Skip to contentSkip to conversation

perf/bpftool: Allow to link libbpf dynamically

PATCH In review Jiri Olsa 11 participants 29 messages Dec 2, 2019 – Dec 5, 2019

Subsystems (7)
lore
Share & tools
lore

Jiri Olsa

author reviewer
#0 lore raw

v4 cover perf/bpftool: Allow to link libbpf dynamically

hi,
adding support to link bpftool with libbpf dynamically, and config change for perf.

It's now possible to use:

  $ make -C tools/bpf/bpftool/ LIBBPF_DYNAMIC=1

which will detect libbpf devel package and if found, link it with bpftool.

It's possible to use arbitrary installed libbpf:

  $ make -C tools/bpf/bpftool/ LIBBPF_DYNAMIC=1 LIBBPF_DIR=/tmp/libbpf/

I based this change on top of Arnaldo's perf/core, because it contains libbpf feature detection code as dependency.

Also available in:
git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
libbpf/dyn

v4 changes:
- based on Toke's v3 post, there's no need for additional API exports:

Since bpftool uses bits of libbpf that are not exported as public API in the .so version, we also pass in libbpf.a to the linker, which allows it to pick up the private functions from the static library without having to expose them as ABI.

- changing some Makefile variable names
- documenting LIBBPF_DYNAMIC and LIBBPF_DIR in the Makefile comment
- extending test_bpftool_build.sh with libbpf dynamic link

thanks,
jirka

---
Jiri Olsa (6):
perf tools: Allow to specify libbpf install directory
bpftool: Allow to link libbpf dynamically
bpftool: Rename BPF_DIR Makefile variable to LIBBPF_SRC_DIR
bpftool: Rename LIBBPF_OUTPUT Makefile variable to LIBBPF_BUILD_OUTPUT
bpftool: Rename LIBBPF_PATH Makefile variable to LIBBPF_BUILD_PATH selftests, bpftool: Add build test for libbpf dynamic linking

 tools/bpf/bpftool/Makefile                        | 54 ++++++++++++++++++++++++++++++++++++++++++++++--------
 tools/perf/Makefile.config                        | 27 ++++++++++++++++++++-------
 tools/testing/selftests/bpf/test_bpftool_build.sh | 53 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 119 insertions(+), 15 deletions(-)
author reviewer
#1 lore raw

PATCH v4 1/6 perf tools: Allow to specify libbpf install directory

Adding LIBBPF_DIR compile variable to allow linking with libbpf installed into specific directory:

  $ make -C tools/lib/bpf/ prefix=/tmp/libbpf/ install_lib install_headers
  $ make -C tools/perf/ LIBBPF_DYNAMIC=1 LIBBPF_DIR=/tmp/libbpf/ VF=1

It might be needed to clean build tree first because features framework does not detect the change properly:

  $ make -C tools/build/feature clean
  $ make -C tools/perf/ clean
---
 tools/perf/Makefile.config | 27 ++++++++++++++++++++-------
 1 file changed, 20 insertions(+), 7 deletions(-)
tools/perf/Makefile.config +20 -7
index c90f4146e5a2..eb9d9b70b7d3 100644
@@ -22,6 +22,14 @@ include $(srctree)/tools/scripts/Makefile.arch
 
 $(call detected_var,SRCARCH)
 
+ifndef lib
+  ifeq ($(SRCARCH)$(IS_64_BIT), x861)
+    lib = lib64
+  else
+    lib = lib
+  endif
+endif # lib
+
 NO_PERF_REGS := 1
 NO_SYSCALL_TABLE := 1
 
@@ -484,11 +492,22 @@ ifndef NO_LIBELF
       CFLAGS += -DHAVE_LIBBPF_SUPPORT
       $(call detected,CONFIG_LIBBPF)
 
+      # for linking with debug library run:
+      # make DEBUG=1 LIBBPF_DIR=/opt/libbpf
+      ifdef LIBBPF_DIR
+        LIBBPF_CFLAGS  := -I$(LIBBPF_DIR)/include
+        LIBBPF_LDFLAGS := -L$(LIBBPF_DIR)/$(lib)
+        FEATURE_CHECK_CFLAGS-libbpf  := $(LIBBPF_CFLAGS)
+        FEATURE_CHECK_LDFLAGS-libbpf := $(LIBBPF_LDFLAGS)
+      endif
+
       # detecting libbpf without LIBBPF_DYNAMIC, so make VF=1 shows libbpf detection status
       $(call feature_check,libbpf)
       ifdef LIBBPF_DYNAMIC
         ifeq ($(feature-libbpf), 1)
           EXTLIBS += -lbpf
+          CFLAGS  += $(LIBBPF_CFLAGS)
+          LDFLAGS += $(LIBBPF_LDFLAGS)
         else
           dummy := $(error Error: No libbpf devel library found, please install libbpf-devel);
         endif
@@ -1037,13 +1056,6 @@ else
 sysconfdir = $(prefix)/etc
 ETC_PERFCONFIG = etc/perfconfig
 endif
-ifndef lib
-ifeq ($(SRCARCH)$(IS_64_BIT), x861)
-lib = lib64
-else
-lib = lib
-endif
-endif # lib
 libdir = $(prefix)/$(lib)
 
 # Shell quote (do not use $(call) to accommodate ancient setups);
@@ -1108,6 +1120,7 @@ ifeq ($(VF),1)
   $(call print_var,LIBUNWIND_DIR)
   $(call print_var,LIBDW_DIR)
   $(call print_var,JDIR)
+  $(call print_var,LIBBPF_DIR)
 
   ifeq ($(dwarf-post-unwind),1)
     $(call feature_print_text,"DWARF post unwind library", $(dwarf-post-unwind-text))
Signature · 1 line
2.21.0
author reviewer
#2 lore raw

PATCH v4 2/6 bpftool: Allow to link libbpf dynamically

In reply to Jiri Olsa

Currently we support only static linking with kernel's libbpf (tools/lib/bpf). This patch adds LIBBPF_DYNAMIC compile variable that triggers libbpf detection and bpf dynamic linking:

  $ make -C tools/bpf/bpftool make LIBBPF_DYNAMIC=1

If libbpf is not installed, build (with LIBBPF_DYNAMIC=1) stops with:

  $ make -C tools/bpf/bpftool LIBBPF_DYNAMIC=1
    Auto-detecting system features:
    ...                        libbfd: [ on  ]
    ...        disassembler-four-args: [ on  ]
    ...                          zlib: [ on  ]
    ...                        libbpf: [ OFF ]

Makefile:102: *** Error: No libbpf devel library found, please install libbpf-devel or libbpf-dev.

Adding LIBBPF_DIR compile variable to allow linking with libbpf installed into specific directory:

  $ make -C tools/lib/bpf/ prefix=/tmp/libbpf/ install_lib install_headers
  $ make -C tools/bpf/bpftool/ LIBBPF_DYNAMIC=1 LIBBPF_DIR=/tmp/libbpf/

It might be needed to clean build tree first because features framework does not detect the change properly:

  $ make -C tools/build/feature clean
  $ make -C tools/bpf/bpftool/ clean

Since bpftool uses bits of libbpf that are not exported as public API in the .so version, we also pass in libbpf.a to the linker, which allows it to pick up the private functions from the static library without having to expose them as ABI.

---
 tools/bpf/bpftool/Makefile | 38 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 38 insertions(+)
tools/bpf/bpftool/Makefile +38 -0
index 39bc6f0f4f0b..be6dea7eb9fd 100644
@@ -1,6 +1,21 @@
 # SPDX-License-Identifier: GPL-2.0-only
+
+# To link dynamically with libbpf run:
+#   make LIBBPF_DYNAMIC=1
+# build will try to detect installed devel package of libbpf.
+#
+# For linking dynamically with libbpf installed at specific PATH run:
+#   make LIBBPF_DYNAMIC=1 LIBBPF_DIR=<PATH>
+
 include ../../scripts/Makefile.include
 include ../../scripts/utilities.mak
+include ../../scripts/Makefile.arch
+
+ifeq ($(LP64), 1)
+  libdir_relative = lib64
+else
+  libdir_relative = lib
+endif
 
 ifeq ($(srctree),)
 srctree := $(patsubst %/,%,$(dir $(CURDIR)))
@@ -63,6 +78,17 @@ RM ?= rm -f
 FEATURE_USER = .bpftool
 FEATURE_TESTS = libbfd disassembler-four-args reallocarray zlib
 FEATURE_DISPLAY = libbfd disassembler-four-args zlib
+ifdef LIBBPF_DYNAMIC
+  FEATURE_TESTS   += libbpf
+  FEATURE_DISPLAY += libbpf
+
+  ifdef LIBBPF_DIR
+    LIBBPF_CFLAGS  := -I$(LIBBPF_DIR)/include
+    LIBBPF_LDFLAGS := -L$(LIBBPF_DIR)/$(libdir_relative)
+    FEATURE_CHECK_CFLAGS-libbpf  := $(LIBBPF_CFLAGS)
+    FEATURE_CHECK_LDFLAGS-libbpf := $(LIBBPF_LDFLAGS)
+  endif
+endif
 
 check_feat := 1
 NON_CHECK_FEAT_TARGETS := clean uninstall doc doc-clean doc-install doc-uninstall
@@ -88,6 +114,18 @@ ifeq ($(feature-reallocarray), 0)
 CFLAGS += -DCOMPAT_NEED_REALLOCARRAY
 endif
 
+ifdef LIBBPF_DYNAMIC
+  ifeq ($(feature-libbpf), 1)
+    # bpftool uses non-exported functions from libbpf, so just add the dynamic
+    # version of libbpf and let the linker figure it out
+    LIBS    := -lbpf $(LIBS)
+    CFLAGS  += $(LIBBPF_CFLAGS)
+    LDFLAGS += $(LIBBPF_LDFLAGS)
+  else
+    dummy := $(error Error: No libbpf devel library found, please install libbpf-devel or libbpf-dev.)
+  endif
+endif
+
 include $(wildcard $(OUTPUT)*.d)
 
 all: $(OUTPUT)bpftool
Signature · 1 line
2.21.0
author reviewer
#3 lore raw

PATCH v4 3/6 bpftool: Rename BPF_DIR Makefile variable to LIBBPF_SRC_DIR

In reply to Jiri Olsa

To properly describe the usage of the variable.

---
 tools/bpf/bpftool/Makefile | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
tools/bpf/bpftool/Makefile +4 -4
index be6dea7eb9fd..512504956315 100644
@@ -29,13 +29,13 @@ else
   Q = @
 endif
 
-BPF_DIR = $(srctree)/tools/lib/bpf/
+LIBBPF_SRC_DIR = $(srctree)/tools/lib/bpf/
 
 ifneq ($(OUTPUT),)
   LIBBPF_OUTPUT = $(OUTPUT)/libbpf/
   LIBBPF_PATH = $(LIBBPF_OUTPUT)
 else
-  LIBBPF_PATH = $(BPF_DIR)
+  LIBBPF_PATH = $(LIBBPF_SRC_DIR)
 endif
 
 LIBBPF = $(LIBBPF_PATH)libbpf.a
@@ -44,11 +44,11 @@ BPFTOOL_VERSION := $(shell make -rR --no-print-directory -sC ../../.. kernelvers
 
 $(LIBBPF): FORCE
 	$(if $(LIBBPF_OUTPUT),@mkdir -p $(LIBBPF_OUTPUT))
-	$(Q)$(MAKE) -C $(BPF_DIR) OUTPUT=$(LIBBPF_OUTPUT) $(LIBBPF_OUTPUT)libbpf.a
+	$(Q)$(MAKE) -C $(LIBBPF_SRC_DIR) OUTPUT=$(LIBBPF_OUTPUT) $(LIBBPF_OUTPUT)libbpf.a
 
 $(LIBBPF)-clean:
 	$(call QUIET_CLEAN, libbpf)
-	$(Q)$(MAKE) -C $(BPF_DIR) OUTPUT=$(LIBBPF_OUTPUT) clean >/dev/null
+	$(Q)$(MAKE) -C $(LIBBPF_SRC_DIR) OUTPUT=$(LIBBPF_OUTPUT) clean >/dev/null
 
 prefix ?= /usr/local
 bash_compdir ?= /usr/share/bash-completion/completions
Signature · 1 line
2.21.0
author reviewer
#4 lore raw

PATCH v4 4/6 bpftool: Rename LIBBPF_OUTPUT Makefile variable to LIBBPF_BUILD_OUTPUT

In reply to Jiri Olsa

To properly describe the usage of the variable.

---
 tools/bpf/bpftool/Makefile | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
tools/bpf/bpftool/Makefile +5 -5
index 512504956315..38f831750ffe 100644
@@ -32,8 +32,8 @@ endif
 LIBBPF_SRC_DIR = $(srctree)/tools/lib/bpf/
 
 ifneq ($(OUTPUT),)
-  LIBBPF_OUTPUT = $(OUTPUT)/libbpf/
-  LIBBPF_PATH = $(LIBBPF_OUTPUT)
+  LIBBPF_BUILD_OUTPUT = $(OUTPUT)/libbpf/
+  LIBBPF_PATH = $(LIBBPF_BUILD_OUTPUT)
 else
   LIBBPF_PATH = $(LIBBPF_SRC_DIR)
 endif
@@ -43,12 +43,12 @@ LIBBPF = $(LIBBPF_PATH)libbpf.a
 BPFTOOL_VERSION := $(shell make -rR --no-print-directory -sC ../../.. kernelversion)
 
 $(LIBBPF): FORCE
-	$(if $(LIBBPF_OUTPUT),@mkdir -p $(LIBBPF_OUTPUT))
-	$(Q)$(MAKE) -C $(LIBBPF_SRC_DIR) OUTPUT=$(LIBBPF_OUTPUT) $(LIBBPF_OUTPUT)libbpf.a
+	$(if $(LIBBPF_BUILD_OUTPUT),@mkdir -p $(LIBBPF_BUILD_OUTPUT))
+	$(Q)$(MAKE) -C $(LIBBPF_SRC_DIR) OUTPUT=$(LIBBPF_BUILD_OUTPUT) $(LIBBPF_BUILD_OUTPUT)libbpf.a
 
 $(LIBBPF)-clean:
 	$(call QUIET_CLEAN, libbpf)
-	$(Q)$(MAKE) -C $(LIBBPF_SRC_DIR) OUTPUT=$(LIBBPF_OUTPUT) clean >/dev/null
+	$(Q)$(MAKE) -C $(LIBBPF_SRC_DIR) OUTPUT=$(LIBBPF_BUILD_OUTPUT) clean >/dev/null
 
 prefix ?= /usr/local
 bash_compdir ?= /usr/share/bash-completion/completions
Signature · 1 line
2.21.0
author reviewer
#5 lore raw

PATCH v4 5/6 bpftool: Rename LIBBPF_PATH Makefile variable to LIBBPF_BUILD_PATH

In reply to Jiri Olsa

To properly describe the usage of the variable.

---
 tools/bpf/bpftool/Makefile | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
tools/bpf/bpftool/Makefile +3 -3
index 38f831750ffe..e66d49bd6aff 100644
@@ -33,12 +33,12 @@ LIBBPF_SRC_DIR = $(srctree)/tools/lib/bpf/
 
 ifneq ($(OUTPUT),)
   LIBBPF_BUILD_OUTPUT = $(OUTPUT)/libbpf/
-  LIBBPF_PATH = $(LIBBPF_BUILD_OUTPUT)
+  LIBBPF_BUILD_PATH = $(LIBBPF_BUILD_OUTPUT)
 else
-  LIBBPF_PATH = $(LIBBPF_SRC_DIR)
+  LIBBPF_BUILD_PATH = $(LIBBPF_SRC_DIR)
 endif
 
-LIBBPF = $(LIBBPF_PATH)libbpf.a
+LIBBPF = $(LIBBPF_BUILD_PATH)libbpf.a
 
 BPFTOOL_VERSION := $(shell make -rR --no-print-directory -sC ../../.. kernelversion)
 
Signature · 1 line
2.21.0
author reviewer
#6 lore raw

PATCH v4 6/6 selftests, bpftool: Add build test for libbpf dynamic linking

In reply to Jiri Olsa

Adding new test to test_bpftool_build.sh script to test the dynamic linkage of libbpf for bpftool:

  $ ./test_bpftool_build.sh
  [SNIP]

... with dynamic libbpf

$PWD: /home/jolsa/kernel/linux-perf/tools/bpf/bpftool
command: make -s -C ../../build/feature clean >/dev/null
command: make -s -C ../../lib/bpf clean >/dev/null
command: make -s -C ../../lib/bpf prefix=/tmp/tmp.fG8O2Ps8ER install_lib install_headers >/dev/null Parsed description of 117 helper function(s)
command: make -s clean >/dev/null
command: make -s LIBBPF_DYNAMIC=1 LIBBPF_DIR=/tmp/tmp.fG8O2Ps8ER >/dev/null
binary: /home/jolsa/kernel/linux-perf/tools/bpf/bpftool/bpftool
binary: linked with libbpf

The test installs libbpf into temp directory
and links bpftool dynamically with it.

---
 .../selftests/bpf/test_bpftool_build.sh       | 53 +++++++++++++++++++
 1 file changed, 53 insertions(+)
tools/testing/selftests/bpf/test_bpftool_build.sh +53 -0
index ac349a5cea7e..e4a6a0520f8e 100755
@@ -85,6 +85,55 @@ make_with_tmpdir() {
 	echo
 }
 
+# Assumes current directory is tools/bpf/bpftool
+make_with_dynamic_libbpf() {
+	TMPDIR=$(mktemp -d)
+	echo -e "\$PWD:    $PWD"
+
+	# It might be needed to clean build tree first because features
+	# framework does not detect the change properly
+	echo -e "command: make -s -C ../../build/feature clean >/dev/null"
+	make $J -s -C ../../build/feature clean >/dev/null
+	if [ $? -ne 0 ] ; then
+		ERROR=1
+	fi
+	echo -e "command: make -s -C ../../lib/bpf clean >/dev/null"
+	make $J -s -C ../../lib/bpf clean >/dev/null
+	if [ $? -ne 0 ] ; then
+		ERROR=1
+	fi
+
+	# Now install libbpf into TMPDIR
+	echo -e "command: make -s -C ../../lib/bpf prefix=$TMPDIR install_lib install_headers >/dev/null"
+	make $J -s -C ../../lib/bpf prefix=$TMPDIR install_lib install_headers >/dev/null
+	if [ $? -ne 0 ] ; then
+		ERROR=1
+	fi
+
+	# And final bpftool build (with clean first) with libbpf dynamic link
+	echo -e "command: make -s clean >/dev/null"
+	if [ $? -ne 0 ] ; then
+		ERROR=1
+	fi
+	echo -e "command: make -s LIBBPF_DYNAMIC=1 LIBBPF_DIR=$TMPDIR >/dev/null"
+	make $J -s LIBBPF_DYNAMIC=1 LIBBPF_DIR=$TMPDIR >/dev/null
+	if [ $? -ne 0 ] ; then
+		ERROR=1
+	fi
+
+	check .
+	ldd bpftool | grep -q libbpf.so
+	if [ $? -ne 0 ] ; then
+		printf "FAILURE: Did not find libbpf linked\n"
+	else
+		echo "binary:  linked with libbpf"
+	fi
+	make -s -C ../../lib/bpf clean
+	make -s clean
+	rm -rf -- $TMPDIR
+	echo
+}
+
 echo "Trying to build bpftool"
 echo -e "... through kbuild\n"
 
@@ -145,3 +194,7 @@ make_and_clean
 make_with_tmpdir OUTPUT
 
 make_with_tmpdir O
+
+echo -e "... with dynamic libbpf\n"
+
+make_with_dynamic_libbpf
Signature · 1 line
2.21.0
#7 lore raw

Re: [PATCH 6/6] selftests, bpftool: Add build test for libbpf dynamic linking

Thanks Jiri ! A few comments inline.

2019-12-02 14:18 UTC+0100 ~ Jiri Olsa <[email protected]>

Quoted 32 lines

Adding new test to test_bpftool_build.sh script to test the dynamic linkage of libbpf for bpftool:

  $ ./test_bpftool_build.sh
  [SNIP]

... with dynamic libbpf

$PWD: /home/jolsa/kernel/linux-perf/tools/bpf/bpftool
command: make -s -C ../../build/feature clean >/dev/null
command: make -s -C ../../lib/bpf clean >/dev/null
command: make -s -C ../../lib/bpf prefix=/tmp/tmp.fG8O2Ps8ER install_lib install_headers >/dev/null Parsed description of 117 helper function(s)
command: make -s clean >/dev/null
command: make -s LIBBPF_DYNAMIC=1 LIBBPF_DIR=/tmp/tmp.fG8O2Ps8ER >/dev/null
binary: /home/jolsa/kernel/linux-perf/tools/bpf/bpftool/bpftool
binary: linked with libbpf

The test installs libbpf into temp directory
and links bpftool dynamically with it.

---
 .../selftests/bpf/test_bpftool_build.sh       | 53 +++++++++++++++++++
 1 file changed, 53 insertions(+)
tools/testing/selftests/bpf/test_bpftool_build.sh +0 -0
index ac349a5cea7e..e4a6a0520f8e 100755
@@ -85,6 +85,55 @@ make_with_tmpdir() {
 	echo

tools/testing/selftests/bpf/test_bpftool_build.sh make_with_tmpdir() { · line 95

Quoted 10 lines
tools/testing/selftests/bpf/test_bpftool_build.sh +8 -0
 }
 
+# Assumes current directory is tools/bpf/bpftool
+make_with_dynamic_libbpf() {
+	TMPDIR=$(mktemp -d)
+	echo -e "\$PWD:    $PWD"
+
+	# It might be needed to clean build tree first because features
+	# framework does not detect the change properly
+	echo -e "command: make -s -C ../../build/feature clean >/dev/null"

(So far I did not echo the "make clean" commands, I printed only the ones used to build bpftool. But that's your call.)

Quoted 12 lines
tools/testing/selftests/bpf/test_bpftool_build.sh +12 -0
+	make $J -s -C ../../build/feature clean >/dev/null
+	if [ $? -ne 0 ] ; then
+		ERROR=1
+	fi
+	echo -e "command: make -s -C ../../lib/bpf clean >/dev/null"
+	make $J -s -C ../../lib/bpf clean >/dev/null
+	if [ $? -ne 0 ] ; then
+		ERROR=1
+	fi
+
+	# Now install libbpf into TMPDIR
+	echo -e "command: make -s -C ../../lib/bpf prefix=$TMPDIR install_lib install_headers >/dev/null"

tools/testing/selftests/bpf/test_bpftool_build.sh make_with_tmpdir() { · line 117

Quoted 10 lines
tools/testing/selftests/bpf/test_bpftool_build.sh +10 -0
+	make $J -s -C ../../lib/bpf prefix=$TMPDIR install_lib install_headers >/dev/null
+	if [ $? -ne 0 ] ; then
+		ERROR=1
+	fi
+
+	# And final bpftool build (with clean first) with libbpf dynamic link
+	echo -e "command: make -s clean >/dev/null"
+	if [ $? -ne 0 ] ; then
+		ERROR=1
+	fi

I do not believe you need to "make clean" here, this should have been done by the previous test in that dir earlier in the script (cd tools/bpf/bpftool; make_and_clean)

tools/testing/selftests/bpf/test_bpftool_build.sh make_with_tmpdir() { · line 126

Quoted 9 lines
tools/testing/selftests/bpf/test_bpftool_build.sh +9 -0
+	echo -e "command: make -s LIBBPF_DYNAMIC=1 LIBBPF_DIR=$TMPDIR >/dev/null"
+	make $J -s LIBBPF_DYNAMIC=1 LIBBPF_DIR=$TMPDIR >/dev/null
+	if [ $? -ne 0 ] ; then
+		ERROR=1
+	fi
+
+	check .
+	ldd bpftool | grep -q libbpf.so
+	if [ $? -ne 0 ] ; then

(Or "if ldd bpftool | grep -q libbpf.so ; then")

tools/testing/selftests/bpf/test_bpftool_build.sh make_with_tmpdir() { · line 127

Quoted 1 line
tools/testing/selftests/bpf/test_bpftool_build.sh +1 -0
+		printf "FAILURE: Did not find libbpf linked\n"

Please also set $(ERROR) here.

(Also, stick to echo rather than mixing with printf? I can't remember why I used one over echo in the check() function, that was probably not on purpose.)

tools/testing/selftests/bpf/test_bpftool_build.sh make_with_tmpdir() { · line 133

Quoted 6 lines
tools/testing/selftests/bpf/test_bpftool_build.sh +6 -0
+	else
+		echo "binary:  linked with libbpf"
+	fi
+	make -s -C ../../lib/bpf clean
+	make -s clean
+	rm -rf -- $TMPDIR

We probably want to clean features too? We tried to check that libbpf was available, but with a very specific $(LIBBPF_DIR), which was temporary and no longer exist. So better to reset features to a clean state?

Quoted 8 lines
tools/testing/selftests/bpf/test_bpftool_build.sh +3 -0
+	echo
+}
+
 echo "Trying to build bpftool"
 echo -e "... through kbuild\n"
 

tools/testing/selftests/bpf/test_bpftool_build.sh make_and_clean · line 200

Quoted 7 lines
tools/testing/selftests/bpf/test_bpftool_build.sh +4 -0
@@ -145,3 +194,7 @@ make_and_clean
 make_with_tmpdir OUTPUT
 
 make_with_tmpdir O
+
+echo -e "... with dynamic libbpf\n"
+
+make_with_dynamic_libbpf

Thanks,
Quentin

#8 lore raw
In reply to Jiri Olsa
Quoted 20 lines · Jiri Olsa

hi,
adding support to link bpftool with libbpf dynamically, and config change for perf.

It's now possible to use:

  $ make -C tools/bpf/bpftool/ LIBBPF_DYNAMIC=1

which will detect libbpf devel package and if found, link it with bpftool.

It's possible to use arbitrary installed libbpf:

  $ make -C tools/bpf/bpftool/ LIBBPF_DYNAMIC=1 LIBBPF_DIR=/tmp/libbpf/

I based this change on top of Arnaldo's perf/core, because it contains libbpf feature detection code as dependency.

Also available in:
git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
libbpf/dyn

Quoted 7 lines

v4 changes:
- based on Toke's v3 post, there's no need for additional API exports:

Since bpftool uses bits of libbpf that are not exported as public API in the .so version, we also pass in libbpf.a to the linker, which allows it to pick up the private functions from the static library without having to expose them as ABI.

Whoever understands how this is supposed to work, can you please explain? From reading this, I think what we **want** is:

- all LIBBPF_API-exposed APIs should be dynamically linked against libbpf.so;
- everything else used from libbpf (e.g., netlink APIs), should come from libbpf.a.

Am I getting the idea right?

If yes, are we sure it actually works like that in practice? I've compiled with LIBBPF_DYNAMIC=1, and what I see is that libelf, libc, zlib, etc functions do have relocations against them in ".rela.plt" section. None of libbpf exposed APIs, though, have any of such relocations. Which to me suggests that they are just statically linked against libbpf.a and libbpf.so is just recorded in ELF as a dynamic library dependency because of this extra -lbpf flag. Which kind of defeats the purpose of this whole endeavor, no?

I'm no linker expert, though, so I apologize if I got it completely wrong, would really appreciate someone to detail this a bit more. Thanks!

Quoted 23 lines

- changing some Makefile variable names
- documenting LIBBPF_DYNAMIC and LIBBPF_DIR in the Makefile comment
- extending test_bpftool_build.sh with libbpf dynamic link

thanks,
jirka

---
Jiri Olsa (6):
perf tools: Allow to specify libbpf install directory
bpftool: Allow to link libbpf dynamically
bpftool: Rename BPF_DIR Makefile variable to LIBBPF_SRC_DIR
bpftool: Rename LIBBPF_OUTPUT Makefile variable to LIBBPF_BUILD_OUTPUT
bpftool: Rename LIBBPF_PATH Makefile variable to LIBBPF_BUILD_PATH selftests, bpftool: Add build test for libbpf dynamic linking

 tools/bpf/bpftool/Makefile                        | 54 ++++++++++++++++++++++++++++++++++++++++++++++--------
 tools/perf/Makefile.config                        | 27 ++++++++++++++++++++-------
 tools/testing/selftests/bpf/test_bpftool_build.sh | 53 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 119 insertions(+), 15 deletions(-)
#9 lore raw
Quoted 47 lines · Andrii Nakryiko
Quoted 20 lines · Jiri Olsa

hi,
adding support to link bpftool with libbpf dynamically, and config change for perf.

It's now possible to use:

  $ make -C tools/bpf/bpftool/ LIBBPF_DYNAMIC=1

which will detect libbpf devel package and if found, link it with bpftool.

It's possible to use arbitrary installed libbpf:

  $ make -C tools/bpf/bpftool/ LIBBPF_DYNAMIC=1 LIBBPF_DIR=/tmp/libbpf/

I based this change on top of Arnaldo's perf/core, because it contains libbpf feature detection code as dependency.

Also available in:
git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
libbpf/dyn

Quoted 7 lines

v4 changes:
- based on Toke's v3 post, there's no need for additional API exports:

Since bpftool uses bits of libbpf that are not exported as public API in the .so version, we also pass in libbpf.a to the linker, which allows it to pick up the private functions from the static library without having to expose them as ABI.

Whoever understands how this is supposed to work, can you please explain? From reading this, I think what we **want** is:

- all LIBBPF_API-exposed APIs should be dynamically linked against libbpf.so;
- everything else used from libbpf (e.g., netlink APIs), should come from libbpf.a.

Am I getting the idea right?

If yes, are we sure it actually works like that in practice? I've compiled with LIBBPF_DYNAMIC=1, and what I see is that libelf, libc, zlib, etc functions do have relocations against them in ".rela.plt" section. None of libbpf exposed APIs, though, have any of such relocations. Which to me suggests that they are just statically linked against libbpf.a and libbpf.so is just recorded in ELF as a dynamic library dependency because of this extra -lbpf flag. Which kind of defeats the purpose of this whole endeavor, no?

Quoted 3 lines

I'm no linker expert, though, so I apologize if I got it completely wrong, would really appreciate someone to detail this a bit more. Thanks!

Ah, that is my mistake: I was getting dynamic libbpf symbols with this approach, but that was because I had the version of libbpf.so in my $LIBDIR that had the patch to expose the netlink APIs as versioned symbols; so it was just pulling in everything from the shared library.

So what I was going for was exactly what you described above; but it seems that doesn't actually work. Too bad, and sorry for wasting your time on this :/

-Toke

#10 lore raw
Quoted 9 lines · Toke Høiland-Jørgensen

Ah, that is my mistake: I was getting dynamic libbpf symbols with this approach, but that was because I had the version of libbpf.so in my $LIBDIR that had the patch to expose the netlink APIs as versioned symbols; so it was just pulling in everything from the shared library.

So what I was going for was exactly what you described above; but it seems that doesn't actually work. Too bad, and sorry for wasting your time on this :/

bpftool is currently tightly coupled with libbpf and very likely in the future the dependency will be even tighter.
In that sense bpftool is an extension of libbpf and libbpf is an extension of bpftool.
Andrii is working on set of patches to generate user space .c code from bpf program.
bpftool will be generating the code that is specific for the version bpftool and for
the version of libbpf. There will be compatibility layers as usual. But in general the situation where a bug in libbpf is so criticial that bpftool needs to repackaged is imo less likely than a bug in bpftool that will require re-packaging of libbpf.
bpftool is quite special. It's not a typical user of libbpf.
The other way around is more correct. libbpf is a user of the code that bpftool generates and both depend on each other.
perf on the other side is what typical user space app that uses libbpf will look like.
I think keeping bpftool in the kernel while packaging libbpf
out of github was an oversight.
I think we need to mirror bpftool into github/libbpf as well
and make sure they stay together. The version of libbpf == version of bpftool. Both should come from the same package and so on.
May be they can be two different packages but
upgrading one should trigger upgrade of another and vice versa.
I think one package would be easier though.
Thoughts?

#11 lore raw
Quoted 29 lines · Alexei Starovoitov
Quoted 9 lines · Toke Høiland-Jørgensen

Ah, that is my mistake: I was getting dynamic libbpf symbols with this approach, but that was because I had the version of libbpf.so in my $LIBDIR that had the patch to expose the netlink APIs as versioned symbols; so it was just pulling in everything from the shared library.

So what I was going for was exactly what you described above; but it seems that doesn't actually work. Too bad, and sorry for wasting your time on this :/

bpftool is currently tightly coupled with libbpf and very likely in the future the dependency will be even tighter.
In that sense bpftool is an extension of libbpf and libbpf is an extension of bpftool.
Andrii is working on set of patches to generate user space .c code from bpf program.
bpftool will be generating the code that is specific for the version bpftool and for
the version of libbpf. There will be compatibility layers as usual. But in general the situation where a bug in libbpf is so criticial that bpftool needs to repackaged is imo less likely than a bug in bpftool that will require re-packaging of libbpf.
bpftool is quite special. It's not a typical user of libbpf. The other way around is more correct. libbpf is a user of the code that bpftool generates and both depend on each other.
perf on the other side is what typical user space app that uses libbpf will look like.
I think keeping bpftool in the kernel while packaging libbpf

Quoted 8 lines

out of github was an oversight.
I think we need to mirror bpftool into github/libbpf as well
and make sure they stay together. The version of libbpf == version of bpftool. Both should come from the same package and so on.
May be they can be two different packages but
upgrading one should trigger upgrade of another and vice versa.
I think one package would be easier though.
Thoughts?

great, makes sense.. Toke already mentioned we might need to move bpftool to libbpf package to solve our issues, now when you guys have already plans for that, it works for us ;-)

thanks for sharing the plan,
jirka

#12 lore raw
In reply to Alexei Starovoitov
Quoted 29 lines · Alexei Starovoitov
Quoted 9 lines · Toke Høiland-Jørgensen

Ah, that is my mistake: I was getting dynamic libbpf symbols with this approach, but that was because I had the version of libbpf.so in my $LIBDIR that had the patch to expose the netlink APIs as versioned symbols; so it was just pulling in everything from the shared library.

So what I was going for was exactly what you described above; but it seems that doesn't actually work. Too bad, and sorry for wasting your time on this :/

bpftool is currently tightly coupled with libbpf and very likely in the future the dependency will be even tighter.
In that sense bpftool is an extension of libbpf and libbpf is an extension of bpftool.
Andrii is working on set of patches to generate user space .c code from bpf program.
bpftool will be generating the code that is specific for the version bpftool and for
the version of libbpf. There will be compatibility layers as usual. But in general the situation where a bug in libbpf is so criticial that bpftool needs to repackaged is imo less likely than a bug in bpftool that will require re-packaging of libbpf.
bpftool is quite special. It's not a typical user of libbpf. The other way around is more correct. libbpf is a user of the code that bpftool generates and both depend on each other.
perf on the other side is what typical user space app that uses libbpf will look like.
I think keeping bpftool in the kernel while packaging libbpf

Quoted 8 lines

out of github was an oversight.
I think we need to mirror bpftool into github/libbpf as well
and make sure they stay together. The version of libbpf == version of bpftool. Both should come from the same package and so on.
May be they can be two different packages but
upgrading one should trigger upgrade of another and vice versa.
I think one package would be easier though.
Thoughts?

Yup, making bpftool explicitly the "libbpf command line interface" makes sense and would help clarify the relationship between the two. As Jiri said, we are already moving in that direction packaging-wise...

-Toke

#13 lore raw
Quoted 41 lines · Toke Høiland-Jørgensen
Quoted 29 lines · Alexei Starovoitov
Quoted 9 lines · Toke Høiland-Jørgensen

Ah, that is my mistake: I was getting dynamic libbpf symbols with this approach, but that was because I had the version of libbpf.so in my $LIBDIR that had the patch to expose the netlink APIs as versioned symbols; so it was just pulling in everything from the shared library.

So what I was going for was exactly what you described above; but it seems that doesn't actually work. Too bad, and sorry for wasting your time on this :/

bpftool is currently tightly coupled with libbpf and very likely in the future the dependency will be even tighter.
In that sense bpftool is an extension of libbpf and libbpf is an extension of bpftool.
Andrii is working on set of patches to generate user space .c code from bpf program.
bpftool will be generating the code that is specific for the version bpftool and for
the version of libbpf. There will be compatibility layers as usual. But in general the situation where a bug in libbpf is so criticial that bpftool needs to repackaged is imo less likely than a bug in bpftool that will require re-packaging of libbpf.
bpftool is quite special. It's not a typical user of libbpf. The other way around is more correct. libbpf is a user of the code that bpftool generates and both depend on each other.
perf on the other side is what typical user space app that uses libbpf will look like.
I think keeping bpftool in the kernel while packaging libbpf

Quoted 8 lines

out of github was an oversight.
I think we need to mirror bpftool into github/libbpf as well
and make sure they stay together. The version of libbpf == version of bpftool. Both should come from the same package and so on.
May be they can be two different packages but
upgrading one should trigger upgrade of another and vice versa.
I think one package would be easier though.
Thoughts?

Quoted 3 lines

Yup, making bpftool explicitly the "libbpf command line interface" makes sense and would help clarify the relationship between the two. As Jiri said, we are already moving in that direction packaging-wise...

Awesome. Let's figure out the logistics.
Should we do:
git mv tools/bpf/bpftool/ tools/lib/bpf/
and appropriate adjustment to Makefiles ?
or keep it where it is and only add to
https://github.com/libbpf/libbpf/blob/master/scripts/sync-kernel.sh ?

maintainer
#14 lore raw
Quoted 44 lines · Alexei Starovoitov
Quoted 39 lines · Toke Høiland-Jørgensen
Quoted 29 lines · Alexei Starovoitov
Quoted 9 lines · Toke Høiland-Jørgensen

Ah, that is my mistake: I was getting dynamic libbpf symbols with this approach, but that was because I had the version of libbpf.so in my $LIBDIR that had the patch to expose the netlink APIs as versioned symbols; so it was just pulling in everything from the shared library.

So what I was going for was exactly what you described above; but it seems that doesn't actually work. Too bad, and sorry for wasting your time on this :/

bpftool is currently tightly coupled with libbpf and very likely in the future the dependency will be even tighter.
In that sense bpftool is an extension of libbpf and libbpf is an extension of bpftool.
Andrii is working on set of patches to generate user space .c code from bpf program.
bpftool will be generating the code that is specific for the version bpftool and for
the version of libbpf. There will be compatibility layers as usual. But in general the situation where a bug in libbpf is so criticial that bpftool needs to repackaged is imo less likely than a bug in bpftool that will require re-packaging of libbpf.
bpftool is quite special. It's not a typical user of libbpf. The other way around is more correct. libbpf is a user of the code that bpftool generates and both depend on each other.
perf on the other side is what typical user space app that uses libbpf will look like.
I think keeping bpftool in the kernel while packaging libbpf

Quoted 8 lines

out of github was an oversight.
I think we need to mirror bpftool into github/libbpf as well
and make sure they stay together. The version of libbpf == version of bpftool. Both should come from the same package and so on.
May be they can be two different packages but
upgrading one should trigger upgrade of another and vice versa.
I think one package would be easier though.
Thoughts?

Quoted 3 lines

Yup, making bpftool explicitly the "libbpf command line interface" makes sense and would help clarify the relationship between the two. As Jiri said, we are already moving in that direction packaging-wise...

Quoted 6 lines

Awesome. Let's figure out the logistics.
Should we do:
git mv tools/bpf/bpftool/ tools/lib/bpf/
and appropriate adjustment to Makefiles ?
or keep it where it is and only add to
https://github.com/libbpf/libbpf/blob/master/scripts/sync-kernel.sh ?

I'd be in preference of the latter aka keeping where it is.

Thanks,
Daniel

#15 lore raw
Quoted 45 lines · Daniel Borkmann
Quoted 44 lines · Alexei Starovoitov
Quoted 42 lines · Toke Høiland-Jørgensen
Quoted 29 lines · Alexei Starovoitov
Quoted 9 lines · Toke Høiland-Jørgensen

Ah, that is my mistake: I was getting dynamic libbpf symbols with this approach, but that was because I had the version of libbpf.so in my $LIBDIR that had the patch to expose the netlink APIs as versioned symbols; so it was just pulling in everything from the shared library.

So what I was going for was exactly what you described above; but it seems that doesn't actually work. Too bad, and sorry for wasting your time on this :/

bpftool is currently tightly coupled with libbpf and very likely in the future the dependency will be even tighter.
In that sense bpftool is an extension of libbpf and libbpf is an extension of bpftool.
Andrii is working on set of patches to generate user space .c code from bpf program.
bpftool will be generating the code that is specific for the version bpftool and for
the version of libbpf. There will be compatibility layers as usual. But in general the situation where a bug in libbpf is so criticial that bpftool needs to repackaged is imo less likely than a bug in bpftool that will require re-packaging of libbpf.
bpftool is quite special. It's not a typical user of libbpf. The other way around is more correct. libbpf is a user of the code that bpftool generates and both depend on each other.
perf on the other side is what typical user space app that uses libbpf will look like.
I think keeping bpftool in the kernel while packaging libbpf

Quoted 8 lines

out of github was an oversight.
I think we need to mirror bpftool into github/libbpf as well
and make sure they stay together. The version of libbpf == version of bpftool. Both should come from the same package and so on.
May be they can be two different packages but
upgrading one should trigger upgrade of another and vice versa.
I think one package would be easier though.
Thoughts?

Yup, making bpftool explicitly the "libbpf command line interface" makes sense and would help clarify the relationship between the two. As Jiri said, we are already moving in that direction packaging-wise...

Quoted 8 lines
Quoted 6 lines

Awesome. Let's figure out the logistics.
Should we do:
git mv tools/bpf/bpftool/ tools/lib/bpf/
and appropriate adjustment to Makefiles ?
or keep it where it is and only add to
https://github.com/libbpf/libbpf/blob/master/scripts/sync-kernel.sh ?

I'd be in preference of the latter aka keeping where it is.

I don't have any strong preference either way. It would make sense to move it to make clear the interdependency (and that bpftool is really the "libbpf cli interface"); but it could also just be kept separate and just document this in the existing bpftool dir.

The github repository may need some surgery, though. So maybe let the changes in the kernel tree depend on what's easiest for that? IDK?

-Toke

#16 lore raw
In reply to Alexei Starovoitov
Quoted 23 lines · Alexei Starovoitov
Quoted 9 lines · Toke Høiland-Jørgensen

Ah, that is my mistake: I was getting dynamic libbpf symbols with this approach, but that was because I had the version of libbpf.so in my $LIBDIR that had the patch to expose the netlink APIs as versioned symbols; so it was just pulling in everything from the shared library.

So what I was going for was exactly what you described above; but it seems that doesn't actually work. Too bad, and sorry for wasting your time on this :/

bpftool is currently tightly coupled with libbpf and very likely in the future the dependency will be even tighter.
In that sense bpftool is an extension of libbpf and libbpf is an extension of bpftool.
Andrii is working on set of patches to generate user space .c code from bpf program.
bpftool will be generating the code that is specific for the version bpftool and for
the version of libbpf. There will be compatibility layers as usual. But in general the situation where a bug in libbpf is so criticial that bpftool needs to repackaged is imo less likely than a bug in

Quoted 8 lines

bpftool that will require re-packaging of libbpf. bpftool is quite special. It's not a typical user of libbpf. The other way around is more correct. libbpf is a user of the code that bpftool generates and both depend on each other. perf on the other side is what typical user space app that uses libbpf will look like.
I think keeping bpftool in the kernel while packaging libbpf out of github was an oversight.

I wonder what big advantage having bpftool in libbpf's Github repo brings, actually? The reason we need libbpf on github is to allow other projects like pahole to be able to use libbpf from submodule. There is no such need for bpftool.

I agree about preference to release them in sync, but that could be easily done by releasing based on corresponding commits in github's libbpf repo and kernel repo. bpftool doesn't have to physically live next to libbpf on Github, does it?

Calling github repo a "mirror" is incorrect. It's not a 1:1 copy of files. We have a completely separate Makefile for libbpf, and we have a bunch of stuff we had to re-implement to detach libbpf code from kernel's non-UAPI headers. Doing this for bpftool as well seems like just more maintenance. Keeping github's Makefile in sync with kernel's Makefile (for libbpf) is PITA, I'd rather avoid similar pains for bpftool without a really good reason.

Quoted 7 lines

I think we need to mirror bpftool into github/libbpf as well
and make sure they stay together. The version of libbpf == version of bpftool. Both should come from the same package and so on.
May be they can be two different packages but
upgrading one should trigger upgrade of another and vice versa.
I think one package would be easier though.
Thoughts?

#17 lore raw
Quoted 9 lines · Andrii Nakryiko

I wonder what big advantage having bpftool in libbpf's Github repo brings, actually? The reason we need libbpf on github is to allow other projects like pahole to be able to use libbpf from submodule. There is no such need for bpftool.

I agree about preference to release them in sync, but that could be easily done by releasing based on corresponding commits in github's libbpf repo and kernel repo. bpftool doesn't have to physically live next to libbpf on Github, does it?

+1

Quoted 7 lines

Calling github repo a "mirror" is incorrect. It's not a 1:1 copy of files. We have a completely separate Makefile for libbpf, and we have a bunch of stuff we had to re-implement to detach libbpf code from kernel's non-UAPI headers. Doing this for bpftool as well seems like just more maintenance. Keeping github's Makefile in sync with kernel's Makefile (for libbpf) is PITA, I'd rather avoid similar pains for bpftool without a really good reason.

Agreed. Having libbpf on GH is definitely useful today, but one can hope a day will come when distroes will get up to speed on packaging libbpf, and perhaps we can retire it? Maybe 2, 3 years from now? Putting bpftool in the same boat is just more baggage.

#18 lore raw
Quoted 21 lines · Jakub Kicinski
Quoted 9 lines · Andrii Nakryiko

I wonder what big advantage having bpftool in libbpf's Github repo brings, actually? The reason we need libbpf on github is to allow other projects like pahole to be able to use libbpf from submodule. There is no such need for bpftool.

I agree about preference to release them in sync, but that could be easily done by releasing based on corresponding commits in github's libbpf repo and kernel repo. bpftool doesn't have to physically live next to libbpf on Github, does it?

+1

Quoted 7 lines

Calling github repo a "mirror" is incorrect. It's not a 1:1 copy of files. We have a completely separate Makefile for libbpf, and we have a bunch of stuff we had to re-implement to detach libbpf code from kernel's non-UAPI headers. Doing this for bpftool as well seems like just more maintenance. Keeping github's Makefile in sync with kernel's Makefile (for libbpf) is PITA, I'd rather avoid similar pains for bpftool without a really good reason.

Quoted 4 lines

Agreed. Having libbpf on GH is definitely useful today, but one can hope a day will come when distroes will get up to speed on packaging libbpf, and perhaps we can retire it? Maybe 2, 3 years from now? Putting bpftool in the same boat is just more baggage.

Distros should be packaging libbpf and bpftool from single repo on github. Kernel tree is for packaging kernel.

#19 lore raw
Quoted 7 lines · Alexei Starovoitov
Quoted 4 lines

Agreed. Having libbpf on GH is definitely useful today, but one can hope a day will come when distroes will get up to speed on packaging libbpf, and perhaps we can retire it? Maybe 2, 3 years from now? Putting bpftool in the same boat is just more baggage.

Distros should be packaging libbpf and bpftool from single repo on github. Kernel tree is for packaging kernel.

Okay, single repo on GitHub:

https://github.com/torvalds/linux

we are in agreement 😝

Jokes aside, you may need to provide some reasoning on this one.. The recommendation for packaging libbpf from GitHub never had any clear justification either AFAICR.

I honestly don't see why location matters. bpftool started out on GitHub but we moved it into the tree for... ease of packaging/distribution(?!) Now it's handy to have it in the tree to reuse the uapi headers.

As much as I don't care if we move it (back) out of the tree - having two copies makes no sense to me. As does having it in the libbpf repo. The sync effort is not warranted. User confusion is not warranted.

The distroes already package bpftool from the kernel sources, people had put in time to get to this stage and there aren't any complaints.

In fact all the BPF projects and test suites we are involved in at Netronome are entirely happy the packaged versions of LLVM and libbpf in Fedora _today_, IOW the GH libbpf is irrelevant to us already.

As for the problem which sparked this discussion - I disagree that bpftool should have "special relationship" with the library. In fact bpftool uses the widest range of libbpf's interfaces of all known projects so it's invaluable for making sure that those interfaces are usable, consistent and complete.

You also said a few times you don't want to merge fixes into bpf/net. That divergence from kernel development process is worrying.

None of this makes very much sense to me. We're diverging from well established development practices without as much as a justification.

Perhaps I'm not clever enough to follow. But if I'm allowed to make an uneducated guess it would be that it's some Facebook internal reason, like it's hard to do backports? :/

maintainer
#20 lore raw

From: Jakub Kicinski <[email protected]>
Date: Wed, 4 Dec 2019 16:23:48 -0800

Quoted 11 lines

Jokes aside, you may need to provide some reasoning on this one.. The recommendation for packaging libbpf from GitHub never had any clear justification either AFAICR.

I honestly don't see why location matters. bpftool started out on GitHub but we moved it into the tree for... ease of packaging/distribution(?!) Now it's handy to have it in the tree to reuse the uapi headers.

As much as I don't care if we move it (back) out of the tree - having two copies makes no sense to me. As does having it in the libbpf repo. The sync effort is not warranted. User confusion is not warranted.

Part of this story has to do with how bug fixes propagate via bpf-next instead of the bpf tree, as I understand it.

But yeah it would be nice to have a clear documentation on all of the reasoning.

On the distro side, people seem to not want to use the separate repo. If you're supporting enterprise customers you don't just sync with upstream, you cherry pick. When cherry picking gets too painful, you sync with upstream possibly eliding upstream new features you don't want to appear in your supported product yet.

I agree with tying bpftool and libbpf into the _resulting_ binary distro package, but I'm not totally convinced about separating them out of the kernel source tree.

#21 lore raw
Quoted 27 lines · David Miller

From: Jakub Kicinski <[email protected]>
Date: Wed, 4 Dec 2019 16:23:48 -0800

Quoted 11 lines

Jokes aside, you may need to provide some reasoning on this one.. The recommendation for packaging libbpf from GitHub never had any clear justification either AFAICR.

I honestly don't see why location matters. bpftool started out on GitHub but we moved it into the tree for... ease of packaging/distribution(?!) Now it's handy to have it in the tree to reuse the uapi headers.

As much as I don't care if we move it (back) out of the tree - having two copies makes no sense to me. As does having it in the libbpf repo. The sync effort is not warranted. User confusion is not warranted.

Part of this story has to do with how bug fixes propagate via bpf-next instead of the bpf tree, as I understand it.

But yeah it would be nice to have a clear documentation on all of the reasoning.

On the distro side, people seem to not want to use the separate repo. If you're supporting enterprise customers you don't just sync with upstream, you cherry pick. When cherry picking gets too painful, you sync with upstream possibly eliding upstream new features you don't want to appear in your supported product yet.

Quoted 3 lines

I agree with tying bpftool and libbpf into the _resulting_ binary distro package, but I'm not totally convinced about separating them out of the kernel source tree.

Looks like there is a confusion here.
I'm not proposing to move bpftool out of kernel tree.
The kernel+libbpf+bpftool+selftests already come as single patch set. bpftool has to stay in the kernel tree otherwise things like skeleton patchset won't be possible to accomplish without a lot of coordination between different trees and propagation delays.

I'm proposing to tweak github/libbpf sync script to sync bpftool sources from kernel into github, so both libbpf and bpftool can be tested and packaged together.
People are working on adding proper CI to github/libbpf. bpftool testing will automatically get more mileage out of that effort.

github/libbpf is self contained. It should be built and tested on many different kernels and build environments (like any user space package should be). That's an important goal of CI. When bpftool is part of github/libbpf it will get the same treatment. I see only advantages and not a single disadvantage of building, testing, packaging bpftool out of github/libbpf.

To support stable libbpf+bpftool releases we can branch in github and push fixes into branches. Same CI can test master and stable branches.

#22 lore raw
In reply to Jakub Kicinski
Quoted 6 lines · Jakub Kicinski
Quoted 5 lines · Alexei Starovoitov
Quoted 4 lines

Agreed. Having libbpf on GH is definitely useful today, but one can hope a day will come when distroes will get up to speed on packaging libbpf, and perhaps we can retire it? Maybe 2, 3 years from now? Putting bpftool in the same boat is just more baggage.

Quoted 6 lines
Quoted 2 lines

Distros should be packaging libbpf and bpftool from single repo on github. Kernel tree is for packaging kernel.

Okay, single repo on GitHub:

https://github.com/torvalds/linux

and how will you git submodule only libbpf part of kernel github into bcc and other projects?

Quoted 2 lines

You also said a few times you don't want to merge fixes into bpf/net. That divergence from kernel development process is worrying.

worrying - why? what exactly the concern you see?
Tying user space release into kernel release and user space process into kernel process makes little sense to me. Packaging is different. Compatibility requirements are different. CI is different. Integration with other projects is different.

libbpf source code is in the kernel tree only because kernel changes plus libbpf changes plus selftests changes come as single patchset. That is really the only reason. Packaging scripts, CI scripts, etc should be kept out of kernel tree. All that stuff belongs at github/libbpf.

Quoted 2 lines

None of this makes very much sense to me. We're diverging from well established development practices without as much as a justification.

The kernel development process was never used for libbpf. Even coding style is different. I'm puzzled why you think user space should be tied to kernel. Everything is so vastly different.
Some people say that 8 weeks to bump libbpf version is too long. Other people say that it's too often.
libbpf version numbers != kernel version numbers.
There is no definition of LTS for libbpf. One day it will be
and the version of libbpf picked for LTS will likely have
nothing to do with kernel LTS choices.
libbpf has to run on all kernels. Newer and older. How do you support that if libbpf is tied with the kernel?

Quoted 3 lines

Perhaps I'm not clever enough to follow. But if I'm allowed to make an uneducated guess it would be that it's some Facebook internal reason, like it's hard to do backports? :/

hard to do backports? of what?

#23 lore raw
Quoted 10 lines · Alexei Starovoitov
Quoted 9 lines · Jakub Kicinski
Quoted 7 lines · Alexei Starovoitov
Quoted 4 lines

Agreed. Having libbpf on GH is definitely useful today, but one can hope a day will come when distroes will get up to speed on packaging libbpf, and perhaps we can retire it? Maybe 2, 3 years from now? Putting bpftool in the same boat is just more baggage.

Distros should be packaging libbpf and bpftool from single repo on github. Kernel tree is for packaging kernel.

Quoted 6 lines
Quoted 3 lines

Okay, single repo on GitHub:

https://github.com/torvalds/linux

and how will you git submodule only libbpf part of kernel github into bcc and other projects?

Why does bcc have to submodule libbpf? Is it in a "special relationship" with libbpf as well?

dnf/apt install libbpf

Or rather:

dnf/apt install bcc

since BCC's user doesn't care about dependencies. The day distroes started packaging libbpf and bpftool the game has changed.

Quoted 4 lines
Quoted 2 lines

You also said a few times you don't want to merge fixes into bpf/net. That divergence from kernel development process is worrying.

worrying - why? what exactly the concern you see?

First it's still in the tree so it just feels wrong to have the process fundamentally different for sections of the tree.

Secondly it's best to stick to the tried and tested processes unless there's a good and stated reason to diverge.

Now we're neither doing standard user library process nor kernel process. Both of which must account for stable fixes (Dave explained the enterprise distro needs in his reply).

Quoted 2 lines

Tying user space release into kernel release and user space process into kernel process makes little sense to me.

I'm not saying we can't do user space, but we should pick one.

I have slight preference for kernel process, because it's familiar and it automatically takes of policy questions, which otherwise consume developer's time.

Please accept iproute2 as an example of a user space toolset closely related to the kernel. If kernel release model and process made no sense in user space, why do iproute2s developers continue to follow it for years?

Let's learn from experience :/

Quoted 1 line

Packaging is different.

There are mostly disadvantages, but the process should be well known. perf has been packaged for years.

iproute2 sometimes lags behind the upstream, more than perf in my experience. Also there's rarely a need to update tools closely related to the kernel without the kernel..

Quoted 1 line

Compatibility requirements are different.

True.

Quoted 1 line

CI is different.

Well, hard to argue about things which don't exist :/

Quoted 1 line

Integration with other projects is different.

IMHO the submodule use case is fading. dnf/apt install, it's just a normal C library, we've done this for decades.

And there is no submodule need for bpftool, which we are discussing.

Quoted 9 lines

libbpf source code is in the kernel tree only because kernel changes plus libbpf changes plus selftests changes come as single patchset. That is really the only reason. Packaging scripts, CI scripts, etc should be kept out of kernel tree. All that stuff belongs at github/libbpf.

Quoted 2 lines

None of this makes very much sense to me. We're diverging from well established development practices without as much as a justification.

The kernel development process was never used for libbpf.

What do you mean? I've sure as hell sent patches to net with Fixes tags, S-o-B and all that jazz for libbpf and bpftool.

Quoted 1 line

Even coding style is different.

Is it? You mean the damn underscores people are making fun of? :/

Quoted 2 lines

I'm puzzled why you think user space
should be tied to kernel. Everything is so vastly different.

Not what I'm saying, I'm saying either run it as user space project in separate repos or kernel project.

You're proposing we do both with some syncs and no clear process.

Quoted 2 lines

Some people say that 8 weeks to bump libbpf version is too long. Other people say that it's too often.

Those are two conflicting requirements, Alexei, how is breaking apart from the kernel going to make any difference?

At least kernel gives us this somewhat reasonable 8 week cadence and we don't have to ponder whether it should be shorter or longer until someone presents us with a compelling reason.

Quoted 1 line

libbpf version numbers != kernel version numbers.

Well, libbpf's numbers are arbitrary, and meaningless. They could as well be kernel numbers, like they are for bpftool. Or dates.

libbpf doesn't have a roadmap either, it's not really a full-on project on its own. What's 0.1.0 gonna be?

Besides stuff lands in libbpf before it hits a major kernel release. So how are you gonna make libbpf releases independently from kernel ones? What if a feature gets a last minute revert in the kernel and it's in libbpf's ABI?

Eh, you're just convincing me it should just stay in the kernel now.

Quoted 3 lines

There is no definition of LTS for libbpf. One day it will be and the version of libbpf picked for LTS will likely have nothing to do with kernel LTS choices.

Again, libbpf inherits the kernel process so there is LTS process, the kernel one, and it's taken care of for us by all the existing stable automation.

By merging everything into -next you're loosing that, with all but a vague mirage of how there may one day be multiple stable branches..

Quoted 2 lines

libbpf has to run on all kernels. Newer and older. How do you support that if libbpf is tied with the kernel?

Say I have built N kernels UM or for a VM, and we have some test
suite: I pull libbpf, build it, run its tests. The only difference between in tree and out of tree is that "pull libbpf" means pulling smaller or larger repo. Doesn't matter that match, it's a low --depth local clone.

Sorry for the long response.

#24 lore raw
Quoted 21 lines · Jakub Kicinski
Quoted 10 lines · Alexei Starovoitov
Quoted 9 lines · Jakub Kicinski
Quoted 7 lines · Alexei Starovoitov
Quoted 4 lines

Agreed. Having libbpf on GH is definitely useful today, but one can hope a day will come when distroes will get up to speed on packaging libbpf, and perhaps we can retire it? Maybe 2, 3 years from now? Putting bpftool in the same boat is just more baggage.

Distros should be packaging libbpf and bpftool from single repo on github. Kernel tree is for packaging kernel.

Quoted 6 lines
Quoted 3 lines

Okay, single repo on GitHub:

https://github.com/torvalds/linux

and how will you git submodule only libbpf part of kernel github into bcc and other projects?

Why does bcc have to submodule libbpf? Is it in a "special relationship" with libbpf as well?

Quoted 8 lines

dnf/apt install libbpf

Or rather:

dnf/apt install bcc

since BCC's user doesn't care about dependencies. The day distroes started packaging libbpf and bpftool the game has changed.

have you ever built bcc ? or bpftrace?
I'm not sure how to answer such 'suggestion'.

Quoted 4 lines

Please accept iproute2 as an example of a user space toolset closely related to the kernel. If kernel release model and process made no sense in user space, why do iproute2s developers continue to follow it for years?

imo iproute2 is an example how things should not be run. But that's a very different topic.

Quoted 4 lines
Quoted 1 line

Packaging is different.

There are mostly disadvantages, but the process should be well known. perf has been packaged for years.

perf was initially seen as something that should match kernel one to one. yet it diverged over years. I think it's a counter example.

Quoted 1 line

What do you mean? I've sure as hell sent patches to net with Fixes tags

which was complete waste of time for people who were sending these patches, for maintainers who applied them and for all stables folks who carried them into kernel stable releases.
Not a single libbpf build was made out of those sources.

Quoted 1 line

S-o-B and all that jazz for libbpf and bpftool.

Many open source projects use SOB. It's not kernel specific.

Quoted 4 lines
Quoted 2 lines

Even coding style is different.

Is it? You mean the damn underscores people are making fun of? :/

Are you trolling? Do you understand why __ is there?

Quoted 1 line

libbpf doesn't have a roadmap either,

I think you're contrasting that with kernel and saying that kernel has a roadmap ? What is kernel roadmap?

Quoted 2 lines

it's not really a full-on project
on its own. What's 0.1.0 gonna be?

whenever this bpf community decides to call it 0.1.0.

Quoted 4 lines

Besides stuff lands in libbpf before it hits a major kernel release. So how are you gonna make libbpf releases independently from kernel ones? What if a feature gets a last minute revert in the kernel and it's in libbpf's ABI?

You mean when kernel gets new feature, then libbpf gets new feature, then libbpf is released, but then kernel feature is reverted? Obviously we should avoid making a libbpf release that relies on kernel features that didn't reach the mainline. Yet there could be plenty of reasons why making libbpf release in the middle of kernel development cycle makes perfect sense.

Also reaching Linus's tree in rc1 is also not a guarantee of non-revert. Yet we release libbpf around rc1 because everyone expects bug-fixes after rc1. So it's an exception that solidifies the rule.

Quoted 8 lines
Quoted 2 lines

libbpf has to run on all kernels. Newer and older. How do you support that if libbpf is tied with the kernel?

Say I have built N kernels UM or for a VM, and we have some test
suite: I pull libbpf, build it, run its tests. The only difference between in tree and out of tree is that "pull libbpf" means pulling smaller or larger repo. Doesn't matter that match, it's a low --depth local clone.

The expected CI is:
1. pull-req proposed.
2. CI picks it up, builds, run tests.
3. humans see results and land or reject pull-req.
Now try to think through how CI on top of full kernel tree will be able to pick just the right commits to start build/test cycle. Is it going to cherry-pick from patchworks? That would be awesome. Yet intel 0bot results show that it's easier said than done. I'm not saying it's not possible. Just complex.
If you have cycles to integrate *-next into kernelci.org, please go ahead.

#25 lore raw
Quoted 26 lines · Alexei Starovoitov
Quoted 25 lines · Jakub Kicinski
Quoted 10 lines · Alexei Starovoitov
Quoted 9 lines · Jakub Kicinski
Quoted 7 lines · Alexei Starovoitov
Quoted 4 lines

Agreed. Having libbpf on GH is definitely useful today, but one can hope a day will come when distroes will get up to speed on packaging libbpf, and perhaps we can retire it? Maybe 2, 3 years from now? Putting bpftool in the same boat is just more baggage.

Distros should be packaging libbpf and bpftool from single repo on github. Kernel tree is for packaging kernel.

Quoted 6 lines
Quoted 3 lines

Okay, single repo on GitHub:

https://github.com/torvalds/linux

and how will you git submodule only libbpf part of kernel github into bcc and other projects?

Why does bcc have to submodule libbpf? Is it in a "special relationship" with libbpf as well?

dnf/apt install libbpf

Or rather:

Quoted 7 lines
Quoted 4 lines

dnf/apt install bcc

since BCC's user doesn't care about dependencies. The day distroes started packaging libbpf and bpftool the game has changed.

have you ever built bcc ? or bpftrace?
I'm not sure how to answer such 'suggestion'.

Perhaps someone else has more patience to explain it - why bcc can't just use binary libbpf distribution (static lib + headers) and link against it like it links against other libraries?

Quoted 7 lines
Quoted 4 lines

Please accept iproute2 as an example of a user space toolset closely related to the kernel. If kernel release model and process made no sense in user space, why do iproute2s developers continue to follow it for years?

imo iproute2 is an example how things should not be run. But that's a very different topic.

Please explain, the topic is how to maintain user space closely related to the kernel.

Share with us what you dislike about iproute2 so we can fix it. Instead of adding parts of it to bpftool and then pretending that the API added to libbpf to facilitate that duplication is some internal bpftool-only magic which then prevents us from dynamic linking..... 😠

Quoted 8 lines
Quoted 4 lines
Quoted 1 line

Packaging is different.

There are mostly disadvantages, but the process should be well known. perf has been packaged for years.

perf was initially seen as something that should match kernel one to one. yet it diverged over years. I think it's a counter example.

Quoted 6 lines
Quoted 1 line

What do you mean? I've sure as hell sent patches to net with Fixes tags

which was complete waste of time for people who were sending these patches, for maintainers who applied them and for all stables folks who carried them into kernel stable releases.
Not a single libbpf build was made out of those sources.

Because libbpf just now entered the distroes, and you suggested the distroes use the GH repo, so sure now it's wasted work.

IIRC there were bpftool crash fixes which landed in Fedora via stable.

Quoted 5 lines
Quoted 3 lines
Quoted 1 line

Even coding style is different.

Is it? You mean the damn underscores people are making fun of? :/

Are you trolling? Do you understand why __ is there?

Not the point. Tell me how the coding style is different. The underscores is the only thing I could think of that's not common in the kernel.

Quoted 4 lines
Quoted 1 line

libbpf doesn't have a roadmap either,

I think you're contrasting that with kernel and saying that kernel has a roadmap ? What is kernel roadmap?

Kernel road map is the same as libbpf's road map.

Quoted 10 lines
Quoted 2 lines

it's not really a full-on project
on its own. What's 0.1.0 gonna be?

whenever this bpf community decides to call it 0.1.0.

Quoted 4 lines

Besides stuff lands in libbpf before it hits a major kernel release. So how are you gonna make libbpf releases independently from kernel ones? What if a feature gets a last minute revert in the kernel and it's in libbpf's ABI?

Quoted 5 lines

You mean when kernel gets new feature, then libbpf gets new feature, then libbpf is released, but then kernel feature is reverted? Obviously we should avoid making a libbpf release that relies on kernel features that didn't reach the mainline. Yet there could be plenty of reasons why making libbpf release in the middle of kernel development cycle makes perfect sense.

But master of libbpf must have all features to test the kernel with, right? So how do we branch of a release in the middle? That's only possible if kernel cycle happens to not have had any features that required libbpf yet?

Or are you thinking 3 tier branching where we'd branch off libbpf release, say 2.6.0 that corresponds to kernel X, but it wouldn't be a stable-only release, and we can still backport features added in kernel X + 1 cycle, features which don't require kernel support, and release libbpf 2.7.0?

Could work but it'd get tricky, cause if we want to break ABI we'd actually need 4 tiers. ABI compat, kernel version, feature version, stable version.

Quoted 2 lines

Also reaching Linus's tree in rc1 is also not a guarantee of non-revert. Yet we release libbpf around rc1 because everyone expects bug-fixes after rc1.

I consider current process to be broken. Hopefully we can improve it.

Quoted 11 lines

So it's an exception that solidifies the rule.

Quoted 8 lines
Quoted 2 lines

libbpf has to run on all kernels. Newer and older. How do you support that if libbpf is tied with the kernel?

Say I have built N kernels UM or for a VM, and we have some test
suite: I pull libbpf, build it, run its tests. The only difference between in tree and out of tree is that "pull libbpf" means pulling smaller or larger repo. Doesn't matter that match, it's a low --depth local clone.

Quoted 10 lines

The expected CI is:
1. pull-req proposed.
2. CI picks it up, builds, run tests.
3. humans see results and land or reject pull-req.
Now try to think through how CI on top of full kernel tree will be able to pick just the right commits to start build/test cycle. Is it going to cherry-pick from patchworks? That would be awesome. Yet intel 0bot results show that it's easier said than done. I'm not saying it's not possible. Just complex.
If you have cycles to integrate *-next into kernelci.org, please go ahead.

Yes, it is very complex, I know. I've been hacking on something along those lines for the last few weeks. Hopefully I'll have results at some point..

First stab is just doing build testing, checkpatch, verify tags etc. Uploading to patchwork, and sending an email if there were failures.

Even that's not easy as a weekend/evening task :( And it requires a lot of manual inspection upfront before it's unleashed on the ML, because it will catch a lot of stupid little stuff and a lot of people will get grumpy.

We need to modernize the process across the board. I don't think having zombie read-only repos on GitHub will give contributors confidence so it's not a step in right direction. We should start from the hard problem, that is the CI itself.

The problem of correlating user space and kernel patches will have to be solved for netdev, because netdev tests depend on iproute2.

#26 lore raw
Quoted 32 lines · Jakub Kicinski
Quoted 31 lines · Alexei Starovoitov
Quoted 29 lines · Jakub Kicinski
Quoted 10 lines · Alexei Starovoitov
Quoted 9 lines · Jakub Kicinski
Quoted 7 lines · Alexei Starovoitov
Quoted 4 lines

Agreed. Having libbpf on GH is definitely useful today, but one can hope a day will come when distroes will get up to speed on packaging libbpf, and perhaps we can retire it? Maybe 2, 3 years from now? Putting bpftool in the same boat is just more baggage.

Distros should be packaging libbpf and bpftool from single repo on github. Kernel tree is for packaging kernel.

Quoted 6 lines
Quoted 3 lines

Okay, single repo on GitHub:

https://github.com/torvalds/linux

and how will you git submodule only libbpf part of kernel github into bcc and other projects?

Why does bcc have to submodule libbpf? Is it in a "special relationship" with libbpf as well?

dnf/apt install libbpf

Or rather:

dnf/apt install bcc

since BCC's user doesn't care about dependencies. The day distroes started packaging libbpf and bpftool the game has changed.

Quoted 6 lines
Quoted 2 lines

have you ever built bcc ? or bpftrace?
I'm not sure how to answer such 'suggestion'.

Perhaps someone else has more patience to explain it - why bcc can't just use binary libbpf distribution (static lib + headers) and link against it like it links against other libraries?

why systemd considered using libbpf as submodule ?
When project like bcc needs to run on different distros and different versions of the same distro such project cannot force users to upgrade core libraries to get features that project might need. libbpf is far from stability of libc, libmnl, libelf. There are mature libraries and actively developed libraries. libbpf is at its infancy stage. We're trying hard to be stable, but accumulated baggage is already huge and it is slowing us down. systemd is considering to use libbpf without being submodule too, but it's mainly driven by systemd's CI limitations and nothing else.

Quoted 1 line

Share with us what you dislike about iproute2 so we can fix it.

let's start with 24-hr review cycle that we're trying hard to keep in bpf/net trees. Can you help with 24-hr review in iproute2 ?

Quoted 2 lines

Because libbpf just now entered the distroes, and you suggested the distroes use the GH repo, so sure now it's wasted work.

You got the timeline wrong. GH repo was done before some distros started packaging libbpf. libbpf is still not available in centos/rhel afaik. But bcc and bpftrace need to work there. Answer? submodules...

Quoted 5 lines
Quoted 1 line

Are you trolling? Do you understand why __ is there?

Not the point. Tell me how the coding style is different. The underscores is the only thing I could think of that's not common in the kernel.

we don't actively enforce xmas tree :)

Quoted 4 lines

But master of libbpf must have all features to test the kernel with, right? So how do we branch of a release in the middle? That's only possible if kernel cycle happens to not have had any features that required libbpf yet?

libbpf rate of changes is higher than bpf bits of the kernel. Amount of patches per week is higher as well.
Something like skeleton work might warrant independent release just to get into the hands of people faster. Why wait 8 weeks for kernel release? No reason.

Quoted 2 lines

Or are you thinking 3 tier branching where we'd branch off libbpf release, say 2.6.0 that corresponds to kernel X,

libbpf does not correspond to kernel.
All libbpf releases must work with any kernel.

Quoted 4 lines

but it wouldn't be a
stable-only release, and we can still backport features added in kernel X + 1 cycle, features which don't require kernel support, and release libbpf 2.7.0?

Say libbpf 0.1 was developed during kernel 5.4.
We can 'backport' all libbpf features from kernel tree 5.5 into libbpf 0.1+ and that libbpf 0.1+ must work with kernels 5.4 and 5.5.
That is what existing github/libbpf CI is testing.

#27 lore raw
In reply to Alexei Starovoitov
Quoted 7 lines · On Wed, 4 Dec 2019 15:39:49 -0800 Alexei Starovoitov…
Quoted 6 lines · Jakub Kicinski
Quoted 5 lines · Andrii Nakryiko

I wonder what big advantage having bpftool in libbpf's Github repo brings, actually? The reason we need libbpf on github is to allow other projects like pahole to be able to use libbpf from submodule. There is no such need for bpftool.

Quoted 6 lines
Quoted 6 lines
Quoted 4 lines

I agree about preference to release them in sync, but that could be easily done by releasing based on corresponding commits in github's libbpf repo and kernel repo. bpftool doesn't have to physically live next to libbpf on Github, does it?

+1

I don't see any advantage of having bpftool in libbpf's GitHub repo.

As Jakub mention we have seen bpftool crash fixes, which would be painful/annoying to maintain fixes for in the libbpf GitHub repo.

As Andrii also points out, it requires more work, as GitHub libbpf have to maintain a separate Makefile for bpftool.

Quoted 8 lines
Quoted 8 lines
Quoted 7 lines

Calling github repo a "mirror" is incorrect. It's not a 1:1 copy of files. We have a completely separate Makefile for libbpf, and we have a bunch of stuff we had to re-implement to detach libbpf code from kernel's non-UAPI headers. Doing this for bpftool as well seems like just more maintenance. Keeping github's Makefile in sync with kernel's Makefile (for libbpf) is PITA, I'd rather avoid similar pains for bpftool without a really good reason.

Quoted 7 lines
Quoted 4 lines

Agreed. Having libbpf on GH is definitely useful today, but one can hope a day will come when distroes will get up to speed on packaging libbpf, and perhaps we can retire it? Maybe 2, 3 years from now? Putting bpftool in the same boat is just more baggage.

Distros should be packaging libbpf and bpftool from single repo on github. Kernel tree is for packaging kernel.

I don't think that is a good idea. You are creating double work and wasting distro developers time. Let me explain:

1. First of all, GitHub libbpf does not have a stable branches (which makes sense, given this is a read-only clone of kernel tree). Thus, distro developers have to maintain that themselves, in their internal package tree (that is based on GitHub libbpf).

2. Kernel BPF changes usually require updates to libbpf, as selftests uses libbpf. Thus, the distro kernel backporter is already required to backport libbpf parts.

This is double work, the code changes to libbpf are now maintained in two places for the distro.

The disadvantage for distros to package libbpf (+ bpftool and perf) off their distro kernel tree is that a fix to libbpf, requires rolling a new kernel minor release. The solution to that is simply that distro package for libbpf have a separate (RPM) spec file, with own versioning, which sources points to distro kernel tree.

Signature · 4 lines
Best regards,
  Jesper Dangaard Brouer
  MSc.CS, Principal Kernel Engineer at Red Hat
  LinkedIn: http://www.linkedin.com/in/brouer
#28 lore raw
Quoted 8 lines · Jesper Dangaard Brouer

I don't think that is a good idea. You are creating double work and wasting distro developers time. Let me explain:

1. First of all, GitHub libbpf does not have a stable branches (which makes sense, given this is a read-only clone of kernel tree). Thus, distro developers have to maintain that themselves, in their internal package tree (that is based on GitHub libbpf).

Quoted 6 lines

2. Kernel BPF changes usually require updates to libbpf, as selftests uses libbpf. Thus, the distro kernel backporter is already required to backport libbpf parts.

This is double work, the code changes to libbpf are now maintained in two places for the distro.

I totally agree with Jesper here.

I don't know how the situatiom with packaging libbpf and bpftool looks like in Fedora/Centos/RHEL now, but in openSUSE we would like to build both of them from the kernel source - use kernel-source package as a requirement and use the kernel tree from /usr/src/linux to build those. We do that for bpftool and perf currently.

So far we are building bpftool and perf without libbpf being dynamically linked, so there is no dependency between those packages, although we would like to change it as soon as we find a consensus on this series of patches.

Quoted 5 lines

The disadvantage for distros to package libbpf (+ bpftool and perf) off their distro kernel tree is that a fix to libbpf, requires rolling a new kernel minor release. The solution to that is simply that distro package for libbpf have a separate (RPM) spec file, with own versioning, which sources points to distro kernel tree.

That's a great idea. So far, we are using the same version for kernel, bfptool and perf, but all of these have separate RPM specs, so we can do that.

Michal