Skip to contentSkip to conversation

bpftool: Switch to independent versioning scheme

PATCH In review Quentin Monnet 2 participants 8 messages Jan 31, 2022 – Feb 2, 2022

Subsystems (5)
lore
Share & tools
lore
#0 lore raw

cover bpftool: Switch to independent versioning scheme

Hi, this set aims at updating the way bpftool versions are numbered. Instead of copying the version from the kernel (given that the sources for the kernel and bpftool are shipped together), introduce an independent versioning scheme. We start at v6.0.0 - incrementing the major version number - and the idea is to update this number from time to time, as new features or bug fixes make their way into bpftool. Please refer to the description of the third commit for details on the motivations.

The patchset also adds the number of the version of libbpf that was used to compile to the output of "bpftool version".

Quentin Monnet (3):
libbpf: Add "libbpversion" make target to print version
bpftool: Add libbpf's version number to "bpftool version" output
bpftool: Update versioning scheme

 tools/bpf/bpftool/Documentation/common_options.rst | 3 ++-
 tools/bpf/bpftool/Makefile                         | 7 ++++---
 tools/bpf/bpftool/main.c                           | 3 +++
 tools/lib/bpf/Makefile                             | 3 +++
 4 files changed, 12 insertions(+), 4 deletions(-)
Signature · 1 line
2.32.0
#1 lore raw

PATCH 1/3 libbpf: Add "libbpversion" make target to print version

Add a target to libbpf's Makefile to print its version number, in a similar way to what running "make kernelversion" at the root of the repository does.

This is to avoid re-implementing the parsing of the libbpf.map file in case some other tools want to extract the version of the libbpf sources they are using.

---
 tools/lib/bpf/Makefile | 3 +++
 1 file changed, 3 insertions(+)
tools/lib/bpf/Makefile +3 -0
index f947b61b2107..e3a1ae7efa33 100644
@@ -108,6 +108,9 @@ MAKEOVERRIDES=
 
 all:
 
+libbpfversion:
+	@echo $(LIBBPF_VERSION)
+
 export srctree OUTPUT CC LD CFLAGS V
 include $(srctree)/tools/build/Makefile.include
 
Signature · 1 line
2.32.0
#2 lore raw

PATCH 2/3 bpftool: Add libbpf's version number to "bpftool version" output

In reply to Quentin Monnet

To help users check what version of libbpf has been used to compile bpftool, embed the version number and print it along with bpftool's own version number.

---
 tools/bpf/bpftool/Documentation/common_options.rst | 3 ++-
 tools/bpf/bpftool/Makefile                         | 2 ++
 tools/bpf/bpftool/main.c                           | 3 +++
 3 files changed, 7 insertions(+), 1 deletion(-)
tools/bpf/bpftool/Documentation/common_options.rst +2 -1
index 908487b9c2ad..24166733d3ae 100644
@@ -4,7 +4,8 @@
 	  Print short help message (similar to **bpftool help**).
 
 -V, --version
-	  Print version number (similar to **bpftool version**), and optional
+	  Print bpftool's version number (similar to **bpftool version**), the
+	  version of libbpf that was used to compile the binary, and optional
 	  features that were included when bpftool was compiled. Optional
 	  features include linking against libbfd to provide the disassembler
 	  for JIT-ted programs (**bpftool prog dump jited**) and usage of BPF
tools/bpf/bpftool/Makefile +2 -0
index 83369f55df61..bd5a8cafac49 100644
@@ -42,6 +42,7 @@ LIBBPF_BOOTSTRAP_INTERNAL_HDRS := $(addprefix $(LIBBPF_BOOTSTRAP_HDRS_DIR)/,hash
 ifeq ($(BPFTOOL_VERSION),)
 BPFTOOL_VERSION := $(shell make -rR --no-print-directory -sC ../../.. kernelversion)
 endif
+LIBBPF_VERSION := $(shell make -r --no-print-directory -sC $(BPF_DIR) libbpfversion)
 
 $(LIBBPF_OUTPUT) $(BOOTSTRAP_OUTPUT) $(LIBBPF_BOOTSTRAP_OUTPUT) $(LIBBPF_HDRS_DIR) $(LIBBPF_BOOTSTRAP_HDRS_DIR):
 	$(QUIET_MKDIR)mkdir -p $@
@@ -84,6 +85,7 @@ CFLAGS += -DPACKAGE='"bpftool"' -D__EXPORTED_HEADERS__ \
 	-I$(srctree)/tools/include \
 	-I$(srctree)/tools/include/uapi
 CFLAGS += -DBPFTOOL_VERSION='"$(BPFTOOL_VERSION)"'
+CFLAGS += -DLIBBPF_VERSION='"$(LIBBPF_VERSION)"'
 ifneq ($(EXTRA_CFLAGS),)
 CFLAGS += $(EXTRA_CFLAGS)
 endif
tools/bpf/bpftool/main.c +3 -0
index 9d01fa9de033..4bda73057980 100644
@@ -89,6 +89,8 @@ static int do_version(int argc, char **argv)
 
 		jsonw_name(json_wtr, "version");
 		jsonw_printf(json_wtr, "\"%s\"", BPFTOOL_VERSION);
+		jsonw_name(json_wtr, "libbpf_version");
+		jsonw_printf(json_wtr, "\"%s\"", LIBBPF_VERSION);
 
 		jsonw_name(json_wtr, "features");
 		jsonw_start_object(json_wtr);	/* features */
@@ -102,6 +104,7 @@ static int do_version(int argc, char **argv)
 		unsigned int nb_features = 0;
 
 		printf("%s v%s\n", bin_name, BPFTOOL_VERSION);
+		printf("using libbpf v%s\n", LIBBPF_VERSION);
 		printf("features:");
 		if (has_libbfd) {
 			printf(" libbfd");
Signature · 1 line
2.32.0
#3 lore raw

Re: [PATCH bpf-next 2/3] bpftool: Add libbpf's version number to "bpftool version" output

Quoted 31 lines · Quentin Monnet

To help users check what version of libbpf has been used to compile bpftool, embed the version number and print it along with bpftool's own version number.

---
 tools/bpf/bpftool/Documentation/common_options.rst | 3 ++-
 tools/bpf/bpftool/Makefile                         | 2 ++
 tools/bpf/bpftool/main.c                           | 3 +++
 3 files changed, 7 insertions(+), 1 deletion(-)
tools/bpf/bpftool/Documentation/common_options.rst +2 -1
index 908487b9c2ad..24166733d3ae 100644
@@ -4,7 +4,8 @@
          Print short help message (similar to **bpftool help**).
 
 -V, --version
-         Print version number (similar to **bpftool version**), and optional
+         Print bpftool's version number (similar to **bpftool version**), the
+         version of libbpf that was used to compile the binary, and optional
          features that were included when bpftool was compiled. Optional
          features include linking against libbfd to provide the disassembler
          for JIT-ted programs (**bpftool prog dump jited**) and usage of BPF
tools/bpf/bpftool/Makefile +0 -0
index 83369f55df61..bd5a8cafac49 100644

tools/bpf/bpftool/Makefile LIBBPF_BOOTSTRAP_INTERNAL_HDRS := $(addprefix $(LIBBPF_BOOTSTRAP_HDRS_DIR)/,hash · line 46

Quoted 5 lines
tools/bpf/bpftool/Makefile +1 -0
@@ -42,6 +42,7 @@ LIBBPF_BOOTSTRAP_INTERNAL_HDRS := $(addprefix $(LIBBPF_BOOTSTRAP_HDRS_DIR)/,hash
 ifeq ($(BPFTOOL_VERSION),)
 BPFTOOL_VERSION := $(shell make -rR --no-print-directory -sC ../../.. kernelversion)
 endif
+LIBBPF_VERSION := $(shell make -r --no-print-directory -sC $(BPF_DIR) libbpfversion)
 

why can't you use libbpf_version_string() API instead?

Quoted 34 lines
tools/bpf/bpftool/Makefile +1 -0
 $(LIBBPF_OUTPUT) $(BOOTSTRAP_OUTPUT) $(LIBBPF_BOOTSTRAP_OUTPUT) $(LIBBPF_HDRS_DIR) $(LIBBPF_BOOTSTRAP_HDRS_DIR):
        $(QUIET_MKDIR)mkdir -p $@
@@ -84,6 +85,7 @@ CFLAGS += -DPACKAGE='"bpftool"' -D__EXPORTED_HEADERS__ \
        -I$(srctree)/tools/include \
        -I$(srctree)/tools/include/uapi
 CFLAGS += -DBPFTOOL_VERSION='"$(BPFTOOL_VERSION)"'
+CFLAGS += -DLIBBPF_VERSION='"$(LIBBPF_VERSION)"'
 ifneq ($(EXTRA_CFLAGS),)
 CFLAGS += $(EXTRA_CFLAGS)
 endif
tools/bpf/bpftool/main.c +3 -1
index 9d01fa9de033..4bda73057980 100644
@@ -89,6 +89,8 @@ static int do_version(int argc, char **argv)
 
                jsonw_name(json_wtr, "version");
                jsonw_printf(json_wtr, "\"%s\"", BPFTOOL_VERSION);
+               jsonw_name(json_wtr, "libbpf_version");
+               jsonw_printf(json_wtr, "\"%s\"", LIBBPF_VERSION);
 
                jsonw_name(json_wtr, "features");
                jsonw_start_object(json_wtr);   /* features */
@@ -102,6 +104,7 @@ static int do_version(int argc, char **argv)
                unsigned int nb_features = 0;
 
                printf("%s v%s\n", bin_name, BPFTOOL_VERSION);
+               printf("using libbpf v%s\n", LIBBPF_VERSION);
                printf("features:");
                if (has_libbfd) {
                        printf(" libbfd");
--

2.32.0

#4 lore raw

Re: [PATCH bpf-next 2/3] bpftool: Add libbpf's version number to "bpftool version" output

2022-02-01 22:59 UTC-0800 ~ Andrii Nakryiko <[email protected]>

Quoted 31 lines · Quentin Monnet
Quoted 30 lines

To help users check what version of libbpf has been used to compile bpftool, embed the version number and print it along with bpftool's own version number.

---
 tools/bpf/bpftool/Documentation/common_options.rst | 3 ++-
 tools/bpf/bpftool/Makefile                         | 2 ++
 tools/bpf/bpftool/main.c                           | 3 +++
 3 files changed, 7 insertions(+), 1 deletion(-)
tools/bpf/bpftool/Documentation/common_options.rst +2 -1
index 908487b9c2ad..24166733d3ae 100644
@@ -4,7 +4,8 @@
          Print short help message (similar to **bpftool help**).
 
 -V, --version
-         Print version number (similar to **bpftool version**), and optional
+         Print bpftool's version number (similar to **bpftool version**), the
+         version of libbpf that was used to compile the binary, and optional
          features that were included when bpftool was compiled. Optional
          features include linking against libbfd to provide the disassembler
          for JIT-ted programs (**bpftool prog dump jited**) and usage of BPF
tools/bpf/bpftool/Makefile +0 -0
index 83369f55df61..bd5a8cafac49 100644
Quoted 8 lines

tools/bpf/bpftool/Makefile LIBBPF_BOOTSTRAP_INTERNAL_HDRS := $(addprefix $(LIBBPF_BOOTSTRAP_HDRS_DIR)/,hash · line 46

Quoted 6 lines
tools/bpf/bpftool/Makefile +1 -0
@@ -42,6 +42,7 @@ LIBBPF_BOOTSTRAP_INTERNAL_HDRS := $(addprefix $(LIBBPF_BOOTSTRAP_HDRS_DIR)/,hash
 ifeq ($(BPFTOOL_VERSION),)
 BPFTOOL_VERSION := $(shell make -rR --no-print-directory -sC ../../.. kernelversion)
 endif
+LIBBPF_VERSION := $(shell make -r --no-print-directory -sC $(BPF_DIR) libbpfversion)
 

why can't you use libbpf_version_string() API instead?

I missed it somehow, thanks for the pointer. It seems to be a recent addition to libbpf, and it was probably not present last time I checked for such an API. I'll use it.

#5 lore raw

PATCH 3/3 bpftool: Update versioning scheme

In reply to Quentin Monnet

Since the notion of versions was introduced for bpftool, it has been following the version number of the kernel (using the version number corresponding to the tree in which bpftool's sources are located). The rationale was that bpftool's features are loosely tied to BPF features in the kernel, and that we could defer versioning to the kernel repository itself.

But this versioning scheme is confusing today, because a bpftool binary should be able to work with both older and newer kernels, even if some of its recent features won't be available on older systems. Furthermore, if bpftool is ported to other systems in the future, keeping a Linux-based version number is not a good option.

It would make more sense to align bpftool's number on libbpf, maybe. When versioning was introduced in bpftool, libbpf was in its initial phase at v0.0.1. Now it moves faster, with regular version bumps. But there are two issues if we want to pick the same numbers. First, that would mean going backward on the numbering, and will be a huge pain for every script trying to determine which bpftool binary is the most recent (not to mention some possible overlap of the numbers in a distant future). Then, bpftool could get new features or bug fixes between two versions libbpf, so maybe we should not completely tie its versions to libbpf, either.

Therefore, this commit introduces an independent versioning scheme for bpftool. The new version is v6.0.0, with its major number incremented over the current 5.16.* returned from the kernel's Makefile. The plan is to update this new number from time to time when bpftool gets new features or new bug fixes. These updates could possibly lead to new releases being tagged on the recently created out-of-tree mirror, at
https://github.com/libbpf/bpftool.

Version number is moved higher in the Makefile, to make it more visible.

---
 tools/bpf/bpftool/Makefile | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)
tools/bpf/bpftool/Makefile +2 -3
index bd5a8cafac49..b7dbdea112d3 100644
@@ -1,6 +1,8 @@
 # SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
 include ../../scripts/Makefile.include
 
+BPFTOOL_VERSION := 6.0.0
+
 ifeq ($(srctree),)
 srctree := $(patsubst %/,%,$(dir $(CURDIR)))
 srctree := $(patsubst %/,%,$(dir $(srctree)))
@@ -39,9 +41,6 @@ LIBBPF_BOOTSTRAP := $(LIBBPF_BOOTSTRAP_OUTPUT)libbpf.a
 LIBBPF_INTERNAL_HDRS := $(addprefix $(LIBBPF_HDRS_DIR)/,hashmap.h nlattr.h)
 LIBBPF_BOOTSTRAP_INTERNAL_HDRS := $(addprefix $(LIBBPF_BOOTSTRAP_HDRS_DIR)/,hashmap.h)
 
-ifeq ($(BPFTOOL_VERSION),)
-BPFTOOL_VERSION := $(shell make -rR --no-print-directory -sC ../../.. kernelversion)
-endif
 LIBBPF_VERSION := $(shell make -r --no-print-directory -sC $(BPF_DIR) libbpfversion)
 
 $(LIBBPF_OUTPUT) $(BOOTSTRAP_OUTPUT) $(LIBBPF_BOOTSTRAP_OUTPUT) $(LIBBPF_HDRS_DIR) $(LIBBPF_BOOTSTRAP_HDRS_DIR):
Signature · 1 line
2.32.0
#6 lore raw

Re: [PATCH bpf-next 3/3] bpftool: Update versioning scheme

Quoted 46 lines · Quentin Monnet

Since the notion of versions was introduced for bpftool, it has been following the version number of the kernel (using the version number corresponding to the tree in which bpftool's sources are located). The rationale was that bpftool's features are loosely tied to BPF features in the kernel, and that we could defer versioning to the kernel repository itself.

But this versioning scheme is confusing today, because a bpftool binary should be able to work with both older and newer kernels, even if some of its recent features won't be available on older systems. Furthermore, if bpftool is ported to other systems in the future, keeping a Linux-based version number is not a good option.

It would make more sense to align bpftool's number on libbpf, maybe. When versioning was introduced in bpftool, libbpf was in its initial phase at v0.0.1. Now it moves faster, with regular version bumps. But there are two issues if we want to pick the same numbers. First, that would mean going backward on the numbering, and will be a huge pain for every script trying to determine which bpftool binary is the most recent (not to mention some possible overlap of the numbers in a distant future). Then, bpftool could get new features or bug fixes between two versions libbpf, so maybe we should not completely tie its versions to libbpf, either.

Therefore, this commit introduces an independent versioning scheme for bpftool. The new version is v6.0.0, with its major number incremented over the current 5.16.* returned from the kernel's Makefile. The plan is to update this new number from time to time when bpftool gets new features or new bug fixes. These updates could possibly lead to new releases being tagged on the recently created out-of-tree mirror, at
https://github.com/libbpf/bpftool.

Version number is moved higher in the Makefile, to make it more visible.

---
 tools/bpf/bpftool/Makefile | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)
tools/bpf/bpftool/Makefile +0 -0
index bd5a8cafac49..b7dbdea112d3 100644

tools/bpf/bpftool/Makefile line 5

Quoted 5 lines
tools/bpf/bpftool/Makefile +2 -0
@@ -1,6 +1,8 @@
 # SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
 include ../../scripts/Makefile.include
 
+BPFTOOL_VERSION := 6.0.0
+

It's going to be a PITA to not forget to update this :( As discussed, I'm fine with this, but I also recalled the versioning approach that libbpf-sys library is using (see [0]). Maybe we could steal some of those ideas. As in, base bpftool version on libbpf (with major version
+ 6 as you do here), but also have "-1", "-2", etc suffixes for bpftool releases for when libbpf version didn't change. Don't know, just throwing out the idea for your consideration.

[0] https://github.com/libbpf/libbpf-sys#versioning

Quoted 16 lines
tools/bpf/bpftool/Makefile +0 -4
 ifeq ($(srctree),)
 srctree := $(patsubst %/,%,$(dir $(CURDIR)))
 srctree := $(patsubst %/,%,$(dir $(srctree)))
@@ -39,9 +41,6 @@ LIBBPF_BOOTSTRAP := $(LIBBPF_BOOTSTRAP_OUTPUT)libbpf.a
 LIBBPF_INTERNAL_HDRS := $(addprefix $(LIBBPF_HDRS_DIR)/,hashmap.h nlattr.h)
 LIBBPF_BOOTSTRAP_INTERNAL_HDRS := $(addprefix $(LIBBPF_BOOTSTRAP_HDRS_DIR)/,hashmap.h)
 
-ifeq ($(BPFTOOL_VERSION),)
-BPFTOOL_VERSION := $(shell make -rR --no-print-directory -sC ../../.. kernelversion)
-endif
 LIBBPF_VERSION := $(shell make -r --no-print-directory -sC $(BPF_DIR) libbpfversion)
 
 $(LIBBPF_OUTPUT) $(BOOTSTRAP_OUTPUT) $(LIBBPF_BOOTSTRAP_OUTPUT) $(LIBBPF_HDRS_DIR) $(LIBBPF_BOOTSTRAP_HDRS_DIR):
--

2.32.0

#7 lore raw

Re: [PATCH bpf-next 3/3] bpftool: Update versioning scheme

2022-02-01 22:59 UTC-0800 ~ Andrii Nakryiko <[email protected]>

Quoted 61 lines · Quentin Monnet
Quoted 46 lines

Since the notion of versions was introduced for bpftool, it has been following the version number of the kernel (using the version number corresponding to the tree in which bpftool's sources are located). The rationale was that bpftool's features are loosely tied to BPF features in the kernel, and that we could defer versioning to the kernel repository itself.

But this versioning scheme is confusing today, because a bpftool binary should be able to work with both older and newer kernels, even if some of its recent features won't be available on older systems. Furthermore, if bpftool is ported to other systems in the future, keeping a Linux-based version number is not a good option.

It would make more sense to align bpftool's number on libbpf, maybe. When versioning was introduced in bpftool, libbpf was in its initial phase at v0.0.1. Now it moves faster, with regular version bumps. But there are two issues if we want to pick the same numbers. First, that would mean going backward on the numbering, and will be a huge pain for every script trying to determine which bpftool binary is the most recent (not to mention some possible overlap of the numbers in a distant future). Then, bpftool could get new features or bug fixes between two versions libbpf, so maybe we should not completely tie its versions to libbpf, either.

Therefore, this commit introduces an independent versioning scheme for bpftool. The new version is v6.0.0, with its major number incremented over the current 5.16.* returned from the kernel's Makefile. The plan is to update this new number from time to time when bpftool gets new features or new bug fixes. These updates could possibly lead to new releases being tagged on the recently created out-of-tree mirror, at
https://github.com/libbpf/bpftool.

Version number is moved higher in the Makefile, to make it more visible.

---
 tools/bpf/bpftool/Makefile | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)
tools/bpf/bpftool/Makefile +0 -0
index bd5a8cafac49..b7dbdea112d3 100644

tools/bpf/bpftool/Makefile line 5

Quoted 5 lines
tools/bpf/bpftool/Makefile +2 -0
@@ -1,6 +1,8 @@
 # SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
 include ../../scripts/Makefile.include
 
+BPFTOOL_VERSION := 6.0.0
+

It's going to be a PITA to not forget to update this :( As discussed, I'm fine with this, but I also recalled the versioning approach that libbpf-sys library is using (see [0]). Maybe we could steal some of those ideas. As in, base bpftool version on libbpf (with major version
+ 6 as you do here), but also have "-1", "-2", etc suffixes for bpftool releases for when libbpf version didn't change. Don't know, just throwing out the idea for your consideration.

Quoted 1 line

[0] https://github.com/libbpf/libbpf-sys#versioning

I've been somewhat torn between having a separate versioning scheme and getting as much flexibility as we want, and aligning on libbpf and having “automatic” version updates. My reasoning is the following.

If aligning on libbpf:

- We may want bpftool releases in-between two libbpf versions. Using pre-release numbers for tagging them is a good idea, although I don't know if something marked as a pre-release will look “official” enough (users may not pick it, thinking it's a beta release?). On the other hand, having bpftool with version numbers that look “official” haven't really been an issue so far.

- If no new feature lands in bpftool for some time, we may move from e.g. 6.7.0 to 6.8.0 when libbpf levels up and have two different versions which are in fact the same.

- Following libbpf's versioning scheme sounds better than kernel's, but ultimately it doesn't make too much sense either, because even though bpftool uses the lib a lot, its behaviour is not that much conditioned by the internal evolution of the library (or by new APIs that it may not use).

Having an independent versioning scheme solves the above, but as you say, it's gonna be painful to update the numbers. Developers will miss it most of the time, and I'm not even exactly sure myself of when to tag a new minor release. I suppose it would be a bit like for docs and completion, with occasional “catch-up” patches to update the version number - not great.

Based on all the above, I think your suggestion is good, and I'll switch back to aligning on libbpf's version. It may not be perfect, but 1) it's certainly an improvement over the current scheme, 2) the issues raised above are minor at the moment, and 3) we can still move to an independent scheme in the future if we realise we need it. Sounds more important to save on the maintenance burden at the moment. I'll send a new version shortly.

Thanks,
Quentin