bpftool: Switch to independent versioning scheme
Subsystems (5)
cover bpftool: Switch to independent versioning scheme
Hi, this set aims at updating the way bpftool versions are numbered. Instead of copying the version from the kernel (given that the sources for the kernel and bpftool are shipped together), introduce an independent versioning scheme. We start at v6.0.0 - incrementing the major version number - and the idea is to update this number from time to time, as new features or bug fixes make their way into bpftool. Please refer to the description of the third commit for details on the motivations.
The patchset also adds the number of the version of libbpf that was used to compile to the output of "bpftool version".
Quentin Monnet (3):
libbpf: Add "libbpversion" make target to print version
bpftool: Add libbpf's version number to "bpftool version" output
bpftool: Update versioning scheme
tools/bpf/bpftool/Documentation/common_options.rst | 3 ++- tools/bpf/bpftool/Makefile | 7 ++++--- tools/bpf/bpftool/main.c | 3 +++ tools/lib/bpf/Makefile | 3 +++ 4 files changed, 12 insertions(+), 4 deletions(-)
Signature · 1 line
2.32.0
PATCH 1/3 libbpf: Add "libbpversion" make target to print version
Add a target to libbpf's Makefile to print its version number, in a similar way to what running "make kernelversion" at the root of the repository does.
This is to avoid re-implementing the parsing of the libbpf.map file in case some other tools want to extract the version of the libbpf sources they are using.
--- tools/lib/bpf/Makefile | 3 +++ 1 file changed, 3 insertions(+)
tools/lib/bpf/Makefile +3 -0
@@ -108,6 +108,9 @@ MAKEOVERRIDES= all: +libbpfversion: + @echo $(LIBBPF_VERSION) + export srctree OUTPUT CC LD CFLAGS V include $(srctree)/tools/build/Makefile.include
Signature · 1 line
2.32.0
PATCH 2/3 bpftool: Add libbpf's version number to "bpftool version" output
In reply to Quentin MonnetTo help users check what version of libbpf has been used to compile bpftool, embed the version number and print it along with bpftool's own version number.
--- tools/bpf/bpftool/Documentation/common_options.rst | 3 ++- tools/bpf/bpftool/Makefile | 2 ++ tools/bpf/bpftool/main.c | 3 +++ 3 files changed, 7 insertions(+), 1 deletion(-)
tools/bpf/bpftool/Documentation/common_options.rst +2 -1
@@ -4,7 +4,8 @@ Print short help message (similar to **bpftool help**). -V, --version - Print version number (similar to **bpftool version**), and optional + Print bpftool's version number (similar to **bpftool version**), the + version of libbpf that was used to compile the binary, and optional features that were included when bpftool was compiled. Optional features include linking against libbfd to provide the disassembler for JIT-ted programs (**bpftool prog dump jited**) and usage of BPF
tools/bpf/bpftool/Makefile +2 -0
@@ -42,6 +42,7 @@ LIBBPF_BOOTSTRAP_INTERNAL_HDRS := $(addprefix $(LIBBPF_BOOTSTRAP_HDRS_DIR)/,hash ifeq ($(BPFTOOL_VERSION),) BPFTOOL_VERSION := $(shell make -rR --no-print-directory -sC ../../.. kernelversion) endif +LIBBPF_VERSION := $(shell make -r --no-print-directory -sC $(BPF_DIR) libbpfversion) $(LIBBPF_OUTPUT) $(BOOTSTRAP_OUTPUT) $(LIBBPF_BOOTSTRAP_OUTPUT) $(LIBBPF_HDRS_DIR) $(LIBBPF_BOOTSTRAP_HDRS_DIR): $(QUIET_MKDIR)mkdir -p $@
@@ -84,6 +85,7 @@ CFLAGS += -DPACKAGE='"bpftool"' -D__EXPORTED_HEADERS__ \ -I$(srctree)/tools/include \ -I$(srctree)/tools/include/uapi CFLAGS += -DBPFTOOL_VERSION='"$(BPFTOOL_VERSION)"' +CFLAGS += -DLIBBPF_VERSION='"$(LIBBPF_VERSION)"' ifneq ($(EXTRA_CFLAGS),) CFLAGS += $(EXTRA_CFLAGS) endif
tools/bpf/bpftool/main.c +3 -0
@@ -89,6 +89,8 @@ static int do_version(int argc, char **argv)jsonw_name (json_wtr,"version" );jsonw_printf (json_wtr,"\"%s\"" ,BPFTOOL_VERSION ); +jsonw_name (json_wtr,"libbpf_version" ); +jsonw_printf (json_wtr,"\"%s\"" ,LIBBPF_VERSION );jsonw_name (json_wtr,"features" );jsonw_start_object (json_wtr);/* features */
@@ -102,6 +104,7 @@ static int do_version(int argc, char **argv)unsigned int nb_features =0 ;printf ("%s v%s\n" , bin_name,BPFTOOL_VERSION ); +printf ("using libbpf v%s\n" ,LIBBPF_VERSION );printf ("features:" );if (has_libbfd) {printf (" libbfd" );
Signature · 1 line
2.32.0
Re: [PATCH bpf-next 2/3] bpftool: Add libbpf's version number to "bpftool version" output
Quoted 31 lines · Quentin Monnet
To help users check what version of libbpf has been used to compile bpftool, embed the version number and print it along with bpftool's own version number.
Signed-off-by Quentin Monnet <[email protected]>--- tools/bpf/bpftool/Documentation/common_options.rst | 3 ++- tools/bpf/bpftool/Makefile | 2 ++ tools/bpf/bpftool/main.c | 3 +++ 3 files changed, 7 insertions(+), 1 deletion(-)tools/bpf/bpftool/Documentation/common_options.rst +2
-1@@ -4,7 +4,8 @@ Print short help message (similar to **bpftool help**). -V, --version - Print version number (similar to **bpftool version**), and optional + Print bpftool's version number (similar to **bpftool version**), the + version of libbpf that was used to compile the binary, and optional features that were included when bpftool was compiled. Optional features include linking against libbfd to provide the disassembler for JIT-ted programs (**bpftool prog dump jited**) and usage of BPFtools/bpf/bpftool/Makefile +0
-0
tools/bpf/bpftool/Makefile LIBBPF_BOOTSTRAP_INTERNAL_HDRS := $(addprefix $(LIBBPF_BOOTSTRAP_HDRS_DIR)/,hash · line 46
Quoted 5 lines
tools/bpf/bpftool/Makefile +1
-0@@ -42,6 +42,7 @@ LIBBPF_BOOTSTRAP_INTERNAL_HDRS := $(addprefix $(LIBBPF_BOOTSTRAP_HDRS_DIR)/,hash ifeq ($(BPFTOOL_VERSION),) BPFTOOL_VERSION := $(shell make -rR --no-print-directory -sC ../../.. kernelversion) endif +LIBBPF_VERSION := $(shell make -r --no-print-directory -sC $(BPF_DIR) libbpfversion)
why can't you use libbpf_version_string() API instead?
Quoted 34 lines
tools/bpf/bpftool/Makefile +1
-0$(LIBBPF_OUTPUT) $(BOOTSTRAP_OUTPUT) $(LIBBPF_BOOTSTRAP_OUTPUT) $(LIBBPF_HDRS_DIR) $(LIBBPF_BOOTSTRAP_HDRS_DIR): $(QUIET_MKDIR)mkdir -p $@@@ -84,6 +85,7 @@ CFLAGS += -DPACKAGE='"bpftool"' -D__EXPORTED_HEADERS__ \ -I$(srctree)/tools/include \ -I$(srctree)/tools/include/uapi CFLAGS += -DBPFTOOL_VERSION='"$(BPFTOOL_VERSION)"' +CFLAGS += -DLIBBPF_VERSION='"$(LIBBPF_VERSION)"' ifneq ($(EXTRA_CFLAGS),) CFLAGS += $(EXTRA_CFLAGS) endiftools/bpf/bpftool/main.c +3
-1@@ -89,6 +89,8 @@ static int do_version(int argc, char **argv)jsonw_name (json_wtr,"version" );jsonw_printf (json_wtr,"\"%s\"" ,BPFTOOL_VERSION ); +jsonw_name (json_wtr,"libbpf_version" ); +jsonw_printf (json_wtr,"\"%s\"" ,LIBBPF_VERSION );jsonw_name (json_wtr,"features" );jsonw_start_object (json_wtr);/* features */ @@ -102,6 +104,7 @@ static int do_version(int argc, char **argv)unsigned int nb_features =0 ;printf ("%s v%s\n" , bin_name,BPFTOOL_VERSION ); +printf ("using libbpf v%s\n" ,LIBBPF_VERSION );printf ("features:" );if (has_libbfd) {printf (" libbfd" ); --2.32.0
Re: [PATCH bpf-next 2/3] bpftool: Add libbpf's version number to "bpftool version" output
2022-02-01 22:59 UTC-0800 ~ Andrii Nakryiko <[email protected]>
Quoted 31 lines · Quentin Monnet
Quoted 30 lines
To help users check what version of libbpf has been used to compile bpftool, embed the version number and print it along with bpftool's own version number.
Signed-off-by Quentin Monnet <[email protected]>--- tools/bpf/bpftool/Documentation/common_options.rst | 3 ++- tools/bpf/bpftool/Makefile | 2 ++ tools/bpf/bpftool/main.c | 3 +++ 3 files changed, 7 insertions(+), 1 deletion(-)tools/bpf/bpftool/Documentation/common_options.rst +2
-1@@ -4,7 +4,8 @@ Print short help message (similar to **bpftool help**). -V, --version - Print version number (similar to **bpftool version**), and optional + Print bpftool's version number (similar to **bpftool version**), the + version of libbpf that was used to compile the binary, and optional features that were included when bpftool was compiled. Optional features include linking against libbfd to provide the disassembler for JIT-ted programs (**bpftool prog dump jited**) and usage of BPFtools/bpf/bpftool/Makefile +0
-0
Quoted 8 lines
tools/bpf/bpftool/Makefile LIBBPF_BOOTSTRAP_INTERNAL_HDRS := $(addprefix $(LIBBPF_BOOTSTRAP_HDRS_DIR)/,hash · line 46
Quoted 6 lines
tools/bpf/bpftool/Makefile +1
-0@@ -42,6 +42,7 @@ LIBBPF_BOOTSTRAP_INTERNAL_HDRS := $(addprefix $(LIBBPF_BOOTSTRAP_HDRS_DIR)/,hash ifeq ($(BPFTOOL_VERSION),) BPFTOOL_VERSION := $(shell make -rR --no-print-directory -sC ../../.. kernelversion) endif +LIBBPF_VERSION := $(shell make -r --no-print-directory -sC $(BPF_DIR) libbpfversion)why can't you use
libbpf_version_string()API instead?
I missed it somehow, thanks for the pointer. It seems to be a recent addition to libbpf, and it was probably not present last time I checked for such an API. I'll use it.
PATCH 3/3 bpftool: Update versioning scheme
In reply to Quentin MonnetSince the notion of versions was introduced for bpftool, it has been following the version number of the kernel (using the version number corresponding to the tree in which bpftool's sources are located). The rationale was that bpftool's features are loosely tied to BPF features in the kernel, and that we could defer versioning to the kernel repository itself.
But this versioning scheme is confusing today, because a bpftool binary should be able to work with both older and newer kernels, even if some of its recent features won't be available on older systems. Furthermore, if bpftool is ported to other systems in the future, keeping a Linux-based version number is not a good option.
It would make more sense to align bpftool's number on libbpf, maybe. When versioning was introduced in bpftool, libbpf was in its initial phase at v0.0.1. Now it moves faster, with regular version bumps. But there are two issues if we want to pick the same numbers. First, that would mean going backward on the numbering, and will be a huge pain for every script trying to determine which bpftool binary is the most recent (not to mention some possible overlap of the numbers in a distant future). Then, bpftool could get new features or bug fixes between two versions libbpf, so maybe we should not completely tie its versions to libbpf, either.
Therefore, this commit introduces an independent versioning scheme for bpftool. The new version is v6.0.0, with its major number incremented over the current 5.16.* returned from the kernel's Makefile. The plan is to update this new number from time to time when bpftool gets new features or new bug fixes. These updates could possibly lead to new releases being tagged on the recently created out-of-tree mirror, at
https://github.com/libbpf/bpftool.
Version number is moved higher in the Makefile, to make it more visible.
--- tools/bpf/bpftool/Makefile | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)
tools/bpf/bpftool/Makefile +2 -3
@@ -1,6 +1,8 @@ # SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) include ../../scripts/Makefile.include +BPFTOOL_VERSION := 6.0.0 + ifeq ($(srctree),) srctree := $(patsubst %/,%,$(dir $(CURDIR))) srctree := $(patsubst %/,%,$(dir $(srctree)))
@@ -39,9 +41,6 @@ LIBBPF_BOOTSTRAP := $(LIBBPF_BOOTSTRAP_OUTPUT)libbpf.a LIBBPF_INTERNAL_HDRS := $(addprefix $(LIBBPF_HDRS_DIR)/,hashmap.h nlattr.h) LIBBPF_BOOTSTRAP_INTERNAL_HDRS := $(addprefix $(LIBBPF_BOOTSTRAP_HDRS_DIR)/,hashmap.h) -ifeq ($(BPFTOOL_VERSION),) -BPFTOOL_VERSION := $(shell make -rR --no-print-directory -sC ../../.. kernelversion) -endif LIBBPF_VERSION := $(shell make -r --no-print-directory -sC $(BPF_DIR) libbpfversion) $(LIBBPF_OUTPUT) $(BOOTSTRAP_OUTPUT) $(LIBBPF_BOOTSTRAP_OUTPUT) $(LIBBPF_HDRS_DIR) $(LIBBPF_BOOTSTRAP_HDRS_DIR):
Signature · 1 line
2.32.0
Re: [PATCH bpf-next 3/3] bpftool: Update versioning scheme
Quoted 46 lines · Quentin Monnet
Since the notion of versions was introduced for bpftool, it has been following the version number of the kernel (using the version number corresponding to the tree in which bpftool's sources are located). The rationale was that bpftool's features are loosely tied to BPF features in the kernel, and that we could defer versioning to the kernel repository itself.
But this versioning scheme is confusing today, because a bpftool binary should be able to work with both older and newer kernels, even if some of its recent features won't be available on older systems. Furthermore, if bpftool is ported to other systems in the future, keeping a Linux-based version number is not a good option.
It would make more sense to align bpftool's number on libbpf, maybe. When versioning was introduced in bpftool, libbpf was in its initial phase at v0.0.1. Now it moves faster, with regular version bumps. But there are two issues if we want to pick the same numbers. First, that would mean going backward on the numbering, and will be a huge pain for every script trying to determine which bpftool binary is the most recent (not to mention some possible overlap of the numbers in a distant future). Then, bpftool could get new features or bug fixes between two versions libbpf, so maybe we should not completely tie its versions to libbpf, either.
Therefore, this commit introduces an independent versioning scheme for bpftool. The new version is v6.0.0, with its major number incremented over the current 5.16.* returned from the kernel's
Makefile. The plan is to update this new number from time to time when bpftool gets new features or new bug fixes. These updates could possibly lead to new releases being tagged on the recently created out-of-tree mirror, at
https://github.com/libbpf/bpftool.Version number is moved higher in the
Makefile, to make it more visible.Suggested-by Andrii Nakryiko <[email protected]>Signed-off-by Quentin Monnet <[email protected]>--- tools/bpf/bpftool/Makefile | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)tools/bpf/bpftool/Makefile +0
-0
tools/bpf/bpftool/Makefile line 5
Quoted 5 lines
tools/bpf/bpftool/Makefile +2
-0@@ -1,6 +1,8 @@ # SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) include ../../scripts/Makefile.include +BPFTOOL_VERSION := 6.0.0 +
It's going to be a PITA to not forget to update this :( As discussed, I'm fine with this, but I also recalled the versioning approach that libbpf-sys library is using (see [0]). Maybe we could steal some of those ideas. As in, base bpftool version on libbpf (with major version
+ 6 as you do here), but also have "-1", "-2", etc suffixes for bpftool releases for when libbpf version didn't change. Don't know, just throwing out the idea for your consideration.
Quoted 16 lines
tools/bpf/bpftool/Makefile +0
-4ifeq ($(srctree),) srctree := $(patsubst %/,%,$(dir $(CURDIR))) srctree := $(patsubst %/,%,$(dir $(srctree)))@@ -39,9 +41,6 @@ LIBBPF_BOOTSTRAP := $(LIBBPF_BOOTSTRAP_OUTPUT)libbpf.a LIBBPF_INTERNAL_HDRS := $(addprefix $(LIBBPF_HDRS_DIR)/,hashmap.h nlattr.h) LIBBPF_BOOTSTRAP_INTERNAL_HDRS := $(addprefix $(LIBBPF_BOOTSTRAP_HDRS_DIR)/,hashmap.h) -ifeq ($(BPFTOOL_VERSION),) -BPFTOOL_VERSION := $(shell make -rR --no-print-directory -sC ../../.. kernelversion) -endif LIBBPF_VERSION := $(shell make -r --no-print-directory -sC $(BPF_DIR) libbpfversion) $(LIBBPF_OUTPUT) $(BOOTSTRAP_OUTPUT) $(LIBBPF_BOOTSTRAP_OUTPUT) $(LIBBPF_HDRS_DIR) $(LIBBPF_BOOTSTRAP_HDRS_DIR): --2.32.0
Re: [PATCH bpf-next 3/3] bpftool: Update versioning scheme
2022-02-01 22:59 UTC-0800 ~ Andrii Nakryiko <[email protected]>
Quoted 61 lines · Quentin Monnet
Quoted 46 lines
Since the notion of versions was introduced for bpftool, it has been following the version number of the kernel (using the version number corresponding to the tree in which bpftool's sources are located). The rationale was that bpftool's features are loosely tied to BPF features in the kernel, and that we could defer versioning to the kernel repository itself.
But this versioning scheme is confusing today, because a bpftool binary should be able to work with both older and newer kernels, even if some of its recent features won't be available on older systems. Furthermore, if bpftool is ported to other systems in the future, keeping a Linux-based version number is not a good option.
It would make more sense to align bpftool's number on libbpf, maybe. When versioning was introduced in bpftool, libbpf was in its initial phase at v0.0.1. Now it moves faster, with regular version bumps. But there are two issues if we want to pick the same numbers. First, that would mean going backward on the numbering, and will be a huge pain for every script trying to determine which bpftool binary is the most recent (not to mention some possible overlap of the numbers in a distant future). Then, bpftool could get new features or bug fixes between two versions libbpf, so maybe we should not completely tie its versions to libbpf, either.
Therefore, this commit introduces an independent versioning scheme for bpftool. The new version is v6.0.0, with its major number incremented over the current 5.16.* returned from the kernel's
Makefile. The plan is to update this new number from time to time when bpftool gets new features or new bug fixes. These updates could possibly lead to new releases being tagged on the recently created out-of-tree mirror, at
https://github.com/libbpf/bpftool.Version number is moved higher in the
Makefile, to make it more visible.Suggested-by Andrii Nakryiko <[email protected]>Signed-off-by Quentin Monnet <[email protected]>--- tools/bpf/bpftool/Makefile | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)tools/bpf/bpftool/Makefile +0
-0tools/bpf/bpftool/Makefile line 5
Quoted 5 lines
tools/bpf/bpftool/Makefile +2
-0@@ -1,6 +1,8 @@ # SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) include ../../scripts/Makefile.include +BPFTOOL_VERSION := 6.0.0 +It's going to be a PITA to not forget to update this :( As discussed, I'm fine with this, but I also recalled the versioning approach that libbpf-sys library is using (see [0]). Maybe we could steal some of those ideas. As in, base bpftool version on libbpf (with major version
+ 6 as you do here), but also have "-1", "-2", etc suffixes for bpftool releases for when libbpf version didn't change. Don't know, just throwing out the idea for your consideration.
Quoted 1 line
I've been somewhat torn between having a separate versioning scheme and getting as much flexibility as we want, and aligning on libbpf and having “automatic” version updates. My reasoning is the following.
If aligning on libbpf:
- We may want bpftool releases in-between two libbpf versions. Using pre-release numbers for tagging them is a good idea, although I don't know if something marked as a pre-release will look “official” enough (users may not pick it, thinking it's a beta release?). On the other hand, having bpftool with version numbers that look “official” haven't really been an issue so far.
- If no new feature lands in bpftool for some time, we may move from e.g. 6.7.0 to 6.8.0 when libbpf levels up and have two different versions which are in fact the same.
- Following libbpf's versioning scheme sounds better than kernel's, but ultimately it doesn't make too much sense either, because even though bpftool uses the lib a lot, its behaviour is not that much conditioned by the internal evolution of the library (or by new APIs that it may not use).
Having an independent versioning scheme solves the above, but as you say, it's gonna be painful to update the numbers. Developers will miss it most of the time, and I'm not even exactly sure myself of when to tag a new minor release. I suppose it would be a bit like for docs and completion, with occasional “catch-up” patches to update the version number - not great.
Based on all the above, I think your suggestion is good, and I'll switch back to aligning on libbpf's version. It may not be perfect, but 1) it's certainly an improvement over the current scheme, 2) the issues raised above are minor at the moment, and 3) we can still move to an independent scheme in the future if we realise we need it. Sounds more important to save on the maintenance burden at the moment. I'll send a new version shortly.
Thanks,
Quentin