Skip to contentSkip to conversation

bpftool: match programs and maps by names

PATCH In review Paul Chaignon 2 participants 6 messages Dec 13, 2019 – Dec 18, 2019

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

v2 cover bpftool: match programs and maps by names

When working with frequently modified BPF programs, both the ID and the tag may change. bpftool currently doesn't provide a "stable" way to match such programs. This patchset allows bpftool to match programs and maps by name.

When given a tag that matches several programs, bpftool currently only considers the first match. The first patch changes that behavior to either process all matching programs (for the show and dump commands) or error out. The second patch implements program lookup by name, with the same behavior as for tags in case of ambiguity. The last patch implements map lookup by name.

Changelogs:
Changes in v2:
- Fix buffer overflow after realloc.
- Add example output to commit message.
- Properly close JSON arrays on errors.
- Fix style errors (line breaks, for loops, exit labels, type for tagname).
- Move do_show code for argc == 2 to do_show_subset functions.
- Rebase.

Paul Chaignon (3):
bpftool: match several programs with same tag
bpftool: match programs by name
bpftool: match maps by name

 .../bpf/bpftool/Documentation/bpftool-map.rst |  12 +-
 .../bpftool/Documentation/bpftool-prog.rst    |  18 +-
 tools/bpf/bpftool/bash-completion/bpftool     | 145 ++++++-
 tools/bpf/bpftool/main.h                      |   4 +-
 tools/bpf/bpftool/map.c                       | 384 +++++++++++++----
 tools/bpf/bpftool/prog.c                      | 388 ++++++++++++------
 6 files changed, 737 insertions(+), 214 deletions(-)
Signature · 1 line
2.24.0
#1 lore raw

PATCH v2 1/3 bpftool: match several programs with same tag

When several BPF programs have the same tag, bpftool matches only the first (in ID order). This patch changes that behavior such that dump and show commands return all matched programs. Commands that require a single program (e.g., pin and attach) will error out if given a tag that matches several. bpftool prog dump will also error out if file or visual are given and several programs have the given tag.

In the case of the dump command, a program header is added before each dump only if the tag matches several programs; this patch doesn't change the output if a single program matches. The output when several programs match thus looks as follows.

$ ./bpftool prog dump xlated tag 6deef7357e7b4530
3: cgroup_skb  tag 6deef7357e7b4530  gpl
   0: (bf) r6 = r1
   [...]
   7: (95) exit

4: cgroup_skb tag 6deef7357e7b4530 gpl
0: (bf) r6 = r1
[...]
7: (95) exit

---
 .../bpftool/Documentation/bpftool-prog.rst    |  16 +-
 tools/bpf/bpftool/prog.c                      | 372 ++++++++++++------
 2 files changed, 268 insertions(+), 120 deletions(-)
tools/bpf/bpftool/Documentation/bpftool-prog.rst +12 -4
index 7a374b3c851d..d377d0cb7923 100644
@@ -53,8 +53,10 @@ DESCRIPTION
 ===========
 	**bpftool prog { show | list }** [*PROG*]
 		  Show information about loaded programs.  If *PROG* is
-		  specified show information only about given program, otherwise
-		  list all programs currently loaded on the system.
+		  specified show information only about given programs,
+		  otherwise list all programs currently loaded on the system.
+		  In case of **tag**, *PROG* may match several programs which
+		  will all be shown.
 
 		  Output will start with program ID followed by program type and
 		  zero or more named attributes (depending on kernel version).
@@ -68,11 +70,15 @@ DESCRIPTION
 		  performed via the **kernel.bpf_stats_enabled** sysctl knob.
 
 	**bpftool prog dump xlated** *PROG* [{ **file** *FILE* | **opcodes** | **visual** | **linum** }]
-		  Dump eBPF instructions of the program from the kernel. By
+		  Dump eBPF instructions of the programs from the kernel. By
 		  default, eBPF will be disassembled and printed to standard
 		  output in human-readable format. In this case, **opcodes**
 		  controls if raw opcodes should be printed as well.
 
+		  In case of **tag**, *PROG* may match several programs which
+		  will all be dumped.  However, if **file** or **visual** is
+		  specified, *PROG* must match a single program.
+
 		  If **file** is specified, the binary image will instead be
 		  written to *FILE*.
 
@@ -80,15 +86,17 @@ DESCRIPTION
 		  built instead, and eBPF instructions will be presented with
 		  CFG in DOT format, on standard output.
 
-		  If the prog has line_info available, the source line will
+		  If the programs have line_info available, the source line will
 		  be displayed by default.  If **linum** is specified,
 		  the filename, line number and line column will also be
 		  displayed on top of the source line.
 
 	**bpftool prog dump jited**  *PROG* [{ **file** *FILE* | **opcodes** | **linum** }]
 		  Dump jited image (host machine code) of the program.
+
 		  If *FILE* is specified image will be written to a file,
 		  otherwise it will be disassembled and printed to stdout.
+		  *PROG* must match a single program when **file** is specified.
 
 		  **opcodes** controls if raw opcodes will be printed.
 
tools/bpf/bpftool/prog.c +256 -116

535 lines not on this page · raw message

Signature · 1 line
2.24.0
#2 lore raw

PATCH v2 2/3 bpftool: match programs by name

In reply to Paul Chaignon

When working with frequently modified BPF programs, both the ID and the tag may change. bpftool currently doesn't provide a "stable" way to match such programs.

This patch implements lookup by name for programs. The show and dump commands will return all programs with the given name, whereas other commands will error out if several programs have the same name.

---
 .../bpf/bpftool/Documentation/bpftool-map.rst |  2 +-
 .../bpftool/Documentation/bpftool-prog.rst    | 12 +++++-----
 tools/bpf/bpftool/bash-completion/bpftool     | 22 ++++++++++++++-----
 tools/bpf/bpftool/main.h                      |  2 +-
 tools/bpf/bpftool/prog.c                      | 22 +++++++++++++++----
 5 files changed, 42 insertions(+), 18 deletions(-)
tools/bpf/bpftool/Documentation/bpftool-map.rst +1 -1
index 1c0f7146aab0..90d848b5b7d3 100644
@@ -41,7 +41,7 @@ MAP COMMANDS
 |
 |	*MAP* := { **id** *MAP_ID* | **pinned** *FILE* }
 |	*DATA* := { [**hex**] *BYTES* }
-|	*PROG* := { **id** *PROG_ID* | **pinned** *FILE* | **tag** *PROG_TAG* }
+|	*PROG* := { **id** *PROG_ID* | **pinned** *FILE* | **tag** *PROG_TAG* | **name** *PROG_NAME* }
 |	*VALUE* := { *DATA* | *MAP* | *PROG* }
 |	*UPDATE_FLAGS* := { **any** | **exist** | **noexist** }
 |	*TYPE* := { **hash** | **array** | **prog_array** | **perf_event_array** | **percpu_hash**
tools/bpf/bpftool/Documentation/bpftool-prog.rst +6 -6
index d377d0cb7923..64ddf8a4c518 100644
@@ -33,7 +33,7 @@ PROG COMMANDS
 |	**bpftool** **prog help**
 |
 |	*MAP* := { **id** *MAP_ID* | **pinned** *FILE* }
-|	*PROG* := { **id** *PROG_ID* | **pinned** *FILE* | **tag** *PROG_TAG* }
+|	*PROG* := { **id** *PROG_ID* | **pinned** *FILE* | **tag** *PROG_TAG* | **name** *PROG_NAME* }
 |	*TYPE* := {
 |		**socket** | **kprobe** | **kretprobe** | **classifier** | **action** |
 |		**tracepoint** | **raw_tracepoint** | **xdp** | **perf_event** | **cgroup/skb** |
@@ -55,8 +55,8 @@ DESCRIPTION
 		  Show information about loaded programs.  If *PROG* is
 		  specified show information only about given programs,
 		  otherwise list all programs currently loaded on the system.
-		  In case of **tag**, *PROG* may match several programs which
-		  will all be shown.
+		  In case of **tag** or **name**, *PROG* may match several
+		  programs which will all be shown.
 
 		  Output will start with program ID followed by program type and
 		  zero or more named attributes (depending on kernel version).
@@ -75,9 +75,9 @@ DESCRIPTION
 		  output in human-readable format. In this case, **opcodes**
 		  controls if raw opcodes should be printed as well.
 
-		  In case of **tag**, *PROG* may match several programs which
-		  will all be dumped.  However, if **file** or **visual** is
-		  specified, *PROG* must match a single program.
+		  In case of **tag** or **name**, *PROG* may match several
+		  programs which will all be dumped.  However, if **file** or
+		  **visual** is specified, *PROG* must match a single program.
 
 		  If **file** is specified, the binary image will instead be
 		  written to *FILE*.
tools/bpf/bpftool/bash-completion/bpftool +16 -6
index 70493a6da206..05b5be4a6ef9 100644
@@ -71,6 +71,12 @@ _bpftool_get_prog_tags()
         command sed -n 's/.*"tag": "\(.*\)",$/\1/p' )" -- "$cur" ) )
 }
 
+_bpftool_get_prog_names()
+{
+    COMPREPLY+=( $( compgen -W "$( bpftool -jp prog 2>&1 | \
+        command sed -n 's/.*"name": "\(.*\)",$/\1/p' )" -- "$cur" ) )
+}
+
 _bpftool_get_btf_ids()
 {
     COMPREPLY+=( $( compgen -W "$( bpftool -jp btf 2>&1 | \
@@ -201,6 +207,10 @@ _bpftool()
             _bpftool_get_prog_tags
             return 0
             ;;
+        name)
+            _bpftool_get_prog_names
+            return 0
+            ;;
         dev)
             _sysfs_get_netdevs
             return 0
@@ -263,7 +273,7 @@ _bpftool()
                     ;;
             esac
 
-            local PROG_TYPE='id pinned tag'
+            local PROG_TYPE='id pinned tag name'
             local MAP_TYPE='id pinned'
             case $command in
                 show|list)
@@ -559,7 +569,7 @@ _bpftool()
                                     return 0
                                     ;;
                                 prog_array)
-                                    local PROG_TYPE='id pinned tag'
+                                    local PROG_TYPE='id pinned tag name'
                                     COMPREPLY+=( $( compgen -W "$PROG_TYPE" \
                                         -- "$cur" ) )
                                     return 0
@@ -644,7 +654,7 @@ _bpftool()
             esac
             ;;
         btf)
-            local PROG_TYPE='id pinned tag'
+            local PROG_TYPE='id pinned tag name'
             local MAP_TYPE='id pinned'
             case $command in
                 dump)
@@ -735,7 +745,7 @@ _bpftool()
                         connect6 sendmsg4 sendmsg6 recvmsg4 recvmsg6 sysctl \
                         getsockopt setsockopt'
                     local ATTACH_FLAGS='multi override'
-                    local PROG_TYPE='id pinned tag'
+                    local PROG_TYPE='id pinned tag name'
                     case $prev in
                         $command)
                             _filedir
@@ -760,7 +770,7 @@ _bpftool()
                             elif [[ "$command" == "attach" ]]; then
                                 # We have an attach type on the command line,
                                 # but it is not the previous word, or
-                                # "id|pinned|tag" (we already checked for
+                                # "id|pinned|tag|name" (we already checked for
                                 # that). This should only leave the case when
                                 # we need attach flags for "attach" commamnd.
                                 _bpftool_one_of_list "$ATTACH_FLAGS"
@@ -786,7 +796,7 @@ _bpftool()
             esac
             ;;
         net)
-            local PROG_TYPE='id pinned tag'
+            local PROG_TYPE='id pinned tag name'
             local ATTACH_TYPES='xdp xdpgeneric xdpdrv xdpoffload'
             case $command in
                 show|list)
tools/bpf/bpftool/main.h +1 -1
index 2899095f8254..a7ead7bb9447 100644
@@ -42,7 +42,7 @@
 #define BPF_TAG_FMT	"%02hhx%02hhx%02hhx%02hhx%02hhx%02hhx%02hhx%02hhx"
 
 #define HELP_SPEC_PROGRAM						\
-	"PROG := { id PROG_ID | pinned FILE | tag PROG_TAG }"
+	"PROG := { id PROG_ID | pinned FILE | tag PROG_TAG | name PROG_NAME }"
 #define HELP_SPEC_OPTIONS						\
 	"OPTIONS := { {-j|--json} [{-p|--pretty}] | {-f|--bpffs} |\n"	\
 	"\t            {-m|--mapcompat} | {-n|--nomount} }"
tools/bpf/bpftool/prog.c +18 -4
index 37948c47aabd..2221bae037f1 100644
@@ -82,7 +82,7 @@ static void print_boot_time(__u64 nsecs, char *buf, unsigned int size)
 		strftime(buf, size, "%FT%T%z", &load_tm);
 }
 
-static int prog_fd_by_tag(unsigned char *tag, int **fds)
+static int prog_fd_by_nametag(void *nametag, int **fds, bool tag)
 {
 	unsigned int id = 0;
 	int fd, nb_fds = 0;
@@ -116,7 +116,8 @@ static int prog_fd_by_tag(unsigned char *tag, int **fds)
 			goto err_close_fd;
 		}
 
-		if (memcmp(tag, info.tag, BPF_TAG_SIZE)) {
+		if ((tag && memcmp(nametag, info.tag, BPF_TAG_SIZE)) ||
+		    (!tag && strncmp(nametag, info.name, BPF_OBJ_NAME_LEN))) {
 			close(fd);
 			continue;
 		}
@@ -174,7 +175,20 @@ static int prog_parse_fds(int *argc, char ***argv, int **fds)
 		}
 		NEXT_ARGP();
 
-		return prog_fd_by_tag(tag, fds);
+		return prog_fd_by_nametag(tag, fds, true);
+	} else if (is_prefix(**argv, "name")) {
+		char *name;
+
+		NEXT_ARGP();
+
+		name = **argv;
+		if (strlen(name) > BPF_OBJ_NAME_LEN - 1) {
+			p_err("can't parse name");
+			return -1;
+		}
+		NEXT_ARGP();
+
+		return prog_fd_by_nametag(name, fds, false);
 	} else if (is_prefix(**argv, "pinned")) {
 		char *path;
 
@@ -189,7 +203,7 @@ static int prog_parse_fds(int *argc, char ***argv, int **fds)
 		return 1;
 	}
 
-	p_err("expected 'id', 'tag' or 'pinned', got: '%s'?", **argv);
+	p_err("expected 'id', 'tag', 'name' or 'pinned', got: '%s'?", **argv);
 	return -1;
 }
 
Signature · 1 line
2.24.0
#3 lore raw

PATCH v2 3/3 bpftool: match maps by name

In reply to Paul Chaignon

This patch implements lookup by name for maps and changes the behavior of lookups by tag to be consistent with prog subcommands. Similarly to program subcommands, the show and dump commands will return all maps with the given name (or tag), whereas other commands will error out if several maps have the same name (resp. tag).

When a map has BTF info, it is dumped in JSON with available BTF info. This patch requires that all matched maps have BTF info before switching the output format to JSON.

---
 .../bpf/bpftool/Documentation/bpftool-map.rst |  10 +-
 tools/bpf/bpftool/bash-completion/bpftool     | 131 +++++-
 tools/bpf/bpftool/main.h                      |   2 +-
 tools/bpf/bpftool/map.c                       | 384 ++++++++++++++----
 4 files changed, 439 insertions(+), 88 deletions(-)
tools/bpf/bpftool/Documentation/bpftool-map.rst +6 -4
index 90d848b5b7d3..cdeae8ae90ba 100644
@@ -39,7 +39,7 @@ MAP COMMANDS
 |	**bpftool** **map freeze**     *MAP*
 |	**bpftool** **map help**
 |
-|	*MAP* := { **id** *MAP_ID* | **pinned** *FILE* }
+|	*MAP* := { **id** *MAP_ID* | **pinned** *FILE* | **name** *MAP_NAME* }
 |	*DATA* := { [**hex**] *BYTES* }
 |	*PROG* := { **id** *PROG_ID* | **pinned** *FILE* | **tag** *PROG_TAG* | **name** *PROG_NAME* }
 |	*VALUE* := { *DATA* | *MAP* | *PROG* }
@@ -55,8 +55,9 @@ DESCRIPTION
 ===========
 	**bpftool map { show | list }**   [*MAP*]
 		  Show information about loaded maps.  If *MAP* is specified
-		  show information only about given map, otherwise list all
-		  maps currently loaded on the system.
+		  show information only about given maps, otherwise list all
+		  maps currently loaded on the system.  In case of **name**,
+		  *MAP* may match several maps which will all be shown.
 
 		  Output will start with map ID followed by map type and
 		  zero or more named attributes (depending on kernel version).
@@ -66,7 +67,8 @@ DESCRIPTION
 		  as *FILE*.
 
 	**bpftool map dump**    *MAP*
-		  Dump all entries in a given *MAP*.
+		  Dump all entries in a given *MAP*.  In case of **name**,
+		  *MAP* may match several maps which will all be dumped.
 
 	**bpftool map update**  *MAP* [**key** *DATA*] [**value** *VALUE*] [*UPDATE_FLAGS*]
 		  Update map entry for a given *KEY*.
tools/bpf/bpftool/bash-completion/bpftool +122 -9

232 lines not on this page · raw message

tools/bpf/bpftool/main.h +1 -1
index a7ead7bb9447..81890f4c8cca 100644
@@ -47,7 +47,7 @@
 	"OPTIONS := { {-j|--json} [{-p|--pretty}] | {-f|--bpffs} |\n"	\
 	"\t            {-m|--mapcompat} | {-n|--nomount} }"
 #define HELP_SPEC_MAP							\
-	"MAP := { id MAP_ID | pinned FILE }"
+	"MAP := { id MAP_ID | pinned FILE | name MAP_NAME }"
 
 static const char * const prog_type_name[] = {
 	[BPF_PROG_TYPE_UNSPEC]			= "unspec",
tools/bpf/bpftool/map.c +310 -74

486 lines not on this page · raw message

Signature · 1 line
2.24.0
#4 lore raw
In reply to Paul Chaignon
Quoted 12 lines · Paul Chaignon

When working with frequently modified BPF programs, both the ID and the tag may change. bpftool currently doesn't provide a "stable" way to match such programs. This patchset allows bpftool to match programs and maps by name.

When given a tag that matches several programs, bpftool currently only considers the first match. The first patch changes that behavior to either process all matching programs (for the show and dump commands) or error out. The second patch implements program lookup by name, with the same behavior as for tags in case of ambiguity. The last patch implements map lookup by name.

Quoted 9 lines

Changelogs:
Changes in v2:
- Fix buffer overflow after realloc.
- Add example output to commit message.
- Properly close JSON arrays on errors.
- Fix style errors (line breaks, for loops, exit labels, type for tagname).
- Move do_show code for argc == 2 to do_show_subset functions.
- Rebase.

Loogs good. Applied.

I found the exact match logic unintuitive though.
Since 'prog show' can print multiple may be allow partial match on name? So 'bpftool p s name tracepoint__' would print all BCC-based programs that attach to tracepoints.
It would be roughly equivalent to 'bpftool p s |grep tracepoint__', but grep captures single line.
There is 'bpftool perf|grep tracepoint' as well, but since the tool matches on name it probably should match partial name too.

#5 lore raw
Quoted 23 lines · Alexei Starovoitov
Quoted 12 lines · Paul Chaignon

When working with frequently modified BPF programs, both the ID and the tag may change. bpftool currently doesn't provide a "stable" way to match such programs. This patchset allows bpftool to match programs and maps by name.

When given a tag that matches several programs, bpftool currently only considers the first match. The first patch changes that behavior to either process all matching programs (for the show and dump commands) or error out. The second patch implements program lookup by name, with the same behavior as for tags in case of ambiguity. The last patch implements map lookup by name.

Quoted 9 lines

Changelogs:
Changes in v2:
- Fix buffer overflow after realloc.
- Add example output to commit message.
- Properly close JSON arrays on errors.
- Fix style errors (line breaks, for loops, exit labels, type for tagname).
- Move do_show code for argc == 2 to do_show_subset functions.
- Rebase.

Quoted 8 lines

Loogs good. Applied.

I found the exact match logic unintuitive though.
Since 'prog show' can print multiple may be allow partial match on name? So 'bpftool p s name tracepoint__' would print all BCC-based programs that attach to tracepoints.
It would be roughly equivalent to 'bpftool p s |grep tracepoint__', but grep captures single line.

I had a look at bcc and it actually removes these prefixes from the program's name (except for kretprobes, but it looks like an oversight).

I still agree a partial match would be good to have since related programs are more likely to share prefixes (e.g., func_entry and func_exit). Maybe matching on prefixes would be enough though? I can't think of use cases for a true partial match and the behavior might be a bit unexpected for users...

Or we go all the way and implement support for * (e.g., tcp_* for all bcc kprobes on tcp functions; *tcp* for all programs containing 'tcp').

Quoted 3 lines

There is 'bpftool perf|grep tracepoint' as well, but since the tool matches on name it probably should match partial name too.