Skip to contentSkip to conversation

tools/bpf: bpftool: add net support

RFC PATCH In review Yonghong Song 2 participants 7 messages Sep 3, 2018 – Sep 5, 2018

Subsystems (5)
lore
Share & tools
lore
#0 lore raw

cover tools/bpf: bpftool: add net support

The functionality to dump network driver and tc related bpf programs are added. Currently, users can already use "ip link show <dev>" and "tc filter show dev <dev> ..." to dump bpf program attachment information for xdp programs and tc bpf programs.
The implementation here allows bpftool as a central place for bpf introspection and users do not need to revert to other tools. Also, we can make command simpler to dump bpf related information, e.g., "bpftool net" is able to dump all xdp and tc bpf programs.

For example,

  $ bpftool net
  xdp [
  ]
  netdev_filters [
  ifindex 2 name handle_icmp flags direct-action flags_gen [not_in_hw ]
            prog_id 3194 tag 846d29c14d0d7d26 act []
  ifindex 2 name handle_egress flags direct-action flags_gen [not_in_hw ]
            prog_id 3193 tag 387d281be9fe77aa
  ]     
  $ bpftool -jp net
  [{    
        "xdp": [],
        "netdev_filters": [{
                "ifindex": 2,
                "name": "handle_icmp",
                "flags": "direct-action",
                "flags_gen": ["not_in_hw"
                ],
                "prog_id": 3194,
                "tag": "846d29c14d0d7d26",
                "act": []
            },{
                "ifindex": 2,
                "name": "handle_egress",
                "flags": "direct-action",
                "flags_gen": ["not_in_hw"
                ],
                "prog_id": 3193,
                "tag": "387d281be9fe77aa"
            }
        ]
    }
  ]

This is labeled as RFC as
. the current command line as "bpftool net {show|list} [dev name]",
does this sound reasonable? When "dev name" is specified, only bpf programs for that particular device are displayed.
. for some netlink attributes, currently I only print out the
raw numbers, maybe I should print better similar to iproute2?
. bpftool documentation and bash completion are not implemented.

Patch #1 sync'd kernel uapi header if_link.h to tools directory. Patch #2 re-organized the bpf/lib bpf.c to have a separate file for netlink related functions.
Patch #3 added additional netlink related functions. Patch #4 implemented "bpftool net" command.

Yonghong Song (4):
tools/bpf: sync kernel uapi header if_link.h to tools tools/bpf: move bpf/lib netlink related functions into a new file tools/bpf: add more netlink functionalities in lib/bpf tools/bpf: bpftool: add net support

 tools/bpf/bpftool/main.c           |   3 +-
 tools/bpf/bpftool/main.h           |   7 +
 tools/bpf/bpftool/net.c            | 219 +++++++++++++++++++
 tools/bpf/bpftool/netlink_dumper.c | 261 +++++++++++++++++++++++
 tools/bpf/bpftool/netlink_dumper.h | 103 +++++++++
 tools/include/uapi/linux/if_link.h |  17 ++
 tools/lib/bpf/Build                |   2 +-
 tools/lib/bpf/bpf.c                | 129 ------------
 tools/lib/bpf/libbpf.h             |  16 ++
 tools/lib/bpf/libbpf_errno.c       |   1 +
 tools/lib/bpf/netlink.c            | 323 +++++++++++++++++++++++++++++
 tools/lib/bpf/nlattr.c             |  33 +--
 tools/lib/bpf/nlattr.h             |  38 ++++
 13 files changed, 1009 insertions(+), 143 deletions(-)
 create mode 100644 tools/bpf/bpftool/net.c
 create mode 100644 tools/bpf/bpftool/netlink_dumper.c
 create mode 100644 tools/bpf/bpftool/netlink_dumper.h
 create mode 100644 tools/lib/bpf/netlink.c
Signature · 1 line
2.17.1
#2 lore raw

RFC PATCH 2/4 tools/bpf: move bpf/lib netlink related functions into a new file

There are no functionality change for this patch.

In the subsequent patches, more netlink related library functions will be added and a separate file is better than cluttering bpf.c.

---
 tools/lib/bpf/Build     |   2 +-
 tools/lib/bpf/bpf.c     | 129 -------------------------------
 tools/lib/bpf/netlink.c | 164 ++++++++++++++++++++++++++++++++++++++++
 3 files changed, 165 insertions(+), 130 deletions(-)
 create mode 100644 tools/lib/bpf/netlink.c
tools/lib/bpf/Build +1 -1
index 13a861135127..512b2c0ba0d2 100644
@@ -1 +1 @@
-libbpf-y := libbpf.o bpf.o nlattr.o btf.o libbpf_errno.o
+libbpf-y := libbpf.o bpf.o nlattr.o btf.o libbpf_errno.o netlink.o
tools/lib/bpf/bpf.c +0 -129

143 lines not on this page · raw message

tools/lib/bpf/netlink.c new file +164 -0

164 lines not on this page · raw message

Signature · 1 line
2.17.1
#1 lore raw

RFC PATCH 1/4 tools/bpf: sync kernel uapi header if_link.h to tools

In reply to Yonghong Song

Among others, this header will be used later for
bpftool net support.

---
 tools/include/uapi/linux/if_link.h | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
tools/include/uapi/linux/if_link.h +17 -0
index cf01b6824244..43391e2d1153 100644
@@ -164,6 +164,8 @@ enum {
 	IFLA_CARRIER_UP_COUNT,
 	IFLA_CARRIER_DOWN_COUNT,
 	IFLA_NEW_IFINDEX,
+	IFLA_MIN_MTU,
+	IFLA_MAX_MTU,
 	__IFLA_MAX
 };
 
@@ -334,6 +336,7 @@ enum {
 	IFLA_BRPORT_GROUP_FWD_MASK,
 	IFLA_BRPORT_NEIGH_SUPPRESS,
 	IFLA_BRPORT_ISOLATED,
+	IFLA_BRPORT_BACKUP_PORT,
 	__IFLA_BRPORT_MAX
 };
 #define IFLA_BRPORT_MAX (__IFLA_BRPORT_MAX - 1)
@@ -459,6 +462,16 @@ enum {
 
 #define IFLA_MACSEC_MAX (__IFLA_MACSEC_MAX - 1)
 
+/* XFRM section */
+enum {
+	IFLA_XFRM_UNSPEC,
+	IFLA_XFRM_LINK,
+	IFLA_XFRM_IF_ID,
+	__IFLA_XFRM_MAX
+};
+
+#define IFLA_XFRM_MAX (__IFLA_XFRM_MAX - 1)
+
 enum macsec_validation_type {
 	MACSEC_VALIDATE_DISABLED = 0,
 	MACSEC_VALIDATE_CHECK = 1,
@@ -920,6 +933,7 @@ enum {
 	XDP_ATTACHED_DRV,
 	XDP_ATTACHED_SKB,
 	XDP_ATTACHED_HW,
+	XDP_ATTACHED_MULTI,
 };
 
 enum {
@@ -928,6 +942,9 @@ enum {
 	IFLA_XDP_ATTACHED,
 	IFLA_XDP_FLAGS,
 	IFLA_XDP_PROG_ID,
+	IFLA_XDP_DRV_PROG_ID,
+	IFLA_XDP_SKB_PROG_ID,
+	IFLA_XDP_HW_PROG_ID,
 	__IFLA_XDP_MAX,
 };
 
Signature · 1 line
2.17.1
#3 lore raw

RFC PATCH 3/4 tools/bpf: add more netlink functionalities in lib/bpf

In reply to Yonghong Song

This patch added a few netlink attribute parsing functions and the netlink API functions to query networking links, tc classes, tc qdiscs and tc filters. For example, the following API is to get networking links:
int nl_get_link(int sock, unsigned int nl_pid,

                  dump_nlmsg_t dump_link_nlmsg,
                  void *cookie);

Note that when the API is called, the user also provided a callback function with the following signature:
int (*dump_nlmsg_t)(void *cookie, void *msg, struct nlattr **tb);

The "cookie" is the parameter the user passed to the API and will be available for the callback function.
The "msg" is the information about the result, e.g., ifinfomsg or tcmsg. The "tb" is the parsed netlink attributes.

---
 tools/lib/bpf/libbpf.h       |  16 ++++
 tools/lib/bpf/libbpf_errno.c |   1 +
 tools/lib/bpf/netlink.c      | 165 ++++++++++++++++++++++++++++++++++-
 tools/lib/bpf/nlattr.c       |  33 ++++---
 tools/lib/bpf/nlattr.h       |  38 ++++++++
 5 files changed, 238 insertions(+), 15 deletions(-)
tools/lib/bpf/libbpf.h +16 -0
index 96c55fac54c3..e3b00e23e181 100644
@@ -46,6 +46,7 @@ enum libbpf_errno {
 	LIBBPF_ERRNO__PROGTYPE,	/* Kernel doesn't support this program type */
 	LIBBPF_ERRNO__WRNGPID,	/* Wrong pid in netlink message */
 	LIBBPF_ERRNO__INVSEQ,	/* Invalid netlink sequence */
+	LIBBPF_ERRNO__NLPARSE,	/* netlink parsing error */
 	__LIBBPF_ERRNO__END,
 };
 
@@ -297,4 +298,19 @@ int bpf_perf_event_read_simple(void *mem, unsigned long size,
 			       unsigned long page_size,
 			       void **buf, size_t *buf_len,
 			       bpf_perf_event_print_t fn, void *priv);
+
+struct nlmsghdr;
+struct nlattr;
+typedef int (*dump_nlmsg_t)(void *cookie, void *msg, struct nlattr **tb);
+typedef int (*__dump_nlmsg_t)(struct nlmsghdr *nlmsg, dump_nlmsg_t,
+			      void *cookie);
+int bpf_netlink_open(unsigned int *nl_pid);
+int nl_get_link(int sock, unsigned int nl_pid, dump_nlmsg_t dump_link_nlmsg,
+		void *cookie);
+int nl_get_class(int sock, unsigned int nl_pid, int ifindex,
+		 dump_nlmsg_t dump_class_nlmsg, void *cookie);
+int nl_get_qdisc(int sock, unsigned int nl_pid, int ifindex,
+		 dump_nlmsg_t dump_qdisc_nlmsg, void *cookie);
+int nl_get_filter(int sock, unsigned int nl_pid, int ifindex, int handle,
+		  dump_nlmsg_t dump_filter_nlmsg, void *cookie);
 #endif
tools/lib/bpf/libbpf_errno.c +1 -0
index d9ba851bd7f9..2464ade3b326 100644
@@ -42,6 +42,7 @@ static const char *libbpf_strerror_table[NR_ERRNO] = {
 	[ERRCODE_OFFSET(PROGTYPE)]	= "Kernel doesn't support this program type",
 	[ERRCODE_OFFSET(WRNGPID)]	= "Wrong pid in netlink message",
 	[ERRCODE_OFFSET(INVSEQ)]	= "Invalid netlink sequence",
+	[ERRCODE_OFFSET(NLPARSE)]	= "Incorrect netlink message parsing",
 };
 
 int libbpf_strerror(int err, char *buf, size_t size)
tools/lib/bpf/netlink.c +162 -3

191 lines not on this page · raw message

tools/lib/bpf/nlattr.c +21 -12
index 4719434278b2..49f514119bdb 100644
@@ -26,11 +26,6 @@ static uint16_t nla_attr_minlen[NLA_TYPE_MAX+1] = {
 	[NLA_FLAG]	= 0,
 };
 
-static int nla_len(const struct nlattr *nla)
-{
-	return nla->nla_len - NLA_HDRLEN;
-}
-
 static struct nlattr *nla_next(const struct nlattr *nla, int *remaining)
 {
 	int totlen = NLA_ALIGN(nla->nla_len);
@@ -46,11 +41,6 @@ static int nla_ok(const struct nlattr *nla, int remaining)
 	       nla->nla_len <= remaining;
 }
 
-static void *nla_data(const struct nlattr *nla)
-{
-	return (char *) nla + NLA_HDRLEN;
-}
-
 static int nla_type(const struct nlattr *nla)
 {
 	return nla->nla_type & NLA_TYPE_MASK;
@@ -114,8 +104,8 @@ static inline int nlmsg_len(const struct nlmsghdr *nlh)
  * @see nla_validate
  * @return 0 on success or a negative error code.
  */
-static int nla_parse(struct nlattr *tb[], int maxtype, struct nlattr *head, int len,
-		     struct nla_policy *policy)
+int nla_parse(struct nlattr *tb[], int maxtype, struct nlattr *head, int len,
+	      struct nla_policy *policy)
 {
 	struct nlattr *nla;
 	int rem, err;
@@ -146,6 +136,25 @@ static int nla_parse(struct nlattr *tb[], int maxtype, struct nlattr *head, int
 	return err;
 }
 
+/**
+ * Create attribute index based on nested attribute
+ * @arg tb              Index array to be filled (maxtype+1 elements).
+ * @arg maxtype         Maximum attribute type expected and accepted.
+ * @arg nla             Nested Attribute.
+ * @arg policy          Attribute validation policy.
+ *
+ * Feeds the stream of attributes nested into the specified attribute
+ * to nla_parse().
+ *
+ * @see nla_parse
+ * @return 0 on success or a negative error code.
+ */
+int nla_parse_nested(struct nlattr *tb[], int maxtype, struct nlattr *nla,
+		     struct nla_policy *policy)
+{
+	return nla_parse(tb, maxtype, nla_data(nla), nla_len(nla), policy);
+}
+
 /* dump netlink extended ack error message */
 int nla_dump_errormsg(struct nlmsghdr *nlh)
 {
tools/lib/bpf/nlattr.h +38 -0
index 931a71f68f93..a6e2396bce7c 100644
@@ -67,6 +67,44 @@ struct nla_policy {
 	     nla_ok(pos, rem); \
 	     pos = nla_next(pos, &(rem)))
 
+/**
+ * nla_data - head of payload
+ * @nla: netlink attribute
+ */
+static inline void *nla_data(const struct nlattr *nla)
+{
+	return (char *) nla + NLA_HDRLEN;
+}
+
+static inline uint8_t nla_getattr_u8(const struct nlattr *nla)
+{
+	return *(uint8_t *)nla_data(nla);
+}
+
+static inline uint32_t nla_getattr_u32(const struct nlattr *nla)
+{
+	return *(uint32_t *)nla_data(nla);
+}
+
+static inline const char *nla_getattr_str(const struct nlattr *nla)
+{
+	return (const char *)nla_data(nla);
+}
+
+/**
+ * nla_len - length of payload
+ * @nla: netlink attribute
+ */
+static inline int nla_len(const struct nlattr *nla)
+{
+	return nla->nla_len - NLA_HDRLEN;
+}
+
+int nla_parse(struct nlattr *tb[], int maxtype, struct nlattr *head, int len,
+	      struct nla_policy *policy);
+int nla_parse_nested(struct nlattr *tb[], int maxtype, struct nlattr *nla,
+		     struct nla_policy *policy);
+
 int nla_dump_errormsg(struct nlmsghdr *nlh);
 
 #endif /* __NLATTR_H */
Signature · 1 line
2.17.1
#4 lore raw

RFC PATCH 4/4 tools/bpf: bpftool: add net support

In reply to Yonghong Song

Add "bpftool net" support. Networking devices are enumerated to dump all xdp information. Also, for each networking device, tc classes and qdiscs are enumerated in order to check bpf filters with these classes and qdiscs. In addition, root handle and clsact ingress/egress are also checked for bpf filters.

For example,

  $ bpftool net
  xdp [
  ]
  netdev_filters [
  ifindex 2 name handle_icmp flags direct-action flags_gen [not_in_hw ]
            prog_id 3194 tag 846d29c14d0d7d26 act []
  ifindex 2 name handle_egress flags direct-action flags_gen [not_in_hw ]
            prog_id 3193 tag 387d281be9fe77aa
  ]
  $ bpftool -jp net
  [{
        "xdp": [],
        "netdev_filters": [{
                "ifindex": 2,
                "name": "handle_icmp",
                "flags": "direct-action",
                "flags_gen": ["not_in_hw"
                ],
                "prog_id": 3194,
                "tag": "846d29c14d0d7d26",
                "act": []
            },{
                "ifindex": 2,
                "name": "handle_egress",
                "flags": "direct-action",
                "flags_gen": ["not_in_hw"
                ],
                "prog_id": 3193,
                "tag": "387d281be9fe77aa"
            }
        ]
    }
  ]
---
 tools/bpf/bpftool/main.c           |   3 +-
 tools/bpf/bpftool/main.h           |   7 +
 tools/bpf/bpftool/net.c            | 219 ++++++++++++++++++++++++
 tools/bpf/bpftool/netlink_dumper.c | 261 +++++++++++++++++++++++++++++
 tools/bpf/bpftool/netlink_dumper.h | 103 ++++++++++++
 5 files changed, 592 insertions(+), 1 deletion(-)
 create mode 100644 tools/bpf/bpftool/net.c
 create mode 100644 tools/bpf/bpftool/netlink_dumper.c
 create mode 100644 tools/bpf/bpftool/netlink_dumper.h
tools/bpf/bpftool/main.c +2 -1
index d15a62be6cf0..79dc3f193547 100644
@@ -85,7 +85,7 @@ static int do_help(int argc, char **argv)
 		"       %s batch file FILE\n"
 		"       %s version\n"
 		"\n"
-		"       OBJECT := { prog | map | cgroup | perf }\n"
+		"       OBJECT := { prog | map | cgroup | perf | net }\n"
 		"       " HELP_SPEC_OPTIONS "\n"
 		"",
 		bin_name, bin_name, bin_name);
@@ -215,6 +215,7 @@ static const struct cmd cmds[] = {
 	{ "map",	do_map },
 	{ "cgroup",	do_cgroup },
 	{ "perf",	do_perf },
+	{ "net",	do_net },
 	{ "version",	do_version },
 	{ 0 }
 };
tools/bpf/bpftool/main.h +7 -0
index 238e734d75b3..f82aeb08a043 100644
@@ -136,6 +136,7 @@ int do_map(int argc, char **arg);
 int do_event_pipe(int argc, char **argv);
 int do_cgroup(int argc, char **arg);
 int do_perf(int argc, char **arg);
+int do_net(int argc, char **arg);
 
 int prog_parse_fd(int *argc, char ***argv);
 int map_parse_fd(int *argc, char ***argv);
@@ -165,4 +166,10 @@ struct btf_dumper {
  */
 int btf_dumper_type(const struct btf_dumper *d, __u32 type_id,
 		    const void *data);
+
+struct nlattr;
+struct ifinfomsg;
+struct tcmsg;
+int do_filter_dump(struct tcmsg *ifinfo, struct nlattr **tb);
+int do_xdp_dump(struct ifinfomsg *ifinfo, struct nlattr **tb);
 #endif
tools/bpf/bpftool/net.c new file +219 -0

219 lines not on this page · raw message

tools/bpf/bpftool/netlink_dumper.c new file +261 -0

261 lines not on this page · raw message

tools/bpf/bpftool/netlink_dumper.h new file +103 -0

103 lines not on this page · raw message

Signature · 1 line
2.17.1
#5 lore raw
In reply to Yonghong Song
Quoted 8 lines · Yonghong Song

The functionality to dump network driver and tc related bpf programs are added. Currently, users can already use "ip link show <dev>" and "tc filter show dev <dev> ..." to dump bpf program attachment information for xdp programs and tc bpf programs.
The implementation here allows bpftool as a central place for bpf introspection and users do not need to revert to other tools. Also, we can make command simpler to dump bpf related information, e.g., "bpftool net" is able to dump all xdp and tc bpf programs.

Why not implement this best-effort, unreliable (name spaces) additional output the same way we added bpffs support, make it a flag to existing list commands?

My knee jerk reaction is that this is duplication of work. iproute2 can show us the filters and xdp programs very easily. Will we add programs attached to sockets as well? And lwtunnels? bpfilter?

Would you be able to give us a convincing user scenario? What kind of information is the user looking for? Are there going to be other sub-commands to the 'net' object?

Quoted 7 lines

For example,

  $ bpftool net
  xdp [
  ]
  netdev_filters [
  ifindex 2 name handle_icmp flags direct-action flags_gen [not_in_hw ]

How do you handle shared blocks here? Does the user really care about the flags? What about ordering of filters?

Quoted 4 lines
            prog_id 3194 tag 846d29c14d0d7d26 act []
  ifindex 2 name handle_egress flags direct-action flags_gen [not_in_hw ]
            prog_id 3193 tag 387d281be9fe77aa

]

#6 lore raw
Quoted 10 lines · Jakub Kicinski
Quoted 8 lines · Yonghong Song

The functionality to dump network driver and tc related bpf programs are added. Currently, users can already use "ip link show <dev>" and "tc filter show dev <dev> ..." to dump bpf program attachment information for xdp programs and tc bpf programs.
The implementation here allows bpftool as a central place for bpf introspection and users do not need to revert to other tools. Also, we can make command simpler to dump bpf related information, e.g., "bpftool net" is able to dump all xdp and tc bpf programs.

Quoted 3 lines

Why not implement this best-effort, unreliable (name spaces) additional output the same way we added bpffs support, make it a flag to existing list commands?

Do you mean to implement something like "bpftool -n prog" to show the
attachments for net-related bpf programs? I feel a separate command "net" is better since it intends to show the context of the bpf program as the same program may be installed in different places. This is similar to other commands like "cgroup" and "perf".

Quoted 4 lines

My knee jerk reaction is that this is duplication of work. iproute2 can show us the filters and xdp programs very easily. Will we add programs attached to sockets as well? And lwtunnels? bpfilter?

This has been discussed in iovisor meeting, but let me summarize here. I understand that iproute2 ip/tc can do exactly what I implemented here. The implementation here is mostly from user experience point of view.

People worried about bpf performance/memory cost in the data center. So they often ask what bpf programs are running in any host and what is the context (attachment point) of all bpf programs? Most these engineers
are not networking/bpf/kernel engineers. So yes, we will need
to add programs attached to sockets, lwtunnels and bpfilters etc. later.
This may be a lower priority for me now since FB does not use them yet.

Currently, we already use bpftool do prog/map perf/cgroup dumps. Extending bpftool is easier for user than using a different command as not everybody is very familiar to esp. tc.

Quoted 4 lines

Would you be able to give us a convincing user scenario? What kind of information is the user looking for? Are there going to be other sub-commands to the 'net' object?

As of now, it just dumped bpf related information with prog_id (plus other attachment specific information) so users can correlate back to the program itself. No plan to add other sub-commands (except "dev <name>") at this point.

Quoted 11 lines
Quoted 8 lines

For example,

   $ bpftool net
   xdp [
   ]
   netdev_filters [
   ifindex 2 name handle_icmp flags direct-action flags_gen [not_in_hw ]

How do you handle shared blocks here? Does the user really care about the flags? What about ordering of filters?

shared block is not handled here. This can be added later.
yes, I can remove flags_gen. For "flags", may or may not. Will go through dumped info and remove those not really needed.

The order of filters will be based on ifindex first and inside ifindex, attached to class first, attached to qdisc second, attached to root/clsact last.

Quoted 5 lines
Quoted 5 lines
             prog_id 3194 tag 846d29c14d0d7d26 act []
   ifindex 2 name handle_egress flags direct-action flags_gen [not_in_hw ]
             prog_id 3193 tag 387d281be9fe77aa

]