Skip to contentSkip to conversation

bpf: sockmap fixes

PATCH In review John Fastabend 2 participants 6 messages Feb 5, 2018 – Feb 6, 2018

Subsystems (5)
Networking Network drivers BPF TCP +1 more subsystems
lore
Share & tools
lore

John Fastabend

author maintainer
#0 lore raw

v4 cover bpf: sockmap fixes

A set of fixes for sockmap to resolve programs referencing sockmaps and closing without deleting all entries in the map and/or not detaching BPF programs attached to the map. Both leaving entries in the map and not detaching programs may result in the map failing to be removed by BPF infrastructure due to reference counts never reaching zero.

For this we pull in the ULP infrastructure to hook into the close() hook of the sock layer. This seemed natural because we have additional sockmap features (to add support for TX hooks) that will also use the ULP infrastructure. This allows us to cleanup entries in the map when socks are closed() and avoid trying to get the sk_state_change() hook to fire in all cases.

The second issue resolved here occurs when users don't detach programs. The gist is a refcnt issue resolved by implementing the release callback. See patch for details.

For testing I ran both sample/sockmap and selftests bpf/test_maps.c. Dave Watson ran TLS test suite on v1 version of the patches without the put_module error path change.

v4 fix missing rcu_unlock()
v3 wrap psock reference in RCU
v2 changes rebased onto bpf-next with small update adding module_put

---

John Fastabend (3):
net: add a UID to use for ULP socket assignment
bpf: sockmap, add sock close() hook to remove socks
bpf: sockmap, fix leaking maps with attached but not detached progs

 include/net/tcp.h    |    8 ++
 kernel/bpf/sockmap.c |  187 +++++++++++++++++++++++++++++++-------------------
 net/ipv4/tcp_ulp.c   |   59 ++++++++++++++--
 net/tls/tls_main.c   |    2 +
 4 files changed, 179 insertions(+), 77 deletions(-)
author maintainer
#1 lore raw

PATCH v4 1/3 net: add a UID to use for ULP socket assignment

Create a UID field and enum that can be used to assign ULPs to sockets. This saves a set of string comparisons if the ULP id is known.

For sockmap, which is added in the next patches, a ULP is used to hook into TCP sockets close state. In this case the ULP being added is done at map insert time and the ULP is known and done on the kernel side. In this case the named lookup is not needed. Because we don't want to expose psock internals to user space socket options a user visible flag is also added. For TLS this is set for BPF it will be cleared.

Alos remove pr_notice, user gets an error code back and should check that rather than rely on logs.

---
 include/net/tcp.h  |    6 +++++
 net/ipv4/tcp_ulp.c |   59 ++++++++++++++++++++++++++++++++++++++++++++++++----
 net/tls/tls_main.c |    2 ++
 3 files changed, 62 insertions(+), 5 deletions(-)
include/net/tcp.h +6 -0
index 5827866..a58292d 100644
@@ -1983,6 +1983,10 @@ static inline void tcp_listendrop(const struct sock *sk)
 #define TCP_ULP_MAX		128
 #define TCP_ULP_BUF_MAX		(TCP_ULP_NAME_MAX*TCP_ULP_MAX)
 
+enum {
+	TCP_ULP_TLS,
+};
+
 struct tcp_ulp_ops {
 	struct list_head	list;
 
@@ -1991,7 +1995,9 @@ struct tcp_ulp_ops {
 	/* cleanup ulp */
 	void (*release)(struct sock *sk);
 
+	int		uid;
 	char		name[TCP_ULP_NAME_MAX];
+	bool		user_visible;
 	struct module	*owner;
 };
 int tcp_register_ulp(struct tcp_ulp_ops *type);
net/ipv4/tcp_ulp.c +54 -5

85 lines not on this page · raw message

net/tls/tls_main.c +2 -0
index 736719c..b0d5fce 100644
@@ -484,6 +484,8 @@ static int tls_init(struct sock *sk)
 
 static struct tcp_ulp_ops tcp_tls_ulp_ops __read_mostly = {
 	.name			= "tls",
+	.uid			= TCP_ULP_TLS,
+	.user_visible		= true,
 	.owner			= THIS_MODULE,
 	.init			= tls_init,
 };
maintainer
#2 lore raw

Re: [bpf PATCH v4 1/3] net: add a UID to use for ULP socket assignment

On 02/05/2018 07:17 PM, John Fastabend wrote:
[...]

Quoted 24 lines
net/ipv4/tcp_ulp.c +20 -0
@@ -124,6 +145,34 @@ int tcp_set_ulp(struct sock *sk, const char *name)
 	if (!ulp_ops)
 		return -ENOENT;
 
+	if (!ulp_ops->user_visible) {
+		module_put(ulp_ops->owner);
+		return -ENOENT;
+	}
+
+	err = ulp_ops->init(sk);
+	if (err) {
+		module_put(ulp_ops->owner);
+		return err;
+	}
+
+	icsk->icsk_ulp_ops = ulp_ops;
+	return 0;
+}
+
+int tcp_set_ulp_id(struct sock *sk, int ulp)
+{
+	struct inet_connection_sock *icsk = inet_csk(sk);
+	const struct tcp_ulp_ops *ulp_ops;
+	int err;

net/ipv4/tcp_ulp.c int tcp_set_ulp(struct sock *sk, const char *name) · line 177

Quoted 10 lines
net/ipv4/tcp_ulp.c +8 -0
+
+	if (icsk->icsk_ulp_ops)
+		return -EEXIST;
+
+	ulp_ops = __tcp_ulp_lookup(ulp);
+	if (!ulp_ops)
+		return -ENOENT;
+
 	err = ulp_ops->init(sk);
 	if (err) {

Just some minor feedback; v5 would have been impractical for just this as the fix looks otherwise fine, but I thought I'd mention it here for a possible follow-up cleanup to address:

tcp_set_ulp() and tcp_set_ulp_id() are quite different from API PoV despite that they almost look the same: tcp_set_ulp() respects user_visible bool while tcp_set_ulp_id() ignores it.

I'm wondering, could we somehow make this more obvious from the API itself?

Perhaps:

int tcp_set_ulp_id(struct sock *sk, int ulp, bool user_visible_only)

And we do:

	[...]
	if (user_visible_only && !ulp_ops->user_visible) {
		module_put(ulp_ops->owner);
		return -ENOENT;
	}

Maybe something like this would make it more obvious for callers, potentially same could be added in tcp_set_ulp(), so both APIs only differ in name vs id but not much more (yeah, one pulls in the module as well, but aside from that). Then, they could also be consolidated, since the only thing different left is __tcp_ulp_find_autoload() vs __tcp_ulp_lookup() to fetch the actual ulp_ops. Wdyt?

Thanks,
Daniel

author maintainer
#3 lore raw

PATCH v4 2/3 bpf: sockmap, add sock close() hook to remove socks

In reply to John Fastabend

The selftests test_maps program was leaving dangling BPF sockmap programs around because not all psock elements were removed from the map. The elements in turn hold a reference on the BPF program they are attached to causing BPF programs to stay open even after test_maps has completed.

The original intent was that sk_state_change() would be called when TCP socks went through TCP_CLOSE state. However, because socks may be in SOCK_DEAD state or the sock may be a listening socket the event is not always triggered.

To resolve this use the ULP infrastructure and register our own proto close() handler. This fixes the above case.

Fixes 174a79ff9515 ("bpf: sockmap with sk redirect support")
---
 include/net/tcp.h    |    2 +
 kernel/bpf/sockmap.c |  168 ++++++++++++++++++++++++++++++--------------------
 2 files changed, 103 insertions(+), 67 deletions(-)
include/net/tcp.h +2 -0
index a58292d..e3fc667 100644
@@ -1985,6 +1985,7 @@ static inline void tcp_listendrop(const struct sock *sk)
 
 enum {
 	TCP_ULP_TLS,
+	TCP_ULP_BPF,
 };
 
 struct tcp_ulp_ops {
@@ -2003,6 +2004,7 @@ struct tcp_ulp_ops {
 int tcp_register_ulp(struct tcp_ulp_ops *type);
 void tcp_unregister_ulp(struct tcp_ulp_ops *type);
 int tcp_set_ulp(struct sock *sk, const char *name);
+int tcp_set_ulp_id(struct sock *sk, const int ulp);
 void tcp_get_available_ulp(char *buf, size_t len);
 void tcp_cleanup_ulp(struct sock *sk);
 
kernel/bpf/sockmap.c +101 -67

228 lines not on this page · raw message

author maintainer
#4 lore raw

PATCH v4 3/3 bpf: sockmap, fix leaking maps with attached but not detached progs

In reply to John Fastabend

When a program is attached to a map we increment the program refcnt to ensure that the program is not removed while it is potentially being referenced from sockmap side. However, if this same program also references the map (this is a reasonably common pattern in my programs) then the verifier will also increment the maps refcnt from the verifier. This is to ensure the map doesn't get garbage collected while the program has a reference to it.

So we are left in a state where the map holds the refcnt on the program stopping it from being removed and releasing the map refcnt. And vice versa the program holds a refcnt on the map stopping it from releasing the refcnt on the prog.

All this is fine as long as users detach the program while the map fd is still around. But, if the user omits this detach command we are left with a dangling map we can no longer release.

To resolve this when the map fd is released decrement the program references and remove any reference from the map to the program. This fixes the issue with possibly dangling map and creates a user side API constraint. That is, the map fd must be held open for programs to be attached to a map.

Fixes 174a79ff9515 ("bpf: sockmap with sk redirect support")
---
 kernel/bpf/sockmap.c |   19 ++++++++++++++-----
 1 file changed, 14 insertions(+), 5 deletions(-)
kernel/bpf/sockmap.c +14 -5
index bd4a6d9..48c3341 100644
@@ -620,11 +620,6 @@ static void sock_map_free(struct bpf_map *map)
 	}
 	rcu_read_unlock();
 
-	if (stab->bpf_verdict)
-		bpf_prog_put(stab->bpf_verdict);
-	if (stab->bpf_parse)
-		bpf_prog_put(stab->bpf_parse);
-
 	sock_map_remove_complete(stab);
 }
 
@@ -900,6 +895,19 @@ static int sock_map_update_elem(struct bpf_map *map,
 	return err;
 }
 
+static void sock_map_release(struct bpf_map *map, struct file *map_file)
+{
+	struct bpf_stab *stab = container_of(map, struct bpf_stab, map);
+	struct bpf_prog *orig;
+
+	orig = xchg(&stab->bpf_parse, NULL);
+	if (orig)
+		bpf_prog_put(orig);
+	orig = xchg(&stab->bpf_verdict, NULL);
+	if (orig)
+		bpf_prog_put(orig);
+}
+
 const struct bpf_map_ops sock_map_ops = {
 	.map_alloc = sock_map_alloc,
 	.map_free = sock_map_free,
@@ -907,6 +915,7 @@ static int sock_map_update_elem(struct bpf_map *map,
 	.map_get_next_key = sock_map_get_next_key,
 	.map_update_elem = sock_map_update_elem,
 	.map_delete_elem = sock_map_delete_elem,
+	.map_release = sock_map_release,
 };
 
 BPF_CALL_4(bpf_sock_map_update, struct bpf_sock_ops_kern *, bpf_sock,
maintainer
#5 lore raw
In reply to John Fastabend
Quoted 17 lines · John Fastabend

A set of fixes for sockmap to resolve programs referencing sockmaps and closing without deleting all entries in the map and/or not detaching BPF programs attached to the map. Both leaving entries in the map and not detaching programs may result in the map failing to be removed by BPF infrastructure due to reference counts never reaching zero.

For this we pull in the ULP infrastructure to hook into the close() hook of the sock layer. This seemed natural because we have additional sockmap features (to add support for TX hooks) that will also use the ULP infrastructure. This allows us to cleanup entries in the map when socks are closed() and avoid trying to get the sk_state_change() hook to fire in all cases.

The second issue resolved here occurs when users don't detach programs. The gist is a refcnt issue resolved by implementing the release callback. See patch for details.

Quoted 7 lines

For testing I ran both sample/sockmap and selftests bpf/test_maps.c. Dave Watson ran TLS test suite on v1 version of the patches without the put_module error path change.

v4 fix missing rcu_unlock()
v3 wrap psock reference in RCU
v2 changes rebased onto bpf-next with small update adding module_put

Series applied to bpf tree, thanks John!