bpf-next: Add socket destroy capability
Subsystems (9)
cover bpf-next: Add socket destroy capability
This patch adds the capability to destroy sockets in BPF. We plan to use the capability in Cilium to force client sockets to reconnect when their remote load-balancing backends are deleted. The other use case is on-the-fly policy enforcement where existing socket connections prevented by policies need to
be terminated.
The use cases, and more details around the selected approach was presented at LPC 2022 -
https://lpc.events/event/16/contributions/1358/.
RFC discussion -
https://lore.kernel.org/netdev/CABG=zsBEh-P4NXk23eBJw7eajB5YJeRS7oPXnTAzs=yob4EMoQ@mail.gmail.com/T/#u.
Notes to the reviewers:
- We evaluated different approaches to allow the diag_destroy handlers to acquire/skip locks from the BPF iterator programs. The commit description has all the details. The selected work queue approach was discussed with Daniel Borkmann.
- We previously discussed the possibility of using sockmap to store sockets to be destroyed as an optimization, so that users may not need to iterate over all the host-wide sockets. This approach needs more discussion on the TCP side, as we may need to extend the logic that checks for certain TCP states while inserting sockets in a sockmap. So I've skipped those self test cases involving sockmap from the patch.
Aditi Ghag (2):
bpf: Add socket destroy capability selftests/bpf: Add tests for bpf_sock_destroy
include/linux/bpf.h | 1 + include/uapi/linux/bpf.h | 17 +++ kernel/bpf/core.c | 1 + kernel/trace/bpf_trace.c | 2 + net/core/filter.c | 70 ++++++++++ tools/include/uapi/linux/bpf.h | 17 +++ .../selftests/bpf/prog_tests/sock_destroy.c | 131 ++++++++++++++++++ .../selftests/bpf/progs/sock_destroy_prog.c | 96 +++++++++++++ 8 files changed, 335 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/sock_destroy.c create mode 100644 tools/testing/selftests/bpf/progs/sock_destroy_prog.c
base-commit: 0e43662e61f2569500ab83b8188c065603530785
Signature · 1 line
2.34.1
PATCH 1/2 bpf: Add socket destroy capability
The socket destroy helper is used to forcefully terminate sockets from certain BPF contexts. We plan to use the capability in Cilium to force client sockets to reconnect when their remote load-balancing backends are deleted. The other use case is on-the-fly policy enforcement where existing socket connections prevented by policies need to be terminated.
The helper is currently exposed to iterator type BPF programs where users can filter, and terminate a set of sockets.
Sockets are destroyed asynchronously using the work queue infrastructure. This allows for current the locking semantics within socket destroy handlers, as BPF iterators invoking the helper acquire *sock* locks. This also allows the helper to be invoked from non-sleepable contexts.
The other approach to skip acquiring locks by passing an argument to the diag_destroy handler didn't work out well for UDP, as the UDP abort function internally invokes another function that ends up acquiring *sock* lock.
While there are sleepable BPF iterators, these are limited to only certain map types. Furthermore, it's limiting in the sense that it wouldn't allow us to extend the helper to other non-sleepable BPF programs.
The work queue infrastructure processes work items from per-cpu structures. As the sock destroy work items are executed asynchronously, we need to ref count sockets before they are added to the work queue. The 'work_pending' check prevents duplicate ref counting of sockets in case users invoke the destroy helper for a socket multiple times. The {READ,WRITE}_ONCE macros ensure that the socket pointer stored in a work queue item isn't clobbered while the item is being processed. As BPF programs are non-preemptible, we can expect that once a socket is ref counted, no other socket can sneak in before the ref counted socket is added to the work queue for asynchronous destroy. Finally, users are expected to retry when the helper fails to queue a work item for a socket to be destroyed in case there is another destroy operation is in progress.
--- include/linux/bpf.h | 1 + include/uapi/linux/bpf.h | 17 +++++++++ kernel/bpf/core.c | 1 + kernel/trace/bpf_trace.c | 2 + net/core/filter.c | 70 ++++++++++++++++++++++++++++++++++ tools/include/uapi/linux/bpf.h | 17 +++++++++ 6 files changed, 108 insertions(+)
include/linux/bpf.h +1 -0
@@ -2676,6 +2676,7 @@ extern const struct bpf_func_proto bpf_get_retval_proto;extern const struct bpf_func_proto bpf_user_ringbuf_drain_proto;extern const struct bpf_func_proto bpf_cgrp_storage_get_proto;extern const struct bpf_func_proto bpf_cgrp_storage_delete_proto; +extern const struct bpf_func_proto bpf_sock_destroy_proto;const struct bpf_func_proto *tracing_prog_func_proto (enum bpf_func_id func_id,const struct bpf_prog *prog);
include/uapi/linux/bpf.h +17 -0
@@ -5484,6 +5484,22 @@ union bpf_attr {* 0 on success. * * **-ENOENT** if the bpf_local_storage cannot be found. +* +* int bpf_sock_destroy(struct sock *sk) +* Description +* Destroy the given socket with **ECONNABORTED** error code. +* +* *sk* must be a non-**NULL** pointer to a socket. +* +* Return +* The socket is destroyed asynchronosuly, so 0 return value may +* not suggest indicate that the socket was successfully destroyed. +* +* On error, may return **EPROTONOSUPPORT**, **EBUSY**, **EINVAL**. +* +* **-EPROTONOSUPPORT** if protocol specific destroy handler is not implemented. +* +* **-EBUSY** if another socket destroy operation is in progress. */ #define ___BPF_FUNC_MAPPER(FN, ctx...) \ FN(unspec, 0, ##ctx) \
@@ -5698,6 +5714,7 @@ union bpf_attr { FN(user_ringbuf_drain, 209, ##ctx) \ FN(cgrp_storage_get, 210, ##ctx) \ FN(cgrp_storage_delete, 211, ##ctx) \ + FN(sock_destroy, 212, ##ctx) \/* */ /* backwards-compatibility macros for users of __BPF_FUNC_MAPPER that don't
kernel/bpf/core.c +1 -0
@@ -2651,6 +2651,7 @@ const struct bpf_func_proto bpf_snprintf_btf_proto __weak;const struct bpf_func_proto bpf_seq_printf_btf_proto__weak ;const struct bpf_func_proto bpf_set_retval_proto__weak ;const struct bpf_func_proto bpf_get_retval_proto__weak ; +const struct bpf_func_proto bpf_sock_destroy_proto__weak ;const struct bpf_func_proto *__weak bpf_get_trace_printk_proto (void ) {
kernel/trace/bpf_trace.c +2 -0
@@ -1930,6 +1930,8 @@ tracing_prog_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)return &bpf_get_socket_ptr_cookie_proto;case BPF_FUNC_xdp_get_buff_len:return &bpf_xdp_get_buff_len_trace_proto; +case BPF_FUNC_sock_destroy: +return &bpf_sock_destroy_proto;#endif case BPF_FUNC_seq_printf: return prog->expected_attach_type == BPF_TRACE_ITER ?
net/core/filter.c +70 -0
79 lines not on this page · raw message
tools/include/uapi/linux/bpf.h +17 -0
@@ -5484,6 +5484,22 @@ union bpf_attr {* 0 on success. * * **-ENOENT** if the bpf_local_storage cannot be found. +* +* int bpf_sock_destroy(void *sk) +* Description +* Destroy the given socket with **ECONNABORTED** error code. +* +* *sk* must be a non-**NULL** pointer to a socket. +* +* Return +* The socket is destroyed asynchronosuly, so 0 return value may +* not indicate that the socket was successfully destroyed. +* +* On error, may return **EPROTONOSUPPORT**, **EBUSY**, **EINVAL**. +* +* **-EPROTONOSUPPORT** if protocol specific destroy handler is not implemented. +* +* **-EBUSY** if another socket destroy operation is in progress. */ #define ___BPF_FUNC_MAPPER(FN, ctx...) \ FN(unspec, 0, ##ctx) \
@@ -5698,6 +5714,7 @@ union bpf_attr { FN(user_ringbuf_drain, 209, ##ctx) \ FN(cgrp_storage_get, 210, ##ctx) \ FN(cgrp_storage_delete, 211, ##ctx) \ + FN(sock_destroy, 212, ##ctx) \/* */ /* backwards-compatibility macros for users of __BPF_FUNC_MAPPER that don't
Signature · 1 line
2.34.1
PATCH 2/2 selftests/bpf: Add tests for bpf_sock_destroy
In reply to Aditi GhagThe test cases for TCP and UDP mirror the
intended usages of the helper.
As the helper destroys sockets asynchronously,
the tests have sleep invocation before validating
if the sockets were destroyed by sending data.
Also, while all the protocol specific helpers set ECONNABORTED error code on the destroyed sockets, only the TCP test case has the validation check. UDP sockets have an overriding error code from the disconnect call during abort.
--- .../selftests/bpf/prog_tests/sock_destroy.c | 131 ++++++++++++++++++ .../selftests/bpf/progs/sock_destroy_prog.c | 96 +++++++++++++ 2 files changed, 227 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/sock_destroy.c create mode 100644 tools/testing/selftests/bpf/progs/sock_destroy_prog.c
tools/testing/selftests/bpf/prog_tests/sock_destroy.c new file +131 -0
@@ -0,0 +1,131 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <test_progs.h> + +#include "sock_destroy_prog.skel.h" +#include "network_helpers.h" + +#define ECONNABORTED 103 + +static int duration; + +static void start_iter_sockets (struct bpf_program *prog) +{ +struct bpf_link *link; +char buf[16 ] = {}; +int iter_fd, len; + + link =bpf_program__attach_iter (prog,NULL ); +if (!ASSERT_OK_PTR (link,"attach_iter" )) +return ; + + iter_fd =bpf_iter_create (bpf_link__fd (link)); +if (!ASSERT_GE (iter_fd,0 ,"create_iter" )) + goto free_link; + +while ((len =read (iter_fd, buf,sizeof (buf))) >0 ) + ; +CHECK (len <0 ,"read" ,"read failed: %s\n" ,strerror (errno)); + +close (iter_fd); + +free_link: +bpf_link__destroy (link); +} + +void test_tcp (struct sock_destroy_prog *skel) +{ +int serv =-1 , clien =-1 , n =0 ; + + serv =start_server (AF_INET6 ,SOCK_STREAM ,NULL ,0 ,0 ); +if (CHECK (serv <0 ,"start_server" ,"failed to start server\n" )) + goto cleanup_serv; + + clien =connect_to_fd (serv,0 ); +if (CHECK (clien <0 ,"connect_to_fd" ,"errno %d\n" , errno)) + goto cleanup_serv; + + serv =accept (serv,NULL ,NULL ); +if (CHECK (serv <0 ,"accept" ,"errno %d\n" , errno)) + goto cleanup; + + n =send (clien,"t" ,1 ,0 ); +if (CHECK (n <0 ,"client_send" ,"client failed to send on socket\n" )) + goto cleanup; + +start_iter_sockets (skel->progs.iter_tcp6); + +// Sockets are destroyed asynchronously. +usleep (1000 ); + n =send (clien,"t" ,1 ,0 ); + +if (CHECK (n >0 ,"client_send" ,"succeeded on destroyed socket\n" )) + goto cleanup; +CHECK (errno !=ECONNABORTED ,"client_send" ,"unexpected error code on destroyed socket\n" ); + + +cleanup: +close (clien); +cleanup_serv: +close (serv); +} + + +void test_udp (struct sock_destroy_prog *skel) +{ +int serv =-1 , clien =-1 , n =0 ; + + serv =start_server (AF_INET6 ,SOCK_DGRAM ,NULL ,0 ,0 ); +if (CHECK (serv <0 ,"start_server" ,"failed to start server\n" )) + goto cleanup_serv; + + clien =connect_to_fd (serv,0 ); +if (CHECK (clien <0 ,"connect_to_fd" ,"errno %d\n" , errno)) + goto cleanup_serv; + + n =send (clien,"t" ,1 ,0 ); +if (CHECK (n <0 ,"client_send" ,"client failed to send on socket\n" )) + goto cleanup; + +start_iter_sockets (skel->progs.iter_udp6); + +// Sockets are destroyed asynchronously. +usleep (1000 ); + + n =send (clien,"t" ,1 ,0 ); +if (CHECK (n >0 ,"client_send" ,"succeeded on destroyed socket\n" )) + goto cleanup; +// UDP sockets have an overriding error code after they are disconnected. + + +cleanup: +close (clien); +cleanup_serv: +close (serv); +} + +void test_sock_destroy (void ) +{ +int cgroup_fd =0 ; +struct sock_destroy_prog *skel; + + skel =sock_destroy_prog__open_and_load (); +if (!ASSERT_OK_PTR (skel,"skel_open" )) +return ; + + cgroup_fd =test__join_cgroup ("/sock_destroy" ); +if (CHECK (cgroup_fd <0 ,"join_cgroup" ,"cgroup creation failed\n" )) + goto close_cgroup_fd; + + skel->links.sock_connect =bpf_program__attach_cgroup ( + skel->progs.sock_connect, cgroup_fd); +if (!ASSERT_OK_PTR (skel->links.sock_connect,"prog_attach" )) + goto close_cgroup_fd; + +test_tcp (skel); +test_udp (skel); + + +close_cgroup_fd: +close (cgroup_fd); +sock_destroy_prog__destroy (skel); +}
tools/testing/selftests/bpf/progs/sock_destroy_prog.c new file +96 -0
96 lines not on this page · raw message
Signature · 1 line
2.34.1
Re: [PATCH 1/2] bpf: Add socket destroy capability
In reply to Aditi GhagQuoted 10 lines · Aditi Ghag
The socket destroy helper is used to forcefully terminate sockets from certain BPF contexts. We plan to use the capability in Cilium to force client sockets to reconnect when their remote load-balancing backends are deleted. The other use case is on-the-fly policy enforcement where existing socket connections prevented by policies need to be terminated.
Quoted 7 lines
The helper is currently exposed to iterator type BPF programs where users can filter, and terminate a set of sockets.
Sockets are destroyed asynchronously using the work queue infrastructure. This allows for current the locking semantics within
s/current the/the current/ ?
Quoted 27 lines
socket destroy handlers, as BPF iterators invoking the helper acquire *sock* locks. This also allows the helper to be invoked from non-sleepable contexts.
The other approach to skip acquiring locks by passing an argument to thediag_destroyhandler didn't work out well for UDP, as the UDP abort function internally invokes another function that ends up acquiring *sock* lock.
While there are sleepable BPF iterators, these are limited to only certain map types. Furthermore, it's limiting in the sense that it wouldn't allow us to extend the helper to other non-sleepable BPF programs.The work queue infrastructure processes work items from per-cpu structures. As the sock destroy work items are executed asynchronously, we need to ref count sockets before they are added to the work queue. The '
work_pending' check prevents duplicate ref counting of sockets in case users invoke the destroy helper for a socket multiple times. The{READ,WRITE}_ONCEmacros ensure that the socket pointer stored in a work queue item isn't clobbered while the item is being processed. As BPF programs
Quoted 8 lines
are non-preemptible, we can expect that once a socket is ref counted, no other socket can sneak in before the ref counted socket is added to the work queue for asynchronous destroy. Finally, users are expected to retry when the helper fails to queue a work item for a socket to be destroyed in case there is another destroy operation is in progress.
nit: maybe reformat to fit into 80 characters per line? A bit hard to read with this narrow formatting..
Quoted 31 lines
Signed-off-by Aditi Ghag <[email protected]>--- include/linux/bpf.h | 1 + include/uapi/linux/bpf.h | 17 +++++++++ kernel/bpf/core.c | 1 + kernel/trace/bpf_trace.c | 2 + net/core/filter.c | 70 ++++++++++++++++++++++++++++++++++ tools/include/uapi/linux/bpf.h | 17 +++++++++ 6 files changed, 108 insertions(+)include/linux/bpf.h +1
-0@@ -2676,6 +2676,7 @@ extern const struct bpf_func_proto bpf_get_retval_proto;extern const struct bpf_func_proto bpf_user_ringbuf_drain_proto;extern const struct bpf_func_proto bpf_cgrp_storage_get_proto;extern const struct bpf_func_proto bpf_cgrp_storage_delete_proto; +extern const struct bpf_func_proto bpf_sock_destroy_proto; const struct bpf_func_proto *tracing_prog_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog);include/uapi/linux/bpf.h +0
-0@@ -5484,6 +5484,22 @@ union bpf_attr {* 0 on success. * * **-ENOENT** if the bpf_local_storage cannot be found.
include/uapi/linux/bpf.h union bpf_attr { · line 5496
Quoted 10 lines
include/uapi/linux/bpf.h +10
-0+* +* int bpf_sock_destroy(struct sock *sk) +* Description +* Destroy the given socket with **ECONNABORTED** error code. +* +* *sk* must be a non-**NULL** pointer to a socket. +* +* Return +* The socket is destroyed asynchronosuly, so 0 return value may +* not suggest indicate that the socket was successfully destroyed.
s/suggest indicate/ with either suggest or indicate?
Quoted 103 lines
include/uapi/linux/bpf.h +4
-0+* +* On error, may return **EPROTONOSUPPORT**, **EBUSY**, **EINVAL**. +* +* **-EPROTONOSUPPORT** if protocol specific destroy handler is not implemented.
+ *
+ * **-EBUSY** if another socket destroy operation is in progress.
*/
#define___BPF_FUNC_MAPPER(FN, ctx...)\
FN(unspec, 0, ##ctx)\include/uapi/linux/bpf.h +1
-0@@ -5698,6 +5714,7 @@ union bpf_attr { FN(user_ringbuf_drain, 209, ##ctx) \ FN(cgrp_storage_get, 210, ##ctx) \ FN(cgrp_storage_delete, 211, ##ctx) \ + FN(sock_destroy, 212, ##ctx) \/* */ /* backwards-compatibility macros for users of __BPF_FUNC_MAPPER that don't
kernel/bpf/core.c +1
-0@@ -2651,6 +2651,7 @@ const struct bpf_func_proto bpf_snprintf_btf_proto__weak ;const struct bpf_func_proto bpf_seq_printf_btf_proto__weak ;const struct bpf_func_proto bpf_set_retval_proto__weak ;const struct bpf_func_proto bpf_get_retval_proto__weak ; +const struct bpf_func_proto bpf_sock_destroy_proto__weak ;const struct bpf_func_proto *__weak bpf_get_trace_printk_proto (void ){kernel/trace/bpf_trace.c +2
-0@@ -1930,6 +1930,8 @@ tracing_prog_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)return &bpf_get_socket_ptr_cookie_proto;case BPF_FUNC_xdp_get_buff_len:return &bpf_xdp_get_buff_len_trace_proto; +case BPF_FUNC_sock_destroy: +return &bpf_sock_destroy_proto;#endif case BPF_FUNC_seq_printf:return
prog->expected_attach_type==BPF_TRACE_ITER?net/core/filter.c +42
-0@@ -11569,6 +11569,8 @@ bpf_sk_base_func_proto(enum bpf_func_id func_id)break ;case BPF_FUNC_ktime_get_coarse_ns:return &bpf_ktime_get_coarse_ns_proto; +case BPF_FUNC_sock_destroy: +return &bpf_sock_destroy_proto;default :return bpf_base_func_proto (func_id); }@@ -11578,3 +11580,71 @@ bpf_sk_base_func_proto(enum bpf_func_id func_id)return func; } + +struct sock_destroy_work { +struct sock *sk; +struct work_struct destroy; +}; + +static DEFINE_PER_CPU(struct sock_destroy_work, sock_destroy_workqueue); + +static void bpf_sock_destroy_fn (struct work_struct *work) +{ +struct sock_destroy_work *sd_work =container_of (work, + struct sock_destroy_work, destroy); +struct sock *sk =READ_ONCE (sd_work->sk); + + sk->sk_prot->diag_destroy (sk,ECONNABORTED ); +sock_put (sk); +} + +static int __init bpf_sock_destroy_workqueue_init (void ) +{ +int cpu; +struct sock_destroy_work *work; + +for_each_possible_cpu (cpu) { + work =per_cpu_ptr (&sock_destroy_workqueue, cpu); +INIT_WORK (&work->destroy, bpf_sock_destroy_fn); + } + +return 0 ; +} +subsys_initcall (bpf_sock_destroy_workqueue_init); + +BPF_CALL_1(bpf_sock_destroy, struct sock *, sk) +{ +struct sock_destroy_work *sd_work; + +if (!sk->sk_prot->diag_destroy) +return -EOPNOTSUPP ; + + sd_work =this_cpu_ptr (&sock_destroy_workqueue);
[..]
net/core/filter.c bpf_sk_base_func_proto(enum bpf_func_id func_id) · line 11626
Quoted 4 lines
net/core/filter.c +4
-0+/* This check prevents duplicate ref counting +* of sockets, in case the handler is invoked +* multiple times for the same socket. +*/
This means this helper can also be called for a single socket during invocation; is it an ok compromise?
I'm also assuming it's still possible that this helper gets called for the same socket on different cpus?
Quoted 63 lines
net/core/filter.c +24
-0+if (work_pending (&sd_work->destroy)) +return -EBUSY ; + +/* Ref counting ensures that the socket +* isn't deleted from underneath us before +* the work queue item is processed. +*/ +if (!refcount_inc_not_zero (&sk->sk_refcnt)) +return -EINVAL ; + +WRITE_ONCE (sd_work->sk, sk); +if (!queue_work (system_wq, &sd_work->destroy)) { +sock_put (sk); +return -EBUSY ; + } + +return 0 ; +} + +const struct bpf_func_proto bpf_sock_destroy_proto = { + .func = bpf_sock_destroy, + .ret_type =RET_INTEGER , + .arg1_type =ARG_PTR_TO_BTF_ID_SOCK_COMMON , +};tools/include/uapi/linux/bpf.h +14
-0@@ -5484,6 +5484,22 @@ union bpf_attr {* 0 on success. * * **-ENOENT** if the bpf_local_storage cannot be found. +* +* int bpf_sock_destroy(void *sk) +* Description +* Destroy the given socket with **ECONNABORTED** error code. +* +* *sk* must be a non-**NULL** pointer to a socket. +* +* Return +* The socket is destroyed asynchronosuly, so 0 return value may +* not indicate that the socket was successfully destroyed. +* +* On error, may return **EPROTONOSUPPORT**, **EBUSY**, **EINVAL**. +* +* **-EPROTONOSUPPORT** if protocol specific destroy handler is not implemented.
+ *
+ * **-EBUSY** if another socket destroy operation is in progress.
*/
#define___BPF_FUNC_MAPPER(FN, ctx...)\
FN(unspec, 0, ##ctx)\tools/include/uapi/linux/bpf.h +1
-0@@ -5698,6 +5714,7 @@ union bpf_attr { FN(user_ringbuf_drain, 209, ##ctx) \ FN(cgrp_storage_get, 210, ##ctx) \ FN(cgrp_storage_delete, 211, ##ctx) \ + FN(sock_destroy, 212, ##ctx) \/* */ /* backwards-compatibility macros for users of __BPF_FUNC_MAPPER that don't
--
2.34.1
Re: [PATCH 2/2] selftests/bpf: Add tests for bpf_sock_destroy
In reply to Aditi GhagQuoted 44 lines · Aditi Ghag
The test cases for TCP and UDP mirror the
intended usages of the helper.As the helper destroys sockets asynchronously,
the tests have sleep invocation before validating
if the sockets were destroyed by sending data.Also, while all the protocol specific helpers set
ECONNABORTEDerror code on the destroyed sockets, only the TCP test case has the validation check. UDP sockets have an overriding error code from the disconnect call during abort.Signed-off-by Aditi Ghag <[email protected]>--- .../selftests/bpf/prog_tests/sock_destroy.c | 131 ++++++++++++++++++ .../selftests/bpf/progs/sock_destroy_prog.c | 96 +++++++++++++ 2 files changed, 227 insertions(+)create mode 100644
tools/testing/selftests/bpf/prog_tests/sock_destroy.ccreate mode 100644tools/testing/selftests/bpf/progs/sock_destroy_prog.ctools/testing/selftests/bpf/prog_tests/sock_destroy.c new file +17
-0@@ -0,0 +1,131 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <test_progs.h> + +#include "sock_destroy_prog.skel.h" +#include "network_helpers.h" + +#define ECONNABORTED 103 + +static int duration; + +static void start_iter_sockets (struct bpf_program *prog) +{ +struct bpf_link *link; +char buf[16 ] = {}; +int iter_fd, len; + + link =bpf_program__attach_iter (prog,NULL );
tools/testing/selftests/bpf/prog_tests/sock_destroy.c line 27
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/sock_destroy.c new file +10
-0+if (!ASSERT_OK_PTR (link,"attach_iter" )) +return ; + + iter_fd =bpf_iter_create (bpf_link__fd (link)); +if (!ASSERT_GE (iter_fd,0 ,"create_iter" )) + goto free_link; + +while ((len =read (iter_fd, buf,sizeof (buf))) >0 ) + ; +CHECK (len <0 ,"read" ,"read failed: %s\n" ,strerror (errno));
CHECK(s) are not super appealing in the new code, can you replace them all with ASSERT(s) ? This should also let you drop 'duration' variable.
Quoted 22 lines
tools/testing/selftests/bpf/prog_tests/sock_destroy.c new file +22
-0+ +close (iter_fd); + +free_link: +bpf_link__destroy (link); +} + +void test_tcp (struct sock_destroy_prog *skel) +{ +int serv =-1 , clien =-1 , n =0 ; + + serv =start_server (AF_INET6 ,SOCK_STREAM ,NULL ,0 ,0 ); +if (CHECK (serv <0 ,"start_server" ,"failed to start server\n" )) + goto cleanup_serv; + + clien =connect_to_fd (serv,0 ); +if (CHECK (clien <0 ,"connect_to_fd" ,"errno %d\n" , errno)) + goto cleanup_serv; + + serv =accept (serv,NULL ,NULL ); +if (CHECK (serv <0 ,"accept" ,"errno %d\n" , errno)) + goto cleanup;
tools/testing/selftests/bpf/prog_tests/sock_destroy.c line 59
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/sock_destroy.c new file +10
-0+ + n =send (clien,"t" ,1 ,0 ); +if (CHECK (n <0 ,"client_send" ,"client failed to send on socket\n" )) + goto cleanup; + +start_iter_sockets (skel->progs.iter_tcp6); + +// Sockets are destroyed asynchronously. +usleep (1000 ); + n =send (clien,"t" ,1 ,0 );
That's racy. Do some kind of:
while (retries--) {
usleep(100);
n = send (...)
if (n < 0)
break;
}
ASSERT_LT(n, 0);
ASSERT_EQ(errno, ECONNABORTED);Quoted 24 lines
tools/testing/selftests/bpf/prog_tests/sock_destroy.c new file +4
-0+ +if (CHECK (n >0 ,"client_send" ,"succeeded on destroyed socket\n" )) + goto cleanup; + CHECK(errno != ECONNABORTED, "client_send", "unexpected error code ondestroyed socket\n");
tools/testing/selftests/bpf/prog_tests/sock_destroy.c new file +19
-0+ + +cleanup: +close (clien); +cleanup_serv: +close (serv); +} + + +void test_udp (struct sock_destroy_prog *skel) +{ +int serv =-1 , clien =-1 , n =0 ; + + serv =start_server (AF_INET6 ,SOCK_DGRAM ,NULL ,0 ,0 ); +if (CHECK (serv <0 ,"start_server" ,"failed to start server\n" )) + goto cleanup_serv; + + clien =connect_to_fd (serv,0 ); +if (CHECK (clien <0 ,"connect_to_fd" ,"errno %d\n" , errno))
tools/testing/selftests/bpf/prog_tests/sock_destroy.c line 92
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/sock_destroy.c new file +10
-0+ goto cleanup_serv; + + n =send (clien,"t" ,1 ,0 ); +if (CHECK (n <0 ,"client_send" ,"client failed to send on socket\n" )) + goto cleanup; + +start_iter_sockets (skel->progs.iter_udp6); + +// Sockets are destroyed asynchronously. +usleep (1000 );
Same here.
Quoted 144 lines
tools/testing/selftests/bpf/prog_tests/sock_destroy.c new file +5
-0+ + n =send (clien,"t" ,1 ,0 ); +if (CHECK (n >0 ,"client_send" ,"succeeded on destroyed socket\n" )) + goto cleanup; +// UDP sockets have an overriding error code after they are disconnected.
tools/testing/selftests/bpf/prog_tests/sock_destroy.c new file +34
-0+ + +cleanup: +close (clien); +cleanup_serv: +close (serv); +} + +void test_sock_destroy (void ) +{ +int cgroup_fd =0 ; +struct sock_destroy_prog *skel; + + skel =sock_destroy_prog__open_and_load (); +if (!ASSERT_OK_PTR (skel,"skel_open" )) +return ; + + cgroup_fd =test__join_cgroup ("/sock_destroy" ); +if (CHECK (cgroup_fd <0 ,"join_cgroup" ,"cgroup creation failed\n" )) + goto close_cgroup_fd; + + skel->links.sock_connect =bpf_program__attach_cgroup ( + skel->progs.sock_connect, cgroup_fd); +if (!ASSERT_OK_PTR (skel->links.sock_connect,"prog_attach" )) + goto close_cgroup_fd; + +test_tcp (skel); +test_udp (skel); + + +close_cgroup_fd: +close (cgroup_fd); +sock_destroy_prog__destroy (skel); +}tools/testing/selftests/bpf/progs/sock_destroy_prog.c new file +96
-1@@ -0,0 +1,96 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include "vmlinux.h" + +#include <bpf/bpf_helpers.h> + +#define AF_INET6 10 + +struct { + __uint(type, BPF_MAP_TYPE_ARRAY); + __uint(max_entries, 1); + __type(key, __u32); + __type(value, __u64); +} tcp_conn_sockets SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_ARRAY); + __uint(max_entries, 1); + __type(key, __u32); + __type(value, __u64); +} udp_conn_sockets SEC(".maps"); + +SEC("cgroup/connect6") +int sock_connect (struct bpf_sock_addr *ctx) +{ +int key =0 ; +__u64 sock_cookie =0 ; +__u32 keyc =0 ; + +if (ctx->family !=AF_INET6 || ctx->user_family !=AF_INET6 ) +return 1 ; + + sock_cookie =bpf_get_socket_cookie (ctx); +if (ctx->protocol ==IPPROTO_TCP ) +bpf_map_update_elem (&tcp_conn_sockets, &key, &sock_cookie,0 ); +else if (ctx->protocol ==IPPROTO_UDP ) +bpf_map_update_elem (&udp_conn_sockets, &keyc, &sock_cookie,0 ); +else +return 1 ; + +return 1 ; +} + +SEC("iter/tcp") +int iter_tcp6 (struct bpf_iter__tcp *ctx) +{ +struct sock_common *sk_common = ctx->sk_common; +struct seq_file *seq = ctx->meta->seq; +__u64 sock_cookie =0 ; +__u64 *val; +int key =0 ; + +if (!sk_common) +return 0 ; + +if (sk_common->skc_family !=AF_INET6 ) +return 0 ; + + sock_cookie =bpf_get_socket_cookie (sk_common); + val =bpf_map_lookup_elem (&tcp_conn_sockets, &key); + +if (!val) +return 0 ; + +if (sock_cookie == *val) +bpf_sock_destroy (sk_common); + +return 0 ; +} + +SEC("iter/udp") +int iter_udp6 (struct bpf_iter__udp *ctx) +{ +struct seq_file *seq = ctx->meta->seq; +struct udp_sock *udp_sk = ctx->udp_sk; +struct sock *sk = (struct sock *) udp_sk; +__u64 sock_cookie =0 ; +int key =0 ; +__u64 *val; + +if (!sk) +return 0 ; + + sock_cookie =bpf_get_socket_cookie (sk); + val =bpf_map_lookup_elem (&udp_conn_sockets, &key); + +if (!val) +return 0 ; + +if (sock_cookie == *val) +bpf_sock_destroy (sk); + +return 0 ; +} + +char _license[] SEC("license") = "GPL"; --2.34.1
Re: [PATCH 1/2] bpf: Add socket destroy capability
In reply to Aditi GhagQuoted 140 lines · Aditi Ghag
The socket destroy helper is used to forcefully terminate sockets from certain BPF contexts. We plan to use the capability in Cilium to force client sockets to reconnect when their remote load-balancing backends are deleted. The other use case is on-the-fly policy enforcement where existing socket connections prevented by policies need to be terminated.
The helper is currently exposed to iterator type BPF programs where users can filter, and terminate a set of sockets.
Sockets are destroyed asynchronously using the work queue infrastructure. This allows for current the locking semantics within socket destroy handlers, as BPF iterators invoking the helper acquire *sock* locks. This also allows the helper to be invoked from non-sleepable contexts.
The other approach to skip acquiring locks by passing an argument to thediag_destroyhandler didn't work out well for UDP, as the UDP abort function internally invokes another function that ends up acquiring *sock* lock.
While there are sleepable BPF iterators, these are limited to only certain map types. Furthermore, it's limiting in the sense that it wouldn't allow us to extend the helper to other non-sleepable BPF programs.The work queue infrastructure processes work items from per-cpu structures. As the sock destroy work items are executed asynchronously, we need to ref count sockets before they are added to the work queue. The '
work_pending' check prevents duplicate ref counting of sockets in case users invoke the destroy helper for a socket multiple times. The{READ,WRITE}_ONCEmacros ensure that the socket pointer stored in a work queue item isn't clobbered while the item is being processed. As BPF programs are non-preemptible, we can expect that once a socket is ref counted, no other socket can sneak in before the ref counted socket is added to the work queue for asynchronous destroy. Finally, users are expected to retry when the helper fails to queue a work item for a socket to be destroyed in case there is another destroy operation is in progress.Signed-off-by Aditi Ghag <[email protected]>--- include/linux/bpf.h | 1 + include/uapi/linux/bpf.h | 17 +++++++++ kernel/bpf/core.c | 1 + kernel/trace/bpf_trace.c | 2 + net/core/filter.c | 70 ++++++++++++++++++++++++++++++++++ tools/include/uapi/linux/bpf.h | 17 +++++++++ 6 files changed, 108 insertions(+)include/linux/bpf.h +1
-0@@ -2676,6 +2676,7 @@ extern const struct bpf_func_proto bpf_get_retval_proto;extern const struct bpf_func_proto bpf_user_ringbuf_drain_proto;extern const struct bpf_func_proto bpf_cgrp_storage_get_proto;extern const struct bpf_func_proto bpf_cgrp_storage_delete_proto; +extern const struct bpf_func_proto bpf_sock_destroy_proto;const struct bpf_func_proto *tracing_prog_func_proto (enum bpf_func_id func_id,const struct bpf_prog *prog);include/uapi/linux/bpf.h +17
-0@@ -5484,6 +5484,22 @@ union bpf_attr {* 0 on success. * * **-ENOENT** if the bpf_local_storage cannot be found. +* +* int bpf_sock_destroy(struct sock *sk) +* Description +* Destroy the given socket with **ECONNABORTED** error code. +* +* *sk* must be a non-**NULL** pointer to a socket. +* +* Return +* The socket is destroyed asynchronosuly, so 0 return value may +* not suggest indicate that the socket was successfully destroyed. +* +* On error, may return **EPROTONOSUPPORT**, **EBUSY**, **EINVAL**. +* +* **-EPROTONOSUPPORT** if protocol specific destroy handler is not implemented. +* +* **-EBUSY** if another socket destroy operation is in progress. */ #define ___BPF_FUNC_MAPPER(FN, ctx...) \ FN(unspec, 0, ##ctx) \@@ -5698,6 +5714,7 @@ union bpf_attr { FN(user_ringbuf_drain, 209, ##ctx) \ FN(cgrp_storage_get, 210, ##ctx) \ FN(cgrp_storage_delete, 211, ##ctx) \ + FN(sock_destroy, 212, ##ctx) \/* */ /* backwards-compatibility macros for users of __BPF_FUNC_MAPPER that don't kernel/bpf/core.c +1
-0@@ -2651,6 +2651,7 @@ const struct bpf_func_proto bpf_snprintf_btf_proto __weak;const struct bpf_func_proto bpf_seq_printf_btf_proto__weak ;const struct bpf_func_proto bpf_set_retval_proto__weak ;const struct bpf_func_proto bpf_get_retval_proto__weak ; +const struct bpf_func_proto bpf_sock_destroy_proto__weak ;const struct bpf_func_proto *__weak bpf_get_trace_printk_proto (void ) {kernel/trace/bpf_trace.c +2
-0@@ -1930,6 +1930,8 @@ tracing_prog_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)return &bpf_get_socket_ptr_cookie_proto;case BPF_FUNC_xdp_get_buff_len:return &bpf_xdp_get_buff_len_trace_proto; +case BPF_FUNC_sock_destroy: +return &bpf_sock_destroy_proto;#endif case BPF_FUNC_seq_printf: return prog->expected_attach_type == BPF_TRACE_ITER ?net/core/filter.c +0
-0
net/core/filter.c bpf_sk_base_func_proto(enum bpf_func_id func_id) · line 11574
Quoted 6 lines
net/core/filter.c +2
-0@@ -11569,6 +11569,8 @@ bpf_sk_base_func_proto(enum bpf_func_id func_id)break ;case BPF_FUNC_ktime_get_coarse_ns:return &bpf_ktime_get_coarse_ns_proto; +case BPF_FUNC_sock_destroy: +return &bpf_sock_destroy_proto;default :
This is a really neat feature! One question though; above it seems to suggest that the helper is only exposed to BPF iterators, but by adding it to bpf_sk_base_func_proto() won't it be available to other BPF programs like sock_addr, sk_filter etc? If I've got that right, we'd definitely want to exclude potentially unprivileged programs from having access to this helper. And to be clear, I'd definitely see value in having it accessible to other BPF program types too like sockops if possible, but just wanted to check.
Quoted 33 lines
net/core/filter.c +27
-0return bpf_base_func_proto (func_id); }@@ -11578,3 +11580,71 @@ bpf_sk_base_func_proto(enum bpf_func_id func_id)return func; } + +struct sock_destroy_work { +struct sock *sk; +struct work_struct destroy; +}; + +static DEFINE_PER_CPU(struct sock_destroy_work, sock_destroy_workqueue); + +static void bpf_sock_destroy_fn (struct work_struct *work) +{ +struct sock_destroy_work *sd_work =container_of (work, + struct sock_destroy_work, destroy); +struct sock *sk =READ_ONCE (sd_work->sk); + + sk->sk_prot->diag_destroy (sk,ECONNABORTED ); +sock_put (sk); +} + +static int __init bpf_sock_destroy_workqueue_init (void ) +{ +int cpu; +struct sock_destroy_work *work; + +for_each_possible_cpu (cpu) { + work =per_cpu_ptr (&sock_destroy_workqueue, cpu); +INIT_WORK (&work->destroy, bpf_sock_destroy_fn); + }
net/core/filter.c bpf_sk_base_func_proto(enum bpf_func_id func_id) · line 11619
Quoted 10 lines
net/core/filter.c +10
-0+ +return 0 ; +} +subsys_initcall (bpf_sock_destroy_workqueue_init); + +BPF_CALL_1(bpf_sock_destroy, struct sock *, sk) +{ +struct sock_destroy_work *sd_work; + +if (!sk->sk_prot->diag_destroy)
risk of a NULL sk here?
Thanks!
Alan
Martin KaFai Lau
reviewerRe: [PATCH 1/2] bpf: Add socket destroy capability
In reply to Aditi GhagQuoted 14 lines · Aditi Ghag
The socket destroy helper is used to forcefully terminate sockets from certain BPF contexts. We plan to use the capability in Cilium to force client sockets to reconnect when their remote load-balancing backends are deleted. The other use case is on-the-fly policy enforcement where existing socket connections prevented by policies need to be terminated.
The helper is currently exposed to iterator type BPF programs where users can filter, and terminate a set of sockets.
Quoted 15 lines
Sockets are destroyed asynchronously using the work queue infrastructure. This allows for current the locking semantics within socket destroy handlers, as BPF iterators invoking the helper acquire *sock* locks. This also allows the helper to be invoked from non-sleepable contexts.
The other approach to skip acquiring locks by passing an argument to thediag_destroyhandler didn't work out well for UDP, as the UDP abort function internally invokes another function that ends up acquiring *sock* lock.
While there are sleepable BPF iterators, these are limited to only certain map types.
bpf-iter program can be sleepable and non sleepable. Both sleepable and non sleepable tcp/unix bpf-iter programs have been able to call bpf_setsockopt() synchronously. bpf_setsockopt() also requires the sock lock to be held first. The situation on calling '.diag_destroy' from bpf-iter should not be much different from calling bpf_setsockopt(). From a quick look at tcp_abort and udp_abort, I don't see they might sleep also and you may want to double check. Even '.diag_destroy' was only usable in sleepable 'bpf-iter' because it might sleep, the common bpf map types are already available to the sleepable programs.
At the kernel side, the tcp and unix iter acquire the lock_sock() first (eg. in bpf_iter_tcp_seq_show()) before calling the bpf-iter prog . At the kernel setsockopt code (eg. do_ip_setsockopt()), it uses sockopt_lock_sock() and avoids doing the lock if it has already been guaranteed by the bpf running context.
For udp, I don't see how the udp_abort acquires the sock lock differently from tcp_abort. I assume the actual problem seen in udp_abort is related to the '->unhash()' part which acquires the udp_table's bucket lock. This is a problem for udp bpf-iter only because the udp bpf-iter did not release the udp_table's bucket lock before calling the bpf prog. The tcp (and unix) bpf-iter releases the bucket lock first before calling the bpf prog. This was done explicitly to allow acquiring the sock lock before calling the bpf prog because otherwise it will have a lock ordering issue. Hence, for this reason, bpf_setsockopt() is only available to tcp and unix bpf-iter but not udp bpf-iter. The udp-iter needs to do similar change like the tcp and unix iter (https://lore.kernel.org/bpf/[email protected]/): batch, release the bucket's lock, lock the sock, and then call bpf prog. This will allow udp-iter to call bpf_setsockopt() like its tcp and unix counterpart. That will also allow udp bpf-iter prog to directly do '.diag_destroy'.
Daniel Borkmann
maintainerRe: [PATCH 1/2] bpf: Add socket destroy capability
Quoted 31 lines · Martin KaFai Lau
Quoted 14 lines · Aditi Ghag
The socket destroy helper is used to forcefully terminate sockets from certain BPF contexts. We plan to use the capability in Cilium to force client sockets to reconnect when their remote load-balancing backends are deleted. The other use case is on-the-fly policy enforcement where existing socket connections prevented by policies need to be terminated.
The helper is currently exposed to iterator type BPF programs where users can filter, and terminate a set of sockets.
Quoted 15 lines
Sockets are destroyed asynchronously using the work queue infrastructure. This allows for current the locking semantics within socket destroy handlers, as BPF iterators invoking the helper acquire *sock* locks. This also allows the helper to be invoked from non-sleepable contexts.
The other approach to skip acquiring locks by passing an argument to thediag_destroyhandler didn't work out well for UDP, as the UDP abort function internally invokes another function that ends up acquiring *sock* lock.
While there are sleepable BPF iterators, these are limited to only certain map types.
Quoted 5 lines
bpf-iter program can be sleepable and non sleepable. Both sleepable and non sleepable tcp/unix bpf-iter programs have been able to call
bpf_setsockopt()synchronously.bpf_setsockopt()also requires the sock lock to be held first. The situation on calling '.diag_destroy' from bpf-iter should not be much different from callingbpf_setsockopt(). From a quick look attcp_abortandudp_abort, I don't see they might sleep also and you may want to double check. Even '.diag_destroy' was only usable in sleepable 'bpf-iter' because it might sleep, the common bpf map types are already available to the sleepable programs.At the kernel side, the tcp and unix iter acquire the
lock_sock()first (eg. inbpf_iter_tcp_seq_show()) before calling the bpf-iter prog . At the kernel setsockopt code (eg.do_ip_setsockopt()), it usessockopt_lock_sock()and avoids doing the lock if it has already been guaranteed by the bpf running context.For udp, I don't see how the
udp_abortacquires the sock lock differently from tcp_abort. I assume the actual problem seen inudp_abortis related to the '->unhash()' part which acquires the udp_table's bucket lock. This is a problem for udp bpf-iter only because the udp bpf-iter did not release the udp_table's bucket lock before calling the bpf prog. The tcp (and unix) bpf-iter releases the bucket lock first before calling the bpf prog. This was done explicitly to allow acquiring the sock lock before calling the bpf prog because otherwise it will have a lock ordering issue. Hence, for this reason,bpf_setsockopt()is only available to tcp and unix bpf-iter but not udp bpf-iter. The udp-iter needs to do similar change like the tcp and unix iter (https://lore.kernel.org/bpf/[email protected]/): batch, release the bucket's lock, lock the sock, and then call bpf prog. This will allow udp-iter to callbpf_setsockopt()like its tcp and unix counterpart. That will also allow udp bpf-iter prog to directly do '.diag_destroy'.
Agree, that sounds very reasonable way forward and would also enable bpf_setsockopt() support for UDP sks. Wrt the workqueue I don't really like that there's a loophole with the -EBUSY if the callback didn't trigger yet, while we need to be able to destroy multiple sks in one iteration when they match the sk cookie previously recorded in the map.. so agree, this needs to happen in a synchronous manner.
Thanks,
Daniel
Re: [PATCH 1/2] bpf: Add socket destroy capability
In reply to Martin KaFai LauThanks for the reviews.
@martin: Thanks, I'll look into the missing batching related logic for UDP. Appreciate the pointers.
OT: While we can use the has_current_bpf_ctx macro to skip taking sock locks in the BPF context, how is the assumption around locking enforced in the code (e.g., setsockopt helper assumes that BPF iter prog acquires the relevant locks)? We could potentially use lockdep_sock_is_held for the sock lock. It's a debug configuration, but I suppose it's enabled in selftests.
Quoted 36 lines · Martin KaFai Lau
Quoted 19 lines · Aditi Ghag
The socket destroy helper is used to forcefully terminate sockets from certain BPF contexts. We plan to use the capability in Cilium to force client sockets to reconnect when their remote load-balancing backends are deleted. The other use case is on-the-fly policy enforcement where existing socket connections prevented by policies need to be terminated.
The helper is currently exposed to iterator type BPF programs where users can filter, and terminate a set of sockets.
Sockets are destroyed asynchronously using the work queue infrastructure. This allows for current the locking semantics within socket destroy handlers, as BPF iterators invoking the helper acquire *sock* locks. This also allows the helper to be invoked from non-sleepable contexts.Quoted 8 lines
The other approach to skip acquiring locks by passing an argument to the
diag_destroyhandler didn't work out well for UDP, as the UDP abort function internally invokes another function that ends up acquiring *sock* lock.
While there are sleepable BPF iterators, these are limited to only certain map types.bpf-iter program can be sleepable and non sleepable. Both sleepable and non sleepable tcp/unix bpf-iter programs have been able to call
bpf_setsockopt()synchronously.bpf_setsockopt()also requires the sock lock to be held first. The situation on calling '.diag_destroy' from bpf-iter should not be much different from callingbpf_setsockopt(). From a quick look attcp_abortandudp_abort, I don't see they might sleep also and you may want to double check. Even '.diag_destroy' was only usable in sleepable 'bpf-iter' because it might sleep, the common bpf map types are already available to the sleepable programs.At the kernel side, the tcp and unix iter acquire the
lock_sock()first (eg. inbpf_iter_tcp_seq_show()) before calling the bpf-iter prog . At the kernel setsockopt code (eg.do_ip_setsockopt()), it usessockopt_lock_sock()and avoids doing the lock if it has already been guaranteed by the bpf running context.For udp, I don't see how the
udp_abortacquires the sock lock differently fromtcp_abort. I assume the actual problem seen inudp_abortis related to the '->unhash()' part which acquires the udp_table's bucket lock. This is a problem for udp bpf-iter only because the udp bpf-iter did not release the udp_table's bucket lock before calling the bpf prog. The tcp (and unix) bpf-iter releases the bucket lock first before calling the bpf prog. This was done explicitly to allow acquiring the sock lock before calling the bpf prog because otherwise it will have a lock ordering issue. Hence, for this reason,bpf_setsockopt()is only available to tcp and unix bpf-iter but not udp bpf-iter. The udp-iter needs to do similar change like the tcp and unix iter (https://lore.kernel.org/bpf/[email protected]/): batch, release the bucket's lock, lock the sock, and then call bpf prog. This will allow udp-iter to callbpf_setsockopt()like its tcp and unix counterpart. That will also allow udp bpf-iter prog to directly do '.diag_destroy'.
Martin KaFai Lau
reviewerRe: [PATCH 1/2] bpf: Add socket destroy capability
In reply to a message not in the archiveQuoted 4 lines · Aditi Ghag
OT: While we can use the
has_current_bpf_ctxmacro to skip taking sock locks in the BPF context, how is the assumption around locking enforced in the code (e.g., setsockopt helper assumes that BPF iter prog acquires the relevant locks)?
The lock condition is enforced by the verifier when loading bpf prog. The bpf_setsockopt helper is only available to the bpf hooks that have already held/owned the sock lock. For example, if a bpf-tc prog tries to call bpf_setsockopt(), the verifier will reject loading the prog because the bpf-tc hook does not hold/own the sock lock.
Quoted 1 line
We could potentially use
lockdep_sock_is_heldfor the sock lock. It's a debug configuration, but I suppose it's enabled in selftests.
Not sure I totally get it. Meaning adding a sock_owned_by_me() check somewhere in the new bpf_sk_destroy() kfunc for debug purpose? hmm...yeah, why not but not sure how that may look like. I think it is some minor implementation details and will be easier to comment with code.
[ btw, please reply inline instead of top posting. ]
Quoted 87 lines
Quoted 82 lines · Martin KaFai Lau
Quoted 14 lines · Aditi Ghag
The socket destroy helper is used to forcefully terminate sockets from certain BPF contexts. We plan to use the capability in Cilium to force client sockets to reconnect when their remote load-balancing backends are deleted. The other use case is on-the-fly policy enforcement where existing socket connections prevented by policies need to be terminated.
The helper is currently exposed to iterator type BPF programs where users can filter, and terminate a set of sockets.
Quoted 15 lines
Sockets are destroyed asynchronously using the work queue infrastructure. This allows for current the locking semantics within socket destroy handlers, as BPF iterators invoking the helper acquire *sock* locks. This also allows the helper to be invoked from non-sleepable contexts.
The other approach to skip acquiring locks by passing an argument to thediag_destroyhandler didn't work out well for UDP, as the UDP abort function internally invokes another function that ends up acquiring *sock* lock.
While there are sleepable BPF iterators, these are limited to only certain map types.bpf-iter program can be sleepable and non sleepable. Both sleepable and non
sleepable tcp/unix bpf-iter programs have been able to callbpf_setsockopt()
synchronously.bpf_setsockopt()also requires the sock lock to be held first.
The situation on calling '.diag_destroy' from bpf-iter should not be much different from callingbpf_setsockopt(). From a quick look attcp_abortandudp_abort, I don't see they might sleep also and you may want to double check.
Even '.diag_destroy' was only usable in sleepable 'bpf-iter' because it might
sleep, the common bpf map types are already available to the sleepable programs.At the kernel side, the tcp and unix iter acquire the
lock_sock()first (eg. inbpf_iter_tcp_seq_show()) before calling the bpf-iter prog . At the kernel setsockopt code (eg.do_ip_setsockopt()), it usessockopt_lock_sock()and avoids
doing the lock if it has already been guaranteed by the bpf running context.For udp, I don't see how the
udp_abortacquires the sock lock differently fromtcp_abort. I assume the actual problem seen inudp_abortis related to the
'->unhash()' part which acquires the udp_table's bucket lock. This is a problem
for udp bpf-iter only because the udp bpf-iter did not release the udp_table's
bucket lock before calling the bpf prog. The tcp (and unix) bpf-iter releases
the bucket lock first before calling the bpf prog. This was done explicitly to
allow acquiring the sock lock before calling the bpf prog because otherwise it
will have a lock ordering issue. Hence, for this reason,bpf_setsockopt()is
only available to tcp and unix bpf-iter but not udp bpf-iter. The udp-iter needs
to do similar change like the tcp and unix iter
(https://lore.kernel.org/bpf/[email protected]/):
batch,
release the bucket's lock, lock the sock, and then call bpf prog. This will
allow udp-iter to callbpf_setsockopt()like its tcp and unix counterpart. That
will also allow udp bpf-iter prog to directly do '.diag_destroy'.
Re: [PATCH 1/2] bpf: Add socket destroy capability
In reply to [email protected]Quoted 217 lines · On Dec 19, 2022, at 10:22 AM, [email protected] wrote:
Quoted 10 lines · Aditi Ghag
The socket destroy helper is used to forcefully terminate sockets from certain BPF contexts. We plan to use the capability in Cilium to force client sockets to reconnect when their remote load-balancing backends are deleted. The other use case is on-the-fly policy enforcement where existing socket connections prevented by policies need to be terminated.
Quoted 7 lines
The helper is currently exposed to iterator type BPF programs where users can filter, and terminate a set of sockets.
Sockets are destroyed asynchronously using the work queue infrastructure. This allows for current the locking semantics within
s/current the/the current/ ?
Quoted 27 lines
socket destroy handlers, as BPF iterators invoking the helper acquire *sock* locks. This also allows the helper to be invoked from non-sleepable contexts.
The other approach to skip acquiring locks by passing an argument to thediag_destroyhandler didn't work out well for UDP, as the UDP abort function internally invokes another function that ends up acquiring *sock* lock.
While there are sleepable BPF iterators, these are limited to only certain map types. Furthermore, it's limiting in the sense that it wouldn't allow us to extend the helper to other non-sleepable BPF programs.The work queue infrastructure processes work items from per-cpu structures. As the sock destroy work items are executed asynchronously, we need to ref count sockets before they are added to the work queue. The '
work_pending' check prevents duplicate ref counting of sockets in case users invoke the destroy helper for a socket multiple times. The{READ,WRITE}_ONCEmacros ensure that the socket pointer stored in a work queue item isn't clobbered while the item is being processed. As BPF programsQuoted 8 lines
are non-preemptible, we can expect that once a socket is ref counted, no other socket can sneak in before the ref counted socket is added to the work queue for asynchronous destroy. Finally, users are expected to retry when the helper fails to queue a work item for a socket to be destroyed in case there is another destroy operation is in progress.
nit: maybe reformat to fit into 80 characters per line? A bit hard to read with this narrow formatting..
Quoted 30 lines
Signed-off-by Aditi Ghag <[email protected]>--- include/linux/bpf.h | 1 + include/uapi/linux/bpf.h | 17 +++++++++ kernel/bpf/core.c | 1 + kernel/trace/bpf_trace.c | 2 + net/core/filter.c | 70 ++++++++++++++++++++++++++++++++++ tools/include/uapi/linux/bpf.h | 17 +++++++++ 6 files changed, 108 insertions(+)include/linux/bpf.h +1
-0@@ -2676,6 +2676,7 @@ extern const struct bpf_func_proto bpf_get_retval_proto;extern const struct bpf_func_proto bpf_user_ringbuf_drain_proto;extern const struct bpf_func_proto bpf_cgrp_storage_get_proto;extern const struct bpf_func_proto bpf_cgrp_storage_delete_proto; +extern const struct bpf_func_proto bpf_sock_destroy_proto;const struct bpf_func_proto *tracing_prog_func_proto (enum bpf_func_id func_id,const struct bpf_prog *prog);include/uapi/linux/bpf.h +0
-0@@ -5484,6 +5484,22 @@ union bpf_attr {* 0 on success. * * **-ENOENT** if the bpf_local_storage cannot be found. include/uapi/linux/bpf.h union bpf_attr { · line 5496
Quoted 10 lines
include/uapi/linux/bpf.h +10
-0+* +* int bpf_sock_destroy(struct sock *sk) +* Description +* Destroy the given socket with **ECONNABORTED** error code. +* +* *sk* must be a non-**NULL** pointer to a socket. +* +* Return +* The socket is destroyed asynchronosuly, so 0 return value may +* not suggest indicate that the socket was successfully destroyed. s/suggest indicate/ with either suggest or indicate?
Quoted 99 lines
include/uapi/linux/bpf.h +7
-0+* +* On error, may return **EPROTONOSUPPORT**, **EBUSY**, **EINVAL**. +* +* **-EPROTONOSUPPORT** if protocol specific destroy handler is not implemented. +* +* **-EBUSY** if another socket destroy operation is in progress. */ #define ___BPF_FUNC_MAPPER(FN, ctx...) \ FN(unspec, 0, ##ctx) \@@ -5698,6 +5714,7 @@ union bpf_attr { FN(user_ringbuf_drain, 209, ##ctx) \ FN(cgrp_storage_get, 210, ##ctx) \ FN(cgrp_storage_delete, 211, ##ctx) \ + FN(sock_destroy, 212, ##ctx) \/* */ /* backwards-compatibility macros for users of __BPF_FUNC_MAPPER that don't kernel/bpf/core.c +1
-0@@ -2651,6 +2651,7 @@ const struct bpf_func_proto bpf_snprintf_btf_proto __weak;const struct bpf_func_proto bpf_seq_printf_btf_proto__weak ;const struct bpf_func_proto bpf_set_retval_proto__weak ;const struct bpf_func_proto bpf_get_retval_proto__weak ; +const struct bpf_func_proto bpf_sock_destroy_proto__weak ;const struct bpf_func_proto *__weak bpf_get_trace_printk_proto (void ) {kernel/trace/bpf_trace.c +2
-0@@ -1930,6 +1930,8 @@ tracing_prog_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)return &bpf_get_socket_ptr_cookie_proto;case BPF_FUNC_xdp_get_buff_len:return &bpf_xdp_get_buff_len_trace_proto; +case BPF_FUNC_sock_destroy: +return &bpf_sock_destroy_proto;#endif case BPF_FUNC_seq_printf: return prog->expected_attach_type == BPF_TRACE_ITER ?net/core/filter.c +42
-0@@ -11569,6 +11569,8 @@ bpf_sk_base_func_proto(enum bpf_func_id func_id)break ;case BPF_FUNC_ktime_get_coarse_ns:return &bpf_ktime_get_coarse_ns_proto; +case BPF_FUNC_sock_destroy: +return &bpf_sock_destroy_proto;default :return bpf_base_func_proto (func_id); }@@ -11578,3 +11580,71 @@ bpf_sk_base_func_proto(enum bpf_func_id func_id)return func; } + +struct sock_destroy_work { +struct sock *sk; +struct work_struct destroy; +}; + +static DEFINE_PER_CPU(struct sock_destroy_work, sock_destroy_workqueue); + +static void bpf_sock_destroy_fn (struct work_struct *work) +{ +struct sock_destroy_work *sd_work =container_of (work, + struct sock_destroy_work, destroy); +struct sock *sk =READ_ONCE (sd_work->sk); + + sk->sk_prot->diag_destroy (sk,ECONNABORTED ); +sock_put (sk); +} + +static int __init bpf_sock_destroy_workqueue_init (void ) +{ +int cpu; +struct sock_destroy_work *work; + +for_each_possible_cpu (cpu) { + work =per_cpu_ptr (&sock_destroy_workqueue, cpu); +INIT_WORK (&work->destroy, bpf_sock_destroy_fn); + } + +return 0 ; +} +subsys_initcall (bpf_sock_destroy_workqueue_init); + +BPF_CALL_1(bpf_sock_destroy, struct sock *, sk) +{ +struct sock_destroy_work *sd_work; + +if (!sk->sk_prot->diag_destroy) +return -EOPNOTSUPP ; + + sd_work =this_cpu_ptr (&sock_destroy_workqueue);[..]
net/core/filter.c bpf_sk_base_func_proto(enum bpf_func_id func_id) · line 11626
Quoted 4 lines
net/core/filter.c +4
-0+/* This check prevents duplicate ref counting +* of sockets, in case the handler is invoked +* multiple times for the same socket. +*/ This means this helper can also be called for a single socket during invocation; is it an ok compromise?
Quoted 2 lines
I'm also assuming it's still possible that this helper gets called for the same socket on different cpus?
Thanks for the review. The v2 patch series refactored this code path significantly, and it executes the destroy handlers synchronously.
Quoted 62 lines
Quoted 62 lines
net/core/filter.c +24
-0+if (work_pending (&sd_work->destroy)) +return -EBUSY ; + +/* Ref counting ensures that the socket +* isn't deleted from underneath us before +* the work queue item is processed. +*/ +if (!refcount_inc_not_zero (&sk->sk_refcnt)) +return -EINVAL ; + +WRITE_ONCE (sd_work->sk, sk); +if (!queue_work (system_wq, &sd_work->destroy)) { +sock_put (sk); +return -EBUSY ; + } + +return 0 ; +} + +const struct bpf_func_proto bpf_sock_destroy_proto = { + .func = bpf_sock_destroy, + .ret_type =RET_INTEGER , + .arg1_type =ARG_PTR_TO_BTF_ID_SOCK_COMMON , +};tools/include/uapi/linux/bpf.h +17
-1@@ -5484,6 +5484,22 @@ union bpf_attr {* 0 on success. * * **-ENOENT** if the bpf_local_storage cannot be found. +* +* int bpf_sock_destroy(void *sk) +* Description +* Destroy the given socket with **ECONNABORTED** error code. +* +* *sk* must be a non-**NULL** pointer to a socket. +* +* Return +* The socket is destroyed asynchronosuly, so 0 return value may +* not indicate that the socket was successfully destroyed. +* +* On error, may return **EPROTONOSUPPORT**, **EBUSY**, **EINVAL**. +* +* **-EPROTONOSUPPORT** if protocol specific destroy handler is not implemented. +* +* **-EBUSY** if another socket destroy operation is in progress. */ #define ___BPF_FUNC_MAPPER(FN, ctx...) \ FN(unspec, 0, ##ctx) \@@ -5698,6 +5714,7 @@ union bpf_attr { FN(user_ringbuf_drain, 209, ##ctx) \ FN(cgrp_storage_get, 210, ##ctx) \ FN(cgrp_storage_delete, 211, ##ctx) \ + FN(sock_destroy, 212, ##ctx) \/* */ /* backwards-compatibility macros for users of __BPF_FUNC_MAPPER that don't -- 2.34.1
Re: [PATCH 1/2] bpf: Add socket destroy capability
In reply to Martin KaFai LauQuoted 32 lines · Martin KaFai Lau
Quoted 19 lines · Aditi Ghag
The socket destroy helper is used to forcefully terminate sockets from certain BPF contexts. We plan to use the capability in Cilium to force client sockets to reconnect when their remote load-balancing backends are deleted. The other use case is on-the-fly policy enforcement where existing socket connections prevented by policies need to be terminated.
The helper is currently exposed to iterator type BPF programs where users can filter, and terminate a set of sockets.
Sockets are destroyed asynchronously using the work queue infrastructure. This allows for current the locking semantics within socket destroy handlers, as BPF iterators invoking the helper acquire *sock* locks. This also allows the helper to be invoked from non-sleepable contexts.Quoted 8 lines
The other approach to skip acquiring locks by passing an argument to the
diag_destroyhandler didn't work out well for UDP, as the UDP abort function internally invokes another function that ends up acquiring *sock* lock.
While there are sleepable BPF iterators, these are limited to only certain map types.
Quoted 5 lines
bpf-iter program can be sleepable and non sleepable. Both sleepable and non sleepable tcp/unix bpf-iter programs have been able to call
bpf_setsockopt()synchronously.bpf_setsockopt()also requires the sock lock to be held first. The situation on calling '.diag_destroy' from bpf-iter should not be much different from callingbpf_setsockopt(). From a quick look attcp_abortandudp_abort, I don't see they might sleep also and you may want to double check. Even '.diag_destroy' was only usable in sleepable 'bpf-iter' because it might sleep, the common bpf map types are already available to the sleepable programs.At the kernel side, the tcp and unix iter acquire the
lock_sock()first (eg. inbpf_iter_tcp_seq_show()) before calling the bpf-iter prog . At the kernel setsockopt code (eg.do_ip_setsockopt()), it usessockopt_lock_sock()and avoids doing the lock if it has already been guaranteed by the bpf running context.For udp, I don't see how the
udp_abortacquires the sock lock differently fromtcp_abort. I assume the actual problem seen inudp_abortis related to the '->unhash()' part which acquires the udp_table's bucket lock. This is a problem for udp bpf-iter only because the udp bpf-iter did not release the udp_table's bucket lock before calling the bpf prog. The tcp (and unix) bpf-iter releases the bucket lock first before calling the bpf prog. This was done explicitly to allow acquiring the sock lock before calling the bpf prog because otherwise it will have a lock ordering issue. Hence, for this reason,bpf_setsockopt()is only available to tcp and unix bpf-iter but not udp bpf-iter. The udp-iter needs to do similar change like the tcp and unix iter (https://lore.kernel.org/bpf/[email protected]/): batch, release the bucket's lock, lock the sock, and then call bpf prog. This will allow udp-iter to callbpf_setsockopt()like its tcp and unix counterpart. That will also allow udp bpf-iter prog to directly do '.diag_destroy'.
Pushed v2 patch series that implements the missing batching support for UDP iterators. Sorry for the delay. Thanks for the well-documented batching code in TCP!
Re: [PATCH 1/2] bpf: Add socket destroy capability
In reply to Alan MaguireQuoted 144 lines · Alan Maguire
Quoted 140 lines · Aditi Ghag
The socket destroy helper is used to forcefully terminate sockets from certain BPF contexts. We plan to use the capability in Cilium to force client sockets to reconnect when their remote load-balancing backends are deleted. The other use case is on-the-fly policy enforcement where existing socket connections prevented by policies need to be terminated.
The helper is currently exposed to iterator type BPF programs where users can filter, and terminate a set of sockets.
Sockets are destroyed asynchronously using the work queue infrastructure. This allows for current the locking semantics within socket destroy handlers, as BPF iterators invoking the helper acquire *sock* locks. This also allows the helper to be invoked from non-sleepable contexts.
The other approach to skip acquiring locks by passing an argument to thediag_destroyhandler didn't work out well for UDP, as the UDP abort function internally invokes another function that ends up acquiring *sock* lock.
While there are sleepable BPF iterators, these are limited to only certain map types. Furthermore, it's limiting in the sense that it wouldn't allow us to extend the helper to other non-sleepable BPF programs.The work queue infrastructure processes work items from per-cpu structures. As the sock destroy work items are executed asynchronously, we need to ref count sockets before they are added to the work queue. The '
work_pending' check prevents duplicate ref counting of sockets in case users invoke the destroy helper for a socket multiple times. The{READ,WRITE}_ONCEmacros ensure that the socket pointer stored in a work queue item isn't clobbered while the item is being processed. As BPF programs are non-preemptible, we can expect that once a socket is ref counted, no other socket can sneak in before the ref counted socket is added to the work queue for asynchronous destroy. Finally, users are expected to retry when the helper fails to queue a work item for a socket to be destroyed in case there is another destroy operation is in progress.Signed-off-by Aditi Ghag <[email protected]>---
include/linux/bpf.h | 1 + include/uapi/linux/bpf.h | 17 +++++++++ kernel/bpf/core.c | 1 + kernel/trace/bpf_trace.c | 2 + net/core/filter.c | 70 ++++++++++++++++++++++++++++++++++ tools/include/uapi/linux/bpf.h | 17 +++++++++ 6 files changed, 108 insertions(+)include/linux/bpf.h +1
-0@@ -2676,6 +2676,7 @@ extern const struct bpf_func_proto bpf_get_retval_proto;extern const struct bpf_func_proto bpf_user_ringbuf_drain_proto;extern const struct bpf_func_proto bpf_cgrp_storage_get_proto;extern const struct bpf_func_proto bpf_cgrp_storage_delete_proto; +extern const struct bpf_func_proto bpf_sock_destroy_proto;const struct
bpf_func_proto*tracing_prog_func_proto(
enumbpf_func_idfunc_id, const structbpf_prog*prog);include/uapi/linux/bpf.h +17
-0@@ -5484,6 +5484,22 @@ union bpf_attr {* 0 on success. * * **-ENOENT** if the bpf_local_storage cannot be found. +* +* int bpf_sock_destroy(struct sock *sk) +* Description +* Destroy the given socket with **ECONNABORTED** error code. +* +* *sk* must be a non-**NULL** pointer to a socket. +* +* Return +* The socket is destroyed asynchronosuly, so 0 return value may +* not suggest indicate that the socket was successfully destroyed. +* +* On error, may return **EPROTONOSUPPORT**, **EBUSY**, **EINVAL**. +* +* **-EPROTONOSUPPORT** if protocol specific destroy handler is not implemented. +* +* **-EBUSY** if another socket destroy operation is in progress. */ #define ___BPF_FUNC_MAPPER(FN, ctx...) \ FN(unspec, 0, ##ctx) \@@ -5698,6 +5714,7 @@ union bpf_attr { FN(user_ringbuf_drain, 209, ##ctx) \ FN(cgrp_storage_get, 210, ##ctx) \ FN(cgrp_storage_delete, 211, ##ctx) \ + FN(sock_destroy, 212, ##ctx) \/* */ /* backwards-compatibility macros for users of __BPF_FUNC_MAPPER that don't kernel/bpf/core.c +1
-0@@ -2651,6 +2651,7 @@ const struct bpf_func_proto bpf_snprintf_btf_proto __weak;const struct bpf_func_proto bpf_seq_printf_btf_proto__weak ;const struct bpf_func_proto bpf_set_retval_proto__weak ;const struct bpf_func_proto bpf_get_retval_proto__weak ; +const struct bpf_func_proto bpf_sock_destroy_proto__weak ;const struct bpf_func_proto *__weak bpf_get_trace_printk_proto (void ) {kernel/trace/bpf_trace.c +2
-0@@ -1930,6 +1930,8 @@ tracing_prog_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)return &bpf_get_socket_ptr_cookie_proto;case BPF_FUNC_xdp_get_buff_len:return &bpf_xdp_get_buff_len_trace_proto; +case BPF_FUNC_sock_destroy: +return &bpf_sock_destroy_proto;#endif case BPF_FUNC_seq_printf: return prog->expected_attach_type == BPF_TRACE_ITER ?net/core/filter.c +0
-0@@ -11569,6 +11569,8 @@ bpf_sk_base_func_proto(enum bpf_func_id func_id)
Quoted 8 lines
net/core/filter.c bpf_sk_base_func_proto(enum bpf_func_id func_id) · line 11574
Quoted 6 lines
net/core/filter.c +2
-0break ;case BPF_FUNC_ktime_get_coarse_ns:return &bpf_ktime_get_coarse_ns_proto; +case BPF_FUNC_sock_destroy: +return &bpf_sock_destroy_proto;default :This is a really neat feature! One question though; above it seems to suggest that the helper is only exposed to BPF iterators, but by
The v2 patch only exposes the capability to BPF iterators. The patch series has details involving locking semantics due to which the kfunc is only available for BPF iterators at the moment.
Quoted 56 lines
adding it to
bpf_sk_base_func_proto()won't it be available to other BPF programs likesock_addr,sk_filteretc? If I've got that right, we'd definitely want to exclude potentially unprivileged programs from having access to this helper. And to be clear, I'd definitely see value in having it accessible to other BPF program types too like sockops if possible, but just wanted to check.Quoted 33 lines
net/core/filter.c +27
-0return bpf_base_func_proto (func_id); }@@ -11578,3 +11580,71 @@ bpf_sk_base_func_proto(enum bpf_func_id func_id)return func; } + +struct sock_destroy_work { +struct sock *sk; +struct work_struct destroy; +}; + +static DEFINE_PER_CPU(struct sock_destroy_work, sock_destroy_workqueue); + +static void bpf_sock_destroy_fn (struct work_struct *work) +{ +struct sock_destroy_work *sd_work =container_of (work, + struct sock_destroy_work, destroy); +struct sock *sk =READ_ONCE (sd_work->sk); + + sk->sk_prot->diag_destroy (sk,ECONNABORTED ); +sock_put (sk); +} + +static int __init bpf_sock_destroy_workqueue_init (void ) +{ +int cpu; +struct sock_destroy_work *work; + +for_each_possible_cpu (cpu) { + work =per_cpu_ptr (&sock_destroy_workqueue, cpu); +INIT_WORK (&work->destroy, bpf_sock_destroy_fn); + }net/core/filter.c bpf_sk_base_func_proto(enum bpf_func_id func_id) · line 11619
Quoted 10 lines
net/core/filter.c +10
-0+ +return 0 ; +} +subsys_initcall (bpf_sock_destroy_workqueue_init); + +BPF_CALL_1(bpf_sock_destroy, struct sock *, sk) +{ +struct sock_destroy_work *sd_work; + +if (!sk->sk_prot->diag_destroy)risk of a NULL sk here?
Thanks!
Alan
Re: [PATCH 2/2] selftests/bpf: Add tests for bpf_sock_destroy
In reply to [email protected]Quoted 53 lines · On Dec 19, 2022, at 10:25 AM, [email protected] wrote:
Quoted 49 lines · Aditi Ghag
The test cases for TCP and UDP mirror the
intended usages of the helper.As the helper destroys sockets asynchronously,
the tests have sleep invocation before validating
if the sockets were destroyed by sending data.Also, while all the protocol specific helpers set
ECONNABORTEDerror code on the destroyed sockets, only the TCP test case has the validation check. UDP sockets have an overriding error code from the disconnect call during abort.Signed-off-by Aditi Ghag <[email protected]>--- .../selftests/bpf/prog_tests/sock_destroy.c | 131 ++++++++++++++++++ .../selftests/bpf/progs/sock_destroy_prog.c | 96 +++++++++++++ 2 files changed, 227 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/sock_destroy.c create mode 100644 tools/testing/selftests/bpf/progs/sock_destroy_prog.ctools/testing/selftests/bpf/prog_tests/sock_destroy.c new file +22
-0@@ -0,0 +1,131 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <test_progs.h> + +#include "sock_destroy_prog.skel.h" +#include "network_helpers.h" + +#define ECONNABORTED 103 + +static int duration; + +static void start_iter_sockets (struct bpf_program *prog) +{ +struct bpf_link *link; +char buf[16 ] = {}; +int iter_fd, len; + + link =bpf_program__attach_iter (prog,NULL ); +if (!ASSERT_OK_PTR (link,"attach_iter" )) +return ; + + iter_fd =bpf_iter_create (bpf_link__fd (link)); +if (!ASSERT_GE (iter_fd,0 ,"create_iter" ))
Quoted 8 lines
tools/testing/selftests/bpf/prog_tests/sock_destroy.c line 27
Quoted 5 lines
tools/testing/selftests/bpf/prog_tests/sock_destroy.c new file +5
-0+ goto free_link; + +while ((len =read (iter_fd, buf,sizeof (buf))) >0 ) + ; +CHECK (len <0 ,"read" ,"read failed: %s\n" ,strerror (errno));CHECK(s) are not super appealing in the new code, can you replace them all with ASSERT(s) ? This should also let you drop 'duration' variable.
Sorry, I missed this comment. I can look into replacing CHECK(s) with ASSERT(s) as a follow-up. Will wait for reviews on patch v2 for now.
Quoted 35 lines
Quoted 23 lines
tools/testing/selftests/bpf/prog_tests/sock_destroy.c new file +22
-0+ +close (iter_fd); + +free_link: +bpf_link__destroy (link); +} + +void test_tcp (struct sock_destroy_prog *skel) +{ +int serv =-1 , clien =-1 , n =0 ; + + serv =start_server (AF_INET6 ,SOCK_STREAM ,NULL ,0 ,0 ); +if (CHECK (serv <0 ,"start_server" ,"failed to start server\n" )) + goto cleanup_serv; + + clien =connect_to_fd (serv,0 ); +if (CHECK (clien <0 ,"connect_to_fd" ,"errno %d\n" , errno)) + goto cleanup_serv; + + serv =accept (serv,NULL ,NULL ); +if (CHECK (serv <0 ,"accept" ,"errno %d\n" , errno)) + goto cleanup;tools/testing/selftests/bpf/prog_tests/sock_destroy.c line 59
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/sock_destroy.c new file +10
-0+ + n =send (clien,"t" ,1 ,0 ); +if (CHECK (n <0 ,"client_send" ,"client failed to send on socket\n" )) + goto cleanup; + +start_iter_sockets (skel->progs.iter_tcp6); + +// Sockets are destroyed asynchronously. +usleep (1000 ); + n =send (clien,"t" ,1 ,0 );That's racy. Do some kind of:
Quoted 8 lines
while (retries--) {
usleep(100);
n = send (...)
if (n < 0)
break;
}
ASSERT_LT(n, 0);
ASSERT_EQ(errno, ECONNABORTED);This isn't applicable anymore as sockets are destroyed synchronously in the latest patches.
Quoted 180 lines
Quoted 24 lines
tools/testing/selftests/bpf/prog_tests/sock_destroy.c new file +23
-0+ +if (CHECK (n >0 ,"client_send" ,"succeeded on destroyed socket\n" )) + goto cleanup; +CHECK (errno !=ECONNABORTED ,"client_send" ,"unexpected error code on destroyed socket\n" ); + + +cleanup: +close (clien); +cleanup_serv: +close (serv); +} + + +void test_udp (struct sock_destroy_prog *skel) +{ +int serv =-1 , clien =-1 , n =0 ; + + serv =start_server (AF_INET6 ,SOCK_DGRAM ,NULL ,0 ,0 ); +if (CHECK (serv <0 ,"start_server" ,"failed to start server\n" )) + goto cleanup_serv; + + clien =connect_to_fd (serv,0 ); +if (CHECK (clien <0 ,"connect_to_fd" ,"errno %d\n" , errno))tools/testing/selftests/bpf/prog_tests/sock_destroy.c line 92
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/sock_destroy.c new file +10
-0+ goto cleanup_serv; + + n =send (clien,"t" ,1 ,0 ); +if (CHECK (n <0 ,"client_send" ,"client failed to send on socket\n" )) + goto cleanup; + +start_iter_sockets (skel->progs.iter_udp6); + +// Sockets are destroyed asynchronously. +usleep (1000 );Same here.
Quoted 143 lines
tools/testing/selftests/bpf/prog_tests/sock_destroy.c new file +39
-0+ + n =send (clien,"t" ,1 ,0 ); +if (CHECK (n >0 ,"client_send" ,"succeeded on destroyed socket\n" )) + goto cleanup; +// UDP sockets have an overriding error code after they are disconnected. + + +cleanup: +close (clien); +cleanup_serv: +close (serv); +} + +void test_sock_destroy (void ) +{ +int cgroup_fd =0 ; +struct sock_destroy_prog *skel; + + skel =sock_destroy_prog__open_and_load (); +if (!ASSERT_OK_PTR (skel,"skel_open" )) +return ; + + cgroup_fd =test__join_cgroup ("/sock_destroy" ); +if (CHECK (cgroup_fd <0 ,"join_cgroup" ,"cgroup creation failed\n" )) + goto close_cgroup_fd; + + skel->links.sock_connect =bpf_program__attach_cgroup ( + skel->progs.sock_connect, cgroup_fd); +if (!ASSERT_OK_PTR (skel->links.sock_connect,"prog_attach" )) + goto close_cgroup_fd; + +test_tcp (skel); +test_udp (skel); + + +close_cgroup_fd: +close (cgroup_fd); +sock_destroy_prog__destroy (skel); +}tools/testing/selftests/bpf/progs/sock_destroy_prog.c new file +96
-1@@ -0,0 +1,96 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include "vmlinux.h" + +#include <bpf/bpf_helpers.h> + +#define AF_INET6 10 + +struct { + __uint(type, BPF_MAP_TYPE_ARRAY); + __uint(max_entries, 1); + __type(key, __u32); + __type(value, __u64); +} tcp_conn_sockets SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_ARRAY); + __uint(max_entries, 1); + __type(key, __u32); + __type(value, __u64); +} udp_conn_sockets SEC(".maps"); + +SEC("cgroup/connect6") +int sock_connect (struct bpf_sock_addr *ctx) +{ +int key =0 ; +__u64 sock_cookie =0 ; +__u32 keyc =0 ; + +if (ctx->family !=AF_INET6 || ctx->user_family !=AF_INET6 ) +return 1 ; + + sock_cookie =bpf_get_socket_cookie (ctx); +if (ctx->protocol ==IPPROTO_TCP ) +bpf_map_update_elem (&tcp_conn_sockets, &key, &sock_cookie,0 ); +else if (ctx->protocol ==IPPROTO_UDP ) +bpf_map_update_elem (&udp_conn_sockets, &keyc, &sock_cookie,0 ); +else +return 1 ; + +return 1 ; +} + +SEC("iter/tcp") +int iter_tcp6 (struct bpf_iter__tcp *ctx) +{ +struct sock_common *sk_common = ctx->sk_common; +struct seq_file *seq = ctx->meta->seq; +__u64 sock_cookie =0 ; +__u64 *val; +int key =0 ; + +if (!sk_common) +return 0 ; + +if (sk_common->skc_family !=AF_INET6 ) +return 0 ; + + sock_cookie =bpf_get_socket_cookie (sk_common); + val =bpf_map_lookup_elem (&tcp_conn_sockets, &key); + +if (!val) +return 0 ; + +if (sock_cookie == *val) +bpf_sock_destroy (sk_common); + +return 0 ; +} + +SEC("iter/udp") +int iter_udp6 (struct bpf_iter__udp *ctx) +{ +struct seq_file *seq = ctx->meta->seq; +struct udp_sock *udp_sk = ctx->udp_sk; +struct sock *sk = (struct sock *) udp_sk; +__u64 sock_cookie =0 ; +int key =0 ; +__u64 *val; + +if (!sk) +return 0 ; + + sock_cookie =bpf_get_socket_cookie (sk); + val =bpf_map_lookup_elem (&udp_conn_sockets, &key); + +if (!val) +return 0 ; + +if (sock_cookie == *val) +bpf_sock_destroy (sk); + +return 0 ; +} + +char _license[] SEC("license") = "GPL"; --2.34.1