Support bpf trampoline for s390x
Subsystems (10)
Ilya Leoshkevich
maintainerv2 cover Support bpf trampoline for s390x
v1: https://lore.kernel.org/bpf/[email protected]/#t
v1 -> v2:
- Fix core_read_macros, sk_assign, test_profiler, test_bpffs (24/31; I'm not quite happy with the fix, but don't have better ideas), and xdp_synproxy. (Andrii)
- Prettify liburandom_read and verify_pkcs7_sig fixes. (Andrii)
- Fix bpf_usdt_arg using barrier_var(); prettify barrier_var(). (Andrii)
- Change BPF_MAX_TRAMP_LINKS to enum and query it using BTF. (Andrii)
- Improve bpf_jit_supports_kfunc_call() description. (Alexei)
- Always check sign_extend() return value.
- Cc: Alexander Gordeev.
Hi,
This series implements poke, trampoline, kfunc, mixing subprogs and tailcalls, and fixes a number of tests on s390x.
The following failures still remain:
#82 get_stack_raw_tp:FAIL
get_stack_print_output:FAIL:user_stack corrupted user stack Known issue:
We cannot reliably unwind userspace on s390x without DWARF.
#101 ksyms_module:FAIL
address of kernel function bpf_testmod_test_mod_kfunc is out of range Known issue:
Kernel and modules are too far away from each other on s390x.
#190 stacktrace_build_id:FAIL
Known issue:
We cannot reliably unwind userspace on s390x without DWARF.
#281 xdp_metadata:FAIL
See patch 31.
None of these seem to be due to the new changes.
Best regards,
Ilya
Ilya Leoshkevich (31):
bpf: Use ARG_CONST_SIZE_OR_ZERO for 3rd argument of
bpf_tcp_raw_gen_syncookie_ipv{4,6}() bpf: Change BPF_MAX_TRAMP_LINKS to enum
selftests/bpf: Query BPF_MAX_TRAMP_LINKS using BTF selftests/bpf: Fix liburandom_read.so linker error selftests/bpf: Fix symlink creation error
selftests/bpf: Fix kfree_skb on s390x
selftests/bpf: Set errno when urand_spawn() fails selftests/bpf: Fix decap_sanity_ns cleanup
selftests/bpf: Fix verify_pkcs7_sig on s390x selftests/bpf: Fix xdp_do_redirect on s390x selftests/bpf: Fix cgrp_local_storage on s390x selftests/bpf: Check stack_mprotect() return value selftests/bpf: Increase SIZEOF_BPF_LOCAL_STORAGE_ELEM on s390x selftests/bpf: Add a sign-extension test for kfuncs selftests/bpf: Fix test_lsm on s390x
selftests/bpf: Fix test_xdp_adjust_tail_grow2 on s390x selftests/bpf: Fix vmlinux test on s390x
selftests/bpf: Fix sk_assign on s390x
selftests/bpf: Fix xdp_synproxy/tc on s390x selftests/bpf: Fix profiler on s390x
libbpf: Simplify barrier_var()
libbpf: Fix unbounded memory access in bpf_usdt_arg()
libbpf: Fix BPF_PROBE_READ{_STR}_INTO() on s390x
bpf: iterators: Split iterators.lskel.h into little- and big- endian
versions
bpf: btf: Add BTF_FMODEL_SIGNED_ARG flag
s390/bpf: Fix a typo in a comment
s390/bpf: Add expoline to tail calls
s390/bpf: Implement bpf_arch_text_poke()
s390/bpf: Implement arch_prepare_bpf_trampoline() s390/bpf: Implement bpf_jit_supports_subprog_tailcalls() s390/bpf: Implement bpf_jit_supports_kfunc_call()
arch/s390/net/bpf_jit_comp.c | 714 +++++++++++++++++-
include/linux/bpf.h | 12 +-
include/linux/btf.h | 15 +-
kernel/bpf/btf.c | 16 +-
kernel/bpf/preload/bpf_preload_kern.c | 6 +-
kernel/bpf/preload/iterators/Makefile | 12 +-
kernel/bpf/preload/iterators/README | 5 +-
.../iterators/iterators.lskel-big-endian.h | 419 ++++++++++
...skel.h => iterators.lskel-little-endian.h} | 0
net/bpf/test_run.c | 9 +
net/core/filter.c | 4 +-
tools/lib/bpf/bpf_core_read.h | 4 +-
tools/lib/bpf/bpf_helpers.h | 2 +-
tools/lib/bpf/usdt.bpf.h | 5 +-
tools/testing/selftests/bpf/Makefile | 5 +-
tools/testing/selftests/bpf/netcnt_common.h | 6 +-
.../selftests/bpf/prog_tests/bpf_cookie.c | 6 +-
.../bpf/prog_tests/cgrp_local_storage.c | 2 +-
.../selftests/bpf/prog_tests/decap_sanity.c | 2 +-
.../selftests/bpf/prog_tests/fexit_stress.c | 22 +-
.../selftests/bpf/prog_tests/kfree_skb.c | 2 +-
.../selftests/bpf/prog_tests/kfunc_call.c | 1 +
.../selftests/bpf/prog_tests/sk_assign.c | 5 +-
.../selftests/bpf/prog_tests/test_lsm.c | 3 +-
.../bpf/prog_tests/trampoline_count.c | 18 +-
tools/testing/selftests/bpf/prog_tests/usdt.c | 1 +
.../bpf/prog_tests/verify_pkcs7_sig.c | 3 +
.../bpf/prog_tests/xdp_adjust_tail.c | 7 +-
.../bpf/prog_tests/xdp_do_redirect.c | 4 +
.../selftests/bpf/progs/kfunc_call_test.c | 18 +
tools/testing/selftests/bpf/progs/lsm.c | 7 +-
.../selftests/bpf/progs/profiler.inc.h | 62 +-
.../selftests/bpf/progs/test_sk_assign.c | 24 +-
.../bpf/progs/test_verify_pkcs7_sig.c | 12 +-
.../selftests/bpf/progs/test_vmlinux.c | 4 +-
.../bpf/progs/test_xdp_adjust_tail_grow.c | 8 +-
.../selftests/bpf/progs/xdp_synproxy_kern.c | 2 +-
tools/testing/selftests/bpf/test_progs.c | 38 +
tools/testing/selftests/bpf/test_progs.h | 2 +
39 files changed, 1350 insertions(+), 137 deletions(-)
create mode 100644 kernel/bpf/preload/iterators/iterators.lskel-big-endian.h
rename kernel/bpf/preload/iterators/{iterators.lskel.h => iterators.lskel-little-endian.h} (100%)Signature · 1 line
2.39.1
Ilya Leoshkevich
maintainerPATCH v2 1/31 bpf: Use ARG_CONST_SIZE_OR_ZERO for 3rd argument of bpf_tcp_raw_gen_syncookie_ipv{4,6}()
These functions already check that th_len < sizeof(*th), and propagating the lower bound (th_len > 0) may be challenging in complex code, e.g. as is the case with xdp_synproxy test on s390x [1]. Switch to ARG_CONST_SIZE_OR_ZERO in order to make the verifier accept code where it cannot prove that th_len > 0.
[1] https://lore.kernel.org/bpf/CAEf4Bzb3uiSHtUbgVWmkWuJ5Sw1UZd4c_iuS4QXtUkXmTTtXuQ@mail.gmail.com/
--- net/core/filter.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
net/core/filter.c +2 -2
@@ -7532,7 +7532,7 @@ static const struct bpf_func_proto bpf_tcp_raw_gen_syncookie_ipv4_proto = { .arg1_type = ARG_PTR_TO_FIXED_SIZE_MEM, .arg1_size = sizeof(struct iphdr), .arg2_type = ARG_PTR_TO_MEM, - .arg3_type = ARG_CONST_SIZE, + .arg3_type = ARG_CONST_SIZE_OR_ZERO, }; BPF_CALL_3(bpf_tcp_raw_gen_syncookie_ipv6, struct ipv6hdr *, iph,
@@ -7564,7 +7564,7 @@ static const struct bpf_func_proto bpf_tcp_raw_gen_syncookie_ipv6_proto = { .arg1_type = ARG_PTR_TO_FIXED_SIZE_MEM, .arg1_size = sizeof(struct ipv6hdr), .arg2_type = ARG_PTR_TO_MEM, - .arg3_type = ARG_CONST_SIZE, + .arg3_type = ARG_CONST_SIZE_OR_ZERO, }; BPF_CALL_2(bpf_tcp_raw_check_syncookie_ipv4, struct iphdr *, iph,
Signature · 1 line
2.39.1
Ilya Leoshkevich
maintainerPATCH v2 2/31 bpf: Change BPF_MAX_TRAMP_LINKS to enum
In reply to Ilya LeoshkevichThis way it's possible to query its value from testcases using BTF.
--- include/linux/bpf.h | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
include/linux/bpf.h +3 -1
@@ -939,7 +939,9 @@ struct btf_func_model {/* Each call __bpf_prog_enter + call bpf_func + call __bpf_prog_exit is ~50 * bytes on x86. */ -#define BPF_MAX_TRAMP_LINKS 38 +enum { +BPF_MAX_TRAMP_LINKS =38 , +};struct bpf_tramp_links {struct bpf_tramp_link *links[BPF_MAX_TRAMP_LINKS ];
Signature · 1 line
2.39.1
Ilya Leoshkevich
maintainerPATCH v2 3/31 selftests/bpf: Query BPF_MAX_TRAMP_LINKS using BTF
In reply to Ilya LeoshkevichDo not hard-code the value, since for s390x it will be smaller than for x86.
--- .../selftests/bpf/prog_tests/fexit_stress.c | 22 +++++++---- .../bpf/prog_tests/trampoline_count.c | 18 +++++++-- tools/testing/selftests/bpf/test_progs.c | 38 +++++++++++++++++++ tools/testing/selftests/bpf/test_progs.h | 2 + 4 files changed, 69 insertions(+), 11 deletions(-)
tools/testing/selftests/bpf/prog_tests/fexit_stress.c +14 -8
@@ -2,14 +2,19 @@/* Copyright (c) 2019 Facebook */ #include <test_progs.h> -/* that's kernel internal BPF_MAX_TRAMP_PROGS define */ -#define CNT 38 -void serial_test_fexit_stress (void ) { -int fexit_fd[CNT ] = {}; -int link_fd[CNT ] = {}; -int err, i; +int bpf_max_tramp_links, err, i; +int *fd, *fexit_fd, *link_fd; + + bpf_max_tramp_links =get_bpf_max_tramp_links (); +if (!ASSERT_GE (bpf_max_tramp_links,1 ,"bpf_max_tramp_links" )) +return ; + fd =calloc (bpf_max_tramp_links *2 ,sizeof (*fd)); +if (!ASSERT_OK_PTR (fd,"fd" )) +return ; + fexit_fd = fd; + link_fd = fd + bpf_max_tramp_links;const struct bpf_insn trace_program[] = {BPF_MOV64_IMM (BPF_REG_0 ,0 ),
@@ -28,7 +33,7 @@ void serial_test_fexit_stress(void) goto out; trace_opts.attach_btf_id = err; -for (i =0 ; i <CNT ; i++) { +for (i =0 ; i < bpf_max_tramp_links; i++) { fexit_fd[i] = bpf_prog_load(BPF_PROG_TYPE_TRACING, NULL, "GPL", trace_program, sizeof(trace_program) / sizeof(struct bpf_insn),
@@ -44,10 +49,11 @@ void serial_test_fexit_stress(void)ASSERT_OK (err,"bpf_prog_test_run_opts" ); out: -for (i =0 ; i <CNT ; i++) { +for (i =0 ; i < bpf_max_tramp_links; i++) {if (link_fd[i])close (link_fd[i]);if (fexit_fd[i])close (fexit_fd[i]); } +free (fd); }
tools/testing/selftests/bpf/prog_tests/trampoline_count.c +15 -3
@@ -2,7 +2,11 @@#define _GNU_SOURCE#include <test_progs.h> +#if defined(__s390x__) +#define MAX_TRAMP_PROGS 27 +#else #define MAX_TRAMP_PROGS 38 +#endif struct inst {struct bpf_object *obj;
@@ -37,14 +41,21 @@ void serial_test_trampoline_count(void) {char *file ="test_trampoline_count.bpf.o" ;char *const progs[] = {"fentry_test" ,"fmod_ret_test" ,"fexit_test" }; -struct inst inst[MAX_TRAMP_PROGS +1 ] = {}; +int bpf_max_tramp_links, err, i, prog_fd;struct bpf_program *prog;struct bpf_link *link; -int prog_fd, err, i; +struct inst *inst;LIBBPF_OPTS (bpf_test_run_opts, opts); + bpf_max_tramp_links =get_bpf_max_tramp_links (); +if (!ASSERT_GE (bpf_max_tramp_links,1 ,"bpf_max_tramp_links" )) +return ; + inst =calloc (bpf_max_tramp_links +1 ,sizeof (*inst)); +if (!ASSERT_OK_PTR (inst,"inst" )) +return ; +/* attach 'allowed' trampoline programs */ -for (i =0 ; i <MAX_TRAMP_PROGS ; i++) { +for (i =0 ; i < bpf_max_tramp_links; i++) { prog =load_prog (file, progs[i %ARRAY_SIZE (progs)], &inst[i]);if (!prog) goto cleanup;
@@ -91,4 +102,5 @@ void serial_test_trampoline_count(void)bpf_link__destroy (inst[i].link);bpf_object__close (inst[i].obj); } +free (inst); }
tools/testing/selftests/bpf/test_progs.c +38 -0
@@ -17,6 +17,7 @@#include <sys/select.h> #include <sys/socket.h> #include <sys/un.h> +#include <bpf/btf.h> static bool verbose (void ) {
@@ -967,6 +968,43 @@ int write_sysctl(const char *sysctl, const char *value)return 0 ; } +int get_bpf_max_tramp_links_from (struct btf *btf) +{ +const struct btf_enum *e; +const struct btf_type *t; +__u32 i, type_cnt; +const char *name; +__u16 j, vlen; + +for (i =1 , type_cnt =btf__type_cnt (btf); i < type_cnt; i++) { + t =btf__type_by_id (btf, i); +if (!t || !btf_is_enum (t) || t->name_off) +continue ; + e =btf_enum (t); +for (j =0 , vlen =btf_vlen (t); j < vlen; j++, e++) { + name =btf__str_by_offset (btf, e->name_off); +if (name && !strcmp (name,"BPF_MAX_TRAMP_LINKS" )) +return e->val; + } + } + +return -1 ; +} + +int get_bpf_max_tramp_links (void ) +{ +struct btf *vmlinux_btf; +int ret; + + vmlinux_btf =btf__load_vmlinux_btf (); +if (!ASSERT_OK_PTR (vmlinux_btf,"vmlinux btf" )) +return -1 ; + ret =get_bpf_max_tramp_links_from (vmlinux_btf); +btf__free (vmlinux_btf); + +return ret; +} +#define MAX_BACKTRACE_SZ 128void crash_handler (int signum) {
tools/testing/selftests/bpf/test_progs.h +2 -0
@@ -394,6 +394,8 @@ int kern_sync_rcu(void);int trigger_module_test_read (int read_sz);int trigger_module_test_write (int write_sz);int write_sysctl (const char *sysctl,const char *value); +int get_bpf_max_tramp_links_from (struct btf *btf); +int get_bpf_max_tramp_links (void );#ifdef __x86_64__#define SYS_NANOSLEEP_KPROBE_NAME "__x64_sys_nanosleep"
Signature · 1 line
2.39.1
Re: [PATCH bpf-next v2 03/31] selftests/bpf: Query BPF_MAX_TRAMP_LINKS using BTF
tools/testing/selftests/bpf/prog_tests/trampoline_count.c line 9
Quoted 13 lines · Ilya Leoshkevich
tools/testing/selftests/bpf/prog_tests/trampoline_count.c +4
-0@@ -2,7 +2,11 @@#define _GNU_SOURCE#include <test_progs.h> +#if defined(__s390x__) +#define MAX_TRAMP_PROGS 27 +#else #define MAX_TRAMP_PROGS 38 +#endif
This was a leftover from v1. I've removed it while applying.
Also dropped sk_assign fix patch 18, since it requires 'tc' to be built with libbpf which might not be the case. Pls figure out a different fix.
Pushed the first 26-1 patches. The last few need a respin to fix a build warn. Thanks! Great stuff.
Re: [PATCH bpf-next v2 03/31] selftests/bpf: Query BPF_MAX_TRAMP_LINKS using BTF
Quoted 18 lines · Alexei Starovoitov
tools/testing/selftests/bpf/prog_tests/trampoline_count.c line 9
Quoted 13 lines · Ilya Leoshkevich
tools/testing/selftests/bpf/prog_tests/trampoline_count.c +4
-0@@ -2,7 +2,11 @@#define _GNU_SOURCE#include <test_progs.h> +#if defined(__s390x__) +#define MAX_TRAMP_PROGS 27 +#else #define MAX_TRAMP_PROGS 38 +#endif This was a leftover from v1. I've removed it while applying.
Quoted 6 lines
Also dropped
sk_assignfix patch 18, since it requires 'tc' to be built with libbpf which might not be the case. Pls figure out a different fix.Pushed the first 26-1 patches. The last few need a respin to fix a build warn. Thanks! Great stuff.
One more thing.
When you respin please update DENYLIST.s390x,
so that BPF CI can confirm that a bunch of tests are now passing. Most of the denylist can be removed, right?
Ilya Leoshkevich
maintainerPATCH v2 4/31 selftests/bpf: Fix liburandom_read.so linker error
In reply to Ilya LeoshkevichWhen building with O=, the following linker error occurs:
clang: error: no such file or directory: 'liburandom_read.so'
Fix by adding $(OUTPUT) to the linker search path.
--- tools/testing/selftests/bpf/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/testing/selftests/bpf/Makefile +1 -1
@@ -189,7 +189,7 @@ $(OUTPUT)/liburandom_read.so: urandom_read_lib1.c urandom_read_lib2.c $(OUTPUT)/urandom_read: urandom_read.c urandom_read_aux.c $(OUTPUT)/liburandom_read.so $(call msg,BINARY,,$@) $(Q)$(CLANG) $(filter-out -static,$(CFLAGS) $(LDFLAGS)) $(filter %.c,$^) \ - liburandom_read.so $(filter-out -static,$(LDLIBS)) \ + -lurandom_read $(filter-out -static,$(LDLIBS)) -L$(OUTPUT) \ -fuse-ld=$(LLD) -Wl,-znoseparate-code -Wl,--build-id=sha1 \ -Wl,-rpath=. -o $@
Signature · 1 line
2.39.1
Ilya Leoshkevich
maintainerPATCH v2 5/31 selftests/bpf: Fix symlink creation error
In reply to Ilya LeoshkevichWhen building with O=, the following error occurs:
ln: failed to create symbolic link 'no_alu32/bpftool': No such file or directory
Adjust the code to account for $(OUTPUT).
--- tools/testing/selftests/bpf/Makefile | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
tools/testing/selftests/bpf/Makefile +2 -1
@@ -519,7 +519,8 @@ $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS) \ $$(call msg,BINARY,,$$@) $(Q)$$(CC) $$(CFLAGS) $$(filter %.a %.o,$$^) $$(LDLIBS) -o $$@ $(Q)$(RESOLVE_BTFIDS) --btf $(TRUNNER_OUTPUT)/btf_data.bpf.o $$@ - $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/bootstrap/bpftool $(if $2,$2/)bpftool + $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/bootstrap/bpftool \ + $(OUTPUT)/$(if $2,$2/)bpftool endef
Signature · 1 line
2.39.1
Ilya Leoshkevich
maintainerPATCH v2 6/31 selftests/bpf: Fix kfree_skb on s390x
In reply to Ilya Leoshkevichh_proto is big-endian; use htons() in order to make comparison work on both little- and big-endian machines.
--- tools/testing/selftests/bpf/prog_tests/kfree_skb.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/testing/selftests/bpf/prog_tests/kfree_skb.c +1 -1
@@ -36,7 +36,7 @@ static void on_sample(void *ctx, int cpu, void *data, __u32 size)"cb32_0 %x != %x\n" , meta->cb32_0, cb.cb32[0])) return; -if (CHECK (pkt_v6->eth.h_proto !=0xdd86 ,"check_eth" , +if (CHECK (pkt_v6->eth.h_proto !=htons (ETH_P_IPV6 ),"check_eth" ,"h_proto %x\n" , pkt_v6->eth.h_proto))return ; if (CHECK(pkt_v6->iph.nexthdr != 6, "check_ip",
Signature · 1 line
2.39.1
Ilya Leoshkevich
maintainerPATCH v2 7/31 selftests/bpf: Set errno when urand_spawn() fails
In reply to Ilya LeoshkevichThe result of urand_spawn() is checked with ASSERT_OK_PTR, which treats NULL as success if errno == 0.
--- tools/testing/selftests/bpf/prog_tests/usdt.c | 1 + 1 file changed, 1 insertion(+)
tools/testing/selftests/bpf/prog_tests/usdt.c +1 -0
@@ -314,6 +314,7 @@ static FILE *urand_spawn(int *pid)if (fscanf (f,"%d" , pid) !=1 ) {pclose (f); + errno =EINVAL ;return NULL ; }
Signature · 1 line
2.39.1
Ilya Leoshkevich
maintainerPATCH v2 8/31 selftests/bpf: Fix decap_sanity_ns cleanup
In reply to Ilya Leoshkevichdecap_sanity prints the following on the 1st run:
decap_sanity: sh: 1: Syntax error: Bad fd number
and the following on the 2nd run:
Cannot create namespace file "/run/netns/decap_sanity_ns": File exists
The problem is that the cleanup command has a typo and does nothing. Fix the typo.
--- tools/testing/selftests/bpf/prog_tests/decap_sanity.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/testing/selftests/bpf/prog_tests/decap_sanity.c +1 -1
@@ -80,6 +80,6 @@ void test_decap_sanity(void)bpf_tc_hook_destroy (&qdisc_hook);close_netns (nstoken); } -system ("ip netns del " NS_TEST " >& /dev/null" ); +system ("ip netns del " NS_TEST " &> /dev/null" );decap_sanity__destroy (skel); }
Signature · 1 line
2.39.1
Ilya Leoshkevich
maintainerPATCH v2 9/31 selftests/bpf: Fix verify_pkcs7_sig on s390x
In reply to Ilya LeoshkevichUse bpf_probe_read_kernel() instead of bpf_probe_read(), which is not defined on all architectures.
While at it, improve the error handling: do not hide the verifier log, and check the return values of bpf_probe_read_kernel() and bpf_copy_from_user().
--- .../selftests/bpf/prog_tests/verify_pkcs7_sig.c | 3 +++ .../selftests/bpf/progs/test_verify_pkcs7_sig.c | 12 ++++++++---- 2 files changed, 11 insertions(+), 4 deletions(-)
tools/testing/selftests/bpf/prog_tests/verify_pkcs7_sig.c +3 -0
@@ -61,6 +61,9 @@ static bool kfunc_not_supported;static int libbpf_print_cb (enum libbpf_print_level level,const char *fmt,va_list args) { +if (level ==LIBBPF_WARN ) +vprintf (fmt, args); +if (strcmp (fmt,"libbpf: extern (func ksym) '%s': not found in kernel or module BTFs\n" ))return 0 ;
tools/testing/selftests/bpf/progs/test_verify_pkcs7_sig.c +8 -4
@@ -59,10 +59,14 @@ int BPF_PROG(bpf, int cmd, union bpf_attr *attr, unsigned int size)if (!data_val)return 0 ; -bpf_probe_read (&value,sizeof (value), &attr->value); - -bpf_copy_from_user (data_val,sizeof (struct data ), - (void *)(unsigned long )value); + ret =bpf_probe_read_kernel (&value,sizeof (value), &attr->value); +if (ret) +return ret; + + ret =bpf_copy_from_user (data_val,sizeof (struct data ), + (void *)(unsigned long )value); +if (ret) +return ret;if (data_val->data_len >sizeof (data_val->data))return -EINVAL ;
Signature · 1 line
2.39.1
Ilya Leoshkevich
maintainerPATCH v2 10/31 selftests/bpf: Fix xdp_do_redirect on s390x
In reply to Ilya Leoshkevichs390x cache line size is 256 bytes, so skb_shared_info must be aligned on a much larger boundary than for x86. This makes the maximum packet size smaller.
--- tools/testing/selftests/bpf/prog_tests/xdp_do_redirect.c | 4 ++++ 1 file changed, 4 insertions(+)
tools/testing/selftests/bpf/prog_tests/xdp_do_redirect.c +4 -0
@@ -65,7 +65,11 @@ static int attach_tc_prog(struct bpf_tc_hook *hook, int fd)/* The maximum permissible size is: PAGE_SIZE - sizeof(struct xdp_page_head) - * sizeof(struct skb_shared_info) - XDP_PACKET_HEADROOM = 3368 bytes */ +#if defined(__s390x__) +#define MAX_PKT_SIZE 3176 +#else #define MAX_PKT_SIZE 3368 +#endif static void test_max_pkt_size (int fd) {char data[MAX_PKT_SIZE +1 ] = {};
Signature · 1 line
2.39.1
Ilya Leoshkevich
maintainerPATCH v2 11/31 selftests/bpf: Fix cgrp_local_storage on s390x
In reply to Ilya LeoshkevichSync the definition of socket_cookie between the eBPF program and the test. Currently the test works by accident, since on little-endian it is sometimes acceptable to access u64 as u32.
--- tools/testing/selftests/bpf/prog_tests/cgrp_local_storage.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/testing/selftests/bpf/prog_tests/cgrp_local_storage.c +1 -1
@@ -16,7 +16,7 @@struct socket_cookie {__u64 cookie_key; -__u32 cookie_value; +__u64 cookie_value; };static void test_tp_btf (int cgroup_fd)
Signature · 1 line
2.39.1
Ilya Leoshkevich
maintainerPATCH v2 12/31 selftests/bpf: Check stack_mprotect() return value
In reply to Ilya LeoshkevichIf stack_mprotect() succeeds, errno is not changed. This can produce misleading error messages, that show stale errno.
--- tools/testing/selftests/bpf/prog_tests/bpf_cookie.c | 6 ++++-- tools/testing/selftests/bpf/prog_tests/test_lsm.c | 3 ++- 2 files changed, 6 insertions(+), 3 deletions(-)
tools/testing/selftests/bpf/prog_tests/bpf_cookie.c +4 -2
@@ -472,6 +472,7 @@ static void lsm_subtest(struct test_bpf_cookie *skel)int prog_fd;int lsm_fd =-1 ;LIBBPF_OPTS (bpf_link_create_opts, link_opts); +int err; skel->bss->lsm_res =0 ;
@@ -482,8 +483,9 @@ static void lsm_subtest(struct test_bpf_cookie *skel)if (!ASSERT_GE (lsm_fd,0 ,"lsm.link_create" )) goto cleanup; -stack_mprotect (); -if (!ASSERT_EQ (errno,EPERM ,"stack_mprotect" )) + err =stack_mprotect (); +if (!ASSERT_EQ (err,-1 ,"stack_mprotect" ) || + !ASSERT_EQ (errno,EPERM ,"stack_mprotect" )) goto cleanup;usleep (1 );
tools/testing/selftests/bpf/prog_tests/test_lsm.c +2 -1
@@ -75,7 +75,8 @@ static int test_lsm(struct lsm *skel) skel->bss->monitored_pid =getpid (); err =stack_mprotect (); -if (!ASSERT_EQ (errno,EPERM ,"stack_mprotect" )) +if (!ASSERT_EQ (err,-1 ,"stack_mprotect" ) || + !ASSERT_EQ (errno,EPERM ,"stack_mprotect" ))return err;ASSERT_EQ (skel->bss->mprotect_count,1 ,"mprotect_count" );
Signature · 1 line
2.39.1
Ilya Leoshkevich
maintainerPATCH v2 13/31 selftests/bpf: Increase SIZEOF_BPF_LOCAL_STORAGE_ELEM on s390x
In reply to Ilya Leoshkevichsizeof(struct bpf_local_storage_elem) is 512 on s390x:
struct bpf_local_storage_elem {
struct hlist_node map_node; /* 0 16 */
struct hlist_node snode; /* 16 16 */
struct bpf_local_storage * local_storage; /* 32 8 */
struct callback_head rcu __attribute__((__aligned__(8))); /* 40 16 */
/* XXX 200 bytes hole, try to pack */
/* --- cacheline 1 boundary (256 bytes) --- */
struct bpf_local_storage_data sdata __attribute__((__aligned__(256))); /* 256 8 */
/* size: 512, cachelines: 2, members: 5 */
/* sum members: 64, holes: 1, sum holes: 200 */
/* padding: 248 */
/* forced alignments: 2, forced holes: 1, sum forced holes: 200 */
} __attribute__((__aligned__(256)));As the existing comment suggests, use a larger number in order to be future-proof.
--- tools/testing/selftests/bpf/netcnt_common.h | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
tools/testing/selftests/bpf/netcnt_common.h +3 -3
@@ -8,11 +8,11 @@/* sizeof(struct bpf_local_storage_elem): * -* It really is about 128 bytes on x86_64, but allocate more to account for -* possible layout changes, different architectures, etc. +* It is about 128 bytes on x86_64 and 512 bytes on s390x, but allocate more to +* account for possible layout changes, different architectures, etc. * The kernel will wrap up to PAGE_SIZE internally anyway. */ -#define SIZEOF_BPF_LOCAL_STORAGE_ELEM 256 +#define SIZEOF_BPF_LOCAL_STORAGE_ELEM 768/* Try to estimate kernel's BPF_LOCAL_STORAGE_MAX_VALUE_SIZE: */ #define BPF_LOCAL_STORAGE_MAX_VALUE_SIZE (0xFFFF - \
Signature · 1 line
2.39.1
Ilya Leoshkevich
maintainerPATCH v2 14/31 selftests/bpf: Add a sign-extension test for kfuncs
In reply to Ilya Leoshkevichs390x ABI requires the caller to zero- or sign-extend the arguments. eBPF already deals with zero-extension (by definition of its ABI), but not with sign-extension.
Add a test to cover that potentially problematic area.
--- net/bpf/test_run.c | 9 +++++++++ .../selftests/bpf/prog_tests/kfunc_call.c | 1 + .../selftests/bpf/progs/kfunc_call_test.c | 18 ++++++++++++++++++ 3 files changed, 28 insertions(+)
net/bpf/test_run.c +9 -0
@@ -550,6 +550,14 @@ struct sock * noinline bpf_kfunc_call_test3(struct sock *sk)return sk; } +long noinline bpf_kfunc_call_test4 (signed char a,short b,int c,long d) +{ +/* Provoke the compiler to assume that the caller has sign-extended a, +* b and c on platforms where this is required (e.g. s390x). +*/ +return (long )a + (long )b + (long )c + d; +} +struct prog_test_member1 {int a; };
@@ -746,6 +754,7 @@ BTF_SET8_START(test_sk_check_kfunc_ids) BTF_ID_FLAGS(func, bpf_kfunc_call_test1) BTF_ID_FLAGS(func, bpf_kfunc_call_test2) BTF_ID_FLAGS(func, bpf_kfunc_call_test3) +BTF_ID_FLAGS(func, bpf_kfunc_call_test4) BTF_ID_FLAGS(func, bpf_kfunc_call_test_acquire, KF_ACQUIRE | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_kfunc_call_memb_acquire, KF_ACQUIRE | KF_RET_NULL)BTF_ID_FLAGS (func, bpf_kfunc_call_test_release,KF_RELEASE )
tools/testing/selftests/bpf/prog_tests/kfunc_call.c +1 -0
@@ -72,6 +72,7 @@ static struct kfunc_test_params kfunc_tests[] = {/* success cases */ TC_TEST (kfunc_call_test1,12 ),TC_TEST (kfunc_call_test2,3 ), +TC_TEST (kfunc_call_test4,-1234 ),TC_TEST (kfunc_call_test_ref_btf_id,0 ),TC_TEST (kfunc_call_test_get_mem,42 ), SYSCALL_TEST(kfunc_syscall_test, 0),
tools/testing/selftests/bpf/progs/kfunc_call_test.c +18 -0
@@ -3,6 +3,7 @@#include <vmlinux.h> #include <bpf/bpf_helpers.h> +extern long bpf_kfunc_call_test4(signed char a, short b, int c, long d) __ksym; extern int bpf_kfunc_call_test2(struct sock *sk, __u32 a, __u32 b) __ksym; extern __u64 bpf_kfunc_call_test1(struct sock *sk, __u32 a, __u64 b, __u32 c, __u64 d) __ksym;
@@ -17,6 +18,23 @@ extern void bpf_kfunc_call_test_mem_len_fail2(__u64 *mem, int len) __ksym;extern int *bpf_kfunc_call_test_get_rdwr_mem (struct prog_test_ref_kfunc *p,const int rdwr_buf_size) __ksym;extern int *bpf_kfunc_call_test_get_rdonly_mem (struct prog_test_ref_kfunc *p,const int rdonly_buf_size) __ksym; +SEC("tc") +int kfunc_call_test4 (struct __sk_buff *skb) +{ +struct bpf_sock *sk = skb->sk; +long tmp; + +if (!sk) +return -1 ; + + sk =bpf_sk_fullsock (sk); +if (!sk) +return -1 ; + + tmp =bpf_kfunc_call_test4 (-3 ,-30 ,-200 ,-1000 ); +return (tmp >>32 ) + tmp; +} + SEC("tc")int kfunc_call_test2 (struct __sk_buff *skb) {
Signature · 1 line
2.39.1
Ilya Leoshkevich
maintainerPATCH v2 15/31 selftests/bpf: Fix test_lsm on s390x
In reply to Ilya LeoshkevichUse syscall macros to access the setdomainname() arguments; currently the code uses gprs[2] instead of orig_gpr2 for the first argument.
--- tools/testing/selftests/bpf/progs/lsm.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
tools/testing/selftests/bpf/progs/lsm.c +4 -3
@@ -6,9 +6,10 @@#include "bpf_misc.h" #include "vmlinux.h" +#include <bpf/bpf_core_read.h> #include <bpf/bpf_helpers.h> #include <bpf/bpf_tracing.h> -#include <errno.h> +#include <errno.h> struct { __uint(type, BPF_MAP_TYPE_ARRAY);
@@ -164,8 +165,8 @@ int copy_test = 0; SEC("fentry.s/" SYS_PREFIX "sys_setdomainname")int BPF_PROG (test_sys_setdomainname ,struct pt_regs *regs) { -void *ptr = (void *)PT_REGS_PARM1 (regs); -int len =PT_REGS_PARM2 (regs); +void *ptr = (void *)PT_REGS_PARM1_SYSCALL (regs); +int len =PT_REGS_PARM2_SYSCALL (regs);int buf =0 ;long ret;
Signature · 1 line
2.39.1
Ilya Leoshkevich
maintainerPATCH v2 16/31 selftests/bpf: Fix test_xdp_adjust_tail_grow2 on s390x
In reply to Ilya Leoshkevichs390x cache line size is 256 bytes, so skb_shared_info must be aligned on a much larger boundary than for x86.
--- tools/testing/selftests/bpf/prog_tests/xdp_adjust_tail.c | 7 ++++++- .../selftests/bpf/progs/test_xdp_adjust_tail_grow.c | 8 +++++++- 2 files changed, 13 insertions(+), 2 deletions(-)
tools/testing/selftests/bpf/prog_tests/xdp_adjust_tail.c +6 -1
@@ -76,10 +76,15 @@ static void test_xdp_adjust_tail_grow2(void) {const char *file ="./test_xdp_adjust_tail_grow.bpf.o" ;char buf[4096 ];/* avoid segfault: large buf to hold grow results */ -int tailroom =320 ;/* SKB_DATA_ALIGN(sizeof(struct skb_shared_info))*/ ;struct bpf_object *obj;int err, cnt, i;int max_grow, prog_fd; +/* SKB_DATA_ALIGN(sizeof(struct skb_shared_info)) */ +#if defined(__s390x__) +int tailroom =512 ; +#else +int tailroom =320 ; +#endif LIBBPF_OPTS(bpf_test_run_opts, tattr, .repeat = 1,
tools/testing/selftests/bpf/progs/test_xdp_adjust_tail_grow.c +7 -1
@@ -9,6 +9,12 @@ int _xdp_adjust_tail_grow(struct xdp_md *xdp)void *data = (void *)(long )xdp->data;int data_len =bpf_xdp_get_buff_len (xdp);int offset =0 ; +/* SKB_DATA_ALIGN(sizeof(struct skb_shared_info)) */ +#if defined(__TARGET_ARCH_s390) +int tailroom =512 ; +#else +int tailroom =320 ; +#endif /* Data length determine test case */
@@ -20,7 +26,7 @@ int _xdp_adjust_tail_grow(struct xdp_md *xdp) offset =128 ; }else if (data_len ==128 ) {/* Max tail grow 3520 */ - offset =4096 -256 -320 - data_len; + offset =4096 -256 - tailroom - data_len; }else if (data_len ==9000 ) { offset =10 ; }else if (data_len ==9001 ) {
Signature · 1 line
2.39.1
Ilya Leoshkevich
maintainerPATCH v2 17/31 selftests/bpf: Fix vmlinux test on s390x
In reply to Ilya LeoshkevichUse a syscall macro to access the nanosleep()'s first argument; currently the code uses gprs[2] instead of orig_gpr2.
--- tools/testing/selftests/bpf/progs/test_vmlinux.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
tools/testing/selftests/bpf/progs/test_vmlinux.c +2 -2
@@ -42,7 +42,7 @@ int BPF_PROG(handle__raw_tp, struct pt_regs *regs, long id)if (id != __NR_nanosleep)return 0 ; - ts = (void *)PT_REGS_PARM1_CORE (regs); + ts = (void *)PT_REGS_PARM1_CORE_SYSCALL (regs);if (bpf_probe_read_user (&tv_nsec,sizeof (ts->tv_nsec), &ts->tv_nsec) || tv_nsec !=MY_TV_NSEC )return 0 ;
@@ -60,7 +60,7 @@ int BPF_PROG(handle__tp_btf, struct pt_regs *regs, long id)if (id != __NR_nanosleep)return 0 ; - ts = (void *)PT_REGS_PARM1_CORE (regs); + ts = (void *)PT_REGS_PARM1_CORE_SYSCALL (regs);if (bpf_probe_read_user (&tv_nsec,sizeof (ts->tv_nsec), &ts->tv_nsec) || tv_nsec !=MY_TV_NSEC )return 0 ;
Signature · 1 line
2.39.1
Ilya Leoshkevich
maintainerPATCH v2 18/31 selftests/bpf: Fix sk_assign on s390x
In reply to Ilya Leoshkevichsk_assign is failing on an s390x machine running Debian "bookworm" for 2 reasons: legacy server_map definition and uninitialized addrlen in recvfrom() call.
Fix by upgrading the server_map definition and dropping addrlen (recvfrom() allows NULL values for src_addr and addrlen).
--- .../selftests/bpf/prog_tests/sk_assign.c | 5 +--- .../selftests/bpf/progs/test_sk_assign.c | 24 +++++-------------- 2 files changed, 7 insertions(+), 22 deletions(-)
tools/testing/selftests/bpf/prog_tests/sk_assign.c +1 -4
@@ -129,15 +129,12 @@ get_port(int fd)static ssize_t rcv_msg (int srv_client,int type) { -struct sockaddr_storage ss;char buf[BUFSIZ ]; -socklen_t slen;if (type ==SOCK_STREAM )return read (srv_client, &buf,sizeof (buf));else -return recvfrom (srv_client, &buf,sizeof (buf),0 , - (struct sockaddr *)&ss, &slen); +return recvfrom (srv_client, &buf,sizeof (buf),0 ,NULL ,NULL ); } static int
tools/testing/selftests/bpf/progs/test_sk_assign.c +6 -18
@@ -16,25 +16,13 @@#include <bpf/bpf_helpers.h> #include <bpf/bpf_endian.h> -/* Pin map under /sys/fs/bpf/tc/globals/<map name> */ -#define PIN_GLOBAL_NS 2 - -/* Must match struct bpf_elf_map layout from iproute2 */ struct { -__u32 type; -__u32 size_key; -__u32 size_value; -__u32 max_elem; -__u32 flags; -__u32 id; -__u32 pinning; -} server_map SEC("maps") = { - .type = BPF_MAP_TYPE_SOCKMAP, - .size_key = sizeof(int), - .size_value = sizeof(__u64), - .max_elem = 1, - .pinning = PIN_GLOBAL_NS, -}; + __uint(type, BPF_MAP_TYPE_SOCKMAP); + __type(key, int); + __type(value, __u64); + __uint(pinning, LIBBPF_PIN_BY_NAME); + __uint(max_entries, 1); +} server_map SEC(".maps"); char _license[] SEC("license") = "GPL";
Signature · 1 line
2.39.1
Ilya Leoshkevich
maintainerPATCH v2 19/31 selftests/bpf: Fix xdp_synproxy/tc on s390x
In reply to Ilya LeoshkevichUse the correct datatype for the values map values; currently the test works by accident, since on little-endian machines it is sometimes acceptable to access u64 as u32.
--- tools/testing/selftests/bpf/progs/xdp_synproxy_kern.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/testing/selftests/bpf/progs/xdp_synproxy_kern.c +1 -1
@@ -310,7 +310,7 @@ static __always_inline void values_get_tcpipopts(__u16 *mss, __u8 *wscale, static __always_inline void values_inc_synacks(void) {__u32 key =1 ; -__u32 *value; +__u64 *value; value =bpf_map_lookup_elem (&values, &key);if (value)
Signature · 1 line
2.39.1
Ilya Leoshkevich
maintainerPATCH v2 20/31 selftests/bpf: Fix profiler on s390x
In reply to Ilya LeoshkevichUse bpf_probe_read_kernel() and bpf_probe_read_kernel_str() instead of bpf_probe_read() and bpf_probe_read_kernel().
--- .../selftests/bpf/progs/profiler.inc.h | 62 ++++++++++++------- 1 file changed, 38 insertions(+), 24 deletions(-)
tools/testing/selftests/bpf/progs/profiler.inc.h +38 -24
@@ -156,10 +156,10 @@ probe_read_lim(void* dst, void* src, unsigned long len, unsigned long max) { len = len < max ? len : max;if (len >1 ) { -if (bpf_probe_read (dst, len, src)) +if (bpf_probe_read_kernel (dst, len, src))return 0 ; }else if (len ==1 ) { -if (bpf_probe_read (dst,1 , src)) +if (bpf_probe_read_kernel (dst,1 , src))return 0 ; }return len;
@@ -216,7 +216,8 @@ static INLINE void* read_full_cgroup_path(struct kernfs_node* cgroup_node,#endif for (int i =0 ; i <MAX_CGROUPS_PATH_DEPTH ; i++) { filepart_length = -bpf_probe_read_str (payload,MAX_PATH ,BPF_CORE_READ (cgroup_node, name)); +bpf_probe_read_kernel_str (payload,MAX_PATH , +BPF_CORE_READ (cgroup_node, name));if (!cgroup_node)return payload;if (cgroup_node == cgroup_root_node)
@@ -303,7 +304,8 @@ static INLINE void* populate_cgroup_info(struct cgroup_data_t* cgroup_data, cgroup_data->cgroup_full_length =0 ;size_t cgroup_root_length = -bpf_probe_read_str (payload,MAX_PATH ,BPF_CORE_READ (root_kernfs, name)); +bpf_probe_read_kernel_str (payload,MAX_PATH , +BPF_CORE_READ (root_kernfs, name));barrier_var (cgroup_root_length);if (cgroup_root_length <=MAX_PATH ) {barrier_var (cgroup_root_length);
@@ -312,7 +314,8 @@ static INLINE void* populate_cgroup_info(struct cgroup_data_t* cgroup_data, }size_t cgroup_proc_length = -bpf_probe_read_str (payload,MAX_PATH ,BPF_CORE_READ (proc_kernfs, name)); +bpf_probe_read_kernel_str (payload,MAX_PATH , +BPF_CORE_READ (proc_kernfs, name));barrier_var (cgroup_proc_length);if (cgroup_proc_length <=MAX_PATH ) {barrier_var (cgroup_proc_length);
@@ -395,7 +398,8 @@ static INLINE int trace_var_sys_kill(void* ctx, int tpid, int sig) arr_struct =bpf_map_lookup_elem (&data_heap, &zero);if (arr_struct ==NULL )return 0 ; -bpf_probe_read (&arr_struct->array[0 ],sizeof (arr_struct->array[0 ]), kill_data); +bpf_probe_read_kernel (&arr_struct->array[0 ], +sizeof (arr_struct->array[0 ]), kill_data); }else {int index =get_var_spid_index (arr_struct, spid);
@@ -409,8 +413,9 @@ static INLINE int trace_var_sys_kill(void* ctx, int tpid, int sig)#endif for (int i =0 ; i <ARRAY_SIZE (arr_struct->array); i++)if (arr_struct->array[i].meta.pid ==0 ) { -bpf_probe_read (&arr_struct->array[i], -sizeof (arr_struct->array[i]), kill_data); +bpf_probe_read_kernel (&arr_struct->array[i], +sizeof (arr_struct->array[i]), + kill_data);bpf_map_update_elem (&var_tpid_to_data, &tpid, arr_struct,0 );
@@ -427,17 +432,17 @@ static INLINE int trace_var_sys_kill(void* ctx, int tpid, int sig)if (delta_sec <STALE_INFO ) { kill_data->kill_count++; kill_data->last_kill_time =bpf_ktime_get_ns (); -bpf_probe_read (&arr_struct->array[index], -sizeof (arr_struct->array[index]), - kill_data); +bpf_probe_read_kernel (&arr_struct->array[index], +sizeof (arr_struct->array[index]), + kill_data); }else {struct var_kill_data_t * kill_data =get_var_kill_data (ctx, spid, tpid, sig);if (kill_data ==NULL )return 0 ; -bpf_probe_read (&arr_struct->array[index], -sizeof (arr_struct->array[index]), - kill_data); +bpf_probe_read_kernel (&arr_struct->array[index], +sizeof (arr_struct->array[index]), + kill_data); } }bpf_map_update_elem (&var_tpid_to_data, &tpid, arr_struct,0 );
@@ -487,8 +492,9 @@ read_absolute_file_path_from_dentry(struct dentry* filp_dentry, void* payload)#pragma unroll#endif for (int i =0 ; i <MAX_PATH_DEPTH ; i++) { - filepart_length =bpf_probe_read_str (payload,MAX_PATH , -BPF_CORE_READ (filp_dentry, d_name.name)); + filepart_length = +bpf_probe_read_kernel_str (payload,MAX_PATH , +BPF_CORE_READ (filp_dentry, d_name.name));barrier_var (filepart_length);if (filepart_length >MAX_PATH )break ;
@@ -572,7 +578,8 @@ ssize_t BPF_KPROBE(kprobe__proc_sys_write, sysctl_data->sysctl_val_length =0 ; sysctl_data->sysctl_path_length =0 ; -size_t sysctl_val_length =bpf_probe_read_str (payload,CTL_MAXNAME , buf); +size_t sysctl_val_length =bpf_probe_read_kernel_str (payload, +CTL_MAXNAME , buf);barrier_var (sysctl_val_length);if (sysctl_val_length <=CTL_MAXNAME ) {barrier_var (sysctl_val_length);
@@ -580,8 +587,10 @@ ssize_t BPF_KPROBE(kprobe__proc_sys_write, payload += sysctl_val_length; } -size_t sysctl_path_length =bpf_probe_read_str (payload,MAX_PATH , -BPF_CORE_READ (filp, f_path.dentry, d_name.name)); +size_t sysctl_path_length = +bpf_probe_read_kernel_str (payload,MAX_PATH , +BPF_CORE_READ (filp, f_path.dentry, + d_name.name));barrier_var (sysctl_path_length);if (sysctl_path_length <=MAX_PATH ) {barrier_var (sysctl_path_length);
@@ -638,7 +647,8 @@ int raw_tracepoint__sched_process_exit(void* ctx)struct var_kill_data_t * past_kill_data = &arr_struct->array[i];if (past_kill_data !=NULL && past_kill_data->kill_target_pid == tpid) { -bpf_probe_read (kill_data,sizeof (*past_kill_data), past_kill_data); +bpf_probe_read_kernel (kill_data,sizeof (*past_kill_data), + past_kill_data);void * payload = kill_data->payload;size_t offset = kill_data->payload_length;if (offset >=MAX_METADATA_PAYLOAD_LEN +MAX_CGROUP_PAYLOAD_LEN )
@@ -656,8 +666,10 @@ int raw_tracepoint__sched_process_exit(void* ctx) payload += comm_length; } -size_t cgroup_proc_length =bpf_probe_read_str (payload,KILL_TARGET_LEN , -BPF_CORE_READ (proc_kernfs, name)); +size_t cgroup_proc_length = +bpf_probe_read_kernel_str (payload, +KILL_TARGET_LEN , +BPF_CORE_READ (proc_kernfs, name));barrier_var (cgroup_proc_length);if (cgroup_proc_length <=KILL_TARGET_LEN ) {barrier_var (cgroup_proc_length);
@@ -718,7 +730,8 @@ int raw_tracepoint__sched_process_exec(struct bpf_raw_tracepoint_args* ctx) proc_exec_data->parent_start_time =BPF_CORE_READ (parent_task, start_time);const char * filename =BPF_CORE_READ (bprm, filename); -size_t bin_path_length =bpf_probe_read_str (payload,MAX_FILENAME_LEN , filename); +size_t bin_path_length = +bpf_probe_read_kernel_str (payload,MAX_FILENAME_LEN , filename);barrier_var (bin_path_length);if (bin_path_length <=MAX_FILENAME_LEN ) {barrier_var (bin_path_length);
@@ -922,7 +935,8 @@ int BPF_KPROBE(kprobe__vfs_symlink, struct inode* dir, struct dentry* dentry, filemod_data->payload); payload =populate_cgroup_info (&filemod_data->cgroup_data, task, payload); -size_t len =bpf_probe_read_str (payload,MAX_FILEPATH_LENGTH , oldname); +size_t len =bpf_probe_read_kernel_str (payload,MAX_FILEPATH_LENGTH , + oldname);barrier_var (len);if (len <=MAX_FILEPATH_LENGTH ) {barrier_var (len);
Signature · 1 line
2.39.1
Ilya Leoshkevich
maintainerPATCH v2 21/31 libbpf: Simplify barrier_var()
In reply to Ilya LeoshkevichUse a single "+r" constraint instead of the separate "=r" and "0".
--- tools/lib/bpf/bpf_helpers.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/lib/bpf/bpf_helpers.h +1 -1
@@ -109,7 +109,7 @@* This is a variable-specific variant of more global barrier(). */ #ifndef barrier_var -#define barrier_var (var) asm volatile("" : "=r"(var) : "0"(var)) +#define barrier_var (var) asm volatile("" : "+r"(var))#endif /*
Signature · 1 line
2.39.1
Ilya Leoshkevich
maintainerPATCH v2 22/31 libbpf: Fix unbounded memory access in bpf_usdt_arg()
In reply to Ilya LeoshkevichLoading programs that use bpf_usdt_arg() on s390x fails with:
; if (arg_num >= BPF_USDT_MAX_ARG_CNT || arg_num >= spec->arg_cnt)
128: (79) r1 = *(u64 *)(r10 -24) ; frame1: R1_w=scalar(umax=4294967295,var_off=(0x0; 0xffffffff)) R10=fp0
129: (25) if r1 > 0xb goto pc+83 ; frame1: R1_w=scalar(umax=11,var_off=(0x0; 0xf))
...
; arg_spec = &spec->args[arg_num];
135: (79) r1 = *(u64 *)(r10 -24) ; frame1: R1_w=scalar(umax=4294967295,var_off=(0x0; 0xffffffff)) R10=fp0
...
; switch (arg_spec->arg_type) {
139: (61) r1 = *(u32 *)(r2 +8)
R2 unbounded memory access, make sure to bounds check any such accessThe reason is that, even though the C code enforces that arg_num < BPF_USDT_MAX_ARG_CNT, the verifier cannot propagate this constraint to the arg_spec assignment yet. Help it by forcing r1 back to stack after comparison.
--- tools/lib/bpf/usdt.bpf.h | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
tools/lib/bpf/usdt.bpf.h +4 -1
@@ -130,7 +130,10 @@ int bpf_usdt_arg(struct pt_regs *ctx, __u64 arg_num, long *res)if (!spec)return -ESRCH ; -if (arg_num >=BPF_USDT_MAX_ARG_CNT || arg_num >= spec->arg_cnt) +if (arg_num >=BPF_USDT_MAX_ARG_CNT ) +return -ENOENT ; +barrier_var (arg_num); +if (arg_num >= spec->arg_cnt)return -ENOENT ; arg_spec = &spec->args[arg_num];
Signature · 1 line
2.39.1
Ilya Leoshkevich
maintainerPATCH v2 23/31 libbpf: Fix BPF_PROBE_READ{_STR}_INTO() on s390x
In reply to Ilya LeoshkevichBPF_PROBE_READ_INTO() and BPF_PROBE_READ_STR_INTO() should map to bpf_probe_read() and bpf_probe_read_str() respectively in order to work correctly on architectures with !ARCH_HAS_NON_OVERLAPPING_ADDRESS_SPACE.
--- tools/lib/bpf/bpf_core_read.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
tools/lib/bpf/bpf_core_read.h +2 -2
@@ -364,7 +364,7 @@ enum bpf_enum_value_kind {/* Non-CO-RE variant of BPF_CORE_READ_INTO() */ #define BPF_PROBE_READ_INTO (dst, src, a, ...) ({ \ - ___core_read(bpf_probe_read, bpf_probe_read, \ + ___core_read(bpf_probe_read_kernel, bpf_probe_read_kernel, \ dst, (src), a, ##__VA_ARGS__) \ })
@@ -400,7 +400,7 @@ enum bpf_enum_value_kind {/* Non-CO-RE variant of BPF_CORE_READ_STR_INTO() */ #define BPF_PROBE_READ_STR_INTO (dst, src, a, ...) ({ \ - ___core_read(bpf_probe_read_str, bpf_probe_read, \ + ___core_read(bpf_probe_read_kernel_str, bpf_probe_read_kernel, \ dst, (src), a, ##__VA_ARGS__) \ })
Signature · 1 line
2.39.1
Ilya Leoshkevich
maintainerPATCH v2 24/31 bpf: iterators: Split iterators.lskel.h into little- and big- endian versions
In reply to Ilya Leoshkevichiterators.lskel.h is little-endian, therefore bpf iterator is currently broken on big-endian systems. Introduce a big-endian version and add instructions regarding its generation. Unfortunately bpftool's cross-endianness capabilities are limited to BTF right now, so the procedure requires access to a big-endian machine or a configured emulator.
---
kernel/bpf/preload/bpf_preload_kern.c | 6 +-
kernel/bpf/preload/iterators/Makefile | 12 +-
kernel/bpf/preload/iterators/README | 5 +-
.../iterators/iterators.lskel-big-endian.h | 419 ++++++++++++++++++
...skel.h => iterators.lskel-little-endian.h} | 0
5 files changed, 435 insertions(+), 7 deletions(-)
create mode 100644 kernel/bpf/preload/iterators/iterators.lskel-big-endian.h
rename kernel/bpf/preload/iterators/{iterators.lskel.h => iterators.lskel-little-endian.h} (100%)kernel/bpf/preload/bpf_preload_kern.c +5 -1
@@ -3,7 +3,11 @@#include <linux/init.h> #include <linux/module.h> #include "bpf_preload.h" -#include "iterators/iterators.lskel.h" +#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__ +#include "iterators/iterators.lskel-little-endian.h" +#else +#include "iterators/iterators.lskel-big-endian.h" +#endif static struct bpf_link *maps_link, *progs_link;static struct iterators_bpf *skel;
kernel/bpf/preload/iterators/Makefile +7 -5
@@ -35,20 +35,22 @@ endif .PHONY: all clean -all: iterators.lskel.h +all: iterators.lskel-little-endian.h + +big: iterators.lskel-big-endian.h clean: $(call msg,CLEAN) $(Q)rm -rf $(OUTPUT) iterators -iterators.lskel.h: $(OUTPUT)/iterators.bpf.o | $(BPFTOOL) +iterators.lskel-%.h: $(OUTPUT)/%/iterators.bpf.o | $(BPFTOOL) $(call msg,GEN-SKEL,$@) $(Q)$(BPFTOOL) gen skeleton -L $< > $@ - -$(OUTPUT)/iterators.bpf.o: iterators.bpf.c $(BPFOBJ) | $(OUTPUT) +$(OUTPUT)/%/iterators.bpf.o: iterators.bpf.c $(BPFOBJ) | $(OUTPUT) $(call msg,BPF,$@) - $(Q)$(CLANG) -g -O2 -target bpf $(INCLUDES) \ + $(Q)mkdir -p $(@D) + $(Q)$(CLANG) -g -O2 -target bpf -m$* $(INCLUDES) \ -c $(filter %.c,$^) -o $@ && \ $(LLVM_STRIP) -g $@
kernel/bpf/preload/iterators/README +4 -1
@@ -1,4 +1,7 @@ WARNING: -If you change "iterators.bpf.c" do "make -j" in this directory to rebuild "iterators.skel.h". +If you change "iterators.bpf.c" do "make -j" in this directory to +rebuild "iterators.lskel-little-endian.h". Then, on a big-endian +machine, do "make -j big" in this directory to rebuild +"iterators.lskel-big-endian.h". Commit both resulting headers. Make sure to have clang 10 installed. See Documentation/bpf/bpf_devel_QA.rst
kernel/bpf/preload/iterators/iterators.lskel-big-endian.h new file +419 -0
419 lines not on this page · raw message
kernel/bpf/preload/iterators/iterators.lskel-little-endian.h renamed from kernel/bpf/preload/iterators/iterators.lskel.h +0 -0
Signature · 1 line
2.39.1
Ilya Leoshkevich
maintainerPATCH v2 25/31 bpf: btf: Add BTF_FMODEL_SIGNED_ARG flag
In reply to Ilya Leoshkevichs390x eBPF JIT needs to know whether a function return value is signed and which function arguments are signed, in order to generate code compliant with the s390x ABI.
--- include/linux/bpf.h | 4 ++++ include/linux/btf.h | 15 ++++++++++----- kernel/bpf/btf.c | 16 +++++++++++++++- 3 files changed, 29 insertions(+), 6 deletions(-)
include/linux/bpf.h +4 -0
@@ -899,8 +899,12 @@ enum bpf_cgroup_storage_type {/* The argument is a structure. */ #define BTF_FMODEL_STRUCT_ARG BIT(0) +/* The argument is signed. */ +#define BTF_FMODEL_SIGNED_ARG BIT(1) +struct btf_func_model {u8 ret_size; +u8 ret_flags;u8 nr_args;u8 arg_size[MAX_BPF_FUNC_ARGS ];u8 arg_flags[MAX_BPF_FUNC_ARGS ];
include/linux/btf.h +10 -5
@@ -236,6 +236,16 @@ static inline bool btf_type_is_small_int(const struct btf_type *t)return btf_type_is_int (t) && t->size <=sizeof (u64); } +static inline u8 btf_int_encoding (const struct btf_type *t) +{ +return BTF_INT_ENCODING (*(u32 *)(t +1 )); +} + +static inline bool btf_type_is_signed_int (const struct btf_type *t) +{ +return btf_type_is_int (t) && (btf_int_encoding (t) &BTF_INT_SIGNED ); +} +static inline bool btf_type_is_enum (const struct btf_type *t) {return BTF_INFO_KIND (t->info) ==BTF_KIND_ENUM ;
@@ -306,11 +316,6 @@ static inline u8 btf_int_offset(const struct btf_type *t)return BTF_INT_OFFSET (*(u32 *)(t +1 )); } -static inline u8 btf_int_encoding (const struct btf_type *t) -{ -return BTF_INT_ENCODING (*(u32 *)(t +1 )); -} -static inline bool btf_type_is_scalar (const struct btf_type *t) {return btf_type_is_int (t) ||btf_type_is_enum (t);
kernel/bpf/btf.c +15 -1
@@ -6453,6 +6453,18 @@ static int __get_type_size(struct btf *btf, u32 btf_id,return -EINVAL ; } +static u8 __get_type_fmodel_flags (const struct btf_type *t) +{ +u8 flags =0 ; + +if (__btf_type_is_struct (t)) + flags |=BTF_FMODEL_STRUCT_ARG ; +if (btf_type_is_signed_int (t)) + flags |=BTF_FMODEL_SIGNED_ARG ; + +return flags; +} + int btf_distill_func_proto(struct bpf_verifier_log *log, struct btf *btf, const struct btf_type *func,
@@ -6473,6 +6485,7 @@ int btf_distill_func_proto(struct bpf_verifier_log *log, m->arg_flags[i] =0 ; } m->ret_size =8 ; + m->ret_flags =0 ; m->nr_args =MAX_BPF_FUNC_REG_ARGS ;return 0 ; }
@@ -6492,6 +6505,7 @@ int btf_distill_func_proto(struct bpf_verifier_log *log,return -EINVAL ; } m->ret_size = ret; + m->ret_flags =__get_type_fmodel_flags (t);for (i =0 ; i < nargs; i++) {if (i == nargs -1 && args[i].type ==0 ) {
@@ -6516,7 +6530,7 @@ int btf_distill_func_proto(struct bpf_verifier_log *log,return -EINVAL ; } m->arg_size[i] = ret; - m->arg_flags[i] =__btf_type_is_struct (t) ?BTF_FMODEL_STRUCT_ARG :0 ; + m->arg_flags[i] =__get_type_fmodel_flags (t); } m->nr_args = nargs;return 0 ;
Signature · 1 line
2.39.1
Ilya Leoshkevich
maintainerPATCH v2 26/31 s390/bpf: Fix a typo in a comment
In reply to Ilya Leoshkevich"desription" should be "description".
--- arch/s390/net/bpf_jit_comp.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/s390/net/bpf_jit_comp.c +1 -1
@@ -510,7 +510,7 @@ static void bpf_skip(struct bpf_jit *jit, int size)* Emit function prologue * * Save registers and create stack frame if necessary. -* See stack frame layout desription in "bpf_jit.h"! +* See stack frame layout description in "bpf_jit.h"! */ static void bpf_jit_prologue (struct bpf_jit *jit,u32 stack_depth) {
Signature · 1 line
2.39.1
Ilya Leoshkevich
maintainerPATCH v2 27/31 s390/bpf: Add expoline to tail calls
In reply to Ilya LeoshkevichAll the indirect jumps in the eBPF JIT already use expolines, except for the tail call one.
--- arch/s390/net/bpf_jit_comp.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-)
arch/s390/net/bpf_jit_comp.c +10 -2
@@ -1393,8 +1393,16 @@ static noinline int bpf_jit_insn(struct bpf_jit *jit, struct bpf_prog *fp,/* lg %r1,bpf_func(%r1) */ EMIT6_DISP_LH (0xe3000000 ,0x0004 ,REG_1 ,REG_1 ,REG_0 , offsetof(struct bpf_prog , bpf_func)); -/* bc 0xf,tail_call_start(%r1) */ -_EMIT4 (0x47f01000 + jit->tail_call_start); +if (nospec_uses_trampoline ()) { + jit->seen |=SEEN_FUNC ; +/* aghi %r1,tail_call_start */ +EMIT4_IMM (0xa70b0000 ,REG_1 , jit->tail_call_start); +/* brcl 0xf,__s390_indirect_jump_r1 */ +EMIT6_PCREL_RILC (0xc0040000 ,0xf , jit->r1_thunk_ip); + }else { +/* bc 0xf,tail_call_start(%r1) */ +_EMIT4 (0x47f01000 + jit->tail_call_start); + }/* out: */ if (jit->prg_buf) { *(u16 *)(jit->prg_buf + patch_1_clrj + 2) =
Signature · 1 line
2.39.1
Ilya Leoshkevich
maintainerPATCH v2 28/31 s390/bpf: Implement bpf_arch_text_poke()
In reply to Ilya Leoshkevichbpf_arch_text_poke() is used to hotpatch eBPF programs and trampolines. s390x has a very strict hotpatching restriction: the only thing that is allowed to be hotpatched is conditional branch mask.
Take the same approach as commit de5012b41e5c ("s390/ftrace: implement hotpatching"): create a conditional jump to a "plt", which loads the target address from memory and jumps to it; then first patch this address, and then the mask.
Trampolines (introduced in the next patch) respect the ftrace calling
convention: the return address is in %r0, and %r1 is clobbered. With that in mind, bpf_arch_text_poke() does not differentiate between jumps and calls.
However, there is a simple optimization for jumps (for the epilogue_ip case): if a jump already points to the destination, then there is no "plt" and we can just flip the mask.
For simplicity, the "plt" template is defined in assembly, and its size is used to define C arrays. There doesn't seem to be a way to convey this size to C as a constant, so it's hardcoded and double-checked during runtime.
--- arch/s390/net/bpf_jit_comp.c | 97 ++++++++++++++++++++++++++++++++++++ 1 file changed, 97 insertions(+)
arch/s390/net/bpf_jit_comp.c +97 -0
@@ -30,6 +30,7 @@#include <asm/facility.h> #include <asm/nospec-branch.h> #include <asm/set_memory.h> +#include <asm/text-patching.h> #include "bpf_jit.h" struct bpf_jit {
@@ -50,6 +51,8 @@ struct bpf_jit {int r14_thunk_ip;/* Address of expoline thunk for 'br %r14' */ int tail_call_start;/* Tail call start offset */ int excnt;/* Number of exception table entries */ +int prologue_plt_ret;/* Return address for prologue hotpatch PLT */ +int prologue_plt;/* Start of prologue hotpatch PLT */ };#define SEEN_MEM BIT(0)/* use mem[] for temporary storage */
@@ -506,6 +509,36 @@ static void bpf_skip(struct bpf_jit *jit, int size) } } +/* +* PLT for hotpatchable calls. The calling convention is the same as for the +* ftrace hotpatch trampolines: %r0 is return address, %r1 is clobbered. +*/ +extern const char bpf_plt[]; +extern const char bpf_plt_ret[]; +extern const char bpf_plt_target[]; +extern const char bpf_plt_end[]; +#define BPF_PLT_SIZE 32 +asm( +".pushsection .rodata\n" +" .align 8\n" +"bpf_plt:\n" +" lgrl %r0,bpf_plt_ret\n" +" lgrl %r1,bpf_plt_target\n" +" br %r1\n" +" .align 8\n" +"bpf_plt_ret: .quad 0\n" +"bpf_plt_target: .quad 0\n" +"bpf_plt_end:\n" +" .popsection\n" +); + +static void bpf_jit_plt (void *plt,void *ret,void *target) +{ +memcpy (plt, bpf_plt,BPF_PLT_SIZE ); + *(void **)((char *)plt + (bpf_plt_ret - bpf_plt)) = ret; + *(void **)((char *)plt + (bpf_plt_target - bpf_plt)) = target; +} +/* * Emit function prologue *
@@ -514,6 +547,11 @@ static void bpf_skip(struct bpf_jit *jit, int size)*/ static void bpf_jit_prologue (struct bpf_jit *jit,u32 stack_depth) { +/* No-op for hotpatching */ +/* brcl 0,prologue_plt */ +EMIT6_PCREL_RILC (0xc0040000 ,0 , jit->prologue_plt); + jit->prologue_plt_ret = jit->prg; +if (jit->seen &SEEN_TAIL_CALL ) {/* xc STK_OFF_TCCNT(4,%r15),STK_OFF_TCCNT(%r15) */ _EMIT6 (0xd703f000 |STK_OFF_TCCNT ,0xf000 |STK_OFF_TCCNT );
@@ -589,6 +627,13 @@ static void bpf_jit_epilogue(struct bpf_jit *jit, u32 stack_depth)/* br %r1 */ _EMIT2 (0x07f1 ); } + + jit->prg =ALIGN (jit->prg,8 ); + jit->prologue_plt = jit->prg; +if (jit->prg_buf) +bpf_jit_plt (jit->prg_buf + jit->prg, + jit->prg_buf + jit->prologue_plt_ret,NULL ); + jit->prg +=BPF_PLT_SIZE ; }static int get_probe_mem_regno (const u8 *insn)
@@ -1776,6 +1821,9 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)struct bpf_jit jit;int pass; +if (WARN_ON_ONCE (bpf_plt_end - bpf_plt !=BPF_PLT_SIZE )) +return orig_fp; +if (!fp->jit_requested)return orig_fp;
@@ -1867,3 +1915,52 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp) tmp : orig_fp);return fp; } + +int bpf_arch_text_poke (void *ip,enum bpf_text_poke_type t, +void *old_addr,void *new_addr) +{ +struct { +u16 opc; +s32 disp; + } __packed insn; +char expected_plt[BPF_PLT_SIZE ]; +char current_plt[BPF_PLT_SIZE ]; +char *plt; +int err; + +/* Verify the branch to be patched. */ + err =copy_from_kernel_nofault (&insn, ip,sizeof (insn)); +if (err <0 ) +return err; +if (insn.opc != (0xc004 | (old_addr ?0xf0 :0 ))) +return -EINVAL ; + +if (t ==BPF_MOD_JUMP && + insn.disp == ((char *)new_addr - (char *)ip) >>1 ) { +/* +* The branch already points to the destination, +* there is no PLT. +*/ + }else { +/* Verify the PLT. */ + plt = (char *)ip + (insn.disp <<1 ); + err =copy_from_kernel_nofault (current_plt, plt,BPF_PLT_SIZE ); +if (err <0 ) +return err; +bpf_jit_plt (expected_plt, (char *)ip +6 , old_addr); +if (memcmp (current_plt, expected_plt,BPF_PLT_SIZE )) +return -EINVAL ; +/* Adjust the call address. */ +s390_kernel_write (plt + (bpf_plt_target - bpf_plt), + &new_addr,sizeof (void *)); + } + +/* Adjust the mask of the branch. */ + insn.opc =0xc004 | (new_addr ?0xf0 :0 ); +s390_kernel_write ((char *)ip +1 , (char *)&insn.opc +1 ,1 ); + +/* Make the new code visible to the other CPUs. */ +text_poke_sync_lock (); + +return 0 ; +}
Signature · 1 line
2.39.1
Ilya Leoshkevich
maintainerPATCH v2 29/31 s390/bpf: Implement arch_prepare_bpf_trampoline()
In reply to Ilya Leoshkevicharch_prepare_bpf_trampoline() is used for direct attachment of eBPF programs to various places, bypassing kprobes. It's responsible for calling a number of eBPF programs before, instead and/or after whatever they are attached to.
Add a s390x implementation, paying attention to the following:
- Reuse the existing JIT infrastructure, where possible.
- Like the existing JIT, prefer making multiple passes instead of backpatching. Currently 2 passes is enough. If literal pool is introduced, this needs to be raised to 3. However, at the moment adding literal pool only makes the code larger. If branch shortening is introduced, the number of passes needs to be increased even further.
- Support both regular and ftrace calling conventions, depending on the trampoline flags.
- Use expolines for indirect calls.
- Handle the mismatch between the eBPF and the s390x ABIs.
- Sign-extend fmod_ret return values.
invoke_bpf_prog() produces about 120 bytes; it might be possible to slightly optimize this, but reaching 50 bytes, like on x86_64, looks
unrealistic: just loading cookie, __bpf_prog_enter, bpf_func, insnsi and __bpf_prog_exit as literals already takes at least 5 * 12 = 60 bytes, and we can't use relative addressing for most of them. Therefore, lower BPF_MAX_TRAMP_LINKS on s390x.
--- arch/s390/net/bpf_jit_comp.c | 541 +++++++++++++++++++++++++++++++++-- include/linux/bpf.h | 4 + 2 files changed, 523 insertions(+), 22 deletions(-)
arch/s390/net/bpf_jit_comp.c +519 -22
@@ -71,6 +71,10 @@ struct bpf_jit {#define REG_0 REG_W0/* Register 0 */ #define REG_1 REG_W1/* Register 1 */ #define REG_2 BPF_REG_1/* Register 2 */ +#define REG_3 BPF_REG_2/* Register 3 */ +#define REG_4 BPF_REG_3/* Register 4 */ +#define REG_7 BPF_REG_6/* Register 7 */ +#define REG_8 BPF_REG_7/* Register 8 */ #define REG_14 BPF_REG_0/* Register 14 */ /*
@@ -595,6 +599,43 @@ static void bpf_jit_prologue(struct bpf_jit *jit, u32 stack_depth) } } +/* +* Emit an expoline for a jump that follows +*/ +static void emit_expoline (struct bpf_jit *jit) +{ +/* exrl %r0,.+10 */ +EMIT6_PCREL_RIL (0xc6000000 , jit->prg +10 ); +/* j . */ +EMIT4_PCREL (0xa7f40000 ,0 ); +} + +/* +* Emit __s390_indirect_jump_r1 thunk if necessary +*/ +static void emit_r1_thunk (struct bpf_jit *jit) +{ +if (nospec_uses_trampoline ()) { + jit->r1_thunk_ip = jit->prg; +emit_expoline (jit); +/* br %r1 */ +_EMIT2 (0x07f1 ); + } +} + +/* +* Call r1 either directly or via __s390_indirect_jump_r1 thunk +*/ +static void call_r1 (struct bpf_jit *jit) +{ +if (nospec_uses_trampoline ()) +/* brasl %r14,__s390_indirect_jump_r1 */ +EMIT6_PCREL_RILB (0xc0050000 ,REG_14 , jit->r1_thunk_ip); +else +/* basr %r14,%r1 */ +EMIT2 (0x0d00 ,REG_14 ,REG_1 ); +} +/* * Function epilogue */
@@ -608,25 +649,13 @@ static void bpf_jit_epilogue(struct bpf_jit *jit, u32 stack_depth)if (nospec_uses_trampoline ()) { jit->r14_thunk_ip = jit->prg;/* Generate __s390_indirect_jump_r14 thunk */ -/* exrl %r0,.+10 */ -EMIT6_PCREL_RIL (0xc6000000 , jit->prg +10 ); -/* j . */ -EMIT4_PCREL (0xa7f40000 ,0 ); +emit_expoline (jit); }/* br %r14 */ _EMIT2 (0x07fe ); -if ((nospec_uses_trampoline ()) && - (is_first_pass (jit) || (jit->seen &SEEN_FUNC ))) { - jit->r1_thunk_ip = jit->prg; -/* Generate __s390_indirect_jump_r1 thunk */ -/* exrl %r0,.+10 */ -EMIT6_PCREL_RIL (0xc6000000 , jit->prg +10 ); -/* j . */ -EMIT4_PCREL (0xa7f40000 ,0 ); -/* br %r1 */ -_EMIT2 (0x07f1 ); - } +if (is_first_pass (jit) || (jit->seen &SEEN_FUNC )) +emit_r1_thunk (jit); jit->prg =ALIGN (jit->prg,8 ); jit->prologue_plt = jit->prg;
@@ -707,6 +736,34 @@ static int bpf_jit_probe_mem(struct bpf_jit *jit, struct bpf_prog *fp,return 0 ; } +/* +* Sign-extend the register if necessary +*/ +static int sign_extend (struct bpf_jit *jit,int r,u8 size,u8 flags) +{ +if (!(flags &BTF_FMODEL_SIGNED_ARG )) +return 0 ; + +switch (size) { +case 1 : +/* lgbr %r,%r */ +EMIT4 (0xb9060000 , r, r); +return 0 ; +case 2 : +/* lghr %r,%r */ +EMIT4 (0xb9070000 , r, r); +return 0 ; +case 4 : +/* lgfr %r,%r */ +EMIT4 (0xb9140000 , r, r); +return 0 ; +case 8 : +return 0 ; +default : +return -1 ; + } +} +/* * Compile one eBPF instruction into s390x code *
@@ -1355,13 +1412,8 @@ static noinline int bpf_jit_insn(struct bpf_jit *jit, struct bpf_prog *fp, jit->seen |=SEEN_FUNC ;/* lgrl %w1,func */ EMIT6_PCREL_RILB (0xc4080000 ,REG_W1 ,_EMIT_CONST_U64 (func)); -if (nospec_uses_trampoline ()) { -/* brasl %r14,__s390_indirect_jump_r1 */ -EMIT6_PCREL_RILB (0xc0050000 ,REG_14 , jit->r1_thunk_ip); - }else { -/* basr %r14,%w1 */ -EMIT2 (0x0d00 ,REG_14 ,REG_W1 ); - } +/* %r1() */ +call_r1 (jit);/* lgr %b0,%r2: load return value into %b0 */ EMIT4 (0xb9040000 ,BPF_REG_0 ,REG_2 );break ;
@@ -1964,3 +2016,448 @@ int bpf_arch_text_poke(void *ip, enum bpf_text_poke_type t,return 0 ; } + +struct bpf_tramp_jit { +struct bpf_jit common; +int orig_stack_args_off;/* Offset of arguments placed on stack by the +* func_addr's original caller +*/ +int stack_size;/* Trampoline stack size */ +int stack_args_off;/* Offset of stack arguments for calling +* func_addr, has to be at the top +*/ +int reg_args_off;/* Offset of register arguments for calling +* func_addr +*/ +int ip_off;/* For bpf_get_func_ip(), has to be at +* (ctx - 16) +*/ +int arg_cnt_off;/* For bpf_get_func_arg_cnt(), has to be at +* (ctx - 8) +*/ +int bpf_args_off;/* Offset of BPF_PROG context, which consists +* of BPF arguments followed by return value +*/ +int retval_off;/* Offset of return value (see above) */ +int r7_r8_off;/* Offset of saved %r7 and %r8, which are used +* for __bpf_prog_enter() return value and +* func_addr respectively +*/ +int r14_off;/* Offset of saved %r14 */ +int run_ctx_off;/* Offset of struct bpf_tramp_run_ctx */ +int do_fexit;/* do_fexit: label */ +}; + +static void load_imm64 (struct bpf_jit *jit,int dst_reg,u64 val) +{ +/* llihf %dst_reg,val_hi */ +EMIT6_IMM (0xc00e0000 , dst_reg, (val >>32 )); +/* oilf %rdst_reg,val_lo */ +EMIT6_IMM (0xc00d0000 , dst_reg, val); +} + +static int invoke_bpf_prog (struct bpf_tramp_jit *tjit, +const struct btf_func_model *m, +struct bpf_tramp_link *tlink,bool save_ret) +{ +struct bpf_jit *jit = &tjit->common; +int cookie_off = tjit->run_ctx_off + + offsetof(struct bpf_tramp_run_ctx , bpf_cookie); +struct bpf_prog *p = tlink->link.prog; +int patch; + +/* +* run_ctx.cookie = tlink->cookie; +*/ + +/* %r0 = tlink->cookie */ +load_imm64 (jit,REG_W0 , tlink->cookie); +/* stg %r0,cookie_off(%r15) */ +EMIT6_DISP_LH (0xe3000000 ,0x0024 ,REG_W0 ,REG_0 ,REG_15 , cookie_off); + +/* +* if ((start = __bpf_prog_enter(p, &run_ctx)) == 0) +* goto skip; +*/ + +/* %r1 = __bpf_prog_enter */ +load_imm64 (jit,REG_1 , (u64 )bpf_trampoline_enter (p)); +/* %r2 = p */ +load_imm64 (jit,REG_2 , (u64 )p); +/* la %r3,run_ctx_off(%r15) */ +EMIT4_DISP (0x41000000 ,REG_3 ,REG_15 , tjit->run_ctx_off); +/* %r1() */ +call_r1 (jit); +/* ltgr %r7,%r2 */ +EMIT4 (0xb9020000 ,REG_7 ,REG_2 ); +/* brcl 8,skip */ + patch = jit->prg; +EMIT6_PCREL_RILC (0xc0040000 ,8 ,0 ); + +/* +* retval = bpf_func(args, p->insnsi); +*/ + +/* %r1 = p->bpf_func */ +load_imm64 (jit,REG_1 , (u64 )p->bpf_func); +/* la %r2,bpf_args_off(%r15) */ +EMIT4_DISP (0x41000000 ,REG_2 ,REG_15 , tjit->bpf_args_off); +/* %r3 = p->insnsi */ +if (!p->jited) +load_imm64 (jit,REG_3 , (u64 )p->insnsi); +/* %r1() */ +call_r1 (jit); +/* stg %r2,retval_off(%r15) */ +if (save_ret) { +if (sign_extend (jit,REG_2 , m->ret_size, m->ret_flags)) +return -1 ; +EMIT6_DISP_LH (0xe3000000 ,0x0024 ,REG_2 ,REG_0 ,REG_15 , + tjit->retval_off); + } + +/* skip: */ +if (jit->prg_buf) + *(u32 *)&jit->prg_buf[patch +2 ] = (jit->prg - patch) >>1 ; + +/* +* __bpf_prog_exit(p, start, &run_ctx); +*/ + +/* %r1 = __bpf_prog_exit */ +load_imm64 (jit,REG_1 , (u64 )bpf_trampoline_exit (p)); +/* %r2 = p */ +load_imm64 (jit,REG_2 , (u64 )p); +/* lgr %r3,%r7 */ +EMIT4 (0xb9040000 ,REG_3 ,REG_7 ); +/* la %r4,run_ctx_off(%r15) */ +EMIT4_DISP (0x41000000 ,REG_4 ,REG_15 , tjit->run_ctx_off); +/* %r1() */ +call_r1 (jit); + +return 0 ; +} + +static int alloc_stack (struct bpf_tramp_jit *tjit,size_t size) +{ +int stack_offset = tjit->stack_size; + + tjit->stack_size += size; +return stack_offset; +} + +/* ABI uses %r2 - %r6 for parameter passing. */ +#define MAX_NR_REG_ARGS 5 + +/* The "L" field of the "mvc" instruction is 8 bits. */ +#define MAX_MVC_SIZE 256 +#define MAX_NR_STACK_ARGS (MAX_MVC_SIZE / sizeof(u64)) + +/* -mfentry generates a 6-byte nop on s390x. */ +#define S390X_PATCH_SIZE 6 + +int __arch_prepare_bpf_trampoline (struct bpf_tramp_image *im, +struct bpf_tramp_jit *tjit, +const struct btf_func_model *m, +u32 flags,struct bpf_tramp_links *tlinks, +void *func_addr) +{ +struct bpf_tramp_links *fmod_ret = &tlinks[BPF_TRAMP_MODIFY_RETURN ]; +struct bpf_tramp_links *fentry = &tlinks[BPF_TRAMP_FENTRY ]; +struct bpf_tramp_links *fexit = &tlinks[BPF_TRAMP_FEXIT ]; +int nr_bpf_args, nr_reg_args, nr_stack_args; +struct bpf_jit *jit = &tjit->common; +int arg, bpf_arg_off; +int i, j; + +/* Support as many stack arguments as "mvc" instruction can handle. */ + nr_reg_args =min_t (int, m->nr_args,MAX_NR_REG_ARGS ); + nr_stack_args = m->nr_args - nr_reg_args; +if (nr_stack_args >MAX_NR_STACK_ARGS ) +return -ENOTSUPP ; + +/* Return to %r14, since func_addr and %r0 are not available. */ +if (!func_addr && !(flags &BPF_TRAMP_F_ORIG_STACK )) + flags |=BPF_TRAMP_F_SKIP_FRAME ; + +/* +* Compute how many arguments we need to pass to BPF programs. +* BPF ABI mirrors that of x86_64: arguments that are 16 bytes or +* smaller are packed into 1 or 2 registers; larger arguments are +* passed via pointers. +* In s390x ABI, arguments that are 8 bytes or smaller are packed into +* a register; larger arguments are passed via pointers. +* We need to deal with this difference. +*/ + nr_bpf_args =0 ; +for (i =0 ; i < m->nr_args; i++) { +if (m->arg_size[i] <=8 ) + nr_bpf_args +=1 ; +else if (m->arg_size[i] <=16 ) + nr_bpf_args +=2 ; +else +return -ENOTSUPP ; + } + +/* +* Calculate the stack layout. +*/ + +/* Reserve STACK_FRAME_OVERHEAD bytes for the callees. */ + tjit->stack_size =STACK_FRAME_OVERHEAD ; + tjit->stack_args_off =alloc_stack (tjit, nr_stack_args *sizeof (u64)); + tjit->reg_args_off =alloc_stack (tjit, nr_reg_args *sizeof (u64)); + tjit->ip_off =alloc_stack (tjit,sizeof (u64)); + tjit->arg_cnt_off =alloc_stack (tjit,sizeof (u64)); + tjit->bpf_args_off =alloc_stack (tjit, nr_bpf_args *sizeof (u64)); + tjit->retval_off =alloc_stack (tjit,sizeof (u64)); + tjit->r7_r8_off =alloc_stack (tjit,2 *sizeof (u64)); + tjit->r14_off =alloc_stack (tjit,sizeof (u64)); + tjit->run_ctx_off =alloc_stack (tjit, +sizeof (struct bpf_tramp_run_ctx )); +/* The caller has already reserved STACK_FRAME_OVERHEAD bytes. */ + tjit->stack_size -=STACK_FRAME_OVERHEAD ; + tjit->orig_stack_args_off = tjit->stack_size +STACK_FRAME_OVERHEAD ; + +/* aghi %r15,-stack_size */ +EMIT4_IMM (0xa70b0000 ,REG_15 , -tjit->stack_size); +/* stmg %r2,%rN,fwd_reg_args_off(%r15) */ +if (nr_reg_args) +EMIT6_DISP_LH (0xeb000000 ,0x0024 ,REG_2 , +REG_2 + (nr_reg_args -1 ),REG_15 , + tjit->reg_args_off); +for (i =0 , j =0 ; i < m->nr_args; i++) { +if (i <MAX_NR_REG_ARGS ) + arg =REG_2 + i; +else + arg = tjit->orig_stack_args_off + + (i -MAX_NR_REG_ARGS ) *sizeof (u64); + bpf_arg_off = tjit->bpf_args_off + j *sizeof (u64); +if (m->arg_size[i] <=8 ) { +if (i <MAX_NR_REG_ARGS ) +/* stg %arg,bpf_arg_off(%r15) */ +EMIT6_DISP_LH (0xe3000000 ,0x0024 , arg, +REG_0 ,REG_15 , bpf_arg_off); +else +/* mvc bpf_arg_off(8,%r15),arg(%r15) */ +_EMIT6 (0xd207f000 | bpf_arg_off, +0xf000 | arg); + j +=1 ; + }else { +if (i <MAX_NR_REG_ARGS ) { +/* mvc bpf_arg_off(16,%r15),0(%arg) */ +_EMIT6 (0xd20ff000 | bpf_arg_off, + reg2hex[arg] <<12 ); + }else { +/* lg %r1,arg(%r15) */ +EMIT6_DISP_LH (0xe3000000 ,0x0004 ,REG_1 ,REG_0 , +REG_15 , arg); +/* mvc bpf_arg_off(16,%r15),0(%r1) */ +_EMIT6 (0xd20ff000 | bpf_arg_off,0x1000 ); + } + j +=2 ; + } + } +/* stmg %r7,%r8,r7_r8_off(%r15) */ +EMIT6_DISP_LH (0xeb000000 ,0x0024 ,REG_7 ,REG_8 ,REG_15 , + tjit->r7_r8_off); +/* stg %r14,r14_off(%r15) */ +EMIT6_DISP_LH (0xe3000000 ,0x0024 ,REG_14 ,REG_0 ,REG_15 , tjit->r14_off); + +if (flags &BPF_TRAMP_F_ORIG_STACK ) { +/* +* The ftrace trampoline puts the return address (which is the +* address of the original function + S390X_PATCH_SIZE) into +* %r0; see ftrace_shared_hotpatch_trampoline_br and +* ftrace_init_nop() for details. +*/ + +/* lgr %r8,%r0 */ +EMIT4 (0xb9040000 ,REG_8 ,REG_0 ); + }else { +/* %r8 = func_addr + S390X_PATCH_SIZE */ +load_imm64 (jit,REG_8 , (u64 )func_addr +S390X_PATCH_SIZE ); + } + +/* +* ip = func_addr; +* arg_cnt = m->nr_args; +*/ + +if (flags &BPF_TRAMP_F_IP_ARG ) { +/* %r0 = func_addr */ +load_imm64 (jit,REG_0 , (u64 )func_addr); +/* stg %r0,ip_off(%r15) */ +EMIT6_DISP_LH (0xe3000000 ,0x0024 ,REG_0 ,REG_0 ,REG_15 , + tjit->ip_off); + } +/* lghi %r0,nr_bpf_args */ +EMIT4_IMM (0xa7090000 ,REG_0 , nr_bpf_args); +/* stg %r0,arg_cnt_off(%r15) */ +EMIT6_DISP_LH (0xe3000000 ,0x0024 ,REG_0 ,REG_0 ,REG_15 , + tjit->arg_cnt_off); + +if (flags &BPF_TRAMP_F_CALL_ORIG ) { +/* +* __bpf_tramp_enter(im); +*/ + +/* %r1 = __bpf_tramp_enter */ +load_imm64 (jit,REG_1 , (u64 )__bpf_tramp_enter); +/* %r2 = im */ +load_imm64 (jit,REG_2 , (u64 )im); +/* %r1() */ +call_r1 (jit); + } + +for (i =0 ; i < fentry->nr_links; i++) +if (invoke_bpf_prog (tjit, m, fentry->links[i], + flags &BPF_TRAMP_F_RET_FENTRY_RET )) +return -EINVAL ; + +if (fmod_ret->nr_links) { +/* +* retval = 0; +*/ + +/* xc retval_off(8,%r15),retval_off(%r15) */ +_EMIT6 (0xd707f000 | tjit->retval_off, +0xf000 | tjit->retval_off); + +for (i =0 ; i < fmod_ret->nr_links; i++) { +if (invoke_bpf_prog (tjit, m, fmod_ret->links[i], true)) +return -EINVAL ; + +/* +* if (retval) +* goto do_fexit; +*/ + +/* ltg %r0,retval_off(%r15) */ +EMIT6_DISP_LH (0xe3000000 ,0x0002 ,REG_0 ,REG_0 ,REG_15 , + tjit->retval_off); +/* brcl 7,do_fexit */ +EMIT6_PCREL_RILC (0xc0040000 ,7 , tjit->do_fexit); + } + } + +if (flags &BPF_TRAMP_F_CALL_ORIG ) { +/* +* retval = func_addr(args); +*/ + +/* lmg %r2,%rN,reg_args_off(%r15) */ +if (nr_reg_args) +EMIT6_DISP_LH (0xeb000000 ,0x0004 ,REG_2 , +REG_2 + (nr_reg_args -1 ),REG_15 , + tjit->reg_args_off); +/* mvc stack_args_off(N,%r15),orig_stack_args_off(%r15) */ +if (nr_stack_args) +_EMIT6 (0xd200f000 | + (nr_stack_args *sizeof (u64) -1 ) <<16 | + tjit->stack_args_off, +0xf000 | tjit->orig_stack_args_off); +/* lgr %r1,%r8 */ +EMIT4 (0xb9040000 ,REG_1 ,REG_8 ); +/* %r1() */ +call_r1 (jit); +/* stg %r2,retval_off(%r15) */ +EMIT6_DISP_LH (0xe3000000 ,0x0024 ,REG_2 ,REG_0 ,REG_15 , + tjit->retval_off); + + im->ip_after_call = jit->prg_buf + jit->prg; + +/* +* The following nop will be patched by bpf_tramp_image_put(). +*/ + +/* brcl 0,im->ip_epilogue */ +EMIT6_PCREL_RILC (0xc0040000 ,0 , (u64 )im->ip_epilogue); + } + +/* do_fexit: */ + tjit->do_fexit = jit->prg; +for (i =0 ; i < fexit->nr_links; i++) +if (invoke_bpf_prog (tjit, m, fexit->links[i], false)) +return -EINVAL ; + +if (flags &BPF_TRAMP_F_CALL_ORIG ) { + im->ip_epilogue = jit->prg_buf + jit->prg; + +/* +* __bpf_tramp_exit(im); +*/ + +/* %r1 = __bpf_tramp_exit */ +load_imm64 (jit,REG_1 , (u64 )__bpf_tramp_exit); +/* %r2 = im */ +load_imm64 (jit,REG_2 , (u64 )im); +/* %r1() */ +call_r1 (jit); + } + +/* lmg %r2,%rN,reg_args_off(%r15) */ +if ((flags &BPF_TRAMP_F_RESTORE_REGS ) && nr_reg_args) +EMIT6_DISP_LH (0xeb000000 ,0x0004 ,REG_2 , +REG_2 + (nr_reg_args -1 ),REG_15 , + tjit->reg_args_off); +/* lgr %r1,%r8 */ +if (!(flags &BPF_TRAMP_F_SKIP_FRAME )) +EMIT4 (0xb9040000 ,REG_1 ,REG_8 ); +/* lmg %r7,%r8,r7_r8_off(%r15) */ +EMIT6_DISP_LH (0xeb000000 ,0x0004 ,REG_7 ,REG_8 ,REG_15 , + tjit->r7_r8_off); +/* lg %r14,r14_off(%r15) */ +EMIT6_DISP_LH (0xe3000000 ,0x0004 ,REG_14 ,REG_0 ,REG_15 , tjit->r14_off); +/* lg %r2,retval_off(%r15) */ +if (flags & (BPF_TRAMP_F_CALL_ORIG |BPF_TRAMP_F_RET_FENTRY_RET )) +EMIT6_DISP_LH (0xe3000000 ,0x0004 ,REG_2 ,REG_0 ,REG_15 , + tjit->retval_off); +/* aghi %r15,stack_size */ +EMIT4_IMM (0xa70b0000 ,REG_15 , tjit->stack_size); +/* Emit an expoline for the following indirect jump. */ +if (nospec_uses_trampoline ()) +emit_expoline (jit); +if (flags &BPF_TRAMP_F_SKIP_FRAME ) +/* br %r14 */ +_EMIT2 (0x07fe ); +else +/* br %r1 */ +_EMIT2 (0x07f1 ); + +emit_r1_thunk (jit); + +return 0 ; +} + +int arch_prepare_bpf_trampoline (struct bpf_tramp_image *im,void *image, +void *image_end,const struct btf_func_model *m, +u32 flags,struct bpf_tramp_links *tlinks, +void *func_addr) +{ +struct bpf_tramp_jit tjit; +int ret; +int i; + +for (i =0 ; i <2 ; i++) { +if (i ==0 ) { +/* Compute offsets, check whether the code fits. */ +memset (&tjit,0 ,sizeof (tjit)); + }else { +/* Generate the code. */ + tjit.common.prg =0 ; + tjit.common.prg_buf = image; + } + ret =__arch_prepare_bpf_trampoline (im, &tjit, m, flags, + tlinks, func_addr); +if (ret <0 ) +return ret; +if (tjit.common.prg > (char *)image_end - (char *)image) +/* +* Use the same error code as for exceeding +* BPF_MAX_TRAMP_LINKS. +*/ +return -E2BIG ; + } + +return ret; +}
include/linux/bpf.h +4 -0
@@ -944,7 +944,11 @@ struct btf_func_model {* bytes on x86. */ enum { +#if defined(__s390x__) +BPF_MAX_TRAMP_LINKS =27 , +#else BPF_MAX_TRAMP_LINKS =38 , +#endif };struct bpf_tramp_links {
Signature · 1 line
2.39.1
kernel test robot
build botRe: [PATCH bpf-next v2 29/31] s390/bpf: Implement arch_prepare_bpf_trampoline()
I love your patch! Perhaps something to improve:
Hi Ilya,
I love your patch! Perhaps something to improve:
[auto build test WARNING on bpf-next/master]
url: https://github.com/intel-lab-lkp/linux/commits/Ilya-Leoshkevich/bpf-Use-ARG_CONST_SIZE_OR_ZERO-for-3rd-argument-of-bpf_tcp_raw_gen_syncookie_ipv-4-6/20230128-143920 base: https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git master patch link: https://lore.kernel.org/r/20230128000650.1516334-30-iii%40linux.ibm.com patch subject: [PATCH bpf-next v2 29/31] s390/bpf: Implement arch_prepare_bpf_trampoline() config: s390-allyesconfig (https://download.01.org/0day-ci/archive/20230128/[email protected]/config) compiler: s390-linux-gcc (GCC) 12.1.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# https://github.com/intel-lab-lkp/linux/commit/067ec74d790af1fd8c02b8d3571cf743d53e3656
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review Ilya-Leoshkevich/bpf-Use-ARG_CONST_SIZE_OR_ZERO-for-3rd-argument-of-bpf_tcp_raw_gen_syncookie_ipv-4-6/20230128-143920 git checkout 067ec74d790af1fd8c02b8d3571cf743d53e3656
# save the config file
mkdir build_dir && cp config build_dir/.config
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=s390 olddefconfig COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=s390 SHELL=/bin/bash arch/s390/net/
If you fix the issue, kindly add following tag where applicable | Reported-by: kernel test robot <[email protected]>
All warnings (new ones prefixed by >>):
>> arch/s390/net/bpf_jit_comp.c:2158:5: warning: no previous prototype for '__arch_prepare_bpf_trampoline' [-Wmissing-prototypes]
2158 | int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
vim +/__arch_prepare_bpf_trampoline +2158 arch/s390/net/bpf_jit_comp.c
2157
> 2158 int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, 2159 struct bpf_tramp_jit *tjit, 2160 const struct btf_func_model *m, 2161 u32 flags, struct bpf_tramp_links *tlinks, 2162 void *func_addr)
2163 {
2164 struct bpf_tramp_links *fmod_ret = &tlinks[BPF_TRAMP_MODIFY_RETURN];
2165 struct bpf_tramp_links *fentry = &tlinks[BPF_TRAMP_FENTRY];
2166 struct bpf_tramp_links *fexit = &tlinks[BPF_TRAMP_FEXIT];
2167 int nr_bpf_args, nr_reg_args, nr_stack_args;
2168 struct bpf_jit *jit = &tjit->common;
2169 int arg, bpf_arg_off;
2170 int i, j;
2171
2172 /* Support as many stack arguments as "mvc" instruction can handle. */
2173 nr_reg_args = min_t(int, m->nr_args, MAX_NR_REG_ARGS);
2174 nr_stack_args = m->nr_args - nr_reg_args;
2175 if (nr_stack_args > MAX_NR_STACK_ARGS)
2176 return -ENOTSUPP;
2177
2178 /* Return to %r14, since func_addr and %r0 are not available. */
2179 if (!func_addr && !(flags & BPF_TRAMP_F_ORIG_STACK))
2180 flags |= BPF_TRAMP_F_SKIP_FRAME;
2181
2182 /*
2183 * Compute how many arguments we need to pass to BPF programs.
2184 * BPF ABI mirrors that of x86_64: arguments that are 16 bytes or
2185 * smaller are packed into 1 or 2 registers; larger arguments are
2186 * passed via pointers.
2187 * In s390x ABI, arguments that are 8 bytes or smaller are packed into
2188 * a register; larger arguments are passed via pointers.
2189 * We need to deal with this difference.
2190 */
2191 nr_bpf_args = 0;
2192 for (i = 0; i < m->nr_args; i++) {
2193 if (m->arg_size[i] <= 8)
2194 nr_bpf_args += 1;
2195 else if (m->arg_size[i] <= 16)
2196 nr_bpf_args += 2;
2197 else
2198 return -ENOTSUPP;
2199 }
2200
2201 /*
2202 * Calculate the stack layout.
2203 */
2204
2205 /* Reserve STACK_FRAME_OVERHEAD bytes for the callees. */
2206 tjit->stack_size = STACK_FRAME_OVERHEAD;
2207 tjit->stack_args_off = alloc_stack(tjit, nr_stack_args * sizeof(u64));
2208 tjit->reg_args_off = alloc_stack(tjit, nr_reg_args * sizeof(u64));
2209 tjit->ip_off = alloc_stack(tjit, sizeof(u64));
2210 tjit->arg_cnt_off = alloc_stack(tjit, sizeof(u64));
2211 tjit->bpf_args_off = alloc_stack(tjit, nr_bpf_args * sizeof(u64));
2212 tjit->retval_off = alloc_stack(tjit, sizeof(u64));
2213 tjit->r7_r8_off = alloc_stack(tjit, 2 * sizeof(u64));
2214 tjit->r14_off = alloc_stack(tjit, sizeof(u64));
2215 tjit->run_ctx_off = alloc_stack(tjit,
2216 sizeof(struct bpf_tramp_run_ctx));
2217 /* The caller has already reserved STACK_FRAME_OVERHEAD bytes. */
2218 tjit->stack_size -= STACK_FRAME_OVERHEAD;
2219 tjit->orig_stack_args_off = tjit->stack_size + STACK_FRAME_OVERHEAD;
2220
2221 /* aghi %r15,-stack_size */
2222 EMIT4_IMM(0xa70b0000, REG_15, -tjit->stack_size);
2223 /* stmg %r2,%rN,fwd_reg_args_off(%r15) */
2224 if (nr_reg_args)
2225 EMIT6_DISP_LH(0xeb000000, 0x0024, REG_2,
2226 REG_2 + (nr_reg_args - 1), REG_15,
2227 tjit->reg_args_off);
2228 for (i = 0, j = 0; i < m->nr_args; i++) {
2229 if (i < MAX_NR_REG_ARGS)
2230 arg = REG_2 + i;
2231 else
2232 arg = tjit->orig_stack_args_off +
2233 (i - MAX_NR_REG_ARGS) * sizeof(u64);
2234 bpf_arg_off = tjit->bpf_args_off + j * sizeof(u64);
2235 if (m->arg_size[i] <= 8) {
2236 if (i < MAX_NR_REG_ARGS)
2237 /* stg %arg,bpf_arg_off(%r15) */
2238 EMIT6_DISP_LH(0xe3000000, 0x0024, arg,
2239 REG_0, REG_15, bpf_arg_off);
2240 else
2241 /* mvc bpf_arg_off(8,%r15),arg(%r15) */
2242 _EMIT6(0xd207f000 | bpf_arg_off,
2243 0xf000 | arg);
2244 j += 1;
2245 } else {
2246 if (i < MAX_NR_REG_ARGS) {
2247 /* mvc bpf_arg_off(16,%r15),0(%arg) */
2248 _EMIT6(0xd20ff000 | bpf_arg_off,
2249 reg2hex[arg] << 12);
2250 } else {
2251 /* lg %r1,arg(%r15) */
2252 EMIT6_DISP_LH(0xe3000000, 0x0004, REG_1, REG_0,
2253 REG_15, arg);
2254 /* mvc bpf_arg_off(16,%r15),0(%r1) */
2255 _EMIT6(0xd20ff000 | bpf_arg_off, 0x1000);
2256 }
2257 j += 2;
2258 }
2259 }
2260 /* stmg %r7,%r8,r7_r8_off(%r15) */
2261 EMIT6_DISP_LH(0xeb000000, 0x0024, REG_7, REG_8, REG_15,
2262 tjit->r7_r8_off);
2263 /* stg %r14,r14_off(%r15) */
2264 EMIT6_DISP_LH(0xe3000000, 0x0024, REG_14, REG_0, REG_15, tjit->r14_off);
2265
2266 if (flags & BPF_TRAMP_F_ORIG_STACK) {
2267 /*
2268 * The ftrace trampoline puts the return address (which is the
2269 * address of the original function + S390X_PATCH_SIZE) into
2270 * %r0; see ftrace_shared_hotpatch_trampoline_br and
2271 * ftrace_init_nop() for details.
2272 */
2273
2274 /* lgr %r8,%r0 */
2275 EMIT4(0xb9040000, REG_8, REG_0);
2276 } else {
2277 /* %r8 = func_addr + S390X_PATCH_SIZE */
2278 load_imm64(jit, REG_8, (u64)func_addr + S390X_PATCH_SIZE);
2279 }
2280
2281 /*
2282 * ip = func_addr;
2283 * arg_cnt = m->nr_args;
2284 */
2285
2286 if (flags & BPF_TRAMP_F_IP_ARG) {
2287 /* %r0 = func_addr */
2288 load_imm64(jit, REG_0, (u64)func_addr);
2289 /* stg %r0,ip_off(%r15) */
2290 EMIT6_DISP_LH(0xe3000000, 0x0024, REG_0, REG_0, REG_15,
2291 tjit->ip_off);
2292 }
2293 /* lghi %r0,nr_bpf_args */
2294 EMIT4_IMM(0xa7090000, REG_0, nr_bpf_args);
2295 /* stg %r0,arg_cnt_off(%r15) */
2296 EMIT6_DISP_LH(0xe3000000, 0x0024, REG_0, REG_0, REG_15,
2297 tjit->arg_cnt_off);
2298
2299 if (flags & BPF_TRAMP_F_CALL_ORIG) {
2300 /*
2301 * __bpf_tramp_enter(im);
2302 */
2303
2304 /* %r1 = __bpf_tramp_enter */
2305 load_imm64(jit, REG_1, (u64)__bpf_tramp_enter);
2306 /* %r2 = im */
2307 load_imm64(jit, REG_2, (u64)im);
2308 /* %r1() */
2309 call_r1(jit);
2310 }
2311
2312 for (i = 0; i < fentry->nr_links; i++)
2313 if (invoke_bpf_prog(tjit, m, fentry->links[i],
2314 flags & BPF_TRAMP_F_RET_FENTRY_RET))
2315 return -EINVAL;
2316
2317 if (fmod_ret->nr_links) {
2318 /*
2319 * retval = 0;
2320 */
2321
2322 /* xc retval_off(8,%r15),retval_off(%r15) */
2323 _EMIT6(0xd707f000 | tjit->retval_off,
2324 0xf000 | tjit->retval_off);
2325
2326 for (i = 0; i < fmod_ret->nr_links; i++) {
2327 if (invoke_bpf_prog(tjit, m, fmod_ret->links[i], true))
2328 return -EINVAL;
2329
2330 /*
2331 * if (retval)
2332 * goto do_fexit;
2333 */
2334
2335 /* ltg %r0,retval_off(%r15) */
2336 EMIT6_DISP_LH(0xe3000000, 0x0002, REG_0, REG_0, REG_15,
2337 tjit->retval_off);
2338 /* brcl 7,do_fexit */
2339 EMIT6_PCREL_RILC(0xc0040000, 7, tjit->do_fexit);
2340 }
2341 }
2342
2343 if (flags & BPF_TRAMP_F_CALL_ORIG) {
2344 /*
2345 * retval = func_addr(args);
2346 */
2347
2348 /* lmg %r2,%rN,reg_args_off(%r15) */
2349 if (nr_reg_args)
2350 EMIT6_DISP_LH(0xeb000000, 0x0004, REG_2,
2351 REG_2 + (nr_reg_args - 1), REG_15,
2352 tjit->reg_args_off);
2353 /* mvc stack_args_off(N,%r15),orig_stack_args_off(%r15) */
2354 if (nr_stack_args)
2355 _EMIT6(0xd200f000 |
2356 (nr_stack_args * sizeof(u64) - 1) << 16 |
2357 tjit->stack_args_off,
2358 0xf000 | tjit->orig_stack_args_off);
2359 /* lgr %r1,%r8 */
2360 EMIT4(0xb9040000, REG_1, REG_8);
2361 /* %r1() */
2362 call_r1(jit);
2363 /* stg %r2,retval_off(%r15) */
2364 EMIT6_DISP_LH(0xe3000000, 0x0024, REG_2, REG_0, REG_15,
2365 tjit->retval_off);
2366
2367 im->ip_after_call = jit->prg_buf + jit->prg;
2368
2369 /*
2370 * The following nop will be patched by bpf_tramp_image_put().
2371 */
2372
2373 /* brcl 0,im->ip_epilogue */
2374 EMIT6_PCREL_RILC(0xc0040000, 0, (u64)im->ip_epilogue);
2375 }
2376
2377 /* do_fexit: */
2378 tjit->do_fexit = jit->prg;
2379 for (i = 0; i < fexit->nr_links; i++)
2380 if (invoke_bpf_prog(tjit, m, fexit->links[i], false))
2381 return -EINVAL;
2382
2383 if (flags & BPF_TRAMP_F_CALL_ORIG) {
2384 im->ip_epilogue = jit->prg_buf + jit->prg;
2385
2386 /*
2387 * __bpf_tramp_exit(im);
2388 */
2389
2390 /* %r1 = __bpf_tramp_exit */
2391 load_imm64(jit, REG_1, (u64)__bpf_tramp_exit);
2392 /* %r2 = im */
2393 load_imm64(jit, REG_2, (u64)im);
2394 /* %r1() */
2395 call_r1(jit);
2396 }
2397
2398 /* lmg %r2,%rN,reg_args_off(%r15) */
2399 if ((flags & BPF_TRAMP_F_RESTORE_REGS) && nr_reg_args)
2400 EMIT6_DISP_LH(0xeb000000, 0x0004, REG_2,
2401 REG_2 + (nr_reg_args - 1), REG_15,
2402 tjit->reg_args_off);
2403 /* lgr %r1,%r8 */
2404 if (!(flags & BPF_TRAMP_F_SKIP_FRAME))
2405 EMIT4(0xb9040000, REG_1, REG_8);
2406 /* lmg %r7,%r8,r7_r8_off(%r15) */
2407 EMIT6_DISP_LH(0xeb000000, 0x0004, REG_7, REG_8, REG_15,
2408 tjit->r7_r8_off);
2409 /* lg %r14,r14_off(%r15) */
2410 EMIT6_DISP_LH(0xe3000000, 0x0004, REG_14, REG_0, REG_15, tjit->r14_off);
2411 /* lg %r2,retval_off(%r15) */
2412 if (flags & (BPF_TRAMP_F_CALL_ORIG | BPF_TRAMP_F_RET_FENTRY_RET))
2413 EMIT6_DISP_LH(0xe3000000, 0x0004, REG_2, REG_0, REG_15,
2414 tjit->retval_off);
2415 /* aghi %r15,stack_size */
2416 EMIT4_IMM(0xa70b0000, REG_15, tjit->stack_size);
2417 /* Emit an expoline for the following indirect jump. */
2418 if (nospec_uses_trampoline())
2419 emit_expoline(jit);
2420 if (flags & BPF_TRAMP_F_SKIP_FRAME)
2421 /* br %r14 */
2422 _EMIT2(0x07fe);
2423 else
2424 /* br %r1 */
2425 _EMIT2(0x07f1);
2426
2427 emit_r1_thunk(jit);
2428
2429 return 0;
2430 }
2431
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests
Ilya Leoshkevich
maintainerPATCH v2 30/31 s390/bpf: Implement bpf_jit_supports_subprog_tailcalls()
In reply to Ilya LeoshkevichAllow mixing subprogs and tail calls by passing the current tail call count to subprogs.
--- arch/s390/net/bpf_jit_comp.c | 37 ++++++++++++++++++++++++++---------- 1 file changed, 27 insertions(+), 10 deletions(-)
arch/s390/net/bpf_jit_comp.c +27 -10
@@ -58,7 +58,6 @@ struct bpf_jit {#define SEEN_MEM BIT(0)/* use mem[] for temporary storage */ #define SEEN_LITERAL BIT(1)/* code uses literals */ #define SEEN_FUNC BIT(2)/* calls C functions */ -#define SEEN_TAIL_CALL BIT(3)/* code uses tail calls */ #define SEEN_STACK (SEEN_FUNC | SEEN_MEM)/*
@@ -549,20 +548,23 @@ static void bpf_jit_plt(void *plt, void *ret, void *target)* Save registers and create stack frame if necessary. * See stack frame layout description in "bpf_jit.h"! */ -static void bpf_jit_prologue (struct bpf_jit *jit,u32 stack_depth) +static void bpf_jit_prologue (struct bpf_jit *jit,struct bpf_prog *fp, +u32 stack_depth) {/* No-op for hotpatching */ /* brcl 0,prologue_plt */ EMIT6_PCREL_RILC (0xc0040000 ,0 , jit->prologue_plt); jit->prologue_plt_ret = jit->prg; -if (jit->seen &SEEN_TAIL_CALL ) { +if (fp->aux->func_idx ==0 ) { +/* Initialize the tail call counter in the main program. */ /* xc STK_OFF_TCCNT(4,%r15),STK_OFF_TCCNT(%r15) */ _EMIT6 (0xd703f000 |STK_OFF_TCCNT ,0xf000 |STK_OFF_TCCNT ); }else {/* -* There are no tail calls. Insert nops in order to have -* tail_call_start at a predictable offset. +* Skip the tail call counter initialization in subprograms. +* Insert nops in order to have tail_call_start at a +* predictable offset. */ bpf_skip (jit,6 ); }
@@ -1410,6 +1412,19 @@ static noinline int bpf_jit_insn(struct bpf_jit *jit, struct bpf_prog *fp,REG_SET_SEEN (BPF_REG_5 ); jit->seen |=SEEN_FUNC ; +/* +* Copy the tail call counter to where the callee expects it. +* +* Note 1: The callee can increment the tail call counter, but +* we do not load it back, since the x86 JIT does not do this +* either. +* +* Note 2: We assume that the verifier does not let us call the +* main program, which clears the tail call counter on entry. +*/ +/* mvc STK_OFF_TCCNT(4,%r15),N(%r15) */ +_EMIT6 (0xd203f000 |STK_OFF_TCCNT , +0xf000 | (STK_OFF_TCCNT +STK_OFF + stack_depth));/* lgrl %w1,func */ EMIT6_PCREL_RILB (0xc4080000 ,REG_W1 ,_EMIT_CONST_U64 (func));/* %r1() */
@@ -1426,10 +1441,7 @@ static noinline int bpf_jit_insn(struct bpf_jit *jit, struct bpf_prog *fp,* B1: pointer to ctx * B2: pointer to bpf_array * B3: index in bpf_array -*/ - jit->seen |=SEEN_TAIL_CALL ; - -/* +* * if (index >= array->map.max_entries) * goto out; */
@@ -1793,7 +1805,7 @@ static int bpf_jit_prog(struct bpf_jit *jit, struct bpf_prog *fp, jit->prg =0 ; jit->excnt =0 ; -bpf_jit_prologue (jit, stack_depth); +bpf_jit_prologue (jit, fp, stack_depth);if (bpf_set_addr (jit,0 ) <0 )return -1 ;for (i =0 ; i < fp->len; i += insn_count) {
@@ -2461,3 +2473,8 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image,return ret; } + +bool bpf_jit_supports_subprog_tailcalls (void ) +{ +return true; +}
Signature · 1 line
2.39.1
Ilya Leoshkevich
maintainerPATCH v2 31/31 s390/bpf: Implement bpf_jit_supports_kfunc_call()
In reply to Ilya LeoshkevichImplement calling kernel functions from eBPF. In general, the eBPF ABI is fairly close to that of s390x, with one important difference: on s390x callers should sign-extend signed arguments. Handle that by using information returned by bpf_jit_find_kfunc_model().
Here is an example of how sign extensions works. Suppose we need to call the following function from BPF:
; long noinline bpf_kfunc_call_test4(signed char a, short b, int c,
long d)
0000000000936a78 <bpf_kfunc_call_test4>:
936a78: c0 04 00 00 00 00 jgnop bpf_kfunc_call_test4
; return (long)a + (long)b + (long)c + d;
936a7e: b9 08 00 45 agr %r4,%r5
936a82: b9 08 00 43 agr %r4,%r3
936a86: b9 08 00 24 agr %r2,%r4
936a8a: c0 f4 00 1e 3b 27 jg <__s390_indirect_jump_r14>As per the s390x ABI, bpf_kfunc_call_test4() has the right to assume that a, b and c are sign-extended by the caller, which results in using 64-bit additions (agr) without any additional conversions. Without sign extension we would have the following on the JITed code side:
; tmp = bpf_kfunc_call_test4(-3, -30, -200, -1000);
; 5: b4 10 00 00 ff ff ff fd w1 = -3
0x3ff7fdcdad4: llilf %r2,0xfffffffd
; 6: b4 20 00 00 ff ff ff e2 w2 = -30
0x3ff7fdcdada: llilf %r3,0xffffffe2
; 7: b4 30 00 00 ff ff ff 38 w3 = -200
0x3ff7fdcdae0: llilf %r4,0xffffff38
; 8: b7 40 00 00 ff ff fc 18 r4 = -1000
0x3ff7fdcdae6: lgfi %r5,-1000
0x3ff7fdcdaec: mvc 64(4,%r15),160(%r15)
0x3ff7fdcdaf2: lgrl %r1,bpf_kfunc_call_test4@GOT
0x3ff7fdcdaf8: brasl %r14,__s390_indirect_jump_r1This first 3 llilfs are 32-bit loads, that need to be sign-extended to 64 bits.
Note: at the moment bpf_jit_find_kfunc_model() does not seem to play nicely with XDP metadata functions: add_kfunc_call() adds an "abstract" bpf_*() version to kfunc_btf_tab, but then fixup_kfunc_call() puts the concrete version into insn->imm, which bpf_jit_find_kfunc_model() cannot find. But this seems to be a common code problem.
--- arch/s390/net/bpf_jit_comp.c | 25 +++++++++++++++++++++++-- 1 file changed, 23 insertions(+), 2 deletions(-)
arch/s390/net/bpf_jit_comp.c +23 -2
@@ -1401,9 +1401,10 @@ static noinline int bpf_jit_insn(struct bpf_jit *jit, struct bpf_prog *fp,*/ case BPF_JMP |BPF_CALL : { -u64 func; +const struct btf_func_model *m;bool func_addr_fixed; -int ret; +int j, ret; +u64 func; ret =bpf_jit_get_func_addr (fp, insn, extra_pass, &func, &func_addr_fixed);
@@ -1425,6 +1426,21 @@ static noinline int bpf_jit_insn(struct bpf_jit *jit, struct bpf_prog *fp,/* mvc STK_OFF_TCCNT(4,%r15),N(%r15) */ _EMIT6 (0xd203f000 |STK_OFF_TCCNT ,0xf000 | (STK_OFF_TCCNT +STK_OFF + stack_depth)); + +/* Sign-extend the kfunc arguments. */ +if (insn->src_reg ==BPF_PSEUDO_KFUNC_CALL ) { + m =bpf_jit_find_kfunc_model (fp, insn); +if (!m) +return -1 ; + +for (j =0 ; j < m->nr_args; j++) { +if (sign_extend (jit,BPF_REG_1 + j, + m->arg_size[j], + m->arg_flags[j])) +return -1 ; + } + } +/* lgrl %w1,func */ EMIT6_PCREL_RILB (0xc4080000 ,REG_W1 ,_EMIT_CONST_U64 (func));/* %r1() */
@@ -1980,6 +1996,11 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)return fp; } +bool bpf_jit_supports_kfunc_call (void ) +{ +return true; +} +int bpf_arch_text_poke (void *ip,enum bpf_text_poke_type t,void *old_addr,void *new_addr) {
Signature · 1 line
2.39.1
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Hello:
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Quoted 12 lines · On Sat, 28 Jan 2023 01:06:19 +0100 you wrote:
v1: https://lore.kernel.org/bpf/[email protected]/#t
v1 -> v2:
- Fixcore_read_macros,sk_assign,test_profiler,test_bpffs(24/31; I'm not quite happy with the fix, but don't have better ideas), andxdp_synproxy. (Andrii)
- Prettifyliburandom_readandverify_pkcs7_sigfixes. (Andrii)
- Fixbpf_usdt_argusingbarrier_var(); prettifybarrier_var(). (Andrii)
- ChangeBPF_MAX_TRAMP_LINKSto enum and query it using BTF. (Andrii)
- Improvebpf_jit_supports_kfunc_call()description. (Alexei)
- Always checksign_extend()return value.
- Cc: Alexander Gordeev.
Quoted 1 line
[...]
Here is the summary with links:
- [bpf-next,v2,01/31] bpf: Use ARG_CONST_SIZE_OR_ZERO for 3rd argument of bpf_tcp_raw_gen_syncookie_ipv{4,6}()
https://git.kernel.org/bpf/bpf-next/c/bf3849755ac6 - [bpf-next,v2,02/31] bpf: Change BPF_MAX_TRAMP_LINKS to enum https://git.kernel.org/bpf/bpf-next/c/390a07a921b3 - [bpf-next,v2,03/31] selftests/bpf: Query BPF_MAX_TRAMP_LINKS using BTF https://git.kernel.org/bpf/bpf-next/c/8fb9fb2f1728 - [bpf-next,v2,04/31] selftests/bpf: Fix liburandom_read.so linker error https://git.kernel.org/bpf/bpf-next/c/b14b01f281f7 - [bpf-next,v2,05/31] selftests/bpf: Fix symlink creation error https://git.kernel.org/bpf/bpf-next/c/6eab2370d142 - [bpf-next,v2,06/31] selftests/bpf: Fix kfree_skb on s390x https://git.kernel.org/bpf/bpf-next/c/31da9be64a11 - [bpf-next,v2,07/31] selftests/bpf: Set errno when urand_spawn() fails https://git.kernel.org/bpf/bpf-next/c/804acdd251e8 - [bpf-next,v2,08/31] selftests/bpf: Fix decap_sanity_ns cleanup https://git.kernel.org/bpf/bpf-next/c/98e13848cf43 - [bpf-next,v2,09/31] selftests/bpf: Fix verify_pkcs7_sig on s390x https://git.kernel.org/bpf/bpf-next/c/56e1a5048319 - [bpf-next,v2,10/31] selftests/bpf: Fix xdp_do_redirect on s390x https://git.kernel.org/bpf/bpf-next/c/06c1865b0b0c - [bpf-next,v2,11/31] selftests/bpf: Fix cgrp_local_storage on s390x https://git.kernel.org/bpf/bpf-next/c/06cea99e683c - [bpf-next,v2,12/31] selftests/bpf: Check stack_mprotect() return value https://git.kernel.org/bpf/bpf-next/c/2934565f04fd - [bpf-next,v2,13/31] selftests/bpf: Increase SIZEOF_BPF_LOCAL_STORAGE_ELEM on s390x https://git.kernel.org/bpf/bpf-next/c/80a611904eef - [bpf-next,v2,14/31] selftests/bpf: Add a sign-extension test for kfuncs https://git.kernel.org/bpf/bpf-next/c/be6b5c10ecc4 - [bpf-next,v2,15/31] selftests/bpf: Fix test_lsm on s390x https://git.kernel.org/bpf/bpf-next/c/207612eb12b9 - [bpf-next,v2,16/31] selftests/bpf: Fix test_xdp_adjust_tail_grow2 on s390x https://git.kernel.org/bpf/bpf-next/c/26e8a0149479 - [bpf-next,v2,17/31] selftests/bpf: Fix vmlinux test on s390x https://git.kernel.org/bpf/bpf-next/c/d504270a233d - [bpf-next,v2,18/31] selftests/bpf: Fix sk_assign on s390x (no matching commit) - [bpf-next,v2,19/31] selftests/bpf: Fix xdp_synproxy/tc on s390x https://git.kernel.org/bpf/bpf-next/c/438a2edf26b7 - [bpf-next,v2,20/31] selftests/bpf: Fix profiler on s390x https://git.kernel.org/bpf/bpf-next/c/1b5e38532581 - [bpf-next,v2,21/31] libbpf: Simplify barrier_var() https://git.kernel.org/bpf/bpf-next/c/e85465e420be - [bpf-next,v2,22/31] libbpf: Fix unbounded memory access in bpf_usdt_arg() https://git.kernel.org/bpf/bpf-next/c/25c76ed42821 - [bpf-next,v2,23/31] libbpf: Fix BPF_PROBE_READ{_STR}_INTO() on s390x https://git.kernel.org/bpf/bpf-next/c/42fae973c2b1 - [bpf-next,v2,24/31] bpf: iterators: Split iterators.lskel.h into little- and big- endian versions https://git.kernel.org/bpf/bpf-next/c/0f0e5f5bd506 - [bpf-next,v2,25/31] bpf: btf: Add BTF_FMODEL_SIGNED_ARG flag https://git.kernel.org/bpf/bpf-next/c/49f67f393ff2 - [bpf-next,v2,26/31] s390/bpf: Fix a typo in a comment https://git.kernel.org/bpf/bpf-next/c/07dcbd7325ce
- [bpf-next,v2,27/31] s390/bpf: Add expoline to tail calls
(no matching commit)
- [bpf-next,v2,28/31] s390/bpf: Implement bpf_arch_text_poke() (no matching commit)
- [bpf-next,v2,29/31] s390/bpf: Implement arch_prepare_bpf_trampoline() (no matching commit)
- [bpf-next,v2,30/31] s390/bpf: Implement bpf_jit_supports_subprog_tailcalls() (no matching commit)
- [bpf-next,v2,31/31] s390/bpf: Implement bpf_jit_supports_kfunc_call() (no matching commit)
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html