bpf: Fix a few test failures with 64K page size
Subsystems (5)
Yonghong Song
reviewerv2 cover bpf: Fix a few test failures with 64K page size
This small patch set tried to fix a few networking related test failure failures due to 64K page size. Please see each individual patch for details.
Changelog:
v1 -> v2:
- v1: https://lore.kernel.org/bpf/[email protected]/
- For xdp_adjust_tail, let kernel test_run can handle various page sizes for xdp progs.
- For two change_tail tests, make code easier to understand.
- Resolved a new test failure (xdp_do_redirect).
Yonghong Song (3):
bpf: Fix an issue in bpf_prog_test_run_xdp when page size greater than
4K
selftests/bpf: Fix two net related test failures with 64K page size selftests/bpf: Fix xdp_do_redirect failure with 64KB page size
net/bpf/test_run.c | 2 +- .../bpf/prog_tests/xdp_adjust_tail.c | 95 +++++++++++++++++-- .../bpf/prog_tests/xdp_do_redirect.c | 13 ++- .../bpf/progs/test_sockmap_change_tail.c | 5 +- .../selftests/bpf/progs/test_tc_change_tail.c | 5 +- .../bpf/progs/test_xdp_adjust_tail_grow.c | 8 +- 6 files changed, 115 insertions(+), 13 deletions(-)
Signature · 1 line
2.47.1
Yonghong Song
reviewerPATCH v2 1/3 bpf: Fix an issue in bpf_prog_test_run_xdp when page size greater than 4K
The bpf selftest xdp_adjust_tail/xdp_adjust_frags_tail_grow failed on arm64 with 64KB page:
xdp_adjust_tail/xdp_adjust_frags_tail_grow:FAIL
In bpf_prog_test_run_xdp(), the xdp->frame_sz is set to 4K, but later on when constructing frags, with 64K page size, the frag data_len could be more than 4K. This will cause problems in bpf_xdp_frags_increase_tail().
To fix the failure, the xdp->frame_sz is set to be PAGE_SIZE so kernel can test different page size properly. With the kernel change, the user space and bpf prog needs adjustment. Currently, the MAX_SKB_FRAGS default value is 17, so for 4K page, the maximum packet size will be less than 68K. To test 64K page, a bigger maximum packet size than 68K is desired. So two different functions are implemented for subtest xdp_adjust_frags_tail_grow. Depending on different page size, different data input/output sizes are used to adapt with different page size.
--- net/bpf/test_run.c | 2 +- .../bpf/prog_tests/xdp_adjust_tail.c | 95 +++++++++++++++++-- .../bpf/progs/test_xdp_adjust_tail_grow.c | 8 +- 3 files changed, 96 insertions(+), 9 deletions(-)
net/bpf/test_run.c +1 -1
@@ -1255,7 +1255,7 @@ int bpf_prog_test_run_xdp(struct bpf_prog *prog, const union bpf_attr *kattr, headroom -= ctx->data; } - max_data_sz =4096 - headroom - tailroom; + max_data_sz =PAGE_SIZE - headroom - tailroom;if (size > max_data_sz) {/* disallow live data mode for jumbo frames */ if (do_live)
tools/testing/selftests/bpf/prog_tests/xdp_adjust_tail.c +88 -7
132 lines not on this page · raw message
tools/testing/selftests/bpf/progs/test_xdp_adjust_tail_grow.c +7 -1
@@ -19,7 +19,9 @@ int _xdp_adjust_tail_grow(struct xdp_md *xdp)/* Data length determine test case */ if (data_len ==54 ) {/* sizeof(pkt_v4) */ - offset =4096 ;/* test too large offset */ + offset =4096 ;/* test too large offset, 4k page size */ + }else if (data_len ==53 ) {/* sizeof(pkt_v4) - 1 */ + offset =65536 ;/* test too large offset, 64k page size */ }else if (data_len ==74 ) {/* sizeof(pkt_v6) */ offset =40 ; }else if (data_len ==64 ) {
@@ -31,6 +33,10 @@ int _xdp_adjust_tail_grow(struct xdp_md *xdp) offset =10 ; }else if (data_len ==9001 ) { offset =4096 ; + }else if (data_len ==90000 ) { + offset =10 ; + }else if (data_len ==90001 ) { + offset =65536 ; }else {return XDP_ABORTED ;/* No matching test */ }
Signature · 1 line
2.47.1
Re: [PATCH bpf-next v2 1/3] bpf: Fix an issue in bpf_prog_test_run_xdp when page size greater than 4K
Quoted 57 lines · Yonghong Song
The bpf selftest xdp_adjust_tail/xdp_adjust_frags_tail_grow failed on arm64 with 64KB page:
xdp_adjust_tail/xdp_adjust_frags_tail_grow:FAILIn
bpf_prog_test_run_xdp(), thexdp->frame_szis set to 4K, but later on when constructing frags, with 64K page size, the fragdata_lencould be more than 4K. This will cause problems inbpf_xdp_frags_increase_tail().To fix the failure, the
xdp->frame_szis set to bePAGE_SIZEso kernel can test different page size properly. With the kernel change, the user space and bpf prog needs adjustment. Currently, theMAX_SKB_FRAGSdefault value is 17, so for 4K page, the maximum packet size will be less than 68K. To test 64K page, a bigger maximum packet size than 68K is desired. So two different functions are implemented for subtestxdp_adjust_frags_tail_grow. Depending on different page size, different data input/output sizes are used to adapt with different page size.Signed-off-by Yonghong Song <[email protected]>--- net/bpf/test_run.c | 2 +- .../bpf/prog_tests/xdp_adjust_tail.c | 95 +++++++++++++++++-- .../bpf/progs/test_xdp_adjust_tail_grow.c | 8 +- 3 files changed, 96 insertions(+), 9 deletions(-)net/bpf/test_run.c +1
-1@@ -1255,7 +1255,7 @@ int bpf_prog_test_run_xdp(struct bpf_prog *prog, const union bpf_attr *kattr, headroom -= ctx->data; } - max_data_sz =4096 - headroom - tailroom; + max_data_sz =PAGE_SIZE - headroom - tailroom;if (size > max_data_sz) {/* disallow live data mode for jumbo frames */ if (do_live)tools/testing/selftests/bpf/prog_tests/xdp_adjust_tail.c +2
-2@@ -37,21 +37,25 @@ static void test_xdp_adjust_tail_shrink(void)bpf_object__close (obj); } -static void test_xdp_adjust_tail_grow (void ) +static void test_xdp_adjust_tail_grow (bool is_64k_pagesize) {const char *file ="./test_xdp_adjust_tail_grow.bpf.o" ;struct bpf_object *obj; -char buf[4096 ];/* avoid segfault: large buf to hold grow results */ +char buf[8192 ];/* avoid segfault: large buf to hold grow results */ __u32 expect_sz;int err, prog_fd; LIBBPF_OPTS(bpf_test_run_opts, topts, .data_in = &pkt_v4,
tools/testing/selftests/bpf/prog_tests/xdp_adjust_tail.c static void test_xdp_adjust_tail_shrink(void) · line 57
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/xdp_adjust_tail.c +4
-1- .data_size_in = sizeof(pkt_v4), .data_out = buf, .data_size_out = sizeof(buf), .repeat = 1, ); +if (is_64k_pagesize) + topts.data_size_in =sizeof (pkt_v4) -1 ; +else + topts.data_size_in =sizeof (pkt_v4);
Please add a comment that magic data size is a special signal to bpf prog:
tools/testing/selftests/bpf/prog_tests/xdp_adjust_tail.c static void test_xdp_adjust_tail_shrink(void)
Quoted 7 lines
tools/testing/selftests/bpf/prog_tests/xdp_adjust_tail.c +3
-1if (data_len ==54 ) {/* sizeof(pkt_v4) */ - offset =4096 ;/* test too large offset */ + offset =4096 ;/* test too large offset, 4k page size */ + }else if (data_len ==53 ) {/* sizeof(pkt_v4) - 1 */ + offset =65536 ;/* test too large offset, 64k page size */ }else if (data_len ==74 ) {/* sizeof(pkt_v6) */ offset =40 ;
and comment about 90000, 90001 sizes.
Yonghong Song
reviewerRe: [PATCH bpf-next v2 1/3] bpf: Fix an issue in bpf_prog_test_run_xdp when page size greater than 4K
Quoted 63 lines · Alexei Starovoitov
Quoted 62 lines · Yonghong Song
The bpf selftest xdp_adjust_tail/xdp_adjust_frags_tail_grow failed on arm64 with 64KB page:
xdp_adjust_tail/xdp_adjust_frags_tail_grow:FAILIn
bpf_prog_test_run_xdp(), thexdp->frame_szis set to 4K, but later on when constructing frags, with 64K page size, the fragdata_lencould be more than 4K. This will cause problems inbpf_xdp_frags_increase_tail().To fix the failure, the
xdp->frame_szis set to bePAGE_SIZEso kernel can test different page size properly. With the kernel change, the user space and bpf prog needs adjustment. Currently, theMAX_SKB_FRAGSdefault value is 17, so for 4K page, the maximum packet size will be less than 68K. To test 64K page, a bigger maximum packet size than 68K is desired. So two different functions are implemented for subtestxdp_adjust_frags_tail_grow. Depending on different page size, different data input/output sizes are used to adapt with different page size.Signed-off-by Yonghong Song <[email protected]>--- net/bpf/test_run.c | 2 +- .../bpf/prog_tests/xdp_adjust_tail.c | 95 +++++++++++++++++-- .../bpf/progs/test_xdp_adjust_tail_grow.c | 8 +- 3 files changed, 96 insertions(+), 9 deletions(-)net/bpf/test_run.c +1
-1@@ -1255,7 +1255,7 @@ int bpf_prog_test_run_xdp(struct bpf_prog *prog, const union bpf_attr *kattr, headroom -= ctx->data; } - max_data_sz =4096 - headroom - tailroom; + max_data_sz =PAGE_SIZE - headroom - tailroom;if (size > max_data_sz) {/* disallow live data mode for jumbo frames */ if (do_live)tools/testing/selftests/bpf/prog_tests/xdp_adjust_tail.c +2
-3@@ -37,21 +37,25 @@ static void test_xdp_adjust_tail_shrink(void)bpf_object__close (obj); } -static void test_xdp_adjust_tail_grow (void ) +static void test_xdp_adjust_tail_grow (bool is_64k_pagesize) {const char *file ="./test_xdp_adjust_tail_grow.bpf.o" ;struct bpf_object *obj; -char buf[4096 ];/* avoid segfault: large buf to hold grow results */ +char buf[8192 ];/* avoid segfault: large buf to hold grow results */ __u32 expect_sz;int err, prog_fd;LIBBPF_OPTS (bpf_test_run_opts, topts, .data_in = &pkt_v4, - .data_size_in = sizeof(pkt_v4), .data_out = buf, .data_size_out = sizeof(buf), .repeat = 1, );
Quoted 6 lines
tools/testing/selftests/bpf/prog_tests/xdp_adjust_tail.c static void test_xdp_adjust_tail_shrink(void) · line 57
Quoted 4 lines
tools/testing/selftests/bpf/prog_tests/xdp_adjust_tail.c +4
-0+if (is_64k_pagesize) + topts.data_size_in =sizeof (pkt_v4) -1 ; +else + topts.data_size_in =sizeof (pkt_v4);Please add a comment that magic data size is a special signal to bpf prog:
Ok, will add a comment to explain this.
Quoted 9 lines
tools/testing/selftests/bpf/prog_tests/xdp_adjust_tail.c static void test_xdp_adjust_tail_shrink(void)
Quoted 8 lines
tools/testing/selftests/bpf/prog_tests/xdp_adjust_tail.c +3
-1if (data_len ==54 ) {/* sizeof(pkt_v4) */ - offset =4096 ;/* test too large offset */ + offset =4096 ;/* test too large offset, 4k page size */ + }else if (data_len ==53 ) {/* sizeof(pkt_v4) - 1 */ + offset =65536 ;/* test too large offset, 64k page size */ }else if (data_len ==74 ) {/* sizeof(pkt_v6) */ offset =40 ;and comment about 90000, 90001 sizes.
Ack as well.
Yonghong Song
reviewerPATCH v2 2/3 selftests/bpf: Fix two net related test failures with 64K page size
In reply to Yonghong SongWhen running BPF selftests on arm64 with a 64K page size, I encountered the following two test failures:
sockmap_basic/sockmap skb_verdict change tail:FAIL tc_change_tail:FAIL
With further debugging, I identified the root cause in the following kernel code within __bpf_skb_change_tail():
u32 max_len = BPF_SKB_MAX_LEN;
u32 min_len = __bpf_skb_min_len(skb);
int ret;
if (unlikely(flags || new_len > max_len || new_len < min_len))
return -EINVAL;With a 4K page size, new_len = 65535 and max_len = 16064, the function returns -EINVAL. However, With a 64K page size, max_len increases to 261824, allowing execution to proceed further in the function. This is because BPF_SKB_MAX_LEN scales with the page size and larger page sizes result in higher max_len values.
Updating the new_len parameter in both tests from 65535 to 256K (0x40000) resolves the failures.
--- tools/testing/selftests/bpf/progs/test_sockmap_change_tail.c | 5 ++++- tools/testing/selftests/bpf/progs/test_tc_change_tail.c | 5 ++++- 2 files changed, 8 insertions(+), 2 deletions(-)
tools/testing/selftests/bpf/progs/test_sockmap_change_tail.c +4 -1
@@ -3,6 +3,9 @@#include <linux/bpf.h> #include <bpf/bpf_helpers.h> +#define PAGE_SIZE 65536/* make it work on 64K page arches */ +#define BPF_SKB_MAX_LEN (PAGE_SIZE << 2) +struct { __uint(type, BPF_MAP_TYPE_SOCKMAP); __uint(max_entries, 1);
@@ -31,7 +34,7 @@ int prog_skb_verdict(struct __sk_buff *skb) change_tail_ret =bpf_skb_change_tail (skb, skb->len +1 ,0 );return SK_PASS ; }else if (data[0 ] =='E' ) {/* Error */ - change_tail_ret =bpf_skb_change_tail (skb,65535 ,0 ); + change_tail_ret =bpf_skb_change_tail (skb,BPF_SKB_MAX_LEN ,0 );return SK_PASS ; }return SK_PASS ;
tools/testing/selftests/bpf/progs/test_tc_change_tail.c +4 -1
@@ -7,6 +7,9 @@#include <linux/udp.h> #include <linux/pkt_cls.h> +#define PAGE_SIZE 65536/* make it work on 64K page arches */ +#define BPF_SKB_MAX_LEN (PAGE_SIZE << 2) +long change_tail_ret =1 ; static __always_inline struct iphdr *parse_ip_header(struct __sk_buff *skb, int *ip_proto)
@@ -94,7 +97,7 @@ int change_tail(struct __sk_buff *skb)bpf_skb_change_tail (skb, len,0 );return TCX_PASS ; }else if (payload[0 ] =='E' ) {/* Error */ - change_tail_ret =bpf_skb_change_tail (skb,65535 ,0 ); + change_tail_ret =bpf_skb_change_tail (skb,BPF_SKB_MAX_LEN ,0 );return TCX_PASS ; }else if (payload[0 ] =='Z' ) {/* Zero */ change_tail_ret =bpf_skb_change_tail (skb,0 ,0 );
Signature · 1 line
2.47.1
Re: [PATCH bpf-next v2 2/3] selftests/bpf: Fix two net related test failures with 64K page size
Quoted 59 lines · Yonghong Song
When running BPF selftests on arm64 with a 64K page size, I encountered the following two test failures:
sockmap_basic/sockmapskb_verdictchange tail:FAIL tc_change_tail:FAILWith further debugging, I identified the root cause in the following kernel code within
__bpf_skb_change_tail():u32 max_len = BPF_SKB_MAX_LEN; u32 min_len = __bpf_skb_min_len(skb); int ret; if (unlikely(flags || new_len > max_len || new_len < min_len)) return -EINVAL;With a 4K page size,
new_len= 65535 andmax_len= 16064, the function returns-EINVAL. However, With a 64K page size,max_lenincreases to 261824, allowing execution to proceed further in the function. This is becauseBPF_SKB_MAX_LENscales with the page size and larger page sizes result in highermax_lenvalues.Updating the
new_lenparameter in both tests from 65535 to 256K (0x40000) resolves the failures.Signed-off-by Yonghong Song <[email protected]>--- tools/testing/selftests/bpf/progs/test_sockmap_change_tail.c | 5 ++++- tools/testing/selftests/bpf/progs/test_tc_change_tail.c | 5 ++++- 2 files changed, 8 insertions(+), 2 deletions(-)tools/testing/selftests/bpf/progs/test_sockmap_change_tail.c +4
-1@@ -3,6 +3,9 @@#include <linux/bpf.h> #include <bpf/bpf_helpers.h> +#define PAGE_SIZE 65536/* make it work on 64K page arches */ +#define BPF_SKB_MAX_LEN (PAGE_SIZE << 2) +struct { __uint(type, BPF_MAP_TYPE_SOCKMAP); __uint(max_entries, 1);@@ -31,7 +34,7 @@ int prog_skb_verdict(struct __sk_buff *skb) change_tail_ret =bpf_skb_change_tail (skb, skb->len +1 ,0 );return SK_PASS ; }else if (data[0 ] =='E' ) {/* Error */ - change_tail_ret =bpf_skb_change_tail (skb,65535 ,0 ); + change_tail_ret =bpf_skb_change_tail (skb,BPF_SKB_MAX_LEN ,0 );return SK_PASS ; }return SK_PASS ;tools/testing/selftests/bpf/progs/test_tc_change_tail.c +0
-0
tools/testing/selftests/bpf/progs/test_tc_change_tail.c line 11
Quoted 5 lines
tools/testing/selftests/bpf/progs/test_tc_change_tail.c +2
-0@@ -7,6 +7,9 @@#include <linux/udp.h> #include <linux/pkt_cls.h> +#define PAGE_SIZE 65536/* make it work on 64K page arches */ +#define BPF_SKB_MAX_LEN (PAGE_SIZE << 2)
If you want it to match the kernel then let's use actual page size? See bpf_arena_common.h and
#ifndef PAGE_SIZE
#define PAGE_SIZE __PAGE_SIZE
Yonghong Song
reviewerRe: [PATCH bpf-next v2 2/3] selftests/bpf: Fix two net related test failures with 64K page size
Quoted 62 lines · Alexei Starovoitov
Quoted 61 lines · Yonghong Song
When running BPF selftests on arm64 with a 64K page size, I encountered the following two test failures:
sockmap_basic/sockmapskb_verdictchange tail:FAIL tc_change_tail:FAILWith further debugging, I identified the root cause in the following kernel code within
__bpf_skb_change_tail():u32 max_len = BPF_SKB_MAX_LEN; u32 min_len = __bpf_skb_min_len(skb); int ret; if (unlikely(flags || new_len > max_len || new_len < min_len)) return -EINVAL;With a 4K page size,
new_len= 65535 andmax_len= 16064, the function returns-EINVAL. However, With a 64K page size,max_lenincreases to 261824, allowing execution to proceed further in the function. This is becauseBPF_SKB_MAX_LENscales with the page size and larger page sizes result in highermax_lenvalues.Updating the
new_lenparameter in both tests from 65535 to 256K (0x40000) resolves the failures.Signed-off-by Yonghong Song <[email protected]>--- tools/testing/selftests/bpf/progs/test_sockmap_change_tail.c | 5 ++++- tools/testing/selftests/bpf/progs/test_tc_change_tail.c | 5 ++++- 2 files changed, 8 insertions(+), 2 deletions(-)tools/testing/selftests/bpf/progs/test_sockmap_change_tail.c +4
-1@@ -3,6 +3,9 @@#include <linux/bpf.h> #include <bpf/bpf_helpers.h> +#define PAGE_SIZE 65536/* make it work on 64K page arches */ +#define BPF_SKB_MAX_LEN (PAGE_SIZE << 2) +struct { __uint(type, BPF_MAP_TYPE_SOCKMAP); __uint(max_entries, 1);@@ -31,7 +34,7 @@ int prog_skb_verdict(struct __sk_buff *skb) change_tail_ret =bpf_skb_change_tail (skb, skb->len +1 ,0 );return SK_PASS ; }else if (data[0 ] =='E' ) {/* Error */ - change_tail_ret =bpf_skb_change_tail (skb,65535 ,0 ); + change_tail_ret =bpf_skb_change_tail (skb,BPF_SKB_MAX_LEN ,0 );return SK_PASS ; }return SK_PASS ;tools/testing/selftests/bpf/progs/test_tc_change_tail.c +0
-0@@ -7,6 +7,9 @@#include <linux/udp.h> #include <linux/pkt_cls.h>
Quoted 6 lines
tools/testing/selftests/bpf/progs/test_tc_change_tail.c line 11
Quoted 2 lines
tools/testing/selftests/bpf/progs/test_tc_change_tail.c +2
-0+#define PAGE_SIZE 65536/* make it work on 64K page arches */ +#define BPF_SKB_MAX_LEN (PAGE_SIZE << 2)If you want it to match the kernel then let's use actual page size? See
bpf_arena_common.hand
#ifndefPAGE_SIZE
#definePAGE_SIZE__PAGE_SIZE
LGTM. This will allow to use actual page size...
Yonghong Song
reviewerPATCH v2 3/3 selftests/bpf: Fix xdp_do_redirect failure with 64KB page size
In reply to Yonghong SongOn arm64 with 64KB page size, the selftest xdp_do_redirect failed like
below:
...
test_xdp_do_redirect:PASS:pkt_count_tc 0 nsec test_max_pkt_size:PASS:prog_run_max_size 0 nsec test_max_pkt_size:FAIL:prog_run_too_big unexpected prog_run_too_big: actual -28 != expected -22
With 64KB page size, the xdp frame size will be much bigger so the existing test will fail.
Adjust various parameters so the test can also work on 64K page size.
--- .../selftests/bpf/prog_tests/xdp_do_redirect.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-)
tools/testing/selftests/bpf/prog_tests/xdp_do_redirect.c +11 -2
@@ -66,16 +66,25 @@ static int attach_tc_prog(struct bpf_tc_hook *hook, int fd)#else #define MAX_PKT_SIZE 3408#endif + +#define PAGE_SIZE_4K 4096 +#define PAGE_SIZE_64K 65536 +static void test_max_pkt_size (int fd) { -char data[MAX_PKT_SIZE +1 ] = {}; +char data[PAGE_SIZE_64K +1 ] = {};int err;DECLARE_LIBBPF_OPTS (bpf_test_run_opts, opts, .data_in = &data, - .data_size_in = MAX_PKT_SIZE, .flags = BPF_F_TEST_XDP_LIVE_FRAMES, .repeat = 1, ); + +if (getpagesize () ==PAGE_SIZE_64K ) + opts.data_size_in =MAX_PKT_SIZE +PAGE_SIZE_64K -PAGE_SIZE_4K ; +else + opts.data_size_in =MAX_PKT_SIZE ; + err =bpf_prog_test_run_opts (fd, &opts);ASSERT_OK (err,"prog_run_max_size" );
Signature · 1 line
2.47.1