bpf: Fix an issue in bpf_prog_test_run_xdp when page size greater than 4K
Subsystems (4)
Yonghong Song
reviewerPATCH 1/2 bpf: Fix an issue in bpf_prog_test_run_xdp when page size greater than 4K
The bpf selftest xdp_adjust_tail/xdp_adjust_frags_tail_grow failed on arm64 with 64KB page:
xdp_adjust_tail/xdp_adjust_frags_tail_grow:FAIL
In bpf_prog_test_run_xdp(), the xdp->frame_sz is set to 4K, but later on when constructing frags, with 64K page size, the frag data_len could be more than 4K. This will cause problems in bpf_xdp_frags_increase_tail().
Limiting the data_len to be 4K for each frag fixed the above test failure.
--- net/bpf/test_run.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
net/bpf/test_run.c +3 -2
@@ -1214,6 +1214,7 @@ int bpf_prog_test_run_xdp(struct bpf_prog *prog, const union bpf_attr *kattr,u32 repeat = kattr->test.repeat;struct netdev_rx_queue *rxqueue;struct skb_shared_info *sinfo; +const u32 frame_sz =4096 ;struct xdp_buff xdp = {};int i, ret = -EINVAL ;struct xdp_md *ctx;
@@ -1255,7 +1256,7 @@ int bpf_prog_test_run_xdp(struct bpf_prog *prog, const union bpf_attr *kattr, headroom -= ctx->data; } - max_data_sz =4096 - headroom - tailroom; + max_data_sz = frame_sz - headroom - tailroom;if (size > max_data_sz) {/* disallow live data mode for jumbo frames */ if (do_live)
@@ -1301,7 +1302,7 @@ int bpf_prog_test_run_xdp(struct bpf_prog *prog, const union bpf_attr *kattr, frag = &sinfo->frags[sinfo->nr_frags++]; data_len =min_t (u32, kattr->test.data_size_in - size, -PAGE_SIZE ); + frame_sz);skb_frag_fill_page_desc (frag, page,0 , data_len); if (copy_from_user(page_address(page), data_in + size,
Signature · 1 line
2.47.1
Yonghong Song
reviewerPATCH 2/2 selftests/bpf: Fix two net related test failures with 64K page size
When running BPF selftests on arm64 with a 64K page size, I encountered the following two test failures:
sockmap_basic/sockmap skb_verdict change tail:FAIL tc_change_tail:FAIL
With further debugging, I identified the root cause in the following kernel code within __bpf_skb_change_tail():
u32 max_len = BPF_SKB_MAX_LEN;
u32 min_len = __bpf_skb_min_len(skb);
int ret;
if (unlikely(flags || new_len > max_len || new_len < min_len))
return -EINVAL;With a 4K page size, new_len = 65535 and max_len = 16064, the function returns -EINVAL. However, With a 64K page size, max_len increases to 261824, allowing execution to proceed further in the function. This is because BPF_SKB_MAX_LEN scales with the page size and larger page sizes result in higher max_len values.
Updating the new_len parameter in both tests from 65535 to 262143 (0x3ffff) resolves the failures.
--- tools/testing/selftests/bpf/progs/test_sockmap_change_tail.c | 2 +- tools/testing/selftests/bpf/progs/test_tc_change_tail.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
tools/testing/selftests/bpf/progs/test_sockmap_change_tail.c +1 -1
@@ -31,7 +31,7 @@ int prog_skb_verdict(struct __sk_buff *skb) change_tail_ret =bpf_skb_change_tail (skb, skb->len +1 ,0 );return SK_PASS ; }else if (data[0 ] =='E' ) {/* Error */ - change_tail_ret =bpf_skb_change_tail (skb,65535 ,0 ); + change_tail_ret =bpf_skb_change_tail (skb,262143 ,0 );return SK_PASS ; }return SK_PASS ;
tools/testing/selftests/bpf/progs/test_tc_change_tail.c +1 -1
@@ -94,7 +94,7 @@ int change_tail(struct __sk_buff *skb)bpf_skb_change_tail (skb, len,0 );return TCX_PASS ; }else if (payload[0 ] =='E' ) {/* Error */ - change_tail_ret =bpf_skb_change_tail (skb,65535 ,0 ); + change_tail_ret =bpf_skb_change_tail (skb,262143 ,0 );return TCX_PASS ; }else if (payload[0 ] =='Z' ) {/* Zero */ change_tail_ret =bpf_skb_change_tail (skb,0 ,0 );
Signature · 1 line
2.47.1
Re: [PATCH bpf-next 2/2] selftests/bpf: Fix two net related test failures with 64K page size
Quoted 48 lines · Yonghong Song
When running BPF selftests on arm64 with a 64K page size, I encountered the following two test failures:
sockmap_basic/sockmapskb_verdictchange tail:FAIL tc_change_tail:FAILWith further debugging, I identified the root cause in the following kernel code within
__bpf_skb_change_tail():u32 max_len = BPF_SKB_MAX_LEN; u32 min_len = __bpf_skb_min_len(skb); int ret; if (unlikely(flags || new_len > max_len || new_len < min_len)) return -EINVAL;With a 4K page size,
new_len= 65535 andmax_len= 16064, the function returns-EINVAL. However, With a 64K page size,max_lenincreases to 261824, allowing execution to proceed further in the function. This is becauseBPF_SKB_MAX_LENscales with the page size and larger page sizes result in highermax_lenvalues.Updating the
new_lenparameter in both tests from 65535 to 262143 (0x3ffff) resolves the failures.Signed-off-by Yonghong Song <[email protected]>--- tools/testing/selftests/bpf/progs/test_sockmap_change_tail.c | 2 +- tools/testing/selftests/bpf/progs/test_tc_change_tail.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)tools/testing/selftests/bpf/progs/test_sockmap_change_tail.c +1
-1@@ -31,7 +31,7 @@ int prog_skb_verdict(struct __sk_buff *skb) change_tail_ret =bpf_skb_change_tail (skb, skb->len +1 ,0 );return SK_PASS ; }else if (data[0 ] =='E' ) {/* Error */ - change_tail_ret =bpf_skb_change_tail (skb,65535 ,0 ); + change_tail_ret =bpf_skb_change_tail (skb,262143 ,0 );return SK_PASS ; }return SK_PASS ;tools/testing/selftests/bpf/progs/test_tc_change_tail.c +0
-0
tools/testing/selftests/bpf/progs/test_tc_change_tail.c int change_tail(struct __sk_buff *skb) · line 100
Quoted 8 lines
tools/testing/selftests/bpf/progs/test_tc_change_tail.c +1
-1@@ -94,7 +94,7 @@ int change_tail(struct __sk_buff *skb)bpf_skb_change_tail (skb, len,0 );return TCX_PASS ; }else if (payload[0 ] =='E' ) {/* Error */ - change_tail_ret =bpf_skb_change_tail (skb,65535 ,0 ); + change_tail_ret =bpf_skb_change_tail (skb,262143 ,0 );return TCX_PASS ; }else if (payload[0 ] =='Z' ) {/* Zero */ change_tail_ret =bpf_skb_change_tail (skb,0 ,0 );
nit: this seems to be exercising BPF_SKB_MAX_LEN case. To make it easier to spot in the future, can we do the following in both tests?
#define PAGE_SIZE 65536 /* make it work on 64K page arches */ #define BPF_SKB_MAX_LEN (PAGE_SIZE << 2)
... = bpf_skb_change_tail(skb, BPF_SKB_MAX_LEN, 0);
Yonghong Song
reviewerRe: [PATCH bpf-next 2/2] selftests/bpf: Fix two net related test failures with 64K page size
Quoted 60 lines · Stanislav Fomichev
Quoted 48 lines · Yonghong Song
When running BPF selftests on arm64 with a 64K page size, I encountered the following two test failures:
sockmap_basic/sockmapskb_verdictchange tail:FAIL tc_change_tail:FAILWith further debugging, I identified the root cause in the following kernel code within
__bpf_skb_change_tail():u32 max_len = BPF_SKB_MAX_LEN; u32 min_len = __bpf_skb_min_len(skb); int ret; if (unlikely(flags || new_len > max_len || new_len < min_len)) return -EINVAL;With a 4K page size,
new_len= 65535 andmax_len= 16064, the function returns-EINVAL. However, With a 64K page size,max_lenincreases to 261824, allowing execution to proceed further in the function. This is becauseBPF_SKB_MAX_LENscales with the page size and larger page sizes result in highermax_lenvalues.Updating the
new_lenparameter in both tests from 65535 to 262143 (0x3ffff) resolves the failures.Signed-off-by Yonghong Song <[email protected]>--- tools/testing/selftests/bpf/progs/test_sockmap_change_tail.c | 2 +- tools/testing/selftests/bpf/progs/test_tc_change_tail.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)tools/testing/selftests/bpf/progs/test_sockmap_change_tail.c +1
-1@@ -31,7 +31,7 @@ int prog_skb_verdict(struct __sk_buff *skb) change_tail_ret =bpf_skb_change_tail (skb, skb->len +1 ,0 );return SK_PASS ; }else if (data[0 ] =='E' ) {/* Error */ - change_tail_ret =bpf_skb_change_tail (skb,65535 ,0 ); + change_tail_ret =bpf_skb_change_tail (skb,262143 ,0 );return SK_PASS ; }return SK_PASS ;tools/testing/selftests/bpf/progs/test_tc_change_tail.c +0
-0tools/testing/selftests/bpf/progs/test_tc_change_tail.c int change_tail(struct __sk_buff *skb) · line 100
Quoted 8 lines
tools/testing/selftests/bpf/progs/test_tc_change_tail.c +1
-1@@ -94,7 +94,7 @@ int change_tail(struct __sk_buff *skb)bpf_skb_change_tail (skb, len,0 );return TCX_PASS ; }else if (payload[0 ] =='E' ) {/* Error */ - change_tail_ret =bpf_skb_change_tail (skb,65535 ,0 ); + change_tail_ret =bpf_skb_change_tail (skb,262143 ,0 );return TCX_PASS ; }else if (payload[0 ] =='Z' ) {/* Zero */ change_tail_ret =bpf_skb_change_tail (skb,0 ,0 );nit: this seems to be exercising
BPF_SKB_MAX_LENcase. To make it easier to spot in the future, can we do the following in both tests?
Quoted 4 lines
#define
PAGE_SIZE65536 /* make it work on 64K page arches */ #defineBPF_SKB_MAX_LEN(PAGE_SIZE<< 2)... =
bpf_skb_change_tail(skb, BPF_SKB_MAX_LEN, 0);
Thanks for suggestions, Will do.
Quoted 31 lines · Yonghong Song
The bpf selftest xdp_adjust_tail/xdp_adjust_frags_tail_grow failed on arm64 with 64KB page:
xdp_adjust_tail/xdp_adjust_frags_tail_grow:FAILIn
bpf_prog_test_run_xdp(), thexdp->frame_szis set to 4K, but later on when constructing frags, with 64K page size, the fragdata_lencould be more than 4K. This will cause problems inbpf_xdp_frags_increase_tail().Limiting the
data_lento be 4K for each frag fixed the above test failure.Signed-off-by Yonghong Song <[email protected]>--- net/bpf/test_run.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)net/bpf/test_run.c +1
-0@@ -1214,6 +1214,7 @@ int bpf_prog_test_run_xdp(struct bpf_prog *prog, const union bpf_attr *kattr,u32 repeat = kattr->test.repeat;struct netdev_rx_queue *rxqueue;struct skb_shared_info *sinfo; +const u32 frame_sz =4096 ;struct xdp_buff xdp = {};int i, ret = -EINVAL ;struct xdp_md *ctx;@@ -1255,7 +1256,7 @@ int bpf_prog_test_run_xdp(struct bpf_prog *prog, const union bpf_attr *kattr, headroom -= ctx->data; }
[..]
net/bpf/test_run.c int bpf_prog_test_run_xdp(struct bpf_prog *prog, const union bpf_attr *kattr, · line 1259
Quoted 2 lines
net/bpf/test_run.c +1
-1- max_data_sz =4096 - headroom - tailroom; + max_data_sz = frame_sz - headroom - tailroom;
I wonder whether we should do s/4096/PAGE_SIZE/ here instead. Have you tried that? If we are on a 64K page arch, we should not try to preserve 4K page limits.
Yonghong Song
reviewerQuoted 34 lines · Stanislav Fomichev
Quoted 31 lines · Yonghong Song
The bpf selftest xdp_adjust_tail/xdp_adjust_frags_tail_grow failed on arm64 with 64KB page:
xdp_adjust_tail/xdp_adjust_frags_tail_grow:FAILIn
bpf_prog_test_run_xdp(), thexdp->frame_szis set to 4K, but later on when constructing frags, with 64K page size, the fragdata_lencould be more than 4K. This will cause problems inbpf_xdp_frags_increase_tail().Limiting the
data_lento be 4K for each frag fixed the above test failure.Signed-off-by Yonghong Song <[email protected]>--- net/bpf/test_run.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)net/bpf/test_run.c +1
-0@@ -1214,6 +1214,7 @@ int bpf_prog_test_run_xdp(struct bpf_prog *prog, const union bpf_attr *kattr,u32 repeat = kattr->test.repeat;struct netdev_rx_queue *rxqueue;struct skb_shared_info *sinfo; +const u32 frame_sz =4096 ;struct xdp_buff xdp = {};int i, ret = -EINVAL ;struct xdp_md *ctx;@@ -1255,7 +1256,7 @@ int bpf_prog_test_run_xdp(struct bpf_prog *prog, const union bpf_attr *kattr, headroom -= ctx->data; }[..]
Quoted 5 lines
net/bpf/test_run.c int bpf_prog_test_run_xdp(struct bpf_prog *prog, const union bpf_attr *kattr, · line 1259
Quoted 2 lines
net/bpf/test_run.c +1
-1- max_data_sz =4096 - headroom - tailroom; + max_data_sz = frame_sz - headroom - tailroom;I wonder whether we should do s/4096/PAGE_SIZE/ here instead. Have you tried that? If we are on a 64K page arch, we should not try to preserve 4K page limits.
The user space test_run input looks like below (in prog_tests/xdp_adjust_tail.c):
buf = malloc(16384); ... topts.data_in = buf; topts.data_out = buf; topts.data_size_in = 9000; topts.data_size_out = 16384; err = bpf_prog_test_run_opts(prog_fd, &topts);
Allowing s/4096/PAGE_SIZE (64K) will have the test failure for the above user space input. I think I can increse buf size data_size_in/data_size_out properly so in the kernel we can do s/4096/PAGE_SIZE.