xsk: fix l2fwd for copy mode + busy poll combo
Subsystems (4)
Maciej Fijalkowski
maintainerPATCH 1/1 xsk: fix l2fwd for copy mode + busy poll combo
While checking AF_XDP copy mode combined with busy poll, strange results were observed. rxdrop and txonly scenarios worked fine, but l2fwd broke immediately.
After a deeper look, it turned out that for l2fwd, Tx side was exiting early due to xsk_no_wakeup() returning true and in the end xsk_generic_xmit() was never called. Note that AF_XDP Tx in copy mode is syscall steered, so the current behavior is broken.
Txonly scenario only worked due to the fact that sk_mark_napi_id_once_xdp() was never called - since Rx side is not in the picture for this case and mentioned function is called in xsk_rcv_check(), sk::sk_napi_id was never set, which in turn meant that xsk_no_wakeup() was returning false (see the sk->sk_napi_id >= MIN_NAPI_ID check in there).
To fix this, prefer busy poll in xsk_sendmsg() only when zero copy is enabled on a given AF_XDP socket. By doing so, busy poll in copy mode would not exit early on Tx side and eventually xsk_generic_xmit() will be called.
--- net/xdp/xsk.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
net/xdp/xsk.c +1 -1
@@ -639,7 +639,7 @@ static int __xsk_sendmsg(struct socket *sock, struct msghdr *m, size_t total_lenif (sk_can_busy_loop (sk))sk_busy_loop (sk,1 );/* only support non-blocking sockets */ -if (xsk_no_wakeup (sk)) +if (xs->zc &&xsk_no_wakeup (sk))return 0 ; pool = xs->pool;
Signature · 1 line
2.27.0
This patch was applied to bpf/bpf.git (master) by Daniel Borkmann <[email protected]>:
Hello:
This patch was applied to bpf/bpf.git (master)
by Daniel Borkmann <[email protected]>:
Quoted 10 lines · On Wed, 6 Apr 2022 17:58:04 +0200 you wrote:
While checking
AF_XDPcopy mode combined with busy poll, strange results were observed. rxdrop and txonly scenarios worked fine, but l2fwd broke immediately.After a deeper look, it turned out that for l2fwd, Tx side was exiting early due to
xsk_no_wakeup()returning true and in the endxsk_generic_xmit()was never called. Note thatAF_XDPTx in copy mode is syscall steered, so the current behavior is broken.[...]
Here is the summary with links:
- [bpf] xsk: fix l2fwd for copy mode + busy poll combo
https://git.kernel.org/bpf/bpf/c/8de8b71b787f
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html