Pass correct tun device's RX queue number to XDP program
Get series with b4
b4 am [email protected]Versions newest first
- v1 Latest1 patch 1 replyIn review
Patches in v1 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/1 | net: tun: record RX queue in skb before do_xdp_generic()drivers/ | +2 | 1 reply |
Reviews trailers per patch
No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.
Cover letter [PATCH 0/1] Pass correct tun device's RX queue number to XDP program
Hi,
I'm trying to run XDP on top of a tun device in multi queue mode as that allows me to easily play on my laptop with an interface that supports multiple queues.
While testing it I found an issue: regardless of the the tun fd (i.e. RX queue) I'm writing packets to, the XDP program thinks all packets are coming from RX queue 0.
I'd like to propose a fix for that (see next mail) but let me first share some context.
Here's what I'm using to setup a tun device with multiple RX/TX queues and write some test packets into all of its RX queues:
struct ifreq ifr = {
.ifr_flags = IFF_TAP | IFF_NO_PI | IFF_MULTI_QUEUE, .ifr_name = "mytun0"
};
int fds[4];
for (int i = 0; i < 4; i++) {
if ((fds[i] = open("/dev/net/tun", O_RDWR)) < 0)
return -1;
if (ioctl(fds[i], TUNSETIFF, (void *)&ifr))
return -1;
}
while (1) {
for (int i = 0 ; i < 4; i++)
write(fds[i], "\x12\x34\x56\x78\x90\xab\x12\x34\x56\x67\x90\xab\x08\x00", 14);
sleep(2);
}Next I reproduced the problem with this XDP program attached to the tun device created with the previous code snippet:
$ cat xdp_kern.c
#include <linux/bpf.h>
#include <bpf_helpers.h> SEC("prog")
unsigned int xdp_prog(struct xdp_md *xdp) {
bpf_printk("rx queue: %d\n", xdp->rx_queue_index);
return XDP_PASS;
}
char _license[] SEC("license") = "GPL";
$ sudo ip link set dev mytun0 xdpdrv obj ./xdp_kern.o
$ sudo bpftool prog tracelog
a.out-22189 [003] .... 14444.594609: 0: rx queue: 0
a.out-22189 [003] .... 14444.594645: 0: rx queue: 0
a.out-22189 [003] .... 14444.594653: 0: rx queue: 0
a.out-22189 [003] .... 14444.594661: 0: rx queue: 0Here the XDP program is reporting all packets coming from RX queue 0 when they should have been coming from RX queues 0..3.
After looking into the tun driver I ended up writing this stap script which reveals the issue:
$ cat rx_queue.stap
probe kernel.function("tun_get_user") {
printf("tun_get_user(): tfile->xdp_rxq.queue_index: %d\n", $tfile->xdp_rxq->queue_index);
} probe kernel.function("netif_receive_generic_xdp") {
printf("netif_receive_generic_xdp(): skb->queue_mapping: %d\n", $skb->queue_mapping);
printf("\n");
}
$ sudo stap ./rx_queue.stap
tun_get_user(): tfile->xdp_rxq.queue_index: 0
netif_receive_generic_xdp(): skb->queue_mapping: 0
tun_get_user(): tfile->xdp_rxq.queue_index: 1
netif_receive_generic_xdp(): skb->queue_mapping: 0
tun_get_user(): tfile->xdp_rxq.queue_index: 2
netif_receive_generic_xdp(): skb->queue_mapping: 0
tun_get_user(): tfile->xdp_rxq.queue_index: 3
netif_receive_generic_xdp(): skb->queue_mapping: 0It looks like the tun driver (specifically tun_get_user()) is not passing correctly the information about the RX queue to do_xdp_generic().
Setting the queue_mapping field in the skb with skb_record_rx_queue() seems to fix the issue.
Same test with the patched kernel:
sudo bpftool prog tracelog
a.out-791 [004] .... 4826.499356: 0: rx queue: 0
a.out-791 [004] .... 4826.499532: 0: rx queue: 1
a.out-791 [004] .... 4826.499541: 0: rx queue: 2
a.out-791 [004] .... 4826.499549: 0: rx queue: 3While at it, I also fixed tun_xdp_one(), which was calling skb_record_rx_queue() after invoking do_xdp_generic().
Gilberto Bertin (1):
net: tun: record RX queue in skb before do_xdp_generic()
drivers/net/tun.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
Signature · 1 line
2.20.1