Skip to content

Pass correct tun device's RX queue number to XDP program

Gilberto Bertin bpf 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest1 patch 1 replyIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/1net: tun: record RX queue in skb before do_xdp_generic()drivers/net/tun.c+2 -11 reply

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [PATCH 0/1] Pass correct tun device's RX queue number to XDP program

Hi,

I'm trying to run XDP on top of a tun device in multi queue mode as that allows me to easily play on my laptop with an interface that supports multiple queues.

While testing it I found an issue: regardless of the the tun fd (i.e. RX queue) I'm writing packets to, the XDP program thinks all packets are coming from RX queue 0.

I'd like to propose a fix for that (see next mail) but let me first share some context.

Here's what I'm using to setup a tun device with multiple RX/TX queues and write some test packets into all of its RX queues:

        struct ifreq ifr = {
                .ifr_flags = IFF_TAP | IFF_NO_PI | IFF_MULTI_QUEUE, .ifr_name = "mytun0"
        };

        int fds[4];
        for (int i = 0; i < 4; i++) {
                if ((fds[i] = open("/dev/net/tun", O_RDWR)) < 0)
                        return -1;
                if (ioctl(fds[i], TUNSETIFF, (void *)&ifr))
                        return -1;
        }

	while (1) {
		for (int i = 0 ; i < 4; i++)
			write(fds[i], "\x12\x34\x56\x78\x90\xab\x12\x34\x56\x67\x90\xab\x08\x00", 14);
		sleep(2);
	}

Next I reproduced the problem with this XDP program attached to the tun device created with the previous code snippet:

        $ cat xdp_kern.c
        #include <linux/bpf.h>
        #include <bpf_helpers.h>
        SEC("prog")
        unsigned int xdp_prog(struct xdp_md *xdp) {
                bpf_printk("rx queue: %d\n", xdp->rx_queue_index);
                return XDP_PASS;
        }

        char _license[] SEC("license") = "GPL";

        $ sudo ip link set dev mytun0 xdpdrv obj ./xdp_kern.o

        $ sudo bpftool prog tracelog
                   a.out-22189 [003] .... 14444.594609: 0: rx queue: 0
                   a.out-22189 [003] .... 14444.594645: 0: rx queue: 0
                   a.out-22189 [003] .... 14444.594653: 0: rx queue: 0
                   a.out-22189 [003] .... 14444.594661: 0: rx queue: 0

Here the XDP program is reporting all packets coming from RX queue 0 when they should have been coming from RX queues 0..3.

After looking into the tun driver I ended up writing this stap script which reveals the issue:

        $ cat rx_queue.stap
        probe kernel.function("tun_get_user") {
                printf("tun_get_user(): tfile->xdp_rxq.queue_index: %d\n", $tfile->xdp_rxq->queue_index);
        }
        probe kernel.function("netif_receive_generic_xdp") {
                printf("netif_receive_generic_xdp(): skb->queue_mapping: %d\n", $skb->queue_mapping);
                printf("\n");
        }

        $ sudo stap ./rx_queue.stap
        tun_get_user(): tfile->xdp_rxq.queue_index: 0
        netif_receive_generic_xdp(): skb->queue_mapping: 0

        tun_get_user(): tfile->xdp_rxq.queue_index: 1
        netif_receive_generic_xdp(): skb->queue_mapping: 0

        tun_get_user(): tfile->xdp_rxq.queue_index: 2
        netif_receive_generic_xdp(): skb->queue_mapping: 0

        tun_get_user(): tfile->xdp_rxq.queue_index: 3
        netif_receive_generic_xdp(): skb->queue_mapping: 0

It looks like the tun driver (specifically tun_get_user()) is not passing correctly the information about the RX queue to do_xdp_generic().

Setting the queue_mapping field in the skb with skb_record_rx_queue() seems to fix the issue.

Same test with the patched kernel:

        sudo bpftool prog tracelog
                   a.out-791   [004] ....  4826.499356: 0: rx queue: 0
                   a.out-791   [004] ....  4826.499532: 0: rx queue: 1
                   a.out-791   [004] ....  4826.499541: 0: rx queue: 2
                   a.out-791   [004] ....  4826.499549: 0: rx queue: 3

While at it, I also fixed tun_xdp_one(), which was calling skb_record_rx_queue() after invoking do_xdp_generic().

Gilberto Bertin (1):
net: tun: record RX queue in skb before do_xdp_generic()

 drivers/net/tun.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
Signature · 1 line
2.20.1