bpf: cpumap: report queue_index to xdp_rxq_info
Subsystems (5)
RFC PATCH 1/1 bpf: cpumap: report queue_index to xdp_rxq_info
When a packet is redirected to a CPU map entry, cpu_map_bpf_prog_run_xdp() reconstructs a minimal xdp_rxq_info from xdp_frame fields (dev_rx and mem_type) before re-running the BPF program on the target CPU. However, queue_index was never preserved across the CPU boundary, so BPF programs running in cpumap context always observe ctx->rx_queue_index == 0, regardless of which hardware queue originally received the packet.
Fix this by storing the originating queue_index in struct xdp_frame, following the same pattern already established for dev_rx and mem_type. The field is populated from rxq->queue_index in xdp_convert_buff_to_frame() during NAPI context, when the rxq_info is still valid, and restored into the reconstructed rxq_info in cpu_map_bpf_prog_run_xdp().
Also use xdpf->queue_index in __xdp_build_skb_from_frame() to call skb_record_rx_queue(), which was previously listed as missing information in that function's comment.
Also propagate queue_index in dpaa_a050385_wa_xdpf(), which manually constructs a new xdp_frame from an uninitialized page. Without this, queue_index would contain stale data from the page allocator.
---
Note: this patch was only compiled, not tested.
drivers/net/ethernet/freescale/dpaa/dpaa_eth.c | 1 + include/net/xdp.h | 4 +++- kernel/bpf/cpumap.c | 2 +- net/core/xdp.c | 2 +- 4 files changed, 6 insertions(+), 3 deletions(-)
drivers/net/ethernet/freescale/dpaa/dpaa_eth.c +1 -0
@@ -2281,6 +2281,7 @@ static int dpaa_a050385_wa_xdpf(struct dpaa_priv *priv, new_xdpf->headroom = priv->tx_headroom; new_xdpf->frame_sz =DPAA_BP_RAW_SIZE ; new_xdpf->mem_type =MEM_TYPE_PAGE_ORDER0 ; + new_xdpf->queue_index = xdpf->queue_index;/* Release the initial buffer */ xdp_return_frame_rx_napi (xdpf);
include/net/xdp.h +3 -1
@@ -297,10 +297,11 @@ struct xdp_frame {u32 headroom;u32 metasize;/* uses lower 8-bits */ /* Lifetime of xdp_rxq_info is limited to NAPI/enqueue time, -* while mem_type is valid on remote CPU. +* while mem_type and queue_index are valid on remote CPU. */ enum xdp_mem_type mem_type:32;struct net_device *dev_rx;/* used by cpumap */ +u32 queue_index;/* used by cpumap */ u32 frame_sz;u32 flags;/* supported values defined in xdp_buff_flags */ };
@@ -441,6 +442,7 @@ struct xdp_frame *xdp_convert_buff_to_frame(struct xdp_buff *xdp)/* rxq only valid until napi_schedule ends, convert to xdp_mem_type */ xdp_frame->mem_type = xdp->rxq->mem.type; + xdp_frame->queue_index = xdp->rxq->queue_index;return xdp_frame; }
kernel/bpf/cpumap.c +1 -1
@@ -197,7 +197,7 @@ static int cpu_map_bpf_prog_run_xdp(struct bpf_cpu_map_entry *rcpu, rxq.dev = xdpf->dev_rx; rxq.mem.type = xdpf->mem_type; -/* TODO: report queue_index to xdp_rxq_info */ + rxq.queue_index = xdpf->queue_index;xdp_convert_frame_to_buff (xdpf, &xdp);
net/core/xdp.c +1 -1
@@ -829,11 +829,11 @@ struct sk_buff *__xdp_build_skb_from_frame(struct xdp_frame *xdpf,/* Essential SKB info: protocol and skb->dev */ skb->protocol =eth_type_trans (skb, dev); +skb_record_rx_queue (skb, xdpf->queue_index);/* Optional SKB info, currently missing: * - HW checksum info (skb->ip_summed) * - HW RX hash (skb_set_hash) -* - RX ring dev queue index (skb_record_rx_queue) */ if (xdpf->mem_type ==MEM_TYPE_PAGE_POOL )
--
2.53.0
Quoted 76 lines · Jose A. Perez de Azpillaga
When a packet is redirected to a CPU map entry,
cpu_map_bpf_prog_run_xdp()reconstructs a minimalxdp_rxq_infofromxdp_framefields (dev_rxandmem_type) before re-running the BPF program on the target CPU. However,queue_indexwas never preserved across the CPU boundary, so BPF programs running in cpumap context always observectx->rx_queue_index== 0, regardless of which hardware queue originally received the packet.Fix this by storing the originating
queue_indexin structxdp_frame, following the same pattern already established fordev_rxandmem_type. The field is populated fromrxq->queue_indexinxdp_convert_buff_to_frame()during NAPI context, when therxq_infois still valid, and restored into the reconstructedrxq_infoincpu_map_bpf_prog_run_xdp().Also use
xdpf->queue_indexin__xdp_build_skb_from_frame()to callskb_record_rx_queue(), which was previously listed as missing information in that function's comment.Also propagate
queue_indexindpaa_a050385_wa_xdpf(), which manually constructs a newxdp_framefrom an uninitialized page. Without this,queue_indexwould contain stale data from the page allocator.Signed-off-by Jose A. Perez de Azpillaga <[email protected]>---
Note: this patch was only compiled, not tested.drivers/net/ethernet/freescale/dpaa/dpaa_eth.c | 1 + include/net/xdp.h | 4 +++- kernel/bpf/cpumap.c | 2 +- net/core/xdp.c | 2 +- 4 files changed, 6 insertions(+), 3 deletions(-)drivers/net/ethernet/freescale/dpaa/dpaa_eth.c +1
-0@@ -2281,6 +2281,7 @@ static int dpaa_a050385_wa_xdpf(struct dpaa_priv *priv, new_xdpf->headroom = priv->tx_headroom; new_xdpf->frame_sz =DPAA_BP_RAW_SIZE ; new_xdpf->mem_type =MEM_TYPE_PAGE_ORDER0 ; + new_xdpf->queue_index = xdpf->queue_index;/* Release the initial buffer */ xdp_return_frame_rx_napi (xdpf);include/net/xdp.h +3
-1@@ -297,10 +297,11 @@ struct xdp_frame {u32 headroom;u32 metasize;/* uses lower 8-bits */ /* Lifetime of xdp_rxq_info is limited to NAPI/enqueue time, -* while mem_type is valid on remote CPU. +* while mem_type and queue_index are valid on remote CPU. */ enum xdp_mem_type mem_type:32;struct net_device *dev_rx;/* used by cpumap */ +u32 queue_index;/* used by cpumap */ u32 frame_sz;u32 flags;/* supported values defined in xdp_buff_flags */ };@@ -441,6 +442,7 @@ struct xdp_frame *xdp_convert_buff_to_frame(struct xdp_buff *xdp)/* rxq only valid until napi_schedule ends, convert to xdp_mem_type */ xdp_frame->mem_type = xdp->rxq->mem.type; + xdp_frame->queue_index = xdp->rxq->queue_index;return xdp_frame; }kernel/bpf/cpumap.c +0
-0
kernel/bpf/cpumap.c static int cpu_map_bpf_prog_run_xdp(struct bpf_cpu_map_entry *rcpu, · line 200
Quoted 5 lines
kernel/bpf/cpumap.c +1
-1@@ -197,7 +197,7 @@ static int cpu_map_bpf_prog_run_xdp(struct bpf_cpu_map_entry *rcpu, rxq.dev = xdpf->dev_rx; rxq.mem.type = xdpf->mem_type; -/* TODO: report queue_index to xdp_rxq_info */ + rxq.queue_index = xdpf->queue_index;
This is like 5th time people attempt to address this TODO.
Just remove that comment. Don't send broken patches.
pw-bot: cr
Quoted 79 lines · Alexei Starovoitov
Quoted 77 lines · Jose A. Perez de Azpillaga
When a packet is redirected to a CPU map entry,
cpu_map_bpf_prog_run_xdp()reconstructs a minimalxdp_rxq_infofromxdp_framefields (dev_rxandmem_type) before re-running the BPF program on the target CPU. However,queue_indexwas never preserved across the CPU boundary, so BPF programs running in cpumap context always observectx->rx_queue_index== 0, regardless of which hardware queue originally received the packet.Fix this by storing the originating
queue_indexin structxdp_frame, following the same pattern already established fordev_rxandmem_type. The field is populated fromrxq->queue_indexinxdp_convert_buff_to_frame()during NAPI context, when therxq_infois still valid, and restored into the reconstructedrxq_infoincpu_map_bpf_prog_run_xdp().Also use
xdpf->queue_indexin__xdp_build_skb_from_frame()to callskb_record_rx_queue(), which was previously listed as missing information in that function's comment.Also propagate
queue_indexindpaa_a050385_wa_xdpf(), which manually constructs a newxdp_framefrom an uninitialized page. Without this,queue_indexwould contain stale data from the page allocator.Signed-off-by Jose A. Perez de Azpillaga <[email protected]>---
Note: this patch was only compiled, not tested.drivers/net/ethernet/freescale/dpaa/dpaa_eth.c | 1 + include/net/xdp.h | 4 +++- kernel/bpf/cpumap.c | 2 +- net/core/xdp.c | 2 +- 4 files changed, 6 insertions(+), 3 deletions(-)drivers/net/ethernet/freescale/dpaa/dpaa_eth.c +1
-0@@ -2281,6 +2281,7 @@ static int dpaa_a050385_wa_xdpf(struct dpaa_priv *priv, new_xdpf->headroom = priv->tx_headroom; new_xdpf->frame_sz =DPAA_BP_RAW_SIZE ; new_xdpf->mem_type =MEM_TYPE_PAGE_ORDER0 ; + new_xdpf->queue_index = xdpf->queue_index;/* Release the initial buffer */ xdp_return_frame_rx_napi (xdpf);include/net/xdp.h +3
-1@@ -297,10 +297,11 @@ struct xdp_frame {u32 headroom;u32 metasize;/* uses lower 8-bits */ /* Lifetime of xdp_rxq_info is limited to NAPI/enqueue time, -* while mem_type is valid on remote CPU. +* while mem_type and queue_index are valid on remote CPU. */ enum xdp_mem_type mem_type:32;struct net_device *dev_rx;/* used by cpumap */ +u32 queue_index;/* used by cpumap */ u32 frame_sz;u32 flags;/* supported values defined in xdp_buff_flags */ };@@ -441,6 +442,7 @@ struct xdp_frame *xdp_convert_buff_to_frame(struct xdp_buff *xdp)/* rxq only valid until napi_schedule ends, convert to xdp_mem_type */ xdp_frame->mem_type = xdp->rxq->mem.type; + xdp_frame->queue_index = xdp->rxq->queue_index;return xdp_frame; }kernel/bpf/cpumap.c +0
-0@@ -197,7 +197,7 @@ static int cpu_map_bpf_prog_run_xdp(struct bpf_cpu_map_entry *rcpu,
Quoted 8 lines
kernel/bpf/cpumap.c static int cpu_map_bpf_prog_run_xdp(struct bpf_cpu_map_entry *rcpu, · line 200
Quoted 4 lines
kernel/bpf/cpumap.c +1
-1rxq.dev = xdpf->dev_rx; rxq.mem.type = xdpf->mem_type; -/* TODO: report queue_index to xdp_rxq_info */ + rxq.queue_index = xdpf->queue_index;This is like 5th time people attempt to address this TODO.
Just remove that comment. Don't send broken patches.
oh... okay. but I have a question, since the bot detected something I didn't and queue_index should be propagated in xdp_convert_zc_to_xdp_frame(), or maybe intentional?
is it better to do as you said, removing the comment, or doing what the bot said with proper test?
--
regards,
jose a. p-a
[email protected]
AI reviewIs the updated comment accurate for all paths?
Quoted 6 lines
include/net/xdp.h +0
-0@@ -297,10 +297,11 @@ struct xdp_frame {u32 headroom;u32 metasize;/* uses lower 8-bits */
include/net/xdp.h struct xdp_frame { · line 307
Quoted 10 lines
include/net/xdp.h +2
-1/* Lifetime of xdp_rxq_info is limited to NAPI/enqueue time, -* while mem_type is valid on remote CPU. +* while mem_type and queue_index are valid on remote CPU. */ enum xdp_mem_type mem_type:32;struct net_device *dev_rx;/* used by cpumap */ +u32 queue_index;/* used by cpumap */ u32 frame_sz;u32 flags;/* supported values defined in xdp_buff_flags */ };
Is the updated comment accurate for all paths?
xdp_convert_buff_to_frame() has an early return for the zero-copy path:
if (xdp->rxq->mem.type == MEM_TYPE_XSK_BUFF_POOL)
return xdp_convert_zc_to_xdp_frame(xdp);xdp_convert_zc_to_xdp_frame() zeroes the frame with memset and explicitly sets mem_type but never sets queue_index:
net/core/xdp.c:xdp_convert_zc_to_xdp_frame() {
...
memset(xdpf, 0, sizeof(*xdpf));
...
xdpf->mem_type = MEM_TYPE_PAGE_ORDER0;
/* queue_index is not set -- remains 0 from memset */
...
}The comment says queue_index is "valid on remote CPU" but for zero-copy converted frames it will always be 0 regardless of which hardware queue originated the packet.
include/net/xdp.h struct xdp_frame *xdp_convert_buff_to_frame(struct xdp_buff *xdp) · line 447
Quoted 7 lines
include/net/xdp.h +1
-0@@ -441,6 +442,7 @@ struct xdp_frame *xdp_convert_buff_to_frame(struct xdp_buff *xdp)/* rxq only valid until napi_schedule ends, convert to xdp_mem_type */ xdp_frame->mem_type = xdp->rxq->mem.type; + xdp_frame->queue_index = xdp->rxq->queue_index;return xdp_frame;
This assignment is only reached for the non-ZC path. Should xdp_convert_zc_to_xdp_frame() also set queue_index from xdp->rxq->queue_index to match?
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/24288456014