xdp: Fixing skb->pp_recycle flag in generic XDP handling
Subsystems (2)
PATCH 1/1 xdp: Fixing skb->pp_recycle flag in generic XDP handling
In the generic XDP processing flow, if an skb with a page pool page (skb->pp_recycle == 1) fails to meet XDP packet requirements, it will undergo head expansion and linearization of fragment data. As a result, skb->head points to a reallocated buffer without any fragments. At this point, the skb will not contain any page pool pages. However, the skb->pp_recycle flag is still set to 1, which is inconsistent with the actual situation. Although it doesn't seem to cause much real harm at the moment(a little nagetive impact on skb_try_coalesce), to avoid potential issues associated with using incorrect skb->pp_recycle information, setting skb->pp_recycle to 0 to reflect the pp state of the skb.
--- net/core/dev.c | 2 ++ 1 file changed, 2 insertions(+)
net/core/dev.c +2 -0
@@ -4934,6 +4934,8 @@ static u32 netif_receive_generic_xdp(struct sk_buff *skb, goto do_drop;if (skb_linearize (skb)) goto do_drop; +if (skb->pp_recycle) + skb->pp_recycle =0 ; } act =bpf_prog_run_generic_xdp (skb, xdp, xdp_prog);
Signature · 1 line
2.40.1
From: Liang Chen <[email protected]>
Date: Wed, 2 Aug 2023 15:04:54 +0800
Quoted 7 lines
In the generic XDP processing flow, if an skb with a page pool page (
skb->pp_recycle== 1) fails to meet XDP packet requirements, it will undergo head expansion and linearization of fragment data. As a result,skb->headpoints to a reallocated buffer without any fragments. At this point, the skb will not contain any page pool pages. However, theskb->pp_recycleflag is still set to 1, which is inconsistent with the actual situation. Although it doesn't seem to cause much real harm at the
This means it must be handled in the function which replaces the head, i.e. pskb_expand_head(). Your change only suppresses one symptom of the issue.
Quoted 1 line
moment(a little nagetive impact on skb_try_coalesce), to avoid potential
^^^^^^^^
negativeQuoted 4 lines
issues associated with using incorrect
skb->pp_recycleinformation, settingskb->pp_recycleto 0 to reflect the pp state of the skb.Signed-off-by Liang Chen <[email protected]>
I won't say for sure, but may be a candidate for the fixes tree, not next. This way it would need a "Fixes:" tag here (above the SoB).
Quoted 9 lines
--- net/core/dev.c | 2 ++ 1 file changed, 2 insertions(+)net/core/dev.c +0
-0
net/core/dev.c static u32 netif_receive_generic_xdp(struct sk_buff *skb, · line 4941
Quoted 8 lines
net/core/dev.c +2
-0@@ -4934,6 +4934,8 @@ static u32 netif_receive_generic_xdp(struct sk_buff *skb, goto do_drop;if (skb_linearize (skb)) goto do_drop; +if (skb->pp_recycle) + skb->pp_recycle =0 ; } act =bpf_prog_run_generic_xdp (skb, xdp, xdp_prog);
Thanks,
Olek
Quoted 12 lines · Alexander Lobakin
From: Liang Chen <[email protected]>
Date: Wed, 2 Aug 2023 15:04:54 +0800Quoted 7 lines
In the generic XDP processing flow, if an skb with a page pool page (
skb->pp_recycle== 1) fails to meet XDP packet requirements, it will undergo head expansion and linearization of fragment data. As a result,skb->headpoints to a reallocated buffer without any fragments. At this point, the skb will not contain any page pool pages. However, theskb->pp_recycleflag is still set to 1, which is inconsistent with the actual situation. Although it doesn't seem to cause much real harm at the
Quoted 4 lines
This means it must be handled in the function which replaces the head, i.e.
pskb_expand_head(). Your change only suppresses one symptom of the issue.
I attempted to do so. But after pskb_expand_head, there may still be skb frags with pp pages left. It is after skb_linearize those frags are removed.
Thanks,
Liang
Quoted 5 lines
Quoted 1 line
moment(a little nagetive impact on skb_try_coalesce), to avoid potential^^^^^^^^ negative
Sure, Thanks.
Quoted 28 lines
Quoted 4 lines
issues associated with using incorrect
skb->pp_recycleinformation, settingskb->pp_recycleto 0 to reflect the pp state of the skb.Signed-off-by Liang Chen <[email protected]>I won't say for sure, but may be a candidate for the fixes tree, not next. This way it would need a "Fixes:" tag here (above the SoB).
Quoted 9 lines
--- net/core/dev.c | 2 ++ 1 file changed, 2 insertions(+)net/core/dev.c +0
-0net/core/dev.c static u32 netif_receive_generic_xdp(struct sk_buff *skb, · line 4941
Quoted 8 lines
net/core/dev.c +2
-0@@ -4934,6 +4934,8 @@ static u32 netif_receive_generic_xdp(struct sk_buff *skb, goto do_drop;if (skb_linearize (skb)) goto do_drop; +if (skb->pp_recycle) + skb->pp_recycle =0 ; } act =bpf_prog_run_generic_xdp (skb, xdp, xdp_prog);Thanks,
Olek
From: Liang Chen <[email protected]>
Date: Thu, 3 Aug 2023 16:25:49 +0800
Quoted 14 lines
Quoted 12 lines · Alexander Lobakin
From: Liang Chen <[email protected]>
Date: Wed, 2 Aug 2023 15:04:54 +0800Quoted 7 lines
In the generic XDP processing flow, if an skb with a page pool page (
skb->pp_recycle== 1) fails to meet XDP packet requirements, it will undergo head expansion and linearization of fragment data. As a result,skb->headpoints to a reallocated buffer without any fragments. At this point, the skb will not contain any page pool pages. However, theskb->pp_recycleflag is still set to 1, which is inconsistent with the actual situation. Although it doesn't seem to cause much real harm at the
Quoted 8 lines
Quoted 4 lines
This means it must be handled in the function which replaces the head, i.e.
pskb_expand_head(). Your change only suppresses one symptom of the issue.I attempted to do so. But after
pskb_expand_head, there may still be skb frags with pp pages left. It is afterskb_linearizethose frags are removed.
Ah, right.
Then you need to handle that in __pskb_pull_tail(). Check at the end of the function whether the skb still has any frags, and if not, clear skb->pp_recycle.
The most correct fix would be to do that in both pskb_expand_head() and __pskb_pull_tail(): iterate over the frags and check if any page still belongs to a page_pool. Then page_pool_return_skb_page() wouldn't hit false-branch after the skb was re-layout.
[...]
Thanks,
Olek
Quoted 32 lines · Alexander Lobakin
From: Liang Chen <[email protected]>
Date: Thu, 3 Aug 2023 16:25:49 +0800Quoted 14 lines
Quoted 12 lines · Alexander Lobakin
From: Liang Chen <[email protected]>
Date: Wed, 2 Aug 2023 15:04:54 +0800Quoted 7 lines
In the generic XDP processing flow, if an skb with a page pool page (
skb->pp_recycle== 1) fails to meet XDP packet requirements, it will undergo head expansion and linearization of fragment data. As a result,skb->headpoints to a reallocated buffer without any fragments. At this point, the skb will not contain any page pool pages. However, theskb->pp_recycleflag is still set to 1, which is inconsistent with the actual situation. Although it doesn't seem to cause much real harm at theQuoted 8 lines
Quoted 4 lines
This means it must be handled in the function which replaces the head, i.e.
pskb_expand_head(). Your change only suppresses one symptom of the issue.I attempted to do so. But after
pskb_expand_head, there may still be skb frags with pp pages left. It is afterskb_linearizethose frags are removed.Ah, right.
Then you need to handle that in__pskb_pull_tail(). Check at the end of the function whether the skb still has any frags, and if not, clearskb->pp_recycle.
Quoted 5 lines
The most correct fix would be to do that in both
pskb_expand_head()and__pskb_pull_tail(): iterate over the frags and check if any page still belongs to apage_pool. Thenpage_pool_return_skb_page()wouldn't hit false-branch after the skb was re-layout.
Yeah, I agree. netif_receive_generic_xdp may not be the only place the flag can get wrong. To make the pp_recycle flag strictly reflect the state of page pool usages, the check should be put in both functions. But, since it's merely an indication that the skb is 'page pool aware,' and considering that the performance impact we observed doesn't affect the native XDP path, addressing this issue doesn't seem worthwhile based on the feedback I've received.
Thanks,
Liang
Quoted 4 lines
[...]
Thanks,
Olek
Jakub Kicinski
maintainerQuoted 10 lines · On Wed, 2 Aug 2023 15:04:54 +0800 Liang Chen wrote:
In the generic XDP processing flow, if an skb with a page pool page (
skb->pp_recycle== 1) fails to meet XDP packet requirements, it will undergo head expansion and linearization of fragment data. As a result,skb->headpoints to a reallocated buffer without any fragments. At this point, the skb will not contain any page pool pages. However, theskb->pp_recycleflag is still set to 1, which is inconsistent with the actual situation. Although it doesn't seem to cause much real harm at themoment(a little nagetive impact on skb_try_coalesce), to avoid potential issues associated with using incorrectskb->pp_recycleinformation, settingskb->pp_recycleto 0 to reflect the pp state of the skb.
pp_recycle just means that the skb is "page pool aware", there's absolutely no harm in having an skb with pp_recycle = 1 and no page pool pages attached.
I vote not to apply this.
Quoted 13 lines · Jakub Kicinski
Quoted 10 lines · On Wed, 2 Aug 2023 15:04:54 +0800 Liang Chen wrote:
In the generic XDP processing flow, if an skb with a page pool page (
skb->pp_recycle== 1) fails to meet XDP packet requirements, it will undergo head expansion and linearization of fragment data. As a result,skb->headpoints to a reallocated buffer without any fragments. At this point, the skb will not contain any page pool pages. However, theskb->pp_recycleflag is still set to 1, which is inconsistent with the actual situation. Although it doesn't seem to cause much real harm at themoment(a little nagetive impact on skb_try_coalesce), to avoid potential issues associated with using incorrectskb->pp_recycleinformation, settingskb->pp_recycleto 0 to reflect the pp state of the skb.
Quoted 3 lines
pp_recyclejust means that the skb is "page pool aware", there's absolutely no harm in having an skb withpp_recycle= 1 and no page pool pages attached.
I don't see it causing an error right now either. But it affects skb_try_coalesce in a negative way from a performance perspective - from->pp_recycle can be falsely true leading to a coalescing failure in "(from->pp_recycle && skb_cloned(from))" test, which otherwise would let the coalesce continue if from->pp_recycle was false. I wonder if that justifies the need for a fix.
Thanks,
Liang
Quoted 2 lines
I vote not to apply this.
Hi Liang
Quoted 19 lines · Liang Chen
Quoted 13 lines · Jakub Kicinski
Quoted 10 lines · On Wed, 2 Aug 2023 15:04:54 +0800 Liang Chen wrote:
In the generic XDP processing flow, if an skb with a page pool page (
skb->pp_recycle== 1) fails to meet XDP packet requirements, it will undergo head expansion and linearization of fragment data. As a result,skb->headpoints to a reallocated buffer without any fragments. At this point, the skb will not contain any page pool pages. However, theskb->pp_recycleflag is still set to 1, which is inconsistent with the actual situation. Although it doesn't seem to cause much real harm at themoment(a little nagetive impact on skb_try_coalesce), to avoid potential issues associated with using incorrectskb->pp_recycleinformation, settingskb->pp_recycleto 0 to reflect the pp state of the skb.Quoted 3 lines
pp_recyclejust means that the skb is "page pool aware", there's absolutely no harm in having an skb withpp_recycle= 1 and no page pool pages attached.
Quoted 6 lines
I don't see it causing an error right now either. But it affects
skb_try_coalescein a negative way from a performance perspective -from->pp_recyclecan be falsely true leading to a coalescing failure in "(from->pp_recycle&&skb_cloned(from))" test, which otherwise would let the coalesce continue iffrom->pp_recyclewas false. I wonder if that justifies the need for a fix.
This applies to non-native XDP code only though, right? IIRC that code was to make xdp easier to use but it's not used when xdp-like performance is needed.
Thanks
/Ilias
Quoted 7 lines
Thanks,
LiangQuoted 2 lines
I vote not to apply this.
Quoted 30 lines · Ilias Apalodimas
Hi Liang
Quoted 19 lines · Liang Chen
Quoted 13 lines · Jakub Kicinski
Quoted 10 lines · On Wed, 2 Aug 2023 15:04:54 +0800 Liang Chen wrote:
In the generic XDP processing flow, if an skb with a page pool page (
skb->pp_recycle== 1) fails to meet XDP packet requirements, it will undergo head expansion and linearization of fragment data. As a result,skb->headpoints to a reallocated buffer without any fragments. At this point, the skb will not contain any page pool pages. However, theskb->pp_recycleflag is still set to 1, which is inconsistent with the actual situation. Although it doesn't seem to cause much real harm at themoment(a little nagetive impact on skb_try_coalesce), to avoid potential issues associated with using incorrectskb->pp_recycleinformation, settingskb->pp_recycleto 0 to reflect the pp state of the skb.Quoted 3 lines
pp_recyclejust means that the skb is "page pool aware", there's absolutely no harm in having an skb withpp_recycle= 1 and no page pool pages attached.Quoted 6 lines
I don't see it causing an error right now either. But it affects
skb_try_coalescein a negative way from a performance perspective -from->pp_recyclecan be falsely true leading to a coalescing failure in "(from->pp_recycle&&skb_cloned(from))" test, which otherwise would let the coalesce continue iffrom->pp_recyclewas false. I wonder if that justifies the need for a fix.
Quoted 3 lines
This applies to non-native XDP code only though, right? IIRC that code was to make xdp easier to use but it's not used when xdp-like performance is needed.
Sure, this applies to non-native XDP code only. Thanks.
Thanks,
Liang
Quoted 10 lines
Thanks
/IliasQuoted 7 lines
Thanks,
LiangQuoted 2 lines
I vote not to apply this.