netvsc fixes to VF frame handling
Subsystems (3)
cover netvsc fixes to VF frame handling
XDP generic would not work with the transparent VF support in the netvsc device. This problem is generic and probably also happens with failsafe, team, bridge, bond and any other device with a receive handler.
Stephen Hemminger (2):
netvsc: invoke xdp_generic from VF frame handler
netvsc: unshare skb in VF rx handler
drivers/net/hyperv/netvsc_drv.c | 12 ++++++++++++ 1 file changed, 12 insertions(+)
Signature · 1 line
2.20.1
RFC 1/2 netvsc: invoke xdp_generic from VF frame handler
XDP generic does not work correctly with the Hyper-V/Azure netvsc device because of packet processing order. Only packets on the synthetic path get seen by the XDP program. The VF device packets are not seen.
By the time the packets that arrive on the VF are handled by netvsc after the first pass of XDP generic (on the VF) has already been done.
A fix for the netvsc device is to do this in the VF packet handler. by directly calling do_xdp_generic() if XDP program is present on the parent device.
A riskier but maybe better alternative would be to do this netdev core code after the receive handler is invoked (if RX_HANDLER_ANOTHER is returned).
--- drivers/net/hyperv/netvsc_drv.c | 6 ++++++ 1 file changed, 6 insertions(+)
drivers/net/hyperv/netvsc_drv.c +6 -0
@@ -1999,9 +1999,15 @@ static rx_handler_result_t netvsc_vf_handle_frame(struct sk_buff **pskb)struct net_device_context *ndev_ctx =netdev_priv (ndev);struct netvsc_vf_pcpu_stats *pcpu_stats =this_cpu_ptr (ndev_ctx->vf_stats); +struct bpf_prog *xdp_prog; skb->dev = ndev; + xdp_prog =rcu_dereference (ndev->xdp_prog); +if (xdp_prog && +do_xdp_generic (xdp_prog, skb) !=XDP_PASS ) +return RX_HANDLER_CONSUMED ; +u64_stats_update_begin (&pcpu_stats->syncp); pcpu_stats->rx_packets++; pcpu_stats->rx_bytes += skb->len;
Signature · 1 line
2.20.1
Re: [RFC 1/2] netvsc: invoke xdp_generic from VF frame handler
Quoted 9 lines · Stephen Hemminger
XDP generic does not work correctly with the Hyper-V/Azure netvsc device because of packet processing order. Only packets on the synthetic path get seen by the XDP program. The VF device packets are not seen.
By the time the packets that arrive on the VF are handled by netvsc after the first pass of XDP generic (on the VF) has already been done.
Quoted 7 lines
A fix for the netvsc device is to do this in the VF packet handler. by directly calling
do_xdp_generic()if XDP program is present on the parent device.A riskier but maybe better alternative would be to do this netdev core code after the receive handler is invoked (if
RX_HANDLER_ANOTHERis returned).
Something like what I propose at https://lore.kernel.org/patchwork/patch/973819/ ?
It belongs to a series that try to make XDP (both native and generic) work for stacked device. But for some reason (probably performance), the maintainer seems not like the idea.
Maybe it's time to reconsider that?
Thanks
Quoted 27 lines
Fixes 0c195567a8f6 ("netvsc: transparent VF management")Signed-off-by Stephen Hemminger <[email protected]>--- drivers/net/hyperv/netvsc_drv.c | 6 ++++++ 1 file changed, 6 insertions(+)drivers/net/hyperv/netvsc_drv.c +6
-0@@ -1999,9 +1999,15 @@ static rx_handler_result_t netvsc_vf_handle_frame(struct sk_buff **pskb)struct net_device_context *ndev_ctx =netdev_priv (ndev);struct netvsc_vf_pcpu_stats *pcpu_stats =this_cpu_ptr (ndev_ctx->vf_stats); +struct bpf_prog *xdp_prog; skb->dev = ndev; + xdp_prog =rcu_dereference (ndev->xdp_prog); +if (xdp_prog && +do_xdp_generic (xdp_prog, skb) !=XDP_PASS ) +return RX_HANDLER_CONSUMED ; +u64_stats_update_begin (&pcpu_stats->syncp); pcpu_stats->rx_packets++; pcpu_stats->rx_bytes += skb->len;
Re: [RFC 1/2] netvsc: invoke xdp_generic from VF frame handler
Quoted 22 lines · On Wed, 15 May 2019 16:12:42 +0800 Jason Wang <[email protected]> wrote:
Quoted 9 lines · Stephen Hemminger
XDP generic does not work correctly with the Hyper-V/Azure netvsc device because of packet processing order. Only packets on the synthetic path get seen by the XDP program. The VF device packets are not seen.
By the time the packets that arrive on the VF are handled by netvsc after the first pass of XDP generic (on the VF) has already been done.
Quoted 7 lines
A fix for the netvsc device is to do this in the VF packet handler. by directly calling
do_xdp_generic()if XDP program is present on the parent device.A riskier but maybe better alternative would be to do this netdev core code after the receive handler is invoked (if
RX_HANDLER_ANOTHERis returned).Something like what I propose at
https://lore.kernel.org/patchwork/patch/973819/ ?
Quoted 7 lines
It belongs to a series that try to make XDP (both native and generic) work for stacked device. But for some reason (probably performance), the maintainer seems not like the idea.
Maybe it's time to reconsider that?
Thanks
I like your generic solution but it introduces a change in semantics. Netvsc always changes device when returning a ANOTHER but do all devices? If some other stacked device did this then there a chance that using XDP on that device would see same packet twice.
Re: [RFC 1/2] netvsc: invoke xdp_generic from VF frame handler
Quoted 28 lines · Stephen Hemminger
Quoted 25 lines · On Wed, 15 May 2019 16:12:42 +0800 Jason Wang <[email protected]> wrote:
Quoted 9 lines · Stephen Hemminger
XDP generic does not work correctly with the Hyper-V/Azure netvsc device because of packet processing order. Only packets on the synthetic path get seen by the XDP program. The VF device packets are not seen.
By the time the packets that arrive on the VF are handled by netvsc after the first pass of XDP generic (on the VF) has already been done.
Quoted 7 lines
A fix for the netvsc device is to do this in the VF packet handler. by directly calling
do_xdp_generic()if XDP program is present on the parent device.A riskier but maybe better alternative would be to do this netdev core code after the receive handler is invoked (if
RX_HANDLER_ANOTHERis returned).Something like what I propose at
https://lore.kernel.org/patchwork/patch/973819/ ?It belongs to a series that try to make XDP (both native and generic) work for stacked device. But for some reason (probably performance), the maintainer seems not like the idea.
Quoted 8 lines
Quoted 3 lines
Maybe it's time to reconsider that?
Thanks
I like your generic solution but it introduces a change in semantics. Netvsc always changes device when returning a ANOTHER but do all devices? If some other stacked device did this then there a chance that using XDP on that device would see same packet twice.
Good point. Can we simply add a check and call XDP only if dev is changed in this case?
Thanks
Haiyang Zhang
maintainerRE: [RFC 1/2] netvsc: invoke xdp_generic from VF frame handler
In reply to Stephen HemmingerQuoted 37 lines
-----Original Message-----
From: Stephen Hemminger <[email protected]>
Sent: Wednesday, May 15, 2019 4:03 AM
To: KY Srinivasan <[email protected]>; Haiyang Zhang <[email protected]>; [email protected]
Cc: [email protected]; Stephen Hemminger <[email protected]>
Subject: [RFC 1/2] netvsc: invokexdp_genericfrom VF frame handlerXDP generic does not work correctly with the Hyper-V/Azure netvsc device because of packet processing order. Only packets on the synthetic path get seen by the XDP program. The VF device packets are not seen.
By the time the packets that arrive on the VF are handled by netvsc after the first pass of XDP generic (on the VF) has already been done.
A fix for the netvsc device is to do this in the VF packet handler. by directly calling
do_xdp_generic()if XDP program is present on the parent device.A riskier but maybe better alternative would be to do this netdev core code after the receive handler is invoked (if
RX_HANDLER_ANOTHERis returned).Fixes 0c195567a8f6 ("netvsc: transparent VF management")Signed-off-by Stephen Hemminger <[email protected]>--- drivers/net/hyperv/netvsc_drv.c | 6 ++++++ 1 file changed, 6 insertions(+)drivers/net/hyperv/netvsc_drv.c +0
-0@@ -1999,9 +1999,15 @@ static rx_handler_result_t netvsc_vf_handle_frame(struct sk_buff **pskb) struct net_device_context *ndev_ctx = netdev_priv(ndev);
drivers/net/hyperv/netvsc_drv.c static rx_handler_result_t · line 2010
Quoted 10 lines
drivers/net/hyperv/netvsc_drv.c +5
-0struct netvsc_vf_pcpu_stats *pcpu_stats =this_cpu_ptr (ndev_ctx->vf_stats); +struct bpf_prog *xdp_prog; skb->dev = ndev; + xdp_prog =rcu_dereference (ndev->xdp_prog); +if (xdp_prog && +do_xdp_generic (xdp_prog, skb) !=XDP_PASS ) +return RX_HANDLER_CONSUMED ;
Looks fine overall.
The function do_xdp_generic() already checks NULL on xdp_prog, so we don't need to check it in our code.
int do_xdp_generic(struct bpf_prog *xdp_prog, struct sk_buff *skb)
{
if (xdp_prog) {Thanks,
- Haiyang
Re: [RFC 1/2] netvsc: invoke xdp_generic from VF frame handler
Quoted 48 lines · On Wed, 15 May 2019 17:50:25 +0000 Haiyang Zhang <[email protected]> wrote:
Quoted 35 lines
-----Original Message-----
From: Stephen Hemminger <[email protected]>
Sent: Wednesday, May 15, 2019 4:03 AM
To: KY Srinivasan <[email protected]>; Haiyang Zhang <[email protected]>; [email protected]
Cc: [email protected]; Stephen Hemminger <[email protected]>
Subject: [RFC 1/2] netvsc: invokexdp_genericfrom VF frame handlerXDP generic does not work correctly with the Hyper-V/Azure netvsc device because of packet processing order. Only packets on the synthetic path get seen by the XDP program. The VF device packets are not seen.
By the time the packets that arrive on the VF are handled by netvsc after the first pass of XDP generic (on the VF) has already been done.
A fix for the netvsc device is to do this in the VF packet handler. by directly calling
do_xdp_generic()if XDP program is present on the parent device.A riskier but maybe better alternative would be to do this netdev core code after the receive handler is invoked (if
RX_HANDLER_ANOTHERis returned).Fixes 0c195567a8f6 ("netvsc: transparent VF management")Signed-off-by Stephen Hemminger <[email protected]>--- drivers/net/hyperv/netvsc_drv.c | 6 ++++++ 1 file changed, 6 insertions(+)drivers/net/hyperv/netvsc_drv.c +0
-0@@ -1999,9 +1999,15 @@ static rx_handler_result_t netvsc_vf_handle_frame(struct sk_buff **pskb) struct net_device_context *ndev_ctx = netdev_priv(ndev);drivers/net/hyperv/netvsc_drv.c static rx_handler_result_t · line 2010
Quoted 10 lines
drivers/net/hyperv/netvsc_drv.c +5
-0struct netvsc_vf_pcpu_stats *pcpu_stats =this_cpu_ptr (ndev_ctx->vf_stats); +struct bpf_prog *xdp_prog; skb->dev = ndev; + xdp_prog =rcu_dereference (ndev->xdp_prog); +if (xdp_prog && +do_xdp_generic (xdp_prog, skb) !=XDP_PASS ) +return RX_HANDLER_CONSUMED ;Looks fine overall.
Quoted 7 lines
The function
do_xdp_generic()already checks NULL onxdp_prog, so we don't need to check it in our code.int
do_xdp_generic(struct bpf_prog *xdp_prog, struct sk_buff *skb){ if (xdp_prog) {
The null check in the netvsc code was just an minor optimization to avoid unnecessary function call in fast path.
RE: [RFC 1/2] netvsc: invoke xdp_generic from VF frame handler
Quoted 68 lines
-----Original Message-----
From: Stephen Hemminger <[email protected]>
Sent: Wednesday, May 15, 2019 1:54 PM
To: Haiyang Zhang <[email protected]>
Cc: KY Srinivasan <[email protected]>; [email protected]; [email protected]; Stephen Hemminger <[email protected]>
Subject: Re: [RFC 1/2] netvsc: invokexdp_genericfrom VF frame handlerQuoted 6 lines · On Wed, 15 May 2019 17:50:25 +0000 Haiyang Zhang <[email protected]> wrote:
Quoted 6 lines
-----Original Message-----
From: Stephen Hemminger <[email protected]>
Sent: Wednesday, May 15, 2019 4:03 AM
To: KY Srinivasan <[email protected]>; Haiyang Zhang <[email protected]>; [email protected]
Cc: [email protected]; Stephen HemmingerQuoted 7 lines
Quoted 7 lines
Subject: [RFC 1/2] netvsc: invoke
xdp_genericfrom VF frame handlerXDP generic does not work correctly with the Hyper-V/Azure netvsc device because of packet processing order. Only packets on the synthetic path get seen by the XDP program. The VF device packets are not seen.
By the time the packets that arrive on the VF are handled by netvsc after
the
Quoted 4 lines
Quoted 4 lines
first pass of XDP generic (on the VF) has already been done.
A fix for the netvsc device is to do this in the VF packet handler. by directly calling
do_xdp_generic()if XDP program is present on theparent
Quoted 5 lines
Quoted 5 lines
device.
A riskier but maybe better alternative would be to do this netdev core code after the receive handler is invoked (if
RX_HANDLER_ANOTHERis returned).Quoted 7 lines
Review
Quoted 7 lines
Fixes 0c195567a8f6 ("netvsc: transparent VF management")Signed-off-by Stephen Hemminger <[email protected]>--- drivers/net/hyperv/netvsc_drv.c | 6 ++++++ 1 file changed, 6 insertions(+)(quoted diff) +0
-0
b/drivers/net/hyperv/netvsc_drv.cQuoted 22 lines
Quoted 6 lines
index
06393b215102..bb0fc1869bde100644drivers/net/hyperv/netvsc_drv.c +0
-0@@ -1999,9 +1999,15 @@ static rx_handler_result_t netvsc_vf_handle_frame(struct sk_buff **pskb) struct net_device_context *ndev_ctx = netdev_priv(ndev);drivers/net/hyperv/netvsc_drv.c static rx_handler_result_t · line 2010
Quoted 10 lines
drivers/net/hyperv/netvsc_drv.c +5
-0struct netvsc_vf_pcpu_stats *pcpu_stats =this_cpu_ptr (ndev_ctx->vf_stats); +struct bpf_prog *xdp_prog; skb->dev = ndev; + xdp_prog =rcu_dereference (ndev->xdp_prog); +if (xdp_prog && +do_xdp_generic (xdp_prog, skb) !=XDP_PASS ) +return RX_HANDLER_CONSUMED ;Looks fine overall.
The function
do_xdp_generic()already checks NULL onxdp_prog, so we don't need to check it in our code.
Quoted 7 lines
Quoted 4 lines
int
do_xdp_generic(struct bpf_prog *xdp_prog, struct sk_buff *skb){ if (xdp_prog) {The null check in the netvsc code was just an minor optimization to avoid unnecessary function call in fast path.
Thanks for the explanation.
RFC 2/2 netvsc: unshare skb in VF rx handler
In reply to Stephen HemmingerThe netvsc VF skb handler should make sure that skb is not shared. Similar logic already exists in bonding and team device drivers.
This does not happen in practice because the mlx device driver does not return shared skb's.
--- drivers/net/hyperv/netvsc_drv.c | 6 ++++++ 1 file changed, 6 insertions(+)
drivers/net/hyperv/netvsc_drv.c +6 -0
@@ -2001,6 +2001,12 @@ static rx_handler_result_t netvsc_vf_handle_frame(struct sk_buff **pskb) = this_cpu_ptr(ndev_ctx->vf_stats);struct bpf_prog *xdp_prog; + skb =skb_share_check (skb,GFP_ATOMIC ); +if (unlikely (!skb)) +return RX_HANDLER_CONSUMED ; + + *pskb = skb; + skb->dev = ndev; xdp_prog =rcu_dereference (ndev->xdp_prog);
Signature · 1 line
2.20.1