Skip to contentSkip to conversation

netvsc fixes to VF frame handling

RFC In review Stephen Hemminger 3 participants 9 messages May 15, 2019 – May 16, 2019

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

cover netvsc fixes to VF frame handling

XDP generic would not work with the transparent VF support in the netvsc device. This problem is generic and probably also happens with failsafe, team, bridge, bond and any other device with a receive handler.

Stephen Hemminger (2):
netvsc: invoke xdp_generic from VF frame handler
netvsc: unshare skb in VF rx handler

 drivers/net/hyperv/netvsc_drv.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)
Signature · 1 line
2.20.1
#1 lore raw

RFC 1/2 netvsc: invoke xdp_generic from VF frame handler

XDP generic does not work correctly with the Hyper-V/Azure netvsc device because of packet processing order. Only packets on the synthetic path get seen by the XDP program. The VF device packets are not seen.

By the time the packets that arrive on the VF are handled by netvsc after the first pass of XDP generic (on the VF) has already been done.

A fix for the netvsc device is to do this in the VF packet handler. by directly calling do_xdp_generic() if XDP program is present on the parent device.

A riskier but maybe better alternative would be to do this netdev core code after the receive handler is invoked (if RX_HANDLER_ANOTHER is returned).

Fixes 0c195567a8f6 ("netvsc: transparent VF management")
---
 drivers/net/hyperv/netvsc_drv.c | 6 ++++++
 1 file changed, 6 insertions(+)
drivers/net/hyperv/netvsc_drv.c +6 -0
index 06393b215102..bb0fc1869bde 100644
@@ -1999,9 +1999,15 @@ static rx_handler_result_t netvsc_vf_handle_frame(struct sk_buff **pskb)
 	struct net_device_context *ndev_ctx = netdev_priv(ndev);
 	struct netvsc_vf_pcpu_stats *pcpu_stats
 		 = this_cpu_ptr(ndev_ctx->vf_stats);
+	struct bpf_prog *xdp_prog;
 
 	skb->dev = ndev;
 
+	xdp_prog = rcu_dereference(ndev->xdp_prog);
+	if (xdp_prog &&
+	    do_xdp_generic(xdp_prog, skb) != XDP_PASS)
+		return RX_HANDLER_CONSUMED;
+
 	u64_stats_update_begin(&pcpu_stats->syncp);
 	pcpu_stats->rx_packets++;
 	pcpu_stats->rx_bytes += skb->len;
Signature · 1 line
2.20.1
#2 lore raw

Re: [RFC 1/2] netvsc: invoke xdp_generic from VF frame handler

Quoted 9 lines · Stephen Hemminger

XDP generic does not work correctly with the Hyper-V/Azure netvsc device because of packet processing order. Only packets on the synthetic path get seen by the XDP program. The VF device packets are not seen.

By the time the packets that arrive on the VF are handled by netvsc after the first pass of XDP generic (on the VF) has already been done.

Quoted 7 lines

A fix for the netvsc device is to do this in the VF packet handler. by directly calling do_xdp_generic() if XDP program is present on the parent device.

A riskier but maybe better alternative would be to do this netdev core code after the receive handler is invoked (if RX_HANDLER_ANOTHER is returned).

Something like what I propose at https://lore.kernel.org/patchwork/patch/973819/ ?

It belongs to a series that try to make XDP (both native and generic) work for stacked device. But for some reason (probably performance), the maintainer seems not like the idea.

Maybe it's time to reconsider that?

Thanks

Quoted 27 lines
Fixes 0c195567a8f6 ("netvsc: transparent VF management")
---
  drivers/net/hyperv/netvsc_drv.c | 6 ++++++
  1 file changed, 6 insertions(+)
drivers/net/hyperv/netvsc_drv.c +6 -0
index 06393b215102..bb0fc1869bde 100644
@@ -1999,9 +1999,15 @@ static rx_handler_result_t netvsc_vf_handle_frame(struct sk_buff **pskb)
  	struct net_device_context *ndev_ctx = netdev_priv(ndev);
  	struct netvsc_vf_pcpu_stats *pcpu_stats
  		 = this_cpu_ptr(ndev_ctx->vf_stats);
+	struct bpf_prog *xdp_prog;
 
  	skb->dev = ndev;
 
+	xdp_prog = rcu_dereference(ndev->xdp_prog);
+	if (xdp_prog &&
+	    do_xdp_generic(xdp_prog, skb) != XDP_PASS)
+		return RX_HANDLER_CONSUMED;
+
  	u64_stats_update_begin(&pcpu_stats->syncp);
  	pcpu_stats->rx_packets++;
  	pcpu_stats->rx_bytes += skb->len;
#3 lore raw

Re: [RFC 1/2] netvsc: invoke xdp_generic from VF frame handler

Quoted 22 lines · On Wed, 15 May 2019 16:12:42 +0800 Jason Wang <[email protected]> wrote:
Quoted 9 lines · Stephen Hemminger

XDP generic does not work correctly with the Hyper-V/Azure netvsc device because of packet processing order. Only packets on the synthetic path get seen by the XDP program. The VF device packets are not seen.

By the time the packets that arrive on the VF are handled by netvsc after the first pass of XDP generic (on the VF) has already been done.

Quoted 7 lines

A fix for the netvsc device is to do this in the VF packet handler. by directly calling do_xdp_generic() if XDP program is present on the parent device.

A riskier but maybe better alternative would be to do this netdev core code after the receive handler is invoked (if RX_HANDLER_ANOTHER is returned).

Something like what I propose at
https://lore.kernel.org/patchwork/patch/973819/ ?

Quoted 7 lines

It belongs to a series that try to make XDP (both native and generic) work for stacked device. But for some reason (probably performance), the maintainer seems not like the idea.

Maybe it's time to reconsider that?

Thanks

I like your generic solution but it introduces a change in semantics. Netvsc always changes device when returning a ANOTHER but do all devices? If some other stacked device did this then there a chance that using XDP on that device would see same packet twice.

#4 lore raw

Re: [RFC 1/2] netvsc: invoke xdp_generic from VF frame handler

Quoted 28 lines · Stephen Hemminger
Quoted 25 lines · On Wed, 15 May 2019 16:12:42 +0800 Jason Wang <[email protected]> wrote:
Quoted 9 lines · Stephen Hemminger

XDP generic does not work correctly with the Hyper-V/Azure netvsc device because of packet processing order. Only packets on the synthetic path get seen by the XDP program. The VF device packets are not seen.

By the time the packets that arrive on the VF are handled by netvsc after the first pass of XDP generic (on the VF) has already been done.

Quoted 7 lines

A fix for the netvsc device is to do this in the VF packet handler. by directly calling do_xdp_generic() if XDP program is present on the parent device.

A riskier but maybe better alternative would be to do this netdev core code after the receive handler is invoked (if RX_HANDLER_ANOTHER is returned).

Something like what I propose at
https://lore.kernel.org/patchwork/patch/973819/ ?

It belongs to a series that try to make XDP (both native and generic) work for stacked device. But for some reason (probably performance), the maintainer seems not like the idea.

Quoted 8 lines
Quoted 3 lines

Maybe it's time to reconsider that?

Thanks

I like your generic solution but it introduces a change in semantics. Netvsc always changes device when returning a ANOTHER but do all devices? If some other stacked device did this then there a chance that using XDP on that device would see same packet twice.

Good point.  Can we simply add a check and call XDP only if dev is changed in this case?

Thanks

maintainer
#5 lore raw

RE: [RFC 1/2] netvsc: invoke xdp_generic from VF frame handler

In reply to Stephen Hemminger
Quoted 37 lines

-----Original Message-----
From: Stephen Hemminger <[email protected]>
Sent: Wednesday, May 15, 2019 4:03 AM
To: KY Srinivasan <[email protected]>; Haiyang Zhang <[email protected]>; [email protected]
Cc: [email protected]; Stephen Hemminger <[email protected]>
Subject: [RFC 1/2] netvsc: invoke xdp_generic from VF frame handler

XDP generic does not work correctly with the Hyper-V/Azure netvsc device because of packet processing order. Only packets on the synthetic path get seen by the XDP program. The VF device packets are not seen.

By the time the packets that arrive on the VF are handled by netvsc after the first pass of XDP generic (on the VF) has already been done.

A fix for the netvsc device is to do this in the VF packet handler. by directly calling do_xdp_generic() if XDP program is present on the parent device.

A riskier but maybe better alternative would be to do this netdev core code after the receive handler is invoked (if RX_HANDLER_ANOTHER is returned).

Fixes 0c195567a8f6 ("netvsc: transparent VF management")
---
 drivers/net/hyperv/netvsc_drv.c | 6 ++++++
 1 file changed, 6 insertions(+)
drivers/net/hyperv/netvsc_drv.c +0 -0
index 06393b215102..bb0fc1869bde 100644
@@ -1999,9 +1999,15 @@ static rx_handler_result_t
 netvsc_vf_handle_frame(struct sk_buff **pskb)
 	struct net_device_context *ndev_ctx = netdev_priv(ndev);

drivers/net/hyperv/netvsc_drv.c static rx_handler_result_t · line 2010

Quoted 10 lines
drivers/net/hyperv/netvsc_drv.c +5 -0
 	struct netvsc_vf_pcpu_stats *pcpu_stats
 		 = this_cpu_ptr(ndev_ctx->vf_stats);
+	struct bpf_prog *xdp_prog;
 
 	skb->dev = ndev;
 
+	xdp_prog = rcu_dereference(ndev->xdp_prog);
+	if (xdp_prog &&
+	    do_xdp_generic(xdp_prog, skb) != XDP_PASS)
+		return RX_HANDLER_CONSUMED;

Looks fine overall.

The function do_xdp_generic() already checks NULL on xdp_prog, so we don't need to check it in our code.

int do_xdp_generic(struct bpf_prog *xdp_prog, struct sk_buff *skb)

{
        if (xdp_prog) {

Thanks,
- Haiyang

#6 lore raw

Re: [RFC 1/2] netvsc: invoke xdp_generic from VF frame handler

Quoted 48 lines · On Wed, 15 May 2019 17:50:25 +0000 Haiyang Zhang <[email protected]> wrote:
Quoted 35 lines

-----Original Message-----
From: Stephen Hemminger <[email protected]>
Sent: Wednesday, May 15, 2019 4:03 AM
To: KY Srinivasan <[email protected]>; Haiyang Zhang <[email protected]>; [email protected]
Cc: [email protected]; Stephen Hemminger <[email protected]>
Subject: [RFC 1/2] netvsc: invoke xdp_generic from VF frame handler

XDP generic does not work correctly with the Hyper-V/Azure netvsc device because of packet processing order. Only packets on the synthetic path get seen by the XDP program. The VF device packets are not seen.

By the time the packets that arrive on the VF are handled by netvsc after the first pass of XDP generic (on the VF) has already been done.

A fix for the netvsc device is to do this in the VF packet handler. by directly calling do_xdp_generic() if XDP program is present on the parent device.

A riskier but maybe better alternative would be to do this netdev core code after the receive handler is invoked (if RX_HANDLER_ANOTHER is returned).

Fixes 0c195567a8f6 ("netvsc: transparent VF management")
---
 drivers/net/hyperv/netvsc_drv.c | 6 ++++++
 1 file changed, 6 insertions(+)
drivers/net/hyperv/netvsc_drv.c +0 -0
index 06393b215102..bb0fc1869bde 100644
@@ -1999,9 +1999,15 @@ static rx_handler_result_t
 netvsc_vf_handle_frame(struct sk_buff **pskb)
 	struct net_device_context *ndev_ctx = netdev_priv(ndev);

drivers/net/hyperv/netvsc_drv.c static rx_handler_result_t · line 2010

Quoted 10 lines
drivers/net/hyperv/netvsc_drv.c +5 -0
 	struct netvsc_vf_pcpu_stats *pcpu_stats
 		 = this_cpu_ptr(ndev_ctx->vf_stats);
+	struct bpf_prog *xdp_prog;
 
 	skb->dev = ndev;
 
+	xdp_prog = rcu_dereference(ndev->xdp_prog);
+	if (xdp_prog &&
+	    do_xdp_generic(xdp_prog, skb) != XDP_PASS)
+		return RX_HANDLER_CONSUMED;  

Looks fine overall.

Quoted 7 lines

The function do_xdp_generic() already checks NULL on xdp_prog, so we don't need to check it in our code.

int do_xdp_generic(struct bpf_prog *xdp_prog, struct sk_buff *skb)

{
        if (xdp_prog) {

The null check in the netvsc code was just an minor optimization to avoid unnecessary function call in fast path.

maintainer Reviewed-by
#7 lore raw

RE: [RFC 1/2] netvsc: invoke xdp_generic from VF frame handler

Quoted 68 lines

-----Original Message-----
From: Stephen Hemminger <[email protected]>
Sent: Wednesday, May 15, 2019 1:54 PM
To: Haiyang Zhang <[email protected]>
Cc: KY Srinivasan <[email protected]>; [email protected]; [email protected]; Stephen Hemminger <[email protected]>
Subject: Re: [RFC 1/2] netvsc: invoke xdp_generic from VF frame handler

Quoted 6 lines · On Wed, 15 May 2019 17:50:25 +0000 Haiyang Zhang <[email protected]> wrote:
Quoted 6 lines

-----Original Message-----
From: Stephen Hemminger <[email protected]>
Sent: Wednesday, May 15, 2019 4:03 AM
To: KY Srinivasan <[email protected]>; Haiyang Zhang <[email protected]>; [email protected]
Cc: [email protected]; Stephen Hemminger

<[email protected]>

Quoted 7 lines
Quoted 7 lines

Subject: [RFC 1/2] netvsc: invoke xdp_generic from VF frame handler

XDP generic does not work correctly with the Hyper-V/Azure netvsc device because of packet processing order. Only packets on the synthetic path get seen by the XDP program. The VF device packets are not seen.

By the time the packets that arrive on the VF are handled by netvsc after

the

Quoted 4 lines
Quoted 4 lines

first pass of XDP generic (on the VF) has already been done.

A fix for the netvsc device is to do this in the VF packet handler. by directly calling do_xdp_generic() if XDP program is present on the

parent

Quoted 5 lines
Quoted 5 lines

device.

A riskier but maybe better alternative would be to do this netdev core code after the receive handler is invoked (if RX_HANDLER_ANOTHER is returned).

Quoted 7 lines

Review

Quoted 7 lines
Fixes 0c195567a8f6 ("netvsc: transparent VF management")
---
 drivers/net/hyperv/netvsc_drv.c | 6 ++++++
 1 file changed, 6 insertions(+)
(quoted diff) +0 -0

b/drivers/net/hyperv/netvsc_drv.c

Quoted 22 lines
Quoted 6 lines

index 06393b215102..bb0fc1869bde 100644

drivers/net/hyperv/netvsc_drv.c +0 -0
@@ -1999,9 +1999,15 @@ static rx_handler_result_t
 netvsc_vf_handle_frame(struct sk_buff **pskb)
 	struct net_device_context *ndev_ctx = netdev_priv(ndev);

drivers/net/hyperv/netvsc_drv.c static rx_handler_result_t · line 2010

Quoted 10 lines
drivers/net/hyperv/netvsc_drv.c +5 -0
 	struct netvsc_vf_pcpu_stats *pcpu_stats
 		 = this_cpu_ptr(ndev_ctx->vf_stats);
+	struct bpf_prog *xdp_prog;
 
 	skb->dev = ndev;
 
+	xdp_prog = rcu_dereference(ndev->xdp_prog);
+	if (xdp_prog &&
+	    do_xdp_generic(xdp_prog, skb) != XDP_PASS)
+		return RX_HANDLER_CONSUMED;

Looks fine overall.

The function do_xdp_generic() already checks NULL on xdp_prog, so we don't need to check it in our code.

Quoted 7 lines
Quoted 4 lines

int do_xdp_generic(struct bpf_prog *xdp_prog, struct sk_buff *skb)

{
        if (xdp_prog) {

The null check in the netvsc code was just an minor optimization to avoid unnecessary function call in fast path.

Thanks for the explanation.

#8 lore raw

RFC 2/2 netvsc: unshare skb in VF rx handler

In reply to Stephen Hemminger

The netvsc VF skb handler should make sure that skb is not shared. Similar logic already exists in bonding and team device drivers.

This does not happen in practice because the mlx device driver does not return shared skb's.

Fixes 0c195567a8f6 ("netvsc: transparent VF management")
---
 drivers/net/hyperv/netvsc_drv.c | 6 ++++++
 1 file changed, 6 insertions(+)
drivers/net/hyperv/netvsc_drv.c +6 -0
index bb0fc1869bde..eb666908b0fa 100644
@@ -2001,6 +2001,12 @@ static rx_handler_result_t netvsc_vf_handle_frame(struct sk_buff **pskb)
 		 = this_cpu_ptr(ndev_ctx->vf_stats);
 	struct bpf_prog *xdp_prog;
 
+	skb = skb_share_check(skb, GFP_ATOMIC);
+	if (unlikely(!skb))
+		return RX_HANDLER_CONSUMED;
+
+	*pskb = skb;
+
 	skb->dev = ndev;
 
 	xdp_prog = rcu_dereference(ndev->xdp_prog);
Signature · 1 line
2.20.1