mlx4/en_netdev: allow offloading VXLAN over VLAN
Subsystems (4)
cover mlx4/en_netdev: allow offloading VXLAN over VLAN
When VXLAN offload is enabled on ConnectX-3 Pro devices, the NIC can segment correctly also VXLAN packet with a VLAN tag: this series ensures that a VLAN created on top of a mlx4_en NIC inherits the tunnel offload capabilities.
Davide Caratti (2):
mlx4/en_netdev: update vlan features with tunnel offloads mlx4/en_netdev: call notifiers when hw_enc_features change
.../net/ethernet/mellanox/mlx4/en_netdev.c | 60 ++++++++++++------- 1 file changed, 37 insertions(+), 23 deletions(-)
Signature · 1 line
2.20.1
PATCH 1/2 mlx4/en_netdev: update vlan features with tunnel offloads
ConnectX-3 Pro can offload transmission of VLAN packets with VXLAN inside:
enable tunnel offloads in dev->vlan_features, like it's done with other NIC drivers (e.g. be2net and ixgbe).
--- .../net/ethernet/mellanox/mlx4/en_netdev.c | 21 ++++++++++--------- 1 file changed, 11 insertions(+), 10 deletions(-)
drivers/net/ethernet/mellanox/mlx4/en_netdev.c +11 -10
@@ -3415,6 +3415,17 @@ int mlx4_en_init_netdev(struct mlx4_en_dev *mdev, int port,if (mdev->LSO_support) dev->hw_features |=NETIF_F_TSO |NETIF_F_TSO6 ; +if (mdev->dev->caps.tunnel_offload_mode == +MLX4_TUNNEL_OFFLOAD_MODE_VXLAN ) { + dev->hw_features |=NETIF_F_GSO_UDP_TUNNEL | +NETIF_F_GSO_UDP_TUNNEL_CSUM | +NETIF_F_GSO_PARTIAL ; + dev->features |=NETIF_F_GSO_UDP_TUNNEL | +NETIF_F_GSO_UDP_TUNNEL_CSUM | +NETIF_F_GSO_PARTIAL ; + dev->gso_partial_features =NETIF_F_GSO_UDP_TUNNEL_CSUM ; + } + dev->vlan_features = dev->hw_features; dev->hw_features |=NETIF_F_RXCSUM |NETIF_F_RXHASH ;
@@ -3483,16 +3494,6 @@ int mlx4_en_init_netdev(struct mlx4_en_dev *mdev, int port, priv->rss_hash_fn =ETH_RSS_HASH_TOP ; } -if (mdev->dev->caps.tunnel_offload_mode ==MLX4_TUNNEL_OFFLOAD_MODE_VXLAN ) { - dev->hw_features |=NETIF_F_GSO_UDP_TUNNEL | -NETIF_F_GSO_UDP_TUNNEL_CSUM | -NETIF_F_GSO_PARTIAL ; - dev->features |=NETIF_F_GSO_UDP_TUNNEL | -NETIF_F_GSO_UDP_TUNNEL_CSUM | -NETIF_F_GSO_PARTIAL ; - dev->gso_partial_features =NETIF_F_GSO_UDP_TUNNEL_CSUM ; - } -/* MTU range: 68 - hw-specific max */ dev->min_mtu =ETH_MIN_MTU ; dev->max_mtu = priv->max_mtu;
Signature · 1 line
2.20.1
Re: [PATCH net-next 1/2] mlx4/en_netdev: update vlan features with tunnel offloads
Quoted 7 lines · Davide Caratti
ConnectX-3 Pro can offload transmission of VLAN packets with VXLAN inside:
enable tunnel offloads indev->vlan_features, like it's done with other
NIC drivers (e.g. be2net and ixgbe).Signed-off-by Davide Caratti <[email protected]>
LGTM
PATCH 2/2 mlx4/en_netdev: call notifiers when hw_enc_features change
In reply to Davide Carattiensure to call netdev_features_change() when the driver flips its hw_enc_features bits.
--- .../net/ethernet/mellanox/mlx4/en_netdev.c | 39 ++++++++++++------- 1 file changed, 26 insertions(+), 13 deletions(-)
drivers/net/ethernet/mellanox/mlx4/en_netdev.c +26 -13
@@ -2628,6 +2628,30 @@ static int mlx4_en_get_phys_port_id(struct net_device *dev,return 0 ; } +#define MLX4_GSO_PARTIAL_FEATURES (NETIF_F_IP_CSUM | NETIF_F_IPV6_CSUM | \ + NETIF_F_RXCSUM | \ + NETIF_F_TSO | NETIF_F_TSO6 | \ + NETIF_F_GSO_UDP_TUNNEL | \ + NETIF_F_GSO_UDP_TUNNEL_CSUM | \ + NETIF_F_GSO_PARTIAL) + +static void mlx4_set_vxlan_offloads (struct net_device *dev,bool enable) +{ +netdev_features_t hw_enc_features; + +rtnl_lock (); + hw_enc_features = dev->hw_enc_features; +if (enable) + dev->hw_enc_features |=MLX4_GSO_PARTIAL_FEATURES ; +else + dev->hw_enc_features &= ~MLX4_GSO_PARTIAL_FEATURES ; + +if (hw_enc_features ^ dev->hw_enc_features) +netdev_features_change (dev); + +rtnl_unlock (); +} +static void mlx4_en_add_vxlan_offloads (struct work_struct *work) {int ret;
@@ -2647,12 +2671,7 @@ static void mlx4_en_add_vxlan_offloads(struct work_struct *work) }/* set offloads */ - priv->dev->hw_enc_features |=NETIF_F_IP_CSUM |NETIF_F_IPV6_CSUM | -NETIF_F_RXCSUM | -NETIF_F_TSO |NETIF_F_TSO6 | -NETIF_F_GSO_UDP_TUNNEL | -NETIF_F_GSO_UDP_TUNNEL_CSUM | -NETIF_F_GSO_PARTIAL ; +mlx4_set_vxlan_offloads (priv->dev, true); }static void mlx4_en_del_vxlan_offloads (struct work_struct *work)
@@ -2661,13 +2680,7 @@ static void mlx4_en_del_vxlan_offloads(struct work_struct *work) struct mlx4_en_priv *priv = container_of(work, struct mlx4_en_priv, vxlan_del_task);/* unset offloads */ - priv->dev->hw_enc_features &= ~(NETIF_F_IP_CSUM |NETIF_F_IPV6_CSUM | -NETIF_F_RXCSUM | -NETIF_F_TSO |NETIF_F_TSO6 | -NETIF_F_GSO_UDP_TUNNEL | -NETIF_F_GSO_UDP_TUNNEL_CSUM | -NETIF_F_GSO_PARTIAL ); - +mlx4_set_vxlan_offloads (priv->dev, false); ret =mlx4_SET_PORT_VXLAN (priv->mdev->dev, priv->port,VXLAN_STEER_BY_OUTER_MAC ,0 );if (ret)
Signature · 1 line
2.20.1
Re: [PATCH net-next 2/2] mlx4/en_netdev: call notifiers when hw_enc_features change
Quoted 4 lines · Davide Caratti
ensure to call
netdev_features_change()when the driver flips itshw_enc_featuresbits.Signed-off-by Davide Caratti <[email protected]>
The patch is correct, but can you explain how did you come to this ? did you encounter any issue with the current code ?
I am asking just because i think the whole dynamic changing of dev-
Quoted 1 line
hw_enc_featuresis redundant since mlx4 has thefeatutres_check
callback.
Quoted 78 lines
--- .../net/ethernet/mellanox/mlx4/en_netdev.c | 39 ++++++++++++----- -- 1 file changed, 26 insertions(+), 13 deletions(-)drivers/net/ethernet/mellanox/mlx4/en_netdev.c +24
-0@@ -2628,6 +2628,30 @@ static int mlx4_en_get_phys_port_id(structnet_device *dev, return 0; } +#define MLX4_GSO_PARTIAL_FEATURES (NETIF_F_IP_CSUM | NETIF_F_IPV6_CSUM | \ + NETIF_F_RXCSUM | \ + NETIF_F_TSO | NETIF_F_TSO6 | \ + NETIF_F_GSO_UDP_TUNNEL | \ + NETIF_F_GSO_UDP_TUNNEL_CSUM | \ + NETIF_F_GSO_PARTIAL) + +static void mlx4_set_vxlan_offloads (struct net_device *dev,bool enable) +{ +netdev_features_t hw_enc_features; + +rtnl_lock (); + hw_enc_features = dev->hw_enc_features; +if (enable) + dev->hw_enc_features |=MLX4_GSO_PARTIAL_FEATURES ; +else + dev->hw_enc_features &= ~MLX4_GSO_PARTIAL_FEATURES ; + +if (hw_enc_features ^ dev->hw_enc_features) +netdev_features_change (dev); + +rtnl_unlock (); +} +static void
mlx4_en_add_vxlan_offloads(struct work_struct *work){drivers/net/ethernet/mellanox/mlx4/en_netdev.c +0
-1int ret;@@ -2647,12 +2671,7 @@ static void mlx4_en_add_vxlan_offloads(struct work_struct *work) }/* set offloads */ - priv->dev->hw_enc_features |= NETIF_F_IP_CSUM |
NETIF_F_IPV6_CSUM|drivers/net/ethernet/mellanox/mlx4/en_netdev.c +1
-6-NETIF_F_RXCSUM | -NETIF_F_TSO |NETIF_F_TSO6 | -NETIF_F_GSO_UDP_TUNNEL | -NETIF_F_GSO_UDP_TUNNEL_CSUM | -NETIF_F_GSO_PARTIAL ; +mlx4_set_vxlan_offloads (priv->dev, true); }static void mlx4_en_del_vxlan_offloads (struct work_struct *work)@@ -2661,13 +2680,7 @@ static void mlx4_en_del_vxlan_offloads(struct work_struct *work) struct mlx4_en_priv *priv = container_of(work, struct mlx4_en_priv, vxlan_del_task);/* unset offloads */ - priv->dev->hw_enc_features &= ~(NETIF_F_IP_CSUM |
NETIF_F_IPV6_CSUM|drivers/net/ethernet/mellanox/mlx4/en_netdev.c +1
-6-NETIF_F_RXCSUM | -NETIF_F_TSO |NETIF_F_TSO6 | -NETIF_F_GSO_UDP_TUNNEL | -NETIF_F_GSO_UDP_TUNNEL_CSUM | - NETIF_F_GSO_PARTIAL); - +mlx4_set_vxlan_offloads (priv->dev, false); ret =mlx4_SET_PORT_VXLAN (priv->mdev->dev, priv->port,VXLAN_STEER_BY_OUTER_MAC ,0 );if (ret)
Re: [PATCH net-next 2/2] mlx4/en_netdev: call notifiers when hw_enc_features change
Quoted 7 lines · Saeed Mahameed
Quoted 4 lines · Davide Caratti
ensure to call
netdev_features_change()when the driver flips itshw_enc_featuresbits.Signed-off-by Davide Caratti <[email protected]>The patch is correct,
hello Saeed, and thanks for looking at this!
Quoted 6 lines
but can you explain how did you come to this ? did you encounter any issue with the current code ?
I am asking just because i think the whole dynamic changing of dev-
Quoted 1 line
hw_enc_featuresis redundant since mlx4 has thefeatutres_checkcallback.
we need it to ensure that vlan_transfer_features() updates
the (new) value of hw_enc_features in the overlying vlan: otherwise, segmentation will happen anyway when skb passes from vxlan to vlan, if the vxlan is added after the vlan device has been created (see: 7dad9937e064 ("net: vlan: add support for tunnel offload") ).
thanks!
Signature · 1 line
davide
Re: [PATCH net-next 2/2] mlx4/en_netdev: call notifiers when hw_enc_features change
Quoted 18 lines · Davide Caratti
Quoted 7 lines · Saeed Mahameed
Quoted 4 lines · Davide Caratti
ensure to call
netdev_features_change()when the driver flips itshw_enc_featuresbits.Signed-off-by Davide Caratti <[email protected]>The patch is correct,
hello Saeed, and thanks for looking at this!
Quoted 6 lines
but can you explain how did you come to this ? did you encounter any issue with the current code ?
I am asking just because i think the whole dynamic changing of dev-
Quoted 1 line
hw_enc_featuresis redundant since mlx4 has thefeatutres_checkcallback.
Quoted 8 lines
we need it to ensure that
vlan_transfer_features()updates the (new) value ofhw_enc_featuresin the overlying vlan: otherwise, segmentation will happen anyway when skb passes from vxlan to vlan, if the
vxlan is added after the vlan device has been created (see:7dad9937e064
("net: vlan: add support for tunnel offload") ).
but in previous patch you made sure that the vlan always sees the correct hw_enc_features on driver load, we don't need to have this dynamic update mechanism, features_check ndo should take care of protocols we don't support.
Quoted 1 line
thanks!
Re: [PATCH net-next 2/2] mlx4/en_netdev: call notifiers when hw_enc_features change
Quoted 28 lines · Saeed Mahameed
Quoted 18 lines · Davide Caratti
Quoted 7 lines · Saeed Mahameed
Quoted 4 lines · Davide Caratti
ensure to call
netdev_features_change()when the driver flips itshw_enc_featuresbits.Signed-off-by Davide Caratti <[email protected]>The patch is correct,
hello Saeed, and thanks for looking at this!
Quoted 6 lines
but can you explain how did you come to this ? did you encounter any issue with the current code ?
I am asking just because i think the whole dynamic changing of dev-
Quoted 1 line
hw_enc_featuresis redundant since mlx4 has thefeatutres_checkcallback.
Quoted 8 lines
we need it to ensure that
vlan_transfer_features()updates the (new) value ofhw_enc_featuresin the overlying vlan: otherwise, segmentation will happen anyway when skb passes from vxlan to vlan, if the
vxlan is added after the vlan device has been created (see:7dad9937e064
("net: vlan: add support for tunnel offload") ).
Quoted 3 lines
but in previous patch you made sure that the vlan always sees the correct
hw_enc_featureson driver load, we don't need to have this dynamic update mechanism,
ok, but the mlx4 driver flips the value of hw_enc_features when VXLAN tunnels are added or removed. So, assume eth0 is a Cx3-pro, and I do:
# ip link add name vlan5 link eth0 type vlan id 5 # ip link add dev vxlan6 type vxlan id 6 [...] dev vlan5
the value of dev->hw_enc_features is 0 for vlan5, and as a consequence VXLAN over VLAN traffic becomes segmented by the VLAN, even if eth0, at the end of this sequence, has the "right" features bits.
Quoted 2 lines
features_checkndo should take care of
protocols we don't support.
I just had a look at mlx4_en_features_check(), I see it checks if the packet is tunneled in VXLAN and the destination port matches the configured value of priv->vxlan_port (when that value is not zero). Now:
Quoted 3 lines · Saeed Mahameed
I am asking just because i think the whole dynamic changing of dev->
hw_enc_featuresis redundant since mlx4 has thefeatutres_checkcallback.
I read your initial proposal again. Would it be correct if I just use patch 1/2, where I add an assignment of
dev->hw_enc_features = NETIF_F_IP_CSUM | NETIF_F_IPV6_CSUM | \
NETIF_F_RXCSUM | \
NETIF_F_TSO | NETIF_F_TSO6 | \
NETIF_F_GSO_UDP_TUNNEL | \
NETIF_F_GSO_UDP_TUNNEL_CSUM | \
NETIF_F_GSO_PARTIAL;in mlx4_en_init_netdev(), and then remove the code that flips dev->hw_enc_features in mlx4_en_add_vxlan_offloads() and mlx4_en_del_vxlan_offloads() ?
thanks,
--
davide
Re: [PATCH net-next 2/2] mlx4/en_netdev: call notifiers when hw_enc_features change
Quoted 41 lines · Davide Caratti
Quoted 33 lines · Saeed Mahameed
Quoted 21 lines · Davide Caratti
Quoted 8 lines · Saeed Mahameed
Quoted 5 lines · Davide Caratti
ensure to call
netdev_features_change()when the driver flips itshw_enc_featuresbits.Signed-off-by Davide Caratti <[email protected]>The patch is correct,
hello Saeed, and thanks for looking at this!
Quoted 8 lines
but can you explain how did you come to this ? did you encounter any issue with the current code ?
I am asking just because i think the whole dynamic changing of dev-
Quoted 2 lines
hw_enc_featuresis redundant since mlx4 has thefeatutres_checkcallback.
Quoted 10 lines
we need it to ensure that
vlan_transfer_features()updates the (new) value ofhw_enc_featuresin the overlying vlan:
otherwise,
segmentation will happen anyway when skb passes from vxlan to vlan,
if the
vxlan is added after the vlan device has been created (see:7dad9937e064
("net: vlan: add support for tunnel offload") ).Quoted 3 lines
but in previous patch you made sure that the vlan always sees the correct
hw_enc_featureson driver load, we don't need to have this dynamic update mechanism,ok, but the mlx4 driver flips the value of
hw_enc_featureswhen VXLAN tunnels are added or removed. So, assume eth0 is a Cx3-pro, and I do:
Quoted 9 lines
# ip link add name vlan5 link eth0 type vlan id 5 # ip link add dev vxlan6 type vxlan id 6 [...] dev vlan5the value of
dev->hw_enc_featuresis 0 for vlan5, and as a consequence
VXLAN over VLAN traffic becomes segmented by the VLAN, even if eth0, at
the end of this sequence, has the "right" features bits.
your patch handled this issue already, no need for flipping and updating features if features check ndo will cover the cases we don't support.
Quoted 24 lines
Quoted 2 lines
features_checkndo should take care of
protocols we don't support.I just had a look at
mlx4_en_features_check(), I see it checks if the packet is tunneled in VXLAN and the destination port matches the configured value ofpriv->vxlan_port(when that value is not zero). Now:Quoted 4 lines · Saeed Mahameed
I am asking just because i think the whole dynamic changing of dev->
hw_enc_featuresis redundant since mlx4 has thefeatutres_check
callback.I read your initial proposal again. Would it be correct if I just use patch 1/2, where I add an assignment of
dev->hw_enc_features = NETIF_F_IP_CSUM | NETIF_F_IPV6_CSUM | \ NETIF_F_RXCSUM | \ NETIF_F_TSO | NETIF_F_TSO6 | \ NETIF_F_GSO_UDP_TUNNEL | \ NETIF_F_GSO_UDP_TUNNEL_CSUM | \ NETIF_F_GSO_PARTIAL;
Quoted 4 lines
in
mlx4_en_init_netdev(), and then remove the code that flipsdev->hw_enc_featuresinmlx4_en_add_vxlan_offloads()andmlx4_en_del_vxlan_offloads()?
yes, this is exactly what I meant.
Thanks,
Saeed.