Skip to contentSkip to conversation

udp_tunnel: convert mlx5 to the new udp_tunnel infrastructure

PATCH In review Jakub Kicinski 2 participants 5 messages Jul 28, 2020 – Jul 30, 2020

Subsystems (6)
lore
Share & tools
lore

Jakub Kicinski

author maintainer
#0 lore raw

v3 cover udp_tunnel: convert mlx5 to the new udp_tunnel infrastructure

This set converts mlx5 to the new infrastructure.

We need a small amount of special handling as mlx5 expects IANA VXLAN port to always be programmed into the device. ethtool will show such ports in a separate, type-less, fake table, at the end:

Tunnel information for eth0:
UDP port table 0:

    Size: 7
    Types: vxlan
    No entries
  UDP port table 1: 
    Size: 1
    Types: none (static entries)
    Entries (1):
        port 4789, vxlan

Saeed - this should apply on top of net-next, independently of the Intel patches, would you mind taking this in for testing and review? I'll post the netdevsim & test once Intel patches re-emerge.

v3: - add missing include.
v2: - don't disable the offload on reprs in patch #2.

Jakub Kicinski (2):
udp_tunnel: add the ability to hard-code IANA VXLAN
mlx5: convert to new udp_tunnel infrastructure

 Documentation/networking/ethtool-netlink.rst  |  3 +
 drivers/net/ethernet/mellanox/mlx5/core/en.h  |  2 -
 .../net/ethernet/mellanox/mlx5/core/en_main.c | 88 ++-----------------
 .../net/ethernet/mellanox/mlx5/core/en_rep.c  |  7 +-
 .../ethernet/mellanox/mlx5/core/lib/vxlan.c   | 87 +++++++++---------
 .../ethernet/mellanox/mlx5/core/lib/vxlan.h   |  6 +-
 include/net/udp_tunnel.h                      |  5 ++
 net/ethtool/tunnels.c                         | 69 +++++++++++++--
 net/ipv4/udp_tunnel_nic.c                     |  7 ++
 9 files changed, 131 insertions(+), 143 deletions(-)
Signature · 1 line
2.26.2
author maintainer
#1 lore raw

PATCH v3 1/2 udp_tunnel: add the ability to hard-code IANA VXLAN

mlx5 has the IANA VXLAN port (4789) hard coded by the device, instead of being added dynamically when tunnels are created.

To support this add a workaround flag to struct udp_tunnel_nic_info. Skipping updates for the port is fairly trivial, dumping the hard coded port via ethtool requires some code duplication. The port is not a part of any real table, we dump it in a special table which has no tunnel types supported and only one entry.

This is the last known workaround / hack needed to convert all drivers to the new infra.

---
 Documentation/networking/ethtool-netlink.rst |  3 +
 include/net/udp_tunnel.h                     |  5 ++
 net/ethtool/tunnels.c                        | 69 +++++++++++++++++---
 net/ipv4/udp_tunnel_nic.c                    |  7 ++
 4 files changed, 76 insertions(+), 8 deletions(-)
Documentation/networking/ethtool-netlink.rst +3 -0
index 7d75f1e32152..d53bcb31645a 100644
@@ -1263,6 +1263,9 @@ Gets information about the tunnel state NIC is aware of.
  | | | | ``ETHTOOL_A_TUNNEL_UDP_ENTRY_TYPE``   | u32    | tunnel type         |
  +-+-+-+---------------------------------------+--------+---------------------+
 
+For UDP tunnel table empty ``ETHTOOL_A_TUNNEL_UDP_TABLE_TYPES`` indicates that
+the table contains static entries, hard-coded by the NIC.
+
 Request translation
 ===================
 
include/net/udp_tunnel.h +5 -0
index dd20ce99740c..94bb7a882250 100644
@@ -193,6 +193,11 @@ enum udp_tunnel_nic_info_flags {
 	UDP_TUNNEL_NIC_INFO_OPEN_ONLY	= BIT(1),
 	/* Device supports only IPv4 tunnels */
 	UDP_TUNNEL_NIC_INFO_IPV4_ONLY	= BIT(2),
+	/* Device has hard-coded the IANA VXLAN port (4789) as VXLAN.
+	 * This port must not be counted towards n_entries of any table.
+	 * Driver will not receive any callback associated with port 4789.
+	 */
+	UDP_TUNNEL_NIC_INFO_STATIC_IANA_VXLAN	= BIT(3),
 };
 
 /**
net/ethtool/tunnels.c +61 -8

117 lines not on this page · raw message

net/ipv4/udp_tunnel_nic.c +7 -0
index f0dbd9905a53..69962165c0e8 100644
@@ -7,6 +7,7 @@
 #include <linux/types.h>
 #include <linux/workqueue.h>
 #include <net/udp_tunnel.h>
+#include <net/vxlan.h>
 
 enum udp_tunnel_nic_table_entry_flags {
 	UDP_TUNNEL_NIC_ENTRY_ADD	= BIT(0),
@@ -504,6 +505,12 @@ __udp_tunnel_nic_add_port(struct net_device *dev, struct udp_tunnel_info *ti)
 		return;
 	if (!netif_running(dev) && info->flags & UDP_TUNNEL_NIC_INFO_OPEN_ONLY)
 		return;
+	if (info->flags & UDP_TUNNEL_NIC_INFO_STATIC_IANA_VXLAN &&
+	    ti->port == htons(IANA_VXLAN_UDP_PORT)) {
+		if (ti->type != UDP_TUNNEL_TYPE_VXLAN)
+			netdev_warn(dev, "device assumes port 4789 will be used by vxlan tunnels\n");
+		return;
+	}
 
 	if (!udp_tunnel_nic_is_capable(dev, utn, ti))
 		return;
Signature · 1 line
2.26.2
author maintainer
#2 lore raw

PATCH v3 2/2 mlx5: convert to new udp_tunnel infrastructure

In reply to Jakub Kicinski

Allocate nic_info dynamically - n_entries is not constant.

Attach the tunnel offload info only to the uplink representor. We expect the "main" netdev to be unregistered in switchdev mode, and there to be only one uplink representor.

Drop the udp_tunnel_drop_rx_info() call, it was not there until
commit b3c2ed21c0bd ("net/mlx5e: Fix VXLAN configuration restore after function reload") so the device doesn't need it, and core should handle reloads and
reset just fine.

v2:
- don't drop the ndos on reprs, and register info on uplink repr.

---
 drivers/net/ethernet/mellanox/mlx5/core/en.h  |  2 -
 .../net/ethernet/mellanox/mlx5/core/en_main.c | 88 ++-----------------
 .../net/ethernet/mellanox/mlx5/core/en_rep.c  |  7 +-
 .../ethernet/mellanox/mlx5/core/lib/vxlan.c   | 87 +++++++++---------
 .../ethernet/mellanox/mlx5/core/lib/vxlan.h   |  6 +-
 5 files changed, 55 insertions(+), 135 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en.h +0 -2
index c44669102626..c3e399f3e834 100644
@@ -1113,8 +1113,6 @@ void mlx5e_build_rss_params(struct mlx5e_rss_params *rss_params,
 void mlx5e_rx_dim_work(struct work_struct *work);
 void mlx5e_tx_dim_work(struct work_struct *work);
 
-void mlx5e_add_vxlan_port(struct net_device *netdev, struct udp_tunnel_info *ti);
-void mlx5e_del_vxlan_port(struct net_device *netdev, struct udp_tunnel_info *ti);
 netdev_features_t mlx5e_features_check(struct sk_buff *skb,
 				       struct net_device *netdev,
 				       netdev_features_t features);
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +5 -83

118 lines not on this page · raw message

drivers/net/ethernet/mellanox/mlx5/core/en_rep.c +5 -2
index c300729fb498..e9b86bc39716 100644
@@ -37,6 +37,7 @@
 #include <net/act_api.h>
 #include <net/devlink.h>
 #include <net/ipv6_stubs.h>
+#include <net/udp_tunnel.h>
 
 #include "eswitch.h"
 #include "esw/chains.h"
@@ -47,6 +48,7 @@
 #include "en/rep/neigh.h"
 #include "fs_core.h"
 #include "lib/mlx5.h"
+#include "lib/vxlan.h"
 #define CREATE_TRACE_POINTS
 #include "diag/en_rep_tracepoint.h"
 
@@ -633,8 +635,8 @@ static const struct net_device_ops mlx5e_netdev_ops_uplink_rep = {
 	.ndo_has_offload_stats	 = mlx5e_rep_has_offload_stats,
 	.ndo_get_offload_stats	 = mlx5e_rep_get_offload_stats,
 	.ndo_change_mtu          = mlx5e_uplink_rep_change_mtu,
-	.ndo_udp_tunnel_add      = mlx5e_add_vxlan_port,
-	.ndo_udp_tunnel_del      = mlx5e_del_vxlan_port,
+	.ndo_udp_tunnel_add      = udp_tunnel_nic_add_port,
+	.ndo_udp_tunnel_del      = udp_tunnel_nic_del_port,
 	.ndo_features_check      = mlx5e_features_check,
 	.ndo_set_vf_mac          = mlx5e_set_vf_mac,
 	.ndo_set_vf_rate         = mlx5e_set_vf_rate,
@@ -705,6 +707,7 @@ static void mlx5e_build_rep_netdev(struct net_device *netdev)
 		/* we want a persistent mac for the uplink rep */
 		mlx5_query_mac_address(mdev, netdev->dev_addr);
 		netdev->ethtool_ops = &mlx5e_uplink_rep_ethtool_ops;
+		mlx5_vxlan_set_netdev_info(mdev->vxlan, netdev);
 		mlx5e_dcbnl_build_rep_netdev(netdev);
 	} else {
 		netdev->netdev_ops = &mlx5e_netdev_ops_rep;
drivers/net/ethernet/mellanox/mlx5/core/lib/vxlan.c +43 -44

144 lines not on this page · raw message

drivers/net/ethernet/mellanox/mlx5/core/lib/vxlan.h +2 -4
index 6d599f4a8acd..6b7eeb94ec35 100644
@@ -48,15 +48,13 @@ static inline bool mlx5_vxlan_allowed(struct mlx5_vxlan *vxlan)
 #if IS_ENABLED(CONFIG_VXLAN)
 struct mlx5_vxlan *mlx5_vxlan_create(struct mlx5_core_dev *mdev);
 void mlx5_vxlan_destroy(struct mlx5_vxlan *vxlan);
-int mlx5_vxlan_add_port(struct mlx5_vxlan *vxlan, u16 port);
-int mlx5_vxlan_del_port(struct mlx5_vxlan *vxlan, u16 port);
+void mlx5_vxlan_set_netdev_info(struct mlx5_vxlan *vxlan, struct net_device *netdev);
 bool mlx5_vxlan_lookup_port(struct mlx5_vxlan *vxlan, u16 port);
 #else
 static inline struct mlx5_vxlan*
 mlx5_vxlan_create(struct mlx5_core_dev *mdev) { return ERR_PTR(-EOPNOTSUPP); }
 static inline void mlx5_vxlan_destroy(struct mlx5_vxlan *vxlan) { return; }
-static inline int mlx5_vxlan_add_port(struct mlx5_vxlan *vxlan, u16 port) { return -EOPNOTSUPP; }
-static inline int mlx5_vxlan_del_port(struct mlx5_vxlan *vxlan, u16 port) { return -EOPNOTSUPP; }
+static inline void mlx5_vxlan_set_netdev_info(struct mlx5_vxlan *vxlan, struct net_device *netdev) { return; }
 static inline bool mlx5_vxlan_lookup_port(struct mlx5_vxlan *vxlan, u16 port) { return false; }
 #endif
 
Signature · 1 line
2.26.2
#3 lore raw

Re: [PATCH net-next v3 2/2] mlx5: convert to new udp_tunnel infrastructure

Quoted 6 lines · Jakub Kicinski

Allocate nic_info dynamically - n_entries is not constant.

Attach the tunnel offload info only to the uplink representor. We expect the "main" netdev to be unregistered in switchdev mode, and there to be only one uplink representor.

Quoted 6 lines

Drop the udp_tunnel_drop_rx_info() call, it was not there until commit b3c2ed21c0bd ("net/mlx5e: Fix VXLAN configuration restore after function reload")
so the device doesn't need it, and core should handle reloads and reset just fine.

Oouf :), I posted comments on v2, please have a look.

#4 lore raw
In reply to Jakub Kicinski
Quoted 22 lines · Jakub Kicinski

This set converts mlx5 to the new infrastructure.

We need a small amount of special handling as mlx5 expects IANA VXLAN port to always be programmed into the device. ethtool will show such ports in a separate, type-less, fake table, at the end:

Tunnel information for eth0:
UDP port table 0:

    Size: 7
    Types: vxlan
    No entries
  UDP port table 1: 
    Size: 1
    Types: none (static entries)
    Entries (1):
        port 4789, vxlan

Saeed - this should apply on top of net-next, independently of the Intel patches, would you mind taking this in for testing and review? I'll post the netdevsim & test once Intel patches re-emerge.

Quoted 7 lines

v3: - add missing include.
v2: - don't disable the offload on reprs in patch #2.

Jakub Kicinski (2):
udp_tunnel: add the ability to hard-code IANA VXLAN
mlx5: convert to new udp_tunnel infrastructure

The two patches are applied to net-next-mlx5.

mlx5 patch was slightly modified to move udp_tunnel netdev specific bits from mlx5/core/lib/vxlan.c to mlx5/core/en_main.c. Jakub already Acknowledge this change.

Thanks,
Saeed.