udp_tunnel: add NIC RX port offload infrastructure
Subsystems (13)
Jakub Kicinski
maintainerv4 cover udp_tunnel: add NIC RX port offload infrastructure
Kernel has a facility to notify drivers about the UDP tunnel ports so that devices can recognize tunneled packets. This is important mostly for RX - devices which don't support CHECKSUM_COMPLETE can report checksums of inner packets, and compute RSS over inner headers. Some drivers also match the UDP tunnel ports also for TX, although doing so may lead to false positives and negatives.
Unfortunately the user experience when trying to take adavantage of these facilities is suboptimal. First of all there is no way for users to check which ports are offloaded. Many drivers resort to printing messages to aid debugging, other use debugfs. Even worse the availability of the RX features (NETIF_F_RX_UDP_TUNNEL_PORT) is established purely on the basis of the driver having the ndos installed. For most drivers, however, the ability to perform offloads is contingent on device capabilities (driver support multiple device and firmware versions). Unless driver resorts to hackish clearing of features set incorrectly by the core - users are left guessing whether their device really supports UDP tunnel port offload or not.
There is currently no way to indicate or configure whether RX features include just the checksum offload or checksum and using inner headers for RSS. Many drivers default to not using inner headers for RSS because most implementations populate the source port with entropy from the inner headers. This, however, is not always the case, for example certain switches are only able to use a fixed source port during encapsulation.
We have also seen many driver authors get the intricacies of UDP tunnel port offloads wrong. Most commonly the drivers forget to perform reference counting, or take sleeping locks in the callbacks.
This work tries to improve the situation by pulling the UDP tunnel port table maintenance out of the drivers. It turns out that almost all drivers maintain a fixed size table of ports (in most cases one per tunnel type), so we can take care of all the refcounting in the core, and let the driver specify if they need to sleep in the callbacks or not. The new common implementation will also support replacing ports - when a port is removed from a full table it will try to find a previously missing port to take its place.
This patch only implements the core functionality along with a few drivers I was hoping to test manually [1] along with a test based on a netdevsim implementation. Following patches will convert all the drivers. Once that's complete we can remove the ndos, and rely directly on the new infrastrucutre.
Then after RSS (RXFH) is converted to netlink we can add the ability to configure the use of inner RSS headers for UDP tunnels.
[1] Unfortunately I wasn't able to, turns out 2 of the devices I had access to were older generation or had old FW, and they did not actually support UDP tunnel port notifications (see the second paragraph). The thrid device appears to program the UDP ports correctly but it generates bad UDP checksums with or without these patches. Long story short - I'd appreciate reviews and testing here..
v4:
- better build fix (hopefully this one does it..)
v3:
- fix build issue;
- improve bnxt changes.
Jakub Kicinski (10):
debugfs: make sure we can remove u32_array files cleanly
udp_tunnel: re-number the offload tunnel types
udp_tunnel: add central NIC RX port offload infrastructure
ethtool: add tunnel info interface
netdevsim: add UDP tunnel port offload support
selftests: net: add a test for UDP tunnel info infra
ixgbe: don't clear UDP tunnel ports when RXCSUM is disabled
ixgbe: convert to new udp_tunnel_nic infra
bnxt: convert to new udp_tunnel_nic infra
mlx4: convert to new udp_tunnel_nic infra
Documentation/filesystems/debugfs.rst | 12 +-
Documentation/networking/ethtool-netlink.rst | 33 +
drivers/net/ethernet/broadcom/bnxt/bnxt.c | 141 +--
drivers/net/ethernet/broadcom/bnxt/bnxt.h | 12 +-
drivers/net/ethernet/intel/ixgbe/ixgbe.h | 3 -
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 200 +---
.../net/ethernet/mellanox/mlx4/en_netdev.c | 107 +--
drivers/net/ethernet/mellanox/mlx4/mlx4_en.h | 2 -
drivers/net/geneve.c | 6 +-
drivers/net/netdevsim/Makefile | 2 +-
drivers/net/netdevsim/dev.c | 1 +
drivers/net/netdevsim/netdev.c | 12 +-
drivers/net/netdevsim/netdevsim.h | 19 +
drivers/net/netdevsim/udp_tunnels.c | 192 ++++
drivers/net/vxlan.c | 6 +-
fs/debugfs/file.c | 27 +-
include/linux/debugfs.h | 12 +-
include/linux/netdevice.h | 8 +
include/net/udp_tunnel.h | 164 +++-
include/uapi/linux/ethtool.h | 2 +
include/uapi/linux/ethtool_netlink.h | 55 ++
mm/cma.h | 3 +
mm/cma_debug.c | 7 +-
net/ethtool/Makefile | 3 +-
net/ethtool/common.c | 9 +
net/ethtool/common.h | 1 +
net/ethtool/netlink.c | 12 +
net/ethtool/netlink.h | 4 +
net/ethtool/strset.c | 5 +
net/ethtool/tunnels.c | 259 +++++
net/ipv4/Makefile | 3 +-
net/ipv4/{udp_tunnel.c => udp_tunnel_core.c} | 0
net/ipv4/udp_tunnel_nic.c | 890 ++++++++++++++++++
net/ipv4/udp_tunnel_stub.c | 7 +
.../drivers/net/netdevsim/udp_tunnel_nic.sh | 786 ++++++++++++++++
35 files changed, 2608 insertions(+), 397 deletions(-)
create mode 100644 drivers/net/netdevsim/udp_tunnels.c
create mode 100644 net/ethtool/tunnels.c
rename net/ipv4/{udp_tunnel.c => udp_tunnel_core.c} (100%)
create mode 100644 net/ipv4/udp_tunnel_nic.c
create mode 100644 net/ipv4/udp_tunnel_stub.c
create mode 100644 tools/testing/selftests/drivers/net/netdevsim/udp_tunnel_nic.shSignature · 1 line
2.26.2
Jakub Kicinski
maintainerPATCH v4 2/10 udp_tunnel: re-number the offload tunnel types
Make it possible to use tunnel types as flags more easily. There doesn't appear to be any user using the type as an array index, so this should make no difference.
--- include/net/udp_tunnel.h | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
include/net/udp_tunnel.h +3 -3
@@ -106,9 +106,9 @@ void setup_udp_tunnel_sock(struct net *net, struct socket *sock,* call this function to perform Tx offloads on outgoing traffic. */ enum udp_parsable_tunnel_type { -UDP_TUNNEL_TYPE_VXLAN ,/* RFC 7348 */ -UDP_TUNNEL_TYPE_GENEVE ,/* draft-ietf-nvo3-geneve */ -UDP_TUNNEL_TYPE_VXLAN_GPE ,/* draft-ietf-nvo3-vxlan-gpe */ +UDP_TUNNEL_TYPE_VXLAN =BIT (0 ),/* RFC 7348 */ +UDP_TUNNEL_TYPE_GENEVE =BIT (1 ),/* draft-ietf-nvo3-geneve */ +UDP_TUNNEL_TYPE_VXLAN_GPE =BIT (2 ),/* draft-ietf-nvo3-vxlan-gpe */ };struct udp_tunnel_info {
Signature · 1 line
2.26.2
Jakub Kicinski
maintainerPATCH v4 1/10 debugfs: make sure we can remove u32_array files cleanly
In reply to Jakub Kicinskidebugfs_create_u32_array() allocates a small structure to wrap the data and size information about the array. If users ever try to remove the file this leads to a leak since nothing ever frees this wrapper.
That said there are no upstream users of debugfs_create_u32_array() that'd remove a u32 array file (we only have one u32 array user in CMA), so there is no real bug here.
Make callers pass a wrapper they allocated. This way the lifetime management of the wrapper is on the caller, and we can avoid the potential leak in debugfs.
--- Documentation/filesystems/debugfs.rst | 12 ++++++++---- fs/debugfs/file.c | 27 +++++++-------------------- include/linux/debugfs.h | 12 +++++++++--- mm/cma.h | 3 +++ mm/cma_debug.c | 7 ++++--- 5 files changed, 31 insertions(+), 30 deletions(-)
Documentation/filesystems/debugfs.rst +8 -4
@@ -185,13 +185,17 @@ byte offsets over a base for the register block. If you want to dump an u32 array in debugfs, you can create file with:: + struct debugfs_u32_array { + u32 *array; + u32 n_elements; + }; + void debugfs_create_u32_array(const char *name, umode_t mode, struct dentry *parent, - u32 *array, u32 elements); + struct debugfs_u32_array *array); -The "array" argument provides data, and the "elements" argument is -the number of elements in the array. Note: Once array is created its -size can not be changed. +The "array" argument wraps a pointer to the array's data and the number +of its elements. Note: Once array is created its size can not be changed. There is a helper function to create device related seq_file::
fs/debugfs/file.c +7 -20
@@ -918,11 +918,6 @@ struct dentry *debugfs_create_blob(const char *name, umode_t mode, }EXPORT_SYMBOL_GPL (debugfs_create_blob); -struct array_data { -void *array; -u32 elements; -}; -static size_t u32_format_array (char *buf,size_t bufsize,u32 *array,int array_size) {
@@ -943,8 +938,8 @@ static size_t u32_format_array(char *buf, size_t bufsize,static int u32_array_open (struct inode *inode,struct file *file) { -struct array_data *data = inode->i_private; -int size, elements = data->elements; +struct debugfs_u32_array *data = inode->i_private; +int size, elements = data->n_elements;char *buf;/*
@@ -959,7 +954,7 @@ static int u32_array_open(struct inode *inode, struct file *file) buf[size] =0 ; file->private_data = buf; -u32_format_array (buf, size, data->array, data->elements); +u32_format_array (buf, size, data->array, data->n_elements);return nonseekable_open (inode, file); }
@@ -996,8 +991,7 @@ static const struct file_operations u32_array_fops = {* @parent: a pointer to the parent dentry for this file. This should be a * directory dentry if set. If this parameter is %NULL, then the * file will be created in the root of the debugfs filesystem. -* @array: u32 array that provides data. -* @elements: total number of elements in the array. +* @array: wrapper struct containing data pointer and size of the array. * * This function creates a file in debugfs with the given name that exports * @array as data. If the @mode variable is so set it can be read from.
@@ -1005,17 +999,10 @@ static const struct file_operations u32_array_fops = {* Once array is created its size can not be changed. */ void debugfs_create_u32_array (const char *name,umode_t mode, -struct dentry *parent,u32 *array,u32 elements) +struct dentry *parent, +struct debugfs_u32_array *array) { -struct array_data *data =kmalloc (sizeof (*data),GFP_KERNEL ); - -if (data ==NULL ) -return ; - - data->array = array; - data->elements = elements; - -debugfs_create_file_unsafe (name, mode, parent, data, &u32_array_fops); +debugfs_create_file_unsafe (name, mode, parent, array, &u32_array_fops); }EXPORT_SYMBOL_GPL (debugfs_create_u32_array);
include/linux/debugfs.h +9 -3
@@ -38,6 +38,11 @@ struct debugfs_regset32 {struct device *dev;/* Optional device for Runtime PM */ }; +struct debugfs_u32_array { +u32 *array; +u32 n_elements; +}; +extern struct dentry *arch_debugfs_dir;#define DEFINE_DEBUGFS_ATTRIBUTE (__fops, __get, __set, __fmt) \
@@ -136,7 +141,8 @@ void debugfs_print_regs32(struct seq_file *s, const struct debugfs_reg32 *regs, int nregs, void __iomem *base, char *prefix);void debugfs_create_u32_array (const char *name,umode_t mode, -struct dentry *parent,u32 *array,u32 elements); +struct dentry *parent, +struct debugfs_u32_array *array); struct dentry *debugfs_create_devm_seqfile(struct device *dev, const char *name, struct dentry *parent,
@@ -316,8 +322,8 @@ static inline bool debugfs_initialized(void) }static inline void debugfs_create_u32_array (const char *name,umode_t mode, -struct dentry *parent,u32 *array, -u32 elements) +struct dentry *parent, +struct debugfs_u32_array *array) { }
mm/cma.h +3 -0
@@ -2,6 +2,8 @@ #ifndef __MM_CMA_H__ #define __MM_CMA_H__ +#include <linux/debugfs.h> + struct cma { unsigned long base_pfn; unsigned long count;
@@ -11,6 +13,7 @@ struct cma {#ifdef CONFIG_CMA_DEBUGFS struct hlist_head mem_head;spinlock_t mem_head_lock; +struct debugfs_u32_array dfs_bitmap;#endif const char *name; };
mm/cma_debug.c +4 -3
@@ -164,7 +164,6 @@ static void cma_debugfs_add_one(struct cma *cma, struct dentry *root_dentry) {struct dentry *tmp;char name[16 ]; -int u32s;scnprintf (name,sizeof (name),"cma-%s" , cma->name);
@@ -180,8 +179,10 @@ static void cma_debugfs_add_one(struct cma *cma, struct dentry *root_dentry)debugfs_create_file ("used" ,0444 , tmp, cma, &cma_used_fops);debugfs_create_file ("maxchunk" ,0444 , tmp, cma, &cma_maxchunk_fops); - u32s =DIV_ROUND_UP (cma_bitmap_maxno (cma),BITS_PER_BYTE *sizeof (u32)); -debugfs_create_u32_array ("bitmap" ,0444 , tmp, (u32 *)cma->bitmap, u32s); + cma->dfs_bitmap.array = (u32 *)cma->bitmap; + cma->dfs_bitmap.n_elements =DIV_ROUND_UP (cma_bitmap_maxno (cma), +BITS_PER_BYTE *sizeof (u32)); +debugfs_create_u32_array ("bitmap" ,0444 , tmp, &cma->dfs_bitmap); }static int __init cma_debugfs_init (void )
Signature · 1 line
2.26.2
Jakub Kicinski
maintainerPATCH v4 4/10 ethtool: add tunnel info interface
In reply to Jakub KicinskiAdd an interface to report offloaded UDP ports via ethtool netlink.
Now that core takes care of tracking which UDP tunnel ports the NICs are aware of we can quite easily export this information out to user space.
The responsibility of writing the netlink dumps is split between ethtool code and udp_tunnel_nic.c - since udp_tunnel module may not always be loaded, yet we should always report the capabilities of the NIC.
$ ethtool --show-tunnels eth0
Tunnel information for eth0:
UDP port table 0:
Size: 4
Types: vxlan
No entries
UDP port table 1:
Size: 4
Types: geneve, vxlan-gpe
Entries (1):
port 1230, vxlan-gpev4:
- back to v2, build fix is now directly in udp_tunnel.h
v3:
- don't compile ETHTOOL_MSG_TUNNEL_INFO_GET in if CONFIG_INET not set.
v2:
- fix string set count,
- reorder enums in the uAPI,
- fix type of ETHTOOL_A_TUNNEL_UDP_TABLE_TYPES to bitset in docs and comments.
--- Documentation/networking/ethtool-netlink.rst | 33 +++ include/net/udp_tunnel.h | 21 ++ include/uapi/linux/ethtool.h | 2 + include/uapi/linux/ethtool_netlink.h | 55 ++++ net/ethtool/Makefile | 3 +- net/ethtool/common.c | 9 + net/ethtool/common.h | 1 + net/ethtool/netlink.c | 12 + net/ethtool/netlink.h | 4 + net/ethtool/strset.c | 5 + net/ethtool/tunnels.c | 259 +++++++++++++++++++ net/ipv4/udp_tunnel_nic.c | 69 +++++ 12 files changed, 472 insertions(+), 1 deletion(-) create mode 100644 net/ethtool/tunnels.c
Documentation/networking/ethtool-netlink.rst +33 -0
@@ -1230,6 +1230,39 @@ used to report the amplitude of the reflection for a given pair. | | | ``ETHTOOL_A_CABLE_AMPLITUDE_mV`` | s16 | Reflection amplitude | +-+-+-----------------------------------------+--------+----------------------+ +TUNNEL_INFO +=========== + +Gets information about the tunnel state NIC is aware of. + +Request contents: + + ===================================== ====== ========================== + ``ETHTOOL_A_TUNNEL_INFO_HEADER`` nested request header + ===================================== ====== ========================== + +Kernel response contents: + + +---------------------------------------------+--------+---------------------+ + | ``ETHTOOL_A_TUNNEL_INFO_HEADER`` | nested | reply header | + +---------------------------------------------+--------+---------------------+ + | ``ETHTOOL_A_TUNNEL_INFO_UDP_PORTS`` | nested | all UDP port tables | + +-+-------------------------------------------+--------+---------------------+ + | | ``ETHTOOL_A_TUNNEL_UDP_TABLE`` | nested | one UDP port table | + +-+-+-----------------------------------------+--------+---------------------+ + | | | ``ETHTOOL_A_TUNNEL_UDP_TABLE_SIZE`` | u32 | max size of the | + | | | | | table | + +-+-+-----------------------------------------+--------+---------------------+ + | | | ``ETHTOOL_A_TUNNEL_UDP_TABLE_TYPES`` | bitset | tunnel types which | + | | | | | table can hold | + +-+-+-----------------------------------------+--------+---------------------+ + | | | ``ETHTOOL_A_TUNNEL_UDP_TABLE_ENTRY`` | nested | offloaded UDP port | + +-+-+-+---------------------------------------+--------+---------------------+ + | | | | ``ETHTOOL_A_TUNNEL_UDP_ENTRY_PORT`` | be16 | UDP port | + +-+-+-+---------------------------------------+--------+---------------------+ + | | | | ``ETHTOOL_A_TUNNEL_UDP_ENTRY_TYPE`` | u32 | tunnel type | + +-+-+-+---------------------------------------+--------+---------------------+ + Request translation ===================
include/net/udp_tunnel.h +21 -0
@@ -255,6 +255,10 @@ struct udp_tunnel_nic_ops {void (*add_port)(struct net_device *dev,struct udp_tunnel_info *ti);void (*del_port)(struct net_device *dev,struct udp_tunnel_info *ti);void (*reset_ntf)(struct net_device *dev); + +size_t (*dump_size)(struct net_device *dev,unsigned int table); +int (*dump_write)(struct net_device *dev,unsigned int table, +struct sk_buff *skb); };#ifdef CONFIG_INET
@@ -318,4 +322,21 @@ static inline void udp_tunnel_nic_reset_ntf(struct net_device *dev)if (udp_tunnel_nic_ops) udp_tunnel_nic_ops->reset_ntf(dev); } + +static inline size_t +udp_tunnel_nic_dump_size (struct net_device *dev,unsigned int table) +{ +if (!udp_tunnel_nic_ops) +return 0 ; +return udp_tunnel_nic_ops->dump_size (dev, table); +} + +static inline int +udp_tunnel_nic_dump_write (struct net_device *dev,unsigned int table, +struct sk_buff *skb) +{ +if (!udp_tunnel_nic_ops) +return 0 ; +return udp_tunnel_nic_ops->dump_write (dev, table, skb); +}#endif
include/uapi/linux/ethtool.h +2 -0
@@ -669,6 +669,7 @@ enum ethtool_link_ext_substate_cable_issue {* @ETH_SS_SOF_TIMESTAMPING: SOF_TIMESTAMPING_* flags * @ETH_SS_TS_TX_TYPES: timestamping Tx types * @ETH_SS_TS_RX_FILTERS: timestamping Rx filters +* @ETH_SS_UDP_TUNNEL_TYPES: UDP tunnel types */ enum ethtool_stringset {ETH_SS_TEST =0 ,
@@ -686,6 +687,7 @@ enum ethtool_stringset {ETH_SS_SOF_TIMESTAMPING ,ETH_SS_TS_TX_TYPES ,ETH_SS_TS_RX_FILTERS , +ETH_SS_UDP_TUNNEL_TYPES ,/* add new constants above here */ ETH_SS_COUNT
include/uapi/linux/ethtool_netlink.h +55 -0
67 lines not on this page · raw message
net/ethtool/Makefile +2 -1
@@ -6,4 +6,5 @@ obj-$(CONFIG_ETHTOOL_NETLINK) += ethtool_nl.o ethtool_nl-y := netlink.o bitset.o strset.o linkinfo.o linkmodes.o \ linkstate.o debug.o wol.o features.o privflags.o rings.o \ - channels.o coalesce.o pause.o eee.o tsinfo.o cabletest.o + channels.o coalesce.o pause.o eee.o tsinfo.o cabletest.o \ + tunnels.o
net/ethtool/common.c +9 -0
@@ -1,5 +1,6 @@// SPDX-License-Identifier: GPL-2.0-only +#include <linux/ethtool_netlink.h> #include <linux/net_tstamp.h> #include <linux/phy.h> #include <linux/rtnetlink.h>
@@ -257,6 +258,14 @@ const char ts_rx_filter_names[][ETH_GSTRING_LEN] = { };static_assert (ARRAY_SIZE (ts_rx_filter_names) == __HWTSTAMP_FILTER_CNT); +const char udp_tunnel_type_names[][ETH_GSTRING_LEN ] = { + [ETHTOOL_UDP_TUNNEL_TYPE_VXLAN ] ="vxlan" , + [ETHTOOL_UDP_TUNNEL_TYPE_GENEVE ] ="geneve" , + [ETHTOOL_UDP_TUNNEL_TYPE_VXLAN_GPE ] ="vxlan-gpe" , +}; +static_assert (ARRAY_SIZE (udp_tunnel_type_names) == + __ETHTOOL_UDP_TUNNEL_TYPE_CNT); +/* return false if legacy contained non-0 deprecated fields * maxtxpkt/maxrxpkt. rest of ksettings always updated */
net/ethtool/common.h +1 -0
@@ -28,6 +28,7 @@ extern const char wol_mode_names[][ETH_GSTRING_LEN];extern const char sof_timestamping_names[][ETH_GSTRING_LEN ];extern const char ts_tx_type_names[][ETH_GSTRING_LEN ];extern const char ts_rx_filter_names[][ETH_GSTRING_LEN ]; +extern const char udp_tunnel_type_names[][ETH_GSTRING_LEN ];int __ethtool_get_link (struct net_device *dev);
net/ethtool/netlink.c +12 -0
@@ -181,6 +181,12 @@ struct sk_buff *ethnl_reply_init(size_t payload, struct net_device *dev, u8 cmd,return NULL ; } +void *ethnl_dump_put (struct sk_buff *skb,struct netlink_callback *cb,u8 cmd) +{ +return genlmsg_put (skb,NETLINK_CB (cb->skb).portid, cb->nlh->nlmsg_seq, + ðtool_genl_family,0 , cmd); +} +void *ethnl_bcastmsg_put (struct sk_buff *skb,u8 cmd) { return genlmsg_put(skb, 0, ++ethnl_bcast_seq, ðtool_genl_family, 0,
@@ -849,6 +855,12 @@ static const struct genl_ops ethtool_genl_ops[] = { .flags = GENL_UNS_ADMIN_PERM, .doit = ethnl_act_cable_test_tdr, }, + { + .cmd = ETHTOOL_MSG_TUNNEL_INFO_GET, + .doit = ethnl_tunnel_info_doit, + .start = ethnl_tunnel_info_start, + .dumpit = ethnl_tunnel_info_dumpit, + }, };static const struct genl_multicast_group ethtool_nl_mcgrps[] = {
net/ethtool/netlink.h +4 -0
@@ -19,6 +19,7 @@ int ethnl_fill_reply_header(struct sk_buff *skb, struct net_device *dev,struct sk_buff *ethnl_reply_init (size_t payload,struct net_device *dev,u8 cmd,u16 hdr_attrtype,struct genl_info *info,void **ehdrp); +void *ethnl_dump_put (struct sk_buff *skb,struct netlink_callback *cb,u8 cmd);void *ethnl_bcastmsg_put (struct sk_buff *skb,u8 cmd);int ethnl_multicast (struct sk_buff *skb,struct net_device *dev);
@@ -361,5 +362,8 @@ int ethnl_set_pause(struct sk_buff *skb, struct genl_info *info);int ethnl_set_eee (struct sk_buff *skb,struct genl_info *info);int ethnl_act_cable_test (struct sk_buff *skb,struct genl_info *info);int ethnl_act_cable_test_tdr (struct sk_buff *skb,struct genl_info *info); +int ethnl_tunnel_info_doit (struct sk_buff *skb,struct genl_info *info); +int ethnl_tunnel_info_start (struct netlink_callback *cb); +int ethnl_tunnel_info_dumpit (struct sk_buff *skb,struct netlink_callback *cb);#endif /* _NET_ETHTOOL_NETLINK_H */
net/ethtool/strset.c +5 -0
@@ -75,6 +75,11 @@ static const struct strset_info info_template[] = { .count = __HWTSTAMP_FILTER_CNT, .strings = ts_rx_filter_names, }, + [ETH_SS_UDP_TUNNEL_TYPES] = { + .per_dev = false, + .count = __ETHTOOL_UDP_TUNNEL_TYPE_CNT, + .strings = udp_tunnel_type_names, + }, }; struct strset_req_info {
net/ethtool/tunnels.c new file +259 -0
259 lines not on this page · raw message
net/ipv4/udp_tunnel_nic.c +69 -0
93 lines not on this page · raw message
Signature · 1 line
2.26.2
Jakub Kicinski
maintainerPATCH v4 3/10 udp_tunnel: add central NIC RX port offload infrastructure
In reply to Jakub KicinskiCater to devices which:
(a) may want to sleep in the callbacks;
(b) only have IPv4 support;
(c) need all the programming to happen while the netdev is up.
Drivers attach UDP tunnel offload info struct to their netdevs, where they declare how many UDP ports of various tunnel types they support. Core takes care of tracking which ports to offload.
Use a fixed-size array since this matches what almost all drivers do, and avoids a complexity and uncertainty around memory allocations in an atomic context.
Make sure that tunnel drivers don't try to replay the ports when new NIC netdev is registered. Automatic replays would mess up reference counting, and will be removed completely once all drivers are converted.
v4:
- use a #define NULL to avoid build issues with CONFIG_INET=n.
---
drivers/net/geneve.c | 6 +-
drivers/net/vxlan.c | 6 +-
include/linux/netdevice.h | 8 +
include/net/udp_tunnel.h | 137 ++++
net/ipv4/Makefile | 3 +-
net/ipv4/{udp_tunnel.c => udp_tunnel_core.c} | 0
net/ipv4/udp_tunnel_nic.c | 821 +++++++++++++++++++
net/ipv4/udp_tunnel_stub.c | 7 +
8 files changed, 983 insertions(+), 5 deletions(-)
rename net/ipv4/{udp_tunnel.c => udp_tunnel_core.c} (100%)
create mode 100644 net/ipv4/udp_tunnel_nic.c
create mode 100644 net/ipv4/udp_tunnel_stub.cdrivers/net/geneve.c +4 -2
@@ -1796,9 +1796,11 @@ static int geneve_netdevice_event(struct notifier_block *unused, event == NETDEV_UDP_TUNNEL_DROP_INFO) {geneve_offload_rx_ports (dev, event ==NETDEV_UDP_TUNNEL_PUSH_INFO ); } else if (event == NETDEV_UNREGISTER) { -geneve_offload_rx_ports (dev, false); +if (!dev->udp_tunnel_nic_info) +geneve_offload_rx_ports (dev, false); } else if (event == NETDEV_REGISTER) { -geneve_offload_rx_ports (dev, true); +if (!dev->udp_tunnel_nic_info) +geneve_offload_rx_ports (dev, true); }return NOTIFY_DONE ;
drivers/net/vxlan.c +4 -2
@@ -4477,10 +4477,12 @@ static int vxlan_netdevice_event(struct notifier_block *unused,struct vxlan_net *vn =net_generic (dev_net (dev), vxlan_net_id);if (event ==NETDEV_UNREGISTER ) { -vxlan_offload_rx_ports (dev, false); +if (!dev->udp_tunnel_nic_info) +vxlan_offload_rx_ports (dev, false);vxlan_handle_lowerdev_unregister (vn, dev); }else if (event ==NETDEV_REGISTER ) { -vxlan_offload_rx_ports (dev, true); +if (!dev->udp_tunnel_nic_info) +vxlan_offload_rx_ports (dev, true); }else if (event ==NETDEV_UDP_TUNNEL_PUSH_INFO || event ==NETDEV_UDP_TUNNEL_DROP_INFO ) {vxlan_offload_rx_ports (dev, event ==NETDEV_UDP_TUNNEL_PUSH_INFO );
include/linux/netdevice.h +8 -0
@@ -65,6 +65,8 @@ struct wpan_dev;struct mpls_dev ;/* UDP Tunnel offloads */ struct udp_tunnel_info ; +struct udp_tunnel_nic_info ; +struct udp_tunnel_nic ;struct bpf_prog ;struct xdp_buff ;
@@ -1836,6 +1838,10 @@ enum netdev_priv_flags {* * @macsec_ops: MACsec offloading ops * +* @udp_tunnel_nic_info: static structure describing the UDP tunnel +* offload capabilities of the device +* @udp_tunnel_nic: UDP tunnel offload state +* * FIXME: cleanup struct net_device such that network protocol info * moves out. */
@@ -2134,6 +2140,8 @@ struct net_device {/* MACsec management functions */ const struct macsec_ops *macsec_ops;#endif +const struct udp_tunnel_nic_info *udp_tunnel_nic_info; +struct udp_tunnel_nic *udp_tunnel_nic; };#define to_net_dev (d) container_of(d, struct net_device, dev)
include/net/udp_tunnel.h +137 -0
147 lines not on this page · raw message
net/ipv4/Makefile +2 -1
@@ -14,7 +14,7 @@ obj-y := route.o inetpeer.o protocol.o \ udp_offload.o arp.o icmp.o devinet.o af_inet.o igmp.o \ fib_frontend.o fib_semantics.o fib_trie.o fib_notifier.o \ inet_fragment.o ping.o ip_tunnel_core.o gre_offload.o \ - metrics.o netlink.o nexthop.o + metrics.o netlink.o nexthop.o udp_tunnel_stub.o obj-$(CONFIG_BPFILTER) += bpfilter/
@@ -29,6 +29,7 @@ gre-y := gre_demux.o obj-$(CONFIG_NET_FOU) += fou.o obj-$(CONFIG_NET_IPGRE_DEMUX) += gre.o obj-$(CONFIG_NET_IPGRE) += ip_gre.o +udp_tunnel-y := udp_tunnel_core.o udp_tunnel_nic.o obj-$(CONFIG_NET_UDP_TUNNEL) += udp_tunnel.o obj-$(CONFIG_NET_IPVTI) += ip_vti.o obj-$(CONFIG_SYN_COOKIES) += syncookies.o
net/ipv4/udp_tunnel_core.c renamed from net/ipv4/udp_tunnel.c +0 -0
net/ipv4/udp_tunnel_nic.c new file +821 -0
821 lines not on this page · raw message
net/ipv4/udp_tunnel_stub.c new file +7 -0
@@ -0,0 +1,7 @@ +// SPDX-License-Identifier: GPL-2.0-only +// Copyright (c) 2020 Facebook Inc. + +#include <net/udp_tunnel.h> + +const struct udp_tunnel_nic_ops *udp_tunnel_nic_ops; +EXPORT_SYMBOL_GPL (udp_tunnel_nic_ops);
Signature · 1 line
2.26.2
Jakub Kicinski
maintainerPATCH v4 5/10 netdevsim: add UDP tunnel port offload support
In reply to Jakub KicinskiAdd UDP tunnel port handlers to our fake driver so we can test the core infra.
--- drivers/net/netdevsim/Makefile | 2 +- drivers/net/netdevsim/dev.c | 1 + drivers/net/netdevsim/netdev.c | 12 +- drivers/net/netdevsim/netdevsim.h | 19 +++ drivers/net/netdevsim/udp_tunnels.c | 192 ++++++++++++++++++++++++++++ 5 files changed, 224 insertions(+), 2 deletions(-) create mode 100644 drivers/net/netdevsim/udp_tunnels.c
drivers/net/netdevsim/Makefile +1 -1
@@ -3,7 +3,7 @@ obj-$(CONFIG_NETDEVSIM) += netdevsim.o netdevsim-objs := \ - netdev.o dev.o fib.o bus.o health.o + netdev.o dev.o fib.o bus.o health.o udp_tunnels.o ifeq ($(CONFIG_BPF_SYSCALL),y) netdevsim-objs += \
drivers/net/netdevsim/dev.c +1 -0
@@ -225,6 +225,7 @@ static int nsim_dev_debugfs_init(struct nsim_dev *nsim_dev)debugfs_create_bool ("fail_trap_policer_counter_get" ,0600 , nsim_dev->ddir, &nsim_dev->fail_trap_policer_counter_get); +nsim_udp_tunnels_debugfs_create (nsim_dev);return 0 ; }
drivers/net/netdevsim/netdev.c +11 -1
@@ -22,6 +22,7 @@#include <net/netlink.h> #include <net/pkt_cls.h> #include <net/rtnetlink.h> +#include <net/udp_tunnel.h> #include "netdevsim.h"
@@ -257,6 +258,8 @@ static const struct net_device_ops nsim_netdev_ops = { .ndo_setup_tc = nsim_setup_tc, .ndo_set_features = nsim_set_features, .ndo_bpf = nsim_bpf, + .ndo_udp_tunnel_add = udp_tunnel_nic_add_port, + .ndo_udp_tunnel_del = udp_tunnel_nic_del_port, .ndo_get_devlink_port = nsim_get_devlink_port, };
@@ -299,10 +302,14 @@ nsim_create(struct nsim_dev *nsim_dev, struct nsim_dev_port *nsim_dev_port)SET_NETDEV_DEV (dev, &ns->nsim_bus_dev->dev); dev->netdev_ops = &nsim_netdev_ops; + err =nsim_udp_tunnels_info_create (nsim_dev, dev); +if (err) + goto err_free_netdev; +rtnl_lock (); err =nsim_bpf_init (ns);if (err) - goto err_free_netdev; + goto err_utn_destroy;nsim_ipsec_init (ns);
@@ -317,6 +324,8 @@ nsim_create(struct nsim_dev *nsim_dev, struct nsim_dev_port *nsim_dev_port)nsim_ipsec_teardown (ns);nsim_bpf_uninit (ns);rtnl_unlock (); +err_utn_destroy: +nsim_udp_tunnels_info_destroy (dev); err_free_netdev:free_netdev (dev);return ERR_PTR (err);
@@ -331,6 +340,7 @@ void nsim_destroy(struct netdevsim *ns)nsim_ipsec_teardown (ns);nsim_bpf_uninit (ns);rtnl_unlock (); +nsim_udp_tunnels_info_destroy (dev);free_netdev (dev); }
drivers/net/netdevsim/netdevsim.h +19 -0
@@ -13,6 +13,7 @@* THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION. */ +#include <linux/debugfs.h> #include <linux/device.h> #include <linux/kernel.h> #include <linux/list.h>
@@ -29,6 +30,7 @@#define NSIM_IPSEC_MAX_SA_COUNT 33#define NSIM_IPSEC_VALID BIT(31) +#define NSIM_UDP_TUNNEL_N_PORTS 4struct nsim_sa {struct xfrm_state *xs;
@@ -72,12 +74,23 @@ struct netdevsim {bool bpf_map_accept;struct nsim_ipsec ipsec; +struct { +u32 inject_error; +u32 sleep; +u32 ports[2 ][NSIM_UDP_TUNNEL_N_PORTS ]; +struct debugfs_u32_array dfs_ports[2 ]; + } udp_ports; };struct netdevsim *nsim_create (struct nsim_dev *nsim_dev,struct nsim_dev_port *nsim_dev_port);void nsim_destroy (struct netdevsim *ns); +void nsim_udp_tunnels_debugfs_create (struct nsim_dev *nsim_dev); +int nsim_udp_tunnels_info_create (struct nsim_dev *nsim_dev, +struct net_device *dev); +void nsim_udp_tunnels_info_destroy (struct net_device *dev); +#ifdef CONFIG_BPF_SYSCALL int nsim_bpf_dev_init (struct nsim_dev *nsim_dev);void nsim_bpf_dev_exit (struct nsim_dev *nsim_dev);
@@ -183,6 +196,12 @@ struct nsim_dev {bool fail_trap_group_set;bool fail_trap_policer_set;bool fail_trap_policer_counter_get; +struct { +bool sync_all; +bool open_only; +bool ipv4_only; +u32 sleep; + } udp_ports; };static inline struct net *nsim_dev_net (struct nsim_dev *nsim_dev)
drivers/net/netdevsim/udp_tunnels.c new file +192 -0
192 lines not on this page · raw message
Signature · 1 line
2.26.2
Jakub Kicinski
maintainerPATCH v4 10/10 mlx4: convert to new udp_tunnel_nic infra
In reply to Jakub KicinskiConvert to new infra, make use of the ability to sleep in the callback.
--- .../net/ethernet/mellanox/mlx4/en_netdev.c | 107 ++++-------------- drivers/net/ethernet/mellanox/mlx4/mlx4_en.h | 2 - 2 files changed, 25 insertions(+), 84 deletions(-)
drivers/net/ethernet/mellanox/mlx4/en_netdev.c +25 -82
@@ -1816,7 +1816,7 @@ int mlx4_en_start_port(struct net_device *dev)queue_work (mdev->workqueue, &priv->rx_mode_task);if (priv->mdev->dev->caps.tunnel_offload_mode ==MLX4_TUNNEL_OFFLOAD_MODE_VXLAN ) -udp_tunnel_get_rx_info (dev); +udp_tunnel_nic_reset_ntf (dev); priv->port_up = true;
@@ -2628,89 +2628,32 @@ static int mlx4_en_get_phys_port_id(struct net_device *dev,return 0 ; } -static void mlx4_en_add_vxlan_offloads (struct work_struct *work) +static int mlx4_udp_tunnel_sync (struct net_device *dev,unsigned int table) { +struct mlx4_en_priv *priv =netdev_priv (dev); +struct udp_tunnel_info ti;int ret; - struct mlx4_en_priv *priv = container_of(work, struct mlx4_en_priv, - vxlan_add_task); - ret =mlx4_config_vxlan_port (priv->mdev->dev, priv->vxlan_port); -if (ret) - goto out; +udp_tunnel_nic_get_port (dev, table,0 , &ti); + priv->vxlan_port = ti.port; - ret =mlx4_SET_PORT_VXLAN (priv->mdev->dev, priv->port, -VXLAN_STEER_BY_OUTER_MAC ,1 ); -out: -if (ret) { -en_err (priv,"failed setting L2 tunnel configuration ret %d\n" , ret); -return ; - } -} - -static void mlx4_en_del_vxlan_offloads (struct work_struct *work) -{ -int ret; - struct mlx4_en_priv *priv = container_of(work, struct mlx4_en_priv, - vxlan_del_task); - ret =mlx4_SET_PORT_VXLAN (priv->mdev->dev, priv->port, -VXLAN_STEER_BY_OUTER_MAC ,0 ); + ret =mlx4_config_vxlan_port (priv->mdev->dev, priv->vxlan_port);if (ret) -en_err (priv,"failed setting L2 tunnel configuration ret %d\n" , ret); +return ret; - priv->vxlan_port =0 ; +return mlx4_SET_PORT_VXLAN (priv->mdev->dev, priv->port, +VXLAN_STEER_BY_OUTER_MAC , + !!priv->vxlan_port); } -static void mlx4_en_add_vxlan_port (struct net_device *dev, -struct udp_tunnel_info *ti) -{ -struct mlx4_en_priv *priv =netdev_priv (dev); -__be16 port = ti->port; -__be16 current_port; - -if (ti->type !=UDP_TUNNEL_TYPE_VXLAN ) -return ; - -if (ti->sa_family !=AF_INET ) -return ; - -if (priv->mdev->dev->caps.tunnel_offload_mode !=MLX4_TUNNEL_OFFLOAD_MODE_VXLAN ) -return ; - - current_port = priv->vxlan_port; -if (current_port && current_port != port) { -en_warn (priv,"vxlan port %d configured, can't add port %d\n" , -ntohs (current_port),ntohs (port)); -return ; - } - - priv->vxlan_port = port; -queue_work (priv->mdev->workqueue, &priv->vxlan_add_task); -} - -static void mlx4_en_del_vxlan_port (struct net_device *dev, -struct udp_tunnel_info *ti) -{ -struct mlx4_en_priv *priv =netdev_priv (dev); -__be16 port = ti->port; -__be16 current_port; - -if (ti->type !=UDP_TUNNEL_TYPE_VXLAN ) -return ; - -if (ti->sa_family !=AF_INET ) -return ; - -if (priv->mdev->dev->caps.tunnel_offload_mode !=MLX4_TUNNEL_OFFLOAD_MODE_VXLAN ) -return ; - - current_port = priv->vxlan_port; -if (current_port != port) { -en_dbg (DRV , priv,"vxlan port %d isn't configured, ignoring\n" ,ntohs (port)); -return ; - } - -queue_work (priv->mdev->workqueue, &priv->vxlan_del_task); -} +static const struct udp_tunnel_nic_info mlx4_udp_tunnels = { + .sync_table = mlx4_udp_tunnel_sync, + .flags =UDP_TUNNEL_NIC_INFO_MAY_SLEEP | +UDP_TUNNEL_NIC_INFO_IPV4_ONLY , + .tables = { + { .n_entries =1 , .tunnel_types =UDP_TUNNEL_TYPE_VXLAN , }, + }, +}; static netdev_features_t mlx4_en_features_check(struct sk_buff *skb, struct net_device *dev,
@@ -2914,8 +2857,8 @@ static const struct net_device_ops mlx4_netdev_ops = { .ndo_rx_flow_steer = mlx4_en_filter_rfs, #endif .ndo_get_phys_port_id = mlx4_en_get_phys_port_id, - .ndo_udp_tunnel_add = mlx4_en_add_vxlan_port, - .ndo_udp_tunnel_del = mlx4_en_del_vxlan_port, + .ndo_udp_tunnel_add = udp_tunnel_nic_add_port, + .ndo_udp_tunnel_del = udp_tunnel_nic_del_port, .ndo_features_check = mlx4_en_features_check, .ndo_set_tx_maxrate = mlx4_en_set_tx_maxrate, .ndo_bpf = mlx4_xdp,
@@ -2948,8 +2891,8 @@ static const struct net_device_ops mlx4_netdev_ops_master = { .ndo_rx_flow_steer = mlx4_en_filter_rfs, #endif .ndo_get_phys_port_id = mlx4_en_get_phys_port_id, - .ndo_udp_tunnel_add = mlx4_en_add_vxlan_port, - .ndo_udp_tunnel_del = mlx4_en_del_vxlan_port, + .ndo_udp_tunnel_add = udp_tunnel_nic_add_port, + .ndo_udp_tunnel_del = udp_tunnel_nic_del_port, .ndo_features_check = mlx4_en_features_check, .ndo_set_tx_maxrate = mlx4_en_set_tx_maxrate, .ndo_bpf = mlx4_xdp,
@@ -3250,8 +3193,6 @@ int mlx4_en_init_netdev(struct mlx4_en_dev *mdev, int port,INIT_WORK (&priv->linkstate_task, mlx4_en_linkstate);INIT_DELAYED_WORK (&priv->stats_task, mlx4_en_do_get_stats);INIT_DELAYED_WORK (&priv->service_task, mlx4_en_service_task); -INIT_WORK (&priv->vxlan_add_task, mlx4_en_add_vxlan_offloads); -INIT_WORK (&priv->vxlan_del_task, mlx4_en_del_vxlan_offloads);#ifdef CONFIG_RFS_ACCEL INIT_LIST_HEAD (&priv->filters);spin_lock_init (&priv->filters_lock);
@@ -3406,6 +3347,8 @@ int mlx4_en_init_netdev(struct mlx4_en_dev *mdev, int port,NETIF_F_GSO_UDP_TUNNEL |NETIF_F_GSO_UDP_TUNNEL_CSUM |NETIF_F_GSO_PARTIAL ; + + dev->udp_tunnel_nic_info = &mlx4_udp_tunnels; } dev->vlan_features = dev->hw_features;
drivers/net/ethernet/mellanox/mlx4/mlx4_en.h +0 -2
@@ -599,8 +599,6 @@ struct mlx4_en_priv {struct work_struct linkstate_task;struct delayed_work stats_task;struct delayed_work service_task; -struct work_struct vxlan_add_task; -struct work_struct vxlan_del_task;struct mlx4_en_perf_stats pstats;struct mlx4_en_pkt_stats pkstats;struct mlx4_en_counter_stats pf_stats;
Signature · 1 line
2.26.2
Jakub Kicinski
maintainerPATCH v4 8/10 ixgbe: convert to new udp_tunnel_nic infra
In reply to Jakub KicinskiMake use of new common udp_tunnel_nic infra. ixgbe supports IPv4 only, and only single VxLAN and Geneve ports (one each).
v2:
- split out the RXCSUM feature handling to separate change;
- declare structs separately;
- use ti.type instead of assuming table 0 is VxLAN;
- move setting netdev->udp_tunnel_nic_info to its own switch.
--- drivers/net/ethernet/intel/ixgbe/ixgbe.h | 3 - drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 180 +++++------------- 2 files changed, 44 insertions(+), 139 deletions(-)
drivers/net/ethernet/intel/ixgbe/ixgbe.h +0 -3
@@ -588,11 +588,9 @@ struct ixgbe_adapter {#define IXGBE_FLAG_FCOE_ENABLED BIT(21)#define IXGBE_FLAG_SRIOV_CAPABLE BIT(22)#define IXGBE_FLAG_SRIOV_ENABLED BIT(23) -#define IXGBE_FLAG_VXLAN_OFFLOAD_CAPABLE BIT(24)#define IXGBE_FLAG_RX_HWTSTAMP_ENABLED BIT(25)#define IXGBE_FLAG_RX_HWTSTAMP_IN_REGISTER BIT(26)#define IXGBE_FLAG_DCB_CAPABLE BIT(27) -#define IXGBE_FLAG_GENEVE_OFFLOAD_CAPABLE BIT(28)u32 flags2;#define IXGBE_FLAG2_RSC_CAPABLE BIT(0)
@@ -606,7 +604,6 @@ struct ixgbe_adapter {#define IXGBE_FLAG2_RSS_FIELD_IPV6_UDP BIT(9)#define IXGBE_FLAG2_PTP_PPS_ENABLED BIT(10)#define IXGBE_FLAG2_PHY_INTERRUPT BIT(11) -#define IXGBE_FLAG2_UDP_TUN_REREG_NEEDED BIT(12)#define IXGBE_FLAG2_VLAN_PROMISC BIT(13)#define IXGBE_FLAG2_EEE_CAPABLE BIT(14)#define IXGBE_FLAG2_EEE_ENABLED BIT(15)
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +44 -136
234 lines not on this page · raw message
Signature · 1 line
2.26.2
Jakub Kicinski
maintainerPATCH v4 9/10 bnxt: convert to new udp_tunnel_nic infra
In reply to Jakub KicinskiConvert to new infra, taking advantage of sleeping in callbacks.
v2:
- use bp->*_fw_dst_port_id != INVALID_HW_RING_ID as indication that the offload is active.
--- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 141 ++++++---------------- drivers/net/ethernet/broadcom/bnxt/bnxt.h | 12 +- 2 files changed, 40 insertions(+), 113 deletions(-)
drivers/net/ethernet/broadcom/bnxt/bnxt.c +38 -103
@@ -4505,10 +4505,12 @@ static int bnxt_hwrm_tunnel_dst_port_free(struct bnxt *bp, u8 tunnel_type)switch (tunnel_type) {case TUNNEL_DST_PORT_FREE_REQ_TUNNEL_TYPE_VXLAN : - req.tunnel_dst_port_id = bp->vxlan_fw_dst_port_id; + req.tunnel_dst_port_id =cpu_to_le16 (bp->vxlan_fw_dst_port_id); + bp->vxlan_fw_dst_port_id =INVALID_HW_RING_ID ;break ;case TUNNEL_DST_PORT_FREE_REQ_TUNNEL_TYPE_GENEVE : - req.tunnel_dst_port_id = bp->nge_fw_dst_port_id; + req.tunnel_dst_port_id =cpu_to_le16 (bp->nge_fw_dst_port_id); + bp->nge_fw_dst_port_id =INVALID_HW_RING_ID ;break ;default :break ;
@@ -4543,10 +4545,11 @@ static int bnxt_hwrm_tunnel_dst_port_alloc(struct bnxt *bp, __be16 port,switch (tunnel_type) {case TUNNEL_DST_PORT_ALLOC_REQ_TUNNEL_TYPE_VXLAN : - bp->vxlan_fw_dst_port_id = resp->tunnel_dst_port_id; + bp->vxlan_fw_dst_port_id = +le16_to_cpu (resp->tunnel_dst_port_id);break ;case TUNNEL_DST_PORT_ALLOC_REQ_TUNNEL_TYPE_GENEVE : - bp->nge_fw_dst_port_id = resp->tunnel_dst_port_id; + bp->nge_fw_dst_port_id =le16_to_cpu (resp->tunnel_dst_port_id);break ;default :break ;
@@ -7470,16 +7473,12 @@ static int bnxt_hwrm_pcie_qstats(struct bnxt *bp)static void bnxt_hwrm_free_tunnel_ports (struct bnxt *bp) { -if (bp->vxlan_port_cnt) { +if (bp->vxlan_fw_dst_port_id !=INVALID_HW_RING_ID )bnxt_hwrm_tunnel_dst_port_free ( bp,TUNNEL_DST_PORT_FREE_REQ_TUNNEL_TYPE_VXLAN ); - } - bp->vxlan_port_cnt =0 ; -if (bp->nge_port_cnt) { +if (bp->nge_fw_dst_port_id !=INVALID_HW_RING_ID )bnxt_hwrm_tunnel_dst_port_free ( bp,TUNNEL_DST_PORT_FREE_REQ_TUNNEL_TYPE_GENEVE ); - } - bp->nge_port_cnt =0 ; }static int bnxt_set_tpa (struct bnxt *bp,bool set_tpa)
@@ -9194,7 +9193,7 @@ static int __bnxt_open_nic(struct bnxt *bp, bool irq_re_init, bool link_re_init) }if (irq_re_init) -udp_tunnel_get_rx_info (bp->dev); +udp_tunnel_nic_reset_ntf (bp->dev);set_bit (BNXT_STATE_OPEN , &bp->state);bnxt_enable_int (bp);
@@ -10353,24 +10352,6 @@ static void bnxt_sp_task(struct work_struct *work)bnxt_cfg_ntp_filters (bp);if (test_and_clear_bit (BNXT_HWRM_EXEC_FWD_REQ_SP_EVENT , &bp->sp_event))bnxt_hwrm_exec_fwd_req (bp); -if (test_and_clear_bit (BNXT_VXLAN_ADD_PORT_SP_EVENT , &bp->sp_event)) { -bnxt_hwrm_tunnel_dst_port_alloc ( - bp, bp->vxlan_port, -TUNNEL_DST_PORT_FREE_REQ_TUNNEL_TYPE_VXLAN ); - } -if (test_and_clear_bit (BNXT_VXLAN_DEL_PORT_SP_EVENT , &bp->sp_event)) { -bnxt_hwrm_tunnel_dst_port_free ( - bp,TUNNEL_DST_PORT_FREE_REQ_TUNNEL_TYPE_VXLAN ); - } -if (test_and_clear_bit (BNXT_GENEVE_ADD_PORT_SP_EVENT , &bp->sp_event)) { -bnxt_hwrm_tunnel_dst_port_alloc ( - bp, bp->nge_port, -TUNNEL_DST_PORT_FREE_REQ_TUNNEL_TYPE_GENEVE ); - } -if (test_and_clear_bit (BNXT_GENEVE_DEL_PORT_SP_EVENT , &bp->sp_event)) { -bnxt_hwrm_tunnel_dst_port_free ( - bp,TUNNEL_DST_PORT_FREE_REQ_TUNNEL_TYPE_GENEVE ); - }if (test_and_clear_bit (BNXT_PERIODIC_STATS_SP_EVENT , &bp->sp_event)) {bnxt_hwrm_port_qstats (bp);bnxt_hwrm_port_qstats_ext (bp);
@@ -10967,6 +10948,9 @@ static int bnxt_init_board(struct pci_dev *pdev, struct net_device *dev)timer_setup (&bp->timer, bnxt_timer,0 ); bp->current_interval =BNXT_TIMER_INTERVAL ; + bp->vxlan_fw_dst_port_id =INVALID_HW_RING_ID ; + bp->nge_fw_dst_port_id =INVALID_HW_RING_ID ; +clear_bit (BNXT_STATE_OPEN , &bp->state);return 0 ;
@@ -11294,84 +11278,33 @@ static void bnxt_cfg_ntp_filters(struct bnxt *bp)#endif /* CONFIG_RFS_ACCEL */ -static void bnxt_udp_tunnel_add (struct net_device *dev, -struct udp_tunnel_info *ti) +static int bnxt_udp_tunnel_sync (struct net_device *netdev,unsigned int table) { -struct bnxt *bp =netdev_priv (dev); - -if (ti->sa_family !=AF_INET6 && ti->sa_family !=AF_INET ) -return ; - -if (!netif_running (dev)) -return ; +struct bnxt *bp =netdev_priv (netdev); +struct udp_tunnel_info ti; +unsigned int cmd; -switch (ti->type) { -case UDP_TUNNEL_TYPE_VXLAN : -if (bp->vxlan_port_cnt && bp->vxlan_port != ti->port) -return ; +udp_tunnel_nic_get_port (netdev, table,0 , &ti); +if (ti.type ==UDP_TUNNEL_TYPE_VXLAN ) + cmd =TUNNEL_DST_PORT_FREE_REQ_TUNNEL_TYPE_VXLAN ; +else + cmd =TUNNEL_DST_PORT_FREE_REQ_TUNNEL_TYPE_GENEVE ; - bp->vxlan_port_cnt++; -if (bp->vxlan_port_cnt ==1 ) { - bp->vxlan_port = ti->port; -set_bit (BNXT_VXLAN_ADD_PORT_SP_EVENT , &bp->sp_event); -bnxt_queue_sp_work (bp); - } -break ; -case UDP_TUNNEL_TYPE_GENEVE : -if (bp->nge_port_cnt && bp->nge_port != ti->port) -return ; +if (ti.port) +return bnxt_hwrm_tunnel_dst_port_alloc (bp, ti.port, cmd); - bp->nge_port_cnt++; -if (bp->nge_port_cnt ==1 ) { - bp->nge_port = ti->port; -set_bit (BNXT_GENEVE_ADD_PORT_SP_EVENT , &bp->sp_event); - } -break ; -default : -return ; - } - -bnxt_queue_sp_work (bp); +return bnxt_hwrm_tunnel_dst_port_free (bp, cmd); } -static void bnxt_udp_tunnel_del (struct net_device *dev, -struct udp_tunnel_info *ti) -{ -struct bnxt *bp =netdev_priv (dev); - -if (ti->sa_family !=AF_INET6 && ti->sa_family !=AF_INET ) -return ; - -if (!netif_running (dev)) -return ; - -switch (ti->type) { -case UDP_TUNNEL_TYPE_VXLAN : -if (!bp->vxlan_port_cnt || bp->vxlan_port != ti->port) -return ; - bp->vxlan_port_cnt--; - -if (bp->vxlan_port_cnt !=0 ) -return ; - -set_bit (BNXT_VXLAN_DEL_PORT_SP_EVENT , &bp->sp_event); -break ; -case UDP_TUNNEL_TYPE_GENEVE : -if (!bp->nge_port_cnt || bp->nge_port != ti->port) -return ; - bp->nge_port_cnt--; - -if (bp->nge_port_cnt !=0 ) -return ; - -set_bit (BNXT_GENEVE_DEL_PORT_SP_EVENT , &bp->sp_event); -break ; -default : -return ; - } - -bnxt_queue_sp_work (bp); -} +static const struct udp_tunnel_nic_info bnxt_udp_tunnels = { + .sync_table = bnxt_udp_tunnel_sync, + .flags =UDP_TUNNEL_NIC_INFO_MAY_SLEEP | +UDP_TUNNEL_NIC_INFO_OPEN_ONLY , + .tables = { + { .n_entries =1 , .tunnel_types =UDP_TUNNEL_TYPE_VXLAN , }, + { .n_entries =1 , .tunnel_types =UDP_TUNNEL_TYPE_GENEVE , }, + }, +}; static int bnxt_bridge_getlink(struct sk_buff *skb, u32 pid, u32 seq, struct net_device *dev, u32 filter_mask,
@@ -11469,8 +11402,8 @@ static const struct net_device_ops bnxt_netdev_ops = { #ifdef CONFIG_RFS_ACCEL .ndo_rx_flow_steer = bnxt_rx_flow_steer, #endif - .ndo_udp_tunnel_add = bnxt_udp_tunnel_add, - .ndo_udp_tunnel_del = bnxt_udp_tunnel_del, + .ndo_udp_tunnel_add = udp_tunnel_nic_add_port, + .ndo_udp_tunnel_del = udp_tunnel_nic_del_port, .ndo_bpf = bnxt_xdp, .ndo_xdp_xmit = bnxt_xdp_xmit, .ndo_bridge_getlink = bnxt_bridge_getlink,
@@ -11958,6 +11891,8 @@ static int bnxt_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)NETIF_F_GSO_UDP_TUNNEL |NETIF_F_GSO_GRE |NETIF_F_GSO_UDP_TUNNEL_CSUM |NETIF_F_GSO_GRE_CSUM |NETIF_F_GSO_IPXIP4 |NETIF_F_GSO_PARTIAL ; + dev->udp_tunnel_nic_info = &bnxt_udp_tunnels; + dev->gso_partial_features =NETIF_F_GSO_UDP_TUNNEL_CSUM |NETIF_F_GSO_GRE_CSUM ; dev->vlan_features = dev->hw_features |NETIF_F_HIGHDMA ;
drivers/net/ethernet/broadcom/bnxt/bnxt.h +2 -10
@@ -1751,12 +1751,8 @@ struct bnxt { ((u64)(maj) << 48 | (u64)(min) << 32 | (u64)(bld) << 16 | (rsv))#define BNXT_FW_MAJ (bp) ((bp)->fw_ver_code >> 48) -__be16 vxlan_port; -u8 vxlan_port_cnt; -__le16 vxlan_fw_dst_port_id; -__be16 nge_port; -u8 nge_port_cnt; -__le16 nge_fw_dst_port_id; +u16 vxlan_fw_dst_port_id; +u16 nge_fw_dst_port_id;u8 port_partition_type;u8 port_count;u16 br_mode;
@@ -1776,16 +1772,12 @@ struct bnxt {#define BNXT_RX_NTP_FLTR_SP_EVENT 1#define BNXT_LINK_CHNG_SP_EVENT 2#define BNXT_HWRM_EXEC_FWD_REQ_SP_EVENT 3 -#define BNXT_VXLAN_ADD_PORT_SP_EVENT 4 -#define BNXT_VXLAN_DEL_PORT_SP_EVENT 5#define BNXT_RESET_TASK_SP_EVENT 6#define BNXT_RST_RING_SP_EVENT 7#define BNXT_HWRM_PF_UNLOAD_SP_EVENT 8#define BNXT_PERIODIC_STATS_SP_EVENT 9#define BNXT_HWRM_PORT_MODULE_SP_EVENT 10#define BNXT_RESET_TASK_SILENT_SP_EVENT 11 -#define BNXT_GENEVE_ADD_PORT_SP_EVENT 12 -#define BNXT_GENEVE_DEL_PORT_SP_EVENT 13#define BNXT_LINK_SPEED_CHNG_SP_EVENT 14#define BNXT_FLOW_STATS_SP_EVENT 15#define BNXT_UPDATE_PHY_SP_EVENT 16
Signature · 1 line
2.26.2
Jakub Kicinski
maintainerPATCH v4 6/10 selftests: net: add a test for UDP tunnel info infra
In reply to Jakub KicinskiAdd validating the UDP tunnel infra works.
$ ./udp_tunnel_nic.sh PASSED all 383 checks
--- .../drivers/net/netdevsim/udp_tunnel_nic.sh | 786 ++++++++++++++++++ 1 file changed, 786 insertions(+) create mode 100644 tools/testing/selftests/drivers/net/netdevsim/udp_tunnel_nic.sh
tools/testing/selftests/drivers/net/netdevsim/udp_tunnel_nic.sh new file +786 -0
@@ -0,0 +1,786 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0-only + +VNI_GEN=$RANDOM +NSIM_ID=$((RANDOM % 1024)) +NSIM_DEV_SYS=/sys/bus/netdevsim/devices/netdevsim$NSIM_ID +NSIM_DEV_DFS=/sys/kernel/debug/netdevsim/netdevsim$NSIM_ID +NSIM_NETDEV= +HAS_ETHTOOL= +EXIT_STATUS=0 +num_cases=0 +num_errors=0 + +clean_up_devs=( ) + +function err_cnt { + echo "ERROR:" $@ + EXIT_STATUS=1 + ((num_errors++)) + ((num_cases++)) +} + +function pass_cnt { + ((num_cases++)) +} + +function cleanup_tuns { + for dev in "${clean_up_devs[@]}"; do + [ -e /sys/class/net/$dev ] && ip link del dev $dev + done + clean_up_devs=( ) +} + +function cleanup_nsim { + if [ -e $NSIM_DEV_SYS ]; then + echo $NSIM_ID > /sys/bus/netdevsim/del_device + fi +} + +function cleanup { + cleanup_tuns + cleanup_nsim +} + +trap cleanup EXIT + +function new_vxlan { + local dev=$1 + local dstport=$2 + local lower=$3 + local ipver=$4 + local flags=$5 + + local group ipfl + + [ "$ipver" != '6' ] && group=239.1.1.1 || group=fff1::1 + [ "$ipver" != '6' ] || ipfl="-6" + + [[ ! "$flags" =~ "external" ]] && flags="$flags id $((VNI_GEN++))" + + ip $ipfl link add $dev type vxlan \ + group $group \ + dev $lower \ + dstport $dstport \ + $flags + + ip link set dev $dev up + + clean_up_devs=("${clean_up_devs[@]}" $dev) + + check_tables +} + +function new_geneve { + local dev=$1 + local dstport=$2 + local ipver=$3 + local flags=$4 + + local group ipfl + + [ "$ipver" != '6' ] && remote=1.1.1.2 || group=::2 + [ "$ipver" != '6' ] || ipfl="-6" + + [[ ! "$flags" =~ "external" ]] && flags="$flags vni $((VNI_GEN++))" + + ip $ipfl link add $dev type geneve \ + remote $remote \ + dstport $dstport \ + $flags + + ip link set dev $dev up + + clean_up_devs=("${clean_up_devs[@]}" $dev) + + check_tables +} + +function del_dev { + local dev=$1 + + ip link del dev $dev + check_tables +} + +# Helpers for netdevsim port/type encoding +function mke { + local port=$1 + local type=$2 + + echo $((port << 16 | type)) +} + +function pre { + local val=$1 + + echo -e "port: $((val >> 16))\ttype: $((val & 0xffff))" +} + +function pre_ethtool { + local val=$1 + local port=$((val >> 16)) + local type=$((val & 0xffff)) + + case $type in + 1) + type_name="vxlan" + ;; + 2) + type_name="geneve" + ;; + 4) + type_name="vxlan-gpe" + ;; + *) + type_name="bit X" + ;; + esac + + echo "port $port, $type_name" +} + +function check_table { + local path=$NSIM_DEV_DFS/ports/$port/udp_ports_table$1 + local -n expected=$2 + local last=$3 + + read -a have < $path + + if [ ${#expected[@]} -ne ${#have[@]} ]; then + echo "check_table: BAD NUMBER OF ITEMS" + return 0 + fi + + for i in "${!expected[@]}"; do + if [ -n "$HAS_ETHTOOL" -a ${expected[i]} -ne 0 ]; then + pp_expected=`pre_ethtool ${expected[i]}` + ethtool --show-tunnels $NSIM_NETDEV | grep "$pp_expected" >/dev/null + if [ $? -ne 0 -a $last -ne 0 ]; then + err_cnt "ethtool table $1 on port $port: $pfx - $msg" + echo " check_table: ethtool does not contain '$pp_expected'" + ethtool --show-tunnels $NSIM_NETDEV + return 0 + + fi + fi + + if [ ${expected[i]} != ${have[i]} ]; then + if [ $last -ne 0 ]; then + err_cnt "table $1 on port $port: $pfx - $msg" + echo " check_table: wrong entry $i" + echo " expected: `pre ${expected[i]}`" + echo " have: `pre ${have[i]}`" + return 0 + fi + return 1 + fi + done + + pass_cnt + return 0 +} + +function check_tables { + # Need retries in case we have workqueue making the changes + local retries=10 + + while ! check_table 0 exp0 $((retries == 0)); do + sleep 0.02 + ((retries--)) + done + while ! check_table 1 exp1 $((retries == 0)); do + sleep 0.02 + ((retries--)) + done +} + +function print_table { + local path=$NSIM_DEV_DFS/ports/$port/udp_ports_table$1 + read -a have < $path + + tree $NSIM_DEV_DFS/ + + echo "Port $port table $1:" + + for i in "${!have[@]}"; do + echo " `pre ${have[i]}`" + done + +} + +function print_tables { + print_table 0 + print_table 1 +} + +function get_netdev_name { + local -n old=$1 + + new=$(ls /sys/class/net) + + for netdev in $new; do + for check in $old; do + [ $netdev == $check ] && break + done + + if [ $netdev != $check ]; then + echo $netdev + break + fi + done +} + +### +### Code start +### + +# Probe ethtool support +ethtool -h | grep show-tunnels 2>&1 >/dev/null && HAS_ETHTOOL=y + +modprobe netdevsim + +# Basic test +pfx="basic" + +for port in 0 1; do + old_netdevs=$(ls /sys/class/net) + if [ $port -eq 0 ]; then + echo $NSIM_ID > /sys/bus/netdevsim/new_device + else + echo 1 > $NSIM_DEV_DFS/udp_ports_open_only + echo 1 > $NSIM_DEV_DFS/udp_ports_sleep + echo 1 > $NSIM_DEV_SYS/new_port + fi + NSIM_NETDEV=`get_netdev_name old_netdevs` + + msg="new NIC device created" + exp0=( 0 0 0 0 ) + exp1=( 0 0 0 0 ) + check_tables + + msg="VxLAN v4 devices" + exp0=( `mke 4789 1` 0 0 0 ) + new_vxlan vxlan0 4789 $NSIM_NETDEV + new_vxlan vxlan1 4789 $NSIM_NETDEV + + msg="VxLAN v4 devices go down" + exp0=( 0 0 0 0 ) + ifconfig vxlan1 down + ifconfig vxlan0 down + check_tables + + msg="VxLAN v6 devices" + exp0=( `mke 4789 1` 0 0 0 ) + new_vxlan vxlanA 4789 $NSIM_NETDEV 6 + + for ifc in vxlan0 vxlan1; do + ifconfig $ifc up + done + + new_vxlan vxlanB 4789 $NSIM_NETDEV 6 + + msg="another VxLAN v6 devices" + exp0=( `mke 4789 1` `mke 4790 1` 0 0 ) + new_vxlan vxlanC 4790 $NSIM_NETDEV 6 + + msg="Geneve device" + exp1=( `mke 6081 2` 0 0 0 ) + new_geneve gnv0 6081 + + msg="NIC device goes down" + ifconfig $NSIM_NETDEV down + if [ $port -eq 1 ]; then + exp0=( 0 0 0 0 ) + exp1=( 0 0 0 0 ) + fi + check_tables + msg="NIC device goes up again" + ifconfig $NSIM_NETDEV up + exp0=( `mke 4789 1` `mke 4790 1` 0 0 ) + exp1=( `mke 6081 2` 0 0 0 ) + check_tables + + cleanup_tuns + + msg="tunnels destroyed" + exp0=( 0 0 0 0 ) + exp1=( 0 0 0 0 ) + check_tables + + modprobe -r geneve + modprobe -r vxlan + modprobe -r udp_tunnel + + check_tables +done + +modprobe -r netdevsim + +# Module tests +pfx="module tests" + +if modinfo netdevsim | grep udp_tunnel >/dev/null; then + err_cnt "netdevsim depends on udp_tunnel" +else + pass_cnt +fi + +modprobe netdevsim + +old_netdevs=$(ls /sys/class/net) +port=0 +echo $NSIM_ID > /sys/bus/netdevsim/new_device +echo 0 > $NSIM_DEV_SYS/del_port +echo 1000 > $NSIM_DEV_DFS/udp_ports_sleep +echo 0 > $NSIM_DEV_SYS/new_port +NSIM_NETDEV=`get_netdev_name old_netdevs` + +msg="create VxLANs" +exp0=( 0 0 0 0 ) # sleep is longer than out wait +new_vxlan vxlan0 10000 $NSIM_NETDEV + +modprobe -r vxlan +modprobe -r udp_tunnel + +msg="remove tunnels" +exp0=( 0 0 0 0 ) +check_tables + +msg="create VxLANs" +exp0=( 0 0 0 0 ) # sleep is longer than out wait +new_vxlan vxlan0 10000 $NSIM_NETDEV + +exp0=( 0 0 0 0 ) + +modprobe -r netdevsim +modprobe netdevsim + +# Overflow the table + +function overflow_table0 { + local pfx=$1 + + msg="create VxLANs 1/5" + exp0=( `mke 10000 1` 0 0 0 ) + new_vxlan vxlan0 10000 $NSIM_NETDEV + + msg="create VxLANs 2/5" + exp0=( `mke 10000 1` `mke 10001 1` 0 0 ) + new_vxlan vxlan1 10001 $NSIM_NETDEV + + msg="create VxLANs 3/5" + exp0=( `mke 10000 1` `mke 10001 1` `mke 10002 1` 0 ) + new_vxlan vxlan2 10002 $NSIM_NETDEV + + msg="create VxLANs 4/5" + exp0=( `mke 10000 1` `mke 10001 1` `mke 10002 1` `mke 10003 1` ) + new_vxlan vxlan3 10003 $NSIM_NETDEV + + msg="create VxLANs 5/5" + new_vxlan vxlan4 10004 $NSIM_NETDEV +} + +function overflow_table1 { + local pfx=$1 + + msg="create GENEVE 1/5" + exp1=( `mke 20000 2` 0 0 0 ) + new_geneve gnv0 20000 + + msg="create GENEVE 2/5" + exp1=( `mke 20000 2` `mke 20001 2` 0 0 ) + new_geneve gnv1 20001 + + msg="create GENEVE 3/5" + exp1=( `mke 20000 2` `mke 20001 2` `mke 20002 2` 0 ) + new_geneve gnv2 20002 + + msg="create GENEVE 4/5" + exp1=( `mke 20000 2` `mke 20001 2` `mke 20002 2` `mke 20003 2` ) + new_geneve gnv3 20003 + + msg="create GENEVE 5/5" + new_geneve gnv4 20004 +} + +echo $NSIM_ID > /sys/bus/netdevsim/new_device +echo 0 > $NSIM_DEV_SYS/del_port + +for port in 0 1; do + if [ $port -ne 0 ]; then + echo 1 > $NSIM_DEV_DFS/udp_ports_open_only + echo 1 > $NSIM_DEV_DFS/udp_ports_sleep + fi + + echo $port > $NSIM_DEV_SYS/new_port + ifconfig $NSIM_NETDEV up + + overflow_table0 "overflow NIC table" + overflow_table1 "overflow NIC table" + + msg="replace VxLAN in overflow table" + exp0=( `mke 10000 1` `mke 10004 1` `mke 10002 1` `mke 10003 1` ) + del_dev vxlan1 + + msg="vacate VxLAN in overflow table" + exp0=( `mke 10000 1` `mke 10004 1` 0 `mke 10003 1` ) + del_dev vxlan2 + + msg="replace GENEVE in overflow table" + exp1=( `mke 20000 2` `mke 20004 2` `mke 20002 2` `mke 20003 2` ) + del_dev gnv1 + + msg="vacate GENEVE in overflow table" + exp1=( `mke 20000 2` `mke 20004 2` 0 `mke 20003 2` ) + del_dev gnv2 + + msg="table sharing - share" + exp1=( `mke 20000 2` `mke 20004 2` `mke 30001 4` `mke 20003 2` ) + new_vxlan vxlanG0 30001 $NSIM_NETDEV 4 "gpe external" + + msg="table sharing - overflow" + new_vxlan vxlanG1 30002 $NSIM_NETDEV 4 "gpe external" + msg="table sharing - overflow v6" + new_vxlan vxlanG2 30002 $NSIM_NETDEV 6 "gpe external" + + exp1=( `mke 20000 2` `mke 30002 4` `mke 30001 4` `mke 20003 2` ) + del_dev gnv4 + + msg="destroy NIC" + echo $port > $NSIM_DEV_SYS/del_port + + cleanup_tuns + exp0=( 0 0 0 0 ) + exp1=( 0 0 0 0 ) +done + +cleanup_nsim + +# Sync all +pfx="sync all" + +echo $NSIM_ID > /sys/bus/netdevsim/new_device +echo 0 > $NSIM_DEV_SYS/del_port +echo 1 > $NSIM_DEV_DFS/udp_ports_sync_all + +for port in 0 1; do + if [ $port -ne 0 ]; then + echo 1 > $NSIM_DEV_DFS/udp_ports_open_only + echo 1 > $NSIM_DEV_DFS/udp_ports_sleep + fi + + echo $port > $NSIM_DEV_SYS/new_port + ifconfig $NSIM_NETDEV up + + overflow_table0 "overflow NIC table" + overflow_table1 "overflow NIC table" + + msg="replace VxLAN in overflow table" + exp0=( `mke 10000 1` `mke 10004 1` `mke 10002 1` `mke 10003 1` ) + del_dev vxlan1 + + msg="vacate VxLAN in overflow table" + exp0=( `mke 10000 1` `mke 10004 1` 0 `mke 10003 1` ) + del_dev vxlan2 + + msg="replace GENEVE in overflow table" + exp1=( `mke 20000 2` `mke 20004 2` `mke 20002 2` `mke 20003 2` ) + del_dev gnv1 + + msg="vacate GENEVE in overflow table" + exp1=( `mke 20000 2` `mke 20004 2` 0 `mke 20003 2` ) + del_dev gnv2 + + msg="table sharing - share" + exp1=( `mke 20000 2` `mke 20004 2` `mke 30001 4` `mke 20003 2` ) + new_vxlan vxlanG0 30001 $NSIM_NETDEV 4 "gpe external" + + msg="table sharing - overflow" + new_vxlan vxlanG1 30002 $NSIM_NETDEV 4 "gpe external" + msg="table sharing - overflow v6" + new_vxlan vxlanG2 30002 $NSIM_NETDEV 6 "gpe external" + + exp1=( `mke 20000 2` `mke 30002 4` `mke 30001 4` `mke 20003 2` ) + del_dev gnv4 + + msg="destroy NIC" + echo $port > $NSIM_DEV_SYS/del_port + + cleanup_tuns + exp0=( 0 0 0 0 ) + exp1=( 0 0 0 0 ) +done + +cleanup_nsim + +# Destroy full NIC +pfx="destroy full" + +echo $NSIM_ID > /sys/bus/netdevsim/new_device +echo 0 > $NSIM_DEV_SYS/del_port + +for port in 0 1; do + if [ $port -ne 0 ]; then + echo 1 > $NSIM_DEV_DFS/udp_ports_open_only + echo 1 > $NSIM_DEV_DFS/udp_ports_sleep + fi + + echo $port > $NSIM_DEV_SYS/new_port + ifconfig $NSIM_NETDEV up + + overflow_table0 "destroy NIC" + overflow_table1 "destroy NIC" + + msg="destroy NIC" + echo $port > $NSIM_DEV_SYS/del_port + + cleanup_tuns + exp0=( 0 0 0 0 ) + exp1=( 0 0 0 0 ) +done + +cleanup_nsim + +# IPv4 only +pfx="IPv4 only" + +echo $NSIM_ID > /sys/bus/netdevsim/new_device +echo 0 > $NSIM_DEV_SYS/del_port +echo 1 > $NSIM_DEV_DFS/udp_ports_ipv4_only + +for port in 0 1; do + if [ $port -ne 0 ]; then + echo 1 > $NSIM_DEV_DFS/udp_ports_open_only + echo 1 > $NSIM_DEV_DFS/udp_ports_sleep + fi + + echo $port > $NSIM_DEV_SYS/new_port + ifconfig $NSIM_NETDEV up + + msg="create VxLANs v6" + new_vxlan vxlanA0 10000 $NSIM_NETDEV 6 + + msg="create VxLANs v6" + new_vxlan vxlanA1 10000 $NSIM_NETDEV 6 + + ip link set dev vxlanA0 down + ip link set dev vxlanA0 up + check_tables + + msg="create VxLANs v4" + exp0=( `mke 10000 1` 0 0 0 ) + new_vxlan vxlan0 10000 $NSIM_NETDEV + + msg="down VxLANs v4" + exp0=( 0 0 0 0 ) + ip link set dev vxlan0 down + check_tables + + msg="up VxLANs v4" + exp0=( `mke 10000 1` 0 0 0 ) + ip link set dev vxlan0 up + check_tables + + msg="destroy VxLANs v4" + exp0=( 0 0 0 0 ) + del_dev vxlan0 + + msg="recreate VxLANs v4" + exp0=( `mke 10000 1` 0 0 0 ) + new_vxlan vxlan0 10000 $NSIM_NETDEV + + del_dev vxlanA0 + del_dev vxlanA1 + + msg="destroy NIC" + echo $port > $NSIM_DEV_SYS/del_port + + cleanup_tuns + exp0=( 0 0 0 0 ) + exp1=( 0 0 0 0 ) +done + +cleanup_nsim + +# Failures +pfx="error injection" + +echo $NSIM_ID > /sys/bus/netdevsim/new_device +echo 0 > $NSIM_DEV_SYS/del_port + +for port in 0 1; do + if [ $port -ne 0 ]; then + echo 1 > $NSIM_DEV_DFS/udp_ports_open_only + echo 1 > $NSIM_DEV_DFS/udp_ports_sleep + fi + + echo $port > $NSIM_DEV_SYS/new_port + ifconfig $NSIM_NETDEV up + + echo 110 > $NSIM_DEV_DFS/ports/$port/udp_ports_inject_error + + msg="1 - create VxLANs v6" + exp0=( 0 0 0 0 ) + new_vxlan vxlanA0 10000 $NSIM_NETDEV 6 + + msg="1 - create VxLANs v4" + exp0=( `mke 10000 1` 0 0 0 ) + new_vxlan vxlan0 10000 $NSIM_NETDEV + + msg="1 - remove VxLANs v4" + del_dev vxlan0 + + msg="1 - remove VxLANs v6" + exp0=( 0 0 0 0 ) + del_dev vxlanA0 + + msg="2 - create GENEVE" + exp1=( `mke 20000 2` 0 0 0 ) + new_geneve gnv0 20000 + + msg="2 - destroy GENEVE" + echo 2 > $NSIM_DEV_DFS/ports/$port/udp_ports_inject_error + exp1=( `mke 20000 2` 0 0 0 ) + del_dev gnv0 + + msg="2 - create second GENEVE" + exp1=( 0 `mke 20001 2` 0 0 ) + new_geneve gnv0 20001 + + msg="destroy NIC" + echo $port > $NSIM_DEV_SYS/del_port + + cleanup_tuns + exp0=( 0 0 0 0 ) + exp1=( 0 0 0 0 ) +done + +cleanup_nsim + +# netdev flags +pfx="netdev flags" + +echo $NSIM_ID > /sys/bus/netdevsim/new_device +echo 0 > $NSIM_DEV_SYS/del_port + +for port in 0 1; do + if [ $port -ne 0 ]; then + echo 1 > $NSIM_DEV_DFS/udp_ports_open_only + echo 1 > $NSIM_DEV_DFS/udp_ports_sleep + fi + + echo $port > $NSIM_DEV_SYS/new_port + ifconfig $NSIM_NETDEV up + + msg="create VxLANs v6" + exp0=( `mke 10000 1` 0 0 0 ) + new_vxlan vxlanA0 10000 $NSIM_NETDEV 6 + + msg="create VxLANs v4" + new_vxlan vxlan0 10000 $NSIM_NETDEV + + msg="turn off" + exp0=( 0 0 0 0 ) + ethtool -K $NSIM_NETDEV rx-udp_tunnel-port-offload off + check_tables + + msg="turn on" + exp0=( `mke 10000 1` 0 0 0 ) + ethtool -K $NSIM_NETDEV rx-udp_tunnel-port-offload on + check_tables + + msg="remove both" + del_dev vxlanA0 + exp0=( 0 0 0 0 ) + del_dev vxlan0 + check_tables + + ethtool -K $NSIM_NETDEV rx-udp_tunnel-port-offload off + + msg="create VxLANs v4 - off" + exp0=( 0 0 0 0 ) + new_vxlan vxlan0 10000 $NSIM_NETDEV + + msg="created off - turn on" + exp0=( `mke 10000 1` 0 0 0 ) + ethtool -K $NSIM_NETDEV rx-udp_tunnel-port-offload on + check_tables + + msg="destroy NIC" + echo $port > $NSIM_DEV_SYS/del_port + + cleanup_tuns + exp0=( 0 0 0 0 ) + exp1=( 0 0 0 0 ) +done + +cleanup_nsim + +# device initiated reset +pfx="reset notification" + +echo $NSIM_ID > /sys/bus/netdevsim/new_device +echo 0 > $NSIM_DEV_SYS/del_port + +for port in 0 1; do + if [ $port -ne 0 ]; then + echo 1 > $NSIM_DEV_DFS/udp_ports_open_only + echo 1 > $NSIM_DEV_DFS/udp_ports_sleep + fi + + echo $port > $NSIM_DEV_SYS/new_port + ifconfig $NSIM_NETDEV up + + msg="create VxLANs v6" + exp0=( `mke 10000 1` 0 0 0 ) + new_vxlan vxlanA0 10000 $NSIM_NETDEV 6 + + msg="create VxLANs v4" + new_vxlan vxlan0 10000 $NSIM_NETDEV + + echo 1 > $NSIM_DEV_DFS/ports/$port/udp_ports_reset + check_tables + + msg="NIC device goes down" + ifconfig $NSIM_NETDEV down + if [ $port -eq 1 ]; then + exp0=( 0 0 0 0 ) + exp1=( 0 0 0 0 ) + fi + check_tables + + echo 1 > $NSIM_DEV_DFS/ports/$port/udp_ports_reset + check_tables + + msg="NIC device goes up again" + ifconfig $NSIM_NETDEV up + exp0=( `mke 10000 1` 0 0 0 ) + check_tables + + msg="remove both" + del_dev vxlanA0 + exp0=( 0 0 0 0 ) + del_dev vxlan0 + check_tables + + echo 1 > $NSIM_DEV_DFS/ports/$port/udp_ports_reset + check_tables + + msg="destroy NIC" + echo $port > $NSIM_DEV_SYS/del_port + + cleanup_tuns + exp0=( 0 0 0 0 ) + exp1=( 0 0 0 0 ) +done + +modprobe -r netdevsim + +if [ $num_errors -eq 0 ]; then + echo "PASSED all $num_cases checks" +else + echo "FAILED $num_errors/$num_cases checks" +fi + +exit $EXIT_STATUS
Signature · 1 line
2.26.2
Jakub Kicinski
maintainerPATCH v4 7/10 ixgbe: don't clear UDP tunnel ports when RXCSUM is disabled
In reply to Jakub KicinskiIt appears the clearing of UDP tunnel ports when RXCSUM is disabled is unnecessary. Driver will not pay attention to checksum bits if RXCSUM is not set, so we can let the hardware parse the packets.
Note that the UDP tunnel port NDO handlers don't pay attention to the state of RXCSUM, so the ports could had been re-programmed, anyway.
This cleanup simplifies later conversion patch.
v2:
- break this out of the following patch.
--- drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 20 ------------------- 1 file changed, 20 deletions(-)
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +0 -20
@@ -9784,26 +9784,6 @@ static int ixgbe_set_features(struct net_device *netdev, netdev->features = features; -if ((adapter->flags &IXGBE_FLAG_VXLAN_OFFLOAD_CAPABLE )) { -if (features &NETIF_F_RXCSUM ) { - adapter->flags2 |=IXGBE_FLAG2_UDP_TUN_REREG_NEEDED ; - }else { -u32 port_mask =IXGBE_VXLANCTRL_VXLAN_UDPPORT_MASK ; - -ixgbe_clear_udp_tunnel_port (adapter, port_mask); - } - } - -if ((adapter->flags &IXGBE_FLAG_GENEVE_OFFLOAD_CAPABLE )) { -if (features &NETIF_F_RXCSUM ) { - adapter->flags2 |=IXGBE_FLAG2_UDP_TUN_REREG_NEEDED ; - }else { -u32 port_mask =IXGBE_VXLANCTRL_GENEVE_UDPPORT_MASK ; - -ixgbe_clear_udp_tunnel_port (adapter, port_mask); - } - } -if ((changed &NETIF_F_HW_L2FW_DOFFLOAD ) && adapter->num_rx_pools >1 )ixgbe_reset_l2fw_offload (adapter);else if (need_reset)
Signature · 1 line
2.26.2
Re: [PATCH net-next v4 09/10] bnxt: convert to new udp_tunnel_nic infra
In reply to Jakub KicinskiQuoted 8 lines · Jakub Kicinski
Convert to new infra, taking advantage of sleeping in callbacks.
v2:
- use bp->*_fw_dst_port_id !=INVALID_HW_RING_IDas indication that the offload is active.Signed-off-by Jakub Kicinski <[email protected]>
Thanks.
David Miller
maintainerFrom: Jakub Kicinski <[email protected]>
Date: Thu, 9 Jul 2020 17:42:43 -0700
...
Quoted 9 lines
This work tries to improve the situation by pulling the UDP tunnel port table maintenance out of the drivers. It turns out that almost all drivers maintain a fixed size table of ports (in most cases one per tunnel type), so we can take care of all the refcounting in the core, and let the driver specify if they need to sleep in the callbacks or not. The new common implementation will also support replacing ports - when a port is removed from a full table it will try to find a previously missing port to take its place.
Quoted 8 lines
This patch only implements the core functionality along with a few drivers I was hoping to test manually [1] along with a test based on a netdevsim implementation. Following patches will convert all the drivers. Once that's complete we can remove the ndos, and rely directly on the new infrastrucutre.
Then after RSS (RXFH) is converted to netlink we can add the ability to configure the use of inner RSS headers for UDP tunnels.
...
Series applied, thanks Jakub.