Skip to contentSkip to conversation

ionic: Add ionic driver

PATCH Superseded Shannon Nelson 4 participants 37 messages Aug 26, 2019 – Aug 28, 2019

Subsystems (4)

v5 of 7 Superseded Latest is v7 →

lore
Share & tools
lore
#0 lore raw

v5 cover ionic: Add ionic driver

This is a patch series that adds the ionic driver, supporting the Pensando ethernet device.

In this initial patchset we implement basic transmit and receive. Later patchsets will add more advanced features.

Our thanks to Saeed Mahameed, David Miller, Andrew Lunn, Michal Kubecek, Jacub Kicinski, Jiri Pirko, and the ever present kbuild test robots for their comments and suggestions.

New in v5:
- code reorganized for more sane layout, with a side benefit of getting rid of a "defined but not used" complaint after patch 5
- added "ionic_" prefix to struct definitions and fixed up remaining reverse christmas tree formatting (I think I got them all...)
- ndo_open and ndo_stop reworked for better error recovery
- interrupt coalescing enabled at driver start
- unnecessary log messaging removed from events
- double copy added in the module prom read to assure a clean copy
- added BQL counting
- fixed a TSO unmap issue found in testing
- generalize a bit-flag wait with timeout
- added devlink into earlier code and dropped patch 19

New in v4:
- use devlink struct alloc for ionic device specific struct
- add support for devlink_port
- fixup devlink fixed vs running version usage
- use bitmap_copy() instead of memcpy() for link_ksettings
- don't bother to zero out the advertising bits before copying in the support bits
- drop unknown xcvr types (will be expanded on later)
- flap the connection to force auto-negotiation
- use is_power_of_2() rather than open code
- simplify set/get_pauseparam use of pause->autoneg
- add a couple comments about NIC status data updated in DMA spaces

New in v3:
- use le32_to_cpu() on queue_count[] values in debugfs
- dma_free_coherent() can handle NULL pointers
- remove unused SS_TEST from ethtool handlers
- one more case of stop the tx ring if there is no room
- remove a couple of stray // comments

New in v2:
- removed debugfs error checking and cut down on debugfs use
- remove redundant bounds checking on incoming values for mtu and ethtool
- don't alloc rx_filter memory until the match type has been checked
- free the ionic struct on remove
- simplified link_up and netif_carrier_ok comparison
- put stats into ethtool -S, out of debugfs
- moved dev_cmd and dev_info dumping to ethtool -d, out of debugfs
- added devlink support
- used kernel's rss init routines rather than open code
- set the Kbuild dependant on 64BIT
- cut down on some unnecessary log messaging
- cleaned up ionic_get_link_ksettings
- cleaned up other little code bits here and there

Shannon Nelson (18):
ionic: Add basic framework for IONIC Network device driver
ionic: Add hardware init and device commands
ionic: Add port management commands
ionic: Add basic lif support
ionic: Add interrupts and doorbells
ionic: Add basic adminq support
ionic: Add adminq action
ionic: Add notifyq support
ionic: Add the basic NDO callbacks for netdev support
ionic: Add management of rx filters
ionic: Add Rx filter and rx_mode ndo support
ionic: Add async link status check and basic stats
ionic: Add initial ethtool support
ionic: Add Tx and Rx handling
ionic: Add netdev-event handling
ionic: Add driver stats
ionic: Add RSS support
ionic: Add coalesce and other features

 .../networking/device_drivers/index.rst       |    1 +
 .../device_drivers/pensando/ionic.rst         |   43 +
 MAINTAINERS                                   |    8 +
 drivers/net/ethernet/Kconfig                  |    1 +
 drivers/net/ethernet/Makefile                 |    1 +
 drivers/net/ethernet/pensando/Kconfig         |   32 +
 drivers/net/ethernet/pensando/Makefile        |    6 +
 drivers/net/ethernet/pensando/ionic/Makefile  |    8 +
 drivers/net/ethernet/pensando/ionic/ionic.h   |   74 +
 .../net/ethernet/pensando/ionic/ionic_bus.h   |   16 +
 .../ethernet/pensando/ionic/ionic_bus_pci.c   |  292 ++
 .../ethernet/pensando/ionic/ionic_debugfs.c   |  269 ++
 .../ethernet/pensando/ionic/ionic_debugfs.h   |   34 +
 .../net/ethernet/pensando/ionic/ionic_dev.c   |  524 ++++
 .../net/ethernet/pensando/ionic/ionic_dev.h   |  306 ++
 .../ethernet/pensando/ionic/ionic_devlink.c   |   95 +
 .../ethernet/pensando/ionic/ionic_devlink.h   |   14 +
 .../ethernet/pensando/ionic/ionic_ethtool.c   |  780 ++++++
 .../ethernet/pensando/ionic/ionic_ethtool.h   |    9 +
 .../net/ethernet/pensando/ionic/ionic_if.h    | 2482 +++++++++++++++++
 .../net/ethernet/pensando/ionic/ionic_lif.c   | 2266 +++++++++++++++
 .../net/ethernet/pensando/ionic/ionic_lif.h   |  302 ++
 .../net/ethernet/pensando/ionic/ionic_main.c  |  554 ++++
 .../net/ethernet/pensando/ionic/ionic_regs.h  |  136 +
 .../ethernet/pensando/ionic/ionic_rx_filter.c |  150 +
 .../ethernet/pensando/ionic/ionic_rx_filter.h |   35 +
 .../net/ethernet/pensando/ionic/ionic_stats.c |  334 +++
 .../net/ethernet/pensando/ionic/ionic_stats.h |   53 +
 .../net/ethernet/pensando/ionic/ionic_txrx.c  |  894 ++++++
 .../net/ethernet/pensando/ionic/ionic_txrx.h  |   15 +
 30 files changed, 9734 insertions(+)
 create mode 100644 Documentation/networking/device_drivers/pensando/ionic.rst
 create mode 100644 drivers/net/ethernet/pensando/Kconfig
 create mode 100644 drivers/net/ethernet/pensando/Makefile
 create mode 100644 drivers/net/ethernet/pensando/ionic/Makefile
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic.h
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_bus.h
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_debugfs.c
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_debugfs.h
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_dev.c
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_dev.h
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_devlink.c
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_devlink.h
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_ethtool.c
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_ethtool.h
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_if.h
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_lif.c
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_lif.h
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_main.c
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_regs.h
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_rx_filter.c
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_rx_filter.h
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_stats.c
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_stats.h
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_txrx.c
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_txrx.h
Signature · 1 line
2.17.1
#1 lore raw

PATCH v5 1/18 ionic: Add basic framework for IONIC Network device driver

This patch adds a basic driver framework for the Pensando IONIC network device. There is no functionality right now other than the ability to load and unload.

---
 .../networking/device_drivers/index.rst       |  1 +
 .../device_drivers/pensando/ionic.rst         | 43 ++++++++++++++
 MAINTAINERS                                   |  8 +++
 drivers/net/ethernet/Kconfig                  |  1 +
 drivers/net/ethernet/Makefile                 |  1 +
 drivers/net/ethernet/pensando/Kconfig         | 32 ++++++++++
 drivers/net/ethernet/pensando/Makefile        |  6 ++
 drivers/net/ethernet/pensando/ionic/Makefile  |  6 ++
 drivers/net/ethernet/pensando/ionic/ionic.h   | 26 ++++++++
 .../net/ethernet/pensando/ionic/ionic_bus.h   | 10 ++++
 .../ethernet/pensando/ionic/ionic_bus_pci.c   | 59 +++++++++++++++++++
 .../ethernet/pensando/ionic/ionic_devlink.c   | 43 ++++++++++++++
 .../ethernet/pensando/ionic/ionic_devlink.h   | 12 ++++
 .../net/ethernet/pensando/ionic/ionic_main.c  | 32 ++++++++++
 14 files changed, 280 insertions(+)
 create mode 100644 Documentation/networking/device_drivers/pensando/ionic.rst
 create mode 100644 drivers/net/ethernet/pensando/Kconfig
 create mode 100644 drivers/net/ethernet/pensando/Makefile
 create mode 100644 drivers/net/ethernet/pensando/ionic/Makefile
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic.h
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_bus.h
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_devlink.c
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_devlink.h
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_main.c
Documentation/networking/device_drivers/index.rst +1 -0
index 2b7fefe72351..57fec66c5419 100644
@@ -23,6 +23,7 @@ Contents:
    intel/ice
    google/gve
    mellanox/mlx5
+   pensando/ionic
 
 .. only::  subproject
 
Documentation/networking/device_drivers/pensando/ionic.rst new file +43 -0

43 lines not on this page · raw message

MAINTAINERS +8 -0
index e352550a6895..be167e04bb5e 100644
@@ -12574,6 +12574,14 @@ L:	[email protected]
 S:	Maintained
 F:	drivers/platform/x86/peaq-wmi.c
 
+PENSANDO ETHERNET DRIVERS
+M:	Shannon Nelson <[email protected]>
+M:	Pensando Drivers <[email protected]>
+L:	[email protected]
+S:	Supported
+F:	Documentation/networking/device_drivers/pensando/ionic.rst
+F:	drivers/net/ethernet/pensando/
+
 PER-CPU MEMORY ALLOCATOR
 M:	Dennis Zhou <[email protected]>
 M:	Tejun Heo <[email protected]>
drivers/net/ethernet/Kconfig +1 -0
index 93a2d4deb27c..2830dc283ce6 100644
@@ -168,6 +168,7 @@ config ETHOC
 
 source "drivers/net/ethernet/packetengines/Kconfig"
 source "drivers/net/ethernet/pasemi/Kconfig"
+source "drivers/net/ethernet/pensando/Kconfig"
 source "drivers/net/ethernet/qlogic/Kconfig"
 source "drivers/net/ethernet/qualcomm/Kconfig"
 source "drivers/net/ethernet/rdc/Kconfig"
drivers/net/ethernet/Makefile +1 -0
index fb9155cffcff..061edd22f507 100644
@@ -97,3 +97,4 @@ obj-$(CONFIG_NET_VENDOR_WIZNET) += wiznet/
 obj-$(CONFIG_NET_VENDOR_XILINX) += xilinx/
 obj-$(CONFIG_NET_VENDOR_XIRCOM) += xircom/
 obj-$(CONFIG_NET_VENDOR_SYNOPSYS) += synopsys/
+obj-$(CONFIG_NET_VENDOR_PENSANDO) += pensando/
drivers/net/ethernet/pensando/Kconfig new file +32 -0
new file mode 100644
index 000000000000..5ea570be8379
@@ -0,0 +1,32 @@
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2019 Pensando Systems, Inc
+#
+# Pensando device configuration
+#
+
+config NET_VENDOR_PENSANDO
+	bool "Pensando devices"
+	default y
+	help
+	  If you have a network (Ethernet) card belonging to this class, say Y.
+
+	  Note that the answer to this question doesn't directly affect the
+	  kernel: saying N will just cause the configurator to skip all
+	  the questions about Pensando cards. If you say Y, you will be asked
+	  for your specific card in the following questions.
+
+if NET_VENDOR_PENSANDO
+
+config IONIC
+	tristate "Pensando Ethernet IONIC Support"
+	depends on 64BIT && PCI
+	help
+	  This enables the support for the Pensando family of Ethernet
+	  adapters.  More specific information on this driver can be
+	  found in
+	  <file:Documentation/networking/device_drivers/pensando/ionic.rst>.
+
+          To compile this driver as a module, choose M here. The module
+          will be called ionic.
+
+endif # NET_VENDOR_PENSANDO
drivers/net/ethernet/pensando/Makefile new file +6 -0
new file mode 100644
index 000000000000..21ce7499c122
@@ -0,0 +1,6 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Makefile for the Pensando network device drivers.
+#
+
+obj-$(CONFIG_IONIC) += ionic/
drivers/net/ethernet/pensando/ionic/Makefile new file +6 -0
new file mode 100644
index 000000000000..f174e8f7bce1
@@ -0,0 +1,6 @@
+# SPDX-License-Identifier: GPL-2.0
+# Copyright(c) 2017 - 2019 Pensando Systems, Inc
+
+obj-$(CONFIG_IONIC) := ionic.o
+
+ionic-y := ionic_main.o ionic_bus_pci.o ionic_devlink.o
drivers/net/ethernet/pensando/ionic/ionic.h new file +26 -0
new file mode 100644
index 000000000000..d40077161214
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+
+#ifndef _IONIC_H_
+#define _IONIC_H_
+
+#define IONIC_DRV_NAME		"ionic"
+#define IONIC_DRV_DESCRIPTION	"Pensando Ethernet NIC Driver"
+#define IONIC_DRV_VERSION	"0.15.0-k"
+
+#define PCI_VENDOR_ID_PENSANDO			0x1dd8
+
+#define PCI_DEVICE_ID_PENSANDO_IONIC_ETH_PF	0x1002
+#define PCI_DEVICE_ID_PENSANDO_IONIC_ETH_VF	0x1003
+
+#define IONIC_SUBDEV_ID_NAPLES_25	0x4000
+#define IONIC_SUBDEV_ID_NAPLES_100_4	0x4001
+#define IONIC_SUBDEV_ID_NAPLES_100_8	0x4002
+
+struct ionic {
+	struct pci_dev *pdev;
+	struct device *dev;
+	struct devlink *dl;
+};
+
+#endif /* _IONIC_H_ */
drivers/net/ethernet/pensando/ionic/ionic_bus.h new file +10 -0
new file mode 100644
index 000000000000..94ba0afc6f38
@@ -0,0 +1,10 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+
+#ifndef _IONIC_BUS_H_
+#define _IONIC_BUS_H_
+
+int ionic_bus_register_driver(void);
+void ionic_bus_unregister_driver(void);
+
+#endif /* _IONIC_BUS_H_ */
drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c new file +59 -0

59 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_devlink.c new file +43 -0

43 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_devlink.h new file +12 -0
new file mode 100644
index 000000000000..1df50874260a
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+
+#ifndef _IONIC_DEVLINK_H_
+#define _IONIC_DEVLINK_H_
+
+#include <net/devlink.h>
+
+struct ionic *ionic_devlink_alloc(struct device *dev);
+void ionic_devlink_free(struct ionic *ionic);
+
+#endif /* _IONIC_DEVLINK_H_ */
drivers/net/ethernet/pensando/ionic/ionic_main.c new file +32 -0
new file mode 100644
index 000000000000..332b528ce921
@@ -0,0 +1,32 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+
+#include <linux/module.h>
+#include <linux/version.h>
+#include <linux/netdevice.h>
+#include <linux/utsname.h>
+
+#include "ionic.h"
+#include "ionic_bus.h"
+
+MODULE_DESCRIPTION(IONIC_DRV_DESCRIPTION);
+MODULE_AUTHOR("Pensando Systems, Inc");
+MODULE_LICENSE("GPL");
+MODULE_VERSION(IONIC_DRV_VERSION);
+
+static int __init ionic_init_module(void)
+{
+	pr_info("%s %s, ver %s\n",
+		IONIC_DRV_NAME, IONIC_DRV_DESCRIPTION, IONIC_DRV_VERSION);
+	return ionic_bus_register_driver();
+}
+
+static void __exit ionic_cleanup_module(void)
+{
+	ionic_bus_unregister_driver();
+
+	pr_info("%s removed\n", IONIC_DRV_NAME);
+}
+
+module_init(ionic_init_module);
+module_exit(ionic_cleanup_module);
Signature · 1 line
2.17.1
#2 lore raw

Re: [PATCH v5 net-next 01/18] ionic: Add basic framework for IONIC Network device driver

drivers/net/ethernet/pensando/ionic/ionic.h line 23

Quoted 4 lines · Shannon Nelson
drivers/net/ethernet/pensando/ionic/ionic.h new file +4 -0
+struct ionic {
+	struct pci_dev *pdev;
+	struct device *dev;
+	struct devlink *dl;

No need for the dl pointer here. priv_to_devlink can be used to obtain the devlink pointer based on priv structure address.

Quoted 1 line

+};

#3 lore raw

Re: [PATCH v5 net-next 01/18] ionic: Add basic framework for IONIC Network device driver

Quoted 9 lines · Jakub Kicinski

drivers/net/ethernet/pensando/ionic/ionic.h line 23

Quoted 4 lines · Shannon Nelson
drivers/net/ethernet/pensando/ionic/ionic.h new file +4 -0
+struct ionic {
+	struct pci_dev *pdev;
+	struct device *dev;
+	struct devlink *dl;

No need for the dl pointer here. priv_to_devlink can be used to obtain the devlink pointer based on priv structure address.

Quoted 1 line

+};

Sure

Thanks,
sln

#4 lore raw

PATCH v5 2/18 ionic: Add hardware init and device commands

In reply to Shannon Nelson

The ionic device has a small set of PCI registers, including a device control and data space, and a large set of message commands.

---
 drivers/net/ethernet/pensando/ionic/Makefile  |    3 +-
 drivers/net/ethernet/pensando/ionic/ionic.h   |   21 +
 .../net/ethernet/pensando/ionic/ionic_bus.h   |    1 +
 .../ethernet/pensando/ionic/ionic_bus_pci.c   |  142 +-
 .../ethernet/pensando/ionic/ionic_debugfs.c   |   67 +
 .../ethernet/pensando/ionic/ionic_debugfs.h   |   24 +
 .../net/ethernet/pensando/ionic/ionic_dev.c   |  136 +
 .../net/ethernet/pensando/ionic/ionic_dev.h   |  143 +
 .../ethernet/pensando/ionic/ionic_devlink.c   |   39 +
 .../ethernet/pensando/ionic/ionic_devlink.h   |    2 +
 .../net/ethernet/pensando/ionic/ionic_if.h    | 2482 +++++++++++++++++
 .../net/ethernet/pensando/ionic/ionic_main.c  |  298 ++
 .../net/ethernet/pensando/ionic/ionic_regs.h  |  133 +
 13 files changed, 3489 insertions(+), 2 deletions(-)
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_debugfs.c
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_debugfs.h
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_dev.c
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_dev.h
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_if.h
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_regs.h
drivers/net/ethernet/pensando/ionic/Makefile +2 -1
index f174e8f7bce1..a23d58519c63 100644
@@ -3,4 +3,5 @@
 
 obj-$(CONFIG_IONIC) := ionic.o
 
-ionic-y := ionic_main.o ionic_bus_pci.o ionic_devlink.o
+ionic-y := ionic_main.o ionic_bus_pci.o ionic_devlink.o ionic_dev.o \
+	   ionic_debugfs.o
drivers/net/ethernet/pensando/ionic/ionic.h +21 -0
index d40077161214..1f3c4a916849 100644
@@ -4,6 +4,10 @@
 #ifndef _IONIC_H_
 #define _IONIC_H_
 
+#include "ionic_if.h"
+#include "ionic_dev.h"
+#include "ionic_devlink.h"
+
 #define IONIC_DRV_NAME		"ionic"
 #define IONIC_DRV_DESCRIPTION	"Pensando Ethernet NIC Driver"
 #define IONIC_DRV_VERSION	"0.15.0-k"
@@ -17,10 +21,27 @@
 #define IONIC_SUBDEV_ID_NAPLES_100_4	0x4001
 #define IONIC_SUBDEV_ID_NAPLES_100_8	0x4002
 
+#define devcmd_timeout  10
+
 struct ionic {
 	struct pci_dev *pdev;
 	struct device *dev;
 	struct devlink *dl;
+	struct devlink_port dl_port;
+	struct ionic_dev idev;
+	struct mutex dev_cmd_lock;	/* lock for dev_cmd operations */
+	struct dentry *dentry;
+	struct ionic_dev_bar bars[IONIC_BARS_MAX];
+	unsigned int num_bars;
+	struct ionic_identity ident;
 };
 
+int ionic_dev_cmd_wait(struct ionic *ionic, unsigned long max_wait);
+int ionic_set_dma_mask(struct ionic *ionic);
+int ionic_setup(struct ionic *ionic);
+
+int ionic_identify(struct ionic *ionic);
+int ionic_init(struct ionic *ionic);
+int ionic_reset(struct ionic *ionic);
+
 #endif /* _IONIC_H_ */
drivers/net/ethernet/pensando/ionic/ionic_bus.h +1 -0
index 94ba0afc6f38..24b4c01ec03f 100644
@@ -4,6 +4,7 @@
 #ifndef _IONIC_BUS_H_
 #define _IONIC_BUS_H_
 
+const char *ionic_bus_info(struct ionic *ionic);
 int ionic_bus_register_driver(void);
 void ionic_bus_unregister_driver(void);
 
drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c +141 -1

172 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_debugfs.c new file +67 -0

67 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_debugfs.h new file +24 -0
new file mode 100644
index 000000000000..7073a8b4e2f9
@@ -0,0 +1,24 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+
+#ifndef _IONIC_DEBUGFS_H_
+#define _IONIC_DEBUGFS_H_
+
+#include <linux/debugfs.h>
+
+#ifdef CONFIG_DEBUG_FS
+
+void ionic_debugfs_create(void);
+void ionic_debugfs_destroy(void);
+void ionic_debugfs_add_dev(struct ionic *ionic);
+void ionic_debugfs_del_dev(struct ionic *ionic);
+void ionic_debugfs_add_ident(struct ionic *ionic);
+#else
+static inline void ionic_debugfs_create(void) { }
+static inline void ionic_debugfs_destroy(void) { }
+static inline void ionic_debugfs_add_dev(struct ionic *ionic) { }
+static inline void ionic_debugfs_del_dev(struct ionic *ionic) { }
+static inline void ionic_debugfs_add_ident(struct ionic *ionic) { }
+#endif
+
+#endif /* _IONIC_DEBUGFS_H_ */
drivers/net/ethernet/pensando/ionic/ionic_dev.c new file +136 -0

136 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_dev.h new file +143 -0

143 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_devlink.c +39 -0
index e24ef6971cd5..1ca1e33cca04 100644
@@ -11,8 +11,28 @@
 static int ionic_dl_info_get(struct devlink *dl, struct devlink_info_req *req,
 			     struct netlink_ext_ack *extack)
 {
+	struct ionic *ionic = devlink_priv(dl);
+	struct ionic_dev *idev = &ionic->idev;
+	char buf[16];
+
 	devlink_info_driver_name_put(req, IONIC_DRV_NAME);
 
+	devlink_info_version_running_put(req,
+					 DEVLINK_INFO_VERSION_GENERIC_FW_MGMT,
+					 idev->dev_info.fw_version);
+
+	snprintf(buf, sizeof(buf), "0x%x", idev->dev_info.asic_type);
+	devlink_info_version_fixed_put(req,
+				       DEVLINK_INFO_VERSION_GENERIC_BOARD_ID,
+				       buf);
+
+	snprintf(buf, sizeof(buf), "0x%x", idev->dev_info.asic_rev);
+	devlink_info_version_fixed_put(req,
+				       DEVLINK_INFO_VERSION_GENERIC_BOARD_REV,
+				       buf);
+
+	devlink_info_serial_number_put(req, idev->dev_info.serial_num);
+
 	return 0;
 }
 
@@ -41,3 +61,22 @@ void ionic_devlink_free(struct ionic *ionic)
 {
 	devlink_free(ionic->dl);
 }
+
+int ionic_devlink_register(struct ionic *ionic)
+{
+	int err;
+
+	err = devlink_register(ionic->dl, ionic->dev);
+	if (err)
+		dev_warn(ionic->dev, "devlink_register failed: %d\n", err);
+
+	return err;
+}
+
+void ionic_devlink_unregister(struct ionic *ionic)
+{
+	if (!ionic || !ionic->dl)
+		return;
+
+	devlink_unregister(ionic->dl);
+}
drivers/net/ethernet/pensando/ionic/ionic_devlink.h +2 -0
index 1df50874260a..0690172fc57a 100644
@@ -8,5 +8,7 @@
 
 struct ionic *ionic_devlink_alloc(struct device *dev);
 void ionic_devlink_free(struct ionic *ionic);
+int ionic_devlink_register(struct ionic *ionic);
+void ionic_devlink_unregister(struct ionic *ionic);
 
 #endif /* _IONIC_DEVLINK_H_ */
drivers/net/ethernet/pensando/ionic/ionic_if.h new file +2482 -0

2,351 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_main.c +298 -0

298 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_regs.h new file +133 -0

133 lines not on this page · raw message

The rest of this diff is not shown. View the raw message.

Signature · 1 line
2.17.1

Part of the diff is not shown here. View the raw message.

#5 lore raw

Re: [PATCH v5 net-next 02/18] ionic: Add hardware init and device commands

drivers/net/ethernet/pensando/ionic/ionic_debugfs.c line 33

Quoted 10 lines · Shannon Nelson
drivers/net/ethernet/pensando/ionic/ionic_debugfs.c new file +10 -0
+void ionic_debugfs_add_dev(struct ionic *ionic)
+{
+	struct dentry *dentry;
+
+	dentry = debugfs_create_dir(ionic_bus_info(ionic), ionic_dir);
+	if (IS_ERR_OR_NULL(dentry))
+		return;
+
+	ionic->dentry = dentry;
+}

Hi Shannon

There was recently a big patchset from GregKH which removed all error checking from drivers calling debugfs calls. I'm pretty sure you don't need this check here.

Quoted 4 lines
drivers/net/ethernet/pensando/ionic/ionic_debugfs.h new file +4 -0
+#ifdef CONFIG_DEBUG_FS
+
+void ionic_debugfs_create(void);
+void ionic_debugfs_destroy(void);

drivers/net/ethernet/pensando/ionic/ionic_debugfs.h line 22

Quoted 10 lines
drivers/net/ethernet/pensando/ionic/ionic_debugfs.h new file +10 -0
+void ionic_debugfs_add_dev(struct ionic *ionic);
+void ionic_debugfs_del_dev(struct ionic *ionic);
+void ionic_debugfs_add_ident(struct ionic *ionic);
+#else
+static inline void ionic_debugfs_create(void) { }
+static inline void ionic_debugfs_destroy(void) { }
+static inline void ionic_debugfs_add_dev(struct ionic *ionic) { }
+static inline void ionic_debugfs_del_dev(struct ionic *ionic) { }
+static inline void ionic_debugfs_add_ident(struct ionic *ionic) { }
+#endif

Is this really needed? I would expect there to be stubs for all the debugfs calls if it is disabled.

Quoted 32 lines
drivers/net/ethernet/pensando/ionic/ionic_if.h new file +31 -0
+/**
+ * union drv_identity - driver identity information
+ * @os_type:          OS type (see enum os_type)
+ * @os_dist:          OS distribution, numeric format
+ * @os_dist_str:      OS distribution, string format
+ * @kernel_ver:       Kernel version, numeric format
+ * @kernel_ver_str:   Kernel version, string format
+ * @driver_ver_str:   Driver version, string format
+ */
+union ionic_drv_identity {
+	struct {
+		__le32 os_type;
+		__le32 os_dist;
+		char   os_dist_str[128];
+		__le32 kernel_ver;
+		char   kernel_ver_str[32];
+		char   driver_ver_str[32];
+	};
+	__le32 words[512];
+};
 
+int ionic_identify(struct ionic *ionic)
+{
+	struct ionic_identity *ident = &ionic->ident;
+	struct ionic_dev *idev = &ionic->idev;
+	size_t sz;
+	int err;
+
+	memset(ident, 0, sizeof(*ident));
+
+	ident->drv.os_type = cpu_to_le32(IONIC_OS_TYPE_LINUX);
+	ident->drv.os_dist = 0;

drivers/net/ethernet/pensando/ionic/ionic_if.h line 298

Quoted 10 lines
drivers/net/ethernet/pensando/ionic/ionic_if.h new file +10 -0
+	strncpy(ident->drv.os_dist_str, utsname()->release,
+		sizeof(ident->drv.os_dist_str) - 1);
+	ident->drv.kernel_ver = cpu_to_le32(LINUX_VERSION_CODE);
+	strncpy(ident->drv.kernel_ver_str, utsname()->version,
+		sizeof(ident->drv.kernel_ver_str) - 1);
+	strncpy(ident->drv.driver_ver_str, IONIC_DRV_VERSION,
+		sizeof(ident->drv.driver_ver_str) - 1);
+
+	mutex_lock(&ionic->dev_cmd_lock);
+

I don't know about others, but from a privacy prospective, i'm not so happy about this. This is a smart NIC. It could be reporting back to Mothership pensando with this information?

I would be happier if there was a privacy statement, right here, saying what this information is used for, and an agreement it is not used for anything else. If that gets violated, you can then only blame yourself when we ripe this out and hard code it to static values.

Andrew

#6 lore raw

Re: [PATCH v5 net-next 02/18] ionic: Add hardware init and device commands

Quoted 26 lines · Andrew Lunn
Quoted 11 lines
drivers/net/ethernet/pensando/ionic/ionic_main.c +11 -0
+int ionic_identify(struct ionic *ionic)
+{
+	struct ionic_identity *ident = &ionic->ident;
+	struct ionic_dev *idev = &ionic->idev;
+	size_t sz;
+	int err;
+
+	memset(ident, 0, sizeof(*ident));
+
+	ident->drv.os_type = cpu_to_le32(IONIC_OS_TYPE_LINUX);
+	ident->drv.os_dist = 0;

drivers/net/ethernet/pensando/ionic/ionic_main.c line 262

Quoted 10 lines
drivers/net/ethernet/pensando/ionic/ionic_main.c +10 -0
+	strncpy(ident->drv.os_dist_str, utsname()->release,
+		sizeof(ident->drv.os_dist_str) - 1);
+	ident->drv.kernel_ver = cpu_to_le32(LINUX_VERSION_CODE);
+	strncpy(ident->drv.kernel_ver_str, utsname()->version,
+		sizeof(ident->drv.kernel_ver_str) - 1);
+	strncpy(ident->drv.driver_ver_str, IONIC_DRV_VERSION,
+		sizeof(ident->drv.driver_ver_str) - 1);
+
+	mutex_lock(&ionic->dev_cmd_lock);
+  

I don't know about others, but from a privacy prospective, i'm not so happy about this. This is a smart NIC. It could be reporting back to Mothership pensando with this information?

Quoted 4 lines

I would be happier if there was a privacy statement, right here, saying what this information is used for, and an agreement it is not used for anything else. If that gets violated, you can then only blame yourself when we ripe this out and hard code it to static values.

FWIW seems like a fair ask to me..

#7 lore raw

Re: [PATCH v5 net-next 02/18] ionic: Add hardware init and device commands

In reply to Andrew Lunn
Quoted 13 lines · Andrew Lunn

drivers/net/ethernet/pensando/ionic/ionic_debugfs.c line 33

Quoted 10 lines · Shannon Nelson
drivers/net/ethernet/pensando/ionic/ionic_debugfs.c new file +10 -0
+void ionic_debugfs_add_dev(struct ionic *ionic)
+{
+	struct dentry *dentry;
+
+	dentry = debugfs_create_dir(ionic_bus_info(ionic), ionic_dir);
+	if (IS_ERR_OR_NULL(dentry))
+		return;
+
+	ionic->dentry = dentry;
+}

Hi Shannon

Quoted 3 lines

There was recently a big patchset from GregKH which removed all error checking from drivers calling debugfs calls. I'm pretty sure you don't need this check here.

With this check I end up either with a valid dentry value or NULL in ionic->dentry.  Without this check I possibly get an error value in ionic->dentry, which can get used later as the parent dentry to try to make a new debugfs node.  Some quick tracing looks like this error value will get dereferenced in a call to inode_lock(), which would likely cause us some heartburn.

I'd prefer to keep this check and leave ionic->dentry as NULL.

I've removed several of the other error checks and messages that were in this code, but left a few of the IS_ERR_OR_NULL checks to be sure we don't try dereferencing similar bogus values.

Quoted 17 lines
Quoted 5 lines
drivers/net/ethernet/pensando/ionic/ionic_debugfs.h new file +4 -0
+#ifdef CONFIG_DEBUG_FS
+
+void ionic_debugfs_create(void);
+void ionic_debugfs_destroy(void);

drivers/net/ethernet/pensando/ionic/ionic_debugfs.h line 22

Quoted 10 lines
drivers/net/ethernet/pensando/ionic/ionic_debugfs.h new file +10 -0
+void ionic_debugfs_add_dev(struct ionic *ionic);
+void ionic_debugfs_del_dev(struct ionic *ionic);
+void ionic_debugfs_add_ident(struct ionic *ionic);
+#else
+static inline void ionic_debugfs_create(void) { }
+static inline void ionic_debugfs_destroy(void) { }
+static inline void ionic_debugfs_add_dev(struct ionic *ionic) { }
+static inline void ionic_debugfs_del_dev(struct ionic *ionic) { }
+static inline void ionic_debugfs_add_ident(struct ionic *ionic) { }
+#endif

Is this really needed? I would expect there to be stubs for all the debugfs calls if it is disabled.

If CONFIG_DEBUG_FS is not enabled, I would prefer this driver's debugfs code to also be left out rather than be compiled in and left useless.

Quoted 37 lines
Quoted 37 lines
drivers/net/ethernet/pensando/ionic/ionic_if.h new file +36 -0
+/**
+ * union drv_identity - driver identity information
+ * @os_type:          OS type (see enum os_type)
+ * @os_dist:          OS distribution, numeric format
+ * @os_dist_str:      OS distribution, string format
+ * @kernel_ver:       Kernel version, numeric format
+ * @kernel_ver_str:   Kernel version, string format
+ * @driver_ver_str:   Driver version, string format
+ */
+union ionic_drv_identity {
+	struct {
+		__le32 os_type;
+		__le32 os_dist;
+		char   os_dist_str[128];
+		__le32 kernel_ver;
+		char   kernel_ver_str[32];
+		char   driver_ver_str[32];
+	};
+	__le32 words[512];
+};
+int ionic_identify(struct ionic *ionic)
+{
+	struct ionic_identity *ident = &ionic->ident;
+	struct ionic_dev *idev = &ionic->idev;
+	size_t sz;
+	int err;
+
+	memset(ident, 0, sizeof(*ident));
+
+	ident->drv.os_type = cpu_to_le32(IONIC_OS_TYPE_LINUX);
+	ident->drv.os_dist = 0;
+	strncpy(ident->drv.os_dist_str, utsname()->release,
+		sizeof(ident->drv.os_dist_str) - 1);
+	ident->drv.kernel_ver = cpu_to_le32(LINUX_VERSION_CODE);
+	strncpy(ident->drv.kernel_ver_str, utsname()->version,
+		sizeof(ident->drv.kernel_ver_str) - 1);
Quoted 8 lines

drivers/net/ethernet/pensando/ionic/ionic_main.c line 262

Quoted 5 lines
drivers/net/ethernet/pensando/ionic/ionic_main.c +5 -0
+	strncpy(ident->drv.driver_ver_str, IONIC_DRV_VERSION,
+		sizeof(ident->drv.driver_ver_str) - 1);
+
+	mutex_lock(&ionic->dev_cmd_lock);
+

I don't know about others, but from a privacy prospective, i'm not so happy about this. This is a smart NIC. It could be reporting back to Mothership pensando with this information?

I suppose the phrase "you can trust us" wouldn't help much here, would it... :-)

Quoted 5 lines

I would be happier if there was a privacy statement, right here, saying what this information is used for, and an agreement it is not used for anything else. If that gets violated, you can then only blame yourself when we ripe this out and hard code it to static values.

That makes perfect sense.

I can add a full description here of how the information will be used, which should help most folks, but I'm sure there will still be some that don't want this info released.

What I'd like to propose here is that I do the hardcoded strings myself for now, and I work up a way for the users to enable the feature as desired, with a reasonable comment here in the code and in the Documentation/.../ionic.rst file.  This might end up as an ethtool priv-flag that defaults to off and can set a NIC value that is remembered for later.

Does that sound reasonable?

Cheers,
sln

#8 lore raw

Re: [PATCH v5 net-next 02/18] ionic: Add hardware init and device commands

Quoted 14 lines · Shannon Nelson
Quoted 13 lines · Andrew Lunn

drivers/net/ethernet/pensando/ionic/ionic_debugfs.c line 33

Quoted 10 lines · Shannon Nelson
drivers/net/ethernet/pensando/ionic/ionic_debugfs.c new file +10 -0
+void ionic_debugfs_add_dev(struct ionic *ionic)
+{
+	struct dentry *dentry;
+
+	dentry = debugfs_create_dir(ionic_bus_info(ionic), ionic_dir);
+	if (IS_ERR_OR_NULL(dentry))
+		return;
+
+	ionic->dentry = dentry;
+}

Hi Shannon

Quoted 8 lines
Quoted 3 lines

There was recently a big patchset from GregKH which removed all error checking from drivers calling debugfs calls. I'm pretty sure you don't need this check here.

With this check I end up either with a valid dentry value or NULL in ionic->dentry.  Without this check I possibly get an error value in ionic->dentry, which can get used later as the parent dentry to try to make a new debugfs node.

Hi Shannon

What you should find is that every debugfs function will have something like:

	if (IS_ERR(dentry))
	   return dentry;
or
	if (IS_ERR(parent))
	   return parent;

If you know of a API which is missing such protection, i'm sure GregKH would like to know. Especially since he just ripped all such protection in driver out. Meaning he just broken some drivers if such IS_ERR() calls are missing in the debugfs core.

Quoted 11 lines
Quoted 4 lines

I would be happier if there was a privacy statement, right here, saying what this information is used for, and an agreement it is not used for anything else. If that gets violated, you can then only blame yourself when we ripe this out and hard code it to static values.

That makes perfect sense.

I can add a full description here of how the information will be used, which should help most folks, but I'm sure there will still be some that don't want this info released.

Quoted 7 lines

What I'd like to propose here is that I do the hardcoded strings myself for now, and I work up a way for the users to enable the feature as desired, with a reasonable comment here in the code and in the Documentation/.../ionic.rst file.  This might end up as an ethtool priv-flag that defaults to off and can set a NIC value that is remembered for later.

Does that sound reasonable?

Yes, that sounds reasonable.

Thanks
Andrew

#9 lore raw

Re: [PATCH v5 net-next 02/18] ionic: Add hardware init and device commands

Quoted 33 lines · Andrew Lunn
Quoted 14 lines · Shannon Nelson
Quoted 13 lines · Andrew Lunn

drivers/net/ethernet/pensando/ionic/ionic_debugfs.c line 33

Quoted 10 lines · Shannon Nelson
drivers/net/ethernet/pensando/ionic/ionic_debugfs.c new file +10 -0
+void ionic_debugfs_add_dev(struct ionic *ionic)
+{
+	struct dentry *dentry;
+
+	dentry = debugfs_create_dir(ionic_bus_info(ionic), ionic_dir);
+	if (IS_ERR_OR_NULL(dentry))
+		return;
+
+	ionic->dentry = dentry;
+}

Hi Shannon

Quoted 7 lines
Quoted 3 lines

There was recently a big patchset from GregKH which removed all error checking from drivers calling debugfs calls. I'm pretty sure you don't need this check here.

With this check I end up either with a valid dentry value or NULL in ionic->dentry.  Without this check I possibly get an error value in ionic->dentry, which can get used later as the parent dentry to try to make a new debugfs node.

Hi Shannon

What you should find is that every debugfs function will have something like:

	if (IS_ERR(dentry))
	   return dentry;
or
	if (IS_ERR(parent))
	   return parent;
Quoted 5 lines

If you know of a API which is missing such protection, i'm sure GregKH would like to know. Especially since he just ripped all such protection in driver out. Meaning he just broken some drivers if such IS_ERR() calls are missing in the debugfs core.

Ah, here's the confusion: there's a start_creating() in the tracefs as well as in the debugfs, and the tracefs code doesn't have all of those checks.

sln

#10 lore raw

Re: [PATCH v5 net-next 02/18] ionic: Add hardware init and device commands

In reply to Shannon Nelson
Quoted 33 lines · Shannon Nelson

The ionic device has a small set of PCI registers, including a device control and data space, and a large set of message commands.

drivers/net/ethernet/pensando/ionic/Makefile +2 -1
index f174e8f7bce1..a23d58519c63 100644
@@ -3,4 +3,5 @@
 
 obj-$(CONFIG_IONIC) := ionic.o
 
-ionic-y := ionic_main.o ionic_bus_pci.o ionic_devlink.o
+ionic-y := ionic_main.o ionic_bus_pci.o ionic_devlink.o ionic_dev.o \
+	   ionic_debugfs.o
drivers/net/ethernet/pensando/ionic/ionic.h +4 -0
index d40077161214..1f3c4a916849 100644
@@ -4,6 +4,10 @@
 #ifndef _IONIC_H_
 #define _IONIC_H_
 
+#include "ionic_if.h"
+#include "ionic_dev.h"
+#include "ionic_devlink.h"
+
 #define IONIC_DRV_NAME		"ionic"
 #define IONIC_DRV_DESCRIPTION	"Pensando Ethernet NIC Driver"
 #define IONIC_DRV_VERSION	"0.15.0-k"

drivers/net/ethernet/pensando/ionic/ionic.h line 24

Quoted 4 lines
drivers/net/ethernet/pensando/ionic/ionic.h +1 -0
@@ -17,10 +21,27 @@
 #define IONIC_SUBDEV_ID_NAPLES_100_4	0x4001
 #define IONIC_SUBDEV_ID_NAPLES_100_8	0x4002
 
+#define devcmd_timeout  10

nit: upper case?

drivers/net/ethernet/pensando/ionic/ionic.h line 30

Quoted 5 lines
drivers/net/ethernet/pensando/ionic/ionic.h +1 -0
 struct ionic {
 	struct pci_dev *pdev;
 	struct device *dev;
 	struct devlink *dl;
+	struct devlink_port dl_port;

devlink_port is not used in this patch

Quoted 33 lines
drivers/net/ethernet/pensando/ionic/ionic.h +15 -0
+	struct ionic_dev idev;
+	struct mutex dev_cmd_lock;	/* lock for dev_cmd operations */
+	struct dentry *dentry;
+	struct ionic_dev_bar bars[IONIC_BARS_MAX];
+	unsigned int num_bars;
+	struct ionic_identity ident;
 };
 
+	err = ionic_init(ionic);
+	if (err) {
+		dev_err(dev, "Cannot init device: %d, aborting\n", err);
+		goto err_out_teardown;
+	}
+
+	err = ionic_devlink_register(ionic);
+	if (err)
+		dev_err(dev, "Cannot register devlink: %d\n", err);
 
drivers/net/ethernet/pensando/ionic/ionic.h +14 -0
 	return 0;
+
+err_out_teardown:
+	ionic_dev_teardown(ionic);
+err_out_unmap_bars:
+	ionic_unmap_bars(ionic);
+	pci_release_regions(pdev);
+err_out_pci_clear_master:
+	pci_clear_master(pdev);
+err_out_pci_disable_device:
+	pci_disable_device(pdev);
+err_out_debugfs_del_dev:
+	ionic_debugfs_del_dev(ionic);
+err_out_clear_drvdata:
+	mutex_destroy(&ionic->dev_cmd_lock);

drivers/net/ethernet/pensando/ionic/ionic.h

Quoted 10 lines
drivers/net/ethernet/pensando/ionic/ionic.h +4 -0
+	ionic_devlink_free(ionic);
+
+	return err;
 }
 
 static void ionic_remove(struct pci_dev *pdev)
 {
 	struct ionic *ionic = pci_get_drvdata(pdev);
 
+	if (!ionic)

How can this be NULL? Usually if this is NULL that means probe() failed but 'err' was not set properly. Perhaps WARN_ON() here?

Quoted 116 lines
drivers/net/ethernet/pensando/ionic/ionic.h +11 -0
+		return;
+
+	ionic_devlink_unregister(ionic);
+	ionic_reset(ionic);
+	ionic_dev_teardown(ionic);
+	ionic_unmap_bars(ionic);
+	pci_release_regions(pdev);
+	pci_clear_master(pdev);
+	pci_disable_device(pdev);
+	ionic_debugfs_del_dev(ionic);
+	mutex_destroy(&ionic->dev_cmd_lock);
 	ionic_devlink_free(ionic);
 }
drivers/net/ethernet/pensando/ionic/ionic_dev.h new file +95 -0
new file mode 100644
index 000000000000..30a5206bba4e
@@ -0,0 +1,143 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+
+#ifndef _IONIC_DEV_H_
+#define _IONIC_DEV_H_
+
+#include <linux/mutex.h>
+#include <linux/workqueue.h>
+
+#include "ionic_if.h"
+#include "ionic_regs.h"
+
+struct ionic_dev_bar {
+	void __iomem *vaddr;
+	phys_addr_t bus_addr;
+	unsigned long len;
+	int res_index;
+};
+
+static inline void ionic_struct_size_checks(void)
+{
+	/* Registers */
+	BUILD_BUG_ON(sizeof(struct ionic_intr) != 32);
+
+	BUILD_BUG_ON(sizeof(struct ionic_doorbell) != 8);
+	BUILD_BUG_ON(sizeof(struct ionic_intr_status) != 8);
+
+	BUILD_BUG_ON(sizeof(union ionic_dev_regs) != 4096);
+	BUILD_BUG_ON(sizeof(union ionic_dev_info_regs) != 2048);
+	BUILD_BUG_ON(sizeof(union ionic_dev_cmd_regs) != 2048);
+
+	BUILD_BUG_ON(sizeof(struct ionic_lif_stats) != 1024);
+
+	BUILD_BUG_ON(sizeof(struct ionic_admin_cmd) != 64);
+	BUILD_BUG_ON(sizeof(struct ionic_admin_comp) != 16);
+	BUILD_BUG_ON(sizeof(struct ionic_nop_cmd) != 64);
+	BUILD_BUG_ON(sizeof(struct ionic_nop_comp) != 16);
+
+	/* Device commands */
+	BUILD_BUG_ON(sizeof(struct ionic_dev_identify_cmd) != 64);
+	BUILD_BUG_ON(sizeof(struct ionic_dev_identify_comp) != 16);
+	BUILD_BUG_ON(sizeof(struct ionic_dev_init_cmd) != 64);
+	BUILD_BUG_ON(sizeof(struct ionic_dev_init_comp) != 16);
+	BUILD_BUG_ON(sizeof(struct ionic_dev_reset_cmd) != 64);
+	BUILD_BUG_ON(sizeof(struct ionic_dev_reset_comp) != 16);
+	BUILD_BUG_ON(sizeof(struct ionic_dev_getattr_cmd) != 64);
+	BUILD_BUG_ON(sizeof(struct ionic_dev_getattr_comp) != 16);
+	BUILD_BUG_ON(sizeof(struct ionic_dev_setattr_cmd) != 64);
+	BUILD_BUG_ON(sizeof(struct ionic_dev_setattr_comp) != 16);
+
+	/* Port commands */
+	BUILD_BUG_ON(sizeof(struct ionic_port_identify_cmd) != 64);
+	BUILD_BUG_ON(sizeof(struct ionic_port_identify_comp) != 16);
+	BUILD_BUG_ON(sizeof(struct ionic_port_init_cmd) != 64);
+	BUILD_BUG_ON(sizeof(struct ionic_port_init_comp) != 16);
+	BUILD_BUG_ON(sizeof(struct ionic_port_reset_cmd) != 64);
+	BUILD_BUG_ON(sizeof(struct ionic_port_reset_comp) != 16);
+	BUILD_BUG_ON(sizeof(struct ionic_port_getattr_cmd) != 64);
+	BUILD_BUG_ON(sizeof(struct ionic_port_getattr_comp) != 16);
+	BUILD_BUG_ON(sizeof(struct ionic_port_setattr_cmd) != 64);
+	BUILD_BUG_ON(sizeof(struct ionic_port_setattr_comp) != 16);
+
+	/* LIF commands */
+	BUILD_BUG_ON(sizeof(struct ionic_lif_init_cmd) != 64);
+	BUILD_BUG_ON(sizeof(struct ionic_lif_init_comp) != 16);
+	BUILD_BUG_ON(sizeof(struct ionic_lif_reset_cmd) != 64);
+	BUILD_BUG_ON(sizeof(ionic_lif_reset_comp) != 16);
+	BUILD_BUG_ON(sizeof(struct ionic_lif_getattr_cmd) != 64);
+	BUILD_BUG_ON(sizeof(struct ionic_lif_getattr_comp) != 16);
+	BUILD_BUG_ON(sizeof(struct ionic_lif_setattr_cmd) != 64);
+	BUILD_BUG_ON(sizeof(struct ionic_lif_setattr_comp) != 16);
+
+	BUILD_BUG_ON(sizeof(struct ionic_q_init_cmd) != 64);
+	BUILD_BUG_ON(sizeof(struct ionic_q_init_comp) != 16);
+	BUILD_BUG_ON(sizeof(struct ionic_q_control_cmd) != 64);
+	BUILD_BUG_ON(sizeof(ionic_q_control_comp) != 16);
+
+	BUILD_BUG_ON(sizeof(struct ionic_rx_mode_set_cmd) != 64);
+	BUILD_BUG_ON(sizeof(ionic_rx_mode_set_comp) != 16);
+	BUILD_BUG_ON(sizeof(struct ionic_rx_filter_add_cmd) != 64);
+	BUILD_BUG_ON(sizeof(struct ionic_rx_filter_add_comp) != 16);
+	BUILD_BUG_ON(sizeof(struct ionic_rx_filter_del_cmd) != 64);
+	BUILD_BUG_ON(sizeof(ionic_rx_filter_del_comp) != 16);
+
+	/* RDMA commands */
+	BUILD_BUG_ON(sizeof(struct ionic_rdma_reset_cmd) != 64);
+	BUILD_BUG_ON(sizeof(struct ionic_rdma_queue_cmd) != 64);
+
+	/* Events */
+	BUILD_BUG_ON(sizeof(struct ionic_notifyq_cmd) != 4);
+	BUILD_BUG_ON(sizeof(union ionic_notifyq_comp) != 64);
+	BUILD_BUG_ON(sizeof(struct ionic_notifyq_event) != 64);
+	BUILD_BUG_ON(sizeof(struct ionic_link_change_event) != 64);
+	BUILD_BUG_ON(sizeof(struct ionic_reset_event) != 64);
+	BUILD_BUG_ON(sizeof(struct ionic_heartbeat_event) != 64);

drivers/net/ethernet/pensando/ionic/ionic_dev.h line 105

Quoted 10 lines
drivers/net/ethernet/pensando/ionic/ionic_dev.h new file +10 -0
+	BUILD_BUG_ON(sizeof(struct ionic_log_event) != 64);
+
+	/* I/O */
+	BUILD_BUG_ON(sizeof(struct ionic_txq_desc) != 16);
+	BUILD_BUG_ON(sizeof(struct ionic_txq_sg_desc) != 128);
+	BUILD_BUG_ON(sizeof(struct ionic_txq_comp) != 16);
+
+	BUILD_BUG_ON(sizeof(struct ionic_rxq_desc) != 16);
+	BUILD_BUG_ON(sizeof(struct ionic_rxq_sg_desc) != 128);
+	BUILD_BUG_ON(sizeof(struct ionic_rxq_comp) != 16);

static_assert() for all of those? That way you don't need this fake function.

Quoted 45 lines
drivers/net/ethernet/pensando/ionic/ionic_dev.h new file +38 -0
+}
+
+struct ionic_devinfo {
+	u8 asic_type;
+	u8 asic_rev;
+	char fw_version[IONIC_DEVINFO_FWVERS_BUFLEN + 1];
+	char serial_num[IONIC_DEVINFO_SERIAL_BUFLEN + 1];
+};
+
+struct ionic_dev {
+	union ionic_dev_info_regs __iomem *dev_info_regs;
+	union ionic_dev_cmd_regs __iomem *dev_cmd_regs;
+
+	u64 __iomem *db_pages;
+	dma_addr_t phy_db_pages;
+
+	struct ionic_intr __iomem *intr_ctrl;
+	u64 __iomem *intr_status;
+
+	struct ionic_devinfo dev_info;
+};
+
+struct ionic;
+
+void ionic_init_devinfo(struct ionic *ionic);
+int ionic_dev_setup(struct ionic *ionic);
+void ionic_dev_teardown(struct ionic *ionic);
+
+void ionic_dev_cmd_go(struct ionic_dev *idev, union ionic_dev_cmd *cmd);
+u8 ionic_dev_cmd_status(struct ionic_dev *idev);
+bool ionic_dev_cmd_done(struct ionic_dev *idev);
+void ionic_dev_cmd_comp(struct ionic_dev *idev, union ionic_dev_cmd_comp *comp);
+
+void ionic_dev_cmd_identify(struct ionic_dev *idev, u8 ver);
+void ionic_dev_cmd_init(struct ionic_dev *idev);
+void ionic_dev_cmd_reset(struct ionic_dev *idev);
+
+#endif /* _IONIC_DEV_H_ */
drivers/net/ethernet/pensando/ionic/ionic_devlink.c +0 -0
index e24ef6971cd5..1ca1e33cca04 100644
@@ -11,8 +11,28 @@
 static int ionic_dl_info_get(struct devlink *dl, struct devlink_info_req *req,
 			     struct netlink_ext_ack *extack)

drivers/net/ethernet/pensando/ionic/ionic_devlink.c line 22

Quoted 10 lines
drivers/net/ethernet/pensando/ionic/ionic_devlink.c +7 -0
 {
+	struct ionic *ionic = devlink_priv(dl);
+	struct ionic_dev *idev = &ionic->idev;
+	char buf[16];
+
 	devlink_info_driver_name_put(req, IONIC_DRV_NAME);
 
+	devlink_info_version_running_put(req,
+					 DEVLINK_INFO_VERSION_GENERIC_FW_MGMT,
+					 idev->dev_info.fw_version);

Are you sure this is not the FW that controls the data path?

drivers/net/ethernet/pensando/ionic/ionic_devlink.c line 27

Quoted 4 lines
drivers/net/ethernet/pensando/ionic/ionic_devlink.c +4 -0
+	snprintf(buf, sizeof(buf), "0x%x", idev->dev_info.asic_type);
+	devlink_info_version_fixed_put(req,
+				       DEVLINK_INFO_VERSION_GENERIC_BOARD_ID,
+				       buf);

Board ID is not ASIC. This is for identifying a board version with all its components which surround the main ASIC.

drivers/net/ethernet/pensando/ionic/ionic_devlink.c line 32

Quoted 4 lines
drivers/net/ethernet/pensando/ionic/ionic_devlink.c +4 -0
+	snprintf(buf, sizeof(buf), "0x%x", idev->dev_info.asic_rev);
+	devlink_info_version_fixed_put(req,
+				       DEVLINK_INFO_VERSION_GENERIC_BOARD_REV,
+				       buf);

ditto

Quoted 15 lines
drivers/net/ethernet/pensando/ionic/ionic_devlink.c +8 -0
+	devlink_info_serial_number_put(req, idev->dev_info.serial_num);
+
 	return 0;
 }
 
@@ -41,3 +61,22 @@ void ionic_devlink_free(struct ionic *ionic)
 {
 	devlink_free(ionic->dl);
 }
+
+int ionic_devlink_register(struct ionic *ionic)
+{
+	int err;
+
+	err = devlink_register(ionic->dl, ionic->dev);

drivers/net/ethernet/pensando/ionic/ionic_devlink.c void ionic_devlink_free(struct ionic *ionic) · line 79

Quoted 10 lines
drivers/net/ethernet/pensando/ionic/ionic_devlink.c +10 -0
+	if (err)
+		dev_warn(ionic->dev, "devlink_register failed: %d\n", err);
+
+	return err;
+}
+
+void ionic_devlink_unregister(struct ionic *ionic)
+{
+	if (!ionic || !ionic->dl)
+		return;

Impossiblu

Quoted 14 lines
drivers/net/ethernet/pensando/ionic/ionic_devlink.c +2 -0
+	devlink_unregister(ionic->dl);
+}
drivers/net/ethernet/pensando/ionic/ionic_devlink.h +2 -0
index 1df50874260a..0690172fc57a 100644
@@ -8,5 +8,7 @@
 
 struct ionic *ionic_devlink_alloc(struct device *dev);
 void ionic_devlink_free(struct ionic *ionic);
+int ionic_devlink_register(struct ionic *ionic);
+void ionic_devlink_unregister(struct ionic *ionic);
 
 #endif /* _IONIC_DEVLINK_H_ */
#11 lore raw

Re: [PATCH v5 net-next 02/18] ionic: Add hardware init and device commands

Quoted 36 lines · Jakub Kicinski
Quoted 35 lines · Shannon Nelson

The ionic device has a small set of PCI registers, including a device control and data space, and a large set of message commands.

drivers/net/ethernet/pensando/ionic/Makefile +2 -1
index f174e8f7bce1..a23d58519c63 100644
@@ -3,4 +3,5 @@
 
  obj-$(CONFIG_IONIC) := ionic.o
 
-ionic-y := ionic_main.o ionic_bus_pci.o ionic_devlink.o
+ionic-y := ionic_main.o ionic_bus_pci.o ionic_devlink.o ionic_dev.o \
+	   ionic_debugfs.o
drivers/net/ethernet/pensando/ionic/ionic.h +4 -0
index d40077161214..1f3c4a916849 100644
@@ -4,6 +4,10 @@
  #ifndef _IONIC_H_
  #define _IONIC_H_
 
+#include "ionic_if.h"
+#include "ionic_dev.h"
+#include "ionic_devlink.h"
+
  #define IONIC_DRV_NAME		"ionic"
  #define IONIC_DRV_DESCRIPTION	"Pensando Ethernet NIC Driver"
  #define IONIC_DRV_VERSION	"0.15.0-k"
@@ -17,10 +21,27 @@
  #define IONIC_SUBDEV_ID_NAPLES_100_4	0x4001
  #define IONIC_SUBDEV_ID_NAPLES_100_8	0x4002
 
Quoted 2 lines

drivers/net/ethernet/pensando/ionic/ionic.h line 24

Quoted 1 line
drivers/net/ethernet/pensando/ionic/ionic.h +1 -0
+#define devcmd_timeout  10

nit: upper case?

Sure

Quoted 7 lines

drivers/net/ethernet/pensando/ionic/ionic.h line 30

Quoted 6 lines
drivers/net/ethernet/pensando/ionic/ionic.h +1 -0
  struct ionic {
  	struct pci_dev *pdev;
  	struct device *dev;
  	struct devlink *dl;
+	struct devlink_port dl_port;

devlink_port is not used in this patch

I can move that to patch 09 where it gets used

Quoted 38 lines
Quoted 38 lines
drivers/net/ethernet/pensando/ionic/ionic.h +32 -0
+	struct ionic_dev idev;
+	struct mutex dev_cmd_lock;	/* lock for dev_cmd operations */
+	struct dentry *dentry;
+	struct ionic_dev_bar bars[IONIC_BARS_MAX];
+	unsigned int num_bars;
+	struct ionic_identity ident;
  };
+	err = ionic_init(ionic);
+	if (err) {
+		dev_err(dev, "Cannot init device: %d, aborting\n", err);
+		goto err_out_teardown;
+	}
+
+	err = ionic_devlink_register(ionic);
+	if (err)
+		dev_err(dev, "Cannot register devlink: %d\n", err);
 
  	return 0;
+
+err_out_teardown:
+	ionic_dev_teardown(ionic);
+err_out_unmap_bars:
+	ionic_unmap_bars(ionic);
+	pci_release_regions(pdev);
+err_out_pci_clear_master:
+	pci_clear_master(pdev);
+err_out_pci_disable_device:
+	pci_disable_device(pdev);
+err_out_debugfs_del_dev:
+	ionic_debugfs_del_dev(ionic);
+err_out_clear_drvdata:
+	mutex_destroy(&ionic->dev_cmd_lock);
+	ionic_devlink_free(ionic);
+
+	return err;
  }
Quoted 7 lines

drivers/net/ethernet/pensando/ionic/ionic.h

Quoted 5 lines
  static void ionic_remove(struct pci_dev *pdev)
  {
  	struct ionic *ionic = pci_get_drvdata(pdev);
drivers/net/ethernet/pensando/ionic/ionic.h +1 -0
+	if (!ionic)

How can this be NULL? Usually if this is NULL that means probe() failed but 'err' was not set properly. Perhaps WARN_ON() here?

Yes, pretty unlikely, but seems worth checking.

Quoted 120 lines
Quoted 120 lines
drivers/net/ethernet/pensando/ionic/ionic.h +11 -0
+		return;
+
+	ionic_devlink_unregister(ionic);
+	ionic_reset(ionic);
+	ionic_dev_teardown(ionic);
+	ionic_unmap_bars(ionic);
+	pci_release_regions(pdev);
+	pci_clear_master(pdev);
+	pci_disable_device(pdev);
+	ionic_debugfs_del_dev(ionic);
+	mutex_destroy(&ionic->dev_cmd_lock);
  	ionic_devlink_free(ionic);
  }
drivers/net/ethernet/pensando/ionic/ionic_dev.h new file +99 -0
new file mode 100644
index 000000000000..30a5206bba4e
@@ -0,0 +1,143 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+
+#ifndef _IONIC_DEV_H_
+#define _IONIC_DEV_H_
+
+#include <linux/mutex.h>
+#include <linux/workqueue.h>
+
+#include "ionic_if.h"
+#include "ionic_regs.h"
+
+struct ionic_dev_bar {
+	void __iomem *vaddr;
+	phys_addr_t bus_addr;
+	unsigned long len;
+	int res_index;
+};
+
+static inline void ionic_struct_size_checks(void)
+{
+	/* Registers */
+	BUILD_BUG_ON(sizeof(struct ionic_intr) != 32);
+
+	BUILD_BUG_ON(sizeof(struct ionic_doorbell) != 8);
+	BUILD_BUG_ON(sizeof(struct ionic_intr_status) != 8);
+
+	BUILD_BUG_ON(sizeof(union ionic_dev_regs) != 4096);
+	BUILD_BUG_ON(sizeof(union ionic_dev_info_regs) != 2048);
+	BUILD_BUG_ON(sizeof(union ionic_dev_cmd_regs) != 2048);
+
+	BUILD_BUG_ON(sizeof(struct ionic_lif_stats) != 1024);
+
+	BUILD_BUG_ON(sizeof(struct ionic_admin_cmd) != 64);
+	BUILD_BUG_ON(sizeof(struct ionic_admin_comp) != 16);
+	BUILD_BUG_ON(sizeof(struct ionic_nop_cmd) != 64);
+	BUILD_BUG_ON(sizeof(struct ionic_nop_comp) != 16);
+
+	/* Device commands */
+	BUILD_BUG_ON(sizeof(struct ionic_dev_identify_cmd) != 64);
+	BUILD_BUG_ON(sizeof(struct ionic_dev_identify_comp) != 16);
+	BUILD_BUG_ON(sizeof(struct ionic_dev_init_cmd) != 64);
+	BUILD_BUG_ON(sizeof(struct ionic_dev_init_comp) != 16);
+	BUILD_BUG_ON(sizeof(struct ionic_dev_reset_cmd) != 64);
+	BUILD_BUG_ON(sizeof(struct ionic_dev_reset_comp) != 16);
+	BUILD_BUG_ON(sizeof(struct ionic_dev_getattr_cmd) != 64);
+	BUILD_BUG_ON(sizeof(struct ionic_dev_getattr_comp) != 16);
+	BUILD_BUG_ON(sizeof(struct ionic_dev_setattr_cmd) != 64);
+	BUILD_BUG_ON(sizeof(struct ionic_dev_setattr_comp) != 16);
+
+	/* Port commands */
+	BUILD_BUG_ON(sizeof(struct ionic_port_identify_cmd) != 64);
+	BUILD_BUG_ON(sizeof(struct ionic_port_identify_comp) != 16);
+	BUILD_BUG_ON(sizeof(struct ionic_port_init_cmd) != 64);
+	BUILD_BUG_ON(sizeof(struct ionic_port_init_comp) != 16);
+	BUILD_BUG_ON(sizeof(struct ionic_port_reset_cmd) != 64);
+	BUILD_BUG_ON(sizeof(struct ionic_port_reset_comp) != 16);
+	BUILD_BUG_ON(sizeof(struct ionic_port_getattr_cmd) != 64);
+	BUILD_BUG_ON(sizeof(struct ionic_port_getattr_comp) != 16);
+	BUILD_BUG_ON(sizeof(struct ionic_port_setattr_cmd) != 64);
+	BUILD_BUG_ON(sizeof(struct ionic_port_setattr_comp) != 16);
+
+	/* LIF commands */
+	BUILD_BUG_ON(sizeof(struct ionic_lif_init_cmd) != 64);
+	BUILD_BUG_ON(sizeof(struct ionic_lif_init_comp) != 16);
+	BUILD_BUG_ON(sizeof(struct ionic_lif_reset_cmd) != 64);
+	BUILD_BUG_ON(sizeof(ionic_lif_reset_comp) != 16);
+	BUILD_BUG_ON(sizeof(struct ionic_lif_getattr_cmd) != 64);
+	BUILD_BUG_ON(sizeof(struct ionic_lif_getattr_comp) != 16);
+	BUILD_BUG_ON(sizeof(struct ionic_lif_setattr_cmd) != 64);
+	BUILD_BUG_ON(sizeof(struct ionic_lif_setattr_comp) != 16);
+
+	BUILD_BUG_ON(sizeof(struct ionic_q_init_cmd) != 64);
+	BUILD_BUG_ON(sizeof(struct ionic_q_init_comp) != 16);
+	BUILD_BUG_ON(sizeof(struct ionic_q_control_cmd) != 64);
+	BUILD_BUG_ON(sizeof(ionic_q_control_comp) != 16);
+
+	BUILD_BUG_ON(sizeof(struct ionic_rx_mode_set_cmd) != 64);
+	BUILD_BUG_ON(sizeof(ionic_rx_mode_set_comp) != 16);
+	BUILD_BUG_ON(sizeof(struct ionic_rx_filter_add_cmd) != 64);
+	BUILD_BUG_ON(sizeof(struct ionic_rx_filter_add_comp) != 16);
+	BUILD_BUG_ON(sizeof(struct ionic_rx_filter_del_cmd) != 64);
+	BUILD_BUG_ON(sizeof(ionic_rx_filter_del_comp) != 16);
+
+	/* RDMA commands */
+	BUILD_BUG_ON(sizeof(struct ionic_rdma_reset_cmd) != 64);
+	BUILD_BUG_ON(sizeof(struct ionic_rdma_queue_cmd) != 64);
+
+	/* Events */
+	BUILD_BUG_ON(sizeof(struct ionic_notifyq_cmd) != 4);
+	BUILD_BUG_ON(sizeof(union ionic_notifyq_comp) != 64);
+	BUILD_BUG_ON(sizeof(struct ionic_notifyq_event) != 64);
+	BUILD_BUG_ON(sizeof(struct ionic_link_change_event) != 64);
+	BUILD_BUG_ON(sizeof(struct ionic_reset_event) != 64);
+	BUILD_BUG_ON(sizeof(struct ionic_heartbeat_event) != 64);
+	BUILD_BUG_ON(sizeof(struct ionic_log_event) != 64);
+
+	/* I/O */
+	BUILD_BUG_ON(sizeof(struct ionic_txq_desc) != 16);
Quoted 8 lines

drivers/net/ethernet/pensando/ionic/ionic_dev.h line 105

Quoted 6 lines
drivers/net/ethernet/pensando/ionic/ionic_dev.h new file +6 -0
+	BUILD_BUG_ON(sizeof(struct ionic_txq_sg_desc) != 128);
+	BUILD_BUG_ON(sizeof(struct ionic_txq_comp) != 16);
+
+	BUILD_BUG_ON(sizeof(struct ionic_rxq_desc) != 16);
+	BUILD_BUG_ON(sizeof(struct ionic_rxq_sg_desc) != 128);
+	BUILD_BUG_ON(sizeof(struct ionic_rxq_comp) != 16);

static_assert() for all of those? That way you don't need this fake function.

Sure

Quoted 53 lines
Quoted 53 lines
drivers/net/ethernet/pensando/ionic/ionic_dev.h new file +38 -0
+}
+
+struct ionic_devinfo {
+	u8 asic_type;
+	u8 asic_rev;
+	char fw_version[IONIC_DEVINFO_FWVERS_BUFLEN + 1];
+	char serial_num[IONIC_DEVINFO_SERIAL_BUFLEN + 1];
+};
+
+struct ionic_dev {
+	union ionic_dev_info_regs __iomem *dev_info_regs;
+	union ionic_dev_cmd_regs __iomem *dev_cmd_regs;
+
+	u64 __iomem *db_pages;
+	dma_addr_t phy_db_pages;
+
+	struct ionic_intr __iomem *intr_ctrl;
+	u64 __iomem *intr_status;
+
+	struct ionic_devinfo dev_info;
+};
+
+struct ionic;
+
+void ionic_init_devinfo(struct ionic *ionic);
+int ionic_dev_setup(struct ionic *ionic);
+void ionic_dev_teardown(struct ionic *ionic);
+
+void ionic_dev_cmd_go(struct ionic_dev *idev, union ionic_dev_cmd *cmd);
+u8 ionic_dev_cmd_status(struct ionic_dev *idev);
+bool ionic_dev_cmd_done(struct ionic_dev *idev);
+void ionic_dev_cmd_comp(struct ionic_dev *idev, union ionic_dev_cmd_comp *comp);
+
+void ionic_dev_cmd_identify(struct ionic_dev *idev, u8 ver);
+void ionic_dev_cmd_init(struct ionic_dev *idev);
+void ionic_dev_cmd_reset(struct ionic_dev *idev);
+
+#endif /* _IONIC_DEV_H_ */
drivers/net/ethernet/pensando/ionic/ionic_devlink.c +4 -0
index e24ef6971cd5..1ca1e33cca04 100644
@@ -11,8 +11,28 @@
  static int ionic_dl_info_get(struct devlink *dl, struct devlink_info_req *req,
  			     struct netlink_ext_ack *extack)
  {
+	struct ionic *ionic = devlink_priv(dl);
+	struct ionic_dev *idev = &ionic->idev;
+	char buf[16];
+
  	devlink_info_driver_name_put(req, IONIC_DRV_NAME);
 
Quoted 4 lines

drivers/net/ethernet/pensando/ionic/ionic_devlink.c line 22

Quoted 3 lines
drivers/net/ethernet/pensando/ionic/ionic_devlink.c +3 -0
+	devlink_info_version_running_put(req,
+					 DEVLINK_INFO_VERSION_GENERIC_FW_MGMT,
+					 idev->dev_info.fw_version);

Are you sure this is not the FW that controls the data path?

There is only one FW rev to report, and this covers mgmt and data.

Quoted 8 lines

drivers/net/ethernet/pensando/ionic/ionic_devlink.c line 27

Quoted 5 lines
drivers/net/ethernet/pensando/ionic/ionic_devlink.c +4 -0
+	snprintf(buf, sizeof(buf), "0x%x", idev->dev_info.asic_type);
+	devlink_info_version_fixed_put(req,
+				       DEVLINK_INFO_VERSION_GENERIC_BOARD_ID,
+				       buf);

Board ID is not ASIC. This is for identifying a board version with all its components which surround the main ASIC.

Quoted 5 lines

drivers/net/ethernet/pensando/ionic/ionic_devlink.c line 32

Quoted 4 lines
drivers/net/ethernet/pensando/ionic/ionic_devlink.c +4 -0
+	snprintf(buf, sizeof(buf), "0x%x", idev->dev_info.asic_rev);
+	devlink_info_version_fixed_put(req,
+				       DEVLINK_INFO_VERSION_GENERIC_BOARD_REV,
+				       buf);

ditto

Since I don't have any board info available at this point, shall I use my own "asic.id" and "asic.rev" strings, or in this patch shall I add something like this to devlink.h and use them here:

/* Part number, identifier of asic design */ #define DEVLINK_INFO_VERSION_GENERIC_ASIC_ID    "asic.id" /* Revision of asic design */
#define DEVLINK_INFO_VERSION_GENERIC_ASIC_REV    "asic.rev"

Quoted 18 lines
Quoted 18 lines
drivers/net/ethernet/pensando/ionic/ionic_devlink.c +11 -0
+	devlink_info_serial_number_put(req, idev->dev_info.serial_num);
+
  	return 0;
  }
 
@@ -41,3 +61,22 @@ void ionic_devlink_free(struct ionic *ionic)
  {
  	devlink_free(ionic->dl);
  }
+
+int ionic_devlink_register(struct ionic *ionic)
+{
+	int err;
+
+	err = devlink_register(ionic->dl, ionic->dev);
+	if (err)
+		dev_warn(ionic->dev, "devlink_register failed: %d\n", err);
+
Quoted 8 lines

drivers/net/ethernet/pensando/ionic/ionic_devlink.c void ionic_devlink_free(struct ionic *ionic) · line 79

Quoted 7 lines
drivers/net/ethernet/pensando/ionic/ionic_devlink.c +7 -0
+	return err;
+}
+
+void ionic_devlink_unregister(struct ionic *ionic)
+{
+	if (!ionic || !ionic->dl)
+		return;

Impossiblu

Sure.

Thanks,
sln

#12 lore raw

Re: [PATCH v5 net-next 02/18] ionic: Add hardware init and device commands

Quoted 14 lines · Shannon Nelson
Quoted 14 lines
Quoted 14 lines
drivers/net/ethernet/pensando/ionic/ionic_devlink.c +4 -0
index e24ef6971cd5..1ca1e33cca04 100644
@@ -11,8 +11,28 @@
  static int ionic_dl_info_get(struct devlink *dl, struct devlink_info_req *req,
  			     struct netlink_ext_ack *extack)
  {
+	struct ionic *ionic = devlink_priv(dl);
+	struct ionic_dev *idev = &ionic->idev;
+	char buf[16];
+
  	devlink_info_driver_name_put(req, IONIC_DRV_NAME);
 
Quoted 6 lines
Quoted 4 lines

drivers/net/ethernet/pensando/ionic/ionic_devlink.c line 22

Quoted 3 lines
drivers/net/ethernet/pensando/ionic/ionic_devlink.c +3 -0
+	devlink_info_version_running_put(req,
+					 DEVLINK_INFO_VERSION_GENERIC_FW_MGMT,
+					 idev->dev_info.fw_version);  

Are you sure this is not the FW that controls the data path?

There is only one FW rev to report, and this covers mgmt and data.

Can you add a key for that? Cause this one clearly says management..

Quoted 14 lines
Quoted 8 lines

drivers/net/ethernet/pensando/ionic/ionic_devlink.c line 27

Quoted 5 lines
drivers/net/ethernet/pensando/ionic/ionic_devlink.c +4 -0
+	snprintf(buf, sizeof(buf), "0x%x", idev->dev_info.asic_type);
+	devlink_info_version_fixed_put(req,
+				       DEVLINK_INFO_VERSION_GENERIC_BOARD_ID,
+				       buf);  

Board ID is not ASIC. This is for identifying a board version with all its components which surround the main ASIC.

Quoted 5 lines

drivers/net/ethernet/pensando/ionic/ionic_devlink.c line 32

Quoted 4 lines
drivers/net/ethernet/pensando/ionic/ionic_devlink.c +4 -0
+	snprintf(buf, sizeof(buf), "0x%x", idev->dev_info.asic_rev);
+	devlink_info_version_fixed_put(req,
+				       DEVLINK_INFO_VERSION_GENERIC_BOARD_REV,
+				       buf);  

ditto

Quoted 8 lines

Since I don't have any board info available at this point, shall I use my own "asic.id" and "asic.rev" strings, or in this patch shall I add something like this to devlink.h and use them here:

/* Part number, identifier of asic design */ #define DEVLINK_INFO_VERSION_GENERIC_ASIC_ID    "asic.id" /* Revision of asic design */
#define DEVLINK_INFO_VERSION_GENERIC_ASIC_REV    "asic.rev"

Yes, please add these to the generic items and document appropriately.

#13 lore raw

Re: [PATCH v5 net-next 02/18] ionic: Add hardware init and device commands

Quoted 15 lines · Jakub Kicinski
Quoted 14 lines · Shannon Nelson
Quoted 14 lines
Quoted 14 lines
drivers/net/ethernet/pensando/ionic/ionic_devlink.c +4 -0
index e24ef6971cd5..1ca1e33cca04 100644
@@ -11,8 +11,28 @@
   static int ionic_dl_info_get(struct devlink *dl, struct devlink_info_req *req,
   			     struct netlink_ext_ack *extack)
   {
+	struct ionic *ionic = devlink_priv(dl);
+	struct ionic_dev *idev = &ionic->idev;
+	char buf[16];
+
   	devlink_info_driver_name_put(req, IONIC_DRV_NAME);
 
Quoted 6 lines
Quoted 5 lines
Quoted 4 lines

drivers/net/ethernet/pensando/ionic/ionic_devlink.c line 22

Quoted 3 lines
drivers/net/ethernet/pensando/ionic/ionic_devlink.c +3 -0
+	devlink_info_version_running_put(req,
+					 DEVLINK_INFO_VERSION_GENERIC_FW_MGMT,
+					 idev->dev_info.fw_version);

Are you sure this is not the FW that controls the data path?

There is only one FW rev to report, and this covers mgmt and data.

Can you add a key for that? Cause this one clearly says management..

Perhaps something like this?

/* Overall FW version */
#define DEVLINK_INFO_VERSION_GENERIC_FW    "fw"

Quoted 9 lines
Quoted 8 lines

Since I don't have any board info available at this point, shall I use my own "asic.id" and "asic.rev" strings, or in this patch shall I add something like this to devlink.h and use them here:

/* Part number, identifier of asic design */ #define DEVLINK_INFO_VERSION_GENERIC_ASIC_ID    "asic.id" /* Revision of asic design */
#define DEVLINK_INFO_VERSION_GENERIC_ASIC_REV    "asic.rev"

Yes, please add these to the generic items and document appropriately.

Sure.  Is there any place besides Documentation/networking/devlink-info-versions.rst?

sln

#14 lore raw

Re: [PATCH v5 net-next 02/18] ionic: Add hardware init and device commands

Quoted 20 lines · Shannon Nelson
Quoted 19 lines · Jakub Kicinski
Quoted 18 lines · Shannon Nelson
Quoted 18 lines
Quoted 7 lines
drivers/net/ethernet/pensando/ionic/ionic_devlink.c +0 -0
index e24ef6971cd5..1ca1e33cca04 100644
@@ -11,8 +11,28 @@
   static int ionic_dl_info_get(struct devlink *dl, struct devlink_info_req *req,
   			     struct netlink_ext_ack *extack)

drivers/net/ethernet/pensando/ionic/ionic_devlink.c line 22

Quoted 10 lines
drivers/net/ethernet/pensando/ionic/ionic_devlink.c +7 -0
   {
+	struct ionic *ionic = devlink_priv(dl);
+	struct ionic_dev *idev = &ionic->idev;
+	char buf[16];
+
   	devlink_info_driver_name_put(req, IONIC_DRV_NAME);
 
+	devlink_info_version_running_put(req,
+					 DEVLINK_INFO_VERSION_GENERIC_FW_MGMT,
+					 idev->dev_info.fw_version);  

Are you sure this is not the FW that controls the data path?

Quoted 7 lines
Quoted 2 lines
Quoted 1 line

There is only one FW rev to report, and this covers mgmt and data.

Can you add a key for that? Cause this one clearly says management..

Perhaps something like this?

/* Overall FW version */
#define DEVLINK_INFO_VERSION_GENERIC_FW    "fw"

Sounds reasonable.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

Since I don't have any board info available at this point, shall I use my own "asic.id" and "asic.rev" strings, or in this patch shall I add something like this to devlink.h and use them here:

Quoted 8 lines
Quoted 5 lines
Quoted 4 lines

/* Part number, identifier of asic design */ #define DEVLINK_INFO_VERSION_GENERIC_ASIC_ID    "asic.id" /* Revision of asic design */
#define DEVLINK_INFO_VERSION_GENERIC_ASIC_REV    "asic.rev"

Yes, please add these to the generic items and document appropriately.

Sure.  Is there any place besides Documentation/networking/devlink-info-versions.rst?

Nope, that's the one.

#15 lore raw

PATCH v5 3/18 ionic: Add port management commands

In reply to Shannon Nelson

The port management commands apply to the physical port associated with the PCI device, which might be shared among several logical interfaces.

---
 drivers/net/ethernet/pensando/ionic/ionic.h   |   4 +
 .../ethernet/pensando/ionic/ionic_bus_pci.c   |  16 +++
 .../net/ethernet/pensando/ionic/ionic_dev.c   | 116 ++++++++++++++++++
 .../net/ethernet/pensando/ionic/ionic_dev.h   |  15 +++
 .../net/ethernet/pensando/ionic/ionic_main.c  |  86 +++++++++++++
 5 files changed, 237 insertions(+)
drivers/net/ethernet/pensando/ionic/ionic.h +4 -0
index 1f3c4a916849..db2ad14899d3 100644
@@ -44,4 +44,8 @@ int ionic_identify(struct ionic *ionic);
 int ionic_init(struct ionic *ionic);
 int ionic_reset(struct ionic *ionic);
 
+int ionic_port_identify(struct ionic *ionic);
+int ionic_port_init(struct ionic *ionic);
+int ionic_port_reset(struct ionic *ionic);
+
 #endif /* _IONIC_H_ */
drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c +16 -0
index 286b4b450a73..804dd43e92a6 100644
@@ -138,12 +138,27 @@ static int ionic_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
 		goto err_out_teardown;
 	}
 
+	/* Configure the ports */
+	err = ionic_port_identify(ionic);
+	if (err) {
+		dev_err(dev, "Cannot identify port: %d, aborting\n", err);
+		goto err_out_reset;
+	}
+
+	err = ionic_port_init(ionic);
+	if (err) {
+		dev_err(dev, "Cannot init port: %d, aborting\n", err);
+		goto err_out_reset;
+	}
+
 	err = ionic_devlink_register(ionic);
 	if (err)
 		dev_err(dev, "Cannot register devlink: %d\n", err);
 
 	return 0;
 
+err_out_reset:
+	ionic_reset(ionic);
 err_out_teardown:
 	ionic_dev_teardown(ionic);
 err_out_unmap_bars:
@@ -170,6 +185,7 @@ static void ionic_remove(struct pci_dev *pdev)
 		return;
 
 	ionic_devlink_unregister(ionic);
+	ionic_port_reset(ionic);
 	ionic_reset(ionic);
 	ionic_dev_teardown(ionic);
 	ionic_unmap_bars(ionic);
drivers/net/ethernet/pensando/ionic/ionic_dev.c +116 -0

119 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_dev.h +15 -0
index 30a5206bba4e..5b83f21af18a 100644
@@ -122,6 +122,10 @@ struct ionic_dev {
 	struct ionic_intr __iomem *intr_ctrl;
 	u64 __iomem *intr_status;
 
+	u32 port_info_sz;
+	struct ionic_port_info *port_info;
+	dma_addr_t port_info_pa;
+
 	struct ionic_devinfo dev_info;
 };
 
@@ -140,4 +144,15 @@ void ionic_dev_cmd_identify(struct ionic_dev *idev, u8 ver);
 void ionic_dev_cmd_init(struct ionic_dev *idev);
 void ionic_dev_cmd_reset(struct ionic_dev *idev);
 
+void ionic_dev_cmd_port_identify(struct ionic_dev *idev);
+void ionic_dev_cmd_port_init(struct ionic_dev *idev);
+void ionic_dev_cmd_port_reset(struct ionic_dev *idev);
+void ionic_dev_cmd_port_state(struct ionic_dev *idev, u8 state);
+void ionic_dev_cmd_port_speed(struct ionic_dev *idev, u32 speed);
+void ionic_dev_cmd_port_mtu(struct ionic_dev *idev, u32 mtu);
+void ionic_dev_cmd_port_autoneg(struct ionic_dev *idev, u8 an_enable);
+void ionic_dev_cmd_port_fec(struct ionic_dev *idev, u8 fec_type);
+void ionic_dev_cmd_port_pause(struct ionic_dev *idev, u8 pause_type);
+void ionic_dev_cmd_port_loopback(struct ionic_dev *idev, u8 loopback_mode);
+
 #endif /* _IONIC_DEV_H_ */
drivers/net/ethernet/pensando/ionic/ionic_main.c +86 -0

92 lines not on this page · raw message

Signature · 1 line
2.17.1
#16 lore raw

Re: [PATCH v5 net-next 03/18] ionic: Add port management commands

Quoted 209 lines · Shannon Nelson

The port management commands apply to the physical port associated with the PCI device, which might be shared among several logical interfaces.

---
 drivers/net/ethernet/pensando/ionic/ionic.h   |   4 +
 .../ethernet/pensando/ionic/ionic_bus_pci.c   |  16 +++
 .../net/ethernet/pensando/ionic/ionic_dev.c   | 116 ++++++++++++++++++
 .../net/ethernet/pensando/ionic/ionic_dev.h   |  15 +++
 .../net/ethernet/pensando/ionic/ionic_main.c  |  86 +++++++++++++
 5 files changed, 237 insertions(+)
drivers/net/ethernet/pensando/ionic/ionic.h +4 -0
index 1f3c4a916849..db2ad14899d3 100644
@@ -44,4 +44,8 @@ int ionic_identify(struct ionic *ionic);
 int ionic_init(struct ionic *ionic);
 int ionic_reset(struct ionic *ionic);
 
+int ionic_port_identify(struct ionic *ionic);
+int ionic_port_init(struct ionic *ionic);
+int ionic_port_reset(struct ionic *ionic);
+
 #endif /* _IONIC_H_ */
drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c +16 -0
index 286b4b450a73..804dd43e92a6 100644
@@ -138,12 +138,27 @@ static int ionic_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
 		goto err_out_teardown;
 	}
 
+	/* Configure the ports */
+	err = ionic_port_identify(ionic);
+	if (err) {
+		dev_err(dev, "Cannot identify port: %d, aborting\n", err);
+		goto err_out_reset;
+	}
+
+	err = ionic_port_init(ionic);
+	if (err) {
+		dev_err(dev, "Cannot init port: %d, aborting\n", err);
+		goto err_out_reset;
+	}
+
 	err = ionic_devlink_register(ionic);
 	if (err)
 		dev_err(dev, "Cannot register devlink: %d\n", err);
 
 	return 0;
 
+err_out_reset:
+	ionic_reset(ionic);
 err_out_teardown:
 	ionic_dev_teardown(ionic);
 err_out_unmap_bars:
@@ -170,6 +185,7 @@ static void ionic_remove(struct pci_dev *pdev)
 		return;
 
 	ionic_devlink_unregister(ionic);
+	ionic_port_reset(ionic);
 	ionic_reset(ionic);
 	ionic_dev_teardown(ionic);
 	ionic_unmap_bars(ionic);
drivers/net/ethernet/pensando/ionic/ionic_dev.c +116 -0
index 0bf1bd6bd7b1..cddd41a43550 100644
@@ -134,3 +134,119 @@ void ionic_dev_cmd_reset(struct ionic_dev *idev)
 
 	ionic_dev_cmd_go(idev, &cmd);
 }
+
+/* Port commands */
+void ionic_dev_cmd_port_identify(struct ionic_dev *idev)
+{
+	union ionic_dev_cmd cmd = {
+		.port_init.opcode = IONIC_CMD_PORT_IDENTIFY,
+		.port_init.index = 0,
+	};
+
+	ionic_dev_cmd_go(idev, &cmd);
+}
+
+void ionic_dev_cmd_port_init(struct ionic_dev *idev)
+{
+	union ionic_dev_cmd cmd = {
+		.port_init.opcode = IONIC_CMD_PORT_INIT,
+		.port_init.index = 0,
+		.port_init.info_pa = cpu_to_le64(idev->port_info_pa),
+	};
+
+	ionic_dev_cmd_go(idev, &cmd);
+}
+
+void ionic_dev_cmd_port_reset(struct ionic_dev *idev)
+{
+	union ionic_dev_cmd cmd = {
+		.port_reset.opcode = IONIC_CMD_PORT_RESET,
+		.port_reset.index = 0,
+	};
+
+	ionic_dev_cmd_go(idev, &cmd);
+}
+
+void ionic_dev_cmd_port_state(struct ionic_dev *idev, u8 state)
+{
+	union ionic_dev_cmd cmd = {
+		.port_setattr.opcode = IONIC_CMD_PORT_SETATTR,
+		.port_setattr.index = 0,
+		.port_setattr.attr = IONIC_PORT_ATTR_STATE,
+		.port_setattr.state = state,
+	};
+
+	ionic_dev_cmd_go(idev, &cmd);
+}
+
+void ionic_dev_cmd_port_speed(struct ionic_dev *idev, u32 speed)
+{
+	union ionic_dev_cmd cmd = {
+		.port_setattr.opcode = IONIC_CMD_PORT_SETATTR,
+		.port_setattr.index = 0,
+		.port_setattr.attr = IONIC_PORT_ATTR_SPEED,
+		.port_setattr.speed = cpu_to_le32(speed),
+	};
+
+	ionic_dev_cmd_go(idev, &cmd);
+}
+
+void ionic_dev_cmd_port_mtu(struct ionic_dev *idev, u32 mtu)
+{
+	union ionic_dev_cmd cmd = {
+		.port_setattr.opcode = IONIC_CMD_PORT_SETATTR,
+		.port_setattr.index = 0,
+		.port_setattr.attr = IONIC_PORT_ATTR_MTU,
+		.port_setattr.mtu = cpu_to_le32(mtu),
+	};
+
+	ionic_dev_cmd_go(idev, &cmd);
+}
+
+void ionic_dev_cmd_port_autoneg(struct ionic_dev *idev, u8 an_enable)
+{
+	union ionic_dev_cmd cmd = {
+		.port_setattr.opcode = IONIC_CMD_PORT_SETATTR,
+		.port_setattr.index = 0,
+		.port_setattr.attr = IONIC_PORT_ATTR_AUTONEG,
+		.port_setattr.an_enable = an_enable,
+	};
+
+	ionic_dev_cmd_go(idev, &cmd);
+}
+
+void ionic_dev_cmd_port_fec(struct ionic_dev *idev, u8 fec_type)
+{
+	union ionic_dev_cmd cmd = {
+		.port_setattr.opcode = IONIC_CMD_PORT_SETATTR,
+		.port_setattr.index = 0,
+		.port_setattr.attr = IONIC_PORT_ATTR_FEC,
+		.port_setattr.fec_type = fec_type,
+	};
+
+	ionic_dev_cmd_go(idev, &cmd);
+}
+
+void ionic_dev_cmd_port_pause(struct ionic_dev *idev, u8 pause_type)
+{
+	union ionic_dev_cmd cmd = {
+		.port_setattr.opcode = IONIC_CMD_PORT_SETATTR,
+		.port_setattr.index = 0,
+		.port_setattr.attr = IONIC_PORT_ATTR_PAUSE,
+		.port_setattr.pause_type = pause_type,
+	};
+
+	ionic_dev_cmd_go(idev, &cmd);
+}
+
+void ionic_dev_cmd_port_loopback(struct ionic_dev *idev, u8 loopback_mode)
+{
+	union ionic_dev_cmd cmd = {
+		.port_setattr.opcode = IONIC_CMD_PORT_SETATTR,
+		.port_setattr.index = 0,
+		.port_setattr.attr = IONIC_PORT_ATTR_LOOPBACK,
+		.port_setattr.loopback_mode = loopback_mode,
+	};
+
+	ionic_dev_cmd_go(idev, &cmd);
+}
drivers/net/ethernet/pensando/ionic/ionic_dev.h +4 -0
index 30a5206bba4e..5b83f21af18a 100644
@@ -122,6 +122,10 @@ struct ionic_dev {
 	struct ionic_intr __iomem *intr_ctrl;
 	u64 __iomem *intr_status;
 
+	u32 port_info_sz;
+	struct ionic_port_info *port_info;
+	dma_addr_t port_info_pa;
+
 	struct ionic_devinfo dev_info;
 };
 
@@ -140,4 +144,15 @@ void ionic_dev_cmd_identify(struct ionic_dev *idev, u8 ver);
 void ionic_dev_cmd_init(struct ionic_dev *idev);
 void ionic_dev_cmd_reset(struct ionic_dev *idev);
 

drivers/net/ethernet/pensando/ionic/ionic_dev.h void ionic_dev_cmd_identify(struct ionic_dev *idev, u8 ver); · line 156

Quoted 10 lines
drivers/net/ethernet/pensando/ionic/ionic_dev.h +10 -0
+void ionic_dev_cmd_port_identify(struct ionic_dev *idev);
+void ionic_dev_cmd_port_init(struct ionic_dev *idev);
+void ionic_dev_cmd_port_reset(struct ionic_dev *idev);
+void ionic_dev_cmd_port_state(struct ionic_dev *idev, u8 state);
+void ionic_dev_cmd_port_speed(struct ionic_dev *idev, u32 speed);
+void ionic_dev_cmd_port_mtu(struct ionic_dev *idev, u32 mtu);
+void ionic_dev_cmd_port_autoneg(struct ionic_dev *idev, u8 an_enable);
+void ionic_dev_cmd_port_fec(struct ionic_dev *idev, u8 fec_type);
+void ionic_dev_cmd_port_pause(struct ionic_dev *idev, u8 pause_type);
+void ionic_dev_cmd_port_loopback(struct ionic_dev *idev, u8 loopback_mode);

I don't think you call most of these functions in this patch.

Quoted 53 lines
drivers/net/ethernet/pensando/ionic/ionic_dev.h +0 -0
 #endif /* _IONIC_DEV_H_ */
drivers/net/ethernet/pensando/ionic/ionic_main.c +44 -0
index f52eb6c50358..47928f184230 100644
@@ -309,6 +309,92 @@ int ionic_reset(struct ionic *ionic)
 	return err;
 }
 
+int ionic_port_identify(struct ionic *ionic)
+{
+	struct ionic_identity *ident = &ionic->ident;
+	struct ionic_dev *idev = &ionic->idev;
+	size_t sz;
+	int err;
+
+	mutex_lock(&ionic->dev_cmd_lock);
+
+	ionic_dev_cmd_port_identify(idev);
+	err = ionic_dev_cmd_wait(ionic, devcmd_timeout);
+	if (!err) {
+		sz = min(sizeof(ident->port), sizeof(idev->dev_cmd_regs->data));
+		memcpy_fromio(&ident->port, &idev->dev_cmd_regs->data, sz);
+	}
+
+	mutex_unlock(&ionic->dev_cmd_lock);
+
+	return err;
+}
+
+int ionic_port_init(struct ionic *ionic)
+{
+	struct ionic_identity *ident = &ionic->ident;
+	struct ionic_dev *idev = &ionic->idev;
+	size_t sz;
+	int err;
+
+	if (idev->port_info)
+		return 0;
+
+	idev->port_info_sz = ALIGN(sizeof(*idev->port_info), PAGE_SIZE);
+	idev->port_info = dma_alloc_coherent(ionic->dev, idev->port_info_sz,
+					     &idev->port_info_pa,
+					     GFP_KERNEL);
+	if (!idev->port_info) {
+		dev_err(ionic->dev, "Failed to allocate port info, aborting\n");
+		return -ENOMEM;
+	}
+
+	sz = min(sizeof(ident->port.config), sizeof(idev->dev_cmd_regs->data));
+
+	mutex_lock(&ionic->dev_cmd_lock);
+

drivers/net/ethernet/pensando/ionic/ionic_main.c int ionic_reset(struct ionic *ionic) · line 365

Quoted 10 lines
drivers/net/ethernet/pensando/ionic/ionic_main.c +10 -0
+	memcpy_toio(&idev->dev_cmd_regs->data, &ident->port.config, sz);
+	ionic_dev_cmd_port_init(idev);
+	err = ionic_dev_cmd_wait(ionic, devcmd_timeout);
+
+	ionic_dev_cmd_port_state(&ionic->idev, IONIC_PORT_ADMIN_STATE_UP);
+	(void)ionic_dev_cmd_wait(ionic, devcmd_timeout);
+
+	mutex_unlock(&ionic->dev_cmd_lock);
+	if (err) {
+		dev_err(ionic->dev, "Failed to init port\n");

The lifetime of port_info seems a little strange. Why is it left in place even if the command failed? Doesn't this leak memory?

drivers/net/ethernet/pensando/ionic/ionic_main.c int ionic_reset(struct ionic *ionic) · line 341

Quoted 4 lines
drivers/net/ethernet/pensando/ionic/ionic_main.c +4 -0
+		return err;
+	}
+
+	return 0;

return err; work for both paths

Quoted 7 lines
drivers/net/ethernet/pensando/ionic/ionic_main.c +7 -0
+}
+
+int ionic_port_reset(struct ionic *ionic)
+{
+	struct ionic_dev *idev = &ionic->idev;
+	int err;
+

drivers/net/ethernet/pensando/ionic/ionic_main.c int ionic_reset(struct ionic *ionic) · line 386

Quoted 10 lines
drivers/net/ethernet/pensando/ionic/ionic_main.c +10 -0
+	if (!idev->port_info)
+		return 0;
+
+	mutex_lock(&ionic->dev_cmd_lock);
+	ionic_dev_cmd_port_reset(idev);
+	err = ionic_dev_cmd_wait(ionic, devcmd_timeout);
+	mutex_unlock(&ionic->dev_cmd_lock);
+	if (err) {
+		dev_err(ionic->dev, "Failed to reset port\n");
+		return err;

Again, memory leak if command fails? (nothing frees port_info)

drivers/net/ethernet/pensando/ionic/ionic_main.c int ionic_reset(struct ionic *ionic) · line 395

Quoted 9 lines
drivers/net/ethernet/pensando/ionic/ionic_main.c +9 -0
+	}
+
+	dma_free_coherent(ionic->dev, idev->port_info_sz,
+			  idev->port_info, idev->port_info_pa);
+
+	idev->port_info = NULL;
+	idev->port_info_pa = 0;
+
+	return err;

Well, with current code err can only be 0 at this point.

Quoted 5 lines
drivers/net/ethernet/pensando/ionic/ionic_main.c +2 -0
+}
+
 static int __init ionic_init_module(void)
 {
 	pr_info("%s %s, ver %s\n",
#17 lore raw

Re: [PATCH v5 net-next 03/18] ionic: Add port management commands

Quoted 6 lines · Jakub Kicinski
Quoted 5 lines · Shannon Nelson

The port management commands apply to the physical port associated with the PCI device, which might be shared among several logical interfaces.

[...]

Quoted 19 lines
Quoted 19 lines
drivers/net/ethernet/pensando/ionic/ionic_dev.h +4 -0
index 30a5206bba4e..5b83f21af18a 100644
@@ -122,6 +122,10 @@ struct ionic_dev {
  	struct ionic_intr __iomem *intr_ctrl;
  	u64 __iomem *intr_status;
 
+	u32 port_info_sz;
+	struct ionic_port_info *port_info;
+	dma_addr_t port_info_pa;
+
  	struct ionic_devinfo dev_info;
  };
 
@@ -140,4 +144,15 @@ void ionic_dev_cmd_identify(struct ionic_dev *idev, u8 ver);
  void ionic_dev_cmd_init(struct ionic_dev *idev);
  void ionic_dev_cmd_reset(struct ionic_dev *idev);
 
Quoted 11 lines

drivers/net/ethernet/pensando/ionic/ionic_dev.h void ionic_dev_cmd_identify(struct ionic_dev *idev, u8 ver); · line 156

Quoted 10 lines
drivers/net/ethernet/pensando/ionic/ionic_dev.h +10 -0
+void ionic_dev_cmd_port_identify(struct ionic_dev *idev);
+void ionic_dev_cmd_port_init(struct ionic_dev *idev);
+void ionic_dev_cmd_port_reset(struct ionic_dev *idev);
+void ionic_dev_cmd_port_state(struct ionic_dev *idev, u8 state);
+void ionic_dev_cmd_port_speed(struct ionic_dev *idev, u32 speed);
+void ionic_dev_cmd_port_mtu(struct ionic_dev *idev, u32 mtu);
+void ionic_dev_cmd_port_autoneg(struct ionic_dev *idev, u8 an_enable);
+void ionic_dev_cmd_port_fec(struct ionic_dev *idev, u8 fec_type);
+void ionic_dev_cmd_port_pause(struct ionic_dev *idev, u8 pause_type);
+void ionic_dev_cmd_port_loopback(struct ionic_dev *idev, u8 loopback_mode);

I don't think you call most of these functions in this patch.

No, but most get used in the ethtool code added a few patches later.  The port_mtu probably won't get used, so I can pull that out.  The port_loopback will get used when I add a loopback test, but I can pull that out for now until that test is added.

Quoted 92 lines
Quoted 54 lines
drivers/net/ethernet/pensando/ionic/ionic_dev.h +0 -0
  #endif /* _IONIC_DEV_H_ */
drivers/net/ethernet/pensando/ionic/ionic_main.c +44 -0
index f52eb6c50358..47928f184230 100644
@@ -309,6 +309,92 @@ int ionic_reset(struct ionic *ionic)
  	return err;
  }
 
+int ionic_port_identify(struct ionic *ionic)
+{
+	struct ionic_identity *ident = &ionic->ident;
+	struct ionic_dev *idev = &ionic->idev;
+	size_t sz;
+	int err;
+
+	mutex_lock(&ionic->dev_cmd_lock);
+
+	ionic_dev_cmd_port_identify(idev);
+	err = ionic_dev_cmd_wait(ionic, devcmd_timeout);
+	if (!err) {
+		sz = min(sizeof(ident->port), sizeof(idev->dev_cmd_regs->data));
+		memcpy_fromio(&ident->port, &idev->dev_cmd_regs->data, sz);
+	}
+
+	mutex_unlock(&ionic->dev_cmd_lock);
+
+	return err;
+}
+
+int ionic_port_init(struct ionic *ionic)
+{
+	struct ionic_identity *ident = &ionic->ident;
+	struct ionic_dev *idev = &ionic->idev;
+	size_t sz;
+	int err;
+
+	if (idev->port_info)
+		return 0;
+
+	idev->port_info_sz = ALIGN(sizeof(*idev->port_info), PAGE_SIZE);
+	idev->port_info = dma_alloc_coherent(ionic->dev, idev->port_info_sz,
+					     &idev->port_info_pa,
+					     GFP_KERNEL);
+	if (!idev->port_info) {
+		dev_err(ionic->dev, "Failed to allocate port info, aborting\n");
+		return -ENOMEM;
+	}
+
+	sz = min(sizeof(ident->port.config), sizeof(idev->dev_cmd_regs->data));
+
+	mutex_lock(&ionic->dev_cmd_lock);
+

drivers/net/ethernet/pensando/ionic/ionic_main.c int ionic_reset(struct ionic *ionic) · line 365

Quoted 10 lines
drivers/net/ethernet/pensando/ionic/ionic_main.c +10 -0
+	memcpy_toio(&idev->dev_cmd_regs->data, &ident->port.config, sz);
+	ionic_dev_cmd_port_init(idev);
+	err = ionic_dev_cmd_wait(ionic, devcmd_timeout);
+
+	ionic_dev_cmd_port_state(&ionic->idev, IONIC_PORT_ADMIN_STATE_UP);
+	(void)ionic_dev_cmd_wait(ionic, devcmd_timeout);
+
+	mutex_unlock(&ionic->dev_cmd_lock);
+	if (err) {
+		dev_err(ionic->dev, "Failed to init port\n");

The lifetime of port_info seems a little strange. Why is it left in place even if the command failed? Doesn't this leak memory?

drivers/net/ethernet/pensando/ionic/ionic_main.c int ionic_reset(struct ionic *ionic) · line 341

Quoted 4 lines
drivers/net/ethernet/pensando/ionic/ionic_main.c +4 -0
+		return err;
+	}
+
+	return 0;

return err; work for both paths

Quoted 7 lines
drivers/net/ethernet/pensando/ionic/ionic_main.c +7 -0
+}
+
+int ionic_port_reset(struct ionic *ionic)
+{
+	struct ionic_dev *idev = &ionic->idev;
+	int err;
+

drivers/net/ethernet/pensando/ionic/ionic_main.c int ionic_reset(struct ionic *ionic) · line 386

Quoted 10 lines
drivers/net/ethernet/pensando/ionic/ionic_main.c +10 -0
+	if (!idev->port_info)
+		return 0;
+
+	mutex_lock(&ionic->dev_cmd_lock);
+	ionic_dev_cmd_port_reset(idev);
+	err = ionic_dev_cmd_wait(ionic, devcmd_timeout);
+	mutex_unlock(&ionic->dev_cmd_lock);
+	if (err) {
+		dev_err(ionic->dev, "Failed to reset port\n");
+		return err;

Again, memory leak if command fails? (nothing frees port_info)

Quoted 10 lines

drivers/net/ethernet/pensando/ionic/ionic_main.c int ionic_reset(struct ionic *ionic) · line 395

Quoted 9 lines
drivers/net/ethernet/pensando/ionic/ionic_main.c +9 -0
+	}
+
+	dma_free_coherent(ionic->dev, idev->port_info_sz,
+			  idev->port_info, idev->port_info_pa);
+
+	idev->port_info = NULL;
+	idev->port_info_pa = 0;
+
+	return err;

Well, with current code err can only be 0 at this point.

I'll revisit these bits.

sln

#18 lore raw

PATCH v5 4/18 ionic: Add basic lif support

In reply to Shannon Nelson

The LIF is the Logical Interface, which represents the external connections. The NIC can multiplex many LIFs to a single port, but in most setups, LIF0 is the primary control for the port.

---
 drivers/net/ethernet/pensando/ionic/Makefile  |   2 +-
 drivers/net/ethernet/pensando/ionic/ionic.h   |   7 +
 .../net/ethernet/pensando/ionic/ionic_bus.h   |   2 +
 .../ethernet/pensando/ionic/ionic_bus_pci.c   |  47 +++
 .../ethernet/pensando/ionic/ionic_debugfs.c   |  39 +++
 .../ethernet/pensando/ionic/ionic_debugfs.h   |   6 +
 .../net/ethernet/pensando/ionic/ionic_dev.c   |  34 ++
 .../net/ethernet/pensando/ionic/ionic_dev.h   |   7 +
 .../net/ethernet/pensando/ionic/ionic_lif.c   | 291 ++++++++++++++++++
 .../net/ethernet/pensando/ionic/ionic_lif.h   |  70 +++++
 .../net/ethernet/pensando/ionic/ionic_main.c  |   1 +
 11 files changed, 505 insertions(+), 1 deletion(-)
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_lif.c
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_lif.h
drivers/net/ethernet/pensando/ionic/Makefile +1 -1
index a23d58519c63..215ed1ea44df 100644
@@ -4,4 +4,4 @@
 obj-$(CONFIG_IONIC) := ionic.o
 
 ionic-y := ionic_main.o ionic_bus_pci.o ionic_devlink.o ionic_dev.o \
-	   ionic_debugfs.o
+	   ionic_debugfs.o ionic_lif.o
drivers/net/ethernet/pensando/ionic/ionic.h +7 -0
index db2ad14899d3..82f0d8868379 100644
@@ -34,6 +34,13 @@ struct ionic {
 	struct ionic_dev_bar bars[IONIC_BARS_MAX];
 	unsigned int num_bars;
 	struct ionic_identity ident;
+	struct list_head lifs;
+	unsigned int nnqs_per_lif;
+	unsigned int neqs_per_lif;
+	unsigned int ntxqs_per_lif;
+	unsigned int nrxqs_per_lif;
+	DECLARE_BITMAP(lifbits, IONIC_LIFS_MAX);
+	unsigned int nintrs;
 };
 
 int ionic_dev_cmd_wait(struct ionic *ionic, unsigned long max_wait);
drivers/net/ethernet/pensando/ionic/ionic_bus.h +2 -0
index 24b4c01ec03f..3b1e2d0ebf8f 100644
@@ -5,6 +5,8 @@
 #define _IONIC_BUS_H_
 
 const char *ionic_bus_info(struct ionic *ionic);
+int ionic_bus_alloc_irq_vectors(struct ionic *ionic, unsigned int nintrs);
+void ionic_bus_free_irq_vectors(struct ionic *ionic);
 int ionic_bus_register_driver(void);
 void ionic_bus_unregister_driver(void);
 
drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c +47 -0

77 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_debugfs.c +39 -0
index 89c2514a8e1e..6caa57c6d1c5 100644
@@ -1,10 +1,12 @@
 // SPDX-License-Identifier: GPL-2.0
 /* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
 
+#include <linux/pci.h>
 #include <linux/netdevice.h>
 
 #include "ionic.h"
 #include "ionic_bus.h"
+#include "ionic_lif.h"
 #include "ionic_debugfs.h"
 
 #ifdef CONFIG_DEBUG_FS
@@ -64,4 +66,41 @@ void ionic_debugfs_add_ident(struct ionic *ionic)
 			    ionic, &identity_fops) ? 0 : -EOPNOTSUPP;
 }
 
+void ionic_debugfs_add_sizes(struct ionic *ionic)
+{
+	debugfs_create_u32("nlifs", 0400, ionic->dentry,
+			   (u32 *)&ionic->ident.dev.nlifs);
+	debugfs_create_u32("nintrs", 0400, ionic->dentry, &ionic->nintrs);
+
+	debugfs_create_u32("ntxqs_per_lif", 0400, ionic->dentry,
+			   (u32 *)&ionic->ident.lif.eth.config.queue_count[IONIC_QTYPE_TXQ]);
+	debugfs_create_u32("nrxqs_per_lif", 0400, ionic->dentry,
+			   (u32 *)&ionic->ident.lif.eth.config.queue_count[IONIC_QTYPE_RXQ]);
+}
+
+static int netdev_show(struct seq_file *seq, void *v)
+{
+	struct net_device *netdev = seq->private;
+
+	seq_printf(seq, "%s\n", netdev->name);
+
+	return 0;
+}
+DEFINE_SHOW_ATTRIBUTE(netdev);
+
+void ionic_debugfs_add_lif(struct ionic_lif *lif)
+{
+	lif->dentry = debugfs_create_dir(lif->name, lif->ionic->dentry);
+	if (IS_ERR_OR_NULL(lif->dentry))
+		return;
+
+	debugfs_create_file("netdev", 0400, lif->dentry,
+			    lif->netdev, &netdev_fops);
+}
+
+void ionic_debugfs_del_lif(struct ionic_lif *lif)
+{
+	debugfs_remove_recursive(lif->dentry);
+	lif->dentry = NULL;
+}
 #endif
drivers/net/ethernet/pensando/ionic/ionic_debugfs.h +6 -0
index 7073a8b4e2f9..f742acf56adf 100644
@@ -13,12 +13,18 @@ void ionic_debugfs_destroy(void);
 void ionic_debugfs_add_dev(struct ionic *ionic);
 void ionic_debugfs_del_dev(struct ionic *ionic);
 void ionic_debugfs_add_ident(struct ionic *ionic);
+void ionic_debugfs_add_sizes(struct ionic *ionic);
+void ionic_debugfs_add_lif(struct ionic_lif *lif);
+void ionic_debugfs_del_lif(struct ionic_lif *lif);
 #else
 static inline void ionic_debugfs_create(void) { }
 static inline void ionic_debugfs_destroy(void) { }
 static inline void ionic_debugfs_add_dev(struct ionic *ionic) { }
 static inline void ionic_debugfs_del_dev(struct ionic *ionic) { }
 static inline void ionic_debugfs_add_ident(struct ionic *ionic) { }
+static inline void ionic_debugfs_add_sizes(struct ionic *ionic) { }
+static inline void ionic_debugfs_add_lif(struct ionic_lif *lif) { }
+static inline void ionic_debugfs_del_lif(struct ionic_lif *lif) { }
 #endif
 
 #endif /* _IONIC_DEBUGFS_H_ */
drivers/net/ethernet/pensando/ionic/ionic_dev.c +34 -0
index cddd41a43550..11c1ab599b88 100644
@@ -250,3 +250,37 @@ void ionic_dev_cmd_port_loopback(struct ionic_dev *idev, u8 loopback_mode)
 
 	ionic_dev_cmd_go(idev, &cmd);
 }
+
+/* LIF commands */
+void ionic_dev_cmd_lif_identify(struct ionic_dev *idev, u8 type, u8 ver)
+{
+	union ionic_dev_cmd cmd = {
+		.lif_identify.opcode = IONIC_CMD_LIF_IDENTIFY,
+		.lif_identify.type = type,
+		.lif_identify.ver = ver,
+	};
+
+	ionic_dev_cmd_go(idev, &cmd);
+}
+
+void ionic_dev_cmd_lif_init(struct ionic_dev *idev, u16 lif_index,
+			    dma_addr_t info_pa)
+{
+	union ionic_dev_cmd cmd = {
+		.lif_init.opcode = IONIC_CMD_LIF_INIT,
+		.lif_init.index = cpu_to_le16(lif_index),
+		.lif_init.info_pa = cpu_to_le64(info_pa),
+	};
+
+	ionic_dev_cmd_go(idev, &cmd);
+}
+
+void ionic_dev_cmd_lif_reset(struct ionic_dev *idev, u16 lif_index)
+{
+	union ionic_dev_cmd cmd = {
+		.lif_init.opcode = IONIC_CMD_LIF_RESET,
+		.lif_init.index = cpu_to_le16(lif_index),
+	};
+
+	ionic_dev_cmd_go(idev, &cmd);
+}
drivers/net/ethernet/pensando/ionic/ionic_dev.h +7 -0
index 5b83f21af18a..1afd53bf3e23 100644
@@ -10,6 +10,8 @@
 #include "ionic_if.h"
 #include "ionic_regs.h"
 
+#define IONIC_LIFS_MAX			1024
+
 struct ionic_dev_bar {
 	void __iomem *vaddr;
 	phys_addr_t bus_addr;
@@ -155,4 +157,9 @@ void ionic_dev_cmd_port_fec(struct ionic_dev *idev, u8 fec_type);
 void ionic_dev_cmd_port_pause(struct ionic_dev *idev, u8 pause_type);
 void ionic_dev_cmd_port_loopback(struct ionic_dev *idev, u8 loopback_mode);
 
+void ionic_dev_cmd_lif_identify(struct ionic_dev *idev, u8 type, u8 ver);
+void ionic_dev_cmd_lif_init(struct ionic_dev *idev, u16 lif_index,
+			    dma_addr_t addr);
+void ionic_dev_cmd_lif_reset(struct ionic_dev *idev, u16 lif_index);
+
 #endif /* _IONIC_DEV_H_ */
drivers/net/ethernet/pensando/ionic/ionic_lif.c new file +291 -0

291 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_lif.h new file +70 -0

70 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_main.c +1 -0
index 47928f184230..2d06cf2c4fe7 100644
@@ -8,6 +8,7 @@
 
 #include "ionic.h"
 #include "ionic_bus.h"
+#include "ionic_lif.h"
 #include "ionic_debugfs.h"
 
 MODULE_DESCRIPTION(IONIC_DRV_DESCRIPTION);
Signature · 1 line
2.17.1
#19 lore raw

Re: [PATCH v5 net-next 04/18] ionic: Add basic lif support

Quoted 14 lines · Shannon Nelson
drivers/net/ethernet/pensando/ionic/ionic_lif.h new file +14 -0
+static inline bool ionic_is_pf(struct ionic *ionic)
+{
+	return ionic->pdev &&
+	       ionic->pdev->device == PCI_DEVICE_ID_PENSANDO_IONIC_ETH_PF;
+}
+
+static inline bool ionic_is_vf(struct ionic *ionic)
+{
+	return ionic->pdev &&
+	       ionic->pdev->device == PCI_DEVICE_ID_PENSANDO_IONIC_ETH_VF;
+}
+
+static inline bool ionic_is_25g(struct ionic *ionic)
+{

drivers/net/ethernet/pensando/ionic/ionic_lif.h line 60

Quoted 10 lines
drivers/net/ethernet/pensando/ionic/ionic_lif.h new file +10 -0
+	return ionic_is_pf(ionic) &&
+	       ionic->pdev->subsystem_device == IONIC_SUBDEV_ID_NAPLES_25;
+}
+
+static inline bool ionic_is_100g(struct ionic *ionic)
+{
+	return ionic_is_pf(ionic) &&
+	       (ionic->pdev->subsystem_device == IONIC_SUBDEV_ID_NAPLES_100_4 ||
+		ionic->pdev->subsystem_device == IONIC_SUBDEV_ID_NAPLES_100_8);
+}

Again, a bunch of unused stuff.

#20 lore raw

Re: [PATCH v5 net-next 04/18] ionic: Add basic lif support

Quoted 18 lines · Jakub Kicinski
Quoted 17 lines · Shannon Nelson
drivers/net/ethernet/pensando/ionic/ionic_lif.h new file +17 -0
+static inline bool ionic_is_pf(struct ionic *ionic)
+{
+	return ionic->pdev &&
+	       ionic->pdev->device == PCI_DEVICE_ID_PENSANDO_IONIC_ETH_PF;
+}
+
+static inline bool ionic_is_vf(struct ionic *ionic)
+{
+	return ionic->pdev &&
+	       ionic->pdev->device == PCI_DEVICE_ID_PENSANDO_IONIC_ETH_VF;
+}
+
+static inline bool ionic_is_25g(struct ionic *ionic)
+{
+	return ionic_is_pf(ionic) &&
+	       ionic->pdev->subsystem_device == IONIC_SUBDEV_ID_NAPLES_25;
+}
Quoted 8 lines

drivers/net/ethernet/pensando/ionic/ionic_lif.h line 60

Quoted 7 lines
drivers/net/ethernet/pensando/ionic/ionic_lif.h new file +7 -0
+
+static inline bool ionic_is_100g(struct ionic *ionic)
+{
+	return ionic_is_pf(ionic) &&
+	       (ionic->pdev->subsystem_device == IONIC_SUBDEV_ID_NAPLES_100_4 ||
+		ionic->pdev->subsystem_device == IONIC_SUBDEV_ID_NAPLES_100_8);
+}

Again, a bunch of unused stuff.

More "near future" support code that didn't get stripped.  I'll pull it out for now.

sln

#21 lore raw

PATCH v5 5/18 ionic: Add interrupts and doorbells

In reply to Shannon Nelson

The ionic interrupt model is based on interrupt control blocks accessed through the PCI BAR. Doorbell registers are used by the driver to signal to the NIC that requests are waiting on the message queues. Interrupts are used by the NIC to signal to the driver that answers are waiting on the completion queues.

---
 drivers/net/ethernet/pensando/ionic/ionic.h   |  3 ++
 .../net/ethernet/pensando/ionic/ionic_bus.h   |  2 +
 .../ethernet/pensando/ionic/ionic_bus_pci.c   | 12 ++++++
 .../net/ethernet/pensando/ionic/ionic_dev.c   |  6 +++
 .../net/ethernet/pensando/ionic/ionic_dev.h   | 22 +++++++++++
 .../net/ethernet/pensando/ionic/ionic_lif.c   | 39 +++++++++++++++++++
 .../net/ethernet/pensando/ionic/ionic_lif.h   |  4 ++
 .../net/ethernet/pensando/ionic/ionic_regs.h  |  3 ++
 8 files changed, 91 insertions(+)
drivers/net/ethernet/pensando/ionic/ionic.h +3 -0
index 82f0d8868379..7b4d9ff4721f 100644
@@ -4,6 +4,8 @@
 #ifndef _IONIC_H_
 #define _IONIC_H_
 
+struct ionic_lif;
+
 #include "ionic_if.h"
 #include "ionic_dev.h"
 #include "ionic_devlink.h"
@@ -41,6 +43,7 @@ struct ionic {
 	unsigned int nrxqs_per_lif;
 	DECLARE_BITMAP(lifbits, IONIC_LIFS_MAX);
 	unsigned int nintrs;
+	DECLARE_BITMAP(intrs, IONIC_INTR_CTRL_REGS_MAX);
 };
 
 int ionic_dev_cmd_wait(struct ionic *ionic, unsigned long max_wait);
drivers/net/ethernet/pensando/ionic/ionic_bus.h +2 -0
index 3b1e2d0ebf8f..6b29e94f81d6 100644
@@ -9,5 +9,7 @@ int ionic_bus_alloc_irq_vectors(struct ionic *ionic, unsigned int nintrs);
 void ionic_bus_free_irq_vectors(struct ionic *ionic);
 int ionic_bus_register_driver(void);
 void ionic_bus_unregister_driver(void);
+void __iomem *ionic_bus_map_dbpage(struct ionic *ionic, int page_num);
+void ionic_bus_unmap_dbpage(struct ionic *ionic, void __iomem *page);
 
 #endif /* _IONIC_BUS_H_ */
drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c +12 -0
index f0e0daee45bc..4f08d915c3d2 100644
@@ -87,6 +87,18 @@ static void ionic_unmap_bars(struct ionic *ionic)
 	}
 }
 
+void __iomem *ionic_bus_map_dbpage(struct ionic *ionic, int page_num)
+{
+	return pci_iomap_range(ionic->pdev,
+			       ionic->bars[IONIC_PCI_BAR_DBELL].res_index,
+			       (u64)page_num << PAGE_SHIFT, PAGE_SIZE);
+}
+
+void ionic_bus_unmap_dbpage(struct ionic *ionic, void __iomem *page)
+{
+	iounmap(page);
+}
+
 static int ionic_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
 {
 	struct device *dev = &pdev->dev;
drivers/net/ethernet/pensando/ionic/ionic_dev.c +6 -0
index 11c1ab599b88..26a2142252dd 100644
@@ -9,6 +9,7 @@
 #include <linux/etherdevice.h>
 #include "ionic.h"
 #include "ionic_dev.h"
+#include "ionic_lif.h"
 
 void ionic_init_devinfo(struct ionic *ionic)
 {
@@ -284,3 +285,8 @@ void ionic_dev_cmd_lif_reset(struct ionic_dev *idev, u16 lif_index)
 
 	ionic_dev_cmd_go(idev, &cmd);
 }
+
+int ionic_db_page_num(struct ionic_lif *lif, int pid)
+{
+	return (lif->hw_index * lif->dbid_count) + pid;
+}
drivers/net/ethernet/pensando/ionic/ionic_dev.h +22 -0
index 1afd53bf3e23..a69d8359eead 100644
@@ -131,8 +131,28 @@ struct ionic_dev {
 	struct ionic_devinfo dev_info;
 };
 
+#define INTR_INDEX_NOT_ASSIGNED		-1
+#define INTR_NAME_MAX_SZ		32
+
+struct ionic_intr_info {
+	char name[INTR_NAME_MAX_SZ];
+	unsigned int index;
+	unsigned int vector;
+	u64 rearm_count;
+	unsigned int cpu;
+	cpumask_t affinity_mask;
+};
+
 struct ionic;
 
+static inline void ionic_intr_init(struct ionic_dev *idev,
+				   struct ionic_intr_info *intr,
+				   unsigned long index)
+{
+	ionic_intr_clean(idev->intr_ctrl, index);
+	intr->index = index;
+}
+
 void ionic_init_devinfo(struct ionic *ionic);
 int ionic_dev_setup(struct ionic *ionic);
 void ionic_dev_teardown(struct ionic *ionic);
@@ -162,4 +182,6 @@ void ionic_dev_cmd_lif_init(struct ionic_dev *idev, u16 lif_index,
 			    dma_addr_t addr);
 void ionic_dev_cmd_lif_reset(struct ionic_dev *idev, u16 lif_index);
 
+int ionic_db_page_num(struct ionic_lif *lif, int pid);
+
 #endif /* _IONIC_DEV_H_ */
drivers/net/ethernet/pensando/ionic/ionic_lif.c +39 -0
index 6044dc4062c2..88effa739c70 100644
@@ -94,6 +94,12 @@ static void ionic_lif_free(struct ionic_lif *lif)
 	lif->info = NULL;
 	lif->info_pa = 0;
 
+	/* unmap doorbell page */
+	ionic_bus_unmap_dbpage(lif->ionic, lif->kern_dbpage);
+	lif->kern_dbpage = NULL;
+	kfree(lif->dbid_inuse);
+	lif->dbid_inuse = NULL;
+
 	/* free netdev & lif */
 	ionic_debugfs_del_lif(lif);
 	list_del(&lif->list);
@@ -136,7 +142,9 @@ void ionic_lifs_deinit(struct ionic *ionic)
 static int ionic_lif_init(struct ionic_lif *lif)
 {
 	struct ionic_dev *idev = &lif->ionic->idev;
+	struct device *dev = lif->ionic->dev;
 	struct ionic_lif_init_comp comp;
+	int dbpage_num;
 	int err;
 
 	ionic_debugfs_add_lif(lif);
@@ -151,9 +159,40 @@ static int ionic_lif_init(struct ionic_lif *lif)
 
 	lif->hw_index = le16_to_cpu(comp.hw_index);
 
+	/* now that we have the hw_index we can figure out our doorbell page */
+	lif->dbid_count = le32_to_cpu(lif->ionic->ident.dev.ndbpgs_per_lif);
+	if (!lif->dbid_count) {
+		dev_err(dev, "No doorbell pages, aborting\n");
+		return -EINVAL;
+	}
+
+	lif->dbid_inuse = bitmap_alloc(lif->dbid_count, GFP_KERNEL);
+	if (!lif->dbid_inuse) {
+		dev_err(dev, "Failed alloc doorbell id bitmap, aborting\n");
+		return -ENOMEM;
+	}
+
+	/* first doorbell id reserved for kernel (dbid aka pid == zero) */
+	set_bit(0, lif->dbid_inuse);
+	lif->kern_pid = 0;
+
+	dbpage_num = ionic_db_page_num(lif, lif->kern_pid);
+	lif->kern_dbpage = ionic_bus_map_dbpage(lif->ionic, dbpage_num);
+	if (!lif->kern_dbpage) {
+		dev_err(dev, "Cannot map dbpage, aborting\n");
+		err = -ENOMEM;
+		goto err_out_free_dbid;
+	}
+
 	set_bit(IONIC_LIF_INITED, lif->state);
 
 	return 0;
+
+err_out_free_dbid:
+	kfree(lif->dbid_inuse);
+	lif->dbid_inuse = NULL;
+
+	return err;
 }
 
 int ionic_lifs_init(struct ionic *ionic)
drivers/net/ethernet/pensando/ionic/ionic_lif.h +4 -0
index cc6f907b30db..d96d350bcfd0 100644
@@ -23,6 +23,8 @@ struct ionic_lif {
 	bool registered;
 	unsigned int index;
 	unsigned int hw_index;
+	unsigned int kern_pid;
+	u64 __iomem *kern_dbpage;
 	unsigned int neqs;
 	unsigned int nxqs;
 
@@ -30,6 +32,8 @@ struct ionic_lif {
 	dma_addr_t info_pa;
 	u32 info_sz;
 
+	unsigned long *dbid_inuse;
+	unsigned int dbid_count;
 	struct dentry *dentry;
 	u32 flags;
 };
drivers/net/ethernet/pensando/ionic/ionic_regs.h +3 -0
index 3523915061ed..03ee5a36472b 100644
@@ -22,6 +22,9 @@ struct ionic_intr {
 	u32 rsvd[3];
 };
 
+#define IONIC_INTR_CTRL_REGS_MAX	2048
+#define IONIC_INTR_CTRL_COAL_MAX	0x3F
+
 /** enum ionic_intr_mask_vals - valid values for mask and mask_assert.
  * @IONIC_INTR_MASK_CLEAR:	unmask interrupt.
  * @IONIC_INTR_MASK_SET:	mask interrupt.
Signature · 1 line
2.17.1
#22 lore raw

PATCH v5 6/18 ionic: Add basic adminq support

In reply to Shannon Nelson

Most of the NIC configuration happens through the AdminQ message queue. NAPI is used for basic interrupt handling and message queue management. These routines are set up to be shared among different types of queues when used in slow-path handling.

---
 drivers/net/ethernet/pensando/ionic/ionic.h   |   3 +
 .../net/ethernet/pensando/ionic/ionic_bus.h   |   1 +
 .../ethernet/pensando/ionic/ionic_bus_pci.c   |   5 +
 .../ethernet/pensando/ionic/ionic_debugfs.c   | 147 ++++++++
 .../ethernet/pensando/ionic/ionic_debugfs.h   |   4 +
 .../net/ethernet/pensando/ionic/ionic_dev.c   | 232 ++++++++++++
 .../net/ethernet/pensando/ionic/ionic_dev.h   | 109 ++++++
 .../net/ethernet/pensando/ionic/ionic_lif.c   | 357 ++++++++++++++++++
 .../net/ethernet/pensando/ionic/ionic_lif.h   |  67 ++++
 .../net/ethernet/pensando/ionic/ionic_main.c  |  26 ++
 10 files changed, 951 insertions(+)
drivers/net/ethernet/pensando/ionic/ionic.h +3 -0
index 7b4d9ff4721f..3e39bd76f3c8 100644
@@ -46,6 +46,9 @@ struct ionic {
 	DECLARE_BITMAP(intrs, IONIC_INTR_CTRL_REGS_MAX);
 };
 
+int ionic_napi(struct napi_struct *napi, int budget, ionic_cq_cb cb,
+	       ionic_cq_done_cb done_cb, void *done_arg);
+
 int ionic_dev_cmd_wait(struct ionic *ionic, unsigned long max_wait);
 int ionic_set_dma_mask(struct ionic *ionic);
 int ionic_setup(struct ionic *ionic);
drivers/net/ethernet/pensando/ionic/ionic_bus.h +1 -0
index 6b29e94f81d6..2f4d08c64910 100644
@@ -4,6 +4,7 @@
 #ifndef _IONIC_BUS_H_
 #define _IONIC_BUS_H_
 
+int ionic_bus_get_irq(struct ionic *ionic, unsigned int num);
 const char *ionic_bus_info(struct ionic *ionic);
 int ionic_bus_alloc_irq_vectors(struct ionic *ionic, unsigned int nintrs);
 void ionic_bus_free_irq_vectors(struct ionic *ionic);
drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c +5 -0
index 4f08d915c3d2..7b6190b96a46 100644
@@ -19,6 +19,11 @@ static const struct pci_device_id ionic_id_table[] = {
 };
 MODULE_DEVICE_TABLE(pci, ionic_id_table);
 
+int ionic_bus_get_irq(struct ionic *ionic, unsigned int num)
+{
+	return pci_irq_vector(ionic->pdev, num);
+}
+
 const char *ionic_bus_info(struct ionic *ionic)
 {
 	return pci_name(ionic->pdev);
drivers/net/ethernet/pensando/ionic/ionic_debugfs.c +147 -0

157 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_debugfs.h +4 -0
index f742acf56adf..c44ebde170b6 100644
@@ -15,7 +15,9 @@ void ionic_debugfs_del_dev(struct ionic *ionic);
 void ionic_debugfs_add_ident(struct ionic *ionic);
 void ionic_debugfs_add_sizes(struct ionic *ionic);
 void ionic_debugfs_add_lif(struct ionic_lif *lif);
+void ionic_debugfs_add_qcq(struct ionic_lif *lif, struct ionic_qcq *qcq);
 void ionic_debugfs_del_lif(struct ionic_lif *lif);
+void ionic_debugfs_del_qcq(struct ionic_qcq *qcq);
 #else
 static inline void ionic_debugfs_create(void) { }
 static inline void ionic_debugfs_destroy(void) { }
@@ -24,7 +26,9 @@ static inline void ionic_debugfs_del_dev(struct ionic *ionic) { }
 static inline void ionic_debugfs_add_ident(struct ionic *ionic) { }
 static inline void ionic_debugfs_add_sizes(struct ionic *ionic) { }
 static inline void ionic_debugfs_add_lif(struct ionic_lif *lif) { }
+static inline void ionic_debugfs_add_qcq(struct ionic_lif *lif, struct ionic_qcq *qcq) { }
 static inline void ionic_debugfs_del_lif(struct ionic_lif *lif) { }
+static inline void ionic_debugfs_del_qcq(struct ionic_qcq *qcq) { }
 #endif
 
 #endif /* _IONIC_DEBUGFS_H_ */
drivers/net/ethernet/pensando/ionic/ionic_dev.c +232 -0

239 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_dev.h +109 -0

134 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_lif.c +357 -0

407 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_lif.h +67 -0

83 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_main.c +26 -0
index 2d06cf2c4fe7..a9d2bac7e02d 100644
@@ -169,6 +169,32 @@ static const char *ionic_opcode_to_str(enum ionic_cmd_opcode opcode)
 	}
 }
 
+int ionic_napi(struct napi_struct *napi, int budget, ionic_cq_cb cb,
+	       ionic_cq_done_cb done_cb, void *done_arg)
+{
+	struct ionic_qcq *qcq = napi_to_qcq(napi);
+	struct ionic_cq *cq = &qcq->cq;
+	u32 work_done, flags = 0;
+
+	work_done = ionic_cq_service(cq, budget, cb, done_cb, done_arg);
+
+	if (work_done < budget && napi_complete_done(napi, work_done)) {
+		flags |= IONIC_INTR_CRED_UNMASK;
+		DEBUG_STATS_INTR_REARM(cq->bound_intr);
+	}
+
+	if (work_done || flags) {
+		flags |= IONIC_INTR_CRED_RESET_COALESCE;
+		ionic_intr_credits(cq->lif->ionic->idev.intr_ctrl,
+				   cq->bound_intr->index,
+				   work_done, flags);
+	}
+
+	DEBUG_STATS_NAPI_POLL(qcq, work_done);
+
+	return work_done;
+}
+
 int ionic_dev_cmd_wait(struct ionic *ionic, unsigned long max_seconds)
 {
 	struct ionic_dev *idev = &ionic->idev;
Signature · 1 line
2.17.1
#23 lore raw

PATCH v5 7/18 ionic: Add adminq action

In reply to Shannon Nelson

Add AdminQ specific message requests and completion handling.

---
 drivers/net/ethernet/pensando/ionic/ionic.h   |   7 ++
 .../net/ethernet/pensando/ionic/ionic_main.c  | 111 ++++++++++++++++++
 2 files changed, 118 insertions(+)
drivers/net/ethernet/pensando/ionic/ionic.h +7 -0
index 3e39bd76f3c8..5a705233449a 100644
@@ -46,9 +46,16 @@ struct ionic {
 	DECLARE_BITMAP(intrs, IONIC_INTR_CTRL_REGS_MAX);
 };
 
+struct ionic_admin_ctx {
+	struct completion work;
+	union ionic_adminq_cmd cmd;
+	union ionic_adminq_comp comp;
+};
+
 int ionic_napi(struct napi_struct *napi, int budget, ionic_cq_cb cb,
 	       ionic_cq_done_cb done_cb, void *done_arg);
 
+int ionic_adminq_post_wait(struct ionic_lif *lif, struct ionic_admin_ctx *ctx);
 int ionic_dev_cmd_wait(struct ionic *ionic, unsigned long max_wait);
 int ionic_set_dma_mask(struct ionic *ionic);
 int ionic_setup(struct ionic *ionic);
drivers/net/ethernet/pensando/ionic/ionic_main.c +111 -0

117 lines not on this page · raw message

Signature · 1 line
2.17.1
#24 lore raw

PATCH v5 8/18 ionic: Add notifyq support

In reply to Shannon Nelson

The AdminQ is fine for sending messages and requests to the NIC, but we also need to have events published from the NIC to the driver. The NotifyQ handles this for us, using the same interrupt as AdminQ.

---
 .../ethernet/pensando/ionic/ionic_debugfs.c   |  16 ++
 .../net/ethernet/pensando/ionic/ionic_lif.c   | 169 +++++++++++++++++-
 .../net/ethernet/pensando/ionic/ionic_lif.h   |   4 +
 3 files changed, 188 insertions(+), 1 deletion(-)
drivers/net/ethernet/pensando/ionic/ionic_debugfs.c +16 -0
index f925b473ca45..d7ee60f0b19c 100644
@@ -126,6 +126,7 @@ void ionic_debugfs_add_qcq(struct ionic_lif *lif, struct ionic_qcq *qcq)
 	struct device *dev = lif->ionic->dev;
 	struct ionic_queue *q = &qcq->q;
 	struct ionic_cq *cq = &qcq->cq;
+	struct dentry *stats_dentry;
 
 	qcq_dentry = debugfs_create_dir(q->name, lif->dentry);
 	if (IS_ERR_OR_NULL(qcq_dentry))
@@ -216,6 +217,21 @@ void ionic_debugfs_add_qcq(struct ionic_lif *lif, struct ionic_qcq *qcq)
 		debugfs_create_regset32("intr_ctrl", 0400, intr_dentry,
 					intr_ctrl_regset);
 	}
+
+	if (qcq->flags & IONIC_QCQ_F_NOTIFYQ) {
+		stats_dentry = debugfs_create_dir("notifyblock", qcq_dentry);
+		if (IS_ERR_OR_NULL(stats_dentry))
+			return;
+
+		debugfs_create_u64("eid", 0400, stats_dentry,
+				   (u64 *)&lif->info->status.eid);
+		debugfs_create_u16("link_status", 0400, stats_dentry,
+				   (u16 *)&lif->info->status.link_status);
+		debugfs_create_u32("link_speed", 0400, stats_dentry,
+				   (u32 *)&lif->info->status.link_speed);
+		debugfs_create_u16("link_down_count", 0400, stats_dentry,
+				   (u16 *)&lif->info->status.link_down_count);
+	}
 }
 
 static int netdev_show(struct seq_file *seq, void *v)
drivers/net/ethernet/pensando/ionic/ionic_lif.c +168 -1

216 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_lif.h +4 -0
index f751274bb1b1..397a4d388df1 100644
@@ -7,6 +7,7 @@
 #include <linux/pci.h>
 
 #define IONIC_ADMINQ_LENGTH	16	/* must be a power of two */
+#define IONIC_NOTIFYQ_LENGTH	64	/* must be a power of two */
 
 #define IONIC_MAX_NUM_NAPI_CNTR		(NAPI_POLL_WEIGHT + 1)
 #define IONIC_MAX_NUM_SG_CNTR		(IONIC_TX_MAX_SG_ELEMS + 1)
@@ -24,6 +25,7 @@ struct ionic_rx_stats {
 #define IONIC_QCQ_F_INITED		BIT(0)
 #define IONIC_QCQ_F_SG			BIT(1)
 #define IONIC_QCQ_F_INTR		BIT(2)
+#define IONIC_QCQ_F_NOTIFYQ		BIT(5)
 
 struct ionic_napi_stats {
 	u64 poll_count;
@@ -76,6 +78,8 @@ struct ionic_lif {
 	u64 __iomem *kern_dbpage;
 	spinlock_t adminq_lock;		/* lock for AdminQ operations */
 	struct ionic_qcq *adminqcq;
+	struct ionic_qcq *notifyqcq;
+	u64 last_eid;
 	unsigned int neqs;
 	unsigned int nxqs;
 
Signature · 1 line
2.17.1
#25 lore raw

PATCH v5 9/18 ionic: Add the basic NDO callbacks for netdev support

In reply to Shannon Nelson

Set up the initial NDO structure and callbacks for netdev to use, and register the netdev. This will allow us to do a few basic operations on the device, but no traffic yet.

---
 drivers/net/ethernet/pensando/ionic/ionic.h   |   1 +
 .../ethernet/pensando/ionic/ionic_bus_pci.c   |  15 +-
 .../net/ethernet/pensando/ionic/ionic_dev.h   |   2 +
 .../ethernet/pensando/ionic/ionic_devlink.c   |  15 +-
 .../net/ethernet/pensando/ionic/ionic_lif.c   | 335 ++++++++++++++++++
 .../net/ethernet/pensando/ionic/ionic_lif.h   |  20 ++
 6 files changed, 386 insertions(+), 2 deletions(-)
drivers/net/ethernet/pensando/ionic/ionic.h +1 -0
index 5a705233449a..82b45e3384c3 100644
@@ -37,6 +37,7 @@ struct ionic {
 	unsigned int num_bars;
 	struct ionic_identity ident;
 	struct list_head lifs;
+	struct ionic_lif *master_lif;
 	unsigned int nnqs_per_lif;
 	unsigned int neqs_per_lif;
 	unsigned int ntxqs_per_lif;
drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c +14 -1
index 7b6190b96a46..9a9ab8cb2cb3 100644
@@ -206,12 +206,24 @@ static int ionic_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
 		goto err_out_free_lifs;
 	}
 
+	err = ionic_lifs_register(ionic);
+	if (err) {
+		dev_err(dev, "Cannot register LIFs: %d, aborting\n", err);
+		goto err_out_deinit_lifs;
+	}
+
 	err = ionic_devlink_register(ionic);
-	if (err)
+	if (err) {
 		dev_err(dev, "Cannot register devlink: %d\n", err);
+		goto err_out_deregister_lifs;
+	}
 
 	return 0;
 
+err_out_deregister_lifs:
+	ionic_lifs_unregister(ionic);
+err_out_deinit_lifs:
+	ionic_lifs_deinit(ionic);
 err_out_free_lifs:
 	ionic_lifs_free(ionic);
 err_out_free_irqs:
@@ -246,6 +258,7 @@ static void ionic_remove(struct pci_dev *pdev)
 		return;
 
 	ionic_devlink_unregister(ionic);
+	ionic_lifs_unregister(ionic);
 	ionic_lifs_deinit(ionic);
 	ionic_lifs_free(ionic);
 	ionic_bus_free_irq_vectors(ionic);
drivers/net/ethernet/pensando/ionic/ionic_dev.h +2 -0
index d7bc6f1f6019..f6cce42f0228 100644
@@ -10,6 +10,8 @@
 #include "ionic_if.h"
 #include "ionic_regs.h"
 
+#define IONIC_MIN_MTU			ETH_MIN_MTU
+#define IONIC_MAX_MTU			9194
 #define IONIC_LIFS_MAX			1024
 
 struct ionic_dev_bar {
drivers/net/ethernet/pensando/ionic/ionic_devlink.c +14 -1
index 1ca1e33cca04..a93df132f17b 100644
@@ -6,6 +6,7 @@
 
 #include "ionic.h"
 #include "ionic_bus.h"
+#include "ionic_lif.h"
 #include "ionic_devlink.h"
 
 static int ionic_dl_info_get(struct devlink *dl, struct devlink_info_req *req,
@@ -67,8 +68,19 @@ int ionic_devlink_register(struct ionic *ionic)
 	int err;
 
 	err = devlink_register(ionic->dl, ionic->dev);
-	if (err)
+	if (err) {
 		dev_warn(ionic->dev, "devlink_register failed: %d\n", err);
+		return err;
+	}
+
+	devlink_port_attrs_set(&ionic->dl_port, DEVLINK_PORT_FLAVOUR_PHYSICAL,
+			       0, false, 0, NULL, 0);
+	err = devlink_port_register(ionic->dl, &ionic->dl_port, 0);
+	if (err)
+		dev_err(ionic->dev, "devlink_port_register failed: %d\n", err);
+	else
+		devlink_port_type_eth_set(&ionic->dl_port,
+					  ionic->master_lif->netdev);
 
 	return err;
 }
@@ -78,5 +90,6 @@ void ionic_devlink_unregister(struct ionic *ionic)
 	if (!ionic || !ionic->dl)
 		return;
 
+	devlink_port_unregister(&ionic->dl_port);
 	devlink_unregister(ionic->dl);
 }
drivers/net/ethernet/pensando/ionic/ionic_lif.c +335 -0

359 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_lif.h +20 -0
index 397a4d388df1..866411a7a1fe 100644
@@ -59,6 +59,8 @@ struct ionic_qcq {
 
 enum ionic_lif_state_flags {
 	IONIC_LIF_INITED,
+	IONIC_LIF_UP,
+	IONIC_LIF_QUEUE_RESET,
 
 	/* leave this as last */
 	IONIC_LIF_STATE_SIZE
@@ -82,6 +84,7 @@ struct ionic_lif {
 	u64 last_eid;
 	unsigned int neqs;
 	unsigned int nxqs;
+	u64 hw_features;
 
 	struct ionic_lif_info *info;
 	dma_addr_t info_pa;
@@ -93,6 +96,17 @@ struct ionic_lif {
 	u32 flags;
 };
 
+static inline int ionic_wait_for_bit(struct ionic_lif *lif, int bitname)
+{
+	unsigned long tlimit = jiffies + HZ;
+
+	while (test_and_set_bit(bitname, lif->state) &&
+	       time_before(jiffies, tlimit))
+		usleep_range(100, 200);
+
+	return test_bit(bitname, lif->state);
+}
+
 static inline bool ionic_is_pf(struct ionic *ionic)
 {
 	return ionic->pdev &&
@@ -122,10 +136,16 @@ int ionic_lifs_alloc(struct ionic *ionic);
 void ionic_lifs_free(struct ionic *ionic);
 void ionic_lifs_deinit(struct ionic *ionic);
 int ionic_lifs_init(struct ionic *ionic);
+int ionic_lifs_register(struct ionic *ionic);
+void ionic_lifs_unregister(struct ionic *ionic);
 int ionic_lif_identify(struct ionic *ionic, u8 lif_type,
 		       union ionic_lif_identity *lif_ident);
 int ionic_lifs_size(struct ionic *ionic);
 
+int ionic_open(struct net_device *netdev);
+int ionic_stop(struct net_device *netdev);
+int ionic_reset_queues(struct ionic_lif *lif);
+
 static inline void debug_stats_napi_poll(struct ionic_qcq *qcq,
 					 unsigned int work_done)
 {
Signature · 1 line
2.17.1
#26 lore raw

PATCH v5 10/18 ionic: Add management of rx filters

In reply to Shannon Nelson

Set up the infrastructure for managing Rx filters. We can't ask the hardware for what filters it has, so we keep a local list of filters that we've pushed into the HW.

---
 drivers/net/ethernet/pensando/ionic/Makefile  |   2 +-
 .../net/ethernet/pensando/ionic/ionic_lif.c   |   6 +
 .../net/ethernet/pensando/ionic/ionic_lif.h   |   2 +
 .../ethernet/pensando/ionic/ionic_rx_filter.c | 150 ++++++++++++++++++
 .../ethernet/pensando/ionic/ionic_rx_filter.h |  35 ++++
 5 files changed, 194 insertions(+), 1 deletion(-)
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_rx_filter.c
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_rx_filter.h
drivers/net/ethernet/pensando/ionic/Makefile +1 -1
index 215ed1ea44df..8c31a90830cf 100644
@@ -4,4 +4,4 @@
 obj-$(CONFIG_IONIC) := ionic.o
 
 ionic-y := ionic_main.o ionic_bus_pci.o ionic_devlink.o ionic_dev.o \
-	   ionic_debugfs.o ionic_lif.o
+	   ionic_debugfs.o ionic_lif.o ionic_rx_filter.o
drivers/net/ethernet/pensando/ionic/ionic_lif.c +6 -0
index f1c152dd4559..373b93403190 100644
@@ -831,6 +831,8 @@ static void ionic_lif_deinit(struct ionic_lif *lif)
 
 	clear_bit(IONIC_LIF_INITED, lif->state);
 
+	ionic_rx_filters_deinit(lif);
+
 	napi_disable(&lif->adminqcq->napi);
 	ionic_lif_qcq_deinit(lif, lif->notifyqcq);
 	ionic_lif_qcq_deinit(lif, lif->adminqcq);
@@ -1010,6 +1012,10 @@ static int ionic_lif_init(struct ionic_lif *lif)
 	if (err)
 		goto err_out_notifyq_deinit;
 
+	err = ionic_rx_filters_init(lif);
+	if (err)
+		goto err_out_notifyq_deinit;
+
 	set_bit(IONIC_LIF_INITED, lif->state);
 
 	return 0;
drivers/net/ethernet/pensando/ionic/ionic_lif.h +2 -0
index 866411a7a1fe..b1cd51a5162c 100644
@@ -5,6 +5,7 @@
 #define _IONIC_LIF_H_
 
 #include <linux/pci.h>
+#include "ionic_rx_filter.h"
 
 #define IONIC_ADMINQ_LENGTH	16	/* must be a power of two */
 #define IONIC_NOTIFYQ_LENGTH	64	/* must be a power of two */
@@ -90,6 +91,7 @@ struct ionic_lif {
 	dma_addr_t info_pa;
 	u32 info_sz;
 
+	struct ionic_rx_filters rx_filters;
 	unsigned long *dbid_inuse;
 	unsigned int dbid_count;
 	struct dentry *dentry;
drivers/net/ethernet/pensando/ionic/ionic_rx_filter.c new file +150 -0

150 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_rx_filter.h new file +35 -0
new file mode 100644
index 000000000000..b6aec9c19918
@@ -0,0 +1,35 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+
+#ifndef _IONIC_RX_FILTER_H_
+#define _IONIC_RX_FILTER_H_
+
+#define IONIC_RXQ_INDEX_ANY		(0xFFFF)
+struct ionic_rx_filter {
+	u32 flow_id;
+	u32 filter_id;
+	u16 rxq_index;
+	struct ionic_rx_filter_add_cmd cmd;
+	struct hlist_node by_hash;
+	struct hlist_node by_id;
+};
+
+#define IONIC_RX_FILTER_HASH_BITS	10
+#define IONIC_RX_FILTER_HLISTS		BIT(IONIC_RX_FILTER_HASH_BITS)
+#define IONIC_RX_FILTER_HLISTS_MASK	(IONIC_RX_FILTER_HLISTS - 1)
+struct ionic_rx_filters {
+	spinlock_t lock;				    /* filter list lock */
+	struct hlist_head by_hash[IONIC_RX_FILTER_HLISTS];  /* by skb hash */
+	struct hlist_head by_id[IONIC_RX_FILTER_HLISTS];    /* by filter_id */
+};
+
+void ionic_rx_filter_free(struct ionic_lif *lif, struct ionic_rx_filter *f);
+int ionic_rx_filter_del(struct ionic_lif *lif, struct ionic_rx_filter *f);
+int ionic_rx_filters_init(struct ionic_lif *lif);
+void ionic_rx_filters_deinit(struct ionic_lif *lif);
+int ionic_rx_filter_save(struct ionic_lif *lif, u32 flow_id, u16 rxq_index,
+			 u32 hash, struct ionic_admin_ctx *ctx);
+struct ionic_rx_filter *ionic_rx_filter_by_vlan(struct ionic_lif *lif, u16 vid);
+struct ionic_rx_filter *ionic_rx_filter_by_addr(struct ionic_lif *lif, const u8 *addr);
+
+#endif /* _IONIC_RX_FILTER_H_ */
Signature · 1 line
2.17.1
#27 lore raw

PATCH v5 11/18 ionic: Add Rx filter and rx_mode ndo support

In reply to Shannon Nelson

Add the Rx filtering and rx_mode NDO callbacks. Also add the deferred work thread handling needed to manage the filter requests outside of the netif_addr_lock spinlock.

---
 .../net/ethernet/pensando/ionic/ionic_lif.c   | 391 +++++++++++++++++-
 .../net/ethernet/pensando/ionic/ionic_lif.h   |  29 ++
 2 files changed, 414 insertions(+), 6 deletions(-)
drivers/net/ethernet/pensando/ionic/ionic_lif.c +385 -6
index 373b93403190..0fcb26b8608e 100644
@@ -12,6 +12,52 @@
 #include "ionic_lif.h"
 #include "ionic_debugfs.h"
 
+static void ionic_lif_rx_mode(struct ionic_lif *lif, unsigned int rx_mode);
+static int ionic_lif_addr_add(struct ionic_lif *lif, const u8 *addr);
+static int ionic_lif_addr_del(struct ionic_lif *lif, const u8 *addr);
+
+static void ionic_lif_deferred_work(struct work_struct *work)
+{
+	struct ionic_lif *lif = container_of(work, struct ionic_lif, deferred.work);
+	struct ionic_deferred *def = &lif->deferred;
+	struct ionic_deferred_work *w = NULL;
+
+	spin_lock_bh(&def->lock);
+	if (!list_empty(&def->list)) {
+		w = list_first_entry(&def->list,
+				     struct ionic_deferred_work, list);
+		list_del(&w->list);
+	}
+	spin_unlock_bh(&def->lock);
+
+	if (w) {
+		switch (w->type) {
+		case IONIC_DW_TYPE_RX_MODE:
+			ionic_lif_rx_mode(lif, w->rx_mode);
+			break;
+		case IONIC_DW_TYPE_RX_ADDR_ADD:
+			ionic_lif_addr_add(lif, w->addr);
+			break;
+		case IONIC_DW_TYPE_RX_ADDR_DEL:
+			ionic_lif_addr_del(lif, w->addr);
+			break;
+		default:
+			break;
+		}
+		kfree(w);
+		schedule_work(&def->work);
+	}
+}
+
+static void ionic_lif_deferred_enqueue(struct ionic_deferred *def,
+				       struct ionic_deferred_work *work)
+{
+	spin_lock_bh(&def->lock);
+	list_add_tail(&work->list, &def->list);
+	spin_unlock_bh(&def->lock);
+	schedule_work(&def->work);
+}
+
 static irqreturn_t ionic_isr(int irq, void *data)
 {
 	struct napi_struct *napi = data;
@@ -400,6 +446,238 @@ static int ionic_adminq_napi(struct napi_struct *napi, int budget)
 	return max(n_work, a_work);
 }
 
+static int ionic_lif_addr_add(struct ionic_lif *lif, const u8 *addr)
+{
+	struct ionic_admin_ctx ctx = {
+		.work = COMPLETION_INITIALIZER_ONSTACK(ctx.work),
+		.cmd.rx_filter_add = {
+			.opcode = IONIC_CMD_RX_FILTER_ADD,
+			.lif_index = cpu_to_le16(lif->index),
+			.match = cpu_to_le16(IONIC_RX_FILTER_MATCH_MAC),
+		},
+	};
+	struct ionic_rx_filter *f;
+	int err;
+
+	/* don't bother if we already have it */
+	spin_lock_bh(&lif->rx_filters.lock);
+	f = ionic_rx_filter_by_addr(lif, addr);
+	spin_unlock_bh(&lif->rx_filters.lock);
+	if (f)
+		return 0;
+
+	netdev_dbg(lif->netdev, "rx_filter add ADDR %pM (id %d)\n", addr,
+		   ctx.comp.rx_filter_add.filter_id);
+
+	memcpy(ctx.cmd.rx_filter_add.mac.addr, addr, ETH_ALEN);
+	err = ionic_adminq_post_wait(lif, &ctx);
+	if (err)
+		return err;
+
+	return ionic_rx_filter_save(lif, 0, IONIC_RXQ_INDEX_ANY, 0, &ctx);
+}
+
+static int ionic_lif_addr_del(struct ionic_lif *lif, const u8 *addr)
+{
+	struct ionic_admin_ctx ctx = {
+		.work = COMPLETION_INITIALIZER_ONSTACK(ctx.work),
+		.cmd.rx_filter_del = {
+			.opcode = IONIC_CMD_RX_FILTER_DEL,
+			.lif_index = cpu_to_le16(lif->index),
+		},
+	};
+	struct ionic_rx_filter *f;
+	int err;
+
+	spin_lock_bh(&lif->rx_filters.lock);
+	f = ionic_rx_filter_by_addr(lif, addr);
+	if (!f) {
+		spin_unlock_bh(&lif->rx_filters.lock);
+		return -ENOENT;
+	}
+
+	ctx.cmd.rx_filter_del.filter_id = cpu_to_le32(f->filter_id);
+	ionic_rx_filter_free(lif, f);
+	spin_unlock_bh(&lif->rx_filters.lock);
+
+	err = ionic_adminq_post_wait(lif, &ctx);
+	if (err)
+		return err;
+
+	netdev_dbg(lif->netdev, "rx_filter del ADDR %pM (id %d)\n", addr,
+		   ctx.cmd.rx_filter_del.filter_id);
+
+	return 0;
+}
+
+static int ionic_lif_addr(struct ionic_lif *lif, const u8 *addr, bool add)
+{
+	struct ionic *ionic = lif->ionic;
+	struct ionic_deferred_work *work;
+	unsigned int nmfilters;
+	unsigned int nufilters;
+
+	if (add) {
+		/* Do we have space for this filter?  We test the counters
+		 * here before checking the need for deferral so that we
+		 * can return an overflow error to the stack.
+		 */
+		nmfilters = le32_to_cpu(ionic->ident.lif.eth.max_mcast_filters);
+		nufilters = le32_to_cpu(ionic->ident.lif.eth.max_ucast_filters);
+
+		if ((is_multicast_ether_addr(addr) && lif->nmcast < nmfilters))
+			lif->nmcast++;
+		else if (!is_multicast_ether_addr(addr) &&
+			 lif->nucast < nufilters)
+			lif->nucast++;
+		else
+			return -ENOSPC;
+	} else {
+		if (is_multicast_ether_addr(addr) && lif->nmcast)
+			lif->nmcast--;
+		else if (!is_multicast_ether_addr(addr) && lif->nucast)
+			lif->nucast--;
+	}
+
+	if (in_interrupt()) {
+		work = kzalloc(sizeof(*work), GFP_ATOMIC);
+		if (!work) {
+			netdev_err(lif->netdev, "%s OOM\n", __func__);
+			return -ENOMEM;
+		}
+		work->type = add ? IONIC_DW_TYPE_RX_ADDR_ADD :
+				   IONIC_DW_TYPE_RX_ADDR_DEL;
+		memcpy(work->addr, addr, ETH_ALEN);
+		netdev_dbg(lif->netdev, "deferred: rx_filter %s %pM\n",
+			   add ? "add" : "del", addr);
+		ionic_lif_deferred_enqueue(&lif->deferred, work);
+	} else {
+		netdev_dbg(lif->netdev, "rx_filter %s %pM\n",
+			   add ? "add" : "del", addr);
+		if (add)
+			return ionic_lif_addr_add(lif, addr);
+		else
+			return ionic_lif_addr_del(lif, addr);
+	}
+
+	return 0;
+}
+
+static int ionic_addr_add(struct net_device *netdev, const u8 *addr)
+{
+	return ionic_lif_addr(netdev_priv(netdev), addr, true);
+}
+
+static int ionic_addr_del(struct net_device *netdev, const u8 *addr)
+{
+	return ionic_lif_addr(netdev_priv(netdev), addr, false);
+}
+
+static void ionic_lif_rx_mode(struct ionic_lif *lif, unsigned int rx_mode)
+{
+	struct ionic_admin_ctx ctx = {
+		.work = COMPLETION_INITIALIZER_ONSTACK(ctx.work),
+		.cmd.rx_mode_set = {
+			.opcode = IONIC_CMD_RX_MODE_SET,
+			.lif_index = cpu_to_le16(lif->index),
+			.rx_mode = cpu_to_le16(rx_mode),
+		},
+	};
+	char buf[128];
+	int err;
+	int i;
+#define REMAIN(__x) (sizeof(buf) - (__x))
+
+	i = snprintf(buf, sizeof(buf), "rx_mode 0x%04x -> 0x%04x:",
+		     lif->rx_mode, rx_mode);
+	if (rx_mode & IONIC_RX_MODE_F_UNICAST)
+		i += snprintf(&buf[i], REMAIN(i), " RX_MODE_F_UNICAST");
+	if (rx_mode & IONIC_RX_MODE_F_MULTICAST)
+		i += snprintf(&buf[i], REMAIN(i), " RX_MODE_F_MULTICAST");
+	if (rx_mode & IONIC_RX_MODE_F_BROADCAST)
+		i += snprintf(&buf[i], REMAIN(i), " RX_MODE_F_BROADCAST");
+	if (rx_mode & IONIC_RX_MODE_F_PROMISC)
+		i += snprintf(&buf[i], REMAIN(i), " RX_MODE_F_PROMISC");
+	if (rx_mode & IONIC_RX_MODE_F_ALLMULTI)
+		i += snprintf(&buf[i], REMAIN(i), " RX_MODE_F_ALLMULTI");
+	netdev_dbg(lif->netdev, "lif%d %s\n", lif->index, buf);
+
+	err = ionic_adminq_post_wait(lif, &ctx);
+	if (err)
+		netdev_warn(lif->netdev, "set rx_mode 0x%04x failed: %d\n",
+			    rx_mode, err);
+	else
+		lif->rx_mode = rx_mode;
+}
+
+static void _ionic_lif_rx_mode(struct ionic_lif *lif, unsigned int rx_mode)
+{
+	struct ionic_deferred_work *work;
+
+	if (in_interrupt()) {
+		work = kzalloc(sizeof(*work), GFP_ATOMIC);
+		if (!work) {
+			netdev_err(lif->netdev, "%s OOM\n", __func__);
+			return;
+		}
+		work->type = IONIC_DW_TYPE_RX_MODE;
+		work->rx_mode = rx_mode;
+		netdev_dbg(lif->netdev, "deferred: rx_mode\n");
+		ionic_lif_deferred_enqueue(&lif->deferred, work);
+	} else {
+		ionic_lif_rx_mode(lif, rx_mode);
+	}
+}
+
+static void ionic_set_rx_mode(struct net_device *netdev)
+{
+	struct ionic_lif *lif = netdev_priv(netdev);
+	struct ionic_identity *ident;
+	unsigned int nfilters;
+	unsigned int rx_mode;
+
+	ident = &lif->ionic->ident;
+
+	rx_mode = IONIC_RX_MODE_F_UNICAST;
+	rx_mode |= (netdev->flags & IFF_MULTICAST) ? IONIC_RX_MODE_F_MULTICAST : 0;
+	rx_mode |= (netdev->flags & IFF_BROADCAST) ? IONIC_RX_MODE_F_BROADCAST : 0;
+	rx_mode |= (netdev->flags & IFF_PROMISC) ? IONIC_RX_MODE_F_PROMISC : 0;
+	rx_mode |= (netdev->flags & IFF_ALLMULTI) ? IONIC_RX_MODE_F_ALLMULTI : 0;
+
+	/* sync unicast addresses
+	 * next check to see if we're in an overflow state
+	 *    if so, we track that we overflowed and enable NIC PROMISC
+	 *    else if the overflow is set and not needed
+	 *       we remove our overflow flag and check the netdev flags
+	 *       to see if we can disable NIC PROMISC
+	 */
+	__dev_uc_sync(netdev, ionic_addr_add, ionic_addr_del);
+	nfilters = le32_to_cpu(ident->lif.eth.max_ucast_filters);
+	if (netdev_uc_count(netdev) + 1 > nfilters) {
+		rx_mode |= IONIC_RX_MODE_F_PROMISC;
+		lif->uc_overflow = true;
+	} else if (lif->uc_overflow) {
+		lif->uc_overflow = false;
+		if (!(netdev->flags & IFF_PROMISC))
+			rx_mode &= ~IONIC_RX_MODE_F_PROMISC;
+	}
+
+	/* same for multicast */
+	__dev_mc_sync(netdev, ionic_addr_add, ionic_addr_del);
+	nfilters = le32_to_cpu(ident->lif.eth.max_mcast_filters);
+	if (netdev_mc_count(netdev) > nfilters) {
+		rx_mode |= IONIC_RX_MODE_F_ALLMULTI;
+		lif->mc_overflow = true;
+	} else if (lif->mc_overflow) {
+		lif->mc_overflow = false;
+		if (!(netdev->flags & IFF_ALLMULTI))
+			rx_mode &= ~IONIC_RX_MODE_F_ALLMULTI;
+	}
+
+	if (lif->rx_mode != rx_mode)
+		_ionic_lif_rx_mode(lif, rx_mode);
+}
+
 static __le64 ionic_netdev_features_to_nic(netdev_features_t features)
 {
 	u64 wanted = 0;
@@ -588,8 +866,26 @@ static int ionic_set_features(struct net_device *netdev,
 
 static int ionic_set_mac_address(struct net_device *netdev, void *sa)
 {
-	netdev_info(netdev, "%s: stubbed\n", __func__);
-	return 0;
+	struct sockaddr *addr = sa;
+	u8 *mac;
+
+	mac = (u8 *)addr->sa_data;
+	if (ether_addr_equal(netdev->dev_addr, mac))
+		return 0;
+
+	if (!is_valid_ether_addr(mac))
+		return -EADDRNOTAVAIL;
+
+	if (!is_zero_ether_addr(netdev->dev_addr)) {
+		netdev_info(netdev, "deleting mac addr %pM\n",
+			    netdev->dev_addr);
+		ionic_addr_del(netdev, netdev->dev_addr);
+	}
+
+	memcpy(netdev->dev_addr, mac, netdev->addr_len);
+	netdev_info(netdev, "updating mac addr %pM\n", mac);
+
+	return ionic_addr_add(netdev, mac);
 }
 
 static int ionic_change_mtu(struct net_device *netdev, int new_mtu)
@@ -624,15 +920,57 @@ static void ionic_tx_timeout(struct net_device *netdev)
 static int ionic_vlan_rx_add_vid(struct net_device *netdev, __be16 proto,
 				 u16 vid)
 {
-	netdev_info(netdev, "%s: stubbed\n", __func__);
-	return 0;
+	struct ionic_lif *lif = netdev_priv(netdev);
+	struct ionic_admin_ctx ctx = {
+		.work = COMPLETION_INITIALIZER_ONSTACK(ctx.work),
+		.cmd.rx_filter_add = {
+			.opcode = IONIC_CMD_RX_FILTER_ADD,
+			.lif_index = cpu_to_le16(lif->index),
+			.match = cpu_to_le16(IONIC_RX_FILTER_MATCH_VLAN),
+			.vlan.vlan = cpu_to_le16(vid),
+		},
+	};
+	int err;
+
+	err = ionic_adminq_post_wait(lif, &ctx);
+	if (err)
+		return err;
+
+	netdev_dbg(netdev, "rx_filter add VLAN %d (id %d)\n", vid,
+		   ctx.comp.rx_filter_add.filter_id);
+
+	return ionic_rx_filter_save(lif, 0, IONIC_RXQ_INDEX_ANY, 0, &ctx);
 }
 
 static int ionic_vlan_rx_kill_vid(struct net_device *netdev, __be16 proto,
 				  u16 vid)
 {
-	netdev_info(netdev, "%s: stubbed\n", __func__);
-	return 0;
+	struct ionic_lif *lif = netdev_priv(netdev);
+	struct ionic_admin_ctx ctx = {
+		.work = COMPLETION_INITIALIZER_ONSTACK(ctx.work),
+		.cmd.rx_filter_del = {
+			.opcode = IONIC_CMD_RX_FILTER_DEL,
+			.lif_index = cpu_to_le16(lif->index),
+		},
+	};
+	struct ionic_rx_filter *f;
+
+	spin_lock_bh(&lif->rx_filters.lock);
+
+	f = ionic_rx_filter_by_vlan(lif, vid);
+	if (!f) {
+		spin_unlock_bh(&lif->rx_filters.lock);
+		return -ENOENT;
+	}
+
+	netdev_dbg(netdev, "rx_filter del VLAN %d (id %d)\n", vid,
+		   le32_to_cpu(ctx.cmd.rx_filter_del.filter_id));
+
+	ctx.cmd.rx_filter_del.filter_id = cpu_to_le32(f->filter_id);
+	ionic_rx_filter_free(lif, f);
+	spin_unlock_bh(&lif->rx_filters.lock);
+
+	return ionic_adminq_post_wait(lif, &ctx);
 }
 
 int ionic_open(struct net_device *netdev)
@@ -668,6 +1006,7 @@ int ionic_stop(struct net_device *netdev)
 static const struct net_device_ops ionic_netdev_ops = {
 	.ndo_open               = ionic_open,
 	.ndo_stop               = ionic_stop,
+	.ndo_set_rx_mode	= ionic_set_rx_mode,
 	.ndo_set_features	= ionic_set_features,
 	.ndo_set_mac_address	= ionic_set_mac_address,
 	.ndo_validate_addr	= eth_validate_addr,
@@ -734,6 +1073,10 @@ static struct ionic_lif *ionic_lif_alloc(struct ionic *ionic, unsigned int index
 
 	spin_lock_init(&lif->adminq_lock);
 
+	spin_lock_init(&lif->deferred.lock);
+	INIT_LIST_HEAD(&lif->deferred.list);
+	INIT_WORK(&lif->deferred.work, ionic_lif_deferred_work);
+
 	/* allocate lif info */
 	lif->info_sz = ALIGN(sizeof(*lif->info), PAGE_SIZE);
 	lif->info = dma_alloc_coherent(dev, lif->info_sz,
@@ -953,6 +1296,37 @@ static int ionic_lif_notifyq_init(struct ionic_lif *lif)
 	return 0;
 }
 
+static int ionic_station_set(struct ionic_lif *lif)
+{
+	struct net_device *netdev = lif->netdev;
+	struct ionic_admin_ctx ctx = {
+		.work = COMPLETION_INITIALIZER_ONSTACK(ctx.work),
+		.cmd.lif_getattr = {
+			.opcode = IONIC_CMD_LIF_GETATTR,
+			.index = cpu_to_le16(lif->index),
+			.attr = IONIC_LIF_ATTR_MAC,
+		},
+	};
+	int err;
+
+	err = ionic_adminq_post_wait(lif, &ctx);
+	if (err)
+		return err;
+
+	if (!is_zero_ether_addr(netdev->dev_addr)) {
+		netdev_dbg(lif->netdev, "deleting station MAC addr %pM\n",
+			   netdev->dev_addr);
+		ionic_lif_addr(lif, netdev->dev_addr, false);
+	}
+	memcpy(netdev->dev_addr, ctx.comp.lif_getattr.mac,
+	       netdev->addr_len);
+	netdev_dbg(lif->netdev, "adding station MAC addr %pM\n",
+		   netdev->dev_addr);
+	ionic_lif_addr(lif, netdev->dev_addr, true);
+
+	return 0;
+}
+
 static int ionic_lif_init(struct ionic_lif *lif)
 {
 	struct ionic_dev *idev = &lif->ionic->idev;
@@ -1016,6 +1390,10 @@ static int ionic_lif_init(struct ionic_lif *lif)
 	if (err)
 		goto err_out_notifyq_deinit;
 
+	err = ionic_station_set(lif);
+	if (err)
+		goto err_out_notifyq_deinit;
+
 	set_bit(IONIC_LIF_INITED, lif->state);
 
 	return 0;
@@ -1072,6 +1450,7 @@ void ionic_lifs_unregister(struct ionic *ionic)
 	 * current model, so don't bother searching the
 	 * ionic->lif for candidates to unregister
 	 */
+	cancel_work_sync(&ionic->master_lif->deferred.work);
 	if (ionic->master_lif->netdev->reg_state == NETREG_REGISTERED)
 		unregister_netdev(ionic->master_lif->netdev);
 }
drivers/net/ethernet/pensando/ionic/ionic_lif.h +29 -0
index b1cd51a5162c..a7dcb7865b89 100644
@@ -58,6 +58,29 @@ struct ionic_qcq {
 #define napi_to_qcq(napi)	container_of(napi, struct ionic_qcq, napi)
 #define napi_to_cq(napi)	(&napi_to_qcq(napi)->cq)
 
+enum ionic_deferred_work_type {
+	IONIC_DW_TYPE_RX_MODE,
+	IONIC_DW_TYPE_RX_ADDR_ADD,
+	IONIC_DW_TYPE_RX_ADDR_DEL,
+	IONIC_DW_TYPE_LINK_STATUS,
+	IONIC_DW_TYPE_LIF_RESET,
+};
+
+struct ionic_deferred_work {
+	struct list_head list;
+	enum ionic_deferred_work_type type;
+	union {
+		unsigned int rx_mode;
+		u8 addr[ETH_ALEN];
+	};
+};
+
+struct ionic_deferred {
+	spinlock_t lock;		/* lock for deferred work list */
+	struct list_head list;
+	struct work_struct work;
+};
+
 enum ionic_lif_state_flags {
 	IONIC_LIF_INITED,
 	IONIC_LIF_UP,
@@ -85,13 +108,19 @@ struct ionic_lif {
 	u64 last_eid;
 	unsigned int neqs;
 	unsigned int nxqs;
+	unsigned int rx_mode;
 	u64 hw_features;
+	bool mc_overflow;
+	unsigned int nmcast;
+	bool uc_overflow;
+	unsigned int nucast;
 
 	struct ionic_lif_info *info;
 	dma_addr_t info_pa;
 	u32 info_sz;
 
 	struct ionic_rx_filters rx_filters;
+	struct ionic_deferred deferred;
 	unsigned long *dbid_inuse;
 	unsigned int dbid_count;
 	struct dentry *dentry;
Signature · 1 line
2.17.1
#28 lore raw

PATCH v5 12/18 ionic: Add async link status check and basic stats

In reply to Shannon Nelson

Add code to handle the link status event, and wire up the basic netdev hardware stats.

---
 .../net/ethernet/pensando/ionic/ionic_lif.c   | 116 +++++++++++++++++-
 .../net/ethernet/pensando/ionic/ionic_lif.h   |   1 +
 2 files changed, 111 insertions(+), 6 deletions(-)
drivers/net/ethernet/pensando/ionic/ionic_lif.c +110 -6
index 0fcb26b8608e..5105ce281983 100644
@@ -15,6 +15,7 @@
 static void ionic_lif_rx_mode(struct ionic_lif *lif, unsigned int rx_mode);
 static int ionic_lif_addr_add(struct ionic_lif *lif, const u8 *addr);
 static int ionic_lif_addr_del(struct ionic_lif *lif, const u8 *addr);
+static void ionic_link_status_check(struct ionic_lif *lif);
 
 static void ionic_lif_deferred_work(struct work_struct *work)
 {
@@ -41,6 +42,9 @@ static void ionic_lif_deferred_work(struct work_struct *work)
 		case IONIC_DW_TYPE_RX_ADDR_DEL:
 			ionic_lif_addr_del(lif, w->addr);
 			break;
+		case IONIC_DW_TYPE_LINK_STATUS:
+			ionic_link_status_check(lif);
+			break;
 		default:
 			break;
 		}
@@ -58,6 +62,54 @@ static void ionic_lif_deferred_enqueue(struct ionic_deferred *def,
 	schedule_work(&def->work);
 }
 
+static void ionic_link_status_check(struct ionic_lif *lif)
+{
+	struct net_device *netdev = lif->netdev;
+	u16 link_status;
+	bool link_up;
+
+	link_status = le16_to_cpu(lif->info->status.link_status);
+	link_up = link_status == IONIC_PORT_OPER_STATUS_UP;
+
+	/* filter out the no-change cases */
+	if (link_up == netif_carrier_ok(netdev))
+		goto link_out;
+
+	if (link_up) {
+		netdev_info(netdev, "Link up - %d Gbps\n",
+			    le32_to_cpu(lif->info->status.link_speed) / 1000);
+
+	} else {
+		netdev_info(netdev, "Link down\n");
+
+		/* carrier off first to avoid watchdog timeout */
+		netif_carrier_off(netdev);
+	}
+
+link_out:
+	clear_bit(IONIC_LIF_LINK_CHECK_REQUESTED, lif->state);
+}
+
+static void ionic_link_status_check_request(struct ionic_lif *lif)
+{
+	struct ionic_deferred_work *work;
+
+	/* we only need one request outstanding at a time */
+	if (test_and_set_bit(IONIC_LIF_LINK_CHECK_REQUESTED, lif->state))
+		return;
+
+	if (in_interrupt()) {
+		work = kzalloc(sizeof(*work), GFP_ATOMIC);
+		if (!work)
+			return;
+
+		work->type = IONIC_DW_TYPE_LINK_STATUS;
+		ionic_lif_deferred_enqueue(&lif->deferred, work);
+	} else {
+		ionic_link_status_check(lif);
+	}
+}
+
 static irqreturn_t ionic_isr(int irq, void *data)
 {
 	struct napi_struct *napi = data;
@@ -382,12 +434,7 @@ static bool ionic_notifyq_service(struct ionic_cq *cq,
 
 	switch (le16_to_cpu(comp->event.ecode)) {
 	case IONIC_EVENT_LINK_CHANGE:
-		netdev_info(netdev, "Notifyq IONIC_EVENT_LINK_CHANGE eid=%lld\n",
-			    eid);
-		netdev_info(netdev,
-			    "  link_status=%d link_speed=%d\n",
-			    le16_to_cpu(comp->link_change.link_status),
-			    le32_to_cpu(comp->link_change.link_speed));
+		ionic_link_status_check_request(lif);
 		break;
 	case IONIC_EVENT_RESET:
 		netdev_info(netdev, "Notifyq IONIC_EVENT_RESET eid=%lld\n",
@@ -446,6 +493,59 @@ static int ionic_adminq_napi(struct napi_struct *napi, int budget)
 	return max(n_work, a_work);
 }
 
+static void ionic_get_stats64(struct net_device *netdev,
+			      struct rtnl_link_stats64 *ns)
+{
+	struct ionic_lif *lif = netdev_priv(netdev);
+	struct ionic_lif_stats *ls;
+
+	memset(ns, 0, sizeof(*ns));
+	ls = &lif->info->stats;
+
+	ns->rx_packets = le64_to_cpu(ls->rx_ucast_packets) +
+			 le64_to_cpu(ls->rx_mcast_packets) +
+			 le64_to_cpu(ls->rx_bcast_packets);
+
+	ns->tx_packets = le64_to_cpu(ls->tx_ucast_packets) +
+			 le64_to_cpu(ls->tx_mcast_packets) +
+			 le64_to_cpu(ls->tx_bcast_packets);
+
+	ns->rx_bytes = le64_to_cpu(ls->rx_ucast_bytes) +
+		       le64_to_cpu(ls->rx_mcast_bytes) +
+		       le64_to_cpu(ls->rx_bcast_bytes);
+
+	ns->tx_bytes = le64_to_cpu(ls->tx_ucast_bytes) +
+		       le64_to_cpu(ls->tx_mcast_bytes) +
+		       le64_to_cpu(ls->tx_bcast_bytes);
+
+	ns->rx_dropped = le64_to_cpu(ls->rx_ucast_drop_packets) +
+			 le64_to_cpu(ls->rx_mcast_drop_packets) +
+			 le64_to_cpu(ls->rx_bcast_drop_packets);
+
+	ns->tx_dropped = le64_to_cpu(ls->tx_ucast_drop_packets) +
+			 le64_to_cpu(ls->tx_mcast_drop_packets) +
+			 le64_to_cpu(ls->tx_bcast_drop_packets);
+
+	ns->multicast = le64_to_cpu(ls->rx_mcast_packets);
+
+	ns->rx_over_errors = le64_to_cpu(ls->rx_queue_empty);
+
+	ns->rx_missed_errors = le64_to_cpu(ls->rx_dma_error) +
+			       le64_to_cpu(ls->rx_queue_disabled) +
+			       le64_to_cpu(ls->rx_desc_fetch_error) +
+			       le64_to_cpu(ls->rx_desc_data_error);
+
+	ns->tx_aborted_errors = le64_to_cpu(ls->tx_dma_error) +
+				le64_to_cpu(ls->tx_queue_disabled) +
+				le64_to_cpu(ls->tx_desc_fetch_error) +
+				le64_to_cpu(ls->tx_desc_data_error);
+
+	ns->rx_errors = ns->rx_over_errors +
+			ns->rx_missed_errors;
+
+	ns->tx_errors = ns->tx_aborted_errors;
+}
+
 static int ionic_lif_addr_add(struct ionic_lif *lif, const u8 *addr)
 {
 	struct ionic_admin_ctx ctx = {
@@ -981,6 +1081,8 @@ int ionic_open(struct net_device *netdev)
 
 	set_bit(IONIC_LIF_UP, lif->state);
 
+	ionic_link_status_check_request(lif);
+
 	return 0;
 }
 
@@ -1006,6 +1108,7 @@ int ionic_stop(struct net_device *netdev)
 static const struct net_device_ops ionic_netdev_ops = {
 	.ndo_open               = ionic_open,
 	.ndo_stop               = ionic_stop,
+	.ndo_get_stats64	= ionic_get_stats64,
 	.ndo_set_rx_mode	= ionic_set_rx_mode,
 	.ndo_set_features	= ionic_set_features,
 	.ndo_set_mac_address	= ionic_set_mac_address,
@@ -1439,6 +1542,7 @@ int ionic_lifs_register(struct ionic *ionic)
 		return err;
 	}
 
+	ionic_link_status_check_request(ionic->master_lif);
 	ionic->master_lif->registered = true;
 
 	return 0;
drivers/net/ethernet/pensando/ionic/ionic_lif.h +1 -0
index a7dcb7865b89..db6dda42e8bd 100644
@@ -84,6 +84,7 @@ struct ionic_deferred {
 enum ionic_lif_state_flags {
 	IONIC_LIF_INITED,
 	IONIC_LIF_UP,
+	IONIC_LIF_LINK_CHECK_REQUESTED,
 	IONIC_LIF_QUEUE_RESET,
 
 	/* leave this as last */
Signature · 1 line
2.17.1
#29 lore raw

PATCH v5 13/18 ionic: Add initial ethtool support

In reply to Shannon Nelson

Add in the basic ethtool callbacks for device information and control.

---
 drivers/net/ethernet/pensando/ionic/Makefile  |   2 +-
 .../net/ethernet/pensando/ionic/ionic_dev.h   |   7 +
 .../ethernet/pensando/ionic/ionic_ethtool.c   | 498 ++++++++++++++++++
 .../ethernet/pensando/ionic/ionic_ethtool.h   |   9 +
 .../net/ethernet/pensando/ionic/ionic_lif.c   |   2 +
 .../net/ethernet/pensando/ionic/ionic_lif.h   |   3 +
 6 files changed, 520 insertions(+), 1 deletion(-)
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_ethtool.c
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_ethtool.h
drivers/net/ethernet/pensando/ionic/Makefile +1 -1
index 8c31a90830cf..a8b85f11cf3c 100644
@@ -4,4 +4,4 @@
 obj-$(CONFIG_IONIC) := ionic.o
 
 ionic-y := ionic_main.o ionic_bus_pci.o ionic_devlink.o ionic_dev.o \
-	   ionic_debugfs.o ionic_lif.o ionic_rx_filter.o
+	   ionic_debugfs.o ionic_lif.o ionic_rx_filter.o ionic_ethtool.o
drivers/net/ethernet/pensando/ionic/ionic_dev.h +7 -0
index f6cce42f0228..1db1e433127a 100644
@@ -12,8 +12,15 @@
 
 #define IONIC_MIN_MTU			ETH_MIN_MTU
 #define IONIC_MAX_MTU			9194
+#define IONIC_MAX_TXRX_DESC		16384
+#define IONIC_MIN_TXRX_DESC		16
+#define IONIC_DEF_TXRX_DESC		4096
 #define IONIC_LIFS_MAX			1024
 
+#define IONIC_DEV_CMD_REG_VERSION	1
+#define IONIC_DEV_INFO_REG_COUNT	32
+#define IONIC_DEV_CMD_REG_COUNT		32
+
 struct ionic_dev_bar {
 	void __iomem *vaddr;
 	phys_addr_t bus_addr;
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +498 -0

498 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_ethtool.h new file +9 -0
new file mode 100644
index 000000000000..38b91b1d70ae
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+
+#ifndef _IONIC_ETHTOOL_H_
+#define _IONIC_ETHTOOL_H_
+
+void ionic_ethtool_set_ops(struct net_device *netdev);
+
+#endif /* _IONIC_ETHTOOL_H_ */
drivers/net/ethernet/pensando/ionic/ionic_lif.c +2 -0
index 5105ce281983..cb77ffd18708 100644
@@ -10,6 +10,7 @@
 #include "ionic.h"
 #include "ionic_bus.h"
 #include "ionic_lif.h"
+#include "ionic_ethtool.h"
 #include "ionic_debugfs.h"
 
 static void ionic_lif_rx_mode(struct ionic_lif *lif, unsigned int rx_mode);
@@ -1161,6 +1162,7 @@ static struct ionic_lif *ionic_lif_alloc(struct ionic *ionic, unsigned int index
 	lif->netdev = netdev;
 	ionic->master_lif = lif;
 	netdev->netdev_ops = &ionic_netdev_ops;
+	ionic_ethtool_set_ops(netdev);
 
 	netdev->watchdog_timeo = 2 * HZ;
 	netdev->min_mtu = IONIC_MIN_MTU;
drivers/net/ethernet/pensando/ionic/ionic_lif.h +3 -0
index db6dda42e8bd..f927e1cfd646 100644
@@ -109,6 +109,8 @@ struct ionic_lif {
 	u64 last_eid;
 	unsigned int neqs;
 	unsigned int nxqs;
+	unsigned int ntxq_descs;
+	unsigned int nrxq_descs;
 	unsigned int rx_mode;
 	u64 hw_features;
 	bool mc_overflow;
@@ -125,6 +127,7 @@ struct ionic_lif {
 	unsigned long *dbid_inuse;
 	unsigned int dbid_count;
 	struct dentry *dentry;
+	u32 rx_coalesce_usecs;
 	u32 flags;
 };
 
Signature · 1 line
2.17.1
#30 lore raw

PATCH v5 14/18 ionic: Add Tx and Rx handling

In reply to Shannon Nelson

Add both the Tx and Rx queue setup and handling. The related stats display comes later. Instead of using the generic napi routines used by the slow-path commands, the Tx and Rx paths are simplified and inlined in one file in order to get better compiler optimizations.

---
 drivers/net/ethernet/pensando/ionic/Makefile  |   3 +-
 .../net/ethernet/pensando/ionic/ionic_lif.c   | 414 ++++++++
 .../net/ethernet/pensando/ionic/ionic_lif.h   |  50 +
 .../net/ethernet/pensando/ionic/ionic_txrx.c  | 896 ++++++++++++++++++
 .../net/ethernet/pensando/ionic/ionic_txrx.h  |  15 +
 5 files changed, 1377 insertions(+), 1 deletion(-)
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_txrx.c
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_txrx.h
drivers/net/ethernet/pensando/ionic/Makefile +2 -1
index a8b85f11cf3c..1cf8117db443 100644
@@ -4,4 +4,5 @@
 obj-$(CONFIG_IONIC) := ionic.o
 
 ionic-y := ionic_main.o ionic_bus_pci.o ionic_devlink.o ionic_dev.o \
-	   ionic_debugfs.o ionic_lif.o ionic_rx_filter.o ionic_ethtool.o
+	   ionic_debugfs.o ionic_lif.o ionic_rx_filter.o ionic_ethtool.o \
+	   ionic_txrx.o
drivers/net/ethernet/pensando/ionic/ionic_lif.c +414 -0

512 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_lif.h +50 -0

107 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_txrx.c new file +896 -0

896 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_txrx.h new file +15 -0
new file mode 100644
index 000000000000..53775c62c85a
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+
+#ifndef _IONIC_TXRX_H_
+#define _IONIC_TXRX_H_
+
+void ionic_rx_flush(struct ionic_cq *cq);
+void ionic_tx_flush(struct ionic_cq *cq);
+
+void ionic_rx_fill(struct ionic_queue *q);
+void ionic_rx_empty(struct ionic_queue *q);
+int ionic_rx_napi(struct napi_struct *napi, int budget);
+netdev_tx_t ionic_start_xmit(struct sk_buff *skb, struct net_device *netdev);
+
+#endif /* _IONIC_TXRX_H_ */
Signature · 1 line
2.17.1
#31 lore raw

Re: [PATCH v5 net-next 14/18] ionic: Add Tx and Rx handling

Quoted 438 lines · Shannon Nelson

Add both the Tx and Rx queue setup and handling. The related stats display comes later. Instead of using the generic napi routines used by the slow-path commands, the Tx and Rx paths are simplified and inlined in one file in order to get better compiler optimizations.

---
 drivers/net/ethernet/pensando/ionic/Makefile  |   3 +-
 .../net/ethernet/pensando/ionic/ionic_lif.c   | 414 ++++++++
 .../net/ethernet/pensando/ionic/ionic_lif.h   |  50 +
 .../net/ethernet/pensando/ionic/ionic_txrx.c  | 896 ++++++++++++++++++
 .../net/ethernet/pensando/ionic/ionic_txrx.h  |  15 +
 5 files changed, 1377 insertions(+), 1 deletion(-)
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_txrx.c
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_txrx.h
drivers/net/ethernet/pensando/ionic/Makefile +2 -1
index a8b85f11cf3c..1cf8117db443 100644
@@ -4,4 +4,5 @@
 obj-$(CONFIG_IONIC) := ionic.o
 
 ionic-y := ionic_main.o ionic_bus_pci.o ionic_devlink.o ionic_dev.o \
-	   ionic_debugfs.o ionic_lif.o ionic_rx_filter.o ionic_ethtool.o
+	   ionic_debugfs.o ionic_lif.o ionic_rx_filter.o ionic_ethtool.o \
+	   ionic_txrx.o
drivers/net/ethernet/pensando/ionic/ionic_lif.c +337 -0
index cb77ffd18708..61572823e993 100644
@@ -10,6 +10,7 @@
 #include "ionic.h"
 #include "ionic_bus.h"
 #include "ionic_lif.h"
+#include "ionic_txrx.h"
 #include "ionic_ethtool.h"
 #include "ionic_debugfs.h"
 
@@ -80,11 +81,17 @@ static void ionic_link_status_check(struct ionic_lif *lif)
 		netdev_info(netdev, "Link up - %d Gbps\n",
 			    le32_to_cpu(lif->info->status.link_speed) / 1000);
 
+		if (test_bit(IONIC_LIF_UP, lif->state)) {
+			netif_tx_wake_all_queues(lif->netdev);
+			netif_carrier_on(netdev);
+		}
 	} else {
 		netdev_info(netdev, "Link down\n");
 
 		/* carrier off first to avoid watchdog timeout */
 		netif_carrier_off(netdev);
+		if (test_bit(IONIC_LIF_UP, lif->state))
+			netif_tx_stop_all_queues(netdev);
 	}
 
 link_out:
@@ -163,6 +170,77 @@ static void ionic_intr_free(struct ionic_lif *lif, int index)
 		clear_bit(index, lif->ionic->intrs);
 }
 
+static int ionic_qcq_enable(struct ionic_qcq *qcq)
+{
+	struct ionic_queue *q = &qcq->q;
+	struct ionic_lif *lif = q->lif;
+	struct ionic_dev *idev;
+	struct device *dev;
+
+	struct ionic_admin_ctx ctx = {
+		.work = COMPLETION_INITIALIZER_ONSTACK(ctx.work),
+		.cmd.q_control = {
+			.opcode = IONIC_CMD_Q_CONTROL,
+			.lif_index = cpu_to_le16(lif->index),
+			.type = q->type,
+			.index = cpu_to_le32(q->index),
+			.oper = IONIC_Q_ENABLE,
+		},
+	};
+
+	idev = &lif->ionic->idev;
+	dev = lif->ionic->dev;
+
+	dev_dbg(dev, "q_enable.index %d q_enable.qtype %d\n",
+		ctx.cmd.q_control.index, ctx.cmd.q_control.type);
+
+	if (qcq->flags & IONIC_QCQ_F_INTR) {
+		irq_set_affinity_hint(qcq->intr.vector,
+				      &qcq->intr.affinity_mask);
+		napi_enable(&qcq->napi);
+		ionic_intr_clean(idev->intr_ctrl, qcq->intr.index);
+		ionic_intr_mask(idev->intr_ctrl, qcq->intr.index,
+				IONIC_INTR_MASK_CLEAR);
+	}
+
+	return ionic_adminq_post_wait(lif, &ctx);
+}
+
+static int ionic_qcq_disable(struct ionic_qcq *qcq)
+{
+	struct ionic_queue *q = &qcq->q;
+	struct ionic_lif *lif = q->lif;
+	struct ionic_dev *idev;
+	struct device *dev;
+
+	struct ionic_admin_ctx ctx = {
+		.work = COMPLETION_INITIALIZER_ONSTACK(ctx.work),
+		.cmd.q_control = {
+			.opcode = IONIC_CMD_Q_CONTROL,
+			.lif_index = cpu_to_le16(lif->index),
+			.type = q->type,
+			.index = cpu_to_le32(q->index),
+			.oper = IONIC_Q_DISABLE,
+		},
+	};
+
+	idev = &lif->ionic->idev;
+	dev = lif->ionic->dev;
+
+	dev_dbg(dev, "q_disable.index %d q_disable.qtype %d\n",
+		ctx.cmd.q_control.index, ctx.cmd.q_control.type);
+
+	if (qcq->flags & IONIC_QCQ_F_INTR) {
+		ionic_intr_mask(idev->intr_ctrl, qcq->intr.index,
+				IONIC_INTR_MASK_SET);
+		synchronize_irq(qcq->intr.vector);
+		irq_set_affinity_hint(qcq->intr.vector, NULL);
+		napi_disable(&qcq->napi);
+	}
+
+	return ionic_adminq_post_wait(lif, &ctx);
+}
+
 static void ionic_lif_qcq_deinit(struct ionic_lif *lif, struct ionic_qcq *qcq)
 {
 	struct ionic_dev *idev = &lif->ionic->idev;
@@ -210,6 +288,9 @@ static void ionic_qcq_free(struct ionic_lif *lif, struct ionic_qcq *qcq)
 
 static void ionic_qcqs_free(struct ionic_lif *lif)
 {
+	struct device *dev = lif->ionic->dev;
+	unsigned int i;
+
 	if (lif->notifyqcq) {
 		ionic_qcq_free(lif, lif->notifyqcq);
 		lif->notifyqcq = NULL;
@@ -219,6 +300,20 @@ static void ionic_qcqs_free(struct ionic_lif *lif)
 		ionic_qcq_free(lif, lif->adminqcq);
 		lif->adminqcq = NULL;
 	}
+
+	for (i = 0; i < lif->nxqs; i++)
+		if (lif->rxqcqs[i].stats)
+			devm_kfree(dev, lif->rxqcqs[i].stats);
+
+	devm_kfree(dev, lif->rxqcqs);
+	lif->rxqcqs = NULL;
+
+	for (i = 0; i < lif->nxqs; i++)
+		if (lif->txqcqs[i].stats)
+			devm_kfree(dev, lif->txqcqs[i].stats);
+
+	devm_kfree(dev, lif->txqcqs);
+	lif->txqcqs = NULL;
 }
 
 static void ionic_link_qcq_interrupts(struct ionic_qcq *src_qcq,
@@ -374,8 +469,11 @@ static int ionic_qcq_alloc(struct ionic_lif *lif, unsigned int type,
 
 static int ionic_qcqs_alloc(struct ionic_lif *lif)
 {
+	struct device *dev = lif->ionic->dev;
+	unsigned int q_list_size;
 	unsigned int flags;
 	int err;
+	int i;
 
 	flags = IONIC_QCQ_F_INTR;
 	err = ionic_qcq_alloc(lif, IONIC_QTYPE_ADMINQ, 0, "admin", flags,
@@ -400,8 +498,49 @@ static int ionic_qcqs_alloc(struct ionic_lif *lif)
 		ionic_link_qcq_interrupts(lif->adminqcq, lif->notifyqcq);
 	}
 
+	q_list_size = sizeof(*lif->txqcqs) * lif->nxqs;
+	err = -ENOMEM;
+	lif->txqcqs = devm_kzalloc(dev, q_list_size, GFP_KERNEL);
+	if (!lif->txqcqs)
+		goto err_out_free_notifyqcq;
+	for (i = 0; i < lif->nxqs; i++) {
+		lif->txqcqs[i].stats = devm_kzalloc(dev,
+						    sizeof(struct ionic_q_stats),
+						    GFP_KERNEL);
+		if (!lif->txqcqs[i].stats)
+			goto err_out_free_tx_stats;
+	}
+
+	lif->rxqcqs = devm_kzalloc(dev, q_list_size, GFP_KERNEL);
+	if (!lif->rxqcqs)
+		goto err_out_free_tx_stats;
+	for (i = 0; i < lif->nxqs; i++) {
+		lif->rxqcqs[i].stats = devm_kzalloc(dev,
+						    sizeof(struct ionic_q_stats),
+						    GFP_KERNEL);
+		if (!lif->rxqcqs[i].stats)
+			goto err_out_free_rx_stats;
+	}
+
 	return 0;
 
+err_out_free_rx_stats:
+	for (i = 0; i < lif->nxqs; i++)
+		if (lif->rxqcqs[i].stats)
+			devm_kfree(dev, lif->rxqcqs[i].stats);
+	devm_kfree(dev, lif->rxqcqs);
+	lif->rxqcqs = NULL;
+err_out_free_tx_stats:
+	for (i = 0; i < lif->nxqs; i++)
+		if (lif->txqcqs[i].stats)
+			devm_kfree(dev, lif->txqcqs[i].stats);
+	devm_kfree(dev, lif->txqcqs);
+	lif->txqcqs = NULL;
+err_out_free_notifyqcq:
+	if (lif->notifyqcq) {
+		ionic_qcq_free(lif, lif->notifyqcq);
+		lif->notifyqcq = NULL;
+	}
 err_out_free_adminqcq:
 	ionic_qcq_free(lif, lif->adminqcq);
 	lif->adminqcq = NULL;
@@ -409,6 +548,107 @@ static int ionic_qcqs_alloc(struct ionic_lif *lif)
 	return err;
 }
 
+static int ionic_lif_txq_init(struct ionic_lif *lif, struct ionic_qcq *qcq)
+{
+	struct device *dev = lif->ionic->dev;
+	struct ionic_queue *q = &qcq->q;
+	struct ionic_cq *cq = &qcq->cq;
+	struct ionic_admin_ctx ctx = {
+		.work = COMPLETION_INITIALIZER_ONSTACK(ctx.work),
+		.cmd.q_init = {
+			.opcode = IONIC_CMD_Q_INIT,
+			.lif_index = cpu_to_le16(lif->index),
+			.type = q->type,
+			.index = cpu_to_le32(q->index),
+			.flags = cpu_to_le16(IONIC_QINIT_F_IRQ |
+					     IONIC_QINIT_F_SG),
+			.intr_index = cpu_to_le16(lif->rxqcqs[q->index].qcq->intr.index),
+			.pid = cpu_to_le16(q->pid),
+			.ring_size = ilog2(q->num_descs),
+			.ring_base = cpu_to_le64(q->base_pa),
+			.cq_ring_base = cpu_to_le64(cq->base_pa),
+			.sg_ring_base = cpu_to_le64(q->sg_base_pa),
+		},
+	};
+	int err;
+
+	dev_dbg(dev, "txq_init.pid %d\n", ctx.cmd.q_init.pid);
+	dev_dbg(dev, "txq_init.index %d\n", ctx.cmd.q_init.index);
+	dev_dbg(dev, "txq_init.ring_base 0x%llx\n", ctx.cmd.q_init.ring_base);
+	dev_dbg(dev, "txq_init.ring_size %d\n", ctx.cmd.q_init.ring_size);
+
+	err = ionic_adminq_post_wait(lif, &ctx);
+	if (err)
+		return err;
+
+	q->hw_type = ctx.comp.q_init.hw_type;
+	q->hw_index = le32_to_cpu(ctx.comp.q_init.hw_index);
+	q->dbval = IONIC_DBELL_QID(q->hw_index);
+
+	dev_dbg(dev, "txq->hw_type %d\n", q->hw_type);
+	dev_dbg(dev, "txq->hw_index %d\n", q->hw_index);
+
+	qcq->flags |= IONIC_QCQ_F_INITED;
+
+	ionic_debugfs_add_qcq(lif, qcq);
+
+	return 0;
+}
+
+static int ionic_lif_rxq_init(struct ionic_lif *lif, struct ionic_qcq *qcq)
+{
+	struct device *dev = lif->ionic->dev;
+	struct ionic_queue *q = &qcq->q;
+	struct ionic_cq *cq = &qcq->cq;
+	struct ionic_admin_ctx ctx = {
+		.work = COMPLETION_INITIALIZER_ONSTACK(ctx.work),
+		.cmd.q_init = {
+			.opcode = IONIC_CMD_Q_INIT,
+			.lif_index = cpu_to_le16(lif->index),
+			.type = q->type,
+			.index = cpu_to_le32(q->index),
+			.flags = cpu_to_le16(IONIC_QINIT_F_IRQ),
+			.intr_index = cpu_to_le16(cq->bound_intr->index),
+			.pid = cpu_to_le16(q->pid),
+			.ring_size = ilog2(q->num_descs),
+			.ring_base = cpu_to_le64(q->base_pa),
+			.cq_ring_base = cpu_to_le64(cq->base_pa),
+		},
+	};
+	int err;
+
+	dev_dbg(dev, "rxq_init.pid %d\n", ctx.cmd.q_init.pid);
+	dev_dbg(dev, "rxq_init.index %d\n", ctx.cmd.q_init.index);
+	dev_dbg(dev, "rxq_init.ring_base 0x%llx\n", ctx.cmd.q_init.ring_base);
+	dev_dbg(dev, "rxq_init.ring_size %d\n", ctx.cmd.q_init.ring_size);
+
+	err = ionic_adminq_post_wait(lif, &ctx);
+	if (err)
+		return err;
+
+	q->hw_type = ctx.comp.q_init.hw_type;
+	q->hw_index = le32_to_cpu(ctx.comp.q_init.hw_index);
+	q->dbval = IONIC_DBELL_QID(q->hw_index);
+
+	dev_dbg(dev, "rxq->hw_type %d\n", q->hw_type);
+	dev_dbg(dev, "rxq->hw_index %d\n", q->hw_index);
+
+	netif_napi_add(lif->netdev, &qcq->napi, ionic_rx_napi,
+		       NAPI_POLL_WEIGHT);
+
+	err = ionic_request_irq(lif, qcq);
+	if (err) {
+		netif_napi_del(&qcq->napi);
+		return err;
+	}
+
+	qcq->flags |= IONIC_QCQ_F_INITED;
+
+	ionic_debugfs_add_qcq(lif, qcq);
+
+	return 0;
+}
+
 static bool ionic_notifyq_service(struct ionic_cq *cq,
 				  struct ionic_cq_info *cq_info)
 {
@@ -1074,17 +1314,180 @@ static int ionic_vlan_rx_kill_vid(struct net_device *netdev, __be16 proto,
 	return ionic_adminq_post_wait(lif, &ctx);
 }
 
+static void ionic_txrx_disable(struct ionic_lif *lif)
+{
+	unsigned int i;
+
+	for (i = 0; i < lif->nxqs; i++) {
+		ionic_qcq_disable(lif->txqcqs[i].qcq);
+		ionic_qcq_disable(lif->rxqcqs[i].qcq);
+	}
+}
+
+static void ionic_txrx_deinit(struct ionic_lif *lif)
+{
+	unsigned int i;
+
+	for (i = 0; i < lif->nxqs; i++) {
+		ionic_lif_qcq_deinit(lif, lif->txqcqs[i].qcq);
+		ionic_tx_flush(&lif->txqcqs[i].qcq->cq);
+
+		ionic_lif_qcq_deinit(lif, lif->rxqcqs[i].qcq);
+		ionic_rx_flush(&lif->rxqcqs[i].qcq->cq);
+		ionic_rx_empty(&lif->rxqcqs[i].qcq->q);
+	}
+}
+
+static void ionic_txrx_free(struct ionic_lif *lif)
+{
+	unsigned int i;
+
+	for (i = 0; i < lif->nxqs; i++) {
+		ionic_qcq_free(lif, lif->txqcqs[i].qcq);
+		lif->txqcqs[i].qcq = NULL;
+
+		ionic_qcq_free(lif, lif->rxqcqs[i].qcq);
+		lif->rxqcqs[i].qcq = NULL;
+	}
+}
+
+static int ionic_txrx_alloc(struct ionic_lif *lif)
+{
+	unsigned int flags;
+	unsigned int i;
+	int err = 0;
+
+	flags = IONIC_QCQ_F_TX_STATS | IONIC_QCQ_F_SG;
+	for (i = 0; i < lif->nxqs; i++) {
+		err = ionic_qcq_alloc(lif, IONIC_QTYPE_TXQ, i, "tx", flags,
+				      lif->ntxq_descs,
+				      sizeof(struct ionic_txq_desc),
+				      sizeof(struct ionic_txq_comp),
+				      sizeof(struct ionic_txq_sg_desc),
+				      lif->kern_pid, &lif->txqcqs[i].qcq);
+		if (err)
+			goto err_out;
+
+		lif->txqcqs[i].qcq->stats = lif->txqcqs[i].stats;
+	}
+
+	flags = IONIC_QCQ_F_RX_STATS | IONIC_QCQ_F_INTR;
+	for (i = 0; i < lif->nxqs; i++) {
+		err = ionic_qcq_alloc(lif, IONIC_QTYPE_RXQ, i, "rx", flags,
+				      lif->nrxq_descs,
+				      sizeof(struct ionic_rxq_desc),
+				      sizeof(struct ionic_rxq_comp),
+				      0, lif->kern_pid, &lif->rxqcqs[i].qcq);
+		if (err)
+			goto err_out;
+
+		lif->rxqcqs[i].qcq->stats = lif->rxqcqs[i].stats;
+
+		ionic_link_qcq_interrupts(lif->rxqcqs[i].qcq,
+					  lif->txqcqs[i].qcq);
+	}
+
+	return 0;
+
+err_out:
+	ionic_txrx_free(lif);
+
+	return err;
+}
+
+static int ionic_txrx_init(struct ionic_lif *lif)
+{
+	unsigned int i;
+	int err;
+
+	for (i = 0; i < lif->nxqs; i++) {
+		err = ionic_lif_txq_init(lif, lif->txqcqs[i].qcq);
+		if (err)
+			goto err_out;
+
+		err = ionic_lif_rxq_init(lif, lif->rxqcqs[i].qcq);
+		if (err) {
+			ionic_lif_qcq_deinit(lif, lif->txqcqs[i-1].qcq);
+			goto err_out;
+		}
+	}

drivers/net/ethernet/pensando/ionic/ionic_lif.c static int ionic_vlan_rx_kill_vid(struct net_device *netdev, __be16 proto, · line 1423

Quoted 10 lines
drivers/net/ethernet/pensando/ionic/ionic_lif.c +10 -0
+
+	ionic_set_rx_mode(lif->netdev);
+
+	return 0;
+
+err_out:
+	for (i--; i > 0; i--) {
+		ionic_lif_qcq_deinit(lif, lif->txqcqs[i-1].qcq);
+		ionic_lif_qcq_deinit(lif, lif->rxqcqs[i-1].qcq);
+	}

The "i--" has been done in for initialization, and ionic_lif_qcq_deinit is called with lif->rxqcqs[i-1], which may cause the last lif->txqcqs or lif->rxqcqs not initialized problem.

It may be more common to do the below:

while (i--) {
	ionic_lif_qcq_deinit(lif, lif->txqcqs[i].qcq);
	ionic_lif_qcq_deinit(lif, lif->rxqcqs[i].qcq);
}
Quoted 18 lines
drivers/net/ethernet/pensando/ionic/ionic_lif.c +18 -0
+
+	return err;
+}
+
+static int ionic_txrx_enable(struct ionic_lif *lif)
+{
+	int i, err;
+
+	for (i = 0; i < lif->nxqs; i++) {
+		err = ionic_qcq_enable(lif->txqcqs[i].qcq);
+		if (err)
+			goto err_out;
+
+		ionic_rx_fill(&lif->rxqcqs[i].qcq->q);
+		err = ionic_qcq_enable(lif->rxqcqs[i].qcq);
+		if (err) {
+			ionic_qcq_disable(lif->txqcqs[i].qcq);
+			goto err_out;

drivers/net/ethernet/pensando/ionic/ionic_lif.c static int ionic_vlan_rx_kill_vid(struct net_device *netdev, __be16 proto, · line 1451

Quoted 10 lines
drivers/net/ethernet/pensando/ionic/ionic_lif.c +10 -0
+		}
+	}
+
+	return 0;
+
+err_out:
+	for (i--; i >= 0 ; i--) {
+		ionic_qcq_disable(lif->rxqcqs[i].qcq);
+		ionic_qcq_disable(lif->txqcqs[i].qcq);
+	}

It may be better to use the above pattern too.

Quoted 544 lines
drivers/net/ethernet/pensando/ionic/ionic_lif.c +39 -0
+
+	return err;
+}
+
 int ionic_open(struct net_device *netdev)
 {
 	struct ionic_lif *lif = netdev_priv(netdev);
+	int err;
 
 	netif_carrier_off(netdev);
 
+	err = ionic_txrx_alloc(lif);
+	if (err)
+		return err;
+
+	err = ionic_txrx_init(lif);
+	if (err)
+		goto err_txrx_free;
+
+	err = ionic_txrx_enable(lif);
+	if (err)
+		goto err_txrx_deinit;
+
+	netif_set_real_num_tx_queues(netdev, lif->nxqs);
+	netif_set_real_num_rx_queues(netdev, lif->nxqs);
+
 	set_bit(IONIC_LIF_UP, lif->state);
 
 	ionic_link_status_check_request(lif);
+	if (netif_carrier_ok(netdev))
+		netif_tx_wake_all_queues(netdev);
 
 	return 0;
+
+err_txrx_deinit:
+	ionic_txrx_deinit(lif);
+err_txrx_free:
+	ionic_txrx_free(lif);
+	return err;
 }
 
 int ionic_stop(struct net_device *netdev)
@@ -1102,6 +1505,12 @@ int ionic_stop(struct net_device *netdev)
 
 	/* carrier off before disabling queues to avoid watchdog timeout */
 	netif_carrier_off(netdev);
+	netif_tx_stop_all_queues(netdev);
+	netif_tx_disable(netdev);
+
+	ionic_txrx_disable(lif);
+	ionic_txrx_deinit(lif);
+	ionic_txrx_free(lif);
 
 	return err;
 }
@@ -1109,6 +1518,7 @@ int ionic_stop(struct net_device *netdev)
 static const struct net_device_ops ionic_netdev_ops = {
 	.ndo_open               = ionic_open,
 	.ndo_stop               = ionic_stop,
+	.ndo_start_xmit		= ionic_start_xmit,
 	.ndo_get_stats64	= ionic_get_stats64,
 	.ndo_set_rx_mode	= ionic_set_rx_mode,
 	.ndo_set_features	= ionic_set_features,
@@ -1173,6 +1583,8 @@ static struct ionic_lif *ionic_lif_alloc(struct ionic *ionic, unsigned int index
 
 	lif->ionic = ionic;
 	lif->index = index;
+	lif->ntxq_descs = IONIC_DEF_TXRX_DESC;
+	lif->nrxq_descs = IONIC_DEF_TXRX_DESC;
 
 	snprintf(lif->name, sizeof(lif->name), "lif%u", index);
 
@@ -1499,6 +1911,8 @@ static int ionic_lif_init(struct ionic_lif *lif)
 	if (err)
 		goto err_out_notifyq_deinit;
 
+	lif->rx_copybreak = IONIC_RX_COPYBREAK_DEFAULT;
+
 	set_bit(IONIC_LIF_INITED, lif->state);
 
 	return 0;
drivers/net/ethernet/pensando/ionic/ionic_lif.h +50 -0
index f927e1cfd646..d2fae13def44 100644
@@ -12,20 +12,38 @@
 
 #define IONIC_MAX_NUM_NAPI_CNTR		(NAPI_POLL_WEIGHT + 1)
 #define IONIC_MAX_NUM_SG_CNTR		(IONIC_TX_MAX_SG_ELEMS + 1)
+#define IONIC_RX_COPYBREAK_DEFAULT		256
 
 struct ionic_tx_stats {
+	u64 dma_map_err;
 	u64 pkts;
 	u64 bytes;
+	u64 clean;
+	u64 linearize;
+	u64 no_csum;
+	u64 csum;
+	u64 crc32_csum;
+	u64 tso;
+	u64 frags;
+	u64 sg_cntr[IONIC_MAX_NUM_SG_CNTR];
 };
 
 struct ionic_rx_stats {
+	u64 dma_map_err;
+	u64 alloc_err;
 	u64 pkts;
 	u64 bytes;
+	u64 csum_none;
+	u64 csum_complete;
+	u64 csum_error;
+	u64 buffers_posted;
 };
 
 #define IONIC_QCQ_F_INITED		BIT(0)
 #define IONIC_QCQ_F_SG			BIT(1)
 #define IONIC_QCQ_F_INTR		BIT(2)
+#define IONIC_QCQ_F_TX_STATS		BIT(3)
+#define IONIC_QCQ_F_RX_STATS		BIT(4)
 #define IONIC_QCQ_F_NOTIFYQ		BIT(5)
 
 struct ionic_napi_stats {
@@ -54,7 +72,14 @@ struct ionic_qcq {
 	struct dentry *dentry;
 };
 
+struct ionic_qcqst {
+	struct ionic_qcq *qcq;
+	struct ionic_q_stats *stats;
+};
+
 #define q_to_qcq(q)		container_of(q, struct ionic_qcq, q)
+#define q_to_tx_stats(q)	(&q_to_qcq(q)->stats->tx)
+#define q_to_rx_stats(q)	(&q_to_qcq(q)->stats->rx)
 #define napi_to_qcq(napi)	container_of(napi, struct ionic_qcq, napi)
 #define napi_to_cq(napi)	(&napi_to_qcq(napi)->cq)
 
@@ -106,11 +131,14 @@ struct ionic_lif {
 	spinlock_t adminq_lock;		/* lock for AdminQ operations */
 	struct ionic_qcq *adminqcq;
 	struct ionic_qcq *notifyqcq;
+	struct ionic_qcqst *txqcqs;
+	struct ionic_qcqst *rxqcqs;
 	u64 last_eid;
 	unsigned int neqs;
 	unsigned int nxqs;
 	unsigned int ntxq_descs;
 	unsigned int nrxq_descs;
+	u32 rx_copybreak;
 	unsigned int rx_mode;
 	u64 hw_features;
 	bool mc_overflow;
@@ -131,6 +159,11 @@ struct ionic_lif {
 	u32 flags;
 };
 
+#define lif_to_txqcq(lif, i)	((lif)->txqcqs[i].qcq)
+#define lif_to_rxqcq(lif, i)	((lif)->rxqcqs[i].qcq)
+#define lif_to_txq(lif, i)	(&lif_to_txqcq((lif), i)->q)
+#define lif_to_rxq(lif, i)	(&lif_to_txqcq((lif), i)->q)
+
 static inline int ionic_wait_for_bit(struct ionic_lif *lif, int bitname)
 {
 	unsigned long tlimit = jiffies + HZ;
@@ -181,6 +214,20 @@ int ionic_open(struct net_device *netdev);
 int ionic_stop(struct net_device *netdev);
 int ionic_reset_queues(struct ionic_lif *lif);
 
+static inline void debug_stats_txq_post(struct ionic_qcq *qcq,
+					struct ionic_txq_desc *desc, bool dbell)
+{
+	u8 num_sg_elems = ((le64_to_cpu(desc->cmd) >> IONIC_TXQ_DESC_NSGE_SHIFT)
+						& IONIC_TXQ_DESC_NSGE_MASK);
+
+	qcq->q.dbell_count += dbell;
+
+	if (num_sg_elems > (IONIC_MAX_NUM_SG_CNTR - 1))
+		num_sg_elems = IONIC_MAX_NUM_SG_CNTR - 1;
+
+	qcq->stats->tx.sg_cntr[num_sg_elems]++;
+}
+
 static inline void debug_stats_napi_poll(struct ionic_qcq *qcq,
 					 unsigned int work_done)
 {
@@ -193,7 +240,10 @@ static inline void debug_stats_napi_poll(struct ionic_qcq *qcq,
 }
 
 #define DEBUG_STATS_CQE_CNT(cq)		((cq)->compl_count++)
+#define DEBUG_STATS_RX_BUFF_CNT(qcq)	((qcq)->stats->rx.buffers_posted++)
 #define DEBUG_STATS_INTR_REARM(intr)	((intr)->rearm_count++)
+#define DEBUG_STATS_TXQ_POST(qcq, txdesc, dbell) \
+	debug_stats_txq_post(qcq, txdesc, dbell)
 #define DEBUG_STATS_NAPI_POLL(qcq, work_done) \
 	debug_stats_napi_poll(qcq, work_done)
 
drivers/net/ethernet/pensando/ionic/ionic_txrx.c new file +340 -0
new file mode 100644
index 000000000000..3ec28f021d28
@@ -0,0 +1,896 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+
+#include <linux/ip.h>
+#include <linux/ipv6.h>
+#include <linux/if_vlan.h>
+#include <net/ip6_checksum.h>
+
+#include "ionic.h"
+#include "ionic_lif.h"
+#include "ionic_txrx.h"
+
+static void ionic_rx_clean(struct ionic_queue *q, struct ionic_desc_info *desc_info,
+			   struct ionic_cq_info *cq_info, void *cb_arg);
+
+static inline void ionic_txq_post(struct ionic_queue *q, bool ring_dbell,
+				  ionic_desc_cb cb_func, void *cb_arg)
+{
+	DEBUG_STATS_TXQ_POST(q_to_qcq(q), q->head->desc, ring_dbell);
+
+	ionic_q_post(q, ring_dbell, cb_func, cb_arg);
+}
+
+static inline void ionic_rxq_post(struct ionic_queue *q, bool ring_dbell,
+				  ionic_desc_cb cb_func, void *cb_arg)
+{
+	ionic_q_post(q, ring_dbell, cb_func, cb_arg);
+
+	DEBUG_STATS_RX_BUFF_CNT(q_to_qcq(q));
+}
+
+static inline struct netdev_queue *q_to_ndq(struct ionic_queue *q)
+{
+	return netdev_get_tx_queue(q->lif->netdev, q->index);
+}
+
+static void ionic_rx_recycle(struct ionic_queue *q, struct ionic_desc_info *desc_info,
+			     struct sk_buff *skb)
+{
+	struct ionic_rxq_desc *old = desc_info->desc;
+	struct ionic_rxq_desc *new = q->head->desc;
+
+	new->addr = old->addr;
+	new->len = old->len;
+
+	ionic_rxq_post(q, true, ionic_rx_clean, skb);
+}
+
+static bool ionic_rx_copybreak(struct ionic_queue *q, struct ionic_desc_info *desc_info,
+			       struct ionic_cq_info *cq_info, struct sk_buff **skb)
+{
+	struct ionic_rxq_comp *comp = cq_info->cq_desc;
+	struct ionic_rxq_desc *desc = desc_info->desc;
+	struct net_device *netdev = q->lif->netdev;
+	struct device *dev = q->lif->ionic->dev;
+	struct sk_buff *new_skb;
+	u16 clen, dlen;
+
+	clen = le16_to_cpu(comp->len);
+	dlen = le16_to_cpu(desc->len);
+	if (clen > q->lif->rx_copybreak) {
+		dma_unmap_single(dev, (dma_addr_t)le64_to_cpu(desc->addr),
+				 dlen, DMA_FROM_DEVICE);
+		return false;
+	}
+
+	new_skb = netdev_alloc_skb_ip_align(netdev, clen);
+	if (!new_skb) {
+		dma_unmap_single(dev, (dma_addr_t)le64_to_cpu(desc->addr),
+				 dlen, DMA_FROM_DEVICE);
+		return false;
+	}
+
+	dma_sync_single_for_cpu(dev, (dma_addr_t)le64_to_cpu(desc->addr),
+				clen, DMA_FROM_DEVICE);
+
+	memcpy(new_skb->data, (*skb)->data, clen);
+
+	ionic_rx_recycle(q, desc_info, *skb);
+	*skb = new_skb;
+
+	return true;
+}
+
+static void ionic_rx_clean(struct ionic_queue *q, struct ionic_desc_info *desc_info,
+			   struct ionic_cq_info *cq_info, void *cb_arg)
+{
+	struct ionic_rxq_comp *comp = cq_info->cq_desc;
+	struct ionic_qcq *qcq = q_to_qcq(q);
+	struct sk_buff *skb = cb_arg;
+	struct ionic_rx_stats *stats;
+	struct net_device *netdev;
+
+	stats = q_to_rx_stats(q);
+	netdev = q->lif->netdev;
+
+	if (comp->status) {
+		ionic_rx_recycle(q, desc_info, skb);
+		return;
+	}
+
+	if (unlikely(test_bit(IONIC_LIF_QUEUE_RESET, q->lif->state))) {
+		/* no packet processing while resetting */
+		ionic_rx_recycle(q, desc_info, skb);
+		return;
+	}
+
+	stats->pkts++;
+	stats->bytes += le16_to_cpu(comp->len);
+
+	ionic_rx_copybreak(q, desc_info, cq_info, &skb);
+
+	skb_put(skb, le16_to_cpu(comp->len));
+	skb->protocol = eth_type_trans(skb, netdev);
+
+	skb_record_rx_queue(skb, q->index);
+
+	if (netdev->features & NETIF_F_RXHASH) {
+		switch (comp->pkt_type_color & IONIC_RXQ_COMP_PKT_TYPE_MASK) {
+		case IONIC_PKT_TYPE_IPV4:
+		case IONIC_PKT_TYPE_IPV6:
+			skb_set_hash(skb, le32_to_cpu(comp->rss_hash),
+				     PKT_HASH_TYPE_L3);
+			break;
+		case IONIC_PKT_TYPE_IPV4_TCP:
+		case IONIC_PKT_TYPE_IPV6_TCP:
+		case IONIC_PKT_TYPE_IPV4_UDP:
+		case IONIC_PKT_TYPE_IPV6_UDP:
+			skb_set_hash(skb, le32_to_cpu(comp->rss_hash),
+				     PKT_HASH_TYPE_L4);
+			break;
+		}
+	}
+
+	if (netdev->features & NETIF_F_RXCSUM) {
+		if (comp->csum_flags & IONIC_RXQ_COMP_CSUM_F_CALC) {
+			skb->ip_summed = CHECKSUM_COMPLETE;
+			skb->csum = (__wsum)le16_to_cpu(comp->csum);
+			stats->csum_complete++;
+		}
+	} else {
+		stats->csum_none++;
+	}
+
+	if ((comp->csum_flags & IONIC_RXQ_COMP_CSUM_F_TCP_BAD) ||
+	    (comp->csum_flags & IONIC_RXQ_COMP_CSUM_F_UDP_BAD) ||
+	    (comp->csum_flags & IONIC_RXQ_COMP_CSUM_F_IP_BAD))
+		stats->csum_error++;
+
+	if (netdev->features & NETIF_F_HW_VLAN_CTAG_RX) {
+		if (comp->csum_flags & IONIC_RXQ_COMP_CSUM_F_VLAN)
+			__vlan_hwaccel_put_tag(skb, htons(ETH_P_8021Q),
+					       le16_to_cpu(comp->vlan_tci));
+	}
+
+	napi_gro_receive(&qcq->napi, skb);
+}
+
+static bool ionic_rx_service(struct ionic_cq *cq, struct ionic_cq_info *cq_info)
+{
+	struct ionic_rxq_comp *comp = cq_info->cq_desc;
+	struct ionic_queue *q = cq->bound_q;
+	struct ionic_desc_info *desc_info;
+
+	if (!color_match(comp->pkt_type_color, cq->done_color))
+		return false;
+
+	/* check for empty queue */
+	if (q->tail->index == q->head->index)
+		return false;
+
+	desc_info = q->tail;
+	if (desc_info->index != le16_to_cpu(comp->comp_index))
+		return false;
+
+	q->tail = desc_info->next;
+
+	/* clean the related q entry, only one per qc completion */
+	ionic_rx_clean(q, desc_info, cq_info, desc_info->cb_arg);
+
+	desc_info->cb = NULL;
+	desc_info->cb_arg = NULL;
+
+	return true;
+}
+
+static u32 ionic_rx_walk_cq(struct ionic_cq *rxcq, u32 limit)
+{
+	u32 work_done = 0;
+
+	while (ionic_rx_service(rxcq, rxcq->tail)) {
+		if (rxcq->tail->last)
+			rxcq->done_color = !rxcq->done_color;
+		rxcq->tail = rxcq->tail->next;
+		DEBUG_STATS_CQE_CNT(rxcq);
+
+		if (++work_done >= limit)
+			break;
+	}
+
+	return work_done;
+}
+
+void ionic_rx_flush(struct ionic_cq *cq)
+{
+	struct ionic_dev *idev = &cq->lif->ionic->idev;
+	u32 work_done;
+
+	work_done = ionic_rx_walk_cq(cq, cq->num_descs);
+
+	if (work_done)
+		ionic_intr_credits(idev->intr_ctrl, cq->bound_intr->index,
+				   work_done, IONIC_INTR_CRED_RESET_COALESCE);
+}
+
+static struct sk_buff *ionic_rx_skb_alloc(struct ionic_queue *q, unsigned int len,
+					  dma_addr_t *dma_addr)
+{
+	struct ionic_lif *lif = q->lif;
+	struct ionic_rx_stats *stats;
+	struct net_device *netdev;
+	struct sk_buff *skb;
+	struct device *dev;
+
+	netdev = lif->netdev;
+	dev = lif->ionic->dev;
+	stats = q_to_rx_stats(q);
+	skb = netdev_alloc_skb_ip_align(netdev, len);
+	if (!skb) {
+		net_warn_ratelimited("%s: SKB alloc failed on %s!\n",
+				     netdev->name, q->name);
+		stats->alloc_err++;
+		return NULL;
+	}
+
+	*dma_addr = dma_map_single(dev, skb->data, len, DMA_FROM_DEVICE);
+	if (dma_mapping_error(dev, *dma_addr)) {
+		dev_kfree_skb(skb);
+		net_warn_ratelimited("%s: DMA single map failed on %s!\n",
+				     netdev->name, q->name);
+		stats->dma_map_err++;
+		return NULL;
+	}
+
+	return skb;
+}
+
+#define IONIC_RX_RING_DOORBELL_STRIDE		((1 << 2) - 1)
+
+void ionic_rx_fill(struct ionic_queue *q)
+{
+	struct net_device *netdev = q->lif->netdev;
+	struct ionic_rxq_desc *desc;
+	struct sk_buff *skb;
+	dma_addr_t dma_addr;
+	bool ring_doorbell;
+	unsigned int len;
+	unsigned int i;
+
+	len = netdev->mtu + ETH_HLEN;
+
+	for (i = ionic_q_space_avail(q); i; i--) {
+		skb = ionic_rx_skb_alloc(q, len, &dma_addr);
+		if (!skb)
+			return;
+
+		desc = q->head->desc;
+		desc->addr = cpu_to_le64(dma_addr);
+		desc->len = cpu_to_le16(len);
+		desc->opcode = IONIC_RXQ_DESC_OPCODE_SIMPLE;
+
+		ring_doorbell = ((q->head->index + 1) &
+				IONIC_RX_RING_DOORBELL_STRIDE) == 0;
+
+		ionic_rxq_post(q, ring_doorbell, ionic_rx_clean, skb);
+	}
+}
+
+static void ionic_rx_fill_cb(void *arg)
+{
+	ionic_rx_fill(arg);
+}
+
+void ionic_rx_empty(struct ionic_queue *q)
+{
+	struct device *dev = q->lif->ionic->dev;
+	struct ionic_desc_info *cur;
+	struct ionic_rxq_desc *desc;
+
+	for (cur = q->tail; cur != q->head; cur = cur->next) {
+		desc = cur->desc;
+
+		dma_unmap_single(dev, le64_to_cpu(desc->addr),
+				 le16_to_cpu(desc->len), DMA_FROM_DEVICE);
+		dev_kfree_skb(cur->cb_arg);
+
+		cur->cb_arg = NULL;
+	}
+}
+
+int ionic_rx_napi(struct napi_struct *napi, int budget)
+{
+	struct ionic_qcq *qcq = napi_to_qcq(napi);
+	struct ionic_cq *rxcq = napi_to_cq(napi);
+	unsigned int qi = rxcq->bound_q->index;
+	struct ionic_dev *idev;
+	struct ionic_lif *lif;
+	struct ionic_cq *txcq;
+	u32 work_done = 0;
+	u32 flags = 0;
+
+	lif = rxcq->bound_q->lif;
+	idev = &lif->ionic->idev;
+	txcq = &lif->txqcqs[qi].qcq->cq;
+
+	ionic_tx_flush(txcq);
+
+	work_done = ionic_rx_walk_cq(rxcq, budget);
+
+	if (work_done)
+		ionic_rx_fill_cb(rxcq->bound_q);
+
+	if (work_done < budget && napi_complete_done(napi, work_done)) {
+		flags |= IONIC_INTR_CRED_UNMASK;
+		DEBUG_STATS_INTR_REARM(rxcq->bound_intr);
+	}
+
+	if (work_done || flags) {
+		flags |= IONIC_INTR_CRED_RESET_COALESCE;
+		ionic_intr_credits(idev->intr_ctrl, rxcq->bound_intr->index,
+				   work_done, flags);
+	}
+
+	DEBUG_STATS_NAPI_POLL(qcq, work_done);
+
+	return work_done;
+}
+
+static dma_addr_t ionic_tx_map_single(struct ionic_queue *q, void *data, size_t len)
+{

drivers/net/ethernet/pensando/ionic/ionic_txrx.c line 350

Quoted 10 lines
drivers/net/ethernet/pensando/ionic/ionic_txrx.c new file +10 -0
+	struct ionic_tx_stats *stats = q_to_tx_stats(q);
+	struct device *dev = q->lif->ionic->dev;
+	dma_addr_t dma_addr;
+
+	dma_addr = dma_map_single(dev, data, len, DMA_TO_DEVICE);
+	if (dma_mapping_error(dev, dma_addr)) {
+		net_warn_ratelimited("%s: DMA single map failed on %s!\n",
+				     q->lif->netdev->name, q->name);
+		stats->dma_map_err++;
+		return 0;

zero may be a valid dma address, maybe check the dma_mapping_error in ionic_tx_tso instead.

Quoted 100 lines
drivers/net/ethernet/pensando/ionic/ionic_txrx.c new file +100 -0
+	}
+	return dma_addr;
+}
+
+static dma_addr_t ionic_tx_map_frag(struct ionic_queue *q, const skb_frag_t *frag,
+				    size_t offset, size_t len)
+{
+	struct ionic_tx_stats *stats = q_to_tx_stats(q);
+	struct device *dev = q->lif->ionic->dev;
+	dma_addr_t dma_addr;
+
+	dma_addr = skb_frag_dma_map(dev, frag, offset, len, DMA_TO_DEVICE);
+	if (dma_mapping_error(dev, dma_addr)) {
+		net_warn_ratelimited("%s: DMA frag map failed on %s!\n",
+				     q->lif->netdev->name, q->name);
+		stats->dma_map_err++;
+		return 0;
+	}
+	return dma_addr;
+}
+
+static void ionic_tx_clean(struct ionic_queue *q, struct ionic_desc_info *desc_info,
+			   struct ionic_cq_info *cq_info, void *cb_arg)
+{
+	struct ionic_txq_sg_desc *sg_desc = desc_info->sg_desc;
+	struct ionic_txq_sg_elem *elem = sg_desc->elems;
+	struct ionic_tx_stats *stats = q_to_tx_stats(q);
+	struct ionic_txq_desc *desc = desc_info->desc;
+	struct device *dev = q->lif->ionic->dev;
+	u8 opcode, flags, nsge;
+	u16 queue_index;
+	unsigned int i;
+	u64 addr;
+
+	decode_txq_desc_cmd(le64_to_cpu(desc->cmd),
+			    &opcode, &flags, &nsge, &addr);
+
+	/* use unmap_single only if either this is not TSO,
+	 * or this is first descriptor of a TSO
+	 */
+	if (opcode != IONIC_TXQ_DESC_OPCODE_TSO ||
+	    flags & IONIC_TXQ_DESC_FLAG_TSO_SOT)
+		dma_unmap_single(dev, (dma_addr_t)addr,
+				 le16_to_cpu(desc->len), DMA_TO_DEVICE);
+	else
+		dma_unmap_page(dev, (dma_addr_t)addr,
+			       le16_to_cpu(desc->len), DMA_TO_DEVICE);
+
+	for (i = 0; i < nsge; i++, elem++)
+		dma_unmap_page(dev, (dma_addr_t)le64_to_cpu(elem->addr),
+			       le16_to_cpu(elem->len), DMA_TO_DEVICE);
+
+	if (cb_arg) {
+		struct sk_buff *skb = cb_arg;
+		u32 len = skb->len;
+
+		queue_index = skb_get_queue_mapping(skb);
+		if (unlikely(__netif_subqueue_stopped(q->lif->netdev,
+						      queue_index))) {
+			netif_wake_subqueue(q->lif->netdev, queue_index);
+			q->wake++;
+		}
+		dev_kfree_skb_any(skb);
+		stats->clean++;
+		netdev_tx_completed_queue(q_to_ndq(q), 1, len);
+	}
+}
+
+void ionic_tx_flush(struct ionic_cq *cq)
+{
+	struct ionic_txq_comp *comp = cq->tail->cq_desc;
+	struct ionic_dev *idev = &cq->lif->ionic->idev;
+	struct ionic_queue *q = cq->bound_q;
+	struct ionic_desc_info *desc_info;
+	unsigned int work_done = 0;
+
+	/* walk the completed cq entries */
+	while (work_done < cq->num_descs &&
+	       color_match(comp->color, cq->done_color)) {
+
+		/* clean the related q entries, there could be
+		 * several q entries completed for each cq completion
+		 */
+		do {
+			desc_info = q->tail;
+			q->tail = desc_info->next;
+			ionic_tx_clean(q, desc_info, cq->tail,
+				       desc_info->cb_arg);
+			desc_info->cb = NULL;
+			desc_info->cb_arg = NULL;
+		} while (desc_info->index != le16_to_cpu(comp->comp_index));
+
+		if (cq->tail->last)
+			cq->done_color = !cq->done_color;
+
+		cq->tail = cq->tail->next;
+		comp = cq->tail->cq_desc;
+		DEBUG_STATS_CQE_CNT(cq);
+
+		work_done++;

drivers/net/ethernet/pensando/ionic/ionic_txrx.c line 460

Quoted 10 lines
drivers/net/ethernet/pensando/ionic/ionic_txrx.c new file +10 -0
+	}
+
+	if (work_done)
+		ionic_intr_credits(idev->intr_ctrl, cq->bound_intr->index,
+				   work_done, 0);
+}
+
+static void ionic_tx_tcp_inner_pseudo_csum(struct sk_buff *skb)
+{
+	skb_cow_head(skb, 0);

May need to check for return error of skb_cow_head.

Quoted 458 lines
drivers/net/ethernet/pensando/ionic/ionic_txrx.c new file +436 -0
+
+	if (skb->protocol == cpu_to_be16(ETH_P_IP)) {
+		inner_ip_hdr(skb)->check = 0;
+		inner_tcp_hdr(skb)->check =
+			~csum_tcpudp_magic(inner_ip_hdr(skb)->saddr,
+					   inner_ip_hdr(skb)->daddr,
+					   0, IPPROTO_TCP, 0);
+	} else if (skb->protocol == cpu_to_be16(ETH_P_IPV6)) {
+		inner_tcp_hdr(skb)->check =
+			~csum_ipv6_magic(&inner_ipv6_hdr(skb)->saddr,
+					 &inner_ipv6_hdr(skb)->daddr,
+					 0, IPPROTO_TCP, 0);
+	}
+}
+
+static void ionic_tx_tcp_pseudo_csum(struct sk_buff *skb)
+{
+	skb_cow_head(skb, 0);
+
+	if (skb->protocol == cpu_to_be16(ETH_P_IP)) {
+		ip_hdr(skb)->check = 0;
+		tcp_hdr(skb)->check =
+			~csum_tcpudp_magic(ip_hdr(skb)->saddr,
+					   ip_hdr(skb)->daddr,
+					   0, IPPROTO_TCP, 0);
+	} else if (skb->protocol == cpu_to_be16(ETH_P_IPV6)) {
+		tcp_hdr(skb)->check =
+			~csum_ipv6_magic(&ipv6_hdr(skb)->saddr,
+					 &ipv6_hdr(skb)->daddr,
+					 0, IPPROTO_TCP, 0);
+	}
+}
+
+static void ionic_tx_tso_post(struct ionic_queue *q, struct ionic_txq_desc *desc,
+			      struct sk_buff *skb,
+			      dma_addr_t addr, u8 nsge, u16 len,
+			      unsigned int hdrlen, unsigned int mss,
+			      bool outer_csum,
+			      u16 vlan_tci, bool has_vlan,
+			      bool start, bool done)
+{
+	u8 flags = 0;
+	u64 cmd;
+
+	flags |= has_vlan ? IONIC_TXQ_DESC_FLAG_VLAN : 0;
+	flags |= outer_csum ? IONIC_TXQ_DESC_FLAG_ENCAP : 0;
+	flags |= start ? IONIC_TXQ_DESC_FLAG_TSO_SOT : 0;
+	flags |= done ? IONIC_TXQ_DESC_FLAG_TSO_EOT : 0;
+
+	cmd = encode_txq_desc_cmd(IONIC_TXQ_DESC_OPCODE_TSO, flags, nsge, addr);
+	desc->cmd = cpu_to_le64(cmd);
+	desc->len = cpu_to_le16(len);
+	desc->vlan_tci = cpu_to_le16(vlan_tci);
+	desc->hdr_len = cpu_to_le16(hdrlen);
+	desc->mss = cpu_to_le16(mss);
+
+	if (done) {
+		skb_tx_timestamp(skb);
+		netdev_tx_sent_queue(q_to_ndq(q), skb->len);
+		ionic_txq_post(q, !netdev_xmit_more(), ionic_tx_clean, skb);
+	} else {
+		ionic_txq_post(q, false, ionic_tx_clean, NULL);
+	}
+}
+
+static struct ionic_txq_desc *ionic_tx_tso_next(struct ionic_queue *q,
+						struct ionic_txq_sg_elem **elem)
+{
+	struct ionic_txq_sg_desc *sg_desc = q->head->sg_desc;
+	struct ionic_txq_desc *desc = q->head->desc;
+
+	*elem = sg_desc->elems;
+	return desc;
+}
+
+static int ionic_tx_tso(struct ionic_queue *q, struct sk_buff *skb)
+{
+	struct ionic_tx_stats *stats = q_to_tx_stats(q);
+	struct ionic_desc_info *abort = q->head;
+	struct ionic_desc_info *rewind = abort;
+	struct ionic_txq_sg_elem *elem;
+	struct ionic_txq_desc *desc;
+	unsigned int frag_left = 0;
+	unsigned int offset = 0;
+	unsigned int len_left;
+	dma_addr_t desc_addr;
+	unsigned int hdrlen;
+	unsigned int nfrags;
+	unsigned int seglen;
+	u64 total_bytes = 0;
+	u64 total_pkts = 0;
+	unsigned int left;
+	unsigned int len;
+	unsigned int mss;
+	skb_frag_t *frag;
+	bool start, done;
+	bool outer_csum;
+	bool has_vlan;
+	u16 desc_len;
+	u8 desc_nsge;
+	u16 vlan_tci;
+	bool encap;
+
+	mss = skb_shinfo(skb)->gso_size;
+	nfrags = skb_shinfo(skb)->nr_frags;
+	len_left = skb->len - skb_headlen(skb);
+	outer_csum = (skb_shinfo(skb)->gso_type & SKB_GSO_GRE_CSUM) ||
+		     (skb_shinfo(skb)->gso_type & SKB_GSO_UDP_TUNNEL_CSUM);
+	has_vlan = !!skb_vlan_tag_present(skb);
+	vlan_tci = skb_vlan_tag_get(skb);
+	encap = skb->encapsulation;
+
+	/* Preload inner-most TCP csum field with IP pseudo hdr
+	 * calculated with IP length set to zero.  HW will later
+	 * add in length to each TCP segment resulting from the TSO.
+	 */
+
+	if (encap)
+		ionic_tx_tcp_inner_pseudo_csum(skb);
+	else
+		ionic_tx_tcp_pseudo_csum(skb);
+
+	if (encap)
+		hdrlen = skb_inner_transport_header(skb) - skb->data +
+			 inner_tcp_hdrlen(skb);
+	else
+		hdrlen = skb_transport_offset(skb) + tcp_hdrlen(skb);
+
+	seglen = hdrlen + mss;
+	left = skb_headlen(skb);
+
+	desc = ionic_tx_tso_next(q, &elem);
+	start = true;
+
+	/* Chop skb->data up into desc segments */
+
+	while (left > 0) {
+		len = min(seglen, left);
+		frag_left = seglen - len;
+		desc_addr = ionic_tx_map_single(q, skb->data + offset, len);
+		if (!desc_addr)
+			goto err_out_abort;
+		desc_len = len;
+		desc_nsge = 0;
+		left -= len;
+		offset += len;
+		if (nfrags > 0 && frag_left > 0)
+			continue;
+		done = (nfrags == 0 && left == 0);
+		ionic_tx_tso_post(q, desc, skb,
+				  desc_addr, desc_nsge, desc_len,
+				  hdrlen, mss,
+				  outer_csum,
+				  vlan_tci, has_vlan,
+				  start, done);
+		total_pkts++;
+		total_bytes += start ? len : len + hdrlen;
+		desc = ionic_tx_tso_next(q, &elem);
+		start = false;
+		seglen = mss;
+	}
+
+	/* Chop skb frags into desc segments */
+
+	for (frag = skb_shinfo(skb)->frags; len_left; frag++) {
+		offset = 0;
+		left = skb_frag_size(frag);
+		len_left -= left;
+		nfrags--;
+		stats->frags++;
+
+		while (left > 0) {
+			if (frag_left > 0) {
+				len = min(frag_left, left);
+				frag_left -= len;
+				elem->addr =
+				    cpu_to_le64(ionic_tx_map_frag(q, frag,
+								  offset, len));
+				if (!elem->addr)
+					goto err_out_abort;
+				elem->len = cpu_to_le16(len);
+				elem++;
+				desc_nsge++;
+				left -= len;
+				offset += len;
+				if (nfrags > 0 && frag_left > 0)
+					continue;
+				done = (nfrags == 0 && left == 0);
+				ionic_tx_tso_post(q, desc, skb, desc_addr,
+						  desc_nsge, desc_len,
+						  hdrlen, mss, outer_csum,
+						  vlan_tci, has_vlan,
+						  start, done);
+				total_pkts++;
+				total_bytes += start ? len : len + hdrlen;
+				desc = ionic_tx_tso_next(q, &elem);
+				start = false;
+			} else {
+				len = min(mss, left);
+				frag_left = mss - len;
+				desc_addr = ionic_tx_map_frag(q, frag,
+							      offset, len);
+				if (!desc_addr)
+					goto err_out_abort;
+				desc_len = len;
+				desc_nsge = 0;
+				left -= len;
+				offset += len;
+				if (nfrags > 0 && frag_left > 0)
+					continue;
+				done = (nfrags == 0 && left == 0);
+				ionic_tx_tso_post(q, desc, skb, desc_addr,
+						  desc_nsge, desc_len,
+						  hdrlen, mss, outer_csum,
+						  vlan_tci, has_vlan,
+						  start, done);
+				total_pkts++;
+				total_bytes += start ? len : len + hdrlen;
+				desc = ionic_tx_tso_next(q, &elem);
+				start = false;
+			}
+		}
+	}
+
+	stats->pkts += total_pkts;
+	stats->bytes += total_bytes;
+	stats->tso++;
+
+	return 0;
+
+err_out_abort:
+	while (rewind->desc != q->head->desc) {
+		ionic_tx_clean(q, rewind, NULL, NULL);
+		rewind = rewind->next;
+	}
+	q->head = abort;
+
+	return -ENOMEM;
+}
+
+static int ionic_tx_calc_csum(struct ionic_queue *q, struct sk_buff *skb)
+{
+	struct ionic_tx_stats *stats = q_to_tx_stats(q);
+	struct ionic_txq_desc *desc = q->head->desc;
+	dma_addr_t addr;
+	bool has_vlan;
+	u8 flags = 0;
+	bool encap;
+	u64 cmd;
+
+	has_vlan = !!skb_vlan_tag_present(skb);
+	encap = skb->encapsulation;
+
+	addr = ionic_tx_map_single(q, skb->data, skb_headlen(skb));
+	if (!addr)
+		return -ENOMEM;
+
+	flags |= has_vlan ? IONIC_TXQ_DESC_FLAG_VLAN : 0;
+	flags |= encap ? IONIC_TXQ_DESC_FLAG_ENCAP : 0;
+
+	cmd = encode_txq_desc_cmd(IONIC_TXQ_DESC_OPCODE_CSUM_PARTIAL,
+				  flags, skb_shinfo(skb)->nr_frags, addr);
+	desc->cmd = cpu_to_le64(cmd);
+	desc->len = cpu_to_le16(skb_headlen(skb));
+	desc->vlan_tci = cpu_to_le16(skb_vlan_tag_get(skb));
+	desc->csum_start = cpu_to_le16(skb_checksum_start_offset(skb));
+	desc->csum_offset = cpu_to_le16(skb->csum_offset);
+
+	if (skb->csum_not_inet)
+		stats->crc32_csum++;
+	else
+		stats->csum++;
+
+	return 0;
+}
+
+static int ionic_tx_calc_no_csum(struct ionic_queue *q, struct sk_buff *skb)
+{
+	struct ionic_tx_stats *stats = q_to_tx_stats(q);
+	struct ionic_txq_desc *desc = q->head->desc;
+	dma_addr_t addr;
+	bool has_vlan;
+	u8 flags = 0;
+	bool encap;
+	u64 cmd;
+
+	has_vlan = !!skb_vlan_tag_present(skb);
+	encap = skb->encapsulation;
+
+	addr = ionic_tx_map_single(q, skb->data, skb_headlen(skb));
+	if (!addr)
+		return -ENOMEM;
+
+	flags |= has_vlan ? IONIC_TXQ_DESC_FLAG_VLAN : 0;
+	flags |= encap ? IONIC_TXQ_DESC_FLAG_ENCAP : 0;
+
+	cmd = encode_txq_desc_cmd(IONIC_TXQ_DESC_OPCODE_CSUM_NONE,
+				  flags, skb_shinfo(skb)->nr_frags, addr);
+	desc->cmd = cpu_to_le64(cmd);
+	desc->len = cpu_to_le16(skb_headlen(skb));
+	desc->vlan_tci = cpu_to_le16(skb_vlan_tag_get(skb));
+
+	stats->no_csum++;
+
+	return 0;
+}
+
+static int ionic_tx_skb_frags(struct ionic_queue *q, struct sk_buff *skb)
+{
+	struct ionic_txq_sg_desc *sg_desc = q->head->sg_desc;
+	unsigned int len_left = skb->len - skb_headlen(skb);
+	struct ionic_txq_sg_elem *elem = sg_desc->elems;
+	struct ionic_tx_stats *stats = q_to_tx_stats(q);
+	dma_addr_t dma_addr;
+	skb_frag_t *frag;
+	u16 len;
+
+	for (frag = skb_shinfo(skb)->frags; len_left; frag++, elem++) {
+		len = skb_frag_size(frag);
+		elem->len = cpu_to_le16(len);
+		dma_addr = ionic_tx_map_frag(q, frag, 0, len);
+		if (!dma_addr)
+			return -ENOMEM;
+		elem->addr = cpu_to_le64(dma_addr);
+		len_left -= len;
+		stats->frags++;
+	}
+
+	return 0;
+}
+
+static int ionic_tx(struct ionic_queue *q, struct sk_buff *skb)
+{
+	struct ionic_tx_stats *stats = q_to_tx_stats(q);
+	int err;
+
+	/* set up the initial descriptor */
+	if (skb->ip_summed == CHECKSUM_PARTIAL)
+		err = ionic_tx_calc_csum(q, skb);
+	else
+		err = ionic_tx_calc_no_csum(q, skb);
+	if (err)
+		return err;
+
+	/* add frags */
+	err = ionic_tx_skb_frags(q, skb);
+	if (err)
+		return err;
+
+	skb_tx_timestamp(skb);
+	stats->pkts++;
+	stats->bytes += skb->len;
+
+	netdev_tx_sent_queue(q_to_ndq(q), skb->len);
+	ionic_txq_post(q, !netdev_xmit_more(), ionic_tx_clean, skb);
+
+	return 0;
+}
+
+static int ionic_tx_descs_needed(struct ionic_queue *q, struct sk_buff *skb)
+{
+	struct ionic_tx_stats *stats = q_to_tx_stats(q);
+	int err;
+
+	/* If TSO, need roundup(skb->len/mss) descs */
+	if (skb_is_gso(skb))
+		return (skb->len / skb_shinfo(skb)->gso_size) + 1;
+
+	/* If non-TSO, just need 1 desc and nr_frags sg elems */
+	if (skb_shinfo(skb)->nr_frags <= IONIC_TX_MAX_SG_ELEMS)
+		return 1;
+
+	/* Too many frags, so linearize */
+	err = skb_linearize(skb);
+	if (err)
+		return err;
+
+	stats->linearize++;
+
+	/* Need 1 desc and zero sg elems */
+	return 1;
+}
+
+netdev_tx_t ionic_start_xmit(struct sk_buff *skb, struct net_device *netdev)
+{
+	u16 queue_index = skb_get_queue_mapping(skb);
+	struct ionic_lif *lif = netdev_priv(netdev);
+	struct ionic_queue *q;
+	int ndescs;
+	int err;
+
+	if (unlikely(!test_bit(IONIC_LIF_UP, lif->state))) {
+		dev_kfree_skb(skb);
+		return NETDEV_TX_OK;
+	}
+
+	if (likely(lif_to_txqcq(lif, queue_index)))
+		q = lif_to_txq(lif, queue_index);
+	else
+		q = lif_to_txq(lif, 0);
+
+	ndescs = ionic_tx_descs_needed(q, skb);
+	if (ndescs < 0)
+		goto err_out_drop;
+
+	if (!ionic_q_has_space(q, ndescs)) {
+		netif_stop_subqueue(netdev, queue_index);
+		q->stop++;
+
+		/* Might race with ionic_tx_clean, check again */
+		smp_rmb();
+		if (ionic_q_has_space(q, ndescs)) {
+			netif_wake_subqueue(netdev, queue_index);
+			q->wake++;
+		} else {
+			return NETDEV_TX_BUSY;
+		}
+	}
+
+	if (skb_is_gso(skb))
+		err = ionic_tx_tso(q, skb);
+	else
+		err = ionic_tx(q, skb);
+
+	if (err)
+		goto err_out_drop;
+
+	return NETDEV_TX_OK;
+
+err_out_drop:
+	netif_stop_subqueue(netdev, queue_index);
+	q->stop++;
+	q->drop++;
+	dev_kfree_skb(skb);
+	return NETDEV_TX_OK;
+}
drivers/net/ethernet/pensando/ionic/ionic_txrx.h new file +15 -0
new file mode 100644
index 000000000000..53775c62c85a
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+
+#ifndef _IONIC_TXRX_H_
+#define _IONIC_TXRX_H_
+
+void ionic_rx_flush(struct ionic_cq *cq);
+void ionic_tx_flush(struct ionic_cq *cq);
+
+void ionic_rx_fill(struct ionic_queue *q);
+void ionic_rx_empty(struct ionic_queue *q);
+int ionic_rx_napi(struct napi_struct *napi, int budget);
+netdev_tx_t ionic_start_xmit(struct sk_buff *skb, struct net_device *netdev);
+
+#endif /* _IONIC_TXRX_H_ */
#32 lore raw

Re: [PATCH v5 net-next 14/18] ionic: Add Tx and Rx handling

Quoted 9 lines · Yunsheng Lin
Quoted 8 lines · Shannon Nelson

Add both the Tx and Rx queue setup and handling. The related stats display comes later. Instead of using the generic napi routines used by the slow-path commands, the Tx and Rx paths are simplified and inlined in one file in order to get better compiler optimizations.

---

[...]

Quoted 30 lines
Quoted 16 lines
drivers/net/ethernet/pensando/ionic/ionic_lif.c +16 -0
+static int ionic_txrx_init(struct ionic_lif *lif)
+{
+	unsigned int i;
+	int err;
+
+	for (i = 0; i < lif->nxqs; i++) {
+		err = ionic_lif_txq_init(lif, lif->txqcqs[i].qcq);
+		if (err)
+			goto err_out;
+
+		err = ionic_lif_rxq_init(lif, lif->rxqcqs[i].qcq);
+		if (err) {
+			ionic_lif_qcq_deinit(lif, lif->txqcqs[i-1].qcq);
+			goto err_out;
+		}
+	}

drivers/net/ethernet/pensando/ionic/ionic_lif.c static int ionic_vlan_rx_kill_vid(struct net_device *netdev, __be16 proto, · line 1423

Quoted 10 lines
drivers/net/ethernet/pensando/ionic/ionic_lif.c +10 -0
+
+	ionic_set_rx_mode(lif->netdev);
+
+	return 0;
+
+err_out:
+	for (i--; i > 0; i--) {
+		ionic_lif_qcq_deinit(lif, lif->txqcqs[i-1].qcq);
+		ionic_lif_qcq_deinit(lif, lif->rxqcqs[i-1].qcq);
+	}

The "i--" has been done in for initialization, and ionic_lif_qcq_deinit is called with lif->rxqcqs[i-1], which may cause the last lif->txqcqs or lif->rxqcqs not initialized problem.

Quoted 5 lines

It may be more common to do the below:

while (i--) {
	ionic_lif_qcq_deinit(lif, lif->txqcqs[i].qcq);
	ionic_lif_qcq_deinit(lif, lif->rxqcqs[i].qcq);
}

Sure.

Quoted 21 lines
Quoted 21 lines
drivers/net/ethernet/pensando/ionic/ionic_lif.c +21 -0
+
+	return err;
+}
+
+static int ionic_txrx_enable(struct ionic_lif *lif)
+{
+	int i, err;
+
+	for (i = 0; i < lif->nxqs; i++) {
+		err = ionic_qcq_enable(lif->txqcqs[i].qcq);
+		if (err)
+			goto err_out;
+
+		ionic_rx_fill(&lif->rxqcqs[i].qcq->q);
+		err = ionic_qcq_enable(lif->rxqcqs[i].qcq);
+		if (err) {
+			ionic_qcq_disable(lif->txqcqs[i].qcq);
+			goto err_out;
+		}
+	}
+
Quoted 8 lines

drivers/net/ethernet/pensando/ionic/ionic_lif.c static int ionic_vlan_rx_kill_vid(struct net_device *netdev, __be16 proto, · line 1451

Quoted 7 lines
drivers/net/ethernet/pensando/ionic/ionic_lif.c +7 -0
+	return 0;
+
+err_out:
+	for (i--; i >= 0 ; i--) {
+		ionic_qcq_disable(lif->rxqcqs[i].qcq);
+		ionic_qcq_disable(lif->txqcqs[i].qcq);
+	}

It may be better to use the above pattern too.

Okay

Quoted 6 lines
Quoted 6 lines
drivers/net/ethernet/pensando/ionic/ionic_txrx.c new file +6 -0
+static dma_addr_t ionic_tx_map_single(struct ionic_queue *q, void *data, size_t len)
+{
+	struct ionic_tx_stats *stats = q_to_tx_stats(q);
+	struct device *dev = q->lif->ionic->dev;
+	dma_addr_t dma_addr;
+
Quoted 8 lines

drivers/net/ethernet/pensando/ionic/ionic_txrx.c line 350

Quoted 6 lines
drivers/net/ethernet/pensando/ionic/ionic_txrx.c new file +6 -0
+	dma_addr = dma_map_single(dev, data, len, DMA_TO_DEVICE);
+	if (dma_mapping_error(dev, dma_addr)) {
+		net_warn_ratelimited("%s: DMA single map failed on %s!\n",
+				     q->lif->netdev->name, q->name);
+		stats->dma_map_err++;
+		return 0;

zero may be a valid dma address, maybe check the dma_mapping_error in ionic_tx_tso instead.

Hmmm, hadn't thought of 0 as a valid address...
I'll need to make a similar adjustment to ionic_tx_map_frag() uses.

Quoted 7 lines
drivers/net/ethernet/pensando/ionic/ionic_txrx.c new file +4 -0
+
+static void ionic_tx_tcp_inner_pseudo_csum(struct sk_buff *skb)
+{
+	skb_cow_head(skb, 0);

May need to check for return error of skb_cow_head.

Sure, and in both places.

Thanks,
sln

#33 lore raw

PATCH v5 15/18 ionic: Add netdev-event handling

In reply to Shannon Nelson

When the netdev gets a new name from userland, pass that name down to the NIC for internal tracking.

---
 drivers/net/ethernet/pensando/ionic/ionic.h   |  2 +
 .../net/ethernet/pensando/ionic/ionic_lif.c   | 62 +++++++++++++++++++
 2 files changed, 64 insertions(+)
drivers/net/ethernet/pensando/ionic/ionic.h +2 -0
index 82b45e3384c3..d46103cb3a13 100644
@@ -45,6 +45,8 @@ struct ionic {
 	DECLARE_BITMAP(lifbits, IONIC_LIFS_MAX);
 	unsigned int nintrs;
 	DECLARE_BITMAP(intrs, IONIC_INTR_CTRL_REGS_MAX);
+	struct work_struct nb_work;
+	struct notifier_block nb;
 };
 
 struct ionic_admin_ctx {
drivers/net/ethernet/pensando/ionic/ionic_lif.c +62 -0

78 lines not on this page · raw message

Signature · 1 line
2.17.1
#34 lore raw

PATCH v5 16/18 ionic: Add driver stats

In reply to Shannon Nelson

Add in the detailed statistics for ethtool -S that the driver keeps as it processes packets. Display of the additional debug statistics can be enabled through the ethtool priv-flags feature.

---
 drivers/net/ethernet/pensando/ionic/Makefile  |   2 +-
 .../ethernet/pensando/ionic/ionic_ethtool.c   | 101 ++++++
 .../net/ethernet/pensando/ionic/ionic_lif.h   |  16 +-
 .../net/ethernet/pensando/ionic/ionic_stats.c | 334 ++++++++++++++++++
 .../net/ethernet/pensando/ionic/ionic_stats.h |  53 +++
 5 files changed, 504 insertions(+), 2 deletions(-)
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_stats.c
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_stats.h
drivers/net/ethernet/pensando/ionic/Makefile +1 -1
index 1cf8117db443..29f304d75261 100644
@@ -5,4 +5,4 @@ obj-$(CONFIG_IONIC) := ionic.o
 
 ionic-y := ionic_main.o ionic_bus_pci.o ionic_devlink.o ionic_dev.o \
 	   ionic_debugfs.o ionic_lif.o ionic_rx_filter.o ionic_ethtool.o \
-	   ionic_txrx.o
+	   ionic_txrx.o ionic_stats.o
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c +101 -0

119 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_lif.h +15 -1
index d2fae13def44..5b7e7127cacb 100644
@@ -12,7 +12,7 @@
 
 #define IONIC_MAX_NUM_NAPI_CNTR		(NAPI_POLL_WEIGHT + 1)
 #define IONIC_MAX_NUM_SG_CNTR		(IONIC_TX_MAX_SG_ELEMS + 1)
-#define IONIC_RX_COPYBREAK_DEFAULT		256
+#define IONIC_RX_COPYBREAK_DEFAULT	256
 
 struct ionic_tx_stats {
 	u64 dma_map_err;
@@ -106,8 +106,22 @@ struct ionic_deferred {
 	struct work_struct work;
 };
 
+struct ionic_lif_sw_stats {
+	u64 tx_packets;
+	u64 tx_bytes;
+	u64 rx_packets;
+	u64 rx_bytes;
+	u64 tx_tso;
+	u64 tx_no_csum;
+	u64 tx_csum;
+	u64 rx_csum_none;
+	u64 rx_csum_complete;
+	u64 rx_csum_error;
+};
+
 enum ionic_lif_state_flags {
 	IONIC_LIF_INITED,
+	IONIC_LIF_SW_DEBUG_STATS,
 	IONIC_LIF_UP,
 	IONIC_LIF_LINK_CHECK_REQUESTED,
 	IONIC_LIF_QUEUE_RESET,
drivers/net/ethernet/pensando/ionic/ionic_stats.c new file +334 -0

334 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_stats.h new file +53 -0

53 lines not on this page · raw message

Signature · 1 line
2.17.1
#35 lore raw

PATCH v5 17/18 ionic: Add RSS support

In reply to Shannon Nelson

Add code to manipulate through ethtool the RSS configuration used by the NIC.

---
 .../ethernet/pensando/ionic/ionic_ethtool.c   | 73 ++++++++++++++++
 .../net/ethernet/pensando/ionic/ionic_lif.c   | 84 +++++++++++++++++++
 .../net/ethernet/pensando/ionic/ionic_lif.h   |  8 ++
 3 files changed, 165 insertions(+)
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c +73 -0
index c6a056efea0e..ce32d5e96903 100644
@@ -486,6 +486,74 @@ static int ionic_set_priv_flags(struct net_device *netdev, u32 priv_flags)
 	return 0;
 }
 
+static int ionic_get_rxnfc(struct net_device *netdev,
+			   struct ethtool_rxnfc *info, u32 *rules)
+{
+	struct ionic_lif *lif = netdev_priv(netdev);
+	int err = 0;
+
+	switch (info->cmd) {
+	case ETHTOOL_GRXRINGS:
+		info->data = lif->nxqs;
+		break;
+	default:
+		netdev_err(netdev, "Command parameter %d is not supported\n",
+			   info->cmd);
+		err = -EOPNOTSUPP;
+	}
+
+	return err;
+}
+
+static u32 ionic_get_rxfh_indir_size(struct net_device *netdev)
+{
+	struct ionic_lif *lif = netdev_priv(netdev);
+
+	return le16_to_cpu(lif->ionic->ident.lif.eth.rss_ind_tbl_sz);
+}
+
+static u32 ionic_get_rxfh_key_size(struct net_device *netdev)
+{
+	return IONIC_RSS_HASH_KEY_SIZE;
+}
+
+static int ionic_get_rxfh(struct net_device *netdev, u32 *indir, u8 *key,
+			  u8 *hfunc)
+{
+	struct ionic_lif *lif = netdev_priv(netdev);
+	unsigned int i, tbl_sz;
+
+	if (indir) {
+		tbl_sz = le16_to_cpu(lif->ionic->ident.lif.eth.rss_ind_tbl_sz);
+		for (i = 0; i < tbl_sz; i++)
+			indir[i] = lif->rss_ind_tbl[i];
+	}
+
+	if (key)
+		memcpy(key, lif->rss_hash_key, IONIC_RSS_HASH_KEY_SIZE);
+
+	if (hfunc)
+		*hfunc = ETH_RSS_HASH_TOP;
+
+	return 0;
+}
+
+static int ionic_set_rxfh(struct net_device *netdev, const u32 *indir,
+			  const u8 *key, const u8 hfunc)
+{
+	struct ionic_lif *lif = netdev_priv(netdev);
+	int err;
+
+	if (hfunc != ETH_RSS_HASH_NO_CHANGE && hfunc != ETH_RSS_HASH_TOP)
+		return -EOPNOTSUPP;
+
+	err = ionic_lif_rss_config(lif, lif->rss_types, key, indir);
+	if (err)
+		return err;
+
+	return 0;
+}
+
 static int ionic_get_module_info(struct net_device *netdev,
 				 struct ethtool_modinfo *modinfo)
 
@@ -585,6 +653,11 @@ static const struct ethtool_ops ionic_ethtool_ops = {
 	.get_sset_count		= ionic_get_sset_count,
 	.get_priv_flags		= ionic_get_priv_flags,
 	.set_priv_flags		= ionic_set_priv_flags,
+	.get_rxnfc		= ionic_get_rxnfc,
+	.get_rxfh_indir_size	= ionic_get_rxfh_indir_size,
+	.get_rxfh_key_size	= ionic_get_rxfh_key_size,
+	.get_rxfh		= ionic_get_rxfh,
+	.set_rxfh		= ionic_set_rxfh,
 	.get_module_info	= ionic_get_module_info,
 	.get_module_eeprom	= ionic_get_module_eeprom,
 	.get_pauseparam		= ionic_get_pauseparam,
drivers/net/ethernet/pensando/ionic/ionic_lif.c +84 -0

124 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_lif.h +8 -0
index 5b7e7127cacb..402d6a8238a1 100644
@@ -164,6 +164,12 @@ struct ionic_lif {
 	dma_addr_t info_pa;
 	u32 info_sz;
 
+	u16 rss_types;
+	u8 rss_hash_key[IONIC_RSS_HASH_KEY_SIZE];
+	u8 *rss_ind_tbl;
+	dma_addr_t rss_ind_tbl_pa;
+	u32 rss_ind_tbl_sz;
+
 	struct ionic_rx_filters rx_filters;
 	struct ionic_deferred deferred;
 	unsigned long *dbid_inuse;
@@ -223,6 +229,8 @@ void ionic_lifs_unregister(struct ionic *ionic);
 int ionic_lif_identify(struct ionic *ionic, u8 lif_type,
 		       union ionic_lif_identity *lif_ident);
 int ionic_lifs_size(struct ionic *ionic);
+int ionic_lif_rss_config(struct ionic_lif *lif, u16 types,
+			 const u8 *key, const u32 *indir);
 
 int ionic_open(struct net_device *netdev);
 int ionic_stop(struct net_device *netdev);
Signature · 1 line
2.17.1
#36 lore raw

PATCH v5 18/18 ionic: Add coalesce and other features

In reply to Shannon Nelson

Interrupt coalescing, tunable copybreak value, and tx timeout.

---
 .../net/ethernet/pensando/ionic/ionic_dev.h   |   1 +
 .../ethernet/pensando/ionic/ionic_ethtool.c   | 108 ++++++++++++++++++
 .../net/ethernet/pensando/ionic/ionic_lif.c   |  28 ++++-
 .../net/ethernet/pensando/ionic/ionic_lif.h   |  30 +++++
 .../net/ethernet/pensando/ionic/ionic_txrx.c  |   2 -
 5 files changed, 166 insertions(+), 3 deletions(-)
drivers/net/ethernet/pensando/ionic/ionic_dev.h +1 -0
index 1db1e433127a..7538edfcbb78 100644
@@ -16,6 +16,7 @@
 #define IONIC_MIN_TXRX_DESC		16
 #define IONIC_DEF_TXRX_DESC		4096
 #define IONIC_LIFS_MAX			1024
+#define IONIC_ITR_COAL_USEC_DEFAULT	64
 
 #define IONIC_DEV_CMD_REG_VERSION	1
 #define IONIC_DEV_INFO_REG_COUNT	32
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c +108 -0

132 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_lif.c +27 -1
index 48f254ab7d46..1ca95576af8d 100644
@@ -3,6 +3,7 @@
 
 #include <linux/netdevice.h>
 #include <linux/etherdevice.h>
+#include <linux/rtnetlink.h>
 #include <linux/interrupt.h>
 #include <linux/pci.h>
 #include <linux/cpumask.h>
@@ -1253,9 +1254,22 @@ static int ionic_change_mtu(struct net_device *netdev, int new_mtu)
 	return err;
 }
 
+static void ionic_tx_timeout_work(struct work_struct *ws)
+{
+	struct ionic_lif *lif = container_of(ws, struct ionic_lif, tx_timeout_work);
+
+	netdev_info(lif->netdev, "Tx Timeout recovery\n");
+
+	rtnl_lock();
+	ionic_reset_queues(lif);
+	rtnl_unlock();
+}
+
 static void ionic_tx_timeout(struct net_device *netdev)
 {
-	netdev_info(netdev, "%s: stubbed\n", __func__);
+	struct ionic_lif *lif = netdev_priv(netdev);
+
+	schedule_work(&lif->tx_timeout_work);
 }
 
 static int ionic_vlan_rx_add_vid(struct net_device *netdev, __be16 proto,
@@ -1415,6 +1429,7 @@ static int ionic_txrx_alloc(struct ionic_lif *lif)
 	unsigned int flags;
 	unsigned int i;
 	int err = 0;
+	u32 coal;
 
 	flags = IONIC_QCQ_F_TX_STATS | IONIC_QCQ_F_SG;
 	for (i = 0; i < lif->nxqs; i++) {
@@ -1431,6 +1446,7 @@ static int ionic_txrx_alloc(struct ionic_lif *lif)
 	}
 
 	flags = IONIC_QCQ_F_RX_STATS | IONIC_QCQ_F_INTR;
+	coal = ionic_coal_usec_to_hw(lif->ionic, lif->rx_coalesce_usecs);
 	for (i = 0; i < lif->nxqs; i++) {
 		err = ionic_qcq_alloc(lif, IONIC_QTYPE_RXQ, i, "rx", flags,
 				      lif->nrxq_descs,
@@ -1442,6 +1458,8 @@ static int ionic_txrx_alloc(struct ionic_lif *lif)
 
 		lif->rxqcqs[i].qcq->stats = lif->rxqcqs[i].stats;
 
+		ionic_intr_coal_init(lif->ionic->idev.intr_ctrl,
+				     lif->rxqcqs[i].qcq->intr.index, coal);
 		ionic_link_qcq_interrupts(lif->rxqcqs[i].qcq,
 					  lif->txqcqs[i].qcq);
 	}
@@ -1620,6 +1638,7 @@ static struct ionic_lif *ionic_lif_alloc(struct ionic *ionic, unsigned int index
 	struct net_device *netdev;
 	struct ionic_lif *lif;
 	int tbl_sz;
+	u32 coal;
 	int err;
 
 	netdev = alloc_etherdev_mqs(sizeof(*lif),
@@ -1649,6 +1668,10 @@ static struct ionic_lif *ionic_lif_alloc(struct ionic *ionic, unsigned int index
 	lif->ntxq_descs = IONIC_DEF_TXRX_DESC;
 	lif->nrxq_descs = IONIC_DEF_TXRX_DESC;
 
+	/* Convert the default coalesce value to actual hw resolution */
+	coal = ionic_coal_usec_to_hw(lif->ionic, IONIC_ITR_COAL_USEC_DEFAULT);
+	lif->rx_coalesce_usecs = ionic_coal_hw_to_usec(lif->ionic, coal);
+
 	snprintf(lif->name, sizeof(lif->name), "lif%u", index);
 
 	spin_lock_init(&lif->adminq_lock);
@@ -1999,6 +2022,8 @@ static int ionic_lif_init(struct ionic_lif *lif)
 
 	set_bit(IONIC_LIF_INITED, lif->state);
 
+	INIT_WORK(&lif->tx_timeout_work, ionic_tx_timeout_work);
+
 	return 0;
 
 err_out_notifyq_deinit:
@@ -2117,6 +2142,7 @@ void ionic_lifs_unregister(struct ionic *ionic)
 	 * ionic->lif for candidates to unregister
 	 */
 	cancel_work_sync(&ionic->master_lif->deferred.work);
+	cancel_work_sync(&ionic->master_lif->tx_timeout_work);
 	if (ionic->master_lif->netdev->reg_state == NETREG_REGISTERED)
 		unregister_netdev(ionic->master_lif->netdev);
 }
drivers/net/ethernet/pensando/ionic/ionic_lif.h +30 -0
index 402d6a8238a1..1f847e15637d 100644
@@ -177,6 +177,7 @@ struct ionic_lif {
 	struct dentry *dentry;
 	u32 rx_coalesce_usecs;
 	u32 flags;
+	struct work_struct tx_timeout_work;
 };
 
 #define lif_to_txqcq(lif, i)	((lif)->txqcqs[i].qcq)
@@ -220,6 +221,35 @@ static inline bool ionic_is_100g(struct ionic *ionic)
 		ionic->pdev->subsystem_device == IONIC_SUBDEV_ID_NAPLES_100_8);
 }
 
+static inline u32 ionic_coal_usec_to_hw(struct ionic *ionic, u32 usecs)
+{
+	u32 mult = le32_to_cpu(ionic->ident.dev.intr_coal_mult);
+	u32 div = le32_to_cpu(ionic->ident.dev.intr_coal_div);
+
+	/* Div-by-zero should never be an issue, but check anyway */
+	if (!div || !mult)
+		return 0;
+
+	/* Round up in case usecs is close to the next hw unit */
+	usecs += (div / mult) >> 1;
+
+	/* Convert from usecs to device units */
+	return (usecs * mult) / div;
+}
+
+static inline u32 ionic_coal_hw_to_usec(struct ionic *ionic, u32 units)
+{
+	u32 mult = le32_to_cpu(ionic->ident.dev.intr_coal_mult);
+	u32 div = le32_to_cpu(ionic->ident.dev.intr_coal_div);
+
+	/* Div-by-zero should never be an issue, but check anyway */
+	if (!div || !mult)
+		return 0;
+
+	/* Convert from device units to usec */
+	return (units * div) / mult;
+}
+
 int ionic_lifs_alloc(struct ionic *ionic);
 void ionic_lifs_free(struct ionic *ionic);
 void ionic_lifs_deinit(struct ionic *ionic);
drivers/net/ethernet/pensando/ionic/ionic_txrx.c +0 -2
index 3ec28f021d28..751809e0f534 100644
@@ -289,11 +289,9 @@ void ionic_rx_empty(struct ionic_queue *q)
 
 	for (cur = q->tail; cur != q->head; cur = cur->next) {
 		desc = cur->desc;
-
 		dma_unmap_single(dev, le64_to_cpu(desc->addr),
 				 le16_to_cpu(desc->len), DMA_FROM_DEVICE);
 		dev_kfree_skb(cur->cb_arg);
-
 		cur->cb_arg = NULL;
 	}
 }
Signature · 1 line
2.17.1