Skip to contentSkip to conversation

Add ionic driver

PATCH Superseded Shannon Nelson 6 participants 56 messages Jul 8, 2019 – Jul 19, 2019

Subsystems (4)

v3 of 7 Superseded Latest is v7 →

lore
Share & tools
lore
#0 lore raw

v3 cover Add ionic driver

This is a patch series that adds the ionic driver, supporting the Pensando ethernet device.

In this initial patchset we implement basic transmit and receive. Later patchsets will add more advanced features.

Our thanks to Andrew Lunn, Michal Kubecek, Jacub Kicinski, and the ever present kbuild test robots for their comments and suggestions.

New in v3:
- use le32_to_cpu() on queue_count[] values in debugfs
- dma_free_coherent() can handle NULL pointers
- remove unused SS_TEST from ethtool handlers
- one more case of stop the tx ring if there is no room
- remove a couple of stray // comments

New in v2:
- removed debugfs error checking and cut down on debugfs use
- remove redundant bounds checking on incoming values for mtu and ethtool
- don't alloc rx_filter memory until the match type has been checked
- free the ionic struct on remove
- simplified link_up and netif_carrier_ok comparison
- put stats into ethtool -S, out of debugfs
- moved dev_cmd and dev_info dumping to ethtool -d, out of debugfs
- added devlink support
- used kernel's rss init routines rather than open code
- set the Kbuild dependant on 64BIT
- cut down on some unnecessary log messaging
- cleaned up ionic_get_link_ksettings
- cleaned up other little code bits here and there

Shannon Nelson (19):
ionic: Add basic framework for IONIC Network device driver
ionic: Add hardware init and device commands
ionic: Add port management commands
ionic: Add basic lif support
ionic: Add interrupts and doorbells
ionic: Add basic adminq support
ionic: Add adminq action
ionic: Add notifyq support
ionic: Add the basic NDO callbacks for netdev support
ionic: Add management of rx filters
ionic: Add Rx filter and rx_mode ndo support
ionic: Add async link status check and basic stats
ionic: Add initial ethtool support
ionic: Add Tx and Rx handling
ionic: Add netdev-event handling
ionic: Add driver stats
ionic: Add RSS support
ionic: Add coalesce and other features
ionic: Add basic devlink interface

 .../networking/device_drivers/index.rst       |    1 +
 .../device_drivers/pensando/ionic.rst         |   64 +
 MAINTAINERS                                   |    8 +
 drivers/net/ethernet/Kconfig                  |    1 +
 drivers/net/ethernet/Makefile                 |    1 +
 drivers/net/ethernet/pensando/Kconfig         |   32 +
 drivers/net/ethernet/pensando/Makefile        |    6 +
 drivers/net/ethernet/pensando/ionic/Makefile  |    8 +
 drivers/net/ethernet/pensando/ionic/ionic.h   |   72 +
 .../net/ethernet/pensando/ionic/ionic_bus.h   |   16 +
 .../ethernet/pensando/ionic/ionic_bus_pci.c   |  291 ++
 .../ethernet/pensando/ionic/ionic_debugfs.c   |  283 ++
 .../ethernet/pensando/ionic/ionic_debugfs.h   |   38 +
 .../net/ethernet/pensando/ionic/ionic_dev.c   |  535 ++++
 .../net/ethernet/pensando/ionic/ionic_dev.h   |  284 ++
 .../ethernet/pensando/ionic/ionic_devlink.c   |   89 +
 .../ethernet/pensando/ionic/ionic_devlink.h   |   12 +
 .../ethernet/pensando/ionic/ionic_ethtool.c   |  788 +++++
 .../ethernet/pensando/ionic/ionic_ethtool.h   |    9 +
 .../net/ethernet/pensando/ionic/ionic_if.h    | 2552 +++++++++++++++++
 .../net/ethernet/pensando/ionic/ionic_lif.c   | 2262 +++++++++++++++
 .../net/ethernet/pensando/ionic/ionic_lif.h   |  270 ++
 .../net/ethernet/pensando/ionic/ionic_main.c  |  553 ++++
 .../net/ethernet/pensando/ionic/ionic_regs.h  |  133 +
 .../ethernet/pensando/ionic/ionic_rx_filter.c |  142 +
 .../ethernet/pensando/ionic/ionic_rx_filter.h |   34 +
 .../net/ethernet/pensando/ionic/ionic_stats.c |  333 +++
 .../net/ethernet/pensando/ionic/ionic_stats.h |   53 +
 .../net/ethernet/pensando/ionic/ionic_txrx.c  |  879 ++++++
 .../net/ethernet/pensando/ionic/ionic_txrx.h  |   15 +
 30 files changed, 9764 insertions(+)
 create mode 100644 Documentation/networking/device_drivers/pensando/ionic.rst
 create mode 100644 drivers/net/ethernet/pensando/Kconfig
 create mode 100644 drivers/net/ethernet/pensando/Makefile
 create mode 100644 drivers/net/ethernet/pensando/ionic/Makefile
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic.h
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_bus.h
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_debugfs.c
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_debugfs.h
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_dev.c
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_dev.h
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_devlink.c
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_devlink.h
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_ethtool.c
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_ethtool.h
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_if.h
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_lif.c
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_lif.h
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_main.c
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_regs.h
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_rx_filter.c
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_rx_filter.h
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_stats.c
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_stats.h
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_txrx.c
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_txrx.h
Signature · 1 line
2.17.1
#1 lore raw

PATCH v3 1/19 ionic: Add basic framework for IONIC Network device driver

This patch adds a basic driver framework for the Pensando IONIC network device. There is no functionality right now other than the ability to load and unload.

---
 .../networking/device_drivers/index.rst       |  1 +
 .../device_drivers/pensando/ionic.rst         | 64 +++++++++++++++++++
 MAINTAINERS                                   |  8 +++
 drivers/net/ethernet/Kconfig                  |  1 +
 drivers/net/ethernet/Makefile                 |  1 +
 drivers/net/ethernet/pensando/Kconfig         | 32 ++++++++++
 drivers/net/ethernet/pensando/Makefile        |  6 ++
 drivers/net/ethernet/pensando/ionic/Makefile  |  6 ++
 drivers/net/ethernet/pensando/ionic/ionic.h   | 25 ++++++++
 .../net/ethernet/pensando/ionic/ionic_bus.h   | 10 +++
 .../ethernet/pensando/ionic/ionic_bus_pci.c   | 58 +++++++++++++++++
 .../net/ethernet/pensando/ionic/ionic_main.c  | 29 +++++++++
 12 files changed, 241 insertions(+)
 create mode 100644 Documentation/networking/device_drivers/pensando/ionic.rst
 create mode 100644 drivers/net/ethernet/pensando/Kconfig
 create mode 100644 drivers/net/ethernet/pensando/Makefile
 create mode 100644 drivers/net/ethernet/pensando/ionic/Makefile
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic.h
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_bus.h
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_main.c
Documentation/networking/device_drivers/index.rst +1 -0
index 2b7fefe72351..57fec66c5419 100644
@@ -23,6 +23,7 @@ Contents:
    intel/ice
    google/gve
    mellanox/mlx5
+   pensando/ionic
 
 .. only::  subproject
 
Documentation/networking/device_drivers/pensando/ionic.rst new file +64 -0

64 lines not on this page · raw message

MAINTAINERS +8 -0
index 449e7cdb3303..0217242e812b 100644
@@ -12364,6 +12364,14 @@ L:	[email protected]
 S:	Maintained
 F:	drivers/platform/x86/peaq-wmi.c
 
+PENSANDO ETHERNET DRIVERS
+M:	Shannon Nelson <[email protected]>
+M:	Pensando Drivers <[email protected]>
+L:	[email protected]
+S:	Supported
+F:	Documentation/networking/device_drivers/pensando/ionic.rst
+F:	drivers/net/ethernet/pensando/
+
 PER-CPU MEMORY ALLOCATOR
 M:	Dennis Zhou <[email protected]>
 M:	Tejun Heo <[email protected]>
drivers/net/ethernet/Kconfig +1 -0
index 93a2d4deb27c..2830dc283ce6 100644
@@ -168,6 +168,7 @@ config ETHOC
 
 source "drivers/net/ethernet/packetengines/Kconfig"
 source "drivers/net/ethernet/pasemi/Kconfig"
+source "drivers/net/ethernet/pensando/Kconfig"
 source "drivers/net/ethernet/qlogic/Kconfig"
 source "drivers/net/ethernet/qualcomm/Kconfig"
 source "drivers/net/ethernet/rdc/Kconfig"
drivers/net/ethernet/Makefile +1 -0
index fb9155cffcff..061edd22f507 100644
@@ -97,3 +97,4 @@ obj-$(CONFIG_NET_VENDOR_WIZNET) += wiznet/
 obj-$(CONFIG_NET_VENDOR_XILINX) += xilinx/
 obj-$(CONFIG_NET_VENDOR_XIRCOM) += xircom/
 obj-$(CONFIG_NET_VENDOR_SYNOPSYS) += synopsys/
+obj-$(CONFIG_NET_VENDOR_PENSANDO) += pensando/
drivers/net/ethernet/pensando/Kconfig new file +32 -0
new file mode 100644
index 000000000000..5ea570be8379
@@ -0,0 +1,32 @@
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2019 Pensando Systems, Inc
+#
+# Pensando device configuration
+#
+
+config NET_VENDOR_PENSANDO
+	bool "Pensando devices"
+	default y
+	help
+	  If you have a network (Ethernet) card belonging to this class, say Y.
+
+	  Note that the answer to this question doesn't directly affect the
+	  kernel: saying N will just cause the configurator to skip all
+	  the questions about Pensando cards. If you say Y, you will be asked
+	  for your specific card in the following questions.
+
+if NET_VENDOR_PENSANDO
+
+config IONIC
+	tristate "Pensando Ethernet IONIC Support"
+	depends on 64BIT && PCI
+	help
+	  This enables the support for the Pensando family of Ethernet
+	  adapters.  More specific information on this driver can be
+	  found in
+	  <file:Documentation/networking/device_drivers/pensando/ionic.rst>.
+
+          To compile this driver as a module, choose M here. The module
+          will be called ionic.
+
+endif # NET_VENDOR_PENSANDO
drivers/net/ethernet/pensando/Makefile new file +6 -0
new file mode 100644
index 000000000000..21ce7499c122
@@ -0,0 +1,6 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Makefile for the Pensando network device drivers.
+#
+
+obj-$(CONFIG_IONIC) += ionic/
drivers/net/ethernet/pensando/ionic/Makefile new file +6 -0
new file mode 100644
index 000000000000..beb3faeccac1
@@ -0,0 +1,6 @@
+# SPDX-License-Identifier: GPL-2.0
+# Copyright(c) 2017 - 2019 Pensando Systems, Inc
+
+obj-$(CONFIG_IONIC) := ionic.o
+
+ionic-y := ionic_main.o ionic_bus_pci.o
drivers/net/ethernet/pensando/ionic/ionic.h new file +25 -0
new file mode 100644
index 000000000000..18c79adef06c
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+
+#ifndef _IONIC_H_
+#define _IONIC_H_
+
+#define DRV_NAME		"ionic"
+#define DRV_DESCRIPTION		"Pensando Ethernet NIC Driver"
+#define DRV_VERSION		"0.11.0-k"
+
+#define PCI_VENDOR_ID_PENSANDO			0x1dd8
+
+#define PCI_DEVICE_ID_PENSANDO_IONIC_ETH_PF	0x1002
+#define PCI_DEVICE_ID_PENSANDO_IONIC_ETH_VF	0x1003
+
+#define IONIC_SUBDEV_ID_NAPLES_25	0x4000
+#define IONIC_SUBDEV_ID_NAPLES_100_4	0x4001
+#define IONIC_SUBDEV_ID_NAPLES_100_8	0x4002
+
+struct ionic {
+	struct pci_dev *pdev;
+	struct device *dev;
+};
+
+#endif /* _IONIC_H_ */
drivers/net/ethernet/pensando/ionic/ionic_bus.h new file +10 -0
new file mode 100644
index 000000000000..94ba0afc6f38
@@ -0,0 +1,10 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+
+#ifndef _IONIC_BUS_H_
+#define _IONIC_BUS_H_
+
+int ionic_bus_register_driver(void);
+void ionic_bus_unregister_driver(void);
+
+#endif /* _IONIC_BUS_H_ */
drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c new file +58 -0

58 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_main.c new file +29 -0
new file mode 100644
index 000000000000..fb49c0ca5095
@@ -0,0 +1,29 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+
+#include <linux/module.h>
+#include <linux/version.h>
+#include <linux/netdevice.h>
+#include <linux/utsname.h>
+
+#include "ionic.h"
+#include "ionic_bus.h"
+
+MODULE_DESCRIPTION(DRV_DESCRIPTION);
+MODULE_AUTHOR("Pensando Systems, Inc");
+MODULE_LICENSE("GPL");
+MODULE_VERSION(DRV_VERSION);
+
+static int __init ionic_init_module(void)
+{
+	pr_info("%s %s, ver %s\n", DRV_NAME, DRV_DESCRIPTION, DRV_VERSION);
+	return ionic_bus_register_driver();
+}
+
+static void __exit ionic_cleanup_module(void)
+{
+	ionic_bus_unregister_driver();
+}
+
+module_init(ionic_init_module);
+module_exit(ionic_cleanup_module);
Signature · 1 line
2.17.1
#2 lore raw

PATCH v3 2/19 ionic: Add hardware init and device commands

In reply to Shannon Nelson

The ionic device has a small set of PCI registers, including a device control and data space, and a large set of message commands.

---
 drivers/net/ethernet/pensando/ionic/Makefile  |    2 +-
 drivers/net/ethernet/pensando/ionic/ionic.h   |   20 +
 .../net/ethernet/pensando/ionic/ionic_bus.h   |    1 +
 .../ethernet/pensando/ionic/ionic_bus_pci.c   |  140 +-
 .../ethernet/pensando/ionic/ionic_debugfs.c   |   67 +
 .../ethernet/pensando/ionic/ionic_debugfs.h   |   28 +
 .../net/ethernet/pensando/ionic/ionic_dev.c   |  132 +
 .../net/ethernet/pensando/ionic/ionic_dev.h   |  144 +
 .../net/ethernet/pensando/ionic/ionic_if.h    | 2552 +++++++++++++++++
 .../net/ethernet/pensando/ionic/ionic_main.c  |  296 ++
 .../net/ethernet/pensando/ionic/ionic_regs.h  |  133 +
 11 files changed, 3512 insertions(+), 3 deletions(-)
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_debugfs.c
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_debugfs.h
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_dev.c
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_dev.h
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_if.h
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_regs.h
drivers/net/ethernet/pensando/ionic/Makefile +1 -1
index beb3faeccac1..49a8e28a5c84 100644
@@ -3,4 +3,4 @@
 
 obj-$(CONFIG_IONIC) := ionic.o
 
-ionic-y := ionic_main.o ionic_bus_pci.o
+ionic-y := ionic_main.o ionic_bus_pci.o ionic_dev.o ionic_debugfs.o
drivers/net/ethernet/pensando/ionic/ionic.h +20 -0
index 18c79adef06c..4fd0ae86556a 100644
@@ -4,6 +4,10 @@
 #ifndef _IONIC_H_
 #define _IONIC_H_
 
+#include "ionic_if.h"
+#include "ionic_dev.h"
+#include "ionic_debugfs.h"
+
 #define DRV_NAME		"ionic"
 #define DRV_DESCRIPTION		"Pensando Ethernet NIC Driver"
 #define DRV_VERSION		"0.11.0-k"
@@ -17,9 +21,25 @@
 #define IONIC_SUBDEV_ID_NAPLES_100_4	0x4001
 #define IONIC_SUBDEV_ID_NAPLES_100_8	0x4002
 
+#define devcmd_timeout  10
+
 struct ionic {
 	struct pci_dev *pdev;
 	struct device *dev;
+	struct ionic_dev idev;
+	struct mutex dev_cmd_lock;	/* lock for dev_cmd operations */
+	struct dentry *dentry;
+	struct ionic_dev_bar bars[IONIC_BARS_MAX];
+	unsigned int num_bars;
+	struct identity ident;
 };
 
+int ionic_dev_cmd_wait(struct ionic *ionic, unsigned long max_wait);
+int ionic_set_dma_mask(struct ionic *ionic);
+int ionic_setup(struct ionic *ionic);
+
+int ionic_identify(struct ionic *ionic);
+int ionic_init(struct ionic *ionic);
+int ionic_reset(struct ionic *ionic);
+
 #endif /* _IONIC_H_ */
drivers/net/ethernet/pensando/ionic/ionic_bus.h +1 -0
index 94ba0afc6f38..24b4c01ec03f 100644
@@ -4,6 +4,7 @@
 #ifndef _IONIC_BUS_H_
 #define _IONIC_BUS_H_
 
+const char *ionic_bus_info(struct ionic *ionic);
 int ionic_bus_register_driver(void);
 void ionic_bus_unregister_driver(void);
 
drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c +138 -2

165 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_debugfs.c new file +67 -0

67 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_debugfs.h new file +28 -0
new file mode 100644
index 000000000000..c3958b0c15b1
@@ -0,0 +1,28 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+
+#ifndef _IONIC_DEBUGFS_H_
+#define _IONIC_DEBUGFS_H_
+
+#include <linux/debugfs.h>
+
+#ifdef CONFIG_DEBUG_FS
+
+void ionic_debugfs_create(void);
+void ionic_debugfs_destroy(void);
+int ionic_debugfs_add_dev(struct ionic *ionic);
+void ionic_debugfs_del_dev(struct ionic *ionic);
+int ionic_debugfs_add_bars(struct ionic *ionic);
+int ionic_debugfs_add_dev_cmd(struct ionic *ionic);
+int ionic_debugfs_add_ident(struct ionic *ionic);
+#else
+static inline void ionic_debugfs_create(void) { }
+static inline void ionic_debugfs_destroy(void) { }
+static inline int ionic_debugfs_add_dev(struct ionic *ionic) { return 0; }
+static inline void ionic_debugfs_del_dev(struct ionic *ionic) { }
+static inline int ionic_debugfs_add_bars(struct ionic *ionic) { return 0; }
+static inline int ionic_debugfs_add_dev_cmd(struct ionic *ionic) { return 0; }
+static inline int ionic_debugfs_add_ident(struct ionic *ionic) { return 0; }
+#endif
+
+#endif /* _IONIC_DEBUGFS_H_ */
drivers/net/ethernet/pensando/ionic/ionic_dev.c new file +132 -0

132 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_dev.h new file +144 -0

144 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_if.h new file +2552 -0

2,417 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_main.c +296 -0

296 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_regs.h new file +133 -0

133 lines not on this page · raw message

The rest of this diff is not shown. View the raw message.

Signature · 1 line
2.17.1

Part of the diff is not shown here. View the raw message.

#3 lore raw

PATCH v3 3/19 ionic: Add port management commands

In reply to Shannon Nelson

The port management commands apply to the physical port associated with the PCI device, which might be shared among several logical interfaces.

---
 drivers/net/ethernet/pensando/ionic/ionic.h   |   4 +
 .../ethernet/pensando/ionic/ionic_bus_pci.c   |  16 +++
 .../net/ethernet/pensando/ionic/ionic_dev.c   | 116 ++++++++++++++++++
 .../net/ethernet/pensando/ionic/ionic_dev.h   |  15 +++
 .../net/ethernet/pensando/ionic/ionic_main.c  |  95 ++++++++++++++
 5 files changed, 246 insertions(+)
drivers/net/ethernet/pensando/ionic/ionic.h +4 -0
index 4fd0ae86556a..f77712feef5c 100644
@@ -42,4 +42,8 @@ int ionic_identify(struct ionic *ionic);
 int ionic_init(struct ionic *ionic);
 int ionic_reset(struct ionic *ionic);
 
+int ionic_port_identify(struct ionic *ionic);
+int ionic_port_init(struct ionic *ionic);
+int ionic_port_reset(struct ionic *ionic);
+
 #endif /* _IONIC_H_ */
drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c +16 -0
index 7eb23f0e87ac..7a41f8995c35 100644
@@ -137,8 +137,23 @@ static int ionic_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
 		goto err_out_teardown;
 	}
 
+	/* Configure the ports */
+	err = ionic_port_identify(ionic);
+	if (err) {
+		dev_err(dev, "Cannot identify port: %d, aborting\n", err);
+		goto err_out_reset;
+	}
+
+	err = ionic_port_init(ionic);
+	if (err) {
+		dev_err(dev, "Cannot init port: %d, aborting\n", err);
+		goto err_out_reset;
+	}
+
 	return 0;
 
+err_out_reset:
+	ionic_reset(ionic);
 err_out_teardown:
 	ionic_dev_teardown(ionic);
 err_out_unmap_bars:
@@ -162,6 +177,7 @@ static void ionic_remove(struct pci_dev *pdev)
 	struct ionic *ionic = pci_get_drvdata(pdev);
 
 	if (ionic) {
+		ionic_port_reset(ionic);
 		ionic_reset(ionic);
 		ionic_dev_teardown(ionic);
 		ionic_unmap_bars(ionic);
drivers/net/ethernet/pensando/ionic/ionic_dev.c +116 -0

119 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_dev.h +15 -0
index 66afab3ee396..fe5e1b0e8d55 100644
@@ -123,6 +123,10 @@ struct ionic_dev {
 	struct ionic_intr __iomem *intr_ctrl;
 	u64 __iomem *intr_status;
 
+	struct port_info *port_info;
+	dma_addr_t port_info_pa;
+	u32 port_info_sz;
+
 	struct ionic_devinfo dev_info;
 };
 
@@ -141,4 +145,15 @@ void ionic_dev_cmd_identify(struct ionic_dev *idev, u8 ver);
 void ionic_dev_cmd_init(struct ionic_dev *idev);
 void ionic_dev_cmd_reset(struct ionic_dev *idev);
 
+void ionic_dev_cmd_port_identify(struct ionic_dev *idev);
+void ionic_dev_cmd_port_init(struct ionic_dev *idev);
+void ionic_dev_cmd_port_reset(struct ionic_dev *idev);
+void ionic_dev_cmd_port_state(struct ionic_dev *idev, u8 state);
+void ionic_dev_cmd_port_speed(struct ionic_dev *idev, u32 speed);
+void ionic_dev_cmd_port_mtu(struct ionic_dev *idev, u32 mtu);
+void ionic_dev_cmd_port_autoneg(struct ionic_dev *idev, u8 an_enable);
+void ionic_dev_cmd_port_fec(struct ionic_dev *idev, u8 fec_type);
+void ionic_dev_cmd_port_pause(struct ionic_dev *idev, u8 pause_type);
+void ionic_dev_cmd_port_loopback(struct ionic_dev *idev, u8 loopback_mode);
+
 #endif /* _IONIC_DEV_H_ */
drivers/net/ethernet/pensando/ionic/ionic_main.c +95 -0

101 lines not on this page · raw message

Signature · 1 line
2.17.1
#4 lore raw

PATCH v3 4/19 ionic: Add basic lif support

In reply to Shannon Nelson

The LIF is the Logical Interface, which represents the external connections. The NIC can multiplex many LIFs to a single port, but in most setups, LIF0 is the primary control for the port.

---
 drivers/net/ethernet/pensando/ionic/Makefile  |   3 +-
 drivers/net/ethernet/pensando/ionic/ionic.h   |   8 +-
 .../net/ethernet/pensando/ionic/ionic_bus.h   |   2 +
 .../ethernet/pensando/ionic/ionic_bus_pci.c   |  48 +++
 .../ethernet/pensando/ionic/ionic_debugfs.c   |  51 +++
 .../ethernet/pensando/ionic/ionic_debugfs.h   |   6 +
 .../net/ethernet/pensando/ionic/ionic_dev.c   |  34 ++
 .../net/ethernet/pensando/ionic/ionic_dev.h   |   7 +
 .../net/ethernet/pensando/ionic/ionic_lif.c   | 292 ++++++++++++++++++
 .../net/ethernet/pensando/ionic/ionic_lif.h   |  70 +++++
 .../net/ethernet/pensando/ionic/ionic_main.c  |   1 +
 11 files changed, 520 insertions(+), 2 deletions(-)
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_lif.c
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_lif.h
drivers/net/ethernet/pensando/ionic/Makefile +2 -1
index 49a8e28a5c84..04d519d00be6 100644
@@ -3,4 +3,5 @@
 
 obj-$(CONFIG_IONIC) := ionic.o
 
-ionic-y := ionic_main.o ionic_bus_pci.o ionic_dev.o ionic_debugfs.o
+ionic-y := ionic_main.o ionic_bus_pci.o ionic_dev.o ionic_debugfs.o \
+	   ionic_lif.o
drivers/net/ethernet/pensando/ionic/ionic.h +7 -1
index f77712feef5c..afff2e074f1a 100644
@@ -6,7 +6,6 @@
 
 #include "ionic_if.h"
 #include "ionic_dev.h"
-#include "ionic_debugfs.h"
 
 #define DRV_NAME		"ionic"
 #define DRV_DESCRIPTION		"Pensando Ethernet NIC Driver"
@@ -32,6 +31,13 @@ struct ionic {
 	struct ionic_dev_bar bars[IONIC_BARS_MAX];
 	unsigned int num_bars;
 	struct identity ident;
+	struct list_head lifs;
+	unsigned int nnqs_per_lif;
+	unsigned int neqs_per_lif;
+	unsigned int ntxqs_per_lif;
+	unsigned int nrxqs_per_lif;
+	DECLARE_BITMAP(lifbits, IONIC_LIFS_MAX);
+	unsigned int nintrs;
 };
 
 int ionic_dev_cmd_wait(struct ionic *ionic, unsigned long max_wait);
drivers/net/ethernet/pensando/ionic/ionic_bus.h +2 -0
index 24b4c01ec03f..3b1e2d0ebf8f 100644
@@ -5,6 +5,8 @@
 #define _IONIC_BUS_H_
 
 const char *ionic_bus_info(struct ionic *ionic);
+int ionic_bus_alloc_irq_vectors(struct ionic *ionic, unsigned int nintrs);
+void ionic_bus_free_irq_vectors(struct ionic *ionic);
 int ionic_bus_register_driver(void);
 void ionic_bus_unregister_driver(void);
 
drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c +48 -0

74 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_debugfs.c +51 -0

65 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_debugfs.h +6 -0
index c3958b0c15b1..cb00166e7c30 100644
@@ -15,6 +15,9 @@ void ionic_debugfs_del_dev(struct ionic *ionic);
 int ionic_debugfs_add_bars(struct ionic *ionic);
 int ionic_debugfs_add_dev_cmd(struct ionic *ionic);
 int ionic_debugfs_add_ident(struct ionic *ionic);
+int ionic_debugfs_add_sizes(struct ionic *ionic);
+int ionic_debugfs_add_lif(struct lif *lif);
+void ionic_debugfs_del_lif(struct lif *lif);
 #else
 static inline void ionic_debugfs_create(void) { }
 static inline void ionic_debugfs_destroy(void) { }
@@ -23,6 +26,9 @@ static inline void ionic_debugfs_del_dev(struct ionic *ionic) { }
 static inline int ionic_debugfs_add_bars(struct ionic *ionic) { return 0; }
 static inline int ionic_debugfs_add_dev_cmd(struct ionic *ionic) { return 0; }
 static inline int ionic_debugfs_add_ident(struct ionic *ionic) { return 0; }
+static inline int ionic_debugfs_add_sizes(struct ionic *ionic) { return 0; }
+static inline int ionic_debugfs_add_lif(struct lif *lif) { return 0; }
+static inline void ionic_debugfs_del_lif(struct lif *lif) { return 0; }
 #endif
 
 #endif /* _IONIC_DEBUGFS_H_ */
drivers/net/ethernet/pensando/ionic/ionic_dev.c +34 -0
index e2eda2bedac6..59d863e45072 100644
@@ -246,3 +246,37 @@ void ionic_dev_cmd_port_loopback(struct ionic_dev *idev, u8 loopback_mode)
 
 	ionic_dev_cmd_go(idev, &cmd);
 }
+
+/* LIF commands */
+void ionic_dev_cmd_lif_identify(struct ionic_dev *idev, u8 type, u8 ver)
+{
+	union dev_cmd cmd = {
+		.lif_identify.opcode = CMD_OPCODE_LIF_IDENTIFY,
+		.lif_identify.type = type,
+		.lif_identify.ver = ver,
+	};
+
+	ionic_dev_cmd_go(idev, &cmd);
+}
+
+void ionic_dev_cmd_lif_init(struct ionic_dev *idev, u16 lif_index,
+			    dma_addr_t info_pa)
+{
+	union dev_cmd cmd = {
+		.lif_init.opcode = CMD_OPCODE_LIF_INIT,
+		.lif_init.index = cpu_to_le16(lif_index),
+		.lif_init.info_pa = cpu_to_le64(info_pa),
+	};
+
+	ionic_dev_cmd_go(idev, &cmd);
+}
+
+void ionic_dev_cmd_lif_reset(struct ionic_dev *idev, u16 lif_index)
+{
+	union dev_cmd cmd = {
+		.lif_init.opcode = CMD_OPCODE_LIF_RESET,
+		.lif_init.index = cpu_to_le16(lif_index),
+	};
+
+	ionic_dev_cmd_go(idev, &cmd);
+}
drivers/net/ethernet/pensando/ionic/ionic_dev.h +7 -0
index fe5e1b0e8d55..a8a84734b433 100644
@@ -10,6 +10,8 @@
 #include "ionic_if.h"
 #include "ionic_regs.h"
 
+#define IONIC_LIFS_MAX			1024
+
 struct ionic_dev_bar {
 	void __iomem *vaddr;
 	phys_addr_t bus_addr;
@@ -156,4 +158,9 @@ void ionic_dev_cmd_port_fec(struct ionic_dev *idev, u8 fec_type);
 void ionic_dev_cmd_port_pause(struct ionic_dev *idev, u8 pause_type);
 void ionic_dev_cmd_port_loopback(struct ionic_dev *idev, u8 loopback_mode);
 
+void ionic_dev_cmd_lif_identify(struct ionic_dev *idev, u8 type, u8 ver);
+void ionic_dev_cmd_lif_init(struct ionic_dev *idev, u16 lif_index,
+			    dma_addr_t addr);
+void ionic_dev_cmd_lif_reset(struct ionic_dev *idev, u16 lif_index);
+
 #endif /* _IONIC_DEV_H_ */
drivers/net/ethernet/pensando/ionic/ionic_lif.c new file +292 -0

292 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_lif.h new file +70 -0

70 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_main.c +1 -0
index 78eab326b067..b732e7345550 100644
@@ -8,6 +8,7 @@
 
 #include "ionic.h"
 #include "ionic_bus.h"
+#include "ionic_lif.h"
 #include "ionic_debugfs.h"
 
 MODULE_DESCRIPTION(DRV_DESCRIPTION);
Signature · 1 line
2.17.1
#5 lore raw

PATCH v3 5/19 ionic: Add interrupts and doorbells

In reply to Shannon Nelson

The ionic interrupt model is based on interrupt control blocks accessed through the PCI BAR. Doorbell registers are used by the driver to signal to the NIC that requests are waiting on the message queues. Interrupts are used by the NIC to signal to the driver that answers are waiting on the completion queues.

---
 drivers/net/ethernet/pensando/ionic/ionic.h   |  3 +
 .../net/ethernet/pensando/ionic/ionic_bus.h   |  2 +
 .../ethernet/pensando/ionic/ionic_bus_pci.c   | 12 ++++
 .../net/ethernet/pensando/ionic/ionic_dev.c   |  6 ++
 .../net/ethernet/pensando/ionic/ionic_dev.h   | 21 ++++++
 .../net/ethernet/pensando/ionic/ionic_lif.c   | 64 +++++++++++++++++++
 .../net/ethernet/pensando/ionic/ionic_lif.h   |  4 ++
 7 files changed, 112 insertions(+)
drivers/net/ethernet/pensando/ionic/ionic.h +3 -0
index afff2e074f1a..c86a08752b3b 100644
@@ -4,6 +4,8 @@
 #ifndef _IONIC_H_
 #define _IONIC_H_
 
+struct lif;
+
 #include "ionic_if.h"
 #include "ionic_dev.h"
 
@@ -38,6 +40,7 @@ struct ionic {
 	unsigned int nrxqs_per_lif;
 	DECLARE_BITMAP(lifbits, IONIC_LIFS_MAX);
 	unsigned int nintrs;
+	DECLARE_BITMAP(intrs, INTR_CTRL_REGS_MAX);
 };
 
 int ionic_dev_cmd_wait(struct ionic *ionic, unsigned long max_wait);
drivers/net/ethernet/pensando/ionic/ionic_bus.h +2 -0
index 3b1e2d0ebf8f..6b29e94f81d6 100644
@@ -9,5 +9,7 @@ int ionic_bus_alloc_irq_vectors(struct ionic *ionic, unsigned int nintrs);
 void ionic_bus_free_irq_vectors(struct ionic *ionic);
 int ionic_bus_register_driver(void);
 void ionic_bus_unregister_driver(void);
+void __iomem *ionic_bus_map_dbpage(struct ionic *ionic, int page_num);
+void ionic_bus_unmap_dbpage(struct ionic *ionic, void __iomem *page);
 
 #endif /* _IONIC_BUS_H_ */
drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c +12 -0
index 0ed724efe117..838d7c423b2c 100644
@@ -87,6 +87,18 @@ static void ionic_unmap_bars(struct ionic *ionic)
 	}
 }
 
+void __iomem *ionic_bus_map_dbpage(struct ionic *ionic, int page_num)
+{
+	return pci_iomap_range(ionic->pdev,
+			       ionic->bars[IONIC_PCI_BAR_DBELL].res_index,
+			       (u64)page_num << PAGE_SHIFT, PAGE_SIZE);
+}
+
+void ionic_bus_unmap_dbpage(struct ionic *ionic, void __iomem *page)
+{
+	iounmap(page);
+}
+
 static int ionic_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
 {
 	struct device *dev = &pdev->dev;
drivers/net/ethernet/pensando/ionic/ionic_dev.c +6 -0
index 59d863e45072..c0710bf200a5 100644
@@ -9,6 +9,7 @@
 #include <linux/etherdevice.h>
 #include "ionic.h"
 #include "ionic_dev.h"
+#include "ionic_lif.h"
 
 void ionic_init_devinfo(struct ionic_dev *idev)
 {
@@ -280,3 +281,8 @@ void ionic_dev_cmd_lif_reset(struct ionic_dev *idev, u16 lif_index)
 
 	ionic_dev_cmd_go(idev, &cmd);
 }
+
+int ionic_db_page_num(struct lif *lif, int pid)
+{
+	return (lif->hw_index * lif->dbid_count) + pid;
+}
drivers/net/ethernet/pensando/ionic/ionic_dev.h +21 -0
index a8a84734b433..6d30adeab8c5 100644
@@ -132,8 +132,27 @@ struct ionic_dev {
 	struct ionic_devinfo dev_info;
 };
 
+#define INTR_INDEX_NOT_ASSIGNED		-1
+#define INTR_NAME_MAX_SZ		32
+
+struct intr {
+	char name[INTR_NAME_MAX_SZ];
+	unsigned int index;
+	unsigned int vector;
+	u64 rearm_count;
+	unsigned int cpu;
+	cpumask_t affinity_mask;
+};
+
 struct ionic;
 
+static inline void ionic_intr_init(struct ionic_dev *idev, struct intr *intr,
+				   unsigned long index)
+{
+	ionic_intr_clean(idev->intr_ctrl, index);
+	intr->index = index;
+}
+
 void ionic_init_devinfo(struct ionic_dev *idev);
 int ionic_dev_setup(struct ionic *ionic);
 void ionic_dev_teardown(struct ionic *ionic);
@@ -163,4 +182,6 @@ void ionic_dev_cmd_lif_init(struct ionic_dev *idev, u16 lif_index,
 			    dma_addr_t addr);
 void ionic_dev_cmd_lif_reset(struct ionic_dev *idev, u16 lif_index);
 
+int ionic_db_page_num(struct lif *lif, int pid);
+
 #endif /* _IONIC_DEV_H_ */
drivers/net/ethernet/pensando/ionic/ionic_lif.c +64 -0

92 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_lif.h +4 -0
index 37f79d36744a..98d0699234e9 100644
@@ -23,6 +23,8 @@ struct lif {
 	bool registered;
 	unsigned int index;
 	unsigned int hw_index;
+	unsigned int kern_pid;
+	u64 __iomem *kern_dbpage;
 	unsigned int neqs;
 	unsigned int nxqs;
 
@@ -30,6 +32,8 @@ struct lif {
 	dma_addr_t info_pa;
 	u32 info_sz;
 
+	unsigned long *dbid_inuse;
+	unsigned int dbid_count;
 	struct dentry *dentry;
 	u32 flags;
 };
Signature · 1 line
2.17.1
#6 lore raw

PATCH v3 6/19 ionic: Add basic adminq support

In reply to Shannon Nelson

Most of the NIC configuration happens through the AdminQ message queue. NAPI is used for basic interrupt handling and message queue management. These routines are set up to be shared among different types of queues when used in slow-path handling.

---
 drivers/net/ethernet/pensando/ionic/ionic.h   |   3 +
 .../net/ethernet/pensando/ionic/ionic_bus.h   |   1 +
 .../ethernet/pensando/ionic/ionic_bus_pci.c   |   5 +
 .../ethernet/pensando/ionic/ionic_debugfs.c   | 149 ++++++++
 .../ethernet/pensando/ionic/ionic_debugfs.h   |   4 +
 .../net/ethernet/pensando/ionic/ionic_dev.c   | 247 +++++++++++++
 .../net/ethernet/pensando/ionic/ionic_dev.h   |  92 +++++
 .../net/ethernet/pensando/ionic/ionic_lif.c   | 328 ++++++++++++++++++
 .../net/ethernet/pensando/ionic/ionic_lif.h   |  72 ++++
 .../net/ethernet/pensando/ionic/ionic_main.c  |  26 ++
 10 files changed, 927 insertions(+)
drivers/net/ethernet/pensando/ionic/ionic.h +3 -0
index c86a08752b3b..c8c93e9efb39 100644
@@ -43,6 +43,9 @@ struct ionic {
 	DECLARE_BITMAP(intrs, INTR_CTRL_REGS_MAX);
 };
 
+int ionic_napi(struct napi_struct *napi, int budget, ionic_cq_cb cb,
+	       ionic_cq_done_cb done_cb, void *done_arg);
+
 int ionic_dev_cmd_wait(struct ionic *ionic, unsigned long max_wait);
 int ionic_set_dma_mask(struct ionic *ionic);
 int ionic_setup(struct ionic *ionic);
drivers/net/ethernet/pensando/ionic/ionic_bus.h +1 -0
index 6b29e94f81d6..2f4d08c64910 100644
@@ -4,6 +4,7 @@
 #ifndef _IONIC_BUS_H_
 #define _IONIC_BUS_H_
 
+int ionic_bus_get_irq(struct ionic *ionic, unsigned int num);
 const char *ionic_bus_info(struct ionic *ionic);
 int ionic_bus_alloc_irq_vectors(struct ionic *ionic, unsigned int nintrs);
 void ionic_bus_free_irq_vectors(struct ionic *ionic);
drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c +5 -0
index 838d7c423b2c..59d1ae7ce532 100644
@@ -19,6 +19,11 @@ static const struct pci_device_id ionic_id_table[] = {
 };
 MODULE_DEVICE_TABLE(pci, ionic_id_table);
 
+int ionic_bus_get_irq(struct ionic *ionic, unsigned int num)
+{
+	return pci_irq_vector(ionic->pdev, num);
+}
+
 const char *ionic_bus_info(struct ionic *ionic)
 {
 	return pci_name(ionic->pdev);
drivers/net/ethernet/pensando/ionic/ionic_debugfs.c +149 -0

159 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_debugfs.h +4 -0
index cb00166e7c30..fa5030d48763 100644
@@ -17,7 +17,9 @@ int ionic_debugfs_add_dev_cmd(struct ionic *ionic);
 int ionic_debugfs_add_ident(struct ionic *ionic);
 int ionic_debugfs_add_sizes(struct ionic *ionic);
 int ionic_debugfs_add_lif(struct lif *lif);
+int ionic_debugfs_add_qcq(struct lif *lif, struct qcq *qcq);
 void ionic_debugfs_del_lif(struct lif *lif);
+void ionic_debugfs_del_qcq(struct qcq *qcq);
 #else
 static inline void ionic_debugfs_create(void) { }
 static inline void ionic_debugfs_destroy(void) { }
@@ -28,7 +30,9 @@ static inline int ionic_debugfs_add_dev_cmd(struct ionic *ionic) { return 0; }
 static inline int ionic_debugfs_add_ident(struct ionic *ionic) { return 0; }
 static inline int ionic_debugfs_add_sizes(struct ionic *ionic) { return 0; }
 static inline int ionic_debugfs_add_lif(struct lif *lif) { return 0; }
+static inline int ionic_debugfs_add_qcq(struct lif *lif, struct qcq *qcq) { return 0; }
 static inline void ionic_debugfs_del_lif(struct lif *lif) { return 0; }
+static inline void ionic_debugfs_del_qcq(struct qcq *qcq) { return 0; }
 #endif
 
 #endif /* _IONIC_DEBUGFS_H_ */
drivers/net/ethernet/pensando/ionic/ionic_dev.c +247 -0

254 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_dev.h +92 -0

111 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_lif.c +328 -0

390 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_lif.h +72 -0

88 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_main.c +26 -0
index b732e7345550..aa73ea81a4b2 100644
@@ -169,6 +169,32 @@ static const char *ionic_opcode_to_str(enum cmd_opcode opcode)
 	}
 }
 
+int ionic_napi(struct napi_struct *napi, int budget, ionic_cq_cb cb,
+	       ionic_cq_done_cb done_cb, void *done_arg)
+{
+	struct qcq *qcq = napi_to_qcq(napi);
+	struct cq *cq = &qcq->cq;
+	u32 work_done, flags = 0;
+
+	work_done = ionic_cq_service(cq, budget, cb, done_cb, done_arg);
+
+	if (work_done < budget && napi_complete_done(napi, work_done)) {
+		flags |= IONIC_INTR_CRED_UNMASK;
+		DEBUG_STATS_INTR_REARM(cq->bound_intr);
+	}
+
+	if (work_done || flags) {
+		flags |= IONIC_INTR_CRED_RESET_COALESCE;
+		ionic_intr_credits(cq->lif->ionic->idev.intr_ctrl,
+				   cq->bound_intr->index,
+				   work_done, flags);
+	}
+
+	DEBUG_STATS_NAPI_POLL(qcq, work_done);
+
+	return work_done;
+}
+
 int ionic_dev_cmd_wait(struct ionic *ionic, unsigned long max_seconds)
 {
 	struct ionic_dev *idev = &ionic->idev;
Signature · 1 line
2.17.1
#7 lore raw

PATCH v3 7/19 ionic: Add adminq action

In reply to Shannon Nelson

Add AdminQ specific message requests and completion handling.

---
 drivers/net/ethernet/pensando/ionic/ionic.h   |   7 ++
 .../net/ethernet/pensando/ionic/ionic_main.c  | 106 ++++++++++++++++++
 2 files changed, 113 insertions(+)
drivers/net/ethernet/pensando/ionic/ionic.h +7 -0
index c8c93e9efb39..87ab13aee89e 100644
@@ -43,9 +43,16 @@ struct ionic {
 	DECLARE_BITMAP(intrs, INTR_CTRL_REGS_MAX);
 };
 
+struct ionic_admin_ctx {
+	struct completion work;
+	union adminq_cmd cmd;
+	union adminq_comp comp;
+};
+
 int ionic_napi(struct napi_struct *napi, int budget, ionic_cq_cb cb,
 	       ionic_cq_done_cb done_cb, void *done_arg);
 
+int ionic_adminq_post_wait(struct lif *lif, struct ionic_admin_ctx *ctx);
 int ionic_dev_cmd_wait(struct ionic *ionic, unsigned long max_wait);
 int ionic_set_dma_mask(struct ionic *ionic);
 int ionic_setup(struct ionic *ionic);
drivers/net/ethernet/pensando/ionic/ionic_main.c +106 -0

112 lines not on this page · raw message

Signature · 1 line
2.17.1
#8 lore raw

PATCH v3 8/19 ionic: Add notifyq support

In reply to Shannon Nelson

The AdminQ is fine for sending messages and requests to the NIC, but we also need to have events published from the NIC to the driver. The NotifyQ handles this for us, using the same interrupt as AdminQ.

---
 .../ethernet/pensando/ionic/ionic_debugfs.c   |  16 ++
 .../net/ethernet/pensando/ionic/ionic_lif.c   | 181 +++++++++++++++++-
 .../net/ethernet/pensando/ionic/ionic_lif.h   |   4 +
 3 files changed, 200 insertions(+), 1 deletion(-)
drivers/net/ethernet/pensando/ionic/ionic_debugfs.c +16 -0
index b5e77d35ce6f..cc13198a16f9 100644
@@ -130,6 +130,7 @@ int ionic_debugfs_add_qcq(struct lif *lif, struct qcq *qcq)
 	struct debugfs_blob_wrapper *desc_blob;
 	struct device *dev = lif->ionic->dev;
 	struct intr *intr = &qcq->intr;
+	struct dentry *stats_dentry;
 	struct queue *q = &qcq->q;
 	struct cq *cq = &qcq->cq;
 
@@ -223,6 +224,21 @@ int ionic_debugfs_add_qcq(struct lif *lif, struct qcq *qcq)
 					intr_ctrl_regset);
 	}
 
+	if (qcq->flags & QCQ_F_NOTIFYQ) {
+		stats_dentry = debugfs_create_dir("notifyblock", qcq_dentry);
+		if (IS_ERR_OR_NULL(stats_dentry))
+			return PTR_ERR(stats_dentry);
+
+		debugfs_create_u64("eid", 0400, stats_dentry,
+				   (u64 *)&lif->info->status.eid);
+		debugfs_create_u16("link_status", 0400, stats_dentry,
+				   (u16 *)&lif->info->status.link_status);
+		debugfs_create_u32("link_speed", 0400, stats_dentry,
+				   (u32 *)&lif->info->status.link_speed);
+		debugfs_create_u16("link_down_count", 0400, stats_dentry,
+				   (u16 *)&lif->info->status.link_down_count);
+	}
+
 	return 0;
 }
 
drivers/net/ethernet/pensando/ionic/ionic_lif.c +180 -1

232 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_lif.h +4 -0
index 28ab92b43a64..80eec0778f40 100644
@@ -7,6 +7,7 @@
 #include <linux/pci.h>
 
 #define IONIC_ADMINQ_LENGTH	16	/* must be a power of two */
+#define IONIC_NOTIFYQ_LENGTH	64	/* must be a power of two */
 
 #define GET_NAPI_CNTR_IDX(work_done)	(work_done)
 #define MAX_NUM_NAPI_CNTR	(NAPI_POLL_WEIGHT + 1)
@@ -26,6 +27,7 @@ struct rx_stats {
 #define QCQ_F_INITED		BIT(0)
 #define QCQ_F_SG		BIT(1)
 #define QCQ_F_INTR		BIT(2)
+#define QCQ_F_NOTIFYQ		BIT(5)
 
 struct napi_stats {
 	u64 poll_count;
@@ -78,6 +80,8 @@ struct lif {
 	u64 __iomem *kern_dbpage;
 	spinlock_t adminq_lock;		/* lock for AdminQ operations */
 	struct qcq *adminqcq;
+	struct qcq *notifyqcq;
+	u64 last_eid;
 	unsigned int neqs;
 	unsigned int nxqs;
 
Signature · 1 line
2.17.1
#9 lore raw

PATCH v3 9/19 ionic: Add the basic NDO callbacks for netdev support

In reply to Shannon Nelson

Set up the initial NDO structure and callbacks for netdev to use, and register the netdev. This will allow us to do a few basic operations on the device, but no traffic yet.

---
 drivers/net/ethernet/pensando/ionic/ionic.h   |   1 +
 .../ethernet/pensando/ionic/ionic_bus_pci.c   |   9 +
 .../net/ethernet/pensando/ionic/ionic_dev.h   |   2 +
 .../net/ethernet/pensando/ionic/ionic_lif.c   | 348 ++++++++++++++++++
 .../net/ethernet/pensando/ionic/ionic_lif.h   |   5 +
 5 files changed, 365 insertions(+)
drivers/net/ethernet/pensando/ionic/ionic.h +1 -0
index 87ab13aee89e..d7eee79b2a10 100644
@@ -34,6 +34,7 @@ struct ionic {
 	unsigned int num_bars;
 	struct identity ident;
 	struct list_head lifs;
+	struct lif *master_lif;
 	unsigned int nnqs_per_lif;
 	unsigned int neqs_per_lif;
 	unsigned int ntxqs_per_lif;
drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c +9 -0
index 59d1ae7ce532..98c12b770c7f 100644
@@ -206,8 +206,16 @@ static int ionic_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
 		goto err_out_free_lifs;
 	}
 
+	err = ionic_lifs_register(ionic);
+	if (err) {
+		dev_err(dev, "Cannot register LIFs: %d, aborting\n", err);
+		goto err_out_deinit_lifs;
+	}
+
 	return 0;
 
+err_out_deinit_lifs:
+	ionic_lifs_deinit(ionic);
 err_out_free_lifs:
 	ionic_lifs_free(ionic);
 err_out_free_irqs:
@@ -239,6 +247,7 @@ static void ionic_remove(struct pci_dev *pdev)
 	struct ionic *ionic = pci_get_drvdata(pdev);
 
 	if (ionic) {
+		ionic_lifs_unregister(ionic);
 		ionic_lifs_deinit(ionic);
 		ionic_lifs_free(ionic);
 		ionic_bus_free_irq_vectors(ionic);
drivers/net/ethernet/pensando/ionic/ionic_dev.h +2 -0
index 7014acd70b98..d44220c1d430 100644
@@ -10,6 +10,8 @@
 #include "ionic_if.h"
 #include "ionic_regs.h"
 
+#define IONIC_MIN_MTU			ETH_MIN_MTU
+#define IONIC_MAX_MTU			9194
 #define IONIC_LIFS_MAX			1024
 
 struct ionic_dev_bar {
drivers/net/ethernet/pensando/ionic/ionic_lif.c +348 -0

386 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_lif.h +5 -0
index 80eec0778f40..ef3f7340a277 100644
@@ -61,6 +61,8 @@ struct qcq {
 
 enum lif_state_flags {
 	LIF_INITED,
+	LIF_UP,
+	LIF_QUEUE_RESET,
 
 	/* leave this as last */
 	LIF_STATE_SIZE
@@ -84,6 +86,7 @@ struct lif {
 	u64 last_eid;
 	unsigned int neqs;
 	unsigned int nxqs;
+	u64 hw_features;
 
 	struct lif_info *info;
 	dma_addr_t info_pa;
@@ -124,6 +127,8 @@ int ionic_lifs_alloc(struct ionic *ionic);
 void ionic_lifs_free(struct ionic *ionic);
 void ionic_lifs_deinit(struct ionic *ionic);
 int ionic_lifs_init(struct ionic *ionic);
+int ionic_lifs_register(struct ionic *ionic);
+void ionic_lifs_unregister(struct ionic *ionic);
 int ionic_lif_identify(struct ionic *ionic, u8 lif_type,
 		       union lif_identity *lif_ident);
 int ionic_lifs_size(struct ionic *ionic);
Signature · 1 line
2.17.1
#10 lore raw

PATCH v3 10/19 ionic: Add management of rx filters

In reply to Shannon Nelson

Set up the infrastructure for managing Rx filters. We can't ask the hardware for what filters it has, so we keep a local list of filters that we've pushed into the HW.

---
 drivers/net/ethernet/pensando/ionic/Makefile  |   4 +-
 .../net/ethernet/pensando/ionic/ionic_lif.c   |   6 +
 .../net/ethernet/pensando/ionic/ionic_lif.h   |   2 +
 .../ethernet/pensando/ionic/ionic_rx_filter.c | 142 ++++++++++++++++++
 .../ethernet/pensando/ionic/ionic_rx_filter.h |  34 +++++
 5 files changed, 186 insertions(+), 2 deletions(-)
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_rx_filter.c
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_rx_filter.h
drivers/net/ethernet/pensando/ionic/Makefile +2 -2
index 04d519d00be6..7d9cdc5f02a1 100644
@@ -3,5 +3,5 @@
 
 obj-$(CONFIG_IONIC) := ionic.o
 
-ionic-y := ionic_main.o ionic_bus_pci.o ionic_dev.o ionic_debugfs.o \
-	   ionic_lif.o
+ionic-y := ionic_main.o ionic_bus_pci.o ionic_dev.o \
+	   ionic_lif.o ionic_rx_filter.o ionic_debugfs.o
drivers/net/ethernet/pensando/ionic/ionic_lif.c +6 -0
index 005b1d908fa1..72ac0fd56b69 100644
@@ -667,6 +667,8 @@ static void ionic_lif_deinit(struct lif *lif)
 
 	clear_bit(LIF_INITED, lif->state);
 
+	ionic_rx_filters_deinit(lif);
+
 	napi_disable(&lif->adminqcq->napi);
 	ionic_lif_qcq_deinit(lif, lif->notifyqcq);
 	ionic_lif_qcq_deinit(lif, lif->adminqcq);
@@ -1033,6 +1035,10 @@ static int ionic_lif_init(struct lif *lif)
 	if (err)
 		goto err_out_notifyq_deinit;
 
+	err = ionic_rx_filters_init(lif);
+	if (err)
+		goto err_out_notifyq_deinit;
+
 	set_bit(LIF_INITED, lif->state);
 
 	return 0;
drivers/net/ethernet/pensando/ionic/ionic_lif.h +2 -0
index ef3f7340a277..9f112aa69033 100644
@@ -5,6 +5,7 @@
 #define _IONIC_LIF_H_
 
 #include <linux/pci.h>
+#include "ionic_rx_filter.h"
 
 #define IONIC_ADMINQ_LENGTH	16	/* must be a power of two */
 #define IONIC_NOTIFYQ_LENGTH	64	/* must be a power of two */
@@ -92,6 +93,7 @@ struct lif {
 	dma_addr_t info_pa;
 	u32 info_sz;
 
+	struct rx_filters rx_filters;
 	unsigned long *dbid_inuse;
 	unsigned int dbid_count;
 	struct dentry *dentry;
drivers/net/ethernet/pensando/ionic/ionic_rx_filter.c new file +142 -0

142 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_rx_filter.h new file +34 -0
new file mode 100644
index 000000000000..c8688124465a
@@ -0,0 +1,34 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+
+#ifndef _IONIC_RX_FILTER_H_
+#define _IONIC_RX_FILTER_H_
+
+#define RXQ_INDEX_ANY		(0xFFFF)
+struct rx_filter {
+	u32 flow_id;
+	u32 filter_id;
+	u16 rxq_index;
+	struct rx_filter_add_cmd cmd;
+	struct hlist_node by_hash;
+	struct hlist_node by_id;
+};
+
+#define RX_FILTER_HLISTS	BIT(10)
+#define RX_FILTER_HLISTS_MASK	(RX_FILTER_HLISTS - 1)
+struct rx_filters {
+	spinlock_t lock;				/* filter list lock */
+	struct hlist_head by_hash[RX_FILTER_HLISTS];	/* by skb hash */
+	struct hlist_head by_id[RX_FILTER_HLISTS];	/* by filter_id */
+};
+
+void ionic_rx_filter_free(struct lif *lif, struct rx_filter *f);
+int ionic_rx_filter_del(struct lif *lif, struct rx_filter *f);
+int ionic_rx_filters_init(struct lif *lif);
+void ionic_rx_filters_deinit(struct lif *lif);
+int ionic_rx_filter_save(struct lif *lif, u32 flow_id, u16 rxq_index,
+			 u32 hash, struct ionic_admin_ctx *ctx);
+struct rx_filter *ionic_rx_filter_by_vlan(struct lif *lif, u16 vid);
+struct rx_filter *ionic_rx_filter_by_addr(struct lif *lif, const u8 *addr);
+
+#endif /* _IONIC_RX_FILTER_H_ */
Signature · 1 line
2.17.1
#11 lore raw

PATCH v3 11/19 ionic: Add Rx filter and rx_mode ndo support

In reply to Shannon Nelson

Add the Rx filtering and rx_mode NDO callbacks. Also add the deferred work thread handling needed to manage the filter requests otuside of the netif_addr_lock spinlock.

---
 .../net/ethernet/pensando/ionic/ionic_lif.c   | 389 +++++++++++++++++-
 .../net/ethernet/pensando/ionic/ionic_lif.h   |  29 ++
 2 files changed, 412 insertions(+), 6 deletions(-)
drivers/net/ethernet/pensando/ionic/ionic_lif.c +383 -6
index 72ac0fd56b69..efcda1337f91 100644
@@ -12,9 +12,55 @@
 #include "ionic_lif.h"
 #include "ionic_debugfs.h"
 
+static void ionic_lif_rx_mode(struct lif *lif, unsigned int rx_mode);
+static int ionic_lif_addr_add(struct lif *lif, const u8 *addr);
+static int ionic_lif_addr_del(struct lif *lif, const u8 *addr);
+
 static int ionic_set_nic_features(struct lif *lif, netdev_features_t features);
 static int ionic_notifyq_clean(struct lif *lif, int budget);
 
+static void ionic_lif_deferred_work(struct work_struct *work)
+{
+	struct lif *lif = container_of(work, struct lif, deferred.work);
+	struct ionic_deferred *def = &lif->deferred;
+	struct ionic_deferred_work *w = NULL;
+
+	spin_lock_bh(&def->lock);
+	if (!list_empty(&def->list)) {
+		w = list_first_entry(&def->list,
+				     struct ionic_deferred_work, list);
+		list_del(&w->list);
+	}
+	spin_unlock_bh(&def->lock);
+
+	if (w) {
+		switch (w->type) {
+		case DW_TYPE_RX_MODE:
+			ionic_lif_rx_mode(lif, w->rx_mode);
+			break;
+		case DW_TYPE_RX_ADDR_ADD:
+			ionic_lif_addr_add(lif, w->addr);
+			break;
+		case DW_TYPE_RX_ADDR_DEL:
+			ionic_lif_addr_del(lif, w->addr);
+			break;
+		default:
+			break;
+		}
+		kfree(w);
+		schedule_work(&def->work);
+	}
+}
+
+static void ionic_lif_deferred_enqueue(struct ionic_deferred *def,
+				       struct ionic_deferred_work *work)
+{
+	spin_lock_bh(&def->lock);
+	list_add_tail(&work->list, &def->list);
+	spin_unlock_bh(&def->lock);
+	schedule_work(&def->work);
+}
+
 int ionic_open(struct net_device *netdev)
 {
 	struct lif *lif = netdev_priv(netdev);
@@ -180,6 +226,235 @@ static int ionic_notifyq_clean(struct lif *lif, int budget)
 	return work_done;
 }
 
+static int ionic_lif_addr_add(struct lif *lif, const u8 *addr)
+{
+	struct ionic_admin_ctx ctx = {
+		.work = COMPLETION_INITIALIZER_ONSTACK(ctx.work),
+		.cmd.rx_filter_add = {
+			.opcode = CMD_OPCODE_RX_FILTER_ADD,
+			.lif_index = cpu_to_le16(lif->index),
+			.match = cpu_to_le16(RX_FILTER_MATCH_MAC),
+		},
+	};
+	struct rx_filter *f;
+	int err;
+
+	/* don't bother if we already have it */
+	spin_lock_bh(&lif->rx_filters.lock);
+	f = ionic_rx_filter_by_addr(lif, addr);
+	spin_unlock_bh(&lif->rx_filters.lock);
+	if (f)
+		return 0;
+
+	netdev_dbg(lif->netdev, "rx_filter add ADDR %pM (id %d)\n", addr,
+		   ctx.comp.rx_filter_add.filter_id);
+
+	memcpy(ctx.cmd.rx_filter_add.mac.addr, addr, ETH_ALEN);
+	err = ionic_adminq_post_wait(lif, &ctx);
+	if (err)
+		return err;
+
+	return ionic_rx_filter_save(lif, 0, RXQ_INDEX_ANY, 0, &ctx);
+}
+
+static int ionic_lif_addr_del(struct lif *lif, const u8 *addr)
+{
+	struct ionic_admin_ctx ctx = {
+		.work = COMPLETION_INITIALIZER_ONSTACK(ctx.work),
+		.cmd.rx_filter_del = {
+			.opcode = CMD_OPCODE_RX_FILTER_DEL,
+			.lif_index = cpu_to_le16(lif->index),
+		},
+	};
+	struct rx_filter *f;
+	int err;
+
+	spin_lock_bh(&lif->rx_filters.lock);
+	f = ionic_rx_filter_by_addr(lif, addr);
+	if (!f) {
+		spin_unlock_bh(&lif->rx_filters.lock);
+		return -ENOENT;
+	}
+
+	ctx.cmd.rx_filter_del.filter_id = cpu_to_le32(f->filter_id);
+	ionic_rx_filter_free(lif, f);
+	spin_unlock_bh(&lif->rx_filters.lock);
+
+	err = ionic_adminq_post_wait(lif, &ctx);
+	if (err)
+		return err;
+
+	netdev_dbg(lif->netdev, "rx_filter del ADDR %pM (id %d)\n", addr,
+		   ctx.cmd.rx_filter_del.filter_id);
+
+	return 0;
+}
+
+static int ionic_lif_addr(struct lif *lif, const u8 *addr, bool add)
+{
+	struct ionic *ionic = lif->ionic;
+	struct ionic_deferred_work *work;
+	unsigned int nmfilters;
+	unsigned int nufilters;
+
+	if (add) {
+		/* Do we have space for this filter?  We test the counters
+		 * here before checking the need for deferral so that we
+		 * can return an overflow error to the stack.
+		 */
+		nmfilters = le32_to_cpu(ionic->ident.lif.eth.max_mcast_filters);
+		nufilters = le32_to_cpu(ionic->ident.lif.eth.max_ucast_filters);
+
+		if ((is_multicast_ether_addr(addr) && lif->nmcast < nmfilters))
+			lif->nmcast++;
+		else if (!is_multicast_ether_addr(addr) &&
+			 lif->nucast < nufilters)
+			lif->nucast++;
+		else
+			return -ENOSPC;
+	} else {
+		if (is_multicast_ether_addr(addr) && lif->nmcast)
+			lif->nmcast--;
+		else if (!is_multicast_ether_addr(addr) && lif->nucast)
+			lif->nucast--;
+	}
+
+	if (in_interrupt()) {
+		work = kzalloc(sizeof(*work), GFP_ATOMIC);
+		if (!work) {
+			netdev_err(lif->netdev, "%s OOM\n", __func__);
+			return -ENOMEM;
+		}
+		work->type = add ? DW_TYPE_RX_ADDR_ADD : DW_TYPE_RX_ADDR_DEL;
+		memcpy(work->addr, addr, ETH_ALEN);
+		netdev_dbg(lif->netdev, "deferred: rx_filter %s %pM\n",
+			   add ? "add" : "del", addr);
+		ionic_lif_deferred_enqueue(&lif->deferred, work);
+	} else {
+		netdev_dbg(lif->netdev, "rx_filter %s %pM\n",
+			   add ? "add" : "del", addr);
+		if (add)
+			return ionic_lif_addr_add(lif, addr);
+		else
+			return ionic_lif_addr_del(lif, addr);
+	}
+
+	return 0;
+}
+
+static int ionic_addr_add(struct net_device *netdev, const u8 *addr)
+{
+	return ionic_lif_addr(netdev_priv(netdev), addr, true);
+}
+
+static int ionic_addr_del(struct net_device *netdev, const u8 *addr)
+{
+	return ionic_lif_addr(netdev_priv(netdev), addr, false);
+}
+
+static void ionic_lif_rx_mode(struct lif *lif, unsigned int rx_mode)
+{
+	struct ionic_admin_ctx ctx = {
+		.work = COMPLETION_INITIALIZER_ONSTACK(ctx.work),
+		.cmd.rx_mode_set = {
+			.opcode = CMD_OPCODE_RX_MODE_SET,
+			.lif_index = cpu_to_le16(lif->index),
+			.rx_mode = cpu_to_le16(rx_mode),
+		},
+	};
+	char buf[128];
+	int err;
+	int i;
+#define REMAIN(__x) (sizeof(buf) - (__x))
+
+	i = snprintf(buf, sizeof(buf), "rx_mode 0x%04x -> 0x%04x:",
+		     lif->rx_mode, rx_mode);
+	if (rx_mode & RX_MODE_F_UNICAST)
+		i += snprintf(&buf[i], REMAIN(i), " RX_MODE_F_UNICAST");
+	if (rx_mode & RX_MODE_F_MULTICAST)
+		i += snprintf(&buf[i], REMAIN(i), " RX_MODE_F_MULTICAST");
+	if (rx_mode & RX_MODE_F_BROADCAST)
+		i += snprintf(&buf[i], REMAIN(i), " RX_MODE_F_BROADCAST");
+	if (rx_mode & RX_MODE_F_PROMISC)
+		i += snprintf(&buf[i], REMAIN(i), " RX_MODE_F_PROMISC");
+	if (rx_mode & RX_MODE_F_ALLMULTI)
+		i += snprintf(&buf[i], REMAIN(i), " RX_MODE_F_ALLMULTI");
+	netdev_dbg(lif->netdev, "lif%d %s\n", lif->index, buf);
+
+	err = ionic_adminq_post_wait(lif, &ctx);
+	if (err)
+		netdev_warn(lif->netdev, "set rx_mode 0x%04x failed: %d\n",
+			    rx_mode, err);
+	else
+		lif->rx_mode = rx_mode;
+}
+
+static void _ionic_lif_rx_mode(struct lif *lif, unsigned int rx_mode)
+{
+	struct ionic_deferred_work *work;
+
+	if (in_interrupt()) {
+		work = kzalloc(sizeof(*work), GFP_ATOMIC);
+		if (!work) {
+			netdev_err(lif->netdev, "%s OOM\n", __func__);
+			return;
+		}
+		work->type = DW_TYPE_RX_MODE;
+		work->rx_mode = rx_mode;
+		netdev_dbg(lif->netdev, "deferred: rx_mode\n");
+		ionic_lif_deferred_enqueue(&lif->deferred, work);
+	} else {
+		ionic_lif_rx_mode(lif, rx_mode);
+	}
+}
+
+static void ionic_set_rx_mode(struct net_device *netdev)
+{
+	struct lif *lif = netdev_priv(netdev);
+	struct identity *ident = &lif->ionic->ident;
+	unsigned int nfilters;
+	unsigned int rx_mode;
+
+	rx_mode = RX_MODE_F_UNICAST;
+	rx_mode |= (netdev->flags & IFF_MULTICAST) ? RX_MODE_F_MULTICAST : 0;
+	rx_mode |= (netdev->flags & IFF_BROADCAST) ? RX_MODE_F_BROADCAST : 0;
+	rx_mode |= (netdev->flags & IFF_PROMISC) ? RX_MODE_F_PROMISC : 0;
+	rx_mode |= (netdev->flags & IFF_ALLMULTI) ? RX_MODE_F_ALLMULTI : 0;
+
+	/* sync unicast addresses
+	 * next check to see if we're in an overflow state
+	 *    if so, we track that we overflowed and enable NIC PROMISC
+	 *    else if the overflow is set and not needed
+	 *       we remove our overflow flag and check the netdev flags
+	 *       to see if we can disable NIC PROMISC
+	 */
+	__dev_uc_sync(netdev, ionic_addr_add, ionic_addr_del);
+	nfilters = le32_to_cpu(ident->lif.eth.max_ucast_filters);
+	if (netdev_uc_count(netdev) + 1 > nfilters) {
+		rx_mode |= RX_MODE_F_PROMISC;
+		lif->uc_overflow = true;
+	} else if (lif->uc_overflow) {
+		lif->uc_overflow = false;
+		if (!(netdev->flags & IFF_PROMISC))
+			rx_mode &= ~RX_MODE_F_PROMISC;
+	}
+
+	/* same for multicast */
+	__dev_mc_sync(netdev, ionic_addr_add, ionic_addr_del);
+	nfilters = le32_to_cpu(ident->lif.eth.max_mcast_filters);
+	if (netdev_mc_count(netdev) > nfilters) {
+		rx_mode |= RX_MODE_F_ALLMULTI;
+		lif->mc_overflow = true;
+	} else if (lif->mc_overflow) {
+		lif->mc_overflow = false;
+		if (!(netdev->flags & IFF_ALLMULTI))
+			rx_mode &= ~RX_MODE_F_ALLMULTI;
+	}
+
+	if (lif->rx_mode != rx_mode)
+		_ionic_lif_rx_mode(lif, rx_mode);
+}
+
 static int ionic_set_features(struct net_device *netdev,
 			      netdev_features_t features)
 {
@@ -196,8 +471,26 @@ static int ionic_set_features(struct net_device *netdev,
 
 static int ionic_set_mac_address(struct net_device *netdev, void *sa)
 {
-	netdev_info(netdev, "%s: stubbed\n", __func__);
-	return 0;
+	struct sockaddr *addr = sa;
+	u8 *mac;
+
+	mac = (u8 *)addr->sa_data;
+	if (ether_addr_equal(netdev->dev_addr, mac))
+		return 0;
+
+	if (!is_valid_ether_addr(mac))
+		return -EADDRNOTAVAIL;
+
+	if (!is_zero_ether_addr(netdev->dev_addr)) {
+		netdev_info(netdev, "deleting mac addr %pM\n",
+			    netdev->dev_addr);
+		ionic_addr_del(netdev, netdev->dev_addr);
+	}
+
+	memcpy(netdev->dev_addr, mac, netdev->addr_len);
+	netdev_info(netdev, "updating mac addr %pM\n", mac);
+
+	return ionic_addr_add(netdev, mac);
 }
 
 static int ionic_change_mtu(struct net_device *netdev, int new_mtu)
@@ -232,20 +525,63 @@ static void ionic_tx_timeout(struct net_device *netdev)
 static int ionic_vlan_rx_add_vid(struct net_device *netdev, __be16 proto,
 				 u16 vid)
 {
-	netdev_info(netdev, "%s: stubbed\n", __func__);
-	return 0;
+	struct lif *lif = netdev_priv(netdev);
+	struct ionic_admin_ctx ctx = {
+		.work = COMPLETION_INITIALIZER_ONSTACK(ctx.work),
+		.cmd.rx_filter_add = {
+			.opcode = CMD_OPCODE_RX_FILTER_ADD,
+			.lif_index = cpu_to_le16(lif->index),
+			.match = cpu_to_le16(RX_FILTER_MATCH_VLAN),
+			.vlan.vlan = cpu_to_le16(vid),
+		},
+	};
+	int err;
+
+	err = ionic_adminq_post_wait(lif, &ctx);
+	if (err)
+		return err;
+
+	netdev_dbg(netdev, "rx_filter add VLAN %d (id %d)\n", vid,
+		   ctx.comp.rx_filter_add.filter_id);
+
+	return ionic_rx_filter_save(lif, 0, RXQ_INDEX_ANY, 0, &ctx);
 }
 
 static int ionic_vlan_rx_kill_vid(struct net_device *netdev, __be16 proto,
 				  u16 vid)
 {
-	netdev_info(netdev, "%s: stubbed\n", __func__);
-	return 0;
+	struct lif *lif = netdev_priv(netdev);
+	struct ionic_admin_ctx ctx = {
+		.work = COMPLETION_INITIALIZER_ONSTACK(ctx.work),
+		.cmd.rx_filter_del = {
+			.opcode = CMD_OPCODE_RX_FILTER_DEL,
+			.lif_index = cpu_to_le16(lif->index),
+		},
+	};
+	struct rx_filter *f;
+
+	spin_lock_bh(&lif->rx_filters.lock);
+
+	f = ionic_rx_filter_by_vlan(lif, vid);
+	if (!f) {
+		spin_unlock_bh(&lif->rx_filters.lock);
+		return -ENOENT;
+	}
+
+	netdev_dbg(netdev, "rx_filter del VLAN %d (id %d)\n", vid,
+		   le32_to_cpu(ctx.cmd.rx_filter_del.filter_id));
+
+	ctx.cmd.rx_filter_del.filter_id = cpu_to_le32(f->filter_id);
+	ionic_rx_filter_free(lif, f);
+	spin_unlock_bh(&lif->rx_filters.lock);
+
+	return ionic_adminq_post_wait(lif, &ctx);
 }
 
 static const struct net_device_ops ionic_netdev_ops = {
 	.ndo_open               = ionic_open,
 	.ndo_stop               = ionic_stop,
+	.ndo_set_rx_mode	= ionic_set_rx_mode,
 	.ndo_set_features	= ionic_set_features,
 	.ndo_set_mac_address	= ionic_set_mac_address,
 	.ndo_validate_addr	= eth_validate_addr,
@@ -546,6 +882,10 @@ static struct lif *ionic_lif_alloc(struct ionic *ionic, unsigned int index)
 
 	spin_lock_init(&lif->adminq_lock);
 
+	spin_lock_init(&lif->deferred.lock);
+	INIT_LIST_HEAD(&lif->deferred.list);
+	INIT_WORK(&lif->deferred.work, ionic_lif_deferred_work);
+
 	/* allocate lif info */
 	lif->info_sz = ALIGN(sizeof(*lif->info), PAGE_SIZE);
 	lif->info = dma_alloc_coherent(dev, lif->info_sz,
@@ -607,6 +947,8 @@ static void ionic_lif_free(struct lif *lif)
 	ionic_qcqs_free(lif);
 	ionic_lif_reset(lif);
 
+	cancel_work_sync(&lif->deferred.work);
+
 	/* free lif info */
 	dma_free_coherent(dev, lif->info_sz, lif->info, lif->info_pa);
 	lif->info = NULL;
@@ -975,6 +1317,37 @@ static int ionic_init_nic_features(struct lif *lif)
 	return 0;
 }
 
+static int ionic_station_set(struct lif *lif)
+{
+	struct net_device *netdev = lif->netdev;
+	struct ionic_admin_ctx ctx = {
+		.work = COMPLETION_INITIALIZER_ONSTACK(ctx.work),
+		.cmd.lif_getattr = {
+			.opcode = CMD_OPCODE_LIF_GETATTR,
+			.index = cpu_to_le16(lif->index),
+			.attr = IONIC_LIF_ATTR_MAC,
+		},
+	};
+	int err;
+
+	err = ionic_adminq_post_wait(lif, &ctx);
+	if (err)
+		return err;
+
+	if (!is_zero_ether_addr(netdev->dev_addr)) {
+		netdev_dbg(lif->netdev, "deleting station MAC addr %pM\n",
+			   netdev->dev_addr);
+		ionic_lif_addr(lif, netdev->dev_addr, false);
+	}
+	memcpy(netdev->dev_addr, ctx.comp.lif_getattr.mac,
+	       netdev->addr_len);
+	netdev_dbg(lif->netdev, "adding station MAC addr %pM\n",
+		   netdev->dev_addr);
+	ionic_lif_addr(lif, netdev->dev_addr, true);
+
+	return 0;
+}
+
 static int ionic_lif_init(struct lif *lif)
 {
 	struct ionic_dev *idev = &lif->ionic->idev;
@@ -1039,6 +1412,10 @@ static int ionic_lif_init(struct lif *lif)
 	if (err)
 		goto err_out_notifyq_deinit;
 
+	err = ionic_station_set(lif);
+	if (err)
+		goto err_out_notifyq_deinit;
+
 	set_bit(LIF_INITED, lif->state);
 
 	return 0;
drivers/net/ethernet/pensando/ionic/ionic_lif.h +29 -0
index 9f112aa69033..20b4fa573f77 100644
@@ -60,6 +60,29 @@ struct qcq {
 #define napi_to_qcq(napi)	container_of(napi, struct qcq, napi)
 #define napi_to_cq(napi)	(&napi_to_qcq(napi)->cq)
 
+enum ionic_deferred_work_type {
+	DW_TYPE_RX_MODE,
+	DW_TYPE_RX_ADDR_ADD,
+	DW_TYPE_RX_ADDR_DEL,
+	DW_TYPE_LINK_STATUS,
+	DW_TYPE_LIF_RESET,
+};
+
+struct ionic_deferred_work {
+	struct list_head list;
+	enum ionic_deferred_work_type type;
+	union {
+		unsigned int rx_mode;
+		u8 addr[ETH_ALEN];
+	};
+};
+
+struct ionic_deferred {
+	spinlock_t lock;		/* lock for deferred work list */
+	struct list_head list;
+	struct work_struct work;
+};
+
 enum lif_state_flags {
 	LIF_INITED,
 	LIF_UP,
@@ -87,13 +110,19 @@ struct lif {
 	u64 last_eid;
 	unsigned int neqs;
 	unsigned int nxqs;
+	unsigned int rx_mode;
 	u64 hw_features;
+	bool mc_overflow;
+	unsigned int nmcast;
+	bool uc_overflow;
+	unsigned int nucast;
 
 	struct lif_info *info;
 	dma_addr_t info_pa;
 	u32 info_sz;
 
 	struct rx_filters rx_filters;
+	struct ionic_deferred deferred;
 	unsigned long *dbid_inuse;
 	unsigned int dbid_count;
 	struct dentry *dentry;
Signature · 1 line
2.17.1
#12 lore raw

PATCH v3 12/19 ionic: Add async link status check and basic stats

In reply to Shannon Nelson

Add code to handle the link status event, and wire up the basic netdev hardware stats.

---
 .../net/ethernet/pensando/ionic/ionic_lif.c   | 116 ++++++++++++++++++
 .../net/ethernet/pensando/ionic/ionic_lif.h   |   1 +
 2 files changed, 117 insertions(+)
drivers/net/ethernet/pensando/ionic/ionic_lif.c +116 -0
index efcda1337f91..f52af9cb6264 100644
@@ -15,6 +15,7 @@
 static void ionic_lif_rx_mode(struct lif *lif, unsigned int rx_mode);
 static int ionic_lif_addr_add(struct lif *lif, const u8 *addr);
 static int ionic_lif_addr_del(struct lif *lif, const u8 *addr);
+static void ionic_link_status_check(struct lif *lif);
 
 static int ionic_set_nic_features(struct lif *lif, netdev_features_t features);
 static int ionic_notifyq_clean(struct lif *lif, int budget);
@@ -44,6 +45,9 @@ static void ionic_lif_deferred_work(struct work_struct *work)
 		case DW_TYPE_RX_ADDR_DEL:
 			ionic_lif_addr_del(lif, w->addr);
 			break;
+		case DW_TYPE_LINK_STATUS:
+			ionic_link_status_check(lif);
+			break;
 		default:
 			break;
 		}
@@ -69,6 +73,7 @@ int ionic_open(struct net_device *netdev)
 
 	set_bit(LIF_UP, lif->state);
 
+	ionic_link_status_check(lif);
 	if (netif_carrier_ok(netdev))
 		netif_tx_wake_all_queues(netdev);
 
@@ -151,6 +156,39 @@ static int ionic_adminq_napi(struct napi_struct *napi, int budget)
 	return max(n_work, a_work);
 }
 
+static void ionic_link_status_check(struct lif *lif)
+{
+	struct net_device *netdev = lif->netdev;
+	u16 link_status;
+	bool link_up;
+
+	clear_bit(LIF_LINK_CHECK_NEEDED, lif->state);
+
+	link_status = le16_to_cpu(lif->info->status.link_status);
+	link_up = link_status == PORT_OPER_STATUS_UP;
+
+	/* filter out the no-change cases */
+	if (link_up == netif_carrier_ok(netdev))
+		return;
+
+	if (link_up) {
+		netdev_info(netdev, "Link up - %d Gbps\n",
+			    le32_to_cpu(lif->info->status.link_speed) / 1000);
+
+		if (test_bit(LIF_UP, lif->state)) {
+			netif_tx_wake_all_queues(lif->netdev);
+			netif_carrier_on(netdev);
+		}
+	} else {
+		netdev_info(netdev, "Link down\n");
+
+		/* carrier off first to avoid watchdog timeout */
+		netif_carrier_off(netdev);
+		if (test_bit(LIF_UP, lif->state))
+			netif_tx_stop_all_queues(netdev);
+	}
+}
+
 static bool ionic_notifyq_service(struct cq *cq, struct cq_info *cq_info)
 {
 	union notifyq_comp *comp = cq_info->cq_desc;
@@ -182,6 +220,9 @@ static bool ionic_notifyq_service(struct cq *cq, struct cq_info *cq_info)
 			    "  link_status=%d link_speed=%d\n",
 			    le16_to_cpu(comp->link_change.link_status),
 			    le32_to_cpu(comp->link_change.link_speed));
+
+		set_bit(LIF_LINK_CHECK_NEEDED, lif->state);
+
 		break;
 	case EVENT_OPCODE_RESET:
 		netdev_info(netdev, "Notifyq EVENT_OPCODE_RESET eid=%lld\n",
@@ -222,10 +263,81 @@ static int ionic_notifyq_clean(struct lif *lif, int budget)
 	if (work_done == budget)
 		goto return_to_napi;
 
+	/* After outstanding events are processed we can check on
+	 * the link status and any outstanding interrupt credits.
+	 *
+	 * We wait until here to check on the link status in case
+	 * there was a long list of link events from a flap episode.
+	 */
+	if (test_bit(LIF_LINK_CHECK_NEEDED, lif->state)) {
+		struct ionic_deferred_work *work;
+
+		work = kzalloc(sizeof(*work), GFP_ATOMIC);
+		if (!work) {
+			netdev_err(lif->netdev, "%s OOM\n", __func__);
+		} else {
+			work->type = DW_TYPE_LINK_STATUS;
+			ionic_lif_deferred_enqueue(&lif->deferred, work);
+		}
+	}
+
 return_to_napi:
 	return work_done;
 }
 
+static void ionic_get_stats64(struct net_device *netdev,
+			      struct rtnl_link_stats64 *ns)
+{
+	struct lif *lif = netdev_priv(netdev);
+	struct lif_stats *ls;
+
+	memset(ns, 0, sizeof(*ns));
+	ls = &lif->info->stats;
+
+	ns->rx_packets = le64_to_cpu(ls->rx_ucast_packets) +
+			 le64_to_cpu(ls->rx_mcast_packets) +
+			 le64_to_cpu(ls->rx_bcast_packets);
+
+	ns->tx_packets = le64_to_cpu(ls->tx_ucast_packets) +
+			 le64_to_cpu(ls->tx_mcast_packets) +
+			 le64_to_cpu(ls->tx_bcast_packets);
+
+	ns->rx_bytes = le64_to_cpu(ls->rx_ucast_bytes) +
+		       le64_to_cpu(ls->rx_mcast_bytes) +
+		       le64_to_cpu(ls->rx_bcast_bytes);
+
+	ns->tx_bytes = le64_to_cpu(ls->tx_ucast_bytes) +
+		       le64_to_cpu(ls->tx_mcast_bytes) +
+		       le64_to_cpu(ls->tx_bcast_bytes);
+
+	ns->rx_dropped = le64_to_cpu(ls->rx_ucast_drop_packets) +
+			 le64_to_cpu(ls->rx_mcast_drop_packets) +
+			 le64_to_cpu(ls->rx_bcast_drop_packets);
+
+	ns->tx_dropped = le64_to_cpu(ls->tx_ucast_drop_packets) +
+			 le64_to_cpu(ls->tx_mcast_drop_packets) +
+			 le64_to_cpu(ls->tx_bcast_drop_packets);
+
+	ns->multicast = le64_to_cpu(ls->rx_mcast_packets);
+
+	ns->rx_over_errors = le64_to_cpu(ls->rx_queue_empty);
+
+	ns->rx_missed_errors = le64_to_cpu(ls->rx_dma_error) +
+			       le64_to_cpu(ls->rx_queue_disabled) +
+			       le64_to_cpu(ls->rx_desc_fetch_error) +
+			       le64_to_cpu(ls->rx_desc_data_error);
+
+	ns->tx_aborted_errors = le64_to_cpu(ls->tx_dma_error) +
+				le64_to_cpu(ls->tx_queue_disabled) +
+				le64_to_cpu(ls->tx_desc_fetch_error) +
+				le64_to_cpu(ls->tx_desc_data_error);
+
+	ns->rx_errors = ns->rx_over_errors +
+			ns->rx_missed_errors;
+
+	ns->tx_errors = ns->tx_aborted_errors;
+}
+
 static int ionic_lif_addr_add(struct lif *lif, const u8 *addr)
 {
 	struct ionic_admin_ctx ctx = {
@@ -581,6 +693,7 @@ static int ionic_vlan_rx_kill_vid(struct net_device *netdev, __be16 proto,
 static const struct net_device_ops ionic_netdev_ops = {
 	.ndo_open               = ionic_open,
 	.ndo_stop               = ionic_stop,
+	.ndo_get_stats64	= ionic_get_stats64,
 	.ndo_set_rx_mode	= ionic_set_rx_mode,
 	.ndo_set_features	= ionic_set_features,
 	.ndo_set_mac_address	= ionic_set_mac_address,
@@ -1418,6 +1531,8 @@ static int ionic_lif_init(struct lif *lif)
 
 	set_bit(LIF_INITED, lif->state);
 
+	ionic_link_status_check(lif);
+
 	return 0;
 
 err_out_notifyq_deinit:
@@ -1461,6 +1576,7 @@ int ionic_lifs_register(struct ionic *ionic)
 		return err;
 	}
 
+	ionic_link_status_check(ionic->master_lif);
 	ionic->master_lif->registered = true;
 
 	return 0;
drivers/net/ethernet/pensando/ionic/ionic_lif.h +1 -0
index 20b4fa573f77..9930b9390c8a 100644
@@ -86,6 +86,7 @@ struct ionic_deferred {
 enum lif_state_flags {
 	LIF_INITED,
 	LIF_UP,
+	LIF_LINK_CHECK_NEEDED,
 	LIF_QUEUE_RESET,
 
 	/* leave this as last */
Signature · 1 line
2.17.1
#13 lore raw

PATCH v3 13/19 ionic: Add initial ethtool support

In reply to Shannon Nelson

Add in the basic ethtool callbacks for device information and control.

---
 drivers/net/ethernet/pensando/ionic/Makefile  |   2 +-
 .../net/ethernet/pensando/ionic/ionic_dev.h   |   3 +
 .../ethernet/pensando/ionic/ionic_ethtool.c   | 509 ++++++++++++++++++
 .../ethernet/pensando/ionic/ionic_ethtool.h   |   9 +
 .../net/ethernet/pensando/ionic/ionic_lif.c   |   2 +
 .../net/ethernet/pensando/ionic/ionic_lif.h   |   8 +
 6 files changed, 532 insertions(+), 1 deletion(-)
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_ethtool.c
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_ethtool.h
drivers/net/ethernet/pensando/ionic/Makefile +1 -1
index 7d9cdc5f02a1..9b19bf57a489 100644
@@ -3,5 +3,5 @@
 
 obj-$(CONFIG_IONIC) := ionic.o
 
-ionic-y := ionic_main.o ionic_bus_pci.o ionic_dev.o \
+ionic-y := ionic_main.o ionic_bus_pci.o ionic_dev.o ionic_ethtool.o \
 	   ionic_lif.o ionic_rx_filter.o ionic_debugfs.o
drivers/net/ethernet/pensando/ionic/ionic_dev.h +3 -0
index d44220c1d430..a95af4c4dbf0 100644
@@ -12,6 +12,9 @@
 
 #define IONIC_MIN_MTU			ETH_MIN_MTU
 #define IONIC_MAX_MTU			9194
+#define IONIC_MAX_TXRX_DESC		16384
+#define IONIC_MIN_TXRX_DESC		16
+#define IONIC_DEF_TXRX_DESC		4096
 #define IONIC_LIFS_MAX			1024
 
 struct ionic_dev_bar {
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +509 -0

509 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_ethtool.h new file +9 -0
new file mode 100644
index 000000000000..38b91b1d70ae
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+
+#ifndef _IONIC_ETHTOOL_H_
+#define _IONIC_ETHTOOL_H_
+
+void ionic_ethtool_set_ops(struct net_device *netdev);
+
+#endif /* _IONIC_ETHTOOL_H_ */
drivers/net/ethernet/pensando/ionic/ionic_lif.c +2 -0
index f52af9cb6264..2bd8ce61c4a0 100644
@@ -10,6 +10,7 @@
 #include "ionic.h"
 #include "ionic_bus.h"
 #include "ionic_lif.h"
+#include "ionic_ethtool.h"
 #include "ionic_debugfs.h"
 
 static void ionic_lif_rx_mode(struct lif *lif, unsigned int rx_mode);
@@ -980,6 +981,7 @@ static struct lif *ionic_lif_alloc(struct ionic *ionic, unsigned int index)
 	lif->netdev = netdev;
 	ionic->master_lif = lif;
 	netdev->netdev_ops = &ionic_netdev_ops;
+	ionic_ethtool_set_ops(netdev);
 
 	netdev->watchdog_timeo = 2 * HZ;
 	netdev->min_mtu = IONIC_MIN_MTU;
drivers/net/ethernet/pensando/ionic/ionic_lif.h +8 -0
index 9930b9390c8a..d8589a306aa5 100644
@@ -111,6 +111,8 @@ struct lif {
 	u64 last_eid;
 	unsigned int neqs;
 	unsigned int nxqs;
+	unsigned int ntxq_descs;
+	unsigned int nrxq_descs;
 	unsigned int rx_mode;
 	u64 hw_features;
 	bool mc_overflow;
@@ -124,6 +126,8 @@ struct lif {
 
 	struct rx_filters rx_filters;
 	struct ionic_deferred deferred;
+	u32 tx_coalesce_usecs;
+	u32 rx_coalesce_usecs;
 	unsigned long *dbid_inuse;
 	unsigned int dbid_count;
 	struct dentry *dentry;
@@ -165,6 +169,10 @@ int ionic_lif_identify(struct ionic *ionic, u8 lif_type,
 		       union lif_identity *lif_ident);
 int ionic_lifs_size(struct ionic *ionic);
 
+int ionic_open(struct net_device *netdev);
+int ionic_stop(struct net_device *netdev);
+int ionic_reset_queues(struct lif *lif);
+
 static inline void debug_stats_napi_poll(struct qcq *qcq,
 					 unsigned int work_done)
 {
Signature · 1 line
2.17.1
#14 lore raw

Re: [PATCH v3 net-next 13/19] ionic: Add initial ethtool support

Quoted 70 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +70 -0
+static int ionic_get_link_ksettings(struct net_device *netdev,
+				    struct ethtool_link_ksettings *ks)
+{
+	struct lif *lif = netdev_priv(netdev);
+	struct ionic_dev *idev = &lif->ionic->idev;
+	int copper_seen = 0;
+
+	ethtool_link_ksettings_zero_link_mode(ks, supported);
+	ethtool_link_ksettings_zero_link_mode(ks, advertising);
+
+	switch (le16_to_cpu(idev->port_info->status.xcvr.pid)) {
+		/* Copper */
+	case XCVR_PID_QSFP_100G_CR4:
+		ethtool_link_ksettings_add_link_mode(ks, supported,
+						     100000baseCR4_Full);
+		copper_seen++;
+		break;
+	case XCVR_PID_QSFP_40GBASE_CR4:
+		ethtool_link_ksettings_add_link_mode(ks, supported,
+						     40000baseCR4_Full);
+		copper_seen++;
+		break;
+	case XCVR_PID_SFP_25GBASE_CR_S:
+	case XCVR_PID_SFP_25GBASE_CR_L:
+	case XCVR_PID_SFP_25GBASE_CR_N:
+		ethtool_link_ksettings_add_link_mode(ks, supported,
+						     25000baseCR_Full);
+		copper_seen++;
+		break;
+	case XCVR_PID_SFP_10GBASE_AOC:
+	case XCVR_PID_SFP_10GBASE_CU:
+		ethtool_link_ksettings_add_link_mode(ks, supported,
+						     10000baseCR_Full);
+		copper_seen++;
+		break;
+
+		/* Fibre */
+	case XCVR_PID_QSFP_100G_SR4:
+	case XCVR_PID_QSFP_100G_AOC:
+		ethtool_link_ksettings_add_link_mode(ks, supported,
+						     100000baseSR4_Full);
+		break;
+	case XCVR_PID_QSFP_100G_LR4:
+		ethtool_link_ksettings_add_link_mode(ks, supported,
+						     100000baseLR4_ER4_Full);
+		break;
+	case XCVR_PID_QSFP_100G_ER4:
+		ethtool_link_ksettings_add_link_mode(ks, supported,
+						     100000baseLR4_ER4_Full);
+		break;
+	case XCVR_PID_QSFP_40GBASE_SR4:
+	case XCVR_PID_QSFP_40GBASE_AOC:
+		ethtool_link_ksettings_add_link_mode(ks, supported,
+						     40000baseSR4_Full);
+		break;
+	case XCVR_PID_QSFP_40GBASE_LR4:
+		ethtool_link_ksettings_add_link_mode(ks, supported,
+						     40000baseLR4_Full);
+		break;
+	case XCVR_PID_SFP_25GBASE_SR:
+	case XCVR_PID_SFP_25GBASE_AOC:
+		ethtool_link_ksettings_add_link_mode(ks, supported,
+						     25000baseSR_Full);
+		break;
+	case XCVR_PID_SFP_10GBASE_SR:
+		ethtool_link_ksettings_add_link_mode(ks, supported,
+						     10000baseSR_Full);
+		break;
+	case XCVR_PID_SFP_10GBASE_LR:
+		ethtool_link_ksettings_add_link_mode(ks, supported,

drivers/net/ethernet/pensando/ionic/ionic_ethtool.c line 129

Quoted 10 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +10 -0
+						     10000baseLR_Full);
+		break;
+	case XCVR_PID_SFP_10GBASE_LRM:
+		ethtool_link_ksettings_add_link_mode(ks, supported,
+						     10000baseLRM_Full);
+		break;
+	case XCVR_PID_SFP_10GBASE_ER:
+		ethtool_link_ksettings_add_link_mode(ks, supported,
+						     10000baseER_Full);
+		break;

I don't know these link modes too well. But only setting a single bit seems odd. What i do know is that an SFP which supports 2500BaseX should also be able to support 1000BaseX. So should a 100G SFP also support 40G, 25G, 10G etc? The SERDES just runs a slower bitstream over the basic bitpipe?

drivers/net/ethernet/pensando/ionic/ionic_ethtool.c line 137

Quoted 8 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +8 -0
+	case XCVR_PID_QSFP_100G_ACC:
+	case XCVR_PID_QSFP_40GBASE_ER4:
+	case XCVR_PID_SFP_25GBASE_LR:
+	case XCVR_PID_SFP_25GBASE_ER:
+		dev_info(lif->ionic->dev, "no decode bits for xcvr type pid=%d / 0x%x\n",
+			 idev->port_info->status.xcvr.pid,
+			 idev->port_info->status.xcvr.pid);
+		break;

Why not add them?

drivers/net/ethernet/pensando/ionic/ionic_ethtool.c line 151

Quoted 2 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +2 -0
+	memcpy(ks->link_modes.advertising, ks->link_modes.supported,
+	       sizeof(ks->link_modes.advertising));

bitmap_copy() would be a better way to do this. You could consider adding a helper to ethtool.h.

Andrew

#15 lore raw

Re: [PATCH v3 net-next 13/19] ionic: Add initial ethtool support

Quoted 99 lines · Andrew Lunn
Quoted 70 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +70 -0
+static int ionic_get_link_ksettings(struct net_device *netdev,
+				    struct ethtool_link_ksettings *ks)
+{
+	struct lif *lif = netdev_priv(netdev);
+	struct ionic_dev *idev = &lif->ionic->idev;
+	int copper_seen = 0;
+
+	ethtool_link_ksettings_zero_link_mode(ks, supported);
+	ethtool_link_ksettings_zero_link_mode(ks, advertising);
+
+	switch (le16_to_cpu(idev->port_info->status.xcvr.pid)) {
+		/* Copper */
+	case XCVR_PID_QSFP_100G_CR4:
+		ethtool_link_ksettings_add_link_mode(ks, supported,
+						     100000baseCR4_Full);
+		copper_seen++;
+		break;
+	case XCVR_PID_QSFP_40GBASE_CR4:
+		ethtool_link_ksettings_add_link_mode(ks, supported,
+						     40000baseCR4_Full);
+		copper_seen++;
+		break;
+	case XCVR_PID_SFP_25GBASE_CR_S:
+	case XCVR_PID_SFP_25GBASE_CR_L:
+	case XCVR_PID_SFP_25GBASE_CR_N:
+		ethtool_link_ksettings_add_link_mode(ks, supported,
+						     25000baseCR_Full);
+		copper_seen++;
+		break;
+	case XCVR_PID_SFP_10GBASE_AOC:
+	case XCVR_PID_SFP_10GBASE_CU:
+		ethtool_link_ksettings_add_link_mode(ks, supported,
+						     10000baseCR_Full);
+		copper_seen++;
+		break;
+
+		/* Fibre */
+	case XCVR_PID_QSFP_100G_SR4:
+	case XCVR_PID_QSFP_100G_AOC:
+		ethtool_link_ksettings_add_link_mode(ks, supported,
+						     100000baseSR4_Full);
+		break;
+	case XCVR_PID_QSFP_100G_LR4:
+		ethtool_link_ksettings_add_link_mode(ks, supported,
+						     100000baseLR4_ER4_Full);
+		break;
+	case XCVR_PID_QSFP_100G_ER4:
+		ethtool_link_ksettings_add_link_mode(ks, supported,
+						     100000baseLR4_ER4_Full);
+		break;
+	case XCVR_PID_QSFP_40GBASE_SR4:
+	case XCVR_PID_QSFP_40GBASE_AOC:
+		ethtool_link_ksettings_add_link_mode(ks, supported,
+						     40000baseSR4_Full);
+		break;
+	case XCVR_PID_QSFP_40GBASE_LR4:
+		ethtool_link_ksettings_add_link_mode(ks, supported,
+						     40000baseLR4_Full);
+		break;
+	case XCVR_PID_SFP_25GBASE_SR:
+	case XCVR_PID_SFP_25GBASE_AOC:
+		ethtool_link_ksettings_add_link_mode(ks, supported,
+						     25000baseSR_Full);
+		break;
+	case XCVR_PID_SFP_10GBASE_SR:
+		ethtool_link_ksettings_add_link_mode(ks, supported,
+						     10000baseSR_Full);
+		break;
+	case XCVR_PID_SFP_10GBASE_LR:
+		ethtool_link_ksettings_add_link_mode(ks, supported,

drivers/net/ethernet/pensando/ionic/ionic_ethtool.c line 129

Quoted 10 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +10 -0
+						     10000baseLR_Full);
+		break;
+	case XCVR_PID_SFP_10GBASE_LRM:
+		ethtool_link_ksettings_add_link_mode(ks, supported,
+						     10000baseLRM_Full);
+		break;
+	case XCVR_PID_SFP_10GBASE_ER:
+		ethtool_link_ksettings_add_link_mode(ks, supported,
+						     10000baseER_Full);
+		break;

I don't know these link modes too well. But only setting a single bit seems odd. What i do know is that an SFP which supports 2500BaseX should also be able to support 1000BaseX. So should a 100G SFP also support 40G, 25G, 10G etc? The SERDES just runs a slower bitstream over the basic bitpipe?

drivers/net/ethernet/pensando/ionic/ionic_ethtool.c line 137

Quoted 8 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +8 -0
+	case XCVR_PID_QSFP_100G_ACC:
+	case XCVR_PID_QSFP_40GBASE_ER4:
+	case XCVR_PID_SFP_25GBASE_LR:
+	case XCVR_PID_SFP_25GBASE_ER:
+		dev_info(lif->ionic->dev, "no decode bits for xcvr type pid=%d / 0x%x\n",
+			 idev->port_info->status.xcvr.pid,
+			 idev->port_info->status.xcvr.pid);
+		break;

Why not add them?

Quoted 5 lines

drivers/net/ethernet/pensando/ionic/ionic_ethtool.c line 151

Quoted 2 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +2 -0
+	memcpy(ks->link_modes.advertising, ks->link_modes.supported,
+	       sizeof(ks->link_modes.advertising));

bitmap_copy() would be a better way to do this. You could consider adding a helper to ethtool.h.

Also, there is no need to zero initialize ks->link_modes.advertising above if it's going to be rewritten here anyway.

Michal

#16 lore raw

Re: [PATCH v3 net-next 13/19] ionic: Add initial ethtool support

Quoted 4 lines · Michal Kubecek

Also, there is no need to zero initialize ks->link_modes.advertising above if it's going to be rewritten here anyway.

Michal

Got it.

Thanks,
sln

#17 lore raw

Re: [PATCH v3 net-next 13/19] ionic: Add initial ethtool support

In reply to Andrew Lunn
Quoted 9 lines · Andrew Lunn

drivers/net/ethernet/pensando/ionic/ionic_ethtool.c line 129

Quoted 4 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +4 -0
+	case XCVR_PID_SFP_10GBASE_ER:
+		ethtool_link_ksettings_add_link_mode(ks, supported,
+						     10000baseER_Full);
+		break;

I don't know these link modes too well. But only setting a single bit seems odd. What i do know is that an SFP which supports 2500BaseX should also be able to support 1000BaseX. So should a 100G SFP also support 40G, 25G, 10G etc? The SERDES just runs a slower bitstream over the basic bitpipe?

Yes, but in this initial release we're not supporting changes to the modes yet.  That flexibility will come later.

Quoted 10 lines

drivers/net/ethernet/pensando/ionic/ionic_ethtool.c line 137

Quoted 9 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +8 -0
+	case XCVR_PID_QSFP_100G_ACC:
+	case XCVR_PID_QSFP_40GBASE_ER4:
+	case XCVR_PID_SFP_25GBASE_LR:
+	case XCVR_PID_SFP_25GBASE_ER:
+		dev_info(lif->ionic->dev, "no decode bits for xcvr type pid=%d / 0x%x\n",
+			 idev->port_info->status.xcvr.pid,
+			 idev->port_info->status.xcvr.pid);
+		break;

Why not add them?

Yes, this has been mentioned before.  I might in the future, but I have my hands full at the moment.

Quoted 6 lines

drivers/net/ethernet/pensando/ionic/ionic_ethtool.c line 151

Quoted 4 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +2 -0
+	memcpy(ks->link_modes.advertising, ks->link_modes.supported,
+	       sizeof(ks->link_modes.advertising));

bitmap_copy() would be a better way to do this. You could consider adding a helper to ethtool.h.

Sure.

Thanks for your comments, and sorry I haven't responded as quickly as I'd like... I'll be going through these and your other comments over the next few days.

sln

#18 lore raw

Re: [PATCH v3 net-next 13/19] ionic: Add initial ethtool support

In reply to Shannon Nelson
Quoted 26 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +26 -0
+static int ionic_set_pauseparam(struct net_device *netdev,
+				struct ethtool_pauseparam *pause)
+{
+	struct lif *lif = netdev_priv(netdev);
+	struct ionic *ionic = lif->ionic;
+	struct ionic_dev *idev = &lif->ionic->idev;
+
+	u32 requested_pause;
+	u32 cur_autoneg;
+	int err;
+
+	cur_autoneg = idev->port_info->config.an_enable ? AUTONEG_ENABLE :
+								AUTONEG_DISABLE;
+	if (pause->autoneg != cur_autoneg) {
+		netdev_info(netdev, "Please use 'ethtool -s ...' to change autoneg\n");
+		return -EOPNOTSUPP;
+	}
+
+	/* change both at the same time */
+	requested_pause = PORT_PAUSE_TYPE_LINK;
+	if (pause->rx_pause)
+		requested_pause |= IONIC_PAUSE_F_RX;
+	if (pause->tx_pause)
+		requested_pause |= IONIC_PAUSE_F_TX;
+
+	if (requested_pause == idev->port_info->config.pause_type)

drivers/net/ethernet/pensando/ionic/ionic_ethtool.c line 300

Quoted 10 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +10 -0
+		return 0;
+
+	idev->port_info->config.pause_type = requested_pause;
+
+	mutex_lock(&ionic->dev_cmd_lock);
+	ionic_dev_cmd_port_pause(idev, requested_pause);
+	err = ionic_dev_cmd_wait(ionic, devcmd_timeout);
+	mutex_unlock(&ionic->dev_cmd_lock);
+	if (err)
+		return err;

Hi Shannon

I've no idea what the firmware black box is doing, but this looks wrong.

pause->autoneg is about if the results of auto-neg should be used or not. If false, just configure the MAC with the pause settings and you are done. If the interface is being forced, so autoneg in general is disabled, just configure the MAC and you are done.

If pause->autoneg is true and the interface is using auto-neg as a whole, you pass the pause values to the PHY for it to advertise and trigger an auto-neg. Once autoneg has completed, and the resolved settings are available, the MAC is configured with the resolved values.

Looking at this code, i don't see any difference between configuring the MAC or configuring the PHY. I would expect pause->autoneg to be part of requested_pause somehow, so the firmware knows what is should do.

Andrew

#19 lore raw

Re: [PATCH v3 net-next 13/19] ionic: Add initial ethtool support

Quoted 52 lines · Andrew Lunn
Quoted 26 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +26 -0
+static int ionic_set_pauseparam(struct net_device *netdev,
+				struct ethtool_pauseparam *pause)
+{
+	struct lif *lif = netdev_priv(netdev);
+	struct ionic *ionic = lif->ionic;
+	struct ionic_dev *idev = &lif->ionic->idev;
+
+	u32 requested_pause;
+	u32 cur_autoneg;
+	int err;
+
+	cur_autoneg = idev->port_info->config.an_enable ? AUTONEG_ENABLE :
+								AUTONEG_DISABLE;
+	if (pause->autoneg != cur_autoneg) {
+		netdev_info(netdev, "Please use 'ethtool -s ...' to change autoneg\n");
+		return -EOPNOTSUPP;
+	}
+
+	/* change both at the same time */
+	requested_pause = PORT_PAUSE_TYPE_LINK;
+	if (pause->rx_pause)
+		requested_pause |= IONIC_PAUSE_F_RX;
+	if (pause->tx_pause)
+		requested_pause |= IONIC_PAUSE_F_TX;
+
+	if (requested_pause == idev->port_info->config.pause_type)

drivers/net/ethernet/pensando/ionic/ionic_ethtool.c line 300

Quoted 10 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +10 -0
+		return 0;
+
+	idev->port_info->config.pause_type = requested_pause;
+
+	mutex_lock(&ionic->dev_cmd_lock);
+	ionic_dev_cmd_port_pause(idev, requested_pause);
+	err = ionic_dev_cmd_wait(ionic, devcmd_timeout);
+	mutex_unlock(&ionic->dev_cmd_lock);
+	if (err)
+		return err;

Hi Shannon

I've no idea what the firmware black box is doing, but this looks wrong.

pause->autoneg is about if the results of auto-neg should be used or not. If false, just configure the MAC with the pause settings and you are done. If the interface is being forced, so autoneg in general is disabled, just configure the MAC and you are done.

If pause->autoneg is true and the interface is using auto-neg as a whole, you pass the pause values to the PHY for it to advertise and trigger an auto-neg. Once autoneg has completed, and the resolved settings are available, the MAC is configured with the resolved values.

Quoted 6 lines

Looking at this code, i don't see any difference between configuring the MAC or configuring the PHY. I would expect pause->autoneg to be part of requested_pause somehow, so the firmware knows what is should do.

Andrew

In this device there's actually very little the driver can do to directly configure the mac or phy besides passing through to the firmware what the user has requested - that happens here for the pause values, and in ionic_set_link_ksettings() for autoneg.  The firmware is managing the port based on these requests with the help of internally configured rules defined in a customer setting.

sln

#20 lore raw

Re: [PATCH v3 net-next 13/19] ionic: Add initial ethtool support

Quoted 58 lines · Shannon Nelson
Quoted 57 lines · Andrew Lunn
Quoted 26 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +26 -0
+static int ionic_set_pauseparam(struct net_device *netdev,
+				struct ethtool_pauseparam *pause)
+{
+	struct lif *lif = netdev_priv(netdev);
+	struct ionic *ionic = lif->ionic;
+	struct ionic_dev *idev = &lif->ionic->idev;
+
+	u32 requested_pause;
+	u32 cur_autoneg;
+	int err;
+
+	cur_autoneg = idev->port_info->config.an_enable ? AUTONEG_ENABLE :
+								AUTONEG_DISABLE;
+	if (pause->autoneg != cur_autoneg) {
+		netdev_info(netdev, "Please use 'ethtool -s ...' to change autoneg\n");
+		return -EOPNOTSUPP;
+	}
+
+	/* change both at the same time */
+	requested_pause = PORT_PAUSE_TYPE_LINK;
+	if (pause->rx_pause)
+		requested_pause |= IONIC_PAUSE_F_RX;
+	if (pause->tx_pause)
+		requested_pause |= IONIC_PAUSE_F_TX;
+
+	if (requested_pause == idev->port_info->config.pause_type)

drivers/net/ethernet/pensando/ionic/ionic_ethtool.c line 300

Quoted 10 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +10 -0
+		return 0;
+
+	idev->port_info->config.pause_type = requested_pause;
+
+	mutex_lock(&ionic->dev_cmd_lock);
+	ionic_dev_cmd_port_pause(idev, requested_pause);
+	err = ionic_dev_cmd_wait(ionic, devcmd_timeout);
+	mutex_unlock(&ionic->dev_cmd_lock);
+	if (err)
+		return err;

Hi Shannon

I've no idea what the firmware black box is doing, but this looks wrong.

pause->autoneg is about if the results of auto-neg should be used or not. If false, just configure the MAC with the pause settings and you are done. If the interface is being forced, so autoneg in general is disabled, just configure the MAC and you are done.

If pause->autoneg is true and the interface is using auto-neg as a whole, you pass the pause values to the PHY for it to advertise and trigger an auto-neg. Once autoneg has completed, and the resolved settings are available, the MAC is configured with the resolved values.

Looking at this code, i don't see any difference between configuring the MAC or configuring the PHY. I would expect pause->autoneg to be part of requested_pause somehow, so the firmware knows what is should do.

Quoted 8 lines
Quoted 1 line

Andrew

In this device there's actually very little the driver can do to directly configure the mac or phy besides passing through to the firmware what the user has requested - that happens here for the pause values, and in ionic_set_link_ksettings() for autoneg.  The firmware is managing the port based on these requests with the help of internally configured rules defined in a customer setting.

I get that. But the firmware needs to conform to what Linux expects. And what i see here does not conform. That is why i gave a bit of detail in my reply.

What exactly does the firmware do? Once we know that, we can figure out when the driver should return -EOPNOTSUPP because of firmware limitations, and what it can configure and should return 0.

Andrew

#21 lore raw

Re: [PATCH v3 net-next 13/19] ionic: Add initial ethtool support

Quoted 66 lines · Andrew Lunn
Quoted 58 lines · Shannon Nelson
Quoted 57 lines · Andrew Lunn
Quoted 26 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +26 -0
+static int ionic_set_pauseparam(struct net_device *netdev,
+				struct ethtool_pauseparam *pause)
+{
+	struct lif *lif = netdev_priv(netdev);
+	struct ionic *ionic = lif->ionic;
+	struct ionic_dev *idev = &lif->ionic->idev;
+
+	u32 requested_pause;
+	u32 cur_autoneg;
+	int err;
+
+	cur_autoneg = idev->port_info->config.an_enable ? AUTONEG_ENABLE :
+								AUTONEG_DISABLE;
+	if (pause->autoneg != cur_autoneg) {
+		netdev_info(netdev, "Please use 'ethtool -s ...' to change autoneg\n");
+		return -EOPNOTSUPP;
+	}
+
+	/* change both at the same time */
+	requested_pause = PORT_PAUSE_TYPE_LINK;
+	if (pause->rx_pause)
+		requested_pause |= IONIC_PAUSE_F_RX;
+	if (pause->tx_pause)
+		requested_pause |= IONIC_PAUSE_F_TX;
+
+	if (requested_pause == idev->port_info->config.pause_type)

drivers/net/ethernet/pensando/ionic/ionic_ethtool.c line 300

Quoted 10 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +10 -0
+		return 0;
+
+	idev->port_info->config.pause_type = requested_pause;
+
+	mutex_lock(&ionic->dev_cmd_lock);
+	ionic_dev_cmd_port_pause(idev, requested_pause);
+	err = ionic_dev_cmd_wait(ionic, devcmd_timeout);
+	mutex_unlock(&ionic->dev_cmd_lock);
+	if (err)
+		return err;

Hi Shannon

I've no idea what the firmware black box is doing, but this looks wrong.

pause->autoneg is about if the results of auto-neg should be used or not. If false, just configure the MAC with the pause settings and you are done. If the interface is being forced, so autoneg in general is disabled, just configure the MAC and you are done.

If pause->autoneg is true and the interface is using auto-neg as a whole, you pass the pause values to the PHY for it to advertise and trigger an auto-neg. Once autoneg has completed, and the resolved settings are available, the MAC is configured with the resolved values.

Looking at this code, i don't see any difference between configuring the MAC or configuring the PHY. I would expect pause->autoneg to be part of requested_pause somehow, so the firmware knows what is should do.

Quoted 7 lines
Quoted 1 line

Andrew

In this device there's actually very little the driver can do to directly configure the mac or phy besides passing through to the firmware what the user has requested - that happens here for the pause values, and in ionic_set_link_ksettings() for autoneg.  The firmware is managing the port based on these requests with the help of internally configured rules defined in a customer setting.

Quoted 7 lines

I get that. But the firmware needs to conform to what Linux expects. And what i see here does not conform. That is why i gave a bit of detail in my reply.

What exactly does the firmware do? Once we know that, we can figure out when the driver should return -EOPNOTSUPP because of firmware limitations, and what it can configure and should return 0.

Because this is fairly smart FW, it handles this as expected.  I can add this as another comment in the code.

sln

#22 lore raw

Re: [PATCH v3 net-next 13/19] ionic: Add initial ethtool support

Quoted 71 lines · Shannon Nelson
Quoted 70 lines · Andrew Lunn
Quoted 58 lines · Shannon Nelson
Quoted 57 lines · Andrew Lunn
Quoted 26 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +26 -0
+static int ionic_set_pauseparam(struct net_device *netdev,
+				struct ethtool_pauseparam *pause)
+{
+	struct lif *lif = netdev_priv(netdev);
+	struct ionic *ionic = lif->ionic;
+	struct ionic_dev *idev = &lif->ionic->idev;
+
+	u32 requested_pause;
+	u32 cur_autoneg;
+	int err;
+
+	cur_autoneg = idev->port_info->config.an_enable ? AUTONEG_ENABLE :
+								AUTONEG_DISABLE;
+	if (pause->autoneg != cur_autoneg) {
+		netdev_info(netdev, "Please use 'ethtool -s ...' to change autoneg\n");
+		return -EOPNOTSUPP;
+	}
+
+	/* change both at the same time */
+	requested_pause = PORT_PAUSE_TYPE_LINK;
+	if (pause->rx_pause)
+		requested_pause |= IONIC_PAUSE_F_RX;
+	if (pause->tx_pause)
+		requested_pause |= IONIC_PAUSE_F_TX;
+
+	if (requested_pause == idev->port_info->config.pause_type)

drivers/net/ethernet/pensando/ionic/ionic_ethtool.c line 300

Quoted 10 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +10 -0
+		return 0;
+
+	idev->port_info->config.pause_type = requested_pause;
+
+	mutex_lock(&ionic->dev_cmd_lock);
+	ionic_dev_cmd_port_pause(idev, requested_pause);
+	err = ionic_dev_cmd_wait(ionic, devcmd_timeout);
+	mutex_unlock(&ionic->dev_cmd_lock);
+	if (err)
+		return err;

Hi Shannon

I've no idea what the firmware black box is doing, but this looks wrong.

pause->autoneg is about if the results of auto-neg should be used or not. If false, just configure the MAC with the pause settings and you are done. If the interface is being forced, so autoneg in general is disabled, just configure the MAC and you are done.

If pause->autoneg is true and the interface is using auto-neg as a whole, you pass the pause values to the PHY for it to advertise and trigger an auto-neg. Once autoneg has completed, and the resolved settings are available, the MAC is configured with the resolved values.

Looking at this code, i don't see any difference between configuring the MAC or configuring the PHY. I would expect pause->autoneg to be part of requested_pause somehow, so the firmware knows what is should do.

Quoted 7 lines
Quoted 1 line

Andrew

In this device there's actually very little the driver can do to directly configure the mac or phy besides passing through to the firmware what the user has requested - that happens here for the pause values, and in ionic_set_link_ksettings() for autoneg.  The firmware is managing the port based on these requests with the help of internally configured rules defined in a customer setting.

I get that. But the firmware needs to conform to what Linux expects. And what i see here does not conform. That is why i gave a bit of detail in my reply.

Quoted 6 lines
Quoted 3 lines

What exactly does the firmware do? Once we know that, we can figure out when the driver should return -EOPNOTSUPP because of firmware limitations, and what it can configure and should return 0.

Because this is fairly smart FW, it handles this as expected.  I can add this as another comment in the code.

Hi Shannon

Looking at the code, i don't see how it can handle this correctly. Please look at my comments, particularly the meaning of pause->autoneg and describe how the firmware does the right thing, given what information it is passed.

Andrew

#23 lore raw

Re: [PATCH v3 net-next 13/19] ionic: Add initial ethtool support

Quoted 75 lines · Andrew Lunn
Quoted 74 lines · Shannon Nelson
Quoted 73 lines · Andrew Lunn
Quoted 58 lines · Shannon Nelson
Quoted 57 lines · Andrew Lunn
Quoted 26 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +26 -0
+static int ionic_set_pauseparam(struct net_device *netdev,
+				struct ethtool_pauseparam *pause)
+{
+	struct lif *lif = netdev_priv(netdev);
+	struct ionic *ionic = lif->ionic;
+	struct ionic_dev *idev = &lif->ionic->idev;
+
+	u32 requested_pause;
+	u32 cur_autoneg;
+	int err;
+
+	cur_autoneg = idev->port_info->config.an_enable ? AUTONEG_ENABLE :
+								AUTONEG_DISABLE;
+	if (pause->autoneg != cur_autoneg) {
+		netdev_info(netdev, "Please use 'ethtool -s ...' to change autoneg\n");
+		return -EOPNOTSUPP;
+	}
+
+	/* change both at the same time */
+	requested_pause = PORT_PAUSE_TYPE_LINK;
+	if (pause->rx_pause)
+		requested_pause |= IONIC_PAUSE_F_RX;
+	if (pause->tx_pause)
+		requested_pause |= IONIC_PAUSE_F_TX;
+
+	if (requested_pause == idev->port_info->config.pause_type)

drivers/net/ethernet/pensando/ionic/ionic_ethtool.c line 300

Quoted 10 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +10 -0
+		return 0;
+
+	idev->port_info->config.pause_type = requested_pause;
+
+	mutex_lock(&ionic->dev_cmd_lock);
+	ionic_dev_cmd_port_pause(idev, requested_pause);
+	err = ionic_dev_cmd_wait(ionic, devcmd_timeout);
+	mutex_unlock(&ionic->dev_cmd_lock);
+	if (err)
+		return err;

Hi Shannon

I've no idea what the firmware black box is doing, but this looks wrong.

pause->autoneg is about if the results of auto-neg should be used or not. If false, just configure the MAC with the pause settings and you are done. If the interface is being forced, so autoneg in general is disabled, just configure the MAC and you are done.

If pause->autoneg is true and the interface is using auto-neg as a whole, you pass the pause values to the PHY for it to advertise and trigger an auto-neg. Once autoneg has completed, and the resolved settings are available, the MAC is configured with the resolved values.

Looking at this code, i don't see any difference between configuring the MAC or configuring the PHY. I would expect pause->autoneg to be part of requested_pause somehow, so the firmware knows what is should do.

Quoted 7 lines
Quoted 1 line

Andrew

In this device there's actually very little the driver can do to directly configure the mac or phy besides passing through to the firmware what the user has requested - that happens here for the pause values, and in ionic_set_link_ksettings() for autoneg.  The firmware is managing the port based on these requests with the help of internally configured rules defined in a customer setting.

I get that. But the firmware needs to conform to what Linux expects. And what i see here does not conform. That is why i gave a bit of detail in my reply.

What exactly does the firmware do? Once we know that, we can figure out when the driver should return -EOPNOTSUPP because of firmware limitations, and what it can configure and should return 0.

Quoted 9 lines
Quoted 2 lines

Because this is fairly smart FW, it handles this as expected.  I can add this as another comment in the code.

Hi Shannon

Looking at the code, i don't see how it can handle this correctly. Please look at my comments, particularly the meaning of pause->autoneg and describe how the firmware does the right thing, given what information it is passed.

I guess I'm not sure how much better I can answer your question. Like several other devices, we don't support selecting pause->autoneg.  The firmware manages the PHY itself, the driver doesn't have direct access to the PHY.  The driver passes the tx and rx pause info down to the firmware in a command request.  The NIC firmware does an autoneg if autoneg is enabled on the port.

sln

#24 lore raw

Re: [PATCH v3 net-next 13/19] ionic: Add initial ethtool support

Quoted 86 lines · Shannon Nelson
Quoted 75 lines · Andrew Lunn
Quoted 74 lines · Shannon Nelson
Quoted 73 lines · Andrew Lunn
Quoted 58 lines · Shannon Nelson
Quoted 57 lines · Andrew Lunn
Quoted 26 lines
(quoted diff) +26 -0
+static int ionic_set_pauseparam(struct net_device *netdev,
+				struct ethtool_pauseparam *pause)
+{
+	struct lif *lif = netdev_priv(netdev);
+	struct ionic *ionic = lif->ionic;
+	struct ionic_dev *idev = &lif->ionic->idev;
+
+	u32 requested_pause;
+	u32 cur_autoneg;
+	int err;
+
+	cur_autoneg = idev->port_info->config.an_enable ? AUTONEG_ENABLE :
+								AUTONEG_DISABLE;
+	if (pause->autoneg != cur_autoneg) {
+		netdev_info(netdev, "Please use 'ethtool -s ...' to change autoneg\n");
+		return -EOPNOTSUPP;
+	}
+
+	/* change both at the same time */
+	requested_pause = PORT_PAUSE_TYPE_LINK;
+	if (pause->rx_pause)
+		requested_pause |= IONIC_PAUSE_F_RX;
+	if (pause->tx_pause)
+		requested_pause |= IONIC_PAUSE_F_TX;
+
+	if (requested_pause == idev->port_info->config.pause_type)

Review

Quoted 10 lines
(quoted diff) +10 -0
+		return 0;
+
+	idev->port_info->config.pause_type = requested_pause;
+
+	mutex_lock(&ionic->dev_cmd_lock);
+	ionic_dev_cmd_port_pause(idev, requested_pause);
+	err = ionic_dev_cmd_wait(ionic, devcmd_timeout);
+	mutex_unlock(&ionic->dev_cmd_lock);
+	if (err)
+		return err;

Hi Shannon

I've no idea what the firmware black box is doing, but this looks wrong.

pause->autoneg is about if the results of auto-neg should be used or not. If false, just configure the MAC with the pause settings and you are done. If the interface is being forced, so autoneg in general is disabled, just configure the MAC and you are done.

If pause->autoneg is true and the interface is using auto-neg as a whole, you pass the pause values to the PHY for it to advertise and trigger an auto-neg. Once autoneg has completed, and the resolved settings are available, the MAC is configured with the resolved values.

Looking at this code, i don't see any difference between configuring the MAC or configuring the PHY. I would expect pause->autoneg to be part of requested_pause somehow, so the firmware knows what is should do.

Quoted 7 lines
Quoted 1 line

Andrew

In this device there's actually very little the driver can do to directly configure the mac or phy besides passing through to the firmware what the user has requested - that happens here for the pause values, and in ionic_set_link_ksettings() for autoneg.  The firmware is managing the port based on these requests with the help of internally configured rules defined in a customer setting.

I get that. But the firmware needs to conform to what Linux expects. And what i see here does not conform. That is why i gave a bit of detail in my reply.

What exactly does the firmware do? Once we know that, we can figure out when the driver should return -EOPNOTSUPP because of firmware limitations, and what it can configure and should return 0.

Quoted 9 lines
Quoted 2 lines

Because this is fairly smart FW, it handles this as expected.  I can add this as another comment in the code.

Hi Shannon

Looking at the code, i don't see how it can handle this correctly. Please look at my comments, particularly the meaning of pause->autoneg and describe how the firmware does the right thing, given what information it is passed.

Quoted 6 lines

I guess I'm not sure how much better I can answer your question. Like several other devices, we don't support selecting pause->autoneg.  The firmware manages the PHY itself, the driver doesn't have direct access to the PHY.  The driver passes the tx and rx pause info down to the firmware in a command request.  The NIC firmware does an autoneg if autoneg is enabled on the port.

Hi Shannon

Thanks. That was the information i was looking for.

Please return -EOPNOTSUPP if pause->autoneg indicates autoneg results should not be used. Your firmware does not support it, so the driver should error out. Also the get function should use a hard coded value for pause->autoneg.

If you ever fix your firmware, you can add full support for pause configuration.

Thanks
Andrew

#25 lore raw

Re: [PATCH v3 net-next 13/19] ionic: Add initial ethtool support

Quoted 96 lines · Andrew Lunn
Quoted 85 lines · Shannon Nelson
Quoted 75 lines · Andrew Lunn
Quoted 74 lines · Shannon Nelson
Quoted 73 lines · Andrew Lunn
Quoted 58 lines · Shannon Nelson
Quoted 57 lines · Andrew Lunn
Quoted 26 lines
(quoted diff) +26 -0
+static int ionic_set_pauseparam(struct net_device *netdev,
+				struct ethtool_pauseparam *pause)
+{
+	struct lif *lif = netdev_priv(netdev);
+	struct ionic *ionic = lif->ionic;
+	struct ionic_dev *idev = &lif->ionic->idev;
+
+	u32 requested_pause;
+	u32 cur_autoneg;
+	int err;
+
+	cur_autoneg = idev->port_info->config.an_enable ? AUTONEG_ENABLE :
+								AUTONEG_DISABLE;
+	if (pause->autoneg != cur_autoneg) {
+		netdev_info(netdev, "Please use 'ethtool -s ...' to change autoneg\n");
+		return -EOPNOTSUPP;
+	}
+
+	/* change both at the same time */
+	requested_pause = PORT_PAUSE_TYPE_LINK;
+	if (pause->rx_pause)
+		requested_pause |= IONIC_PAUSE_F_RX;
+	if (pause->tx_pause)
+		requested_pause |= IONIC_PAUSE_F_TX;
+
+	if (requested_pause == idev->port_info->config.pause_type)

Review

Quoted 10 lines
(quoted diff) +10 -0
+		return 0;
+
+	idev->port_info->config.pause_type = requested_pause;
+
+	mutex_lock(&ionic->dev_cmd_lock);
+	ionic_dev_cmd_port_pause(idev, requested_pause);
+	err = ionic_dev_cmd_wait(ionic, devcmd_timeout);
+	mutex_unlock(&ionic->dev_cmd_lock);
+	if (err)
+		return err;

Hi Shannon

I've no idea what the firmware black box is doing, but this looks wrong.

pause->autoneg is about if the results of auto-neg should be used or not. If false, just configure the MAC with the pause settings and you are done. If the interface is being forced, so autoneg in general is disabled, just configure the MAC and you are done.

If pause->autoneg is true and the interface is using auto-neg as a whole, you pass the pause values to the PHY for it to advertise and trigger an auto-neg. Once autoneg has completed, and the resolved settings are available, the MAC is configured with the resolved values.

Looking at this code, i don't see any difference between configuring the MAC or configuring the PHY. I would expect pause->autoneg to be part of requested_pause somehow, so the firmware knows what is should do.

Quoted 7 lines
Quoted 1 line

Andrew

In this device there's actually very little the driver can do to directly configure the mac or phy besides passing through to the firmware what the user has requested - that happens here for the pause values, and in ionic_set_link_ksettings() for autoneg.  The firmware is managing the port based on these requests with the help of internally configured rules defined in a customer setting.

I get that. But the firmware needs to conform to what Linux expects. And what i see here does not conform. That is why i gave a bit of detail in my reply.

What exactly does the firmware do? Once we know that, we can figure out when the driver should return -EOPNOTSUPP because of firmware limitations, and what it can configure and should return 0.

Quoted 9 lines
Quoted 2 lines

Because this is fairly smart FW, it handles this as expected.  I can add this as another comment in the code.

Hi Shannon

Looking at the code, i don't see how it can handle this correctly. Please look at my comments, particularly the meaning of pause->autoneg and describe how the firmware does the right thing, given what information it is passed.

Quoted 6 lines

I guess I'm not sure how much better I can answer your question. Like several other devices, we don't support selecting pause->autoneg.  The firmware manages the PHY itself, the driver doesn't have direct access to the PHY.  The driver passes the tx and rx pause info down to the firmware in a command request.  The NIC firmware does an autoneg if autoneg is enabled on the port.

Hi Shannon

Thanks. That was the information i was looking for.

Quoted 9 lines

Please return -EOPNOTSUPP if pause->autoneg indicates autoneg results should not be used. Your firmware does not support it, so the driver should error out. Also the get function should use a hard coded value for pause->autoneg.

If you ever fix your firmware, you can add full support for pause configuration.

Thanks for the help,
sln

#26 lore raw

Re: [PATCH v3 net-next 13/19] ionic: Add initial ethtool support

In reply to Shannon Nelson

drivers/net/ethernet/pensando/ionic/ionic_ethtool.c line 451

Quoted 13 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +13 -0
+static int ionic_get_module_eeprom(struct net_device *netdev,
+				   struct ethtool_eeprom *ee,
+				   u8 *data)
+{
+	struct lif *lif = netdev_priv(netdev);
+	struct ionic_dev *idev = &lif->ionic->idev;
+	struct xcvr_status *xcvr;
+	u32 len;
+
+	/* copy the module bytes into data */
+	xcvr = &idev->port_info->status.xcvr;
+	len = min_t(u32, sizeof(xcvr->sprom), ee->len);
+	memcpy(data, xcvr->sprom, len);

Hi Shannon

This also looks odd. Where is the call into the firmware to get the eeprom contents? Even though it is called 'eeprom', the data is not static. It contains real time diagnostic values, temperature, transmit power, receiver power, voltages etc.

Quoted 10 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +10 -0
+
+	dev_dbg(&lif->netdev->dev, "notifyblock eid=0x%llx link_status=0x%x link_speed=0x%x link_down_cnt=0x%x\n",
+		lif->info->status.eid,
+		lif->info->status.link_status,
+		lif->info->status.link_speed,
+		lif->info->status.link_down_count);
+	dev_dbg(&lif->netdev->dev, "  port_status id=0x%x status=0x%x speed=0x%x\n",
+		idev->port_info->status.id,
+		idev->port_info->status.status,
+		idev->port_info->status.speed);

drivers/net/ethernet/pensando/ionic/ionic_ethtool.c line 471

Quoted 10 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +10 -0
+	dev_dbg(&lif->netdev->dev, "    xcvr status state=0x%x phy=0x%x pid=0x%x\n",
+		xcvr->state, xcvr->phy, xcvr->pid);
+	dev_dbg(&lif->netdev->dev, "  port_config state=0x%x speed=0x%x mtu=0x%x an_enable=0x%x fec_type=0x%x pause_type=0x%x loopback_mode=0x%x\n",
+		idev->port_info->config.state,
+		idev->port_info->config.speed,
+		idev->port_info->config.mtu,
+		idev->port_info->config.an_enable,
+		idev->port_info->config.fec_type,
+		idev->port_info->config.pause_type,
+		idev->port_info->config.loopback_mode);

It is a funny place to have all the debug code.

Andrew

#27 lore raw

Re: [PATCH v3 net-next 13/19] ionic: Add initial ethtool support

Quoted 15 lines · Andrew Lunn

drivers/net/ethernet/pensando/ionic/ionic_ethtool.c line 451

Quoted 13 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +13 -0
+static int ionic_get_module_eeprom(struct net_device *netdev,
+				   struct ethtool_eeprom *ee,
+				   u8 *data)
+{
+	struct lif *lif = netdev_priv(netdev);
+	struct ionic_dev *idev = &lif->ionic->idev;
+	struct xcvr_status *xcvr;
+	u32 len;
+
+	/* copy the module bytes into data */
+	xcvr = &idev->port_info->status.xcvr;
+	len = min_t(u32, sizeof(xcvr->sprom), ee->len);
+	memcpy(data, xcvr->sprom, len);

Hi Shannon

Quoted 4 lines

This also looks odd. Where is the call into the firmware to get the eeprom contents? Even though it is called 'eeprom', the data is not static. It contains real time diagnostic values, temperature, transmit power, receiver power, voltages etc.

idev->port_info is a memory mapped space that the device keeps up-to-date.

Quoted 14 lines
Quoted 14 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +13 -0
+
+	dev_dbg(&lif->netdev->dev, "notifyblock eid=0x%llx link_status=0x%x link_speed=0x%x link_down_cnt=0x%x\n",
+		lif->info->status.eid,
+		lif->info->status.link_status,
+		lif->info->status.link_speed,
+		lif->info->status.link_down_count);
+	dev_dbg(&lif->netdev->dev, "  port_status id=0x%x status=0x%x speed=0x%x\n",
+		idev->port_info->status.id,
+		idev->port_info->status.status,
+		idev->port_info->status.speed);
+	dev_dbg(&lif->netdev->dev, "    xcvr status state=0x%x phy=0x%x pid=0x%x\n",
+		xcvr->state, xcvr->phy, xcvr->pid);
+	dev_dbg(&lif->netdev->dev, "  port_config state=0x%x speed=0x%x mtu=0x%x an_enable=0x%x fec_type=0x%x pause_type=0x%x loopback_mode=0x%x\n",
Quoted 8 lines

drivers/net/ethernet/pensando/ionic/ionic_ethtool.c line 471

Quoted 7 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +7 -0
+		idev->port_info->config.state,
+		idev->port_info->config.speed,
+		idev->port_info->config.mtu,
+		idev->port_info->config.an_enable,
+		idev->port_info->config.fec_type,
+		idev->port_info->config.pause_type,
+		idev->port_info->config.loopback_mode);

It is a funny place to have all the debug code.

Someone wanted a hook for getting some link info on the fly on request.

sln

Quoted 2 lines

Andrew

#28 lore raw

Re: [PATCH v3 net-next 13/19] ionic: Add initial ethtool support

Quoted 16 lines · Shannon Nelson
Quoted 15 lines · Andrew Lunn

drivers/net/ethernet/pensando/ionic/ionic_ethtool.c line 451

Quoted 13 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +13 -0
+static int ionic_get_module_eeprom(struct net_device *netdev,
+				   struct ethtool_eeprom *ee,
+				   u8 *data)
+{
+	struct lif *lif = netdev_priv(netdev);
+	struct ionic_dev *idev = &lif->ionic->idev;
+	struct xcvr_status *xcvr;
+	u32 len;
+
+	/* copy the module bytes into data */
+	xcvr = &idev->port_info->status.xcvr;
+	len = min_t(u32, sizeof(xcvr->sprom), ee->len);
+	memcpy(data, xcvr->sprom, len);

Hi Shannon

Quoted 6 lines
Quoted 4 lines

This also looks odd. Where is the call into the firmware to get the eeprom contents? Even though it is called 'eeprom', the data is not static. It contains real time diagnostic values, temperature, transmit power, receiver power, voltages etc.

idev->port_info is a memory mapped space that the device keeps up-to-date.

Hi Shannon

It at least needs a comment. How frequently does the device update this chunk of memory? It would be good to comment about that as well. Or do MMIO reads block while i2c operations occur to update the memory?

Quoted 22 lines
Quoted 13 lines
Quoted 13 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +13 -0
+
+	dev_dbg(&lif->netdev->dev, "notifyblock eid=0x%llx link_status=0x%x link_speed=0x%x link_down_cnt=0x%x\n",
+		lif->info->status.eid,
+		lif->info->status.link_status,
+		lif->info->status.link_speed,
+		lif->info->status.link_down_count);
+	dev_dbg(&lif->netdev->dev, "  port_status id=0x%x status=0x%x speed=0x%x\n",
+		idev->port_info->status.id,
+		idev->port_info->status.status,
+		idev->port_info->status.speed);
+	dev_dbg(&lif->netdev->dev, "    xcvr status state=0x%x phy=0x%x pid=0x%x\n",
+		xcvr->state, xcvr->phy, xcvr->pid);
+	dev_dbg(&lif->netdev->dev, "  port_config state=0x%x speed=0x%x mtu=0x%x an_enable=0x%x fec_type=0x%x pause_type=0x%x loopback_mode=0x%x\n",
Quoted 8 lines

drivers/net/ethernet/pensando/ionic/ionic_ethtool.c line 471

Quoted 7 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +7 -0
+		idev->port_info->config.state,
+		idev->port_info->config.speed,
+		idev->port_info->config.mtu,
+		idev->port_info->config.an_enable,
+		idev->port_info->config.fec_type,
+		idev->port_info->config.pause_type,
+		idev->port_info->config.loopback_mode);

It is a funny place to have all the debug code.

Quoted 1 line

Someone wanted a hook for getting some link info on the fly on request.

I would suggest deleting it all. Most of it is already available via ethtool.

Andrew

#29 lore raw

Re: [PATCH v3 net-next 13/19] ionic: Add initial ethtool support

Quoted 21 lines · Andrew Lunn
Quoted 20 lines · Shannon Nelson
Quoted 19 lines · Andrew Lunn

drivers/net/ethernet/pensando/ionic/ionic_ethtool.c line 451

Quoted 13 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +13 -0
+static int ionic_get_module_eeprom(struct net_device *netdev,
+				   struct ethtool_eeprom *ee,
+				   u8 *data)
+{
+	struct lif *lif = netdev_priv(netdev);
+	struct ionic_dev *idev = &lif->ionic->idev;
+	struct xcvr_status *xcvr;
+	u32 len;
+
+	/* copy the module bytes into data */
+	xcvr = &idev->port_info->status.xcvr;
+	len = min_t(u32, sizeof(xcvr->sprom), ee->len);
+	memcpy(data, xcvr->sprom, len);

Hi Shannon

This also looks odd. Where is the call into the firmware to get the eeprom contents? Even though it is called 'eeprom', the data is not static. It contains real time diagnostic values, temperature, transmit power, receiver power, voltages etc.

Quoted 7 lines
Quoted 1 line

idev->port_info is a memory mapped space that the device keeps up-to-date.

Hi Shannon

It at least needs a comment. How frequently does the device update this chunk of memory? It would be good to comment about that as well. Or do MMIO reads block while i2c operations occur to update the memory?

The device keeps this updated when changes happen internally so that there is no need to block on MMIO read.  Sure, I'll add a little more commentary here and perhaps in a couple other similar places.

Quoted 22 lines
Quoted 22 lines
Quoted 22 lines
Quoted 11 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +10 -0
+
+	dev_dbg(&lif->netdev->dev, "notifyblock eid=0x%llx link_status=0x%x link_speed=0x%x link_down_cnt=0x%x\n",
+		lif->info->status.eid,
+		lif->info->status.link_status,
+		lif->info->status.link_speed,
+		lif->info->status.link_down_count);
+	dev_dbg(&lif->netdev->dev, "  port_status id=0x%x status=0x%x speed=0x%x\n",
+		idev->port_info->status.id,
+		idev->port_info->status.status,
+		idev->port_info->status.speed);

drivers/net/ethernet/pensando/ionic/ionic_ethtool.c line 471

Quoted 10 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +10 -0
+	dev_dbg(&lif->netdev->dev, "    xcvr status state=0x%x phy=0x%x pid=0x%x\n",
+		xcvr->state, xcvr->phy, xcvr->pid);
+	dev_dbg(&lif->netdev->dev, "  port_config state=0x%x speed=0x%x mtu=0x%x an_enable=0x%x fec_type=0x%x pause_type=0x%x loopback_mode=0x%x\n",
+		idev->port_info->config.state,
+		idev->port_info->config.speed,
+		idev->port_info->config.mtu,
+		idev->port_info->config.an_enable,
+		idev->port_info->config.fec_type,
+		idev->port_info->config.pause_type,
+		idev->port_info->config.loopback_mode);

It is a funny place to have all the debug code.

Quoted 4 lines
Quoted 1 line

Someone wanted a hook for getting some link info on the fly on request.

I would suggest deleting it all. Most of it is already available via ethtool.

Sure.

sln

#30 lore raw

Re: [PATCH v3 net-next 13/19] ionic: Add initial ethtool support

Quoted 25 lines · Shannon Nelson
Quoted 24 lines · Andrew Lunn
Quoted 16 lines · Shannon Nelson
Quoted 15 lines · Andrew Lunn

drivers/net/ethernet/pensando/ionic/ionic_ethtool.c line 451

Quoted 13 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +13 -0
+static int ionic_get_module_eeprom(struct net_device *netdev,
+				   struct ethtool_eeprom *ee,
+				   u8 *data)
+{
+	struct lif *lif = netdev_priv(netdev);
+	struct ionic_dev *idev = &lif->ionic->idev;
+	struct xcvr_status *xcvr;
+	u32 len;
+
+	/* copy the module bytes into data */
+	xcvr = &idev->port_info->status.xcvr;
+	len = min_t(u32, sizeof(xcvr->sprom), ee->len);
+	memcpy(data, xcvr->sprom, len);

Hi Shannon

Quoted 5 lines
Quoted 4 lines

This also looks odd. Where is the call into the firmware to get the eeprom contents? Even though it is called 'eeprom', the data is not static. It contains real time diagnostic values, temperature, transmit power, receiver power, voltages etc.

idev->port_info is a memory mapped space that the device keeps up-to-date.

Hi Shannon

Quoted 7 lines
Quoted 4 lines

It at least needs a comment. How frequently does the device update this chunk of memory? It would be good to comment about that as well. Or do MMIO reads block while i2c operations occur to update the memory?

The device keeps this updated when changes happen internally so that there is no need to block on MMIO read.

Hi Shannon

I'm thinking about the diagnostic page. RX and TX power, temperature, alarms etc. These are real time values, so you should read them on demand, or at last only cache them for a short time.

Andrew

#31 lore raw

Re: [PATCH v3 net-next 13/19] ionic: Add initial ethtool support

Quoted 30 lines · Andrew Lunn
Quoted 29 lines · Shannon Nelson
Quoted 28 lines · Andrew Lunn
Quoted 16 lines · Shannon Nelson
Quoted 15 lines · Andrew Lunn

drivers/net/ethernet/pensando/ionic/ionic_ethtool.c line 451

Quoted 13 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +13 -0
+static int ionic_get_module_eeprom(struct net_device *netdev,
+				   struct ethtool_eeprom *ee,
+				   u8 *data)
+{
+	struct lif *lif = netdev_priv(netdev);
+	struct ionic_dev *idev = &lif->ionic->idev;
+	struct xcvr_status *xcvr;
+	u32 len;
+
+	/* copy the module bytes into data */
+	xcvr = &idev->port_info->status.xcvr;
+	len = min_t(u32, sizeof(xcvr->sprom), ee->len);
+	memcpy(data, xcvr->sprom, len);

Hi Shannon

Quoted 5 lines
Quoted 4 lines

This also looks odd. Where is the call into the firmware to get the eeprom contents? Even though it is called 'eeprom', the data is not static. It contains real time diagnostic values, temperature, transmit power, receiver power, voltages etc.

idev->port_info is a memory mapped space that the device keeps up-to-date.

Hi Shannon

It at least needs a comment. How frequently does the device update this chunk of memory? It would be good to comment about that as well. Or do MMIO reads block while i2c operations occur to update the memory?

Quoted 9 lines
Quoted 2 lines

The device keeps this updated when changes happen internally so that there is no need to block on MMIO read.

Hi Shannon

I'm thinking about the diagnostic page. RX and TX power, temperature, alarms etc. These are real time values, so you should read them on demand, or at last only cache them for a short time.

They *are* read on demand.  The port_info and lif_info structs are dma mapped spaces that the device keeps up-to-date with PCI writes in the background so that the driver can do a quick memory read for current data.

sln

#32 lore raw

Re: [PATCH v3 net-next 13/19] ionic: Add initial ethtool support

In reply to Shannon Nelson

drivers/net/ethernet/pensando/ionic/ionic_ethtool.c line 482

Quoted 7 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +7 -0
+static int ionic_nway_reset(struct net_device *netdev)
+{
+	struct lif *lif = netdev_priv(netdev);
+	int err = 0;
+
+	if (netif_running(netdev))
+		err = ionic_reset_queues(lif);

What does ionic_reset_queues() do? It sounds nothing like restarting auto negotiation?

Andrew

#33 lore raw

Re: [PATCH v3 net-next 13/19] ionic: Add initial ethtool support

Quoted 11 lines · Andrew Lunn

drivers/net/ethernet/pensando/ionic/ionic_ethtool.c line 482

Quoted 7 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +7 -0
+static int ionic_nway_reset(struct net_device *netdev)
+{
+	struct lif *lif = netdev_priv(netdev);
+	int err = 0;
+
+	if (netif_running(netdev))
+		err = ionic_reset_queues(lif);

What does ionic_reset_queues() do? It sounds nothing like restarting auto negotiation?

Andrew

Basically, it's a rip-it-all-down-and-start-over way of restarting the connection, and is also useful for fixing queues that are misbehaving.  It's a little old-fashioned, taken from the ixgbe example, but is effective when there isn't an actual "restart auto-negotiation" command in the firmware.

I'll try to make it a little more evident.

sln

#34 lore raw

Re: [PATCH v3 net-next 13/19] ionic: Add initial ethtool support

Quoted 11 lines · Shannon Nelson
Quoted 10 lines · Andrew Lunn

drivers/net/ethernet/pensando/ionic/ionic_ethtool.c line 482

Quoted 7 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +7 -0
+static int ionic_nway_reset(struct net_device *netdev)
+{
+	struct lif *lif = netdev_priv(netdev);
+	int err = 0;
+
+	if (netif_running(netdev))
+		err = ionic_reset_queues(lif);

What does ionic_reset_queues() do? It sounds nothing like restarting auto negotiation?

Quoted 5 lines
Quoted 1 line

Andrew

Basically, it's a rip-it-all-down-and-start-over way of restarting the connection, and is also useful for fixing queues that are misbehaving.  It's a little old-fashioned, taken from the ixgbe example, but is effective when there isn't an actual "restart auto-negotiation" command in the firmware.

O.K. More comments please.

Did you consider throwing the firmware away and just letting Linux control the hardware? It would make this all much more transparent and debuggable.

Andrew

#35 lore raw

Re: [PATCH v3 net-next 13/19] ionic: Add initial ethtool support

Quoted 12 lines · Andrew Lunn
Quoted 11 lines · Shannon Nelson
Quoted 10 lines · Andrew Lunn

drivers/net/ethernet/pensando/ionic/ionic_ethtool.c line 482

Quoted 7 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +7 -0
+static int ionic_nway_reset(struct net_device *netdev)
+{
+	struct lif *lif = netdev_priv(netdev);
+	int err = 0;
+
+	if (netif_running(netdev))
+		err = ionic_reset_queues(lif);

What does ionic_reset_queues() do? It sounds nothing like restarting auto negotiation?

Quoted 6 lines
Quoted 5 lines
Quoted 1 line

Andrew

Basically, it's a rip-it-all-down-and-start-over way of restarting the connection, and is also useful for fixing queues that are misbehaving.  It's a little old-fashioned, taken from the ixgbe example, but is effective when there isn't an actual "restart auto-negotiation" command in the firmware.

O.K. More comments please.

After a little more discussion with the nic connection folks, they've said to just flap the link to force a new auto-negotiation, so that's what I'm adding - set state to off, then back on.  And yes, another comment here.

Quoted 5 lines

Did you consider throwing the firmware away and just letting Linux control the hardware? It would make this all much more transparent and debuggable.

Yes, the tension between direct driver bit twiddling and smartnic offloading continues.

sln

#36 lore raw

Re: [PATCH v3 net-next 13/19] ionic: Add initial ethtool support

In reply to Shannon Nelson
Quoted 14 lines · Shannon Nelson

Add in the basic ethtool callbacks for device information and control.

---
 drivers/net/ethernet/pensando/ionic/Makefile  |   2 +-
 .../net/ethernet/pensando/ionic/ionic_dev.h   |   3 +
 .../ethernet/pensando/ionic/ionic_ethtool.c   | 509 ++++++++++++++++++
 .../ethernet/pensando/ionic/ionic_ethtool.h   |   9 +
 .../net/ethernet/pensando/ionic/ionic_lif.c   |   2 +
 .../net/ethernet/pensando/ionic/ionic_lif.h   |   8 +
 6 files changed, 532 insertions(+), 1 deletion(-)
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_ethtool.c
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_ethtool.h

...

Quoted 45 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +45 -0
+static int ionic_set_link_ksettings(struct net_device *netdev,
+				    const struct ethtool_link_ksettings *ks)
+{
+	struct lif *lif = netdev_priv(netdev);
+	struct ionic *ionic = lif->ionic;
+	struct ionic_dev *idev = &lif->ionic->idev;
+	u8 fec_type = PORT_FEC_TYPE_NONE;
+	u32 req_rs, req_fc;
+	int err = 0;
+
+	/* set autoneg */
+	if (ks->base.autoneg != idev->port_info->config.an_enable) {
+		mutex_lock(&ionic->dev_cmd_lock);
+		ionic_dev_cmd_port_autoneg(idev, ks->base.autoneg);
+		err = ionic_dev_cmd_wait(ionic, devcmd_timeout);
+		mutex_unlock(&ionic->dev_cmd_lock);
+		if (err)
+			return err;
+
+		idev->port_info->config.an_enable = ks->base.autoneg;
+	}
+
+	/* set speed */
+	if (ks->base.speed != le32_to_cpu(idev->port_info->config.speed)) {
+		mutex_lock(&ionic->dev_cmd_lock);
+		ionic_dev_cmd_port_speed(idev, ks->base.speed);
+		err = ionic_dev_cmd_wait(ionic, devcmd_timeout);
+		mutex_unlock(&ionic->dev_cmd_lock);
+		if (err)
+			return err;
+
+		idev->port_info->config.speed = cpu_to_le32(ks->base.speed);
+	}
+
+	/* set FEC */
+	req_rs = ethtool_link_ksettings_test_link_mode(ks, advertising, FEC_RS);
+	req_fc = ethtool_link_ksettings_test_link_mode(ks, advertising, FEC_BASER);
+	if (req_rs && req_fc) {
+		netdev_info(netdev, "Only select one FEC mode at a time\n");
+		return -EINVAL;
+
+	} else if (req_fc &&
+		   idev->port_info->config.fec_type != PORT_FEC_TYPE_FC) {
+		fec_type = PORT_FEC_TYPE_FC;
+	} else if (req_rs &&

drivers/net/ethernet/pensando/ionic/ionic_ethtool.c line 238

Quoted 10 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +10 -0
+		   idev->port_info->config.fec_type != PORT_FEC_TYPE_RS) {
+		fec_type = PORT_FEC_TYPE_RS;
+	} else if (!(req_rs | req_fc) &&
+		 idev->port_info->config.fec_type != PORT_FEC_TYPE_NONE) {
+		fec_type = PORT_FEC_TYPE_NONE;
+	}
+
+	if (fec_type != idev->port_info->config.fec_type) {
+		mutex_lock(&ionic->dev_cmd_lock);
+		ionic_dev_cmd_port_fec(idev, PORT_FEC_TYPE_NONE);

The second argument should be fec_type, I believe.

Quoted 10 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +10 -0
+		err = ionic_dev_cmd_wait(ionic, devcmd_timeout);
+		mutex_unlock(&ionic->dev_cmd_lock);
+		if (err)
+			return err;
+
+		idev->port_info->config.fec_type = fec_type;
+	}
+
+	return 0;
+}

...

Quoted 8 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +8 -0
+static int ionic_set_ringparam(struct net_device *netdev,
+			       struct ethtool_ringparam *ring)
+{
+	struct lif *lif = netdev_priv(netdev);
+	bool running;
+	int i, j;
+
+	if (ring->rx_mini_pending || ring->rx_jumbo_pending) {

drivers/net/ethernet/pensando/ionic/ionic_ethtool.c line 344

Quoted 10 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +10 -0
+		netdev_info(netdev, "Changing jumbo or mini descriptors not supported\n");
+		return -EINVAL;
+	}
+
+	i = ring->tx_pending & (ring->tx_pending - 1);
+	j = ring->rx_pending & (ring->rx_pending - 1);
+	if (i || j) {
+		netdev_info(netdev, "Descriptor count must be a power of 2\n");
+		return -EINVAL;
+	}

You can use is_power_of_2() here (it wouldn't allow 0 but you probably don't want to allow that either).

Michal

Quoted 22 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +22 -0
+
+	/* if nothing to do return success */
+	if (ring->tx_pending == lif->ntxq_descs &&
+	    ring->rx_pending == lif->nrxq_descs)
+		return 0;
+
+	while (test_and_set_bit(LIF_QUEUE_RESET, lif->state))
+		usleep_range(200, 400);
+
+	running = test_bit(LIF_UP, lif->state);
+	if (running)
+		ionic_stop(netdev);
+
+	lif->ntxq_descs = ring->tx_pending;
+	lif->nrxq_descs = ring->rx_pending;
+
+	if (running)
+		ionic_open(netdev);
+	clear_bit(LIF_QUEUE_RESET, lif->state);
+
+	return 0;
+}
#37 lore raw

Re: [PATCH v3 net-next 13/19] ionic: Add initial ethtool support

Quoted 10 lines · Michal Kubecek

drivers/net/ethernet/pensando/ionic/ionic_ethtool.c line 238

Quoted 8 lines · Shannon Nelson

Add in the basic ethtool callbacks for device information and control.

drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +4 -0
+
+	if (fec_type != idev->port_info->config.fec_type) {
+		mutex_lock(&ionic->dev_cmd_lock);
+		ionic_dev_cmd_port_fec(idev, PORT_FEC_TYPE_NONE);

The second argument should be fec_type, I believe.

Yep.

Quoted 24 lines
Quoted 11 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +10 -0
+		err = ionic_dev_cmd_wait(ionic, devcmd_timeout);
+		mutex_unlock(&ionic->dev_cmd_lock);
+		if (err)
+			return err;
+
+		idev->port_info->config.fec_type = fec_type;
+	}
+
+	return 0;
+}

...

Quoted 12 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +12 -0
+static int ionic_set_ringparam(struct net_device *netdev,
+			       struct ethtool_ringparam *ring)
+{
+	struct lif *lif = netdev_priv(netdev);
+	bool running;
+	int i, j;
+
+	if (ring->rx_mini_pending || ring->rx_jumbo_pending) {
+		netdev_info(netdev, "Changing jumbo or mini descriptors not supported\n");
+		return -EINVAL;
+	}
+
Quoted 8 lines

drivers/net/ethernet/pensando/ionic/ionic_ethtool.c line 344

Quoted 6 lines
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c new file +6 -0
+	i = ring->tx_pending & (ring->tx_pending - 1);
+	j = ring->rx_pending & (ring->rx_pending - 1);
+	if (i || j) {
+		netdev_info(netdev, "Descriptor count must be a power of 2\n");
+		return -EINVAL;
+	}

You can use is_power_of_2() here (it wouldn't allow 0 but you probably don't want to allow that either).

Sure.

Thanks,
sln

#38 lore raw

PATCH v3 14/19 ionic: Add Tx and Rx handling

In reply to Shannon Nelson

Add both the Tx and Rx queue setup and handling. The related stats display comes later. Instead of using the generic napi routines used by the slow-path commands, the Tx and Rx paths are simplified and inlined in one file in order to get better compiler optimizations.

---
 drivers/net/ethernet/pensando/ionic/Makefile  |   2 +-
 .../net/ethernet/pensando/ionic/ionic_lif.c   | 387 ++++++++
 .../net/ethernet/pensando/ionic/ionic_lif.h   |  52 ++
 .../net/ethernet/pensando/ionic/ionic_txrx.c  | 879 ++++++++++++++++++
 .../net/ethernet/pensando/ionic/ionic_txrx.h  |  15 +
 5 files changed, 1334 insertions(+), 1 deletion(-)
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_txrx.c
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_txrx.h
drivers/net/ethernet/pensando/ionic/Makefile +1 -1
index 9b19bf57a489..0e2dc53f08d4 100644
@@ -4,4 +4,4 @@
 obj-$(CONFIG_IONIC) := ionic.o
 
 ionic-y := ionic_main.o ionic_bus_pci.o ionic_dev.o ionic_ethtool.o \
-	   ionic_lif.o ionic_rx_filter.o ionic_debugfs.o
+	   ionic_lif.o ionic_rx_filter.o ionic_txrx.o ionic_debugfs.o
drivers/net/ethernet/pensando/ionic/ionic_lif.c +387 -0

482 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_lif.h +52 -0

109 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_txrx.c new file +879 -0

879 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_txrx.h new file +15 -0
new file mode 100644
index 000000000000..2391a0eec65a
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+
+#ifndef _IONIC_TXRX_H_
+#define _IONIC_TXRX_H_
+
+void ionic_rx_flush(struct cq *cq);
+void ionic_tx_flush(struct cq *cq);
+
+void ionic_rx_fill(struct queue *q);
+void ionic_rx_empty(struct queue *q);
+int ionic_rx_napi(struct napi_struct *napi, int budget);
+netdev_tx_t ionic_start_xmit(struct sk_buff *skb, struct net_device *netdev);
+
+#endif /* _IONIC_TXRX_H_ */
Signature · 1 line
2.17.1
#39 lore raw

PATCH v3 15/19 ionic: Add netdev-event handling

In reply to Shannon Nelson

When the netdev gets a new name from userland, pass that name down to the NIC for internal tracking.

---
 drivers/net/ethernet/pensando/ionic/ionic.h   |  2 +
 .../net/ethernet/pensando/ionic/ionic_lif.c   | 65 +++++++++++++++++++
 2 files changed, 67 insertions(+)
drivers/net/ethernet/pensando/ionic/ionic.h +2 -0
index d7eee79b2a10..9b720187b549 100644
@@ -42,6 +42,8 @@ struct ionic {
 	DECLARE_BITMAP(lifbits, IONIC_LIFS_MAX);
 	unsigned int nintrs;
 	DECLARE_BITMAP(intrs, INTR_CTRL_REGS_MAX);
+	struct work_struct nb_work;
+	struct notifier_block nb;
 };
 
 struct ionic_admin_ctx {
drivers/net/ethernet/pensando/ionic/ionic_lif.c +65 -0

81 lines not on this page · raw message

Signature · 1 line
2.17.1
#40 lore raw

PATCH v3 16/19 ionic: Add driver stats

In reply to Shannon Nelson

Add in the detailed statistics for ethtool -S that the driver keeps as it processes packets. Display of the additional debug statistics can be enabled through the ethtool priv-flags feature.

---
 drivers/net/ethernet/pensando/ionic/Makefile  |   3 +-
 .../ethernet/pensando/ionic/ionic_ethtool.c   | 101 ++++++
 .../net/ethernet/pensando/ionic/ionic_lif.h   |  14 +
 .../net/ethernet/pensando/ionic/ionic_stats.c | 333 ++++++++++++++++++
 .../net/ethernet/pensando/ionic/ionic_stats.h |  53 +++
 5 files changed, 503 insertions(+), 1 deletion(-)
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_stats.c
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_stats.h
drivers/net/ethernet/pensando/ionic/Makefile +2 -1
index 0e2dc53f08d4..4f3cfbf36c23 100644
@@ -4,4 +4,5 @@
 obj-$(CONFIG_IONIC) := ionic.o
 
 ionic-y := ionic_main.o ionic_bus_pci.o ionic_dev.o ionic_ethtool.o \
-	   ionic_lif.o ionic_rx_filter.o ionic_txrx.o ionic_debugfs.o
+	   ionic_lif.o ionic_rx_filter.o ionic_txrx.o ionic_debugfs.o \
+	   ionic_stats.o
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c +101 -0

119 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_lif.h +14 -0
index 88d5bf8f58a1..f93bfa2b4393 100644
@@ -108,8 +108,22 @@ struct ionic_deferred {
 	struct work_struct work;
 };
 
+struct lif_sw_stats {
+	u64 tx_packets;
+	u64 tx_bytes;
+	u64 rx_packets;
+	u64 rx_bytes;
+	u64 tx_tso;
+	u64 tx_no_csum;
+	u64 tx_csum;
+	u64 rx_csum_none;
+	u64 rx_csum_complete;
+	u64 rx_csum_error;
+};
+
 enum lif_state_flags {
 	LIF_INITED,
+	LIF_SW_DEBUG_STATS,
 	LIF_UP,
 	LIF_LINK_CHECK_NEEDED,
 	LIF_QUEUE_RESET,
drivers/net/ethernet/pensando/ionic/ionic_stats.c new file +333 -0

333 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_stats.h new file +53 -0

53 lines not on this page · raw message

Signature · 1 line
2.17.1
#41 lore raw

PATCH v3 17/19 ionic: Add RSS support

In reply to Shannon Nelson

Add code to manipulate through ethtool the RSS configuration used by the NIC.

---
 .../ethernet/pensando/ionic/ionic_ethtool.c   | 73 ++++++++++++++++
 .../net/ethernet/pensando/ionic/ionic_lif.c   | 87 +++++++++++++++++++
 .../net/ethernet/pensando/ionic/ionic_lif.h   |  8 ++
 3 files changed, 168 insertions(+)
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c +73 -0
index 6785c0cd4689..403882a5e09e 100644
@@ -476,6 +476,74 @@ static int ionic_set_channels(struct net_device *netdev,
 	return 0;
 }
 
+static int ionic_get_rxnfc(struct net_device *netdev,
+			   struct ethtool_rxnfc *info, u32 *rules)
+{
+	struct lif *lif = netdev_priv(netdev);
+	int err = 0;
+
+	switch (info->cmd) {
+	case ETHTOOL_GRXRINGS:
+		info->data = lif->nxqs;
+		break;
+	default:
+		netdev_err(netdev, "Command parameter %d is not supported\n",
+			   info->cmd);
+		err = -EOPNOTSUPP;
+	}
+
+	return err;
+}
+
+static u32 ionic_get_rxfh_indir_size(struct net_device *netdev)
+{
+	struct lif *lif = netdev_priv(netdev);
+
+	return le16_to_cpu(lif->ionic->ident.lif.eth.rss_ind_tbl_sz);
+}
+
+static u32 ionic_get_rxfh_key_size(struct net_device *netdev)
+{
+	return IONIC_RSS_HASH_KEY_SIZE;
+}
+
+static int ionic_get_rxfh(struct net_device *netdev, u32 *indir, u8 *key,
+			  u8 *hfunc)
+{
+	struct lif *lif = netdev_priv(netdev);
+	unsigned int i, tbl_sz;
+
+	if (indir) {
+		tbl_sz = le16_to_cpu(lif->ionic->ident.lif.eth.rss_ind_tbl_sz);
+		for (i = 0; i < tbl_sz; i++)
+			indir[i] = lif->rss_ind_tbl[i];
+	}
+
+	if (key)
+		memcpy(key, lif->rss_hash_key, IONIC_RSS_HASH_KEY_SIZE);
+
+	if (hfunc)
+		*hfunc = ETH_RSS_HASH_TOP;
+
+	return 0;
+}
+
+static int ionic_set_rxfh(struct net_device *netdev, const u32 *indir,
+			  const u8 *key, const u8 hfunc)
+{
+	struct lif *lif = netdev_priv(netdev);
+	int err;
+
+	if (hfunc != ETH_RSS_HASH_NO_CHANGE && hfunc != ETH_RSS_HASH_TOP)
+		return -EOPNOTSUPP;
+
+	err = ionic_lif_rss_config(lif, lif->rss_types, key, indir);
+	if (err)
+		return err;
+
+	return 0;
+}
+
 static u32 ionic_get_priv_flags(struct net_device *netdev)
 {
 	struct lif *lif = netdev_priv(netdev);
@@ -594,6 +662,11 @@ static const struct ethtool_ops ionic_ethtool_ops = {
 	.get_strings		= ionic_get_strings,
 	.get_ethtool_stats	= ionic_get_stats,
 	.get_sset_count		= ionic_get_sset_count,
+	.get_rxnfc		= ionic_get_rxnfc,
+	.get_rxfh_indir_size	= ionic_get_rxfh_indir_size,
+	.get_rxfh_key_size	= ionic_get_rxfh_key_size,
+	.get_rxfh		= ionic_get_rxfh,
+	.set_rxfh		= ionic_set_rxfh,
 	.get_priv_flags		= ionic_get_priv_flags,
 	.set_priv_flags		= ionic_set_priv_flags,
 	.get_module_info	= ionic_get_module_info,
drivers/net/ethernet/pensando/ionic/ionic_lif.c +87 -0

133 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_lif.h +8 -0
index f93bfa2b4393..0e6908f959f2 100644
@@ -166,6 +166,12 @@ struct lif {
 	dma_addr_t info_pa;
 	u32 info_sz;
 
+	u16 rss_types;
+	u8 rss_hash_key[IONIC_RSS_HASH_KEY_SIZE];
+	u8 *rss_ind_tbl;
+	dma_addr_t rss_ind_tbl_pa;
+	u32 rss_ind_tbl_sz;
+
 	struct rx_filters rx_filters;
 	struct ionic_deferred deferred;
 	u32 tx_coalesce_usecs;
@@ -215,6 +221,8 @@ void ionic_lifs_unregister(struct ionic *ionic);
 int ionic_lif_identify(struct ionic *ionic, u8 lif_type,
 		       union lif_identity *lif_ident);
 int ionic_lifs_size(struct ionic *ionic);
+int ionic_lif_rss_config(struct lif *lif, u16 types,
+			 const u8 *key, const u32 *indir);
 
 int ionic_open(struct net_device *netdev);
 int ionic_stop(struct net_device *netdev);
Signature · 1 line
2.17.1
#42 lore raw

PATCH v3 18/19 ionic: Add coalesce and other features

In reply to Shannon Nelson

Interrupt coalescing, tunable copybreak value, and tx timeout.

---
 drivers/net/ethernet/pensando/ionic/ionic.h   |   2 +-
 .../ethernet/pensando/ionic/ionic_ethtool.c   | 105 ++++++++++++++++++
 .../net/ethernet/pensando/ionic/ionic_lif.c   |  13 ++-
 .../net/ethernet/pensando/ionic/ionic_lif.h   |   1 +
 4 files changed, 119 insertions(+), 2 deletions(-)
drivers/net/ethernet/pensando/ionic/ionic.h +1 -1
index 9b720187b549..cd08166f73a9 100644
@@ -11,7 +11,7 @@ struct lif;
 
 #define DRV_NAME		"ionic"
 #define DRV_DESCRIPTION		"Pensando Ethernet NIC Driver"
-#define DRV_VERSION		"0.11.0-k"
+#define DRV_VERSION		"0.11.0-44-k"
 
 #define PCI_VENDOR_ID_PENSANDO			0x1dd8
 
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c +105 -0

129 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_lif.c +12 -1
index 68a9975e34c6..8473b065763b 100644
@@ -744,9 +744,19 @@ static int ionic_change_mtu(struct net_device *netdev, int new_mtu)
 	return err;
 }
 
+static void ionic_tx_timeout_work(struct work_struct *ws)
+{
+	struct lif *lif = container_of(ws, struct lif, tx_timeout_work);
+
+	netdev_info(lif->netdev, "Tx Timeout recovery\n");
+	ionic_reset_queues(lif);
+}
+
 static void ionic_tx_timeout(struct net_device *netdev)
 {
-	netdev_info(netdev, "%s: stubbed\n", __func__);
+	struct lif *lif = netdev_priv(netdev);
+
+	schedule_work(&lif->tx_timeout_work);
 }
 
 static int ionic_vlan_rx_add_vid(struct net_device *netdev, __be16 proto,
@@ -2009,6 +2019,7 @@ static int ionic_lif_init(struct lif *lif)
 
 	ionic_link_status_check(lif);
 
+	INIT_WORK(&lif->tx_timeout_work, ionic_tx_timeout_work);
 	return 0;
 
 err_out_notifyq_deinit:
drivers/net/ethernet/pensando/ionic/ionic_lif.h +1 -0
index 0e6908f959f2..76cc519acd5a 100644
@@ -180,6 +180,7 @@ struct lif {
 	unsigned int dbid_count;
 	struct dentry *dentry;
 	u32 flags;
+	struct work_struct tx_timeout_work;
 };
 
 #define lif_to_txqcq(lif, i)	((lif)->txqcqs[i].qcq)
Signature · 1 line
2.17.1
#43 lore raw

PATCH v3 19/19 ionic: Add basic devlink interface

In reply to Shannon Nelson

Add a devlink interface for access to information that isn't normally available through ethtool or the iplink interface.

Example:

	$ ./devlink -j -p dev info pci/0000:b6:00.0
	{
	    "info": {
		"pci/0000:b6:00.0": {
		    "driver": "ionic",
		    "serial_number": "FLM18420073",
		    "versions": {
			"fixed": {
			    "fw_version": "0.11.0-50",
			    "fw_status": "0x1",
			    "fw_heartbeat": "0x716ce",
			    "asic_type": "0x0",
			    "asic_rev": "0x0"
			}
		    }
		}
	    }
	}
---
 drivers/net/ethernet/pensando/ionic/Makefile  |  2 +-
 drivers/net/ethernet/pensando/ionic/ionic.h   |  1 +
 .../ethernet/pensando/ionic/ionic_bus_pci.c   |  7 ++
 .../ethernet/pensando/ionic/ionic_devlink.c   | 89 +++++++++++++++++++
 .../ethernet/pensando/ionic/ionic_devlink.h   | 12 +++
 5 files changed, 110 insertions(+), 1 deletion(-)
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_devlink.c
 create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_devlink.h
drivers/net/ethernet/pensando/ionic/Makefile +1 -1
index 4f3cfbf36c23..ce187c7b33a8 100644
@@ -5,4 +5,4 @@ obj-$(CONFIG_IONIC) := ionic.o
 
 ionic-y := ionic_main.o ionic_bus_pci.o ionic_dev.o ionic_ethtool.o \
 	   ionic_lif.o ionic_rx_filter.o ionic_txrx.o ionic_debugfs.o \
-	   ionic_stats.o
+	   ionic_stats.o ionic_devlink.o
drivers/net/ethernet/pensando/ionic/ionic.h +1 -0
index cd08166f73a9..a0034bc5b4a1 100644
@@ -44,6 +44,7 @@ struct ionic {
 	DECLARE_BITMAP(intrs, INTR_CTRL_REGS_MAX);
 	struct work_struct nb_work;
 	struct notifier_block nb;
+	struct devlink *dl;
 };
 
 struct ionic_admin_ctx {
drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c +7 -0
index 98c12b770c7f..a8c99254489f 100644
@@ -10,6 +10,7 @@
 #include "ionic_bus.h"
 #include "ionic_lif.h"
 #include "ionic_debugfs.h"
+#include "ionic_devlink.h"
 
 /* Supported devices */
 static const struct pci_device_id ionic_id_table[] = {
@@ -212,9 +213,14 @@ static int ionic_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
 		goto err_out_deinit_lifs;
 	}
 
+	err = ionic_devlink_register(ionic);
+	if (err)
+		dev_err(dev, "Cannot register devlink (ignored): %d\n", err);
+
 	return 0;
 
 err_out_deinit_lifs:
+	ionic_devlink_unregister(ionic);
 	ionic_lifs_deinit(ionic);
 err_out_free_lifs:
 	ionic_lifs_free(ionic);
@@ -247,6 +253,7 @@ static void ionic_remove(struct pci_dev *pdev)
 	struct ionic *ionic = pci_get_drvdata(pdev);
 
 	if (ionic) {
+		ionic_devlink_unregister(ionic);
 		ionic_lifs_unregister(ionic);
 		ionic_lifs_deinit(ionic);
 		ionic_lifs_free(ionic);
drivers/net/ethernet/pensando/ionic/ionic_devlink.c new file +89 -0

89 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_devlink.h new file +12 -0
new file mode 100644
index 000000000000..35528884e29f
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+
+#ifndef _IONIC_DEVLINK_H_
+#define _IONIC_DEVLINK_H_
+
+#include <net/devlink.h>
+
+int ionic_devlink_register(struct ionic *ionic);
+void ionic_devlink_unregister(struct ionic *ionic);
+
+#endif /* _IONIC_DEVLINK_H_ */
Signature · 1 line
2.17.1
#44 lore raw

Re: [PATCH v3 net-next 19/19] ionic: Add basic devlink interface

Quoted 145 lines · Mon, Jul 08, 2019 at 09:25:32PM CEST, [email protected] wrote:

Add a devlink interface for access to information that isn't normally available through ethtool or the iplink interface.

Example:

	$ ./devlink -j -p dev info pci/0000:b6:00.0
	{
	    "info": {
		"pci/0000:b6:00.0": {
		    "driver": "ionic",
		    "serial_number": "FLM18420073",
		    "versions": {
			"fixed": {
			    "fw_version": "0.11.0-50",
			    "fw_status": "0x1",
			    "fw_heartbeat": "0x716ce",
			    "asic_type": "0x0",
			    "asic_rev": "0x0"
			}
		    }
		}
	    }
	}

---

drivers/net/ethernet/pensando/ionic/Makefile  |  2 +-
drivers/net/ethernet/pensando/ionic/ionic.h   |  1 +
.../ethernet/pensando/ionic/ionic_bus_pci.c   |  7 ++
.../ethernet/pensando/ionic/ionic_devlink.c   | 89 +++++++++++++++++++
.../ethernet/pensando/ionic/ionic_devlink.h   | 12 +++
5 files changed, 110 insertions(+), 1 deletion(-)

create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_devlink.c create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_devlink.h

drivers/net/ethernet/pensando/ionic/Makefile +1 -1
index 4f3cfbf36c23..ce187c7b33a8 100644
@@ -5,4 +5,4 @@ obj-$(CONFIG_IONIC) := ionic.o
 
 ionic-y := ionic_main.o ionic_bus_pci.o ionic_dev.o ionic_ethtool.o \
 	   ionic_lif.o ionic_rx_filter.o ionic_txrx.o ionic_debugfs.o \
-	   ionic_stats.o
+	   ionic_stats.o ionic_devlink.o
drivers/net/ethernet/pensando/ionic/ionic.h +1 -0
index cd08166f73a9..a0034bc5b4a1 100644
@@ -44,6 +44,7 @@ struct ionic {
 	DECLARE_BITMAP(intrs, INTR_CTRL_REGS_MAX);
 	struct work_struct nb_work;
 	struct notifier_block nb;
+	struct devlink *dl;
 };
 
 struct ionic_admin_ctx {
drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c +5 -0
index 98c12b770c7f..a8c99254489f 100644
@@ -10,6 +10,7 @@
 #include "ionic_bus.h"
 #include "ionic_lif.h"
 #include "ionic_debugfs.h"
+#include "ionic_devlink.h"
 
 /* Supported devices */
 static const struct pci_device_id ionic_id_table[] = {
@@ -212,9 +213,14 @@ static int ionic_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
 		goto err_out_deinit_lifs;
 	}
 
+	err = ionic_devlink_register(ionic);
+	if (err)
+		dev_err(dev, "Cannot register devlink (ignored): %d\n", err);
+
 	return 0;
 

err_out_deinit_lifs:

drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c +1 -0
+	ionic_devlink_unregister(ionic);
	ionic_lifs_deinit(ionic);
err_out_free_lifs:
	ionic_lifs_free(ionic);
drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c +1 -0
@@ -247,6 +253,7 @@ static void ionic_remove(struct pci_dev *pdev)
 	struct ionic *ionic = pci_get_drvdata(pdev);
 
 	if (ionic) {
+		ionic_devlink_unregister(ionic);
 		ionic_lifs_unregister(ionic);
 		ionic_lifs_deinit(ionic);
 		ionic_lifs_free(ionic);
drivers/net/ethernet/pensando/ionic/ionic_devlink.c new file +48 -0
new file mode 100644
index 000000000000..fbbfcdde292f
@@ -0,0 +1,89 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+
+#include <linux/module.h>
+#include <linux/netdevice.h>
+
+#include "ionic.h"
+#include "ionic_bus.h"
+#include "ionic_lif.h"
+#include "ionic_devlink.h"
+
+struct ionic_devlink {
+	struct ionic *ionic;
+};
+
+static int ionic_dl_info_get(struct devlink *dl, struct devlink_info_req *req,
+			     struct netlink_ext_ack *extack)
+{
+	struct ionic *ionic = *(struct ionic **)devlink_priv(dl);
+	struct ionic_dev *idev = &ionic->idev;
+	char buf[16];
+	u32 val;
+
+	devlink_info_driver_name_put(req, DRV_NAME);
+
+	devlink_info_version_fixed_put(req, "fw_version",
+				       idev->dev_info.fw_version);
+
+	val = ioread8(&idev->dev_info_regs->fw_status);
+	snprintf(buf, sizeof(buf), "0x%x", val);
+	devlink_info_version_fixed_put(req, "fw_status", buf);
+
+	val = ioread32(&idev->dev_info_regs->fw_heartbeat);
+	snprintf(buf, sizeof(buf), "0x%x", val);
+	devlink_info_version_fixed_put(req, "fw_heartbeat", buf);
+
+	snprintf(buf, sizeof(buf), "0x%x", idev->dev_info.asic_type);
+	devlink_info_version_fixed_put(req, "asic_type", buf);
+
+	snprintf(buf, sizeof(buf), "0x%x", idev->dev_info.asic_rev);
+	devlink_info_version_fixed_put(req, "asic_rev", buf);
+
+	devlink_info_serial_number_put(req, idev->dev_info.serial_num);
+
+	return 0;
+}
+
+static const struct devlink_ops ionic_dl_ops = {

drivers/net/ethernet/pensando/ionic/ionic_devlink.c line 58

Quoted 10 lines
drivers/net/ethernet/pensando/ionic/ionic_devlink.c new file +10 -0
+	.info_get	= ionic_dl_info_get,
+};
+
+int ionic_devlink_register(struct ionic *ionic)
+{
+	struct devlink *dl;
+	struct ionic **ip;
+	int err;
+
+	dl = devlink_alloc(&ionic_dl_ops, sizeof(struct ionic *));

Oups. Something is wrong with your flow. The devlink alloc is allocating the structure that holds private data (per-device data) for you. This is misuse :/

You are missing one parent device struct apparently.

Oh, I think I see something like it. The unused "struct ionic_devlink".

Quoted 52 lines
drivers/net/ethernet/pensando/ionic/ionic_devlink.c new file +31 -0
+	if (!dl) {
+		dev_warn(ionic->dev, "devlink_alloc failed");
+		return -ENOMEM;
+	}
+
+	ip = (struct ionic **)devlink_priv(dl);
+	*ip = ionic;
+	ionic->dl = dl;
+
+	err = devlink_register(dl, ionic->dev);
+	if (err) {
+		dev_warn(ionic->dev, "devlink_register failed: %d\n", err);
+		goto err_dl_free;
+	}
+
+	return 0;
+
+err_dl_free:
+	ionic->dl = NULL;
+	devlink_free(dl);
+	return err;
+}
+
+void ionic_devlink_unregister(struct ionic *ionic)
+{
+	if (!ionic->dl)
+		return;
+
+	devlink_unregister(ionic->dl);
+	devlink_free(ionic->dl);
+}
drivers/net/ethernet/pensando/ionic/ionic_devlink.h new file +12 -0
new file mode 100644
index 000000000000..35528884e29f
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+
+#ifndef _IONIC_DEVLINK_H_
+#define _IONIC_DEVLINK_H_
+
+#include <net/devlink.h>
+
+int ionic_devlink_register(struct ionic *ionic);
+void ionic_devlink_unregister(struct ionic *ionic);
+
+#endif /* _IONIC_DEVLINK_H_ */
Signature · 1 line
2.17.1
#45 lore raw

Re: [PATCH v3 net-next 19/19] ionic: Add basic devlink interface

Quoted 160 lines · Jiri Pirko
Quoted 145 lines · Mon, Jul 08, 2019 at 09:25:32PM CEST, [email protected] wrote:

Add a devlink interface for access to information that isn't normally available through ethtool or the iplink interface.

Example:

	$ ./devlink -j -p dev info pci/0000:b6:00.0
	{
	    "info": {
		"pci/0000:b6:00.0": {
		    "driver": "ionic",
		    "serial_number": "FLM18420073",
		    "versions": {
			"fixed": {
			    "fw_version": "0.11.0-50",
			    "fw_status": "0x1",
			    "fw_heartbeat": "0x716ce",
			    "asic_type": "0x0",
			    "asic_rev": "0x0"
			}
		    }
		}
	    }
	}

---

drivers/net/ethernet/pensando/ionic/Makefile  |  2 +-
drivers/net/ethernet/pensando/ionic/ionic.h   |  1 +
.../ethernet/pensando/ionic/ionic_bus_pci.c   |  7 ++
.../ethernet/pensando/ionic/ionic_devlink.c   | 89 +++++++++++++++++++
.../ethernet/pensando/ionic/ionic_devlink.h   | 12 +++
5 files changed, 110 insertions(+), 1 deletion(-)

create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_devlink.c create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_devlink.h

drivers/net/ethernet/pensando/ionic/Makefile +1 -1
index 4f3cfbf36c23..ce187c7b33a8 100644
@@ -5,4 +5,4 @@ obj-$(CONFIG_IONIC) := ionic.o
 
 ionic-y := ionic_main.o ionic_bus_pci.o ionic_dev.o ionic_ethtool.o \
 	   ionic_lif.o ionic_rx_filter.o ionic_txrx.o ionic_debugfs.o \
-	   ionic_stats.o
+	   ionic_stats.o ionic_devlink.o
drivers/net/ethernet/pensando/ionic/ionic.h +1 -0
index cd08166f73a9..a0034bc5b4a1 100644
@@ -44,6 +44,7 @@ struct ionic {
 	DECLARE_BITMAP(intrs, INTR_CTRL_REGS_MAX);
 	struct work_struct nb_work;
 	struct notifier_block nb;
+	struct devlink *dl;
 };
 
 struct ionic_admin_ctx {
drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c +5 -0
index 98c12b770c7f..a8c99254489f 100644
@@ -10,6 +10,7 @@
 #include "ionic_bus.h"
 #include "ionic_lif.h"
 #include "ionic_debugfs.h"
+#include "ionic_devlink.h"
 
 /* Supported devices */
 static const struct pci_device_id ionic_id_table[] = {
@@ -212,9 +213,14 @@ static int ionic_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
 		goto err_out_deinit_lifs;
 	}
 
+	err = ionic_devlink_register(ionic);
+	if (err)
+		dev_err(dev, "Cannot register devlink (ignored): %d\n", err);
+
 	return 0;
 

err_out_deinit_lifs:

drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c +1 -0
+	ionic_devlink_unregister(ionic);
	ionic_lifs_deinit(ionic);
err_out_free_lifs:
	ionic_lifs_free(ionic);
drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c +1 -0
@@ -247,6 +253,7 @@ static void ionic_remove(struct pci_dev *pdev)
 	struct ionic *ionic = pci_get_drvdata(pdev);
 
 	if (ionic) {
+		ionic_devlink_unregister(ionic);
 		ionic_lifs_unregister(ionic);
 		ionic_lifs_deinit(ionic);
 		ionic_lifs_free(ionic);
drivers/net/ethernet/pensando/ionic/ionic_devlink.c new file +48 -0
new file mode 100644
index 000000000000..fbbfcdde292f
@@ -0,0 +1,89 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+
+#include <linux/module.h>
+#include <linux/netdevice.h>
+
+#include "ionic.h"
+#include "ionic_bus.h"
+#include "ionic_lif.h"
+#include "ionic_devlink.h"
+
+struct ionic_devlink {
+	struct ionic *ionic;
+};
+
+static int ionic_dl_info_get(struct devlink *dl, struct devlink_info_req *req,
+			     struct netlink_ext_ack *extack)
+{
+	struct ionic *ionic = *(struct ionic **)devlink_priv(dl);
+	struct ionic_dev *idev = &ionic->idev;
+	char buf[16];
+	u32 val;
+
+	devlink_info_driver_name_put(req, DRV_NAME);
+
+	devlink_info_version_fixed_put(req, "fw_version",
+				       idev->dev_info.fw_version);
+
+	val = ioread8(&idev->dev_info_regs->fw_status);
+	snprintf(buf, sizeof(buf), "0x%x", val);
+	devlink_info_version_fixed_put(req, "fw_status", buf);
+
+	val = ioread32(&idev->dev_info_regs->fw_heartbeat);
+	snprintf(buf, sizeof(buf), "0x%x", val);
+	devlink_info_version_fixed_put(req, "fw_heartbeat", buf);
+
+	snprintf(buf, sizeof(buf), "0x%x", idev->dev_info.asic_type);
+	devlink_info_version_fixed_put(req, "asic_type", buf);
+
+	snprintf(buf, sizeof(buf), "0x%x", idev->dev_info.asic_rev);
+	devlink_info_version_fixed_put(req, "asic_rev", buf);
+
+	devlink_info_serial_number_put(req, idev->dev_info.serial_num);
+
+	return 0;
+}
+
+static const struct devlink_ops ionic_dl_ops = {

drivers/net/ethernet/pensando/ionic/ionic_devlink.c line 58

Quoted 10 lines
drivers/net/ethernet/pensando/ionic/ionic_devlink.c new file +10 -0
+	.info_get	= ionic_dl_info_get,
+};
+
+int ionic_devlink_register(struct ionic *ionic)
+{
+	struct devlink *dl;
+	struct ionic **ip;
+	int err;
+
+	dl = devlink_alloc(&ionic_dl_ops, sizeof(struct ionic *));

Oups. Something is wrong with your flow. The devlink alloc is allocating the structure that holds private data (per-device data) for you. This is misuse :/

Quoted 3 lines

You are missing one parent device struct apparently.

Oh, I think I see something like it. The unused "struct ionic_devlink".

If I'm not mistaken, the alloc is only allocating enough for a pointer, not the whole per device struct, and a few lines down from here the pointer to the new devlink struct is assigned to ionic->dl.  This was based on what I found in the qed driver's qed_devlink_register(), and it all seems to work.

That unused struct ionic_devlink does need to go away, it was superfluous after working out a better typecast off of devlink_priv().

I'll remove the unused struct ionic_devlink, but I think the rest is okay.

sln

Quoted 54 lines
Quoted 54 lines
drivers/net/ethernet/pensando/ionic/ionic_devlink.c new file +31 -0
+	if (!dl) {
+		dev_warn(ionic->dev, "devlink_alloc failed");
+		return -ENOMEM;
+	}
+
+	ip = (struct ionic **)devlink_priv(dl);
+	*ip = ionic;
+	ionic->dl = dl;
+
+	err = devlink_register(dl, ionic->dev);
+	if (err) {
+		dev_warn(ionic->dev, "devlink_register failed: %d\n", err);
+		goto err_dl_free;
+	}
+
+	return 0;
+
+err_dl_free:
+	ionic->dl = NULL;
+	devlink_free(dl);
+	return err;
+}
+
+void ionic_devlink_unregister(struct ionic *ionic)
+{
+	if (!ionic->dl)
+		return;
+
+	devlink_unregister(ionic->dl);
+	devlink_free(ionic->dl);
+}
drivers/net/ethernet/pensando/ionic/ionic_devlink.h new file +12 -0
new file mode 100644
index 000000000000..35528884e29f
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+
+#ifndef _IONIC_DEVLINK_H_
+#define _IONIC_DEVLINK_H_
+
+#include <net/devlink.h>
+
+int ionic_devlink_register(struct ionic *ionic);
+void ionic_devlink_unregister(struct ionic *ionic);
+
+#endif /* _IONIC_DEVLINK_H_ */
Signature · 1 line
2.17.1
#46 lore raw

Re: [PATCH v3 net-next 19/19] ionic: Add basic devlink interface

Quoted 161 lines · Mon, Jul 08, 2019 at 09:58:09PM CEST, [email protected] wrote:
Quoted 160 lines · Jiri Pirko
Quoted 145 lines · Mon, Jul 08, 2019 at 09:25:32PM CEST, [email protected] wrote:

Add a devlink interface for access to information that isn't normally available through ethtool or the iplink interface.

Example:

	$ ./devlink -j -p dev info pci/0000:b6:00.0
	{
	    "info": {
		"pci/0000:b6:00.0": {
		    "driver": "ionic",
		    "serial_number": "FLM18420073",
		    "versions": {
			"fixed": {
			    "fw_version": "0.11.0-50",
			    "fw_status": "0x1",
			    "fw_heartbeat": "0x716ce",
			    "asic_type": "0x0",
			    "asic_rev": "0x0"
			}
		    }
		}
	    }
	}

---

drivers/net/ethernet/pensando/ionic/Makefile  |  2 +-
drivers/net/ethernet/pensando/ionic/ionic.h   |  1 +
.../ethernet/pensando/ionic/ionic_bus_pci.c   |  7 ++
.../ethernet/pensando/ionic/ionic_devlink.c   | 89 +++++++++++++++++++
.../ethernet/pensando/ionic/ionic_devlink.h   | 12 +++
5 files changed, 110 insertions(+), 1 deletion(-)

create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_devlink.c create mode 100644 drivers/net/ethernet/pensando/ionic/ionic_devlink.h

drivers/net/ethernet/pensando/ionic/Makefile +1 -1
index 4f3cfbf36c23..ce187c7b33a8 100644
@@ -5,4 +5,4 @@ obj-$(CONFIG_IONIC) := ionic.o
 
 ionic-y := ionic_main.o ionic_bus_pci.o ionic_dev.o ionic_ethtool.o \
 	   ionic_lif.o ionic_rx_filter.o ionic_txrx.o ionic_debugfs.o \
-	   ionic_stats.o
+	   ionic_stats.o ionic_devlink.o
drivers/net/ethernet/pensando/ionic/ionic.h +1 -0
index cd08166f73a9..a0034bc5b4a1 100644
@@ -44,6 +44,7 @@ struct ionic {
 	DECLARE_BITMAP(intrs, INTR_CTRL_REGS_MAX);
 	struct work_struct nb_work;
 	struct notifier_block nb;
+	struct devlink *dl;
 };
 
 struct ionic_admin_ctx {
drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c +5 -0
index 98c12b770c7f..a8c99254489f 100644
@@ -10,6 +10,7 @@
 #include "ionic_bus.h"
 #include "ionic_lif.h"
 #include "ionic_debugfs.h"
+#include "ionic_devlink.h"
 
 /* Supported devices */
 static const struct pci_device_id ionic_id_table[] = {
@@ -212,9 +213,14 @@ static int ionic_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
 		goto err_out_deinit_lifs;
 	}
 
+	err = ionic_devlink_register(ionic);
+	if (err)
+		dev_err(dev, "Cannot register devlink (ignored): %d\n", err);
+
 	return 0;
 

err_out_deinit_lifs:

drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c +1 -0
+	ionic_devlink_unregister(ionic);
	ionic_lifs_deinit(ionic);
err_out_free_lifs:
	ionic_lifs_free(ionic);
drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c +1 -0
@@ -247,6 +253,7 @@ static void ionic_remove(struct pci_dev *pdev)
 	struct ionic *ionic = pci_get_drvdata(pdev);
 
 	if (ionic) {
+		ionic_devlink_unregister(ionic);
 		ionic_lifs_unregister(ionic);
 		ionic_lifs_deinit(ionic);
 		ionic_lifs_free(ionic);
drivers/net/ethernet/pensando/ionic/ionic_devlink.c new file +48 -0
new file mode 100644
index 000000000000..fbbfcdde292f
@@ -0,0 +1,89 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+
+#include <linux/module.h>
+#include <linux/netdevice.h>
+
+#include "ionic.h"
+#include "ionic_bus.h"
+#include "ionic_lif.h"
+#include "ionic_devlink.h"
+
+struct ionic_devlink {
+	struct ionic *ionic;
+};
+
+static int ionic_dl_info_get(struct devlink *dl, struct devlink_info_req *req,
+			     struct netlink_ext_ack *extack)
+{
+	struct ionic *ionic = *(struct ionic **)devlink_priv(dl);
+	struct ionic_dev *idev = &ionic->idev;
+	char buf[16];
+	u32 val;
+
+	devlink_info_driver_name_put(req, DRV_NAME);
+
+	devlink_info_version_fixed_put(req, "fw_version",
+				       idev->dev_info.fw_version);
+
+	val = ioread8(&idev->dev_info_regs->fw_status);
+	snprintf(buf, sizeof(buf), "0x%x", val);
+	devlink_info_version_fixed_put(req, "fw_status", buf);
+
+	val = ioread32(&idev->dev_info_regs->fw_heartbeat);
+	snprintf(buf, sizeof(buf), "0x%x", val);
+	devlink_info_version_fixed_put(req, "fw_heartbeat", buf);
+
+	snprintf(buf, sizeof(buf), "0x%x", idev->dev_info.asic_type);
+	devlink_info_version_fixed_put(req, "asic_type", buf);
+
+	snprintf(buf, sizeof(buf), "0x%x", idev->dev_info.asic_rev);
+	devlink_info_version_fixed_put(req, "asic_rev", buf);
+
+	devlink_info_serial_number_put(req, idev->dev_info.serial_num);
+
+	return 0;
+}
+
+static const struct devlink_ops ionic_dl_ops = {

drivers/net/ethernet/pensando/ionic/ionic_devlink.c line 58

Quoted 10 lines
drivers/net/ethernet/pensando/ionic/ionic_devlink.c new file +10 -0
+	.info_get	= ionic_dl_info_get,
+};
+
+int ionic_devlink_register(struct ionic *ionic)
+{
+	struct devlink *dl;
+	struct ionic **ip;
+	int err;
+
+	dl = devlink_alloc(&ionic_dl_ops, sizeof(struct ionic *));

Oups. Something is wrong with your flow. The devlink alloc is allocating the structure that holds private data (per-device data) for you. This is misuse :/

Quoted 8 lines
Quoted 3 lines

You are missing one parent device struct apparently.

Oh, I think I see something like it. The unused "struct ionic_devlink".

If I'm not mistaken, the alloc is only allocating enough for a pointer, not the whole per device struct, and a few lines down from here the pointer to the new devlink struct is assigned to ionic->dl.  This was based on what I found in the qed driver's qed_devlink_register(), and it all seems to work.

I'm not saying your code won't work. What I say is that you should have a struct for device that would be allocated by devlink_alloc()

The ionic struct should be associated with devlink_port. That you are missing too.

Quoted 63 lines

That unused struct ionic_devlink does need to go away, it was superfluous after working out a better typecast off of devlink_priv().

I'll remove the unused struct ionic_devlink, but I think the rest is okay.

sln

Quoted 54 lines
Quoted 54 lines
drivers/net/ethernet/pensando/ionic/ionic_devlink.c new file +31 -0
+	if (!dl) {
+		dev_warn(ionic->dev, "devlink_alloc failed");
+		return -ENOMEM;
+	}
+
+	ip = (struct ionic **)devlink_priv(dl);
+	*ip = ionic;
+	ionic->dl = dl;
+
+	err = devlink_register(dl, ionic->dev);
+	if (err) {
+		dev_warn(ionic->dev, "devlink_register failed: %d\n", err);
+		goto err_dl_free;
+	}
+
+	return 0;
+
+err_dl_free:
+	ionic->dl = NULL;
+	devlink_free(dl);
+	return err;
+}
+
+void ionic_devlink_unregister(struct ionic *ionic)
+{
+	if (!ionic->dl)
+		return;
+
+	devlink_unregister(ionic->dl);
+	devlink_free(ionic->dl);
+}
drivers/net/ethernet/pensando/ionic/ionic_devlink.h new file +12 -0
new file mode 100644
index 000000000000..35528884e29f
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+
+#ifndef _IONIC_DEVLINK_H_
+#define _IONIC_DEVLINK_H_
+
+#include <net/devlink.h>
+
+int ionic_devlink_register(struct ionic *ionic);
+void ionic_devlink_unregister(struct ionic *ionic);
+
+#endif /* _IONIC_DEVLINK_H_ */
Signature · 1 line
2.17.1
#47 lore raw

Re: [PATCH v3 net-next 19/19] ionic: Add basic devlink interface

Quoted 22 lines · Jiri Pirko
Quoted 21 lines · Mon, Jul 08, 2019 at 09:58:09PM CEST, [email protected] wrote:
Quoted 20 lines · Jiri Pirko

drivers/net/ethernet/pensando/ionic/ionic_devlink.c line 58

Quoted 13 lines · Mon, Jul 08, 2019 at 09:25:32PM CEST, [email protected] wrote:
drivers/net/ethernet/pensando/ionic/ionic_devlink.c new file +12 -0
+
+static const struct devlink_ops ionic_dl_ops = {
+	.info_get	= ionic_dl_info_get,
+};
+
+int ionic_devlink_register(struct ionic *ionic)
+{
+	struct devlink *dl;
+	struct ionic **ip;
+	int err;
+
+	dl = devlink_alloc(&ionic_dl_ops, sizeof(struct ionic *));

Oups. Something is wrong with your flow. The devlink alloc is allocating the structure that holds private data (per-device data) for you. This is misuse :/

You are missing one parent device struct apparently.

Quoted 7 lines
Quoted 5 lines
Quoted 1 line

Oh, I think I see something like it. The unused "struct ionic_devlink".

If I'm not mistaken, the alloc is only allocating enough for a pointer, not the whole per device struct, and a few lines down from here the pointer to the new devlink struct is assigned to ionic->dl.  This was based on what I found in the qed driver's qed_devlink_register(), and it all seems to work.

I'm not saying your code won't work. What I say is that you should have a struct for device that would be allocated by devlink_alloc()

Is there a particular reason why?  I appreciate that devlink_alloc() can give you this device specific space, just as alloc_etherdev_mq() can, but is there a specific reason why this should be used instead of setting up simply a pointer to a space that has already been allocated?  There are several drivers that are using it the way I've setup here, which happened to be the first examples I followed - are they doing something different that makes this valid for them?

Quoted 3 lines

The ionic struct should be associated with devlink_port. That you are missing too.

We don't support any of devlink_port features at this point, just the simple device information.

sln

Quoted 62 lines
Quoted 62 lines

That unused struct ionic_devlink does need to go away, it was superfluous after working out a better typecast off of devlink_priv().

I'll remove the unused struct ionic_devlink, but I think the rest is okay.

sln

Quoted 53 lines
Quoted 53 lines
drivers/net/ethernet/pensando/ionic/ionic_devlink.c new file +31 -0
+	if (!dl) {
+		dev_warn(ionic->dev, "devlink_alloc failed");
+		return -ENOMEM;
+	}
+
+	ip = (struct ionic **)devlink_priv(dl);
+	*ip = ionic;
+	ionic->dl = dl;
+
+	err = devlink_register(dl, ionic->dev);
+	if (err) {
+		dev_warn(ionic->dev, "devlink_register failed: %d\n", err);
+		goto err_dl_free;
+	}
+
+	return 0;
+
+err_dl_free:
+	ionic->dl = NULL;
+	devlink_free(dl);
+	return err;
+}
+
+void ionic_devlink_unregister(struct ionic *ionic)
+{
+	if (!ionic->dl)
+		return;
+
+	devlink_unregister(ionic->dl);
+	devlink_free(ionic->dl);
+}
drivers/net/ethernet/pensando/ionic/ionic_devlink.h new file +12 -0
new file mode 100644
index 000000000000..35528884e29f
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+
+#ifndef _IONIC_DEVLINK_H_
+#define _IONIC_DEVLINK_H_
+
+#include <net/devlink.h>
+
+int ionic_devlink_register(struct ionic *ionic);
+void ionic_devlink_unregister(struct ionic *ionic);
+
+#endif /* _IONIC_DEVLINK_H_ */
Signature · 1 line
2.17.1
#48 lore raw

Re: [PATCH v3 net-next 19/19] ionic: Add basic devlink interface

Quoted 28 lines · Tue, Jul 09, 2019 at 12:58:00AM CEST, [email protected] wrote:
Quoted 27 lines · Jiri Pirko
Quoted 26 lines · Mon, Jul 08, 2019 at 09:58:09PM CEST, [email protected] wrote:
Quoted 18 lines · Jiri Pirko

drivers/net/ethernet/pensando/ionic/ionic_devlink.c line 58

Quoted 13 lines · Mon, Jul 08, 2019 at 09:25:32PM CEST, [email protected] wrote:
drivers/net/ethernet/pensando/ionic/ionic_devlink.c new file +12 -0
+
+static const struct devlink_ops ionic_dl_ops = {
+	.info_get	= ionic_dl_info_get,
+};
+
+int ionic_devlink_register(struct ionic *ionic)
+{
+	struct devlink *dl;
+	struct ionic **ip;
+	int err;
+
+	dl = devlink_alloc(&ionic_dl_ops, sizeof(struct ionic *));

Oups. Something is wrong with your flow. The devlink alloc is allocating the structure that holds private data (per-device data) for you. This is misuse :/

Quoted 3 lines

You are missing one parent device struct apparently.

Oh, I think I see something like it. The unused "struct ionic_devlink".

If I'm not mistaken, the alloc is only allocating enough for a pointer, not the whole per device struct, and a few lines down from here the pointer to the new devlink struct is assigned to ionic->dl.  This was based on what I found in the qed driver's qed_devlink_register(), and it all seems to work.

Quoted 5 lines
Quoted 2 lines

I'm not saying your code won't work. What I say is that you should have a struct for device that would be allocated by devlink_alloc()

Is there a particular reason why?  I appreciate that devlink_alloc() can give you this device specific space, just as alloc_etherdev_mq() can, but is there

Yes. Devlink manipulates with the whole device. However, alloc_etherdev_mq() allocates only net_device. These are 2 different things. devlink port relates 1:1 to net_device. However, devlink instance can have multiple ports. What I say is do it correctly.

Quoted 5 lines

a specific reason why this should be used instead of setting up simply a pointer to a space that has already been allocated?  There are several drivers that are using it the way I've setup here, which happened to be the first examples I followed - are they doing something different that makes this valid for them?

Nope. I'll look at that and fix.

Quoted 7 lines
Quoted 4 lines

The ionic struct should be associated with devlink_port. That you are missing too.

We don't support any of devlink_port features at this point, just the simple device information.

No problem, you can still register devlink_port. You don't have to do much in order to do so.

Quoted 66 lines

sln

Quoted 62 lines
Quoted 62 lines

That unused struct ionic_devlink does need to go away, it was superfluous after working out a better typecast off of devlink_priv().

I'll remove the unused struct ionic_devlink, but I think the rest is okay.

sln

Quoted 53 lines
Quoted 53 lines
drivers/net/ethernet/pensando/ionic/ionic_devlink.c new file +31 -0
+	if (!dl) {
+		dev_warn(ionic->dev, "devlink_alloc failed");
+		return -ENOMEM;
+	}
+
+	ip = (struct ionic **)devlink_priv(dl);
+	*ip = ionic;
+	ionic->dl = dl;
+
+	err = devlink_register(dl, ionic->dev);
+	if (err) {
+		dev_warn(ionic->dev, "devlink_register failed: %d\n", err);
+		goto err_dl_free;
+	}
+
+	return 0;
+
+err_dl_free:
+	ionic->dl = NULL;
+	devlink_free(dl);
+	return err;
+}
+
+void ionic_devlink_unregister(struct ionic *ionic)
+{
+	if (!ionic->dl)
+		return;
+
+	devlink_unregister(ionic->dl);
+	devlink_free(ionic->dl);
+}
drivers/net/ethernet/pensando/ionic/ionic_devlink.h new file +12 -0
new file mode 100644
index 000000000000..35528884e29f
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+
+#ifndef _IONIC_DEVLINK_H_
+#define _IONIC_DEVLINK_H_
+
+#include <net/devlink.h>
+
+int ionic_devlink_register(struct ionic *ionic);
+void ionic_devlink_unregister(struct ionic *ionic);
+
+#endif /* _IONIC_DEVLINK_H_ */
Signature · 1 line
2.17.1
#49 lore raw

Re: [PATCH v3 net-next 19/19] ionic: Add basic devlink interface

Quoted 28 lines · Jiri Pirko
Quoted 27 lines · Tue, Jul 09, 2019 at 12:58:00AM CEST, [email protected] wrote:
Quoted 26 lines · Jiri Pirko
Quoted 25 lines · Mon, Jul 08, 2019 at 09:58:09PM CEST, [email protected] wrote:
Quoted 17 lines · Jiri Pirko

drivers/net/ethernet/pensando/ionic/ionic_devlink.c line 58

Quoted 12 lines · Mon, Jul 08, 2019 at 09:25:32PM CEST, [email protected] wrote:
drivers/net/ethernet/pensando/ionic/ionic_devlink.c new file +12 -0
+
+static const struct devlink_ops ionic_dl_ops = {
+	.info_get	= ionic_dl_info_get,
+};
+
+int ionic_devlink_register(struct ionic *ionic)
+{
+	struct devlink *dl;
+	struct ionic **ip;
+	int err;
+
+	dl = devlink_alloc(&ionic_dl_ops, sizeof(struct ionic *));

Oups. Something is wrong with your flow. The devlink alloc is allocating the structure that holds private data (per-device data) for you. This is misuse :/

Quoted 3 lines

You are missing one parent device struct apparently.

Oh, I think I see something like it. The unused "struct ionic_devlink".

If I'm not mistaken, the alloc is only allocating enough for a pointer, not the whole per device struct, and a few lines down from here the pointer to the new devlink struct is assigned to ionic->dl.  This was based on what I found in the qed driver's qed_devlink_register(), and it all seems to work.

Quoted 8 lines
Quoted 4 lines
Quoted 2 lines

I'm not saying your code won't work. What I say is that you should have a struct for device that would be allocated by devlink_alloc()

Is there a particular reason why?  I appreciate that devlink_alloc() can give you this device specific space, just as alloc_etherdev_mq() can, but is there

Yes. Devlink manipulates with the whole device. However, alloc_etherdev_mq() allocates only net_device. These are 2 different things. devlink port relates 1:1 to net_device. However, devlink instance can have multiple ports. What I say is do it correctly.

So what you are saying is that anyone who wants to add even the smallest devlink feature to their driver needs to rework their basic device memory setup to do it the devlink way.  I can see where some folks may have a problem with this.

Quoted 10 lines
Quoted 7 lines

a specific reason why this should be used instead of setting up simply a pointer to a space that has already been allocated?  There are several drivers that are using it the way I've setup here, which happened to be the first examples I followed - are they doing something different that makes this valid for them?

Nope. I'll look at that and fix.

Quoted 6 lines
Quoted 4 lines
Quoted 2 lines

The ionic struct should be associated with devlink_port. That you are missing too.

We don't support any of devlink_port features at this point, just the simple device information.

No problem, you can still register devlink_port. You don't have to do much in order to do so.

Is there any write-up to help guide developers new to devlink in using the interface correctly?  I haven't found much yet, but perhaps I've missed something.  The manpages are somewhat useful in showing what the user might do, but they really don't help much in guiding the developer through these details.

sln

#50 lore raw

Re: [PATCH v3 net-next 19/19] ionic: Add basic devlink interface

Quoted 38 lines · Tue, Jul 09, 2019 at 09:13:53PM CEST, [email protected] wrote:
Quoted 28 lines · Jiri Pirko
Quoted 27 lines · Tue, Jul 09, 2019 at 12:58:00AM CEST, [email protected] wrote:
Quoted 26 lines · Jiri Pirko
Quoted 25 lines · Mon, Jul 08, 2019 at 09:58:09PM CEST, [email protected] wrote:
Quoted 17 lines · Jiri Pirko

Review

Quoted 12 lines · Mon, Jul 08, 2019 at 09:25:32PM CEST, [email protected] wrote:
(quoted diff) +12 -0
+
+static const struct devlink_ops ionic_dl_ops = {
+	.info_get	= ionic_dl_info_get,
+};
+
+int ionic_devlink_register(struct ionic *ionic)
+{
+	struct devlink *dl;
+	struct ionic **ip;
+	int err;
+
+	dl = devlink_alloc(&ionic_dl_ops, sizeof(struct ionic *));

Oups. Something is wrong with your flow. The devlink alloc is allocating the structure that holds private data (per-device data) for you. This is misuse :/

Quoted 3 lines

You are missing one parent device struct apparently.

Oh, I think I see something like it. The unused "struct ionic_devlink".

If I'm not mistaken, the alloc is only allocating enough for a pointer, not the whole per device struct, and a few lines down from here the pointer to the new devlink struct is assigned to ionic->dl.  This was based on what I found in the qed driver's qed_devlink_register(), and it all seems to work.

Quoted 8 lines
Quoted 4 lines
Quoted 2 lines

I'm not saying your code won't work. What I say is that you should have a struct for device that would be allocated by devlink_alloc()

Is there a particular reason why?  I appreciate that devlink_alloc() can give you this device specific space, just as alloc_etherdev_mq() can, but is there

Yes. Devlink manipulates with the whole device. However, alloc_etherdev_mq() allocates only net_device. These are 2 different things. devlink port relates 1:1 to net_device. However, devlink instance can have multiple ports. What I say is do it correctly.

Quoted 4 lines

So what you are saying is that anyone who wants to add even the smallest devlink feature to their driver needs to rework their basic device memory setup to do it the devlink way.  I can see where some folks may have a problem with this.

It's just about having a structure to hold device data. You don't have to rework anything, just add this small one.

Quoted 15 lines
Quoted 15 lines
Quoted 8 lines

a specific reason why this should be used instead of setting up simply a pointer to a space that has already been allocated?  There are several drivers that are using it the way I've setup here, which happened to be the first examples I followed - are they doing something different that makes this valid for them?

Nope. I'll look at that and fix.

Quoted 4 lines
Quoted 2 lines

The ionic struct should be associated with devlink_port. That you are missing too.

We don't support any of devlink_port features at this point, just the simple device information.

Quoted 8 lines
Quoted 2 lines

No problem, you can still register devlink_port. You don't have to do much in order to do so.

Is there any write-up to help guide developers new to devlink in using the interface correctly?  I haven't found much yet, but perhaps I've missed something.  The manpages are somewhat useful in showing what the user might do, but they really don't help much in guiding the developer through these details.

That is not job of a manpage. See the rest of the code to get inspired.

Quoted 3 lines

sln

#51 lore raw

Re: [PATCH v3 net-next 19/19] ionic: Add basic devlink interface

Quoted 18 lines · Jiri Pirko
Quoted 17 lines · Tue, Jul 09, 2019 at 09:13:53PM CEST, [email protected] wrote:
Quoted 16 lines · Jiri Pirko
Quoted 11 lines · Tue, Jul 09, 2019 at 12:58:00AM CEST, [email protected] wrote:
Quoted 8 lines · Jiri Pirko
Quoted 4 lines · Mon, Jul 08, 2019 at 09:58:09PM CEST, [email protected] wrote:

If I'm not mistaken, the alloc is only allocating enough for a pointer, not the whole per device struct, and a few lines down from here the pointer to the new devlink struct is assigned to ionic->dl.  This was based on what I found in the qed driver's qed_devlink_register(), and it all seems to work.

I'm not saying your code won't work. What I say is that you should have a struct for device that would be allocated by devlink_alloc()

Is there a particular reason why?  I appreciate that devlink_alloc() can give you this device specific space, just as alloc_etherdev_mq() can, but is there

Yes. Devlink manipulates with the whole device. However, alloc_etherdev_mq() allocates only net_device. These are 2 different things. devlink port relates 1:1 to net_device. However, devlink instance can have multiple ports. What I say is do it correctly.

Quoted 6 lines
Quoted 4 lines

So what you are saying is that anyone who wants to add even the smallest devlink feature to their driver needs to rework their basic device memory setup to do it the devlink way.  I can see where some folks may have a problem with this.

It's just about having a structure to hold device data. You don't have to rework anything, just add this small one.

Well, there's a bit of logic rework to and a little data twiddling - not too bad in our case.  Others may not be thrilled depending on how they've already implemented their drivers.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines

The ionic struct should be associated with devlink_port. That you are missing too.

We don't support any of devlink_port features at this point, just the simple device information.

Quoted 9 lines
Quoted 7 lines
Quoted 2 lines

No problem, you can still register devlink_port. You don't have to do much in order to do so.

Is there any write-up to help guide developers new to devlink in using the interface correctly?  I haven't found much yet, but perhaps I've missed something.  The manpages are somewhat useful in showing what the user might do, but they really don't help much in guiding the developer through these details.

That is not job of a manpage. See the rest of the code to get inspired.

Sure, we should all be able to poke through the code and figure out the basics - "use the Force, read the source" - but as software engineers we should be including some bits of documentation to help those new to the feature to steer away from pitfalls and use the feature correctly.  We're all busy with our own projects and only have limited time to dig into and understand someone else's code; if there's not a guide, we'll do what we can to get it working and then move on, with no guarantee that we followed the original intent.

There's a Documentation page on the devlink-health feature, and a brief bit on devlink-params, but I haven't seen anything yet that spells out the "proper" way to use the devlink framework.  Of course, the open-source spirit is for me to scratch my own itch and take care of the need myself: I'd be happy to get a brief doc started, but if the original developers can take a few minutes to at least sketch some notes down about important bits like "the device struct should be associated with devlink_port" and why it should, then we have a chance at saving a lot of other people's time, and perhaps we can fill out the details correctly and not miss something important.

sln

#52 lore raw

Re: [PATCH v3 net-next 19/19] ionic: Add basic devlink interface

In reply to Shannon Nelson
Quoted 4 lines · Shannon Nelson

Add a devlink interface for access to information that isn't normally available through ethtool or the iplink interface.

Example:

Quoted 9 lines
	$ ./devlink -j -p dev info pci/0000:b6:00.0
	{
	    "info": {
		"pci/0000:b6:00.0": {
		    "driver": "ionic",
		    "serial_number": "FLM18420073",
		    "versions": {
			"fixed": {
			    "fw_version": "0.11.0-50",

Hm. Fixed is for hardware components. Seeing FW version reported as fixed seems counter intuitive. You probably want "running"?

Quoted 1 line
			    "fw_status": "0x1",

I don't think this is the right interface to report status-like information. Perhaps devlink health reporters?

Quoted 1 line
			    "fw_heartbeat": "0x716ce",

Ditto, perhaps best to report it in health stuff?

Quoted 2 lines
			    "asic_type": "0x0",
			    "asic_rev": "0x0"

These seem like legit "fixed" versions 👍

Quoted 7 lines
			}
		    }
		}
	    }
	}

Isn't this a new patch? Perhaps you'd be best off upstreaming the first batch of support and add features later? It'd be easier on reviewers so we don't have to keep re-checking the first 16 patches..

#53 lore raw

Re: [PATCH v3 net-next 19/19] ionic: Add basic devlink interface

Quoted 14 lines · Jakub Kicinski
Quoted 4 lines · Shannon Nelson

Add a devlink interface for access to information that isn't normally available through ethtool or the iplink interface.

Example:

Quoted 9 lines
	$ ./devlink -j -p dev info pci/0000:b6:00.0
	{
	    "info": {
		"pci/0000:b6:00.0": {
		    "driver": "ionic",
		    "serial_number": "FLM18420073",
		    "versions": {
			"fixed": {
			    "fw_version": "0.11.0-50",
Quoted 2 lines

Hm. Fixed is for hardware components. Seeing FW version reported as fixed seems counter intuitive. You probably want "running"?

Sure.

Quoted 7 lines
Quoted 2 lines
			    "fw_status": "0x1",

I don't think this is the right interface to report status-like information. Perhaps devlink health reporters?

Quoted 1 line
			    "fw_heartbeat": "0x716ce",

Ditto, perhaps best to report it in health stuff?

I haven't dug too far into the health stuff, but on the surface it looks like a lot of infrastructure for a couple of simple values. I'm tempted to put them back into debugfs for the moment rather than add that much more interface goo.

Quoted 11 lines
Quoted 3 lines
			    "asic_type": "0x0",
			    "asic_rev": "0x0"

These seem like legit "fixed" versions 👍

Quoted 6 lines
			}
		    }
		}
	    }
	}
Quoted 4 lines
Quoted 1 line

Isn't this a new patch? Perhaps you'd be best off upstreaming the first batch of support and add features later? It'd be easier on reviewers so we don't have to keep re-checking the first 16 patches..

Yes, and I commented about this in v2 notes: in the tension between trying to address comments, keep the line count down, keep the basic feature set, and keep the patches self-consistent and simple, I added this one patch for the devlink goodies that were requested. At least the total line count went down.

sln

maintainer
#54 lore raw
In reply to Shannon Nelson

From: Shannon Nelson <[email protected]>
Date: Mon, 8 Jul 2019 12:25:13 -0700

Quoted 2 lines

This is a patch series that adds the ionic driver, supporting the Pensando ethernet device.

...

I think with the review comments and feedback still coming in you will have to wait until the next merge window, sorry.

#55 lore raw
Quoted 9 lines · David Miller

From: Shannon Nelson <[email protected]>
Date: Mon, 8 Jul 2019 12:25:13 -0700

Quoted 2 lines

This is a patch series that adds the ionic driver, supporting the Pensando ethernet device.

...

I think with the review comments and feedback still coming in you will have to wait until the next merge window, sorry.

Yep, that's what I was expecting - I'll have another patchset version ready by then.

Cheers,
sln