Add Mellanox BlueField Gigabit Ethernet driver
Subsystems (2)
PATCH v6 1/1 Add Mellanox BlueField Gigabit Ethernet driver
This patch adds build and driver logic for the "mlxbf_gige" Ethernet driver from Mellanox Technologies. The second generation BlueField SoC from Mellanox supports an out-of-band GigaBit Ethernet management port to the Arm subsystem. This driver supports TCP/IP network connectivity for that port, and provides back-end routines to handle basic ethtool requests.
The driver interfaces to the Gigabit Ethernet block of BlueField SoC via MMIO accesses to registers, which contain control information or pointers describing transmit and receive resources. There is a single transmit queue, and the port supports transmit ring sizes of 4 to 256 entries. There is a single receive queue, and the port supports receive ring sizes of 32 to 32K entries. The transmit and receive rings are allocated from DMA coherent memory. There is a 16-bit producer and consumer index per ring to denote software ownership and hardware ownership, respectively.
The main driver logic such as probe(), remove(), and netdev ops are in "mlxbf_gige_main.c". Logic in "mlxbf_gige_rx.c" and "mlxbf_gige_tx.c" handles the packet processing for receive and transmit respectively.
The logic in "mlxbf_gige_ethtool.c" supports the handling of some basic ethtool requests: get driver info, get ring parameters, get registers, and get statistics.
The logic in "mlxbf_gige_mdio.c" is the driver controlling the Mellanox BlueField hardware that interacts with a PHY device via MDIO/MDC pins. This driver does the following:
- At driver probe time, it configures several BlueField MDIO parameters such as sample rate, full drive, voltage and MDC
- It defines functions to read and write MDIO registers and registers the MDIO bus.
- It defines the phy interrupt handler reporting a link up/down status change
- This driver's probe is invoked from the main driver logic while the phy interrupt handler is registered in ndo_open.
Driver limitations
- Only supports 1Gbps speed
- Only supports GMII protocol
- Supports maximum packet size of 2KB
- Does not support scatter-gather buffering
Testing
- Successful build of kernel for ARM64, ARM32, X86_64
- Tested ARM64 build on FastModels & Palladium
- Tested ARM64 build on several Mellanox boards that are built with the BlueField-2 SoC. The testing includes coverage in the areas of networking (e.g. ping, iperf, ifconfig, route), file transfers (e.g. SCP), and various ethtool options relevant to this driver.
v5 -> v6
Fixed use of COMPILE_TEST for ARM32 build; changed driver to not depend on CONFIG_ACPI for ARM32 build
v4 -> v5
Created a separate interrupt controller for the GPIO PHY interrupt and as a result, the GIGE driver no longer depends on GPIO driver Updated the logic in mlxbf_gige_adjust_link() to store the negotiated pause settings into the driver's private settings.
Modified logic to only change enable bit in RX_DMA register Changed logic to only map and unmap the actual length of the TX SKB, instead using the default size.
Added better error handling to open() method
Modified receive packet logic to use polarity bit to signify ownership (software vs. hardware) of the RX CQE slot
v3 -> v4
Main driver module broken out into rx, tx, intr, and ethtool modules Removed some GPIO PHY interrupt logic; moved to GPIO_MLXBF2 driver
v2 -> v3
Added logic to handle PHY link up/down interrupts
Use streaming DMA mapping for packet buffers
Changed logic to use standard iopoll methods
Changed PHY logic to not allow C45 transactions
Enhanced the error handling in open() method
Enhanced start_xmit() method to use xmit_more mechanism Added support for ndo_get_stats64
Removed standard stats from "ethtool -S" output
v1 -> v2:
Fixed all warnings raised by "make C=1" and "make W=1"
a) Changed logic in mlxbf_gige_rx_deinit() and mlxbf_gige_tx_deinit() to initialize relevant pointers as NULL, not 0
b) Change mlxbf_gige_get_mac_rx_filter() to return void, as this function's return status is not used by caller
c) Fixed type definition of "buff" in mlxbf_gige_get_regs()
--- drivers/net/ethernet/mellanox/Kconfig | 1 + drivers/net/ethernet/mellanox/Makefile | 1 + .../net/ethernet/mellanox/mlxbf_gige/Kconfig | 13 + .../net/ethernet/mellanox/mlxbf_gige/Makefile | 11 + .../ethernet/mellanox/mlxbf_gige/mlxbf_gige.h | 193 ++++++++ .../mellanox/mlxbf_gige/mlxbf_gige_ethtool.c | 140 ++++++ .../mellanox/mlxbf_gige/mlxbf_gige_gpio.c | 212 ++++++++ .../mellanox/mlxbf_gige/mlxbf_gige_intr.c | 143 ++++++ .../mellanox/mlxbf_gige/mlxbf_gige_main.c | 459 ++++++++++++++++++ .../mellanox/mlxbf_gige/mlxbf_gige_mdio.c | 187 +++++++ .../mellanox/mlxbf_gige/mlxbf_gige_regs.h | 78 +++ .../mellanox/mlxbf_gige/mlxbf_gige_rx.c | 318 ++++++++++++ .../mellanox/mlxbf_gige/mlxbf_gige_tx.c | 284 +++++++++++ 13 files changed, 2040 insertions(+) create mode 100644 drivers/net/ethernet/mellanox/mlxbf_gige/Kconfig create mode 100644 drivers/net/ethernet/mellanox/mlxbf_gige/Makefile create mode 100644 drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige.h create mode 100644 drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_ethtool.c create mode 100644 drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_gpio.c create mode 100644 drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_intr.c create mode 100644 drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_main.c create mode 100644 drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_mdio.c create mode 100644 drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_regs.h create mode 100644 drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_rx.c create mode 100644 drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_tx.c
drivers/net/ethernet/mellanox/Kconfig +1 -0
@@ -22,5 +22,6 @@ source "drivers/net/ethernet/mellanox/mlx4/Kconfig" source "drivers/net/ethernet/mellanox/mlx5/core/Kconfig" source "drivers/net/ethernet/mellanox/mlxsw/Kconfig" source "drivers/net/ethernet/mellanox/mlxfw/Kconfig" +source "drivers/net/ethernet/mellanox/mlxbf_gige/Kconfig" endif # NET_VENDOR_MELLANOX
drivers/net/ethernet/mellanox/Makefile +1 -0
@@ -7,3 +7,4 @@ obj-$(CONFIG_MLX4_CORE) += mlx4/ obj-$(CONFIG_MLX5_CORE) += mlx5/core/ obj-$(CONFIG_MLXSW_CORE) += mlxsw/ obj-$(CONFIG_MLXFW) += mlxfw/ +obj-$(CONFIG_MLXBF_GIGE) += mlxbf_gige/
drivers/net/ethernet/mellanox/mlxbf_gige/Kconfig new file +13 -0
@@ -0,0 +1,13 @@ +# SPDX-License-Identifier: GPL-2.0-only OR BSD-3-Clause +# +# Mellanox GigE driver configuration +# + +config MLXBF_GIGE + tristate "Mellanox Technologies BlueField Gigabit Ethernet support" + depends on (ARM64 && ACPI) || COMPILE_TEST + select PHYLIB + help + The second generation BlueField SoC from Mellanox Technologies + supports an out-of-band Gigabit Ethernet management port to the + Arm subsystem.
drivers/net/ethernet/mellanox/mlxbf_gige/Makefile new file +11 -0
@@ -0,0 +1,11 @@ +# SPDX-License-Identifier: GPL-2.0-only OR BSD-3-Clause + +obj-$(CONFIG_MLXBF_GIGE) += mlxbf_gige.o + +mlxbf_gige-y := mlxbf_gige_ethtool.o \ + mlxbf_gige_gpio.o \ + mlxbf_gige_intr.o \ + mlxbf_gige_main.o \ + mlxbf_gige_mdio.o \ + mlxbf_gige_rx.o \ + mlxbf_gige_tx.o
drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige.h new file +193 -0
193 lines not on this page · raw message
drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_ethtool.c new file +140 -0
140 lines not on this page · raw message
drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_gpio.c new file +212 -0
212 lines not on this page · raw message
drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_intr.c new file +143 -0
143 lines not on this page · raw message
drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_main.c new file +459 -0
459 lines not on this page · raw message
drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_mdio.c new file +187 -0
187 lines not on this page · raw message
drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_regs.h new file +78 -0
78 lines not on this page · raw message
drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_rx.c new file +318 -0
318 lines not on this page · raw message
drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_tx.c new file +284 -0
284 lines not on this page · raw message
Signature · 1 line
2.30.1
Quoted 54 lines · David Thompson
This patch adds build and driver logic for the "
mlxbf_gige" Ethernet driver from Mellanox Technologies. The second generation BlueField SoC from Mellanox supports an out-of-band GigaBit Ethernet management port to the Arm subsystem. This driver supports TCP/IP network connectivity for that port, and provides back-end routines to handle basic ethtool requests.The driver interfaces to the Gigabit Ethernet block of BlueField SoC via MMIO accesses to registers, which contain control information or pointers describing transmit and receive resources. There is a single transmit queue, and the port supports transmit ring sizes of 4 to 256 entries. There is a single receive queue, and the port supports receive ring sizes of 32 to 32K entries. The transmit and receive rings are allocated from DMA coherent memory. There is a 16-bit producer and consumer index per ring to denote software ownership and hardware ownership, respectively.
The main driver logic such as
probe(),remove(), and netdev ops are in "mlxbf_gige_main.c". Logic in "mlxbf_gige_rx.c" and "mlxbf_gige_tx.c" handles the packet processing for receive and transmit respectively.The logic in "
mlxbf_gige_ethtool.c" supports the handling of some basic ethtool requests: get driver info, get ring parameters, get registers, and get statistics.The logic in "
mlxbf_gige_mdio.c" is the driver controlling the Mellanox BlueField hardware that interacts with a PHY device via MDIO/MDC pins. This driver does the following:
- At driver probe time, it configures several BlueField MDIO parameters such as sample rate, full drive, voltage and MDC
- It defines functions to read and write MDIO registers and registers the MDIO bus.
- It defines the phy interrupt handler reporting a link up/down status change
- This driver's probe is invoked from the main driver logic while the phy interrupt handler is registered inndo_open.Driver limitations
- Only supports 1Gbps speed
- Only supports GMII protocol
- Supports maximum packet size of 2KB
- Does not support scatter-gather bufferingTesting
- Successful build of kernel for ARM64, ARM32,X86_64
- Tested ARM64 build on FastModels & Palladium
- Tested ARM64 build on several Mellanox boards that are built with the BlueField-2 SoC. The testing includes coverage in the areas of networking (e.g. ping, iperf, ifconfig, route), file transfers (e.g. SCP), and various ethtool options relevant to this driver.
Quoted 1 line
v5 -> v6
Please put changelog under "---" below your SOBs. We don't need to see this history in the git log.
Quoted 33 lines
Fixed use of
COMPILE_TESTfor ARM32 build; changed driver to not depend onCONFIG_ACPIfor ARM32 build
v4 -> v5
Created a separate interrupt controller for the GPIO PHY interrupt and as a result, the GIGE driver no longer depends on GPIO driver Updated the logic inmlxbf_gige_adjust_link()to store the negotiated pause settings into the driver's private settings.
Modified logic to only change enable bit inRX_DMAregister Changed logic to only map and unmap the actual length of the TX SKB, instead using the default size.
Added better error handling toopen()method
Modified receive packet logic to use polarity bit to signify ownership (software vs. hardware) of the RX CQE slot
v3 -> v4
Main driver module broken out into rx, tx, intr, and ethtool modules Removed some GPIO PHY interrupt logic; moved toGPIO_MLXBF2driver
v2 -> v3
Added logic to handle PHY link up/down interrupts
Use streaming DMA mapping for packet buffers
Changed logic to use standard iopoll methods
Changed PHY logic to not allow C45 transactions
Enhanced the error handling inopen()method
Enhancedstart_xmit()method to usexmit_moremechanism Added support forndo_get_stats64
Removed standard stats from "ethtool -S" output
v1 -> v2:
Fixed all warnings raised by "make C=1" and "make W=1"
a) Changed logic inmlxbf_gige_rx_deinit()andmlxbf_gige_tx_deinit()to initialize relevant pointers as NULL, not 0
b) Changemlxbf_gige_get_mac_rx_filter()to return void, as this function's return status is not used by caller
c) Fixed type definition of "buff" inmlxbf_gige_get_regs()
Quoted 4 lines
Signed-off-by David Thompson <[email protected]>Signed-off-by Asmaa Mnebhi <[email protected]>Reviewed-by Liming Sun <[email protected]>---
The patch generates checkpatch warnings.
CHECK: spinlock_t definition without comment
#272: FILE: drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige.h:87:
+ spinlock_t lock;
CHECK: spinlock_t definition without comment
#273: FILE: drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige.h:88:
+ spinlock_t gpio_lock;
CHECK: Macro argument 'tx_wqe_addr' may be better as '(tx_wqe_addr)' to avoid precedence issues #328: FILE: drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige.h:143:
+#define MLXBF_GIGE_TX_WQE_PKT_LEN(tx_wqe_addr) \
+ (*(tx_wqe_addr + 1) & MLXBF_GIGE_TX_WQE_PKT_LEN_MASK)
CHECK: multiple assignments should be avoided
#1271: FILE: drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_main.c:374:
+ phydev->irq = priv->mdiobus->irq[addr] = priv->phy_irq;
Thanks
Quoted 62 lines
-----Original Message-----
From: Leon Romanovsky <[email protected]>
Sent: Wednesday, June 2, 2021 12:39 AM
To: David Thompson <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; Liming Sun <[email protected]>; Asmaa Mnebhi <[email protected]>
Subject: Re: [PATCH net-next v6] Add Mellanox BlueField Gigabit Ethernet driverQuoted 53 lines · David Thompson
This patch adds build and driver logic for the "
mlxbf_gige" Ethernet driver from Mellanox Technologies. The second generation BlueField SoC from Mellanox supports an out-of-band GigaBit Ethernet management port to the Arm subsystem. This driver supports TCP/IP network connectivity for that port, and provides back-end routines to handle basic ethtool requests.The driver interfaces to the Gigabit Ethernet block of BlueField SoC via MMIO accesses to registers, which contain control information or pointers describing transmit and receive resources. There is a single transmit queue, and the port supports transmit ring sizes of 4 to 256 entries.
There is a single receive queue, and the port supports receive ring sizes of 32 to 32K entries. The transmit and receive rings are allocated from DMA coherent memory. There is a 16-bit producer and consumer index per ring to denote software ownership and hardware ownership, respectively.The main driver logic such as
probe(),remove(), and netdev ops are in "mlxbf_gige_main.c". Logic in "mlxbf_gige_rx.c"
and "mlxbf_gige_tx.c" handles the packet processing for receive and transmit respectively.The logic in "
mlxbf_gige_ethtool.c" supports the handling of some basic ethtool requests: get driver info, get ring parameters, get registers, and get statistics.The logic in "
mlxbf_gige_mdio.c" is the driver controlling the Mellanox BlueField hardware that interacts with a PHY device via MDIO/MDC pins. This driver does the following:
- At driver probe time, it configures several BlueField MDIO parameters such as sample rate, full drive, voltage and MDC
- It defines functions to read and write MDIO registers and registers the MDIO bus.
- It defines the phy interrupt handler reporting a link up/down status change
- This driver's probe is invoked from the main driver logic while the phy interrupt handler is registered inndo_open.Driver limitations
- Only supports 1Gbps speed
- Only supports GMII protocol
- Supports maximum packet size of 2KB
- Does not support scatter-gather bufferingTesting
- Successful build of kernel for ARM64, ARM32,X86_64
- Tested ARM64 build on FastModels & Palladium
- Tested ARM64 build on several Mellanox boards that are built with the BlueField-2 SoC. The testing includes coverage in the areas of networking (e.g. ping, iperf, ifconfig, route), file transfers (e.g. SCP), and various ethtool options relevant to this driver.
Quoted 5 lines
Quoted 1 line
v5 -> v6
Please put changelog under "---" below your SOBs. We don't need to see this history in the git log.
Sure, will adjust this in the next patch version.
Quoted 54 lines
Quoted 33 lines
Fixed use of
COMPILE_TESTfor ARM32 build; changed driver to not depend onCONFIG_ACPIfor ARM32 build
v4 -> v5
Created a separate interrupt controller for the GPIO PHY interrupt and as a result, the GIGE driver no longer depends on GPIO driver Updated the logic inmlxbf_gige_adjust_link()to store the negotiated pause settings into the driver's private settings.
Modified logic to only change enable bit inRX_DMAregister Changed logic to only map and unmap the actual length of the TX SKB, instead using the default size.
Added better error handling toopen()method
Modified receive packet logic to use polarity bit to signify ownership (software vs. hardware) of the RX CQE slot
v3 -> v4
Main driver module broken out into rx, tx, intr, and ethtool modules Removed some GPIO PHY interrupt logic; moved toGPIO_MLXBF2driver
v2 -> v3
Added logic to handle PHY link up/down interrupts
Use streaming DMA mapping for packet buffers
Changed logic to use standard iopoll methods
Changed PHY logic to not allow C45 transactions
Enhanced the error handling inopen()method
Enhancedstart_xmit()method to usexmit_moremechanism Added support forndo_get_stats64
Removed standard stats from "ethtool -S" output
v1 -> v2:
Fixed all warnings raised by "make C=1" and "make W=1"
a) Changed logic inmlxbf_gige_rx_deinit()andmlxbf_gige_tx_deinit()to initialize relevant pointers as NULL, not 0
b) Changemlxbf_gige_get_mac_rx_filter()to return void, as this function's return status is not used by caller
c) Fixed type definition of "buff" inmlxbf_gige_get_regs()Quoted 4 lines
Signed-off-by David Thompson <[email protected]>Signed-off-by Asmaa Mnebhi <[email protected]>Reviewed-by Liming Sun <[email protected]>---
The patch generates checkpatch warnings.
CHECK:
spinlock_tdefinition without comment
#272: FILE:drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige.h:87:
+spinlock_tlock;CHECK:
spinlock_tdefinition without comment
#273: FILE:drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige.h:88:
+spinlock_tgpio_lock;CHECK: Macro argument '
tx_wqe_addr' may be better as '(tx_wqe_addr)' to
avoid precedence issues
#328: FILE:drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige.h:143:
+#defineMLXBF_GIGE_TX_WQE_PKT_LEN(tx_wqe_addr)\
+ (*(tx_wqe_addr + 1) &MLXBF_GIGE_TX_WQE_PKT_LEN_MASK)
Quoted 6 lines
CHECK: multiple assignments should be avoided
#1271: FILE:drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_main.c:374:
+phydev->irq= priv->mdiobus->irq[addr] =priv->phy_irq;Thanks
Leon,
Thank you for pointing out these checkpatch warnings. We'll address these in the next patch version.
- Dave