Skip to contentSkip to conversation

mlxbf_gige: add BlueField-3 support

PATCH Applied David Thompson 3 participants 8 messages Jan 12, 2023 – Jan 14, 2023

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

v3 cover mlxbf_gige: add BlueField-3 support

This patch series adds driver logic to the "mlxbf_gige" Ethernet driver in order to support the third generation BlueField SoC (BF3). The existing "mlxbf_gige" driver is extended with BF3-specific logic and run-time decisions are made by the driver depending on the SoC generation (BF2 vs. BF3).

The BF3 SoC is similar to BF2 SoC with regards to transmit and receive packet processing:
* Driver rings usage; consumer & producer indices
* Single queue for receive and transmit
* DMA operation

The differences between BF3 and BF2 SoC are:
* In addition to supporting 1Gbps interface speed, the BF3 SoC adds support for 10Mbps and 100Mbps interface speeds
* BF3 requires SerDes config logic to support its SGMII interface
* BF3 adds support for "ethtool -s" for interface speed config
* BF3 utilizes different MDIO logic for accessing the board-level PHY device

Testing
- Successful build of kernel for ARM64, ARM32, X86_64
- Tested ARM64 build on FastModels, Palladium, SoC

v2 -> v3:
- "mlxbf_gige: add BlueField-3 Serdes configuration" patch has been removed from the patch series, as this configuration is no longer needed within the Linux driver.
- "mlxbf_gige: add set_link_ksettings ethtool callback" patch has been split into two patches, moving the white space change to its own patch

v1 -> v2:
- Fixed build failures in "build_32bit" and "build_allmodconfig_warn"
- Removed use of spinlock in BF3 "adjust_link" callback
- Added use of ARRAY_SIZE() where appropriate
- Added "set_link_ksettings" ethtool callback for BF2 and BF3

David Thompson (4):
mlxbf_gige: add MDIO support for BlueField-3
mlxbf_gige: support 10M/100M/1G speeds on BlueField-3
mlxbf_gige: add "set_link_ksettings" ethtool callback
mlxbf_gige: fix white space in mlxbf_gige_eth_ioctl

 .../ethernet/mellanox/mlxbf_gige/mlxbf_gige.h |  27 +++
 .../mellanox/mlxbf_gige/mlxbf_gige_ethtool.c  |   1 +
 .../mellanox/mlxbf_gige/mlxbf_gige_main.c     | 109 +++++++++--
 .../mellanox/mlxbf_gige/mlxbf_gige_mdio.c     | 172 +++++++++++++-----
 .../mellanox/mlxbf_gige/mlxbf_gige_mdio_bf2.h |  53 ++++++
 .../mellanox/mlxbf_gige/mlxbf_gige_mdio_bf3.h |  54 ++++++
 .../mellanox/mlxbf_gige/mlxbf_gige_regs.h     |  22 +++
 7 files changed, 372 insertions(+), 66 deletions(-)
 create mode 100644 drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_mdio_bf2.h
 create mode 100644 drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_mdio_bf3.h
Signature · 1 line
2.30.1
#1 lore raw

PATCH v3 1/4 mlxbf_gige: add MDIO support for BlueField-3

This patch adds initial MDIO support for the BlueField-3 SoC. Separate header files for the BlueField-2 and the BlueField-3 SoCs have been created. These header files hold the SoC-specific MDIO macros since the register offsets and bit fields have changed. Also, in BlueField-3 there is a separate register for writing and reading the MDIO data. Finally, instead of having "if" statements everywhere to differentiate between SoC-specific logic, a mlxbf_gige_mdio_gw_t struct was created for this purpose.

---
 .../ethernet/mellanox/mlxbf_gige/mlxbf_gige.h |  19 ++
 .../mellanox/mlxbf_gige/mlxbf_gige_main.c     |   2 +
 .../mellanox/mlxbf_gige/mlxbf_gige_mdio.c     | 172 +++++++++++++-----
 .../mellanox/mlxbf_gige/mlxbf_gige_mdio_bf2.h |  53 ++++++
 .../mellanox/mlxbf_gige/mlxbf_gige_mdio_bf3.h |  54 ++++++
 .../mellanox/mlxbf_gige/mlxbf_gige_regs.h     |   1 +
 6 files changed, 251 insertions(+), 50 deletions(-)
 create mode 100644 drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_mdio_bf2.h
 create mode 100644 drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_mdio_bf3.h
drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige.h +19 -0
index 5a1027b07215..421a0b1b766c 100644
@@ -67,6 +67,23 @@ struct mlxbf_gige_stats {
 	u64 rx_filter_discard_pkts;
 };
 
+struct mlxbf_gige_reg_param {
+	u32 mask;
+	u32 shift;
+};
+
+struct mlxbf_gige_mdio_gw {
+	u32 gw_address;
+	u32 read_data_address;
+	struct mlxbf_gige_reg_param busy;
+	struct mlxbf_gige_reg_param write_data;
+	struct mlxbf_gige_reg_param read_data;
+	struct mlxbf_gige_reg_param devad;
+	struct mlxbf_gige_reg_param partad;
+	struct mlxbf_gige_reg_param opcode;
+	struct mlxbf_gige_reg_param st1;
+};
+
 struct mlxbf_gige {
 	void __iomem *base;
 	void __iomem *llu_base;
@@ -102,6 +119,8 @@ struct mlxbf_gige {
 	u8 valid_polarity;
 	struct napi_struct napi;
 	struct mlxbf_gige_stats stats;
+	u8 hw_version;
+	struct mlxbf_gige_mdio_gw *mdio_gw;
 };
 
 /* Rx Work Queue Element definitions */
drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_main.c +2 -0
index 2292d63a279c..e08c07e914c1 100644
@@ -315,6 +315,8 @@ static int mlxbf_gige_probe(struct platform_device *pdev)
 
 	spin_lock_init(&priv->lock);
 
+	priv->hw_version = readq(base + MLXBF_GIGE_VERSION);
+
 	/* Attach MDIO device */
 	err = mlxbf_gige_mdio_probe(pdev, priv);
 	if (err)
drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_mdio.c +122 -50

256 lines not on this page · raw message

drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_mdio_bf2.h new file +53 -0

53 lines not on this page · raw message

drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_mdio_bf3.h new file +54 -0

54 lines not on this page · raw message

drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_regs.h +1 -0
index 7be3a793984d..8d52dbef4adf 100644
@@ -10,6 +10,7 @@
 
 #define MLXBF_GIGE_VERSION                            0x0000
 #define MLXBF_GIGE_VERSION_BF2                        0x0
+#define MLXBF_GIGE_VERSION_BF3                        0x1
 #define MLXBF_GIGE_STATUS                             0x0010
 #define MLXBF_GIGE_STATUS_READY                       BIT(0)
 #define MLXBF_GIGE_INT_STATUS                         0x0028
Signature · 1 line
2.30.1
#2 lore raw

PATCH v3 2/4 mlxbf_gige: support 10M/100M/1G speeds on BlueField-3

In reply to David Thompson

The BlueField-3 OOB interface supports 10Mbps, 100Mbps, and 1Gbps speeds. The external PHY is responsible for autonegotiating the speed with the link partner. Once the autonegotiation is done, the BlueField PLU needs to be configured accordingly.

This patch does two things:
1) Initialize the advertised control flow/duplex/speed in the probe based on the BlueField SoC generation (2 or 3)
2) Adjust the PLU speed config in the PHY interrupt handler

---
 .../ethernet/mellanox/mlxbf_gige/mlxbf_gige.h |   8 ++
 .../mellanox/mlxbf_gige/mlxbf_gige_main.c     | 105 +++++++++++++++---
 .../mellanox/mlxbf_gige/mlxbf_gige_regs.h     |  21 ++++
 3 files changed, 119 insertions(+), 15 deletions(-)
drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige.h +8 -0
index 421a0b1b766c..a453b9cd9033 100644
@@ -14,6 +14,7 @@
 #include <linux/irqreturn.h>
 #include <linux/netdevice.h>
 #include <linux/irq.h>
+#include <linux/phy.h>
 
 /* The silicon design supports a maximum RX ring size of
  * 32K entries. Based on current testing this maximum size
@@ -84,6 +85,12 @@ struct mlxbf_gige_mdio_gw {
 	struct mlxbf_gige_reg_param st1;
 };
 
+struct mlxbf_gige_link_cfg {
+	void (*set_phy_link_mode)(struct phy_device *phydev);
+	void (*adjust_link)(struct net_device *netdev);
+	phy_interface_t phy_mode;
+};
+
 struct mlxbf_gige {
 	void __iomem *base;
 	void __iomem *llu_base;
@@ -121,6 +128,7 @@ struct mlxbf_gige {
 	struct mlxbf_gige_stats stats;
 	u8 hw_version;
 	struct mlxbf_gige_mdio_gw *mdio_gw;
+	int prev_speed;
 };
 
 /* Rx Work Queue Element definitions */
drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_main.c +90 -15

128 lines not on this page · raw message

drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_regs.h +21 -0
index 8d52dbef4adf..cd0973229c9b 100644
@@ -8,6 +8,8 @@
 #ifndef __MLXBF_GIGE_REGS_H__
 #define __MLXBF_GIGE_REGS_H__
 
+#include <linux/bitfield.h>
+
 #define MLXBF_GIGE_VERSION                            0x0000
 #define MLXBF_GIGE_VERSION_BF2                        0x0
 #define MLXBF_GIGE_VERSION_BF3                        0x1
@@ -78,4 +80,23 @@
  */
 #define MLXBF_GIGE_MMIO_REG_SZ                        (MLXBF_GIGE_MAC_CFG + 8)
 
+#define MLXBF_GIGE_PLU_TX_REG0                        0x80
+#define MLXBF_GIGE_PLU_TX_IPG_SIZE_MASK               GENMASK(11, 0)
+#define MLXBF_GIGE_PLU_TX_SGMII_MODE_MASK             GENMASK(15, 14)
+
+#define MLXBF_GIGE_PLU_RX_REG0                        0x10
+#define MLXBF_GIGE_PLU_RX_SGMII_MODE_MASK             GENMASK(25, 24)
+
+#define MLXBF_GIGE_1G_SGMII_MODE                      0x0
+#define MLXBF_GIGE_10M_SGMII_MODE                     0x1
+#define MLXBF_GIGE_100M_SGMII_MODE                    0x2
+
+/* ipg_size default value for 1G is fixed by HW to 11 + End = 12.
+ * So for 100M it is 12 * 10 - 1 = 119
+ * For 10M, it is 12 * 100 - 1 = 1199
+ */
+#define MLXBF_GIGE_1G_IPG_SIZE                        11
+#define MLXBF_GIGE_100M_IPG_SIZE                      119
+#define MLXBF_GIGE_10M_IPG_SIZE                       1199
+
 #endif /* !defined(__MLXBF_GIGE_REGS_H__) */
Signature · 1 line
2.30.1
Reviewed-by on v3 2/4
#3 lore raw

Re: [PATCH net-next v3 2/4] mlxbf_gige: support 10M/100M/1G speeds on BlueField-3

Quoted 5 lines · David Thompson

The BlueField-3 OOB interface supports 10Mbps, 100Mbps, and 1Gbps speeds. The external PHY is responsible for autonegotiating the speed with the link partner. Once the autonegotiation is done, the BlueField PLU needs to be configured accordingly.

Quoted 7 lines

This patch does two things:
1) Initialize the advertised control flow/duplex/speed in the probe based on the BlueField SoC generation (2 or 3)
2) Adjust the PLU speed config in the PHY interrupt handler

Andrew

#4 lore raw

PATCH v3 3/4 mlxbf_gige: add "set_link_ksettings" ethtool callback

In reply to David Thompson

This patch extends the "ethtool_ops" data structure to include the "set_link_ksettings" callback. This change enables configuration of the various interface speeds that the BlueField-3 supports (10Mbps, 100Mbps, and 1Gbps).

---
 drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_ethtool.c | 1 +
 1 file changed, 1 insertion(+)
drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_ethtool.c +1 -0
index 41ebef25a930..253d7ad9b809 100644
@@ -135,4 +135,5 @@ const struct ethtool_ops mlxbf_gige_ethtool_ops = {
 	.nway_reset		= phy_ethtool_nway_reset,
 	.get_pauseparam		= mlxbf_gige_get_pauseparam,
 	.get_link_ksettings	= phy_ethtool_get_link_ksettings,
+	.set_link_ksettings	= phy_ethtool_set_link_ksettings,
 };
Signature · 1 line
2.30.1
Reviewed-by on v3 3/4
#5 lore raw

Re: [PATCH net-next v3 3/4] mlxbf_gige: add "set_link_ksettings" ethtool callback

Quoted 7 lines · David Thompson

This patch extends the "ethtool_ops" data structure to include the "set_link_ksettings" callback. This change enables configuration of the various interface speeds that the BlueField-3 supports (10Mbps, 100Mbps, and 1Gbps).

Andrew

#6 lore raw

PATCH v3 4/4 mlxbf_gige: fix white space in mlxbf_gige_eth_ioctl

In reply to David Thompson

This patch fixes the white space issue raised by checkpatch:
CHECK: Alignment should match open parenthesis +static int mlxbf_gige_eth_ioctl(struct net_device *netdev,
+ struct ifreq *ifr, int cmd)

---
 drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_main.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_main.c +1 -1
index 32d7030eb2cf..694de9513b9f 100644
@@ -205,7 +205,7 @@ static int mlxbf_gige_stop(struct net_device *netdev)
 }
 
 static int mlxbf_gige_eth_ioctl(struct net_device *netdev,
-			       struct ifreq *ifr, int cmd)
+				struct ifreq *ifr, int cmd)
 {
 	if (!(netif_running(netdev)))
 		return -EINVAL;
Signature · 1 line
2.30.1
#7 lore raw
In reply to David Thompson
This series was applied to netdev/net-next.git (master) by Jakub Kicinski <[email protected]>:

Hello:

This series was applied to netdev/net-next.git (master) by Jakub Kicinski <[email protected]>:

Quoted 8 lines · On Thu, 12 Jan 2023 15:26:05 -0500 you wrote:

This patch series adds driver logic to the "mlxbf_gige" Ethernet driver in order to support the third generation BlueField SoC (BF3). The existing "mlxbf_gige" driver is extended with BF3-specific logic and run-time decisions are made by the driver depending on the SoC generation (BF2 vs. BF3).

[...]

Here is the summary with links:
- [net-next,v3,1/4] mlxbf_gige: add MDIO support for BlueField-3

    https://git.kernel.org/netdev/net-next/c/2321d69f92aa
  - [net-next,v3,2/4] mlxbf_gige: support 10M/100M/1G speeds on BlueField-3
    https://git.kernel.org/netdev/net-next/c/20d03d4d9437
  - [net-next,v3,3/4] mlxbf_gige: add "set_link_ksettings" ethtool callback
    https://git.kernel.org/netdev/net-next/c/cedd97737a1f
  - [net-next,v3,4/4] mlxbf_gige: fix white space in mlxbf_gige_eth_ioctl
    https://git.kernel.org/netdev/net-next/c/e1cc8ce46200

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html