Skip to contentSkip to conversation

bnxt_en: Updates

PATCH Applied Michael Chan 3 participants 8 messages Nov 7, 2022 – Nov 8, 2022

Subsystems (3)
lore
Share & tools
lore

Michael Chan

author maintainer
#0 lore raw

cover bnxt_en: Updates

This small patchset adds an improvement to the configuration of ethtool RSS tuple hash and a PTP improvement when running in a multi-host environment.

Edwin Peer (2):
bnxt_en: refactor VNIC RSS update functions
bnxt_en: update RSS config using difference algorithm

Pavan Chebbi (1):
bnxt_en: Add a non-real time mode to access NIC clock

 drivers/net/ethernet/broadcom/bnxt/bnxt.c     | 72 ++++++++++++++-----
 drivers/net/ethernet/broadcom/bnxt/bnxt.h     |  2 +
 .../net/ethernet/broadcom/bnxt/bnxt_ethtool.c |  2 +
 drivers/net/ethernet/broadcom/bnxt/bnxt_hsi.h | 47 ++++++++++++
 drivers/net/ethernet/broadcom/bnxt/bnxt_ptp.c | 45 ++++++++----
 drivers/net/ethernet/broadcom/bnxt/bnxt_ptp.h |  7 +-
 6 files changed, 142 insertions(+), 33 deletions(-)
Signature · 1 line
2.18.1
author maintainer
#1 lore raw

PATCH 1/3 bnxt_en: refactor VNIC RSS update functions

From: Edwin Peer <[email protected]>

Extract common code into a new function. This will avoid duplication in the next patch, which changes the update algorithm for both the P5 and legacy code paths.

No functional changes.

---
 drivers/net/ethernet/broadcom/bnxt/bnxt.c | 34 +++++++++++------------
 1 file changed, 16 insertions(+), 18 deletions(-)
drivers/net/ethernet/broadcom/bnxt/bnxt.c +16 -18
index 03272aa48511..15edb6cb3656 100644
@@ -5250,7 +5250,7 @@ int bnxt_get_nr_rss_ctxs(struct bnxt *bp, int rx_rings)
 	return 1;
 }
 
-static void __bnxt_fill_hw_rss_tbl(struct bnxt *bp, struct bnxt_vnic_info *vnic)
+static void bnxt_fill_hw_rss_tbl(struct bnxt *bp, struct bnxt_vnic_info *vnic)
 {
 	bool no_rss = !(vnic->flags & BNXT_VNIC_RSS_FLAG);
 	u16 i, j;
@@ -5263,8 +5263,8 @@ static void __bnxt_fill_hw_rss_tbl(struct bnxt *bp, struct bnxt_vnic_info *vnic)
 	}
 }
 
-static void __bnxt_fill_hw_rss_tbl_p5(struct bnxt *bp,
-				      struct bnxt_vnic_info *vnic)
+static void bnxt_fill_hw_rss_tbl_p5(struct bnxt *bp,
+				    struct bnxt_vnic_info *vnic)
 {
 	__le16 *ring_tbl = vnic->rss_table;
 	struct bnxt_rx_ring_info *rxr;
@@ -5285,12 +5285,19 @@ static void __bnxt_fill_hw_rss_tbl_p5(struct bnxt *bp,
 	}
 }
 
-static void bnxt_fill_hw_rss_tbl(struct bnxt *bp, struct bnxt_vnic_info *vnic)
+static void
+__bnxt_hwrm_vnic_set_rss(struct bnxt *bp, struct hwrm_vnic_rss_cfg_input *req,
+			 struct bnxt_vnic_info *vnic)
 {
 	if (bp->flags & BNXT_FLAG_CHIP_P5)
-		__bnxt_fill_hw_rss_tbl_p5(bp, vnic);
+		bnxt_fill_hw_rss_tbl_p5(bp, vnic);
 	else
-		__bnxt_fill_hw_rss_tbl(bp, vnic);
+		bnxt_fill_hw_rss_tbl(bp, vnic);
+
+	req->hash_type = cpu_to_le32(bp->rss_hash_cfg);
+	req->hash_mode_flags = VNIC_RSS_CFG_REQ_HASH_MODE_FLAGS_DEFAULT;
+	req->ring_grp_tbl_addr = cpu_to_le64(vnic->rss_table_dma_addr);
+	req->hash_key_tbl_addr = cpu_to_le64(vnic->rss_hash_key_dma_addr);
 }
 
 static int bnxt_hwrm_vnic_set_rss(struct bnxt *bp, u16 vnic_id, bool set_rss)
@@ -5307,14 +5314,8 @@ static int bnxt_hwrm_vnic_set_rss(struct bnxt *bp, u16 vnic_id, bool set_rss)
 	if (rc)
 		return rc;
 
-	if (set_rss) {
-		bnxt_fill_hw_rss_tbl(bp, vnic);
-		req->hash_type = cpu_to_le32(bp->rss_hash_cfg);
-		req->hash_mode_flags = VNIC_RSS_CFG_REQ_HASH_MODE_FLAGS_DEFAULT;
-		req->ring_grp_tbl_addr = cpu_to_le64(vnic->rss_table_dma_addr);
-		req->hash_key_tbl_addr =
-			cpu_to_le64(vnic->rss_hash_key_dma_addr);
-	}
+	if (set_rss)
+		__bnxt_hwrm_vnic_set_rss(bp, req, vnic);
 	req->rss_ctx_idx = cpu_to_le16(vnic->fw_rss_cos_lb_ctx[0]);
 	return hwrm_req_send(bp, req);
 }
@@ -5335,10 +5336,7 @@ static int bnxt_hwrm_vnic_set_rss_p5(struct bnxt *bp, u16 vnic_id, bool set_rss)
 	if (!set_rss)
 		return hwrm_req_send(bp, req);
 
-	bnxt_fill_hw_rss_tbl(bp, vnic);
-	req->hash_type = cpu_to_le32(bp->rss_hash_cfg);
-	req->hash_mode_flags = VNIC_RSS_CFG_REQ_HASH_MODE_FLAGS_DEFAULT;
-	req->hash_key_tbl_addr = cpu_to_le64(vnic->rss_hash_key_dma_addr);
+	__bnxt_hwrm_vnic_set_rss(bp, req, vnic);
 	ring_tbl_map = vnic->rss_table_dma_addr;
 	nr_ctxs = bnxt_get_nr_rss_ctxs(bp, bp->rx_nr_rings);
 
Signature · 1 line
2.18.1
Reviewed-by on v1 1/3
#2 lore raw

Re: [PATCH net-next 1/3] bnxt_en: refactor VNIC RSS update functions

Quoted 13 lines · Michael Chan

From: Edwin Peer <[email protected]>

Extract common code into a new function. This will avoid duplication in the next patch, which changes the update algorithm for both the P5 and legacy code paths.

No functional changes.

---
 drivers/net/ethernet/broadcom/bnxt/bnxt.c | 34 +++++++++++------------
 1 file changed, 16 insertions(+), 18 deletions(-)

<...>

drivers/net/ethernet/broadcom/bnxt/bnxt.c static void __bnxt_fill_hw_rss_tbl_p5(struct bnxt *bp, · line 5290

Quoted 3 lines
drivers/net/ethernet/broadcom/bnxt/bnxt.c +3 -0
+static void
+__bnxt_hwrm_vnic_set_rss(struct bnxt *bp, struct hwrm_vnic_rss_cfg_input *req,
+			 struct bnxt_vnic_info *vnic)

The "__" prefix in __func_name() usually means in kernel that this function is locked externally.

Thanks,

author maintainer
#3 lore raw

PATCH 2/3 bnxt_en: update RSS config using difference algorithm

In reply to Michael Chan

From: Edwin Peer <[email protected]>

Hardware is unable to realize all legal firmware interface state values for hash_type. For example, if 4-tuple TCP_IPV4 hash is enabled, 4-tuple UDP_IPV4 hash must also be enabled. By providing the bits the user intended to change instead of the possible illegal intermediate states, the firmware is able to make better compromises when deciding which bits to ignore.

With this new mechansim, we can now report the actual configured hash back to the user. Add bnxt_hwrm_update_rss_hash_cfg() to report the actual hash after user configuration.

---
 drivers/net/ethernet/broadcom/bnxt/bnxt.c     | 35 +++++++++++++-
 drivers/net/ethernet/broadcom/bnxt/bnxt.h     |  2 +
 .../net/ethernet/broadcom/bnxt/bnxt_ethtool.c |  2 +
 drivers/net/ethernet/broadcom/bnxt/bnxt_hsi.h | 47 +++++++++++++++++++
 4 files changed, 85 insertions(+), 1 deletion(-)
drivers/net/ethernet/broadcom/bnxt/bnxt.c +34 -1
index 15edb6cb3656..d18e55b16375 100644
@@ -5294,7 +5294,15 @@ __bnxt_hwrm_vnic_set_rss(struct bnxt *bp, struct hwrm_vnic_rss_cfg_input *req,
 	else
 		bnxt_fill_hw_rss_tbl(bp, vnic);
 
-	req->hash_type = cpu_to_le32(bp->rss_hash_cfg);
+	if (bp->rss_hash_delta) {
+		req->hash_type = cpu_to_le32(bp->rss_hash_delta);
+		if (bp->rss_hash_cfg & bp->rss_hash_delta)
+			req->flags |= VNIC_RSS_CFG_REQ_FLAGS_HASH_TYPE_INCLUDE;
+		else
+			req->flags |= VNIC_RSS_CFG_REQ_FLAGS_HASH_TYPE_EXCLUDE;
+	} else {
+		req->hash_type = cpu_to_le32(bp->rss_hash_cfg);
+	}
 	req->hash_mode_flags = VNIC_RSS_CFG_REQ_HASH_MODE_FLAGS_DEFAULT;
 	req->ring_grp_tbl_addr = cpu_to_le64(vnic->rss_table_dma_addr);
 	req->hash_key_tbl_addr = cpu_to_le64(vnic->rss_hash_key_dma_addr);
@@ -5355,6 +5363,25 @@ static int bnxt_hwrm_vnic_set_rss_p5(struct bnxt *bp, u16 vnic_id, bool set_rss)
 	return rc;
 }
 
+static void bnxt_hwrm_update_rss_hash_cfg(struct bnxt *bp)
+{
+	struct bnxt_vnic_info *vnic = &bp->vnic_info[0];
+	struct hwrm_vnic_rss_qcfg_output *resp;
+	struct hwrm_vnic_rss_qcfg_input *req;
+
+	if (hwrm_req_init(bp, req, HWRM_VNIC_RSS_QCFG))
+		return;
+
+	/* all contexts configured to same hash_type, zero always exists */
+	req->rss_ctx_idx = cpu_to_le16(vnic->fw_rss_cos_lb_ctx[0]);
+	resp = hwrm_req_hold(bp, req);
+	if (!hwrm_req_send(bp, req)) {
+		bp->rss_hash_cfg = le32_to_cpu(resp->hash_type) ?: bp->rss_hash_cfg;
+		bp->rss_hash_delta = 0;
+	}
+	hwrm_req_drop(bp, req);
+}
+
 static int bnxt_hwrm_vnic_set_hds(struct bnxt *bp, u16 vnic_id)
 {
 	struct bnxt_vnic_info *vnic = &bp->vnic_info[vnic_id];
@@ -5612,6 +5639,8 @@ static int bnxt_hwrm_vnic_qcaps(struct bnxt *bp)
 		    (BNXT_CHIP_P5_THOR(bp) &&
 		     !(bp->fw_cap & BNXT_FW_CAP_EXT_HW_STATS_SUPPORTED)))
 			bp->fw_cap |= BNXT_FW_CAP_VLAN_RX_STRIP;
+		if (flags & VNIC_QCAPS_RESP_FLAGS_RSS_HASH_TYPE_DELTA_CAP)
+			bp->fw_cap |= BNXT_FW_CAP_RSS_HASH_TYPE_DELTA;
 		bp->max_tpa_v2 = le16_to_cpu(resp->max_aggs_supported);
 		if (bp->max_tpa_v2) {
 			if (BNXT_CHIP_P5_THOR(bp))
@@ -8806,6 +8835,8 @@ static int bnxt_init_chip(struct bnxt *bp, bool irq_re_init)
 	rc = bnxt_setup_vnic(bp, 0);
 	if (rc)
 		goto err_out;
+	if (bp->fw_cap & BNXT_FW_CAP_RSS_HASH_TYPE_DELTA)
+		bnxt_hwrm_update_rss_hash_cfg(bp);
 
 	if (bp->flags & BNXT_FLAG_RFS) {
 		rc = bnxt_alloc_rfs_vnics(bp);
@@ -12241,6 +12272,8 @@ static void bnxt_set_dflt_rss_hash_type(struct bnxt *bp)
 			   VNIC_RSS_CFG_REQ_HASH_TYPE_TCP_IPV4 |
 			   VNIC_RSS_CFG_REQ_HASH_TYPE_IPV6 |
 			   VNIC_RSS_CFG_REQ_HASH_TYPE_TCP_IPV6;
+	if (bp->fw_cap & BNXT_FW_CAP_RSS_HASH_TYPE_DELTA)
+		bp->rss_hash_delta = bp->rss_hash_cfg;
 	if (BNXT_CHIP_P4_PLUS(bp) && bp->hwrm_spec_code >= 0x10501) {
 		bp->flags |= BNXT_FLAG_UDP_RSS_CAP;
 		bp->rss_hash_cfg |= VNIC_RSS_CFG_REQ_HASH_TYPE_UDP_IPV4 |
drivers/net/ethernet/broadcom/bnxt/bnxt.h +2 -0
index 91a1ba0a914d..7611bed4a196 100644
@@ -1900,6 +1900,7 @@ struct bnxt {
 	u16			*rss_indir_tbl;
 	u16			rss_indir_tbl_entries;
 	u32			rss_hash_cfg;
+	u32			rss_hash_delta;
 
 	u16			max_mtu;
 	u8			max_tc;
@@ -1965,6 +1966,7 @@ struct bnxt {
 	#define BNXT_FW_CAP_CFA_RFS_RING_TBL_IDX_V2	0x00010000
 	#define BNXT_FW_CAP_PCIE_STATS_SUPPORTED	0x00020000
 	#define BNXT_FW_CAP_EXT_STATS_SUPPORTED		0x00040000
+	#define BNXT_FW_CAP_RSS_HASH_TYPE_DELTA		0x00080000
 	#define BNXT_FW_CAP_ERR_RECOVER_RELOAD		0x00100000
 	#define BNXT_FW_CAP_HOT_RESET			0x00200000
 	#define BNXT_FW_CAP_PTP_RTC			0x00400000
drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c +2 -0
index cc89e5eabcb9..a841b0a47cfd 100644
@@ -1234,6 +1234,8 @@ static int bnxt_srxfh(struct bnxt *bp, struct ethtool_rxnfc *cmd)
 	if (bp->rss_hash_cfg == rss_hash_cfg)
 		return 0;
 
+	if (bp->fw_cap & BNXT_FW_CAP_RSS_HASH_TYPE_DELTA)
+		bp->rss_hash_delta = bp->rss_hash_cfg ^ rss_hash_cfg;
 	bp->rss_hash_cfg = rss_hash_cfg;
 	if (netif_running(bp->dev)) {
 		bnxt_close_nic(bp, false, false);
drivers/net/ethernet/broadcom/bnxt/bnxt_hsi.h +47 -0

53 lines not on this page · raw message

Signature · 1 line
2.18.1
Reviewed-by on v1 2/3
#4 lore raw

Re: [PATCH net-next 2/3] bnxt_en: update RSS config using difference algorithm

Quoted 13 lines · Michael Chan

From: Edwin Peer <[email protected]>

Hardware is unable to realize all legal firmware interface state values for hash_type. For example, if 4-tuple TCP_IPV4 hash is enabled, 4-tuple UDP_IPV4 hash must also be enabled. By providing the bits the user intended to change instead of the possible illegal intermediate states, the firmware is able to make better compromises when deciding which bits to ignore.

With this new mechansim, we can now report the actual configured hash back to the user. Add bnxt_hwrm_update_rss_hash_cfg() to report the actual hash after user configuration.

Quoted 9 lines
---
 drivers/net/ethernet/broadcom/bnxt/bnxt.c     | 35 +++++++++++++-
 drivers/net/ethernet/broadcom/bnxt/bnxt.h     |  2 +
 .../net/ethernet/broadcom/bnxt/bnxt_ethtool.c |  2 +
 drivers/net/ethernet/broadcom/bnxt/bnxt_hsi.h | 47 +++++++++++++++++++
 4 files changed, 85 insertions(+), 1 deletion(-)

Thanks,

author maintainer
#5 lore raw

PATCH 3/3 bnxt_en: Add a non-real time mode to access NIC clock

In reply to Michael Chan

From: Pavan Chebbi <[email protected]>

When using a PHC that is shared between multiple hosts, in order to achieve consistent timestamps across all hosts, we need to isolate the PHC from any host making frequency adjustments.

This patch adds a non-real time mode for this purpose. The implementation is based on a free running NIC hardware timer which is used as the timestamper time-base. Each host implements individual adjustments to a local timecounter based on the NIC free running timer.

---
 drivers/net/ethernet/broadcom/bnxt/bnxt.c     |  5 ++-
 drivers/net/ethernet/broadcom/bnxt/bnxt_ptp.c | 45 ++++++++++++++-----
 drivers/net/ethernet/broadcom/bnxt/bnxt_ptp.h |  7 ++-
 3 files changed, 42 insertions(+), 15 deletions(-)
drivers/net/ethernet/broadcom/bnxt/bnxt.c +4 -1
index d18e55b16375..d9ad325f7840 100644
@@ -6985,8 +6985,11 @@ static int bnxt_hwrm_func_qcfg(struct bnxt *bp)
 		if (flags & FUNC_QCFG_RESP_FLAGS_FW_DCBX_AGENT_ENABLED)
 			bp->fw_cap |= BNXT_FW_CAP_DCBX_AGENT;
 	}
-	if (BNXT_PF(bp) && (flags & FUNC_QCFG_RESP_FLAGS_MULTI_HOST))
+	if (BNXT_PF(bp) && (flags & FUNC_QCFG_RESP_FLAGS_MULTI_HOST)) {
 		bp->flags |= BNXT_FLAG_MULTI_HOST;
+		if (bp->fw_cap & BNXT_FW_CAP_PTP_RTC)
+			bp->fw_cap &= ~BNXT_FW_CAP_PTP_RTC;
+	}
 	if (flags & FUNC_QCFG_RESP_FLAGS_RING_MONITOR_ENABLED)
 		bp->fw_cap |= BNXT_FW_CAP_RING_MONITOR;
 
drivers/net/ethernet/broadcom/bnxt/bnxt_ptp.c +33 -12

65 lines not on this page · raw message

drivers/net/ethernet/broadcom/bnxt/bnxt_ptp.h +5 -2
index 4ce0a14c1e23..34162e07a119 100644
@@ -17,6 +17,8 @@
 #define BNXT_PTP_GRC_WIN_BASE	0x6000
 
 #define BNXT_MAX_PHC_DRIFT	31000000
+#define BNXT_CYCLES_SHIFT	23
+#define BNXT_DEVCLK_FREQ	1000000
 #define BNXT_LO_TIMER_MASK	0x0000ffffffffUL
 #define BNXT_HI_TIMER_MASK	0xffff00000000UL
 
@@ -88,8 +90,9 @@ struct bnxt_ptp_cfg {
 	u64			old_time;
 	unsigned long		next_period;
 	unsigned long		next_overflow_check;
-	/* 48-bit PHC overflows in 78 hours.  Check overflow every 19 hours. */
-	#define BNXT_PHC_OVERFLOW_PERIOD	(19 * 3600 * HZ)
+	u32			cmult;
+	/* a 23b shift cyclecounter will overflow in ~36 mins.  Check overflow every 18 mins. */
+	#define BNXT_PHC_OVERFLOW_PERIOD	(18 * 60 * HZ)
 
 	u16			tx_seqid;
 	u16			tx_hdr_off;
Signature · 1 line
2.18.1
Reviewed-by on v1 3/3
#6 lore raw

Re: [PATCH net-next 3/3] bnxt_en: Add a non-real time mode to access NIC clock

Quoted 17 lines · Michael Chan

From: Pavan Chebbi <[email protected]>

When using a PHC that is shared between multiple hosts, in order to achieve consistent timestamps across all hosts, we need to isolate the PHC from any host making frequency adjustments.

This patch adds a non-real time mode for this purpose. The implementation is based on a free running NIC hardware timer which is used as the timestamper time-base. Each host implements individual adjustments to a local timecounter based on the NIC free running timer.

Quoted 6 lines
---
 drivers/net/ethernet/broadcom/bnxt/bnxt.c     |  5 ++-
 drivers/net/ethernet/broadcom/bnxt/bnxt_ptp.c | 45 ++++++++++++++-----
 drivers/net/ethernet/broadcom/bnxt/bnxt_ptp.h |  7 ++-
 3 files changed, 42 insertions(+), 15 deletions(-)

Thanks,

#7 lore raw
In reply to Michael Chan
This series was applied to netdev/net-next.git (master) by Paolo Abeni <[email protected]>:

Hello:

This series was applied to netdev/net-next.git (master) by Paolo Abeni <[email protected]>:

Quoted 9 lines · On Sun, 6 Nov 2022 19:16:29 -0500 you wrote:

This small patchset adds an improvement to the configuration of ethtool RSS tuple hash and a PTP improvement when running in a multi-host environment.

Edwin Peer (2):
bnxt_en: refactor VNIC RSS update functions
bnxt_en: update RSS config using difference algorithm

[...]

Here is the summary with links:
- [net-next,1/3] bnxt_en: refactor VNIC RSS update functions

    https://git.kernel.org/netdev/net-next/c/41d2dd42bfa1
  - [net-next,2/3] bnxt_en: update RSS config using difference algorithm
    https://git.kernel.org/netdev/net-next/c/98a4322b70e8
  - [net-next,3/3] bnxt_en: Add a non-real time mode to access NIC clock
    https://git.kernel.org/netdev/net-next/c/85036aee1938

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html