Skip to contentSkip to conversation

mlxsw: Preparations for support of CFF flood mode

PATCH Applied Petr Machata 2 participants 16 messages Nov 20, 2023 – Nov 21, 2023

Subsystems (3)
lore
Share & tools
lore

Petr Machata

author maintainer
#0 lore raw

cover mlxsw: Preparations for support of CFF flood mode

PGT is an in-HW table that maps addresses to sets of ports. Then when some HW process needs a set of ports as an argument, instead of embedding the actual set in the dynamic configuration, what gets configured is the address referencing the set. The HW then works with the appropriate PGT entry.

Among other allocations, the PGT currently contains two large blocks for bridge flooding: one for 802.1q and one for 802.1d. Within each of these blocks are three tables, for unknown-unicast, multicast and broadcast flooding:

      . . . |    802.1q    |    802.1d    | . . .
            | UC | MC | BC | UC | MC | BC |

\______ _____/ \_____ ______/
v v
FID flood vectors

Thus each FID (which corresponds to an 802.1d bridge or one VLAN in an 802.1q bridge) uses three flood vectors spread across a fairly large region of PGT.

This way of organizing the flood table (called "controlled") is not very flexible. E.g. to decrease a bridge scale and store more IP MC vectors, one would need to completely rewrite the bridge PGT blocks, or resort to hacks such as storing individual MC flood vectors into unused part of the bridge table.

In order to address these shortcomings, Spectrum-2 and above support what is called CFF flood mode, for Compressed FID Flooding. In CFF flood mode, each FID has a little table of its own, with three entries adjacent to each other, one for unknown-UC, one for MC, one for BC. This allows for a much more fine-grained approach to PGT management, where bits of it are allocated on demand.

      . . . | FID | FID | FID | FID | FID | . . .
            |U|M|B|U|M|B|U|M|B|U|M|B|U|M|B|

\_____________ _____________/
v
FID flood vectors

Besides the FID table organization, the CFF flood mode also impacts Router Subport (RSP) table. This table contains flood vectors for rFIDs, which are FIDs that reference front panel ports or LAGs. The RSP table contains two entries per front panel port and LAG, one for unknown-UC traffic, and one for everything else. Currently, the FW allocates and manages the table in its own part of PGT. rFIDs are marked with flood_rsp bit and managed specially. In CFF mode, rFIDs are managed as all other FIDs. The driver therefore has to allocate and maintain the flood vectors. Like with bridge FIDs, this is more work, but increases flexibility of the system.

The FW currently supports both the controlled and CFF flood modes. To shed complexity, in the future it should only support CFF flood mode. Hence this patchset, which is the first in series of two to add CFF flood mode support to mlxsw.

There are FW versions out there that do not support CFF flood mode, and on Spectrum-1 in particular, there is no plan to support it at all. mlxsw will therefore have to support both controlled flood mode as well as CFF.

Another aspect is that at least on Spectrum-1, there are FW versions out there that claim to support CFF flood mode, but then reject or ignore configurations enabling the same. The driver thus has to have a say in whether an attempt to configure CFF flood mode should even be made.

Much like with the LAG mode, the feature is therefore expressed in terms of "does the driver prefer CFF flood mode?", and "what flood mode the PCI module managed to configure the FW with". This gives to the driver a chance to determine whether CFF flood mode configuration should be attempted.

In this patchset, we lay the ground with new definitions, registers and their fields, and some minor code shaping. The next patchset will be more focused on introducing necessary abstractions and implementation.

- Patches #1 and #2 add CFF-related items to the command interface.

- Patch #3 adds a new resource, for maximum number of flood profiles supported. (A flood profile is a mapping between traffic type and offset in the per-FID flood vector table.)

- Patches #4 to #8 adjust reg.h. The SFFP register is added, which is used for configuring the abovementioned traffic-type-to-offset mapping. The SFMR, register, which serves for FID configuration, is extended with fields specific to CFF mode. And other minor adjustments.

- Patches #9 and #10 add the plumbing for CFF mode: a way to request that CFF flood mode be configured, and a way to query the flood mode that was actually configured.

- Patch #11 removes dead code.

- Patches #12 and #13 add helpers that the next patchset will make use of. Patch #14 moves RIF setup ahead so that FID code can make use of it.

Petr Machata (14):
mlxsw: cmd: Add cmd_mbox.query_fw.cff_support
mlxsw: cmd: Add MLXSW_CMD_MBOX_CONFIG_PROFILE_FLOOD_MODE_CFF
mlxsw: resources: Add max_cap_nve_flood_prf
mlxsw: reg: Add Switch FID Flooding Profiles Register
mlxsw: reg: Mark SFGC & some SFMR fields as reserved in CFF mode
mlxsw: reg: Drop unnecessary writes from mlxsw_reg_sfmr_pack()
mlxsw: reg: Extract flood-mode specific part of mlxsw_reg_sfmr_pack()
mlxsw: reg: Add to SFMR register the fields related to CFF flood mode
mlxsw: core, pci: Add plumbing related to CFF mode
mlxsw: pci: Permit enabling CFF mode
mlxsw: spectrum_fid: Drop unnecessary conditions
mlxsw: spectrum_fid: Extract SFMR packing into a helper
mlxsw: spectrum_router: Add a helper to get subport number from a RIF
mlxsw: spectrum_router: Call RIF setup before obtaining FID

 drivers/net/ethernet/mellanox/mlxsw/cmd.h     | 11 +++
 drivers/net/ethernet/mellanox/mlxsw/core.c    |  7 ++
 drivers/net/ethernet/mellanox/mlxsw/core.h    |  9 +++
 drivers/net/ethernet/mellanox/mlxsw/pci.c     | 27 ++++++-
 drivers/net/ethernet/mellanox/mlxsw/reg.h     | 78 +++++++++++++++++--
 .../net/ethernet/mellanox/mlxsw/resources.h   |  2 +
 .../net/ethernet/mellanox/mlxsw/spectrum.h    |  2 +
 .../ethernet/mellanox/mlxsw/spectrum_fid.c    | 46 ++++++-----
 .../ethernet/mellanox/mlxsw/spectrum_router.c | 20 ++++-
 9 files changed, 170 insertions(+), 32 deletions(-)
Signature · 1 line
2.41.0
author maintainer
#1 lore raw

PATCH 1/14 mlxsw: cmd: Add cmd_mbox.query_fw.cff_support

PGT, a port-group table is an in-HW block of specialized memory that holds sets of ports. Allocated within the PGT are series of flood tables that describe to which ports traffic of various types (unknown UC, BC, MC) should be flooded from which FID. The hitherto-used layout of these flood tables is being replaced with a more flexible scheme, called compressed FID flooding (CFF). CFF can be configured through CONFIG_PROFILE.flood_mode.

cff_support determines whether CONFIG_PROFILE.flood_mode can be set to CFF.

---
 drivers/net/ethernet/mellanox/mlxsw/cmd.h | 6 ++++++
 1 file changed, 6 insertions(+)
drivers/net/ethernet/mellanox/mlxsw/cmd.h +6 -0
index e827c78be114..b45c9a04fcc4 100644
@@ -282,6 +282,12 @@ MLXSW_ITEM32(cmd_mbox, query_fw, fw_day, 0x14, 0, 8);
  */
 MLXSW_ITEM32(cmd_mbox, query_fw, lag_mode_support, 0x18, 1, 1);
 
+/* cmd_mbox_query_fw_cff_support
+ * 0: CONFIG_PROFILE.flood_mode = 5 (CFF) is not supported by FW
+ * 1: CONFIG_PROFILE.flood_mode = 5 (CFF) is supported by FW
+ */
+MLXSW_ITEM32(cmd_mbox, query_fw, cff_support, 0x18, 2, 1);
+
 /* cmd_mbox_query_fw_clr_int_base_offset
  * Clear Interrupt register's offset from clr_int_bar register
  * in PCI address space.
Signature · 1 line
2.41.0
author maintainer
#2 lore raw

PATCH 2/14 mlxsw: cmd: Add MLXSW_CMD_MBOX_CONFIG_PROFILE_FLOOD_MODE_CFF

In reply to Petr Machata

PGT, a port-group table is an in-HW block of specialized memory that holds sets of ports. Allocated within the PGT are series of flood tables that describe to which ports traffic of various types (unknown UC, BC, MC) should be flooded from which FID. The hitherto-used layout of these flood tables is being replaced with a more flexible scheme, called compressed FID flooding (CFF). CFF can be configured through CONFIG_PROFILE.flood_mode.

In this patch, add MLXSW_CMD_MBOX_CONFIG_PROFILE_FLOOD_MODE_CFF, the value to use to enable the CFF mode.

---
 drivers/net/ethernet/mellanox/mlxsw/cmd.h | 5 +++++
 1 file changed, 5 insertions(+)
drivers/net/ethernet/mellanox/mlxsw/cmd.h +5 -0
index b45c9a04fcc4..e3271c845ee6 100644
@@ -785,6 +785,11 @@ enum mlxsw_cmd_mbox_config_profile_flood_mode {
 	 * used.
 	 */
 	MLXSW_CMD_MBOX_CONFIG_PROFILE_FLOOD_MODE_CONTROLLED = 4,
+	/* CFF - Compressed FID Flood (CFF) mode.
+	 * Reserved when legacy bridge model is used.
+	 * Supported only by Spectrum-2+.
+	 */
+	MLXSW_CMD_MBOX_CONFIG_PROFILE_FLOOD_MODE_CFF = 5,
 };
 
 /* cmd_mbox_config_profile_flood_mode
Signature · 1 line
2.41.0
author maintainer
#3 lore raw

PATCH 3/14 mlxsw: resources: Add max_cap_nve_flood_prf

In reply to Petr Machata

max_cap_nve_flood_prf describes maximum number of NVE flooding profiles. The same value then applies for flooding profiles for flooding in CFF mode.

---
 drivers/net/ethernet/mellanox/mlxsw/resources.h | 2 ++
 1 file changed, 2 insertions(+)
drivers/net/ethernet/mellanox/mlxsw/resources.h +2 -0
index 89dd2777ec4d..9d7977ebe186 100644
@@ -27,6 +27,7 @@ enum mlxsw_res_id {
 	MLXSW_RES_ID_FID,
 	MLXSW_RES_ID_MAX_LAG,
 	MLXSW_RES_ID_MAX_LAG_MEMBERS,
+	MLXSW_RES_ID_MAX_NVE_FLOOD_PRF,
 	MLXSW_RES_ID_GUARANTEED_SHARED_BUFFER,
 	MLXSW_RES_ID_CELL_SIZE,
 	MLXSW_RES_ID_MAX_HEADROOM_SIZE,
@@ -88,6 +89,7 @@ static u16 mlxsw_res_ids[] = {
 	[MLXSW_RES_ID_FID] = 0x2512,
 	[MLXSW_RES_ID_MAX_LAG] = 0x2520,
 	[MLXSW_RES_ID_MAX_LAG_MEMBERS] = 0x2521,
+	[MLXSW_RES_ID_MAX_NVE_FLOOD_PRF] = 0x2522,
 	[MLXSW_RES_ID_GUARANTEED_SHARED_BUFFER] = 0x2805,	/* Bytes */
 	[MLXSW_RES_ID_CELL_SIZE] = 0x2803,	/* Bytes */
 	[MLXSW_RES_ID_MAX_HEADROOM_SIZE] = 0x2811,	/* Bytes */
Signature · 1 line
2.41.0
author maintainer
#4 lore raw

PATCH 4/14 mlxsw: reg: Add Switch FID Flooding Profiles Register

In reply to Petr Machata

The SFFP register populates the fid flooding profile tables used for the NVE flooding and Compressed-FID Flooding (CFF).

---
 drivers/net/ethernet/mellanox/mlxsw/reg.h | 45 +++++++++++++++++++++++
 1 file changed, 45 insertions(+)
drivers/net/ethernet/mellanox/mlxsw/reg.h +45 -0
index e26e9d06bd72..3472f70b2482 100644
@@ -2168,6 +2168,50 @@ static inline void mlxsw_reg_spvc_pack(char *payload, u16 local_port, bool et1,
 	mlxsw_reg_spvc_et0_set(payload, et0);
 }
 
+/* SFFP - Switch FID Flooding Profiles Register
+ * --------------------------------------------
+ * The SFFP register populates the fid flooding profile tables used for the NVE
+ * flooding and Compressed-FID Flooding (CFF).
+ *
+ * Reserved on Spectrum-1.
+ */
+#define MLXSW_REG_SFFP_ID 0x2029
+#define MLXSW_REG_SFFP_LEN 0x0C
+
+MLXSW_REG_DEFINE(sffp, MLXSW_REG_SFFP_ID, MLXSW_REG_SFFP_LEN);
+
+/* reg_sffp_profile_id
+ * Profile ID a.k.a. SFMR.nve_flood_prf_id or SFMR.cff_prf_id
+ * Range 0..max_cap_nve_flood_prf-1
+ * Access: Index
+ */
+MLXSW_ITEM32(reg, sffp, profile_id, 0x00, 16, 2);
+
+/* reg_sffp_type
+ * The traffic type to reach the flooding table.
+ * Same as SFGC.type
+ * Access: Index
+ */
+MLXSW_ITEM32(reg, sffp, type, 0x00, 0, 4);
+
+/* reg_sffp_flood_offset
+ * Flood offset. Offset to add to SFMR.cff_mid_base to get the final PGT address
+ * for FID flood; or offset to add to SFMR.nve_tunnel_flood_ptr to get KVD
+ * pointer for NVE underlay.
+ * Access: RW
+ */
+MLXSW_ITEM32(reg, sffp, flood_offset, 0x04, 0, 3);
+
+static inline void mlxsw_reg_sffp_pack(char *payload, u8 profile_id,
+				       enum mlxsw_reg_sfgc_type type,
+				       u8 flood_offset)
+{
+	MLXSW_REG_ZERO(sffp, payload);
+	mlxsw_reg_sffp_profile_id_set(payload, profile_id);
+	mlxsw_reg_sffp_type_set(payload, type);
+	mlxsw_reg_sffp_flood_offset_set(payload, flood_offset);
+}
+
 /* SPEVET - Switch Port Egress VLAN EtherType
  * ------------------------------------------
  * The switch port egress VLAN EtherType configures which EtherType to push at
@@ -12946,6 +12990,7 @@ static const struct mlxsw_reg_info *mlxsw_reg_infos[] = {
 	MLXSW_REG(spvmlr),
 	MLXSW_REG(spfsr),
 	MLXSW_REG(spvc),
+	MLXSW_REG(sffp),
 	MLXSW_REG(spevet),
 	MLXSW_REG(smpe),
 	MLXSW_REG(smid2),
Signature · 1 line
2.41.0
author maintainer
#5 lore raw

PATCH 5/14 mlxsw: reg: Mark SFGC & some SFMR fields as reserved in CFF mode

In reply to Petr Machata

Some existing fields and the whole register of SFGC are reserved in CFF mode. Backport the reservation note to these fields.

---
 drivers/net/ethernet/mellanox/mlxsw/reg.h | 6 ++++++
 1 file changed, 6 insertions(+)
drivers/net/ethernet/mellanox/mlxsw/reg.h +6 -0
index 3472f70b2482..ec0adddd4598 100644
@@ -1024,6 +1024,8 @@ static inline void mlxsw_reg_spaft_pack(char *payload, u16 local_port,
  * ------------------------------------------
  * The following register controls the association of flooding tables and MIDs
  * to packet types used for flooding.
+ *
+ * Reserved when CONFIG_PROFILE.flood_mode = CFF.
  */
 #define MLXSW_REG_SFGC_ID 0x2011
 #define MLXSW_REG_SFGC_LEN 0x14
@@ -1862,6 +1864,7 @@ MLXSW_ITEM32(reg, sfmr, fid, 0x00, 0, 16);
  * Access: RW
  *
  * Note: Reserved when legacy bridge model is used.
+ * Reserved when CONFIG_PROFILE.flood_mode = CFF.
  */
 MLXSW_ITEM32(reg, sfmr, flood_rsp, 0x08, 31, 1);
 
@@ -1872,6 +1875,7 @@ MLXSW_ITEM32(reg, sfmr, flood_rsp, 0x08, 31, 1);
  * Access: RW
  *
  * Note: Reserved when legacy bridge model is used and when flood_rsp=1.
+ * Reserved when CONFIG_PROFILE.flood_mode = CFF
  */
 MLXSW_ITEM32(reg, sfmr, flood_bridge_type, 0x08, 28, 1);
 
@@ -1880,6 +1884,8 @@ MLXSW_ITEM32(reg, sfmr, flood_bridge_type, 0x08, 28, 1);
  * Used to point into the flooding table selected by SFGC register if
  * the table is of type FID-Offset. Otherwise, this field is reserved.
  * Access: RW
+ *
+ * Note: Reserved when CONFIG_PROFILE.flood_mode = CFF
  */
 MLXSW_ITEM32(reg, sfmr, fid_offset, 0x08, 0, 16);
 
Signature · 1 line
2.41.0
author maintainer
#6 lore raw

PATCH 6/14 mlxsw: reg: Drop unnecessary writes from mlxsw_reg_sfmr_pack()

In reply to Petr Machata

The MLXSW_REG_ZERO at the beginning of the function wipes the whole payload. There's no need to set vtfp and vv to false explicitly.

---
 drivers/net/ethernet/mellanox/mlxsw/reg.h | 2 --
 1 file changed, 2 deletions(-)
drivers/net/ethernet/mellanox/mlxsw/reg.h +0 -2
index ec0adddd4598..e8f7a4741bd3 100644
@@ -1973,8 +1973,6 @@ static inline void mlxsw_reg_sfmr_pack(char *payload,
 	mlxsw_reg_sfmr_op_set(payload, op);
 	mlxsw_reg_sfmr_fid_set(payload, fid);
 	mlxsw_reg_sfmr_fid_offset_set(payload, fid_offset);
-	mlxsw_reg_sfmr_vtfp_set(payload, false);
-	mlxsw_reg_sfmr_vv_set(payload, false);
 	mlxsw_reg_sfmr_flood_rsp_set(payload, flood_rsp);
 	mlxsw_reg_sfmr_flood_bridge_type_set(payload, bridge_type);
 	mlxsw_reg_sfmr_smpe_valid_set(payload, smpe_valid);
Signature · 1 line
2.41.0
author maintainer
#7 lore raw

PATCH 7/14 mlxsw: reg: Extract flood-mode specific part of mlxsw_reg_sfmr_pack()

In reply to Petr Machata

In CFF mode, it is necessary to set a different set of SFMR fields. Leave in mlxsw_reg_sfmr_pack() only the common bits, and move the parts relevant to controlled flood mode directly to the call site.

---
 drivers/net/ethernet/mellanox/mlxsw/reg.h         |  5 -----
 .../net/ethernet/mellanox/mlxsw/spectrum_fid.c    | 15 ++++++++++-----
 2 files changed, 10 insertions(+), 10 deletions(-)
drivers/net/ethernet/mellanox/mlxsw/reg.h +0 -5
index e8f7a4741bd3..bd709f7fcae1 100644
@@ -1965,16 +1965,11 @@ MLXSW_ITEM32(reg, sfmr, smpe, 0x28, 0, 16);
 
 static inline void mlxsw_reg_sfmr_pack(char *payload,
 				       enum mlxsw_reg_sfmr_op op, u16 fid,
-				       u16 fid_offset, bool flood_rsp,
-				       enum mlxsw_reg_bridge_type bridge_type,
 				       bool smpe_valid, u16 smpe)
 {
 	MLXSW_REG_ZERO(sfmr, payload);
 	mlxsw_reg_sfmr_op_set(payload, op);
 	mlxsw_reg_sfmr_fid_set(payload, fid);
-	mlxsw_reg_sfmr_fid_offset_set(payload, fid_offset);
-	mlxsw_reg_sfmr_flood_rsp_set(payload, flood_rsp);
-	mlxsw_reg_sfmr_flood_bridge_type_set(payload, bridge_type);
 	mlxsw_reg_sfmr_smpe_valid_set(payload, smpe_valid);
 	mlxsw_reg_sfmr_smpe_set(payload, smpe);
 }
drivers/net/ethernet/mellanox/mlxsw/spectrum_fid.c +10 -5
index e954b8cd2ee8..6a509913bdc7 100644
@@ -433,9 +433,12 @@ static int mlxsw_sp_fid_op(const struct mlxsw_sp_fid *fid, bool valid)
 	smpe = fid->fid_family->smpe_index_valid ? fid->fid_index : 0;
 
 	mlxsw_reg_sfmr_pack(sfmr_pl, mlxsw_sp_sfmr_op(valid), fid->fid_index,
-			    fid->fid_offset, fid->fid_family->flood_rsp,
-			    fid->fid_family->bridge_type,
 			    fid->fid_family->smpe_index_valid, smpe);
+	mlxsw_reg_sfmr_fid_offset_set(sfmr_pl, fid->fid_offset);
+	mlxsw_reg_sfmr_flood_rsp_set(sfmr_pl, fid->fid_family->flood_rsp);
+	mlxsw_reg_sfmr_flood_bridge_type_set(sfmr_pl,
+					     fid->fid_family->bridge_type);
+
 	return mlxsw_reg_write(mlxsw_sp->core, MLXSW_REG(sfmr), sfmr_pl);
 }
 
@@ -449,10 +452,12 @@ static int mlxsw_sp_fid_edit_op(const struct mlxsw_sp_fid *fid,
 	smpe = fid->fid_family->smpe_index_valid ? fid->fid_index : 0;
 
 	mlxsw_reg_sfmr_pack(sfmr_pl, MLXSW_REG_SFMR_OP_CREATE_FID,
-			    fid->fid_index, fid->fid_offset,
-			    fid->fid_family->flood_rsp,
-			    fid->fid_family->bridge_type,
+			    fid->fid_index,
 			    fid->fid_family->smpe_index_valid, smpe);
+	mlxsw_reg_sfmr_fid_offset_set(sfmr_pl, fid->fid_offset);
+	mlxsw_reg_sfmr_flood_rsp_set(sfmr_pl, fid->fid_family->flood_rsp);
+	mlxsw_reg_sfmr_flood_bridge_type_set(sfmr_pl,
+					     fid->fid_family->bridge_type);
 	mlxsw_reg_sfmr_vv_set(sfmr_pl, fid->vni_valid);
 	mlxsw_reg_sfmr_vni_set(sfmr_pl, be32_to_cpu(fid->vni));
 	mlxsw_reg_sfmr_vtfp_set(sfmr_pl, fid->nve_flood_index_valid);
Signature · 1 line
2.41.0
author maintainer
#8 lore raw

PATCH 8/14 mlxsw: reg: Add to SFMR register the fields related to CFF flood mode

In reply to Petr Machata

Add the field cff_mid_base, which specifies at which point in PGT the per-FID flood table is stored. Add cff_prf_id, the profile ID, which determines on which row of the flood table a flood vector can be found for a given traffic type.

---
 drivers/net/ethernet/mellanox/mlxsw/reg.h | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)
drivers/net/ethernet/mellanox/mlxsw/reg.h +20 -0
index bd709f7fcae1..3aae4467e431 100644
@@ -1944,6 +1944,26 @@ MLXSW_ITEM32(reg, sfmr, irif_v, 0x14, 24, 1);
  */
 MLXSW_ITEM32(reg, sfmr, irif, 0x14, 0, 16);
 
+/* reg_sfmr_cff_mid_base
+ * Pointer to PGT table.
+ * Range: 0..(cap_max_pgt-1)
+ * Access: RW
+ *
+ * Note: Reserved when SwitchX/-2 and Spectrum-1.
+ * Supported when CONFIG_PROFILE.flood_mode = CFF.
+ */
+MLXSW_ITEM32(reg, sfmr, cff_mid_base, 0x20, 0, 16);
+
+/* reg_sfmr_cff_prf_id
+ * Compressed Fid Flooding profile_id
+ * Range 0..(max_cap_nve_flood_prf-1)
+ * Access: RW
+ *
+ * Note: Reserved when SwitchX/-2 and Spectrum-1
+ * Supported only when CONFIG_PROFLE.flood_mode = CFF.
+ */
+MLXSW_ITEM32(reg, sfmr, cff_prf_id, 0x24, 0, 2);
+
 /* reg_sfmr_smpe_valid
  * SMPE is valid.
  * Access: RW
Signature · 1 line
2.41.0
author maintainer
#9 lore raw

PATCH 9/14 mlxsw: core, pci: Add plumbing related to CFF mode

In reply to Petr Machata

CFF mode, for Compressed FID Flooding, is a way of organizing flood vectors in the PGT table. The bus module determines whether CFF is supported, can configure flood mode to CFF if it is, and knows what flood mode has been configured. Therefore add a bus callback to determine the configured flood mode. Also add to core an API to query it.

Since after this patch, we rely on mlxsw_pci->flood_mode being set, it becomes a coding error if a driver invokes this function with a set of fields that misses the initialization. Warn and bail out in that case.

The CFF mode is not used as of this patch. The code to actually use it will be added later.

---
 drivers/net/ethernet/mellanox/mlxsw/core.c |  7 +++++++
 drivers/net/ethernet/mellanox/mlxsw/core.h |  3 +++
 drivers/net/ethernet/mellanox/mlxsw/pci.c  | 18 ++++++++++++++++++
 3 files changed, 28 insertions(+)
drivers/net/ethernet/mellanox/mlxsw/core.c +7 -0
index f23421f038f3..e4d7739bd7c8 100644
@@ -211,6 +211,13 @@ mlxsw_core_lag_mode(struct mlxsw_core *mlxsw_core)
 }
 EXPORT_SYMBOL(mlxsw_core_lag_mode);
 
+enum mlxsw_cmd_mbox_config_profile_flood_mode
+mlxsw_core_flood_mode(struct mlxsw_core *mlxsw_core)
+{
+	return mlxsw_core->bus->flood_mode(mlxsw_core->bus_priv);
+}
+EXPORT_SYMBOL(mlxsw_core_flood_mode);
+
 void *mlxsw_core_driver_priv(struct mlxsw_core *mlxsw_core)
 {
 	return mlxsw_core->driver_priv;
drivers/net/ethernet/mellanox/mlxsw/core.h +3 -0
index 764d14bd5bc0..a93e9c38848a 100644
@@ -38,6 +38,8 @@ unsigned int mlxsw_core_max_ports(const struct mlxsw_core *mlxsw_core);
 int mlxsw_core_max_lag(struct mlxsw_core *mlxsw_core, u16 *p_max_lag);
 enum mlxsw_cmd_mbox_config_profile_lag_mode
 mlxsw_core_lag_mode(struct mlxsw_core *mlxsw_core);
+enum mlxsw_cmd_mbox_config_profile_flood_mode
+mlxsw_core_flood_mode(struct mlxsw_core *mlxsw_core);
 
 void *mlxsw_core_driver_priv(struct mlxsw_core *mlxsw_core);
 
@@ -489,6 +491,7 @@ struct mlxsw_bus {
 	u32 (*read_utc_sec)(void *bus_priv);
 	u32 (*read_utc_nsec)(void *bus_priv);
 	enum mlxsw_cmd_mbox_config_profile_lag_mode (*lag_mode)(void *bus_priv);
+	enum mlxsw_cmd_mbox_config_profile_flood_mode (*flood_mode)(void *priv);
 	u8 features;
 };
 
drivers/net/ethernet/mellanox/mlxsw/pci.c +18 -0
index 5b1f2483a3cc..845edd43032b 100644
@@ -106,7 +106,9 @@ struct mlxsw_pci {
 	u64 utc_sec_offset;
 	u64 utc_nsec_offset;
 	bool lag_mode_support;
+	bool cff_support;
 	enum mlxsw_cmd_mbox_config_profile_lag_mode lag_mode;
+	enum mlxsw_cmd_mbox_config_profile_flood_mode flood_mode;
 	struct mlxsw_pci_queue_type_group queues[MLXSW_PCI_QUEUE_TYPE_COUNT];
 	u32 doorbell_offset;
 	struct mlxsw_core *core;
@@ -1251,6 +1253,10 @@ static int mlxsw_pci_config_profile(struct mlxsw_pci *mlxsw_pci, char *mbox,
 			mbox, 1);
 		mlxsw_cmd_mbox_config_profile_flood_mode_set(
 			mbox, profile->flood_mode);
+		mlxsw_pci->flood_mode = profile->flood_mode;
+	} else {
+		WARN_ON(1);
+		return -EINVAL;
 	}
 	if (profile->used_max_ib_mc) {
 		mlxsw_cmd_mbox_config_profile_set_max_ib_mc_set(
@@ -1654,6 +1660,9 @@ static int mlxsw_pci_init(void *bus_priv, struct mlxsw_core *mlxsw_core,
 
 	mlxsw_pci->lag_mode_support =
 		mlxsw_cmd_mbox_query_fw_lag_mode_support_get(mbox);
+	mlxsw_pci->cff_support =
+		mlxsw_cmd_mbox_query_fw_cff_support_get(mbox);
+
 	num_pages = mlxsw_cmd_mbox_query_fw_fw_pages_get(mbox);
 	err = mlxsw_pci_fw_area_init(mlxsw_pci, mbox, num_pages);
 	if (err)
@@ -1970,6 +1979,14 @@ mlxsw_pci_lag_mode(void *bus_priv)
 	return mlxsw_pci->lag_mode;
 }
 
+static enum mlxsw_cmd_mbox_config_profile_flood_mode
+mlxsw_pci_flood_mode(void *bus_priv)
+{
+	struct mlxsw_pci *mlxsw_pci = bus_priv;
+
+	return mlxsw_pci->flood_mode;
+}
+
 static const struct mlxsw_bus mlxsw_pci_bus = {
 	.kind			= "pci",
 	.init			= mlxsw_pci_init,
@@ -1982,6 +1999,7 @@ static const struct mlxsw_bus mlxsw_pci_bus = {
 	.read_utc_sec		= mlxsw_pci_read_utc_sec,
 	.read_utc_nsec		= mlxsw_pci_read_utc_nsec,
 	.lag_mode		= mlxsw_pci_lag_mode,
+	.flood_mode		= mlxsw_pci_flood_mode,
 	.features		= MLXSW_BUS_F_TXRX | MLXSW_BUS_F_RESET,
 };
 
Signature · 1 line
2.41.0
author maintainer
#10 lore raw

PATCH 10/14 mlxsw: pci: Permit enabling CFF mode

In reply to Petr Machata

There are FW versions out there that do not support CFF flood mode, and on Spectrum-1 in particular, there is no plan to support it at all. mlxsw will therefore have to support both controlled flood mode as well as CFF. There are also FW versions out there that claim to support CFF flood mode, but then reject or ignore configurations enabling the same. The driver thus has to have a say in whether an attempt to configure CFF flood mode should even be made, and what to use as a fallback.

Hence express the feature in terms of "does the driver prefer CFF flood mode?", and "what flood mode the PCI module managed to configure the FW with". This gives to the driver a chance to determine whether CFF flood mode configuration should be attempted.

The latter bit was added in previous patches. In this patch, add the bit that allows the driver to determine whether CFF enablement should be attempted, and the enablement code itself.

---
 drivers/net/ethernet/mellanox/mlxsw/core.h | 6 ++++++
 drivers/net/ethernet/mellanox/mlxsw/pci.c  | 9 ++++++++-
 2 files changed, 14 insertions(+), 1 deletion(-)
drivers/net/ethernet/mellanox/mlxsw/core.h +6 -0
index a93e9c38848a..6d11225594dd 100644
@@ -324,7 +324,12 @@ struct mlxsw_config_profile {
 	u16	max_regions;
 	u8	max_flood_tables;
 	u8	max_vid_flood_tables;
+
+	/* Flood mode to use if used_flood_mode. If flood_mode_prefer_cff,
+	 * the backup flood mode (if any) when CFF unsupported.
+	 */
 	u8	flood_mode;
+
 	u8	max_fid_offset_flood_tables;
 	u16	fid_offset_flood_table_size;
 	u8	max_fid_flood_tables;
@@ -340,6 +345,7 @@ struct mlxsw_config_profile {
 	u8	kvd_hash_double_parts;
 	u8	cqe_time_stamp_type;
 	bool	lag_mode_prefer_sw;
+	bool	flood_mode_prefer_cff;
 	struct mlxsw_swid_config swid_config[MLXSW_CONFIG_PROFILE_SWID_COUNT];
 };
 
drivers/net/ethernet/mellanox/mlxsw/pci.c +8 -1
index 845edd43032b..0d58f13a7c7d 100644
@@ -1248,7 +1248,14 @@ static int mlxsw_pci_config_profile(struct mlxsw_pci *mlxsw_pci, char *mbox,
 		mlxsw_cmd_mbox_config_profile_fid_flood_table_size_set(
 			mbox, profile->fid_flood_table_size);
 	}
-	if (profile->used_flood_mode) {
+	if (profile->flood_mode_prefer_cff && mlxsw_pci->cff_support) {
+		enum mlxsw_cmd_mbox_config_profile_flood_mode flood_mode =
+			MLXSW_CMD_MBOX_CONFIG_PROFILE_FLOOD_MODE_CFF;
+
+		mlxsw_cmd_mbox_config_profile_set_flood_mode_set(mbox, 1);
+		mlxsw_cmd_mbox_config_profile_flood_mode_set(mbox, flood_mode);
+		mlxsw_pci->flood_mode = flood_mode;
+	} else if (profile->used_flood_mode) {
 		mlxsw_cmd_mbox_config_profile_set_flood_mode_set(
 			mbox, 1);
 		mlxsw_cmd_mbox_config_profile_flood_mode_set(
Signature · 1 line
2.41.0
author maintainer
#11 lore raw

PATCH 11/14 mlxsw: spectrum_fid: Drop unnecessary conditions

In reply to Petr Machata

The caller already only calls mlxsw_sp_fid_flood_tables_init() and mlxsw_sp_fid_flood_tables_fini() if (fid_family->flood_tables). There is no configuration where the pointer is non-NULL, but the number of tables is zero. So drop the conditions.

---
 drivers/net/ethernet/mellanox/mlxsw/spectrum_fid.c | 6 ------
 1 file changed, 6 deletions(-)
drivers/net/ethernet/mellanox/mlxsw/spectrum_fid.c +0 -6
index 6a509913bdc7..d7fc579f3b29 100644
@@ -1692,9 +1692,6 @@ mlxsw_sp_fid_flood_tables_init(struct mlxsw_sp_fid_family *fid_family)
 	int err;
 	int i;
 
-	if (!fid_family->nr_flood_tables)
-		return 0;
-
 	pgt_size = mlxsw_sp_fid_family_pgt_size(fid_family);
 	err = mlxsw_sp_pgt_mid_alloc_range(mlxsw_sp, &fid_family->pgt_base,
 					   pgt_size);
@@ -1723,9 +1720,6 @@ mlxsw_sp_fid_flood_tables_fini(struct mlxsw_sp_fid_family *fid_family)
 	struct mlxsw_sp *mlxsw_sp = fid_family->mlxsw_sp;
 	u16 pgt_size;
 
-	if (!fid_family->nr_flood_tables)
-		return;
-
 	pgt_size = mlxsw_sp_fid_family_pgt_size(fid_family);
 	mlxsw_sp_pgt_mid_free_range(mlxsw_sp, fid_family->pgt_base, pgt_size);
 }
Signature · 1 line
2.41.0
author maintainer
#12 lore raw

PATCH 12/14 mlxsw: spectrum_fid: Extract SFMR packing into a helper

In reply to Petr Machata

Both mlxsw_sp_fid_op() and mlxsw_sp_fid_edit_op() pack the core of SFMR the same way. Extract the common code into a helper and call that. Extract out of that a wrapper that just calls mlxsw_reg_sfmr_pack(), because it will be useful for the dummy family later on.

---
 .../ethernet/mellanox/mlxsw/spectrum_fid.c    | 45 ++++++++++---------
 1 file changed, 25 insertions(+), 20 deletions(-)
drivers/net/ethernet/mellanox/mlxsw/spectrum_fid.c +25 -20
index d7fc579f3b29..aad4bb17dfb1 100644
@@ -424,21 +424,35 @@ static enum mlxsw_reg_sfmr_op mlxsw_sp_sfmr_op(bool valid)
 		       MLXSW_REG_SFMR_OP_DESTROY_FID;
 }
 
+static void mlxsw_sp_fid_pack(char *sfmr_pl,
+			      const struct mlxsw_sp_fid *fid,
+			      enum mlxsw_reg_sfmr_op op)
+{
+	u16 smpe;
+
+	smpe = fid->fid_family->smpe_index_valid ? fid->fid_index : 0;
+
+	mlxsw_reg_sfmr_pack(sfmr_pl, op, fid->fid_index,
+			    fid->fid_family->smpe_index_valid, smpe);
+}
+
+static void mlxsw_sp_fid_pack_ctl(char *sfmr_pl,
+				  const struct mlxsw_sp_fid *fid,
+				  enum mlxsw_reg_sfmr_op op)
+{
+	mlxsw_sp_fid_pack(sfmr_pl, fid, op);
+	mlxsw_reg_sfmr_fid_offset_set(sfmr_pl, fid->fid_offset);
+	mlxsw_reg_sfmr_flood_rsp_set(sfmr_pl, fid->fid_family->flood_rsp);
+	mlxsw_reg_sfmr_flood_bridge_type_set(sfmr_pl,
+					     fid->fid_family->bridge_type);
+}
+
 static int mlxsw_sp_fid_op(const struct mlxsw_sp_fid *fid, bool valid)
 {
 	struct mlxsw_sp *mlxsw_sp = fid->fid_family->mlxsw_sp;
 	char sfmr_pl[MLXSW_REG_SFMR_LEN];
-	u16 smpe;
-
-	smpe = fid->fid_family->smpe_index_valid ? fid->fid_index : 0;
-
-	mlxsw_reg_sfmr_pack(sfmr_pl, mlxsw_sp_sfmr_op(valid), fid->fid_index,
-			    fid->fid_family->smpe_index_valid, smpe);
-	mlxsw_reg_sfmr_fid_offset_set(sfmr_pl, fid->fid_offset);
-	mlxsw_reg_sfmr_flood_rsp_set(sfmr_pl, fid->fid_family->flood_rsp);
-	mlxsw_reg_sfmr_flood_bridge_type_set(sfmr_pl,
-					     fid->fid_family->bridge_type);
 
+	mlxsw_sp_fid_pack_ctl(sfmr_pl, fid, mlxsw_sp_sfmr_op(valid));
 	return mlxsw_reg_write(mlxsw_sp->core, MLXSW_REG(sfmr), sfmr_pl);
 }
 
@@ -447,17 +461,8 @@ static int mlxsw_sp_fid_edit_op(const struct mlxsw_sp_fid *fid,
 {
 	struct mlxsw_sp *mlxsw_sp = fid->fid_family->mlxsw_sp;
 	char sfmr_pl[MLXSW_REG_SFMR_LEN];
-	u16 smpe;
 
-	smpe = fid->fid_family->smpe_index_valid ? fid->fid_index : 0;
-
-	mlxsw_reg_sfmr_pack(sfmr_pl, MLXSW_REG_SFMR_OP_CREATE_FID,
-			    fid->fid_index,
-			    fid->fid_family->smpe_index_valid, smpe);
-	mlxsw_reg_sfmr_fid_offset_set(sfmr_pl, fid->fid_offset);
-	mlxsw_reg_sfmr_flood_rsp_set(sfmr_pl, fid->fid_family->flood_rsp);
-	mlxsw_reg_sfmr_flood_bridge_type_set(sfmr_pl,
-					     fid->fid_family->bridge_type);
+	mlxsw_sp_fid_pack_ctl(sfmr_pl, fid, MLXSW_REG_SFMR_OP_CREATE_FID);
 	mlxsw_reg_sfmr_vv_set(sfmr_pl, fid->vni_valid);
 	mlxsw_reg_sfmr_vni_set(sfmr_pl, be32_to_cpu(fid->vni));
 	mlxsw_reg_sfmr_vtfp_set(sfmr_pl, fid->nve_flood_index_valid);
Signature · 1 line
2.41.0
author maintainer
#13 lore raw

PATCH 13/14 mlxsw: spectrum_router: Add a helper to get subport number from a RIF

In reply to Petr Machata

In the CFF flood mode, responsibility for management of the PGT entries for rFIDs is moved from FW to the driver. All rFIDs are based off either a front panel port, or a LAG port. The flood vectors for port-based rFIDs enable just the port itself, the ones for LAG-based rFIDs enable all member ports of the LAG in question.

Since all rFIDs based off the same port have the same flood vector, and similarly for LAG-based rFIDs, the flood entries are shared. The PGT address of the flood vector is therefore determined based on the port (or LAG) number of the RIF connected with the rFID.

Add a helper to determine subport number given a RIF, to be used in these calculations.

---
 drivers/net/ethernet/mellanox/mlxsw/spectrum.h     |  2 ++
 .../net/ethernet/mellanox/mlxsw/spectrum_router.c  | 14 ++++++++++++++
 2 files changed, 16 insertions(+)
drivers/net/ethernet/mellanox/mlxsw/spectrum.h +2 -0
index c70333b460ea..800c461deefa 100644
@@ -753,6 +753,8 @@ union mlxsw_sp_l3addr {
 };
 
 u16 mlxsw_sp_rif_index(const struct mlxsw_sp_rif *rif);
+int mlxsw_sp_rif_subport_port(const struct mlxsw_sp_rif *rif,
+			      u16 *port, bool *is_lag);
 int mlxsw_sp_router_init(struct mlxsw_sp *mlxsw_sp,
 			 struct netlink_ext_ack *extack);
 void mlxsw_sp_router_fini(struct mlxsw_sp *mlxsw_sp);
drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c +14 -0
index 82a95125d9ca..a358ceb4e1d0 100644
@@ -8660,6 +8660,20 @@ mlxsw_sp_rif_subport_rif(const struct mlxsw_sp_rif *rif)
 	return container_of(rif, struct mlxsw_sp_rif_subport, common);
 }
 
+int mlxsw_sp_rif_subport_port(const struct mlxsw_sp_rif *rif,
+			      u16 *port, bool *is_lag)
+{
+	struct mlxsw_sp_rif_subport *rif_subport;
+
+	if (WARN_ON(rif->ops->type != MLXSW_SP_RIF_TYPE_SUBPORT))
+		return -EINVAL;
+
+	rif_subport = mlxsw_sp_rif_subport_rif(rif);
+	*is_lag = rif_subport->lag;
+	*port = *is_lag ? rif_subport->lag_id : rif_subport->system_port;
+	return 0;
+}
+
 static struct mlxsw_sp_rif *
 mlxsw_sp_rif_subport_get(struct mlxsw_sp *mlxsw_sp,
 			 const struct mlxsw_sp_rif_params *params,
Signature · 1 line
2.41.0
author maintainer
#14 lore raw

PATCH 14/14 mlxsw: spectrum_router: Call RIF setup before obtaining FID

In reply to Petr Machata

For subport RIFs, the setup initializes, among other things, RIF port and LAG numbers. Those are important to determine where in the PGT the RIF FID will be stored. Therefore, call the RIF setup before fid_get.

---
 drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c +3 -3
index a358ceb4e1d0..2c255ed9b8a9 100644
@@ -8419,6 +8419,9 @@ mlxsw_sp_rif_create(struct mlxsw_sp *mlxsw_sp,
 	rif->ops = ops;
 	rif->rif_entries = rif_entries;
 
+	if (ops->setup)
+		ops->setup(rif, params);
+
 	if (ops->fid_get) {
 		fid = ops->fid_get(rif, params, extack);
 		if (IS_ERR(fid)) {
@@ -8428,9 +8431,6 @@ mlxsw_sp_rif_create(struct mlxsw_sp *mlxsw_sp,
 		rif->fid = fid;
 	}
 
-	if (ops->setup)
-		ops->setup(rif, params);
-
 	err = ops->configure(rif, extack);
 	if (err)
 		goto err_configure;
Signature · 1 line
2.41.0
#15 lore raw
In reply to Petr Machata
This series was applied to netdev/net-next.git (main) by Jakub Kicinski <[email protected]>:

Hello:

This series was applied to netdev/net-next.git (main) by Jakub Kicinski <[email protected]>:

Quoted 6 lines · On Mon, 20 Nov 2023 19:25:17 +0100 you wrote:

PGT is an in-HW table that maps addresses to sets of ports. Then when some HW process needs a set of ports as an argument, instead of embedding the actual set in the dynamic configuration, what gets configured is the address referencing the set. The HW then works with the appropriate PGT entry.

Quoted 6 lines

Among other allocations, the PGT currently contains two large blocks for bridge flooding: one for 802.1q and one for 802.1d. Within each of these blocks are three tables, for unknown-unicast, multicast and broadcast flooding:

[...]

Here is the summary with links:
- [net-next,01/14] mlxsw: cmd: Add cmd_mbox.query_fw.cff_support

    https://git.kernel.org/netdev/net-next/c/8405d6626289
  - [net-next,02/14] mlxsw: cmd: Add MLXSW_CMD_MBOX_CONFIG_PROFILE_FLOOD_MODE_CFF
    https://git.kernel.org/netdev/net-next/c/50ee67789b82
  - [net-next,03/14] mlxsw: resources: Add max_cap_nve_flood_prf
    https://git.kernel.org/netdev/net-next/c/2d19da927719
  - [net-next,04/14] mlxsw: reg: Add Switch FID Flooding Profiles Register
    https://git.kernel.org/netdev/net-next/c/e1e4ce6c6d54
  - [net-next,05/14] mlxsw: reg: Mark SFGC & some SFMR fields as reserved in CFF mode
    https://git.kernel.org/netdev/net-next/c/7eb902954b62
  - [net-next,06/14] mlxsw: reg: Drop unnecessary writes from mlxsw_reg_sfmr_pack()
    https://git.kernel.org/netdev/net-next/c/642d6a2033d8
  - [net-next,07/14] mlxsw: reg: Extract flood-mode specific part of mlxsw_reg_sfmr_pack()
    https://git.kernel.org/netdev/net-next/c/446bc1e9dec6
  - [net-next,08/14] mlxsw: reg: Add to SFMR register the fields related to CFF flood mode
    https://git.kernel.org/netdev/net-next/c/6b10371c386c
  - [net-next,09/14] mlxsw: core, pci: Add plumbing related to CFF mode
    https://git.kernel.org/netdev/net-next/c/095915956867
  - [net-next,10/14] mlxsw: pci: Permit enabling CFF mode
    https://git.kernel.org/netdev/net-next/c/9aad19a363f6
  - [net-next,11/14] mlxsw: spectrum_fid: Drop unnecessary conditions
    https://git.kernel.org/netdev/net-next/c/b51c876c2297
  - [net-next,12/14] mlxsw: spectrum_fid: Extract SFMR packing into a helper
    https://git.kernel.org/netdev/net-next/c/2b7bccd1f167
  - [net-next,13/14] mlxsw: spectrum_router: Add a helper to get subport number from a RIF
    https://git.kernel.org/netdev/net-next/c/27851dfaa3d6
  - [net-next,14/14] mlxsw: spectrum_router: Call RIF setup before obtaining FID
    https://git.kernel.org/netdev/net-next/c/f7ebb4023765

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html