Skip to contentSkip to conversation

net: dsa: netc: add PTP support for NETC switch

PATCH New [email protected] 1 participant 10 messages Nov 19

Subsystems (9)
lore
Share & tools
lore
#0 lore raw

v8 cover net: dsa: netc: add PTP support for NETC switch

From: Wei Fang <[email protected]>

This patch set adds PTP timestamping support to the NETC switch, so that the switch ports can be used for IEEE 1588 PTP synchronization.

The NETC switch has no time registers of its own and instead shares the PTP time of the NETC Timer, which is a separate PCIe function driven by its own driver. The first 3 patches prepare the Timer driver for this
role: tidy up its 64-bit register access, drop an ineffective reset, and export netc_timer_get_current_time() so the switch driver can read the current PTP time across the driver boundary in a race-free way.

The switch redirects PTP frames to the CPU through the ingress port filter table (IPFT). A few patches harden and simplify the existing IPFT and host flood handling first, then enable the ingress port filtering lookup by default so PTP filter rules take effect without extra software tracking.

On top of that groundwork, the switch driver gains RX timestamping and both flavours of TX timestamping. Timestamps are carried between the hardware and the driver through dedicated switch tags: the ingress and transmit timestamps ride in To_Host tags, and per-frame requests ride in To_Port tags. Two-step TX matches a queued skb clone against the timestamp response echoed back by the hardware. One-step TX defers the Sync frame to a per-port work, where the driver reads a fresh PTP time and writes it into the Sync frame's timestamp field, then programs the single-step register and transmits the frame so the hardware can add the residence time to the correction field on the wire.

Together these changes let standard user-space PTP stacks drive hardware timestamping on the NETC switch ports.

---
v8 changes:
1. Rebase patch 4 due to the code base has been changed
2. Check the return value of pcie_flr() in netc_switch_pci_init()
3. Remove retry in netc_port_set_host_flood()
4. Add IPFT_FAF_L4_CODE mask to frm_attr_flags_mask
v7 link: https://lore.kernel.org/imx/[email protected]/
v6 link: https://lore.kernel.org/imx/[email protected]/
v5 link: https://lore.kernel.org/imx/[email protected]/
v4 link: https://lore.kernel.org/imx/[email protected]/
v3 link: https://lore.kernel.org/imx/[email protected]/
v2 link: https://lore.kernel.org/imx/[email protected]/
v1 link: https://lore.kernel.org/imx/[email protected]/
---

Wei Fang (9):
ptp: netc: use ioread64_lo_hi/iowrite64_lo_hi for 64-bit register
access
ptp: netc: remove unnecessary pcie_flr() call in probe
ptp: netc: export netc_timer_get_current_time() for cross-driver use
net: dsa: netc: tie setup resources and reset to DSA setup/teardown
net: dsa: netc: use entry ID instead of pointer to track host flood
rule
net: dsa: netc: check return value of ntmp_ipft_delete_entry()
net: dsa: netc: enable ingress port filtering lookup by default
net: dsa: netc: add PTP two-step timestamping support
net: dsa: netc: add PTP one-step timestamping support

 drivers/net/dsa/netc/Kconfig          |   2 +
 drivers/net/dsa/netc/Makefile         |   3 +-
 drivers/net/dsa/netc/netc_main.c      | 398 +++++++++---
 drivers/net/dsa/netc/netc_platform.c  |   1 +
 drivers/net/dsa/netc/netc_ptp.c       | 890 ++++++++++++++++++++++++++
 drivers/net/dsa/netc/netc_switch.h    |  67 +-
 drivers/net/dsa/netc/netc_switch_hw.h |   5 +
 drivers/ptp/ptp_netc.c                | 136 ++--
 include/linux/dsa/tag_netc.h          |  38 ++
 include/linux/fsl/netc_global.h       |  10 +
 net/dsa/tag_netc.c                    | 228 ++++++-
 11 files changed, 1610 insertions(+), 168 deletions(-)
 create mode 100644 drivers/net/dsa/netc/netc_ptp.c
Signature · 1 line
2.34.1
#1 lore raw

PATCH v8 1/9 ptp: netc: use ioread64_lo_hi/iowrite64_lo_hi for 64-bit register access

From: Wei Fang <[email protected]>

Replace the open-coded 64-bit register read/write sequences with ioread64_lo_hi() and iowrite64_lo_hi() helpers. Introduce two new macros netc_timer_rd64() and netc_timer_wr64() that wrap these helpers and use them throughout the driver. This reduces boilerplate and makes the intent of each operation clearer.

The high-half register defines (NETC_TMR_*_H) are kept to document the register map; they are not used directly since netc_timer_rd/wr64() address the 64-bit register pair via the low-half offset, relying on the hardware layout where H is always at L + 4.

---
 drivers/ptp/ptp_netc.c | 72 ++++++++++++------------------------------
 1 file changed, 21 insertions(+), 51 deletions(-)
drivers/ptp/ptp_netc.c +21 -51
index 1a327a7561e39..25f059efc831f 100644
@@ -127,6 +127,17 @@ struct netc_timer {
 
 #define netc_timer_rd(p, o)		netc_read((p)->base + (o))
 #define netc_timer_wr(p, o, v)		netc_write((p)->base + (o), v)
+
+/* The 64-bit timer registers consist of a low (L) and high (H) register pair.
+ * Hardware requires a strict access order: for writes, TMR_xxx_L must be
+ * written first, which latches the value into a shadow register; the write
+ * to TMR_xxx_H then atomically transfers both shadow registers into the live
+ * counter. For reads, TMR_xxx_L must be read first to capture a coherent
+ * snapshot. iowrite64_lo_hi() and ioread64_lo_hi() enforce this L-before-H
+ * ordering.
+ */
+#define netc_timer_rd64(p, o)		ioread64_lo_hi((p)->base + (o))
+#define netc_timer_wr64(p, o, v)	iowrite64_lo_hi(v, (p)->base + (o))
 #define ptp_to_netc_timer(ptp)		container_of((ptp), struct netc_timer, caps)
 
 static const char *const timer_clk_src[] = {
@@ -136,66 +147,28 @@ static const char *const timer_clk_src[] = {
 
 static void netc_timer_cnt_write(struct netc_timer *priv, u64 ns)
 {
-	u32 tmr_cnt_h = upper_32_bits(ns);
-	u32 tmr_cnt_l = lower_32_bits(ns);
-
-	/* Writes to the TMR_CNT_L register copies the written value
-	 * into the shadow TMR_CNT_L register. Writes to the TMR_CNT_H
-	 * register copies the values written into the shadow TMR_CNT_H
-	 * register. Contents of the shadow registers are copied into
-	 * the TMR_CNT_L and TMR_CNT_H registers following a write into
-	 * the TMR_CNT_H register. So the user must writes to TMR_CNT_L
-	 * register first. Other H/L registers should have the same
-	 * behavior.
-	 */
-	netc_timer_wr(priv, NETC_TMR_CNT_L, tmr_cnt_l);
-	netc_timer_wr(priv, NETC_TMR_CNT_H, tmr_cnt_h);
+	netc_timer_wr64(priv, NETC_TMR_CNT_L, ns);
 }
 
 static u64 netc_timer_offset_read(struct netc_timer *priv)
 {
-	u32 tmr_off_l, tmr_off_h;
-	u64 offset;
-
-	tmr_off_l = netc_timer_rd(priv, NETC_TMR_OFF_L);
-	tmr_off_h = netc_timer_rd(priv, NETC_TMR_OFF_H);
-	offset = (((u64)tmr_off_h) << 32) | tmr_off_l;
-
-	return offset;
+	return netc_timer_rd64(priv, NETC_TMR_OFF_L);
 }
 
 static void netc_timer_offset_write(struct netc_timer *priv, u64 offset)
 {
-	u32 tmr_off_h = upper_32_bits(offset);
-	u32 tmr_off_l = lower_32_bits(offset);
-
-	netc_timer_wr(priv, NETC_TMR_OFF_L, tmr_off_l);
-	netc_timer_wr(priv, NETC_TMR_OFF_H, tmr_off_h);
+	netc_timer_wr64(priv, NETC_TMR_OFF_L, offset);
 }
 
 static u64 netc_timer_cur_time_read(struct netc_timer *priv)
 {
-	u32 time_h, time_l;
-	u64 ns;
-
-	/* The user should read NETC_TMR_CUR_TIME_L first to
-	 * get correct current time.
-	 */
-	time_l = netc_timer_rd(priv, NETC_TMR_CUR_TIME_L);
-	time_h = netc_timer_rd(priv, NETC_TMR_CUR_TIME_H);
-	ns = (u64)time_h << 32 | time_l;
-
-	return ns;
+	return netc_timer_rd64(priv, NETC_TMR_CUR_TIME_L);
 }
 
 static void netc_timer_alarm_write(struct netc_timer *priv,
 				   u64 alarm, int index)
 {
-	u32 alarm_h = upper_32_bits(alarm);
-	u32 alarm_l = lower_32_bits(alarm);
-
-	netc_timer_wr(priv, NETC_TMR_ALARM_L(index), alarm_l);
-	netc_timer_wr(priv, NETC_TMR_ALARM_H(index), alarm_h);
+	netc_timer_wr64(priv, NETC_TMR_ALARM_L(index), alarm);
 }
 
 static u32 netc_timer_get_integral_period(struct netc_timer *priv)
@@ -500,22 +473,19 @@ static void netc_timer_handle_etts_event(struct netc_timer *priv, int index,
 					 bool update_event)
 {
 	struct ptp_clock_event event;
-	u32 etts_l = 0, etts_h = 0;
+	u64 etts = 0;
 
-	while (netc_timer_rd(priv, NETC_TMR_STAT) & TMR_STAT_ETS_VLD(index)) {
-		etts_l = netc_timer_rd(priv, NETC_TMR_ETTS_L(index));
-		etts_h = netc_timer_rd(priv, NETC_TMR_ETTS_H(index));
-	}
+	while (netc_timer_rd(priv, NETC_TMR_STAT) & TMR_STAT_ETS_VLD(index))
+		etts = netc_timer_rd64(priv, NETC_TMR_ETTS_L(index));
 
 	/* Invalid time stamp */
-	if (!etts_l && !etts_h)
+	if (!etts)
 		return;
 
 	if (update_event) {
 		event.type = PTP_CLOCK_EXTTS;
 		event.index = index;
-		event.timestamp = (u64)etts_h << 32;
-		event.timestamp |= etts_l;
+		event.timestamp = etts;
 		ptp_clock_event(priv->clock, &event);
 	}
 }
Signature · 1 line
2.34.1
#2 lore raw

PATCH v8 2/9 ptp: netc: remove unnecessary pcie_flr() call in probe

In reply to [email protected]

From: Wei Fang <[email protected]>

According to the NETC reference manual, function level reset is not applicable to the timer as a supporting function. Remove the pcie_flr() call from netc_timer_pci_probe() as it has no effect.

---
 drivers/ptp/ptp_netc.c | 1 -
 1 file changed, 1 deletion(-)
drivers/ptp/ptp_netc.c +0 -1
index 25f059efc831f..87cfea180e859 100644
@@ -777,7 +777,6 @@ static int netc_timer_pci_probe(struct pci_dev *pdev)
 	if (!priv)
 		return -ENOMEM;
 
-	pcie_flr(pdev);
 	err = pci_enable_device_mem(pdev);
 	if (err)
 		return dev_err_probe(dev, err, "Failed to enable device\n");
Signature · 1 line
2.34.1
#3 lore raw

PATCH v8 3/9 ptp: netc: export netc_timer_get_current_time() for cross-driver use

In reply to [email protected]

From: Wei Fang <[email protected]>

The NETC Switch does not have its own time registers and must obtain the current PTP time from the NETC Timer bound to it. As they are separate PCIe functions with independent drivers, add netc_timer_get_current_time() to the Timer driver and export it so the Switch driver can call it. There is only one switch in the NETC IP, and it is the sole caller of this function. ENETC driver will not use this function as it has its own time registers.

The Timer may be unbound while the Switch is calling this function, which would free priv and unmap the MMIO region under an in-flight read. To keep priv valid for the whole read, track each probed Timer in a module-private list keyed by its pci_dev, and protect the lookup and the register read with a global spinlock. A list hit proves the pci_dev is bound to this driver, so priv is valid. remove() deletes the entry under the same lock before tearing the device down, so no read can observe a stale priv.

---
 drivers/ptp/ptp_netc.c          | 63 ++++++++++++++++++++++++++++++++-
 include/linux/fsl/netc_global.h | 10 ++++++
 2 files changed, 72 insertions(+), 1 deletion(-)
drivers/ptp/ptp_netc.c +62 -1
index 87cfea180e859..370f7ad43a0a8 100644
@@ -1,7 +1,7 @@
 // SPDX-License-Identifier: (GPL-2.0+ OR BSD-3-Clause)
 /*
  * NXP NETC V4 Timer driver
- * Copyright 2025 NXP
+ * Copyright 2025-2026 NXP
  */
 
 #include <linux/bitfield.h>
@@ -123,8 +123,12 @@ struct netc_timer {
 	u8 fs_alarm_num;
 	u8 fs_alarm_bitmap;
 	struct netc_pp pp[NETC_TMR_FIPER_NUM]; /* periodic pulse */
+	struct list_head node;
 };
 
+static LIST_HEAD(netc_timer_list);
+static DEFINE_SPINLOCK(netc_timer_list_lock);
+
 #define netc_timer_rd(p, o)		netc_read((p)->base + (o))
 #define netc_timer_wr(p, o, v)		netc_write((p)->base + (o), v)
 
@@ -985,6 +989,10 @@ static int netc_timer_probe(struct pci_dev *pdev,
 
 	enable_irq(priv->irq);
 
+	spin_lock_bh(&netc_timer_list_lock);
+	list_add(&priv->node, &netc_timer_list);
+	spin_unlock_bh(&netc_timer_list_lock);
+
 	return 0;
 
 free_msix_irq:
@@ -999,6 +1007,10 @@ static void netc_timer_remove(struct pci_dev *pdev)
 {
 	struct netc_timer *priv = pci_get_drvdata(pdev);
 
+	spin_lock_bh(&netc_timer_list_lock);
+	list_del(&priv->node);
+	spin_unlock_bh(&netc_timer_list_lock);
+
 	disable_irq(priv->irq);
 	ptp_clock_unregister(priv->clock);
 	netc_timer_wr(priv, NETC_TMR_TEMASK, 0);
@@ -1021,5 +1033,54 @@ static struct pci_driver netc_timer_driver = {
 };
 module_pci_driver(netc_timer_driver);
 
+/**
+ * netc_timer_get_current_time - read the current PTP time from the NETC Timer
+ * @pdev: PCI device of the NETC Timer
+ * @ns: Output, the current PTP clock time in nanoseconds
+ *
+ * Read TMR_CUR_TIME from the NETC Timer bound to @pdev. The lookup and read
+ * run under netc_timer_list_lock, so the Timer cannot be unbound and its priv
+ * freed during the read. The caller must hold a reference on @pdev.
+ *
+ * Context: Process or softirq context. Must not be called from hardirq.
+ *
+ * Return: 0 on success, -ENODEV if the Timer is not present (not yet probed
+ *         or already removed).
+ */
+int netc_timer_get_current_time(struct pci_dev *pdev, u64 *ns)
+{
+	struct netc_timer *priv = NULL;
+	struct netc_timer *tmp;
+	unsigned long flags;
+	int err = 0;
+
+	/* Serialize against driver unbind, so holding it here ensures that
+	 * priv remains valid for the entire duration of the register read.
+	 */
+	spin_lock_bh(&netc_timer_list_lock);
+
+	list_for_each_entry(tmp, &netc_timer_list, node) {
+		if (tmp->pdev == pdev) {
+			priv = tmp;
+			break;
+		}
+	}
+
+	if (!priv) {
+		err = -ENODEV;
+		goto netc_timer_list_unlock;
+	}
+
+	spin_lock_irqsave(&priv->lock, flags);
+	*ns = netc_timer_cur_time_read(priv);
+	spin_unlock_irqrestore(&priv->lock, flags);
+
+netc_timer_list_unlock:
+	spin_unlock_bh(&netc_timer_list_lock);
+
+	return err;
+}
+EXPORT_SYMBOL_GPL(netc_timer_get_current_time);
+
 MODULE_DESCRIPTION("NXP NETC Timer PTP Driver");
 MODULE_LICENSE("Dual BSD/GPL");
include/linux/fsl/netc_global.h +10 -0
index 5b8ff528d3699..d4a26c17f99a4 100644
@@ -6,6 +6,7 @@
 
 #include <linux/io.h>
 #include <linux/io-64-nonatomic-lo-hi.h>
+#include <linux/pci.h>
 
 static inline u32 netc_read(void __iomem *reg)
 {
@@ -22,4 +23,13 @@ static inline u64 netc_read64(void __iomem *reg)
 	return ioread64(reg);
 }
 
+#if IS_REACHABLE(CONFIG_PTP_NETC_V4_TIMER)
+int netc_timer_get_current_time(struct pci_dev *pdev, u64 *ns);
+#else
+static inline int netc_timer_get_current_time(struct pci_dev *pdev, u64 *ns)
+{
+	return -ENODEV;
+}
+#endif
+
 #endif
Signature · 1 line
2.34.1
#4 lore raw

PATCH v8 4/9 net: dsa: netc: tie setup resources and reset to DSA setup/teardown

In reply to [email protected]

From: Wei Fang <[email protected]>

The NETC switch reached a known hardware state via a one-time PCIe FLR in probe(), and used devm_ allocations in netc_setup() for its ports, EMDIO buses, reference clocks, command BD rings and buffer pool table. Both are tied to the PCI device lifetime, not to the DSA netc_setup()/netc_teardown() lifetime.

In a multi-switch DSA tree, unbinding any switch tears down the whole tree, so netc_teardown() runs on every NETC switch in it. When a switch is bound again and completes the tree, dsa_register_switch() sets up the whole tree, so netc_setup() also runs on the switches that stayed bound. Thus netc_teardown() and netc_setup() can run again without a PCI re-probe of the NETC switch. Two problems then appear:

- No fresh FLR occurs, so the hardware tables keep their stale entries while netc_setup() builds empty software state. Hardware and software drift out of sync.

- The devm_ resources are not released by netc_teardown(), so each re-setup leaks memory and re-registers the same EMDIO bus, which fails.

Add netc_switch_reset() and call it at the start of netc_setup() so every setup begins from a clean hardware state. The probe path already performs a FLR in netc_switch_pci_init(), so the first setup will skip the reset because the priv->configured flag is not set. Any later setup is driven by a sibling switch completing the tree, under dsa2_mutex, and resets the switch with pcie_flr(). Save and restore the config space around the FLR, and hold pci_cfg_access_lock() so that user space config access and other PCI resets cannot run concurrently. Do not take the device lock, because an unbinding NETC switch holds it while waiting for dsa2_mutex in dsa_unregister_switch().

Convert the setup-path devm_ allocations to explicit alloc and free, and release them in netc_teardown() and on the error paths, so their lifetime matches the DSA setup and teardown.

The NETC switch does not support multi-switch yet, so there is no functional problem today. This only reworks the driver so that multi-switch can be added later without hitting these issues.

---
 drivers/net/dsa/netc/netc_main.c   | 186 ++++++++++++++++++++++++-----
 drivers/net/dsa/netc/netc_switch.h |   1 +
 2 files changed, 155 insertions(+), 32 deletions(-)
drivers/net/dsa/netc/netc_main.c +154 -32
index 1225ff8153d44..f724f95add36b 100644
@@ -135,7 +135,7 @@ static int netc_port_get_info_from_dt(struct netc_port *np,
 				      struct device *dev)
 {
 	if (of_find_property(node, "clock-names", NULL)) {
-		np->ref_clk = devm_get_clk_from_child(dev, node, "ref");
+		np->ref_clk = of_clk_get_by_name(node, "ref");
 		if (IS_ERR(np->ref_clk)) {
 			dev_err(dev, "Port %d cannot get reference clock\n",
 				np->dp->index);
@@ -146,6 +146,24 @@ static int netc_port_get_info_from_dt(struct netc_port *np,
 	return 0;
 }
 
+static void netc_port_free_emdio_bus(struct netc_port *np)
+{
+	struct device *dev = np->switch_priv->dev;
+	struct enetc_mdio_priv *mdio_priv;
+	struct enetc_hw *hw;
+
+	if (!np->emdio)
+		return;
+
+	mdio_priv = np->emdio->priv;
+	hw = mdio_priv->hw;
+
+	mdiobus_unregister(np->emdio);
+	mdiobus_free(np->emdio);
+	devm_kfree(dev, hw);
+	np->emdio = NULL;
+}
+
 static int netc_port_create_emdio_bus(struct netc_port *np,
 				      struct device_node *node)
 {
@@ -161,9 +179,11 @@ static int netc_port_create_emdio_bus(struct netc_port *np,
 		return dev_err_probe(dev, PTR_ERR(hw),
 				     "Failed to allocate enetc_hw\n");
 
-	bus = devm_mdiobus_alloc_size(dev, sizeof(*mdio_priv));
-	if (!bus)
-		return -ENOMEM;
+	bus = mdiobus_alloc_size(sizeof(*mdio_priv));
+	if (!bus) {
+		err = -ENOMEM;
+		goto free_enetc_hw;
+	}
 
 	bus->name = "NXP NETC switch external MDIO Bus";
 	bus->read = enetc_mdio_read_c22;
@@ -177,14 +197,22 @@ static int netc_port_create_emdio_bus(struct netc_port *np,
 	snprintf(bus->id, MII_BUS_ID_SIZE, "%s-p%d-emdio",
 		 dev_name(dev), np->dp->index);
 
-	err = devm_of_mdiobus_register(dev, bus, node);
-	if (err)
-		return dev_err_probe(dev, err,
-				     "Cannot register EMDIO bus\n");
+	err = of_mdiobus_register(bus, node);
+	if (err) {
+		dev_err_probe(dev, err, "Cannot register EMDIO bus\n");
+		goto free_mdiobus;
+	}
 
 	np->emdio = bus;
 
 	return 0;
+
+free_mdiobus:
+	mdiobus_free(bus);
+free_enetc_hw:
+	devm_kfree(dev, hw);
+
+	return err;
 }
 
 static int netc_port_create_mdio_bus(struct netc_port *np,
@@ -238,6 +266,33 @@ static void netc_get_switch_capabilities(struct netc_switch *priv)
 	priv->num_bp = FIELD_GET(BPCAPR_NUM_BP, val);
 }
 
+static void netc_free_all_ports(struct netc_switch *priv)
+{
+	if (!priv->ports)
+		return;
+
+	for (int i = 0; i < priv->info->num_ports; i++) {
+		struct netc_port *np = priv->ports[i];
+
+		if (!np)
+			continue;
+
+		netc_port_free_emdio_bus(np);
+
+		if (!IS_ERR_OR_NULL(np->ref_clk)) {
+			clk_put(np->ref_clk);
+			np->ref_clk = NULL;
+		}
+
+		kfree(np);
+		priv->ports[i] = NULL;
+	}
+
+	kfree(priv->ports);
+	priv->ports = NULL;
+	priv->port_bitmap = 0;
+}
+
 static int netc_init_all_ports(struct netc_switch *priv)
 {
 	struct device *dev = priv->dev;
@@ -246,8 +301,7 @@ static int netc_init_all_ports(struct netc_switch *priv)
 	int ett_offset = 0;
 	int err;
 
-	priv->ports = devm_kzalloc_objs(dev, struct netc_port *,
-					priv->info->num_ports);
+	priv->ports = kzalloc_objs(*priv->ports, priv->info->num_ports);
 	if (!priv->ports)
 		return -ENOMEM;
 
@@ -259,9 +313,11 @@ static int netc_init_all_ports(struct netc_switch *priv)
 	 * difficult for us to cover.
 	 */
 	for (int i = 0; i < priv->info->num_ports; i++) {
-		np = devm_kzalloc_obj(dev, *np);
-		if (!np)
-			return -ENOMEM;
+		np = kzalloc_obj(*np);
+		if (!np) {
+			err = -ENOMEM;
+			goto free_all_ports;
+		}
 
 		np->switch_priv = priv;
 		np->iobase = priv->regs.port + PORT_IOBASE(i);
@@ -277,18 +333,23 @@ static int netc_init_all_ports(struct netc_switch *priv)
 
 		err = netc_port_get_info_from_dt(np, dp->dn, dev);
 		if (err)
-			return err;
+			goto free_all_ports;
 
 		if (dsa_port_is_user(dp)) {
 			err = netc_port_create_mdio_bus(np, dp->dn);
 			if (err) {
 				dev_err(dev, "Failed to create MDIO bus\n");
-				return err;
+				goto free_all_ports;
 			}
 		}
 	}
 
 	return 0;
+
+free_all_ports:
+	netc_free_all_ports(priv);
+
+	return err;
 }
 
 static void netc_init_ntmp_tbl_versions(struct netc_switch *priv)
@@ -307,8 +368,7 @@ static int netc_init_all_cbdrs(struct netc_switch *priv)
 
 	ntmp->cbdr_num = NETC_CBDR_NUM;
 	ntmp->dev = priv->dev;
-	ntmp->ring = devm_kzalloc_objs(ntmp->dev, struct netc_cbdr,
-				       ntmp->cbdr_num);
+	ntmp->ring = kzalloc_objs(*ntmp->ring, ntmp->cbdr_num);
 	if (!ntmp->ring)
 		return -ENOMEM;
 
@@ -334,6 +394,9 @@ static int netc_init_all_cbdrs(struct netc_switch *priv)
 	for (i--; i >= 0; i--)
 		ntmp_free_cbdr(&ntmp->ring[i]);
 
+	kfree(ntmp->ring);
+	ntmp->ring = NULL;
+
 	return err;
 }
 
@@ -343,6 +406,9 @@ static void netc_remove_all_cbdrs(struct netc_switch *priv)
 
 	for (int i = 0; i < NETC_CBDR_NUM; i++)
 		ntmp_free_cbdr(&ntmp->ring[i]);
+
+	kfree(ntmp->ring);
+	ntmp->ring = NULL;
 }
 
 static u32 netc_num_available_ports(struct netc_switch *priv)
@@ -840,30 +906,74 @@ static void netc_ipv_to_buffer_pool_mapping(struct netc_switch *priv)
 	}
 }
 
+static void netc_switch_free_bpt_list(struct netc_switch *priv)
+{
+	kfree(priv->bpt_list);
+	priv->bpt_list = NULL;
+}
+
 static int netc_switch_bpt_default_config(struct netc_switch *priv)
 {
+	int err;
+
 	if (priv->num_bp < priv->info->num_ports)
 		return -EINVAL;
 
-	priv->bpt_list = devm_kzalloc_objs(priv->dev, struct bpt_cfge_data,
-					   priv->num_bp);
+	priv->bpt_list = kzalloc_objs(*priv->bpt_list, priv->num_bp);
 	if (!priv->bpt_list)
 		return -ENOMEM;
 
 	/* Initialize the maximum threshold of each buffer pool entry */
 	for (int i = 0; i < priv->num_bp; i++) {
 		struct bpt_cfge_data *cfge = &priv->bpt_list[i];
-		int err;
 
 		cfge->max_thresh = cpu_to_le16(NETC_BP_THRESH);
 		err = ntmp_bpt_update_entry(&priv->ntmp, i, cfge);
 		if (err)
-			return err;
+			goto free_bpt_list;
 	}
 
 	netc_ipv_to_buffer_pool_mapping(priv);
 
 	return 0;
+
+free_bpt_list:
+	netc_switch_free_bpt_list(priv);
+
+	return err;
+}
+
+static int netc_switch_reset(struct netc_switch *priv)
+{
+	struct pci_dev *pdev = priv->pdev;
+	int err;
+
+	if (!priv->configured) {
+		/* The hardware will be configured after netc_switch_reset()
+		 * is called, so set priv->configured to true.
+		 */
+		priv->configured = true;
+		return 0;
+	}
+
+	/* Block user space config access and other PCI resets, which take
+	 * pci_cfg_access_lock() via pci_dev_lock(), during the FLR.
+	 */
+	pci_cfg_access_lock(pdev);
+	err = pci_save_state(pdev);
+	if (err)
+		goto unlock_cfg_access;
+
+	err = pcie_flr(pdev);
+	if (err)
+		dev_err(priv->dev, "FLR failed: %pe\n", ERR_PTR(err));
+
+	pci_restore_state(pdev);
+
+unlock_cfg_access:
+	pci_cfg_access_unlock(pdev);
+
+	return err;
 }
 
 static int netc_setup(struct dsa_switch *ds)
@@ -872,6 +982,10 @@ static int netc_setup(struct dsa_switch *ds)
 	struct dsa_port *dp;
 	int err;
 
+	err = netc_switch_reset(priv);
+	if (err)
+		return err;
+
 	err = netc_init_switch_id(priv);
 	if (err)
 		return err;
@@ -884,7 +998,7 @@ static int netc_setup(struct dsa_switch *ds)
 
 	err = netc_init_ntmp_user(priv);
 	if (err)
-		return err;
+		goto free_all_ports;
 
 	INIT_HLIST_HEAD(&priv->fdb_list);
 	mutex_init(&priv->fdbt_lock);
@@ -907,22 +1021,25 @@ static int netc_setup(struct dsa_switch *ds)
 
 	err = netc_add_standalone_vlan_entry(priv);
 	if (err)
-		goto free_lock_and_ntmp_user;
+		goto free_bpt_list;
 
 	err = netc_add_standalone_fdb_bcast_entry(priv);
 	if (err)
-		goto free_lock_and_ntmp_user;
+		goto free_bpt_list;
 
 	return 0;
 
+free_bpt_list:
+	netc_switch_free_bpt_list(priv);
 free_lock_and_ntmp_user:
-	/* No need to clear the hardware state, netc_setup() is only called
-	 * when the driver is bound, and FLR will be performed to reset the
-	 * hardware state.
+	/* No need to clear the hardware state. netc_setup() resets the
+	 * hardware via FLR, so the next setup starts from a clean state.
 	 */
 	mutex_destroy(&priv->fdbt_lock);
 	mutex_destroy(&priv->vft_lock);
 	netc_free_ntmp_user(priv);
+free_all_ports:
+	netc_free_all_ports(priv);
 
 	return err;
 }
@@ -942,10 +1059,10 @@ static void netc_free_host_flood_rules(struct netc_switch *priv)
 	dsa_switch_for_each_user_port(dp, priv->ds) {
 		struct netc_port *np = priv->ports[dp->index];
 
-		/* No need to clear the hardware IPFT entry. Because PCIe
-		 * FLR will be performed when the switch is re-registered,
-		 * it will reset hardware state. So only need to free the
-		 * memory to avoid memory leak.
+		/* No need to clear the hardware IPFT entry. netc_setup()
+		 * resets the hardware via FLR, so the next setup starts
+		 * from a clean state. Only free the memory here to avoid
+		 * a leak.
 		 */
 		kfree(np->host_flood);
 		np->host_flood = NULL;
@@ -960,6 +1077,8 @@ static void netc_teardown(struct dsa_switch *ds)
 	netc_destroy_all_lists(priv);
 	netc_free_host_flood_rules(priv);
 	netc_free_ntmp_user(priv);
+	netc_switch_free_bpt_list(priv);
+	netc_free_all_ports(priv);
 }
 
 static bool netc_port_is_emdio_consumer(struct device_node *node)
@@ -1042,7 +1161,10 @@ static int netc_switch_pci_init(struct pci_dev *pdev)
 	void __iomem *base;
 	int err;
 
-	pcie_flr(pdev);
+	err = pcie_flr(pdev);
+	if (err)
+		return dev_err_probe(dev, err, "FLR failed\n");
+
 	err = pcim_enable_device(pdev);
 	if (err)
 		return dev_err_probe(dev, err, "Failed to enable device\n");
drivers/net/dsa/netc/netc_switch.h +1 -0
index 305f2a92e2f9a..de2af536c5945 100644
@@ -118,6 +118,7 @@ struct netc_switch {
 	struct device *dev;
 	struct dsa_switch *ds;
 	u16 revision;
+	bool configured;
 
 	const struct netc_switch_info *info;
 	struct netc_switch_regs regs;
Signature · 1 line
2.34.1
#5 lore raw

PATCH v8 5/9 net: dsa: netc: use entry ID instead of pointer to track host flood rule

In reply to [email protected]

From: Wei Fang <[email protected]>

Replace the struct ipft_entry_data pointer in struct netc_port with a plain u32 entry ID (ipft_hf_eid), using NTMP_NULL_ENTRY_ID as the sentinel value. The ipft_entry_data allocation is now freed immediately inside netc_port_add_host_flood_rule() after the hardware entry is committed, so no heap memory survives beyond that function. As a result, netc_free_host_flood_rules() is no longer needed and can be removed.

---
 drivers/net/dsa/netc/netc_main.c   | 62 +++++++++++-------------------
 drivers/net/dsa/netc/netc_switch.h |  2 +-
 2 files changed, 24 insertions(+), 40 deletions(-)
drivers/net/dsa/netc/netc_main.c +23 -39
index f724f95add36b..7d0893bc0f163 100644
@@ -321,6 +321,11 @@ static int netc_init_all_ports(struct netc_switch *priv)
 
 		np->switch_priv = priv;
 		np->iobase = priv->regs.port + PORT_IOBASE(i);
+		/* The ipft_hf_eid is initialized to an invalid entry
+		 * ID because the host flood rule (IPFT entry) has not
+		 * been created.
+		 */
+		np->ipft_hf_eid = NTMP_NULL_ENTRY_ID;
 		netc_port_get_capability(np);
 		priv->ports[i] = np;
 	}
@@ -1052,30 +1057,12 @@ static void netc_destroy_all_lists(struct netc_switch *priv)
 	mutex_destroy(&priv->vft_lock);
 }
 
-static void netc_free_host_flood_rules(struct netc_switch *priv)
-{
-	struct dsa_port *dp;
-
-	dsa_switch_for_each_user_port(dp, priv->ds) {
-		struct netc_port *np = priv->ports[dp->index];
-
-		/* No need to clear the hardware IPFT entry. netc_setup()
-		 * resets the hardware via FLR, so the next setup starts
-		 * from a clean state. Only free the memory here to avoid
-		 * a leak.
-		 */
-		kfree(np->host_flood);
-		np->host_flood = NULL;
-	}
-}
-
 static void netc_teardown(struct dsa_switch *ds)
 {
 	struct netc_switch *priv = ds->priv;
 
 	disable_delayed_work_sync(&priv->fdbt_ageing_work);
 	netc_destroy_all_lists(priv);
-	netc_free_host_flood_rules(priv);
 	netc_free_ntmp_user(priv);
 	netc_switch_free_bpt_list(priv);
 	netc_free_all_ports(priv);
@@ -1878,37 +1865,36 @@ static int netc_port_add_host_flood_rule(struct netc_port *np,
 	cfge->cfg = cpu_to_le32(cfg);
 
 	err = ntmp_ipft_add_entry(&priv->ntmp, host_flood);
-	if (err) {
-		kfree(host_flood);
-		return err;
-	}
+	if (err)
+		goto free_host_flood;
 
 	np->uc = uc;
 	np->mc = mc;
-	np->host_flood = host_flood;
+	np->ipft_hf_eid = host_flood->entry_id;
 	/* Enable ingress port filter table lookup */
 	netc_port_wr(np, NETC_PIPFCR, PIPFCR_EN);
 
-	return 0;
+free_host_flood:
+	kfree(host_flood);
+
+	return err;
 }
 
-static void netc_port_remove_host_flood(struct netc_port *np,
-					struct ipft_entry_data *host_flood)
+static void netc_port_remove_host_flood(struct netc_port *np, u32 entry_id)
 {
 	struct netc_switch *priv = np->switch_priv;
 	bool disable_host_flood = false;
 
-	if (!host_flood)
+	if (entry_id == NTMP_NULL_ENTRY_ID)
 		return;
 
-	if (np->host_flood == host_flood)
+	if (np->ipft_hf_eid == entry_id)
 		disable_host_flood = true;
 
-	ntmp_ipft_delete_entry(&priv->ntmp, host_flood->entry_id);
-	kfree(host_flood);
+	ntmp_ipft_delete_entry(&priv->ntmp, entry_id);
 
 	if (disable_host_flood) {
-		np->host_flood = NULL;
+		np->ipft_hf_eid = NTMP_NULL_ENTRY_ID;
 		np->uc = false;
 		np->mc = false;
 		netc_port_wr(np, NETC_PIPFCR, 0);
@@ -1919,7 +1905,7 @@ static void netc_port_set_host_flood(struct dsa_switch *ds, int port,
 				     bool uc, bool mc)
 {
 	struct netc_port *np = NETC_PORT(ds, port);
-	struct ipft_entry_data *old_host_flood;
+	u32 old_entry_id;
 
 	/* Do not add host flood rule to ingress port filter table when
 	 * the port has joined a bridge. Otherwise, the ingress frames
@@ -1927,7 +1913,7 @@ static void netc_port_set_host_flood(struct dsa_switch *ds, int port,
 	 * will be redirected directly to the CPU port.
 	 */
 	if (dsa_port_bridge_dev_get(np->dp)) {
-		netc_port_remove_host_flood(np, np->host_flood);
+		netc_port_remove_host_flood(np, np->ipft_hf_eid);
 
 		return;
 	}
@@ -1937,20 +1923,18 @@ static void netc_port_set_host_flood(struct dsa_switch *ds, int port,
 
 	/* IPFT does not support in-place updates to the KEYE element,
 	 * we need to add a new entry and then delete the old one. So
-	 * save the old entry first.
+	 * save the old entry ID first.
 	 */
-	old_host_flood = np->host_flood;
-	np->host_flood = NULL;
+	old_entry_id = np->ipft_hf_eid;
 
 	if (netc_port_add_host_flood_rule(np, uc, mc)) {
-		np->host_flood = old_host_flood;
 		dev_err(ds->dev, "Failed to add host flood rule on port %d\n",
 			port);
 		return;
 	}
 
 	/* Remove the old host flood entry */
-	netc_port_remove_host_flood(np, old_host_flood);
+	netc_port_remove_host_flood(np, old_entry_id);
 }
 
 static int netc_single_vlan_aware_bridge(struct dsa_switch *ds,
@@ -2139,7 +2123,7 @@ static int netc_port_bridge_join(struct dsa_switch *ds, int port,
 	netc_port_set_pvid(np, vlan_unaware_pvid);
 
 out:
-	netc_port_remove_host_flood(np, np->host_flood);
+	netc_port_remove_host_flood(np, np->ipft_hf_eid);
 
 	if (atomic_inc_return(&priv->br_cnt) == 1)
 		schedule_delayed_work(&priv->fdbt_ageing_work,
drivers/net/dsa/netc/netc_switch.h +1 -1
index de2af536c5945..b0e1eb4621531 100644
@@ -84,7 +84,7 @@ struct netc_port {
 	u16 uc:1;
 	u16 mc:1;
 	u16 pvid;
-	struct ipft_entry_data *host_flood;
+	u32 ipft_hf_eid; /* Must be initialized to NTMP_NULL_ENTRY_ID */
 };
 
 struct netc_switch_regs {
Signature · 1 line
2.34.1
#6 lore raw

PATCH v8 6/9 net: dsa: netc: check return value of ntmp_ipft_delete_entry()

In reply to [email protected]

From: Wei Fang <[email protected]>

The driver ignored the return value of ntmp_ipft_delete_entry(), so a failed deletion left the IPFT entry untracked and impossible to clean up later. IPFT is going to serve more features, such as trapping PTP packets and flow policing, so make the deletion path robust to prepare for these future uses.

To this end, propagate the error to the callers and update the driver state only after the entry has really been deleted, so that the software state always stays consistent with the hardware.

Once the deletion can fail, the order in which a rule is replaced also needs to change. Since IPFT does not support in-place updates to the KEYE element, replacing an existing rule requires deleting the old entry and then adding the new one. Keeping the previous add-then-delete order means that if the delete fails, a port would have to track two or more stale host flood rules, which needlessly complicates the driver. So switch to a delete-then-add order instead. This order is not
perfect: between the delete and the add there is a brief window in which packets that would have matched the rule find no match and are dropped. The window is very short and its impact is acceptable, so this is a deliberate trade-off in favour of simpler software.

Another issue is that if adding a new rule fails, host flooding will be disabled indefinitely. However, there is no good way to fully resolve this, because port_set_host_flood() returns void and the upper layer is not aware of the error. So if this corner case does occur, log an error to help users find out why host flooding is not working.

Note that, a delete failure has never been observed in practice and is highly unlikely to happen, so this patch is an improvement over the previous approach rather than a fix for an issue seen in the field.

---
 drivers/net/dsa/netc/netc_main.c | 83 ++++++++++++++++++++------------
 1 file changed, 52 insertions(+), 31 deletions(-)
drivers/net/dsa/netc/netc_main.c +52 -31
index 7d0893bc0f163..e1ec7aecab0f9 100644
@@ -1825,14 +1825,8 @@ static int netc_port_add_host_flood_rule(struct netc_port *np,
 	u32 cfg;
 	int err;
 
-	if (!uc && !mc) {
-		/* Disable ingress port filter table lookup */
-		netc_port_wr(np, NETC_PIPFCR, 0);
-		np->uc = false;
-		np->mc = false;
-
+	if (!uc && !mc)
 		return 0;
-	}
 
 	host_flood = kzalloc_obj(*host_flood);
 	if (!host_flood)
@@ -1880,32 +1874,36 @@ static int netc_port_add_host_flood_rule(struct netc_port *np,
 	return err;
 }
 
-static void netc_port_remove_host_flood(struct netc_port *np, u32 entry_id)
+static int netc_port_remove_host_flood(struct netc_port *np)
 {
 	struct netc_switch *priv = np->switch_priv;
-	bool disable_host_flood = false;
+	u32 entry_id = np->ipft_hf_eid;
+	int err;
 
 	if (entry_id == NTMP_NULL_ENTRY_ID)
-		return;
+		return 0;
 
-	if (np->ipft_hf_eid == entry_id)
-		disable_host_flood = true;
+	/* The command BD ring is always enabled here, with the ring
+	 * enabled, the hardware completes a command well within 1 ms,
+	 * so -ETIMEDOUT cannot happen here. Any error therefore means
+	 * that the entry has not been deleted, so keep tracking it.
+	 */
+	err = ntmp_ipft_delete_entry(&priv->ntmp, entry_id);
+	if (err)
+		return err;
 
-	ntmp_ipft_delete_entry(&priv->ntmp, entry_id);
+	np->ipft_hf_eid = NTMP_NULL_ENTRY_ID;
+	np->uc = false;
+	np->mc = false;
+	netc_port_wr(np, NETC_PIPFCR, 0);
 
-	if (disable_host_flood) {
-		np->ipft_hf_eid = NTMP_NULL_ENTRY_ID;
-		np->uc = false;
-		np->mc = false;
-		netc_port_wr(np, NETC_PIPFCR, 0);
-	}
+	return 0;
 }
 
 static void netc_port_set_host_flood(struct dsa_switch *ds, int port,
 				     bool uc, bool mc)
 {
 	struct netc_port *np = NETC_PORT(ds, port);
-	u32 old_entry_id;
 
 	/* Do not add host flood rule to ingress port filter table when
 	 * the port has joined a bridge. Otherwise, the ingress frames
@@ -1913,7 +1911,12 @@ static void netc_port_set_host_flood(struct dsa_switch *ds, int port,
 	 * will be redirected directly to the CPU port.
 	 */
 	if (dsa_port_bridge_dev_get(np->dp)) {
-		netc_port_remove_host_flood(np, np->ipft_hf_eid);
+		if (!netc_port_remove_host_flood(np))
+			return;
+
+		dev_err(ds->dev,
+			"Failed to delete host flood rule on bridge port %d\n",
+			port);
 
 		return;
 	}
@@ -1922,19 +1925,26 @@ static void netc_port_set_host_flood(struct dsa_switch *ds, int port,
 		return;
 
 	/* IPFT does not support in-place updates to the KEYE element,
-	 * we need to add a new entry and then delete the old one. So
-	 * save the old entry ID first.
+	 * we need to delete the old one and then add the new rule. If
+	 * the deletion fails, return immediately.
 	 */
-	old_entry_id = np->ipft_hf_eid;
-
-	if (netc_port_add_host_flood_rule(np, uc, mc)) {
-		dev_err(ds->dev, "Failed to add host flood rule on port %d\n",
+	if (netc_port_remove_host_flood(np)) {
+		dev_err(ds->dev,
+			"Failed to delete old host flood rule on port %d\n",
 			port);
+
 		return;
 	}
 
-	/* Remove the old host flood entry */
-	netc_port_remove_host_flood(np, old_entry_id);
+	/* Restoring the previous configuration is pointless because
+	 * .port_set_host_flood() returns void, so the upper layer cannot
+	 * detect the error and the RX flags have changed. Furthermore, we
+	 * cannot guarantee that we can successfully restore the previous
+	 * state.
+	 */
+	if (netc_port_add_host_flood_rule(np, uc, mc))
+		dev_err(ds->dev,
+			"Failed to add host flood rule on port %d\n", port);
 }
 
 static int netc_single_vlan_aware_bridge(struct dsa_switch *ds,
@@ -2099,6 +2109,8 @@ static int netc_port_bridge_join(struct dsa_switch *ds, int port,
 	struct netc_port *np = NETC_PORT(ds, port);
 	struct netc_switch *priv = ds->priv;
 	u16 vlan_unaware_pvid;
+	bool uc = np->uc;
+	bool mc = np->mc;
 	int err;
 
 	if (!bridge.num) {
@@ -2110,6 +2122,12 @@ static int netc_port_bridge_join(struct dsa_switch *ds, int port,
 	if (err)
 		return err;
 
+	err = netc_port_remove_host_flood(np);
+	if (err) {
+		NL_SET_ERR_MSG_MOD(extack, "Failed to delete host flood rule");
+		return err;
+	}
+
 	netc_port_set_mlo(np, MLO_NOT_OVERRIDE);
 
 	if (br_vlan_enabled(bridge.dev))
@@ -2123,8 +2141,6 @@ static int netc_port_bridge_join(struct dsa_switch *ds, int port,
 	netc_port_set_pvid(np, vlan_unaware_pvid);
 
 out:
-	netc_port_remove_host_flood(np, np->ipft_hf_eid);
-
 	if (atomic_inc_return(&priv->br_cnt) == 1)
 		schedule_delayed_work(&priv->fdbt_ageing_work,
 				      READ_ONCE(priv->fdbt_ageing_delay));
@@ -2134,6 +2150,11 @@ static int netc_port_bridge_join(struct dsa_switch *ds, int port,
 disable_mlo:
 	netc_port_set_mlo(np, MLO_DISABLE);
 
+	if (netc_port_add_host_flood_rule(np, uc, mc))
+		dev_err(ds->dev,
+			"Failed to restore host flood rule on port %d\n",
+			port);
+
 	return err;
 }
 
Signature · 1 line
2.34.1
#7 lore raw

PATCH v8 7/9 net: dsa: netc: enable ingress port filtering lookup by default

In reply to [email protected]

From: Wei Fang <[email protected]>

The ingress port filtering lookup function involves performing a lookup against the ingress port filter table (IPFT). If the frame matches an entry, subsequent frame processing functions will perform corresponding operations based on the parameters specified in that entry. If no entry matches the frame, the frame is allowed in, and passed to the next frame processing function.

In the future, the IPFT will be used to filter PTP frames and support tc flower. Tracking in software whether any IPFT entry exists for a port in order to decide whether to enable the ingress port filtering lookup would increase code complexity unnecessarily. Since enabling the lookup when the IPFT is empty has no effect on RX frames, enable the ingress port filtering lookup unconditionally during driver initialization to simplify code logic.

The only concern is that when deleting an IPFT entry fails, the hardware will continue filtering RX frames based on that entry. However, we have no choice. We cannot disable ingress port filter lookup because this would deactivate other IPFT entries, which is obviously not what we want. Moreover, deletion failures are almost impossible under normal circumstances, and we don't need to be bothered by this minor issue.

---
 drivers/net/dsa/netc/netc_main.c | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)
drivers/net/dsa/netc/netc_main.c +6 -3
index e1ec7aecab0f9..d7e96cb8d263d 100644
@@ -618,6 +618,12 @@ static void netc_port_fixed_config(struct netc_port *np)
 	netc_port_rmw(np, NETC_PCR, PCR_L2DOSE | PCR_L3DOSE,
 		      PCR_L2DOSE | PCR_L3DOSE);
 
+	/* Enable ingress port filter table lookup, if no match is found,
+	 * the frame is allowed and passed to the next frame processing
+	 * function.
+	 */
+	netc_port_wr(np, NETC_PIPFCR, PIPFCR_EN);
+
 	/* Set the quanta value of TX PAUSE frame */
 	netc_mac_port_wr(np, NETC_PM_PAUSE_QUANTA(0), NETC_PAUSE_QUANTA);
 
@@ -1865,8 +1871,6 @@ static int netc_port_add_host_flood_rule(struct netc_port *np,
 	np->uc = uc;
 	np->mc = mc;
 	np->ipft_hf_eid = host_flood->entry_id;
-	/* Enable ingress port filter table lookup */
-	netc_port_wr(np, NETC_PIPFCR, PIPFCR_EN);
 
 free_host_flood:
 	kfree(host_flood);
@@ -1895,7 +1899,6 @@ static int netc_port_remove_host_flood(struct netc_port *np)
 	np->ipft_hf_eid = NTMP_NULL_ENTRY_ID;
 	np->uc = false;
 	np->mc = false;
-	netc_port_wr(np, NETC_PIPFCR, 0);
 
 	return 0;
 }
Signature · 1 line
2.34.1
#8 lore raw

PATCH v8 8/9 net: dsa: netc: add PTP two-step timestamping support

In reply to [email protected]

From: Wei Fang <[email protected]>

Add two-step TX timestamping and RX timestamping for the NETC switch.

For RX, install ingress port filter table (IPFT) rules that redirect PTP frames to the CPU port, covering L2, L4 over IPv4 and L4 over IPv6 according to the hwtstamp rx_filter. The hardware inserts a To_Host subtype 1 tag carrying the ingress timestamp, which the tagger extracts into the skb control buffer for netc_port_rxtstamp() to report.

Track the installed rules only through the per-port ptp_ipft_eid[] handles, and let netc_port_hwtstamp_get() derive the active filter from them at read time, so a delete or rollback that fails partway does not leave software reporting a filter that no longer matches the hardware. The four L4 handles are installed and removed as one unit, so treat any surviving L4 handle as L4 active to avoid hiding rules that still trap frames to the CPU.

For two-step TX, clone the skb, assign a per-port timestamp request ID, and queue the clone. netc_xmit() carries the same ID in the To_Port subtype 2 tag of the transmitted frame. The hardware then returns a To_Host subtype 2 response frame echoing that ID and the transmit timestamp. The tagger dispatches both to the driver through the txtstamp_handler callback in netc_tagger_data, which matches the ID to the queued clone and completes it via skb_complete_tx_timestamp(). Non-PTP frames keep using the To_Port subtype 0 tag on the fast path.

The response frame carries no payload and is only 26 bytes long. As eth_type_trans() advances skb->data past the DMAC, SMAC and TPID, the tag starts at (skb->data - 2), so the pskb_may_pull() check must use (NETC_TAG_MAX_LEN - 2) to avoid dropping the response frame.

Manage request IDs per port: each transmit takes the first ID not in flight, and an ID is freed once its clone leaves the queue. A per-port delayed work drops unmatched clones that have been queued for at least 5 seconds, far beyond the hardware's normal response latency.

The response tag carries only the 4-bit ID and no PTP sequenceId, so a clone can be matched by that ID alone. If a frame cannot leave the port for over 5 seconds, its clone is dropped and the ID is reused while the response may still be in flight, so a stale response can complete the wrong clone. This only occurs after the link has been unable to transmit for that long, when PTP has already lost sync, so the mismatch affects only the first few frames right after recovery and is absorbed by the PTP servo. Accept this as a trade-off imposed by the hardware matching limit.

Select the TX tag from ptp_flag in netc_xmit(), and implement the DSA connect_tag_protocol() callback to wire netc_port_txtstamp_handler() into tagger_data->txtstamp_handler, so the tagger can hand the parsed response ID and timestamp back to the driver. Grab the PTP timer's pci_dev in netc_setup() so get_ts_info() can report its PHC index, and release it on teardown and error paths.

---
 drivers/net/dsa/netc/Kconfig         |   1 +
 drivers/net/dsa/netc/Makefile        |   3 +-
 drivers/net/dsa/netc/netc_main.c     |  82 ++++-
 drivers/net/dsa/netc/netc_platform.c |   1 +
 drivers/net/dsa/netc/netc_ptp.c      | 502 +++++++++++++++++++++++++++
 drivers/net/dsa/netc/netc_switch.h   |  41 +++
 include/linux/dsa/tag_netc.h         |  21 ++
 net/dsa/tag_netc.c                   | 165 ++++++++-
 8 files changed, 793 insertions(+), 23 deletions(-)
 create mode 100644 drivers/net/dsa/netc/netc_ptp.c
drivers/net/dsa/netc/Kconfig +1 -0
index 793f7691a24f3..8770b65d0f62d 100644
@@ -4,6 +4,7 @@ config NET_DSA_NETC_SWITCH
 	depends on ARM64 || COMPILE_TEST
 	depends on NET_DSA && PCI
 	depends on NET_VENDOR_FREESCALE
+	depends on PTP_1588_CLOCK_OPTIONAL
 	select NET_DSA_TAG_NETC
 	select FSL_ENETC_MDIO
 	select NXP_NTMP
drivers/net/dsa/netc/Makefile +2 -1
index f40b13c702e07..572b833ad80fc 100644
@@ -1,3 +1,4 @@
 # SPDX-License-Identifier: GPL-2.0-only
 obj-$(CONFIG_NET_DSA_NETC_SWITCH) += nxp-netc-switch.o
-nxp-netc-switch-objs := netc_main.o netc_platform.o netc_ethtool.o
+nxp-netc-switch-objs := netc_main.o netc_platform.o netc_ethtool.o \
+			netc_ptp.o
drivers/net/dsa/netc/netc_main.c +72 -10

192 lines not on this page · raw message

drivers/net/dsa/netc/netc_platform.c +1 -0
index 34aeb6fceb3cb..4fd0ce6770c33 100644
@@ -50,6 +50,7 @@ static void imx94_switch_phylink_get_caps(int port,
 
 static const struct netc_switch_info imx94_info = {
 	.num_ports = 4,
+	.tmr_devfn = PCI_DEVFN(0, 1),
 	.phylink_get_caps = imx94_switch_phylink_get_caps,
 };
 
drivers/net/dsa/netc/netc_ptp.c new file +502 -0

502 lines not on this page · raw message

drivers/net/dsa/netc/netc_switch.h +41 -0

76 lines not on this page · raw message

include/linux/dsa/tag_netc.h +21 -0
index fe964722e5b03..5a567af200940 100644
@@ -10,5 +10,26 @@
 #include <net/dsa.h>
 
 #define NETC_TAG_MAX_LEN			14
+#define NETC_PTP_FLAG_TWOSTEP			BIT(1)
+
+struct netc_skb_cb {
+	u64 ptp_tx_time;
+	u64 tstamp;
+	bool rx_tstamp_valid;
+	u8 ptp_flag;
+	u8 ts_req_id;
+};
+
+#define NETC_SKB_CB(skb)	((struct netc_skb_cb *)((skb)->cb))
+
+/**
+ * struct netc_tagger_data - NETC tagger/switch-driver shared operations
+ * @txtstamp_handler: Called by the tagger when a two-step transmit timestamp
+ *	response is received, to deliver the timestamp to the switch driver.
+ */
+struct netc_tagger_data {
+	void (*txtstamp_handler)(struct dsa_switch *ds, int port,
+				 u8 ts_req_id, u64 ts);
+};
 
 #endif
net/dsa/tag_netc.c +153 -12

242 lines not on this page · raw message

Signature · 1 line
2.34.1
#9 lore raw

PATCH v8 9/9 net: dsa: netc: add PTP one-step timestamping support

In reply to [email protected]

From: Wei Fang <[email protected]>

Add one-step Sync offload for NETC switch ports and advertise HWTSTAMP_TX_ONESTEP_SYNC.

Each port has a single PM_SINGLE_STEP register, which holds the offset of the correction field in the Sync frame and tells the hardware whether the UDP checksum needs to be updated. As the register is shared by all frames of the port, one-step Sync frames have to be sent one at a time. Therefore, let the tagger hand each one-step Sync over to the switch driver, which queues it on a per-port queue and drains the queue from a work.

For every frame, the work reads the current PTP time, programs PM_SINGLE_STEP, writes the time into the originTimestamp field and lets the tagger insert a To_Port subtype 3 tag carrying the lower 30 bits of that time together with a timestamp request ID. The hardware captures the SFD transmit time, adds the difference between it and the software timestamp to the correction field, and sends a response frame echoing the request ID once the Sync frame has left the port. Receiving the response schedules the work for the next frame.

A dropped frame produces no response, so reuse tstamp_timeout_work to reclaim its request ID after 5 seconds, which is far longer than the expected transmit time. When that reclaimed ID belongs to the in-flight frame, also resume the drain from the timeout work, otherwise the queued frames would never be sent unless new traffic happened to arrive.

The tag timestamp is only 30 bits wide and the hardware accounts for a single wrap at most, so the correction field of the in-flight frame can no longer be correct once it is sent out more than 2^30 ns after its software timestamp was read. There is no point in holding the queue for such a frame until the 5 second timeout. So when a new Sync is enqueued while a frame is in flight, also schedule the work if more than NETC_ONESTEP_VALID_WINDOW has passed since the in-flight frame was dequeued for transmission. Measure this time with jiffies_64 instead of the PTP time, as the PHC may be stepped backwards. As the check is only done at enqueue time, the queued frames still wait for tstamp_timeout_work if no new Sync arrives.

If the originTimestamp field of the sync frame is not in the linear buffer of the skb or the correction offset exceeds the hardware capability, the frame will be dropped. Frames are also dropped when the ts_req_id pool, shared with the two-step path, is exhausted. All such drops are counted in the interface tx_dropped statistics.

It is worth mentioning that due to the introduction of netc_timer_get_current_time(), a new dependency has been added to NET_DSA_NETC_SWITCH.

---
 drivers/net/dsa/netc/Kconfig          |   1 +
 drivers/net/dsa/netc/netc_main.c      |   6 +-
 drivers/net/dsa/netc/netc_ptp.c       | 392 +++++++++++++++++++++++++-
 drivers/net/dsa/netc/netc_switch.h    |  23 ++
 drivers/net/dsa/netc/netc_switch_hw.h |   5 +
 include/linux/dsa/tag_netc.h          |  17 ++
 net/dsa/tag_netc.c                    |  69 ++++-
 7 files changed, 507 insertions(+), 6 deletions(-)
drivers/net/dsa/netc/Kconfig +1 -0
index 8770b65d0f62d..7c930579f1c94 100644
@@ -5,6 +5,7 @@ config NET_DSA_NETC_SWITCH
 	depends on NET_DSA && PCI
 	depends on NET_VENDOR_FREESCALE
 	depends on PTP_1588_CLOCK_OPTIONAL
+	depends on PTP_NETC_V4_TIMER || PTP_NETC_V4_TIMER=n
 	select NET_DSA_TAG_NETC
 	select FSL_ENETC_MDIO
 	select NXP_NTMP
drivers/net/dsa/netc/netc_main.c +5 -1
index ddbb12573cfe4..6c19b3a811aa9 100644
@@ -75,6 +75,7 @@ static int netc_connect_tag_protocol(struct dsa_switch *ds,
 
 	tagger_data = ds->tagger_data;
 	tagger_data->txtstamp_handler = netc_port_txtstamp_handler;
+	tagger_data->onestep_sync_enqueue = netc_port_onestep_sync_enqueue;
 
 	return 0;
 }
@@ -94,7 +95,7 @@ static void netc_port_rmw(struct netc_port *np, u32 reg,
 	netc_port_wr(np, reg, new);
 }
 
-static void netc_mac_port_wr(struct netc_port *np, u32 reg, u32 val)
+void netc_mac_port_wr(struct netc_port *np, u32 reg, u32 val)
 {
 	if (is_netc_pseudo_port(np))
 		return;
@@ -1702,6 +1703,7 @@ static void netc_port_disable(struct dsa_switch *ds, int port)
 	if (!np->enable)
 		return;
 
+	netc_port_disable_onestep(np);
 	clk_disable_unprepare(np->ref_clk);
 	np->enable = false;
 }
@@ -2562,6 +2564,7 @@ static void netc_mac_link_up(struct phylink_config *config,
 	netc_port_set_rx_pause(np, rx_pause);
 	netc_port_mac_tx_enable(np);
 	netc_port_mac_rx_enable(np);
+	netc_port_enable_onestep(np);
 }
 
 static void netc_mac_link_down(struct phylink_config *config,
@@ -2572,6 +2575,7 @@ static void netc_mac_link_down(struct phylink_config *config,
 	struct netc_port *np;
 
 	np = NETC_PORT(dp->ds, dp->index);
+	netc_port_disable_onestep(np);
 	netc_port_mac_rx_graceful_stop(np);
 	netc_port_mac_tx_graceful_stop(np);
 	netc_port_remove_dynamic_entries(np);
drivers/net/dsa/netc/netc_ptp.c +390 -2

480 lines not on this page · raw message

drivers/net/dsa/netc/netc_switch.h +23 -0
index 5a8e57b01f35e..ae5e24c91f1cb 100644
@@ -87,6 +87,13 @@ enum netc_host_reason {
 	NETC_HR_PTP_TRAP   = 9,
 };
 
+enum netc_onestep_state {
+	NETC_ONESTEP_IDLE	= 0,
+	NETC_ONESTEP_SCHEDULED,
+	NETC_ONESTEP_IN_FLIGHT,
+	NETC_ONESTEP_PORT_INACTIVE,
+};
+
 struct netc_port {
 	void __iomem *iobase;
 	struct netc_switch *switch_priv;
@@ -102,6 +109,17 @@ struct netc_port {
 	u16 pvid;
 	u32 ipft_hf_eid; /* Must be initialized to NTMP_NULL_ENTRY_ID */
 
+	/* Serialize access to onestep_queue, onestep_state, onestep_tx_time
+	 * and onestep_ts_req_id
+	 */
+	spinlock_t onestep_lock;
+	u8 onestep_state;
+	u8 onestep_ts_req_id;
+	u64 onestep_tx_time;
+	/* skb queue for one-step Sync frames */
+	struct sk_buff_head onestep_queue;
+	struct work_struct onestep_work;
+
 	/* Serialize access to tstamp_queue */
 	spinlock_t tstamp_lock;
 	/* skb queue for TX timestamp frames */
@@ -213,6 +231,7 @@ static inline void netc_del_vlan_entry(struct netc_vlan_entry *entry)
 }
 
 int netc_switch_platform_probe(struct netc_switch *priv);
+void netc_mac_port_wr(struct netc_port *np, u32 reg, u32 val);
 
 /* ethtool APIs */
 void netc_port_get_pause_stats(struct dsa_switch *ds, int port,
@@ -244,5 +263,9 @@ void netc_port_txtstamp_handler(struct dsa_switch *ds, int port,
 bool netc_port_rxtstamp(struct dsa_switch *ds, int port, struct sk_buff *skb,
 			unsigned int type);
 void netc_port_txtstamp(struct dsa_switch *ds, int port, struct sk_buff *skb);
+void netc_port_disable_onestep(struct netc_port *np);
+void netc_port_enable_onestep(struct netc_port *np);
+void netc_port_onestep_sync_enqueue(struct dsa_switch *ds, int port,
+				    struct sk_buff *skb);
 
 #endif
drivers/net/dsa/netc/netc_switch_hw.h +5 -0
index 1404ae41c7bcb..37d1dd7ec2c7f 100644
@@ -203,6 +203,11 @@ enum netc_stg_stage {
 #define   SSP_10M			1
 #define   SSP_1G			2
 
+#define NETC_PM_SINGLE_STEP(a)		(0x10c0 + (a) * 0x400)
+#define  PM_SINGLE_STEP_CH		BIT(6)
+#define  PM_SINGLE_STEP_OFFSET		GENMASK(15, 7)
+#define  PM_SINGLE_STEP_EN		BIT(31)
+
 /* Port MAC 0/1 Receive Ethernet Octets Counter */
 #define NETC_PM_REOCT(a)		(0x1100 + (a) * 0x400)
 
include/linux/dsa/tag_netc.h +17 -0
index 5a567af200940..baf0f93bd9946 100644
@@ -10,7 +10,9 @@
 #include <net/dsa.h>
 
 #define NETC_TAG_MAX_LEN			14
+#define NETC_PTP_FLAG_ONESTEP			BIT(0)
 #define NETC_PTP_FLAG_TWOSTEP			BIT(1)
+#define NETC_PTP_FLAG_DROP			BIT(2)
 
 struct netc_skb_cb {
 	u64 ptp_tx_time;
@@ -18,6 +20,9 @@ struct netc_skb_cb {
 	bool rx_tstamp_valid;
 	u8 ptp_flag;
 	u8 ts_req_id;
+	bool is_udp;
+	u16 correction_offset;
+	u16 timestamp_offset;
 };
 
 #define NETC_SKB_CB(skb)	((struct netc_skb_cb *)((skb)->cb))
@@ -26,10 +31,22 @@ struct netc_skb_cb {
  * struct netc_tagger_data - NETC tagger/switch-driver shared operations
  * @txtstamp_handler: Called by the tagger when a two-step transmit timestamp
  *	response is received, to deliver the timestamp to the switch driver.
+ * @onestep_sync_enqueue: Called from the tagger xmit path for a one-step Sync
+ *	frame. The switch driver takes ownership of the skb and queues it for
+ *	deferred transmission from process context, where the shared
+ *	PM_SINGLE_STEP register can be programmed and the PTP timer read. The
+ *	tagger must not touch the skb after this call and returns NULL to
+ *	dsa_user_xmit().
+ * @onestep_sync_xmit: Called by the switch driver to transmit a deferred
+ *	one-step Sync frame directly to the conduit, bypassing dsa_user_xmit().
  */
 struct netc_tagger_data {
 	void (*txtstamp_handler)(struct dsa_switch *ds, int port,
 				 u8 ts_req_id, u64 ts);
+	void (*onestep_sync_enqueue)(struct dsa_switch *ds, int port,
+				     struct sk_buff *skb);
+	netdev_tx_t (*onestep_sync_xmit)(struct sk_buff *skb,
+					 struct net_device *ndev);
 };
 
 #endif
net/dsa/tag_netc.c +66 -3

113 lines not on this page · raw message

Signature · 1 line
2.34.1