Skip to contentSkip to conversation

Broadcom PTP PHY support

PATCH Superseded Jonathan Lemon 4 participants 13 messages May 18, 2022 – May 31, 2022

Subsystems (4)

v5 of 9 Superseded Latest is v9 →

lore
Share & tools
lore
#0 lore raw

v5 cover Broadcom PTP PHY support

This adds PTP support for the Broadcom PHY BCM54210E (and the specific variant BCM54213PE that the rpi-5.15 branch uses).

This has only been tested on the RPI CM4, which has one port.

There are other Broadcom chips which may benefit from using the same framework here, although with different register sets.

v4->v5:
Reorder so bcm-phy-lib.h shows up first, fixing dependency issues. Use set_normalized_timespec for adjusting ns in adjtime Return upscaled config setting from hwtstamp

v3->v4:
Squash bcm-phy-lib.h and broadcom.c changes into one patch Reorder so the main patch shows up first.

v2->v3:
Rearrange patches so they apply in order
Use ERR_CAST

v1->v2:
Squash Kconfig into main patch
Move config checks into bcm-phy-lib.h
Fix delta_ns calculations in adjtime
Uppercase mode selector macros
Only use NSE_INIT when necessary
Remove the inserted Broadcom RX timestamp from the PTP packet Add perout (chip generated) and fsync out (timer generated) Remove PHY_ID_BCM54213PE special casing (needed for rpi tree)

Thanks to Lasse Johnsen <[email protected]> for pointing out that the chip's periodic output generation isn't sync'd to any time base.

Jonathan Lemon (2):
net: phy: broadcom: Add Broadcom PTP hooks to bcm-phy-lib
net: phy: broadcom: Add PTP support for some Broadcom PHYs.

 drivers/net/phy/Kconfig       |  10 +
 drivers/net/phy/Makefile      |   1 +
 drivers/net/phy/bcm-phy-lib.h |  14 +
 drivers/net/phy/bcm-phy-ptp.c | 868 ++++++++++++++++++++++++++++++++++
 drivers/net/phy/broadcom.c    |  23 +-
 5 files changed, 912 insertions(+), 4 deletions(-)
 create mode 100644 drivers/net/phy/bcm-phy-ptp.c
Signature · 1 line
2.34.3
#1 lore raw

PATCH v5 1/2 net: phy: broadcom: Add Broadcom PTP hooks to bcm-phy-lib

Add 'struct bcm_ptp_private' to bcm54xx_phy_priv which points to an optional PTP structure attached to the PHY. This is allocated on probe if PHY PTP support is configured, and if the driver supports PTP for the specified PHY.

Add the bcm_ptp_probe() and bcm_ptp_config_init() API functions to the bcm-phy library.

---
 drivers/net/phy/bcm-phy-lib.h | 14 ++++++++++++++
 drivers/net/phy/broadcom.c    | 23 +++++++++++++++++++----
 2 files changed, 33 insertions(+), 4 deletions(-)
drivers/net/phy/bcm-phy-lib.h +14 -0
index c3842f87c33b..c2cc3c2766a1 100644
@@ -87,4 +87,18 @@ int bcm_phy_cable_test_start_rdb(struct phy_device *phydev);
 int bcm_phy_cable_test_start(struct phy_device *phydev);
 int bcm_phy_cable_test_get_status(struct phy_device *phydev, bool *finished);
 
+#if IS_ENABLED(CONFIG_BCM_NET_PHYPTP)
+struct bcm_ptp_private *bcm_ptp_probe(struct phy_device *phydev);
+void bcm_ptp_config_init(struct phy_device *phydev);
+#else
+static inline struct bcm_ptp_private *bcm_ptp_probe(struct phy_device *phydev)
+{
+	return NULL;
+}
+
+static inline void bcm_ptp_config_init(struct phy_device *phydev)
+{
+}
+#endif
+
 #endif /* _LINUX_BCM_PHY_LIB_H */
drivers/net/phy/broadcom.c +19 -4
index e36809aa6d30..a7722599b5f9 100644
@@ -27,6 +27,11 @@ MODULE_DESCRIPTION("Broadcom PHY driver");
 MODULE_AUTHOR("Maciej W. Rozycki");
 MODULE_LICENSE("GPL");
 
+struct bcm54xx_phy_priv {
+	u64	*stats;
+	struct bcm_ptp_private *ptp;
+};
+
 static int bcm54xx_config_clock_delay(struct phy_device *phydev)
 {
 	int rc, val;
@@ -313,6 +318,14 @@ static void bcm54xx_adjust_rxrefclk(struct phy_device *phydev)
 		bcm_phy_write_shadow(phydev, BCM54XX_SHD_APD, val);
 }
 
+static void bcm54xx_ptp_config_init(struct phy_device *phydev)
+{
+	struct bcm54xx_phy_priv *priv = phydev->priv;
+
+	if (priv->ptp)
+		bcm_ptp_config_init(phydev);
+}
+
 static int bcm54xx_config_init(struct phy_device *phydev)
 {
 	int reg, err, val;
@@ -390,6 +403,8 @@ static int bcm54xx_config_init(struct phy_device *phydev)
 		bcm_phy_write_exp(phydev, BCM_EXP_MULTICOLOR, val);
 	}
 
+	bcm54xx_ptp_config_init(phydev);
+
 	return 0;
 }
 
@@ -741,10 +756,6 @@ static irqreturn_t brcm_fet_handle_interrupt(struct phy_device *phydev)
 	return IRQ_HANDLED;
 }
 
-struct bcm54xx_phy_priv {
-	u64	*stats;
-};
-
 static int bcm54xx_phy_probe(struct phy_device *phydev)
 {
 	struct bcm54xx_phy_priv *priv;
@@ -761,6 +772,10 @@ static int bcm54xx_phy_probe(struct phy_device *phydev)
 	if (!priv->stats)
 		return -ENOMEM;
 
+	priv->ptp = bcm_ptp_probe(phydev);
+	if (IS_ERR(priv->ptp))
+		return PTR_ERR(priv->ptp);
+
 	return 0;
 }
 
Signature · 1 line
2.34.3
#2 lore raw

PATCH v5 2/2 net: phy: broadcom: Add PTP support for some Broadcom PHYs.

In reply to Jonathan Lemon

This adds PTP support for BCM54210E Broadcom PHYs, in particular, the BCM54213PE, as used in the Rasperry PI CM4. It has only been tested on that hardware.

---
 drivers/net/phy/Kconfig       |  10 +
 drivers/net/phy/Makefile      |   1 +
 drivers/net/phy/bcm-phy-ptp.c | 868 ++++++++++++++++++++++++++++++++++
 3 files changed, 879 insertions(+)
 create mode 100644 drivers/net/phy/bcm-phy-ptp.c
drivers/net/phy/Kconfig +10 -0
index 9fee639ee5c8..68d3fc6beaae 100644
@@ -108,6 +108,16 @@ config BROADCOM_PHY
 	  Currently supports the BCM5411, BCM5421, BCM5461, BCM54616S, BCM5464,
 	  BCM5481, BCM54810 and BCM5482 PHYs.
 
+config BCM_NET_PHYPTP
+	tristate "Broadcom PHY PTP support"
+	depends on NETWORK_PHY_TIMESTAMPING
+	depends on PHYLIB
+	depends on PTP_1588_CLOCK
+	depends on BROADCOM_PHY
+	depends on NET_PTP_CLASSIFY
+	help
+	  Supports PTP timestamping for certain Broadcom PHYs.
+
 config BCM54140_PHY
 	tristate "Broadcom BCM54140 PHY"
 	depends on PHYLIB
drivers/net/phy/Makefile +1 -0
index b12b1d86fc99..f7138d3c896b 100644
@@ -47,6 +47,7 @@ obj-$(CONFIG_BCM84881_PHY)	+= bcm84881.o
 obj-$(CONFIG_BCM87XX_PHY)	+= bcm87xx.o
 obj-$(CONFIG_BCM_CYGNUS_PHY)	+= bcm-cygnus.o
 obj-$(CONFIG_BCM_NET_PHYLIB)	+= bcm-phy-lib.o
+obj-$(CONFIG_BCM_NET_PHYPTP)	+= bcm-phy-ptp.o
 obj-$(CONFIG_BROADCOM_PHY)	+= broadcom.o
 obj-$(CONFIG_CICADA_PHY)	+= cicada.o
 obj-$(CONFIG_CORTINA_PHY)	+= cortina.o
drivers/net/phy/bcm-phy-ptp.c new file +868 -0

868 lines not on this page · raw message

Signature · 1 line
2.34.3
#3 lore raw

Re: [PATCH net-next v5 2/2] net: phy: broadcom: Add PTP support for some Broadcom PHYs.

Quoted 6 lines · Jonathan Lemon

This adds PTP support for BCM54210E Broadcom PHYs, in particular, the BCM54213PE, as used in the Rasperry PI CM4. It has only been tested on that hardware.

---

[snip]

Looks good to me, just one question below:

Quoted 5 lines
drivers/net/phy/bcm-phy-ptp.c new file +5 -0
+static void bcm_ptp_init(struct bcm_ptp_private *priv)
+{
+	priv->nse_ctrl = NSE_GMODE_EN;
+
+	mutex_init(&priv->mutex);

drivers/net/phy/bcm-phy-ptp.c line 835

Quoted 10 lines
drivers/net/phy/bcm-phy-ptp.c new file +10 -0
+	skb_queue_head_init(&priv->tx_queue);
+
+	priv->mii_ts.rxtstamp = bcm_ptp_rxtstamp;
+	priv->mii_ts.txtstamp = bcm_ptp_txtstamp;
+	priv->mii_ts.hwtstamp = bcm_ptp_hwtstamp;
+	priv->mii_ts.ts_info = bcm_ptp_ts_info;
+
+	priv->phydev->mii_ts = &priv->mii_ts;
+
+	INIT_DELAYED_WORK(&priv->out_work, bcm_ptp_fsync_work);

Do we need to make sure that we cancel the workqueue in an bcm_ptp_exit() function?

I would imagine that the Ethernet MAC attached to that PHY device having stopped its receiver and transmitter should ensure no more packets coming in or out, however since this is a delayed/asynchronous work, do not we need to protect against use after free?

Signature · 1 line
Florian
#4 lore raw

Re: [PATCH net-next v5 2/2] net: phy: broadcom: Add PTP support for some Broadcom PHYs.

Quoted 32 lines · Florian Fainelli
Quoted 6 lines · Jonathan Lemon

This adds PTP support for BCM54210E Broadcom PHYs, in particular, the BCM54213PE, as used in the Rasperry PI CM4. It has only been tested on that hardware.

---

[snip]

Looks good to me, just one question below:

Quoted 5 lines
drivers/net/phy/bcm-phy-ptp.c new file +5 -0
+static void bcm_ptp_init(struct bcm_ptp_private *priv)
+{
+	priv->nse_ctrl = NSE_GMODE_EN;
+
+	mutex_init(&priv->mutex);

drivers/net/phy/bcm-phy-ptp.c line 835

Quoted 10 lines
drivers/net/phy/bcm-phy-ptp.c new file +10 -0
+	skb_queue_head_init(&priv->tx_queue);
+
+	priv->mii_ts.rxtstamp = bcm_ptp_rxtstamp;
+	priv->mii_ts.txtstamp = bcm_ptp_txtstamp;
+	priv->mii_ts.hwtstamp = bcm_ptp_hwtstamp;
+	priv->mii_ts.ts_info = bcm_ptp_ts_info;
+
+	priv->phydev->mii_ts = &priv->mii_ts;
+
+	INIT_DELAYED_WORK(&priv->out_work, bcm_ptp_fsync_work);

Do we need to make sure that we cancel the workqueue in an bcm_ptp_exit() function?

Quoted 4 lines

I would imagine that the Ethernet MAC attached to that PHY device having stopped its receiver and transmitter should ensure no more packets coming in or out, however since this is a delayed/asynchronous work, do not we need to protect against use after free?

The workqueue is just mamually creatimg a 1PPS pulse on the SYNC_OUT pin, no packet activity. Arguably, the .suspend hook could stop all work, but that seems out of scope here? (and this phy does not suspend/resume)

Signature · 1 line
Jonathan   (one handed typist for a few weeks)
#5 lore raw

Re: [PATCH net-next v5 2/2] net: phy: broadcom: Add PTP support for some Broadcom PHYs.

Quoted 33 lines · Jonathan Lemon
Quoted 32 lines · Florian Fainelli
Quoted 6 lines · Jonathan Lemon

This adds PTP support for BCM54210E Broadcom PHYs, in particular, the BCM54213PE, as used in the Rasperry PI CM4. It has only been tested on that hardware.

---

[snip]

Looks good to me, just one question below:

Quoted 5 lines
drivers/net/phy/bcm-phy-ptp.c new file +5 -0
+static void bcm_ptp_init(struct bcm_ptp_private *priv)
+{
+	priv->nse_ctrl = NSE_GMODE_EN;
+
+	mutex_init(&priv->mutex);

drivers/net/phy/bcm-phy-ptp.c line 835

Quoted 10 lines
drivers/net/phy/bcm-phy-ptp.c new file +10 -0
+	skb_queue_head_init(&priv->tx_queue);
+
+	priv->mii_ts.rxtstamp = bcm_ptp_rxtstamp;
+	priv->mii_ts.txtstamp = bcm_ptp_txtstamp;
+	priv->mii_ts.hwtstamp = bcm_ptp_hwtstamp;
+	priv->mii_ts.ts_info = bcm_ptp_ts_info;
+
+	priv->phydev->mii_ts = &priv->mii_ts;
+
+	INIT_DELAYED_WORK(&priv->out_work, bcm_ptp_fsync_work);

Do we need to make sure that we cancel the workqueue in an bcm_ptp_exit() function?

Quoted 8 lines
Quoted 4 lines

I would imagine that the Ethernet MAC attached to that PHY device having stopped its receiver and transmitter should ensure no more packets coming in or out, however since this is a delayed/asynchronous work, do not we need to protect against use after free?

The workqueue is just mamually creatimg a 1PPS pulse on the SYNC_OUT pin, no packet activity. Arguably, the .suspend hook could stop all work, but that seems out of scope here? (and this phy does not suspend/resume)

The BCM54210E entry does have a suspend/resume entry so it seems to me that we do need to cancel the workqueue as the PHY library will not do that on our behalf. What I imagine could happen is that this workqueue generates spurious MDIO accesses *after* both the PHY and the bus have been suspended (and their driver's clock possibly gated already).

Signature · 1 line
Florian
#6 lore raw

Re: [PATCH net-next v5 2/2] net: phy: broadcom: Add PTP support for some Broadcom PHYs.

Quoted 45 lines · Florian Fainelli
Quoted 33 lines · Jonathan Lemon
Quoted 32 lines · Florian Fainelli
Quoted 6 lines · Jonathan Lemon

This adds PTP support for BCM54210E Broadcom PHYs, in particular, the BCM54213PE, as used in the Rasperry PI CM4. It has only been tested on that hardware.

---

[snip]

Looks good to me, just one question below:

Quoted 5 lines
drivers/net/phy/bcm-phy-ptp.c new file +5 -0
+static void bcm_ptp_init(struct bcm_ptp_private *priv)
+{
+	priv->nse_ctrl = NSE_GMODE_EN;
+
+	mutex_init(&priv->mutex);

drivers/net/phy/bcm-phy-ptp.c line 835

Quoted 10 lines
drivers/net/phy/bcm-phy-ptp.c new file +10 -0
+	skb_queue_head_init(&priv->tx_queue);
+
+	priv->mii_ts.rxtstamp = bcm_ptp_rxtstamp;
+	priv->mii_ts.txtstamp = bcm_ptp_txtstamp;
+	priv->mii_ts.hwtstamp = bcm_ptp_hwtstamp;
+	priv->mii_ts.ts_info = bcm_ptp_ts_info;
+
+	priv->phydev->mii_ts = &priv->mii_ts;
+
+	INIT_DELAYED_WORK(&priv->out_work, bcm_ptp_fsync_work);

Do we need to make sure that we cancel the workqueue in an bcm_ptp_exit() function?

Quoted 8 lines
Quoted 4 lines

I would imagine that the Ethernet MAC attached to that PHY device having stopped its receiver and transmitter should ensure no more packets coming in or out, however since this is a delayed/asynchronous work, do not we need to protect against use after free?

The workqueue is just mamually creatimg a 1PPS pulse on the SYNC_OUT pin, no packet activity. Arguably, the .suspend hook could stop all work, but that seems out of scope here? (and this phy does not suspend/resume)

Quoted 5 lines

The BCM54210E entry does have a suspend/resume entry so it seems to me that we do need to cancel the workqueue as the PHY library will not do that on our behalf. What I imagine could happen is that this workqueue generates spurious MDIO accesses *after* both the PHY and the bus have been suspended (and their driver's clock possibly gated already).

Yes, you're right. I was looking at the rpi-5.15.y tree, which doesn't have these hooks yet. I'll add a call to stop the workqueue.

Actually, in the next series, I'll break out the extts/perout into separate patch.

Thanks for pointing this out!

Signature · 1 line
Jonathan
#7 lore raw

Re: [PATCH net-next v5 2/2] net: phy: broadcom: Add PTP support for some Broadcom PHYs.

In reply to Jonathan Lemon

drivers/net/phy/bcm-phy-ptp.c line 294

Quoted 10 lines · Jonathan Lemon
drivers/net/phy/bcm-phy-ptp.c new file +10 -0
+static int bcm_ptp_adjtime_locked(struct bcm_ptp_private *priv,
+				  s64 delta_ns)
+{
+	struct timespec64 ts;
+	int err;
+
+	err = bcm_ptp_gettime_locked(priv, &ts, NULL);
+	if (!err) {
+		set_normalized_timespec64(&ts, ts.tv_sec,
+					  ts.tv_nsec + delta_ns);

This also takes a LONG time when delta is large...

drivers/net/phy/bcm-phy-ptp.c line 298

Quoted 4 lines
drivers/net/phy/bcm-phy-ptp.c new file +4 -0
+		err = bcm_ptp_settime_locked(priv, &ts);
+	}
+	return err;
+}

Try this instead:

	s64 ns;

	err = bcm_ptp_gettime_locked(priv, &ts, NULL);
	if (err) {
		return err;
	}
	ns = timespec64_to_ns(&ts);		
	ns += delta_ns;
	ts = ns_to_timespec64(ns);
	err = bcm_ptp_settime_locked(priv, &ts);

	return err;

Thanks,
Richard

#8 lore raw

Re: [PATCH net-next v5 2/2] net: phy: broadcom: Add PTP support for some Broadcom PHYs.

Quoted 14 lines · Richard Cochran

drivers/net/phy/bcm-phy-ptp.c line 294

Quoted 10 lines · Jonathan Lemon
drivers/net/phy/bcm-phy-ptp.c new file +10 -0
+static int bcm_ptp_adjtime_locked(struct bcm_ptp_private *priv,
+				  s64 delta_ns)
+{
+	struct timespec64 ts;
+	int err;
+
+	err = bcm_ptp_gettime_locked(priv, &ts, NULL);
+	if (!err) {
+		set_normalized_timespec64(&ts, ts.tv_sec,
+					  ts.tv_nsec + delta_ns);

This also takes a LONG time when delta is large...

Didn't we just go through this? What constitutes a "large" offset here? The current version seems acceptable to me:

root@rpi:~/src/rpi # time phc_ctl /dev/ptp0 -- adj 86400 phc_ctl[766492.486]: adjusted clock by 86400.000000 seconds

real 0m0.009s
user 0m0.002s
sys 0m0.007s

root@rpi:~/src/rpi # time phc_ctl /dev/ptp0 -- adj -86400 phc_ctl[766494.647]: adjusted clock by -86400.000000 seconds

real 0m0.009s
user 0m0.009s
sys 0m0.000s

Signature · 1 line
Jonathan
#9 lore raw

Re: [PATCH net-next v5 2/2] net: phy: broadcom: Add PTP support for some Broadcom PHYs.

Quoted 16 lines · Jonathan Lemon
Quoted 14 lines · Richard Cochran

drivers/net/phy/bcm-phy-ptp.c line 294

Quoted 10 lines · Jonathan Lemon
drivers/net/phy/bcm-phy-ptp.c new file +10 -0
+static int bcm_ptp_adjtime_locked(struct bcm_ptp_private *priv,
+				  s64 delta_ns)
+{
+	struct timespec64 ts;
+	int err;
+
+	err = bcm_ptp_gettime_locked(priv, &ts, NULL);
+	if (!err) {
+		set_normalized_timespec64(&ts, ts.tv_sec,
+					  ts.tv_nsec + delta_ns);

This also takes a LONG time when delta is large...

Quoted 2 lines

Didn't we just go through this? What constitutes a "large" offset here? The current version seems acceptable to me:

When the PHY boots, it starts from time zero.

Then as a client it needs to adjust to today, something like:

1653951762.413809006 or Mon May 30 16:02:42 2022

(that means adding 1,653,951,762,413,809,006 nanoseconds)

Try that and see how long it takes to apply the adjustment.

Thanks,
Richard

#10 lore raw

Re: [PATCH net-next v5 2/2] net: phy: broadcom: Add PTP support for some Broadcom PHYs.

Quoted 26 lines · Richard Cochran
Quoted 16 lines · Jonathan Lemon
Quoted 15 lines · Richard Cochran

drivers/net/phy/bcm-phy-ptp.c line 294

Quoted 12 lines · Jonathan Lemon
drivers/net/phy/bcm-phy-ptp.c new file +10 -0
+static int bcm_ptp_adjtime_locked(struct bcm_ptp_private *priv,
+                  s64 delta_ns)
+{
+    struct timespec64 ts;
+    int err;
+
+    err = bcm_ptp_gettime_locked(priv, &ts, NULL);
+    if (!err) {
+        set_normalized_timespec64(&ts, ts.tv_sec,
+                      ts.tv_nsec + delta_ns);

This also takes a LONG time when delta is large...

Quoted 2 lines

Didn't we just go through this? What constitutes a "large" offset here? The current version seems acceptable to me:

When the PHY boots, it starts from time zero.

Quoted 7 lines

Then as a client it needs to adjust to today, something like:

1653951762.413809006 or Mon May 30 16:02:42 2022

(that means adding 1,653,951,762,413,809,006 nanoseconds)

Try that and see how long it takes to apply the adjustment.

That’s certainly a big adjustment. I was expecting something along the lines of a settime() call, followed up by adjtime().

Will fix. Seems like there should be a better call sequence for this, though.

Sent from my iPhone

#11 lore raw
In reply to Jonathan Lemon
Quoted 2 lines · Jonathan Lemon

This adds PTP support for the Broadcom PHY BCM54210E (and the specific variant BCM54213PE that the rpi-5.15 branch uses).

I'm interested in this series. Can you please include me on CC?

Thanks,
Richard

#12 lore raw
Quoted 5 lines · Richard Cochran
Quoted 2 lines · Jonathan Lemon

This adds PTP support for the Broadcom PHY BCM54210E (and the specific variant BCM54213PE that the rpi-5.15 branch uses).

I'm interested in this series. Can you please include me on CC?

Sure thing! (I thought I had already, if not, it was an unintentional oversight.)

Signature · 1 line
Jonathan