Skip to contentSkip to conversation

net: stmmac: correct MAC propagation delay

PATCH Applied Johannes Zink 5 participants 9 messages Aug 1, 2023 – Aug 7, 2023

Subsystems (5)
lore
Share & tools
lore
#0 lore raw

v3 cover net: stmmac: correct MAC propagation delay

---
Changes in v3:
- work in Richard's review feedback. Thank you for reviewing my patch:
- as some of the hardware may have no or invalid correction value
registers: introduce feature switch which can be enabled in the glue
code drivers depending on the actual hardware support
- only enable the feature on the i.MX8MP for the time being, as the patch
improves timing accuracy and is tested for this hardware
- Link to v2: https://lore.kernel.org/r/20230719-stmmac_correct_mac_delay-v2-1-3366f38ee9a6@pengutronix.de

Changes in v2:
- fix builds for 32bit, this was found by the kernel build bot

	Reported-by: kernel test robot <[email protected]>
	Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

- while at it also fix an overflow by shifting a u32 constant from macro by 10bits
by casting the constant to u64
- Link to v1: https://lore.kernel.org/r/20230719-stmmac_correct_mac_delay-v1-1-768aa4d09334@pengutronix.de

---
Johannes Zink (2):
net: stmmac: correct MAC propagation delay
net: stmmac: dwmac-imx: enable MAC propagation delay correction for i.MX8MP

 drivers/net/ethernet/stmicro/stmmac/dwmac-imx.c    |  5 +++
 drivers/net/ethernet/stmicro/stmmac/hwif.h         |  3 ++
 .../net/ethernet/stmicro/stmmac/stmmac_hwtstamp.c  | 43 ++++++++++++++++++++++
 drivers/net/ethernet/stmicro/stmmac/stmmac_main.c  |  6 +++
 drivers/net/ethernet/stmicro/stmmac/stmmac_ptp.h   |  6 +++
 include/linux/stmmac.h                             |  1 +
 6 files changed, 64 insertions(+)

---
base-commit: 01e6f8ad8d26ced14b0cf288c42e55d03a7c5070
change-id: 20230719-stmmac_correct_mac_delay-4278cb9d9bc1

Best regards,

Signature · 7 lines
Johannes Zink <[email protected]>


_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
#1 lore raw

PATCH v3 1/2 net: stmmac: correct MAC propagation delay

The IEEE1588 Standard specifies that the timestamps of Packets must be captured when the PTP message timestamp point (leading edge of first octet after the start of frame delimiter) crosses the boundary between the node and the network. As the MAC latches the timestamp at an internal point, the captured timestamp must be corrected for the additional data transmission latency, as described in the publicly available datasheet [1].

This patch only corrects for the MAC-Internal delay, which can be read out from the MAC_Ingress_Timestamp_Latency register on DWMAC version 5, since the Phy framework currently does not support querying the Phy ingress and egress latency. The Closs Domain Crossing Circuits errors as indicated in [1] are already being accounted in the stmmac_get_tx_hwtstamp() function and are not corrected here.

As the Latency varies for different link speeds and MII modes of operation, the correction value needs to be updated on each link state change.

As the delay also causes a phase shift in the timestamp counter compared to the rest of the network, this correction will also reduce phase error when generating PPS outputs from the timestamp counter.

Since the correction registers may be unavailable on some hardware and no feature bits are documented for dynamically detection of the MAC propagation delay readout, introduce a feature bit to explicitely enable MAC delay Correction in the gluecode driver.

[1] i.MX8MP Reference Manual, rev.1 Section 11.7.2.5.3 "Timestamp correction"

---

Changes in v3:
- worked in Richard's review findings. Thank your for your review:
- don't do the correction unconditionally, since the correction registers are not guaranteed to be available on any hardware
- refactored function names to better reflect the functionality
- clarified commit message
Changes in v2:
- fix builds for 32bit, this was found by the kernel build bot

	Reported-by: kernel test robot <[email protected]>
	Closes:
	https://lore.kernel.org/oe-kbuild-all/[email protected]/
  - while at it also fix an overflow by shifting a u32 constant from macro by 10bits
    by casting the constant to u64
---
 drivers/net/ethernet/stmicro/stmmac/hwif.h         |  3 ++
 .../net/ethernet/stmicro/stmmac/stmmac_hwtstamp.c  | 43 ++++++++++++++++++++++
 drivers/net/ethernet/stmicro/stmmac/stmmac_main.c  |  6 +++
 drivers/net/ethernet/stmicro/stmmac/stmmac_ptp.h   |  6 +++
 include/linux/stmmac.h                             |  1 +
 5 files changed, 59 insertions(+)
drivers/net/ethernet/stmicro/stmmac/hwif.h +3 -0
index 652af8f6e75f..238f17c50a1e 100644
@@ -532,6 +532,7 @@ struct stmmac_hwtimestamp {
 	void (*get_systime) (void __iomem *ioaddr, u64 *systime);
 	void (*get_ptptime)(void __iomem *ioaddr, u64 *ptp_time);
 	void (*timestamp_interrupt)(struct stmmac_priv *priv);
+	void (*hwtstamp_correct_latency)(struct stmmac_priv *priv);
 };
 
 #define stmmac_config_hw_tstamping(__priv, __args...) \
@@ -550,6 +551,8 @@ struct stmmac_hwtimestamp {
 	stmmac_do_void_callback(__priv, ptp, get_ptptime, __args)
 #define stmmac_timestamp_interrupt(__priv, __args...) \
 	stmmac_do_void_callback(__priv, ptp, timestamp_interrupt, __args)
+#define stmmac_hwtstamp_correct_latency(__priv, __args...) \
+	stmmac_do_void_callback(__priv, ptp, hwtstamp_correct_latency, __args)
 
 struct stmmac_tx_queue;
 struct stmmac_rx_queue;
drivers/net/ethernet/stmicro/stmmac/stmmac_hwtstamp.c +43 -0

53 lines not on this page · raw message

drivers/net/ethernet/stmicro/stmmac/stmmac_main.c +6 -0
index e1f1c034d325..99aa5360b3ff 100644
@@ -909,6 +909,9 @@ static int stmmac_init_ptp(struct stmmac_priv *priv)
 	priv->hwts_tx_en = 0;
 	priv->hwts_rx_en = 0;
 
+	if (priv->plat->flags & STMMAC_FLAG_HWTSTAMP_CORRECT_LATENCY)
+		stmmac_hwtstamp_correct_latency(priv, priv);
+
 	return 0;
 }
 
@@ -1094,6 +1097,9 @@ static void stmmac_mac_link_up(struct phylink_config *config,
 
 	if (priv->dma_cap.fpesel)
 		stmmac_fpe_link_state_handle(priv, true);
+
+	if (priv->plat->flags & STMMAC_FLAG_HWTSTAMP_CORRECT_LATENCY)
+		stmmac_hwtstamp_correct_latency(priv, priv);
 }
 
 static const struct phylink_mac_ops stmmac_phylink_mac_ops = {
drivers/net/ethernet/stmicro/stmmac/stmmac_ptp.h +6 -0
index bf619295d079..d1fe4b46f162 100644
@@ -26,6 +26,12 @@
 #define	PTP_ACR		0x40	/* Auxiliary Control Reg */
 #define	PTP_ATNR	0x48	/* Auxiliary Timestamp - Nanoseconds Reg */
 #define	PTP_ATSR	0x4c	/* Auxiliary Timestamp - Seconds Reg */
+#define	PTP_TS_INGR_CORR_NS	0x58	/* Ingress timestamp correction nanoseconds */
+#define	PTP_TS_EGR_CORR_NS	0x5C	/* Egress timestamp correction nanoseconds*/
+#define	PTP_TS_INGR_CORR_SNS	0x60	/* Ingress timestamp correction subnanoseconds */
+#define	PTP_TS_EGR_CORR_SNS	0x64	/* Egress timestamp correction subnanoseconds */
+#define	PTP_TS_INGR_LAT	0x68	/* MAC internal Ingress Latency */
+#define	PTP_TS_EGR_LAT	0x6c	/* MAC internal Egress Latency */
 
 #define	PTP_STNSUR_ADDSUB_SHIFT	31
 #define	PTP_DIGITAL_ROLLOVER_MODE	0x3B9ACA00	/* 10e9-1 ns */
include/linux/stmmac.h +1 -0
index 3d0702510224..652404c03944 100644
@@ -218,6 +218,7 @@ struct dwmac4_addrs {
 #define STMMAC_FLAG_INT_SNAPSHOT_EN		BIT(9)
 #define STMMAC_FLAG_RX_CLK_RUNS_IN_LPI		BIT(10)
 #define STMMAC_FLAG_EN_TX_LPI_CLOCKGATING	BIT(11)
+#define STMMAC_FLAG_HWTSTAMP_CORRECT_LATENCY	BIT(12)
 
 struct plat_stmmacenet_data {
 	int bus_id;
Signature · 7 lines
2.39.2


_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
#2 lore raw

PATCH v3 2/2 net: stmmac: dwmac-imx: enable MAC propagation delay correction for i.MX8MP

In reply to Johannes Zink

As the i.MX8MP supports reading MAC propagation delay and correcting the Hardware timestamp counter for additional delays [1], enable the feature for this SoC.

This reduces phase error of the PPS output from the PTP Hardware Clock from approx 150ns to 100ns.

[1] i.MX8MP Reference Manual, rev.1 Section 11.7.2.5.3 "Timestamp correction"

---
 drivers/net/ethernet/stmicro/stmmac/dwmac-imx.c | 5 +++++
 1 file changed, 5 insertions(+)
drivers/net/ethernet/stmicro/stmmac/dwmac-imx.c +5 -0
index 92e06a96757a..645a9454a490 100644
@@ -42,6 +42,7 @@
 
 struct imx_dwmac_ops {
 	u32 addr_width;
+	u32 flags;
 	bool mac_rgmii_txclk_auto_adj;
 
 	int (*fix_soc_reset)(void *priv, void __iomem *ioaddr);
@@ -311,6 +312,9 @@ static int imx_dwmac_probe(struct platform_device *pdev)
 		goto err_parse_dt;
 	}
 
+	if (data->flags & STMMAC_FLAG_HWTSTAMP_CORRECT_LATENCY)
+		plat_dat->flags |= STMMAC_FLAG_HWTSTAMP_CORRECT_LATENCY;
+
 	plat_dat->host_dma_width = dwmac->ops->addr_width;
 	plat_dat->init = imx_dwmac_init;
 	plat_dat->exit = imx_dwmac_exit;
@@ -350,6 +354,7 @@ static struct imx_dwmac_ops imx8mp_dwmac_data = {
 	.addr_width = 34,
 	.mac_rgmii_txclk_auto_adj = false,
 	.set_intf_mode = imx8mp_set_intf_mode,
+	.flags = STMMAC_FLAG_HWTSTAMP_CORRECT_LATENCY,
 };
 
 static struct imx_dwmac_ops imx8dxl_dwmac_data = {
Signature · 7 lines
2.39.2


_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Tested-by on the v3 series
#3 lore raw
In reply to Johannes Zink
Quoted 14 lines · Johannes Zink

---
Changes in v3:
- work in Richard's review feedback. Thank you for reviewing my patch:
- as some of the hardware may have no or invalid correction value
registers: introduce feature switch which can be enabled in the glue
code drivers depending on the actual hardware support
- only enable the feature on the i.MX8MP for the time being, as the patch
improves timing accuracy and is tested for this hardware
- Link to v2: https://lore.kernel.org/r/20230719-stmmac_correct_mac_delay-v2-1-3366f38ee9a6@pengutronix.de

Changes in v2:
- fix builds for 32bit, this was found by the kernel build bot

	Reported-by: kernel test robot <[email protected]>
	Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
Quoted 8 lines

- while at it also fix an overflow by shifting a u32 constant from macro by 10bits
by casting the constant to u64
- Link to v1: https://lore.kernel.org/r/20230719-stmmac_correct_mac_delay-v1-1-768aa4d09334@pengutronix.de

---
Johannes Zink (2):
net: stmmac: correct MAC propagation delay
net: stmmac: dwmac-imx: enable MAC propagation delay correction for i.MX8MP

Tested on imx8mp <-> TSN Switch <-> x86 with i225:

Before your patch:

|ptp4l -i eth0 -f configs/gPTP.cfg --summary_interval=5 -m
|ptp4l[139.274]: rms    9 max   27 freq +29264 +/-  13 delay   347 +/-   2
|ptp4l[171.279]: rms   10 max   24 freq +29257 +/-  13 delay   344 +/-   2
|ptp4l[203.283]: rms   10 max   24 freq +29254 +/-  13 delay   347 +/-   2
|ptp4l[235.288]: rms    9 max   24 freq +29255 +/-  13 delay   346 +/-   1
|ptp4l[267.292]: rms    9 max   28 freq +29257 +/-  13 delay   347 +/-   2

After:

|ptp4l -i eth0 -f configs/gPTP.cfg --summary_interval=5 -m
|ptp4l[214.186]: rms    9 max   29 freq +28868 +/-  16 delay   326 +/-   2
|ptp4l[246.190]: rms    8 max   22 freq +28902 +/-  15 delay   329 +/-   2
|ptp4l[278.194]: rms    9 max   24 freq +28930 +/-  15 delay   325 +/-   1
|ptp4l[310.199]: rms    9 max   25 freq +28956 +/-  15 delay   327 +/-   3
|ptp4l[342.203]: rms    9 max   27 freq +28977 +/-  14 delay   327 +/-   1

And the derived register values:

|[   15.864016] KURT: PTP_TS_INGR_CORR_NS: 3147483248 PTP_TS_INGR_CORR_SNS: 0
|[   15.870862] KURT: PTP_TS_EGR_CORR_NS: 400 PTP_TS_EGR_CORR_SNS: 0
|[   20.000962] KURT: PTP_TS_INGR_CORR_NS: 3147483636 PTP_TS_INGR_CORR_SNS: 0
|[   20.007809] KURT: PTP_TS_EGR_CORR_NS: 12 PTP_TS_EGR_CORR_SNS: 0

So, seems to work:

Thanks,
Kurt

#4 lore raw

Hi Kurt,

Quoted 46 lines · Kurt Kanzenbach
Quoted 14 lines · Johannes Zink

---
Changes in v3:
- work in Richard's review feedback. Thank you for reviewing my patch:
- as some of the hardware may have no or invalid correction value
registers: introduce feature switch which can be enabled in the glue
code drivers depending on the actual hardware support
- only enable the feature on the i.MX8MP for the time being, as the patch
improves timing accuracy and is tested for this hardware
- Link to v2: https://lore.kernel.org/r/20230719-stmmac_correct_mac_delay-v2-1-3366f38ee9a6@pengutronix.de

Changes in v2:
- fix builds for 32bit, this was found by the kernel build bot

	Reported-by: kernel test robot <[email protected]>
	Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
Quoted 8 lines

- while at it also fix an overflow by shifting a u32 constant from macro by 10bits
by casting the constant to u64
- Link to v1: https://lore.kernel.org/r/20230719-stmmac_correct_mac_delay-v1-1-768aa4d09334@pengutronix.de

---
Johannes Zink (2):
net: stmmac: correct MAC propagation delay
net: stmmac: dwmac-imx: enable MAC propagation delay correction for i.MX8MP

Tested on imx8mp <-> TSN Switch <-> x86 with i225:

Before your patch:

|ptp4l -i eth0 -f configs/gPTP.cfg --summary_interval=5 -m
|ptp4l[139.274]: rms    9 max   27 freq +29264 +/-  13 delay   347 +/-   2
|ptp4l[171.279]: rms   10 max   24 freq +29257 +/-  13 delay   344 +/-   2
|ptp4l[203.283]: rms   10 max   24 freq +29254 +/-  13 delay   347 +/-   2
|ptp4l[235.288]: rms    9 max   24 freq +29255 +/-  13 delay   346 +/-   1
|ptp4l[267.292]: rms    9 max   28 freq +29257 +/-  13 delay   347 +/-   2

After:

|ptp4l -i eth0 -f configs/gPTP.cfg --summary_interval=5 -m
|ptp4l[214.186]: rms    9 max   29 freq +28868 +/-  16 delay   326 +/-   2
|ptp4l[246.190]: rms    8 max   22 freq +28902 +/-  15 delay   329 +/-   2
|ptp4l[278.194]: rms    9 max   24 freq +28930 +/-  15 delay   325 +/-   1
|ptp4l[310.199]: rms    9 max   25 freq +28956 +/-  15 delay   327 +/-   3
|ptp4l[342.203]: rms    9 max   27 freq +28977 +/-  14 delay   327 +/-   1

And the derived register values:

Quoted 8 lines
|[   15.864016] KURT: PTP_TS_INGR_CORR_NS: 3147483248 PTP_TS_INGR_CORR_SNS: 0
|[   15.870862] KURT: PTP_TS_EGR_CORR_NS: 400 PTP_TS_EGR_CORR_SNS: 0
|[   20.000962] KURT: PTP_TS_INGR_CORR_NS: 3147483636 PTP_TS_INGR_CORR_SNS: 0
|[   20.007809] KURT: PTP_TS_EGR_CORR_NS: 12 PTP_TS_EGR_CORR_SNS: 0

So, seems to work:

Thank you for testing!
Johannes

Quoted 3 lines

Thanks,
Kurt

Signature · 4 lines
Pengutronix e.K.                | Johannes Zink                  |
Steuerwalder Str. 21            | https://www.pengutronix.de/    |
31137 Hildesheim, Germany       | Phone: +49-5121-206917-0       |
Amtsgericht Hildesheim, HRA 2686| Fax:   +49-5121-206917-5555    |
#5 lore raw
Quoted 54 lines · Johannes Zink

Hi Kurt,

Quoted 51 lines · Kurt Kanzenbach
Quoted 14 lines · Johannes Zink

---
Changes in v3:
- work in Richard's review feedback. Thank you for reviewing my patch:
- as some of the hardware may have no or invalid correction value
registers: introduce feature switch which can be enabled in the glue
code drivers depending on the actual hardware support
- only enable the feature on the i.MX8MP for the time being, as the patch
improves timing accuracy and is tested for this hardware
- Link to v2: https://lore.kernel.org/r/20230719-stmmac_correct_mac_delay-v2-1-3366f38ee9a6@pengutronix.de

Changes in v2:
- fix builds for 32bit, this was found by the kernel build bot

	Reported-by: kernel test robot <[email protected]>
	Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
Quoted 8 lines

- while at it also fix an overflow by shifting a u32 constant from macro by 10bits
by casting the constant to u64
- Link to v1: https://lore.kernel.org/r/20230719-stmmac_correct_mac_delay-v1-1-768aa4d09334@pengutronix.de

---
Johannes Zink (2):
net: stmmac: correct MAC propagation delay
net: stmmac: dwmac-imx: enable MAC propagation delay correction for i.MX8MP

Tested on imx8mp <-> TSN Switch <-> x86 with i225:

Before your patch:

|ptp4l -i eth0 -f configs/gPTP.cfg --summary_interval=5 -m
|ptp4l[139.274]: rms    9 max   27 freq +29264 +/-  13 delay   347 +/-   2
|ptp4l[171.279]: rms   10 max   24 freq +29257 +/-  13 delay   344 +/-   2
|ptp4l[203.283]: rms   10 max   24 freq +29254 +/-  13 delay   347 +/-   2
|ptp4l[235.288]: rms    9 max   24 freq +29255 +/-  13 delay   346 +/-   1
|ptp4l[267.292]: rms    9 max   28 freq +29257 +/-  13 delay   347 +/-   2

After:

|ptp4l -i eth0 -f configs/gPTP.cfg --summary_interval=5 -m
|ptp4l[214.186]: rms    9 max   29 freq +28868 +/-  16 delay   326 +/-   2
|ptp4l[246.190]: rms    8 max   22 freq +28902 +/-  15 delay   329 +/-   2
|ptp4l[278.194]: rms    9 max   24 freq +28930 +/-  15 delay   325 +/-   1
|ptp4l[310.199]: rms    9 max   25 freq +28956 +/-  15 delay   327 +/-   3
|ptp4l[342.203]: rms    9 max   27 freq +28977 +/-  14 delay   327 +/-   1

And the derived register values:

|[   15.864016] KURT: PTP_TS_INGR_CORR_NS: 3147483248 PTP_TS_INGR_CORR_SNS: 0
|[   15.870862] KURT: PTP_TS_EGR_CORR_NS: 400 PTP_TS_EGR_CORR_SNS: 0
|[   20.000962] KURT: PTP_TS_INGR_CORR_NS: 3147483636 PTP_TS_INGR_CORR_SNS: 0
|[   20.007809] KURT: PTP_TS_EGR_CORR_NS: 12 PTP_TS_EGR_CORR_SNS: 0
Quoted 6 lines
Quoted 3 lines

So, seems to work:

Thank you for testing!
Johannes

AFAICT from the manuals the MAC propagation delay should be corrected for the Intel TSN NIC(s) as well. I'll follow up with testing and a patch when this set is merged.

Thanks,
Kurt

maintainer
#6 lore raw
In reply to Johannes Zink
Quoted 1 line · On Tue, 01 Aug 2023 17:44:28 +0200 Johannes Zink wrote:

---

Richard? Sure would be nice to have an official ack from you on this one so I don't have to revert it again ;)

#7 lore raw
Quoted 2 lines · Jakub Kicinski

Richard? Sure would be nice to have an official ack from you on this one so I don't have to revert it again ;)

No objections to this version, as the correction is behind a feature flag that is opt-in per device flavor.

Thanks,
Richard

#8 lore raw
In reply to Johannes Zink
This series was applied to netdev/net-next.git (main) by Jakub Kicinski <[email protected]>:

Hello:

This series was applied to netdev/net-next.git (main) by Jakub Kicinski <[email protected]>:

Quoted 11 lines · On Tue, 01 Aug 2023 17:44:28 +0200 you wrote:

---
Changes in v3:
- work in Richard's review feedback. Thank you for reviewing my patch:
- as some of the hardware may have no or invalid correction value
registers: introduce feature switch which can be enabled in the glue
code drivers depending on the actual hardware support
- only enable the feature on the i.MX8MP for the time being, as the patch
improves timing accuracy and is tested for this hardware
- Link to v2: https://lore.kernel.org/r/20230719-stmmac_correct_mac_delay-v2-1-3366f38ee9a6@pengutronix.de

[...]

Here is the summary with links:
- [v3,1/2] net: stmmac: correct MAC propagation delay

    https://git.kernel.org/netdev/net-next/c/26cfb838aa00
  - [v3,2/2] net: stmmac: dwmac-imx: enable MAC propagation delay correction for i.MX8MP
    https://git.kernel.org/netdev/net-next/c/6cb2e613c796

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html