Unlock new potential in SJA1105 with PTP system timestamping
Vladimir Oltean
maintainercover Unlock new potential in SJA1105 with PTP system timestamping
The SJA1105 being an automotive switch means it is designed to live in a set-and-forget environment, far from the configure-at-runtime nature of Linux. Frequently resetting the switch to change its static config means it loses track of its PTP time, which is not good.
This patch series implements PTP system timestamping for this switch (using the API introduced for SPI here:
https://www.mail-archive.com/[email protected]/msg316725.html),
adding the following benefits to the driver:
- When under control of a user space PTP servo loop (ptp4l, phc2sys), the loss of sync during a switch reset is much more manageable, and the switch still remains in the s2 (locked servo) state.
- When synchronizing the switch using the software technique (based on reading clock A and writing the value to clock B, as opposed to relying on hardware timestamping), e.g. by using phc2sys, the sync accuracy is vastly improved due to the fact that the actual switch PTP time can now be more precisely correlated with something of better precision (CLOCK_REALTIME). The issue is that SPI transfers are inherently bad for measuring time with low jitter, but the newly introduced API aims to alleviate that issue somewhat.
This series is also a requirement for a future patch set that adds full time-aware scheduling offload support for the switch.
Vladimir Oltean (3):
net: dsa: sja1105: Implement the .gettimex64 system call for PTP
net: dsa: sja1105: Restore PTP time after switch reset
net: dsa: sja1105: Disallow management xmit during switch reset
drivers/net/dsa/sja1105/sja1105.h | 6 +- drivers/net/dsa/sja1105/sja1105_main.c | 42 +++++++++- drivers/net/dsa/sja1105/sja1105_ptp.c | 103 ++++++++++++++++++------- drivers/net/dsa/sja1105/sja1105_ptp.h | 24 +++++- drivers/net/dsa/sja1105/sja1105_spi.c | 54 +++++++++---- 5 files changed, 177 insertions(+), 52 deletions(-)
Signature · 1 line
2.17.1
Vladimir Oltean
maintainerPATCH 1/3 net: dsa: sja1105: Implement the .gettimex64 system call for PTP
Through the PTP_SYS_OFFSET_EXTENDED ioctl, it is possible for userspace applications (i.e. phc2sys) to compensate for the delays incurred while reading the PHC's time.
The task itself of taking the software timestamp is delegated to the SPI subsystem, through the newly introduced API in struct spi_transfer. The goal is to cross-timestamp I/O operations on the switch's PTP clock with values in the local system clock (CLOCK_REALTIME). For that we need to understand a bit of the hardware internals.
The 'read PTP time' message is a 12 byte structure, first 4 bytes of which represent the SPI header, and the last 8 bytes represent the 64-bit PTP time. The switch itself starts processing the command immediately after receiving the last bit of the address, i.e. at the middle of byte 3 (last byte of header). The PTP time is shadowed to a buffer register in the switch, and retrieved atomically during the subsequent SPI frames.
A similar thing goes on for the 'write PTP time' message, although in that case the switch waits until the 64-bit PTP time becomes fully available before taking any action. So the byte that needs to be software-timestamped is byte 11 (last) of the transfer.
The patch creates a common (and local) sja1105_xfer implementation for the SPI I/O, and offers 3 front-ends:
- sja1105_xfer_u32 and sja1105_xfer_u64: these are capable of optionally requesting a PTP timestamp
- sja1105_xfer_buf: this is for large transfers (e.g. the static config buffer) and other misc data, and there is no point in giving timestamping capabilities to this.
--- drivers/net/dsa/sja1105/sja1105.h | 6 ++-- drivers/net/dsa/sja1105/sja1105_main.c | 3 +- drivers/net/dsa/sja1105/sja1105_ptp.c | 26 ++++++++------ drivers/net/dsa/sja1105/sja1105_spi.c | 48 +++++++++++++++++++++----- 4 files changed, 61 insertions(+), 22 deletions(-)
drivers/net/dsa/sja1105/sja1105.h +4 -2
@@ -122,9 +122,11 @@ int sja1105_xfer_buf(const struct sja1105_private *priv, sja1105_spi_rw_mode_t rw, u64 reg_addr, u8 *buf, size_t len);int sja1105_xfer_u32 (const struct sja1105_private *priv, -sja1105_spi_rw_mode_t rw,u64 reg_addr,u32 *value); +sja1105_spi_rw_mode_t rw,u64 reg_addr,u32 *value, +struct ptp_system_timestamp *ptp_sts);int sja1105_xfer_u64 (const struct sja1105_private *priv, -sja1105_spi_rw_mode_t rw,u64 reg_addr,u64 *value); +sja1105_spi_rw_mode_t rw,u64 reg_addr,u64 *value, +struct ptp_system_timestamp *ptp_sts);int sja1105_static_config_upload (struct sja1105_private *priv);int sja1105_inhibit_tx (const struct sja1105_private *priv,unsigned long port_bitmap,bool tx_inhibited);
drivers/net/dsa/sja1105/sja1105_main.c +2 -1
@@ -1973,7 +1973,8 @@ static int sja1105_check_device_id(struct sja1105_private *priv)u64 part_no;int rc; - rc =sja1105_xfer_u32 (priv,SPI_READ , regs->device_id, &device_id); + rc =sja1105_xfer_u32 (priv,SPI_READ , regs->device_id, &device_id, +NULL );if (rc <0 )return rc;
drivers/net/dsa/sja1105/sja1105_ptp.c +16 -10
@@ -1,6 +1,7 @@// SPDX-License-Identifier: GPL-2.0 /* Copyright (c) 2019, Vladimir Oltean <[email protected]> */ +#include <linux/spi/spi.h> #include "sja1105.h" /* The adjfine API clamps ppb between [-32,768,000, 32,768,000], and
@@ -335,11 +336,13 @@ static int sja1105_ptpegr_ts_poll(struct dsa_switch *ds, int port, u64 *ts) }/* Caller must hold ptp_data->lock */ -static int sja1105_ptpclkval_read (struct sja1105_private *priv,u64 *ticks) +static int sja1105_ptpclkval_read (struct sja1105_private *priv,u64 *ticks, +struct ptp_system_timestamp *ptp_sts) {const struct sja1105_regs *regs = priv->info->regs; -return sja1105_xfer_u64 (priv,SPI_READ , regs->ptpclkval, ticks); +return sja1105_xfer_u64 (priv,SPI_READ , regs->ptpclkval, ticks, + ptp_sts); }/* Caller must hold ptp_data->lock */
@@ -347,7 +350,8 @@ static int sja1105_ptpclkval_write(struct sja1105_private *priv, u64 ticks) {const struct sja1105_regs *regs = priv->info->regs; -return sja1105_xfer_u64 (priv,SPI_WRITE , regs->ptpclkval, &ticks); +return sja1105_xfer_u64 (priv,SPI_WRITE , regs->ptpclkval, &ticks, +NULL ); }#define rxtstamp_to_tagger (d) \
@@ -370,7 +374,7 @@ static void sja1105_rxtstamp_work(struct work_struct *work)u64 ticks, ts;int rc; - rc =sja1105_ptpclkval_read (priv, &ticks); + rc =sja1105_ptpclkval_read (priv, &ticks,NULL );if (rc <0 ) {dev_err (ds->dev,"Failed to read PTP clock: %d\n" , rc);kfree_skb (skb);
@@ -441,8 +445,9 @@ int sja1105_ptp_reset(struct dsa_switch *ds)return rc; } -static int sja1105_ptp_gettime (struct ptp_clock_info *ptp, -struct timespec64 *ts) +static int sja1105_ptp_gettimex (struct ptp_clock_info *ptp, +struct timespec64 *ts, +struct ptp_system_timestamp *ptp_sts) {struct sja1105_ptp_data *ptp_data =ptp_caps_to_data (ptp);struct sja1105_private *priv =ptp_data_to_sja1105 (ptp_data);
@@ -451,7 +456,7 @@ static int sja1105_ptp_gettime(struct ptp_clock_info *ptp,mutex_lock (&ptp_data->lock); - rc =sja1105_ptpclkval_read (priv, &ticks); + rc =sja1105_ptpclkval_read (priv, &ticks, ptp_sts); *ts =ns_to_timespec64 (sja1105_ticks_to_ns (ticks));mutex_unlock (&ptp_data->lock);
@@ -516,7 +521,8 @@ static int sja1105_ptp_adjfine(struct ptp_clock_info *ptp, long scaled_ppm)mutex_lock (&ptp_data->lock); - rc =sja1105_xfer_u32 (priv,SPI_WRITE , regs->ptpclkrate, &clkrate32); + rc =sja1105_xfer_u32 (priv,SPI_WRITE , regs->ptpclkrate, &clkrate32, +NULL );mutex_unlock (&ptp_data->lock);
@@ -558,7 +564,7 @@ int sja1105_ptp_clock_register(struct dsa_switch *ds) .name = "SJA1105 PHC", .adjfine = sja1105_ptp_adjfine, .adjtime = sja1105_ptp_adjtime, - .gettime64 = sja1105_ptp_gettime, + .gettimex64 = sja1105_ptp_gettimex, .settime64 = sja1105_ptp_settime, .max_adj = SJA1105_MAX_ADJ_PPB, };
@@ -604,7 +610,7 @@ void sja1105_ptp_txtstamp_skb(struct dsa_switch *ds, int slot,mutex_lock (&ptp_data->lock); - rc =sja1105_ptpclkval_read (priv, &ticks); + rc =sja1105_ptpclkval_read (priv, &ticks,NULL );if (rc <0 ) {dev_err (ds->dev,"Failed to read PTP clock: %d\n" , rc);kfree_skb (skb);
drivers/net/dsa/sja1105/sja1105_spi.c +39 -9
108 lines not on this page · raw message
Signature · 1 line
2.17.1
Vladimir Oltean
maintainerPATCH 2/3 net: dsa: sja1105: Restore PTP time after switch reset
In reply to Vladimir OlteanThe PTP time of the switch is not preserved when uploading a new static configuration. Work around this hardware oddity by reading its PTP time before a static config upload, and restoring it afterwards.
Static config changes are expected to occur at runtime even in scenarios directly related to PTP, i.e. the Time-Aware Scheduler of the switch is programmed in this way.
Perhaps the larger implication of this patch is that the PTP .gettimex64 and .settime functions need to be exposed to sja1105_main.c, where the PTP lock needs to be held during this entire process. So their core implementation needs to move to some common functions which get exposed in sja1105_ptp.h.
--- drivers/net/dsa/sja1105/sja1105_main.c | 35 ++++++++++- drivers/net/dsa/sja1105/sja1105_ptp.c | 81 +++++++++++++++++++------- drivers/net/dsa/sja1105/sja1105_ptp.h | 24 +++++++- drivers/net/dsa/sja1105/sja1105_spi.c | 6 -- 4 files changed, 114 insertions(+), 32 deletions(-)
drivers/net/dsa/sja1105/sja1105_main.c +34 -1
@@ -1349,9 +1349,15 @@ static void sja1105_bridge_leave(struct dsa_switch *ds, int port,*/ int sja1105_static_config_reload (struct sja1105_private *priv) { +struct ptp_system_timestamp ptp_sts_before; +struct ptp_system_timestamp ptp_sts_after;struct sja1105_mac_config_entry *mac;int speed_mbps[SJA1105_NUM_PORTS ]; +struct dsa_switch *ds = priv->ds; +s64 t1, t2, t3, t4; +s64 t12, t34;int rc, i; +s64 now; mac = priv->static_config.tables[BLK_IDX_MAC_CONFIG ].entries;
@@ -1365,10 +1371,37 @@ int sja1105_static_config_reload(struct sja1105_private *priv) mac[i].speed =SJA1105_SPEED_AUTO ; } +/* No PTP operations can run right now */ +mutex_lock (&priv->ptp_data.lock); + + rc =__sja1105_ptp_gettimex (ds, &now, &ptp_sts_before); +if (rc <0 ) + goto out_unlock_ptp; +/* Reset switch and send updated static configuration */ rc =sja1105_static_config_upload (priv);if (rc <0 ) - goto out; + goto out_unlock_ptp; + + rc =__sja1105_ptp_settime (ds,0 , &ptp_sts_after); +if (rc <0 ) + goto out_unlock_ptp; + + t1 =timespec64_to_ns (&ptp_sts_before.pre_ts); + t2 =timespec64_to_ns (&ptp_sts_before.post_ts); + t3 =timespec64_to_ns (&ptp_sts_after.pre_ts); + t4 =timespec64_to_ns (&ptp_sts_after.post_ts); +/* Mid point, corresponds to pre-reset PTPCLKVAL */ + t12 = t1 + (t2 - t1) /2 ; +/* Mid point, corresponds to post-reset PTPCLKVAL, aka 0 */ + t34 = t3 + (t4 - t3) /2 ; +/* Advance PTPCLKVAL by the time it took since its readout */ + now += (t34 - t12); + +__sja1105_ptp_adjtime (ds, now); + +out_unlock_ptp: +mutex_unlock (&priv->ptp_data.lock);/* Configure the CGU (PLLs) for MII and RMII PHYs. * For these interfaces there is no dynamic configuration
drivers/net/dsa/sja1105/sja1105_ptp.c +59 -22
145 lines not on this page · raw message
drivers/net/dsa/sja1105/sja1105_ptp.h +21 -3
@@ -51,8 +51,6 @@ int sja1105_get_ts_info(struct dsa_switch *ds, int port,void sja1105_ptp_txtstamp_skb (struct dsa_switch *ds,int slot,struct sk_buff *clone); -int sja1105_ptp_reset (struct dsa_switch *ds); -bool sja1105_port_rxtstamp (struct dsa_switch *ds,int port,struct sk_buff *skb,unsigned int type);
@@ -63,6 +61,14 @@ int sja1105_hwtstamp_get(struct dsa_switch *ds, int port, struct ifreq *ifr);int sja1105_hwtstamp_set (struct dsa_switch *ds,int port,struct ifreq *ifr); +int __sja1105_ptp_gettimex (struct dsa_switch *ds,u64 *ns, +struct ptp_system_timestamp *sts); + +int __sja1105_ptp_settime (struct dsa_switch *ds,u64 ns, +struct ptp_system_timestamp *ptp_sts); + +int __sja1105_ptp_adjtime (struct dsa_switch *ds,s64 delta); +#else struct sja1105_ptp_cmd ;
@@ -87,7 +93,19 @@ static inline void sja1105_ptp_txtstamp_skb(struct dsa_switch *ds, int slot, { } -static inline int sja1105_ptp_reset (struct dsa_switch *ds) +static inline int __sja1105_ptp_gettimex (struct dsa_switch *ds,u64 *ns, +struct ptp_system_timestamp *sts) +{ +return 0 ; +} + +static inline int __sja1105_ptp_settime (struct dsa_switch *ds,u64 ns, +struct ptp_system_timestamp *ptp_sts) +{ +return 0 ; +} + +static inline int __sja1105_ptp_adjtime (struct dsa_switch *ds,s64 delta) {return 0 ; }
drivers/net/dsa/sja1105/sja1105_spi.c +0 -6
@@ -511,12 +511,6 @@ int sja1105_static_config_upload(struct sja1105_private *priv)dev_info (dev,"Succeeded after %d tried\n" ,RETRIES - retries); } - rc =sja1105_ptp_reset (priv->ds); -if (rc <0 ) -dev_err (dev,"Failed to reset PTP clock: %d\n" , rc); - -dev_info (dev,"Reset switch and programmed static config\n" ); - out:kfree (config_buf);return rc;
Signature · 1 line
2.17.1
Vladimir Oltean
maintainerPATCH 3/3 net: dsa: sja1105: Disallow management xmit during switch reset
In reply to Vladimir OlteanThe purpose here is to avoid ptp4l fail due to this condition:
timed out while polling for tx timestamp
increasing tx_timestamp_timeout may correct this issue, but it is likely caused by a driver bug port 1: send peer delay request failed
So either reset the switch before the management frame was sent, or after it was timestamped as well, but not in the middle.
The condition may arise either due to a true timeout (i.e. because re-uploading the static config takes time), or due to the TX timestamp actually getting lost due to reset. For the former we can increase tx_timestamp_timeout in userspace, for the latter we need this patch.
Locking all traffic during switch reset does not make sense at all, though. Forcing all CPU-originated traffic to potentially block waiting for a sleepable context to send > 800 bytes over SPI is not a good idea. Flows that are autonomously forwarded by the switch will get dropped anyway during switch reset no matter what. So just let all other CPU-originated traffic be dropped as well.
--- drivers/net/dsa/sja1105/sja1105_main.c | 4 ++++ 1 file changed, 4 insertions(+)
drivers/net/dsa/sja1105/sja1105_main.c +4 -0
@@ -1359,6 +1359,8 @@ int sja1105_static_config_reload(struct sja1105_private *priv)int rc, i;s64 now; +mutex_lock (&priv->mgmt_lock); + mac = priv->static_config.tables[BLK_IDX_MAC_CONFIG ].entries;/* Back up the dynamic link speed changed by sja1105_adjust_port_config
@@ -1417,6 +1419,8 @@ int sja1105_static_config_reload(struct sja1105_private *priv) goto out; } out: +mutex_unlock (&priv->mgmt_lock); +return rc; }
Signature · 1 line
2.17.1
Quoted 21 lines · Vladimir Oltean
The SJA1105 being an automotive switch means it is designed to live in a set-and-forget environment, far from the configure-at-runtime nature of Linux. Frequently resetting the switch to change its static config means it loses track of its PTP time, which is not good.
This patch series implements PTP system timestamping for this switch (using the API introduced for SPI here:
https://www.mail-archive.com/[email protected]/msg316725.html),
adding the following benefits to the driver:
- When under control of a user space PTP servo loop (ptp4l, phc2sys), the loss of sync during a switch reset is much more manageable, and the switch still remains in the s2 (locked servo) state.
- When synchronizing the switch using the software technique (based on reading clock A and writing the value to clock B, as opposed to relying on hardware timestamping), e.g. by using phc2sys, the sync accuracy is vastly improved due to the fact that the actual switch PTP time can now be more precisely correlated with something of better precision (CLOCK_REALTIME). The issue is that SPI transfers are inherently bad for measuring time with low jitter, but the newly introduced API aims to alleviate that issue somewhat.
Quoted 7 lines
This series is also a requirement for a future patch set that adds full time-aware scheduling offload support for the switch.
Vladimir Oltean (3):
net: dsa: sja1105: Implement the .gettimex64 system call for PTP
net: dsa: sja1105: Restore PTP time after switch reset
net: dsa: sja1105: Disallow management xmit during switch reset
For the series:
Vladimir Oltean
maintainerQuoted 26 lines · Richard Cochran
Quoted 24 lines · Vladimir Oltean
The SJA1105 being an automotive switch means it is designed to live in a set-and-forget environment, far from the configure-at-runtime nature of Linux. Frequently resetting the switch to change its static config means it loses track of its PTP time, which is not good.
This patch series implements PTP system timestamping for this switch (using the API introduced for SPI here:
https://www.mail-archive.com/[email protected]/msg316725.html),
adding the following benefits to the driver:
- When under control of a user space PTP servo loop (ptp4l, phc2sys), the loss of sync during a switch reset is much more manageable, and the switch still remains in the s2 (locked servo) state.
- When synchronizing the switch using the software technique (based on reading clock A and writing the value to clock B, as opposed to relying on hardware timestamping), e.g. by using phc2sys, the sync accuracy is vastly improved due to the fact that the actual switch PTP time can now be more precisely correlated with something of better precision (CLOCK_REALTIME). The issue is that SPI transfers are inherently bad for measuring time with low jitter, but the newly introduced API aims to alleviate that issue somewhat.This series is also a requirement for a future patch set that adds full time-aware scheduling offload support for the switch.
Quoted 8 lines
Quoted 4 lines
Vladimir Oltean (3):
net: dsa: sja1105: Implement the .gettimex64 system call for PTP
net: dsa: sja1105: Restore PTP time after switch reset
net: dsa: sja1105: Disallow management xmit during switch resetFor the series:
Acked-by Richard Cochran <[email protected]>
Thanks Richard.
David, I noticed you put these patches in Patchwork in the 'Needs Review / ACK' state. Do you need more in-depth review, or did you just miss Richard's tag, since Patchwork ignores them if they're posted on the cover letter?
[ in fact, I have a similar question for the "Accomodate DSA front-end into Ocelot" series, which also has 1 R and 1 A on the cover letter ]
Regards,
-Vladimir
David Miller
maintainerFrom: Vladimir Oltean <[email protected]>
Date: Mon, 11 Nov 2019 15:19:26 +0200
Quoted 4 lines
David, I noticed you put these patches in Patchwork in the 'Needs Review / ACK' state. Do you need more in-depth review, or did you just miss Richard's tag, since Patchwork ignores them if they're posted on the cover letter?
Nothing needs to happen, I will take care of this series and apply it.
Thanks.
David Miller
maintainerFrom: Vladimir Oltean <[email protected]>
Date: Sat, 9 Nov 2019 13:32:21 +0200
Quoted 21 lines
The SJA1105 being an automotive switch means it is designed to live in a set-and-forget environment, far from the configure-at-runtime nature of Linux. Frequently resetting the switch to change its static config means it loses track of its PTP time, which is not good.
This patch series implements PTP system timestamping for this switch (using the API introduced for SPI here:
https://www.mail-archive.com/[email protected]/msg316725.html),
adding the following benefits to the driver:
- When under control of a user space PTP servo loop (ptp4l, phc2sys), the loss of sync during a switch reset is much more manageable, and the switch still remains in the s2 (locked servo) state.
- When synchronizing the switch using the software technique (based on reading clock A and writing the value to clock B, as opposed to relying on hardware timestamping), e.g. by using phc2sys, the sync accuracy is vastly improved due to the fact that the actual switch PTP time can now be more precisely correlated with something of better precision (CLOCK_REALTIME). The issue is that SPI transfers are inherently bad for measuring time with low jitter, but the newly introduced API aims to alleviate that issue somewhat.
Quoted 2 lines
This series is also a requirement for a future patch set that adds full time-aware scheduling offload support for the switch.
Series applied, thank you.