Support exposing raw cycle counters in PTP and mlx5
Subsystems (5)
Tariq Toukan
maintainerv2 cover Support exposing raw cycle counters in PTP and mlx5
Hi,
This series by Carolina adds support in ptp and usage in mlx5 for exposing the raw free-running cycle counter of PTP hardware clocks.
This is V2. Find previous one here:
https://lore.kernel.org/all/[email protected]/
Find detailed description by Carolina below [1].
Regards,
Tariq
V2:
- Extend the cover letter with more motivation and use cases.
[1]
This patch series introduces support for exposing the raw free-running cycle counter of PTP hardware clocks. When the device is in free-running mode, it emits timestamps as raw cycle values instead of nanoseconds. These values may be passed directly to user space through:
- fwctl: exposes internal device event records that include raw cycle-based timestamps.
- DPDK: retrieves CQEs that contain raw cycle counters, which are passed to user space unmodified.
To address this, the series introduces two new ioctl commands that allow userspace to query the device's raw cycle counter together with host time:
- PTP_SYS_OFFSET_PRECISE_CYCLES
- PTP_SYS_OFFSET_EXTENDED_CYCLES
These commands work like their existing counterparts but return the device timestamp in cycle units instead of real-time nanoseconds. This allows user space to collect (cycle, time) pairs and build a mapping between the device’s free-running clock and host time.
This can also be useful in the XDP fast path: if a driver inserts the raw cycle value into metadata instead of a real-time timestamp, it can avoid the overhead of converting cycles to time in the kernel. Then userspace can resolve the cycle-to-time mapping using this ioctl when needed.
The ioctl enables user space to correlate those with host time, without requiring the PHC to be synchronized, so long as the drift remains stable during collection.
Adds the new PTP ioctls and integrates support in ptp_ioctl():
- ptp: Add ioctl commands to expose raw cycle counter values
Support for exposing raw cycles in mlx5:
- net/mlx5: Extract MTCTR register read logic into helper function
- net/mlx5: Support getcyclesx and getcrosscycles
Carolina Jubran (3):
ptp: Add ioctl commands to expose raw cycle counter values net/mlx5: Extract MTCTR register read logic into helper function net/mlx5: Support getcyclesx and getcrosscycles
.../ethernet/mellanox/mlx5/core/lib/clock.c | 113 ++++++++++++++++-- drivers/ptp/ptp_chardev.c | 34 ++++-- include/uapi/linux/ptp_clock.h | 4 + 3 files changed, 130 insertions(+), 21 deletions(-)
base-commit: bc4c0a48bdad7f225740b8e750fdc1da6d85e1eb
Signature · 1 line
2.40.1
Tariq Toukan
maintainerPATCH v2 1/3 ptp: Add ioctl commands to expose raw cycle counter values
From: Carolina Jubran <[email protected]>
Introduce two new ioctl commands, PTP_SYS_OFFSET_PRECISE_CYCLES and PTP_SYS_OFFSET_EXTENDED_CYCLES, to allow user space to access the raw free-running cycle counter from PTP devices.
These ioctls are variants of the existing PRECISE and EXTENDED offset queries, but instead of returning device time in realtime, they return the raw cycle counter value.
--- drivers/ptp/ptp_chardev.c | 34 ++++++++++++++++++++++++++-------- include/uapi/linux/ptp_clock.h | 4 ++++ 2 files changed, 30 insertions(+), 8 deletions(-)
drivers/ptp/ptp_chardev.c +26 -8
@@ -285,17 +285,21 @@ static long ptp_enable_pps(struct ptp_clock *ptp, bool enable)return ops->enable (ops, &req, enable); } -static long ptp_sys_offset_precise (struct ptp_clock *ptp,void __user *arg) +typedef int (*ptp_crosststamp_fn )(struct ptp_clock_info *, +struct system_device_crosststamp *); + +static long ptp_sys_offset_precise (struct ptp_clock *ptp,void __user *arg, +ptp_crosststamp_fn crosststamp_fn) {struct ptp_sys_offset_precise precise_offset;struct system_device_crosststamp xtstamp;struct timespec64 ts;int err; -if (!ptp->info->getcrosststamp) +if (!crosststamp_fn)return -EOPNOTSUPP ; - err = ptp->info->getcrosststamp (ptp->info, &xtstamp); + err =crosststamp_fn (ptp->info, &xtstamp);if (err)return err;
@@ -313,12 +317,17 @@ static long ptp_sys_offset_precise(struct ptp_clock *ptp, void __user *arg)return copy_to_user (arg, &precise_offset,sizeof (precise_offset)) ? -EFAULT :0 ; } -static long ptp_sys_offset_extended (struct ptp_clock *ptp,void __user *arg) +typedef int (*ptp_gettimex_fn )(struct ptp_clock_info *, +struct timespec64 *, +struct ptp_system_timestamp *); + +static long ptp_sys_offset_extended (struct ptp_clock *ptp,void __user *arg, +ptp_gettimex_fn gettimex_fn) { struct ptp_sys_offset_extended *extoff __free(kfree) = NULL;struct ptp_system_timestamp sts; -if (!ptp->info->gettimex64) +if (!gettimex_fn)return -EOPNOTSUPP ; extoff =memdup_user (arg,sizeof (*extoff));
@@ -346,7 +355,7 @@ static long ptp_sys_offset_extended(struct ptp_clock *ptp, void __user *arg)struct timespec64 ts;int err; - err = ptp->info->gettimex64 (ptp->info, &ts, &sts); + err =gettimex_fn (ptp->info, &ts, &sts);if (err)return err;
@@ -497,11 +506,13 @@ long ptp_ioctl(struct posix_clock_context *pccontext, unsigned int cmd,case PTP_SYS_OFFSET_PRECISE :case PTP_SYS_OFFSET_PRECISE2 : -return ptp_sys_offset_precise (ptp, argptr); +return ptp_sys_offset_precise (ptp, argptr, + ptp->info->getcrosststamp);case PTP_SYS_OFFSET_EXTENDED :case PTP_SYS_OFFSET_EXTENDED2 : -return ptp_sys_offset_extended (ptp, argptr); +return ptp_sys_offset_extended (ptp, argptr, + ptp->info->gettimex64);case PTP_SYS_OFFSET :case PTP_SYS_OFFSET2 :
@@ -523,6 +534,13 @@ long ptp_ioctl(struct posix_clock_context *pccontext, unsigned int cmd,case PTP_MASK_EN_SINGLE :return ptp_mask_en_single (pccontext->private_clkdata, argptr); +case PTP_SYS_OFFSET_PRECISE_CYCLES : +return ptp_sys_offset_precise (ptp, argptr, + ptp->info->getcrosscycles); + +case PTP_SYS_OFFSET_EXTENDED_CYCLES : +return ptp_sys_offset_extended (ptp, argptr, + ptp->info->getcyclesx64);default :return -ENOTTY ; }
include/uapi/linux/ptp_clock.h +4 -0
@@ -245,6 +245,10 @@ struct ptp_pin_desc { _IOWR(PTP_CLK_MAGIC, 18, struct ptp_sys_offset_extended)#define PTP_MASK_CLEAR_ALL _IO(PTP_CLK_MAGIC, 19)#define PTP_MASK_EN_SINGLE _IOW(PTP_CLK_MAGIC, 20, unsigned int) +#define PTP_SYS_OFFSET_PRECISE_CYCLES \ + _IOWR(PTP_CLK_MAGIC, 21, struct ptp_sys_offset_precise) +#define PTP_SYS_OFFSET_EXTENDED_CYCLES \ + _IOWR(PTP_CLK_MAGIC, 22, struct ptp_sys_offset_extended)struct ptp_extts_event {struct ptp_clock_time t;/* Time event occurred. */
Signature · 1 line
2.40.1
Paolo Abeni
maintainerRe: [PATCH net-next V2 1/3] ptp: Add ioctl commands to expose raw cycle counter values
Quoted 6 lines · Tariq Toukan
From: Carolina Jubran <[email protected]>
Introduce two new ioctl commands,
PTP_SYS_OFFSET_PRECISE_CYCLESandPTP_SYS_OFFSET_EXTENDED_CYCLES, to allow user space to access the raw free-running cycle counter from PTP devices.
Quoted 7 lines
These ioctls are variants of the existing PRECISE and EXTENDED offset queries, but instead of returning device time in realtime, they return the raw cycle counter value.
Signed-off-by Carolina Jubran <[email protected]>Reviewed-by Dragos Tatulea <[email protected]>Signed-off-by Tariq Toukan <[email protected]>
Hi Richard,
can we have a formal ack here?
Thanks,
Paolo
Mark Bloch
maintainerRe: [PATCH net-next V2 1/3] ptp: Add ioctl commands to expose raw cycle counter values
Quoted 11 lines · Paolo Abeni
Quoted 10 lines · Tariq Toukan
From: Carolina Jubran <[email protected]>
Introduce two new ioctl commands,
PTP_SYS_OFFSET_PRECISE_CYCLESandPTP_SYS_OFFSET_EXTENDED_CYCLES, to allow user space to access the raw free-running cycle counter from PTP devices.These ioctls are variants of the existing PRECISE and EXTENDED offset queries, but instead of returning device time in realtime, they return the raw cycle counter value.
Quoted 7 lines
Quoted 3 lines
Signed-off-by Carolina Jubran <[email protected]>Reviewed-by Dragos Tatulea <[email protected]>Signed-off-by Tariq Toukan <[email protected]>Hi Richard,
can we have a formal ack here?
Gentle reminder.
Mark
Quoted 5 lines
Thanks,
Paolo
Richard Cochran
maintainerRe: [PATCH net-next V2 1/3] ptp: Add ioctl commands to expose raw cycle counter values
Quoted 6 lines · Mark Bloch
Quoted 1 line · Paolo Abeni
can we have a formal ack here?
Gentle reminder.
I'm travelling ATM. Please ping again next week.
Thanks,
Richard
Re: [PATCH net-next V2 1/3] ptp: Add ioctl commands to expose raw cycle counter values
Quoted 9 lines · Richard Cochran
Quoted 4 lines · Mark Bloch
Quoted 1 line · Paolo Abeni
can we have a formal ack here?
Gentle reminder.
I'm travelling ATM. Please ping again next week.
Thanks,
Richard
Hi Richard,
Just a kind reminder.
Thanks!
Carolina
Richard Cochran
maintainerRe: [PATCH net-next V2 1/3] ptp: Add ioctl commands to expose raw cycle counter values
Quoted 6 lines · Carolina Jubran
Quoted 4 lines · Richard Cochran
Quoted 3 lines · Mark Bloch
Quoted 1 line · Paolo Abeni
can we have a formal ack here?
Looks good to me.
Thanks,
Richard
Re: [PATCH net-next V2 1/3] ptp: Add ioctl commands to expose raw cycle counter values
Quoted 5 lines · Richard Cochran
Quoted 4 lines · Carolina Jubran
Quoted 2 lines · Richard Cochran
Quoted 1 line · Mark Bloch
Quoted 8 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines · Paolo Abeni
Quoted 1 line
can we have a formal ack here?
Looks good to me.
Thanks,
Richard
Hi Paolo,
A kind reminder, we got the needed ack here.
Yet patchwork state is still 'Needs ACK'.
Regards,
Tariq
Paolo Abeni
maintainerRe: [PATCH net-next V2 1/3] ptp: Add ioctl commands to expose raw cycle counter values
Quoted 11 lines · Tariq Toukan
Quoted 10 lines · Richard Cochran
Quoted 6 lines · Carolina Jubran
Quoted 4 lines · Richard Cochran
Quoted 3 lines · Mark Bloch
Quoted 1 line · Paolo Abeni
can we have a formal ack here?
Looks good to me.
Quoted 7 lines
Quoted 3 lines
Thanks,
RichardHi Paolo,
A kind reminder, we got the needed ack here.
Yet patchwork state is still 'Needs ACK'.
I was traveling in the past days.
I see Jakub has some perplexity WRT this series. I read that as "I prefer don't touch it" (as opposed to a "no-go") and I think that the feature makes sense.
I'm applying this now.
/P
Re: [PATCH net-next V2 1/3] ptp: Add ioctl commands to expose raw cycle counter values
In reply to Tariq ToukanQuoted 6 lines · Tariq Toukan
From: Carolina Jubran <[email protected]>
Introduce two new ioctl commands,
PTP_SYS_OFFSET_PRECISE_CYCLESandPTP_SYS_OFFSET_EXTENDED_CYCLES, to allow user space to access the raw free-running cycle counter from PTP devices.
Quoted 7 lines
These ioctls are variants of the existing PRECISE and EXTENDED offset queries, but instead of returning device time in realtime, they return the raw cycle counter value.
Signed-off-by Carolina Jubran <[email protected]>Reviewed-by Dragos Tatulea <[email protected]>Signed-off-by Tariq Toukan <[email protected]>
Tariq Toukan
maintainerPATCH v2 2/3 net/mlx5: Extract MTCTR register read logic into helper function
In reply to Tariq ToukanFrom: Carolina Jubran <[email protected]>
Refactor the MTCTR register reading logic into a dedicated helper to lay the groundwork for the next patch.
--- .../ethernet/mellanox/mlx5/core/lib/clock.c | 39 +++++++++++++------ 1 file changed, 27 insertions(+), 12 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/lib/clock.c +27 -12
@@ -247,27 +247,24 @@ static bool mlx5_is_ptm_source_time_available(struct mlx5_core_dev *dev)return !!MLX5_GET (mtptm_reg, out, psta); } -static int mlx5_mtctr_syncdevicetime (ktime_t *device_time, -struct system_counterval_t *sys_counterval, -void *ctx) +static int mlx5_mtctr_read (struct mlx5_core_dev *mdev, +bool real_time_mode, +struct system_counterval_t *sys_counterval, +u64 *device) {u32 out[MLX5_ST_SZ_DW (mtctr_reg)] = {0 };u32 in[MLX5_ST_SZ_DW (mtctr_reg)] = {0 }; -struct mlx5_core_dev *mdev = ctx; -bool real_time_mode; -u64 host, device; +u64 host;int err; - real_time_mode =mlx5_real_time_mode (mdev); -MLX5_SET (mtctr_reg, in, first_clock_timestamp_request,MLX5_MTCTR_REQUEST_PTM_ROOT_CLOCK );MLX5_SET (mtctr_reg, in, second_clock_timestamp_request, real_time_mode ?MLX5_MTCTR_REQUEST_REAL_TIME_CLOCK : -MLX5_MTCTR_REQUEST_FREE_RUNNING_COUNTER ); +MLX5_MTCTR_REQUEST_FREE_RUNNING_COUNTER ); - err =mlx5_core_access_reg (mdev, in,sizeof (in), out,sizeof (out),MLX5_REG_MTCTR , -0 ,0 ); + err =mlx5_core_access_reg (mdev, in,sizeof (in), out,sizeof (out), +MLX5_REG_MTCTR ,0 ,0 );if (err)return err;
@@ -281,8 +278,26 @@ static int mlx5_mtctr_syncdevicetime(ktime_t *device_time, .cs_id = CSID_X86_ART, .use_nsecs = true, }; + *device =MLX5_GET64 (mtctr_reg, out, second_clock_timestamp); + +return 0 ; +} + +static int mlx5_mtctr_syncdevicetime (ktime_t *device_time, +struct system_counterval_t *sys_counterval, +void *ctx) +{ +struct mlx5_core_dev *mdev = ctx; +bool real_time_mode; +u64 device; +int err; + + real_time_mode =mlx5_real_time_mode (mdev); + + err =mlx5_mtctr_read (mdev, real_time_mode, sys_counterval, &device); +if (err) +return err; - device =MLX5_GET64 (mtctr_reg, out, second_clock_timestamp);if (real_time_mode) *device_time =ns_to_ktime (REAL_TIME_TO_NS (device >>32 , device &U32_MAX ));else
Signature · 1 line
2.40.1
Tariq Toukan
maintainerPATCH v2 3/3 net/mlx5: Support getcyclesx and getcrosscycles
In reply to Tariq ToukanFrom: Carolina Jubran <[email protected]>
Implement the getcyclesx64 and getcrosscycles callbacks in ptp_info to expose the device’s raw free-running counter.
--- .../ethernet/mellanox/mlx5/core/lib/clock.c | 74 ++++++++++++++++++- 1 file changed, 73 insertions(+), 1 deletion(-)
drivers/net/ethernet/mellanox/mlx5/core/lib/clock.c +73 -1
@@ -306,6 +306,23 @@ static int mlx5_mtctr_syncdevicetime(ktime_t *device_time,return 0 ; } +static int +mlx5_mtctr_syncdevicecyclestime (ktime_t *device_time, +struct system_counterval_t *sys_counterval, +void *ctx) +{ +struct mlx5_core_dev *mdev = ctx; +u64 device; +int err; + + err =mlx5_mtctr_read (mdev, false, sys_counterval, &device); +if (err) +return err; + *device_time =ns_to_ktime (device); + +return 0 ; +} +static int mlx5_ptp_getcrosststamp (struct ptp_clock_info *ptp,struct system_device_crosststamp *cts) {
@@ -330,6 +347,32 @@ static int mlx5_ptp_getcrosststamp(struct ptp_clock_info *ptp,mlx5_clock_unlock (clock);return err; } + +static int mlx5_ptp_getcrosscycles (struct ptp_clock_info *ptp, +struct system_device_crosststamp *cts) +{ +struct mlx5_clock *clock = + container_of(ptp, struct mlx5_clock, ptp_info); +struct system_time_snapshot history_begin = {0 }; +struct mlx5_core_dev *mdev; +int err; + +mlx5_clock_lock (clock); + mdev =mlx5_clock_mdev_get (clock); + +if (!mlx5_is_ptm_source_time_available (mdev)) { + err = -EBUSY ; + goto unlock; + } + +ktime_get_snapshot (&history_begin); + + err =get_device_system_crosststamp (mlx5_mtctr_syncdevicecyclestime, + mdev, &history_begin, cts); +unlock: +mlx5_clock_unlock (clock); +return err; +}#endif /* CONFIG_X86 */ static u64 mlx5_read_time(struct mlx5_core_dev *dev,
@@ -528,6 +571,24 @@ static int mlx5_ptp_gettimex(struct ptp_clock_info *ptp, struct timespec64 *ts,return 0 ; } +static int mlx5_ptp_getcyclesx (struct ptp_clock_info *ptp, +struct timespec64 *ts, +struct ptp_system_timestamp *sts) +{ + struct mlx5_clock *clock = container_of(ptp, struct mlx5_clock, + ptp_info); +struct mlx5_core_dev *mdev; +u64 cycles; + +mlx5_clock_lock (clock); + mdev =mlx5_clock_mdev_get (clock); + + cycles =mlx5_read_time (mdev, sts, false); + *ts =ns_to_timespec64 (cycles); +mlx5_clock_unlock (clock); +return 0 ; +} +static int mlx5_ptp_adjtime_real_time (struct mlx5_core_dev *mdev,s64 delta) {u32 in[MLX5_ST_SZ_DW (mtutc_reg)] = {};
@@ -1244,6 +1305,7 @@ static void mlx5_init_timer_max_freq_adjustment(struct mlx5_core_dev *mdev)static void mlx5_init_timer_clock (struct mlx5_core_dev *mdev) {struct mlx5_clock *clock = mdev->clock; +bool expose_cycles;/* Configure the PHC */ clock->ptp_info = mlx5_ptp_clock_info;
@@ -1251,12 +1313,22 @@ static void mlx5_init_timer_clock(struct mlx5_core_dev *mdev)if (MLX5_CAP_MCAM_REG (mdev, mtutc))mlx5_init_timer_max_freq_adjustment (mdev); + expose_cycles = !MLX5_CAP_GEN (mdev, disciplined_fr_counter) || + !mlx5_real_time_mode (mdev); +#ifdef CONFIG_X86 if (MLX5_CAP_MCAM_REG3 (mdev, mtptm) && -MLX5_CAP_MCAM_REG3 (mdev, mtctr) &&boot_cpu_has (X86_FEATURE_ART )) +MLX5_CAP_MCAM_REG3 (mdev, mtctr) &&boot_cpu_has (X86_FEATURE_ART )) { clock->ptp_info.getcrosststamp = mlx5_ptp_getcrosststamp; +if (expose_cycles) + clock->ptp_info.getcrosscycles = + mlx5_ptp_getcrosscycles; + }#endif /* CONFIG_X86 */ +if (expose_cycles) + clock->ptp_info.getcyclesx64 = mlx5_ptp_getcyclesx; +mlx5_timecounter_init (mdev);mlx5_init_clock_info (mdev);mlx5_init_overflow_period (mdev);
Signature · 1 line
2.40.1
This series was applied to netdev/net-next.git (main) by Paolo Abeni <[email protected]>:
Hello:
This series was applied to netdev/net-next.git (main) by Paolo Abeni <[email protected]>:
Quoted 9 lines · On Tue, 12 Aug 2025 17:17:05 +0300 you wrote:
Hi,
This series by Carolina adds support in ptp and usage in mlx5 for exposing the raw free-running cycle counter of PTP hardware clocks.
This is V2. Find previous one here:
https://lore.kernel.org/all/[email protected]/[...]
Here is the summary with links:
- [net-next,V2,1/3] ptp: Add ioctl commands to expose raw cycle counter values
https://git.kernel.org/netdev/net-next/c/faf23f54d366 - [net-next,V2,2/3] net/mlx5: Extract MTCTR register read logic into helper function https://git.kernel.org/netdev/net-next/c/96c345c3c54c - [net-next,V2,3/3] net/mlx5: Support getcyclesx and getcrosscycles https://git.kernel.org/netdev/net-next/c/a3fb485505ca
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html