Use ixgbe_ptp_reset on linkup/linkdown for X550
Subsystems (3)
PATCH 1/1 Use ixgbe_ptp_reset on linkup/linkdown for X550
From: Steve Payne <[email protected]>
For an unknown reason, when ixgbe_ptp_start_cyclecounter is called from ixgbe_watchdog_link_is_down the PHC on the NIC jumps backward by a seemingly inconsistent amount, which causes discontinuities in time synchronization. Explicitly reset the NIC's PHC to CLOCK_REALTIME whenever the NIC goes up or down by calling ixgbe_ptp_reset instead of the bare ixgbe_ptp_start_cyclecounter.
--- drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +2 -2
@@ -7462,7 +7462,7 @@ static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter) adapter->last_rx_ptp_check = jiffies;if (test_bit (__IXGBE_PTP_RUNNING, &adapter->state)) -ixgbe_ptp_start_cyclecounter (adapter); +ixgbe_ptp_reset (adapter);switch (link_speed) {case IXGBE_LINK_SPEED_10GB_FULL :
@@ -7527,7 +7527,7 @@ static void ixgbe_watchdog_link_is_down(struct ixgbe_adapter *adapter) adapter->flags2 |=IXGBE_FLAG2_SEARCH_FOR_SFP ;if (test_bit (__IXGBE_PTP_RUNNING, &adapter->state)) -ixgbe_ptp_start_cyclecounter (adapter); +ixgbe_ptp_reset (adapter);e_info (drv,"NIC Link is Down\n" );netif_carrier_off (netdev);
Signature · 1 line
2.32.0
Quoted 20 lines
-----Original Message-----
From: [email protected] <[email protected]>
Sent: Monday, August 01, 2022 6:38 AM
To: Brandeburg, Jesse <[email protected]>; [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: [PATCH] Useixgbe_ptp_reseton linkup/linkdown for X550From: Steve Payne <[email protected]>
For an unknown reason, when
ixgbe_ptp_start_cyclecounteris called fromixgbe_watchdog_link_is_downthe PHC on the NIC jumps backward by a seemingly inconsistent amount, which causes discontinuities in time synchronization. Explicitly reset the NIC's PHC toCLOCK_REALTIMEwhenever the NIC goes up or down by callingixgbe_ptp_resetinstead of the bareixgbe_ptp_start_cyclecounter.
Quoted 3 lines
Signed-off-by Steve Payne <[email protected]>Signed-off-by Alison Chaiken <[email protected]>
Resetting PTP could be a problem if the clock was not being synchronized with the kernel CLOCK_REALTIME, and does result in some loss of timer precision either way due to the delays involved with setting the time.
Do you have an example of the clock jump? How much is it? How often is it? Every time? More information would help in order to debug what is going wrong here.
Thanks,
Jake
Quoted 31 lines
--- drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +1
-1@@ -7462,7 +7462,7 @@ static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter) adapter->last_rx_ptp_check = jiffies;if (test_bit (__IXGBE_PTP_RUNNING, &adapter->state)) -ixgbe_ptp_start_cyclecounter (adapter); +ixgbe_ptp_reset (adapter);switch (link_speed) {drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +1
-1case IXGBE_LINK_SPEED_10GB_FULL :@@ -7527,7 +7527,7 @@ static void ixgbe_watchdog_link_is_down(struct ixgbe_adapter *adapter) adapter->flags2 |= IXGBE_FLAG2_SEARCH_FOR_SFP;if (test_bit (__IXGBE_PTP_RUNNING, &adapter->state)) -ixgbe_ptp_start_cyclecounter (adapter); +ixgbe_ptp_reset (adapter);e_info (drv,"NIC Link is Down\n" );
netif_carrier_off(netdev);
--
2.32.0
A message that is not in the archive (sent off-list or not yet received): CAFzL-7tX845o2kJmE4o8EhbeD-=vkR6rmaiz_ZEWfSD4W+iWEA@mail.gmail.com
Quoted 30 lines · Alison Chaiken
---------- Forwarded message ---------
From: Keller, Jacob E <[email protected]>
Date: Mon, Aug 1, 2022 at 2:34 PM
Subject: RE: [PATCH] Useixgbe_ptp_reseton linkup/linkdown for X550
To: [email protected] <[email protected]>, Brandeburg, Jesse < [email protected]>, [email protected] < [email protected]>
Cc: [email protected] <[email protected]>, [email protected] < [email protected]>, [email protected] <[email protected]>, [email protected] <[email protected]>Quoted 18 lines
-----Original Message-----
From: [email protected] <[email protected]>
Sent: Monday, August 01, 2022 6:38 AM
To: Brandeburg, Jesse <[email protected]>; [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: [PATCH] Useixgbe_ptp_reseton linkup/linkdown for X550From: Steve Payne <[email protected]>
For an unknown reason, when
ixgbe_ptp_start_cyclecounteris called fromixgbe_watchdog_link_is_downthe PHC on the NIC jumps backward by a seemingly inconsistent amount, which causes discontinuities in time synchronization. Explicitly reset the NIC's PHC toCLOCK_REALTIMEwhenever the NIC goes up or down by callingixgbe_ptp_resetinstead of the bareixgbe_ptp_start_cyclecounter.
Quoted 7 lines
Quoted 3 lines
Signed-off-by Steve Payne <[email protected]>Signed-off-by Alison Chaiken <[email protected]>Resetting PTP could be a problem if the clock was not being synchronized with the kernel
CLOCK_REALTIME,
That is true, but most likely not really important, as the unmitigated problem also introduces significant discontinuities in time. Basically, this patch does not make things worse.
Quoted 3 lines
and does result in some loss of timer precision either way due to the delays involved with setting the time.
That precision loss is negligible compared to jumps resulting from link
down/up, and should be corrected by normal PTP operation very quickly.
Quoted 2 lines
Do you have an example of the clock jump? How much is it?
2021-02-12T09:24:37.741191+00:00 bench-12 phc2sys: [195230.451] CLOCK_REALTIME phc offset 61 s2 freq -36503 delay 2298 2021-02-12T09:24:38.741315+00:00 bench-12 phc2sys: [195231.451] CLOCK_REALTIME phc offset 169 s2 freq -36377 delay 2294 2021-02-12T09:24:39.741407+00:00 bench-12 phc2sys: [195232.451] CLOCK_REALTIME phc offset 195213702387037 s2 freq +100000000 delay 2301 2021-02-12T09:24:40.741489+00:00 bench-12 phc2sys: [195233.452]
CLOCK_REALTIME phc offset 195213591220495 s2 freq +100000000 delay 2081
Quoted 2 lines
How often is it? Every time?
Every time (though the specific amount differs, it is usually at similar magnitude)
More information would help in order to debug what is going wrong here.
Quoted 27 lines
Thanks,
JakeQuoted 23 lines
--- drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +1
-1@@ -7462,7 +7462,7 @@ static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter) adapter->last_rx_ptp_check = jiffies;if (test_bit (__IXGBE_PTP_RUNNING, &adapter->state)) -ixgbe_ptp_start_cyclecounter (adapter); +ixgbe_ptp_reset (adapter);switch (link_speed) {case
IXGBE_LINK_SPEED_10GB_FULL:drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +0
-0@@ -7527,7 +7527,7 @@ static void ixgbe_watchdog_link_is_down(struct ixgbe_adapter *adapter) adapter->flags2 |= IXGBE_FLAG2_SEARCH_FOR_SFP;
Quoted 10 lines
Quoted 8 lines
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +1
-1if (test_bit (__IXGBE_PTP_RUNNING, &adapter->state)) -ixgbe_ptp_start_cyclecounter (adapter); +ixgbe_ptp_reset (adapter);e_info (drv,"NIC Link is Down\n" );netif_carrier_off(netdev);--
2.32.0
_______________________________________________
Intel-wired-lan mailing list
[email protected]
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
PATCH 1/1 Use ixgbe_ptp_reset on linkup/linkdown for X550
Quoted 18 lines
Quoted 18 lines
-----Original Message-----
From: [email protected] <[email protected]>
Sent: Monday, August 01, 2022 6:38 AM
To: Brandeburg, Jesse <[email protected]>; [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: [PATCH] Useixgbe_ptp_reseton linkup/linkdown for X550From: Steve Payne <[email protected]>
For an unknown reason, when
ixgbe_ptp_start_cyclecounteris called fromixgbe_watchdog_link_is_downthe PHC on the NIC jumps backward by a seemingly inconsistent amount, which causes discontinuities in time synchronization. Explicitly reset the NIC's PHC toCLOCK_REALTIMEwhenever the NIC goes up or down by callingixgbe_ptp_resetinstead of the bareixgbe_ptp_start_cyclecounter.
Quoted 5 lines
Quoted 3 lines
Signed-off-by Steve Payne <[email protected]>Signed-off-by Alison Chaiken <[email protected]>Resetting PTP could be a problem if the clock was not being synchronized with the kernel
CLOCK_REALTIME,
That is true, but most likely not really important, as the unmitigated problem also introduces significant discontinuities in time. Basically, this patch does not make things worse.
Quoted 2 lines
and does result in some loss of timer precision either way due to the delays involved with setting the time.
That precision loss is negligible compared to jumps resulting from
link down/up, and should be corrected by normal PTP operation very quickly.
Quoted 2 lines
Do you have an example of the clock jump? How much is it?
2021-02-12T09:24:37.741191+00:00 bench-12 phc2sys: [195230.451] CLOCK_REALTIME phc offset 61 s2 freq -36503 delay 2298 2021-02-12T09:24:38.741315+00:00 bench-12 phc2sys: [195231.451] CLOCK_REALTIME phc offset 169 s2 freq -36377 delay 2294 2021-02-12T09:24:39.741407+00:00 bench-12 phc2sys: [195232.451] CLOCK_REALTIME phc offset 195213702387037 s2 freq +100000000 delay 2301 2021-02-12T09:24:40.741489+00:00 bench-12 phc2sys: [195233.452]
CLOCK_REALTIME phc offset 195213591220495 s2 freq +100000000 delay 2081
Quoted 2 lines
How often is it? Every time?
Every time (though the specific amount differs, it is usually at similar magnitude)
Quoted 37 lines
More information would help in order to debug what is going wrong here.
Thanks,
JakeQuoted 31 lines
--- drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +1
-1@@ -7462,7 +7462,7 @@ static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter) adapter->last_rx_ptp_check = jiffies;if (test_bit (__IXGBE_PTP_RUNNING, &adapter->state)) -ixgbe_ptp_start_cyclecounter (adapter); +ixgbe_ptp_reset (adapter);switch (link_speed) {case
IXGBE_LINK_SPEED_10GB_FULL:drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +1
-1@@ -7527,7 +7527,7 @@ static void ixgbe_watchdog_link_is_down(struct ixgbe_adapter *adapter) adapter->flags2 |= IXGBE_FLAG2_SEARCH_FOR_SFP;if (test_bit (__IXGBE_PTP_RUNNING, &adapter->state)) -ixgbe_ptp_start_cyclecounter (adapter); +ixgbe_ptp_reset (adapter);e_info (drv,"NIC Link is Down\n" );netif_carrier_off(netdev);--
2.32.0
Quoted 22 lines · Ilya Evenbach
Quoted 22 lines
Quoted 21 lines
-----Original Message-----
From: [email protected] <[email protected]>
Sent: Monday, August 01, 2022 6:38 AM
To: Brandeburg, Jesse <[email protected]>; [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: [PATCH] Useixgbe_ptp_reseton linkup/linkdown for X550From: Steve Payne <[email protected]>
For an unknown reason, when
ixgbe_ptp_start_cyclecounteris called fromixgbe_watchdog_link_is_downthe PHC on the NIC jumps backward by a seemingly inconsistent amount, which causes discontinuities in time synchronization. Explicitly reset the NIC's PHC toCLOCK_REALTIMEwhenever the NIC goes up or down by callingixgbe_ptp_resetinstead of the bareixgbe_ptp_start_cyclecounter.Signed-off-by Steve Payne <[email protected]>Signed-off-by Alison Chaiken <[email protected]>
Quoted 6 lines
Quoted 1 line
Resetting PTP could be a problem if the clock was not being synchronized with the kernel
CLOCK_REALTIME,That is true, but most likely not really important, as the unmitigated problem also introduces significant discontinuities in time. Basically, this patch does not make things worse.
Sure, but I am trying to see if I can understand *why* things get wonky. I suspect the issue is caused because of how we're resetting the cyclecounter.
Quoted 7 lines
Quoted 2 lines
and does result in some loss of timer precision either way due to the delays involved with setting the time.
That precision loss is negligible compared to jumps resulting from
link down/up, and should be corrected by normal PTP operation very quickly.
Only if CLOCK_REALTIME is actually being synchronized. Yes, that is generally true, but its not necessarily guaranteed.
Quoted 11 lines
Quoted 2 lines
Do you have an example of the clock jump? How much is it?
2021-02-12T09:24:37.741191+00:00 bench-12 phc2sys: [195230.451] CLOCK_REALTIME phc offset 61 s2 freq -36503 delay 2298 2021-02-12T09:24:38.741315+00:00 bench-12 phc2sys: [195231.451] CLOCK_REALTIME phc offset 169 s2 freq -36377 delay 2294 2021-02-12T09:24:39.741407+00:00 bench-12 phc2sys: [195232.451] CLOCK_REALTIME phc offset 195213702387037 s2 freq +100000000 delay 2301 2021-02-12T09:24:40.741489+00:00 bench-12 phc2sys: [195233.452]
Quoted 3 lines
CLOCK_REALTIMEphc offset 195213591220495 s2 freq +100000000 delay 2081
Thanks.
I think what's actually going on is a bug in the ixgbe_ptp_start_cyclecounter function where the system time registers are being reset.
What hardware are you operating on? Do you know if its an X550 board? It looks like this has been the case since a9763f3cb54c ("ixgbe: Update PTP to support X550EM_x devices").
The start_cyclecounter was never supposed to modify the current time registers, but resetting it to 0 as it does for X550 devices would give the exact behavior you're seeing.
_______________________________________________
Intel-wired-lan mailing list
[email protected]
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
Quoted 63 lines · Jacob Keller
Quoted 22 lines · Ilya Evenbach
Quoted 22 lines
Quoted 21 lines
-----Original Message-----
From: [email protected] <[email protected]>
Sent: Monday, August 01, 2022 6:38 AM
To: Brandeburg, Jesse <[email protected]>; [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: [PATCH] Useixgbe_ptp_reseton linkup/linkdown for X550From: Steve Payne <[email protected]>
For an unknown reason, when
ixgbe_ptp_start_cyclecounteris called fromixgbe_watchdog_link_is_downthe PHC on the NIC jumps backward by a seemingly inconsistent amount, which causes discontinuities in time synchronization. Explicitly reset the NIC's PHC toCLOCK_REALTIMEwhenever the NIC goes up or down by callingixgbe_ptp_resetinstead of the bareixgbe_ptp_start_cyclecounter.Signed-off-by Steve Payne <[email protected]>Signed-off-by Alison Chaiken <[email protected]>Quoted 6 lines
Quoted 1 line
Resetting PTP could be a problem if the clock was not being synchronized with the kernel
CLOCK_REALTIME,That is true, but most likely not really important, as the unmitigated problem also introduces significant discontinuities in time. Basically, this patch does not make things worse.
Sure, but I am trying to see if I can understand *why* things get wonky. I suspect the issue is caused because of how we're resetting the cyclecounter.
Quoted 7 lines
Quoted 2 lines
and does result in some loss of timer precision either way due to the delays involved with setting the time.
That precision loss is negligible compared to jumps resulting from
link down/up, and should be corrected by normal PTP operation very quickly.Only if
CLOCK_REALTIMEis actually being synchronized. Yes, that is generally true, but its not necessarily guaranteed.Quoted 11 lines
Quoted 2 lines
Do you have an example of the clock jump? How much is it?
2021-02-12T09:24:37.741191+00:00 bench-12 phc2sys: [195230.451] CLOCK_REALTIME phc offset 61 s2 freq -36503 delay 2298 2021-02-12T09:24:38.741315+00:00 bench-12 phc2sys: [195231.451] CLOCK_REALTIME phc offset 169 s2 freq -36377 delay 2294 2021-02-12T09:24:39.741407+00:00 bench-12 phc2sys: [195232.451] CLOCK_REALTIME phc offset 195213702387037 s2 freq +100000000 delay 2301 2021-02-12T09:24:40.741489+00:00 bench-12 phc2sys: [195233.452]Quoted 3 lines
CLOCK_REALTIMEphc offset 195213591220495 s2 freq +100000000 delay 2081
Quoted 7 lines
Thanks.
I think what's actually going on is a bug in the
ixgbe_ptp_start_cyclecounterfunction where the system time registers are being reset.What hardware are you operating on? Do you know if its an X550 board?
Indeed it is.
Quoted 3 lines
It
looks like this has been the case sincea9763f3cb54c("ixgbe: Update PTP to supportX550EM_xdevices").
The current test results come from v5.15.49-rt47. We observed the same problem in 5.4.93-rt51, which contains a9763f3cb54c.
Quoted 3 lines
The
start_cyclecounterwas never supposed to modify the current time registers, but resetting it to 0 as it does for X550 devices would give the exact behavior you're seeing.
That certainly sounds plausible.
Thanks,
Alison Chaiken
Aurora Innovation
Quoted 85 lines · Alison Chaiken
Quoted 63 lines · Jacob Keller
Quoted 22 lines · Ilya Evenbach
Quoted 22 lines
Quoted 21 lines
-----Original Message-----
From: [email protected] <[email protected]>
Sent: Monday, August 01, 2022 6:38 AM
To: Brandeburg, Jesse <[email protected]>; [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: [PATCH] Useixgbe_ptp_reseton linkup/linkdown for X550From: Steve Payne <[email protected]>
For an unknown reason, when
ixgbe_ptp_start_cyclecounteris called fromixgbe_watchdog_link_is_downthe PHC on the NIC jumps backward by a seemingly inconsistent amount, which causes discontinuities in time synchronization. Explicitly reset the NIC's PHC toCLOCK_REALTIMEwhenever the NIC goes up or down by callingixgbe_ptp_resetinstead of the bareixgbe_ptp_start_cyclecounter.Signed-off-by Steve Payne <[email protected]>Signed-off-by Alison Chaiken <[email protected]>Quoted 6 lines
Quoted 1 line
Resetting PTP could be a problem if the clock was not being synchronized with the kernel
CLOCK_REALTIME,That is true, but most likely not really important, as the unmitigated problem also introduces significant discontinuities in time. Basically, this patch does not make things worse.
Sure, but I am trying to see if I can understand *why* things get wonky. I suspect the issue is caused because of how we're resetting the cyclecounter.
Quoted 7 lines
Quoted 2 lines
and does result in some loss of timer precision either way due to the delays involved with setting the time.
That precision loss is negligible compared to jumps resulting from
link down/up, and should be corrected by normal PTP operation very quickly.Only if
CLOCK_REALTIMEis actually being synchronized. Yes, that is generally true, but its not necessarily guaranteed.Quoted 11 lines
Quoted 2 lines
Do you have an example of the clock jump? How much is it?
2021-02-12T09:24:37.741191+00:00 bench-12 phc2sys: [195230.451] CLOCK_REALTIME phc offset 61 s2 freq -36503 delay 2298 2021-02-12T09:24:38.741315+00:00 bench-12 phc2sys: [195231.451] CLOCK_REALTIME phc offset 169 s2 freq -36377 delay 2294 2021-02-12T09:24:39.741407+00:00 bench-12 phc2sys: [195232.451] CLOCK_REALTIME phc offset 195213702387037 s2 freq +100000000 delay 2301 2021-02-12T09:24:40.741489+00:00 bench-12 phc2sys: [195233.452]Quoted 3 lines
CLOCK_REALTIMEphc offset 195213591220495 s2 freq +100000000 delay 2081Quoted 7 lines
Thanks.
I think what's actually going on is a bug in the
ixgbe_ptp_start_cyclecounterfunction where the system time registers are being reset.What hardware are you operating on? Do you know if its an X550 board?
Indeed it is.
Quoted 3 lines
It
looks like this has been the case sincea9763f3cb54c("ixgbe: Update PTP to supportX550EM_xdevices").The current test results come from v5.15.49-rt47. We observed the same problem in 5.4.93-rt51, which contains
a9763f3cb54c.Quoted 3 lines
The
start_cyclecounterwas never supposed to modify the current time registers, but resetting it to 0 as it does for X550 devices would give the exact behavior you're seeing.
Quoted 5 lines
That certainly sounds plausible.
Thanks,
Alison Chaiken
Aurora Innovation
I just posted a fix which moves the SYSTIME clearing out of start_cyclecounter and into ixgbe_ptp_reset. I'm fairly confident that its the correct fix, based on the function comments. I think the implementor for the X550 simply didn't understand the separation of ixgbe_ptp_start_cyclecouter and ixgbe_ptp_reset.
Thanks,
Jake
Quoted 68 lines · Jacob Keller
Quoted 22 lines · Ilya Evenbach
Quoted 22 lines
Quoted 21 lines
-----Original Message-----
From: [email protected] <[email protected]>
Sent: Monday, August 01, 2022 6:38 AM
To: Brandeburg, Jesse <[email protected]>; [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: [PATCH] Useixgbe_ptp_reseton linkup/linkdown for X550From: Steve Payne <[email protected]>
For an unknown reason, when
ixgbe_ptp_start_cyclecounteris called fromixgbe_watchdog_link_is_downthe PHC on the NIC jumps backward by a seemingly inconsistent amount, which causes discontinuities in time synchronization. Explicitly reset the NIC's PHC toCLOCK_REALTIMEwhenever the NIC goes up or down by callingixgbe_ptp_resetinstead of the bareixgbe_ptp_start_cyclecounter.Signed-off-by Steve Payne <[email protected]>Signed-off-by Alison Chaiken <[email protected]>Quoted 6 lines
Quoted 1 line
Resetting PTP could be a problem if the clock was not being synchronized with the kernel
CLOCK_REALTIME,That is true, but most likely not really important, as the unmitigated problem also introduces significant discontinuities in time. Basically, this patch does not make things worse.
Sure, but I am trying to see if I can understand *why* things get wonky. I suspect the issue is caused because of how we're resetting the cyclecounter.
Quoted 7 lines
Quoted 2 lines
and does result in some loss of timer precision either way due to the delays involved with setting the time.
That precision loss is negligible compared to jumps resulting from
link down/up, and should be corrected by normal PTP operation very quickly.Only if
CLOCK_REALTIMEis actually being synchronized. Yes, that is generally true, but its not necessarily guaranteed.Quoted 11 lines
Quoted 2 lines
Do you have an example of the clock jump? How much is it?
2021-02-12T09:24:37.741191+00:00 bench-12 phc2sys: [195230.451] CLOCK_REALTIME phc offset 61 s2 freq -36503 delay 2298 2021-02-12T09:24:38.741315+00:00 bench-12 phc2sys: [195231.451] CLOCK_REALTIME phc offset 169 s2 freq -36377 delay 2294 2021-02-12T09:24:39.741407+00:00 bench-12 phc2sys: [195232.451] CLOCK_REALTIME phc offset 195213702387037 s2 freq +100000000 delay 2301 2021-02-12T09:24:40.741489+00:00 bench-12 phc2sys: [195233.452]Quoted 3 lines
CLOCK_REALTIMEphc offset 195213591220495 s2 freq +100000000 delay 2081Thanks.
I think what's actually going on is a bug in the
ixgbe_ptp_start_cyclecounterfunction where the system time registers are being reset.
Quoted 7 lines
What hardware are you operating on? Do you know if its an X550 board? It looks like this has been the case since
a9763f3cb54c("ixgbe: Update PTP to supportX550EM_xdevices").The
start_cyclecounterwas never supposed to modify the current time registers, but resetting it to 0 as it does for X550 devices would give the exact behavior you're seeing.
I just posted an alternative fix which I believe resolves this issue.
Thanks,
Jake
How about this Intel X550 PTP fix for 6.1?
https://lists.osuosl.org/pipermail/intel-wired-lan/Week-of-Mon-20220801/029590.html
Thanks,
Alison Chaiken
Aurora Innovation
Quoted 82 lines · Jacob Keller
Quoted 68 lines · Jacob Keller
Quoted 22 lines · Ilya Evenbach
Quoted 22 lines
Quoted 21 lines
-----Original Message-----
From: [email protected] <[email protected]>
Sent: Monday, August 01, 2022 6:38 AM
To: Brandeburg, Jesse <[email protected]>; [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: [PATCH] Useixgbe_ptp_reseton linkup/linkdown for X550From: Steve Payne <[email protected]>
For an unknown reason, when
ixgbe_ptp_start_cyclecounteris called fromixgbe_watchdog_link_is_downthe PHC on the NIC jumps backward by a seemingly inconsistent amount, which causes discontinuities in time synchronization. Explicitly reset the NIC's PHC toCLOCK_REALTIMEwhenever the NIC goes up or down by callingixgbe_ptp_resetinstead of the bareixgbe_ptp_start_cyclecounter.Signed-off-by Steve Payne <[email protected]>Signed-off-by Alison Chaiken <[email protected]>Quoted 6 lines
Quoted 1 line
Resetting PTP could be a problem if the clock was not being synchronized with the kernel
CLOCK_REALTIME,That is true, but most likely not really important, as the unmitigated problem also introduces significant discontinuities in time. Basically, this patch does not make things worse.
Sure, but I am trying to see if I can understand *why* things get wonky. I suspect the issue is caused because of how we're resetting the cyclecounter.
Quoted 7 lines
Quoted 2 lines
and does result in some loss of timer precision either way due to the delays involved with setting the time.
That precision loss is negligible compared to jumps resulting from
link down/up, and should be corrected by normal PTP operation very quickly.Only if
CLOCK_REALTIMEis actually being synchronized. Yes, that is generally true, but its not necessarily guaranteed.Quoted 11 lines
Quoted 2 lines
Do you have an example of the clock jump? How much is it?
2021-02-12T09:24:37.741191+00:00 bench-12 phc2sys: [195230.451] CLOCK_REALTIME phc offset 61 s2 freq -36503 delay 2298 2021-02-12T09:24:38.741315+00:00 bench-12 phc2sys: [195231.451] CLOCK_REALTIME phc offset 169 s2 freq -36377 delay 2294 2021-02-12T09:24:39.741407+00:00 bench-12 phc2sys: [195232.451] CLOCK_REALTIME phc offset 195213702387037 s2 freq +100000000 delay 2301 2021-02-12T09:24:40.741489+00:00 bench-12 phc2sys: [195233.452]Quoted 3 lines
CLOCK_REALTIMEphc offset 195213591220495 s2 freq +100000000 delay 2081Thanks.
I think what's actually going on is a bug in the
ixgbe_ptp_start_cyclecounterfunction where the system time registers are being reset.Quoted 7 lines
What hardware are you operating on? Do you know if its an X550 board? It looks like this has been the case since
a9763f3cb54c("ixgbe: Update PTP to supportX550EM_xdevices").The
start_cyclecounterwas never supposed to modify the current time registers, but resetting it to 0 as it does for X550 devices would give the exact behavior you're seeing.I just posted an alternative fix which I believe resolves this issue.
Thanks,
Jake
Quoted 8 lines · Alison Chaiken
How about this Intel X550 PTP fix for 6.1?
https://lists.osuosl.org/pipermail/intel-wired-lan/Week-of-Mon-20220801/029590.html
Thanks,
Alison Chaiken
Aurora Innovation
This was already submitted to net as [1], and it looks like it's already applied to the stable Linux 5.19, 5.15, and 5.10 stable trees.
I've confirmed that it's in the v6.0 tag as commit 25d7a5f5a6bb ("ixgbe:
stop resetting SYSTIME in ixgbe_ptp_start_cyclecounter")
[1]
https://lore.kernel.org/netdev/[email protected]/
Hope that helps!
Thanks,
Jake