Skip to contentSkip to conversation

x86: avoid HPET use on certain Intel platforms

PATCH In review Jan Beulich 3 participants 8 messages Nov 22, 2019 – Nov 25, 2019

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 x86: avoid HPET use on certain Intel platforms

Linux commit fc5db58539b49351e76f19817ed1102bf7c712d0 says

"Some Coffee Lake platforms have a skewed HPET timer once the SoCs entered
PC10, which in consequence marks TSC as unstable because HPET is used as watchdog clocksource for TSC."

Follow this for Xen as well. Looking at its patch context made me notice they have a pre-existing quirk for Bay Trail as well. The comment there, however, points at a Cherry Trail document. Looking at the datasheets of both, there appear to be similar issues, so go beyond Linux'es coverage and exclude both. Also key the disable on the PCI IDs of the actual affected devices, rather than those of 00:00.0.

Apply the workarounds only when the use of HPET was not explicitly requested on the command line and when use of (deep) C-states was not disabled.

Adjust a few types in touched or nearby code at the same time.

xen/arch/x86/time.c +34 -4
@@ -18,6 +18,7 @@
 #include <xen/timer.h>
 #include <xen/smp.h>
 #include <xen/irq.h>
+#include <xen/pci_ids.h>
 #include <xen/softirq.h>
 #include <xen/efi.h>
 #include <xen/cpuidle.h>
@@ -367,12 +368,41 @@ static u64 read_hpet_count(void)
     return hpet_read32(HPET_COUNTER);
 }
 
-static s64 __init init_hpet(struct platform_timesource *pts)
+static int64_t __init init_hpet(struct platform_timesource *pts)
 {
-    u64 hpet_rate = hpet_setup(), start;
-    u32 count, target;
+    uint64_t hpet_rate, start;
+    uint32_t count, target;
 
-    if ( hpet_rate == 0 )
+    if ( hpet_address && strcmp(opt_clocksource, pts->id) &&
+         cpuidle_using_deep_cstate() )
+    {
+        if ( pci_conf_read16(PCI_SBDF(0, 0, 0x1f, 0),
+                             PCI_VENDOR_ID) == PCI_VENDOR_ID_INTEL )
+            switch ( pci_conf_read16(PCI_SBDF(0, 0, 0x1f, 0), PCI_DEVICE_ID) )
+            {
+            /* HPET on Bay Trail platforms will halt in deep C states. */
+            case 0x0f1c:
+            /* HPET on Cherry Trail platforms will halt in deep C states. */
+            case 0x229c:
+                hpet_address = 0;
+                break;
+            }
+
+        /*
+         * Some Coffee Lake platforms have a skewed HPET timer once the SoCs
+         * entered PC10.
+         */
+        if ( pci_conf_read16(PCI_SBDF(0, 0, 0, 0),
+                             PCI_VENDOR_ID) == PCI_VENDOR_ID_INTEL &&
+             pci_conf_read16(PCI_SBDF(0, 0, 0, 0),
+                             PCI_DEVICE_ID) == 0x3ec4 )
+            hpet_address = 0;
+
+        if ( !hpet_address )
+            printk("Disabling HPET for being unreliable\n");
+    }
+
+    if ( (hpet_rate = hpet_setup()) == 0 )
         return 0;
 
     pts->frequency = hpet_rate;

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

Acked-by on v1 1/1
#1 lore raw
Quoted 13 lines · Jan Beulich

Linux commit fc5db58539b49351e76f19817ed1102bf7c712d0 says

"Some Coffee Lake platforms have a skewed HPET timer once the SoCs entered
PC10, which in consequence marks TSC as unstable because HPET is used as watchdog clocksource for TSC."

Follow this for Xen as well. Looking at its patch context made me notice they have a pre-existing quirk for Bay Trail as well. The comment there, however, points at a Cherry Trail document. Looking at the datasheets of both, there appear to be similar issues, so go beyond Linux'es coverage and exclude both. Also key the disable on the PCI IDs of the actual affected devices, rather than those of 00:00.0.

Quoted 5 lines

Apply the workarounds only when the use of HPET was not explicitly requested on the command line and when use of (deep) C-states was not disabled.

Adjust a few types in touched or nearby code at the same time.

Reported-by ?

Quoted 1 line

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#2 lore raw
Quoted 14 lines · Andrew Cooper
Quoted 13 lines · Jan Beulich

Linux commit fc5db58539b49351e76f19817ed1102bf7c712d0 says

"Some Coffee Lake platforms have a skewed HPET timer once the SoCs entered
PC10, which in consequence marks TSC as unstable because HPET is used as watchdog clocksource for TSC."

Follow this for Xen as well. Looking at its patch context made me notice they have a pre-existing quirk for Bay Trail as well. The comment there, however, points at a Cherry Trail document. Looking at the datasheets of both, there appear to be similar issues, so go beyond Linux'es coverage and exclude both. Also key the disable on the PCI IDs of the actual affected devices, rather than those of 00:00.0.

Quoted 7 lines
Quoted 5 lines

Apply the workarounds only when the use of HPET was not explicitly requested on the command line and when use of (deep) C-states was not disabled.

Adjust a few types in touched or nearby code at the same time.

Reported-by ?

The Linux commit has a Suggested-by, but no Reported-by. Do you want me to copy that one? Or else do you have any suggestion as to who the reporter was?

Quoted 3 lines
Quoted 1 line

Thanks.

Jan

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#3 lore raw
Quoted 19 lines · Jan Beulich
Quoted 18 lines · Andrew Cooper
Quoted 17 lines · Jan Beulich

Linux commit fc5db58539b49351e76f19817ed1102bf7c712d0 says

"Some Coffee Lake platforms have a skewed HPET timer once the SoCs entered
PC10, which in consequence marks TSC as unstable because HPET is used as watchdog clocksource for TSC."

Follow this for Xen as well. Looking at its patch context made me notice they have a pre-existing quirk for Bay Trail as well. The comment there, however, points at a Cherry Trail document. Looking at the datasheets of both, there appear to be similar issues, so go beyond Linux'es coverage and exclude both. Also key the disable on the PCI IDs of the actual affected devices, rather than those of 00:00.0.

Apply the workarounds only when the use of HPET was not explicitly requested on the command line and when use of (deep) C-states was not disabled.

Quoted 5 lines
Quoted 2 lines
Quoted 1 line

Adjust a few types in touched or nearby code at the same time.

Reported-by ?

The Linux commit has a Suggested-by, but no Reported-by. Do you want me to copy that one? Or else do you have any suggestion as to who the reporter was?

Well - this patch was identified by someone on xen-devel, which I presume was your basis for looking into it.

~Andrew

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#4 lore raw
Quoted 20 lines · Andrew Cooper
Quoted 19 lines · Jan Beulich
Quoted 18 lines · Andrew Cooper
Quoted 17 lines · Jan Beulich

Linux commit fc5db58539b49351e76f19817ed1102bf7c712d0 says

"Some Coffee Lake platforms have a skewed HPET timer once the SoCs entered
PC10, which in consequence marks TSC as unstable because HPET is used as watchdog clocksource for TSC."

Follow this for Xen as well. Looking at its patch context made me notice they have a pre-existing quirk for Bay Trail as well. The comment there, however, points at a Cherry Trail document. Looking at the datasheets of both, there appear to be similar issues, so go beyond Linux'es coverage and exclude both. Also key the disable on the PCI IDs of the actual affected devices, rather than those of 00:00.0.

Apply the workarounds only when the use of HPET was not explicitly requested on the command line and when use of (deep) C-states was not disabled.

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Quoted 1 line

Adjust a few types in touched or nearby code at the same time.

Reported-by ?

The Linux commit has a Suggested-by, but no Reported-by. Do you want me to copy that one? Or else do you have any suggestion as to who the reporter was?

Well - this patch was identified by someone on xen-devel, which I presume was your basis for looking into it.

No, it was me spotting the Linux commit in the stable 5.3 ChangeLog.

Jan

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#5 lore raw
In reply to Andrew Cooper
Quoted 9 lines · Andrew Cooper
Quoted 8 lines · Jan Beulich
Quoted 7 lines · Andrew Cooper
Quoted 6 lines · Jan Beulich

Linux commit fc5db58539b49351e76f19817ed1102bf7c712d0 says

"Some Coffee Lake platforms have a skewed HPET timer once the SoCs entered
PC10, which in consequence marks TSC as unstable because HPET is used as watchdog clocksource for TSC."

Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
Quoted 1 line

Adjust a few types in touched or nearby code at the same time.

Reported-by ?

The Linux commit has a Suggested-by, but no Reported-by. Do you want me to copy that one? Or else do you have any suggestion as to who the reporter was?

Well - this patch was identified by someone on xen-devel, which I presume was your basis for looking into it.

https://lists.xenproject.org/archives/html/xen-devel/2019-11/msg00662.html

BTW: Xeon E-2136 @ C242 has 8086:3eca as ID. One needs to check with Intel which combinations are really affected.

Regards Andreas

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#6 lore raw
Quoted 2 lines · Andreas Kinzler

BTW: Xeon E-2136 @ C242 has 8086:3eca as ID. One needs to check with Intel which combinations are really affected.

Are you saying you observed the same issue on such a (server processor) system as well? Neither its datasheet nor its specification update (which I specifically downloaded and looked through just because of your remark) have any mention of a similar issue. I also take it that the code comment inherited from Linux says "SoCs" for a reason.

Jan

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#7 lore raw
Quoted 5 lines · Jan Beulich
Quoted 2 lines · Andreas Kinzler

BTW: Xeon E-2136 @ C242 has 8086:3eca as ID. One needs to check with Intel which combinations are really affected.

Are you saying you observed the same issue on such a (server processor) system as well? Neither its datasheet nor its specification update

The whole thread starting with https://lists.xenproject.org/archives/html/xen-devel/2019-10/msg00966.html was about Xeon E-2136.

Setting a limit to PC7 greatly reduced the drift (https://lists.xenproject.org/archives/html/xen-devel/2019-11/msg01044.html)

Quoted 3 lines

(which I specifically downloaded and looked through just because of your remark) have any mention of a similar issue. I also take it that the code comment inherited from Linux says "SoCs" for a reason.

Even the kernel mailing list postings lack official confirmation from Intel. That is why I said: someone (with internal Intel knowledge) needs to confirm which combinations are affected.

Regards Andreas

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel