text-tsx fails on Intel core 8th gen system
Hi,
I've noticed that tools/tests/tsx/test-tsx fails on a system with Intel Core i7-8750H. Specific error I get:
[user@dom0 tsx]$ ./test-tsx
TSX tests
Got 16 CPUs
Testing MSR_TSX_FORCE_ABORT consistency
CPU0 val 0
Testing MSR_TSX_CTRL consistency
Testing MSR_MCU_OPT_CTRL consistency
CPU0 val 0
Testing RTM behaviour
Got #UD
Host reports RTM, but appears unavailable
Testing PV default/max policies
Max: RTM 1, HLE 1, TSX_FORCE_ABORT 0, RTM_ALWAYS_ABORT 0, TSX_CTRL 0
Def: RTM 0, HLE 0, TSX_FORCE_ABORT 0, RTM_ALWAYS_ABORT 0, TSX_CTRL 0 HLE/RTM offered to guests despite not being available
Testing HVM default/max policies
Max: RTM 1, HLE 1, TSX_FORCE_ABORT 0, RTM_ALWAYS_ABORT 0, TSX_CTRL 0
Def: RTM 0, HLE 0, TSX_FORCE_ABORT 0, RTM_ALWAYS_ABORT 0, TSX_CTRL 0 HLE/RTM offered to guests despite not being available
Testing PV guest
Created d8
Cur: RTM 0, HLE 0, TSX_FORCE_ABORT 0, RTM_ALWAYS_ABORT 0, TSX_CTRL 0
Cur: RTM 1, HLE 1, TSX_FORCE_ABORT 0, RTM_ALWAYS_ABORT 0, TSX_CTRL 0
Testing HVM guest
Created d9
Cur: RTM 0, HLE 0, TSX_FORCE_ABORT 0, RTM_ALWAYS_ABORT 0, TSX_CTRL 0
Cur: RTM 1, HLE 1, TSX_FORCE_ABORT 0, RTM_ALWAYS_ABORT 0, TSX_CTRL 0
[user@dom0 tsx]$ echo $?
1
When I try it on a newer system (11th gen) then it works fine (exit code 0, just "Got #UD", no "Host reports RTM, but appears unavailable" line).
/proc/cpuinfo says:
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 158
model name : Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
stepping : 10
microcode : 0xf6
cpu MHz : 2207.990
cache size : 9216 KB
physical id : 0
siblings : 6
core id : 0
cpu cores : 1
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu de tsc msr pae mce cx8 apic sep mca cmov pat clflush acpi mmx fxsr sse sse2 ss ht syscall nx rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid tsc_known_freq pni pclmulqdq monitor est ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch cpuid_fault ssbd ibrs ibpb stibp fsgsbase bmi1 avx2 bmi2 erms rdseed adx clflushopt xsaveopt xsavec xgetbv1 md_clear arch_capabilities
bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit srbds mmio_stale_data retbleed
bogomips : 4415.98
clflush size : 64
cache_alignment : 64
address sizes : 39 bits physical, 48 bits virtual
power management:
...
Full xen-cpuid detail output attached.
Just in case, I'm attaching also full xl dmesg, but I don't see anything related there.
Signature · 3 lines
Best Regards, Marek Marczykowski-Górecki Invisible Things Lab
nr_features: 18
Static sets:
Known bfebfbff:fffef3ff:ee500800:2469bfff:0000000f:ffbfffff:3a405fdf:00000780:779fd205:fc91ef1c:00001c30:38000144:00000001:00000037:00000000:00040000:1fbeffff:00000000 [00] CPUID 0x00000001.edx fpu vme de pse tsc msr pae mce cx8 apic sysenter mtrr pge mca cmov pat pse36 clflush ds acpi mmx fxsr sse sse2 ss htt tm pbe [01] CPUID 0x00000001.ecx sse3 pclmulqdq dtes64 monitor ds-cpl vmx smx est tm2 ssse3 fma cx16 xtpr pdcm pcid dca sse41 sse42 x2apic movebe popcnt tsc-dl aesni xsave osxsave avx f16c rdrnd hyper [02] CPUID 0x80000001.edx syscall nx mmx+ fxsr+ pg1g rdtscp lm 3dnow+ 3dnow [03] CPUID 0x80000001.ecx lahf-lm cmp svm extapic cr8d lzcnt sse4a msse 3dnowpf osvw ibs xop skinit wdt lwp fma4 nodeid tbm topoext dbx monitorx [04] CPUID 0x0000000d:1.eax xsaveopt xsavec xgetbv1 xsaves [05] CPUID 0x00000007:0.ebx fsgsbase tsc-adj sgx bmi1 hle avx2 fdp-exn smep bmi2 erms invpcid rtm pqm depfpp mpx pqe avx512f avx512dq rdseed adx smap avx512-ifma clflushopt clwb proc-trace avx512pf avx512er avx512cd sha avx512bw avx512vl [06] CPUID 0x00000007:0.ecx prefetchwt1 avx512-vbmi umip pku ospke avx512-vbmi2 cet-ss gfni vaes vpclmulqdq avx512-vnni avx512-bitalg avx512-vpopcntdq rdpid cldemote movdiri movdir64b enqcmd [07] CPUID 0x80000007.edx hw-pstate itsc cpb efro [08] CPUID 0x80000008.ebx clzero rstr-fp-err-ptrs wbnoinvd ibpb ibrs amd-stibp ibrs-always stibp-always ibrs-fast ibrs-same-mode no-lmsl ppin amd-ssbd virt-ssbd ssb-no psfd btc-no ibpb-ret [09] CPUID 0x00000007:0.edx avx512-4vnniw avx512-4fmaps fsrm avx512-vp2intersect srbds-ctrl md-clear rtm-always-abort tsx-force-abort serialize hybrid tsxldtrk cet-ibt avx512-fp16 ibrsb stibp l1d-flush arch-caps core-caps ssbd [10] CPUID 0x00000007:1.eax avx-vnni avx512-bf16 fzrm fsrs fsrcs [11] CPUID 0x80000021.eax lfence+ nscb auto-ibrs sbpb ibpb-brtype srso-no [12] CPUID 0x00000007:1.ebx ppin [13] CPUID 0x00000007:2.edx intel-psfd ipred-ctrl rrsba-ctrl bhi-ctrl mcdt-no [14] CPUID 0x00000007:1.ecx [15] CPUID 0x00000007:1.edx cet-sss [16] MSR_ARCH_CAPS.lo rdcl-no eibrs rsba skip-l1dfl intel-ssb-no mds-no if-pschange-mc-no tsx-ctrl taa-no mcu-ctrl misc-pkg-ctrl energy-ctrl doitm sbdr-ssdp-no fbsdp-no psdp-no fb-clear fb-clear-ctrl rrsba bhi-no xapic-status ovrclk-status pbrsb-no gds-ctrl gds-no rfds-no rfds-clear [17] MSR_ARCH_CAPS.hi Special 10000200:c8200000:00000000:00000002:00000000:01002850:00000010:00000000:02000000:20000c00:00000000:00000000:00000000:00000000:00000000:00000000:100a0004:00000000 [00] CPUID 0x00000001.edx apic htt [01] CPUID 0x00000001.ecx x2apic osxsave rdrnd hyper [02] CPUID 0x80000001.edx [03] CPUID 0x80000001.ecx cmp [04] CPUID 0x0000000d:1.eax [05] CPUID 0x00000007:0.ebx hle fdp-exn rtm depfpp clwb [06] CPUID 0x00000007:0.ecx ospke [07] CPUID 0x80000007.edx [08] CPUID 0x80000008.ebx virt-ssbd [09] CPUID 0x00000007:0.edx md-clear rtm-always-abort arch-caps [10] CPUID 0x00000007:1.eax [11] CPUID 0x80000021.eax [12] CPUID 0x00000007:1.ebx [13] CPUID 0x00000007:2.edx [14] CPUID 0x00000007:1.ecx [15] CPUID 0x00000007:1.edx [16] MSR_ARCH_CAPS.lo rsba fb-clear rrsba rfds-clear [17] MSR_ARCH_CAPS.hi PV Max 1fc9cbf5:f6f83203:ea500800:042109e3:00000007:fdaf0b39:1a405f43:00000100:64001005:ac01451c:00001c30:38000044:00000000:00000021:00000000:00000000:1d12e173:00000000 [00] CPUID 0x00000001.edx fpu de tsc msr pae mce cx8 apic sysenter mca cmov pat clflush acpi mmx fxsr sse sse2 ss htt [01] CPUID 0x00000001.ecx sse3 pclmulqdq ssse3 fma cx16 sse41 sse42 x2apic movebe popcnt aesni xsave avx f16c rdrnd hyper [02] CPUID 0x80000001.edx syscall nx mmx+ fxsr+ rdtscp lm 3dnow+ 3dnow [03] CPUID 0x80000001.ecx lahf-lm cmp lzcnt sse4a msse 3dnowpf xop fma4 tbm dbx [04] CPUID 0x0000000d:1.eax xsaveopt xsavec xgetbv1 [05] CPUID 0x00000007:0.ebx fsgsbase bmi1 hle avx2 bmi2 erms rtm avx512f avx512dq rdseed adx avx512-ifma clflushopt clwb avx512pf avx512er avx512cd sha avx512bw avx512vl [06] CPUID 0x00000007:0.ecx prefetchwt1 avx512-vbmi avx512-vbmi2 gfni vaes vpclmulqdq avx512-vnni avx512-bitalg avx512-vpopcntdq rdpid cldemote movdiri movdir64b [07] CPUID 0x80000007.edx itsc [08] CPUID 0x80000008.ebx clzero rstr-fp-err-ptrs ibpb ssb-no btc-no ibpb-ret [09] CPUID 0x00000007:0.edx avx512-4vnniw avx512-4fmaps fsrm avx512-vp2intersect md-clear serialize tsxldtrk ibrsb stibp arch-caps ssbd [10] CPUID 0x00000007:1.eax avx-vnni avx512-bf16 fzrm fsrs fsrcs [11] CPUID 0x80000021.eax lfence+ nscb sbpb ibpb-brtype srso-no [12] CPUID 0x00000007:1.ebx [13] CPUID 0x00000007:2.edx intel-psfd mcdt-no [14] CPUID 0x00000007:1.ecx [15] CPUID 0x00000007:1.edx [16] MSR_ARCH_CAPS.lo rdcl-no eibrs intel-ssb-no mds-no if-pschange-mc-no taa-no sbdr-ssdp-no fbsdp-no psdp-no fb-clear bhi-no pbrsb-no gds-no rfds-no rfds-clear [17] MSR_ARCH_CAPS.hi PV Default 1fc9cbf5:f6f83203:ea500800:042109e3:00000007:fdaf0b29:02405f43:00000000:64001005:ac00441c:00001c30:38000044:00000000:00000021:00000000:00000000:1d12e173:00000000 [00] CPUID 0x00000001.edx fpu de tsc msr pae mce cx8 apic sysenter mca cmov pat clflush acpi mmx fxsr sse sse2 ss htt [01] CPUID 0x00000001.ecx sse3 pclmulqdq ssse3 fma cx16 sse41 sse42 x2apic movebe popcnt aesni xsave avx f16c rdrnd hyper [02] CPUID 0x80000001.edx syscall nx mmx+ fxsr+ rdtscp lm 3dnow+ 3dnow [03] CPUID 0x80000001.ecx lahf-lm cmp lzcnt sse4a msse 3dnowpf xop fma4 tbm dbx [04] CPUID 0x0000000d:1.eax xsaveopt xsavec xgetbv1 [05] CPUID 0x00000007:0.ebx fsgsbase bmi1 avx2 bmi2 erms rtm avx512f avx512dq rdseed adx avx512-ifma clflushopt clwb avx512pf avx512er avx512cd sha avx512bw avx512vl [06] CPUID 0x00000007:0.ecx prefetchwt1 avx512-vbmi avx512-vbmi2 gfni vaes vpclmulqdq avx512-vnni avx512-bitalg avx512-vpopcntdq rdpid cldemote [07] CPUID 0x80000007.edx [08] CPUID 0x80000008.ebx clzero rstr-fp-err-ptrs ibpb ssb-no btc-no ibpb-ret [09] CPUID 0x00000007:0.edx avx512-4vnniw avx512-4fmaps fsrm md-clear serialize ibrsb stibp arch-caps ssbd [10] CPUID 0x00000007:1.eax avx-vnni avx512-bf16 fzrm fsrs fsrcs [11] CPUID 0x80000021.eax lfence+ nscb sbpb ibpb-brtype srso-no [12] CPUID 0x00000007:1.ebx [13] CPUID 0x00000007:2.edx intel-psfd mcdt-no [14] CPUID 0x00000007:1.ecx [15] CPUID 0x00000007:1.edx [16] MSR_ARCH_CAPS.lo rdcl-no eibrs intel-ssb-no mds-no if-pschange-mc-no taa-no sbdr-ssdp-no fbsdp-no psdp-no fb-clear bhi-no pbrsb-no gds-no rfds-no rfds-clear [17] MSR_ARCH_CAPS.hi HVM Shadow Max 1fcbfbff:f7f83203:ea500800:042109f3:0000000f:fdbf4bbb:1a405f47:00000100:751fd005:bc01451c:00001c30:38000044:00000000:00000021:00000000:00000000:1d12e173:00000000 [00] CPUID 0x00000001.edx fpu vme de pse tsc msr pae mce cx8 apic sysenter mtrr pge mca cmov pat pse36 clflush acpi mmx fxsr sse sse2 ss htt [01] CPUID 0x00000001.ecx sse3 pclmulqdq ssse3 fma cx16 sse41 sse42 x2apic movebe popcnt tsc-dl aesni xsave avx f16c rdrnd hyper [02] CPUID 0x80000001.edx syscall nx mmx+ fxsr+ rdtscp lm 3dnow+ 3dnow [03] CPUID 0x80000001.ecx lahf-lm cmp cr8d lzcnt sse4a msse 3dnowpf xop fma4 tbm dbx [04] CPUID 0x0000000d:1.eax xsaveopt xsavec xgetbv1 xsaves [05] CPUID 0x00000007:0.ebx fsgsbase tsc-adj bmi1 hle avx2 smep bmi2 erms rtm mpx avx512f avx512dq rdseed adx smap avx512-ifma clflushopt clwb avx512pf avx512er avx512cd sha avx512bw avx512vl [06] CPUID 0x00000007:0.ecx prefetchwt1 avx512-vbmi umip avx512-vbmi2 gfni vaes vpclmulqdq avx512-vnni avx512-bitalg avx512-vpopcntdq rdpid cldemote movdiri movdir64b [07] CPUID 0x80000007.edx itsc [08] CPUID 0x80000008.ebx clzero rstr-fp-err-ptrs ibpb ibrs amd-stibp ibrs-always stibp-always ibrs-fast ibrs-same-mode no-lmsl amd-ssbd ssb-no psfd btc-no ibpb-ret [09] CPUID 0x00000007:0.edx avx512-4vnniw avx512-4fmaps fsrm avx512-vp2intersect md-clear serialize tsxldtrk ibrsb stibp l1d-flush arch-caps ssbd [10] CPUID 0x00000007:1.eax avx-vnni avx512-bf16 fzrm fsrs fsrcs [11] CPUID 0x80000021.eax lfence+ nscb sbpb ibpb-brtype srso-no [12] CPUID 0x00000007:1.ebx [13] CPUID 0x00000007:2.edx intel-psfd mcdt-no [14] CPUID 0x00000007:1.ecx [15] CPUID 0x00000007:1.edx [16] MSR_ARCH_CAPS.lo rdcl-no eibrs intel-ssb-no mds-no if-pschange-mc-no taa-no sbdr-ssdp-no fbsdp-no psdp-no fb-clear bhi-no pbrsb-no gds-no rfds-no rfds-clear [17] MSR_ARCH_CAPS.hi HVM Shadow Default 1fcbfbff:f7f83203:ea500800:042109f3:0000000f:fdbf0bab:02405f47:00000000:751fd005:bc00441c:00001c30:38000044:00000000:00000021:00000000:00000000:1d12e173:00000000 [00] CPUID 0x00000001.edx fpu vme de pse tsc msr pae mce cx8 apic sysenter mtrr pge mca cmov pat pse36 clflush acpi mmx fxsr sse sse2 ss htt [01] CPUID 0x00000001.ecx sse3 pclmulqdq ssse3 fma cx16 sse41 sse42 x2apic movebe popcnt tsc-dl aesni xsave avx f16c rdrnd hyper [02] CPUID 0x80000001.edx syscall nx mmx+ fxsr+ rdtscp lm 3dnow+ 3dnow [03] CPUID 0x80000001.ecx lahf-lm cmp cr8d lzcnt sse4a msse 3dnowpf xop fma4 tbm dbx [04] CPUID 0x0000000d:1.eax xsaveopt xsavec xgetbv1 xsaves [05] CPUID 0x00000007:0.ebx fsgsbase tsc-adj bmi1 avx2 smep bmi2 erms rtm avx512f avx512dq rdseed adx smap avx512-ifma clflushopt clwb avx512pf avx512er avx512cd sha avx512bw avx512vl [06] CPUID 0x00000007:0.ecx prefetchwt1 avx512-vbmi umip avx512-vbmi2 gfni vaes vpclmulqdq avx512-vnni avx512-bitalg avx512-vpopcntdq rdpid cldemote [07] CPUID 0x80000007.edx [08] CPUID 0x80000008.ebx clzero rstr-fp-err-ptrs ibpb ibrs amd-stibp ibrs-always stibp-always ibrs-fast ibrs-same-mode no-lmsl amd-ssbd ssb-no psfd btc-no ibpb-ret [09] CPUID 0x00000007:0.edx avx512-4vnniw avx512-4fmaps fsrm md-clear serialize ibrsb stibp l1d-flush arch-caps ssbd [10] CPUID 0x00000007:1.eax avx-vnni avx512-bf16 fzrm fsrs fsrcs [11] CPUID 0x80000021.eax lfence+ nscb sbpb ibpb-brtype srso-no [12] CPUID 0x00000007:1.ebx [13] CPUID 0x00000007:2.edx intel-psfd mcdt-no [14] CPUID 0x00000007:1.ecx [15] CPUID 0x00000007:1.edx [16] MSR_ARCH_CAPS.lo rdcl-no eibrs intel-ssb-no mds-no if-pschange-mc-no taa-no sbdr-ssdp-no fbsdp-no psdp-no fb-clear bhi-no pbrsb-no gds-no rfds-no rfds-clear [17] MSR_ARCH_CAPS.hi HVM Hap Max 1fcbfbff:f7fa3223:ee500800:042109f7:0000000f:fdbf4fbb:1a405f4f:00000100:751fd005:bc01451c:00001c30:38000044:00000000:00000021:00000000:00000000:1d12e173:00000000 [00] CPUID 0x00000001.edx fpu vme de pse tsc msr pae mce cx8 apic sysenter mtrr pge mca cmov pat pse36 clflush acpi mmx fxsr sse sse2 ss htt [01] CPUID 0x00000001.ecx sse3 pclmulqdq vmx ssse3 fma cx16 pcid sse41 sse42 x2apic movebe popcnt tsc-dl aesni xsave avx f16c rdrnd hyper [02] CPUID 0x80000001.edx syscall nx mmx+ fxsr+ pg1g rdtscp lm 3dnow+ 3dnow [03] CPUID 0x80000001.ecx lahf-lm cmp svm cr8d lzcnt sse4a msse 3dnowpf xop fma4 tbm dbx [04] CPUID 0x0000000d:1.eax xsaveopt xsavec xgetbv1 xsaves [05] CPUID 0x00000007:0.ebx fsgsbase tsc-adj bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx avx512f avx512dq rdseed adx smap avx512-ifma clflushopt clwb avx512pf avx512er avx512cd sha avx512bw avx512vl [06] CPUID 0x00000007:0.ecx prefetchwt1 avx512-vbmi umip pku avx512-vbmi2 gfni vaes vpclmulqdq avx512-vnni avx512-bitalg avx512-vpopcntdq rdpid cldemote movdiri movdir64b [07] CPUID 0x80000007.edx itsc [08] CPUID 0x80000008.ebx clzero rstr-fp-err-ptrs ibpb ibrs amd-stibp ibrs-always stibp-always ibrs-fast ibrs-same-mode no-lmsl amd-ssbd ssb-no psfd btc-no ibpb-ret [09] CPUID 0x00000007:0.edx avx512-4vnniw avx512-4fmaps fsrm avx512-vp2intersect md-clear serialize tsxldtrk ibrsb stibp l1d-flush arch-caps ssbd [10] CPUID 0x00000007:1.eax avx-vnni avx512-bf16 fzrm fsrs fsrcs [11] CPUID 0x80000021.eax lfence+ nscb sbpb ibpb-brtype srso-no [12] CPUID 0x00000007:1.ebx [13] CPUID 0x00000007:2.edx intel-psfd mcdt-no [14] CPUID 0x00000007:1.ecx [15] CPUID 0x00000007:1.edx [16] MSR_ARCH_CAPS.lo rdcl-no eibrs intel-ssb-no mds-no if-pschange-mc-no taa-no sbdr-ssdp-no fbsdp-no psdp-no fb-clear bhi-no pbrsb-no gds-no rfds-no rfds-clear [17] MSR_ARCH_CAPS.hi HVM Hap Default 1fcbfbff:f7fa3203:ee500800:042109f3:0000000f:fdbf0fab:02405f4f:00000000:751fd005:bc00441c:00001c30:38000044:00000000:00000021:00000000:00000000:1d12e173:00000000 [00] CPUID 0x00000001.edx fpu vme de pse tsc msr pae mce cx8 apic sysenter mtrr pge mca cmov pat pse36 clflush acpi mmx fxsr sse sse2 ss htt [01] CPUID 0x00000001.ecx sse3 pclmulqdq ssse3 fma cx16 pcid sse41 sse42 x2apic movebe popcnt tsc-dl aesni xsave avx f16c rdrnd hyper [02] CPUID 0x80000001.edx syscall nx mmx+ fxsr+ pg1g rdtscp lm 3dnow+ 3dnow [03] CPUID 0x80000001.ecx lahf-lm cmp cr8d lzcnt sse4a msse 3dnowpf xop fma4 tbm dbx [04] CPUID 0x0000000d:1.eax xsaveopt xsavec xgetbv1 xsaves [05] CPUID 0x00000007:0.ebx fsgsbase tsc-adj bmi1 avx2 smep bmi2 erms invpcid rtm avx512f avx512dq rdseed adx smap avx512-ifma clflushopt clwb avx512pf avx512er avx512cd sha avx512bw avx512vl [06] CPUID 0x00000007:0.ecx prefetchwt1 avx512-vbmi umip pku avx512-vbmi2 gfni vaes vpclmulqdq avx512-vnni avx512-bitalg avx512-vpopcntdq rdpid cldemote [07] CPUID 0x80000007.edx [08] CPUID 0x80000008.ebx clzero rstr-fp-err-ptrs ibpb ibrs amd-stibp ibrs-always stibp-always ibrs-fast ibrs-same-mode no-lmsl amd-ssbd ssb-no psfd btc-no ibpb-ret [09] CPUID 0x00000007:0.edx avx512-4vnniw avx512-4fmaps fsrm md-clear serialize ibrsb stibp l1d-flush arch-caps ssbd [10] CPUID 0x00000007:1.eax avx-vnni avx512-bf16 fzrm fsrs fsrcs [11] CPUID 0x80000021.eax lfence+ nscb sbpb ibpb-brtype srso-no
[12] CPUID 0x00000007:1.ebx
[13] CPUID 0x00000007:2.edx intel-psfd mcdt-no
[14] CPUID 0x00000007:1.ecx
[15] CPUID 0x00000007:1.edx
[16] MSR_ARCH_CAPS.lo rdcl-no eibrs intel-ssb-no mds-no if-pschange-mc-no taa-no sbdr-ssdp-no fbsdp-no psdp-no fb-clear bhi-no pbrsb-no gds-no rfds-no rfds-clear
[17] MSR_ARCH_CAPS.hi
Dynamic sets:
Raw bfebfbff:7ffafbbf:2c100800:00000121:0000000f:029c67af:40000000:00000100:00000000:bc002e00:00000000:00000000:00000000:00000000:00000000:00000000:02000c04:00000000 [00] CPUID 0x00000001.edx fpu vme de pse tsc msr pae mce cx8 apic sysenter mtrr pge mca cmov pat pse36 clflush ds acpi mmx fxsr sse sse2 ss htt tm pbe [01] CPUID 0x00000001.ecx sse3 pclmulqdq dtes64 monitor ds-cpl vmx est tm2 ssse3 sdgb fma cx16 xtpr pdcm pcid sse41 sse42 x2apic movebe popcnt tsc-dl aesni xsave osxsave avx f16c rdrnd [02] CPUID 0x80000001.edx syscall nx pg1g rdtscp lm [03] CPUID 0x80000001.ecx lahf-lm lzcnt 3dnowpf [04] CPUID 0x0000000d:1.eax xsaveopt xsavec xgetbv1 xsaves [05] CPUID 0x00000007:0.ebx fsgsbase tsc-adj sgx bmi1 avx2 smep bmi2 erms invpcid depfpp mpx rdseed adx smap clflushopt proc-trace [06] CPUID 0x00000007:0.ecx sgx-lc [07] CPUID 0x80000007.edx itsc [08] CPUID 0x80000008.ebx [09] CPUID 0x00000007:0.edx srbds-ctrl md-clear rtm-always-abort tsx-force-abort ibrsb stibp l1d-flush arch-caps ssbd [10] CPUID 0x00000007:1.eax [11] CPUID 0x80000021.eax [12] CPUID 0x00000007:1.ebx [13] CPUID 0x00000007:2.edx [14] CPUID 0x00000007:1.ecx [15] CPUID 0x00000007:1.edx [16] MSR_ARCH_CAPS.lo rsba misc-pkg-ctrl energy-ctrl gds-ctrl [17] MSR_ARCH_CAPS.hi Host bfebfbff:77faf3bf:2c100800:00000121:0000000f:029c6fbf:00000000:00000100:00000000:bc002e00:00000000:00000000:00000000:00000000:00000000:00000000:0e000c04:00000000 [00] CPUID 0x00000001.edx fpu vme de pse tsc msr pae mce cx8 apic sysenter mtrr pge mca cmov pat pse36 clflush ds acpi mmx fxsr sse sse2 ss htt tm pbe [01] CPUID 0x00000001.ecx sse3 pclmulqdq dtes64 monitor ds-cpl vmx est tm2 ssse3 fma cx16 xtpr pdcm pcid sse41 sse42 x2apic movebe popcnt tsc-dl aesni xsave avx f16c rdrnd [02] CPUID 0x80000001.edx syscall nx pg1g rdtscp lm [03] CPUID 0x80000001.ecx lahf-lm lzcnt 3dnowpf [04] CPUID 0x0000000d:1.eax xsaveopt xsavec xgetbv1 xsaves [05] CPUID 0x00000007:0.ebx fsgsbase tsc-adj sgx bmi1 hle avx2 smep bmi2 erms invpcid rtm depfpp mpx rdseed adx smap clflushopt proc-trace [06] CPUID 0x00000007:0.ecx [07] CPUID 0x80000007.edx itsc [08] CPUID 0x80000008.ebx [09] CPUID 0x00000007:0.edx srbds-ctrl md-clear rtm-always-abort tsx-force-abort ibrsb stibp l1d-flush arch-caps ssbd [10] CPUID 0x00000007:1.eax [11] CPUID 0x80000021.eax [12] CPUID 0x00000007:1.ebx [13] CPUID 0x00000007:2.edx [14] CPUID 0x00000007:1.ecx [15] CPUID 0x00000007:1.edx [16] MSR_ARCH_CAPS.lo rsba misc-pkg-ctrl energy-ctrl gds-ctrl gds-no rfds-no [17] MSR_ARCH_CAPS.hi PV Default 1fc9cbf5:f6f83203:28100800:00000121:00000007:008c0329:00000000:00000000:00001000:ac000400:00000000:00000000:00000000:00000000:00000000:00000000:0c000004:00000000 [00] CPUID 0x00000001.edx fpu de tsc msr pae mce cx8 apic sysenter mca cmov pat clflush acpi mmx fxsr sse sse2 ss htt [01] CPUID 0x00000001.ecx sse3 pclmulqdq ssse3 fma cx16 sse41 sse42 x2apic movebe popcnt aesni xsave avx f16c rdrnd hyper [02] CPUID 0x80000001.edx syscall nx rdtscp lm [03] CPUID 0x80000001.ecx lahf-lm lzcnt 3dnowpf [04] CPUID 0x0000000d:1.eax xsaveopt xsavec xgetbv1 [05] CPUID 0x00000007:0.ebx fsgsbase bmi1 avx2 bmi2 erms rdseed adx clflushopt [06] CPUID 0x00000007:0.ecx [07] CPUID 0x80000007.edx [08] CPUID 0x80000008.ebx ibpb [09] CPUID 0x00000007:0.edx md-clear ibrsb stibp arch-caps ssbd [10] CPUID 0x00000007:1.eax [11] CPUID 0x80000021.eax [12] CPUID 0x00000007:1.ebx [13] CPUID 0x00000007:2.edx [14] CPUID 0x00000007:1.ecx [15] CPUID 0x00000007:1.edx [16] MSR_ARCH_CAPS.lo rsba gds-no rfds-no [17] MSR_ARCH_CAPS.hi HVM Default 1fcbfbff:f7fa3203:2c100800:00000121:0000000f:009c07ab:00000000:00000000:00101000:bc000400:00000000:00000000:00000000:00000000:00000000:00000000:0c000004:00000000 [00] CPUID 0x00000001.edx fpu vme de pse tsc msr pae mce cx8 apic sysenter mtrr pge mca cmov pat pse36 clflush acpi mmx fxsr sse sse2 ss htt [01] CPUID 0x00000001.ecx sse3 pclmulqdq ssse3 fma cx16 pcid sse41 sse42 x2apic movebe popcnt tsc-dl aesni xsave avx f16c rdrnd hyper [02] CPUID 0x80000001.edx syscall nx pg1g rdtscp lm [03] CPUID 0x80000001.ecx lahf-lm lzcnt 3dnowpf [04] CPUID 0x0000000d:1.eax xsaveopt xsavec xgetbv1 xsaves [05] CPUID 0x00000007:0.ebx fsgsbase tsc-adj bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt [06] CPUID 0x00000007:0.ecx [07] CPUID 0x80000007.edx [08] CPUID 0x80000008.ebx ibpb no-lmsl [09] CPUID 0x00000007:0.edx md-clear ibrsb stibp l1d-flush arch-caps ssbd [10] CPUID 0x00000007:1.eax [11] CPUID 0x80000021.eax [12] CPUID 0x00000007:1.ebx [13] CPUID 0x00000007:2.edx [14] CPUID 0x00000007:1.ecx [15] CPUID 0x00000007:1.edx [16] MSR_ARCH_CAPS.lo rsba gds-no rfds-no [17] MSR_ARCH_CAPS.hi PV Max 1fc9cbf5:f6f83203:28100800:00000121:00000007:008c0b39:00000000:00000100:00001000:ac000400:00000000:00000000:00000000:00000000:00000000:00000000:1c020004:00000000 [00] CPUID 0x00000001.edx fpu de tsc msr pae mce cx8 apic sysenter mca cmov pat clflush acpi mmx fxsr sse sse2 ss htt [01] CPUID 0x00000001.ecx sse3 pclmulqdq ssse3 fma cx16 sse41 sse42 x2apic movebe popcnt aesni xsave avx f16c rdrnd hyper [02] CPUID 0x80000001.edx syscall nx rdtscp lm [03] CPUID 0x80000001.ecx lahf-lm lzcnt 3dnowpf [04] CPUID 0x0000000d:1.eax xsaveopt xsavec xgetbv1 [05] CPUID 0x00000007:0.ebx fsgsbase bmi1 hle avx2 bmi2 erms rtm rdseed adx clflushopt [06] CPUID 0x00000007:0.ecx [07] CPUID 0x80000007.edx itsc [08] CPUID 0x80000008.ebx ibpb [09] CPUID 0x00000007:0.edx md-clear ibrsb stibp arch-caps ssbd [10] CPUID 0x00000007:1.eax [11] CPUID 0x80000021.eax [12] CPUID 0x00000007:1.ebx [13] CPUID 0x00000007:2.edx [14] CPUID 0x00000007:1.ecx [15] CPUID 0x00000007:1.edx [16] MSR_ARCH_CAPS.lo rsba fb-clear gds-no rfds-no rfds-clear [17] MSR_ARCH_CAPS.hi HVM Max 1fcbfbff:f7fa3223:2c100800:00000121:0000000f:009c4fbb:00000000:00000100:00101000:bc000400:00000000:00000000:00000000:00000000:00000000:00000000:1c020004:00000000 [00] CPUID 0x00000001.edx fpu vme de pse tsc msr pae mce cx8 apic sysenter mtrr pge mca cmov pat pse36 clflush acpi mmx fxsr sse sse2 ss htt [01] CPUID 0x00000001.ecx sse3 pclmulqdq vmx ssse3 fma cx16 pcid sse41 sse42 x2apic movebe popcnt tsc-dl aesni xsave avx f16c rdrnd hyper [02] CPUID 0x80000001.edx syscall nx pg1g rdtscp lm [03] CPUID 0x80000001.ecx lahf-lm lzcnt 3dnowpf [04] CPUID 0x0000000d:1.eax xsaveopt xsavec xgetbv1 xsaves [05] CPUID 0x00000007:0.ebx fsgsbase tsc-adj bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx rdseed adx smap clflushopt [06] CPUID 0x00000007:0.ecx [07] CPUID 0x80000007.edx itsc [08] CPUID 0x80000008.ebx ibpb no-lmsl [09] CPUID 0x00000007:0.edx md-clear ibrsb stibp l1d-flush arch-caps ssbd
[10] CPUID 0x00000007:1.eax
[11] CPUID 0x80000021.eax
[12] CPUID 0x00000007:1.ebx
[13] CPUID 0x00000007:2.edx
[14] CPUID 0x00000007:1.ecx
[15] CPUID 0x00000007:1.edx
[16] MSR_ARCH_CAPS.lo rsba fb-clear gds-no rfds-no rfds-clear
[17] MSR_ARCH_CAPS.hi
(XEN) Built-in command line: ept=exec-sp spec-ctrl=unpriv-mmio
Xen 4.17.3
(XEN) Xen version 4.17.3 (mockbuild@[unknown]) (gcc (GCC) 12.3.1 20230508 (Red Hat 12.3.1-1)) debug=n Tue Mar 12 20:15:53 GMT 2024
(XEN) Latest ChangeSet:
(XEN) Bootloader: GRUB 2.06
(XEN) Command line: placeholder console=none dom0_mem=min:1024M dom0_mem=max:4096M ucode=scan smt=off gnttab_max_frames=2048 gnttab_max_maptrack_frames=4096 no-real-mode edd=off (XEN) Xen image load base address: 0x49800000
(XEN) Video information:
(XEN) VGA is graphics mode 1024x768, 32 bpp
(XEN) VBE/DDC methods: none; EDID transfer time: 0 seconds
(XEN) Disc information:
(XEN) Found 0 MBR signatures
(XEN) Found 1 EDD information structures
(XEN) EFI RAM map:
(XEN) [0000000000000000, 000000000009efff] (usable) (XEN) [000000000009f000, 00000000000fffff] (reserved) (XEN) [0000000000100000, 000000004a999fff] (usable) (XEN) [000000004a99a000, 000000004fa2efff] (reserved) (XEN) [000000004fa2f000, 000000004fca9fff] (ACPI NVS) (XEN) [000000004fcaa000, 000000004fd0efff] (ACPI data) (XEN) [000000004fd0f000, 000000004fd0ffff] (usable) (XEN) [000000004fd10000, 0000000059ffffff] (reserved) (XEN) [00000000fe010000, 00000000fe010fff] (reserved) (XEN) [0000000100000000, 00000002a3ffffff] (usable) (XEN) ACPI: RSDP 4FD0E014, 0024 (r2 LENOVO) (XEN) ACPI: XSDT 4FD0C188, 0114 (r1 LENOVO TP-N2C 1510 PTEC 2) (XEN) ACPI: FACP 4D2EC000, 0114 (r6 LENOVO TP-N2C 1510 PTEC 2) (XEN) ACPI: DSDT 4D2B9000, 2E277 (r2 LENOVO CFL 20170001 INTL 20160422) (XEN) ACPI: FACS 4FAE2000, 0040 (XEN) ACPI: SSDT 4D346000, 17DF (r2 LENOVO CpuSsdt 3000 INTL 20160527) (XEN) ACPI: SSDT 4D345000, 056D (r2 LENOVO CtdpB 1000 INTL 20160527) (XEN) ACPI: SSDT 4D30B000, 4310 (r2 LENOVO DptfTabl 1000 INTL 20160527) (XEN) ACPI: SSDT 4D2F5000, 3189 (r2 LENOVO SaSsdt 3000 INTL 20160527) (XEN) ACPI: SSDT 4D2F2000, 2A8B (r2 LENOVO PegSsdt 1000 INTL 20160527) (XEN) ACPI: SSDT 4D2F1000, 0612 (r2 LENOVO Tpm2Tabl 1000 INTL 20160527) (XEN) ACPI: TPM2 4D2F0000, 0034 (r4 LENOVO TP-N2C 1510 PTEC 2) (XEN) ACPI: UEFI 4FB03000, 0042 (r1 LENOVO TP-N2C 1510 PTEC 2) (XEN) ACPI: SSDT 4D2ED000, 0530 (r2 LENOVO PerfTune 1000 INTL 20160527) (XEN) ACPI: HPET 4D2EB000, 0038 (r1 LENOVO TP-N2C 1510 PTEC 2) (XEN) ACPI: APIC 4D2EA000, 012C (r3 LENOVO TP-N2C 1510 PTEC 2) (XEN) ACPI: MCFG 4D2E9000, 003C (r1 LENOVO TP-N2C 1510 PTEC 2) (XEN) ACPI: ECDT 4D2E8000, 0053 (r1 LENOVO TP-N2C 1510 PTEC 2) (XEN) ACPI: SSDT 4D2B7000, 1A92 (r2 LENOVO ProjSsdt 10 INTL 20160527) (XEN) ACPI: NHLT 4D2B5000, 1771 (r0 LENOVO TP-N2C 1510 PTEC 2) (XEN) ACPI: BOOT 4D2B4000, 0028 (r1 LENOVO TP-N2C 1510 PTEC 2) (XEN) ACPI: SSDT 4D2B3000, 0FA2 (r2 LENOVO UsbCTabl 1000 INTL 20160527) (XEN) ACPI: LPIT 4D2B2000, 005C (r1 LENOVO TP-N2C 1510 PTEC 2) (XEN) ACPI: WSMT 4D2B1000, 0028 (r1 LENOVO TP-N2C 1510 PTEC 2) (XEN) ACPI: SSDT 4D2AF000, 149F (r2 LENOVO TbtTypeC 0 INTL 20160527) (XEN) ACPI: DBGP 4D2AE000, 0034 (r1 LENOVO TP-N2C 1510 PTEC 2) (XEN) ACPI: DBG2 4D2AD000, 0054 (r0 LENOVO TP-N2C 1510 PTEC 2) (XEN) ACPI: POAT 4D2AC000, 0055 (r3 LENOVO TP-N2C 1510 PTEC 2) (XEN) ACPI: BATB 4D0CE000, 004A (r2 LENOVO TP-N2C 1510 PTEC 2) (XEN) ACPI: DMAR 4B2CB000, 0070 (r1 LENOVO TP-N2C 1510 PTEC 2) (XEN) ACPI: FPDT 4B2CA000, 0034 (r1 LENOVO TP-N2C 1510 PTEC 2) (XEN) ACPI: BGRT 4B2C9000, 0038 (r1 LENOVO TP-N2C 1510 PTEC 2) (XEN) ACPI: UEFI 4FADF000, 012A (r1 LENOVO TP-N2C 1510 PTEC 2) (XEN) ACPI: SSDT 4B2C7000, 1B28 (r1 LENOVO NvOptTbl 1000 INTL 20160527)
(XEN) System RAM: 7913MB (8103144kB)
(XEN) Domain heap initialised
(XEN) ACPI: Invalid sleep control/status register data: 0:0x8:0x3 0:0x8:0x3 (XEN) ACPI: 32/64X FACS address mismatch in FADT - 4fae2000/0000000000000000, using 32 (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-119 (XEN) PCI: Not using MCFG for segment 0000 bus 00-ff
(XEN) Switched to APIC driver x2apic_mixed
(XEN) microcode: CPU0 updated from revision 0xf4 to 0xf6, date = 2023-07-26 (XEN) CPU0: TSC: ratio: 184 / 2
(XEN) CPU0: bus: 100 MHz base: 2200 MHz max: 4100 MHz
(XEN) CPU0: 800 ... 2200 MHz
(XEN) xstate: size: 0x440 and states: 0x1f
(XEN) Speculative mitigation facilities:
(XEN) Hardware hints: RSBA (XEN) Hardware features: IBPB IBRS STIBP SSBD L1D_FLUSH MD_CLEAR SRBDS_CTRL GDS_CTRL (XEN) Compiled-in support: INDIRECT_THUNK HARDEN_ARRAY HARDEN_BRANCH HARDEN_GUEST_ACCESS (XEN) Xen settings: BTI-Thunk: JMP, SPEC_CTRL: IBRS+ STIBP+ SSBD-, Other: SRB_LOCK+ IBPB-ctxt L1D_FLUSH VERW BRANCH_HARDEN (XEN) L1TF: believed vulnerable, maxphysaddr L1D 46, CPUID 39, Safe address 8000000000 (XEN) Support for HVM VMs: MSR_SPEC_CTRL MSR_VIRT_SPEC_CTRL RSB EAGER_FPU (XEN) Support for PV VMs: MSR_SPEC_CTRL EAGER_FPU VERW (XEN) XPTI (64-bit PV only): Dom0 enabled, DomU enabled (with PCID) (XEN) PV L1TF shadowing: Dom0 disabled, DomU enabled
(XEN) Using scheduler: SMP Credit Scheduler rev2 (credit2)
(XEN) Initializing Credit2 scheduler
(XEN) Disabling HPET for being unreliable
(XEN) Platform timer is 3.580MHz ACPI PM Timer
(XEN) Detected 2207.990 MHz processor.
(XEN) Unknown cachability for MFNs 0xa0-0xff
(XEN) Unknown cachability for MFNs 0x58000-0x59fff
(XEN) 0000:00:14.3: unexpected initial MSI-X state (MASKALL=0, ENABLE=1), fixing (XEN) Intel VT-d iommu 0 supported page sizes: 4kB, 2MB, 1GB
(XEN) Intel VT-d Snoop Control enabled.
(XEN) Intel VT-d Dom0 DMA Passthrough not enabled.
(XEN) Intel VT-d Queued Invalidation enabled.
(XEN) Intel VT-d Interrupt Remapping enabled.
(XEN) Intel VT-d Posted Interrupt not enabled.
(XEN) Intel VT-d Shared EPT tables enabled.
(XEN) I/O virtualisation enabled
(XEN) - Dom0 mode: Relaxed
(XEN) Interrupt remapping enabled
(XEN) Enabled directed EOI with ioapic_ack_old on!
(XEN) Enabling APIC mode. Using 1 I/O APICs
(XEN) ENABLING IO-APIC IRQs
(XEN) -> Using old ACK method
(XEN) Allocated console ring of 32 KiB.
(XEN) VMX: Supported advanced features:
(XEN) - APIC MMIO access virtualisation (XEN) - APIC TPR shadow (XEN) - Extended Page Tables (EPT) (XEN) - Virtual-Processor Identifiers (VPID) (XEN) - Virtual NMI (XEN) - MSR direct-access bitmap (XEN) - Unrestricted Guest (XEN) - VM Functions (XEN) - Virtualisation Exceptions (XEN) - Page Modification Logging
(XEN) HVM: ASIDs enabled.
(XEN) HVM: VMX enabled
(XEN) HVM: Hardware Assisted Paging (HAP) detected (XEN) HVM: HAP page sizes: 4kB, 2MB, 1GB
(XEN) Brought up 6 CPUs
(XEN) Scheduling granularity: cpu, 1 CPU per sched-resource (XEN) Initializing Credit2 scheduler
(XEN) Dom0 has maximum 952 PIRQs
(XEN) Xen kernel: 64-bit, lsb
(XEN) Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x5000000 (XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN) Dom0 alloc.: 0000000288000000->0000000290000000 (1007276 pages to be allocated)
(XEN) Init. ramdisk: 00000002a1eac000->00000002a3fff5dc (XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN) Loaded kernel: ffffffff81000000->ffffffff85000000 (XEN) Phys-Mach map: 0000008000000000->0000008000800000 (XEN) Start info: ffffffff85000000->ffffffff850004b8 (XEN) Page tables: ffffffff85001000->ffffffff8502e000 (XEN) Boot stack: ffffffff8502e000->ffffffff8502f000 (XEN) TOTAL: ffffffff80000000->ffffffff85400000 (XEN) ENTRY ADDRESS: ffffffff834fa1c0
(XEN) Dom0 has maximum 6 VCPUs
(XEN) Initial low memory virq threshold set at 0x4000 pages. (XEN) Scrubbing Free RAM in background
(XEN) Std. Loglevel: Errors and warnings
(XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings) (XEN) *** Serial input to DOM0 (type 'CTRL-a' three times to switch input) (XEN) Freed 648kB init memory
Quoted 5 lines · Marek Marczykowski-Górecki
Hi,
I've noticed that
tools/tests/tsx/test-tsxfails on a system with Intel Core i7-8750H. Specific error I get:
Quoted 11 lines
[user@dom0 tsx]$ ./test-tsx
TSX tests
Got 16 CPUs
TestingMSR_TSX_FORCE_ABORTconsistency
CPU0 val 0
TestingMSR_TSX_CTRLconsistency
TestingMSR_MCU_OPT_CTRLconsistency
CPU0 val 0
Testing RTM behaviour
Got #UD
Host reports RTM, but appears unavailable
Isn't this ...
Quoted 10 lines
Testing PV default/max policies
Max: RTM 1, HLE 1,TSX_FORCE_ABORT0,RTM_ALWAYS_ABORT0,TSX_CTRL0
Def: RTM 0, HLE 0,TSX_FORCE_ABORT0,RTM_ALWAYS_ABORT0,TSX_CTRL0 HLE/RTM offered to guests despite not being available
Testing HVM default/max policies
Max: RTM 1, HLE 1,TSX_FORCE_ABORT0,RTM_ALWAYS_ABORT0,TSX_CTRL0
Def: RTM 0, HLE 0,TSX_FORCE_ABORT0,RTM_ALWAYS_ABORT0,TSX_CTRL0 HLE/RTM offered to guests despite not being available
Testing PV guest
Created d8
Quoted 8 lines
Cur: RTM 0, HLE 0,
TSX_FORCE_ABORT0,RTM_ALWAYS_ABORT0,TSX_CTRL0
Cur: RTM 1, HLE 1,TSX_FORCE_ABORT0,RTM_ALWAYS_ABORT0,TSX_CTRL0
Testing HVM guest
Created d9
Cur: RTM 0, HLE 0,TSX_FORCE_ABORT0,RTM_ALWAYS_ABORT0,TSX_CTRL0
Cur: RTM 1, HLE 1,TSX_FORCE_ABORT0,RTM_ALWAYS_ABORT0,TSX_CTRL0
[user@dom0 tsx]$ echo $?
1
... the reason for this?
Jan
Quoted 39 lines · Jan Beulich
Quoted 5 lines · Marek Marczykowski-Górecki
Hi,
I've noticed that
tools/tests/tsx/test-tsxfails on a system with Intel Core i7-8750H. Specific error I get:Quoted 11 lines
[user@dom0 tsx]$ ./test-tsx
TSX tests
Got 16 CPUs
TestingMSR_TSX_FORCE_ABORTconsistency
CPU0 val 0
TestingMSR_TSX_CTRLconsistency
TestingMSR_MCU_OPT_CTRLconsistency
CPU0 val 0
Testing RTM behaviour
Got #UD
Host reports RTM, but appears unavailableIsn't this ...
Quoted 10 lines
Testing PV default/max policies
Max: RTM 1, HLE 1,TSX_FORCE_ABORT0,RTM_ALWAYS_ABORT0,TSX_CTRL0
Def: RTM 0, HLE 0,TSX_FORCE_ABORT0,RTM_ALWAYS_ABORT0,TSX_CTRL0 HLE/RTM offered to guests despite not being available
Testing HVM default/max policies
Max: RTM 1, HLE 1,TSX_FORCE_ABORT0,RTM_ALWAYS_ABORT0,TSX_CTRL0
Def: RTM 0, HLE 0,TSX_FORCE_ABORT0,RTM_ALWAYS_ABORT0,TSX_CTRL0 HLE/RTM offered to guests despite not being available
Testing PV guest
Created d8Quoted 8 lines
Cur: RTM 0, HLE 0,
TSX_FORCE_ABORT0,RTM_ALWAYS_ABORT0,TSX_CTRL0
Cur: RTM 1, HLE 1,TSX_FORCE_ABORT0,RTM_ALWAYS_ABORT0,TSX_CTRL0
Testing HVM guest
Created d9
Cur: RTM 0, HLE 0,TSX_FORCE_ABORT0,RTM_ALWAYS_ABORT0,TSX_CTRL0
Cur: RTM 1, HLE 1,TSX_FORCE_ABORT0,RTM_ALWAYS_ABORT0,TSX_CTRL0
[user@dom0 tsx]$ echo $?
1
Quoted 1 line
... the reason for this?
I think so, but the question is why it behaves this way. Could be an issue with MSR/CPUID values presented by Xen, or values Xen gets from the CPU.
Signature · 3 lines
Best Regards, Marek Marczykowski-Górecki Invisible Things Lab
Quoted 40 lines · Marek Marczykowski-Górecki
Quoted 39 lines · Jan Beulich
Quoted 5 lines · Marek Marczykowski-Górecki
Hi,
I've noticed that
tools/tests/tsx/test-tsxfails on a system with Intel Core i7-8750H. Specific error I get:Quoted 11 lines
[user@dom0 tsx]$ ./test-tsx
TSX tests
Got 16 CPUs
TestingMSR_TSX_FORCE_ABORTconsistency
CPU0 val 0
TestingMSR_TSX_CTRLconsistency
TestingMSR_MCU_OPT_CTRLconsistency
CPU0 val 0
Testing RTM behaviour
Got #UD
Host reports RTM, but appears unavailableIsn't this ...
Quoted 18 lines
Testing PV default/max policies
Max: RTM 1, HLE 1,TSX_FORCE_ABORT0,RTM_ALWAYS_ABORT0,TSX_CTRL0
Def: RTM 0, HLE 0,TSX_FORCE_ABORT0,RTM_ALWAYS_ABORT0,TSX_CTRL0 HLE/RTM offered to guests despite not being available
Testing HVM default/max policies
Max: RTM 1, HLE 1,TSX_FORCE_ABORT0,RTM_ALWAYS_ABORT0,TSX_CTRL0
Def: RTM 0, HLE 0,TSX_FORCE_ABORT0,RTM_ALWAYS_ABORT0,TSX_CTRL0 HLE/RTM offered to guests despite not being available
Testing PV guest
Created d8
Cur: RTM 0, HLE 0,TSX_FORCE_ABORT0,RTM_ALWAYS_ABORT0,TSX_CTRL0
Cur: RTM 1, HLE 1,TSX_FORCE_ABORT0,RTM_ALWAYS_ABORT0,TSX_CTRL0
Testing HVM guest
Created d9
Cur: RTM 0, HLE 0,TSX_FORCE_ABORT0,RTM_ALWAYS_ABORT0,TSX_CTRL0
Cur: RTM 1, HLE 1,TSX_FORCE_ABORT0,RTM_ALWAYS_ABORT0,TSX_CTRL0
[user@dom0 tsx]$ echo $?
1
Quoted 5 lines
Quoted 1 line
... the reason for this?
I think so, but the question is why it behaves this way. Could be an issue with MSR/CPUID values presented by Xen, or values Xen gets from the CPU.
Can't test_rtm_behaviour() be run even without Xen underneath? Maybe this could be run irrespective of xc_interface_open() failing?
Jan
Quoted 5 lines · Marek Marczykowski-Górecki
Hi,
I've noticed that
tools/tests/tsx/test-tsxfails on a system with Intel Core i7-8750H. Specific error I get:
Quoted 11 lines
[user@dom0 tsx]$ ./test-tsx
TSX tests
Got 16 CPUs
TestingMSR_TSX_FORCE_ABORTconsistency
CPU0 val 0
TestingMSR_TSX_CTRLconsistency
TestingMSR_MCU_OPT_CTRLconsistency
CPU0 val 0
Testing RTM behaviour
Got #UD
Host reports RTM, but appears unavailable
Hmm - I should make this failure report more obviously distinguishable from the general logging.
This is reporting a consistency-check failure, with a mismatch between actual-behaviour and what's in CPUID (host policy in practice).
This is CoffeeLake, and was one of the CPUs which had TSX taken out, but something looks wonky.
Quoted 2 lines
When I try it on a newer system (11th gen) then it works fine (exit code 0, just "Got #UD", no "Host reports RTM, but appears unavailable" line).
RocketLake was after the decision to remove TSX from the client line, so will either genuinely not have the silicon, or it should be properly fused out.
Anyway, back to CoffeeLake.
The Raw policy shows rtm-always-abort and tsx-force-abort. Test-tsx says the value in MSR_TSX_FORCE_ABORT is 0, but that shouldn't be the case seeing as the RTM/HLE bits are hidden in real CPUID, but the CPUID_HIDE bit is clear.
We do intentionally force RTM_ALWAYS_ABORT in some cases, because it self-hides in some cases. I wonder if we've got bug in that path.
From the state in Raw, we then synthesise HLE/RTM in the Host policy because MSR_TSX_FORCE_ABORT only exists on TSX-capable systems. However, if XBEGIN is really #UD-ing, we can't offer it as an opt-in to guests.
Let me see about putting some debugging together.
~Andrew
PATCH 1/1 x86/tsx: Cope with RTM_ALWAYS_ABORT vs RTM mismatch
In reply to Marek Marczykowski-GóreckiIt turns out there is something wonky on some but not all CPUs with MSR_TSX_FORCE_ABORT. The presence of RTM_ALWAYS_ABORT causes Xen to think it's safe to offer HLE/RTM to guests, but in this case, XBEGIN instructions genuinely #UD.
Spot this case and try to back out as cleanly as we can.
---
CC: Jan Beulich <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Marek Marczykowski-Górecki <[email protected]>
In the meantime, I'll see if anyone at Intel knows what's going on. Because these parts are fully out of support now, it's very unlikely that we're going to get a fix.
--- xen/arch/x86/tsx.c | 55 +++++++++++++++++++++++++++++++++++++--------- 1 file changed, 45 insertions(+), 10 deletions(-)
xen/arch/x86/tsx.c +45 -10
@@ -1,5 +1,6 @@#include <xen/init.h> #include <xen/param.h> +#include <asm/microcode.h> #include <asm/msr.h> /*
@@ -9,6 +10,7 @@* -1 => Default, altered to 0/1 (if unspecified) by: * - TAA heuristics/settings for speculative safety * - "TSX vs PCR3" select for TSX memory ordering safety +* -2 => Implicit tsx=0 (from RTM_ALWAYS_ABORT vs RTM mismatch) * -3 => Implicit tsx=1 (feed-through from spec-ctrl=0) * * This is arranged such that the bottom bit encodes whether TSX is actually
@@ -114,11 +116,50 @@ void tsx_init(void)if ( cpu_has_tsx_force_abort ) { +uint64_t val; +/* -* On an early TSX-enable Skylake part subject to the memory +* On an early TSX-enabled Skylake part subject to the memory * ordering erratum, with at least the March 2019 microcode. */ +rdmsrl (MSR_TSX_FORCE_ABORT , val); + +/* +* At the time of writing (April 2024), it was discovered that +* some parts (e.g. CoffeeLake 8th Gen, 06-9e-0a, ucode 0xf6) +* advertise RTM_ALWAYS_ABORT, but XBEGIN instructions #UD. Other +* similar parts (e.g. KabyLake Xeon-E3, 06-9e-09, ucode 0xf8) +* operate as expected. +* +* In this case: +* - RTM_ALWAYS_ABORT and MSR_TSX_FORCE_ABORT are enumerated. +* - XBEGIN instructions genuinely #UD. +* - MSR_TSX_FORCE_ABORT is write-discard and fails to hold its +* value. +* - HLE and RTM are not enumerated, despite +* MSR_TSX_FORCE_ABORT.TSX_CPUID_CLEAR being clear. +* +* Spot this case, and treat it as if no TSX is available at all. +* This will prevent Xen from thinking it's safe to offer HLE/RTM +* to VMs. +*/ +if ( val ==0 && cpu_has_rtm_always_abort && !cpu_has_rtm ) + { +printk (XENLOG_ERR +"FIRMWARE BUG: CPU %02x-%02x-%02x, ucode 0x%08x: RTM_ALWAYS_ABORT vs RTM mismatch\n" , + boot_cpu_data.x86, boot_cpu_data.x86_model, + boot_cpu_data.x86_mask,this_cpu (cpu_sig).rev); + +setup_clear_cpu_cap (X86_FEATURE_RTM_ALWAYS_ABORT ); +setup_clear_cpu_cap (X86_FEATURE_TSX_FORCE_ABORT ); + +if ( opt_tsx <0 ) + opt_tsx =-2 ; + + goto done_setup; + } +/* * Probe for the June 2021 microcode which de-features TSX on * client parts. (Note - this is a subset of parts impacted by
@@ -128,15 +169,8 @@ void tsx_init(void)* read as zero if TSX_FORCE_ABORT.ENABLE_RTM has been set before * we run. */ -if ( !has_rtm_always_abort ) - { -uint64_t val; - -rdmsrl (MSR_TSX_FORCE_ABORT , val); - -if ( val &TSX_ENABLE_RTM ) - has_rtm_always_abort = true; - } +if ( val &TSX_ENABLE_RTM ) + has_rtm_always_abort = true;/* * If no explicit tsx= option is provided, pick a default.
@@ -191,6 +225,7 @@ void tsx_init(void)setup_force_cpu_cap (X86_FEATURE_RTM ); } } + done_setup:/* * Note: MSR_TSX_CTRL is enumerated on TSX-enabled MDS_NO and later parts.
base-commit: 6117179dd99958e4ef2687617d12c9b15bdbae24
Signature · 1 line
2.30.2
Re: [PATCH] x86/tsx: Cope with RTM_ALWAYS_ABORT vs RTM mismatch
Quoted 8 lines · Andrew Cooper
It turns out there is something wonky on some but not all CPUs with
MSR_TSX_FORCE_ABORT. The presence ofRTM_ALWAYS_ABORTcauses Xen to think it's safe to offer HLE/RTM to guests, but in this case, XBEGIN instructions genuinely #UD.Spot this case and try to back out as cleanly as we can.
Signed-off-by Andrew Cooper <[email protected]>
Thanks, this makes the test exit with 0, and print just "Got #UD" now in the "Testing RTM behaviour" section.
Quoted 114 lines
---
CC: Jan Beulich <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Marek Marczykowski-Górecki <[email protected]>In the meantime, I'll see if anyone at Intel knows what's going on. Because these parts are fully out of support now, it's very unlikely that we're going to get a fix.
--- xen/arch/x86/tsx.c | 55 +++++++++++++++++++++++++++++++++++++--------- 1 file changed, 45 insertions(+), 10 deletions(-)xen/arch/x86/tsx.c +45
-10@@ -1,5 +1,6 @@#include <xen/init.h> #include <xen/param.h> +#include <asm/microcode.h> #include <asm/msr.h> /* @@ -9,6 +10,7 @@* -1 => Default, altered to 0/1 (if unspecified) by: * - TAA heuristics/settings for speculative safety * - "TSX vs PCR3" select for TSX memory ordering safety +* -2 => Implicit tsx=0 (from RTM_ALWAYS_ABORT vs RTM mismatch) * -3 => Implicit tsx=1 (feed-through from spec-ctrl=0) * * This is arranged such that the bottom bit encodes whether TSX is actually @@ -114,11 +116,50 @@ void tsx_init(void)if ( cpu_has_tsx_force_abort ) { +uint64_t val; +/* -* On an early TSX-enable Skylake part subject to the memory +* On an early TSX-enabled Skylake part subject to the memory * ordering erratum, with at least the March 2019 microcode. */ +rdmsrl (MSR_TSX_FORCE_ABORT , val); + +/* +* At the time of writing (April 2024), it was discovered that +* some parts (e.g. CoffeeLake 8th Gen, 06-9e-0a, ucode 0xf6) +* advertise RTM_ALWAYS_ABORT, but XBEGIN instructions #UD. Other +* similar parts (e.g. KabyLake Xeon-E3, 06-9e-09, ucode 0xf8) +* operate as expected. +* +* In this case: +* - RTM_ALWAYS_ABORT and MSR_TSX_FORCE_ABORT are enumerated. +* - XBEGIN instructions genuinely #UD. +* - MSR_TSX_FORCE_ABORT is write-discard and fails to hold its +* value. +* - HLE and RTM are not enumerated, despite +* MSR_TSX_FORCE_ABORT.TSX_CPUID_CLEAR being clear. +* +* Spot this case, and treat it as if no TSX is available at all. +* This will prevent Xen from thinking it's safe to offer HLE/RTM +* to VMs. +*/ +if ( val ==0 && cpu_has_rtm_always_abort && !cpu_has_rtm ) + { +printk (XENLOG_ERR +"FIRMWARE BUG: CPU %02x-%02x-%02x, ucode 0x%08x: RTM_ALWAYS_ABORT vs RTM mismatch\n" , + boot_cpu_data.x86, boot_cpu_data.x86_model, + boot_cpu_data.x86_mask,this_cpu (cpu_sig).rev); + +setup_clear_cpu_cap (X86_FEATURE_RTM_ALWAYS_ABORT ); +setup_clear_cpu_cap (X86_FEATURE_TSX_FORCE_ABORT ); + +if ( opt_tsx <0 ) + opt_tsx =-2 ; + + goto done_setup; + } +/* * Probe for the June 2021 microcode which de-features TSX on * client parts. (Note - this is a subset of parts impacted by @@ -128,15 +169,8 @@ void tsx_init(void)* read as zero if TSX_FORCE_ABORT.ENABLE_RTM has been set before * we run. */ -if ( !has_rtm_always_abort ) - { -uint64_t val; - -rdmsrl (MSR_TSX_FORCE_ABORT , val); - -if ( val &TSX_ENABLE_RTM ) - has_rtm_always_abort = true; - } +if ( val &TSX_ENABLE_RTM ) + has_rtm_always_abort = true;/* * If no explicit tsx= option is provided, pick a default. @@ -191,6 +225,7 @@ void tsx_init(void)setup_force_cpu_cap (X86_FEATURE_RTM ); } } + done_setup:/* * Note: MSR_TSX_CTRL is enumerated on TSX-enabled MDS_NO and later parts. base-commit:
6117179dd99958e4ef2687617d12c9b15bdbae24Signature · 1 line
2.30.2
Signature · 3 lines
Best Regards, Marek Marczykowski-Górecki Invisible Things Lab
Re: [PATCH] x86/tsx: Cope with RTM_ALWAYS_ABORT vs RTM mismatch
In reply to Andrew CooperQuoted 26 lines · Andrew Cooper
xen/arch/x86/tsx.c +10
-1@@ -9,6 +10,7 @@* -1 => Default, altered to 0/1 (if unspecified) by: * - TAA heuristics/settings for speculative safety * - "TSX vs PCR3" select for TSX memory ordering safety +* -2 => Implicit tsx=0 (from RTM_ALWAYS_ABORT vs RTM mismatch) * -3 => Implicit tsx=1 (feed-through from spec-ctrl=0) * * This is arranged such that the bottom bit encodes whether TSX is actually @@ -114,11 +116,50 @@ void tsx_init(void)if ( cpu_has_tsx_force_abort ) { +uint64_t val; +/* -* On an early TSX-enable Skylake part subject to the memory +* On an early TSX-enabled Skylake part subject to the memory * ordering erratum, with at least the March 2019 microcode. */ +rdmsrl (MSR_TSX_FORCE_ABORT , val); + +/* +* At the time of writing (April 2024), it was discovered that +* some parts (e.g. CoffeeLake 8th Gen, 06-9e-0a, ucode 0xf6) +* advertise RTM_ALWAYS_ABORT, but XBEGIN instructions #UD. Other
xen/arch/x86/tsx.c void tsx_init(void) · line 141
Quoted 10 lines
xen/arch/x86/tsx.c +10
-0+* similar parts (e.g. KabyLake Xeon-E3, 06-9e-09, ucode 0xf8) +* operate as expected. +* +* In this case: +* - RTM_ALWAYS_ABORT and MSR_TSX_FORCE_ABORT are enumerated. +* - XBEGIN instructions genuinely #UD. +* - MSR_TSX_FORCE_ABORT is write-discard and fails to hold its +* value. +* - HLE and RTM are not enumerated, despite +* MSR_TSX_FORCE_ABORT.TSX_CPUID_CLEAR being clear.
Of these 4 items you use the first and last here. It took me some time to figure that the middle two are (aiui) only informational, and that you assume that first and last together are sufficient to uniquely identify the problematic parts. Separating the two groups a little might be helpful.
For the write-discard property, how was that determined? Does it affect all writable bits?
xen/arch/x86/tsx.c void tsx_init(void) · line 150
Quoted 8 lines
xen/arch/x86/tsx.c +8
-0+* Spot this case, and treat it as if no TSX is available at all. +* This will prevent Xen from thinking it's safe to offer HLE/RTM +* to VMs. +*/ +if ( val ==0 && cpu_has_rtm_always_abort && !cpu_has_rtm ) + { + printk(XENLOG_ERR + "FIRMWARE BUG: CPU %02x-%02x-%02x, ucode 0x%08x: RTM_ALWAYS_ABORT vs RTM mismatch\n",
This isn't really firmware, is it? At least I wouldn't call microcode (assuming that's where the bad behavior is rooted) firmware.
xen/arch/x86/tsx.c void tsx_init(void) · line 154
Quoted 4 lines
xen/arch/x86/tsx.c +4
-0+ boot_cpu_data.x86, boot_cpu_data.x86_model, + boot_cpu_data.x86_mask, this_cpu(cpu_sig).rev); + +setup_clear_cpu_cap (X86_FEATURE_RTM_ALWAYS_ABORT );
Instead of the "goto" below, wouldn't it be better to also force has_rtm_always_abort to false along with this, thus skipping the setup_force_cpu_cap(X86_FEATURE_RTM_ALWAYS_ABORT) further down? That would leave things a little less awkward flow-wise, imo. The one thing not becoming clear from the commentary above is whether cpu_has_tsx_ctrl might be true, and hence RTM/HLE still becoming (wrongly) set, if done that way.
Jan
Quoted 39 lines
xen/arch/x86/tsx.c +11
-9+setup_clear_cpu_cap (X86_FEATURE_TSX_FORCE_ABORT ); + +if ( opt_tsx <0 ) + opt_tsx =-2 ; + + goto done_setup; + } +/* * Probe for the June 2021 microcode which de-features TSX on * client parts. (Note - this is a subset of parts impacted by @@ -128,15 +169,8 @@ void tsx_init(void)* read as zero if TSX_FORCE_ABORT.ENABLE_RTM has been set before * we run. */ -if ( !has_rtm_always_abort ) - { -uint64_t val; - -rdmsrl (MSR_TSX_FORCE_ABORT , val); - -if ( val &TSX_ENABLE_RTM ) - has_rtm_always_abort = true; - } +if ( val &TSX_ENABLE_RTM ) + has_rtm_always_abort = true;/* * If no explicit tsx= option is provided, pick a default. @@ -191,6 +225,7 @@ void tsx_init(void)setup_force_cpu_cap (X86_FEATURE_RTM ); } } + done_setup:/* * Note: MSR_TSX_CTRL is enumerated on TSX-enabled MDS_NO and later parts. base-commit:
6117179dd99958e4ef2687617d12c9b15bdbae24
Re: [PATCH] x86/tsx: Cope with RTM_ALWAYS_ABORT vs RTM mismatch
Quoted 33 lines · Jan Beulich
Quoted 32 lines · Andrew Cooper
xen/arch/x86/tsx.c +16
-1@@ -9,6 +10,7 @@* -1 => Default, altered to 0/1 (if unspecified) by: * - TAA heuristics/settings for speculative safety * - "TSX vs PCR3" select for TSX memory ordering safety +* -2 => Implicit tsx=0 (from RTM_ALWAYS_ABORT vs RTM mismatch) * -3 => Implicit tsx=1 (feed-through from spec-ctrl=0) * * This is arranged such that the bottom bit encodes whether TSX is actually @@ -114,11 +116,50 @@ void tsx_init(void)if ( cpu_has_tsx_force_abort ) { +uint64_t val; +/* -* On an early TSX-enable Skylake part subject to the memory +* On an early TSX-enabled Skylake part subject to the memory * ordering erratum, with at least the March 2019 microcode. */ +rdmsrl (MSR_TSX_FORCE_ABORT , val); + +/* +* At the time of writing (April 2024), it was discovered that +* some parts (e.g. CoffeeLake 8th Gen, 06-9e-0a, ucode 0xf6) +* advertise RTM_ALWAYS_ABORT, but XBEGIN instructions #UD. Other +* similar parts (e.g. KabyLake Xeon-E3, 06-9e-09, ucode 0xf8) +* operate as expected. +* +* In this case: +* - RTM_ALWAYS_ABORT and MSR_TSX_FORCE_ABORT are enumerated. +* - XBEGIN instructions genuinely #UD.
Quoted 8 lines
xen/arch/x86/tsx.c void tsx_init(void) · line 141
Quoted 4 lines
xen/arch/x86/tsx.c +4
-0+* - MSR_TSX_FORCE_ABORT is write-discard and fails to hold its +* value. +* - HLE and RTM are not enumerated, despite +* MSR_TSX_FORCE_ABORT.TSX_CPUID_CLEAR being clear. Of these 4 items you use the first and last here. It took me some time to figure that the middle two are (aiui) only informational, and that you assume that first and last together are sufficient to uniquely identify the problematic parts. Separating the two groups a little might be helpful.
All 4 points are relevant to the if() expression.
Quoted 3 lines
For the write-discard property, how was that determined? Does it affect all writable bits?
Marek kindly ran a debugging patch for me last night to try and figure out what was going on.
Currently, Xen tries to set 0x2 (TSX_CPUID_CLEAR) and debugging showed it being read back as 0.
I didn't check anything else, but I have a strong suspicion that I know exactly what's going wrong here.
The property the if() condition is mainly looking for is !RTM && !(MSR_TFA.CPUID_CLEAR) because that's an illegal state in a
Quoted 11 lines
xen/arch/x86/tsx.c void tsx_init(void) · line 150
Quoted 9 lines
xen/arch/x86/tsx.c +8
-0+* Spot this case, and treat it as if no TSX is available at all. +* This will prevent Xen from thinking it's safe to offer HLE/RTM +* to VMs. +*/ +if ( val ==0 && cpu_has_rtm_always_abort && !cpu_has_rtm ) + { + printk(XENLOG_ERR + "FIRMWARE BUG: CPU %02x-%02x-%02x, ucode 0x%08x: RTM_ALWAYS_ABORT vs RTM mismatch\n",This isn't really firmware, is it? At least I wouldn't call microcode (assuming that's where the bad behavior is rooted) firmware.
Microcode is absolutely part of the system firmware.
Quoted 8 lines
xen/arch/x86/tsx.c void tsx_init(void) · line 154
Quoted 5 lines
xen/arch/x86/tsx.c +4
-0+ boot_cpu_data.x86, boot_cpu_data.x86_model, + boot_cpu_data.x86_mask, this_cpu(cpu_sig).rev); + +setup_clear_cpu_cap (X86_FEATURE_RTM_ALWAYS_ABORT );Instead of the "goto" below, wouldn't it be better to also force
has_rtm_always_abortto false along with this, thus skipping thesetup_force_cpu_cap(X86_FEATURE_RTM_ALWAYS_ABORT)further down?
I considered that and dismissed it. It is more fragile, in a case were really do want to treat this case as if TSX genuinely doesn't exist.
Quoted 4 lines
That would
leave things a little less awkward flow-wise, imo. The one thing not becoming clear from the commentary above is whethercpu_has_tsx_ctrlmight be true, and hence RTM/HLE still becoming (wrongly) set, if done that way.
MSR_TSX_CTRL and MSR_TSX_FORCE_ABORT exist on disjoint sets of CPUs. (The split being MDS_NO).
This is discussed explicitly lower down in the function, beyond the if ( once ) block.
~Andrew
Re: [PATCH] x86/tsx: Cope with RTM_ALWAYS_ABORT vs RTM mismatch
Quoted 43 lines · Andrew Cooper
Quoted 33 lines · Jan Beulich
Quoted 32 lines · Andrew Cooper
xen/arch/x86/tsx.c +16
-1@@ -9,6 +10,7 @@* -1 => Default, altered to 0/1 (if unspecified) by: * - TAA heuristics/settings for speculative safety * - "TSX vs PCR3" select for TSX memory ordering safety +* -2 => Implicit tsx=0 (from RTM_ALWAYS_ABORT vs RTM mismatch) * -3 => Implicit tsx=1 (feed-through from spec-ctrl=0) * * This is arranged such that the bottom bit encodes whether TSX is actually @@ -114,11 +116,50 @@ void tsx_init(void)if ( cpu_has_tsx_force_abort ) { +uint64_t val; +/* -* On an early TSX-enable Skylake part subject to the memory +* On an early TSX-enabled Skylake part subject to the memory * ordering erratum, with at least the March 2019 microcode. */ +rdmsrl (MSR_TSX_FORCE_ABORT , val); + +/* +* At the time of writing (April 2024), it was discovered that +* some parts (e.g. CoffeeLake 8th Gen, 06-9e-0a, ucode 0xf6) +* advertise RTM_ALWAYS_ABORT, but XBEGIN instructions #UD. Other +* similar parts (e.g. KabyLake Xeon-E3, 06-9e-09, ucode 0xf8) +* operate as expected. +* +* In this case: +* - RTM_ALWAYS_ABORT and MSR_TSX_FORCE_ABORT are enumerated. +* - XBEGIN instructions genuinely #UD. Quoted 8 lines
xen/arch/x86/tsx.c void tsx_init(void) · line 141
Quoted 4 lines
xen/arch/x86/tsx.c +4
-0+* - MSR_TSX_FORCE_ABORT is write-discard and fails to hold its +* value. +* - HLE and RTM are not enumerated, despite +* MSR_TSX_FORCE_ABORT.TSX_CPUID_CLEAR being clear. Of these 4 items you use the first and last here. It took me some time to figure that the middle two are (aiui) only informational, and that you assume that first and last together are sufficient to uniquely identify the problematic parts. Separating the two groups a little might be helpful.
Quoted 1 line
All 4 points are relevant to the
if()expression.
In which way? You don't probe XBEGIN to see whether you get back #UD. And you also don't probe the MSR to see whether written bits are discarded.
Quoted 11 lines
Quoted 2 lines
For the write-discard property, how was that determined? Does it affect all writable bits?
Marek kindly ran a debugging patch for me last night to try and figure out what was going on.
Currently, Xen tries to set
0x2(TSX_CPUID_CLEAR) and debugging showed it being read back as 0.I didn't check anything else, but I have a strong suspicion that I know exactly what's going wrong here.
Hmm, at the risk of upsetting you: Is a suspicion really enough for a firm statement in a comment?
Quoted 12 lines
The property the
if()condition is mainly looking for is !RTM && !(MSR_TFA.CPUID_CLEAR) because that's an illegal state in aQuoted 9 lines
Quoted 9 lines
xen/arch/x86/tsx.c +8
-0+* Spot this case, and treat it as if no TSX is available at all. +* This will prevent Xen from thinking it's safe to offer HLE/RTM +* to VMs. +*/ +if ( val ==0 && cpu_has_rtm_always_abort && !cpu_has_rtm ) + { + printk(XENLOG_ERR + "FIRMWARE BUG: CPU %02x-%02x-%02x, ucode 0x%08x: RTM_ALWAYS_ABORT vs RTM mismatch\n",
Quoted 4 lines
Quoted 2 lines
This isn't really firmware, is it? At least I wouldn't call microcode (assuming that's where the bad behavior is rooted) firmware.
Microcode is absolutely part of the system firmware.
The ucode ahead of being loaded into CPUs is, sure. But once in the CPU (and there may not be any loading at least in theory), it's not anymore. It becomes part of the CPU then, albeit I still wouldn't call it "hardware".
Plus saying "firmware" suggests that firmware vendors could do anything about the situation, when I don't think they can.
Jan
Re: [PATCH] x86/tsx: Cope with RTM_ALWAYS_ABORT vs RTM mismatch
Quoted 7 lines · Jan Beulich
Quoted 6 lines · Andrew Cooper
Quoted 2 lines · Jan Beulich
For the write-discard property, how was that determined? Does it affect all writable bits?
Marek kindly ran a debugging patch for me last night to try and figure out what was going on.
Quoted 7 lines
Quoted 5 lines
Currently, Xen tries to set
0x2(TSX_CPUID_CLEAR) and debugging showed it being read back as 0.I didn't check anything else, but I have a strong suspicion that I know exactly what's going wrong here.
Hmm, at the risk of upsetting you: Is a suspicion really enough for a firm statement in a comment?
The statement is all demonstrable properties.
The suspicion is about *why* we've ended up with the properties we have, and is based on my involvement in the original planning for this.
Quoted 11 lines
Quoted 11 lines
The property the
if()condition is mainly looking for is !RTM && !(MSR_TFA.CPUID_CLEAR) because that's an illegal state in aQuoted 8 lines
Quoted 8 lines
xen/arch/x86/tsx.c +8
-0+* Spot this case, and treat it as if no TSX is available at all. +* This will prevent Xen from thinking it's safe to offer HLE/RTM +* to VMs. +*/ +if ( val ==0 && cpu_has_rtm_always_abort && !cpu_has_rtm ) + { + printk(XENLOG_ERR + "FIRMWARE BUG: CPU %02x-%02x-%02x, ucode 0x%08x: RTM_ALWAYS_ABORT vs RTM mismatch\n",
Quoted 5 lines
Quoted 3 lines
Quoted 2 lines
This isn't really firmware, is it? At least I wouldn't call microcode (assuming that's where the bad behavior is rooted) firmware.
Microcode is absolutely part of the system firmware.
The ucode ahead of being loaded into CPUs is, sure. But once in the CPU (and there may not be any loading at least in theory), it's not anymore.
You appear to have a very singular impression of what does and does not constitute firmware.
If you can change Intel and AMD's mind on this matter, feel free to submit a patch changing the wording here.
~Andrew
Re: [PATCH] x86/tsx: Cope with RTM_ALWAYS_ABORT vs RTM mismatch
Quoted 11 lines · Andrew Cooper
Quoted 10 lines · Jan Beulich
Quoted 9 lines · Andrew Cooper
Quoted 8 lines · Jan Beulich
Quoted 8 lines
xen/arch/x86/tsx.c +8
-0+* Spot this case, and treat it as if no TSX is available at all. +* This will prevent Xen from thinking it's safe to offer HLE/RTM +* to VMs. +*/ +if ( val ==0 && cpu_has_rtm_always_abort && !cpu_has_rtm ) + { + printk(XENLOG_ERR + "FIRMWARE BUG: CPU %02x-%02x-%02x, ucode 0x%08x: RTM_ALWAYS_ABORT vs RTM mismatch\n",
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
Quoted 2 lines
This isn't really firmware, is it? At least I wouldn't call microcode (assuming that's where the bad behavior is rooted) firmware.
Microcode is absolutely part of the system firmware.
The ucode ahead of being loaded into CPUs is, sure. But once in the CPU (and there may not be any loading at least in theory), it's not anymore.
You appear to have a very singular impression of what does and does not constitute firmware.
Not so singular, I would say: https://en.wikipedia.org/wiki/Firmware
The only mention of microcode there is for historical context, afaics.
Jan
Quoted 4 lines
If you can change Intel and AMD's mind on this matter, feel free to submit a patch changing the wording here.
~Andrew
PATCH v2 1/1 x86/tsx: Cope with RTM_ALWAYS_ABORT vs RTM mismatch
In reply to Marek Marczykowski-GóreckiIt turns out there is something wonky on some but not all CPUs with MSR_TSX_FORCE_ABORT. The presence of RTM_ALWAYS_ABORT causes Xen to think it's safe to offer HLE/RTM to guests, but in this case, XBEGIN instructions genuinely #UD.
Spot this case and try to back out as cleanly as we can.
---
CC: Jan Beulich <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Marek Marczykowski-Górecki <[email protected]>
v2:
* Adjust wording.
--- xen/arch/x86/tsx.c | 55 +++++++++++++++++++++++++++++++++++++--------- 1 file changed, 45 insertions(+), 10 deletions(-)
xen/arch/x86/tsx.c +45 -10
@@ -1,5 +1,6 @@#include <xen/init.h> #include <xen/param.h> +#include <asm/microcode.h> #include <asm/msr.h> /*
@@ -9,6 +10,7 @@* -1 => Default, altered to 0/1 (if unspecified) by: * - TAA heuristics/settings for speculative safety * - "TSX vs PCR3" select for TSX memory ordering safety +* -2 => Implicit tsx=0 (from RTM_ALWAYS_ABORT vs RTM mismatch) * -3 => Implicit tsx=1 (feed-through from spec-ctrl=0) * * This is arranged such that the bottom bit encodes whether TSX is actually
@@ -114,11 +116,50 @@ void tsx_init(void)if ( cpu_has_tsx_force_abort ) { +uint64_t val; +/* -* On an early TSX-enable Skylake part subject to the memory +* On an early TSX-enabled Skylake part subject to the memory * ordering erratum, with at least the March 2019 microcode. */ +rdmsrl (MSR_TSX_FORCE_ABORT , val); + +/* +* At the time of writing (April 2024), it was discovered that +* some parts (e.g. CoffeeLake 8th Gen, 06-9e-0a, ucode 0xf6) +* advertise RTM_ALWAYS_ABORT, but XBEGIN instructions #UD. Other +* similar parts (e.g. KabyLake Xeon-E3, 06-9e-09, ucode 0xf8) +* operate as expected. +* +* In this case: +* - RTM_ALWAYS_ABORT and MSR_TSX_FORCE_ABORT are enumerated. +* - XBEGIN instructions genuinely #UD. +* - MSR_TSX_FORCE_ABORT appears to be write-discard and fails to +* hold its value. +* - HLE and RTM are not enumerated, despite +* MSR_TSX_FORCE_ABORT.TSX_CPUID_CLEAR being clear. +* +* Spot RTM being unavailable without CLEAR_CPUID being set, and +* treat it as if no TSX is available at all. This will prevent +* Xen from thinking it's safe to offer HLE/RTM to VMs. +*/ +if ( val ==0 && cpu_has_rtm_always_abort && !cpu_has_rtm ) + { +printk (XENLOG_ERR +"FIRMWARE BUG: CPU %02x-%02x-%02x, ucode 0x%08x: RTM_ALWAYS_ABORT vs RTM mismatch\n" , + boot_cpu_data.x86, boot_cpu_data.x86_model, + boot_cpu_data.x86_mask,this_cpu (cpu_sig).rev); + +setup_clear_cpu_cap (X86_FEATURE_RTM_ALWAYS_ABORT ); +setup_clear_cpu_cap (X86_FEATURE_TSX_FORCE_ABORT ); + +if ( opt_tsx <0 ) + opt_tsx =-2 ; + + goto done_probe; + } +/* * Probe for the June 2021 microcode which de-features TSX on * client parts. (Note - this is a subset of parts impacted by
@@ -128,15 +169,8 @@ void tsx_init(void)* read as zero if TSX_FORCE_ABORT.ENABLE_RTM has been set before * we run. */ -if ( !has_rtm_always_abort ) - { -uint64_t val; - -rdmsrl (MSR_TSX_FORCE_ABORT , val); - -if ( val &TSX_ENABLE_RTM ) - has_rtm_always_abort = true; - } +if ( val &TSX_ENABLE_RTM ) + has_rtm_always_abort = true;/* * If no explicit tsx= option is provided, pick a default.
@@ -191,6 +225,7 @@ void tsx_init(void)setup_force_cpu_cap (X86_FEATURE_RTM ); } } + done_probe:/* * Note: MSR_TSX_CTRL is enumerated on TSX-enabled MDS_NO and later parts.
base-commit: 270588b9b2b751b0bb6b36f4853cb13005e4706f
Signature · 1 line
2.30.2
Re: [PATCH v2] x86/tsx: Cope with RTM_ALWAYS_ABORT vs RTM mismatch
Quoted 9 lines · Andrew Cooper
It turns out there is something wonky on some but not all CPUs with
MSR_TSX_FORCE_ABORT. The presence ofRTM_ALWAYS_ABORTcauses Xen to think it's safe to offer HLE/RTM to guests, but in this case, XBEGIN instructions genuinely #UD.Spot this case and try to back out as cleanly as we can.
Signed-off-by Andrew Cooper <[email protected]>