clocksource: hyper-v: Use lapic timer in a TDX VM without paravisor
Subsystems (5)
Dexuan Cui
maintainerPATCH 1/1 clocksource: hyper-v: Use lapic timer in a TDX VM without paravisor
In a TDX VM without paravisor, currently the default timer is the Hyper-V timer, which depends on the slow VM Reference Counter MSR: the Hyper-V TSC page is not enabled in such a VM because the VM uses Invariant TSC as a better clocksource and it's challenging to mark the Hyper-V TSC page shared in very early boot.
Lower the rating of the Hyper-V timer so the local APIC timer becomes the the default timer in such a VM. This change should cause no perceivable performance difference.
--- arch/x86/kernel/cpu/mshyperv.c | 6 +++++- drivers/clocksource/hyperv_timer.c | 16 +++++++++++++++- 2 files changed, 20 insertions(+), 2 deletions(-)
arch/x86/kernel/cpu/mshyperv.c +5 -1
@@ -449,9 +449,13 @@ static void __init ms_hyperv_init_platform(void) ms_hyperv.hints &= ~HV_X64_APIC_ACCESS_RECOMMENDED ;if (!ms_hyperv.paravisor_present) { -/* To be supported: more work is required. */ +/* Use Invariant TSC as a better clocksource. */ ms_hyperv.features &= ~HV_MSR_REFERENCE_TSC_AVAILABLE ; +/* Use the Ref Counter in case Invariant TSC is unavailable. */ +if (!(ms_hyperv.features &HV_ACCESS_TSC_INVARIANT )) +pr_warn ("Hyper-V: Invariant TSC is unavailable\n" ); +/* HV_MSR_CRASH_CTL is unsupported. */ ms_hyperv.misc_features &= ~HV_FEATURE_GUEST_CRASH_MSR_AVAILABLE ;
drivers/clocksource/hyperv_timer.c +15 -1
@@ -137,7 +137,21 @@ static int hv_stimer_init(unsigned int cpu) ce->name ="Hyper-V clockevent" ; ce->features =CLOCK_EVT_FEAT_ONESHOT ; ce->cpumask =cpumask_of (cpu); - ce->rating =1000 ; + +/* +* Lower the rating of the Hyper-V timer in a TDX VM without paravisor, +* so the local APIC timer (lapic_clockevent) is the default timer in +* such a VM. The Hyper-V timer is not preferred in such a VM because +* it depends on the slow VM Reference Counter MSR (the Hyper-V TSC +* page is not enbled in such a VM because the VM uses Invariant TSC +* as a better clocksource and it's challenging to mark the Hyper-V +* TSC page shared in very early boot). +*/ +if (!ms_hyperv.paravisor_present &&hv_isolation_type_tdx ()) + ce->rating =90 ; +else + ce->rating =1000 ; + ce->set_state_shutdown = hv_ce_shutdown; ce->set_state_oneshot = hv_ce_set_oneshot; ce->set_next_event = hv_ce_set_next_event;
Signature · 1 line
2.25.1
Quoted 53 lines · Dexuan Cui
In a TDX VM without paravisor, currently the default timer is the Hyper-V timer, which depends on the slow VM Reference Counter MSR: the Hyper-V TSC page is not enabled in such a VM because the VM uses Invariant TSC as a better clocksource and it's challenging to mark the Hyper-V TSC page shared in very early boot.
Lower the rating of the Hyper-V timer so the local APIC timer becomes the the default timer in such a VM. This change should cause no perceivable performance difference.
Cc [email protected] # 6.6+Signed-off-by Dexuan Cui <[email protected]>--- arch/x86/kernel/cpu/mshyperv.c | 6 +++++- drivers/clocksource/hyperv_timer.c | 16 +++++++++++++++- 2 files changed, 20 insertions(+), 2 deletions(-)arch/x86/kernel/cpu/mshyperv.c +5
-1@@ -449,9 +449,13 @@ static void __init ms_hyperv_init_platform(void) ms_hyperv.hints &= ~HV_X64_APIC_ACCESS_RECOMMENDED ;if (!ms_hyperv.paravisor_present) { -/* To be supported: more work is required. */ +/* Use Invariant TSC as a better clocksource. */ ms_hyperv.features &= ~HV_MSR_REFERENCE_TSC_AVAILABLE ; +/* Use the Ref Counter in case Invariant TSC is unavailable. */ +if (!(ms_hyperv.features &HV_ACCESS_TSC_INVARIANT )) +pr_warn ("Hyper-V: Invariant TSC is unavailable\n" ); +/* HV_MSR_CRASH_CTL is unsupported. */ ms_hyperv.misc_features &= ~HV_FEATURE_GUEST_CRASH_MSR_AVAILABLE ;drivers/clocksource/hyperv_timer.c +8
-1@@ -137,7 +137,21 @@ static int hv_stimer_init(unsigned int cpu) ce->name ="Hyper-V clockevent" ; ce->features =CLOCK_EVT_FEAT_ONESHOT ; ce->cpumask =cpumask_of (cpu); - ce->rating =1000 ; + +/* +* Lower the rating of the Hyper-V timer in a TDX VM without paravisor, +* so the local APIC timer (lapic_clockevent) is the default timer in +* such a VM. The Hyper-V timer is not preferred in such a VM because +* it depends on the slow VM Reference Counter MSR (the Hyper-V TSC +* page is not enbled in such a VM because the VM uses Invariant TSC +* as a better clocksource and it's challenging to mark the Hyper-V
drivers/clocksource/hyperv_timer.c static int hv_stimer_init(unsigned int cpu) · line 157
Quoted 10 lines
drivers/clocksource/hyperv_timer.c +7
-0+* TSC page shared in very early boot). +*/ +if (!ms_hyperv.paravisor_present &&hv_isolation_type_tdx ()) + ce->rating =90 ; +else + ce->rating =1000 ; + ce->set_state_shutdown = hv_ce_shutdown; ce->set_state_oneshot = hv_ce_set_oneshot; ce->set_next_event = hv_ce_set_next_event;
LGTM.
Signature · 2 lines
Thank you, Roman
From: Dexuan Cui <[email protected]> Sent: Tuesday, June 18, 2024 5:25 PM
Quoted 23 lines
In a TDX VM without paravisor, currently the default timer is the Hyper-V timer, which depends on the slow VM Reference Counter MSR: the Hyper-V TSC page is not enabled in such a VM because the VM uses Invariant TSC as a better clocksource and it's challenging to mark the Hyper-V TSC page shared in very early boot.
Lower the rating of the Hyper-V timer so the local APIC timer becomes the the default timer in such a VM. This change should cause no perceivable performance difference.
Cc [email protected] # 6.6+Signed-off-by Dexuan Cui <[email protected]>--- arch/x86/kernel/cpu/mshyperv.c | 6 +++++- drivers/clocksource/hyperv_timer.c | 16 +++++++++++++++- 2 files changed, 20 insertions(+), 2 deletions(-)arch/x86/kernel/cpu/mshyperv.c +0
-0
arch/x86/kernel/cpu/mshyperv.c static void __init ms_hyperv_init_platform(void) · line 452
Quoted 5 lines
arch/x86/kernel/cpu/mshyperv.c +1
-1@@ -449,9 +449,13 @@ static void __init ms_hyperv_init_platform(void) ms_hyperv.hints &= ~HV_X64_APIC_ACCESS_RECOMMENDED ;if (!ms_hyperv.paravisor_present) { -/* To be supported: more work is required. */ +/* Use Invariant TSC as a better clocksource. */
I got confused by this comment, partly because I've forgotten the meaning of the ms_hyperv.feature flags. :-( Perhaps you could be more explicit in the comment and say "Mark the Hyper-V TSC page feature as disabled in a TDX VM so that the Invariant TSC, which is a better clocksource anyway, is used instead."
arch/x86/kernel/cpu/mshyperv.c static void __init ms_hyperv_init_platform(void) · line 457
Quoted 5 lines
arch/x86/kernel/cpu/mshyperv.c +3
-0ms_hyperv.features &= ~HV_MSR_REFERENCE_TSC_AVAILABLE ; +/* Use the Ref Counter in case Invariant TSC is unavailable. */ +if (!(ms_hyperv.features &HV_ACCESS_TSC_INVARIANT )) +pr_warn ("Hyper-V: Invariant TSC is unavailable\n" );
The above comment was even more confusing, because the code block doesn't do anything except print a message. The code doesn't force the use of the Ref Counter. I'd suggest something like: "The Invariant TSC is expected to be available, but if not, print a warning message. The slower Hyper-V MSR-based Ref Counter should end up being the clocksource."
Michael
Quoted 34 lines
arch/x86/kernel/cpu/mshyperv.c +1
-0+/* HV_MSR_CRASH_CTL is unsupported. */ ms_hyperv.misc_features &= ~HV_FEATURE_GUEST_CRASH_MSR_AVAILABLE ;drivers/clocksource/hyperv_timer.c +15
-2@@ -137,7 +137,21 @@ static int hv_stimer_init(unsigned int cpu) ce->name ="Hyper-V clockevent" ; ce->features =CLOCK_EVT_FEAT_ONESHOT ; ce->cpumask =cpumask_of (cpu); - ce->rating =1000 ; + +/* +* Lower the rating of the Hyper-V timer in a TDX VM without paravisor, +* so the local APIC timer (lapic_clockevent) is the default timer in +* such a VM. The Hyper-V timer is not preferred in such a VM because +* it depends on the slow VM Reference Counter MSR (the Hyper-V TSC +* page is not enbled in such a VM because the VM uses Invariant TSC +* as a better clocksource and it's challenging to mark the Hyper-V +* TSC page shared in very early boot). +*/ +if (!ms_hyperv.paravisor_present &&hv_isolation_type_tdx ()) + ce->rating =90 ; +else + ce->rating =1000 ; + ce->set_state_shutdown = hv_ce_shutdown; ce->set_state_oneshot = hv_ce_set_oneshot; ce->set_next_event = hv_ce_set_next_event; --2.25.1
Dexuan Cui
maintainerQuoted 32 lines
From: Michael Kelley <[email protected]>
Sent: Thursday, June 20, 2024 2:19 PM
To: Dexuan Cui <[email protected]>; KY Srinivasan [...]arch/x86/kernel/cpu/mshyperv.c static void __init ms_hyperv_init_platform(void) · line 449
Quoted 4 lines
arch/x86/kernel/cpu/mshyperv.c +0
-0@@ -449,9 +449,13 @@ static void __init ms_hyperv_init_platform(void) ms_hyperv.hints &=~HV_X64_APIC_ACCESS_RECOMMENDED;
arch/x86/kernel/cpu/mshyperv.c static void __init ms_hyperv_init_platform(void) · line 451
Quoted 3 lines
arch/x86/kernel/cpu/mshyperv.c +0
-1if (!ms_hyperv.paravisor_present) { -/* To be supported: more work is required. */
arch/x86/kernel/cpu/mshyperv.c static void __init ms_hyperv_init_platform(void) · line 451
Quoted 1 line
arch/x86/kernel/cpu/mshyperv.c +1
-0+/* Use Invariant TSC as a better clocksource. */
I got confused by this comment, partly because I've forgotten the meaning of the ms_hyperv.feature flags. :-( Perhaps you could be more explicit in the comment and say "Mark the Hyper-V TSC page feature as disabled in a TDX VM so that the Invariant TSC, which is a better clocksource anyway, is used instead."
arch/x86/kernel/cpu/mshyperv.c static void __init ms_hyperv_init_platform(void) · line 452
Quoted 1 line
arch/x86/kernel/cpu/mshyperv.c +0
-0ms_hyperv.features &=~HV_MSR_REFERENCE_TSC_AVAILABLE;
arch/x86/kernel/cpu/mshyperv.c static void __init ms_hyperv_init_platform(void) · line 453
Quoted 2 lines
arch/x86/kernel/cpu/mshyperv.c +1
-0+/* Use the Ref Counter in case Invariant TSC is unavailable. */
arch/x86/kernel/cpu/mshyperv.c static void __init ms_hyperv_init_platform(void) · line 454
Quoted 1 line
arch/x86/kernel/cpu/mshyperv.c +1
-0+ if (!(ms_hyperv.features &
HV_ACCESS_TSC_INVARIANT))arch/x86/kernel/cpu/mshyperv.c static void __init ms_hyperv_init_platform(void) · line 455
Quoted 1 line
arch/x86/kernel/cpu/mshyperv.c +1
-0+ pr_warn("Hyper-V: Invariant TSC isunavailable\n");
Quoted 8 lines
The above comment was even more confusing, because the code block doesn't do anything except print a message. The code doesn't force the use of the Ref Counter. I'd suggest something like: "The Invariant TSC is expected to be available, but if not, print a warning message. The slower Hyper-V MSR-based Ref Counter should end up being the clocksource."
Michael
Thanks for the good "comments"! :-)
I'm going to post v2 with the change below.
arch/x86/kernel/cpu/mshyperv.c +15 -1
@@ -449,9 +449,23 @@ static void __init ms_hyperv_init_platform(void) ms_hyperv.hints &= ~HV_X64_APIC_ACCESS_RECOMMENDED ;if (!ms_hyperv.paravisor_present) { -/* To be supported: more work is required. */ +/* +* Mark the Hyper-V TSC page feature as disabled +* in a TDX VM without paravisor so that the +* Invariant TSC, which is a better clocksource +* anyway, is used instead. +*/ ms_hyperv.features &= ~HV_MSR_REFERENCE_TSC_AVAILABLE ; +/* +* The Invariant TSC is expected to be available +* in a TDX VM without paravisor, but if not, +* print a warning message. The slower Hyper-V MSR-based +* Ref Counter should end up being the clocksource. +*/ +if (!(ms_hyperv.features &HV_ACCESS_TSC_INVARIANT )) +pr_warn ("Hyper-V: Invariant TSC is unavailable\n" ); +/* HV_MSR_CRASH_CTL is unsupported. */ ms_hyperv.misc_features &= ~HV_FEATURE_GUEST_CRASH_MSR_AVAILABLE ;