i386: Hyper-V nested sleep states enlightenment
PATCH 1/1 i386: Hyper-V nested sleep states enlightenment
Hyper-V running nested on KVM (e.g. Windows with Virtualization-Based Security) allows hibernation of its root partition only if the parent hypervisor sets bit 20 of HV_CPUID_ENLIGHTMENT_INFO.EAX. KVM never sets it, so such guests report "The hypervisor does not support hibernation".
Add a "hv-nested-sleep" property that sets the bit. It is a hint only and needs no KVM support. With it, Windows 11 26H2 with VBS, HVCI and Credential Guard hibernates and resumes.
The nested hypervisor uses the bit for one more thing, on resume:
instead of writing IA32_TSC itself, it issues hypercall 0x103 with the TSC and the parent reference time saved at suspend, asking the parent to restore its TSC. KVM returns HV_STATUS_INVALID_HYPERCALL_CODE; the nested hypervisor ignores the status and already compensates the TSC jump in software, so the guest clock stays correct.
--- docs/system/i386/hyperv.rst | 10 ++++++++++ target/i386/cpu.c | 1 + target/i386/cpu.h | 1 + target/i386/kvm/hyperv-proto.h | 1 + target/i386/kvm/kvm.c | 5 +++++ 5 files changed, 18 insertions(+)
docs/system/i386/hyperv.rst +10 -0
@@ -187,6 +187,16 @@ Existing enlightenments Note: enabling this feature on old hardware (without APICv/AVIC support) may have negative effect on guest's performance. +``hv-nested-sleep`` + Tells a guest that runs its own Hyper-V (Windows with Virtualization-Based + Security, or with the Hyper-V role) that sleep states are supported for a + nested hypervisor (CPUID 0x40000004.EAX bit 20). Without it, Hyper-V running + on top of KVM disables hibernation (S4) for its root partition: Windows + reports "The hypervisor does not support hibernation", and an S4 attempt + makes the nested hypervisor reset the machine. No KVM support is needed: the + bit is a hint only. On resume the nested hypervisor issues a hypercall that + KVM does not implement; the nested hypervisor ignores the failure. + ``hv-no-nonarch-coresharing`` = on/off/auto This enlightenment tells guest OS that virtual processors will never share a physical core unless they are reported as sibling SMT threads. This information
target/i386/cpu.c +1 -0
@@ -10797,6 +10797,7 @@ static const Property x86_cpu_properties[] = { HYPERV_FEAT_TLBFLUSH_DIRECT, 0), DEFINE_PROP_ON_OFF_AUTO("hv-no-nonarch-coresharing", X86CPU, hyperv_no_nonarch_cs, ON_OFF_AUTO_OFF), + DEFINE_PROP_BOOL("hv-nested-sleep", X86CPU, hyperv_nested_sleep, false),#ifdef CONFIG_SYNDBG DEFINE_PROP_BIT64 ("hv-syndbg" ,X86CPU , hyperv_features, HYPERV_FEAT_SYNDBG, 0),
target/i386/cpu.h +1 -0
@@ -2374,6 +2374,7 @@ struct ArchCPU {uint64_t hyperv_features;bool hyperv_passthrough;OnOffAuto hyperv_no_nonarch_cs; +bool hyperv_nested_sleep;uint32_t hyperv_vendor_id[3 ];uint32_t hyperv_interface_id[4 ];uint32_t hyperv_limits[3 ];
target/i386/kvm/hyperv-proto.h +1 -0
@@ -81,6 +81,7 @@#define HV_EX_PROCESSOR_MASKS_RECOMMENDED (1u << 11)#define HV_ENLIGHTENED_VMCS_RECOMMENDED (1u << 14)#define HV_NO_NONARCH_CORESHARING (1u << 18) +#define HV_NESTED_SLEEP_SUPPORTED (1u << 20)/* * HV_CPUID_SYNDBG_PLATFORM_CAPABILITIES.EAX bits
target/i386/kvm/kvm.c +5 -0
@@ -1630,6 +1630,11 @@ static int hyperv_fill_cpuids(CPUState *cs,HV_NO_NONARCH_CORESHARING ; } +/* Only a hint for a nested Hyper-V: no KVM support is needed */ +if (cpu->hyperv_nested_sleep) { + c->eax |=HV_NESTED_SLEEP_SUPPORTED ; + } + c = &cpuid_ent[cpuid_i++]; c->function =HV_CPUID_IMPLEMENT_LIMITS ; c->eax = cpu->hv_max_vps;
Signature · 1 line
2.56.0
Quoted 9 lines · Andrea Oliveri
Hyper-V running nested on KVM (e.g. Windows with Virtualization-Based Security) allows hibernation of its root partition only if the parent hypervisor sets bit 20 of HV_CPUID_ENLIGHTMENT_INFO.EAX. KVM never sets it, so such guests report "The hypervisor does not support hibernation".
Add a "hv-nested-sleep" property that sets the bit. It is a hint only and needs no KVM support. With it, Windows 11 26H2 with VBS, HVCI and Credential Guard hibernates and resumes.
Quoted 8 lines
The nested hypervisor uses the bit for one more thing, on resume:
instead of writingIA32_TSCitself, it issues hypercall0x103with the TSC and the parent reference time saved at suspend, asking the parent to restore its TSC. KVM returnsHV_STATUS_INVALID_HYPERCALL_CODE; the nested hypervisor ignores the status and already compensates the TSC jump in software, so the guest clock stays correct.Signed-off-by Andrea Oliveri <[email protected]>
Please take a look at the competing proposal:
QEMU:
https://lore.kernel.org/qemu-devel/[email protected]/
KVM: https://lore.kernel.org/kvm/[email protected]/
Quoted 81 lines
--- docs/system/i386/hyperv.rst | 10 ++++++++++ target/i386/cpu.c | 1 + target/i386/cpu.h | 1 + target/i386/kvm/hyperv-proto.h | 1 + target/i386/kvm/kvm.c | 5 +++++ 5 files changed, 18 insertions(+)docs/system/i386/hyperv.rst +10
-0@@ -187,6 +187,16 @@ Existing enlightenments Note: enabling this feature on old hardware (without APICv/AVIC support) may have negative effect on guest's performance. +``hv-nested-sleep`` + Tells a guest that runs its own Hyper-V (Windows with Virtualization-Based + Security, or with the Hyper-V role) that sleep states are supported for a + nested hypervisor (CPUID 0x40000004.EAX bit 20). Without it, Hyper-V running + on top of KVM disables hibernation (S4) for its root partition: Windows + reports "The hypervisor does not support hibernation", and an S4 attempt + makes the nested hypervisor reset the machine. No KVM support is needed: the + bit is a hint only. On resume the nested hypervisor issues a hypercall that + KVM does not implement; the nested hypervisor ignores the failure. + ``hv-no-nonarch-coresharing`` = on/off/auto This enlightenment tells guest OS that virtual processors will never share a physical core unless they are reported as sibling SMT threads. This informationtarget/i386/cpu.c +1
-0@@ -10797,6 +10797,7 @@ static const Property x86_cpu_properties[] = { HYPERV_FEAT_TLBFLUSH_DIRECT, 0), DEFINE_PROP_ON_OFF_AUTO("hv-no-nonarch-coresharing", X86CPU, hyperv_no_nonarch_cs, ON_OFF_AUTO_OFF), + DEFINE_PROP_BOOL("hv-nested-sleep", X86CPU, hyperv_nested_sleep, false),#ifdef CONFIG_SYNDBG DEFINE_PROP_BIT64 ("hv-syndbg" ,X86CPU , hyperv_features, HYPERV_FEAT_SYNDBG, 0),target/i386/cpu.h +1
-0@@ -2374,6 +2374,7 @@ struct ArchCPU {uint64_t hyperv_features;bool hyperv_passthrough;OnOffAuto hyperv_no_nonarch_cs; +bool hyperv_nested_sleep;uint32_t hyperv_vendor_id[3 ];uint32_t hyperv_interface_id[4 ];uint32_t hyperv_limits[3 ];target/i386/kvm/hyperv-proto.h +1
-0@@ -81,6 +81,7 @@#define HV_EX_PROCESSOR_MASKS_RECOMMENDED (1u << 11)#define HV_ENLIGHTENED_VMCS_RECOMMENDED (1u << 14)#define HV_NO_NONARCH_CORESHARING (1u << 18) +#define HV_NESTED_SLEEP_SUPPORTED (1u << 20)/* * HV_CPUID_SYNDBG_PLATFORM_CAPABILITIES.EAX bits target/i386/kvm/kvm.c +5
-0@@ -1630,6 +1630,11 @@ static int hyperv_fill_cpuids(CPUState *cs,HV_NO_NONARCH_CORESHARING ; } +/* Only a hint for a nested Hyper-V: no KVM support is needed */ +if (cpu->hyperv_nested_sleep) { + c->eax |=HV_NESTED_SLEEP_SUPPORTED ; + } + c = &cpuid_ent[cpuid_i++]; c->function =HV_CPUID_IMPLEMENT_LIMITS ; c->eax = cpu->hv_max_vps;
Signature · 1 line
Vitaly
Oh, I didn't see these two patches. They patch exact the same bug as mine. Sorry, please retire mine and continue with the existing ones (that patches also KVM hypercall side).
Sorry again
Andrea
Il giorno lun 19 ott 2026 alle ore 11:11 Vitaly Kuznetsov <[email protected]> ha scritto:
Quoted 112 lines
Quoted 9 lines · Andrea Oliveri
Hyper-V running nested on KVM (e.g. Windows with Virtualization-Based Security) allows hibernation of its root partition only if the parent hypervisor sets bit 20 of HV_CPUID_ENLIGHTMENT_INFO.EAX. KVM never sets it, so such guests report "The hypervisor does not support hibernation".
Add a "hv-nested-sleep" property that sets the bit. It is a hint only and needs no KVM support. With it, Windows 11 26H2 with VBS, HVCI and Credential Guard hibernates and resumes.
Quoted 8 lines
The nested hypervisor uses the bit for one more thing, on resume:
instead of writingIA32_TSCitself, it issues hypercall0x103with the TSC and the parent reference time saved at suspend, asking the parent to restore its TSC. KVM returnsHV_STATUS_INVALID_HYPERCALL_CODE; the nested hypervisor ignores the status and already compensates the TSC jump in software, so the guest clock stays correct.Signed-off-by Andrea Oliveri <[email protected]>Please take a look at the competing proposal:
QEMU:
https://lore.kernel.org/qemu-devel/[email protected]/
KVM: https://lore.kernel.org/kvm/[email protected]/Quoted 71 lines
--- docs/system/i386/hyperv.rst | 10 ++++++++++ target/i386/cpu.c | 1 + target/i386/cpu.h | 1 + target/i386/kvm/hyperv-proto.h | 1 + target/i386/kvm/kvm.c | 5 +++++ 5 files changed, 18 insertions(+)docs/system/i386/hyperv.rst +10
-0@@ -187,6 +187,16 @@ Existing enlightenments Note: enabling this feature on old hardware (without APICv/AVIC support) may have negative effect on guest's performance. +``hv-nested-sleep`` + Tells a guest that runs its own Hyper-V (Windows with Virtualization-Based + Security, or with the Hyper-V role) that sleep states are supported for a + nested hypervisor (CPUID 0x40000004.EAX bit 20). Without it, Hyper-V running + on top of KVM disables hibernation (S4) for its root partition: Windows + reports "The hypervisor does not support hibernation", and an S4 attempt + makes the nested hypervisor reset the machine. No KVM support is needed: the + bit is a hint only. On resume the nested hypervisor issues a hypercall that + KVM does not implement; the nested hypervisor ignores the failure. + ``hv-no-nonarch-coresharing`` = on/off/auto This enlightenment tells guest OS that virtual processors will never share a physical core unless they are reported as sibling SMT threads. This informationtarget/i386/cpu.c +1
-0@@ -10797,6 +10797,7 @@ static const Property x86_cpu_properties[] = { HYPERV_FEAT_TLBFLUSH_DIRECT, 0), DEFINE_PROP_ON_OFF_AUTO("hv-no-nonarch-coresharing", X86CPU, hyperv_no_nonarch_cs, ON_OFF_AUTO_OFF), + DEFINE_PROP_BOOL("hv-nested-sleep", X86CPU, hyperv_nested_sleep, false),#ifdef CONFIG_SYNDBG DEFINE_PROP_BIT64 ("hv-syndbg" ,X86CPU , hyperv_features, HYPERV_FEAT_SYNDBG, 0),target/i386/cpu.h +1
-0@@ -2374,6 +2374,7 @@ struct ArchCPU {uint64_t hyperv_features;bool hyperv_passthrough;OnOffAuto hyperv_no_nonarch_cs; +bool hyperv_nested_sleep;uint32_t hyperv_vendor_id[3 ];uint32_t hyperv_interface_id[4 ];uint32_t hyperv_limits[3 ];target/i386/kvm/hyperv-proto.h +1
-0@@ -81,6 +81,7 @@#define HV_EX_PROCESSOR_MASKS_RECOMMENDED (1u << 11)#define HV_ENLIGHTENED_VMCS_RECOMMENDED (1u << 14)#define HV_NO_NONARCH_CORESHARING (1u << 18) +#define HV_NESTED_SLEEP_SUPPORTED (1u << 20)/* * HV_CPUID_SYNDBG_PLATFORM_CAPABILITIES.EAX bits target/i386/kvm/kvm.c +0
-0@@ -1630,6 +1630,11 @@ static int hyperv_fill_cpuids(CPUState *cs,HV_NO_NONARCH_CORESHARING ;target/i386/kvm/kvm.c static int hyperv_fill_cpuids(CPUState *cs, · line 1640
Quoted 10 lines
target/i386/kvm/kvm.c +5
-0} +/* Only a hint for a nested Hyper-V: no KVM support is needed */ +if (cpu->hyperv_nested_sleep) { + c->eax |=HV_NESTED_SLEEP_SUPPORTED ; + } + c = &cpuid_ent[cpuid_i++]; c->function =HV_CPUID_IMPLEMENT_LIMITS ; c->eax = cpu->hv_max_vps;--
Vitaly