i386: KVM: expand Hyper-V features early and provide simple 'hv-default=on' option
v3 cover i386: KVM: expand Hyper-V features early and provide simple 'hv-default=on' option
This series is a successor of "[PATCH RFC v3 00/23] i386: KVM: expand Hyper-V features early" and "[PATCH v2 2/2] i386: provide simple 'hyperv=on'" option to x86 machine types".
Changes:
- Make 'hv-default' a CPU option and not a machine type option [Igor].
- Introduce a simple qtest [Eduardo].
- Update linux headers from 5.11-rc2.
Description:
Upper layer tools like libvirt want to figure out which Hyper-V features are supported by the underlying stack (QEMU/KVM) but currently they are unable to do so. We have a nice 'hv_passthrough' CPU flag supported by QEMU but it has no effect on e.g. QMP's
query-cpu-model-expansion type=full model={"name":"host","props":{"hv-passthrough":true}}
command as we parse Hyper-V features after creating KVM vCPUs and not at feature expansion time. To support the use-case we first need to make KVM_GET_SUPPORTED_HV_CPUID ioctl a system-wide ioctl as the existing vCPU version can't be used that early. This is what KVM part does. With that done, we can make early Hyper-V feature expansion (this series).
In addition, provide a simple 'hv-default' option which enables (and requires from KVM) all currently supported Hyper-V enlightenments. Unlike 'hv_passthrough' mode, this is going to be migratable.
Vitaly Kuznetsov (19):
linux-headers: update against 5.11-rc2
i386: introduce kvm_hv_evmcs_available()
i386: keep hyperv_vendor string up-to-date
i386: invert hyperv_spinlock_attempts setting logic with
hv_passthrough
i386: always fill Hyper-V CPUID feature leaves from X86CPU data
i386: stop using env->features[] for filling Hyper-V CPUIDs
i386: introduce hyperv_feature_supported()
i386: introduce hv_cpuid_get_host()
i386: drop FEAT_HYPERV feature leaves
i386: introduce hv_cpuid_cache
i386: split hyperv_handle_properties() into
hyperv_expand_features()/hyperv_fill_cpuids()
i386: move eVMCS enablement to hyperv_init_vcpu()
i386: switch hyperv_expand_features() to using error_setg()
i386: adjust the expected KVM_GET_SUPPORTED_HV_CPUID array size
i386: prefer system KVM_GET_SUPPORTED_HV_CPUID ioctl over vCPU's one
i386: use global kvm_state in hyperv_enabled() check
i386: expand Hyper-V features during CPU feature expansion time
i386: provide simple 'hv-default=on' option
qtest/hyperv: Introduce a simple hyper-v test
MAINTAINERS | 1 + docs/hyperv.txt | 16 +- .../infiniband/hw/vmw_pvrdma/pvrdma_verbs.h | 2 +- include/standard-headers/drm/drm_fourcc.h | 175 +++++- include/standard-headers/linux/const.h | 36 ++ include/standard-headers/linux/ethtool.h | 2 +- include/standard-headers/linux/fuse.h | 30 +- include/standard-headers/linux/kernel.h | 9 +- include/standard-headers/linux/pci_regs.h | 16 + include/standard-headers/linux/vhost_types.h | 9 + include/standard-headers/linux/virtio_gpu.h | 82 +++ include/standard-headers/linux/virtio_ids.h | 44 +- linux-headers/asm-arm64/kvm.h | 3 - linux-headers/asm-generic/unistd.h | 6 +- linux-headers/asm-mips/unistd_n32.h | 1 + linux-headers/asm-mips/unistd_n64.h | 1 + linux-headers/asm-mips/unistd_o32.h | 1 + linux-headers/asm-powerpc/unistd_32.h | 1 + linux-headers/asm-powerpc/unistd_64.h | 1 + linux-headers/asm-s390/unistd_32.h | 1 + linux-headers/asm-s390/unistd_64.h | 1 + linux-headers/asm-x86/kvm.h | 1 + linux-headers/asm-x86/unistd_32.h | 1 + linux-headers/asm-x86/unistd_64.h | 1 + linux-headers/asm-x86/unistd_x32.h | 1 + linux-headers/linux/kvm.h | 56 +- linux-headers/linux/userfaultfd.h | 9 + linux-headers/linux/vfio.h | 1 + linux-headers/linux/vhost.h | 4 + scripts/update-linux-headers.sh | 5 +- target/i386/cpu.c | 151 ++--- target/i386/cpu.h | 11 +- target/i386/kvm/kvm-stub.c | 10 + target/i386/kvm/kvm.c | 524 ++++++++++-------- target/i386/kvm/kvm_i386.h | 2 + tests/qtest/hyperv-test.c | 238 ++++++++ tests/qtest/meson.build | 3 +- 37 files changed, 1074 insertions(+), 382 deletions(-) create mode 100644 include/standard-headers/linux/const.h create mode 100644 tests/qtest/hyperv-test.c
Signature · 1 line
2.29.2
PATCH v3 1/19 linux-headers: update against 5.11-rc2
commit e71ba9452f0b5b2e8dc8aa5445198cd9214a6a62
<linux/const.h> needs to be included as some constants were moved from <linux/kernel.h>.
--- .../infiniband/hw/vmw_pvrdma/pvrdma_verbs.h | 2 +- include/standard-headers/drm/drm_fourcc.h | 175 +++++++++++++++++- include/standard-headers/linux/const.h | 36 ++++ include/standard-headers/linux/ethtool.h | 2 +- include/standard-headers/linux/fuse.h | 30 ++- include/standard-headers/linux/kernel.h | 9 +- include/standard-headers/linux/pci_regs.h | 16 ++ include/standard-headers/linux/vhost_types.h | 9 + include/standard-headers/linux/virtio_gpu.h | 82 ++++++++ include/standard-headers/linux/virtio_ids.h | 44 +++-- linux-headers/asm-arm64/kvm.h | 3 - linux-headers/asm-generic/unistd.h | 6 +- linux-headers/asm-mips/unistd_n32.h | 1 + linux-headers/asm-mips/unistd_n64.h | 1 + linux-headers/asm-mips/unistd_o32.h | 1 + linux-headers/asm-powerpc/unistd_32.h | 1 + linux-headers/asm-powerpc/unistd_64.h | 1 + linux-headers/asm-s390/unistd_32.h | 1 + linux-headers/asm-s390/unistd_64.h | 1 + linux-headers/asm-x86/kvm.h | 1 + linux-headers/asm-x86/unistd_32.h | 1 + linux-headers/asm-x86/unistd_64.h | 1 + linux-headers/asm-x86/unistd_x32.h | 1 + linux-headers/linux/kvm.h | 56 +++++- linux-headers/linux/userfaultfd.h | 9 + linux-headers/linux/vfio.h | 1 + linux-headers/linux/vhost.h | 4 + scripts/update-linux-headers.sh | 5 +- 28 files changed, 458 insertions(+), 42 deletions(-) create mode 100644 include/standard-headers/linux/const.h
include/standard-headers/drivers/infiniband/hw/vmw_pvrdma/pvrdma_verbs.h +1 -1
@@ -176,7 +176,7 @@ struct pvrdma_port_attr {uint8_t subnet_timeout;uint8_t init_type_reply;uint8_t active_width; -uint16_t active_speed; +uint8_t active_speed;uint8_t phys_state;uint8_t reserved[2 ]; };
include/standard-headers/drm/drm_fourcc.h +174 -1
205 lines not on this page · raw message
include/standard-headers/linux/const.h new file +36 -0
@@ -0,0 +1,36 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +/* const.h: Macros for dealing with constants. */ + +#ifndef _LINUX_CONST_H +#define _LINUX_CONST_H + +/* Some constant macros are used in both assembler and +* C code. Therefore we cannot annotate them always with +* 'UL' and other type specifiers unilaterally. We +* use the following macros to deal with this. +* +* Similarly, _AT() will cast an expression with a type in C, but +* leave it unchanged in asm. +*/ + +#ifdef __ASSEMBLY__ +#define _AC (X ,Y ) X +#define _AT (T ,X ) X +#else +#define __AC (X ,Y ) (X##Y) +#define _AC (X ,Y ) __AC(X,Y) +#define _AT (T ,X ) ((T)(X)) +#endif + +#define _UL (x) (_AC(x, UL)) +#define _ULL (x) (_AC(x, ULL)) + +#define _BITUL (x) (_UL(1) << (x)) +#define _BITULL (x) (_ULL(1) << (x)) + +#define __ALIGN_KERNEL (x, a) __ALIGN_KERNEL_MASK(x, (typeof(x))(a) - 1) +#define __ALIGN_KERNEL_MASK (x, mask) (((x) + (mask)) & ~(mask)) + +#define __KERNEL_DIV_ROUND_UP (n, d) (((n) + (d) - 1) / (d)) + +#endif /* _LINUX_CONST_H */
include/standard-headers/linux/ethtool.h +1 -1
@@ -16,7 +16,7 @@#include "net/eth.h" -#include "standard-headers/linux/kernel.h" +#include "standard-headers/linux/const.h" #include "standard-headers/linux/types.h" #include "standard-headers/linux/if_ether.h"
include/standard-headers/linux/fuse.h +25 -5
@@ -175,6 +175,10 @@* * 7.32 * - add flags to fuse_attr, add FUSE_ATTR_SUBMOUNT, add FUSE_SUBMOUNTS +* +* 7.33 +* - add FUSE_HANDLE_KILLPRIV_V2, FUSE_WRITE_KILL_SUIDGID, FATTR_KILL_SUIDGID +* - add FUSE_OPEN_KILL_SUIDGID */ #ifndef _LINUX_FUSE_H
@@ -206,7 +210,7 @@#define FUSE_KERNEL_VERSION 7/** Minor version number of this interface */ -#define FUSE_KERNEL_MINOR_VERSION 32 +#define FUSE_KERNEL_MINOR_VERSION 33/** The node ID of the root inode */ #define FUSE_ROOT_ID 1
@@ -267,6 +271,7 @@ struct fuse_file_lock {#define FATTR_MTIME_NOW (1 << 8)#define FATTR_LOCKOWNER (1 << 9)#define FATTR_CTIME (1 << 10) +#define FATTR_KILL_SUIDGID (1 << 11)/** * Flags returned by the OPEN request
@@ -316,6 +321,11 @@ struct fuse_file_lock {* foffset and moffset fields in struct * fuse_setupmapping_out and fuse_removemapping_one. * FUSE_SUBMOUNTS: kernel supports auto-mounting directory submounts +* FUSE_HANDLE_KILLPRIV_V2: fs kills suid/sgid/cap on write/chown/trunc. +* Upon write/truncate suid/sgid is only killed if caller +* does not have CAP_FSETID. Additionally upon +* write/truncate sgid is killed only if file has group +* execute permission. (Same as Linux VFS behavior). */ #define FUSE_ASYNC_READ (1 << 0)#define FUSE_POSIX_LOCKS (1 << 1)
@@ -345,6 +355,7 @@ struct fuse_file_lock {#define FUSE_EXPLICIT_INVAL_DATA (1 << 25)#define FUSE_MAP_ALIGNMENT (1 << 26)#define FUSE_SUBMOUNTS (1 << 27) +#define FUSE_HANDLE_KILLPRIV_V2 (1 << 28)/** * CUSE INIT request/reply flags
@@ -374,11 +385,14 @@ struct fuse_file_lock {* * FUSE_WRITE_CACHE: delayed write from page cache, file handle is guessed * FUSE_WRITE_LOCKOWNER: lock_owner field is valid -* FUSE_WRITE_KILL_PRIV: kill suid and sgid bits +* FUSE_WRITE_KILL_SUIDGID: kill suid and sgid bits */ #define FUSE_WRITE_CACHE (1 << 0)#define FUSE_WRITE_LOCKOWNER (1 << 1) -#define FUSE_WRITE_KILL_PRIV (1 << 2) +#define FUSE_WRITE_KILL_SUIDGID (1 << 2) + +/* Obsolete alias; this flag implies killing suid/sgid only. */ +#define FUSE_WRITE_KILL_PRIV FUSE_WRITE_KILL_SUIDGID/** * Read flags
@@ -427,6 +441,12 @@ struct fuse_file_lock {*/ #define FUSE_ATTR_SUBMOUNT (1 << 0) +/** +* Open flags +* FUSE_OPEN_KILL_SUIDGID: Kill suid and sgid if executable +*/ +#define FUSE_OPEN_KILL_SUIDGID (1 << 0) +enum fuse_opcode {FUSE_LOOKUP =1 ,FUSE_FORGET =2 ,/* no reply */
@@ -588,14 +608,14 @@ struct fuse_setattr_in {struct fuse_open_in {uint32_t flags; -uint32_t unused; +uint32_t open_flags;/* FUSE_OPEN_... */ };struct fuse_create_in {uint32_t flags;uint32_t mode;uint32_t umask; -uint32_t padding; +uint32_t open_flags;/* FUSE_OPEN_... */ };struct fuse_open_out {
include/standard-headers/linux/kernel.h +1 -8
@@ -3,13 +3,6 @@#define _LINUX_KERNEL_H#include "standard-headers/linux/sysinfo.h" - -/* -* 'kernel.h' contains some often-used function prototypes etc -*/ -#define __ALIGN_KERNEL (x, a) __ALIGN_KERNEL_MASK(x, (typeof(x))(a) - 1) -#define __ALIGN_KERNEL_MASK (x, mask) (((x) + (mask)) & ~(mask)) - -#define __KERNEL_DIV_ROUND_UP (n, d) (((n) + (d) - 1) / (d)) +#include "standard-headers/linux/const.h" #endif /* _LINUX_KERNEL_H */
include/standard-headers/linux/pci_regs.h +16 -0
@@ -531,6 +531,7 @@#define PCI_EXP_LNKCAP_SLS_8_0GB 0x00000003/* LNKCAP2 SLS Vector bit 2 */ #define PCI_EXP_LNKCAP_SLS_16_0GB 0x00000004/* LNKCAP2 SLS Vector bit 3 */ #define PCI_EXP_LNKCAP_SLS_32_0GB 0x00000005/* LNKCAP2 SLS Vector bit 4 */ +#define PCI_EXP_LNKCAP_SLS_64_0GB 0x00000006/* LNKCAP2 SLS Vector bit 5 */ #define PCI_EXP_LNKCAP_MLW 0x000003f0/* Maximum Link Width */ #define PCI_EXP_LNKCAP_ASPMS 0x00000c00/* ASPM Support */ #define PCI_EXP_LNKCAP_ASPM_L0S 0x00000400/* ASPM L0s Support */
@@ -562,6 +563,7 @@#define PCI_EXP_LNKSTA_CLS_8_0GB 0x0003/* Current Link Speed 8.0GT/s */ #define PCI_EXP_LNKSTA_CLS_16_0GB 0x0004/* Current Link Speed 16.0GT/s */ #define PCI_EXP_LNKSTA_CLS_32_0GB 0x0005/* Current Link Speed 32.0GT/s */ +#define PCI_EXP_LNKSTA_CLS_64_0GB 0x0006/* Current Link Speed 64.0GT/s */ #define PCI_EXP_LNKSTA_NLW 0x03f0/* Negotiated Link Width */ #define PCI_EXP_LNKSTA_NLW_X1 0x0010/* Current Link Width x1 */ #define PCI_EXP_LNKSTA_NLW_X2 0x0020/* Current Link Width x2 */
@@ -670,6 +672,7 @@#define PCI_EXP_LNKCAP2_SLS_8_0GB 0x00000008/* Supported Speed 8GT/s */ #define PCI_EXP_LNKCAP2_SLS_16_0GB 0x00000010/* Supported Speed 16GT/s */ #define PCI_EXP_LNKCAP2_SLS_32_0GB 0x00000020/* Supported Speed 32GT/s */ +#define PCI_EXP_LNKCAP2_SLS_64_0GB 0x00000040/* Supported Speed 64GT/s */ #define PCI_EXP_LNKCAP2_CROSSLINK 0x00000100/* Crosslink supported */ #define PCI_EXP_LNKCTL2 48/* Link Control 2 */ #define PCI_EXP_LNKCTL2_TLS 0x000f
@@ -678,6 +681,7 @@#define PCI_EXP_LNKCTL2_TLS_8_0GT 0x0003/* Supported Speed 8GT/s */ #define PCI_EXP_LNKCTL2_TLS_16_0GT 0x0004/* Supported Speed 16GT/s */ #define PCI_EXP_LNKCTL2_TLS_32_0GT 0x0005/* Supported Speed 32GT/s */ +#define PCI_EXP_LNKCTL2_TLS_64_0GT 0x0006/* Supported Speed 64GT/s */ #define PCI_EXP_LNKCTL2_ENTER_COMP 0x0010/* Enter Compliance */ #define PCI_EXP_LNKCTL2_TX_MARGIN 0x0380/* Transmit Margin */ #define PCI_EXP_LNKCTL2_HASD 0x0020/* HW Autonomous Speed Disable */
@@ -723,6 +727,7 @@#define PCI_EXT_CAP_ID_DPC 0x1D/* Downstream Port Containment */ #define PCI_EXT_CAP_ID_L1SS 0x1E/* L1 PM Substates */ #define PCI_EXT_CAP_ID_PTM 0x1F/* Precision Time Measurement */ +#define PCI_EXT_CAP_ID_DVSEC 0x23/* Designated Vendor-Specific */ #define PCI_EXT_CAP_ID_DLF 0x25/* Data Link Feature */ #define PCI_EXT_CAP_ID_PL_16GT 0x26/* Physical Layer 16.0 GT/s */ #define PCI_EXT_CAP_ID_MAX PCI_EXT_CAP_ID_PL_16GT
@@ -831,6 +836,13 @@#define PCI_PWR_CAP_BUDGET (x) ((x) & 1)/* Included in system budget */ #define PCI_EXT_CAP_PWR_SIZEOF 16 +/* Root Complex Event Collector Endpoint Association */ +#define PCI_RCEC_RCIEP_BITMAP 4/* Associated Bitmap for RCiEPs */ +#define PCI_RCEC_BUSN 8/* RCEC Associated Bus Numbers */ +#define PCI_RCEC_BUSN_REG_VER 0x02/* Least version with BUSN present */ +#define PCI_RCEC_BUSN_NEXT (x) (((x) >> 8) & 0xff) +#define PCI_RCEC_BUSN_LAST (x) (((x) >> 16) & 0xff) +/* Vendor-Specific (VSEC, PCI_EXT_CAP_ID_VNDR) */ #define PCI_VNDR_HEADER 4/* Vendor-Specific Header */ #define PCI_VNDR_HEADER_ID (x) ((x) & 0xffff)
@@ -1066,6 +1078,10 @@#define PCI_L1SS_CTL1_LTR_L12_TH_SCALE 0xe0000000/* LTR_L1.2_THRESHOLD_Scale */ #define PCI_L1SS_CTL2 0x0c/* Control 2 Register */ +/* Designated Vendor-Specific (DVSEC, PCI_EXT_CAP_ID_DVSEC) */ +#define PCI_DVSEC_HEADER1 0x4/* Designated Vendor-Specific Header1 */ +#define PCI_DVSEC_HEADER2 0x8/* Designated Vendor-Specific Header2 */ +/* Data Link Feature */ #define PCI_DLF_CAP 0x04/* Capabilities Register */ #define PCI_DLF_EXCHANGE_ENABLE 0x80000000/* Data Link Feature Exchange Enable */
include/standard-headers/linux/vhost_types.h +9 -0
@@ -138,6 +138,15 @@ struct vhost_vdpa_config {uint8_t buf[0 ]; }; +/* vhost vdpa IOVA range +* @first: First address that can be mapped by vhost-vDPA +* @last: Last address that can be mapped by vhost-vDPA +*/ +struct vhost_vdpa_iova_range { +uint64_t first; +uint64_t last; +}; +/* Feature bits */ /* Log all write descriptors. Can be changed while device is active. */ #define VHOST_F_LOG_ALL 26
include/standard-headers/linux/virtio_gpu.h +82 -0
116 lines not on this page · raw message
include/standard-headers/linux/virtio_ids.h +25 -19
49 lines not on this page · raw message
linux-headers/asm-arm64/kvm.h +0 -3
@@ -156,9 +156,6 @@ struct kvm_sync_regs {__u64 device_irq_level; }; -struct kvm_arch_memory_slot { -}; -/* * PMU filter structure. Describe a range of events with a particular * action. To be used with KVM_ARM_VCPU_PMU_V3_FILTER.
linux-headers/asm-generic/unistd.h +4 -2
@@ -517,7 +517,7 @@ __SC_COMP(__NR_settimeofday, sys_settimeofday, compat_sys_settimeofday) __SC_3264(__NR_adjtimex, sys_adjtimex_time32, sys_adjtimex)#endif -/* kernel/timer.c */ +/* kernel/sys.c */ #define __NR_getpid 172 __SYSCALL(__NR_getpid, sys_getpid)#define __NR_getppid 173
@@ -859,9 +859,11 @@ __SYSCALL(__NR_pidfd_getfd, sys_pidfd_getfd) __SYSCALL(__NR_faccessat2, sys_faccessat2)#define __NR_process_madvise 440 __SYSCALL(__NR_process_madvise, sys_process_madvise) +#define __NR_epoll_pwait2 441 +__SC_COMP(__NR_epoll_pwait2, sys_epoll_pwait2, compat_sys_epoll_pwait2)#undef __NR_syscalls -#define __NR_syscalls 441 +#define __NR_syscalls 442/* * 32 bit systems traditionally used different
linux-headers/asm-mips/unistd_n32.h +1 -0
@@ -370,6 +370,7 @@#define __NR_pidfd_getfd (__NR_Linux + 438)#define __NR_faccessat2 (__NR_Linux + 439)#define __NR_process_madvise (__NR_Linux + 440) +#define __NR_epoll_pwait2 (__NR_Linux + 441)#endif /* _ASM_MIPS_UNISTD_N32_H */
linux-headers/asm-mips/unistd_n64.h +1 -0
@@ -346,6 +346,7 @@#define __NR_pidfd_getfd (__NR_Linux + 438)#define __NR_faccessat2 (__NR_Linux + 439)#define __NR_process_madvise (__NR_Linux + 440) +#define __NR_epoll_pwait2 (__NR_Linux + 441)#endif /* _ASM_MIPS_UNISTD_N64_H */
linux-headers/asm-mips/unistd_o32.h +1 -0
@@ -416,6 +416,7 @@#define __NR_pidfd_getfd (__NR_Linux + 438)#define __NR_faccessat2 (__NR_Linux + 439)#define __NR_process_madvise (__NR_Linux + 440) +#define __NR_epoll_pwait2 (__NR_Linux + 441)#endif /* _ASM_MIPS_UNISTD_O32_H */
linux-headers/asm-powerpc/unistd_32.h +1 -0
@@ -423,6 +423,7 @@#define __NR_pidfd_getfd 438#define __NR_faccessat2 439#define __NR_process_madvise 440 +#define __NR_epoll_pwait2 441#endif /* _ASM_POWERPC_UNISTD_32_H */
linux-headers/asm-powerpc/unistd_64.h +1 -0
@@ -395,6 +395,7 @@#define __NR_pidfd_getfd 438#define __NR_faccessat2 439#define __NR_process_madvise 440 +#define __NR_epoll_pwait2 441#endif /* _ASM_POWERPC_UNISTD_64_H */
linux-headers/asm-s390/unistd_32.h +1 -0
@@ -413,5 +413,6 @@#define __NR_pidfd_getfd 438#define __NR_faccessat2 439#define __NR_process_madvise 440 +#define __NR_epoll_pwait2 441#endif /* _ASM_S390_UNISTD_32_H */
linux-headers/asm-s390/unistd_64.h +1 -0
@@ -361,5 +361,6 @@#define __NR_pidfd_getfd 438#define __NR_faccessat2 439#define __NR_process_madvise 440 +#define __NR_epoll_pwait2 441#endif /* _ASM_S390_UNISTD_64_H */
linux-headers/asm-x86/kvm.h +1 -0
@@ -12,6 +12,7 @@#define KVM_PIO_PAGE_OFFSET 1#define KVM_COALESCED_MMIO_PAGE_OFFSET 2 +#define KVM_DIRTY_LOG_PAGE_OFFSET 64#define DE_VECTOR 0#define DB_VECTOR 1
linux-headers/asm-x86/unistd_32.h +1 -0
@@ -431,6 +431,7 @@#define __NR_pidfd_getfd 438#define __NR_faccessat2 439#define __NR_process_madvise 440 +#define __NR_epoll_pwait2 441#endif /* _ASM_X86_UNISTD_32_H */
linux-headers/asm-x86/unistd_64.h +1 -0
@@ -353,6 +353,7 @@#define __NR_pidfd_getfd 438#define __NR_faccessat2 439#define __NR_process_madvise 440 +#define __NR_epoll_pwait2 441#endif /* _ASM_X86_UNISTD_64_H */
linux-headers/asm-x86/unistd_x32.h +1 -0
@@ -306,6 +306,7 @@#define __NR_pidfd_getfd (__X32_SYSCALL_BIT + 438)#define __NR_faccessat2 (__X32_SYSCALL_BIT + 439)#define __NR_process_madvise (__X32_SYSCALL_BIT + 440) +#define __NR_epoll_pwait2 (__X32_SYSCALL_BIT + 441)#define __NR_rt_sigaction (__X32_SYSCALL_BIT + 512)#define __NR_rt_sigreturn (__X32_SYSCALL_BIT + 513)#define __NR_ioctl (__X32_SYSCALL_BIT + 514)
linux-headers/linux/kvm.h +55 -1
84 lines not on this page · raw message
linux-headers/linux/userfaultfd.h +9 -0
@@ -257,4 +257,13 @@ struct uffdio_writeprotect { __u64 mode; }; +/* +* Flags for the userfaultfd(2) system call itself. +*/ + +/* +* Create a userfaultfd that can handle page faults only in user mode. +*/ +#define UFFD_USER_MODE_ONLY 1 +#endif /* _LINUX_USERFAULTFD_H */
linux-headers/linux/vfio.h +1 -0
@@ -820,6 +820,7 @@ enum {enum {VFIO_CCW_IO_IRQ_INDEX ,VFIO_CCW_CRW_IRQ_INDEX , +VFIO_CCW_REQ_IRQ_INDEX ,VFIO_CCW_NUM_IRQS };
linux-headers/linux/vhost.h +4 -0
@@ -146,4 +146,8 @@/* Set event fd for config interrupt*/ #define VHOST_VDPA_SET_CONFIG_CALL _IOW(VHOST_VIRTIO, 0x77, int) + +/* Get the valid iova range */ +#define VHOST_VDPA_GET_IOVA_RANGE _IOR(VHOST_VIRTIO, 0x78, \ + struct vhost_vdpa_iova_range)#endif
scripts/update-linux-headers.sh +4 -1
@@ -42,6 +42,7 @@ cp_portable() { -e 'drm.h' \ -e 'limits' \ -e 'linux/kernel' \ + -e 'linux/const' \ -e 'linux/sysinfo' \ -e 'asm-generic/kvm_para' \ > /dev/null
@@ -190,7 +191,9 @@ for i in "$tmpdir"/include/linux/*virtio*.h \ "$tmpdir/include/linux/input.h" \ "$tmpdir/include/linux/input-event-codes.h" \ "$tmpdir/include/linux/pci_regs.h" \ - "$tmpdir/include/linux/ethtool.h" "$tmpdir/include/linux/kernel.h" \ + "$tmpdir/include/linux/ethtool.h" \ + "$tmpdir/include/linux/kernel.h" \ + "$tmpdir/include/linux/const.h" \ "$tmpdir/include/linux/vhost_types.h" \ "$tmpdir/include/linux/sysinfo.h"; do cp_portable "$i" "$output/include/standard-headers/linux"
Signature · 1 line
2.29.2
PATCH v3 2/19 i386: introduce kvm_hv_evmcs_available()
In reply to Vitaly KuznetsovEnlightened VMCS feature is hardware specific, it is only supported on Intel CPUs. Introduce a simple kvm_hv_evmcs_available() helper, it will be used to filter out 'hv_evmcs' when 'hyperv=on' option is added to X86MachineClass.
--- target/i386/kvm/kvm-stub.c | 5 +++++ target/i386/kvm/kvm.c | 8 ++++++++ target/i386/kvm/kvm_i386.h | 1 + 3 files changed, 14 insertions(+)
target/i386/kvm/kvm-stub.c +5 -0
@@ -39,3 +39,8 @@ bool kvm_hv_vpindex_settable(void) {return false; } + +bool kvm_hv_evmcs_available (void ) +{ +return false; +}
target/i386/kvm/kvm.c +8 -0
@@ -95,6 +95,7 @@ static bool has_msr_hv_crash;static bool has_msr_hv_reset;static bool has_msr_hv_vpindex;static bool hv_vpindex_settable; +static bool hv_evmcs_available;static bool has_msr_hv_runtime;static bool has_msr_hv_synic;static bool has_msr_hv_stimer;
@@ -192,6 +193,11 @@ bool kvm_hv_vpindex_settable(void)return hv_vpindex_settable; } +bool kvm_hv_evmcs_available (void ) +{ +return hv_evmcs_available; +} +static int kvm_get_tsc (CPUState *cs) {X86CPU *cpu =X86_CPU (cs);
@@ -2146,6 +2152,8 @@ int kvm_arch_init(MachineState *ms, KVMState *s) has_pit_state2 =kvm_check_extension (s,KVM_CAP_PIT_STATE2 ); hv_vpindex_settable =kvm_check_extension (s,KVM_CAP_HYPERV_VP_INDEX ); + hv_evmcs_available = +kvm_check_extension (s,KVM_CAP_HYPERV_ENLIGHTENED_VMCS ); has_exception_payload =kvm_check_extension (s,KVM_CAP_EXCEPTION_PAYLOAD );if (has_exception_payload) {
target/i386/kvm/kvm_i386.h +1 -0
@@ -47,6 +47,7 @@ bool kvm_has_x2apic_api(void);bool kvm_has_waitpkg (void );bool kvm_hv_vpindex_settable (void ); +bool kvm_hv_evmcs_available (void );uint64_t kvm_swizzle_msi_ext_dest_id (uint64_t address);
Signature · 1 line
2.29.2
PATCH v3 3/19 i386: keep hyperv_vendor string up-to-date
In reply to Vitaly KuznetsovWhen cpu->hyperv_vendor is not set manually we default to "Microsoft Hv" and in 'hv_passthrough' mode we get the information from the host. This information is stored in cpu->hyperv_vendor_id[] array but we don't update cpu->hyperv_vendor string so e.g. QMP's query-cpu-model-expansion output is incorrect.
--- target/i386/cpu.c | 19 +++++++++---------- target/i386/kvm/kvm.c | 4 ++++ 2 files changed, 13 insertions(+), 10 deletions(-)
target/i386/cpu.c +9 -10
@@ -6549,17 +6549,16 @@ static void x86_cpu_hyperv_realize(X86CPU *cpu)/* Hyper-V vendor id */ if (!cpu->hyperv_vendor) { -memcpy (cpu->hyperv_vendor_id,"Microsoft Hv" ,12 ); - }else { - len =strlen (cpu->hyperv_vendor); - -if (len >12 ) { -warn_report ("hv-vendor-id truncated to 12 characters" ); - len =12 ; - } -memset (cpu->hyperv_vendor_id,0 ,12 ); -memcpy (cpu->hyperv_vendor_id, cpu->hyperv_vendor, len); +object_property_set_str (OBJECT (cpu),"hv-vendor-id" ,"Microsoft Hv" , + &error_abort); + } + len =strlen (cpu->hyperv_vendor); +if (len >12 ) { +warn_report ("hv-vendor-id truncated to 12 characters" ); + len =12 ; } +memset (cpu->hyperv_vendor_id,0 ,12 ); +memcpy (cpu->hyperv_vendor_id, cpu->hyperv_vendor, len);/* 'Hv#1' interface identification*/ cpu->hyperv_interface_id[0 ] =0x31237648 ;
target/i386/kvm/kvm.c +4 -0
@@ -1218,6 +1218,10 @@ static int hyperv_handle_properties(CPUState *cs, cpu->hyperv_vendor_id[0 ] = c->ebx; cpu->hyperv_vendor_id[1 ] = c->ecx; cpu->hyperv_vendor_id[2 ] = c->edx; + cpu->hyperv_vendor =g_realloc (cpu->hyperv_vendor, +sizeof (cpu->hyperv_vendor_id) +1 ); +memcpy (cpu->hyperv_vendor, cpu->hyperv_vendor_id, +sizeof (cpu->hyperv_vendor_id)); } c =cpuid_find_entry (cpuid,HV_CPUID_INTERFACE ,0 );
Signature · 1 line
2.29.2
PATCH v3 4/19 i386: invert hyperv_spinlock_attempts setting logic with hv_passthrough
In reply to Vitaly KuznetsovThere is no need to have this special case: like all other Hyper-V enlightenments we can just use kernel's supplied value in hv_passthrough mode.
--- target/i386/kvm/kvm.c | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-)
target/i386/kvm/kvm.c +1 -5
@@ -1258,11 +1258,7 @@ static int hyperv_handle_properties(CPUState *cs, c =cpuid_find_entry (cpuid,HV_CPUID_ENLIGHTMENT_INFO ,0 );if (c) { env->features[FEAT_HV_RECOMM_EAX ] = c->eax; - -/* hv-spinlocks may have been overriden */ -if (cpu->hyperv_spinlock_attempts !=HYPERV_SPINLOCK_NEVER_NOTIFY ) { - c->ebx = cpu->hyperv_spinlock_attempts; - } + cpu->hyperv_spinlock_attempts = c->ebx; } c =cpuid_find_entry (cpuid,HV_CPUID_NESTED_FEATURES ,0 );if (c) {
Signature · 1 line
2.29.2
PATCH v3 5/19 i386: always fill Hyper-V CPUID feature leaves from X86CPU data
In reply to Vitaly KuznetsovWe have all the required data in X86CPU already and as we are about to split hyperv_handle_properties() into hyperv_expand_features()/ hyperv_fill_cpuids() we can remove the blind copy. The functional change is that QEMU won't pass CPUID leaves it doesn't currently know about to the guest but arguably this is a good change.
--- target/i386/kvm/kvm.c | 9 --------- 1 file changed, 9 deletions(-)
target/i386/kvm/kvm.c +0 -9
@@ -1210,9 +1210,6 @@ static int hyperv_handle_properties(CPUState *cs, }if (cpu->hyperv_passthrough) { -memcpy (cpuid_ent, &cpuid->entries[0 ], - cpuid->nent *sizeof (cpuid->entries[0 ])); - c =cpuid_find_entry (cpuid,HV_CPUID_VENDOR_AND_MAX_FUNCTIONS ,0 );if (c) { cpu->hyperv_vendor_id[0 ] = c->ebx;
@@ -1311,12 +1308,6 @@ static int hyperv_handle_properties(CPUState *cs, goto free; } -if (cpu->hyperv_passthrough) { -/* We already copied all feature words from KVM as is */ - r = cpuid->nent; - goto free; - } - c = &cpuid_ent[cpuid_i++]; c->function =HV_CPUID_VENDOR_AND_MAX_FUNCTIONS ; c->eax = hyperv_feat_enabled(cpu, HYPERV_FEAT_EVMCS) ?
Signature · 1 line
2.29.2
PATCH v3 6/19 i386: stop using env->features[] for filling Hyper-V CPUIDs
In reply to Vitaly KuznetsovAs a preparatory patch to dropping Hyper-V CPUID leaves from feature_word_info[] stop using env->features[] as a temporary storage of Hyper-V CPUIDs, just build Hyper-V CPUID leaves directly from kvm_hyperv_properties[] data.
--- target/i386/cpu.h | 1 + target/i386/kvm/kvm.c | 80 +++++++++++++++++++++++-------------------- 2 files changed, 43 insertions(+), 38 deletions(-)
target/i386/cpu.h +1 -0
@@ -1667,6 +1667,7 @@ struct X86CPU {uint32_t hyperv_interface_id[4 ];uint32_t hyperv_version_id[4 ];uint32_t hyperv_limits[3 ]; +uint32_t hyperv_nested[4 ];bool check_cpuid;bool enforce_cpuid;
target/i386/kvm/kvm.c +42 -38
148 lines not on this page · raw message
Signature · 1 line
2.29.2
PATCH v3 7/19 i386: introduce hyperv_feature_supported()
In reply to Vitaly KuznetsovClean up hv_cpuid_check_and_set() by separating hyperv_feature_supported() off it. No functional change intended.
--- target/i386/kvm/kvm.c | 49 ++++++++++++++++++++++++++----------------- 1 file changed, 30 insertions(+), 19 deletions(-)
target/i386/kvm/kvm.c +30 -19
@@ -1110,13 +1110,33 @@ static int hv_cpuid_get_fw(struct kvm_cpuid2 *cpuid, int fw, uint32_t *r)return 0 ; } +static bool hyperv_feature_supported (struct kvm_cpuid2 *cpuid,int feature) +{ +uint32_t r, fw, bits; +int i; + +for (i =0 ; i <ARRAY_SIZE (kvm_hyperv_properties[feature].flags); i++) { + fw = kvm_hyperv_properties[feature].flags[i].fw; + bits = kvm_hyperv_properties[feature].flags[i].bits; + +if (!fw) { +continue ; + } + +if (hv_cpuid_get_fw (cpuid, fw, &r) || (r & bits) != bits) { +return false; + } + } + +return true; +} +static int hv_cpuid_check_and_set (CPUState *cs,struct kvm_cpuid2 *cpuid,int feature) {X86CPU *cpu =X86_CPU (cs); -uint32_t r, fw, bits;uint64_t deps; -int i, dep_feat; +int dep_feat;if (!hyperv_feat_enabled (cpu, feature) && !cpu->hyperv_passthrough) {return 0 ;
@@ -1135,23 +1155,14 @@ static int hv_cpuid_check_and_set(CPUState *cs, struct kvm_cpuid2 *cpuid, deps &= ~(1ull << dep_feat); } -for (i =0 ; i <ARRAY_SIZE (kvm_hyperv_properties[feature].flags); i++) { - fw = kvm_hyperv_properties[feature].flags[i].fw; - bits = kvm_hyperv_properties[feature].flags[i].bits; - -if (!fw) { -continue ; - } - -if (hv_cpuid_get_fw (cpuid, fw, &r) || (r & bits) != bits) { -if (hyperv_feat_enabled (cpu, feature)) { -fprintf (stderr, -"Hyper-V %s is not supported by kernel\n" , - kvm_hyperv_properties[feature].desc); -return 1 ; - }else { -return 0 ; - } +if (!hyperv_feature_supported (cpuid, feature)) { +if (hyperv_feat_enabled (cpu, feature)) { +fprintf (stderr, +"Hyper-V %s is not supported by kernel\n" , + kvm_hyperv_properties[feature].desc); +return 1 ; + }else { +return 0 ; } }
Signature · 1 line
2.29.2
PATCH v3 8/19 i386: introduce hv_cpuid_get_host()
In reply to Vitaly KuznetsovAs a preparation to implementing hv_cpuid_cache intro introduce hv_cpuid_get_host(). No functional change intended.
--- target/i386/kvm/kvm.c | 100 +++++++++++++++++++++++------------------- 1 file changed, 56 insertions(+), 44 deletions(-)
target/i386/kvm/kvm.c +56 -44
@@ -1110,6 +1110,19 @@ static int hv_cpuid_get_fw(struct kvm_cpuid2 *cpuid, int fw, uint32_t *r)return 0 ; } +static uint32_t hv_cpuid_get_host (struct kvm_cpuid2 *cpuid,uint32_t func, +int reg) +{ +struct kvm_cpuid_entry2 *entry; + + entry =cpuid_find_entry (cpuid, func,0 ); +if (!entry) { +return 0 ; + } + +return cpuid_entry_get_reg (entry, reg); +} +static bool hyperv_feature_supported (struct kvm_cpuid2 *cpuid,int feature) {uint32_t r, fw, bits;
@@ -1206,7 +1219,7 @@ static int hyperv_handle_properties(CPUState *cs, {X86CPU *cpu =X86_CPU (cs);struct kvm_cpuid2 *cpuid; -struct kvm_cpuid_entry2 *c, *c2; +struct kvm_cpuid_entry2 *c;uint32_t cpuid_i =0 ;int r;
@@ -1238,45 +1251,46 @@ static int hyperv_handle_properties(CPUState *cs, }if (cpu->hyperv_passthrough) { - c =cpuid_find_entry (cpuid,HV_CPUID_VENDOR_AND_MAX_FUNCTIONS ,0 ); -if (c) { - cpu->hyperv_vendor_id[0 ] = c->ebx; - cpu->hyperv_vendor_id[1 ] = c->ecx; - cpu->hyperv_vendor_id[2 ] = c->edx; - cpu->hyperv_vendor =g_realloc (cpu->hyperv_vendor, -sizeof (cpu->hyperv_vendor_id) +1 ); -memcpy (cpu->hyperv_vendor, cpu->hyperv_vendor_id, -sizeof (cpu->hyperv_vendor_id)); - } - - c =cpuid_find_entry (cpuid,HV_CPUID_INTERFACE ,0 ); -if (c) { - cpu->hyperv_interface_id[0 ] = c->eax; - cpu->hyperv_interface_id[1 ] = c->ebx; - cpu->hyperv_interface_id[2 ] = c->ecx; - cpu->hyperv_interface_id[3 ] = c->edx; - } - - c =cpuid_find_entry (cpuid,HV_CPUID_VERSION ,0 ); -if (c) { - cpu->hyperv_version_id[0 ] = c->eax; - cpu->hyperv_version_id[1 ] = c->ebx; - cpu->hyperv_version_id[2 ] = c->ecx; - cpu->hyperv_version_id[3 ] = c->edx; - } - - c =cpuid_find_entry (cpuid,HV_CPUID_IMPLEMENT_LIMITS ,0 ); -if (c) { - cpu->hv_max_vps = c->eax; - cpu->hyperv_limits[0 ] = c->ebx; - cpu->hyperv_limits[1 ] = c->ecx; - cpu->hyperv_limits[2 ] = c->edx; - } - - c =cpuid_find_entry (cpuid,HV_CPUID_ENLIGHTMENT_INFO ,0 ); -if (c) { - cpu->hyperv_spinlock_attempts = c->ebx; - } + cpu->hyperv_vendor_id[0 ] = +hv_cpuid_get_host (cpuid,HV_CPUID_VENDOR_AND_MAX_FUNCTIONS ,R_EBX ); + cpu->hyperv_vendor_id[1 ] = +hv_cpuid_get_host (cpuid,HV_CPUID_VENDOR_AND_MAX_FUNCTIONS ,R_ECX ); + cpu->hyperv_vendor_id[2 ] = +hv_cpuid_get_host (cpuid,HV_CPUID_VENDOR_AND_MAX_FUNCTIONS ,R_EDX ); + cpu->hyperv_vendor =g_realloc (cpu->hyperv_vendor, +sizeof (cpu->hyperv_vendor_id) +1 ); +memcpy (cpu->hyperv_vendor, cpu->hyperv_vendor_id, +sizeof (cpu->hyperv_vendor_id)); + + cpu->hyperv_interface_id[0 ] = +hv_cpuid_get_host (cpuid,HV_CPUID_INTERFACE ,R_EAX ); + cpu->hyperv_interface_id[1 ] = +hv_cpuid_get_host (cpuid,HV_CPUID_INTERFACE ,R_EBX ); + cpu->hyperv_interface_id[2 ] = +hv_cpuid_get_host (cpuid,HV_CPUID_INTERFACE ,R_ECX ); + cpu->hyperv_interface_id[3 ] = +hv_cpuid_get_host (cpuid,HV_CPUID_INTERFACE ,R_EDX ); + + cpu->hyperv_version_id[0 ] = +hv_cpuid_get_host (cpuid,HV_CPUID_VERSION ,R_EAX ); + cpu->hyperv_version_id[1 ] = +hv_cpuid_get_host (cpuid,HV_CPUID_VERSION ,R_EBX ); + cpu->hyperv_version_id[2 ] = +hv_cpuid_get_host (cpuid,HV_CPUID_VERSION ,R_ECX ); + cpu->hyperv_version_id[3 ] = +hv_cpuid_get_host (cpuid,HV_CPUID_VERSION ,R_EDX ); + + cpu->hv_max_vps =hv_cpuid_get_host (cpuid,HV_CPUID_IMPLEMENT_LIMITS , +R_EAX ); + cpu->hyperv_limits[0 ] = +hv_cpuid_get_host (cpuid,HV_CPUID_IMPLEMENT_LIMITS ,R_EBX ); + cpu->hyperv_limits[1 ] = +hv_cpuid_get_host (cpuid,HV_CPUID_IMPLEMENT_LIMITS ,R_ECX ); + cpu->hyperv_limits[2 ] = +hv_cpuid_get_host (cpuid,HV_CPUID_IMPLEMENT_LIMITS ,R_EDX ); + + cpu->hyperv_spinlock_attempts = +hv_cpuid_get_host (cpuid,HV_CPUID_ENLIGHTMENT_INFO ,R_EBX ); }/* Features */
@@ -1350,10 +1364,8 @@ static int hyperv_handle_properties(CPUState *cs,if (cpu->hyperv_no_nonarch_cs ==ON_OFF_AUTO_ON ) { c->eax |=HV_NO_NONARCH_CORESHARING ; }else if (cpu->hyperv_no_nonarch_cs ==ON_OFF_AUTO_AUTO ) { - c2 =cpuid_find_entry (cpuid,HV_CPUID_ENLIGHTMENT_INFO ,0 ); -if (c2) { - c->eax |= c2->eax &HV_NO_NONARCH_CORESHARING ; - } + c->eax |=hv_cpuid_get_host (cpuid,HV_CPUID_ENLIGHTMENT_INFO ,R_EAX ) & +HV_NO_NONARCH_CORESHARING ; } c = &cpuid_ent[cpuid_i++];
Signature · 1 line
2.29.2
PATCH v3 9/19 i386: drop FEAT_HYPERV feature leaves
In reply to Vitaly KuznetsovHyper-V feature leaves are weird. We have some of them in feature_word_info[] array but we don't use feature_word_info magic to enable them. Neither do we use feature_dependencies[] mechanism to validate the configuration as it doesn't allign well with Hyper-V's many-to-many dependency chains. Some of the feature leaves hold not only feature bits, but also values. E.g. FEAT_HV_NESTED_EAX contains both features and the supported Enlightened VMCS range.
Hyper-V features are already represented in 'struct X86CPU' with uint64_t hyperv_features so duplicating them in env->features adds little (or zero) benefits. THe other half of Hyper-V emulation features is also stored with values in hyperv_vendor_id[], hyperv_limits[],... so env->features[] is already incomplete.
Remove Hyper-V feature leaves from env->features[] completely. kvm_hyperv_properties[] is converted to using raw CPUID func/reg pairs for features, this allows us to get rid of hv_cpuid_get_fw() conversion.
--- target/i386/cpu.c | 90 +---------------------------------- target/i386/cpu.h | 5 -- target/i386/kvm/kvm.c | 108 ++++++++++++++---------------------------- 3 files changed, 37 insertions(+), 166 deletions(-)
target/i386/cpu.c +1 -89
@@ -832,94 +832,6 @@ static FeatureWordInfo feature_word_info[FEATURE_WORDS] = {*/ .no_autoenable_flags = ~0U, }, -/* -* .feat_names are commented out for Hyper-V enlightenments because we -* don't want to have two different ways for enabling them on QEMU command -* line. Some features (e.g. "hyperv_time", "hyperv_vapic", ...) require -* enabling several feature bits simultaneously, exposing these bits -* individually may just confuse guests. -*/ - [FEAT_HYPERV_EAX] = { - .type = CPUID_FEATURE_WORD, - .feat_names = { -NULL /* hv_msr_vp_runtime_access */ ,NULL /* hv_msr_time_refcount_access */ , -NULL /* hv_msr_synic_access */ ,NULL /* hv_msr_stimer_access */ , -NULL /* hv_msr_apic_access */ ,NULL /* hv_msr_hypercall_access */ , -NULL /* hv_vpindex_access */ ,NULL /* hv_msr_reset_access */ , -NULL /* hv_msr_stats_access */ ,NULL /* hv_reftsc_access */ , -NULL /* hv_msr_idle_access */ ,NULL /* hv_msr_frequency_access */ , -NULL /* hv_msr_debug_access */ ,NULL /* hv_msr_reenlightenment_access */ , -NULL ,NULL , -NULL ,NULL ,NULL ,NULL , -NULL ,NULL ,NULL ,NULL , -NULL ,NULL ,NULL ,NULL , - NULL, NULL, NULL, NULL, - }, - .cpuid = { .eax = 0x40000003, .reg = R_EAX, }, - }, - [FEAT_HYPERV_EBX] = { - .type =CPUID_FEATURE_WORD , - .feat_names = { -NULL /* hv_create_partitions */ ,NULL /* hv_access_partition_id */ , -NULL /* hv_access_memory_pool */ ,NULL /* hv_adjust_message_buffers */ , -NULL /* hv_post_messages */ ,NULL /* hv_signal_events */ , -NULL /* hv_create_port */ ,NULL /* hv_connect_port */ , -NULL /* hv_access_stats */ ,NULL ,NULL ,NULL /* hv_debugging */ , -NULL /* hv_cpu_power_management */ ,NULL /* hv_configure_profiler */ , -NULL ,NULL , -NULL ,NULL ,NULL ,NULL , -NULL ,NULL ,NULL ,NULL , -NULL ,NULL ,NULL ,NULL , - NULL, NULL, NULL, NULL, - }, - .cpuid = { .eax = 0x40000003, .reg = R_EBX, }, - }, - [FEAT_HYPERV_EDX] = { - .type =CPUID_FEATURE_WORD , - .feat_names = { -NULL /* hv_mwait */ ,NULL /* hv_guest_debugging */ , -NULL /* hv_perf_monitor */ ,NULL /* hv_cpu_dynamic_part */ , -NULL /* hv_hypercall_params_xmm */ ,NULL /* hv_guest_idle_state */ , -NULL ,NULL , -NULL ,NULL ,NULL /* hv_guest_crash_msr */ ,NULL , -NULL ,NULL ,NULL ,NULL , -NULL ,NULL ,NULL ,NULL , -NULL ,NULL ,NULL ,NULL , -NULL ,NULL ,NULL ,NULL , - NULL, NULL, NULL, NULL, - }, - .cpuid = { .eax = 0x40000003, .reg = R_EDX, }, - }, - [FEAT_HV_RECOMM_EAX] = { - .type =CPUID_FEATURE_WORD , - .feat_names = { -NULL /* hv_recommend_pv_as_switch */ , -NULL /* hv_recommend_pv_tlbflush_local */ , -NULL /* hv_recommend_pv_tlbflush_remote */ , -NULL /* hv_recommend_msr_apic_access */ , -NULL /* hv_recommend_msr_reset */ , -NULL /* hv_recommend_relaxed_timing */ , -NULL /* hv_recommend_dma_remapping */ , -NULL /* hv_recommend_int_remapping */ , -NULL /* hv_recommend_x2apic_msrs */ , -NULL /* hv_recommend_autoeoi_deprecation */ , -NULL /* hv_recommend_pv_ipi */ , -NULL /* hv_recommend_ex_hypercalls */ , -NULL /* hv_hypervisor_is_nested */ , -NULL /* hv_recommend_int_mbec */ , -NULL /* hv_recommend_evmcs */ , -NULL , -NULL ,NULL ,NULL ,NULL , -NULL ,NULL ,NULL ,NULL , -NULL ,NULL ,NULL ,NULL , - NULL, NULL, NULL, NULL, - }, - .cpuid = { .eax = 0x40000004, .reg = R_EAX, }, - }, - [FEAT_HV_NESTED_EAX] = { - .type =CPUID_FEATURE_WORD , - .cpuid = { .eax = 0x4000000A, .reg = R_EAX, }, - }, [FEAT_SVM] = { .type = CPUID_FEATURE_WORD, .feat_names = {
@@ -6953,7 +6865,7 @@ static GuestPanicInformation *x86_cpu_get_crash_info(CPUState *cs)CPUX86State *env = &cpu->env;GuestPanicInformation *panic_info =NULL ; -if (env->features[FEAT_HYPERV_EDX ] &HV_GUEST_CRASH_MSR_AVAILABLE ) { +if (hyperv_feat_enabled (cpu,HYPERV_FEAT_CRASH )) { panic_info =g_malloc0 (sizeof (GuestPanicInformation)); panic_info->type =GUEST_PANIC_INFORMATION_TYPE_HYPER_V ;
target/i386/cpu.h +0 -5
@@ -516,11 +516,6 @@ typedef enum FeatureWord {FEAT_C000_0001_EDX ,/* CPUID[C000_0001].EDX */ FEAT_KVM ,/* CPUID[4000_0001].EAX (KVM_CPUID_FEATURES) */ FEAT_KVM_HINTS ,/* CPUID[4000_0001].EDX */ -FEAT_HYPERV_EAX ,/* CPUID[4000_0003].EAX */ -FEAT_HYPERV_EBX ,/* CPUID[4000_0003].EBX */ -FEAT_HYPERV_EDX ,/* CPUID[4000_0003].EDX */ -FEAT_HV_RECOMM_EAX ,/* CPUID[4000_0004].EAX */ -FEAT_HV_NESTED_EAX ,/* CPUID[4000_000A].EAX */ FEAT_SVM ,/* CPUID[8000_000A].EDX */ FEAT_XSAVE ,/* CPUID[EAX=0xd,ECX=1].EAX */ FEAT_6_EAX, /* CPUID[6].EAX */
target/i386/kvm/kvm.c +36 -72
250 lines not on this page · raw message
Signature · 1 line
2.29.2
PATCH v3 10/19 i386: introduce hv_cpuid_cache
In reply to Vitaly KuznetsovJust like with cpuid_cache, it makes no sense to call KVM_GET_SUPPORTED_HV_CPUID more than once and instead of (ab)using env->features[] and/or trying to keep all the code in one place, it is better to introduce persistent hv_cpuid_cache and hv_cpuid_get_host() accessor to it.
Note, hv_cpuid_get_fw() is converted to using hv_cpuid_get_host() just to be removed later with Hyper-V specific feature words.
--- target/i386/kvm/kvm.c | 109 ++++++++++++++++++++++-------------------- 1 file changed, 56 insertions(+), 53 deletions(-)
target/i386/kvm/kvm.c +56 -53
@@ -126,6 +126,7 @@ static int has_exception_payload;static bool has_msr_mcg_ext_ctl;static struct kvm_cpuid2 *cpuid_cache; +static struct kvm_cpuid2 *hv_cpuid_cache;static struct kvm_msr_list *kvm_feature_msrs;int kvm_has_pit_state2 (void )
@@ -1069,10 +1070,25 @@ static struct kvm_cpuid2 *get_supported_hv_cpuid_legacy(CPUState *cs)return cpuid; } -static uint32_t hv_cpuid_get_host (struct kvm_cpuid2 *cpuid,uint32_t func, -int reg) +static uint32_t hv_cpuid_get_host (CPUState *cs,uint32_t func,int reg) {struct kvm_cpuid_entry2 *entry; +struct kvm_cpuid2 *cpuid; + +if (hv_cpuid_cache) { + cpuid = hv_cpuid_cache; + }else { +if (kvm_check_extension (kvm_state,KVM_CAP_HYPERV_CPUID ) >0 ) { + cpuid =get_supported_hv_cpuid (cs); + }else { + cpuid =get_supported_hv_cpuid_legacy (cs); + } + hv_cpuid_cache = cpuid; + } + +if (!cpuid) { +return 0 ; + } entry =cpuid_find_entry (cpuid, func,0 );if (!entry) {
@@ -1082,7 +1098,7 @@ static uint32_t hv_cpuid_get_host(struct kvm_cpuid2 *cpuid, uint32_t func,return cpuid_entry_get_reg (entry, reg); } -static bool hyperv_feature_supported (struct kvm_cpuid2 *cpuid,int feature) +static bool hyperv_feature_supported (CPUState *cs,int feature) {uint32_t func, bits;int i, reg;
@@ -1097,7 +1113,7 @@ static bool hyperv_feature_supported(struct kvm_cpuid2 *cpuid, int feature)continue ; } -if ((hv_cpuid_get_host (cpuid, func, reg) & bits) != bits) { +if ((hv_cpuid_get_host (cs, func, reg) & bits) != bits) {return false; } }
@@ -1105,8 +1121,7 @@ static bool hyperv_feature_supported(struct kvm_cpuid2 *cpuid, int feature)return true; } -static int hv_cpuid_check_and_set (CPUState *cs,struct kvm_cpuid2 *cpuid, -int feature) +static int hv_cpuid_check_and_set (CPUState *cs,int feature) {X86CPU *cpu =X86_CPU (cs);uint64_t deps;
@@ -1129,7 +1144,7 @@ static int hv_cpuid_check_and_set(CPUState *cs, struct kvm_cpuid2 *cpuid, deps &= ~(1ull << dep_feat); } -if (!hyperv_feature_supported (cpuid, feature)) { +if (!hyperv_feature_supported (cs, feature)) {if (hyperv_feat_enabled (cpu, feature)) { fprintf(stderr, "Hyper-V %s is not supported by kernel\n",
@@ -1182,7 +1197,6 @@ static int hyperv_handle_properties(CPUState *cs, struct kvm_cpuid_entry2 *cpuid_ent) {X86CPU *cpu =X86_CPU (cs); -struct kvm_cpuid2 *cpuid;struct kvm_cpuid_entry2 *c;uint32_t cpuid_i =0 ;int r;
@@ -1208,71 +1222,65 @@ static int hyperv_handle_properties(CPUState *cs, } } -if (kvm_check_extension (cs->kvm_state,KVM_CAP_HYPERV_CPUID ) >0 ) { - cpuid =get_supported_hv_cpuid (cs); - }else { - cpuid =get_supported_hv_cpuid_legacy (cs); - } -if (cpu->hyperv_passthrough) { cpu->hyperv_vendor_id[0 ] = -hv_cpuid_get_host (cpuid,HV_CPUID_VENDOR_AND_MAX_FUNCTIONS ,R_EBX ); +hv_cpuid_get_host (cs,HV_CPUID_VENDOR_AND_MAX_FUNCTIONS ,R_EBX ); cpu->hyperv_vendor_id[1 ] = -hv_cpuid_get_host (cpuid,HV_CPUID_VENDOR_AND_MAX_FUNCTIONS ,R_ECX ); +hv_cpuid_get_host (cs,HV_CPUID_VENDOR_AND_MAX_FUNCTIONS ,R_ECX ); cpu->hyperv_vendor_id[2 ] = -hv_cpuid_get_host (cpuid,HV_CPUID_VENDOR_AND_MAX_FUNCTIONS ,R_EDX ); +hv_cpuid_get_host (cs,HV_CPUID_VENDOR_AND_MAX_FUNCTIONS ,R_EDX ); cpu->hyperv_vendor =g_realloc (cpu->hyperv_vendor,sizeof (cpu->hyperv_vendor_id) +1 );memcpy (cpu->hyperv_vendor, cpu->hyperv_vendor_id,sizeof (cpu->hyperv_vendor_id)); cpu->hyperv_interface_id[0 ] = -hv_cpuid_get_host (cpuid,HV_CPUID_INTERFACE ,R_EAX ); +hv_cpuid_get_host (cs,HV_CPUID_INTERFACE ,R_EAX ); cpu->hyperv_interface_id[1 ] = -hv_cpuid_get_host (cpuid,HV_CPUID_INTERFACE ,R_EBX ); +hv_cpuid_get_host (cs,HV_CPUID_INTERFACE ,R_EBX ); cpu->hyperv_interface_id[2 ] = -hv_cpuid_get_host (cpuid,HV_CPUID_INTERFACE ,R_ECX ); +hv_cpuid_get_host (cs,HV_CPUID_INTERFACE ,R_ECX ); cpu->hyperv_interface_id[3 ] = -hv_cpuid_get_host (cpuid,HV_CPUID_INTERFACE ,R_EDX ); +hv_cpuid_get_host (cs,HV_CPUID_INTERFACE ,R_EDX ); cpu->hyperv_version_id[0 ] = -hv_cpuid_get_host (cpuid,HV_CPUID_VERSION ,R_EAX ); +hv_cpuid_get_host (cs,HV_CPUID_VERSION ,R_EAX ); cpu->hyperv_version_id[1 ] = -hv_cpuid_get_host (cpuid,HV_CPUID_VERSION ,R_EBX ); +hv_cpuid_get_host (cs,HV_CPUID_VERSION ,R_EBX ); cpu->hyperv_version_id[2 ] = -hv_cpuid_get_host (cpuid,HV_CPUID_VERSION ,R_ECX ); +hv_cpuid_get_host (cs,HV_CPUID_VERSION ,R_ECX ); cpu->hyperv_version_id[3 ] = -hv_cpuid_get_host (cpuid,HV_CPUID_VERSION ,R_EDX ); +hv_cpuid_get_host (cs,HV_CPUID_VERSION ,R_EDX ); - cpu->hv_max_vps =hv_cpuid_get_host (cpuid,HV_CPUID_IMPLEMENT_LIMITS , + cpu->hv_max_vps =hv_cpuid_get_host (cs,HV_CPUID_IMPLEMENT_LIMITS ,R_EAX ); cpu->hyperv_limits[0 ] = -hv_cpuid_get_host (cpuid,HV_CPUID_IMPLEMENT_LIMITS ,R_EBX ); +hv_cpuid_get_host (cs,HV_CPUID_IMPLEMENT_LIMITS ,R_EBX ); cpu->hyperv_limits[1 ] = -hv_cpuid_get_host (cpuid,HV_CPUID_IMPLEMENT_LIMITS ,R_ECX ); +hv_cpuid_get_host (cs,HV_CPUID_IMPLEMENT_LIMITS ,R_ECX ); cpu->hyperv_limits[2 ] = -hv_cpuid_get_host (cpuid,HV_CPUID_IMPLEMENT_LIMITS ,R_EDX ); +hv_cpuid_get_host (cs,HV_CPUID_IMPLEMENT_LIMITS ,R_EDX ); cpu->hyperv_spinlock_attempts = -hv_cpuid_get_host (cpuid,HV_CPUID_ENLIGHTMENT_INFO ,R_EBX ); +hv_cpuid_get_host (cs,HV_CPUID_ENLIGHTMENT_INFO ,R_EBX ); }/* Features */ - r =hv_cpuid_check_and_set (cs, cpuid,HYPERV_FEAT_RELAXED ); - r |=hv_cpuid_check_and_set (cs, cpuid,HYPERV_FEAT_VAPIC ); - r |=hv_cpuid_check_and_set (cs, cpuid,HYPERV_FEAT_TIME ); - r |=hv_cpuid_check_and_set (cs, cpuid,HYPERV_FEAT_CRASH ); - r |=hv_cpuid_check_and_set (cs, cpuid,HYPERV_FEAT_RESET ); - r |=hv_cpuid_check_and_set (cs, cpuid,HYPERV_FEAT_VPINDEX ); - r |=hv_cpuid_check_and_set (cs, cpuid,HYPERV_FEAT_RUNTIME ); - r |=hv_cpuid_check_and_set (cs, cpuid,HYPERV_FEAT_SYNIC ); - r |=hv_cpuid_check_and_set (cs, cpuid,HYPERV_FEAT_STIMER ); - r |=hv_cpuid_check_and_set (cs, cpuid,HYPERV_FEAT_FREQUENCIES ); - r |=hv_cpuid_check_and_set (cs, cpuid,HYPERV_FEAT_REENLIGHTENMENT ); - r |=hv_cpuid_check_and_set (cs, cpuid,HYPERV_FEAT_TLBFLUSH ); - r |=hv_cpuid_check_and_set (cs, cpuid,HYPERV_FEAT_EVMCS ); - r |=hv_cpuid_check_and_set (cs, cpuid,HYPERV_FEAT_IPI ); - r |=hv_cpuid_check_and_set (cs, cpuid,HYPERV_FEAT_STIMER_DIRECT ); + r =hv_cpuid_check_and_set (cs,HYPERV_FEAT_RELAXED ); + r |=hv_cpuid_check_and_set (cs,HYPERV_FEAT_VAPIC ); + r |=hv_cpuid_check_and_set (cs,HYPERV_FEAT_TIME ); + r |=hv_cpuid_check_and_set (cs,HYPERV_FEAT_CRASH ); + r |=hv_cpuid_check_and_set (cs,HYPERV_FEAT_RESET ); + r |=hv_cpuid_check_and_set (cs,HYPERV_FEAT_VPINDEX ); + r |=hv_cpuid_check_and_set (cs,HYPERV_FEAT_RUNTIME ); + r |=hv_cpuid_check_and_set (cs,HYPERV_FEAT_SYNIC ); + r |=hv_cpuid_check_and_set (cs,HYPERV_FEAT_STIMER ); + r |=hv_cpuid_check_and_set (cs,HYPERV_FEAT_FREQUENCIES ); + r |=hv_cpuid_check_and_set (cs,HYPERV_FEAT_REENLIGHTENMENT ); + r |=hv_cpuid_check_and_set (cs,HYPERV_FEAT_TLBFLUSH ); + r |=hv_cpuid_check_and_set (cs,HYPERV_FEAT_EVMCS ); + r |=hv_cpuid_check_and_set (cs,HYPERV_FEAT_IPI ); + r |=hv_cpuid_check_and_set (cs,HYPERV_FEAT_STIMER_DIRECT );/* Additional dependencies not covered by kvm_hyperv_properties[] */ if (hyperv_feat_enabled(cpu, HYPERV_FEAT_SYNIC) &&
@@ -1285,8 +1293,7 @@ static int hyperv_handle_properties(CPUState *cs, }if (r) { - r = -ENOSYS ; - goto free; +return -ENOSYS ; } c = &cpuid_ent[cpuid_i++];
@@ -1328,7 +1335,7 @@ static int hyperv_handle_properties(CPUState *cs,if (cpu->hyperv_no_nonarch_cs ==ON_OFF_AUTO_ON ) { c->eax |=HV_NO_NONARCH_CORESHARING ; }else if (cpu->hyperv_no_nonarch_cs ==ON_OFF_AUTO_AUTO ) { - c->eax |=hv_cpuid_get_host (cpuid,HV_CPUID_ENLIGHTMENT_INFO ,R_EAX ) & + c->eax |=hv_cpuid_get_host (cs,HV_CPUID_ENLIGHTMENT_INFO ,R_EAX ) &HV_NO_NONARCH_CORESHARING ; }
@@ -1353,12 +1360,8 @@ static int hyperv_handle_properties(CPUState *cs, c->function =HV_CPUID_NESTED_FEATURES ; c->eax = cpu->hyperv_nested[0 ]; } - r = cpuid_i; -free: -g_free (cpuid); - -return r; +return cpuid_i; }static Error *hv_passthrough_mig_blocker;
Signature · 1 line
2.29.2
PATCH v3 11/19 i386: split hyperv_handle_properties() into hyperv_expand_features()/hyperv_fill_cpuids()
In reply to Vitaly KuznetsovThe intention is to call hyperv_expand_features() early, before vCPUs are created and use the acquired data later when we set guest visible CPUID data.
--- target/i386/kvm/kvm.c | 34 ++++++++++++++++++++++++---------- 1 file changed, 24 insertions(+), 10 deletions(-)
target/i386/kvm/kvm.c +24 -10
@@ -1189,16 +1189,15 @@ static uint32_t hv_build_cpuid_leaf(CPUState *cs, uint32_t func, int reg) }/* -* Fill in Hyper-V CPUIDs. Returns the number of entries filled in cpuid_ent in -* case of success, errno < 0 in case of failure and 0 when no Hyper-V -* extentions are enabled. +* Expand Hyper-V CPU features. In partucular, check that all the requested +* features are supported by the host and the sanity of the configuration +* (that all the required dependencies are included). Also, this takes care +* of 'hv_passthrough' mode and fills the environment with all supported +* Hyper-V features. */ -static int hyperv_handle_properties (CPUState *cs, -struct kvm_cpuid_entry2 *cpuid_ent) +static int hyperv_expand_features (CPUState *cs) {X86CPU *cpu =X86_CPU (cs); -struct kvm_cpuid_entry2 *c; -uint32_t cpuid_i =0 ;int r;if (!hyperv_enabled (cpu))
@@ -1296,6 +1295,19 @@ static int hyperv_handle_properties(CPUState *cs,return -ENOSYS ; } +return 0 ; +} + +/* +* Fill in Hyper-V CPUIDs. Returns the number of entries filled in cpuid_ent. +*/ +static int hyperv_fill_cpuids (CPUState *cs, +struct kvm_cpuid_entry2 *cpuid_ent) +{ +X86CPU *cpu =X86_CPU (cs); +struct kvm_cpuid_entry2 *c; +uint32_t cpuid_i =0 ; + c = &cpuid_ent[cpuid_i++]; c->function =HV_CPUID_VENDOR_AND_MAX_FUNCTIONS ; c->eax = hyperv_feat_enabled(cpu, HYPERV_FEAT_EVMCS) ?
@@ -1503,11 +1515,13 @@ int kvm_arch_init_vcpu(CPUState *cs) env->apic_bus_freq =KVM_APIC_BUS_FREQUENCY ;/* Paravirtualization CPUIDs */ - r =hyperv_handle_properties (cs, cpuid_data.entries); + r =hyperv_expand_features (cs);if (r <0 ) {return r; - }else if (r >0 ) { - cpuid_i = r; + } + +if (hyperv_enabled (cpu)) { + cpuid_i =hyperv_fill_cpuids (cs, cpuid_data.entries); kvm_base =KVM_CPUID_SIGNATURE_NEXT ; has_msr_hv_hypercall = true; }
Signature · 1 line
2.29.2
PATCH v3 12/19 i386: move eVMCS enablement to hyperv_init_vcpu()
In reply to Vitaly Kuznetsovhyperv_expand_features() will be called before we create vCPU so evmcs enablement should go away. hyperv_init_vcpu() looks like the right place.
--- target/i386/kvm/kvm.c | 60 ++++++++++++++++++++++++++----------------- 1 file changed, 37 insertions(+), 23 deletions(-)
target/i386/kvm/kvm.c +37 -23
@@ -965,6 +965,7 @@ static struct kvm_cpuid2 *get_supported_hv_cpuid(CPUState *cs) {struct kvm_cpuid2 *cpuid;int max =7 ;/* 0x40000000..0x40000005, 0x4000000A */ +int i;/* * When the buffer is too small, KVM_GET_SUPPORTED_HV_CPUID fails with
@@ -974,6 +975,22 @@ static struct kvm_cpuid2 *get_supported_hv_cpuid(CPUState *cs)while ((cpuid =try_get_hv_cpuid (cs, max)) ==NULL ) { max++; } + +/* +* KVM_GET_SUPPORTED_HV_CPUID does not set EVMCS CPUID bit before +* KVM_CAP_HYPERV_ENLIGHTENED_VMCS is enabled but we want to get the +* information early, just check for the capability and set the bit +* manually. +*/ +if (kvm_check_extension (cs->kvm_state, +KVM_CAP_HYPERV_ENLIGHTENED_VMCS ) >0 ) { +for (i =0 ; i < cpuid->nent; i++) { +if (cpuid->entries[i].function ==HV_CPUID_ENLIGHTMENT_INFO ) { + cpuid->entries[i].eax |=HV_ENLIGHTENED_VMCS_RECOMMENDED ; + } + } + } +return cpuid; }
@@ -1203,24 +1220,6 @@ static int hyperv_expand_features(CPUState *cs)if (!hyperv_enabled (cpu))return 0 ; -if (hyperv_feat_enabled (cpu,HYPERV_FEAT_EVMCS ) || - cpu->hyperv_passthrough) { -uint16_t evmcs_version; - - r =kvm_vcpu_enable_cap (cs,KVM_CAP_HYPERV_ENLIGHTENED_VMCS ,0 , - (uintptr_t )&evmcs_version); - -if (hyperv_feat_enabled (cpu,HYPERV_FEAT_EVMCS ) && r) { -fprintf (stderr,"Hyper-V %s is not supported by kernel\n" , - kvm_hyperv_properties[HYPERV_FEAT_EVMCS ].desc); -return -ENOSYS ; - } - -if (!r) { - cpu->hyperv_nested[0 ] = evmcs_version; - } - } -if (cpu->hyperv_passthrough) { cpu->hyperv_vendor_id[0 ] =hv_cpuid_get_host (cs,HV_CPUID_VENDOR_AND_MAX_FUNCTIONS ,R_EBX );
@@ -1457,6 +1456,21 @@ static int hyperv_init_vcpu(X86CPU *cpu) } } +if (hyperv_feat_enabled (cpu,HYPERV_FEAT_EVMCS )) { +uint16_t evmcs_version; + + ret =kvm_vcpu_enable_cap (cs,KVM_CAP_HYPERV_ENLIGHTENED_VMCS ,0 , + (uintptr_t )&evmcs_version); + +if (ret <0 ) { +fprintf (stderr,"Hyper-V %s is not supported by kernel\n" , + kvm_hyperv_properties[HYPERV_FEAT_EVMCS ].desc); +return ret; + } + + cpu->hyperv_nested[0 ] = evmcs_version; + } +return 0 ; }
@@ -1521,6 +1535,11 @@ int kvm_arch_init_vcpu(CPUState *cs) }if (hyperv_enabled (cpu)) { + r =hyperv_init_vcpu (cpu); +if (r) { +return r; + } + cpuid_i =hyperv_fill_cpuids (cs, cpuid_data.entries); kvm_base =KVM_CPUID_SIGNATURE_NEXT ; has_msr_hv_hypercall = true;
@@ -1870,11 +1889,6 @@ int kvm_arch_init_vcpu(CPUState *cs)kvm_init_msrs (cpu); - r =hyperv_init_vcpu (cpu); -if (r) { - goto fail; - } -return 0 ; fail:
Signature · 1 line
2.29.2
PATCH v3 13/19 i386: switch hyperv_expand_features() to using error_setg()
In reply to Vitaly KuznetsovUse standard error_setg() mechanism in hyperv_expand_features().
--- target/i386/kvm/kvm.c | 101 +++++++++++++++++++++++++----------------- 1 file changed, 61 insertions(+), 40 deletions(-)
target/i386/kvm/kvm.c +61 -40
@@ -1138,7 +1138,7 @@ static bool hyperv_feature_supported(CPUState *cs, int feature)return true; } -static int hv_cpuid_check_and_set (CPUState *cs,int feature) +static int hv_cpuid_check_and_set (CPUState *cs,int feature,Error **errp) {X86CPU *cpu =X86_CPU (cs);uint64_t deps;
@@ -1152,20 +1152,18 @@ static int hv_cpuid_check_and_set(CPUState *cs, int feature)while (deps) { dep_feat =ctz64 (deps);if (!(hyperv_feat_enabled (cpu, dep_feat))) { -fprintf (stderr, -"Hyper-V %s requires Hyper-V %s\n" , - kvm_hyperv_properties[feature].desc, - kvm_hyperv_properties[dep_feat].desc); -return 1 ; +error_setg (errp,"Hyper-V %s requires Hyper-V %s" , + kvm_hyperv_properties[feature].desc, + kvm_hyperv_properties[dep_feat].desc); +return 1 ; } deps &= ~(1ull << dep_feat); }if (!hyperv_feature_supported (cs, feature)) {if (hyperv_feat_enabled (cpu, feature)) { -fprintf (stderr, -"Hyper-V %s is not supported by kernel\n" , - kvm_hyperv_properties[feature].desc); +error_setg (errp,"Hyper-V %s is not supported by kernel" , + kvm_hyperv_properties[feature].desc);return 1 ; }else {return 0 ;
@@ -1212,13 +1210,12 @@ static uint32_t hv_build_cpuid_leaf(CPUState *cs, uint32_t func, int reg)* of 'hv_passthrough' mode and fills the environment with all supported * Hyper-V features. */ -static int hyperv_expand_features (CPUState *cs) +static void hyperv_expand_features (CPUState *cs,Error **errp) {X86CPU *cpu =X86_CPU (cs); -int r;if (!hyperv_enabled (cpu)) -return 0 ; +return ;if (cpu->hyperv_passthrough) { cpu->hyperv_vendor_id[0] =
@@ -1264,37 +1261,60 @@ static int hyperv_expand_features(CPUState *cs) }/* Features */ - r =hv_cpuid_check_and_set (cs,HYPERV_FEAT_RELAXED ); - r |=hv_cpuid_check_and_set (cs,HYPERV_FEAT_VAPIC ); - r |=hv_cpuid_check_and_set (cs,HYPERV_FEAT_TIME ); - r |=hv_cpuid_check_and_set (cs,HYPERV_FEAT_CRASH ); - r |=hv_cpuid_check_and_set (cs,HYPERV_FEAT_RESET ); - r |=hv_cpuid_check_and_set (cs,HYPERV_FEAT_VPINDEX ); - r |=hv_cpuid_check_and_set (cs,HYPERV_FEAT_RUNTIME ); - r |=hv_cpuid_check_and_set (cs,HYPERV_FEAT_SYNIC ); - r |=hv_cpuid_check_and_set (cs,HYPERV_FEAT_STIMER ); - r |=hv_cpuid_check_and_set (cs,HYPERV_FEAT_FREQUENCIES ); - r |=hv_cpuid_check_and_set (cs,HYPERV_FEAT_REENLIGHTENMENT ); - r |=hv_cpuid_check_and_set (cs,HYPERV_FEAT_TLBFLUSH ); - r |=hv_cpuid_check_and_set (cs,HYPERV_FEAT_EVMCS ); - r |=hv_cpuid_check_and_set (cs,HYPERV_FEAT_IPI ); - r |=hv_cpuid_check_and_set (cs,HYPERV_FEAT_STIMER_DIRECT ); +if (hv_cpuid_check_and_set (cs,HYPERV_FEAT_RELAXED , errp)) { +return ; + } +if (hv_cpuid_check_and_set (cs,HYPERV_FEAT_VAPIC , errp)) { +return ; + } +if (hv_cpuid_check_and_set (cs,HYPERV_FEAT_TIME , errp)) { +return ; + } +if (hv_cpuid_check_and_set (cs,HYPERV_FEAT_CRASH , errp)) { +return ; + } +if (hv_cpuid_check_and_set (cs,HYPERV_FEAT_RESET , errp)) { +return ; + } +if (hv_cpuid_check_and_set (cs,HYPERV_FEAT_VPINDEX , errp)) { +return ; + } +if (hv_cpuid_check_and_set (cs,HYPERV_FEAT_RUNTIME , errp)) { +return ; + } +if (hv_cpuid_check_and_set (cs,HYPERV_FEAT_SYNIC , errp)) { +return ; + } +if (hv_cpuid_check_and_set (cs,HYPERV_FEAT_STIMER , errp)) { +return ; + } +if (hv_cpuid_check_and_set (cs,HYPERV_FEAT_FREQUENCIES , errp)) { +return ; + } +if (hv_cpuid_check_and_set (cs,HYPERV_FEAT_REENLIGHTENMENT , errp)) { +return ; + } +if (hv_cpuid_check_and_set (cs,HYPERV_FEAT_TLBFLUSH , errp)) { +return ; + } +if (hv_cpuid_check_and_set (cs,HYPERV_FEAT_EVMCS , errp)) { +return ; + } +if (hv_cpuid_check_and_set (cs,HYPERV_FEAT_IPI , errp)) { +return ; + } +if (hv_cpuid_check_and_set (cs,HYPERV_FEAT_STIMER_DIRECT , errp)) { +return ; + }/* Additional dependencies not covered by kvm_hyperv_properties[] */ if (hyperv_feat_enabled (cpu,HYPERV_FEAT_SYNIC ) && !cpu->hyperv_synic_kvm_only && !hyperv_feat_enabled (cpu,HYPERV_FEAT_VPINDEX )) { -fprintf (stderr,"Hyper-V %s requires Hyper-V %s\n" , - kvm_hyperv_properties[HYPERV_FEAT_SYNIC ].desc, - kvm_hyperv_properties[HYPERV_FEAT_VPINDEX ].desc); - r |=1 ; - } - -if (r) { -return -ENOSYS ; +error_setg (errp,"Hyper-V %s requires Hyper-V %s" , + kvm_hyperv_properties[HYPERV_FEAT_SYNIC ].desc, + kvm_hyperv_properties[HYPERV_FEAT_VPINDEX ].desc); } - -return 0 ; }/*
@@ -1529,9 +1549,10 @@ int kvm_arch_init_vcpu(CPUState *cs) env->apic_bus_freq =KVM_APIC_BUS_FREQUENCY ;/* Paravirtualization CPUIDs */ - r =hyperv_expand_features (cs); -if (r <0 ) { -return r; +hyperv_expand_features (cs, &local_err); +if (local_err) { +error_report_err (local_err); +return -ENOSYS ; }if (hyperv_enabled (cpu)) {
Signature · 1 line
2.29.2
PATCH v3 14/19 i386: adjust the expected KVM_GET_SUPPORTED_HV_CPUID array size
In reply to Vitaly KuznetsovSYNDBG leaves were recently (Linux-5.8) added to KVM but we haven't updated the expected size of KVM_GET_SUPPORTED_HV_CPUID output in KVM so we now make serveral tries before succeeding. Update the default.
--- target/i386/kvm/kvm.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
target/i386/kvm/kvm.c +2 -1
@@ -964,7 +964,8 @@ static struct kvm_cpuid2 *try_get_hv_cpuid(CPUState *cs, int max)static struct kvm_cpuid2 *get_supported_hv_cpuid (CPUState *cs) {struct kvm_cpuid2 *cpuid; -int max =7 ;/* 0x40000000..0x40000005, 0x4000000A */ +/* 0x40000000..0x40000005, 0x4000000A, 0x40000080..0x40000080 leaves */ +int max =10 ;int i;/*
Signature · 1 line
2.29.2
PATCH v3 15/19 i386: prefer system KVM_GET_SUPPORTED_HV_CPUID ioctl over vCPU's one
In reply to Vitaly KuznetsovKVM_GET_SUPPORTED_HV_CPUID was made a system wide ioctl which can be called prior to creating vCPUs and we are going to use that to expand Hyper-V cpu features early. Use it when it is supported by KVM.
--- target/i386/kvm/kvm.c | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-)
target/i386/kvm/kvm.c +13 -4
@@ -931,7 +931,8 @@ static struct { }, }; -static struct kvm_cpuid2 *try_get_hv_cpuid (CPUState *cs,int max) +static struct kvm_cpuid2 *try_get_hv_cpuid (CPUState *cs,int max, +bool do_sys_ioctl) {struct kvm_cpuid2 *cpuid;int r, size;
@@ -940,7 +941,11 @@ static struct kvm_cpuid2 *try_get_hv_cpuid(CPUState *cs, int max) cpuid =g_malloc0 (size); cpuid->nent = max; - r =kvm_vcpu_ioctl (cs,KVM_GET_SUPPORTED_HV_CPUID , cpuid); +if (do_sys_ioctl) { + r =kvm_ioctl (kvm_state,KVM_GET_SUPPORTED_HV_CPUID , cpuid); + }else { + r =kvm_vcpu_ioctl (cs,KVM_GET_SUPPORTED_HV_CPUID , cpuid); + }if (r ==0 && cpuid->nent >= max) { r = -E2BIG ; }
@@ -967,13 +972,17 @@ static struct kvm_cpuid2 *get_supported_hv_cpuid(CPUState *cs)/* 0x40000000..0x40000005, 0x4000000A, 0x40000080..0x40000080 leaves */ int max =10 ;int i; +bool do_sys_ioctl; + + do_sys_ioctl = +kvm_check_extension (kvm_state,KVM_CAP_SYS_HYPERV_CPUID ) >0 ;/* * When the buffer is too small, KVM_GET_SUPPORTED_HV_CPUID fails with * -E2BIG, however, it doesn't report back the right size. Keep increasing * it and re-trying until we succeed. */ -while ((cpuid =try_get_hv_cpuid (cs, max)) ==NULL ) { +while ((cpuid =try_get_hv_cpuid (cs, max, do_sys_ioctl)) ==NULL ) { max++; }
@@ -983,7 +992,7 @@ static struct kvm_cpuid2 *get_supported_hv_cpuid(CPUState *cs)* information early, just check for the capability and set the bit * manually. */ -if (kvm_check_extension (cs->kvm_state, +if (!do_sys_ioctl &&kvm_check_extension (cs->kvm_state,KVM_CAP_HYPERV_ENLIGHTENED_VMCS ) >0 ) {for (i =0 ; i < cpuid->nent; i++) {if (cpuid->entries[i].function ==HV_CPUID_ENLIGHTMENT_INFO ) {
Signature · 1 line
2.29.2
PATCH v3 16/19 i386: use global kvm_state in hyperv_enabled() check
In reply to Vitaly KuznetsovThere is no need to use vCPU-specific kvm state in hyperv_enabled() check and we need to do that when feature expansion happens early, before vCPU specific KVM state is created.
--- target/i386/kvm/kvm.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
target/i386/kvm/kvm.c +1 -2
@@ -718,8 +718,7 @@ unsigned long kvm_arch_vcpu_id(CPUState *cs)static bool hyperv_enabled (X86CPU *cpu) { -CPUState *cs =CPU (cpu); -return kvm_check_extension (cs->kvm_state,KVM_CAP_HYPERV ) >0 && +return kvm_check_extension (kvm_state,KVM_CAP_HYPERV ) >0 && ((cpu->hyperv_spinlock_attempts !=HYPERV_SPINLOCK_NEVER_NOTIFY ) || cpu->hyperv_features || cpu->hyperv_passthrough); }
Signature · 1 line
2.29.2
PATCH v3 17/19 i386: expand Hyper-V features during CPU feature expansion time
In reply to Vitaly KuznetsovTo make Hyper-V features appear in e.g. QMP query-cpu-model-expansion we need to expand and set the corresponding CPUID leaves early. Modify x86_cpu_get_supported_feature_word() to call newly intoduced Hyper-V specific kvm_hv_get_supported_cpuid() instead of kvm_arch_get_supported_cpuid(). We can't use kvm_arch_get_supported_cpuid() as Hyper-V specific CPUID leaves intersect with KVM's.
Note, early expansion will only happen when KVM supports system wide KVM_GET_SUPPORTED_HV_CPUID ioctl (KVM_CAP_SYS_HYPERV_CPUID).
--- target/i386/cpu.c | 4 ++++ target/i386/kvm/kvm-stub.c | 5 +++++ target/i386/kvm/kvm.c | 15 ++++++++++++--- target/i386/kvm/kvm_i386.h | 1 + 4 files changed, 22 insertions(+), 3 deletions(-)
target/i386/cpu.c +4 -0
@@ -6398,6 +6398,10 @@ static void x86_cpu_expand_features(X86CPU *cpu, Error **errp)if (env->cpuid_xlevel2 ==UINT32_MAX ) { env->cpuid_xlevel2 = env->cpuid_min_xlevel2; } + +if (kvm_enabled ()) { +kvm_hyperv_expand_features (cpu, errp); + } }/*
target/i386/kvm/kvm-stub.c +5 -0
@@ -44,3 +44,8 @@ bool kvm_hv_evmcs_available(void) {return false; } + +void kvm_hyperv_expand_features (X86CPU *cpu,Error **errp) +{ +return ; +}
target/i386/kvm/kvm.c +12 -3
@@ -1219,13 +1219,22 @@ static uint32_t hv_build_cpuid_leaf(CPUState *cs, uint32_t func, int reg)* of 'hv_passthrough' mode and fills the environment with all supported * Hyper-V features. */ -static void hyperv_expand_features (CPUState *cs,Error **errp) +void kvm_hyperv_expand_features (X86CPU *cpu,Error **errp) { -X86CPU *cpu =X86_CPU (cs); +CPUState *cs =CPU (cpu);if (!hyperv_enabled (cpu))return ; +/* +* When kvm_hyperv_expand_features is called at CPU feature expansion +* time per-CPU kvm_state is not available yet so we can only proceed +* when KVM_CAP_SYS_HYPERV_CPUID is supported. +*/ +if (!cs->kvm_state && + !kvm_check_extension (kvm_state,KVM_CAP_SYS_HYPERV_CPUID )) +return ; +if (cpu->hyperv_passthrough) { cpu->hyperv_vendor_id[0 ] =hv_cpuid_get_host (cs,HV_CPUID_VENDOR_AND_MAX_FUNCTIONS ,R_EBX );
@@ -1558,7 +1567,7 @@ int kvm_arch_init_vcpu(CPUState *cs) env->apic_bus_freq =KVM_APIC_BUS_FREQUENCY ;/* Paravirtualization CPUIDs */ -hyperv_expand_features (cs, &local_err); +kvm_hyperv_expand_features (cpu, &local_err);if (local_err) {error_report_err (local_err);return -ENOSYS ;
target/i386/kvm/kvm_i386.h +1 -0
@@ -48,6 +48,7 @@ bool kvm_has_waitpkg(void);bool kvm_hv_vpindex_settable (void );bool kvm_hv_evmcs_available (void ); +void kvm_hyperv_expand_features (X86CPU *cpu,Error **errp);uint64_t kvm_swizzle_msi_ext_dest_id (uint64_t address);
Signature · 1 line
2.29.2
PATCH v3 18/19 i386: provide simple 'hv-default=on' option
In reply to Vitaly KuznetsovEnabling Hyper-V emulation for a Windows VM is a tiring experience as it requires listing all currently supported enlightenments ("hv-*" CPU features) explicitly. We do have 'hv-passthrough' mode enabling everything but it can't be used in production as it prevents migration.
Introduce a simple 'hv-default=on' CPU flag enabling all currently supported Hyper-V enlightenments. Later, when new enlightenments get implemented, compat_props mechanism will be used to disable them for legacy machine types, this will keep 'hv-default=on' configurations migratable.
--- docs/hyperv.txt | 16 +++++++++++++--- target/i386/cpu.c | 38 ++++++++++++++++++++++++++++++++++++++ target/i386/cpu.h | 5 +++++ 3 files changed, 56 insertions(+), 3 deletions(-)
docs/hyperv.txt +13 -3
@@ -17,10 +17,20 @@ compatible hypervisor and use Hyper-V specific features. 2. Setup ========= -No Hyper-V enlightenments are enabled by default by either KVM or QEMU. In -QEMU, individual enlightenments can be enabled through CPU flags, e.g: +All currently supported Hyper-V enlightenments can be enabled by specifying +'hv-default=on' CPU flag: - qemu-system-x86_64 --enable-kvm --cpu host,hv_relaxed,hv_vpindex,hv_time, ... + qemu-system-x86_64 --enable-kvm --cpu host,hv-default ... + +Alternatively, it is possible to do fine-grained enablement through CPU flags, +e.g: + + qemu-system-x86_64 --enable-kvm --cpu host,hv-relaxed,hv-vpindex,hv-time ... + +It is also possible to disable individual enlightenments from the default list, +this can be used for debugging purposes: + + qemu-system-x86_64 --enable-kvm --cpu host,hv-default=on,hv-evmcs=off ... Sometimes there are dependencies between enlightenments, QEMU is supposed to check that the supplied configuration is sane.
target/i386/cpu.c +38 -0
@@ -4552,6 +4552,24 @@ static void x86_cpuid_set_tsc_freq(Object *obj, Visitor *v, const char *name, cpu->env.tsc_khz = cpu->env.user_tsc_khz = value /1000 ; } +static bool x86_hv_default_get (Object *obj,Error **errp) +{ +X86CPU *cpu =X86_CPU (obj); + +return cpu->hyperv_default; +} + +static void x86_hv_default_set (Object *obj,bool value,Error **errp) +{ +X86CPU *cpu =X86_CPU (obj); + + cpu->hyperv_default = value; + +if (value) { + cpu->hyperv_features |= cpu->hyperv_default_features; + } +} +/* Generic getter for "feature-words" and "filtered-features" properties */ static void x86_cpu_get_feature_words(Object *obj, Visitor *v, const char *name, void *opaque,
@@ -6955,10 +6973,26 @@ static void x86_cpu_initfn(Object *obj)object_property_add_alias (obj,"pause_filter" , obj,"pause-filter" );object_property_add_alias (obj,"sse4_1" , obj,"sse4.1" );object_property_add_alias (obj,"sse4_2" , obj,"sse4.2" ); +object_property_add_alias (obj,"hv_default" , obj,"hv-default" );if (xcc->model) {x86_cpu_load_model (cpu, xcc->model); } + +/* Hyper-V features enabled with 'hv-default=on' */ + cpu->hyperv_default_features =BIT (HYPERV_FEAT_RELAXED ) | +BIT (HYPERV_FEAT_VAPIC ) |BIT (HYPERV_FEAT_TIME ) | +BIT (HYPERV_FEAT_CRASH ) |BIT (HYPERV_FEAT_RESET ) | +BIT (HYPERV_FEAT_VPINDEX ) |BIT (HYPERV_FEAT_RUNTIME ) | +BIT (HYPERV_FEAT_SYNIC ) |BIT (HYPERV_FEAT_STIMER ) | +BIT (HYPERV_FEAT_FREQUENCIES ) |BIT (HYPERV_FEAT_REENLIGHTENMENT ) | +BIT (HYPERV_FEAT_TLBFLUSH ) |BIT (HYPERV_FEAT_IPI ) | +BIT (HYPERV_FEAT_STIMER_DIRECT ); + +/* Enlightened VMCS is only available on Intel/VMX */ +if (kvm_hv_evmcs_available ()) { + cpu->hyperv_default_features |=BIT (HYPERV_FEAT_EVMCS ); + } }static int64_t x86_cpu_get_arch_id (CPUState *cs)
@@ -7285,6 +7319,10 @@ static void x86_cpu_common_class_init(ObjectClass *oc, void *data) x86_cpu_get_crash_info_qom, NULL, NULL, NULL);#endif +object_class_property_add_bool (oc,"hv-default" , + x86_hv_default_get, + x86_hv_default_set); +for (w =0 ; w <FEATURE_WORDS ; w++) {int bitnr;for (bitnr =0 ; bitnr <64 ; bitnr++) {
target/i386/cpu.h +5 -0
@@ -1657,6 +1657,11 @@ struct X86CPU {bool hyperv_synic_kvm_only;uint64_t hyperv_features;bool hyperv_passthrough; + +/* 'hv-default' enablement */ +uint64_t hyperv_default_features; +bool hyperv_default; +OnOffAuto hyperv_no_nonarch_cs;uint32_t hyperv_vendor_id[3 ];uint32_t hyperv_interface_id[4 ];
Signature · 1 line
2.29.2
Re: [PATCH v3 18/19] i386: provide simple 'hv-default=on' option
Quoted 27 lines · On Thu, 7 Jan 2021 16:14:49 +0100 Vitaly Kuznetsov <[email protected]> wrote:
Enabling Hyper-V emulation for a Windows VM is a tiring experience as it requires listing all currently supported enlightenments ("hv-*" CPU features) explicitly. We do have 'hv-passthrough' mode enabling everything but it can't be used in production as it prevents migration.
Introduce a simple 'hv-default=on' CPU flag enabling all currently supported Hyper-V enlightenments. Later, when new enlightenments get implemented,
compat_propsmechanism will be used to disable them for legacy machine types, this will keep 'hv-default=on' configurations migratable.Signed-off-by Vitaly Kuznetsov <[email protected]>--- docs/hyperv.txt | 16 +++++++++++++--- target/i386/cpu.c | 38 ++++++++++++++++++++++++++++++++++++++ target/i386/cpu.h | 5 +++++ 3 files changed, 56 insertions(+), 3 deletions(-)docs/hyperv.txt +0
-2@@ -17,10 +17,20 @@ compatible hypervisor and use Hyper-V specific features. 2. Setup ========= -No Hyper-V enlightenments are enabled by default by either KVM or QEMU. In -QEMU, individual enlightenments can be enabled through CPU flags, e.g:
docs/hyperv.txt compatible hypervisor and use Hyper-V specific features. · line 28
Quoted 10 lines
docs/hyperv.txt +8
-1+All currently supported Hyper-V enlightenments can be enabled by specifying +'hv-default=on' CPU flag: - qemu-system-x86_64 --enable-kvm --cpu host,hv_relaxed,hv_vpindex,hv_time, ... + qemu-system-x86_64 --enable-kvm --cpu host,hv-default ... + +Alternatively, it is possible to do fine-grained enablement through CPU flags, +e.g: + + qemu-system-x86_64 --enable-kvm --cpu host,hv-relaxed,hv-vpindex,hv-time ...
I'd put here not '...' but rather recommended list of flags, and update it every time when new feature added if necessary.
(not to mention that if we had it to begin with, then new 'hv-default' won't be necessary, I still see it as functionality duplication but I will not oppose it)
Quoted 20 lines
docs/hyperv.txt +4
-0+It is also possible to disable individual enlightenments from the default list, +this can be used for debugging purposes: + + qemu-system-x86_64 --enable-kvm --cpu host,hv-default=on,hv-evmcs=off ... Sometimes there are dependencies between enlightenments, QEMU is supposed to check that the supplied configuration is sane.target/i386/cpu.c +5
-0@@ -4552,6 +4552,24 @@ static void x86_cpuid_set_tsc_freq(Object *obj, Visitor *v, const char *name, cpu->env.tsc_khz = cpu->env.user_tsc_khz = value /1000 ; } +static bool x86_hv_default_get (Object *obj,Error **errp) +{ +X86CPU *cpu =X86_CPU (obj); + +return cpu->hyperv_default;
target/i386/cpu.c static void x86_cpuid_set_tsc_freq(Object *obj, Visitor *v, const char *name, · line 4569
Quoted 10 lines
target/i386/cpu.c +10
-0+} + +static void x86_hv_default_set (Object *obj,bool value,Error **errp) +{ +X86CPU *cpu =X86_CPU (obj); + + cpu->hyperv_default = value; + +if (value) { + cpu->hyperv_features |= cpu->hyperv_default_features;
s/|="/=/ please,
i.e. no option overrides whatever was specified before to keep semantics consistent.
Quoted 20 lines
target/i386/cpu.c +9
-0+ } +} +/* Generic getter for "feature-words" and "filtered-features" properties */ static void x86_cpu_get_feature_words(Object *obj, Visitor *v, const char *name, void *opaque,@@ -6955,10 +6973,26 @@ static void x86_cpu_initfn(Object *obj)object_property_add_alias (obj,"pause_filter" , obj,"pause-filter" );object_property_add_alias (obj,"sse4_1" , obj,"sse4.1" );object_property_add_alias (obj,"sse4_2" , obj,"sse4.2" ); +object_property_add_alias (obj,"hv_default" , obj,"hv-default" );if (xcc->model) {x86_cpu_load_model (cpu, xcc->model); } + +/* Hyper-V features enabled with 'hv-default=on' */ + cpu->hyperv_default_features =BIT (HYPERV_FEAT_RELAXED ) | +BIT (HYPERV_FEAT_VAPIC ) |BIT (HYPERV_FEAT_TIME ) | + BIT(HYPERV_FEAT_CRASH) | BIT(HYPERV_FEAT_RESET) |
target/i386/cpu.c static void x86_cpu_initfn(Object *obj) · line 6995
Quoted 10 lines
target/i386/cpu.c +10
-0+BIT (HYPERV_FEAT_VPINDEX ) |BIT (HYPERV_FEAT_RUNTIME ) | +BIT (HYPERV_FEAT_SYNIC ) |BIT (HYPERV_FEAT_STIMER ) | +BIT (HYPERV_FEAT_FREQUENCIES ) |BIT (HYPERV_FEAT_REENLIGHTENMENT ) | +BIT (HYPERV_FEAT_TLBFLUSH ) |BIT (HYPERV_FEAT_IPI ) | +BIT (HYPERV_FEAT_STIMER_DIRECT ); + +/* Enlightened VMCS is only available on Intel/VMX */ +if (kvm_hv_evmcs_available ()) { + cpu->hyperv_default_features |=BIT (HYPERV_FEAT_EVMCS ); + }
what if VVM is migrated to another host without evmcs, will it change CPUID?
Quoted 30 lines
target/i386/cpu.c +4
-0}static int64_t x86_cpu_get_arch_id (CPUState *cs)@@ -7285,6 +7319,10 @@ static void x86_cpu_common_class_init(ObjectClass *oc, void *data) x86_cpu_get_crash_info_qom, NULL, NULL, NULL);#endif +object_class_property_add_bool (oc,"hv-default" , + x86_hv_default_get, + x86_hv_default_set); +for (w =0 ; w <FEATURE_WORDS ; w++) {int bitnr;for (bitnr =0 ; bitnr <64 ; bitnr++) {target/i386/cpu.h +5
-0@@ -1657,6 +1657,11 @@ struct X86CPU {bool hyperv_synic_kvm_only;uint64_t hyperv_features;bool hyperv_passthrough; + +/* 'hv-default' enablement */ +uint64_t hyperv_default_features; +bool hyperv_default; +OnOffAuto hyperv_no_nonarch_cs;uint32_t hyperv_vendor_id[3 ];uint32_t hyperv_interface_id[4 ];
Re: [PATCH v3 18/19] i386: provide simple 'hv-default=on' option
Quoted 33 lines · Igor Mammedov
Quoted 30 lines · On Thu, 7 Jan 2021 16:14:49 +0100 Vitaly Kuznetsov <[email protected]> wrote:
Enabling Hyper-V emulation for a Windows VM is a tiring experience as it requires listing all currently supported enlightenments ("hv-*" CPU features) explicitly. We do have 'hv-passthrough' mode enabling everything but it can't be used in production as it prevents migration.
Introduce a simple 'hv-default=on' CPU flag enabling all currently supported Hyper-V enlightenments. Later, when new enlightenments get implemented,
compat_propsmechanism will be used to disable them for legacy machine types, this will keep 'hv-default=on' configurations migratable.Signed-off-by Vitaly Kuznetsov <[email protected]>--- docs/hyperv.txt | 16 +++++++++++++--- target/i386/cpu.c | 38 ++++++++++++++++++++++++++++++++++++++ target/i386/cpu.h | 5 +++++ 3 files changed, 56 insertions(+), 3 deletions(-)docs/hyperv.txt +2
-2@@ -17,10 +17,20 @@ compatible hypervisor and use Hyper-V specific features. 2. Setup ========= -No Hyper-V enlightenments are enabled by default by either KVM or QEMU. In -QEMU, individual enlightenments can be enabled through CPU flags, e.g: +All currently supported Hyper-V enlightenments can be enabled by specifying +'hv-default=on' CPU flag:
Quoted 11 lines
docs/hyperv.txt compatible hypervisor and use Hyper-V specific features. · line 28
Quoted 7 lines
docs/hyperv.txt +6
-1- qemu-system-x86_64 --enable-kvm --cpu host,hv_relaxed,hv_vpindex,hv_time, ... + qemu-system-x86_64 --enable-kvm --cpu host,hv-default ... + +Alternatively, it is possible to do fine-grained enablement through CPU flags, +e.g: + + qemu-system-x86_64 --enable-kvm --cpu host,hv-relaxed,hv-vpindex,hv-time ...I'd put here not '...' but rather recommended list of flags, and update it every time when new feature added if necessary.
This is an example of fine-grained enablement, there is no point to put all the existing flags there (hv-default is the only recommended way now, the rest is 'expert'/'debugging').
Quoted 3 lines
(not to mention that if we had it to begin with, then new 'hv-default' won't be necessary, I still see it as functionality duplication but I will not oppose it)
Unfortunately, upper layer tools don't read this doc and update themselves to enable new features when they appear. Similarly, if when these tools use '-machine q35' they get all the new features we add automatically, right?
Quoted 26 lines
Quoted 26 lines
docs/hyperv.txt +4
-0+It is also possible to disable individual enlightenments from the default list, +this can be used for debugging purposes: + + qemu-system-x86_64 --enable-kvm --cpu host,hv-default=on,hv-evmcs=off ... Sometimes there are dependencies between enlightenments, QEMU is supposed to check that the supplied configuration is sane.target/i386/cpu.c +10
-0@@ -4552,6 +4552,24 @@ static void x86_cpuid_set_tsc_freq(Object *obj, Visitor *v, const char *name, cpu->env.tsc_khz = cpu->env.user_tsc_khz = value /1000 ; } +static bool x86_hv_default_get (Object *obj,Error **errp) +{ +X86CPU *cpu =X86_CPU (obj); + +return cpu->hyperv_default; +} + +static void x86_hv_default_set (Object *obj,bool value,Error **errp) +{ +X86CPU *cpu =X86_CPU (obj);
Quoted 9 lines
target/i386/cpu.c static void x86_cpuid_set_tsc_freq(Object *obj, Visitor *v, const char *name, · line 4569
Quoted 5 lines
target/i386/cpu.c +5
-0+ + cpu->hyperv_default = value; + +if (value) { + cpu->hyperv_features |= cpu->hyperv_default_features;s/|="/=/ please,
i.e. no option overrides whatever was specified before to keep semantics consistent.
Hm,
this doesn't matter for the most recent machine type as hyperv_default_features has all the features but imagine you're running an older machine type which doesn't have 'hv_feature'. Now your suggestion is
if I do:
'hv_default,hv_feature=on' I will get "hyperv_default_features | hv_feature"
but if I do
'hv_feature=on,hv_default' I will just get 'hyperv_default_features' (as hv_default enablement will overwrite everything)
How is this consistent?
Quoted 24 lines
Quoted 24 lines
target/i386/cpu.c +13
-0+ } +} +/* Generic getter for "feature-words" and "filtered-features" properties */ static void x86_cpu_get_feature_words(Object *obj, Visitor *v, const char *name, void *opaque,@@ -6955,10 +6973,26 @@ static void x86_cpu_initfn(Object *obj)object_property_add_alias (obj,"pause_filter" , obj,"pause-filter" );object_property_add_alias (obj,"sse4_1" , obj,"sse4.1" );object_property_add_alias (obj,"sse4_2" , obj,"sse4.2" ); +object_property_add_alias (obj,"hv_default" , obj,"hv-default" );if (xcc->model) {x86_cpu_load_model (cpu, xcc->model); } + +/* Hyper-V features enabled with 'hv-default=on' */ + cpu->hyperv_default_features =BIT (HYPERV_FEAT_RELAXED ) | +BIT (HYPERV_FEAT_VAPIC ) |BIT (HYPERV_FEAT_TIME ) | +BIT (HYPERV_FEAT_CRASH ) |BIT (HYPERV_FEAT_RESET ) | +BIT (HYPERV_FEAT_VPINDEX ) |BIT (HYPERV_FEAT_RUNTIME ) | +BIT (HYPERV_FEAT_SYNIC ) |BIT (HYPERV_FEAT_STIMER ) | +BIT (HYPERV_FEAT_FREQUENCIES ) |BIT (HYPERV_FEAT_REENLIGHTENMENT ) | + BIT(HYPERV_FEAT_TLBFLUSH) | BIT(HYPERV_FEAT_IPI) |
Quoted 9 lines
target/i386/cpu.c static void x86_cpu_initfn(Object *obj) · line 6995
Quoted 6 lines
target/i386/cpu.c +6
-0+BIT (HYPERV_FEAT_STIMER_DIRECT ); + +/* Enlightened VMCS is only available on Intel/VMX */ +if (kvm_hv_evmcs_available ()) { + cpu->hyperv_default_features |=BIT (HYPERV_FEAT_EVMCS ); + }what if VVM is migrated to another host without evmcs, will it change CPUID?
Evmcs is tightly coupled with VMX, we can't migrate when it's not there.
Quoted 31 lines
Quoted 30 lines
target/i386/cpu.c +4
-0}static int64_t x86_cpu_get_arch_id (CPUState *cs)@@ -7285,6 +7319,10 @@ static void x86_cpu_common_class_init(ObjectClass *oc, void *data) x86_cpu_get_crash_info_qom, NULL, NULL, NULL);#endif +object_class_property_add_bool (oc,"hv-default" , + x86_hv_default_get, + x86_hv_default_set); +for (w =0 ; w <FEATURE_WORDS ; w++) {int bitnr;for (bitnr =0 ; bitnr <64 ; bitnr++) {target/i386/cpu.h +5
-0@@ -1657,6 +1657,11 @@ struct X86CPU {bool hyperv_synic_kvm_only;uint64_t hyperv_features;bool hyperv_passthrough; + +/* 'hv-default' enablement */ +uint64_t hyperv_default_features; +bool hyperv_default; +OnOffAuto hyperv_no_nonarch_cs;uint32_t hyperv_vendor_id[3 ];uint32_t hyperv_interface_id[4 ];
Signature · 1 line
Vitaly
Re: [PATCH v3 18/19] i386: provide simple 'hv-default=on' option
Quoted 35 lines · On Fri, 15 Jan 2021 10:20:23 +0100 Vitaly Kuznetsov <[email protected]> wrote:
Quoted 33 lines · Igor Mammedov
Quoted 30 lines · On Thu, 7 Jan 2021 16:14:49 +0100 Vitaly Kuznetsov <[email protected]> wrote:
Enabling Hyper-V emulation for a Windows VM is a tiring experience as it requires listing all currently supported enlightenments ("hv-*" CPU features) explicitly. We do have 'hv-passthrough' mode enabling everything but it can't be used in production as it prevents migration.
Introduce a simple 'hv-default=on' CPU flag enabling all currently supported Hyper-V enlightenments. Later, when new enlightenments get implemented,
compat_propsmechanism will be used to disable them for legacy machine types, this will keep 'hv-default=on' configurations migratable.Signed-off-by Vitaly Kuznetsov <[email protected]>--- docs/hyperv.txt | 16 +++++++++++++--- target/i386/cpu.c | 38 ++++++++++++++++++++++++++++++++++++++ target/i386/cpu.h | 5 +++++ 3 files changed, 56 insertions(+), 3 deletions(-)docs/hyperv.txt +2
-2@@ -17,10 +17,20 @@ compatible hypervisor and use Hyper-V specific features. 2. Setup ========= -No Hyper-V enlightenments are enabled by default by either KVM or QEMU. In -QEMU, individual enlightenments can be enabled through CPU flags, e.g: +All currently supported Hyper-V enlightenments can be enabled by specifying +'hv-default=on' CPU flag:
Quoted 11 lines
Quoted 11 lines
docs/hyperv.txt compatible hypervisor and use Hyper-V specific features. · line 28
Quoted 7 lines
docs/hyperv.txt +6
-1- qemu-system-x86_64 --enable-kvm --cpu host,hv_relaxed,hv_vpindex,hv_time, ... + qemu-system-x86_64 --enable-kvm --cpu host,hv-default ... + +Alternatively, it is possible to do fine-grained enablement through CPU flags, +e.g: + + qemu-system-x86_64 --enable-kvm --cpu host,hv-relaxed,hv-vpindex,hv-time ...I'd put here not '...' but rather recommended list of flags, and update it every time when new feature added if necessary.
1)
Quoted 3 lines
This is an example of fine-grained enablement, there is no point to put all the existing flags there (hv-default is the only recommended way now, the rest is 'expert'/'debugging').
so users are kept in dark what hv-default disables/enables (and it might depend on machine version on top that). Doesn't look like a good documentation to me (sure everyone can go and read source code for it and try to figure out how it's supposed to work)
Quoted 7 lines
Quoted 4 lines
(not to mention that if we had it to begin with, then new 'hv-default' won't be necessary, I still see it as functionality duplication but I will not oppose it)
Unfortunately, upper layer tools don't read this doc and update themselves to enable new features when they appear.
rant: (just merge all libvirt into QEMU, and make VM configuration less low-level. why stop there, just merge with yet another upper layer, it would save us a lot on communication protocols and simplify VM creation even more,
and no one will have to read docs and write anything new on top.) There should be limit somewhere, where QEMU job ends and others pile hw abstraction layers on top of it.
Quoted 2 lines
Similarly, if when these tools use '-machine q35' they get all the new features we add automatically, right?
it depends, in case of CPUs, new features usually 'off' by default for existing models. In case of bugs, features sometimes could be flipped and versioned machines were used to keep broken CPU models on old machine types.
Quoted 35 lines
Quoted 25 lines
Quoted 25 lines
docs/hyperv.txt +4
-0+It is also possible to disable individual enlightenments from the default list, +this can be used for debugging purposes: + + qemu-system-x86_64 --enable-kvm --cpu host,hv-default=on,hv-evmcs=off ... Sometimes there are dependencies between enlightenments, QEMU is supposed to check that the supplied configuration is sane.target/i386/cpu.c +10
-0@@ -4552,6 +4552,24 @@ static void x86_cpuid_set_tsc_freq(Object *obj, Visitor *v, const char *name, cpu->env.tsc_khz = cpu->env.user_tsc_khz = value /1000 ; } +static bool x86_hv_default_get (Object *obj,Error **errp) +{ +X86CPU *cpu =X86_CPU (obj); + +return cpu->hyperv_default; +} + +static void x86_hv_default_set (Object *obj,bool value,Error **errp) +{ +X86CPU *cpu =X86_CPU (obj);Quoted 9 lines
target/i386/cpu.c static void x86_cpuid_set_tsc_freq(Object *obj, Visitor *v, const char *name, · line 4569
Quoted 5 lines
target/i386/cpu.c +5
-0+ + cpu->hyperv_default = value; + +if (value) { + cpu->hyperv_features |= cpu->hyperv_default_features;s/|="/=/ please,
i.e. no option overrides whatever was specified before to keep semantics consistent.
Quoted 6 lines
Hm,
this doesn't matter for the most recent machine type as
hyperv_default_featureshas all the features but imagine you're running an older machine type which doesn't have 'hv_feature'. Now your
normally one shouldn't use new feature with old machine type as it makes VM non-migratable to older QEMU that has this machine type but not this feature.
nitpicking:
according to (1) user should not use 'hv_feature' on old machine since hv_default should cover all their needs (well they don't know what hv_default actually is).
Quoted 4 lines
suggestion is
if I do:
Quoted 8 lines
'hv_default,hv_feature=on' I will get "
hyperv_default_features|hv_feature"but if I do
'hv_feature=on,hv_default' I will just get '
hyperv_default_features' (ashv_defaultenablement will overwrite everything)How is this consistent?
usual semantics for properties, is that the latest property overwrites, the previous property value parsed from left to right.
(i.e. if one asked for hv_default, one gets it related CPUID bit set/unset, if one needs more than that one should add more related features after that.
Quoted 34 lines
Quoted 24 lines
Quoted 24 lines
target/i386/cpu.c +13
-0+ } +} +/* Generic getter for "feature-words" and "filtered-features" properties */ static void x86_cpu_get_feature_words(Object *obj, Visitor *v, const char *name, void *opaque,@@ -6955,10 +6973,26 @@ static void x86_cpu_initfn(Object *obj)object_property_add_alias (obj,"pause_filter" , obj,"pause-filter" );object_property_add_alias (obj,"sse4_1" , obj,"sse4.1" );object_property_add_alias (obj,"sse4_2" , obj,"sse4.2" ); +object_property_add_alias (obj,"hv_default" , obj,"hv-default" );if (xcc->model) {x86_cpu_load_model (cpu, xcc->model); } + +/* Hyper-V features enabled with 'hv-default=on' */ + cpu->hyperv_default_features =BIT (HYPERV_FEAT_RELAXED ) | +BIT (HYPERV_FEAT_VAPIC ) |BIT (HYPERV_FEAT_TIME ) | +BIT (HYPERV_FEAT_CRASH ) |BIT (HYPERV_FEAT_RESET ) | +BIT (HYPERV_FEAT_VPINDEX ) |BIT (HYPERV_FEAT_RUNTIME ) | +BIT (HYPERV_FEAT_SYNIC ) |BIT (HYPERV_FEAT_STIMER ) | +BIT (HYPERV_FEAT_FREQUENCIES ) |BIT (HYPERV_FEAT_REENLIGHTENMENT ) | + BIT(HYPERV_FEAT_TLBFLUSH) | BIT(HYPERV_FEAT_IPI) |Quoted 9 lines
target/i386/cpu.c static void x86_cpu_initfn(Object *obj) · line 6995
Quoted 6 lines
target/i386/cpu.c +6
-0+BIT (HYPERV_FEAT_STIMER_DIRECT ); + +/* Enlightened VMCS is only available on Intel/VMX */ +if (kvm_hv_evmcs_available ()) { + cpu->hyperv_default_features |=BIT (HYPERV_FEAT_EVMCS ); + }what if VVM is migrated to another host without evmcs, will it change CPUID?
Quoted 2 lines
Evmcs is tightly coupled with VMX, we can't migrate when it's not there.
Are you saying mgmt will check and refuse to migrate to such host?
Quoted 33 lines
Quoted 32 lines
Quoted 31 lines
target/i386/cpu.c +4
-0}static int64_t x86_cpu_get_arch_id (CPUState *cs)@@ -7285,6 +7319,10 @@ static void x86_cpu_common_class_init(ObjectClass *oc, void *data) x86_cpu_get_crash_info_qom, NULL, NULL, NULL);#endif +object_class_property_add_bool (oc,"hv-default" , + x86_hv_default_get, + x86_hv_default_set); +for (w =0 ; w <FEATURE_WORDS ; w++) {int bitnr;for (bitnr =0 ; bitnr <64 ; bitnr++) {target/i386/cpu.h +5
-0@@ -1657,6 +1657,11 @@ struct X86CPU {bool hyperv_synic_kvm_only;uint64_t hyperv_features;bool hyperv_passthrough; + +/* 'hv-default' enablement */ +uint64_t hyperv_default_features; +bool hyperv_default; +OnOffAuto hyperv_no_nonarch_cs;uint32_t hyperv_vendor_id[3 ];uint32_t hyperv_interface_id[4 ];
Re: [PATCH v3 18/19] i386: provide simple 'hv-default=on' option
Quoted 52 lines · Igor Mammedov
Quoted 35 lines · On Fri, 15 Jan 2021 10:20:23 +0100 Vitaly Kuznetsov <[email protected]> wrote:
Quoted 33 lines · Igor Mammedov
Quoted 30 lines · On Thu, 7 Jan 2021 16:14:49 +0100 Vitaly Kuznetsov <[email protected]> wrote:
Enabling Hyper-V emulation for a Windows VM is a tiring experience as it requires listing all currently supported enlightenments ("hv-*" CPU features) explicitly. We do have 'hv-passthrough' mode enabling everything but it can't be used in production as it prevents migration.
Introduce a simple 'hv-default=on' CPU flag enabling all currently supported Hyper-V enlightenments. Later, when new enlightenments get implemented,
compat_propsmechanism will be used to disable them for legacy machine types, this will keep 'hv-default=on' configurations migratable.Signed-off-by Vitaly Kuznetsov <[email protected]>--- docs/hyperv.txt | 16 +++++++++++++--- target/i386/cpu.c | 38 ++++++++++++++++++++++++++++++++++++++ target/i386/cpu.h | 5 +++++ 3 files changed, 56 insertions(+), 3 deletions(-)docs/hyperv.txt +2
-2@@ -17,10 +17,20 @@ compatible hypervisor and use Hyper-V specific features. 2. Setup ========= -No Hyper-V enlightenments are enabled by default by either KVM or QEMU. In -QEMU, individual enlightenments can be enabled through CPU flags, e.g: +All currently supported Hyper-V enlightenments can be enabled by specifying +'hv-default=on' CPU flag:Quoted 11 lines
Quoted 11 lines
docs/hyperv.txt compatible hypervisor and use Hyper-V specific features. · line 28
Quoted 7 lines
docs/hyperv.txt +6
-1- qemu-system-x86_64 --enable-kvm --cpu host,hv_relaxed,hv_vpindex,hv_time, ... + qemu-system-x86_64 --enable-kvm --cpu host,hv-default ... + +Alternatively, it is possible to do fine-grained enablement through CPU flags, +e.g: + + qemu-system-x86_64 --enable-kvm --cpu host,hv-relaxed,hv-vpindex,hv-time ...I'd put here not '...' but rather recommended list of flags, and update it every time when new feature added if necessary.
1)
Quoted 7 lines
Quoted 3 lines
This is an example of fine-grained enablement, there is no point to put all the existing flags there (hv-default is the only recommended way now, the rest is 'expert'/'debugging').
so users are kept in dark what hv-default disables/enables (and it might depend on machine version on top that). Doesn't look like a good documentation to me (sure everyone can go and read source code for it and try to figure out how it's supposed to work)
'hv-default' enables *all* currently supported enlightenments. When using with an old machine type, it will enable *all* Hyper-V enlightenmnets which were supported when the corresponding machine type was released. I don't think we document all other cases when a machine type is modified (i.e. where can I read how pc-q35-5.1 is different from pc-q35-5.0 if I refuse to read the source code?)
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
(not to mention that if we had it to begin with, then new 'hv-default' won't be necessary, I still see it as functionality duplication but I will not oppose it)
Quoted 8 lines
Quoted 2 lines
Unfortunately, upper layer tools don't read this doc and update themselves to enable new features when they appear.
rant: (just merge all libvirt into QEMU, and make VM configuration less low-level. why stop there, just merge with yet another upper layer, it would save us a lot on communication protocols and simplify VM creation even more,
and no one will have to read docs and write anything new on top.) There should be limit somewhere, where QEMU job ends and others pile hw abstraction layers on top of it.
We have '-machine q35' and we don't require to list all the devices from it. We have '-cpu Skylake-Server' and we don't require to configure all the features manually. Why can't we have similar enablement for Hyper-V emulation where we can't even see a real need for anything but 'enable everything' option?
There is no 'one libvirt to rule them all' (fortunately or unfortunately). And sometimes QEMU is the uppermost layer and there's no 'libvirt' on top of it, this is also a perfectly valid use-case.
Quoted 8 lines
Quoted 3 lines
Similarly, if when these tools use '-machine q35' they get all the new features we add automatically, right?
it depends, in case of CPUs, new features usually 'off' by default for existing models. In case of bugs, features sometimes could be flipped and versioned machines were used to keep broken CPU models on old machine types.
That's why I was saying that Hyper-V enlightenments hardly resemble 'hardware' CPU features.
Quoted 42 lines
Quoted 36 lines
Quoted 26 lines
Quoted 26 lines
docs/hyperv.txt +4
-0+It is also possible to disable individual enlightenments from the default list, +this can be used for debugging purposes: + + qemu-system-x86_64 --enable-kvm --cpu host,hv-default=on,hv-evmcs=off ... Sometimes there are dependencies between enlightenments, QEMU is supposed to check that the supplied configuration is sane.target/i386/cpu.c +10
-0@@ -4552,6 +4552,24 @@ static void x86_cpuid_set_tsc_freq(Object *obj, Visitor *v, const char *name, cpu->env.tsc_khz = cpu->env.user_tsc_khz = value /1000 ; } +static bool x86_hv_default_get (Object *obj,Error **errp) +{ +X86CPU *cpu =X86_CPU (obj); + +return cpu->hyperv_default; +} + +static void x86_hv_default_set (Object *obj,bool value,Error **errp) +{ +X86CPU *cpu =X86_CPU (obj);Quoted 9 lines
target/i386/cpu.c static void x86_cpuid_set_tsc_freq(Object *obj, Visitor *v, const char *name, · line 4569
Quoted 5 lines
target/i386/cpu.c +5
-0+ + cpu->hyperv_default = value; + +if (value) { + cpu->hyperv_features |= cpu->hyperv_default_features;s/|="/=/ please,
i.e. no option overrides whatever was specified before to keep semantics consistent.Quoted 6 lines
Hm,
this doesn't matter for the most recent machine type as
hyperv_default_featureshas all the features but imagine you're running an older machine type which doesn't have 'hv_feature'. Now your
Quoted 7 lines
normally one shouldn't use new feature with old machine type as it makes VM non-migratable to older QEMU that has this machine type but not this feature.
nitpicking:
according to (1) user should not use 'hv_feature' on old machine sincehv_defaultshould cover all their needs (well they don't know whathv_defaultactually is).
Normally yes but I can imagine sticking to some old machine type for other-than-hyperv-enlightenments purposes and still wanting to add a newly introduced enlightenment. Migration is not always a must.
Quoted 9 lines
Quoted 9 lines
suggestion is
if I do:
'hv_default,hv_feature=on' I will get "
hyperv_default_features|hv_feature"but if I do
Quoted 9 lines
Quoted 4 lines
'hv_feature=on,hv_default' I will just get '
hyperv_default_features' (ashv_defaultenablement will overwrite everything)How is this consistent?
usual semantics for properties, is that the latest property overwrites, the previous property value parsed from left to right.
(i.e. if one asked forhv_default, one gets it related CPUID bit set/unset, if one needs more than that one should add more related features after that.
This semantics probably doesn't apply to 'hv-default' case IMO as my brain refuses to accept the fact that
'hv_default,hv_feature' != 'hv_feature,hv_default'
which should express the same desire 'the default set PLUS the feature I want'.
I think I prefer sanity over purity in this case.
Quoted 35 lines
Quoted 35 lines
Quoted 34 lines
Quoted 21 lines
target/i386/cpu.c +9
-0+ } +} +/* Generic getter for "feature-words" and "filtered-features" properties */ static void x86_cpu_get_feature_words(Object *obj, Visitor *v, const char *name, void *opaque,@@ -6955,10 +6973,26 @@ static void x86_cpu_initfn(Object *obj)object_property_add_alias (obj,"pause_filter" , obj,"pause-filter" );object_property_add_alias (obj,"sse4_1" , obj,"sse4.1" );object_property_add_alias (obj,"sse4_2" , obj,"sse4.2" ); +object_property_add_alias (obj,"hv_default" , obj,"hv-default" );if (xcc->model) {x86_cpu_load_model (cpu, xcc->model); } + +/* Hyper-V features enabled with 'hv-default=on' */ + cpu->hyperv_default_features =BIT (HYPERV_FEAT_RELAXED ) | +BIT (HYPERV_FEAT_VAPIC ) |BIT (HYPERV_FEAT_TIME ) | + BIT(HYPERV_FEAT_CRASH) | BIT(HYPERV_FEAT_RESET) |target/i386/cpu.c static void x86_cpu_initfn(Object *obj) · line 6995
Quoted 10 lines
target/i386/cpu.c +10
-0+BIT (HYPERV_FEAT_VPINDEX ) |BIT (HYPERV_FEAT_RUNTIME ) | +BIT (HYPERV_FEAT_SYNIC ) |BIT (HYPERV_FEAT_STIMER ) | +BIT (HYPERV_FEAT_FREQUENCIES ) |BIT (HYPERV_FEAT_REENLIGHTENMENT ) | +BIT (HYPERV_FEAT_TLBFLUSH ) |BIT (HYPERV_FEAT_IPI ) | +BIT (HYPERV_FEAT_STIMER_DIRECT ); + +/* Enlightened VMCS is only available on Intel/VMX */ +if (kvm_hv_evmcs_available ()) { + cpu->hyperv_default_features |=BIT (HYPERV_FEAT_EVMCS ); + }what if VVM is migrated to another host without evmcs, will it change CPUID?
Quoted 5 lines
Quoted 2 lines
Evmcs is tightly coupled with VMX, we can't migrate when it's not there.
Are you saying mgmt will check and refuse to migrate to such host?
Is it possible to migrate a VM from a VMX-enabled host to a VMX-disabled one if VMX feature was exposed to the VM? Probably not, you will fail to create a VM on the destination host. Evmcs doesn't change anything in this regard, there are no hosts where VMX is available but EVMCS is not.
Quoted 34 lines
Quoted 33 lines
Quoted 32 lines
Quoted 31 lines
target/i386/cpu.c +4
-0}static int64_t x86_cpu_get_arch_id (CPUState *cs)@@ -7285,6 +7319,10 @@ static void x86_cpu_common_class_init(ObjectClass *oc, void *data) x86_cpu_get_crash_info_qom, NULL, NULL, NULL);#endif +object_class_property_add_bool (oc,"hv-default" , + x86_hv_default_get, + x86_hv_default_set); +for (w =0 ; w <FEATURE_WORDS ; w++) {int bitnr;for (bitnr =0 ; bitnr <64 ; bitnr++) {target/i386/cpu.h +5
-0@@ -1657,6 +1657,11 @@ struct X86CPU {bool hyperv_synic_kvm_only;uint64_t hyperv_features;bool hyperv_passthrough; + +/* 'hv-default' enablement */ +uint64_t hyperv_default_features; +bool hyperv_default; +OnOffAuto hyperv_no_nonarch_cs;uint32_t hyperv_vendor_id[3 ];uint32_t hyperv_interface_id[4 ];
Signature · 1 line
Vitaly
Re: [PATCH v3 18/19] i386: provide simple 'hv-default=on' option
Quoted 97 lines · On Wed, 20 Jan 2021 15:38:33 +0100 Vitaly Kuznetsov <[email protected]> wrote:
Quoted 52 lines · Igor Mammedov
Quoted 35 lines · On Fri, 15 Jan 2021 10:20:23 +0100 Vitaly Kuznetsov <[email protected]> wrote:
Quoted 33 lines · Igor Mammedov
Quoted 30 lines · On Thu, 7 Jan 2021 16:14:49 +0100 Vitaly Kuznetsov <[email protected]> wrote:
Enabling Hyper-V emulation for a Windows VM is a tiring experience as it requires listing all currently supported enlightenments ("hv-*" CPU features) explicitly. We do have 'hv-passthrough' mode enabling everything but it can't be used in production as it prevents migration.
Introduce a simple 'hv-default=on' CPU flag enabling all currently supported Hyper-V enlightenments. Later, when new enlightenments get implemented,
compat_propsmechanism will be used to disable them for legacy machine types, this will keep 'hv-default=on' configurations migratable.Signed-off-by Vitaly Kuznetsov <[email protected]>--- docs/hyperv.txt | 16 +++++++++++++--- target/i386/cpu.c | 38 ++++++++++++++++++++++++++++++++++++++ target/i386/cpu.h | 5 +++++ 3 files changed, 56 insertions(+), 3 deletions(-)docs/hyperv.txt +2
-2@@ -17,10 +17,20 @@ compatible hypervisor and use Hyper-V specific features. 2. Setup ========= -No Hyper-V enlightenments are enabled by default by either KVM or QEMU. In -QEMU, individual enlightenments can be enabled through CPU flags, e.g: +All currently supported Hyper-V enlightenments can be enabled by specifying +'hv-default=on' CPU flag:Quoted 11 lines
Quoted 11 lines
docs/hyperv.txt compatible hypervisor and use Hyper-V specific features. · line 28
Quoted 7 lines
docs/hyperv.txt +6
-1- qemu-system-x86_64 --enable-kvm --cpu host,hv_relaxed,hv_vpindex,hv_time, ... + qemu-system-x86_64 --enable-kvm --cpu host,hv-default ... + +Alternatively, it is possible to do fine-grained enablement through CPU flags, +e.g: + + qemu-system-x86_64 --enable-kvm --cpu host,hv-relaxed,hv-vpindex,hv-time ...I'd put here not '...' but rather recommended list of flags, and update it every time when new feature added if necessary.
1)
Quoted 7 lines
Quoted 3 lines
This is an example of fine-grained enablement, there is no point to put all the existing flags there (hv-default is the only recommended way now, the rest is 'expert'/'debugging').
so users are kept in dark what hv-default disables/enables (and it might depend on machine version on top that). Doesn't look like a good documentation to me (sure everyone can go and read source code for it and try to figure out how it's supposed to work)
'hv-default' enables *all* currently supported enlightenments. When using with an old machine type, it will enable *all* Hyper-V enlightenmnets which were supported when the corresponding machine type was released. I don't think we document all other cases when a machine type is modified (i.e. where can I read how pc-q35-5.1 is different from pc-q35-5.0 if I refuse to read the source code?)
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
(not to mention that if we had it to begin with, then new 'hv-default' won't be necessary, I still see it as functionality duplication but I will not oppose it)
Quoted 8 lines
Quoted 2 lines
Unfortunately, upper layer tools don't read this doc and update themselves to enable new features when they appear.
rant: (just merge all libvirt into QEMU, and make VM configuration less low-level. why stop there, just merge with yet another upper layer, it would save us a lot on communication protocols and simplify VM creation even more,
and no one will have to read docs and write anything new on top.) There should be limit somewhere, where QEMU job ends and others pile hw abstraction layers on top of it.We have '-machine q35' and we don't require to list all the devices from it. We have '-cpu Skylake-Server' and we don't require to configure all the features manually. Why can't we have similar enablement for Hyper-V emulation where we can't even see a real need for anything but 'enable everything' option?
There is no 'one libvirt to rule them all' (fortunately or unfortunately). And sometimes QEMU is the uppermost layer and there's no 'libvirt' on top of it, this is also a perfectly valid use-case.
Quoted 3 lines
Quoted 3 lines
Similarly, if when these tools use '-machine q35' they get all the new features we add automatically, right?
Quoted 8 lines
Quoted 5 lines
it depends, in case of CPUs, new features usually 'off' by default for existing models. In case of bugs, features sometimes could be flipped and versioned machines were used to keep broken CPU models on old machine types.
That's why I was saying that Hyper-V enlightenments hardly resemble 'hardware' CPU features.
Well, Microsoft chose to implement them as hardware concept (CPUID leaf), and I prefer to treat them the same way as any other CPUID bits.
Quoted 68 lines
Quoted 43 lines
Quoted 37 lines
Quoted 27 lines
Quoted 27 lines
docs/hyperv.txt +4
-0+It is also possible to disable individual enlightenments from the default list, +this can be used for debugging purposes: + + qemu-system-x86_64 --enable-kvm --cpu host,hv-default=on,hv-evmcs=off ... Sometimes there are dependencies between enlightenments, QEMU is supposed to check that the supplied configuration is sane.target/i386/cpu.c +10
-0@@ -4552,6 +4552,24 @@ static void x86_cpuid_set_tsc_freq(Object *obj, Visitor *v, const char *name, cpu->env.tsc_khz = cpu->env.user_tsc_khz = value /1000 ; } +static bool x86_hv_default_get (Object *obj,Error **errp) +{ +X86CPU *cpu =X86_CPU (obj); + +return cpu->hyperv_default; +} + +static void x86_hv_default_set (Object *obj,bool value,Error **errp) +{ +X86CPU *cpu =X86_CPU (obj);Quoted 9 lines
target/i386/cpu.c static void x86_cpuid_set_tsc_freq(Object *obj, Visitor *v, const char *name, · line 4569
Quoted 5 lines
target/i386/cpu.c +5
-0+ + cpu->hyperv_default = value; + +if (value) { + cpu->hyperv_features |= cpu->hyperv_default_features;s/|="/=/ please,
i.e. no option overrides whatever was specified before to keep semantics consistent.Quoted 6 lines
Hm,
this doesn't matter for the most recent machine type as
hyperv_default_featureshas all the features but imagine you're running an older machine type which doesn't have 'hv_feature'. Now yourQuoted 7 lines
normally one shouldn't use new feature with old machine type as it makes VM non-migratable to older QEMU that has this machine type but not this feature.
nitpicking:
according to (1) user should not use 'hv_feature' on old machine sincehv_defaultshould cover all their needs (well they don't know whathv_defaultactually is).Normally yes but I can imagine sticking to some old machine type for other-than-hyperv-enlightenments purposes and still wanting to add a newly introduced enlightenment. Migration is not always a must.
Quoted 13 lines
Quoted 13 lines
suggestion is
if I do:
'hv_default,hv_feature=on' I will get "
hyperv_default_features|hv_feature"but if I do
'hv_feature=on,hv_default' I will just get '
hyperv_default_features' (ashv_defaultenablement will overwrite everything)How is this consistent?
Quoted 8 lines
Quoted 5 lines
usual semantics for properties, is that the latest property overwrites, the previous property value parsed from left to right.
(i.e. if one asked forhv_default, one gets it related CPUID bit set/unset, if one needs more than that one should add more related features after that.This semantics probably doesn't apply to 'hv-default' case IMO as my brain refuses to accept the fact that
it's difficult probably because 'hv-default' is 'alias' property that covers all individual hv-foo features in one go and that individual features are exposed to user, but otherwise it is just a property that sets CPUID features or like any other property, and should be treated like such.
Quoted 4 lines
'hv_default,hv_feature' != 'hv_feature,hv_default'
which should express the same desire 'the default set PLUS the feature I want'.
if hv_default were touching different data, I'd agree.
But in the end hv_default boils down to the same CPUID bits as individual features:
hv_default,hv_f2 => (hv_f1=on,hv_f2=off),hv_f2=on
!=
hv_f2,hv_default => hv_f2=on,(hv_f1=on,hv_f2=off)
Quoted 1 line
I think I prefer sanity over purity in this case.
what is sanity to one could be insanity for another, so I pointed out the way properties expected to work today.
But you are adding new semantic ('combine') to property/features parsing (instead of current 'set' policy), and users will have to be aware of this new behavior and add/maintain code for this special case. (maybe I worry in vain, and no one will read docs and know about this new property anyways)
That will also push x86 CPUs consolidation farther away from other targets, where there aren't any special casing for features parsing, just simple left to right parsing with the latest property having overwriting previously set value.
We are trying hard to reduce special cases and unify interfaces for same components to simplify qemu and make it predictable/easier for users.
Quoted 37 lines
Quoted 37 lines
Quoted 36 lines
Quoted 24 lines
Quoted 24 lines
target/i386/cpu.c +13
-0+ } +} +/* Generic getter for "feature-words" and "filtered-features" properties */ static void x86_cpu_get_feature_words(Object *obj, Visitor *v, const char *name, void *opaque,@@ -6955,10 +6973,26 @@ static void x86_cpu_initfn(Object *obj)object_property_add_alias (obj,"pause_filter" , obj,"pause-filter" );object_property_add_alias (obj,"sse4_1" , obj,"sse4.1" );object_property_add_alias (obj,"sse4_2" , obj,"sse4.2" ); +object_property_add_alias (obj,"hv_default" , obj,"hv-default" );if (xcc->model) {x86_cpu_load_model (cpu, xcc->model); } + +/* Hyper-V features enabled with 'hv-default=on' */ + cpu->hyperv_default_features =BIT (HYPERV_FEAT_RELAXED ) | +BIT (HYPERV_FEAT_VAPIC ) |BIT (HYPERV_FEAT_TIME ) | +BIT (HYPERV_FEAT_CRASH ) |BIT (HYPERV_FEAT_RESET ) | +BIT (HYPERV_FEAT_VPINDEX ) |BIT (HYPERV_FEAT_RUNTIME ) | +BIT (HYPERV_FEAT_SYNIC ) |BIT (HYPERV_FEAT_STIMER ) | +BIT (HYPERV_FEAT_FREQUENCIES ) |BIT (HYPERV_FEAT_REENLIGHTENMENT ) | + BIT(HYPERV_FEAT_TLBFLUSH) | BIT(HYPERV_FEAT_IPI) |Quoted 9 lines
target/i386/cpu.c static void x86_cpu_initfn(Object *obj) · line 6995
Quoted 6 lines
target/i386/cpu.c +6
-0+BIT (HYPERV_FEAT_STIMER_DIRECT ); + +/* Enlightened VMCS is only available on Intel/VMX */ +if (kvm_hv_evmcs_available ()) { + cpu->hyperv_default_features |=BIT (HYPERV_FEAT_EVMCS ); + }what if VVM is migrated to another host without evmcs, will it change CPUID?
Evmcs is tightly coupled with VMX, we can't migrate when it's not there.
Quoted 7 lines
Quoted 2 lines
Are you saying mgmt will check and refuse to migrate to such host?
Is it possible to migrate a VM from a VMX-enabled host to a VMX-disabled one if VMX feature was exposed to the VM? Probably not, you will fail to create a VM on the destination host. Evmcs doesn't change anything in this regard, there are no hosts where VMX is available but EVMCS is not.
I'm not sure how evmcs should be handled,
can you point out what in this series makes sure that migration fails or makes qemu not able to start in case kvm_hv_evmcs_available() returns false.
So far I read snippet above as a problem:
1:
host supports evmcs:
and exposes HYPERV_FEAT_EVMCS in CPUID
2: we migrate to host without evmcs
2.1 start target QEMU, it happily creates vCPUs without HYPERV_FEAT_EVMCS in CPUID 2.2 if I'm not mistaken CPUID is not part of migration stream,
nothing could check and fail migration
2.3 guest runs fine till it tries to use non existing feature, ...
Quoted 34 lines
Quoted 33 lines
Quoted 32 lines
Quoted 31 lines
Quoted 30 lines
target/i386/cpu.c +4
-0}static int64_t x86_cpu_get_arch_id (CPUState *cs)@@ -7285,6 +7319,10 @@ static void x86_cpu_common_class_init(ObjectClass *oc, void *data) x86_cpu_get_crash_info_qom, NULL, NULL, NULL);#endif +object_class_property_add_bool (oc,"hv-default" , + x86_hv_default_get, + x86_hv_default_set); +for (w =0 ; w <FEATURE_WORDS ; w++) {int bitnr;for (bitnr =0 ; bitnr <64 ; bitnr++) {target/i386/cpu.h +5
-0@@ -1657,6 +1657,11 @@ struct X86CPU {bool hyperv_synic_kvm_only;uint64_t hyperv_features;bool hyperv_passthrough; + +/* 'hv-default' enablement */ +uint64_t hyperv_default_features; +bool hyperv_default; +OnOffAuto hyperv_no_nonarch_cs;uint32_t hyperv_vendor_id[3 ];uint32_t hyperv_interface_id[4 ];
Re: [PATCH v3 18/19] i386: provide simple 'hv-default=on' option
Quoted 196 lines · Igor Mammedov
Quoted 97 lines · On Wed, 20 Jan 2021 15:38:33 +0100 Vitaly Kuznetsov <[email protected]> wrote:
Quoted 52 lines · Igor Mammedov
Quoted 35 lines · On Fri, 15 Jan 2021 10:20:23 +0100 Vitaly Kuznetsov <[email protected]> wrote:
Quoted 33 lines · Igor Mammedov
Quoted 30 lines · On Thu, 7 Jan 2021 16:14:49 +0100 Vitaly Kuznetsov <[email protected]> wrote:
Enabling Hyper-V emulation for a Windows VM is a tiring experience as it requires listing all currently supported enlightenments ("hv-*" CPU features) explicitly. We do have 'hv-passthrough' mode enabling everything but it can't be used in production as it prevents migration.
Introduce a simple 'hv-default=on' CPU flag enabling all currently supported Hyper-V enlightenments. Later, when new enlightenments get implemented,
compat_propsmechanism will be used to disable them for legacy machine types, this will keep 'hv-default=on' configurations migratable.Signed-off-by Vitaly Kuznetsov <[email protected]>--- docs/hyperv.txt | 16 +++++++++++++--- target/i386/cpu.c | 38 ++++++++++++++++++++++++++++++++++++++ target/i386/cpu.h | 5 +++++ 3 files changed, 56 insertions(+), 3 deletions(-)docs/hyperv.txt +2
-2@@ -17,10 +17,20 @@ compatible hypervisor and use Hyper-V specific features. 2. Setup ========= -No Hyper-V enlightenments are enabled by default by either KVM or QEMU. In -QEMU, individual enlightenments can be enabled through CPU flags, e.g: +All currently supported Hyper-V enlightenments can be enabled by specifying +'hv-default=on' CPU flag:Quoted 11 lines
Quoted 11 lines
docs/hyperv.txt compatible hypervisor and use Hyper-V specific features. · line 28
Quoted 7 lines
docs/hyperv.txt +6
-1- qemu-system-x86_64 --enable-kvm --cpu host,hv_relaxed,hv_vpindex,hv_time, ... + qemu-system-x86_64 --enable-kvm --cpu host,hv-default ... + +Alternatively, it is possible to do fine-grained enablement through CPU flags, +e.g: + + qemu-system-x86_64 --enable-kvm --cpu host,hv-relaxed,hv-vpindex,hv-time ...I'd put here not '...' but rather recommended list of flags, and update it every time when new feature added if necessary.
1)
Quoted 7 lines
Quoted 3 lines
This is an example of fine-grained enablement, there is no point to put all the existing flags there (hv-default is the only recommended way now, the rest is 'expert'/'debugging').
so users are kept in dark what hv-default disables/enables (and it might depend on machine version on top that). Doesn't look like a good documentation to me (sure everyone can go and read source code for it and try to figure out how it's supposed to work)
'hv-default' enables *all* currently supported enlightenments. When using with an old machine type, it will enable *all* Hyper-V enlightenmnets which were supported when the corresponding machine type was released. I don't think we document all other cases when a machine type is modified (i.e. where can I read how pc-q35-5.1 is different from pc-q35-5.0 if I refuse to read the source code?)
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
(not to mention that if we had it to begin with, then new 'hv-default' won't be necessary, I still see it as functionality duplication but I will not oppose it)
Quoted 8 lines
Quoted 2 lines
Unfortunately, upper layer tools don't read this doc and update themselves to enable new features when they appear.
rant: (just merge all libvirt into QEMU, and make VM configuration less low-level. why stop there, just merge with yet another upper layer, it would save us a lot on communication protocols and simplify VM creation even more,
and no one will have to read docs and write anything new on top.) There should be limit somewhere, where QEMU job ends and others pile hw abstraction layers on top of it.We have '-machine q35' and we don't require to list all the devices from it. We have '-cpu Skylake-Server' and we don't require to configure all the features manually. Why can't we have similar enablement for Hyper-V emulation where we can't even see a real need for anything but 'enable everything' option?
There is no 'one libvirt to rule them all' (fortunately or unfortunately). And sometimes QEMU is the uppermost layer and there's no 'libvirt' on top of it, this is also a perfectly valid use-case.
Quoted 3 lines
Quoted 3 lines
Similarly, if when these tools use '-machine q35' they get all the new features we add automatically, right?
Quoted 8 lines
Quoted 5 lines
it depends, in case of CPUs, new features usually 'off' by default for existing models. In case of bugs, features sometimes could be flipped and versioned machines were used to keep broken CPU models on old machine types.
That's why I was saying that Hyper-V enlightenments hardly resemble 'hardware' CPU features.
Well, Microsoft chose to implement them as hardware concept (CPUID leaf), and I prefer to treat them the same way as any other CPUID bits.
Quoted 68 lines
Quoted 43 lines
Quoted 37 lines
Quoted 27 lines
Quoted 27 lines
docs/hyperv.txt +4
-0+It is also possible to disable individual enlightenments from the default list, +this can be used for debugging purposes: + + qemu-system-x86_64 --enable-kvm --cpu host,hv-default=on,hv-evmcs=off ... Sometimes there are dependencies between enlightenments, QEMU is supposed to check that the supplied configuration is sane.target/i386/cpu.c +10
-0@@ -4552,6 +4552,24 @@ static void x86_cpuid_set_tsc_freq(Object *obj, Visitor *v, const char *name, cpu->env.tsc_khz = cpu->env.user_tsc_khz = value /1000 ; } +static bool x86_hv_default_get (Object *obj,Error **errp) +{ +X86CPU *cpu =X86_CPU (obj); + +return cpu->hyperv_default; +} + +static void x86_hv_default_set (Object *obj,bool value,Error **errp) +{ +X86CPU *cpu =X86_CPU (obj);Quoted 9 lines
target/i386/cpu.c static void x86_cpuid_set_tsc_freq(Object *obj, Visitor *v, const char *name, · line 4569
Quoted 5 lines
target/i386/cpu.c +5
-0+ + cpu->hyperv_default = value; + +if (value) { + cpu->hyperv_features |= cpu->hyperv_default_features;s/|="/=/ please,
i.e. no option overrides whatever was specified before to keep semantics consistent.Quoted 6 lines
Hm,
this doesn't matter for the most recent machine type as
hyperv_default_featureshas all the features but imagine you're running an older machine type which doesn't have 'hv_feature'. Now yourQuoted 7 lines
normally one shouldn't use new feature with old machine type as it makes VM non-migratable to older QEMU that has this machine type but not this feature.
nitpicking:
according to (1) user should not use 'hv_feature' on old machine sincehv_defaultshould cover all their needs (well they don't know whathv_defaultactually is).Normally yes but I can imagine sticking to some old machine type for other-than-hyperv-enlightenments purposes and still wanting to add a newly introduced enlightenment. Migration is not always a must.
Quoted 13 lines
Quoted 13 lines
suggestion is
if I do:
'hv_default,hv_feature=on' I will get "
hyperv_default_features|hv_feature"but if I do
'hv_feature=on,hv_default' I will just get '
hyperv_default_features' (ashv_defaultenablement will overwrite everything)How is this consistent?
Quoted 8 lines
Quoted 5 lines
usual semantics for properties, is that the latest property overwrites, the previous property value parsed from left to right.
(i.e. if one asked forhv_default, one gets it related CPUID bit set/unset, if one needs more than that one should add more related features after that.This semantics probably doesn't apply to 'hv-default' case IMO as my brain refuses to accept the fact that
it's difficult probably because 'hv-default' is 'alias' property that covers all individual hv-foo features in one go and that individual features are exposed to user, but otherwise it is just a property that sets CPUID features or like any other property, and should be treated like such.
Quoted 4 lines
'hv_default,hv_feature' != 'hv_feature,hv_default'
which should express the same desire 'the default set PLUS the feature I want'.
Quoted 7 lines
if
hv_defaultwere touching different data, I'd agree.
But in the endhv_defaultboils down to the same CPUID bits as individual features:hv_default,hv_f2 => (hv_f1=on,hv_f2=off),hv_f2=on
!=hv_f2,hv_default => hv_f2=on,(hv_f1=on,hv_f2=off)
I don't know why you chose to define "hv_default" as hv_f1=on,hv_f2=off. If hv_f2 is not enabled by hv_default, it doesn't need to be touched by hv_default at all.
Quoted 12 lines
Quoted 3 lines
I think I prefer sanity over purity in this case.
what is sanity to one could be insanity for another, so I pointed out the way properties expected to work today.
But you are adding new semantic ('combine') to property/features parsing (instead of current 'set' policy), and users will have to be aware of this new behavior and add/maintain code for this special case. (maybe I worry in vain, and no one will read docs and know about this new property anyways)
Quoted 7 lines
That will also push x86 CPUs consolidation farther away from other targets, where there aren't any special casing for features parsing, just simple left to right parsing with the latest property having overwriting previously set value.
We are trying hard to reduce special cases and unify interfaces for same components to simplify qemu and make it predictable/easier for users.
What you are proposing diverges from other targets, actually. See target/s390x/cpu_models.c:set_feature_group() for example. Enabling a feature group in s390x only enables a set of feature bits, and doesn't touch the rest.
In other words, if hv_default includes hv_f1+hv_f2 (and not hv_f3 or hv_f4), this means:
hv_default,hv_f3=on,hv_f4=off => (hv_f1=on,hv_f2=on),hv_f3=on,hv_f4=off
==
hv_f3=on,hv_f4=off,hv_default => hv_f3=on,hv_f4=off,(hv_f2=on,hv_f2=on)
That would also mean:
hv_default,hv_f1=on,hv_f2=off => (hv_f1=on,hv_f2=on),hv_f1=on,hv_f2=off
!=
hv_f1=on,hv_f2=off,hv_default => hv_f1=on,hv_f2=off,(hv_f2=on,hv_f2=on)
That's the behavior implemented by Vitaly.
Quoted 1 line
[...]
Signature · 1 line
Eduardo
Re: [PATCH v3 18/19] i386: provide simple 'hv-default=on' option
Quoted 201 lines · On Wed, 20 Jan 2021 15:49:09 -0500 Eduardo Habkost <[email protected]> wrote:
Quoted 200 lines · Igor Mammedov
Quoted 97 lines · On Wed, 20 Jan 2021 15:38:33 +0100 Vitaly Kuznetsov <[email protected]> wrote:
Quoted 52 lines · Igor Mammedov
Quoted 35 lines · On Fri, 15 Jan 2021 10:20:23 +0100 Vitaly Kuznetsov <[email protected]> wrote:
Quoted 33 lines · Igor Mammedov
Quoted 30 lines · On Thu, 7 Jan 2021 16:14:49 +0100 Vitaly Kuznetsov <[email protected]> wrote:
Enabling Hyper-V emulation for a Windows VM is a tiring experience as it requires listing all currently supported enlightenments ("hv-*" CPU features) explicitly. We do have 'hv-passthrough' mode enabling everything but it can't be used in production as it prevents migration.
Introduce a simple 'hv-default=on' CPU flag enabling all currently supported Hyper-V enlightenments. Later, when new enlightenments get implemented,
compat_propsmechanism will be used to disable them for legacy machine types, this will keep 'hv-default=on' configurations migratable.Signed-off-by Vitaly Kuznetsov <[email protected]>--- docs/hyperv.txt | 16 +++++++++++++--- target/i386/cpu.c | 38 ++++++++++++++++++++++++++++++++++++++ target/i386/cpu.h | 5 +++++ 3 files changed, 56 insertions(+), 3 deletions(-)docs/hyperv.txt +2
-2@@ -17,10 +17,20 @@ compatible hypervisor and use Hyper-V specific features. 2. Setup ========= -No Hyper-V enlightenments are enabled by default by either KVM or QEMU. In -QEMU, individual enlightenments can be enabled through CPU flags, e.g: +All currently supported Hyper-V enlightenments can be enabled by specifying +'hv-default=on' CPU flag:Quoted 11 lines
Quoted 11 lines
docs/hyperv.txt compatible hypervisor and use Hyper-V specific features. · line 28
Quoted 7 lines
docs/hyperv.txt +6
-1- qemu-system-x86_64 --enable-kvm --cpu host,hv_relaxed,hv_vpindex,hv_time, ... + qemu-system-x86_64 --enable-kvm --cpu host,hv-default ... + +Alternatively, it is possible to do fine-grained enablement through CPU flags, +e.g: + + qemu-system-x86_64 --enable-kvm --cpu host,hv-relaxed,hv-vpindex,hv-time ...I'd put here not '...' but rather recommended list of flags, and update it every time when new feature added if necessary.
1)
Quoted 7 lines
Quoted 3 lines
This is an example of fine-grained enablement, there is no point to put all the existing flags there (hv-default is the only recommended way now, the rest is 'expert'/'debugging').
so users are kept in dark what hv-default disables/enables (and it might depend on machine version on top that). Doesn't look like a good documentation to me (sure everyone can go and read source code for it and try to figure out how it's supposed to work)
'hv-default' enables *all* currently supported enlightenments. When using with an old machine type, it will enable *all* Hyper-V enlightenmnets which were supported when the corresponding machine type was released. I don't think we document all other cases when a machine type is modified (i.e. where can I read how pc-q35-5.1 is different from pc-q35-5.0 if I refuse to read the source code?)
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
(not to mention that if we had it to begin with, then new 'hv-default' won't be necessary, I still see it as functionality duplication but I will not oppose it)
Quoted 8 lines
Quoted 2 lines
Unfortunately, upper layer tools don't read this doc and update themselves to enable new features when they appear.
rant: (just merge all libvirt into QEMU, and make VM configuration less low-level. why stop there, just merge with yet another upper layer, it would save us a lot on communication protocols and simplify VM creation even more,
and no one will have to read docs and write anything new on top.) There should be limit somewhere, where QEMU job ends and others pile hw abstraction layers on top of it.We have '-machine q35' and we don't require to list all the devices from it. We have '-cpu Skylake-Server' and we don't require to configure all the features manually. Why can't we have similar enablement for Hyper-V emulation where we can't even see a real need for anything but 'enable everything' option?
There is no 'one libvirt to rule them all' (fortunately or unfortunately). And sometimes QEMU is the uppermost layer and there's no 'libvirt' on top of it, this is also a perfectly valid use-case.
Quoted 3 lines
Quoted 3 lines
Similarly, if when these tools use '-machine q35' they get all the new features we add automatically, right?
Quoted 8 lines
Quoted 5 lines
it depends, in case of CPUs, new features usually 'off' by default for existing models. In case of bugs, features sometimes could be flipped and versioned machines were used to keep broken CPU models on old machine types.
That's why I was saying that Hyper-V enlightenments hardly resemble 'hardware' CPU features.
Well, Microsoft chose to implement them as hardware concept (CPUID leaf), and I prefer to treat them the same way as any other CPUID bits.
Quoted 68 lines
Quoted 43 lines
Quoted 37 lines
Quoted 27 lines
Quoted 27 lines
docs/hyperv.txt +4
-0+It is also possible to disable individual enlightenments from the default list, +this can be used for debugging purposes: + + qemu-system-x86_64 --enable-kvm --cpu host,hv-default=on,hv-evmcs=off ... Sometimes there are dependencies between enlightenments, QEMU is supposed to check that the supplied configuration is sane.target/i386/cpu.c +10
-0@@ -4552,6 +4552,24 @@ static void x86_cpuid_set_tsc_freq(Object *obj, Visitor *v, const char *name, cpu->env.tsc_khz = cpu->env.user_tsc_khz = value /1000 ; } +static bool x86_hv_default_get (Object *obj,Error **errp) +{ +X86CPU *cpu =X86_CPU (obj); + +return cpu->hyperv_default; +} + +static void x86_hv_default_set (Object *obj,bool value,Error **errp) +{ +X86CPU *cpu =X86_CPU (obj);Quoted 9 lines
target/i386/cpu.c static void x86_cpuid_set_tsc_freq(Object *obj, Visitor *v, const char *name, · line 4569
Quoted 5 lines
target/i386/cpu.c +5
-0+ + cpu->hyperv_default = value; + +if (value) { + cpu->hyperv_features |= cpu->hyperv_default_features;s/|="/=/ please,
i.e. no option overrides whatever was specified before to keep semantics consistent.Quoted 6 lines
Hm,
this doesn't matter for the most recent machine type as
hyperv_default_featureshas all the features but imagine you're running an older machine type which doesn't have 'hv_feature'. Now yourQuoted 7 lines
normally one shouldn't use new feature with old machine type as it makes VM non-migratable to older QEMU that has this machine type but not this feature.
nitpicking:
according to (1) user should not use 'hv_feature' on old machine sincehv_defaultshould cover all their needs (well they don't know whathv_defaultactually is).Normally yes but I can imagine sticking to some old machine type for other-than-hyperv-enlightenments purposes and still wanting to add a newly introduced enlightenment. Migration is not always a must.
Quoted 13 lines
Quoted 13 lines
suggestion is
if I do:
'hv_default,hv_feature=on' I will get "
hyperv_default_features|hv_feature"but if I do
'hv_feature=on,hv_default' I will just get '
hyperv_default_features' (ashv_defaultenablement will overwrite everything)How is this consistent?
Quoted 8 lines
Quoted 5 lines
usual semantics for properties, is that the latest property overwrites, the previous property value parsed from left to right.
(i.e. if one asked forhv_default, one gets it related CPUID bit set/unset, if one needs more than that one should add more related features after that.This semantics probably doesn't apply to 'hv-default' case IMO as my brain refuses to accept the fact that
it's difficult probably because 'hv-default' is 'alias' property that covers all individual hv-foo features in one go and that individual features are exposed to user, but otherwise it is just a property that sets CPUID features or like any other property, and should be treated like such.
Quoted 4 lines
'hv_default,hv_feature' != 'hv_feature,hv_default'
which should express the same desire 'the default set PLUS the feature I want'.
if
hv_defaultwere touching different data, I'd agree.
But in the endhv_defaultboils down to the same CPUID bits as individual features:
Quoted 7 lines
Quoted 3 lines
hv_default,hv_f2 => (hv_f1=on,hv_f2=off),hv_f2=on
!=hv_f2,hv_default => hv_f2=on,(hv_f1=on,hv_f2=off)
I don't know why you chose to define "
hv_default" as hv_f1=on,hv_f2=off. Ifhv_f2is not enabled byhv_default, it doesn't need to be touched byhv_defaultat all.
Essentially I was thinking about hv_default=on as setting default value of hv CPUID leaf i.e. like doc claims, 'all' hv_* features (including turned off and unused bits) which always sets leaf to its default state.
Now lets consider following possible situation
using combine' approach (leaf |= some_bits):
QEMU-6.0: initially we have all possible features enabled
hv_default = (hv_f1=on,hv_f2=on)
hv_f2=on,hv_default=on == hv_f1=on,hv_f2=on
QEMU-6.1: disabled hv_f2=off that was causing problems
hv_default = (hv_f1=on,hv_f2=off)
however due to ORing hv_default doesn't fix issue for the same CLI (i.e. it doesn't have expected effect)
hv_f2=on,hv_default=on => hv_f1=on,hv_f2=on
if one would use usual 'set' semantics (leaf = all_bits),
then new hv_default value will have desired effect despite of botched CLI, just by virtue of property following typical 'last set' semantics:
=> hv_f1=on,hv_f2=off
If we assume that we 'never ever' will need to disable feature bits than it doesn't matter which approach to use, however a look at pc_compat arrays shows that features are being enabled/disabled all the time.
PS:
I'd rename hv_default => hv_set_default,
since we would need hv_default[_value] property later on to set compat value based on machine type version.
Quoted 18 lines
Quoted 10 lines
Quoted 1 line
I think I prefer sanity over purity in this case.
what is sanity to one could be insanity for another, so I pointed out the way properties expected to work today.
But you are adding new semantic ('combine') to property/features parsing (instead of current 'set' policy), and users will have to be aware of this new behavior and add/maintain code for this special case. (maybe I worry in vain, and no one will read docs and know about this new property anyways)
Quoted 7 lines
That will also push x86 CPUs consolidation farther away from other targets, where there aren't any special casing for features parsing, just simple left to right parsing with the latest property having overwriting previously set value.
We are trying hard to reduce special cases and unify interfaces for same components to simplify qemu and make it predictable/easier for users.
Quoted 4 lines
What you are proposing diverges from other targets, actually. See target/s390x/cpu_models.c:set_feature_group() for example. Enabling a feature group in s390x only enables a set of feature bits, and doesn't touch the rest.
Looking at code, it has the same issue as I described above
Quoted 17 lines
In other words, if
hv_defaultincludes hv_f1+hv_f2 (and nothv_f3orhv_f4), this means:hv_default,hv_f3=on,hv_f4=off => (hv_f1=on,hv_f2=on),hv_f3=on,hv_f4=off == hv_f3=on,hv_f4=off,hv_default => hv_f3=on,hv_f4=off,(hv_f2=on,hv_f2=on)That would also mean:
hv_default,hv_f1=on,hv_f2=off => (hv_f1=on,hv_f2=on),hv_f1=on,hv_f2=off != hv_f1=on,hv_f2=off,hv_default => hv_f1=on,hv_f2=off,(hv_f2=on,hv_f2=on)That's the behavior implemented by Vitaly.
Quoted 1 line
[...]
Re: [PATCH v3 18/19] i386: provide simple 'hv-default=on' option
Quoted 237 lines · On Thu, 21 Jan 2021 14:27:04 +0100 Igor Mammedov <[email protected]> wrote:
Quoted 201 lines · On Wed, 20 Jan 2021 15:49:09 -0500 Eduardo Habkost <[email protected]> wrote:
Quoted 200 lines · Igor Mammedov
Quoted 97 lines · On Wed, 20 Jan 2021 15:38:33 +0100 Vitaly Kuznetsov <[email protected]> wrote:
Quoted 52 lines · Igor Mammedov
Quoted 35 lines · On Fri, 15 Jan 2021 10:20:23 +0100 Vitaly Kuznetsov <[email protected]> wrote:
Quoted 33 lines · Igor Mammedov
Quoted 30 lines · On Thu, 7 Jan 2021 16:14:49 +0100 Vitaly Kuznetsov <[email protected]> wrote:
Enabling Hyper-V emulation for a Windows VM is a tiring experience as it requires listing all currently supported enlightenments ("hv-*" CPU features) explicitly. We do have 'hv-passthrough' mode enabling everything but it can't be used in production as it prevents migration.
Introduce a simple 'hv-default=on' CPU flag enabling all currently supported Hyper-V enlightenments. Later, when new enlightenments get implemented,
compat_propsmechanism will be used to disable them for legacy machine types, this will keep 'hv-default=on' configurations migratable.Signed-off-by Vitaly Kuznetsov <[email protected]>--- docs/hyperv.txt | 16 +++++++++++++--- target/i386/cpu.c | 38 ++++++++++++++++++++++++++++++++++++++ target/i386/cpu.h | 5 +++++ 3 files changed, 56 insertions(+), 3 deletions(-)docs/hyperv.txt +2
-2@@ -17,10 +17,20 @@ compatible hypervisor and use Hyper-V specific features. 2. Setup ========= -No Hyper-V enlightenments are enabled by default by either KVM or QEMU. In -QEMU, individual enlightenments can be enabled through CPU flags, e.g: +All currently supported Hyper-V enlightenments can be enabled by specifying +'hv-default=on' CPU flag:Quoted 11 lines
Quoted 11 lines
docs/hyperv.txt compatible hypervisor and use Hyper-V specific features. · line 28
Quoted 7 lines
docs/hyperv.txt +6
-1- qemu-system-x86_64 --enable-kvm --cpu host,hv_relaxed,hv_vpindex,hv_time, ... + qemu-system-x86_64 --enable-kvm --cpu host,hv-default ... + +Alternatively, it is possible to do fine-grained enablement through CPU flags, +e.g: + + qemu-system-x86_64 --enable-kvm --cpu host,hv-relaxed,hv-vpindex,hv-time ...I'd put here not '...' but rather recommended list of flags, and update it every time when new feature added if necessary.
1)
Quoted 7 lines
Quoted 3 lines
This is an example of fine-grained enablement, there is no point to put all the existing flags there (hv-default is the only recommended way now, the rest is 'expert'/'debugging').
so users are kept in dark what hv-default disables/enables (and it might depend on machine version on top that). Doesn't look like a good documentation to me (sure everyone can go and read source code for it and try to figure out how it's supposed to work)
'hv-default' enables *all* currently supported enlightenments. When using with an old machine type, it will enable *all* Hyper-V enlightenmnets which were supported when the corresponding machine type was released. I don't think we document all other cases when a machine type is modified (i.e. where can I read how pc-q35-5.1 is different from pc-q35-5.0 if I refuse to read the source code?)
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
(not to mention that if we had it to begin with, then new 'hv-default' won't be necessary, I still see it as functionality duplication but I will not oppose it)
Quoted 8 lines
Quoted 2 lines
Unfortunately, upper layer tools don't read this doc and update themselves to enable new features when they appear.
rant: (just merge all libvirt into QEMU, and make VM configuration less low-level. why stop there, just merge with yet another upper layer, it would save us a lot on communication protocols and simplify VM creation even more,
and no one will have to read docs and write anything new on top.) There should be limit somewhere, where QEMU job ends and others pile hw abstraction layers on top of it.We have '-machine q35' and we don't require to list all the devices from it. We have '-cpu Skylake-Server' and we don't require to configure all the features manually. Why can't we have similar enablement for Hyper-V emulation where we can't even see a real need for anything but 'enable everything' option?
There is no 'one libvirt to rule them all' (fortunately or unfortunately). And sometimes QEMU is the uppermost layer and there's no 'libvirt' on top of it, this is also a perfectly valid use-case.
Quoted 3 lines
Quoted 3 lines
Similarly, if when these tools use '-machine q35' they get all the new features we add automatically, right?
Quoted 8 lines
Quoted 5 lines
it depends, in case of CPUs, new features usually 'off' by default for existing models. In case of bugs, features sometimes could be flipped and versioned machines were used to keep broken CPU models on old machine types.
That's why I was saying that Hyper-V enlightenments hardly resemble 'hardware' CPU features.
Well, Microsoft chose to implement them as hardware concept (CPUID leaf), and I prefer to treat them the same way as any other CPUID bits.
Quoted 68 lines
Quoted 43 lines
Quoted 37 lines
Quoted 27 lines
Quoted 27 lines
docs/hyperv.txt +4
-0+It is also possible to disable individual enlightenments from the default list, +this can be used for debugging purposes: + + qemu-system-x86_64 --enable-kvm --cpu host,hv-default=on,hv-evmcs=off ... Sometimes there are dependencies between enlightenments, QEMU is supposed to check that the supplied configuration is sane.target/i386/cpu.c +10
-0@@ -4552,6 +4552,24 @@ static void x86_cpuid_set_tsc_freq(Object *obj, Visitor *v, const char *name, cpu->env.tsc_khz = cpu->env.user_tsc_khz = value /1000 ; } +static bool x86_hv_default_get (Object *obj,Error **errp) +{ +X86CPU *cpu =X86_CPU (obj); + +return cpu->hyperv_default; +} + +static void x86_hv_default_set (Object *obj,bool value,Error **errp) +{ +X86CPU *cpu =X86_CPU (obj);Quoted 9 lines
target/i386/cpu.c static void x86_cpuid_set_tsc_freq(Object *obj, Visitor *v, const char *name, · line 4569
Quoted 5 lines
target/i386/cpu.c +5
-0+ + cpu->hyperv_default = value; + +if (value) { + cpu->hyperv_features |= cpu->hyperv_default_features;s/|="/=/ please,
i.e. no option overrides whatever was specified before to keep semantics consistent.Quoted 6 lines
Hm,
this doesn't matter for the most recent machine type as
hyperv_default_featureshas all the features but imagine you're running an older machine type which doesn't have 'hv_feature'. Now yourQuoted 7 lines
normally one shouldn't use new feature with old machine type as it makes VM non-migratable to older QEMU that has this machine type but not this feature.
nitpicking:
according to (1) user should not use 'hv_feature' on old machine sincehv_defaultshould cover all their needs (well they don't know whathv_defaultactually is).Normally yes but I can imagine sticking to some old machine type for other-than-hyperv-enlightenments purposes and still wanting to add a newly introduced enlightenment. Migration is not always a must.
Quoted 13 lines
Quoted 13 lines
suggestion is
if I do:
'hv_default,hv_feature=on' I will get "
hyperv_default_features|hv_feature"but if I do
'hv_feature=on,hv_default' I will just get '
hyperv_default_features' (ashv_defaultenablement will overwrite everything)How is this consistent?
Quoted 8 lines
Quoted 5 lines
usual semantics for properties, is that the latest property overwrites, the previous property value parsed from left to right.
(i.e. if one asked forhv_default, one gets it related CPUID bit set/unset, if one needs more than that one should add more related features after that.This semantics probably doesn't apply to 'hv-default' case IMO as my brain refuses to accept the fact that
it's difficult probably because 'hv-default' is 'alias' property that covers all individual hv-foo features in one go and that individual features are exposed to user, but otherwise it is just a property that sets CPUID features or like any other property, and should be treated like such.
Quoted 4 lines
'hv_default,hv_feature' != 'hv_feature,hv_default'
which should express the same desire 'the default set PLUS the feature I want'.
if
hv_defaultwere touching different data, I'd agree.
But in the endhv_defaultboils down to the same CPUID bits as individual features:Quoted 7 lines
Quoted 3 lines
hv_default,hv_f2 => (hv_f1=on,hv_f2=off),hv_f2=on
!=hv_f2,hv_default => hv_f2=on,(hv_f1=on,hv_f2=off)
I don't know why you chose to define "
hv_default" as hv_f1=on,hv_f2=off. Ifhv_f2is not enabled byhv_default, it doesn't need to be touched byhv_defaultat all.Essentially I was thinking about hv_default=on as setting default value of hv CPUID leaf i.e. like doc claims, 'all' hv_* features (including turned off and unused bits) which always sets leaf to its default state.
Now lets consider following possible situation
using combine' approach (leaf |=some_bits):QEMU-6.0: initially we have all possible features enabled
hv_default = (hv_f1=on,hv_f2=on)hv_f2=on,hv_default=on == hv_f1=on,hv_f2=on
QEMU-6.1: disabled hv_f2=off that was causing problems
hv_default= (hv_f1=on,hv_f2=off)however due to ORing
hv_defaultdoesn't fix issue for the same CLI (i.e. it doesn't have expected effect)hv_f2=on,hv_default=on => hv_f1=on,hv_f2=on
if one would use usual 'set' semantics (leaf =
all_bits),
then newhv_defaultvalue will have desired effect despite of botched CLI, just by virtue of property following typical 'last set' semantics:
Quoted 6 lines
=> hv_f1=on,hv_f2=off
If we assume that we 'never ever' will need to disable feature bits than it doesn't matter which approach to use, however a look at
pc_compatarrays shows that features are being enabled/disabled all the time.
Also there should be a good reason for adding new semantics and deviating from typical property behavior.
Re: [PATCH v3 18/19] i386: provide simple 'hv-default=on' option
In reply to Igor MammedovQuoted 219 lines · Igor Mammedov
Quoted 201 lines · On Wed, 20 Jan 2021 15:49:09 -0500 Eduardo Habkost <[email protected]> wrote:
Quoted 200 lines · Igor Mammedov
Quoted 97 lines · On Wed, 20 Jan 2021 15:38:33 +0100 Vitaly Kuznetsov <[email protected]> wrote:
Quoted 52 lines · Igor Mammedov
Quoted 35 lines · On Fri, 15 Jan 2021 10:20:23 +0100 Vitaly Kuznetsov <[email protected]> wrote:
Quoted 33 lines · Igor Mammedov
Quoted 30 lines · On Thu, 7 Jan 2021 16:14:49 +0100 Vitaly Kuznetsov <[email protected]> wrote:
Enabling Hyper-V emulation for a Windows VM is a tiring experience as it requires listing all currently supported enlightenments ("hv-*" CPU features) explicitly. We do have 'hv-passthrough' mode enabling everything but it can't be used in production as it prevents migration.
Introduce a simple 'hv-default=on' CPU flag enabling all currently supported Hyper-V enlightenments. Later, when new enlightenments get implemented,
compat_propsmechanism will be used to disable them for legacy machine types, this will keep 'hv-default=on' configurations migratable.Signed-off-by Vitaly Kuznetsov <[email protected]>--- docs/hyperv.txt | 16 +++++++++++++--- target/i386/cpu.c | 38 ++++++++++++++++++++++++++++++++++++++ target/i386/cpu.h | 5 +++++ 3 files changed, 56 insertions(+), 3 deletions(-)docs/hyperv.txt +2
-2@@ -17,10 +17,20 @@ compatible hypervisor and use Hyper-V specific features. 2. Setup ========= -No Hyper-V enlightenments are enabled by default by either KVM or QEMU. In -QEMU, individual enlightenments can be enabled through CPU flags, e.g: +All currently supported Hyper-V enlightenments can be enabled by specifying +'hv-default=on' CPU flag:Quoted 11 lines
Quoted 11 lines
docs/hyperv.txt compatible hypervisor and use Hyper-V specific features. · line 28
Quoted 7 lines
docs/hyperv.txt +6
-1- qemu-system-x86_64 --enable-kvm --cpu host,hv_relaxed,hv_vpindex,hv_time, ... + qemu-system-x86_64 --enable-kvm --cpu host,hv-default ... + +Alternatively, it is possible to do fine-grained enablement through CPU flags, +e.g: + + qemu-system-x86_64 --enable-kvm --cpu host,hv-relaxed,hv-vpindex,hv-time ...I'd put here not '...' but rather recommended list of flags, and update it every time when new feature added if necessary.
1)
Quoted 7 lines
Quoted 3 lines
This is an example of fine-grained enablement, there is no point to put all the existing flags there (hv-default is the only recommended way now, the rest is 'expert'/'debugging').
so users are kept in dark what hv-default disables/enables (and it might depend on machine version on top that). Doesn't look like a good documentation to me (sure everyone can go and read source code for it and try to figure out how it's supposed to work)
'hv-default' enables *all* currently supported enlightenments. When using with an old machine type, it will enable *all* Hyper-V enlightenmnets which were supported when the corresponding machine type was released. I don't think we document all other cases when a machine type is modified (i.e. where can I read how pc-q35-5.1 is different from pc-q35-5.0 if I refuse to read the source code?)
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
(not to mention that if we had it to begin with, then new 'hv-default' won't be necessary, I still see it as functionality duplication but I will not oppose it)
Quoted 8 lines
Quoted 2 lines
Unfortunately, upper layer tools don't read this doc and update themselves to enable new features when they appear.
rant: (just merge all libvirt into QEMU, and make VM configuration less low-level. why stop there, just merge with yet another upper layer, it would save us a lot on communication protocols and simplify VM creation even more,
and no one will have to read docs and write anything new on top.) There should be limit somewhere, where QEMU job ends and others pile hw abstraction layers on top of it.We have '-machine q35' and we don't require to list all the devices from it. We have '-cpu Skylake-Server' and we don't require to configure all the features manually. Why can't we have similar enablement for Hyper-V emulation where we can't even see a real need for anything but 'enable everything' option?
There is no 'one libvirt to rule them all' (fortunately or unfortunately). And sometimes QEMU is the uppermost layer and there's no 'libvirt' on top of it, this is also a perfectly valid use-case.
Quoted 3 lines
Quoted 3 lines
Similarly, if when these tools use '-machine q35' they get all the new features we add automatically, right?
Quoted 8 lines
Quoted 5 lines
it depends, in case of CPUs, new features usually 'off' by default for existing models. In case of bugs, features sometimes could be flipped and versioned machines were used to keep broken CPU models on old machine types.
That's why I was saying that Hyper-V enlightenments hardly resemble 'hardware' CPU features.
Well, Microsoft chose to implement them as hardware concept (CPUID leaf), and I prefer to treat them the same way as any other CPUID bits.
Quoted 68 lines
Quoted 43 lines
Quoted 37 lines
Quoted 27 lines
Quoted 27 lines
docs/hyperv.txt +4
-0+It is also possible to disable individual enlightenments from the default list, +this can be used for debugging purposes: + + qemu-system-x86_64 --enable-kvm --cpu host,hv-default=on,hv-evmcs=off ... Sometimes there are dependencies between enlightenments, QEMU is supposed to check that the supplied configuration is sane.target/i386/cpu.c +10
-0@@ -4552,6 +4552,24 @@ static void x86_cpuid_set_tsc_freq(Object *obj, Visitor *v, const char *name, cpu->env.tsc_khz = cpu->env.user_tsc_khz = value /1000 ; } +static bool x86_hv_default_get (Object *obj,Error **errp) +{ +X86CPU *cpu =X86_CPU (obj); + +return cpu->hyperv_default; +} + +static void x86_hv_default_set (Object *obj,bool value,Error **errp) +{ +X86CPU *cpu =X86_CPU (obj);Quoted 9 lines
target/i386/cpu.c static void x86_cpuid_set_tsc_freq(Object *obj, Visitor *v, const char *name, · line 4569
Quoted 5 lines
target/i386/cpu.c +5
-0+ + cpu->hyperv_default = value; + +if (value) { + cpu->hyperv_features |= cpu->hyperv_default_features;s/|="/=/ please,
i.e. no option overrides whatever was specified before to keep semantics consistent.Quoted 6 lines
Hm,
this doesn't matter for the most recent machine type as
hyperv_default_featureshas all the features but imagine you're running an older machine type which doesn't have 'hv_feature'. Now yourQuoted 7 lines
normally one shouldn't use new feature with old machine type as it makes VM non-migratable to older QEMU that has this machine type but not this feature.
nitpicking:
according to (1) user should not use 'hv_feature' on old machine sincehv_defaultshould cover all their needs (well they don't know whathv_defaultactually is).Normally yes but I can imagine sticking to some old machine type for other-than-hyperv-enlightenments purposes and still wanting to add a newly introduced enlightenment. Migration is not always a must.
Quoted 13 lines
Quoted 13 lines
suggestion is
if I do:
'hv_default,hv_feature=on' I will get "
hyperv_default_features|hv_feature"but if I do
'hv_feature=on,hv_default' I will just get '
hyperv_default_features' (ashv_defaultenablement will overwrite everything)How is this consistent?
Quoted 8 lines
Quoted 5 lines
usual semantics for properties, is that the latest property overwrites, the previous property value parsed from left to right.
(i.e. if one asked forhv_default, one gets it related CPUID bit set/unset, if one needs more than that one should add more related features after that.This semantics probably doesn't apply to 'hv-default' case IMO as my brain refuses to accept the fact that
it's difficult probably because 'hv-default' is 'alias' property that covers all individual hv-foo features in one go and that individual features are exposed to user, but otherwise it is just a property that sets CPUID features or like any other property, and should be treated like such.
Quoted 4 lines
'hv_default,hv_feature' != 'hv_feature,hv_default'
which should express the same desire 'the default set PLUS the feature I want'.
if
hv_defaultwere touching different data, I'd agree.
But in the endhv_defaultboils down to the same CPUID bits as individual features:Quoted 7 lines
Quoted 3 lines
hv_default,hv_f2 => (hv_f1=on,hv_f2=off),hv_f2=on
!=hv_f2,hv_default => hv_f2=on,(hv_f1=on,hv_f2=off)
I don't know why you chose to define "
hv_default" as hv_f1=on,hv_f2=off. Ifhv_f2is not enabled byhv_default, it doesn't need to be touched byhv_defaultat all.Essentially I was thinking about hv_default=on as setting default value of hv CPUID leaf i.e. like doc claims, 'all' hv_* features (including turned off and unused bits) which always sets leaf to its default state.
Now lets consider following possible situation
using combine' approach (leaf |=some_bits):
Quoted 8 lines
QEMU-6.0: initially we have all possible features enabled
hv_default = (hv_f1=on,hv_f2=on)hv_f2=on,hv_default=on == hv_f1=on,hv_f2=on
QEMU-6.1: disabled hv_f2=off that was causing problems
hv_default= (hv_f1=on,hv_f2=off)
Why would we choose to do that?
If we decide f2 shouldn't be part of the default, we'll redefine hv_default as:
hv_default = (hv_f1=on)
Quoted 10 lines
however due to ORing
hv_defaultdoesn't fix issue for the same CLI (i.e. it doesn't have expected effect)hv_f2=on,hv_default=on => hv_f1=on,hv_f2=on
if one would use usual 'set' semantics (leaf =
all_bits),
then newhv_defaultvalue will have desired effect despite of botched CLI, just by virtue of property following typical 'last set' semantics:
Quoted 6 lines
=> hv_f1=on,hv_f2=off
If we assume that we 'never ever' will need to disable feature bits than it doesn't matter which approach to use, however a look at
pc_compatarrays shows that features are being enabled/disabled all the time.
I'm pretty sure that "hv_default=on will also disable features that appear in the command line" will not be a requirement.
Quoted 24 lines
PS:
I'd renamehv_default=>hv_set_default,
since we would need hv_default[_value] property later on to set compat value based on machine type version.Quoted 18 lines
Quoted 17 lines
Quoted 1 line
I think I prefer sanity over purity in this case.
what is sanity to one could be insanity for another, so I pointed out the way properties expected to work today.
But you are adding new semantic ('combine') to property/features parsing (instead of current 'set' policy), and users will have to be aware of this new behavior and add/maintain code for this special case. (maybe I worry in vain, and no one will read docs and know about this new property anyways)
That will also push x86 CPUs consolidation farther away from other targets, where there aren't any special casing for features parsing, just simple left to right parsing with the latest property having overwriting previously set value.
We are trying hard to reduce special cases and unify interfaces for same components to simplify qemu and make it predictable/easier for users.
Quoted 5 lines
Quoted 4 lines
What you are proposing diverges from other targets, actually. See target/s390x/cpu_models.c:set_feature_group() for example. Enabling a feature group in s390x only enables a set of feature bits, and doesn't touch the rest.
Looking at code, it has the same issue as I described above
I don't see why that's an issue. This is how feature groups were designed, and it works.
Quoted 20 lines
Quoted 19 lines
In other words, if
hv_defaultincludes hv_f1+hv_f2 (and nothv_f3orhv_f4), this means:hv_default,hv_f3=on,hv_f4=off => (hv_f1=on,hv_f2=on),hv_f3=on,hv_f4=off == hv_f3=on,hv_f4=off,hv_default => hv_f3=on,hv_f4=off,(hv_f2=on,hv_f2=on)That would also mean:
hv_default,hv_f1=on,hv_f2=off => (hv_f1=on,hv_f2=on),hv_f1=on,hv_f2=off != hv_f1=on,hv_f2=off,hv_default => hv_f1=on,hv_f2=off,(hv_f2=on,hv_f2=on)That's the behavior implemented by Vitaly.
Quoted 1 line
[...]
Signature · 1 line
Eduardo
Re: [PATCH v3 18/19] i386: provide simple 'hv-default=on' option
Quoted 248 lines · Eduardo Habkost
Quoted 219 lines · Igor Mammedov
Quoted 201 lines · On Wed, 20 Jan 2021 15:49:09 -0500 Eduardo Habkost <[email protected]> wrote:
Quoted 200 lines · Igor Mammedov
Quoted 97 lines · On Wed, 20 Jan 2021 15:38:33 +0100 Vitaly Kuznetsov <[email protected]> wrote:
Quoted 52 lines · Igor Mammedov
Quoted 35 lines · On Fri, 15 Jan 2021 10:20:23 +0100 Vitaly Kuznetsov <[email protected]> wrote:
Quoted 33 lines · Igor Mammedov
Quoted 30 lines · On Thu, 7 Jan 2021 16:14:49 +0100 Vitaly Kuznetsov <[email protected]> wrote:
Enabling Hyper-V emulation for a Windows VM is a tiring experience as it requires listing all currently supported enlightenments ("hv-*" CPU features) explicitly. We do have 'hv-passthrough' mode enabling everything but it can't be used in production as it prevents migration.
Introduce a simple 'hv-default=on' CPU flag enabling all currently supported Hyper-V enlightenments. Later, when new enlightenments get implemented,
compat_propsmechanism will be used to disable them for legacy machine types, this will keep 'hv-default=on' configurations migratable.Signed-off-by Vitaly Kuznetsov <[email protected]>--- docs/hyperv.txt | 16 +++++++++++++--- target/i386/cpu.c | 38 ++++++++++++++++++++++++++++++++++++++ target/i386/cpu.h | 5 +++++ 3 files changed, 56 insertions(+), 3 deletions(-)docs/hyperv.txt +2
-2@@ -17,10 +17,20 @@ compatible hypervisor and use Hyper-V specific features. 2. Setup ========= -No Hyper-V enlightenments are enabled by default by either KVM or QEMU. In -QEMU, individual enlightenments can be enabled through CPU flags, e.g: +All currently supported Hyper-V enlightenments can be enabled by specifying +'hv-default=on' CPU flag:Quoted 11 lines
Quoted 11 lines
docs/hyperv.txt compatible hypervisor and use Hyper-V specific features. · line 28
Quoted 7 lines
docs/hyperv.txt +6
-1- qemu-system-x86_64 --enable-kvm --cpu host,hv_relaxed,hv_vpindex,hv_time, ... + qemu-system-x86_64 --enable-kvm --cpu host,hv-default ... + +Alternatively, it is possible to do fine-grained enablement through CPU flags, +e.g: + + qemu-system-x86_64 --enable-kvm --cpu host,hv-relaxed,hv-vpindex,hv-time ...I'd put here not '...' but rather recommended list of flags, and update it every time when new feature added if necessary.
1)
Quoted 7 lines
Quoted 3 lines
This is an example of fine-grained enablement, there is no point to put all the existing flags there (hv-default is the only recommended way now, the rest is 'expert'/'debugging').
so users are kept in dark what hv-default disables/enables (and it might depend on machine version on top that). Doesn't look like a good documentation to me (sure everyone can go and read source code for it and try to figure out how it's supposed to work)
'hv-default' enables *all* currently supported enlightenments. When using with an old machine type, it will enable *all* Hyper-V enlightenmnets which were supported when the corresponding machine type was released. I don't think we document all other cases when a machine type is modified (i.e. where can I read how pc-q35-5.1 is different from pc-q35-5.0 if I refuse to read the source code?)
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
(not to mention that if we had it to begin with, then new 'hv-default' won't be necessary, I still see it as functionality duplication but I will not oppose it)
Quoted 8 lines
Quoted 2 lines
Unfortunately, upper layer tools don't read this doc and update themselves to enable new features when they appear.
rant: (just merge all libvirt into QEMU, and make VM configuration less low-level. why stop there, just merge with yet another upper layer, it would save us a lot on communication protocols and simplify VM creation even more,
and no one will have to read docs and write anything new on top.) There should be limit somewhere, where QEMU job ends and others pile hw abstraction layers on top of it.We have '-machine q35' and we don't require to list all the devices from it. We have '-cpu Skylake-Server' and we don't require to configure all the features manually. Why can't we have similar enablement for Hyper-V emulation where we can't even see a real need for anything but 'enable everything' option?
There is no 'one libvirt to rule them all' (fortunately or unfortunately). And sometimes QEMU is the uppermost layer and there's no 'libvirt' on top of it, this is also a perfectly valid use-case.
Quoted 3 lines
Quoted 3 lines
Similarly, if when these tools use '-machine q35' they get all the new features we add automatically, right?
Quoted 8 lines
Quoted 5 lines
it depends, in case of CPUs, new features usually 'off' by default for existing models. In case of bugs, features sometimes could be flipped and versioned machines were used to keep broken CPU models on old machine types.
That's why I was saying that Hyper-V enlightenments hardly resemble 'hardware' CPU features.
Well, Microsoft chose to implement them as hardware concept (CPUID leaf), and I prefer to treat them the same way as any other CPUID bits.
Quoted 68 lines
Quoted 43 lines
Quoted 37 lines
Quoted 27 lines
Quoted 27 lines
docs/hyperv.txt +4
-0+It is also possible to disable individual enlightenments from the default list, +this can be used for debugging purposes: + + qemu-system-x86_64 --enable-kvm --cpu host,hv-default=on,hv-evmcs=off ... Sometimes there are dependencies between enlightenments, QEMU is supposed to check that the supplied configuration is sane.target/i386/cpu.c +10
-0@@ -4552,6 +4552,24 @@ static void x86_cpuid_set_tsc_freq(Object *obj, Visitor *v, const char *name, cpu->env.tsc_khz = cpu->env.user_tsc_khz = value /1000 ; } +static bool x86_hv_default_get (Object *obj,Error **errp) +{ +X86CPU *cpu =X86_CPU (obj); + +return cpu->hyperv_default; +} + +static void x86_hv_default_set (Object *obj,bool value,Error **errp) +{ +X86CPU *cpu =X86_CPU (obj);Quoted 9 lines
target/i386/cpu.c static void x86_cpuid_set_tsc_freq(Object *obj, Visitor *v, const char *name, · line 4569
Quoted 5 lines
target/i386/cpu.c +5
-0+ + cpu->hyperv_default = value; + +if (value) { + cpu->hyperv_features |= cpu->hyperv_default_features;s/|="/=/ please,
i.e. no option overrides whatever was specified before to keep semantics consistent.Quoted 6 lines
Hm,
this doesn't matter for the most recent machine type as
hyperv_default_featureshas all the features but imagine you're running an older machine type which doesn't have 'hv_feature'. Now yourQuoted 7 lines
normally one shouldn't use new feature with old machine type as it makes VM non-migratable to older QEMU that has this machine type but not this feature.
nitpicking:
according to (1) user should not use 'hv_feature' on old machine sincehv_defaultshould cover all their needs (well they don't know whathv_defaultactually is).Normally yes but I can imagine sticking to some old machine type for other-than-hyperv-enlightenments purposes and still wanting to add a newly introduced enlightenment. Migration is not always a must.
Quoted 13 lines
Quoted 13 lines
suggestion is
if I do:
'hv_default,hv_feature=on' I will get "
hyperv_default_features|hv_feature"but if I do
'hv_feature=on,hv_default' I will just get '
hyperv_default_features' (ashv_defaultenablement will overwrite everything)How is this consistent?
Quoted 8 lines
Quoted 5 lines
usual semantics for properties, is that the latest property overwrites, the previous property value parsed from left to right.
(i.e. if one asked forhv_default, one gets it related CPUID bit set/unset, if one needs more than that one should add more related features after that.This semantics probably doesn't apply to 'hv-default' case IMO as my brain refuses to accept the fact that
it's difficult probably because 'hv-default' is 'alias' property that covers all individual hv-foo features in one go and that individual features are exposed to user, but otherwise it is just a property that sets CPUID features or like any other property, and should be treated like such.
Quoted 4 lines
'hv_default,hv_feature' != 'hv_feature,hv_default'
which should express the same desire 'the default set PLUS the feature I want'.
if
hv_defaultwere touching different data, I'd agree.
But in the endhv_defaultboils down to the same CPUID bits as individual features:Quoted 7 lines
Quoted 3 lines
hv_default,hv_f2 => (hv_f1=on,hv_f2=off),hv_f2=on
!=hv_f2,hv_default => hv_f2=on,(hv_f1=on,hv_f2=off)
I don't know why you chose to define "
hv_default" as hv_f1=on,hv_f2=off. Ifhv_f2is not enabled byhv_default, it doesn't need to be touched byhv_defaultat all.Essentially I was thinking about hv_default=on as setting default value of hv CPUID leaf i.e. like doc claims, 'all' hv_* features (including turned off and unused bits) which always sets leaf to its default state.
Now lets consider following possible situation
using combine' approach (leaf |=some_bits):Quoted 8 lines
QEMU-6.0: initially we have all possible features enabled
hv_default = (hv_f1=on,hv_f2=on)hv_f2=on,hv_default=on == hv_f1=on,hv_f2=on
QEMU-6.1: disabled hv_f2=off that was causing problems
hv_default= (hv_f1=on,hv_f2=off)Why would we choose to do that?
If we decide f2 shouldn't be part of the default, we'll redefine
hv_defaultas:
hv_default= (hv_f1=on)Quoted 12 lines
however due to ORing
hv_defaultdoesn't fix issue for the same CLI (i.e. it doesn't have expected effect)hv_f2=on,hv_default=on => hv_f1=on,hv_f2=on
if one would use usual 'set' semantics (leaf =
all_bits),
then newhv_defaultvalue will have desired effect despite of botched CLI, just by virtue of property following typical 'last set' semantics:=> hv_f1=on,hv_f2=off
Quoted 7 lines
Quoted 4 lines
If we assume that we 'never ever' will need to disable feature bits than it doesn't matter which approach to use, however a look at
pc_compatarrays shows that features are being enabled/disabled all the time.I'm pretty sure that "hv_default=on will also disable features that appear in the command line" will not be a requirement.
Is there a definitive conclusion to this?
In reading the thread I realised that the patch I sent adding "kvm-no-defaults" may fall foul of the "process properties in order" rule.
That is..
kvm_clock=on,kvm_no_defaults=on
==
kvm_no_defaults=on,kvm_clock=on
...which would not be the case if the properties are processed strictly in order (because in the first case kvm_no_defaults would disable kvm_clock).
Quoted 49 lines
Quoted 24 lines
PS:
I'd renamehv_default=>hv_set_default,
since we would need hv_default[_value] property later on to set compat value based on machine type version.Quoted 18 lines
Quoted 17 lines
Quoted 1 line
I think I prefer sanity over purity in this case.
what is sanity to one could be insanity for another, so I pointed out the way properties expected to work today.
But you are adding new semantic ('combine') to property/features parsing (instead of current 'set' policy), and users will have to be aware of this new behavior and add/maintain code for this special case. (maybe I worry in vain, and no one will read docs and know about this new property anyways)
That will also push x86 CPUs consolidation farther away from other targets, where there aren't any special casing for features parsing, just simple left to right parsing with the latest property having overwriting previously set value.
We are trying hard to reduce special cases and unify interfaces for same components to simplify qemu and make it predictable/easier for users.Quoted 5 lines
Quoted 4 lines
What you are proposing diverges from other targets, actually. See target/s390x/cpu_models.c:set_feature_group() for example. Enabling a feature group in s390x only enables a set of feature bits, and doesn't touch the rest.
Looking at code, it has the same issue as I described above
I don't see why that's an issue. This is how feature groups were designed, and it works.
Quoted 15 lines
Quoted 15 lines
In other words, if
hv_defaultincludes hv_f1+hv_f2 (and nothv_f3orhv_f4), this means:hv_default,hv_f3=on,hv_f4=off => (hv_f1=on,hv_f2=on),hv_f3=on,hv_f4=off == hv_f3=on,hv_f4=off,hv_default => hv_f3=on,hv_f4=off,(hv_f2=on,hv_f2=on)That would also mean:
hv_default,hv_f1=on,hv_f2=off => (hv_f1=on,hv_f2=on),hv_f1=on,hv_f2=off != hv_f1=on,hv_f2=off,hv_default => hv_f1=on,hv_f2=off,(hv_f2=on,hv_f2=on)
Quoted 8 lines
Quoted 5 lines
Quoted 4 lines
That's the behavior implemented by Vitaly.
Quoted 1 line
[...]
Signature · 1 line
Eduardo
dme.
Signature · 1 line
There's too many people on the bus from the airport.
Re: [PATCH v3 18/19] i386: provide simple 'hv-default=on' option
In reply to Igor MammedovQuoted 38 lines · Igor Mammedov
Quoted 17 lines · On Wed, 20 Jan 2021 15:38:33 +0100 Vitaly Kuznetsov <[email protected]> wrote:
Quoted 15 lines · Igor Mammedov
Quoted 12 lines · On Fri, 15 Jan 2021 10:20:23 +0100 Vitaly Kuznetsov <[email protected]> wrote:
suggestion is
if I do:
'hv_default,hv_feature=on' I will get "
hyperv_default_features|hv_feature"but if I do
'hv_feature=on,hv_default' I will just get '
hyperv_default_features' (ashv_defaultenablement will overwrite everything)How is this consistent?
Quoted 8 lines
Quoted 5 lines
usual semantics for properties, is that the latest property overwrites, the previous property value parsed from left to right.
(i.e. if one asked forhv_default, one gets it related CPUID bit set/unset, if one needs more than that one should add more related features after that.This semantics probably doesn't apply to 'hv-default' case IMO as my brain refuses to accept the fact that
it's difficult probably because 'hv-default' is 'alias' property that covers all individual hv-foo features in one go and that individual features are exposed to user, but otherwise it is just a property that sets CPUID features or like any other property, and should be treated like such.
Quoted 4 lines
'hv_default,hv_feature' != 'hv_feature,hv_default'
which should express the same desire 'the default set PLUS the feature I want'.
Quoted 8 lines
if
hv_defaultwere touching different data, I'd agree.
But in the endhv_defaultboils down to the same CPUID bits as individual features:hv_default,hv_f2 => (hv_f1=on,hv_f2=off),hv_f2=on
!=hv_f2,hv_default => hv_f2=on,(hv_f1=on,hv_f2=off)
In your case I treat 'hv_default' as 'hv_f1=on' and it says nothing about 'hv_f2' - neither it is enabled, nor it is disabled because when the corresponding machine type was released it just wasn't there.
Quoted 11 lines
Quoted 2 lines
I think I prefer sanity over purity in this case.
what is sanity to one could be insanity for another, so I pointed out the way properties expected to work today.
But you are adding new semantic ('combine') to property/features parsing (instead of current 'set' policy), and users will have to be aware of this new behavior and add/maintain code for this special case. (maybe I worry in vain, and no one will read docs and know about this new property anyways)
Quoted 4 lines
That will also push x86 CPUs consolidation farther away from other targets, where there aren't any special casing for features parsing, just simple left to right parsing with the latest property having overwriting previously set value.
In case this is somewhat important I suggest we get back to adding 'hyperv=on' machine type option and not do the 'aliasing' with 'hv_default'. I think it would be possible to support
'-M q35,hyper=on -cpu host,hv-stimer-direct=off'
even if we need to add a custom handler for Hyper-V feature setting instead of just using bits in u64 as we need to remember both what was enabled and what was disabled to combine this with machine type property correctly.
Quoted 3 lines
We are trying hard to reduce special cases and unify interfaces for same components to simplify qemu and make it predictable/easier for users.
That's exactly the reason why we need simpler Hyper-V feature enablement! :-)
Quoted 46 lines
Quoted 38 lines
Quoted 38 lines
Quoted 37 lines
Quoted 25 lines
Quoted 25 lines
target/i386/cpu.c +13
-0+ } +} +/* Generic getter for "feature-words" and "filtered-features" properties */ static void x86_cpu_get_feature_words(Object *obj, Visitor *v, const char *name, void *opaque,@@ -6955,10 +6973,26 @@ static void x86_cpu_initfn(Object *obj)object_property_add_alias (obj,"pause_filter" , obj,"pause-filter" );object_property_add_alias (obj,"sse4_1" , obj,"sse4.1" );object_property_add_alias (obj,"sse4_2" , obj,"sse4.2" ); +object_property_add_alias (obj,"hv_default" , obj,"hv-default" );if (xcc->model) {x86_cpu_load_model (cpu, xcc->model); } + +/* Hyper-V features enabled with 'hv-default=on' */ + cpu->hyperv_default_features =BIT (HYPERV_FEAT_RELAXED ) | +BIT (HYPERV_FEAT_VAPIC ) |BIT (HYPERV_FEAT_TIME ) | +BIT (HYPERV_FEAT_CRASH ) |BIT (HYPERV_FEAT_RESET ) | +BIT (HYPERV_FEAT_VPINDEX ) |BIT (HYPERV_FEAT_RUNTIME ) | +BIT (HYPERV_FEAT_SYNIC ) |BIT (HYPERV_FEAT_STIMER ) | +BIT (HYPERV_FEAT_FREQUENCIES ) |BIT (HYPERV_FEAT_REENLIGHTENMENT ) | + BIT(HYPERV_FEAT_TLBFLUSH) | BIT(HYPERV_FEAT_IPI) |Quoted 9 lines
target/i386/cpu.c static void x86_cpu_initfn(Object *obj) · line 6995
Quoted 6 lines
target/i386/cpu.c +6
-0+BIT (HYPERV_FEAT_STIMER_DIRECT ); + +/* Enlightened VMCS is only available on Intel/VMX */ +if (kvm_hv_evmcs_available ()) { + cpu->hyperv_default_features |=BIT (HYPERV_FEAT_EVMCS ); + }what if VVM is migrated to another host without evmcs, will it change CPUID?
Evmcs is tightly coupled with VMX, we can't migrate when it's not there.
Quoted 7 lines
Quoted 2 lines
Are you saying mgmt will check and refuse to migrate to such host?
Is it possible to migrate a VM from a VMX-enabled host to a VMX-disabled one if VMX feature was exposed to the VM? Probably not, you will fail to create a VM on the destination host. Evmcs doesn't change anything in this regard, there are no hosts where VMX is available but EVMCS is not.
Quoted 8 lines
I'm not sure how evmcs should be handled,
can you point out what in this series makes sure that migration fails or makes qemu not able to start in casekvm_hv_evmcs_available()returns false.So far I read snippet above as a problem:
1:
host supports evmcs:
and exposesHYPERV_FEAT_EVMCSin CPUID
Host with EVMCS is Intel
Quoted 1 line
2: we migrate to host without evmcs
Host without EVMCS is AMD, there are no other options. It is a pure software feature available for KVM-intel. And if your KVM is so old that it doesn't know anything about EVMCS, a bunch of other options from 'hv-default' will not start as well.
Quoted 2 lines
2.1 start target QEMU, it happily creates vCPUs without
HYPERV_FEAT_EVMCSin CPUID
No, it doesn't as on host1 we had at least VMX CPU feature enabled (or a CPU model implying it) to make this all work.
Quoted 3 lines
2.2 if I'm not mistaken CPUID is not part of migration stream,
nothing could check and fail migration
2.3 guest runs fine till it tries to use non existing feature, ..
I'm also very sceptical about possibilities for migration Windows/Hyper-V VMs from Intel to AMD. Hyper-V doesn't even boot if you don't have fresh-enough CPU so the common denominator for Intel/AMD would definitely not work.
Signature · 1 line
Vitaly
Re: [PATCH v3 18/19] i386: provide simple 'hv-default=on' option
Quoted 156 lines · On Thu, 21 Jan 2021 09:45:33 +0100 Vitaly Kuznetsov <[email protected]> wrote:
Quoted 38 lines · Igor Mammedov
Quoted 17 lines · On Wed, 20 Jan 2021 15:38:33 +0100 Vitaly Kuznetsov <[email protected]> wrote:
Quoted 15 lines · Igor Mammedov
Quoted 12 lines · On Fri, 15 Jan 2021 10:20:23 +0100 Vitaly Kuznetsov <[email protected]> wrote:
suggestion is
if I do:
'hv_default,hv_feature=on' I will get "
hyperv_default_features|hv_feature"but if I do
'hv_feature=on,hv_default' I will just get '
hyperv_default_features' (ashv_defaultenablement will overwrite everything)How is this consistent?
Quoted 8 lines
Quoted 5 lines
usual semantics for properties, is that the latest property overwrites, the previous property value parsed from left to right.
(i.e. if one asked forhv_default, one gets it related CPUID bit set/unset, if one needs more than that one should add more related features after that.This semantics probably doesn't apply to 'hv-default' case IMO as my brain refuses to accept the fact that
it's difficult probably because 'hv-default' is 'alias' property that covers all individual hv-foo features in one go and that individual features are exposed to user, but otherwise it is just a property that sets CPUID features or like any other property, and should be treated like such.
Quoted 4 lines
'hv_default,hv_feature' != 'hv_feature,hv_default'
which should express the same desire 'the default set PLUS the feature I want'.
Quoted 8 lines
if
hv_defaultwere touching different data, I'd agree.
But in the endhv_defaultboils down to the same CPUID bits as individual features:hv_default,hv_f2 => (hv_f1=on,hv_f2=off),hv_f2=on
!=hv_f2,hv_default => hv_f2=on,(hv_f1=on,hv_f2=off)
In your case I treat '
hv_default' as 'hv_f1=on' and it says nothing about 'hv_f2' - neither it is enabled, nor it is disabled because when the corresponding machine type was released it just wasn't there.Quoted 11 lines
Quoted 2 lines
I think I prefer sanity over purity in this case.
what is sanity to one could be insanity for another, so I pointed out the way properties expected to work today.
But you are adding new semantic ('combine') to property/features parsing (instead of current 'set' policy), and users will have to be aware of this new behavior and add/maintain code for this special case. (maybe I worry in vain, and no one will read docs and know about this new property anyways)
Quoted 4 lines
That will also push x86 CPUs consolidation farther away from other targets, where there aren't any special casing for features parsing, just simple left to right parsing with the latest property having overwriting previously set value.
In case this is somewhat important I suggest we get back to adding 'hyperv=on' machine type option and not do the 'aliasing' with '
hv_default'. I think it would be possible to support'-M q35,hyper=on -cpu host,hv-stimer-direct=off'
even if we need to add a custom handler for Hyper-V feature setting instead of just using bits in u64 as we need to remember both what was enabled and what was disabled to combine this with machine type property correctly.
Quoted 3 lines
We are trying hard to reduce special cases and unify interfaces for same components to simplify qemu and make it predictable/easier for users.
That's exactly the reason why we need simpler Hyper-V feature enablement! :-)
Quoted 46 lines
Quoted 38 lines
Quoted 38 lines
Quoted 37 lines
Quoted 25 lines
Quoted 25 lines
(quoted diff) +13
-0+ } +} + /* Generic getter for "feature-words" and "filtered-features" properties */ static void x86_cpu_get_feature_words(Object *obj, Visitor *v, const char *name, void *opaque,@@ -6955,10 +6973,26 @@ static void x86_cpu_initfn(Object *obj) object_property_add_alias(obj, "pause_filter", obj, "pause-filter"); object_property_add_alias(obj, "sse4_1", obj, "sse4.1"); object_property_add_alias(obj, "sse4_2", obj, "sse4.2"); + object_property_add_alias(obj, "hv_default", obj, "hv-default"); if (xcc->model) { x86_cpu_load_model(cpu, xcc->model); } + + /* Hyper-V features enabled with 'hv-default=on' */ + cpu->hyperv_default_features = BIT(HYPERV_FEAT_RELAXED) | + BIT(HYPERV_FEAT_VAPIC) | BIT(HYPERV_FEAT_TIME) | + BIT(HYPERV_FEAT_CRASH) | BIT(HYPERV_FEAT_RESET) | + BIT(HYPERV_FEAT_VPINDEX) | BIT(HYPERV_FEAT_RUNTIME) | + BIT(HYPERV_FEAT_SYNIC) | BIT(HYPERV_FEAT_STIMER) | + BIT(HYPERV_FEAT_FREQUENCIES) | BIT(HYPERV_FEAT_REENLIGHTENMENT) | + BIT(HYPERV_FEAT_TLBFLUSH) | BIT(HYPERV_FEAT_IPI) |Quoted 9 lines
Review static void x86_cpu_initfn(Object *obj) · line 6995
Quoted 6 lines
(quoted diff) +6
-0+ BIT(HYPERV_FEAT_STIMER_DIRECT); + + /* Enlightened VMCS is only available on Intel/VMX */ + if (kvm_hv_evmcs_available()) { + cpu->hyperv_default_features |= BIT(HYPERV_FEAT_EVMCS); + }what if VVM is migrated to another host without evmcs, will it change CPUID?
Evmcs is tightly coupled with VMX, we can't migrate when it's not there.
Quoted 7 lines
Quoted 2 lines
Are you saying mgmt will check and refuse to migrate to such host?
Is it possible to migrate a VM from a VMX-enabled host to a VMX-disabled one if VMX feature was exposed to the VM? Probably not, you will fail to create a VM on the destination host. Evmcs doesn't change anything in this regard, there are no hosts where VMX is available but EVMCS is not.
Quoted 8 lines
I'm not sure how evmcs should be handled,
can you point out what in this series makes sure that migration fails or makes qemu not able to start in casekvm_hv_evmcs_available()returns false.So far I read snippet above as a problem:
1:
host supports evmcs:
and exposesHYPERV_FEAT_EVMCSin CPUIDHost with EVMCS is Intel
Quoted 1 line
2: we migrate to host without evmcs
Host without EVMCS is AMD, there are no other options. It is a pure software feature available for KVM-intel. And if your KVM is so old that it doesn't know anything about EVMCS, a bunch of other options from 'hv-default' will not start as well.
Quoted 2 lines
2.1 start target QEMU, it happily creates vCPUs without
HYPERV_FEAT_EVMCSin CPUIDNo, it doesn't as on host1 we had at least VMX CPU feature enabled (or a CPU model implying it) to make this all work.
Quoted 8 lines
Quoted 3 lines
2.2 if I'm not mistaken CPUID is not part of migration stream,
nothing could check and fail migration
2.3 guest runs fine till it tries to use non existing feature, ..I'm also very sceptical about possibilities for migration Windows/Hyper-V VMs from Intel to AMD. Hyper-V doesn't even boot if you don't have fresh-enough CPU so the common denominator for Intel/AMD would definitely not work.
Like you said host doesn't have to be AMD, just old enough kernel will do the job. What exactly will prevent migration 'successfully' completing?
The way it's currently written migration stream won't prevent it.
One way that might solve issue is to add subsection that's enabled when kvm_hv_evmcs_available() == true, and check on target that the feature is available or fail migration.
Maybe Eduardo or David can add more how to deal with it if needed.
Re: [PATCH v3 18/19] i386: provide simple 'hv-default=on' option
Quoted 27 lines · Igor Mammedov
Quoted 24 lines · On Thu, 21 Jan 2021 09:45:33 +0100 Vitaly Kuznetsov <[email protected]> wrote:
Quoted 5 lines
So far I read snippet above as a problem:
1:
host supports evmcs:
and exposesHYPERV_FEAT_EVMCSin CPUIDHost with EVMCS is Intel
Quoted 1 line
2: we migrate to host without evmcs
Host without EVMCS is AMD, there are no other options. It is a pure software feature available for KVM-intel. And if your KVM is so old that it doesn't know anything about EVMCS, a bunch of other options from 'hv-default' will not start as well.
Quoted 2 lines
2.1 start target QEMU, it happily creates vCPUs without
HYPERV_FEAT_EVMCSin CPUIDNo, it doesn't as on host1 we had at least VMX CPU feature enabled (or a CPU model implying it) to make this all work.
Quoted 3 lines
2.2 if I'm not mistaken CPUID is not part of migration stream,
nothing could check and fail migration
2.3 guest runs fine till it tries to use non existing feature, ..
Quoted 8 lines
Quoted 4 lines
I'm also very sceptical about possibilities for migration Windows/Hyper-V VMs from Intel to AMD. Hyper-V doesn't even boot if you don't have fresh-enough CPU so the common denominator for Intel/AMD would definitely not work.
Like you said host doesn't have to be AMD, just old enough kernel will do the job. What exactly will prevent migration 'successfully' completing?
First, you can't start a VM with 'hv-default' with an old-enough kernel because it won't have many other 'hv-' enlightenments implemented. 'hv-default' will only work for a 'recent enough' kernel (>= 5.0 when hv-stimer-direct was implemented).
You can probably try doing '-cpu xxx,hv_default,hv-stimer-direct=off' to trigger the problem but then KVM should also support nested state migration to actually migrate a VM using VMX and EVMCS support for it also emerged in 5.0. I believe that trying to call KVM_SET_NESTED_STATE (which only appeared in 4.19 btw) on something in between will fail.
Quoted 5 lines
The way it's currently written migration stream won't prevent it.
One way that might solve issue is to add subsection that's enabled when
kvm_hv_evmcs_available()== true, and check on target that the feature is available or fail migration.
Yes, we can but I don't think there's a real issue worth fighting for. Nested migration was so broken in upstream KVM untill recently that I don't see why 'old kernel' can be a problem at all. And, again, Intel to AMD migration is likely off question.
Quoted 3 lines
Maybe Eduardo or David can add more how to deal with it if needed.
Signature · 1 line
Vitaly
Re: [PATCH v3 18/19] i386: provide simple 'hv-default=on' option
In reply to Igor MammedovQuoted 7 lines · Igor Mammedov
Quoted 4 lines · On Fri, 15 Jan 2021 10:20:23 +0100 Vitaly Kuznetsov <[email protected]> wrote:
Quoted 2 lines · Igor Mammedov
On Thu, 7 Jan 2021 16:14:49 +0100
Vitaly Kuznetsov <[email protected]> wrote:
[...]
Quoted 19 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
docs/hyperv.txt +2
-2-No Hyper-V enlightenments are enabled by default by either KVM or QEMU. In -QEMU, individual enlightenments can be enabled through CPU flags, e.g: +All currently supported Hyper-V enlightenments can be enabled by specifying +'hv-default=on' CPU flag:Quoted 11 lines
Quoted 11 lines
docs/hyperv.txt compatible hypervisor and use Hyper-V specific features. · line 28
Quoted 7 lines
docs/hyperv.txt +6
-1- qemu-system-x86_64 --enable-kvm --cpu host,hv_relaxed,hv_vpindex,hv_time, ... + qemu-system-x86_64 --enable-kvm --cpu host,hv-default ... + +Alternatively, it is possible to do fine-grained enablement through CPU flags, +e.g: + + qemu-system-x86_64 --enable-kvm --cpu host,hv-relaxed,hv-vpindex,hv-time ...I'd put here not '...' but rather recommended list of flags, and update it every time when new feature added if necessary.
1)
Quoted 7 lines
Quoted 3 lines
This is an example of fine-grained enablement, there is no point to put all the existing flags there (hv-default is the only recommended way now, the rest is 'expert'/'debugging').
so users are kept in dark what hv-default disables/enables (and it might depend on machine version on top that). Doesn't look like a good documentation to me (sure everyone can go and read source code for it and try to figure out how it's supposed to work)
Why is this a problem? This is not different from CPU feature flags hidden by CPU model names. Or virtio feature bits hidden by virtio devices and machine type compat code.
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
(not to mention that if we had it to begin with, then new 'hv-default' won't be necessary, I still see it as functionality duplication but I will not oppose it)
Quoted 8 lines
Quoted 2 lines
Unfortunately, upper layer tools don't read this doc and update themselves to enable new features when they appear.
rant: (just merge all libvirt into QEMU, and make VM configuration less low-level. why stop there, just merge with yet another upper layer, it would save us a lot on communication protocols and simplify VM creation even more,
and no one will have to read docs and write anything new on top.) There should be limit somewhere, where QEMU job ends and others pile hw abstraction layers on top of it.
If there should be a limit somewhere, I'd say low level hyperv flags are a very long way from crossing that limit. It is completely reasonable to provide a higher level knob for them in QEMU. Vitaly is trying to solve a real problem here, because the existing solution is not working.
We only need to offer more complex and lower level interfaces if we have to. I don't think we have real world examples where libvirt or management software developers/users are asking for low level hyperv feature flag knobs, do we?
Quoted 9 lines
Quoted 3 lines
Similarly, if when these tools use '-machine q35' they get all the new features we add automatically, right?
it depends, in case of CPUs, new features usually 'off' by default for existing models. In case of bugs, features sometimes could be flipped and versioned machines were used to keep broken CPU models on old machine types.
[...]
Signature · 1 line
Eduardo
PATCH v3 19/19 qtest/hyperv: Introduce a simple hyper-v test
In reply to Vitaly KuznetsovFor the beginning, just test 'hv-default', 'hv-passthrough' and a couple of custom Hyper-V enlightenments configurations through QMP. Later, it would be great to complement this by checking CPUID values from within the guest.
--- MAINTAINERS | 1 + tests/qtest/hyperv-test.c | 238 ++++++++++++++++++++++++++++++++++++++ tests/qtest/meson.build | 3 +- 3 files changed, 241 insertions(+), 1 deletion(-) create mode 100644 tests/qtest/hyperv-test.c
MAINTAINERS +1 -0
@@ -1495,6 +1495,7 @@ F: hw/isa/apm.c F: include/hw/isa/apm.h F: tests/test-x86-cpuid.c F: tests/qtest/test-x86-cpuid-compat.c +F: tests/qtest/hyperv-test.c PC Chipset M: Michael S. Tsirkin <[email protected]>
tests/qtest/hyperv-test.c new file +238 -0
238 lines not on this page · raw message
tests/qtest/meson.build +2 -1
@@ -64,7 +64,8 @@ qtests_i386 = \ 'vmgenid-test', 'migration-test', 'test-x86-cpuid-compat', - 'numa-test'] + 'numa-test', + 'hyperv-test'] dbus_daemon = find_program('dbus-daemon', required: false) if dbus_daemon.found() and config_host.has_key('GDBUS_CODEGEN')
Signature · 1 line
2.29.2