Support x2APIC mode with TCG accelerator
v3 cover Support x2APIC mode with TCG accelerator
Hi everyone,
This series implements x2APIC mode in userspace local APIC and the RDMSR/WRMSR helper to access x2APIC registers in x2APIC mode. Intel iommu and AMD iommu are adjusted to support x2APIC interrupt remapping. With this series, we can now boot Linux kernel into x2APIC mode with TCG accelerator using either Intel or AMD iommu.
Testing the emulated userspace APIC with kvm-unit-tests, disable test device with this patch
lib/x86/fwcfg.c +1 -0
@@ -27,6 +27,7 @@ static void read_cfg_override(void)if ((str =getenv ("TEST_DEVICE" ))) no_test_device = !atol (str); + no_test_device = true;if ((str =getenv ("MEMLIMIT" ))) fw_override[FW_CFG_MAX_RAM ] =atol (str) *1024 *1024 ;
~ env QEMU=/home/minh/Desktop/oss/qemu/build/qemu-system-x86_64 ACCEL=tcg \
./run_tests.sh -v -g apic
TESTNAME=apic-split TIMEOUT=90s ACCEL=tcg ./x86/run x86/apic.flat -smp 2 -cpu qemu64,+x2apic,+tsc-deadline -machine kernel_irqchip=split FAIL apic-split (54 tests, 8 unexpected failures, 1 skipped) TESTNAME=ioapic-split TIMEOUT=90s ACCEL=tcg ./x86/run x86/ioapic.flat -smp 1 -cpu qemu64 -machine kernel_irqchip=split PASS ioapic-split (19 tests) TESTNAME=x2apic TIMEOUT=30 ACCEL=tcg ./x86/run x86/apic.flat -smp 2 -cpu qemu64,+x2apic,+tsc-deadline FAIL x2apic (54 tests, 8 unexpected failures, 1 skipped) TESTNAME=xapic TIMEOUT=60 ACCEL=tcg ./x86/run x86/apic.flat -smp 2 -cpu qemu64,-x2apic,+tsc-deadline -machine pit=off FAIL xapic (43 tests, 6 unexpected failures, 2 skipped)
FAIL: apic_disable: *0xfee00030: 50014
FAIL: apic_disable: *0xfee00080: f0
FAIL: apic_disable: *0xfee00030: 50014
FAIL: apic_disable: *0xfee00080: f0
FAIL: apicbase: relocate apic
These errors are because we don't disable MMIO region when switching to x2APIC and don't support relocate MMIO region yet. This is a problem because, MMIO region is the same for all CPUs, in order to support these we need to figure out how to allocate and manage different MMIO regions for each CPUs. This can be an improvement in the future.
FAIL: nmi-after-sti
FAIL: multiple nmi
These errors are in the way we handle CPU_INTERRUPT_NMI in core TCG.
FAIL: TMCCT should stay at zero
This error is related to APIC timer which should be addressed in separate patch.
Version 3 changes,
- Patch 2:
+ Allow APIC ID > 255 only when x2APIC feature is supported on CPU
+ Make physical destination mode IPI which has destination id 0xffffffff a broadcast to xAPIC CPUs
+ Make cluster address 0xf in cluster model of xAPIC logical destination mode a broadcast to all clusters
+ Create new extended_log_dest to store APIC_LDR information in x2APIC instead of extending log_dest for backward compatibility in vmstate
Version 2 changes,
- Add support for APIC ID larger than 255
- Adjust AMD iommu for x2APIC suuport
- Reorganize and split patch 1,2 into patch 1,2,3 in version 2
Thanks,
Quang Minh.
Bui Quang Minh (5):
i386/tcg: implement x2APIC registers MSR access
apic: add support for x2APIC mode
apic, i386/tcg: add x2apic transitions
intel_iommu: allow Extended Interrupt Mode when using userspace APIC
amd_iommu: report x2APIC support to the operating system
hw/i386/acpi-build.c | 28 +- hw/i386/amd_iommu.c | 21 +- hw/i386/amd_iommu.h | 16 +- hw/i386/intel_iommu.c | 11 - hw/i386/x86.c | 8 +- hw/intc/apic.c | 395 +++++++++++++++++++++------ hw/intc/apic_common.c | 16 +- hw/intc/trace-events | 4 +- include/hw/i386/apic.h | 6 +- include/hw/i386/apic_internal.h | 7 +- target/i386/cpu-sysemu.c | 18 +- target/i386/cpu.c | 5 +- target/i386/cpu.h | 9 + target/i386/tcg/sysemu/misc_helper.c | 31 +++ 14 files changed, 436 insertions(+), 139 deletions(-)
Signature · 1 line
2.25.1
Quoted 10 lines · Bui Quang Minh
Hi everyone,
This series implements x2APIC mode in userspace local APIC and the RDMSR/WRMSR helper to access x2APIC registers in x2APIC mode. Intel iommu and AMD iommu are adjusted to support x2APIC interrupt remapping. With this series, we can now boot Linux kernel into x2APIC mode with TCG accelerator using either Intel or AMD iommu.
Testing the emulated userspace APIC with kvm-unit-tests, disable test device with this patch
Threading's broken with this posting.
Quoted 92 lines
lib/x86/fwcfg.c +1
-0@@ -27,6 +27,7 @@ static void read_cfg_override(void)if ((str =getenv ("TEST_DEVICE" ))) no_test_device = !atol (str); + no_test_device = true;if ((str =getenv ("MEMLIMIT" ))) fw_override[FW_CFG_MAX_RAM ] =atol (str) *1024 *1024 ;~ env QEMU=/home/minh/Desktop/oss/qemu/build/qemu-system-x86_64 ACCEL=tcg \
./run_tests.sh -v -g apicTESTNAME=apic-split TIMEOUT=90s ACCEL=tcg ./x86/run x86/apic.flat -smp 2 -cpu qemu64,+x2apic,+tsc-deadline -machine kernel_irqchip=split FAIL apic-split (54 tests, 8 unexpected failures, 1 skipped) TESTNAME=ioapic-split TIMEOUT=90s ACCEL=tcg ./x86/run x86/ioapic.flat -smp 1 -cpu qemu64 -machine kernel_irqchip=split PASS ioapic-split (19 tests) TESTNAME=x2apic TIMEOUT=30 ACCEL=tcg ./x86/run x86/apic.flat -smp 2 -cpu qemu64,+x2apic,+tsc-deadline FAIL x2apic (54 tests, 8 unexpected failures, 1 skipped) TESTNAME=xapic TIMEOUT=60 ACCEL=tcg ./x86/run x86/apic.flat -smp 2 -cpu qemu64,-x2apic,+tsc-deadline -machine pit=off FAIL xapic (43 tests, 6 unexpected failures, 2 skipped)
FAIL:
apic_disable: *0xfee00030: 50014
FAIL:apic_disable: *0xfee00080: f0
FAIL:apic_disable: *0xfee00030: 50014
FAIL:apic_disable: *0xfee00080: f0
FAIL: apicbase: relocate apicThese errors are because we don't disable MMIO region when switching to x2APIC and don't support relocate MMIO region yet. This is a problem because, MMIO region is the same for all CPUs, in order to support these we need to figure out how to allocate and manage different MMIO regions for each CPUs. This can be an improvement in the future.
FAIL: nmi-after-sti
FAIL: multiple nmiThese errors are in the way we handle
CPU_INTERRUPT_NMIin core TCG.FAIL: TMCCT should stay at zero
This error is related to APIC timer which should be addressed in separate patch.
Version 3 changes,
- Patch 2:
+ Allow APIC ID > 255 only when x2APIC feature is supported on CPU
+ Make physical destination mode IPI which has destination id0xffffffffa broadcast to xAPIC CPUs
+ Make cluster address0xfin cluster model of xAPIC logical destination mode a broadcast to all clusters
+ Create newextended_log_destto storeAPIC_LDRinformation in x2APIC instead of extendinglog_destfor backward compatibility in vmstateVersion 2 changes,
- Add support for APIC ID larger than 255
- Adjust AMD iommu for x2APIC suuport
- Reorganize and split patch 1,2 into patch 1,2,3 in version 2Thanks,
Quang Minh.Bui Quang Minh (5):
i386/tcg: implement x2APIC registers MSR access
apic: add support for x2APIC mode
apic, i386/tcg: add x2apic transitions
intel_iommu: allow Extended Interrupt Mode when using userspace APIC
amd_iommu: report x2APIC support to the operating systemhw/i386/acpi-build.c | 28 +- hw/i386/amd_iommu.c | 21 +- hw/i386/amd_iommu.h | 16 +- hw/i386/intel_iommu.c | 11 - hw/i386/x86.c | 8 +- hw/intc/apic.c | 395 +++++++++++++++++++++------ hw/intc/apic_common.c | 16 +- hw/intc/trace-events | 4 +- include/hw/i386/apic.h | 6 +- include/hw/i386/apic_internal.h | 7 +- target/i386/cpu-sysemu.c | 18 +- target/i386/cpu.c | 5 +- target/i386/cpu.h | 9 + target/i386/tcg/sysemu/misc_helper.c | 31 +++ 14 files changed, 436 insertions(+), 139 deletions(-)Signature · 1 line
2.25.1
Quoted 9 lines · Michael S. Tsirkin
Quoted 8 lines · Bui Quang Minh
Hi everyone,
This series implements x2APIC mode in userspace local APIC and the RDMSR/WRMSR helper to access x2APIC registers in x2APIC mode. Intel iommu and AMD iommu are adjusted to support x2APIC interrupt remapping. With this series, we can now boot Linux kernel into x2APIC mode with TCG accelerator using either Intel or AMD iommu.
Quoted 5 lines
Quoted 2 lines
Testing the emulated userspace APIC with kvm-unit-tests, disable test device with this patch
Threading's broken with this posting.
Oh, thank you, I will repost this series with correct threading. Quang Minh.