enable x2APIC without interrupt remapping under KVM

KVM would like to provide x2APIC interface to a guest without emulating interrupt remapping device. The reason KVM prefers guest to use x2APIC is that x2APIC interface is better virtualizable and provides better performance than mmio xAPIC interface:

- msr exits are faster than mmio (no page table walk, emulation)
- no need to read back ICR to look at the busy bit
- one 64 bit ICR write instead of two 32 bit writes
- shared code with the Hyper-V paravirt interface

Included patch changes x2APIC enabling logic to enable it even if IR initialization failed, but kernel runs under KVM and no apic id is greater than 255 (if there is one spec requires BIOS to move to x2apic mode before starting an OS).

Signed-off-by Gleb Natapov <[email protected]>

---

v2:
Add help message to Kconfig
Force physical mode if IR is not available.

arch/x86/Kconfig +0 -0

index d1430ef..3e5b6ea 100644

Quoted 28 lines · Gleb Natapov

KVM would like to provide x2APIC interface to a guest without emulating interrupt remapping device. The reason KVM prefers guest to use x2APIC is that x2APIC interface is better virtualizable and provides better performance than mmio xAPIC interface:

- msr exits are faster than mmio (no page table walk, emulation)
- no need to read back ICR to look at the busy bit
- one 64 bit ICR write instead of two 32 bit writes
- shared code with the Hyper-V paravirt interface

Included patch changes x2APIC enabling logic to enable it even if IR initialization failed, but kernel runs under KVM and no apic id is greater than 255 (if there is one spec requires BIOS to move to x2apic mode before starting an OS).

Signed-off-by Gleb Natapov <[email protected]>

---

v2:
Add help message to Kconfig
Force physical mode if IR is not available.

arch/x86/Kconfig +0 -0

index d1430ef..3e5b6ea 100644
@@ -260,12 +260,15 @@ config SMP
 

arch/x86/Kconfig config SMP · line 263

Quoted 4 lines

arch/x86/Kconfig +1 -1

 config X86_X2APIC
 	bool "Support x2apic"
-	depends on X86_LOCAL_APIC && X86_64 && INTR_REMAP
+	depends on X86_LOCAL_APIC && X86_64

Have you tried to build this with X86_X2APIC set but INTR_REMAP disabled?

arch/x86/kernel/apic/apic.c void __init enable_IR_x2apic(void) · line 1405

Quoted 4 lines · Gleb Natapov

arch/x86/kernel/apic/apic.c +0 -2

-	local_irq_save(flags);
 	mask_IO_APIC_setup(ioapic_entries);
-	mask_8259A();

Is there a reason why the 8259 mask/unmask operations are separated from io-apic mask/unmask operations.

Can we keep it together so that it will be easy to read and understand that we first do the interrupt subsystem mask, try enabling IR and x2apic and unmask the interrupt subsystem.

Otherwise I am ok with this change.

thanks,
suresh

I left interrupt masking in enable_IR_x2apic() because interrupt should be masked during transition to x2apic mode, so it can't be moved to enable_IR(). I moved io-apic into enable_IR() because the state of io-apic depend on whether IR was enabled or not, so I left it close to IR enabling logic.

Also io-apic masking can fail, but we still want to
enable x2apic on KVM if possible, and moving io-apic masking to enable_IR_x2apic() will complicate the logic.

Quoted 5 lines

Can we keep it together so that it will be easy to read and understand that we first do the interrupt subsystem mask, try enabling IR and x2apic and unmask the interrupt subsystem.

For io-apic this is not 100% symmetric. We mask io-apic, enable IR and if this succeeds we do not unmask io-apic.