Skip to contentSkip to conversation

KVM: arm64: Fix CPUHP logic for protected KVM

PATCH In review Raghavendra Rao Ananta 2 participants 3 messages Jul 19, 2023

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 KVM: arm64: Fix CPUHP logic for protected KVM

For protected kvm, the CPU hotplug 'down' logic currently brings down the timer and vGIC, essentially disabling interrupts. However, because of how the 'kvm_arm_hardware_enabled' flag is designed, it never re-enables them back on the CPU hotplug 'up' path. Hence, clean up the logic to maintain the CPU hotplug up/down symmetry.

Fixes 466d27e48d7c ("KVM: arm64: Simplify the CPUHP logic")
---
 arch/arm64/kvm/arm.c | 14 ++++----------
 1 file changed, 4 insertions(+), 10 deletions(-)
arch/arm64/kvm/arm.c +4 -10
index c2c14059f6a8..010ebfa69650 100644
@@ -1867,14 +1867,10 @@ static void _kvm_arch_hardware_enable(void *discard)
 
 int kvm_arch_hardware_enable(void)
 {
-	int was_enabled = __this_cpu_read(kvm_arm_hardware_enabled);
-
 	_kvm_arch_hardware_enable(NULL);
 
-	if (!was_enabled) {
-		kvm_vgic_cpu_up();
-		kvm_timer_cpu_up();
-	}
+	kvm_vgic_cpu_up();
+	kvm_timer_cpu_up();
 
 	return 0;
 }
@@ -1889,10 +1885,8 @@ static void _kvm_arch_hardware_disable(void *discard)
 
 void kvm_arch_hardware_disable(void)
 {
-	if (__this_cpu_read(kvm_arm_hardware_enabled)) {
-		kvm_timer_cpu_down();
-		kvm_vgic_cpu_down();
-	}
+	kvm_timer_cpu_down();
+	kvm_vgic_cpu_down();
 
 	if (!is_protected_kvm_enabled())
 		_kvm_arch_hardware_disable(NULL);
Signature · 1 line
2.41.0.487.g6d72f3e995-goog
#1 lore raw
Quoted 5 lines · Raghavendra Rao Ananta

For protected kvm, the CPU hotplug 'down' logic currently brings down the timer and vGIC, essentially disabling interrupts. However, because of how the 'kvm_arm_hardware_enabled' flag is designed, it never re-enables them back on the CPU hotplug 'up' path. Hence, clean up the logic to maintain the CPU hotplug up/down symmetry.

Correct me if I am wrong, but this issue exists outside of cpu hotplug, right? init_subsystems() calls _kvm_arch_hardware_enable() on all cores, which only sets up the hyp cpu context and not the percpu interrupts. Similar issue exists for the cpu that calls do_pkvm_init().

I'll also note kvm_arm_hardware_enabled is deceptively vague, as it only keeps track of whether or not the hyp cpu context has been initialized. May send a cleanup here in a bit.

Perhaps this for the changelog:

KVM: arm64: Fix hardware enable/disable flows for pKVM

When running in protected mode, the hyp stub is disabled after pKVM is initialized, meaning the host cannot enable/disable the hyp at runtime. As such, kvm_arm_hardware_enabled is always 1 after initialization, and kvm_arch_hardware_enable() never enables the vgic maintenance irq or timer irqs.

Unconditionally enable/disable the vgic + timer irqs in the respective calls, instead relying on the percpu bookkeeping in the generic code to keep track of which cpus have the interrupts unmasked.

Quoted 45 lines
Fixes 466d27e48d7c ("KVM: arm64: Simplify the CPUHP logic")
---
 arch/arm64/kvm/arm.c | 14 ++++----------
 1 file changed, 4 insertions(+), 10 deletions(-)
arch/arm64/kvm/arm.c +4 -10
index c2c14059f6a8..010ebfa69650 100644
@@ -1867,14 +1867,10 @@ static void _kvm_arch_hardware_enable(void *discard)
 
 int kvm_arch_hardware_enable(void)
 {
-	int was_enabled = __this_cpu_read(kvm_arm_hardware_enabled);
-
 	_kvm_arch_hardware_enable(NULL);
 
-	if (!was_enabled) {
-		kvm_vgic_cpu_up();
-		kvm_timer_cpu_up();
-	}
+	kvm_vgic_cpu_up();
+	kvm_timer_cpu_up();
 
 	return 0;
 }
@@ -1889,10 +1885,8 @@ static void _kvm_arch_hardware_disable(void *discard)
 
 void kvm_arch_hardware_disable(void)
 {
-	if (__this_cpu_read(kvm_arm_hardware_enabled)) {
-		kvm_timer_cpu_down();
-		kvm_vgic_cpu_down();
-	}
+	kvm_timer_cpu_down();
+	kvm_vgic_cpu_down();
 
 	if (!is_protected_kvm_enabled())
 		_kvm_arch_hardware_disable(NULL);
Signature · 1 line
2.41.0.487.g6d72f3e995-goog
Signature · 2 lines
Thanks,
Oliver
#2 lore raw
Quoted 8 lines · Oliver Upton
Quoted 5 lines · Raghavendra Rao Ananta

For protected kvm, the CPU hotplug 'down' logic currently brings down the timer and vGIC, essentially disabling interrupts. However, because of how the 'kvm_arm_hardware_enabled' flag is designed, it never re-enables them back on the CPU hotplug 'up' path. Hence, clean up the logic to maintain the CPU hotplug up/down symmetry.

Quoted 5 lines

Correct me if I am wrong, but this issue exists outside of cpu hotplug, right? init_subsystems() calls _kvm_arch_hardware_enable() on all cores, which only sets up the hyp cpu context and not the percpu interrupts. Similar issue exists for the cpu that calls do_pkvm_init().

Ah, perhaps I looked at the from a different perspective, but this makes sense too.

Quoted 14 lines

I'll also note kvm_arm_hardware_enabled is deceptively vague, as it only keeps track of whether or not the hyp cpu context has been initialized. May send a cleanup here in a bit.

Perhaps this for the changelog:

KVM: arm64: Fix hardware enable/disable flows for pKVM

When running in protected mode, the hyp stub is disabled after pKVM is initialized, meaning the host cannot enable/disable the hyp at runtime. As such, kvm_arm_hardware_enabled is always 1 after initialization, and kvm_arch_hardware_enable() never enables the vgic maintenance irq or timer irqs.

Quoted 4 lines

Unconditionally enable/disable the vgic + timer irqs in the respective calls, instead relying on the percpu bookkeeping in the generic code to keep track of which cpus have the interrupts unmasked.

Sure, we can use this for v2.

Thanks,
Raghavendra

Quoted 49 lines
Quoted 43 lines
Fixes 466d27e48d7c ("KVM: arm64: Simplify the CPUHP logic")
---
 arch/arm64/kvm/arm.c | 14 ++++----------
 1 file changed, 4 insertions(+), 10 deletions(-)
arch/arm64/kvm/arm.c +4 -11
index c2c14059f6a8..010ebfa69650 100644
@@ -1867,14 +1867,10 @@ static void _kvm_arch_hardware_enable(void *discard)
 
 int kvm_arch_hardware_enable(void)
 {
-     int was_enabled = __this_cpu_read(kvm_arm_hardware_enabled);
-
      _kvm_arch_hardware_enable(NULL);
 
-     if (!was_enabled) {
-             kvm_vgic_cpu_up();
-             kvm_timer_cpu_up();
-     }
+     kvm_vgic_cpu_up();
+     kvm_timer_cpu_up();
 
      return 0;
 }
@@ -1889,10 +1885,8 @@ static void _kvm_arch_hardware_disable(void *discard)
 
 void kvm_arch_hardware_disable(void)
 {
-     if (__this_cpu_read(kvm_arm_hardware_enabled)) {
-             kvm_timer_cpu_down();
-             kvm_vgic_cpu_down();
-     }
+     kvm_timer_cpu_down();
+     kvm_vgic_cpu_down();
 
      if (!is_protected_kvm_enabled())
              _kvm_arch_hardware_disable(NULL);
--
Quoted 2 lines

2.41.0.487.g6d72f3e995-goog

--
Thanks,
Oliver