Skip to contentSkip to conversation

x86/irq: forward pending interrupts to new destination in fixup_irqs()

PATCH In review Roger Pau Monne 3 participants 3 messages Jun 19, 2024 – Jun 20, 2024

lore
Share & tools
lore
#0 lore raw

PATCH v4 1/1 x86/irq: forward pending interrupts to new destination in fixup_irqs()

fixup_irqs() is used to evacuate interrupts from to be offlined CPUs. Given the CPU is to become offline, the normal migration logic used by Xen where the vector in the previous target(s) is left configured until the interrupt is received on the new destination is not suitable.

Instead attempt to do as much as possible in order to prevent loosing interrupts. If fixup_irqs() is called from the CPU to be offlined (as is currently the case for CPU hot unplug) attempt to forward pending vectors when interrupts that target the current CPU are migrated to a different destination.

Additionally, for interrupts that have already been moved from the current CPU prior to the call to fixup_irqs() but that haven't been delivered to the new destination (iow: interrupts with move_in_progress set and the current CPU set in ->arch.old_cpu_mask) also check whether the previous vector is pending and forward it to the new destination.

This allows us to remove the window with interrupts enabled at the bottom of fixup_irqs(). Such window wasn't safe anyway: references to the CPU to become offline are removed from interrupts masks, but the per-CPU vector_irq[] array is not updated to reflect those changes (as the CPU is going offline anyway).

---
Changes since v3:
- Move the IRR check after the cpumask_copy().

Changes since v2:
- Remove interrupt enabled window from fixup_irqs().
- Adjust comments and commit message.

Changes since v1:
- Rename to apic_irr_read().

---
 xen/arch/x86/include/asm/apic.h |  5 ++++
 xen/arch/x86/irq.c              | 46 ++++++++++++++++++++++++++++-----
 2 files changed, 45 insertions(+), 6 deletions(-)
xen/arch/x86/include/asm/apic.h +5 -0
index d1cb001fb4ab..7bd66dc6e151 100644
@@ -132,6 +132,11 @@ static inline bool apic_isr_read(uint8_t vector)
             (vector & 0x1f)) & 1;
 }
 
+static inline bool apic_irr_read(unsigned int vector)
+{
+    return apic_read(APIC_IRR + (vector / 32 * 0x10)) & (1U << (vector % 32));
+}
+
 static inline u32 get_apic_id(void)
 {
     u32 id = apic_read(APIC_ID);
xen/arch/x86/irq.c +40 -6

76 lines not on this page · raw message

Signature · 1 line
2.45.2
Reviewed-by on v4 1/1
#1 lore raw
Quoted 16 lines · Roger Pau Monne

fixup_irqs() is used to evacuate interrupts from to be offlined CPUs. Given the CPU is to become offline, the normal migration logic used by Xen where the vector in the previous target(s) is left configured until the interrupt is received on the new destination is not suitable.

Instead attempt to do as much as possible in order to prevent loosing interrupts. If fixup_irqs() is called from the CPU to be offlined (as is currently the case for CPU hot unplug) attempt to forward pending vectors when interrupts that target the current CPU are migrated to a different destination.

Additionally, for interrupts that have already been moved from the current CPU prior to the call to fixup_irqs() but that haven't been delivered to the new destination (iow: interrupts with move_in_progress set and the current CPU set in ->arch.old_cpu_mask) also check whether the previous vector is pending and forward it to the new destination.

Quoted 6 lines

This allows us to remove the window with interrupts enabled at the bottom of fixup_irqs(). Such window wasn't safe anyway: references to the CPU to become offline are removed from interrupts masks, but the per-CPU vector_irq[] array is not updated to reflect those changes (as the CPU is going offline anyway).

#2 lore raw
Quoted 37 lines · Jan Beulich
Quoted 36 lines · Roger Pau Monne

fixup_irqs() is used to evacuate interrupts from to be offlined CPUs.  Given
the CPU is to become offline, the normal migration logic used by Xen where the
vector in the previous target(s) is left configured until the interrupt is
received on the new destination is not suitable.

Instead attempt to do as much as possible in order to prevent loosing
interrupts.  If fixup_irqs() is called from the CPU to be offlined (as is
currently the case for CPU hot unplug) attempt to forward pending vectors when
interrupts that target the current CPU are migrated to a different destination.

Additionally, for interrupts that have already been moved from the current CPU
prior to the call to fixup_irqs() but that haven't been delivered to the new
destination (iow: interrupts with move_in_progress set and the current CPU set
in ->arch.old_cpu_mask) also check whether the previous vector is pending and
forward it to the new destination.

This allows us to remove the window with interrupts enabled at the bottom of
fixup_irqs().  Such window wasn't safe anyway: references to the CPU to become
offline are removed from interrupts masks, but the per-CPU vector_irq[] array
is not updated to reflect those changes (as the CPU is going offline anyway).

Quoted 3 lines
Quoted 1 line

~ Oleksii