x86/PCI/MSI: Make sure that irq reservation mode works everywhere
cover x86/PCI/MSI: Make sure that irq reservation mode works everywhere
Dexuan reported that the recent rework of the vector allocation mode in x86 broke HyperV PCI passtrough because the rework missed to add the MSI_FLAG_MUST_REACTIVATE flag to the HyperV/PCI interrupt domain info.
The simple solution would be to set the flag in the HyperV/PCI driver but it's better to make this generic and let the PCI/MSI core code set the flag when reservation mode is enabled. That ensures that future users of this wont trip over the same problem.
Thanks,
tglx
--- arch/x86/Kconfig | 2 +- arch/x86/kernel/apic/msi.c | 5 ++--- drivers/pci/msi.c | 2 ++ kernel/irq/Kconfig | 3 +++ 4 files changed, 8 insertions(+), 4 deletions(-)
PATCH 1/3 genirq: Add config option for reservation mode
The interrupt reservation mode requires reactivation of PCI/MSI interrupts. Create a config option, so the PCI code can set the corresponding flag when required.
--- kernel/irq/Kconfig | 3 +++ 1 file changed, 3 insertions(+)
kernel/irq/Kconfig +3 -0
@@ -100,6 +100,9 @@ config IRQ_TIMINGS config GENERIC_IRQ_MATRIX_ALLOCATOR bool +config GENERIC_IRQ_RESERVATION_MODE + bool + config IRQ_DOMAIN_DEBUG bool "Expose hardware/virtual IRQ mapping via debugfs" depends on IRQ_DOMAIN && DEBUG_FS
tip-bot for Thomas Gleixner
merge bot[tip:x86/apic] genirq: Add config option for reservation mode
genirq: Add config option for reservation mode
Commit-ID: 2b5175c4fa974b6aa05bbd2ee8d443a8036a1714 Gitweb: https://git.kernel.org/tip/2b5175c4fa974b6aa05bbd2ee8d443a8036a1714 Author: Thomas Gleixner <[email protected]> AuthorDate: Tue, 17 Oct 2017 09:54:57 +0200 Committer: Thomas Gleixner <[email protected]> CommitDate: Wed, 18 Oct 2017 15:38:30 +0200
genirq: Add config option for reservation mode
The interrupt reservation mode requires reactivation of PCI/MSI interrupts. Create a config option, so the PCI code can set the corresponding flag when required.
--- kernel/irq/Kconfig | 3 +++ 1 file changed, 3 insertions(+)
kernel/irq/Kconfig +3 -0
@@ -100,6 +100,9 @@ config IRQ_TIMINGS config GENERIC_IRQ_MATRIX_ALLOCATOR bool +config GENERIC_IRQ_RESERVATION_MODE + bool + config IRQ_DOMAIN_DEBUG bool "Expose hardware/virtual IRQ mapping via debugfs" depends on IRQ_DOMAIN && DEBUG_FS
PATCH 2/3 PCI/MSI: Set MSI_FLAG_MUST_REACTIVATE in core code
In reply to Thomas GleixnerIf interrupt reservation mode is enabled then the PCI/MSI interrupts must be reactivated after early activation.
Make sure that all callers of pci_msi_create_irq_domain() have the MSI_FLAG_MUST_REACTIVATE set when reservation mode is enabled.
--- drivers/pci/msi.c | 2 ++ 1 file changed, 2 insertions(+)
drivers/pci/msi.c +2 -0
@@ -1441,6 +1441,8 @@ struct irq_domain *pci_msi_create_irq_dopci_msi_domain_update_chip_ops (info); info->flags |=MSI_FLAG_ACTIVATE_EARLY ; +if (IS_ENABLED (CONFIG_GENERIC_IRQ_RESERVATION )) + info->flags |=MSI_FLAG_MUST_REACTIVATE ; domain =msi_create_irq_domain (fwnode, info, parent);if (!domain)
Re: [patch 2/3] PCI/MSI: Set MSI_FLAG_MUST_REACTIVATE in core code
Quoted 14 lines · Thomas Gleixner
If interrupt reservation mode is enabled then the PCI/MSI interrupts must be reactivated after early activation.
Make sure that all callers of
pci_msi_create_irq_domain()have theMSI_FLAG_MUST_REACTIVATEset when reservation mode is enabled.Signed-off-by Thomas Gleixner <[email protected]>--- drivers/pci/msi.c | 2 ++ 1 file changed, 2 insertions(+)
drivers/pci/msi.c struct irq_domain *pci_msi_create_irq_do · line 1444
Quoted 4 lines
drivers/pci/msi.c +1
-0@@ -1441,6 +1441,8 @@ struct irq_domain *pci_msi_create_irq_dopci_msi_domain_update_chip_ops (info); info->flags |=MSI_FLAG_ACTIVATE_EARLY ; +if (IS_ENABLED (CONFIG_GENERIC_IRQ_RESERVATION ))
Bah. quilt refresh seems overrated. Correct patch below.
Thanks,
tglx
8<----------------
drivers/pci/msi.c +2 -0
@@ -1441,6 +1441,8 @@ struct irq_domain *pci_msi_create_irq_dopci_msi_domain_update_chip_ops (info); info->flags |=MSI_FLAG_ACTIVATE_EARLY ; +if (IS_ENABLED (CONFIG_GENERIC_IRQ_RESERVATION_MODE )) + info->flags |=MSI_FLAG_MUST_REACTIVATE ; domain =msi_create_irq_domain (fwnode, info, parent);if (!domain)
tip-bot for Thomas Gleixner
merge bot[tip:x86/apic] PCI/MSI: Set MSI_FLAG_MUST_REACTIVATE in core code
In reply to Thomas GleixnerPCI/MSI: Set MSI_FLAG_MUST_REACTIVATE in core code
Commit-ID: 25e960efc63852b84d1c3739aef586285b177395 Gitweb: https://git.kernel.org/tip/25e960efc63852b84d1c3739aef586285b177395 Author: Thomas Gleixner <[email protected]> AuthorDate: Tue, 17 Oct 2017 09:54:58 +0200 Committer: Thomas Gleixner <[email protected]> CommitDate: Wed, 18 Oct 2017 15:38:31 +0200
PCI/MSI: Set MSI_FLAG_MUST_REACTIVATE in core code
If interrupt reservation mode is enabled then the PCI/MSI interrupts must be reactivated after early activation.
Make sure that all callers of pci_msi_create_irq_domain() have the MSI_FLAG_MUST_REACTIVATE set when reservation mode is enabled.
--- drivers/pci/msi.c | 2 ++ 1 file changed, 2 insertions(+)
drivers/pci/msi.c +2 -0
@@ -1441,6 +1441,8 @@ struct irq_domain *pci_msi_create_irq_domain(struct fwnode_handle *fwnode,pci_msi_domain_update_chip_ops (info); info->flags |=MSI_FLAG_ACTIVATE_EARLY ; +if (IS_ENABLED (CONFIG_GENERIC_IRQ_RESERVATION_MODE )) + info->flags |=MSI_FLAG_MUST_REACTIVATE ; domain =msi_create_irq_domain (fwnode, info, parent);if (!domain)
PATCH 3/3 x86/vector/msi: Select CONFIG_GENERIC_IRQ_RESERVATION_MODE
In reply to Thomas GleixnerSelect CONFIG_GENERIC_IRQ_RESERVATION_MODE so PCI/MSI domains get the MSI_FLAG_MUST_REACTIVATE flag set in pci_msi_create_irq_domain().
Remove the explicit setters of this flag in the apic/msi code as they are not longer required.
--- arch/x86/Kconfig | 2 +- arch/x86/kernel/apic/msi.c | 5 ++--- 2 files changed, 3 insertions(+), 4 deletions(-)
arch/x86/Kconfig +1 -1
@@ -95,7 +95,7 @@ config X86 select GENERIC_IRQ_MATRIX_ALLOCATOR if X86_LOCAL_APIC select GENERIC_IRQ_MIGRATION if SMP select GENERIC_IRQ_PROBE - select GENERIC_IRQ_SHOW + select GENERIC_IRQ_RESERVATION_MODE select GENERIC_PENDING_IRQ if SMP select GENERIC_SMP_IDLE_THREAD select GENERIC_STRNCPY_FROM_USER
arch/x86/kernel/apic/msi.c +2 -3
@@ -129,7 +129,7 @@ static struct msi_domain_ops pci_msi_domstatic struct msi_domain_info pci_msi_domain_info = { .flags =MSI_FLAG_USE_DEF_DOM_OPS |MSI_FLAG_USE_DEF_CHIP_OPS | -MSI_FLAG_PCI_MSIX |MSI_FLAG_MUST_REACTIVATE , +MSI_FLAG_PCI_MSIX , .ops = &pci_msi_domain_ops, .chip = &pci_msi_controller, .handler = handle_edge_irq,
@@ -167,8 +167,7 @@ static struct irq_chip pci_msi_ir_controstatic struct msi_domain_info pci_msi_ir_domain_info = { .flags =MSI_FLAG_USE_DEF_DOM_OPS |MSI_FLAG_USE_DEF_CHIP_OPS | -MSI_FLAG_MULTI_PCI_MSI |MSI_FLAG_PCI_MSIX | -MSI_FLAG_MUST_REACTIVATE , +MSI_FLAG_MULTI_PCI_MSI |MSI_FLAG_PCI_MSIX , .ops = &pci_msi_domain_ops, .chip = &pci_msi_ir_controller, .handler = handle_edge_irq,
RE: [patch 3/3] x86/vector/msi: Select CONFIG_GENERIC_IRQ_RESERVATION_MODE
arch/x86/Kconfig config X86 · line 101
Quoted 13 lines
From: Thomas Gleixner [mailto:[email protected]]
Sent: Tuesday, October 17, 2017 12:55 AMarch/x86/Kconfig +1
-1@@ -95,7 +95,7 @@ config X86 select GENERIC_IRQ_MATRIX_ALLOCATOR if X86_LOCAL_APIC select GENERIC_IRQ_MIGRATION if SMP select GENERIC_IRQ_PROBE - select GENERIC_IRQ_SHOW + select GENERIC_IRQ_RESERVATION_MODE select GENERIC_PENDING_IRQ if SMP select GENERIC_SMP_IDLE_THREAD select GENERIC_STRNCPY_FROM_USER
Hi Thomas,
It looks you removed GENERIC_IRQ_SHOW by accident?
I got this build error:
...
LD vmlinux.o
MODPOST vmlinux.o
fs/proc/interrupts.o:(.rodata+0x118): undefined reference to `show_interrup= ts'
Makefile:1000: recipe for target 'vmlinux' failed
make: *** [vmlinux] Error 1
Adding GENERIC_IRQ_SHOW back can fix the build failure, and with the pachset Hyper-V PCI pass-through can work fine.
Thanks,
-- Dexuan
RE: [patch 3/3] x86/vector/msi: Select CONFIG_GENERIC_IRQ_RESERVATION_MODE
Quoted 8 lines · Dexuan Cui
Quoted 8 lines
From: Thomas Gleixner [mailto:[email protected]]
Sent: Tuesday, October 17, 2017 12:55 AMarch/x86/Kconfig +0
-0@@ -95,7 +95,7 @@ config X86 select GENERIC_IRQ_MATRIX_ALLOCATOR if X86_LOCAL_APIC select GENERIC_IRQ_MIGRATION if SMP select GENERIC_IRQ_PROBE
Quoted 8 lines
arch/x86/Kconfig config X86 · line 101
Quoted 5 lines
arch/x86/Kconfig +1
-1- select GENERIC_IRQ_SHOW + select GENERIC_IRQ_RESERVATION_MODE select GENERIC_PENDING_IRQ if SMP select GENERIC_SMP_IDLE_THREAD select GENERIC_STRNCPY_FROM_USERHi Thomas,
It looks you removedGENERIC_IRQ_SHOWby accident?
Indeed. My patch foo today is more than bad....
Quoted 10 lines
I got this build error:
...
LD vmlinux.o
MODPOST vmlinux.o
fs/proc/interrupts.o:(.rodata+0x118): undefined reference to `show_interrupts' Makefile:1000: recipe for target 'vmlinux' failed
make: *** [vmlinux] Error 1Adding
GENERIC_IRQ_SHOWback can fix the build failure, and with the pachset Hyper-V PCI pass-through can work fine.
Thanks for testing.
tglx
tip-bot for Thomas Gleixner
merge bot[tip:x86/apic] x86/vector/msi: Select CONFIG_GENERIC_IRQ_RESERVATION_MODE
In reply to Thomas Gleixnerx86/vector/msi: Select CONFIG_GENERIC_IRQ_RESERVATION_MODE
Commit-ID: c201c91799d687c0a6d8c3272950f51aad5ffebe Gitweb: https://git.kernel.org/tip/c201c91799d687c0a6d8c3272950f51aad5ffebe Author: Thomas Gleixner <[email protected]> AuthorDate: Tue, 17 Oct 2017 09:54:59 +0200 Committer: Thomas Gleixner <[email protected]> CommitDate: Wed, 18 Oct 2017 15:38:31 +0200
x86/vector/msi: Select CONFIG_GENERIC_IRQ_RESERVATION_MODE
Select CONFIG_GENERIC_IRQ_RESERVATION_MODE so PCI/MSI domains get the MSI_FLAG_MUST_REACTIVATE flag set in pci_msi_create_irq_domain().
Remove the explicit setters of this flag in the apic/msi code as they are not longer required.
--- arch/x86/Kconfig | 1 + arch/x86/kernel/apic/msi.c | 5 ++--- 2 files changed, 3 insertions(+), 3 deletions(-)
arch/x86/Kconfig +1 -0
@@ -95,6 +95,7 @@ config X86 select GENERIC_IRQ_MATRIX_ALLOCATOR if X86_LOCAL_APIC select GENERIC_IRQ_MIGRATION if SMP select GENERIC_IRQ_PROBE + select GENERIC_IRQ_RESERVATION_MODE select GENERIC_IRQ_SHOW select GENERIC_PENDING_IRQ if SMP select GENERIC_SMP_IDLE_THREAD
arch/x86/kernel/apic/msi.c +2 -3
@@ -129,7 +129,7 @@ static struct msi_domain_ops pci_msi_domain_ops = {static struct msi_domain_info pci_msi_domain_info = { .flags =MSI_FLAG_USE_DEF_DOM_OPS |MSI_FLAG_USE_DEF_CHIP_OPS | -MSI_FLAG_PCI_MSIX |MSI_FLAG_MUST_REACTIVATE , +MSI_FLAG_PCI_MSIX , .ops = &pci_msi_domain_ops, .chip = &pci_msi_controller, .handler = handle_edge_irq,
@@ -167,8 +167,7 @@ static struct irq_chip pci_msi_ir_controller = {static struct msi_domain_info pci_msi_ir_domain_info = { .flags =MSI_FLAG_USE_DEF_DOM_OPS |MSI_FLAG_USE_DEF_CHIP_OPS | -MSI_FLAG_MULTI_PCI_MSI |MSI_FLAG_PCI_MSIX | -MSI_FLAG_MUST_REACTIVATE , +MSI_FLAG_MULTI_PCI_MSI |MSI_FLAG_PCI_MSIX , .ops = &pci_msi_domain_ops, .chip = &pci_msi_ir_controller, .handler = handle_edge_irq,
Quoted 11 lines · Thomas Gleixner
Dexuan reported that the recent rework of the vector allocation mode in x86 broke HyperV PCI passtrough because the rework missed to add the
MSI_FLAG_MUST_REACTIVATEflag to the HyperV/PCI interrupt domain info.The simple solution would be to set the flag in the HyperV/PCI driver but it's better to make this generic and let the PCI/MSI core code set the flag when reservation mode is enabled. That ensures that future users of this wont trip over the same problem.
Thanks,
Quoted 8 lines
tglx
--- arch/x86/Kconfig | 2 +- arch/x86/kernel/apic/msi.c | 5 ++--- drivers/pci/msi.c | 2 ++ kernel/irq/Kconfig | 3 +++ 4 files changed, 8 insertions(+), 4 deletions(-)
This mentions 4900be83602b ("x86/vector/msi: Switch to global reservation mode"), which I don't have, so I assume it's an x86 thing. So I guess you'll probably merge this via the same tree?
Here's my ack for the PCI part:
Quoted 21 lines · Bjorn Helgaas
Quoted 11 lines · Thomas Gleixner
Dexuan reported that the recent rework of the vector allocation mode in x86 broke HyperV PCI passtrough because the rework missed to add the
MSI_FLAG_MUST_REACTIVATEflag to the HyperV/PCI interrupt domain info.The simple solution would be to set the flag in the HyperV/PCI driver but it's better to make this generic and let the PCI/MSI core code set the flag when reservation mode is enabled. That ensures that future users of this wont trip over the same problem.
Thanks,
Quoted 8 lines
tglx
--- arch/x86/Kconfig | 2 +- arch/x86/kernel/apic/msi.c | 5 ++--- drivers/pci/msi.c | 2 ++ kernel/irq/Kconfig | 3 +++ 4 files changed, 8 insertions(+), 4 deletions(-)
Quoted 3 lines
This mentions
4900be83602b("x86/vector/msi: Switch to global reservation mode"), which I don't have, so I assume it's an x86 thing. So I guess you'll probably merge this via the same tree?
Yes, it's in tip/x86/apic branch and the fixups are applied there now as well.
Thanks,
tglx