Skip to content

x86, PCI, XEN, genirq ...: Prepare for device MSI

Thomas Gleixner linux-hyperv linux-iommu linux-next linux-pci linux-s390 lkml oe-kbuild-all xen-devel 2 versions – Applied

Status from this message: The following commit has been merged into the x86/irq branch of tip:

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v2 Latest46 patches 75 repliesApplied
  2. v1 RFC38 patches 32 repliesSuperseded

Changes in v2 from the cover letter

- Addressed various review comments and addressed the 0day fallout.
- Corrected the XEN logic (Jürgen)
- Make the arch fallback in PCI/MSI opt-in not opt-out (Bjorn)

- Fixed the compose MSI message inconsistency

- Ensure that the necessary flags are set for device SMI

- Make the irq bus logic work for affinity setting to prepare support for IMS storage in queue memory. It turned out to be less scary than I feared.

- Remove leftovers in iommu/intel|amd

- Reworked the IMS POC driver to cover queue storage so Jason can have a look whether that fits the needs of MLX devices.

Patches in v2 Applied

#Subject and filesSizeRepliesApplied
1/46iommu/amd: Prevent NULL pointer dereferencedrivers/iommu/amd/iommu.c+2 -21 reply
2/46x86/init: Remove unused init opsarch/x86/include/asm/mpspec.h arch/x86/include/asm/x86_init.h arch/x86/kernel/mpparse.c +1 more+4 -460 replies
3/46PCI: vmd: Dont abuse vector irqomain as parentdrivers/pci/controller/vmd.c+2 -10 replies
4/46genirq/chip: Use the first chip in irq_chip_compose_msi_msg()arch/x86/kernel/apic/msi.c kernel/irq/chip.c+14 -53 replies
5/46x86/msi: Move compose message callback where it belongsarch/x86/include/asm/apic.h arch/x86/kernel/apic/msi.c arch/x86/kernel/apic/vector.c+12 -90 replies
6/46x86/msi: Remove pointless vcpu_affinity callbackarch/x86/kernel/apic/msi.c+0 -10 replies
7/46x86/irq: Rename X86_IRQ_ALLOC_TYPE_MSI* to reflect PCI dependencyarch/x86/include/asm/hw_irq.h arch/x86/kernel/apic/msi.c drivers/iommu/amd/iommu.c +1 more+26 -260 replies
8/46x86/irq: Add allocation type for parent domain retrievalarch/x86/include/asm/hw_irq.h arch/x86/kernel/apic/io_apic.c arch/x86/kernel/apic/msi.c +3 more+15 -90 replies
9/46iommu/vt-d: Consolidate irq domain getterdrivers/iommu/intel/irq_remapping.c+24 -430 replies
10/46iommu/amd: Consolidate irq domain getterdrivers/iommu/amd/iommu.c+21 -440 replies
11/46iommu/irq_remapping: Consolidate irq domain lookuparch/x86/include/asm/irq_remapping.h arch/x86/kernel/apic/io_apic.c arch/x86/kernel/apic/msi.c +5 more+6 -400 replies
12/46x86/irq: Prepare consolidation of irq_alloc_infoarch/x86/include/asm/hw_irq.h+16 -60 replies
13/46x86/msi: Consolidate HPET allocationarch/x86/include/asm/hw_irq.h arch/x86/kernel/apic/msi.c drivers/iommu/amd/iommu.c +1 more+10 -170 replies
14/46x86/ioapic: Consolidate IOAPIC allocationarch/x86/include/asm/hw_irq.h arch/x86/kernel/apic/io_apic.c arch/x86/kernel/devicetree.c +3 more+66 -651 reply
15/46x86/irq: Consolidate DMAR irq allocationarch/x86/include/asm/hw_irq.h arch/x86/kernel/apic/msi.c+5 -114 replies
16/46x86/irq: Consolidate UV domain allocationarch/x86/include/asm/hw_irq.h arch/x86/platform/uv/uv_irq.c+20 -170 replies
17/46PCI/MSI: Rework pci_msi_domain_calc_hwirq()arch/x86/kernel/apic/msi.c drivers/pci/msi.c include/linux/msi.h+6 -81 reply
18/46x86/msi: Consolidate MSI allocationarch/x86/include/asm/hw_irq.h arch/x86/kernel/apic/msi.c drivers/iommu/amd/iommu.c +2 more+9 -171 reply
19/46x86/msi: Use generic MSI domain opsarch/x86/include/asm/msi.h arch/x86/kernel/apic/msi.c drivers/pci/controller/pci-hyperv.c +2 more+0 -352 replies
20/46x86/irq: Move apic_post_init() invocation to one placearch/x86/kernel/apic/apic.c arch/x86/kernel/apic/probe_32.c arch/x86/kernel/apic/probe_64.c+3 -60 replies
21/46x86/pci: Reducde #ifdeffery in PCI init codearch/x86/include/asm/pci_x86.h arch/x86/pci/init.c+14 -70 replies
22/46x86/irq: Initialize PCI/MSI domain at PCI init timearch/x86/include/asm/irqdomain.h arch/x86/include/asm/x86_init.h arch/x86/kernel/apic/msi.c +3 more+32 -170 replies
23/46irqdomain/msi: Provide DOMAIN_BUS_VMD_MSIinclude/linux/irqdomain.h kernel/irq/msi.c+7 -12 replies
24/46PCI: vmd: Mark VMD irqdomain with DOMAIN_BUS_VMD_MSIdrivers/pci/controller/vmd.c+6 -06 replies
25/46PCI/MSI: Provide pci_dev_has_special_msi_domain() helperdrivers/pci/msi.c include/linux/msi.h+23 -00 replies
26/46x86/xen: Make xen_msi_init() static and rename it to xen_hvm_msi_init()arch/x86/pci/xen.c+2 -20 replies
27/46x86/xen: Rework MSI teardownarch/x86/pci/xen.c+18 -51 reply
28/46x86/xen: Consolidate XEN-MSI initarch/x86/pci/xen.c+32 -191 reply
29/46irqdomain/msi: Allow to override msi_domain_alloc/free_irqs()include/linux/msi.h kernel/irq/msi.c+75 -224 replies
30/46x86/xen: Wrap XEN MSI management into irqdomainarch/x86/pci/xen.c+63 -02 replies
31/46iommm/vt-d: Store irq domain in struct devicedrivers/iommu/intel/dmar.c drivers/iommu/intel/irq_remapping.c include/linux/intel-iommu.h+26 -00 replies
32/46iommm/amd: Store irq domain in struct devicedrivers/iommu/amd/iommu.c+16 -10 replies
33/46x86/pci: Set default irq domain in pcibios_add_device()arch/x86/include/asm/irqdomain.h arch/x86/kernel/apic/msi.c arch/x86/pci/common.c+20 -20 replies
34/46PCI/MSI: Make arch_.*_msi_irq[s] fallbacks selectablearch/ia64/Kconfig arch/mips/Kconfig arch/powerpc/Kconfig +7 more+40 -613 replies
35/46x86/irq: Cleanup the arch_*_msi_irqs() leftoversarch/x86/Kconfig arch/x86/include/asm/pci.h arch/x86/include/asm/x86_init.h +3 more+0 -600 replies
36/46x86/irq: Make most MSI ops XEN privatearch/x86/include/asm/x86_init.h arch/x86/pci/xen.c+14 -90 replies
37/46iommu/vt-d: Remove domain search for PCI/MSI[X]drivers/iommu/intel/irq_remapping.c+0 -30 replies
38/46iommu/amd: Remove domain search for PCI/MSIdrivers/iommu/amd/iommu.c+0 -30 replies
39/46x86/irq: Add DEV_MSI allocation typearch/x86/include/asm/hw_irq.h+1 -00 replies
40/46x86/msi: Rename and rework pci_msi_prepare() to cover non-PCI MSIarch/x86/include/asm/msi.h arch/x86/kernel/apic/msi.c drivers/pci/controller/pci-hyperv.c +1 more+28 -90 replies
41/46platform-msi: Provide default irq_chip:: Ackdrivers/base/platform-msi.c+2 -01 reply
42/46genirq/proc: Take buslock on affinity writekernel/irq/manage.c+3 -30 replies
43/46genirq/msi: Provide and use msi_domain_set_default_info_flags()drivers/pci/msi.c include/linux/msi.h kernel/irq/msi.c+26 -62 replies
44/46platform-msi: Add device MSI infrastructuredrivers/base/platform-msi.c include/linux/irqdomain.h include/linux/msi.h +1 more+160 -00 replies
45/46irqdomain/msi: Provide msi_alloc/free_store() callbacksinclude/linux/msi.h kernel/irq/msi.c+19 -00 replies
46/46irqchip: Add IMS (Interrupt Message Storm) driver - NOT FOR MERGINGdrivers/irqchip/Kconfig drivers/irqchip/Makefile drivers/irqchip/irq-ims-msi.c +1 more+458 -01 reply

Reviews trailers per patch

16 of 46 patches have a Reviewed-by or Acked-by of their own; the series has one for all of them.

PatchBjorn HelgaasJuergen GrossMarc ZyngierJon DerrickJoerg RoedelWei LiuLu BaoluItay Aveksis
SeriesA-b A-b T-b
1/46A-b R-b
2/46
3/46
4/46R-b
5/46
6/46
7/46
8/46
9/46
10/46
11/46
12/46
13/46
14/46A-b
15/46
16/46
17/46A-b A-b
18/46A-b
19/46R-b
20/46
21/46
22/46
23/46A-b A-b
24/46A-b A-b
25/46A-b
26/46R-b
27/46R-b
28/46R-b
29/46R-b
30/46A-b
31/46
32/46
33/46
34/46A-b
35/46
36/46
37/46
38/46
39/46
40/46
41/46A-b
42/46
43/46
44/46
45/46
46/46

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 8 reviewers.

Read the v2 thread

Cover letter [patch V2 00/46] x86, PCI, XEN, genirq ...: Prepare for device MSI

This is the second version of providing a base to support device MSI (non PCI based) and on top of that support for IMS (Interrupt Message Storm) based devices in a halfways architecture independent way.

The first version can be found here:

    https://lore.kernel.org/r/[email protected]

It's still a mixed bag of bug fixes, cleanups and general improvements which are worthwhile independent of device MSI.

There are quite a bunch of issues to solve:

- X86 does not use the device::msi_domain pointer for historical reasons and due to XEN, which makes it impossible to create an architecture agnostic device MSI infrastructure.

- X86 has it's own msi_alloc_info data type which is pointlessly different from the generic version and does not allow to share code.

- The logic of composing MSI messages in an hierarchy is busted at the core level and of course some (x86) drivers depend on that.

- A few minor shortcomings as usual

This series addresses that in several steps:

1) Accidental bug fixes

iommu/amd: Prevent NULL pointer dereference

2) Janitoring

x86/init: Remove unused init ops
PCI: vmd: Dont abuse vector irqomain as parent x86/msi: Remove pointless vcpu_affinity callback

3) Sanitizing the composition of MSI messages in a hierarchy

genirq/chip: Use the first chip in irq_chip_compose_msi_msg() x86/msi: Move compose message callback where it belongs

4) Simplification of the x86 specific interrupt allocation mechanism

x86/irq: Rename X86_IRQ_ALLOC_TYPE_MSI* to reflect PCI dependency x86/irq: Add allocation type for parent domain retrieval iommu/vt-d: Consolidate irq domain getter
iommu/amd: Consolidate irq domain getter iommu/irq_remapping: Consolidate irq domain lookup

5) Consolidation of the X86 specific interrupt allocation mechanism to be as close as possible to the generic MSI allocation mechanism which allows to get rid of quite a bunch of x86'isms which are pointless

x86/irq: Prepare consolidation of irq_alloc_info x86/msi: Consolidate HPET allocation x86/ioapic: Consolidate IOAPIC allocation x86/irq: Consolidate DMAR irq allocation x86/irq: Consolidate UV domain allocation PCI/MSI: Rework pci_msi_domain_calc_hwirq() x86/msi: Consolidate MSI allocation x86/msi: Use generic MSI domain ops

6) x86 specific cleanups to remove the dependency on arch_*_msi_irqs()

x86/irq: Move apic_post_init() invocation to one place x86/pci: Reducde #ifdeffery in PCI init code
x86/irq: Initialize PCI/MSI domain at PCI init time irqdomain/msi: Provide DOMAIN_BUS_VMD_MSI
PCI: vmd: Mark VMD irqdomain with DOMAIN_BUS_VMD_MSI PCI/MSI: Provide pci_dev_has_special_msi_domain() helper x86/xen: Make xen_msi_init() static and rename it to xen_hvm_msi_init() x86/xen: Rework MSI teardown
x86/xen: Consolidate XEN-MSI init
irqdomain/msi: Allow to override msi_domain_alloc/free_irqs() x86/xen: Wrap XEN MSI management into irqdomain
iommm/vt-d: Store irq domain in struct device
iommm/amd: Store irq domain in struct device
x86/pci: Set default irq domain in pcibios_add_device() PCI/MSI: Make arch_.*_msi_irq[s] fallbacks selectable x86/irq: Cleanup the arch_*_msi_irqs() leftovers
x86/irq: Make most MSI ops XEN private
iommu/vt-d: Remove domain search for PCI/MSI[X]
iommu/amd: Remove domain search for PCI/MSI

7) X86 specific preparation for device MSI

x86/irq: Add DEV_MSI allocation type
x86/msi: Rename and rework pci_msi_prepare() to cover non-PCI MSI

8) Generic device MSI infrastructure
platform-msi: Provide default irq_chip:: Ack genirq/proc: Take buslock on affinity write genirq/msi: Provide and use msi_domain_set_default_info_flags()
platform-msi: Add device MSI infrastructure irqdomain/msi: Provide msi_alloc/free_store() callbacks

9) POC of IMS (Interrupt Message Storm) irq domain and irqchip implementations for both device array and queue storage.

irqchip: Add IMS (Interrupt Message Storm) driver - NOT FOR MERGING

Changes vs. V1:

- Addressed various review comments and addressed the 0day fallout.
- Corrected the XEN logic (Jürgen)
- Make the arch fallback in PCI/MSI opt-in not opt-out (Bjorn)

- Fixed the compose MSI message inconsistency

- Ensure that the necessary flags are set for device SMI

- Make the irq bus logic work for affinity setting to prepare support for IMS storage in queue memory. It turned out to be less scary than I feared.

- Remove leftovers in iommu/intel|amd

- Reworked the IMS POC driver to cover queue storage so Jason can have a look whether that fits the needs of MLX devices.

The whole lot is also available from git:

git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git device-msi

This has been tested on Intel/AMD/KVM but lacks testing on:

- HYPERV (-ENODEV)
- VMD enabled systems (-ENODEV)
- XEN (-ENOCLUE)
- IMS (-ENODEV)

- Any non-X86 code which might depend on the broken compose MSI message logic. Marc excpects not much fallout, but agrees that we need to fix it anyway.

#1 - #3 should be applied unconditionally for obvious reasons
#4 - #6 are wortwhile cleanups which should be done independent of device MSI

#7 - #8 look promising to cleanup the platform MSI implementation
independent of #8, but I neither had cycles nor the stomach to tackle that.

#9 is obviously just for the folks interested in IMS

Thanks,

tglx
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu