genirq, PCI/MSI: Support for per device MSI and PCI/IMS - Part 3 implementation
Status from this message: The following commit has been merged into the irq/core branch of tip:
Get series with b4
b4 am [email protected]Versions newest first · linked by author and title
- v3 Latest posting 2/233 patches 38 repliesApplied
- v3 posting 1/222 patches 11 repliesApplied
- v2 posting 2/233 patches 37 repliesApplied
Earlier versions · 2
| Version | Posted | Patches | Replies | Status |
|---|---|---|---|---|
| v2 posting 1/2 | 21 patches | 9 replies | Superseded | |
| v1 | 33 patches | 52 replies | Superseded |
Changes in v3 from the cover letter
- Rework the domain size initialization and handling (Kevin)
- Enable IMS only when on real hardware (Kevin)
- Rename the PCI/MSI irqchip functions (Kevin)
- Update change logs and comments (Kevin)
Patches in v3 posting 2/2 Applied
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/33 | genirq/msi: Rearrange MSI domain flagsinclude/ | +34 | 0 replies | |
| 2/33 | genirq/msi: Provide struct msi_parent_opsinclude/ include/ kernel/ | +67 | 0 replies | |
| 3/33 | genirq/msi: Provide data structs for per device domainsinclude/ | +15 | 0 replies | |
| 4/33 | genirq/msi: Add size info to struct msi_domain_infoinclude/ kernel/ | +16 | 0 replies | |
| 5/33 | genirq/msi: Split msi_create_irq_domain()kernel/ | +20 | 0 replies | |
| 6/33 | genirq/irqdomain: Add irq_domain::dev for per device MSI domainsinclude/ | +4 | 0 replies | |
| 7/33 | genirq/msi: Provide msi_create/free_device_irq_domain()include/ kernel/ | +144 | 0 replies | |
| 8/33 | genirq/msi: Provide msi_match_device_domain()include/ kernel/ | +28 | 0 replies | |
| 9/33 | genirq/msi: Add range checking to msi_insert_desc()kernel/ | +47 | 24 replies | |
| 10/33 | PCI/MSI: Split __pci_write_msi_msg()drivers/ | +54 | 0 replies | |
| 11/33 | genirq/msi: Provide BUS_DEVICE_PCI_MSI[X]include/ kernel/ | +6 | 0 replies | |
| 12/33 | PCI/MSI: Add support for per device MSI[X] domainsdrivers/ drivers/ drivers/ | +201 | 1 reply | |
| 13/33 | x86/apic/vector: Provide MSI parent domainarch/ arch/ arch/ +2 more | +138 | 7 replies | |
| 14/33 | PCI/MSI: Remove unused pci_dev_has_special_msi_domain()drivers/ include/ | +0 | 0 replies | |
| 15/33 | iommu/vt-d: Switch to MSI parent domainsarch/ drivers/ drivers/ +1 more | +15 | 0 replies | |
| 16/33 | iommu/amd: Switch to MSI base domainsarch/ drivers/ drivers/ +1 more | +15 | 0 replies | |
| 17/33 | x86/apic/msi: Remove arch_create_remap_msi_irq_domain()arch/ arch/ | +1 | 0 replies | |
| 18/33 | genirq/msi: Provide struct msi_mapinclude/ | +13 | 0 replies | |
| 19/33 | genirq/msi: Provide msi_desc::msi_datainclude/ include/ | +54 | 0 replies | |
| 20/33 | genirq/msi: Provide msi_domain_ops::prepare_desc()include/ kernel/ | +8 | 0 replies | |
| 21/33 | genirq/msi: Provide msi_domain_alloc_irq_at()include/ include/ kernel/ | +108 | 1 reply | |
| 22/33 | genirq/msi: Provide MSI_FLAG_MSIX_ALLOC_DYNinclude/ | +2 | 0 replies | |
| 23/33 | PCI/MSI: Split MSI-X descriptor setupdrivers/ drivers/ | +47 | 0 replies | |
| 24/33 | PCI/MSI: Provide prepare_desc() MSI domain opdrivers/ | +9 | 0 replies | |
| 25/33 | PCI/MSI: Provide post-enable dynamic allocation interfaces for MSI-Xdrivers/ drivers/ include/ | +75 | 0 replies | |
| 26/33 | x86/apic/msi: Enable MSI_FLAG_PCI_MSIX_ALLOC_DYNarch/ | +1 | 0 replies | |
| 27/33 | genirq/msi: Provide constants for PCI/IMS supportinclude/ include/ include/ | +4 | 0 replies | |
| 28/33 | PCI/MSI: Provide IMS (Interrupt Message Store) supportdrivers/ include/ | +64 | 2 replies | |
| 29/33 | PCI/MSI: Provide pci_ims_alloc/free_irq()drivers/ include/ | +53 | 1 reply | |
| 30/33 | x86/apic/msi: Enable PCI/IMSarch/ | +5 | 0 replies | |
| 31/33 | iommu/vt-d: Enable PCI/IMSdrivers/ | +16 | 0 replies | |
| 32/33 | iommu/amd: Enable PCI/IMSdrivers/ | +15 | 0 replies | |
| 33/33 | irqchip: Add IDXD Interrupt Message Store driverdrivers/ drivers/ drivers/ +1 more | +173 | 0 replies |
Reviews trailers per patch
9 of 33 patches have a Reviewed-by or Acked-by of their own; the series has one for all of them.
| Patch | Jason Gunthorpe | Bjorn Helgaas | Kevin Tian | Marc Zyngier | Kalle Valo |
|---|---|---|---|---|---|
| Series | R-b | A-b | |||
| 1/33 | R-b | ||||
| 2/33 | |||||
| 3/33 | |||||
| 4/33 | |||||
| 5/33 | |||||
| 6/33 | |||||
| 7/33 | |||||
| 8/33 | |||||
| 9/33 | |||||
| 10/33 | A-b | ||||
| 11/33 | |||||
| 12/33 | A-b | ||||
| 13/33 | T-b | ||||
| 14/33 | A-b | ||||
| 15/33 | |||||
| 16/33 | |||||
| 17/33 | |||||
| 18/33 | |||||
| 19/33 | |||||
| 20/33 | |||||
| 21/33 | |||||
| 22/33 | R-b | ||||
| 23/33 | A-b | ||||
| 24/33 | R-b | A-b | |||
| 25/33 | A-b | ||||
| 26/33 | |||||
| 27/33 | |||||
| 28/33 | |||||
| 29/33 | A-b | ||||
| 30/33 | |||||
| 31/33 | |||||
| 32/33 | |||||
| 33/33 |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 5 reviewers.
Cover letter [patch V3 00/33] genirq, PCI/MSI: Support for per device MSI and PCI/IMS - Part 3 implementation
This is V3 of the third part of the effort to provide support for per device MSI interrupt domains.
Version 2 of this part can be found here:
https://lore.kernel.org/all/[email protected]
This is based on the second part which is available here:
https://lore.kernel.org/all/[email protected]
and from git:
git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git devmsi-v3-part2
This third part is available from git too:
git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git devmsi-v3-part3
This part provides the actual per device domain implementation and related functionality:
1) Provide infrastructure to create and remove per device MSI domains
2) Implement per device MSI domains in the PCI/MSI code and make them conditional on the availability of a suitable parent MSI domain. This allows to convert the existing domains one by one and keeps both the legacy and the current "global" PCI/MSI domain model working.
3) Convert the related x86 MSI domains over (vector and remapping).
4) Provide core infrastructure for dynamic allocations
5) Provide PCI/MSI-X interfaces for device drivers to do post MSI-X enable allocation/free
6) Enable dynamic allocation support on the x86 MSI parent domains
7) Provide infrastructure to create PCI/IMS domains
8) Enable IMS support on the x86 MSI parent domains
9) Provide a driver for IDXD which demonstrates how IMS domains look like.
Changes vs. v2:
- Rework the domain size initialization and handling (Kevin)
- Enable IMS only when on real hardware (Kevin)
- Rename the PCI/MSI irqchip functions (Kevin)
- Update change logs and comments (Kevin)
The delta patch vs. V3 is attached below. It's not completely accurate as it has some changes from part 2 intermingled, but you get the idea.
Thanks,
tglx
---
drivers/iommu/amd/iommu.c +12 -1
@@ -3656,6 +3656,13 @@ static const struct msi_parent_ops amdvi_msi_parent_ops = { .init_dev_msi_info = msi_parent_init_dev_msi_info, }; +static const struct msi_parent_ops virt_amdvi_msi_parent_ops = { + .supported_flags =X86_VECTOR_MSI_FLAGS_SUPPORTED | +MSI_FLAG_MULTI_PCI_MSI , + .prefix ="vIR-" , + .init_dev_msi_info = msi_parent_init_dev_msi_info, +}; +int amd_iommu_create_irq_domain (struct amd_iommu *iommu) {struct fwnode_handle *fn;
@@ -3672,7 +3679,11 @@ int amd_iommu_create_irq_domain(struct amd_iommu *iommu)irq_domain_update_bus_token (iommu->ir_domain,DOMAIN_BUS_AMDVI ); iommu->ir_domain->flags |=IRQ_DOMAIN_FLAG_MSI_PARENT ; - iommu->ir_domain->msi_parent_ops = &amdvi_msi_parent_ops; + +if (amd_iommu_np_cache) + iommu->ir_domain->msi_parent_ops = &virt_amdvi_msi_parent_ops; +else + iommu->ir_domain->msi_parent_ops = &amdvi_msi_parent_ops;return 0 ; }
drivers/iommu/intel/irq_remapping.c +13 -2
@@ -82,7 +82,7 @@ static const struct irq_domain_ops intel_ir_domain_ops;static void iommu_disable_irq_remapping (struct intel_iommu *iommu);static int __init parse_ioapics_under_ir (void ); -static const struct msi_parent_ops dmar_msi_parent_ops; +static const struct msi_parent_ops dmar_msi_parent_ops, virt_dmar_msi_parent_ops;static bool ir_pre_enabled (struct intel_iommu *iommu) {
@@ -577,7 +577,11 @@ static int intel_setup_irq_remapping(struct intel_iommu *iommu)irq_domain_update_bus_token (iommu->ir_domain,DOMAIN_BUS_DMAR ); iommu->ir_domain->flags |=IRQ_DOMAIN_FLAG_MSI_PARENT ; - iommu->ir_domain->msi_parent_ops = &dmar_msi_parent_ops; + +if (cap_caching_mode (iommu->cap)) + iommu->ir_domain->msi_parent_ops = &virt_dmar_msi_parent_ops; +else + iommu->ir_domain->msi_parent_ops = &dmar_msi_parent_ops; ir_table->base =page_address (pages); ir_table->bitmap = bitmap;
@@ -1436,6 +1440,13 @@ static const struct msi_parent_ops dmar_msi_parent_ops = { .init_dev_msi_info = msi_parent_init_dev_msi_info, }; +static const struct msi_parent_ops virt_dmar_msi_parent_ops = { + .supported_flags =X86_VECTOR_MSI_FLAGS_SUPPORTED | +MSI_FLAG_MULTI_PCI_MSI , + .prefix ="vIR-" , + .init_dev_msi_info = msi_parent_init_dev_msi_info, +}; +/* * Support of Interrupt Remapping Unit Hotplug */
drivers/irqchip/Kconfig +1 -1
@@ -699,7 +699,7 @@ config PCI_INTEL_IDXD_IMS tristate "Intel IDXD Interrupt Message Store controller" depends on PCI_MSI help - Support for Intel IDXD IMS Interrupt Message Store controller + Support for Intel IDXD Interrupt Message Store (IMS) controller with IMS slot storage in a slot array in device memory endmenu
drivers/irqchip/irq-pci-intel-idxd.c +1 -1
@@ -128,7 +128,7 @@ static const struct msi_domain_template idxd_ims_template = {/** * pci_intel_idxd_create_ims_domain - Create a IDXD IMS domain * @pdev: IDXD PCI device to operate on -* @slots: Pointer to the mapped slot memory arrray +* @slots: Pointer to the mapped slot memory array * @nr_slots: The number of slots in the array * * Returns: True on success, false otherwise
drivers/pci/msi/api.c +2 -2
@@ -141,7 +141,7 @@ EXPORT_SYMBOL_GPL(pci_msix_can_alloc_dyn);* Return: A struct msi_map * * On success msi_map::index contains the allocated index (>= 0) and -* msi_map::virq the allocated Linux interrupt number (> 0). +* msi_map::virq contains the allocated Linux interrupt number (> 0). * * On fail msi_map::index contains the error code and msi_map::virq * is set to 0.
@@ -376,7 +376,7 @@ EXPORT_SYMBOL(pci_irq_get_affinity);* the index - if there is a hardware table - or in case of purely software * managed IMS implementation the association happens via the * irq_write_msi_msg() callback of the implementation specific interrupt -* chip, which utilizes the provided @cookie to store the MSI message in +* chip, which utilizes the provided @icookie to store the MSI message in * the appropriate place. * * Return: A struct msi_map
drivers/pci/msi/irqdomain.c +9 -9
@@ -148,14 +148,14 @@ static void pci_device_domain_set_desc(msi_alloc_info_t *arg, struct msi_desc *d arg->hwirq = desc->msi_index; } -static void pci_mask_msi (struct irq_data *data) +static void pci_irq_mask_msi (struct irq_data *data) {struct msi_desc *desc =irq_data_get_msi_desc (data);pci_msi_mask (desc,BIT (data->irq - desc->irq)); } -static void pci_unmask_msi (struct irq_data *data) +static void pci_irq_unmask_msi (struct irq_data *data) {struct msi_desc *desc =irq_data_get_msi_desc (data);
@@ -176,8 +176,8 @@ static void pci_unmask_msi(struct irq_data *data)static struct msi_domain_template pci_msi_template = { .chip = { .name ="PCI-MSI" , - .irq_mask = pci_mask_msi, - .irq_unmask = pci_unmask_msi, + .irq_mask = pci_irq_mask_msi, + .irq_unmask = pci_irq_unmask_msi, .irq_write_msi_msg = pci_msi_domain_write_msg, .flags =IRQCHIP_ONESHOT_SAFE , },
@@ -192,12 +192,12 @@ static struct msi_domain_template pci_msi_template = { }, }; -static void pci_mask_msix (struct irq_data *data) +static void pci_irq_mask_msix (struct irq_data *data) {pci_msix_mask (irq_data_get_msi_desc (data)); } -static void pci_unmask_msix (struct irq_data *data) +static void pci_irq_unmask_msix (struct irq_data *data) {pci_msix_unmask (irq_data_get_msi_desc (data)); }
@@ -213,8 +213,8 @@ static void pci_msix_prepare_desc(struct irq_domain *domain, msi_alloc_info_t *astatic struct msi_domain_template pci_msix_template = { .chip = { .name ="PCI-MSIX" , - .irq_mask = pci_mask_msix, - .irq_unmask = pci_unmask_msix, + .irq_mask = pci_irq_mask_msix, + .irq_unmask = pci_irq_unmask_msix, .irq_write_msi_msg = pci_msi_domain_write_msg, .flags =IRQCHIP_ONESHOT_SAFE , },
@@ -302,7 +302,7 @@ bool pci_setup_msi_device_domain(struct pci_dev *pdev)*/ bool pci_setup_msix_device_domain (struct pci_dev *pdev,unsigned int hwsize) { -if (WARN_ON_ONCE (pdev->msix_enabled)) +if (WARN_ON_ONCE (pdev->msi_enabled))return false;if (pci_match_device_domain (pdev,DOMAIN_BUS_PCI_DEVICE_MSIX ))
include/linux/msi.h +4 -1
@@ -455,7 +463,10 @@ struct msi_domain_ops {* struct msi_domain_info - MSI interrupt domain data * @flags: Flags to decribe features and capabilities * @bus_token: The domain bus token -* @hwsize: The hardware table size (0 if unknown/unlimited) +* @hwsize: The hardware table size or the software index limit. +* If 0 then the size is considered unlimited and +* gets initialized to the maximum software index limit +* by the domain creation code. * @ops: The callback data structure * @chip: Optional: associated interrupt chip * @chip_data: Optional: associated interrupt chip data
kernel/irq/msi.c +53 -44
219 lines not on this page · raw message