MSI: Track device proxying when allocating MSIs
Subsystems (4)
Marc Zyngier
maintainercover MSI: Track device proxying when allocating MSIs
This series aims to plug a gap found with John's "managed interrupts for platform devices" series.
We have a couple of cases where we allocate MSIs for a device, but the endpoint generating the actual interrupts (let's call it the "logical device") isn't the one being seen generating the MSIs (the "effective device"):
- PCI devices behind a non transparent bridge
- Devices attached to a wire-to-MSI bridge
For interrupt infrastructures such as the ITS (where the effective device ID is part of the translation), it is important decouple the lifetime of the effective device mapping from the interrupt allocation on behalf of the logical device. Yes, I have a headache too.
To achieve this split, let's tag the such allocations with a new flag that convey the fact that the effective device isn't the logical one. This mostly affects the platform-MSI subsystem.
Subsequent patches add handling of this new flag in the GICv3 ITS (though there may be scope for something more generic in the case of the last patch).
Marc Zyngier (3):
platform-msi: Track shared domain allocation irqchip/gic-v3-its: Tag ITS device as shared if allocating for a proxy
device
irqchip/gic-v3-its: Flag device allocation as proxied if behind a PCI
bridge
drivers/base/platform-msi.c | 7 +++++++ drivers/irqchip/irq-gic-v3-its-pci-msi.c | 11 ++++++++--- drivers/irqchip/irq-gic-v3-its.c | 3 +++ include/asm-generic/msi.h | 4 ++++ 4 files changed, 22 insertions(+), 3 deletions(-)
Signature · 1 line
2.28.0
Marc Zyngier
maintainerPATCH 1/3 platform-msi: Track shared domain allocation
We have two flavours of platform-MSI:
- MSIs generated by devices for themselves (the usual case)
- MSIs generated on behalf of other devices, as the generating device is some form of bridge (either a wire-to-MSI bridge, or even a non-transparent PCI bridge that repaints the PCI requester ID).
In the latter case, the underlying interrupt architecture may need to track this in order to keep the mapping alive even when no MSI are currently being generated.
Add a set of flags to the generic msi_alloc_info_t structure, as well as the MSI_ALLOC_FLAGS_PROXY_DEVICE flag that will get advertized by the platform-MSI code when allocating an irqdomain for a device.
--- drivers/base/platform-msi.c | 7 +++++++ include/asm-generic/msi.h | 4 ++++ 2 files changed, 11 insertions(+)
drivers/base/platform-msi.c +7 -0
@@ -59,9 +59,15 @@ static int platform_msi_init(struct irq_domain *domain,return irq_domain_set_hwirq_and_chip (domain, virq, hwirq, info->chip, info->chip_data); } + +static void platform_msi_set_proxy_dev (msi_alloc_info_t *arg) +{ + arg->flags |=MSI_ALLOC_FLAGS_PROXY_DEVICE ; +}#else #define platform_msi_set_desc NULL#define platform_msi_init NULL +#define platform_msi_set_proxy_dev (x) do {} while(0)#endif static void platform_msi_update_dom_ops (struct msi_domain_info *info)
@@ -343,6 +349,7 @@ __platform_msi_create_device_domain(struct device *dev,if (!domain) goto free_priv; +platform_msi_set_proxy_dev (&data->arg); err =msi_domain_prepare_irqs (domain->parent, dev, nvec, &data->arg);if (err) goto free_domain;
include/asm-generic/msi.h +4 -0
@@ -22,12 +22,16 @@ struct msi_desc;typedef struct msi_alloc_info {struct msi_desc *desc;irq_hw_number_t hwirq; +unsigned long flags;union {unsigned long ul;void *ptr; } scratchpad[NUM_MSI_ALLOC_SCRATCHPAD_REGS ]; }msi_alloc_info_t ; +/* Device generating MSIs is proxying for another device */ +#define MSI_ALLOC_FLAGS_PROXY_DEVICE (1UL << 0) +#define GENERIC_MSI_DOMAIN_OPS 1#endif
Signature · 1 line
2.28.0
Marc Zyngier
maintainerPATCH 2/3 irqchip/gic-v3-its: Tag ITS device as shared if allocating for a proxy device
In reply to Marc ZyngierThe ITS already has some notion of "shared" devices. Let's map the MSI_ALLOC_FLAGS_PROXY_DEVICE flag onto this internal property.
--- drivers/irqchip/irq-gic-v3-its.c | 3 +++ 1 file changed, 3 insertions(+)
drivers/irqchip/irq-gic-v3-its.c +3 -0
@@ -3488,6 +3488,9 @@ static int its_msi_prepare(struct irq_domain *domain, struct device *dev, goto out; } +if (info->flags &MSI_ALLOC_FLAGS_PROXY_DEVICE ) + its_dev->shared = true; +pr_debug ("ITT %d entries, %d bits\n" , nvec,ilog2 (nvec)); out:mutex_unlock (&its->dev_alloc_lock);
Signature · 1 line
2.28.0
Marc Zyngier
maintainerPATCH 3/3 irqchip/gic-v3-its: Flag device allocation as proxied if behind a PCI bridge
In reply to Marc ZyngierAn aliasing PCI bridge is another case where we should flag the corresponding allocation as "proxied", as MSIs are coming with the bridge's RID, and not the originating device's.
--- drivers/irqchip/irq-gic-v3-its-pci-msi.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-)
drivers/irqchip/irq-gic-v3-its-pci-msi.c +8 -3
@@ -67,11 +67,16 @@ static int its_pci_msi_prepare(struct irq_domain *domain, struct device *dev,/* * If pdev is downstream of any aliasing bridges, take an upper * bound of how many other vectors could map to the same DevID. +* Also tell the ITS that the signalling will come from a proxy +* device, and that special allocation rules apply. */ pci_for_each_dma_alias (pdev, its_get_pci_alias, &alias_dev); -if (alias_dev != pdev && alias_dev->subordinate) -pci_walk_bus (alias_dev->subordinate, its_pci_msi_vec_count, - &alias_count); +if (alias_dev != pdev) { +if (alias_dev->subordinate) +pci_walk_bus (alias_dev->subordinate, + its_pci_msi_vec_count, &alias_count); + info->flags |=MSI_ALLOC_FLAGS_PROXY_DEVICE ; + }/* ITS specific DeviceID, as the core ITS ignores dev. */ info->scratchpad[0 ].ul =pci_msi_domain_get_msi_rid (domain, pdev);
Signature · 1 line
2.28.0
Quoted 16 lines · Marc Zyngier
This series aims to plug a gap found with John's "managed interrupts for platform devices" series.
We have a couple of cases where we allocate MSIs for a device, but the endpoint generating the actual interrupts (let's call it the "logical device") isn't the one being seen generating the MSIs (the "effective device"):
- PCI devices behind a non transparent bridge
- Devices attached to a wire-to-MSI bridgeFor interrupt infrastructures such as the ITS (where the effective device ID is part of the translation), it is important decouple the lifetime of the effective device mapping from the interrupt allocation on behalf of the logical device. Yes, I have a headache too.
Quoted 7 lines
To achieve this split, let's tag the such allocations with a new flag that convey the fact that the effective device isn't the logical one. This mostly affects the platform-MSI subsystem.
Subsequent patches add handling of this new flag in the GICv3 ITS (though there may be scope for something more generic in the case of the last patch).
Hi Marc,
Thanks!
Quoted 14 lines
Marc Zyngier (3):
platform-msi: Track shared domain allocation irqchip/gic-v3-its: Tag ITS device as shared if allocating for a proxy
device
irqchip/gic-v3-its: Flag device allocation as proxied if behind a PCI
bridgedrivers/base/platform-msi.c | 7 +++++++ drivers/irqchip/irq-gic-v3-its-pci-msi.c | 11 ++++++++--- drivers/irqchip/irq-gic-v3-its.c | 3 +++ include/asm-generic/msi.h | 4 ++++ 4 files changed, 22 insertions(+), 3 deletions(-)
[irqchip: irq/irqchip-next] irqchip/gic-v3-its: Tag ITS device as shared if allocating for a proxy device
In reply to Marc ZyngierThe following commit has been merged into the irq/irqchip-next branch of irqchip:
The following commit has been merged into the irq/irqchip-next branch of irqchip:
Commit-ID: 5fe71d271df8c05e1060c0184764eba18b17a96f Gitweb: https://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms/5fe71d271df8c05e1060c0184764eba18b17a96f Author: Marc Zyngier <[email protected]> AuthorDate: Sun, 29 Nov 2020 13:52:07 Committer: Marc Zyngier <[email protected]> CommitterDate: Fri, 11 Dec 2020 14:47:50
irqchip/gic-v3-its: Tag ITS device as shared if allocating for a proxy device
The ITS already has some notion of "shared" devices. Let's map the MSI_ALLOC_FLAGS_PROXY_DEVICE flag onto this internal property.
--- drivers/irqchip/irq-gic-v3-its.c | 3 +++ 1 file changed, 3 insertions(+)
drivers/irqchip/irq-gic-v3-its.c +3 -0
@@ -3487,6 +3487,9 @@ static int its_msi_prepare(struct irq_domain *domain, struct device *dev, goto out; } +if (info->flags &MSI_ALLOC_FLAGS_PROXY_DEVICE ) + its_dev->shared = true; +pr_debug ("ITT %d entries, %d bits\n" , nvec,ilog2 (nvec)); out:mutex_unlock (&its->dev_alloc_lock);
[irqchip: irq/irqchip-next] irqchip/gic-v3-its: Flag device allocation as proxied if behind a PCI bridge
In reply to Marc ZyngierThe following commit has been merged into the irq/irqchip-next branch of irqchip:
The following commit has been merged into the irq/irqchip-next branch of irqchip:
Commit-ID: 34dd263fce3114147f21698f8e55e05b9e8185bd Gitweb: https://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms/34dd263fce3114147f21698f8e55e05b9e8185bd Author: Marc Zyngier <[email protected]> AuthorDate: Sun, 29 Nov 2020 13:52:08 Committer: Marc Zyngier <[email protected]> CommitterDate: Fri, 11 Dec 2020 14:47:50
irqchip/gic-v3-its: Flag device allocation as proxied if behind a PCI bridge
An aliasing PCI bridge is another case where we should flag the corresponding allocation as "proxied", as MSIs are coming with the bridge's RID, and not the originating device's.
--- drivers/irqchip/irq-gic-v3-its-pci-msi.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-)
drivers/irqchip/irq-gic-v3-its-pci-msi.c +8 -3
@@ -67,11 +67,16 @@ static int its_pci_msi_prepare(struct irq_domain *domain, struct device *dev,/* * If pdev is downstream of any aliasing bridges, take an upper * bound of how many other vectors could map to the same DevID. +* Also tell the ITS that the signalling will come from a proxy +* device, and that special allocation rules apply. */ pci_for_each_dma_alias (pdev, its_get_pci_alias, &alias_dev); -if (alias_dev != pdev && alias_dev->subordinate) -pci_walk_bus (alias_dev->subordinate, its_pci_msi_vec_count, - &alias_count); +if (alias_dev != pdev) { +if (alias_dev->subordinate) +pci_walk_bus (alias_dev->subordinate, + its_pci_msi_vec_count, &alias_count); + info->flags |=MSI_ALLOC_FLAGS_PROXY_DEVICE ; + }/* ITS specific DeviceID, as the core ITS ignores dev. */ info->scratchpad[0 ].ul =pci_msi_domain_get_msi_rid (domain, pdev);
[irqchip: irq/irqchip-next] platform-msi: Track shared domain allocation
In reply to Marc ZyngierThe following commit has been merged into the irq/irqchip-next branch of irqchip:
The following commit has been merged into the irq/irqchip-next branch of irqchip:
Commit-ID: 91f90daa4fb2b77db7aa25ef2e0206f2e3962665 Gitweb: https://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms/91f90daa4fb2b77db7aa25ef2e0206f2e3962665 Author: Marc Zyngier <[email protected]> AuthorDate: Sun, 29 Nov 2020 13:52:06 Committer: Marc Zyngier <[email protected]> CommitterDate: Fri, 11 Dec 2020 14:47:50
platform-msi: Track shared domain allocation
We have two flavours of platform-MSI:
- MSIs generated by devices for themselves (the usual case)
- MSIs generated on behalf of other devices, as the generating device is some form of bridge (either a wire-to-MSI bridge, or even a non-transparent PCI bridge that repaints the PCI requester ID).
In the latter case, the underlying interrupt architecture may need to track this in order to keep the mapping alive even when no MSI are currently being generated.
Add a set of flags to the generic msi_alloc_info_t structure, as well as the MSI_ALLOC_FLAGS_PROXY_DEVICE flag that will get advertized by the platform-MSI code when allocating an irqdomain for a device.
--- drivers/base/platform-msi.c | 7 +++++++ include/asm-generic/msi.h | 4 ++++ 2 files changed, 11 insertions(+)
drivers/base/platform-msi.c +7 -0
@@ -59,9 +59,15 @@ static int platform_msi_init(struct irq_domain *domain,return irq_domain_set_hwirq_and_chip (domain, virq, hwirq, info->chip, info->chip_data); } + +static void platform_msi_set_proxy_dev (msi_alloc_info_t *arg) +{ + arg->flags |=MSI_ALLOC_FLAGS_PROXY_DEVICE ; +}#else #define platform_msi_set_desc NULL#define platform_msi_init NULL +#define platform_msi_set_proxy_dev (x) do {} while(0)#endif static void platform_msi_update_dom_ops (struct msi_domain_info *info)
@@ -343,6 +349,7 @@ __platform_msi_create_device_domain(struct device *dev,if (!domain) goto free_priv; +platform_msi_set_proxy_dev (&data->arg); err =msi_domain_prepare_irqs (domain->parent, dev, nvec, &data->arg);if (err) goto free_domain;
include/asm-generic/msi.h +4 -0
@@ -22,12 +22,16 @@ struct msi_desc;typedef struct msi_alloc_info {struct msi_desc *desc;irq_hw_number_t hwirq; +unsigned long flags;union {unsigned long ul;void *ptr; } scratchpad[NUM_MSI_ALLOC_SCRATCHPAD_REGS ]; }msi_alloc_info_t ; +/* Device generating MSIs is proxying for another device */ +#define MSI_ALLOC_FLAGS_PROXY_DEVICE (1UL << 0) +#define GENERIC_MSI_DOMAIN_OPS 1#endif