genirq, PCI/MSI: Support for per device MSI and PCI/IMS - Part 1 cleanups
Subsystems (25)
[tip: irq/core] PCI/MSI: Reject MSI-X early
In reply to Thomas GleixnerThe following commit has been merged into the irq/core branch of tip:
The following commit has been merged into the irq/core branch of tip:
Commit-ID: 99f3d279765725920aa5924fa445537a20129a6f Gitweb: https://git.kernel.org/tip/99f3d279765725920aa5924fa445537a20129a6f Author: Thomas Gleixner <[email protected]> AuthorDate: Fri, 11 Nov 2022 14:55:11 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Thu, 17 Nov 2022 15:15:22 +01:00
PCI/MSI: Reject MSI-X early
Similar to PCI multi-MSI reject MSI-X enablement when a irq domain is attached to the device which does not support MSI-X.
--- drivers/pci/msi/msi.c | 4 ++++ 1 file changed, 4 insertions(+)
drivers/pci/msi/msi.c +4 -0
@@ -762,6 +762,10 @@ int __pci_enable_msix_range(struct pci_dev *dev, struct msix_entry *entries, intif (WARN_ON_ONCE (dev->msix_enabled))return -EINVAL ; +/* Check MSI-X early on irq domain enabled architectures */ +if (!pci_msi_domain_supports (dev,MSI_FLAG_PCI_MSIX ,ALLOW_LEGACY )) +return -ENOTSUPP ; +if (!pci_msi_supported (dev, nvec) || dev->current_state !=PCI_D0 )return -EINVAL ;
David Woodhouse
maintainerPATCH 1/1 x86/xen: Set MSI_FLAG_PCI_MSIX support in Xen MSI domain
In reply to Thomas GleixnerThe Xen MSI → PIRQ magic does support MSI-X, so advertise it.
(In fact it's better off with MSI-X than MSI, because it's actually broken by design for 32-bit MSI, since it puts the high bits of the PIRQ# into the high 32 bits of the MSI message address, instead of the Extended Destination ID field which is in bits 4-11.
Strictly speaking, this really fixes a much older commit 2e4386eba0c0 ("x86/xen: Wrap XEN MSI management into irqdomain") which failed to set the flag. But that never really mattered until __pci_enable_msix_range() started to check and bail out early. So in 6.2-rc we see failures e.g. to bring up networking on an Amazon EC2 m4.16xlarge instance:
[ 41.498694] ena 0000:00:03.0 (unnamed net_device) (uninitialized): Failed to enable MSI-X. irq_cnt -524 [ 41.498705] ena 0000:00:03.0: Can not reserve msix vectors [ 41.498712] ena 0000:00:03.0: Failed to enable and set the admin interrupts
Side note: This is the first bug found, and first patch tested, by running Xen guests under QEMU/KVM instead of running under actual Xen.
--- arch/x86/pci/xen.c | 1 + 1 file changed, 1 insertion(+)
arch/x86/pci/xen.c +1 -0
@@ -433,6 +433,7 @@ static struct msi_domain_ops xen_pci_msi_domain_ops = { };static struct msi_domain_info xen_pci_msi_domain_info = { + .flags =MSI_FLAG_PCI_MSIX , .ops = &xen_pci_msi_domain_ops, };
Signature · 1 line
2.34.1
[tip: x86/urgent] x86/pci/xen: Set MSI_FLAG_PCI_MSIX support in Xen MSI domain
The following commit has been merged into the x86/urgent branch of tip:
The following commit has been merged into the x86/urgent branch of tip:
Commit-ID: 0a3a58de319552525507a3497da86df039a5e4e0 Gitweb: https://git.kernel.org/tip/0a3a58de319552525507a3497da86df039a5e4e0 Author: David Woodhouse <[email protected]> AuthorDate: Sun, 15 Jan 2023 22:14:19 Committer: Thomas Gleixner <[email protected]> CommitterDate: Mon, 16 Jan 2023 20:40:44 +01:00
x86/pci/xen: Set MSI_FLAG_PCI_MSIX support in Xen MSI domain
The Xen MSI → PIRQ magic does support MSI-X, so advertise it.
(In fact it's better off with MSI-X than MSI, because it's actually broken by design for 32-bit MSI, since it puts the high bits of the PIRQ# into the high 32 bits of the MSI message address, instead of the Extended Destination ID field which is in bits 4-11.
Strictly speaking, this really fixes a much older commit 2e4386eba0c0 ("x86/xen: Wrap XEN MSI management into irqdomain") which failed to set the flag. But that never really mattered until __pci_enable_msix_range() started to check and bail out early. So in 6.2-rc we see failures e.g. to bring up networking on an Amazon EC2 m4.16xlarge instance:
[ 41.498694] ena 0000:00:03.0 (unnamed net_device) (uninitialized): Failed to enable MSI-X. irq_cnt -524 [ 41.498705] ena 0000:00:03.0: Can not reserve msix vectors [ 41.498712] ena 0000:00:03.0: Failed to enable and set the admin interrupts
Side note: This is the first bug found, and first patch tested, by running Xen guests under QEMU/KVM instead of running under actual Xen.
--- arch/x86/pci/xen.c | 1 + 1 file changed, 1 insertion(+)
arch/x86/pci/xen.c +1 -0
@@ -433,6 +433,7 @@ static struct msi_domain_ops xen_pci_msi_domain_ops = { };static struct msi_domain_info xen_pci_msi_domain_info = { + .flags =MSI_FLAG_PCI_MSIX , .ops = &xen_pci_msi_domain_ops, };
PATCH 36/39 PCI/MSI: Validate MSIX contiguous restriction early
In reply to Thomas GleixnerWith interrupt domains the sanity check for MSI-X vector validation can be done _before_ any allocation happens. The sanity check only applies to the allocation functions which have an 'entries' array argument. The entries array is filled by the caller with the requested MSI-X indicies. Some drivers have gaps in the index space which is not supported on all architectures.
The PCI/MSI irqdomain has a 'feature' bit to enforce this validation late during the allocation phase.
Just do it right away before doing any other work along with the other sanity checks on that array.
--- drivers/pci/msi/msi.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-)
drivers/pci/msi/msi.c +9 -2
@@ -725,13 +725,17 @@ static int msix_capability_init(struct preturn ret; } -static bool pci_msix_validate_entries (struct msix_entry *entries,int nvec,int hwsize) +static bool pci_msix_validate_entries (struct pci_dev *dev,struct msix_entry *entries, +int nvec,int hwsize) { +bool nogap;int i, j;if (!entries)return true; + nogap =pci_msi_domain_supports (dev,MSI_FLAG_MSIX_CONTIGUOUS ,DENY_LEGACY ); +for (i =0 ; i < nvec; i++) {/* Entry within hardware limit? */ if (entries[i].entry >= hwsize)
@@ -742,6 +746,9 @@ static bool pci_msix_validate_entries(stif (entries[i].entry == entries[j].entry)return false; } +/* Check for unsupported gaps */ +if (nogap && entries[i].entry != i) +return false; }return true; }
@@ -773,7 +780,7 @@ int __pci_enable_msix_range(struct pci_dif (hwsize <0 )return hwsize; -if (!pci_msix_validate_entries (entries, nvec, hwsize)) +if (!pci_msix_validate_entries (dev, entries, nvec, hwsize))return -EINVAL ;/* PCI_IRQ_VIRTUAL is a horrible hack! */
Re: [patch 36/39] PCI/MSI: Validate MSIX contiguous restriction early
Quoted 6 lines · Thomas Gleixner
With interrupt domains the sanity check for MSI-X vector validation can be done _before_ any allocation happens. The sanity check only applies to the allocation functions which have an 'entries' array argument. The entries array is filled by the caller with the requested MSI-X indicies. Some drivers have gaps in the index space which is not supported on all architectures.
Quoted 7 lines
The PCI/MSI irqdomain has a 'feature' bit to enforce this validation late during the allocation phase.
Just do it right away before doing any other work along with the other sanity checks on that array.
Signed-off-by Thomas Gleixner <[email protected]>
s/indicies/indices/ (commit log)
s/irqdomain/irq domain/? IIRC previous logs used "irq domain" s/MSIX/MSI-X/ (subject line)
Quoted 45 lines
--- drivers/pci/msi/msi.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-)drivers/pci/msi/msi.c +9
-2@@ -725,13 +725,17 @@ static int msix_capability_init(struct preturn ret; } -static bool pci_msix_validate_entries (struct msix_entry *entries,int nvec,int hwsize) +static bool pci_msix_validate_entries (struct pci_dev *dev,struct msix_entry *entries, +int nvec,int hwsize) { +bool nogap;int i, j;if (!entries)return true; + nogap =pci_msi_domain_supports (dev,MSI_FLAG_MSIX_CONTIGUOUS ,DENY_LEGACY ); +for (i =0 ; i < nvec; i++) {/* Entry within hardware limit? */ if (entries[i].entry >= hwsize)@@ -742,6 +746,9 @@ static bool pci_msix_validate_entries(stif (entries[i].entry == entries[j].entry)return false; } +/* Check for unsupported gaps */ +if (nogap && entries[i].entry != i) +return false; }return true; }@@ -773,7 +780,7 @@ int __pci_enable_msix_range(struct pci_dif (hwsize <0 )return hwsize; -if (!pci_msix_validate_entries (entries, nvec, hwsize)) +if (!pci_msix_validate_entries (dev, entries, nvec, hwsize))return -EINVAL ;/* PCI_IRQ_VIRTUAL is a horrible hack! */
Re: [patch 36/39] PCI/MSI: Validate MSIX contiguous restriction early
In reply to Thomas GleixnerQuoted 9 lines · Thomas Gleixner
With interrupt domains the sanity check for MSI-X vector validation can be done _before_ any allocation happens. The sanity check only applies to the allocation functions which have an 'entries' array argument. The entries array is filled by the caller with the requested MSI-X indicies. Some drivers have gaps in the index space which is not supported on all architectures.
The PCI/MSI irqdomain has a 'feature' bit to enforce this validation late during the allocation phase.
Quoted 7 lines
Just do it right away before doing any other work along with the other sanity checks on that array.
Signed-off-by Thomas Gleixner <[email protected]>--- drivers/pci/msi/msi.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-)
Jason
[tip: irq/core] PCI/MSI: Validate MSI-X contiguous restriction early
In reply to Thomas GleixnerThe following commit has been merged into the irq/core branch of tip:
The following commit has been merged into the irq/core branch of tip:
Commit-ID: 4644d22eb673f173252610a93aaaba4c2bff7b41 Gitweb: https://git.kernel.org/tip/4644d22eb673f173252610a93aaaba4c2bff7b41 Author: Thomas Gleixner <[email protected]> AuthorDate: Fri, 11 Nov 2022 14:55:12 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Thu, 17 Nov 2022 15:15:22 +01:00
PCI/MSI: Validate MSI-X contiguous restriction early
With interrupt domains the sanity check for MSI-X vector validation can be done _before_ any allocation happens. The sanity check only applies to the allocation functions which have an 'entries' array argument. The entries array is filled by the caller with the requested MSI-X indices. Some drivers have gaps in the index space which is not supported on all architectures.
The PCI/MSI irq domain has a 'feature' bit to enforce this validation late during the allocation phase.
Just do it right away before doing any other work along with the other sanity checks on that array.
--- drivers/pci/msi/msi.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-)
drivers/pci/msi/msi.c +9 -2
@@ -725,13 +725,17 @@ out_disable:return ret; } -static bool pci_msix_validate_entries (struct msix_entry *entries,int nvec,int hwsize) +static bool pci_msix_validate_entries (struct pci_dev *dev,struct msix_entry *entries, +int nvec,int hwsize) { +bool nogap;int i, j;if (!entries)return true; + nogap =pci_msi_domain_supports (dev,MSI_FLAG_MSIX_CONTIGUOUS ,DENY_LEGACY ); +for (i =0 ; i < nvec; i++) {/* Entry within hardware limit? */ if (entries[i].entry >= hwsize)
@@ -742,6 +746,9 @@ static bool pci_msix_validate_entries(struct msix_entry *entries, int nvec, intif (entries[i].entry == entries[j].entry)return false; } +/* Check for unsupported gaps */ +if (nogap && entries[i].entry != i) +return false; }return true; }
@@ -773,7 +780,7 @@ int __pci_enable_msix_range(struct pci_dev *dev, struct msix_entry *entries, intif (hwsize <0 )return hwsize; -if (!pci_msix_validate_entries (entries, nvec, hwsize)) +if (!pci_msix_validate_entries (dev, entries, nvec, hwsize))return -EINVAL ;/* PCI_IRQ_VIRTUAL is a horrible hack! */
PATCH 37/39 PCI/MSI: Remove redundant msi_check() callback
In reply to Thomas GleixnerAll these sanity checks are now done _before_ any allocation work happens. No point in doing it twice.
--- drivers/pci/msi/irqdomain.c | 48 -------------------------------------------- 1 file changed, 48 deletions(-)
drivers/pci/msi/irqdomain.c +0 -48
@@ -64,51 +64,6 @@ static irq_hw_number_t pci_msi_domain_ca (pci_domain_nr (dev->bus) &0xFFFFFFFF ) <<27 ; } -static inline bool pci_msi_desc_is_multi_msi (struct msi_desc *desc) -{ -return !desc->pci.msi_attrib.is_msix && desc->nvec_used >1 ; -} - -/** -* pci_msi_domain_check_cap - Verify that @domain supports the capabilities -* for @dev -* @domain: The interrupt domain to check -* @info: The domain info for verification -* @dev: The device to check -* -* Returns: -* 0 if the functionality is supported -* 1 if Multi MSI is requested, but the domain does not support it -* -ENOTSUPP otherwise -*/ -static int pci_msi_domain_check_cap (struct irq_domain *domain, -struct msi_domain_info *info, -struct device *dev) -{ -struct msi_desc *desc =msi_first_desc (dev,MSI_DESC_ALL ); - -/* Special handling to support __pci_enable_msi_range() */ -if (pci_msi_desc_is_multi_msi (desc) && - !(info->flags &MSI_FLAG_MULTI_PCI_MSI )) -return 1 ; - -if (desc->pci.msi_attrib.is_msix) { -if (!(info->flags &MSI_FLAG_PCI_MSIX )) -return -ENOTSUPP ; - -if (info->flags &MSI_FLAG_MSIX_CONTIGUOUS ) { -unsigned int idx =0 ; - -/* Check for gaps in the entry indices */ -msi_for_each_desc (desc, dev,MSI_DESC_ALL ) { -if (desc->msi_index != idx++) -return -ENOTSUPP ; - } - } - } -return 0 ; -} -static void pci_msi_domain_set_desc (msi_alloc_info_t *arg,struct msi_desc *desc) {
@@ -118,7 +73,6 @@ static void pci_msi_domain_set_desc(msi_static struct msi_domain_ops pci_msi_domain_ops_default = { .set_desc = pci_msi_domain_set_desc, - .msi_check = pci_msi_domain_check_cap, };static void pci_msi_domain_update_dom_ops (struct msi_domain_info *info)
@@ -130,8 +84,6 @@ static void pci_msi_domain_update_dom_op }else {if (ops->set_desc ==NULL ) ops->set_desc = pci_msi_domain_set_desc; -if (ops->msi_check ==NULL ) - ops->msi_check = pci_msi_domain_check_cap; } }
Re: [patch 37/39] PCI/MSI: Remove redundant msi_check() callback
Quoted 4 lines · Thomas Gleixner
All these sanity checks are now done _before_ any allocation work happens. No point in doing it twice.
Signed-off-by Thomas Gleixner <[email protected]>
Quoted 76 lines
--- drivers/pci/msi/irqdomain.c | 48 -------------------------------------------- 1 file changed, 48 deletions(-)drivers/pci/msi/irqdomain.c +0
-48@@ -64,51 +64,6 @@ static irq_hw_number_t pci_msi_domain_ca (pci_domain_nr (dev->bus) &0xFFFFFFFF ) <<27 ; } -static inline bool pci_msi_desc_is_multi_msi (struct msi_desc *desc) -{ -return !desc->pci.msi_attrib.is_msix && desc->nvec_used >1 ; -} - -/** -* pci_msi_domain_check_cap - Verify that @domain supports the capabilities -* for @dev -* @domain: The interrupt domain to check -* @info: The domain info for verification -* @dev: The device to check -* -* Returns: -* 0 if the functionality is supported -* 1 if Multi MSI is requested, but the domain does not support it -* -ENOTSUPP otherwise -*/ -static int pci_msi_domain_check_cap (struct irq_domain *domain, -struct msi_domain_info *info, -struct device *dev) -{ -struct msi_desc *desc =msi_first_desc (dev,MSI_DESC_ALL ); - -/* Special handling to support __pci_enable_msi_range() */ -if (pci_msi_desc_is_multi_msi (desc) && - !(info->flags &MSI_FLAG_MULTI_PCI_MSI )) -return 1 ; - -if (desc->pci.msi_attrib.is_msix) { -if (!(info->flags &MSI_FLAG_PCI_MSIX )) -return -ENOTSUPP ; - -if (info->flags &MSI_FLAG_MSIX_CONTIGUOUS ) { -unsigned int idx =0 ; - -/* Check for gaps in the entry indices */ -msi_for_each_desc (desc, dev,MSI_DESC_ALL ) { -if (desc->msi_index != idx++) -return -ENOTSUPP ; - } - } - } -return 0 ; -} -static void pci_msi_domain_set_desc (msi_alloc_info_t *arg,struct msi_desc *desc) {@@ -118,7 +73,6 @@ static void pci_msi_domain_set_desc(msi_static struct msi_domain_ops pci_msi_domain_ops_default = { .set_desc = pci_msi_domain_set_desc, - .msi_check = pci_msi_domain_check_cap, };static void pci_msi_domain_update_dom_ops (struct msi_domain_info *info)@@ -130,8 +84,6 @@ static void pci_msi_domain_update_dom_op }else {if (ops->set_desc ==NULL ) ops->set_desc = pci_msi_domain_set_desc; -if (ops->msi_check ==NULL ) - ops->msi_check = pci_msi_domain_check_cap; } }
Re: [patch 37/39] PCI/MSI: Remove redundant msi_check() callback
In reply to Thomas GleixnerQuoted 7 lines · Thomas Gleixner
All these sanity checks are now done _before_ any allocation work happens. No point in doing it twice.
Signed-off-by Thomas Gleixner <[email protected]>--- drivers/pci/msi/irqdomain.c | 48 -------------------------------------------- 1 file changed, 48 deletions(-)
Much clearer this way
Jason
[tip: irq/core] PCI/MSI: Remove redundant msi_check() callback
In reply to Thomas GleixnerThe following commit has been merged into the irq/core branch of tip:
The following commit has been merged into the irq/core branch of tip:
Commit-ID: 9c03b2589da2b2b259a960f2a325731f6b38d115 Gitweb: https://git.kernel.org/tip/9c03b2589da2b2b259a960f2a325731f6b38d115 Author: Thomas Gleixner <[email protected]> AuthorDate: Fri, 11 Nov 2022 14:55:14 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Thu, 17 Nov 2022 15:15:22 +01:00
PCI/MSI: Remove redundant msi_check() callback
All these sanity checks are now done _before_ any allocation work happens. No point in doing it twice.
--- drivers/pci/msi/irqdomain.c | 48 +------------------------------------ 1 file changed, 48 deletions(-)
drivers/pci/msi/irqdomain.c +0 -48
@@ -64,51 +64,6 @@ static irq_hw_number_t pci_msi_domain_calc_hwirq(struct msi_desc *desc) (pci_domain_nr (dev->bus) &0xFFFFFFFF ) <<27 ; } -static inline bool pci_msi_desc_is_multi_msi (struct msi_desc *desc) -{ -return !desc->pci.msi_attrib.is_msix && desc->nvec_used >1 ; -} - -/** -* pci_msi_domain_check_cap - Verify that @domain supports the capabilities -* for @dev -* @domain: The interrupt domain to check -* @info: The domain info for verification -* @dev: The device to check -* -* Returns: -* 0 if the functionality is supported -* 1 if Multi MSI is requested, but the domain does not support it -* -ENOTSUPP otherwise -*/ -static int pci_msi_domain_check_cap (struct irq_domain *domain, -struct msi_domain_info *info, -struct device *dev) -{ -struct msi_desc *desc =msi_first_desc (dev,MSI_DESC_ALL ); - -/* Special handling to support __pci_enable_msi_range() */ -if (pci_msi_desc_is_multi_msi (desc) && - !(info->flags &MSI_FLAG_MULTI_PCI_MSI )) -return 1 ; - -if (desc->pci.msi_attrib.is_msix) { -if (!(info->flags &MSI_FLAG_PCI_MSIX )) -return -ENOTSUPP ; - -if (info->flags &MSI_FLAG_MSIX_CONTIGUOUS ) { -unsigned int idx =0 ; - -/* Check for gaps in the entry indices */ -msi_for_each_desc (desc, dev,MSI_DESC_ALL ) { -if (desc->msi_index != idx++) -return -ENOTSUPP ; - } - } - } -return 0 ; -} -static void pci_msi_domain_set_desc (msi_alloc_info_t *arg,struct msi_desc *desc) {
@@ -118,7 +73,6 @@ static void pci_msi_domain_set_desc(msi_alloc_info_t *arg,static struct msi_domain_ops pci_msi_domain_ops_default = { .set_desc = pci_msi_domain_set_desc, - .msi_check = pci_msi_domain_check_cap, };static void pci_msi_domain_update_dom_ops (struct msi_domain_info *info)
@@ -130,8 +84,6 @@ static void pci_msi_domain_update_dom_ops(struct msi_domain_info *info) }else {if (ops->set_desc ==NULL ) ops->set_desc = pci_msi_domain_set_desc; -if (ops->msi_check ==NULL ) - ops->msi_check = pci_msi_domain_check_cap; } }
PATCH 38/39 genirq/msi: Remove msi_domain_ops::msi_check()
In reply to Thomas GleixnerNo more users.
--- include/linux/msi.h | 4 ---- kernel/irq/msi.c | 17 +---------------- 2 files changed, 1 insertion(+), 20 deletions(-)
include/linux/msi.h +0 -4
@@ -288,7 +288,6 @@ struct msi_domain_info;* @get_hwirq: Retrieve the resulting hw irq number * @msi_init: Domain specific init function for MSI interrupts * @msi_free: Domain specific function to free a MSI interrupts -* @msi_check: Callback for verification of the domain/info/dev data * @msi_prepare: Prepare the allocation of the interrupts in the domain * @set_desc: Set the msi descriptor for an interrupt * @domain_alloc_irqs: Optional function to override the default allocation
@@ -326,9 +325,6 @@ struct msi_domain_ops {void (*msi_free)(struct irq_domain *domain,struct msi_domain_info *info,unsigned int virq); -int (*msi_check)(struct irq_domain *domain, -struct msi_domain_info *info, -struct device *dev);int (*msi_prepare)(struct irq_domain *domain,struct device *dev,int nvec,msi_alloc_info_t *arg);
kernel/irq/msi.c +1 -16
@@ -617,17 +617,9 @@ static int msi_domain_ops_init(struct irreturn 0 ; } -static int msi_domain_ops_check (struct irq_domain *domain, -struct msi_domain_info *info, -struct device *dev) -{ -return 0 ; -} -static struct msi_domain_ops msi_domain_ops_default = { .get_hwirq = msi_domain_ops_get_hwirq, .msi_init = msi_domain_ops_init, - .msi_check = msi_domain_ops_check, .msi_prepare = msi_domain_ops_prepare, .set_desc = msi_domain_ops_set_desc, .domain_alloc_irqs = __msi_domain_alloc_irqs,
@@ -655,8 +647,6 @@ static void msi_domain_update_dom_ops(st ops->get_hwirq = msi_domain_ops_default.get_hwirq;if (ops->msi_init ==NULL ) ops->msi_init = msi_domain_ops_default.msi_init; -if (ops->msi_check ==NULL ) - ops->msi_check = msi_domain_ops_default.msi_check;if (ops->msi_prepare ==NULL ) ops->msi_prepare = msi_domain_ops_default.msi_prepare;if (ops->set_desc ==NULL )
@@ -707,13 +697,8 @@ int msi_domain_prepare_irqs(struct irq_d {struct msi_domain_info *info = domain->host_data;struct msi_domain_ops *ops = info->ops; -int ret; - - ret = ops->msi_check (domain, info, dev); -if (ret ==0 ) - ret = ops->msi_prepare (domain, dev, nvec, arg); -return ret; +return ops->msi_prepare (domain, dev, nvec, arg); } int msi_domain_populate_irqs(struct irq_domain *domain, struct device *dev,
Re: [patch 38/39] genirq/msi: Remove msi_domain_ops::msi_check()
Quoted 7 lines · Thomas Gleixner
No more users.
Signed-off-by Thomas Gleixner <[email protected]>--- include/linux/msi.h | 4 ---- kernel/irq/msi.c | 17 +---------------- 2 files changed, 1 insertion(+), 20 deletions(-)
Jason
[tip: irq/core] genirq/msi: Remove msi_domain_ops:: Msi_check()
In reply to Thomas GleixnerThe following commit has been merged into the irq/core branch of tip:
The following commit has been merged into the irq/core branch of tip:
Commit-ID: 2569f62ca473584a8ba389f455fc00805389f7da Gitweb: https://git.kernel.org/tip/2569f62ca473584a8ba389f455fc00805389f7da Author: Thomas Gleixner <[email protected]> AuthorDate: Fri, 11 Nov 2022 14:55:15 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Thu, 17 Nov 2022 15:15:22 +01:00
genirq/msi: Remove msi_domain_ops:: Msi_check()
No more users.
--- include/linux/msi.h | 4 ---- kernel/irq/msi.c | 17 +---------------- 2 files changed, 1 insertion(+), 20 deletions(-)
include/linux/msi.h +0 -4
@@ -292,7 +292,6 @@ struct msi_domain_info;* @get_hwirq: Retrieve the resulting hw irq number * @msi_init: Domain specific init function for MSI interrupts * @msi_free: Domain specific function to free a MSI interrupts -* @msi_check: Callback for verification of the domain/info/dev data * @msi_prepare: Prepare the allocation of the interrupts in the domain * @set_desc: Set the msi descriptor for an interrupt * @domain_alloc_irqs: Optional function to override the default allocation
@@ -330,9 +329,6 @@ struct msi_domain_ops {void (*msi_free)(struct irq_domain *domain,struct msi_domain_info *info,unsigned int virq); -int (*msi_check)(struct irq_domain *domain, -struct msi_domain_info *info, -struct device *dev);int (*msi_prepare)(struct irq_domain *domain,struct device *dev,int nvec,msi_alloc_info_t *arg);
kernel/irq/msi.c +1 -16
@@ -617,17 +617,9 @@ static int msi_domain_ops_init(struct irq_domain *domain,return 0 ; } -static int msi_domain_ops_check (struct irq_domain *domain, -struct msi_domain_info *info, -struct device *dev) -{ -return 0 ; -} -static struct msi_domain_ops msi_domain_ops_default = { .get_hwirq = msi_domain_ops_get_hwirq, .msi_init = msi_domain_ops_init, - .msi_check = msi_domain_ops_check, .msi_prepare = msi_domain_ops_prepare, .set_desc = msi_domain_ops_set_desc, .domain_alloc_irqs = __msi_domain_alloc_irqs,
@@ -655,8 +647,6 @@ static void msi_domain_update_dom_ops(struct msi_domain_info *info) ops->get_hwirq = msi_domain_ops_default.get_hwirq;if (ops->msi_init ==NULL ) ops->msi_init = msi_domain_ops_default.msi_init; -if (ops->msi_check ==NULL ) - ops->msi_check = msi_domain_ops_default.msi_check;if (ops->msi_prepare ==NULL ) ops->msi_prepare = msi_domain_ops_default.msi_prepare;if (ops->set_desc ==NULL )
@@ -707,13 +697,8 @@ int msi_domain_prepare_irqs(struct irq_domain *domain, struct device *dev, {struct msi_domain_info *info = domain->host_data;struct msi_domain_ops *ops = info->ops; -int ret; - - ret = ops->msi_check (domain, info, dev); -if (ret ==0 ) - ret = ops->msi_prepare (domain, dev, nvec, arg); -return ret; +return ops->msi_prepare (domain, dev, nvec, arg); } int msi_domain_populate_irqs(struct irq_domain *domain, struct device *dev,
PATCH 39/39 x86/apic: Remove X86_IRQ_ALLOC_CONTIGUOUS_VECTORS
In reply to Thomas GleixnerNow that the PCI/MSI core code does early checking for multi-MSI support X86_IRQ_ALLOC_CONTIGUOUS_VECTORS is not required anymore.
Remove the flag and rely on MSI_FLAG_MULTI_PCI_MSI.
--- arch/x86/include/asm/irqdomain.h | 4 +--- arch/x86/kernel/apic/msi.c | 6 ++---- arch/x86/kernel/apic/vector.c | 4 ---- drivers/iommu/amd/iommu.c | 7 ------- drivers/iommu/intel/irq_remapping.c | 7 ------- drivers/pci/controller/pci-hyperv.c | 15 +-------------- 6 files changed, 4 insertions(+), 39 deletions(-)
arch/x86/include/asm/irqdomain.h +1 -3
@@ -7,9 +7,7 @@#ifdef CONFIG_X86_LOCAL_APIC enum { -/* Allocate contiguous CPU vectors */ -X86_IRQ_ALLOC_CONTIGUOUS_VECTORS =0x1 , -X86_IRQ_ALLOC_LEGACY =0x2 , +X86_IRQ_ALLOC_LEGACY =0x1 , };extern int x86_fwspec_is_ioapic (struct irq_fwspec *fwspec);
arch/x86/kernel/apic/msi.c +2 -4
@@ -161,12 +161,10 @@ int pci_msi_prepare(struct irq_domain *d msi_alloc_info_t *arg) {init_irq_alloc_info (arg,NULL ); -if (to_pci_dev (dev)->msix_enabled) { +if (to_pci_dev (dev)->msix_enabled) arg->type =X86_IRQ_ALLOC_TYPE_PCI_MSIX ; - }else { +else arg->type =X86_IRQ_ALLOC_TYPE_PCI_MSI ; - arg->flags |=X86_IRQ_ALLOC_CONTIGUOUS_VECTORS ; - }return 0 ; }
arch/x86/kernel/apic/vector.c +0 -4
@@ -539,10 +539,6 @@ static int x86_vector_alloc_irqs(structif (disable_apic)return -ENXIO ; -/* Currently vector allocator can't guarantee contiguous allocations */ -if ((info->flags &X86_IRQ_ALLOC_CONTIGUOUS_VECTORS ) && nr_irqs >1 ) -return -ENOSYS ; -/* * Catch any attempt to touch the cascade interrupt on a PIC * equipped system.
drivers/iommu/amd/iommu.c +0 -7
@@ -3297,13 +3297,6 @@ static int irq_remapping_alloc(struct irif (nr_irqs >1 && info->type !=X86_IRQ_ALLOC_TYPE_PCI_MSI )return -EINVAL ; -/* -* With IRQ remapping enabled, don't need contiguous CPU vectors -* to support multiple MSI interrupts. -*/ -if (info->type ==X86_IRQ_ALLOC_TYPE_PCI_MSI ) - info->flags &= ~X86_IRQ_ALLOC_CONTIGUOUS_VECTORS ; - sbdf =get_devid (info);if (sbdf <0 )return -EINVAL ;
drivers/iommu/intel/irq_remapping.c +0 -7
@@ -1337,13 +1337,6 @@ static int intel_irq_remapping_alloc(strif (nr_irqs >1 && info->type !=X86_IRQ_ALLOC_TYPE_PCI_MSI )return -EINVAL ; -/* -* With IRQ remapping enabled, don't need contiguous CPU vectors -* to support multiple MSI interrupts. -*/ -if (info->type ==X86_IRQ_ALLOC_TYPE_PCI_MSI ) - info->flags &= ~X86_IRQ_ALLOC_CONTIGUOUS_VECTORS ; - ret =irq_domain_alloc_irqs_parent (domain, virq, nr_irqs, arg);if (ret <0 )return ret;
drivers/pci/controller/pci-hyperv.c +1 -14
@@ -611,20 +611,7 @@ static unsigned int hv_msi_get_int_vectoreturn cfg->vector; } -static int hv_msi_prepare (struct irq_domain *domain,struct device *dev, -int nvec,msi_alloc_info_t *info) -{ -int ret =pci_msi_prepare (domain, dev, nvec, info); - -/* -* By using the interrupt remapper in the hypervisor IOMMU, contiguous -* CPU vectors is not needed for multi-MSI -*/ -if (info->type ==X86_IRQ_ALLOC_TYPE_PCI_MSI ) - info->flags &= ~X86_IRQ_ALLOC_CONTIGUOUS_VECTORS ; - -return ret; -} +#define hv_msi_prepare pci_msi_prepare/** * hv_arch_irq_unmask() - "Unmask" the IRQ by setting its current
Re: [patch 39/39] x86/apic: Remove X86_IRQ_ALLOC_CONTIGUOUS_VECTORS
Quoted 4 lines · Thomas Gleixner
Now that the PCI/MSI core code does early checking for multi-MSI support
X86_IRQ_ALLOC_CONTIGUOUS_VECTORSis not required anymore.Remove the flag and rely on
MSI_FLAG_MULTI_PCI_MSI.
Jason
Re: [patch 39/39] x86/apic: Remove X86_IRQ_ALLOC_CONTIGUOUS_VECTORS
Jason, Bjorn, Ashok!
Quoted 7 lines · Jason Gunthorpe
Quoted 4 lines · Thomas Gleixner
Now that the PCI/MSI core code does early checking for multi-MSI support
X86_IRQ_ALLOC_CONTIGUOUS_VECTORSis not required anymore.Remove the flag and rely on
MSI_FLAG_MULTI_PCI_MSI.Reviewed-by Jason Gunthorpe <[email protected]>
Thanks for taking the time to go through this pile!
tglx
[tip: irq/core] x86/apic: Remove X86_IRQ_ALLOC_CONTIGUOUS_VECTORS
In reply to Thomas GleixnerThe following commit has been merged into the irq/core branch of tip:
The following commit has been merged into the irq/core branch of tip:
Commit-ID: d474d92d70250d43e7ce0c7cb8623f31ee7c40f6 Gitweb: https://git.kernel.org/tip/d474d92d70250d43e7ce0c7cb8623f31ee7c40f6 Author: Thomas Gleixner <[email protected]> AuthorDate: Fri, 11 Nov 2022 14:55:17 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Thu, 17 Nov 2022 15:15:22 +01:00
x86/apic: Remove X86_IRQ_ALLOC_CONTIGUOUS_VECTORS
Now that the PCI/MSI core code does early checking for multi-MSI support X86_IRQ_ALLOC_CONTIGUOUS_VECTORS is not required anymore.
Remove the flag and rely on MSI_FLAG_MULTI_PCI_MSI.
--- arch/x86/include/asm/irqdomain.h | 4 +--- arch/x86/kernel/apic/msi.c | 6 ++---- arch/x86/kernel/apic/vector.c | 4 ---- drivers/iommu/amd/iommu.c | 7 ------- drivers/iommu/intel/irq_remapping.c | 7 ------- drivers/pci/controller/pci-hyperv.c | 15 +-------------- 6 files changed, 4 insertions(+), 39 deletions(-)
arch/x86/include/asm/irqdomain.h +1 -3
@@ -7,9 +7,7 @@#ifdef CONFIG_X86_LOCAL_APIC enum { -/* Allocate contiguous CPU vectors */ -X86_IRQ_ALLOC_CONTIGUOUS_VECTORS =0x1 , -X86_IRQ_ALLOC_LEGACY =0x2 , +X86_IRQ_ALLOC_LEGACY =0x1 , };extern int x86_fwspec_is_ioapic (struct irq_fwspec *fwspec);
arch/x86/kernel/apic/msi.c +2 -4
@@ -161,12 +161,10 @@ int pci_msi_prepare(struct irq_domain *domain, struct device *dev, int nvec, msi_alloc_info_t *arg) {init_irq_alloc_info (arg,NULL ); -if (to_pci_dev (dev)->msix_enabled) { +if (to_pci_dev (dev)->msix_enabled) arg->type =X86_IRQ_ALLOC_TYPE_PCI_MSIX ; - }else { +else arg->type =X86_IRQ_ALLOC_TYPE_PCI_MSI ; - arg->flags |=X86_IRQ_ALLOC_CONTIGUOUS_VECTORS ; - }return 0 ; }
arch/x86/kernel/apic/vector.c +0 -4
@@ -539,10 +539,6 @@ static int x86_vector_alloc_irqs(struct irq_domain *domain, unsigned int virq,if (disable_apic)return -ENXIO ; -/* Currently vector allocator can't guarantee contiguous allocations */ -if ((info->flags &X86_IRQ_ALLOC_CONTIGUOUS_VECTORS ) && nr_irqs >1 ) -return -ENOSYS ; -/* * Catch any attempt to touch the cascade interrupt on a PIC * equipped system.
drivers/iommu/amd/iommu.c +0 -7
@@ -3297,13 +3297,6 @@ static int irq_remapping_alloc(struct irq_domain *domain, unsigned int virq,if (nr_irqs >1 && info->type !=X86_IRQ_ALLOC_TYPE_PCI_MSI )return -EINVAL ; -/* -* With IRQ remapping enabled, don't need contiguous CPU vectors -* to support multiple MSI interrupts. -*/ -if (info->type ==X86_IRQ_ALLOC_TYPE_PCI_MSI ) - info->flags &= ~X86_IRQ_ALLOC_CONTIGUOUS_VECTORS ; - sbdf =get_devid (info);if (sbdf <0 )return -EINVAL ;
drivers/iommu/intel/irq_remapping.c +0 -7
@@ -1337,13 +1337,6 @@ static int intel_irq_remapping_alloc(struct irq_domain *domain,if (nr_irqs >1 && info->type !=X86_IRQ_ALLOC_TYPE_PCI_MSI )return -EINVAL ; -/* -* With IRQ remapping enabled, don't need contiguous CPU vectors -* to support multiple MSI interrupts. -*/ -if (info->type ==X86_IRQ_ALLOC_TYPE_PCI_MSI ) - info->flags &= ~X86_IRQ_ALLOC_CONTIGUOUS_VECTORS ; - ret =irq_domain_alloc_irqs_parent (domain, virq, nr_irqs, arg);if (ret <0 )return ret;
drivers/pci/controller/pci-hyperv.c +1 -14
@@ -611,20 +611,7 @@ static unsigned int hv_msi_get_int_vector(struct irq_data *data)return cfg->vector; } -static int hv_msi_prepare (struct irq_domain *domain,struct device *dev, -int nvec,msi_alloc_info_t *info) -{ -int ret =pci_msi_prepare (domain, dev, nvec, info); - -/* -* By using the interrupt remapper in the hypervisor IOMMU, contiguous -* CPU vectors is not needed for multi-MSI -*/ -if (info->type ==X86_IRQ_ALLOC_TYPE_PCI_MSI ) - info->flags &= ~X86_IRQ_ALLOC_CONTIGUOUS_VECTORS ; - -return ret; -} +#define hv_msi_prepare pci_msi_prepare/** * hv_arch_irq_unmask() - "Unmask" the IRQ by setting its current
Quoted 9 lines
From: Thomas Gleixner <[email protected]>
Sent: Friday, November 11, 2022 9:54 PMEnough of history and theory. Here comes part 1:
This is just a cleanup and a reorganisation of the PCI/MSI code which became quite an unreadable mess over time. There is no intentional functional change in this series.
Quoted 6 lines
It's just a separate step to make the subsequent changes in the infrastructure easier both to implement and to review.
Thanks,
tglx
The entire series looks good to me except a couple nits replied in individual patches: