Skip to contentSkip to conversation

PCI/MSI: Fix MSI hwirq truncation

PATCH Superseded Vidya Sagar 5 participants 18 messages Jan 5, 2024 – Feb 19, 2024

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 PCI/MSI: Fix MSI hwirq truncation

While calculating the hwirq number for an MSI interrupt, the higher bits (i.e. from bit-5 onwards a.k.a domain_nr >= 32) of the PCI domain number gets truncated because of the shifted value casting to u32. This for example is resulting in same hwirq number for devices 0019:00:00.0 and 0039:00:00.0.

So, cast the PCI domain number to u64 before left shifting it to calculate hwirq number.

---
 drivers/pci/msi/irqdomain.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/pci/msi/irqdomain.c +1 -1
index c8be056c248d..cfd84a899c82 100644
@@ -61,7 +61,7 @@ static irq_hw_number_t pci_msi_domain_calc_hwirq(struct msi_desc *desc)
 
 	return (irq_hw_number_t)desc->msi_index |
 		pci_dev_id(dev) << 11 |
-		(pci_domain_nr(dev->bus) & 0xFFFFFFFF) << 27;
+		((irq_hw_number_t)(pci_domain_nr(dev->bus) & 0xFFFFFFFF)) << 27;
 }
 
 static void pci_msi_domain_set_desc(msi_alloc_info_t *arg,
Signature · 1 line
2.25.1
#1 lore raw
Quoted 8 lines · Vidya Sagar

While calculating the hwirq number for an MSI interrupt, the higher bits (i.e. from bit-5 onwards a.k.a domain_nr >= 32) of the PCI domain number gets truncated because of the shifted value casting to u32. This for example is resulting in same hwirq number for devices 0019:00:00.0 and 0039:00:00.0.

So, cast the PCI domain number to u64 before left shifting it to calculate hwirq number.

This seems like a bug fix. Why not add Fixes: tag?

Is this issue detected in a platform test? Maybe including the relevant details about it will help reproduce the issue.

Quoted 19 lines
---
 drivers/pci/msi/irqdomain.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/pci/msi/irqdomain.c +1 -1
index c8be056c248d..cfd84a899c82 100644
@@ -61,7 +61,7 @@ static irq_hw_number_t pci_msi_domain_calc_hwirq(struct msi_desc *desc)
 
 	return (irq_hw_number_t)desc->msi_index |
 		pci_dev_id(dev) << 11 |
-		(pci_domain_nr(dev->bus) & 0xFFFFFFFF) << 27;
+		((irq_hw_number_t)(pci_domain_nr(dev->bus) & 0xFFFFFFFF)) << 27;
 }
 
 static void pci_msi_domain_set_desc(msi_alloc_info_t *arg,
Signature · 2 lines
Sathyanarayanan Kuppuswamy
Linux Kernel Developer
#2 lore raw
Quoted 10 lines · Kuppuswamy Sathyanarayanan

External email: Use caution opening links or attachments

Quoted 6 lines · Vidya Sagar

While calculating the hwirq number for an MSI interrupt, the higher bits (i.e. from bit-5 onwards a.k.a domain_nr >= 32) of the PCI domain number gets truncated because of the shifted value casting to u32. This for example is resulting in same hwirq number for devices 0019:00:00.0 and 0039:00:00.0.

Quoted 4 lines
Quoted 2 lines

So, cast the PCI domain number to u64 before left shifting it to calculate hwirq number.

This seems like a bug fix. Why not add Fixes: tag?

This doesn't seem to be introduced by any recent change as such. It has been from the time the original change appeared in the mainline. I'll give the commit ID of the original change for 'Fixes'.

Quoted 3 lines

Is this issue detected in a platform test? Maybe including the relevant details about it will help reproduce the issue.

The issue is detected while running ARM's ACS test suite. But, I couldn't give that info as there was some bug in the test suite itself for which I raised an issue at https://gitlab.arm.com/linux-arm/linux-acs/-/issues/2

Quoted 24 lines
Quoted 12 lines
---
  drivers/pci/msi/irqdomain.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
drivers/pci/msi/irqdomain.c +0 -0
index c8be056c248d..cfd84a899c82 100644

drivers/pci/msi/irqdomain.c static irq_hw_number_t pci_msi_domain_calc_hwirq(struct msi_desc *desc) · line 67

Quoted 8 lines
drivers/pci/msi/irqdomain.c +1 -1
@@ -61,7 +61,7 @@ static irq_hw_number_t pci_msi_domain_calc_hwirq(struct msi_desc *desc)
 
       return (irq_hw_number_t)desc->msi_index |
               pci_dev_id(dev) << 11 |
-             (pci_domain_nr(dev->bus) & 0xFFFFFFFF) << 27;
+             ((irq_hw_number_t)(pci_domain_nr(dev->bus) & 0xFFFFFFFF)) << 27;
  }
 
  static void pci_msi_domain_set_desc(msi_alloc_info_t *arg,

--
Sathyanarayanan Kuppuswamy
Linux Kernel Developer

#3 lore raw

PATCH v2 1/1 PCI/MSI: Fix MSI hwirq truncation

In reply to Vidya Sagar

While calculating the hwirq number for an MSI interrupt, the higher bits (i.e. from bit-5 onwards a.k.a domain_nr >= 32) of the PCI domain number gets truncated because of the shifted value casting to u32. This for example is resulting in same hwirq number for devices 0019:00:00.0 and 0039:00:00.0.

So, cast the PCI domain number to u64 before left shifting it to calculate hwirq number.

Fixes 3878eaefb89a ("PCI/MSI: Enhance core to support hierarchy irqdomain")

---
V2:
* Added Fixes tag

 drivers/pci/msi/irqdomain.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/pci/msi/irqdomain.c +1 -1
index c8be056c248d..cfd84a899c82 100644
@@ -61,7 +61,7 @@ static irq_hw_number_t pci_msi_domain_calc_hwirq(struct msi_desc *desc)
 
 	return (irq_hw_number_t)desc->msi_index |
 		pci_dev_id(dev) << 11 |
-		(pci_domain_nr(dev->bus) & 0xFFFFFFFF) << 27;
+		((irq_hw_number_t)(pci_domain_nr(dev->bus) & 0xFFFFFFFF)) << 27;
 }
 
 static void pci_msi_domain_set_desc(msi_alloc_info_t *arg,
Signature · 1 line
2.25.1
Tested-by on v2 1/1
#4 lore raw

Thanks Vidya.

-ShankerD

Quoted 31 lines · Vidya Sagar

While calculating the hwirq number for an MSI interrupt, the higher bits (i.e. from bit-5 onwards a.k.a domain_nr >= 32) of the PCI domain number gets truncated because of the shifted value casting to u32. This for example is resulting in same hwirq number for devices 0019:00:00.0 and 0039:00:00.0.

So, cast the PCI domain number to u64 before left shifting it to calculate hwirq number.

Fixes 3878eaefb89a ("PCI/MSI: Enhance core to support hierarchy irqdomain")

---
V2:
* Added Fixes tag

  drivers/pci/msi/irqdomain.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
drivers/pci/msi/irqdomain.c +1 -1
index c8be056c248d..cfd84a899c82 100644
@@ -61,7 +61,7 @@ static irq_hw_number_t pci_msi_domain_calc_hwirq(struct msi_desc *desc)
 
  	return (irq_hw_number_t)desc->msi_index |
  		pci_dev_id(dev) << 11 |
-		(pci_domain_nr(dev->bus) & 0xFFFFFFFF) << 27;
+		((irq_hw_number_t)(pci_domain_nr(dev->bus) & 0xFFFFFFFF)) << 27;
  }
 
  static void pci_msi_domain_set_desc(msi_alloc_info_t *arg,
#5 lore raw
In reply to Vidya Sagar
Quoted 23 lines · Vidya Sagar

While calculating the hwirq number for an MSI interrupt, the higher bits (i.e. from bit-5 onwards a.k.a domain_nr >= 32) of the PCI domain number gets truncated because of the shifted value casting to u32. This for example is resulting in same hwirq number for devices 0019:00:00.0 and 0039:00:00.0.

So, cast the PCI domain number to u64 before left shifting it to calculate hwirq number.

Fixes 3878eaefb89a ("PCI/MSI: Enhance core to support hierarchy irqdomain")

---
V2:
* Added Fixes tag

 drivers/pci/msi/irqdomain.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/pci/msi/irqdomain.c +0 -0
index c8be056c248d..cfd84a899c82 100644

drivers/pci/msi/irqdomain.c static irq_hw_number_t pci_msi_domain_calc_hwirq(struct msi_desc *desc) · line 64

Quoted 5 lines
drivers/pci/msi/irqdomain.c +1 -1
@@ -61,7 +61,7 @@ static irq_hw_number_t pci_msi_domain_calc_hwirq(struct msi_desc *desc)
 
 	return (irq_hw_number_t)desc->msi_index |
 		pci_dev_id(dev) << 11 |
-		(pci_domain_nr(dev->bus) & 0xFFFFFFFF) << 27;
+		((irq_hw_number_t)(pci_domain_nr(dev->bus) & 0xFFFFFFFF)) << 27;

This is not casting to u64. It's casting to unsigned long:

typedef unsigned long irq_hw_number_t;

So this works only correctly on 64bit. On 32bit kernels unsigned long is still 32bit. It's probably arguable that the 32bit case is not a problem, but the changelog and the change do not match. This needs a proper explanation why we don't care about this on 32bit.

Thanks,

tglx

#6 lore raw
Quoted 36 lines · Thomas Gleixner

External email: Use caution opening links or attachments

Quoted 23 lines · Vidya Sagar

While calculating the hwirq number for an MSI interrupt, the higher bits (i.e. from bit-5 onwards a.k.a domain_nr >= 32) of the PCI domain number gets truncated because of the shifted value casting to u32. This for example is resulting in same hwirq number for devices 0019:00:00.0 and 0039:00:00.0.

So, cast the PCI domain number to u64 before left shifting it to calculate hwirq number.

Fixes 3878eaefb89a ("PCI/MSI: Enhance core to support hierarchy irqdomain")

---
V2:
* Added Fixes tag

  drivers/pci/msi/irqdomain.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
drivers/pci/msi/irqdomain.c +0 -0
index c8be056c248d..cfd84a899c82 100644

drivers/pci/msi/irqdomain.c static irq_hw_number_t pci_msi_domain_calc_hwirq(struct msi_desc *desc) · line 64

Quoted 5 lines
drivers/pci/msi/irqdomain.c +1 -1
@@ -61,7 +61,7 @@ static irq_hw_number_t pci_msi_domain_calc_hwirq(struct msi_desc *desc)
 
       return (irq_hw_number_t)desc->msi_index |
               pci_dev_id(dev) << 11 |
-             (pci_domain_nr(dev->bus) & 0xFFFFFFFF) << 27;
+             ((irq_hw_number_t)(pci_domain_nr(dev->bus) & 0xFFFFFFFF)) << 27;

This is not casting to u64. It's casting to unsigned long:

Quoted 6 lines
   typedef unsigned long irq_hw_number_t;

So this works only correctly on 64bit. On 32bit kernels unsigned long is still 32bit. It's probably arguable that the 32bit case is not a problem, but the changelog and the change do not match. This needs a proper explanation why we don't care about this on 32bit.

Agree that the change and the commit message do not match. I'll update the commit message to use 'irq_hw_number_t' instead of u64.

Thanks,
Vidya Sagar

Quoted 4 lines

Thanks,

tglx

#7 lore raw

PATCH v3 1/1 PCI/MSI: Fix MSI hwirq truncation

In reply to Vidya Sagar

While calculating the hwirq number for an MSI interrupt, the higher bits (i.e. from bit-5 onwards a.k.a domain_nr >= 32) of the PCI domain number gets truncated because of the shifted value casting to return type of pci_domain_nr() which is 'int'. This for example is resulting in same hwirq number for devices 0019:00:00.0 and 0039:00:00.0.

So, cast the PCI domain number to 'irq_hw_number_t' before left shifting it to calculate hwirq number.

Fixes 3878eaefb89a ("PCI/MSI: Enhance core to support hierarchy irqdomain")

---
V3:
* Addressed review comments from Thomas Gleixner
* Added Tested-By: Shanker Donthineni <[email protected]>

V2:
* Added Fixes tag

 drivers/pci/msi/irqdomain.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/pci/msi/irqdomain.c +1 -1
index c8be056c248d..cfd84a899c82 100644
@@ -61,7 +61,7 @@ static irq_hw_number_t pci_msi_domain_calc_hwirq(struct msi_desc *desc)
 
 	return (irq_hw_number_t)desc->msi_index |
 		pci_dev_id(dev) << 11 |
-		(pci_domain_nr(dev->bus) & 0xFFFFFFFF) << 27;
+		((irq_hw_number_t)(pci_domain_nr(dev->bus) & 0xFFFFFFFF)) << 27;
 }
 
 static void pci_msi_domain_set_desc(msi_alloc_info_t *arg,
Signature · 1 line
2.25.1
#8 lore raw
Quoted 8 lines · Vidya Sagar

While calculating the hwirq number for an MSI interrupt, the higher bits (i.e. from bit-5 onwards a.k.a domain_nr >= 32) of the PCI domain number gets truncated because of the shifted value casting to return type of pci_domain_nr() which is 'int'. This for example is resulting in same hwirq number for devices 0019:00:00.0 and 0039:00:00.0.

So, cast the PCI domain number to 'irq_hw_number_t' before left shifting it to calculate hwirq number.

This still does not explain that this fixes it only on 64-bit platforms and why we don't care for 32-bit systems.

#9 lore raw
Quoted 10 lines · Thomas Gleixner

External email: Use caution opening links or attachments

Quoted 6 lines · Vidya Sagar

While calculating the hwirq number for an MSI interrupt, the higher bits (i.e. from bit-5 onwards a.k.a domain_nr >= 32) of the PCI domain number gets truncated because of the shifted value casting to return type of pci_domain_nr() which is 'int'. This for example is resulting in same hwirq number for devices 0019:00:00.0 and 0039:00:00.0.

Quoted 5 lines
Quoted 2 lines

So, cast the PCI domain number to 'irq_hw_number_t' before left shifting it to calculate hwirq number.

This still does not explain that this fixes it only on 64-bit platforms and why we don't care for 32-bit systems.

Agree that this fixes the issue only on 64-bit platforms. It doesn't change the behavior on 32-bit platforms. My understanding is that the issue surfaces only if there are too many PCIe controllers in the system which usually is the case in modern server systems and it is arguable if the server systems really run 32-bit kernels.

One way to fix it for both 32-bit and 64-bit systems is by changing the type of 'hwirq' to u64. This may cause two memory reads in 32-bit systems whenever 'hwirq' is accessed and that may intern cause some perf impact?? Is this the way you think I should be handling it?

Quoted 1 line
#10 lore raw
Quoted 5 lines · Vidya Sagar
Quoted 4 lines · Thomas Gleixner
Quoted 2 lines · Vidya Sagar

So, cast the PCI domain number to 'irq_hw_number_t' before left shifting it to calculate hwirq number.

Quoted 7 lines
Quoted 2 lines

This still does not explain that this fixes it only on 64-bit platforms and why we don't care for 32-bit systems.

Agree that this fixes the issue only on 64-bit platforms. It doesn't change the behavior on 32-bit platforms. My understanding is that the issue surfaces only if there are too many PCIe controllers in the system which usually is the case in modern server systems and it is arguable if the server systems really run 32-bit kernels.

Arguably people who do that can keep the pieces.

Quoted 4 lines

One way to fix it for both 32-bit and 64-bit systems is by changing the type of 'hwirq' to u64. This may cause two memory reads in 32-bit systems whenever 'hwirq' is accessed and that may intern cause some perf impact?? Is this the way you think I should be handling it?

No. Leave it as is. What I'm asking for is that it's properly documented in the changelog.

Thanks,

tglx

#11 lore raw
Quoted 19 lines · Thomas Gleixner

External email: Use caution opening links or attachments

Quoted 5 lines · Vidya Sagar
Quoted 4 lines · Thomas Gleixner
Quoted 2 lines · Vidya Sagar

So, cast the PCI domain number to 'irq_hw_number_t' before left shifting it to calculate hwirq number.

Quoted 7 lines
Quoted 2 lines

This still does not explain that this fixes it only on 64-bit platforms and why we don't care for 32-bit systems.

Agree that this fixes the issue only on 64-bit platforms. It doesn't change the behavior on 32-bit platforms. My understanding is that the issue surfaces only if there are too many PCIe controllers in the system which usually is the case in modern server systems and it is arguable if the server systems really run 32-bit kernels.

Arguably people who do that can keep the pieces.

Quoted 7 lines
Quoted 4 lines

One way to fix it for both 32-bit and 64-bit systems is by changing the type of 'hwirq' to u64. This may cause two memory reads in 32-bit systems whenever 'hwirq' is accessed and that may intern cause some perf impact?? Is this the way you think I should be handling it?

No. Leave it as is. What I'm asking for is that it's properly documented in the changelog.

Sure. I'll add this extra information in the change log.

Quoted 5 lines

Thanks,

tglx

#12 lore raw

PATCH v4 1/1 PCI/MSI: Fix MSI hwirq truncation

In reply to Vidya Sagar

While calculating the hwirq number for an MSI interrupt, the higher bits (i.e. from bit-5 onwards a.k.a domain_nr >= 32) of the PCI domain number gets truncated because of the shifted value casting to return type of pci_domain_nr() which is 'int'. This for example is resulting in same hwirq number for devices 0019:00:00.0 and 0039:00:00.0.

So, cast the PCI domain number to 'irq_hw_number_t' before left shifting it to calculate hwirq number. Please note that this fixes the issue only on 64-bit systems and doesn't change the behavior in 32-bit systems i.e. the 32-bit systems continue to have the issue. Since the issue surfaces only if there are too many PCIe controllers in the system which usually is the case in modern server systems and they don't tend to run 32-bit kernels.

Fixes 3878eaefb89a ("PCI/MSI: Enhance core to support hierarchy irqdomain")

---
V4:
* Added extra information in the change log about the impact of this patch in a 32-bit system as suggested by Thomas

V3:
* Addressed review comments from Thomas Gleixner
* Added Tested-By: Shanker Donthineni <[email protected]>

V2:
* Added Fixes tag

 drivers/pci/msi/irqdomain.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/pci/msi/irqdomain.c +1 -1
index c8be056c248d..cfd84a899c82 100644
@@ -61,7 +61,7 @@ static irq_hw_number_t pci_msi_domain_calc_hwirq(struct msi_desc *desc)
 
 	return (irq_hw_number_t)desc->msi_index |
 		pci_dev_id(dev) << 11 |
-		(pci_domain_nr(dev->bus) & 0xFFFFFFFF) << 27;
+		((irq_hw_number_t)(pci_domain_nr(dev->bus) & 0xFFFFFFFF)) << 27;
 }
 
 static void pci_msi_domain_set_desc(msi_alloc_info_t *arg,
Signature · 1 line
2.25.1
#13 lore raw

Hi Thomas,
Does this patch look fine to you?
If yes, would you mind giving an Ack?

Thanks,
Vidya Sagar

Quoted 45 lines · Vidya Sagar

While calculating the hwirq number for an MSI interrupt, the higher bits (i.e. from bit-5 onwards a.k.a domain_nr >= 32) of the PCI domain number gets truncated because of the shifted value casting to return type of pci_domain_nr() which is 'int'. This for example is resulting in same hwirq number for devices 0019:00:00.0 and 0039:00:00.0.

So, cast the PCI domain number to 'irq_hw_number_t' before left shifting it to calculate hwirq number. Please note that this fixes the issue only on 64-bit systems and doesn't change the behavior in 32-bit systems i.e. the 32-bit systems continue to have the issue. Since the issue surfaces only if there are too many PCIe controllers in the system which usually is the case in modern server systems and they don't tend to run 32-bit kernels.

Fixes 3878eaefb89a ("PCI/MSI: Enhance core to support hierarchy irqdomain")

---
V4:
* Added extra information in the change log about the impact of this patch in a 32-bit system as suggested by Thomas

V3:
* Addressed review comments from Thomas Gleixner
* Added Tested-By: Shanker Donthineni <[email protected]>

V2:
* Added Fixes tag

  drivers/pci/msi/irqdomain.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
drivers/pci/msi/irqdomain.c +1 -1
index c8be056c248d..cfd84a899c82 100644
@@ -61,7 +61,7 @@ static irq_hw_number_t pci_msi_domain_calc_hwirq(struct msi_desc *desc)
 
  	return (irq_hw_number_t)desc->msi_index |
  		pci_dev_id(dev) << 11 |
-		(pci_domain_nr(dev->bus) & 0xFFFFFFFF) << 27;
+		((irq_hw_number_t)(pci_domain_nr(dev->bus) & 0xFFFFFFFF)) << 27;
  }
 
  static void pci_msi_domain_set_desc(msi_alloc_info_t *arg,
#14 lore raw

Hi Thomas,
Sorry to bother you.
Would you mind giving an Ack to this patch?

Thanks,
Vidya Sagar

Quoted 52 lines · Vidya Sagar

Hi Thomas,
Does this patch look fine to you?
If yes, would you mind giving an Ack?

Thanks,
Vidya Sagar

Quoted 44 lines · Vidya Sagar

While calculating the hwirq number for an MSI interrupt, the higher bits (i.e. from bit-5 onwards a.k.a domain_nr >= 32) of the PCI domain number gets truncated because of the shifted value casting to return type of pci_domain_nr() which is 'int'. This for example is resulting in same hwirq number for devices 0019:00:00.0 and 0039:00:00.0.

So, cast the PCI domain number to 'irq_hw_number_t' before left shifting it to calculate hwirq number. Please note that this fixes the issue only on 64-bit systems and doesn't change the behavior in 32-bit systems i.e. the 32-bit systems continue to have the issue. Since the issue surfaces only if there are too many PCIe controllers in the system which usually is the case in modern server systems and they don't tend to run 32-bit kernels.

Fixes 3878eaefb89a ("PCI/MSI: Enhance core to support hierarchy irqdomain")

---
V4:
* Added extra information in the change log about the impact of this patch in a 32-bit system as suggested by Thomas

V3:
* Addressed review comments from Thomas Gleixner
* Added Tested-By: Shanker Donthineni <[email protected]>

V2:
* Added Fixes tag

  drivers/pci/msi/irqdomain.c | 2 +-

  1 file changed, 1 insertion(+), 1 deletion(-)
drivers/pci/msi/irqdomain.c +0 -0
index c8be056c248d..cfd84a899c82 100644
@@ -61,7 +61,7 @@ static irq_hw_number_t 
 pci_msi_domain_calc_hwirq(struct msi_desc *desc)

      return (irq_hw_number_t)desc->msi_index | pci_dev_id(dev) << 11 |
-        (pci_domain_nr(dev->bus) & 0xFFFFFFFF) << 27;
+        ((irq_hw_number_t)(pci_domain_nr(dev->bus) & 0xFFFFFFFF)) << 27; }
  static void pci_msi_domain_set_desc(msi_alloc_info_t *arg,

#15 lore raw

Hi Thomas / Bjorn,
Can you please guide me on getting this patch merged?

Thanks,
Vidya Sagar

Quoted 61 lines · Vidya Sagar

Hi Thomas,
Sorry to bother you.
Would you mind giving an Ack to this patch?

Thanks,
Vidya Sagar

Quoted 53 lines · Vidya Sagar

Hi Thomas,
Does this patch look fine to you?
If yes, would you mind giving an Ack?

Thanks,
Vidya Sagar

Quoted 45 lines · Vidya Sagar

While calculating the hwirq number for an MSI interrupt, the higher bits (i.e. from bit-5 onwards a.k.a domain_nr >= 32) of the PCI domain number gets truncated because of the shifted value casting to return type of pci_domain_nr() which is 'int'. This for example is resulting in same hwirq number for devices 0019:00:00.0 and 0039:00:00.0.

So, cast the PCI domain number to 'irq_hw_number_t' before left shifting it to calculate hwirq number. Please note that this fixes the issue only on 64-bit systems and doesn't change the behavior in 32-bit systems i.e. the 32-bit systems continue to have the issue. Since the issue surfaces only if there are too many PCIe controllers in the system which usually is the case in modern server systems and they don't tend to run 32-bit kernels.

Fixes 3878eaefb89a ("PCI/MSI: Enhance core to support hierarchy irqdomain")

---
V4:
* Added extra information in the change log about the impact of this patch in a 32-bit system as suggested by Thomas

V3:
* Addressed review comments from Thomas Gleixner
* Added Tested-By: Shanker Donthineni <[email protected]>

V2:
* Added Fixes tag

  drivers/pci/msi/irqdomain.c | 2 +-

  1 file changed, 1 insertion(+), 1 deletion(-)
drivers/pci/msi/irqdomain.c +0 -0
index c8be056c248d..cfd84a899c82 100644
@@ -61,7 +61,7 @@ static irq_hw_number_t 
 pci_msi_domain_calc_hwirq(struct msi_desc *desc)

      return (irq_hw_number_t)desc->msi_index | pci_dev_id(dev) << 11 |
-        (pci_domain_nr(dev->bus) & 0xFFFFFFFF) << 27;
+        ((irq_hw_number_t)(pci_domain_nr(dev->bus) & 0xFFFFFFFF)) << 27;
  }
  static void pci_msi_domain_set_desc(msi_alloc_info_t *arg,

#16 lore raw
Quoted 2 lines · Vidya Sagar

Hi Thomas / Bjorn,
Can you please guide me on getting this patch merged?

It's in my backlog...

merge bot Applied
#17 lore raw

[tip: irq/urgent] PCI/MSI: Prevent MSI hardware interrupt number truncation

In reply to Vidya Sagar
The following commit has been merged into the irq/urgent branch of tip:

The following commit has been merged into the irq/urgent branch of tip:

Commit-ID:     db744ddd59be798c2627efbfc71f707f5a935a40
Gitweb:        https://git.kernel.org/tip/db744ddd59be798c2627efbfc71f707f5a935a40
Author:        Vidya Sagar <[email protected]>
AuthorDate:    Mon, 15 Jan 2024 19:26:49 +05:30
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Mon, 19 Feb 2024 16:11:01 +01:00

PCI/MSI: Prevent MSI hardware interrupt number truncation

While calculating the hardware interrupt number for a MSI interrupt, the higher bits (i.e. from bit-5 onwards a.k.a domain_nr >= 32) of the PCI domain number gets truncated because of the shifted value casting to return type of pci_domain_nr() which is 'int'. This for example is resulting in same hardware interrupt number for devices 0019:00:00.0 and 0039:00:00.0.

To address this cast the PCI domain number to 'irq_hw_number_t' before left shifting it to calculate the hardware interrupt number.

Please note that this fixes the issue only on 64-bit systems and doesn't change the behavior for 32-bit systems i.e. the 32-bit systems continue to have the issue. Since the issue surfaces only if there are too many PCIe controllers in the system which usually is the case in modern server systems and they don't tend to run 32-bit kernels.

Fixes 3878eaefb89a ("PCI/MSI: Enhance core to support hierarchy irqdomain")
---
 drivers/pci/msi/irqdomain.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/pci/msi/irqdomain.c +1 -1
index c8be056..cfd84a8 100644
@@ -61,7 +61,7 @@ static irq_hw_number_t pci_msi_domain_calc_hwirq(struct msi_desc *desc)
 
 	return (irq_hw_number_t)desc->msi_index |
 		pci_dev_id(dev) << 11 |
-		(pci_domain_nr(dev->bus) & 0xFFFFFFFF) << 27;
+		((irq_hw_number_t)(pci_domain_nr(dev->bus) & 0xFFFFFFFF)) << 27;
 }
 
 static void pci_msi_domain_set_desc(msi_alloc_info_t *arg,