Skip to contentSkip to conversation

hv: fix msi affinity when device requests all possible CPU's

PATCH In review Stephen Hemminger 4 participants 7 messages Jun 28, 2017 – Jul 5, 2017 1 message not in the archive PCI

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 hv: fix msi affinity when device requests all possible CPU's

When Intel 10G (ixgbevf) is passed to a Hyper-V guest with SR-IOV, the driver requests affinity with all possible CPU's (0-239) even those CPU's are not online (and will never be). Because of this the device is unable to correctly get MSI interrupt's setup.

This was caused by the change in 4.12 that converted this affinity into all possible CPU's (0-31) but then host reports
an error since this is larger than the number of online cpu's.

Previously, this worked (up to 4.12-rc1) because only online cpu's would be put in mask passed to the host.

This patch applies only to 4.12.
The driver in linux-next needs a a different fix because of the changes to PCI host protocol version.

Fixes 433fcf6b7b31 ("PCI: hv: Specify CPU_AFFINITY_ALL for MSI affinity when >= 32 CPUs")
---
 drivers/pci/host/pci-hyperv.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
drivers/pci/host/pci-hyperv.c +3 -1
index 84936383e269..3cadfcca3ae9 100644
@@ -900,10 +900,12 @@ static void hv_compose_msi_msg(struct irq_data *data, struct msi_msg *msg)
 	 * processors because Hyper-V only supports 64 in a guest.
 	 */
 	affinity = irq_data_get_affinity_mask(data);
+	cpumask_and(affinity, affinity, cpu_online_mask);
+
 	if (cpumask_weight(affinity) >= 32) {
 		int_pkt->int_desc.cpu_mask = CPU_AFFINITY_ALL;
 	} else {
-		for_each_cpu_and(cpu, affinity, cpu_online_mask) {
+		for_each_cpu(cpu, affinity) {
 			int_pkt->int_desc.cpu_mask |=
 				(1ULL << vmbus_cpu_number_to_vp_number(cpu));
 		}
Signature · 1 line
2.11.0

A message that is not in the archive (sent off-list or not yet received): DM5PR21MB0476CFFA739B74FBB3B6F42DA0D20@DM5PR21MB0476.namprd21.prod.outlook.com

#2 lore raw
Quoted 7 lines

-----Original Message-----
From: Stephen Hemminger [mailto:[email protected]]
Sent: Wednesday, June 28, 2017 4:22 PM
To: KY Srinivasan <[email protected]>; [email protected]
Cc: [email protected]; [email protected]; Stephen Hemminger <[email protected]>
Subject: [PATCH] hv: fix msi affinity when device requests all possible C=

PU's

Quoted 2 lines

=20
When Intel 10G (ixgbevf) is passed to a Hyper-V guest with SR-IOV, the dr=

iver

Quoted 1 line

requests affinity with all possible CPU's (0-239) even those CPU's are no=

t online

Quoted 1 line

(and will never be). Because of this the device is unable to correctly ge=

t MSI

Quoted 3 lines

interrupt's setup.
=20
This was caused by the change in 4.12 that converted this affinity into a=

ll

Quoted 1 line

possible CPU's (0-31) but then host reports an error since this is larger=

than the

Quoted 3 lines

number of online cpu's.
=20
Previously, this worked (up to 4.12-rc1) because only online cpu's would =

be put

Quoted 4 lines

in mask passed to the host.
=20
This patch applies only to 4.12.
The driver in linux-next needs a a different fix because of the changes t=

o PCI

Quoted 1 line

host protocol version.

The vPCI patch in linux-next has the issue fixed already.

Regards,
Jork

#3 lore raw

Patch still needed for 4.12

-----Original Message-----
From: Jork Loeser=20
Sent: Thursday, June 29, 2017 3:08 PM
To: [email protected]; KY Srinivasan <[email protected]>; bhelgaas= @google.com
Cc: [email protected]; [email protected]; Stephen Hemmin= ger <[email protected]>
Subject: RE: [PATCH] hv: fix msi affinity when device requests all possible=
CPU's

Quoted 7 lines

-----Original Message-----
From: Stephen Hemminger [mailto:[email protected]]
Sent: Wednesday, June 28, 2017 4:22 PM
To: KY Srinivasan <[email protected]>; [email protected]
Cc: [email protected]; [email protected]; Stephen Hemminger <[email protected]>
Subject: [PATCH] hv: fix msi affinity when device requests all possible C=

PU's

Quoted 2 lines

=20
When Intel 10G (ixgbevf) is passed to a Hyper-V guest with SR-IOV, the dr=

iver

Quoted 1 line

requests affinity with all possible CPU's (0-239) even those CPU's are no=

t online

Quoted 1 line

(and will never be). Because of this the device is unable to correctly ge=

t MSI

Quoted 3 lines

interrupt's setup.
=20
This was caused by the change in 4.12 that converted this affinity into a=

ll

Quoted 1 line

possible CPU's (0-31) but then host reports an error since this is larger=

than the

Quoted 3 lines

number of online cpu's.
=20
Previously, this worked (up to 4.12-rc1) because only online cpu's would =

be put

Quoted 4 lines

in mask passed to the host.
=20
This patch applies only to 4.12.
The driver in linux-next needs a a different fix because of the changes t=

o PCI

Quoted 1 line

host protocol version.

The vPCI patch in linux-next has the issue fixed already.

Regards,
Jork

#4 lore raw
In reply to Stephen Hemminger
Quoted 9 lines · Stephen Hemminger

When Intel 10G (ixgbevf) is passed to a Hyper-V guest with SR-IOV, the driver requests affinity with all possible CPU's (0-239) even those CPU's are not online (and will never be). Because of this the device is unable to correctly get MSI interrupt's setup.

This was caused by the change in 4.12 that converted this affinity into all possible CPU's (0-31) but then host reports
an error since this is larger than the number of online cpu's.

Quoted 6 lines

Previously, this worked (up to 4.12-rc1) because only online cpu's would be put in mask passed to the host.

This patch applies only to 4.12.
The driver in linux-next needs a a different fix because of the changes to PCI host protocol version.

If Linus decides to postpone v4.12 a week, I can ask him to pull this. But I suspect he will release v4.12 today. In that case, I don't know what to do with this other than maybe send it to Greg for a -stable release.

Quoted 27 lines
Fixes 433fcf6b7b31 ("PCI: hv: Specify CPU_AFFINITY_ALL for MSI affinity when >= 32 CPUs")
---
 drivers/pci/host/pci-hyperv.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
drivers/pci/host/pci-hyperv.c +3 -1
index 84936383e269..3cadfcca3ae9 100644
@@ -900,10 +900,12 @@ static void hv_compose_msi_msg(struct irq_data *data, struct msi_msg *msg)
 	 * processors because Hyper-V only supports 64 in a guest.
 	 */
 	affinity = irq_data_get_affinity_mask(data);
+	cpumask_and(affinity, affinity, cpu_online_mask);
+
 	if (cpumask_weight(affinity) >= 32) {
 		int_pkt->int_desc.cpu_mask = CPU_AFFINITY_ALL;
 	} else {
-		for_each_cpu_and(cpu, affinity, cpu_online_mask) {
+		for_each_cpu(cpu, affinity) {
 			int_pkt->int_desc.cpu_mask |=
 				(1ULL << vmbus_cpu_number_to_vp_number(cpu));
 		}
Signature · 1 line
2.11.0
#5 lore raw
Quoted 13 lines · On Sun, 2 Jul 2017 16:38:19 -0500 Bjorn Helgaas <[email protected]> wrote:
Quoted 12 lines · Stephen Hemminger

When Intel 10G (ixgbevf) is passed to a Hyper-V guest with SR-IOV, the driver requests affinity with all possible CPU's (0-239) even those CPU's are not online (and will never be). Because of this the device is unable to correctly get MSI interrupt's setup.

This was caused by the change in 4.12 that converted this affinity into all possible CPU's (0-31) but then host reports
an error since this is larger than the number of online cpu's.

Previously, this worked (up to 4.12-rc1) because only online cpu's would be put in mask passed to the host.

Quoted 7 lines
Quoted 3 lines

This patch applies only to 4.12.
The driver in linux-next needs a a different fix because of the changes to PCI host protocol version.

If Linus decides to postpone v4.12 a week, I can ask him to pull this. But I suspect he will release v4.12 today. In that case, I don't know what to do with this other than maybe send it to Greg for a -stable release.

Looks like this will have to be queued for 4.12 stable.

#6 lore raw
Quoted 20 lines · Stephen Hemminger
Quoted 17 lines · On Sun, 2 Jul 2017 16:38:19 -0500 Bjorn Helgaas <[email protected]> wrote:
Quoted 15 lines · Stephen Hemminger

When Intel 10G (ixgbevf) is passed to a Hyper-V guest with SR-IOV, the driver requests affinity with all possible CPU's (0-239) even those CPU's are not online (and will never be). Because of this the device is unable to correctly get MSI interrupt's setup.

This was caused by the change in 4.12 that converted this affinity into all possible CPU's (0-31) but then host reports
an error since this is larger than the number of online cpu's.

Previously, this worked (up to 4.12-rc1) because only online cpu's would be put in mask passed to the host.

This patch applies only to 4.12.
The driver in linux-next needs a a different fix because of the changes to PCI host protocol version.

Quoted 5 lines
Quoted 3 lines

If Linus decides to postpone v4.12 a week, I can ask him to pull this. But I suspect he will release v4.12 today. In that case, I don't know what to do with this other than maybe send it to Greg for a -stable release.

Looks like this will have to be queued for 4.12 stable.

I assume you'll take care of this, right? It sounds like there's nothing to do for upstream because it needs a different fix.

Bjorn

#7 lore raw
Quoted 25 lines · On Wed, 5 Jul 2017 14:49:33 -0500 Bjorn Helgaas <[email protected]> wrote:
Quoted 24 lines · Stephen Hemminger
Quoted 20 lines · On Sun, 2 Jul 2017 16:38:19 -0500 Bjorn Helgaas <[email protected]> wrote:
Quoted 9 lines · Stephen Hemminger

When Intel 10G (ixgbevf) is passed to a Hyper-V guest with SR-IOV, the driver requests affinity with all possible CPU's (0-239) even those CPU's are not online (and will never be). Because of this the device is unable to correctly get MSI interrupt's setup.

This was caused by the change in 4.12 that converted this affinity into all possible CPU's (0-31) but then host reports
an error since this is larger than the number of online cpu's.

Quoted 6 lines

Previously, this worked (up to 4.12-rc1) because only online cpu's would be put in mask passed to the host.

This patch applies only to 4.12.
The driver in linux-next needs a a different fix because of the changes to PCI host protocol version.

If Linus decides to postpone v4.12 a week, I can ask him to pull this. But I suspect he will release v4.12 today. In that case, I don't know what to do with this other than maybe send it to Greg for a -stable release.

Quoted 6 lines
Quoted 1 line

Looks like this will have to be queued for 4.12 stable.

I assume you'll take care of this, right? It sounds like there's nothing to do for upstream because it needs a different fix.

Bjorn

Already fixed in Linux-next. The code is different for PCI 1.2 version and never had the bug.