Skip to contentSkip to conversation

PCI: hv: Fix NUMA node assignment when kernel boots with custom NUMA topology

PATCH Superseded [email protected] 4 participants 4 messages Jan 19, 2022 – Jan 26, 2022

Subsystems (4)

v3 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

PATCH v3 1/1 PCI: hv: Fix NUMA node assignment when kernel boots with custom NUMA topology

From: Long Li <[email protected]>

When kernel boots with a NUMA topology with some NUMA nodes offline, the PCI driver should only set an online NUMA node on the device. This can happen during KDUMP where some NUMA nodes are not made online by the KDUMP kernel.

This patch also fixes the case where kernel is booting with "numa=off".

Fixes 999dd956d838 ("PCI: hv: Add support for protocol 1.3 and support PCI_BUS_RELATIONS2")

Change log:
v2: use numa_map_to_online_node() to assign a node to device (suggested by Michael Kelly <[email protected]>)

v3: add "Fixes" and check for num_possible_nodes()

---
 drivers/pci/controller/pci-hyperv.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)
drivers/pci/controller/pci-hyperv.c +11 -2
index 6c9efeefae1b..b5276e81bb44 100644
@@ -2129,8 +2129,17 @@ static void hv_pci_assign_numa_node(struct hv_pcibus_device *hbus)
 		if (!hv_dev)
 			continue;
 
-		if (hv_dev->desc.flags & HV_PCI_DEVICE_FLAG_NUMA_AFFINITY)
-			set_dev_node(&dev->dev, hv_dev->desc.virtual_numa_node);
+		if (hv_dev->desc.flags & HV_PCI_DEVICE_FLAG_NUMA_AFFINITY &&
+		    hv_dev->desc.virtual_numa_node < num_possible_nodes())
+			/*
+			 * The kernel may boot with some NUMA nodes offline
+			 * (e.g. in a KDUMP kernel) or with NUMA disabled via
+			 * "numa=off". In those cases, adjust the host provided
+			 * NUMA node to a valid NUMA node used by the kernel.
+			 */
+			set_dev_node(&dev->dev,
+				     numa_map_to_online_node(
+					     hv_dev->desc.virtual_numa_node));
 
 		put_pcichild(hv_dev);
 	}
Signature · 1 line
2.25.1
Reviewed-by on v3 1/1
#1 lore raw

From: [email protected] <[email protected]> Sent: Wednesday, January 19, 2022 11:59 AM

Quoted 44 lines

When kernel boots with a NUMA topology with some NUMA nodes offline, the PCI driver should only set an online NUMA node on the device. This can happen during KDUMP where some NUMA nodes are not made online by the KDUMP kernel.

This patch also fixes the case where kernel is booting with "numa=off".

Fixes 999dd956d838 ("PCI: hv: Add support for protocol 1.3 and support PCI_BUS_RELATIONS2")

Change log:
v2: use numa_map_to_online_node() to assign a node to device (suggested by Michael Kelly <[email protected]>)

v3: add "Fixes" and check for num_possible_nodes()

---
 drivers/pci/controller/pci-hyperv.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)
drivers/pci/controller/pci-hyperv.c +11 -2
index 6c9efeefae1b..b5276e81bb44 100644
@@ -2129,8 +2129,17 @@ static void hv_pci_assign_numa_node(struct
 hv_pcibus_device *hbus)
 		if (!hv_dev)
 			continue;
 
-		if (hv_dev->desc.flags & HV_PCI_DEVICE_FLAG_NUMA_AFFINITY)
-			set_dev_node(&dev->dev, hv_dev->desc.virtual_numa_node);
+		if (hv_dev->desc.flags & HV_PCI_DEVICE_FLAG_NUMA_AFFINITY &&
+		    hv_dev->desc.virtual_numa_node < num_possible_nodes())
+			/*
+			 * The kernel may boot with some NUMA nodes offline
+			 * (e.g. in a KDUMP kernel) or with NUMA disabled via
+			 * "numa=off". In those cases, adjust the host provided
+			 * NUMA node to a valid NUMA node used by the kernel.
+			 */
+			set_dev_node(&dev->dev,
+				     numa_map_to_online_node(
+					     hv_dev->desc.virtual_numa_node));
 
 		put_pcichild(hv_dev);
Quoted 3 lines

}
--
2.25.1

#2 lore raw
In reply to [email protected]
Quoted 10 lines · On Wed, Jan 19, 2022 at 11:59:06AM -0800, [email protected] wrote:

From: Long Li <[email protected]>

When kernel boots with a NUMA topology with some NUMA nodes offline, the PCI driver should only set an online NUMA node on the device. This can happen during KDUMP where some NUMA nodes are not made online by the KDUMP kernel.

This patch also fixes the case where kernel is booting with "numa=off".

Fixes 999dd956d838 ("PCI: hv: Add support for protocol 1.3 and support PCI_BUS_RELATIONS2")

No blank line here, please

Quoted 1 line

Everything below needs to be under "---" marker.

Thanks

Quoted 37 lines

Change log:
v2: use numa_map_to_online_node() to assign a node to device (suggested by Michael Kelly <[email protected]>)

v3: add "Fixes" and check for num_possible_nodes()

---
 drivers/pci/controller/pci-hyperv.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)
drivers/pci/controller/pci-hyperv.c +11 -2
index 6c9efeefae1b..b5276e81bb44 100644
@@ -2129,8 +2129,17 @@ static void hv_pci_assign_numa_node(struct hv_pcibus_device *hbus)
 		if (!hv_dev)
 			continue;
 
-		if (hv_dev->desc.flags & HV_PCI_DEVICE_FLAG_NUMA_AFFINITY)
-			set_dev_node(&dev->dev, hv_dev->desc.virtual_numa_node);
+		if (hv_dev->desc.flags & HV_PCI_DEVICE_FLAG_NUMA_AFFINITY &&
+		    hv_dev->desc.virtual_numa_node < num_possible_nodes())
+			/*
+			 * The kernel may boot with some NUMA nodes offline
+			 * (e.g. in a KDUMP kernel) or with NUMA disabled via
+			 * "numa=off". In those cases, adjust the host provided
+			 * NUMA node to a valid NUMA node used by the kernel.
+			 */
+			set_dev_node(&dev->dev,
+				     numa_map_to_online_node(
+					     hv_dev->desc.virtual_numa_node));
 
 		put_pcichild(hv_dev);
 	}
Signature · 1 line
2.25.1
#3 lore raw
Quoted 17 lines

Subject: Re: [Patch v3] PCI: hv: Fix NUMA node assignment when kernel boots with custom NUMA topology

Quoted 5 lines · On Wed, Jan 19, 2022 at 11:59:06AM -0800, [email protected] wrote:

From: Long Li <[email protected]>

When kernel boots with a NUMA topology with some NUMA nodes offline, the PCI driver should only set an online NUMA node on the device. This can happen during KDUMP where some NUMA nodes are not made online by

the KDUMP kernel.

Quoted 6 lines

This patch also fixes the case where kernel is booting with "numa=off".

Fixes 999dd956d838 ("PCI: hv: Add support for protocol 1.3 and

support PCI_BUS_RELATIONS2")

Quoted 5 lines

No blank line here, please

Quoted 1 line

Everything below needs to be under "---" marker.

I'm sending v4 to address the comments.

Long

Quoted 45 lines

Thanks

drivers/pci/controller/pci-hyperv.c static void hv_pci_assign_numa_node(struct

Quoted 16 lines

Change log:
v2: use numa_map_to_online_node() to assign a node to device (suggested by Michael Kelly <[email protected]>)

v3: add "Fixes" and check for num_possible_nodes()

---
 drivers/pci/controller/pci-hyperv.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)
drivers/pci/controller/pci-hyperv.c +0 -0
index 6c9efeefae1b..b5276e81bb44 100644
@@ -2129,8 +2129,17 @@ static void hv_pci_assign_numa_node(struct

hv_pcibus_device *hbus)

drivers/pci/controller/pci-hyperv.c static void hv_pci_assign_numa_node(struct hv_pcibus_device *hbus)

Quoted 4 lines
drivers/pci/controller/pci-hyperv.c +0 -1
 		if (!hv_dev)
 			continue;
 
-		if (hv_dev->desc.flags &

HV_PCI_DEVICE_FLAG_NUMA_AFFINITY)

drivers/pci/controller/pci-hyperv.c static void hv_pci_assign_numa_node(struct · line 2133

Quoted 3 lines
drivers/pci/controller/pci-hyperv.c +1 -1
-			set_dev_node(&dev->dev, hv_dev-
 desc.virtual_numa_node);
+		if (hv_dev->desc.flags &

HV_PCI_DEVICE_FLAG_NUMA_AFFINITY &&

Quoted 16 lines
drivers/pci/controller/pci-hyperv.c +10 -0
+		    hv_dev->desc.virtual_numa_node < num_possible_nodes())
+			/*
+			 * The kernel may boot with some NUMA nodes offline
+			 * (e.g. in a KDUMP kernel) or with NUMA disabled via
+			 * "numa=off". In those cases, adjust the host provided
+			 * NUMA node to a valid NUMA node used by the kernel.
+			 */
+			set_dev_node(&dev->dev,
+				     numa_map_to_online_node(
+					     hv_dev->desc.virtual_numa_node));
 
 		put_pcichild(hv_dev);

}
--
2.25.1