Skip to contentSkip to conversation

Add support for the PCI host bridge device-tree node creation.

PATCH Superseded Herve Codina 1 participant 7 messages Nov 14, 2024

Subsystems (3)

v3 of 8 Superseded Latest is v8 →

lore
Share & tools
lore
#0 lore raw

v3 cover Add support for the PCI host bridge device-tree node creation.

Hi,

This series adds support for creating a device-tree node for the PCI host bridge on non device-tree based system.

Creating device-tree nodes for PCI devices and PCI-PCI bridges already exists upstream. It was added in commit 407d1a51921e ("PCI: Create device tree node for bridge"). Created device-tree nodes need a parent node to be attached to. For the first level devices, on device-tree based system, this parent node (i.e. the PCI host bridge) is described in the base device-tree. The PCI bus related to this bridge (PCI root bus) inherit of the PCI host bridge device-tree node.

The LAN966x PCI device driver was recently accepted [1] and relies on this feature.

On system where the base hardware is not described by a device-tree, the PCI host bridge to which first level created PCI devices need to be attach to does not exist. This is the case for instance on ACPI described systems such as x86.

This series goal is to handle this case.

In order to have the PCI host bridge device-tree node available even on x86, this top level node is created (if not already present) based on information computed by the PCI core. It follows the same mechanism as the one used for PCI devices device-tree node creation.

As for device-tree based system, the PCI root bus handled by the PCI host bridge inherit of this created node.

In order to have this feature available, a number of changes are needed:
- Patch 1 and 2: Introduce and use device_{add,remove}_of_node(). This function will also be used in the root PCI bus node creation.

- Patch 3 and 4: Improve existing functions to reuse them in the root PCI bus node creation.

- Patch 5: Set #address-cells and #size-cells in the empty device-tree root node.

- Patch 6: The PCI host bridge device-tree node creation itself.

With those modifications, the LAN966x PCI device is working on x86 systems and all device-tree kunit tests (including the of_unittest_pci_node test) pass successfully with the following command:

  qemu-system-x86_64 -machine q35 -nographic \
    -kernel arch/x86_64/boot/bzImage --append console=ttyS0 \
    -device pcie-root-port,port=0x10,chassis=9,id=pci.9,bus=pcie.0,multifunction=on,addr=0x3 \
    -device pcie-root-port,port=0x11,chassis=10,id=pci.10,bus=pcie.0,addr=0x3.0x1 \
    -device x3130-upstream,id=pci.11,bus=pci.9,addr=0x0 \
    -device xio3130-downstream,port=0x0,chassis=11,id=pci.12,bus=pci.11,multifunction=on,addr=0x0 \
    -device i82801b11-bridge,id=pci.13,bus=pcie.0,addr=0x4 \
    -device pci-bridge,chassis_nr=14,id=pci.14,bus=pci.13,addr=0x0 \
    -device pci-testdev,bus=pci.12,addr=0x0

[1] https://lore.kernel.org/lkml/[email protected]/

Compare to previous iteration, this v3 series fixes patch 5 commit log and sets #size-cells to 2 in the empty root DT node.

Best regards,
Hervé Codina

Changes v2 -> v3
v2: https://lore.kernel.org/lkml/[email protected]/

- Patch 5
Fix commit log.
Use 2 for #size-cells.

- Patches 1 to 4 and 6
No changes

Changes v1 -> v2
v1: https://lore.kernel.org/lkml/[email protected]/

- Patch 1
Remove Cc: stable

- Patch 2
Remove Fixup tag and Cc: stable

- Patches 3 and 4
No changes

- Patch 5
Add #address-cells/#size-cells in the empty root DT node instead of updating default values for x86.
Update commit log and commit title.

- Patch 6
Create device-tree node for the PCI host bridge and reuse it for the PCI root bus. Rename functions accordingly.
Use "pci" instead of "pci-root" for the PCI host bridge node name. Use "res->start - windows->offset" for the PCI bus addresses. Update commit log and commit title.

Herve Codina (6):
driver core: Introduce device_{add,remove}_of_node()
PCI: of: Use device_{add,remove}_of_node() to attach of_node to
existing device
PCI: of_property: Add support for NULL pdev in of_pci_set_address()
PCI: of_property: Constify parameter in of_pci_get_addr_flags()
of: Add #address-cells/#size-cells in the device-tree root empty node
PCI: of: Create device-tree PCI host bridge node

 drivers/base/core.c       |  52 +++++++++++++++++
 drivers/of/empty_root.dts |   9 ++-
 drivers/pci/of.c          |  98 +++++++++++++++++++++++++++++++-
 drivers/pci/of_property.c | 114 ++++++++++++++++++++++++++++++++++++--
 drivers/pci/pci.h         |   6 ++
 drivers/pci/probe.c       |   2 +
 drivers/pci/remove.c      |   2 +
 include/linux/device.h    |   2 +
 8 files changed, 277 insertions(+), 8 deletions(-)
Signature · 1 line
2.47.0
#1 lore raw

PATCH v3 1/6 driver core: Introduce device_{add,remove}_of_node()

An of_node can be set to a device using device_set_node(). This function cannot prevent any of_node and/or fwnode overwrites.

When adding an of_node on an already present device, the following operations need to be done:
- Attach the of_node if no of_node were already attached
- Attach the of_node as a fwnode if no fwnode were already attached

This is the purpose of device_add_of_node(). device_remove_of_node() reverts the operations done by device_add_of_node().

---
 drivers/base/core.c    | 52 ++++++++++++++++++++++++++++++++++++++++++
 include/linux/device.h |  2 ++
 2 files changed, 54 insertions(+)
drivers/base/core.c +52 -0
index 24c572031403..0aa63371f55d 100644
@@ -5118,6 +5118,58 @@ void set_secondary_fwnode(struct device *dev, struct fwnode_handle *fwnode)
 }
 EXPORT_SYMBOL_GPL(set_secondary_fwnode);
 
+/**
+ * device_remove_of_node - Remove an of_node from a device
+ * @dev: device whose device-tree node is being removed
+ */
+void device_remove_of_node(struct device *dev)
+{
+	dev = get_device(dev);
+	if (!dev)
+		return;
+
+	if (!dev->of_node)
+		goto end;
+
+	if (dev->fwnode == of_fwnode_handle(dev->of_node))
+		dev->fwnode = NULL;
+
+	of_node_put(dev->of_node);
+	dev->of_node = NULL;
+
+end:
+	put_device(dev);
+}
+EXPORT_SYMBOL_GPL(device_remove_of_node);
+
+/**
+ * device_add_of_node - Add an of_node to an existing device
+ * @dev: device whose device-tree node is being added
+ * @of_node: of_node to add
+ */
+void device_add_of_node(struct device *dev, struct device_node *of_node)
+{
+	if (!of_node)
+		return;
+
+	dev = get_device(dev);
+	if (!dev)
+		return;
+
+	if (WARN(dev->of_node, "%s: Cannot replace node %pOF with %pOF\n",
+		 dev_name(dev), dev->of_node, of_node))
+		goto end;
+
+	dev->of_node = of_node_get(of_node);
+
+	if (!dev->fwnode)
+		dev->fwnode = of_fwnode_handle(of_node);
+
+end:
+	put_device(dev);
+}
+EXPORT_SYMBOL_GPL(device_add_of_node);
+
 /**
  * device_set_of_node_from_dev - reuse device-tree node of another device
  * @dev: device whose device-tree node is being set
include/linux/device.h +2 -0
index b4bde8d22697..e3aa25ce1f90 100644
@@ -1146,6 +1146,8 @@ int device_online(struct device *dev);
 void set_primary_fwnode(struct device *dev, struct fwnode_handle *fwnode);
 void set_secondary_fwnode(struct device *dev, struct fwnode_handle *fwnode);
 void device_set_node(struct device *dev, struct fwnode_handle *fwnode);
+void device_add_of_node(struct device *dev, struct device_node *of_node);
+void device_remove_of_node(struct device *dev);
 void device_set_of_node_from_dev(struct device *dev, const struct device *dev2);
 
 static inline struct device_node *dev_of_node(struct device *dev)
Signature · 1 line
2.47.0
#2 lore raw

PATCH v3 2/6 PCI: of: Use device_{add,remove}_of_node() to attach of_node to existing device

In reply to Herve Codina

The commit 407d1a51921e ("PCI: Create device tree node for bridge") creates of_node for PCI devices. The newly created of_node is attached to an existing device. This is done setting directly pdev->dev.of_node in the code.

Even if pdev->dev.of_node cannot be previously set, this doesn't handle the fwnode field of the struct device. Indeed, this field needs to be set if it hasn't already been set.

device_{add,remove}_of_node() have been introduced to handle this case.

Use them instead of the direct setting.

---
 drivers/pci/of.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/pci/of.c +2 -2
index dacea3fc5128..141ffbb1b3e6 100644
@@ -655,8 +655,8 @@ void of_pci_remove_node(struct pci_dev *pdev)
 	np = pci_device_to_OF_node(pdev);
 	if (!np || !of_node_check_flag(np, OF_DYNAMIC))
 		return;
-	pdev->dev.of_node = NULL;
 
+	device_remove_of_node(&pdev->dev);
 	of_changeset_revert(np->data);
 	of_changeset_destroy(np->data);
 	of_node_put(np);
@@ -713,7 +713,7 @@ void of_pci_make_dev_node(struct pci_dev *pdev)
 		goto out_free_node;
 
 	np->data = cset;
-	pdev->dev.of_node = np;
+	device_add_of_node(&pdev->dev, np);
 	kfree(name);
 
 	return;
Signature · 1 line
2.47.0
#3 lore raw

PATCH v3 3/6 PCI: of_property: Add support for NULL pdev in of_pci_set_address()

In reply to Herve Codina

The pdev (pointer to a struct pci_dev) parameter of of_pci_set_address() cannot be NULL.

In order to reuse of_pci_set_address() when creating the PCI root bus node, this function needs to support a NULL pdev parameter. Indeed, in the case of the PCI root bus node creation, no pdev are available and of_pci_set_address() will be used with the bridge windows.

Allow to call of_pci_set_address() with a NULL pdev.

---
 drivers/pci/of_property.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)
drivers/pci/of_property.c +7 -3
index 5a0b98e69795..59cc5c851fc3 100644
@@ -54,9 +54,13 @@ enum of_pci_prop_compatible {
 static void of_pci_set_address(struct pci_dev *pdev, u32 *prop, u64 addr,
 			       u32 reg_num, u32 flags, bool reloc)
 {
-	prop[0] = FIELD_PREP(OF_PCI_ADDR_FIELD_BUS, pdev->bus->number) |
-		FIELD_PREP(OF_PCI_ADDR_FIELD_DEV, PCI_SLOT(pdev->devfn)) |
-		FIELD_PREP(OF_PCI_ADDR_FIELD_FUNC, PCI_FUNC(pdev->devfn));
+	if (pdev)
+		prop[0] = FIELD_PREP(OF_PCI_ADDR_FIELD_BUS, pdev->bus->number) |
+			  FIELD_PREP(OF_PCI_ADDR_FIELD_DEV, PCI_SLOT(pdev->devfn)) |
+			  FIELD_PREP(OF_PCI_ADDR_FIELD_FUNC, PCI_FUNC(pdev->devfn));
+	else
+		prop[0] = 0;
+
 	prop[0] |= flags | reg_num;
 	if (!reloc) {
 		prop[0] |= OF_PCI_ADDR_FIELD_NONRELOC;
Signature · 1 line
2.47.0
#4 lore raw

PATCH v3 4/6 PCI: of_property: Constify parameter in of_pci_get_addr_flags()

In reply to Herve Codina

The res parameter has no reason to be a pointer to an un-const struct resource. Indeed, struct resource is not supposed to be modified by the function.

Constify the res parameter.

---
 drivers/pci/of_property.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/pci/of_property.c +1 -1
index 59cc5c851fc3..e56159cc48e8 100644
@@ -69,7 +69,7 @@ static void of_pci_set_address(struct pci_dev *pdev, u32 *prop, u64 addr,
 	}
 }
 
-static int of_pci_get_addr_flags(struct resource *res, u32 *flags)
+static int of_pci_get_addr_flags(const struct resource *res, u32 *flags)
 {
 	u32 ss;
 
Signature · 1 line
2.47.0
#5 lore raw

PATCH v3 5/6 of: Add #address-cells/#size-cells in the device-tree root empty node

In reply to Herve Codina

On systems where ACPI is enabled or when a device-tree is not passed to the kernel by the bootloader, a device-tree root empty node is created. This device-tree root empty node does not have the #address-cells and the #size-cells properties

This leads to the use of the default address cells and size cells values which are defined in the code to 1 for the address cells value and 1 for the size cells value.

According to the devicetree specification and the OpenFirmware standard (IEEE 1275-1994) the default value for #address-cells should be 2.

Also, according to the devicetree specification, the #address-cells and the #size-cells are required properties in the root node.

The device tree compiler already uses 2 as default value for address cells and 1 for size cells. The powerpc PROM code also uses 2 as default value for address cells and 1 for size cells. Modern implementation should have the #address-cells and the #size-cells properties set and should not rely on default values.

On x86, this root empty node is used and the code default values are used.

In preparation of the support for device-tree overlay on PCI devices feature on x86 (i.e. the creation of the PCI root bus device-tree node), the default value for #address-cells needs to be updated. Indeed, on x86_64, addresses are on 64bits and the upper part of an address is needed for correct address translations. On x86_32 having the default value updated does not lead to issues while the upper part of a 64-bit value is zero.

Changing the default value for all architectures may break device-tree compatibility. Indeed, existing dts file without the #address-cells property set in the root node will not be compatible with this modification.

Instead of updating default values, add both required #address-cells and #size-cells properties in the device-tree empty node.

Use 2 for both properties value in order to fully support 64-bit addresses and sizes on systems using this empty root node.

---
 drivers/of/empty_root.dts | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
drivers/of/empty_root.dts +8 -1
index cf9e97a60f48..cbe169ba3db5 100644
@@ -2,5 +2,12 @@
 /dts-v1/;
 
 / {
-
+	/*
+	 * #address-cells/#size-cells are required properties at root node.
+	 * Use 2 cells for both address cells and size cells in order to fully
+	 * support 64-bit addresses and sizes on systems using this empty root
+	 * node.
+	 */
+	#address-cells = <0x02>;
+	#size-cells = <0x02>;
 };
Signature · 1 line
2.47.0
#6 lore raw

PATCH v3 6/6 PCI: of: Create device-tree PCI host bridge node

In reply to Herve Codina

PCI devices device-tree nodes can be already created. This was introduced by commit 407d1a51921e ("PCI: Create device tree node for bridge").

In order to have device-tree nodes related to PCI devices attached on their PCI root bus (the PCI bus handled by the PCI host bridge), a PCI root bus device-tree node is needed. This root bus node will be used as the parent node of the first level devices scanned on the bus. On device-tree based systems, this PCI root bus device tree node is set to the node of the related PCI host bridge. The PCI host bridge node is available in the device-tree used to describe the hardware passed at boot.

On non device-tree based system (such as ACPI), a device-tree node for the PCI host bridge or for the root bus do not exist. Indeed, the PCI host bridge is not described in a device-tree used at boot simply because no device-tree are passed at boot.

The device-tree PCI host bridge node creation needs to be done at runtime. This is done in the same way as for the creation of the PCI device nodes. I.e. node and properties are created based on computed information done by the PCI core. Also, as is done on device-tree based systems, this PCI host bridge node is used for the PCI root bus.

---
 drivers/pci/of.c          |  94 ++++++++++++++++++++++++++++++++++-
 drivers/pci/of_property.c | 102 ++++++++++++++++++++++++++++++++++++++
 drivers/pci/pci.h         |   6 +++
 drivers/pci/probe.c       |   2 +
 drivers/pci/remove.c      |   2 +
 5 files changed, 205 insertions(+), 1 deletion(-)
drivers/pci/of.c +93 -1
index 141ffbb1b3e6..316c31d13aa8 100644
@@ -726,7 +726,99 @@ void of_pci_make_dev_node(struct pci_dev *pdev)
 out_free_name:
 	kfree(name);
 }
-#endif
+
+void of_pci_remove_host_bridge_node(struct pci_host_bridge *bridge)
+{
+	struct device_node *np;
+
+	np = pci_bus_to_OF_node(bridge->bus);
+	if (!np || !of_node_check_flag(np, OF_DYNAMIC))
+		return;
+
+	device_remove_of_node(&bridge->bus->dev);
+	device_remove_of_node(&bridge->dev);
+	of_changeset_revert(np->data);
+	of_changeset_destroy(np->data);
+	of_node_put(np);
+}
+
+void of_pci_make_host_bridge_node(struct pci_host_bridge *bridge)
+{
+	struct device_node *np = NULL;
+	struct of_changeset *cset;
+	const char *name;
+	int ret;
+
+	/*
+	 * If there is already a device-tree node linked to the PCI bus handled
+	 * by this bridge (i.e. the PCI root bus), nothing to do.
+	 */
+	if (pci_bus_to_OF_node(bridge->bus))
+		return;
+
+	/* The root bus has no node. Check that the host bridge has no node too */
+	if (bridge->dev.of_node) {
+		pr_err("PCI host bridge of_node already set");
+		return;
+	}
+
+	/* Check if there is a DT root node to attach the created node */
+	if (!of_root) {
+		pr_err("of_root node is NULL, cannot create PCI host bridge node\n");
+		return;
+	}
+
+	name = kasprintf(GFP_KERNEL, "pci@%x,%x", pci_domain_nr(bridge->bus),
+			 bridge->bus->number);
+	if (!name)
+		return;
+
+	cset = kmalloc(sizeof(*cset), GFP_KERNEL);
+	if (!cset)
+		goto out_free_name;
+	of_changeset_init(cset);
+
+	np = of_changeset_create_node(cset, of_root, name);
+	if (!np)
+		goto out_destroy_cset;
+
+	ret = of_pci_add_host_bridge_properties(bridge, cset, np);
+	if (ret)
+		goto out_free_node;
+
+	/*
+	 * This of_node will be added to an existing device. The of_node parent
+	 * is the root OF node and so this node will be handled by the platform
+	 * bus. Avoid any new device creation.
+	 */
+	of_node_set_flag(np, OF_POPULATED);
+	np->fwnode.dev = &bridge->dev;
+	fwnode_dev_initialized(&np->fwnode, true);
+
+	ret = of_changeset_apply(cset);
+	if (ret)
+		goto out_free_node;
+
+	np->data = cset;
+
+	/* Add the of_node to host bridge and the root bus */
+	device_add_of_node(&bridge->dev, np);
+	device_add_of_node(&bridge->bus->dev, np);
+
+	kfree(name);
+
+	return;
+
+out_free_node:
+	of_node_put(np);
+out_destroy_cset:
+	of_changeset_destroy(cset);
+	kfree(cset);
+out_free_name:
+	kfree(name);
+}
+
+#endif /* CONFIG_PCI_DYNAMIC_OF_NODES */
 
 #endif /* CONFIG_PCI */
 
drivers/pci/of_property.c +102 -0

105 lines not on this page · raw message

drivers/pci/pci.h +6 -0
index 14d00ce45bfa..f2092a119f2c 100644
@@ -802,9 +802,15 @@ void of_pci_make_dev_node(struct pci_dev *pdev);
 void of_pci_remove_node(struct pci_dev *pdev);
 int of_pci_add_properties(struct pci_dev *pdev, struct of_changeset *ocs,
 			  struct device_node *np);
+void of_pci_make_host_bridge_node(struct pci_host_bridge *bridge);
+void of_pci_remove_host_bridge_node(struct pci_host_bridge *bridge);
+int of_pci_add_host_bridge_properties(struct pci_host_bridge *bridge, struct of_changeset *ocs,
+				      struct device_node *np);
 #else
 static inline void of_pci_make_dev_node(struct pci_dev *pdev) { }
 static inline void of_pci_remove_node(struct pci_dev *pdev) { }
+static inline void of_pci_make_host_bridge_node(struct pci_host_bridge *bridge) { }
+static inline void of_pci_remove_host_bridge_node(struct pci_host_bridge *bridge) { }
 #endif
 
 #ifdef CONFIG_PCIEAER
drivers/pci/probe.c +2 -0
index 4f68414c3086..bfe89ce8d800 100644
@@ -1049,6 +1049,8 @@ static int pci_register_host_bridge(struct pci_host_bridge *bridge)
 		dev_info(&bus->dev, "root bus resource %pR%s\n", res, addr);
 	}
 
+	of_pci_make_host_bridge_node(bridge);
+
 	down_write(&pci_bus_sem);
 	list_add_tail(&bus->node, &pci_root_buses);
 	up_write(&pci_bus_sem);
drivers/pci/remove.c +2 -0
index e4ce1145aa3e..2c379c9ab5b4 100644
@@ -160,6 +160,8 @@ void pci_stop_root_bus(struct pci_bus *bus)
 					 &bus->devices, bus_list)
 		pci_stop_bus_device(child);
 
+	of_pci_remove_host_bridge_node(host_bridge);
+
 	/* stop the host bridge */
 	device_release_driver(&host_bridge->dev);
 }
Signature · 1 line
2.47.0