Skip to contentSkip to conversation

PCI: pci_resource_alignment() improvement + cleanups

PATCH Applied Ilpo Järvinen 2 participants 15 messages Apr 29 – Jun 12

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover PCI: pci_resource_alignment() improvement + cleanups

pci_resource_alignment() returns 0 when resource is already assigned and in case of disabled bridge windows. This has caused problems to calculations relying on pci_resource_alignment():

https://lore.kernel.org/linux-pci/LV8P221MB1472A24B9975F7C8E8D6BF929947A@LV8P221MB1472.NAMP221.PROD.OUTLOOK.COM/

This series reworks pci_resource_alignment() interface to return always non-zero alignment if the resource exists. For assigned bridge windows, the calculation is using heuristic based on size and start address alignment as calculating the alignment again is costly (would require sizing the entire sub-hierarchy).

As pci_resource_alignment() is becoming more complicated, it's also moved to setup-res.c. While moving pci_resource_alignment()'s arguments are converted into const to tell compiler it can rely on resource remaining the same across the call.

This was intended to be part of a larger series that addresses some shortcomings in pci=realloc. The pci=realloc changes will recalculate bridge window sizes considering also assigned resources which required making these changes to pci_resource_alignment().

As this also relates to the issue linked above, I'm sending it already now without pci=realloc changes that are still incomplete. The first patches originate from the large pci=realloc work but seem generally useful even if independent of the alignment improvements so I've included them here without reorganizing the series to contain only alignment related changes.

Ilpo Järvinen (11):
PCI: Log all resource claims
PCI: Rename added to add_list
PCI: Consolidate add_list (aka realloc_head) empty sanity checks
PCI: Remove const removal cast
resource: Make resource_alignment() input const resource powerpc/pseries: Make pseries_get_iov_fw_value() & pnv_iov_get()
pci_dev const
PCI: Make pci_sriov_resource_alignment() pci_dev const
PCI: Convert pci_resource_alignment() input parameters to const
PCI: Move pci_resource_alignment() to setup-res.c file
PCI: Lower bound bridge windown alignment
PCI: Return valid alignment for assigned resources

 arch/powerpc/include/asm/machdep.h         |  2 +-
 arch/powerpc/kernel/pci-common.c           |  2 +-
 arch/powerpc/platforms/powernv/pci-sriov.c |  4 +-
 arch/powerpc/platforms/powernv/pci.h       |  5 ++-
 arch/powerpc/platforms/pseries/setup.c     |  5 ++-
 drivers/pci/iov.c                          |  7 +--
 drivers/pci/pci.h                          | 24 ++++-------
 drivers/pci/setup-bus.c                    | 50 ++++++++++++----------
 drivers/pci/setup-cardbus.c                |  2 +-
 drivers/pci/setup-res.c                    | 37 ++++++++++++++++
 include/linux/ioport.h                     |  2 +-
 include/linux/pci.h                        |  8 ++--
 kernel/resource.c                          |  2 +-
 13 files changed, 94 insertions(+), 56 deletions(-)

base-commit: 254f49634ee16a731174d2ae34bc50bd5f45e731

Signature · 1 line
2.39.5
#1 lore raw

PATCH 1/11 PCI: Log all resource claims

There are two ways to graft resource into resource tree in PCI, pci_assign_resource() and pci_claim_resource(). Only the former logs the action which complicated troubleshooting the cases where resources are assigned by pci_claim_resource() which mostly assigns the addresses inherited from the FW.

Add logging into pci_claim_resource() to make troubleshooting easier.

---
 drivers/pci/setup-res.c | 2 ++
 1 file changed, 2 insertions(+)
drivers/pci/setup-res.c +2 -0
index fbc05cda96ee..0d203325562b 100644
@@ -167,6 +167,8 @@ int pci_claim_resource(struct pci_dev *dev, int resource)
 		return -EBUSY;
 	}
 
+	pci_dbg(dev, "%s %pR: claiming\n", res_name, res);
+
 	return 0;
 }
 EXPORT_SYMBOL(pci_claim_resource);
Signature · 1 line
2.39.5
#2 lore raw

PATCH 2/11 PCI: Rename added to add_list

In reply to Ilpo Järvinen

The resource fitting algorithm uses different names from the list holding the optional sizes: added, add_head, add_list, and realloc_head. 'add_list' sounds the most natural and some of the related variables also use 'add' such as 'add_size'.

To reduce variation, rename 'added' and 'add_head' to 'add_list'. Also rename some 'realloc_head' cases selectively to 'add_list'.

While it would be nice to rename every 'realloc_head' to 'add_list' for consistency, it might create a backport headache with all to work going into this algorithm that may need to be eventually backported. Thus, it's better to leave 'realloc_head' as is for now.

---
 drivers/pci/pci.h       |  2 +-
 drivers/pci/setup-bus.c | 26 +++++++++++++-------------
 2 files changed, 14 insertions(+), 14 deletions(-)
drivers/pci/pci.h +1 -1
index 4a14f88e543a..4fcf5a25ad9e 100644
@@ -515,7 +515,7 @@ int pci_dev_res_add_to_list(struct list_head *head, struct pci_dev *dev,
 void __pci_bus_size_bridges(struct pci_bus *bus,
 			struct list_head *realloc_head);
 void __pci_bus_assign_resources(const struct pci_bus *bus,
-				struct list_head *realloc_head,
+				struct list_head *add_list,
 				struct list_head *fail_head);
 bool pci_bus_clip_resource(struct pci_dev *dev, int idx);
 void pci_walk_bus_locked(struct pci_bus *top,
drivers/pci/setup-bus.c +13 -13
index 4cf120ebe5ad..3765693e95f0 100644
@@ -756,13 +756,13 @@ static void __assign_resources_sorted(struct list_head *head,
 }
 
 static void pdev_assign_resources_sorted(struct pci_dev *dev,
-					 struct list_head *add_head,
+					 struct list_head *add_list,
 					 struct list_head *fail_head)
 {
 	LIST_HEAD(head);
 
 	pdev_sort_resources(dev, &head);
-	__assign_resources_sorted(&head, add_head, fail_head);
+	__assign_resources_sorted(&head, add_list, fail_head);
 
 }
 
@@ -1502,13 +1502,13 @@ static void pdev_assign_fixed_resources(struct pci_dev *dev)
 }
 
 void __pci_bus_assign_resources(const struct pci_bus *bus,
-				struct list_head *realloc_head,
+				struct list_head *add_list,
 				struct list_head *fail_head)
 {
 	struct pci_bus *b;
 	struct pci_dev *dev;
 
-	pbus_assign_resources_sorted(bus, realloc_head, fail_head);
+	pbus_assign_resources_sorted(bus, add_list, fail_head);
 
 	list_for_each_entry(dev, &bus->devices, bus_list) {
 		pdev_assign_fixed_resources(dev);
@@ -1517,7 +1517,7 @@ void __pci_bus_assign_resources(const struct pci_bus *bus,
 		if (!b)
 			continue;
 
-		__pci_bus_assign_resources(b, realloc_head, fail_head);
+		__pci_bus_assign_resources(b, add_list, fail_head);
 
 		switch (dev->hdr_type) {
 		case PCI_HEADER_TYPE_BRIDGE:
@@ -1613,19 +1613,19 @@ void pci_bus_claim_resources(struct pci_bus *b)
 EXPORT_SYMBOL(pci_bus_claim_resources);
 
 static void __pci_bridge_assign_resources(const struct pci_dev *bridge,
-					  struct list_head *add_head,
+					  struct list_head *add_list,
 					  struct list_head *fail_head)
 {
 	struct pci_bus *b;
 
 	pdev_assign_resources_sorted((struct pci_dev *)bridge,
-					 add_head, fail_head);
+				     add_list, fail_head);
 
 	b = bridge->subordinate;
 	if (!b)
 		return;
 
-	__pci_bus_assign_resources(b, add_head, fail_head);
+	__pci_bus_assign_resources(b, add_list, fail_head);
 
 	switch (bridge->class >> 8) {
 	case PCI_CLASS_BRIDGE_PCI:
@@ -2303,7 +2303,7 @@ static int pbus_reassign_bridge_resources(struct pci_bus *bus, struct resource *
 	unsigned long type = res->flags;
 	struct pci_dev_resource *dev_res;
 	struct pci_dev *bridge = NULL;
-	LIST_HEAD(added);
+	LIST_HEAD(add_list);
 	LIST_HEAD(failed);
 	unsigned int i;
 	int ret = 0;
@@ -2337,10 +2337,10 @@ static int pbus_reassign_bridge_resources(struct pci_bus *bus, struct resource *
 	if (!bridge)
 		return -ENOENT;
 
-	__pci_bus_size_bridges(bridge->subordinate, &added);
-	__pci_bridge_assign_resources(bridge, &added, &failed);
-	if (WARN_ON_ONCE(!list_empty(&added)))
-		pci_dev_res_free_list(&added);
+	__pci_bus_size_bridges(bridge->subordinate, &add_list);
+	__pci_bridge_assign_resources(bridge, &add_list, &failed);
+	if (WARN_ON_ONCE(!list_empty(&add_list)))
+		pci_dev_res_free_list(&add_list);
 
 	if (!list_empty(&failed)) {
 		if (pci_required_resource_failed(&failed, type))
Signature · 1 line
2.39.5
#3 lore raw

PATCH 3/11 PCI: Consolidate add_list (aka realloc_head) empty sanity checks

In reply to Ilpo Järvinen

Callers of __pci_bridge_assign_resources() and __pci_bus_assign_resources() perform WARN_ON_ONCE(list_empty(add_list))) checks to sanity check that all optional sizes were processed (and removed) from the list. The empty list sanity check is duplicated code so the more appropriate place for it would be inside the called function.

Placing the empty list check into __pci_bus_assign_resources() also ensures all callsites do perform the sanity check which currently is not the case when being called from enable_slot(). This inconsistency was noted by Sashiko though only inside its in depth log but not flagged as a real problem, possibly because this is only a sanity check that should never fire. Nonetheless, this sanity check has been very useful to catch problems early in the past so it's good to do it consistenty everywhere.

As __pci_bus_assign_resources() is recursive function, it needs to be renamed to __pci_bus_assign_resources_one() to only perform the empty list check at the end of processing the entire hierarchy in __pci_bus_assign_resources().

Suggested-by sashiko.dev # Sanity check missing from enable_slot()
---
 drivers/pci/setup-bus.c | 29 +++++++++++++++++------------
 1 file changed, 17 insertions(+), 12 deletions(-)
drivers/pci/setup-bus.c +17 -12
index 3765693e95f0..1e0e28efe8b8 100644
@@ -1501,9 +1501,9 @@ static void pdev_assign_fixed_resources(struct pci_dev *dev)
 	}
 }
 
-void __pci_bus_assign_resources(const struct pci_bus *bus,
-				struct list_head *add_list,
-				struct list_head *fail_head)
+static void __pci_bus_assign_resources_one(const struct pci_bus *bus,
+					   struct list_head *add_list,
+					   struct list_head *fail_head)
 {
 	struct pci_bus *b;
 	struct pci_dev *dev;
@@ -1517,7 +1517,7 @@ void __pci_bus_assign_resources(const struct pci_bus *bus,
 		if (!b)
 			continue;
 
-		__pci_bus_assign_resources(b, add_list, fail_head);
+		__pci_bus_assign_resources_one(b, add_list, fail_head);
 
 		switch (dev->hdr_type) {
 		case PCI_HEADER_TYPE_BRIDGE:
@@ -1537,6 +1537,16 @@ void __pci_bus_assign_resources(const struct pci_bus *bus,
 	}
 }
 
+void __pci_bus_assign_resources(const struct pci_bus *bus,
+				struct list_head *add_list,
+				struct list_head *fail_head)
+{
+	__pci_bus_assign_resources_one(bus, add_list, fail_head);
+
+	if (WARN_ON_ONCE(add_list && !list_empty(add_list)))
+		pci_dev_res_free_list(add_list);
+}
+
 void pci_bus_assign_resources(const struct pci_bus *bus)
 {
 	__pci_bus_assign_resources(bus, NULL, NULL);
@@ -1641,6 +1651,9 @@ static void __pci_bridge_assign_resources(const struct pci_dev *bridge,
 			 pci_domain_nr(b), b->number);
 		break;
 	}
+
+	if (WARN_ON_ONCE(add_list && !list_empty(add_list)))
+		pci_dev_res_free_list(add_list);
 }
 
 static void pci_bridge_release_resources(struct pci_bus *bus,
@@ -2205,8 +2218,6 @@ void pci_assign_unassigned_root_bus_resources(struct pci_bus *bus)
 
 		/* Depth last, allocate resources and update the hardware. */
 		__pci_bus_assign_resources(bus, add_list, &fail_head);
-		if (WARN_ON_ONCE(add_list && !list_empty(add_list)))
-			pci_dev_res_free_list(add_list);
 		tried_times++;
 
 		/* Any device complain? */
@@ -2268,8 +2279,6 @@ void pci_assign_unassigned_bridge_resources(struct pci_dev *bridge)
 		pci_bridge_distribute_available_resources(bridge, &add_list);
 
 		__pci_bridge_assign_resources(bridge, &add_list, &fail_head);
-		if (WARN_ON_ONCE(!list_empty(&add_list)))
-			pci_dev_res_free_list(&add_list);
 		tried_times++;
 
 		if (list_empty(&fail_head))
@@ -2339,8 +2348,6 @@ static int pbus_reassign_bridge_resources(struct pci_bus *bus, struct resource *
 
 	__pci_bus_size_bridges(bridge->subordinate, &add_list);
 	__pci_bridge_assign_resources(bridge, &add_list, &failed);
-	if (WARN_ON_ONCE(!list_empty(&add_list)))
-		pci_dev_res_free_list(&add_list);
 
 	if (!list_empty(&failed)) {
 		if (pci_required_resource_failed(&failed, type))
@@ -2473,7 +2480,5 @@ void pci_assign_unassigned_bus_resources(struct pci_bus *bus)
 			__pci_bus_size_bridges(dev->subordinate, &add_list);
 	up_read(&pci_bus_sem);
 	__pci_bus_assign_resources(bus, &add_list, NULL);
-	if (WARN_ON_ONCE(!list_empty(&add_list)))
-		pci_dev_res_free_list(&add_list);
 }
 EXPORT_SYMBOL_GPL(pci_assign_unassigned_bus_resources);
Signature · 1 line
2.39.5
#4 lore raw

PATCH 4/11 PCI: Remove const removal cast

In reply to Ilpo Järvinen

__pci_bridge_assign_resources() inputs const pci_dev *bridge, but then immediately casts const away to pass the bridge to pdev_assign_resources_sorted().

As pdev_assign_resources_sorted() performs assignment of resources, it is not possible to make its input parameter to const. Neither of the __pci_bridge_assign_resources() callers requires the bridge parameter to be const.

Thus, simply remove the out of place cast and convert the input parameter to non-const.

---
 drivers/pci/setup-bus.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)
drivers/pci/setup-bus.c +2 -3
index 1e0e28efe8b8..c0a949f2c995 100644
@@ -1622,14 +1622,13 @@ void pci_bus_claim_resources(struct pci_bus *b)
 }
 EXPORT_SYMBOL(pci_bus_claim_resources);
 
-static void __pci_bridge_assign_resources(const struct pci_dev *bridge,
+static void __pci_bridge_assign_resources(struct pci_dev *bridge,
 					  struct list_head *add_list,
 					  struct list_head *fail_head)
 {
 	struct pci_bus *b;
 
-	pdev_assign_resources_sorted((struct pci_dev *)bridge,
-				     add_list, fail_head);
+	pdev_assign_resources_sorted(bridge, add_list, fail_head);
 
 	b = bridge->subordinate;
 	if (!b)
Signature · 1 line
2.39.5
#5 lore raw

PATCH 5/11 resource: Make resource_alignment() input const resource

In reply to Ilpo Järvinen

resource_alignment() does not need to change resource so it can be made const.

---
 include/linux/ioport.h | 2 +-
 kernel/resource.c      | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
include/linux/ioport.h +1 -1
index 3c73c9c0d4f7..f7930b3dfd0a 100644
@@ -261,7 +261,7 @@ extern int allocate_resource(struct resource *root, struct resource *new,
 struct resource *lookup_resource(struct resource *root, resource_size_t start);
 int adjust_resource(struct resource *res, resource_size_t start,
 		    resource_size_t size);
-resource_size_t resource_alignment(struct resource *res);
+resource_size_t resource_alignment(const struct resource *res);
 
 /**
  * resource_set_size - Calculate resource end address from size and start
kernel/resource.c +1 -1
index d02a53fb95d8..3d17e3196a3e 100644
@@ -1238,7 +1238,7 @@ reserve_region_with_split(struct resource *root, resource_size_t start,
  *
  * Returns alignment on success, 0 (invalid alignment) on failure.
  */
-resource_size_t resource_alignment(struct resource *res)
+resource_size_t resource_alignment(const struct resource *res)
 {
 	switch (res->flags & (IORESOURCE_SIZEALIGN | IORESOURCE_STARTALIGN)) {
 	case IORESOURCE_SIZEALIGN:
Signature · 1 line
2.39.5
#6 lore raw

PATCH 6/11 powerpc/pseries: Make pseries_get_iov_fw_value() & pnv_iov_get() pci_dev const

In reply to Ilpo Järvinen

Convert input pci_dev for pseries_get_iov_fw_value() and pnv_iov_get() to const to be able to convert pcibios_iov_resource_alignment() as well in an upcoming change.

---
 arch/powerpc/platforms/powernv/pci.h   | 2 +-
 arch/powerpc/platforms/pseries/setup.c | 3 ++-
 2 files changed, 3 insertions(+), 2 deletions(-)
arch/powerpc/platforms/powernv/pci.h +1 -1
index 42075501663b..032b2081aedb 100644
@@ -251,7 +251,7 @@ struct pnv_iov_data {
 	struct resource holes[PCI_SRIOV_NUM_BARS];
 };
 
-static inline struct pnv_iov_data *pnv_iov_get(struct pci_dev *pdev)
+static inline struct pnv_iov_data *pnv_iov_get(const struct pci_dev *pdev)
 {
 	return pdev->dev.archdata.iov_data;
 }
arch/powerpc/platforms/pseries/setup.c +2 -1
index 50b26ed8432d..b670c6fdfcea 100644
@@ -658,7 +658,8 @@ enum get_iov_fw_value_index {
 	WDW_SIZE      = 3     /*  Get Window Size */
 };
 
-static resource_size_t pseries_get_iov_fw_value(struct pci_dev *dev, int resno,
+static resource_size_t pseries_get_iov_fw_value(const struct pci_dev *dev,
+						int resno,
 						enum get_iov_fw_value_index value)
 {
 	const int *indexes;
Signature · 1 line
2.39.5
#7 lore raw

PATCH 7/11 PCI: Make pci_sriov_resource_alignment() pci_dev const

In reply to Ilpo Järvinen

pci_sriov_resource_alignment() inputs struct pci_dev which it should not need to alter to calculate alignment.

Make pci_dev pci_sriov_resource_alignment() inputs const. It requires making pci_iov_resource_size() input const as well.

---
 arch/powerpc/include/asm/machdep.h         | 2 +-
 arch/powerpc/kernel/pci-common.c           | 2 +-
 arch/powerpc/platforms/powernv/pci-sriov.c | 4 ++--
 arch/powerpc/platforms/powernv/pci.h       | 3 ++-
 arch/powerpc/platforms/pseries/setup.c     | 2 +-
 drivers/pci/iov.c                          | 7 ++++---
 drivers/pci/pci.h                          | 5 +++--
 include/linux/pci.h                        | 8 +++++---
 8 files changed, 19 insertions(+), 14 deletions(-)
arch/powerpc/include/asm/machdep.h +1 -1
index 3298eec123a3..256f9309bf4f 100644
@@ -169,7 +169,7 @@ struct machdep_calls {
 
 #ifdef CONFIG_PCI_IOV
 	void (*pcibios_fixup_sriov)(struct pci_dev *pdev);
-	resource_size_t (*pcibios_iov_resource_alignment)(struct pci_dev *, int resno);
+	resource_size_t (*pcibios_iov_resource_alignment)(const struct pci_dev *, int resno);
 	int (*pcibios_sriov_enable)(struct pci_dev *pdev, u16 num_vfs);
 	int (*pcibios_sriov_disable)(struct pci_dev *pdev);
 #endif /* CONFIG_PCI_IOV */
arch/powerpc/kernel/pci-common.c +1 -1
index 8efe95a0c4ff..3c4ca90e2ab7 100644
@@ -254,7 +254,7 @@ resource_size_t pcibios_default_alignment(void)
 }
 
 #ifdef CONFIG_PCI_IOV
-resource_size_t pcibios_iov_resource_alignment(struct pci_dev *pdev, int resno)
+resource_size_t pcibios_iov_resource_alignment(const struct pci_dev *pdev, int resno)
 {
 	if (ppc_md.pcibios_iov_resource_alignment)
 		return ppc_md.pcibios_iov_resource_alignment(pdev, resno);
arch/powerpc/platforms/powernv/pci-sriov.c +2 -2
index 7105a573aec4..8652078801f2 100644
@@ -244,8 +244,8 @@ void pnv_pci_ioda_fixup_iov(struct pci_dev *pdev)
 	}
 }
 
-resource_size_t pnv_pci_iov_resource_alignment(struct pci_dev *pdev,
-						      int resno)
+resource_size_t pnv_pci_iov_resource_alignment(const struct pci_dev *pdev,
+					       int resno)
 {
 	resource_size_t align = pci_iov_resource_size(pdev, resno);
 	struct pnv_phb *phb = pci_bus_to_pnvhb(pdev->bus);
arch/powerpc/platforms/powernv/pci.h +2 -1
index 032b2081aedb..3ac718d471c2 100644
@@ -257,7 +257,8 @@ static inline struct pnv_iov_data *pnv_iov_get(const struct pci_dev *pdev)
 }
 
 void pnv_pci_ioda_fixup_iov(struct pci_dev *pdev);
-resource_size_t pnv_pci_iov_resource_alignment(struct pci_dev *pdev, int resno);
+resource_size_t pnv_pci_iov_resource_alignment(const struct pci_dev *pdev,
+					       int resno);
 
 int pnv_pcibios_sriov_enable(struct pci_dev *pdev, u16 num_vfs);
 int pnv_pcibios_sriov_disable(struct pci_dev *pdev);
arch/powerpc/platforms/pseries/setup.c +1 -1
index b670c6fdfcea..1223dc961242 100644
@@ -789,7 +789,7 @@ static void pseries_pci_fixup_iov_resources(struct pci_dev *pdev)
 		pseries_disable_sriov_resources(pdev);
 }
 
-static resource_size_t pseries_pci_iov_resource_alignment(struct pci_dev *pdev,
+static resource_size_t pseries_pci_iov_resource_alignment(const struct pci_dev *pdev,
 							  int resno)
 {
 	const __be32 *reg;
drivers/pci/iov.c +4 -3
index 91ac4e37ecb9..c86409835f73 100644
@@ -150,7 +150,7 @@ static void virtfn_remove_bus(struct pci_bus *physbus, struct pci_bus *virtbus)
 		pci_remove_bus(virtbus);
 }
 
-resource_size_t pci_iov_resource_size(struct pci_dev *dev, int resno)
+resource_size_t pci_iov_resource_size(const struct pci_dev *dev, int resno)
 {
 	if (!dev->is_physfn)
 		return 0;
@@ -1084,7 +1084,7 @@ void pci_iov_update_resource(struct pci_dev *dev, int resno)
 	}
 }
 
-resource_size_t __weak pcibios_iov_resource_alignment(struct pci_dev *dev,
+resource_size_t __weak pcibios_iov_resource_alignment(const struct pci_dev *dev,
 						      int resno)
 {
 	return pci_iov_resource_size(dev, resno);
@@ -1100,7 +1100,8 @@ resource_size_t __weak pcibios_iov_resource_alignment(struct pci_dev *dev,
  * the VF BAR size multiplied by the number of VFs.  The alignment
  * is just the VF BAR size.
  */
-resource_size_t pci_sriov_resource_alignment(struct pci_dev *dev, int resno)
+resource_size_t pci_sriov_resource_alignment(const struct pci_dev *dev,
+					     int resno)
 {
 	return pcibios_iov_resource_alignment(dev, resno);
 }
drivers/pci/pci.h +3 -2
index 4fcf5a25ad9e..710803be3a79 100644
@@ -947,7 +947,8 @@ int pci_iov_init(struct pci_dev *dev);
 void pci_iov_release(struct pci_dev *dev);
 void pci_iov_remove(struct pci_dev *dev);
 void pci_iov_update_resource(struct pci_dev *dev, int resno);
-resource_size_t pci_sriov_resource_alignment(struct pci_dev *dev, int resno);
+resource_size_t pci_sriov_resource_alignment(const struct pci_dev *dev,
+					     int resno);
 void pci_restore_iov_state(struct pci_dev *dev);
 int pci_iov_bus_range(struct pci_bus *bus);
 void pci_iov_resource_set_size(struct pci_dev *dev, int resno, int size);
@@ -981,7 +982,7 @@ static inline int pci_iov_init(struct pci_dev *dev)
 static inline void pci_iov_release(struct pci_dev *dev) { }
 static inline void pci_iov_remove(struct pci_dev *dev) { }
 static inline void pci_iov_update_resource(struct pci_dev *dev, int resno) { }
-static inline resource_size_t pci_sriov_resource_alignment(struct pci_dev *dev,
+static inline resource_size_t pci_sriov_resource_alignment(const struct pci_dev *dev,
 							   int resno)
 {
 	return 0;
include/linux/pci.h +5 -3
index 2c4454583c11..0ced3bbd08c0 100644
@@ -2540,7 +2540,7 @@ int pci_vfs_assigned(struct pci_dev *dev);
 int pci_sriov_set_totalvfs(struct pci_dev *dev, u16 numvfs);
 int pci_sriov_get_totalvfs(struct pci_dev *dev);
 int pci_sriov_configure_simple(struct pci_dev *dev, int nr_virtfn);
-resource_size_t pci_iov_resource_size(struct pci_dev *dev, int resno);
+resource_size_t pci_iov_resource_size(const struct pci_dev *dev, int resno);
 int pci_iov_vf_bar_set_size(struct pci_dev *dev, int resno, int size);
 u32 pci_iov_vf_bar_get_sizes(struct pci_dev *dev, int resno, int num_vfs);
 void pci_vf_drivers_autoprobe(struct pci_dev *dev, bool probe);
@@ -2548,7 +2548,8 @@ void pci_vf_drivers_autoprobe(struct pci_dev *dev, bool probe);
 /* Arch may override these (weak) */
 int pcibios_sriov_enable(struct pci_dev *pdev, u16 num_vfs);
 int pcibios_sriov_disable(struct pci_dev *pdev);
-resource_size_t pcibios_iov_resource_alignment(struct pci_dev *dev, int resno);
+resource_size_t pcibios_iov_resource_alignment(const struct pci_dev *dev,
+					       int resno);
 #else
 static inline int pci_iov_virtfn_bus(struct pci_dev *dev, int id)
 {
@@ -2593,7 +2594,8 @@ static inline int pci_sriov_set_totalvfs(struct pci_dev *dev, u16 numvfs)
 static inline int pci_sriov_get_totalvfs(struct pci_dev *dev)
 { return 0; }
 #define pci_sriov_configure_simple	NULL
-static inline resource_size_t pci_iov_resource_size(struct pci_dev *dev, int resno)
+static inline resource_size_t pci_iov_resource_size(const struct pci_dev *dev,
+						    int resno)
 { return 0; }
 static inline int pci_iov_vf_bar_set_size(struct pci_dev *dev, int resno, int size)
 { return -ENODEV; }
Signature · 1 line
2.39.5
#8 lore raw

PATCH 8/11 PCI: Convert pci_resource_alignment() input parameters to const

In reply to Ilpo Järvinen

pci_resource_alignment() calculates resource alignment and should not alter its input structs. Make its input parameters const.

It requires making also pci_cardbus_resource_alignment() input const.

---
 drivers/pci/pci.h           | 8 ++++----
 drivers/pci/setup-cardbus.c | 2 +-
 2 files changed, 5 insertions(+), 5 deletions(-)
drivers/pci/pci.h +4 -4
index 710803be3a79..e0fcc33dfef6 100644
@@ -419,7 +419,7 @@ static inline bool pci_is_cardbus_bridge(struct pci_dev *dev)
 	return dev->hdr_type == PCI_HEADER_TYPE_CARDBUS;
 }
 #ifdef CONFIG_CARDBUS
-unsigned long pci_cardbus_resource_alignment(struct resource *res);
+unsigned long pci_cardbus_resource_alignment(const struct resource *res);
 int pci_bus_size_cardbus_bridge(struct pci_bus *bus,
 				struct list_head *realloc_head);
 int pci_cardbus_scan_bridge_extend(struct pci_bus *bus, struct pci_dev *dev,
@@ -428,7 +428,7 @@ int pci_cardbus_scan_bridge_extend(struct pci_bus *bus, struct pci_dev *dev,
 int pci_setup_cardbus(char *str);
 
 #else
-static inline unsigned long pci_cardbus_resource_alignment(struct resource *res)
+static inline unsigned long pci_cardbus_resource_alignment(const struct resource *res)
 {
 	return 0;
 }
@@ -1044,8 +1044,8 @@ static inline void pci_suspend_ptm(struct pci_dev *dev) { }
 static inline void pci_resume_ptm(struct pci_dev *dev) { }
 #endif
 
-static inline resource_size_t pci_resource_alignment(struct pci_dev *dev,
-						     struct resource *res)
+static inline resource_size_t pci_resource_alignment(const struct pci_dev *dev,
+						     const struct resource *res)
 {
 	int resno = pci_resource_num(dev, res);
 
drivers/pci/setup-cardbus.c +1 -1
index 1ebd13a1f730..0cba404080ad 100644
@@ -22,7 +22,7 @@
 static unsigned long pci_cardbus_io_size = DEFAULT_CARDBUS_IO_SIZE;
 static unsigned long pci_cardbus_mem_size = DEFAULT_CARDBUS_MEM_SIZE;
 
-unsigned long pci_cardbus_resource_alignment(struct resource *res)
+unsigned long pci_cardbus_resource_alignment(const struct resource *res)
 {
 	if (res->flags & IORESOURCE_IO)
 		return pci_cardbus_io_size;
Signature · 1 line
2.39.5
#9 lore raw

PATCH 9/11 PCI: Move pci_resource_alignment() to setup-res.c file

In reply to Ilpo Järvinen

pci_resource_alignment() is a bit one the complex side to have in a header so put it into setup-res.c.

---
 drivers/pci/pci.h       | 13 ++-----------
 drivers/pci/setup-res.c | 12 ++++++++++++
 2 files changed, 14 insertions(+), 11 deletions(-)
drivers/pci/pci.h +2 -11
index e0fcc33dfef6..472b6c2f7c4d 100644
@@ -1044,17 +1044,8 @@ static inline void pci_suspend_ptm(struct pci_dev *dev) { }
 static inline void pci_resume_ptm(struct pci_dev *dev) { }
 #endif
 
-static inline resource_size_t pci_resource_alignment(const struct pci_dev *dev,
-						     const struct resource *res)
-{
-	int resno = pci_resource_num(dev, res);
-
-	if (pci_resource_is_iov(resno))
-		return pci_sriov_resource_alignment(dev, resno);
-	if (dev->class >> 8 == PCI_CLASS_BRIDGE_CARDBUS)
-		return pci_cardbus_resource_alignment(res);
-	return resource_alignment(res);
-}
+resource_size_t pci_resource_alignment(const struct pci_dev *dev,
+				       const struct resource *res);
 
 resource_size_t pci_min_window_alignment(struct pci_bus *bus,
 					 unsigned long type);
drivers/pci/setup-res.c +12 -0
index 0d203325562b..18e8775ea848 100644
@@ -246,6 +246,18 @@ static int pci_revert_fw_address(struct resource *res, struct pci_dev *dev,
 	return 0;
 }
 
+resource_size_t pci_resource_alignment(const struct pci_dev *dev,
+				       const struct resource *res)
+{
+	int resno = pci_resource_num(dev, res);
+
+	if (pci_resource_is_iov(resno))
+		return pci_sriov_resource_alignment(dev, resno);
+	if (dev->class >> 8 == PCI_CLASS_BRIDGE_CARDBUS)
+		return pci_cardbus_resource_alignment(res);
+	return resource_alignment(res);
+}
+
 /*
  * For mem bridge windows, try to relocate tail remainder space to space
  * before res->start if there's enough free space there. This enables
Signature · 1 line
2.39.5
#10 lore raw

PATCH 10/11 PCI: Lower bound bridge window alignment

In reply to Ilpo Järvinen

pci_resource_alignment() does not consider bridge windows special, yet their alignment is subject to different requirements from BAR alignment.

Add lower bound to bridge window alignment to help callers out to always have large enough alignment.

---
 drivers/pci/setup-res.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)
drivers/pci/setup-res.c +11 -1
index 18e8775ea848..c15bce20815d 100644
@@ -19,7 +19,10 @@
 #include <linux/errno.h>
 #include <linux/ioport.h>
 #include <linux/cache.h>
+#include <linux/minmax.h>
 #include <linux/slab.h>
+#include <linux/types.h>
+
 #include "pci.h"
 
 static void pci_std_update_resource(struct pci_dev *dev, int resno)
@@ -250,12 +253,19 @@ resource_size_t pci_resource_alignment(const struct pci_dev *dev,
 				       const struct resource *res)
 {
 	int resno = pci_resource_num(dev, res);
+	resource_size_t min_align = 0;
 
 	if (pci_resource_is_iov(resno))
 		return pci_sriov_resource_alignment(dev, resno);
+
 	if (dev->class >> 8 == PCI_CLASS_BRIDGE_CARDBUS)
 		return pci_cardbus_resource_alignment(res);
-	return resource_alignment(res);
+
+	if (pci_resource_is_bridge_win(resno) &&
+	    (res->flags & (IORESOURCE_IO|IORESOURCE_MEM)))
+		min_align = pci_min_window_alignment(dev->bus, res->flags);
+
+	return max(resource_alignment(res), min_align);
 }
 
 /*
Signature · 1 line
2.39.5
#11 lore raw

Re: [PATCH 10/11] PCI: Lower bound bridge window alignment

Quoted 8 lines · Ilpo Järvinen

pci_resource_alignment() does not consider bridge windows special, yet their alignment is subject to different requirements from BAR alignment.

Add lower bound to bridge window alignment to help callers out to always have large enough alignment.

Hi Bjorn,

Could you please pull this change and the subsequent one, the commits in pci/resource:

ae09d28ecbbc ("PCI: Lower bound bridge window alignment") cf996b886675 ("PCI: Return valid alignment for assigned resources")

The rest of the changes in this series seem okay and can proceed if you're okay with keeping a partial series.

The reason for this request is the dev->bus vs dev->subordinate issue sashiko mentioned in its review.

I've tried to come up with a solution to that but it has become so ugly I have not been very happy with it. Maybe there's no other way but the problem boils down with pci_min_window_alignment() having to be capable of dealing two cases, each lacking one of the key pointers (bus or bridge dev):

1) root bus without bus->self
2) bridge without a subordinate bus struct (if subordinate bus' alloc failed)

Only way to solve that I could think of is passing both the bus and the bridge device to pci_min_window_alignment() and the related arch side function pcibios_window_alignment(). The ugliest parts then involve getting a bus pointer compatible with both of those cases, like this:

	struct pci_controller *phb = pci_bus_to_host(bus ? bus : bridge->bus);

But it's too late in the cycle now to try even that, IMO, so better to wait to the next cycle.

Signature · 1 line
 i.
#12 lore raw

Re: [PATCH 10/11] PCI: Lower bound bridge window alignment

Quoted 11 lines · Ilpo Järvinen
Quoted 8 lines · Ilpo Järvinen

pci_resource_alignment() does not consider bridge windows special, yet their alignment is subject to different requirements from BAR alignment.

Add lower bound to bridge window alignment to help callers out to always have large enough alignment.

Quoted 7 lines

Hi Bjorn,

Could you please pull this change and the subsequent one, the commits in pci/resource:

ae09d28ecbbc ("PCI: Lower bound bridge window alignment") cf996b886675 ("PCI: Return valid alignment for assigned resources")

Sure, I dropped these two patches from pci/resource. The rest are still heading to v7.2. Thanks!

Quoted 28 lines

The rest of the changes in this series seem okay and can proceed if you're okay with keeping a partial series.

The reason for this request is the dev->bus vs dev->subordinate issue sashiko mentioned in its review.

I've tried to come up with a solution to that but it has become so ugly I have not been very happy with it. Maybe there's no other way but the problem boils down with pci_min_window_alignment() having to be capable of dealing two cases, each lacking one of the key pointers (bus or bridge dev):

1) root bus without bus->self
2) bridge without a subordinate bus struct (if subordinate bus' alloc failed)

Only way to solve that I could think of is passing both the bus and the bridge device to pci_min_window_alignment() and the related arch side function pcibios_window_alignment(). The ugliest parts then involve getting a bus pointer compatible with both of those cases, like this:

	struct pci_controller *phb = pci_bus_to_host(bus ? bus : bridge->bus);

But it's too late in the cycle now to try even that, IMO, so better to wait to the next cycle.

Signature · 1 line
 i.
#13 lore raw

PATCH 11/11 PCI: Return valid alignment for assigned resources

In reply to Ilpo Järvinen

When a resource is assigned, IORESOURCE_STARTALIGN flag is cleared, resulting in pci_resource_alignment() returning 0 in some situations (at least for bridge windows).

Add heuristic to pci_resource_alignment() which mimics start and size alignment by taking minimum of those for an assigned resource. It may overestimate alignment when start has large alignment but the exact alignment information is not available and regenerating it by sizing the bridge again is costly.

---
 drivers/pci/setup-res.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)
drivers/pci/setup-res.c +13 -0
index c15bce20815d..03098f159ec9 100644
@@ -13,6 +13,7 @@
  *	     Resource sorting
  */
 
+#include <linux/bitops.h>
 #include <linux/kernel.h>
 #include <linux/export.h>
 #include <linux/pci.h>
@@ -265,6 +266,18 @@ resource_size_t pci_resource_alignment(const struct pci_dev *dev,
 	    (res->flags & (IORESOURCE_IO|IORESOURCE_MEM)))
 		min_align = pci_min_window_alignment(dev->bus, res->flags);
 
+	if (resource_assigned(res)) {
+		resource_size_t start_align = 1, size_align;
+
+		size_align = roundup_pow_of_two(resource_size(res));
+		if (res->start)
+			start_align <<= __ffs(res->start);
+		else
+			start_align = size_align;
+
+		return max(min(start_align, size_align), min_align);
+	}
+
 	return max(resource_alignment(res), min_align);
 }
 
Signature · 1 line
2.39.5
#14 lore raw
In reply to Ilpo Järvinen
Quoted 43 lines · Ilpo Järvinen

pci_resource_alignment() returns 0 when resource is already assigned and in case of disabled bridge windows. This has caused problems to calculations relying on pci_resource_alignment():

https://lore.kernel.org/linux-pci/LV8P221MB1472A24B9975F7C8E8D6BF929947A@LV8P221MB1472.NAMP221.PROD.OUTLOOK.COM/

This series reworks pci_resource_alignment() interface to return always non-zero alignment if the resource exists. For assigned bridge windows, the calculation is using heuristic based on size and start address alignment as calculating the alignment again is costly (would require sizing the entire sub-hierarchy).

As pci_resource_alignment() is becoming more complicated, it's also moved to setup-res.c. While moving pci_resource_alignment()'s arguments are converted into const to tell compiler it can rely on resource remaining the same across the call.

This was intended to be part of a larger series that addresses some shortcomings in pci=realloc. The pci=realloc changes will recalculate bridge window sizes considering also assigned resources which required making these changes to pci_resource_alignment().

As this also relates to the issue linked above, I'm sending it already now without pci=realloc changes that are still incomplete. The first patches originate from the large pci=realloc work but seem generally useful even if independent of the alignment improvements so I've included them here without reorganizing the series to contain only alignment related changes.

Ilpo Järvinen (11):
PCI: Log all resource claims
PCI: Rename added to add_list
PCI: Consolidate add_list (aka realloc_head) empty sanity checks
PCI: Remove const removal cast
resource: Make resource_alignment() input const resource powerpc/pseries: Make pseries_get_iov_fw_value() & pnv_iov_get()
pci_dev const
PCI: Make pci_sriov_resource_alignment() pci_dev const
PCI: Convert pci_resource_alignment() input parameters to const
PCI: Move pci_resource_alignment() to setup-res.c file
PCI: Lower bound bridge windown alignment
PCI: Return valid alignment for assigned resources

Quoted 14 lines
 arch/powerpc/include/asm/machdep.h         |  2 +-
 arch/powerpc/kernel/pci-common.c           |  2 +-
 arch/powerpc/platforms/powernv/pci-sriov.c |  4 +-
 arch/powerpc/platforms/powernv/pci.h       |  5 ++-
 arch/powerpc/platforms/pseries/setup.c     |  5 ++-
 drivers/pci/iov.c                          |  7 +--
 drivers/pci/pci.h                          | 24 ++++-------
 drivers/pci/setup-bus.c                    | 50 ++++++++++++----------
 drivers/pci/setup-cardbus.c                |  2 +-
 drivers/pci/setup-res.c                    | 37 ++++++++++++++++
 include/linux/ioport.h                     |  2 +-
 include/linux/pci.h                        |  8 ++--
 kernel/resource.c                          |  2 +-
 13 files changed, 94 insertions(+), 56 deletions(-)

Applied to pci/resource for v7.2, thanks!

Powerpc folks, the arch/powerpc changes here are minor (just adding const), but heads up in case you see any issue.