Skip to contentSkip to conversation

PCI : bridge resource reallocation patchset -- followup

PATCH In review Yinghai Lu 2 participants 6 messages Feb 24, 2012

Subsystems (2)
lore
Share & tools
lore

Yinghai Lu

author
#0 lore raw

cover PCI : bridge resource reallocation patchset -- followup

e084882: PCI: only enable pci realloc when SRIOV bar is not assigned 22f7543: PCI: print out suggestion about using pci=realloc
c81bbda: PCI: Make pci bridge reallocating enabled/disabled
b1b520d: PCI: Retry on type IORESOURCE_IO allocation.

are left over after Jesse pickup more of them.
it will try to auto detect if need to use pci=realloc, and print out suggestion.

could get from

git://git.kernel.org/pub/scm/linux/kernel/git/yinghai/linux-yinghai.git for-pci-next

-v3: Resend according to Jesse. Update ack and etc.
last one add CONFIG_PCI_REALLOC_ENABLE_DETECT according to Jesse.
-v4: left over that need refreshing against pci/linux-next...
resending accoring to Jesse.

Thanks

Yinghai

 Documentation/kernel-parameters.txt |    9 +++-
 drivers/pci/Kconfig                 |   10 +++++
 drivers/pci/pci.c                   |    4 +-
 drivers/pci/pci.h                   |    2 +-
 drivers/pci/setup-bus.c             |   75 +++++++++++++++++++++++++++--------
 5 files changed, 79 insertions(+), 21 deletions(-)
author
#1 lore raw

PATCH 1/4 PCI: Retry on type IORESOURCE_IO allocation.

During reenabling pci reallocating for pci bridge by clean the small size in bridge and assign with requested + optional size for first several try, Ram mention could have problem with one case

	https://bugzilla.kernel.org/show_bug.cgi?id=15960

After checking the booting log in

	https://lkml.org/lkml/2010/4/19/44
	[regression, bisected] Xonar DX invalid PCI I/O range since 977d17bb174

We should not stop too early for io ports.

	Apr 19 10:19:38 [kernel] pci 0000:04:00.0: BAR 7: can't assign io (size 0x4000)
	Apr 19 10:19:38 [kernel] pci 0000:05:01.0: BAR 8: assigned [mem 0x80400000-0x805fffff]
	Apr 19 10:19:38 [kernel] pci 0000:05:01.0: BAR 7: can't assign io (size 0x2000)
	Apr 19 10:19:38 [kernel] pci 0000:05:02.0: BAR 7: can't assign io (size 0x1000)
	Apr 19 10:19:38 [kernel] pci 0000:05:03.0: BAR 7: can't assign io (size 0x1000)
	Apr 19 10:19:38 [kernel] pci 0000:08:00.0: BAR 7: can't assign io (size 0x1000)
	Apr 19 10:19:38 [kernel] pci 0000:09:04.0: BAR 0: can't assign io (size 0x100)

and clear 00:1c.0 to retry again.

The patch remove the IORESOUCE_IO checking, and try one more time. and we will have chance to get allocation for 00:1c.0 io port range because from 0x4000 to 0x8000 could be used.

---
 drivers/pci/setup-bus.c |   12 +-----------
 1 files changed, 1 insertions(+), 11 deletions(-)
drivers/pci/setup-bus.c +1 -11
index 2991a89..162edfb 100644
@@ -1292,7 +1292,6 @@ pci_assign_unassigned_resources(void)
 	struct pci_dev_resource *fail_res;
 	unsigned long type_mask = IORESOURCE_IO | IORESOURCE_MEM |
 				  IORESOURCE_PREFETCH;
-	unsigned long failed_type;
 	int pci_try_num = 1;
 
 	/* don't realloc if asked to do so */
@@ -1327,16 +1326,7 @@ again:
 	if (list_empty(&fail_head))
 		goto enable_and_dump;
 
-	failed_type = 0;
-	list_for_each_entry(fail_res, &fail_head, list)
-		failed_type |= fail_res->flags;
-
-	/*
-	 * io port are tight, don't try extra
-	 * or if reach the limit, don't want to try more
-	 */
-	failed_type &= type_mask;
-	if ((failed_type == IORESOURCE_IO) || (tried_times >= pci_try_num)) {
+	if (tried_times >= pci_try_num) {
 		free_list(&fail_head);
 		goto enable_and_dump;
 	}
Signature · 1 line
1.7.7
author
#2 lore raw

PATCH 2/4 PCI: Make pci bridge reallocating enabled/disabled

In reply to Yinghai Lu

Let the user could enable and disable with pci=realloc=on or pci=realloc=off

Also
1. move variable and functions near the place they are used.
2. change macro to function
3. change related functions and variable to static and _init
4. update parameter description accordingly.

-v2: still honor pci=realloc, and treat it as pci=realloc=on
also use enum instead of ...
-v3: update kernel-paramenters.txt according to Jesse.

---
 Documentation/kernel-parameters.txt |    9 +++++++--
 drivers/pci/pci.c                   |    4 +++-
 drivers/pci/pci.h                   |    2 +-
 drivers/pci/setup-bus.c             |   34 +++++++++++++++++++++++++++-------
 4 files changed, 38 insertions(+), 11 deletions(-)
Documentation/kernel-parameters.txt +7 -2
index 7fb7a4b..7dc523e 100644
@@ -2109,8 +2109,13 @@ bytes respectively. Such letter suffixes can also be entirely omitted.
 				the default.
 				off: Turn ECRC off
 				on: Turn ECRC on.
-		realloc		reallocate PCI resources if allocations done by BIOS
-				are erroneous.
+		realloc=	Enable/disable reallocating PCI bridge resources
+				if allocations done by BIOS are too small to
+				accommodate resources required by all child
+				devices.
+				off: Turn realloc off
+				on: Turn realloc on
+		realloc		same as realloc=on
 
 	pcie_aspm=	[PCIE] Forcibly enable or disable PCIe Active State Power
 			Management.
drivers/pci/pci.c +3 -1
index 8f30736..e9f9dc1 100644
@@ -3772,8 +3772,10 @@ static int __init pci_setup(char *str)
 				pci_no_msi();
 			} else if (!strcmp(str, "noaer")) {
 				pci_no_aer();
+			} else if (!strncmp(str, "realloc=", 8)) {
+				pci_realloc_get_opt(str + 8);
 			} else if (!strncmp(str, "realloc", 7)) {
-				pci_realloc();
+				pci_realloc_get_opt("on");
 			} else if (!strcmp(str, "nodomains")) {
 				pci_no_domains();
 			} else if (!strncmp(str, "cbiosize=", 9)) {
drivers/pci/pci.h +1 -1
index 586ac9b..1fc63b3 100644
@@ -149,7 +149,7 @@ static inline void pci_no_msi(void) { }
 static inline void pci_msi_init_pci_dev(struct pci_dev *dev) { }
 #endif
 
-extern void pci_realloc(void);
+void pci_realloc_get_opt(char *);
 
 static inline int pci_no_d1d2(struct pci_dev *dev)
 {
drivers/pci/setup-bus.c +27 -7
index 162edfb..219722d 100644
@@ -48,13 +48,6 @@ static void free_list(struct list_head *head)
 	}
 }
 
-int pci_realloc_enable = 0;
-#define pci_realloc_enabled() pci_realloc_enable
-void pci_realloc(void)
-{
-	pci_realloc_enable = 1;
-}
-
 /**
  * add_to_list() - add a new resource tracker to the list
  * @head:	Head of the list
@@ -1273,6 +1266,33 @@ static int __init pci_get_max_depth(void)
 	return depth;
 }
 
+/*
+ * -1: undefined, will auto detect later
+ *  0: disabled by user
+ *  1: disabled by auto detect
+ *  2: enabled by user
+ *  3: enabled by auto detect
+ */
+enum enable_type {
+	undefined = -1,
+	user_disabled,
+	auto_disabled,
+	user_enabled,
+	auto_enabled,
+};
+
+static enum enable_type pci_realloc_enable __initdata = undefined;
+void __init pci_realloc_get_opt(char *str)
+{
+	if (!strncmp(str, "off", 3))
+		pci_realloc_enable = user_disabled;
+	else if (!strncmp(str, "on", 2))
+		pci_realloc_enable = user_enabled;
+}
+static bool __init pci_realloc_enabled(void)
+{
+	return pci_realloc_enable >= user_enabled;
+}
 
 /*
  * first try will not touch pci bridge res
Signature · 1 line
1.7.7
author
#3 lore raw

PATCH 3/4 PCI: print out suggestion about using pci=realloc

In reply to Yinghai Lu

let user know they could try if pci=realloc could help.

-v2: update suggestion text.

---
 drivers/pci/setup-bus.c |    3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)
drivers/pci/setup-bus.c +3 -0
index 219722d..e21e1c2 100644
@@ -1347,6 +1347,9 @@ again:
 		goto enable_and_dump;
 
 	if (tried_times >= pci_try_num) {
+		if (pci_realloc_enable == undefined)
+			printk(KERN_INFO "Some PCI device resources are unassigned, try booting with pci=realloc\n");
+
 		free_list(&fail_head);
 		goto enable_and_dump;
 	}
Signature · 1 line
1.7.7
author
#4 lore raw

PATCH 4/4 PCI: only enable pci realloc when SRIOV bar is not assigned

In reply to Yinghai Lu

If bios does not assign those BAR or wrong address, then kernel will try to do pci realloc.

in that case, user still can use pci=realloc=off to override it.

-v2: According to Jesse, adding one CONFIG option for distribution to
disable it or enable it.

---
 drivers/pci/Kconfig     |   10 ++++++++++
 drivers/pci/setup-bus.c |   28 ++++++++++++++++++++++++++++
 2 files changed, 38 insertions(+), 0 deletions(-)
drivers/pci/Kconfig +10 -0
index 37856f7..771a037 100644
@@ -31,6 +31,16 @@ config PCI_DEBUG
 
 	  When in doubt, say N.
 
+config PCI_REALLOC_ENABLE_AUTO
+	bool "PCI Realloc Enable Auto"
+	depends on PCI
+	help
+	  Say Y here if you want the PCI core to detect if pci realloc
+	  need to be enabled. You can always use pci=realloc=on or
+	  pci=realloc=off to override it.
+
+	  When in doubt, say N.
+
 config PCI_STUB
 	tristate "PCI Stub driver"
 	depends on PCI
drivers/pci/setup-bus.c +28 -0
index e21e1c2..c9214a1 100644
@@ -1294,6 +1294,31 @@ static bool __init pci_realloc_enabled(void)
 	return pci_realloc_enable >= user_enabled;
 }
 
+static void __init pci_realloc_detect(void)
+{
+#if defined(CONFIG_PCI_IOV) && defined(CONFIG_PCI_REALLOC_ENABLE_AUTO)
+	struct pci_dev *dev = NULL;
+
+	if (pci_realloc_enable != undefined)
+		return;
+
+	for_each_pci_dev(dev) {
+		int i;
+
+		for (i = PCI_IOV_RESOURCES; i <= PCI_IOV_RESOURCE_END; i++) {
+			struct resource *r = &dev->resource[i];
+
+			/* Not assigned, or rejected by kernel ? */
+			if (r->flags && !r->start) {
+				pci_realloc_enable = auto_enabled;
+
+				return;
+			}
+		}
+	}
+#endif
+}
+
 /*
  * first try will not touch pci bridge res
  * second  and later try will clear small leaf bridge res
@@ -1315,6 +1340,7 @@ pci_assign_unassigned_resources(void)
 	int pci_try_num = 1;
 
 	/* don't realloc if asked to do so */
+	pci_realloc_detect();
 	if (pci_realloc_enabled()) {
 		int max_depth = pci_get_max_depth();
 
@@ -1349,6 +1375,8 @@ again:
 	if (tried_times >= pci_try_num) {
 		if (pci_realloc_enable == undefined)
 			printk(KERN_INFO "Some PCI device resources are unassigned, try booting with pci=realloc\n");
+		else if (pci_realloc_enable == auto_enabled)
+			printk(KERN_INFO "Automatically enabled pci realloc, if you have problem, try booting with pci=realloc=off\n");
 
 		free_list(&fail_head);
 		goto enable_and_dump;
Signature · 1 line
1.7.7
#5 lore raw

Re: [PATCH 4/4] PCI: only enable pci realloc when SRIOV bar is not assigned

Quoted 10 lines · On Thu, 23 Feb 2012 19:23:32 -0800 Yinghai Lu <[email protected]> wrote:

If bios does not assign those BAR or wrong address, then kernel will try to do pci realloc.

in that case, user still can use pci=realloc=off to override it.

-v2: According to Jesse, adding one CONFIG option for distribution to
disable it or enable it.

---

Ok applied these, thanks for the cleanups Yinghai.

Signature · 1 line
Jesse Barnes, Intel Open Source Technology Center