Skip to contentSkip to conversation

PCI: rcar: Add the suspend/resume for pcie-rcar driver

PATCH Applied [email protected] 5 participants 12 messages Mar 15, 2020 – Apr 28, 2020

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

PATCH v3 1/1 PCI: rcar: Add the suspend/resume for pcie-rcar driver

From: Kazufumi Ikeda <[email protected]>

This adds the suspend/resume supports for pcie-rcar. The resume handler reprograms the hardware based on the software state kept in specific device structures. Also it doesn't need to save any registers.

---
V2: - Change return type of rcar_pcie_hw_enable() to void
- Drop default: case in switch statement in rcar_pcie_hw_enable()
- Sort variables in rcar_pcie_resume()
V3: - Update on top of next-20200313

---
 drivers/pci/controller/pcie-rcar.c | 86 +++++++++++++++++++++++++-----
 1 file changed, 74 insertions(+), 12 deletions(-)
drivers/pci/controller/pcie-rcar.c +74 -12
index 759c6542c5c8..f86513638b8a 100644
@@ -452,6 +452,32 @@ static void rcar_pcie_force_speedup(struct rcar_pcie *pcie)
 		 (macsr & LINK_SPEED) == LINK_SPEED_5_0GTS ? "5" : "2.5");
 }
 
+static void rcar_pcie_hw_enable(struct rcar_pcie *pci)
+{
+	struct resource_entry *win;
+	LIST_HEAD(res);
+	int i = 0;
+
+	/* Try setting 5 GT/s link speed */
+	rcar_pcie_force_speedup(pci);
+
+	/* Setup PCI resources */
+	resource_list_for_each_entry(win, &pci->resources) {
+		struct resource *res = win->res;
+
+		if (!res->flags)
+			continue;
+
+		switch (resource_type(res)) {
+		case IORESOURCE_IO:
+		case IORESOURCE_MEM:
+			rcar_pcie_setup_window(i, pci, win);
+			i++;
+			break;
+		}
+	}
+}
+
 static int rcar_pcie_enable(struct rcar_pcie *pcie)
 {
 	struct device *dev = pcie->dev;
@@ -891,11 +917,25 @@ static void rcar_pcie_unmap_msi(struct rcar_pcie *pcie)
 	irq_domain_remove(msi->domain);
 }
 
+static void rcar_pcie_hw_enable_msi(struct rcar_pcie *pcie)
+{
+	struct rcar_msi *msi = &pcie->msi;
+	unsigned long base;
+
+	/* setup MSI data target */
+	base = virt_to_phys((void *)msi->pages);
+
+	rcar_pci_write_reg(pcie, lower_32_bits(base) | MSIFE, PCIEMSIALR);
+	rcar_pci_write_reg(pcie, upper_32_bits(base), PCIEMSIAUR);
+
+	/* enable all MSI interrupts */
+	rcar_pci_write_reg(pcie, 0xffffffff, PCIEMSIIER);
+}
+
 static int rcar_pcie_enable_msi(struct rcar_pcie *pcie)
 {
 	struct device *dev = pcie->dev;
 	struct rcar_msi *msi = &pcie->msi;
-	phys_addr_t base;
 	int err, i;
 
 	mutex_init(&msi->lock);
@@ -934,17 +974,7 @@ static int rcar_pcie_enable_msi(struct rcar_pcie *pcie)
 
 	/* setup MSI data target */
 	msi->pages = __get_free_pages(GFP_KERNEL, 0);
-	if (!msi->pages) {
-		err = -ENOMEM;
-		goto err;
-	}
-	base = virt_to_phys((void *)msi->pages);
-
-	rcar_pci_write_reg(pcie, lower_32_bits(base) | MSIFE, PCIEMSIALR);
-	rcar_pci_write_reg(pcie, upper_32_bits(base), PCIEMSIAUR);
-
-	/* enable all MSI interrupts */
-	rcar_pci_write_reg(pcie, 0xffffffff, PCIEMSIIER);
+	rcar_pcie_hw_enable_msi(pcie);
 
 	return 0;
 
@@ -1219,6 +1249,37 @@ static int rcar_pcie_probe(struct platform_device *pdev)
 	return err;
 }
 
+static int rcar_pcie_resume(struct device *dev)
+{
+	struct rcar_pcie *pcie = dev_get_drvdata(dev);
+	int (*hw_init_fn)(struct rcar_pcie *);
+	unsigned int data;
+	int err;
+
+	err = rcar_pcie_parse_map_dma_ranges(pcie);
+	if (err)
+		return 0;
+
+	/* Failure to get a link might just be that no cards are inserted */
+	hw_init_fn = of_device_get_match_data(dev);
+	err = hw_init_fn(pcie);
+	if (err) {
+		dev_info(dev, "PCIe link down\n");
+		return 0;
+	}
+
+	data = rcar_pci_read_reg(pcie, MACSR);
+	dev_info(dev, "PCIe x%d: link up\n", (data >> 20) & 0x3f);
+
+	/* Enable MSI */
+	if (IS_ENABLED(CONFIG_PCI_MSI))
+		rcar_pcie_hw_enable_msi(pcie);
+
+	rcar_pcie_hw_enable(pcie);
+
+	return 0;
+}
+
 static int rcar_pcie_resume_noirq(struct device *dev)
 {
 	struct rcar_pcie *pcie = dev_get_drvdata(dev);
@@ -1234,6 +1295,7 @@ static int rcar_pcie_resume_noirq(struct device *dev)
 }
 
 static const struct dev_pm_ops rcar_pcie_pm_ops = {
+	SET_SYSTEM_SLEEP_PM_OPS(NULL, rcar_pcie_resume)
 	.resume_noirq = rcar_pcie_resume_noirq,
 };
 
Signature · 1 line
2.25.0
#1 lore raw
Thank you for the patch! Yet something to improve:

Hi,

Thank you for the patch! Yet something to improve:

[auto build test ERROR on pci/next]
[also build test ERROR on v5.6-rc6 next-20200312]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please see https://stackoverflow.com/a/37406982]

url:    https://github.com/0day-ci/linux/commits/marek-vasut-gmail-com/PCI-rcar-Add-the-suspend-resume-for-pcie-rcar-driver/20200317-023853
base:   https://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci.git next
config: parisc-allyesconfig (attached as .config)
compiler: hppa-linux-gcc (GCC) 9.2.0

reproduce:
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
GCC_VERSION=9.2.0 make.cross ARCH=parisc

If you fix the issue, kindly add following tag

All errors (new ones prefixed by >>):

drivers/pci/controller/pcie-rcar.c: In function 'rcar_pcie_hw_enable':

>> drivers/pci/controller/pcie-rcar.c:474:35: error: passing argument 3 of 'rcar_pcie_setup_window' from incompatible pointer type [-Werror=incompatible-pointer-types]
     474 |    rcar_pcie_setup_window(i, pci, win);
         |                                   ^~~
         |                                   |
         |                                   struct resource_entry *
   drivers/pci/controller/pcie-rcar.c:336:25: note: expected 'struct resource *' but argument is of type 'struct resource_entry *'
     336 |        struct resource *res)
         |        ~~~~~~~~~~~~~~~~~^~~

At top level:
drivers/pci/controller/pcie-rcar.c:1252:12: warning: 'rcar_pcie_resume' defined but not used [-Wunused-function]
1252 | static int rcar_pcie_resume(struct device *dev)
| ^~~~~~~~~~~~~~~~
cc1: some warnings being treated as errors

vim +/rcar_pcie_setup_window +474 drivers/pci/controller/pcie-rcar.c

454
455 static void rcar_pcie_hw_enable(struct rcar_pcie *pci)
456 {
457 struct resource_entry *win;
458 LIST_HEAD(res);
459 int i = 0;
460
461 /* Try setting 5 GT/s link speed */
462 rcar_pcie_force_speedup(pci);
463
464 /* Setup PCI resources */
465 resource_list_for_each_entry(win, &pci->resources) {
466 struct resource *res = win->res;
467
468 if (!res->flags)
469 continue;
470
471 switch (resource_type(res)) {
472 case IORESOURCE_IO:
473 case IORESOURCE_MEM:

 > 474				rcar_pcie_setup_window(i, pci, win);

475 i++;
476 break;
477 }
478 }
479 }
480

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

#2 lore raw
In reply to [email protected]
Quoted 113 lines · On Sat, Mar 14, 2020 at 08:12:32PM +0100, [email protected] wrote:

From: Kazufumi Ikeda <[email protected]>

This adds the suspend/resume supports for pcie-rcar. The resume handler reprograms the hardware based on the software state kept in specific device structures. Also it doesn't need to save any registers.

---
V2: - Change return type of rcar_pcie_hw_enable() to void
- Drop default: case in switch statement in rcar_pcie_hw_enable()
- Sort variables in rcar_pcie_resume()
V3: - Update on top of next-20200313

---
 drivers/pci/controller/pcie-rcar.c | 86 +++++++++++++++++++++++++-----
 1 file changed, 74 insertions(+), 12 deletions(-)
drivers/pci/controller/pcie-rcar.c +45 -12
index 759c6542c5c8..f86513638b8a 100644
@@ -452,6 +452,32 @@ static void rcar_pcie_force_speedup(struct rcar_pcie *pcie)
 		 (macsr & LINK_SPEED) == LINK_SPEED_5_0GTS ? "5" : "2.5");
 }
 
+static void rcar_pcie_hw_enable(struct rcar_pcie *pci)
+{
+	struct resource_entry *win;
+	LIST_HEAD(res);
+	int i = 0;
+
+	/* Try setting 5 GT/s link speed */
+	rcar_pcie_force_speedup(pci);
+
+	/* Setup PCI resources */
+	resource_list_for_each_entry(win, &pci->resources) {
+		struct resource *res = win->res;
+
+		if (!res->flags)
+			continue;
+
+		switch (resource_type(res)) {
+		case IORESOURCE_IO:
+		case IORESOURCE_MEM:
+			rcar_pcie_setup_window(i, pci, win);
+			i++;
+			break;
+		}
+	}
+}
+
 static int rcar_pcie_enable(struct rcar_pcie *pcie)
 {
 	struct device *dev = pcie->dev;
@@ -891,11 +917,25 @@ static void rcar_pcie_unmap_msi(struct rcar_pcie *pcie)
 	irq_domain_remove(msi->domain);
 }
 
+static void rcar_pcie_hw_enable_msi(struct rcar_pcie *pcie)
+{
+	struct rcar_msi *msi = &pcie->msi;
+	unsigned long base;
+
+	/* setup MSI data target */
+	base = virt_to_phys((void *)msi->pages);
+
+	rcar_pci_write_reg(pcie, lower_32_bits(base) | MSIFE, PCIEMSIALR);
+	rcar_pci_write_reg(pcie, upper_32_bits(base), PCIEMSIAUR);
+
+	/* enable all MSI interrupts */
+	rcar_pci_write_reg(pcie, 0xffffffff, PCIEMSIIER);
+}
+
 static int rcar_pcie_enable_msi(struct rcar_pcie *pcie)
 {
 	struct device *dev = pcie->dev;
 	struct rcar_msi *msi = &pcie->msi;
-	phys_addr_t base;
 	int err, i;
 
 	mutex_init(&msi->lock);
@@ -934,17 +974,7 @@ static int rcar_pcie_enable_msi(struct rcar_pcie *pcie)
 
 	/* setup MSI data target */
 	msi->pages = __get_free_pages(GFP_KERNEL, 0);
-	if (!msi->pages) {
-		err = -ENOMEM;
-		goto err;
-	}
-	base = virt_to_phys((void *)msi->pages);
-
-	rcar_pci_write_reg(pcie, lower_32_bits(base) | MSIFE, PCIEMSIALR);
-	rcar_pci_write_reg(pcie, upper_32_bits(base), PCIEMSIAUR);
-
-	/* enable all MSI interrupts */
-	rcar_pci_write_reg(pcie, 0xffffffff, PCIEMSIIER);
+	rcar_pcie_hw_enable_msi(pcie);
 
 	return 0;
 
@@ -1219,6 +1249,37 @@ static int rcar_pcie_probe(struct platform_device *pdev)
 	return err;
 }
 
+static int rcar_pcie_resume(struct device *dev)
+{
+	struct rcar_pcie *pcie = dev_get_drvdata(dev);

drivers/pci/controller/pcie-rcar.c static int rcar_pcie_probe(struct platform_device *pdev) · line 1264

Quoted 10 lines
drivers/pci/controller/pcie-rcar.c +10 -0
+	int (*hw_init_fn)(struct rcar_pcie *);
+	unsigned int data;
+	int err;
+
+	err = rcar_pcie_parse_map_dma_ranges(pcie);
+	if (err)
+		return 0;
+
+	/* Failure to get a link might just be that no cards are inserted */
+	hw_init_fn = of_device_get_match_data(dev);

Hi Marek,

happy to apply it as is, I was wondering if it is work taking this look-up out of the resume path, it is not supposed to change anyway, you can even save the function pointer at probe.

Again, happy to apply it as-is, just let me know please.

Thanks,
Lorenzo

Quoted 32 lines
drivers/pci/controller/pcie-rcar.c +19 -0
+	err = hw_init_fn(pcie);
+	if (err) {
+		dev_info(dev, "PCIe link down\n");
+		return 0;
+	}
+
+	data = rcar_pci_read_reg(pcie, MACSR);
+	dev_info(dev, "PCIe x%d: link up\n", (data >> 20) & 0x3f);
+
+	/* Enable MSI */
+	if (IS_ENABLED(CONFIG_PCI_MSI))
+		rcar_pcie_hw_enable_msi(pcie);
+
+	rcar_pcie_hw_enable(pcie);
+
+	return 0;
+}
+
 static int rcar_pcie_resume_noirq(struct device *dev)
 {
 	struct rcar_pcie *pcie = dev_get_drvdata(dev);
@@ -1234,6 +1295,7 @@ static int rcar_pcie_resume_noirq(struct device *dev)
 }
 
 static const struct dev_pm_ops rcar_pcie_pm_ops = {
+	SET_SYSTEM_SLEEP_PM_OPS(NULL, rcar_pcie_resume)
 	.resume_noirq = rcar_pcie_resume_noirq,
 };
 
Signature · 1 line
2.25.0
#3 lore raw

On 3/20/20 11:12 AM, Lorenzo Pieralisi wrote:
[...]

Quoted 7 lines
Quoted 7 lines
drivers/pci/controller/pcie-rcar.c +7 -0
+static int rcar_pcie_resume(struct device *dev)
+{
+	struct rcar_pcie *pcie = dev_get_drvdata(dev);
+	int (*hw_init_fn)(struct rcar_pcie *);
+	unsigned int data;
+	int err;
+
Quoted 8 lines

drivers/pci/controller/pcie-rcar.c static int rcar_pcie_probe(struct platform_device *pdev) · line 1264

Quoted 6 lines
drivers/pci/controller/pcie-rcar.c +6 -0
+	err = rcar_pcie_parse_map_dma_ranges(pcie);
+	if (err)
+		return 0;
+
+	/* Failure to get a link might just be that no cards are inserted */
+	hw_init_fn = of_device_get_match_data(dev);

Hi Marek,

Hi,

Quoted 5 lines

happy to apply it as is, I was wondering if it is work taking this look-up out of the resume path, it is not supposed to change anyway, you can even save the function pointer at probe.

Again, happy to apply it as-is, just let me know please.

I just sent subsequent patch to address this:
PCI: pcie-rcar: Cache PHY init function pointer

#4 lore raw
In reply to [email protected]

drivers/pci/controller/pcie-rcar.c

Quoted 27 lines · On Sat, Mar 14, 2020 at 08:12:32PM +0100, [email protected] wrote:

From: Kazufumi Ikeda <[email protected]>

This adds the suspend/resume supports for pcie-rcar. The resume handler reprograms the hardware based on the software state kept in specific device structures. Also it doesn't need to save any registers.

---
V2: - Change return type of rcar_pcie_hw_enable() to void
- Drop default: case in switch statement in rcar_pcie_hw_enable()
- Sort variables in rcar_pcie_resume()
V3: - Update on top of next-20200313

---
 drivers/pci/controller/pcie-rcar.c | 86 +++++++++++++++++++++++++-----
 1 file changed, 74 insertions(+), 12 deletions(-)
drivers/pci/controller/pcie-rcar.c +0 -0
index 759c6542c5c8..f86513638b8a 100644

Applied to pci/rcar for v5.18, thanks.

Lorenzo

Quoted 128 lines
drivers/pci/controller/pcie-rcar.c +74 -12
@@ -452,6 +452,32 @@ static void rcar_pcie_force_speedup(struct rcar_pcie *pcie)
 		 (macsr & LINK_SPEED) == LINK_SPEED_5_0GTS ? "5" : "2.5");
 }
 
+static void rcar_pcie_hw_enable(struct rcar_pcie *pci)
+{
+	struct resource_entry *win;
+	LIST_HEAD(res);
+	int i = 0;
+
+	/* Try setting 5 GT/s link speed */
+	rcar_pcie_force_speedup(pci);
+
+	/* Setup PCI resources */
+	resource_list_for_each_entry(win, &pci->resources) {
+		struct resource *res = win->res;
+
+		if (!res->flags)
+			continue;
+
+		switch (resource_type(res)) {
+		case IORESOURCE_IO:
+		case IORESOURCE_MEM:
+			rcar_pcie_setup_window(i, pci, win);
+			i++;
+			break;
+		}
+	}
+}
+
 static int rcar_pcie_enable(struct rcar_pcie *pcie)
 {
 	struct device *dev = pcie->dev;
@@ -891,11 +917,25 @@ static void rcar_pcie_unmap_msi(struct rcar_pcie *pcie)
 	irq_domain_remove(msi->domain);
 }
 
+static void rcar_pcie_hw_enable_msi(struct rcar_pcie *pcie)
+{
+	struct rcar_msi *msi = &pcie->msi;
+	unsigned long base;
+
+	/* setup MSI data target */
+	base = virt_to_phys((void *)msi->pages);
+
+	rcar_pci_write_reg(pcie, lower_32_bits(base) | MSIFE, PCIEMSIALR);
+	rcar_pci_write_reg(pcie, upper_32_bits(base), PCIEMSIAUR);
+
+	/* enable all MSI interrupts */
+	rcar_pci_write_reg(pcie, 0xffffffff, PCIEMSIIER);
+}
+
 static int rcar_pcie_enable_msi(struct rcar_pcie *pcie)
 {
 	struct device *dev = pcie->dev;
 	struct rcar_msi *msi = &pcie->msi;
-	phys_addr_t base;
 	int err, i;
 
 	mutex_init(&msi->lock);
@@ -934,17 +974,7 @@ static int rcar_pcie_enable_msi(struct rcar_pcie *pcie)
 
 	/* setup MSI data target */
 	msi->pages = __get_free_pages(GFP_KERNEL, 0);
-	if (!msi->pages) {
-		err = -ENOMEM;
-		goto err;
-	}
-	base = virt_to_phys((void *)msi->pages);
-
-	rcar_pci_write_reg(pcie, lower_32_bits(base) | MSIFE, PCIEMSIALR);
-	rcar_pci_write_reg(pcie, upper_32_bits(base), PCIEMSIAUR);
-
-	/* enable all MSI interrupts */
-	rcar_pci_write_reg(pcie, 0xffffffff, PCIEMSIIER);
+	rcar_pcie_hw_enable_msi(pcie);
 
 	return 0;
 
@@ -1219,6 +1249,37 @@ static int rcar_pcie_probe(struct platform_device *pdev)
 	return err;
 }
 
+static int rcar_pcie_resume(struct device *dev)
+{
+	struct rcar_pcie *pcie = dev_get_drvdata(dev);
+	int (*hw_init_fn)(struct rcar_pcie *);
+	unsigned int data;
+	int err;
+
+	err = rcar_pcie_parse_map_dma_ranges(pcie);
+	if (err)
+		return 0;
+
+	/* Failure to get a link might just be that no cards are inserted */
+	hw_init_fn = of_device_get_match_data(dev);
+	err = hw_init_fn(pcie);
+	if (err) {
+		dev_info(dev, "PCIe link down\n");
+		return 0;
+	}
+
+	data = rcar_pci_read_reg(pcie, MACSR);
+	dev_info(dev, "PCIe x%d: link up\n", (data >> 20) & 0x3f);
+
+	/* Enable MSI */
+	if (IS_ENABLED(CONFIG_PCI_MSI))
+		rcar_pcie_hw_enable_msi(pcie);
+
+	rcar_pcie_hw_enable(pcie);
+
+	return 0;
+}
+
 static int rcar_pcie_resume_noirq(struct device *dev)
 {
 	struct rcar_pcie *pcie = dev_get_drvdata(dev);
@@ -1234,6 +1295,7 @@ static int rcar_pcie_resume_noirq(struct device *dev)
 }
 
 static const struct dev_pm_ops rcar_pcie_pm_ops = {
+	SET_SYSTEM_SLEEP_PM_OPS(NULL, rcar_pcie_resume)
 	.resume_noirq = rcar_pcie_resume_noirq,
 };
 
Signature · 1 line
2.25.0
#5 lore raw
In reply to [email protected]

[+cc Vaibhav]

Alternate less redundant subject:

PCI: rcar: Add suspend/resume support

Quoted 5 lines · On Sat, Mar 14, 2020 at 08:12:32PM +0100, [email protected] wrote:

From: Kazufumi Ikeda <[email protected]>

This adds the suspend/resume supports for pcie-rcar. The resume handler reprograms the hardware based on the software state kept in specific device structures. Also it doesn't need to save any registers.

s/This adds the/Add/
s/supports/support/

Quoted 139 lines

---
V2: - Change return type of rcar_pcie_hw_enable() to void
- Drop default: case in switch statement in rcar_pcie_hw_enable()
- Sort variables in rcar_pcie_resume()
V3: - Update on top of next-20200313

---
 drivers/pci/controller/pcie-rcar.c | 86 +++++++++++++++++++++++++-----
 1 file changed, 74 insertions(+), 12 deletions(-)
drivers/pci/controller/pcie-rcar.c +73 -12
index 759c6542c5c8..f86513638b8a 100644
@@ -452,6 +452,32 @@ static void rcar_pcie_force_speedup(struct rcar_pcie *pcie)
 		 (macsr & LINK_SPEED) == LINK_SPEED_5_0GTS ? "5" : "2.5");
 }
 
+static void rcar_pcie_hw_enable(struct rcar_pcie *pci)
+{
+	struct resource_entry *win;
+	LIST_HEAD(res);
+	int i = 0;
+
+	/* Try setting 5 GT/s link speed */
+	rcar_pcie_force_speedup(pci);
+
+	/* Setup PCI resources */
+	resource_list_for_each_entry(win, &pci->resources) {
+		struct resource *res = win->res;
+
+		if (!res->flags)
+			continue;
+
+		switch (resource_type(res)) {
+		case IORESOURCE_IO:
+		case IORESOURCE_MEM:
+			rcar_pcie_setup_window(i, pci, win);
+			i++;
+			break;
+		}
+	}
+}
+
 static int rcar_pcie_enable(struct rcar_pcie *pcie)
 {
 	struct device *dev = pcie->dev;
@@ -891,11 +917,25 @@ static void rcar_pcie_unmap_msi(struct rcar_pcie *pcie)
 	irq_domain_remove(msi->domain);
 }
 
+static void rcar_pcie_hw_enable_msi(struct rcar_pcie *pcie)
+{
+	struct rcar_msi *msi = &pcie->msi;
+	unsigned long base;
+
+	/* setup MSI data target */
+	base = virt_to_phys((void *)msi->pages);
+
+	rcar_pci_write_reg(pcie, lower_32_bits(base) | MSIFE, PCIEMSIALR);
+	rcar_pci_write_reg(pcie, upper_32_bits(base), PCIEMSIAUR);
+
+	/* enable all MSI interrupts */
+	rcar_pci_write_reg(pcie, 0xffffffff, PCIEMSIIER);
+}
+
 static int rcar_pcie_enable_msi(struct rcar_pcie *pcie)
 {
 	struct device *dev = pcie->dev;
 	struct rcar_msi *msi = &pcie->msi;
-	phys_addr_t base;
 	int err, i;
 
 	mutex_init(&msi->lock);
@@ -934,17 +974,7 @@ static int rcar_pcie_enable_msi(struct rcar_pcie *pcie)
 
 	/* setup MSI data target */
 	msi->pages = __get_free_pages(GFP_KERNEL, 0);
-	if (!msi->pages) {
-		err = -ENOMEM;
-		goto err;
-	}
-	base = virt_to_phys((void *)msi->pages);
-
-	rcar_pci_write_reg(pcie, lower_32_bits(base) | MSIFE, PCIEMSIALR);
-	rcar_pci_write_reg(pcie, upper_32_bits(base), PCIEMSIAUR);
-
-	/* enable all MSI interrupts */
-	rcar_pci_write_reg(pcie, 0xffffffff, PCIEMSIIER);
+	rcar_pcie_hw_enable_msi(pcie);
 
 	return 0;
 
@@ -1219,6 +1249,37 @@ static int rcar_pcie_probe(struct platform_device *pdev)
 	return err;
 }
 
+static int rcar_pcie_resume(struct device *dev)
+{
+	struct rcar_pcie *pcie = dev_get_drvdata(dev);
+	int (*hw_init_fn)(struct rcar_pcie *);
+	unsigned int data;
+	int err;
+
+	err = rcar_pcie_parse_map_dma_ranges(pcie);
+	if (err)
+		return 0;
+
+	/* Failure to get a link might just be that no cards are inserted */
+	hw_init_fn = of_device_get_match_data(dev);
+	err = hw_init_fn(pcie);
+	if (err) {
+		dev_info(dev, "PCIe link down\n");
+		return 0;
+	}
+
+	data = rcar_pci_read_reg(pcie, MACSR);
+	dev_info(dev, "PCIe x%d: link up\n", (data >> 20) & 0x3f);
+
+	/* Enable MSI */
+	if (IS_ENABLED(CONFIG_PCI_MSI))
+		rcar_pcie_hw_enable_msi(pcie);
+
+	rcar_pcie_hw_enable(pcie);
+
+	return 0;
+}
+
 static int rcar_pcie_resume_noirq(struct device *dev)
 {
 	struct rcar_pcie *pcie = dev_get_drvdata(dev);

drivers/pci/controller/pcie-rcar.c static int rcar_pcie_resume_noirq(struct device *dev) · line 1298

Quoted 4 lines
drivers/pci/controller/pcie-rcar.c +1 -0
@@ -1234,6 +1295,7 @@ static int rcar_pcie_resume_noirq(struct device *dev)
 }
 
 static const struct dev_pm_ops rcar_pcie_pm_ops = {
+	SET_SYSTEM_SLEEP_PM_OPS(NULL, rcar_pcie_resume)

This causes the following warning when CONFIG_PM_SLEEP is not set:

drivers/pci/controller/pcie-rcar.c:1253:12: warning: ‘rcar_pcie_resume’ defined but not used [-Wunused-function]
1253 | static int rcar_pcie_resume(struct device *dev)

	|            ^~~~~~~~~~~~~~~~

Most people seem to be using __maybe_unused on the suspend/resume functions to avoid this, e.g., 226e6b866d74 ("gpio: pch: Convert to dev_pm_ops").

Quoted 6 lines
drivers/pci/controller/pcie-rcar.c +0 -0
 	.resume_noirq = rcar_pcie_resume_noirq,
 };
 
Signature · 1 line
2.25.0
#6 lore raw

Hi Bjorn,

Quoted 5 lines · Bjorn Helgaas

[+cc Vaibhav]

Alternate less redundant subject:

PCI: rcar: Add suspend/resume support

Note that there's both pcie-rcar.c (this driver, for R-Car Gen2 and Gen3 PCIe) and pci-rcar-gen2.c (for R-Car Gen2 PCI).
People tend to use the prefix "PCI: rcar: " for both :-(

Gr{oetje,eeting}s,

Geert

Signature · 5 lines
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds
#7 lore raw

[+cc Lorenzo]

Quoted 10 lines · Geert Uytterhoeven
Quoted 5 lines · Bjorn Helgaas

[+cc Vaibhav]

Alternate less redundant subject:

PCI: rcar: Add suspend/resume support

Note that there's both pcie-rcar.c (this driver, for R-Car Gen2 and Gen3 PCIe) and pci-rcar-gen2.c (for R-Car Gen2 PCI).
People tend to use the prefix "PCI: rcar: " for both :-(

Yeah, that's pretty broken, thanks for pointing this out!

For most drivers we use a chipset name ("keystone", "imx6", "tegra", etc) as the changlog tag. That's nice because it gives space for multiple drivers from the same vendor, but I don't know anything similarly specific for the R-Car drivers.

pci-rcar-gen2.c seems to be for some sort of internal Conventional PCI bus? The "gen2" is confusing because "Gen 2" is more commonly used for PCIe than for Conventional PCI.

I would propose keeping "rcar" for the PCIe driver and using "rcar-pci" for the Conventional PCI one, but the Conventional PCI one (pci-rcar-gen2.c) seems pretty inactive. The most recent commits are from 2018, and they're trivial cleanups. So I'm doubtful that anybody will remember when the next change comes in.

Bjorn

#8 lore raw

Hi Bjorn,

Quoted 12 lines · Bjorn Helgaas
Quoted 10 lines · Geert Uytterhoeven
Quoted 5 lines · Bjorn Helgaas

[+cc Vaibhav]

Alternate less redundant subject:

PCI: rcar: Add suspend/resume support

Note that there's both pcie-rcar.c (this driver, for R-Car Gen2 and Gen3 PCIe) and pci-rcar-gen2.c (for R-Car Gen2 PCI).
People tend to use the prefix "PCI: rcar: " for both :-(

Quoted 8 lines

Yeah, that's pretty broken, thanks for pointing this out!

For most drivers we use a chipset name ("keystone", "imx6", "tegra", etc) as the changlog tag. That's nice because it gives space for multiple drivers from the same vendor, but I don't know anything similarly specific for the R-Car drivers.

pci-rcar-gen2.c seems to be for some sort of internal Conventional PCI

AFAIUI it's some internal PCI glue to the *HCI USB controller.

Quoted 2 lines

bus? The "gen2" is confusing because "Gen 2" is more commonly used for PCIe than for Conventional PCI.

The "Gen2" applies to "R-Car", not to "PCI".

Quoted 2 lines

I would propose keeping "rcar" for the PCIe driver and using "rcar-pci" for the Conventional PCI one, but the Conventional PCI one

(/me resists against bike-shedding)

Quoted 3 lines

(pci-rcar-gen2.c) seems pretty inactive. The most recent commits are from 2018, and they're trivial cleanups. So I'm doubtful that anybody will remember when the next change comes in.

I guess pci-rcar-gen2.c is simpler and more mature ;-) R-Car Gen2 SoCs have both (internal) PCI and PCIe, so the two drivers can be used together on the same hardware.

Gr{oetje,eeting}s,

Geert

Signature · 5 lines
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds
#9 lore raw
Quoted 22 lines · Geert Uytterhoeven

Hi Bjorn,

Quoted 19 lines · Bjorn Helgaas
Quoted 10 lines · Geert Uytterhoeven
Quoted 5 lines · Bjorn Helgaas

[+cc Vaibhav]

Alternate less redundant subject:

PCI: rcar: Add suspend/resume support

Note that there's both pcie-rcar.c (this driver, for R-Car Gen2 and Gen3 PCIe) and pci-rcar-gen2.c (for R-Car Gen2 PCI).
People tend to use the prefix "PCI: rcar: " for both :-(

Yeah, that's pretty broken, thanks for pointing this out!

For most drivers we use a chipset name ("keystone", "imx6", "tegra", etc) as the changlog tag. That's nice because it gives space for multiple drivers from the same vendor, but I don't know anything similarly specific for the R-Car drivers.

Quoted 8 lines
Quoted 1 line

pci-rcar-gen2.c seems to be for some sort of internal Conventional PCI

AFAIUI it's some internal PCI glue to the *HCI USB controller.

Quoted 2 lines

bus? The "gen2" is confusing because "Gen 2" is more commonly used for PCIe than for Conventional PCI.

The "Gen2" applies to "R-Car", not to "PCI".

Wicked :) !

Quoted 4 lines
Quoted 2 lines

I would propose keeping "rcar" for the PCIe driver and using "rcar-pci" for the Conventional PCI one, but the Conventional PCI one

(/me resists against bike-shedding)

I'd agree with Bjorn - I don't know, internal vs external seems artificial. Certainly gen2 is misleading, it does not take much to improve it.

Quoted 7 lines
Quoted 3 lines

(pci-rcar-gen2.c) seems pretty inactive. The most recent commits are from 2018, and they're trivial cleanups. So I'm doubtful that anybody will remember when the next change comes in.

I guess pci-rcar-gen2.c is simpler and more mature ;-) R-Car Gen2 SoCs have both (internal) PCI and PCIe, so the two drivers can be used together on the same hardware.

I'd remove gen2 to start with, you are better placed to know the internals to come up with something significant.

Lorenzo

#10 lore raw

Hi Lorenzo,

Quoted 23 lines · Lorenzo Pieralisi
Quoted 22 lines · Geert Uytterhoeven
Quoted 20 lines · Bjorn Helgaas
Quoted 10 lines · Geert Uytterhoeven
Quoted 5 lines · Bjorn Helgaas

[+cc Vaibhav]

Alternate less redundant subject:

PCI: rcar: Add suspend/resume support

Note that there's both pcie-rcar.c (this driver, for R-Car Gen2 and Gen3 PCIe) and pci-rcar-gen2.c (for R-Car Gen2 PCI).
People tend to use the prefix "PCI: rcar: " for both :-(

Yeah, that's pretty broken, thanks for pointing this out!

For most drivers we use a chipset name ("keystone", "imx6", "tegra", etc) as the changlog tag. That's nice because it gives space for multiple drivers from the same vendor, but I don't know anything similarly specific for the R-Car drivers.

pci-rcar-gen2.c seems to be for some sort of internal Conventional PCI

Quoted 8 lines
Quoted 6 lines

AFAIUI it's some internal PCI glue to the *HCI USB controller.

Quoted 2 lines

bus? The "gen2" is confusing because "Gen 2" is more commonly used for PCIe than for Conventional PCI.

The "Gen2" applies to "R-Car", not to "PCI".

Wicked :) !

pcie-rcar.c supports R-Car Gen1, Gen2, and Gen3.

Quoted 8 lines
Quoted 4 lines
Quoted 2 lines

I would propose keeping "rcar" for the PCIe driver and using "rcar-pci" for the Conventional PCI one, but the Conventional PCI one

(/me resists against bike-shedding)

I'd agree with Bjorn - I don't know, internal vs external seems artificial. Certainly gen2 is misleading, it does not take much to improve it.

We have lots of drivers in other subsystems with "rcar-gen2" or "rcar-gen3" as part of their names.

Quoted 10 lines
Quoted 7 lines
Quoted 3 lines

(pci-rcar-gen2.c) seems pretty inactive. The most recent commits are from 2018, and they're trivial cleanups. So I'm doubtful that anybody will remember when the next change comes in.

I guess pci-rcar-gen2.c is simpler and more mature ;-) R-Car Gen2 SoCs have both (internal) PCI and PCIe, so the two drivers can be used together on the same hardware.

I'd remove gen2 to start with, you are better placed to know the internals to come up with something significant.

So we're back at "PCI: rcar: ...", for both ;-)

I'd say the main difference between the two drivers is PCI vs. PCIe.

Gr{oetje,eeting}s,

Geert

Signature · 5 lines
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds
#11 lore raw
In reply to Bjorn Helgaas

On 4/24/20 9:57 PM, Bjorn Helgaas wrote:
Hi,

[...]

Quoted 8 lines

drivers/pci/controller/pcie-rcar.c static int rcar_pcie_resume_noirq(struct device *dev) · line 1298

Quoted 5 lines
drivers/pci/controller/pcie-rcar.c +1 -0
@@ -1234,6 +1295,7 @@ static int rcar_pcie_resume_noirq(struct device *dev)
 }
 
 static const struct dev_pm_ops rcar_pcie_pm_ops = {
+	SET_SYSTEM_SLEEP_PM_OPS(NULL, rcar_pcie_resume)

This causes the following warning when CONFIG_PM_SLEEP is not set:

Quoted 7 lines

drivers/pci/controller/pcie-rcar.c:1253:12: warning: ‘rcar_pcie_resume’ defined but not used [-Wunused-function]
1253 | static int rcar_pcie_resume(struct device *dev)

	|            ^~~~~~~~~~~~~~~~

Most people seem to be using __maybe_unused on the suspend/resume functions to avoid this, e.g., 226e6b866d74 ("gpio: pch: Convert to dev_pm_ops").

Should be fixed by:
PCI: pcie-rcar: Mark rcar_pcie_resume() with __maybe_unused