Skip to contentSkip to conversation

PCI: rcar: Add the initialization of PCIe link in resume_noirq()

PATCH Applied [email protected] 4 participants 8 messages Mar 25, 2019 – Mar 29, 2019

Subsystems (5)
lore
Share & tools
lore
#0 lore raw

PATCH v3 1/1 PCI: rcar: Add the initialization of PCIe link in resume_noirq()

From: Kazufumi Ikeda <[email protected]>

Reestablish the PCIe link very early in the resume process in case it went down to prevent PCI accesses from hanging the bus. Such accesses can happen early in the PCI resume process, as early as the SUSPEND_RESUME_NOIRQ step, thus the link must be reestablished in the driver resume_noirq() callback.

Fixes e015f88c368d ("PCI: rcar: Add support for R-Car H3 to pcie-rcar")
[[email protected]: reformatted commit log]
Cc Geert Uytterhoeven <[email protected]>
Cc Phil Edworthy <[email protected]>
Cc Simon Horman <[email protected]>
Cc Wolfram Sang <[email protected]>

---
V2: - Use BIT() macro for (1 << n)
- Since polling in rcar_pcie_wait_for_dl() uses udelay(), do not add extra changes to this function anymore
- Make resume_noirq return early and clean up parenthesis therein
V3: - Add missing PMSR register definition, dropped due to patch reshuffling

---
 drivers/pci/controller/pcie-rcar.c | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)
drivers/pci/controller/pcie-rcar.c +21 -0
index c8febb009454..6a4e435bd35f 100644
@@ -46,6 +46,7 @@
 
 /* Transfer control */
 #define PCIETCTLR		0x02000
+#define  DL_DOWN		BIT(3)
 #define  CFINIT			1
 #define PCIETSTR		0x02004
 #define  DATA_LINK_ACTIVE	1
@@ -94,6 +95,7 @@
 #define MACCTLR			0x011058
 #define  SPEED_CHANGE		BIT(24)
 #define  SCRAMBLE_DISABLE	BIT(27)
+#define PMSR			0x01105c
 #define MACS2R			0x011078
 #define MACCGSPSETR		0x011084
 #define  SPCNGRSN		BIT(31)
@@ -1130,6 +1132,7 @@ static int rcar_pcie_probe(struct platform_device *pdev)
 	pcie = pci_host_bridge_priv(bridge);
 
 	pcie->dev = dev;
+	platform_set_drvdata(pdev, pcie);
 
 	err = pci_parse_request_of_pci_ranges(dev, &pcie->resources, NULL);
 	if (err)
@@ -1221,10 +1224,28 @@ static int rcar_pcie_probe(struct platform_device *pdev)
 	return err;
 }
 
+static int rcar_pcie_resume_noirq(struct device *dev)
+{
+	struct rcar_pcie *pcie = dev_get_drvdata(dev);
+
+	if (rcar_pci_read_reg(pcie, PMSR) &&
+	    !(rcar_pci_read_reg(pcie, PCIETCTLR) & DL_DOWN))
+		return 0;
+
+	/* Re-establish the PCIe link */
+	rcar_pci_write_reg(pcie, CFINIT, PCIETCTLR);
+	return rcar_pcie_wait_for_dl(pcie);
+}
+
+static const struct dev_pm_ops rcar_pcie_pm_ops = {
+	.resume_noirq = rcar_pcie_resume_noirq,
+};
+
 static struct platform_driver rcar_pcie_driver = {
 	.driver = {
 		.name = "rcar-pcie",
 		.of_match_table = rcar_pcie_of_match,
+		.pm = &rcar_pcie_pm_ops,
 		.suppress_bind_attrs = true,
 	},
 	.probe = rcar_pcie_probe,
Signature · 1 line
2.20.1
Reviewed-by on v3 1/1
#1 lore raw
Quoted 8 lines · On Mon, Mar 25, 2019 at 8:43 PM <[email protected]> wrote:

From: Kazufumi Ikeda <[email protected]>

Reestablish the PCIe link very early in the resume process in case it went down to prevent PCI accesses from hanging the bus. Such accesses can happen early in the PCI resume process, as early as the SUSPEND_RESUME_NOIRQ step, thus the link must be reestablished in the driver resume_noirq() callback.

Quoted 8 lines
Fixes e015f88c368d ("PCI: rcar: Add support for R-Car H3 to pcie-rcar")
[[email protected]: reformatted commit log]

Gr{oetje,eeting}s,

Geert

Signature · 5 lines
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds
#2 lore raw
In reply to [email protected]
Quoted 28 lines · On Mon, Mar 25, 2019 at 08:43:19PM +0100, [email protected] wrote:

From: Kazufumi Ikeda <[email protected]>

Reestablish the PCIe link very early in the resume process in case it went down to prevent PCI accesses from hanging the bus. Such accesses can happen early in the PCI resume process, as early as the SUSPEND_RESUME_NOIRQ step, thus the link must be reestablished in the driver resume_noirq() callback.

Fixes e015f88c368d ("PCI: rcar: Add support for R-Car H3 to pcie-rcar")
[[email protected]: reformatted commit log]
Cc Geert Uytterhoeven <[email protected]>
Cc Phil Edworthy <[email protected]>
Cc Simon Horman <[email protected]>
Cc Wolfram Sang <[email protected]>

---
V2: - Use BIT() macro for (1 << n)
- Since polling in rcar_pcie_wait_for_dl() uses udelay(), do not add extra changes to this function anymore
- Make resume_noirq return early and clean up parenthesis therein
V3: - Add missing PMSR register definition, dropped due to patch reshuffling

Quoted 3 lines
---
 drivers/pci/controller/pcie-rcar.c | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

Applied to pci/controller-fixes for one of the upcoming -rc*.

Thanks,
Lorenzo

Quoted 60 lines
drivers/pci/controller/pcie-rcar.c +21 -0
index c8febb009454..6a4e435bd35f 100644
@@ -46,6 +46,7 @@
 
 /* Transfer control */
 #define PCIETCTLR		0x02000
+#define  DL_DOWN		BIT(3)
 #define  CFINIT			1
 #define PCIETSTR		0x02004
 #define  DATA_LINK_ACTIVE	1
@@ -94,6 +95,7 @@
 #define MACCTLR			0x011058
 #define  SPEED_CHANGE		BIT(24)
 #define  SCRAMBLE_DISABLE	BIT(27)
+#define PMSR			0x01105c
 #define MACS2R			0x011078
 #define MACCGSPSETR		0x011084
 #define  SPCNGRSN		BIT(31)
@@ -1130,6 +1132,7 @@ static int rcar_pcie_probe(struct platform_device *pdev)
 	pcie = pci_host_bridge_priv(bridge);
 
 	pcie->dev = dev;
+	platform_set_drvdata(pdev, pcie);
 
 	err = pci_parse_request_of_pci_ranges(dev, &pcie->resources, NULL);
 	if (err)
@@ -1221,10 +1224,28 @@ static int rcar_pcie_probe(struct platform_device *pdev)
 	return err;
 }
 
+static int rcar_pcie_resume_noirq(struct device *dev)
+{
+	struct rcar_pcie *pcie = dev_get_drvdata(dev);
+
+	if (rcar_pci_read_reg(pcie, PMSR) &&
+	    !(rcar_pci_read_reg(pcie, PCIETCTLR) & DL_DOWN))
+		return 0;
+
+	/* Re-establish the PCIe link */
+	rcar_pci_write_reg(pcie, CFINIT, PCIETCTLR);
+	return rcar_pcie_wait_for_dl(pcie);
+}
+
+static const struct dev_pm_ops rcar_pcie_pm_ops = {
+	.resume_noirq = rcar_pcie_resume_noirq,
+};
+
 static struct platform_driver rcar_pcie_driver = {
 	.driver = {
 		.name = "rcar-pcie",
 		.of_match_table = rcar_pcie_of_match,
+		.pm = &rcar_pcie_pm_ops,
 		.suppress_bind_attrs = true,
 	},
 	.probe = rcar_pcie_probe,
Signature · 1 line
2.20.1
#3 lore raw
In reply to [email protected]
Quoted 9 lines · On Mon, Mar 25, 2019 at 08:43:19PM +0100, [email protected] wrote:

From: Kazufumi Ikeda <[email protected]>

Reestablish the PCIe link very early in the resume process in case it went down to prevent PCI accesses from hanging the bus. Such accesses can happen early in the PCI resume process, as early as the SUSPEND_RESUME_NOIRQ step, thus the link must be reestablished in the driver resume_noirq() callback.

Fixes e015f88c368d ("PCI: rcar: Add support for R-Car H3 to pcie-rcar")

I'm fine with the fix itself, but since e015f88c368d appeared more than two years ago in v4.5, the justification for merging this after the merge window is a little weak.

Is there a more recent change that exposed this problem? The usual situation is that we merged something during the v5.1 merge window that caused a regression, and we're now fixing that before v5.1 final.

Bjorn

Quoted 83 lines

---
V2: - Use BIT() macro for (1 << n)
- Since polling in rcar_pcie_wait_for_dl() uses udelay(), do not add extra changes to this function anymore
- Make resume_noirq return early and clean up parenthesis therein
V3: - Add missing PMSR register definition, dropped due to patch reshuffling

---
 drivers/pci/controller/pcie-rcar.c | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)
drivers/pci/controller/pcie-rcar.c +21 -0
index c8febb009454..6a4e435bd35f 100644
@@ -46,6 +46,7 @@
 
 /* Transfer control */
 #define PCIETCTLR		0x02000
+#define  DL_DOWN		BIT(3)
 #define  CFINIT			1
 #define PCIETSTR		0x02004
 #define  DATA_LINK_ACTIVE	1
@@ -94,6 +95,7 @@
 #define MACCTLR			0x011058
 #define  SPEED_CHANGE		BIT(24)
 #define  SCRAMBLE_DISABLE	BIT(27)
+#define PMSR			0x01105c
 #define MACS2R			0x011078
 #define MACCGSPSETR		0x011084
 #define  SPCNGRSN		BIT(31)
@@ -1130,6 +1132,7 @@ static int rcar_pcie_probe(struct platform_device *pdev)
 	pcie = pci_host_bridge_priv(bridge);
 
 	pcie->dev = dev;
+	platform_set_drvdata(pdev, pcie);
 
 	err = pci_parse_request_of_pci_ranges(dev, &pcie->resources, NULL);
 	if (err)
@@ -1221,10 +1224,28 @@ static int rcar_pcie_probe(struct platform_device *pdev)
 	return err;
 }
 
+static int rcar_pcie_resume_noirq(struct device *dev)
+{
+	struct rcar_pcie *pcie = dev_get_drvdata(dev);
+
+	if (rcar_pci_read_reg(pcie, PMSR) &&
+	    !(rcar_pci_read_reg(pcie, PCIETCTLR) & DL_DOWN))
+		return 0;
+
+	/* Re-establish the PCIe link */
+	rcar_pci_write_reg(pcie, CFINIT, PCIETCTLR);
+	return rcar_pcie_wait_for_dl(pcie);
+}
+
+static const struct dev_pm_ops rcar_pcie_pm_ops = {
+	.resume_noirq = rcar_pcie_resume_noirq,
+};
+
 static struct platform_driver rcar_pcie_driver = {
 	.driver = {
 		.name = "rcar-pcie",
 		.of_match_table = rcar_pcie_of_match,
+		.pm = &rcar_pcie_pm_ops,
 		.suppress_bind_attrs = true,
 	},
 	.probe = rcar_pcie_probe,
Signature · 1 line
2.20.1
#4 lore raw
Quoted 11 lines · Bjorn Helgaas
Quoted 9 lines · On Mon, Mar 25, 2019 at 08:43:19PM +0100, [email protected] wrote:

From: Kazufumi Ikeda <[email protected]>

Reestablish the PCIe link very early in the resume process in case it went down to prevent PCI accesses from hanging the bus. Such accesses can happen early in the PCI resume process, as early as the SUSPEND_RESUME_NOIRQ step, thus the link must be reestablished in the driver resume_noirq() callback.

Fixes e015f88c368d ("PCI: rcar: Add support for R-Car H3 to pcie-rcar")
Quoted 7 lines

I'm fine with the fix itself, but since e015f88c368d appeared more than two years ago in v4.5, the justification for merging this after the merge window is a little weak.

Is there a more recent change that exposed this problem? The usual situation is that we merged something during the v5.1 merge window that caused a regression, and we're now fixing that before v5.1 final.

I do not think that's urgent enough to merge it in one of the -rc* but I won't speak for the authors - it will trickle back anyway through the stable mechanism, I just queued it as a fix since that's how it qualifies.

So postponing it to v5.2 is fine by me, just let me know how it is best to handle it.

Thanks,
Lorenzo

Quoted 85 lines

Bjorn

Quoted 83 lines

---
V2: - Use BIT() macro for (1 << n)
- Since polling in rcar_pcie_wait_for_dl() uses udelay(), do not add extra changes to this function anymore
- Make resume_noirq return early and clean up parenthesis therein
V3: - Add missing PMSR register definition, dropped due to patch reshuffling

---
 drivers/pci/controller/pcie-rcar.c | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)
drivers/pci/controller/pcie-rcar.c +21 -0
index c8febb009454..6a4e435bd35f 100644
@@ -46,6 +46,7 @@
 
 /* Transfer control */
 #define PCIETCTLR		0x02000
+#define  DL_DOWN		BIT(3)
 #define  CFINIT			1
 #define PCIETSTR		0x02004
 #define  DATA_LINK_ACTIVE	1
@@ -94,6 +95,7 @@
 #define MACCTLR			0x011058
 #define  SPEED_CHANGE		BIT(24)
 #define  SCRAMBLE_DISABLE	BIT(27)
+#define PMSR			0x01105c
 #define MACS2R			0x011078
 #define MACCGSPSETR		0x011084
 #define  SPCNGRSN		BIT(31)
@@ -1130,6 +1132,7 @@ static int rcar_pcie_probe(struct platform_device *pdev)
 	pcie = pci_host_bridge_priv(bridge);
 
 	pcie->dev = dev;
+	platform_set_drvdata(pdev, pcie);
 
 	err = pci_parse_request_of_pci_ranges(dev, &pcie->resources, NULL);
 	if (err)
@@ -1221,10 +1224,28 @@ static int rcar_pcie_probe(struct platform_device *pdev)
 	return err;
 }
 
+static int rcar_pcie_resume_noirq(struct device *dev)
+{
+	struct rcar_pcie *pcie = dev_get_drvdata(dev);
+
+	if (rcar_pci_read_reg(pcie, PMSR) &&
+	    !(rcar_pci_read_reg(pcie, PCIETCTLR) & DL_DOWN))
+		return 0;
+
+	/* Re-establish the PCIe link */
+	rcar_pci_write_reg(pcie, CFINIT, PCIETCTLR);
+	return rcar_pcie_wait_for_dl(pcie);
+}
+
+static const struct dev_pm_ops rcar_pcie_pm_ops = {
+	.resume_noirq = rcar_pcie_resume_noirq,
+};
+
 static struct platform_driver rcar_pcie_driver = {
 	.driver = {
 		.name = "rcar-pcie",
 		.of_match_table = rcar_pcie_of_match,
+		.pm = &rcar_pcie_pm_ops,
 		.suppress_bind_attrs = true,
 	},
 	.probe = rcar_pcie_probe,
Signature · 1 line
2.20.1
#5 lore raw
In reply to Bjorn Helgaas

Hi Bjorn,

Quoted 9 lines · Bjorn Helgaas
Quoted 8 lines · On Mon, Mar 25, 2019 at 08:43:19PM +0100, [email protected] wrote:

From: Kazufumi Ikeda <[email protected]>

Reestablish the PCIe link very early in the resume process in case it went down to prevent PCI accesses from hanging the bus. Such accesses can happen early in the PCI resume process, as early as the SUSPEND_RESUME_NOIRQ step, thus the link must be reestablished in the driver resume_noirq() callback.

Quoted 5 lines
Quoted 1 line
Fixes e015f88c368d ("PCI: rcar: Add support for R-Car H3 to pcie-rcar")

I'm fine with the fix itself, but since e015f88c368d appeared more than two years ago in v4.5, the justification for merging this after the merge window is a little weak.

V1 of this fix was posted in November 2017, but IIRC, the series became the target of some bike-shedding...

Quoted 3 lines

Is there a more recent change that exposed this problem? The usual situation is that we merged something during the v5.1 merge window that caused a regression, and we're now fixing that before v5.1 final.

There are several reasons most people couldn't even run suspend/resume cycles on their systems:
1. Early releases of the affected boards came with firmware revisions with non-functional PSCI system suspend,
2. Preparing the PMIC for suspend required ugly assistance from i2cset in userspace, until the Linux driver learned to take care of that itself in v4.18/v4.19.

I guess the fix can survive postponing to v5.2, though...

Thanks!

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

#6 lore raw
Quoted 31 lines · Geert Uytterhoeven
Quoted 9 lines · Bjorn Helgaas
Quoted 8 lines · On Mon, Mar 25, 2019 at 08:43:19PM +0100, [email protected] wrote:

From: Kazufumi Ikeda <[email protected]>

Reestablish the PCIe link very early in the resume process in case it went down to prevent PCI accesses from hanging the bus. Such accesses can happen early in the PCI resume process, as early as the SUSPEND_RESUME_NOIRQ step, thus the link must be reestablished in the driver resume_noirq() callback.

Quoted 5 lines
Quoted 1 line
Fixes e015f88c368d ("PCI: rcar: Add support for R-Car H3 to pcie-rcar")

I'm fine with the fix itself, but since e015f88c368d appeared more than two years ago in v4.5, the justification for merging this after the merge window is a little weak.

V1 of this fix was posted in November 2017, but IIRC, the series became the target of some bike-shedding...

Quoted 3 lines

Is there a more recent change that exposed this problem? The usual situation is that we merged something during the v5.1 merge window that caused a regression, and we're now fixing that before v5.1 final.

There are several reasons most people couldn't even run suspend/resume cycles on their systems:
1. Early releases of the affected boards came with firmware revisions with non-functional PSCI system suspend,
2. Preparing the PMIC for suspend required ugly assistance from i2cset in userspace, until the Linux driver learned to take care of that itself in v4.18/v4.19.

Quoted 1 line

I guess the fix can survive postponing to v5.2, though...

Ok, I'll merge it to -next for v5.2, thanks.

Bjorn

#7 lore raw
Quoted 32 lines · Bjorn Helgaas
Quoted 31 lines · Geert Uytterhoeven
Quoted 9 lines · Bjorn Helgaas
Quoted 8 lines · On Mon, Mar 25, 2019 at 08:43:19PM +0100, [email protected] wrote:

From: Kazufumi Ikeda <[email protected]>

Reestablish the PCIe link very early in the resume process in case it went down to prevent PCI accesses from hanging the bus. Such accesses can happen early in the PCI resume process, as early as the SUSPEND_RESUME_NOIRQ step, thus the link must be reestablished in the driver resume_noirq() callback.

Quoted 5 lines
Quoted 1 line
Fixes e015f88c368d ("PCI: rcar: Add support for R-Car H3 to pcie-rcar")

I'm fine with the fix itself, but since e015f88c368d appeared more than two years ago in v4.5, the justification for merging this after the merge window is a little weak.

V1 of this fix was posted in November 2017, but IIRC, the series became the target of some bike-shedding...

Quoted 3 lines

Is there a more recent change that exposed this problem? The usual situation is that we merged something during the v5.1 merge window that caused a regression, and we're now fixing that before v5.1 final.

There are several reasons most people couldn't even run suspend/resume cycles on their systems:
1. Early releases of the affected boards came with firmware revisions with non-functional PSCI system suspend,
2. Preparing the PMIC for suspend required ugly assistance from i2cset in userspace, until the Linux driver learned to take care of that itself in v4.18/v4.19.

Quoted 3 lines
Quoted 1 line

I guess the fix can survive postponing to v5.2, though...

Ok, I'll merge it to -next for v5.2, thanks.

ACK.

Signature · 2 lines
Best regards,
Marek Vasut