PCI: apple: Follow the PCIe specifications when resetting the port

PATCH In review Marc Zyngier 4 participants 11 messages Nov 22, 2021 – Nov 23, 2021

Subsystems (6)

Device tree bindings Arm (32-bit) PCI PCI native host bridge and endpoint drivers +2 more subsystems

lore

Share & tools

lore

Marc Zyngier

author maintainer

2021-11-22 10:42 UTC #0 lore raw

PATCH v2 1/1 PCI: apple: Follow the PCIe specifications when resetting the port

While the Apple PCIe driver works correctly when directly booted from the firmware, it fails to initialise when the kernel is booted from a bootloader using PCIe such as u-boot.

That's beacuse we're missing a proper reset of the port (we only clear the reset, but never assert it).

The PCIe spec requirements are two-fold:

- #PERST must be asserted before setting up the clocks, and stay asserted for at least 100us (Tperst-clk).

- Once #PERST is deasserted, the OS must wait for at least 100ms "from the end of a Conventional Reset" before we can start talking to the devices

Implementing this results in a booting system.

Fixes 1e33888fbe44 ("PCI: apple: Add initial hardware bring-up")

Signed-off-by Marc Zyngier <[email protected]>

Cc Alyssa Rosenzweig <[email protected]>

Cc Lorenzo Pieralisi <[email protected]>

Cc Bjorn Helgaas <[email protected]>

Cc Pali Rohár <[email protected]>

---
 drivers/pci/controller/pcie-apple.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

drivers/pci/controller/pcie-apple.c +10 -0

index 1bf4d75b61be..957960a733c4 100644
@@ -539,13 +539,23 @@ static int apple_pcie_setup_port(struct apple_pcie *pcie,
 
 	rmw_set(PORT_APPCLK_EN, port->base + PORT_APPCLK);
 
+	/* Engage #PERST before setting up the clock */
+	gpiod_set_value(reset, 0);
+
 	ret = apple_pcie_setup_refclk(pcie, port);
 	if (ret < 0)
 		return ret;
 
+	/* The minimal Tperst-clk value is 100us (PCIe CMS r2.0, 2.6.2) */
+	usleep_range(100, 200);
+
+	/* Deassert #PERST */
 	rmw_set(PORT_PERST_OFF, port->base + PORT_PERST);
 	gpiod_set_value(reset, 1);
 
+	/* Wait for 100ms after #PERST deassertion (PCIe r2.0, 6.6.1) */
+	msleep(100);
+
 	ret = readl_relaxed_poll_timeout(port->base + PORT_STATUS, stat,
 					 stat & PORT_STATUS_READY, 100, 250000);
 	if (ret < 0) {

Signature · 1 line

2.30.2

Pali Rohár

Acked-by on v2 1/1

2021-11-22 12:03 UTC #1 lore raw

Quoted 16 lines · On Monday 22 November 2021 10:41:56 Marc Zyngier wrote:

While the Apple PCIe driver works correctly when directly booted from the firmware, it fails to initialise when the kernel is booted from a bootloader using PCIe such as u-boot.
That's beacuse we're missing a proper reset of the port (we only clear the reset, but never assert it).
The PCIe spec requirements are two-fold:
- #PERST must be asserted before setting up the clocks, and stay asserted for at least 100us (Tperst-clk).
- Once #PERST is deasserted, the OS must wait for at least 100ms "from the end of a Conventional Reset" before we can start talking to the devices

Quoted 8 lines

Implementing this results in a booting system.
Fixes 1e33888fbe44 ("PCI: apple: Add initial hardware bring-up")
Signed-off-by Marc Zyngier <[email protected]>
Cc Alyssa Rosenzweig <[email protected]>
Cc Lorenzo Pieralisi <[email protected]>
Cc Bjorn Helgaas <[email protected]>
Cc Pali Rohár <[email protected]>

Looks good, but see comment below.

Acked-by Pali Rohár <[email protected]>

Quoted 15 lines

---
 drivers/pci/controller/pcie-apple.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

drivers/pci/controller/pcie-apple.c +3 -0

index 1bf4d75b61be..957960a733c4 100644
@@ -539,13 +539,23 @@ static int apple_pcie_setup_port(struct apple_pcie *pcie,
 
 	rmw_set(PORT_APPCLK_EN, port->base + PORT_APPCLK);
 
+	/* Engage #PERST before setting up the clock */
+	gpiod_set_value(reset, 0);
+

drivers/pci/controller/pcie-apple.c static int apple_pcie_setup_port(struct apple_pcie *pcie, · line 554

Quoted 10 lines

drivers/pci/controller/pcie-apple.c +4 -0

 	ret = apple_pcie_setup_refclk(pcie, port);
 	if (ret < 0)
 		return ret;
 
+	/* The minimal Tperst-clk value is 100us (PCIe CMS r2.0, 2.6.2) */
+	usleep_range(100, 200);
+
+	/* Deassert #PERST */
 	rmw_set(PORT_PERST_OFF, port->base + PORT_PERST);
 	gpiod_set_value(reset, 1);

+ Luca

Just one comment. PERST# (PCIe Reset) is active-low signal. De-asserting means to really set value to 1.

But there was a discussion that de-asserting should be done by call:
gpiod_set_value(reset, 0);

https://lore.kernel.org/linux-pci/[email protected]/

Could we make this new pcie-apple.c driver to use gpiod_set_value(reset, 0) for de-asserting, like in other drivers?

Quoted 10 lines

drivers/pci/controller/pcie-apple.c +3 -0

+	/* Wait for 100ms after #PERST deassertion (PCIe r2.0, 6.6.1) */
+	msleep(100);
+
 	ret = readl_relaxed_poll_timeout(port->base + PORT_STATUS, stat,
 					 stat & PORT_STATUS_READY, 100, 250000);
 	if (ret < 0) {

Signature · 1 line

2.30.2

Marc Zyngier

author maintainer

2021-11-22 14:43 UTC #2 lore raw

Quoted 27 lines · Pali Rohár

Quoted 16 lines · On Monday 22 November 2021 10:41:56 Marc Zyngier wrote:
While the Apple PCIe driver works correctly when directly booted from the firmware, it fails to initialise when the kernel is booted from a bootloader using PCIe such as u-boot.
That's beacuse we're missing a proper reset of the port (we only clear the reset, but never assert it).
The PCIe spec requirements are two-fold:
- #PERST must be asserted before setting up the clocks, and stay asserted for at least 100us (Tperst-clk).
- Once #PERST is deasserted, the OS must wait for at least 100ms "from the end of a Conventional Reset" before we can start talking to the devices
Quoted 8 lines
Implementing this results in a booting system.
Fixes 1e33888fbe44 ("PCI: apple: Add initial hardware bring-up")
Signed-off-by Marc Zyngier <[email protected]>
Cc Alyssa Rosenzweig <[email protected]>
Cc Lorenzo Pieralisi <[email protected]>
Cc Bjorn Helgaas <[email protected]>
Cc Pali Rohár <[email protected]>

Quoted 3 lines

Looks good, but see comment below.
Acked-by Pali Rohár <[email protected]>

Thanks for that.

Quoted 32 lines

Quoted 16 lines
---
 drivers/pci/controller/pcie-apple.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
drivers/pci/controller/pcie-apple.c +3 -0
index 1bf4d75b61be..957960a733c4 100644
@@ -539,13 +539,23 @@ static int apple_pcie_setup_port(struct apple_pcie *pcie,
 
 	rmw_set(PORT_APPCLK_EN, port->base + PORT_APPCLK);
 
+	/* Engage #PERST before setting up the clock */
+	gpiod_set_value(reset, 0);
+
drivers/pci/controller/pcie-apple.c static int apple_pcie_setup_port(struct apple_pcie *pcie, · line 554
Quoted 10 lines
drivers/pci/controller/pcie-apple.c +4 -0
 	ret = apple_pcie_setup_refclk(pcie, port);
 	if (ret < 0)
 		return ret;
 
+	/* The minimal Tperst-clk value is 100us (PCIe CMS r2.0, 2.6.2) */
+	usleep_range(100, 200);
+
+	/* Deassert #PERST */
 	rmw_set(PORT_PERST_OFF, port->base + PORT_PERST);
 	gpiod_set_value(reset, 1);
+ Luca
Just one comment. PERST# (PCIe Reset) is active-low signal. De-asserting means to really set value to 1.

Quoted 7 lines

But there was a discussion that de-asserting should be done by call:
gpiod_set_value(reset, 0);
https://lore.kernel.org/linux-pci/[email protected]/
Could we make this new pcie-apple.c driver to use gpiod_set_value(reset, 0) for de-asserting, like in other drivers?

I guess it depends whether you care about the assertion or the signal itself. I think we may have a bug in the way the GPIOs are handled at the moment, as it makes no difference whether I register the GPIO are active high or active low...

I guess that will be yet another thing to debug, but in the meantime we have a reliable reset.

Thanks,

M.

Signature · 1 line

Without deviation from the norm, progress is not possible.

Luca Ceresoli

2021-11-22 21:32 UTC #3 lore raw

Hi Marc,

Quoted 29 lines · Marc Zyngier

Quoted 27 lines · Pali Rohár
Quoted 24 lines · On Monday 22 November 2021 10:41:56 Marc Zyngier wrote:
While the Apple PCIe driver works correctly when directly booted from the firmware, it fails to initialise when the kernel is booted from a bootloader using PCIe such as u-boot.
That's beacuse we're missing a proper reset of the port (we only clear the reset, but never assert it).
The PCIe spec requirements are two-fold:
- #PERST must be asserted before setting up the clocks, and stay asserted for at least 100us (Tperst-clk).
- Once #PERST is deasserted, the OS must wait for at least 100ms "from the end of a Conventional Reset" before we can start talking to the devices
Implementing this results in a booting system.
Fixes 1e33888fbe44 ("PCI: apple: Add initial hardware bring-up")
Signed-off-by Marc Zyngier <[email protected]>
Cc Alyssa Rosenzweig <[email protected]>
Cc Lorenzo Pieralisi <[email protected]>
Cc Bjorn Helgaas <[email protected]>
Cc Pali Rohár <[email protected]>

Quoted 45 lines

Quoted 3 lines
Looks good, but see comment below.
Acked-by Pali Rohár <[email protected]>
Thanks for that.
Quoted 39 lines
Quoted 16 lines
---
 drivers/pci/controller/pcie-apple.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
drivers/pci/controller/pcie-apple.c +3 -0
index 1bf4d75b61be..957960a733c4 100644
@@ -539,13 +539,23 @@ static int apple_pcie_setup_port(struct apple_pcie *pcie,
 
 	rmw_set(PORT_APPCLK_EN, port->base + PORT_APPCLK);
 
+	/* Engage #PERST before setting up the clock */
+	gpiod_set_value(reset, 0);
+
drivers/pci/controller/pcie-apple.c static int apple_pcie_setup_port(struct apple_pcie *pcie, · line 554
Quoted 10 lines
drivers/pci/controller/pcie-apple.c +4 -0
 	ret = apple_pcie_setup_refclk(pcie, port);
 	if (ret < 0)
 		return ret;
 
+	/* The minimal Tperst-clk value is 100us (PCIe CMS r2.0, 2.6.2) */
+	usleep_range(100, 200);
+
+	/* Deassert #PERST */
 	rmw_set(PORT_PERST_OFF, port->base + PORT_PERST);
 	gpiod_set_value(reset, 1);
+ Luca
Just one comment. PERST# (PCIe Reset) is active-low signal. De-asserting means to really set value to 1.
But there was a discussion that de-asserting should be done by call:
gpiod_set_value(reset, 0);
https://lore.kernel.org/linux-pci/[email protected]/
Could we make this new pcie-apple.c driver to use gpiod_set_value(reset, 0) for de-asserting, like in other drivers?

I agree it should be done right from the beginning since this is a new driver. Fixing it later is a painful process.

Quoted 7 lines

I guess it depends whether you care about the assertion or the signal itself. I think we may have a bug in the way the GPIOs are handled at the moment, as it makes no difference whether I register the GPIO are active high or active low...
I guess that will be yet another thing to debug, but in the meantime we have a reliable reset.

Strange, in my case the "active low" pin polarity is correctly picked up from device tree by the gpiolib code, thus using gpio_set_value(gpiod,
1) asserts the pin as it should, resulting in an electrically low pin.

Signature · 1 line

Luca

Marc Zyngier

author maintainer

2021-11-23 08:48 UTC #4 lore raw

Luca,

Quoted 7 lines · Luca Ceresoli

Quoted 7 lines
Quoted 7 lines
Just one comment. PERST# (PCIe Reset) is active-low signal. De-asserting means to really set value to 1.
But there was a discussion that de-asserting should be done by call:
gpiod_set_value(reset, 0);

Quoted 7 lines

Quoted 4 lines
Quoted 4 lines
https://lore.kernel.org/linux-pci/[email protected]/
Could we make this new pcie-apple.c driver to use gpiod_set_value(reset, 0) for de-asserting, like in other drivers?
I agree it should be done right from the beginning since this is a new driver. Fixing it later is a painful process.

No more painful than anything else. At this stage, using a positive or negative polarity is immaterial, as there is no core infrastructure making any use of this behaviour (every single driver must reinvent its own square wheel). If such an infrastructure existed, that'd indeed be a requirement. For now, this is merely a convention.

Quoted 11 lines

Quoted 7 lines
I guess it depends whether you care about the assertion or the signal itself. I think we may have a bug in the way the GPIOs are handled at the moment, as it makes no difference whether I register the GPIO are active high or active low...
I guess that will be yet another thing to debug, but in the meantime we have a reliable reset.
Strange, in my case the "active low" pin polarity is correctly picked up from device tree by the gpiolib code, thus using gpio_set_value(gpiod,
1) asserts the pin as it should, resulting in an electrically low pin.

As I said, this looks like a bug, probably in the M1 DT. I'll try to look into it when I get the time.

M.

Signature · 1 line

Without deviation from the norm, progress is not possible.

Mark Kettenis

2021-11-23 12:27 UTC #5 lore raw

Quoted 36 lines

Date: Tue, 23 Nov 2021 08:48:50 +0000
From: Marc Zyngier <[email protected]>
Luca,
Quoted 7 lines · Luca Ceresoli
Quoted 7 lines
Quoted 7 lines
Just one comment. PERST# (PCIe Reset) is active-low signal. De-asserting means to really set value to 1.
But there was a discussion that de-asserting should be done by call:
gpiod_set_value(reset, 0);
Quoted 7 lines
Quoted 4 lines
Quoted 4 lines
https://lore.kernel.org/linux-pci/[email protected]/
Could we make this new pcie-apple.c driver to use gpiod_set_value(reset, 0) for de-asserting, like in other drivers?
I agree it should be done right from the beginning since this is a new driver. Fixing it later is a painful process.
No more painful than anything else. At this stage, using a positive or negative polarity is immaterial, as there is no core infrastructure making any use of this behaviour (every single driver must reinvent its own square wheel). If such an infrastructure existed, that'd indeed be a requirement. For now, this is merely a convention.
Quoted 8 lines
Quoted 7 lines
I guess it depends whether you care about the assertion or the signal itself. I think we may have a bug in the way the GPIOs are handled at the moment, as it makes no difference whether I register the GPIO are active high or active low...
I guess that will be yet another thing to debug, but in the meantime we have a reliable reset.

Quoted 6 lines

Quoted 3 lines
Strange, in my case the "active low" pin polarity is correctly picked up from device tree by the gpiolib code, thus using gpio_set_value(gpiod,
1) asserts the pin as it should, resulting in an electrically low pin.
As I said, this looks like a bug, probably in the M1 DT. I'll try to look into it when I get the time.

So the diff below is what the changes look like for U-Boot. The U-Boot Apple PCIe driver has not been submitted upstream yet, so making this change is no problem.

arch/arm/dts/t8103-j274.dts +3 -3

index aef1ae29b6..3777337033 100644
@@ -65,7 +65,7 @@
 		device_type = "pci";
 		reg = <0x0 0x0 0x0 0x0 0x0>;
 		pwren-gpios = <&smc 13 0>;
-		reset-gpios = <&pinctrl_ap 152 0>;
+		reset-gpios = <&pinctrl_ap 152 GPIO_ACTIVE_LOW>;
 		max-link-speed = <2>;
 
 		#address-cells = <3>;

@@ -76,7 +76,7 @@
 	pci1: pci@1,0 {
 		device_type = "pci";
 		reg = <0x800 0x0 0x0 0x0 0x0>;
-		reset-gpios = <&pinctrl_ap 153 0>;
+		reset-gpios = <&pinctrl_ap 153 GPIO_ACTIVE_LOW>;
 		max-link-speed = <2>;
 
 		#address-cells = <3>;

@@ -87,7 +87,7 @@
 	pci2: pci@2,0 {
 		device_type = "pci";
 		reg = <0x1000 0x0 0x0 0x0 0x0>;
-		reset-gpios = <&pinctrl_ap 33 0>;
+		reset-gpios = <&pinctrl_ap 33 GPIO_ACTIVE_LOW>;
 		max-link-speed = <1>;
 
 		#address-cells = <3>;

drivers/pci/pcie_apple.c +2 -2

index bef6043adb..89eec70d81 100644
@@ -210,7 +210,7 @@ static int apple_pcie_setup_port(struct apple_pcie_priv *pcie, unsigned idx)
 		return 0;
 
 	dm_gpio_set_dir_flags(&pcie->perstn[idx], GPIOD_IS_OUT);
-	dm_gpio_set_value(&pcie->perstn[idx], 0);
+	dm_gpio_set_value(&pcie->perstn[idx], 1);
 
 	rmwl(0, PORT_APPCLK_EN, pcie->base_port[idx] + PORT_APPCLK);

@@ -221,7 +221,7 @@ static int apple_pcie_setup_port(struct apple_pcie_priv *pcie, unsigned idx)
 	apple_pcie_port_pwren(pcie, idx);
 
 	rmwl(0, PORT_PERST_OFF, pcie->base_port[idx] + PORT_PERST);
-	dm_gpio_set_value(&pcie->perstn[idx], 1);
+	dm_gpio_set_value(&pcie->perstn[idx], 0);
 
 	res = readl_poll_timeout(pcie->base_port[idx] + PORT_STATUS, stat, (stat & PORT_STATUS_READY), 100, 250000);
 	if (res < 0) {

Mark Kettenis

2021-11-22 21:50 UTC #6 lore raw

In reply to Marc Zyngier

Quoted 75 lines

Date: Mon, 22 Nov 2021 14:43:37 +0000
From: Marc Zyngier <[email protected]>
Quoted 27 lines · Pali Rohár
Quoted 16 lines · On Monday 22 November 2021 10:41:56 Marc Zyngier wrote:
While the Apple PCIe driver works correctly when directly booted from the firmware, it fails to initialise when the kernel is booted from a bootloader using PCIe such as u-boot.
That's beacuse we're missing a proper reset of the port (we only clear the reset, but never assert it).
The PCIe spec requirements are two-fold:
- #PERST must be asserted before setting up the clocks, and stay asserted for at least 100us (Tperst-clk).
- Once #PERST is deasserted, the OS must wait for at least 100ms "from the end of a Conventional Reset" before we can start talking to the devices
Quoted 8 lines
Implementing this results in a booting system.
Fixes 1e33888fbe44 ("PCI: apple: Add initial hardware bring-up")
Signed-off-by Marc Zyngier <[email protected]>
Cc Alyssa Rosenzweig <[email protected]>
Cc Lorenzo Pieralisi <[email protected]>
Cc Bjorn Helgaas <[email protected]>
Cc Pali Rohár <[email protected]>
Quoted 3 lines
Looks good, but see comment below.
Acked-by Pali Rohár <[email protected]>
Thanks for that.
Quoted 37 lines
Quoted 16 lines
---
 drivers/pci/controller/pcie-apple.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
drivers/pci/controller/pcie-apple.c +3 -0
index 1bf4d75b61be..957960a733c4 100644
@@ -539,13 +539,23 @@ static int apple_pcie_setup_port(struct apple_pcie *pcie,
 
 	rmw_set(PORT_APPCLK_EN, port->base + PORT_APPCLK);
 
+	/* Engage #PERST before setting up the clock */
+	gpiod_set_value(reset, 0);
+
drivers/pci/controller/pcie-apple.c static int apple_pcie_setup_port(struct apple_pcie *pcie, · line 554
Quoted 10 lines
drivers/pci/controller/pcie-apple.c +4 -0
 	ret = apple_pcie_setup_refclk(pcie, port);
 	if (ret < 0)
 		return ret;
 
+	/* The minimal Tperst-clk value is 100us (PCIe CMS r2.0, 2.6.2) */
+	usleep_range(100, 200);
+
+	/* Deassert #PERST */
 	rmw_set(PORT_PERST_OFF, port->base + PORT_PERST);
 	gpiod_set_value(reset, 1);
+ Luca
Just one comment. PERST# (PCIe Reset) is active-low signal. De-asserting means to really set value to 1.
But there was a discussion that de-asserting should be done by call:
gpiod_set_value(reset, 0);
https://lore.kernel.org/linux-pci/[email protected]/

Quoted 7 lines

Quoted 2 lines
Could we make this new pcie-apple.c driver to use gpiod_set_value(reset, 0) for de-asserting, like in other drivers?
I guess it depends whether you care about the assertion or the signal itself. I think we may have a bug in the way the GPIOs are handled at the moment, as it makes no difference whether I register the GPIO are active high or active low...

That's unfortunate. But maybe that's an opportunity to fix the devicetree to use GPIO_ACTIVE_LOW for these GPIOs?

Marc Zyngier

author maintainer

2021-11-23 12:24 UTC #7 lore raw

Quoted 79 lines · Mark Kettenis

Quoted 79 lines
Date: Mon, 22 Nov 2021 14:43:37 +0000
From: Marc Zyngier <[email protected]>
Quoted 27 lines · Pali Rohár
Quoted 16 lines · On Monday 22 November 2021 10:41:56 Marc Zyngier wrote:
While the Apple PCIe driver works correctly when directly booted from the firmware, it fails to initialise when the kernel is booted from a bootloader using PCIe such as u-boot.
That's beacuse we're missing a proper reset of the port (we only clear the reset, but never assert it).
The PCIe spec requirements are two-fold:
- #PERST must be asserted before setting up the clocks, and stay asserted for at least 100us (Tperst-clk).
- Once #PERST is deasserted, the OS must wait for at least 100ms "from the end of a Conventional Reset" before we can start talking to the devices
Quoted 8 lines
Implementing this results in a booting system.
Fixes 1e33888fbe44 ("PCI: apple: Add initial hardware bring-up")
Signed-off-by Marc Zyngier <[email protected]>
Cc Alyssa Rosenzweig <[email protected]>
Cc Lorenzo Pieralisi <[email protected]>
Cc Bjorn Helgaas <[email protected]>
Cc Pali Rohár <[email protected]>
Quoted 3 lines
Looks good, but see comment below.
Acked-by Pali Rohár <[email protected]>
Thanks for that.
Quoted 39 lines
Quoted 16 lines
---
 drivers/pci/controller/pcie-apple.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
drivers/pci/controller/pcie-apple.c +3 -0
index 1bf4d75b61be..957960a733c4 100644
@@ -539,13 +539,23 @@ static int apple_pcie_setup_port(struct apple_pcie *pcie,
 
 	rmw_set(PORT_APPCLK_EN, port->base + PORT_APPCLK);
 
+	/* Engage #PERST before setting up the clock */
+	gpiod_set_value(reset, 0);
+
drivers/pci/controller/pcie-apple.c static int apple_pcie_setup_port(struct apple_pcie *pcie, · line 554
Quoted 10 lines
drivers/pci/controller/pcie-apple.c +4 -0
 	ret = apple_pcie_setup_refclk(pcie, port);
 	if (ret < 0)
 		return ret;
 
+	/* The minimal Tperst-clk value is 100us (PCIe CMS r2.0, 2.6.2) */
+	usleep_range(100, 200);
+
+	/* Deassert #PERST */
 	rmw_set(PORT_PERST_OFF, port->base + PORT_PERST);
 	gpiod_set_value(reset, 1);
+ Luca
Just one comment. PERST# (PCIe Reset) is active-low signal. De-asserting means to really set value to 1.
But there was a discussion that de-asserting should be done by call:
gpiod_set_value(reset, 0);
https://lore.kernel.org/linux-pci/[email protected]/
Could we make this new pcie-apple.c driver to use gpiod_set_value(reset, 0) for de-asserting, like in other drivers?

Quoted 7 lines

Quoted 4 lines
I guess it depends whether you care about the assertion or the signal itself. I think we may have a bug in the way the GPIOs are handled at the moment, as it makes no difference whether I register the GPIO are active high or active low...
That's unfortunate. But maybe that's an opportunity to fix the devicetree to use GPIO_ACTIVE_LOW for these GPIOs?

Indeed. The following hack does the right thing, and I can then reverse the polarity of the reset in the Linux driver. Of course, it breaks u-boot at the same time (and I suspect OpenBSD would be equally affected).

So if we are going down that road, we may need a flag day where all the moving parts change. I don't really mind not being able to boot older kernels, but this goes beyond Linux at this point.

Thanks,

M.

arch/arm64/boot/dts/apple/t8103.dtsi +4 -3

index d2e9afde3729..cad1ab920304 100644
@@ -10,6 +10,7 @@
 #include <dt-bindings/interrupt-controller/apple-aic.h>
 #include <dt-bindings/interrupt-controller/irq.h>
 #include <dt-bindings/pinctrl/apple.h>
+#include <dt-bindings/gpio/gpio.h>
 
 / {
 	compatible = "apple,t8103", "apple,arm-platform";

@@ -293,7 +294,7 @@ pcie0: pcie@690000000 {
 			port00: pci@0,0 {
 				device_type = "pci";
 				reg = <0x0 0x0 0x0 0x0 0x0>;
-				reset-gpios = <&pinctrl_ap 152 0>;
+				reset-gpios = <&pinctrl_ap 152 GPIO_ACTIVE_LOW>;
 				max-link-speed = <2>;
 
 				#address-cells = <3>;

@@ -313,7 +314,7 @@ port00: pci@0,0 {
 			port01: pci@1,0 {
 				device_type = "pci";
 				reg = <0x800 0x0 0x0 0x0 0x0>;
-				reset-gpios = <&pinctrl_ap 153 0>;
+				reset-gpios = <&pinctrl_ap 153 GPIO_ACTIVE_LOW>;
 				max-link-speed = <2>;
 
 				#address-cells = <3>;

@@ -333,7 +334,7 @@ port01: pci@1,0 {
 			port02: pci@2,0 {
 				device_type = "pci";
 				reg = <0x1000 0x0 0x0 0x0 0x0>;
-				reset-gpios = <&pinctrl_ap 33 0>;
+				reset-gpios = <&pinctrl_ap 33 GPIO_ACTIVE_LOW>;
 				max-link-speed = <1>;
 
 				#address-cells = <3>;

Signature · 1 line

Without deviation from the norm, progress is not possible.

Pali Rohár

2021-11-23 12:27 UTC #8 lore raw

Quoted 86 lines · On Tuesday 23 November 2021 12:24:10 Marc Zyngier wrote:

Quoted 84 lines · Mark Kettenis
Quoted 83 lines
Date: Mon, 22 Nov 2021 14:43:37 +0000
From: Marc Zyngier <[email protected]>
Quoted 27 lines · Pali Rohár
Quoted 16 lines · On Monday 22 November 2021 10:41:56 Marc Zyngier wrote:
While the Apple PCIe driver works correctly when directly booted from the firmware, it fails to initialise when the kernel is booted from a bootloader using PCIe such as u-boot.
That's beacuse we're missing a proper reset of the port (we only clear the reset, but never assert it).
The PCIe spec requirements are two-fold:
- #PERST must be asserted before setting up the clocks, and stay asserted for at least 100us (Tperst-clk).
- Once #PERST is deasserted, the OS must wait for at least 100ms "from the end of a Conventional Reset" before we can start talking to the devices
Quoted 8 lines
Implementing this results in a booting system.
Fixes 1e33888fbe44 ("PCI: apple: Add initial hardware bring-up")
Signed-off-by Marc Zyngier <[email protected]>
Cc Alyssa Rosenzweig <[email protected]>
Cc Lorenzo Pieralisi <[email protected]>
Cc Bjorn Helgaas <[email protected]>
Cc Pali Rohár <[email protected]>
Quoted 3 lines
Looks good, but see comment below.
Acked-by Pali Rohár <[email protected]>
Thanks for that.
Quoted 32 lines
Quoted 16 lines
---
 drivers/pci/controller/pcie-apple.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
drivers/pci/controller/pcie-apple.c +3 -0
index 1bf4d75b61be..957960a733c4 100644
@@ -539,13 +539,23 @@ static int apple_pcie_setup_port(struct apple_pcie *pcie,
 
 	rmw_set(PORT_APPCLK_EN, port->base + PORT_APPCLK);
 
+	/* Engage #PERST before setting up the clock */
+	gpiod_set_value(reset, 0);
+
drivers/pci/controller/pcie-apple.c static int apple_pcie_setup_port(struct apple_pcie *pcie, · line 554
Quoted 10 lines
drivers/pci/controller/pcie-apple.c +4 -0
 	ret = apple_pcie_setup_refclk(pcie, port);
 	if (ret < 0)
 		return ret;
 
+	/* The minimal Tperst-clk value is 100us (PCIe CMS r2.0, 2.6.2) */
+	usleep_range(100, 200);
+
+	/* Deassert #PERST */
 	rmw_set(PORT_PERST_OFF, port->base + PORT_PERST);
 	gpiod_set_value(reset, 1);
+ Luca
Just one comment. PERST# (PCIe Reset) is active-low signal. De-asserting means to really set value to 1.
Quoted 7 lines
But there was a discussion that de-asserting should be done by call:
gpiod_set_value(reset, 0);
https://lore.kernel.org/linux-pci/[email protected]/
Could we make this new pcie-apple.c driver to use gpiod_set_value(reset, 0) for de-asserting, like in other drivers?
I guess it depends whether you care about the assertion or the signal itself. I think we may have a bug in the way the GPIOs are handled at the moment, as it makes no difference whether I register the GPIO are active high or active low...

Quoted 7 lines

Quoted 2 lines
That's unfortunate. But maybe that's an opportunity to fix the devicetree to use GPIO_ACTIVE_LOW for these GPIOs?
Indeed. The following hack does the right thing, and I can then reverse the polarity of the reset in the Linux driver. Of course, it breaks u-boot at the same time (and I suspect OpenBSD would be equally affected).

U-Boot has its own copy of DTS files, right? So it can be fixed in U-Boot driver together with U-Boot DTS file at the same time.

Quoted 50 lines

So if we are going down that road, we may need a flag day where all the moving parts change. I don't really mind not being able to boot older kernels, but this goes beyond Linux at this point.

Thanks,

M.

arch/arm64/boot/dts/apple/t8103.dtsi +4 -3

index d2e9afde3729..cad1ab920304 100644
@@ -10,6 +10,7 @@
 #include <dt-bindings/interrupt-controller/apple-aic.h>
 #include <dt-bindings/interrupt-controller/irq.h>
 #include <dt-bindings/pinctrl/apple.h>
+#include <dt-bindings/gpio/gpio.h>
 
 / {
 	compatible = "apple,t8103", "apple,arm-platform";

@@ -293,7 +294,7 @@ pcie0: pcie@690000000 {
 			port00: pci@0,0 {
 				device_type = "pci";
 				reg = <0x0 0x0 0x0 0x0 0x0>;
-				reset-gpios = <&pinctrl_ap 152 0>;
+				reset-gpios = <&pinctrl_ap 152 GPIO_ACTIVE_LOW>;
 				max-link-speed = <2>;
 
 				#address-cells = <3>;

@@ -313,7 +314,7 @@ port00: pci@0,0 {
 			port01: pci@1,0 {
 				device_type = "pci";
 				reg = <0x800 0x0 0x0 0x0 0x0>;
-				reset-gpios = <&pinctrl_ap 153 0>;
+				reset-gpios = <&pinctrl_ap 153 GPIO_ACTIVE_LOW>;
 				max-link-speed = <2>;
 
 				#address-cells = <3>;

@@ -333,7 +334,7 @@ port01: pci@1,0 {
 			port02: pci@2,0 {
 				device_type = "pci";
 				reg = <0x1000 0x0 0x0 0x0 0x0>;
-				reset-gpios = <&pinctrl_ap 33 0>;
+				reset-gpios = <&pinctrl_ap 33 GPIO_ACTIVE_LOW>;
 				max-link-speed = <1>;
 
 				#address-cells = <3>;

Signature · 2 lines

-- 
Without deviation from the norm, progress is not possible.

Marc Zyngier

author maintainer

2021-11-23 13:09 UTC #9 lore raw

Quoted 3 lines · Pali Rohár

U-Boot has its own copy of DTS files, right? So it can be fixed in U-Boot driver together with U-Boot DTS file at the same time.

No, there is only a single DTS, solely provided by either u-boot or the earlier firmware. The version in the kernel tree is purely for documentation purposes.

M.

Signature · 1 line

Without deviation from the norm, progress is not possible.

Mark Kettenis

2021-11-23 12:49 UTC #10 lore raw

In reply to Marc Zyngier

Quoted 89 lines

Date: Tue, 23 Nov 2021 12:24:10 +0000
From: Marc Zyngier <[email protected]>
Quoted 84 lines · Mark Kettenis
Quoted 83 lines
Date: Mon, 22 Nov 2021 14:43:37 +0000
From: Marc Zyngier <[email protected]>
Quoted 27 lines · Pali Rohár
Quoted 16 lines · On Monday 22 November 2021 10:41:56 Marc Zyngier wrote:
While the Apple PCIe driver works correctly when directly booted from the firmware, it fails to initialise when the kernel is booted from a bootloader using PCIe such as u-boot.
That's beacuse we're missing a proper reset of the port (we only clear the reset, but never assert it).
The PCIe spec requirements are two-fold:
- #PERST must be asserted before setting up the clocks, and stay asserted for at least 100us (Tperst-clk).
- Once #PERST is deasserted, the OS must wait for at least 100ms "from the end of a Conventional Reset" before we can start talking to the devices
Quoted 8 lines
Implementing this results in a booting system.
Fixes 1e33888fbe44 ("PCI: apple: Add initial hardware bring-up")
Signed-off-by Marc Zyngier <[email protected]>
Cc Alyssa Rosenzweig <[email protected]>
Cc Lorenzo Pieralisi <[email protected]>
Cc Bjorn Helgaas <[email protected]>
Cc Pali Rohár <[email protected]>
Quoted 3 lines
Looks good, but see comment below.
Acked-by Pali Rohár <[email protected]>
Thanks for that.
Quoted 32 lines
Quoted 16 lines
---
 drivers/pci/controller/pcie-apple.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
drivers/pci/controller/pcie-apple.c +3 -0
index 1bf4d75b61be..957960a733c4 100644
@@ -539,13 +539,23 @@ static int apple_pcie_setup_port(struct apple_pcie *pcie,
 
 	rmw_set(PORT_APPCLK_EN, port->base + PORT_APPCLK);
 
+	/* Engage #PERST before setting up the clock */
+	gpiod_set_value(reset, 0);
+
drivers/pci/controller/pcie-apple.c static int apple_pcie_setup_port(struct apple_pcie *pcie, · line 554
Quoted 10 lines
drivers/pci/controller/pcie-apple.c +4 -0
 	ret = apple_pcie_setup_refclk(pcie, port);
 	if (ret < 0)
 		return ret;
 
+	/* The minimal Tperst-clk value is 100us (PCIe CMS r2.0, 2.6.2) */
+	usleep_range(100, 200);
+
+	/* Deassert #PERST */
 	rmw_set(PORT_PERST_OFF, port->base + PORT_PERST);
 	gpiod_set_value(reset, 1);
+ Luca
Just one comment. PERST# (PCIe Reset) is active-low signal. De-asserting means to really set value to 1.
Quoted 7 lines
But there was a discussion that de-asserting should be done by call:
gpiod_set_value(reset, 0);
https://lore.kernel.org/linux-pci/[email protected]/
Could we make this new pcie-apple.c driver to use gpiod_set_value(reset, 0) for de-asserting, like in other drivers?
I guess it depends whether you care about the assertion or the signal itself. I think we may have a bug in the way the GPIOs are handled at the moment, as it makes no difference whether I register the GPIO are active high or active low...

Quoted 7 lines

Quoted 2 lines
That's unfortunate. But maybe that's an opportunity to fix the devicetree to use GPIO_ACTIVE_LOW for these GPIOs?
Indeed. The following hack does the right thing, and I can then reverse the polarity of the reset in the Linux driver. Of course, it breaks u-boot at the same time (and I suspect OpenBSD would be equally affected).

As I said in my other reply (that clearly crossed this message) the U-Boot PCIe driver has not been upstreamed yet. OpenBSD (and also NetBSD) rely on the configuration done by U-Boot and don't mess with the reset GPIO at this moment.

Quoted 3 lines

So if we are going down that road, we may need a flag day where all the moving parts change. I don't really mind not being able to boot older kernels, but this goes beyond Linux at this point.

So I don't think this is needed. The earlier we fix this the better!

Quoted 47 lines

arch/arm64/boot/dts/apple/t8103.dtsi +4 -3

index d2e9afde3729..cad1ab920304 100644
@@ -10,6 +10,7 @@
 #include <dt-bindings/interrupt-controller/apple-aic.h>
 #include <dt-bindings/interrupt-controller/irq.h>
 #include <dt-bindings/pinctrl/apple.h>
+#include <dt-bindings/gpio/gpio.h>
 
 / {
 	compatible = "apple,t8103", "apple,arm-platform";

@@ -293,7 +294,7 @@ pcie0: pcie@690000000 {
 			port00: pci@0,0 {
 				device_type = "pci";
 				reg = <0x0 0x0 0x0 0x0 0x0>;
-				reset-gpios = <&pinctrl_ap 152 0>;
+				reset-gpios = <&pinctrl_ap 152 GPIO_ACTIVE_LOW>;
 				max-link-speed = <2>;
 
 				#address-cells = <3>;

@@ -313,7 +314,7 @@ port00: pci@0,0 {
 			port01: pci@1,0 {
 				device_type = "pci";
 				reg = <0x800 0x0 0x0 0x0 0x0>;
-				reset-gpios = <&pinctrl_ap 153 0>;
+				reset-gpios = <&pinctrl_ap 153 GPIO_ACTIVE_LOW>;
 				max-link-speed = <2>;
 
 				#address-cells = <3>;

@@ -333,7 +334,7 @@ port01: pci@1,0 {
 			port02: pci@2,0 {
 				device_type = "pci";
 				reg = <0x1000 0x0 0x0 0x0 0x0>;
-				reset-gpios = <&pinctrl_ap 33 0>;
+				reset-gpios = <&pinctrl_ap 33 GPIO_ACTIVE_LOW>;
 				max-link-speed = <1>;
 
 				#address-cells = <3>;

Signature · 7 lines

-- 
Without deviation from the norm, progress is not possible.

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel