Skip to contentSkip to conversation

PCI: Add Intel PCIe Driver and respective dt-binding yaml file

PATCH Superseded Dilip Kota 10 participants 55 messages Oct 21, 2019 – Nov 4, 2019

Subsystems (5)

v4 of 11 Superseded Latest is v11 →

lore
Share & tools
lore

Dilip Kota

author
#0 lore raw

v4 cover PCI: Add Intel PCIe Driver and respective dt-binding yaml file

Intel PCIe is synopsys based controller utilizes the Designware framework for host initialization and intel application specific register configurations.

Changes on v4:
Add lane resizing API in PCIe DesignWare driver. Intel PCIe driver uses it for lane resizing which
is being exposed through sysfs attributes.
Add Intel PCIe sysfs attributes is in separate patch. Address review comments given on v3.

Changes on v3:
Compared to v2, map_irq() patch is removed as it is no longer
required for Intel PCIe driver. Intel PCIe driver does platform specific interrupt configuration during core initialization. So changed the subject line too.
Address v2 review comments for DT binding and PCIe driver

Dilip Kota (3):
dt-bindings: PCI: intel: Add YAML schemas for the PCIe RC controller
dwc: PCI: intel: PCIe RC controller driver
pci: intel: Add sysfs attributes to configure pcie link

 .../devicetree/bindings/pci/intel-gw-pcie.yaml     | 135 ++++
 drivers/pci/controller/dwc/Kconfig                 |  10 +
 drivers/pci/controller/dwc/Makefile                |   1 +
 drivers/pci/controller/dwc/pcie-designware.c       |  43 ++
 drivers/pci/controller/dwc/pcie-designware.h       |  15 +
 drivers/pci/controller/dwc/pcie-intel-gw.c         | 700 +++++++++++++++
 include/uapi/linux/pci_regs.h                      |   1 +
 7 files changed, 905 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml
 create mode 100644 drivers/pci/controller/dwc/pcie-intel-gw.c
Signature · 1 line
2.11.0
author
#1 lore raw

PATCH v4 1/3 dt-bindings: PCI: intel: Add YAML schemas for the PCIe RC controller

Add YAML shcemas for PCIe RC controller on Intel Gateway SoCs which is Synopsys DesignWare based PCIe core.

changes on v4:
Add "snps,dw-pcie" compatible.
Rename phy-names property value to pcie. And maximum and minimum values to num-lanes. Add ref for reset-assert-ms entry and update the
description for easy understanding.
Remove pcie core interrupt entry.

changes on v3:
Add the appropriate License-Identifier
Rename intel,rst-interval to 'reset-assert-us'
Add additionalProperties: false
Rename phy-names to 'pciephy'
Remove the dtsi node split of SoC and board in the example Add #interrupt-cells = <1>; or else interrupt parsing will fail Name yaml file with compatible name

---
 .../devicetree/bindings/pci/intel-gw-pcie.yaml     | 135 +++++++++++++++++++++
 1 file changed, 135 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml
Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml new file +135 -0
new file mode 100644
index 000000000000..49dd87ec1e3d
@@ -0,0 +1,135 @@
+# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/pci/intel-gw-pcie.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: PCIe RC controller on Intel Gateway SoCs
+
+maintainers:
+  - Dilip Kota <[email protected]>
+
+properties:
+  compatible:
+    items:
+      - const: intel,lgm-pcie
+      - const: snps,dw-pcie
+
+  device_type:
+    const: pci
+
+  "#address-cells":
+    const: 3
+
+  "#size-cells":
+    const: 2
+
+  reg:
+    items:
+      - description: Controller control and status registers.
+      - description: PCIe configuration registers.
+      - description: Controller application registers.
+
+  reg-names:
+    items:
+      - const: dbi
+      - const: config
+      - const: app
+
+  ranges:
+    description: Ranges for the PCI memory and I/O regions.
+
+  resets:
+    maxItems: 1
+
+  clocks:
+    description: PCIe registers interface clock.
+
+  phys:
+    maxItems: 1
+
+  phy-names:
+    const: pcie
+
+  reset-gpios:
+    maxItems: 1
+
+  num-lanes:
+    minimum: 1
+    maximum: 2
+    description: Number of lanes to use for this port.
+
+  linux,pci-domain:
+    $ref: /schemas/types.yaml#/definitions/uint32
+    description: PCI domain ID.
+
+  '#interrupt-cells':
+    const: 1
+
+  interrupt-map-mask:
+    description: Standard PCI IRQ mapping properties.
+
+  interrupt-map:
+    description: Standard PCI IRQ mapping properties.
+
+  max-link-speed:
+    description: Specify PCI Gen for link capability.
+
+  bus-range:
+    description: Range of bus numbers associated with this controller.
+
+  reset-assert-ms:
+    $ref: /schemas/types.yaml#/definitions/uint32
+    description: |
+      Delay after asserting reset to the PCIe device.
+      Some devices need an interval upto 500ms. By default it is 100ms.
+
+required:
+  - compatible
+  - device_type
+  - reg
+  - reg-names
+  - ranges
+  - resets
+  - clocks
+  - phys
+  - phy-names
+  - reset-gpios
+  - num-lanes
+  - linux,pci-domain
+  - interrupt-map
+  - interrupt-map-mask
+
+additionalProperties: false
+
+examples:
+  - |
+    pcie10:pcie@d0e00000 {
+      compatible = "intel,lgm-pcie", "snps,dw-pcie";
+      device_type = "pci";
+      #address-cells = <3>;
+      #size-cells = <2>;
+      reg = <0xd0e00000 0x1000>,
+            <0xd2000000 0x800000>,
+            <0xd0a41000 0x1000>;
+      reg-names = "dbi", "config", "app";
+      linux,pci-domain = <0>;
+      max-link-speed = <4>;
+      bus-range = <0x00 0x08>;
+      interrupt-parent = <&ioapic1>;
+      #interrupt-cells = <1>;
+      interrupt-map-mask = <0 0 0 0x7>;
+      interrupt-map = <0 0 0 1 &ioapic1 27 1>,
+                      <0 0 0 2 &ioapic1 28 1>,
+                      <0 0 0 3 &ioapic1 29 1>,
+                      <0 0 0 4 &ioapic1 30 1>;
+      ranges = <0x02000000 0 0xd4000000 0xd4000000 0 0x04000000>;
+      resets = <&rcu0 0x50 0>;
+      clocks = <&cgu0 LGM_GCLK_PCIE10>;
+      phys = <&cb0phy0>;
+      phy-names = "pcie";
+      status = "okay";
+      reset-assert-ms = <500>;
+      reset-gpios = <&gpio0 3 GPIO_ACTIVE_LOW>;
+      num-lanes = <2>;
+    };
Signature · 1 line
2.11.0
Reviewed-by on v4 1/3
#2 lore raw

Re: [PATCH v4 1/3] dt-bindings: PCI: intel: Add YAML schemas for the PCIe RC controller

Quoted 1 line · Dilip Kota

Add YAML shcemas for PCIe RC controller on Intel Gateway SoCs

s/shcemas/schemas/

Quoted 1 line

which is Synopsys DesignWare based PCIe core.

The revision history below doesn't need to be in the commit mesage and so you should add a '---' before the following (and thanks for the detailed history).

Besides that:

Quoted 168 lines

changes on v4:
Add "snps,dw-pcie" compatible.
Rename phy-names property value to pcie. And maximum and minimum values to num-lanes. Add ref for reset-assert-ms entry and update the
description for easy understanding.
Remove pcie core interrupt entry.

changes on v3:
Add the appropriate License-Identifier
Rename intel,rst-interval to 'reset-assert-us'
Add additionalProperties: false
Rename phy-names to 'pciephy'
Remove the dtsi node split of SoC and board in the example Add #interrupt-cells = <1>; or else interrupt parsing will fail Name yaml file with compatible name

---
 .../devicetree/bindings/pci/intel-gw-pcie.yaml     | 135 +++++++++++++++++++++
 1 file changed, 135 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml
Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml new file +135 -0
new file mode 100644
index 000000000000..49dd87ec1e3d
@@ -0,0 +1,135 @@
+# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/pci/intel-gw-pcie.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: PCIe RC controller on Intel Gateway SoCs
+
+maintainers:
+  - Dilip Kota <[email protected]>
+
+properties:
+  compatible:
+    items:
+      - const: intel,lgm-pcie
+      - const: snps,dw-pcie
+
+  device_type:
+    const: pci
+
+  "#address-cells":
+    const: 3
+
+  "#size-cells":
+    const: 2
+
+  reg:
+    items:
+      - description: Controller control and status registers.
+      - description: PCIe configuration registers.
+      - description: Controller application registers.
+
+  reg-names:
+    items:
+      - const: dbi
+      - const: config
+      - const: app
+
+  ranges:
+    description: Ranges for the PCI memory and I/O regions.
+
+  resets:
+    maxItems: 1
+
+  clocks:
+    description: PCIe registers interface clock.
+
+  phys:
+    maxItems: 1
+
+  phy-names:
+    const: pcie
+
+  reset-gpios:
+    maxItems: 1
+
+  num-lanes:
+    minimum: 1
+    maximum: 2
+    description: Number of lanes to use for this port.
+
+  linux,pci-domain:
+    $ref: /schemas/types.yaml#/definitions/uint32
+    description: PCI domain ID.
+
+  '#interrupt-cells':
+    const: 1
+
+  interrupt-map-mask:
+    description: Standard PCI IRQ mapping properties.
+
+  interrupt-map:
+    description: Standard PCI IRQ mapping properties.
+
+  max-link-speed:
+    description: Specify PCI Gen for link capability.
+
+  bus-range:
+    description: Range of bus numbers associated with this controller.
+
+  reset-assert-ms:
+    $ref: /schemas/types.yaml#/definitions/uint32
+    description: |
+      Delay after asserting reset to the PCIe device.
+      Some devices need an interval upto 500ms. By default it is 100ms.
+
+required:
+  - compatible
+  - device_type
+  - reg
+  - reg-names
+  - ranges
+  - resets
+  - clocks
+  - phys
+  - phy-names
+  - reset-gpios
+  - num-lanes
+  - linux,pci-domain
+  - interrupt-map
+  - interrupt-map-mask
+
+additionalProperties: false
+
+examples:
+  - |
+    pcie10:pcie@d0e00000 {
+      compatible = "intel,lgm-pcie", "snps,dw-pcie";
+      device_type = "pci";
+      #address-cells = <3>;
+      #size-cells = <2>;
+      reg = <0xd0e00000 0x1000>,
+            <0xd2000000 0x800000>,
+            <0xd0a41000 0x1000>;
+      reg-names = "dbi", "config", "app";
+      linux,pci-domain = <0>;
+      max-link-speed = <4>;
+      bus-range = <0x00 0x08>;
+      interrupt-parent = <&ioapic1>;
+      #interrupt-cells = <1>;
+      interrupt-map-mask = <0 0 0 0x7>;
+      interrupt-map = <0 0 0 1 &ioapic1 27 1>,
+                      <0 0 0 2 &ioapic1 28 1>,
+                      <0 0 0 3 &ioapic1 29 1>,
+                      <0 0 0 4 &ioapic1 30 1>;
+      ranges = <0x02000000 0 0xd4000000 0xd4000000 0 0x04000000>;
+      resets = <&rcu0 0x50 0>;
+      clocks = <&cgu0 LGM_GCLK_PCIE10>;
+      phys = <&cb0phy0>;
+      phy-names = "pcie";
+      status = "okay";
+      reset-assert-ms = <500>;
+      reset-gpios = <&gpio0 3 GPIO_ACTIVE_LOW>;
+      num-lanes = <2>;
+    };
Signature · 1 line
2.11.0
author
#3 lore raw

Re: [PATCH v4 1/3] dt-bindings: PCI: intel: Add YAML schemas for the PCIe RC controller

Hi Andrew Murray,

Quoted 4 lines · Andrew Murray
Quoted 1 line · Dilip Kota

Add YAML shcemas for PCIe RC controller on Intel Gateway SoCs

s/shcemas/schemas/

Quoted 8 lines
Quoted 1 line

which is Synopsys DesignWare based PCIe core.

The revision history below doesn't need to be in the commit mesage and so you should add a '---' before the following (and thanks for the detailed history).

Besides that:

Thank you for the review. I will fix the conventions

Regards,
Dilip

Quoted 168 lines
Quoted 168 lines

changes on v4:
Add "snps,dw-pcie" compatible.
Rename phy-names property value to pcie. And maximum and minimum values to num-lanes. Add ref for reset-assert-ms entry and update the
description for easy understanding.
Remove pcie core interrupt entry.

changes on v3:
Add the appropriate License-Identifier
Rename intel,rst-interval to 'reset-assert-us'
Add additionalProperties: false
Rename phy-names to 'pciephy'
Remove the dtsi node split of SoC and board in the example Add #interrupt-cells = <1>; or else interrupt parsing will fail Name yaml file with compatible name

---
  .../devicetree/bindings/pci/intel-gw-pcie.yaml     | 135 +++++++++++++++++++++
  1 file changed, 135 insertions(+)

create mode 100644 Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml

Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml new file +135 -0
new file mode 100644
index 000000000000..49dd87ec1e3d
@@ -0,0 +1,135 @@
+# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/pci/intel-gw-pcie.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: PCIe RC controller on Intel Gateway SoCs
+
+maintainers:
+  - Dilip Kota <[email protected]>
+
+properties:
+  compatible:
+    items:
+      - const: intel,lgm-pcie
+      - const: snps,dw-pcie
+
+  device_type:
+    const: pci
+
+  "#address-cells":
+    const: 3
+
+  "#size-cells":
+    const: 2
+
+  reg:
+    items:
+      - description: Controller control and status registers.
+      - description: PCIe configuration registers.
+      - description: Controller application registers.
+
+  reg-names:
+    items:
+      - const: dbi
+      - const: config
+      - const: app
+
+  ranges:
+    description: Ranges for the PCI memory and I/O regions.
+
+  resets:
+    maxItems: 1
+
+  clocks:
+    description: PCIe registers interface clock.
+
+  phys:
+    maxItems: 1
+
+  phy-names:
+    const: pcie
+
+  reset-gpios:
+    maxItems: 1
+
+  num-lanes:
+    minimum: 1
+    maximum: 2
+    description: Number of lanes to use for this port.
+
+  linux,pci-domain:
+    $ref: /schemas/types.yaml#/definitions/uint32
+    description: PCI domain ID.
+
+  '#interrupt-cells':
+    const: 1
+
+  interrupt-map-mask:
+    description: Standard PCI IRQ mapping properties.
+
+  interrupt-map:
+    description: Standard PCI IRQ mapping properties.
+
+  max-link-speed:
+    description: Specify PCI Gen for link capability.
+
+  bus-range:
+    description: Range of bus numbers associated with this controller.
+
+  reset-assert-ms:
+    $ref: /schemas/types.yaml#/definitions/uint32
+    description: |
+      Delay after asserting reset to the PCIe device.
+      Some devices need an interval upto 500ms. By default it is 100ms.
+
+required:
+  - compatible
+  - device_type
+  - reg
+  - reg-names
+  - ranges
+  - resets
+  - clocks
+  - phys
+  - phy-names
+  - reset-gpios
+  - num-lanes
+  - linux,pci-domain
+  - interrupt-map
+  - interrupt-map-mask
+
+additionalProperties: false
+
+examples:
+  - |
+    pcie10:pcie@d0e00000 {
+      compatible = "intel,lgm-pcie", "snps,dw-pcie";
+      device_type = "pci";
+      #address-cells = <3>;
+      #size-cells = <2>;
+      reg = <0xd0e00000 0x1000>,
+            <0xd2000000 0x800000>,
+            <0xd0a41000 0x1000>;
+      reg-names = "dbi", "config", "app";
+      linux,pci-domain = <0>;
+      max-link-speed = <4>;
+      bus-range = <0x00 0x08>;
+      interrupt-parent = <&ioapic1>;
+      #interrupt-cells = <1>;
+      interrupt-map-mask = <0 0 0 0x7>;
+      interrupt-map = <0 0 0 1 &ioapic1 27 1>,
+                      <0 0 0 2 &ioapic1 28 1>,
+                      <0 0 0 3 &ioapic1 29 1>,
+                      <0 0 0 4 &ioapic1 30 1>;
+      ranges = <0x02000000 0 0xd4000000 0xd4000000 0 0x04000000>;
+      resets = <&rcu0 0x50 0>;
+      clocks = <&cgu0 LGM_GCLK_PCIE10>;
+      phys = <&cb0phy0>;
+      phy-names = "pcie";
+      status = "okay";
+      reset-assert-ms = <500>;
+      reset-gpios = <&gpio0 3 GPIO_ACTIVE_LOW>;
+      num-lanes = <2>;
+    };
Signature · 1 line
2.11.0
#4 lore raw

Re: [PATCH v4 1/3] dt-bindings: PCI: intel: Add YAML schemas for the PCIe RC controller

In reply to Dilip Kota

Hi Dilip,

thank you for the update
two nit-picks below, apart from those it looks good to me

On Mon, Oct 21, 2019 at 8:39 AM Dilip Kota <[email protected]> wrote:
[...]

Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml line 107

Quoted 3 lines
Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml new file +3 -0
+examples:
+  - |
+    pcie10:pcie@d0e00000 {

why no space between pcie10 and pcie@d0e00000?

[...]

Quoted 1 line

+ status = "okay";

examples should not use the status property, see commit 4da722ca19f30f ("dt-bindings: Remove "status" from examples")

Martin

author
#5 lore raw

Re: [PATCH v4 1/3] dt-bindings: PCI: intel: Add YAML schemas for the PCIe RC controller

Hi Martin,

Quoted 4 lines · Martin Blumenstingl

Hi Dilip,

thank you for the update
two nit-picks below, apart from those it looks good to me

Thanks for reviewing it.

Quoted 7 lines · Dilip Kota

[...]

Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml line 107

Quoted 3 lines
Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml new file +3 -0
+examples:
+  - |
+    pcie10:pcie@d0e00000 {

why no space between pcie10 and pcie@d0e00000?

Agree, will fix it in the next patch version.

Quoted 5 lines

[...]

Quoted 1 line

+ status = "okay";

examples should not use the status property, see commit 4da722ca19f30f ("dt-bindings: Remove "status" from examples")

Agree, will fix it in the next patch version. (sorry for the late reply, i am back today from sick leave).

Regards,
Dilip

Quoted 3 lines

Martin

maintainer
#6 lore raw

Re: [PATCH v4 1/3] dt-bindings: PCI: intel: Add YAML schemas for the PCIe RC controller

In reply to Dilip Kota
Quoted 20 lines · Dilip Kota

Add YAML shcemas for PCIe RC controller on Intel Gateway SoCs which is Synopsys DesignWare based PCIe core.

changes on v4:
Add "snps,dw-pcie" compatible.
Rename phy-names property value to pcie. And maximum and minimum values to num-lanes. Add ref for reset-assert-ms entry and update the
description for easy understanding.
Remove pcie core interrupt entry.

changes on v3:
Add the appropriate License-Identifier
Rename intel,rst-interval to 'reset-assert-us'
Add additionalProperties: false
Rename phy-names to 'pciephy'
Remove the dtsi node split of SoC and board in the example Add #interrupt-cells = <1>; or else interrupt parsing will fail Name yaml file with compatible name

Quoted 5 lines
---
 .../devicetree/bindings/pci/intel-gw-pcie.yaml     | 135 +++++++++++++++++++++
 1 file changed, 135 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml

Fails to validate:

Error: Documentation/devicetree/bindings/pci/intel-gw-pcie.example.dts:38.27-28 syntax error
FATAL ERROR: Unable to parse input tree
scripts/Makefile.lib:321: recipe for target 'Documentation/devicetree/bindings/pci/intel-gw-pcie.example.dt.yaml' failed

Please run 'make -k dt_binding_check' (-k because there are some unrelated failures).

Quoted 37 lines
Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml new file +30 -0
new file mode 100644
index 000000000000..49dd87ec1e3d
@@ -0,0 +1,135 @@
+# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/pci/intel-gw-pcie.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: PCIe RC controller on Intel Gateway SoCs
+
+maintainers:
+  - Dilip Kota <[email protected]>
+
+properties:
+  compatible:
+    items:
+      - const: intel,lgm-pcie
+      - const: snps,dw-pcie
+
+  device_type:
+    const: pci
+
+  "#address-cells":
+    const: 3
+
+  "#size-cells":
+    const: 2
+
+  reg:
+    items:
+      - description: Controller control and status registers.
+      - description: PCIe configuration registers.

Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml line 40

Quoted 10 lines
Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml new file +10 -0
+      - description: Controller application registers.
+
+  reg-names:
+    items:
+      - const: dbi
+      - const: config
+      - const: app
+
+  ranges:
+    description: Ranges for the PCI memory and I/O regions.

How many entries do you expect? Add a 'maxItems' to define.

Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml line 46

Quoted 6 lines
Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml new file +6 -0
+
+  resets:
+    maxItems: 1
+
+  clocks:
+    description: PCIe registers interface clock.

How many clocks?

Quoted 8 lines
Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml new file +8 -0
+
+  phys:
+    maxItems: 1
+
+  phy-names:
+    const: pcie
+
+  reset-gpios:

Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml line 64

Quoted 10 lines
Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml new file +10 -0
+    maxItems: 1
+
+  num-lanes:
+    minimum: 1
+    maximum: 2
+    description: Number of lanes to use for this port.
+
+  linux,pci-domain:
+    $ref: /schemas/types.yaml#/definitions/uint32
+    description: PCI domain ID.

Just a value of 'true' is fine here.

Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml line 76

Quoted 12 lines
Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml new file +12 -0
+
+  '#interrupt-cells':
+    const: 1
+
+  interrupt-map-mask:
+    description: Standard PCI IRQ mapping properties.
+
+  interrupt-map:
+    description: Standard PCI IRQ mapping properties.
+
+  max-link-speed:
+    description: Specify PCI Gen for link capability.

Allowed values? Default?

Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml line 82

Quoted 6 lines
Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml new file +6 -0
+
+  bus-range:
+    description: Range of bus numbers associated with this controller.
+
+  reset-assert-ms:
+    $ref: /schemas/types.yaml#/definitions/uint32

Don't need a type for standard units.

Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml line 85

Quoted 3 lines
Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml new file +3 -0
+    description: |
+      Delay after asserting reset to the PCIe device.
+      Some devices need an interval upto 500ms. By default it is 100ms.

Express as a schema:

maximum: 500
default: 100

Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml line 98

Quoted 13 lines
Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml new file +13 -0
+
+required:
+  - compatible
+  - device_type
+  - reg
+  - reg-names
+  - ranges
+  - resets
+  - clocks
+  - phys
+  - phy-names
+  - reset-gpios
+  - num-lanes

Shouldn't be required. It should have a default.

Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml line 99

Quoted 1 line
Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml new file +1 -0
+  - linux,pci-domain

Is this really required? AIUI, domains are optional and only used if you have more than one host.

Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml line 107

Quoted 8 lines
Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml new file +8 -0
+  - interrupt-map
+  - interrupt-map-mask
+
+additionalProperties: false
+
+examples:
+  - |
+    pcie10:pcie@d0e00000 {

space ^

Quoted 11 lines
Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml new file +11 -0
+      compatible = "intel,lgm-pcie", "snps,dw-pcie";
+      device_type = "pci";
+      #address-cells = <3>;
+      #size-cells = <2>;
+      reg = <0xd0e00000 0x1000>,
+            <0xd2000000 0x800000>,
+            <0xd0a41000 0x1000>;
+      reg-names = "dbi", "config", "app";
+      linux,pci-domain = <0>;
+      max-link-speed = <4>;
+      bus-range = <0x00 0x08>;

Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml line 128

Quoted 10 lines
Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml new file +10 -0
+      interrupt-parent = <&ioapic1>;
+      #interrupt-cells = <1>;
+      interrupt-map-mask = <0 0 0 0x7>;
+      interrupt-map = <0 0 0 1 &ioapic1 27 1>,
+                      <0 0 0 2 &ioapic1 28 1>,
+                      <0 0 0 3 &ioapic1 29 1>,
+                      <0 0 0 4 &ioapic1 30 1>;
+      ranges = <0x02000000 0 0xd4000000 0xd4000000 0 0x04000000>;
+      resets = <&rcu0 0x50 0>;
+      clocks = <&cgu0 LGM_GCLK_PCIE10>;

You need to include any defines you use. That's why the example fails to build.

Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml line 131

Quoted 3 lines
Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml new file +3 -0
+      phys = <&cb0phy0>;
+      phy-names = "pcie";
+      status = "okay";

Don't show status in examples.

Quoted 7 lines
Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml new file +4 -0
+      reset-assert-ms = <500>;
+      reset-gpios = <&gpio0 3 GPIO_ACTIVE_LOW>;
+      num-lanes = <2>;
+    };
Signature · 1 line
2.11.0
author
#7 lore raw

Re: [PATCH v4 1/3] dt-bindings: PCI: intel: Add YAML schemas for the PCIe RC controller

Quoted 74 lines · Rob Herring
Quoted 20 lines · Dilip Kota

Add YAML shcemas for PCIe RC controller on Intel Gateway SoCs which is Synopsys DesignWare based PCIe core.

changes on v4:
Add "snps,dw-pcie" compatible.
Rename phy-names property value to pcie. And maximum and minimum values to num-lanes. Add ref for reset-assert-ms entry and update the
description for easy understanding.
Remove pcie core interrupt entry.

changes on v3:
Add the appropriate License-Identifier
Rename intel,rst-interval to 'reset-assert-us'
Add additionalProperties: false
Rename phy-names to 'pciephy'
Remove the dtsi node split of SoC and board in the example Add #interrupt-cells = <1>; or else interrupt parsing will fail Name yaml file with compatible name

Quoted 5 lines
---
  .../devicetree/bindings/pci/intel-gw-pcie.yaml     | 135 +++++++++++++++++++++
  1 file changed, 135 insertions(+)

create mode 100644 Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml

Fails to validate:

Error: Documentation/devicetree/bindings/pci/intel-gw-pcie.example.dts:38.27-28 syntax error
FATAL ERROR: Unable to parse input tree
scripts/Makefile.lib:321: recipe for target 'Documentation/devicetree/bindings/pci/intel-gw-pcie.example.dt.yaml' failed

Please run 'make -k dt_binding_check' (-k because there are some unrelated failures).

Quoted 39 lines
Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml new file +33 -0
new file mode 100644
index 000000000000..49dd87ec1e3d
@@ -0,0 +1,135 @@
+# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/pci/intel-gw-pcie.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: PCIe RC controller on Intel Gateway SoCs
+
+maintainers:
+  - Dilip Kota <[email protected]>
+
+properties:
+  compatible:
+    items:
+      - const: intel,lgm-pcie
+      - const: snps,dw-pcie
+
+  device_type:
+    const: pci
+
+  "#address-cells":
+    const: 3
+
+  "#size-cells":
+    const: 2
+
+  reg:
+    items:
+      - description: Controller control and status registers.
+      - description: PCIe configuration registers.
+      - description: Controller application registers.
+
+  reg-names:
Quoted 8 lines

Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml line 40

Quoted 7 lines
Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml new file +7 -0
+    items:
+      - const: dbi
+      - const: config
+      - const: app
+
+  ranges:
+    description: Ranges for the PCI memory and I/O regions.

How many entries do you expect? Add a 'maxItems' to define.

Agree will add it.

Quoted 8 lines

Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml line 46

Quoted 7 lines
Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml new file +6 -0
+
+  resets:
+    maxItems: 1
+
+  clocks:
+    description: PCIe registers interface clock.

How many clocks?

One. I will mention maxItems: 1

Quoted 12 lines
Quoted 12 lines
Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml new file +11 -0
+
+  phys:
+    maxItems: 1
+
+  phy-names:
+    const: pcie
+
+  reset-gpios:
+    maxItems: 1
+
+  num-lanes:
Quoted 8 lines

Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml line 64

Quoted 7 lines
Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml new file +7 -0
+    minimum: 1
+    maximum: 2
+    description: Number of lanes to use for this port.
+
+  linux,pci-domain:
+    $ref: /schemas/types.yaml#/definitions/uint32
+    description: PCI domain ID.

Just a value of 'true' is fine here.

Ok.

Quoted 14 lines

Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml line 76

Quoted 13 lines
Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml new file +12 -0
+
+  '#interrupt-cells':
+    const: 1
+
+  interrupt-map-mask:
+    description: Standard PCI IRQ mapping properties.
+
+  interrupt-map:
+    description: Standard PCI IRQ mapping properties.
+
+  max-link-speed:
+    description: Specify PCI Gen for link capability.

Allowed values? Default?

Sure, will add it.

Quoted 8 lines

Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml line 82

Quoted 7 lines
Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml new file +6 -0
+
+  bus-range:
+    description: Range of bus numbers associated with this controller.
+
+  reset-assert-ms:
+    $ref: /schemas/types.yaml#/definitions/uint32

Don't need a type for standard units.

Ok.

Quoted 8 lines

Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml line 85

Quoted 4 lines
Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml new file +3 -0
+    description: |
+      Delay after asserting reset to the PCIe device.
+      Some devices need an interval upto 500ms. By default it is 100ms.

Express as a schema:

maximum: 500
default: 100

Sure i will update it.

Quoted 15 lines
Quoted 4 lines
Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml new file +3 -0
+
+required:
+  - compatible

Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml line 98

Quoted 10 lines
Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml new file +10 -0
+  - device_type
+  - reg
+  - reg-names
+  - ranges
+  - resets
+  - clocks
+  - phys
+  - phy-names
+  - reset-gpios
+  - num-lanes

Shouldn't be required. It should have a default.

Agree, will fix it.

Quoted 4 lines

Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml line 99

Quoted 2 lines
Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml new file +1 -0
+  - linux,pci-domain

Is this really required? AIUI, domains are optional and only used if you have more than one host.

Yes, not required. I will update,

Quoted 10 lines

Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml line 107

Quoted 9 lines
Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml new file +8 -0
+  - interrupt-map
+  - interrupt-map-mask
+
+additionalProperties: false
+
+examples:
+  - |
+    pcie10:pcie@d0e00000 {

space ^

Agree, i will fix it.

Quoted 22 lines
Quoted 12 lines
Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml new file +11 -0
+      compatible = "intel,lgm-pcie", "snps,dw-pcie";
+      device_type = "pci";
+      #address-cells = <3>;
+      #size-cells = <2>;
+      reg = <0xd0e00000 0x1000>,
+            <0xd2000000 0x800000>,
+            <0xd0a41000 0x1000>;
+      reg-names = "dbi", "config", "app";
+      linux,pci-domain = <0>;
+      max-link-speed = <4>;
+      bus-range = <0x00 0x08>;

Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml line 128

Quoted 10 lines
Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml new file +10 -0
+      interrupt-parent = <&ioapic1>;
+      #interrupt-cells = <1>;
+      interrupt-map-mask = <0 0 0 0x7>;
+      interrupt-map = <0 0 0 1 &ioapic1 27 1>,
+                      <0 0 0 2 &ioapic1 28 1>,
+                      <0 0 0 3 &ioapic1 29 1>,
+                      <0 0 0 4 &ioapic1 30 1>;
+      ranges = <0x02000000 0 0xd4000000 0xd4000000 0 0x04000000>;
+      resets = <&rcu0 0x50 0>;
+      clocks = <&cgu0 LGM_GCLK_PCIE10>;
Quoted 2 lines

You need to include any defines you use. That's why the example fails to build.

Yes, i will add it.

Quoted 5 lines

Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml line 131

Quoted 4 lines
Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml new file +3 -0
+      phys = <&cb0phy0>;
+      phy-names = "pcie";
+      status = "okay";

Don't show status in examples.

OK, will fix it.

Thanks for reviewing it.

Regards,
Dilip

Quoted 8 lines
Quoted 8 lines
Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml new file +4 -0
+      reset-assert-ms = <500>;
+      reset-gpios = <&gpio0 3 GPIO_ACTIVE_LOW>;
+      num-lanes = <2>;
+    };
Signature · 1 line
2.11.0
author
#8 lore raw

Re: [PATCH v4 1/3] dt-bindings: PCI: intel: Add YAML schemas for the PCIe RC controller

Quoted 6 lines · Dilip Kota
Quoted 4 lines · Rob Herring
Quoted 3 lines · Dilip Kota

Add YAML shcemas for PCIe RC controller on Intel Gateway SoCs which is Synopsys DesignWare based PCIe core.

[...]

Quoted 8 lines
Quoted 7 lines
Quoted 1 line

+
+  linux,pci-domain:
+    $ref: /schemas/types.yaml#/definitions/uint32 +    description: PCI domain ID.
Just a value of 'true' is fine here.

Ok.

dt_binding_check is failing for adding 'true' alone. It is passing only if "$ref" is mentioned.

Regards,
Dilip

maintainer
#9 lore raw

Re: [PATCH v4 1/3] dt-bindings: PCI: intel: Add YAML schemas for the PCIe RC controller

Quoted 12 lines · Dilip Kota
Quoted 6 lines · Dilip Kota
Quoted 4 lines · Rob Herring
Quoted 3 lines · Dilip Kota

Add YAML shcemas for PCIe RC controller on Intel Gateway SoCs which is Synopsys DesignWare based PCIe core.

[...]

Quoted 2 lines
Quoted 2 lines
Quoted 1 line
Quoted 8 lines
Quoted 6 lines
Quoted 5 lines
Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml new file +4 -0
+
+  linux,pci-domain:
+    $ref: /schemas/types.yaml#/definitions/uint32
+    description: PCI domain ID.

Just a value of 'true' is fine here.

Ok.

dt_binding_check is failing for adding 'true' alone. It is passing only if "$ref" is mentioned.

Update dtschema. That should be fixed.

Rob

author
#10 lore raw

PATCH v4 2/3 dwc: PCI: intel: PCIe RC controller driver

In reply to Dilip Kota

Add support to PCIe RC controller on Intel Gateway SoCs. PCIe controller is based of Synopsys DesignWare pci core.

Intel PCIe driver requires Upconfig support, fast training sequence configuration and link speed change. So adding the respective helper functions in the pcie DesignWare framework. It also programs hardware autonomous speed during speed configuration so defining it in pci_regs.h.

Changes on v4:
Rename the driver naming and description to
"PCIe RC controller on Intel Gateway SoCs".
Use PCIe core register macros defined in pci_regs.h
and remove respective local definitions.
Remove pcie core interrupt handling.
Move pcie link control speed change, upconfig and FTS. configuration functions to DesignWare framework.
Use of_pci_get_max_link_speed().
Mark dependency on X86 and COMPILE_TEST in Kconfig.
Remove lanes and add n_fts variables in intel_pcie_port structure. Rename rst_interval variable to rst_intrvl in intel_pcie_port structure. Remove intel_pcie_mem_iatu() as it is already perfomed in dw_setup_rc() Move sysfs attributes specific code to separate patch. Remove redundant error handling.
Reduce LoCs by doing variable initializations while declaration itself. Add extra line after closing parenthesis.
Move intel_pcie_ep_rst_init() out of get_resources()

changes on v3:
Rename PCIe app logic registers with PCIE_APP prefix. PCIE_IOP_CTRL configuration is not required. Remove respective code. Remove wrapper functions for clk enable/disable APIs.
Use platform_get_resource_byname() instead of
devm_platform_ioremap_resource() to be similar with DWC framework.
Rename phy name to "pciephy".
Modify debug message in msi_init() callback to be more specific. Remove map_irq() callback.
Enable the INTx interrupts at the time of PCIe initialization. Reduce memory fragmentation by using variable "struct dw_pcie pci"
instead of allocating memory.
Reduce the delay to 100us during enpoint initialization
intel_pcie_ep_rst_init().
Call dw_pcie_host_deinit() during remove() instead of directly
calling PCIe core APIs.
Rename "intel,rst-interval" to "reset-assert-ms".
Remove unused APP logic Interrupt bit macro definitions. Use dwc framework's dw_pcie_setup_rc() for PCIe host specific
configuration instead of redefining the same functionality in the driver.
Move the whole DT parsing specific code to intel_pcie_get_resources()

---
 drivers/pci/controller/dwc/Kconfig           |  10 +
 drivers/pci/controller/dwc/Makefile          |   1 +
 drivers/pci/controller/dwc/pcie-designware.c |  34 ++
 drivers/pci/controller/dwc/pcie-designware.h |  12 +
 drivers/pci/controller/dwc/pcie-intel-gw.c   | 590 +++++++++++++++++++++++++++
 include/uapi/linux/pci_regs.h                |   1 +
 6 files changed, 648 insertions(+)
 create mode 100644 drivers/pci/controller/dwc/pcie-intel-gw.c
drivers/pci/controller/dwc/Kconfig +10 -0
index 0ba988b5b5bc..b33ed1cc873d 100644
@@ -82,6 +82,16 @@ config PCIE_DW_PLAT_EP
 	  order to enable device-specific features PCI_DW_PLAT_EP must be
 	  selected.
 
+config PCIE_INTEL_GW
+        bool "Intel Gateway PCIe host controller support"
+	depends on OF && (X86 || COMPILE_TEST)
+	select PCIE_DW_HOST
+	help
+          Say 'Y' here to enable support for PCIe Host controller driver.
+	  The PCIe controller on Intel Gateway SoCs is based on the Synopsys
+	  DesignWare pcie core and therefore uses the DesignWare core
+	  functions for the driver implementation.
+
 config PCI_EXYNOS
 	bool "Samsung Exynos PCIe controller"
 	depends on SOC_EXYNOS5440 || COMPILE_TEST
drivers/pci/controller/dwc/Makefile +1 -0
index b30336181d46..99db83cd2f35 100644
@@ -3,6 +3,7 @@ obj-$(CONFIG_PCIE_DW) += pcie-designware.o
 obj-$(CONFIG_PCIE_DW_HOST) += pcie-designware-host.o
 obj-$(CONFIG_PCIE_DW_EP) += pcie-designware-ep.o
 obj-$(CONFIG_PCIE_DW_PLAT) += pcie-designware-plat.o
+obj-$(CONFIG_PCIE_INTEL_GW) += pcie-intel-gw.o
 obj-$(CONFIG_PCI_DRA7XX) += pci-dra7xx.o
 obj-$(CONFIG_PCI_EXYNOS) += pci-exynos.o
 obj-$(CONFIG_PCI_IMX6) += pci-imx6.o
drivers/pci/controller/dwc/pcie-designware.c +34 -0
index 820488dfeaed..4c391bfd681a 100644
@@ -474,6 +474,40 @@ int dw_pcie_link_up(struct dw_pcie *pci)
 		(!(val & PCIE_PORT_DEBUG1_LINK_IN_TRAINING)));
 }
 
+
+void dw_pcie_upconfig_setup(struct dw_pcie *pci)
+{
+	u32 val;
+
+	val = dw_pcie_readl_dbi(pci, PCIE_PORT_MULTI_LANE_CTRL);
+	dw_pcie_writel_dbi(pci, PCIE_PORT_MULTI_LANE_CTRL,
+			   val | PORT_MLTI_UPCFG_SUPPORT);
+}
+
+void dw_pcie_link_speed_change(struct dw_pcie *pci, bool enable)
+{
+	u32 val;
+
+	val = dw_pcie_readl_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL);
+
+	if (enable)
+		val |= PORT_LOGIC_SPEED_CHANGE;
+	else
+		val &= ~PORT_LOGIC_SPEED_CHANGE;
+
+	dw_pcie_writel_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL, val);
+}
+
+void dw_pcie_link_set_n_fts(struct dw_pcie *pci, u32 n_fts)
+{
+	u32 val;
+
+	val = dw_pcie_readl_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL);
+	val &= ~PORT_LOGIC_N_FTS;
+	val |= n_fts;
+	dw_pcie_writel_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL, val);
+}
+
 static u8 dw_pcie_iatu_unroll_enabled(struct dw_pcie *pci)
 {
 	u32 val;
drivers/pci/controller/dwc/pcie-designware.h +12 -0
index 5a18e94e52c8..3beac10e4a4c 100644
@@ -30,7 +30,12 @@
 #define LINK_WAIT_IATU			9
 
 /* Synopsys-specific PCIe configuration registers */
+#define PCIE_PORT_AFR			0x70C
+#define PORT_AFR_N_FTS_MASK		GENMASK(15, 8)
+#define PORT_AFR_CC_N_FTS_MASK		GENMASK(23, 16)
+
 #define PCIE_PORT_LINK_CONTROL		0x710
+#define PORT_LINK_DLL_LINK_EN		BIT(5)
 #define PORT_LINK_MODE_MASK		GENMASK(21, 16)
 #define PORT_LINK_MODE(n)		FIELD_PREP(PORT_LINK_MODE_MASK, n)
 #define PORT_LINK_MODE_1_LANES		PORT_LINK_MODE(0x1)
@@ -46,6 +51,7 @@
 #define PCIE_PORT_DEBUG1_LINK_IN_TRAINING	BIT(29)
 
 #define PCIE_LINK_WIDTH_SPEED_CONTROL	0x80C
+#define PORT_LOGIC_N_FTS		GENMASK(7, 0)
 #define PORT_LOGIC_SPEED_CHANGE		BIT(17)
 #define PORT_LOGIC_LINK_WIDTH_MASK	GENMASK(12, 8)
 #define PORT_LOGIC_LINK_WIDTH(n)	FIELD_PREP(PORT_LOGIC_LINK_WIDTH_MASK, n)
@@ -60,6 +66,9 @@
 #define PCIE_MSI_INTR0_MASK		0x82C
 #define PCIE_MSI_INTR0_STATUS		0x830
 
+#define PCIE_PORT_MULTI_LANE_CTRL	0x8C0
+#define PORT_MLTI_UPCFG_SUPPORT		BIT(7)
+
 #define PCIE_ATU_VIEWPORT		0x900
 #define PCIE_ATU_REGION_INBOUND		BIT(31)
 #define PCIE_ATU_REGION_OUTBOUND	0
@@ -273,6 +282,9 @@ void dw_pcie_write_dbi2(struct dw_pcie *pci, u32 reg, size_t size, u32 val);
 u32 dw_pcie_read_atu(struct dw_pcie *pci, u32 reg, size_t size);
 void dw_pcie_write_atu(struct dw_pcie *pci, u32 reg, size_t size, u32 val);
 int dw_pcie_link_up(struct dw_pcie *pci);
+void dw_pcie_upconfig_setup(struct dw_pcie *pci);
+void dw_pcie_link_speed_change(struct dw_pcie *pci, bool enable);
+void dw_pcie_link_set_n_fts(struct dw_pcie *pci, u32 n_fts);
 int dw_pcie_wait_for_link(struct dw_pcie *pci);
 void dw_pcie_prog_outbound_atu(struct dw_pcie *pci, int index,
 			       int type, u64 cpu_addr, u64 pci_addr,
drivers/pci/controller/dwc/pcie-intel-gw.c new file +590 -0

590 lines not on this page · raw message

include/uapi/linux/pci_regs.h +1 -0
index 29d6e93fd15e..f6e7e402f879 100644
@@ -673,6 +673,7 @@
 #define  PCI_EXP_LNKCTL2_TLS_8_0GT	0x0003 /* Supported Speed 8GT/s */
 #define  PCI_EXP_LNKCTL2_TLS_16_0GT	0x0004 /* Supported Speed 16GT/s */
 #define  PCI_EXP_LNKCTL2_TLS_32_0GT	0x0005 /* Supported Speed 32GT/s */
+#define  PCI_EXP_LNKCTL2_HASD		0x0200 /* Hw Autonomous Speed Disable */
 #define PCI_EXP_LNKSTA2		50	/* Link Status 2 */
 #define PCI_CAP_EXP_ENDPOINT_SIZEOF_V2	52	/* v2 endpoints with link end here */
 #define PCI_EXP_SLTCAP2		52	/* Slot Capabilities 2 */
Signature · 1 line
2.11.0
#11 lore raw

RE: [PATCH v4 2/3] dwc: PCI: intel: PCIe RC controller driver

Hi

Quoted 8 lines · Dilip Kota

Add support to PCIe RC controller on Intel Gateway SoCs. PCIe controller is based of Synopsys DesignWare pci core.

Intel PCIe driver requires Upconfig support, fast training sequence configuration and link speed change. So adding the respective helper functions in the pcie DesignWare framework. It also programs hardware autonomous speed during speed configuration so defining it in pci_regs.h.

Please do the replacement in all of your patches

s/pcie/PCIe
s/pci/PCI

Also I think the correct term is Upconfigure and not Upconfig

Quoted 277 lines

Changes on v4:
Rename the driver naming and description to
"PCIe RC controller on Intel Gateway SoCs".
Use PCIe core register macros defined in pci_regs.h
and remove respective local definitions.
Remove pcie core interrupt handling.
Move pcie link control speed change, upconfig and FTS. configuration functions to DesignWare framework.
Use of_pci_get_max_link_speed().
Mark dependency on X86 and COMPILE_TEST in Kconfig.
Remove lanes and add n_fts variables in intel_pcie_port structure. Rename rst_interval variable to rst_intrvl in intel_pcie_port structure. Remove intel_pcie_mem_iatu() as it is already perfomed in dw_setup_rc() Move sysfs attributes specific code to separate patch. Remove redundant error handling.
Reduce LoCs by doing variable initializations while declaration itself. Add extra line after closing parenthesis.
Move intel_pcie_ep_rst_init() out of get_resources()

changes on v3:
Rename PCIe app logic registers with PCIE_APP prefix. PCIE_IOP_CTRL configuration is not required. Remove respective code. Remove wrapper functions for clk enable/disable APIs.
Use platform_get_resource_byname() instead of
devm_platform_ioremap_resource() to be similar with DWC framework.
Rename phy name to "pciephy".
Modify debug message in msi_init() callback to be more specific. Remove map_irq() callback.
Enable the INTx interrupts at the time of PCIe initialization. Reduce memory fragmentation by using variable "struct dw_pcie pci"
instead of allocating memory.
Reduce the delay to 100us during enpoint initialization
intel_pcie_ep_rst_init().
Call dw_pcie_host_deinit() during remove() instead of directly
calling PCIe core APIs.
Rename "intel,rst-interval" to "reset-assert-ms".
Remove unused APP logic Interrupt bit macro definitions. Use dwc framework's dw_pcie_setup_rc() for PCIe host specific
configuration instead of redefining the same functionality in the driver.
Move the whole DT parsing specific code to intel_pcie_get_resources()

---
 drivers/pci/controller/dwc/Kconfig           |  10 +
 drivers/pci/controller/dwc/Makefile          |   1 +
 drivers/pci/controller/dwc/pcie-designware.c |  34 ++
 drivers/pci/controller/dwc/pcie-designware.h |  12 +
 drivers/pci/controller/dwc/pcie-intel-gw.c   | 590 +++++++++++++++++++++++++++
 include/uapi/linux/pci_regs.h                |   1 +
 6 files changed, 648 insertions(+)
 create mode 100644 drivers/pci/controller/dwc/pcie-intel-gw.c
drivers/pci/controller/dwc/Kconfig +10 -0
index 0ba988b5b5bc..b33ed1cc873d 100644
@@ -82,6 +82,16 @@ config PCIE_DW_PLAT_EP
 	  order to enable device-specific features PCI_DW_PLAT_EP must be
 	  selected.
 
+config PCIE_INTEL_GW
+        bool "Intel Gateway PCIe host controller support"
+	depends on OF && (X86 || COMPILE_TEST)
+	select PCIE_DW_HOST
+	help
+          Say 'Y' here to enable support for PCIe Host controller driver.
+	  The PCIe controller on Intel Gateway SoCs is based on the Synopsys
+	  DesignWare pcie core and therefore uses the DesignWare core
+	  functions for the driver implementation.
+
 config PCI_EXYNOS
 	bool "Samsung Exynos PCIe controller"
 	depends on SOC_EXYNOS5440 || COMPILE_TEST
drivers/pci/controller/dwc/Makefile +1 -0
index b30336181d46..99db83cd2f35 100644
@@ -3,6 +3,7 @@ obj-$(CONFIG_PCIE_DW) += pcie-designware.o
 obj-$(CONFIG_PCIE_DW_HOST) += pcie-designware-host.o
 obj-$(CONFIG_PCIE_DW_EP) += pcie-designware-ep.o
 obj-$(CONFIG_PCIE_DW_PLAT) += pcie-designware-plat.o
+obj-$(CONFIG_PCIE_INTEL_GW) += pcie-intel-gw.o
 obj-$(CONFIG_PCI_DRA7XX) += pci-dra7xx.o
 obj-$(CONFIG_PCI_EXYNOS) += pci-exynos.o
 obj-$(CONFIG_PCI_IMX6) += pci-imx6.o
drivers/pci/controller/dwc/pcie-designware.c +34 -0
index 820488dfeaed..4c391bfd681a 100644
@@ -474,6 +474,40 @@ int dw_pcie_link_up(struct dw_pcie *pci)
 		(!(val & PCIE_PORT_DEBUG1_LINK_IN_TRAINING)));
 }
 
+
+void dw_pcie_upconfig_setup(struct dw_pcie *pci)
+{
+	u32 val;
+
+	val = dw_pcie_readl_dbi(pci, PCIE_PORT_MULTI_LANE_CTRL);
+	dw_pcie_writel_dbi(pci, PCIE_PORT_MULTI_LANE_CTRL,
+			   val | PORT_MLTI_UPCFG_SUPPORT);
+}
+
+void dw_pcie_link_speed_change(struct dw_pcie *pci, bool enable)
+{
+	u32 val;
+
+	val = dw_pcie_readl_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL);
+
+	if (enable)
+		val |= PORT_LOGIC_SPEED_CHANGE;
+	else
+		val &= ~PORT_LOGIC_SPEED_CHANGE;
+
+	dw_pcie_writel_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL, val);
+}
+
+void dw_pcie_link_set_n_fts(struct dw_pcie *pci, u32 n_fts)
+{
+	u32 val;
+
+	val = dw_pcie_readl_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL);
+	val &= ~PORT_LOGIC_N_FTS;
+	val |= n_fts;
+	dw_pcie_writel_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL, val);
+}
+
 static u8 dw_pcie_iatu_unroll_enabled(struct dw_pcie *pci)
 {
 	u32 val;
drivers/pci/controller/dwc/pcie-designware.h +12 -0
index 5a18e94e52c8..3beac10e4a4c 100644
@@ -30,7 +30,12 @@
 #define LINK_WAIT_IATU			9
 
 /* Synopsys-specific PCIe configuration registers */
+#define PCIE_PORT_AFR			0x70C
+#define PORT_AFR_N_FTS_MASK		GENMASK(15, 8)
+#define PORT_AFR_CC_N_FTS_MASK		GENMASK(23, 16)
+
 #define PCIE_PORT_LINK_CONTROL		0x710
+#define PORT_LINK_DLL_LINK_EN		BIT(5)
 #define PORT_LINK_MODE_MASK		GENMASK(21, 16)
 #define PORT_LINK_MODE(n)		FIELD_PREP(PORT_LINK_MODE_MASK, n)
 #define PORT_LINK_MODE_1_LANES		PORT_LINK_MODE(0x1)
@@ -46,6 +51,7 @@
 #define PCIE_PORT_DEBUG1_LINK_IN_TRAINING	BIT(29)
 
 #define PCIE_LINK_WIDTH_SPEED_CONTROL	0x80C
+#define PORT_LOGIC_N_FTS		GENMASK(7, 0)
 #define PORT_LOGIC_SPEED_CHANGE		BIT(17)
 #define PORT_LOGIC_LINK_WIDTH_MASK	GENMASK(12, 8)
 #define PORT_LOGIC_LINK_WIDTH(n)	FIELD_PREP(PORT_LOGIC_LINK_WIDTH_MASK, n)
@@ -60,6 +66,9 @@
 #define PCIE_MSI_INTR0_MASK		0x82C
 #define PCIE_MSI_INTR0_STATUS		0x830
 
+#define PCIE_PORT_MULTI_LANE_CTRL	0x8C0
+#define PORT_MLTI_UPCFG_SUPPORT		BIT(7)
+
 #define PCIE_ATU_VIEWPORT		0x900
 #define PCIE_ATU_REGION_INBOUND		BIT(31)
 #define PCIE_ATU_REGION_OUTBOUND	0
@@ -273,6 +282,9 @@ void dw_pcie_write_dbi2(struct dw_pcie *pci, u32 reg, size_t size, u32 val);
 u32 dw_pcie_read_atu(struct dw_pcie *pci, u32 reg, size_t size);
 void dw_pcie_write_atu(struct dw_pcie *pci, u32 reg, size_t size, u32 val);
 int dw_pcie_link_up(struct dw_pcie *pci);
+void dw_pcie_upconfig_setup(struct dw_pcie *pci);
+void dw_pcie_link_speed_change(struct dw_pcie *pci, bool enable);
+void dw_pcie_link_set_n_fts(struct dw_pcie *pci, u32 n_fts);
 int dw_pcie_wait_for_link(struct dw_pcie *pci);
 void dw_pcie_prog_outbound_atu(struct dw_pcie *pci, int index,
 			       int type, u64 cpu_addr, u64 pci_addr,
drivers/pci/controller/dwc/pcie-intel-gw.c new file +94 -0
new file mode 100644
index 000000000000..9142c70db808
@@ -0,0 +1,590 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * PCIe host controller driver for Intel Gateway SoCs
+ *
+ * Copyright (c) 2019 Intel Corporation.
+ */
+
+#include <linux/bitfield.h>
+#include <linux/clk.h>
+#include <linux/gpio/consumer.h>
+#include <linux/interrupt.h>
+#include <linux/iopoll.h>
+#include <linux/of_irq.h>
+#include <linux/of_pci.h>
+#include <linux/of_platform.h>
+#include <linux/pci_regs.h>
+#include <linux/phy/phy.h>
+#include <linux/platform_device.h>
+#include <linux/reset.h>
+
+#include "../../pci.h"
+#include "pcie-designware.h"
+
+#define PCIE_CAP_OFST			0x70
+
+#define PORT_AFR_N_FTS_GEN12_DFT	(SZ_128 - 1)
+#define PORT_AFR_N_FTS_GEN3		180
+#define PORT_AFR_N_FTS_GEN4		196
+
+/* PCIe Application logic Registers */
+#define PCIE_APP_CCR			0x10
+#define PCIE_APP_CCR_LTSSM_ENABLE	BIT(0)
+
+#define PCIE_APP_MSG_CR			0x30
+#define PCIE_APP_MSG_XMT_PM_TURNOFF	BIT(0)
+
+#define PCIE_APP_PMC			0x44
+#define PCIE_APP_PMC_IN_L2		BIT(20)
+
+#define PCIE_APP_IRNEN			0xF4
+#define PCIE_APP_IRNCR			0xF8
+#define PCIE_APP_IRN_AER_REPORT		BIT(0)
+#define PCIE_APP_IRN_PME		BIT(2)
+#define PCIE_APP_IRN_RX_VDM_MSG		BIT(4)
+#define PCIE_APP_IRN_PM_TO_ACK		BIT(9)
+#define PCIE_APP_IRN_LINK_AUTO_BW_STAT	BIT(11)
+#define PCIE_APP_IRN_BW_MGT		BIT(12)
+#define PCIE_APP_IRN_MSG_LTR		BIT(18)
+#define PCIE_APP_IRN_SYS_ERR_RC		BIT(29)
+#define PCIE_APP_INTX_OFST		12
+
+#define PCIE_APP_IRN_INT \
+			(PCIE_APP_IRN_AER_REPORT | PCIE_APP_IRN_PME | \
+			PCIE_APP_IRN_RX_VDM_MSG | PCIE_APP_IRN_SYS_ERR_RC | \
+			PCIE_APP_IRN_PM_TO_ACK | PCIE_APP_IRN_MSG_LTR | \
+			PCIE_APP_IRN_BW_MGT | PCIE_APP_IRN_LINK_AUTO_BW_STAT | \
+			(PCIE_APP_INTX_OFST + PCI_INTERRUPT_INTA) | \
+			(PCIE_APP_INTX_OFST + PCI_INTERRUPT_INTB) | \
+			(PCIE_APP_INTX_OFST + PCI_INTERRUPT_INTC) | \
+			(PCIE_APP_INTX_OFST + PCI_INTERRUPT_INTD))
+
+#define BUS_IATU_OFFS			SZ_256M
+#define RST_INTRVL_DFT_MS		100
+
+enum {
+	PCIE_LINK_SPEED_AUTO = 0,
+	PCIE_LINK_SPEED_GEN1,
+	PCIE_LINK_SPEED_GEN2,
+	PCIE_LINK_SPEED_GEN3,
+	PCIE_LINK_SPEED_GEN4,
+};
+
+struct intel_pcie_soc {
+	unsigned int pcie_ver;
+	unsigned int pcie_atu_offset;
+	u32 num_viewport;
+};
+
+struct intel_pcie_port {
+	struct dw_pcie		pci;
+	unsigned int		id; /* Physical RC Index */
+	void __iomem		*app_base;
+	struct gpio_desc	*reset_gpio;
+	u32			rst_intrvl;
+	u32			max_speed;
+	u32			link_gen;
+	u32			max_width;
+	u32			n_fts;
+	struct clk		*core_clk;
+	struct reset_control	*core_rst;
+	struct phy		*phy;
+};
+
+static void pcie_update_bits(void __iomem *base, u32 mask, u32 val, u32 ofs)

drivers/pci/controller/dwc/pcie-intel-gw.c line 104

Quoted 10 lines
drivers/pci/controller/dwc/pcie-intel-gw.c new file +10 -0
+{
+	u32 orig, tmp;
+
+	orig = readl(base + ofs);
+
+	tmp = (orig & ~mask) | (val & mask);
+
+	if (tmp != orig)
+		writel(tmp, base + ofs);
+}

I'd suggest to the a take on FIELD_PREP() and FIELD_GET() and use more intuitive names such as new and old, instead of orig and tmp.

Quoted 89 lines
drivers/pci/controller/dwc/pcie-intel-gw.c new file +89 -0
+static inline u32 pcie_app_rd(struct intel_pcie_port *lpp, u32 ofs)
+{
+	return readl(lpp->app_base + ofs);
+}
+
+static inline void pcie_app_wr(struct intel_pcie_port *lpp, u32 val, u32 ofs)
+{
+	writel(val, lpp->app_base + ofs);
+}
+
+static void pcie_app_wr_mask(struct intel_pcie_port *lpp,
+			     u32 mask, u32 val, u32 ofs)
+{
+	pcie_update_bits(lpp->app_base, mask, val, ofs);
+}
+
+static inline u32 pcie_rc_cfg_rd(struct intel_pcie_port *lpp, u32 ofs)
+{
+	return dw_pcie_readl_dbi(&lpp->pci, ofs);
+}
+
+static inline void pcie_rc_cfg_wr(struct intel_pcie_port *lpp, u32 val, u32 ofs)
+{
+	dw_pcie_writel_dbi(&lpp->pci, ofs, val);
+}
+
+static void pcie_rc_cfg_wr_mask(struct intel_pcie_port *lpp,
+				u32 mask, u32 val, u32 ofs)
+{
+	pcie_update_bits(lpp->pci.dbi_base, mask, val, ofs);
+}
+
+static void intel_pcie_ltssm_enable(struct intel_pcie_port *lpp)
+{
+	pcie_app_wr_mask(lpp, PCIE_APP_CCR_LTSSM_ENABLE,
+			 PCIE_APP_CCR_LTSSM_ENABLE, PCIE_APP_CCR);
+}
+
+static void intel_pcie_ltssm_disable(struct intel_pcie_port *lpp)
+{
+	pcie_app_wr_mask(lpp, PCIE_APP_CCR_LTSSM_ENABLE, 0, PCIE_APP_CCR);
+}
+
+static void intel_pcie_link_setup(struct intel_pcie_port *lpp)
+{
+	u32 val;
+
+	val = pcie_rc_cfg_rd(lpp, PCIE_CAP_OFST + PCI_EXP_LNKCAP);
+	lpp->max_speed = FIELD_GET(PCI_EXP_LNKCAP_SLS, val);
+	lpp->max_width = FIELD_GET(PCI_EXP_LNKCAP_MLW, val);
+
+	val = pcie_rc_cfg_rd(lpp, PCIE_CAP_OFST + PCI_EXP_LNKCTL);
+
+	val &= ~(PCI_EXP_LNKCTL_LD | PCI_EXP_LNKCTL_ASPMC);
+	val |= (PCI_EXP_LNKSTA_SLC << 16) | PCI_EXP_LNKCTL_CCC |
+	       PCI_EXP_LNKCTL_RCB;
+	pcie_rc_cfg_wr(lpp, val, PCIE_CAP_OFST + PCI_EXP_LNKCTL);
+}
+
+static void intel_pcie_max_speed_setup(struct intel_pcie_port *lpp)
+{
+	u32 reg, val;
+
+	reg = pcie_rc_cfg_rd(lpp, PCIE_CAP_OFST + PCI_EXP_LNKCTL2);
+	switch (lpp->link_gen) {
+	case PCIE_LINK_SPEED_GEN1:
+		reg &= ~PCI_EXP_LNKCTL2_TLS;
+		reg |= PCI_EXP_LNKCTL2_HASD|
+			PCI_EXP_LNKCTL2_TLS_2_5GT;
+		break;
+	case PCIE_LINK_SPEED_GEN2:
+		reg &= ~PCI_EXP_LNKCTL2_TLS;
+		reg |= PCI_EXP_LNKCTL2_HASD|
+			PCI_EXP_LNKCTL2_TLS_5_0GT;
+		break;
+	case PCIE_LINK_SPEED_GEN3:
+		reg &= ~PCI_EXP_LNKCTL2_TLS;
+		reg |= PCI_EXP_LNKCTL2_HASD|
+			PCI_EXP_LNKCTL2_TLS_8_0GT;
+		break;
+	case PCIE_LINK_SPEED_GEN4:
+		reg &= ~PCI_EXP_LNKCTL2_TLS;
+		reg |= PCI_EXP_LNKCTL2_HASD|
+			PCI_EXP_LNKCTL2_TLS_16_0GT;
+		break;
+	default:
+		/* Use hardware capability */
+		val = pcie_rc_cfg_rd(lpp, PCIE_CAP_OFST + PCI_EXP_LNKCAP);
+		val = FIELD_GET(PCI_EXP_LNKCAP_SLS, val);

drivers/pci/controller/dwc/pcie-intel-gw.c line 204

Quoted 10 lines
drivers/pci/controller/dwc/pcie-intel-gw.c new file +10 -0
+		reg &= ~PCI_EXP_LNKCTL2_HASD;
+		reg |= val;
+		break;
+	}
+
+	pcie_rc_cfg_wr(lpp, reg, PCIE_CAP_OFST + PCI_EXP_LNKCTL2);
+	dw_pcie_link_set_n_fts(&lpp->pci, lpp->n_fts);
+}
+
+

Reduce the number of empty lines here.

Quoted 391 lines
drivers/pci/controller/dwc/pcie-intel-gw.c new file +386 -0
+
+static void intel_pcie_port_logic_setup(struct intel_pcie_port *lpp)
+{
+	u32 val, mask;
+
+	switch (lpp->max_speed) {
+	case PCIE_LINK_SPEED_GEN3:
+		lpp->n_fts = PORT_AFR_N_FTS_GEN3;
+		break;
+	case PCIE_LINK_SPEED_GEN4:
+		lpp->n_fts = PORT_AFR_N_FTS_GEN4;
+		break;
+	default:
+		lpp->n_fts = PORT_AFR_N_FTS_GEN12_DFT;
+		break;
+	}
+
+	mask = PORT_AFR_N_FTS_MASK | PORT_AFR_CC_N_FTS_MASK;
+	val = FIELD_PREP(PORT_AFR_N_FTS_MASK, lpp->n_fts) |
+	       FIELD_PREP(PORT_AFR_CC_N_FTS_MASK, lpp->n_fts);
+	pcie_rc_cfg_wr_mask(lpp, mask, val, PCIE_PORT_AFR);
+
+	/* Port Link Control Register */
+	pcie_rc_cfg_wr_mask(lpp, PORT_LINK_DLL_LINK_EN,
+			    PORT_LINK_DLL_LINK_EN, PCIE_PORT_LINK_CONTROL);
+}
+
+static void intel_pcie_rc_setup(struct intel_pcie_port *lpp)
+{
+	intel_pcie_ltssm_disable(lpp);
+	intel_pcie_link_setup(lpp);
+	dw_pcie_setup_rc(&lpp->pci.pp);
+	dw_pcie_upconfig_setup(&lpp->pci);
+	intel_pcie_port_logic_setup(lpp);
+	intel_pcie_max_speed_setup(lpp);
+}
+
+static int intel_pcie_ep_rst_init(struct intel_pcie_port *lpp)
+{
+	struct device *dev = lpp->pci.dev;
+	int ret;
+
+	lpp->reset_gpio = devm_gpiod_get(dev, "reset", GPIOD_OUT_LOW);
+	if (IS_ERR(lpp->reset_gpio)) {
+		ret = PTR_ERR(lpp->reset_gpio);
+		if (ret != -EPROBE_DEFER)
+			dev_err(dev, "failed to request PCIe GPIO: %d\n", ret);
+		return ret;
+	}
+
+	/* Make initial reset last for 100us */
+	usleep_range(100, 200);
+
+	return 0;
+}
+
+static void intel_pcie_core_rst_assert(struct intel_pcie_port *lpp)
+{
+	reset_control_assert(lpp->core_rst);
+}
+
+static void intel_pcie_core_rst_deassert(struct intel_pcie_port *lpp)
+{
+	/*
+	 * One micro-second delay to make sure the reset pulse
+	 * wide enough so that core reset is clean.
+	 */
+	udelay(1);
+	reset_control_deassert(lpp->core_rst);
+
+	/*
+	 * Some SoC core reset also reset PHY, more delay needed
+	 * to make sure the reset process is done.
+	 */
+	usleep_range(1000, 2000);
+}
+
+static void intel_pcie_device_rst_assert(struct intel_pcie_port *lpp)
+{
+	gpiod_set_value_cansleep(lpp->reset_gpio, 1);
+}
+
+static void intel_pcie_device_rst_deassert(struct intel_pcie_port *lpp)
+{
+	msleep(lpp->rst_intrvl);
+	gpiod_set_value_cansleep(lpp->reset_gpio, 0);
+}
+
+static int intel_pcie_app_logic_setup(struct intel_pcie_port *lpp)
+{
+	intel_pcie_device_rst_deassert(lpp);
+	intel_pcie_ltssm_enable(lpp);
+
+	return dw_pcie_wait_for_link(&lpp->pci);
+}
+
+static void intel_pcie_core_irq_disable(struct intel_pcie_port *lpp)
+{
+	pcie_app_wr(lpp, 0, PCIE_APP_IRNEN);
+	pcie_app_wr(lpp, PCIE_APP_IRN_INT,  PCIE_APP_IRNCR);
+}
+
+static int intel_pcie_get_resources(struct platform_device *pdev)
+{
+	struct intel_pcie_port *lpp = platform_get_drvdata(pdev);
+	struct dw_pcie *pci = &lpp->pci;
+	struct device *dev = pci->dev;
+	struct resource *res;
+	int ret;
+
+	ret = device_property_read_u32(dev, "linux,pci-domain", &lpp->id);
+	if (ret) {
+		dev_err(dev, "failed to get domain id, errno %d\n", ret);
+		return ret;
+	}
+
+	res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "dbi");
+
+	pci->dbi_base = devm_ioremap_resource(dev, res);
+	if (IS_ERR(pci->dbi_base))
+		return PTR_ERR(pci->dbi_base);
+
+	lpp->core_clk = devm_clk_get(dev, NULL);
+	if (IS_ERR(lpp->core_clk)) {
+		ret = PTR_ERR(lpp->core_clk);
+		if (ret != -EPROBE_DEFER)
+			dev_err(dev, "failed to get clks: %d\n", ret);
+		return ret;
+	}
+
+	lpp->core_rst = devm_reset_control_get(dev, NULL);
+	if (IS_ERR(lpp->core_rst)) {
+		ret = PTR_ERR(lpp->core_rst);
+		if (ret != -EPROBE_DEFER)
+			dev_err(dev, "failed to get resets: %d\n", ret);
+		return ret;
+	}
+
+	ret = device_property_match_string(dev, "device_type", "pci");
+	if (ret) {
+		dev_err(dev, "failed to find pci device type: %d\n", ret);
+		return ret;
+	}
+
+	if (device_property_read_u32(dev, "reset-assert-ms", &lpp->rst_intrvl))
+		lpp->rst_intrvl = RST_INTRVL_DFT_MS;
+
+	ret = of_pci_get_max_link_speed(dev->of_node);
+		lpp->link_gen = ret < 0 ? 0 : ret;
+
+	res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "app");
+
+	lpp->app_base = devm_ioremap_resource(dev, res);
+	if (IS_ERR(lpp->app_base))
+		return PTR_ERR(lpp->app_base);
+
+	lpp->phy = devm_phy_get(dev, "pcie");
+	if (IS_ERR(lpp->phy)) {
+		ret = PTR_ERR(lpp->phy);
+		if (ret != -EPROBE_DEFER)
+			dev_err(dev, "couldn't get pcie-phy: %d\n", ret);
+		return ret;
+	}
+
+	return 0;
+}
+
+static void intel_pcie_deinit_phy(struct intel_pcie_port *lpp)
+{
+	phy_exit(lpp->phy);
+}
+
+static int intel_pcie_wait_l2(struct intel_pcie_port *lpp)
+{
+	u32 value;
+	int ret;
+
+	if (lpp->max_speed < PCIE_LINK_SPEED_GEN3)
+		return 0;
+
+	/* Send PME_TURN_OFF message */
+	pcie_app_wr_mask(lpp, PCIE_APP_MSG_XMT_PM_TURNOFF,
+			 PCIE_APP_MSG_XMT_PM_TURNOFF, PCIE_APP_MSG_CR);
+
+	/* Read PMC status and wait for falling into L2 link state */
+	ret = readl_poll_timeout(lpp->app_base + PCIE_APP_PMC, value,
+				 (value & PCIE_APP_PMC_IN_L2), 20,
+				 jiffies_to_usecs(5 * HZ));
+	if (ret)
+		dev_err(lpp->pci.dev, "PCIe link enter L2 timeout!\n");
+
+	return ret;
+}
+
+static void intel_pcie_turn_off(struct intel_pcie_port *lpp)
+{
+	if (dw_pcie_link_up(&lpp->pci))
+		intel_pcie_wait_l2(lpp);
+
+	/* Put endpoint device in reset state */
+	intel_pcie_device_rst_assert(lpp);
+	pcie_rc_cfg_wr_mask(lpp, PCI_COMMAND_MEMORY, 0, PCI_COMMAND);
+}
+
+static int intel_pcie_host_setup(struct intel_pcie_port *lpp)
+{
+	int ret;
+
+	intel_pcie_core_rst_assert(lpp);
+	intel_pcie_device_rst_assert(lpp);
+
+	ret = phy_init(lpp->phy);
+	if (ret)
+		return ret;
+
+	intel_pcie_core_rst_deassert(lpp);
+
+	ret = clk_prepare_enable(lpp->core_clk);
+	if (ret) {
+		dev_err(lpp->pci.dev, "Core clock enable failed: %d\n", ret);
+		goto clk_err;
+	}
+
+	intel_pcie_rc_setup(lpp);
+	ret = intel_pcie_app_logic_setup(lpp);
+	if (ret)
+		goto app_init_err;
+
+	/* Enable integrated interrupts */
+	pcie_app_wr_mask(lpp, PCIE_APP_IRN_INT, PCIE_APP_IRN_INT,
+			 PCIE_APP_IRNEN);
+	return 0;
+
+app_init_err:
+	clk_disable_unprepare(lpp->core_clk);
+clk_err:
+	intel_pcie_core_rst_assert(lpp);
+	intel_pcie_deinit_phy(lpp);
+
+	return ret;
+}
+
+static void __intel_pcie_remove(struct intel_pcie_port *lpp)
+{
+	intel_pcie_core_irq_disable(lpp);
+	intel_pcie_turn_off(lpp);
+	clk_disable_unprepare(lpp->core_clk);
+	intel_pcie_core_rst_assert(lpp);
+	intel_pcie_deinit_phy(lpp);
+}
+
+static int intel_pcie_remove(struct platform_device *pdev)
+{
+	struct intel_pcie_port *lpp = platform_get_drvdata(pdev);
+	struct pcie_port *pp = &lpp->pci.pp;
+
+	dw_pcie_host_deinit(pp);
+	__intel_pcie_remove(lpp);
+
+	return 0;
+}
+
+static int __maybe_unused intel_pcie_suspend_noirq(struct device *dev)
+{
+	struct intel_pcie_port *lpp = dev_get_drvdata(dev);
+	int ret;
+
+	intel_pcie_core_irq_disable(lpp);
+	ret = intel_pcie_wait_l2(lpp);
+	if (ret)
+		return ret;
+
+	intel_pcie_deinit_phy(lpp);
+	clk_disable_unprepare(lpp->core_clk);
+	return ret;
+}
+
+static int __maybe_unused intel_pcie_resume_noirq(struct device *dev)
+{
+	struct intel_pcie_port *lpp = dev_get_drvdata(dev);
+
+	return intel_pcie_host_setup(lpp);
+}
+
+static int intel_pcie_rc_init(struct pcie_port *pp)
+{
+	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
+	struct intel_pcie_port *lpp = dev_get_drvdata(pci->dev);
+
+	return intel_pcie_host_setup(lpp);
+}
+
+int intel_pcie_msi_init(struct pcie_port *pp)
+{
+	/* PCIe MSI/MSIx is handled by MSI in x86 processor */
+	return 0;
+}
+
+u64 intel_pcie_cpu_addr(struct dw_pcie *pcie, u64 cpu_addr)
+{
+	return cpu_addr + BUS_IATU_OFFS;
+}
+
+static const struct dw_pcie_ops intel_pcie_ops = {
+	.cpu_addr_fixup = intel_pcie_cpu_addr,
+};
+
+static const struct dw_pcie_host_ops intel_pcie_dw_ops = {
+	.host_init =		intel_pcie_rc_init,
+	.msi_host_init =	intel_pcie_msi_init,
+};
+
+static const struct intel_pcie_soc pcie_data = {
+	.pcie_ver =		0x520A,
+	.pcie_atu_offset =	0xC0000,
+	.num_viewport =		3,
+};
+
+static int intel_pcie_probe(struct platform_device *pdev)
+{
+	const struct intel_pcie_soc *data;
+	struct device *dev = &pdev->dev;
+	struct intel_pcie_port *lpp;
+	struct pcie_port *pp;
+	struct dw_pcie *pci;
+	int ret;
+
+	lpp = devm_kzalloc(dev, sizeof(*lpp), GFP_KERNEL);
+	if (!lpp)
+		return -ENOMEM;
+
+	platform_set_drvdata(pdev, lpp);
+	pci = &lpp->pci;
+	pci->dev = dev;
+	pp = &pci->pp;
+
+	ret = intel_pcie_get_resources(pdev);
+	if (ret)
+		return ret;
+
+	ret = intel_pcie_ep_rst_init(lpp);
+	if (ret)
+		return ret;
+
+	data = device_get_match_data(dev);
+	pci->ops = &intel_pcie_ops;
+	pci->version = data->pcie_ver;
+	pci->atu_base = pci->dbi_base + data->pcie_atu_offset;
+	pp->ops = &intel_pcie_dw_ops;
+
+	ret = dw_pcie_host_init(pp);
+	if (ret) {
+		dev_err(dev, "cannot initialize host\n");
+		return ret;
+	}
+
+	/* Intel PCIe doesn't configure IO region, so configure
+	 * viewport to not to access IO region during register
+	 * read write operations.
+	 */
+	pci->num_viewport = data->num_viewport;
+	dev_info(dev, "Intel PCIe Root Complex Port %d init done\n", lpp->id);
+
+	return ret;
+}
+
+static const struct dev_pm_ops intel_pcie_pm_ops = {
+	SET_NOIRQ_SYSTEM_SLEEP_PM_OPS(intel_pcie_suspend_noirq,
+				      intel_pcie_resume_noirq)
+};
+
+static const struct of_device_id of_intel_pcie_match[] = {
+	{ .compatible = "intel,lgm-pcie", .data = &pcie_data },
+	{}
+};
+
+static struct platform_driver intel_pcie_driver = {
+	.probe = intel_pcie_probe,
+	.remove = intel_pcie_remove,
+	.driver = {
+		.name = "intel-gw-pcie",
+		.of_match_table = of_intel_pcie_match,
+		.pm = &intel_pcie_pm_ops,
+	},
+};
+builtin_platform_driver(intel_pcie_driver);
include/uapi/linux/pci_regs.h +0 -0
index 29d6e93fd15e..f6e7e402f879 100644

include/uapi/linux/pci_regs.h line 676

Quoted 4 lines
include/uapi/linux/pci_regs.h +1 -0
@@ -673,6 +673,7 @@
 #define  PCI_EXP_LNKCTL2_TLS_8_0GT	0x0003 /* Supported Speed 8GT/s */
 #define  PCI_EXP_LNKCTL2_TLS_16_0GT	0x0004 /* Supported Speed 16GT/s */
 #define  PCI_EXP_LNKCTL2_TLS_32_0GT	0x0005 /* Supported Speed 32GT/s */
+#define  PCI_EXP_LNKCTL2_HASD		0x0200 /* Hw Autonomous Speed Disable */

s/Hw/HW

Quoted 5 lines
include/uapi/linux/pci_regs.h +0 -0
 #define PCI_EXP_LNKSTA2		50	/* Link Status 2 */
 #define PCI_CAP_EXP_ENDPOINT_SIZEOF_V2	52	/* v2 endpoints with link end here */
 #define PCI_EXP_SLTCAP2		52	/* Slot Capabilities 2 */
Signature · 1 line
2.11.0
author
#12 lore raw

Re: [PATCH v4 2/3] dwc: PCI: intel: PCIe RC controller driver

Hi Gustavo Pimentel,

Quoted 13 lines · Gustavo Pimentel

Hi

Quoted 8 lines · Dilip Kota

Add support to PCIe RC controller on Intel Gateway SoCs. PCIe controller is based of Synopsys DesignWare pci core.

Intel PCIe driver requires Upconfig support, fast training sequence configuration and link speed change. So adding the respective helper functions in the pcie DesignWare framework. It also programs hardware autonomous speed during speed configuration so defining it in pci_regs.h.

Quoted 6 lines

Please do the replacement in all of your patches

s/pcie/PCIe
s/pci/PCI

Also I think the correct term is Upconfigure and not Upconfig

Yes, i will update it.

Quoted 281 lines
Quoted 281 lines

Changes on v4:
Rename the driver naming and description to
"PCIe RC controller on Intel Gateway SoCs".
Use PCIe core register macros defined in pci_regs.h
and remove respective local definitions.
Remove pcie core interrupt handling.
Move pcie link control speed change, upconfig and FTS. configuration functions to DesignWare framework.
Use of_pci_get_max_link_speed().
Mark dependency on X86 and COMPILE_TEST in Kconfig.
Remove lanes and add n_fts variables in intel_pcie_port structure. Rename rst_interval variable to rst_intrvl in intel_pcie_port structure. Remove intel_pcie_mem_iatu() as it is already perfomed in dw_setup_rc() Move sysfs attributes specific code to separate patch. Remove redundant error handling.
Reduce LoCs by doing variable initializations while declaration itself. Add extra line after closing parenthesis.
Move intel_pcie_ep_rst_init() out of get_resources()

changes on v3:
Rename PCIe app logic registers with PCIE_APP prefix. PCIE_IOP_CTRL configuration is not required. Remove respective code. Remove wrapper functions for clk enable/disable APIs.
Use platform_get_resource_byname() instead of
devm_platform_ioremap_resource() to be similar with DWC framework.
Rename phy name to "pciephy".
Modify debug message in msi_init() callback to be more specific. Remove map_irq() callback.
Enable the INTx interrupts at the time of PCIe initialization. Reduce memory fragmentation by using variable "struct dw_pcie pci"
instead of allocating memory.
Reduce the delay to 100us during enpoint initialization
intel_pcie_ep_rst_init().
Call dw_pcie_host_deinit() during remove() instead of directly
calling PCIe core APIs.
Rename "intel,rst-interval" to "reset-assert-ms".
Remove unused APP logic Interrupt bit macro definitions. Use dwc framework's dw_pcie_setup_rc() for PCIe host specific
configuration instead of redefining the same functionality in the driver.
Move the whole DT parsing specific code to intel_pcie_get_resources()

---
  drivers/pci/controller/dwc/Kconfig           |  10 +
  drivers/pci/controller/dwc/Makefile          |   1 +
  drivers/pci/controller/dwc/pcie-designware.c |  34 ++
  drivers/pci/controller/dwc/pcie-designware.h |  12 +
  drivers/pci/controller/dwc/pcie-intel-gw.c   | 590 +++++++++++++++++++++++++++
  include/uapi/linux/pci_regs.h                |   1 +
  6 files changed, 648 insertions(+)

create mode 100644 drivers/pci/controller/dwc/pcie-intel-gw.c

drivers/pci/controller/dwc/Kconfig +10 -0
index 0ba988b5b5bc..b33ed1cc873d 100644
@@ -82,6 +82,16 @@ config PCIE_DW_PLAT_EP
  	  order to enable device-specific features PCI_DW_PLAT_EP must be
  	  selected.
 
+config PCIE_INTEL_GW
+        bool "Intel Gateway PCIe host controller support"
+	depends on OF && (X86 || COMPILE_TEST)
+	select PCIE_DW_HOST
+	help
+          Say 'Y' here to enable support for PCIe Host controller driver.
+	  The PCIe controller on Intel Gateway SoCs is based on the Synopsys
+	  DesignWare pcie core and therefore uses the DesignWare core
+	  functions for the driver implementation.
+
  config PCI_EXYNOS
  	bool "Samsung Exynos PCIe controller"
  	depends on SOC_EXYNOS5440 || COMPILE_TEST
drivers/pci/controller/dwc/Makefile +1 -0
index b30336181d46..99db83cd2f35 100644
@@ -3,6 +3,7 @@ obj-$(CONFIG_PCIE_DW) += pcie-designware.o
  obj-$(CONFIG_PCIE_DW_HOST) += pcie-designware-host.o
  obj-$(CONFIG_PCIE_DW_EP) += pcie-designware-ep.o
  obj-$(CONFIG_PCIE_DW_PLAT) += pcie-designware-plat.o
+obj-$(CONFIG_PCIE_INTEL_GW) += pcie-intel-gw.o
  obj-$(CONFIG_PCI_DRA7XX) += pci-dra7xx.o
  obj-$(CONFIG_PCI_EXYNOS) += pci-exynos.o
  obj-$(CONFIG_PCI_IMX6) += pci-imx6.o
drivers/pci/controller/dwc/pcie-designware.c +34 -0
index 820488dfeaed..4c391bfd681a 100644
@@ -474,6 +474,40 @@ int dw_pcie_link_up(struct dw_pcie *pci)
  		(!(val & PCIE_PORT_DEBUG1_LINK_IN_TRAINING)));
  }
 
+
+void dw_pcie_upconfig_setup(struct dw_pcie *pci)
+{
+	u32 val;
+
+	val = dw_pcie_readl_dbi(pci, PCIE_PORT_MULTI_LANE_CTRL);
+	dw_pcie_writel_dbi(pci, PCIE_PORT_MULTI_LANE_CTRL,
+			   val | PORT_MLTI_UPCFG_SUPPORT);
+}
+
+void dw_pcie_link_speed_change(struct dw_pcie *pci, bool enable)
+{
+	u32 val;
+
+	val = dw_pcie_readl_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL);
+
+	if (enable)
+		val |= PORT_LOGIC_SPEED_CHANGE;
+	else
+		val &= ~PORT_LOGIC_SPEED_CHANGE;
+
+	dw_pcie_writel_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL, val);
+}
+
+void dw_pcie_link_set_n_fts(struct dw_pcie *pci, u32 n_fts)
+{
+	u32 val;
+
+	val = dw_pcie_readl_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL);
+	val &= ~PORT_LOGIC_N_FTS;
+	val |= n_fts;
+	dw_pcie_writel_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL, val);
+}
+
  static u8 dw_pcie_iatu_unroll_enabled(struct dw_pcie *pci)
  {
  	u32 val;
drivers/pci/controller/dwc/pcie-designware.h +12 -0
index 5a18e94e52c8..3beac10e4a4c 100644
@@ -30,7 +30,12 @@
  #define LINK_WAIT_IATU			9
 
  /* Synopsys-specific PCIe configuration registers */
+#define PCIE_PORT_AFR			0x70C
+#define PORT_AFR_N_FTS_MASK		GENMASK(15, 8)
+#define PORT_AFR_CC_N_FTS_MASK		GENMASK(23, 16)
+
  #define PCIE_PORT_LINK_CONTROL		0x710
+#define PORT_LINK_DLL_LINK_EN		BIT(5)
  #define PORT_LINK_MODE_MASK		GENMASK(21, 16)
  #define PORT_LINK_MODE(n)		FIELD_PREP(PORT_LINK_MODE_MASK, n)
  #define PORT_LINK_MODE_1_LANES		PORT_LINK_MODE(0x1)
@@ -46,6 +51,7 @@
  #define PCIE_PORT_DEBUG1_LINK_IN_TRAINING	BIT(29)
 
  #define PCIE_LINK_WIDTH_SPEED_CONTROL	0x80C
+#define PORT_LOGIC_N_FTS		GENMASK(7, 0)
  #define PORT_LOGIC_SPEED_CHANGE		BIT(17)
  #define PORT_LOGIC_LINK_WIDTH_MASK	GENMASK(12, 8)
  #define PORT_LOGIC_LINK_WIDTH(n)	FIELD_PREP(PORT_LOGIC_LINK_WIDTH_MASK, n)
@@ -60,6 +66,9 @@
  #define PCIE_MSI_INTR0_MASK		0x82C
  #define PCIE_MSI_INTR0_STATUS		0x830
 
+#define PCIE_PORT_MULTI_LANE_CTRL	0x8C0
+#define PORT_MLTI_UPCFG_SUPPORT		BIT(7)
+
  #define PCIE_ATU_VIEWPORT		0x900
  #define PCIE_ATU_REGION_INBOUND		BIT(31)
  #define PCIE_ATU_REGION_OUTBOUND	0
@@ -273,6 +282,9 @@ void dw_pcie_write_dbi2(struct dw_pcie *pci, u32 reg, size_t size, u32 val);
  u32 dw_pcie_read_atu(struct dw_pcie *pci, u32 reg, size_t size);
  void dw_pcie_write_atu(struct dw_pcie *pci, u32 reg, size_t size, u32 val);
  int dw_pcie_link_up(struct dw_pcie *pci);
+void dw_pcie_upconfig_setup(struct dw_pcie *pci);
+void dw_pcie_link_speed_change(struct dw_pcie *pci, bool enable);
+void dw_pcie_link_set_n_fts(struct dw_pcie *pci, u32 n_fts);
  int dw_pcie_wait_for_link(struct dw_pcie *pci);
  void dw_pcie_prog_outbound_atu(struct dw_pcie *pci, int index,
  			       int type, u64 cpu_addr, u64 pci_addr,
drivers/pci/controller/dwc/pcie-intel-gw.c new file +98 -0
new file mode 100644
index 000000000000..9142c70db808
@@ -0,0 +1,590 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * PCIe host controller driver for Intel Gateway SoCs
+ *
+ * Copyright (c) 2019 Intel Corporation.
+ */
+
+#include <linux/bitfield.h>
+#include <linux/clk.h>
+#include <linux/gpio/consumer.h>
+#include <linux/interrupt.h>
+#include <linux/iopoll.h>
+#include <linux/of_irq.h>
+#include <linux/of_pci.h>
+#include <linux/of_platform.h>
+#include <linux/pci_regs.h>
+#include <linux/phy/phy.h>
+#include <linux/platform_device.h>
+#include <linux/reset.h>
+
+#include "../../pci.h"
+#include "pcie-designware.h"
+
+#define PCIE_CAP_OFST			0x70
+
+#define PORT_AFR_N_FTS_GEN12_DFT	(SZ_128 - 1)
+#define PORT_AFR_N_FTS_GEN3		180
+#define PORT_AFR_N_FTS_GEN4		196
+
+/* PCIe Application logic Registers */
+#define PCIE_APP_CCR			0x10
+#define PCIE_APP_CCR_LTSSM_ENABLE	BIT(0)
+
+#define PCIE_APP_MSG_CR			0x30
+#define PCIE_APP_MSG_XMT_PM_TURNOFF	BIT(0)
+
+#define PCIE_APP_PMC			0x44
+#define PCIE_APP_PMC_IN_L2		BIT(20)
+
+#define PCIE_APP_IRNEN			0xF4
+#define PCIE_APP_IRNCR			0xF8
+#define PCIE_APP_IRN_AER_REPORT		BIT(0)
+#define PCIE_APP_IRN_PME		BIT(2)
+#define PCIE_APP_IRN_RX_VDM_MSG		BIT(4)
+#define PCIE_APP_IRN_PM_TO_ACK		BIT(9)
+#define PCIE_APP_IRN_LINK_AUTO_BW_STAT	BIT(11)
+#define PCIE_APP_IRN_BW_MGT		BIT(12)
+#define PCIE_APP_IRN_MSG_LTR		BIT(18)
+#define PCIE_APP_IRN_SYS_ERR_RC		BIT(29)
+#define PCIE_APP_INTX_OFST		12
+
+#define PCIE_APP_IRN_INT \
+			(PCIE_APP_IRN_AER_REPORT | PCIE_APP_IRN_PME | \
+			PCIE_APP_IRN_RX_VDM_MSG | PCIE_APP_IRN_SYS_ERR_RC | \
+			PCIE_APP_IRN_PM_TO_ACK | PCIE_APP_IRN_MSG_LTR | \
+			PCIE_APP_IRN_BW_MGT | PCIE_APP_IRN_LINK_AUTO_BW_STAT | \
+			(PCIE_APP_INTX_OFST + PCI_INTERRUPT_INTA) | \
+			(PCIE_APP_INTX_OFST + PCI_INTERRUPT_INTB) | \
+			(PCIE_APP_INTX_OFST + PCI_INTERRUPT_INTC) | \
+			(PCIE_APP_INTX_OFST + PCI_INTERRUPT_INTD))
+
+#define BUS_IATU_OFFS			SZ_256M
+#define RST_INTRVL_DFT_MS		100
+
+enum {
+	PCIE_LINK_SPEED_AUTO = 0,
+	PCIE_LINK_SPEED_GEN1,
+	PCIE_LINK_SPEED_GEN2,
+	PCIE_LINK_SPEED_GEN3,
+	PCIE_LINK_SPEED_GEN4,
+};
+
+struct intel_pcie_soc {
+	unsigned int pcie_ver;
+	unsigned int pcie_atu_offset;
+	u32 num_viewport;
+};
+
+struct intel_pcie_port {
+	struct dw_pcie		pci;
+	unsigned int		id; /* Physical RC Index */
+	void __iomem		*app_base;
+	struct gpio_desc	*reset_gpio;
+	u32			rst_intrvl;
+	u32			max_speed;
+	u32			link_gen;
+	u32			max_width;
+	u32			n_fts;
+	struct clk		*core_clk;
+	struct reset_control	*core_rst;
+	struct phy		*phy;
+};
+
+static void pcie_update_bits(void __iomem *base, u32 mask, u32 val, u32 ofs)
+{
+	u32 orig, tmp;
+
+	orig = readl(base + ofs);
Quoted 8 lines

drivers/pci/controller/dwc/pcie-intel-gw.c line 104

Quoted 6 lines
drivers/pci/controller/dwc/pcie-intel-gw.c new file +6 -0
+
+	tmp = (orig & ~mask) | (val & mask);
+
+	if (tmp != orig)
+		writel(tmp, base + ofs);
+}

I'd suggest to the a take on FIELD_PREP() and FIELD_GET() and use more intuitive names such as new and old, instead of orig and tmp.

Sure, i will update it.

Quoted 93 lines
Quoted 93 lines
drivers/pci/controller/dwc/pcie-intel-gw.c new file +92 -0
+static inline u32 pcie_app_rd(struct intel_pcie_port *lpp, u32 ofs)
+{
+	return readl(lpp->app_base + ofs);
+}
+
+static inline void pcie_app_wr(struct intel_pcie_port *lpp, u32 val, u32 ofs)
+{
+	writel(val, lpp->app_base + ofs);
+}
+
+static void pcie_app_wr_mask(struct intel_pcie_port *lpp,
+			     u32 mask, u32 val, u32 ofs)
+{
+	pcie_update_bits(lpp->app_base, mask, val, ofs);
+}
+
+static inline u32 pcie_rc_cfg_rd(struct intel_pcie_port *lpp, u32 ofs)
+{
+	return dw_pcie_readl_dbi(&lpp->pci, ofs);
+}
+
+static inline void pcie_rc_cfg_wr(struct intel_pcie_port *lpp, u32 val, u32 ofs)
+{
+	dw_pcie_writel_dbi(&lpp->pci, ofs, val);
+}
+
+static void pcie_rc_cfg_wr_mask(struct intel_pcie_port *lpp,
+				u32 mask, u32 val, u32 ofs)
+{
+	pcie_update_bits(lpp->pci.dbi_base, mask, val, ofs);
+}
+
+static void intel_pcie_ltssm_enable(struct intel_pcie_port *lpp)
+{
+	pcie_app_wr_mask(lpp, PCIE_APP_CCR_LTSSM_ENABLE,
+			 PCIE_APP_CCR_LTSSM_ENABLE, PCIE_APP_CCR);
+}
+
+static void intel_pcie_ltssm_disable(struct intel_pcie_port *lpp)
+{
+	pcie_app_wr_mask(lpp, PCIE_APP_CCR_LTSSM_ENABLE, 0, PCIE_APP_CCR);
+}
+
+static void intel_pcie_link_setup(struct intel_pcie_port *lpp)
+{
+	u32 val;
+
+	val = pcie_rc_cfg_rd(lpp, PCIE_CAP_OFST + PCI_EXP_LNKCAP);
+	lpp->max_speed = FIELD_GET(PCI_EXP_LNKCAP_SLS, val);
+	lpp->max_width = FIELD_GET(PCI_EXP_LNKCAP_MLW, val);
+
+	val = pcie_rc_cfg_rd(lpp, PCIE_CAP_OFST + PCI_EXP_LNKCTL);
+
+	val &= ~(PCI_EXP_LNKCTL_LD | PCI_EXP_LNKCTL_ASPMC);
+	val |= (PCI_EXP_LNKSTA_SLC << 16) | PCI_EXP_LNKCTL_CCC |
+	       PCI_EXP_LNKCTL_RCB;
+	pcie_rc_cfg_wr(lpp, val, PCIE_CAP_OFST + PCI_EXP_LNKCTL);
+}
+
+static void intel_pcie_max_speed_setup(struct intel_pcie_port *lpp)
+{
+	u32 reg, val;
+
+	reg = pcie_rc_cfg_rd(lpp, PCIE_CAP_OFST + PCI_EXP_LNKCTL2);
+	switch (lpp->link_gen) {
+	case PCIE_LINK_SPEED_GEN1:
+		reg &= ~PCI_EXP_LNKCTL2_TLS;
+		reg |= PCI_EXP_LNKCTL2_HASD|
+			PCI_EXP_LNKCTL2_TLS_2_5GT;
+		break;
+	case PCIE_LINK_SPEED_GEN2:
+		reg &= ~PCI_EXP_LNKCTL2_TLS;
+		reg |= PCI_EXP_LNKCTL2_HASD|
+			PCI_EXP_LNKCTL2_TLS_5_0GT;
+		break;
+	case PCIE_LINK_SPEED_GEN3:
+		reg &= ~PCI_EXP_LNKCTL2_TLS;
+		reg |= PCI_EXP_LNKCTL2_HASD|
+			PCI_EXP_LNKCTL2_TLS_8_0GT;
+		break;
+	case PCIE_LINK_SPEED_GEN4:
+		reg &= ~PCI_EXP_LNKCTL2_TLS;
+		reg |= PCI_EXP_LNKCTL2_HASD|
+			PCI_EXP_LNKCTL2_TLS_16_0GT;
+		break;
+	default:
+		/* Use hardware capability */
+		val = pcie_rc_cfg_rd(lpp, PCIE_CAP_OFST + PCI_EXP_LNKCAP);
+		val = FIELD_GET(PCI_EXP_LNKCAP_SLS, val);
+		reg &= ~PCI_EXP_LNKCTL2_HASD;
+		reg |= val;
+		break;
Quoted 8 lines

drivers/pci/controller/dwc/pcie-intel-gw.c line 204

Quoted 7 lines
drivers/pci/controller/dwc/pcie-intel-gw.c new file +7 -0
+	}
+
+	pcie_rc_cfg_wr(lpp, reg, PCIE_CAP_OFST + PCI_EXP_LNKCTL2);
+	dw_pcie_link_set_n_fts(&lpp->pci, lpp->n_fts);
+}
+
+

Reduce the number of empty lines here.

Ok, will remove it.

Quoted 389 lines
Quoted 389 lines
drivers/pci/controller/dwc/pcie-intel-gw.c new file +386 -0
+
+static void intel_pcie_port_logic_setup(struct intel_pcie_port *lpp)
+{
+	u32 val, mask;
+
+	switch (lpp->max_speed) {
+	case PCIE_LINK_SPEED_GEN3:
+		lpp->n_fts = PORT_AFR_N_FTS_GEN3;
+		break;
+	case PCIE_LINK_SPEED_GEN4:
+		lpp->n_fts = PORT_AFR_N_FTS_GEN4;
+		break;
+	default:
+		lpp->n_fts = PORT_AFR_N_FTS_GEN12_DFT;
+		break;
+	}
+
+	mask = PORT_AFR_N_FTS_MASK | PORT_AFR_CC_N_FTS_MASK;
+	val = FIELD_PREP(PORT_AFR_N_FTS_MASK, lpp->n_fts) |
+	       FIELD_PREP(PORT_AFR_CC_N_FTS_MASK, lpp->n_fts);
+	pcie_rc_cfg_wr_mask(lpp, mask, val, PCIE_PORT_AFR);
+
+	/* Port Link Control Register */
+	pcie_rc_cfg_wr_mask(lpp, PORT_LINK_DLL_LINK_EN,
+			    PORT_LINK_DLL_LINK_EN, PCIE_PORT_LINK_CONTROL);
+}
+
+static void intel_pcie_rc_setup(struct intel_pcie_port *lpp)
+{
+	intel_pcie_ltssm_disable(lpp);
+	intel_pcie_link_setup(lpp);
+	dw_pcie_setup_rc(&lpp->pci.pp);
+	dw_pcie_upconfig_setup(&lpp->pci);
+	intel_pcie_port_logic_setup(lpp);
+	intel_pcie_max_speed_setup(lpp);
+}
+
+static int intel_pcie_ep_rst_init(struct intel_pcie_port *lpp)
+{
+	struct device *dev = lpp->pci.dev;
+	int ret;
+
+	lpp->reset_gpio = devm_gpiod_get(dev, "reset", GPIOD_OUT_LOW);
+	if (IS_ERR(lpp->reset_gpio)) {
+		ret = PTR_ERR(lpp->reset_gpio);
+		if (ret != -EPROBE_DEFER)
+			dev_err(dev, "failed to request PCIe GPIO: %d\n", ret);
+		return ret;
+	}
+
+	/* Make initial reset last for 100us */
+	usleep_range(100, 200);
+
+	return 0;
+}
+
+static void intel_pcie_core_rst_assert(struct intel_pcie_port *lpp)
+{
+	reset_control_assert(lpp->core_rst);
+}
+
+static void intel_pcie_core_rst_deassert(struct intel_pcie_port *lpp)
+{
+	/*
+	 * One micro-second delay to make sure the reset pulse
+	 * wide enough so that core reset is clean.
+	 */
+	udelay(1);
+	reset_control_deassert(lpp->core_rst);
+
+	/*
+	 * Some SoC core reset also reset PHY, more delay needed
+	 * to make sure the reset process is done.
+	 */
+	usleep_range(1000, 2000);
+}
+
+static void intel_pcie_device_rst_assert(struct intel_pcie_port *lpp)
+{
+	gpiod_set_value_cansleep(lpp->reset_gpio, 1);
+}
+
+static void intel_pcie_device_rst_deassert(struct intel_pcie_port *lpp)
+{
+	msleep(lpp->rst_intrvl);
+	gpiod_set_value_cansleep(lpp->reset_gpio, 0);
+}
+
+static int intel_pcie_app_logic_setup(struct intel_pcie_port *lpp)
+{
+	intel_pcie_device_rst_deassert(lpp);
+	intel_pcie_ltssm_enable(lpp);
+
+	return dw_pcie_wait_for_link(&lpp->pci);
+}
+
+static void intel_pcie_core_irq_disable(struct intel_pcie_port *lpp)
+{
+	pcie_app_wr(lpp, 0, PCIE_APP_IRNEN);
+	pcie_app_wr(lpp, PCIE_APP_IRN_INT,  PCIE_APP_IRNCR);
+}
+
+static int intel_pcie_get_resources(struct platform_device *pdev)
+{
+	struct intel_pcie_port *lpp = platform_get_drvdata(pdev);
+	struct dw_pcie *pci = &lpp->pci;
+	struct device *dev = pci->dev;
+	struct resource *res;
+	int ret;
+
+	ret = device_property_read_u32(dev, "linux,pci-domain", &lpp->id);
+	if (ret) {
+		dev_err(dev, "failed to get domain id, errno %d\n", ret);
+		return ret;
+	}
+
+	res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "dbi");
+
+	pci->dbi_base = devm_ioremap_resource(dev, res);
+	if (IS_ERR(pci->dbi_base))
+		return PTR_ERR(pci->dbi_base);
+
+	lpp->core_clk = devm_clk_get(dev, NULL);
+	if (IS_ERR(lpp->core_clk)) {
+		ret = PTR_ERR(lpp->core_clk);
+		if (ret != -EPROBE_DEFER)
+			dev_err(dev, "failed to get clks: %d\n", ret);
+		return ret;
+	}
+
+	lpp->core_rst = devm_reset_control_get(dev, NULL);
+	if (IS_ERR(lpp->core_rst)) {
+		ret = PTR_ERR(lpp->core_rst);
+		if (ret != -EPROBE_DEFER)
+			dev_err(dev, "failed to get resets: %d\n", ret);
+		return ret;
+	}
+
+	ret = device_property_match_string(dev, "device_type", "pci");
+	if (ret) {
+		dev_err(dev, "failed to find pci device type: %d\n", ret);
+		return ret;
+	}
+
+	if (device_property_read_u32(dev, "reset-assert-ms", &lpp->rst_intrvl))
+		lpp->rst_intrvl = RST_INTRVL_DFT_MS;
+
+	ret = of_pci_get_max_link_speed(dev->of_node);
+		lpp->link_gen = ret < 0 ? 0 : ret;
+
+	res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "app");
+
+	lpp->app_base = devm_ioremap_resource(dev, res);
+	if (IS_ERR(lpp->app_base))
+		return PTR_ERR(lpp->app_base);
+
+	lpp->phy = devm_phy_get(dev, "pcie");
+	if (IS_ERR(lpp->phy)) {
+		ret = PTR_ERR(lpp->phy);
+		if (ret != -EPROBE_DEFER)
+			dev_err(dev, "couldn't get pcie-phy: %d\n", ret);
+		return ret;
+	}
+
+	return 0;
+}
+
+static void intel_pcie_deinit_phy(struct intel_pcie_port *lpp)
+{
+	phy_exit(lpp->phy);
+}
+
+static int intel_pcie_wait_l2(struct intel_pcie_port *lpp)
+{
+	u32 value;
+	int ret;
+
+	if (lpp->max_speed < PCIE_LINK_SPEED_GEN3)
+		return 0;
+
+	/* Send PME_TURN_OFF message */
+	pcie_app_wr_mask(lpp, PCIE_APP_MSG_XMT_PM_TURNOFF,
+			 PCIE_APP_MSG_XMT_PM_TURNOFF, PCIE_APP_MSG_CR);
+
+	/* Read PMC status and wait for falling into L2 link state */
+	ret = readl_poll_timeout(lpp->app_base + PCIE_APP_PMC, value,
+				 (value & PCIE_APP_PMC_IN_L2), 20,
+				 jiffies_to_usecs(5 * HZ));
+	if (ret)
+		dev_err(lpp->pci.dev, "PCIe link enter L2 timeout!\n");
+
+	return ret;
+}
+
+static void intel_pcie_turn_off(struct intel_pcie_port *lpp)
+{
+	if (dw_pcie_link_up(&lpp->pci))
+		intel_pcie_wait_l2(lpp);
+
+	/* Put endpoint device in reset state */
+	intel_pcie_device_rst_assert(lpp);
+	pcie_rc_cfg_wr_mask(lpp, PCI_COMMAND_MEMORY, 0, PCI_COMMAND);
+}
+
+static int intel_pcie_host_setup(struct intel_pcie_port *lpp)
+{
+	int ret;
+
+	intel_pcie_core_rst_assert(lpp);
+	intel_pcie_device_rst_assert(lpp);
+
+	ret = phy_init(lpp->phy);
+	if (ret)
+		return ret;
+
+	intel_pcie_core_rst_deassert(lpp);
+
+	ret = clk_prepare_enable(lpp->core_clk);
+	if (ret) {
+		dev_err(lpp->pci.dev, "Core clock enable failed: %d\n", ret);
+		goto clk_err;
+	}
+
+	intel_pcie_rc_setup(lpp);
+	ret = intel_pcie_app_logic_setup(lpp);
+	if (ret)
+		goto app_init_err;
+
+	/* Enable integrated interrupts */
+	pcie_app_wr_mask(lpp, PCIE_APP_IRN_INT, PCIE_APP_IRN_INT,
+			 PCIE_APP_IRNEN);
+	return 0;
+
+app_init_err:
+	clk_disable_unprepare(lpp->core_clk);
+clk_err:
+	intel_pcie_core_rst_assert(lpp);
+	intel_pcie_deinit_phy(lpp);
+
+	return ret;
+}
+
+static void __intel_pcie_remove(struct intel_pcie_port *lpp)
+{
+	intel_pcie_core_irq_disable(lpp);
+	intel_pcie_turn_off(lpp);
+	clk_disable_unprepare(lpp->core_clk);
+	intel_pcie_core_rst_assert(lpp);
+	intel_pcie_deinit_phy(lpp);
+}
+
+static int intel_pcie_remove(struct platform_device *pdev)
+{
+	struct intel_pcie_port *lpp = platform_get_drvdata(pdev);
+	struct pcie_port *pp = &lpp->pci.pp;
+
+	dw_pcie_host_deinit(pp);
+	__intel_pcie_remove(lpp);
+
+	return 0;
+}
+
+static int __maybe_unused intel_pcie_suspend_noirq(struct device *dev)
+{
+	struct intel_pcie_port *lpp = dev_get_drvdata(dev);
+	int ret;
+
+	intel_pcie_core_irq_disable(lpp);
+	ret = intel_pcie_wait_l2(lpp);
+	if (ret)
+		return ret;
+
+	intel_pcie_deinit_phy(lpp);
+	clk_disable_unprepare(lpp->core_clk);
+	return ret;
+}
+
+static int __maybe_unused intel_pcie_resume_noirq(struct device *dev)
+{
+	struct intel_pcie_port *lpp = dev_get_drvdata(dev);
+
+	return intel_pcie_host_setup(lpp);
+}
+
+static int intel_pcie_rc_init(struct pcie_port *pp)
+{
+	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
+	struct intel_pcie_port *lpp = dev_get_drvdata(pci->dev);
+
+	return intel_pcie_host_setup(lpp);
+}
+
+int intel_pcie_msi_init(struct pcie_port *pp)
+{
+	/* PCIe MSI/MSIx is handled by MSI in x86 processor */
+	return 0;
+}
+
+u64 intel_pcie_cpu_addr(struct dw_pcie *pcie, u64 cpu_addr)
+{
+	return cpu_addr + BUS_IATU_OFFS;
+}
+
+static const struct dw_pcie_ops intel_pcie_ops = {
+	.cpu_addr_fixup = intel_pcie_cpu_addr,
+};
+
+static const struct dw_pcie_host_ops intel_pcie_dw_ops = {
+	.host_init =		intel_pcie_rc_init,
+	.msi_host_init =	intel_pcie_msi_init,
+};
+
+static const struct intel_pcie_soc pcie_data = {
+	.pcie_ver =		0x520A,
+	.pcie_atu_offset =	0xC0000,
+	.num_viewport =		3,
+};
+
+static int intel_pcie_probe(struct platform_device *pdev)
+{
+	const struct intel_pcie_soc *data;
+	struct device *dev = &pdev->dev;
+	struct intel_pcie_port *lpp;
+	struct pcie_port *pp;
+	struct dw_pcie *pci;
+	int ret;
+
+	lpp = devm_kzalloc(dev, sizeof(*lpp), GFP_KERNEL);
+	if (!lpp)
+		return -ENOMEM;
+
+	platform_set_drvdata(pdev, lpp);
+	pci = &lpp->pci;
+	pci->dev = dev;
+	pp = &pci->pp;
+
+	ret = intel_pcie_get_resources(pdev);
+	if (ret)
+		return ret;
+
+	ret = intel_pcie_ep_rst_init(lpp);
+	if (ret)
+		return ret;
+
+	data = device_get_match_data(dev);
+	pci->ops = &intel_pcie_ops;
+	pci->version = data->pcie_ver;
+	pci->atu_base = pci->dbi_base + data->pcie_atu_offset;
+	pp->ops = &intel_pcie_dw_ops;
+
+	ret = dw_pcie_host_init(pp);
+	if (ret) {
+		dev_err(dev, "cannot initialize host\n");
+		return ret;
+	}
+
+	/* Intel PCIe doesn't configure IO region, so configure
+	 * viewport to not to access IO region during register
+	 * read write operations.
+	 */
+	pci->num_viewport = data->num_viewport;
+	dev_info(dev, "Intel PCIe Root Complex Port %d init done\n", lpp->id);
+
+	return ret;
+}
+
+static const struct dev_pm_ops intel_pcie_pm_ops = {
+	SET_NOIRQ_SYSTEM_SLEEP_PM_OPS(intel_pcie_suspend_noirq,
+				      intel_pcie_resume_noirq)
+};
+
+static const struct of_device_id of_intel_pcie_match[] = {
+	{ .compatible = "intel,lgm-pcie", .data = &pcie_data },
+	{}
+};
+
+static struct platform_driver intel_pcie_driver = {
+	.probe = intel_pcie_probe,
+	.remove = intel_pcie_remove,
+	.driver = {
+		.name = "intel-gw-pcie",
+		.of_match_table = of_intel_pcie_match,
+		.pm = &intel_pcie_pm_ops,
+	},
+};
+builtin_platform_driver(intel_pcie_driver);
include/uapi/linux/pci_regs.h +0 -0
index 29d6e93fd15e..f6e7e402f879 100644
Quoted 8 lines

include/uapi/linux/pci_regs.h line 676

Quoted 7 lines
include/uapi/linux/pci_regs.h +1 -0
@@ -673,6 +673,7 @@
  #define  PCI_EXP_LNKCTL2_TLS_8_0GT	0x0003 /* Supported Speed 8GT/s */
  #define  PCI_EXP_LNKCTL2_TLS_16_0GT	0x0004 /* Supported Speed 16GT/s */
  #define  PCI_EXP_LNKCTL2_TLS_32_0GT	0x0005 /* Supported Speed 32GT/s */
+#define  PCI_EXP_LNKCTL2_HASD		0x0200 /* Hw Autonomous Speed Disable */

s/Hw/HW

Sure, i will correct it.

Thanks for the valuable inputs.

Regards,
Dilip

Quoted 7 lines
Quoted 6 lines
include/uapi/linux/pci_regs.h +0 -0
  #define PCI_EXP_LNKSTA2		50	/* Link Status 2 */
  #define PCI_CAP_EXP_ENDPOINT_SIZEOF_V2	52	/* v2 endpoints with link end here */
  #define PCI_EXP_SLTCAP2		52	/* Slot Capabilities 2 */
Signature · 1 line
2.11.0
author
#13 lore raw

Re: [PATCH v4 2/3] dwc: PCI: intel: PCIe RC controller driver

Hi Gustavo Pimentel,

Quoted 312 lines · Dilip Kota

Hi Gustavo Pimentel,

Quoted 13 lines · Gustavo Pimentel

Hi

Quoted 8 lines · Dilip Kota

Add support to PCIe RC controller on Intel Gateway SoCs. PCIe controller is based of Synopsys DesignWare pci core.

Intel PCIe driver requires Upconfig support, fast training sequence configuration and link speed change. So adding the respective helper functions in the pcie DesignWare framework. It also programs hardware autonomous speed during speed configuration so defining it in pci_regs.h.

Quoted 6 lines

Please do the replacement in all of your patches

s/pcie/PCIe
s/pci/PCI

Also I think the correct term is Upconfigure and not Upconfig

Yes, i will update it.

Quoted 281 lines
Quoted 281 lines

Changes on v4:
    Rename the driver naming and description to
     "PCIe RC controller on Intel Gateway SoCs".
    Use PCIe core register macros defined in pci_regs.h
     and remove respective local definitions.
    Remove pcie core interrupt handling.
    Move pcie link control speed change, upconfig and FTS. configuration functions to DesignWare framework.
    Use of_pci_get_max_link_speed().
    Mark dependency on X86 and COMPILE_TEST in Kconfig.
    Remove lanes and add n_fts variables in intel_pcie_port structure. Rename rst_interval variable to rst_intrvl in intel_pcie_port structure. Remove intel_pcie_mem_iatu() as it is already perfomed in dw_setup_rc() Move sysfs attributes specific code to separate patch. Remove redundant error handling.
    Reduce LoCs by doing variable initializations while declaration itself. Add extra line after closing parenthesis.
    Move intel_pcie_ep_rst_init() out of get_resources()

changes on v3:
    Rename PCIe app logic registers with PCIE_APP prefix. PCIE_IOP_CTRL configuration is not required. Remove respective code. Remove wrapper functions for clk enable/disable APIs.
    Use platform_get_resource_byname() instead of devm_platform_ioremap_resource() to be similar with DWC framework. Rename phy name to "pciephy".
    Modify debug message in msi_init() callback to be more specific. Remove map_irq() callback.
    Enable the INTx interrupts at the time of PCIe initialization. Reduce memory fragmentation by using variable "struct dw_pcie pci" instead of allocating memory.
    Reduce the delay to 100us during enpoint initialization intel_pcie_ep_rst_init().
    Call  dw_pcie_host_deinit() during remove() instead of directly calling PCIe core APIs.
    Rename "intel,rst-interval" to "reset-assert-ms".
    Remove unused APP logic Interrupt bit macro definitions. Use dwc framework's dw_pcie_setup_rc() for PCIe host specific configuration instead of redefining the same functionality in the driver.
    Move the whole DT parsing specific code to intel_pcie_get_resources()

---
  drivers/pci/controller/dwc/Kconfig           |  10 + drivers/pci/controller/dwc/Makefile          |   1 + drivers/pci/controller/dwc/pcie-designware.c |  34 ++ drivers/pci/controller/dwc/pcie-designware.h |  12 + drivers/pci/controller/dwc/pcie-intel-gw.c   | 590
+++++++++++++++++++++++++++
  include/uapi/linux/pci_regs.h                |   1 +

  6 files changed, 648 insertions(+)

  create mode 100644 drivers/pci/controller/dwc/pcie-intel-gw.c

drivers/pci/controller/dwc/Kconfig +0 -0
index 0ba988b5b5bc..b33ed1cc873d 100644
@@ -82,6 +82,16 @@ config PCIE_DW_PLAT_EP

        order to enable device-specific features PCI_DW_PLAT_EP must be selected.
  +config PCIE_INTEL_GW
+        bool "Intel Gateway PCIe host controller support"
+    depends on OF && (X86 || COMPILE_TEST)
+    select PCIE_DW_HOST
+    help
+          Say 'Y' here to enable support for PCIe Host controller driver.
+      The PCIe controller on Intel Gateway SoCs is based on the Synopsys
+      DesignWare pcie core and therefore uses the DesignWare core +      functions for the driver implementation.
+
  config PCI_EXYNOS
      bool "Samsung Exynos PCIe controller"
      depends on SOC_EXYNOS5440 || COMPILE_TEST

drivers/pci/controller/dwc/Makefile +0 -0
index b30336181d46..99db83cd2f35 100644
@@ -3,6 +3,7 @@ obj-$(CONFIG_PCIE_DW) += pcie-designware.o

  obj-$(CONFIG_PCIE_DW_HOST) += pcie-designware-host.o obj-$(CONFIG_PCIE_DW_EP) += pcie-designware-ep.o obj-$(CONFIG_PCIE_DW_PLAT) += pcie-designware-plat.o +obj-$(CONFIG_PCIE_INTEL_GW) += pcie-intel-gw.o obj-$(CONFIG_PCI_DRA7XX) += pci-dra7xx.o obj-$(CONFIG_PCI_EXYNOS) += pci-exynos.o obj-$(CONFIG_PCI_IMX6) += pci-imx6.o

drivers/pci/controller/dwc/pcie-designware.c +0 -0
index 820488dfeaed..4c391bfd681a 100644
@@ -474,6 +474,40 @@ int dw_pcie_link_up(struct dw_pcie *pci)
           (!(val & PCIE_PORT_DEBUG1_LINK_IN_TRAINING)));
   }

  +

drivers/pci/controller/dwc/pcie-designware.c +33 -0
+void dw_pcie_upconfig_setup(struct dw_pcie *pci)
+{
+    u32 val;
+
+    val = dw_pcie_readl_dbi(pci, PCIE_PORT_MULTI_LANE_CTRL);
+    dw_pcie_writel_dbi(pci, PCIE_PORT_MULTI_LANE_CTRL,
+               val | PORT_MLTI_UPCFG_SUPPORT);
+}
+
+void dw_pcie_link_speed_change(struct dw_pcie *pci, bool enable)
+{
+    u32 val;
+
+    val = dw_pcie_readl_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL);
+
+    if (enable)
+        val |= PORT_LOGIC_SPEED_CHANGE;
+    else
+        val &= ~PORT_LOGIC_SPEED_CHANGE;
+
+    dw_pcie_writel_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL, val);
+}
+
+void dw_pcie_link_set_n_fts(struct dw_pcie *pci, u32 n_fts)
+{
+    u32 val;
+
+    val = dw_pcie_readl_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL);
+    val &= ~PORT_LOGIC_N_FTS;
+    val |= n_fts;
+    dw_pcie_writel_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL, val);
+}
+

  static u8 dw_pcie_iatu_unroll_enabled(struct dw_pcie *pci) {
      u32 val;

drivers/pci/controller/dwc/pcie-designware.h +6 -0
index 5a18e94e52c8..3beac10e4a4c 100644
@@ -30,7 +30,12 @@
   #define LINK_WAIT_IATU            9
     /* Synopsys-specific PCIe configuration registers */
+#define PCIE_PORT_AFR            0x70C
+#define PORT_AFR_N_FTS_MASK        GENMASK(15, 8)
+#define PORT_AFR_CC_N_FTS_MASK        GENMASK(23, 16)
+
   #define PCIE_PORT_LINK_CONTROL        0x710
+#define PORT_LINK_DLL_LINK_EN        BIT(5)
   #define PORT_LINK_MODE_MASK        GENMASK(21, 16)
   #define PORT_LINK_MODE(n) FIELD_PREP(PORT_LINK_MODE_MASK, n)
   #define PORT_LINK_MODE_1_LANES        PORT_LINK_MODE(0x1)
@@ -46,6 +51,7 @@
   #define PCIE_PORT_DEBUG1_LINK_IN_TRAINING    BIT(29)
     #define PCIE_LINK_WIDTH_SPEED_CONTROL    0x80C
+#define PORT_LOGIC_N_FTS        GENMASK(7, 0)
   #define PORT_LOGIC_SPEED_CHANGE        BIT(17)
   #define PORT_LOGIC_LINK_WIDTH_MASK    GENMASK(12, 8)
   #define PORT_LOGIC_LINK_WIDTH(n) FIELD_PREP(PORT_LOGIC_LINK_WIDTH_MASK, n)
@@ -60,6 +66,9 @@
   #define PCIE_MSI_INTR0_MASK        0x82C
   #define PCIE_MSI_INTR0_STATUS        0x830

  +#define PCIE_PORT_MULTI_LANE_CTRL    0x8C0

drivers/pci/controller/dwc/pcie-designware.h +2 -0
+#define PORT_MLTI_UPCFG_SUPPORT        BIT(7)
+

  #define PCIE_ATU_VIEWPORT        0x900
  #define PCIE_ATU_REGION_INBOUND        BIT(31)
  #define PCIE_ATU_REGION_OUTBOUND    0

drivers/pci/controller/dwc/pcie-designware.h +3 -0
@@ -273,6 +282,9 @@ void dw_pcie_write_dbi2(struct dw_pcie *pci, u32 
 reg, size_t size, u32 val);
   u32 dw_pcie_read_atu(struct dw_pcie *pci, u32 reg, size_t size);
   void dw_pcie_write_atu(struct dw_pcie *pci, u32 reg, size_t size, u32 val);
   int dw_pcie_link_up(struct dw_pcie *pci);
+void dw_pcie_upconfig_setup(struct dw_pcie *pci);
+void dw_pcie_link_speed_change(struct dw_pcie *pci, bool enable);
+void dw_pcie_link_set_n_fts(struct dw_pcie *pci, u32 n_fts);
   int dw_pcie_wait_for_link(struct dw_pcie *pci);
   void dw_pcie_prog_outbound_atu(struct dw_pcie *pci, int index,

                     int type, u64 cpu_addr, u64 pci_addr,

drivers/pci/controller/dwc/pcie-intel-gw.c new file +94 -0
new file mode 100644
index 000000000000..9142c70db808
@@ -0,0 +1,590 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * PCIe host controller driver for Intel Gateway SoCs
+ *
+ * Copyright (c) 2019 Intel Corporation.
+ */
+
+#include <linux/bitfield.h>
+#include <linux/clk.h>
+#include <linux/gpio/consumer.h>
+#include <linux/interrupt.h>
+#include <linux/iopoll.h>
+#include <linux/of_irq.h>
+#include <linux/of_pci.h>
+#include <linux/of_platform.h>
+#include <linux/pci_regs.h>
+#include <linux/phy/phy.h>
+#include <linux/platform_device.h>
+#include <linux/reset.h>
+
+#include "../../pci.h"
+#include "pcie-designware.h"
+
+#define PCIE_CAP_OFST            0x70
+
+#define PORT_AFR_N_FTS_GEN12_DFT    (SZ_128 - 1)
+#define PORT_AFR_N_FTS_GEN3        180
+#define PORT_AFR_N_FTS_GEN4        196
+
+/* PCIe Application logic Registers */
+#define PCIE_APP_CCR            0x10
+#define PCIE_APP_CCR_LTSSM_ENABLE    BIT(0)
+
+#define PCIE_APP_MSG_CR            0x30
+#define PCIE_APP_MSG_XMT_PM_TURNOFF    BIT(0)
+
+#define PCIE_APP_PMC            0x44
+#define PCIE_APP_PMC_IN_L2        BIT(20)
+
+#define PCIE_APP_IRNEN            0xF4
+#define PCIE_APP_IRNCR            0xF8
+#define PCIE_APP_IRN_AER_REPORT        BIT(0)
+#define PCIE_APP_IRN_PME        BIT(2)
+#define PCIE_APP_IRN_RX_VDM_MSG        BIT(4)
+#define PCIE_APP_IRN_PM_TO_ACK        BIT(9)
+#define PCIE_APP_IRN_LINK_AUTO_BW_STAT    BIT(11)
+#define PCIE_APP_IRN_BW_MGT        BIT(12)
+#define PCIE_APP_IRN_MSG_LTR        BIT(18)
+#define PCIE_APP_IRN_SYS_ERR_RC        BIT(29)
+#define PCIE_APP_INTX_OFST        12
+
+#define PCIE_APP_IRN_INT \
+            (PCIE_APP_IRN_AER_REPORT | PCIE_APP_IRN_PME | \
+            PCIE_APP_IRN_RX_VDM_MSG | PCIE_APP_IRN_SYS_ERR_RC | \
+            PCIE_APP_IRN_PM_TO_ACK | PCIE_APP_IRN_MSG_LTR | \
+            PCIE_APP_IRN_BW_MGT | PCIE_APP_IRN_LINK_AUTO_BW_STAT | \
+            (PCIE_APP_INTX_OFST + PCI_INTERRUPT_INTA) | \
+            (PCIE_APP_INTX_OFST + PCI_INTERRUPT_INTB) | \
+            (PCIE_APP_INTX_OFST + PCI_INTERRUPT_INTC) | \
+            (PCIE_APP_INTX_OFST + PCI_INTERRUPT_INTD))
+
+#define BUS_IATU_OFFS            SZ_256M
+#define RST_INTRVL_DFT_MS        100
+
+enum {
+    PCIE_LINK_SPEED_AUTO = 0,
+    PCIE_LINK_SPEED_GEN1,
+    PCIE_LINK_SPEED_GEN2,
+    PCIE_LINK_SPEED_GEN3,
+    PCIE_LINK_SPEED_GEN4,
+};
+
+struct intel_pcie_soc {
+    unsigned int pcie_ver;
+    unsigned int pcie_atu_offset;
+    u32 num_viewport;
+};
+
+struct intel_pcie_port {
+    struct dw_pcie        pci;
+    unsigned int        id; /* Physical RC Index */
+    void __iomem        *app_base;
+    struct gpio_desc    *reset_gpio;
+    u32            rst_intrvl;
+    u32            max_speed;
+    u32            link_gen;
+    u32            max_width;
+    u32            n_fts;
+    struct clk        *core_clk;
+    struct reset_control    *core_rst;
+    struct phy        *phy;
+};
+
+static void pcie_update_bits(void __iomem *base, u32 mask, u32 val, 

u32 ofs)

drivers/pci/controller/dwc/pcie-intel-gw.c new file +4 -0
+{
+    u32 orig, tmp;
+
+    orig = readl(base + ofs);
Quoted 8 lines

drivers/pci/controller/dwc/pcie-intel-gw.c line 104

Quoted 6 lines
drivers/pci/controller/dwc/pcie-intel-gw.c new file +6 -0
+
+    tmp = (orig & ~mask) | (val & mask);
+
+    if (tmp != orig)
+        writel(tmp, base + ofs);
+}

I'd suggest to the a take on FIELD_PREP() and FIELD_GET() and use more intuitive names such as new and old, instead of orig and tmp.

Quoted 1 line

Sure, i will update it.

I tried using FIELD_PREP and FIELD_GET but it is failing because FIELD_PREP and FIELD_GET are expecting mask should be constant macro. u32 or u32 const are not accepted.

Regards,
Dilip

Quoted 520 lines
Quoted 95 lines
Quoted 95 lines
drivers/pci/controller/dwc/pcie-intel-gw.c new file +6 -0
+static inline u32 pcie_app_rd(struct intel_pcie_port *lpp, u32 ofs)
+{
+    return readl(lpp->app_base + ofs);
+}
+
+static inline void pcie_app_wr(struct intel_pcie_port *lpp, u32 

val, u32 ofs)

drivers/pci/controller/dwc/pcie-intel-gw.c new file +16 -0
+{
+    writel(val, lpp->app_base + ofs);
+}
+
+static void pcie_app_wr_mask(struct intel_pcie_port *lpp,
+                 u32 mask, u32 val, u32 ofs)
+{
+    pcie_update_bits(lpp->app_base, mask, val, ofs);
+}
+
+static inline u32 pcie_rc_cfg_rd(struct intel_pcie_port *lpp, u32 ofs)
+{
+    return dw_pcie_readl_dbi(&lpp->pci, ofs);
+}
+
+static inline void pcie_rc_cfg_wr(struct intel_pcie_port *lpp, u32 

val, u32 ofs)

drivers/pci/controller/dwc/pcie-intel-gw.c new file +70 -0
+{
+    dw_pcie_writel_dbi(&lpp->pci, ofs, val);
+}
+
+static void pcie_rc_cfg_wr_mask(struct intel_pcie_port *lpp,
+                u32 mask, u32 val, u32 ofs)
+{
+    pcie_update_bits(lpp->pci.dbi_base, mask, val, ofs);
+}
+
+static void intel_pcie_ltssm_enable(struct intel_pcie_port *lpp)
+{
+    pcie_app_wr_mask(lpp, PCIE_APP_CCR_LTSSM_ENABLE,
+             PCIE_APP_CCR_LTSSM_ENABLE, PCIE_APP_CCR);
+}
+
+static void intel_pcie_ltssm_disable(struct intel_pcie_port *lpp)
+{
+    pcie_app_wr_mask(lpp, PCIE_APP_CCR_LTSSM_ENABLE, 0, PCIE_APP_CCR);
+}
+
+static void intel_pcie_link_setup(struct intel_pcie_port *lpp)
+{
+    u32 val;
+
+    val = pcie_rc_cfg_rd(lpp, PCIE_CAP_OFST + PCI_EXP_LNKCAP);
+    lpp->max_speed = FIELD_GET(PCI_EXP_LNKCAP_SLS, val);
+    lpp->max_width = FIELD_GET(PCI_EXP_LNKCAP_MLW, val);
+
+    val = pcie_rc_cfg_rd(lpp, PCIE_CAP_OFST + PCI_EXP_LNKCTL);
+
+    val &= ~(PCI_EXP_LNKCTL_LD | PCI_EXP_LNKCTL_ASPMC);
+    val |= (PCI_EXP_LNKSTA_SLC << 16) | PCI_EXP_LNKCTL_CCC |
+           PCI_EXP_LNKCTL_RCB;
+    pcie_rc_cfg_wr(lpp, val, PCIE_CAP_OFST + PCI_EXP_LNKCTL);
+}
+
+static void intel_pcie_max_speed_setup(struct intel_pcie_port *lpp)
+{
+    u32 reg, val;
+
+    reg = pcie_rc_cfg_rd(lpp, PCIE_CAP_OFST + PCI_EXP_LNKCTL2);
+    switch (lpp->link_gen) {
+    case PCIE_LINK_SPEED_GEN1:
+        reg &= ~PCI_EXP_LNKCTL2_TLS;
+        reg |= PCI_EXP_LNKCTL2_HASD|
+            PCI_EXP_LNKCTL2_TLS_2_5GT;
+        break;
+    case PCIE_LINK_SPEED_GEN2:
+        reg &= ~PCI_EXP_LNKCTL2_TLS;
+        reg |= PCI_EXP_LNKCTL2_HASD|
+            PCI_EXP_LNKCTL2_TLS_5_0GT;
+        break;
+    case PCIE_LINK_SPEED_GEN3:
+        reg &= ~PCI_EXP_LNKCTL2_TLS;
+        reg |= PCI_EXP_LNKCTL2_HASD|
+            PCI_EXP_LNKCTL2_TLS_8_0GT;
+        break;
+    case PCIE_LINK_SPEED_GEN4:
+        reg &= ~PCI_EXP_LNKCTL2_TLS;
+        reg |= PCI_EXP_LNKCTL2_HASD|
+            PCI_EXP_LNKCTL2_TLS_16_0GT;
+        break;
+    default:
+        /* Use hardware capability */
+        val = pcie_rc_cfg_rd(lpp, PCIE_CAP_OFST + PCI_EXP_LNKCAP);
+        val = FIELD_GET(PCI_EXP_LNKCAP_SLS, val);
+        reg &= ~PCI_EXP_LNKCTL2_HASD;
+        reg |= val;
+        break;
Quoted 8 lines

drivers/pci/controller/dwc/pcie-intel-gw.c line 204

Quoted 7 lines

+    }

drivers/pci/controller/dwc/pcie-intel-gw.c new file +6 -0
+
+    pcie_rc_cfg_wr(lpp, reg, PCIE_CAP_OFST + PCI_EXP_LNKCTL2);
+    dw_pcie_link_set_n_fts(&lpp->pci, lpp->n_fts);
+}
+
+

Reduce the number of empty lines here.

Ok, will remove it.

Quoted 393 lines
Quoted 393 lines
drivers/pci/controller/dwc/pcie-intel-gw.c new file +83 -0
+
+static void intel_pcie_port_logic_setup(struct intel_pcie_port *lpp)
+{
+    u32 val, mask;
+
+    switch (lpp->max_speed) {
+    case PCIE_LINK_SPEED_GEN3:
+        lpp->n_fts = PORT_AFR_N_FTS_GEN3;
+        break;
+    case PCIE_LINK_SPEED_GEN4:
+        lpp->n_fts = PORT_AFR_N_FTS_GEN4;
+        break;
+    default:
+        lpp->n_fts = PORT_AFR_N_FTS_GEN12_DFT;
+        break;
+    }
+
+    mask = PORT_AFR_N_FTS_MASK | PORT_AFR_CC_N_FTS_MASK;
+    val = FIELD_PREP(PORT_AFR_N_FTS_MASK, lpp->n_fts) |
+           FIELD_PREP(PORT_AFR_CC_N_FTS_MASK, lpp->n_fts);
+    pcie_rc_cfg_wr_mask(lpp, mask, val, PCIE_PORT_AFR);
+
+    /* Port Link Control Register */
+    pcie_rc_cfg_wr_mask(lpp, PORT_LINK_DLL_LINK_EN,
+                PORT_LINK_DLL_LINK_EN, PCIE_PORT_LINK_CONTROL);
+}
+
+static void intel_pcie_rc_setup(struct intel_pcie_port *lpp)
+{
+    intel_pcie_ltssm_disable(lpp);
+    intel_pcie_link_setup(lpp);
+    dw_pcie_setup_rc(&lpp->pci.pp);
+    dw_pcie_upconfig_setup(&lpp->pci);
+    intel_pcie_port_logic_setup(lpp);
+    intel_pcie_max_speed_setup(lpp);
+}
+
+static int intel_pcie_ep_rst_init(struct intel_pcie_port *lpp)
+{
+    struct device *dev = lpp->pci.dev;
+    int ret;
+
+    lpp->reset_gpio = devm_gpiod_get(dev, "reset", GPIOD_OUT_LOW);
+    if (IS_ERR(lpp->reset_gpio)) {
+        ret = PTR_ERR(lpp->reset_gpio);
+        if (ret != -EPROBE_DEFER)
+            dev_err(dev, "failed to request PCIe GPIO: %d\n", ret);
+        return ret;
+    }
+
+    /* Make initial reset last for 100us */
+    usleep_range(100, 200);
+
+    return 0;
+}
+
+static void intel_pcie_core_rst_assert(struct intel_pcie_port *lpp)
+{
+    reset_control_assert(lpp->core_rst);
+}
+
+static void intel_pcie_core_rst_deassert(struct intel_pcie_port *lpp)
+{
+    /*
+     * One micro-second delay to make sure the reset pulse
+     * wide enough so that core reset is clean.
+     */
+    udelay(1);
+    reset_control_deassert(lpp->core_rst);
+
+    /*
+     * Some SoC core reset also reset PHY, more delay needed
+     * to make sure the reset process is done.
+     */
+    usleep_range(1000, 2000);
+}
+
+static void intel_pcie_device_rst_assert(struct intel_pcie_port *lpp)
+{
+    gpiod_set_value_cansleep(lpp->reset_gpio, 1);
+}
+
+static void intel_pcie_device_rst_deassert(struct intel_pcie_port 

*lpp)

drivers/pci/controller/dwc/pcie-intel-gw.c new file +62 -0
+{
+    msleep(lpp->rst_intrvl);
+    gpiod_set_value_cansleep(lpp->reset_gpio, 0);
+}
+
+static int intel_pcie_app_logic_setup(struct intel_pcie_port *lpp)
+{
+    intel_pcie_device_rst_deassert(lpp);
+    intel_pcie_ltssm_enable(lpp);
+
+    return dw_pcie_wait_for_link(&lpp->pci);
+}
+
+static void intel_pcie_core_irq_disable(struct intel_pcie_port *lpp)
+{
+    pcie_app_wr(lpp, 0, PCIE_APP_IRNEN);
+    pcie_app_wr(lpp, PCIE_APP_IRN_INT,  PCIE_APP_IRNCR);
+}
+
+static int intel_pcie_get_resources(struct platform_device *pdev)
+{
+    struct intel_pcie_port *lpp = platform_get_drvdata(pdev);
+    struct dw_pcie *pci = &lpp->pci;
+    struct device *dev = pci->dev;
+    struct resource *res;
+    int ret;
+
+    ret = device_property_read_u32(dev, "linux,pci-domain", &lpp->id);
+    if (ret) {
+        dev_err(dev, "failed to get domain id, errno %d\n", ret);
+        return ret;
+    }
+
+    res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "dbi");
+
+    pci->dbi_base = devm_ioremap_resource(dev, res);
+    if (IS_ERR(pci->dbi_base))
+        return PTR_ERR(pci->dbi_base);
+
+    lpp->core_clk = devm_clk_get(dev, NULL);
+    if (IS_ERR(lpp->core_clk)) {
+        ret = PTR_ERR(lpp->core_clk);
+        if (ret != -EPROBE_DEFER)
+            dev_err(dev, "failed to get clks: %d\n", ret);
+        return ret;
+    }
+
+    lpp->core_rst = devm_reset_control_get(dev, NULL);
+    if (IS_ERR(lpp->core_rst)) {
+        ret = PTR_ERR(lpp->core_rst);
+        if (ret != -EPROBE_DEFER)
+            dev_err(dev, "failed to get resets: %d\n", ret);
+        return ret;
+    }
+
+    ret = device_property_match_string(dev, "device_type", "pci");
+    if (ret) {
+        dev_err(dev, "failed to find pci device type: %d\n", ret);
+        return ret;
+    }
+
+    if (device_property_read_u32(dev, "reset-assert-ms", 

&lpp->rst_intrvl))
+        lpp->rst_intrvl = RST_INTRVL_DFT_MS;

drivers/pci/controller/dwc/pcie-intel-gw.c new file +216 -0
+
+    ret = of_pci_get_max_link_speed(dev->of_node);
+        lpp->link_gen = ret < 0 ? 0 : ret;
+
+    res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "app");
+
+    lpp->app_base = devm_ioremap_resource(dev, res);
+    if (IS_ERR(lpp->app_base))
+        return PTR_ERR(lpp->app_base);
+
+    lpp->phy = devm_phy_get(dev, "pcie");
+    if (IS_ERR(lpp->phy)) {
+        ret = PTR_ERR(lpp->phy);
+        if (ret != -EPROBE_DEFER)
+            dev_err(dev, "couldn't get pcie-phy: %d\n", ret);
+        return ret;
+    }
+
+    return 0;
+}
+
+static void intel_pcie_deinit_phy(struct intel_pcie_port *lpp)
+{
+    phy_exit(lpp->phy);
+}
+
+static int intel_pcie_wait_l2(struct intel_pcie_port *lpp)
+{
+    u32 value;
+    int ret;
+
+    if (lpp->max_speed < PCIE_LINK_SPEED_GEN3)
+        return 0;
+
+    /* Send PME_TURN_OFF message */
+    pcie_app_wr_mask(lpp, PCIE_APP_MSG_XMT_PM_TURNOFF,
+             PCIE_APP_MSG_XMT_PM_TURNOFF, PCIE_APP_MSG_CR);
+
+    /* Read PMC status and wait for falling into L2 link state */
+    ret = readl_poll_timeout(lpp->app_base + PCIE_APP_PMC, value,
+                 (value & PCIE_APP_PMC_IN_L2), 20,
+                 jiffies_to_usecs(5 * HZ));
+    if (ret)
+        dev_err(lpp->pci.dev, "PCIe link enter L2 timeout!\n");
+
+    return ret;
+}
+
+static void intel_pcie_turn_off(struct intel_pcie_port *lpp)
+{
+    if (dw_pcie_link_up(&lpp->pci))
+        intel_pcie_wait_l2(lpp);
+
+    /* Put endpoint device in reset state */
+    intel_pcie_device_rst_assert(lpp);
+    pcie_rc_cfg_wr_mask(lpp, PCI_COMMAND_MEMORY, 0, PCI_COMMAND);
+}
+
+static int intel_pcie_host_setup(struct intel_pcie_port *lpp)
+{
+    int ret;
+
+    intel_pcie_core_rst_assert(lpp);
+    intel_pcie_device_rst_assert(lpp);
+
+    ret = phy_init(lpp->phy);
+    if (ret)
+        return ret;
+
+    intel_pcie_core_rst_deassert(lpp);
+
+    ret = clk_prepare_enable(lpp->core_clk);
+    if (ret) {
+        dev_err(lpp->pci.dev, "Core clock enable failed: %d\n", ret);
+        goto clk_err;
+    }
+
+    intel_pcie_rc_setup(lpp);
+    ret = intel_pcie_app_logic_setup(lpp);
+    if (ret)
+        goto app_init_err;
+
+    /* Enable integrated interrupts */
+    pcie_app_wr_mask(lpp, PCIE_APP_IRN_INT, PCIE_APP_IRN_INT,
+             PCIE_APP_IRNEN);
+    return 0;
+
+app_init_err:
+    clk_disable_unprepare(lpp->core_clk);
+clk_err:
+    intel_pcie_core_rst_assert(lpp);
+    intel_pcie_deinit_phy(lpp);
+
+    return ret;
+}
+
+static void __intel_pcie_remove(struct intel_pcie_port *lpp)
+{
+    intel_pcie_core_irq_disable(lpp);
+    intel_pcie_turn_off(lpp);
+    clk_disable_unprepare(lpp->core_clk);
+    intel_pcie_core_rst_assert(lpp);
+    intel_pcie_deinit_phy(lpp);
+}
+
+static int intel_pcie_remove(struct platform_device *pdev)
+{
+    struct intel_pcie_port *lpp = platform_get_drvdata(pdev);
+    struct pcie_port *pp = &lpp->pci.pp;
+
+    dw_pcie_host_deinit(pp);
+    __intel_pcie_remove(lpp);
+
+    return 0;
+}
+
+static int __maybe_unused intel_pcie_suspend_noirq(struct device *dev)
+{
+    struct intel_pcie_port *lpp = dev_get_drvdata(dev);
+    int ret;
+
+    intel_pcie_core_irq_disable(lpp);
+    ret = intel_pcie_wait_l2(lpp);
+    if (ret)
+        return ret;
+
+    intel_pcie_deinit_phy(lpp);
+    clk_disable_unprepare(lpp->core_clk);
+    return ret;
+}
+
+static int __maybe_unused intel_pcie_resume_noirq(struct device *dev)
+{
+    struct intel_pcie_port *lpp = dev_get_drvdata(dev);
+
+    return intel_pcie_host_setup(lpp);
+}
+
+static int intel_pcie_rc_init(struct pcie_port *pp)
+{
+    struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
+    struct intel_pcie_port *lpp = dev_get_drvdata(pci->dev);
+
+    return intel_pcie_host_setup(lpp);
+}
+
+int intel_pcie_msi_init(struct pcie_port *pp)
+{
+    /* PCIe MSI/MSIx is handled by MSI in x86 processor */
+    return 0;
+}
+
+u64 intel_pcie_cpu_addr(struct dw_pcie *pcie, u64 cpu_addr)
+{
+    return cpu_addr + BUS_IATU_OFFS;
+}
+
+static const struct dw_pcie_ops intel_pcie_ops = {
+    .cpu_addr_fixup = intel_pcie_cpu_addr,
+};
+
+static const struct dw_pcie_host_ops intel_pcie_dw_ops = {
+    .host_init =        intel_pcie_rc_init,
+    .msi_host_init =    intel_pcie_msi_init,
+};
+
+static const struct intel_pcie_soc pcie_data = {
+    .pcie_ver =        0x520A,
+    .pcie_atu_offset =    0xC0000,
+    .num_viewport =        3,
+};
+
+static int intel_pcie_probe(struct platform_device *pdev)
+{
+    const struct intel_pcie_soc *data;
+    struct device *dev = &pdev->dev;
+    struct intel_pcie_port *lpp;
+    struct pcie_port *pp;
+    struct dw_pcie *pci;
+    int ret;
+
+    lpp = devm_kzalloc(dev, sizeof(*lpp), GFP_KERNEL);
+    if (!lpp)
+        return -ENOMEM;
+
+    platform_set_drvdata(pdev, lpp);
+    pci = &lpp->pci;
+    pci->dev = dev;
+    pp = &pci->pp;
+
+    ret = intel_pcie_get_resources(pdev);
+    if (ret)
+        return ret;
+
+    ret = intel_pcie_ep_rst_init(lpp);
+    if (ret)
+        return ret;
+
+    data = device_get_match_data(dev);
+    pci->ops = &intel_pcie_ops;
+    pci->version = data->pcie_ver;
+    pci->atu_base = pci->dbi_base + data->pcie_atu_offset;
+    pp->ops = &intel_pcie_dw_ops;
+
+    ret = dw_pcie_host_init(pp);
+    if (ret) {
+        dev_err(dev, "cannot initialize host\n");
+        return ret;
+    }
+
+    /* Intel PCIe doesn't configure IO region, so configure
+     * viewport to not to access IO region during register
+     * read write operations.
+     */
+    pci->num_viewport = data->num_viewport;
+    dev_info(dev, "Intel PCIe Root Complex Port %d init done\n", 

lpp->id);

drivers/pci/controller/dwc/pcie-intel-gw.c new file +24 -0
+
+    return ret;
+}
+
+static const struct dev_pm_ops intel_pcie_pm_ops = {
+    SET_NOIRQ_SYSTEM_SLEEP_PM_OPS(intel_pcie_suspend_noirq,
+                      intel_pcie_resume_noirq)
+};
+
+static const struct of_device_id of_intel_pcie_match[] = {
+    { .compatible = "intel,lgm-pcie", .data = &pcie_data },
+    {}
+};
+
+static struct platform_driver intel_pcie_driver = {
+    .probe = intel_pcie_probe,
+    .remove = intel_pcie_remove,
+    .driver = {
+        .name = "intel-gw-pcie",
+        .of_match_table = of_intel_pcie_match,
+        .pm = &intel_pcie_pm_ops,
+    },
+};
+builtin_platform_driver(intel_pcie_driver);
include/uapi/linux/pci_regs.h +0 -1
index 29d6e93fd15e..f6e7e402f879 100644
--- a/include/uapi/linux/pci_regs.h
Quoted 8 lines
Quoted 7 lines

+++ b/include/uapi/linux/pci_regs.h

include/uapi/linux/pci_regs.h +1 -0
@@ -673,6 +673,7 @@
   #define  PCI_EXP_LNKCTL2_TLS_8_0GT    0x0003 /* Supported Speed 8GT/s */
   #define  PCI_EXP_LNKCTL2_TLS_16_0GT    0x0004 /* Supported Speed 16GT/s */
   #define  PCI_EXP_LNKCTL2_TLS_32_0GT    0x0005 /* Supported Speed 32GT/s */
+#define  PCI_EXP_LNKCTL2_HASD        0x0200 /* Hw Autonomous Speed 

Disable */

s/Hw/HW

Sure, i will correct it.

Thanks for the valuable inputs.

Regards,
Dilip

Quoted 7 lines
Quoted 6 lines

  #define PCI_EXP_LNKSTA2        50 /* Link Status 2 */
  #define PCI_CAP_EXP_ENDPOINT_SIZEOF_V2    52    /* v2 endpoints with link end here */ #define PCI_EXP_SLTCAP2        52    /* Slot Capabilities 2 */

Signature · 1 line
2.11.0
#14 lore raw

Re: [PATCH v4 2/3] dwc: PCI: intel: PCIe RC controller driver

Quoted 5 lines · Dilip Kota
Quoted 4 lines · Dilip Kota
Quoted 3 lines · Gustavo Pimentel

On Mon, Oct 21, 2019 at 7:39:19, Dilip Kota
<[email protected]>
wrote:

First of all, it's a good behaviour to avoid way long quoting.

Quoted 14 lines
Quoted 14 lines
Quoted 14 lines

drivers/pci/controller/dwc/pcie-intel-gw.c line 104

Quoted 12 lines

+static void pcie_update_bits(void __iomem *base, u32 mask, u32 val, u32 ofs)

drivers/pci/controller/dwc/pcie-intel-gw.c new file +10 -0
+{
+    u32 orig, tmp;
+
+    orig = readl(base + ofs);
+
+    tmp = (orig & ~mask) | (val & mask);
+
+    if (tmp != orig)
+        writel(tmp, base + ofs);
+}

I'd suggest to the a take on FIELD_PREP() and FIELD_GET() and use more intuitive names such as new and old, instead of orig and tmp.

Quoted 5 lines
Quoted 1 line

Sure, i will update it.

I tried using FIELD_PREP and FIELD_GET but it is failing because FIELD_PREP and FIELD_GET
are expecting mask should be constant macro. u32 or u32 const are not accepted.

If you look at bitfield.h carefully you may find in particular u32_replace_bits().

Signature · 2 lines
With Best Regards,
Andy Shevchenko
#15 lore raw

Re: [PATCH v4 2/3] dwc: PCI: intel: PCIe RC controller driver

Quoted 23 lines · On Tue, Oct 22, 2019 at 02:44:13PM +0300, [email protected] wrote:
Quoted 5 lines · Dilip Kota
Quoted 4 lines · Dilip Kota
Quoted 3 lines · Gustavo Pimentel

On Mon, Oct 21, 2019 at 7:39:19, Dilip Kota
<[email protected]>
wrote:

First of all, it's a good behaviour to avoid way long quoting.

Quoted 14 lines
Quoted 14 lines
Quoted 14 lines

drivers/pci/controller/dwc/pcie-intel-gw.c line 104

Quoted 12 lines

+static void pcie_update_bits(void __iomem *base, u32 mask, u32 val, u32 ofs)

drivers/pci/controller/dwc/pcie-intel-gw.c new file +10 -0
+{
+    u32 orig, tmp;
+
+    orig = readl(base + ofs);
+
+    tmp = (orig & ~mask) | (val & mask);
+
+    if (tmp != orig)
+        writel(tmp, base + ofs);
+}

I'd suggest to the a take on FIELD_PREP() and FIELD_GET() and use more intuitive names such as new and old, instead of orig and tmp.

Quoted 8 lines
Quoted 5 lines
Quoted 1 line

Sure, i will update it.

I tried using FIELD_PREP and FIELD_GET but it is failing because FIELD_PREP and FIELD_GET
are expecting mask should be constant macro. u32 or u32 const are not accepted.

If you look at bitfield.h carefully you may find in particular u32_replace_bits().

Thanks Andy - I was looking for something like this when I was reviewing the patch but wasn't able to find it.

Andrew Murray

Quoted 6 lines
Signature · 3 lines
-- 
With Best Regards,
Andy Shevchenko
author
#16 lore raw

Re: [PATCH v4 2/3] dwc: PCI: intel: PCIe RC controller driver

In reply to [email protected]
Quoted 7 lines · On 10/22/2019 7:44 PM, [email protected] wrote:
Quoted 5 lines · Dilip Kota
Quoted 4 lines · Dilip Kota
Quoted 3 lines · Gustavo Pimentel

On Mon, Oct 21, 2019 at 7:39:19, Dilip Kota
<[email protected]>
wrote:

First of all, it's a good behaviour to avoid way long quoting.

(Sorry for the late reply, I am back today from sick leave.) Noted. Will take care of it.

Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines

drivers/pci/controller/dwc/pcie-intel-gw.c line 104

Quoted 13 lines

+static void pcie_update_bits(void __iomem *base, u32 mask, u32 val, u32 ofs)

drivers/pci/controller/dwc/pcie-intel-gw.c new file +10 -0
+{
+    u32 orig, tmp;
+
+    orig = readl(base + ofs);
+
+    tmp = (orig & ~mask) | (val & mask);
+
+    if (tmp != orig)
+        writel(tmp, base + ofs);
+}

I'd suggest to the a take on FIELD_PREP() and FIELD_GET() and use more intuitive names such as new and old, instead of orig and tmp.

Quoted 7 lines
Quoted 5 lines
Quoted 1 line

Sure, i will update it.

I tried using FIELD_PREP and FIELD_GET but it is failing because FIELD_PREP and FIELD_GET
are expecting mask should be constant macro. u32 or u32 const are not accepted.

If you look at bitfield.h carefully you may find in particular u32_replace_bits().

Thanks for pointing it.
But, I found bitfields are not contiguous during few function calls, so cannot use them here.

Regards,
Dilip

Quoted 1 line
#17 lore raw

Re: [PATCH v4 2/3] dwc: PCI: intel: PCIe RC controller driver

In reply to Dilip Kota
Quoted 67 lines · Dilip Kota

Add support to PCIe RC controller on Intel Gateway SoCs. PCIe controller is based of Synopsys DesignWare pci core.

Intel PCIe driver requires Upconfig support, fast training sequence configuration and link speed change. So adding the respective helper functions in the pcie DesignWare framework. It also programs hardware autonomous speed during speed configuration so defining it in pci_regs.h.

Changes on v4:
Rename the driver naming and description to
"PCIe RC controller on Intel Gateway SoCs".
Use PCIe core register macros defined in pci_regs.h
and remove respective local definitions.
Remove pcie core interrupt handling.
Move pcie link control speed change, upconfig and FTS. configuration functions to DesignWare framework.
Use of_pci_get_max_link_speed().
Mark dependency on X86 and COMPILE_TEST in Kconfig.
Remove lanes and add n_fts variables in intel_pcie_port structure. Rename rst_interval variable to rst_intrvl in intel_pcie_port structure. Remove intel_pcie_mem_iatu() as it is already perfomed in dw_setup_rc() Move sysfs attributes specific code to separate patch. Remove redundant error handling.
Reduce LoCs by doing variable initializations while declaration itself. Add extra line after closing parenthesis.
Move intel_pcie_ep_rst_init() out of get_resources()

changes on v3:
Rename PCIe app logic registers with PCIE_APP prefix. PCIE_IOP_CTRL configuration is not required. Remove respective code. Remove wrapper functions for clk enable/disable APIs.
Use platform_get_resource_byname() instead of
devm_platform_ioremap_resource() to be similar with DWC framework.
Rename phy name to "pciephy".
Modify debug message in msi_init() callback to be more specific. Remove map_irq() callback.
Enable the INTx interrupts at the time of PCIe initialization. Reduce memory fragmentation by using variable "struct dw_pcie pci"
instead of allocating memory.
Reduce the delay to 100us during enpoint initialization
intel_pcie_ep_rst_init().
Call dw_pcie_host_deinit() during remove() instead of directly
calling PCIe core APIs.
Rename "intel,rst-interval" to "reset-assert-ms".
Remove unused APP logic Interrupt bit macro definitions. Use dwc framework's dw_pcie_setup_rc() for PCIe host specific
configuration instead of redefining the same functionality in the driver.
Move the whole DT parsing specific code to intel_pcie_get_resources()

---
 drivers/pci/controller/dwc/Kconfig           |  10 +
 drivers/pci/controller/dwc/Makefile          |   1 +
 drivers/pci/controller/dwc/pcie-designware.c |  34 ++
 drivers/pci/controller/dwc/pcie-designware.h |  12 +
 drivers/pci/controller/dwc/pcie-intel-gw.c   | 590 +++++++++++++++++++++++++++
 include/uapi/linux/pci_regs.h                |   1 +
 6 files changed, 648 insertions(+)
 create mode 100644 drivers/pci/controller/dwc/pcie-intel-gw.c
drivers/pci/controller/dwc/Kconfig +0 -0
index 0ba988b5b5bc..b33ed1cc873d 100644

drivers/pci/controller/dwc/Kconfig config PCIE_DW_PLAT_EP · line 90

Quoted 9 lines
drivers/pci/controller/dwc/Kconfig +6 -0
@@ -82,6 +82,16 @@ config PCIE_DW_PLAT_EP
 	  order to enable device-specific features PCI_DW_PLAT_EP must be
 	  selected.
 
+config PCIE_INTEL_GW
+        bool "Intel Gateway PCIe host controller support"
+	depends on OF && (X86 || COMPILE_TEST)
+	select PCIE_DW_HOST
+	help
+          Say 'Y' here to enable support for PCIe Host controller driver.

This sentence is very generic, we don't even say what controller driver.

drivers/pci/controller/dwc/Kconfig config PCIE_DW_PLAT_EP · line 93

Quoted 3 lines
drivers/pci/controller/dwc/Kconfig +3 -0
+	  The PCIe controller on Intel Gateway SoCs is based on the Synopsys
+	  DesignWare pcie core and therefore uses the DesignWare core
+	  functions for the driver implementation.

Thanks for changing the driver name, though the description hasn't really changed since the last revision. In particular, Christoph's feedback mentioned that it would be useful to describe where this IP block may be used - is there any reason why we can't make this more useful for users?

Quoted 37 lines
drivers/pci/controller/dwc/Kconfig +1 -0
+
 config PCI_EXYNOS
 	bool "Samsung Exynos PCIe controller"
 	depends on SOC_EXYNOS5440 || COMPILE_TEST
drivers/pci/controller/dwc/Makefile +1 -0
index b30336181d46..99db83cd2f35 100644
@@ -3,6 +3,7 @@ obj-$(CONFIG_PCIE_DW) += pcie-designware.o
 obj-$(CONFIG_PCIE_DW_HOST) += pcie-designware-host.o
 obj-$(CONFIG_PCIE_DW_EP) += pcie-designware-ep.o
 obj-$(CONFIG_PCIE_DW_PLAT) += pcie-designware-plat.o
+obj-$(CONFIG_PCIE_INTEL_GW) += pcie-intel-gw.o
 obj-$(CONFIG_PCI_DRA7XX) += pci-dra7xx.o
 obj-$(CONFIG_PCI_EXYNOS) += pci-exynos.o
 obj-$(CONFIG_PCI_IMX6) += pci-imx6.o
drivers/pci/controller/dwc/pcie-designware.c +13 -0
index 820488dfeaed..4c391bfd681a 100644
@@ -474,6 +474,40 @@ int dw_pcie_link_up(struct dw_pcie *pci)
 		(!(val & PCIE_PORT_DEBUG1_LINK_IN_TRAINING)));
 }
 
+
+void dw_pcie_upconfig_setup(struct dw_pcie *pci)
+{
+	u32 val;
+
+	val = dw_pcie_readl_dbi(pci, PCIE_PORT_MULTI_LANE_CTRL);
+	dw_pcie_writel_dbi(pci, PCIE_PORT_MULTI_LANE_CTRL,
+			   val | PORT_MLTI_UPCFG_SUPPORT);
+}
+
+void dw_pcie_link_speed_change(struct dw_pcie *pci, bool enable)
+{
+	u32 val;

drivers/pci/controller/dwc/pcie-designware.c int dw_pcie_link_up(struct dw_pcie *pci) · line 499

Quoted 10 lines
drivers/pci/controller/dwc/pcie-designware.c +10 -0
+
+	val = dw_pcie_readl_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL);
+
+	if (enable)
+		val |= PORT_LOGIC_SPEED_CHANGE;
+	else
+		val &= ~PORT_LOGIC_SPEED_CHANGE;
+
+	dw_pcie_writel_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL, val);
+}

It's always great to see code move from specific drivers to parent drivers as it helps reduce duplication of code.

Though I notice that imx6_pcie_establish_link (pci-imx6.c) and dw_pcie_setup_rc (pcie-designware-host.c) also does this. Rather than have duplicated code, I think it makes sense to adopt those users to this new code.

In any case, the above function isn't used here. Can you add it in any patch that does use it please?

drivers/pci/controller/dwc/pcie-designware.c int dw_pcie_link_up(struct dw_pcie *pci) · line 509

Quoted 10 lines
drivers/pci/controller/dwc/pcie-designware.c +10 -0
+
+void dw_pcie_link_set_n_fts(struct dw_pcie *pci, u32 n_fts)
+{
+	u32 val;
+
+	val = dw_pcie_readl_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL);
+	val &= ~PORT_LOGIC_N_FTS;
+	val |= n_fts;
+	dw_pcie_writel_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL, val);
+}

I notice that pcie-artpec6.c (artpec6_pcie_set_nfts) also writes the FTS and defines a bunch of macros to support this. It doesn't make sense to duplicate this there. Therefore I think we need to update pcie-artpec6.c to use this new function.

Quoted 66 lines
drivers/pci/controller/dwc/pcie-designware.c +1 -0
+
 static u8 dw_pcie_iatu_unroll_enabled(struct dw_pcie *pci)
 {
 	u32 val;
drivers/pci/controller/dwc/pcie-designware.h +12 -0
index 5a18e94e52c8..3beac10e4a4c 100644
@@ -30,7 +30,12 @@
 #define LINK_WAIT_IATU			9
 
 /* Synopsys-specific PCIe configuration registers */
+#define PCIE_PORT_AFR			0x70C
+#define PORT_AFR_N_FTS_MASK		GENMASK(15, 8)
+#define PORT_AFR_CC_N_FTS_MASK		GENMASK(23, 16)
+
 #define PCIE_PORT_LINK_CONTROL		0x710
+#define PORT_LINK_DLL_LINK_EN		BIT(5)
 #define PORT_LINK_MODE_MASK		GENMASK(21, 16)
 #define PORT_LINK_MODE(n)		FIELD_PREP(PORT_LINK_MODE_MASK, n)
 #define PORT_LINK_MODE_1_LANES		PORT_LINK_MODE(0x1)
@@ -46,6 +51,7 @@
 #define PCIE_PORT_DEBUG1_LINK_IN_TRAINING	BIT(29)
 
 #define PCIE_LINK_WIDTH_SPEED_CONTROL	0x80C
+#define PORT_LOGIC_N_FTS		GENMASK(7, 0)
 #define PORT_LOGIC_SPEED_CHANGE		BIT(17)
 #define PORT_LOGIC_LINK_WIDTH_MASK	GENMASK(12, 8)
 #define PORT_LOGIC_LINK_WIDTH(n)	FIELD_PREP(PORT_LOGIC_LINK_WIDTH_MASK, n)
@@ -60,6 +66,9 @@
 #define PCIE_MSI_INTR0_MASK		0x82C
 #define PCIE_MSI_INTR0_STATUS		0x830
 
+#define PCIE_PORT_MULTI_LANE_CTRL	0x8C0
+#define PORT_MLTI_UPCFG_SUPPORT		BIT(7)
+
 #define PCIE_ATU_VIEWPORT		0x900
 #define PCIE_ATU_REGION_INBOUND		BIT(31)
 #define PCIE_ATU_REGION_OUTBOUND	0
@@ -273,6 +282,9 @@ void dw_pcie_write_dbi2(struct dw_pcie *pci, u32 reg, size_t size, u32 val);
 u32 dw_pcie_read_atu(struct dw_pcie *pci, u32 reg, size_t size);
 void dw_pcie_write_atu(struct dw_pcie *pci, u32 reg, size_t size, u32 val);
 int dw_pcie_link_up(struct dw_pcie *pci);
+void dw_pcie_upconfig_setup(struct dw_pcie *pci);
+void dw_pcie_link_speed_change(struct dw_pcie *pci, bool enable);
+void dw_pcie_link_set_n_fts(struct dw_pcie *pci, u32 n_fts);
 int dw_pcie_wait_for_link(struct dw_pcie *pci);
 void dw_pcie_prog_outbound_atu(struct dw_pcie *pci, int index,
 			       int type, u64 cpu_addr, u64 pci_addr,
drivers/pci/controller/dwc/pcie-intel-gw.c new file +11 -0
new file mode 100644
index 000000000000..9142c70db808
@@ -0,0 +1,590 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * PCIe host controller driver for Intel Gateway SoCs
+ *
+ * Copyright (c) 2019 Intel Corporation.
+ */
+
+#include <linux/bitfield.h>
+#include <linux/clk.h>
+#include <linux/gpio/consumer.h>
+#include <linux/interrupt.h>

drivers/pci/controller/dwc/pcie-intel-gw.c line 21

Quoted 10 lines
drivers/pci/controller/dwc/pcie-intel-gw.c new file +10 -0
+#include <linux/iopoll.h>
+#include <linux/of_irq.h>
+#include <linux/of_pci.h>
+#include <linux/of_platform.h>
+#include <linux/pci_regs.h>
+#include <linux/phy/phy.h>
+#include <linux/platform_device.h>
+#include <linux/reset.h>
+
+#include "../../pci.h"

I expected this to be removed, is it needed now?

Quoted 174 lines
drivers/pci/controller/dwc/pcie-intel-gw.c new file +174 -0
+#include "pcie-designware.h"
+
+#define PCIE_CAP_OFST			0x70
+
+#define PORT_AFR_N_FTS_GEN12_DFT	(SZ_128 - 1)
+#define PORT_AFR_N_FTS_GEN3		180
+#define PORT_AFR_N_FTS_GEN4		196
+
+/* PCIe Application logic Registers */
+#define PCIE_APP_CCR			0x10
+#define PCIE_APP_CCR_LTSSM_ENABLE	BIT(0)
+
+#define PCIE_APP_MSG_CR			0x30
+#define PCIE_APP_MSG_XMT_PM_TURNOFF	BIT(0)
+
+#define PCIE_APP_PMC			0x44
+#define PCIE_APP_PMC_IN_L2		BIT(20)
+
+#define PCIE_APP_IRNEN			0xF4
+#define PCIE_APP_IRNCR			0xF8
+#define PCIE_APP_IRN_AER_REPORT		BIT(0)
+#define PCIE_APP_IRN_PME		BIT(2)
+#define PCIE_APP_IRN_RX_VDM_MSG		BIT(4)
+#define PCIE_APP_IRN_PM_TO_ACK		BIT(9)
+#define PCIE_APP_IRN_LINK_AUTO_BW_STAT	BIT(11)
+#define PCIE_APP_IRN_BW_MGT		BIT(12)
+#define PCIE_APP_IRN_MSG_LTR		BIT(18)
+#define PCIE_APP_IRN_SYS_ERR_RC		BIT(29)
+#define PCIE_APP_INTX_OFST		12
+
+#define PCIE_APP_IRN_INT \
+			(PCIE_APP_IRN_AER_REPORT | PCIE_APP_IRN_PME | \
+			PCIE_APP_IRN_RX_VDM_MSG | PCIE_APP_IRN_SYS_ERR_RC | \
+			PCIE_APP_IRN_PM_TO_ACK | PCIE_APP_IRN_MSG_LTR | \
+			PCIE_APP_IRN_BW_MGT | PCIE_APP_IRN_LINK_AUTO_BW_STAT | \
+			(PCIE_APP_INTX_OFST + PCI_INTERRUPT_INTA) | \
+			(PCIE_APP_INTX_OFST + PCI_INTERRUPT_INTB) | \
+			(PCIE_APP_INTX_OFST + PCI_INTERRUPT_INTC) | \
+			(PCIE_APP_INTX_OFST + PCI_INTERRUPT_INTD))
+
+#define BUS_IATU_OFFS			SZ_256M
+#define RST_INTRVL_DFT_MS		100
+
+enum {
+	PCIE_LINK_SPEED_AUTO = 0,
+	PCIE_LINK_SPEED_GEN1,
+	PCIE_LINK_SPEED_GEN2,
+	PCIE_LINK_SPEED_GEN3,
+	PCIE_LINK_SPEED_GEN4,
+};
+
+struct intel_pcie_soc {
+	unsigned int pcie_ver;
+	unsigned int pcie_atu_offset;
+	u32 num_viewport;
+};
+
+struct intel_pcie_port {
+	struct dw_pcie		pci;
+	unsigned int		id; /* Physical RC Index */
+	void __iomem		*app_base;
+	struct gpio_desc	*reset_gpio;
+	u32			rst_intrvl;
+	u32			max_speed;
+	u32			link_gen;
+	u32			max_width;
+	u32			n_fts;
+	struct clk		*core_clk;
+	struct reset_control	*core_rst;
+	struct phy		*phy;
+};
+
+static void pcie_update_bits(void __iomem *base, u32 mask, u32 val, u32 ofs)
+{
+	u32 orig, tmp;
+
+	orig = readl(base + ofs);
+
+	tmp = (orig & ~mask) | (val & mask);
+
+	if (tmp != orig)
+		writel(tmp, base + ofs);
+}
+
+static inline u32 pcie_app_rd(struct intel_pcie_port *lpp, u32 ofs)
+{
+	return readl(lpp->app_base + ofs);
+}
+
+static inline void pcie_app_wr(struct intel_pcie_port *lpp, u32 val, u32 ofs)
+{
+	writel(val, lpp->app_base + ofs);
+}
+
+static void pcie_app_wr_mask(struct intel_pcie_port *lpp,
+			     u32 mask, u32 val, u32 ofs)
+{
+	pcie_update_bits(lpp->app_base, mask, val, ofs);
+}
+
+static inline u32 pcie_rc_cfg_rd(struct intel_pcie_port *lpp, u32 ofs)
+{
+	return dw_pcie_readl_dbi(&lpp->pci, ofs);
+}
+
+static inline void pcie_rc_cfg_wr(struct intel_pcie_port *lpp, u32 val, u32 ofs)
+{
+	dw_pcie_writel_dbi(&lpp->pci, ofs, val);
+}
+
+static void pcie_rc_cfg_wr_mask(struct intel_pcie_port *lpp,
+				u32 mask, u32 val, u32 ofs)
+{
+	pcie_update_bits(lpp->pci.dbi_base, mask, val, ofs);
+}
+
+static void intel_pcie_ltssm_enable(struct intel_pcie_port *lpp)
+{
+	pcie_app_wr_mask(lpp, PCIE_APP_CCR_LTSSM_ENABLE,
+			 PCIE_APP_CCR_LTSSM_ENABLE, PCIE_APP_CCR);
+}
+
+static void intel_pcie_ltssm_disable(struct intel_pcie_port *lpp)
+{
+	pcie_app_wr_mask(lpp, PCIE_APP_CCR_LTSSM_ENABLE, 0, PCIE_APP_CCR);
+}
+
+static void intel_pcie_link_setup(struct intel_pcie_port *lpp)
+{
+	u32 val;
+
+	val = pcie_rc_cfg_rd(lpp, PCIE_CAP_OFST + PCI_EXP_LNKCAP);
+	lpp->max_speed = FIELD_GET(PCI_EXP_LNKCAP_SLS, val);
+	lpp->max_width = FIELD_GET(PCI_EXP_LNKCAP_MLW, val);
+
+	val = pcie_rc_cfg_rd(lpp, PCIE_CAP_OFST + PCI_EXP_LNKCTL);
+
+	val &= ~(PCI_EXP_LNKCTL_LD | PCI_EXP_LNKCTL_ASPMC);
+	val |= (PCI_EXP_LNKSTA_SLC << 16) | PCI_EXP_LNKCTL_CCC |
+	       PCI_EXP_LNKCTL_RCB;
+	pcie_rc_cfg_wr(lpp, val, PCIE_CAP_OFST + PCI_EXP_LNKCTL);
+}
+
+static void intel_pcie_max_speed_setup(struct intel_pcie_port *lpp)
+{
+	u32 reg, val;
+
+	reg = pcie_rc_cfg_rd(lpp, PCIE_CAP_OFST + PCI_EXP_LNKCTL2);
+	switch (lpp->link_gen) {
+	case PCIE_LINK_SPEED_GEN1:
+		reg &= ~PCI_EXP_LNKCTL2_TLS;
+		reg |= PCI_EXP_LNKCTL2_HASD|
+			PCI_EXP_LNKCTL2_TLS_2_5GT;
+		break;
+	case PCIE_LINK_SPEED_GEN2:
+		reg &= ~PCI_EXP_LNKCTL2_TLS;
+		reg |= PCI_EXP_LNKCTL2_HASD|
+			PCI_EXP_LNKCTL2_TLS_5_0GT;
+		break;
+	case PCIE_LINK_SPEED_GEN3:
+		reg &= ~PCI_EXP_LNKCTL2_TLS;
+		reg |= PCI_EXP_LNKCTL2_HASD|
+			PCI_EXP_LNKCTL2_TLS_8_0GT;
+		break;
+	case PCIE_LINK_SPEED_GEN4:
+		reg &= ~PCI_EXP_LNKCTL2_TLS;
+		reg |= PCI_EXP_LNKCTL2_HASD|
+			PCI_EXP_LNKCTL2_TLS_16_0GT;
+		break;
+	default:
+		/* Use hardware capability */
+		val = pcie_rc_cfg_rd(lpp, PCIE_CAP_OFST + PCI_EXP_LNKCAP);
+		val = FIELD_GET(PCI_EXP_LNKCAP_SLS, val);
+		reg &= ~PCI_EXP_LNKCTL2_HASD;

drivers/pci/controller/dwc/pcie-intel-gw.c line 205

Quoted 10 lines
drivers/pci/controller/dwc/pcie-intel-gw.c new file +10 -0
+		reg |= val;
+		break;
+	}
+
+	pcie_rc_cfg_wr(lpp, reg, PCIE_CAP_OFST + PCI_EXP_LNKCTL2);
+	dw_pcie_link_set_n_fts(&lpp->pci, lpp->n_fts);
+}
+
+
+

Lots of space here isn't needed.

Quoted 104 lines
drivers/pci/controller/dwc/pcie-intel-gw.c new file +104 -0
+static void intel_pcie_port_logic_setup(struct intel_pcie_port *lpp)
+{
+	u32 val, mask;
+
+	switch (lpp->max_speed) {
+	case PCIE_LINK_SPEED_GEN3:
+		lpp->n_fts = PORT_AFR_N_FTS_GEN3;
+		break;
+	case PCIE_LINK_SPEED_GEN4:
+		lpp->n_fts = PORT_AFR_N_FTS_GEN4;
+		break;
+	default:
+		lpp->n_fts = PORT_AFR_N_FTS_GEN12_DFT;
+		break;
+	}
+
+	mask = PORT_AFR_N_FTS_MASK | PORT_AFR_CC_N_FTS_MASK;
+	val = FIELD_PREP(PORT_AFR_N_FTS_MASK, lpp->n_fts) |
+	       FIELD_PREP(PORT_AFR_CC_N_FTS_MASK, lpp->n_fts);
+	pcie_rc_cfg_wr_mask(lpp, mask, val, PCIE_PORT_AFR);
+
+	/* Port Link Control Register */
+	pcie_rc_cfg_wr_mask(lpp, PORT_LINK_DLL_LINK_EN,
+			    PORT_LINK_DLL_LINK_EN, PCIE_PORT_LINK_CONTROL);
+}
+
+static void intel_pcie_rc_setup(struct intel_pcie_port *lpp)
+{
+	intel_pcie_ltssm_disable(lpp);
+	intel_pcie_link_setup(lpp);
+	dw_pcie_setup_rc(&lpp->pci.pp);
+	dw_pcie_upconfig_setup(&lpp->pci);
+	intel_pcie_port_logic_setup(lpp);
+	intel_pcie_max_speed_setup(lpp);
+}
+
+static int intel_pcie_ep_rst_init(struct intel_pcie_port *lpp)
+{
+	struct device *dev = lpp->pci.dev;
+	int ret;
+
+	lpp->reset_gpio = devm_gpiod_get(dev, "reset", GPIOD_OUT_LOW);
+	if (IS_ERR(lpp->reset_gpio)) {
+		ret = PTR_ERR(lpp->reset_gpio);
+		if (ret != -EPROBE_DEFER)
+			dev_err(dev, "failed to request PCIe GPIO: %d\n", ret);
+		return ret;
+	}
+
+	/* Make initial reset last for 100us */
+	usleep_range(100, 200);
+
+	return 0;
+}
+
+static void intel_pcie_core_rst_assert(struct intel_pcie_port *lpp)
+{
+	reset_control_assert(lpp->core_rst);
+}
+
+static void intel_pcie_core_rst_deassert(struct intel_pcie_port *lpp)
+{
+	/*
+	 * One micro-second delay to make sure the reset pulse
+	 * wide enough so that core reset is clean.
+	 */
+	udelay(1);
+	reset_control_deassert(lpp->core_rst);
+
+	/*
+	 * Some SoC core reset also reset PHY, more delay needed
+	 * to make sure the reset process is done.
+	 */
+	usleep_range(1000, 2000);
+}
+
+static void intel_pcie_device_rst_assert(struct intel_pcie_port *lpp)
+{
+	gpiod_set_value_cansleep(lpp->reset_gpio, 1);
+}
+
+static void intel_pcie_device_rst_deassert(struct intel_pcie_port *lpp)
+{
+	msleep(lpp->rst_intrvl);
+	gpiod_set_value_cansleep(lpp->reset_gpio, 0);
+}
+
+static int intel_pcie_app_logic_setup(struct intel_pcie_port *lpp)
+{
+	intel_pcie_device_rst_deassert(lpp);
+	intel_pcie_ltssm_enable(lpp);
+
+	return dw_pcie_wait_for_link(&lpp->pci);
+}
+
+static void intel_pcie_core_irq_disable(struct intel_pcie_port *lpp)
+{
+	pcie_app_wr(lpp, 0, PCIE_APP_IRNEN);
+	pcie_app_wr(lpp, PCIE_APP_IRN_INT,  PCIE_APP_IRNCR);
+}
+
+static int intel_pcie_get_resources(struct platform_device *pdev)
+{
+	struct intel_pcie_port *lpp = platform_get_drvdata(pdev);

drivers/pci/controller/dwc/pcie-intel-gw.c line 319

Quoted 10 lines
drivers/pci/controller/dwc/pcie-intel-gw.c new file +10 -0
+	struct dw_pcie *pci = &lpp->pci;
+	struct device *dev = pci->dev;
+	struct resource *res;
+	int ret;
+
+	ret = device_property_read_u32(dev, "linux,pci-domain", &lpp->id);
+	if (ret) {
+		dev_err(dev, "failed to get domain id, errno %d\n", ret);
+		return ret;
+	}

Why is this a required property?

Quoted 24 lines
drivers/pci/controller/dwc/pcie-intel-gw.c new file +24 -0
+
+	res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "dbi");
+
+	pci->dbi_base = devm_ioremap_resource(dev, res);
+	if (IS_ERR(pci->dbi_base))
+		return PTR_ERR(pci->dbi_base);
+
+	lpp->core_clk = devm_clk_get(dev, NULL);
+	if (IS_ERR(lpp->core_clk)) {
+		ret = PTR_ERR(lpp->core_clk);
+		if (ret != -EPROBE_DEFER)
+			dev_err(dev, "failed to get clks: %d\n", ret);
+		return ret;
+	}
+
+	lpp->core_rst = devm_reset_control_get(dev, NULL);
+	if (IS_ERR(lpp->core_rst)) {
+		ret = PTR_ERR(lpp->core_rst);
+		if (ret != -EPROBE_DEFER)
+			dev_err(dev, "failed to get resets: %d\n", ret);
+		return ret;
+	}
+
+	ret = device_property_match_string(dev, "device_type", "pci");

drivers/pci/controller/dwc/pcie-intel-gw.c line 353

Quoted 10 lines
drivers/pci/controller/dwc/pcie-intel-gw.c new file +10 -0
+	if (ret) {
+		dev_err(dev, "failed to find pci device type: %d\n", ret);
+		return ret;
+	}
+
+	if (device_property_read_u32(dev, "reset-assert-ms", &lpp->rst_intrvl))
+		lpp->rst_intrvl = RST_INTRVL_DFT_MS;
+
+	ret = of_pci_get_max_link_speed(dev->of_node);
+		lpp->link_gen = ret < 0 ? 0 : ret;

I think the spacing may be a little off above.

Quoted 242 lines
drivers/pci/controller/dwc/pcie-intel-gw.c new file +237 -0
+
+	res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "app");
+
+	lpp->app_base = devm_ioremap_resource(dev, res);
+	if (IS_ERR(lpp->app_base))
+		return PTR_ERR(lpp->app_base);
+
+	lpp->phy = devm_phy_get(dev, "pcie");
+	if (IS_ERR(lpp->phy)) {
+		ret = PTR_ERR(lpp->phy);
+		if (ret != -EPROBE_DEFER)
+			dev_err(dev, "couldn't get pcie-phy: %d\n", ret);
+		return ret;
+	}
+
+	return 0;
+}
+
+static void intel_pcie_deinit_phy(struct intel_pcie_port *lpp)
+{
+	phy_exit(lpp->phy);
+}
+
+static int intel_pcie_wait_l2(struct intel_pcie_port *lpp)
+{
+	u32 value;
+	int ret;
+
+	if (lpp->max_speed < PCIE_LINK_SPEED_GEN3)
+		return 0;
+
+	/* Send PME_TURN_OFF message */
+	pcie_app_wr_mask(lpp, PCIE_APP_MSG_XMT_PM_TURNOFF,
+			 PCIE_APP_MSG_XMT_PM_TURNOFF, PCIE_APP_MSG_CR);
+
+	/* Read PMC status and wait for falling into L2 link state */
+	ret = readl_poll_timeout(lpp->app_base + PCIE_APP_PMC, value,
+				 (value & PCIE_APP_PMC_IN_L2), 20,
+				 jiffies_to_usecs(5 * HZ));
+	if (ret)
+		dev_err(lpp->pci.dev, "PCIe link enter L2 timeout!\n");
+
+	return ret;
+}
+
+static void intel_pcie_turn_off(struct intel_pcie_port *lpp)
+{
+	if (dw_pcie_link_up(&lpp->pci))
+		intel_pcie_wait_l2(lpp);
+
+	/* Put endpoint device in reset state */
+	intel_pcie_device_rst_assert(lpp);
+	pcie_rc_cfg_wr_mask(lpp, PCI_COMMAND_MEMORY, 0, PCI_COMMAND);
+}
+
+static int intel_pcie_host_setup(struct intel_pcie_port *lpp)
+{
+	int ret;
+
+	intel_pcie_core_rst_assert(lpp);
+	intel_pcie_device_rst_assert(lpp);
+
+	ret = phy_init(lpp->phy);
+	if (ret)
+		return ret;
+
+	intel_pcie_core_rst_deassert(lpp);
+
+	ret = clk_prepare_enable(lpp->core_clk);
+	if (ret) {
+		dev_err(lpp->pci.dev, "Core clock enable failed: %d\n", ret);
+		goto clk_err;
+	}
+
+	intel_pcie_rc_setup(lpp);
+	ret = intel_pcie_app_logic_setup(lpp);
+	if (ret)
+		goto app_init_err;
+
+	/* Enable integrated interrupts */
+	pcie_app_wr_mask(lpp, PCIE_APP_IRN_INT, PCIE_APP_IRN_INT,
+			 PCIE_APP_IRNEN);
+	return 0;
+
+app_init_err:
+	clk_disable_unprepare(lpp->core_clk);
+clk_err:
+	intel_pcie_core_rst_assert(lpp);
+	intel_pcie_deinit_phy(lpp);
+
+	return ret;
+}
+
+static void __intel_pcie_remove(struct intel_pcie_port *lpp)
+{
+	intel_pcie_core_irq_disable(lpp);
+	intel_pcie_turn_off(lpp);
+	clk_disable_unprepare(lpp->core_clk);
+	intel_pcie_core_rst_assert(lpp);
+	intel_pcie_deinit_phy(lpp);
+}
+
+static int intel_pcie_remove(struct platform_device *pdev)
+{
+	struct intel_pcie_port *lpp = platform_get_drvdata(pdev);
+	struct pcie_port *pp = &lpp->pci.pp;
+
+	dw_pcie_host_deinit(pp);
+	__intel_pcie_remove(lpp);
+
+	return 0;
+}
+
+static int __maybe_unused intel_pcie_suspend_noirq(struct device *dev)
+{
+	struct intel_pcie_port *lpp = dev_get_drvdata(dev);
+	int ret;
+
+	intel_pcie_core_irq_disable(lpp);
+	ret = intel_pcie_wait_l2(lpp);
+	if (ret)
+		return ret;
+
+	intel_pcie_deinit_phy(lpp);
+	clk_disable_unprepare(lpp->core_clk);
+	return ret;
+}
+
+static int __maybe_unused intel_pcie_resume_noirq(struct device *dev)
+{
+	struct intel_pcie_port *lpp = dev_get_drvdata(dev);
+
+	return intel_pcie_host_setup(lpp);
+}
+
+static int intel_pcie_rc_init(struct pcie_port *pp)
+{
+	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
+	struct intel_pcie_port *lpp = dev_get_drvdata(pci->dev);
+
+	return intel_pcie_host_setup(lpp);
+}
+
+int intel_pcie_msi_init(struct pcie_port *pp)
+{
+	/* PCIe MSI/MSIx is handled by MSI in x86 processor */
+	return 0;
+}
+
+u64 intel_pcie_cpu_addr(struct dw_pcie *pcie, u64 cpu_addr)
+{
+	return cpu_addr + BUS_IATU_OFFS;
+}
+
+static const struct dw_pcie_ops intel_pcie_ops = {
+	.cpu_addr_fixup = intel_pcie_cpu_addr,
+};
+
+static const struct dw_pcie_host_ops intel_pcie_dw_ops = {
+	.host_init =		intel_pcie_rc_init,
+	.msi_host_init =	intel_pcie_msi_init,
+};
+
+static const struct intel_pcie_soc pcie_data = {
+	.pcie_ver =		0x520A,
+	.pcie_atu_offset =	0xC0000,
+	.num_viewport =		3,
+};
+
+static int intel_pcie_probe(struct platform_device *pdev)
+{
+	const struct intel_pcie_soc *data;
+	struct device *dev = &pdev->dev;
+	struct intel_pcie_port *lpp;
+	struct pcie_port *pp;
+	struct dw_pcie *pci;
+	int ret;
+
+	lpp = devm_kzalloc(dev, sizeof(*lpp), GFP_KERNEL);
+	if (!lpp)
+		return -ENOMEM;
+
+	platform_set_drvdata(pdev, lpp);
+	pci = &lpp->pci;
+	pci->dev = dev;
+	pp = &pci->pp;
+
+	ret = intel_pcie_get_resources(pdev);
+	if (ret)
+		return ret;
+
+	ret = intel_pcie_ep_rst_init(lpp);
+	if (ret)
+		return ret;
+
+	data = device_get_match_data(dev);
+	pci->ops = &intel_pcie_ops;
+	pci->version = data->pcie_ver;
+	pci->atu_base = pci->dbi_base + data->pcie_atu_offset;
+	pp->ops = &intel_pcie_dw_ops;
+
+	ret = dw_pcie_host_init(pp);
+	if (ret) {
+		dev_err(dev, "cannot initialize host\n");
+		return ret;
+	}
+
+	/* Intel PCIe doesn't configure IO region, so configure
+	 * viewport to not to access IO region during register
+	 * read write operations.
+	 */
+	pci->num_viewport = data->num_viewport;
+	dev_info(dev, "Intel PCIe Root Complex Port %d init done\n", lpp->id);
+
+	return ret;
+}
+
+static const struct dev_pm_ops intel_pcie_pm_ops = {
+	SET_NOIRQ_SYSTEM_SLEEP_PM_OPS(intel_pcie_suspend_noirq,
+				      intel_pcie_resume_noirq)
+};
+
+static const struct of_device_id of_intel_pcie_match[] = {
+	{ .compatible = "intel,lgm-pcie", .data = &pcie_data },
+	{}
+};
+
+static struct platform_driver intel_pcie_driver = {
+	.probe = intel_pcie_probe,
+	.remove = intel_pcie_remove,
+	.driver = {
+		.name = "intel-gw-pcie",
+		.of_match_table = of_intel_pcie_match,
+		.pm = &intel_pcie_pm_ops,
+	},
+};
+builtin_platform_driver(intel_pcie_driver);
include/uapi/linux/pci_regs.h +0 -0
index 29d6e93fd15e..f6e7e402f879 100644

include/uapi/linux/pci_regs.h line 676

Quoted 4 lines
include/uapi/linux/pci_regs.h +1 -0
@@ -673,6 +673,7 @@
 #define  PCI_EXP_LNKCTL2_TLS_8_0GT	0x0003 /* Supported Speed 8GT/s */
 #define  PCI_EXP_LNKCTL2_TLS_16_0GT	0x0004 /* Supported Speed 16GT/s */
 #define  PCI_EXP_LNKCTL2_TLS_32_0GT	0x0005 /* Supported Speed 32GT/s */
+#define  PCI_EXP_LNKCTL2_HASD		0x0200 /* Hw Autonomous Speed Disable */

I think this should be 0x0020.

Thanks,

Andrew Murray

Quoted 6 lines
include/uapi/linux/pci_regs.h +0 -0
 #define PCI_EXP_LNKSTA2		50	/* Link Status 2 */
 #define PCI_CAP_EXP_ENDPOINT_SIZEOF_V2	52	/* v2 endpoints with link end here */
 #define PCI_EXP_SLTCAP2		52	/* Slot Capabilities 2 */
Signature · 1 line
2.11.0
author
#18 lore raw

Re: [PATCH v4 2/3] dwc: PCI: intel: PCIe RC controller driver

Hi Andrew Murray,

Quoted 79 lines · Andrew Murray
Quoted 67 lines · Dilip Kota

Add support to PCIe RC controller on Intel Gateway SoCs. PCIe controller is based of Synopsys DesignWare pci core.

Intel PCIe driver requires Upconfig support, fast training sequence configuration and link speed change. So adding the respective helper functions in the pcie DesignWare framework. It also programs hardware autonomous speed during speed configuration so defining it in pci_regs.h.

Changes on v4:
Rename the driver naming and description to
"PCIe RC controller on Intel Gateway SoCs".
Use PCIe core register macros defined in pci_regs.h
and remove respective local definitions.
Remove pcie core interrupt handling.
Move pcie link control speed change, upconfig and FTS. configuration functions to DesignWare framework.
Use of_pci_get_max_link_speed().
Mark dependency on X86 and COMPILE_TEST in Kconfig.
Remove lanes and add n_fts variables in intel_pcie_port structure. Rename rst_interval variable to rst_intrvl in intel_pcie_port structure. Remove intel_pcie_mem_iatu() as it is already perfomed in dw_setup_rc() Move sysfs attributes specific code to separate patch. Remove redundant error handling.
Reduce LoCs by doing variable initializations while declaration itself. Add extra line after closing parenthesis.
Move intel_pcie_ep_rst_init() out of get_resources()

changes on v3:
Rename PCIe app logic registers with PCIE_APP prefix. PCIE_IOP_CTRL configuration is not required. Remove respective code. Remove wrapper functions for clk enable/disable APIs.
Use platform_get_resource_byname() instead of
devm_platform_ioremap_resource() to be similar with DWC framework.
Rename phy name to "pciephy".
Modify debug message in msi_init() callback to be more specific. Remove map_irq() callback.
Enable the INTx interrupts at the time of PCIe initialization. Reduce memory fragmentation by using variable "struct dw_pcie pci"
instead of allocating memory.
Reduce the delay to 100us during enpoint initialization
intel_pcie_ep_rst_init().
Call dw_pcie_host_deinit() during remove() instead of directly
calling PCIe core APIs.
Rename "intel,rst-interval" to "reset-assert-ms".
Remove unused APP logic Interrupt bit macro definitions. Use dwc framework's dw_pcie_setup_rc() for PCIe host specific
configuration instead of redefining the same functionality in the driver.
Move the whole DT parsing specific code to intel_pcie_get_resources()

---
  drivers/pci/controller/dwc/Kconfig           |  10 +
  drivers/pci/controller/dwc/Makefile          |   1 +
  drivers/pci/controller/dwc/pcie-designware.c |  34 ++
  drivers/pci/controller/dwc/pcie-designware.h |  12 +
  drivers/pci/controller/dwc/pcie-intel-gw.c   | 590 +++++++++++++++++++++++++++
  include/uapi/linux/pci_regs.h                |   1 +
  6 files changed, 648 insertions(+)

create mode 100644 drivers/pci/controller/dwc/pcie-intel-gw.c

drivers/pci/controller/dwc/Kconfig +0 -0
index 0ba988b5b5bc..b33ed1cc873d 100644

drivers/pci/controller/dwc/Kconfig config PCIE_DW_PLAT_EP · line 90

Quoted 9 lines
drivers/pci/controller/dwc/Kconfig +6 -0
@@ -82,6 +82,16 @@ config PCIE_DW_PLAT_EP
  	  order to enable device-specific features PCI_DW_PLAT_EP must be
  	  selected.
 
+config PCIE_INTEL_GW
+        bool "Intel Gateway PCIe host controller support"
+	depends on OF && (X86 || COMPILE_TEST)
+	select PCIE_DW_HOST
+	help
+          Say 'Y' here to enable support for PCIe Host controller driver.

This sentence is very generic, we don't even say what controller driver.

Quoted 7 lines

drivers/pci/controller/dwc/Kconfig config PCIE_DW_PLAT_EP · line 93

Quoted 3 lines
drivers/pci/controller/dwc/Kconfig +3 -0
+	  The PCIe controller on Intel Gateway SoCs is based on the Synopsys
+	  DesignWare pcie core and therefore uses the DesignWare core
+	  functions for the driver implementation.

Thanks for changing the driver name, though the description hasn't really changed since the last revision. In particular, Christoph's feedback mentioned that it would be useful to describe where this IP block may be used - is there any reason why we can't make this more useful for users?

I will add more information.

Quoted 51 lines
Quoted 38 lines
drivers/pci/controller/dwc/Kconfig +1 -0
+
  config PCI_EXYNOS
  	bool "Samsung Exynos PCIe controller"
  	depends on SOC_EXYNOS5440 || COMPILE_TEST
drivers/pci/controller/dwc/Makefile +1 -0
index b30336181d46..99db83cd2f35 100644
@@ -3,6 +3,7 @@ obj-$(CONFIG_PCIE_DW) += pcie-designware.o
  obj-$(CONFIG_PCIE_DW_HOST) += pcie-designware-host.o
  obj-$(CONFIG_PCIE_DW_EP) += pcie-designware-ep.o
  obj-$(CONFIG_PCIE_DW_PLAT) += pcie-designware-plat.o
+obj-$(CONFIG_PCIE_INTEL_GW) += pcie-intel-gw.o
  obj-$(CONFIG_PCI_DRA7XX) += pci-dra7xx.o
  obj-$(CONFIG_PCI_EXYNOS) += pci-exynos.o
  obj-$(CONFIG_PCI_IMX6) += pci-imx6.o
drivers/pci/controller/dwc/pcie-designware.c +13 -0
index 820488dfeaed..4c391bfd681a 100644
@@ -474,6 +474,40 @@ int dw_pcie_link_up(struct dw_pcie *pci)
  		(!(val & PCIE_PORT_DEBUG1_LINK_IN_TRAINING)));
  }
 
+
+void dw_pcie_upconfig_setup(struct dw_pcie *pci)
+{
+	u32 val;
+
+	val = dw_pcie_readl_dbi(pci, PCIE_PORT_MULTI_LANE_CTRL);
+	dw_pcie_writel_dbi(pci, PCIE_PORT_MULTI_LANE_CTRL,
+			   val | PORT_MLTI_UPCFG_SUPPORT);
+}
+
+void dw_pcie_link_speed_change(struct dw_pcie *pci, bool enable)
+{
+	u32 val;

drivers/pci/controller/dwc/pcie-designware.c int dw_pcie_link_up(struct dw_pcie *pci) · line 499

Quoted 10 lines
drivers/pci/controller/dwc/pcie-designware.c +10 -0
+
+	val = dw_pcie_readl_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL);
+
+	if (enable)
+		val |= PORT_LOGIC_SPEED_CHANGE;
+	else
+		val &= ~PORT_LOGIC_SPEED_CHANGE;
+
+	dw_pcie_writel_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL, val);
+}

It's always great to see code move from specific drivers to parent drivers as it helps reduce duplication of code.

Quoted 7 lines

Though I notice that imx6_pcie_establish_link (pci-imx6.c) and dw_pcie_setup_rc (pcie-designware-host.c) also does this. Rather than have duplicated code, I think it makes sense to adopt those users to this new code.

In any case, the above function isn't used here. Can you add it in any patch that does use it please?

In the earlier patch version this function is being called in intel_pcie_rc_setup().
After noticing this operation is being performed in dw_pcie_setup_rc(), i removed the dw_pcie_link_speed_change() function call, but missed to move the function definition to sysfs attribute patch as it is not being called here.

I will move it to the sysfs attribute patch.
Thanks for pointing it.

Quoted 15 lines

drivers/pci/controller/dwc/pcie-designware.c int dw_pcie_link_up(struct dw_pcie *pci) · line 509

Quoted 11 lines
drivers/pci/controller/dwc/pcie-designware.c +10 -0
+
+void dw_pcie_link_set_n_fts(struct dw_pcie *pci, u32 n_fts)
+{
+	u32 val;
+
+	val = dw_pcie_readl_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL);
+	val &= ~PORT_LOGIC_N_FTS;
+	val |= n_fts;
+	dw_pcie_writel_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL, val);
+}

I notice that pcie-artpec6.c (artpec6_pcie_set_nfts) also writes the FTS and defines a bunch of macros to support this. It doesn't make sense to duplicate this there. Therefore I think we need to update pcie-artpec6.c to use this new function.

I think we can do in a separate patch after these changes get merged and keep this patch series for intel PCIe driver and required changes in PCIe DesignWare framework.

Quoted 70 lines
Quoted 70 lines
drivers/pci/controller/dwc/pcie-designware.c +1 -0
+
  static u8 dw_pcie_iatu_unroll_enabled(struct dw_pcie *pci)
  {
  	u32 val;
drivers/pci/controller/dwc/pcie-designware.h +12 -0
index 5a18e94e52c8..3beac10e4a4c 100644
@@ -30,7 +30,12 @@
  #define LINK_WAIT_IATU			9
 
  /* Synopsys-specific PCIe configuration registers */
+#define PCIE_PORT_AFR			0x70C
+#define PORT_AFR_N_FTS_MASK		GENMASK(15, 8)
+#define PORT_AFR_CC_N_FTS_MASK		GENMASK(23, 16)
+
  #define PCIE_PORT_LINK_CONTROL		0x710
+#define PORT_LINK_DLL_LINK_EN		BIT(5)
  #define PORT_LINK_MODE_MASK		GENMASK(21, 16)
  #define PORT_LINK_MODE(n)		FIELD_PREP(PORT_LINK_MODE_MASK, n)
  #define PORT_LINK_MODE_1_LANES		PORT_LINK_MODE(0x1)
@@ -46,6 +51,7 @@
  #define PCIE_PORT_DEBUG1_LINK_IN_TRAINING	BIT(29)
 
  #define PCIE_LINK_WIDTH_SPEED_CONTROL	0x80C
+#define PORT_LOGIC_N_FTS		GENMASK(7, 0)
  #define PORT_LOGIC_SPEED_CHANGE		BIT(17)
  #define PORT_LOGIC_LINK_WIDTH_MASK	GENMASK(12, 8)
  #define PORT_LOGIC_LINK_WIDTH(n)	FIELD_PREP(PORT_LOGIC_LINK_WIDTH_MASK, n)
@@ -60,6 +66,9 @@
  #define PCIE_MSI_INTR0_MASK		0x82C
  #define PCIE_MSI_INTR0_STATUS		0x830
 
+#define PCIE_PORT_MULTI_LANE_CTRL	0x8C0
+#define PORT_MLTI_UPCFG_SUPPORT		BIT(7)
+
  #define PCIE_ATU_VIEWPORT		0x900
  #define PCIE_ATU_REGION_INBOUND		BIT(31)
  #define PCIE_ATU_REGION_OUTBOUND	0
@@ -273,6 +282,9 @@ void dw_pcie_write_dbi2(struct dw_pcie *pci, u32 reg, size_t size, u32 val);
  u32 dw_pcie_read_atu(struct dw_pcie *pci, u32 reg, size_t size);
  void dw_pcie_write_atu(struct dw_pcie *pci, u32 reg, size_t size, u32 val);
  int dw_pcie_link_up(struct dw_pcie *pci);
+void dw_pcie_upconfig_setup(struct dw_pcie *pci);
+void dw_pcie_link_speed_change(struct dw_pcie *pci, bool enable);
+void dw_pcie_link_set_n_fts(struct dw_pcie *pci, u32 n_fts);
  int dw_pcie_wait_for_link(struct dw_pcie *pci);
  void dw_pcie_prog_outbound_atu(struct dw_pcie *pci, int index,
  			       int type, u64 cpu_addr, u64 pci_addr,
drivers/pci/controller/dwc/pcie-intel-gw.c new file +14 -0
new file mode 100644
index 000000000000..9142c70db808
@@ -0,0 +1,590 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * PCIe host controller driver for Intel Gateway SoCs
+ *
+ * Copyright (c) 2019 Intel Corporation.
+ */
+
+#include <linux/bitfield.h>
+#include <linux/clk.h>
+#include <linux/gpio/consumer.h>
+#include <linux/interrupt.h>
+#include <linux/iopoll.h>
+#include <linux/of_irq.h>
+#include <linux/of_pci.h>
Quoted 8 lines

drivers/pci/controller/dwc/pcie-intel-gw.c line 21

Quoted 7 lines
drivers/pci/controller/dwc/pcie-intel-gw.c new file +7 -0
+#include <linux/of_platform.h>
+#include <linux/pci_regs.h>
+#include <linux/phy/phy.h>
+#include <linux/platform_device.h>
+#include <linux/reset.h>
+
+#include "../../pci.h"

I expected this to be removed, is it needed now?

In the earlier patch you suggested to use "of_pci_get_max_link_speed()" instead of device_get_*. And, pci.h is required for it.

Quoted 178 lines
Quoted 178 lines
drivers/pci/controller/dwc/pcie-intel-gw.c new file +177 -0
+#include "pcie-designware.h"
+
+#define PCIE_CAP_OFST			0x70
+
+#define PORT_AFR_N_FTS_GEN12_DFT	(SZ_128 - 1)
+#define PORT_AFR_N_FTS_GEN3		180
+#define PORT_AFR_N_FTS_GEN4		196
+
+/* PCIe Application logic Registers */
+#define PCIE_APP_CCR			0x10
+#define PCIE_APP_CCR_LTSSM_ENABLE	BIT(0)
+
+#define PCIE_APP_MSG_CR			0x30
+#define PCIE_APP_MSG_XMT_PM_TURNOFF	BIT(0)
+
+#define PCIE_APP_PMC			0x44
+#define PCIE_APP_PMC_IN_L2		BIT(20)
+
+#define PCIE_APP_IRNEN			0xF4
+#define PCIE_APP_IRNCR			0xF8
+#define PCIE_APP_IRN_AER_REPORT		BIT(0)
+#define PCIE_APP_IRN_PME		BIT(2)
+#define PCIE_APP_IRN_RX_VDM_MSG		BIT(4)
+#define PCIE_APP_IRN_PM_TO_ACK		BIT(9)
+#define PCIE_APP_IRN_LINK_AUTO_BW_STAT	BIT(11)
+#define PCIE_APP_IRN_BW_MGT		BIT(12)
+#define PCIE_APP_IRN_MSG_LTR		BIT(18)
+#define PCIE_APP_IRN_SYS_ERR_RC		BIT(29)
+#define PCIE_APP_INTX_OFST		12
+
+#define PCIE_APP_IRN_INT \
+			(PCIE_APP_IRN_AER_REPORT | PCIE_APP_IRN_PME | \
+			PCIE_APP_IRN_RX_VDM_MSG | PCIE_APP_IRN_SYS_ERR_RC | \
+			PCIE_APP_IRN_PM_TO_ACK | PCIE_APP_IRN_MSG_LTR | \
+			PCIE_APP_IRN_BW_MGT | PCIE_APP_IRN_LINK_AUTO_BW_STAT | \
+			(PCIE_APP_INTX_OFST + PCI_INTERRUPT_INTA) | \
+			(PCIE_APP_INTX_OFST + PCI_INTERRUPT_INTB) | \
+			(PCIE_APP_INTX_OFST + PCI_INTERRUPT_INTC) | \
+			(PCIE_APP_INTX_OFST + PCI_INTERRUPT_INTD))
+
+#define BUS_IATU_OFFS			SZ_256M
+#define RST_INTRVL_DFT_MS		100
+
+enum {
+	PCIE_LINK_SPEED_AUTO = 0,
+	PCIE_LINK_SPEED_GEN1,
+	PCIE_LINK_SPEED_GEN2,
+	PCIE_LINK_SPEED_GEN3,
+	PCIE_LINK_SPEED_GEN4,
+};
+
+struct intel_pcie_soc {
+	unsigned int pcie_ver;
+	unsigned int pcie_atu_offset;
+	u32 num_viewport;
+};
+
+struct intel_pcie_port {
+	struct dw_pcie		pci;
+	unsigned int		id; /* Physical RC Index */
+	void __iomem		*app_base;
+	struct gpio_desc	*reset_gpio;
+	u32			rst_intrvl;
+	u32			max_speed;
+	u32			link_gen;
+	u32			max_width;
+	u32			n_fts;
+	struct clk		*core_clk;
+	struct reset_control	*core_rst;
+	struct phy		*phy;
+};
+
+static void pcie_update_bits(void __iomem *base, u32 mask, u32 val, u32 ofs)
+{
+	u32 orig, tmp;
+
+	orig = readl(base + ofs);
+
+	tmp = (orig & ~mask) | (val & mask);
+
+	if (tmp != orig)
+		writel(tmp, base + ofs);
+}
+
+static inline u32 pcie_app_rd(struct intel_pcie_port *lpp, u32 ofs)
+{
+	return readl(lpp->app_base + ofs);
+}
+
+static inline void pcie_app_wr(struct intel_pcie_port *lpp, u32 val, u32 ofs)
+{
+	writel(val, lpp->app_base + ofs);
+}
+
+static void pcie_app_wr_mask(struct intel_pcie_port *lpp,
+			     u32 mask, u32 val, u32 ofs)
+{
+	pcie_update_bits(lpp->app_base, mask, val, ofs);
+}
+
+static inline u32 pcie_rc_cfg_rd(struct intel_pcie_port *lpp, u32 ofs)
+{
+	return dw_pcie_readl_dbi(&lpp->pci, ofs);
+}
+
+static inline void pcie_rc_cfg_wr(struct intel_pcie_port *lpp, u32 val, u32 ofs)
+{
+	dw_pcie_writel_dbi(&lpp->pci, ofs, val);
+}
+
+static void pcie_rc_cfg_wr_mask(struct intel_pcie_port *lpp,
+				u32 mask, u32 val, u32 ofs)
+{
+	pcie_update_bits(lpp->pci.dbi_base, mask, val, ofs);
+}
+
+static void intel_pcie_ltssm_enable(struct intel_pcie_port *lpp)
+{
+	pcie_app_wr_mask(lpp, PCIE_APP_CCR_LTSSM_ENABLE,
+			 PCIE_APP_CCR_LTSSM_ENABLE, PCIE_APP_CCR);
+}
+
+static void intel_pcie_ltssm_disable(struct intel_pcie_port *lpp)
+{
+	pcie_app_wr_mask(lpp, PCIE_APP_CCR_LTSSM_ENABLE, 0, PCIE_APP_CCR);
+}
+
+static void intel_pcie_link_setup(struct intel_pcie_port *lpp)
+{
+	u32 val;
+
+	val = pcie_rc_cfg_rd(lpp, PCIE_CAP_OFST + PCI_EXP_LNKCAP);
+	lpp->max_speed = FIELD_GET(PCI_EXP_LNKCAP_SLS, val);
+	lpp->max_width = FIELD_GET(PCI_EXP_LNKCAP_MLW, val);
+
+	val = pcie_rc_cfg_rd(lpp, PCIE_CAP_OFST + PCI_EXP_LNKCTL);
+
+	val &= ~(PCI_EXP_LNKCTL_LD | PCI_EXP_LNKCTL_ASPMC);
+	val |= (PCI_EXP_LNKSTA_SLC << 16) | PCI_EXP_LNKCTL_CCC |
+	       PCI_EXP_LNKCTL_RCB;
+	pcie_rc_cfg_wr(lpp, val, PCIE_CAP_OFST + PCI_EXP_LNKCTL);
+}
+
+static void intel_pcie_max_speed_setup(struct intel_pcie_port *lpp)
+{
+	u32 reg, val;
+
+	reg = pcie_rc_cfg_rd(lpp, PCIE_CAP_OFST + PCI_EXP_LNKCTL2);
+	switch (lpp->link_gen) {
+	case PCIE_LINK_SPEED_GEN1:
+		reg &= ~PCI_EXP_LNKCTL2_TLS;
+		reg |= PCI_EXP_LNKCTL2_HASD|
+			PCI_EXP_LNKCTL2_TLS_2_5GT;
+		break;
+	case PCIE_LINK_SPEED_GEN2:
+		reg &= ~PCI_EXP_LNKCTL2_TLS;
+		reg |= PCI_EXP_LNKCTL2_HASD|
+			PCI_EXP_LNKCTL2_TLS_5_0GT;
+		break;
+	case PCIE_LINK_SPEED_GEN3:
+		reg &= ~PCI_EXP_LNKCTL2_TLS;
+		reg |= PCI_EXP_LNKCTL2_HASD|
+			PCI_EXP_LNKCTL2_TLS_8_0GT;
+		break;
+	case PCIE_LINK_SPEED_GEN4:
+		reg &= ~PCI_EXP_LNKCTL2_TLS;
+		reg |= PCI_EXP_LNKCTL2_HASD|
+			PCI_EXP_LNKCTL2_TLS_16_0GT;
+		break;
+	default:
+		/* Use hardware capability */
+		val = pcie_rc_cfg_rd(lpp, PCIE_CAP_OFST + PCI_EXP_LNKCAP);
+		val = FIELD_GET(PCI_EXP_LNKCAP_SLS, val);
+		reg &= ~PCI_EXP_LNKCTL2_HASD;
+		reg |= val;
+		break;
+	}
Quoted 8 lines

drivers/pci/controller/dwc/pcie-intel-gw.c line 205

Quoted 7 lines
drivers/pci/controller/dwc/pcie-intel-gw.c new file +7 -0
+
+	pcie_rc_cfg_wr(lpp, reg, PCIE_CAP_OFST + PCI_EXP_LNKCTL2);
+	dw_pcie_link_set_n_fts(&lpp->pci, lpp->n_fts);
+}
+
+
+

Lots of space here isn't needed.

i will remove it.

Quoted 108 lines
Quoted 108 lines
drivers/pci/controller/dwc/pcie-intel-gw.c new file +107 -0
+static void intel_pcie_port_logic_setup(struct intel_pcie_port *lpp)
+{
+	u32 val, mask;
+
+	switch (lpp->max_speed) {
+	case PCIE_LINK_SPEED_GEN3:
+		lpp->n_fts = PORT_AFR_N_FTS_GEN3;
+		break;
+	case PCIE_LINK_SPEED_GEN4:
+		lpp->n_fts = PORT_AFR_N_FTS_GEN4;
+		break;
+	default:
+		lpp->n_fts = PORT_AFR_N_FTS_GEN12_DFT;
+		break;
+	}
+
+	mask = PORT_AFR_N_FTS_MASK | PORT_AFR_CC_N_FTS_MASK;
+	val = FIELD_PREP(PORT_AFR_N_FTS_MASK, lpp->n_fts) |
+	       FIELD_PREP(PORT_AFR_CC_N_FTS_MASK, lpp->n_fts);
+	pcie_rc_cfg_wr_mask(lpp, mask, val, PCIE_PORT_AFR);
+
+	/* Port Link Control Register */
+	pcie_rc_cfg_wr_mask(lpp, PORT_LINK_DLL_LINK_EN,
+			    PORT_LINK_DLL_LINK_EN, PCIE_PORT_LINK_CONTROL);
+}
+
+static void intel_pcie_rc_setup(struct intel_pcie_port *lpp)
+{
+	intel_pcie_ltssm_disable(lpp);
+	intel_pcie_link_setup(lpp);
+	dw_pcie_setup_rc(&lpp->pci.pp);
+	dw_pcie_upconfig_setup(&lpp->pci);
+	intel_pcie_port_logic_setup(lpp);
+	intel_pcie_max_speed_setup(lpp);
+}
+
+static int intel_pcie_ep_rst_init(struct intel_pcie_port *lpp)
+{
+	struct device *dev = lpp->pci.dev;
+	int ret;
+
+	lpp->reset_gpio = devm_gpiod_get(dev, "reset", GPIOD_OUT_LOW);
+	if (IS_ERR(lpp->reset_gpio)) {
+		ret = PTR_ERR(lpp->reset_gpio);
+		if (ret != -EPROBE_DEFER)
+			dev_err(dev, "failed to request PCIe GPIO: %d\n", ret);
+		return ret;
+	}
+
+	/* Make initial reset last for 100us */
+	usleep_range(100, 200);
+
+	return 0;
+}
+
+static void intel_pcie_core_rst_assert(struct intel_pcie_port *lpp)
+{
+	reset_control_assert(lpp->core_rst);
+}
+
+static void intel_pcie_core_rst_deassert(struct intel_pcie_port *lpp)
+{
+	/*
+	 * One micro-second delay to make sure the reset pulse
+	 * wide enough so that core reset is clean.
+	 */
+	udelay(1);
+	reset_control_deassert(lpp->core_rst);
+
+	/*
+	 * Some SoC core reset also reset PHY, more delay needed
+	 * to make sure the reset process is done.
+	 */
+	usleep_range(1000, 2000);
+}
+
+static void intel_pcie_device_rst_assert(struct intel_pcie_port *lpp)
+{
+	gpiod_set_value_cansleep(lpp->reset_gpio, 1);
+}
+
+static void intel_pcie_device_rst_deassert(struct intel_pcie_port *lpp)
+{
+	msleep(lpp->rst_intrvl);
+	gpiod_set_value_cansleep(lpp->reset_gpio, 0);
+}
+
+static int intel_pcie_app_logic_setup(struct intel_pcie_port *lpp)
+{
+	intel_pcie_device_rst_deassert(lpp);
+	intel_pcie_ltssm_enable(lpp);
+
+	return dw_pcie_wait_for_link(&lpp->pci);
+}
+
+static void intel_pcie_core_irq_disable(struct intel_pcie_port *lpp)
+{
+	pcie_app_wr(lpp, 0, PCIE_APP_IRNEN);
+	pcie_app_wr(lpp, PCIE_APP_IRN_INT,  PCIE_APP_IRNCR);
+}
+
+static int intel_pcie_get_resources(struct platform_device *pdev)
+{
+	struct intel_pcie_port *lpp = platform_get_drvdata(pdev);
+	struct dw_pcie *pci = &lpp->pci;
+	struct device *dev = pci->dev;
+	struct resource *res;
Quoted 8 lines

drivers/pci/controller/dwc/pcie-intel-gw.c line 319

Quoted 7 lines
drivers/pci/controller/dwc/pcie-intel-gw.c new file +7 -0
+	int ret;
+
+	ret = device_property_read_u32(dev, "linux,pci-domain", &lpp->id);
+	if (ret) {
+		dev_err(dev, "failed to get domain id, errno %d\n", ret);
+		return ret;
+	}

Why is this a required property?

I marked it required property because lpp->id is being used during debug prints. -> sprintf(buf, "Port %2u Width x%u Speed %s GT/s\n", lpp->id,
 -> dev_info(dev, "Intel PCIe Root Complex Port %d init done\n", lpp->id);

Hmmm.. I found, domain id can be traversed from pci_host_bridge structure after dw_pcie_host_init(). I will remove the code for getting the pci-domain here.

Quoted 28 lines
Quoted 28 lines
drivers/pci/controller/dwc/pcie-intel-gw.c new file +27 -0
+
+	res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "dbi");
+
+	pci->dbi_base = devm_ioremap_resource(dev, res);
+	if (IS_ERR(pci->dbi_base))
+		return PTR_ERR(pci->dbi_base);
+
+	lpp->core_clk = devm_clk_get(dev, NULL);
+	if (IS_ERR(lpp->core_clk)) {
+		ret = PTR_ERR(lpp->core_clk);
+		if (ret != -EPROBE_DEFER)
+			dev_err(dev, "failed to get clks: %d\n", ret);
+		return ret;
+	}
+
+	lpp->core_rst = devm_reset_control_get(dev, NULL);
+	if (IS_ERR(lpp->core_rst)) {
+		ret = PTR_ERR(lpp->core_rst);
+		if (ret != -EPROBE_DEFER)
+			dev_err(dev, "failed to get resets: %d\n", ret);
+		return ret;
+	}
+
+	ret = device_property_match_string(dev, "device_type", "pci");
+	if (ret) {
+		dev_err(dev, "failed to find pci device type: %d\n", ret);
+		return ret;
Quoted 8 lines

drivers/pci/controller/dwc/pcie-intel-gw.c line 353

Quoted 7 lines
drivers/pci/controller/dwc/pcie-intel-gw.c new file +7 -0
+	}
+
+	if (device_property_read_u32(dev, "reset-assert-ms", &lpp->rst_intrvl))
+		lpp->rst_intrvl = RST_INTRVL_DFT_MS;
+
+	ret = of_pci_get_max_link_speed(dev->of_node);
+		lpp->link_gen = ret < 0 ? 0 : ret;

I think the spacing may be a little off above.

Yes, typo error. I will correct it.

Quoted 240 lines
Quoted 240 lines
drivers/pci/controller/dwc/pcie-intel-gw.c new file +237 -0
+
+	res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "app");
+
+	lpp->app_base = devm_ioremap_resource(dev, res);
+	if (IS_ERR(lpp->app_base))
+		return PTR_ERR(lpp->app_base);
+
+	lpp->phy = devm_phy_get(dev, "pcie");
+	if (IS_ERR(lpp->phy)) {
+		ret = PTR_ERR(lpp->phy);
+		if (ret != -EPROBE_DEFER)
+			dev_err(dev, "couldn't get pcie-phy: %d\n", ret);
+		return ret;
+	}
+
+	return 0;
+}
+
+static void intel_pcie_deinit_phy(struct intel_pcie_port *lpp)
+{
+	phy_exit(lpp->phy);
+}
+
+static int intel_pcie_wait_l2(struct intel_pcie_port *lpp)
+{
+	u32 value;
+	int ret;
+
+	if (lpp->max_speed < PCIE_LINK_SPEED_GEN3)
+		return 0;
+
+	/* Send PME_TURN_OFF message */
+	pcie_app_wr_mask(lpp, PCIE_APP_MSG_XMT_PM_TURNOFF,
+			 PCIE_APP_MSG_XMT_PM_TURNOFF, PCIE_APP_MSG_CR);
+
+	/* Read PMC status and wait for falling into L2 link state */
+	ret = readl_poll_timeout(lpp->app_base + PCIE_APP_PMC, value,
+				 (value & PCIE_APP_PMC_IN_L2), 20,
+				 jiffies_to_usecs(5 * HZ));
+	if (ret)
+		dev_err(lpp->pci.dev, "PCIe link enter L2 timeout!\n");
+
+	return ret;
+}
+
+static void intel_pcie_turn_off(struct intel_pcie_port *lpp)
+{
+	if (dw_pcie_link_up(&lpp->pci))
+		intel_pcie_wait_l2(lpp);
+
+	/* Put endpoint device in reset state */
+	intel_pcie_device_rst_assert(lpp);
+	pcie_rc_cfg_wr_mask(lpp, PCI_COMMAND_MEMORY, 0, PCI_COMMAND);
+}
+
+static int intel_pcie_host_setup(struct intel_pcie_port *lpp)
+{
+	int ret;
+
+	intel_pcie_core_rst_assert(lpp);
+	intel_pcie_device_rst_assert(lpp);
+
+	ret = phy_init(lpp->phy);
+	if (ret)
+		return ret;
+
+	intel_pcie_core_rst_deassert(lpp);
+
+	ret = clk_prepare_enable(lpp->core_clk);
+	if (ret) {
+		dev_err(lpp->pci.dev, "Core clock enable failed: %d\n", ret);
+		goto clk_err;
+	}
+
+	intel_pcie_rc_setup(lpp);
+	ret = intel_pcie_app_logic_setup(lpp);
+	if (ret)
+		goto app_init_err;
+
+	/* Enable integrated interrupts */
+	pcie_app_wr_mask(lpp, PCIE_APP_IRN_INT, PCIE_APP_IRN_INT,
+			 PCIE_APP_IRNEN);
+	return 0;
+
+app_init_err:
+	clk_disable_unprepare(lpp->core_clk);
+clk_err:
+	intel_pcie_core_rst_assert(lpp);
+	intel_pcie_deinit_phy(lpp);
+
+	return ret;
+}
+
+static void __intel_pcie_remove(struct intel_pcie_port *lpp)
+{
+	intel_pcie_core_irq_disable(lpp);
+	intel_pcie_turn_off(lpp);
+	clk_disable_unprepare(lpp->core_clk);
+	intel_pcie_core_rst_assert(lpp);
+	intel_pcie_deinit_phy(lpp);
+}
+
+static int intel_pcie_remove(struct platform_device *pdev)
+{
+	struct intel_pcie_port *lpp = platform_get_drvdata(pdev);
+	struct pcie_port *pp = &lpp->pci.pp;
+
+	dw_pcie_host_deinit(pp);
+	__intel_pcie_remove(lpp);
+
+	return 0;
+}
+
+static int __maybe_unused intel_pcie_suspend_noirq(struct device *dev)
+{
+	struct intel_pcie_port *lpp = dev_get_drvdata(dev);
+	int ret;
+
+	intel_pcie_core_irq_disable(lpp);
+	ret = intel_pcie_wait_l2(lpp);
+	if (ret)
+		return ret;
+
+	intel_pcie_deinit_phy(lpp);
+	clk_disable_unprepare(lpp->core_clk);
+	return ret;
+}
+
+static int __maybe_unused intel_pcie_resume_noirq(struct device *dev)
+{
+	struct intel_pcie_port *lpp = dev_get_drvdata(dev);
+
+	return intel_pcie_host_setup(lpp);
+}
+
+static int intel_pcie_rc_init(struct pcie_port *pp)
+{
+	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
+	struct intel_pcie_port *lpp = dev_get_drvdata(pci->dev);
+
+	return intel_pcie_host_setup(lpp);
+}
+
+int intel_pcie_msi_init(struct pcie_port *pp)
+{
+	/* PCIe MSI/MSIx is handled by MSI in x86 processor */
+	return 0;
+}
+
+u64 intel_pcie_cpu_addr(struct dw_pcie *pcie, u64 cpu_addr)
+{
+	return cpu_addr + BUS_IATU_OFFS;
+}
+
+static const struct dw_pcie_ops intel_pcie_ops = {
+	.cpu_addr_fixup = intel_pcie_cpu_addr,
+};
+
+static const struct dw_pcie_host_ops intel_pcie_dw_ops = {
+	.host_init =		intel_pcie_rc_init,
+	.msi_host_init =	intel_pcie_msi_init,
+};
+
+static const struct intel_pcie_soc pcie_data = {
+	.pcie_ver =		0x520A,
+	.pcie_atu_offset =	0xC0000,
+	.num_viewport =		3,
+};
+
+static int intel_pcie_probe(struct platform_device *pdev)
+{
+	const struct intel_pcie_soc *data;
+	struct device *dev = &pdev->dev;
+	struct intel_pcie_port *lpp;
+	struct pcie_port *pp;
+	struct dw_pcie *pci;
+	int ret;
+
+	lpp = devm_kzalloc(dev, sizeof(*lpp), GFP_KERNEL);
+	if (!lpp)
+		return -ENOMEM;
+
+	platform_set_drvdata(pdev, lpp);
+	pci = &lpp->pci;
+	pci->dev = dev;
+	pp = &pci->pp;
+
+	ret = intel_pcie_get_resources(pdev);
+	if (ret)
+		return ret;
+
+	ret = intel_pcie_ep_rst_init(lpp);
+	if (ret)
+		return ret;
+
+	data = device_get_match_data(dev);
+	pci->ops = &intel_pcie_ops;
+	pci->version = data->pcie_ver;
+	pci->atu_base = pci->dbi_base + data->pcie_atu_offset;
+	pp->ops = &intel_pcie_dw_ops;
+
+	ret = dw_pcie_host_init(pp);
+	if (ret) {
+		dev_err(dev, "cannot initialize host\n");
+		return ret;
+	}
+
+	/* Intel PCIe doesn't configure IO region, so configure
+	 * viewport to not to access IO region during register
+	 * read write operations.
+	 */
+	pci->num_viewport = data->num_viewport;
+	dev_info(dev, "Intel PCIe Root Complex Port %d init done\n", lpp->id);
+
+	return ret;
+}
+
+static const struct dev_pm_ops intel_pcie_pm_ops = {
+	SET_NOIRQ_SYSTEM_SLEEP_PM_OPS(intel_pcie_suspend_noirq,
+				      intel_pcie_resume_noirq)
+};
+
+static const struct of_device_id of_intel_pcie_match[] = {
+	{ .compatible = "intel,lgm-pcie", .data = &pcie_data },
+	{}
+};
+
+static struct platform_driver intel_pcie_driver = {
+	.probe = intel_pcie_probe,
+	.remove = intel_pcie_remove,
+	.driver = {
+		.name = "intel-gw-pcie",
+		.of_match_table = of_intel_pcie_match,
+		.pm = &intel_pcie_pm_ops,
+	},
+};
+builtin_platform_driver(intel_pcie_driver);
include/uapi/linux/pci_regs.h +0 -0
index 29d6e93fd15e..f6e7e402f879 100644
Quoted 8 lines

include/uapi/linux/pci_regs.h line 676

Quoted 7 lines
include/uapi/linux/pci_regs.h +1 -0
@@ -673,6 +673,7 @@
  #define  PCI_EXP_LNKCTL2_TLS_8_0GT	0x0003 /* Supported Speed 8GT/s */
  #define  PCI_EXP_LNKCTL2_TLS_16_0GT	0x0004 /* Supported Speed 16GT/s */
  #define  PCI_EXP_LNKCTL2_TLS_32_0GT	0x0005 /* Supported Speed 32GT/s */
+#define  PCI_EXP_LNKCTL2_HASD		0x0200 /* Hw Autonomous Speed Disable */

I think this should be 0x0020.

Yes, my miss, i will update.
Thanks for pointing it.

Thanks for reviewing and providing the inputs.
Regards,
Dilip

Quoted 11 lines

Thanks,

Andrew Murray

Quoted 6 lines
include/uapi/linux/pci_regs.h +0 -0
  #define PCI_EXP_LNKSTA2		50	/* Link Status 2 */
  #define PCI_CAP_EXP_ENDPOINT_SIZEOF_V2	52	/* v2 endpoints with link end here */
  #define PCI_EXP_SLTCAP2		52	/* Slot Capabilities 2 */
Signature · 1 line
2.11.0
#19 lore raw

Re: [PATCH v4 2/3] dwc: PCI: intel: PCIe RC controller driver

Quoted 19 lines · Dilip Kota

Hi Andrew Murray,

Quoted 16 lines · Andrew Murray

drivers/pci/controller/dwc/pcie-designware.c int dw_pcie_link_up(struct dw_pcie *pci) · line 509

Quoted 10 lines · Dilip Kota
drivers/pci/controller/dwc/pcie-designware.c +10 -0
+
+void dw_pcie_link_set_n_fts(struct dw_pcie *pci, u32 n_fts)
+{
+	u32 val;
+
+	val = dw_pcie_readl_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL);
+	val &= ~PORT_LOGIC_N_FTS;
+	val |= n_fts;
+	dw_pcie_writel_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL, val);
+}

I notice that pcie-artpec6.c (artpec6_pcie_set_nfts) also writes the FTS and defines a bunch of macros to support this. It doesn't make sense to duplicate this there. Therefore I think we need to update pcie-artpec6.c to use this new function.

Quoted 3 lines

I think we can do in a separate patch after these changes get merged and keep this patch series for intel PCIe driver and required changes in PCIe DesignWare framework.

The pcie-artpec6.c is a DWC driver as well. So I think we can do all this together. This helps reduce the technical debt that will otherwise build up in duplicated code.

Quoted 29 lines
Quoted 20 lines
Quoted 20 lines
drivers/pci/controller/dwc/pcie-intel-gw.c new file +14 -0
new file mode 100644
index 000000000000..9142c70db808
@@ -0,0 +1,590 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * PCIe host controller driver for Intel Gateway SoCs
+ *
+ * Copyright (c) 2019 Intel Corporation.
+ */
+
+#include <linux/bitfield.h>
+#include <linux/clk.h>
+#include <linux/gpio/consumer.h>
+#include <linux/interrupt.h>
+#include <linux/iopoll.h>
+#include <linux/of_irq.h>
+#include <linux/of_pci.h>
Quoted 8 lines

drivers/pci/controller/dwc/pcie-intel-gw.c line 21

Quoted 7 lines
drivers/pci/controller/dwc/pcie-intel-gw.c new file +7 -0
+#include <linux/of_platform.h>
+#include <linux/pci_regs.h>
+#include <linux/phy/phy.h>
+#include <linux/platform_device.h>
+#include <linux/reset.h>
+
+#include "../../pci.h"

I expected this to be removed, is it needed now?

Quoted 3 lines

In the earlier patch you suggested to use "of_pci_get_max_link_speed()" instead of device_get_*.
And, pci.h is required for it.

OK that makes sense.

Quoted 15 lines
Quoted 7 lines
Quoted 7 lines
drivers/pci/controller/dwc/pcie-intel-gw.c new file +7 -0
+
+static int intel_pcie_get_resources(struct platform_device *pdev)
+{
+	struct intel_pcie_port *lpp = platform_get_drvdata(pdev);
+	struct dw_pcie *pci = &lpp->pci;
+	struct device *dev = pci->dev;
+	struct resource *res;
Quoted 8 lines

drivers/pci/controller/dwc/pcie-intel-gw.c line 319

Quoted 7 lines
drivers/pci/controller/dwc/pcie-intel-gw.c new file +7 -0
+	int ret;
+
+	ret = device_property_read_u32(dev, "linux,pci-domain", &lpp->id);
+	if (ret) {
+		dev_err(dev, "failed to get domain id, errno %d\n", ret);
+		return ret;
+	}

Why is this a required property?

Quoted 8 lines

I marked it required property because lpp->id is being used during debug prints.
  -> sprintf(buf, "Port %2u Width x%u Speed %s GT/s\n", lpp->id, -> dev_info(dev, "Intel PCIe Root Complex Port %d init done\n", lpp->id);

Hmmm.. I found, domain id can be traversed from pci_host_bridge structure after dw_pcie_host_init().
I will remove the code for getting the pci-domain here.

Excellent.

Quoted 14 lines
Quoted 4 lines

drivers/pci/controller/dwc/pcie-intel-gw.c line 36

Quoted 3 lines
drivers/pci/controller/dwc/pcie-intel-gw.c new file +1 -0
+#define  PCI_EXP_LNKCTL2_HASD		0x0200 /* Hw Autonomous Speed Disable */

I think this should be 0x0020.

Yes, my miss, i will update.
Thanks for pointing it.

Thanks for reviewing and providing the inputs.
Regards,
Dilip

Quoted 4 lines

Thanks,

Andrew Murray

Thanks,

Andrew Murray

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

#define PCI_EXP_LNKSTA2 50 /* Link Status 2 */
#define PCI_CAP_EXP_ENDPOINT_SIZEOF_V2 52 /* v2 endpoints with link end here */
#define PCI_EXP_SLTCAP2 52 /* Slot Capabilities 2 */

Signature · 1 line
2.11.0
author
#20 lore raw

Re: [PATCH v4 2/3] dwc: PCI: intel: PCIe RC controller driver

Quoted 19 lines · Andrew Murray
Quoted 18 lines · Dilip Kota

Hi Andrew Murray,

Quoted 15 lines · Andrew Murray

drivers/pci/controller/dwc/pcie-designware.c int dw_pcie_link_up(struct dw_pcie *pci) · line 509

Quoted 10 lines · Dilip Kota
drivers/pci/controller/dwc/pcie-designware.c +10 -0
+
+void dw_pcie_link_set_n_fts(struct dw_pcie *pci, u32 n_fts)
+{
+	u32 val;
+
+	val = dw_pcie_readl_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL);
+	val &= ~PORT_LOGIC_N_FTS;
+	val |= n_fts;
+	dw_pcie_writel_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL, val);
+}

I notice that pcie-artpec6.c (artpec6_pcie_set_nfts) also writes the FTS and defines a bunch of macros to support this. It doesn't make sense to duplicate this there. Therefore I think we need to update pcie-artpec6.c to use this new function.

Quoted 6 lines
Quoted 3 lines

I think we can do in a separate patch after these changes get merged and keep this patch series for intel PCIe driver and required changes in PCIe DesignWare framework.

The pcie-artpec6.c is a DWC driver as well. So I think we can do all this together. This helps reduce the technical debt that will otherwise build up in duplicated code.

I agree with you to remove duplicated code, but at this point not sure what all drivers has defined FTS configuration. Reviewing all other DWC drivers and removing them can be done in one single separate patch.

Regards,
Dilip

#21 lore raw

Re: [PATCH v4 2/3] dwc: PCI: intel: PCIe RC controller driver

Quoted 24 lines · Dilip Kota
Quoted 22 lines · Andrew Murray
Quoted 21 lines · Dilip Kota

Hi Andrew Murray,

Quoted 15 lines · Andrew Murray

drivers/pci/controller/dwc/pcie-designware.c int dw_pcie_link_up(struct dw_pcie *pci) · line 509

Quoted 10 lines · Dilip Kota
drivers/pci/controller/dwc/pcie-designware.c +10 -0
+
+void dw_pcie_link_set_n_fts(struct dw_pcie *pci, u32 n_fts)
+{
+	u32 val;
+
+	val = dw_pcie_readl_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL);
+	val &= ~PORT_LOGIC_N_FTS;
+	val |= n_fts;
+	dw_pcie_writel_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL, val);
+}

I notice that pcie-artpec6.c (artpec6_pcie_set_nfts) also writes the FTS and defines a bunch of macros to support this. It doesn't make sense to duplicate this there. Therefore I think we need to update pcie-artpec6.c to use this new function.

I think we can do in a separate patch after these changes get merged and keep this patch series for intel PCIe driver and required changes in PCIe DesignWare framework.

Quoted 7 lines
Quoted 3 lines

The pcie-artpec6.c is a DWC driver as well. So I think we can do all this together. This helps reduce the technical debt that will otherwise build up in duplicated code.

I agree with you to remove duplicated code, but at this point not sure what all drivers has defined FTS configuration.
Reviewing all other DWC drivers and removing them can be done in one single separate patch.

I'm not asking to set up an FTS configuration for all DWC drivers, but instead to move this helper function you've created to somewhere like pcie-designware.c and call it from this driver and pcie-artpec6.c.

Thanks,

Andrew Murray

Quoted 3 lines

Regards,
Dilip

author
#22 lore raw

Re: [PATCH v4 2/3] dwc: PCI: intel: PCIe RC controller driver

Quoted 27 lines · Andrew Murray
Quoted 26 lines · Dilip Kota
Quoted 25 lines · Andrew Murray
Quoted 18 lines · Dilip Kota

Hi Andrew Murray,

Quoted 15 lines · Andrew Murray

drivers/pci/controller/dwc/pcie-designware.c int dw_pcie_link_up(struct dw_pcie *pci) · line 509

Quoted 10 lines · Dilip Kota
drivers/pci/controller/dwc/pcie-designware.c +10 -0
+
+void dw_pcie_link_set_n_fts(struct dw_pcie *pci, u32 n_fts)
+{
+	u32 val;
+
+	val = dw_pcie_readl_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL);
+	val &= ~PORT_LOGIC_N_FTS;
+	val |= n_fts;
+	dw_pcie_writel_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL, val);
+}

I notice that pcie-artpec6.c (artpec6_pcie_set_nfts) also writes the FTS and defines a bunch of macros to support this. It doesn't make sense to duplicate this there. Therefore I think we need to update pcie-artpec6.c to use this new function.

Quoted 3 lines

I think we can do in a separate patch after these changes get merged and keep this patch series for intel PCIe driver and required changes in PCIe DesignWare framework.

The pcie-artpec6.c is a DWC driver as well. So I think we can do all this together. This helps reduce the technical debt that will otherwise build up in duplicated code.

Quoted 7 lines
Quoted 4 lines

I agree with you to remove duplicated code, but at this point not sure what all drivers has defined FTS configuration.
Reviewing all other DWC drivers and removing them can be done in one single separate patch.

I'm not asking to set up an FTS configuration for all DWC drivers, but instead to move this helper function you've created to somewhere like pcie-designware.c and call it from this driver and pcie-artpec6.c.

What i mean is, we need to check how many of the current DWC drivers are configuring the FTS and call the helper function.
Today i have grep all the DWC based drivers and i see pcie-artpec6.c is the only driver doing FTS configuration.

I will add the helper function call in pcie-artpec6.c in the next patch version.

Regards,
Dilip

Quoted 7 lines

Thanks,

Andrew Murray

Quoted 2 lines

Regards,
Dilip

#23 lore raw

Re: [PATCH v4 2/3] dwc: PCI: intel: PCIe RC controller driver

Quoted 36 lines · Dilip Kota
Quoted 27 lines · Andrew Murray
Quoted 26 lines · Dilip Kota
Quoted 25 lines · Andrew Murray
Quoted 18 lines · Dilip Kota

Hi Andrew Murray,

Quoted 15 lines · Andrew Murray

Review

Quoted 10 lines · Dilip Kota
(quoted diff) +10 -0
+
+void dw_pcie_link_set_n_fts(struct dw_pcie *pci, u32 n_fts)
+{
+	u32 val;
+
+	val = dw_pcie_readl_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL);
+	val &= ~PORT_LOGIC_N_FTS;
+	val |= n_fts;
+	dw_pcie_writel_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL, val);
+}

I notice that pcie-artpec6.c (artpec6_pcie_set_nfts) also writes the FTS and defines a bunch of macros to support this. It doesn't make sense to duplicate this there. Therefore I think we need to update pcie-artpec6.c to use this new function.

Quoted 3 lines

I think we can do in a separate patch after these changes get merged and keep this patch series for intel PCIe driver and required changes in PCIe DesignWare framework.

The pcie-artpec6.c is a DWC driver as well. So I think we can do all this together. This helps reduce the technical debt that will otherwise build up in duplicated code.

Quoted 7 lines
Quoted 4 lines

I agree with you to remove duplicated code, but at this point not sure what all drivers has defined FTS configuration.
Reviewing all other DWC drivers and removing them can be done in one single separate patch.

I'm not asking to set up an FTS configuration for all DWC drivers, but instead to move this helper function you've created to somewhere like pcie-designware.c and call it from this driver and pcie-artpec6.c.

Quoted 8 lines

What i mean is, we need to check how many of the current DWC drivers are configuring the FTS
and call the helper function.
Today i have grep all the DWC based drivers and i see pcie-artpec6.c is the only driver doing FTS configuration.

I will add the helper function call in pcie-artpec6.c in the next patch version.

Thanks that's very much appreciated.

Thanks,

Andrew Murray

Quoted 13 lines

Regards,
Dilip

Quoted 7 lines

Thanks,

Andrew Murray

Quoted 2 lines

Regards,
Dilip

#24 lore raw

Re: [PATCH v4 2/3] dwc: PCI: intel: PCIe RC controller driver

In reply to Dilip Kota
Quoted 13 lines · Dilip Kota

Add support to PCIe RC controller on Intel Gateway SoCs. PCIe controller is based of Synopsys DesignWare pci core.

Intel PCIe driver requires Upconfig support, fast training sequence configuration and link speed change. So adding the respective helper functions in the pcie DesignWare framework. It also programs hardware autonomous speed during speed configuration so defining it in pci_regs.h.

drivers/pci/controller/dwc/pcie-intel-gw.c new file +3 -0
+static void intel_pcie_link_setup(struct intel_pcie_port *lpp)
+{
+	u32 val;

drivers/pci/controller/dwc/pcie-intel-gw.c line 161

Quoted 10 lines
drivers/pci/controller/dwc/pcie-intel-gw.c new file +10 -0
+
+	val = pcie_rc_cfg_rd(lpp, PCIE_CAP_OFST + PCI_EXP_LNKCAP);
+	lpp->max_speed = FIELD_GET(PCI_EXP_LNKCAP_SLS, val);
+	lpp->max_width = FIELD_GET(PCI_EXP_LNKCAP_MLW, val);
+
+	val = pcie_rc_cfg_rd(lpp, PCIE_CAP_OFST + PCI_EXP_LNKCTL);
+
+	val &= ~(PCI_EXP_LNKCTL_LD | PCI_EXP_LNKCTL_ASPMC);
+	val |= (PCI_EXP_LNKSTA_SLC << 16) | PCI_EXP_LNKCTL_CCC |
+	       PCI_EXP_LNKCTL_RCB;

Link Control is only 16 bits wide, so "PCI_EXP_LNKSTA_SLC << 16" wouldn't make sense. But I guess you're writing a device-specific register that is not actually the Link Control as documented in PCIe r5.0, sec 7.5.3.7, even though the bits are similar?

Likewise, PCI_EXP_LNKCTL_RCB is RO for Root Ports, but maybe you're telling the device what it should advertise in its Link Control?

PCI_EXP_LNKCTL_CCC is RW. But doesn't it depend on the components on both ends of the link? Do you know what device is at the other end? I would have assumed that you'd have to start with CCC==0, which should be most conservative, then set CCC=1 only if you know both ends have a common clock.

Quoted 31 lines
drivers/pci/controller/dwc/pcie-intel-gw.c new file +30 -0
+	pcie_rc_cfg_wr(lpp, val, PCIE_CAP_OFST + PCI_EXP_LNKCTL);
+}
+
 
+static void intel_pcie_max_speed_setup(struct intel_pcie_port *lpp)
+{
+	u32 reg, val;
+
+	reg = pcie_rc_cfg_rd(lpp, PCIE_CAP_OFST + PCI_EXP_LNKCTL2);
+	switch (lpp->link_gen) {
+	case PCIE_LINK_SPEED_GEN1:
+		reg &= ~PCI_EXP_LNKCTL2_TLS;
+		reg |= PCI_EXP_LNKCTL2_HASD|
+			PCI_EXP_LNKCTL2_TLS_2_5GT;
+		break;
+	case PCIE_LINK_SPEED_GEN2:
+		reg &= ~PCI_EXP_LNKCTL2_TLS;
+		reg |= PCI_EXP_LNKCTL2_HASD|
+			PCI_EXP_LNKCTL2_TLS_5_0GT;
+		break;
+	case PCIE_LINK_SPEED_GEN3:
+		reg &= ~PCI_EXP_LNKCTL2_TLS;
+		reg |= PCI_EXP_LNKCTL2_HASD|
+			PCI_EXP_LNKCTL2_TLS_8_0GT;
+		break;
+	case PCIE_LINK_SPEED_GEN4:
+		reg &= ~PCI_EXP_LNKCTL2_TLS;
+		reg |= PCI_EXP_LNKCTL2_HASD|
+			PCI_EXP_LNKCTL2_TLS_16_0GT;
+		break;
+	default:

drivers/pci/controller/dwc/pcie-intel-gw.c line 201

Quoted 10 lines
drivers/pci/controller/dwc/pcie-intel-gw.c new file +10 -0
+		/* Use hardware capability */
+		val = pcie_rc_cfg_rd(lpp, PCIE_CAP_OFST + PCI_EXP_LNKCAP);
+		val = FIELD_GET(PCI_EXP_LNKCAP_SLS, val);
+		reg &= ~PCI_EXP_LNKCTL2_HASD;
+		reg |= val;
+		break;
+	}
+
+	pcie_rc_cfg_wr(lpp, reg, PCIE_CAP_OFST + PCI_EXP_LNKCTL2);
+	dw_pcie_link_set_n_fts(&lpp->pci, lpp->n_fts);

There are other users of of_pci_get_max_link_speed() that look sort of similar to this (dra7xx_pcie_establish_link(), ks_pcie_set_link_speed(), tegra_pcie_prepare_host()). Do these *need* to be different, or is there something that could be factored out?

drivers/pci/controller/dwc/pcie-designware.c int dw_pcie_link_up(struct dw_pcie *pci) · line 492

Quoted 4 lines
drivers/pci/controller/dwc/pcie-designware.c +4 -0
+}
+
+
+

Remove extra blank lines here.

drivers/pci/controller/dwc/pcie-intel-gw.c line 564

Quoted 7 lines
drivers/pci/controller/dwc/pcie-intel-gw.c new file +5 -0
+static void intel_pcie_port_logic_setup(struct intel_pcie_port *lpp)
...
 
+	/* Intel PCIe doesn't configure IO region, so configure
+	 * viewport to not to access IO region during register
+	 * read write operations.
+	 */

This comment doesn't describe the code. Is there supposed to be some code here that configures the viewport? Where do we tell the viewport not to access IO?

I guess maybe this means something like "tell dw_pcie_access_other_conf() not to program an outbound ATU for I/O"? I don't know that structure well enough to write this in a way that makes sense, but this code doesn't look like it's configuring any viewports.

Please use usual multi-line comment style, i.e.,

/*
* Intel PCIe ...
*/

Quoted 5 lines
drivers/pci/controller/dwc/pcie-intel-gw.c new file +5 -0
+	pci->num_viewport = data->num_viewport;
+	dev_info(dev, "Intel PCIe Root Complex Port %d init done\n", lpp->id);
+
+	return ret;
+}
author
#25 lore raw

Re: [PATCH v4 2/3] dwc: PCI: intel: PCIe RC controller driver

Hi Bjorn Helgaas,

Quoted 19 lines · Bjorn Helgaas
Quoted 18 lines · Dilip Kota

Add support to PCIe RC controller on Intel Gateway SoCs. PCIe controller is based of Synopsys DesignWare pci core.

Intel PCIe driver requires Upconfig support, fast training sequence configuration and link speed change. So adding the respective helper functions in the pcie DesignWare framework. It also programs hardware autonomous speed during speed configuration so defining it in pci_regs.h.

drivers/pci/controller/dwc/pcie-intel-gw.c new file +9 -0
+static void intel_pcie_link_setup(struct intel_pcie_port *lpp)
+{
+	u32 val;
+
+	val = pcie_rc_cfg_rd(lpp, PCIE_CAP_OFST + PCI_EXP_LNKCAP);
+	lpp->max_speed = FIELD_GET(PCI_EXP_LNKCAP_SLS, val);
+	lpp->max_width = FIELD_GET(PCI_EXP_LNKCAP_MLW, val);
+
+	val = pcie_rc_cfg_rd(lpp, PCIE_CAP_OFST + PCI_EXP_LNKCTL);
Quoted 8 lines

drivers/pci/controller/dwc/pcie-intel-gw.c line 161

Quoted 4 lines
drivers/pci/controller/dwc/pcie-intel-gw.c new file +4 -0
+
+	val &= ~(PCI_EXP_LNKCTL_LD | PCI_EXP_LNKCTL_ASPMC);
+	val |= (PCI_EXP_LNKSTA_SLC << 16) | PCI_EXP_LNKCTL_CCC |
+	       PCI_EXP_LNKCTL_RCB;

Link Control is only 16 bits wide, so "PCI_EXP_LNKSTA_SLC << 16" wouldn't make sense. But I guess you're writing a device-specific register that is not actually the Link Control as documented in PCIe r5.0, sec 7.5.3.7, even though the bits are similar?

It is not device specific.
You are correct about register size that pcie spec mentions PCIE_EXP_LNKCTL at 0x10 and PCIE_EXP_LNKSTS at 0x12 each of 2bytes. Accessing 4bytes at offset 0x10 ends up accessing LNK control and status register. In Synopsys PCIe controller LINK_CONTROL_LINK_STATUS_REG is of 4bytes size at offset 0x10,
In both the cases everything is same except the size of the register, so i used PCIE_EXP_LNKCTL macro which is already defined in pci_regs.h

Quoted 3 lines

Likewise, PCI_EXP_LNKCTL_RCB is RO for Root Ports, but maybe you're telling the device what it should advertise in its Link Control?

You are correct, PCI_EXP_LNKCTL_RCB is RO. I will remove it.

Quoted 6 lines

PCI_EXP_LNKCTL_CCC is RW. But doesn't it depend on the components on both ends of the link? Do you know what device is at the other end? I would have assumed that you'd have to start with CCC==0, which should be most conservative, then set CCC=1 only if you know both ends have a common clock.

PCIe RC and endpoint device are having the common clock so set the CCC=1.

Quoted 37 lines
Quoted 37 lines
drivers/pci/controller/dwc/pcie-intel-gw.c new file +36 -0
+	pcie_rc_cfg_wr(lpp, val, PCIE_CAP_OFST + PCI_EXP_LNKCTL);
+}
+
+static void intel_pcie_max_speed_setup(struct intel_pcie_port *lpp)
+{
+	u32 reg, val;
+
+	reg = pcie_rc_cfg_rd(lpp, PCIE_CAP_OFST + PCI_EXP_LNKCTL2);
+	switch (lpp->link_gen) {
+	case PCIE_LINK_SPEED_GEN1:
+		reg &= ~PCI_EXP_LNKCTL2_TLS;
+		reg |= PCI_EXP_LNKCTL2_HASD|
+			PCI_EXP_LNKCTL2_TLS_2_5GT;
+		break;
+	case PCIE_LINK_SPEED_GEN2:
+		reg &= ~PCI_EXP_LNKCTL2_TLS;
+		reg |= PCI_EXP_LNKCTL2_HASD|
+			PCI_EXP_LNKCTL2_TLS_5_0GT;
+		break;
+	case PCIE_LINK_SPEED_GEN3:
+		reg &= ~PCI_EXP_LNKCTL2_TLS;
+		reg |= PCI_EXP_LNKCTL2_HASD|
+			PCI_EXP_LNKCTL2_TLS_8_0GT;
+		break;
+	case PCIE_LINK_SPEED_GEN4:
+		reg &= ~PCI_EXP_LNKCTL2_TLS;
+		reg |= PCI_EXP_LNKCTL2_HASD|
+			PCI_EXP_LNKCTL2_TLS_16_0GT;
+		break;
+	default:
+		/* Use hardware capability */
+		val = pcie_rc_cfg_rd(lpp, PCIE_CAP_OFST + PCI_EXP_LNKCAP);
+		val = FIELD_GET(PCI_EXP_LNKCAP_SLS, val);
+		reg &= ~PCI_EXP_LNKCTL2_HASD;
+		reg |= val;
+		break;
Quoted 8 lines

drivers/pci/controller/dwc/pcie-intel-gw.c line 201

Quoted 4 lines
drivers/pci/controller/dwc/pcie-intel-gw.c new file +4 -0
+	}
+
+	pcie_rc_cfg_wr(lpp, reg, PCIE_CAP_OFST + PCI_EXP_LNKCTL2);
+	dw_pcie_link_set_n_fts(&lpp->pci, lpp->n_fts);

There are other users of of_pci_get_max_link_speed() that look sort of similar to this (dra7xx_pcie_establish_link(), ks_pcie_set_link_speed(), tegra_pcie_prepare_host()). Do these *need* to be different, or is there something that could be factored out?

dra7xx_pcie_establish_link() and ks_pcie_set_link_speed() are doing speed configuration for GEN1 and GEN1 &2 respectively.

intel_pcie_max_speed_setup() can be moved to dwc framework and dra7xx and ks_pcie drivers can call.

Quoted 6 lines

drivers/pci/controller/dwc/pcie-designware.c int dw_pcie_link_up(struct dw_pcie *pci) · line 492

Quoted 5 lines
drivers/pci/controller/dwc/pcie-designware.c +4 -0
+}
+
+
+

Remove extra blank lines here.

i will remove it.

Quoted 11 lines

drivers/pci/controller/dwc/pcie-intel-gw.c line 564

Quoted 7 lines
drivers/pci/controller/dwc/pcie-intel-gw.c new file +5 -0
+static void intel_pcie_port_logic_setup(struct intel_pcie_port *lpp)
...
+	/* Intel PCIe doesn't configure IO region, so configure
+	 * viewport to not to access IO region during register
+	 * read write operations.
+	 */

This comment doesn't describe the code. Is there supposed to be some code here that configures the viewport? Where do we tell the viewport not to access IO?

Quoted 5 lines

I guess maybe this means something like "tell dw_pcie_access_other_conf() not to program an outbound ATU for I/O"? I don't know that structure well enough to write this in a way that makes sense, but this code doesn't look like it's configuring any viewports.

Yes, you are correct. Telling not to program an outbound ATU for IO. I will update the description.

Quoted 5 lines

Please use usual multi-line comment style, i.e.,

   /*
    * Intel PCIe ...
    */

Sure, i will correct it.

Thanks for reviewing and providing the valuable inputs! Regards,
Dilip

Quoted 5 lines
Quoted 5 lines
drivers/pci/controller/dwc/pcie-intel-gw.c new file +5 -0
+	pci->num_viewport = data->num_viewport;
+	dev_info(dev, "Intel PCIe Root Complex Port %d init done\n", lpp->id);
+
+	return ret;
+}
#26 lore raw

Re: [PATCH v4 2/3] dwc: PCI: intel: PCIe RC controller driver

Quoted 30 lines · Dilip Kota
Quoted 21 lines · Bjorn Helgaas
Quoted 20 lines · Dilip Kota

Add support to PCIe RC controller on Intel Gateway SoCs. PCIe controller is based of Synopsys DesignWare pci core.

Intel PCIe driver requires Upconfig support, fast training sequence configuration and link speed change. So adding the respective helper functions in the pcie DesignWare framework. It also programs hardware autonomous speed during speed configuration so defining it in pci_regs.h.

drivers/pci/controller/dwc/pcie-intel-gw.c new file +11 -0
+static void intel_pcie_link_setup(struct intel_pcie_port *lpp)
+{
+	u32 val;
+
+	val = pcie_rc_cfg_rd(lpp, PCIE_CAP_OFST + PCI_EXP_LNKCAP);
+	lpp->max_speed = FIELD_GET(PCI_EXP_LNKCAP_SLS, val);
+	lpp->max_width = FIELD_GET(PCI_EXP_LNKCAP_MLW, val);
+
+	val = pcie_rc_cfg_rd(lpp, PCIE_CAP_OFST + PCI_EXP_LNKCTL);
+
+	val &= ~(PCI_EXP_LNKCTL_LD | PCI_EXP_LNKCTL_ASPMC);
Quoted 8 lines

drivers/pci/controller/dwc/pcie-intel-gw.c line 161

Quoted 2 lines
drivers/pci/controller/dwc/pcie-intel-gw.c new file +2 -0
+	val |= (PCI_EXP_LNKSTA_SLC << 16) | PCI_EXP_LNKCTL_CCC |
+	       PCI_EXP_LNKCTL_RCB;

PCI_EXP_LNKCTL_CCC is RW. But doesn't it depend on the components on both ends of the link? Do you know what device is at the other end? I would have assumed that you'd have to start with CCC==0, which should be most conservative, then set CCC=1 only if you know both ends have a common clock.

Quoted 1 line

PCIe RC and endpoint device are having the common clock so set the CCC=1.

How do you know what the endpoint device is? Is this driver only for a specific embedded configuration where the endpoint is always soldered down? There's no possibility of this RC being used with a connector?

Shouldn't this be either discoverable or configurable via DT or something? pcie_aspm_configure_common_clock() seems to do something similar, but I can't really vouch for its correctness.

Bjorn

author
#27 lore raw

Re: [PATCH v4 2/3] dwc: PCI: intel: PCIe RC controller driver

Quoted 31 lines · Bjorn Helgaas
Quoted 29 lines · Dilip Kota
Quoted 20 lines · Bjorn Helgaas
Quoted 19 lines · Dilip Kota

Add support to PCIe RC controller on Intel Gateway SoCs. PCIe controller is based of Synopsys DesignWare pci core.

Intel PCIe driver requires Upconfig support, fast training sequence configuration and link speed change. So adding the respective helper functions in the pcie DesignWare framework. It also programs hardware autonomous speed during speed configuration so defining it in pci_regs.h.

drivers/pci/controller/dwc/pcie-intel-gw.c new file +10 -0
+static void intel_pcie_link_setup(struct intel_pcie_port *lpp)
+{
+	u32 val;
+
+	val = pcie_rc_cfg_rd(lpp, PCIE_CAP_OFST + PCI_EXP_LNKCAP);
+	lpp->max_speed = FIELD_GET(PCI_EXP_LNKCAP_SLS, val);
+	lpp->max_width = FIELD_GET(PCI_EXP_LNKCAP_MLW, val);
+
+	val = pcie_rc_cfg_rd(lpp, PCIE_CAP_OFST + PCI_EXP_LNKCTL);
+
Quoted 8 lines

drivers/pci/controller/dwc/pcie-intel-gw.c line 161

Quoted 3 lines
drivers/pci/controller/dwc/pcie-intel-gw.c new file +3 -0
+	val &= ~(PCI_EXP_LNKCTL_LD | PCI_EXP_LNKCTL_ASPMC);
+	val |= (PCI_EXP_LNKSTA_SLC << 16) | PCI_EXP_LNKCTL_CCC |
+	       PCI_EXP_LNKCTL_RCB;

PCI_EXP_LNKCTL_CCC is RW. But doesn't it depend on the components on both ends of the link? Do you know what device is at the other end? I would have assumed that you'd have to start with CCC==0, which should be most conservative, then set CCC=1 only if you know both ends have a common clock.

Quoted 1 line

PCIe RC and endpoint device are having the common clock so set the CCC=1.

Quoted 8 lines

How do you know what the endpoint device is? Is this driver only for a specific embedded configuration where the endpoint is always soldered down? There's no possibility of this RC being used with a connector?

Shouldn't this be either discoverable or configurable via DT or something? pcie_aspm_configure_common_clock() seems to do something similar, but I can't really vouch for its correctness.

(sorry for the late reply, i am back today from sick leave)

I see pcie_aspm_configure_common_clock() is getting called during pcie root bus bridge scanning and programming the CCC.

So, CCC configuration can be removed here in intel_pcie_link_setup().

Regards,
Dilip

Quoted 2 lines

Bjorn

#28 lore raw

Re: [PATCH v4 2/3] dwc: PCI: intel: PCIe RC controller driver

In reply to Dilip Kota
Thank you for the patch! Yet something to improve:

Hi Dilip,

Thank you for the patch! Yet something to improve:

[auto build test ERROR on pci/next]
[cannot apply to v5.4-rc4 next-20191023]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please see https://stackoverflow.com/a/37406982]

url:    https://github.com/0day-ci/linux/commits/Dilip-Kota/PCI-Add-Intel-PCIe-Driver-and-respective-dt-binding-yaml-file/20191024-103204
base:   https://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci.git next
config: sparc64-allmodconfig (attached as .config)
compiler: sparc64-linux-gcc (GCC) 7.4.0

reproduce:
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
GCC_VERSION=7.4.0 make.cross ARCH=sparc64

If you fix the issue, kindly add following tag

All error/warnings (new ones prefixed by >>):

>> drivers/pci/controller/dwc/pcie-designware-host.c:72:15: error: variable 'dw_pcie_msi_domain_info' has initializer but incomplete type
    static struct msi_domain_info dw_pcie_msi_domain_info = {
                  ^~~~~~~~~~~~~~~
>> drivers/pci/controller/dwc/pcie-designware-host.c:73:3: error: 'struct msi_domain_info' has no member named 'flags'
     .flags = (MSI_FLAG_USE_DEF_DOM_OPS | MSI_FLAG_USE_DEF_CHIP_OPS |
      ^~~~~
>> drivers/pci/controller/dwc/pcie-designware-host.c:73:12: error: 'MSI_FLAG_USE_DEF_DOM_OPS' undeclared here (not in a function); did you mean 'SIMPLE_DEV_PM_OPS'?
     .flags = (MSI_FLAG_USE_DEF_DOM_OPS | MSI_FLAG_USE_DEF_CHIP_OPS |
               ^~~~~~~~~~~~~~~~~~~~~~~~
               SIMPLE_DEV_PM_OPS
>> drivers/pci/controller/dwc/pcie-designware-host.c:73:39: error: 'MSI_FLAG_USE_DEF_CHIP_OPS' undeclared here (not in a function); did you mean 'MSI_FLAG_USE_DEF_DOM_OPS'?
     .flags = (MSI_FLAG_USE_DEF_DOM_OPS | MSI_FLAG_USE_DEF_CHIP_OPS |
                                          ^~~~~~~~~~~~~~~~~~~~~~~~~
                                          MSI_FLAG_USE_DEF_DOM_OPS
>> drivers/pci/controller/dwc/pcie-designware-host.c:74:6: error: 'MSI_FLAG_PCI_MSIX' undeclared here (not in a function); did you mean 'CONFIG_PCI_MSI'?
         MSI_FLAG_PCI_MSIX | MSI_FLAG_MULTI_PCI_MSI),
         ^~~~~~~~~~~~~~~~~
         CONFIG_PCI_MSI
>> drivers/pci/controller/dwc/pcie-designware-host.c:74:26: error: 'MSI_FLAG_MULTI_PCI_MSI' undeclared here (not in a function); did you mean 'MSI_FLAG_PCI_MSIX'?
         MSI_FLAG_PCI_MSIX | MSI_FLAG_MULTI_PCI_MSI),
                             ^~~~~~~~~~~~~~~~~~~~~~
                             MSI_FLAG_PCI_MSIX
>> drivers/pci/controller/dwc/pcie-designware-host.c:73:11: warning: excess elements in struct initializer
     .flags = (MSI_FLAG_USE_DEF_DOM_OPS | MSI_FLAG_USE_DEF_CHIP_OPS |
              ^
   drivers/pci/controller/dwc/pcie-designware-host.c:73:11: note: (near initialization for 'dw_pcie_msi_domain_info')
>> drivers/pci/controller/dwc/pcie-designware-host.c:75:3: error: 'struct msi_domain_info' has no member named 'chip'
     .chip = &dw_pcie_msi_irq_chip,
      ^~~~
   drivers/pci/controller/dwc/pcie-designware-host.c:75:10: warning: excess elements in struct initializer
     .chip = &dw_pcie_msi_irq_chip,
             ^
   drivers/pci/controller/dwc/pcie-designware-host.c:75:10: note: (near initialization for 'dw_pcie_msi_domain_info')
   drivers/pci/controller/dwc/pcie-designware-host.c: In function 'dw_pcie_allocate_domains':
>> drivers/pci/controller/dwc/pcie-designware-host.c:266:19: error: implicit declaration of function 'pci_msi_create_irq_domain'; did you mean 'pci_msi_get_device_domain'? [-Werror=implicit-function-declaration]
     pp->msi_domain = pci_msi_create_irq_domain(fwnode,
                      ^~~~~~~~~~~~~~~~~~~~~~~~~
                      pci_msi_get_device_domain
>> drivers/pci/controller/dwc/pcie-designware-host.c:266:17: warning: assignment makes pointer from integer without a cast [-Wint-conversion]
     pp->msi_domain = pci_msi_create_irq_domain(fwnode,
                    ^
   drivers/pci/controller/dwc/pcie-designware-host.c: At top level:
>> drivers/pci/controller/dwc/pcie-designware-host.c:72:31: error: storage size of 'dw_pcie_msi_domain_info' isn't known
    static struct msi_domain_info dw_pcie_msi_domain_info = {
                                  ^~~~~~~~~~~~~~~~~~~~~~~

cc1: some warnings being treated as errors

vim +/dw_pcie_msi_domain_info +72 drivers/pci/controller/dwc/pcie-designware-host.c

7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06   71  
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  @72  static struct msi_domain_info dw_pcie_msi_domain_info = {
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  @73  	.flags	= (MSI_FLAG_USE_DEF_DOM_OPS | MSI_FLAG_USE_DEF_CHIP_OPS |
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  @74  		   MSI_FLAG_PCI_MSIX | MSI_FLAG_MULTI_PCI_MSI),
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  @75  	.chip	= &dw_pcie_msi_irq_chip,
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15   76  };
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15   77  
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15   78  /* MSI int handler */
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15   79  irqreturn_t dw_handle_msi_irq(struct pcie_port *pp)
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15   80  {
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15   81  	int i, pos, irq;
1f319cb0538a10 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06   82  	u32 val, num_ctrls;
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15   83  	irqreturn_t ret = IRQ_NONE;
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15   84  
1f319cb0538a10 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06   85  	num_ctrls = pp->num_vectors / MAX_MSI_IRQS_PER_CTRL;
1f319cb0538a10 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06   86  
1f319cb0538a10 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06   87  	for (i = 0; i < num_ctrls; i++) {
76cbf066b1ab75 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-05-14   88  		dw_pcie_rd_own_conf(pp, PCIE_MSI_INTR0_STATUS +
76cbf066b1ab75 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-05-14   89  					(i * MSI_REG_CTRL_BLOCK_SIZE),
76cbf066b1ab75 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-05-14   90  				    4, &val);
dbe4a09e8bbcf8 drivers/pci/dwc/pcie-designware-host.c            Bjorn Helgaas          2017-03-16   91  		if (!val)
dbe4a09e8bbcf8 drivers/pci/dwc/pcie-designware-host.c            Bjorn Helgaas          2017-03-16   92  			continue;
dbe4a09e8bbcf8 drivers/pci/dwc/pcie-designware-host.c            Bjorn Helgaas          2017-03-16   93  
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15   94  		ret = IRQ_HANDLED;
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15   95  		pos = 0;
76cbf066b1ab75 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-05-14   96  		while ((pos = find_next_bit((unsigned long *) &val,
76cbf066b1ab75 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-05-14   97  					    MAX_MSI_IRQS_PER_CTRL,
76cbf066b1ab75 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-05-14   98  					    pos)) != MAX_MSI_IRQS_PER_CTRL) {
76cbf066b1ab75 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-05-14   99  			irq = irq_find_mapping(pp->irq_domain,
76cbf066b1ab75 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-05-14  100  					       (i * MAX_MSI_IRQS_PER_CTRL) +
76cbf066b1ab75 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-05-14  101  					       pos);
8c934095fa2f33 drivers/pci/dwc/pcie-designware-host.c            Faiz Abbas             2017-08-10  102  			generic_handle_irq(irq);
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  103  			pos++;
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  104  		}
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  105  	}
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  106  
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  107  	return ret;
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  108  }
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  109  
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  110  /* Chained MSI interrupt service routine */
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  111  static void dw_chained_msi_isr(struct irq_desc *desc)
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  112  {
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  113  	struct irq_chip *chip = irq_desc_get_chip(desc);
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  114  	struct pcie_port *pp;
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  115  
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  116  	chained_irq_enter(chip, desc);
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  117  
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  118  	pp = irq_desc_get_handler_data(desc);
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  119  	dw_handle_msi_irq(pp);
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  120  
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  121  	chained_irq_exit(chip, desc);
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  122  }
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  123  
59ea68b3f17294 drivers/pci/controller/dwc/pcie-designware-host.c Gustavo Pimentel       2019-01-31  124  static void dw_pci_setup_msi_msg(struct irq_data *d, struct msi_msg *msg)
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  125  {
59ea68b3f17294 drivers/pci/controller/dwc/pcie-designware-host.c Gustavo Pimentel       2019-01-31  126  	struct pcie_port *pp = irq_data_get_irq_chip_data(d);
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  127  	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  128  	u64 msi_target;
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  129  
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  130  	msi_target = (u64)pp->msi_data;
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  131  
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  132  	msg->address_lo = lower_32_bits(msi_target);
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  133  	msg->address_hi = upper_32_bits(msi_target);
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  134  
59ea68b3f17294 drivers/pci/controller/dwc/pcie-designware-host.c Gustavo Pimentel       2019-01-31  135  	msg->data = d->hwirq;
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  136  
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  137  	dev_dbg(pci->dev, "msi#%d address_hi %#x address_lo %#x\n",
59ea68b3f17294 drivers/pci/controller/dwc/pcie-designware-host.c Gustavo Pimentel       2019-01-31  138  		(int)d->hwirq, msg->address_hi, msg->address_lo);
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  139  }
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  140  
fd5288a362ab55 drivers/pci/controller/dwc/pcie-designware-host.c Gustavo Pimentel       2019-01-31  141  static int dw_pci_msi_set_affinity(struct irq_data *d,
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  142  				   const struct cpumask *mask, bool force)
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  143  {
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  144  	return -EINVAL;
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  145  }
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  146  
40e9892ef94ce8 drivers/pci/controller/dwc/pcie-designware-host.c Gustavo Pimentel       2019-01-31  147  static void dw_pci_bottom_mask(struct irq_data *d)
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  148  {
40e9892ef94ce8 drivers/pci/controller/dwc/pcie-designware-host.c Gustavo Pimentel       2019-01-31  149  	struct pcie_port *pp = irq_data_get_irq_chip_data(d);
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  150  	unsigned int res, bit, ctrl;
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  151  	unsigned long flags;
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  152  
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  153  	raw_spin_lock_irqsave(&pp->lock, flags);
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  154  
40e9892ef94ce8 drivers/pci/controller/dwc/pcie-designware-host.c Gustavo Pimentel       2019-01-31  155  	ctrl = d->hwirq / MAX_MSI_IRQS_PER_CTRL;
76cbf066b1ab75 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-05-14  156  	res = ctrl * MSI_REG_CTRL_BLOCK_SIZE;
40e9892ef94ce8 drivers/pci/controller/dwc/pcie-designware-host.c Gustavo Pimentel       2019-01-31  157  	bit = d->hwirq % MAX_MSI_IRQS_PER_CTRL;
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  158  
657722570a555c drivers/pci/controller/dwc/pcie-designware-host.c Gustavo Pimentel       2019-01-31  159  	pp->irq_mask[ctrl] |= BIT(bit);
830920e065e90d drivers/pci/controller/dwc/pcie-designware-host.c Marc Zyngier           2018-11-13  160  	dw_pcie_wr_own_conf(pp, PCIE_MSI_INTR0_MASK + res, 4,
a348d015f0de7a drivers/pci/controller/dwc/pcie-designware-host.c Gustavo Pimentel       2019-01-31  161  			    pp->irq_mask[ctrl]);
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  162  
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  163  	raw_spin_unlock_irqrestore(&pp->lock, flags);
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  164  }
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  165  
40e9892ef94ce8 drivers/pci/controller/dwc/pcie-designware-host.c Gustavo Pimentel       2019-01-31  166  static void dw_pci_bottom_unmask(struct irq_data *d)
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  167  {
40e9892ef94ce8 drivers/pci/controller/dwc/pcie-designware-host.c Gustavo Pimentel       2019-01-31  168  	struct pcie_port *pp = irq_data_get_irq_chip_data(d);
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  169  	unsigned int res, bit, ctrl;
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  170  	unsigned long flags;
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  171  
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  172  	raw_spin_lock_irqsave(&pp->lock, flags);
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  173  
40e9892ef94ce8 drivers/pci/controller/dwc/pcie-designware-host.c Gustavo Pimentel       2019-01-31  174  	ctrl = d->hwirq / MAX_MSI_IRQS_PER_CTRL;
76cbf066b1ab75 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-05-14  175  	res = ctrl * MSI_REG_CTRL_BLOCK_SIZE;
40e9892ef94ce8 drivers/pci/controller/dwc/pcie-designware-host.c Gustavo Pimentel       2019-01-31  176  	bit = d->hwirq % MAX_MSI_IRQS_PER_CTRL;
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  177  
657722570a555c drivers/pci/controller/dwc/pcie-designware-host.c Gustavo Pimentel       2019-01-31  178  	pp->irq_mask[ctrl] &= ~BIT(bit);
830920e065e90d drivers/pci/controller/dwc/pcie-designware-host.c Marc Zyngier           2018-11-13  179  	dw_pcie_wr_own_conf(pp, PCIE_MSI_INTR0_MASK + res, 4,
a348d015f0de7a drivers/pci/controller/dwc/pcie-designware-host.c Gustavo Pimentel       2019-01-31  180  			    pp->irq_mask[ctrl]);
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  181  
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  182  	raw_spin_unlock_irqrestore(&pp->lock, flags);
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  183  }
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  184  
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  185  static void dw_pci_bottom_ack(struct irq_data *d)
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  186  {
3f7bb2ec20ce07 drivers/pci/controller/dwc/pcie-designware-host.c Marc Zyngier           2018-11-13  187  	struct pcie_port *pp  = irq_data_get_irq_chip_data(d);
3f7bb2ec20ce07 drivers/pci/controller/dwc/pcie-designware-host.c Marc Zyngier           2018-11-13  188  	unsigned int res, bit, ctrl;
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  189  
3f7bb2ec20ce07 drivers/pci/controller/dwc/pcie-designware-host.c Marc Zyngier           2018-11-13  190  	ctrl = d->hwirq / MAX_MSI_IRQS_PER_CTRL;
3f7bb2ec20ce07 drivers/pci/controller/dwc/pcie-designware-host.c Marc Zyngier           2018-11-13  191  	res = ctrl * MSI_REG_CTRL_BLOCK_SIZE;
3f7bb2ec20ce07 drivers/pci/controller/dwc/pcie-designware-host.c Marc Zyngier           2018-11-13  192  	bit = d->hwirq % MAX_MSI_IRQS_PER_CTRL;
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  193  
657722570a555c drivers/pci/controller/dwc/pcie-designware-host.c Gustavo Pimentel       2019-01-31  194  	dw_pcie_wr_own_conf(pp, PCIE_MSI_INTR0_STATUS + res, 4, BIT(bit));
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  195  }
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  196  
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  197  static struct irq_chip dw_pci_msi_bottom_irq_chip = {
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  198  	.name = "DWPCI-MSI",
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  199  	.irq_ack = dw_pci_bottom_ack,
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  200  	.irq_compose_msi_msg = dw_pci_setup_msi_msg,
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  201  	.irq_set_affinity = dw_pci_msi_set_affinity,
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  202  	.irq_mask = dw_pci_bottom_mask,
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  203  	.irq_unmask = dw_pci_bottom_unmask,
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  204  };
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  205  
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  206  static int dw_pcie_irq_domain_alloc(struct irq_domain *domain,
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  207  				    unsigned int virq, unsigned int nr_irqs,
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  208  				    void *args)
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  209  {
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  210  	struct pcie_port *pp = domain->host_data;
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  211  	unsigned long flags;
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  212  	u32 i;
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  213  	int bit;
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  214  
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  215  	raw_spin_lock_irqsave(&pp->lock, flags);
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  216  
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  217  	bit = bitmap_find_free_region(pp->msi_irq_in_use, pp->num_vectors,
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  218  				      order_base_2(nr_irqs));
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  219  
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  220  	raw_spin_unlock_irqrestore(&pp->lock, flags);
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  221  
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  222  	if (bit < 0)
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  223  		return -ENOSPC;
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  224  
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  225  	for (i = 0; i < nr_irqs; i++)
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  226  		irq_domain_set_info(domain, virq + i, bit + i,
9f67437b3a0858 drivers/pci/controller/dwc/pcie-designware-host.c Kishon Vijay Abraham I 2019-03-21  227  				    pp->msi_irq_chip,
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  228  				    pp, handle_edge_irq,
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  229  				    NULL, NULL);
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  230  
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  231  	return 0;
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  232  }
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  233  
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  234  static void dw_pcie_irq_domain_free(struct irq_domain *domain,
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  235  				    unsigned int virq, unsigned int nr_irqs)
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  236  {
4cfae0f1f8ce16 drivers/pci/controller/dwc/pcie-designware-host.c Gustavo Pimentel       2019-01-31  237  	struct irq_data *d = irq_domain_get_irq_data(domain, virq);
4cfae0f1f8ce16 drivers/pci/controller/dwc/pcie-designware-host.c Gustavo Pimentel       2019-01-31  238  	struct pcie_port *pp = irq_data_get_irq_chip_data(d);
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  239  	unsigned long flags;
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  240  
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  241  	raw_spin_lock_irqsave(&pp->lock, flags);
b4a8a51caf7de4 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-05-14  242  
4cfae0f1f8ce16 drivers/pci/controller/dwc/pcie-designware-host.c Gustavo Pimentel       2019-01-31  243  	bitmap_release_region(pp->msi_irq_in_use, d->hwirq,
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  244  			      order_base_2(nr_irqs));
b4a8a51caf7de4 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-05-14  245  
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  246  	raw_spin_unlock_irqrestore(&pp->lock, flags);
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  247  }
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  248  
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  249  static const struct irq_domain_ops dw_pcie_msi_domain_ops = {
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  250  	.alloc	= dw_pcie_irq_domain_alloc,
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  251  	.free	= dw_pcie_irq_domain_free,
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  252  };
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  253  
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  254  int dw_pcie_allocate_domains(struct pcie_port *pp)
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  255  {
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  256  	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  257  	struct fwnode_handle *fwnode = of_node_to_fwnode(pci->dev->of_node);
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  258  
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  259  	pp->irq_domain = irq_domain_create_linear(fwnode, pp->num_vectors,
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  260  					       &dw_pcie_msi_domain_ops, pp);
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  261  	if (!pp->irq_domain) {
b4a8a51caf7de4 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-05-14  262  		dev_err(pci->dev, "Failed to create IRQ domain\n");
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  263  		return -ENOMEM;
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  264  	}
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  265  
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06 @266  	pp->msi_domain = pci_msi_create_irq_domain(fwnode,
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  267  						   &dw_pcie_msi_domain_info,
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  268  						   pp->irq_domain);
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  269  	if (!pp->msi_domain) {
b4a8a51caf7de4 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-05-14  270  		dev_err(pci->dev, "Failed to create MSI domain\n");
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  271  		irq_domain_remove(pp->irq_domain);
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  272  		return -ENOMEM;
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel       2018-03-06  273  	}
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  274  
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  275  	return 0;
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  276  }
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I 2017-02-15  277  

:::::: The code at line 72 was first introduced by commit
:::::: 7c5925afbc58c6d6b384e1dc051bb992969bf787 PCI: dwc: Move MSI IRQs allocation to IRQ domains hierarchical API

:::::: TO: Gustavo Pimentel <[email protected]> :::::: CC: Lorenzo Pieralisi <[email protected]>

---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation

#29 lore raw

Re: [PATCH v4 2/3] dwc: PCI: intel: PCIe RC controller driver

In reply to Dilip Kota
Thank you for the patch! Perhaps something to improve:

Hi Dilip,

Thank you for the patch! Perhaps something to improve:

[auto build test WARNING on pci/next]
[cannot apply to v5.4-rc4 next-20191025]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please see https://stackoverflow.com/a/37406982]

url:    https://github.com/0day-ci/linux/commits/Dilip-Kota/PCI-Add-Intel-PCIe-Driver-and-respective-dt-binding-yaml-file/20191024-103204
base:   https://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci.git next
reproduce:
        # apt-get install sparse
        # sparse version: v0.6.1-dirty
        make ARCH=x86_64 allmodconfig
        make C=1 CF='-fdiagnostic-prefix -D__CHECK_ENDIAN__'

If you fix the issue, kindly add following tag

sparse warnings: (new ones prefixed by >>)

>> drivers/pci/controller/dwc/pcie-intel-gw.c:497:5: sparse: sparse: symbol 'intel_pcie_msi_init' was not declared. Should it be static?
>> drivers/pci/controller/dwc/pcie-intel-gw.c:503:5: sparse: sparse: symbol 'intel_pcie_cpu_addr' was not declared. Should it be static?

Please review and possibly fold the followup patch.

---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation

#30 lore raw

[RFC PATCH] dwc: PCI: intel: intel_pcie_msi_init() can be static

In reply to Dilip Kota
Fixes: 3686a0f46840 ("dwc: PCI: intel: PCIe RC controller driver")
Fixes 3686a0f46840 ("dwc: PCI: intel: PCIe RC controller driver")
---
 pcie-intel-gw.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/pci/controller/dwc/pcie-intel-gw.c +2 -2
index 9142c70db8085..352d0d03171b2 100644
@@ -494,13 +494,13 @@ static int intel_pcie_rc_init(struct pcie_port *pp)
 	return intel_pcie_host_setup(lpp);
 }
 
-int intel_pcie_msi_init(struct pcie_port *pp)
+static int intel_pcie_msi_init(struct pcie_port *pp)
 {
 	/* PCIe MSI/MSIx is handled by MSI in x86 processor */
 	return 0;
 }
 
-u64 intel_pcie_cpu_addr(struct dw_pcie *pcie, u64 cpu_addr)
+static u64 intel_pcie_cpu_addr(struct dw_pcie *pcie, u64 cpu_addr)
 {
 	return cpu_addr + BUS_IATU_OFFS;
 }
author
#31 lore raw

PATCH v4 3/3 pci: intel: Add sysfs attributes to configure pcie link

In reply to Dilip Kota

PCIe RC driver on Intel Gateway SoCs have a requirement of changing link width and speed on the fly.
So add the sysfs attributes to show and store the link properties.
Add the respective link resize function in pcie DesignWare framework so that Intel PCIe driver can use during link width configuration on the fly.

---
 drivers/pci/controller/dwc/pcie-designware.c |   9 +++
 drivers/pci/controller/dwc/pcie-designware.h |   3 +
 drivers/pci/controller/dwc/pcie-intel-gw.c   | 112 ++++++++++++++++++++++++++-
 3 files changed, 123 insertions(+), 1 deletion(-)
drivers/pci/controller/dwc/pcie-designware.c +9 -0
index 4c391bfd681a..662fdcb4f2d6 100644
@@ -474,6 +474,15 @@ int dw_pcie_link_up(struct dw_pcie *pci)
 		(!(val & PCIE_PORT_DEBUG1_LINK_IN_TRAINING)));
 }
 
+void dw_pcie_link_width_resize(struct dw_pcie *pci, u32 lane_width)
+{
+	u32 val;
+
+	val =  dw_pcie_readl_dbi(pci, PCIE_PORT_MULTI_LANE_CTRL);
+	val &= ~(PORT_MLTI_LNK_WDTH_CHNG | PORT_MLTI_LNK_WDTH);
+	val |= PORT_MLTI_LNK_WDTH_CHNG | lane_width;
+	dw_pcie_writel_dbi(pci, PCIE_PORT_MULTI_LANE_CTRL, val);
+}
 
 void dw_pcie_upconfig_setup(struct dw_pcie *pci)
 {
drivers/pci/controller/dwc/pcie-designware.h +3 -0
index 3beac10e4a4c..fcf0442341fd 100644
@@ -67,6 +67,8 @@
 #define PCIE_MSI_INTR0_STATUS		0x830
 
 #define PCIE_PORT_MULTI_LANE_CTRL	0x8C0
+#define PORT_MLTI_LNK_WDTH		GENMASK(5, 0)
+#define PORT_MLTI_LNK_WDTH_CHNG		BIT(6)
 #define PORT_MLTI_UPCFG_SUPPORT		BIT(7)
 
 #define PCIE_ATU_VIEWPORT		0x900
@@ -282,6 +284,7 @@ void dw_pcie_write_dbi2(struct dw_pcie *pci, u32 reg, size_t size, u32 val);
 u32 dw_pcie_read_atu(struct dw_pcie *pci, u32 reg, size_t size);
 void dw_pcie_write_atu(struct dw_pcie *pci, u32 reg, size_t size, u32 val);
 int dw_pcie_link_up(struct dw_pcie *pci);
+void dw_pcie_link_width_resize(struct dw_pcie *pci, u32 lane_width);
 void dw_pcie_upconfig_setup(struct dw_pcie *pci);
 void dw_pcie_link_speed_change(struct dw_pcie *pci, bool enable);
 void dw_pcie_link_set_n_fts(struct dw_pcie *pci, u32 n_fts);
drivers/pci/controller/dwc/pcie-intel-gw.c +111 -1

131 lines not on this page · raw message

Signature · 1 line
2.11.0
#32 lore raw

RE: [PATCH v4 3/3] pci: intel: Add sysfs attributes to configure pcie link

Quoted 94 lines · Dilip Kota

PCIe RC driver on Intel Gateway SoCs have a requirement of changing link width and speed on the fly.
So add the sysfs attributes to show and store the link properties.
Add the respective link resize function in pcie DesignWare framework so that Intel PCIe driver can use during link width configuration on the fly.

---
 drivers/pci/controller/dwc/pcie-designware.c |   9 +++
 drivers/pci/controller/dwc/pcie-designware.h |   3 +
 drivers/pci/controller/dwc/pcie-intel-gw.c   | 112 ++++++++++++++++++++++++++-
 3 files changed, 123 insertions(+), 1 deletion(-)
drivers/pci/controller/dwc/pcie-designware.c +9 -0
index 4c391bfd681a..662fdcb4f2d6 100644
@@ -474,6 +474,15 @@ int dw_pcie_link_up(struct dw_pcie *pci)
 		(!(val & PCIE_PORT_DEBUG1_LINK_IN_TRAINING)));
 }
 
+void dw_pcie_link_width_resize(struct dw_pcie *pci, u32 lane_width)
+{
+	u32 val;
+
+	val =  dw_pcie_readl_dbi(pci, PCIE_PORT_MULTI_LANE_CTRL);
+	val &= ~(PORT_MLTI_LNK_WDTH_CHNG | PORT_MLTI_LNK_WDTH);
+	val |= PORT_MLTI_LNK_WDTH_CHNG | lane_width;
+	dw_pcie_writel_dbi(pci, PCIE_PORT_MULTI_LANE_CTRL, val);
+}
 
 void dw_pcie_upconfig_setup(struct dw_pcie *pci)
 {
drivers/pci/controller/dwc/pcie-designware.h +3 -0
index 3beac10e4a4c..fcf0442341fd 100644
@@ -67,6 +67,8 @@
 #define PCIE_MSI_INTR0_STATUS		0x830
 
 #define PCIE_PORT_MULTI_LANE_CTRL	0x8C0
+#define PORT_MLTI_LNK_WDTH		GENMASK(5, 0)
+#define PORT_MLTI_LNK_WDTH_CHNG		BIT(6)
 #define PORT_MLTI_UPCFG_SUPPORT		BIT(7)
 
 #define PCIE_ATU_VIEWPORT		0x900
@@ -282,6 +284,7 @@ void dw_pcie_write_dbi2(struct dw_pcie *pci, u32 reg, size_t size, u32 val);
 u32 dw_pcie_read_atu(struct dw_pcie *pci, u32 reg, size_t size);
 void dw_pcie_write_atu(struct dw_pcie *pci, u32 reg, size_t size, u32 val);
 int dw_pcie_link_up(struct dw_pcie *pci);
+void dw_pcie_link_width_resize(struct dw_pcie *pci, u32 lane_width);
 void dw_pcie_upconfig_setup(struct dw_pcie *pci);
 void dw_pcie_link_speed_change(struct dw_pcie *pci, bool enable);
 void dw_pcie_link_set_n_fts(struct dw_pcie *pci, u32 n_fts);
drivers/pci/controller/dwc/pcie-intel-gw.c +23 -0
index 9142c70db808..b9be0921671d 100644
@@ -146,6 +146,22 @@ static void intel_pcie_ltssm_disable(struct intel_pcie_port *lpp)
 	pcie_app_wr_mask(lpp, PCIE_APP_CCR_LTSSM_ENABLE, 0, PCIE_APP_CCR);
 }
 
+static const char *pcie_link_gen_to_str(int gen)
+{
+	switch (gen) {
+	case PCIE_LINK_SPEED_GEN1:
+		return "2.5";
+	case PCIE_LINK_SPEED_GEN2:
+		return "5.0";
+	case PCIE_LINK_SPEED_GEN3:
+		return "8.0";
+	case PCIE_LINK_SPEED_GEN4:
+		return "16.0";
+	default:
+		return "???";
+	}
+}
+
 static void intel_pcie_link_setup(struct intel_pcie_port *lpp)
 {
 	u32 val;
@@ -444,6 +460,91 @@ static int intel_pcie_host_setup(struct intel_pcie_port *lpp)
 	return ret;
 }
 
+static ssize_t pcie_link_status_show(struct device *dev,
+				     struct device_attribute *attr, char *buf)
+{
+	struct intel_pcie_port *lpp = dev_get_drvdata(dev);
+	u32 reg, width, gen;
+
+	reg = pcie_rc_cfg_rd(lpp, PCIE_CAP_OFST + PCI_EXP_LNKCTL);

drivers/pci/controller/dwc/pcie-intel-gw.c static int intel_pcie_host_setup(struct intel_pcie_port *lpp) · line 479

Quoted 10 lines
drivers/pci/controller/dwc/pcie-intel-gw.c +10 -0
+	width = FIELD_GET(PCI_EXP_LNKSTA_NLW, reg >> 16);
+	gen = FIELD_GET(PCI_EXP_LNKSTA_CLS, reg >> 16);
+
+	if (gen > lpp->max_speed)
+		return -EINVAL;
+
+	return sprintf(buf, "Port %2u Width x%u Speed %s GT/s\n", lpp->id,
+		       width, pcie_link_gen_to_str(gen));
+}
+static DEVICE_ATTR_RO(pcie_link_status);

Dilip please check pci.h there are there already enums and strings relatively to PCIe speed and width, that you can use.

Quoted 92 lines
drivers/pci/controller/dwc/pcie-intel-gw.c +78 -1
+
+static ssize_t pcie_speed_store(struct device *dev,
+				struct device_attribute *attr,
+				const char *buf, size_t len)
+{
+	struct intel_pcie_port *lpp = dev_get_drvdata(dev);
+	unsigned long val;
+	int ret;
+
+	ret = kstrtoul(buf, 10, &val);
+	if (ret)
+		return ret;
+
+	if (val > lpp->max_speed)
+		return -EINVAL;
+
+	lpp->link_gen = val;
+	intel_pcie_max_speed_setup(lpp);
+	dw_pcie_link_speed_change(&lpp->pci, false);
+	dw_pcie_link_speed_change(&lpp->pci, true);
+
+	return len;
+}
+static DEVICE_ATTR_WO(pcie_speed);
+
+/*
+ * Link width change on the fly is not always successful.
+ * It also depends on the partner.
+ */
+static ssize_t pcie_width_store(struct device *dev,
+				struct device_attribute *attr,
+				const char *buf, size_t len)
+{
+	struct intel_pcie_port *lpp = dev_get_drvdata(dev);
+	unsigned long val;
+	int ret;
+
+	lpp = dev_get_drvdata(dev);
+
+	ret = kstrtoul(buf, 10, &val);
+	if (ret)
+		return ret;
+
+	if (val > lpp->max_width)
+		return -EINVAL;
+
+	/* HW auto bandwidth negotiation must be enabled */
+	pcie_rc_cfg_wr_mask(lpp, PCI_EXP_LNKCTL_HAWD, 0,
+			    PCIE_CAP_OFST + PCI_EXP_LNKCTL);
+	dw_pcie_link_width_resize(&lpp->pci, val);
+
+	return len;
+}
+static DEVICE_ATTR_WO(pcie_width);
+
+static struct attribute *pcie_cfg_attrs[] = {
+	&dev_attr_pcie_link_status.attr,
+	&dev_attr_pcie_speed.attr,
+	&dev_attr_pcie_width.attr,
+	NULL,
+};
+ATTRIBUTE_GROUPS(pcie_cfg);
+
+static int intel_pcie_sysfs_init(struct intel_pcie_port *lpp)
+{
+	return devm_device_add_groups(lpp->pci.dev, pcie_cfg_groups);
+}
+
 static void __intel_pcie_remove(struct intel_pcie_port *lpp)
 {
 	intel_pcie_core_irq_disable(lpp);
@@ -490,8 +591,17 @@ static int intel_pcie_rc_init(struct pcie_port *pp)
 {
 	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
 	struct intel_pcie_port *lpp = dev_get_drvdata(pci->dev);
+	int ret;
 
-	return intel_pcie_host_setup(lpp);
+	ret = intel_pcie_host_setup(lpp);
+	if (ret)
+		return ret;
+
+	ret = intel_pcie_sysfs_init(lpp);
+	if (ret)
+		__intel_pcie_remove(lpp);
+
+	return ret;
 }
 
 int intel_pcie_msi_init(struct pcie_port *pp)
Signature · 1 line
2.11.0
author
#33 lore raw

Re: [PATCH v4 3/3] pci: intel: Add sysfs attributes to configure pcie link

Hi Gustavo Pimentel,

Quoted 101 lines · Gustavo Pimentel
Quoted 98 lines · Dilip Kota

PCIe RC driver on Intel Gateway SoCs have a requirement of changing link width and speed on the fly.
So add the sysfs attributes to show and store the link properties.
Add the respective link resize function in pcie DesignWare framework so that Intel PCIe driver can use during link width configuration on the fly.

---
  drivers/pci/controller/dwc/pcie-designware.c |   9 +++
  drivers/pci/controller/dwc/pcie-designware.h |   3 +
  drivers/pci/controller/dwc/pcie-intel-gw.c   | 112 ++++++++++++++++++++++++++-
  3 files changed, 123 insertions(+), 1 deletion(-)
drivers/pci/controller/dwc/pcie-designware.c +9 -0
index 4c391bfd681a..662fdcb4f2d6 100644
@@ -474,6 +474,15 @@ int dw_pcie_link_up(struct dw_pcie *pci)
  		(!(val & PCIE_PORT_DEBUG1_LINK_IN_TRAINING)));
  }
 
+void dw_pcie_link_width_resize(struct dw_pcie *pci, u32 lane_width)
+{
+	u32 val;
+
+	val =  dw_pcie_readl_dbi(pci, PCIE_PORT_MULTI_LANE_CTRL);
+	val &= ~(PORT_MLTI_LNK_WDTH_CHNG | PORT_MLTI_LNK_WDTH);
+	val |= PORT_MLTI_LNK_WDTH_CHNG | lane_width;
+	dw_pcie_writel_dbi(pci, PCIE_PORT_MULTI_LANE_CTRL, val);
+}
 
  void dw_pcie_upconfig_setup(struct dw_pcie *pci)
  {
drivers/pci/controller/dwc/pcie-designware.h +3 -0
index 3beac10e4a4c..fcf0442341fd 100644
@@ -67,6 +67,8 @@
  #define PCIE_MSI_INTR0_STATUS		0x830
 
  #define PCIE_PORT_MULTI_LANE_CTRL	0x8C0
+#define PORT_MLTI_LNK_WDTH		GENMASK(5, 0)
+#define PORT_MLTI_LNK_WDTH_CHNG		BIT(6)
  #define PORT_MLTI_UPCFG_SUPPORT		BIT(7)
 
  #define PCIE_ATU_VIEWPORT		0x900
@@ -282,6 +284,7 @@ void dw_pcie_write_dbi2(struct dw_pcie *pci, u32 reg, size_t size, u32 val);
  u32 dw_pcie_read_atu(struct dw_pcie *pci, u32 reg, size_t size);
  void dw_pcie_write_atu(struct dw_pcie *pci, u32 reg, size_t size, u32 val);
  int dw_pcie_link_up(struct dw_pcie *pci);
+void dw_pcie_link_width_resize(struct dw_pcie *pci, u32 lane_width);
  void dw_pcie_upconfig_setup(struct dw_pcie *pci);
  void dw_pcie_link_speed_change(struct dw_pcie *pci, bool enable);
  void dw_pcie_link_set_n_fts(struct dw_pcie *pci, u32 n_fts);
drivers/pci/controller/dwc/pcie-intel-gw.c +27 -0
index 9142c70db808..b9be0921671d 100644
@@ -146,6 +146,22 @@ static void intel_pcie_ltssm_disable(struct intel_pcie_port *lpp)
  	pcie_app_wr_mask(lpp, PCIE_APP_CCR_LTSSM_ENABLE, 0, PCIE_APP_CCR);
  }
 
+static const char *pcie_link_gen_to_str(int gen)
+{
+	switch (gen) {
+	case PCIE_LINK_SPEED_GEN1:
+		return "2.5";
+	case PCIE_LINK_SPEED_GEN2:
+		return "5.0";
+	case PCIE_LINK_SPEED_GEN3:
+		return "8.0";
+	case PCIE_LINK_SPEED_GEN4:
+		return "16.0";
+	default:
+		return "???";
+	}
+}
+
  static void intel_pcie_link_setup(struct intel_pcie_port *lpp)
  {
  	u32 val;
@@ -444,6 +460,91 @@ static int intel_pcie_host_setup(struct intel_pcie_port *lpp)
  	return ret;
  }
 
+static ssize_t pcie_link_status_show(struct device *dev,
+				     struct device_attribute *attr, char *buf)
+{
+	struct intel_pcie_port *lpp = dev_get_drvdata(dev);
+	u32 reg, width, gen;
+
+	reg = pcie_rc_cfg_rd(lpp, PCIE_CAP_OFST + PCI_EXP_LNKCTL);
+	width = FIELD_GET(PCI_EXP_LNKSTA_NLW, reg >> 16);
+	gen = FIELD_GET(PCI_EXP_LNKSTA_CLS, reg >> 16);
+
+	if (gen > lpp->max_speed)
Quoted 8 lines

drivers/pci/controller/dwc/pcie-intel-gw.c static int intel_pcie_host_setup(struct intel_pcie_port *lpp) · line 479

Quoted 6 lines
drivers/pci/controller/dwc/pcie-intel-gw.c +6 -0
+		return -EINVAL;
+
+	return sprintf(buf, "Port %2u Width x%u Speed %s GT/s\n", lpp->id,
+		       width, pcie_link_gen_to_str(gen));
+}
+static DEVICE_ATTR_RO(pcie_link_status);

Dilip please check pci.h there are there already enums and strings relatively to PCIe speed and width, that you can use.

Yes i can see a global array "pcie_link_speed[]" and a macro PCIE_SPEED2STR[]. I will update the driver.
Whereas width enum, it is not required here as directly storing the register value. Thanks for pointing it.

Regards,
Dilip

Quoted 94 lines
Quoted 93 lines
drivers/pci/controller/dwc/pcie-intel-gw.c +78 -1
+
+static ssize_t pcie_speed_store(struct device *dev,
+				struct device_attribute *attr,
+				const char *buf, size_t len)
+{
+	struct intel_pcie_port *lpp = dev_get_drvdata(dev);
+	unsigned long val;
+	int ret;
+
+	ret = kstrtoul(buf, 10, &val);
+	if (ret)
+		return ret;
+
+	if (val > lpp->max_speed)
+		return -EINVAL;
+
+	lpp->link_gen = val;
+	intel_pcie_max_speed_setup(lpp);
+	dw_pcie_link_speed_change(&lpp->pci, false);
+	dw_pcie_link_speed_change(&lpp->pci, true);
+
+	return len;
+}
+static DEVICE_ATTR_WO(pcie_speed);
+
+/*
+ * Link width change on the fly is not always successful.
+ * It also depends on the partner.
+ */
+static ssize_t pcie_width_store(struct device *dev,
+				struct device_attribute *attr,
+				const char *buf, size_t len)
+{
+	struct intel_pcie_port *lpp = dev_get_drvdata(dev);
+	unsigned long val;
+	int ret;
+
+	lpp = dev_get_drvdata(dev);
+
+	ret = kstrtoul(buf, 10, &val);
+	if (ret)
+		return ret;
+
+	if (val > lpp->max_width)
+		return -EINVAL;
+
+	/* HW auto bandwidth negotiation must be enabled */
+	pcie_rc_cfg_wr_mask(lpp, PCI_EXP_LNKCTL_HAWD, 0,
+			    PCIE_CAP_OFST + PCI_EXP_LNKCTL);
+	dw_pcie_link_width_resize(&lpp->pci, val);
+
+	return len;
+}
+static DEVICE_ATTR_WO(pcie_width);
+
+static struct attribute *pcie_cfg_attrs[] = {
+	&dev_attr_pcie_link_status.attr,
+	&dev_attr_pcie_speed.attr,
+	&dev_attr_pcie_width.attr,
+	NULL,
+};
+ATTRIBUTE_GROUPS(pcie_cfg);
+
+static int intel_pcie_sysfs_init(struct intel_pcie_port *lpp)
+{
+	return devm_device_add_groups(lpp->pci.dev, pcie_cfg_groups);
+}
+
  static void __intel_pcie_remove(struct intel_pcie_port *lpp)
  {
  	intel_pcie_core_irq_disable(lpp);
@@ -490,8 +591,17 @@ static int intel_pcie_rc_init(struct pcie_port *pp)
  {
  	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
  	struct intel_pcie_port *lpp = dev_get_drvdata(pci->dev);
+	int ret;
 
-	return intel_pcie_host_setup(lpp);
+	ret = intel_pcie_host_setup(lpp);
+	if (ret)
+		return ret;
+
+	ret = intel_pcie_sysfs_init(lpp);
+	if (ret)
+		__intel_pcie_remove(lpp);
+
+	return ret;
  }
 
  int intel_pcie_msi_init(struct pcie_port *pp)
Signature · 1 line
2.11.0
#34 lore raw

Re: [PATCH v4 3/3] pci: intel: Add sysfs attributes to configure pcie link

In reply to Dilip Kota
Quoted 155 lines · Dilip Kota

PCIe RC driver on Intel Gateway SoCs have a requirement of changing link width and speed on the fly.
So add the sysfs attributes to show and store the link properties.
Add the respective link resize function in pcie DesignWare framework so that Intel PCIe driver can use during link width configuration on the fly.

---
 drivers/pci/controller/dwc/pcie-designware.c |   9 +++
 drivers/pci/controller/dwc/pcie-designware.h |   3 +
 drivers/pci/controller/dwc/pcie-intel-gw.c   | 112 ++++++++++++++++++++++++++-
 3 files changed, 123 insertions(+), 1 deletion(-)
drivers/pci/controller/dwc/pcie-designware.c +9 -0
index 4c391bfd681a..662fdcb4f2d6 100644
@@ -474,6 +474,15 @@ int dw_pcie_link_up(struct dw_pcie *pci)
 		(!(val & PCIE_PORT_DEBUG1_LINK_IN_TRAINING)));
 }
 
+void dw_pcie_link_width_resize(struct dw_pcie *pci, u32 lane_width)
+{
+	u32 val;
+
+	val =  dw_pcie_readl_dbi(pci, PCIE_PORT_MULTI_LANE_CTRL);
+	val &= ~(PORT_MLTI_LNK_WDTH_CHNG | PORT_MLTI_LNK_WDTH);
+	val |= PORT_MLTI_LNK_WDTH_CHNG | lane_width;
+	dw_pcie_writel_dbi(pci, PCIE_PORT_MULTI_LANE_CTRL, val);
+}
 
 void dw_pcie_upconfig_setup(struct dw_pcie *pci)
 {
drivers/pci/controller/dwc/pcie-designware.h +3 -0
index 3beac10e4a4c..fcf0442341fd 100644
@@ -67,6 +67,8 @@
 #define PCIE_MSI_INTR0_STATUS		0x830
 
 #define PCIE_PORT_MULTI_LANE_CTRL	0x8C0
+#define PORT_MLTI_LNK_WDTH		GENMASK(5, 0)
+#define PORT_MLTI_LNK_WDTH_CHNG		BIT(6)
 #define PORT_MLTI_UPCFG_SUPPORT		BIT(7)
 
 #define PCIE_ATU_VIEWPORT		0x900
@@ -282,6 +284,7 @@ void dw_pcie_write_dbi2(struct dw_pcie *pci, u32 reg, size_t size, u32 val);
 u32 dw_pcie_read_atu(struct dw_pcie *pci, u32 reg, size_t size);
 void dw_pcie_write_atu(struct dw_pcie *pci, u32 reg, size_t size, u32 val);
 int dw_pcie_link_up(struct dw_pcie *pci);
+void dw_pcie_link_width_resize(struct dw_pcie *pci, u32 lane_width);
 void dw_pcie_upconfig_setup(struct dw_pcie *pci);
 void dw_pcie_link_speed_change(struct dw_pcie *pci, bool enable);
 void dw_pcie_link_set_n_fts(struct dw_pcie *pci, u32 n_fts);
drivers/pci/controller/dwc/pcie-intel-gw.c +84 -0
index 9142c70db808..b9be0921671d 100644
@@ -146,6 +146,22 @@ static void intel_pcie_ltssm_disable(struct intel_pcie_port *lpp)
 	pcie_app_wr_mask(lpp, PCIE_APP_CCR_LTSSM_ENABLE, 0, PCIE_APP_CCR);
 }
 
+static const char *pcie_link_gen_to_str(int gen)
+{
+	switch (gen) {
+	case PCIE_LINK_SPEED_GEN1:
+		return "2.5";
+	case PCIE_LINK_SPEED_GEN2:
+		return "5.0";
+	case PCIE_LINK_SPEED_GEN3:
+		return "8.0";
+	case PCIE_LINK_SPEED_GEN4:
+		return "16.0";
+	default:
+		return "???";
+	}
+}
+
 static void intel_pcie_link_setup(struct intel_pcie_port *lpp)
 {
 	u32 val;
@@ -444,6 +460,91 @@ static int intel_pcie_host_setup(struct intel_pcie_port *lpp)
 	return ret;
 }
 
+static ssize_t pcie_link_status_show(struct device *dev,
+				     struct device_attribute *attr, char *buf)
+{
+	struct intel_pcie_port *lpp = dev_get_drvdata(dev);
+	u32 reg, width, gen;
+
+	reg = pcie_rc_cfg_rd(lpp, PCIE_CAP_OFST + PCI_EXP_LNKCTL);
+	width = FIELD_GET(PCI_EXP_LNKSTA_NLW, reg >> 16);
+	gen = FIELD_GET(PCI_EXP_LNKSTA_CLS, reg >> 16);
+
+	if (gen > lpp->max_speed)
+		return -EINVAL;
+
+	return sprintf(buf, "Port %2u Width x%u Speed %s GT/s\n", lpp->id,
+		       width, pcie_link_gen_to_str(gen));
+}
+static DEVICE_ATTR_RO(pcie_link_status);
+
+static ssize_t pcie_speed_store(struct device *dev,
+				struct device_attribute *attr,
+				const char *buf, size_t len)
+{
+	struct intel_pcie_port *lpp = dev_get_drvdata(dev);
+	unsigned long val;
+	int ret;
+
+	ret = kstrtoul(buf, 10, &val);
+	if (ret)
+		return ret;
+
+	if (val > lpp->max_speed)
+		return -EINVAL;
+
+	lpp->link_gen = val;
+	intel_pcie_max_speed_setup(lpp);
+	dw_pcie_link_speed_change(&lpp->pci, false);
+	dw_pcie_link_speed_change(&lpp->pci, true);
+
+	return len;
+}
+static DEVICE_ATTR_WO(pcie_speed);
+
+/*
+ * Link width change on the fly is not always successful.
+ * It also depends on the partner.
+ */
+static ssize_t pcie_width_store(struct device *dev,
+				struct device_attribute *attr,
+				const char *buf, size_t len)
+{
+	struct intel_pcie_port *lpp = dev_get_drvdata(dev);
+	unsigned long val;
+	int ret;
+
+	lpp = dev_get_drvdata(dev);
+
+	ret = kstrtoul(buf, 10, &val);
+	if (ret)
+		return ret;
+
+	if (val > lpp->max_width)
+		return -EINVAL;
+
+	/* HW auto bandwidth negotiation must be enabled */
+	pcie_rc_cfg_wr_mask(lpp, PCI_EXP_LNKCTL_HAWD, 0,
+			    PCIE_CAP_OFST + PCI_EXP_LNKCTL);
+	dw_pcie_link_width_resize(&lpp->pci, val);
+

drivers/pci/controller/dwc/pcie-intel-gw.c static int intel_pcie_host_setup(struct intel_pcie_port *lpp) · line 540

Quoted 10 lines
drivers/pci/controller/dwc/pcie-intel-gw.c +10 -0
+	return len;
+}
+static DEVICE_ATTR_WO(pcie_width);
+
+static struct attribute *pcie_cfg_attrs[] = {
+	&dev_attr_pcie_link_status.attr,
+	&dev_attr_pcie_speed.attr,
+	&dev_attr_pcie_width.attr,
+	NULL,
+};

Is there a reason that these are limited only to the Intel driver and not the wider set of DWC drivers?

Is there anything specific here about the Intel GW driver?

Thanks,

Andrew Murray

Quoted 32 lines
drivers/pci/controller/dwc/pcie-intel-gw.c +17 -1
+ATTRIBUTE_GROUPS(pcie_cfg);
+
+static int intel_pcie_sysfs_init(struct intel_pcie_port *lpp)
+{
+	return devm_device_add_groups(lpp->pci.dev, pcie_cfg_groups);
+}
+
 static void __intel_pcie_remove(struct intel_pcie_port *lpp)
 {
 	intel_pcie_core_irq_disable(lpp);
@@ -490,8 +591,17 @@ static int intel_pcie_rc_init(struct pcie_port *pp)
 {
 	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
 	struct intel_pcie_port *lpp = dev_get_drvdata(pci->dev);
+	int ret;
 
-	return intel_pcie_host_setup(lpp);
+	ret = intel_pcie_host_setup(lpp);
+	if (ret)
+		return ret;
+
+	ret = intel_pcie_sysfs_init(lpp);
+	if (ret)
+		__intel_pcie_remove(lpp);
+
+	return ret;
 }
 
 int intel_pcie_msi_init(struct pcie_port *pp)
Signature · 1 line
2.11.0
#35 lore raw

Re: [PATCH v4 3/3] pci: intel: Add sysfs attributes to configure pcie link

Quoted 3 lines · Andrew Murray
Quoted 2 lines · Dilip Kota

PCIe RC driver on Intel Gateway SoCs have a requirement of changing link width and speed on the fly.

Please add more details about why this is needed. Since you're adding sysfs files, it sounds like it's not actually the *driver* that needs this; it's something in userspace?

The normal scenario is that the hardware negotiates link widths and speeds without any software involvement (PCIe r5.0, sec 1.2).

If this is to work around hardware defects, we should try to do that inside the kernel because we can't expect userspace to do it reliably.

As Andrew points out below, this all sounds like it should be generic rather than Intel-specific.

Quoted 16 lines
Quoted 16 lines

So add the sysfs attributes to show and store the link properties.
Add the respective link resize function in pcie DesignWare framework so that Intel PCIe driver can use during link width configuration on the fly.
...

drivers/pci/controller/dwc/pcie-intel-gw.c +9 -0
+static ssize_t pcie_link_status_show(struct device *dev,
+				     struct device_attribute *attr, char *buf)
+{
+	struct intel_pcie_port *lpp = dev_get_drvdata(dev);
+	u32 reg, width, gen;
+
+	reg = pcie_rc_cfg_rd(lpp, PCIE_CAP_OFST + PCI_EXP_LNKCTL);
+	width = FIELD_GET(PCI_EXP_LNKSTA_NLW, reg >> 16);
+	gen = FIELD_GET(PCI_EXP_LNKSTA_CLS, reg >> 16);
Quoted 8 lines

drivers/pci/controller/dwc/pcie-intel-gw.c static int intel_pcie_host_setup(struct intel_pcie_port *lpp) · line 479

Quoted 8 lines
drivers/pci/controller/dwc/pcie-intel-gw.c +8 -0
+
+	if (gen > lpp->max_speed)
+		return -EINVAL;
+
+	return sprintf(buf, "Port %2u Width x%u Speed %s GT/s\n", lpp->id,
+		       width, pcie_link_gen_to_str(gen));
+}
+static DEVICE_ATTR_RO(pcie_link_status);

We already have generic current_link_speed and current_link_width files.

Quoted 65 lines
Quoted 50 lines
drivers/pci/controller/dwc/pcie-intel-gw.c +50 -0
+static ssize_t pcie_speed_store(struct device *dev,
+				struct device_attribute *attr,
+				const char *buf, size_t len)
+{
+	struct intel_pcie_port *lpp = dev_get_drvdata(dev);
+	unsigned long val;
+	int ret;
+
+	ret = kstrtoul(buf, 10, &val);
+	if (ret)
+		return ret;
+
+	if (val > lpp->max_speed)
+		return -EINVAL;
+
+	lpp->link_gen = val;
+	intel_pcie_max_speed_setup(lpp);
+	dw_pcie_link_speed_change(&lpp->pci, false);
+	dw_pcie_link_speed_change(&lpp->pci, true);
+
+	return len;
+}
+static DEVICE_ATTR_WO(pcie_speed);
+
+/*
+ * Link width change on the fly is not always successful.
+ * It also depends on the partner.
+ */
+static ssize_t pcie_width_store(struct device *dev,
+				struct device_attribute *attr,
+				const char *buf, size_t len)
+{
+	struct intel_pcie_port *lpp = dev_get_drvdata(dev);
+	unsigned long val;
+	int ret;
+
+	lpp = dev_get_drvdata(dev);
+
+	ret = kstrtoul(buf, 10, &val);
+	if (ret)
+		return ret;
+
+	if (val > lpp->max_width)
+		return -EINVAL;
+
+	/* HW auto bandwidth negotiation must be enabled */
+	pcie_rc_cfg_wr_mask(lpp, PCI_EXP_LNKCTL_HAWD, 0,
+			    PCIE_CAP_OFST + PCI_EXP_LNKCTL);
+	dw_pcie_link_width_resize(&lpp->pci, val);
+

drivers/pci/controller/dwc/pcie-intel-gw.c static int intel_pcie_host_setup(struct intel_pcie_port *lpp) · line 540

Quoted 10 lines
drivers/pci/controller/dwc/pcie-intel-gw.c +10 -0
+	return len;
+}
+static DEVICE_ATTR_WO(pcie_width);
+
+static struct attribute *pcie_cfg_attrs[] = {
+	&dev_attr_pcie_link_status.attr,
+	&dev_attr_pcie_speed.attr,
+	&dev_attr_pcie_width.attr,
+	NULL,
+};

Is there a reason that these are limited only to the Intel driver and not the wider set of DWC drivers?

Is there anything specific here about the Intel GW driver?

author
#36 lore raw

Re: [PATCH v4 3/3] pci: intel: Add sysfs attributes to configure pcie link

Hi Bjorn Helgaas,

Quoted 7 lines · Bjorn Helgaas
Quoted 3 lines · Andrew Murray
Quoted 2 lines · Dilip Kota

PCIe RC driver on Intel Gateway SoCs have a requirement of changing link width and speed on the fly.

Please add more details about why this is needed. Since you're adding sysfs files, it sounds like it's not actually the *driver* that needs this; it's something in userspace?

We have use cases to change the link speed and width on the fly.
One is EMI check and other is power saving.
Some battery backed applications have to switch PCIe link from higher GEN to GEN1 and width to x1. During the cases like external power supply got disconnected or broken. Once external power supply is connected then switch PCIe link to higher GEN and width.

Quoted 27 lines

The normal scenario is that the hardware negotiates link widths and speeds without any software involvement (PCIe r5.0, sec 1.2).

If this is to work around hardware defects, we should try to do that inside the kernel because we can't expect userspace to do it reliably.

As Andrew points out below, this all sounds like it should be generic rather than Intel-specific.

Quoted 17 lines
Quoted 17 lines

So add the sysfs attributes to show and store the link properties.
Add the respective link resize function in pcie DesignWare framework so that Intel PCIe driver can use during link width configuration on the fly.
...

drivers/pci/controller/dwc/pcie-intel-gw.c +11 -0
+static ssize_t pcie_link_status_show(struct device *dev,
+				     struct device_attribute *attr, char *buf)
+{
+	struct intel_pcie_port *lpp = dev_get_drvdata(dev);
+	u32 reg, width, gen;
+
+	reg = pcie_rc_cfg_rd(lpp, PCIE_CAP_OFST + PCI_EXP_LNKCTL);
+	width = FIELD_GET(PCI_EXP_LNKSTA_NLW, reg >> 16);
+	gen = FIELD_GET(PCI_EXP_LNKSTA_CLS, reg >> 16);
+
+	if (gen > lpp->max_speed)
Quoted 8 lines
Quoted 6 lines

drivers/pci/controller/dwc/pcie-intel-gw.c static int intel_pcie_host_setup(struct intel_pcie_port *lpp) · line 479

Quoted 6 lines
drivers/pci/controller/dwc/pcie-intel-gw.c +6 -0
+		return -EINVAL;
+
+	return sprintf(buf, "Port %2u Width x%u Speed %s GT/s\n", lpp->id,
+		       width, pcie_link_gen_to_str(gen));
+}
+static DEVICE_ATTR_RO(pcie_link_status);

We already have generic current_link_speed and current_link_width files.

Thanks for pointing it. I will remove the pcie_link_status.

Regards,
Dilip

Quoted 65 lines
Quoted 65 lines
Quoted 51 lines
drivers/pci/controller/dwc/pcie-intel-gw.c +50 -0
+static ssize_t pcie_speed_store(struct device *dev,
+				struct device_attribute *attr,
+				const char *buf, size_t len)
+{
+	struct intel_pcie_port *lpp = dev_get_drvdata(dev);
+	unsigned long val;
+	int ret;
+
+	ret = kstrtoul(buf, 10, &val);
+	if (ret)
+		return ret;
+
+	if (val > lpp->max_speed)
+		return -EINVAL;
+
+	lpp->link_gen = val;
+	intel_pcie_max_speed_setup(lpp);
+	dw_pcie_link_speed_change(&lpp->pci, false);
+	dw_pcie_link_speed_change(&lpp->pci, true);
+
+	return len;
+}
+static DEVICE_ATTR_WO(pcie_speed);
+
+/*
+ * Link width change on the fly is not always successful.
+ * It also depends on the partner.
+ */
+static ssize_t pcie_width_store(struct device *dev,
+				struct device_attribute *attr,
+				const char *buf, size_t len)
+{
+	struct intel_pcie_port *lpp = dev_get_drvdata(dev);
+	unsigned long val;
+	int ret;
+
+	lpp = dev_get_drvdata(dev);
+
+	ret = kstrtoul(buf, 10, &val);
+	if (ret)
+		return ret;
+
+	if (val > lpp->max_width)
+		return -EINVAL;
+
+	/* HW auto bandwidth negotiation must be enabled */
+	pcie_rc_cfg_wr_mask(lpp, PCI_EXP_LNKCTL_HAWD, 0,
+			    PCIE_CAP_OFST + PCI_EXP_LNKCTL);
+	dw_pcie_link_width_resize(&lpp->pci, val);
+

drivers/pci/controller/dwc/pcie-intel-gw.c static int intel_pcie_host_setup(struct intel_pcie_port *lpp) · line 540

Quoted 10 lines
drivers/pci/controller/dwc/pcie-intel-gw.c +10 -0
+	return len;
+}
+static DEVICE_ATTR_WO(pcie_width);
+
+static struct attribute *pcie_cfg_attrs[] = {
+	&dev_attr_pcie_link_status.attr,
+	&dev_attr_pcie_speed.attr,
+	&dev_attr_pcie_width.attr,
+	NULL,
+};

Is there a reason that these are limited only to the Intel driver and not the wider set of DWC drivers?

Is there anything specific here about the Intel GW driver?

#37 lore raw

Re: [PATCH v4 3/3] pci: intel: Add sysfs attributes to configure pcie link

[+cc Rafael, linux-pm, beginning of discussion at
https://lore.kernel.org/r/d8574605f8e70f41ce1e88ccfb56b63c8f85e4df.1571638827.git.eswara.kota@linux.intel.com]

Quoted 9 lines · Dilip Kota
Quoted 7 lines · Bjorn Helgaas
Quoted 3 lines · Andrew Murray
Quoted 2 lines · Dilip Kota

PCIe RC driver on Intel Gateway SoCs have a requirement of changing link width and speed on the fly.

Please add more details about why this is needed. Since you're adding sysfs files, it sounds like it's not actually the *driver* that needs this; it's something in userspace?

Quoted 6 lines

We have use cases to change the link speed and width on the fly. One is EMI check and other is power saving. Some battery backed applications have to switch PCIe link from higher GEN to GEN1 and width to x1. During the cases like external power supply got disconnected or broken. Once external power supply is connected then switch PCIe link to higher GEN and width.

That sounds plausible, but of course nothing there is specific to the Intel Gateway, so we should implement this generically so it would work on all hardware.

I'm not sure what the interface should look like -- should it be a low-level interface as you propose where userspace would have to identify each link of interest, or is there some system-wide power/performance knob that could tune all links? Cc'd Rafael and linux-pm in case they have ideas.

Bjorn

author
#38 lore raw

Re: [PATCH v4 3/3] pci: intel: Add sysfs attributes to configure pcie link

Quoted 18 lines · Bjorn Helgaas

[+cc Rafael, linux-pm, beginning of discussion at
https://lore.kernel.org/r/d8574605f8e70f41ce1e88ccfb56b63c8f85e4df.1571638827.git.eswara.kota@linux.intel.com]

Quoted 8 lines · Dilip Kota
Quoted 7 lines · Bjorn Helgaas
Quoted 3 lines · Andrew Murray
Quoted 2 lines · Dilip Kota

PCIe RC driver on Intel Gateway SoCs have a requirement of changing link width and speed on the fly.

Please add more details about why this is needed. Since you're adding sysfs files, it sounds like it's not actually the *driver* that needs this; it's something in userspace?

Quoted 6 lines

We have use cases to change the link speed and width on the fly. One is EMI check and other is power saving. Some battery backed applications have to switch PCIe link from higher GEN to GEN1 and width to x1. During the cases like external power supply got disconnected or broken. Once external power supply is connected then switch PCIe link to higher GEN and width.

Quoted 3 lines

That sounds plausible, but of course nothing there is specific to the Intel Gateway, so we should implement this generically so it would work on all hardware.

Agree.

Quoted 6 lines

I'm not sure what the interface should look like -- should it be a low-level interface as you propose where userspace would have to identify each link of interest, or is there some system-wide power/performance knob that could tune all links? Cc'd Rafael and linux-pm in case they have ideas.

To my knowledge sysfs is the appropriate way to go. If there are any other best possible knobs, will be helpful.

Regards,
Dilip

Quoted 2 lines

Bjorn

#39 lore raw

Re: [PATCH v4 3/3] pci: intel: Add sysfs attributes to configure pcie link

[+cc Heiner, Rajat]

Quoted 24 lines · Dilip Kota
Quoted 18 lines · Bjorn Helgaas

[+cc Rafael, linux-pm, beginning of discussion at
https://lore.kernel.org/r/d8574605f8e70f41ce1e88ccfb56b63c8f85e4df.1571638827.git.eswara.kota@linux.intel.com]

Quoted 8 lines · Dilip Kota
Quoted 7 lines · Bjorn Helgaas
Quoted 3 lines · Andrew Murray
Quoted 2 lines · Dilip Kota

PCIe RC driver on Intel Gateway SoCs have a requirement of changing link width and speed on the fly.

Please add more details about why this is needed. Since you're adding sysfs files, it sounds like it's not actually the *driver* that needs this; it's something in userspace?

Quoted 6 lines

We have use cases to change the link speed and width on the fly. One is EMI check and other is power saving. Some battery backed applications have to switch PCIe link from higher GEN to GEN1 and width to x1. During the cases like external power supply got disconnected or broken. Once external power supply is connected then switch PCIe link to higher GEN and width.

Quoted 3 lines

That sounds plausible, but of course nothing there is specific to the Intel Gateway, so we should implement this generically so it would work on all hardware.

Agree.

Quoted 1 line
Quoted 8 lines
Quoted 5 lines

I'm not sure what the interface should look like -- should it be a low-level interface as you propose where userspace would have to identify each link of interest, or is there some system-wide power/performance knob that could tune all links? Cc'd Rafael and linux-pm in case they have ideas.

To my knowledge sysfs is the appropriate way to go. If there are any other best possible knobs, will be helpful.

I agree sysfs is the right place for it; my question was whether we should have files like:

/sys/.../0000:00:1f.3/pcie_speed
/sys/.../0000:00:1f.3/pcie_width

as I think this patch would add (BTW, please include sample paths like the above in the commit log), or whether there should be a more global thing that would affect all the links in the system.

I think the low-level files like you propose would be better because one might want to tune link performance differently for different types of devices and workloads.

We also have to decide if these files should be associated with the device at the upstream or downstream end of the link. For ASPM, the current proposal [1] has the files at the downstream end on the theory that the GPU, NIC, NVMe device, etc is the user-recognizable one. Also, neither ASPM nor link speed/width make any sense unless there *is* a device at the downstream end, so putting them there automatically makes them visible only when they're useful.

Rafael had some concerns about the proposed ASPM interface [2], but I don't know what they are yet.

For ASPM we added a "link_pm" directory, and maybe that's too specific. Maybe it should be a generic "link_mgt" or even "pcie" directory that could contain both the ASPM and width/speed files.

There's also a change coming to put AER stats in something like this:

/sys/.../0000:00:1f.3/aer_stats/correctable_rx_err /sys/.../0000:00:1f.3/aer_stats/correctable_timeout /sys/.../0000:00:1f.3/aer_stats/fatal_TLP
...

It would certainly be good to have some organizational scheme or we'll end up with a real hodge-podge.

[1] https://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci.git/commit/?h=pci/aspm&id=ad46fe1c733656611788e2cd59793e891ed7ded7
[2] https://lore.kernel.org/r/CAJZ5v0jdxR4roEUC_Hs3puCzGY4ThdLsi_XcxfBUUxqruP4z7A@mail.gmail.com

#40 lore raw

Re: [PATCH v4 3/3] pci: intel: Add sysfs attributes to configure pcie link

Quoted 59 lines · Bjorn Helgaas

[+cc Heiner, Rajat]

Quoted 24 lines · Dilip Kota
Quoted 18 lines · Bjorn Helgaas

[+cc Rafael, linux-pm, beginning of discussion at
https://lore.kernel.org/r/d8574605f8e70f41ce1e88ccfb56b63c8f85e4df.1571638827.git.eswara.kota@linux.intel.com]

Quoted 8 lines · Dilip Kota
Quoted 7 lines · Bjorn Helgaas
Quoted 3 lines · Andrew Murray
Quoted 2 lines · Dilip Kota

PCIe RC driver on Intel Gateway SoCs have a requirement of changing link width and speed on the fly.

Please add more details about why this is needed. Since you're adding sysfs files, it sounds like it's not actually the *driver* that needs this; it's something in userspace?

Quoted 6 lines

We have use cases to change the link speed and width on the fly. One is EMI check and other is power saving. Some battery backed applications have to switch PCIe link from higher GEN to GEN1 and width to x1. During the cases like external power supply got disconnected or broken. Once external power supply is connected then switch PCIe link to higher GEN and width.

Quoted 3 lines

That sounds plausible, but of course nothing there is specific to the Intel Gateway, so we should implement this generically so it would work on all hardware.

Agree.

Quoted 1 line
Quoted 8 lines
Quoted 5 lines

I'm not sure what the interface should look like -- should it be a low-level interface as you propose where userspace would have to identify each link of interest, or is there some system-wide power/performance knob that could tune all links? Cc'd Rafael and linux-pm in case they have ideas.

To my knowledge sysfs is the appropriate way to go. If there are any other best possible knobs, will be helpful.

I agree sysfs is the right place for it; my question was whether we should have files like:

/sys/.../0000:00:1f.3/pcie_speed
/sys/.../0000:00:1f.3/pcie_width

as I think this patch would add (BTW, please include sample paths like the above in the commit log), or whether there should be a more global thing that would affect all the links in the system.

I think the low-level files like you propose would be better because one might want to tune link performance differently for different types of devices and workloads.

We also have to decide if these files should be associated with the device at the upstream or downstream end of the link. For ASPM, the current proposal [1] has the files at the downstream end on the theory that the GPU, NIC, NVMe device, etc is the user-recognizable one. Also, neither ASPM nor link speed/width make any sense unless there *is* a device at the downstream end, so putting them there automatically makes them visible only when they're useful.

Quoted 2 lines

Rafael had some concerns about the proposed ASPM interface [2], but I don't know what they are yet.

I was talking about the existing ASPM interface in sysfs. The new one I still have to review, but I'm kind of wondering what about people who used the old one? Would it be supported going forward?

Quoted 16 lines

For ASPM we added a "link_pm" directory, and maybe that's too specific. Maybe it should be a generic "link_mgt" or even "pcie" directory that could contain both the ASPM and width/speed files.

There's also a change coming to put AER stats in something like this:

/sys/.../0000:00:1f.3/aer_stats/correctable_rx_err /sys/.../0000:00:1f.3/aer_stats/correctable_timeout /sys/.../0000:00:1f.3/aer_stats/fatal_TLP
...

It would certainly be good to have some organizational scheme or we'll end up with a real hodge-podge.

[1] https://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci.git/commit/?h=pci/aspm&id=ad46fe1c733656611788e2cd59793e891ed7ded7
[2] https://lore.kernel.org/r/CAJZ5v0jdxR4roEUC_Hs3puCzGY4ThdLsi_XcxfBUUxqruP4z7A@mail.gmail.com

#41 lore raw

Re: [PATCH v4 3/3] pci: intel: Add sysfs attributes to configure pcie link

Quoted 60 lines · Rafael J. Wysocki
Quoted 59 lines · Bjorn Helgaas

[+cc Heiner, Rajat]

Quoted 24 lines · Dilip Kota
Quoted 18 lines · Bjorn Helgaas

[+cc Rafael, linux-pm, beginning of discussion at
https://lore.kernel.org/r/d8574605f8e70f41ce1e88ccfb56b63c8f85e4df.1571638827.git.eswara.kota@linux.intel.com]

Quoted 8 lines · Dilip Kota
Quoted 7 lines · Bjorn Helgaas
Quoted 3 lines · Andrew Murray
Quoted 2 lines · Dilip Kota

PCIe RC driver on Intel Gateway SoCs have a requirement of changing link width and speed on the fly.

Please add more details about why this is needed. Since you're adding sysfs files, it sounds like it's not actually the *driver* that needs this; it's something in userspace?

Quoted 6 lines

We have use cases to change the link speed and width on the fly. One is EMI check and other is power saving. Some battery backed applications have to switch PCIe link from higher GEN to GEN1 and width to x1. During the cases like external power supply got disconnected or broken. Once external power supply is connected then switch PCIe link to higher GEN and width.

Quoted 3 lines

That sounds plausible, but of course nothing there is specific to the Intel Gateway, so we should implement this generically so it would work on all hardware.

Agree.

Quoted 1 line
Quoted 8 lines
Quoted 5 lines

I'm not sure what the interface should look like -- should it be a low-level interface as you propose where userspace would have to identify each link of interest, or is there some system-wide power/performance knob that could tune all links? Cc'd Rafael and linux-pm in case they have ideas.

To my knowledge sysfs is the appropriate way to go. If there are any other best possible knobs, will be helpful.

I agree sysfs is the right place for it; my question was whether we should have files like:

/sys/.../0000:00:1f.3/pcie_speed
/sys/.../0000:00:1f.3/pcie_width

as I think this patch would add (BTW, please include sample paths like the above in the commit log), or whether there should be a more global thing that would affect all the links in the system.

I think the low-level files like you propose would be better because one might want to tune link performance differently for different types of devices and workloads.

We also have to decide if these files should be associated with the device at the upstream or downstream end of the link. For ASPM, the current proposal [1] has the files at the downstream end on the theory that the GPU, NIC, NVMe device, etc is the user-recognizable one. Also, neither ASPM nor link speed/width make any sense unless there *is* a device at the downstream end, so putting them there automatically makes them visible only when they're useful.

Quoted 6 lines
Quoted 2 lines

Rafael had some concerns about the proposed ASPM interface [2], but I don't know what they are yet.

I was talking about the existing ASPM interface in sysfs. The new one I still have to review, but I'm kind of wondering what about people who used the old one? Would it be supported going forward?

The old one interface was enabled by CONFIG_PCIEASPM_DEBUG. Red Hat doesn't enable that. Ubuntu does. I *thought* we heard from a Canonical person who said they didn't have any tools that used it, but I can't find that now. I don't know about SUSE.

So the idea was to drop it on the theory that nobody is using it. Possibly that's too aggressive.

Bjorn

#42 lore raw

Re: [PATCH v4 3/3] pci: intel: Add sysfs attributes to configure pcie link

Quoted 68 lines · CET Bjorn Helgaas
Quoted 60 lines · Rafael J. Wysocki
Quoted 59 lines · Bjorn Helgaas

[+cc Heiner, Rajat]

Quoted 24 lines · Dilip Kota
Quoted 18 lines · Bjorn Helgaas

[+cc Rafael, linux-pm, beginning of discussion at
https://lore.kernel.org/r/d8574605f8e70f41ce1e88ccfb56b63c8f85e4df.1571638827.git.eswara.kota@linux.intel.com]

Quoted 8 lines · Dilip Kota
Quoted 7 lines · Bjorn Helgaas
Quoted 3 lines · Andrew Murray
Quoted 2 lines · Dilip Kota

PCIe RC driver on Intel Gateway SoCs have a requirement of changing link width and speed on the fly.

Please add more details about why this is needed. Since you're adding sysfs files, it sounds like it's not actually the *driver* that needs this; it's something in userspace?

Quoted 6 lines

We have use cases to change the link speed and width on the fly. One is EMI check and other is power saving. Some battery backed applications have to switch PCIe link from higher GEN to GEN1 and width to x1. During the cases like external power supply got disconnected or broken. Once external power supply is connected then switch PCIe link to higher GEN and width.

Quoted 3 lines

That sounds plausible, but of course nothing there is specific to the Intel Gateway, so we should implement this generically so it would work on all hardware.

Agree.

Quoted 1 line
Quoted 8 lines
Quoted 5 lines

I'm not sure what the interface should look like -- should it be a low-level interface as you propose where userspace would have to identify each link of interest, or is there some system-wide power/performance knob that could tune all links? Cc'd Rafael and linux-pm in case they have ideas.

To my knowledge sysfs is the appropriate way to go. If there are any other best possible knobs, will be helpful.

I agree sysfs is the right place for it; my question was whether we should have files like:

/sys/.../0000:00:1f.3/pcie_speed
/sys/.../0000:00:1f.3/pcie_width

as I think this patch would add (BTW, please include sample paths like the above in the commit log), or whether there should be a more global thing that would affect all the links in the system.

I think the low-level files like you propose would be better because one might want to tune link performance differently for different types of devices and workloads.

We also have to decide if these files should be associated with the device at the upstream or downstream end of the link. For ASPM, the current proposal [1] has the files at the downstream end on the theory that the GPU, NIC, NVMe device, etc is the user-recognizable one. Also, neither ASPM nor link speed/width make any sense unless there *is* a device at the downstream end, so putting them there automatically makes them visible only when they're useful.

Quoted 6 lines
Quoted 2 lines

Rafael had some concerns about the proposed ASPM interface [2], but I don't know what they are yet.

I was talking about the existing ASPM interface in sysfs. The new one I still have to review, but I'm kind of wondering what about people who used the old one? Would it be supported going forward?

Quoted 7 lines

The old one interface was enabled by CONFIG_PCIEASPM_DEBUG. Red Hat doesn't enable that. Ubuntu does. I *thought* we heard from a Canonical person who said they didn't have any tools that used it, but I can't find that now. I don't know about SUSE.

So the idea was to drop it on the theory that nobody is using it. Possibly that's too aggressive.

Well, one problem is that the "old" (actually existing) I/F has made it to one of my OSS EU presentation slides (I did not talk to this particular slide, but it is there in the deck that's available for downloading), so who knows who is going to use it. :-)

So I guess that there's a risk that needs to be taken into consideration.

What could be done, in principle, would be to make the new I/F depend on CONFIG_PCIEASPM_DEBUG being unset and provide the "old" one when it is set.

In any case, the pcie_aspm.policy module parameter cannot be dropped, because AFAICS there is quite a bit of user space using it (e.g. TLP).

Cheers!

#43 lore raw

Re: [PATCH v4 3/3] pci: intel: Add sysfs attributes to configure pcie link

Quoted 24 lines · Rafael J. Wysocki
Quoted 10 lines · CET Bjorn Helgaas
Quoted 8 lines · Rafael J. Wysocki
Quoted 2 lines · Bjorn Helgaas

Rafael had some concerns about the proposed ASPM interface [2], but I don't know what they are yet.

I was talking about the existing ASPM interface in sysfs. The new one I still have to review, but I'm kind of wondering what about people who used the old one? Would it be supported going forward?

Quoted 7 lines

The old one interface was enabled by CONFIG_PCIEASPM_DEBUG. Red Hat doesn't enable that. Ubuntu does. I *thought* we heard from a Canonical person who said they didn't have any tools that used it, but I can't find that now. I don't know about SUSE.

So the idea was to drop it on the theory that nobody is using it. Possibly that's too aggressive.

Well, one problem is that the "old" (actually existing) I/F has made it to one of my OSS EU presentation slides (I did not talk to this particular slide, but it is there in the deck that's available for downloading), so who knows who is going to use it. :-)

Quoted 4 lines

So I guess that there's a risk that needs to be taken into consideration.

What could be done, in principle, would be to make the new I/F depend on CONFIG_PCIEASPM_DEBUG being unset and provide the "old" one when it is set.

I would prefer to enable the new interface unconditionally to make it easier for userspace tools like powertop to use it.

I think the existing and new interfaces could coexist, with the existing interface being enabled by CONFIG_PCIEASPM_DEBUG as it is today. The patch that removes the existing interface is the last in the series and could easily be dropped.

Quoted 2 lines

In any case, the pcie_aspm.policy module parameter cannot be dropped, because AFAICS there is quite a bit of user space using it (e.g. TLP).

What is TLP? Since CONFIG_PCIEASPM is a bool, aspm.o is built in statically if enabled, so pcie_aspm.policy is effectively a boot-time kernel parameter, right? We don't have a plan to remove it.

Bjorn

author
#44 lore raw

Re: [PATCH v4 3/3] pci: intel: Add sysfs attributes to configure pcie link

In reply to Bjorn Helgaas
Quoted 36 lines · Bjorn Helgaas

[+cc Heiner, Rajat]

Quoted 22 lines · Dilip Kota
Quoted 18 lines · Bjorn Helgaas

[+cc Rafael, linux-pm, beginning of discussion at
https://lore.kernel.org/r/d8574605f8e70f41ce1e88ccfb56b63c8f85e4df.1571638827.git.eswara.kota@linux.intel.com]

Quoted 8 lines · Dilip Kota
Quoted 7 lines · Bjorn Helgaas
Quoted 3 lines · Andrew Murray
Quoted 2 lines · Dilip Kota

PCIe RC driver on Intel Gateway SoCs have a requirement of changing link width and speed on the fly.

Please add more details about why this is needed. Since you're adding sysfs files, it sounds like it's not actually the *driver* that needs this; it's something in userspace?

Quoted 6 lines

We have use cases to change the link speed and width on the fly. One is EMI check and other is power saving. Some battery backed applications have to switch PCIe link from higher GEN to GEN1 and width to x1. During the cases like external power supply got disconnected or broken. Once external power supply is connected then switch PCIe link to higher GEN and width.

Quoted 3 lines

That sounds plausible, but of course nothing there is specific to the Intel Gateway, so we should implement this generically so it would work on all hardware.

Quoted 8 lines

Agree.

Quoted 5 lines

I'm not sure what the interface should look like -- should it be a low-level interface as you propose where userspace would have to identify each link of interest, or is there some system-wide power/performance knob that could tune all links? Cc'd Rafael and linux-pm in case they have ideas.

To my knowledge sysfs is the appropriate way to go. If there are any other best possible knobs, will be helpful.

I agree sysfs is the right place for it; my question was whether we should have files like:

Quoted 6 lines
   /sys/.../0000:00:1f.3/pcie_speed
   /sys/.../0000:00:1f.3/pcie_width

as I think this patch would add (BTW, please include sample paths like the above in the commit log), or whether there should be a more global thing that would affect all the links in the system.

Sure, i will add them.

Quoted 5 lines

I think the low-level files like you propose would be better because one might want to tune link performance differently for different types of devices and workloads.

Quoted 7 lines

We also have to decide if these files should be associated with the device at the upstream or downstream end of the link. For ASPM, the current proposal [1] has the files at the downstream end on the theory that the GPU, NIC, NVMe device, etc is the user-recognizable one. Also, neither ASPM nor link speed/width make any sense unless there *is* a device at the downstream end, so putting them there automatically makes them visible only when they're useful.

This patch places the speed and width in the host controller directory. /sys/.../xxx.pcie/pcie_speed
/sys/.../xxx.pcie/pcie_width

I agree with you partially,  because i am having couple of points making me to keep speed and width change entries in controller directory:

-- For changing the speed/width with device node, software ends up traversing to the controller from the device and do the operations.
-- Change speed and width are performed at controller level,
-- Keeping speed and width in controller gives a perspective (to the user) of changing
them only once irrespective of no. of devices.
-- For speed and link change in Synopsys PCIe controller, specific registers need to be configured. This prevents or complicates adding the speed and width change functionality in pci-sysfs or pci framework.

Regards,
Dilip

#45 lore raw

Re: [PATCH v4 3/3] pci: intel: Add sysfs attributes to configure pcie link

Quoted 60 lines · Dilip Kota
Quoted 35 lines · Bjorn Helgaas
Quoted 23 lines · Dilip Kota
Quoted 19 lines · Bjorn Helgaas

[+cc Rafael, linux-pm, beginning of discussion at
https://lore.kernel.org/r/d8574605f8e70f41ce1e88ccfb56b63c8f85e4df.1571638827.git.eswara.kota@linux.intel.com]

Quoted 9 lines · Dilip Kota
Quoted 8 lines · Bjorn Helgaas
Quoted 3 lines · Andrew Murray
Quoted 2 lines · Dilip Kota

PCIe RC driver on Intel Gateway SoCs have a requirement of changing link width and speed on the fly.

Please add more details about why this is needed. Since you're adding sysfs files, it sounds like it's not actually the *driver* that needs this; it's something in userspace?

Quoted 6 lines

We have use cases to change the link speed and width on the fly. One is EMI check and other is power saving. Some battery backed applications have to switch PCIe link from higher GEN to GEN1 and width to x1. During the cases like external power supply got disconnected or broken. Once external power supply is connected then switch PCIe link to higher GEN and width.

Quoted 3 lines

That sounds plausible, but of course nothing there is specific to the Intel Gateway, so we should implement this generically so it would work on all hardware.

Quoted 8 lines

Agree.

Quoted 5 lines

I'm not sure what the interface should look like -- should it be a low-level interface as you propose where userspace would have to identify each link of interest, or is there some system-wide power/performance knob that could tune all links? Cc'd Rafael and linux-pm in case they have ideas.

To my knowledge sysfs is the appropriate way to go. If there are any other best possible knobs, will be helpful.

I agree sysfs is the right place for it; my question was whether we should have files like:

Quoted 6 lines
   /sys/.../0000:00:1f.3/pcie_speed
   /sys/.../0000:00:1f.3/pcie_width

as I think this patch would add (BTW, please include sample paths like the above in the commit log), or whether there should be a more global thing that would affect all the links in the system.

Sure, i will add them.

Quoted 5 lines

I think the low-level files like you propose would be better because one might want to tune link performance differently for different types of devices and workloads.

Quoted 7 lines

We also have to decide if these files should be associated with the device at the upstream or downstream end of the link. For ASPM, the current proposal [1] has the files at the downstream end on the theory that the GPU, NIC, NVMe device, etc is the user-recognizable one. Also, neither ASPM nor link speed/width make any sense unless there *is* a device at the downstream end, so putting them there automatically makes them visible only when they're useful.

This patch places the speed and width in the host controller directory. /sys/.../xxx.pcie/pcie_speed
/sys/.../xxx.pcie/pcie_width

Quoted 8 lines

I agree with you partially,  because i am having couple of points making me to keep speed and width change entries in controller directory:

-- For changing the speed/width with device node, software ends up
traversing to the controller from the device and do the operations.
-- Change speed and width are performed at controller level,

The controller is effectively a Root Complex, which may contain several Root Ports. I have the impression that the Synopsys controller only supports a single Root Port, but that's just a detail of the Synopsys implementation. I think it should be possible to configure the width/speed of each Root Port individually.

Quoted 2 lines

-- Keeping speed and width in controller gives a perspective (to the
user) of changing them only once irrespective of no. of devices.

What if there's a switch? If we change the width/speed of the link between the Root Port and the Switch Upstream Port, that doesn't do anything about the links from the Switch Downstream Ports.

Quoted 4 lines

-- For speed and link change in Synopsys PCIe controller, specific
registers need to be configured. This prevents or complicates adding the speed and width change functionality in pci-sysfs or pci framework.

Don't the Link Control and related registers in PCIe spec give us enough control to manage the link width/speed of *all* links, including those from Root Ports and Switch Downstream Ports?

If the Synopsys controller requires controller-specific registers, that sounds to me like it doesn't quite conform to the spec. Maybe that means we would need some sort of quirk or controller callback?

Bjorn

author
#46 lore raw

Re: [PATCH v4 3/3] pci: intel: Add sysfs attributes to configure pcie link

Quoted 73 lines · Bjorn Helgaas
Quoted 58 lines · Dilip Kota
Quoted 35 lines · Bjorn Helgaas
Quoted 23 lines · Dilip Kota
Quoted 19 lines · Bjorn Helgaas

[+cc Rafael, linux-pm, beginning of discussion at
https://lore.kernel.org/r/d8574605f8e70f41ce1e88ccfb56b63c8f85e4df.1571638827.git.eswara.kota@linux.intel.com]

Quoted 9 lines · Dilip Kota
Quoted 8 lines · Bjorn Helgaas
Quoted 3 lines · Andrew Murray
Quoted 2 lines · Dilip Kota

PCIe RC driver on Intel Gateway SoCs have a requirement of changing link width and speed on the fly.

Please add more details about why this is needed. Since you're adding sysfs files, it sounds like it's not actually the *driver* that needs this; it's something in userspace?

Quoted 6 lines

We have use cases to change the link speed and width on the fly. One is EMI check and other is power saving. Some battery backed applications have to switch PCIe link from higher GEN to GEN1 and width to x1. During the cases like external power supply got disconnected or broken. Once external power supply is connected then switch PCIe link to higher GEN and width.

Quoted 3 lines

That sounds plausible, but of course nothing there is specific to the Intel Gateway, so we should implement this generically so it would work on all hardware.

Quoted 8 lines

Agree.

Quoted 5 lines

I'm not sure what the interface should look like -- should it be a low-level interface as you propose where userspace would have to identify each link of interest, or is there some system-wide power/performance knob that could tune all links? Cc'd Rafael and linux-pm in case they have ideas.

To my knowledge sysfs is the appropriate way to go. If there are any other best possible knobs, will be helpful.

I agree sysfs is the right place for it; my question was whether we should have files like:

Quoted 6 lines
    /sys/.../0000:00:1f.3/pcie_speed
    /sys/.../0000:00:1f.3/pcie_width

as I think this patch would add (BTW, please include sample paths like the above in the commit log), or whether there should be a more global thing that would affect all the links in the system.

Sure, i will add them.

Quoted 11 lines

I think the low-level files like you propose would be better because one might want to tune link performance differently for different types of devices and workloads.

We also have to decide if these files should be associated with the device at the upstream or downstream end of the link. For ASPM, the current proposal [1] has the files at the downstream end on the theory that the GPU, NIC, NVMe device, etc is the user-recognizable one. Also, neither ASPM nor link speed/width make any sense unless there *is* a device at the downstream end, so putting them there automatically makes them visible only when they're useful.

This patch places the speed and width in the host controller directory. /sys/.../xxx.pcie/pcie_speed
/sys/.../xxx.pcie/pcie_width

Quoted 8 lines

I agree with you partially,  because i am having couple of points making me to keep speed and width change entries in controller directory:

-- For changing the speed/width with device node, software ends up
traversing to the controller from the device and do the operations.
-- Change speed and width are performed at controller level,

The controller is effectively a Root Complex, which may contain several Root Ports. I have the impression that the Synopsys controller only supports a single Root Port, but that's just a detail of the Synopsys implementation. I think it should be possible to configure the width/speed of each Root Port individually.

Quoted 5 lines
Quoted 2 lines

-- Keeping speed and width in controller gives a perspective (to the
user) of changing them only once irrespective of no. of devices.

What if there's a switch? If we change the width/speed of the link between the Root Port and the Switch Upstream Port, that doesn't do anything about the links from the Switch Downstream Ports.

I missed to evaluate the multiple root port and switch scenarios, thanks for pointing it. Then, placing the link speed and width change entries in the device node will be appropriate. Software will traverse to the respective port or bus through the device node and does the changes.

Quoted 5 lines
Quoted 5 lines

-- For speed and link change in Synopsys PCIe controller, specific
registers need to be configured. This prevents or complicates adding the speed and width change functionality in pci-sysfs or pci framework.

Quoted 7 lines

Don't the Link Control and related registers in PCIe spec give us enough control to manage the link width/speed of *all* links, including those from Root Ports and Switch Downstream Ports?

If the Synopsys controller requires controller-specific registers, that sounds to me like it doesn't quite conform to the spec. Maybe that means we would need some sort of quirk or controller callback?

Yes, Synopsys has specific registers configuration for link width resizing and speed change. I will evaluate the possible mechanism for plugging in the controller specific changes to the framework.

Regards,
Dilip

Quoted 2 lines

Bjorn

#47 lore raw

Re: [PATCH v4 3/3] pci: intel: Add sysfs attributes to configure pcie link

Quoted 95 lines · Dilip Kota
Quoted 73 lines · Bjorn Helgaas
Quoted 58 lines · Dilip Kota
Quoted 35 lines · Bjorn Helgaas
Quoted 23 lines · Dilip Kota
Quoted 19 lines · Bjorn Helgaas

[+cc Rafael, linux-pm, beginning of discussion at
https://lore.kernel.org/r/d8574605f8e70f41ce1e88ccfb56b63c8f85e4df.1571638827.git.eswara.kota@linux.intel.com]

Quoted 9 lines · Dilip Kota
Quoted 8 lines · Bjorn Helgaas
Quoted 3 lines · Andrew Murray
Quoted 2 lines · Dilip Kota

PCIe RC driver on Intel Gateway SoCs have a requirement of changing link width and speed on the fly.

Please add more details about why this is needed. Since you're adding sysfs files, it sounds like it's not actually the *driver* that needs this; it's something in userspace?

Quoted 6 lines

We have use cases to change the link speed and width on the fly. One is EMI check and other is power saving. Some battery backed applications have to switch PCIe link from higher GEN to GEN1 and width to x1. During the cases like external power supply got disconnected or broken. Once external power supply is connected then switch PCIe link to higher GEN and width.

Quoted 3 lines

That sounds plausible, but of course nothing there is specific to the Intel Gateway, so we should implement this generically so it would work on all hardware.

Quoted 8 lines

Agree.

Quoted 5 lines

I'm not sure what the interface should look like -- should it be a low-level interface as you propose where userspace would have to identify each link of interest, or is there some system-wide power/performance knob that could tune all links? Cc'd Rafael and linux-pm in case they have ideas.

To my knowledge sysfs is the appropriate way to go. If there are any other best possible knobs, will be helpful.

I agree sysfs is the right place for it; my question was whether we should have files like:

Quoted 6 lines
    /sys/.../0000:00:1f.3/pcie_speed
    /sys/.../0000:00:1f.3/pcie_width

as I think this patch would add (BTW, please include sample paths like the above in the commit log), or whether there should be a more global thing that would affect all the links in the system.

Sure, i will add them.

Quoted 11 lines

I think the low-level files like you propose would be better because one might want to tune link performance differently for different types of devices and workloads.

We also have to decide if these files should be associated with the device at the upstream or downstream end of the link. For ASPM, the current proposal [1] has the files at the downstream end on the theory that the GPU, NIC, NVMe device, etc is the user-recognizable one. Also, neither ASPM nor link speed/width make any sense unless there *is* a device at the downstream end, so putting them there automatically makes them visible only when they're useful.

This patch places the speed and width in the host controller directory. /sys/.../xxx.pcie/pcie_speed
/sys/.../xxx.pcie/pcie_width

Quoted 8 lines

I agree with you partially,  because i am having couple of points making me to keep speed and width change entries in controller directory:

-- For changing the speed/width with device node, software ends up
traversing to the controller from the device and do the operations.
-- Change speed and width are performed at controller level,

The controller is effectively a Root Complex, which may contain several Root Ports. I have the impression that the Synopsys controller only supports a single Root Port, but that's just a detail of the Synopsys implementation. I think it should be possible to configure the width/speed of each Root Port individually.

Quoted 5 lines
Quoted 2 lines

-- Keeping speed and width in controller gives a perspective (to the
user) of changing them only once irrespective of no. of devices.

What if there's a switch? If we change the width/speed of the link between the Root Port and the Switch Upstream Port, that doesn't do anything about the links from the Switch Downstream Ports.

I missed to evaluate the multiple root port and switch scenarios, thanks for pointing it.
Then, placing the link speed and width change entries in the device node will be appropriate.
Software will traverse to the respective port or bus through the device node and does the changes.

Quoted 9 lines
Quoted 5 lines

-- For speed and link change in Synopsys PCIe controller, specific
registers need to be configured. This prevents or complicates adding the speed and width change functionality in pci-sysfs or pci framework.

Don't the Link Control and related registers in PCIe spec give us enough control to manage the link width/speed of *all* links, including those from Root Ports and Switch Downstream Ports?

Quoted 7 lines
Quoted 3 lines

If the Synopsys controller requires controller-specific registers, that sounds to me like it doesn't quite conform to the spec. Maybe that means we would need some sort of quirk or controller callback?

Yes, Synopsys has specific registers configuration for link width resizing and speed change.
I will evaluate the possible mechanism for plugging in the controller specific changes to the framework.

According to the spec, "Software is permitted to restrict the maximum speed of Link operation and set the perferred Link speed by setting the value in the Target Link Speed field in the Upstream component." - This is the Link Control 2 Register, and a link retrain should then be triggered.

With regards to this proposed sysfs API - I wonder if this implies we should also disable 'Hardware Autonomous Speed Disable' to prevent a link speed change for device specific reasons?

In my view, this means we *can* have a sysfs control for limiting the link speed using standard PCI means - though callbacks and quirks may be needed for host bridge controllers and similar.

With regards to link width, I can't see any obvious software initiated means to change the link width (they are all RO) - though a device can change its own link width so long as it's 'Hardware Autonomous Width Disable' bit is clear. So whilst there may be some benefit for the initial links of a few host bridge controllers that may opt-in to some framework for this - such an API wouldn't benefit the majority of links in a PCI fabric. Perhaps this (width) should be DWC specific.

Thanks,

Andrew Murray

Quoted 5 lines

Regards,
Dilip

Quoted 2 lines

Bjorn

#48 lore raw

Re: [PATCH v4 3/3] pci: intel: Add sysfs attributes to configure pcie link

In reply to Bjorn Helgaas
Quoted 25 lines · Bjorn Helgaas

[+cc Rafael, linux-pm, beginning of discussion at
https://lore.kernel.org/r/d8574605f8e70f41ce1e88ccfb56b63c8f85e4df.1571638827.git.eswara.kota@linux.intel.com]

Quoted 9 lines · Dilip Kota
Quoted 7 lines · Bjorn Helgaas
Quoted 3 lines · Andrew Murray
Quoted 2 lines · Dilip Kota

PCIe RC driver on Intel Gateway SoCs have a requirement of changing link width and speed on the fly.

Please add more details about why this is needed. Since you're adding sysfs files, it sounds like it's not actually the *driver* that needs this; it's something in userspace?

Quoted 6 lines

We have use cases to change the link speed and width on the fly. One is EMI check and other is power saving. Some battery backed applications have to switch PCIe link from higher GEN to GEN1 and width to x1. During the cases like external power supply got disconnected or broken. Once external power supply is connected then switch PCIe link to higher GEN and width.

That sounds plausible, but of course nothing there is specific to the Intel Gateway, so we should implement this generically so it would work on all hardware.

Quoted 5 lines

I'm not sure what the interface should look like -- should it be a low-level interface as you propose where userspace would have to identify each link of interest, or is there some system-wide power/performance knob that could tune all links? Cc'd Rafael and linux-pm in case they have ideas.

Frankly, I need some time to think about this and, in case you are wondering about whether or not it has been discussed with me already, it hasn't.

At this point I can only say that since we have an ASPM interface, which IMO is not fantastic, it may be good to come up with a common link management interface.

Cheers!

#49 lore raw

Re: [PATCH v4 3/3] pci: intel: Add sysfs attributes to configure pcie link

[+cc Heiner for ASPM conversation]

Quoted 32 lines · Rafael J. Wysocki
Quoted 25 lines · Bjorn Helgaas

[+cc Rafael, linux-pm, beginning of discussion at
https://lore.kernel.org/r/d8574605f8e70f41ce1e88ccfb56b63c8f85e4df.1571638827.git.eswara.kota@linux.intel.com]

Quoted 9 lines · Dilip Kota
Quoted 7 lines · Bjorn Helgaas
Quoted 3 lines · Andrew Murray
Quoted 2 lines · Dilip Kota

PCIe RC driver on Intel Gateway SoCs have a requirement of changing link width and speed on the fly.

Please add more details about why this is needed. Since you're adding sysfs files, it sounds like it's not actually the *driver* that needs this; it's something in userspace?

Quoted 6 lines

We have use cases to change the link speed and width on the fly. One is EMI check and other is power saving. Some battery backed applications have to switch PCIe link from higher GEN to GEN1 and width to x1. During the cases like external power supply got disconnected or broken. Once external power supply is connected then switch PCIe link to higher GEN and width.

That sounds plausible, but of course nothing there is specific to the Intel Gateway, so we should implement this generically so it would work on all hardware.

Quoted 5 lines

I'm not sure what the interface should look like -- should it be a low-level interface as you propose where userspace would have to identify each link of interest, or is there some system-wide power/performance knob that could tune all links? Cc'd Rafael and linux-pm in case they have ideas.

Quoted 7 lines

Frankly, I need some time to think about this and, in case you are wondering about whether or not it has been discussed with me already, it hasn't.

At this point I can only say that since we have an ASPM interface, which IMO is not fantastic, it may be good to come up with a common link management interface.

The ASPM interface hasn't been merged yet, so if you have better ideas, now is the time. That one is definitely very low-level, partly because the first use case is working around defects in a specific device.

Some sort of unification of link management does sound like a good idea.

Bjorn

author
#50 lore raw

Re: [PATCH v4 3/3] pci: intel: Add sysfs attributes to configure pcie link

In reply to Andrew Murray

Hi Andrew Murray,

Quoted 169 lines · Andrew Murray
Quoted 155 lines · Dilip Kota

PCIe RC driver on Intel Gateway SoCs have a requirement of changing link width and speed on the fly.
So add the sysfs attributes to show and store the link properties.
Add the respective link resize function in pcie DesignWare framework so that Intel PCIe driver can use during link width configuration on the fly.

---
  drivers/pci/controller/dwc/pcie-designware.c |   9 +++
  drivers/pci/controller/dwc/pcie-designware.h |   3 +
  drivers/pci/controller/dwc/pcie-intel-gw.c   | 112 ++++++++++++++++++++++++++-
  3 files changed, 123 insertions(+), 1 deletion(-)
drivers/pci/controller/dwc/pcie-designware.c +9 -0
index 4c391bfd681a..662fdcb4f2d6 100644
@@ -474,6 +474,15 @@ int dw_pcie_link_up(struct dw_pcie *pci)
  		(!(val & PCIE_PORT_DEBUG1_LINK_IN_TRAINING)));
  }
 
+void dw_pcie_link_width_resize(struct dw_pcie *pci, u32 lane_width)
+{
+	u32 val;
+
+	val =  dw_pcie_readl_dbi(pci, PCIE_PORT_MULTI_LANE_CTRL);
+	val &= ~(PORT_MLTI_LNK_WDTH_CHNG | PORT_MLTI_LNK_WDTH);
+	val |= PORT_MLTI_LNK_WDTH_CHNG | lane_width;
+	dw_pcie_writel_dbi(pci, PCIE_PORT_MULTI_LANE_CTRL, val);
+}
 
  void dw_pcie_upconfig_setup(struct dw_pcie *pci)
  {
drivers/pci/controller/dwc/pcie-designware.h +3 -0
index 3beac10e4a4c..fcf0442341fd 100644
@@ -67,6 +67,8 @@
  #define PCIE_MSI_INTR0_STATUS		0x830
 
  #define PCIE_PORT_MULTI_LANE_CTRL	0x8C0
+#define PORT_MLTI_LNK_WDTH		GENMASK(5, 0)
+#define PORT_MLTI_LNK_WDTH_CHNG		BIT(6)
  #define PORT_MLTI_UPCFG_SUPPORT		BIT(7)
 
  #define PCIE_ATU_VIEWPORT		0x900
@@ -282,6 +284,7 @@ void dw_pcie_write_dbi2(struct dw_pcie *pci, u32 reg, size_t size, u32 val);
  u32 dw_pcie_read_atu(struct dw_pcie *pci, u32 reg, size_t size);
  void dw_pcie_write_atu(struct dw_pcie *pci, u32 reg, size_t size, u32 val);
  int dw_pcie_link_up(struct dw_pcie *pci);
+void dw_pcie_link_width_resize(struct dw_pcie *pci, u32 lane_width);
  void dw_pcie_upconfig_setup(struct dw_pcie *pci);
  void dw_pcie_link_speed_change(struct dw_pcie *pci, bool enable);
  void dw_pcie_link_set_n_fts(struct dw_pcie *pci, u32 n_fts);
drivers/pci/controller/dwc/pcie-intel-gw.c +84 -0
index 9142c70db808..b9be0921671d 100644
@@ -146,6 +146,22 @@ static void intel_pcie_ltssm_disable(struct intel_pcie_port *lpp)
  	pcie_app_wr_mask(lpp, PCIE_APP_CCR_LTSSM_ENABLE, 0, PCIE_APP_CCR);
  }
 
+static const char *pcie_link_gen_to_str(int gen)
+{
+	switch (gen) {
+	case PCIE_LINK_SPEED_GEN1:
+		return "2.5";
+	case PCIE_LINK_SPEED_GEN2:
+		return "5.0";
+	case PCIE_LINK_SPEED_GEN3:
+		return "8.0";
+	case PCIE_LINK_SPEED_GEN4:
+		return "16.0";
+	default:
+		return "???";
+	}
+}
+
  static void intel_pcie_link_setup(struct intel_pcie_port *lpp)
  {
  	u32 val;
@@ -444,6 +460,91 @@ static int intel_pcie_host_setup(struct intel_pcie_port *lpp)
  	return ret;
  }
 
+static ssize_t pcie_link_status_show(struct device *dev,
+				     struct device_attribute *attr, char *buf)
+{
+	struct intel_pcie_port *lpp = dev_get_drvdata(dev);
+	u32 reg, width, gen;
+
+	reg = pcie_rc_cfg_rd(lpp, PCIE_CAP_OFST + PCI_EXP_LNKCTL);
+	width = FIELD_GET(PCI_EXP_LNKSTA_NLW, reg >> 16);
+	gen = FIELD_GET(PCI_EXP_LNKSTA_CLS, reg >> 16);
+
+	if (gen > lpp->max_speed)
+		return -EINVAL;
+
+	return sprintf(buf, "Port %2u Width x%u Speed %s GT/s\n", lpp->id,
+		       width, pcie_link_gen_to_str(gen));
+}
+static DEVICE_ATTR_RO(pcie_link_status);
+
+static ssize_t pcie_speed_store(struct device *dev,
+				struct device_attribute *attr,
+				const char *buf, size_t len)
+{
+	struct intel_pcie_port *lpp = dev_get_drvdata(dev);
+	unsigned long val;
+	int ret;
+
+	ret = kstrtoul(buf, 10, &val);
+	if (ret)
+		return ret;
+
+	if (val > lpp->max_speed)
+		return -EINVAL;
+
+	lpp->link_gen = val;
+	intel_pcie_max_speed_setup(lpp);
+	dw_pcie_link_speed_change(&lpp->pci, false);
+	dw_pcie_link_speed_change(&lpp->pci, true);
+
+	return len;
+}
+static DEVICE_ATTR_WO(pcie_speed);
+
+/*
+ * Link width change on the fly is not always successful.
+ * It also depends on the partner.
+ */
+static ssize_t pcie_width_store(struct device *dev,
+				struct device_attribute *attr,
+				const char *buf, size_t len)
+{
+	struct intel_pcie_port *lpp = dev_get_drvdata(dev);
+	unsigned long val;
+	int ret;
+
+	lpp = dev_get_drvdata(dev);
+
+	ret = kstrtoul(buf, 10, &val);
+	if (ret)
+		return ret;
+
+	if (val > lpp->max_width)
+		return -EINVAL;
+
+	/* HW auto bandwidth negotiation must be enabled */
+	pcie_rc_cfg_wr_mask(lpp, PCI_EXP_LNKCTL_HAWD, 0,
+			    PCIE_CAP_OFST + PCI_EXP_LNKCTL);
+	dw_pcie_link_width_resize(&lpp->pci, val);
+

drivers/pci/controller/dwc/pcie-intel-gw.c static int intel_pcie_host_setup(struct intel_pcie_port *lpp) · line 540

Quoted 10 lines
drivers/pci/controller/dwc/pcie-intel-gw.c +10 -0
+	return len;
+}
+static DEVICE_ATTR_WO(pcie_width);
+
+static struct attribute *pcie_cfg_attrs[] = {
+	&dev_attr_pcie_link_status.attr,
+	&dev_attr_pcie_speed.attr,
+	&dev_attr_pcie_width.attr,
+	NULL,
+};

Is there a reason that these are limited only to the Intel driver and not the wider set of DWC drivers?

Quoted 1 line

Is there anything specific here about the Intel GW driver?

Yes, they need intel_pcie_max_speed_setup() and pcie_link_gen_to_str().
Once intel_pcie_max_speed_setup() moved to DesignWare framework (as per Bjorn Helgaas inputs) and use pcie_link_speed[] array instead of pcie_link_gen_to_str() (as per gustavo pimentel inputs) we can move this to PCIe DesignWare framework or to pci sysfs file.

Regards,
Dilip

Quoted 37 lines

Thanks,

Andrew Murray

Quoted 32 lines
drivers/pci/controller/dwc/pcie-intel-gw.c +17 -1
+ATTRIBUTE_GROUPS(pcie_cfg);
+
+static int intel_pcie_sysfs_init(struct intel_pcie_port *lpp)
+{
+	return devm_device_add_groups(lpp->pci.dev, pcie_cfg_groups);
+}
+
  static void __intel_pcie_remove(struct intel_pcie_port *lpp)
  {
  	intel_pcie_core_irq_disable(lpp);
@@ -490,8 +591,17 @@ static int intel_pcie_rc_init(struct pcie_port *pp)
  {
  	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
  	struct intel_pcie_port *lpp = dev_get_drvdata(pci->dev);
+	int ret;
 
-	return intel_pcie_host_setup(lpp);
+	ret = intel_pcie_host_setup(lpp);
+	if (ret)
+		return ret;
+
+	ret = intel_pcie_sysfs_init(lpp);
+	if (ret)
+		__intel_pcie_remove(lpp);
+
+	return ret;
  }
 
  int intel_pcie_msi_init(struct pcie_port *pp)
Signature · 1 line
2.11.0
#51 lore raw

Re: [PATCH v4 3/3] pci: intel: Add sysfs attributes to configure pcie link

Quoted 53 lines · Dilip Kota

Hi Andrew Murray,

Quoted 50 lines · Andrew Murray
Quoted 36 lines · Dilip Kota

PCIe RC driver on Intel Gateway SoCs have a requirement of changing link width and speed on the fly.
So add the sysfs attributes to show and store the link properties.
Add the respective link resize function in pcie DesignWare framework so that Intel PCIe driver can use during link width configuration on the fly.

drivers/pci/controller/dwc/pcie-intel-gw.c +26 -0
+/*
+ * Link width change on the fly is not always successful.
+ * It also depends on the partner.
+ */
+static ssize_t pcie_width_store(struct device *dev,
+				struct device_attribute *attr,
+				const char *buf, size_t len)
+{
+	struct intel_pcie_port *lpp = dev_get_drvdata(dev);
+	unsigned long val;
+	int ret;
+
+	lpp = dev_get_drvdata(dev);
+
+	ret = kstrtoul(buf, 10, &val);
+	if (ret)
+		return ret;
+
+	if (val > lpp->max_width)
+		return -EINVAL;
+
+	/* HW auto bandwidth negotiation must be enabled */
+	pcie_rc_cfg_wr_mask(lpp, PCI_EXP_LNKCTL_HAWD, 0,
+			    PCIE_CAP_OFST + PCI_EXP_LNKCTL);
+	dw_pcie_link_width_resize(&lpp->pci, val);
+

drivers/pci/controller/dwc/pcie-intel-gw.c static int intel_pcie_host_setup(struct intel_pcie_port *lpp) · line 540

Quoted 10 lines
drivers/pci/controller/dwc/pcie-intel-gw.c +10 -0
+	return len;
+}
+static DEVICE_ATTR_WO(pcie_width);
+
+static struct attribute *pcie_cfg_attrs[] = {
+	&dev_attr_pcie_link_status.attr,
+	&dev_attr_pcie_speed.attr,
+	&dev_attr_pcie_width.attr,
+	NULL,
+};

Is there a reason that these are limited only to the Intel driver and not the wider set of DWC drivers?

Quoted 7 lines
Quoted 1 line

Is there anything specific here about the Intel GW driver?

Yes, they need intel_pcie_max_speed_setup() and pcie_link_gen_to_str(). Once intel_pcie_max_speed_setup() moved to DesignWare framework (as per Bjorn Helgaas inputs) and use pcie_link_speed[] array instead of pcie_link_gen_to_str() (as per gustavo pimentel inputs) we can move this to PCIe DesignWare framework or to pci sysfs file.

I think the key concern here is this: If you introduce sysfs controls that represent generic PCI concepts (such as changing the link speed) - the concept isn't limited to a particular host controller, it's limited to PCI. Therefore the sysfs control should also apply more widely to all PCI controllers. This is important as otherwise you may end up getting a slightly different user interface to achieve the same consequence depending on the host-controller in use.

If each controller driver has a different way of doing things, then it lends itself to having some set of ops that they can all implement. Or perhaps there is a middle-ground solution where this applies just to DWC devices and not all devices.

Thanks,

Andrew Murray

Quoted 39 lines

Regards,
Dilip

Quoted 35 lines

Thanks,

Andrew Murray

Quoted 30 lines
drivers/pci/controller/dwc/pcie-intel-gw.c +17 -1
+ATTRIBUTE_GROUPS(pcie_cfg);
+
+static int intel_pcie_sysfs_init(struct intel_pcie_port *lpp)
+{
+	return devm_device_add_groups(lpp->pci.dev, pcie_cfg_groups);
+}
+
  static void __intel_pcie_remove(struct intel_pcie_port *lpp)
  {
  	intel_pcie_core_irq_disable(lpp);
@@ -490,8 +591,17 @@ static int intel_pcie_rc_init(struct pcie_port *pp)
  {
  	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
  	struct intel_pcie_port *lpp = dev_get_drvdata(pci->dev);
+	int ret;
-	return intel_pcie_host_setup(lpp);
+	ret = intel_pcie_host_setup(lpp);
+	if (ret)
+		return ret;
+
+	ret = intel_pcie_sysfs_init(lpp);
+	if (ret)
+		__intel_pcie_remove(lpp);
+
+	return ret;
  }
  int intel_pcie_msi_init(struct pcie_port *pp)
Signature · 1 line
2.11.0
author
#52 lore raw

Re: [PATCH v4 3/3] pci: intel: Add sysfs attributes to configure pcie link

Quoted 53 lines · Andrew Murray
Quoted 39 lines
Quoted 39 lines
Quoted 26 lines
drivers/pci/controller/dwc/pcie-intel-gw.c +26 -0
+/*
+ * Link width change on the fly is not always successful.
+ * It also depends on the partner.
+ */
+static ssize_t pcie_width_store(struct device *dev,
+				struct device_attribute *attr,
+				const char *buf, size_t len)
+{
+	struct intel_pcie_port *lpp = dev_get_drvdata(dev);
+	unsigned long val;
+	int ret;
+
+	lpp = dev_get_drvdata(dev);
+
+	ret = kstrtoul(buf, 10, &val);
+	if (ret)
+		return ret;
+
+	if (val > lpp->max_width)
+		return -EINVAL;
+
+	/* HW auto bandwidth negotiation must be enabled */
+	pcie_rc_cfg_wr_mask(lpp, PCI_EXP_LNKCTL_HAWD, 0,
+			    PCIE_CAP_OFST + PCI_EXP_LNKCTL);
+	dw_pcie_link_width_resize(&lpp->pci, val);
+

drivers/pci/controller/dwc/pcie-intel-gw.c static int intel_pcie_host_setup(struct intel_pcie_port *lpp) · line 540

Quoted 10 lines
drivers/pci/controller/dwc/pcie-intel-gw.c +10 -0
+	return len;
+}
+static DEVICE_ATTR_WO(pcie_width);
+
+static struct attribute *pcie_cfg_attrs[] = {
+	&dev_attr_pcie_link_status.attr,
+	&dev_attr_pcie_speed.attr,
+	&dev_attr_pcie_width.attr,
+	NULL,
+};

Is there a reason that these are limited only to the Intel driver and not the wider set of DWC drivers?

Quoted 6 lines
Quoted 1 line

Is there anything specific here about the Intel GW driver?

Yes, they need intel_pcie_max_speed_setup() and pcie_link_gen_to_str(). Once intel_pcie_max_speed_setup() moved to DesignWare framework (as per Bjorn Helgaas inputs) and use pcie_link_speed[] array instead of pcie_link_gen_to_str() (as per gustavo pimentel inputs) we can move this to PCIe DesignWare framework or to pci sysfs file.

I think the key concern here is this: If you introduce sysfs controls that represent generic PCI concepts (such as changing the link speed) - the concept isn't limited to a particular host controller, it's limited to PCI. Therefore the sysfs control should also apply more widely to all PCI controllers. This is important as otherwise you may end up getting a slightly different user interface to achieve the same consequence depending on the host-controller in use.

Quoted 4 lines

If each controller driver has a different way of doing things, then it lends itself to having some set of ops that they can all implement. Or perhaps there is a middle-ground solution where this applies just to DWC devices and not all devices.

I see the better way is to move the implementation to PCIe DesignWare framework because of the registers programming.

Regards,
Dilip

Quoted 5 lines

Thanks,

Andrew Murray

Quoted 1 line
#53 lore raw
In reply to Dilip Kota

Hi,

Quoted 1 line · Dilip Kota

Intel PCIe is synopsys based controller utilizes the Designware

Please do this general replacement in all your patches.

s/synopsys/Synopsys

and

s/Designware/DesignWare

Quoted 35 lines

framework for host initialization and intel application specific register configurations.

Changes on v4:
Add lane resizing API in PCIe DesignWare driver. Intel PCIe driver uses it for lane resizing which
is being exposed through sysfs attributes.
Add Intel PCIe sysfs attributes is in separate patch. Address review comments given on v3.

Changes on v3:
Compared to v2, map_irq() patch is removed as it is no longer
required for Intel PCIe driver. Intel PCIe driver does platform specific interrupt configuration during core initialization. So changed the subject line too.
Address v2 review comments for DT binding and PCIe driver

Dilip Kota (3):
dt-bindings: PCI: intel: Add YAML schemas for the PCIe RC controller
dwc: PCI: intel: PCIe RC controller driver
pci: intel: Add sysfs attributes to configure pcie link

 .../devicetree/bindings/pci/intel-gw-pcie.yaml     | 135 ++++
 drivers/pci/controller/dwc/Kconfig                 |  10 +
 drivers/pci/controller/dwc/Makefile                |   1 +
 drivers/pci/controller/dwc/pcie-designware.c       |  43 ++
 drivers/pci/controller/dwc/pcie-designware.h       |  15 +
 drivers/pci/controller/dwc/pcie-intel-gw.c         | 700 +++++++++++++++
 include/uapi/linux/pci_regs.h                      |   1 +
 7 files changed, 905 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml
 create mode 100644 drivers/pci/controller/dwc/pcie-intel-gw.c
Signature · 1 line
2.11.0
author
#54 lore raw

Hi Gustavo Pimentel,

Quoted 6 lines · Gustavo Pimentel

Hi,

On Mon, Oct 21, 2019 at 7:39:17, Dilip Kota <[email protected]> wrote:

Quoted 8 lines
Quoted 1 line

Intel PCIe is synopsys based controller utilizes the Designware

Please do this general replacement in all your patches.

s/synopsys/Synopsys

and

s/Designware/DesignWare

Sure, i will update it in the next patch version.
(In the all other patches naming is proper, i got missed it here.) I will ensure and take care of it.

Regards,
Dilip

Quoted 37 lines
Quoted 36 lines

framework for host initialization and intel application specific register configurations.

Changes on v4:
Add lane resizing API in PCIe DesignWare driver. Intel PCIe driver uses it for lane resizing which
is being exposed through sysfs attributes.
Add Intel PCIe sysfs attributes is in separate patch. Address review comments given on v3.

Changes on v3:
Compared to v2, map_irq() patch is removed as it is no longer
required for Intel PCIe driver. Intel PCIe driver does platform specific interrupt configuration during core initialization. So changed the subject line too.
Address v2 review comments for DT binding and PCIe driver

Dilip Kota (3):
dt-bindings: PCI: intel: Add YAML schemas for the PCIe RC controller
dwc: PCI: intel: PCIe RC controller driver
pci: intel: Add sysfs attributes to configure pcie link

  .../devicetree/bindings/pci/intel-gw-pcie.yaml     | 135 ++++
  drivers/pci/controller/dwc/Kconfig                 |  10 +
  drivers/pci/controller/dwc/Makefile                |   1 +
  drivers/pci/controller/dwc/pcie-designware.c       |  43 ++
  drivers/pci/controller/dwc/pcie-designware.h       |  15 +
  drivers/pci/controller/dwc/pcie-intel-gw.c         | 700 +++++++++++++++
  include/uapi/linux/pci_regs.h                      |   1 +
  7 files changed, 905 insertions(+)

create mode 100644 Documentation/devicetree/bindings/pci/intel-gw-pcie.yaml create mode 100644 drivers/pci/controller/dwc/pcie-intel-gw.c

Signature · 1 line
2.11.0