Add Rockchip 3568001/2 errata workarounds and enable ITS on RK356x
Subsystems (6)
cover Add Rockchip 3568001/2 errata workarounds and enable ITS on RK356x
Enable GIC ITS support on Rockchip RK3566/RK3568 SoCs by adding necessary GIC erratum workarounds and moving RK356x PCIe MSI to use ITS instead of MBI.
Dmitry Osipenko (4):
irqchip/gic-v3: Add Rockchip 3568002 erratum workaround
arm64: dts: rockchip: rk356x: Add dma-noncoherent property to GIC node
arm64: dts: rockchip: rk356x: Add MSI controller node
arm64: dts: rockchip: rk356x: Move PCIe MSI to use GIC ITS instead of
MBI
Documentation/arch/arm64/silicon-errata.rst | 2 ++ arch/arm64/Kconfig | 9 ++++++++ arch/arm64/boot/dts/rockchip/rk356x-base.dtsi | 14 ++++++++++- drivers/irqchip/irq-gic-v3-its.c | 23 ++++++++++++++++++- 4 files changed, 46 insertions(+), 2 deletions(-)
Signature · 1 line
2.48.1
PATCH 1/4 irqchip/gic-v3: Add Rockchip 3568002 erratum workaround
Rockchip RK3566/RK3568 GIC600 integration has DDR addressing limited to first 4GB of DRAM. Rockchip assigned Erratum ID #3568002 for this issue. Add driver quirk for this Rockchip GIC Erratum.
Note, that the 0x0201743b ID is not Rockchip 356x specific and thus there is an extra of_machine_is_compatible() check. Rockchip 3588 uses same ID and it is not affected by this errata.
--- Documentation/arch/arm64/silicon-errata.rst | 2 ++ arch/arm64/Kconfig | 9 ++++++++ drivers/irqchip/irq-gic-v3-its.c | 23 ++++++++++++++++++++- 3 files changed, 33 insertions(+), 1 deletion(-)
Documentation/arch/arm64/silicon-errata.rst +2 -0
@@ -284,6 +284,8 @@ stable kernels. +----------------+-----------------+-----------------+-----------------------------+ | Rockchip | RK3588 | #3588001 | ROCKCHIP_ERRATUM_3588001 | +----------------+-----------------+-----------------+-----------------------------+ +| Rockchip | RK3568 | #3568002 | ROCKCHIP_ERRATUM_3568002 | ++----------------+-----------------+-----------------+-----------------------------+ +----------------+-----------------+-----------------+-----------------------------+ | Fujitsu | A64FX | E#010001 | FUJITSU_ERRATUM_010001 | +----------------+-----------------+-----------------+-----------------------------+
arch/arm64/Kconfig +9 -0
@@ -1302,6 +1302,15 @@ config NVIDIA_CARMEL_CNP_ERRATUM If unsure, say Y. +config ROCKCHIP_ERRATUM_3568002 + bool "Rockchip 3568002: can not support DDR addresses higher than 4G" + default y + help + The Rockchip RK3566 and RK3568 GIC600 SoC integrations have DDR + addressing limited to first 4GB. + + If unsure, say Y. + config ROCKCHIP_ERRATUM_3588001 bool "Rockchip 3588001: GIC600 can not support shareability attributes" default y
drivers/irqchip/irq-gic-v3-its.c +22 -1
@@ -205,13 +205,15 @@ static DEFINE_IDA(its_vpeid_ida);#define gic_data_rdist_rd_base () (gic_data_rdist()->rd_base)#define gic_data_rdist_vlpi_base () (gic_data_rdist_rd_base() + SZ_128K) +static gfp_t gfp_flags_quirk; +static struct page *its_alloc_pages_node (int node,gfp_t gfp,unsigned int order) {struct page *page;int ret =0 ; - page =alloc_pages_node (node, gfp, order); + page =alloc_pages_node (node, gfp | gfp_flags_quirk, order);if (!page)return NULL ;
@@ -4887,6 +4889,17 @@ static bool __maybe_unused its_enable_quirk_hip09_162100801(void *data)return true; } +static bool __maybe_unused its_enable_rk3568002 (void *data) +{ +if (!of_machine_is_compatible ("rockchip,rk3566" ) && + !of_machine_is_compatible ("rockchip,rk3568" )) +return false; + + gfp_flags_quirk |=GFP_DMA32 ; + +return true; +} + static const struct gic_quirk its_quirks[] = { #ifdef CONFIG_CAVIUM_ERRATUM_22375 {
@@ -4954,6 +4967,14 @@ static const struct gic_quirk its_quirks[] = { .property = "dma-noncoherent", .init = its_set_non_coherent, }, +#ifdef CONFIG_ROCKCHIP_ERRATUM_3568002 + { + .desc = "ITS: Rockchip erratum RK3568002", + .iidr = 0x0201743b, + .mask = 0xffffffff, + .init = its_enable_rk3568002, + }, +#endif { } };
Signature · 7 lines
2.48.1 _______________________________________________ Linux-rockchip mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-rockchip
Re: [PATCH v1 1/4] irqchip/gic-v3: Add Rockchip 3568002 erratum workaround
Quoted 4 lines · Dmitry Osipenko
Rockchip RK3566/RK3568 GIC600 integration has DDR addressing limited to first 4GB of DRAM. Rockchip assigned Erratum ID #3568002 for this issue. Add driver quirk for this Rockchip GIC Erratum.
Thanks for taking the time to submit this. It only took 5 years for this erratum to be published...
However, my understanding of this issue is that the integration is limited to the first 32bit of physical address space, not the first 32bit of RAM. If the memory is placed as physical address 0, then they represent the same space. But this is still an important distinction.
Quoted 4 lines
Note, that the
0x0201743bID is not Rockchip 356x specific and thus there is an extraof_machine_is_compatible()check. Rockchip 3588 uses same ID and it is not affected by this errata.
This ID is that of ARM's GIC600, which is a very common GICv3 implementation, and is not Rockchip-specific. Please capture this in the commit message.
Quoted 28 lines
Signed-off-by Dmitry Osipenko <[email protected]>--- Documentation/arch/arm64/silicon-errata.rst | 2 ++ arch/arm64/Kconfig | 9 ++++++++ drivers/irqchip/irq-gic-v3-its.c | 23 ++++++++++++++++++++- 3 files changed, 33 insertions(+), 1 deletion(-)Documentation/arch/arm64/silicon-errata.rst +2
-0@@ -284,6 +284,8 @@ stable kernels. +----------------+-----------------+-----------------+-----------------------------+ | Rockchip | RK3588 | #3588001 | ROCKCHIP_ERRATUM_3588001 | +----------------+-----------------+-----------------+-----------------------------+ +| Rockchip | RK3568 | #3568002 | ROCKCHIP_ERRATUM_3568002 | ++----------------+-----------------+-----------------+-----------------------------+ +----------------+-----------------+-----------------+-----------------------------+ | Fujitsu | A64FX | E#010001 | FUJITSU_ERRATUM_010001 | +----------------+-----------------+-----------------+-----------------------------+arch/arm64/Kconfig +0
-0@@ -1302,6 +1302,15 @@ config NVIDIA_CARMEL_CNP_ERRATUM If unsure, say Y.
arch/arm64/Kconfig config NVIDIA_CARMEL_CNP_ERRATUM · line 1313
Quoted 10 lines
arch/arm64/Kconfig +9
-0+config ROCKCHIP_ERRATUM_3568002 + bool "Rockchip 3568002: can not support DDR addresses higher than 4G" + default y + help + The Rockchip RK3566 and RK3568 GIC600 SoC integrations have DDR + addressing limited to first 4GB. + + If unsure, say Y. +
s/DDR addresses/physical addresses/
Quoted 47 lines
arch/arm64/Kconfig +0
-0config ROCKCHIP_ERRATUM_3588001 bool "Rockchip 3588001: GIC600 can not support shareability attributes" default ydrivers/irqchip/irq-gic-v3-its.c +15
-1@@ -205,13 +205,15 @@ static DEFINE_IDA(its_vpeid_ida);#define gic_data_rdist_rd_base () (gic_data_rdist()->rd_base)#define gic_data_rdist_vlpi_base () (gic_data_rdist_rd_base() + SZ_128K) +static gfp_t gfp_flags_quirk; +static struct page *its_alloc_pages_node (int node,gfp_t gfp,unsigned int order) {struct page *page;int ret =0 ; - page =alloc_pages_node (node, gfp, order); + page =alloc_pages_node (node, gfp | gfp_flags_quirk, order);if (!page)return NULL ;@@ -4887,6 +4889,17 @@ static bool __maybe_unused its_enable_quirk_hip09_162100801(void *data)return true; } +static bool __maybe_unused its_enable_rk3568002 (void *data) +{ +if (!of_machine_is_compatible ("rockchip,rk3566" ) && + !of_machine_is_compatible ("rockchip,rk3568" )) +return false; + + gfp_flags_quirk |=GFP_DMA32 ; + +return true; +} + static const struct gic_quirk its_quirks[] = { #ifdef CONFIG_CAVIUM_ERRATUM_22375 {@@ -4954,6 +4967,14 @@ static const struct gic_quirk its_quirks[] = { .property = "dma-noncoherent", .init = its_set_non_coherent, }, +#ifdef CONFIG_ROCKCHIP_ERRATUM_3568002
drivers/irqchip/irq-gic-v3-its.c static const struct gic_quirk its_quirks[] = { · line 4980
Quoted 10 lines
drivers/irqchip/irq-gic-v3-its.c +7
-0+ { + .desc = "ITS: Rockchip erratum RK3568002", + .iidr = 0x0201743b, + .mask = 0xffffffff, + .init = its_enable_rk3568002, + }, +#endif { } };
Another thing is that this patch conflates ITS and redistributors. As it turns out, we use the same allocator for both, but they are distinct architectural concepts, even if GIC600 is a monolithic implementation. It is OK for now, but it will have to be revisited if we ever move the redistributor management outside of the ITS driver.
With the other comments addressed:
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Re: [PATCH v1 1/4] irqchip/gic-v3: Add Rockchip 3568002 erratum workaround
Quoted 9 lines · Marc Zyngier
Quoted 4 lines · Dmitry Osipenko
Rockchip RK3566/RK3568 GIC600 integration has DDR addressing limited to first 4GB of DRAM. Rockchip assigned Erratum ID #3568002 for this issue. Add driver quirk for this Rockchip GIC Erratum.
Thanks for taking the time to submit this. It only took 5 years for this erratum to be published...
The erratum document itself actually is dated by 5 years ago. Only wish the doc was made publicly available, which would've accelerated the upstreaming process.
Quoted 4 lines
However, my understanding of this issue is that the integration is limited to the first 32bit of physical address space, not the first 32bit of RAM. If the memory is placed as physical address 0, then they represent the same space. But this is still an important distinction.
Indeed, will correct the description in v2.
Quoted 7 lines
Quoted 3 lines
Note, that the
0x0201743bID is not Rockchip 356x specific and thus there is an extraof_machine_is_compatible()check. Rockchip 3588 uses same ID and it is not affected by this errata.This ID is that of ARM's GIC600, which is a very common GICv3 implementation, and is not Rockchip-specific. Please capture this in the commit message.
Ack
...
Quoted 11 lines
arch/arm64/Kconfig config NVIDIA_CARMEL_CNP_ERRATUM · line 1313
Quoted 9 lines
arch/arm64/Kconfig +9
-0+config ROCKCHIP_ERRATUM_3568002 + bool "Rockchip 3568002: can not support DDR addresses higher than 4G" + default y + help + The Rockchip RK3566 and RK3568 GIC600 SoC integrations have DDR + addressing limited to first 4GB. + + If unsure, say Y. +s/DDR addresses/physical addresses/
Ack
...
Quoted 9 lines
Another thing is that this patch conflates ITS and redistributors. As it turns out, we use the same allocator for both, but they are distinct architectural concepts, even if GIC600 is a monolithic implementation. It is OK for now, but it will have to be revisited if we ever move the redistributor management outside of the ITS driver.
With the other comments addressed:
Acked-by Marc Zyngier <[email protected]>
Thanks for the review!
Signature · 7 lines
Best regards, Dmitry _______________________________________________ Linux-rockchip mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-rockchip
Marc Zyngier
maintainerRe: [PATCH v1 1/4] irqchip/gic-v3: Add Rockchip 3568002 erratum workaround
Quoted 9 lines · Dmitry Osipenko
Quoted 7 lines · Marc Zyngier
Quoted 4 lines · Dmitry Osipenko
Rockchip RK3566/RK3568 GIC600 integration has DDR addressing limited to first 4GB of DRAM. Rockchip assigned Erratum ID #3568002 for this issue. Add driver quirk for this Rockchip GIC Erratum.
Quoted 6 lines
Quoted 2 lines
Thanks for taking the time to submit this. It only took 5 years for this erratum to be published...
The erratum document itself actually is dated by 5 years ago. Only wish the doc was made publicly available, which would've accelerated the upstreaming process.
The funny thing is that RockChip was very public about the issue in 2019, but refused to acknowledge that it was a bug (see the list archives). 5 years lost with bad performance and bad upstream support, only to finally admit the bleeding obvious.
On the other hand, we're getting close to 10 years of rk3399, and the botched integration of GIC500 still hasn't been officially disclosed. I guess this counts as progress ;-).
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
PATCH 2/4 arm64: dts: rockchip: rk356x: Add dma-noncoherent property to GIC node
In reply to Dmitry OsipenkoRockchip 3566/8 SoCs are affected by the GIC integration issue where GIC reports that it supports programmable shareability in a feature register, while in fact it doesn't support this feature. Rockchip assigned Errata ID #3568001 for the issue.
Add dma-noncoherent GIC property, denoting that a SW quirk is required for the GIC.
--- arch/arm64/boot/dts/rockchip/rk356x-base.dtsi | 1 + 1 file changed, 1 insertion(+)
arch/arm64/boot/dts/rockchip/rk356x-base.dtsi +1 -0
@@ -284,6 +284,7 @@ gic: interrupt-controller@fd400000 { mbi-alias = <0x0 0xfd410000>; mbi-ranges = <296 24>; msi-controller; + dma-noncoherent; }; usb_host0_ehci: usb@fd800000 {
Signature · 7 lines
2.48.1 _______________________________________________ Linux-rockchip mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-rockchip
PATCH 3/4 arm64: dts: rockchip: rk356x: Add MSI controller node
In reply to Dmitry OsipenkoRockchip 356x SoC's GIC has two hardware integration issues that affect MSI functionality of the GIC. Previously, both these GIC limitations were worked around by using MBI for MSI instead of ITS because kernel GIC driver didn't have necessary quirks.
The first limitation is about RK356x GIC not supporting programmable shareability. Rockchip assigned Errata ID #3568001 for this issue.
Second limitation is about GIC AXI master interface addressing only first 4GB of DRAM. Rockchip assigned Errata ID #3568002 for this issue.
Now that kernel supports quirks for both of the erratums, add MSI controller node to RK356x device-tree.
--- arch/arm64/boot/dts/rockchip/rk356x-base.dtsi | 11 +++++++++++ 1 file changed, 11 insertions(+)
arch/arm64/boot/dts/rockchip/rk356x-base.dtsi +11 -0
@@ -284,7 +284,18 @@ gic: interrupt-controller@fd400000 { mbi-alias = <0x0 0xfd410000>; mbi-ranges = <296 24>; msi-controller; + ranges; + #address-cells = <2>; + #size-cells = <2>; dma-noncoherent; + + its: msi-controller@fd440000 { + compatible = "arm,gic-v3-its"; + reg = <0x0 0xfd440000 0 0x20000>; + dma-noncoherent; + msi-controller; + #msi-cells = <1>; + }; }; usb_host0_ehci: usb@fd800000 {
Signature · 7 lines
2.48.1 _______________________________________________ Linux-rockchip mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-rockchip
Marc Zyngier
maintainerRe: [PATCH v1 3/4] arm64: dts: rockchip: rk356x: Add MSI controller node
Quoted 33 lines · Dmitry Osipenko
Rockchip 356x SoC's GIC has two hardware integration issues that affect MSI functionality of the GIC. Previously, both these GIC limitations were worked around by using MBI for MSI instead of ITS because kernel GIC driver didn't have necessary quirks.
The first limitation is about RK356x GIC not supporting programmable shareability. Rockchip assigned Errata ID #3568001 for this issue.
Second limitation is about GIC AXI master interface addressing only first 4GB of DRAM. Rockchip assigned Errata ID #3568002 for this issue.
Now that kernel supports quirks for both of the erratums, add MSI controller node to RK356x device-tree.
Signed-off-by Dmitry Osipenko <[email protected]>--- arch/arm64/boot/dts/rockchip/rk356x-base.dtsi | 11 +++++++++++ 1 file changed, 11 insertions(+)arch/arm64/boot/dts/rockchip/rk356x-base.dtsi +4
-0@@ -284,7 +284,18 @@ gic: interrupt-controller@fd400000 { mbi-alias = <0x0 0xfd410000>; mbi-ranges = <296 24>; msi-controller; + ranges; + #address-cells = <2>; + #size-cells = <2>; dma-noncoherent; +
arch/arm64/boot/dts/rockchip/rk356x-base.dtsi gic: interrupt-controller@fd400000 { · line 301
Quoted 10 lines
arch/arm64/boot/dts/rockchip/rk356x-base.dtsi +7
-0+ its: msi-controller@fd440000 { + compatible = "arm,gic-v3-its"; + reg = <0x0 0xfd440000 0 0x20000>; + dma-noncoherent; + msi-controller; + #msi-cells = <1>; + }; }; usb_host0_ehci: usb@fd800000 {
You can merge this patch with the previous one. Marking the GIC non-coherent is pointless if no ITS is available, because there is no point in allocating memory for them.
Thanks,
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Re: [PATCH v1 3/4] arm64: dts: rockchip: rk356x: Add MSI controller node
Quoted 44 lines · Marc Zyngier
Quoted 42 lines · Dmitry Osipenko
Rockchip 356x SoC's GIC has two hardware integration issues that affect MSI functionality of the GIC. Previously, both these GIC limitations were worked around by using MBI for MSI instead of ITS because kernel GIC driver didn't have necessary quirks.
The first limitation is about RK356x GIC not supporting programmable shareability. Rockchip assigned Errata ID #3568001 for this issue.
Second limitation is about GIC AXI master interface addressing only first 4GB of DRAM. Rockchip assigned Errata ID #3568002 for this issue.
Now that kernel supports quirks for both of the erratums, add MSI controller node to RK356x device-tree.
Signed-off-by Dmitry Osipenko <[email protected]>--- arch/arm64/boot/dts/rockchip/rk356x-base.dtsi | 11 +++++++++++ 1 file changed, 11 insertions(+)arch/arm64/boot/dts/rockchip/rk356x-base.dtsi +11
-0@@ -284,7 +284,18 @@ gic: interrupt-controller@fd400000 { mbi-alias = <0x0 0xfd410000>; mbi-ranges = <296 24>; msi-controller; + ranges; + #address-cells = <2>; + #size-cells = <2>; dma-noncoherent; + + its: msi-controller@fd440000 { + compatible = "arm,gic-v3-its"; + reg = <0x0 0xfd440000 0 0x20000>; + dma-noncoherent; + msi-controller; + #msi-cells = <1>; + }; };
Quoted 5 lines
arch/arm64/boot/dts/rockchip/rk356x-base.dtsi gic: interrupt-controller@fd400000 { · line 301
Quoted 1 line
arch/arm64/boot/dts/rockchip/rk356x-base.dtsi +0
-0usb_host0_ehci: usb@fd800000 {You can merge this patch with the previous one. Marking the GIC non-coherent is pointless if no ITS is available, because there is no point in allocating memory for them.
Ack
Signature · 7 lines
Best regards, Dmitry _______________________________________________ Linux-rockchip mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-rockchip
PATCH 4/4 arm64: dts: rockchip: rk356x: Move PCIe MSI to use GIC ITS instead of MBI
In reply to Dmitry OsipenkoRockchip 356x device-tree now supports GIC ITS. Move PCIe controller's MSI to use ITS instead of MBI. This removes extra CPU overhead of handling PCIe MBIs by letting GIC's ITS to serve the PCIe MSIs.
--- arch/arm64/boot/dts/rockchip/rk356x-base.dtsi | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/arm64/boot/dts/rockchip/rk356x-base.dtsi +1 -1
@@ -969,7 +969,7 @@ pcie2x1: pcie@fe260000 { num-ib-windows = <6>; num-ob-windows = <2>; max-link-speed = <2>; - msi-map = <0x0 &gic 0x0 0x1000>; + msi-map = <0x0 &its 0x0 0x1000>; num-lanes = <1>; phys = <&combphy2 PHY_TYPE_PCIE>; phy-names = "pcie-phy";
Signature · 7 lines
2.48.1 _______________________________________________ Linux-rockchip mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-rockchip