Skip to contentSkip to conversation

mips: bmips: improve handling of RAC and CBR addr

PATCH In review Christian Marangi 2 participants 6 messages Jun 20, 2024 – Jun 27, 2024

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

v8 cover mips: bmips: improve handling of RAC and CBR addr

Hi,

this simple series improve handling of RAC and CBR address and try to upstream these simple patch we have in OpenWrt for a while.

The first patch fix a straight kernel panic where some Bootloader might enable RAC but misconfigure the CBR address. The current logic only check if RAC is enabled but doesn't verify if the CBR address is usable.

The DMA sync function cause a kernel panic for invalid write. (as CBR is 0 or something like 0xa)

The second is preparation for making the CBR address configurable in DT. Since this address doesn't change, we can cache it and reference it with a local variable instead of calling the register to access the value.

The 4th patch make it configurable with 2 DT property, one to actually set the reg and the other to force set it.

The first property is used when CBR is set to 0. The second property is to force it if the Bootloader sets it to something wrong.

If the CBR value is not 0 and is not forced with the second property a WARN is printed and the DT value is ignored.

The 4th patch enable RAC on BMIPS4350.

These has been tested on BCM6358 (HG556a) and BCM6368 (VH4032N) and reported correct functionality.

Changes v8:
- Drop fix patch (applied to mips-fixes)
- Rework CBR cache patch to handle bcm47xx not compiling smp-bmips.c Changes v7:
- Add ACK and Reviewed-by tag for dt patch from v5 Changes v6:
- Add missing patch that got lost in v5
- Fix missing header for legacy bcm47xx
- Fix compilation error with gcc 10.2.1
Changes v5:
- Add Ack tags
- Improve DT descriptions as suggested by Conor
Changes v4:
- Fix compilation error with legacy brcm target
- Improve property description in DT commit (give
CBR meaning and drop reference to linux functions)
- Use only __read_mostly as we can't add variable to multiple data sections
- In patch 4 use local cbr variable instead of global one.
Changes v3:
- Drop broken-cbr-reg property
- Fix anyOf+const with enum
Changes v2:
- Prefix brcm vendor in the added property
- Drop last patch (cpu switch from DMA sync)
- Validate CBR addr from DT to be outside DRAM
- Reduce indentation in DT CBR check
- Reduce delta and use local variable for CBR where possible
- Fix and improve typo and spelling mistake
- Use 0xf instead of 0xa for BCM6358 RAC enable

Christian Marangi (3):
mips: bmips: rework and cache CBR addr handling
dt-bindings: mips: brcm: Document brcm,bmips-cbr-reg property
mips: bmips: setup: make CBR address configurable

Daniel González Cabanelas (1):
mips: bmips: enable RAC on BMIPS4350

 .../devicetree/bindings/mips/brcm/soc.yaml    | 24 +++++++++++++
 arch/mips/bcm47xx/prom.c                      |  3 ++
 arch/mips/bcm47xx/setup.c                     |  8 +++++
 arch/mips/bcm63xx/prom.c                      |  3 ++
 arch/mips/bcm63xx/setup.c                     |  8 +++++
 arch/mips/bmips/dma.c                         |  2 +-
 arch/mips/bmips/setup.c                       | 35 +++++++++++++++++--
 arch/mips/include/asm/bmips.h                 |  1 +
 arch/mips/kernel/smp-bmips.c                  | 22 ++++++++++--
 9 files changed, 101 insertions(+), 5 deletions(-)
Signature · 1 line
2.45.1
#1 lore raw

PATCH v8 1/4 mips: bmips: rework and cache CBR addr handling

Rework the handling of the CBR address and cache it. This address doesn't change and can be cached instead of reading the register every time.

This is in preparation of permitting to tweak the CBR address in DT with broken SoC or bootloader.

bmips_cbr_addr is defined in setup.c for each arch to keep compatibility with legacy brcm47xx/brcm63xx and generic BMIPS target.

---
 arch/mips/bcm47xx/prom.c      | 3 +++
 arch/mips/bcm47xx/setup.c     | 4 ++++
 arch/mips/bcm63xx/prom.c      | 3 +++
 arch/mips/bcm63xx/setup.c     | 4 ++++
 arch/mips/bmips/dma.c         | 2 +-
 arch/mips/bmips/setup.c       | 7 ++++++-
 arch/mips/include/asm/bmips.h | 1 +
 arch/mips/kernel/smp-bmips.c  | 4 ++--
 8 files changed, 24 insertions(+), 4 deletions(-)
arch/mips/bcm47xx/prom.c +3 -0
index 58fb7c2dc3b8..66e3ee2b04e6 100644
@@ -32,6 +32,7 @@
 #include <linux/ssb/ssb_driver_chipcommon.h>
 #include <linux/ssb/ssb_regs.h>
 #include <linux/smp.h>
+#include <asm/bmips.h>
 #include <asm/bootinfo.h>
 #include <bcm47xx.h>
 #include <bcm47xx_board.h>
@@ -110,6 +111,8 @@ static __init void prom_init_mem(void)
 
 void __init prom_init(void)
 {
+	/* Cache CBR addr before CPU/DMA setup */
+	bmips_cbr_addr = BMIPS_GET_CBR();
 	prom_init_mem();
 	setup_8250_early_printk_port(CKSEG1ADDR(BCM47XX_SERIAL_ADDR), 0, 0);
 }
arch/mips/bcm47xx/setup.c +4 -0
index 94bf839576c1..2f1ee0560aba 100644
@@ -37,6 +37,7 @@
 #include <linux/ssb/ssb.h>
 #include <linux/ssb/ssb_embedded.h>
 #include <linux/bcma/bcma_soc.h>
+#include <asm/bmips.h>
 #include <asm/bootinfo.h>
 #include <asm/idle.h>
 #include <asm/prom.h>
@@ -45,6 +46,9 @@
 #include <bcm47xx.h>
 #include <bcm47xx_board.h>
 
+/* CBR addr doesn't change and we can cache it */
+void __iomem *bmips_cbr_addr __read_mostly;
+
 union bcm47xx_bus bcm47xx_bus;
 EXPORT_SYMBOL(bcm47xx_bus);
 
arch/mips/bcm63xx/prom.c +3 -0
index c3a2ea62c5c3..f21dd168171a 100644
@@ -22,6 +22,9 @@ void __init prom_init(void)
 {
 	u32 reg, mask;
 
+	/* Cache CBR addr before CPU/DMA setup */
+	bmips_cbr_addr = BMIPS_GET_CBR();
+
 	bcm63xx_cpu_init();
 
 	/* stop any running watchdog */
arch/mips/bcm63xx/setup.c +4 -0
index c13ddb544a23..16ea8945ae3a 100644
@@ -12,6 +12,7 @@
 #include <linux/memblock.h>
 #include <linux/ioport.h>
 #include <linux/pm.h>
+#include <asm/bmips.h>
 #include <asm/bootinfo.h>
 #include <asm/time.h>
 #include <asm/reboot.h>
@@ -22,6 +23,9 @@
 #include <bcm63xx_io.h>
 #include <bcm63xx_gpio.h>
 
+/* CBR addr doesn't change and we can cache it */
+void __iomem *bmips_cbr_addr __read_mostly;
+
 void bcm63xx_machine_halt(void)
 {
 	pr_info("System halted\n");
arch/mips/bmips/dma.c +1 -1
index 3779e7855bd7..2bc9c0d4402f 100644
@@ -9,7 +9,7 @@ bool bmips_rac_flush_disable;
 
 void arch_sync_dma_for_cpu_all(void)
 {
-	void __iomem *cbr = BMIPS_GET_CBR();
+	void __iomem *cbr = bmips_cbr_addr;
 	u32 cfg;
 
 	if (boot_cpu_type() != CPU_BMIPS3300 &&
arch/mips/bmips/setup.c +6 -1
index 66a8ba19c287..6dd166c3d643 100644
@@ -34,6 +34,9 @@
 #define REG_BCM6328_OTP		((void __iomem *)CKSEG1ADDR(0x1000062c))
 #define BCM6328_TP1_DISABLED	BIT(9)
 
+/* CBR addr doesn't change and we can cache it */
+void __iomem *bmips_cbr_addr __read_mostly;
+
 extern bool bmips_rac_flush_disable;
 
 static const unsigned long kbase = VMLINUX_LOAD_ADDRESS & 0xfff00000;
@@ -111,7 +114,7 @@ static void bcm6358_quirks(void)
 	 * because the bootloader is not initializing it properly.
 	 */
 	bmips_rac_flush_disable = !!(read_c0_brcm_cmt_local() & (1 << 31)) ||
-				  !!BMIPS_GET_CBR();
+				  !!bmips_cbr_addr;
 }
 
 static void bcm6368_quirks(void)
@@ -144,6 +147,8 @@ static void __init bmips_init_cfe(void)
 
 void __init prom_init(void)
 {
+	/* Cache CBR addr before CPU/DMA setup */
+	bmips_cbr_addr = BMIPS_GET_CBR();
 	bmips_init_cfe();
 	bmips_cpu_setup();
 	register_bmips_smp_ops();
arch/mips/include/asm/bmips.h +1 -0
index 581a6a3c66e4..3a1cdfddb987 100644
@@ -81,6 +81,7 @@ extern char bmips_smp_movevec[];
 extern char bmips_smp_int_vec[];
 extern char bmips_smp_int_vec_end[];
 
+extern void __iomem *bmips_cbr_addr;
 extern int bmips_smp_enabled;
 extern int bmips_cpu_offset;
 extern cpumask_t bmips_booted_mask;
arch/mips/kernel/smp-bmips.c +2 -2
index b3dbf9ecb0d6..a4f84667a901 100644
@@ -518,7 +518,7 @@ static void bmips_set_reset_vec(int cpu, u32 val)
 		info.val = val;
 		bmips_set_reset_vec_remote(&info);
 	} else {
-		void __iomem *cbr = BMIPS_GET_CBR();
+		void __iomem *cbr = bmips_cbr_addr;
 
 		if (cpu == 0)
 			__raw_writel(val, cbr + BMIPS_RELO_VECTOR_CONTROL_0);
@@ -591,7 +591,7 @@ asmlinkage void __weak plat_wired_tlb_setup(void)
 
 void bmips_cpu_setup(void)
 {
-	void __iomem __maybe_unused *cbr = BMIPS_GET_CBR();
+	void __iomem __maybe_unused *cbr = bmips_cbr_addr;
 	u32 __maybe_unused cfg;
 
 	switch (current_cpu_type()) {
Signature · 1 line
2.45.1
#2 lore raw

PATCH v8 2/4 dt-bindings: mips: brcm: Document brcm,bmips-cbr-reg property

In reply to Christian Marangi

Document brcm,bmips-cbr-reg property.

Some SoC suffer from a BUG where CBR(Core Base Register) address might be badly or never initialized by the Bootloader or reading it from co-processor registers, if the system boots from secondary CPU, results in invalid address.

The CBR address is always the same on the SoC.

Usage of this property is to give an address also in these broken configuration/bootloader.

---
 .../devicetree/bindings/mips/brcm/soc.yaml    | 24 +++++++++++++++++++
 1 file changed, 24 insertions(+)
Documentation/devicetree/bindings/mips/brcm/soc.yaml +24 -0
index 975945ca2888..0cc634482a6a 100644
@@ -55,6 +55,16 @@ properties:
          under the "cpus" node.
         $ref: /schemas/types.yaml#/definitions/uint32
 
+      brcm,bmips-cbr-reg:
+        description: Reference address of the CBR.
+          Some SoC suffer from a BUG where CBR(Core Base Register)
+          address might be badly or never initialized by the Bootloader
+          or reading it from co-processor registers, if the system boots
+          from secondary CPU, results in invalid address.
+          The CBR address is always the same on the SoC hence it
+          can be provided in DT to handle these broken case.
+        $ref: /schemas/types.yaml#/definitions/uint32
+
     patternProperties:
       "^cpu@[0-9]$":
         type: object
@@ -64,6 +74,20 @@ properties:
     required:
       - mips-hpt-frequency
 
+if:
+  properties:
+    compatible:
+      contains:
+        enum:
+          - brcm,bcm6358
+          - brcm,bcm6368
+
+then:
+  properties:
+    cpus:
+      required:
+        - brcm,bmips-cbr-reg
+
 additionalProperties: true
 
 examples:
Signature · 1 line
2.45.1
#3 lore raw

PATCH v8 3/4 mips: bmips: setup: make CBR address configurable

In reply to Christian Marangi

Add support to provide CBR address from DT to handle broken SoC/Bootloader that doesn't correctly init it. This permits to use the RAC flush even in these condition.

To provide a CBR address from DT, the property "brcm,bmips-cbr-reg" needs to be set in the "cpus" node. On DT init, this property presence will be checked and will set the bmips_cbr_addr value accordingly. Also bmips_rac_flush_disable will be set to false as RAC flush can be correctly supported.

The CBR address from DT will overwrite the cached one and the one set in the CBR register will be ignored.

Also the DT CBR address is validated on being outside DRAM window.

---
 arch/mips/bcm47xx/setup.c |  6 +++++-
 arch/mips/bcm63xx/setup.c |  6 +++++-
 arch/mips/bmips/setup.c   | 30 ++++++++++++++++++++++++++++--
 3 files changed, 38 insertions(+), 4 deletions(-)
arch/mips/bcm47xx/setup.c +5 -1
index 2f1ee0560aba..247be207f293 100644
@@ -46,7 +46,11 @@
 #include <bcm47xx.h>
 #include <bcm47xx_board.h>
 
-/* CBR addr doesn't change and we can cache it */
+/*
+ * CBR addr doesn't change and we can cache it.
+ * For broken SoC/Bootloader CBR addr might also be provided via DT
+ * with "brcm,bmips-cbr-reg" in the "cpus" node.
+ */
 void __iomem *bmips_cbr_addr __read_mostly;
 
 union bcm47xx_bus bcm47xx_bus;
arch/mips/bcm63xx/setup.c +5 -1
index 16ea8945ae3a..81529084bc75 100644
@@ -23,7 +23,11 @@
 #include <bcm63xx_io.h>
 #include <bcm63xx_gpio.h>
 
-/* CBR addr doesn't change and we can cache it */
+/*
+ * CBR addr doesn't change and we can cache it.
+ * For broken SoC/Bootloader CBR addr might also be provided via DT
+ * with "brcm,bmips-cbr-reg" in the "cpus" node.
+ */
 void __iomem *bmips_cbr_addr __read_mostly;
 
 void bcm63xx_machine_halt(void)
arch/mips/bmips/setup.c +28 -2
index 6dd166c3d643..2572fd49a6e9 100644
@@ -34,7 +34,11 @@
 #define REG_BCM6328_OTP		((void __iomem *)CKSEG1ADDR(0x1000062c))
 #define BCM6328_TP1_DISABLED	BIT(9)
 
-/* CBR addr doesn't change and we can cache it */
+/*
+ * CBR addr doesn't change and we can cache it.
+ * For broken SoC/Bootloader CBR addr might also be provided via DT
+ * with "brcm,bmips-cbr-reg" in the "cpus" node.
+ */
 void __iomem *bmips_cbr_addr __read_mostly;
 
 extern bool bmips_rac_flush_disable;
@@ -208,13 +212,35 @@ void __init plat_mem_setup(void)
 void __init device_tree_init(void)
 {
 	struct device_node *np;
+	u32 addr;
 
 	unflatten_and_copy_device_tree();
 
 	/* Disable SMP boot unless both CPUs are listed in DT and !disabled */
 	np = of_find_node_by_name(NULL, "cpus");
-	if (np && of_get_available_child_count(np) <= 1)
+	if (!np)
+		return;
+
+	if (of_get_available_child_count(np) <= 1)
 		bmips_smp_enabled = 0;
+
+	/* Check if DT provide a CBR address */
+	if (of_property_read_u32(np, "brcm,bmips-cbr-reg", &addr))
+		goto exit;
+
+	/* Make sure CBR address is outside DRAM window */
+	if (addr >= (u32)memblock_start_of_DRAM() &&
+	    addr < (u32)memblock_end_of_DRAM()) {
+		WARN(1, "DT CBR %x inside DRAM window. Ignoring DT CBR.\n",
+		     addr);
+		goto exit;
+	}
+
+	bmips_cbr_addr = (void __iomem *)addr;
+	/* Since CBR is provided by DT, enable RAC flush */
+	bmips_rac_flush_disable = false;
+
+exit:
 	of_node_put(np);
 }
 
Signature · 1 line
2.45.1
#4 lore raw

PATCH v8 4/4 mips: bmips: enable RAC on BMIPS4350

In reply to Christian Marangi

From: Daniel González Cabanelas <[email protected]>

The data RAC is left disabled by the bootloader in some SoCs, at least in the core it boots from.
Enabling this feature increases the performance up to +30% depending on the task.

---
 arch/mips/kernel/smp-bmips.c | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)
arch/mips/kernel/smp-bmips.c +18 -0
index a4f84667a901..35b8d810833c 100644
@@ -592,6 +592,7 @@ asmlinkage void __weak plat_wired_tlb_setup(void)
 void bmips_cpu_setup(void)
 {
 	void __iomem __maybe_unused *cbr = bmips_cbr_addr;
+	u32 __maybe_unused rac_addr;
 	u32 __maybe_unused cfg;
 
 	switch (current_cpu_type()) {
@@ -620,6 +621,23 @@ void bmips_cpu_setup(void)
 		__raw_readl(cbr + BMIPS_RAC_ADDRESS_RANGE);
 		break;
 
+	case CPU_BMIPS4350:
+		rac_addr = BMIPS_RAC_CONFIG_1;
+
+		if (!(read_c0_brcm_cmt_local() & (1 << 31)))
+			rac_addr = BMIPS_RAC_CONFIG;
+
+		/* Enable data RAC */
+		cfg = __raw_readl(cbr + rac_addr);
+		__raw_writel(cfg | 0xf, cbr + rac_addr);
+		__raw_readl(cbr + rac_addr);
+
+		/* Flush stale data out of the readahead cache */
+		cfg = __raw_readl(cbr + BMIPS_RAC_CONFIG);
+		__raw_writel(cfg | 0x100, cbr + BMIPS_RAC_CONFIG);
+		__raw_readl(cbr + BMIPS_RAC_CONFIG);
+		break;
+
 	case CPU_BMIPS4380:
 		/* CBG workaround for early BMIPS4380 CPUs */
 		switch (read_c0_prid()) {
Signature · 1 line
2.45.1
#5 lore raw
In reply to Christian Marangi
Quoted 70 lines · Christian Marangi

Hi,

this simple series improve handling of RAC and CBR address and try to upstream these simple patch we have in OpenWrt for a while.

The first patch fix a straight kernel panic where some Bootloader might enable RAC but misconfigure the CBR address. The current logic only check if RAC is enabled but doesn't verify if the CBR address is usable.

The DMA sync function cause a kernel panic for invalid write. (as CBR is 0 or something like 0xa)

The second is preparation for making the CBR address configurable in DT. Since this address doesn't change, we can cache it and reference it with a local variable instead of calling the register to access the value.

The 4th patch make it configurable with 2 DT property, one to actually set the reg and the other to force set it.

The first property is used when CBR is set to 0. The second property is to force it if the Bootloader sets it to something wrong.

If the CBR value is not 0 and is not forced with the second property a WARN is printed and the DT value is ignored.

The 4th patch enable RAC on BMIPS4350.

These has been tested on BCM6358 (HG556a) and BCM6368 (VH4032N) and reported correct functionality.

Changes v8:
- Drop fix patch (applied to mips-fixes)
- Rework CBR cache patch to handle bcm47xx not compiling smp-bmips.c Changes v7:
- Add ACK and Reviewed-by tag for dt patch from v5 Changes v6:
- Add missing patch that got lost in v5
- Fix missing header for legacy bcm47xx
- Fix compilation error with gcc 10.2.1
Changes v5:
- Add Ack tags
- Improve DT descriptions as suggested by Conor
Changes v4:
- Fix compilation error with legacy brcm target
- Improve property description in DT commit (give
CBR meaning and drop reference to linux functions)
- Use only __read_mostly as we can't add variable to multiple data sections
- In patch 4 use local cbr variable instead of global one.
Changes v3:
- Drop broken-cbr-reg property
- Fix anyOf+const with enum
Changes v2:
- Prefix brcm vendor in the added property
- Drop last patch (cpu switch from DMA sync)
- Validate CBR addr from DT to be outside DRAM
- Reduce indentation in DT CBR check
- Reduce delta and use local variable for CBR where possible
- Fix and improve typo and spelling mistake
- Use 0xf instead of 0xa for BCM6358 RAC enable

Christian Marangi (3):
mips: bmips: rework and cache CBR addr handling
dt-bindings: mips: brcm: Document brcm,bmips-cbr-reg property
mips: bmips: setup: make CBR address configurable

Daniel González Cabanelas (1):
mips: bmips: enable RAC on BMIPS4350

Quoted 10 lines
 .../devicetree/bindings/mips/brcm/soc.yaml    | 24 +++++++++++++
 arch/mips/bcm47xx/prom.c                      |  3 ++
 arch/mips/bcm47xx/setup.c                     |  8 +++++
 arch/mips/bcm63xx/prom.c                      |  3 ++
 arch/mips/bcm63xx/setup.c                     |  8 +++++
 arch/mips/bmips/dma.c                         |  2 +-
 arch/mips/bmips/setup.c                       | 35 +++++++++++++++++--
 arch/mips/include/asm/bmips.h                 |  1 +
 arch/mips/kernel/smp-bmips.c                  | 22 ++++++++++--
 9 files changed, 101 insertions(+), 5 deletions(-)

series applied to mips-next.

Thomas.

Signature · 2 lines
Crap can work. Given enough thrust pigs will fly, but it's not necessarily a
good idea.                                                [ RFC1925, 2.3 ]