ARM: BCM63xx: SMP support
Subsystems (6)
cover ARM: BCM63xx: SMP support
Hi all,
This patch series implements SMP support on the Broadcom BCM63138 DSL platform.
There is a number of decisions being made here to simplify things:
- the power on sequence for the secondary CPU is a bit of low-level code that uses the on-chip PMB bus, which will later be supported as a proper reset controller driver, it was considered first to use a reset controller for that, but this was later dropped because:
- reset controllers cannot be utilized at SMP init time (too early)
- moving reset controllers init earlier has been rejected
- the second CPU power on sequence is fairly specific, even within the context of the BCM63138 SoC, no other peripheral has that level of detail
- VFP needs to be disabled when SMP is used because the second CPU does not have any VFP unit and will just hang doing a VFP/NEON operation
Florian Fainelli (7):
Documentation: DT: Add Broadcom BCM63138 PMB binding
ARM: dts: BCM63xx: Add PMB busses nodes
Documentation: DT: Document SMP DT nodes and properties for BCM63138
ARM: dts: BCM63xx: Add SMP nodes and required properties
ARM: BCM63xx: Add Broadcom BCM63xx PMB controller helpers
ARM: BCM63xx: Add secondary CPU PMB initialization sequence
ARM: BCM63xx: Add SMP support for BCM63138
.../devicetree/bindings/arm/bcm/bcm63138.txt | 42 ++++ .../bindings/reset/brcm,bcm63138-pmb.txt | 19 ++ arch/arm/boot/dts/bcm63138.dtsi | 20 ++ arch/arm/mach-bcm/Makefile | 7 +- arch/arm/mach-bcm/bcm63xx_headsmp.S | 23 +++ arch/arm/mach-bcm/bcm63xx_pmb.c | 222 +++++++++++++++++++++ arch/arm/mach-bcm/bcm63xx_smp.c | 171 ++++++++++++++++ arch/arm/mach-bcm/bcm63xx_smp.h | 9 + include/linux/bcm63xx_pmb.h | 76 +++++++ 9 files changed, 588 insertions(+), 1 deletion(-) create mode 100644 Documentation/devicetree/bindings/reset/brcm,bcm63138-pmb.txt create mode 100644 arch/arm/mach-bcm/bcm63xx_headsmp.S create mode 100644 arch/arm/mach-bcm/bcm63xx_pmb.c create mode 100644 arch/arm/mach-bcm/bcm63xx_smp.c create mode 100644 arch/arm/mach-bcm/bcm63xx_smp.h create mode 100644 include/linux/bcm63xx_pmb.h
Signature · 1 line
2.1.0
PATCH 1/7 Documentation: DT: Add Broadcom BCM63138 PMB binding
Add a Device Tree binding for the Broadcom BCM63138 Processor Monitor Bus, which is an internal bus used to access different power and reset signals within a BCM63138 System-on-a-Chip.
--- .../devicetree/bindings/reset/brcm,bcm63138-pmb.txt | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) create mode 100644 Documentation/devicetree/bindings/reset/brcm,bcm63138-pmb.txt
Documentation/devicetree/bindings/reset/brcm,bcm63138-pmb.txt new file +19 -0
@@ -0,0 +1,19 @@ +Broadcom BCM63138 Processor Monitor Bus binding +=============================================== + +Please also refer to reset.txt in this directory for common reset +controller binding usage. + +Require properties: + +- compatible: must be "brcm,bcm63138-pmb" +- reg: base register address and size for this bus controller +- #reset-cells: must be 2 first cell is the internal bus number, and the third + cell is the address on this bus + +Example: + pmb0: reset-controller at 4800c0 { + compatible = "brcm,bcm63138-pmb"; + reg = <0x4800c0 0x10>; + #reset-cells = <2>; + };
Signature · 1 line
2.1.0
PATCH 1/7 Documentation: DT: Add Broadcom BCM63138 PMB binding
2015-04-17 15:33 GMT-07:00 Florian Fainelli <[email protected]>:
Quoted 10 lines
Add a Device Tree binding for the Broadcom BCM63138 Processor Monitor Bus, which is an internal bus used to access different power and reset signals within a BCM63138 System-on-a-Chip.
Signed-off-by Florian Fainelli <[email protected]>---
+- compatible: must be "brcm,bcm63138-pmb" +- reg: base register address and size for this bus controller +- #reset-cells: must be 2 first cell is the internal bus number, and the third + cell is the address on this bus
I missed the fact that we need to represent the number of zones per devices as a third cell here since some devices may have multiple zones to be powered on. Will re-spin this series along with addressing Russell's comments in patch 7.
--
Florian
PATCH 2/7 ARM: dts: BCM63xx: Add PMB busses nodes
In reply to Florian FainelliAdd the two BCM63138 PMB busses nodes found on this System-on-a-Chip as described in their corresponding binding document.
--- arch/arm/boot/dts/bcm63138.dtsi | 12 ++++++++++++ 1 file changed, 12 insertions(+)
arch/arm/boot/dts/bcm63138.dtsi +12 -0
@@ -105,6 +105,18 @@ reg = <0x1e620 0x20>; interrupts = <GIC_PPI 14 IRQ_TYPE_LEVEL_HIGH>; }; + + pmb0: reset-controller at 4800c0 { + compatible = "brcm,bcm63138-pmb"; + reg = <0x4800c0 0x10>; + #reset-cells = <2>; + }; + + pmb1: reset-controller at 4800e0 { + compatible = "brcm,bcm63138-pmb"; + reg = <0x4800e0 0x10>; + #reset-cells = <2>; + }; }; /* Legacy UBUS base */
Signature · 1 line
2.1.0
PATCH 3/7 Documentation: DT: Document SMP DT nodes and properties for BCM63138
In reply to Florian FainelliAdd binding documentation for the additional nodes and properties required to get the secondary CPU online on the BCM63138 SoC.
--- .../devicetree/bindings/arm/bcm/bcm63138.txt | 42 ++++++++++++++++++++++ 1 file changed, 42 insertions(+)
Documentation/devicetree/bindings/arm/bcm/bcm63138.txt +42 -0
@@ -7,3 +7,45 @@ following properties: Required root node property: compatible: should be "brcm,bcm63138" + +An optional Boot lookup table Device Tree node is required for secondary CPU +initialization as well as a 'resets' phandle to the correct PMB controller as +defined in reset/brcm,bcm63138-pmb.txt for this secondary CPU, and an +'enable-method' property. + +Required properties for the Boot lookup table node: +- compatible: should be "brcm,bcm63138-bootlut" +- reg: register base address and length for the Boot Lookup table + +Optional properties for the primary CPU node: +- enable-method: should be "brcm,bcm63138" + +Optional properties for the secondary CPU node: +- enable-method: should be "brcm,bcm63138" +- resets: phandle to the relevant PMB controller, one integer indicating the internal + bus number, and a second integer indicating the address of the CPU in the PMB + internal bus number. + +Example: + + cpus { + cpu at 0 { + compatible = "arm,cotex-a9"; + reg = <0>; + ... + enable-method = "brcm,bcm63138"; + }; + + cpu at 1 { + compatible = "arm,cortex-a9"; + reg = <1>; + ... + enable-method = "brcm,bcm63138"; + resets = <&pmb0 0 4>; + }; + }; + + bootlut: bootlut at 8000 { + compatible = "brcm,bcm63138-bootlut"; + reg = <0x8000 0x50>; + };
Signature · 1 line
2.1.0
PATCH 4/7 ARM: dts: BCM63xx: Add SMP nodes and required properties
In reply to Florian FainelliUpdate bcm63138.dtsi with the following:
- enable-method for both CPU nodes
- brcm,bcm63138-bootlut node
- resets properties to point to the correct PMB controller to release the secondary CPU from reset
--- arch/arm/boot/dts/bcm63138.dtsi | 8 ++++++++ 1 file changed, 8 insertions(+)
arch/arm/boot/dts/bcm63138.dtsi +8 -0
@@ -26,6 +26,7 @@ compatible = "arm,cortex-a9"; next-level-cache = <&L2>; reg = <0>; + enable-method = "brcm,bcm63138"; }; cpu at 1 {
@@ -33,6 +34,8 @@ compatible = "arm,cortex-a9"; next-level-cache = <&L2>; reg = <1>; + enable-method = "brcm,bcm63138"; + resets = <&pmb0 0 4>; }; };
@@ -143,5 +146,10 @@ clock-names = "periph"; status = "disabled"; }; + + bootlut: bootlut at 8000 { + compatible = "brcm,bcm63138-bootlut"; + reg = <0x8000 0x50>; + }; }; };
Signature · 1 line
2.1.0
PATCH 5/7 ARM: BCM63xx: Add Broadcom BCM63xx PMB controller helpers
In reply to Florian FainelliThis patch adds both common register definitions and helper functions used to issue read/write commands to the Broadcom BCM63xx PMB controller which is used to power on and release from reset internal on-chip peripherals such as the integrated Ethernet switch, AHCI, USB, as well as the secondary CPU core.
This is going to be utilized by the BCM63138 SMP code, as well as by the BCM63138 reset controller later.
--- include/linux/bcm63xx_pmb.h | 76 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 76 insertions(+) create mode 100644 include/linux/bcm63xx_pmb.h
include/linux/bcm63xx_pmb.h new file +76 -0
@@ -0,0 +1,76 @@ +#ifndef __BCM63XX_PMB_H +#define __BCM63XX_PMB_H + +#include <linux/io.h> +#include <linux/types.h> +#include <linux/delay.h> + +/* PMB Master controller register */ +#define PMB_CTRL 0x00 +#define PMC_PMBM_START (1 << 31) +#define PMC_PMBM_TIMEOUT (1 << 30) +#define PMC_PMBM_SLAVE_ERR (1 << 29) +#define PMC_PMBM_BUSY (1 << 28) +#define PMC_PMBM_READ (0 << 20) +#define PMC_PMBM_WRITE (1 << 20) +#define PMB_WR_DATA 0x04 +#define PMB_TIMEOUT 0x08 +#define PMB_RD_DATA 0x0C + +#define PMB_BUS_ID_SHIFT 8 + +/* Perform the low-level PMB master operation, shared between reads and +* writes, caller must hold the spinlock +*/ +static inline int __bpcm_do_op (void __iomem *master,unsigned int addr, +u32 off,u32 op) +{ +unsigned int timeout =1000 ; +u32 cmd; + + cmd = (PMC_PMBM_START | op | (addr &0xff ) <<12 | off); +__raw_writel (cmd, master +PMB_CTRL ); +do { + cmd =__raw_readl (master +PMB_CTRL ); +if (!(cmd &PMC_PMBM_START )) +return 0 ; + +if (cmd &PMC_PMBM_SLAVE_ERR ) +return -EIO ; + +if (cmd &PMC_PMBM_TIMEOUT ) +return -ETIMEDOUT ; + +udelay (1 ); + }while (timeout-- >0 ); + +return -ETIMEDOUT ; +} + +static inline int bpcm_rd (void __iomem *master,unsigned int addr, +u32 off,u32 *val) +{ +int ret =0 ; + + ret =__bpcm_do_op (master, addr, off >>2 ,PMC_PMBM_READ ); + *val =__raw_readl (master +PMB_RD_DATA ); + +return ret; +} + +static inline int bpcm_wr (void __iomem *master,unsigned int addr, +u32 off,u32 val) +{ +int ret =0 ; + +__raw_writel (val, master +PMB_WR_DATA ); +/* Ensure that writes to the PMB_WR_DATA registers are taken +* into account before attempting to start the PMB transaction +*/ +mb (); + ret =__bpcm_do_op (master, addr, off >>2 ,PMC_PMBM_WRITE ); + +return ret; +} + +#endif /* __BCM63XX_PMB_H */
Signature · 1 line
2.1.0
PATCH 6/7 ARM: BCM63xx: Add secondary CPU PMB initialization sequence
In reply to Florian FainelliThe sequence to initialize a secondary CPU using the BCM63138 PMB is extremely specific and represents much more code than any other on-chip peripheral (AHCI, USB 3.0 or integrated Ethernet switch), as such we keep that code local and utilize Device Tree to lookup all the resources we need from the CPU device tree node.
--- arch/arm/mach-bcm/bcm63xx_pmb.c | 222 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 222 insertions(+) create mode 100644 arch/arm/mach-bcm/bcm63xx_pmb.c
arch/arm/mach-bcm/bcm63xx_pmb.c new file +222 -0
@@ -0,0 +1,222 @@ +/* +* Broadcom BCM63138 PMB initialization for secondary CPU(s) +* +* Copyright (C) 2015 Broadcom Corporation +* Author: Florian Fainelli <[email protected]> +* +* This program is free software; you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published by +* the Free Software Foundation; either version 2 of the License, or +* (at your option) any later version. +*/ +#include <linux/kernel.h> +#include <linux/io.h> +#include <linux/spinlock.h> +#include <linux/bcm63xx_pmb.h> +#include <linux/of.h> +#include <linux/of_address.h> + +#include "bcm63xx_smp.h" + +/* ARM Control register definitions */ +#define CORE_PWR_CTRL_SHIFT 0 +#define CORE_PWR_CTRL_MASK 0x3 +#define PLL_PWR_ON BIT(8) +#define PLL_LDO_PWR_ON BIT(9) +#define PLL_CLAMP_ON BIT(10) +#define CPU_RESET_N (x) BIT(13 + (x)) +#define NEON_RESET_N BIT(15) +#define PWR_CTRL_STATUS_SHIFT 28 +#define PWR_CTRL_STATUS_MASK 0x3 +#define PWR_DOWN_SHIFT 30 +#define PWR_DOWN_MASK 0x3 + +/* CPU Power control register definitions */ +#define MEM_PWR_OK BIT(0) +#define MEM_PWR_ON BIT(1) +#define MEM_CLAMP_ON BIT(2) +#define MEM_PWR_OK_STATUS BIT(4) +#define MEM_PWR_ON_STATUS BIT(5) +#define MEM_PDA_SHIFT 8 +#define MEM_PDA_MASK 0xf +#define MEM_PDA_CPU_MASK 0x1 +#define MEM_PDA_NEON_MASK 0xf +#define CLAMP_ON BIT(15) +#define PWR_OK_SHIFT 16 +#define PWR_OK_MASK 0xf +#define PWR_ON_SHIFT 20 +#define PWR_CPU_MASK 0x03 +#define PWR_NEON_MASK 0x01 +#define PWR_ON_MASK 0xf +#define PWR_OK_STATUS_SHIFT 24 +#define PWR_OK_STATUS_MASK 0xf +#define PWR_ON_STATUS_SHIFT 28 +#define PWR_ON_STATUS_MASK 0xf + +#define ARM_CONTROL 0x30 +#define ARM_PWR_CONTROL_BASE 0x34 +#define ARM_PWR_CONTROL (x) (ARM_PWR_CONTROL_BASE + (x) * 0x4) +#define ARM_NEON_L2 0x3c + +/* Perform a value write, then spin until the value shifted by +* shift is seen, masked with mask and is different from cond. +*/ +static int bpcm_wr_rd_mask (void __iomem *master, +unsigned int addr,u32 off,u32 *val, +u32 shift,u32 mask,u32 cond) +{ +int ret; + + ret =bpcm_wr (master, addr, off, *val); +if (ret) +return ret; + +do { + ret =bpcm_rd (master, addr, off, val); +if (ret) +return ret; + +cpu_relax (); + }while (((*val >> shift) & mask) != cond); + +return ret; +} + +/* Global lock to serialize accesses to the PMB registers while we +* are bringing up the secondary CPU +*/ +static DEFINE_SPINLOCK (pmb_lock); + +static int bcm63xx_pmb_get_resources (struct device_node *dn, +void __iomem **base, +unsigned int *cpu, +unsigned int *bus, +unsigned int *addr) +{ +struct device_node *pmb_dn; +struct of_phandle_args args; +int ret; + + ret =of_property_read_u32 (dn,"reg" , cpu); +if (ret) { +pr_err ("CPU is missing a reg node\n" ); +return ret; + } + + ret =of_parse_phandle_with_args (dn,"resets" ,"#reset-cells" , +0 , &args); +if (ret) { +pr_err ("CPU is missing a resets phandle\n" ); +return ret; + } + + pmb_dn = args.np; +if (args.args_count !=2 ) { +pr_err ("reset-controller does not conform to reset-cells\n" ); +return -EINVAL ; + } + + *base =of_iomap (args.np,0 ); +if (!*base) { +pr_err ("failed remapping PMB register\n" ); +return -ENOMEM ; + } + + *bus = args.args[0 ]; + *addr = args.args[1 ]; + +return 0 ; +} + +int bcm63xx_pmb_power_on_cpu (struct device_node *dn) +{ +void __iomem *base; +unsigned int cpu, bus, addr; +unsigned long flags; +u32 val, ctrl; +int ret; + + ret =bcm63xx_pmb_get_resources (dn, &base, &cpu, &bus, &addr); +if (ret) +return ret; + +/* We would not know how to enable a third and greater CPU */ +WARN_ON (cpu >1 ); + +spin_lock_irqsave (&pmb_lock, flags); + +/* Check if the CPU is already on and save the ARM_CONTROL register +* value since we will use it later for CPU de-assert once done with +* the CPU-specific power sequence +*/ + ret =bpcm_rd (base, addr,ARM_CONTROL , &ctrl); +if (ret) +return ret; + +if (ctrl &CPU_RESET_N (cpu)) { +pr_info ("PMB: CPU%d is already powered on\n" , cpu); + ret =0 ; + goto out; + } + +/* Power on PLL */ + ret =bpcm_rd (base, addr,ARM_PWR_CONTROL (cpu), &val); +if (ret) + goto out; + + val |= (PWR_CPU_MASK <<PWR_ON_SHIFT ); + + ret =bpcm_wr_rd_mask (base, addr,ARM_PWR_CONTROL (cpu), &val, +PWR_ON_STATUS_SHIFT ,PWR_CPU_MASK ,PWR_CPU_MASK ); +if (ret) + goto out; + + val |= (PWR_CPU_MASK <<PWR_OK_SHIFT ); + + ret =bpcm_wr_rd_mask (base, addr,ARM_PWR_CONTROL (cpu), &val, +PWR_OK_STATUS_SHIFT ,PWR_CPU_MASK ,PWR_CPU_MASK ); +if (ret) + goto out; + + val &= ~CLAMP_ON ; + + ret =bpcm_wr (base, addr,ARM_PWR_CONTROL (cpu), val); +if (ret) + goto out; + +/* Power on CPU<N> RAM */ + val &= ~(MEM_PDA_MASK <<MEM_PDA_SHIFT ); + + ret =bpcm_wr (base, addr,ARM_PWR_CONTROL (cpu), val); +if (ret) + goto out; + + val |=MEM_PWR_ON ; + + ret =bpcm_wr_rd_mask (base, addr,ARM_PWR_CONTROL (cpu), &val, +0 ,MEM_PWR_ON_STATUS ,MEM_PWR_ON_STATUS ); +if (ret) + goto out; + + val |=MEM_PWR_OK ; + + ret =bpcm_wr_rd_mask (base, addr,ARM_PWR_CONTROL (cpu), &val, +0 ,MEM_PWR_OK_STATUS ,MEM_PWR_OK_STATUS ); +if (ret) + goto out; + + val &= ~MEM_CLAMP_ON ; + + ret =bpcm_wr (base, addr,ARM_PWR_CONTROL (cpu), val); +if (ret) + goto out; + +/* De-assert CPU reset */ + ctrl |=CPU_RESET_N (cpu); + + ret =bpcm_wr (base, addr,ARM_CONTROL , ctrl); +out: +spin_unlock_irqrestore (&pmb_lock, flags); +iounmap (base); +return ret; +}
Signature · 1 line
2.1.0
PATCH 7/7 ARM: BCM63xx: Add SMP support for BCM63138
In reply to Florian FainelliAdd support for booting the secondary CPU on BCM63138, this involves:
- locating the bootlut to write the reset vector
- powering up the second CPU when we need to using the DT-supplied PMB references
- disabling VFP when enabled such that we can keep having SMP
--- arch/arm/mach-bcm/Makefile | 7 +- arch/arm/mach-bcm/bcm63xx_headsmp.S | 23 +++++ arch/arm/mach-bcm/bcm63xx_smp.c | 171 ++++++++++++++++++++++++++++++++++++ arch/arm/mach-bcm/bcm63xx_smp.h | 9 ++ 4 files changed, 209 insertions(+), 1 deletion(-) create mode 100644 arch/arm/mach-bcm/bcm63xx_headsmp.S create mode 100644 arch/arm/mach-bcm/bcm63xx_smp.c create mode 100644 arch/arm/mach-bcm/bcm63xx_smp.h
arch/arm/mach-bcm/Makefile +6 -1
@@ -38,7 +38,12 @@ obj-$(CONFIG_ARCH_BCM2835) += board_bcm2835.o obj-$(CONFIG_ARCH_BCM_5301X) += bcm_5301x.o # BCM63XXx -obj-$(CONFIG_ARCH_BCM_63XX) := bcm63xx.o +ifeq ($(CONFIG_ARCH_BCM_63XX),y) +obj-y += bcm63xx.o +CFLAGS_bcm63xx_headsmp.o += -march=armv7-a +obj-$(CONFIG_SMP) += bcm63xx_smp.o bcm63xx_headsmp.o \ + bcm63xx_pmb.o +endif ifeq ($(CONFIG_ARCH_BRCMSTB),y) CFLAGS_platsmp-brcmstb.o += -march=armv7-a
arch/arm/mach-bcm/bcm63xx_headsmp.S new file +23 -0
@@ -0,0 +1,23 @@ +/* + * Copyright (C) 2015, Broadcom Corporation + * All Rights Reserved + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + */ +#include <linux/linkage.h> +#include <linux/init.h> +#include <asm/assembler.h> + +ENTRY(bcm63138_secondary_startup) + ARM_BE8(setend be) + /* + * L1 cache does have unpredictable contents at power-up clean its + * contents without flushing + */ + bl v7_invalidate_l1 + nop + + b secondary_startup +ENDPROC(bcm63138_secondary_startup)
arch/arm/mach-bcm/bcm63xx_smp.c new file +171 -0
171 lines not on this page · raw message
arch/arm/mach-bcm/bcm63xx_smp.h new file +9 -0
@@ -0,0 +1,9 @@ +#ifndef __BCM63XX_SMP_H +#define __BCM63XX_SMP_H + +struct device_node ; + +extern void bcm63138_secondary_startup (void ); +extern int bcm63xx_pmb_power_on_cpu (struct device_node *dn); + +#endif /* __BCM63XX_SMP_H */
Signature · 1 line
2.1.0
PATCH 7/7 ARM: BCM63xx: Add SMP support for BCM63138
Quoted 2 lines · Florian Fainelli
(quoted diff) +2
-0+ +extern unsigned int VFP_arch;
...
arch/arm/mach-bcm/bcm63xx_smp.c line 85
Quoted 13 lines
arch/arm/mach-bcm/bcm63xx_smp.c new file +13
-0+/* The BCM63138 SoC has two Cortex-A9 CPUs, CPU0 features a complete +* and fully functional VFP unit that can be used, but CPU1 does not. +* Since we will not be able to trap kernel-mode NEON to force +* migration to CPU0, just do not advertise VFP support at all. +* +* This will make vfp_init bail out and do not attempt to use VFP at +* all, for kernel-mode NEON, we do not want to introduce any +* conditionals in hot-paths, so we just restrict the system to UP. +*/ +#ifdef CONFIG_VFP +if (ncores >1 ) { +pr_warn ("SMP: secondary CPUs lack VFP unit, disabling VFP\n" ); + VFP_arch =1 ;
What makes you think that doing this hack would be somehow acceptable?
Please, do the job properly. Add a function hook into the VFP code called "vfp_disable()" so that it's obvious that you want to disable it - and, because that code will be localised to VFP, it then becomes acceptable to set VFP_arch=1. Don't rely on current VFP behaviour through setting VFP_arch directly in from platform code - that's nothing more than a hack and a disgusting layering violation.
Signature · 2 lines
FTTC broadband for 0.8mile line: currently at 10.5Mbps down 400kbps up according to speedtest.net.