Skip to contentSkip to conversation

ARM: tegra: add CPU hotplug support

PATCH Superseded Joseph Lo 2 participants 9 messages Aug 15, 2012 – Aug 16, 2012

Subsystems (3)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Joseph Lo

author
#0 lore raw

v2 cover ARM: tegra: add CPU hotplug support

This patch set adds CPU hotplug support for Tegra20 and Tegra30 SoCs.

The first 3 patches make the CPU clock and reset control be a module and the abstraction APIs for TegraXX SoCs. And clean up some codes for later usage. Then add CPU hotplug support for Tegra20 and Tegra30.

Verified on Seaboard(Tegra20) and Cardhu(Tegra30)

V2:
* add one more patch for enable HOTPLUG_CPU [6/6]
* move all the SoC specific CPU shutdown code into platform_cpu_die [4/6]

Joseph Lo (6):
ARM: tegra: introduce tegra_cpu_car_ops structures
ARM: tegra: replace the CPU CAR access code by tegra_cpu_car_ops
ARM: tegra: clean up the common assembly macros into sleep.h
ARM: tegra30: add CPU hotplug support
ARM: tegra20: add CPU hotplug support
ARM: tegra: defconfig: enable HOTPLUG_CPU

 arch/arm/configs/tegra_defconfig          |    1 +
 arch/arm/mach-tegra/Makefile              |    2 +
 arch/arm/mach-tegra/clock.c               |    4 +
 arch/arm/mach-tegra/common.c              |    3 +
 arch/arm/mach-tegra/headsmp.S             |    6 +-
 arch/arm/mach-tegra/hotplug.c             |  118 ++++++++---------------------
 arch/arm/mach-tegra/platsmp.c             |   29 +------
 arch/arm/mach-tegra/sleep-t20.S           |   82 ++++++++++++++++++++
 arch/arm/mach-tegra/sleep-t30.S           |  107 ++++++++++++++++++++++++++
 arch/arm/mach-tegra/sleep.S               |   33 +--------
 arch/arm/mach-tegra/sleep.h               |   80 +++++++++++++++++++
 arch/arm/mach-tegra/tegra20_clocks.c      |   70 +++++++++++++++++
 arch/arm/mach-tegra/tegra20_clocks_data.c |    2 +
 arch/arm/mach-tegra/tegra30_clocks.c      |   72 +++++++++++++++++
 arch/arm/mach-tegra/tegra30_clocks_data.c |    3 +
 arch/arm/mach-tegra/tegra_cpu_car.h       |   87 +++++++++++++++++++++
 16 files changed, 552 insertions(+), 147 deletions(-)
 create mode 100644 arch/arm/mach-tegra/sleep-t20.S
 create mode 100644 arch/arm/mach-tegra/sleep-t30.S
 create mode 100644 arch/arm/mach-tegra/sleep.h
 create mode 100644 arch/arm/mach-tegra/tegra_cpu_car.h
author
#1 lore raw

PATCH v2 1/6 ARM: tegra: introduce tegra_cpu_car_ops structures

The tegra_cpu_car_ops provide the interface for CPU to control it's clock gating and reset status. The other drivers should use this for CPU control. And should not directly access CAR registers to control CPU.

---
 arch/arm/mach-tegra/clock.c               |    4 +
 arch/arm/mach-tegra/tegra20_clocks.c      |   70 +++++++++++++++++++++++
 arch/arm/mach-tegra/tegra20_clocks_data.c |    2 +
 arch/arm/mach-tegra/tegra30_clocks.c      |   72 ++++++++++++++++++++++++
 arch/arm/mach-tegra/tegra30_clocks_data.c |    3 +
 arch/arm/mach-tegra/tegra_cpu_car.h       |   87 +++++++++++++++++++++++++++++
 6 files changed, 238 insertions(+), 0 deletions(-)
 create mode 100644 arch/arm/mach-tegra/tegra_cpu_car.h
arch/arm/mach-tegra/clock.c +4 -0
index 632133f..fd82085 100644
@@ -31,6 +31,10 @@
 
 #include "board.h"
 #include "clock.h"
+#include "tegra_cpu_car.h"
+
+/* Global data of Tegra CPU CAR ops */
+struct tegra_cpu_car_ops *tegra_cpu_car_ops;
 
 /*
  * Locking:
arch/arm/mach-tegra/tegra20_clocks.c +70 -0

85 lines not on this page · raw message

arch/arm/mach-tegra/tegra20_clocks_data.c +2 -0
index b5c518e..56552f9 100644
@@ -34,6 +34,7 @@
 #include "fuse.h"
 #include "tegra2_emc.h"
 #include "tegra20_clocks.h"
+#include "tegra_cpu_car.h"
 
 /* Clock definitions */
 
@@ -1110,4 +1111,5 @@ void __init tegra2_init_clocks(void)
 	}
 
 	init_audio_sync_clock_mux();
+	tegra20_cpu_car_ops_init();
 }
arch/arm/mach-tegra/tegra30_clocks.c +72 -0

87 lines not on this page · raw message

arch/arm/mach-tegra/tegra30_clocks_data.c +3 -0
index c924240..8252f88 100644
@@ -32,6 +32,7 @@
 #include "clock.h"
 #include "fuse.h"
 #include "tegra30_clocks.h"
+#include "tegra_cpu_car.h"
 
 #define DEFINE_CLK_TEGRA(_name, _rate, _ops, _flags,		\
 		   _parent_names, _parents, _parent)		\
@@ -1367,4 +1368,6 @@ void __init tegra30_init_clocks(void)
 
 	for (i = 0; i < ARRAY_SIZE(tegra_clk_out_list); i++)
 		tegra30_init_one_clock(tegra_clk_out_list[i]);
+
+	tegra30_cpu_car_ops_init();
 }
arch/arm/mach-tegra/tegra_cpu_car.h new file +87 -0

87 lines not on this page · raw message

Signature · 1 line
1.7.0.4
author
#2 lore raw

PATCH v2 2/6 ARM: tegra: replace the CPU CAR access code by tegra_cpu_car_ops

In reply to Joseph Lo

Replacing the code that directly access to CAR registers with tegra_cpu_car_ops. This ops hides CPU CAR access inside and provides control interface for it.

---
 arch/arm/mach-tegra/platsmp.c |   29 +++++------------------------
 1 files changed, 5 insertions(+), 24 deletions(-)
arch/arm/mach-tegra/platsmp.c +5 -24
index 1a208db..96ed171 100644
@@ -31,6 +31,7 @@
 #include "fuse.h"
 #include "flowctrl.h"
 #include "reset.h"
+#include "tegra_cpu_car.h"
 
 extern void tegra_secondary_startup(void);
 
@@ -38,17 +39,6 @@ static void __iomem *scu_base = IO_ADDRESS(TEGRA_ARM_PERIF_BASE);
 
 #define EVP_CPU_RESET_VECTOR \
 	(IO_ADDRESS(TEGRA_EXCEPTION_VECTORS_BASE) + 0x100)
-#define CLK_RST_CONTROLLER_CLK_CPU_CMPLX \
-	(IO_ADDRESS(TEGRA_CLK_RESET_BASE) + 0x4c)
-#define CLK_RST_CONTROLLER_RST_CPU_CMPLX_SET \
-	(IO_ADDRESS(TEGRA_CLK_RESET_BASE) + 0x340)
-#define CLK_RST_CONTROLLER_RST_CPU_CMPLX_CLR \
-	(IO_ADDRESS(TEGRA_CLK_RESET_BASE) + 0x344)
-#define CLK_RST_CONTROLLER_CLK_CPU_CMPLX_CLR \
-	(IO_ADDRESS(TEGRA_CLK_RESET_BASE) + 0x34c)
-
-#define CPU_CLOCK(cpu)	(0x1<<(8+cpu))
-#define CPU_RESET(cpu)	(0x1111ul<<(cpu))
 
 void __cpuinit platform_secondary_init(unsigned int cpu)
 {
@@ -63,13 +53,8 @@ void __cpuinit platform_secondary_init(unsigned int cpu)
 
 static int tegra20_power_up_cpu(unsigned int cpu)
 {
-	u32 reg;
-
 	/* Enable the CPU clock. */
-	reg = readl(CLK_RST_CONTROLLER_CLK_CPU_CMPLX);
-	writel(reg & ~CPU_CLOCK(cpu), CLK_RST_CONTROLLER_CLK_CPU_CMPLX);
-	barrier();
-	reg = readl(CLK_RST_CONTROLLER_CLK_CPU_CMPLX);
+	tegra_enable_cpu_clock(cpu);
 
 	/* Clear flow controller CSR. */
 	flowctrl_write_cpu_csr(cpu, 0);
@@ -79,7 +64,6 @@ static int tegra20_power_up_cpu(unsigned int cpu)
 
 static int tegra30_power_up_cpu(unsigned int cpu)
 {
-	u32 reg;
 	int ret, pwrgateid;
 	unsigned long timeout;
 
@@ -103,8 +87,7 @@ static int tegra30_power_up_cpu(unsigned int cpu)
 	}
 
 	/* CPU partition is powered. Enable the CPU clock. */
-	writel(CPU_CLOCK(cpu), CLK_RST_CONTROLLER_CLK_CPU_CMPLX_CLR);
-	reg = readl(CLK_RST_CONTROLLER_CLK_CPU_CMPLX_CLR);
+	tegra_enable_cpu_clock(cpu);
 	udelay(10);
 
 	/* Remove I/O clamps. */
@@ -128,8 +111,7 @@ int __cpuinit boot_secondary(unsigned int cpu, struct task_struct *idle)
 	 * via the flow controller). This will have no effect on first boot
 	 * of the CPU since it should already be in reset.
 	 */
-	writel(CPU_RESET(cpu), CLK_RST_CONTROLLER_RST_CPU_CMPLX_SET);
-	dmb();
+	tegra_put_cpu_in_reset(cpu);
 
 	/*
 	 * Unhalt the CPU. If the flow controller was used to power-gate the
@@ -155,8 +137,7 @@ int __cpuinit boot_secondary(unsigned int cpu, struct task_struct *idle)
 		goto done;
 
 	/* Take the CPU out of reset. */
-	writel(CPU_RESET(cpu), CLK_RST_CONTROLLER_RST_CPU_CMPLX_CLR);
-	wmb();
+	tegra_cpu_out_of_reset(cpu);
 done:
 	return status;
 }
Signature · 1 line
1.7.0.4
author
#3 lore raw

PATCH v2 3/6 ARM: tegra: clean up the common assembly macros into sleep.h

In reply to Joseph Lo

There are some common macros for Tegra low-level assembly code. Clean up them into one header file and move the definitions that will be re-used into it as well.

---
 arch/arm/mach-tegra/headsmp.S |    6 +---
 arch/arm/mach-tegra/sleep.S   |   33 +-----------------------
 arch/arm/mach-tegra/sleep.h   |   56 +++++++++++++++++++++++++++++++++++++++++
 3 files changed, 58 insertions(+), 37 deletions(-)
 create mode 100644 arch/arm/mach-tegra/sleep.h
arch/arm/mach-tegra/headsmp.S +1 -5
index fef9c2c..6addc78 100644
@@ -7,17 +7,13 @@
 
 #include "flowctrl.h"
 #include "reset.h"
+#include "sleep.h"
 
 #define APB_MISC_GP_HIDREV	0x804
 #define PMC_SCRATCH41	0x140
 
 #define RESET_DATA(x)	((TEGRA_RESET_##x)*4)
 
-	.macro mov32, reg, val
-	movw	\reg, #:lower16:\val
-	movt	\reg, #:upper16:\val
-	.endm
-
         .section ".text.head", "ax"
 	__CPUINIT
 
arch/arm/mach-tegra/sleep.S +1 -32
index d29b156..ea81554 100644
@@ -29,36 +29,5 @@
 #include <mach/iomap.h>
 
 #include "flowctrl.h"
+#include "sleep.h"
 
-#define TEGRA_FLOW_CTRL_VIRT (TEGRA_FLOW_CTRL_BASE - IO_PPSB_PHYS \
-					+ IO_PPSB_VIRT)
-
-/* returns the offset of the flow controller halt register for a cpu */
-.macro cpu_to_halt_reg rd, rcpu
-	cmp	\rcpu, #0
-	subne	\rd, \rcpu, #1
-	movne	\rd, \rd, lsl #3
-	addne	\rd, \rd, #0x14
-	moveq	\rd, #0
-.endm
-
-/* returns the offset of the flow controller csr register for a cpu */
-.macro cpu_to_csr_reg rd, rcpu
-	cmp	\rcpu, #0
-	subne	\rd, \rcpu, #1
-	movne	\rd, \rd, lsl #3
-	addne	\rd, \rd, #0x18
-	moveq	\rd, #8
-.endm
-
-/* returns the ID of the current processor */
-.macro cpu_id, rd
-	mrc	p15, 0, \rd, c0, c0, 5
-	and	\rd, \rd, #0xF
-.endm
-
-/* loads a 32-bit value into a register without a data access */
-.macro mov32, reg, val
-	movw	\reg, #:lower16:\val
-	movt	\reg, #:upper16:\val
-.endm
arch/arm/mach-tegra/sleep.h new file +56 -0

56 lines not on this page · raw message

Signature · 1 line
1.7.0.4
author
#4 lore raw

PATCH v2 4/6 ARM: tegra30: add CPU hotplug support

In reply to Joseph Lo

Hotplug function put CPUs in offline or online state at runtime. When the CPU been put in the offline state, it was been clock and power gated. Except primary CPU other CPUs can be hotplugged.

Based on the work by:
Scott Williams <[email protected]>
Colin Cross <[email protected]>
Gary King <[email protected]>

---
 arch/arm/mach-tegra/Makefile    |    1 +
 arch/arm/mach-tegra/common.c    |    2 +
 arch/arm/mach-tegra/hotplug.c   |  110 +++++++++------------------------------
 arch/arm/mach-tegra/sleep-t30.S |  107 +++++++++++++++++++++++++++++++++++++
 arch/arm/mach-tegra/sleep.h     |   21 +++++++
 5 files changed, 155 insertions(+), 86 deletions(-)
 create mode 100644 arch/arm/mach-tegra/sleep-t30.S
arch/arm/mach-tegra/Makefile +1 -0
index 060592a..211e279 100644
@@ -17,6 +17,7 @@ obj-$(CONFIG_ARCH_TEGRA_2x_SOC)         += tegra20_clocks_data.o
 obj-$(CONFIG_ARCH_TEGRA_2x_SOC)		+= tegra2_emc.o
 obj-$(CONFIG_ARCH_TEGRA_3x_SOC)		+= tegra30_clocks.o
 obj-$(CONFIG_ARCH_TEGRA_3x_SOC)		+= tegra30_clocks_data.o
+obj-$(CONFIG_ARCH_TEGRA_3x_SOC)		+= sleep-t30.o
 obj-$(CONFIG_SMP)			+= platsmp.o headsmp.o
 obj-$(CONFIG_SMP)                       += reset.o
 obj-$(CONFIG_HOTPLUG_CPU)               += hotplug.o
arch/arm/mach-tegra/common.c +2 -0
index f3654f8..0652056 100644
@@ -34,6 +34,7 @@
 #include "fuse.h"
 #include "pmc.h"
 #include "apbio.h"
+#include "sleep.h"
 
 /*
  * Storage for debug-macro.S's state.
@@ -147,6 +148,7 @@ void __init tegra30_init_early(void)
 	tegra_init_cache(0x441, 0x551);
 	tegra_pmc_init();
 	tegra_powergate_init();
+	tegra30_hotplug_init();
 }
 #endif
 
arch/arm/mach-tegra/hotplug.c +24 -86

140 lines not on this page · raw message

arch/arm/mach-tegra/sleep-t30.S new file +107 -0

107 lines not on this page · raw message

arch/arm/mach-tegra/sleep.h +21 -0
index d0c7a8b..2ad9b6d 100644
@@ -19,6 +19,8 @@
 
 #include <mach/iomap.h>
 
+#define TEGRA_ARM_PERIF_VIRT (TEGRA_ARM_PERIF_BASE - IO_CPU_PHYS \
+					+ IO_CPU_VIRT)
 #define TEGRA_FLOW_CTRL_VIRT (TEGRA_FLOW_CTRL_BASE - IO_PPSB_PHYS \
 					+ IO_PPSB_VIRT)
 
@@ -52,5 +54,24 @@
 	movw	\reg, #:lower16:\val
 	movt	\reg, #:upper16:\val
 .endm
+
+/* Macro to exit SMP coherency. */
+.macro exit_smp, tmp1, tmp2
+	mrc	p15, 0, \tmp1, c1, c0, 1	@ ACTLR
+	bic	\tmp1, \tmp1, #(1<<6) | (1<<0)	@ clear ACTLR.SMP | ACTLR.FW
+	mcr	p15, 0, \tmp1, c1, c0, 1	@ ACTLR
+	isb
+	cpu_id	\tmp1
+	mov	\tmp1, \tmp1, lsl #2
+	mov	\tmp2, #0xf
+	mov	\tmp2, \tmp2, lsl \tmp1
+	mov32	\tmp1, TEGRA_ARM_PERIF_VIRT + 0xC
+	str	\tmp2, [\tmp1]			@ invalidate SCU tags for CPU
+	dsb
+.endm
+#else
+
+void tegra30_hotplug_init(void);
+
 #endif
 #endif
Signature · 1 line
1.7.0.4
author
#5 lore raw

PATCH v2 5/6 ARM: tegra20: add CPU hotplug support

In reply to Joseph Lo

Hotplug function put CPU in offline or online mode at runtime. When the CPU been put into offline, it was been clock gated. The offline CPU can be power gated, when the remaining CPU goes into LP2.

Based on the worked by:
Colin Cross <[email protected]>
Gary King <[email protected]>

---
 arch/arm/mach-tegra/Makefile    |    1 +
 arch/arm/mach-tegra/common.c    |    1 +
 arch/arm/mach-tegra/hotplug.c   |    8 ++++
 arch/arm/mach-tegra/sleep-t20.S |   82 +++++++++++++++++++++++++++++++++++++++
 arch/arm/mach-tegra/sleep.h     |    3 +
 5 files changed, 95 insertions(+), 0 deletions(-)
 create mode 100644 arch/arm/mach-tegra/sleep-t20.S
arch/arm/mach-tegra/Makefile +1 -0
index 211e279..959a429 100644
@@ -15,6 +15,7 @@ obj-$(CONFIG_CPU_IDLE)			+= sleep.o
 obj-$(CONFIG_ARCH_TEGRA_2x_SOC)         += tegra20_clocks.o
 obj-$(CONFIG_ARCH_TEGRA_2x_SOC)         += tegra20_clocks_data.o
 obj-$(CONFIG_ARCH_TEGRA_2x_SOC)		+= tegra2_emc.o
+obj-$(CONFIG_ARCH_TEGRA_2x_SOC)		+= sleep-t20.o
 obj-$(CONFIG_ARCH_TEGRA_3x_SOC)		+= tegra30_clocks.o
 obj-$(CONFIG_ARCH_TEGRA_3x_SOC)		+= tegra30_clocks_data.o
 obj-$(CONFIG_ARCH_TEGRA_3x_SOC)		+= sleep-t30.o
arch/arm/mach-tegra/common.c +1 -0
index 0652056..0560538 100644
@@ -136,6 +136,7 @@ void __init tegra20_init_early(void)
 	tegra_init_cache(0x331, 0x441);
 	tegra_pmc_init();
 	tegra_powergate_init();
+	tegra20_hotplug_init();
 }
 #endif
 #ifdef CONFIG_ARCH_TEGRA_3x_SOC
arch/arm/mach-tegra/hotplug.c +8 -0
index be92d4c..d02a354 100644
@@ -56,6 +56,14 @@ int platform_cpu_disable(unsigned int cpu)
 	return cpu == 0 ? -EPERM : 0;
 }
 
+#ifdef CONFIG_ARCH_TEGRA_2x_SOC
+extern void tegra20_hotplug_shutdown(void);
+void __init tegra20_hotplug_init(void)
+{
+	tegra_hotplug_shutdown = tegra20_hotplug_shutdown;
+}
+#endif
+
 #ifdef CONFIG_ARCH_TEGRA_3x_SOC
 extern void tegra30_hotplug_shutdown(void);
 void __init tegra30_hotplug_init(void)
arch/arm/mach-tegra/sleep-t20.S new file +82 -0

82 lines not on this page · raw message

arch/arm/mach-tegra/sleep.h +3 -0
index 2ad9b6d..42236e1 100644
@@ -23,6 +23,8 @@
 					+ IO_CPU_VIRT)
 #define TEGRA_FLOW_CTRL_VIRT (TEGRA_FLOW_CTRL_BASE - IO_PPSB_PHYS \
 					+ IO_PPSB_VIRT)
+#define TEGRA_CLK_RESET_VIRT (TEGRA_CLK_RESET_BASE - IO_PPSB_PHYS \
+					+ IO_PPSB_VIRT)
 
 #ifdef __ASSEMBLY__
 /* returns the offset of the flow controller halt register for a cpu */
@@ -71,6 +73,7 @@
 .endm
 #else
 
+void tegra20_hotplug_init(void);
 void tegra30_hotplug_init(void);
 
 #endif
Signature · 1 line
1.7.0.4
author
#6 lore raw

PATCH v2 6/6 ARM: tegra: defconfig: enable HOTPLUG_CPU

In reply to Joseph Lo

Enable HOTPLUG_CPU after the SoC specific code to hotplug a CPU been merged.

---
 arch/arm/configs/tegra_defconfig |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)
arch/arm/configs/tegra_defconfig +1 -0
index db22453..e16fd61 100644
@@ -31,6 +31,7 @@ CONFIG_MACH_PAZ00=y
 CONFIG_MACH_TRIMSLICE=y
 CONFIG_TEGRA_EMC_SCALING_ENABLE=y
 CONFIG_SMP=y
+CONFIG_HOTPLUG_CPU=y
 CONFIG_PREEMPT=y
 CONFIG_AEABI=y
 # CONFIG_OABI_COMPAT is not set
Signature · 1 line
1.7.0.4
#7 lore raw

v2 cover ARM: tegra: add CPU hotplug support

In reply to Joseph Lo
Quoted 10 lines · Joseph Lo

This patch set adds CPU hotplug support for Tegra20 and Tegra30 SoCs.

The first 3 patches make the CPU clock and reset control be a module and the abstraction APIs for TegraXX SoCs. And clean up some codes for later usage. Then add CPU hotplug support for Tegra20 and Tegra30.

Verified on Seaboard(Tegra20) and Cardhu(Tegra30)

V2:
* add one more patch for enable HOTPLUG_CPU [6/6]

The build problem when HOTPLUG_CPU is disabled is still present. The kernel needs to build with this option disabled.

Enabling HOTPLUG_CPU in defconfig doesn't force everyone to enable that configuration option; not everyone uses tegra_defconfig, people don't always update to the latest defconfig, people should be free to disable functionality they don't need and still get a buildable & working kernel, etc.

Besides, even if defconfig were a legitimate solution to this problem, this series still wouldn't work with "git bisect", since the defconfig patch comes after the patch that causes the build failure.

Quoted 1 line

* move all the SoC specific CPU shutdown code into platform_cpu_die [4/6]

The reboot problem is indeed solved.

P.S. It'd be nice to include a summary of what changed in each affected patch, as well as in the cover letter.

author
#8 lore raw

v2 cover ARM: tegra: add CPU hotplug support

Hi Stephen,

Quoted 12 lines · Stephen Warren
Quoted 10 lines · Joseph Lo

This patch set adds CPU hotplug support for Tegra20 and Tegra30 SoCs.

The first 3 patches make the CPU clock and reset control be a module and the abstraction APIs for TegraXX SoCs. And clean up some codes for later usage. Then add CPU hotplug support for Tegra20 and Tegra30.

Verified on Seaboard(Tegra20) and Cardhu(Tegra30)

V2:
* add one more patch for enable HOTPLUG_CPU [6/6]

Quoted 9 lines

The build problem when HOTPLUG_CPU is disabled is still present. The kernel needs to build with this option disabled.

Enabling HOTPLUG_CPU in defconfig doesn't force everyone to enable that configuration option; not everyone uses tegra_defconfig, people don't always update to the latest defconfig, people should be free to disable functionality they don't need and still get a buildable & working kernel, etc.

Obviously, I didn't get your problem at your first reply. Sorry.

I fixed that and re-checked it again with Tegra20-only, Tegra30-only and both supported kernel. And checked with "git bisect" as well. It should be OK now.

Thanks,
Joseph

Quoted 4 lines

Besides, even if defconfig were a legitimate solution to this problem, this series still wouldn't work with "git bisect", since the defconfig patch comes after the patch that causes the build failure.