Skip to contentSkip to conversation

ARM: Flush L2 cache on soft_restart

RFC PATCH New Taras Kondratiuk 3 participants 6 messages Oct 2, 2013 – Oct 3, 2013 Arm (32-bit)

lore
Share & tools
lore
#0 lore raw

RFC PATCH 1/1 ARM: Flush L2 cache on soft_restart

Kexec disables L2 cache before jumping to reboot code, but it doesn't flush it. So often just copied reboot code gets corrupted, because part of it is stored in L2 cache and have not reached memory.

Flushing cache prevents this corruption.

I'm facing this issue on Pandaboard ES, but it looks like similar issue is observed on TC2 board [1].

[1] http://www.spinics.net/lists/arm-kernel/msg264339.html

---
 arch/arm/kernel/process.c |    6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
arch/arm/kernel/process.c +4 -2
index 94f6b05..e359b62 100644
@@ -103,9 +103,11 @@ void soft_restart(unsigned long addr)
 	local_irq_disable();
 	local_fiq_disable();
 
-	/* Disable the L2 if we're the last man standing. */
-	if (num_online_cpus() == 1)
+	/* Flush and disable the L2 if we're the last man standing. */
+	if (num_online_cpus() == 1) {
+		outer_flush_all();
 		outer_disable();
+	}
 
 	/* Change to the new stack and continue with the reset. */
 	call_with_stack(__soft_restart, (void *)addr, (void *)stack);
Signature · 1 line
1.7.9.5
#1 lore raw

RFC PATCH 1/1 ARM: Flush L2 cache on soft_restart

Quoted 9 lines · Taras Kondratiuk

Kexec disables L2 cache before jumping to reboot code, but it doesn't flush it. So often just copied reboot code gets corrupted, because part of it is stored in L2 cache and have not reached memory.

Flushing cache prevents this corruption.

I'm facing this issue on Pandaboard ES, but it looks like similar issue is observed on TC2 board [1].

TC2 doesn't have an outer cache, so that report is not relevant to this patch.

Quoted 12 lines

[1] http://www.spinics.net/lists/arm-kernel/msg264339.html

---
 arch/arm/kernel/process.c |    6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
arch/arm/kernel/process.c +0 -0
index 94f6b05..e359b62 100644

arch/arm/kernel/process.c void soft_restart(unsigned long addr) · line 109

Quoted 9 lines
arch/arm/kernel/process.c +3 -2
@@ -103,9 +103,11 @@ void soft_restart(unsigned long addr)
 	local_irq_disable();
 	local_fiq_disable();
 
-	/* Disable the L2 if we're the last man standing. */
-	if (num_online_cpus() == 1)
+	/* Flush and disable the L2 if we're the last man standing. */
+	if (num_online_cpus() == 1) {
+		outer_flush_all();
 		outer_disable();

l2x0_disable already contains a flush, so this doesn't change anything.

Will

#2 lore raw

RFC PATCH 1/1 ARM: Flush L2 cache on soft_restart

Quoted 9 lines · Will Deacon
Quoted 8 lines · Taras Kondratiuk
arch/arm/kernel/process.c +0 -0
index 94f6b05..e359b62 100644
@@ -103,9 +103,11 @@ void soft_restart(unsigned long addr)
      local_irq_disable();
      local_fiq_disable();
 
Quoted 8 lines

arch/arm/kernel/process.c void soft_restart(unsigned long addr) · line 109

Quoted 6 lines
arch/arm/kernel/process.c +3 -2
-     /* Disable the L2 if we're the last man standing. */
-     if (num_online_cpus() == 1)
+     /* Flush and disable the L2 if we're the last man standing. */
+     if (num_online_cpus() == 1) {
+             outer_flush_all();
              outer_disable();

l2x0_disable already contains a flush, so this doesn't change anything.

Unfortunately not everybody uses l2x0_disable().
SoC's that use SMC calls for L2 cache maintenance have its own implementation of outer_cache.disable which usually doesn't flush cache implicitly.

Signature · 2 lines
Regards,
Taras Kondratiuk
#3 lore raw

RFC PATCH 1/1 ARM: Flush L2 cache on soft_restart

Quoted 19 lines · Taras Kondratiuk
Quoted 9 lines · Will Deacon
Quoted 8 lines · Taras Kondratiuk
arch/arm/kernel/process.c +0 -0
index 94f6b05..e359b62 100644
@@ -103,9 +103,11 @@ void soft_restart(unsigned long addr)
      local_irq_disable();
      local_fiq_disable();
 
Quoted 8 lines

arch/arm/kernel/process.c void soft_restart(unsigned long addr) · line 109

Quoted 6 lines
arch/arm/kernel/process.c +3 -2
-     /* Disable the L2 if we're the last man standing. */
-     if (num_online_cpus() == 1)
+     /* Flush and disable the L2 if we're the last man standing. */
+     if (num_online_cpus() == 1) {
+             outer_flush_all();
              outer_disable();

l2x0_disable already contains a flush, so this doesn't change anything.

Quoted 3 lines

Unfortunately not everybody uses l2x0_disable().
SoC's that use SMC calls for L2 cache maintenance have its own implementation of outer_cache.disable which usually doesn't flush cache implicitly.

In which case, we should probably fix the disabling code to make a flush then update callers not to bother with redundant flushing. The flushing during the disable code is likely required anyway if there's any synchronisation going on.

Will

#4 lore raw

RFC PATCH 1/1 ARM: Flush L2 cache on soft_restart

Quoted 20 lines · Will Deacon
Quoted 19 lines · Taras Kondratiuk
Quoted 17 lines · Will Deacon
Quoted 5 lines · Taras Kondratiuk
arch/arm/kernel/process.c +0 -0
index 94f6b05..e359b62 100644

arch/arm/kernel/process.c void soft_restart(unsigned long addr) · line 109

Quoted 9 lines
arch/arm/kernel/process.c +3 -2
@@ -103,9 +103,11 @@ void soft_restart(unsigned long addr)
      local_irq_disable();
      local_fiq_disable();
 
-     /* Disable the L2 if we're the last man standing. */
-     if (num_online_cpus() == 1)
+     /* Flush and disable the L2 if we're the last man standing. */
+     if (num_online_cpus() == 1) {
+             outer_flush_all();
              outer_disable();

l2x0_disable already contains a flush, so this doesn't change anything.

Quoted 8 lines
Quoted 3 lines

Unfortunately not everybody uses l2x0_disable().
SoC's that use SMC calls for L2 cache maintenance have its own implementation of outer_cache.disable which usually doesn't flush cache implicitly.

In which case, we should probably fix the disabling code to make a flush then update callers not to bother with redundant flushing. The flushing during the disable code is likely required anyway if there's any synchronisation going on.

It makes sense, but a history of the current code looks a bit confusing. Implicit flush was added in l2x0_disable() as a "side effect" of commit 38a8914f9ac2379293944f613e6ca24b61373de8
"ARM: 6987/1: l2x0: fix disabling function to avoid deadlock", while initially disable was just a disable.

Maybe it worth to explicitly document that .disable callback must flush cache?

Signature · 2 lines
Regards,
Taras Kondratiuk
#5 lore raw

RFC PATCH 1/1 ARM: Flush L2 cache on soft_restart

In reply to Will Deacon
Quoted 29 lines · Will Deacon
Quoted 9 lines · Taras Kondratiuk

Kexec disables L2 cache before jumping to reboot code, but it doesn't flush it. So often just copied reboot code gets corrupted, because part of it is stored in L2 cache and have not reached memory.

Flushing cache prevents this corruption.

I'm facing this issue on Pandaboard ES, but it looks like similar issue is observed on TC2 board [1].

TC2 doesn't have an outer cache, so that report is not relevant to this patch.

Quoted 15 lines

[1] http://www.spinics.net/lists/arm-kernel/msg264339.html

---
 arch/arm/kernel/process.c |    6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
arch/arm/kernel/process.c +0 -0
index 94f6b05..e359b62 100644
@@ -103,9 +103,11 @@ void soft_restart(unsigned long addr)
 	local_irq_disable();
 	local_fiq_disable();
 
Quoted 8 lines

arch/arm/kernel/process.c void soft_restart(unsigned long addr) · line 109

Quoted 6 lines
arch/arm/kernel/process.c +3 -2
-	/* Disable the L2 if we're the last man standing. */
-	if (num_online_cpus() == 1)
+	/* Flush and disable the L2 if we're the last man standing. */
+	if (num_online_cpus() == 1) {
+		outer_flush_all();
 		outer_disable();

l2x0_disable already contains a flush, so this doesn't change anything.

The disable call _has_ to contain the flush as that's the only way it can be disabled to avoid any loss of data when the outer cache stops being searched for cache hits.