Skip to contentSkip to conversation

Help fix SMP boot issues with L2 cache

PATCH New Russell King - ARM Linux 3 participants 6 messages Feb 12, 2010 – Feb 19, 2010 Arm (32-bit)

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 Help fix SMP boot issues with L2 cache

ARM: Add L2 cache handling to smp boot support

The page table and secondary data which we're asking the secondary CPU to make use of has to hit RAM to ensure that the secondary CPU can see it since it may not be taking part in coherency or cache searches at this point.

 arch/arm/kernel/smp.c |    4 +++-
 1 files changed, 3 insertions(+), 1 deletions(-)
arch/arm/kernel/smp.c +3 -1
index 57162af..577543f 100644
@@ -99,6 +99,7 @@ int __cpuinit __cpu_up(unsigned int cpu)
 	*pmd = __pmd((PHYS_OFFSET & PGDIR_MASK) |
 		     PMD_TYPE_SECT | PMD_SECT_AP_WRITE);
 	flush_pmd_entry(pmd);
+	outer_clean_range(__pa(pmd), __pa(pmd + 1));
 
 	/*
 	 * We need to tell the secondary core where to find
@@ -106,7 +107,8 @@ int __cpuinit __cpu_up(unsigned int cpu)
 	 */
 	secondary_data.stack = task_stack_page(idle) + THREAD_START_SP;
 	secondary_data.pgdir = virt_to_phys(pgd);
-	wmb();
+	__cpuc_flush_dcache_area(&secondary_data, sizeof(secondary_data));
+	outer_clean_range(__pa(&secondary_data), __pa(&secondary_data + 1));
 
 	/*
 	 * Now bring the CPU into our world.
#1 lore raw

PATCH 1/1 Help fix SMP boot issues with L2 cache

Quoted 25 lines · Russell King - ARM Linux

ARM: Add L2 cache handling to smp boot support

The page table and secondary data which we're asking the secondary CPU to make use of has to hit RAM to ensure that the secondary CPU can see it since it may not be taking part in coherency or cache searches at this point.

 arch/arm/kernel/smp.c |    4 +++-
 1 files changed, 3 insertions(+), 1 deletions(-)
arch/arm/kernel/smp.c +1 -0
index 57162af..577543f 100644
@@ -99,6 +99,7 @@ int __cpuinit __cpu_up(unsigned int cpu)
 	*pmd = __pmd((PHYS_OFFSET & PGDIR_MASK) |
 		     PMD_TYPE_SECT | PMD_SECT_AP_WRITE);
 	flush_pmd_entry(pmd);
+	outer_clean_range(__pa(pmd), __pa(pmd + 1));
 
 	/*
 	 * We need to tell the secondary core where to find

arch/arm/kernel/smp.c int __cpuinit __cpu_up(unsigned int cpu) · line 114

Quoted 9 lines
arch/arm/kernel/smp.c +2 -1
@@ -106,7 +107,8 @@ int __cpuinit __cpu_up(unsigned int cpu)
 	 */
 	secondary_data.stack = task_stack_page(idle) + THREAD_START_SP;
 	secondary_data.pgdir = virt_to_phys(pgd);
-	wmb();
+	__cpuc_flush_dcache_area(&secondary_data, sizeof(secondary_data));
+	outer_clean_range(__pa(&secondary_data), __pa(&secondary_data + 1));
 
 	/*
 	 * Now bring the CPU into our world.

But is the outer_cache initialised at this point? I recall for RealView I had to change the l2x0_init to an early_initcall:

http://linux-arm.org/git?p=linux-2.6-stable.git;a=commitdiff;h=70e1b8d308f635eaeb7dad14fc5c79f082a147ef;hp=f034a4ab31b0663a5b68c65503122bed1eaca349

I think I have a patch for 2.6.33, I'll dig it out on Monday. But this was only needed when running the kernel in non-secure mode and the L2 was pre-initialised by the secure monitor (it may be the case on some Omap boards).

Signature · 1 line
Catalin
#2 lore raw

PATCH 1/1 Help fix SMP boot issues with L2 cache

Quoted 26 lines · Catalin Marinas
Quoted 25 lines · Russell King - ARM Linux

ARM: Add L2 cache handling to smp boot support
The page table and secondary data which we're asking the
secondary CPU
to make use of has to hit RAM to ensure that the secondary CPU can see it since it may not be taking part in coherency or cache searches at this point.

 arch/arm/kernel/smp.c |    4 +++-
 1 files changed, 3 insertions(+), 1 deletions(-)
arch/arm/kernel/smp.c +1 -0
index 57162af..577543f 100644
@@ -99,6 +99,7 @@ int __cpuinit __cpu_up(unsigned int cpu)
 	*pmd = __pmd((PHYS_OFFSET & PGDIR_MASK) |
 		     PMD_TYPE_SECT | PMD_SECT_AP_WRITE);
 	flush_pmd_entry(pmd);
+	outer_clean_range(__pa(pmd), __pa(pmd + 1));
  	/*
 	 * We need to tell the secondary core where to find
@@ -106,7 +107,8 @@ int __cpuinit __cpu_up(unsigned int cpu)
 	 */
 	secondary_data.stack = task_stack_page(idle) + THREAD_START_SP;
Quoted 8 lines

arch/arm/kernel/smp.c int __cpuinit __cpu_up(unsigned int cpu) · line 114

Quoted 6 lines
arch/arm/kernel/smp.c +2 -1
 	secondary_data.pgdir = virt_to_phys(pgd);
-	wmb();
+	__cpuc_flush_dcache_area(&secondary_data, sizeof(secondary_data));
+	outer_clean_range(__pa(&secondary_data), __pa(&secondary_data + 1));
  	/*
 	 * Now bring the CPU into our world.

But is the outer_cache initialised at this point?

If it isn't, the calls are no-ops, so are harmless. However, there are platforms where the L2 is already initialized at this point, and therefore this needs to be handled.

Whether platforms have initialized their L2 cache support while the hardware is enabled is their own lookout at this point - that's outside of the scope of this patch.

Quoted 4 lines

I recall for RealView
I had to change the l2x0_init to an early_initcall:

http://linux-arm.org/git?p=linux-2.6-stable.git;a=commitdiff;h=70e1b8d308f635eaeb7dad14fc5c79f082a147ef;hp=f034a4ab31b0663a5b68c65503122bed1eaca349

And that's what I'm addressing with this patch - the need to patch every platsmp.c file with that kind of dirty hack. With my patch, the SMP generic data structures will be covered by the appropriate code, rather than having every SMP platform having to do that itself.

Please update your patch for mine - the only thing you need to look after is 'pen_release'.

#3 lore raw

PATCH 1/1 Help fix SMP boot issues with L2 cache

Ping. Acks would be nice.

Quoted 61 lines · Russell King - ARM Linux
Quoted 26 lines · Catalin Marinas
Quoted 25 lines · Russell King - ARM Linux

ARM: Add L2 cache handling to smp boot support
The page table and secondary data which we're asking the
secondary CPU
to make use of has to hit RAM to ensure that the secondary CPU can see it since it may not be taking part in coherency or cache searches at this point.

 arch/arm/kernel/smp.c |    4 +++-
 1 files changed, 3 insertions(+), 1 deletions(-)
arch/arm/kernel/smp.c +1 -0
index 57162af..577543f 100644
@@ -99,6 +99,7 @@ int __cpuinit __cpu_up(unsigned int cpu)
 	*pmd = __pmd((PHYS_OFFSET & PGDIR_MASK) |
 		     PMD_TYPE_SECT | PMD_SECT_AP_WRITE);
 	flush_pmd_entry(pmd);
+	outer_clean_range(__pa(pmd), __pa(pmd + 1));
  	/*
 	 * We need to tell the secondary core where to find
@@ -106,7 +107,8 @@ int __cpuinit __cpu_up(unsigned int cpu)
 	 */
 	secondary_data.stack = task_stack_page(idle) + THREAD_START_SP;
Quoted 8 lines

arch/arm/kernel/smp.c int __cpuinit __cpu_up(unsigned int cpu) · line 114

Quoted 6 lines
arch/arm/kernel/smp.c +2 -1
 	secondary_data.pgdir = virt_to_phys(pgd);
-	wmb();
+	__cpuc_flush_dcache_area(&secondary_data, sizeof(secondary_data));
+	outer_clean_range(__pa(&secondary_data), __pa(&secondary_data + 1));
  	/*
 	 * Now bring the CPU into our world.

But is the outer_cache initialised at this point?

If it isn't, the calls are no-ops, so are harmless. However, there are platforms where the L2 is already initialized at this point, and therefore this needs to be handled.

Whether platforms have initialized their L2 cache support while the hardware is enabled is their own lookout at this point - that's outside of the scope of this patch.

Quoted 4 lines

I recall for RealView
I had to change the l2x0_init to an early_initcall:

http://linux-arm.org/git?p=linux-2.6-stable.git;a=commitdiff;h=70e1b8d308f635eaeb7dad14fc5c79f082a147ef;hp=f034a4ab31b0663a5b68c65503122bed1eaca349

And that's what I'm addressing with this patch - the need to patch every platsmp.c file with that kind of dirty hack. With my patch, the SMP generic data structures will be covered by the appropriate code, rather than having every SMP platform having to do that itself.

Please update your patch for mine - the only thing you need to look after is 'pen_release'.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

#4 lore raw

PATCH 1/1 Help fix SMP boot issues with L2 cache

Quoted 7 lines

-----Original Message-----
From: Russell King - ARM Linux [mailto:linux at arm.linux.org.uk]
Sent: Thursday, February 18, 2010 10:43 PM
To: Catalin Marinas; Shilimkar, Santosh
Cc: linux-arm-kernel at lists.infradead.org
Subject: Re: [PATCH] Help fix SMP boot issues with L2 cache

Quoted 38 lines

Ping. Acks would be nice.

Quoted 35 lines · Russell King - ARM Linux
Quoted 34 lines · Catalin Marinas
Quoted 23 lines · Russell King - ARM Linux

ARM: Add L2 cache handling to smp boot support
The page table and secondary data which we're asking the
secondary CPU
to make use of has to hit RAM to ensure that the secondary CPU can see it since it may not be taking part in coherency or cache searches at this point.

 arch/arm/kernel/smp.c |    4 +++-
 1 files changed, 3 insertions(+), 1 deletions(-)
arch/arm/kernel/smp.c +1 -0
index 57162af..577543f 100644
@@ -99,6 +99,7 @@ int __cpuinit __cpu_up(unsigned int cpu)
 	*pmd = __pmd((PHYS_OFFSET & PGDIR_MASK) |
 		     PMD_TYPE_SECT | PMD_SECT_AP_WRITE);
 	flush_pmd_entry(pmd);
+	outer_clean_range(__pa(pmd), __pa(pmd + 1));
  	/*
 	 * We need to tell the secondary core where to find

arch/arm/kernel/smp.c int __cpuinit __cpu_up(unsigned int cpu) · line 113

Quoted 8 lines
arch/arm/kernel/smp.c +2 -1
@@ -106,7 +107,8 @@ int __cpuinit __cpu_up(unsigned int cpu)
 	 */
 	secondary_data.stack = task_stack_page(idle) + THREAD_START_SP;
 	secondary_data.pgdir = virt_to_phys(pgd);
-	wmb();
+	__cpuc_flush_dcache_area(&secondary_data, sizeof(secondary_data));
+	outer_clean_range(__pa(&secondary_data), __pa(&secondary_data + 1));
  	/*
 	 * Now bring the CPU into our world.

But is the outer_cache initialised at this point?

Outer cache is initialized by this time, so it's good. Infact this is the right place to flush the secondary data and page tables instead of every arm-smp platform doing that as part of "boot_secondary".

Regards,
Santosh

#5 lore raw

PATCH 1/1 Help fix SMP boot issues with L2 cache

In reply to Russell King - ARM Linux
Quoted 36 lines · Russell King - ARM Linux
Quoted 26 lines · Catalin Marinas
Quoted 25 lines · Russell King - ARM Linux

ARM: Add L2 cache handling to smp boot support
The page table and secondary data which we're asking the
secondary CPU
to make use of has to hit RAM to ensure that the secondary CPU can see it since it may not be taking part in coherency or cache searches at this point.

 arch/arm/kernel/smp.c |    4 +++-
 1 files changed, 3 insertions(+), 1 deletions(-)
arch/arm/kernel/smp.c +1 -0
index 57162af..577543f 100644
@@ -99,6 +99,7 @@ int __cpuinit __cpu_up(unsigned int cpu)
     *pmd = __pmd((PHYS_OFFSET & PGDIR_MASK) |
                  PMD_TYPE_SECT | PMD_SECT_AP_WRITE);
     flush_pmd_entry(pmd);
+    outer_clean_range(__pa(pmd), __pa(pmd + 1));
     /*
      * We need to tell the secondary core where to find
@@ -106,7 +107,8 @@ int __cpuinit __cpu_up(unsigned int cpu)
      */
     secondary_data.stack = task_stack_page(idle) + THREAD_START_SP;
Quoted 8 lines

arch/arm/kernel/smp.c int __cpuinit __cpu_up(unsigned int cpu) · line 114

Quoted 6 lines
arch/arm/kernel/smp.c +2 -1
     secondary_data.pgdir = virt_to_phys(pgd);
-    wmb();
+    __cpuc_flush_dcache_area(&secondary_data, sizeof(secondary_data));
+    outer_clean_range(__pa(&secondary_data), __pa(&secondary_data + 1));
     /*
      * Now bring the CPU into our world.

But is the outer_cache initialised at this point?

Quoted 3 lines

If it isn't, the calls are no-ops, so are harmless. However, there are platforms where the L2 is already initialized at this point, and therefore this needs to be handled.

Yes, for what it is intended the patch is fine:

Signature · 1 line
Catalin