DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1445B20672
Date: Thu, 6 Dec 2018 14:37:03 +0000
From: Russell King - ARM Linux <linux@armlinux.org.uk>
To: Krzysztof Kozlowski <krzk@kernel.org>
Subject: Re: [PATCH v2 5/5] ARM: spectre-v2: per-CPU vtables to work around
 big.Little systems
Message-ID: <20181206143703.GV30658@n2100.armlinux.org.uk>
References: <20181106123700.GA30658@n2100.armlinux.org.uk>
 <CGME20181106124037epcas4p35f0ed44d430e44310e764190f15c2883@epcas4p3.samsung.com>
 <E1gK0dU-0007Mp-4h@rmk-PC.armlinux.org.uk>
 <dda0be26-5ac8-a158-3210-4bc6861bede2@samsung.com>
 <20181206100052.GR30658@n2100.armlinux.org.uk>
 <CAJKOXPdUFfOXqMkgdv4fjhpbrmjNmw_f9cMj03a2h65snNJ4qw@mail.gmail.com>
 <20181206123956.GS30658@n2100.armlinux.org.uk>
 <CAJKOXPf=nGN53EK=+bjhKyaY_knMnY5J+JSW2Jb6KRQuJy37vA@mail.gmail.com>
 <20181206140717.GU30658@n2100.armlinux.org.uk>
 <CAJKOXPdUABPbni+1H=tTvA5wKe8Rc8EPTFKtLGMwQ8yLcb3=gg@mail.gmail.com>
MIME-Version: 1.0
Content-Disposition: inline
In-Reply-To: <CAJKOXPdUABPbni+1H=tTvA5wKe8Rc8EPTFKtLGMwQ8yLcb3=gg@mail.gmail.com>
User-Agent: Mutt/1.5.23 (2014-03-12)
Precedence: list
Cc: mark.rutland@arm.com, julien.thierry@arm.com, marc.zyngier@arm.com,
 will.deacon@arm.com, Morten.Rasmussen@arm.com, Qais.Yousef@arm.com,
 dietmar.eggemann@arm.com, linux-arm-kernel@lists.infradead.org,
 Marek Szyprowski <m.szyprowski@samsung.com>
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Sender: "linux-arm-kernel" <linux-arm-kernel-bounces@lists.infradead.org>
Errors-To: linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org

On Thu, Dec 06, 2018 at 03:30:22PM +0100, Krzysztof Kozlowski wrote:
> On Thu, 6 Dec 2018 at 15:07, Russell King - ARM Linux
> <linux@armlinux.org.uk> wrote:
> >
> > On Thu, Dec 06, 2018 at 02:54:10PM +0100, Krzysztof Kozlowski wrote:
> > > On Thu, 6 Dec 2018 at 13:40, Russell King - ARM Linux
> > > <linux@armlinux.org.uk> wrote:
> > > >
> > > > On Thu, Dec 06, 2018 at 11:24:27AM +0100, Krzysztof Kozlowski wrote:
> > > > > On Thu, 6 Dec 2018 at 11:01, Russell King - ARM Linux
> > > > > <linux@armlinux.org.uk> wrote:
> > > > > > I've no idea based on what you've supplied given that the SoC
> > > > > > maintainers are responsible for writing the code to deal with hotplug
> > > > > > etc, and Exynos's code there is something of a maze.  It's not clear
> > > > > > which bits are being used.  I think you at the very least need to debug
> > > > > > to find out whether the problem is at CPU down or CPU up.
> > > > > >
> > > > > > From the ARM architecture point of view, for Cortex A9, all the
> > > > > > processor function instances should be identical.  The only difference
> > > > > > as a result of the patch is that we'll be calling smp_processor_id()
> > > > > > early (which should be fine), and indirecting through the cpu_vtable[]
> > > > > > array rather than merely dereferencing the processor struct.
> > > > > >
> > > > > > What about checking dmesg - messages from offline CPUs do not appear
> > > > > > on the console(s) but are still logged in the kernel log.
> > > > > >
> > > > > > You could try making PROC_VTABLE() the same as PROC_TABLE() (iow,
> > > > > > always access cpu_vtable[0]) to see whether it's the smp_processor_id()
> > > > > > that's causing your problem or not.  If it is, then try and work out
> > > > > > which of the processor functions is causing it by restoring
> > > > > > PROC_VTABLE() and then switching each from PROC_VTABLE() to
> > > > > > PROC_TABLE() until it does work.
> > > > >
> > > > > Thanks for hints!
> > > >
> > > > So I can plan, how long do you think it will take to get some results
> > > > from my suggestions above?
> > >
> > > For Suspend to RAM, on v4.20-rc3, this warning appears:
> > > [   84.046722] WARNING: CPU: 1 PID: 0 at
> > > ../arch/arm/include/asm/proc-fns.h:124
> > > secondary_start_kernel+0x214/0x26c
> > > (difference between dcache_clean_area)
> > > The secondary CPUs bringup fails with ETIMEDOUT.
> >
> > That basically means that the dcache_clean_area method for CPU1
> > differs from the dcache_clean_area method for CPU0.  If all your
> > CPUs are identical, that definitely should not be happening.
> >
> > Hmm.  Interestingly, OMAP4430 passes hotplug tests just fine.
> >
> > Please try this patch.
> 
> This fixes both hotplug and suspend to RAM. I was trying to narrow why
> the pointers to all processor functions differ. During first boot they
> were OK but it seems they were changed just before suspend.

Thanks for testing.  I think this is probably a better patch which
should end up with the same result.

I suspect no one else has noticed because most people have big.Little
support disabled - that'd explain why it doesn't show up on OMAP4.

diff --git a/arch/arm/mm/proc-macros.S b/arch/arm/mm/proc-macros.S
index 81d0efb055c6..44f9776139a8 100644
--- a/arch/arm/mm/proc-macros.S
+++ b/arch/arm/mm/proc-macros.S
@@ -274,6 +274,13 @@
 	.endm
 
 .macro define_processor_functions name:req, dabort:req, pabort:req, nommu=0, suspend=0, bugs=0
+/*
+ * If we are building for big.Little with branch predictor hardening,
+ * we need the processor function tables to remain available after boot.
+ */
+#if defined(CONFIG_BIG_LITTLE) && defined(CONFIG_HARDEN_BRANCH_PREDICTOR)
+	.rodata
+#endif
 	.type	\name\()_processor_functions, #object
 	.align 2
 ENTRY(\name\()_processor_functions)
@@ -309,6 +316,9 @@ ENTRY(\name\()_processor_functions)
 	.endif
 
 	.size	\name\()_processor_functions, . - \name\()_processor_functions
+#if defined(CONFIG_BIG_LITTLE) && defined(CONFIG_HARDEN_BRANCH_PREDICTOR)
+	.previous
+#endif
 .endm
 
 .macro define_cache_functions name:req


-- 
RMK's Patch system: http://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
According to speedtest.net: 11.9Mbps down 500kbps up

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel