NFS problems on a MPC5200-based board
Hi,
This is a follow-up on NFS problems on an MPC5200-based board reported here a while back:
http://www.nabble.com/-PATCH--Add-support-for-the-digsy-MTC-board.-to21750004.html#a21792612
To recap: while using NFS, especially while mounting the root filesystem over NFS, the system is really slow and displays a bunch of "nfs: server 192.168.1.1 not responding, still trying" messages. Sometimes it is able to get to the login prompt, sometimes not. In cases where the login is successful, the system is still extremely sluggish (console hangs for tens of seconds and longer).
git bisect narrows down the troublesome commit as:
commit 4c456a67f501b8b15542c7c21c28812bf88f484b
Author: Gerhard Pircher <[email protected]> Date: Fri Jan 23 06:51:28 2009 +0000
powerpc/mm: Fix handling of _PAGE_COHERENT in BAT setup code
_PAGE_COHERENT is now always set in _PAGE_RAM resp. PAGE_KERNEL. Thus it has to be masked out, if the BAT mapping should be non cacheable or CPU_FTR_NEED_COHERENT is not set.
This will work on normal SMP setups because we force-set CPU_FTR_NEED_COHERENT as part of CPU_FTR_COMMON on SMP.
We have tested recent mainline kernel (past 2.6.29-rc7) with the 4c456a6... commit reverted and NFS problems went away.
Other people have also reported similar problems (original posters on Cc):
http://www.nabble.com/-PATCH--Add-support-for-the-digsy-MTC-board.-tp21750004p21792825.html
http://www.nabble.com/-PATCH--Add-support-for-the-digsy-MTC-board.-tp21750004p21792612.html
The commit in question does not look directly related to NFS/networking; moreover it is a fix for some other problem, so just reverting it is not an option, it seems (?). So how do we go about having NFS operational again? Any comments?
Regards,
Bartlomiej Sieka
Quoted 7 lines · Bartłomiej Sięka
Hi,
This is a follow-up on NFS problems on an MPC5200-based board reported here a while back:
http://www.nabble.com/-PATCH--Add-support-for-the-digsy-MTC-board.-to21750004.html#a21792612
Quoted 8 lines
To recap: while using NFS, especially while mounting the root filesystem over NFS, the system is really slow and displays a bunch of "nfs: server 192.168.1.1 not responding, still trying" messages. Sometimes it is able to get to the login prompt, sometimes not. In cases where the login is successful, the system is still extremely sluggish (console hangs for tens of seconds and longer).
git bisect narrows down the troublesome commit as:
Maybe you need to set CPU_FTR_NEED_COHERENT for the 5200 ?
Cheers,
Ben.
Quoted 33 lines
commit
4c456a67f501b8b15542c7c21c28812bf88f484bAuthor: Gerhard Pircher <[email protected]> Date: Fri Jan 23 06:51:28 2009 +0000powerpc/mm: Fix handling of
_PAGE_COHERENTin BAT setup code
_PAGE_COHERENTis now always set in_PAGE_RAMresp.PAGE_KERNEL. Thus it has to be masked out, if the BAT mapping should be non cacheable orCPU_FTR_NEED_COHERENTis not set.This will work on normal SMP setups because we force-set
CPU_FTR_NEED_COHERENTas part ofCPU_FTR_COMMONon SMP.Signed-off-by Gerhard Pircher <[email protected]>Signed-off-by Benjamin Herrenschmidt <[email protected]>We have tested recent mainline kernel (past 2.6.29-rc7) with the 4c456a6...
commit reverted and NFS problems went away.Other people have also reported similar problems (original posters on Cc):
http://www.nabble.com/-PATCH--Add-support-for-the-digsy-MTC-board.-tp21750004p21792825.html
http://www.nabble.com/-PATCH--Add-support-for-the-digsy-MTC-board.-tp21750004p21792612.htmlThe commit in question does not look directly related to NFS/ networking; moreover it is a fix for some other problem, so just reverting it is not an option, it seems (?). So how do we go about having NFS operational again? Any comments?
Regards,
Bartlomiej Sieka
-------- Original-Nachricht --------
Quoted 22 lines
Datum: Thu, 12 Mar 2009 08:39:26 +1100
Von: Benjamin Herrenschmidt <[email protected]>
An: "Bartłomiej Sięka" <[email protected]>
CC: [email protected], [email protected], Grant Likely <[email protected]>
Betreff: Re: NFS problems on a MPC5200-based boardQuoted 6 lines · Bartłomiej Sięka
Hi,
This is a follow-up on NFS problems on an MPC5200-based board reported here a while back:
http://www.nabble.com/-PATCH--Add-support-for-the-digsy-MTC-board.-to21750004.html#a21792612
Quoted 8 lines
To recap: while using NFS, especially while mounting the root filesystem over NFS, the system is really slow and displays a bunch of "nfs: server 192.168.1.1 not responding, still trying" messages. Sometimes it is able to get to the login prompt, sometimes not. In cases where the login is successful, the system is still extremely sluggish (console hangs for tens of seconds and longer).
Quoted 3 lines
Quoted 1 line
git bisect narrows down the troublesome commit as:
Maybe you need to set
CPU_FTR_NEED_COHERENTfor the 5200 ?
I would say the same, as the patch just replicates the CPU_FTR_NEED_COHERENT handling of the hash page table code.
regards,
Gerhard
Quoted 35 lines
Quoted 17 lines
commit
4c456a67f501b8b15542c7c21c28812bf88f484bAuthor: Gerhard Pircher <[email protected]> Date: Fri Jan 23 06:51:28 2009 +0000powerpc/mm: Fix handling of
_PAGE_COHERENTin BAT setup code
_PAGE_COHERENTis now always set in_PAGE_RAMresp.PAGE_KERNEL. Thus it has to be masked out, if the BAT mapping should be non cacheable orCPU_FTR_NEED_COHERENTis not set.This will work on normal SMP setups because we force-set
CPU_FTR_NEED_COHERENTas part ofCPU_FTR_COMMONon SMP.Signed-off-by Gerhard Pircher <[email protected]>Signed-off-by Benjamin Herrenschmidt <[email protected]>Quoted 7 lines
We have tested recent mainline kernel (past 2.6.29-rc7) with the 4c456a6...
commit reverted and NFS problems went away.Other people have also reported similar problems (original posters on Cc):
http://www.nabble.com/-PATCH--Add-support-for-the-digsy-MTC-board.-tp21750004p21792825.html
Quoted 1 line
http://www.nabble.com/-PATCH--Add-support-for-the-digsy-MTC-board.-tp21750004p21792612.html
Quoted 8 lines
The commit in question does not look directly related to NFS/ networking; moreover it is a fix for some other problem, so just reverting it is not an option, it seems (?). So how do we go about having NFS operational again? Any comments?
Regards,
Bartlomiej Sieka
Signature · 1 line
Psssst! Schon vom neuen GMX MultiMessenger gehört? Der kann`s mit allen: http://www.gmx.net/de/go/multimessenger01
PATCH 1/1 powerpc: Enable CPU_FTR_NEED_COHERENT for MPC52xx
BestComm, a DMA engine in MPC52xx SoC, requires snooping when CPU caches are enabled to work properly.
Adding CPU_FTR_NEED_COHERENT fixes NFS problems on MPC52xx machines introduced by 'powerpc/mm: Fix handling of _PAGE_COHERENT in BAT setup code'.
---
This patch was succesfully tested on our MPC5200-based.
Ben, Gerhard could you ack this patch, as it implements your suggestion.
arch/powerpc/include/asm/cputable.h | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-)
arch/powerpc/include/asm/cputable.h +3 -1
@@ -241,9 +241,11 @@ extern const char *powerpc_base_platform;/* We need to mark all pages as being coherent if we're SMP or we have a * 74[45]x and an MPC107 host bridge. Also 83xx and PowerQUICC II * require it for PCI "streaming/prefetch" to work properly. +* This is also required by 52xx family. */ #if defined(CONFIG_SMP ) || defined(CONFIG_MPC10X_BRIDGE ) \ - || defined(CONFIG_PPC_83xx) || defined(CONFIG_8260 ) + || defined(CONFIG_PPC_83xx) || defined(CONFIG_8260 ) \ + || defined(CONFIG_PPC_MPC52xx)#define CPU_FTR_COMMON CPU_FTR_NEED_COHERENT#else #define CPU_FTR_COMMON 0
Signature · 1 line
1.6.0.6
Re: [PATCH] powerpc: Enable CPU_FTR_NEED_COHERENT for MPC52xx
Quoted 5 lines · Piotr Ziecik
BestComm, a DMA engine in MPC52xx SoC, requires snooping when CPU caches are enabled to work properly.
Adding
CPU_FTR_NEED_COHERENTfixes NFS problems on MPC52xx machines introduced by 'powerpc/mm: Fix handling of_PAGE_COHERENTin BAT setup co=
de'.
arch/powerpc/include/a=
Quoted 11 lines
Signed-off-by Piotr Ziecik <[email protected]>---
This patch was succesfully tested on our MPC5200-based.
Ben, Gerhard could you ack this patch, as it implements your suggestion.=A0arch/powerpc/include/asm/cputable.h | =A0 =A04 +++- =A01 files changed, 3 insertions(+), 1 deletions(-)
arch/powerpc/include/a= renamed from arch/powerpc/include/asm/cputable.h +0
-0
sm/cputable.h
Quoted 5 lines
index b18358f..22dc2d8 100644
arch/powerpc/include/asm/cputable.h +0
-0@@ -241,9 +241,11 @@ extern const char *powerpc_base_platform;=A0/* We need to mark all pages as being coherent if we're SMP or we have=
a
Quoted 11 lines
=A0* 74[45]x and an MPC107 host bridge. Also 83xx and PowerQUICC II =A0* require it for PCI "streaming/prefetch" to work properly.
+ * This is also required by 52xx family.
=A0*/
=A0#ifdefined(CONFIG_SMP)||defined(CONFIG_MPC10X_BRIDGE)\- =A0 =A0 =A0 || defined(CONFIG_PPC_83xx) || defined(CONFIG_8260) + =A0 =A0 =A0 || defined(CONFIG_PPC_83xx) || defined(CONFIG_8260) \ + =A0 =A0 =A0 || defined(CONFIG_PPC_MPC52xx)#define
CPU_FTR_COMMONCPU_FTR_NEED_COHERENT
#else
#defineCPU_FTR_COMMON0
Aside from the fact that MPC10X_BRIDGE, PPC83xx and 8260 are already doing it, adding the feature bit this way isn't multiplatform friendly. Essentially it means that all selected platforms will have CPU_FTR_NEED_COHERENT enabled if CONFIG_PPC_MPC52xx is enabled.
Adding a new section to cputable.c for PVR=3D0x8082201x and applying the bit there would contain the impact to only G2 cores, but it's not perfect. Some of the 82xx parts have the same PVR and so would also be impacted.
Maybe I'm fretting over nothing. Ben, what's the right thing to do here?
Otherwise, I've pulled this into my tree and while I'm not affected by the bug, I don't see any new problems with the patch either.
g.
--=20
Grant Likely, B.Sc., P.Eng.
Secret Lab Technologies Ltd.
Re: [PATCH] powerpc: Enable CPU_FTR_NEED_COHERENT for MPC52xx
On Mon, Mar 16, 2009 at 9:54 PM, Grant Likely <[email protected]> w= rote:
Quoted 6 lines · Piotr Ziecik
Quoted 5 lines
BestComm, a DMA engine in MPC52xx SoC, requires snooping when CPU caches are enabled to work properly.
Adding
CPU_FTR_NEED_COHERENTfixes NFS problems on MPC52xx machines introduced by 'powerpc/mm: Fix handling of_PAGE_COHERENTin BAT setup c=
ode'.
Quoted 6 lines
Quoted 6 lines
[...]
=A0#ifdefined(CONFIG_SMP)||defined(CONFIG_MPC10X_BRIDGE)\- =A0 =A0 =A0 || defined(CONFIG_PPC_83xx) || defined(CONFIG_8260) + =A0 =A0 =A0 || defined(CONFIG_PPC_83xx) || defined(CONFIG_8260) \ + =A0 =A0 =A0 || defined(CONFIG_PPC_MPC52xx)=A0#define
CPU_FTR_COMMON=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0CPU_FTR_NEE=
D_COHERENT
Quoted 7 lines
Quoted 2 lines
=A0#else
=A0#defineCPU_FTR_COMMON=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A00Aside from the fact that
MPC10X_BRIDGE, PPC83xx and 8260 are already doing it, adding the feature bit this way isn't multiplatform friendly. =A0Essentially it means that all selected platforms will haveCPU_FTR_NEED_COHERENTenabled ifCONFIG_PPC_MPC52xxis enabled.
Here's my counter-patch. It contains the change to just G2_LE cores when MPC52xx is selected. However, this change will also affect some of the MPC82xx parts. However CPU_FTR_NEED_COHERENT shouldn't actually hurt anything, so maybe it would be better to just enable it unconditionally for the G2_LE core.
Kumar/Ben, thoughts?
g.
---
arch/powerpc/include/asm= renamed from arch/powerpc/include/asm/cputable.h +0 -0
/cput
index 4911104..48d7f5f 100644
arch/powerpc/include/asm/cputable.h +7 -0
@@ -348,8 +348,15 @@ extern const char *powerpc_base_platform; CPU_FTR_PPC_LE | CPU_FTR_NEED_PAIRED_STWCX)#define CPU_FTRS_82XX (CPU_FTR_COMMON | \ CPU_FTR_MAYBE_CAN_DOZE | CPU_FTR_USE_TB) + +#if defined(CONFIG_PPC_MPC52xx) +#define CPU_FTRS_G2_LE (CPU_FTR_COMMON | CPU_FTR_MAYBE_CAN_DOZE | \ + CPU_FTR_USE_TB | CPU_FTR_MAYBE_CAN_NAP | CPU_FTR_NEED_COHERENT) +#else #define CPU_FTRS_G2_LE (CPU_FTR_COMMON | CPU_FTR_MAYBE_CAN_DOZE | \ CPU_FTR_USE_TB | CPU_FTR_MAYBE_CAN_NAP) +#endif +#define CPU_FTRS_E300 (CPU_FTR_MAYBE_CAN_DOZE | \ CPU_FTR_USE_TB | CPU_FTR_MAYBE_CAN_NAP | \ CPU_FTR_COMMON)
--=20
Grant Likely, B.Sc., P.Eng.
Secret Lab Technologies Ltd.
Re: [PATCH] powerpc: Enable CPU_FTR_NEED_COHERENT for MPC52xx
Quoted 54 lines · Grant Likely
On Mon, Mar 16, 2009 at 9:54 PM, Grant Likely <[email protected]
Quoted 16 lines
wrote:
Review
Quoted 13 lines · Piotr Ziecik
BestComm, a DMA engine in MPC52xx SoC, requires snooping when CPU caches are enabled to work properly.
Adding
CPU_FTR_NEED_COHERENTfixes NFS problems on MPC52xx machines introduced by 'powerpc/mm: Fix handling of_PAGE_COHERENTin BAT setup code'. [...]
#ifdefined(CONFIG_SMP)||defined(CONFIG_MPC10X_BRIDGE)\(quoted diff) +2
-1- || defined(CONFIG_PPC_83xx) || defined(CONFIG_8260) + || defined(CONFIG_PPC_83xx) || defined(CONFIG_8260) \ + || defined(CONFIG_PPC_MPC52xx) #define CPU_FTR_COMMON CPU_FTR_NEED_COHERENT #else #define CPU_FTR_COMMON 0Quoted 4 lines
Aside from the fact that
MPC10X_BRIDGE, PPC83xx and 8260 are already doing it, adding the feature bit this way isn't multiplatform friendly. Essentially it means that all selected platforms will haveCPU_FTR_NEED_COHERENTenabled ifCONFIG_PPC_MPC52xxis enabled.Here's my counter-patch. It contains the change to just
G2_LEcores when MPC52xx is selected. However, this change will also affect some of the MPC82xx parts. HoweverCPU_FTR_NEED_COHERENTshouldn't actually hurt anything, so maybe it would be better to just enable it unconditionally for theG2_LEcore.Kumar/Ben, thoughts?
g.
---
arch/powerpc/include/asm/cputable.h +7
-0@@ -348,8 +348,15 @@ extern const char *powerpc_base_platform; CPU_FTR_PPC_LE | CPU_FTR_NEED_PAIRED_STWCX)#define CPU_FTRS_82XX (CPU_FTR_COMMON | \ CPU_FTR_MAYBE_CAN_DOZE | CPU_FTR_USE_TB) + +#if defined(CONFIG_PPC_MPC52xx) +#define CPU_FTRS_G2_LE (CPU_FTR_COMMON | CPU_FTR_MAYBE_CAN_DOZE | \ + CPU_FTR_USE_TB | CPU_FTR_MAYBE_CAN_NAP | CPU_FTR_NEED_COHERENT) +#else #define CPU_FTRS_G2_LE (CPU_FTR_COMMON | CPU_FTR_MAYBE_CAN_DOZE | \ CPU_FTR_USE_TB | CPU_FTR_MAYBE_CAN_NAP) +#endif +#define CPU_FTRS_E300 (CPU_FTR_MAYBE_CAN_DOZE | \ CPU_FTR_USE_TB | CPU_FTR_MAYBE_CAN_NAP | \
Quoted 1 line
CPU_FTR_COMMON)
Doing this via a static CPU FTR fixup isn't really the best way to handle it. I was thinking about this the other day in my patch to actually make G2/e300 cores respect _PAGE_COHERENT. We really should set this via a platform fixup. Just not sure if that's soon enough.
- k
Re: [PATCH] powerpc: Enable CPU_FTR_NEED_COHERENT for MPC52xx
On Tue, Mar 17, 2009 at 6:10 AM, Kumar Gala <[email protected]> wrot= e:
Quoted 18 lines · Grant Likely
Quoted 17 lines
arch/powerpc/include/asm/cputable.h +5
-0@@ -348,8 +348,15 @@ extern const char *powerpc_base_platform; =A0 =A0 =A0 =A0 =A0 CPU_FTR_PPC_LE | CPU_FTR_NEED_PAIRED_STWCX)#define CPU_FTRS_82XX =A0(CPU_FTR_COMMON | \ =A0 =A0 =A0 =A0 =A0 CPU_FTR_MAYBE_CAN_DOZE | CPU_FTR_USE_TB) + +#if defined(CONFIG_PPC_MPC52xx) +#define CPU_FTRS_G2_LE (CPU_FTR_COMMON | CPU_FTR_MAYBE_CAN_DOZE | \ + =A0 =A0 =A0 =A0 =A0 CPU_FTR_USE_TB | CPU_FTR_MAYBE_CAN_NAP | CPU_FTR_NEED_COHERENT) +#else #define CPU_FTRS_G2_LE (CPU_FTR_COMMON | CPU_FTR_MAYBE_CAN_DOZE | \ =A0 =A0 =A0 =A0 =A0 CPU_FTR_USE_TB | CPU_FTR_MAYBE_CAN_NAP)
Quoted 8 lines
Quoted 5 lines
arch/powerpc/include/asm/cputable.h +2
-0+#endif +#define CPU_FTRS_E300 =A0(CPU_FTR_MAYBE_CAN_DOZE | \ =A0 =A0 =A0 =A0 =A0 CPU_FTR_USE_TB | CPU_FTR_MAYBE_CAN_NAP | \=A0 =A0 =A0 =A0 =A0
CPU_FTR_COMMON)Doing this via a static CPU FTR fixup isn't really the best way to handle it. =A0I was thinking about this the other day in my patch to actually ma=
ke
Quoted 2 lines
G2/e300 cores respect
_PAGE_COHERENT. =A0We really should set this via a platform fixup. =A0Just not sure if that's soon enough.
I agree, but as Ben pointed out last night on IRC, feature fixup (early_init) occurs well before platform probe time. Platform code cannot fix it up until someone does the work of making platform probe time earlier. It's non-trivial.
However, I need to get a fix in for this ASAP, otherwise the 5200 will be broken in 2.6.29. I don't see this patch as a final solution, but it works as a stop-gap until platform probing can be reworked.
g.
--=20
Grant Likely, B.Sc., P.Eng.
Secret Lab Technologies Ltd.
Re: [PATCH] powerpc: Enable CPU_FTR_NEED_COHERENT for MPC52xx
Quoted 30 lines · Grant Likely
Quoted 20 lines · Kumar Gala
Quoted 19 lines · Grant Likely
arch/powerpc/include/asm/cputable.h +7
-0@@ -348,8 +348,15 @@ extern const char *powerpc_base_platform; CPU_FTR_PPC_LE | CPU_FTR_NEED_PAIRED_STWCX)#define CPU_FTRS_82XX (CPU_FTR_COMMON | \ CPU_FTR_MAYBE_CAN_DOZE | CPU_FTR_USE_TB) + +#if defined(CONFIG_PPC_MPC52xx) +#define CPU_FTRS_G2_LE (CPU_FTR_COMMON | CPU_FTR_MAYBE_CAN_DOZE | \ + CPU_FTR_USE_TB | CPU_FTR_MAYBE_CAN_NAP | CPU_FTR_NEED_COHERENT) +#else #define CPU_FTRS_G2_LE (CPU_FTR_COMMON | CPU_FTR_MAYBE_CAN_DOZE | \ CPU_FTR_USE_TB | CPU_FTR_MAYBE_CAN_NAP) +#endif +Quoted 8 lines
Quoted 3 lines
arch/powerpc/include/asm/cputable.h +0
-0#define CPU_FTRS_E300 (CPU_FTR_MAYBE_CAN_DOZE | \ CPU_FTR_USE_TB | CPU_FTR_MAYBE_CAN_NAP | \
CPU_FTR_COMMON)Doing this via a static CPU FTR fixup isn't really the best way to handle it. I was thinking about this the other day in my patch to actually make G2/e300 cores respect
_PAGE_COHERENT. We really should set this via a platform fixup. Just not sure if that's soon enough.
Quoted 8 lines
I agree, but as Ben pointed out last night on IRC, feature fixup (
early_init) occurs well before platform probe time. Platform code cannot fix it up until someone does the work of making platform probe time earlier. It's non-trivial.However, I need to get a fix in for this ASAP, otherwise the 5200 will be broken in 2.6.29. I don't see this patch as a final solution, but it works as a stop-gap until platform probing can be reworked.
So for .29 lets just go with the easy solution of adding CONFIG_PPC_MPC52xx to the list that set CPU_FTR_NEED_COHERENT (10x, 83xx, etc..).
- k