ucc_geth: Fix hangs after switching from full to half duplex
Subsystems (2)
PATCH 3/3 ucc_geth: Fix hangs after switching from full to half duplex
MPC8360 QE UCC ethernet controllers hang when changing link duplex under a load (a bit of NFS activity is enough).
PHY: mdio@e0102120:00 - Link is Up - 1000/Full sh-3.00# ethtool -s eth0 speed 100 duplex half autoneg off
PHY: mdio@e0102120:00 - Link is Down
PHY: mdio@e0102120:00 - Link is Up - 100/Half NETDEV WATCHDOG: eth0 (ucc_geth): transmit queue 0 timed out
------------[ cut here ]------------ Badness at c01fcbd0 [verbose debug info unavailable] NIP: c01fcbd0 LR: c01fcbd0 CTR: c0194e44 ...
The cure is to disable the controller before changing speed/duplex and enable it afterwards.
--- drivers/net/ucc_geth.c | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-)
drivers/net/ucc_geth.c +4 -0
@@ -1631,9 +1631,13 @@ static void adjust_link(struct net_device *dev) ugeth->oldspeed = phydev->speed; } +ugeth_disable (ugeth,COMM_DIR_RX_AND_TX ); +out_be32 (&ug_regs->maccfg2, tempval);out_be32 (&uf_regs->upsmr, upsmr); +ugeth_enable (ugeth,COMM_DIR_RX_AND_TX ); +if (!ugeth->oldlink) { new_state =1 ; ugeth->oldlink =1 ;
Signature · 1 line
1.6.3.3
PATCH v2 3/3 ucc_geth: Fix hangs after switching from full to half duplex
MPC8360 QE UCC ethernet controllers hang when changing link duplex under a load (a bit of NFS activity is enough).
PHY: mdio@e0102120:00 - Link is Up - 1000/Full sh-3.00# ethtool -s eth0 speed 100 duplex half autoneg off
PHY: mdio@e0102120:00 - Link is Down
PHY: mdio@e0102120:00 - Link is Up - 100/Half NETDEV WATCHDOG: eth0 (ucc_geth): transmit queue 0 timed out
------------[ cut here ]------------ Badness at c01fcbd0 [verbose debug info unavailable] NIP: c01fcbd0 LR: c01fcbd0 CTR: c0194e44 ...
The cure is to disable the controller before changing speed/duplex and enable it afterwards.
Since ugeth_graceful_stop_{tx,rx} now may be called from an atomic context, switch the two functions from msleep() to mdelay().
---
v2: Switch ugeth_graceful_stop_{tx,rx} to mdelay. Noticed with
CONFIG_DEBUG_SPINLOCK_SLEEP.
drivers/net/ucc_geth.c | 8 ++++++-- 1 files changed, 6 insertions(+), 2 deletions(-)
drivers/net/ucc_geth.c +6 -2
@@ -1432,7 +1432,7 @@ static int ugeth_graceful_stop_tx(struct ucc_geth_private *ugeth)/* Wait for command to complete */ do { -msleep (10 ); +mdelay (10 ); temp =in_be32 (uccf->p_ucce); }while (!(temp &UCC_GETH_UCCE_GRA ) && --i);
@@ -1464,7 +1464,7 @@ static int ugeth_graceful_stop_rx(struct ucc_geth_private *ugeth) ucc_num);qe_issue_cmd (QE_GRACEFUL_STOP_RX , cecr_subblock,QE_CR_PROTOCOL_ETHERNET ,0 ); -msleep (10 ); +mdelay (10 ); temp =in_8 (&ugeth->p_rx_glbl_pram->rxgstpack); }while (!(temp &GRACEFUL_STOP_ACKNOWLEDGE_RX ) && --i);
@@ -1631,9 +1631,13 @@ static void adjust_link(struct net_device *dev) ugeth->oldspeed = phydev->speed; } +ugeth_disable (ugeth,COMM_DIR_RX_AND_TX ); +out_be32 (&ug_regs->maccfg2, tempval);out_be32 (&uf_regs->upsmr, upsmr); +ugeth_enable (ugeth,COMM_DIR_RX_AND_TX ); +if (!ugeth->oldlink) { new_state =1 ; ugeth->oldlink =1 ;
Signature · 1 line
1.6.3.3
Quoted 13 lines · Anton Vorontsov
MPC8360 QE UCC ethernet controllers hang when changing link duplex under a load (a bit of NFS activity is enough).
PHY: mdio@e0102120:00 - Link is Up - 1000/Full sh-3.00# ethtool -s eth0 speed 100 duplex half autoneg off
PHY: mdio@e0102120:00 - Link is Down
PHY: mdio@e0102120:00 - Link is Up - 100/Half NETDEV WATCHDOG: eth0 (ucc_geth): transmit queue 0 timed out------------[ cut here ]------------ Badness at c01fcbd0 [verbose debug info unavailable] NIP: c01fcbd0 LR: c01fcbd0 CTR: c0194e44 ...
Quoted 5 lines
The cure is to disable the controller before changing speed/duplex and enable it afterwards.
Since ugeth_graceful_stop_{tx,rx} now may be called from an atomic context, switch the two functions from
msleep()tomdelay().
Ouch. Can we put this in a workqueue or something?
-Scott
Quoted 14 lines · Scott Wood
Quoted 13 lines · Anton Vorontsov
MPC8360 QE UCC ethernet controllers hang when changing link duplex under a load (a bit of NFS activity is enough).
PHY: mdio@e0102120:00 - Link is Up - 1000/Full sh-3.00# ethtool -s eth0 speed 100 duplex half autoneg off
PHY: mdio@e0102120:00 - Link is Down
PHY: mdio@e0102120:00 - Link is Up - 100/Half NETDEV WATCHDOG: eth0 (ucc_geth): transmit queue 0 timed out------------[ cut here ]------------ Badness at c01fcbd0 [verbose debug info unavailable] NIP: c01fcbd0 LR: c01fcbd0 CTR: c0194e44 ...
Quoted 7 lines
Quoted 5 lines
The cure is to disable the controller before changing speed/duplex and enable it afterwards.
Since ugeth_graceful_stop_{tx,rx} now may be called from an atomic context, switch the two functions from
msleep()tomdelay().Ouch.
Yeah, right... delaying for 10ms with irqs off isn't good.
Quoted 1 line
Can we put this in a workqueue or something?
adjust_link() itself isn't called from an atomic context.
It's we are grabbing ugeth->lock, i.e. a spinlock. I don't see why the lock is needed in adjust_link() in its current form, but if we're going to disable the controller for some time, we'll have to make sure that no start_xmit() or NAPI is running, scheduled or will be scheduled until we say so.
I think that lock-less, and thus completely sleep-able variant of adjust_link is doable.
Thanks,
Signature · 3 lines
Anton Vorontsov email: [email protected] irc://irc.freenode.net/bd2
Quoted 21 lines · Anton Vorontsov
Quoted 20 lines · Scott Wood
Quoted 18 lines · Anton Vorontsov
MPC8360 QE UCC ethernet controllers hang when changing link duplex under a load (a bit of NFS activity is enough).
PHY: mdio@e0102120:00 - Link is Up - 1000/Full sh-3.00# ethtool -s eth0 speed 100 duplex half autoneg off
PHY: mdio@e0102120:00 - Link is Down
PHY: mdio@e0102120:00 - Link is Up - 100/Half NETDEV WATCHDOG: eth0 (ucc_geth): transmit queue 0 timed out------------[ cut here ]------------ Badness at c01fcbd0 [verbose debug info unavailable] NIP: c01fcbd0 LR: c01fcbd0 CTR: c0194e44 ...The cure is to disable the controller before changing speed/duplex and enable it afterwards.
Since ugeth_graceful_stop_{tx,rx} now may be called from an atomic context, switch the two functions from
msleep()tomdelay().
Quoted 7 lines
Quoted 1 line
Ouch.
Yeah, right... delaying for 10ms with irqs off isn't good.
Quoted 1 line
Can we put this in a workqueue or something?
adjust_link()itself isn't called from an atomic context.
Oops. I though that phylib calls us from a workqueue, not a timer. Hm...
Will be a little bit more work..
Signature · 3 lines
Anton Vorontsov email: [email protected] irc://irc.freenode.net/bd2
PATCH v3 3/3 ucc_geth: Fix hangs after switching from full to half duplex
MPC8360 QE UCC ethernet controllers hang when changing link duplex under a load (a bit of NFS activity is enough).
PHY: mdio@e0102120:00 - Link is Up - 1000/Full sh-3.00# ethtool -s eth0 speed 100 duplex half autoneg off
PHY: mdio@e0102120:00 - Link is Down
PHY: mdio@e0102120:00 - Link is Up - 100/Half NETDEV WATCHDOG: eth0 (ucc_geth): transmit queue 0 timed out
------------[ cut here ]------------ Badness at c01fcbd0 [verbose debug info unavailable] NIP: c01fcbd0 LR: c01fcbd0 CTR: c0194e44 ...
The cure is to disable the controller before changing speed/duplex and enable it afterwards.
Though, disabling the controller might take quite a while, so we better not grab any spinlocks in adjust_link(). Instead, we quiesce the driver's activity, and only then disable the controller.
---
Quoted 26 lines · Anton Vorontsov
Quoted 25 lines · Anton Vorontsov
Quoted 14 lines · Scott Wood
Quoted 13 lines · Anton Vorontsov
MPC8360 QE UCC ethernet controllers hang when changing link duplex under a load (a bit of NFS activity is enough).
PHY: mdio@e0102120:00 - Link is Up - 1000/Full sh-3.00# ethtool -s eth0 speed 100 duplex half autoneg off
PHY: mdio@e0102120:00 - Link is Down
PHY: mdio@e0102120:00 - Link is Up - 100/Half NETDEV WATCHDOG: eth0 (ucc_geth): transmit queue 0 timed out------------[ cut here ]------------ Badness at c01fcbd0 [verbose debug info unavailable] NIP: c01fcbd0 LR: c01fcbd0 CTR: c0194e44 ...Quoted 7 lines
Quoted 5 lines
The cure is to disable the controller before changing speed/duplex and enable it afterwards.
Since ugeth_graceful_stop_{tx,rx} now may be called from an atomic context, switch the two functions from
msleep()tomdelay().Ouch.
Yeah, right... delaying for 10ms with irqs off isn't good.
Quoted 7 lines
Quoted 3 lines
Quoted 1 line
Can we put this in a workqueue or something?
adjust_link()itself isn't called from an atomic context.Oops. I though that phylib calls us from a workqueue, not a timer. Hm...
Will be a little bit more work..
Ignore me. I'm working on two kernel versions in parallel (2.6.21 and mainline), and it's 2.6.21 where phylib uses a timer. Mainline is OK.
How about this patch?
drivers/net/ucc_geth.c | 36 +++++++++++++++++++++++++++++++----- 1 files changed, 31 insertions(+), 5 deletions(-)
drivers/net/ucc_geth.c +31 -5
@@ -1560,6 +1560,25 @@ static int ugeth_disable(struct ucc_geth_private *ugeth, enum comm_dir mode)return 0 ; } +static void ugeth_quiesce (struct ucc_geth_private *ugeth) +{ +/* Wait for and prevent any further xmits. */ +netif_tx_disable (ugeth->ndev); + +/* Disable the interrupt to avoid NAPI rescheduling. */ +disable_irq (ugeth->ug_info->uf_info.irq); + +/* Stop NAPI, and possibly wait for its completion. */ +napi_disable (&ugeth->napi); +} + +static void ugeth_activate (struct ucc_geth_private *ugeth) +{ +napi_enable (&ugeth->napi); +enable_irq (ugeth->ug_info->uf_info.irq); +netif_tx_wake_all_queues (ugeth->ndev); +} +/* Called every time the controller might need to be made * aware of new link state. The PHY code conveys this * information through variables in the ugeth structure, and this
@@ -1573,14 +1592,11 @@ static void adjust_link(struct net_device *dev)struct ucc_geth __iomem *ug_regs;struct ucc_fast __iomem *uf_regs;struct phy_device *phydev = ugeth->phydev; -unsigned long flags;int new_state =0 ; ug_regs = ugeth->ug_regs; uf_regs = ugeth->uccf->uf_regs; -spin_lock_irqsave (&ugeth->lock, flags); -if (phydev->link) {u32 tempval =in_be32 (&ug_regs->maccfg2);u32 upsmr =in_be32 (&uf_regs->upsmr);
@@ -1631,9 +1647,21 @@ static void adjust_link(struct net_device *dev) ugeth->oldspeed = phydev->speed; } +/* +* To change the MAC configuration we need to disable the +* controller. To do so, we have to either grab ugeth->lock, +* which is a bad idea since 'graceful stop' commands might +* take quite a while, or we can quiesce driver's activity. +*/ +ugeth_quiesce (ugeth); +ugeth_disable (ugeth,COMM_DIR_RX_AND_TX ); +out_be32 (&ug_regs->maccfg2, tempval);out_be32 (&uf_regs->upsmr, upsmr); +ugeth_enable (ugeth,COMM_DIR_RX_AND_TX ); +ugeth_activate (ugeth); +if (!ugeth->oldlink) { new_state =1 ; ugeth->oldlink =1 ;
@@ -1647,8 +1675,6 @@ static void adjust_link(struct net_device *dev)if (new_state &&netif_msg_link (ugeth))phy_print_status (phydev); - -spin_unlock_irqrestore (&ugeth->lock, flags); }/* Initialize TBI PHY interface for communicating with the
Signature · 1 line
1.6.3.3
From: Anton Vorontsov <[email protected]>
Date: Fri, 11 Sep 2009 01:48:12 +0400
Quoted 13 lines
MPC8360 QE UCC ethernet controllers hang when changing link duplex under a load (a bit of NFS activity is enough).
PHY: mdio@e0102120:00 - Link is Up - 1000/Full sh-3.00# ethtool -s eth0 speed 100 duplex half autoneg off
PHY: mdio@e0102120:00 - Link is Down
PHY: mdio@e0102120:00 - Link is Up - 100/Half NETDEV WATCHDOG: eth0 (ucc_geth): transmit queue 0 timed out------------[ cut here ]------------ Badness at c01fcbd0 [verbose debug info unavailable] NIP: c01fcbd0 LR: c01fcbd0 CTR: c0194e44 ...
Quoted 8 lines
The cure is to disable the controller before changing speed/duplex and enable it afterwards.
Though, disabling the controller might take quite a while, so we better not grab any spinlocks in
adjust_link(). Instead, we quiesce the driver's activity, and only then disable the controller.Signed-off-by Anton Vorontsov <[email protected]>
Applied.