net: ks8851: Fix deadlock with the SPI chip variant
Subsystems (3)
PATCH v2 1/1 net: ks8851: Fix deadlock with the SPI chip variant
From: Ronald Wahl <[email protected]>
When SMP is enabled and spinlocks are actually functional then there is a deadlock with the 'statelock' spinlock between ks8851_start_xmit_spi and ks8851_irq:
watchdog: BUG: soft lockup - CPU#0 stuck for 27s!
call trace:
queued_spin_lock_slowpath+0x100/0x284
do_raw_spin_lock+0x34/0x44
ks8851_start_xmit_spi+0x30/0xb8
ks8851_start_xmit+0x14/0x20
netdev_start_xmit+0x40/0x6c
dev_hard_start_xmit+0x6c/0xbc
sch_direct_xmit+0xa4/0x22c
__qdisc_run+0x138/0x3fc
qdisc_run+0x24/0x3c
net_tx_action+0xf8/0x130
handle_softirqs+0x1ac/0x1f0
__do_softirq+0x14/0x20
____do_softirq+0x10/0x1c
call_on_irq_stack+0x3c/0x58
do_softirq_own_stack+0x1c/0x28
__irq_exit_rcu+0x54/0x9c
irq_exit_rcu+0x10/0x1c
el1_interrupt+0x38/0x50
el1h_64_irq_handler+0x18/0x24
el1h_64_irq+0x64/0x68
__netif_schedule+0x6c/0x80
netif_tx_wake_queue+0x38/0x48
ks8851_irq+0xb8/0x2c8
irq_thread_fn+0x2c/0x74
irq_thread+0x10c/0x1b0
kthread+0xc8/0xd8
ret_from_fork+0x10/0x20This issue has not been identified earlier because tests were done on a device with SMP disabled and so spinlocks were actually NOPs.
Now use spin_(un)lock_bh for TX queue related locking to avoid execution of softirq work synchronously that would lead to a deadlock.
---
V2: - use spin_lock_bh instead of moving netif_wake_queue outside of
locked region (doing the same in the start_xmit function)
- add missing net: tag
drivers/net/ethernet/micrel/ks8851_common.c | 4 ++-- drivers/net/ethernet/micrel/ks8851_spi.c | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-)
drivers/net/ethernet/micrel/ks8851_common.c +2 -2
@@ -352,11 +352,11 @@ static irqreturn_t ks8851_irq(int irq, void *_ks)netif_dbg (ks, intr, ks->netdev,"%s: txspace %d\n" , __func__, tx_space); -spin_lock (&ks->statelock); +spin_lock_bh (&ks->statelock); ks->tx_space = tx_space;if (netif_queue_stopped (ks->netdev))netif_wake_queue (ks->netdev); -spin_unlock (&ks->statelock); +spin_unlock_bh (&ks->statelock); }if (status &IRQ_SPIBEI ) {
drivers/net/ethernet/micrel/ks8851_spi.c +2 -2
@@ -385,7 +385,7 @@ static netdev_tx_t ks8851_start_xmit_spi(struct sk_buff *skb,netif_dbg (ks, tx_queued, ks->netdev,"%s: skb %p, %d@%p\n" , __func__, skb, skb->len, skb->data); -spin_lock (&ks->statelock); +spin_lock_bh (&ks->statelock);if (ks->queued_len + needed > ks->tx_space) {netif_stop_queue (dev);
@@ -395,7 +395,7 @@ static netdev_tx_t ks8851_start_xmit_spi(struct sk_buff *skb,skb_queue_tail (&ks->txq, skb); } -spin_unlock (&ks->statelock); +spin_unlock_bh (&ks->statelock);if (ret ==NETDEV_TX_OK )schedule_work (&kss->tx_work);
--
2.45.2
Jakub Kicinski
maintainerQuoted 12 lines · On Thu, 4 Jul 2024 19:47:56 +0200 Ronald Wahl wrote:
drivers/net/ethernet/micrel/ks8851_spi.c +1
-1@@ -385,7 +385,7 @@ static netdev_tx_t ks8851_start_xmit_spi(struct sk_buff *skb,netif_dbg (ks, tx_queued, ks->netdev,"%s: skb %p, %d@%p\n" , __func__, skb, skb->len, skb->data); -spin_lock (&ks->statelock); +spin_lock_bh (&ks->statelock);if (ks->queued_len + needed > ks->tx_space) {netif_stop_queue (dev);
drivers/net/ethernet/micrel/ks8851_spi.c static netdev_tx_t ks8851_start_xmit_spi(struct sk_buff *skb, · line 398
Quoted 5 lines
drivers/net/ethernet/micrel/ks8851_spi.c +1
-1@@ -395,7 +395,7 @@ static netdev_tx_t ks8851_start_xmit_spi(struct sk_buff *skb,skb_queue_tail (&ks->txq, skb); } -spin_unlock (&ks->statelock); +spin_unlock_bh (&ks->statelock);
this one probably can stay as spin_lock() since networking stack only calls xmit in BH context. But I see 2 other spin_lock(statelock) in the driver which I'm not as sure about. Any taking of this lock has to be _bh() unless you're sure the caller is already in BH.
Quoted 16 lines · Jakub Kicinski
Quoted 15 lines · On Thu, 4 Jul 2024 19:47:56 +0200 Ronald Wahl wrote:
drivers/net/ethernet/micrel/ks8851_spi.c +1
-1@@ -385,7 +385,7 @@ static netdev_tx_t ks8851_start_xmit_spi(struct sk_buff *skb,netif_dbg (ks, tx_queued, ks->netdev,"%s: skb %p, %d@%p\n" , __func__, skb, skb->len, skb->data); -spin_lock (&ks->statelock); +spin_lock_bh (&ks->statelock);if (ks->queued_len + needed > ks->tx_space) {netif_stop_queue (dev);@@ -395,7 +395,7 @@ static netdev_tx_t ks8851_start_xmit_spi(struct sk_buff *skb,skb_queue_tail (&ks->txq, skb); }
Quoted 5 lines
drivers/net/ethernet/micrel/ks8851_spi.c static netdev_tx_t ks8851_start_xmit_spi(struct sk_buff *skb, · line 398
Quoted 2 lines
drivers/net/ethernet/micrel/ks8851_spi.c +1
-1-spin_unlock (&ks->statelock); +spin_unlock_bh (&ks->statelock);this one probably can stay as
spin_lock()since networking stack only calls xmit in BH context.
I already suspected this it was more a mental hint here. I will remove it.
Quoted 3 lines
But I see 2 other
spin_lock(statelock)in the
driver which I'm not as sure about. Any taking of this lock has to be_bh()unless you're sure the caller is already in BH.
The other two instances are not in BH context as far as I know but also do not interfere with BH. The one in ks8861_tx_work protects only variable assignments used only inside the driver and the one in ks8851_set_rx_mode also only does some driver local variable stuff and a schedule_work which as far as I know has nothing to do with BH because workqueues are running in process context. Am I wrong here?
- ron
________________________________
Ce message, ainsi que tous les fichiers joints à ce message, peuvent contenir des informations sensibles et/ ou confidentielles ne devant pas être divulguées. Si vous n'êtes pas le destinataire de ce message (ou que vous recevez ce message par erreur), nous vous remercions de le notifier immédiatement à son expéditeur, et de détruire ce message. Toute copie, divulgation, modification, utilisation ou diffusion, non autorisée, directe ou indirecte, de tout ou partie de ce message, est strictement interdite.
This e-mail, and any document attached hereby, may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized, direct or indirect, copying, disclosure, distribution or other use of the material or parts thereof is strictly forbidden.
Quoted 31 lines · Ronald Wahl
Quoted 18 lines · Jakub Kicinski
Quoted 17 lines · On Thu, 4 Jul 2024 19:47:56 +0200 Ronald Wahl wrote:
drivers/net/ethernet/micrel/ks8851_spi.c +1
-1@@ -385,7 +385,7 @@ static netdev_tx_t ks8851_start_xmit_spi(struct sk_buff *skb,netif_dbg (ks, tx_queued, ks->netdev,"%s: skb %p, %d@%p\n" , __func__, skb, skb->len, skb->data); -spin_lock (&ks->statelock); +spin_lock_bh (&ks->statelock);if (ks->queued_len + needed > ks->tx_space) {netif_stop_queue(dev);drivers/net/ethernet/micrel/ks8851_spi.c +0
-0@@ -395,7 +395,7 @@ static netdev_tx_t ks8851_start_xmit_spi(struct sk_buff *skb,skb_queue_tail (&ks->txq, skb); }Quoted 5 lines
drivers/net/ethernet/micrel/ks8851_spi.c static netdev_tx_t ks8851_start_xmit_spi(struct · line 399
Quoted 2 lines
drivers/net/ethernet/micrel/ks8851_spi.c +1
-1-spin_unlock (&ks->statelock); +spin_unlock_bh (&ks->statelock);this one probably can stay as
spin_lock()since networking stack only calls xmit in BH context.I already suspected this it was more a mental hint here. I will remove it.
Quoted 3 lines
But I see 2 other
spin_lock(statelock)in the
driver which I'm not as sure about. Any taking of this lock has to be_bh()unless you're sure the caller is already in BH.
Quoted 6 lines
The other two instances are not in BH context as far as I know but also do not interfere with BH. The one in
ks8861_tx_workprotects only variable assignments used only inside the driver and the one inks8851_set_rx_modealso only does some driver local variable stuff and aschedule_workwhich as far as I know has nothing to do with BH because workqueues are running in process context. Am I wrong here?
I guess I found a misunderstanding on my side: I was assuming that a softirq cannot asynchronously interrupt a spin lock protected section. Maybe this is wrong. In the one place where I'm waking the queue again the spin_lock_bh avoids synchronously triggering the BH processing while still holding a spinlock.
I will use the _bh variants on the two other places.
- ron
________________________________
Ce message, ainsi que tous les fichiers joints à ce message, peuvent contenir des informations sensibles et/ ou confidentielles ne devant pas être divulguées. Si vous n'êtes pas le destinataire de ce message (ou que vous recevez ce message par erreur), nous vous remercions de le notifier immédiatement à son expéditeur, et de détruire ce message. Toute copie, divulgation, modification, utilisation ou diffusion, non autorisée, directe ou indirecte, de tout ou partie de ce message, est strictement interdite.
This e-mail, and any document attached hereby, may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized, direct or indirect, copying, disclosure, distribution or other use of the material or parts thereof is strictly forbidden.
Greg KH
maintainerQuoted 2 lines · Ronald Wahl
This e-mail, and any document attached hereby, may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized, direct or indirect, copying, disclosure, distribution or other use of the material or parts thereof is strictly forbidden.
Now deleted