net: ks8851: Reinstate disabling of BHs around IRQ handler
Subsystems (3)
PATCH v2 1/1 net: ks8851: Reinstate disabling of BHs around IRQ handler
If CONFIG_PREEMPT_RT=y is set AND the driver executes ks8851_irq() AND KSZ_ISR register bit IRQ_RXI is set AND ks8851_rx_pkts() detects that there are packets in the RX FIFO, then netdev_alloc_skb_ip_align() is called to allocate SKBs. If netdev_alloc_skb_ip_align() is called with BH enabled, local_bh_enable() at the end of netdev_alloc_skb_ip_align() will call __local_bh_enable_ip(), which will call __do_softirq(), which may trigger net_tx_action() softirq, which may ultimately call the xmit callback ks8851_start_xmit_par(). The ks8851_start_xmit_par() will try to lock struct ks8851_net_par .lock spinlock, which is already locked by ks8851_irq() from which ks8851_start_xmit_par() was called. This leads to a deadlock, which is reported by the kernel, including a trace listed below.
Fix the problem by disabling BH around the IRQ handler, thus preventing the net_tx_action() softirq from triggering during the IRQ handler. The net_tx_action() softirq is now triggered at the end of the IRQ handler, once all the other IRQ handler actions have been completed.
__schedule from schedule_rtlock+0x1c/0x34 schedule_rtlock from rtlock_slowlock_locked+0x538/0x894 rtlock_slowlock_locked from rt_spin_lock+0x44/0x5c rt_spin_lock from ks8851_start_xmit_par+0x68/0x1a0 ks8851_start_xmit_par from netdev_start_xmit+0x1c/0x40 netdev_start_xmit from dev_hard_start_xmit+0xec/0x1b0 dev_hard_start_xmit from sch_direct_xmit+0xb8/0x25c sch_direct_xmit from __qdisc_run+0x20c/0x4fc __qdisc_run from qdisc_run+0x1c/0x28
qdisc_run from net_tx_action+0x1f4/0x244 net_tx_action from handle_softirqs+0x1c0/0x29c handle_softirqs from __local_bh_enable_ip+0xdc/0xf4 __local_bh_enable_ip from __netdev_alloc_skb+0x140/0x194 __netdev_alloc_skb from ks8851_irq+0x348/0x4d8 ks8851_irq from irq_thread_fn+0x24/0x64 irq_thread_fn from irq_thread+0x110/0x1dc irq_thread from kthread+0x104/0x10c
kthread from ret_from_fork+0x14/0x28
---
Cc: "David S. Miller" <[email protected]>
Cc: Andrew Lunn <[email protected]>
Cc: Eric Dumazet <[email protected]>
Cc: Jakub Kicinski <[email protected]>
Cc: Nicolai Buchwitz <[email protected]>
Cc: Paolo Abeni <[email protected]>
Cc: Ronald Wahl <[email protected]>
Cc: Yicong Hui <[email protected]>
Cc: [email protected]
Cc: [email protected]
---
V2: Register dedicated IRQ handler wrapper which disables BH for the
parallel variant of the MAC, the variant which uses spinlocks as the locking primitive. Use stock IRQ handler with BH unchanged for the SPI variant, which uses mutexes as locking primitive.
--- drivers/net/ethernet/micrel/ks8851.h | 2 ++ drivers/net/ethernet/micrel/ks8851_common.c | 22 ++++++++++++++++++++- drivers/net/ethernet/micrel/ks8851_par.c | 1 + drivers/net/ethernet/micrel/ks8851_spi.c | 1 + 4 files changed, 25 insertions(+), 1 deletion(-)
drivers/net/ethernet/micrel/ks8851.h +2 -0
@@ -394,6 +394,8 @@ struct ks8851_net {u16 rc_rxqcr;u16 rc_ccr; +bool no_bh_in_irq_handler; +struct mii_if_info mii;struct ks8851_rxctrl rxctrl;
drivers/net/ethernet/micrel/ks8851_common.c +21 -1
@@ -385,6 +385,24 @@ static irqreturn_t ks8851_irq(int irq, void *_ks)return IRQ_HANDLED ; } +/** +* ks8851_irq_nobh - IRQ handler with BH disabled +* @irq: IRQ number +* @_ks: cookie +* +* Wrapper which calls ks8851_irq() with BH disabled. +*/ +static irqreturn_t ks8851_irq_nobh (int irq,void *_ks) +{ +irqreturn_t ret; + +local_bh_disable (); + ret =ks8851_irq (irq, _ks); +local_bh_enable (); + +return ret; +} +/** * ks8851_flush_tx_work - flush outstanding TX work * @ks: The device state
@@ -408,7 +426,9 @@ static int ks8851_net_open(struct net_device *dev)unsigned long flags;int ret; - ret =request_threaded_irq (dev->irq,NULL , ks8851_irq, + ret =request_threaded_irq (dev->irq,NULL , + ks->no_bh_in_irq_handler ? + ks8851_irq_nobh : ks8851_irq,IRQF_TRIGGER_LOW |IRQF_ONESHOT , dev->name, ks);if (ret <0 ) {
drivers/net/ethernet/micrel/ks8851_par.c +1 -0
@@ -288,6 +288,7 @@ static int ks8851_probe_par(struct platform_device *pdev) ks->rdfifo = ks8851_rdfifo_par; ks->wrfifo = ks8851_wrfifo_par; ks->start_xmit = ks8851_start_xmit_par; + ks->no_bh_in_irq_handler = true;#define STD_IRQ (IRQ_LCI |/* Link Change */ \ IRQ_RXI | /* RX done */ \
drivers/net/ethernet/micrel/ks8851_spi.c +1 -0
@@ -424,6 +424,7 @@ static int ks8851_probe_spi(struct spi_device *spi) ks->wrfifo = ks8851_wrfifo_spi; ks->start_xmit = ks8851_start_xmit_spi; ks->flush_tx_work = ks8851_flush_tx_work_spi; + ks->no_bh_in_irq_handler = false;#define STD_IRQ (IRQ_LCI |/* Link Change */ \ IRQ_TXI | /* TX done */ \
Signature · 1 line
2.53.0
Hi Marek
Quoted 28 lines · Marek Vasut
[...]
drivers/net/ethernet/micrel/ks8851_common.c +19
-0+bool no_bh_in_irq_handler; +/** +* ks8851_irq_nobh - IRQ handler with BH disabled +* @irq: IRQ number +* @_ks: cookie +* +* Wrapper which calls ks8851_irq() with BH disabled. +*/ +static irqreturn_t ks8851_irq_nobh (int irq,void *_ks) +{ +irqreturn_t ret; + +local_bh_disable (); + ret =ks8851_irq (irq, _ks); +local_bh_enable (); + +return ret; +} +/** * ks8851_flush_tx_work - flush outstanding TX work * @ks: The device state
drivers/net/ethernet/micrel/ks8851_common.c static int ks8851_net_open(struct net_device *dev) · line 431
Quoted 7 lines
drivers/net/ethernet/micrel/ks8851_common.c +3
-1@@ -408,7 +426,9 @@ static int ks8851_net_open(struct net_device *dev)unsigned long flags;int ret; - ret = request_threaded_irq(dev->irq, NULL, ks8851_irq, + ret = request_threaded_irq(dev->irq, NULL, + ks->no_bh_in_irq_handler ? + ks8851_irq_nobh : ks8851_irq,
This works, but wouldn't it be simpler to put the BH disable into the PAR lock/unlock directly?
static void ks8851_lock_par(...)
{
local_bh_disable();
spin_lock_irqsave(&ksp->lock, *flags);
}
static void ks8851_unlock_par(...)
{
spin_unlock_irqrestore(&ksp->lock, *flags);
local_bh_enable();
}No flag, no wrapper, no conditional in request_threaded_irq. And it protects all PAR lock/unlock callsites, not just the IRQ handler.
Quoted 1 line
[...]
Regards
Nicolai
On 4/9/26 8:52 AM, Nicolai Buchwitz wrote:
Hello Nicolai,
Quoted 24 lines
drivers/net/ethernet/micrel/ks8851_common.c static int ks8851_net_open(struct net_device *dev) · line 431
Quoted 8 lines
drivers/net/ethernet/micrel/ks8851_common.c +3
-1@@ -408,7 +426,9 @@ static int ks8851_net_open(struct net_device *dev) unsigned long flags; int ret; - ret = request_threaded_irq(dev->irq, NULL, ks8851_irq, + ret = request_threaded_irq(dev->irq, NULL, + ks->no_bh_in_irq_handler ? + ks8851_irq_nobh : ks8851_irq,This works, but wouldn't it be simpler to put the BH disable into the PAR lock/unlock directly?
static void
ks8851_lock_par(...)
{
local_bh_disable();
spin_lock_irqsave(&ksp->lock, *flags);
}static void
ks8851_unlock_par(...)
{
spin_unlock_irqrestore(&ksp->lock, *flags);local_bh_enable();
}
Quoted 3 lines
No flag, no wrapper, no conditional in
request_threaded_irq. And it protects all PAR lock/unlock callsites, not just the IRQ handler.
That is exactly why I wrapped the IRQ handler, because the BH should be disabled ONLY around the IRQ handler, not around the other call sites.
Nicolai Buchwitz
Reviewed-byQuoted 29 lines · Marek Vasut
On 4/9/26 8:52 AM, Nicolai Buchwitz wrote:
Hello Nicolai,
Quoted 25 lines
drivers/net/ethernet/micrel/ks8851_common.c static int ks8851_net_open(struct net_device · line 432
Quoted 9 lines
drivers/net/ethernet/micrel/ks8851_common.c +3
-1@@ -408,7 +426,9 @@ static int ks8851_net_open(struct net_device *dev) unsigned long flags; int ret; - ret = request_threaded_irq(dev->irq, NULL, ks8851_irq, + ret = request_threaded_irq(dev->irq, NULL, + ks->no_bh_in_irq_handler ? + ks8851_irq_nobh : ks8851_irq,This works, but wouldn't it be simpler to put the BH disable into the PAR lock/unlock directly?
static void
ks8851_lock_par(...)
{
local_bh_disable();
spin_lock_irqsave(&ksp->lock, *flags);
}static void
ks8851_unlock_par(...)
{
spin_unlock_irqrestore(&ksp->lock, *flags);local_bh_enable();
}
Quoted 4 lines
Quoted 3 lines
No flag, no wrapper, no conditional in
request_threaded_irq. And it protects all PAR lock/unlock callsites, not just the IRQ handler.That is exactly why I wrapped the IRQ handler, because the BH should be disabled ONLY around the IRQ handler, not around the other call sites.
Jakub Kicinski
maintainerQuoted 4 lines · On Wed, 8 Apr 2026 18:24:58 +0200 Marek Vasut wrote:
If
CONFIG_PREEMPT_RT=yis set AND the driver executesks8851_irq()ANDKSZ_ISRregister bitIRQ_RXIis set ANDks8851_rx_pkts()detects that there are packets in the RX FIFO, thennetdev_alloc_skb_ip_align()is called to allocate SKBs. Ifnetdev_alloc_skb_ip_align()is called with
Quoted 8 lines
BH enabled,
local_bh_enable()at the end ofnetdev_alloc_skb_ip_align()will call__local_bh_enable_ip(), which will call__do_softirq(), which may triggernet_tx_action()softirq, which may ultimately call the xmit callbackks8851_start_xmit_par(). Theks8851_start_xmit_par()will try to lock structks8851_net_par.lock spinlock, which is already locked byks8851_irq()from whichks8851_start_xmit_par()was called. This leads to a deadlock, which is reported by the kernel, including a trace listed below.
lock_par is a spinlock, and AFAIU softirqs run in their on thread on RT. I'm not following.
The patch looks way to "advanced" for a driver. Something is going very wrong here. Or the commit message must be updated to explain it better to people like me. Or both.
Signature · 1 line
pw-bot: cr
Quoted 14 lines · Jakub Kicinski
Quoted 4 lines · On Wed, 8 Apr 2026 18:24:58 +0200 Marek Vasut wrote:
If
CONFIG_PREEMPT_RT=yis set AND the driver executesks8851_irq()ANDKSZ_ISRregister bitIRQ_RXIis set ANDks8851_rx_pkts()detects that there are packets in the RX FIFO, thennetdev_alloc_skb_ip_align()is called to allocate SKBs. Ifnetdev_alloc_skb_ip_align()is called withQuoted 8 lines
BH enabled,
local_bh_enable()at the end ofnetdev_alloc_skb_ip_align()will call__local_bh_enable_ip(), which will call__do_softirq(), which may triggernet_tx_action()softirq, which may ultimately call the xmit callbackks8851_start_xmit_par(). Theks8851_start_xmit_par()will try to lock structks8851_net_par.lock spinlock, which is already locked byks8851_irq()from whichks8851_start_xmit_par()was called. This leads to a deadlock, which is reported by the kernel, including a trace listed below.
Quoted 2 lines
lock_paris a spinlock, and AFAIU softirqs run in their on thread on RT. I'm not following.
Please look at the backtrace in the commit message, this part, please read from bottom to top to observe the failure in chronological order. It does not seem the handle_softirqs() is running in its own thread, separate from the IRQ thread ?
rt_spin_lock from ks8851_start_xmit_par+0x68/0x1a0 ks8851_start_xmit_par from netdev_start_xmit+0x1c/0x40 <---- this
tries to grab the same PAR spinlock, and deadlocks
netdev_start_xmit from dev_hard_start_xmit+0xec/0x1b0 dev_hard_start_xmit from sch_direct_xmit+0xb8/0x25c sch_direct_xmit from __qdisc_run+0x20c/0x4fc
__qdisc_run from qdisc_run+0x1c/0x28
qdisc_run from net_tx_action+0x1f4/0x244
net_tx_action from handle_softirqs+0x1c0/0x29c handle_softirqs from __local_bh_enable_ip+0xdc/0xf4 __local_bh_enable_ip from __netdev_alloc_skb+0x140/0x194 __netdev_alloc_skb from ks8851_irq+0x348/0x4d8 <---- this is called
from ks8851_rx_pkts() via netdev_alloc_skb_ip_align()
ks8851_irq from irq_thread_fn+0x24/0x64 <-------- this here runs with
the PAR spinlock held
Quoted 3 lines
The patch looks way to "advanced" for a driver. Something is going very wrong here. Or the commit message must be updated to explain it better to people like me. Or both.
Does the backtrace make the problem clearer, with the annotation above ?
Jakub Kicinski
maintainerQuoted 39 lines · On Sun, 12 Apr 2026 18:27:28 +0200 Marek Vasut wrote:
Quoted 14 lines · Jakub Kicinski
Quoted 4 lines · On Wed, 8 Apr 2026 18:24:58 +0200 Marek Vasut wrote:
If
CONFIG_PREEMPT_RT=yis set AND the driver executesks8851_irq()ANDKSZ_ISRregister bitIRQ_RXIis set ANDks8851_rx_pkts()detects that there are packets in the RX FIFO, thennetdev_alloc_skb_ip_align()is called to allocate SKBs. Ifnetdev_alloc_skb_ip_align()is called withQuoted 8 lines
BH enabled,
local_bh_enable()at the end ofnetdev_alloc_skb_ip_align()will call__local_bh_enable_ip(), which will call__do_softirq(), which may triggernet_tx_action()softirq, which may ultimately call the xmit callbackks8851_start_xmit_par(). Theks8851_start_xmit_par()will try to lock structks8851_net_par.lock spinlock, which is already locked byks8851_irq()from whichks8851_start_xmit_par()was called. This leads to a deadlock, which is reported by the kernel, including a trace listed below.Quoted 2 lines
lock_paris a spinlock, and AFAIU softirqs run in their on thread on RT. I'm not following.Please look at the backtrace in the commit message, this part, please read from bottom to top to observe the failure in chronological order. It does not seem the
handle_softirqs()is running in its own thread, separate from the IRQ thread ?rt_spin_lock from ks8851_start_xmit_par+0x68/0x1a0 ks8851_start_xmit_par from netdev_start_xmit+0x1c/0x40 <---- thistries to grab the same PAR spinlock, and deadlocks
netdev_start_xmitfromdev_hard_start_xmit+0xec/0x1b0dev_hard_start_xmitfromsch_direct_xmit+0xb8/0x25csch_direct_xmitfrom__qdisc_run+0x20c/0x4fc
__qdisc_runfromqdisc_run+0x1c/0x28
qdisc_runfromnet_tx_action+0x1f4/0x244
net_tx_actionfromhandle_softirqs+0x1c0/0x29chandle_softirqsfrom__local_bh_enable_ip+0xdc/0xf4__local_bh_enable_ipfrom__netdev_alloc_skb+0x140/0x194__netdev_alloc_skbfromks8851_irq+0x348/0x4d8<---- this is called
fromks8851_rx_pkts()vianetdev_alloc_skb_ip_align()
ks8851_irqfromirq_thread_fn+0x24/0x64<-------- this here runs with
the PAR spinlock held
Quoted 5 lines
Quoted 3 lines
The patch looks way to "advanced" for a driver. Something is going very wrong here. Or the commit message must be updated to explain it better to people like me. Or both.
Does the backtrace make the problem clearer, with the annotation above ?
Sebastian, do you have any recommendation here? tl;dr is that the driver does
spin_lock_irqsave() __netdev_alloc_skb() spin_unlock_irqrestore()
And __netdev_alloc_skb() does:
if (in_hardirq() || irqs_disabled()) {
nc = this_cpu_ptr(&netdev_alloc_cache);
data = page_frag_alloc(nc, len, gfp_mask);
pfmemalloc = page_frag_cache_is_pfmemalloc(nc);
} else {
local_bh_disable();
local_lock_nested_bh(&napi_alloc_cache.bh_lock);
nc = this_cpu_ptr(&napi_alloc_cache.page);
data = page_frag_alloc(nc, len, gfp_mask);
pfmemalloc = page_frag_cache_is_pfmemalloc(nc);
local_unlock_nested_bh(&napi_alloc_cache.bh_lock);
local_bh_enable();
}the local_bh_enable() seems to kick in BH processing inline, and BH processing takes the same spin lock the driver is already holding.
Quoted 3 lines · Jakub Kicinski
Quoted 1 line
Does the backtrace make the problem clearer, with the annotation above ?
Sebastian, do you have any recommendation here? tl;dr is that the driver does
…
What about this:
drivers/net/ethernet/micrel/ks8851_par.c +2 -2
@@ -63,7 +63,7 @@ static void ks8851_lock_par(struct ks8851_net *ks, unsigned long *flags) {struct ks8851_net_par *ksp =to_ks8851_par (ks); -spin_lock_irqsave (&ksp->lock, *flags); +spin_lock_bh (&ksp->lock); }/**
@@ -77,7 +77,7 @@ static void ks8851_unlock_par(struct ks8851_net *ks, unsigned long *flags) {struct ks8851_net_par *ksp =to_ks8851_par (ks); -spin_unlock_irqrestore (&ksp->lock, *flags); +spin_unlock_bh (&ksp->lock); }/**
I don't see why it needs to disable interrupts. This seems to be used by the _par driver and the _common part. The comments refer to DMA but I see only FIFO access.
And while at it, I would recommend to
drivers/net/ethernet/micrel/ks8851_common.c +4 -1
@@ -378,9 +378,12 @@ static irqreturn_t ks8851_irq(int irq, void *_ks)if (status &IRQ_LCI )mii_check_link (&ks->mii); -if (status &IRQ_RXI ) +if (status &IRQ_RXI ) { +local_bh_disable ();while ((skb =__skb_dequeue (&rxq)))netif_rx (skb); +local_bh_enable (); + }return IRQ_HANDLED ; }
Because otherwise it will kick-off backlog NAPI after every packet if multiple packets are available.
Sebastian
Quoted 30 lines · Sebastian Andrzej Siewior
Quoted 3 lines · Jakub Kicinski
Quoted 1 line
Does the backtrace make the problem clearer, with the annotation above ?
Sebastian, do you have any recommendation here? tl;dr is that the driver does
…
What about this:
drivers/net/ethernet/micrel/ks8851_par.c +2
-2@@ -63,7 +63,7 @@ static void ks8851_lock_par(struct ks8851_net *ks, unsigned long *flags) {struct ks8851_net_par *ksp =to_ks8851_par (ks); -spin_lock_irqsave (&ksp->lock, *flags); +spin_lock_bh (&ksp->lock); }/** @@ -77,7 +77,7 @@ static void ks8851_unlock_par(struct ks8851_net *ks, unsigned long *flags) {struct ks8851_net_par *ksp =to_ks8851_par (ks); -spin_unlock_irqrestore (&ksp->lock, *flags); +spin_unlock_bh (&ksp->lock); }/**
Quoted 1 line
I don't see why it needs to disable interrupts.
Because when the lock is held, the PAR code shouldn't be interrupted by an interrupt, otherwise it would completely mess up the state of the KS8851 MAC. The spinlock does not protect only the IRQ handler, it protects also ks8851_start_xmit_par() and ks8851_write_mac_addr() and ks8851_read_mac_addr() and ks8851_net_open() and ks8851_net_stop() and other sites which call ks8851_lock()/ks8851_unlock() which cannot be executed concurrently, but where BHs can be enabled.
Quoted 3 lines
? This seems to be used by
the_pardriver and the_commonpart. The comments refer to DMA but I see only FIFO access.
The KS8851 does its own internal DMA into the SRAM, from which the data are copied by the driver into system DRAM.
Quoted 21 lines
And while at it, I would recommend to
drivers/net/ethernet/micrel/ks8851_common.c +4
-1@@ -378,9 +378,12 @@ static irqreturn_t ks8851_irq(int irq, void *_ks)if (status &IRQ_LCI )mii_check_link (&ks->mii); -if (status &IRQ_RXI ) +if (status &IRQ_RXI ) { +local_bh_disable ();while ((skb =__skb_dequeue (&rxq)))netif_rx (skb); +local_bh_enable (); + }return IRQ_HANDLED ; }
Quoted 2 lines
Because otherwise it will kick-off backlog NAPI after every packet if multiple packets are available.
I think this patch will do the same, but the above should be done for the SPI part ?
Quoted 9 lines · Marek Vasut
Quoted 1 line
I don't see why it needs to disable interrupts.
Because when the lock is held, the PAR code shouldn't be interrupted by an interrupt, otherwise it would completely mess up the state of the KS8851 MAC. The spinlock does not protect only the IRQ handler, it protects also
ks8851_start_xmit_par()andks8851_write_mac_addr()andks8851_read_mac_addr()andks8851_net_open()andks8851_net_stop()and other sites which call ks8851_lock()/ks8851_unlock() which cannot be executed concurrently, but where BHs can be enabled.
I need check this once brain is at full power again. But which interrupt? Your interrupt is threaded. So that should be okay.
Quoted 6 lines
Quoted 3 lines
? This seems to be used by
the_pardriver and the_commonpart. The comments refer to DMA but I see only FIFO access.The KS8851 does its own internal DMA into the SRAM, from which the data are copied by the driver into system DRAM.
So this no interrupt involved as "dma completed" and you do your manual "memcpy".
Quoted 19 lines
Quoted 19 lines
And while at it, I would recommend to
drivers/net/ethernet/micrel/ks8851_common.c +4
-1@@ -378,9 +378,12 @@ static irqreturn_t ks8851_irq(int irq, void *_ks)if (status &IRQ_LCI )mii_check_link (&ks->mii); -if (status &IRQ_RXI ) +if (status &IRQ_RXI ) { +local_bh_disable ();while ((skb =__skb_dequeue (&rxq)))netif_rx (skb); +local_bh_enable (); + }return IRQ_HANDLED ; }
Quoted 4 lines
Quoted 2 lines
Because otherwise it will kick-off backlog NAPI after every packet if multiple packets are available.
I think this patch will do the same, but the above should be done for the SPI part ?
Yes, both. This the SPI/ Mutex part does not matter. You inject one packet into netif_rx() then if will add it to its internal NAPI and schedule a softirq, process it. It would be more efficient to queue multiple packets and process them all at the local_bh_enable() time.
Sebastian
Quoted 11 lines · To Marek Vasut
Quoted 9 lines · Marek Vasut
Quoted 1 line
I don't see why it needs to disable interrupts.
Because when the lock is held, the PAR code shouldn't be interrupted by an interrupt, otherwise it would completely mess up the state of the KS8851 MAC. The spinlock does not protect only the IRQ handler, it protects also
ks8851_start_xmit_par()andks8851_write_mac_addr()andks8851_read_mac_addr()andks8851_net_open()andks8851_net_stop()and other sites which call ks8851_lock()/ks8851_unlock() which cannot be executed concurrently, but where BHs can be enabled.
Quoted 2 lines
I need check this once brain is at full power again. But which interrupt? Your interrupt is threaded. So that should be okay.
I don't understand. There is no point in using spin_lock_irqsave() in ks8851_lock_par(). You don't protect against interrupts because none of the user actually run in an interrupt. As far as I can see, the interrupt is threaded and the mdio phy link checks should come from the workqueue.
What is wrong is that the ndo_start_xmit callback can be invoked from a softirq and such you must disable BHs while acquiring a lock which can be accessed from both contexts. Therefore spin_lock() is not sufficient, it needs the _bh() and _irq() brings no additional value here.
Sebastian
Quoted 12 lines · Sebastian Andrzej Siewior
Quoted 11 lines · To Marek Vasut
Quoted 9 lines · Marek Vasut
Quoted 1 line
I don't see why it needs to disable interrupts.
Because when the lock is held, the PAR code shouldn't be interrupted by an interrupt, otherwise it would completely mess up the state of the KS8851 MAC. The spinlock does not protect only the IRQ handler, it protects also
ks8851_start_xmit_par()andks8851_write_mac_addr()andks8851_read_mac_addr()andks8851_net_open()andks8851_net_stop()and other sites which call ks8851_lock()/ks8851_unlock() which cannot be executed concurrently, but where BHs can be enabled.
Quoted 8 lines
Quoted 2 lines
I need check this once brain is at full power again. But which interrupt? Your interrupt is threaded. So that should be okay.
I don't understand. There is no point in using
spin_lock_irqsave()inks8851_lock_par(). You don't protect against interrupts because none of the user actually run in an interrupt. As far as I can see, the interrupt is threaded and the mdio phy link checks should come from the workqueue.
Ha, now that the IRQ handler is indeed only threaded, I can use spin_lock_bh() indeed. I will send a V3 like that.
Quoted 4 lines
What is wrong is that the
ndo_start_xmitcallback can be invoked from a softirq and such you must disable BHs while acquiring a lock which can be accessed from both contexts. Thereforespin_lock()is not sufficient, it needs the_bh()and_irq()brings no additional value here.
Jakub Kicinski
maintainerQuoted 7 lines · On Mon, 13 Apr 2026 14:57:44 +0200 Sebastian Andrzej Siewior wrote:
Quoted 3 lines · Jakub Kicinski
Quoted 1 line
Does the backtrace make the problem clearer, with the annotation above ?
Sebastian, do you have any recommendation here? tl;dr is that the driver does
…
What about this:
Thanks for taking a look (according to you auto-reply immediately after a vacation ;))
TBH changing the driver feels like a workaround / invitation for a whack-a-mole game. I'd prefer to fix the skb allocation.
Is there any way we can check if any locks which were _irq() on non-RT are held?
Quoted 11 lines · Jakub Kicinski
Quoted 7 lines · On Mon, 13 Apr 2026 14:57:44 +0200 Sebastian Andrzej Siewior wrote:
Quoted 3 lines · Jakub Kicinski
Quoted 1 line
Does the backtrace make the problem clearer, with the annotation above ?
Sebastian, do you have any recommendation here? tl;dr is that the driver does
…
What about this:
Thanks for taking a look (according to you auto-reply immediately after a vacation ;))
;)
Quoted 2 lines
TBH changing the driver feels like a workaround / invitation for a whack-a-mole game. I'd prefer to fix the skb allocation.
The problem is that _irq() implicitly disables bh processing but this does not happen. Forcing this is possible but expensive. However, I did remove lock from bh_disable() on RT.
Marek: from which kernel version was this backtrace?
Quoted 2 lines
Is there any way we can check if any locks which were
_irq()on non-RT are held?
lockdep has a list of locks which are acquired but it does not see if it is _irq() or not. It only records it was acquired.
Sebastian
Quoted 8 lines · Sebastian Andrzej Siewior
Quoted 2 lines
TBH changing the driver feels like a workaround / invitation for a whack-a-mole game. I'd prefer to fix the skb allocation.
The problem is that
_irq()implicitly disables bh processing but this does not happen. Forcing this is possible but expensive. However, I did remove lock frombh_disable()on RT.Marek: from which kernel version was this backtrace?
That was v6.12.79-rt17-rebase where it was easy to trigger , here is a fresh one from next-20260413 with b44596ffe1b4 ("ARM: Allow to enable RT") from stable-rt added on top:
"
rcu: INFO: rcu_preempt detected stalls on CPUs/tasks: rcu: Tasks blocked on level-0 rcu_node (CPUs 0-1): P127/5:b..l rcu: (detected by 1, t=2102 jiffies, g=-503, q=1 ncpus=2) task:irq/68-eth1 state:D stack:0 pid:127 tgid:127 ppid:2 task_flags:0x4208040 flags:0x00000000 Call trace: __schedule from schedule_rtlock+0x1c/0x34 schedule_rtlock from rtlock_slowlock_locked+0x548/0x904 rtlock_slowlock_locked from rt_spin_lock+0x60/0x9c rt_spin_lock from ks8851_start_xmit_par+0x74/0x1a8 ks8851_start_xmit_par from netdev_start_xmit+0x20/0x44 netdev_start_xmit from dev_hard_start_xmit+0xd0/0x188 dev_hard_start_xmit from sch_direct_xmit+0xb8/0x25c sch_direct_xmit from __qdisc_run+0x1f8/0x4ec __qdisc_run from qdisc_run+0x1c/0x28 qdisc_run from net_tx_action+0x1f0/0x268 net_tx_action from handle_softirqs+0x1a4/0x270 handle_softirqs from __local_bh_enable_ip+0xcc/0xe0 __local_bh_enable_ip from __alloc_skb+0xd8/0x128 __alloc_skb from __netdev_alloc_skb+0x3c/0x19c __netdev_alloc_skb from ks8851_irq+0x388/0x4d4 ks8851_irq from irq_thread_fn+0x24/0x64 irq_thread_fn from irq_thread+0x178/0x28c irq_thread from kthread+0x12c/0x138 kthread from ret_from_fork+0x14/0x28 Exception stack(0xf4c99fb0 to 0xf4c99ff8)
9fa0: 00000000 00000000 00000000 00000000
9fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 9fe0: 00000000 00000000 00000000 00000000 00000013 00000000
"
Quoted 21 lines · Jakub Kicinski
Quoted 21 lines
rt_spin_lock from ks8851_start_xmit_par+0x68/0x1a0 ks8851_start_xmit_par from netdev_start_xmit+0x1c/0x40 <---- thistries to grab the same PAR spinlock, and deadlocks
netdev_start_xmitfromdev_hard_start_xmit+0xec/0x1b0dev_hard_start_xmitfromsch_direct_xmit+0xb8/0x25csch_direct_xmitfrom__qdisc_run+0x20c/0x4fc
__qdisc_runfromqdisc_run+0x1c/0x28
qdisc_runfromnet_tx_action+0x1f4/0x244
net_tx_actionfromhandle_softirqs+0x1c0/0x29chandle_softirqsfrom__local_bh_enable_ip+0xdc/0xf4__local_bh_enable_ipfrom__netdev_alloc_skb+0x140/0x194__netdev_alloc_skbfromks8851_irq+0x348/0x4d8<---- this is called
fromks8851_rx_pkts()vianetdev_alloc_skb_ip_align()
ks8851_irqfromirq_thread_fn+0x24/0x64<-------- this here runs with
the PAR spinlock heldQuoted 3 lines
The patch looks way to "advanced" for a driver. Something is going very wrong here. Or the commit message must be updated to explain it better to people like me. Or both.
Quoted 7 lines
Quoted 1 line
Does the backtrace make the problem clearer, with the annotation above ?
Sebastian, do you have any recommendation here? tl;dr is that the driver does
spin_lock_irqsave() __netdev_alloc_skb() spin_unlock_irqrestore()
So that is what happens in the backtrace. But not as of v7.0 if I look at ks8851_irq():
| if (status & IRQ_TXI) {
| unsigned short tx_space = ks8851_rdreg16(ks, KS_TXMIR);
|
| netif_dbg(ks, intr, ks->netdev,
| "%s: txspace %d\n", __func__, tx_space);
|
| spin_lock_bh(&ks->statelock);disables bh
| ks->tx_space = tx_space; | if (netif_queue_stopped(ks->netdev)) | netif_wake_queue(ks->netdev);
wakes queue, raise softirq, net-tx which does the qdisc_run() as seen in the backtrace
| spin_unlock_bh(&ks->statelock);
enables bh and runs it
| }
So this I understand and it would lead to a similar backtrace. However this shouldn't occur from __netdev_alloc_skb().
Quoted 18 lines
And
__netdev_alloc_skb()does:if (in_hardirq() || irqs_disabled()) { nc = this_cpu_ptr(&netdev_alloc_cache); data = page_frag_alloc(nc, len, gfp_mask); pfmemalloc = page_frag_cache_is_pfmemalloc(nc); } else { local_bh_disable(); local_lock_nested_bh(&napi_alloc_cache.bh_lock); nc = this_cpu_ptr(&napi_alloc_cache.page); data = page_frag_alloc(nc, len, gfp_mask); pfmemalloc = page_frag_cache_is_pfmemalloc(nc); local_unlock_nested_bh(&napi_alloc_cache.bh_lock); local_bh_enable(); }
Quoted 3 lines
the
local_bh_enable()seems to kick in BH processing inline, and BH processing takes the same spin lock the driver is already holding.
Yes, it does. But there is nothing between local_bh_disable() and local_bh_enable() that raises the softirq. Looking at v6.9 there is the following instead:
| spin_lock(&ks->statelock); | ks->tx_space = tx_space; | if (netif_queue_stopped(ks->netdev)) | netif_wake_queue(ks->netdev); | spin_unlock(&ks->statelock);
So no _bh() here. So here netif_wake_queue() woke ksoftirqd to handle it. _Later_ there is this alloc_skb which does local_bh_disable()/ enable() and the latter will look at pending softirqs. They are still set from before because ksoftirqd had no chance processing them. And now you see the deadlock from within __netdev_alloc_skb().
I *think* lockdep will yell here on RT.
Looking at current kernel from !RT perspective, this isn't good either. We have:
| ks8851_irq
| {
| ks8851_lock()
| -> spin_lock_irqsave()irqs are off
| if (status & IRQ_TXI) {
| spin_lock_bh(&ks->statelock);
| if (netif_queue_stopped(ks->netdev))
| netif_wake_queue(ks->netdev);raise softirq
| spin_unlock_bh(&ks->statelock);
bh enable with disabled interrupts. And __local_bh_enable_ip() has this gem:
|void __local_bh_enable_ip(unsigned long ip, unsigned int cnt)
| {
| WARN_ON_ONCE(in_hardirq());
| lockdep_assert_irqs_enabled();
| #ifdef CONFIG_TRACE_IRQFLAGS
| local_irq_disable();
| #endifso lockep will yell if interrupts are disabled. And handle_softirqs() will enable interrupts before handling softirqs and restore them later on. But CONFIG_TRACE_IRQFLAGS will keep them enabled. Since the lock is not acquired in hardirq, it has no other deadlock problem.
What I don't understand is why this is limited to PREEMPT_RT. !RT is also affected by this:
- ks8851_irq() acquires the lock, disables interrupts
- netif_wake_queue() raises the softirq
- spin_unlock_bh(&ks->statelock) enables BH and handles softirqs, and goes to ks8851_start_xmit()
This is only possible in newer kernels due to 0913ec336a6c0 ("net:
ks8851: Fix deadlock with the SPI chip variant") because of the irq_disabled() check in skb allocation.
So. Using _bh instead _irq remains my recommendation. Lockdep should already yell on !RT here.
Sebastian