firmware: tegra: bpmp: Make atomic_tx_lock a raw_spinlock
Subsystems (2)
PATCH 1/1 firmware: tegra: bpmp: Make atomic_tx_lock a raw_spinlock
The atomic_tx_lock was first introduced by commit 1abb081e41a7 ("firmware: tegra: Simplify channel management") as a spinlock_t. It is used only in tegra_bpmp_transfer_atomic() to ensure mutual exclusion.
Since the merging of PREEMPT_RT support into mainline Linux kernel in v6.12, a spinlock becomes a sleeping lock when CONFIG_PREEMPT_RT is enabled. As tegra_bpmp_transfer_atomic() is called with interrupt disabled, acquiring a sleeping lock will lead to the following bug report when booting up a PREEMPT_RT kernel on an tegra based arm64 system with Boot and Power Management Processor (BPMP).
BUG: sleeping function called from invalid context at kernel/locking/spinlock_rt.c:48
in_atomic(): 0, irqs_disabled(): 1, non_block: 0, pid: 1, name: swapper/0
preempt_count: 0, expected: 0
RCU nest depth: 0, expected: 0
2 locks held by swapper/0/1:
#0: ffff42bda2d5b0f0 (&dev->mutex){....}-{3:3}, at: __driver_attach+0x110/0x2c0
#1: ffff42bda4da5138 (&bpmp->atomic_tx_lock){....}-{2:2}, at: tegra_bpmp_transfer_atomic+0x118/0x3c0Fix it by changing the type of atomic_tx_lock in the tegra_bpmp structure to raw_spinlock_t and use raw_spinlock APIs to access it.
--- drivers/firmware/tegra/bpmp.c | 8 ++++---- include/soc/tegra/bpmp.h | 2 +- 2 files changed, 5 insertions(+), 5 deletions(-)
drivers/firmware/tegra/bpmp.c +4 -4
@@ -365,16 +365,16 @@ int tegra_bpmp_transfer_atomic(struct tegra_bpmp *bpmp, channel = bpmp->tx_channel; -spin_lock (&bpmp->atomic_tx_lock); +raw_spin_lock (&bpmp->atomic_tx_lock); err =tegra_bpmp_channel_write (channel, msg->mrq,MSG_ACK , msg->tx.data, msg->tx.size);if (err <0 ) { -spin_unlock (&bpmp->atomic_tx_lock); +raw_spin_unlock (&bpmp->atomic_tx_lock);return err; } -spin_unlock (&bpmp->atomic_tx_lock); +raw_spin_unlock (&bpmp->atomic_tx_lock); err =tegra_bpmp_ring_doorbell (bpmp);if (err <0 )
@@ -763,7 +763,7 @@ static int tegra_bpmp_probe(struct platform_device *pdev)if (!bpmp->threaded.busy)return -ENOMEM ; -spin_lock_init (&bpmp->atomic_tx_lock); +raw_spin_lock_init (&bpmp->atomic_tx_lock); bpmp->tx_channel =devm_kzalloc (&pdev->dev,sizeof (*bpmp->tx_channel),GFP_KERNEL );if (!bpmp->tx_channel)
include/soc/tegra/bpmp.h +1 -1
@@ -79,7 +79,7 @@ struct tegra_bpmp {struct mbox_chan *channel; } mbox; -spinlock_t atomic_tx_lock; +raw_spinlock_t atomic_tx_lock;struct tegra_bpmp_channel *tx_channel, *rx_channel, *threaded_channels;struct {
Signature · 1 line
2.53.0
Sebastian Andrzej Siewior
maintainerQuoted 19 lines · Waiman Long
The
atomic_tx_lockwas first introduced by commit1abb081e41a7("firmware: tegra: Simplify channel management") as aspinlock_t. It is used only integra_bpmp_transfer_atomic()to ensure mutual exclusion.Since the merging of
PREEMPT_RTsupport into mainline Linux kernel in v6.12, a spinlock becomes a sleeping lock whenCONFIG_PREEMPT_RTis enabled. Astegra_bpmp_transfer_atomic()is called with interrupt disabled, acquiring a sleeping lock will lead to the following bug report when booting up aPREEMPT_RTkernel on an tegra based arm64 system with Boot and Power Management Processor (BPMP).BUG: sleeping function called from invalid context at kernel/locking/spinlock_rt.c:48 in_atomic(): 0, irqs_disabled(): 1, non_block: 0, pid: 1, name: swapper/0 preempt_count: 0, expected: 0 RCU nest depth: 0, expected: 0 2 locks held by swapper/0/1: #0: ffff42bda2d5b0f0 (&dev->mutex){....}-{3:3}, at: __driver_attach+0x110/0x2c0 #1: ffff42bda4da5138 (&bpmp->atomic_tx_lock){....}-{2:2}, at: tegra_bpmp_transfer_atomic+0x118/0x3c0
Quoted 2 lines
Fix it by changing the type of
atomic_tx_lockin thetegra_bpmpstructure toraw_spinlock_tand useraw_spinlockAPIs to access it.
Do you have a backtrace why interrupts are disabled to begin with?
Sebastian
A message that is not in the archive (sent off-list or not yet received): [email protected]
Sebastian Andrzej Siewior
maintainerQuoted 4 lines · Waiman Long
__might_resched+0x254/0x330 rt_spin_lock+0x70/0x140 tegra_bpmp_transfer_atomic+0x118/0x3c0 tegra_bpmp_probe+0x564/0x6f0
So this is tegra_bpmp_ping().
Quoted 5 lines
I know that interrupt is disabled becasue of the following code.
346 int
tegra_bpmp_transfer_atomic(struct tegra_bpmp *bpmp, 347 struct tegra_bpmp_message *msg)348 { 349 structtegra_bpmp_channel*channel; 350 int err; 351 352 if (WARN_ON(!irqs_disabled())) 353 return-EPERM;
Well, yes. It disables interrupts just probably to document the time it took for the transfer so it can write it then via dev_dbg(). It is hard to tell what the worst-case delay here is but it is probably not important if this is just boot time/ driver probe. Maybe. What I am bit more concerned if the actual path of this tegra_bpmp_transfer_atomic() invocation via i2c driver is actually invoked with disabled interrupts on PREEMPT_RT. Because that might not be the case. I've been looking at the i2c call chain and it is not obvious what the actual call chain is. There is just i2c_in_atomic_xfer_mode() check. So it may or may not be used in the end.
Quoted 1 line
Cheers, Longman
Sebastian
Quoted 12 lines · Sebastian Andrzej Siewior
Quoted 4 lines · Waiman Long
__might_resched+0x254/0x330 rt_spin_lock+0x70/0x140 tegra_bpmp_transfer_atomic+0x118/0x3c0 tegra_bpmp_probe+0x564/0x6f0So this is
tegra_bpmp_ping().Quoted 5 lines
I know that interrupt is disabled becasue of the following code.
346 int
tegra_bpmp_transfer_atomic(struct tegra_bpmp *bpmp, 347 struct tegra_bpmp_message *msg)348 { 349 structtegra_bpmp_channel*channel; 350 int err; 351 352 if (WARN_ON(!irqs_disabled())) 353 return-EPERM;
Quoted 11 lines
Well, yes. It disables interrupts just probably to document the time it took for the transfer so it can write it then via
dev_dbg(). It is hard to tell what the worst-case delay here is but it is probably not important if this is just boot time/ driver probe. Maybe. What I am bit more concerned if the actual path of thistegra_bpmp_transfer_atomic()invocation via i2c driver is actually invoked with disabled interrupts onPREEMPT_RT. Because that might not be the case. I've been looking at the i2c call chain and it is not obvious what the actual call chain is. There is justi2c_in_atomic_xfer_mode()check. So it may or may not be used in the end.
tegra_bpmp_channel_write() calls tegra_bpmp_wait_request_channel_free() which waits until the channel is freed. So it is hard to tell how long does that take. In my case, the bug report happened at boot time.
Anyway, are there other comment about this patch?
Thanks,
Longman