sbitmap: Protect swap_lock from hardirq
Subsystems (2)
PATCH 1/1 sbitmap: Protect swap_lock from hardirq
The original report is actually one real deadlock:
[ 106.132865] Possible interrupt unsafe locking scenario:
[ 106.132865]
[ 106.133659] CPU0 CPU1
[ 106.134194] ---- ----
[ 106.134733] lock(&(&sb->map[i].swap_lock)->rlock);
[ 106.135318] local_irq_disable();
[ 106.136014] lock(&sbq->ws[i].wait);
[ 106.136747] lock(&(&hctx->dispatch_wait_lock)->rlock);
[ 106.137742] <Interrupt>
[ 106.138110] lock(&sbq->ws[i].wait);Because we may call blk_mq_get_driver_tag() directly from blk_mq_dispatch_rq_list() without holding any lock, then HARDIRQ may come and the above DEADLOCK is triggered.
ab53dcfb3e7b ("sbitmap: Protect swap_lock from hardirq") tries to fix this issue by using 'spin_lock_bh', which isn't enough because we complete request from hardirq context direclty in case of multiqueue.
--- lib/sbitmap.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
lib/sbitmap.c +3 -2
@@ -27,8 +27,9 @@ static inline bool sbitmap_deferred_clear(struct sbitmap *sb, int index) {unsigned long mask, val;bool ret = false; +unsigned long flags; -spin_lock_bh (&sb->map[index].swap_lock); +spin_lock_irqsave (&sb->map[index].swap_lock, flags);if (!sb->map[index].cleared) goto out_unlock;
@@ -49,7 +50,7 @@ static inline bool sbitmap_deferred_clear(struct sbitmap *sb, int index) ret = true; out_unlock: -spin_unlock_bh (&sb->map[index].swap_lock); +spin_unlock_irqrestore (&sb->map[index].swap_lock, flags);return ret; }
Signature · 1 line
2.14.4
Jens Axboe
maintainerQuoted 13 lines · Ming Lei
The original report is actually one real deadlock:
[ 106.132865] Possible interrupt unsafe locking scenario: [ 106.132865] [ 106.133659] CPU0 CPU1 [ 106.134194] ---- ---- [ 106.134733] lock(&(&sb->map[i].swap_lock)->rlock); [ 106.135318] local_irq_disable(); [ 106.136014] lock(&sbq->ws[i].wait); [ 106.136747] lock(&(&hctx->dispatch_wait_lock)->rlock); [ 106.137742] <Interrupt> [ 106.138110] lock(&sbq->ws[i].wait);
Quoted 7 lines
Because we may call
blk_mq_get_driver_tag()directly fromblk_mq_dispatch_rq_list()without holding any lock, then HARDIRQ may come and the above DEADLOCK is triggered.
ab53dcfb3e7b("sbitmap: Protectswap_lockfrom hardirq") tries to fix this issue by using 'spin_lock_bh', which isn't enough because we complete request from hardirq context direclty in case of multiqueue.
Thanks Ming, I'll queue this up for shipping this week.
Signature · 1 line
Jens Axboe
Quoted 2 lines · Jens Axboe
Thanks Ming, I'll queue this up for shipping this week.
Oops. I _just_ applied it to my tree as a follow-up to Steven's softirq version. I just hadn't had time to build test and push out yet.
Linus
Jens Axboe
maintainerQuoted 7 lines · Linus Torvalds
Quoted 2 lines · Jens Axboe
Thanks Ming, I'll queue this up for shipping this week.
Oops. I _just_ applied it to my tree as a follow-up to Steven's softirq version. I just hadn't had time to build test and push out yet.
No big deal, fwiw, this is what I queued up:
which has a spelling error fixed, and the indentation a bit nicer for the locking scenario. But I can just drop it.
Signature · 1 line
Jens Axboe