WARNING: CPU: 0 PID: 1444 at kernel/sched/core.c:7088 __might_sleep+0x51/0x16f()
Subsystems (3)
Greetings,
Here is a microcode/load_module error triggered by debug check commit 64c2181bc433b17f04da8fe8592aa83cceac9606 ("sched: Debug nested sleeps"):
[main] Setsockopt(1 8 80d1000 4) on fd 21 [1:2:1] [main] Setsockopt(1 2f 80d1000 4) on fd 22 [4:2:60]
[ 14.027148] ------------[ cut here ]------------ [ 14.027864] WARNING: CPU: 0 PID: 210 at kernel/sched/core.c:7088 __might_sleep+0x40/0x68() [ 14.029295] do not call blocking ops when !TASK_RUNNING; state=2 set at [<c144e379>] prepare_to_wait+0x35/0x56 [ 14.030590] Modules linked in: [ 14.031136] CPU: 0 PID: 210 Comm: trinity-main Not tainted 3.16.0-02167-g254135e #972 [ 14.032263] 00000000 c0f4de4c c0f4de24 c196630c c0f4de3c c142f01a c1447632 c0f1dbb0 [ 14.033480] 00000002 b0066140 c0f4de54 c142f057 00000009 c0f4de4c c1b3bac8 c0f4de68 [ 14.034640] c0f4de88 c1447632 c1b3bb12 00001bb0 c1b3bac8 00000002 c144e379 c144e379 [ 14.035983] Call Trace: [ 14.036355] [<c196630c>] dump_stack+0x16/0x18 [ 14.037005] [<c142f01a>] warn_slowpath_common+0x55/0x6c [ 14.037715] [<c1447632>] ? __might_sleep+0x40/0x68 [ 14.038372] [<c142f057>] warn_slowpath_fmt+0x26/0x2a [ 14.039097] [<c1447632>] __might_sleep+0x40/0x68 [ 14.039787] [<c144e379>] ? prepare_to_wait+0x35/0x56 [ 14.040595] [<c144e379>] ? prepare_to_wait+0x35/0x56 [ 14.041272] [<c14a837e>] kmem_cache_alloc+0x39/0xb0 [ 14.041934] [<c18fa2de>] ? __alloc_skb+0x3c/0x154 [ 14.042572] [<c18fa2de>] __alloc_skb+0x3c/0x154 [ 14.043339] [<c145117a>] ? mark_held_locks+0x44/0x60 [ 14.044141] [<c1946093>] sigd_enq2+0x2a/0xff [ 14.044836] [<c1946188>] sigd_enq+0x20/0x2a [ 14.045405] [<c19467fb>] svc_listen+0x8b/0x11f [ 14.046009] [<c144e5a6>] ? __wake_up_sync+0xd/0xd [ 14.046653] [<c18f4132>] SyS_listen+0x37/0x51 [ 14.047423] [<c18f4ce5>] SyS_socketcall+0x90/0x1c0 [ 14.048328] [<c145136e>] ? trace_hardirqs_on+0xb/0xd [ 14.049061] [<c19729f6>] ? restore_all+0xf/0xf [ 14.049665] [<c19729bd>] syscall_call+0x7/0x7 [ 14.050253] [<c1970000>] ? __ww_mutex_lock_interruptible+0x165/0x573 [ 14.051147] ---[ end trace 6f1365c63eafedde ]---
[main] Setsockopt(1 2d 80d1000 f0) on fd 25 [1:1:1]
Thanks,
Fengguang
Quoted 3 lines · Fengguang Wu
Greetings,
Here is a microcode/load_module error triggered by debug check commit
Sorry for the copy&paste error! This warning is not related to microcode/load_module..
Thanks,
Fengguang
Quoted 8 lines · Fengguang Wu
Quoted 3 lines · Fengguang Wu
Greetings,
Here is a microcode/load_module error triggered by debug check commit
Sorry for the copy&paste error! This warning is not related to microcode/load_module..
Quoted 7 lines
Thanks,
Fengguang
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Hey Fenngaug,
I looking through this trace.
213 skb = kmem_cache_alloc_node(cache, gfp_mask & ~__GFP_DMA, node); 214 if (!skb)
215 goto out;
This lines seem to be an issue as after tracing to were you are getting a warning,might_sleep is written as
might_sleep_if(flags & __GFP_WAIT);
This means that the flag for GFP_WAIT is probably set and sure enough you would be getting an error after sleeping in __alloc_skb.
Regards Nick
Quoted 38 lines · Fengguang Wu
Greetings,
Here is a microcode/load_module error triggered by debug check commit
64c2181bc433b17f04da8fe8592aa83cceac9606("sched: Debug nested sleeps"):[main] Setsockopt(1 8 80d1000 4) on fd 21 [1:2:1] [main] Setsockopt(1 2f 80d1000 4) on fd 22 [4:2:60]
[ 14.027148] ------------[ cut here ]------------ [ 14.027864] WARNING: CPU: 0 PID: 210 at kernel/sched/core.c:7088 __might_sleep+0x40/0x68() [ 14.029295] do not call blocking ops when !TASK_RUNNING; state=2 set at [<c144e379>] prepare_to_wait+0x35/0x56 [ 14.030590] Modules linked in: [ 14.031136] CPU: 0 PID: 210 Comm: trinity-main Not tainted 3.16.0-02167-g254135e #972 [ 14.032263] 00000000 c0f4de4c c0f4de24 c196630c c0f4de3c c142f01a c1447632 c0f1dbb0 [ 14.033480] 00000002 b0066140 c0f4de54 c142f057 00000009 c0f4de4c c1b3bac8 c0f4de68 [ 14.034640] c0f4de88 c1447632 c1b3bb12 00001bb0 c1b3bac8 00000002 c144e379 c144e379 [ 14.035983] Call Trace: [ 14.036355] [<c196630c>] dump_stack+0x16/0x18 [ 14.037005] [<c142f01a>] warn_slowpath_common+0x55/0x6c [ 14.037715] [<c1447632>] ? __might_sleep+0x40/0x68 [ 14.038372] [<c142f057>] warn_slowpath_fmt+0x26/0x2a [ 14.039097] [<c1447632>] __might_sleep+0x40/0x68 [ 14.039787] [<c144e379>] ? prepare_to_wait+0x35/0x56 [ 14.040595] [<c144e379>] ? prepare_to_wait+0x35/0x56 [ 14.041272] [<c14a837e>] kmem_cache_alloc+0x39/0xb0 [ 14.041934] [<c18fa2de>] ? __alloc_skb+0x3c/0x154 [ 14.042572] [<c18fa2de>] __alloc_skb+0x3c/0x154 [ 14.043339] [<c145117a>] ? mark_held_locks+0x44/0x60 [ 14.044141] [<c1946093>] sigd_enq2+0x2a/0xff [ 14.044836] [<c1946188>] sigd_enq+0x20/0x2a [ 14.045405] [<c19467fb>] svc_listen+0x8b/0x11f [ 14.046009] [<c144e5a6>] ? __wake_up_sync+0xd/0xd [ 14.046653] [<c18f4132>] SyS_listen+0x37/0x51 [ 14.047423] [<c18f4ce5>] SyS_socketcall+0x90/0x1c0 [ 14.048328] [<c145136e>] ? trace_hardirqs_on+0xb/0xd [ 14.049061] [<c19729f6>] ? restore_all+0xf/0xf [ 14.049665] [<c19729bd>] syscall_call+0x7/0x7 [ 14.050253] [<c1970000>] ? __ww_mutex_lock_interruptible+0x165/0x573 [ 14.051147] ---[ end trace 6f1365c63eafedde ]---
Quoted 1 line
[main] Setsockopt(1 2d 80d1000 f0) on fd 25 [1:1:1]
---
Subject: atm: Fix blocking in wait loop
One should not call blocking primitives inside a wait loop, since both require task_struct::state to sleep, so the inner will destroy the outer state.
In this instance sigd_enq() will possible sleep for alloc_skb(), now if I understand the code right, we do not actually need to call sigd_enq() after the initial prepare_to_wait(), because we test the termination condition before schedule() anyhow.
So we can simply move it up a bit and avoid the entire confusion.
--- net/atm/svc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
net/atm/svc.c +1 -1
@@ -297,8 +297,8 @@ static int svc_listen(struct socket *sock, int backlog) goto out; }set_bit (ATM_VF_WAITING , &vcc->flags); -prepare_to_wait (sk_sleep (sk), &wait,TASK_UNINTERRUPTIBLE );sigd_enq (vcc, as_listen,NULL ,NULL , &vcc->local); +prepare_to_wait (sk_sleep (sk), &wait,TASK_UNINTERRUPTIBLE );while (test_bit (ATM_VF_WAITING , &vcc->flags) && sigd) {schedule ();prepare_to_wait (sk_sleep (sk), &wait,TASK_UNINTERRUPTIBLE );
Quoted 23 lines · On Thu, 7 Aug 2014 17:17:41 +0200 Peter Zijlstra <[email protected]> wrote:
Subject: atm: Fix blocking in wait loop
One should not call blocking primitives inside a wait loop, since both require
task_struct::stateto sleep, so the inner will destroy the outer state.In this instance
sigd_enq()will possible sleep foralloc_skb(), now if I understand the code right, we do not actually need to callsigd_enq()after the initialprepare_to_wait(), because we test the termination condition beforeschedule()anyhow.So we can simply move it up a bit and avoid the entire confusion.
Signed-off-by Peter Zijlstra <[email protected]>--- net/atm/svc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)net/atm/svc.c +0
-0
net/atm/svc.c static int svc_listen(struct socket *sock, int backlog) · line 304
Quoted 9 lines
net/atm/svc.c +1
-1@@ -297,8 +297,8 @@ static int svc_listen(struct socket *sock, int backlog) goto out; }set_bit (ATM_VF_WAITING , &vcc->flags); -prepare_to_wait (sk_sleep (sk), &wait,TASK_UNINTERRUPTIBLE );sigd_enq (vcc, as_listen,NULL ,NULL , &vcc->local); +prepare_to_wait (sk_sleep (sk), &wait,TASK_UNINTERRUPTIBLE );while (test_bit (ATM_VF_WAITING , &vcc->flags) && sigd) {schedule ();prepare_to_wait (sk_sleep (sk), &wait,TASK_UNINTERRUPTIBLE );
This isn't the only place that we queue a message for the signalling daemon after a prepare_to_wait() uninterruptibly so this patch would be incomplete as is.
What bothers me is the TASK_UNINTERRUPTIBLE -- I don't have a good reason why any of these should be sleeping uninterruptibly.
Quoted 32 lines · chas williams - CONTRACTOR
Quoted 29 lines · On Thu, 7 Aug 2014 17:17:41 +0200 Peter Zijlstra <[email protected]> wrote:
Subject: atm: Fix blocking in wait loop
One should not call blocking primitives inside a wait loop, since both require
task_struct::stateto sleep, so the inner will destroy the outer state.In this instance
sigd_enq()will possible sleep foralloc_skb(), now if I understand the code right, we do not actually need to callsigd_enq()after the initialprepare_to_wait(), because we test the termination condition beforeschedule()anyhow.So we can simply move it up a bit and avoid the entire confusion.
Signed-off-by Peter Zijlstra <[email protected]>--- net/atm/svc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)net/atm/svc.c +1
-1@@ -297,8 +297,8 @@ static int svc_listen(struct socket *sock, int backlog) goto out; }set_bit (ATM_VF_WAITING , &vcc->flags); -prepare_to_wait (sk_sleep (sk), &wait,TASK_UNINTERRUPTIBLE );sigd_enq (vcc, as_listen,NULL ,NULL , &vcc->local); +prepare_to_wait (sk_sleep (sk), &wait,TASK_UNINTERRUPTIBLE );
Quoted 8 lines
net/atm/svc.c static int svc_listen(struct socket *sock, int backlog) · line 304
Quoted 3 lines
net/atm/svc.c +0
-0while (test_bit (ATM_VF_WAITING , &vcc->flags) && sigd) {schedule ();prepare_to_wait (sk_sleep (sk), &wait,TASK_UNINTERRUPTIBLE );This isn't the only place that we queue a message for the signalling daemon after a
prepare_to_wait()uninterruptibly so this patch would be incomplete as is.
I'm not sure I follow, this is the only place you do so while then going to sleep. All other sites don't sleep while they're enqueued on the waitqueue.
Quoted 2 lines
What bothers me is the
TASK_UNINTERRUPTIBLE-- I don't have a good reason why any of these should be sleeping uninterruptibly.
That's a whole different story, there's tons of ugly in there, but its all ancient code.
Quoted 7 lines · Peter Zijlstra
Quoted 3 lines
This isn't the only place that we queue a message for the signalling daemon after a
prepare_to_wait()uninterruptibly so this patch would be incomplete as is.I'm not sure I follow, this is the only place you do so while then going to sleep. All other sites don't sleep while they're enqueued on the waitqueue.
Argh, my brain is fried, you're quite right. I'll go have another stab at it tomorrow or so.
PATCH 1/1 atm/svc: Fix blocking in wait loop
One should not call blocking primitives inside a wait loop, since both require task_struct::state to sleep, so the inner will destroy the outer state.
sigd_enq() will possibly sleep for alloc_skb(). Move sigd_enq() before prepare_to_wait() to avoid sleeping while waiting interruptibly. You do not actually need to call sigd_enq() after the initial prepare_to_wait() because we test the termination condition before calling schedule().
Based on suggestions from Peter Zijlstra.
--- net/atm/svc.c | 60 +++++++++++++++++++++++++++++++---------------------------- 1 file changed, 32 insertions(+), 28 deletions(-)
net/atm/svc.c +32 -28
@@ -50,12 +50,12 @@ static void svc_disconnect(struct atm_vcc *vcc)pr_debug ("%p\n" , vcc);if (test_bit (ATM_VF_REGIS , &vcc->flags)) { -prepare_to_wait (sk_sleep (sk), &wait,TASK_UNINTERRUPTIBLE );sigd_enq (vcc, as_close,NULL ,NULL ,NULL ); -while (!test_bit (ATM_VF_RELEASED , &vcc->flags) && sigd) { +for (;;) { +prepare_to_wait (sk_sleep (sk), &wait,TASK_UNINTERRUPTIBLE ); +if (test_bit (ATM_VF_RELEASED , &vcc->flags) || !sigd) +break ;schedule (); -prepare_to_wait (sk_sleep (sk), &wait, -TASK_UNINTERRUPTIBLE ); }finish_wait (sk_sleep (sk), &wait); }
@@ -126,11 +126,12 @@ static int svc_bind(struct socket *sock, struct sockaddr *sockaddr, } vcc->local = *addr;set_bit (ATM_VF_WAITING , &vcc->flags); -prepare_to_wait (sk_sleep (sk), &wait,TASK_UNINTERRUPTIBLE );sigd_enq (vcc, as_bind,NULL ,NULL , &vcc->local); -while (test_bit (ATM_VF_WAITING , &vcc->flags) && sigd) { -schedule (); +for (;;) {prepare_to_wait (sk_sleep (sk), &wait,TASK_UNINTERRUPTIBLE ); +if (!test_bit (ATM_VF_WAITING , &vcc->flags) || !sigd) +break ; +schedule (); }finish_wait (sk_sleep (sk), &wait);clear_bit (ATM_VF_REGIS , &vcc->flags);/* doesn't count */
@@ -202,15 +203,14 @@ static int svc_connect(struct socket *sock, struct sockaddr *sockaddr, } vcc->remote = *addr;set_bit (ATM_VF_WAITING , &vcc->flags); -prepare_to_wait (sk_sleep (sk), &wait,TASK_INTERRUPTIBLE );sigd_enq (vcc, as_connect,NULL ,NULL , &vcc->remote);if (flags &O_NONBLOCK ) { -finish_wait (sk_sleep (sk), &wait); sock->state =SS_CONNECTING ; error = -EINPROGRESS ; goto out; } error =0 ; +prepare_to_wait (sk_sleep (sk), &wait,TASK_INTERRUPTIBLE );while (test_bit (ATM_VF_WAITING , &vcc->flags) && sigd) {schedule ();if (!signal_pending (current)) {
@@ -297,11 +297,12 @@ static int svc_listen(struct socket *sock, int backlog) goto out; }set_bit (ATM_VF_WAITING , &vcc->flags); -prepare_to_wait (sk_sleep (sk), &wait,TASK_UNINTERRUPTIBLE );sigd_enq (vcc, as_listen,NULL ,NULL , &vcc->local); -while (test_bit (ATM_VF_WAITING , &vcc->flags) && sigd) { -schedule (); +for (;;) {prepare_to_wait (sk_sleep (sk), &wait,TASK_UNINTERRUPTIBLE ); +if (!test_bit (ATM_VF_WAITING , &vcc->flags) || !sigd) +break ; +schedule (); }finish_wait (sk_sleep (sk), &wait);if (!sigd) {
@@ -387,15 +388,15 @@ static int svc_accept(struct socket *sock, struct socket *newsock, int flags) }/* wait should be short, so we ignore the non-blocking flag */ set_bit (ATM_VF_WAITING , &new_vcc->flags); -prepare_to_wait (sk_sleep (sk_atm (new_vcc)), &wait, -TASK_UNINTERRUPTIBLE );sigd_enq (new_vcc, as_accept, old_vcc,NULL ,NULL ); -while (test_bit (ATM_VF_WAITING , &new_vcc->flags) && sigd) { +for (;;) { +prepare_to_wait (sk_sleep (sk_atm (new_vcc)), &wait, +TASK_UNINTERRUPTIBLE ); +if (!test_bit (ATM_VF_WAITING , &new_vcc->flags) || !sigd) +break ;release_sock (sk);schedule ();lock_sock (sk); -prepare_to_wait (sk_sleep (sk_atm (new_vcc)), &wait, -TASK_UNINTERRUPTIBLE ); }finish_wait (sk_sleep (sk_atm (new_vcc)), &wait);if (!sigd) {
@@ -433,12 +434,14 @@ int svc_change_qos(struct atm_vcc *vcc, struct atm_qos *qos)DEFINE_WAIT (wait);set_bit (ATM_VF_WAITING , &vcc->flags); -prepare_to_wait (sk_sleep (sk), &wait,TASK_UNINTERRUPTIBLE );sigd_enq2 (vcc, as_modify,NULL ,NULL , &vcc->local, qos,0 ); -while (test_bit (ATM_VF_WAITING , &vcc->flags) && - !test_bit (ATM_VF_RELEASED , &vcc->flags) && sigd) { -schedule (); +for (;;) {prepare_to_wait (sk_sleep (sk), &wait,TASK_UNINTERRUPTIBLE ); +if (!test_bit (ATM_VF_WAITING , &vcc->flags) || +test_bit (ATM_VF_RELEASED , &vcc->flags) || !sigd) { +break ; + } +schedule (); }finish_wait (sk_sleep (sk), &wait);if (!sigd)
@@ -529,18 +532,18 @@ static int svc_addparty(struct socket *sock, struct sockaddr *sockaddr,lock_sock (sk);set_bit (ATM_VF_WAITING , &vcc->flags); -prepare_to_wait (sk_sleep (sk), &wait,TASK_INTERRUPTIBLE );sigd_enq (vcc, as_addparty,NULL ,NULL , (struct sockaddr_atmsvc *) sockaddr);if (flags &O_NONBLOCK ) { -finish_wait (sk_sleep (sk), &wait); error = -EINPROGRESS ; goto out; }pr_debug ("added wait queue\n" ); -while (test_bit (ATM_VF_WAITING , &vcc->flags) && sigd) { -schedule (); +for (;;) {prepare_to_wait (sk_sleep (sk), &wait,TASK_INTERRUPTIBLE ); +if (!test_bit (ATM_VF_WAITING , &vcc->flags) || !sigd) +break ; +schedule (); }finish_wait (sk_sleep (sk), &wait); error =xchg (&sk->sk_err_soft,0 );
@@ -558,11 +561,12 @@ static int svc_dropparty(struct socket *sock, int ep_ref)lock_sock (sk);set_bit (ATM_VF_WAITING , &vcc->flags); -prepare_to_wait (sk_sleep (sk), &wait,TASK_INTERRUPTIBLE );sigd_enq2 (vcc, as_dropparty,NULL ,NULL ,NULL ,NULL , ep_ref); -while (test_bit (ATM_VF_WAITING , &vcc->flags) && sigd) { -schedule (); +for (;;) {prepare_to_wait (sk_sleep (sk), &wait,TASK_INTERRUPTIBLE ); +if (!test_bit (ATM_VF_WAITING , &vcc->flags) || !sigd) +break ; +schedule (); }finish_wait (sk_sleep (sk), &wait);if (!sigd) {
Signature · 1 line
1.9.3
Re: [PATCH] atm/svc: Fix blocking in wait loop
From: chas williams - CONTRACTOR <[email protected]>
Date: Tue, 12 Aug 2014 08:12:26 -0400
Quoted 9 lines
One should not call blocking primitives inside a wait loop, since both require
task_struct::stateto sleep, so the inner will destroy the outer state.
sigd_enq()will possibly sleep foralloc_skb(). Movesigd_enq()beforeprepare_to_wait()to avoid sleeping while waiting interruptibly. You do not actually need to callsigd_enq()after the initialprepare_to_wait()because we test the termination condition before callingschedule().
Quoted 4 lines
Based on suggestions from Peter Zijlstra.
Signed-off-by Chas Williams <[email protected]>Acked-by Peter Zijlstra <[email protected]>
Applied.