kernel hang in inet_init (PREEMPT_RCU problem)
Subsystems (2)
A message that is not in the archive (sent off-list or not yet received): [email protected]/
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 1 line · On Thu, Feb 28, 2008 at 12:53:03PM -0800, [email protected]…
...
Quoted 2 lines
------- Comment #17 from [email protected] 2008-02-28 12:53 ------- bingo with classic rcu
Congratulations Yannick!
So, it seems mainly RCU vs. timer(?) problem. I send CC of this message to RCU author, linux-kernel and netdev lists.
I'm not sure how much time you could/will to offer for this debugging. Anyway, if it's possible I think it would be interesting to check with CONFIG_PREEMPT_RCU = y again, but CONFIG_HOTPLUG_CPU = n ("Processor type and features --> Support for suspend on SMP and ...") in the meantime.
Many thanks,
Jarek P.
Quoted 29 lines
[ 0.770854] NET: Registered protocol family 2 [ 0.770868] 4294668327 net/ipv4/af_inet.c 1387 [ 0.770884] 4294668327 net/ipv4/af_inet.c 1390 [ 0.770888] 4294668327 net/ipv4/af_inet.c 1393 [ 0.770891] 4294668327 net/ipv4/af_inet.c 1402 [ 0.770893] 4294668327 net/ipv4/af_inet.c 1407 [ 0.773078] 0 4294668328l [ 0.774036] 0 4294668329l [ 0.775037] 0 4294668330l [ 0.775040] 4294668330 net/ipv4/af_inet.c 1419 [ 0.775077] 4294668330 net/ipv4/af_inet.c 1426 [ 0.775077] 4294668330 net/ipv4/route.c 2998 [ 0.775077] 4294668330 net/ipv4/route.c 3009 [ 0.775077] 4294668330 net/ipv4/route.c 3016 [ 0.775077] IP route cache hash table entries: 32768 (order: 5, 131072 bytes) [ 0.775077] 4294668330 net/ipv4/route.c 3027 [ 0.775492] TCP established hash table entries: 131072 (order: 8, 1048576 bytes) [ 0.776318] TCP bind hash table entries: 65536 (order: 9, 2359296 bytes) [ 0.785670] TCP: Hash tables configured (established 131072 bind 65536) [ 0.785715] TCP reno registeredSignature · 4 lines
Configure bugmail: http://bugzilla.kernel.org/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is. You are the assignee for the bug, or are watching the assignee.
Quoted 10 lines · Jarek Poplawski
Quoted 1 line · On Thu, Feb 28, 2008 at 12:53:03PM -0800, [email protected]…
...
Quoted 2 lines
------- Comment #17 from [email protected] 2008-02-28 12:53 ------- bingo with classic rcu
Congratulations Yannick!
So, it seems mainly RCU vs. timer(?) problem. I send CC of this message to RCU author, linux-kernel and netdev lists.
Interesting! Any chance of a pointer to the original messages? A quick Google search and a look at LKML showed me nothing. :-/
Thanx, Paul
Quoted 39 lines
I'm not sure how much time you could/will to offer for this debugging. Anyway, if it's possible I think it would be interesting to check with
CONFIG_PREEMPT_RCU= y again, butCONFIG_HOTPLUG_CPU= n ("Processor type and features --> Support for suspend on SMP and ...") in the meantime.Many thanks,
Jarek P.Quoted 29 lines
[ 0.770854] NET: Registered protocol family 2 [ 0.770868] 4294668327 net/ipv4/af_inet.c 1387 [ 0.770884] 4294668327 net/ipv4/af_inet.c 1390 [ 0.770888] 4294668327 net/ipv4/af_inet.c 1393 [ 0.770891] 4294668327 net/ipv4/af_inet.c 1402 [ 0.770893] 4294668327 net/ipv4/af_inet.c 1407 [ 0.773078] 0 4294668328l [ 0.774036] 0 4294668329l [ 0.775037] 0 4294668330l [ 0.775040] 4294668330 net/ipv4/af_inet.c 1419 [ 0.775077] 4294668330 net/ipv4/af_inet.c 1426 [ 0.775077] 4294668330 net/ipv4/route.c 2998 [ 0.775077] 4294668330 net/ipv4/route.c 3009 [ 0.775077] 4294668330 net/ipv4/route.c 3016 [ 0.775077] IP route cache hash table entries: 32768 (order: 5, 131072 bytes) [ 0.775077] 4294668330 net/ipv4/route.c 3027 [ 0.775492] TCP established hash table entries: 131072 (order: 8, 1048576 bytes) [ 0.776318] TCP bind hash table entries: 65536 (order: 9, 2359296 bytes) [ 0.785670] TCP: Hash tables configured (established 131072 bind 65536) [ 0.785715] TCP reno registeredSignature · 4 lines
Configure bugmail: http://bugzilla.kernel.org/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is. You are the assignee for the bug, or are watching the assignee.
Quoted 7 lines · Paul E. McKenney
Quoted 6 lines · Jarek Poplawski
Quoted 1 line · On Thu, Feb 28, 2008 at 12:53:03PM -0800, [email protected]…
...
Quoted 2 lines
------- Comment #17 from [email protected] 2008-02-28 12:53 ------- bingo with classic rcu
Quoted 7 lines
Quoted 4 lines
Congratulations Yannick!
So, it seems mainly RCU vs. timer(?) problem. I send CC of this message to RCU author, linux-kernel and netdev lists.
Interesting! Any chance of a pointer to the original messages? A quick Google search and a look at LKML showed me nothing. :-/
Thanks to Raphael for pointing me at the above bugzilla URL!
Yannick,
Would you be willing to try your test again after applying the patch in http://lkml.org/lkml/2008/2/2/11 and rebuilding your kernel?
This patch does not appear to be in 2.6.25-rc3. :-(
I am reposting this patch separately.
Thanx, Paul
(In reply to comment #20)
Quoted 11 lines
Reply-To: [email protected]
Would you be willing to try your test again after applying the patch in http://lkml.org/lkml/2008/2/2/11 and rebuilding your kernel?
This patch does not appear to be in 2.6.25-rc3. :-(
I am reposting this patch separately.
Thanx, Paul
Congratulations Paul and thanks a lot to Jarek , It works, no more hang or slow boot :
[1.061675] Calling initcall 0xc03b8e10: inet_init+0x0/0x3ba() [1.061895] NET: Registered protocol family 2 [1.339007] IP route cache hash table entries: 32768 (order: 5, 131072 bytes) [1.339470] TCP established hash table entries: 131072 (order: 8, 1048576 bytes) [1.340375] TCP bind hash table entries: 65536 (order: 9, 2359296 bytes) [1.349809] TCP: Hash tables configured (established 131072 bind 65536) [1.349906] TCP reno registered [1.098514] initcall 0xc03b8e10: inet_init+0x0/0x3ba() returned 0. [1.098604] initcall 0xc03b8e10 ran for 32 msecs: inet_init+0x0/0x3ba()
Yannick
Quoted 15 lines · Yannick
(In reply to comment #20)
Quoted 11 lines
Reply-To: [email protected]
Would you be willing to try your test again after applying the patch in http://lkml.org/lkml/2008/2/2/11 and rebuilding your kernel?
This patch does not appear to be in 2.6.25-rc3. :-(
I am reposting this patch separately.
Thanx, Paul
Congratulations Paul and thanks a lot to Jarek , It works, no more hang or slow boot :
Quoted 10 lines
[1.061675] Calling initcall 0xc03b8e10: inet_init+0x0/0x3ba() [1.061895] NET: Registered protocol family 2 [1.339007] IP route cache hash table entries: 32768 (order: 5, 131072 bytes) [1.339470] TCP established hash table entries: 131072 (order: 8, 1048576 bytes) [1.340375] TCP bind hash table entries: 65536 (order: 9, 2359296 bytes) [1.349809] TCP: Hash tables configured (established 131072 bind 65536) [1.349906] TCP reno registered [1.098514] initcall 0xc03b8e10: inet_init+0x0/0x3ba() returned 0. [1.098604] initcall 0xc03b8e10 ran for 32 msecs: inet_init+0x0/0x3ba()
Thank you very much for testing this, Yannick!
Andrew and Ingo have this patch on track to merge into a later -rc of 2.6.25, so we will get there. ;-)
Thanx, Paul