Skip to contentSkip to conversation

More issues found on kernel.org

DISCUSSION J.H. 3 participants 4 messages Oct 18, 2010 – Oct 20, 2010

lore
Share & tools
lore

J.H.

author
#0 lore raw

Not that the current discussion on IMA, and the recent problems found with XFS were enough, I've started seeing, rather regularly, what I've reported in bugzilla

https://bugzilla.kernel.org/show_bug.cgi?id=20702

It looks like a double free is happening somewhere, and the issue *SEEMS* to be limited to the dynamic web boxes (bugzilla, wiki's, etc) and those are the only boxes I have running drbd and ocfs2.

Once the initial problem hits, the box more or less grinds to a halt and will eventually kernel panic and reboot. Explicitly trying to reboot the box results in a solidly hung box requiring a hard reset.

Anyone have any thoughts, any extra debugging if/when this should happen again that would be useful?

- John 'Warthog9' Hawley

#1 lore raw
Quoted 9 lines · J.H.

Not that the current discussion on IMA, and the recent problems found with XFS were enough, I've started seeing, rather regularly, what I've reported in bugzilla

https://bugzilla.kernel.org/show_bug.cgi?id=20702

It looks like a double free is happening somewhere, and the issue *SEEMS* to be limited to the dynamic web boxes (bugzilla, wiki's, etc) and those are the only boxes I have running drbd and ocfs2.

Obviously with no ocfs2 in the stack traces, it's hard to say
anything from that perspective. Do you have any idea what file snmpd is closing?

Joel

Signature · 10 lines
"I'm so tired of being tired,
 Sure as night will follow day.
 Most things I worry about
 Never happen anyway."

Joel Becker
Consulting Software Developer
Oracle
E-mail: [email protected]
Phone: (650) 506-8127
author
#2 lore raw
Quoted 7 lines · Joel Becker
Quoted 6 lines · J.H.

Not that the current discussion on IMA, and the recent problems found with XFS were enough, I've started seeing, rather regularly, what I've reported in bugzilla

https://bugzilla.kernel.org/show_bug.cgi?id=20702

Quoted 7 lines
Quoted 3 lines

It looks like a double free is happening somewhere, and the issue *SEEMS* to be limited to the dynamic web boxes (bugzilla, wiki's, etc) and those are the only boxes I have running drbd and ocfs2.

Obviously with no ocfs2 in the stack traces, it's hard to say
anything from that perspective. Do you have any idea what file snmpd is closing?

Wasn't pointing the finger at ocfs2, or drbd for that matter, was noting that was running on the box as those are the only two boxes with it, and those are the boxes having issues right now. I'm at the point where I have no idea *WHAT* was causing the problem just trying to get as much info out there for debugging as possible.

As to what files snmpd was closing, no idea. I'm using snmpd both for monitoring of the boxes, but HP's utilities are using it for a pile of things as well, including disk monitoring and such. Could have been just about anything unfortunately and I'm not sure there's a good way to trap that if/when it happens again.

If I get to that state again is there anything that would be useful (from a debugging perspective) to snag before the box falls over, I might be able to get some sysrq requests back if anyone would find that helpful, and might be able to poke around a bit, not sure how far I can get before it becomes unusable yet.

- John 'Warthog9' Hawley

#3 lore raw
In reply to J.H.
Quoted 10 lines · J.H.

Not that the current discussion on IMA, and the recent problems found with XFS were enough, I've started seeing, rather regularly, what I've reported in bugzilla

https://bugzilla.kernel.org/show_bug.cgi?id=20702

It looks like a double free is happening somewhere, and the issue *SEEMS* to be limited to the dynamic web boxes (bugzilla, wiki's, etc) and those are the only boxes I have running drbd and ocfs2.

Quoted 6 lines

Once the initial problem hits, the box more or less grinds to a halt and will eventually kernel panic and reboot. Explicitly trying to reboot the box results in a solidly hung box requiring a hard reset.

Anyone have any thoughts, any extra debugging if/when this should happen again that would be useful?

If you are building your own kernel, I would turn on slab debugging. Nearly all the double frees I have come across in the last years have been tracked down using slab debug which traps double frees. It does not always point at the culprit, but at least narrows the field.

Robin