filp usage when cpu busy
Hi,
With 2.6.16 x86_64 on a 4 core machine, I noticed that the filp usage (according to /proc/slabinfo) shoots up and keeps on increasing sharply when one of the CPUs is (1) locked up, or (2) very busy doing a lot of printk()'s with KERN_EMERG.
In the case of (1), it's permanent until it runs out of memory eventually. For (2), it's temporary; filp count comes back down when the printk()'s are done.
I can't think of any relationship between a busy/ locked-up CPU and filp count. The system is still functional. New short-lived processes kept being created, but the overall number of processes is stable.
Does anyone know why filp count would go up like that?
Thanks,
bc
Quoted 13 lines · bc Wong (chimwong)
Hi,
With 2.6.16
x86_64on a 4 core machine, I noticed that the filp usage (according to/proc/slabinfo) shoots up and keeps on increasing sharply when one of the CPUs is (1) locked up, or (2) very busy doing a lot of printk()'s withKERN_EMERG.In the case of (1), it's permanent until it runs out of memory eventually. For (2), it's temporary; filp count comes back down when the printk()'s are done.
Quoted 8 lines
I can't think of any relationship between a busy/ locked-up CPU and filp count. The system is still functional. New short-lived processes kept being created, but the overall number of processes is stable.
Does anyone know why filp count would go up like that?
Yeah, it's probably because filp structures are freed by RCU, and if you have a locked up CPU then it can't go through a quiescent state so RCU stops freeing your filps.
If you add some cond_resched()s to your code, you should find that RCU will force a reschedule and things will work (actually, for 2.6.16, I'm not sure if RCU had the code to force a reschedule... it's force_quiescent_state() in kernel/rcupdate.c upstream).
Quoted 27 lines · Nick Piggin
Quoted 13 lines · bc Wong (chimwong)
Hi,
With 2.6.16
x86_64on a 4 core machine, I noticed that the filp usage (according to/proc/slabinfo) shoots up and keeps on increasing sharply when one of the CPUs is (1) locked up, or (2) very busy doing a lot of printk()'s withKERN_EMERG.In the case of (1), it's permanent until it runs out of memory eventually. For (2), it's temporary; filp count comes back down when the printk()'s are done.
Quoted 8 lines
I can't think of any relationship between a busy/ locked-up CPU and filp count. The system is still functional. New short-lived processes kept being created, but the overall number of processes is stable.
Does anyone know why filp count would go up like that?
Yeah, it's probably because filp structures are freed by RCU, and if you have a locked up CPU then it can't go through a quiescent state so RCU stops freeing your filps.
Quoted 5 lines
If you add some cond_resched()s to your code, you should find that RCU will force a reschedule and things will work (actually, for 2.6.16, I'm not sure if RCU had the code to force a reschedule... it's
force_quiescent_state()inkernel/rcupdate.cupstream).
Thanks! You're absolutely right. Btw, 2.6.16 does
have force_quiescent_state().
Cheers,
bc