Skip to contentSkip to conversation

Counting currently open file descriptors per process.

DISCUSSION wmealing 1 participant 1 message Jun 11, 2014

lore
Share & tools
lore

wmealing

author
#0 lore raw

Gday,

I'm seeking some guidance on how to best (or if) to implement a feature.

Please CC me on any reply, I am not subscribed to this list.

The feature is, "an application would like to know how many files another process has open".

Quoted 1 line

From user space, the cheapest way would be to use the call

syscall(SYS_getdents ...) in the proc/pid/fd directory.

Alternatively from kernel space one could achieve a similar behavior by iterating through the tasks fdtable, as i have attempted to here:

https://gist.github.com/wmealing/c0836bc6a38f8f90aa0d

Colleagues of mine have pointed out that this may have performance impacts for tools that frequently parse /proc/pid/status.

I have compiled a kernel with the above patch and here are the performance stats.

System settings

# sysctl -w fs.file-max=5000000
fs.file-max = 5000000

Increase this sessions limits.

# ulimit -n 1000000

test.py had 500002 files open each time.

Here are some of the performance benchmarks on an idle system:

# time cat /proc/`pidof python test.py`/status |grep FD
FDSize:	524288
FDCount: 500002

real 0m0.008s
user 0m0.002s
sys 0m0.004s

# time ./test-getdents /proc/`pidof python test.py`/fd &> /dev/null 

real 0m0.631s
user 0m0.001s
sys 0m0.485s

or this time with readdir(3)

# time ./test-opendir /proc/`pidof python test.py`/fd &> /dev/null

real 0m0.129s
user 0m0.001s
sys 0m0.007s

(which oddly seems faster?)

My benchmark values above are not meant for micro-benchmarking but rather as a scale to know how far behind the code is.

Is the current method of getting a live fd count acceptable, if not how should it be done ?

Thanks for your time.

Wade Mealing.

[1] https://github.com/wmealing/live-fd-count/
git repo with code used in the above.