Skip to contentSkip to conversation

RFC: /proc/<pid>/sched should contain cumulative data for all threads in process

DISCUSSION Chris Friesen 3 participants 6 messages Feb 7, 2011 – Feb 9, 2011

lore
Share & tools
lore
#0 lore raw

Hi,

We've got a tool that gathers lots of scheduling data for each process (not task/thread) on the system.

For /proc/<pid>/{stat,io} this is straightforward, as the per-thread values are summed together for the process as a whole.

However, /proc/<pid>/sched only shows the data for the individual thread with the same tid as the pid. To get a per-process view we need to manually scan all the threads and sum them--and this can get expensive due to all the extra file operations, parsing, etc.

Was this a concious design decision, or just an oversight? Would a patch converting it to whole-process values be accepted or is it enough of a standard interface that we can't break existing apps that expect the current behaviour?

Thanks,
Chris

Signature · 5 lines
Chris Friesen
Software Developer
GENBAND
[email protected]
www.genband.com
#1 lore raw
Quoted 13 lines · Chris Friesen

Hi,

We've got a tool that gathers lots of scheduling data for each process (not task/thread) on the system.

For /proc/<pid>/{stat,io} this is straightforward, as the per-thread values are summed together for the process as a whole.

However, /proc/<pid>/sched only shows the data for the individual thread with the same tid as the pid. To get a per-process view we need to manually scan all the threads and sum them--and this can get expensive due to all the extra file operations, parsing, etc.

Quoted 4 lines

Was this a concious design decision, or just an oversight? Would a patch converting it to whole-process values be accepted or is it enough of a standard interface that we can't break existing apps that expect the current behaviour?

I'd as soon remove all that stuff than extend it, its an abi liability, esp since you're talking about tools parsing this stuff.

#2 lore raw
Quoted 14 lines · Peter Zijlstra
Quoted 13 lines · Chris Friesen

Hi,

We've got a tool that gathers lots of scheduling data for each process (not task/thread) on the system.

For /proc/<pid>/{stat,io} this is straightforward, as the per-thread values are summed together for the process as a whole.

However, /proc/<pid>/sched only shows the data for the individual thread with the same tid as the pid. To get a per-process view we need to manually scan all the threads and sum them--and this can get expensive due to all the extra file operations, parsing, etc.

Quoted 7 lines
Quoted 4 lines

Was this a concious design decision, or just an oversight? Would a patch converting it to whole-process values be accepted or is it enough of a standard interface that we can't break existing apps that expect the current behaviour?

I'd as soon remove all that stuff than extend it, its an abi liability, esp since you're talking about tools parsing this stuff.

So assuming a tool would want to capture such stats of the system, what would be its options? Could we do all this via system-wide counters and perf stat alike cheap, transparent gathering without having to patch/rebuild the kernel?

Thanks,

Ingo

#3 lore raw
Quoted 21 lines · Ingo Molnar
Quoted 19 lines · Peter Zijlstra
Quoted 17 lines · Chris Friesen

Hi,

We've got a tool that gathers lots of scheduling data for each process (not task/thread) on the system.

For /proc/<pid>/{stat,io} this is straightforward, as the per-thread values are summed together for the process as a whole.

However, /proc/<pid>/sched only shows the data for the individual thread with the same tid as the pid. To get a per-process view we need to manually scan all the threads and sum them--and this can get expensive due to all the extra file operations, parsing, etc.

Was this a concious design decision, or just an oversight? Would a patch converting it to whole-process values be accepted or is it enough of a standard interface that we can't break existing apps that expect the current behaviour?

Quoted 6 lines
Quoted 2 lines

I'd as soon remove all that stuff than extend it, its an abi liability, esp since you're talking about tools parsing this stuff.

So assuming a tool would want to capture such stats of the system, what would be its options? Could we do all this via system-wide counters and perf stat alike cheap, transparent gathering without having to patch/rebuild the kernel?

Very much depends on what is wanted, but most of the stuff inside those files is very specific to the implementation and pinning any of that to an ABI is like silly.

#4 lore raw
Quoted 3 lines · Peter Zijlstra

Very much depends on what is wanted, but most of the stuff inside those files is very specific to the implementation and pinning any of that to an ABI is like silly.

Why? If the implementation changes then those values lose meaning and the most correct value to report is *zero* and that's it.

That's how such things were always done, it was never a problem in the past 15 years. Why should it start being a problem suddenly?

Thanks,

Ingo

#5 lore raw
In reply to Peter Zijlstra
Quoted 9 lines · Peter Zijlstra
Quoted 3 lines · Ingo Molnar

So assuming a tool would want to capture such stats of the system, what would be its options? Could we do all this via system-wide counters and perf stat alike cheap, transparent gathering without having to patch/rebuild the kernel?

Very much depends on what is wanted, but most of the stuff inside those files is very specific to the implementation and pinning any of that to an ABI is like silly.

Currently we're using the following fields from
/proc/<pid>/task/<tid>/sched:

sum_exec_runtime
wait_sum
wait_max
exec_max
iowait_sum
iowait_count
nr_switches

If there's a better way to get this information (with the precision available from this interface) then I'd love to hear about it.

Chris

Signature · 5 lines
Chris Friesen
Software Developer
GENBAND
[email protected]
www.genband.com