Add containerstats (v3)
cover Add containerstats (v3)
Changelog (v3)
1. Use the container iterator (Paul Menage)
2. delayacct_set_flag() has now been moved to delayacct_blkio_start (Andrew Morton)
Changelog (v2)
1. Instead of parsing long container path's use the dentry to match the container for which stats are required. The user space application opens the container directory and passes the file descriptor, which is used to determine the container for which stats are required. This approach was suggested by Paul Menage
This patch is inspired by the discussion at http://lkml.org/lkml/2007/4/11/187
and implements per container statistics as suggested by Andrew Morton in http://lkml.org/lkml/2007/4/11/263. The patch is on top of 2.6.21-mm1 with Paul's containers v9 patches (forward ported)
This patch implements per container statistics infrastructure and re-uses code from the taskstats interface. A new set of container operations are registered with commands and attributes. It should be very easy to *extend* per container statistics, by adding members to the containerstats structure.
The current model for containerstats is a pull, a push model (to post statistics on interesting events), should be very easy to add. Currently user space requests for statistics by passing the container file descriptor. Statistics about the state of all the tasks in the container is returned to user space.
TODO's/NOTE:
This patch provides an infrastructure for implementing container statistics. Based on the needs of each controller, we can incrementally add more statistics, event based support for notification of statistics, accumulation of taskstats into container statistics in the future.
Sample output
# ./containerstats -C /container/a sleeping 2, blocked 0, running 1, stopped 0, uninterruptible 0
# ./containerstats -C /container/ sleeping 154, blocked 0, running 0, stopped 0, uninterruptible 0
If the approach looks good, I'll enhance and post the user space utility for the same
Feedback, comments, test results are always welcome!
---
Documentation/accounting/containerstats.txt | 27 ++++++++++ include/linux/Kbuild | 1 include/linux/container.h | 8 +++ include/linux/containerstats.h | 70 ++++++++++++++++++++++++++++ include/linux/delayacct.h | 13 +++++ kernel/container.c | 54 +++++++++++++++++++++ kernel/taskstats.c | 66 ++++++++++++++++++++++++++ 7 files changed, 239 insertions(+)
Documentation/accounting/containerstats.txt new file +27 -0
@@ -0,0 +1,27 @@ +Containerstats is inspired by the discussion at +http://lkml.org/lkml/2007/4/11/187 and implements per container statistics as +suggested by Andrew Morton in http://lkml.org/lkml/2007/4/11/263. + +Per container statistics infrastructure re-uses code from the taskstats +interface. A new set of container operations are registered with commands +and attributes specific to containers. It should be very easy to +extend per container statistics, by adding members to the containerstats +structure. + +The current model for containerstats is a pull, a push model (to post +statistics on interesting events), should be very easy to add. Currently +user space requests for statistics by passing the container path. +Statistics about the state of all the tasks in the container is returned to +user space. + +NOTE: We currently rely on delay accounting for extracting information +about tasks blocked on I/O. If CONFIG_TASK_DELAY_ACCT is disabled, this +information will not be available. + +To extract container statistics a utility very similar to getdelays.c +has been developed, the sample output of the utility is shown below + +~/balbir/containerstats # ./getdelays -C "/container/a" +sleeping 1, blocked 0, running 1, stopped 0, uninterruptible 0 +~/balbir/containerstats # ./getdelays -C "/container" +sleeping 155, blocked 0, running 1, stopped 0, uninterruptible 2
include/linux/container.h renamed from include/linux/container.h~containers-taskstats +8 -0
@@ -13,6 +13,7 @@#include <linux/cpumask.h> #include <linux/nodemask.h> #include <linux/rcupdate.h> +#include <linux/containerstats.h> #ifdef CONFIG_CONTAINERS
@@ -27,6 +28,8 @@ extern void container_unlock(void);extern void container_fork (struct task_struct *p);extern void container_fork_callbacks (struct task_struct *p);extern void container_exit (struct task_struct *p,int run_callbacks); +extern int containerstats_build (struct containerstats *stats, +struct dentry *dentry);extern struct file_operations proc_container_operations;
@@ -288,6 +291,11 @@ static inline void container_exit(structstatic inline void container_lock (void ) {}static inline void container_unlock (void ) {} +static inline int containerstats_build (struct containerstats *stats, +struct dentry *dentry) +{ +return -EINVAL ; +}#endif /* !CONFIG_CONTAINERS */
include/linux/containerstats.h new file +70 -0
70 lines not on this page · raw message
include/linux/delayacct.h renamed from include/linux/delayacct.h~containers-taskstats +13 -0
@@ -26,6 +26,7 @@* Used to set current->delays->flags */ #define DELAYACCT_PF_SWAPIN 0x00000001/* I am doing a swapin */ +#define DELAYACCT_PF_BLKIO 0x00000002/* I am waiting on IO */ #ifdef CONFIG_TASK_DELAY_ACCT
@@ -39,6 +40,14 @@ extern void __delayacct_blkio_end(void);extern int __delayacct_add_tsk (struct taskstats *,struct task_struct *);extern __u64 __delayacct_blkio_ticks (struct task_struct *); +static inline int delayacct_is_task_waiting_on_io (struct task_struct *p) +{ +if (p->delays) +return (p->delays->flags &DELAYACCT_PF_BLKIO ); +else +return 0 ; +} +static inline void delayacct_set_flag (int flag) {if (current->delays)
@@ -71,6 +80,7 @@ static inline void delayacct_tsk_free(ststatic inline void delayacct_blkio_start (void ) { +delayacct_set_flag (DELAYACCT_PF_BLKIO );if (current->delays)__delayacct_blkio_start (); }
@@ -79,6 +89,7 @@ static inline void delayacct_blkio_end(v {if (current->delays)__delayacct_blkio_end (); +delayacct_clear_flag (DELAYACCT_PF_BLKIO ); } static inline int delayacct_add_tsk(struct taskstats *d,
@@ -116,6 +127,8 @@ static inline int delayacct_add_tsk(stru {return 0 ; }static inline __u64 delayacct_blkio_ticks (struct task_struct *tsk) {return 0 ; } +static inline int delayacct_is_task_waiting_on_io (struct task_struct *p) +{return 0 ; }#endif /* CONFIG_TASK_DELAY_ACCT */ #endif
include/linux/Kbuild renamed from include/linux/Kbuild~containers-taskstats +1 -0
@@ -47,6 +47,7 @@ header-y += coff.h header-y += comstats.h header-y += consolemap.h header-y += const.h +header-y += containerstats.h header-y += cycx_cfm.h header-y += dlm_device.h header-y += dlm_netlink.h
kernel/container.c renamed from kernel/container.c~containers-taskstats +54 -0
66 lines not on this page · raw message
diff -puN kernel/sched.c~containers-taskstats kernel/sched.c
kernel/taskstats.c renamed from kernel/taskstats.c~containers-taskstats +66 -0
98 lines not on this page · raw message
diff -puN kernel/cpuset.c~containers-taskstats kernel/cpuset.c diff -puN kernel/delayacct.c~containers-taskstats kernel/delayacct.c _
Signature · 4 lines
Warm Regards, Balbir Singh Linux Technology Center IBM, ISTL
PATCH 1/1 Update getdelays to become containerstats aware
Update the getdelays utility to become containerstats aware. A new -C option has been added. It takes in a container path and prints out a summary of task states in the container.
---
Documentation/accounting/getdelays.c | 43 ++++++++++++++++++++++++++++++++++- 1 file changed, 42 insertions(+), 1 deletion(-)
Documentation/accounting/getdelays.c renamed from Documentation/accounting/getdelays.c~containerstats-update-getdelays +42 -1
@@ -26,6 +26,7 @@#include <linux/genetlink.h> #include <linux/taskstats.h> +#include <linux/containerstats.h> /* * Generic macros for dealing with netlink sockets. Might be duplicated
@@ -79,6 +80,7 @@ static void usage(void)fprintf (stderr," -i: print IO accounting (works only with -p)\n" );fprintf (stderr," -l: listen forever\n" );fprintf (stderr," -v: debug on\n" ); +fprintf (stderr," -C: container path\n" ); }/*
@@ -213,6 +215,14 @@ void task_context_switch_counts(struct t t->nvcsw, t->nivcsw); } +void print_containerstats (struct containerstats *c) +{ +printf ("sleeping %llu, blocked %llu, running %llu, stopped %llu, " +"uninterruptible %llu\n" , c->nr_sleeping, c->nr_io_wait, + c->nr_running, c->nr_stopped, c->nr_uninterruptible); +} + +void print_ioacct (struct taskstats *t) { printf("%s: read=%llu, write=%llu, cancelled_write=%llu\n",
@@ -240,11 +250,14 @@ int main(int argc, char *argv[])int maskset =0 ;char *logfile =NULL ;int loop =0 ; +int containerset =0 ; +char containerpath[1024 ]; +int cfd =0 ;struct msgtemplate msg;while (1 ) { - c =getopt (argc, argv,"qdiw:r:m:t:p:vl" ); + c =getopt (argc, argv,"qdiw:r:m:t:p:vlC:" );if (c <0 )break ;
@@ -261,6 +274,10 @@ int main(int argc, char *argv[])printf ("printing task/process context switch rates\n" ); print_task_context_switch_counts =1 ;break ; +case 'C' : + containerset =1 ; +strncpy (containerpath, optarg,strlen (optarg) +1 ); +break ;case 'w' : logfile =strdup (optarg);printf ("write to file %s\n" , logfile);
@@ -335,6 +352,11 @@ int main(int argc, char *argv[]) } } +if (tid && containerset) { +fprintf (stderr,"Select either -t or -C, not both\n" ); + goto err; + } +if (tid) { rc =send_cmd (nl_sd, id, mypid,TASKSTATS_CMD_GET , cmd_type, &tid,sizeof (__u32));
@@ -345,6 +367,20 @@ int main(int argc, char *argv[]) } } +if (containerset) { + cfd =open (containerpath,O_RDONLY ); +if (cfd <0 ) { +perror ("error opening container file" ); + goto err; + } + rc =send_cmd (nl_sd, id, mypid,CONTAINERSTATS_CMD_GET , +CONTAINERSTATS_CMD_ATTR_FD , &cfd,sizeof (__u32)); +if (rc <0 ) { +perror ("error sending containerstats command" ); + goto err; + } + } + do { int i;
@@ -423,6 +459,9 @@ int main(int argc, char *argv[]) }break ; +case CONTAINERSTATS_TYPE_CONTAINER_STATS : +print_containerstats (NLA_DATA (na)); +break ;default :fprintf (stderr,"Unknown nla_type %d\n" , na->nla_type);
@@ -444,5 +483,7 @@ err:close (nl_sd);if (fd)close (fd); +if (cfd) +close (cfd);return 0 ; }
_
Signature · 4 lines
Warm Regards, Balbir Singh Linux Technology Center IBM, ISTL
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 11 lines · Andrew Morton
Quoted 2 lines · On Fri, 08 Jun 2007 23:43:46 +0530 Balbir Singh <[email protected]>…
This patch implements per container statistics infrastructure and re-uses code from the taskstats interface.
boggle.
Symbol: CONTAINERS [=y]
Selected by:CONTAINER_DEBUG|| CPUSETS && SMP ||CONTAINER_CPUACCT
Quoted 6 lines
Paul, that's just bizarre. How come it was done this way?
<struggles for a while, works out how to make
CONFIG_CONTAINERSgo away>OK, so
taskstats.cstill compiles, but I'm surprised. Shouldn't all that newly-added container stuff intaskstats.cbe insideCONFIG_CONTAINERS?
Hi, Andrew,
I've added a definition of containerstats_build() even when CONFIG_CONTAINERS is turned off, it simply returns -EINVAL. That's why taskstats.c compiles, I could add an #ifdef and move the containers commands under CONFIG_CONTAINERS, but I felt taskstats.c could be clean without any #ifdef's hanging around (as much as possible).
In the next iteration, I could move out all the code to containerstats.c and make containerstats.c depend on CONFIG_CONTAINERS
Signature · 4 lines
Warm Regards, Balbir Singh Linux Technology Center IBM, ISTL
Quoted 6 lines · Andrew Morton
Quoted 2 lines · On Fri, 08 Jun 2007 23:43:46 +0530 Balbir Singh <[email protected]>…
This patch implements per container statistics infrastructure and re-uses code from the taskstats interface.
Quoted 6 lines
boggle.
Symbol: CONTAINERS [=y]
Selected by:CONTAINER_DEBUG|| CPUSETS && SMP ||CONTAINER_CPUACCTPaul, that's just bizarre. How come it was done this way?
Containers on their own without any subsystems aren't hugely useful. So the plan was that any container subsystem would "select CONTAINERS" to cause the container framework to be built. I'm happy to change it to invert the dependencies if you'd prefer.
Paul
Quoted 12 lines · On Sat, 9 Jun 2007 00:09:55 -0700 "Paul Menage" <[email protected]> wrote:
Quoted 11 lines · Andrew Morton
Quoted 2 lines · On Fri, 08 Jun 2007 23:43:46 +0530 Balbir Singh <[email protected]>…
This patch implements per container statistics infrastructure and re-uses code from the taskstats interface.
boggle.
Symbol: CONTAINERS [=y]
Selected by:CONTAINER_DEBUG|| CPUSETS && SMP ||CONTAINER_CPUACCT
Quoted 7 lines
Quoted 1 line
Paul, that's just bizarre. How come it was done this way?
Containers on their own without any subsystems aren't hugely useful. So the plan was that any container subsystem would "select CONTAINERS" to cause the container framework to be built. I'm happy to change it to invert the dependencies if you'd prefer.
- CONTAINER_DEBUG should depend on CONTAINERS
- the CPUSETS && SMP is weird and should be deleted, unless I'm missing something
- CONTAINERS should depend on CPUSETS
- That leaves CONTAINER_CPUACCT. One _could_ `select' CONTAINERS if CONTAINER_CPUACCT but I suspect that'll screw up the above (select does odd things). I think it's reasonable to have a CONTAINERS menu and under that, all the clients of it: CONTAINER_CPUACCT, CONTAINER_MEMORY_CONTROL, etc.
Quoted 2 lines · Andrew Morton
-
CONTAINER_DEBUGshould depend on CONTAINERS
CONTAINER_DEBUG is actually a container subsystem whose sole purpose is to provide debugging information about any hierarchy that it's mounted as a part of. So in some senses it's in the same boat as something like cpusets or the RSS controller. CONFIG_CONTAINER_DEBUG doesn't affect any of the container framework code.
Quoted 3 lines
- the CPUSETS && SMP is weird and should be deleted, unless I'm missing something
Cpusets depends on SMP in the vanilla tree, so that's not anything new that I added.
Quoted 2 lines
- CONTAINERS should depend on CPUSETS
You mean the other way around?
Quoted 2 lines
- That leaves
CONTAINER_CPUACCT.
Really, CONTAINER_CPUACCT should have the same relationship to CONTAINERS as CPUSETS does.
Paul
Quoted 9 lines · On Sat, 9 Jun 2007 01:07:53 -0700 "Paul Menage" <[email protected]> wrote:
Quoted 2 lines · Andrew Morton
-
CONTAINER_DEBUGshould depend on CONTAINERS
CONTAINER_DEBUGis actually a container subsystem whose sole purpose is to provide debugging information about any hierarchy that it's mounted as a part of. So in some senses it's in the same boat as something like cpusets or the RSS controller.CONFIG_CONTAINER_DEBUGdoesn't affect any of the container framework code.
Oh, that's right.
But it still should depend on CONTAINERS
Quoted 6 lines
Quoted 3 lines
- the CPUSETS && SMP is weird and should be deleted, unless I'm missing something
Cpusets depends on SMP in the vanilla tree, so that's not anything new that I added.
Oh, OK, so CPUSETS is nor a client of CONTAINERS: so it depneds on CONTAINERS
Quoted 4 lines
Quoted 2 lines
- CONTAINERS should depend on CPUSETS
You mean the other way around?
yup
Quoted 6 lines
Quoted 2 lines
- That leaves
CONTAINER_CPUACCT.Really,
CONTAINER_CPUACCTshould have the same relationship to CONTAINERS as CPUSETS does.
Would it not be simplest to have CONTAINERS as the top-level user-configurable item and to then have everything else depend on it?
select is a nasty thing - we repeatedly have problems when using it.
Quoted 4 lines · Andrew Morton
Would it not be simplest to have CONTAINERS as the top-level user-configurable item and to then have everything else depend on it?
Yes, OK - it can go that way around too. I guess my thought was that people would be more interested in enabling/disabling the subsystems themselves (cpusets, RSS, etc) than the underlying framework. But if select has problems then it's no trouble to invert it.
Paul