Skip to contentSkip to conversation

CKRM after a major overhaul

RFC PATCH In review [email protected] 10 participants 35 messages Apr 21, 2006 – Apr 28, 2006

Subsystems (5)
lore
Share & tools
lore
#0 lore raw

cover CKRM after a major overhaul

CKRM has gone through a major overhaul by removing some of the complexity, cutting down on features and moving portions to userspace.

Diffstat of this patchset (including the numtasks controller that follows) is:

	23 files changed, 2475 insertions(+), 5 deletions(-)

including Documentaion and comments.

This patchset will be followed with two controllers:
- a simple controller, numtasks to control number of tasks
- CPU controller, to control CPU resource.
--

Brief Intro for CKRM:

Class-based Kernel Resource Management (CKRM) enables control of system resource usage and monitoring of resource usage through user-defined groups of tasks called classes.

Class is a group of tasks that is grouped by the administrator.

By assigning tasks to classes, administrators can monitor and bound the resource usage of any system resource with a resource controller. Resources amenable to such control include CPU ticks, physical pages, disk I/O bandwidth, number of open file handles, and number of tasks to name a few.

Userspace interfaces with CKRM through a configfs subsystem: Resource Control File System (RCFS). Users create and delete classes simply by issuing mkdir or rmdir commands. Once created the user may set the resource share of a class and alter the group of tasks bound to those classes by writing to files in the class directory. Similarly, to monitor the subsequent resource utilization of the class, users read files in the class directory.

Users control different resource shares of a class independent of other resource. In other words, CPU share of class can be very different from memory share and that of I/O share.

Resource controllers implement a small set of functions that respond to changes in resource shares, class creation/deletion and class membership. Given a class and its shares the controller then manages resource usage of tasks in that class. For instance, a CPU resource controller might manipulate the timeslice of each task according to its class' remaining CPU share.

--

Patch Descriptions:

This set of patches implements classes, resource controller registration, and the RCFS interface. Subsequent sets of patches add specific resource controllers.

More details are available in the doumentation patch.

Patch descriptions:
01/12: ckrm_core
- Provides register/unregister functions for a controller

02/12: ckrm_core_class_support
- Provides functions to alloc and free a user defined class
- Provides utility functions to walk through the class hierarchy

03/12: ckrm_core_handle_shares
- Provides functions to set/get shares of a class
- Defines a teardown function that is intended to be called when user disables CKRM (by umount of configfs or rmmod of rcfs)

04/12: ckrm_tasksupport
- Adds logic to support adding/removing task to/from a class
- Provides an interface to set a task's class

05/12: ckrm_tasksupport_fork_exit_init
- Initializes and clears ckrm specific information at fork() and exit()
- Inititalizes ckrm (called from start_kernel)

06/12: ckrm_tasksupport_procsupport
- Adds an interface in /proc to get the class name of a task.

07/12 - ckrm_configfs_rcfs
Creates configfs interface(RCFS) for managing CKRM. Hooks up with configfs. Provides functions for creating and deleting classes.

08/12 - ckrm_configfs_rcfs_attr_support
Adds the basic attribute store and show functions.

09/12 - 04ckrm_configfs_rcfs_stats
Adds attr_store and attr_show support for stats file.

10/12 - ckrm_configfs_rcfs_shares
Adds attr_store and attr_show support for shares file.

11/12 - ckrm_configfs_rcfs_members
Adds attr_store and attr_show support for members file.

12/12 - ckrm_docs
Documentation describing important CKRM elements such as classes, shares, controllers, and the interface provided to userspace via RCFS

Signature · 4 lines
----------------------------------------------------------------------
    Chandra Seetharaman               | Be careful what you choose....
              - [email protected]   |      .......you may get it.
----------------------------------------------------------------------
#1 lore raw

RFC PATCH 1/12 Register/Unregister interface for Controllers

01/12 - ckrm_core

This patch defines data structures for defining a class and resource controller.
Provides register/unregister functions for a controller. Provides utility functions to get a controller's data structure. --

 include/linux/ckrm.h     |   76 +++++++++++++++++
 include/linux/ckrm_rc.h  |   67 ++++++++++++++
 init/Kconfig             |   14 +++
 kernel/Makefile          |    1 
 kernel/ckrm/Makefile     |    1 
 kernel/ckrm/ckrm.c       |  210 +++++++++++++++++++++++++++++++++++++++++++++++
 kernel/ckrm/ckrm_local.h |   13 ++
 7 files changed, 382 insertions(+)
include/linux/ckrm.h new file +76 -0
Index: linux2617-rc2/include/linux/ckrm.h
===================================================================
@@ -0,0 +1,76 @@
+/*
+ *  ckrm.h - Header file to be used by Class-based Kernel Resource
+ *  Management (CKRM).
+ *
+ * Copyright (C) Hubertus Franke, IBM Corp. 2003, 2004
+ *		(C) Shailabh Nagar,  IBM Corp. 2003, 2004
+ *		(C) Chandra Seetharaman, IBM Corp. 2003, 2004, 2005
+ *
+ * Provides data structures, macros and kernel APIs
+ *
+ * More details at http://ckrm.sf.net
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ */
+
+#ifndef _LINUX_CKRM_H
+#define _LINUX_CKRM_H
+
+#ifdef CONFIG_CKRM
+#include <linux/spinlock.h>
+#include <linux/list.h>
+#include <linux/kref.h>
+
+#define CKRM_SHARE_UNCHANGED	(-1)	/* implicitly specified by userspace,
+					 * never stored in a class' shares
+					 * struct, and never displayed */
+#define CKRM_SHARE_UNSUPPORTED	(-2)	/* If the resource controller doesn't
+					 * support user changing a shares value
+					 * it sets the corresponding share
+					 * value to UNSUPPORTED when it returns
+					 * the newly allocated shares data
+					 * structure */
+#define CKRM_SHARE_DONT_CARE	(-3)
+
+#define CKRM_SHARE_DEFAULT_DIVISOR 	(100)
+
+#define CKRM_MAX_RES_CTLRS	8	/* max # of resource controllers */
+
+#define CKRM_NO_CLASS		NULL
+#define CKRM_NO_SHARE		NULL
+#define CKRM_NO_RES_ID		CKRM_MAX_RES_CTLRS /* Invalid ID */
+
+/*
+ * Share quantities are a child's fraction of the parent's resource
+ * specified by a divisor in the parent and a dividend in the child.
+ *
+ * Shares are represented as a relative quantity between parent and child
+ * to simplify locking when propagating modifications to the shares of a
+ * class. Only the parent and the children of the modified class need to be
+ * locked.
+*/
+struct ckrm_shares {
+};
+
+/*
+ * Class is the grouping of tasks with shares of each resource that has
+ * registered a resource controller (see include/linux/ckrm_rc.h).
+ */
+struct ckrm_class {
+	int depth;		/* depth of this class. root == 0 */
+	spinlock_t class_lock;	/* protects task_list, shares and children
+				 * When grabbing class_lock in a hierarchy,
+				 * grab parent's class_lock first.
+				 * If resource controller uses a class
+				 * specific lock, grab class_lock before
+				 * grabbing resource specific lock */
+	struct ckrm_shares *shares[CKRM_MAX_RES_CTLRS];/* resource shares */
+	struct list_head class_list;	/* entry in list of all classes */
+};
+
+#endif /* CONFIG_CKRM */
+#endif /* _LINUX_CKRM_H */
include/linux/ckrm_rc.h new file +67 -0

67 lines not on this page · raw message

kernel/ckrm/ckrm.c new file +210 -0

210 lines not on this page · raw message

kernel/Makefile +1 -0
Index: linux2617-rc2/kernel/Makefile
===================================================================
@@ -38,6 +38,7 @@ obj-$(CONFIG_GENERIC_HARDIRQS) += irq/
 obj-$(CONFIG_SECCOMP) += seccomp.o
 obj-$(CONFIG_RCU_TORTURE_TEST) += rcutorture.o
 obj-$(CONFIG_RELAY) += relay.o
+obj-$(CONFIG_CKRM) += ckrm/
 
 ifneq ($(CONFIG_SCHED_NO_NO_OMIT_FRAME_POINTER),y)
 # According to Alan Modra <[email protected]>, the -fno-omit-frame-pointer is
kernel/ckrm/Makefile new file +1 -0
Index: linux2617-rc2/kernel/ckrm/Makefile
===================================================================
@@ -0,0 +1 @@
+obj-y = ckrm.o
init/Kconfig +14 -0
Index: linux2617-rc2/init/Kconfig
===================================================================
@@ -150,6 +150,20 @@ config BSD_PROCESS_ACCT_V3
 	  for processing it. A preliminary version of these tools is available
 	  at <http://www.physik3.uni-rostock.de/tim/kernel/utils/acct/>.
 
+menu "Class Based Kernel Resource Management"
+
+config CKRM
+	bool "Class Based Kernel Resource Management Core"
+	depends on EXPERIMENTAL
+	help
+	  Class-based Kernel Resource Management is a framework for controlling
+	  and monitoring resource allocation of user-defined groups of tasks.
+	  For more information, please visit http://ckrm.sf.net.
+
+	  If you say Y here, enable the Resource Class File System and at least
+	  one of the resource controllers below. Say N if you are unsure.
+
+endmenu
 config SYSCTL
 	bool "Sysctl support"
 	---help---
kernel/ckrm/ckrm_local.h new file +13 -0
Index: linux2617-rc2/kernel/ckrm/ckrm_local.h
===================================================================
@@ -0,0 +1,13 @@
+/*
+ * Contains function definitions that are local to ckrm core.
+ * NOT to be included by controllers.
+ */
+
+#include <linux/ckrm_rc.h>
+
+extern struct ckrm_controller *ckrm_get_controller_by_name(const char *);
+extern struct ckrm_controller *ckrm_get_controller_by_id(unsigned int);
+extern void ckrm_put_controller(struct ckrm_controller *);
+extern struct ckrm_class *ckrm_alloc_class(struct ckrm_class *, const char *);
+extern int ckrm_free_class(struct ckrm_class *);
+extern void ckrm_release_class(struct kref *);
Signature · 4 lines
----------------------------------------------------------------------
    Chandra Seetharaman               | Be careful what you choose....
              - [email protected]   |      .......you may get it.
----------------------------------------------------------------------
#2 lore raw

RFC PATCH 2/12 Class creation/deletion

In reply to [email protected]

02/12 - ckrm_core_class_support

Provides functions to alloc and free a user defined class. Provides utility macro to walk through the class hierarchy --

 include/linux/ckrm.h    |    8 ++
 include/linux/ckrm_rc.h |    9 ++
 kernel/ckrm/ckrm.c      |  171 ++++++++++++++++++++++++++++++++++++++++++++----
 3 files changed, 175 insertions(+), 13 deletions(-)
include/linux/ckrm.h +8 -0
Index: linux2617-rc2/include/linux/ckrm.h
===================================================================
@@ -61,6 +61,8 @@ struct ckrm_shares {
  * registered a resource controller (see include/linux/ckrm_rc.h).
  */
 struct ckrm_class {
+	const char *name;
+	struct kref ref;
 	int depth;		/* depth of this class. root == 0 */
 	spinlock_t class_lock;	/* protects task_list, shares and children
 				 * When grabbing class_lock in a hierarchy,
@@ -70,6 +72,12 @@ struct ckrm_class {
 				 * grabbing resource specific lock */
 	struct ckrm_shares *shares[CKRM_MAX_RES_CTLRS];/* resource shares */
 	struct list_head class_list;	/* entry in list of all classes */
+
+	struct list_head task_list;	/* this class's tasks */
+
+	struct ckrm_class *parent;
+	struct list_head siblings;	/* entry in list of siblings */
+	struct list_head children;	/* head of children */
 };
 
 #endif /* CONFIG_CKRM */
kernel/ckrm/ckrm.c +158 -13

228 lines not on this page · raw message

include/linux/ckrm_rc.h +9 -0
Index: linux2617-rc2/include/linux/ckrm_rc.h
===================================================================
@@ -64,4 +64,13 @@ struct ckrm_controller {
 
 extern int ckrm_register_controller(struct ckrm_controller *);
 extern int ckrm_unregister_controller(struct ckrm_controller *);
+extern struct ckrm_class ckrm_default_class;
+static inline int ckrm_is_class_root(const struct ckrm_class* class)
+{
+	return (class == &ckrm_default_class);
+}
+
+#define for_each_child(child, parent)	\
+	list_for_each_entry(child, &parent->children, siblings)
+
 #endif /* _LINUX_CKRM_RC_H */
Signature · 4 lines
----------------------------------------------------------------------
    Chandra Seetharaman               | Be careful what you choose....
              - [email protected]   |      .......you may get it.
----------------------------------------------------------------------
#3 lore raw

RFC PATCH 3/12 Share Handling

In reply to [email protected]

03/12 - ckrm_core_handle_shares

Provides functions to set/get shares of a specific resource of a class Defines a teardown function that is intended to be called when user disables CKRM (by umount of RCFS)
--

 include/linux/ckrm.h      |   14 ++
 include/linux/ckrm_rc.h   |   10 +
 kernel/ckrm/Makefile      |    2 
 kernel/ckrm/ckrm.c        |   24 ++++
 kernel/ckrm/ckrm_local.h  |    6 +
 kernel/ckrm/ckrm_shares.c |  242 ++++++++++++++++++++++++++++++++++++++++++++++
 6 files changed, 297 insertions(+), 1 deletion(-)
include/linux/ckrm.h +14 -0
Index: linux2617-rc2/include/linux/ckrm.h
===================================================================
@@ -54,6 +54,20 @@
  * locked.
 */
 struct ckrm_shares {
+	/* shares only set by userspace */
+	int min_shares; /* minimun fraction of parent's resources allowed */
+	int max_shares; /* maximum fraction of parent's resources allowed */
+	int child_shares_divisor; /* >= 1, may not be DONT_CARE */
+
+	/*
+	 * share values invisible to userspace.  adjusted when userspace
+	 * sets shares
+	 */
+	int unused_min_shares;
+		/* 0 <= unused_min_shares <= (child_shares_divisor -
+		 * 			Sum of min_shares of children)
+		 */
+	int cur_max_shares; /* max(children's max_shares). need better name */
 };
 
 /*
kernel/ckrm/ckrm_shares.c new file +242 -0

242 lines not on this page · raw message

kernel/ckrm/Makefile +1 -1
Index: linux2617-rc2/kernel/ckrm/Makefile
===================================================================
@@ -1 +1 @@
-obj-y = ckrm.o
+obj-y = ckrm.o ckrm_shares.o
kernel/ckrm/ckrm.c +24 -0
Index: linux2617-rc2/kernel/ckrm/ckrm.c
===================================================================
@@ -174,6 +174,8 @@ static void do_free_shares_struct(struct
 	if (shares == NULL)
 		return;
 
+	ckrm_set_shares_to_default(class, ctlr);
+
 	spin_lock(&class->class_lock);
 	class->shares[ctlr->ctlr_id] = NULL;
 	spin_unlock(&class->class_lock);
@@ -345,6 +347,26 @@ int ckrm_unregister_controller(struct ck
 	return remove_controller(ctlr);
 }
 
+/*
+ * Bring the state of CKRM to the initial state.
+ * Only shares of the default class need to be changed to default values.
+ * At this point no user-defined classes should exist.
+ */
+void ckrm_teardown(void)
+{
+	int i;
+	struct ckrm_controller *ctlr;
+
+	BUG_ON(ckrm_num_classes != 0);
+	for (i = 0; i < CKRM_MAX_RES_CTLRS; i++) {
+		ctlr = ckrm_get_controller_by_id(i);
+		if (ctlr) {
+			ckrm_set_shares_to_default(&ckrm_default_class, ctlr);
+			ckrm_put_controller(ctlr);
+		}
+	}
+}
+
 EXPORT_SYMBOL_GPL(ckrm_register_controller);
 EXPORT_SYMBOL_GPL(ckrm_unregister_controller);
 EXPORT_SYMBOL_GPL(ckrm_alloc_class);
@@ -353,3 +375,5 @@ EXPORT_SYMBOL_GPL(ckrm_default_class);
 EXPORT_SYMBOL_GPL(ckrm_get_controller_by_name);
 EXPORT_SYMBOL_GPL(ckrm_get_controller_by_id);
 EXPORT_SYMBOL_GPL(ckrm_put_controller);
+EXPORT_SYMBOL_GPL(ckrm_set_controller_shares);
+EXPORT_SYMBOL_GPL(ckrm_teardown);
include/linux/ckrm_rc.h +10 -0
Index: linux2617-rc2/include/linux/ckrm_rc.h
===================================================================
@@ -73,4 +73,14 @@ static inline int ckrm_is_class_root(con
 #define for_each_child(child, parent)	\
 	list_for_each_entry(child, &parent->children, siblings)
 
+/* Get controller specific shares structure for the given class */
+static inline struct ckrm_shares *ckrm_get_controller_shares(
+			struct ckrm_class *class, struct ckrm_controller *ctlr)
+{
+	if (class && ctlr)
+		return class->shares[ctlr->ctlr_id];
+	else
+		return CKRM_NO_SHARE;
+}
+
 #endif /* _LINUX_CKRM_RC_H */
kernel/ckrm/ckrm_local.h +6 -0
Index: linux2617-rc2/kernel/ckrm/ckrm_local.h
===================================================================
@@ -11,3 +11,9 @@ extern void ckrm_put_controller(struct c
 extern struct ckrm_class *ckrm_alloc_class(struct ckrm_class *, const char *);
 extern int ckrm_free_class(struct ckrm_class *);
 extern void ckrm_release_class(struct kref *);
+extern int ckrm_set_controller_shares(struct ckrm_class *,
+			struct ckrm_controller *, const struct ckrm_shares *);
+/* Set the shares for the given class and resource to default values */
+extern void ckrm_set_shares_to_default(struct ckrm_class *,
+						struct ckrm_controller *);
+extern void ckrm_teardown(void);
Signature · 4 lines
----------------------------------------------------------------------
    Chandra Seetharaman               | Be careful what you choose....
              - [email protected]   |      .......you may get it.
----------------------------------------------------------------------
#4 lore raw

RFC PATCH 4/12 Add task logic to class

In reply to [email protected]

04/12 - ckrm_tasksupport

Adds logic to support adding/removing task to/from a class Provides interface to set a task's class
--

 include/linux/sched.h    |    4 +
 kernel/ckrm/Makefile     |    2 
 kernel/ckrm/ckrm.c       |   24 +++++++
 kernel/ckrm/ckrm_local.h |    1 
 kernel/ckrm/ckrm_task.c  |  144 +++++++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 174 insertions(+), 1 deletion(-)
include/linux/sched.h +4 -0
Index: linux2617-rc2/include/linux/sched.h
===================================================================
@@ -888,6 +888,10 @@ struct task_struct {
 	 * cache last used pipe for splice
 	 */
 	struct pipe_inode_info *splice_pipe;
+#ifdef CONFIG_CKRM
+	struct ckrm_class *class;
+	struct list_head member_list; /* list of tasks in class */
+#endif /* CONFIG_CKRM */
 };
 
 static inline pid_t process_group(struct task_struct *tsk)
kernel/ckrm/ckrm_task.c new file +144 -0

144 lines not on this page · raw message

kernel/ckrm/Makefile +1 -1
Index: linux2617-rc2/kernel/ckrm/Makefile
===================================================================
@@ -1 +1 @@
-obj-y = ckrm.o ckrm_shares.o
+obj-y = ckrm.o ckrm_shares.o ckrm_task.o
kernel/ckrm/ckrm.c +24 -0
Index: linux2617-rc2/kernel/ckrm/ckrm.c
===================================================================
@@ -251,6 +251,26 @@ static int add_controller(struct ckrm_co
 	return ret;
 }
 /*
+ * Helper function to move all tasks in a class to/from the registering
+ * /unregistering resource controller.
+ *
+ * Assumes ctlr is valid and class is initialized with resource
+ * controller's shares.
+ */
+static void move_tasks(struct ckrm_class *class, struct ckrm_controller *ctlr,
+		struct ckrm_shares *from, struct ckrm_shares *to)
+{
+	struct task_struct *tsk;
+
+	if (!ctlr->move_task)
+		return;
+	spin_lock(&class->class_lock);
+	list_for_each_entry(tsk, &class->task_list, member_list)
+		ctlr->move_task(tsk, from, to);
+	spin_unlock(&class->class_lock);
+}
+
+/*
  * Interface for registering a resource controller.
  *
  * Returns the 0 on success, -errno for failure.
@@ -287,6 +307,8 @@ int ckrm_register_controller(struct ckrm
 		kref_get(&class->ref);
 		read_unlock(&ckrm_class_lock);
   		do_alloc_shares_struct(class, ctlr);
+		move_tasks(class, ctlr, CKRM_NO_SHARE,
+					class->shares[ctlr->ctlr_id]);
 		if (prev_class)
 			kref_put(&prev_class->ref, ckrm_release_class);
 		prev_class = class;
@@ -333,6 +355,8 @@ int ckrm_unregister_controller(struct ck
 	list_for_each_entry_reverse(class, &ckrm_classes, class_list) {
 		kref_get(&class->ref);
 		read_unlock(&ckrm_class_lock);
+		move_tasks(class, ctlr, class->shares[ctlr->ctlr_id],
+							CKRM_NO_SHARE);
   		do_free_shares_struct(class, ctlr);
 		if (prev_class)
 			kref_put(&prev_class->ref, ckrm_release_class);
kernel/ckrm/ckrm_local.h +1 -0
Index: linux2617-rc2/kernel/ckrm/ckrm_local.h
===================================================================
@@ -17,3 +17,4 @@ extern int ckrm_set_controller_shares(st
 extern void ckrm_set_shares_to_default(struct ckrm_class *,
 						struct ckrm_controller *);
 extern void ckrm_teardown(void);
+extern int ckrm_setclass(pid_t, struct ckrm_class *);
Signature · 4 lines
----------------------------------------------------------------------
    Chandra Seetharaman               | Be careful what you choose....
              - [email protected]   |      .......you may get it.
----------------------------------------------------------------------
#5 lore raw

RFC PATCH 5/12 Init and clear class info in task

In reply to [email protected]

05/12 - ckrm_tasksupport_fork_exit_init

Initializes and clears ckrm specific information in a task at fork() and exit().
Inititalizes ckrm (called from start_kernel)
--

 include/linux/ckrm.h     |    7 ++++++
 init/main.c              |    2 +
 kernel/ckrm/ckrm.c       |   11 +++++++++
 kernel/ckrm/ckrm_local.h |    1 
 kernel/ckrm/ckrm_task.c  |   52 +++++++++++++++++++++++++++++++++++++++++++++++
 kernel/exit.c            |    2 +
 kernel/fork.c            |    2 +
 7 files changed, 77 insertions(+)
kernel/exit.c +2 -0
Index: linux2617-rc2/kernel/exit.c
===================================================================
@@ -35,6 +35,7 @@
 #include <linux/futex.h>
 #include <linux/compat.h>
 #include <linux/pipe_fs_i.h>
+#include <linux/ckrm.h>
 
 #include <asm/uaccess.h>
 #include <asm/unistd.h>
@@ -731,6 +732,7 @@ static void exit_notify(struct task_stru
 	struct task_struct *t;
 	struct list_head ptrace_dead, *_p, *_n;
 
+	ckrm_clear_task(tsk);
 	if (signal_pending(tsk) && !(tsk->signal->flags & SIGNAL_GROUP_EXIT)
 	    && !thread_group_empty(tsk)) {
 		/*
kernel/fork.c +2 -0
Index: linux2617-rc2/kernel/fork.c
===================================================================
@@ -44,6 +44,7 @@
 #include <linux/rmap.h>
 #include <linux/acct.h>
 #include <linux/cn_proc.h>
+#include <linux/ckrm.h>
 
 #include <asm/pgtable.h>
 #include <asm/pgalloc.h>
@@ -1214,6 +1215,7 @@ static task_t *copy_process(unsigned lon
 	total_forks++;
 	spin_unlock(&current->sighand->siglock);
 	write_unlock_irq(&tasklist_lock);
+	ckrm_init_task(p);
 	proc_fork_connector(p);
 	return p;
 
init/main.c +2 -0
Index: linux2617-rc2/init/main.c
===================================================================
@@ -47,6 +47,7 @@
 #include <linux/rmap.h>
 #include <linux/mempolicy.h>
 #include <linux/key.h>
+#include <linux/ckrm.h>
 
 #include <asm/io.h>
 #include <asm/bugs.h>
@@ -541,6 +542,7 @@ asmlinkage void __init start_kernel(void
 	proc_root_init();
 #endif
 	cpuset_init();
+	ckrm_init();
 
 	check_bugs();
 
kernel/ckrm/ckrm.c +11 -0
Index: linux2617-rc2/kernel/ckrm/ckrm.c
===================================================================
@@ -231,6 +231,7 @@ int ckrm_free_class(struct ckrm_class *c
 		return -EBUSY;
 	}
 	spin_unlock(&class->class_lock);
+	ckrm_move_tasks_to_parent(class);
 	kref_put(&class->ref, ckrm_release_class);
 	return 0;
 }
@@ -391,6 +392,16 @@ void ckrm_teardown(void)
 	}
 }
 
+void ckrm_init(void)
+{
+	write_lock(&ckrm_class_lock);
+	list_add_tail(&ckrm_default_class.class_list, &ckrm_classes);
+	write_unlock(&ckrm_class_lock);
+	kref_init(&ckrm_default_class.ref);
+	init_task.class = &ckrm_default_class;
+	ckrm_init_task(&init_task);
+}
+
 EXPORT_SYMBOL_GPL(ckrm_register_controller);
 EXPORT_SYMBOL_GPL(ckrm_unregister_controller);
 EXPORT_SYMBOL_GPL(ckrm_alloc_class);
kernel/ckrm/ckrm_task.c +52 -0

56 lines not on this page · raw message

include/linux/ckrm.h +7 -0
Index: linux2617-rc2/include/linux/ckrm.h
===================================================================
@@ -94,5 +94,12 @@ struct ckrm_class {
 	struct list_head children;	/* head of children */
 };
 
+extern void ckrm_init_task(struct task_struct *);
+extern void ckrm_clear_task(struct task_struct *);
+extern void ckrm_init(void);
+#else /* CONFIG_CKRM */
+static inline void ckrm_init_task(struct task_struct *tsk) { }
+static inline void ckrm_clear_task(struct task_struct *tsk) { }
+static inline void ckrm_init(void) { }
 #endif /* CONFIG_CKRM */
 #endif /* _LINUX_CKRM_H */
kernel/ckrm/ckrm_local.h +1 -0
Index: linux2617-rc2/kernel/ckrm/ckrm_local.h
===================================================================
@@ -18,3 +18,4 @@ extern void ckrm_set_shares_to_default(s
 						struct ckrm_controller *);
 extern void ckrm_teardown(void);
 extern int ckrm_setclass(pid_t, struct ckrm_class *);
+extern void ckrm_move_tasks_to_parent(struct ckrm_class *);
Signature · 4 lines
----------------------------------------------------------------------
    Chandra Seetharaman               | Be careful what you choose....
              - [email protected]   |      .......you may get it.
----------------------------------------------------------------------
#6 lore raw

RFC PATCH 6/12 Add proc interface to get class info of task

In reply to [email protected]

06/12: ckrm_tasksupport_procsupport

Adds an interface in /proc to get the class name of a task. --

 fs/proc/base.c          |   19 +++++++++++++++++++
 include/linux/ckrm.h    |    2 ++
 kernel/ckrm/ckrm_task.c |   32 +++++++++++++++++++++++++++++++-
 3 files changed, 52 insertions(+), 1 deletion(-)
fs/proc/base.c +19 -0
Index: linux2617-rc2/fs/proc/base.c
===================================================================
@@ -70,6 +70,7 @@
 #include <linux/ptrace.h>
 #include <linux/seccomp.h>
 #include <linux/cpuset.h>
+#include <linux/ckrm.h>
 #include <linux/audit.h>
 #include <linux/poll.h>
 #include "internal.h"
@@ -115,6 +116,9 @@ enum pid_directory_inos {
 #ifdef CONFIG_CPUSETS
 	PROC_TGID_CPUSET,
 #endif
+#ifdef CONFIG_CKRM
+	PROC_TGID_CKRM_CLASS,
+#endif
 #ifdef CONFIG_SECURITY
 	PROC_TGID_ATTR,
 	PROC_TGID_ATTR_CURRENT,
@@ -156,6 +160,9 @@ enum pid_directory_inos {
 #ifdef CONFIG_CPUSETS
 	PROC_TID_CPUSET,
 #endif
+#ifdef CONFIG_CKRM
+	PROC_TID_CKRM_CLASS,
+#endif
 #ifdef CONFIG_SECURITY
 	PROC_TID_ATTR,
 	PROC_TID_ATTR_CURRENT,
@@ -219,6 +226,9 @@ static struct pid_entry tgid_base_stuff[
 #ifdef CONFIG_CPUSETS
 	E(PROC_TGID_CPUSET,    "cpuset",  S_IFREG|S_IRUGO),
 #endif
+#ifdef CONFIG_CKRM
+	E(PROC_TGID_CKRM_CLASS,"ckrm_class",S_IFREG|S_IRUGO),
+#endif
 	E(PROC_TGID_OOM_SCORE, "oom_score",S_IFREG|S_IRUGO),
 	E(PROC_TGID_OOM_ADJUST,"oom_adj", S_IFREG|S_IRUGO|S_IWUSR),
 #ifdef CONFIG_AUDITSYSCALL
@@ -261,6 +271,9 @@ static struct pid_entry tid_base_stuff[]
 #ifdef CONFIG_CPUSETS
 	E(PROC_TID_CPUSET,     "cpuset",  S_IFREG|S_IRUGO),
 #endif
+#ifdef CONFIG_CKRM
+	E(PROC_TID_CKRM_CLASS, "ckrm_class",S_IFREG|S_IRUGO),
+#endif
 	E(PROC_TID_OOM_SCORE,  "oom_score",S_IFREG|S_IRUGO),
 	E(PROC_TID_OOM_ADJUST, "oom_adj", S_IFREG|S_IRUGO|S_IWUSR),
 #ifdef CONFIG_AUDITSYSCALL
@@ -1814,6 +1827,12 @@ static struct dentry *proc_pident_lookup
 			inode->i_fop = &proc_cpuset_operations;
 			break;
 #endif
+#ifdef CONFIG_CKRM
+		case PROC_TID_CKRM_CLASS:
+		case PROC_TGID_CKRM_CLASS:
+			inode->i_fop = &proc_ckrm_class_operations;
+			break;
+#endif
 		case PROC_TID_OOM_SCORE:
 		case PROC_TGID_OOM_SCORE:
 			inode->i_fop = &proc_info_file_operations;
kernel/ckrm/ckrm_task.c +31 -1
Index: linux2617-rc2/kernel/ckrm/ckrm_task.c
===================================================================
@@ -13,7 +13,8 @@
  * (at your option) any later version.
  *
  */
-#include <linux/sched.h>
+#include <linux/seq_file.h>
+#include <linux/proc_fs.h>
 #include <linux/module.h>
 #include "ckrm_local.h"
 
@@ -193,4 +194,33 @@ next_task:
 	kref_put(&class->ref, ckrm_release_class);
 }
 
+static int proc_ckrm_class_show(struct seq_file *m, void *v)
+{
+	struct task_struct *tsk = m->private;
+	struct ckrm_class *class = tsk->class;
+
+	if (!class)
+		return -EINVAL;
+
+	kref_get(&class->ref);
+	seq_puts(m, "/");
+	if (!ckrm_is_class_root(class))
+		seq_puts(m, class->name);
+	seq_putc(m, '\n');
+	kref_put(&class->ref, ckrm_release_class);
+	return 0;
+}
+
+static int ckrm_class_open(struct inode *inode, struct file *file)
+{
+	struct task_struct *tsk = PROC_I(inode)->task;
+	return single_open(file, proc_ckrm_class_show, tsk);
+}
+
+struct file_operations proc_ckrm_class_operations = {
+	.open		= ckrm_class_open,
+	.read		= seq_read,
+	.llseek 	= seq_lseek,
+	.release	= single_release,
+};
 EXPORT_SYMBOL_GPL(ckrm_setclass);
include/linux/ckrm.h +2 -0
Index: linux2617-rc2/include/linux/ckrm.h
===================================================================
@@ -94,6 +94,8 @@ struct ckrm_class {
 	struct list_head children;	/* head of children */
 };
 
+extern struct file_operations proc_ckrm_class_operations;
+
 extern void ckrm_init_task(struct task_struct *);
 extern void ckrm_clear_task(struct task_struct *);
 extern void ckrm_init(void);
Signature · 4 lines
----------------------------------------------------------------------
    Chandra Seetharaman               | Be careful what you choose....
              - [email protected]   |      .......you may get it.
----------------------------------------------------------------------
#7 lore raw

RFC PATCH 7/12 Configfs based filesystem user interface - RCFS

In reply to [email protected]

07/12 - ckrm_configfs_rcfs

Create the filesystem(RCFS) for managing CKRM. Hooks up with configfs. Provides functions for creating and deleting classes.
--

 init/Kconfig            |   12 +++
 kernel/ckrm/Makefile    |    1 
 kernel/ckrm/ckrm_rcfs.c |  160 ++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 173 insertions(+)
init/Kconfig +12 -0
Index: linux2617-rc2/init/Kconfig
===================================================================
@@ -163,6 +163,18 @@ config CKRM
 	  If you say Y here, enable the Resource Class File System and at least
 	  one of the resource controllers below. Say N if you are unsure.
 
+config CKRM_RCFS
+	tristate "Resource Control File System (User API for CKRM)"
+	depends on CKRM
+	select CONFIGFS_FS
+	default m
+	help
+	  RCFS is the filesystem API for CKRM. Compiling it as a module permits
+	  users to only load RCFS if they intend to use CKRM.
+
+	  Say M if unsure, Y to save on module loading. N doesn't make sense
+	  when CKRM has been configured.
+
 endmenu
 config SYSCTL
 	bool "Sysctl support"
kernel/ckrm/ckrm_rcfs.c new file +160 -0

160 lines not on this page · raw message

kernel/ckrm/Makefile +1 -0
Index: linux2617-rc2/kernel/ckrm/Makefile
===================================================================
@@ -1 +1,2 @@
 obj-y = ckrm.o ckrm_shares.o ckrm_task.o
+obj-$(CONFIG_CKRM_RCFS) += ckrm_rcfs.o
Signature · 4 lines
----------------------------------------------------------------------
    Chandra Seetharaman               | Be careful what you choose....
              - [email protected]   |      .......you may get it.
----------------------------------------------------------------------
#8 lore raw

RFC PATCH 8/12 Add attribute support to RCFS

In reply to [email protected]

08/12 - ckrm_configfs_rcfs_attr_support

Adds the basic attribute store and show functions. --

 kernel/ckrm/ckrm_rcfs.c |   51 +++++++++++++++++++++++++++++++++++++++++++++++-
 1 files changed, 50 insertions(+), 1 deletion(-)
kernel/ckrm/ckrm_rcfs.c +50 -1
Index: linux2617-rc2/kernel/ckrm/ckrm_rcfs.c
===================================================================
@@ -14,13 +14,21 @@
  * as published by the Free Software Foundation.
  *
  */
+#include <linux/ctype.h>
 #include <linux/module.h>
 #include <linux/configfs.h>
+#include <linux/parser.h>
 #include "ckrm_local.h"
 
 static struct configfs_subsystem rcfs_subsys;
 static struct config_item_type rcfs_class_type;
 
+struct class_attribute {
+	struct configfs_attribute configfs_attr;
+	ssize_t (*show)(struct ckrm_class *, char *);
+	int (*store)(struct ckrm_class *, const char *);
+};
+
 struct rcfs_class {
 	char *name;
 	struct ckrm_class *core;
@@ -56,6 +64,40 @@ static inline struct ckrm_class *item_to
 	return group_to_ckrm_class(to_config_group(item));
 }
 
+static ssize_t rcfs_attr_show(struct config_item *item,
+	      		      struct configfs_attribute *attr, char *buf)
+{
+	struct class_attribute *class_attr;
+	struct ckrm_class *class = item_to_ckrm_class(item);
+
+	class_attr = container_of(attr, struct class_attribute, configfs_attr);
+	return class_attr->show(class, buf);
+}
+
+static ssize_t rcfs_attr_store(struct config_item *item,
+			       struct configfs_attribute *attr, const char *buf,
+			       size_t count)
+{
+	char *filtered_buf, *p;
+	ssize_t rc;
+	struct class_attribute *class_attr;
+	struct ckrm_class *class = item_to_ckrm_class(item);
+
+	class_attr = container_of(attr, struct class_attribute, configfs_attr);
+	filtered_buf = kzalloc(count + 1, GFP_KERNEL);
+	if (!filtered_buf)
+		return -ENOMEM;
+	strncpy(filtered_buf, buf, count);
+	for (p = filtered_buf; isprint(*p); ++p)
+		;
+	*p = '\0';
+	rc = class_attr->store(class, filtered_buf);
+	kfree(filtered_buf);
+	if (rc)
+		return rc;
+	return count;
+}
+
 /*
  * This is the function that is called when a 'mkdir' command
  * is issued under our filesystem
@@ -117,17 +159,24 @@ static void rcfs_class_release_item(stru
 }
 
 static struct configfs_item_operations rcfs_class_item_ops = {
-	.release	= rcfs_class_release_item,
+	.release		= rcfs_class_release_item,
+	.show_attribute		= rcfs_attr_show,
+	.store_attribute	= rcfs_attr_store,
 };
 
 static struct configfs_group_operations rcfs_class_group_ops = {
 	.make_group     = make_rcfs_class,
 };
 
+static struct configfs_attribute *class_attrs[] = {
+	NULL
+};
+
 static struct config_item_type rcfs_class_type = {
 	.ct_owner	= THIS_MODULE,
 	.ct_item_ops    = &rcfs_class_item_ops,
 	.ct_group_ops	= &rcfs_class_group_ops,
+	.ct_attrs       = class_attrs
 };
 
 static struct configfs_subsystem rcfs_subsys = {
Signature · 4 lines
----------------------------------------------------------------------
    Chandra Seetharaman               | Be careful what you choose....
              - [email protected]   |      .......you may get it.
----------------------------------------------------------------------
#9 lore raw

RFC PATCH 9/12 Add stats file support to RCFS

In reply to [email protected]

09/12 - ckrm_configfs_rcfs_stats

Adds attr_store and attr_show support for stats file. --

 kernel/ckrm/ckrm_rcfs.c |  114 +++++++++++++++++++++++++++++++++++++++++++++++-
 1 files changed, 112 insertions(+), 2 deletions(-)
kernel/ckrm/ckrm_rcfs.c +112 -2
Index: linux2617-rc2/kernel/ckrm/ckrm_rcfs.c
===================================================================
@@ -20,8 +20,104 @@
 #include <linux/parser.h>
 #include "ckrm_local.h"
 
-static struct configfs_subsystem rcfs_subsys;
-static struct config_item_type rcfs_class_type;
+#define CKRM_NAME_LEN 20
+
+#define RES_STRING "res"
+
+static ssize_t show_stats(struct ckrm_class *class, char *buf)
+{
+	int i, j = 0, rc = 0;
+	size_t buf_size = PAGE_SIZE-1; /* allow only PAGE_SIZE # of bytes */
+	struct ckrm_controller *ctlr;
+	struct ckrm_shares *shares;
+
+	for (i = 0; i < CKRM_MAX_RES_CTLRS; i++, j = 0) {
+		if (buf_size <= 0)
+			break;
+		ctlr = ckrm_get_controller_by_id(i);
+		if (!ctlr)
+			 continue;
+		shares = ckrm_get_controller_shares(class, ctlr);
+		if (shares && ctlr->show_stats)
+			j = ctlr->show_stats(shares, buf, buf_size);
+		ckrm_put_controller(ctlr);
+		rc += j;
+		buf += j;
+		buf_size -= j;
+	}
+	if (i < CKRM_MAX_RES_CTLRS)
+		rc = -ENOSPC;
+	return rc;
+}
+
+enum parse_token_t {
+	parse_res_type, parse_err
+};
+
+static match_table_t parse_tokens = {
+	{parse_res_type, RES_STRING"=%s"},
+	{parse_err, NULL}
+};
+
+static int ckrm_stats_parse(const char *options,
+				char **resname, char **remaining_line)
+{
+	char *p, *str;
+	int rc = -EINVAL;
+
+	if (!options)
+		return -EINVAL;
+
+	while ((p = strsep((char **)&options, ",")) != NULL) {
+		substring_t args[MAX_OPT_ARGS];
+		int token;
+
+		if (!*p)
+			continue;
+		token = match_token(p, parse_tokens, args);
+		if (token == parse_res_type) {
+			*resname = match_strdup(args);
+			str = p + strlen(p) + 1;
+			*remaining_line = kmalloc(strlen(str) + 1, GFP_KERNEL);
+			if (*remaining_line == NULL) {
+				kfree(*resname);
+				*resname = NULL;
+				rc = -ENOMEM;
+			} else {
+				strcpy(*remaining_line, str);
+				rc = 0;
+			}
+			break;
+		}
+	}
+	return rc;
+}
+
+static int reset_stats(struct ckrm_class *class, const char *str)
+{
+	int rc;
+	char *resname = NULL, *statstr = NULL;
+	struct ckrm_controller *ctlr;
+	struct ckrm_shares *shares;
+
+	rc = ckrm_stats_parse(str, &resname, &statstr);
+	if (rc)
+		return rc;
+
+	ctlr = ckrm_get_controller_by_name(resname);
+	if (!ctlr) {
+		rc = -EINVAL;
+		goto done;
+	}
+	shares = ckrm_get_controller_shares(class, ctlr);
+	if (shares && ctlr->reset_stats)
+		rc = ctlr->reset_stats(shares, statstr);
+	ckrm_put_controller(ctlr);
+done:
+	kfree(resname);
+	kfree(statstr);
+	return rc;
+}
 
 struct class_attribute {
 	struct configfs_attribute configfs_attr;
@@ -29,6 +125,19 @@ struct class_attribute {
 	int (*store)(struct ckrm_class *, const char *);
 };
 
+struct class_attribute stats_attr = {
+	.configfs_attr = {
+		.ca_name = "stats",
+		.ca_owner = THIS_MODULE,
+		.ca_mode = S_IRUGO | S_IWUSR
+	},
+	.show = show_stats,
+	.store = reset_stats
+};
+
+static struct configfs_subsystem rcfs_subsys;
+static struct config_item_type rcfs_class_type;
+
 struct rcfs_class {
 	char *name;
 	struct ckrm_class *core;
@@ -169,6 +278,7 @@ static struct configfs_group_operations 
 };
 
 static struct configfs_attribute *class_attrs[] = {
+	&stats_attr.configfs_attr,
 	NULL
 };
 
Signature · 4 lines
----------------------------------------------------------------------
    Chandra Seetharaman               | Be careful what you choose....
              - [email protected]   |      .......you may get it.
----------------------------------------------------------------------
#10 lore raw

RFC PATCH 10/12 Add shares file support to RCFS

In reply to [email protected]

10/12 - ckrm_configfs_rcfs_shares

Adds attr_store and attr_show support for shares file. --

 kernel/ckrm/ckrm_rcfs.c |  136 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 136 insertions(+)
kernel/ckrm/ckrm_rcfs.c +136 -0
Index: linux2617-rc2/kernel/ckrm/ckrm_rcfs.c
===================================================================
@@ -23,6 +23,9 @@
 #define CKRM_NAME_LEN 20
 
 #define RES_STRING "res"
+#define MIN_SHARES_STRING "min_shares"
+#define MAX_SHARES_STRING "max_shares"
+#define CHILD_SHARES_DIVISOR_STRING "child_shares_divisor"
 
 static ssize_t show_stats(struct ckrm_class *class, char *buf)
 {
@@ -119,6 +122,128 @@ done:
 	return rc;
 }
 
+
+enum share_token_t {
+	MIN_SHARES_TOKEN,
+	MAX_SHARES_TOKEN,
+	CHILD_SHARES_DIVISOR_TOKEN,
+	RESOURCE_TYPE_TOKEN,
+	ERROR_TOKEN
+};
+
+/* Token matching for parsing input to this magic file */
+static match_table_t shares_tokens = {
+	{RESOURCE_TYPE_TOKEN, RES_STRING"=%s"},
+	{MIN_SHARES_TOKEN, MIN_SHARES_STRING"=%d"},
+	{MAX_SHARES_TOKEN, MAX_SHARES_STRING"=%d"},
+	{CHILD_SHARES_DIVISOR_TOKEN, CHILD_SHARES_DIVISOR_STRING"=%d"},
+	{ERROR_TOKEN, NULL}
+};
+
+static int shares_parse(const char *options, char **resname,
+					struct ckrm_shares *shares)
+{
+	char *p;
+	int option, rc = -EINVAL;
+
+	*resname = NULL;
+	if (!options)
+		goto done;
+	while ((p = strsep((char **)&options, ",")) != NULL) {
+		substring_t args[MAX_OPT_ARGS];
+		int token;
+
+		if (!*p)
+			continue;
+		token = match_token(p, shares_tokens, args);
+		switch (token) {
+		case RESOURCE_TYPE_TOKEN:
+			if (*resname)
+				goto done;
+			*resname = match_strdup(args);
+			break;
+		case MIN_SHARES_TOKEN:
+			if (match_int(args, &option))
+				goto done;
+			shares->min_shares = option;
+			break;
+		case MAX_SHARES_TOKEN:
+			if (match_int(args, &option))
+				goto done;
+			shares->max_shares = option;
+			break;
+		case CHILD_SHARES_DIVISOR_TOKEN:
+			if (match_int(args, &option))
+				goto done;
+			shares->child_shares_divisor = option;
+			break;
+		default:
+			goto done;
+		}
+	}
+	rc = 0;
+done:
+	if (rc) {
+		kfree(*resname);
+		*resname = NULL;
+	}
+	return rc;
+}
+
+static int set_shares(struct ckrm_class *class, const char *str)
+{
+	char *resname = NULL;
+	int rc;
+	struct ckrm_controller *ctlr;
+	struct ckrm_shares shares = {
+		.min_shares = CKRM_SHARE_UNCHANGED,
+		.max_shares = CKRM_SHARE_UNCHANGED,
+		.child_shares_divisor = CKRM_SHARE_UNCHANGED,
+	};
+
+	rc = shares_parse(str, &resname, &shares);
+	if (!rc) {
+		ctlr = ckrm_get_controller_by_name(resname);
+		if (ctlr) {
+			rc = ckrm_set_controller_shares(class, ctlr, &shares);
+			ckrm_put_controller(ctlr);
+		} else
+			rc = -EINVAL;
+		kfree(resname);
+	}
+	return rc;
+}
+
+static ssize_t show_shares(struct ckrm_class *class, char *buf)
+{
+	int i;
+	ssize_t j, rc = 0, bufsize = PAGE_SIZE;
+	struct ckrm_shares *shares;
+	struct ckrm_controller *ctlr;
+
+	for (i = 0; i < CKRM_MAX_RES_CTLRS; i++) {
+		ctlr = ckrm_get_controller_by_id(i);
+		if (!ctlr)
+			continue;
+		shares = ckrm_get_controller_shares(class, ctlr);
+		if (shares) {
+			if (bufsize <= 0)
+				break;
+			j = snprintf(buf, bufsize, "%s=%s,%s=%d,%s=%d,%s=%d\n",
+				RES_STRING, ctlr->name,
+				MIN_SHARES_STRING, shares->min_shares,
+				MAX_SHARES_STRING, shares->max_shares,
+				CHILD_SHARES_DIVISOR_STRING,
+				shares->child_shares_divisor);
+			rc += j; buf += j; bufsize -= j;
+		}
+		ckrm_put_controller(ctlr);
+	}
+	if (i < CKRM_MAX_RES_CTLRS)
+		rc = -ENOSPC;
+	return rc;
+}
+
 struct class_attribute {
 	struct configfs_attribute configfs_attr;
 	ssize_t (*show)(struct ckrm_class *, char *);
@@ -135,6 +260,16 @@ struct class_attribute stats_attr = {
 	.store = reset_stats
 };
 
+struct class_attribute shares_attr = {
+	.configfs_attr = {
+		.ca_name = "shares",
+		.ca_owner = THIS_MODULE,
+		.ca_mode = S_IRUGO | S_IWUSR
+	},
+	.show = show_shares,
+	.store = set_shares
+};
+
 static struct configfs_subsystem rcfs_subsys;
 static struct config_item_type rcfs_class_type;
 
@@ -279,6 +414,7 @@ static struct configfs_group_operations 
 
 static struct configfs_attribute *class_attrs[] = {
 	&stats_attr.configfs_attr,
+	&shares_attr.configfs_attr,
 	NULL
 };
 
Signature · 4 lines
----------------------------------------------------------------------
    Chandra Seetharaman               | Be careful what you choose....
              - [email protected]   |      .......you may get it.
----------------------------------------------------------------------
#11 lore raw

RFC PATCH 11/12 Add members file support to RCFS

In reply to [email protected]

11/12 - ckrm_configfs_rcfs_members

Adds attr_store and attr_show support for members file. --

 kernel/ckrm/ckrm_rcfs.c |   49 +++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 49 insertions(+)
kernel/ckrm/ckrm_rcfs.c +49 -0
Index: linux2617-rc2/kernel/ckrm/ckrm_rcfs.c
===================================================================
@@ -244,6 +244,43 @@ static ssize_t show_shares(struct ckrm_c
 	return rc;
 }
 
+/*
+ * Given a buffer with a pid in it, add the task with that pid to the class.
+ * Ignores entire buffer after the first pid is parsed.
+ */
+static int add_member(struct ckrm_class *class, const char *str)
+{
+	pid_t pid;
+
+	pid = (pid_t) simple_strtol(str, NULL, 0);
+	if (pid <= 0)
+		return -EINVAL; /* Not a valid pid */
+	return ckrm_setclass(pid, class);
+}
+
+/*
+ * Lists pids of tasks that belong to the given class.
+ */
+static ssize_t show_members(struct ckrm_class *class, char *buf)
+{
+	ssize_t i, rc = 0, bufsize = PAGE_SIZE;
+	struct task_struct *tsk;
+
+	spin_lock(&class->class_lock);
+	list_for_each_entry(tsk, &class->task_list, member_list) {
+		if (bufsize <= 0) {
+			rc = -ENOSPC;
+			break;
+		}
+		if (!tsk->pid)	/* Ignore swappers */
+			continue;
+		i = snprintf(buf, bufsize, "%ld\n", (long)tsk->pid);
+		buf += i; rc += i; bufsize -= i;
+	}
+	spin_unlock(&class->class_lock);
+	return rc;
+}
+
 struct class_attribute {
 	struct configfs_attribute configfs_attr;
 	ssize_t (*show)(struct ckrm_class *, char *);
@@ -270,6 +307,17 @@ struct class_attribute shares_attr = {
 	.store = set_shares
 };
 
+struct class_attribute members_attr = {
+	.configfs_attr = {
+		.ca_name = "members",
+		.ca_owner = THIS_MODULE,
+		.ca_mode = S_IRUGO | S_IWUSR
+	},
+	.show = show_members,
+	.store = add_member
+};
+
+
 static struct configfs_subsystem rcfs_subsys;
 static struct config_item_type rcfs_class_type;
 
@@ -415,6 +463,7 @@ static struct configfs_group_operations 
 static struct configfs_attribute *class_attrs[] = {
 	&stats_attr.configfs_attr,
 	&shares_attr.configfs_attr,
+	&members_attr.configfs_attr,
 	NULL
 };
 
Signature · 4 lines
----------------------------------------------------------------------
    Chandra Seetharaman               | Be careful what you choose....
              - [email protected]   |      .......you may get it.
----------------------------------------------------------------------
#12 lore raw

RFC PATCH 12/12 Documentation for CKRM

In reply to [email protected]

12/12 - ckrm_docs

Documentation describing important CKRM elements such as classes, shares, controllers, and the interface provided to userspace via RCFS --

 Documentation/ckrm/ckrm_basics  |   65 ++++++++++++++++++++++++++++++++++++++++
 Documentation/ckrm/ckrm_install |   54 +++++++++++++++++++++++++++++++++
 Documentation/ckrm/ckrm_usage   |   52 ++++++++++++++++++++++++++++++++
 3 files changed, 171 insertions(+)
Documentation/ckrm/ckrm_basics new file +65 -0
Index: linux-2.6.16/Documentation/ckrm/ckrm_basics
===================================================================
@@ -0,0 +1,65 @@
+CKRM Basics
+-------------
+A brief review of CKRM concepts and terminology will help make installation
+and testing easier. For more details, please visit http://ckrm.sf.net.
+
+Concept:
+User defines a class, associate some amount of resources to the class, and
+associates tasks with the class. Tasks belonging to that class will be
+bound by the amount of resources that are assigned to that class.
+
+RCFS depicts a CKRM class as a directory. Hierarchy of classes can be
+created in which children of a class share resources allotted to
+the parent. Tasks can be classified to any class which is at any level.
+There is no correlation between parent-child relationship of tasks and
+the parent-child relationship of classes they belong to.
+
+During fork(), class is inherited by a task. A privileged user can
+reassign a task to any class.
+
+Characteristics of a class can be accessed/changed through the following
+files under the directory representing the class:
+
+shares:  allows changing shares of different resource managed by the class
+stats:   shows statistics associated with each resource managed by the class
+members: allows assignment of tasks to a class and shows tasks that are
+         assigned to a class.
+
+Resource allocation of a class is proportional to the amount of resources
+available to the class's parent.
+Resource allocation for a class is controlled by the parameters:
+
+min_shares: Minimum amount shares that can be allocated by a class. A
+            special value of DONT_CARE(-3) means that there is no minimum
+	    shares of a resource specified. This class may not get
+	    any resource if the system is running short on resources.
+max_shares: Specifies the maximum amount of resource that is allowed to be
+           allocated by a class. A special value DONT_CARE(-3) means
+	   there is no specific limit is specified, this class can get all
+	   the resources available.
+child_shares_divisor: total guarantee that is allowed among the children of
+           this class. In other words, the sum of "guarantee"s of all
+	   children of this class cannot exceed this number.
+
+Any of these parameters can have a special value, UNSUPPORTED(-2) meaning
+that the specific controller does not support this parameter. User
+request to change the value will be ignored.
+
+None of these parameters neither absolute nor have any units associated with
+them. These are just numbers that are used to calculate the absolute number
+of resource available for a specific class.
+
+In order to make them independent of the type of resource and handle
+complexities like hotplug none of these parameters have units associated
+with them. Furthermore they are not percentages. They are called shares
+because an appropriate analogy would be shares in a stock market.
+
+The absolute amount (for example no. of tasks) of minimum shares available
+for a class is calculuated as:
+
+	absolute minimum shares = (parent's absolute amount of resource) *
+			(class's min_shares / parent's child_shares_divisor)
+
+Maximum shares is also calculated in the same way.
+
+Root class is allocated all the resources available in the system. In other
Documentation/ckrm/ckrm_install new file +54 -0

54 lines not on this page · raw message

Documentation/ckrm/ckrm_usage new file +52 -0

52 lines not on this page · raw message

Signature · 4 lines
----------------------------------------------------------------------
    Chandra Seetharaman               | Be careful what you choose....
              - [email protected]   |      .......you may get it.
----------------------------------------------------------------------
#13 lore raw
In reply to [email protected]
Quoted 2 lines · On Thu, 2006-04-20 at 19:24 -0700, [email protected] wrote:

CKRM has gone through a major overhaul by removing some of the complexity, cutting down on features and moving portions to userspace.

What do you want done with these patches? Do you think they are ready for mainline? -mm? Or, are you just posting here for comments?

-- Dave

#14 lore raw
Quoted 7 lines · Dave Hansen
Quoted 2 lines · On Thu, 2006-04-20 at 19:24 -0700, [email protected] wrote:

CKRM has gone through a major overhaul by removing some of the complexity, cutting down on features and moving portions to userspace.

What do you want done with these patches? Do you think they are ready for mainline? -mm? Or, are you just posting here for comments?

We think it is ready for -mm. But, want to go through a review cycle in lkml before i request Andrew for that.

Thanks for asking,

chandra

Quoted 12 lines

-- Dave

-------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
ckrm-tech mailing list
https://lists.sourceforge.net/lists/listinfo/ckrm-tech

Signature · 4 lines
----------------------------------------------------------------------
    Chandra Seetharaman               | Be careful what you choose....
              - [email protected]   |      .......you may get it.
----------------------------------------------------------------------
#15 lore raw
Quoted 6 lines · Chandra Seetharaman
Quoted 4 lines · Dave Hansen
Quoted 2 lines · On Thu, 2006-04-20 at 19:24 -0700, [email protected] wrote:

CKRM has gone through a major overhaul by removing some of the complexity, cutting down on features and moving portions to userspace.

Quoted 8 lines
Quoted 3 lines

What do you want done with these patches? Do you think they are ready for mainline? -mm? Or, are you just posting here for comments?

We think it is ready for -mm. But, want to go through a review cycle in lkml before i request Andrew for that.

From a quick scan, the overall code quality is probably the best I've seen

for an initial submission of this magnitude. I had a few minor issues and questions, but it'd need a couple of hours to go through it all.

So. Send 'em over when you're ready.

I have one concern. If we merge this framework into mainline then we'd (quite reasonably) expect to see an ongoing dribble of new controllers being submitted. But we haven't seen those controllers yet. So there is a risk that you'll submit a major new controller (most likely a net or memory controller) and it will provoke a reviewer revolt. We'd then be in a situation of cant-go-forward, cant-go-backward.

It would increase the comfort level if we could see what the major controllers look like before committing. But that's unreasonable.

Could I ask that you briefly enumerate

a) which controllers you think we'll need in the forseeable future

b) what they need to do

c) pointer to prototype code if poss

Thanks.

#16 lore raw
Quoted 11 lines · Andrew Morton
Quoted 10 lines · Chandra Seetharaman
Quoted 7 lines · Dave Hansen
Quoted 2 lines · On Thu, 2006-04-20 at 19:24 -0700, [email protected] wrote:

CKRM has gone through a major overhaul by removing some of the complexity, cutting down on features and moving portions to userspace.

What do you want done with these patches? Do you think they are ready for mainline? -mm? Or, are you just posting here for comments?

Quoted 5 lines
Quoted 2 lines

We think it is ready for -mm. But, want to go through a review cycle in lkml before i request Andrew for that.

From a quick scan, the overall code quality is probably the best I've seen for an initial submission of this magnitude. I had a few minor issues and

Thanks, and thanks to all that helped.

Quoted 3 lines

questions, but it'd need a couple of hours to go through it all.

So. Send 'em over when you're ready.

Great. I will wait for couple of days for comments and then send them your way.

Quoted 8 lines

I have one concern. If we merge this framework into mainline then we'd (quite reasonably) expect to see an ongoing dribble of new controllers being submitted. But we haven't seen those controllers yet. So there is a risk that you'll submit a major new controller (most likely a net or memory controller) and it will provoke a reviewer revolt. We'd then be in a situation of cant-go-forward, cant-go-backward.

I totally understand your concern.

CKRM's design is not tied with a specific implementation of a controller. It allows hooking up different controllers for the same resource. If a controller is considered complex, it can cut some of the features and be made simpler. Or a simpler controller can replace an earlier complex controller without affecting the user interface.

This flexibility feature reduces the "cant-go-forward, cant-go-back" problem, somewhat.

FYI, we found out that managing network resources was not falling into this task based model and we had to invent complex layering to accommodate it. So, we dropped our plans for network support.

One can write controller for any resource that can be accounted at task level. The corresponding subsystem stakeholders can ensure that it is clean, and at acceptable level.

Quoted 2 lines

It would increase the comfort level if we could see what the major controllers look like before committing. But that's unreasonable.

You might have seen the CPU controller (different implementation than what we had earlier) and the numtasks controller (can prevent fork bombs) that followed this patchset.

Quoted 5 lines

Could I ask that you briefly enumerate

a) which controllers you think we'll need in the forseeable future

Our main object is to provide resource control for the hardware
resources: CPU, I/O and memory.

We have already posted the CPU controller.

We have two implementations of memory controller and a I/O controller.

Memory controller is understandably more complex and controversial, and that is the reason we haven't posted it this time around (we are looking at ways to simplify the design and hence the complexity). Both the memory controllers has been posted to linux-mm.

I/O controller is based on CFQ-scheduler.

Quoted 1 line

b) what they need to do

Both memory controllers provide control for LRU lists.

- One maintains the active/inactive lists per class for each zone. It is of order O(1). Current code is little complex. We are looking at ways to simplify it.

- Another creates pseudo zones under each zones (by splitting the number of pages available in a zone) and attaches them with each class.

I/O Controller that we are working on is based on CFQ scheduler and provides bandwidth control.

Quoted 2 lines

c) pointer to prototype code if poss

Both the memory controllers are fully functional. We need to trim them down.

active/inactive list per class memory controller:
http://prdownloads.sourceforge.net/ckrm/mem_rc-f0.4-2615-v2.tz?download

pzone based memory controller:
http://marc.theaimsgroup.com/?l=ckrm-tech&m=113867467006531&w=2

i/o controller: This controller is not ported to the framework posted, but can be taken for a prototype version. New version would be simpler though.

http://prdownloads.sourceforge.net/ckrm/io_rc.tar.bz2?download

Thanks & Regards,

chandra

Quoted 2 lines

Thanks.

Signature · 4 lines
----------------------------------------------------------------------
    Chandra Seetharaman               | Be careful what you choose....
              - [email protected]   |      .......you may get it.
----------------------------------------------------------------------
#17 lore raw
Quoted 9 lines · Chandra Seetharaman
Quoted 3 lines

c) pointer to prototype code if poss

Both the memory controllers are fully functional. We need to trim them down.

active/inactive list per class memory controller:
http://prdownloads.sourceforge.net/ckrm/mem_rc-f0.4-2615-v2.tz?download

Oh my gosh. That converts memory reclaim from per-zone LRU to per-CKRM-class LRU. If configured.

This is huge. It means that we have basically two quite different versions
of memory reclaim to test and maintain. This is a problem.

(I hope that's the before-we-added-comments version of the patch btw).

Quoted 4 lines

pzone based memory controller:
http://marc.theaimsgroup.com/?l=ckrm-tech&m=113867467006531&w=2

From a super-quick scan that looks saner. Is it effective? Is this the

way you're planning on proceeding?

This requirement is basically a glorified RLIMIT_RSS manager, isn't it? Just that it covers a group of mm's and not just the one mm?

Do you attempt to manage just pagecache? So if class A tries to read 10GB from disk, does that get more aggressively reclaimed based on class A's resource limits?

This all would have been more comfortable if done on top of the 2.4 kernel's virtual scanner.

(btw, using the term "class" to identify a group of tasks isn't very comfortable - it's an instance, not a class...)

Worried.

#18 lore raw
Quoted 5 lines · Andrew Morton
Quoted 4 lines · Chandra Seetharaman
Quoted 3 lines

c) pointer to prototype code if poss

Quoted 8 lines
Quoted 5 lines

Both the memory controllers are fully functional. We need to trim them down.

active/inactive list per class memory controller:
http://prdownloads.sourceforge.net/ckrm/mem_rc-f0.4-2615-v2.tz?download

Oh my gosh. That converts memory reclaim from per-zone LRU to per-CKRM-class LRU. If configured.

Yes. We originally had an implementation that would use the existing per-zone LRU, but the reclamation path was O(n), where n is the number of classes. So, we moved towards a O(1) algorithm.

Quoted 3 lines

This is huge. It means that we have basically two quite different versions
of memory reclaim to test and maintain. This is a problem.

Understood, will work and come up with an acceptable memory controller.

Quoted 2 lines

(I hope that's the before-we-added-comments version of the patch btw).

Yes, indeed :). As I told earlier this patch is not ready for lkml or - mm yet.

Quoted 7 lines
Quoted 3 lines

pzone based memory controller:
http://marc.theaimsgroup.com/?l=ckrm-tech&m=113867467006531&w=2

From a super-quick scan that looks saner. Is it effective? Is this the way you're planning on proceeding?

Yes, it is effective, and the reclamation is O(1) too. It has couple of problems by design, (1) doesn't handle shared pages and (2) doesn't provide support for both min_shares and max_shares.

Quoted 2 lines

This requirement is basically a glorified RLIMIT_RSS manager, isn't it? Just that it covers a group of mm's and not just the one mm?

Yes, that is the core object of ckrm, associate resources to a group of tasks.

Quoted 4 lines

Do you attempt to manage just pagecache? So if class A tries to read 10GB from disk, does that get more aggressively reclaimed based on class A's resource limits?

Yes, it would get more aggressively reclaimed. But, if you have the I/O controller also configured appropriately only class A will be affected.

Quoted 6 lines

This all would have been more comfortable if done on top of the 2.4 kernel's virtual scanner.

(btw, using the term "class" to identify a group of tasks isn't very comfortable - it's an instance, not a class...)

We could go with "Resource Group" as Matt suggested.

Quoted 2 lines

Valerie, KUROSAWA, Please free to add any more details.

Quoted 1 line

Worried.

Signature · 4 lines
----------------------------------------------------------------------
    Chandra Seetharaman               | Be careful what you choose....
              - [email protected]   |      .......you may get it.
----------------------------------------------------------------------
#19 lore raw
Quoted 9 lines · On Fri, 21 Apr 2006 22:28:45 -0700 Chandra Seetharaman <[email protected]>…
Quoted 5 lines
Quoted 2 lines

pzone based memory controller:
http://marc.theaimsgroup.com/?l=ckrm-tech&m=113867467006531&w=2

From a super-quick scan that looks saner. Is it effective? Is this the way you're planning on proceeding?

Yes, it is effective, and the reclamation is O(1) too. It has couple of problems by design, (1) doesn't handle shared pages and (2) doesn't provide support for both min_shares and max_shares.

Right. I wanted to show proof-of-cencept of the pzone based controller and implemented minimal features necessary as the memory controller. So, the pzone based controller still needs development and some cleanup.

#20 lore raw
Quoted 11 lines
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

pzone based memory controller:
http://marc.theaimsgroup.com/?l=ckrm-tech&m=113867467006531&w=2

From a super-quick scan that looks saner. Is it effective? Is this the way you're planning on proceeding?

Yes, it is effective, and the reclamation is O(1) too. It has couple of problems by design, (1) doesn't handle shared pages and (2) doesn't provide support for both min_shares and max_shares.

Right. I wanted to show proof-of-cencept of the pzone based controller and implemented minimal features necessary as the memory controller. So, the pzone based controller still needs development and some cleanup.

Just out of curiosity, how it was meassured that it is effective? How does it work when there is a global memory shortage in the system?

Thanks,
Kirill

#21 lore raw
Quoted 4 lines · On Mon, 24 Apr 2006 08:39:52 +0400 Kirill Korotaev <[email protected]> wrote:
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines

pzone based memory controller:
http://marc.theaimsgroup.com/?l=ckrm-tech&m=113867467006531&w=2

From a super-quick scan that looks saner. Is it effective? Is this the way you're planning on proceeding?

Quoted 8 lines
Quoted 7 lines
Quoted 3 lines

Yes, it is effective, and the reclamation is O(1) too. It has couple of problems by design, (1) doesn't handle shared pages and (2) doesn't provide support for both min_shares and max_shares.

Right. I wanted to show proof-of-cencept of the pzone based controller and implemented minimal features necessary as the memory controller. So, the pzone based controller still needs development and some cleanup.

Just out of curiosity, how it was meassured that it is effective?

I don't have any benchmark numbers yet, so I can't explain the effectiveness with numbers. I've been looking for the way to measure the cost of pzones correctly, but I've not found it out yet.

Quoted 1 line

How does it work when there is a global memory shortage in the system?

I guess you are referring to the situation that global memory is running out but there are free pages in pzones. These free pages in pzones are handled as reserved for pzone users and not used even in global memory shortage.

Thanks,

Signature · 1 line
KUROSAWA, Takahiro
#22 lore raw
Quoted 16 lines
Quoted 9 lines
Quoted 7 lines
Quoted 3 lines

Yes, it is effective, and the reclamation is O(1) too. It has couple of problems by design, (1) doesn't handle shared pages and (2) doesn't provide support for both min_shares and max_shares.

Right. I wanted to show proof-of-cencept of the pzone based controller and implemented minimal features necessary as the memory controller. So, the pzone based controller still needs development and some cleanup.

Just out of curiosity, how it was meassured that it is effective?

I don't have any benchmark numbers yet, so I can't explain the effectiveness with numbers. I've been looking for the way to measure the cost of pzones correctly, but I've not found it out yet.

Quoted 6 lines
Quoted 1 line

How does it work when there is a global memory shortage in the system?

I guess you are referring to the situation that global memory is running out but there are free pages in pzones. These free pages in pzones are handled as reserved for pzone users and not used even in global memory shortage.

ok. Let me explain what I mean.
Imagine the situation with global memory shortage. In kernel, there are threads which do some job behalf the user, e.g. kjournald, loop etc. If the user has some pzone memory, but these threads fail to do their job some nasty things can happen (ext3 problems, deadlocks, OOM etc.) If such behaviour is ok for you, then great. But did you consider it?

Also, I can't understand how it works with OOM killer. If pzones has enough memory, but there is a global shortage, who will be killed?

Thanks,
Kirill

#23 lore raw
Quoted 27 lines · On Mon, 24 Apr 2006 10:45:59 +0400 Kirill Korotaev <[email protected]> wrote:
Quoted 14 lines
Quoted 9 lines
Quoted 7 lines
Quoted 3 lines

Yes, it is effective, and the reclamation is O(1) too. It has couple of problems by design, (1) doesn't handle shared pages and (2) doesn't provide support for both min_shares and max_shares.

Right. I wanted to show proof-of-cencept of the pzone based controller and implemented minimal features necessary as the memory controller. So, the pzone based controller still needs development and some cleanup.

Just out of curiosity, how it was meassured that it is effective?

I don't have any benchmark numbers yet, so I can't explain the effectiveness with numbers. I've been looking for the way to measure the cost of pzones correctly, but I've not found it out yet.

Quoted 6 lines
Quoted 1 line

How does it work when there is a global memory shortage in the system?

I guess you are referring to the situation that global memory is running out but there are free pages in pzones. These free pages in pzones are handled as reserved for pzone users and not used even in global memory shortage.

ok. Let me explain what I mean.
Imagine the situation with global memory shortage. In kernel, there are threads which do some job behalf the user, e.g. kjournald, loop etc. If the user has some pzone memory, but these threads fail to do their job some nasty things can happen (ext3 problems, deadlocks, OOM etc.) If such behaviour is ok for you, then great. But did you consider it?

Quoted 2 lines

Also, I can't understand how it works with OOM killer. If pzones has enough memory, but there is a global shortage, who will be killed?

I understand.
IMHO, only the system processes should use global memory. User processes that may cause such memory shortage should be enclosed in pzones first.

Thanks,

Signature · 1 line
KUROSAWA, Takahiro
#24 lore raw
In reply to Chandra Seetharaman

Hi Chandra,

Quoted 29 lines
Quoted 10 lines
Quoted 7 lines
Quoted 1 line

c) pointer to prototype code if poss

Both the memory controllers are fully functional. We need to trim them down.

active/inactive list per class memory controller:
http://prdownloads.sourceforge.net/ckrm/mem_rc-f0.4-2615-v2.tz?download

Oh my gosh. That converts memory reclaim from per-zone LRU to per-CKRM-class LRU. If configured.

Yes. We originally had an implementation that would use the existing per-zone LRU, but the reclamation path was O(n), where n is the number of classes. So, we moved towards a O(1) algorithm.

Quoted 3 lines

This is huge. It means that we have basically two quite different versions
of memory reclaim to test and maintain. This is a problem.

Understood, will work and come up with an acceptable memory controller.

Quoted 2 lines

(I hope that's the before-we-added-comments version of the patch btw).

Yes, indeed :). As I told earlier this patch is not ready for lkml or - mm yet.

Quoted 4 lines
Quoted 3 lines

pzone based memory controller:
http://marc.theaimsgroup.com/?l=ckrm-tech&m=113867467006531&w=2

Quoted 7 lines
Quoted 3 lines

From a super-quick scan that looks saner. Is it effective? Is this the way you're planning on proceeding?

Yes, it is effective, and the reclamation is O(1) too. It has couple of problems by design, (1) doesn't handle shared pages and (2) doesn't provide support for both min_shares and max_shares.

I'm not sure all of them have to be managed under ckrm_core and rcfs in kernel.

These functions you mentioned can be implemented in user space to minimize the overhead in usual VM operations because it isn't expected quick response to resize it. It is a bit different from that of CPU resource.

You don't need to invent everything. I think you can reuse what NUMA team is doing instead. This approach may not fit in your rcfs, though.

Quoted 5 lines
Quoted 2 lines

This requirement is basically a glorified RLIMIT_RSS manager, isn't it? Just that it covers a group of mm's and not just the one mm?

Yes, that is the core object of ckrm, associate resources to a group of tasks.

Thanks,
Hirokazu Takahahsi.

#25 lore raw
Quoted 1 line · Hirokazu Takahashi

Hi Chandra,

<snip>

Quoted 11 lines
Quoted 3 lines

Yes, it is effective, and the reclamation is O(1) too. It has couple of problems by design, (1) doesn't handle shared pages and (2) doesn't provide support for both min_shares and max_shares.

I'm not sure all of them have to be managed under ckrm_core and rcfs in kernel.

These functions you mentioned can be implemented in user space to minimize the overhead in usual VM operations because it isn't expected quick response to resize it. It is a bit different from that of CPU resource.

Agree, that is where the additional complexity arise from.

If the user can achieve the same results with user space solution that would be good too.

Thanks

chandra

Quoted 12 lines

You don't need to invent everything. I think you can reuse what NUMA team is doing instead. This approach may not fit in your rcfs, though.

Quoted 5 lines
Quoted 2 lines

This requirement is basically a glorified RLIMIT_RSS manager, isn't it? Just that it covers a group of mm's and not just the one mm?

Yes, that is the core object of ckrm, associate resources to a group of tasks.

Thanks,
Hirokazu Takahahsi.

Signature · 4 lines
----------------------------------------------------------------------
    Chandra Seetharaman               | Be careful what you choose....
              - [email protected]   |      .......you may get it.
----------------------------------------------------------------------
#26 lore raw
In reply to Andrew Morton

On Fri, 2006-04-21 at 19:13 -0700, Andrew Morton wrote:

<snip> (I'll let those more familiar with the memory controller efforts comment on those concerns)

Quoted 2 lines

(btw, using the term "class" to identify a group of tasks isn't very comfortable - it's an instance, not a class...)

Yes, I can see how this would be uncomfortable. How about replacing "class" with "resource group"?

Cheers,
-Matt Helsley

#27 lore raw
Quoted 6 lines · Matt Helsley
Quoted 3 lines

(btw, using the term "class" to identify a group of tasks isn't very comfortable - it's an instance, not a class...)

Yes, I can see how this would be uncomfortable. How about replacing "class" with "resource group"?

Much more comfortable, thanks ;)

#28 lore raw
In reply to Andrew Morton
Quoted 2 lines · Andrew

(btw, using the term "class" to identify a group of tasks isn't very comfortable - it's an instance, not a class...)

Bless you. I objected to the term 'class' a long time ago, but failed to advance my case in a successful fashion.

Matt replied:

Quoted 1 line

"resource group"?

Nice.

Signature · 3 lines
                  I won't rest till it's the best ...
                  Programmer, Linux Scalability
                  Paul Jackson <[email protected]> 1.925.600.0401
#29 lore raw
Quoted 6 lines · Paul Jackson
Quoted 2 lines · Andrew

(btw, using the term "class" to identify a group of tasks isn't very comfortable - it's an instance, not a class...)

Bless you. I objected to the term 'class' a long time ago, but failed to advance my case in a successful fashion.

Well, I wouldn't say you were entirely unsuccessful. I distinctly
remembered your case and I tried to think of suitable names during the recent changes. Please take a look at the latest set of patches and see if you think the names are clearer.

Quoted 4 lines

Matt replied:

Quoted 1 line

"resource group"?

Nice.

Cheers,
-Matt Helsley

#30 lore raw
In reply to Andrew Morton
Quoted 32 lines · Andrew Morton
Quoted 9 lines · Chandra Seetharaman
Quoted 3 lines

c) pointer to prototype code if poss

Both the memory controllers are fully functional. We need to trim them down.

active/inactive list per class memory controller:
http://prdownloads.sourceforge.net/ckrm/mem_rc-f0.4-2615-v2.tz?download

Oh my gosh. That converts memory reclaim from per-zone LRU to per-CKRM-class LRU. If configured.

This is huge. It means that we have basically two quite different versions
of memory reclaim to test and maintain. This is a problem.

(I hope that's the before-we-added-comments version of the patch btw).

Quoted 2 lines

pzone based memory controller:
http://marc.theaimsgroup.com/?l=ckrm-tech&m=113867467006531&w=2

From a super-quick scan that looks saner. Is it effective? Is this the way you're planning on proceeding?

This requirement is basically a glorified RLIMIT_RSS manager, isn't it? Just that it covers a group of mm's and not just the one mm?

Do you attempt to manage just pagecache? So if class A tries to read 10GB from disk, does that get more aggressively reclaimed based on class A's resource limits?

Quoted 8 lines

This all would have been more comfortable if done on top of the 2.4 kernel's virtual scanner.

(btw, using the term "class" to identify a group of tasks isn't very comfortable - it's an instance, not a class...)

Worried.

The object of this infrastructure is to get a unified interface for resource management, irrespective of the resource that is being managed.

As I mentioned in my earlier email, subsystem experts are the ones who will finally decide what type resource controller they will accept. With VM experts' direction and advice, i am positive that we will get an excellent memory controller (as well as other controllers).

As you might have noticed, we have gone through major changes to come to community's acceptance levels. We are now making use of all possible features (kref, process event connector, configfs, module parameter, kzalloc) in this infrastructure.

Having a CPU controller, two memory controllers, an I/O controller and a numtasks controller proves that the infrastructure does handle major resources nicely and is also capable of managing virtual resources.

Hope i reduced your worries (at least some :).

regards,

chandra

Quoted 10 lines

-------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
ckrm-tech mailing list
https://lists.sourceforge.net/lists/listinfo/ckrm-tech

Signature · 4 lines
----------------------------------------------------------------------
    Chandra Seetharaman               | Be careful what you choose....
              - [email protected]   |      .......you may get it.
----------------------------------------------------------------------
#31 lore raw
Quoted 14 lines
Quoted 1 line

Worried.

The object of this infrastructure is to get a unified interface for resource management, irrespective of the resource that is being managed.

As I mentioned in my earlier email, subsystem experts are the ones who will finally decide what type resource controller they will accept. With VM experts' direction and advice, i am positive that we will get an excellent memory controller (as well as other controllers).

As you might have noticed, we have gone through major changes to come to community's acceptance levels. We are now making use of all possible features (kref, process event connector, configfs, module parameter, kzalloc) in this infrastructure.

Quoted 5 lines

Having a CPU controller, two memory controllers, an I/O controller and a numtasks controller proves that the infrastructure does handle major resources nicely and is also capable of managing virtual resources.

Hope i reduced your worries (at least some :).

Not all :) Let me explain.

Until you provided something more complex then numtasks, this infrastructure is pure theory. For example, in your infrastracture, when you will add memory resource controller with data sharing, you will face that changing CKRM class of the tasks is almost impossible in a suitable way. Another possible situation: hierarchical classes with shared memory are even more complicated thing.

In both cases you can end up with a poor/complicated/slow solution or dropping some of your infrastructre features (changing class on the fly, hierarchy) or which is worse IMHO with incosistency between controllers and interfaces.

Thanks,
Kirill

#32 lore raw
Quoted 18 lines · Kirill Korotaev
Quoted 18 lines
Quoted 1 line

Worried.

The object of this infrastructure is to get a unified interface for resource management, irrespective of the resource that is being managed.

As I mentioned in my earlier email, subsystem experts are the ones who will finally decide what type resource controller they will accept. With VM experts' direction and advice, i am positive that we will get an excellent memory controller (as well as other controllers).

As you might have noticed, we have gone through major changes to come to community's acceptance levels. We are now making use of all possible features (kref, process event connector, configfs, module parameter, kzalloc) in this infrastructure.

Having a CPU controller, two memory controllers, an I/O controller and a numtasks controller proves that the infrastructure does handle major resources nicely and is also capable of managing virtual resources.

Quoted 7 lines
Quoted 1 line

Hope i reduced your worries (at least some :).

Not all :) Let me explain.

Until you provided something more complex then numtasks, this infrastructure is pure theory. For example, in your infrastracture, when you will add memory resource controller with data sharing, you will face that changing CKRM class of the tasks is almost impossible in a suitable

I do not see a problem here, there could be 2 solutions:
- do not account shared pages against the resource group(put them in the default resource group (as some other OSs do)).
- when you are moving the task to a different class, calculate the resource group's usage depending on how many users are using a specific page.

Quoted 2 lines

way. Another possible situation: hierarchical classes with shared memory are even more complicated thing.

Hierarchy is not an issue. Resource controller can calculate the absolute number of resources (say no. of pages in this case) when the shares are assigned and then treat all resource groups as flat.

Quoted 5 lines

In both cases you can end up with a poor/complicated/slow solution or dropping some of your infrastructre features (changing class on the fly, hierarchy) or which is worse IMHO with incosistency between controllers and interfaces.

I am not convinced (based on the above explanations).

Quoted 4 lines

Thanks,
Kirill

Signature · 4 lines
----------------------------------------------------------------------
    Chandra Seetharaman               | Be careful what you choose....
              - [email protected]   |      .......you may get it.
----------------------------------------------------------------------
#33 lore raw
In reply to Chandra Seetharaman

Hi Chandra,

Quoted 10 lines
Quoted 4 lines

Could I ask that you briefly enumerate

a) which controllers you think we'll need in the forseeable future

Our main object is to provide resource control for the hardware
resources: CPU, I/O and memory.

We have already posted the CPU controller.

Quoted 10 lines

We have two implementations of memory controller and a I/O controller.

Memory controller is understandably more complex and controversial, and that is the reason we haven't posted it this time around (we are looking at ways to simplify the design and hence the complexity). Both the memory controllers has been posted to linux-mm.

I/O controller is based on CFQ-scheduler.

Quoted 1 line

b) what they need to do

	(snip)
Quoted 4 lines

I/O Controller that we are working on is based on CFQ scheduler and provides bandwidth control.

Quoted 2 lines

c) pointer to prototype code if poss

	(snip)
Quoted 3 lines

i/o controller: This controller is not ported to the framework posted, but can be taken for a prototype version. New version would be simpler though.

I think controlling I/O bandwidth is right way to go.

However, I think you need to change the design of the controller a bit. A lot of I/O requests processes issue will be handled by other contexts. There are AIO, journaling, pdflush and vmscan, which some kernel threads treat instead of the processes.

The current design looks not to care about this.

Thanks,
Hirokazu Takahashi.

#34 lore raw
Quoted 11 lines · Hirokazu Takahashi
Quoted 7 lines

i/o controller: This controller is not ported to the framework posted, but can be taken for a prototype version. New version would be simpler though.

I think controlling I/O bandwidth is right way to go.

Thanks. Obviously we agree heartily :-)

Quoted 8 lines

However, I think you need to change the design of the controller a bit. A lot of I/O requests processes issue will be handled by other contexts. There are AIO, journaling, pdflush and vmscan, which some kernel threads treat instead of the processes.

The current design looks not to care about this.

Yes. The current design, which builds directly on top of the CFQ scheduler, does not attempt to treat kernel threads specially in order to account the I/O they're doing on behalf of others properly. This was mainly because of the desire to keep the controller simple.

I suspect pdflush and vmscan I/O is never going to be properly attributable and journaling may be possible but unlikely to be worth it given the risks of throttling it ? AIO is likely to be something we can address if there is consensus that one is willing to pay the price of tracking the source through the I/O submission layers.

I suppose this would be a good time to dust off the I/O controller and post it so discussions can become more concrete.

But as always, changes in the design and implementation are always welcome....

Regards,
Shailabh

Quoted 4 lines

Thanks,
Hirokazu Takahashi.