cpusets: dynamical scheduler domain flags

From: Michael Wang <[email protected]>

This patch set provide a way for user to dynamically configure the scheduler domain flags, which usually to be static.

We can do the configuration through cpuset cgroup, new file will be found under each hierarchy:

sched_smt_domain_flag
-- appear when CONFIG_SCHED_SMT enabled, stands for the domain flags
of cpuset on smt level

sched_mc_domain_flag
-- appear when CONFIG_SCHED_MC enabled, stands for the domain flags
of cpuset on mc level

sched_book_domain_flag
-- appear when CONFIG_SCHED_BOOK enabled, stands for the domain flags
of cpuset on book level

sched_cpu_domain_flag
-- stands for the domain flags of cpuset on cpu level

sched_enable_domain_flag
-- set to 1 if want to use the dynamical domain flags, domain will
be rebuild

Examples:

Currently the domain flags bit are:

#define SD_LOAD_BALANCE         0x0001  /* Do load balancing on this domain. */
#define SD_BALANCE_NEWIDLE      0x0002  /* Balance when about to become idle */
#define SD_BALANCE_EXEC         0x0004  /* Balance on exec */
#define SD_BALANCE_FORK         0x0008  /* Balance on fork, clone */
#define SD_BALANCE_WAKE         0x0010  /* Balance on wakeup */
#define SD_WAKE_AFFINE          0x0020  /* Wake task to waking CPU */
#define SD_PREFER_LOCAL         0x0040  /* Prefer to keep tasks local to this domain */
#define SD_SHARE_CPUPOWER       0x0080  /* Domain members share cpu power */
#define SD_SHARE_PKG_RESOURCES  0x0200  /* Domain members share cpu pkg resources */
#define SD_SERIALIZE            0x0400  /* Only a single load balancing instance */
#define SD_ASYM_PACKING         0x0800  /* Place busy groups earlier in the domain */
#define SD_PREFER_SIBLING       0x1000  /* Prefer to place tasks in a sibling domain */
#define SD_OVERLAP              0x2000  /* sched_domains of this level overlap */
#define SD_NUMA                 0x4000  /* cross-node balancing */

If we want to set SMT domain to be:
'SD_LOAD_BALANCE | SD_BALANCE_WAKE | SD_OVERLAP'
and other level domain only has SD_LOAD_BALANCE, then we can use:

	echo 0x1011 > /sys/fs/cgroup/cpuset/cpuset.sched_smt_domain_flag
	echo 0x1 > /sys/fs/cgroup/cpuset/cpuset.sched_mc_domain_flag
	echo 0x1 > /sys/fs/cgroup/cpuset/cpuset.sched_book_domain_flag
	echo 0x1 > /sys/fs/cgroup/cpuset/cpuset.sched_cpu_domain_flag
	echo 1 > /sys/fs/cgroup/cpuset/cpuset.sched_enable_domain_flag

including:
cpusets: add basic variables
cpusets: add functions and code for initialization
cpusets: enable the dynamical domain flags
cpusets: add fundamental functions for recording
cpusets: add the configuration facility

---
 b/include/linux/sched.h |   22 ++++++++
 b/kernel/cpuset.c       |    7 ++
 b/kernel/sched/core.c   |    2 
 include/linux/sched.h   |   10 ++-
 kernel/cpuset.c         |  131 +++++++++++++++++++++++++++++++++++++++++++++++-
 kernel/sched/core.c     |   10 +++
 6 files changed, 176 insertions(+), 6 deletions(-)

NAK.. you don't get to expose all this nonsense in a 'stable' ABI.

You shouldn't need to prod at them to begin with.

So is that means expose those domain flags to user is a bad idea at all?

Is this caused by that people don't need such feature or this feature has do harm to kernel's stability?(it could be not...)

Regards,
Michael Wang

You can set/clear flags with scripts now, ie domain flags are already exposed.. as defined by the running kernel.

SD_SHARE_PKG_RESOURCES is a good flag look at. What does flipping that switch do, and what did it stop doing recently? So yeah, methinks exporting flags via cpusets is a bad idea. Not only is existence of any particular flag volatile, functionality behind it is volatile as well, so having a button to poke does undefined things. (not to mention non-exclusive sets)

-Mike

I think I got your and peter's opinion, so we could not make sure the kernel could still work well if some flags was enabled because their behave are always changing, and it's impossible to maintain such volatile feature.

Actually I got this idea after reading:

http://marc.info/?l=linux-kernel&m=130822782111533

But looks like I don't get the point, so what we want is building the domain according to some system topology designed by producer?

Regards,
Michael Wang