optional non-interactive mode for cpu scheduler

optional non-interactive mode for cpu scheduler

Some environments desire strict control over cpu resources based on nice values without interactive determination.

Add a sysctl (/proc/sys/kernel/interactive) to disable dynamic priority behaviour and interactive reinsertion.

Index: linux-2.6.10-rc1-mm2/include/linux/sched.h
===================================================================
@@ -738,6 +738,7 @@ extern int task_prio(const task_t *p);
 extern int task_nice(const task_t *p);
 extern int task_curr(const task_t *p);
 extern int idle_cpu(int cpu);
+extern int sched_interactive;
 
 void yield(void);
 

Index: linux-2.6.10-rc1-mm2/include/linux/sysctl.h
===================================================================
@@ -134,6 +134,7 @@ enum
 	KERN_SPARC_SCONS_PWROFF=64, /* int: serial console power-off halt */
 	KERN_HZ_TIMER=65,	/* int: hz timer on or off */
 	KERN_UNKNOWN_NMI_PANIC=66, /* int: unknown nmi panic flag */
+	KERN_INTERACTIVE=67,	/* int: dynamic priorities */
 };
 
 

Index: linux-2.6.10-rc1-mm2/kernel/sched.c
===================================================================
@@ -150,8 +150,8 @@
 #define DELTA(p) \
 	(SCALE(TASK_NICE(p), 40, MAX_BONUS) + INTERACTIVE_DELTA)
 
-#define TASK_INTERACTIVE(p) \
-	((p)->prio <= (p)->static_prio - DELTA(p))
+#define TASK_INTERACTIVE(p) ((sched_interactive) && \
+	((p)->prio <= (p)->static_prio - DELTA(p)))
 
 #define INTERACTIVE_SLEEP(p) \
 	(JIFFIES_TO_NS(MAX_SLEEP_AVG * \

@@ -602,6 +602,13 @@ static inline void enqueue_task_head(str
 }
 
 /*
+ * This is a sysctl which allows dynamic priorities to be applied based on
+ * interactive behaviour, and selective interactive reinsertion into the
+ * active array despite timeslice expiration.
+ */
+int sched_interactive = 1;
+
+/*
  * effective_prio - return the priority that is based on the static
  * priority but is modified by bonuses/penalties.
  *

@@ -622,6 +629,10 @@ static int effective_prio(task_t *p)
 	if (rt_task(p))
 		return p->prio;
 
+	/* Dynamic priorities are not used with interactive mode off */
+	if (!sched_interactive)
+		return p->static_prio;
+
 	bonus = CURRENT_BONUS(p) - MAX_BONUS / 2;
 
 	prio = p->static_prio - bonus;

Index: linux-2.6.10-rc1-mm2/kernel/sysctl.c
===================================================================
@@ -624,6 +624,14 @@ static ctl_table kern_table[] = {
 		.proc_handler   = &proc_unknown_nmi_panic,
 	},
 #endif
+	{
+		.ctl_name	= KERN_INTERACTIVE,
+		.procname	= "interactive",
+		.data		= &sched_interactive,
+		.maxlen		= sizeof (int),
+		.mode		= 0644,
+		.proc_handler	= &proc_dointvec,
+	},
 	{ .ctl_name = 0 }
 };
 

The better cpu proportion guarantee without low latency of such a policy would be desirable to video encoding in the background while capturing in the foreground as one immediately recognisable purpose, and there are likely numerous others, so I agree it's a good idea.

However the non-interactive mode addresses a number of different needs that seem to have come up. Specifically:
I have had users report great success with such a mode on my own scheduler in multiple X session setups where very choppy behaviour occurs in mainline. Many high performance computing people do not wish interactivity code modifying their choice of latency/distribution - admittedly this is a soft one.

What are your thoughts on this?

Con

i think the following scheme would work better:

- introduce a new SCHED_CPUBOUND policy
- return ->static_prio + 5 for such tasks
- keep their timeslice based off ->static_prio

the point is this: such tasks would thus be automatically and perpetually considered 'CPU hogs'. Applications cannot abuse this mechanism because they get the maximum 'penalty'.

and as a bonus, no magic sysctl and inherently more flexibility.

(note that this scheme has advantages above nice +5 because nice +5 still has the interactivity stuff on which can create priority fluctuations and may thus affect workloads.)

if you agree with this scheme, would you be interested in implementing this?

Ingo

Heh I wasnt implying we should move to staircase to fix our problems (then I'd have nothing special for -ck would I?). I was simply trying to reproduce that behaviour with a similar mode. As for the choppiness, the reports were that it would go away if X was run nice+19 which implies no dynamic priority adjustment was the answer.

Assuming they wanted to run everything SCHED_CPUBOUND, yes.

Your solution has quite some merit to it :)

I'll look into coding it later this week (thanks for suggesting I do it btw). This ordeal has left me seriously sleep deprived :P

Since we're considering providing a special cpu policy for high latency high cpu usage, does that mean we can now talk about other policies like batch, isochronous etc? And in the medium to long term future, gang and group?

Regards,
Con

:-|

SCHED_ISO would be interesting, but all SCHED_BATCH patches that i've seen so far were fundamentally broken. [ none protects against the possibility of a simple CPU hog starving a SCHED_BATCH task in kernel mode holding say /home's i_sem forever. None except the one i wrote a couple of years ago that is ;-) ]

but obviously any new scheduling policy first needs considerable testing, exposure and concensus. The main thing that makes SCHED_CPUBOUND possibly objectionable is that it could easily be used as a flag to 'turn off the interactivity code', which is wrong and just prolongs the fixing of interactivity-estimator bugs. Scientific apps burn CPU time exclusively and they have a stable priority at the low end of the range.

One exception would be CPU-bound code with multiple threads which interact with each other - one always runs but the others always sleep. A possible solution would be to exclude all inter-task synchronization methods from the 'interactivity boost' and only hard-device-waits would be considered true 'waiting', such as keyboard, mouse, disk or network IO.

Ingo

since SCHED_CPUBOUND would be inherited across fork(), it should be rather easy to start an X session with all tasks as SCHED_CPUBOUND.

but i think the above rather points in the direction of some genuine weakness in the interactivity code (i know, for which the fix is staircase ;) which would be nice to debug.

well, SCHED_CPUBOUND would solve their needs too, right?

Ingo

Alternatively, you might try a system where each "hard-device-wait" gets a specific interactivity rating, the default would be 1.0, but one could specify that /dev/input/mice gets a rating of 10.0. Then when handling inter-process communication, a wait on IPC connected to some local program would cause a change in interactivity rating based on the current interactivity of the other process. EX:

gpm waits on /dev/input/mice, gets a high interactivity rating. X waits on /dev/gpmdata, gets a lesser but still significant rating.

This clearly has some flaws, but it may be useful in some scenarios.

Cheers,
Kyle Moffett

-----BEGIN GEEK CODE BLOCK-----
Version: 3.12
GCM/CS/IT/U d- s++: a17 C++++>$ UB/L/X/*++++(+)>$ P+++(++++)>$ L++++(+++) E W++(+) N+++(++) o? K? w--- O? M++ V? PS+() PE+(-) Y+ PGP+++ t+(+++) 5 X R? tv-(--) b++++(++) DI+ D+ G e->++++$ h!*()>++$ r !y?(-) ------END GEEK CODE BLOCK------

Cool! I've been toying with this too :)

I guess the one I wrote for staircase is inadequate too. Although in the field the implementation has been safe as far as I can tell.

I'm thinking of holding off for a bit to allow those current changes to be tried in -mm for a bit.

I have two more questions - there are already userspace tools and older out-of-tree kernels (inluding my current one) that use SCHED_BATCH and SCHED_ISO.

Should we respect the values for these policies and use numbering consistent with them (meaning SCHED_BATCH at 3 would be reserved but not used) or should we dish out values according to when they're implemented and demand userspace be updated.

Should we move to a policy bitmask numbering system and/or make SCHED_CPUBOUND, SCHED_ISO etc subpolicies of SCHED_NORMAL?

Regards,
Con