Interactivity improvements

I have a few ideas about the interactivity problem that I thought i would share.

First, the whole problem with interactive tasks turning into CPU hogs, and vice-versa. The current implementation is fairly good at estimating the interactivity of a new process isn't it? I assume it keeps some sort of statitistice on what a process has been doing in the past, so if a process starts to deviate from it's previous behavior quite a bit (say an interactive task starts finishing time slices, or a CPU hog starts sleeping a lot) couldn't we reset the priority, etc and let the interactivity estimator re-estimate the interactivity as if it was a new task. That way no task would get a real chance to starve others, while keeping interactive tasks interactive (interactive tasks that become CPU hogs for short peroids of time would have a pretty good chance to smarten up before they really get penalized).

Another point is compilers, they tend to do a lot of disk I/O then become major CPU hogs, could we have some sort or heuristic that reduces the bonuses for sleeping on block I/O rather than other kinds of I/O (say pipes and network I/O in the case of X). This would prevent tasks like compilers from getting real bonuses for reading alot of files at startup.

Finally, the interactivity estimator seems to be quite a bit of code, which certain people have no real useful (in servers for example) and I would imagine that it does reduce throughput, which is not a big deal in desktops, but in a server environment it's not good, so maybe a CONFIG_INTERACTIVE_ESTIMATOR or something similar would be an idea to keep the server people happy, just have an option to completely get rid of the extra overhead of having a really nice interactivity estimator. I could be an idiot though, and I imagine that I will be needing some asbestos for saying this, but I thought I would voice my opinion.

What about compilers chewing on source files coming in over NFS rather than resident on local block devices? The network waits need to be broken out into NFS versus other, or UDP versus TCP or something. e.g. waits due to the user not having typed anything yet, or moved the mouse, are going to be on TCP connections.

Richard \\\ SuperH Core+Debug Architect /// .. At home ..
  P.    /// [email protected]  ///  [email protected]
Curnow  \\\ http://www.superh.com/    ///  www.rc0.org.uk

Maybe if we had it reduce sleeping bonuses if it's waiting on filesystem access, this would cover NFS as the kernel does consider it a filesystem, this would cover SMB, AFS, etc as well.

We are already doing this.

Network interactivity is dealing with sockets, and such. Accessing NFS (and other network filesystems) deals with a virtual block device, and gives similar patterns to a local block device.

Patrick McLean <[email protected]> [2003-08-07]:

I'd be interested in whatever you come up with for this, as I use NFSS a lot.

-- wli

Well, it might be easy to separate user mode blocking in filesystem operations as opposed to usermode sleeping on file descriptors on a file system type base.

But maybe we do not need to differentiate anyway, perhaps we can somehow detect blocking reads which stress the hardware, vs. blocking reads which actually lowers the load. Perhaps we need a fd flag for that, where FDs pointing to fifo, socket and pipes will be free, and FDs pointing to some devices (e.g. not tty) and files will be tainted and get penalty on blocking.

Another optimisation would be to only penalt a blocking process if there is more than one, to avoid renicing processes on otherwise unloaded but slow systems.

Greetings
Bernd

eckes privat - http://www.eckes.org/
Project Freefire - http://www.freefire.org/

One comment on two processes bouncing a semaphor...

If certain kinds of sleep, such as sleeping on a semaphor, do not add any interactivity points, but the act of sleeping on a semaphor DOES cause SHARING of interactivity points, then those two processes would not be able to achieve interactive status based on their sleep patterns.

The next thing to do is to somehow notice that the process never sleeps on I/O or any other sleeping that WOULD indicate interactivity.

The situation is that both processes really ARE CPU hogs, but we don't detect that based on using up the timeslice. The solution is to track what kinds of sleep a process does (counters of the various sorts), which I'm sure already happens, but the difference is that you do not penalize for using up the time slice. You penalize for NOT doing interactive-indicating I/O.

This is really quite simple. All you're doing is you stop considering time-slice expiration in your heuristics, at least directly. You have a whole list of different sleep-reasons (Schlafgruende?), it's just that timeslice expiration isn't one of them, so whenever your sleep-reasons isn't in the list, you infer that it's timeslice expiration.

That means you have to catch all other reasons and make sure you've accounted for them, but you get the idea.

Simpler still would be to treat sleeping on a semaphor (or a pipe or some other IPC) as a timeslice expiration. That would be equivalent. Now, if the other process is interactive, then you have a fight on your hands.

Does the interactive process accumulate and share interactivity points fast enough to keep the other one moving well? If one process is interactive, then, REALLY, you would want the sleeping of the second process to be a don't-care. But how can you tell?

If one process is a CPU hog (or perceived as such), and it is blocking against another process, thus sharing interactivity points, does that mean the CPU hog shares negative points with the other process?

Please pardon my stream-of-consciousness style in this post.

In the past, I proposed to have at least 2 schedulers available (much like we have for I/O schedulers): one for servers, which doesn't mess with bonuses and interactivity too much and gives best throughput for batch processing (OLTP and in general, non-interactive loads), and another one for desktops or Terminal Servers.

Don't know if this is feasible, however.

Actually, it should improve I/O throughput. What it might hurt is computational performance, but only at the expense of benefiting other processes.

The reason it benefits throughput is that file I/O is definitely marked interactive, and that results in file I/O being able to quickly wake up, dispatch the I/O, and go back to sleep. Its the usual treatment given to I/O, and it works.

Robert Love