Skip to contentSkip to conversation

sched: low latency feedback to userspace

RFC Maximilian Krüger 2 participants 4 messages May 9, 2016

lore
Share & tools
lore
#0 lore raw

I am planning to extend the CFS as part of my master thesis. I want user-threads to allow to decide whether to enter a critical section or call sched_yield()
For tight synchronized workloads it might be useful, to only start certain short tasks, when they still can be completed in the current time slice without being interrupted by the scheduler.
Since low latency is key, my current plan is to use a shared-mapped page for signaling and only use a syscall for the setup. I'd be curious, if you might find this useful in general and if there is a chance for getting this accepted upstream, given my benchmarks can prove gives the intended benefits.

While these are very general questions, I have one detail question concerning how good the chances for getting this upstream are. Would you prefer, if I implemented the setup with an additional syscall, or as an extensions of existing syscalls (probably sched_setattr, sched_getattr). greetings,
Maximilian Krüger

#1 lore raw
Quoted 3 lines · Maximilian Krüger

I am planning to extend the CFS as part of my master thesis. I want user-threads to allow to decide whether to enter a critical section or call sched_yield()

sched_yield() for anything other than SCHED_FIFO / SCHED_DEADLINE is a 'bug'. That is, calling sched_yield() outside of those two cases is undefined behaviour and the kernel is free to eat your granny and set your pet on fire.

Quoted 8 lines

For tight synchronized workloads it might be useful, to only start certain short tasks, when they still can be completed in the current time slice without being interrupted by the scheduler.
Since low latency is key, my current plan is to use a shared-mapped page for signaling and only use a syscall for the setup. I'd be curious, if you might find this useful in general and if there is a chance for getting this accepted upstream, given my benchmarks can prove gives the intended benefits.

Schemes like this have been proposed many times (Google might find them for you in your favourite LKML archive) and shot down the same number of times.

Such proposals always end up needing to define a 'limit', which is artificial and subject to creep, also such soft preempt-disable or boost schemes have very open ended semantics. They also become useless if _everyone_ requests them at the same time -- something not unlikely since every userspace program thinks it is the most important thing under the sun.

Would something like:

http://lkml.kernel.org/r/20151027235635.16059.11630.stgit@pjt-glaptop.roam.corp.google.com

and

http://lkml.kernel.org/r/[email protected]

work for what you want to achieve? If not; please explain in more detail why you want what you propose.

#2 lore raw
Quoted 2 lines

sched_yield() for anything other than SCHED_FIFO / SCHED_DEADLINE is a 'bug'. That is, calling sched_yield() outside of those two cases is undefined behaviour and the kernel is free to eat your granny and set your pet on fire.

okay, fair.c/yield_task_fair() does not exactly sound, as if it would set my granny on fire or eat my pet, nor does man 2 yield, but correct me if I'm wrong.

Quoted 10 lines

Schemes like this have been proposed many times (Google might find them for you in your favourite LKML archive) and shot down the same number of times.

Such proposals always end up needing to define a 'limit', which is artificial and subject to creep, also such soft preempt-disable or boost schemes have very open ended semantics. They also become useless if _everyone_ requests them at the same time -- something not unlikely since every userspace program thinks it is the most important thing under the sun.

I'm totally with you in this point, which is, why I under no circumstances will prevent
preemption or sacrifice the scheduler fairness, so these tasks might run longer uninterrupted, even though this still is up to discussion. Which is, why I would argue, that there would not be a benefit for tasks to use this, just because they think that they are more important than anyone else.

Quoted 4 lines

Would something like:

Quoted 8 lines
   http://lkml.kernel.org/r/20151027235635.16059.11630.stgit@pjt-glaptop.roam.corp.google.com

and

   http://lkml.kernel.org/r/[email protected]

work for what you want to achieve? If not; please explain in more detail why you want what you propose.

the second patchset actually looks useful to me, but I very much agree with the comments, that this thing looks overly complicated. So yes, the interface I have in mind will be similar on an abstract level, but I don't intend any generic interface or anything close to this complexity.

thanks for the the quick feedback,
Max

#3 lore raw
Quoted 9 lines · Maximilian Krüger
Quoted 5 lines

sched_yield() for anything other than SCHED_FIFO / SCHED_DEADLINE is a 'bug'. That is, calling sched_yield() outside of those two cases is undefined behaviour and the kernel is free to eat your granny and set your pet on fire.

okay, fair.c/yield_task_fair() does not exactly sound, as if it would set my granny on fire or eat my pet, nor does man 2 yield, but correct me if I'm wrong.

So barring the co-operative multitasking model (which is still somewhat employed in places), sched_yield() is basically undefined except for FIFO / DEADLINE.

Now, every OS does 'something', and we do too. But that something could really include anything from nothing to setting pet on fire.

That said; there is a lot of crufty code out there that calls sched_yield() for all sorts of reasons, but mostly bad ones.

I have seen code like:

	while (!event)
		sched_yield();

to wait for completion of 'event'. Now imagine what would happen if this code ends up running at a real-time priority?

And, no, you really cannot say that will never happen as an application developer, because the application user might start you with a different scheduling class (for whatever reason), or the system might have unconditional priority inheritance and you get caught in a boost chain, or whatever.

The worst part is; we've had people reject patches when we fixed their yield abuse for them :/