Summary

Peter, at al.

The follow ups to this mail contain the changes to man-pages for the new sched_setattr() and sched_getattr() system calls, as well as the changes to the sched(7) man page to document the SCHED_DEADLINE policy, all new changes in kernel 3.14.

I used the text you submitted, but _heavily_ edited it, and added a _lot_ of further details. Therefore, could you (and Juri, and Dario?) please carefully review the text.

The follow-up mails are:

[1/2] The sched_setattr.2 page that documents sched_setattr(2)
and sched_getattr(2).

[2/2] The revisions to the sched(7) page.

In each case, I've posted the formatted output for easy reading, and attached the page source to the message.

Thanks,

Michael

Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/

Hello Peter et al.

Here is the sched_attr.2 page as rendered text, with the raw page source attached. Please carefully review.

Cheers,

Michael

NAME
sched_setattr, sched_getattr - set and get scheduling policy and attributes

SYNOPSIS

       #include <sched.h>

       int sched_setattr(pid_t pid, const struct sched_attr *attr,
                         unsigned int flags);

       int sched_getattr(pid_t pid, const struct sched_attr *attr,
                         unsigned int size, unsigned int flags);

DESCRIPTION
sched_setattr()
The sched_setattr() system call sets the scheduling policy and associated attributes for the thread whose ID is specified in pid. If pid equals zero, the scheduling policy and attributes of the calling thread will be set.

Currently, Linux supports the following "normal" (i.e., non- real-time) scheduling policies as values that may be specified in policy:

SCHED_OTHER the standard round-robin time-sharing policy;

SCHED_BATCH for "batch" style execution of processes; and

SCHED_IDLE for running very low priority background jobs.

Various "real-time" policies are also supported, for special time-critical applications that need precise control over the way in which runnable threads are selected for execution. For the rules governing when a process may use these policies, see sched(7). The real-time policies that may be specified in pol‐ icy are:

SCHED_FIFO a first-in, first-out policy; and

SCHED_RR a round-robin policy.

Linux also provides the following policy:

SCHED_DEADLINE
a deadline scheduling policy; see sched(7) for details.

The attr argument is a pointer to a structure that defines the new scheduling policy and attributes for the specified thread. This structure has the following form:

struct sched_attr {
u32 size; /* Size of this structure */
u32 sched_policy; /* Policy (SCHED_*) */
u64 sched_flags; /* Flags */
s32 sched_nice; /* Nice value (SCHED_OTHER,
SCHED_BATCH) */
u32 sched_priority; /* Static priority (SCHED_FIFO,
SCHED_RR) */
/* Remaining fields are for SCHED_DEADLINE */ u64 sched_runtime;
u64 sched_deadline;
u64 sched_period;
};

The fields of this structure are as follows:

size This field should be set to the size of the structure in
bytes, as in sizeof(struct sched_attr). If the provided structure is smaller than the kernel structure, any
additional fields are assumed to be '0'. If the pro‐
vided structure is larger than the kernel structure, the kernel verifies that all additional fields are 0; if they are not, sched_setattr() fails with the error E2BIG and updates size to contain the size of the kernel structure.

The above behavior when the size of the user-space sched_attr structure does not match the size of the ker‐ nel structure allows for future extensibility of the interface. Malformed applications that pass oversize structures won't break in the future if the size of the
kernel sched_attr structure is increased. In the
future, it could also allow applications that know about a larger user-space sched_attr structure to determine whether they are running on an older kernel that does not support the larger structure.

sched_policy
This field specifies the scheduling policy, as one of the SCHED_* values listed above.

sched_flags
This field contains flags controlling scheduling behav‐
ior. Only one such flag is currently defined:
SCHED_FLAG_RESET_ON_FORK. As a result of including this flag, children created by fork(2) do not inherit privi‐ leged scheduling policies. See sched(7) for details.

sched_nice
This field specifies the nice value to be set when spec‐
ifying sched_policy as SCHED_OTHER or SCHED_BATCH. The
nice value is a number in the range -20 (high priority) to +19 (low priority); see setpriority(2).

sched_priority
This field specifies the static priority to be set when specifying sched_policy as SCHED_FIFO or SCHED_RR. The allowed range of priorities for these policies can be
determined using sched_get_priority_min(2) and
sched_get_priority_max(2). For other policies, this
field must be specified as 0.

sched_runtime
This field specifies the "Runtime" parameter for dead‐ line scheduling. The value is expressed in nanoseconds. This field, and the next two fields, are used only for SCHED_DEADLINE scheduling; for further details, see sched(7).

sched_deadline
This field specifies the "Deadline" parameter for dead‐ line scheduling. The value is expressed in nanoseconds.

sched_period
This field specifies the "Period" parameter for deadline scheduling. The value is expressed in nanoseconds.

The flags argument is provided to allow for future extensions to the interface; in the current implementation it must be specified as 0.

sched_getattr()
The sched_getattr() system call fetches the scheduling policy and the associated attributes for the thread whose ID is speci‐
fied in pid. If pid equals zero, the scheduling policy and
attributes of the calling thread will be retrieved.

The size argument should be set to the size of the sched_attr structure as known to user space. The value must be at least as large as the size of the initially published sched_attr structure, or the call fails with the error EINVAL.

The retrieved scheduling attributes are placed in the fields of the sched_attr structure pointed to by attr. The kernel sets attr.size to the size of its sched_attr structure.

If the caller-provided attr buffer is larger than the kernel's sched_attr structure, the additional bytes in the user-space structure are not touched. If the caller-provided structure is smaller than the kernel sched_attr structure and the kernel
needs to return values outside the provided space,
sched_getattr() fails with the error E2BIG. As with
sched_setattr(), these semantics allow for future extensibility of the interface.

The flags argument is provided to allow for future extensions to the interface; in the current implementation it must be specified as 0.

RETURN VALUE
On success, sched_setattr() and sched_getattr() return 0. On
error, -1 is returned, and errno is set to indicate the cause of the error.

ERRORS
sched_getattr() and sched_setattr() can both fail for the fol‐ lowing reasons:

EINVAL attr is NULL; or pid is negative; or flags is not zero.

ESRCH The thread whose ID is pid could not be found.

In addition, sched_getattr() can fail for the following rea‐ sons:

E2BIG The buffer specified by size and attr is too small.

EINVAL size is invalid; that is, it is smaller than the initial
version of the sched_attr structure (48 bytes) or larger than the system page size.

In addition, sched_setattr() can fail for the following rea‐ sons:

E2BIG The buffer specified by size and attr is larger than the
kernel structure, and one or more of the excess bytes is nonzero.

EBUSY SCHED_DEADLINE admission control failure, see sched(7).

EINVAL attr.sched_policy is not one of the recognized policies;
attr.sched_flags contains a flag other than
SCHED_FLAG_RESET_ON_FORK; or attr.sched_priority is
invalid; or attr.sched_policy is SCHED_DEADLINE and the deadline scheduling parameters in attr are invalid.

EPERM The caller does not have appropriate privileges.

EPERM The caller's CPU affinity mask does not include all CPUs
in the system (see sched_setaffinity(2)).

VERSIONS
These system calls first appeared in Linux 3.14.

CONFORMING TO
These system calls are nonstandard Linux extensions.

NOTES
sched_setattr() provides a superset of the functionality of sched_setscheduler(2), sched_setparam(2), nice(2), and (other than the ability to set the priority of all processes belonging to a specified user or all processes in a specified group) set‐ priority(2). Analogously, sched_getattr() provides a superset
of the functionality of sched_getscheduler(2), sched_get‐
param(2), and (partially) getpriority(2).

BUGS
In Linux versions up to 3.15, sched_settattr() failed with the error EFAULT instead of E2BIG for the case described in ERRORS.

SEE ALSO

       nice(2), sched_get_priority_max(2), sched_get_priority_min(2),
       sched_getaffinity(2), sched_getscheduler(2), sched_getparam(2),
       sched_rr_get_interval(2), sched_setaffinity(2),
       sched_setscheduler(2), sched_setparam(2), sched_yield(2),
       setpriority(2), pthread_getschedparam(3),
       pthread_setschedparam(3), pthread_setschedprio(3),
       capabilities(7), cpuset(7), sched(7)

Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/

Hello Peter et al.

Here is the section of the sched(7) page that describes SCHED_DEADLINE, as rendered text, with the (entire) raw page source attached. Please carefully review.

Cheers,

Michael

SCHED_DEADLINE: Sporadic task model deadline scheduling
Since version 3.14, Linux provides a deadline scheduling pol‐ icy (SCHED_DEADLINE). This policy is currently implemented using GEDF (Global Earliest Deadline First) in conjunction with CBS (Constant Bandwidth Server). To set and fetch this policy and associated attributes, one must use the Linux-spe‐ cific sched_setattr(2) and sched_getattr(2) system calls.

A sporadic task is one that has a sequence of jobs, where each job is activated at most once per period. Each job also has a relative deadline, before which it should finish execu‐ tion, and a computation time, which is the CPU time necessary for executing the job. The moment when a task wakes up because a new job has to be executed is called the arrival time (also referred to as the request time or release time). The start time is the time at which a task starts its execu‐ tion. The absolute deadline is thus obtained by adding the relative deadline to the arrival time.

The following diagram clarifies these terms:

arrival/wakeup absolute deadline
| start time |
| | |
v v v
-----x--------xooooooooooooooooo-------x--------x---
|<- comp. time ->|
|<------- relative deadline ----->| |<-------------- period ------------------>|

When setting a SCHED_DEADLINE policy for a thread using sched_setattr(2), one can specify three parameters: Runtime,
Deadline, and Period. These parameters do not necessarily
correspond to the aforementioned terms: usual practice is to set Runtime to something bigger than the average computation time (or worst-case execution time for hard real-time tasks), Deadline to the relative deadline, and Period to the period of the task. Thus, for SCHED_DEADLINE scheduling, we have:

arrival/wakeup absolute deadline
| start time |
| | |
v v v
-----x--------xooooooooooooooooo-------x--------x---
|<-- Runtime --->|
|<----------- Deadline ---------->| |<-------------- Period ------------------>|

The three deadline-scheduling parameters correspond to the sched_runtime, sched_deadline, and sched_period fields of the
sched_attr structure; see sched_setattr(2). These fields
express value in nanoseconds. If sched_period is specified as 0, then it is made the same as sched_deadline.

The kernel requires that:

sched_runtime <= sched_deadline <= sched_period

In addition, under the current implementation, all of the parameter values must be at least 1024 (i.e., just over one microsecond, which is the resolution of the implementation). If any of these checks fails, sched_setattr(2) fails with the error EINVAL.

The CBS guarantees non-interference between tasks, by throt‐ tling threads that attempt to over-run their specified Run‐ time.

To ensure deadline scheduling guarantees, the kernel must prevent situations where the set of SCHED_DEADLINE threads is not feasible (schedulable) within the given constraints. The kernel thus performs an admittance test when setting or
changing SCHED_DEADLINE policy and attributes. This admis‐
sion test calculates whether the change is feasible; if it is not sched_setattr(2) fails with the error EBUSY.

For example, it is required (but not necessarily sufficient) for the total utilization to be less than or equal to the total number of CPUs available, where, since each thread can maximally run for Runtime per Period, that thread's utiliza‐ tion is its Runtime divided by its Period.

In order to fulfil the guarantees that are made when a thread is admitted to the SCHED_DEADLINE policy, SCHED_DEADLINE threads are the highest priority (user controllable) threads in the system; if any SCHED_DEADLINE thread is runnable, it will preempt any thread scheduled under one of the other policies.

A call to fork(2) by a thread scheduled under the SCHED_DEAD‐ LINE policy will fail with the error EAGAIN, unless the thread has its reset-on-fork flag set (see below).

A SCHED_DEADLINE thread that calls sched_yield(2) will yield the current job and wait for a new period to begin.

Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/

I've read (as careful as one still can after seeing too much of this) both texts and they appear to be good.

Thanks for polishing, they are indeed clearer than what I provided.

Hello,

                           |<-- Runtime   --->|

I originally drew this slightly bigger than comp. time above because we usually don't want tasks to be throttled in the average case. It's just a rule of thumb.

And below 2^63, as per the last bug fix we discussed.

Thanks,

- Juri
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

Hi,

Apart from nitpicks in 2/2, this looks great to me.

Thanks a lot!

Best,

- Juri

Hello Juri,

[...]

Ahh -- yes, I see now that I messed this up when I tweaked the ASCII art. Thanks for catching that. Fixed now.

Fixed.

Cheers,

Michael

Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Thanks for checking, Peter.

Good!

Cheers,

Michael

Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/

Thanks for checking the pages, Juri.

Cheers,

Michael

Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html