Skip to contentSkip to conversation

Starvation of one RT task when the runtime of another exceeds period.

DISCUSSION Daniel K. 2 participants 2 messages Jun 18, 2008

lore
Share & tools
lore

Daniel K.

author
#0 lore raw

I will demonstrate how to get an RT task stuck, and not rescheduled by (ab)using cgroups and RT scheduling. This is on a 4 core system running 2.6.26-rc6 with two patches applied to make it work at all.

http://marc.info/[email protected]
http://marc.info/?i=1213789854.16944.216.camel@twins

mkdir /dev/cgroup
mount -t cgroup -o cpu,cpuset cgroup /dev/cgroup

# Set up cgroup 0
mkdir /dev/cgroup/0
echo 3 > /dev/cgroup/0/cpuset.cpus
echo 0 > /dev/cgroup/0/cpuset.mems
echo 100000 > /dev/cgroup/0/cpu.rt_period_us
echo 5000 > /dev/cgroup/0/cpu.rt_runtime_us

# Set up cgroup 1
mkdir /dev/cgroup/1
echo 3 > /dev/cgroup/1/cpuset.cpus
echo 0 > /dev/cgroup/1/cpuset.mems
echo 100000 > /dev/cgroup/1/cpu.rt_period_us
echo 5000 > /dev/cgroup/1/cpu.rt_runtime_us

# Start task 1, and assign it to cgroup 0
schedtool -R -p 1 -e burnP6 &
[1] 3309
echo 3309 > /dev/cgroup/0/tasks

At this point task 1 use 20% CPU.

# Start task 2, and assign it to cgroup 1
schedtool -R -p 1 -e burnP6 &
[2] 3313
echo 3313 > /dev/cgroup/1/tasks

At this point task 2 use 20% CPU.
Both tasks use 40% of CPU core#3 in total.

# Assign an insane amount of runtime (over 100%, ref. my other mail) echo 30000 > /dev/cgroup/1/cpu.rt_runtime_us

Now, task 2 use 100% of the CPU, and completely starves task 1, which ceases to get scheduled.

# Cut down on the insanity
echo 5000 > /dev/cgroup/1/cpu.rt_runtime_us

Now task 2 use only 20% of the CPU again, task 1 does still not get scheduled.

Let's call this state 'stuck'

I can make task 1 get unstuck by assigning its PID to another cgroup.

# Kick task 1, so it gets scheduled again.
echo 3309 > /dev/cgroup/1/tasks

Assuming we go back to state 'stuck', a 'killall burnP6' will only kill task 2, task 1 is still waiting for someone to come and kick it in the butt. As soon as that happens, it will get killed as well.

One time even both tasks got stuck and did not get scheduled, and I needed to kick both tasks to get them going again.

Well, this wasn't really a question, but I'm sure this is not how it's supposed to behave?

Daniel K.

#1 lore raw
Quoted 62 lines · Daniel K.

I will demonstrate how to get an RT task stuck, and not rescheduled by (ab)using cgroups and RT scheduling. This is on a 4 core system running 2.6.26-rc6 with two patches applied to make it work at all.

http://marc.info/[email protected]
http://marc.info/?i=1213789854.16944.216.camel@twins

mkdir /dev/cgroup
mount -t cgroup -o cpu,cpuset cgroup /dev/cgroup

# Set up cgroup 0
mkdir /dev/cgroup/0
echo 3 > /dev/cgroup/0/cpuset.cpus
echo 0 > /dev/cgroup/0/cpuset.mems
echo 100000 > /dev/cgroup/0/cpu.rt_period_us
echo 5000 > /dev/cgroup/0/cpu.rt_runtime_us

# Set up cgroup 1
mkdir /dev/cgroup/1
echo 3 > /dev/cgroup/1/cpuset.cpus
echo 0 > /dev/cgroup/1/cpuset.mems
echo 100000 > /dev/cgroup/1/cpu.rt_period_us
echo 5000 > /dev/cgroup/1/cpu.rt_runtime_us

# Start task 1, and assign it to cgroup 0
schedtool -R -p 1 -e burnP6 &
[1] 3309
echo 3309 > /dev/cgroup/0/tasks

At this point task 1 use 20% CPU.

# Start task 2, and assign it to cgroup 1
schedtool -R -p 1 -e burnP6 &
[2] 3313
echo 3313 > /dev/cgroup/1/tasks

At this point task 2 use 20% CPU.
Both tasks use 40% of CPU core#3 in total.

# Assign an insane amount of runtime (over 100%, ref. my other mail) echo 30000 > /dev/cgroup/1/cpu.rt_runtime_us

Now, task 2 use 100% of the CPU, and completely starves task 1, which ceases to get scheduled.

# Cut down on the insanity
echo 5000 > /dev/cgroup/1/cpu.rt_runtime_us

Now task 2 use only 20% of the CPU again, task 1 does still not get scheduled.

Let's call this state 'stuck'

I can make task 1 get unstuck by assigning its PID to another cgroup.

# Kick task 1, so it gets scheduled again.
echo 3309 > /dev/cgroup/1/tasks

Assuming we go back to state 'stuck', a 'killall burnP6' will only kill task 2, task 1 is still waiting for someone to come and kick it in the butt. As soon as that happens, it will get killed as well.

Quoted 5 lines

One time even both tasks got stuck and did not get scheduled, and I needed to kick both tasks to get them going again.

Well, this wasn't really a question, but I'm sure this is not how it's supposed to behave?

Humm, indeed, I have a suspicion, but would need to think about this a bit - again thanks for the detailed report.