Skip to contentSkip to conversation

CFQ IOsched / CGROUPs: How can a process with realtime/0 IO priority starve for several seconds?

DISCUSSION marcel partap 1 participant 1 message Jun 13, 2011

lore
Share & tools
lore

marcel partap

author
#0 lore raw

Greetings Kernelings.
Today i have a very curious issue to bring forward that has been bothering me for some time. But because i just got myself my first basic CGROUP setup, i considered hickups like this to be a thing of the past: Under heavy IO load, mpd starts skipping, sometimes even for a few seconds. That with mpd being in my 'swift' CGROUP which has, as can be seen below, a blkio.weight of 1000 - double the default of 500 and 100 times 'idle' class processes like munin-graph get. Nevertheless, it repeatedly happens that multiple processes accumulate in IOWAIT stage. What i would expect is that the drastically higher blkio.weight would prevent mpd to starve on disk input (it needs 30k/s or something), instead it stutters a lot making me want to #bash my head against a #shell or sumthing ^^ When i noticed the mpd process showed up as besteffort/4 in iotop, i re-ioniced it to realtime/0. That did nothing to improve the situation! That's in line with my earlier notion that a process in IOWAIT seems not to be unblocked by re-ionicing it to realtime!? That really confuses me. Doesn't realtime IO scheduling mean that... ???

So how comes? *How can a process even with high blkio.weight and realtime/0 IO priority starve for several seconds?*

Thx for any insights and please CC me.

#best regards/marcel.

P.S.: i did notice i did not have the CONFIG_CFQ_GROUP_IOSCHED option set, and i am going to try that now. But i guess that doesn't quite touch my basic question.

appendix
--------

Quoted 18 lines

CONFIG_CGROUPS=y
CONFIG_CGROUP_FREEZER=y
CONFIG_CGROUP_DEVICE=y
CONFIG_CGROUP_CPUACCT=y
CONFIG_CGROUP_MEM_RES_CTLR=y
CONFIG_CGROUP_MEM_RES_CTLR_SWAP=y
CONFIG_CGROUP_MEM_RES_CTLR_SWAP_ENABLED=y
CONFIG_CGROUP_PERF=y
CONFIG_CGROUP_SCHED=y
CONFIG_FAIR_GROUP_SCHED=y
CONFIG_RT_GROUP_SCHED=y
CONFIG_BLK_CGROUP=y
CONFIG_DEBUG_BLK_CGROUP=y
CONFIG_SCHED_AUTOGROUP=y
CONFIG_IOSCHED_NOOP=y
CONFIG_IOSCHED_DEADLINE=y
CONFIG_IOSCHED_CFQ=y
# CONFIG_CFQ_GROUP_IOSCHED is not set

Quoted 8 lines

CONFIG_DEFAULT_IOSCHED="cfq"
CONFIG_SCHED_OMIT_FRAME_POINTER=y
CONFIG_SCHED_MC=y
CONFIG_SCHED_HRTICK=y
CONFIG_NET_SCHED=y
CONFIG_SCHED_DEBUG=y
CONFIG_SCHEDSTATS=y
CONFIG_SCHED_TRACER=y

a screenshot of the situation: http://tinyurl.com/3-0-0-rc2-iosched-troubles-png (iotop in accumulative mode)

/etc/cgroup/cgconfig.conf

Quoted 35 lines
mount {
        blkio = /sys/fs/cgroup;
        cpu = /sys/fs/cgroup;
        memory = /sys/fs/cgroup;
}
group . {
        blkio {
                blkio.weight = 500;
        }
}
group idle {
        blkio {
                blkio.weight = 10;
        }
        cpu {
                cpu.shares = 512;
        }
        memory {
                memory.swappiness = 90;
        }
}
group swift {
        blkio {
                blkio.weight = 1000;
        }
        cpu {
                cpu.shares = 2048;
        }
        memory {
                memory.swappiness = 1;
        }
}