IO scheduler based IO controller V9
Subsystems (20)
Hi All,
Here is the V9 of the IO controller patches generated on top of 2.6.31-rc7.
For ease of patching, a consolidated patch is available here.
http://people.redhat.com/~vgoyal/io-controller/io-scheduler-based-io-controller-v9.patch
Changes from V8
===============
- Implemented bdi like congestion semantics for io group also. Now once an io group gets congested, we don't clear the congestion flag until number of requests goes below nr_congestion_off.
This helps in getting rid of Buffered write performance regression we were observing with io controller patches.
Gui, can you please test it and see if this version is better in terms of your buffered write tests.
- Moved some of the functions from blk-core.c to elevator-fq.c. This reduces CONFIG_GROUP_IOSCHED ifdefs in blk-core.c and code looks little more clean.
- Fixed issue of add_front where we go left on rb-tree if add_front is specified in case of preemption.
- Requeue async ioq after one round of dispatch. This helps emulationg CFQ behavior.
- Pulled in v11 of io tracking patches and modified config option so that if CONFIG_TRACK_ASYNC_CONTEXT is not enabled, blkio is not compiled in.
- Fixed some block tracepoints which were broken because of per group request list changes.
- Fixed some logging messages.
- Got rid of extra call to update_prio as pointed out by Jerome and Gui.
- Merged the fix from jerome for a crash while chaning prio.
- Got rid of redundant slice_start assignment as pointed by Gui.
- Merged a elv_ioq_nr_dispatched() cleanup from Gui.
- Fixed a compilation issue if CONFIG_BLOCK=n.
What problem are we trying to solve
===================================
Provide group IO scheduling feature in Linux along the lines of other resource controllers like cpu.
IOW, provide facility so that a user can group applications using cgroups and control the amount of disk time/bandwidth received by a group based on its weight.
How to solve the problem
=========================
Different people have solved the issue differetnly. At least there are now three patchsets available (including this one).
IO throttling
-------------
This is a bandwidth controller which keeps track of IO rate of a group and throttles the process in the group if it exceeds the user specified limit.
dm-ioband
---------
This is a proportional bandwidth controller implemented as device mapper driver and provides fair access in terms of amount of IO done (not in terms of disk time as CFQ does).
So one will setup one or more dm-ioband devices on top of physical/logical block device, configure the ioband device and pass information like grouping etc. Now this device will keep track of bios flowing through it and control the flow of bios based on group policies.
IO scheduler based IO controller
--------------------------------
Here I have viewed the problem of IO contoller as hierarchical group scheduling (along the lines of CFS group scheduling) issue. Currently one can view linux IO schedulers as flat where there is one root group and all the IO belongs to
that group.
This patchset basically modifies IO schedulers to also support hierarchical group scheduling. CFQ already provides fairness among different processes. I have extended it support group IO schduling. Also took some of the code out of CFQ and put in a common layer so that same group scheduling code can be used by noop, deadline and AS to support group scheduling.
Pros/Cons
=========
There are pros and cons to each of the approach. Following are some of the thoughts.
- IO throttling is a max bandwidth controller and not a proportional one. Additionaly it provides fairness in terms of amount of IO done (and not in terms of disk time as CFQ does).
Personally, I think that proportional weight controller is useful to more people than just max bandwidth controller. In addition, IO scheduler based controller can also be enhanced to do max bandwidth control, if need be.
- dm-ioband also provides fairness in terms of amount of IO done not in terms of disk time. So a seeky process can still run away with lot more disk time. Now this is an interesting question that how fairness among groups should be viewed and what is more relevant. Should fairness be based on amount of IO done or amount of disk time consumed as CFQ does. IO scheduler based controller provides fairness in terms of disk time used.
- IO throttling and dm-ioband both are second level controller. That is these controllers are implemented in higher layers than io schedulers. So they control the IO at higher layer based on group policies and later IO schedulers take care of dispatching these bios to disk.
Implementing a second level controller has the advantage of being able to provide bandwidth control even on logical block devices in the IO stack which don't have any IO schedulers attached to these. But they can also interefere with IO scheduling policy of underlying IO scheduler and change the effective behavior. Following are some of the issues which I think should be visible in second level controller in one form or other.
Prio with-in group
------------------
A second level controller can potentially interefere with behavior of different prio processes with-in a group. bios are buffered at higher layer in single queue and release of bios is FIFO and not proportionate to the ioprio of the process. This can result in a particular prio level not getting fair share.
Buffering at higher layer can delay read requests for more than slice idle period of CFQ (default 8 ms). That means, it is possible that we are waiting for a request from the queue but it is buffered at higher layer and then idle timer will fire. It means that queue will losse its share at the same time overall throughput will be impacted as we lost those 8 ms.
Read Vs Write
-------------
Writes can overwhelm readers hence second level controller FIFO release will run into issue here. If there is a single queue maintained then reads will suffer large latencies. If there separate queues for reads and writes then it will be hard to decide in what ratio to dispatch reads and writes as it is IO scheduler's decision to decide when and how much read/write to dispatch. This is another place where higher level controller will not be in sync with lower level io scheduler and can change the effective policies of underlying io scheduler.
Fairness in terms of disk time / size of IO
---------------------------------------------
An higher level controller will most likely be limited to providing fairness in terms of size of IO done and will find it hard to provide fairness in terms of disk time used (as CFQ provides between various prio levels). This is because only IO scheduler knows how much disk time a queue has used.
Not sure how useful it is to have fairness in terms of secotrs as CFQ has been providing fairness in terms of disk time. So a seeky application will still run away with lot of disk time and bring down the overall throughput of the the disk more than usual.
CFQ IO context Issues
---------------------
Buffering at higher layer means submission of bios later with the help of a worker thread. This changes the io context information at CFQ layer which assigns the request to submitting thread. Change of io context info again leads to issues of idle timer expiry and issue of a process not getting fair share and reduced throughput.
Throughput with noop, deadline and AS
---------------------------------------------
I think an higher level controller will result in reduced overall throughput (as compared to io scheduler based io controller) and more seeks with noop, deadline and AS.
The reason being, that it is likely that IO with-in a group will be related and will be relatively close as compared to IO across the groups. For example, thread pool of kvm-qemu doing IO for virtual machine. In case of higher level control, IO from various groups will go into a single queue at lower level controller and it might happen that IO is now interleaved (G1, G2, G1, G3, G4....) causing more seeks and reduced throughput. (Agreed that merging will help up to some extent but still....).
Instead, in case of lower level controller, IO scheduler maintains one queue per group hence there is no interleaving of IO between groups. And if IO is related with-in group, then we shoud get reduced number/amount of seek and higher throughput.
Latency can be a concern but that can be controlled by reducing the time slice length of the queue.
- IO scheduler based controller has the limitation that it works only with the bottom most devices in the IO stack where IO scheduler is attached. Now the question comes that how important/relevant it is to control bandwidth at higher level logical devices also. The actual contention for resources is at the leaf block device so it probably makes sense to do any kind of control there and not at the intermediate devices. Secondly probably it also means better use of available resources.
For example, assume a user has created a linear logical device lv0 using three underlying disks sda, sdb and sdc. Also assume there are two tasks T1 and T2 in two groups doing IO on lv0. Also assume that weights of groups are in the ratio of 2:1 so T1 should get double the BW of T2 on lv0 device.
T1 T2 \ / lv0 / | \ sda sdb sdc
Now if IO control is done at lv0 level, then if T1 is doing IO to only sda, and T2's IO is going to sdc. In this case there is no need of resource management as both the IOs don't have any contention where it matters. If we try to do IO control at lv0 device, it will not be an optimal usage of resources and will bring down overall throughput.
IMHO, IO scheduler based IO controller is a reasonable approach to solve the problem of group bandwidth control, and can do hierarchical IO scheduling more tightly and efficiently. But I am all ears to alternative approaches and suggestions how doing things can be done better.
TODO
====
- code cleanups, testing, bug fixing, optimizations, benchmarking etc...
- More testing to make sure there are no regressions in CFQ.
Open Issues
===========
- Currently for async requests like buffered writes, we get the io group information from the page instead of the task context. How important it is to determine the context from page?
Can we put all the pdflush threads into a separate group and control system wide buffered write bandwidth. Any buffered writes submitted by the process directly will any way go to right group.
If it is acceptable then we can drop all the code associated with async io context and that should simplify the patchset a lot.
Testing
=======
I have divided testing results in three sections.
- Latency
- Throughput and Fairness
- Group Fairness
Because I have enhanced CFQ to also do group scheduling, one of the concerns has been that existing CFQ should not regress at least in flat setup. If one creates groups and puts tasks in those, then this is new environment and some properties can change because groups have this additional requirement of providing isolation also.
Environment
==========
A 7200 RPM SATA drive with queue depth of 31. Ext3 filesystem.
Latency Testing
++++++++++++++++
Test1: fsync-test with torture test from linus as background writer
------------------------------------------------------------
I looked at Ext3 fsync latency thread and picked fsync-test from Theodore Ts'o and torture test from Linus as background writer to see how are the fsync completion latencies. Following are the results.
Vanilla CFQ IOC IOC (with map async)
=========== ================= ==================== fsync time: 0.2515 fsync time: 0.8580 fsync time: 0.0531 fsync time: 0.1082 fsync time: 0.1408 fsync time: 0.8907 fsync time: 0.2106 fsync time: 0.3228 fsync time: 0.2709 fsync time: 0.2591 fsync time: 0.0978 fsync time: 0.3198 fsync time: 0.2776 fsync time: 0.3035 fsync time: 0.0886 fsync time: 0.2530 fsync time: 0.0903 fsync time: 0.3035 fsync time: 0.2271 fsync time: 0.2712 fsync time: 0.0961 fsync time: 0.1057 fsync time: 0.3357 fsync time: 0.1048 fsync time: 0.1699 fsync time: 0.3175 fsync time: 0.2582 fsync time: 0.1923 fsync time: 0.2964 fsync time: 0.0876 fsync time: 0.1805 fsync time: 0.0971 fsync time: 0.2546 fsync time: 0.2944 fsync time: 0.2728 fsync time: 0.3059 fsync time: 0.1420 fsync time: 0.1079 fsync time: 0.2973 fsync time: 0.2650 fsync time: 0.3103 fsync time: 0.2032 fsync time: 0.1581 fsync time: 0.1987 fsync time: 0.2926 fsync time: 0.2656 fsync time: 0.3048 fsync time: 0.1934 fsync time: 0.2666 fsync time: 0.3092 fsync time: 0.2954 fsync time: 0.1272 fsync time: 0.0165 fsync time: 0.2952 fsync time: 0.2655 fsync time: 0.2827 fsync time: 0.2394 fsync time: 0.0147 fsync time: 0.0068 fsync time: 0.0454 fsync time: 0.2296 fsync time: 0.2923 fsync time: 0.2936 fsync time: 0.0069 fsync time: 0.3021 fsync time: 0.0397 fsync time: 0.2668 fsync time: 0.1032 fsync time: 0.2762 fsync time: 0.1932 fsync time: 0.0962 fsync time: 0.2946 fsync time: 0.1895 fsync time: 0.3545 fsync time: 0.0774 fsync time: 0.2577 fsync time: 0.2406 fsync time: 0.3027 fsync time: 0.4935 fsync time: 0.7193 fsync time: 0.2984 fsync time: 0.2804 fsync time: 0.3251 fsync time: 0.1057 fsync time: 0.2685 fsync time: 0.1001 fsync time: 0.3145 fsync time: 0.1946 fsync time: 0.2525 fsync time: 0.2992
IOC--> With IO controller patches applied. CONFIG_TRACK_ASYNC_CONTEXT=n IOC(map async) --> IO controller patches with CONFIG_TRACK_ASYNC_CONTEXT=y
If CONFIG_TRACK_ASYNC_CONTEXT=y, async requests are mapped to the group based on cgroup info stored in page otherwise these are mapped to the cgroup submitting task belongs to.
Notes:
- It looks like that max fsync time is a bit higher with IO controller patches. Wil dig more into it later.
Test2: read small files with multiple sequential readers (10) runnning
======================================================================
Took Ingo's small file reader test and ran it while 10 sequential readers were running.
Vanilla CFQ IOC (flat) IOC (10 readers in 10 groups) 0.12 seconds 0.11 seconds 1.62 seconds 0.05 seconds 0.05 seconds 1.18 seconds 0.05 seconds 0.05 seconds 1.17 seconds 0.03 seconds 0.04 seconds 1.18 seconds 1.15 seconds 1.17 seconds 1.29 seconds 1.18 seconds 1.16 seconds 1.17 seconds 1.17 seconds 1.16 seconds 1.17 seconds 1.18 seconds 1.15 seconds 1.28 seconds 1.17 seconds 1.15 seconds 1.17 seconds 1.16 seconds 1.18 seconds 1.18 seconds 1.15 seconds 1.15 seconds 1.17 seconds 1.17 seconds 1.15 seconds 1.18 seconds 1.17 seconds 1.15 seconds 1.17 seconds 1.17 seconds 1.16 seconds 1.18 seconds 1.17 seconds 1.15 seconds 1.17 seconds 0.04 seconds 0.04 seconds 1.18 seconds 1.17 seconds 1.16 seconds 1.17 seconds 1.18 seconds 1.15 seconds 1.17 seconds 1.18 seconds 1.15 seconds 1.28 seconds 1.18 seconds 1.15 seconds 1.18 seconds 1.17 seconds 1.16 seconds 1.18 seconds 1.17 seconds 1.18 seconds 1.17 seconds 1.17 seconds 1.15 seconds 1.17 seconds 1.16 seconds 1.16 seconds 1.17 seconds 1.17 seconds 1.15 seconds 1.17 seconds 1.16 seconds 1.15 seconds 1.17 seconds 1.15 seconds 1.15 seconds 1.18 seconds 1.18 seconds 1.16 seconds 1.17 seconds 1.16 seconds 1.16 seconds 1.17 seconds 1.17 seconds 1.16 seconds 1.17 seconds 1.16 seconds 1.16 seconds 1.17 seconds
In third column, 10 readers have been put into 10 groups instead of running into root group. Small file reader runs in to root group.
Notes: It looks like that here read latencies remain same as with vanilla CFQ.
Test3: read small files with multiple writers (8) runnning
==========================================================
Again running small file reader test with 8 buffered writers running with prio 0 to 7.
Latency results are in seconds. Tried to capture the output with multiple configurations of IO controller to see the effect.
Vanilla IOC IOC IOC IOC IOC IOC
(flat)(groups) (groups) (map) (map) (map)
(f=0) (f=1) (flat) (groups) (groups)
(f=0) (f=1)
0.25 0.03 0.31 0.25 0.29 1.25 0.39
0.27 0.28 0.28 0.30 0.41 0.90 0.80
0.25 0.24 0.23 0.37 0.27 1.17 0.24
0.14 0.14 0.14 0.13 0.15 0.10 1.11
0.14 0.16 0.13 0.16 0.15 0.06 0.58
0.16 0.11 0.15 0.12 0.19 0.05 0.14
0.03 0.17 0.12 0.17 0.04 0.12 0.12
0.13 0.13 0.13 0.14 0.03 0.05 0.05
0.18 0.13 0.17 0.09 0.09 0.05 0.07
0.11 0.18 0.16 0.18 0.14 0.05 0.12
0.28 0.14 0.15 0.15 0.13 0.02 0.04
0.16 0.14 0.14 0.12 0.15 0.00 0.13
0.14 0.13 0.14 0.13 0.13 0.02 0.02
0.13 0.11 0.12 0.14 0.15 0.06 0.01
0.27 0.28 0.32 0.24 0.25 0.01 0.01
0.14 0.15 0.18 0.15 0.13 0.06 0.02
0.15 0.13 0.13 0.13 0.13 0.00 0.04
0.15 0.13 0.15 0.14 0.15 0.01 0.05
0.11 0.17 0.15 0.13 0.13 0.02 0.00
0.17 0.13 0.17 0.12 0.18 0.39 0.01
0.18 0.16 0.14 0.16 0.14 0.89 0.47
0.13 0.13 0.14 0.04 0.12 0.64 0.78
0.16 0.15 0.19 0.11 0.16 0.67 1.17
0.04 0.12 0.14 0.04 0.18 0.67 0.63
0.03 0.13 0.17 0.11 0.15 0.61 0.69
0.15 0.16 0.13 0.14 0.13 0.77 0.66
0.12 0.12 0.15 0.11 0.13 0.92 0.73
0.15 0.12 0.15 0.16 0.13 0.70 0.73
0.11 0.13 0.15 0.10 0.18 0.73 0.82
0.16 0.19 0.15 0.16 0.14 0.71 0.74
0.28 0.05 0.26 0.22 0.17 2.91 0.79
0.13 0.05 0.14 0.14 0.14 0.44 0.65
0.16 0.22 0.18 0.13 0.26 0.31 0.65
0.10 0.13 0.12 0.11 0.16 0.25 0.66
0.13 0.14 0.16 0.15 0.12 0.17 0.76
0.19 0.11 0.12 0.14 0.17 0.20 0.71
0.16 0.15 0.14 0.15 0.11 0.19 0.68
0.13 0.13 0.13 0.13 0.16 0.04 0.78
0.14 0.16 0.15 0.17 0.15 1.20 0.80
0.17 0.13 0.14 0.18 0.14 0.76 0.63f(0/1)--> refers to "fairness" tunable. This is new tunable part of CFQ. It
set, we wait for requests from one queue to finish before new queue is scheduled in.
group ---> writers are running into individual groups and not in root group. map---> buffered writes are mapped to group using info stored in page.
Notes: Except the case of column 6 and 7 when writeres are in separate groups and we are mapping their writes to respective group, latencies seem to be fine. I think the latencies are higher for the last two cases because now the reader can't preempt the writer.
root / \ \ \ R G1 G2 G3 | | | W W W
Test4: Random Reader test in presece of 4 sequential readers and 4 buffered
writers
============================================================================
Used fio to this time to run one random reader and see how does it fair in the presence of 4 sequential readers and 4 writers.
I have just pasted the output of random reader from fio.
Vanilla Kernel, Three runs
--------------------------
read : io=20,512KiB, bw=349KiB/s, iops=10, runt= 60075msec clat (usec): min=944, max=2,675K, avg=93715.04, stdev=305815.90
read : io=13,696KiB, bw=233KiB/s, iops=7, runt= 60035msec clat (msec): min=2, max=1,812, avg=140.26, stdev=382.55
read : io=13,824KiB, bw=235KiB/s, iops=7, runt= 60185msec clat (usec): min=766, max=2,025K, avg=139310.55, stdev=383647.54
IO controller kernel, Three runs
--------------------------------
read : io=10,304KiB, bw=175KiB/s, iops=5, runt= 60083msec clat (msec): min=2, max=2,654, avg=186.59, stdev=524.08
read : io=10,176KiB, bw=173KiB/s, iops=5, runt= 60054msec clat (usec): min=792, max=2,567K, avg=188841.70, stdev=517154.75
read : io=11,040KiB, bw=188KiB/s, iops=5, runt= 60003msec clat (usec): min=779, max=2,625K, avg=173915.56, stdev=508118.60
Notes:
- Looks like vanilla CFQ gives a bit more disk access to random reader. Will dig into it.
Throughput and Fairness
+++++++++++++++++++++++
Test5: Bandwidth distribution between 4 sequential readers and 4 buffered
writers
==========================================================================
Used fio to launch 4 sequential readers and 4 buffered writers and watched how BW is distributed.
Vanilla kernel, Three sets
--------------------------
read : io=962MiB, bw=16,818KiB/s, iops=513, runt= 60008msec read : io=969MiB, bw=16,920KiB/s, iops=516, runt= 60077msec read : io=978MiB, bw=17,063KiB/s, iops=520, runt= 60096msec read : io=922MiB, bw=16,106KiB/s, iops=491, runt= 60057msec
write: io=235MiB, bw=4,099KiB/s, iops=125, runt= 60049msec
write: io=226MiB, bw=3,944KiB/s, iops=120, runt= 60049msec
write: io=215MiB, bw=3,747KiB/s, iops=114, runt= 60049msec
write: io=207MiB, bw=3,606KiB/s, iops=110, runt= 60049msec
READ: io=3,832MiB, aggrb=66,868KiB/s, minb=16,106KiB/s, maxb=17,063KiB/s, mint=60008msec, maxt=60096msec
WRITE: io=882MiB, aggrb=15,398KiB/s, minb=3,606KiB/s, maxb=4,099KiB/s, mint=60049msec, maxt=60049msec
read : io=1,002MiB, bw=17,513KiB/s, iops=534, runt= 60020msec read : io=979MiB, bw=17,085KiB/s, iops=521, runt= 60080msec read : io=953MiB, bw=16,637KiB/s, iops=507, runt= 60092msec read : io=920MiB, bw=16,057KiB/s, iops=490, runt= 60108msec
write: io=215MiB, bw=3,560KiB/s, iops=108, runt= 63289msec
write: io=136MiB, bw=2,361KiB/s, iops=72, runt= 60502msec
write: io=127MiB, bw=2,101KiB/s, iops=64, runt= 63289msec
write: io=233MiB, bw=3,852KiB/s, iops=117, runt= 63289msec
READ: io=3,855MiB, aggrb=67,256KiB/s, minb=16,057KiB/s, maxb=17,513KiB/s, mint=60020msec, maxt=60108msec
WRITE: io=711MiB, aggrb=11,771KiB/s, minb=2,101KiB/s, maxb=3,852KiB/s, mint=60502msec, maxt=63289msec
read : io=985MiB, bw=17,179KiB/s, iops=524, runt= 60149msec read : io=974MiB, bw=17,025KiB/s, iops=519, runt= 60002msec read : io=962MiB, bw=16,772KiB/s, iops=511, runt= 60170msec read : io=932MiB, bw=16,280KiB/s, iops=496, runt= 60057msec
write: io=177MiB, bw=2,933KiB/s, iops=89, runt= 63094msec
write: io=152MiB, bw=2,637KiB/s, iops=80, runt= 60323msec
write: io=240MiB, bw=3,983KiB/s, iops=121, runt= 63094msec
write: io=147MiB, bw=2,439KiB/s, iops=74, runt= 63094msec
READ: io=3,855MiB, aggrb=67,174KiB/s, minb=16,280KiB/s, maxb=17,179KiB/s, mint=60002msec, maxt=60170msec
WRITE: io=715MiB, aggrb=11,877KiB/s, minb=2,439KiB/s, maxb=3,983KiB/s, mint=60323msec, maxt=63094msec
IO controller kernel three sets
-------------------------------
read : io=944MiB, bw=16,483KiB/s, iops=503, runt= 60055msec read : io=941MiB, bw=16,433KiB/s, iops=501, runt= 60073msec read : io=900MiB, bw=15,713KiB/s, iops=479, runt= 60040msec read : io=866MiB, bw=15,112KiB/s, iops=461, runt= 60086msec
write: io=244MiB, bw=4,262KiB/s, iops=130, runt= 60040msec
write: io=177MiB, bw=3,085KiB/s, iops=94, runt= 60042msec
write: io=158MiB, bw=2,758KiB/s, iops=84, runt= 60041msec
write: io=180MiB, bw=3,137KiB/s, iops=95, runt= 60040msec
READ: io=3,651MiB, aggrb=63,718KiB/s, minb=15,112KiB/s, maxb=16,483KiB/s, mint=60040msec, maxt=60086msec
WRITE: io=758MiB, aggrb=13,243KiB/s, minb=2,758KiB/s, maxb=4,262KiB/s, mint=60040msec, maxt=60042msec
read : io=960MiB, bw=16,734KiB/s, iops=510, runt= 60137msec read : io=917MiB, bw=16,001KiB/s, iops=488, runt= 60122msec read : io=897MiB, bw=15,683KiB/s, iops=478, runt= 60004msec read : io=908MiB, bw=15,824KiB/s, iops=482, runt= 60149msec
write: io=209MiB, bw=3,563KiB/s, iops=108, runt= 61400msec
write: io=177MiB, bw=3,030KiB/s, iops=92, runt= 61400msec
write: io=200MiB, bw=3,409KiB/s, iops=104, runt= 61400msec
write: io=204MiB, bw=3,489KiB/s, iops=106, runt= 61400msec
READ: io=3,682MiB, aggrb=64,194KiB/s, minb=15,683KiB/s, maxb=16,734KiB/s, mint=60004msec, maxt=60149msec
WRITE: io=790MiB, aggrb=13,492KiB/s, minb=3,030KiB/s, maxb=3,563KiB/s, mint=61400msec, maxt=61400msec
read : io=968MiB, bw=16,867KiB/s, iops=514, runt= 60158msec read : io=925MiB, bw=16,135KiB/s, iops=492, runt= 60142msec read : io=875MiB, bw=15,286KiB/s, iops=466, runt= 60003msec read : io=872MiB, bw=15,221KiB/s, iops=464, runt= 60049msec
write: io=213MiB, bw=3,720KiB/s, iops=113, runt= 60162msec
write: io=203MiB, bw=3,536KiB/s, iops=107, runt= 60163msec
write: io=208MiB, bw=3,620KiB/s, iops=110, runt= 60162msec
write: io=203MiB, bw=3,538KiB/s, iops=107, runt= 60163msec
READ: io=3,640MiB, aggrb=63,439KiB/s, minb=15,221KiB/s, maxb=16,867KiB/s, mint=60003msec, maxt=60158msec
WRITE: io=827MiB, aggrb=14,415KiB/s, minb=3,536KiB/s, maxb=3,720KiB/s, mint=60162msec, maxt=60163msec
Notes: It looks like vanilla CFQ favors readers a bit more over writers as
compared to io controller cfq. Will dig into it.
Test6: Bandwidth distribution between readers of diff prio
==========================================================
Using fio, ran 8 readers of prio 0 to 7 and let it run for 30 seconds and watched for overall throughput and who got how much IO done.
Vanilla kernel, Three sets
---------------------------
read : io=454MiB, bw=15,865KiB/s, iops=484, runt= 30004msec read : io=382MiB, bw=13,330KiB/s, iops=406, runt= 30086msec read : io=325MiB, bw=11,330KiB/s, iops=345, runt= 30074msec read : io=294MiB, bw=10,253KiB/s, iops=312, runt= 30062msec read : io=238MiB, bw=8,321KiB/s, iops=253, runt= 30048msec read : io=145MiB, bw=5,061KiB/s, iops=154, runt= 30032msec read : io=99MiB, bw=3,456KiB/s, iops=105, runt= 30021msec read : io=67,040KiB, bw=2,280KiB/s, iops=69, runt= 30108msec
READ: io=2,003MiB, aggrb=69,767KiB/s, minb=2,280KiB/s, maxb=15,865KiB/s, mint=30004msec, maxt=30108msec
read : io=450MiB, bw=15,727KiB/s, iops=479, runt= 30001msec read : io=371MiB, bw=12,966KiB/s, iops=395, runt= 30040msec read : io=325MiB, bw=11,321KiB/s, iops=345, runt= 30099msec read : io=296MiB, bw=10,332KiB/s, iops=315, runt= 30086msec read : io=238MiB, bw=8,319KiB/s, iops=253, runt= 30056msec read : io=152MiB, bw=5,290KiB/s, iops=161, runt= 30070msec read : io=100MiB, bw=3,483KiB/s, iops=106, runt= 30020msec read : io=68,832KiB, bw=2,340KiB/s, iops=71, runt= 30118msec
READ: io=2,000MiB, aggrb=69,631KiB/s, minb=2,340KiB/s, maxb=15,727KiB/s, mint=30001msec, maxt=30118msec
read : io=450MiB, bw=15,691KiB/s, iops=478, runt= 30068msec read : io=369MiB, bw=12,882KiB/s, iops=393, runt= 30032msec read : io=364MiB, bw=12,732KiB/s, iops=388, runt= 30015msec read : io=283MiB, bw=9,889KiB/s, iops=301, runt= 30002msec read : io=228MiB, bw=7,935KiB/s, iops=242, runt= 30091msec read : io=144MiB, bw=5,018KiB/s, iops=153, runt= 30103msec read : io=97,760KiB, bw=3,327KiB/s, iops=101, runt= 30083msec read : io=66,784KiB, bw=2,276KiB/s, iops=69, runt= 30046msec
READ: io=1,999MiB, aggrb=69,625KiB/s, minb=2,276KiB/s, maxb=15,691KiB/s, mint=30002msec, maxt=30103msec
IO controller kernel, Three sets
--------------------------------
read : io=404MiB, bw=14,103KiB/s, iops=430, runt= 30072msec read : io=344MiB, bw=11,999KiB/s, iops=366, runt= 30035msec read : io=294MiB, bw=10,257KiB/s, iops=313, runt= 30052msec read : io=254MiB, bw=8,888KiB/s, iops=271, runt= 30021msec read : io=238MiB, bw=8,311KiB/s, iops=253, runt= 30086msec read : io=177MiB, bw=6,202KiB/s, iops=189, runt= 30001msec read : io=158MiB, bw=5,517KiB/s, iops=168, runt= 30118msec read : io=99MiB, bw=3,464KiB/s, iops=105, runt= 30107msec
READ: io=1,971MiB, aggrb=68,604KiB/s, minb=3,464KiB/s, maxb=14,103KiB/s, mint=30001msec, maxt=30118msec
read : io=375MiB, bw=13,066KiB/s, iops=398, runt= 30110msec read : io=326MiB, bw=11,409KiB/s, iops=348, runt= 30003msec read : io=308MiB, bw=10,758KiB/s, iops=328, runt= 30066msec read : io=256MiB, bw=8,937KiB/s, iops=272, runt= 30091msec read : io=232MiB, bw=8,088KiB/s, iops=246, runt= 30041msec read : io=192MiB, bw=6,695KiB/s, iops=204, runt= 30077msec read : io=144MiB, bw=5,014KiB/s, iops=153, runt= 30051msec read : io=96,224KiB, bw=3,281KiB/s, iops=100, runt= 30026msec
READ: io=1,928MiB, aggrb=67,145KiB/s, minb=3,281KiB/s, maxb=13,066KiB/s, mint=30003msec, maxt=30110msec
read : io=405MiB, bw=14,162KiB/s, iops=432, runt= 30021msec read : io=354MiB, bw=12,386KiB/s, iops=378, runt= 30007msec read : io=303MiB, bw=10,567KiB/s, iops=322, runt= 30062msec read : io=261MiB, bw=9,126KiB/s, iops=278, runt= 30040msec read : io=228MiB, bw=7,946KiB/s, iops=242, runt= 30048msec read : io=178MiB, bw=6,222KiB/s, iops=189, runt= 30074msec read : io=152MiB, bw=5,286KiB/s, iops=161, runt= 30093msec read : io=99MiB, bw=3,446KiB/s, iops=105, runt= 30110msec
READ: io=1,981MiB, aggrb=68,996KiB/s, minb=3,446KiB/s, maxb=14,162KiB/s, mint=30007msec, maxt=30110msec
Notes:
- It looks like overall throughput is 1-3% less in case of io controller.
- Bandwidth distribution between various prio levels has changed a bit. CFQ seems to have 100ms slice length for prio4 and then this slice increases by 20% for each prio level as prio increases and decreases by 20% as prio levels decrease. So Io controller does not seem to be doing too bad as in meeting that distribution.
Group Fairness
+++++++++++++++
Test7 (Isolation between two KVM virtual machines)
==================================================
Created two KVM virtual machines. Partitioned a disk on host in two partitions and gave one partition to each virtual machine. Put both the virtual machines in two different cgroup of weight 1000 and 500 each. Virtual machines created ext3 file system on the partitions exported from host and did buffered writes. Host seems writes as synchronous and virtual machine with higher weight gets double the disk time of virtual machine of lower weight. Used deadline scheduler in this test case.
Some more details about configuration are in documentation patch.
Test8 (Fairness for synchronous reads)
======================================
- Two dd in two cgroups with cgrop weights 1000 and 500. Ran two "dd" in those cgroups (With CFQ scheduler and /sys/block/<device>/queue/fairness = 1)
Higher weight dd finishes first and at that point of time my script takes care of reading cgroup files io.disk_time and io.disk_sectors for both the groups and display the results.
dd if=/mnt/$BLOCKDEV/zerofile1 of=/dev/null &
dd if=/mnt/$BLOCKDEV/zerofile2 of=/dev/null &
group1 time=8:16 2452 group1 sectors=8:16 457856 group2 time=8:16 1317 group2 sectors=8:16 247008
234179072 bytes (234 MB) copied, 3.90912 s, 59.9 MB/s 234179072 bytes (234 MB) copied, 5.15548 s, 45.4 MB/s
First two fields in time and sectors statistics represent major and minor number of the device. Third field represents disk time in milliseconds and number of sectors transferred respectively.
This patchset tries to provide fairness in terms of disk time received. group1 got almost double of group2 disk time (At the time of first dd finish). These time and sectors statistics can be read using io.disk_time and io.disk_sector files in cgroup. More about it in documentation file.
Test9 (Reader Vs Buffered Writes)
================================
Buffered writes can be problematic and can overwhelm readers, especially with noop and deadline. IO controller can provide isolation between readers and buffered (async) writers.
First I ran the test without io controller to see the severity of the issue. Ran a hostile writer and then after 10 seconds started a reader and then monitored the completion time of reader. Reader reads a 256 MB file. Tested this with noop scheduler.
sample script
------------
sync
echo 3 > /proc/sys/vm/drop_caches
time dd if=/dev/zero of=/mnt/sdb/reader-writer-zerofile bs=4K count=2097152 conv=fdatasync &
sleep 10
time dd if=/mnt/sdb/256M-file of=/dev/null &
Results
-------
8589934592 bytes (8.6 GB) copied, 106.045 s, 81.0 MB/s (Writer) 268435456 bytes (268 MB) copied, 96.5237 s, 2.8 MB/s (Reader)
Now it was time to test io controller whether it can provide isolation between readers and writers with noop. I created two cgroups of weight 1000 each and put reader in group1 and writer in group 2 and ran the test again. Upon comletion of reader, my scripts read io.disk_time and io.disk_sectors cgroup files to get an estimate how much disk time each group got and how many sectors each group did IO for.
For more accurate accounting of disk time for buffered writes with queuing hardware I had to set /sys/block/<disk>/queue/iosched/fairness to "1".
sample script
-------------
echo $$ > /cgroup/bfqio/test2/tasks
dd if=/dev/zero of=/mnt/$BLOCKDEV/testzerofile bs=4K count=2097152 & sleep 10
echo noop > /sys/block/$BLOCKDEV/queue/scheduler
echo 1 > /sys/block/$BLOCKDEV/queue/iosched/fairness echo $$ > /cgroup/bfqio/test1/tasks
dd if=/mnt/$BLOCKDEV/256M-file of=/dev/null &
wait $!
# Some code for reading cgroup files upon completion of reader.
-------------------------
Results
=======
268435456 bytes (268 MB) copied, 6.92248 s, 38.8 MB/s
group1 time=8:16 3185 group1 sectors=8:16 524824
group2 time=8:16 3190 group2 sectors=8:16 503848
Note, reader finishes now much lesser time and both group1 and group2 got almost 3 seconds of disk time. Hence io-controller provides isolation from buffered writes.
Test10 (AIO)
===========
AIO reads
-----------
Set up two fio, AIO read jobs in two cgroup with weight 1000 and 500 respectively. I am using cfq scheduler. Following are some lines from my test script.
---------------------------------------------------------------
echo 1000 > /cgroup/bfqio/test1/io.weight
echo 500 > /cgroup/bfqio/test2/io.weight
fio_args="--ioengine=libaio --rw=read --size=512M --direct=1" echo 1 > /sys/block/$BLOCKDEV/queue/iosched/fairness
echo $$ > /cgroup/bfqio/test1/tasks
fio $fio_args --name=test1 --directory=/mnt/$BLOCKDEV/fio1/ --output=/mnt/$BLOCKDEV/fio1/test1.log --exec_postrun="../read-and-display-group-stats.sh $maj_dev $minor_dev" &
echo $$ > /cgroup/bfqio/test2/tasks
fio $fio_args --name=test2 --directory=/mnt/$BLOCKDEV/fio2/ --output=/mnt/$BLOCKDEV/fio2/test2.log &
----------------------------------------------------------------
test1 and test2 are two groups with weight 1000 and 500 respectively. "read-and-display-group-stats.sh" is one small script which reads the test1 and test2 cgroup files to determine how much disk time each group got till first fio job finished.
Results
------
test1 statistics: time=8:16 17955 sectors=8:16 1049656 dq=8:16 2
test2 statistics: time=8:16 9217 sectors=8:16 602592 dq=8:16 1
Above shows that by the time first fio (higher weight), finished, group test1 got 17686 ms of disk time and group test2 got 9036 ms of disk time. similarly the statistics for number of sectors transferred are also shown.
Note that disk time given to group test1 is almost double of group2 disk time.
AIO writes
----------
Set up two fio, AIO direct write jobs in two cgroup with weight 1000 and 500 respectively. I am using cfq scheduler. Following are some lines from my test script.
------------------------------------------------
echo 1000 > /cgroup/bfqio/test1/io.weight
echo 500 > /cgroup/bfqio/test2/io.weight fio_args="--ioengine=libaio --rw=write --size=512M --direct=1"
echo 1 > /sys/block/$BLOCKDEV/queue/iosched/fairness
echo $$ > /cgroup/bfqio/test1/tasks
fio $fio_args --name=test1 --directory=/mnt/$BLOCKDEV/fio1/ --output=/mnt/$BLOCKDEV/fio1/test1.log --exec_postrun="../read-and-display-group-stats.sh $maj_dev $minor_dev" &
echo $$ > /cgroup/bfqio/test2/tasks
fio $fio_args --name=test2 --directory=/mnt/$BLOCKDEV/fio2/ --output=/mnt/$BLOCKDEV/fio2/test2.log &
-------------------------------------------------
test1 and test2 are two groups with weight 1000 and 500 respectively. "read-and-display-group-stats.sh" is one small script which reads the test1 and test2 cgroup files to determine how much disk time each group got till first fio job finished.
Following are the results.
test1 statistics: time=8:16 25452 sectors=8:16 1049664 dq=8:16 2
test2 statistics: time=8:16 12939 sectors=8:16 532184 dq=8:16 4
Above shows that by the time first fio (higher weight), finished, group test1 got almost double the disk time of group test2.
Test11 (Fairness for async writes, Buffered Write Vs Buffered Write)
===================================================================
Fairness for async writes is tricky and biggest reason is that async writes are cached in higher layers (page cahe) as well as possibly in file system layer also (btrfs, xfs etc), and are dispatched to lower layers not necessarily in proportional manner.
For example, consider two dd threads reading /dev/zero as input file and doing writes of huge files. Very soon we will cross vm_dirty_ratio and dd thread will be forced to write out some pages to disk before more pages can be dirtied. But not necessarily dirty pages of same thread are picked. It can very well pick the inode of lesser priority dd thread and do some writeout. So effectively higher weight dd is doing writeouts of lower weight dd pages and we don't see service differentation.
IOW, the core problem with buffered write fairness is that higher weight thread does not throw enought IO traffic at IO controller to keep the queue continuously backlogged. In my testing, there are many .2 to .8 second intervals where higher weight queue is empty and in that duration lower weight queue get lots of job done giving the impression that there was no service differentiation.
In summary, from IO controller point of view async writes support is there. Because page cache has not been designed in such a manner that higher prio/weight writer can do more write out as compared to lower prio/weight writer, gettting service differentiation is hard and it is visible in some cases and not visible in some cases.
Previous versions of the patches were posted here.
------------------------------------------------
(V1) http://lkml.org/lkml/2009/3/11/486
(V2) http://lkml.org/lkml/2009/5/5/275
(V3) http://lkml.org/lkml/2009/5/26/472
(V4) http://lkml.org/lkml/2009/6/8/580
(V5) http://lkml.org/lkml/2009/6/19/279
(V6) http://lkml.org/lkml/2009/7/2/369
(V7) http://lkml.org/lkml/2009/7/24/253
(V8) http://lkml.org/lkml/2009/8/16/204
Thanks
Vivek
PATCH 1/23 io-controller: Documentation
o Documentation for io-controller.
--- Documentation/block/00-INDEX | 2 + Documentation/block/io-controller.txt | 407 +++++++++++++++++++++++++++++++++ 2 files changed, 409 insertions(+), 0 deletions(-) create mode 100644 Documentation/block/io-controller.txt
Documentation/block/00-INDEX +2 -0
@@ -10,6 +10,8 @@ capability.txt - Generic Block Device Capability (/sys/block/<disk>/capability) deadline-iosched.txt - Deadline IO scheduler tunables +io-controller.txt + - IO controller for provding hierarchical IO scheduling ioprio.txt - Block io priorities (in CFQ scheduler) request.txt
Documentation/block/io-controller.txt new file +407 -0
407 lines not on this page · raw message
Signature · 1 line
1.6.0.6
A message that is not in the archive (sent off-list or not yet received): 1251495072-7780-1-git-send-email-vgoyal-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org
PATCH 1/23 io-controller: Documentation
o Documentation for io-controller.
--- Documentation/block/00-INDEX | 2 + Documentation/block/io-controller.txt | 407 +++++++++++++++++++++++++++++++++ 2 files changed, 409 insertions(+), 0 deletions(-) create mode 100644 Documentation/block/io-controller.txt
Documentation/block/00-INDEX +2 -0
@@ -10,6 +10,8 @@ capability.txt - Generic Block Device Capability (/sys/block/<disk>/capability) deadline-iosched.txt - Deadline IO scheduler tunables +io-controller.txt + - IO controller for provding hierarchical IO scheduling ioprio.txt - Block io priorities (in CFQ scheduler) request.txt
Documentation/block/io-controller.txt new file +407 -0
407 lines not on this page · raw message
Signature · 1 line
1.6.0.6
PATCH 2/23 io-controller: Core of the elevator fair queuing
In reply to a message not in the archiveo This is core of the io scheduler implemented at elevator layer. This is a mix of cpu CFS scheduler and CFQ IO scheduler. Some of the bits from CFS have to be derived so that we can support hierarchical scheduling. Without cgroups or with-in group, we should essentially get same behavior as CFQ.
o This patch only shows non-hierarchical bits. Hierarhical code comes in later patches.
o This code is the building base of introducing fair queuing logic in common elevator layer so that it can be used by all the four IO schedulers.
--- block/Makefile | 2 +- block/elevator-fq.c | 404 +++++++++++++++++++++++++++++++++++++++++++++++++++ block/elevator-fq.h | 148 +++++++++++++++++++ 3 files changed, 553 insertions(+), 1 deletions(-) create mode 100644 block/elevator-fq.c create mode 100644 block/elevator-fq.h
block/Makefile +1 -1
@@ -5,7 +5,7 @@ obj-$(CONFIG_BLOCK) := elevator.o blk-core.o blk-tag.o blk-sysfs.o \ blk-barrier.o blk-settings.o blk-ioc.o blk-map.o \ blk-exec.o blk-merge.o blk-softirq.o blk-timeout.o \ - ioctl.o genhd.o scsi_ioctl.o + ioctl.o genhd.o scsi_ioctl.o elevator-fq.o obj-$(CONFIG_BLK_DEV_BSG) += bsg.o obj-$(CONFIG_IOSCHED_NOOP) += noop-iosched.o
block/elevator-fq.c new file +404 -0
404 lines not on this page · raw message
block/elevator-fq.h new file +148 -0
148 lines not on this page · raw message
Signature · 1 line
1.6.0.6
PATCH 3/23 io-controller: Common flat fair queuing code in elevaotor layer
In reply to a message not in the archiveThis is common fair queuing code in elevator layer. This is controlled by config option CONFIG_ELV_FAIR_QUEUING. This patch initially only introduces flat fair queuing support where there is only one group, "root group" and all the tasks belong to root group.
This elevator layer changes are backward compatible. That means any ioscheduler using old interfaces will continue to work.
This is essentially a lot of CFQ logic moved into common layer so that other IO schedulers can make use of that in hierarhical scheduling setup.
--- block/Kconfig.iosched | 13 + block/Makefile | 3 +- block/as-iosched.c | 2 +- block/blk.h | 6 + block/cfq-iosched.c | 2 +- block/deadline-iosched.c | 3 +- block/elevator-fq.c | 985 ++++++++++++++++++++++++++++++++++++++++++++-- block/elevator-fq.h | 229 +++++++++++ block/elevator.c | 63 +++- block/noop-iosched.c | 2 +- include/linux/blkdev.h | 14 + include/linux/elevator.h | 50 +++- 12 files changed, 1330 insertions(+), 42 deletions(-)
block/Kconfig.iosched +13 -0
@@ -2,6 +2,19 @@ if BLOCK menu "IO Schedulers" +config ELV_FAIR_QUEUING + bool "Elevator Fair Queuing Support" + default n + ---help--- + Traditionally only cfq had notion of multiple queues and it did + fair queuing at its own. With the cgroups and need of controlling + IO, now even the simple io schedulers like noop, deadline, as will + have one queue per cgroup and will need hierarchical fair queuing. + Instead of every io scheduler implementing its own fair queuing + logic, this option enables fair queuing in elevator layer so that + other ioschedulers can make use of it. + If unsure, say N. + config IOSCHED_NOOP bool default y
block/Makefile +2 -1
@@ -5,7 +5,7 @@ obj-$(CONFIG_BLOCK) := elevator.o blk-core.o blk-tag.o blk-sysfs.o \ blk-barrier.o blk-settings.o blk-ioc.o blk-map.o \ blk-exec.o blk-merge.o blk-softirq.o blk-timeout.o \ - ioctl.o genhd.o scsi_ioctl.o elevator-fq.o + ioctl.o genhd.o scsi_ioctl.o obj-$(CONFIG_BLK_DEV_BSG) += bsg.o obj-$(CONFIG_IOSCHED_NOOP) += noop-iosched.o
@@ -15,3 +15,4 @@ obj-$(CONFIG_IOSCHED_CFQ) += cfq-iosched.o obj-$(CONFIG_BLOCK_COMPAT) += compat_ioctl.o obj-$(CONFIG_BLK_DEV_INTEGRITY) += blk-integrity.o +obj-$(CONFIG_ELV_FAIR_QUEUING) += elevator-fq.o
block/as-iosched.c +1 -1
@@ -1351,7 +1351,7 @@ static void as_exit_queue(struct elevator_queue *e)/* * initialize elevator private data (as_data). */ -static void *as_init_queue (struct request_queue *q) +static void *as_init_queue (struct request_queue *q,struct elevator_queue *eq) {struct as_data *ad;
block/blk.h +6 -0
@@ -1,6 +1,8 @@#ifndef BLK_INTERNAL_H #define BLK_INTERNAL_H +#include "elevator-fq.h" +/* Amount of time in which a process may batch requests */ #define BLK_BATCH_TIME (HZ/50UL)
@@ -71,6 +73,8 @@ static inline void elv_activate_rq(struct request_queue *q, struct request *rq) {struct elevator_queue *e = q->elevator; +elv_activate_rq_fair (q, rq); +if (e->ops->elevator_activate_req_fn) e->ops->elevator_activate_req_fn (q, rq); }
@@ -79,6 +83,8 @@ static inline void elv_deactivate_rq(struct request_queue *q, struct request *rq {struct elevator_queue *e = q->elevator; +elv_deactivate_rq_fair (q, rq); +if (e->ops->elevator_deactivate_req_fn) e->ops->elevator_deactivate_req_fn (q, rq); }
block/cfq-iosched.c +1 -1
@@ -2448,7 +2448,7 @@ static void cfq_exit_queue(struct elevator_queue *e)kfree (cfqd); } -static void *cfq_init_queue (struct request_queue *q) +static void *cfq_init_queue (struct request_queue *q,struct elevator_queue *eq) {struct cfq_data *cfqd;int i;
block/deadline-iosched.c +2 -1
@@ -347,7 +347,8 @@ static void deadline_exit_queue(struct elevator_queue *e)/* * initialize elevator private data (deadline_data). */ -static void *deadline_init_queue (struct request_queue *q) +static void * +deadline_init_queue (struct request_queue *q,struct elevator_queue *eq) {struct deadline_data *dd;
block/elevator-fq.c +961 -24
1,064 lines not on this page · raw message
block/elevator-fq.h +229 -0
270 lines not on this page · raw message
block/elevator.c +51 -12
138 lines not on this page · raw message
block/noop-iosched.c +1 -1
@@ -65,7 +65,7 @@ noop_latter_request(struct request_queue *q, struct request *rq) return list_entry(rq->queuelist.next, struct request, queuelist); } -static void *noop_init_queue (struct request_queue *q) +static void *noop_init_queue (struct request_queue *q,struct elevator_queue *eq) {struct noop_data *nd;
include/linux/blkdev.h +14 -0
@@ -229,6 +229,11 @@ struct request {/* for bidi */ struct request *next_rq; + +#ifdef CONFIG_ELV_FAIR_QUEUING +/* io queue request belongs to */ +struct io_queue *ioq; +#endif };static inline unsigned short req_get_ioprio (struct request *req)
@@ -236,6 +241,15 @@ static inline unsigned short req_get_ioprio(struct request *req)return req->ioprio; } +static inline struct io_queue *req_ioq (struct request *req) +{ +#ifdef CONFIG_ELV_FAIR_QUEUING +return req->ioq; +#else +return NULL ; +#endif +} +/* * State information carried for REQ_TYPE_PM_SUSPEND and REQ_TYPE_PM_RESUME * requests. Some step values could eventually be made generic.
include/linux/elevator.h +49 -1
80 lines not on this page · raw message
Signature · 1 line
1.6.0.6
PATCH 4/23 io-controller: Modify cfq to make use of flat elevator fair queuing
In reply to a message not in the archiveThis patch changes cfq to use fair queuing code from elevator layer.
--- block/Kconfig.iosched | 3 +- block/cfq-iosched.c | 980 +++++++++++-------------------------------------- 2 files changed, 217 insertions(+), 766 deletions(-)
block/Kconfig.iosched +2 -1
@@ -3,7 +3,7 @@ if BLOCK menu "IO Schedulers" config ELV_FAIR_QUEUING - bool "Elevator Fair Queuing Support" + bool default n ---help--- Traditionally only cfq had notion of multiple queues and it did
@@ -46,6 +46,7 @@ config IOSCHED_DEADLINE config IOSCHED_CFQ tristate "CFQ I/O scheduler" + select ELV_FAIR_QUEUING default y ---help--- The CFQ I/O scheduler tries to distribute bandwidth equally
block/cfq-iosched.c +215 -765
1,571 lines not on this page · raw message
Signature · 1 line
1.6.0.6
PATCH 5/23 io-controller: Core scheduler changes to support hierarhical scheduling
In reply to a message not in the archiveo This patch introduces core changes in fair queuing scheduler to support hierarhical/group scheduling. It is enabled by CONFIG_GROUP_IOSCHED.
--- block/elevator-fq.c | 158 ++++++++++++++++++++++++++++++++++++++++++++++++--- block/elevator-fq.h | 19 ++++++ init/Kconfig | 8 +++ 3 files changed, 177 insertions(+), 8 deletions(-)
block/elevator-fq.c +150 -8
@@ -137,6 +137,88 @@ static inline struct io_group *iog_of(struct io_entity *entity)return NULL ; } +#ifdef CONFIG_GROUP_IOSCHED +/* check for entity->parent so that loop is not executed for root entity. */ +#define for_each_entity (entity) \ + for (; entity && entity->parent; entity = entity->parent) + +/* Do the two (enqueued) entities belong to the same group ? */ +static inline int +is_same_group (struct io_entity *entity,struct io_entity *new_entity) +{ +if (parent_entity (entity) ==parent_entity (new_entity)) +return 1 ; + +return 0 ; +} + +/* return depth at which a io entity is present in the hierarchy */ +static inline int depth_entity (struct io_entity *entity) +{ +int depth =0 ; + +for_each_entity (entity) + depth++; + +return depth; +} + +static void find_matching_io_entity (struct io_entity **entity, +struct io_entity **new_entity) +{ +int entity_depth, new_entity_depth; + +/* +* preemption test can be made between sibling entities who are in the +* same group i.e who have a common parent. Walk up the hierarchy of +* both entities until we find their ancestors who are siblings of +* common parent. +*/ + +/* First walk up until both entities are at same depth */ + entity_depth =depth_entity (*entity); + new_entity_depth =depth_entity (*new_entity); + +while (entity_depth > new_entity_depth) { + entity_depth--; + *entity =parent_entity (*entity); + } + +while (new_entity_depth > entity_depth) { + new_entity_depth--; + *new_entity =parent_entity (*new_entity); + } + +while (!is_same_group (*entity, *new_entity)) { + *entity =parent_entity (*entity); + *new_entity =parent_entity (*new_entity); + } +} +struct io_group *ioq_to_io_group (struct io_queue *ioq) +{ +return iog_of (parent_entity (&ioq->entity)); +} +EXPORT_SYMBOL (ioq_to_io_group); + +static inline struct io_sched_data * +io_entity_sched_data (struct io_entity *entity) +{ +return &iog_of (parent_entity (entity))->sched_data; +} + +#else /* GROUP_IOSCHED */ +#define for_each_entity (entity) \ + for (; entity != NULL; entity = NULL) + +static void find_matching_io_entity (struct io_entity **entity, +struct io_entity **new_entity) { } + +static inline int +is_same_group (struct io_entity *entity,struct io_entity *new_entity) +{ +return 1 ; +} +static inline struct elv_fq_data *efqd_of (struct io_entity *entity) {return ioq_of (entity)->efqd;
@@ -155,6 +237,7 @@ io_entity_sched_data(struct io_entity *entity) return &efqd->root_group->sched_data; } +#endif /* GROUP_IOSCHED */ static inline void init_io_entity_service_tree (struct io_entity *entity,struct io_entity *parent)
@@ -171,8 +254,10 @@ static void entity_served(struct io_entity *entity, unsigned long served, unsigned long nr_sectors) { - entity->vdisktime += elv_delta_fair(served, entity); -update_min_vdisktime (entity->st); + for_each_entity(entity) { + entity->vdisktime += elv_delta_fair(served, entity); +update_min_vdisktime (entity->st); + } } static void place_entity(struct io_service_tree *st, struct io_entity *entity,
@@ -388,14 +473,23 @@ static void put_prev_ioq(struct io_queue *ioq) {struct io_entity *entity = &ioq->entity; -put_prev_io_entity (entity); +for_each_entity (entity) { +put_prev_io_entity (entity); + } }static void dequeue_ioq (struct io_queue *ioq) {struct io_entity *entity = &ioq->entity; -dequeue_io_entity (entity); +for_each_entity (entity) { +struct io_sched_data *sd =io_entity_sched_data (entity); + +dequeue_io_entity (entity); +/* Don't dequeue parent if it has other entities besides us */ +if (sd->nr_active) +break ; + }elv_put_ioq (ioq);return ; }
@@ -406,7 +500,12 @@ static void enqueue_ioq(struct io_queue *ioq)struct io_entity *entity = &ioq->entity;elv_get_ioq (ioq); -enqueue_io_entity (entity); + +for_each_entity (entity) { +if (entity->on_st) +break ; +enqueue_io_entity (entity); + } } static inline void
@@ -638,6 +737,38 @@ void elv_io_group_set_async_queue(struct io_group *iog, int ioprio_class, }EXPORT_SYMBOL (elv_io_group_set_async_queue); +#ifdef CONFIG_GROUP_IOSCHED + +static void io_free_root_group (struct elevator_queue *e) +{ +struct io_group *iog = e->efqd->root_group; + +put_io_group_queues (e, iog); +kfree (iog); +} + +static struct io_group *io_alloc_root_group (struct request_queue *q, +struct elevator_queue *e,void *key) +{ +struct io_group *iog; +int i; + + iog =kmalloc_node (sizeof (*iog),GFP_KERNEL | __GFP_ZERO, q->node); +if (iog ==NULL ) +return NULL ; + + iog->entity.parent =NULL ; + iog->entity.my_sd = &iog->sched_data; + iog->key = key; + +for (i =0 ; i <IO_IOPRIO_CLASSES ; i++) + iog->sched_data.service_tree[i] =ELV_SERVICE_TREE_INIT ; + +return iog; +} + +#else /* CONFIG_GROUP_IOSCHED */ +static struct io_group *io_alloc_root_group (struct request_queue *q,struct elevator_queue *e,void *key) {
@@ -666,6 +797,8 @@ static void io_free_root_group(struct elevator_queue *e) kfree(iog); } +#endif /* CONFIG_GROUP_IOSCHED */ +/* * Should be called after ioq prio and class has been initialized as prio * class data will be used to determine which service tree in the group
@@ -691,9 +824,11 @@ static struct io_queue *elv_get_next_ioq(struct request_queue *q)return NULL ; sd = &efqd->root_group->sched_data; - entity =lookup_next_io_entity (sd); -if (!entity) -return NULL ; +for (; sd !=NULL ; sd = entity->my_sd) { + entity =lookup_next_io_entity (sd); +if (!entity) +return NULL ; + } ioq =ioq_of (entity);return ioq;
@@ -894,6 +1029,13 @@ static int elv_should_preempt(struct request_queue *q, struct io_queue *new_ioq, new_entity = &new_ioq->entity;/* +* In hierarchical setup, one need to traverse up the hierarchy +* till both the queues are children of same parent to make a +* decision whether to do the preemption or not. +*/ +find_matching_io_entity (&entity, &new_entity); + +/* * Allow an RT request to pre-empt an ongoing non-RT cfqq timeslice. */
block/elevator-fq.h +19 -0
@@ -86,6 +86,23 @@ struct io_queue {void *sched_queue; }; +#ifdef CONFIG_GROUP_IOSCHED /* CONFIG_GROUP_IOSCHED */ +struct io_group { + struct io_entity entity; + atomic_t ref; + struct io_sched_data sched_data; + /* + * async queue for each priority case for RT and BE class. + * Used only for cfq. + */ + + struct io_queue *async_queue[2][IOPRIO_BE_NR]; + struct io_queue *async_idle_queue; + void *key; +}; + +#else /* CONFIG_GROUP_IOSCHED */ + struct io_group { struct io_entity entity; struct io_sched_data sched_data;
@@ -99,6 +116,8 @@ struct io_group {void *key; }; +#endif /* CONFIG_GROUP_IOSCHED */ +struct elv_fq_data {struct io_group *root_group;
init/Kconfig +8 -0
@@ -612,6 +612,14 @@ config CGROUP_MEM_RES_CTLR_SWAP Now, memory usage of swap_cgroup is 2 bytes per entry. If swap page size is 4096bytes, 512k per 1Gbytes of swap. +config GROUP_IOSCHED + bool "Group IO Scheduler" + depends on CGROUPS && ELV_FAIR_QUEUING + default n + ---help--- + This feature lets IO scheduler recognize task groups and control + disk bandwidth allocation to such task groups. + endif # CGROUPS config MM_OWNER
Signature · 1 line
1.6.0.6
PATCH 6/23 io-controller: cgroup related changes for hierarchical group support
In reply to a message not in the archiveo This patch introduces some of the cgroup related code for io controller.
--- block/blk-ioc.c | 3 + block/elevator-fq.c | 167 +++++++++++++++++++++++++++++++++++++++++ block/elevator-fq.h | 14 ++++ include/linux/cgroup_subsys.h | 6 ++ include/linux/iocontext.h | 5 + 5 files changed, 195 insertions(+), 0 deletions(-)
block/blk-ioc.c +3 -0
@@ -95,6 +95,9 @@ struct io_context *alloc_io_context(gfp_t gfp_flags, int node)spin_lock_init (&ret->lock); ret->ioprio_changed =0 ; ret->ioprio =0 ; +#ifdef CONFIG_GROUP_IOSCHED + ret->cgroup_changed =0 ; +#endif ret->last_waited = jiffies;/* doesn't matter... */ ret->nr_batch_requests =0 ;/* because this is 0 */ ret->aic =NULL ;
block/elevator-fq.c +167 -0
173 lines not on this page · raw message
block/elevator-fq.h +14 -0
@@ -13,6 +13,7 @@#ifdef CONFIG_BLOCK #include <linux/blkdev.h> +#include <linux/cgroup.h> #ifndef _ELV_SCHED_H#define _ELV_SCHED_H
@@ -91,6 +92,8 @@ struct io_group {struct io_entity entity;atomic_t ref;struct io_sched_data sched_data; +struct hlist_node group_node; +unsigned short iocg_id;/* * async queue for each priority case for RT and BE class. * Used only for cfq.
@@ -101,6 +104,17 @@ struct io_group {void *key; }; +struct io_cgroup { +struct cgroup_subsys_state css; + +unsigned int weight; +unsigned short ioprio_class; + +spinlock_t lock; +struct hlist_head group_data; +}; + +#else /* CONFIG_GROUP_IOSCHED */ struct io_group {
include/linux/cgroup_subsys.h +6 -0
@@ -60,3 +60,9 @@ SUBSYS(net_cls)#endif /* */ + +#ifdef CONFIG_GROUP_IOSCHED +SUBSYS(io) +#endif + +/* */
include/linux/iocontext.h +5 -0
@@ -73,6 +73,11 @@ struct io_context {unsigned short ioprio;unsigned short ioprio_changed; +#ifdef CONFIG_GROUP_IOSCHED +/* If task changes the cgroup, elevator processes it asynchronously */ +unsigned short cgroup_changed; +#endif +/* * For request batching */
Signature · 1 line
1.6.0.6
PATCH 7/23 io-controller: Common hierarchical fair queuing code in elevaotor layer
In reply to a message not in the archiveo This patch enables hierarchical fair queuing in common layer. It is controlled by config option CONFIG_GROUP_IOSCHED.
o Requests keep a reference on ioq and ioq keeps keep a reference on groups. For async queues in CFQ, and single ioq in other schedulers, io_group also keeps are reference on io_queue. This reference on ioq is dropped when the queue is released (elv_release_ioq). So the queue can be freed.
When a queue is released, it puts the reference to io_group and the io_group is released after all the queues are released. Child groups also take reference on parent groups, and release it when they are destroyed.
o Reads of iocg->group_data are not always iocg->lock; so all the operations on that list are still protected by RCU. All modifications to iocg->group_data should always done under iocg->lock.
Whenever iocg->lock and queue_lock can both be held, queue_lock should be held first. This avoids all deadlocks. In order to avoid race between cgroup deletion and elevator switch the following algorithm is used:
- Cgroup deletion path holds iocg->lock and removes iog entry to iocg->group_data list. Then it drops iocg->lock, holds queue_lock and destroys iog. So in this path, we never hold iocg->lock and queue_lock at the same time. Also, since we remove iog from iocg->group_data under iocg->lock, we can't race with elevator switch.
- Elevator switch path does not remove iog from iocg->group_data list directly. It first hold iocg->lock, scans iocg->group_data again to see if iog is still there; it removes iog only if it finds iog there. Otherwise, cgroup deletion must have removed it from the list, and cgroup deletion is responsible for removing iog.
So the path which removes iog from iocg->group_data list does the final removal of iog by calling __io_destroy_group() function.
--- block/cfq-iosched.c | 2 + block/elevator-fq.c | 479 +++++++++++++++++++++++++++++++++++++++++++++++++-- block/elevator-fq.h | 35 ++++ block/elevator.c | 4 + 4 files changed, 509 insertions(+), 11 deletions(-)
block/cfq-iosched.c +2 -0
@@ -1356,6 +1356,8 @@ alloc_cfqq:/* call it after cfq has initialized queue prio */ elv_init_ioq_io_group (ioq, iog); +/* ioq reference on iog */ +elv_get_iog (iog);cfq_log_cfqq (cfqd, cfqq,"alloced" ); }else { cfqq = &cfqd->oom_cfqq;
block/elevator-fq.c +468 -11
592 lines not on this page · raw message
block/elevator-fq.h +35 -0
@@ -93,6 +93,7 @@ struct io_group {atomic_t ref;struct io_sched_data sched_data;struct hlist_node group_node; +struct hlist_node elv_data_node;unsigned short iocg_id;/* * async queue for each priority case for RT and BE class.
@@ -102,6 +103,7 @@ struct io_group {struct io_queue *async_queue[2 ][IOPRIO_BE_NR ];struct io_queue *async_idle_queue;void *key; +struct rcu_head rcu_head; };struct io_cgroup {
@@ -135,6 +137,9 @@ struct io_group {struct elv_fq_data {struct io_group *root_group; +/* List of io groups hanging on this elevator */ +struct hlist_head group_list; +struct request_queue *queue;struct elevator_queue *eq;unsigned int busy_queues;
@@ -315,6 +320,28 @@ static inline struct io_queue *elv_get_oom_ioq(struct elevator_queue *eq)return &eq->efqd->oom_ioq; } +#ifdef CONFIG_GROUP_IOSCHED + +extern int elv_io_group_allow_merge (struct request *rq,struct bio *bio); +extern void elv_put_iog (struct io_group *iog); +extern struct io_group *elv_io_get_io_group (struct request_queue *q, +int create); + +static inline void elv_get_iog (struct io_group *iog) +{ +atomic_inc (&iog->ref); +} + +#else /* !GROUP_IOSCHED */ + +static inline int elv_io_group_allow_merge (struct request *rq,struct bio *bio) +{ +return 1 ; +} + +static inline void elv_get_iog (struct io_group *iog) {} +static inline void elv_put_iog (struct io_group *iog) {} +static inline struct io_group *elv_io_get_io_group (struct request_queue *q,int create) {
@@ -322,6 +349,8 @@ elv_io_get_io_group(struct request_queue *q, int create) return q->elevator->efqd->root_group; } +#endif /* GROUP_IOSCHED */ +extern ssize_t elv_slice_sync_show (struct elevator_queue *q,char *name);extern ssize_t elv_slice_sync_store (struct elevator_queue *q,const char *name,size_t count);
@@ -405,6 +434,12 @@ static inline void *elv_select_ioq(struct request_queue *q, int force) {return NULL ; } + +static inline int elv_io_group_allow_merge (struct request *rq,struct bio *bio) + +{ +return 1 ; +}#endif /* CONFIG_ELV_FAIR_QUEUING */ #endif /* _ELV_SCHED_H */ #endif /* CONFIG_BLOCK */
block/elevator.c +4 -0
@@ -122,6 +122,10 @@ int elv_rq_merge_ok(struct request *rq, struct bio *bio) !bio_failfast_driver(bio) != !blk_failfast_driver(rq))return 0 ; +/* If rq and bio belongs to different groups, dont allow merging */ +if (!elv_io_group_allow_merge (rq, bio)) +return 0 ; +if (!elv_iosched_allow_merge (rq, bio))return 0 ;
Signature · 1 line
1.6.0.6
PATCH 8/23 io-controller: cfq changes to use hierarchical fair queuing code in elevaotor layer
In reply to a message not in the archiveMake cfq hierarhical.
--- block/Kconfig.iosched | 8 ++++++ block/cfq-iosched.c | 63 ++++++++++++++++++++++++++++++++++++++++++++++-- init/Kconfig | 2 +- 3 files changed, 69 insertions(+), 4 deletions(-)
block/Kconfig.iosched +8 -0
@@ -54,6 +54,14 @@ config IOSCHED_CFQ working environment, suitable for desktop systems. This is the default I/O scheduler. +config IOSCHED_CFQ_HIER + bool "CFQ Hierarchical Scheduling support" + depends on IOSCHED_CFQ && CGROUPS + select GROUP_IOSCHED + default n + ---help--- + Enable hierarhical scheduling in cfq. + choice prompt "Default I/O scheduler" default DEFAULT_CFQ
block/cfq-iosched.c +60 -3
87 lines not on this page · raw message
init/Kconfig +1 -1
@@ -613,7 +613,7 @@ config CGROUP_MEM_RES_CTLR_SWAP size is 4096bytes, 512k per 1Gbytes of swap. config GROUP_IOSCHED - bool "Group IO Scheduler" + bool depends on CGROUPS && ELV_FAIR_QUEUING default n ---help---
Signature · 1 line
1.6.0.6
PATCH 9/23 io-controller: Export disk time used and nr sectors dipatched through cgroups
In reply to a message not in the archiveo This patch exports some statistics through cgroup interface. Two of the statistics currently exported are actual disk time assigned to the cgroup and actual number of sectors dispatched to disk on behalf of this cgroup.
--- block/elevator-fq.c | 76 +++++++++++++++++++++++++++++++++++++++++++++++++++ block/elevator-fq.h | 10 +++++++ 2 files changed, 86 insertions(+), 0 deletions(-)
block/elevator-fq.c +76 -0
@@ -13,6 +13,7 @@#include <linux/blkdev.h> #include <linux/blktrace_api.h> +#include <linux/seq_file.h> #include "elevator-fq.h" const int elv_slice_sync =HZ /10 ;
@@ -257,6 +258,8 @@ entity_served(struct io_entity *entity, unsigned long served,for_each_entity (entity) { entity->vdisktime +=elv_delta_fair (served, entity);update_min_vdisktime (entity->st); + entity->total_time += served; + entity->total_sectors += nr_sectors; } }
@@ -854,6 +857,66 @@ STORE_FUNCTION(weight, IO_WEIGHT_MIN, IO_WEIGHT_MAX);STORE_FUNCTION (ioprio_class,IOPRIO_CLASS_RT ,IOPRIO_CLASS_IDLE );#undef STORE_FUNCTION +static int io_cgroup_disk_time_read (struct cgroup *cgroup, +struct cftype *cftype,struct seq_file *m) +{ +struct io_cgroup *iocg; +struct io_group *iog; +struct hlist_node *n; + +if (!cgroup_lock_live_group (cgroup)) +return -ENODEV ; + + iocg =cgroup_to_io_cgroup (cgroup); + +rcu_read_lock (); +hlist_for_each_entry_rcu (iog, n, &iocg->group_data, group_node) { +/* +* There might be groups which are not functional and +* waiting to be reclaimed upon cgoup deletion. +*/ +if (iog->key) { +seq_printf (m,"%u:%u %lu\n" ,MAJOR (iog->dev), +MINOR (iog->dev), + iog->entity.total_time); + } + } +rcu_read_unlock (); +cgroup_unlock (); + +return 0 ; +} + +static int io_cgroup_disk_sectors_read (struct cgroup *cgroup, +struct cftype *cftype,struct seq_file *m) +{ +struct io_cgroup *iocg; +struct io_group *iog; +struct hlist_node *n; + +if (!cgroup_lock_live_group (cgroup)) +return -ENODEV ; + + iocg =cgroup_to_io_cgroup (cgroup); + +rcu_read_lock (); +hlist_for_each_entry_rcu (iog, n, &iocg->group_data, group_node) { +/* +* There might be groups which are not functional and +* waiting to be reclaimed upon cgoup deletion. +*/ +if (iog->key) { +seq_printf (m,"%u:%u %lu\n" ,MAJOR (iog->dev), +MINOR (iog->dev), + iog->entity.total_sectors); + } + } +rcu_read_unlock (); +cgroup_unlock (); + +return 0 ; +} +struct cftype io_files[] = { { .name ="weight" ,
@@ -865,6 +928,14 @@ struct cftype io_files[] = { .read_u64 = io_cgroup_ioprio_class_read, .write_u64 = io_cgroup_ioprio_class_write, }, + { + .name = "disk_time", + .read_seq_string = io_cgroup_disk_time_read, + }, + { + .name = "disk_sectors", + .read_seq_string = io_cgroup_disk_sectors_read, + }, };static int iocg_populate (struct cgroup_subsys *subsys,struct cgroup *cgroup)
@@ -956,6 +1027,8 @@ io_group_chain_alloc(struct request_queue *q, void *key, struct cgroup *cgroup)struct io_cgroup *iocg;struct io_group *iog, *leaf =NULL , *prev =NULL ;gfp_t flags =GFP_ATOMIC | __GFP_ZERO; +unsigned int major, minor; +struct backing_dev_info *bdi = &q->backing_dev_info;for (; cgroup !=NULL ; cgroup = cgroup->parent) { iocg =cgroup_to_io_cgroup (cgroup);
@@ -976,6 +1049,9 @@ io_group_chain_alloc(struct request_queue *q, void *key, struct cgroup *cgroup) iog->iocg_id =css_id (&iocg->css); +sscanf (dev_name (bdi->dev),"%u:%u" , &major, &minor); + iog->dev =MKDEV (major, minor); +io_group_init_entity (iocg, iog);atomic_set (&iog->ref,0 );
block/elevator-fq.h +10 -0
@@ -53,6 +53,13 @@ struct io_entity {unsigned short ioprio, ioprio_class;int ioprio_changed; + +/* +* Keep track of total service received by this entity. Keep the +* stats both for time slices and number of sectors dispatched +*/ +unsigned long total_time; +unsigned long total_sectors; };/*
@@ -104,6 +111,9 @@ struct io_group {struct io_queue *async_idle_queue;void *key;struct rcu_head rcu_head; + +/* The device MKDEV(major, minor), this group has been created for */ +dev_t dev; };struct io_cgroup {
Signature · 1 line
1.6.0.6
PATCH 10/23 io-controller: Debug hierarchical IO scheduling
In reply to a message not in the archiveo Littile debugging aid for hierarchical IO scheduling.
o Enabled under CONFIG_DEBUG_GROUP_IOSCHED
o Currently it outputs more debug messages in blktrace output which helps a great deal in debugging in hierarchical setup. It also creates additional cgroup interfaces io.disk_queue and io.disk_dequeue to output some more debugging data.
--- block/Kconfig.iosched | 8 +++ block/elevator-fq.c | 168 ++++++++++++++++++++++++++++++++++++++++++++++++- block/elevator-fq.h | 29 +++++++++ 3 files changed, 202 insertions(+), 3 deletions(-)
block/Kconfig.iosched +8 -0
@@ -90,6 +90,14 @@ config DEFAULT_IOSCHED default "cfq" if DEFAULT_CFQ default "noop" if DEFAULT_NOOP +config DEBUG_GROUP_IOSCHED + bool "Debug Hierarchical Scheduling support" + depends on CGROUPS && GROUP_IOSCHED + default n + ---help--- + Enable some debugging hooks for hierarchical scheduling support. + Currently it just outputs more information in blktrace output. + endmenu endif
block/elevator-fq.c +165 -3
243 lines not on this page · raw message
block/elevator-fq.h +29 -0
@@ -114,6 +114,16 @@ struct io_group {/* The device MKDEV(major, minor), this group has been created for */ dev_t dev; +#ifdef CONFIG_DEBUG_GROUP_IOSCHED +/* How many times this group has been added to active tree */ +unsigned long queue; + +/* How many times this group has been removed from active tree */ +unsigned long dequeue; + +/* Store cgroup path */ +char path[128 ]; +#endif };struct io_cgroup {
@@ -170,10 +180,29 @@ struct elv_fq_data { };/* Logging facilities. */ +#ifdef CONFIG_DEBUG_GROUP_IOSCHED +#define elv_log_ioq(efqd, ioq, fmt, args...) \ +{ \ + blk_add_trace_msg((efqd)->queue, "elv%d%c %s " fmt, (ioq)->pid, \ + elv_ioq_sync(ioq) ? 'S' : 'A', \ + ioq_to_io_group(ioq)->path, ##args); \ +} + +#define elv_log_iog(efqd, iog, fmt, args...) \ +{ \ + blk_add_trace_msg((efqd)->queue, "elv %s " fmt, (iog)->path, ##args); \ +} + +#else #define elv_log_ioq(efqd, ioq, fmt, args...) \ blk_add_trace_msg((efqd)->queue, "elv%d%c " fmt, (ioq)->pid, \ elv_ioq_sync(ioq) ? 'S' : 'A', ##args) +#define elv_log_iog(efqd, iog, fmt, args...) \ + blk_add_trace_msg((efqd)->queue, "elv " fmt, ##args) + +#endif + #define elv_log(efqd, fmt, args...) \ blk_add_trace_msg((efqd)->queue, "elv " fmt, ##args)
Signature · 1 line
1.6.0.6
PATCH 11/23 io-controller: Introduce group idling
In reply to a message not in the archiveo It is not always that IO from a process or group is continuous. There are cases of dependent reads where next read is not issued till previous read has finished. For such cases, CFQ introduced the notion of slice_idle, where we idle on the queue for sometime hoping next request will come and that's how fairness is provided otherwise queue will be deleted immediately from the service tree and this process will not get the fair share.
o This patch introduces the similar concept at group level. Idle on the group for a period of "group_idle" which is tunable through sysfs interface. So if a group is empty and about to be deleted, we idle for the next request.
o This patch also introduces the notion of wait busy where we wait for one extra group_idle period even if queue has consumed its time slice. The reason being that group will loose its share upon removal from service tree as some other entity will be picked for dispatch and vtime jump will take place.
--- block/cfq-iosched.c | 5 +- block/elevator-fq.c | 207 +++++++++++++++++++++++++++++++++++++++++++++++++-- block/elevator-fq.h | 44 +++++++++++- 3 files changed, 247 insertions(+), 9 deletions(-)
block/cfq-iosched.c +4 -1
@@ -980,7 +980,7 @@ static int cfq_dispatch_requests(struct request_queue *q, int force)*/ if (elv_nr_busy_ioq (q->elevator) >1 && ((!cfq_cfqq_sync (cfqq) && cfqq->slice_dispatch >=cfq_prio_to_maxrq (cfqd, cfqq)) || -cfq_class_idle (cfqq))) { + (cfq_class_idle (cfqq) && !elv_iog_should_idle (cfqq->ioq)))) {cfq_slice_expired (cfqd); }
@@ -2121,6 +2121,9 @@ static struct elv_fs_entry cfq_attrs[] = {CFQ_ATTR (slice_idle),ELV_ATTR (slice_sync),ELV_ATTR (slice_async), +#ifdef CONFIG_GROUP_IOSCHED +ELV_ATTR (group_idle), +#endif __ATTR_NULL };
block/elevator-fq.c +200 -7
359 lines not on this page · raw message
block/elevator-fq.h +43 -1
74 lines not on this page · raw message
Signature · 1 line
1.6.0.6
PATCH 12/23 io-controller: Wait for requests to complete from last queue before new queue is scheduled
In reply to a message not in the archiveo Currently one can dispatch requests from multiple queues to the disk. This is true for hardware which supports queuing. So if a disk support queue depth of 31 it is possible that 20 requests are dispatched from queue 1 and then next queue is scheduled in which dispatches more requests.
o This multiple queue dispatch introduces issues for accurate accounting of disk time consumed by a particular queue. For example, if one async queue is scheduled in, it can dispatch 31 requests to the disk and then it will be expired and a new sync queue might get scheduled in. These 31 requests might take a long time to finish but this time is never accounted to the async queue which dispatched these requests.
o This patch introduces the functionality where we wait for all the requests to finish from previous queue before next queue is scheduled in. That way a queue is more accurately accounted for disk time it has consumed. Note this still does not take care of errors introduced by disk write caching.
o Because above behavior can result in reduced throughput, this behavior will be enabled only if user sets "fairness" tunable to 1.
o This patch helps in achieving more isolation between reads and buffered writes in different cgroups. buffered writes typically utilize full queue depth and then expire the queue. On the contarary, sequential reads typicaly driver queue depth of 1. So despite the fact that writes are using more disk time it is never accounted to write queue because we don't wait for requests to finish after dispatching these. This patch helps do more accurate accounting of disk time, especially for buffered writes hence providing better fairness hence better isolation between two cgroups running read and write workloads.
--- block/cfq-iosched.c | 1 + block/elevator-fq.c | 21 ++++++++++++++++++++- block/elevator-fq.h | 10 +++++++++- 3 files changed, 30 insertions(+), 2 deletions(-)
block/cfq-iosched.c +1 -0
@@ -2123,6 +2123,7 @@ static struct elv_fs_entry cfq_attrs[] = { ELV_ATTR(slice_async),#ifdef CONFIG_GROUP_IOSCHED ELV_ATTR (group_idle), +ELV_ATTR (fairness), #endif __ATTR_NULL };
block/elevator-fq.c +20 -1
@@ -681,6 +681,8 @@ SHOW_FUNCTION(elv_slice_sync_show, efqd->elv_slice[1], 1);EXPORT_SYMBOL (elv_slice_sync_show);SHOW_FUNCTION (elv_slice_async_show, efqd->elv_slice[0 ],1 );EXPORT_SYMBOL (elv_slice_async_show); +SHOW_FUNCTION (elv_fairness_show, efqd->fairness,0 ); +EXPORT_SYMBOL (elv_fairness_show);#undef SHOW_FUNCTION#define STORE_FUNCTION (__FUNC, __PTR,MIN ,MAX , __CONV) \
@@ -705,6 +707,8 @@ STORE_FUNCTION(elv_slice_sync_store, &efqd->elv_slice[1], 1, UINT_MAX, 1);EXPORT_SYMBOL (elv_slice_sync_store);STORE_FUNCTION (elv_slice_async_store, &efqd->elv_slice[0 ],1 ,UINT_MAX ,1 );EXPORT_SYMBOL (elv_slice_async_store); +STORE_FUNCTION (elv_fairness_store, &efqd->fairness,0 ,1 ,0 ); +EXPORT_SYMBOL (elv_fairness_store);#undef STORE_FUNCTIONvoid elv_schedule_dispatch (struct request_queue *q)
@@ -2271,6 +2275,17 @@ void *elv_select_ioq(struct request_queue *q, int force) } expire: +if (efqd->fairness && !force && ioq && ioq->dispatched) { +/* +* If there are request dispatched from this queue, don't +* dispatch requests from new queue till all the requests from +* this queue have completed. +*/ +elv_log_ioq (efqd, ioq,"select: wait for requests to finish" +" disp=%lu" , ioq->dispatched); + ioq =NULL ; + goto keep_queue; + }elv_slice_expired (q); new_queue: ioq =elv_set_active_ioq (q, new_ioq);
@@ -2386,6 +2401,10 @@ void elv_ioq_completed_request(struct request_queue *q, struct request *rq) goto done; } +/* Wait for requests to finish from this queue */ +if (efqd->fairness &&elv_ioq_nr_dispatched (ioq)) + goto done; +/* Expire the queue */ elv_slice_expired (q); goto done;
@@ -2396,7 +2415,7 @@ void elv_ioq_completed_request(struct request_queue *q, struct request *rq)* If this is the last queue in the group and we did not * decide to idle on queue, idle on group. */ -if (elv_iog_should_idle (ioq) && !ioq->dispatched +if (elv_iog_should_idle (ioq) && !elv_ioq_nr_dispatched (ioq) && !timer_pending (&efqd->idle_slice_timer)) {/* * If queue has used up its slice, wait for the
block/elevator-fq.h +9 -1
@@ -180,6 +180,12 @@ struct elv_fq_data {/* Fallback dummy ioq for extreme OOM conditions */ struct io_queue oom_ioq; + +/* +* If set to 1, waits for all request completions from current +* queue before new queue is scheduled in +*/ +unsigned int fairness; };/* Logging facilities. */
@@ -437,7 +443,9 @@ extern ssize_t elv_slice_sync_store(struct elevator_queue *q, const char *name,extern ssize_t elv_slice_async_show (struct elevator_queue *q,char *name);extern ssize_t elv_slice_async_store (struct elevator_queue *q,const char *name,size_t count); - +extern ssize_t elv_fairness_show (struct elevator_queue *q,char *name); +extern ssize_t elv_fairness_store (struct elevator_queue *q,const char *name, +size_t count);/* Functions used by elevator.c */ extern struct elv_fq_data *elv_alloc_fq_data (struct request_queue *q,struct elevator_queue *e);
Signature · 1 line
1.6.0.6
PATCH 13/23 io-controller: Separate out queue and data
In reply to a message not in the archiveo So far noop, deadline and AS had one common structure called *_data which contained both the queue information where requests are queued and also common data used for scheduling. This patch breaks down this common structure in two parts, *_queue and *_data. This is along the lines of cfq where all the reuquests are queued in queue and common data and tunables are part of data.
o It does not change the functionality but this re-organization helps once noop, deadline and AS are changed to use hierarchical fair queuing.
o looks like queue_empty function is not required and we can check for q->nr_sorted in elevator layer to see if ioscheduler queues are empty or not.
--- block/as-iosched.c | 208 ++++++++++++++++++++++++++-------------------- block/deadline-iosched.c | 117 ++++++++++++++++---------- block/elevator.c | 111 +++++++++++++++++++++---- block/noop-iosched.c | 59 ++++++------- include/linux/elevator.h | 9 ++- 5 files changed, 320 insertions(+), 184 deletions(-)
block/as-iosched.c +118 -90
@@ -76,13 +76,7 @@ enum anticipation_status {* or timed out */ }; -struct as_data { -/* -* run time data -*/ - -struct request_queue *q;/* the "owner" queue */ - +struct as_queue {/* * requests (as_rq s) are present on both sort_list and fifo_list */
@@ -90,6 +84,14 @@ struct as_data {struct list_head fifo_list[2 ];struct request *next_rq[2 ];/* next in sort order */ +unsigned long last_check_fifo[2 ]; +int write_batch_count;/* max # of reqs in a write batch */ +int current_write_count;/* how many requests left this batch */ +int write_batch_idled;/* has the write batch gone idle? */ +}; + +struct as_data { +struct request_queue *q;/* the "owner" queue */ sector_t last_sector[2 ];/* last SYNC & ASYNC sectors */ unsigned long exit_prob;/* probability a task will exit while
@@ -103,21 +105,17 @@ struct as_data {sector_t new_seek_mean;unsigned long current_batch_expires; -unsigned long last_check_fifo[2 ];int changed_batch;/* 1: waiting for old batch to end */ int new_batch;/* 1: waiting on first read complete */ -int batch_data_dir;/* current batch SYNC / ASYNC */ -int write_batch_count;/* max # of reqs in a write batch */ -int current_write_count;/* how many requests left this batch */ -int write_batch_idled;/* has the write batch gone idle? */ enum anticipation_status antic_status;unsigned long antic_start;/* jiffies: when it started */ struct timer_list antic_timer;/* anticipatory scheduling timer */ -struct work_struct antic_work;/* Deferred unplugging */ +struct work_struct antic_work;/* Deferred unplugging */ struct io_context *io_context;/* Identify the expected process */ int ioc_finished;/* IO associated with io_context is finished */ int nr_dispatched; +int batch_data_dir;/* current batch SYNC / ASYNC */ /* * settings that change how the i/o scheduler behaves
@@ -258,13 +256,14 @@ static void as_put_io_context(struct request *rq)/* * rb tree support functions */ -#define RQ_RB_ROOT (ad, rq) (&(ad)->sort_list[rq_is_sync((rq))]) +#define RQ_RB_ROOT (asq, rq) (&(asq)->sort_list[rq_is_sync((rq))])static void as_add_rq_rb (struct as_data *ad,struct request *rq) {struct request *alias; +struct as_queue *asq =elv_get_sched_queue (ad->q, rq); -while ((unlikely (alias =elv_rb_add (RQ_RB_ROOT (ad, rq), rq)))) { +while ((unlikely (alias =elv_rb_add (RQ_RB_ROOT (asq, rq), rq)))) {as_move_to_dispatch (ad, alias);as_antic_stop (ad); }
@@ -272,7 +271,9 @@ static void as_add_rq_rb(struct as_data *ad, struct request *rq)static inline void as_del_rq_rb (struct as_data *ad,struct request *rq) { -elv_rb_del (RQ_RB_ROOT (ad, rq), rq); +struct as_queue *asq =elv_get_sched_queue (ad->q, rq); + +elv_rb_del (RQ_RB_ROOT (asq, rq), rq); }/*
@@ -366,7 +367,7 @@ as_choose_req(struct as_data *ad, struct request *rq1, struct request *rq2)* what request to process next. Anticipation works on top of this. */ static struct request * -as_find_next_rq (struct as_data *ad,struct request *last) +as_find_next_rq (struct as_data *ad,struct as_queue *asq,struct request *last) {struct rb_node *rbnext =rb_next (&last->rb_node);struct rb_node *rbprev =rb_prev (&last->rb_node);
@@ -382,7 +383,7 @@ as_find_next_rq(struct as_data *ad, struct request *last)else {const int data_dir =rq_is_sync (last); - rbnext =rb_first (&ad->sort_list[data_dir]); + rbnext =rb_first (&asq->sort_list[data_dir]);if (rbnext && rbnext != &last->rb_node) next =rb_entry_rq (rbnext); }
@@ -789,9 +790,10 @@ static int as_can_anticipate(struct as_data *ad, struct request *rq)static void as_update_rq (struct as_data *ad,struct request *rq) {const int data_dir =rq_is_sync (rq); +struct as_queue *asq =elv_get_sched_queue (ad->q, rq);/* keep the next_rq cache up to date */ - ad->next_rq[data_dir] =as_choose_req (ad, rq, ad->next_rq[data_dir]); + asq->next_rq[data_dir] =as_choose_req (ad, rq, asq->next_rq[data_dir]);/* * have we been anticipating this request?
@@ -812,25 +814,26 @@ static void update_write_batch(struct as_data *ad) {unsigned long batch = ad->batch_expire[BLK_RW_ASYNC ];long write_time; +struct as_queue *asq =elv_get_sched_queue (ad->q,NULL ); write_time = (jiffies - ad->current_batch_expires) + batch;if (write_time <0 ) write_time =0 ; -if (write_time > batch && !ad->write_batch_idled) { +if (write_time > batch && !asq->write_batch_idled) {if (write_time > batch *3 ) - ad->write_batch_count /=2 ; + asq->write_batch_count /=2 ;else - ad->write_batch_count--; - }else if (write_time < batch && ad->current_write_count ==0 ) { + asq->write_batch_count--; + }else if (write_time < batch && asq->current_write_count ==0 ) {if (batch > write_time *3 ) - ad->write_batch_count *=2 ; + asq->write_batch_count *=2 ;else - ad->write_batch_count++; + asq->write_batch_count++; } -if (ad->write_batch_count <1 ) - ad->write_batch_count =1 ; +if (asq->write_batch_count <1 ) + asq->write_batch_count =1 ; }/*
@@ -901,6 +904,7 @@ static void as_remove_queued_request(struct request_queue *q,const int data_dir =rq_is_sync (rq);struct as_data *ad = q->elevator->elevator_data;struct io_context *ioc; +struct as_queue *asq =elv_get_sched_queue (q, rq);WARN_ON (RQ_STATE (rq) !=AS_RQ_QUEUED );
@@ -914,8 +918,8 @@ static void as_remove_queued_request(struct request_queue *q,* Update the "next_rq" cache if we are about to remove its * entry */ -if (ad->next_rq[data_dir] == rq) - ad->next_rq[data_dir] =as_find_next_rq (ad, rq); +if (asq->next_rq[data_dir] == rq) + asq->next_rq[data_dir] =as_find_next_rq (ad, asq, rq);rq_fifo_clear (rq);as_del_rq_rb (ad, rq);
@@ -929,23 +933,23 @@ static void as_remove_queued_request(struct request_queue *q,* * See as_antic_expired comment. */ -static int as_fifo_expired (struct as_data *ad,int adir) +static int as_fifo_expired (struct as_data *ad,struct as_queue *asq,int adir) {struct request *rq;long delta_jif; - delta_jif = jiffies - ad->last_check_fifo[adir]; + delta_jif = jiffies - asq->last_check_fifo[adir];if (unlikely (delta_jif <0 )) delta_jif = -delta_jif;if (delta_jif < ad->fifo_expire[adir])return 0 ; - ad->last_check_fifo[adir] = jiffies; + asq->last_check_fifo[adir] = jiffies; -if (list_empty (&ad->fifo_list[adir])) +if (list_empty (&asq->fifo_list[adir]))return 0 ; - rq =rq_entry_fifo (ad->fifo_list[adir].next); + rq =rq_entry_fifo (asq->fifo_list[adir].next);return time_after (jiffies,rq_fifo_time (rq)); }
@@ -954,7 +958,7 @@ static int as_fifo_expired(struct as_data *ad, int adir)* as_batch_expired returns true if the current batch has expired. A batch * is a set of reads or a set of writes. */ -static inline int as_batch_expired (struct as_data *ad) +static inline int as_batch_expired (struct as_data *ad,struct as_queue *asq) {if (ad->changed_batch || ad->new_batch)return 0 ;
@@ -964,7 +968,7 @@ static inline int as_batch_expired(struct as_data *ad)return time_after (jiffies, ad->current_batch_expires);return time_after (jiffies, ad->current_batch_expires) - || ad->current_write_count ==0 ; + || asq->current_write_count ==0 ; }/*
@@ -973,6 +977,7 @@ static inline int as_batch_expired(struct as_data *ad)static void as_move_to_dispatch (struct as_data *ad,struct request *rq) {const int data_dir =rq_is_sync (rq); +struct as_queue *asq =elv_get_sched_queue (ad->q, rq);BUG_ON (RB_EMPTY_NODE (&rq->rb_node));
@@ -995,12 +1000,12 @@ static void as_move_to_dispatch(struct as_data *ad, struct request *rq) ad->io_context =NULL ; } -if (ad->current_write_count !=0 ) - ad->current_write_count--; +if (asq->current_write_count !=0 ) + asq->current_write_count--; } ad->ioc_finished =0 ; - ad->next_rq[data_dir] =as_find_next_rq (ad, rq); + asq->next_rq[data_dir] =as_find_next_rq (ad, asq, rq);/* * take it off the sort and fifo list, add to dispatch queue
@@ -1024,9 +1029,16 @@ static void as_move_to_dispatch(struct as_data *ad, struct request *rq)static int as_dispatch_request (struct request_queue *q,int force) {struct as_data *ad = q->elevator->elevator_data; -const int reads = !list_empty (&ad->fifo_list[BLK_RW_SYNC ]); -const int writes = !list_empty (&ad->fifo_list[BLK_RW_ASYNC ]);struct request *rq; +struct as_queue *asq =elv_select_sched_queue (q, force); +int reads, writes; + +if (!asq) +return 0 ; + + reads = !list_empty (&asq->fifo_list[BLK_RW_SYNC ]); + writes = !list_empty (&asq->fifo_list[BLK_RW_ASYNC ]); +if (unlikely (force)) {/*
@@ -1042,25 +1054,25 @@ static int as_dispatch_request(struct request_queue *q, int force) ad->changed_batch =0 ; ad->new_batch =0 ; -while (ad->next_rq[BLK_RW_SYNC ]) { -as_move_to_dispatch (ad, ad->next_rq[BLK_RW_SYNC ]); +while (asq->next_rq[BLK_RW_SYNC ]) { +as_move_to_dispatch (ad, asq->next_rq[BLK_RW_SYNC ]); dispatched++; } - ad->last_check_fifo[BLK_RW_SYNC ] = jiffies; + asq->last_check_fifo[BLK_RW_SYNC ] = jiffies; -while (ad->next_rq[BLK_RW_ASYNC ]) { -as_move_to_dispatch (ad, ad->next_rq[BLK_RW_ASYNC ]); +while (asq->next_rq[BLK_RW_ASYNC ]) { +as_move_to_dispatch (ad, asq->next_rq[BLK_RW_ASYNC ]); dispatched++; } - ad->last_check_fifo[BLK_RW_ASYNC ] = jiffies; + asq->last_check_fifo[BLK_RW_ASYNC ] = jiffies;return dispatched; }/* Signal that the write batch was uncontended, so we can't time it */ if (ad->batch_data_dir ==BLK_RW_ASYNC && !reads) { -if (ad->current_write_count ==0 || !writes) - ad->write_batch_idled =1 ; +if (asq->current_write_count ==0 || !writes) + asq->write_batch_idled =1 ; } if (!(reads || writes)
@@ -1069,14 +1081,14 @@ static int as_dispatch_request(struct request_queue *q, int force) || ad->changed_batch)return 0 ; -if (!(reads && writes &&as_batch_expired (ad))) { +if (!(reads && writes &&as_batch_expired (ad, asq))) {/* * batch is still running or no reads or no writes */ - rq = ad->next_rq[ad->batch_data_dir]; + rq = asq->next_rq[ad->batch_data_dir];if (ad->batch_data_dir ==BLK_RW_SYNC && ad->antic_expire) { -if (as_fifo_expired (ad,BLK_RW_SYNC )) +if (as_fifo_expired (ad, asq,BLK_RW_SYNC )) goto fifo_expired;if (as_can_anticipate (ad, rq)) {
@@ -1100,7 +1112,7 @@ static int as_dispatch_request(struct request_queue *q, int force)*/ if (reads) { -BUG_ON (RB_EMPTY_ROOT (&ad->sort_list[BLK_RW_SYNC ])); +BUG_ON (RB_EMPTY_ROOT (&asq->sort_list[BLK_RW_SYNC ]));if (writes && ad->batch_data_dir ==BLK_RW_SYNC )/*
@@ -1113,8 +1125,8 @@ static int as_dispatch_request(struct request_queue *q, int force) ad->changed_batch =1 ; } ad->batch_data_dir =BLK_RW_SYNC ; - rq =rq_entry_fifo (ad->fifo_list[BLK_RW_SYNC ].next); - ad->last_check_fifo[ad->batch_data_dir] = jiffies; + rq =rq_entry_fifo (asq->fifo_list[BLK_RW_SYNC ].next); + asq->last_check_fifo[ad->batch_data_dir] = jiffies; goto dispatch_request; }
@@ -1124,7 +1136,7 @@ static int as_dispatch_request(struct request_queue *q, int force)if (writes) { dispatch_writes: -BUG_ON (RB_EMPTY_ROOT (&ad->sort_list[BLK_RW_ASYNC ])); +BUG_ON (RB_EMPTY_ROOT (&asq->sort_list[BLK_RW_ASYNC ]));if (ad->batch_data_dir ==BLK_RW_SYNC ) { ad->changed_batch =1 ;
@@ -1137,10 +1149,10 @@ dispatch_writes: ad->new_batch =0 ; } ad->batch_data_dir =BLK_RW_ASYNC ; - ad->current_write_count = ad->write_batch_count; - ad->write_batch_idled =0 ; - rq =rq_entry_fifo (ad->fifo_list[BLK_RW_ASYNC ].next); - ad->last_check_fifo[BLK_RW_ASYNC ] = jiffies; + asq->current_write_count = asq->write_batch_count; + asq->write_batch_idled =0 ; + rq =rq_entry_fifo (asq->fifo_list[BLK_RW_ASYNC ].next); + asq->last_check_fifo[BLK_RW_ASYNC ] = jiffies; goto dispatch_request; }
@@ -1152,9 +1164,9 @@ dispatch_request:* If a request has expired, service it. */ -if (as_fifo_expired (ad, ad->batch_data_dir)) { +if (as_fifo_expired (ad, asq, ad->batch_data_dir)) { fifo_expired: - rq =rq_entry_fifo (ad->fifo_list[ad->batch_data_dir].next); + rq =rq_entry_fifo (asq->fifo_list[ad->batch_data_dir].next); }if (ad->changed_batch) {
@@ -1187,6 +1199,7 @@ static void as_add_request(struct request_queue *q, struct request *rq) {struct as_data *ad = q->elevator->elevator_data;int data_dir; +struct as_queue *asq =elv_get_sched_queue (q, rq);RQ_SET_STATE (rq,AS_RQ_NEW );
@@ -1205,7 +1218,7 @@ static void as_add_request(struct request_queue *q, struct request *rq)* set expire time and add to fifo list */ rq_set_fifo_time (rq, jiffies + ad->fifo_expire[data_dir]); -list_add_tail (&rq->queuelist, &ad->fifo_list[data_dir]); +list_add_tail (&rq->queuelist, &asq->fifo_list[data_dir]);as_update_rq (ad, rq);/* keep state machine up to date */ RQ_SET_STATE (rq,AS_RQ_QUEUED );
@@ -1227,31 +1240,20 @@ static void as_deactivate_request(struct request_queue *q, struct request *rq)atomic_inc (&RQ_IOC (rq)->aic->nr_dispatched); } -/* -* as_queue_empty tells us if there are requests left in the device. It may -* not be the case that a driver can get the next request even if the queue -* is not empty - it is used in the block layer to check for plugging and -* merging opportunities -*/ -static int as_queue_empty (struct request_queue *q) -{ -struct as_data *ad = q->elevator->elevator_data; - -return list_empty (&ad->fifo_list[BLK_RW_ASYNC ]) - &&list_empty (&ad->fifo_list[BLK_RW_SYNC ]); -} -static int as_merge (struct request_queue *q,struct request **req,struct bio *bio) { -struct as_data *ad = q->elevator->elevator_data;sector_t rb_key = bio->bi_sector +bio_sectors (bio);struct request *__rq; +struct as_queue *asq =elv_get_sched_queue_current (q); + +if (!asq) +return ELEVATOR_NO_MERGE ;/* * check for front merge */ - __rq =elv_rb_find (&ad->sort_list[bio_data_dir (bio)], rb_key); + __rq =elv_rb_find (&asq->sort_list[bio_data_dir (bio)], rb_key);if (__rq &&elv_rq_merge_ok (__rq, bio)) { *req = __rq;return ELEVATOR_FRONT_MERGE ;
@@ -1334,6 +1336,41 @@ static int as_may_queue(struct request_queue *q, int rw)return ret; } +/* Called with queue lock held */ +static void *as_alloc_as_queue (struct request_queue *q, +struct elevator_queue *eq,gfp_t gfp_mask) +{ +struct as_queue *asq; +struct as_data *ad = eq->elevator_data; + + asq =kmalloc_node (sizeof (*asq), gfp_mask | __GFP_ZERO, q->node); +if (asq ==NULL ) + goto out; + +INIT_LIST_HEAD (&asq->fifo_list[BLK_RW_SYNC ]); +INIT_LIST_HEAD (&asq->fifo_list[BLK_RW_ASYNC ]); + asq->sort_list[BLK_RW_SYNC ] =RB_ROOT ; + asq->sort_list[BLK_RW_ASYNC ] =RB_ROOT ; +if (ad) + asq->write_batch_count = ad->batch_expire[BLK_RW_ASYNC ] /10 ; +else + asq->write_batch_count = default_write_batch_expire /10 ; + +if (asq->write_batch_count <2 ) + asq->write_batch_count =2 ; +out: +return asq; +} + +static void as_free_as_queue (struct elevator_queue *e,void *sched_queue) +{ +struct as_queue *asq = sched_queue; + +BUG_ON (!list_empty (&asq->fifo_list[BLK_RW_SYNC ])); +BUG_ON (!list_empty (&asq->fifo_list[BLK_RW_ASYNC ])); +kfree (asq); +} +static void as_exit_queue (struct elevator_queue *e) {struct as_data *ad = e->elevator_data;
@@ -1341,9 +1378,6 @@ static void as_exit_queue(struct elevator_queue *e)del_timer_sync (&ad->antic_timer);cancel_work_sync (&ad->antic_work); -BUG_ON (!list_empty (&ad->fifo_list[BLK_RW_SYNC ])); -BUG_ON (!list_empty (&ad->fifo_list[BLK_RW_ASYNC ])); -put_io_context (ad->io_context);kfree (ad); }
@@ -1367,10 +1401,6 @@ static void *as_init_queue(struct request_queue *q, struct elevator_queue *eq)init_timer (&ad->antic_timer);INIT_WORK (&ad->antic_work, as_work_handler); -INIT_LIST_HEAD (&ad->fifo_list[BLK_RW_SYNC ]); -INIT_LIST_HEAD (&ad->fifo_list[BLK_RW_ASYNC ]); - ad->sort_list[BLK_RW_SYNC ] =RB_ROOT ; - ad->sort_list[BLK_RW_ASYNC ] =RB_ROOT ; ad->fifo_expire[BLK_RW_SYNC ] = default_read_expire; ad->fifo_expire[BLK_RW_ASYNC ] = default_write_expire; ad->antic_expire = default_antic_expire;
@@ -1378,9 +1408,6 @@ static void *as_init_queue(struct request_queue *q, struct elevator_queue *eq) ad->batch_expire[BLK_RW_ASYNC ] = default_write_batch_expire; ad->current_batch_expires = jiffies + ad->batch_expire[BLK_RW_SYNC ]; - ad->write_batch_count = ad->batch_expire[BLK_RW_ASYNC ] /10 ; -if (ad->write_batch_count <2 ) - ad->write_batch_count =2 ;return ad; }
@@ -1478,7 +1505,6 @@ static struct elevator_type iosched_as = { .elevator_add_req_fn = as_add_request, .elevator_activate_req_fn = as_activate_request, .elevator_deactivate_req_fn = as_deactivate_request, - .elevator_queue_empty_fn = as_queue_empty, .elevator_completed_req_fn = as_completed_request, .elevator_former_req_fn = elv_rb_former_request, .elevator_latter_req_fn = elv_rb_latter_request,
@@ -1486,6 +1512,8 @@ static struct elevator_type iosched_as = { .elevator_init_fn = as_init_queue, .elevator_exit_fn = as_exit_queue, .trim = as_trim, + .elevator_alloc_sched_queue_fn = as_alloc_as_queue, + .elevator_free_sched_queue_fn = as_free_as_queue, }, .elevator_attrs = as_attrs,
block/deadline-iosched.c +71 -46
266 lines not on this page · raw message
block/elevator.c +96 -15
179 lines not on this page · raw message
block/noop-iosched.c +28 -31
111 lines not on this page · raw message
include/linux/elevator.h +7 -2
@@ -30,8 +30,10 @@ typedef void (elevator_deactivate_req_fn) (struct request_queue *, struct requestypedef void *(elevator_init_fn ) (struct request_queue *,struct elevator_queue *);typedef void (elevator_exit_fn ) (struct elevator_queue *); -#ifdef CONFIG_ELV_FAIR_QUEUING +typedef void * (elevator_alloc_sched_queue_fn ) (struct request_queue *q, +struct elevator_queue *eq,gfp_t );typedef void (elevator_free_sched_queue_fn ) (struct elevator_queue *,void *); +#ifdef CONFIG_ELV_FAIR_QUEUING typedef void (elevator_active_ioq_set_fn ) (struct request_queue *,void *,int );typedef void (elevator_active_ioq_reset_fn ) (struct request_queue *,void *);typedef void (elevator_arm_slice_timer_fn ) (struct request_queue *,void *);
@@ -68,8 +70,9 @@ struct elevator_opselevator_exit_fn *elevator_exit_fn;void (*trim)(struct io_context *); -#ifdef CONFIG_ELV_FAIR_QUEUING +elevator_alloc_sched_queue_fn *elevator_alloc_sched_queue_fn;elevator_free_sched_queue_fn *elevator_free_sched_queue_fn; +#ifdef CONFIG_ELV_FAIR_QUEUING elevator_active_ioq_set_fn *elevator_active_ioq_set_fn;elevator_active_ioq_reset_fn *elevator_active_ioq_reset_fn;
@@ -109,6 +112,7 @@ struct elevator_queue {struct elevator_ops *ops;void *elevator_data; +void *sched_queue;struct kobject kobj;struct elevator_type *elevator_type;struct mutex sysfs_lock;
@@ -255,5 +259,6 @@ static inline int elv_iosched_fair_queuing_enabled(struct elevator_queue *e)#endif /* ELV_IOSCHED_FAIR_QUEUING */ extern void *elv_get_sched_queue (struct request_queue *q,struct request *rq);extern void *elv_select_sched_queue (struct request_queue *q,int force); +extern void *elv_get_sched_queue_current (struct request_queue *q);#endif /* CONFIG_BLOCK */ #endif
Signature · 1 line
1.6.0.6
PATCH 14/23 io-conroller: Prepare elevator layer for single queue schedulers
In reply to a message not in the archiveElevator layer now has support for hierarchical fair queuing. cfq has been migrated to make use of it and now it is time to do groundwork for noop, deadline and AS.
noop deadline and AS don't maintain separate queues for different processes. There is only one single queue. Effectively one can think that in hierarchical setup, there will be one queue per cgroup where requests from all the processes in the cgroup will be queued.
Generally io scheduler takes care of creating queues. Because there is only one queue here, we have modified common layer to take care of queue creation and some other functionality. This special casing helps in keeping the changes to noop, deadline and AS to the minimum.
--- block/as-iosched.c | 2 +- block/deadline-iosched.c | 2 +- block/elevator-fq.c | 211 +++++++++++++++++++++++++++++++++++++++++++++- block/elevator-fq.h | 36 ++++++++ block/elevator.c | 37 ++++++++- block/noop-iosched.c | 2 +- include/linux/elevator.h | 18 ++++- 7 files changed, 300 insertions(+), 8 deletions(-)
block/as-iosched.c +1 -1
@@ -1338,7 +1338,7 @@ static int as_may_queue(struct request_queue *q, int rw)/* Called with queue lock held */ static void *as_alloc_as_queue (struct request_queue *q, -struct elevator_queue *eq,gfp_t gfp_mask) +struct elevator_queue *eq,gfp_t gfp_mask,struct io_queue *ioq) {struct as_queue *asq;struct as_data *ad = eq->elevator_data;
block/deadline-iosched.c +1 -1
@@ -341,7 +341,7 @@ dispatch_request: }static void *deadline_alloc_deadline_queue (struct request_queue *q, -struct elevator_queue *eq,gfp_t gfp_mask) +struct elevator_queue *eq,gfp_t gfp_mask,struct io_queue *ioq) {struct deadline_queue *dq;
block/elevator-fq.c +210 -1
253 lines not on this page · raw message
block/elevator-fq.h +36 -0
@@ -125,6 +125,9 @@ struct io_group {/* Store cgroup path */ char path[128 ];#endif + +/* Single ioq per group, used for noop, deadline, anticipatory */ +struct io_queue *ioq; };struct io_cgroup {
@@ -418,6 +421,11 @@ static inline void elv_get_iog(struct io_group *iog) atomic_inc(&iog->ref); } +extern int elv_set_request_ioq (struct request_queue *q,struct request *rq, +gfp_t gfp_mask); +extern void elv_reset_request_ioq (struct request_queue *q,struct request *rq); +extern struct io_queue *elv_lookup_ioq_current (struct request_queue *q); +#else /* !GROUP_IOSCHED */ static inline int elv_io_group_allow_merge (struct request *rq,struct bio *bio)
@@ -435,6 +443,20 @@ elv_io_get_io_group(struct request_queue *q, int create) return q->elevator->efqd->root_group; } +static inline int +elv_set_request_ioq (struct request_queue *q,struct request *rq,gfp_t gfp_mask) +{ +return 0 ; +} + +static inline void +elv_reset_request_ioq (struct request_queue *q,struct request *rq) { } + +static inline struct io_queue *elv_lookup_ioq_current (struct request_queue *q) +{ +return NULL ; +} +#endif /* GROUP_IOSCHED */ extern ssize_t elv_slice_sync_show (struct elevator_queue *q,char *name);
@@ -529,6 +551,20 @@ static inline int elv_io_group_allow_merge(struct request *rq, struct bio *bio) {return 1 ; } +static inline int +elv_set_request_ioq (struct request_queue *q,struct request *rq,gfp_t gfp_mask) +{ +return 0 ; +} + +static inline void +elv_reset_request_ioq (struct request_queue *q,struct request *rq) { } + +static inline struct io_queue *elv_lookup_ioq_current (struct request_queue *q) +{ +return NULL ; +} +#endif /* CONFIG_ELV_FAIR_QUEUING */ #endif /* _ELV_SCHED_H */ #endif /* CONFIG_BLOCK */
block/elevator.c +35 -2
@@ -228,9 +228,17 @@ elevator_alloc_sched_queue(struct request_queue *q, struct elevator_queue *eq) {void *sched_queue =NULL ; +/* +* If fair queuing is enabled, then queue allocation takes place +* during set_request() functions when request actually comes +* in. +*/ +if (elv_iosched_fair_queuing_enabled (eq)) +return NULL ; +if (eq->ops->elevator_alloc_sched_queue_fn) { sched_queue = eq->ops->elevator_alloc_sched_queue_fn (q, eq, -GFP_KERNEL ); +GFP_KERNEL ,NULL );if (!sched_queue)return ERR_PTR (-ENOMEM );
@@ -861,6 +869,13 @@ int elv_set_request(struct request_queue *q, struct request *rq, gfp_t gfp_mask) {struct elevator_queue *e = q->elevator; +/* +* Optimization for noop, deadline and AS which maintain only single +* ioq per io group +*/ +if (elv_iosched_single_ioq (e)) +return elv_set_request_ioq (q, rq, gfp_mask); +if (e->ops->elevator_set_req_fn)return e->ops->elevator_set_req_fn (q, rq, gfp_mask);
@@ -872,6 +887,15 @@ void elv_put_request(struct request_queue *q, struct request *rq) {struct elevator_queue *e = q->elevator; +/* +* Optimization for noop, deadline and AS which maintain only single +* ioq per io group +*/ +if (elv_iosched_single_ioq (e)) { +elv_reset_request_ioq (q, rq); +return ; + } +if (e->ops->elevator_put_req_fn) e->ops->elevator_put_req_fn (rq); }
@@ -1256,9 +1280,18 @@ EXPORT_SYMBOL(elv_select_sched_queue);/* * Get the io scheduler queue pointer for current task. +* +* If fair queuing is enabled, determine the io group of task and retrieve +* the ioq pointer from that. This is used by only single queue ioschedulers +* for retrieving the queue associated with the group to decide whether the +* new bio can do a front merge or not. */ void *elv_get_sched_queue_current (struct request_queue *q) { -return q->elevator->sched_queue; +/* Fair queuing is not enabled. There is only one queue. */ +if (!elv_iosched_fair_queuing_enabled (q->elevator)) +return q->elevator->sched_queue; + +return elv_ioq_sched_queue (elv_lookup_ioq_current (q)); }EXPORT_SYMBOL (elv_get_sched_queue_current);
block/noop-iosched.c +1 -1
@@ -62,7 +62,7 @@ noop_latter_request(struct request_queue *q, struct request *rq) }static void *noop_alloc_noop_queue (struct request_queue *q, -struct elevator_queue *eq,gfp_t gfp_mask) +struct elevator_queue *eq,gfp_t gfp_mask,struct io_queue *ioq) {struct noop_queue *nq;
include/linux/elevator.h +16 -2
@@ -30,9 +30,9 @@ typedef void (elevator_deactivate_req_fn) (struct request_queue *, struct requestypedef void *(elevator_init_fn ) (struct request_queue *,struct elevator_queue *);typedef void (elevator_exit_fn ) (struct elevator_queue *); -typedef void * (elevator_alloc_sched_queue_fn ) (struct request_queue *q, -struct elevator_queue *eq,gfp_t );typedef void (elevator_free_sched_queue_fn ) (struct elevator_queue *,void *); +typedef void * (elevator_alloc_sched_queue_fn ) (struct request_queue *q, +struct elevator_queue *eq,gfp_t ,struct io_queue *ioq);#ifdef CONFIG_ELV_FAIR_QUEUING typedef void (elevator_active_ioq_set_fn ) (struct request_queue *,void *,int );typedef void (elevator_active_ioq_reset_fn ) (struct request_queue *,void *);
@@ -245,17 +245,31 @@ enum {/* iosched wants to use fair queuing logic of elevator layer */ #define ELV_IOSCHED_NEED_FQ 1 +/* iosched maintains only single ioq per group.*/ +#define ELV_IOSCHED_SINGLE_IOQ 2 +static inline int elv_iosched_fair_queuing_enabled (struct elevator_queue *e) {return (e->elevator_type->elevator_features) &ELV_IOSCHED_NEED_FQ ; } +static inline int elv_iosched_single_ioq (struct elevator_queue *e) +{ +return (e->elevator_type->elevator_features) &ELV_IOSCHED_SINGLE_IOQ ; +} +#else /* ELV_IOSCHED_FAIR_QUEUING */ static inline int elv_iosched_fair_queuing_enabled (struct elevator_queue *e) {return 0 ; } + +static inline int elv_iosched_single_ioq (struct elevator_queue *e) +{ +return 0 ; +} +#endif /* ELV_IOSCHED_FAIR_QUEUING */ extern void *elv_get_sched_queue (struct request_queue *q,struct request *rq);extern void *elv_select_sched_queue (struct request_queue *q,int force);
Signature · 1 line
1.6.0.6
PATCH 15/23 io-controller: noop changes for hierarchical fair queuing
In reply to a message not in the archiveThis patch changes noop to use queue scheduling code from elevator layer. One can go back to old noop by deselecting CONFIG_IOSCHED_NOOP_HIER.
--- block/Kconfig.iosched | 11 +++++++++++ block/noop-iosched.c | 14 ++++++++++++++ 2 files changed, 25 insertions(+), 0 deletions(-)
block/Kconfig.iosched +11 -0
@@ -25,6 +25,17 @@ config IOSCHED_NOOP that do their own scheduling and require only minimal assistance from the kernel. +config IOSCHED_NOOP_HIER + bool "Noop Hierarchical Scheduling support" + depends on IOSCHED_NOOP && CGROUPS + select ELV_FAIR_QUEUING + select GROUP_IOSCHED + default n + ---help--- + Enable hierarhical scheduling in noop. In this mode noop keeps + one IO queue per cgroup instead of a global queue. Elevator + fair queuing logic ensures fairness among various queues. + config IOSCHED_AS tristate "Anticipatory I/O scheduler" default y
block/noop-iosched.c +14 -0
@@ -6,6 +6,7 @@#include <linux/bio.h> #include <linux/module.h> #include <linux/init.h> +#include "elevator-fq.h" struct noop_queue {struct list_head queue;
@@ -82,6 +83,15 @@ static void noop_free_noop_queue(struct elevator_queue *e, void *sched_queue)kfree (nq); } +#ifdef CONFIG_IOSCHED_NOOP_HIER +static struct elv_fs_entry noop_attrs[] = { +ELV_ATTR (fairness), +ELV_ATTR (slice_sync), +ELV_ATTR (group_idle), + __ATTR_NULL +}; +#endif +static struct elevator_type elevator_noop = { .ops = { .elevator_merge_req_fn = noop_merged_requests,
@@ -92,6 +102,10 @@ static struct elevator_type elevator_noop = { .elevator_alloc_sched_queue_fn = noop_alloc_noop_queue, .elevator_free_sched_queue_fn = noop_free_noop_queue, }, +#ifdef CONFIG_IOSCHED_NOOP_HIER + .elevator_features = ELV_IOSCHED_NEED_FQ | ELV_IOSCHED_SINGLE_IOQ, + .elevator_attrs = noop_attrs, +#endif .elevator_name = "noop", .elevator_owner = THIS_MODULE, };
Signature · 1 line
1.6.0.6
PATCH 16/23 io-controller: deadline changes for hierarchical fair queuing
In reply to a message not in the archiveThis patch changes deadline to use queue scheduling code from elevator layer. One can go back to old deadline by selecting CONFIG_IOSCHED_DEADLINE_HIER.
--- block/Kconfig.iosched | 11 +++++++++++ block/deadline-iosched.c | 9 +++++++++ 2 files changed, 20 insertions(+), 0 deletions(-)
block/Kconfig.iosched +11 -0
@@ -55,6 +55,17 @@ config IOSCHED_DEADLINE a disk at any one time, its behaviour is almost identical to the anticipatory I/O scheduler and so is a good choice. +config IOSCHED_DEADLINE_HIER + bool "Deadline Hierarchical Scheduling support" + depends on IOSCHED_DEADLINE && CGROUPS + select ELV_FAIR_QUEUING + select GROUP_IOSCHED + default n + ---help--- + Enable hierarhical scheduling in deadline. In this mode deadline keeps + one IO queue per cgroup instead of a global queue. Elevator + fair queuing logic ensures fairness among various queues. + config IOSCHED_CFQ tristate "CFQ I/O scheduler" select ELV_FAIR_QUEUING
block/deadline-iosched.c +9 -0
@@ -13,6 +13,7 @@#include <linux/init.h> #include <linux/compiler.h> #include <linux/rbtree.h> +#include "elevator-fq.h" /* * See Documentation/block/deadline-iosched.txt
@@ -461,6 +462,11 @@ static struct elv_fs_entry deadline_attrs[] = {DD_ATTR (writes_starved),DD_ATTR (front_merges),DD_ATTR (fifo_batch), +#ifdef CONFIG_IOSCHED_DEADLINE_HIER +ELV_ATTR (fairness), +ELV_ATTR (slice_sync), +ELV_ATTR (group_idle), +#endif __ATTR_NULL };
@@ -478,6 +484,9 @@ static struct elevator_type iosched_deadline = { .elevator_alloc_sched_queue_fn = deadline_alloc_deadline_queue, .elevator_free_sched_queue_fn = deadline_free_deadline_queue, }, +#ifdef CONFIG_IOSCHED_DEADLINE_HIER + .elevator_features = ELV_IOSCHED_NEED_FQ | ELV_IOSCHED_SINGLE_IOQ, +#endif .elevator_attrs = deadline_attrs, .elevator_name = "deadline", .elevator_owner = THIS_MODULE,
Signature · 1 line
1.6.0.6
PATCH 17/23 io-controller: anticipatory changes for hierarchical fair queuing
In reply to a message not in the archiveThis patch changes anticipatory scheduler to use queue scheduling code from elevator layer. One can go back to old as by deselecting CONFIG_IOSCHED_AS_HIER. Even with CONFIG_IOSCHED_AS_HIER=y, with-out any other cgroup created, AS behavior should remain the same as old.
o AS is a single queue ioschduler, that means there is one AS queue per group.
o common layer code select the queue to dispatch from based on fairness, and then AS code selects the request with-in group.
o AS runs reads and writes batches with-in group. So common layer runs timed group queues and with-in group time, AS runs timed batches of reads and writes.
o Note: Previously AS write batch length was adjusted synamically whenever a W->R batch data direction took place and when first request from the read batch completed.
Now write batch updation takes place when last request from the write batch has finished during W->R transition.
o AS runs its own anticipation logic to anticipate on reads. common layer also does the anticipation on the group if think time of the group is with-in slice_idle.
o Introduced few debugging messages in AS.
--- block/Kconfig.iosched | 12 ++ block/as-iosched.c | 295 +++++++++++++++++++++++++++++++++++++++++++++- block/elevator-fq.c | 89 ++++++++++++-- block/elevator-fq.h | 2 + include/linux/elevator.h | 2 + 5 files changed, 382 insertions(+), 18 deletions(-)
block/Kconfig.iosched +12 -0
@@ -45,6 +45,18 @@ config IOSCHED_AS deadline I/O scheduler, it can also be slower in some cases especially some database loads. +config IOSCHED_AS_HIER + bool "Anticipatory Hierarchical Scheduling support" + depends on IOSCHED_AS && CGROUPS + select ELV_FAIR_QUEUING + select GROUP_IOSCHED + default n + ---help--- + Enable hierarhical scheduling in anticipatory. In this mode + anticipatory keeps one IO queue per cgroup instead of a global + queue. Elevator fair queuing logic ensures fairness among various + queues. + config IOSCHED_DEADLINE tristate "Deadline I/O scheduler" default y
block/as-iosched.c +289 -6
497 lines not on this page · raw message
block/elevator-fq.c +77 -12
150 lines not on this page · raw message
block/elevator-fq.h +2 -0
@@ -229,6 +229,7 @@ enum elv_queue_state_flags { ELV_QUEUE_FLAG_idle_window,/* elevator slice idling enabled */ ELV_QUEUE_FLAG_slice_new,/* no requests dispatched in slice */ ELV_QUEUE_FLAG_sync,/* synchronous queue */ + ELV_QUEUE_FLAG_must_expire, /* expire queue even slice is left */ };#define ELV_IO_QUEUE_FLAG_FNS (name) \
@@ -251,6 +252,7 @@ ELV_IO_QUEUE_FLAG_FNS(must_dispatch) ELV_IO_QUEUE_FLAG_FNS(idle_window) ELV_IO_QUEUE_FLAG_FNS(slice_new) ELV_IO_QUEUE_FLAG_FNS(sync) +ELV_IO_QUEUE_FLAG_FNS(must_expire)#ifdef CONFIG_GROUP_IOSCHED
include/linux/elevator.h +2 -0
@@ -41,6 +41,7 @@ typedef int (elevator_should_preempt_fn) (struct request_queue*, void*, struct request*);typedef struct io_queue * (elevator_close_cooperator_fn ) (struct request_queue *,void *); +typedef int (elevator_expire_ioq_fn ) (struct request_queue *,void *,int ,int );#endif struct elevator_ops
@@ -79,6 +80,7 @@ struct elevator_opselevator_arm_slice_timer_fn *elevator_arm_slice_timer_fn;elevator_should_preempt_fn *elevator_should_preempt_fn;elevator_close_cooperator_fn *elevator_close_cooperator_fn; +elevator_expire_ioq_fn *elevator_expire_ioq_fn;#endif };
Signature · 1 line
1.6.0.6
PATCH 18/23 io-controller: blkio_cgroup patches from Ryo to track async bios.
In reply to a message not in the archiveo blkio_cgroup patches from Ryo to track async bios.
o This functionality is used to determine the group of async IO from page instead of context of submitting task.
--- block/blk-ioc.c | 36 +++--- fs/buffer.c | 2 + fs/direct-io.c | 2 + include/linux/biotrack.h | 100 ++++++++++++++ include/linux/cgroup_subsys.h | 6 + include/linux/iocontext.h | 1 + include/linux/memcontrol.h | 6 + include/linux/mmzone.h | 4 +- include/linux/page_cgroup.h | 5 +- init/Kconfig | 16 +++ mm/Makefile | 4 +- mm/biotrack.c | 293 +++++++++++++++++++++++++++++++++++++++++ mm/bounce.c | 2 + mm/filemap.c | 2 + mm/memcontrol.c | 6 + mm/memory.c | 5 + mm/page-writeback.c | 2 + mm/page_cgroup.c | 23 ++-- mm/swap_state.c | 2 + 19 files changed, 486 insertions(+), 31 deletions(-) create mode 100644 include/linux/biotrack.h create mode 100644 mm/biotrack.c
block/blk-ioc.c +20 -16
@@ -84,27 +84,31 @@ void exit_io_context(void) } } +void init_io_context (struct io_context *ioc) +{ +atomic_long_set (&ioc->refcount,1 ); +atomic_set (&ioc->nr_tasks,1 ); +spin_lock_init (&ioc->lock); + ioc->ioprio_changed =0 ; + ioc->ioprio =0 ; +#ifdef CONFIG_GROUP_IOSCHED + ioc->cgroup_changed =0 ; +#endif + ioc->last_waited = jiffies;/* doesn't matter... */ + ioc->nr_batch_requests =0 ;/* because this is 0 */ + ioc->aic =NULL ; +INIT_RADIX_TREE (&ioc->radix_root,GFP_ATOMIC | __GFP_HIGH); +INIT_HLIST_HEAD (&ioc->cic_list); + ioc->ioc_data =NULL ; +} +struct io_context *alloc_io_context (gfp_t gfp_flags,int node) {struct io_context *ret; ret =kmem_cache_alloc_node (iocontext_cachep, gfp_flags, node); -if (ret) { -atomic_long_set (&ret->refcount,1 ); -atomic_set (&ret->nr_tasks,1 ); -spin_lock_init (&ret->lock); - ret->ioprio_changed =0 ; - ret->ioprio =0 ; -#ifdef CONFIG_GROUP_IOSCHED - ret->cgroup_changed =0 ; -#endif - ret->last_waited = jiffies;/* doesn't matter... */ - ret->nr_batch_requests =0 ;/* because this is 0 */ - ret->aic =NULL ; -INIT_RADIX_TREE (&ret->radix_root,GFP_ATOMIC | __GFP_HIGH); -INIT_HLIST_HEAD (&ret->cic_list); - ret->ioc_data =NULL ; - } +if (ret) +init_io_context (ret);return ret; }
fs/buffer.c +2 -0
@@ -36,6 +36,7 @@#include <linux/buffer_head.h> #include <linux/task_io_accounting_ops.h> #include <linux/bio.h> +#include <linux/biotrack.h> #include <linux/notifier.h> #include <linux/cpu.h> #include <linux/bitops.h>
@@ -668,6 +669,7 @@ static void __set_page_dirty(struct page *page,if (page->mapping) {/* Race with truncate? */ WARN_ON_ONCE (warn && !PageUptodate (page));account_page_dirtied (page, mapping); +blkio_cgroup_reset_owner_pagedirty (page, current->mm);radix_tree_tag_set (&mapping->page_tree,page_index (page),PAGECACHE_TAG_DIRTY ); }
fs/direct-io.c +2 -0
@@ -33,6 +33,7 @@#include <linux/err.h> #include <linux/blkdev.h> #include <linux/buffer_head.h> +#include <linux/biotrack.h> #include <linux/rwsem.h> #include <linux/uio.h> #include <asm/atomic.h>
@@ -797,6 +798,7 @@ static int do_direct_IO(struct dio *dio) ret =PTR_ERR (page); goto out; } +blkio_cgroup_reset_owner (page, current->mm);while (block_in_page < blocks_per_page) {unsigned offset_in_page = block_in_page << blkbits;
include/linux/biotrack.h new file +100 -0
100 lines not on this page · raw message
include/linux/cgroup_subsys.h +6 -0
@@ -43,6 +43,12 @@ SUBSYS(mem_cgroup)/* */ +#ifdef CONFIG_CGROUP_BLKIO +SUBSYS(blkio_cgroup) +#endif + +/* */ +#ifdef CONFIG_CGROUP_DEVICE SUBSYS(devices)#endif
include/linux/iocontext.h +1 -0
@@ -109,6 +109,7 @@ int put_io_context(struct io_context *ioc);void exit_io_context (void );struct io_context *get_io_context (gfp_t gfp_flags,int node);struct io_context *alloc_io_context (gfp_t gfp_flags,int node); +void init_io_context (struct io_context *ioc);void copy_io_context (struct io_context **pdst,struct io_context **psrc);#else static inline void exit_io_context (void )
include/linux/memcontrol.h +6 -0
@@ -37,6 +37,8 @@ struct mm_struct;* (Of course, if memcg does memory allocation in future, GFP_KERNEL is sane.) */ +extern void __init_mem_page_cgroup (struct page_cgroup *pc); +extern int mem_cgroup_newpage_charge (struct page *page,struct mm_struct *mm,gfp_t gfp_mask);/* for swap handling */
@@ -121,6 +123,10 @@ void mem_cgroup_update_mapped_file_stat(struct page *page, int val);#else /* CONFIG_CGROUP_MEM_RES_CTLR */ struct mem_cgroup ; +static inline void __init_mem_page_cgroup (struct page_cgroup *pc) +{ +} +static inline int mem_cgroup_newpage_charge (struct page *page,struct mm_struct *mm,gfp_t gfp_mask) {
include/linux/mmzone.h +2 -2
@@ -605,7 +605,7 @@ typedef struct pglist_data {int nr_zones;#ifdef CONFIG_FLAT_NODE_MEM_MAP /* means !SPARSEMEM */ struct page *node_mem_map; -#ifdef CONFIG_CGROUP_MEM_RES_CTLR +#ifdef CONFIG_CGROUP_PAGE struct page_cgroup *node_page_cgroup;#endif #endif
@@ -956,7 +956,7 @@ struct mem_section {/* See declaration of similar field in struct zone */ unsigned long *pageblock_flags; -#ifdef CONFIG_CGROUP_MEM_RES_CTLR +#ifdef CONFIG_CGROUP_PAGE /* * If !SPARSEMEM, pgdat doesn't have page_cgroup pointer. We use * section. (see memcontrol.h/page_cgroup.h about this.)
include/linux/page_cgroup.h +3 -2
@@ -1,7 +1,7 @@#ifndef __LINUX_PAGE_CGROUP_H#define __LINUX_PAGE_CGROUP_H -#ifdef CONFIG_CGROUP_MEM_RES_CTLR +#ifdef CONFIG_CGROUP_PAGE #include <linux/bit_spinlock.h> /* * Page Cgroup can be considered as an extended mem_map.
@@ -14,6 +14,7 @@ struct page_cgroup {unsigned long flags;struct mem_cgroup *mem_cgroup;struct page *page; +unsigned long blkio_cgroup_id;struct list_head lru;/* per cgroup LRU list */ };
@@ -83,7 +84,7 @@ static inline void unlock_page_cgroup(struct page_cgroup *pc) bit_spin_unlock(PCG_LOCK, &pc->flags); } -#else /* CONFIG_CGROUP_MEM_RES_CTLR */ +#else /* CONFIG_CGROUP_PAGE */ struct page_cgroup ;static inline void __meminit pgdat_page_cgroup_init (struct pglist_data *pgdat)
init/Kconfig +16 -0
@@ -622,6 +622,22 @@ config GROUP_IOSCHED endif # CGROUPS +config CGROUP_BLKIO + bool + depends on CGROUPS && BLOCK + select MM_OWNER + default n + ---help--- + Provides a Resource Controller which enables to track the onwner + of every Block I/O requests. + The information this subsystem provides can be used from any + kind of module such as dm-ioband device mapper modules or + the cfq-scheduler. + +config CGROUP_PAGE + def_bool y + depends on CGROUP_MEM_RES_CTLR || CGROUP_BLKIO + config MM_OWNER bool
mm/Makefile +3 -1
@@ -39,6 +39,8 @@ else obj-$(CONFIG_SMP) += allocpercpu.o endif obj-$(CONFIG_QUICKLIST) += quicklist.o -obj-$(CONFIG_CGROUP_MEM_RES_CTLR) += memcontrol.o page_cgroup.o +obj-$(CONFIG_CGROUP_MEM_RES_CTLR) += memcontrol.o +obj-$(CONFIG_CGROUP_PAGE) += page_cgroup.o +obj-$(CONFIG_CGROUP_BLKIO) += biotrack.o obj-$(CONFIG_DEBUG_KMEMLEAK) += kmemleak.o obj-$(CONFIG_DEBUG_KMEMLEAK_TEST) += kmemleak-test.o
mm/biotrack.c new file +293 -0
293 lines not on this page · raw message
mm/bounce.c +2 -0
@@ -13,6 +13,7 @@#include <linux/init.h> #include <linux/hash.h> #include <linux/highmem.h> +#include <linux/biotrack.h> #include <asm/tlbflush.h> #include <trace/events/block.h>
@@ -210,6 +211,7 @@ static void __blk_queue_bounce(struct request_queue *q, struct bio **bio_orig, to->bv_len = from->bv_len; to->bv_offset = from->bv_offset;inc_zone_page_state (to->bv_page,NR_BOUNCE ); +blkio_cgroup_copy_owner (to->bv_page, page);if (rw ==WRITE ) {char *vto, *vfrom;
mm/filemap.c +2 -0
@@ -33,6 +33,7 @@#include <linux/cpuset.h> #include <linux/hardirq.h> /* for BUG_ON(!in_atomic()) only */ #include <linux/memcontrol.h> +#include <linux/biotrack.h> #include <linux/mm_inline.h> /* for page_is_file_cache() */ #include "internal.h"
@@ -464,6 +465,7 @@ int add_to_page_cache_locked(struct page *page, struct address_space *mapping, gfp_mask & GFP_RECLAIM_MASK);if (error) goto out; +blkio_cgroup_set_owner (page, current->mm); error =radix_tree_preload (gfp_mask & ~__GFP_HIGHMEM);if (error ==0 ) {
mm/memcontrol.c +6 -0
@@ -129,6 +129,12 @@ struct mem_cgroup_lru_info {struct mem_cgroup_per_node *nodeinfo[MAX_NUMNODES ]; }; +void __meminit __init_mem_page_cgroup (struct page_cgroup *pc) +{ + pc->mem_cgroup =NULL ; +INIT_LIST_HEAD (&pc->lru); +} +/* * The memory controller data structure. The memory controller controls both * page cache and RSS per cgroup. We would eventually like to provide
mm/memory.c +5 -0
@@ -51,6 +51,7 @@#include <linux/init.h> #include <linux/writeback.h> #include <linux/memcontrol.h> +#include <linux/biotrack.h> #include <linux/mmu_notifier.h> #include <linux/kallsyms.h> #include <linux/swapops.h>
@@ -2116,6 +2117,7 @@ gotten:*/ ptep_clear_flush_notify (vma, address, page_table);page_add_new_anon_rmap (new_page, vma, address); +blkio_cgroup_set_owner (new_page, mm);set_pte_at (mm, address, page_table, entry);update_mmu_cache (vma, address, entry);if (old_page) {
@@ -2581,6 +2583,7 @@ static int do_swap_page(struct mm_struct *mm, struct vm_area_struct *vma,flush_icache_page (vma, page);set_pte_at (mm, address, page_table, pte);page_add_anon_rmap (page, vma, address); +blkio_cgroup_reset_owner (page, mm);/* It's better to call commit-charge after rmap is established */ mem_cgroup_commit_charge_swapin (page, ptr);
@@ -2645,6 +2648,7 @@ static int do_anonymous_page(struct mm_struct *mm, struct vm_area_struct *vma, goto release;inc_mm_counter (mm, anon_rss);page_add_new_anon_rmap (page, vma, address); +blkio_cgroup_set_owner (page, mm);set_pte_at (mm, address, page_table, entry);/* No need to invalidate - it was non-present before */
@@ -2792,6 +2796,7 @@ static int __do_fault(struct mm_struct *mm, struct vm_area_struct *vma,if (anon) {inc_mm_counter (mm, anon_rss);page_add_new_anon_rmap (page, vma, address); +blkio_cgroup_set_owner (page, mm); }else {inc_mm_counter (mm, file_rss);page_add_file_rmap (page);
mm/page-writeback.c +2 -0
@@ -23,6 +23,7 @@#include <linux/init.h> #include <linux/backing-dev.h> #include <linux/task_io_accounting_ops.h> +#include <linux/biotrack.h> #include <linux/blkdev.h> #include <linux/mpage.h> #include <linux/rmap.h>
@@ -1247,6 +1248,7 @@ int __set_page_dirty_nobuffers(struct page *page)BUG_ON (mapping2 != mapping);WARN_ON_ONCE (!PagePrivate (page) && !PageUptodate (page));account_page_dirtied (page, mapping); +blkio_cgroup_reset_owner_pagedirty (page, current->mm);radix_tree_tag_set (&mapping->page_tree,page_index (page),PAGECACHE_TAG_DIRTY ); }
mm/page_cgroup.c +13 -10
@@ -9,14 +9,15 @@#include <linux/vmalloc.h> #include <linux/cgroup.h> #include <linux/swapops.h> +#include <linux/biotrack.h> static void __meminit __init_page_cgroup (struct page_cgroup *pc,unsigned long pfn) { pc->flags =0 ; - pc->mem_cgroup =NULL ; pc->page =pfn_to_page (pfn); -INIT_LIST_HEAD (&pc->lru); +__init_mem_page_cgroup (pc); +__init_blkio_page_cgroup (pc); }static unsigned long total_usage;
@@ -74,7 +75,7 @@ void __init page_cgroup_init_flatmem(void)int nid, fail; -if (mem_cgroup_disabled ()) +if (mem_cgroup_disabled () &&blkio_cgroup_disabled ())return ;for_each_online_node (nid) {
@@ -83,12 +84,13 @@ void __init page_cgroup_init_flatmem(void) goto fail; }printk (KERN_INFO "allocated %ld bytes of page_cgroup\n" , total_usage); -printk (KERN_INFO "please try 'cgroup_disable=memory' option if you" -" don't want memory cgroups\n" ); +printk (KERN_INFO "please try 'cgroup_disable=memory,blkio' option" +" if you don't want memory and blkio cgroups\n" );return ; fail:printk (KERN_CRIT "allocation of page_cgroup failed.\n" ); -printk (KERN_CRIT "please try 'cgroup_disable=memory' boot option\n" ); +printk (KERN_CRIT +"please try 'cgroup_disable=memory,blkio' boot option\n" );panic ("Out of memory" ); }
@@ -245,7 +247,7 @@ void __init page_cgroup_init(void)unsigned long pfn;int fail =0 ; -if (mem_cgroup_disabled ()) +if (mem_cgroup_disabled () &&blkio_cgroup_disabled ())return ;for (pfn =0 ; !fail && pfn < max_pfn; pfn +=PAGES_PER_SECTION ) {
@@ -254,14 +256,15 @@ void __init page_cgroup_init(void) fail =init_section_page_cgroup (pfn); }if (fail) { -printk (KERN_CRIT "try 'cgroup_disable=memory' boot option\n" ); +printk (KERN_CRIT +"try 'cgroup_disable=memory,blkio' boot option\n" );panic ("Out of memory" ); }else {hotplug_memory_notifier (page_cgroup_callback,0 ); }printk (KERN_INFO "allocated %ld bytes of page_cgroup\n" , total_usage); -printk (KERN_INFO "please try 'cgroup_disable=memory' option if you don't" -" want memory cgroups\n" ); +printk (KERN_INFO "please try 'cgroup_disable=memory,blkio' option" +" if you don't want memory and blkio cgroups\n" ); }void __meminit pgdat_page_cgroup_init (struct pglist_data *pgdat)
mm/swap_state.c +2 -0
@@ -18,6 +18,7 @@#include <linux/pagevec.h> #include <linux/migrate.h> #include <linux/page_cgroup.h> +#include <linux/biotrack.h> #include <asm/pgtable.h>
@@ -307,6 +308,7 @@ struct page *read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask,*/ __set_page_locked (new_page);SetPageSwapBacked (new_page); +blkio_cgroup_set_owner (new_page, current->mm); err =add_to_swap_cache (new_page, entry, gfp_mask &GFP_KERNEL );if (likely (!err)) {/*
Signature · 1 line
1.6.0.6
PATCH 19/23 io-controller: map async requests to appropriate cgroup
In reply to a message not in the archiveo So far we were assuming that a bio/rq belongs to the task who is submitting it. It did not hold good in case of async writes. This patch makes use of blkio_cgroup pataches to attribute the aysnc writes to right group instead of task submitting the bio.
o For sync requests, we continue to assume that io belongs to the task submitting it. Only in case of async requests, we make use of io tracking patches to track the owner cgroup.
o So far cfq always caches the async queue pointer. With async requests now not necessarily being tied to submitting task io context, caching the pointer will not help for async queues. This patch introduces a new config option CONFIG_TRACK_ASYNC_CONTEXT. If this option is not set, cfq retains old behavior where async queue pointer is cached in task context. If it is set, async queue pointer is not cached and we take help of bio tracking patches to determine group bio belongs to and then map it to async queue of that group.
--- block/Kconfig.iosched | 16 +++++ block/as-iosched.c | 2 +- block/blk-core.c | 7 +- block/cfq-iosched.c | 152 ++++++++++++++++++++++++++++++++++++---------- block/deadline-iosched.c | 2 +- block/elevator-fq.c | 91 +++++++++++++++++++++++---- block/elevator-fq.h | 31 ++++++--- block/elevator.c | 15 +++-- include/linux/elevator.h | 22 ++++++- 9 files changed, 266 insertions(+), 72 deletions(-)
block/Kconfig.iosched +16 -0
@@ -132,6 +132,22 @@ config DEBUG_GROUP_IOSCHED Enable some debugging hooks for hierarchical scheduling support. Currently it just outputs more information in blktrace output. +config TRACK_ASYNC_CONTEXT + bool "Determine async request context from bio" + depends on GROUP_IOSCHED + select CGROUP_BLKIO + default n + ---help--- + Normally async request is attributed to the task submitting the + request. With group ioscheduling, for accurate accounting of + async writes, one needs to map the request to original task/cgroup + which originated the request and not the submitter of the request. + + Currently there are generic io tracking patches to provide facility + to map bio to original owner. If this option is set, for async + request, original owner of the bio is decided by using io tracking + patches otherwise we continue to attribute the request to the + submitting thread. endmenu endif
block/as-iosched.c +1 -1
@@ -1515,7 +1515,7 @@ as_merge(struct request_queue *q, struct request **req, struct bio *bio) {sector_t rb_key = bio->bi_sector +bio_sectors (bio);struct request *__rq; -struct as_queue *asq =elv_get_sched_queue_current (q); +struct as_queue *asq =elv_get_sched_queue_bio (q, bio);if (!asq)return ELEVATOR_NO_MERGE ;
block/blk-core.c +4 -3
@@ -619,7 +619,8 @@ static inline void blk_free_request(struct request_queue *q, struct request *rq) }static struct request * -blk_alloc_request (struct request_queue *q,int flags,int priv,gfp_t gfp_mask) +blk_alloc_request (struct request_queue *q,struct bio *bio,int flags,int priv, +gfp_t gfp_mask) {struct request *rq =mempool_alloc (q->rq.rq_pool, gfp_mask);
@@ -631,7 +632,7 @@ blk_alloc_request(struct request_queue *q, int flags, int priv, gfp_t gfp_mask) rq->cmd_flags = flags |REQ_ALLOCED ;if (priv) { -if (unlikely (elv_set_request (q, rq, gfp_mask))) { +if (unlikely (elv_set_request (q, rq, bio, gfp_mask))) {mempool_free (rq, q->rq.rq_pool);return NULL ; }
@@ -772,7 +773,7 @@ static struct request *get_request(struct request_queue *q, int rw_flags, rw_flags |=REQ_IO_STAT ;spin_unlock_irq (q->queue_lock); - rq =blk_alloc_request (q, rw_flags, priv, gfp_mask); + rq =blk_alloc_request (q, bio, rw_flags, priv, gfp_mask);if (unlikely (!rq)) {/* * Allocation failed presumably due to memory. Undo anything
block/cfq-iosched.c +119 -33
252 lines not on this page · raw message
block/deadline-iosched.c +1 -1
@@ -134,7 +134,7 @@ deadline_merge(struct request_queue *q, struct request **req, struct bio *bio)int ret;struct deadline_queue *dq; - dq =elv_get_sched_queue_current (q); + dq =elv_get_sched_queue_bio (q, bio);if (!dq)return ELEVATOR_NO_MERGE ;
block/elevator-fq.c +77 -14
171 lines not on this page · raw message
block/elevator-fq.h +21 -10
@@ -414,7 +414,9 @@ static inline struct io_queue *elv_get_oom_ioq(struct elevator_queue *eq)extern int elv_io_group_allow_merge (struct request *rq,struct bio *bio);extern void elv_put_iog (struct io_group *iog);extern struct io_group *elv_io_get_io_group (struct request_queue *q, -int create); +struct page *page,int create); +extern struct io_group *elv_io_get_io_group_bio (struct request_queue *q, +struct bio *bio,int create);extern ssize_t elv_group_idle_show (struct elevator_queue *q,char *name);extern ssize_t elv_group_idle_store (struct elevator_queue *q,const char *name,size_t count);
@@ -424,9 +426,10 @@ static inline void elv_get_iog(struct io_group *iog) }extern int elv_set_request_ioq (struct request_queue *q,struct request *rq, -gfp_t gfp_mask); +struct bio *bio,gfp_t gfp_mask);extern void elv_reset_request_ioq (struct request_queue *q,struct request *rq); -extern struct io_queue *elv_lookup_ioq_current (struct request_queue *q); +extern struct io_queue *elv_lookup_ioq_bio (struct request_queue *q, +struct bio *bio);#else /* !GROUP_IOSCHED */
@@ -439,14 +442,20 @@ static inline void elv_get_iog(struct io_group *iog) {}static inline void elv_put_iog (struct io_group *iog) {}static inline struct io_group * -elv_io_get_io_group (struct request_queue *q,int create) +elv_io_get_io_group (struct request_queue *q,struct page *page,int create) {/* In flat mode, there is only root group */ return q->elevator->efqd->root_group; } -static inline int -elv_set_request_ioq (struct request_queue *q,struct request *rq,gfp_t gfp_mask) +static inline struct io_group * +elv_io_get_io_group_bio (struct request_queue *q,struct bio *bio,int create) +{ +return q->elevator->efqd->root_group; +} + +static inline int elv_set_request_ioq (struct request_queue *q, +struct request *rq,struct bio *bio,gfp_t gfp_mask) {return 0 ; }
@@ -454,7 +463,8 @@ elv_set_request_ioq(struct request_queue *q, struct request *rq, gfp_t gfp_mask)static inline void elv_reset_request_ioq (struct request_queue *q,struct request *rq) { } -static inline struct io_queue *elv_lookup_ioq_current (struct request_queue *q) +static inline struct io_queue * +elv_lookup_ioq_bio (struct request_queue *q,struct bio *bio) {return NULL ; }
@@ -553,8 +563,8 @@ static inline int elv_io_group_allow_merge(struct request *rq, struct bio *bio) {return 1 ; } -static inline int -elv_set_request_ioq (struct request_queue *q,struct request *rq,gfp_t gfp_mask) +static inline int elv_set_request_ioq (struct request_queue *q, +struct request *rq,struct bio *bio,gfp_t gfp_mask) {return 0 ; }
@@ -562,7 +572,8 @@ elv_set_request_ioq(struct request_queue *q, struct request *rq, gfp_t gfp_mask)static inline void elv_reset_request_ioq (struct request_queue *q,struct request *rq) { } -static inline struct io_queue *elv_lookup_ioq_current (struct request_queue *q) +static inline struct io_queue *elv_lookup_ioq_bio (struct request_queue *q, +struct bio *bio) {return NULL ; }
block/elevator.c +8 -7
@@ -865,7 +865,8 @@ struct request *elv_former_request(struct request_queue *q, struct request *rq)return NULL ; } -int elv_set_request (struct request_queue *q,struct request *rq,gfp_t gfp_mask) +int elv_set_request (struct request_queue *q,struct request *rq, +struct bio *bio,gfp_t gfp_mask) {struct elevator_queue *e = q->elevator;
@@ -874,10 +875,10 @@ int elv_set_request(struct request_queue *q, struct request *rq, gfp_t gfp_mask)* ioq per io group */ if (elv_iosched_single_ioq (e)) -return elv_set_request_ioq (q, rq, gfp_mask); +return elv_set_request_ioq (q, rq, bio, gfp_mask);if (e->ops->elevator_set_req_fn) -return e->ops->elevator_set_req_fn (q, rq, gfp_mask); +return e->ops->elevator_set_req_fn (q, rq, bio, gfp_mask); rq->elevator_private =NULL ;return 0 ;
@@ -1279,19 +1280,19 @@ void *elv_select_sched_queue(struct request_queue *q, int force)EXPORT_SYMBOL (elv_select_sched_queue);/* -* Get the io scheduler queue pointer for current task. +* Get the io scheduler queue pointer for the group bio belongs to. * * If fair queuing is enabled, determine the io group of task and retrieve * the ioq pointer from that. This is used by only single queue ioschedulers * for retrieving the queue associated with the group to decide whether the * new bio can do a front merge or not. */ -void *elv_get_sched_queue_current (struct request_queue *q) +void *elv_get_sched_queue_bio (struct request_queue *q,struct bio *bio) {/* Fair queuing is not enabled. There is only one queue. */ if (!elv_iosched_fair_queuing_enabled (q->elevator))return q->elevator->sched_queue; -return elv_ioq_sched_queue (elv_lookup_ioq_current (q)); +return elv_ioq_sched_queue (elv_lookup_ioq_bio (q, bio)); } -EXPORT_SYMBOL (elv_get_sched_queue_current); +EXPORT_SYMBOL (elv_get_sched_queue_bio);
include/linux/elevator.h +19 -3
@@ -22,7 +22,8 @@ typedef struct request *(elevator_request_list_fn) (struct request_queue *, strutypedef void (elevator_completed_req_fn ) (struct request_queue *,struct request *);typedef int (elevator_may_queue_fn ) (struct request_queue *,int ); -typedef int (elevator_set_req_fn ) (struct request_queue *,struct request *,gfp_t ); +typedef int (elevator_set_req_fn ) (struct request_queue *,struct request *, +struct bio *bio,gfp_t );typedef void (elevator_put_req_fn ) (struct request *);typedef void (elevator_activate_req_fn ) (struct request_queue *,struct request *);typedef void (elevator_deactivate_req_fn ) (struct request_queue *,struct request *);
@@ -146,7 +147,8 @@ extern void elv_unregister_queue(struct request_queue *q);extern int elv_may_queue (struct request_queue *,int );extern void elv_abort_queue (struct request_queue *);extern void elv_completed_request (struct request_queue *,struct request *); -extern int elv_set_request (struct request_queue *,struct request *,gfp_t ); +extern int elv_set_request (struct request_queue *,struct request *, +struct bio *bio,gfp_t );extern void elv_put_request (struct request_queue *,struct request *);extern void elv_drain_elevator (struct request_queue *);
@@ -275,6 +277,20 @@ static inline int elv_iosched_single_ioq(struct elevator_queue *e)#endif /* ELV_IOSCHED_FAIR_QUEUING */ extern void *elv_get_sched_queue (struct request_queue *q,struct request *rq);extern void *elv_select_sched_queue (struct request_queue *q,int force); -extern void *elv_get_sched_queue_current (struct request_queue *q); +extern void *elv_get_sched_queue_bio (struct request_queue *q,struct bio *bio); + +/* +* This is equivalent of rq_is_sync()/cfq_bio_sync() function where we +* determine whether an rq/bio is sync or not. There are cases like during +* merging and during * request allocation, where we don't have rq but bio +* and needs to find out * if this bio will be considered as sync or async by +* elevator/iosched. This function is useful in such cases. +*/ +static inline int elv_bio_sync (struct bio *bio) +{ +if ((bio_data_dir (bio) ==READ ) ||bio_sync (bio)) +return 1 ; +return 0 ; +}#endif /* CONFIG_BLOCK */ #endif
Signature · 1 line
1.6.0.6
PATCH 20/23 io-controller: Per cgroup request descriptor support
In reply to a message not in the archiveo Currently a request queue has got fixed number of request descriptors for sync and async requests. Once the request descriptors are consumed, new processes are put to sleep and they effectively become serialized. Because sync and async queues are separate, async requests don't impact sync ones but if one is looking for fairness between async requests, that is not achievable if request queue descriptors become bottleneck.
o Make request descriptor's per io group so that if there is lots of IO going on in one cgroup, it does not impact the IO of other group.
o This patch implements the per cgroup request descriptors. request pool per queue is still common but every group will have its own wait list and its own count of request descriptors allocated to that group for sync and async queues. So effectively request_list becomes per io group property and not a global request queue feature.
o Currently one can define q->nr_requests to limit request descriptors allocated for the queue. Now there is another tunable q->nr_group_requests which controls the requests descriptr limit per group. q->nr_requests supercedes q->nr_group_requests to make sure if there are lots of groups present, we don't end up allocating too many request descriptors on the queue.
--- block/blk-core.c | 317 +++++++++++++++++++++++++++++++++--------- block/blk-settings.c | 1 + block/blk-sysfs.c | 59 ++++++-- block/elevator-fq.c | 36 +++++ block/elevator-fq.h | 29 ++++ block/elevator.c | 7 +- include/linux/blkdev.h | 47 ++++++- include/trace/events/block.h | 6 +- kernel/trace/blktrace.c | 6 +- 9 files changed, 421 insertions(+), 87 deletions(-)
block/blk-core.c +250 -65
@@ -460,20 +460,53 @@ void blk_cleanup_queue(struct request_queue *q) }EXPORT_SYMBOL (blk_cleanup_queue); -static int blk_init_free_list (struct request_queue *q) +struct request_list * +blk_get_request_list (struct request_queue *q,struct bio *bio) +{ +#ifdef CONFIG_GROUP_IOSCHED +/* +* Determine which request list bio will be allocated from. This +* is dependent on which io group bio belongs to +*/ +return elv_get_request_list_bio (q, bio); +#else +return &q->rq; +#endif +} + +static struct request_list *rq_rl (struct request_queue *q,struct request *rq) +{ +#ifdef CONFIG_GROUP_IOSCHED +int priv = rq->cmd_flags &REQ_ELVPRIV ; + +return elv_get_request_list_rq (q, rq, priv); +#else +return &q->rq; +#endif +} + +void blk_init_request_list (struct request_list *rl) { -struct request_list *rl = &q->rq; rl->count[BLK_RW_SYNC ] = rl->count[BLK_RW_ASYNC ] =0 ; - rl->starved[BLK_RW_SYNC ] = rl->starved[BLK_RW_ASYNC ] =0 ; - rl->elvpriv =0 ;init_waitqueue_head (&rl->wait[BLK_RW_SYNC ]);init_waitqueue_head (&rl->wait[BLK_RW_ASYNC ]); +} - rl->rq_pool =mempool_create_node (BLKDEV_MIN_RQ , mempool_alloc_slab, - mempool_free_slab, request_cachep, q->node); +static int blk_init_free_list (struct request_queue *q) +{ +/* +* In case of group scheduling, request list is inside group and is +* initialized when group is instanciated. +*/ +#ifndef CONFIG_GROUP_IOSCHED +blk_init_request_list (&q->rq); +#endif + q->rq_data.rq_pool =mempool_create_node (BLKDEV_MIN_RQ , + mempool_alloc_slab, mempool_free_slab, + request_cachep, q->node); -if (!rl->rq_pool) +if (!q->rq_data.rq_pool)return -ENOMEM ;return 0 ;
@@ -581,6 +614,9 @@ blk_init_queue_node(request_fn_proc *rfn, spinlock_t *lock, int node_id) q->queue_flags =QUEUE_FLAG_DEFAULT ; q->queue_lock = lock; +/* init starved waiter wait queue */ +init_waitqueue_head (&q->rq_data.starved_wait); +/* * This also sets hw/phys segments, boundary and size */
@@ -615,14 +651,14 @@ static inline void blk_free_request(struct request_queue *q, struct request *rq) {if (rq->cmd_flags &REQ_ELVPRIV )elv_put_request (q, rq); -mempool_free (rq, q->rq.rq_pool); +mempool_free (rq, q->rq_data.rq_pool); }static struct request *blk_alloc_request (struct request_queue *q,struct bio *bio,int flags,int priv,gfp_t gfp_mask) { -struct request *rq =mempool_alloc (q->rq.rq_pool, gfp_mask); +struct request *rq =mempool_alloc (q->rq_data.rq_pool, gfp_mask);if (!rq)return NULL ;
@@ -633,7 +669,7 @@ blk_alloc_request(struct request_queue *q, struct bio *bio, int flags, int priv,if (priv) {if (unlikely (elv_set_request (q, rq, bio, gfp_mask))) { -mempool_free (rq, q->rq.rq_pool); +mempool_free (rq, q->rq_data.rq_pool);return NULL ; } rq->cmd_flags |=REQ_ELVPRIV ;
@@ -676,18 +712,18 @@ static void ioc_set_batching(struct request_queue *q, struct io_context *ioc) ioc->last_waited = jiffies; } -static void __freed_request (struct request_queue *q,int sync) +static void __freed_request (struct request_queue *q,int sync, +struct request_list *rl) { -struct request_list *rl = &q->rq; - -if (rl->count[sync] <queue_congestion_off_threshold (q)) +if (q->rq_data.count[sync] <queue_congestion_off_threshold (q))blk_clear_queue_congested (q, sync); -if (rl->count[sync] +1 <= q->nr_requests) { +if (q->rq_data.count[sync] +1 <= q->nr_requests) +blk_clear_queue_full (q, sync); + +if (rl->count[sync] +1 <= q->nr_group_requests) {if (waitqueue_active (&rl->wait[sync]))wake_up (&rl->wait[sync]); - -blk_clear_queue_full (q, sync); } }
@@ -695,63 +731,130 @@ static void __freed_request(struct request_queue *q, int sync)* A request has just been released. Account for it, update the full and * congestion status, wake up any waiters. Called under q->queue_lock. */ -static void freed_request (struct request_queue *q,int sync,int priv) +static void freed_request (struct request_queue *q,int sync,int priv, +struct request_list *rl) { -struct request_list *rl = &q->rq; +/* +* There is a window during request allocation where request is +* mapped to one group but by the time a queue for the group is +* allocated, it is possible that original cgroup/io group has been +* deleted and now io queue is allocated in a different group (root) +* altogether. +* +* One solution to the problem is that rq should take io group +* reference. But it looks too much to do that to solve this issue. +* The only side affect to the hard to hit issue seems to be that +* we will try to decrement the rl->count for a request list which +* did not allocate that request. Chcek for rl->count going less than +* zero and do not decrement it if that's the case. +*/ + +if (priv && rl->count[sync] >0 ) + rl->count[sync]--; + +BUG_ON (!q->rq_data.count[sync]); + q->rq_data.count[sync]--; - rl->count[sync]--;if (priv) - rl->elvpriv--; + q->rq_data.elvpriv--; -__freed_request (q, sync); +__freed_request (q, sync, rl);if (unlikely (rl->starved[sync ^1 ])) -__freed_request (q, sync ^1 ); +__freed_request (q, sync ^1 , rl); + +/* Wake up the starved process on global list, if any */ +if (unlikely (q->rq_data.starved)) { +if (waitqueue_active (&q->rq_data.starved_wait)) +wake_up (&q->rq_data.starved_wait); + q->rq_data.starved--; + } +} + +/* +* Returns whether one can sleep on this request list or not. There are +* cases (elevator switch) where request list might not have allocated +* any request descriptor but we deny request allocation due to gloabl +* limits. In that case one should sleep on global list as on this request +* list no wakeup will take place. +* +* Also sets the request list starved flag if there are no requests pending +* in the direction of rq. +* +* Return 1 --> sleep on request list, 0 --> sleep on global list +*/ +static int can_sleep_on_request_list (struct request_list *rl,int is_sync) +{ +if (unlikely (rl->count[is_sync] ==0 )) { +/* +* If there is a request pending in other direction +* in same io group, then set the starved flag of +* the group request list. Otherwise, we need to +* make this process sleep in global starved list +* to make sure it will not sleep indefinitely. +*/ +if (rl->count[is_sync ^1 ] !=0 ) { + rl->starved[is_sync] =1 ; +return 1 ; + }else +return 0 ; + } + +return 1 ; }/* * Get a free request, queue_lock must be held. -* Returns NULL on failure, with queue_lock held. +* Returns NULL on failure, with queue_lock held. Also sets the "reason" field +* in case of failure. This reason field helps caller decide to whether sleep +* on per group list or global per queue list. +* reason = 0 sleep on per group list +* reason = 1 sleep on global list +* * Returns !NULL on success, with queue_lock *not held*. */ static struct request *get_request (struct request_queue *q,int rw_flags, -struct bio *bio,gfp_t gfp_mask) +struct bio *bio,gfp_t gfp_mask, +struct request_list *rl,int *reason) {struct request *rq =NULL ; -struct request_list *rl = &q->rq;struct io_context *ioc =NULL ;const bool is_sync =rw_is_sync (rw_flags) !=0 ;int may_queue, priv; +int sleep_on_global =0 ; may_queue =elv_may_queue (q, rw_flags);if (may_queue ==ELV_MQUEUE_NO ) goto rq_starved; -if (rl->count[is_sync]+1 >=queue_congestion_on_threshold (q)) { -if (rl->count[is_sync]+1 >= q->nr_requests) { - ioc =current_io_context (GFP_ATOMIC , q->node); -/* -* The queue will fill after this allocation, so set -* it as full, and mark this process as "batching". -* This process will be allowed to complete a batch of -* requests, others will be blocked. -*/ -if (!blk_queue_full (q, is_sync)) { -ioc_set_batching (q, ioc); -blk_set_queue_full (q, is_sync); - }else { -if (may_queue !=ELV_MQUEUE_MUST - && !ioc_batching (q, ioc)) { -/* -* The queue is full and the allocating -* process is not a "batcher", and not -* exempted by the IO scheduler -*/ - goto out; - } +if (q->rq_data.count[is_sync]+1 >=queue_congestion_on_threshold (q)) +blk_set_queue_congested (q, is_sync); + +/* queue full seems redundant now */ +if (q->rq_data.count[is_sync]+1 >= q->nr_requests) +blk_set_queue_full (q, is_sync); + +if (rl->count[is_sync]+1 >= q->nr_group_requests) { + ioc =current_io_context (GFP_ATOMIC , q->node); +/* +* The queue request descriptor group will fill after this +* allocation, so set it as full, and mark this process as +* "batching". This process will be allowed to complete a +* batch of requests, others will be blocked. +*/ +if (rl->count[is_sync] <= q->nr_group_requests) +ioc_set_batching (q, ioc); +else { +if (may_queue !=ELV_MQUEUE_MUST + && !ioc_batching (q, ioc)) { +/* +* The queue is full and the allocating +* process is not a "batcher", and not +* exempted by the IO scheduler +*/ + goto out; } } -blk_set_queue_congested (q, is_sync); }/*
@@ -759,21 +862,60 @@ static struct request *get_request(struct request_queue *q, int rw_flags,* limit of requests, otherwise we could have thousands of requests * allocated with any setting of ->nr_requests */ -if (rl->count[is_sync] >= (3 * q->nr_requests /2 )) + +if (q->rq_data.count[is_sync] >= (3 * q->nr_requests /2 )) { +/* +* Queue is too full for allocation. On which request queue +* the task should sleep? Generally it should sleep on its +* request list but if elevator switch is happening, in that +* window, request descriptors are allocated from global +* pool and are not accounted against any particular request +* list as group is going away. +* +* So it might happen that request list does not have any +* requests allocated at all and if process sleeps on per +* group request list, it will not be woken up. In such case, +* make it sleep on global starved list. +*/ +if (test_bit (QUEUE_FLAG_ELVSWITCH , &q->queue_flags) + || !can_sleep_on_request_list (rl, is_sync)) + sleep_on_global =1 ; + goto out; + } + +/* +* Allocation of request is allowed from queue perspective. Now check +* from per group request list +*/ + +if (rl->count[is_sync] >= (3 * q->nr_group_requests /2 )) goto out; - rl->count[is_sync]++; rl->starved[is_sync] =0 ; + q->rq_data.count[is_sync]++; + priv = !test_bit (QUEUE_FLAG_ELVSWITCH , &q->queue_flags); -if (priv) - rl->elvpriv++; +if (priv) { + q->rq_data.elvpriv++; +/* +* Account the request to request list only if request is +* going to elevator. During elevator switch, there will +* be small window where group is going away and new group +* will not be allocated till elevator switch is complete. +* So till then instead of slowing down the application, +* we will continue to allocate request from total common +* pool instead of per group limit +*/ + rl->count[is_sync]++; + }if (blk_queue_io_stat (q)) rw_flags |=REQ_IO_STAT ;spin_unlock_irq (q->queue_lock); rq =blk_alloc_request (q, bio, rw_flags, priv, gfp_mask); +if (unlikely (!rq)) {/* * Allocation failed presumably due to memory. Undo anything
@@ -783,7 +925,7 @@ static struct request *get_request(struct request_queue *q, int rw_flags,* wait queue, but this is pretty rare. */ spin_lock_irq (q->queue_lock); -freed_request (q, is_sync, priv); +freed_request (q, is_sync, priv, rl);/* * in the very unlikely event that allocation failed and no
@@ -793,9 +935,8 @@ static struct request *get_request(struct request_queue *q, int rw_flags,* rq mempool into READ and WRITE */ rq_starved: -if (unlikely (rl->count[is_sync] ==0 )) - rl->starved[is_sync] =1 ; - +if (!can_sleep_on_request_list (rl, is_sync)) + sleep_on_global =1 ; goto out; }
@@ -810,6 +951,8 @@ rq_starved:trace_block_getrq (q, bio, rw_flags &1 ); out: +if (reason && sleep_on_global) + *reason =1 ;return rq; }
@@ -823,16 +966,39 @@ static struct request *get_request_wait(struct request_queue *q, int rw_flags, struct bio *bio) {const bool is_sync =rw_is_sync (rw_flags) !=0 ; +int sleep_on_global =0 ;struct request *rq; +struct request_list *rl =blk_get_request_list (q, bio); - rq =get_request (q, rw_flags, bio,GFP_NOIO ); + rq =get_request (q, rw_flags, bio,GFP_NOIO , rl, &sleep_on_global);while (!rq) {DEFINE_WAIT (wait);struct io_context *ioc; -struct request_list *rl = &q->rq; -prepare_to_wait_exclusive (&rl->wait[is_sync], &wait, -TASK_UNINTERRUPTIBLE ); +if (sleep_on_global) { +/* +* Task failed allocation and needs to wait and +* try again. There are no requests pending from +* the io group hence need to sleep on global +* wait queue. Most likely the allocation failed +* because of memory issues. +*/ + + q->rq_data.starved++; +prepare_to_wait_exclusive (&q->rq_data.starved_wait, + &wait,TASK_UNINTERRUPTIBLE ); + }else { +/* +* We are about to sleep on a request list and we +* drop queue lock. After waking up, we will do +* finish_wait() on request list and in the mean +* time group might be gone. Take a reference to +* the group now. +*/ +prepare_to_wait_exclusive (&rl->wait[is_sync], &wait, +TASK_UNINTERRUPTIBLE ); +elv_get_rl_iog (rl); + }trace_block_sleeprq (q, bio, rw_flags &1 );
@@ -850,9 +1016,25 @@ static struct request *get_request_wait(struct request_queue *q, int rw_flags,ioc_set_batching (q, ioc);spin_lock_irq (q->queue_lock); -finish_wait (&rl->wait[is_sync], &wait); - rq =get_request (q, rw_flags, bio,GFP_NOIO ); +if (sleep_on_global) { +finish_wait (&q->rq_data.starved_wait, &wait); + sleep_on_global =0 ; + }else { +/* +* We had taken a reference to the rl/iog. Put that now +*/ +finish_wait (&rl->wait[is_sync], &wait); +elv_put_rl_iog (rl); + } + +/* +* After the sleep check the rl again in case cgrop bio +* belonged to is gone and it is mapped to root group now +*/ + rl =blk_get_request_list (q, bio); + rq =get_request (q, rw_flags, bio,GFP_NOIO , rl, + &sleep_on_global); };return rq;
@@ -861,14 +1043,16 @@ static struct request *get_request_wait(struct request_queue *q, int rw_flags,struct request *blk_get_request (struct request_queue *q,int rw,gfp_t gfp_mask) {struct request *rq; +struct request_list *rl;BUG_ON (rw !=READ && rw !=WRITE );spin_lock_irq (q->queue_lock); + rl =blk_get_request_list (q,NULL );if (gfp_mask & __GFP_WAIT) { rq =get_request_wait (q, rw,NULL ); }else { - rq =get_request (q, rw,NULL , gfp_mask); + rq =get_request (q, rw,NULL , gfp_mask, rl,NULL );if (!rq)spin_unlock_irq (q->queue_lock); }
@@ -1085,12 +1269,13 @@ void __blk_put_request(struct request_queue *q, struct request *req)if (req->cmd_flags &REQ_ALLOCED ) {int is_sync =rq_is_sync (req) !=0 ;int priv = req->cmd_flags &REQ_ELVPRIV ; +struct request_list *rl =rq_rl (q, req);BUG_ON (!list_empty (&req->queuelist));BUG_ON (!hlist_unhashed (&req->hash));blk_free_request (q, req); -freed_request (q, is_sync, priv); +freed_request (q, is_sync, priv, rl); } }EXPORT_SYMBOL_GPL (__blk_put_request);
block/blk-settings.c +1 -0
@@ -149,6 +149,7 @@ void blk_queue_make_request(struct request_queue *q, make_request_fn *mfn)* set defaults */ q->nr_requests =BLKDEV_MAX_RQ ; + q->nr_group_requests =BLKDEV_MAX_GROUP_RQ ; q->make_request_fn = mfn;blk_queue_dma_alignment (q,511 );
block/blk-sysfs.c +47 -12
113 lines not on this page · raw message
block/elevator-fq.c +36 -0
@@ -925,6 +925,39 @@ static struct io_cgroup *cgroup_to_io_cgroup(struct cgroup *cgroup) struct io_cgroup, css); } +struct request_list * +elv_get_request_list_bio (struct request_queue *q,struct bio *bio) +{ +struct io_group *iog; + +if (!elv_iosched_fair_queuing_enabled (q->elevator)) + iog = q->elevator->efqd->root_group; +else + iog =elv_io_get_io_group_bio (q, bio,1 ); + +BUG_ON (!iog); +return &iog->rl; +} + +struct request_list * +elv_get_request_list_rq (struct request_queue *q,struct request *rq,int priv) +{ +struct io_group *iog; + +if (!elv_iosched_fair_queuing_enabled (q->elevator)) +return &q->elevator->efqd->root_group->rl; + +BUG_ON (priv && !rq->ioq); + +if (priv) + iog =ioq_to_io_group (rq->ioq); +else + iog = q->elevator->efqd->root_group; + +BUG_ON (!iog); +return &iog->rl; +} +/* * Search the io_group for efqd into the hash table (by now only a list) * of bgrp. Must be called under rcu_read_lock().
@@ -1281,6 +1314,8 @@ io_group_chain_alloc(struct request_queue *q, void *key, struct cgroup *cgroup)elv_get_iog (iog);io_group_path (iog); +blk_init_request_list (&iog->rl); +if (leaf ==NULL ) { leaf = iog; prev = leaf;
@@ -1502,6 +1537,7 @@ static struct io_group *io_alloc_root_group(struct request_queue *q,for (i =0 ; i <IO_IOPRIO_CLASSES ; i++) iog->sched_data.service_tree[i] =ELV_SERVICE_TREE_INIT ; +blk_init_request_list (&iog->rl);spin_lock_irq (&iocg->lock);rcu_assign_pointer (iog->key, key);hlist_add_head_rcu (&iog->group_node, &iocg->group_data);
block/elevator-fq.h +29 -0
@@ -128,6 +128,9 @@ struct io_group {/* Single ioq per group, used for noop, deadline, anticipatory */ struct io_queue *ioq; + +/* request list associated with the group */ +struct request_list rl; };struct io_cgroup {
@@ -425,11 +428,31 @@ static inline void elv_get_iog(struct io_group *iog)atomic_inc (&iog->ref); } +static inline struct io_group *rl_iog (struct request_list *rl) +{ + return container_of(rl, struct io_group, rl); +} + +static inline void elv_get_rl_iog (struct request_list *rl) +{ +elv_get_iog (rl_iog (rl)); +} + +static inline void elv_put_rl_iog (struct request_list *rl) +{ +elv_put_iog (rl_iog (rl)); +} +extern int elv_set_request_ioq (struct request_queue *q,struct request *rq,struct bio *bio,gfp_t gfp_mask);extern void elv_reset_request_ioq (struct request_queue *q,struct request *rq);extern struct io_queue *elv_lookup_ioq_bio (struct request_queue *q,struct bio *bio); +struct request_list * +elv_get_request_list_bio (struct request_queue *q,struct bio *bio); + +struct request_list * +elv_get_request_list_rq (struct request_queue *q,struct request *rq,int priv);#else /* !GROUP_IOSCHED */
@@ -469,6 +492,9 @@ elv_lookup_ioq_bio(struct request_queue *q, struct bio *bio) return NULL; } +static inline void elv_get_rl_iog (struct request_list *rl) { } +static inline void elv_put_rl_iog (struct request_list *rl) { } +#endif /* GROUP_IOSCHED */ extern ssize_t elv_slice_sync_show (struct elevator_queue *q,char *name);
@@ -578,6 +604,9 @@ static inline struct io_queue *elv_lookup_ioq_bio(struct request_queue *q,return NULL ; } +static inline void elv_get_rl_iog (struct request_list *rl) { } +static inline void elv_put_rl_iog (struct request_list *rl) { } +#endif /* CONFIG_ELV_FAIR_QUEUING */ #endif /* _ELV_SCHED_H */ #endif /* CONFIG_BLOCK */
block/elevator.c +4 -3
@@ -678,7 +678,7 @@ void elv_quiesce_start(struct request_queue *q)* make sure we don't have any requests in flight */ elv_drain_elevator (q); -while (q->rq.elvpriv) { +while (q->rq_data.elvpriv) {__blk_run_queue (q);spin_unlock_irq (q->queue_lock);msleep (10 );
@@ -777,8 +777,9 @@ void elv_insert(struct request_queue *q, struct request *rq, int where) }if (unplug_it &&blk_queue_plugged (q)) { -int nrq = q->rq.count[BLK_RW_SYNC ] + q->rq.count[BLK_RW_ASYNC ] - -queue_in_flight (q); +int nrq = q->rq_data.count[BLK_RW_SYNC ] + + q->rq_data.count[BLK_RW_ASYNC ] - +queue_in_flight (q);if (nrq >= q->unplug_thresh)__generic_unplug_device (q);
include/linux/blkdev.h +45 -2
88 lines not on this page · raw message
include/trace/events/block.h +4 -2
@@ -397,7 +397,8 @@ TRACE_EVENT(block_unplug_timer, ),TP_fast_assign ( - __entry->nr_rq = q->rq.count[READ] + q->rq.count[WRITE]; + __entry->nr_rq = q->rq_data.count[READ ] + + q->rq_data.count[WRITE];memcpy (__entry->comm, current->comm,TASK_COMM_LEN ); ),
@@ -416,7 +417,8 @@ TRACE_EVENT(block_unplug_io, ),TP_fast_assign ( - __entry->nr_rq = q->rq.count[READ] + q->rq.count[WRITE]; + __entry->nr_rq = q->rq_data.count[READ ] + + q->rq_data.count[WRITE];memcpy (__entry->comm, current->comm,TASK_COMM_LEN ); ),
kernel/trace/blktrace.c +4 -2
@@ -786,7 +786,8 @@ static void blk_add_trace_unplug_io(struct request_queue *q)struct blk_trace *bt = q->blk_trace;if (bt) { -unsigned int pdu = q->rq.count[READ ] + q->rq.count[WRITE ]; +unsigned int pdu = q->rq_data.count[READ ] + + q->rq_data.count[WRITE ];__be64 rpdu =cpu_to_be64 (pdu); __blk_add_trace(bt, 0, 0, 0, BLK_TA_UNPLUG_IO, 0,
@@ -799,7 +800,8 @@ static void blk_add_trace_unplug_timer(struct request_queue *q)struct blk_trace *bt = q->blk_trace;if (bt) { -unsigned int pdu = q->rq.count[READ ] + q->rq.count[WRITE ]; +unsigned int pdu = q->rq_data.count[READ ] + + q->rq_data.count[WRITE ];__be64 rpdu =cpu_to_be64 (pdu); __blk_add_trace(bt, 0, 0, 0, BLK_TA_UNPLUG_TIMER, 0,
Signature · 1 line
1.6.0.6
PATCH 21/23 io-controller: Per io group bdi congestion interface
In reply to a message not in the archiveo So far there used to be only one pair or queue of request descriptors (one for sync and one for async) per device and number of requests allocated used to decide whether associated bdi is congested or not.
Now with per io group request descriptor infrastructure, there is a pair of request descriptor queue per io group per device. So it might happen that overall request queue is not congested but a particular io group bio belongs to is congested.
Or, it could be otherwise that group is not congested but overall queue is congested. This can happen if user has not properly set the request descriptors limits for queue and groups.
(q->nr_requests < nr_groups * q->nr_group_requests)
Hence there is a need for new interface which can query deivce congestion status per group. This group is determined by the "struct page" IO will be done for. If page is null, then group is determined from the current task context.
o This patch introduces new set of function bdi_*_congested_group(), which take "struct page" as addition argument. These functions will call the block layer and in trun elevator to find out if the io group the page will go into is congested or not.
o Currently I have introduced the core functions and migrated most of the users. But there might be still some left. This is an ongoing TODO item.
--- block/blk-core.c | 26 ++++++++ block/blk-sysfs.c | 6 +- block/elevator-fq.c | 135 +++++++++++++++++++++++++++++++++++++++++++ block/elevator-fq.h | 24 +++++++- drivers/md/dm-table.c | 11 ++- drivers/md/dm.c | 7 +- drivers/md/dm.h | 3 +- drivers/md/linear.c | 7 ++- drivers/md/multipath.c | 7 ++- drivers/md/raid0.c | 6 +- drivers/md/raid1.c | 9 ++- drivers/md/raid10.c | 6 +- drivers/md/raid5.c | 2 +- fs/afs/write.c | 8 ++- fs/btrfs/disk-io.c | 6 +- fs/btrfs/extent_io.c | 12 ++++ fs/btrfs/volumes.c | 8 ++- fs/cifs/file.c | 11 ++++ fs/ext2/ialloc.c | 2 +- fs/gfs2/aops.c | 12 ++++ fs/nilfs2/segbuf.c | 3 +- fs/xfs/linux-2.6/xfs_aops.c | 2 +- fs/xfs/linux-2.6/xfs_buf.c | 2 +- include/linux/backing-dev.h | 63 +++++++++++++++++++- include/linux/blkdev.h | 5 ++ mm/backing-dev.c | 74 ++++++++++++++++++++++- mm/page-writeback.c | 11 ++++ mm/readahead.c | 2 +- 28 files changed, 430 insertions(+), 40 deletions(-)
block/blk-core.c +26 -0
@@ -90,6 +90,27 @@ void blk_queue_congestion_threshold(struct request_queue *q) q->nr_congestion_off = nr; } +#ifdef CONFIG_GROUP_IOSCHED +int blk_queue_io_group_congested (struct backing_dev_info *bdi,int bdi_bits, +struct page *page) +{ +int ret =0 ; +struct request_queue *q = bdi->unplug_io_data; + +if (!q || !q->elevator) +return bdi_congested (bdi, bdi_bits); + +/* Do we need to hold queue lock? */ +if (bdi_bits & (1 << BDI_sync_congested)) + ret |=elv_page_io_group_congested (q, page,1 ); + +if (bdi_bits & (1 << BDI_async_congested)) + ret |=elv_page_io_group_congested (q, page,0 ); + +return ret; +} +#endif +/** * blk_get_backing_dev_info - get the address of a queue's backing_dev_info * @bdev: device
@@ -721,6 +742,8 @@ static void __freed_request(struct request_queue *q, int sync,if (q->rq_data.count[sync] +1 <= q->nr_requests)blk_clear_queue_full (q, sync); +elv_freed_request (rl, sync); +if (rl->count[sync] +1 <= q->nr_group_requests) {if (waitqueue_active (&rl->wait[sync]))wake_up (&rl->wait[sync]);
@@ -830,6 +853,9 @@ static struct request *get_request(struct request_queue *q, int rw_flags,if (q->rq_data.count[is_sync]+1 >=queue_congestion_on_threshold (q))blk_set_queue_congested (q, is_sync); +/* check if io group will get congested after this allocation*/ +elv_get_request (rl, is_sync); +/* queue full seems redundant now */ if (q->rq_data.count[is_sync]+1 >= q->nr_requests)blk_set_queue_full (q, is_sync);
block/blk-sysfs.c +3 -3
@@ -83,9 +83,8 @@ static ssize_t queue_group_requests_show(struct request_queue *q, char *page)return queue_var_show (q->nr_group_requests, (page)); } -static ssize_t -queue_group_requests_store (struct request_queue *q,const char *page, -size_t count) +static ssize_t queue_group_requests_store (struct request_queue *q, +const char *page,size_t count) {unsigned long nr;int ret =queue_var_store (&nr, page, count);
@@ -95,6 +94,7 @@ queue_group_requests_store(struct request_queue *q, const char *page,spin_lock_irq (q->queue_lock); q->nr_group_requests = nr; +elv_updated_nr_group_requests (q);spin_unlock_irq (q->queue_lock);return ret; }
block/elevator-fq.c +135 -0
153 lines not on this page · raw message
block/elevator-fq.h +23 -1
@@ -95,6 +95,13 @@ struct io_queue { }; #ifdef CONFIG_GROUP_IOSCHED /* CONFIG_GROUP_IOSCHED */ + +enum io_group_state { + IOG_async_congested, /* The async queue of group is getting full */ + IOG_sync_congested, /* The sync queue of group is getting full */ + IOG_unused, /* Available bits start here */ +}; + struct io_group { struct io_entity entity; atomic_t ref;
@@ -129,6 +136,11 @@ struct io_group {/* Single ioq per group, used for noop, deadline, anticipatory */ struct io_queue *ioq; +/* io group congestion on and off threshold for request descriptors */ +unsigned int nr_congestion_on; +unsigned int nr_congestion_off; + +unsigned long state;/* request list associated with the group */ struct request_list rl; };
@@ -453,6 +465,11 @@ elv_get_request_list_bio(struct request_queue *q, struct bio *bio);struct request_list *elv_get_request_list_rq (struct request_queue *q,struct request *rq,int priv); +extern int elv_page_io_group_congested (struct request_queue *q, +struct page *page,int sync); +extern void elv_freed_request (struct request_list *rl,int sync); +extern void elv_get_request (struct request_list *rl,int sync); +extern void elv_updated_nr_group_requests (struct request_queue *q);#else /* !GROUP_IOSCHED */
@@ -491,9 +508,11 @@ elv_lookup_ioq_bio(struct request_queue *q, struct bio *bio) {return NULL ; } -static inline void elv_get_rl_iog (struct request_list *rl) { }static inline void elv_put_rl_iog (struct request_list *rl) { } +static inline void elv_updated_nr_group_requests (struct request_queue *q) { } +static inline void elv_freed_request (struct request_list *rl,int sync) { } +static inline void elv_get_request (struct request_list *rl,int sync) { }#endif /* GROUP_IOSCHED */
@@ -606,6 +625,9 @@ static inline struct io_queue *elv_lookup_ioq_bio(struct request_queue *q,static inline void elv_get_rl_iog (struct request_list *rl) { }static inline void elv_put_rl_iog (struct request_list *rl) { } +static inline void elv_updated_nr_group_requests (struct request_queue *q) { } +static inline void elv_freed_request (struct request_list *rl,int sync) { } +static inline void elv_get_request (struct request_list *rl,int sync) { }#endif /* CONFIG_ELV_FAIR_QUEUING */ #endif /* _ELV_SCHED_H */
drivers/md/dm-table.c +7 -4
@@ -1170,7 +1170,8 @@ int dm_table_resume_targets(struct dm_table *t)return 0 ; } -int dm_table_any_congested (struct dm_table *t,int bdi_bits) +int dm_table_any_congested (struct dm_table *t,int bdi_bits,struct page *page, +int group) {struct dm_dev_internal *dd;struct list_head *devices =dm_table_get_devices (t);
@@ -1180,9 +1181,11 @@ int dm_table_any_congested(struct dm_table *t, int bdi_bits)struct request_queue *q =bdev_get_queue (dd->dm_dev.bdev);char b[BDEVNAME_SIZE ]; -if (likely (q)) - r |=bdi_congested (&q->backing_dev_info, bdi_bits); -else +if (likely (q)) { +struct backing_dev_info *bdi = &q->backing_dev_info; + r |= group ?bdi_congested_group (bdi, bdi_bits, page) + :bdi_congested (bdi, bdi_bits); + }else DMWARN_LIMIT ("%s: any_congested: nonexistent device %s" ,dm_device_name (t->md),bdevname (dd->dm_dev.bdev, b));
drivers/md/dm.c +4 -3
@@ -1608,7 +1608,8 @@ static void dm_unplug_all(struct request_queue *q) } } -static int dm_any_congested (void *congested_data,int bdi_bits) +static int dm_any_congested (void *congested_data,int bdi_bits, +struct page *page,int group) {int r = bdi_bits;struct mapped_device *md = congested_data;
@@ -1625,8 +1626,8 @@ static int dm_any_congested(void *congested_data, int bdi_bits) r = md->queue->backing_dev_info.state & bdi_bits; else - r =dm_table_any_congested (map, bdi_bits); - + r =dm_table_any_congested (map, bdi_bits, page, + group);dm_table_put (map); } }
drivers/md/dm.h +2 -1
@@ -57,7 +57,8 @@ struct list_head *dm_table_get_devices(struct dm_table *t);void dm_table_presuspend_targets (struct dm_table *t);void dm_table_postsuspend_targets (struct dm_table *t);int dm_table_resume_targets (struct dm_table *t); -int dm_table_any_congested (struct dm_table *t,int bdi_bits); +int dm_table_any_congested (struct dm_table *t,int bdi_bits,struct page *page, +int group);int dm_table_any_busy_target (struct dm_table *t);int dm_table_set_type (struct dm_table *t);unsigned dm_table_get_type (struct dm_table *t);
drivers/md/linear.c +5 -2
@@ -102,7 +102,7 @@ static void linear_unplug(struct request_queue *q)rcu_read_unlock (); } -static int linear_congested (void *data,int bits) +static int linear_congested (void *data,int bits,struct page *page,int group) {mddev_t *mddev = data;linear_conf_t *conf;
@@ -113,7 +113,10 @@ static int linear_congested(void *data, int bits)for (i =0 ; i < mddev->raid_disks && !ret ; i++) {struct request_queue *q =bdev_get_queue (conf->disks[i].rdev->bdev); - ret |=bdi_congested (&q->backing_dev_info, bits); +struct backing_dev_info *bdi = &q->backing_dev_info; + + ret |= group ?bdi_congested_group (bdi, bits, page) : +bdi_congested (bdi, bits); }rcu_read_unlock ();
drivers/md/multipath.c +5 -2
@@ -192,7 +192,8 @@ static void multipath_status (struct seq_file *seq, mddev_t *mddev)seq_printf (seq,"]" ); } -static int multipath_congested (void *data,int bits) +static int multipath_congested (void *data,int bits,struct page *page, +int group) {mddev_t *mddev = data;multipath_conf_t *conf = mddev->private;
@@ -203,8 +204,10 @@ static int multipath_congested(void *data, int bits)mdk_rdev_t *rdev =rcu_dereference (conf->multipaths[i].rdev);if (rdev && !test_bit (Faulty, &rdev->flags)) {struct request_queue *q =bdev_get_queue (rdev->bdev); +struct backing_dev_info *bdi = &q->backing_dev_info; - ret |=bdi_congested (&q->backing_dev_info, bits); + ret |= group ?bdi_congested_group (bdi, bits, page) + :bdi_congested (bdi, bits);/* Just like multipath_map, we just check the * first available device */
drivers/md/raid0.c +4 -2
@@ -37,7 +37,7 @@ static void raid0_unplug(struct request_queue *q) } } -static int raid0_congested (void *data,int bits) +static int raid0_congested (void *data,int bits,struct page *page,int group) {mddev_t *mddev = data;raid0_conf_t *conf = mddev->private;
@@ -46,8 +46,10 @@ static int raid0_congested(void *data, int bits)for (i =0 ; i < mddev->raid_disks && !ret ; i++) {struct request_queue *q =bdev_get_queue (devlist[i]->bdev); +struct backing_dev_info *bdi = &q->backing_dev_info; - ret |=bdi_congested (&q->backing_dev_info, bits); + ret |= group ?bdi_congested_group (bdi, bits, page) + :bdi_congested (bdi, bits); }return ret; }
drivers/md/raid1.c +6 -3
@@ -570,7 +570,7 @@ static void raid1_unplug(struct request_queue *q)md_wakeup_thread (mddev->thread); } -static int raid1_congested (void *data,int bits) +static int raid1_congested (void *data,int bits,struct page *page,int group) {mddev_t *mddev = data;conf_t *conf = mddev->private;
@@ -581,14 +581,17 @@ static int raid1_congested(void *data, int bits)mdk_rdev_t *rdev =rcu_dereference (conf->mirrors[i].rdev);if (rdev && !test_bit (Faulty, &rdev->flags)) {struct request_queue *q =bdev_get_queue (rdev->bdev); +struct backing_dev_info *bdi = &q->backing_dev_info;/* Note the '|| 1' - when read_balance prefers * non-congested targets, it can be removed */ if ((bits & (1 <<BDI_async_congested)) ||1 ) - ret |=bdi_congested (&q->backing_dev_info, bits); + ret |= group ?bdi_congested_group (bdi, bits, + page) :bdi_congested (bdi, bits);else - ret &=bdi_congested (&q->backing_dev_info, bits); + ret &= group ?bdi_congested_group (bdi, bits, + page) :bdi_congested (bdi, bits); } }rcu_read_unlock ();
drivers/md/raid10.c +4 -2
@@ -625,7 +625,7 @@ static void raid10_unplug(struct request_queue *q)md_wakeup_thread (mddev->thread); } -static int raid10_congested (void *data,int bits) +static int raid10_congested (void *data,int bits,struct page *page,int group) {mddev_t *mddev = data;conf_t *conf = mddev->private;
@@ -636,8 +636,10 @@ static int raid10_congested(void *data, int bits)mdk_rdev_t *rdev =rcu_dereference (conf->mirrors[i].rdev);if (rdev && !test_bit (Faulty, &rdev->flags)) {struct request_queue *q =bdev_get_queue (rdev->bdev); +struct backing_dev_info *bdi = &q->backing_dev_info; - ret |=bdi_congested (&q->backing_dev_info, bits); + ret |= group ?bdi_congested_group (bdi, bits, page) + :bdi_congested (bdi, bits); } }rcu_read_unlock ();
drivers/md/raid5.c +1 -1
@@ -3323,7 +3323,7 @@ static void raid5_unplug_device(struct request_queue *q)unplug_slaves (mddev); } -static int raid5_congested (void *data,int bits) +static int raid5_congested (void *data,int bits,struct page *page,int group) {mddev_t *mddev = data;raid5_conf_t *conf = mddev->private;
fs/afs/write.c +7 -1
@@ -455,7 +455,7 @@ int afs_writepage(struct page *page, struct writeback_control *wbc) } wbc->nr_to_write -= ret; -if (wbc->nonblocking &&bdi_write_congested (bdi)) +if (wbc->nonblocking &&bdi_or_group_write_congested (bdi, page)) wbc->encountered_congestion =1 ;_leave (" = 0" );
@@ -491,6 +491,12 @@ static int afs_writepages_region(struct address_space *mapping,return 0 ; } +if (wbc->nonblocking &&bdi_write_congested_group (bdi, page)) { + wbc->encountered_congestion =1 ; +page_cache_release (page); +break ; + } +/* at this point we hold neither mapping->tree_lock nor lock on * the page itself: the page may be truncated or invalidated * (changing page->mapping to NULL), or even swizzled back from
fs/btrfs/disk-io.c +4 -2
@@ -1249,7 +1249,8 @@ struct btrfs_root *btrfs_read_fs_root(struct btrfs_fs_info *fs_info,return root; } -static int btrfs_congested_fn (void *congested_data,int bdi_bits) +static int btrfs_congested_fn (void *congested_data,int bdi_bits, +struct page *page,int group) {struct btrfs_fs_info *info = (struct btrfs_fs_info *)congested_data;int ret =0 ;
@@ -1260,7 +1261,8 @@ static int btrfs_congested_fn(void *congested_data, int bdi_bits)if (!device->bdev)continue ; bdi =blk_get_backing_dev_info (device->bdev); -if (bdi &&bdi_congested (bdi, bdi_bits)) { +if (bdi && (group ?bdi_congested_group (bdi, bdi_bits, page) : +bdi_congested (bdi, bdi_bits))) { ret =1 ;break ; }
fs/btrfs/extent_io.c +12 -0
@@ -2368,6 +2368,18 @@ retry:unsigned i; scanned =1 ; + +/* +* If the io group page will go into is congested, bail out. +*/ +if (wbc->nonblocking + &&bdi_write_congested_group (bdi, pvec.pages[0 ])) { + wbc->encountered_congestion =1 ; + done =1 ; +pagevec_release (&pvec); +break ; + } +for (i =0 ; i < nr_pages; i++) {struct page *page = pvec.pages[i];
fs/btrfs/volumes.c +6 -2
@@ -165,6 +165,7 @@ static noinline int run_scheduled_bios(struct btrfs_device *device)unsigned long limit;unsigned long last_waited =0 ;int force_reg =0 ; +struct page *page; bdi =blk_get_backing_dev_info (device->bdev); fs_info = device->dev_root->fs_info;
@@ -276,8 +277,11 @@ loop_lock:* is now congested. Back off and let other work structs * run instead */ -if (pending &&bdi_write_congested (bdi) && batch_run >32 && - fs_info->fs_devices->open_devices >1 ) { +if (pending) + page =bio_iovec_idx (pending,0 )->bv_page; + +if (pending &&bdi_or_group_write_congested (bdi, page) && + num_run >32 && fs_info->fs_devices->open_devices >1 ) {struct io_context *ioc; ioc = current->io_context;
fs/cifs/file.c +11 -0
@@ -1470,6 +1470,17 @@ retry: n_iov =0 ; bytes_to_write =0 ; +/* +* If the io group page will go into is congested, bail out. +*/ +if (wbc->nonblocking && +bdi_write_congested_group (bdi, pvec.pages[0 ])) { + wbc->encountered_congestion =1 ; + done =1 ; +pagevec_release (&pvec); +break ; + } +for (i =0 ; i < nr_pages; i++) { page = pvec.pages[i];/*
fs/ext2/ialloc.c +1 -1
@@ -179,7 +179,7 @@ static void ext2_preread_inode(struct inode *inode)struct backing_dev_info *bdi; bdi = inode->i_mapping->backing_dev_info; -if (bdi_read_congested (bdi)) +if (bdi_or_group_read_congested (bdi,NULL ))return ;if (bdi_write_congested (bdi))return ;
fs/gfs2/aops.c +12 -0
@@ -371,6 +371,18 @@ retry: PAGECACHE_TAG_DIRTY, min(end - index, (pgoff_t)PAGEVEC_SIZE-1) + 1))) { scanned =1 ; + +/* +* If io group page belongs to is congested. bail out. +*/ +if (wbc->nonblocking + &&bdi_write_congested_group (bdi, pvec.pages[0 ])) { + wbc->encountered_congestion =1 ; + done =1 ; +pagevec_release (&pvec); +break ; + } + ret =gfs2_write_jdata_pagevec (mapping, wbc, &pvec, nr_pages, end);if (ret) done =1 ;
fs/nilfs2/segbuf.c +2 -1
@@ -266,8 +266,9 @@ static int nilfs_submit_seg_bio(struct nilfs_write_info *wi, int mode) {struct bio *bio = wi->bio;int err; +struct page *page =bio_iovec_idx (bio,0 )->bv_page; -if (wi->nbio >0 &&bdi_write_congested (wi->bdi)) { +if (wi->nbio >0 &&bdi_or_group_write_congested (wi->bdi, page)) {wait_for_completion (&wi->bio_event); wi->nbio--;if (unlikely (atomic_read (&wi->err))) {
fs/xfs/linux-2.6/xfs_aops.c +1 -1
@@ -891,7 +891,7 @@ xfs_convert_page( bdi = inode->i_mapping->backing_dev_info; wbc->nr_to_write--; -if (bdi_write_congested (bdi)) { +if (bdi_or_group_write_congested (bdi, page)) { wbc->encountered_congestion =1 ; done =1 ; }else if (wbc->nr_to_write <=0 ) {
fs/xfs/linux-2.6/xfs_buf.c +1 -1
@@ -714,7 +714,7 @@ xfs_buf_readahead(struct backing_dev_info *bdi; bdi = target->bt_mapping->backing_dev_info; -if (bdi_read_congested (bdi)) +if (bdi_or_group_read_congested (bdi,NULL ))return ; flags |= (XBF_TRYLOCK |XBF_ASYNC |XBF_READ_AHEAD );
include/linux/backing-dev.h +60 -3
87 lines not on this page · raw message
include/linux/blkdev.h +5 -0
@@ -846,6 +846,11 @@ static inline void blk_set_queue_congested(struct request_queue *q, int sync)set_bdi_congested (&q->backing_dev_info, sync); } +#ifdef CONFIG_GROUP_IOSCHED +extern int blk_queue_io_group_congested (struct backing_dev_info *bdi, +int bdi_bits,struct page *page); +#endif +extern void blk_start_queue (struct request_queue *q);extern void blk_stop_queue (struct request_queue *q);extern void blk_sync_queue (struct request_queue *q);
mm/backing-dev.c +71 -3
96 lines not on this page · raw message
mm/page-writeback.c +11 -0
@@ -985,6 +985,17 @@ retry:if (nr_pages ==0 )break ; +/* +* If the io group page will go into is congested, bail out. +*/ +if (wbc->nonblocking + &&bdi_write_congested_group (bdi, pvec.pages[0 ])) { + wbc->encountered_congestion =1 ; + done =1 ; +pagevec_release (&pvec); +break ; + } +for (i =0 ; i < nr_pages; i++) {struct page *page = pvec.pages[i];
mm/readahead.c +1 -1
@@ -542,7 +542,7 @@ page_cache_async_readahead(struct address_space *mapping,/* * Defer asynchronous read-ahead on IO congestion. */ -if (bdi_read_congested (mapping->backing_dev_info)) +if (bdi_or_group_read_congested (mapping->backing_dev_info,NULL ))return ;/* do read-ahead */
Signature · 1 line
1.6.0.6
PATCH 22/23 io-controller: Support per cgroup per device weights and io class
In reply to a message not in the archiveThis patch enables per-cgroup per-device weight and ioprio_class handling. A new cgroup interface "policy" is introduced. You can make use of this file to configure weight and ioprio_class for each device in a given cgroup. The original "weight" and "ioprio_class" files are still available. If you don't do special configuration for a particular device, "weight" and "ioprio_class" are used as default values in this device.
You can use the following format to play with the new interface. #echo dev_major:dev_minor weight ioprio_class > /patch/to/cgroup/policy weight=0 means removing the policy for device.
Examples:
Configure weight=300 ioprio_class=2 on /dev/hdb (8:16) in this cgroup
# echo "8:16 300 2" > io.policy # cat io.policy dev weight class 8:16 300 2
Configure weight=500 ioprio_class=1 on /dev/hda (8:0) in this cgroup
# echo "8:0 500 1" > io.policy # cat io.policy dev weight class 8:0 500 1 8:16 300 2
Remove the policy for /dev/hda in this cgroup
# echo 8:0 0 1 > io.policy # cat io.policy dev weight class 8:16 300 2
Changelog (v1 -> v2)
- Rename some structures
- Use spin_lock_irqsave() and spin_lock_irqrestore() version to prevent from enabling the interrupts unconditionally.
- Fix policy setup bug when switching to another io scheduler.
- If a policy is available for a specific device, don't update weight and io class when writing "weight" and "iprio_class".
- Fix a bug when parsing policy string.
--- block/elevator-fq.c | 263 ++++++++++++++++++++++++++++++++++++++++++++++++++- block/elevator-fq.h | 10 ++ 2 files changed, 269 insertions(+), 4 deletions(-)
block/elevator-fq.c +259 -4
@@ -15,6 +15,7 @@#include <linux/blktrace_api.h> #include <linux/seq_file.h> #include <linux/biotrack.h> +#include <linux/genhd.h> #include "elevator-fq.h" const int elv_slice_sync =HZ /10 ;
@@ -866,12 +867,26 @@ EXPORT_SYMBOL(elv_io_group_set_async_queue);#ifdef CONFIG_GROUP_IOSCHED static void iocg_destroy (struct cgroup_subsys *subsys,struct cgroup *cgroup); -static void io_group_init_entity (struct io_cgroup *iocg,struct io_group *iog) +static struct io_policy_node *policy_search_node (const struct io_cgroup *iocg, +dev_t dev); +static void +io_group_init_entity (struct io_cgroup *iocg,struct io_group *iog,dev_t dev) {struct io_entity *entity = &iog->entity; +struct io_policy_node *pn; +unsigned long flags; + +spin_lock_irqsave (&iocg->lock, flags); + pn =policy_search_node (iocg, dev); +if (pn) { + entity->weight = pn->weight; + entity->ioprio_class = pn->ioprio_class; + }else { + entity->weight = iocg->weight; + entity->ioprio_class = iocg->ioprio_class; + } +spin_unlock_irqrestore (&iocg->lock, flags); - entity->weight = iocg->weight; - entity->ioprio_class = iocg->ioprio_class; entity->ioprio_changed =1 ; entity->my_sd = &iog->sched_data; }
@@ -1111,6 +1126,229 @@ io_cgroup_lookup_group(struct io_cgroup *iocg, void *key)return NULL ; } +static int io_cgroup_policy_read (struct cgroup *cgrp,struct cftype *cft, +struct seq_file *m) +{ +struct io_cgroup *iocg; +struct io_policy_node *pn; + + iocg =cgroup_to_io_cgroup (cgrp); + +if (list_empty (&iocg->policy_list)) + goto out; + +seq_printf (m,"dev\tweight\tclass\n" ); + +spin_lock_irq (&iocg->lock); +list_for_each_entry (pn, &iocg->policy_list, node) { +seq_printf (m,"%u:%u\t%u\t%hu\n" ,MAJOR (pn->dev), +MINOR (pn->dev), pn->weight, pn->ioprio_class); + } +spin_unlock_irq (&iocg->lock); +out: +return 0 ; +} + +static inline void policy_insert_node (struct io_cgroup *iocg, +struct io_policy_node *pn) +{ +list_add (&pn->node, &iocg->policy_list); +} + +/* Must be called with iocg->lock held */ +static inline void policy_delete_node (struct io_policy_node *pn) +{ +list_del (&pn->node); +} + +/* Must be called with iocg->lock held */ +static struct io_policy_node *policy_search_node (const struct io_cgroup *iocg, +dev_t dev) +{ +struct io_policy_node *pn; + +if (list_empty (&iocg->policy_list)) +return NULL ; + +list_for_each_entry (pn, &iocg->policy_list, node) { +if (pn->dev == dev) +return pn; + } + +return NULL ; +} + +static int check_dev_num (dev_t dev) +{ +int part =0 ; +struct gendisk *disk; + + disk =get_gendisk (dev, &part); +if (!disk || part) +return -ENODEV ; + +return 0 ; +} + +static int policy_parse_and_set (char *buf,struct io_policy_node *newpn) +{ +char *s[4 ], *p, *major_s =NULL , *minor_s =NULL ; +int ret; +unsigned long major, minor, temp; +int i =0 ; +dev_t dev; + +memset (s,0 ,sizeof (s)); +while ((p =strsep (&buf," " )) !=NULL ) { +if (!*p) +continue ; + s[i++] = p; + +/* Prevent from inputing too many things */ +if (i ==4 ) +break ; + } + +if (i !=3 ) +return -EINVAL ; + + p =strsep (&s[0 ],":" ); +if (p !=NULL ) + major_s = p; +else +return -EINVAL ; + + minor_s = s[0 ]; +if (!minor_s) +return -EINVAL ; + + ret =strict_strtoul (major_s,10 , &major); +if (ret) +return -EINVAL ; + + ret =strict_strtoul (minor_s,10 , &minor); +if (ret) +return -EINVAL ; + + dev =MKDEV (major, minor); + + ret =check_dev_num (dev); +if (ret) +return ret; + + newpn->dev = dev; + +if (s[1 ] ==NULL ) +return -EINVAL ; + + ret =strict_strtoul (s[1 ],10 , &temp); +if (ret || temp >IO_WEIGHT_MAX ) +return -EINVAL ; + + newpn->weight = temp; + +if (s[2 ] ==NULL ) +return -EINVAL ; + + ret =strict_strtoul (s[2 ],10 , &temp); +if (ret || temp <IOPRIO_CLASS_RT || temp >IOPRIO_CLASS_IDLE ) +return -EINVAL ; + newpn->ioprio_class = temp; + +return 0 ; +} + +static void update_iog_weight_prio (struct io_group *iog,struct io_cgroup *iocg, +struct io_policy_node *pn) +{ +if (pn->weight) { + iog->entity.weight = pn->weight; + iog->entity.ioprio_class = pn->ioprio_class; +/* +* iog weight and ioprio_class updating actually happens if +* ioprio_changed is set. So ensure ioprio_changed is not set +* until new weight and new ioprio_class are updated. +*/ +smp_wmb (); + iog->entity.ioprio_changed =1 ; + }else { + iog->entity.weight = iocg->weight; + iog->entity.ioprio_class = iocg->ioprio_class; + +/* The same as above */ +smp_wmb (); + iog->entity.ioprio_changed =1 ; + } +} + +static int io_cgroup_policy_write (struct cgroup *cgrp,struct cftype *cft, +const char *buffer) +{ +struct io_cgroup *iocg; +struct io_policy_node *newpn, *pn; +char *buf; +int ret =0 ; +int keep_newpn =0 ; +struct hlist_node *n; +struct io_group *iog; + + buf =kstrdup (buffer,GFP_KERNEL ); +if (!buf) +return -ENOMEM ; + + newpn =kzalloc (sizeof (*newpn),GFP_KERNEL ); +if (!newpn) { + ret = -ENOMEM ; + goto free_buf; + } + + ret =policy_parse_and_set (buf, newpn); +if (ret) + goto free_newpn; + +if (!cgroup_lock_live_group (cgrp)) { + ret = -ENODEV ; + goto free_newpn; + } + + iocg =cgroup_to_io_cgroup (cgrp); +spin_lock_irq (&iocg->lock); + + pn =policy_search_node (iocg, newpn->dev); +if (!pn) { +if (newpn->weight !=0 ) { +policy_insert_node (iocg, newpn); + keep_newpn =1 ; + } + goto update_io_group; + } + +if (newpn->weight ==0 ) { +/* weight == 0 means deleteing a policy */ +policy_delete_node (pn); + goto update_io_group; + } + + pn->weight = newpn->weight; + pn->ioprio_class = newpn->ioprio_class; + +update_io_group: +hlist_for_each_entry (iog, n, &iocg->group_data, group_node) { +if (iog->dev == newpn->dev) +update_iog_weight_prio (iog, iocg, newpn); + } +spin_unlock_irq (&iocg->lock); + +cgroup_unlock (); + +free_newpn: +if (!keep_newpn) +kfree (newpn); +free_buf: +kfree (buf); +return ret; +} +#define SHOW_FUNCTION (__VAR) \ static u64 io_cgroup_##__VAR##_read(struct cgroup *cgroup, \ struct cftype *cftype) \
@@ -1143,6 +1381,7 @@ static int io_cgroup_##__VAR##_write(struct cgroup *cgroup, \struct io_cgroup *iocg; \struct io_group *iog; \struct hlist_node *n; \ +struct io_policy_node *pn; \ \if (val < (__MIN) || val > (__MAX)) \return -EINVAL ; \
@@ -1155,6 +1394,9 @@ static int io_cgroup_##__VAR##_write(struct cgroup *cgroup, \spin_lock_irq (&iocg->lock); \ iocg->__VAR = (unsigned long )val; \ hlist_for_each_entry(iog, n, &iocg->group_data, group_node) { \ + pn =policy_search_node (iocg, iog->dev); \ +if (pn) \ +continue ; \ iog->entity.__VAR = (unsigned long )val; \smp_wmb (); \ iog->entity.ioprio_changed =1 ; \
@@ -1290,6 +1532,12 @@ static int io_cgroup_disk_dequeue_read(struct cgroup *cgroup,struct cftype io_files[] = { { + .name ="policy" , + .read_seq_string = io_cgroup_policy_read, + .write_string = io_cgroup_policy_write, + .max_write_len =256 , + }, + { .name ="weight" , .read_u64 = io_cgroup_weight_read, .write_u64 = io_cgroup_weight_write,
@@ -1340,6 +1588,7 @@ static struct cgroup_subsys_state *iocg_create(struct cgroup_subsys *subsys,INIT_HLIST_HEAD (&iocg->group_data); iocg->weight =IO_WEIGHT_DEFAULT ; iocg->ioprio_class =IOPRIO_CLASS_BE ; +INIT_LIST_HEAD (&iocg->policy_list);return &iocg->css; }
@@ -1433,7 +1682,7 @@ io_group_chain_alloc(struct request_queue *q, void *key, struct cgroup *cgroup)sscanf (dev_name (bdi->dev),"%u:%u" , &major, &minor); iog->dev =MKDEV (major, minor); -io_group_init_entity (iocg, iog); +io_group_init_entity (iocg, iog, iog->dev);atomic_set (&iog->ref,0 );
@@ -1761,6 +2010,7 @@ static void iocg_destroy(struct cgroup_subsys *subsys, struct cgroup *cgroup)struct io_group *iog;struct elv_fq_data *efqd;unsigned long uninitialized_var (flags ); +struct io_policy_node *pn, *pntmp;/* * io groups are linked in two lists. One list is maintained
@@ -1800,6 +2050,11 @@ remove_entry: goto remove_entry; done: +list_for_each_entry_safe (pn, pntmp, &iocg->policy_list, node) { +policy_delete_node (pn); +kfree (pn); + } +free_css_id (&io_subsys, &iocg->css);rcu_read_unlock ();BUG_ON (!hlist_empty (&iocg->group_data));
block/elevator-fq.h +10 -0
@@ -145,12 +145,22 @@ struct io_group {struct request_list rl; }; +struct io_policy_node { +struct list_head node; +dev_t dev; +unsigned int weight; +unsigned short ioprio_class; +}; +struct io_cgroup {struct cgroup_subsys_state css;unsigned int weight;unsigned short ioprio_class; +/* list of io_policy_node */ +struct list_head policy_list; +spinlock_t lock;struct hlist_head group_data; };
Signature · 1 line
1.6.0.6
PATCH 23/23 io-controller: debug elevator fair queuing support
In reply to a message not in the archiveo More debugging help to debug elevator fair queuing support. Enabled under CONFIG_DEBUG_ELV_FAIR_QUEUING. Currently it prints vdisktime related trace messages in blktrace.
--- block/Kconfig.iosched | 9 +++++++++ block/elevator-fq.c | 47 ++++++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 55 insertions(+), 1 deletions(-)
block/Kconfig.iosched +9 -0
@@ -15,6 +15,15 @@ config ELV_FAIR_QUEUING other ioschedulers can make use of it. If unsure, say N. +config DEBUG_ELV_FAIR_QUEUING + bool "Debug elevator fair queuing" + depends on ELV_FAIR_QUEUING + default n + ---help--- + Enable some debugging hooks for elevator fair queuing support. + Currently it just outputs more information about vdisktime in + blktrace output . + config IOSCHED_NOOP bool default y
block/elevator-fq.c +46 -1
73 lines not on this page · raw message
Signature · 1 line
1.6.0.6
Quoted 14 lines · Vivek Goyal
Hi All,
Here is the V9 of the IO controller patches generated on top of 2.6.31-rc7.
For ease of patching, a consolidated patch is available here.
http://people.redhat.com/~vgoyal/io-controller/io-scheduler-based-io-controller-v9.patch
Changes from V8
===============
- Implemented bdi like congestion semantics for io group also. Now once an io group gets congested, we don't clear the congestion flag until number of requests goes belownr_congestion_off.
Quoted 5 lines
This helps in getting rid of Buffered write performance regression we were observing with io controller patches.
Gui, can you please test it and see if this version is better in terms of your buffered write tests.
Ok, will do.
Signature · 2 lines
Regards Gui Jianfeng
Quoted 14 lines · Vivek Goyal
Hi All,
Here is the V9 of the IO controller patches generated on top of 2.6.31-rc7.
For ease of patching, a consolidated patch is available here.
http://people.redhat.com/~vgoyal/io-controller/io-scheduler-based-io-controller-v9.patch
Changes from V8
===============
- Implemented bdi like congestion semantics for io group also. Now once an io group gets congested, we don't clear the congestion flag until number of requests goes belownr_congestion_off.
Quoted 5 lines
This helps in getting rid of Buffered write performance regression we were observing with io controller patches.
Gui, can you please test it and see if this version is better in terms of your buffered write tests.
Hi Vivek,
Here are some performance numbers generated by fio test. It seems V9 performance is better than V8 especially for write case.
Normal Read | Random Read | Normal Write | Random Write
V8(Avg of 3 timers) 64667 KiB/s 3387 KiB/s 59197 KiB/s 9327 KiB/s
V9(Avg of 3 timers) 65947 KiB/s 3528 KiB/s 61654 KiB/s 9744 KiB/s
Performance +2.0% +4.1% +4.2% +4.5%
Hi Vivek,
I happened to encount a bug when i test IO Controller V9. When there are three tasks to run concurrently in three group, that is, one is parent group, and other two tasks are running in two different child groups respectively to read or write files in some disk, say disk "hdb", The task may hang up, and other tasks which access into "hdb" will also hang up.
The bug only happens when using AS io scheduler. The following scirpt can reproduce this bug in my box.
===========
#!/bin/sh
mkdir /cgroup
mount -t cgroup -o io,blkio io /cgroup
echo anticipatory > /sys/block/hdb/queue/scheduler
mkdir /cgroup/test1
echo 100 > /cgroup/test1/io.weight
mkdir /cgroup/test2
echo 400 > /cgroup/test2/io.weight
mkdir /cgroup/test2/test3
echo 400 > /cgroup/test2/test3/io.weight
mkdir /cgroup/test2/test4
echo 400 > /cgroup/test2/test4/io.weight
#./rwio -r -f /hdb2/2000M.3 &
dd if=/hdb2/2000M.3 of=/dev/null &
pid4=$!
echo $pid4 > /cgroup/test2/test3/tasks
echo "pid4: $pid4"
#./rwio -r -f /hdb2/2000M.1 &
dd if=/hdb2/2000M.1 of=/dev/null &
pid1=$!
echo $pid1 > /cgroup/test1/tasks
echo "pid1 $pid1"
#./rwio -r -f /hdb2/2000M.2 &
dd if=/hdb2/2000M.2 of=/dev/null &
pid2=$!
echo $pid2 > /cgroup/test2/test4/tasks
echo "pid2 $pid2"
sleep 20
for ((;1;))
{
ps -p $pid1 > /dev/null 2>&1
if [ $? -ne 0 ]; then
break
fi
kill -9 $pid1 > /dev/null 2>&1
}
for ((;1;))
{
ps -p $pid2 > /dev/null 2>&1
if [ $? -ne 0 ]; then
break
fi
kill -9 $pid2 > /dev/null 2>&1
}kill -9 $pid4 > /dev/null 2>&1
rmdir /cgroup/test2/test3
rmdir /cgroup/test2/test4
rmdir /cgroup/test2
rmdir /cgroup/test1
umount /cgroup
rmdir /cgroup
Quoted 9 lines · Vivek Goyal
Hi All,
Here is the V9 of the IO controller patches generated on top of 2.6.31-rc7.
For ease of patching, a consolidated patch is available here.
http://people.redhat.com/~vgoyal/io-controller/io-scheduler-based-io-controller-v9.patch
Found few issues during testing. Sending 3 more patches for this series. After some more testing, will merge these patches in higher level patches and post V10.
Thanks
Vivek
[PATCH 24/23] io-controller: Don't leave a queue active when a disk is idle
In reply to a message not in the archiveo It is possible that when there is only a single queue in the system, it remains unexpired for a long time (because there is no IO activity on the disk). So when next request comes in after a long time, it might make scheduler think that all this while queue used the disk and it will assign a high vdisktime to the queue. Hence make sure queue is expired once all the requests have completed from the queue.
o Also avoid unnecessarily expiring a queue when it has got one request dispatched to the queue and waiting for it to finish and it does not have more requests queued to dispatch.
--- block/elevator-fq.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
block/elevator-fq.c +15 -0
@@ -2947,6 +2947,10 @@ void *elv_select_ioq(struct request_queuif (ioq ==NULL ) goto new_queue; +/* There is only one active queue which is empty. Nothing to dispatch */ +if (elv_nr_busy_ioq (q->elevator) ==1 && !ioq->nr_queued) +return NULL ; + iog =ioq_to_io_group (ioq);/*
@@ -3236,6 +3240,17 @@ void elv_ioq_completed_request(struct reelse elv_iog_arm_slice_timer (q, iog,0 ); } + +/* +* if this is only queue and it has completed all its requests +* and has nothing to dispatch, expire it. We don't want to +* keep it around idle otherwise later when it is expired, all +* this idle time will be added to queue's disk time used. +*/ +if (efqd->busy_queues ==1 && !ioq->dispatched && + !ioq->nr_queued && !timer_pending (&efqd->idle_slice_timer)) { +elv_slice_expired (q); + } } done:if (!efqd->rq_in_driver)
[PATCH 25/23] io-controller: fix queue vs group fairness
In reply to a message not in the archiveo I found an issue during test and that is if there is a mix of queue and group at same level, there can be fairness issue. For example, consider following case.
root / \ T1 G1 | T2
T1 and T2 are two tasks with prio 0 and 7 respectively and G1 is the group with weight 900.
Task T1 prio 0 is mapped to weight 900 and it will get slice length of 180ms and then queue will expire and will be put after G1. (Note, in case of reader most liekly next request will come after queue expiry hence queue will be deleted and once the request comes in again, it will added to tree fresh. A fresh queue is added at the end of the tree. So it will be put after G1.).
Now G1 will get to run (effectivly T2 will run), T2 has prio 7, which will map to weight 200 and get slice length of 40ms and will expire after that. Now G1 will a new vtime which is effectively charge of 40ms.
Now to get fairness G1 should run more but instead T1 will be running as we gave it a vtime, same as G1.
The core issue here is that for readers, when slice expires, queue is empty and not backlogged hence it gets deleted from the tree. Because CFQ only operates in flat mode, it did a smart thing and did not keep a track of history. Instead it provides slice lenghts according to prio and if in one round of dispatch one gets fairness it is fine, otherwise upon queue expiry you will be placed at the end of service tree.
This does not work in hierarchical setups where group's slice lenght is determined not by group' weight but by the weight of the queue which will run under the group.
Hence we need to keep track of histroy and assign a new vtime based on disk time used by the current queue at the time of expiry.
But here io scheduler is little different from CFS that at the time of expiry most of the time reader's queue is empty. So one will end up deleting it from the service tree and next request comes with-in 1 ms and it gets into the tree again like a new process.
So we need to keep track of process io queue's vdisktime, even it after got deleted from io scheduler's service tree and use that same vdisktime if that queue gets backlogged again. But trusting a ioq's vdisktime is bad because it can lead to issues if a service tree min_vtime wrap around takes place between two requests of the queue. (Agreed that it can be not that easy to hit but it is possible).
Hence, keep a cache of io queues serviced recently and when a queue gets backlogged, if it is found in cache, use that vdisktime otherwise assign a new vdisktime. This cache of io queues (idle tree), is basically the idea implemented by BFQ guys. I had gotten rid of idle trees in V9 and now I am bringing it back. (Now I understand it better. :-)).
There is one good side affect of keeping the cache of recently service io queues. Now CFQ can differentiate between streaming readers and new processes doing IO. Now for a new queue (which is not in the cache), we can assign a lower vdisktime and for a streaming reader, we assign vdisktime based on disk time used. This way small file readers or the processes doing small amount of IO will have reduced latencies at the cost of little reduced throughput of streaming readers.
--- block/cfq-iosched.c | 2 block/elevator-fq.c | 252 ++++++++++++++++++++++++++++++++++++++++++++++++---- block/elevator-fq.h | 9 + 3 files changed, 246 insertions(+), 17 deletions(-)
block/elevator-fq.c +236 -16
@@ -52,6 +52,8 @@ static struct kmem_cache *elv_ioq_pool; #define elv_log_entity(entity, fmt, args...)#endif +static void check_idle_tree_release (struct io_service_tree *st); +static inline struct io_queue *ioq_of (struct io_entity *entity) {if (entity->my_sd ==NULL )
@@ -109,6 +111,11 @@ elv_prio_to_slice(struct elv_fq_data *efreturn elv_weight_slice (efqd,elv_ioq_sync (ioq), ioq->entity.weight); } +static inline int vdisktime_gt (u64 a,u64 b) +{ +return (s64)(a - b) >0 ; +} +static inline u64 max_vdisktime (u64 min_vdisktime,u64 vdisktime) {s64 delta = (s64)(vdisktime - min_vdisktime);
@@ -145,6 +152,7 @@ static void update_min_vdisktime(struct } st->min_vdisktime =max_vdisktime (st->min_vdisktime, vdisktime); +check_idle_tree_release (st); }static inline struct io_entity *parent_entity (struct io_entity *entity)
@@ -411,27 +419,46 @@ static void place_entity(struct io_servistruct rb_node *parent;struct io_entity *entry;int nr_active = st->nr_active -1 ; +struct io_queue *ioq =ioq_of (entity); +int sync =1 ; + +if (ioq) + sync =elv_ioq_sync (ioq); + +if (add_front || !nr_active) { + vdisktime = st->min_vdisktime; + goto done; + } + +if (sync && entity->vdisktime + &&vdisktime_gt (entity->vdisktime, st->min_vdisktime)) { +/* vdisktime still in future. Use old vdisktime */ + vdisktime = entity->vdisktime; + goto done; + }/* -* Currently put entity at the end of last entity. This probably will -* require adjustments as we move along +* Effectively a new queue. Assign sync queue a lower vdisktime so +* we can achieve better latencies for small file readers. For async +* queues, put them at the end of the existing queue. +* Group entities are always considered sync. */ -if (io_entity_class_idle (entity)) { - vdisktime =elv_delta_fair (ELV_IDLE_DELAY , entity); - parent =rb_last (&st->active); -if (parent) { - entry = rb_entry(parent, struct io_entity, rb_node); - vdisktime += entry->vdisktime; - } - }else if (!add_front && nr_active) { - parent =rb_last (&st->active); -if (parent) { - entry = rb_entry(parent, struct io_entity, rb_node); - vdisktime = entry->vdisktime; - } - }else +if (sync) { vdisktime = st->min_vdisktime; + goto done; + } +/* +* Put entity at the end of the tree. Effectively async queues use +* this path. +*/ + parent =rb_last (&st->active); +if (parent) { + entry = rb_entry(parent, struct io_entity, rb_node); + vdisktime = entry->vdisktime; + }else + vdisktime = st->min_vdisktime; +done: entity->vdisktime =max_vdisktime (st->min_vdisktime, vdisktime); elv_log_entity(entity, "place_entity: vdisktime=%llu" " min_vdisktime=%llu", entity->vdisktime,
@@ -447,6 +474,122 @@ static inline void io_entity_update_prio*/ init_io_entity_service_tree (entity,parent_entity (entity)); entity->ioprio_changed =0 ; + +/* +* Assign this entity a fresh vdisktime instead of using +* previous one as prio class will lead to service tree +* change and this vdisktime will not be valid on new +* service tree. +* +* TODO: Handle the case of only prio change. +*/ + entity->vdisktime =0 ; + } +} + +static void +__dequeue_io_entity_idle (struct io_service_tree *st,struct io_entity *entity) +{ +if (st->rb_leftmost_idle == &entity->rb_node) { +struct rb_node *next_node; + + next_node =rb_next (&entity->rb_node); + st->rb_leftmost_idle = next_node; + } + +rb_erase (&entity->rb_node, &st->idle); +RB_CLEAR_NODE (&entity->rb_node); +} + +static void dequeue_io_entity_idle (struct io_entity *entity) +{ +struct io_queue *ioq =ioq_of (entity); + +__dequeue_io_entity_idle (entity->st, entity); + entity->on_idle_st =0 ; +if (ioq) +elv_put_ioq (ioq); +} + +static void +__enqueue_io_entity_idle (struct io_service_tree *st,struct io_entity *entity) +{ +struct rb_node **node = &st->idle.rb_node; +struct rb_node *parent =NULL ; +struct io_entity *entry; +int leftmost =1 ; + +while (*node !=NULL ) { + parent = *node; + entry = rb_entry(parent, struct io_entity, rb_node); + +if (vdisktime_gt (entry->vdisktime, entity->vdisktime)) + node = &parent->rb_left; +else { + node = &parent->rb_right; + leftmost =0 ; + } + } + +/* +* Maintain a cache of leftmost tree entries (it is frequently +* used) +*/ +if (leftmost) + st->rb_leftmost_idle = &entity->rb_node; + +rb_link_node (&entity->rb_node, parent, node); +rb_insert_color (&entity->rb_node, &st->idle); +} + +static void enqueue_io_entity_idle (struct io_entity *entity) +{ +struct io_queue *ioq =ioq_of (entity); +struct io_group *parent_iog; + +/* +* Don't put an entity on idle tree if it has been marked for deletion. +* We are not expecting more io from this entity. No need to cache it +*/ + +if (entity->exiting) +return ; + +/* +* If parent group is exiting, don't put on idle tree. May be task got +* moved to a different cgroup and original cgroup got deleted +*/ + parent_iog =iog_of (parent_entity (entity)); +if (parent_iog->entity.exiting) +return ; + +if (ioq) +elv_get_ioq (ioq); +__enqueue_io_entity_idle (entity->st, entity); + entity->on_idle_st =1 ; +} + +static void check_idle_tree_release (struct io_service_tree *st) +{ +struct io_entity *leftmost; + +if (!st->rb_leftmost_idle) +return ; + + leftmost = rb_entry(st->rb_leftmost_idle, struct io_entity, rb_node); + +if (vdisktime_gt (st->min_vdisktime, leftmost->vdisktime)) +dequeue_io_entity_idle (leftmost); +} + +static void flush_idle_tree (struct io_service_tree *st) +{ +struct io_entity *entity; + +while (st->rb_leftmost_idle) { + entity = rb_entry(st->rb_leftmost_idle, struct io_entity, + rb_node); +dequeue_io_entity_idle (entity); } }
@@ -483,6 +626,9 @@ static void dequeue_io_entity(struct io_ st->nr_active--; sd->nr_active--;debug_update_stats_dequeue (entity); + +if (vdisktime_gt (entity->vdisktime, st->min_vdisktime)) +enqueue_io_entity_idle (entity); } static void
@@ -524,6 +670,16 @@ static void enqueue_io_entity(struct io_struct io_service_tree *st;struct io_sched_data *sd =io_entity_sched_data (entity); +if (entity->on_idle_st) +dequeue_io_entity_idle (entity); +else +/* +* This entity was not in idle tree cache. Zero out vdisktime +* so that we don't rely on old vdisktime instead assign a +* fresh one. +*/ + entity->vdisktime =0 ; +io_entity_update_prio (entity); st = entity->st; st->nr_active++;
@@ -574,6 +730,8 @@ static void requeue_io_entity(struct io_struct io_service_tree *st = entity->st;struct io_entity *next_entity; + entity->vdisktime =0 ; +if (add_front) { next_entity =__lookup_next_io_entity (st);
@@ -1937,11 +2095,18 @@ static void io_free_root_group(struct el {struct io_group *iog = e->efqd->root_group;struct io_cgroup *iocg = &io_root_cgroup; +struct io_service_tree *st; +int i;spin_lock_irq (&iocg->lock);hlist_del_rcu (&iog->group_node);spin_unlock_irq (&iocg->lock); +for (i =0 ; i <IO_IOPRIO_CLASSES ; i++) { + st = iog->sched_data.service_tree + i; +flush_idle_tree (st); + } +put_io_group_queues (e, iog);elv_put_iog (iog); }
@@ -2039,9 +2204,29 @@ EXPORT_SYMBOL(elv_put_iog);*/ static void __io_destroy_group (struct elv_fq_data *efqd,struct io_group *iog) { +struct io_service_tree *st; +int i; +struct io_entity *entity = &iog->entity; + +/* +* Mark io group for deletion so that no new entry goes in +* idle tree. Any active queue which is removed from active +* tree will not be put in to idle tree. +*/ + entity->exiting =1 ; + +/* We flush idle tree now, and don't put things in there any more. */ +for (i =0 ; i <IO_IOPRIO_CLASSES ; i++) { + st = iog->sched_data.service_tree + i; +flush_idle_tree (st); + } +hlist_del (&iog->elv_data_node);put_io_group_queues (efqd->eq, iog); +if (entity->on_idle_st) +dequeue_io_entity_idle (entity); +/* * Put the reference taken at the time of creation so that when all * queues are gone, group can be destroyed.
@@ -2374,7 +2559,13 @@ static struct io_group *io_alloc_root_grstatic void io_free_root_group (struct elevator_queue *e) {struct io_group *iog = e->efqd->root_group; +struct io_service_tree *st; +int i; +for (i =0 ; i <IO_IOPRIO_CLASSES ; i++) { + st = iog->sched_data.service_tree + i; +flush_idle_tree (st); + }put_io_group_queues (e, iog);kfree (iog); }
@@ -3257,6 +3448,35 @@ done:elv_schedule_dispatch (q); } +/* +* The process associted with ioq (in case of cfq), is going away. Mark it +* for deletion. +*/ +void elv_exit_ioq (struct io_queue *ioq) +{ +struct io_entity *entity = &ioq->entity; + +/* +* Async ioq's belong to io group and are cleaned up once group is +* being deleted. Not need to do any cleanup here even if cfq has +* dropped the reference to the queue +*/ +if (!elv_ioq_sync (ioq)) +return ; + +/* +* This queue is still under service. Just mark it so that once all +* the IO from queue is done, it is not put back in idle tree. +*/ +if (entity->on_st) { + entity->exiting =1 ; +return ; + }else if (entity->on_idle_st) { +/* Remove ioq from idle tree */ +dequeue_io_entity_idle (entity); + } +} +EXPORT_SYMBOL (elv_exit_ioq);static void elv_slab_kill (void ) {/*
block/cfq-iosched.c +2 -0
@@ -1138,6 +1138,7 @@ static void cfq_exit_cfqq(struct cfq_datelv_schedule_dispatch (cfqd->queue); } +elv_exit_ioq (cfqq->ioq);cfq_put_queue (cfqq); }
@@ -1373,6 +1374,7 @@ static void changed_cgroup(struct io_con*/ if (iog != __iog) {cic_set_cfqq (cic,NULL ,1 ); +elv_exit_ioq (sync_cfqq->ioq);cfq_put_queue (sync_cfqq); } }
block/elevator-fq.h +8 -1
@@ -33,6 +33,10 @@ struct io_service_tree {u64 min_vdisktime;struct rb_node *rb_leftmost;unsigned int nr_active; + +/* A cache of io entities which were served and expired */ +struct rb_root idle; +struct rb_node *rb_leftmost_idle; };struct io_sched_data {
@@ -44,9 +48,12 @@ struct io_sched_data {struct io_entity {struct rb_node rb_node;int on_st; +int on_idle_st;u64 vdisktime;unsigned int weight;struct io_entity *parent; +/* This io entity (queue or group) has been marked for deletion */ +unsigned int exiting;struct io_sched_data *my_sd;struct io_service_tree *st;
@@ -572,7 +579,7 @@ extern struct io_queue *elv_alloc_ioq(stextern void elv_free_ioq (struct io_queue *ioq);extern struct io_group *ioq_to_io_group (struct io_queue *ioq);extern int elv_iog_should_idle (struct io_queue *ioq); - +extern void elv_exit_ioq (struct io_queue *ioq);#else /* CONFIG_ELV_FAIR_QUEUING */ static inline struct elv_fq_data *elv_alloc_fq_data (struct request_queue *q,struct elevator_queue *e)
[PATCH 26/23] io-controller: fix writer preemption with in a group
In reply to a message not in the archiveo Found another issue during testing. Consider following hierarchy.
root / \ R1 G1 /\ R2 W
Generally in CFQ when readers and writers are running, reader immediately preempts writers and hence reader gets the better bandwidth. In case of hierarchical setup, it becomes little more tricky. In above diagram, G1 is a group and R1, R2 are readers and W is writer tasks.
Now assume W runs and then R1 runs and then R2 runs. After R2 has used its time slice, if R1 is schedule in, after couple of ms, R1 will get backlogged again in group G1, (streaming reader). But it will not preempt R1 as R1 is also a reader and also because preemption across group is not allowed for isolation reasons. Hence R2 will get backlogged in G1 and will get a vdisktime much higher than W. So when G2 gets scheduled again, W will get to run its full slice length despite the fact R2 is queue on same service tree.
The core issue here is that apart from regular preemptions (preemption across classes), CFQ also has this special notion of preemption with-in class and that can lead to issues active task is running in a differnt group than where new queue gets backlogged.
To solve the issue keep a track of this event (I am calling it late preemption). When a group becomes eligible to run again, if late_preemption is set, check if there are sync readers backlogged, and if yes, expire the writer after one round of dispatch.
This solves the issue of reader not getting enough bandwidth in hierarchical setups.
--- block/elevator-fq.c | 125 ++++++++++++++++++++++++++++++++++++++++++++++------ block/elevator-fq.h | 2 2 files changed, 114 insertions(+), 13 deletions(-)
block/elevator-fq.h +2 -0
@@ -43,6 +43,7 @@ struct io_sched_data {struct io_entity *active_entity;int nr_active;struct io_service_tree service_tree[IO_IOPRIO_CLASSES ]; +int nr_sync; };struct io_entity {
@@ -150,6 +151,7 @@ struct io_group {unsigned long state;/* request list associated with the group */ struct request_list rl; +int late_preemption; };struct io_policy_node {
block/elevator-fq.c +112 -13
225 lines not on this page · raw message
Quoted 3 lines · Vivek Goyal
Hi All,
Here is the V9 of the IO controller patches generated on top of 2.6.31-rc7.
Hi Vivek,
I've run some postgresql benchmarks for io-controller. Tests have been made with 2.6.31-rc6 kernel, without io-controller patches (when relevant) and with io-controller v8 and v9 patches.
I set up two instances of the TPC-H database, each running in their own io-cgroup. I ran two clients to these databases and tested on each that simple request:
$ select count(*) from LINEITEM; where LINEITEM is the biggest table of TPC-H (6001215 entries, 720MB). That request generates a steady stream of IOs.
Time is measure by psql (\timing switched on). Each test is run twice or more if there is any significant difference between the first two runs. Before each run, the cache is flush:
$ echo 3 > /proc/sys/vm/drop_caches
Results with 2 groups of same io policy (BE) and same io weight (1000):
w/o io-scheduler io-scheduler v8 io-scheduler v9
first second first second first second
DB DB DB DB DB DB
CFQ 48.4s 48.4s 48.2s 48.2s 48.1s 48.5s
Noop 138.0s 138.0s 48.3s 48.4s 48.5s 48.8s
AS 46.3s 47.0s 48.5s 48.7s 48.3s 48.5s
Deadl. 137.1s 137.1s 48.2s 48.3s 48.3s 48.5s
As you can see, there is no significant difference for CFQ scheduler. There is big improvement for noop and deadline schedulers (why is that happening?). The performance with anticipatory scheduler is a bit lower (~4%).
Results with 2 groups of same io policy (BE), different io weights and CFQ scheduler:
io-scheduler v8 io-scheduler v9
weights = 1000, 500 35.6s 46.7s 35.6s 46.7s
weigths = 1000, 250 29.2s 45.8s 29.2s 45.6s
The result in term of fairness is close to what we can expect from the ideal theoric case: with io weights of 1000 and 500 (1000 and 250), the first request get 2/3 (4/5) of io time as long as it runs and thus finish in about 3/4 (5/8) of total time.
Results with 2 groups of different io policies, same io weight and CFQ scheduler:
io-scheduler v8 io-scheduler v9
policy = RT, BE 22.5s 45.3s 22.4s 45.0s
policy = BE, IDLE 22.6s 44.8s 22.4s 45.0s
Here again, the result in term of fairness is very close from what we expect.
Thanks,
Jerome
PATCH 2/23 io-controller: Core of the elevator fair queuing
In reply to Vivek Goyalo This is core of the io scheduler implemented at elevator layer. This is a mix of cpu CFS scheduler and CFQ IO scheduler. Some of the bits from CFS have to be derived so that we can support hierarchical scheduling. Without cgroups or with-in group, we should essentially get same behavior as CFQ.
o This patch only shows non-hierarchical bits. Hierarhical code comes in later patches.
o This code is the building base of introducing fair queuing logic in common elevator layer so that it can be used by all the four IO schedulers.
--- block/Makefile | 2 +- block/elevator-fq.c | 404 +++++++++++++++++++++++++++++++++++++++++++++++++++ block/elevator-fq.h | 148 +++++++++++++++++++ 3 files changed, 553 insertions(+), 1 deletions(-) create mode 100644 block/elevator-fq.c create mode 100644 block/elevator-fq.h
block/Makefile +1 -1
@@ -5,7 +5,7 @@ obj-$(CONFIG_BLOCK) := elevator.o blk-core.o blk-tag.o blk-sysfs.o \ blk-barrier.o blk-settings.o blk-ioc.o blk-map.o \ blk-exec.o blk-merge.o blk-softirq.o blk-timeout.o \ - ioctl.o genhd.o scsi_ioctl.o + ioctl.o genhd.o scsi_ioctl.o elevator-fq.o obj-$(CONFIG_BLK_DEV_BSG) += bsg.o obj-$(CONFIG_IOSCHED_NOOP) += noop-iosched.o
block/elevator-fq.c new file +404 -0
404 lines not on this page · raw message
block/elevator-fq.h new file +148 -0
148 lines not on this page · raw message
Signature · 1 line
1.6.0.6
Re: [PATCH 02/23] io-controller: Core of the elevator fair queuing
Quoted 8 lines · Vivek Goyal
o This is core of the io scheduler implemented at elevator layer. This is a mix of cpu CFS scheduler and CFQ IO scheduler. Some of the bits from CFS have to be derived so that we can support hierarchical scheduling. Without cgroups or with-in group, we should essentially get same behavior as CFQ.
o This patch only shows non-hierarchical bits. Hierarhical code comes in later patches.
Quoted 7 lines
o This code is the building base of introducing fair queuing logic in common elevator layer so that it can be used by all the four IO schedulers.
Signed-off-by Fabio Checconi <[email protected]>Signed-off-by Paolo Valente <[email protected]>Signed-off-by Nauman Rafique <[email protected]>Signed-off-by Vivek Goyal <[email protected]>
Signature · 1 line
All rights reversed.
A message that is not in the archive (sent off-list or not yet received): 1251495072-7780-3-git-send-email-vgoyal-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org
Rik van Riel
Acked-byRe: [PATCH 02/23] io-controller: Core of the elevator fair queuing
Quoted 8 lines · Vivek Goyal
o This is core of the io scheduler implemented at elevator layer. This is a mix of cpu CFS scheduler and CFQ IO scheduler. Some of the bits from CFS have to be derived so that we can support hierarchical scheduling. Without cgroups or with-in group, we should essentially get same behavior as CFQ.
o This patch only shows non-hierarchical bits. Hierarhical code comes in later patches.
Quoted 7 lines
o This code is the building base of introducing fair queuing logic in common elevator layer so that it can be used by all the four IO schedulers.
Signed-off-by Fabio Checconi <[email protected]>Signed-off-by Paolo Valente <[email protected]>Signed-off-by Nauman Rafique <[email protected]>Signed-off-by Vivek Goyal <[email protected]>
Signature · 1 line
All rights reversed.
PATCH 3/23 io-controller: Common flat fair queuing code in elevaotor layer
In reply to Vivek GoyalThis is common fair queuing code in elevator layer. This is controlled by config option CONFIG_ELV_FAIR_QUEUING. This patch initially only introduces flat fair queuing support where there is only one group, "root group" and all the tasks belong to root group.
This elevator layer changes are backward compatible. That means any ioscheduler using old interfaces will continue to work.
This is essentially a lot of CFQ logic moved into common layer so that other IO schedulers can make use of that in hierarhical scheduling setup.
--- block/Kconfig.iosched | 13 + block/Makefile | 3 +- block/as-iosched.c | 2 +- block/blk.h | 6 + block/cfq-iosched.c | 2 +- block/deadline-iosched.c | 3 +- block/elevator-fq.c | 985 ++++++++++++++++++++++++++++++++++++++++++++-- block/elevator-fq.h | 229 +++++++++++ block/elevator.c | 63 +++- block/noop-iosched.c | 2 +- include/linux/blkdev.h | 14 + include/linux/elevator.h | 50 +++- 12 files changed, 1330 insertions(+), 42 deletions(-)
block/Kconfig.iosched +13 -0
@@ -2,6 +2,19 @@ if BLOCK menu "IO Schedulers" +config ELV_FAIR_QUEUING + bool "Elevator Fair Queuing Support" + default n + ---help--- + Traditionally only cfq had notion of multiple queues and it did + fair queuing at its own. With the cgroups and need of controlling + IO, now even the simple io schedulers like noop, deadline, as will + have one queue per cgroup and will need hierarchical fair queuing. + Instead of every io scheduler implementing its own fair queuing + logic, this option enables fair queuing in elevator layer so that + other ioschedulers can make use of it. + If unsure, say N. + config IOSCHED_NOOP bool default y
block/Makefile +2 -1
@@ -5,7 +5,7 @@ obj-$(CONFIG_BLOCK) := elevator.o blk-core.o blk-tag.o blk-sysfs.o \ blk-barrier.o blk-settings.o blk-ioc.o blk-map.o \ blk-exec.o blk-merge.o blk-softirq.o blk-timeout.o \ - ioctl.o genhd.o scsi_ioctl.o elevator-fq.o + ioctl.o genhd.o scsi_ioctl.o obj-$(CONFIG_BLK_DEV_BSG) += bsg.o obj-$(CONFIG_IOSCHED_NOOP) += noop-iosched.o
@@ -15,3 +15,4 @@ obj-$(CONFIG_IOSCHED_CFQ) += cfq-iosched.o obj-$(CONFIG_BLOCK_COMPAT) += compat_ioctl.o obj-$(CONFIG_BLK_DEV_INTEGRITY) += blk-integrity.o +obj-$(CONFIG_ELV_FAIR_QUEUING) += elevator-fq.o
block/as-iosched.c +1 -1
@@ -1351,7 +1351,7 @@ static void as_exit_queue(struct elevator_queue *e)/* * initialize elevator private data (as_data). */ -static void *as_init_queue (struct request_queue *q) +static void *as_init_queue (struct request_queue *q,struct elevator_queue *eq) {struct as_data *ad;
block/blk.h +6 -0
@@ -1,6 +1,8 @@#ifndef BLK_INTERNAL_H #define BLK_INTERNAL_H +#include "elevator-fq.h" +/* Amount of time in which a process may batch requests */ #define BLK_BATCH_TIME (HZ/50UL)
@@ -71,6 +73,8 @@ static inline void elv_activate_rq(struct request_queue *q, struct request *rq) {struct elevator_queue *e = q->elevator; +elv_activate_rq_fair (q, rq); +if (e->ops->elevator_activate_req_fn) e->ops->elevator_activate_req_fn (q, rq); }
@@ -79,6 +83,8 @@ static inline void elv_deactivate_rq(struct request_queue *q, struct request *rq {struct elevator_queue *e = q->elevator; +elv_deactivate_rq_fair (q, rq); +if (e->ops->elevator_deactivate_req_fn) e->ops->elevator_deactivate_req_fn (q, rq); }
block/cfq-iosched.c +1 -1
@@ -2448,7 +2448,7 @@ static void cfq_exit_queue(struct elevator_queue *e)kfree (cfqd); } -static void *cfq_init_queue (struct request_queue *q) +static void *cfq_init_queue (struct request_queue *q,struct elevator_queue *eq) {struct cfq_data *cfqd;int i;
block/deadline-iosched.c +2 -1
@@ -347,7 +347,8 @@ static void deadline_exit_queue(struct elevator_queue *e)/* * initialize elevator private data (deadline_data). */ -static void *deadline_init_queue (struct request_queue *q) +static void * +deadline_init_queue (struct request_queue *q,struct elevator_queue *eq) {struct deadline_data *dd;
block/elevator-fq.c +961 -24
1,064 lines not on this page · raw message
block/elevator-fq.h +229 -0
270 lines not on this page · raw message
block/elevator.c +51 -12
138 lines not on this page · raw message
block/noop-iosched.c +1 -1
@@ -65,7 +65,7 @@ noop_latter_request(struct request_queue *q, struct request *rq) return list_entry(rq->queuelist.next, struct request, queuelist); } -static void *noop_init_queue (struct request_queue *q) +static void *noop_init_queue (struct request_queue *q,struct elevator_queue *eq) {struct noop_data *nd;
include/linux/blkdev.h +14 -0
@@ -229,6 +229,11 @@ struct request {/* for bidi */ struct request *next_rq; + +#ifdef CONFIG_ELV_FAIR_QUEUING +/* io queue request belongs to */ +struct io_queue *ioq; +#endif };static inline unsigned short req_get_ioprio (struct request *req)
@@ -236,6 +241,15 @@ static inline unsigned short req_get_ioprio(struct request *req)return req->ioprio; } +static inline struct io_queue *req_ioq (struct request *req) +{ +#ifdef CONFIG_ELV_FAIR_QUEUING +return req->ioq; +#else +return NULL ; +#endif +} +/* * State information carried for REQ_TYPE_PM_SUSPEND and REQ_TYPE_PM_RESUME * requests. Some step values could eventually be made generic.
include/linux/elevator.h +49 -1
80 lines not on this page · raw message
Signature · 1 line
1.6.0.6
Re: [PATCH 03/23] io-controller: Common flat fair queuing code in elevaotor layer
Quoted 11 lines · Vivek Goyal
This is common fair queuing code in elevator layer. This is controlled by config option
CONFIG_ELV_FAIR_QUEUING. This patch initially only introduces flat fair queuing support where there is only one group, "root group" and all the tasks belong to root group.This elevator layer changes are backward compatible. That means any ioscheduler using old interfaces will continue to work.
This is essentially a lot of CFQ logic moved into common layer so that other IO schedulers can make use of that in hierarhical scheduling setup.
Quoted 6 lines
Signed-off-by Nauman Rafique <[email protected]>Signed-off-by Fabio Checconi <[email protected]>Signed-off-by Paolo Valente <[email protected]>Signed-off-by Aristeu Rozanski <[email protected]>Signed-off-by Gui Jianfeng <[email protected]>Signed-off-by Vivek Goyal <[email protected]>
Signature · 1 line
All rights reversed.
A message that is not in the archive (sent off-list or not yet received): 1251495072-7780-4-git-send-email-vgoyal-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org
Rik van Riel
Acked-byRe: [PATCH 03/23] io-controller: Common flat fair queuing code in elevaotor layer
Quoted 11 lines · Vivek Goyal
This is common fair queuing code in elevator layer. This is controlled by config option
CONFIG_ELV_FAIR_QUEUING. This patch initially only introduces flat fair queuing support where there is only one group, "root group" and all the tasks belong to root group.This elevator layer changes are backward compatible. That means any ioscheduler using old interfaces will continue to work.
This is essentially a lot of CFQ logic moved into common layer so that other IO schedulers can make use of that in hierarhical scheduling setup.
Quoted 6 lines
Signed-off-by Nauman Rafique <[email protected]>Signed-off-by Fabio Checconi <[email protected]>Signed-off-by Paolo Valente <[email protected]>Signed-off-by Aristeu Rozanski <[email protected]>Signed-off-by Gui Jianfeng <[email protected]>Signed-off-by Vivek Goyal <[email protected]>
Signature · 1 line
All rights reversed.
PATCH 4/23 io-controller: Modify cfq to make use of flat elevator fair queuing
In reply to Vivek GoyalThis patch changes cfq to use fair queuing code from elevator layer.
--- block/Kconfig.iosched | 3 +- block/cfq-iosched.c | 980 +++++++++++-------------------------------------- 2 files changed, 217 insertions(+), 766 deletions(-)
block/Kconfig.iosched +2 -1
@@ -3,7 +3,7 @@ if BLOCK menu "IO Schedulers" config ELV_FAIR_QUEUING - bool "Elevator Fair Queuing Support" + bool default n ---help--- Traditionally only cfq had notion of multiple queues and it did
@@ -46,6 +46,7 @@ config IOSCHED_DEADLINE config IOSCHED_CFQ tristate "CFQ I/O scheduler" + select ELV_FAIR_QUEUING default y ---help--- The CFQ I/O scheduler tries to distribute bandwidth equally
block/cfq-iosched.c +215 -765
1,571 lines not on this page · raw message
Signature · 1 line
1.6.0.6
Re: [PATCH 04/23] io-controller: Modify cfq to make use of flat elevator fair queuing
Quoted 7 lines · Vivek Goyal
This patch changes cfq to use fair queuing code from elevator layer.
Signed-off-by Nauman Rafique <[email protected]>Signed-off-by Fabio Checconi <[email protected]>Signed-off-by Paolo Valente <[email protected]>Signed-off-by Gui Jianfeng <[email protected]>Signed-off-by Vivek Goyal <[email protected]>
Signature · 1 line
All rights reversed.
A message that is not in the archive (sent off-list or not yet received): 1251495072-7780-5-git-send-email-vgoyal-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org
Rik van Riel
Acked-byRe: [PATCH 04/23] io-controller: Modify cfq to make use of flat elevator fair queuing
Quoted 7 lines · Vivek Goyal
This patch changes cfq to use fair queuing code from elevator layer.
Signed-off-by Nauman Rafique <[email protected]>Signed-off-by Fabio Checconi <[email protected]>Signed-off-by Paolo Valente <[email protected]>Signed-off-by Gui Jianfeng <[email protected]>Signed-off-by Vivek Goyal <[email protected]>
Signature · 1 line
All rights reversed.
PATCH 5/23 io-controller: Core scheduler changes to support hierarhical scheduling
In reply to Vivek Goyalo This patch introduces core changes in fair queuing scheduler to support hierarhical/group scheduling. It is enabled by CONFIG_GROUP_IOSCHED.
--- block/elevator-fq.c | 158 ++++++++++++++++++++++++++++++++++++++++++++++++--- block/elevator-fq.h | 19 ++++++ init/Kconfig | 8 +++ 3 files changed, 177 insertions(+), 8 deletions(-)
block/elevator-fq.c +150 -8
@@ -137,6 +137,88 @@ static inline struct io_group *iog_of(struct io_entity *entity)return NULL ; } +#ifdef CONFIG_GROUP_IOSCHED +/* check for entity->parent so that loop is not executed for root entity. */ +#define for_each_entity (entity) \ + for (; entity && entity->parent; entity = entity->parent) + +/* Do the two (enqueued) entities belong to the same group ? */ +static inline int +is_same_group (struct io_entity *entity,struct io_entity *new_entity) +{ +if (parent_entity (entity) ==parent_entity (new_entity)) +return 1 ; + +return 0 ; +} + +/* return depth at which a io entity is present in the hierarchy */ +static inline int depth_entity (struct io_entity *entity) +{ +int depth =0 ; + +for_each_entity (entity) + depth++; + +return depth; +} + +static void find_matching_io_entity (struct io_entity **entity, +struct io_entity **new_entity) +{ +int entity_depth, new_entity_depth; + +/* +* preemption test can be made between sibling entities who are in the +* same group i.e who have a common parent. Walk up the hierarchy of +* both entities until we find their ancestors who are siblings of +* common parent. +*/ + +/* First walk up until both entities are at same depth */ + entity_depth =depth_entity (*entity); + new_entity_depth =depth_entity (*new_entity); + +while (entity_depth > new_entity_depth) { + entity_depth--; + *entity =parent_entity (*entity); + } + +while (new_entity_depth > entity_depth) { + new_entity_depth--; + *new_entity =parent_entity (*new_entity); + } + +while (!is_same_group (*entity, *new_entity)) { + *entity =parent_entity (*entity); + *new_entity =parent_entity (*new_entity); + } +} +struct io_group *ioq_to_io_group (struct io_queue *ioq) +{ +return iog_of (parent_entity (&ioq->entity)); +} +EXPORT_SYMBOL (ioq_to_io_group); + +static inline struct io_sched_data * +io_entity_sched_data (struct io_entity *entity) +{ +return &iog_of (parent_entity (entity))->sched_data; +} + +#else /* GROUP_IOSCHED */ +#define for_each_entity (entity) \ + for (; entity != NULL; entity = NULL) + +static void find_matching_io_entity (struct io_entity **entity, +struct io_entity **new_entity) { } + +static inline int +is_same_group (struct io_entity *entity,struct io_entity *new_entity) +{ +return 1 ; +} +static inline struct elv_fq_data *efqd_of (struct io_entity *entity) {return ioq_of (entity)->efqd;
@@ -155,6 +237,7 @@ io_entity_sched_data(struct io_entity *entity) return &efqd->root_group->sched_data; } +#endif /* GROUP_IOSCHED */ static inline void init_io_entity_service_tree (struct io_entity *entity,struct io_entity *parent)
@@ -171,8 +254,10 @@ static void entity_served(struct io_entity *entity, unsigned long served, unsigned long nr_sectors) { - entity->vdisktime += elv_delta_fair(served, entity); -update_min_vdisktime (entity->st); + for_each_entity(entity) { + entity->vdisktime += elv_delta_fair(served, entity); +update_min_vdisktime (entity->st); + } } static void place_entity(struct io_service_tree *st, struct io_entity *entity,
@@ -388,14 +473,23 @@ static void put_prev_ioq(struct io_queue *ioq) {struct io_entity *entity = &ioq->entity; -put_prev_io_entity (entity); +for_each_entity (entity) { +put_prev_io_entity (entity); + } }static void dequeue_ioq (struct io_queue *ioq) {struct io_entity *entity = &ioq->entity; -dequeue_io_entity (entity); +for_each_entity (entity) { +struct io_sched_data *sd =io_entity_sched_data (entity); + +dequeue_io_entity (entity); +/* Don't dequeue parent if it has other entities besides us */ +if (sd->nr_active) +break ; + }elv_put_ioq (ioq);return ; }
@@ -406,7 +500,12 @@ static void enqueue_ioq(struct io_queue *ioq)struct io_entity *entity = &ioq->entity;elv_get_ioq (ioq); -enqueue_io_entity (entity); + +for_each_entity (entity) { +if (entity->on_st) +break ; +enqueue_io_entity (entity); + } } static inline void
@@ -638,6 +737,38 @@ void elv_io_group_set_async_queue(struct io_group *iog, int ioprio_class, }EXPORT_SYMBOL (elv_io_group_set_async_queue); +#ifdef CONFIG_GROUP_IOSCHED + +static void io_free_root_group (struct elevator_queue *e) +{ +struct io_group *iog = e->efqd->root_group; + +put_io_group_queues (e, iog); +kfree (iog); +} + +static struct io_group *io_alloc_root_group (struct request_queue *q, +struct elevator_queue *e,void *key) +{ +struct io_group *iog; +int i; + + iog =kmalloc_node (sizeof (*iog),GFP_KERNEL | __GFP_ZERO, q->node); +if (iog ==NULL ) +return NULL ; + + iog->entity.parent =NULL ; + iog->entity.my_sd = &iog->sched_data; + iog->key = key; + +for (i =0 ; i <IO_IOPRIO_CLASSES ; i++) + iog->sched_data.service_tree[i] =ELV_SERVICE_TREE_INIT ; + +return iog; +} + +#else /* CONFIG_GROUP_IOSCHED */ +static struct io_group *io_alloc_root_group (struct request_queue *q,struct elevator_queue *e,void *key) {
@@ -666,6 +797,8 @@ static void io_free_root_group(struct elevator_queue *e) kfree(iog); } +#endif /* CONFIG_GROUP_IOSCHED */ +/* * Should be called after ioq prio and class has been initialized as prio * class data will be used to determine which service tree in the group
@@ -691,9 +824,11 @@ static struct io_queue *elv_get_next_ioq(struct request_queue *q)return NULL ; sd = &efqd->root_group->sched_data; - entity =lookup_next_io_entity (sd); -if (!entity) -return NULL ; +for (; sd !=NULL ; sd = entity->my_sd) { + entity =lookup_next_io_entity (sd); +if (!entity) +return NULL ; + } ioq =ioq_of (entity);return ioq;
@@ -894,6 +1029,13 @@ static int elv_should_preempt(struct request_queue *q, struct io_queue *new_ioq, new_entity = &new_ioq->entity;/* +* In hierarchical setup, one need to traverse up the hierarchy +* till both the queues are children of same parent to make a +* decision whether to do the preemption or not. +*/ +find_matching_io_entity (&entity, &new_entity); + +/* * Allow an RT request to pre-empt an ongoing non-RT cfqq timeslice. */
block/elevator-fq.h +19 -0
@@ -86,6 +86,23 @@ struct io_queue {void *sched_queue; }; +#ifdef CONFIG_GROUP_IOSCHED /* CONFIG_GROUP_IOSCHED */ +struct io_group { + struct io_entity entity; + atomic_t ref; + struct io_sched_data sched_data; + /* + * async queue for each priority case for RT and BE class. + * Used only for cfq. + */ + + struct io_queue *async_queue[2][IOPRIO_BE_NR]; + struct io_queue *async_idle_queue; + void *key; +}; + +#else /* CONFIG_GROUP_IOSCHED */ + struct io_group { struct io_entity entity; struct io_sched_data sched_data;
@@ -99,6 +116,8 @@ struct io_group {void *key; }; +#endif /* CONFIG_GROUP_IOSCHED */ +struct elv_fq_data {struct io_group *root_group;
init/Kconfig +8 -0
@@ -612,6 +612,14 @@ config CGROUP_MEM_RES_CTLR_SWAP Now, memory usage of swap_cgroup is 2 bytes per entry. If swap page size is 4096bytes, 512k per 1Gbytes of swap. +config GROUP_IOSCHED + bool "Group IO Scheduler" + depends on CGROUPS && ELV_FAIR_QUEUING + default n + ---help--- + This feature lets IO scheduler recognize task groups and control + disk bandwidth allocation to such task groups. + endif # CGROUPS config MM_OWNER
Signature · 1 line
1.6.0.6
Re: [PATCH 05/23] io-controller: Core scheduler changes to support hierarhical scheduling
Quoted 7 lines · Vivek Goyal
o This patch introduces core changes in fair queuing scheduler to support hierarhical/group scheduling. It is enabled by
CONFIG_GROUP_IOSCHED.Signed-off-by Fabio Checconi <[email protected]>Signed-off-by Paolo Valente <[email protected]>Signed-off-by Nauman Rafique <[email protected]>Signed-off-by Vivek Goyal <[email protected]>
Signature · 1 line
All rights reversed.
A message that is not in the archive (sent off-list or not yet received): 1251495072-7780-6-git-send-email-vgoyal-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org
Rik van Riel
Acked-byRe: [PATCH 05/23] io-controller: Core scheduler changes to support hierarhical scheduling
Quoted 7 lines · Vivek Goyal
o This patch introduces core changes in fair queuing scheduler to support hierarhical/group scheduling. It is enabled by
CONFIG_GROUP_IOSCHED.Signed-off-by Fabio Checconi <[email protected]>Signed-off-by Paolo Valente <[email protected]>Signed-off-by Nauman Rafique <[email protected]>Signed-off-by Vivek Goyal <[email protected]>
Signature · 1 line
All rights reversed.
PATCH 6/23 io-controller: cgroup related changes for hierarchical group support
In reply to Vivek Goyalo This patch introduces some of the cgroup related code for io controller.
--- block/blk-ioc.c | 3 + block/elevator-fq.c | 167 +++++++++++++++++++++++++++++++++++++++++ block/elevator-fq.h | 14 ++++ include/linux/cgroup_subsys.h | 6 ++ include/linux/iocontext.h | 5 + 5 files changed, 195 insertions(+), 0 deletions(-)
block/blk-ioc.c +3 -0
@@ -95,6 +95,9 @@ struct io_context *alloc_io_context(gfp_t gfp_flags, int node)spin_lock_init (&ret->lock); ret->ioprio_changed =0 ; ret->ioprio =0 ; +#ifdef CONFIG_GROUP_IOSCHED + ret->cgroup_changed =0 ; +#endif ret->last_waited = jiffies;/* doesn't matter... */ ret->nr_batch_requests =0 ;/* because this is 0 */ ret->aic =NULL ;
block/elevator-fq.c +167 -0
173 lines not on this page · raw message
block/elevator-fq.h +14 -0
@@ -13,6 +13,7 @@#ifdef CONFIG_BLOCK #include <linux/blkdev.h> +#include <linux/cgroup.h> #ifndef _ELV_SCHED_H#define _ELV_SCHED_H
@@ -91,6 +92,8 @@ struct io_group {struct io_entity entity;atomic_t ref;struct io_sched_data sched_data; +struct hlist_node group_node; +unsigned short iocg_id;/* * async queue for each priority case for RT and BE class. * Used only for cfq.
@@ -101,6 +104,17 @@ struct io_group {void *key; }; +struct io_cgroup { +struct cgroup_subsys_state css; + +unsigned int weight; +unsigned short ioprio_class; + +spinlock_t lock; +struct hlist_head group_data; +}; + +#else /* CONFIG_GROUP_IOSCHED */ struct io_group {
include/linux/cgroup_subsys.h +6 -0
@@ -60,3 +60,9 @@ SUBSYS(net_cls)#endif /* */ + +#ifdef CONFIG_GROUP_IOSCHED +SUBSYS(io) +#endif + +/* */
include/linux/iocontext.h +5 -0
@@ -73,6 +73,11 @@ struct io_context {unsigned short ioprio;unsigned short ioprio_changed; +#ifdef CONFIG_GROUP_IOSCHED +/* If task changes the cgroup, elevator processes it asynchronously */ +unsigned short cgroup_changed; +#endif +/* * For request batching */
Signature · 1 line
1.6.0.6
Re: [PATCH 06/23] io-controller: cgroup related changes for hierarchical group support
Quoted 7 lines · Vivek Goyal
o This patch introduces some of the cgroup related code for io controller.
Signed-off-by Fabio Checconi <[email protected]>Signed-off-by Paolo Valente <[email protected]>Signed-off-by Nauman Rafique <[email protected]>Signed-off-by Gui Jianfeng <[email protected]>Signed-off-by Vivek Goyal <[email protected]>
Signature · 1 line
All rights reversed.
A message that is not in the archive (sent off-list or not yet received): 1251495072-7780-7-git-send-email-vgoyal-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org
Rik van Riel
Acked-byRe: [PATCH 06/23] io-controller: cgroup related changes for hierarchical group support
Quoted 7 lines · Vivek Goyal
o This patch introduces some of the cgroup related code for io controller.
Signed-off-by Fabio Checconi <[email protected]>Signed-off-by Paolo Valente <[email protected]>Signed-off-by Nauman Rafique <[email protected]>Signed-off-by Gui Jianfeng <[email protected]>Signed-off-by Vivek Goyal <[email protected]>
Signature · 1 line
All rights reversed.
PATCH 7/23 io-controller: Common hierarchical fair queuing code in elevaotor layer
In reply to Vivek Goyalo This patch enables hierarchical fair queuing in common layer. It is controlled by config option CONFIG_GROUP_IOSCHED.
o Requests keep a reference on ioq and ioq keeps keep a reference on groups. For async queues in CFQ, and single ioq in other schedulers, io_group also keeps are reference on io_queue. This reference on ioq is dropped when the queue is released (elv_release_ioq). So the queue can be freed.
When a queue is released, it puts the reference to io_group and the io_group is released after all the queues are released. Child groups also take reference on parent groups, and release it when they are destroyed.
o Reads of iocg->group_data are not always iocg->lock; so all the operations on that list are still protected by RCU. All modifications to iocg->group_data should always done under iocg->lock.
Whenever iocg->lock and queue_lock can both be held, queue_lock should be held first. This avoids all deadlocks. In order to avoid race between cgroup deletion and elevator switch the following algorithm is used:
- Cgroup deletion path holds iocg->lock and removes iog entry to iocg->group_data list. Then it drops iocg->lock, holds queue_lock and destroys iog. So in this path, we never hold iocg->lock and queue_lock at the same time. Also, since we remove iog from iocg->group_data under iocg->lock, we can't race with elevator switch.
- Elevator switch path does not remove iog from iocg->group_data list directly. It first hold iocg->lock, scans iocg->group_data again to see if iog is still there; it removes iog only if it finds iog there. Otherwise, cgroup deletion must have removed it from the list, and cgroup deletion is responsible for removing iog.
So the path which removes iog from iocg->group_data list does the final removal of iog by calling __io_destroy_group() function.
--- block/cfq-iosched.c | 2 + block/elevator-fq.c | 479 +++++++++++++++++++++++++++++++++++++++++++++++++-- block/elevator-fq.h | 35 ++++ block/elevator.c | 4 + 4 files changed, 509 insertions(+), 11 deletions(-)
block/cfq-iosched.c +2 -0
@@ -1356,6 +1356,8 @@ alloc_cfqq:/* call it after cfq has initialized queue prio */ elv_init_ioq_io_group (ioq, iog); +/* ioq reference on iog */ +elv_get_iog (iog);cfq_log_cfqq (cfqd, cfqq,"alloced" ); }else { cfqq = &cfqd->oom_cfqq;
block/elevator-fq.c +468 -11
592 lines not on this page · raw message
block/elevator-fq.h +35 -0
@@ -93,6 +93,7 @@ struct io_group {atomic_t ref;struct io_sched_data sched_data;struct hlist_node group_node; +struct hlist_node elv_data_node;unsigned short iocg_id;/* * async queue for each priority case for RT and BE class.
@@ -102,6 +103,7 @@ struct io_group {struct io_queue *async_queue[2 ][IOPRIO_BE_NR ];struct io_queue *async_idle_queue;void *key; +struct rcu_head rcu_head; };struct io_cgroup {
@@ -135,6 +137,9 @@ struct io_group {struct elv_fq_data {struct io_group *root_group; +/* List of io groups hanging on this elevator */ +struct hlist_head group_list; +struct request_queue *queue;struct elevator_queue *eq;unsigned int busy_queues;
@@ -315,6 +320,28 @@ static inline struct io_queue *elv_get_oom_ioq(struct elevator_queue *eq)return &eq->efqd->oom_ioq; } +#ifdef CONFIG_GROUP_IOSCHED + +extern int elv_io_group_allow_merge (struct request *rq,struct bio *bio); +extern void elv_put_iog (struct io_group *iog); +extern struct io_group *elv_io_get_io_group (struct request_queue *q, +int create); + +static inline void elv_get_iog (struct io_group *iog) +{ +atomic_inc (&iog->ref); +} + +#else /* !GROUP_IOSCHED */ + +static inline int elv_io_group_allow_merge (struct request *rq,struct bio *bio) +{ +return 1 ; +} + +static inline void elv_get_iog (struct io_group *iog) {} +static inline void elv_put_iog (struct io_group *iog) {} +static inline struct io_group *elv_io_get_io_group (struct request_queue *q,int create) {
@@ -322,6 +349,8 @@ elv_io_get_io_group(struct request_queue *q, int create) return q->elevator->efqd->root_group; } +#endif /* GROUP_IOSCHED */ +extern ssize_t elv_slice_sync_show (struct elevator_queue *q,char *name);extern ssize_t elv_slice_sync_store (struct elevator_queue *q,const char *name,size_t count);
@@ -405,6 +434,12 @@ static inline void *elv_select_ioq(struct request_queue *q, int force) {return NULL ; } + +static inline int elv_io_group_allow_merge (struct request *rq,struct bio *bio) + +{ +return 1 ; +}#endif /* CONFIG_ELV_FAIR_QUEUING */ #endif /* _ELV_SCHED_H */ #endif /* CONFIG_BLOCK */
block/elevator.c +4 -0
@@ -122,6 +122,10 @@ int elv_rq_merge_ok(struct request *rq, struct bio *bio) !bio_failfast_driver(bio) != !blk_failfast_driver(rq))return 0 ; +/* If rq and bio belongs to different groups, dont allow merging */ +if (!elv_io_group_allow_merge (rq, bio)) +return 0 ; +if (!elv_iosched_allow_merge (rq, bio))return 0 ;
Signature · 1 line
1.6.0.6
Re: [PATCH 07/23] io-controller: Common hierarchical fair queuing code in elevaotor layer
Quoted 2 lines · Vivek Goyal
o This patch enables hierarchical fair queuing in common layer. It is controlled by config option
CONFIG_GROUP_IOSCHED.
Signature · 1 line
All rights reversed.
A message that is not in the archive (sent off-list or not yet received): 1251495072-7780-8-git-send-email-vgoyal-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org
Rik van Riel
Acked-byRe: [PATCH 07/23] io-controller: Common hierarchical fair queuing code in elevaotor layer
Quoted 2 lines · Vivek Goyal
o This patch enables hierarchical fair queuing in common layer. It is controlled by config option
CONFIG_GROUP_IOSCHED.
Signature · 1 line
All rights reversed.
Re: [PATCH 07/23] io-controller: Common hierarchical fair queuing code in elevaotor layer
In reply to a message not in the archiveHi,
Vivek Goyal wrote, on 08/28/2009 05:30 PM:
Quoted 6 lines
(quoted diff) +6
-0+static struct io_group *io_find_alloc_group(struct request_queue *q, + struct cgroup *cgroup, struct elv_fq_data *efqd, + int create) +{ + struct io_cgroup *iocg = cgroup_to_io_cgroup(cgroup); + struct io_group *iog = NULL;
Review
Quoted 10 lines
(quoted diff) +10
-0+ /* Note: Use efqd as key */ + void *key = efqd; + + /* + * Take a refenrece to css object. Don't want to map a bio to + * a group if it has been marked for deletion + */ + + if (!css_tryget(&iocg->css)) + return iog;
cgroup_to_io_cgroup() returns NULL if only blkio subsystem is mounted but io subsystem is not. It can cause NULL pointer access.
--- block/elevator-fq.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
block/elevator-fq.c +1 -1
@@ -1827,7 +1827,7 @@ static struct io_group *io_find_alloc_group(struct request_queue *q,* a group if it has been marked for deletion */ -if (!css_tryget (&iocg->css)) +if (!iocg || !css_tryget (&iocg->css))return iog; iog =io_cgroup_lookup_group (iocg, key);
Signature · 7 lines
1.6.2.5
--
IKEDA, Munehiro
NEC Corporation of America
m-ikeda-MDRzhb/z0dd8UrSeD/[email protected]Re: [PATCH 07/23] io-controller: Common hierarchical fair queuing code in elevaotor layer
In reply to Vivek GoyalHi,
Vivek Goyal wrote, on 08/28/2009 05:30 PM:
Quoted 6 lines
(quoted diff) +6
-0+static struct io_group *io_find_alloc_group(struct request_queue *q, + struct cgroup *cgroup, struct elv_fq_data *efqd, + int create) +{ + struct io_cgroup *iocg = cgroup_to_io_cgroup(cgroup); + struct io_group *iog = NULL;
Review
Quoted 10 lines
(quoted diff) +10
-0+ /* Note: Use efqd as key */ + void *key = efqd; + + /* + * Take a refenrece to css object. Don't want to map a bio to + * a group if it has been marked for deletion + */ + + if (!css_tryget(&iocg->css)) + return iog;
cgroup_to_io_cgroup() returns NULL if only blkio subsystem is mounted but io subsystem is not. It can cause NULL pointer access.
--- block/elevator-fq.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
block/elevator-fq.c +1 -1
@@ -1827,7 +1827,7 @@ static struct io_group *io_find_alloc_group(struct request_queue *q,* a group if it has been marked for deletion */ -if (!css_tryget (&iocg->css)) +if (!iocg || !css_tryget (&iocg->css))return iog; iog =io_cgroup_lookup_group (iocg, key);
Signature · 7 lines
1.6.2.5
--
IKEDA, Munehiro
NEC Corporation of America
[email protected]Re: [PATCH 07/23] io-controller: Common hierarchical fair queuing code in elevaotor layer
Quoted 15 lines · Munehiro Ikeda
Hi,
Vivek Goyal wrote, on 08/28/2009 05:30 PM:
Quoted 12 lines
(quoted diff) +12
-0+static struct io_group *io_find_alloc_group(struct request_queue *q, + struct cgroup *cgroup, struct elv_fq_data *efqd, + int create) +{ + struct io_cgroup *iocg = cgroup_to_io_cgroup(cgroup); + struct io_group *iog = NULL; + /* Note: Use efqd as key */ + void *key = efqd; + + /* + * Take a refenrece to css object. Don't want to map a bio to + * a group if it has been marked for deletion
Quoted 9 lines
Review
Quoted 4 lines
(quoted diff) +4
-0+ */ + + if (!css_tryget(&iocg->css)) + return iog;
cgroup_to_io_cgroup()returns NULL if only blkio subsystem is mounted but io subsystem is not. It can cause NULL pointer access.
Good catch Muuhh. Thanks. Applied for next release.
Thanks
Vivek
Quoted 26 lines
Signed-off-by Munehiro "Muuhh" Ikeda <[email protected]>--- block/elevator-fq.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)block/elevator-fq.c +1
-1@@ -1827,7 +1827,7 @@ static struct io_group *io_find_alloc_group(struct request_queue *q,* a group if it has been marked for deletion */ -if (!css_tryget (&iocg->css)) +if (!iocg || !css_tryget (&iocg->css))return iog; iog =io_cgroup_lookup_group (iocg, key);Signature · 7 lines
1.6.2.5 -- IKEDA, Munehiro NEC Corporation of America [email protected]
A message that is not in the archive (sent off-list or not yet received): 4A9F3319.8040509-MDRzhb/z0dd8UrSeD/[email protected]
Re: [PATCH 07/23] io-controller: Common hierarchical fair queuing code in elevaotor layer
Quoted 15 lines · Munehiro Ikeda
Hi,
Vivek Goyal wrote, on 08/28/2009 05:30 PM:
Quoted 12 lines
(quoted diff) +12
-0+static struct io_group *io_find_alloc_group(struct request_queue *q, + struct cgroup *cgroup, struct elv_fq_data *efqd, + int create) +{ + struct io_cgroup *iocg = cgroup_to_io_cgroup(cgroup); + struct io_group *iog = NULL; + /* Note: Use efqd as key */ + void *key = efqd; + + /* + * Take a refenrece to css object. Don't want to map a bio to + * a group if it has been marked for deletion
Quoted 9 lines
Review
Quoted 4 lines
(quoted diff) +4
-0+ */ + + if (!css_tryget(&iocg->css)) + return iog;
cgroup_to_io_cgroup()returns NULL if only blkio subsystem is mounted but io subsystem is not. It can cause NULL pointer access.
Good catch Muuhh. Thanks. Applied for next release.
Thanks
Vivek
Quoted 26 lines
--- block/elevator-fq.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)block/elevator-fq.c +1
-1@@ -1827,7 +1827,7 @@ static struct io_group *io_find_alloc_group(struct request_queue *q,* a group if it has been marked for deletion */ -if (!css_tryget (&iocg->css)) +if (!iocg || !css_tryget (&iocg->css))return iog; iog =io_cgroup_lookup_group (iocg, key);Signature · 7 lines
1.6.2.5 -- IKEDA, Munehiro NEC Corporation of America m-ikeda-MDRzhb/z0dd8UrSeD/[email protected]
PATCH 8/23 io-controller: cfq changes to use hierarchical fair queuing code in elevaotor layer
In reply to Vivek GoyalMake cfq hierarhical.
--- block/Kconfig.iosched | 8 ++++++ block/cfq-iosched.c | 63 ++++++++++++++++++++++++++++++++++++++++++++++-- init/Kconfig | 2 +- 3 files changed, 69 insertions(+), 4 deletions(-)
block/Kconfig.iosched +8 -0
@@ -54,6 +54,14 @@ config IOSCHED_CFQ working environment, suitable for desktop systems. This is the default I/O scheduler. +config IOSCHED_CFQ_HIER + bool "CFQ Hierarchical Scheduling support" + depends on IOSCHED_CFQ && CGROUPS + select GROUP_IOSCHED + default n + ---help--- + Enable hierarhical scheduling in cfq. + choice prompt "Default I/O scheduler" default DEFAULT_CFQ
block/cfq-iosched.c +60 -3
87 lines not on this page · raw message
init/Kconfig +1 -1
@@ -613,7 +613,7 @@ config CGROUP_MEM_RES_CTLR_SWAP size is 4096bytes, 512k per 1Gbytes of swap. config GROUP_IOSCHED - bool "Group IO Scheduler" + bool depends on CGROUPS && ELV_FAIR_QUEUING default n ---help---
Signature · 1 line
1.6.0.6
Re: [PATCH 08/23] io-controller: cfq changes to use hierarchical fair queuing code in elevaotor layer
Quoted 7 lines · Vivek Goyal
Make cfq hierarhical.
Signed-off-by Nauman Rafique <[email protected]>Signed-off-by Fabio Checconi <[email protected]>Signed-off-by Paolo Valente <[email protected]>Signed-off-by Aristeu Rozanski <[email protected]>Signed-off-by Vivek Goyal <[email protected]>
Signature · 1 line
All rights reversed.
A message that is not in the archive (sent off-list or not yet received): 1251495072-7780-9-git-send-email-vgoyal-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org
Rik van Riel
Acked-byRe: [PATCH 08/23] io-controller: cfq changes to use hierarchical fair queuing code in elevaotor layer
Quoted 7 lines · Vivek Goyal
Make cfq hierarhical.
Signed-off-by Nauman Rafique <[email protected]>Signed-off-by Fabio Checconi <[email protected]>Signed-off-by Paolo Valente <[email protected]>Signed-off-by Aristeu Rozanski <[email protected]>Signed-off-by Vivek Goyal <[email protected]>
Signature · 1 line
All rights reversed.
PATCH 9/23 io-controller: Export disk time used and nr sectors dipatched through cgroups
In reply to Vivek Goyalo This patch exports some statistics through cgroup interface. Two of the statistics currently exported are actual disk time assigned to the cgroup and actual number of sectors dispatched to disk on behalf of this cgroup.
--- block/elevator-fq.c | 76 +++++++++++++++++++++++++++++++++++++++++++++++++++ block/elevator-fq.h | 10 +++++++ 2 files changed, 86 insertions(+), 0 deletions(-)
block/elevator-fq.c +76 -0
@@ -13,6 +13,7 @@#include <linux/blkdev.h> #include <linux/blktrace_api.h> +#include <linux/seq_file.h> #include "elevator-fq.h" const int elv_slice_sync =HZ /10 ;
@@ -257,6 +258,8 @@ entity_served(struct io_entity *entity, unsigned long served,for_each_entity (entity) { entity->vdisktime +=elv_delta_fair (served, entity);update_min_vdisktime (entity->st); + entity->total_time += served; + entity->total_sectors += nr_sectors; } }
@@ -854,6 +857,66 @@ STORE_FUNCTION(weight, IO_WEIGHT_MIN, IO_WEIGHT_MAX);STORE_FUNCTION (ioprio_class,IOPRIO_CLASS_RT ,IOPRIO_CLASS_IDLE );#undef STORE_FUNCTION +static int io_cgroup_disk_time_read (struct cgroup *cgroup, +struct cftype *cftype,struct seq_file *m) +{ +struct io_cgroup *iocg; +struct io_group *iog; +struct hlist_node *n; + +if (!cgroup_lock_live_group (cgroup)) +return -ENODEV ; + + iocg =cgroup_to_io_cgroup (cgroup); + +rcu_read_lock (); +hlist_for_each_entry_rcu (iog, n, &iocg->group_data, group_node) { +/* +* There might be groups which are not functional and +* waiting to be reclaimed upon cgoup deletion. +*/ +if (iog->key) { +seq_printf (m,"%u:%u %lu\n" ,MAJOR (iog->dev), +MINOR (iog->dev), + iog->entity.total_time); + } + } +rcu_read_unlock (); +cgroup_unlock (); + +return 0 ; +} + +static int io_cgroup_disk_sectors_read (struct cgroup *cgroup, +struct cftype *cftype,struct seq_file *m) +{ +struct io_cgroup *iocg; +struct io_group *iog; +struct hlist_node *n; + +if (!cgroup_lock_live_group (cgroup)) +return -ENODEV ; + + iocg =cgroup_to_io_cgroup (cgroup); + +rcu_read_lock (); +hlist_for_each_entry_rcu (iog, n, &iocg->group_data, group_node) { +/* +* There might be groups which are not functional and +* waiting to be reclaimed upon cgoup deletion. +*/ +if (iog->key) { +seq_printf (m,"%u:%u %lu\n" ,MAJOR (iog->dev), +MINOR (iog->dev), + iog->entity.total_sectors); + } + } +rcu_read_unlock (); +cgroup_unlock (); + +return 0 ; +} +struct cftype io_files[] = { { .name ="weight" ,
@@ -865,6 +928,14 @@ struct cftype io_files[] = { .read_u64 = io_cgroup_ioprio_class_read, .write_u64 = io_cgroup_ioprio_class_write, }, + { + .name = "disk_time", + .read_seq_string = io_cgroup_disk_time_read, + }, + { + .name = "disk_sectors", + .read_seq_string = io_cgroup_disk_sectors_read, + }, };static int iocg_populate (struct cgroup_subsys *subsys,struct cgroup *cgroup)
@@ -956,6 +1027,8 @@ io_group_chain_alloc(struct request_queue *q, void *key, struct cgroup *cgroup)struct io_cgroup *iocg;struct io_group *iog, *leaf =NULL , *prev =NULL ;gfp_t flags =GFP_ATOMIC | __GFP_ZERO; +unsigned int major, minor; +struct backing_dev_info *bdi = &q->backing_dev_info;for (; cgroup !=NULL ; cgroup = cgroup->parent) { iocg =cgroup_to_io_cgroup (cgroup);
@@ -976,6 +1049,9 @@ io_group_chain_alloc(struct request_queue *q, void *key, struct cgroup *cgroup) iog->iocg_id =css_id (&iocg->css); +sscanf (dev_name (bdi->dev),"%u:%u" , &major, &minor); + iog->dev =MKDEV (major, minor); +io_group_init_entity (iocg, iog);atomic_set (&iog->ref,0 );
block/elevator-fq.h +10 -0
@@ -53,6 +53,13 @@ struct io_entity {unsigned short ioprio, ioprio_class;int ioprio_changed; + +/* +* Keep track of total service received by this entity. Keep the +* stats both for time slices and number of sectors dispatched +*/ +unsigned long total_time; +unsigned long total_sectors; };/*
@@ -104,6 +111,9 @@ struct io_group {struct io_queue *async_idle_queue;void *key;struct rcu_head rcu_head; + +/* The device MKDEV(major, minor), this group has been created for */ +dev_t dev; };struct io_cgroup {
Signature · 1 line
1.6.0.6
Re: [PATCH 09/23] io-controller: Export disk time used and nr sectors dipatched through cgroups
Quoted 6 lines · Vivek Goyal
o This patch exports some statistics through cgroup interface. Two of the statistics currently exported are actual disk time assigned to the cgroup and actual number of sectors dispatched to disk on behalf of this cgroup.
Signed-off-by Gui Jianfeng <[email protected]>Signed-off-by Vivek Goyal <[email protected]>
Signature · 1 line
All rights reversed.
A message that is not in the archive (sent off-list or not yet received): 1251495072-7780-10-git-send-email-vgoyal-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org
Rik van Riel
Acked-byRe: [PATCH 09/23] io-controller: Export disk time used and nr sectors dipatched through cgroups
Quoted 6 lines · Vivek Goyal
o This patch exports some statistics through cgroup interface. Two of the statistics currently exported are actual disk time assigned to the cgroup and actual number of sectors dispatched to disk on behalf of this cgroup.
Signed-off-by Gui Jianfeng <[email protected]>Signed-off-by Vivek Goyal <[email protected]>
Signature · 1 line
All rights reversed.
PATCH 10/23 io-controller: Debug hierarchical IO scheduling
In reply to Vivek Goyalo Littile debugging aid for hierarchical IO scheduling.
o Enabled under CONFIG_DEBUG_GROUP_IOSCHED
o Currently it outputs more debug messages in blktrace output which helps a great deal in debugging in hierarchical setup. It also creates additional cgroup interfaces io.disk_queue and io.disk_dequeue to output some more debugging data.
--- block/Kconfig.iosched | 8 +++ block/elevator-fq.c | 168 ++++++++++++++++++++++++++++++++++++++++++++++++- block/elevator-fq.h | 29 +++++++++ 3 files changed, 202 insertions(+), 3 deletions(-)
block/Kconfig.iosched +8 -0
@@ -90,6 +90,14 @@ config DEFAULT_IOSCHED default "cfq" if DEFAULT_CFQ default "noop" if DEFAULT_NOOP +config DEBUG_GROUP_IOSCHED + bool "Debug Hierarchical Scheduling support" + depends on CGROUPS && GROUP_IOSCHED + default n + ---help--- + Enable some debugging hooks for hierarchical scheduling support. + Currently it just outputs more information in blktrace output. + endmenu endif
block/elevator-fq.c +165 -3
243 lines not on this page · raw message
block/elevator-fq.h +29 -0
@@ -114,6 +114,16 @@ struct io_group {/* The device MKDEV(major, minor), this group has been created for */ dev_t dev; +#ifdef CONFIG_DEBUG_GROUP_IOSCHED +/* How many times this group has been added to active tree */ +unsigned long queue; + +/* How many times this group has been removed from active tree */ +unsigned long dequeue; + +/* Store cgroup path */ +char path[128 ]; +#endif };struct io_cgroup {
@@ -170,10 +180,29 @@ struct elv_fq_data { };/* Logging facilities. */ +#ifdef CONFIG_DEBUG_GROUP_IOSCHED +#define elv_log_ioq(efqd, ioq, fmt, args...) \ +{ \ + blk_add_trace_msg((efqd)->queue, "elv%d%c %s " fmt, (ioq)->pid, \ + elv_ioq_sync(ioq) ? 'S' : 'A', \ + ioq_to_io_group(ioq)->path, ##args); \ +} + +#define elv_log_iog(efqd, iog, fmt, args...) \ +{ \ + blk_add_trace_msg((efqd)->queue, "elv %s " fmt, (iog)->path, ##args); \ +} + +#else #define elv_log_ioq(efqd, ioq, fmt, args...) \ blk_add_trace_msg((efqd)->queue, "elv%d%c " fmt, (ioq)->pid, \ elv_ioq_sync(ioq) ? 'S' : 'A', ##args) +#define elv_log_iog(efqd, iog, fmt, args...) \ + blk_add_trace_msg((efqd)->queue, "elv " fmt, ##args) + +#endif + #define elv_log(efqd, fmt, args...) \ blk_add_trace_msg((efqd)->queue, "elv " fmt, ##args)
Signature · 1 line
1.6.0.6
Re: [PATCH 10/23] io-controller: Debug hierarchical IO scheduling
Quoted 11 lines · Vivek Goyal
o Littile debugging aid for hierarchical IO scheduling.
o Enabled under
CONFIG_DEBUG_GROUP_IOSCHEDo Currently it outputs more debug messages in blktrace output which helps a great deal in debugging in hierarchical setup. It also creates additional cgroup interfaces io.disk_queue and io.disk_dequeue to output some more debugging data.
Signed-off-by Gui Jianfeng <[email protected]>Signed-off-by Vivek Goyal <[email protected]>
Signature · 1 line
All rights reversed.
A message that is not in the archive (sent off-list or not yet received): 1251495072-7780-11-git-send-email-vgoyal-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org
Rik van Riel
Acked-byRe: [PATCH 10/23] io-controller: Debug hierarchical IO scheduling
Quoted 11 lines · Vivek Goyal
o Littile debugging aid for hierarchical IO scheduling.
o Enabled under
CONFIG_DEBUG_GROUP_IOSCHEDo Currently it outputs more debug messages in blktrace output which helps a great deal in debugging in hierarchical setup. It also creates additional cgroup interfaces io.disk_queue and io.disk_dequeue to output some more debugging data.
Signed-off-by Gui Jianfeng <[email protected]>Signed-off-by Vivek Goyal <[email protected]>
Signature · 1 line
All rights reversed.
PATCH 11/23 io-controller: Introduce group idling
In reply to Vivek Goyalo It is not always that IO from a process or group is continuous. There are cases of dependent reads where next read is not issued till previous read has finished. For such cases, CFQ introduced the notion of slice_idle, where we idle on the queue for sometime hoping next request will come and that's how fairness is provided otherwise queue will be deleted immediately from the service tree and this process will not get the fair share.
o This patch introduces the similar concept at group level. Idle on the group for a period of "group_idle" which is tunable through sysfs interface. So if a group is empty and about to be deleted, we idle for the next request.
o This patch also introduces the notion of wait busy where we wait for one extra group_idle period even if queue has consumed its time slice. The reason being that group will loose its share upon removal from service tree as some other entity will be picked for dispatch and vtime jump will take place.
--- block/cfq-iosched.c | 5 +- block/elevator-fq.c | 207 +++++++++++++++++++++++++++++++++++++++++++++++++-- block/elevator-fq.h | 44 +++++++++++- 3 files changed, 247 insertions(+), 9 deletions(-)
block/cfq-iosched.c +4 -1
@@ -980,7 +980,7 @@ static int cfq_dispatch_requests(struct request_queue *q, int force)*/ if (elv_nr_busy_ioq (q->elevator) >1 && ((!cfq_cfqq_sync (cfqq) && cfqq->slice_dispatch >=cfq_prio_to_maxrq (cfqd, cfqq)) || -cfq_class_idle (cfqq))) { + (cfq_class_idle (cfqq) && !elv_iog_should_idle (cfqq->ioq)))) {cfq_slice_expired (cfqd); }
@@ -2121,6 +2121,9 @@ static struct elv_fs_entry cfq_attrs[] = {CFQ_ATTR (slice_idle),ELV_ATTR (slice_sync),ELV_ATTR (slice_async), +#ifdef CONFIG_GROUP_IOSCHED +ELV_ATTR (group_idle), +#endif __ATTR_NULL };
block/elevator-fq.c +200 -7
359 lines not on this page · raw message
block/elevator-fq.h +43 -1
74 lines not on this page · raw message
Signature · 1 line
1.6.0.6
Re: [PATCH 11/23] io-controller: Introduce group idling
Quoted 12 lines · Vivek Goyal
o It is not always that IO from a process or group is continuous. There are cases of dependent reads where next read is not issued till previous read has finished. For such cases, CFQ introduced the notion of
slice_idle, where we idle on the queue for sometime hoping next request will come and that's how fairness is provided otherwise queue will be deleted immediately from the service tree and this process will not get the fair share.o This patch introduces the similar concept at group level. Idle on the group for a period of "
group_idle" which is tunable through sysfs interface. So if a group is empty and about to be deleted, we idle for the next request.
Quoted 7 lines
o This patch also introduces the notion of wait busy where we wait for one extra
group_idleperiod even if queue has consumed its time slice. The reason being that group will loose its share upon removal from service tree as some other entity will be picked for dispatch and vtime jump will take place.Signed-off-by Vivek Goyal <[email protected]>
Signature · 1 line
All rights reversed.
A message that is not in the archive (sent off-list or not yet received): 1251495072-7780-12-git-send-email-vgoyal-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org
Rik van Riel
Acked-byRe: [PATCH 11/23] io-controller: Introduce group idling
Quoted 12 lines · Vivek Goyal
o It is not always that IO from a process or group is continuous. There are cases of dependent reads where next read is not issued till previous read has finished. For such cases, CFQ introduced the notion of
slice_idle, where we idle on the queue for sometime hoping next request will come and that's how fairness is provided otherwise queue will be deleted immediately from the service tree and this process will not get the fair share.o This patch introduces the similar concept at group level. Idle on the group for a period of "
group_idle" which is tunable through sysfs interface. So if a group is empty and about to be deleted, we idle for the next request.
Quoted 7 lines
o This patch also introduces the notion of wait busy where we wait for one extra
group_idleperiod even if queue has consumed its time slice. The reason being that group will loose its share upon removal from service tree as some other entity will be picked for dispatch and vtime jump will take place.Signed-off-by Vivek Goyal <[email protected]>
Signature · 1 line
All rights reversed.
PATCH 1/1 io-controller: Fix another bug that causing system hanging
In reply to a message not in the archiveVivek Goyal wrote:
...
Quoted 51 lines
* If io scheduler has functionality of keeping track of close cooperator, check
* with it if it has got a closely co-operating queue.(quoted diff) +26
-2@@ -2057,6 +2171,7 @@ void *elv_select_ioq(struct request_queue *q, int force) { struct elv_fq_data *efqd = q->elevator->efqd; struct io_queue *new_ioq = NULL, *ioq = elv_active_ioq(q->elevator); + struct io_group *iog; if (!elv_nr_busy_ioq(q->elevator)) return NULL;@@ -2064,6 +2179,8 @@ void *elv_select_ioq(struct request_queue *q, int force) if (ioq == NULL) goto new_queue; + iog = ioq_to_io_group(ioq); + /* * Force dispatch. Continue to dispatch from current queue as long * as it has requests.@@ -2075,11 +2192,47 @@ void *elv_select_ioq(struct request_queue *q, int force) goto expire; } + /* We are waiting for this group to become busy before it expires.*/ + if (elv_iog_wait_busy(iog)) { + ioq = NULL; + goto keep_queue; + } + /* * The active queue has run out of time, expire it and select new. */ - if (elv_ioq_slice_used(ioq) && !elv_ioq_must_dispatch(ioq)) - goto expire; + if ((elv_ioq_slice_used(ioq) || elv_ioq_class_idle(ioq)) + && !elv_ioq_must_dispatch(ioq)) { + /* + * Queue has used up its slice. Wait busy is not on otherwise + * we wouldn't have been here. If this group will be deleted + * after the queue expiry, then make sure we have onece + * done wait busy on the group in an attempt to make it + * backlogged. + * + * Following check helps in two conditions. + * - If there are requests dispatched from the queue and + * select_ioq() comes before a request completed from the + * queue and got a chance to arm any of the idle timers. + * + * - If at request completion time slice had not expired and + * we armed either a ioq timer or group timer but when + * select_ioq() hits, slice has expired and it will expire
Review void *elv_select_ioq(struct request_queue *q, int force) · line 2230
Quoted 10 lines
(quoted diff) +10
-0+ * the queue without doing busy wait on group. + * + * In similar situations cfq lets delte the queue even if + * idle timer is armed. That does not impact fairness in non + * hierarhical setup due to weighted slice lengths. But in + * hierarchical setup where group slice lengths are derived + * from queue and is not proportional to group's weight, it + * harms the fairness of the group. + */ + if (elv_iog_should_idle(ioq) && !elv_iog_wait_busy_done(iog)) {
Hi Vivek,
Here is another bug which will cause task hanging when accessing into a certain disk. For the moment, last ioq(corresponding CGroup has been removed) is optimized not to expire unitl another ioq get backlogged. Here just checking "iog_wait_busy_done" flag is not sufficient, because idle timer can be inactive at that moment. This will cause the ioq keeping service all the time and won't stop, causing the whole system hanging. This patch adds extra check for "iog_wait_busy" to make sure that the idle timer is pending, and this ioq will be expired after timer is up.
--- block/elevator-fq.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-)
block/elevator-fq.c +2 -1
@@ -3364,7 +3364,8 @@ void *elv_select_ioq(struct request_queue *q, int force)* harms the fairness of the group. */ slice_expired =1 ; -if (elv_iog_should_idle (ioq) && !elv_iog_wait_busy_done (iog)) { +if (elv_iog_should_idle (ioq) && !elv_iog_wait_busy_done (iog) && +elv_iog_wait_busy (iog)) { ioq =NULL ; goto keep_queue; }else
Signature · 1 line
1.5.4.rc3
PATCH 1/1 io-controller: Fix another bug that causing system hanging
In reply to Vivek GoyalVivek Goyal wrote:
...
Quoted 51 lines
* If io scheduler has functionality of keeping track of close cooperator, check
* with it if it has got a closely co-operating queue.(quoted diff) +26
-2@@ -2057,6 +2171,7 @@ void *elv_select_ioq(struct request_queue *q, int force) { struct elv_fq_data *efqd = q->elevator->efqd; struct io_queue *new_ioq = NULL, *ioq = elv_active_ioq(q->elevator); + struct io_group *iog; if (!elv_nr_busy_ioq(q->elevator)) return NULL;@@ -2064,6 +2179,8 @@ void *elv_select_ioq(struct request_queue *q, int force) if (ioq == NULL) goto new_queue; + iog = ioq_to_io_group(ioq); + /* * Force dispatch. Continue to dispatch from current queue as long * as it has requests.@@ -2075,11 +2192,47 @@ void *elv_select_ioq(struct request_queue *q, int force) goto expire; } + /* We are waiting for this group to become busy before it expires.*/ + if (elv_iog_wait_busy(iog)) { + ioq = NULL; + goto keep_queue; + } + /* * The active queue has run out of time, expire it and select new. */ - if (elv_ioq_slice_used(ioq) && !elv_ioq_must_dispatch(ioq)) - goto expire; + if ((elv_ioq_slice_used(ioq) || elv_ioq_class_idle(ioq)) + && !elv_ioq_must_dispatch(ioq)) { + /* + * Queue has used up its slice. Wait busy is not on otherwise + * we wouldn't have been here. If this group will be deleted + * after the queue expiry, then make sure we have onece + * done wait busy on the group in an attempt to make it + * backlogged. + * + * Following check helps in two conditions. + * - If there are requests dispatched from the queue and + * select_ioq() comes before a request completed from the + * queue and got a chance to arm any of the idle timers. + * + * - If at request completion time slice had not expired and + * we armed either a ioq timer or group timer but when + * select_ioq() hits, slice has expired and it will expire
Review void *elv_select_ioq(struct request_queue *q, int force) · line 2230
Quoted 10 lines
(quoted diff) +10
-0+ * the queue without doing busy wait on group. + * + * In similar situations cfq lets delte the queue even if + * idle timer is armed. That does not impact fairness in non + * hierarhical setup due to weighted slice lengths. But in + * hierarchical setup where group slice lengths are derived + * from queue and is not proportional to group's weight, it + * harms the fairness of the group. + */ + if (elv_iog_should_idle(ioq) && !elv_iog_wait_busy_done(iog)) {
Hi Vivek,
Here is another bug which will cause task hanging when accessing into a certain disk. For the moment, last ioq(corresponding CGroup has been removed) is optimized not to expire unitl another ioq get backlogged. Here just checking "iog_wait_busy_done" flag is not sufficient, because idle timer can be inactive at that moment. This will cause the ioq keeping service all the time and won't stop, causing the whole system hanging. This patch adds extra check for "iog_wait_busy" to make sure that the idle timer is pending, and this ioq will be expired after timer is up.
--- block/elevator-fq.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-)
block/elevator-fq.c +2 -1
@@ -3364,7 +3364,8 @@ void *elv_select_ioq(struct request_queue *q, int force)* harms the fairness of the group. */ slice_expired =1 ; -if (elv_iog_should_idle (ioq) && !elv_iog_wait_busy_done (iog)) { +if (elv_iog_should_idle (ioq) && !elv_iog_wait_busy_done (iog) && +elv_iog_wait_busy (iog)) { ioq =NULL ; goto keep_queue; }else
Signature · 1 line
1.5.4.rc3
Re: [PATCH] io-controller: Fix another bug that causing system hanging
Quoted 75 lines · Gui Jianfeng
Vivek Goyal wrote:
...Quoted 51 lines
* If io scheduler has functionality of keeping track of close cooperator, check
* with it if it has got a closely co-operating queue.(quoted diff) +26
-2@@ -2057,6 +2171,7 @@ void *elv_select_ioq(struct request_queue *q, int force) { struct elv_fq_data *efqd = q->elevator->efqd; struct io_queue *new_ioq = NULL, *ioq = elv_active_ioq(q->elevator); + struct io_group *iog; if (!elv_nr_busy_ioq(q->elevator)) return NULL;@@ -2064,6 +2179,8 @@ void *elv_select_ioq(struct request_queue *q, int force) if (ioq == NULL) goto new_queue; + iog = ioq_to_io_group(ioq); + /* * Force dispatch. Continue to dispatch from current queue as long * as it has requests.@@ -2075,11 +2192,47 @@ void *elv_select_ioq(struct request_queue *q, int force) goto expire; } + /* We are waiting for this group to become busy before it expires.*/ + if (elv_iog_wait_busy(iog)) { + ioq = NULL; + goto keep_queue; + } + /* * The active queue has run out of time, expire it and select new. */ - if (elv_ioq_slice_used(ioq) && !elv_ioq_must_dispatch(ioq)) - goto expire; + if ((elv_ioq_slice_used(ioq) || elv_ioq_class_idle(ioq)) + && !elv_ioq_must_dispatch(ioq)) { + /* + * Queue has used up its slice. Wait busy is not on otherwise + * we wouldn't have been here. If this group will be deleted + * after the queue expiry, then make sure we have onece + * done wait busy on the group in an attempt to make it + * backlogged. + * + * Following check helps in two conditions. + * - If there are requests dispatched from the queue and + * select_ioq() comes before a request completed from the + * queue and got a chance to arm any of the idle timers. + * + * - If at request completion time slice had not expired and + * we armed either a ioq timer or group timer but when + * select_ioq() hits, slice has expired and it will expireReview void *elv_select_ioq(struct request_queue *q, int force) · line 2230
Quoted 10 lines
(quoted diff) +10
-0+ * the queue without doing busy wait on group. + * + * In similar situations cfq lets delte the queue even if + * idle timer is armed. That does not impact fairness in non + * hierarhical setup due to weighted slice lengths. But in + * hierarchical setup where group slice lengths are derived + * from queue and is not proportional to group's weight, it + * harms the fairness of the group. + */ + if (elv_iog_should_idle(ioq) && !elv_iog_wait_busy_done(iog)) {Hi Vivek,
Here is another bug which will cause task hanging when accessing into a certain disk. For the moment, last ioq(corresponding CGroup has been removed) is optimized not to expire unitl another ioq get backlogged. Here just checking "
iog_wait_busy_done" flag is not sufficient, because idle timer can be inactive at that moment. This will cause the ioq keeping service all the time and won't stop, causing the whole system hanging. This patch adds extra check for "iog_wait_busy" to make sure that the idle timer is pending, and this ioq will be expired after timer is up.
Quoted 1 line
Signed-off-by Gui Jianfeng <[email protected]>
Good point. I think keeping the signle queue around with-in child group is getting complicated.
For the time being I will continue to expire the single ioq of child group even if other competing queues are not present. (Bring back the check of efqd->root_group->ioq).
Once rest of the things stablize, we can revisit this optimzation later that don't expire the single queue in child groups also.
Thanks
Vivek
Quoted 23 lines
--- block/elevator-fq.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-)block/elevator-fq.c +2
-1@@ -3364,7 +3364,8 @@ void *elv_select_ioq(struct request_queue *q, int force)* harms the fairness of the group. */ slice_expired =1 ; -if (elv_iog_should_idle (ioq) && !elv_iog_wait_busy_done (iog)) { +if (elv_iog_should_idle (ioq) && !elv_iog_wait_busy_done (iog) && +elv_iog_wait_busy (iog)) { ioq =NULL ; goto keep_queue; }else Signature · 1 line
1.5.4.rc3
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH] io-controller: Fix another bug that causing system hanging
Quoted 75 lines · Gui Jianfeng
Vivek Goyal wrote:
...Quoted 51 lines
* If io scheduler has functionality of keeping track of close cooperator, check
* with it if it has got a closely co-operating queue.(quoted diff) +26
-2@@ -2057,6 +2171,7 @@ void *elv_select_ioq(struct request_queue *q, int force) { struct elv_fq_data *efqd = q->elevator->efqd; struct io_queue *new_ioq = NULL, *ioq = elv_active_ioq(q->elevator); + struct io_group *iog; if (!elv_nr_busy_ioq(q->elevator)) return NULL;@@ -2064,6 +2179,8 @@ void *elv_select_ioq(struct request_queue *q, int force) if (ioq == NULL) goto new_queue; + iog = ioq_to_io_group(ioq); + /* * Force dispatch. Continue to dispatch from current queue as long * as it has requests.@@ -2075,11 +2192,47 @@ void *elv_select_ioq(struct request_queue *q, int force) goto expire; } + /* We are waiting for this group to become busy before it expires.*/ + if (elv_iog_wait_busy(iog)) { + ioq = NULL; + goto keep_queue; + } + /* * The active queue has run out of time, expire it and select new. */ - if (elv_ioq_slice_used(ioq) && !elv_ioq_must_dispatch(ioq)) - goto expire; + if ((elv_ioq_slice_used(ioq) || elv_ioq_class_idle(ioq)) + && !elv_ioq_must_dispatch(ioq)) { + /* + * Queue has used up its slice. Wait busy is not on otherwise + * we wouldn't have been here. If this group will be deleted + * after the queue expiry, then make sure we have onece + * done wait busy on the group in an attempt to make it + * backlogged. + * + * Following check helps in two conditions. + * - If there are requests dispatched from the queue and + * select_ioq() comes before a request completed from the + * queue and got a chance to arm any of the idle timers. + * + * - If at request completion time slice had not expired and + * we armed either a ioq timer or group timer but when + * select_ioq() hits, slice has expired and it will expireReview void *elv_select_ioq(struct request_queue *q, int force) · line 2230
Quoted 10 lines
(quoted diff) +10
-0+ * the queue without doing busy wait on group. + * + * In similar situations cfq lets delte the queue even if + * idle timer is armed. That does not impact fairness in non + * hierarhical setup due to weighted slice lengths. But in + * hierarchical setup where group slice lengths are derived + * from queue and is not proportional to group's weight, it + * harms the fairness of the group. + */ + if (elv_iog_should_idle(ioq) && !elv_iog_wait_busy_done(iog)) {Hi Vivek,
Here is another bug which will cause task hanging when accessing into a certain disk. For the moment, last ioq(corresponding CGroup has been removed) is optimized not to expire unitl another ioq get backlogged. Here just checking "
iog_wait_busy_done" flag is not sufficient, because idle timer can be inactive at that moment. This will cause the ioq keeping service all the time and won't stop, causing the whole system hanging. This patch adds extra check for "iog_wait_busy" to make sure that the idle timer is pending, and this ioq will be expired after timer is up.
Quoted 1 line
Signed-off-by Gui Jianfeng <[email protected]>
Good point. I think keeping the signle queue around with-in child group is getting complicated.
For the time being I will continue to expire the single ioq of child group even if other competing queues are not present. (Bring back the check of efqd->root_group->ioq).
Once rest of the things stablize, we can revisit this optimzation later that don't expire the single queue in child groups also.
Thanks
Vivek
Quoted 23 lines
--- block/elevator-fq.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-)block/elevator-fq.c +2
-1@@ -3364,7 +3364,8 @@ void *elv_select_ioq(struct request_queue *q, int force)* harms the fairness of the group. */ slice_expired =1 ; -if (elv_iog_should_idle (ioq) && !elv_iog_wait_busy_done (iog)) { +if (elv_iog_should_idle (ioq) && !elv_iog_wait_busy_done (iog) && +elv_iog_wait_busy (iog)) { ioq =NULL ; goto keep_queue; }else Signature · 1 line
1.5.4.rc3
PATCH 12/23 io-controller: Wait for requests to complete from last queue before new queue is scheduled
In reply to Vivek Goyalo Currently one can dispatch requests from multiple queues to the disk. This is true for hardware which supports queuing. So if a disk support queue depth of 31 it is possible that 20 requests are dispatched from queue 1 and then next queue is scheduled in which dispatches more requests.
o This multiple queue dispatch introduces issues for accurate accounting of disk time consumed by a particular queue. For example, if one async queue is scheduled in, it can dispatch 31 requests to the disk and then it will be expired and a new sync queue might get scheduled in. These 31 requests might take a long time to finish but this time is never accounted to the async queue which dispatched these requests.
o This patch introduces the functionality where we wait for all the requests to finish from previous queue before next queue is scheduled in. That way a queue is more accurately accounted for disk time it has consumed. Note this still does not take care of errors introduced by disk write caching.
o Because above behavior can result in reduced throughput, this behavior will be enabled only if user sets "fairness" tunable to 1.
o This patch helps in achieving more isolation between reads and buffered writes in different cgroups. buffered writes typically utilize full queue depth and then expire the queue. On the contarary, sequential reads typicaly driver queue depth of 1. So despite the fact that writes are using more disk time it is never accounted to write queue because we don't wait for requests to finish after dispatching these. This patch helps do more accurate accounting of disk time, especially for buffered writes hence providing better fairness hence better isolation between two cgroups running read and write workloads.
--- block/cfq-iosched.c | 1 + block/elevator-fq.c | 21 ++++++++++++++++++++- block/elevator-fq.h | 10 +++++++++- 3 files changed, 30 insertions(+), 2 deletions(-)
block/cfq-iosched.c +1 -0
@@ -2123,6 +2123,7 @@ static struct elv_fs_entry cfq_attrs[] = { ELV_ATTR(slice_async),#ifdef CONFIG_GROUP_IOSCHED ELV_ATTR (group_idle), +ELV_ATTR (fairness), #endif __ATTR_NULL };
block/elevator-fq.c +20 -1
@@ -681,6 +681,8 @@ SHOW_FUNCTION(elv_slice_sync_show, efqd->elv_slice[1], 1);EXPORT_SYMBOL (elv_slice_sync_show);SHOW_FUNCTION (elv_slice_async_show, efqd->elv_slice[0 ],1 );EXPORT_SYMBOL (elv_slice_async_show); +SHOW_FUNCTION (elv_fairness_show, efqd->fairness,0 ); +EXPORT_SYMBOL (elv_fairness_show);#undef SHOW_FUNCTION#define STORE_FUNCTION (__FUNC, __PTR,MIN ,MAX , __CONV) \
@@ -705,6 +707,8 @@ STORE_FUNCTION(elv_slice_sync_store, &efqd->elv_slice[1], 1, UINT_MAX, 1);EXPORT_SYMBOL (elv_slice_sync_store);STORE_FUNCTION (elv_slice_async_store, &efqd->elv_slice[0 ],1 ,UINT_MAX ,1 );EXPORT_SYMBOL (elv_slice_async_store); +STORE_FUNCTION (elv_fairness_store, &efqd->fairness,0 ,1 ,0 ); +EXPORT_SYMBOL (elv_fairness_store);#undef STORE_FUNCTIONvoid elv_schedule_dispatch (struct request_queue *q)
@@ -2271,6 +2275,17 @@ void *elv_select_ioq(struct request_queue *q, int force) } expire: +if (efqd->fairness && !force && ioq && ioq->dispatched) { +/* +* If there are request dispatched from this queue, don't +* dispatch requests from new queue till all the requests from +* this queue have completed. +*/ +elv_log_ioq (efqd, ioq,"select: wait for requests to finish" +" disp=%lu" , ioq->dispatched); + ioq =NULL ; + goto keep_queue; + }elv_slice_expired (q); new_queue: ioq =elv_set_active_ioq (q, new_ioq);
@@ -2386,6 +2401,10 @@ void elv_ioq_completed_request(struct request_queue *q, struct request *rq) goto done; } +/* Wait for requests to finish from this queue */ +if (efqd->fairness &&elv_ioq_nr_dispatched (ioq)) + goto done; +/* Expire the queue */ elv_slice_expired (q); goto done;
@@ -2396,7 +2415,7 @@ void elv_ioq_completed_request(struct request_queue *q, struct request *rq)* If this is the last queue in the group and we did not * decide to idle on queue, idle on group. */ -if (elv_iog_should_idle (ioq) && !ioq->dispatched +if (elv_iog_should_idle (ioq) && !elv_ioq_nr_dispatched (ioq) && !timer_pending (&efqd->idle_slice_timer)) {/* * If queue has used up its slice, wait for the
block/elevator-fq.h +9 -1
@@ -180,6 +180,12 @@ struct elv_fq_data {/* Fallback dummy ioq for extreme OOM conditions */ struct io_queue oom_ioq; + +/* +* If set to 1, waits for all request completions from current +* queue before new queue is scheduled in +*/ +unsigned int fairness; };/* Logging facilities. */
@@ -437,7 +443,9 @@ extern ssize_t elv_slice_sync_store(struct elevator_queue *q, const char *name,extern ssize_t elv_slice_async_show (struct elevator_queue *q,char *name);extern ssize_t elv_slice_async_store (struct elevator_queue *q,const char *name,size_t count); - +extern ssize_t elv_fairness_show (struct elevator_queue *q,char *name); +extern ssize_t elv_fairness_store (struct elevator_queue *q,const char *name, +size_t count);/* Functions used by elevator.c */ extern struct elv_fq_data *elv_alloc_fq_data (struct request_queue *q,struct elevator_queue *e);
Signature · 1 line
1.6.0.6
Re: [PATCH 12/23] io-controller: Wait for requests to complete from last queue before new queue is scheduled
Quoted 5 lines · Vivek Goyal
o Because above behavior can result in reduced throughput, this behavior will be enabled only if user sets "fairness" tunable to 1.
o This patch helps in achieving more isolation between reads and buffered writes in different cgroups.
Could be a nice help against SATA NCQ request starvation.
Quoted 1 line
Signed-off-by Vivek Goyal <[email protected]>
Signature · 1 line
All rights reversed.
A message that is not in the archive (sent off-list or not yet received): 1251495072-7780-13-git-send-email-vgoyal-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org
Rik van Riel
Acked-byRe: [PATCH 12/23] io-controller: Wait for requests to complete from last queue before new queue is scheduled
Quoted 5 lines · Vivek Goyal
o Because above behavior can result in reduced throughput, this behavior will be enabled only if user sets "fairness" tunable to 1.
o This patch helps in achieving more isolation between reads and buffered writes in different cgroups.
Could be a nice help against SATA NCQ request starvation.
Quoted 1 line
Signed-off-by Vivek Goyal <[email protected]>
Signature · 1 line
All rights reversed.
PATCH 13/23 io-controller: Separate out queue and data
In reply to Vivek Goyalo So far noop, deadline and AS had one common structure called *_data which contained both the queue information where requests are queued and also common data used for scheduling. This patch breaks down this common structure in two parts, *_queue and *_data. This is along the lines of cfq where all the reuquests are queued in queue and common data and tunables are part of data.
o It does not change the functionality but this re-organization helps once noop, deadline and AS are changed to use hierarchical fair queuing.
o looks like queue_empty function is not required and we can check for q->nr_sorted in elevator layer to see if ioscheduler queues are empty or not.
--- block/as-iosched.c | 208 ++++++++++++++++++++++++++-------------------- block/deadline-iosched.c | 117 ++++++++++++++++---------- block/elevator.c | 111 +++++++++++++++++++++---- block/noop-iosched.c | 59 ++++++------- include/linux/elevator.h | 9 ++- 5 files changed, 320 insertions(+), 184 deletions(-)
block/as-iosched.c +118 -90
@@ -76,13 +76,7 @@ enum anticipation_status {* or timed out */ }; -struct as_data { -/* -* run time data -*/ - -struct request_queue *q;/* the "owner" queue */ - +struct as_queue {/* * requests (as_rq s) are present on both sort_list and fifo_list */
@@ -90,6 +84,14 @@ struct as_data {struct list_head fifo_list[2 ];struct request *next_rq[2 ];/* next in sort order */ +unsigned long last_check_fifo[2 ]; +int write_batch_count;/* max # of reqs in a write batch */ +int current_write_count;/* how many requests left this batch */ +int write_batch_idled;/* has the write batch gone idle? */ +}; + +struct as_data { +struct request_queue *q;/* the "owner" queue */ sector_t last_sector[2 ];/* last SYNC & ASYNC sectors */ unsigned long exit_prob;/* probability a task will exit while
@@ -103,21 +105,17 @@ struct as_data {sector_t new_seek_mean;unsigned long current_batch_expires; -unsigned long last_check_fifo[2 ];int changed_batch;/* 1: waiting for old batch to end */ int new_batch;/* 1: waiting on first read complete */ -int batch_data_dir;/* current batch SYNC / ASYNC */ -int write_batch_count;/* max # of reqs in a write batch */ -int current_write_count;/* how many requests left this batch */ -int write_batch_idled;/* has the write batch gone idle? */ enum anticipation_status antic_status;unsigned long antic_start;/* jiffies: when it started */ struct timer_list antic_timer;/* anticipatory scheduling timer */ -struct work_struct antic_work;/* Deferred unplugging */ +struct work_struct antic_work;/* Deferred unplugging */ struct io_context *io_context;/* Identify the expected process */ int ioc_finished;/* IO associated with io_context is finished */ int nr_dispatched; +int batch_data_dir;/* current batch SYNC / ASYNC */ /* * settings that change how the i/o scheduler behaves
@@ -258,13 +256,14 @@ static void as_put_io_context(struct request *rq)/* * rb tree support functions */ -#define RQ_RB_ROOT (ad, rq) (&(ad)->sort_list[rq_is_sync((rq))]) +#define RQ_RB_ROOT (asq, rq) (&(asq)->sort_list[rq_is_sync((rq))])static void as_add_rq_rb (struct as_data *ad,struct request *rq) {struct request *alias; +struct as_queue *asq =elv_get_sched_queue (ad->q, rq); -while ((unlikely (alias =elv_rb_add (RQ_RB_ROOT (ad, rq), rq)))) { +while ((unlikely (alias =elv_rb_add (RQ_RB_ROOT (asq, rq), rq)))) {as_move_to_dispatch (ad, alias);as_antic_stop (ad); }
@@ -272,7 +271,9 @@ static void as_add_rq_rb(struct as_data *ad, struct request *rq)static inline void as_del_rq_rb (struct as_data *ad,struct request *rq) { -elv_rb_del (RQ_RB_ROOT (ad, rq), rq); +struct as_queue *asq =elv_get_sched_queue (ad->q, rq); + +elv_rb_del (RQ_RB_ROOT (asq, rq), rq); }/*
@@ -366,7 +367,7 @@ as_choose_req(struct as_data *ad, struct request *rq1, struct request *rq2)* what request to process next. Anticipation works on top of this. */ static struct request * -as_find_next_rq (struct as_data *ad,struct request *last) +as_find_next_rq (struct as_data *ad,struct as_queue *asq,struct request *last) {struct rb_node *rbnext =rb_next (&last->rb_node);struct rb_node *rbprev =rb_prev (&last->rb_node);
@@ -382,7 +383,7 @@ as_find_next_rq(struct as_data *ad, struct request *last)else {const int data_dir =rq_is_sync (last); - rbnext =rb_first (&ad->sort_list[data_dir]); + rbnext =rb_first (&asq->sort_list[data_dir]);if (rbnext && rbnext != &last->rb_node) next =rb_entry_rq (rbnext); }
@@ -789,9 +790,10 @@ static int as_can_anticipate(struct as_data *ad, struct request *rq)static void as_update_rq (struct as_data *ad,struct request *rq) {const int data_dir =rq_is_sync (rq); +struct as_queue *asq =elv_get_sched_queue (ad->q, rq);/* keep the next_rq cache up to date */ - ad->next_rq[data_dir] =as_choose_req (ad, rq, ad->next_rq[data_dir]); + asq->next_rq[data_dir] =as_choose_req (ad, rq, asq->next_rq[data_dir]);/* * have we been anticipating this request?
@@ -812,25 +814,26 @@ static void update_write_batch(struct as_data *ad) {unsigned long batch = ad->batch_expire[BLK_RW_ASYNC ];long write_time; +struct as_queue *asq =elv_get_sched_queue (ad->q,NULL ); write_time = (jiffies - ad->current_batch_expires) + batch;if (write_time <0 ) write_time =0 ; -if (write_time > batch && !ad->write_batch_idled) { +if (write_time > batch && !asq->write_batch_idled) {if (write_time > batch *3 ) - ad->write_batch_count /=2 ; + asq->write_batch_count /=2 ;else - ad->write_batch_count--; - }else if (write_time < batch && ad->current_write_count ==0 ) { + asq->write_batch_count--; + }else if (write_time < batch && asq->current_write_count ==0 ) {if (batch > write_time *3 ) - ad->write_batch_count *=2 ; + asq->write_batch_count *=2 ;else - ad->write_batch_count++; + asq->write_batch_count++; } -if (ad->write_batch_count <1 ) - ad->write_batch_count =1 ; +if (asq->write_batch_count <1 ) + asq->write_batch_count =1 ; }/*
@@ -901,6 +904,7 @@ static void as_remove_queued_request(struct request_queue *q,const int data_dir =rq_is_sync (rq);struct as_data *ad = q->elevator->elevator_data;struct io_context *ioc; +struct as_queue *asq =elv_get_sched_queue (q, rq);WARN_ON (RQ_STATE (rq) !=AS_RQ_QUEUED );
@@ -914,8 +918,8 @@ static void as_remove_queued_request(struct request_queue *q,* Update the "next_rq" cache if we are about to remove its * entry */ -if (ad->next_rq[data_dir] == rq) - ad->next_rq[data_dir] =as_find_next_rq (ad, rq); +if (asq->next_rq[data_dir] == rq) + asq->next_rq[data_dir] =as_find_next_rq (ad, asq, rq);rq_fifo_clear (rq);as_del_rq_rb (ad, rq);
@@ -929,23 +933,23 @@ static void as_remove_queued_request(struct request_queue *q,* * See as_antic_expired comment. */ -static int as_fifo_expired (struct as_data *ad,int adir) +static int as_fifo_expired (struct as_data *ad,struct as_queue *asq,int adir) {struct request *rq;long delta_jif; - delta_jif = jiffies - ad->last_check_fifo[adir]; + delta_jif = jiffies - asq->last_check_fifo[adir];if (unlikely (delta_jif <0 )) delta_jif = -delta_jif;if (delta_jif < ad->fifo_expire[adir])return 0 ; - ad->last_check_fifo[adir] = jiffies; + asq->last_check_fifo[adir] = jiffies; -if (list_empty (&ad->fifo_list[adir])) +if (list_empty (&asq->fifo_list[adir]))return 0 ; - rq =rq_entry_fifo (ad->fifo_list[adir].next); + rq =rq_entry_fifo (asq->fifo_list[adir].next);return time_after (jiffies,rq_fifo_time (rq)); }
@@ -954,7 +958,7 @@ static int as_fifo_expired(struct as_data *ad, int adir)* as_batch_expired returns true if the current batch has expired. A batch * is a set of reads or a set of writes. */ -static inline int as_batch_expired (struct as_data *ad) +static inline int as_batch_expired (struct as_data *ad,struct as_queue *asq) {if (ad->changed_batch || ad->new_batch)return 0 ;
@@ -964,7 +968,7 @@ static inline int as_batch_expired(struct as_data *ad)return time_after (jiffies, ad->current_batch_expires);return time_after (jiffies, ad->current_batch_expires) - || ad->current_write_count ==0 ; + || asq->current_write_count ==0 ; }/*
@@ -973,6 +977,7 @@ static inline int as_batch_expired(struct as_data *ad)static void as_move_to_dispatch (struct as_data *ad,struct request *rq) {const int data_dir =rq_is_sync (rq); +struct as_queue *asq =elv_get_sched_queue (ad->q, rq);BUG_ON (RB_EMPTY_NODE (&rq->rb_node));
@@ -995,12 +1000,12 @@ static void as_move_to_dispatch(struct as_data *ad, struct request *rq) ad->io_context =NULL ; } -if (ad->current_write_count !=0 ) - ad->current_write_count--; +if (asq->current_write_count !=0 ) + asq->current_write_count--; } ad->ioc_finished =0 ; - ad->next_rq[data_dir] =as_find_next_rq (ad, rq); + asq->next_rq[data_dir] =as_find_next_rq (ad, asq, rq);/* * take it off the sort and fifo list, add to dispatch queue
@@ -1024,9 +1029,16 @@ static void as_move_to_dispatch(struct as_data *ad, struct request *rq)static int as_dispatch_request (struct request_queue *q,int force) {struct as_data *ad = q->elevator->elevator_data; -const int reads = !list_empty (&ad->fifo_list[BLK_RW_SYNC ]); -const int writes = !list_empty (&ad->fifo_list[BLK_RW_ASYNC ]);struct request *rq; +struct as_queue *asq =elv_select_sched_queue (q, force); +int reads, writes; + +if (!asq) +return 0 ; + + reads = !list_empty (&asq->fifo_list[BLK_RW_SYNC ]); + writes = !list_empty (&asq->fifo_list[BLK_RW_ASYNC ]); +if (unlikely (force)) {/*
@@ -1042,25 +1054,25 @@ static int as_dispatch_request(struct request_queue *q, int force) ad->changed_batch =0 ; ad->new_batch =0 ; -while (ad->next_rq[BLK_RW_SYNC ]) { -as_move_to_dispatch (ad, ad->next_rq[BLK_RW_SYNC ]); +while (asq->next_rq[BLK_RW_SYNC ]) { +as_move_to_dispatch (ad, asq->next_rq[BLK_RW_SYNC ]); dispatched++; } - ad->last_check_fifo[BLK_RW_SYNC ] = jiffies; + asq->last_check_fifo[BLK_RW_SYNC ] = jiffies; -while (ad->next_rq[BLK_RW_ASYNC ]) { -as_move_to_dispatch (ad, ad->next_rq[BLK_RW_ASYNC ]); +while (asq->next_rq[BLK_RW_ASYNC ]) { +as_move_to_dispatch (ad, asq->next_rq[BLK_RW_ASYNC ]); dispatched++; } - ad->last_check_fifo[BLK_RW_ASYNC ] = jiffies; + asq->last_check_fifo[BLK_RW_ASYNC ] = jiffies;return dispatched; }/* Signal that the write batch was uncontended, so we can't time it */ if (ad->batch_data_dir ==BLK_RW_ASYNC && !reads) { -if (ad->current_write_count ==0 || !writes) - ad->write_batch_idled =1 ; +if (asq->current_write_count ==0 || !writes) + asq->write_batch_idled =1 ; } if (!(reads || writes)
@@ -1069,14 +1081,14 @@ static int as_dispatch_request(struct request_queue *q, int force) || ad->changed_batch)return 0 ; -if (!(reads && writes &&as_batch_expired (ad))) { +if (!(reads && writes &&as_batch_expired (ad, asq))) {/* * batch is still running or no reads or no writes */ - rq = ad->next_rq[ad->batch_data_dir]; + rq = asq->next_rq[ad->batch_data_dir];if (ad->batch_data_dir ==BLK_RW_SYNC && ad->antic_expire) { -if (as_fifo_expired (ad,BLK_RW_SYNC )) +if (as_fifo_expired (ad, asq,BLK_RW_SYNC )) goto fifo_expired;if (as_can_anticipate (ad, rq)) {
@@ -1100,7 +1112,7 @@ static int as_dispatch_request(struct request_queue *q, int force)*/ if (reads) { -BUG_ON (RB_EMPTY_ROOT (&ad->sort_list[BLK_RW_SYNC ])); +BUG_ON (RB_EMPTY_ROOT (&asq->sort_list[BLK_RW_SYNC ]));if (writes && ad->batch_data_dir ==BLK_RW_SYNC )/*
@@ -1113,8 +1125,8 @@ static int as_dispatch_request(struct request_queue *q, int force) ad->changed_batch =1 ; } ad->batch_data_dir =BLK_RW_SYNC ; - rq =rq_entry_fifo (ad->fifo_list[BLK_RW_SYNC ].next); - ad->last_check_fifo[ad->batch_data_dir] = jiffies; + rq =rq_entry_fifo (asq->fifo_list[BLK_RW_SYNC ].next); + asq->last_check_fifo[ad->batch_data_dir] = jiffies; goto dispatch_request; }
@@ -1124,7 +1136,7 @@ static int as_dispatch_request(struct request_queue *q, int force)if (writes) { dispatch_writes: -BUG_ON (RB_EMPTY_ROOT (&ad->sort_list[BLK_RW_ASYNC ])); +BUG_ON (RB_EMPTY_ROOT (&asq->sort_list[BLK_RW_ASYNC ]));if (ad->batch_data_dir ==BLK_RW_SYNC ) { ad->changed_batch =1 ;
@@ -1137,10 +1149,10 @@ dispatch_writes: ad->new_batch =0 ; } ad->batch_data_dir =BLK_RW_ASYNC ; - ad->current_write_count = ad->write_batch_count; - ad->write_batch_idled =0 ; - rq =rq_entry_fifo (ad->fifo_list[BLK_RW_ASYNC ].next); - ad->last_check_fifo[BLK_RW_ASYNC ] = jiffies; + asq->current_write_count = asq->write_batch_count; + asq->write_batch_idled =0 ; + rq =rq_entry_fifo (asq->fifo_list[BLK_RW_ASYNC ].next); + asq->last_check_fifo[BLK_RW_ASYNC ] = jiffies; goto dispatch_request; }
@@ -1152,9 +1164,9 @@ dispatch_request:* If a request has expired, service it. */ -if (as_fifo_expired (ad, ad->batch_data_dir)) { +if (as_fifo_expired (ad, asq, ad->batch_data_dir)) { fifo_expired: - rq =rq_entry_fifo (ad->fifo_list[ad->batch_data_dir].next); + rq =rq_entry_fifo (asq->fifo_list[ad->batch_data_dir].next); }if (ad->changed_batch) {
@@ -1187,6 +1199,7 @@ static void as_add_request(struct request_queue *q, struct request *rq) {struct as_data *ad = q->elevator->elevator_data;int data_dir; +struct as_queue *asq =elv_get_sched_queue (q, rq);RQ_SET_STATE (rq,AS_RQ_NEW );
@@ -1205,7 +1218,7 @@ static void as_add_request(struct request_queue *q, struct request *rq)* set expire time and add to fifo list */ rq_set_fifo_time (rq, jiffies + ad->fifo_expire[data_dir]); -list_add_tail (&rq->queuelist, &ad->fifo_list[data_dir]); +list_add_tail (&rq->queuelist, &asq->fifo_list[data_dir]);as_update_rq (ad, rq);/* keep state machine up to date */ RQ_SET_STATE (rq,AS_RQ_QUEUED );
@@ -1227,31 +1240,20 @@ static void as_deactivate_request(struct request_queue *q, struct request *rq)atomic_inc (&RQ_IOC (rq)->aic->nr_dispatched); } -/* -* as_queue_empty tells us if there are requests left in the device. It may -* not be the case that a driver can get the next request even if the queue -* is not empty - it is used in the block layer to check for plugging and -* merging opportunities -*/ -static int as_queue_empty (struct request_queue *q) -{ -struct as_data *ad = q->elevator->elevator_data; - -return list_empty (&ad->fifo_list[BLK_RW_ASYNC ]) - &&list_empty (&ad->fifo_list[BLK_RW_SYNC ]); -} -static int as_merge (struct request_queue *q,struct request **req,struct bio *bio) { -struct as_data *ad = q->elevator->elevator_data;sector_t rb_key = bio->bi_sector +bio_sectors (bio);struct request *__rq; +struct as_queue *asq =elv_get_sched_queue_current (q); + +if (!asq) +return ELEVATOR_NO_MERGE ;/* * check for front merge */ - __rq =elv_rb_find (&ad->sort_list[bio_data_dir (bio)], rb_key); + __rq =elv_rb_find (&asq->sort_list[bio_data_dir (bio)], rb_key);if (__rq &&elv_rq_merge_ok (__rq, bio)) { *req = __rq;return ELEVATOR_FRONT_MERGE ;
@@ -1334,6 +1336,41 @@ static int as_may_queue(struct request_queue *q, int rw)return ret; } +/* Called with queue lock held */ +static void *as_alloc_as_queue (struct request_queue *q, +struct elevator_queue *eq,gfp_t gfp_mask) +{ +struct as_queue *asq; +struct as_data *ad = eq->elevator_data; + + asq =kmalloc_node (sizeof (*asq), gfp_mask | __GFP_ZERO, q->node); +if (asq ==NULL ) + goto out; + +INIT_LIST_HEAD (&asq->fifo_list[BLK_RW_SYNC ]); +INIT_LIST_HEAD (&asq->fifo_list[BLK_RW_ASYNC ]); + asq->sort_list[BLK_RW_SYNC ] =RB_ROOT ; + asq->sort_list[BLK_RW_ASYNC ] =RB_ROOT ; +if (ad) + asq->write_batch_count = ad->batch_expire[BLK_RW_ASYNC ] /10 ; +else + asq->write_batch_count = default_write_batch_expire /10 ; + +if (asq->write_batch_count <2 ) + asq->write_batch_count =2 ; +out: +return asq; +} + +static void as_free_as_queue (struct elevator_queue *e,void *sched_queue) +{ +struct as_queue *asq = sched_queue; + +BUG_ON (!list_empty (&asq->fifo_list[BLK_RW_SYNC ])); +BUG_ON (!list_empty (&asq->fifo_list[BLK_RW_ASYNC ])); +kfree (asq); +} +static void as_exit_queue (struct elevator_queue *e) {struct as_data *ad = e->elevator_data;
@@ -1341,9 +1378,6 @@ static void as_exit_queue(struct elevator_queue *e)del_timer_sync (&ad->antic_timer);cancel_work_sync (&ad->antic_work); -BUG_ON (!list_empty (&ad->fifo_list[BLK_RW_SYNC ])); -BUG_ON (!list_empty (&ad->fifo_list[BLK_RW_ASYNC ])); -put_io_context (ad->io_context);kfree (ad); }
@@ -1367,10 +1401,6 @@ static void *as_init_queue(struct request_queue *q, struct elevator_queue *eq)init_timer (&ad->antic_timer);INIT_WORK (&ad->antic_work, as_work_handler); -INIT_LIST_HEAD (&ad->fifo_list[BLK_RW_SYNC ]); -INIT_LIST_HEAD (&ad->fifo_list[BLK_RW_ASYNC ]); - ad->sort_list[BLK_RW_SYNC ] =RB_ROOT ; - ad->sort_list[BLK_RW_ASYNC ] =RB_ROOT ; ad->fifo_expire[BLK_RW_SYNC ] = default_read_expire; ad->fifo_expire[BLK_RW_ASYNC ] = default_write_expire; ad->antic_expire = default_antic_expire;
@@ -1378,9 +1408,6 @@ static void *as_init_queue(struct request_queue *q, struct elevator_queue *eq) ad->batch_expire[BLK_RW_ASYNC ] = default_write_batch_expire; ad->current_batch_expires = jiffies + ad->batch_expire[BLK_RW_SYNC ]; - ad->write_batch_count = ad->batch_expire[BLK_RW_ASYNC ] /10 ; -if (ad->write_batch_count <2 ) - ad->write_batch_count =2 ;return ad; }
@@ -1478,7 +1505,6 @@ static struct elevator_type iosched_as = { .elevator_add_req_fn = as_add_request, .elevator_activate_req_fn = as_activate_request, .elevator_deactivate_req_fn = as_deactivate_request, - .elevator_queue_empty_fn = as_queue_empty, .elevator_completed_req_fn = as_completed_request, .elevator_former_req_fn = elv_rb_former_request, .elevator_latter_req_fn = elv_rb_latter_request,
@@ -1486,6 +1512,8 @@ static struct elevator_type iosched_as = { .elevator_init_fn = as_init_queue, .elevator_exit_fn = as_exit_queue, .trim = as_trim, + .elevator_alloc_sched_queue_fn = as_alloc_as_queue, + .elevator_free_sched_queue_fn = as_free_as_queue, }, .elevator_attrs = as_attrs,
block/deadline-iosched.c +71 -46
266 lines not on this page · raw message
block/elevator.c +96 -15
179 lines not on this page · raw message
block/noop-iosched.c +28 -31
111 lines not on this page · raw message
include/linux/elevator.h +7 -2
@@ -30,8 +30,10 @@ typedef void (elevator_deactivate_req_fn) (struct request_queue *, struct requestypedef void *(elevator_init_fn ) (struct request_queue *,struct elevator_queue *);typedef void (elevator_exit_fn ) (struct elevator_queue *); -#ifdef CONFIG_ELV_FAIR_QUEUING +typedef void * (elevator_alloc_sched_queue_fn ) (struct request_queue *q, +struct elevator_queue *eq,gfp_t );typedef void (elevator_free_sched_queue_fn ) (struct elevator_queue *,void *); +#ifdef CONFIG_ELV_FAIR_QUEUING typedef void (elevator_active_ioq_set_fn ) (struct request_queue *,void *,int );typedef void (elevator_active_ioq_reset_fn ) (struct request_queue *,void *);typedef void (elevator_arm_slice_timer_fn ) (struct request_queue *,void *);
@@ -68,8 +70,9 @@ struct elevator_opselevator_exit_fn *elevator_exit_fn;void (*trim)(struct io_context *); -#ifdef CONFIG_ELV_FAIR_QUEUING +elevator_alloc_sched_queue_fn *elevator_alloc_sched_queue_fn;elevator_free_sched_queue_fn *elevator_free_sched_queue_fn; +#ifdef CONFIG_ELV_FAIR_QUEUING elevator_active_ioq_set_fn *elevator_active_ioq_set_fn;elevator_active_ioq_reset_fn *elevator_active_ioq_reset_fn;
@@ -109,6 +112,7 @@ struct elevator_queue {struct elevator_ops *ops;void *elevator_data; +void *sched_queue;struct kobject kobj;struct elevator_type *elevator_type;struct mutex sysfs_lock;
@@ -255,5 +259,6 @@ static inline int elv_iosched_fair_queuing_enabled(struct elevator_queue *e)#endif /* ELV_IOSCHED_FAIR_QUEUING */ extern void *elv_get_sched_queue (struct request_queue *q,struct request *rq);extern void *elv_select_sched_queue (struct request_queue *q,int force); +extern void *elv_get_sched_queue_current (struct request_queue *q);#endif /* CONFIG_BLOCK */ #endif
Signature · 1 line
1.6.0.6
Re: [PATCH 13/23] io-controller: Separate out queue and data
Quoted 10 lines · Vivek Goyal
o So far noop, deadline and AS had one common structure called
*_datawhich contained both the queue information where requests are queued and also common data used for scheduling. This patch breaks down this common structure in two parts,*_queueand*_data. This is along the lines of cfq where all the reuquests are queued in queue and common data and tunables are part of data.o It does not change the functionality but this re-organization helps once noop, deadline and AS are changed to use hierarchical fair queuing.
Quoted 7 lines
o looks like
queue_emptyfunction is not required and we can check forq->nr_sortedin elevator layer to see if ioscheduler queues are empty or not.Signed-off-by Nauman Rafique <[email protected]>Signed-off-by Gui Jianfeng <[email protected]>Signed-off-by Vivek Goyal <[email protected]>
A message that is not in the archive (sent off-list or not yet received): 1251495072-7780-14-git-send-email-vgoyal-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org
Rik van Riel
Acked-byRe: [PATCH 13/23] io-controller: Separate out queue and data
Quoted 10 lines · Vivek Goyal
o So far noop, deadline and AS had one common structure called
*_datawhich contained both the queue information where requests are queued and also common data used for scheduling. This patch breaks down this common structure in two parts,*_queueand*_data. This is along the lines of cfq where all the reuquests are queued in queue and common data and tunables are part of data.o It does not change the functionality but this re-organization helps once noop, deadline and AS are changed to use hierarchical fair queuing.
Quoted 7 lines
o looks like
queue_emptyfunction is not required and we can check forq->nr_sortedin elevator layer to see if ioscheduler queues are empty or not.Signed-off-by Nauman Rafique <[email protected]>Signed-off-by Gui Jianfeng <[email protected]>Signed-off-by Vivek Goyal <[email protected]>
PATCH 14/23 io-conroller: Prepare elevator layer for single queue schedulers
In reply to Vivek GoyalElevator layer now has support for hierarchical fair queuing. cfq has been migrated to make use of it and now it is time to do groundwork for noop, deadline and AS.
noop deadline and AS don't maintain separate queues for different processes. There is only one single queue. Effectively one can think that in hierarchical setup, there will be one queue per cgroup where requests from all the processes in the cgroup will be queued.
Generally io scheduler takes care of creating queues. Because there is only one queue here, we have modified common layer to take care of queue creation and some other functionality. This special casing helps in keeping the changes to noop, deadline and AS to the minimum.
--- block/as-iosched.c | 2 +- block/deadline-iosched.c | 2 +- block/elevator-fq.c | 211 +++++++++++++++++++++++++++++++++++++++++++++- block/elevator-fq.h | 36 ++++++++ block/elevator.c | 37 ++++++++- block/noop-iosched.c | 2 +- include/linux/elevator.h | 18 ++++- 7 files changed, 300 insertions(+), 8 deletions(-)
block/as-iosched.c +1 -1
@@ -1338,7 +1338,7 @@ static int as_may_queue(struct request_queue *q, int rw)/* Called with queue lock held */ static void *as_alloc_as_queue (struct request_queue *q, -struct elevator_queue *eq,gfp_t gfp_mask) +struct elevator_queue *eq,gfp_t gfp_mask,struct io_queue *ioq) {struct as_queue *asq;struct as_data *ad = eq->elevator_data;
block/deadline-iosched.c +1 -1
@@ -341,7 +341,7 @@ dispatch_request: }static void *deadline_alloc_deadline_queue (struct request_queue *q, -struct elevator_queue *eq,gfp_t gfp_mask) +struct elevator_queue *eq,gfp_t gfp_mask,struct io_queue *ioq) {struct deadline_queue *dq;
block/elevator-fq.c +210 -1
253 lines not on this page · raw message
block/elevator-fq.h +36 -0
@@ -125,6 +125,9 @@ struct io_group {/* Store cgroup path */ char path[128 ];#endif + +/* Single ioq per group, used for noop, deadline, anticipatory */ +struct io_queue *ioq; };struct io_cgroup {
@@ -418,6 +421,11 @@ static inline void elv_get_iog(struct io_group *iog) atomic_inc(&iog->ref); } +extern int elv_set_request_ioq (struct request_queue *q,struct request *rq, +gfp_t gfp_mask); +extern void elv_reset_request_ioq (struct request_queue *q,struct request *rq); +extern struct io_queue *elv_lookup_ioq_current (struct request_queue *q); +#else /* !GROUP_IOSCHED */ static inline int elv_io_group_allow_merge (struct request *rq,struct bio *bio)
@@ -435,6 +443,20 @@ elv_io_get_io_group(struct request_queue *q, int create) return q->elevator->efqd->root_group; } +static inline int +elv_set_request_ioq (struct request_queue *q,struct request *rq,gfp_t gfp_mask) +{ +return 0 ; +} + +static inline void +elv_reset_request_ioq (struct request_queue *q,struct request *rq) { } + +static inline struct io_queue *elv_lookup_ioq_current (struct request_queue *q) +{ +return NULL ; +} +#endif /* GROUP_IOSCHED */ extern ssize_t elv_slice_sync_show (struct elevator_queue *q,char *name);
@@ -529,6 +551,20 @@ static inline int elv_io_group_allow_merge(struct request *rq, struct bio *bio) {return 1 ; } +static inline int +elv_set_request_ioq (struct request_queue *q,struct request *rq,gfp_t gfp_mask) +{ +return 0 ; +} + +static inline void +elv_reset_request_ioq (struct request_queue *q,struct request *rq) { } + +static inline struct io_queue *elv_lookup_ioq_current (struct request_queue *q) +{ +return NULL ; +} +#endif /* CONFIG_ELV_FAIR_QUEUING */ #endif /* _ELV_SCHED_H */ #endif /* CONFIG_BLOCK */
block/elevator.c +35 -2
@@ -228,9 +228,17 @@ elevator_alloc_sched_queue(struct request_queue *q, struct elevator_queue *eq) {void *sched_queue =NULL ; +/* +* If fair queuing is enabled, then queue allocation takes place +* during set_request() functions when request actually comes +* in. +*/ +if (elv_iosched_fair_queuing_enabled (eq)) +return NULL ; +if (eq->ops->elevator_alloc_sched_queue_fn) { sched_queue = eq->ops->elevator_alloc_sched_queue_fn (q, eq, -GFP_KERNEL ); +GFP_KERNEL ,NULL );if (!sched_queue)return ERR_PTR (-ENOMEM );
@@ -861,6 +869,13 @@ int elv_set_request(struct request_queue *q, struct request *rq, gfp_t gfp_mask) {struct elevator_queue *e = q->elevator; +/* +* Optimization for noop, deadline and AS which maintain only single +* ioq per io group +*/ +if (elv_iosched_single_ioq (e)) +return elv_set_request_ioq (q, rq, gfp_mask); +if (e->ops->elevator_set_req_fn)return e->ops->elevator_set_req_fn (q, rq, gfp_mask);
@@ -872,6 +887,15 @@ void elv_put_request(struct request_queue *q, struct request *rq) {struct elevator_queue *e = q->elevator; +/* +* Optimization for noop, deadline and AS which maintain only single +* ioq per io group +*/ +if (elv_iosched_single_ioq (e)) { +elv_reset_request_ioq (q, rq); +return ; + } +if (e->ops->elevator_put_req_fn) e->ops->elevator_put_req_fn (rq); }
@@ -1256,9 +1280,18 @@ EXPORT_SYMBOL(elv_select_sched_queue);/* * Get the io scheduler queue pointer for current task. +* +* If fair queuing is enabled, determine the io group of task and retrieve +* the ioq pointer from that. This is used by only single queue ioschedulers +* for retrieving the queue associated with the group to decide whether the +* new bio can do a front merge or not. */ void *elv_get_sched_queue_current (struct request_queue *q) { -return q->elevator->sched_queue; +/* Fair queuing is not enabled. There is only one queue. */ +if (!elv_iosched_fair_queuing_enabled (q->elevator)) +return q->elevator->sched_queue; + +return elv_ioq_sched_queue (elv_lookup_ioq_current (q)); }EXPORT_SYMBOL (elv_get_sched_queue_current);
block/noop-iosched.c +1 -1
@@ -62,7 +62,7 @@ noop_latter_request(struct request_queue *q, struct request *rq) }static void *noop_alloc_noop_queue (struct request_queue *q, -struct elevator_queue *eq,gfp_t gfp_mask) +struct elevator_queue *eq,gfp_t gfp_mask,struct io_queue *ioq) {struct noop_queue *nq;
include/linux/elevator.h +16 -2
@@ -30,9 +30,9 @@ typedef void (elevator_deactivate_req_fn) (struct request_queue *, struct requestypedef void *(elevator_init_fn ) (struct request_queue *,struct elevator_queue *);typedef void (elevator_exit_fn ) (struct elevator_queue *); -typedef void * (elevator_alloc_sched_queue_fn ) (struct request_queue *q, -struct elevator_queue *eq,gfp_t );typedef void (elevator_free_sched_queue_fn ) (struct elevator_queue *,void *); +typedef void * (elevator_alloc_sched_queue_fn ) (struct request_queue *q, +struct elevator_queue *eq,gfp_t ,struct io_queue *ioq);#ifdef CONFIG_ELV_FAIR_QUEUING typedef void (elevator_active_ioq_set_fn ) (struct request_queue *,void *,int );typedef void (elevator_active_ioq_reset_fn ) (struct request_queue *,void *);
@@ -245,17 +245,31 @@ enum {/* iosched wants to use fair queuing logic of elevator layer */ #define ELV_IOSCHED_NEED_FQ 1 +/* iosched maintains only single ioq per group.*/ +#define ELV_IOSCHED_SINGLE_IOQ 2 +static inline int elv_iosched_fair_queuing_enabled (struct elevator_queue *e) {return (e->elevator_type->elevator_features) &ELV_IOSCHED_NEED_FQ ; } +static inline int elv_iosched_single_ioq (struct elevator_queue *e) +{ +return (e->elevator_type->elevator_features) &ELV_IOSCHED_SINGLE_IOQ ; +} +#else /* ELV_IOSCHED_FAIR_QUEUING */ static inline int elv_iosched_fair_queuing_enabled (struct elevator_queue *e) {return 0 ; } + +static inline int elv_iosched_single_ioq (struct elevator_queue *e) +{ +return 0 ; +} +#endif /* ELV_IOSCHED_FAIR_QUEUING */ extern void *elv_get_sched_queue (struct request_queue *q,struct request *rq);extern void *elv_select_sched_queue (struct request_queue *q,int force);
Signature · 1 line
1.6.0.6
Re: [PATCH 14/23] io-conroller: Prepare elevator layer for single queue schedulers
Quoted 9 lines · Vivek Goyal
Elevator layer now has support for hierarchical fair queuing. cfq has been migrated to make use of it and now it is time to do groundwork for noop, deadline and AS.
noop deadline and AS don't maintain separate queues for different processes. There is only one single queue. Effectively one can think that in hierarchical setup, there will be one queue per cgroup where requests from all the processes in the cgroup will be queued.
Quoted 8 lines
Generally io scheduler takes care of creating queues. Because there is only one queue here, we have modified common layer to take care of queue creation and some other functionality. This special casing helps in keeping the changes to noop, deadline and AS to the minimum.
Signed-off-by Nauman Rafique <[email protected]>Signed-off-by Gui Jianfeng <[email protected]>Signed-off-by Vivek Goyal <[email protected]>
Signature · 1 line
All rights reversed.
A message that is not in the archive (sent off-list or not yet received): 1251495072-7780-15-git-send-email-vgoyal-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org
Rik van Riel
Acked-byRe: [PATCH 14/23] io-conroller: Prepare elevator layer for single queue schedulers
Quoted 9 lines · Vivek Goyal
Elevator layer now has support for hierarchical fair queuing. cfq has been migrated to make use of it and now it is time to do groundwork for noop, deadline and AS.
noop deadline and AS don't maintain separate queues for different processes. There is only one single queue. Effectively one can think that in hierarchical setup, there will be one queue per cgroup where requests from all the processes in the cgroup will be queued.
Quoted 8 lines
Generally io scheduler takes care of creating queues. Because there is only one queue here, we have modified common layer to take care of queue creation and some other functionality. This special casing helps in keeping the changes to noop, deadline and AS to the minimum.
Signed-off-by Nauman Rafique <[email protected]>Signed-off-by Gui Jianfeng <[email protected]>Signed-off-by Vivek Goyal <[email protected]>
Signature · 1 line
All rights reversed.
PATCH 15/23 io-controller: noop changes for hierarchical fair queuing
In reply to Vivek GoyalThis patch changes noop to use queue scheduling code from elevator layer. One can go back to old noop by deselecting CONFIG_IOSCHED_NOOP_HIER.
--- block/Kconfig.iosched | 11 +++++++++++ block/noop-iosched.c | 14 ++++++++++++++ 2 files changed, 25 insertions(+), 0 deletions(-)
block/Kconfig.iosched +11 -0
@@ -25,6 +25,17 @@ config IOSCHED_NOOP that do their own scheduling and require only minimal assistance from the kernel. +config IOSCHED_NOOP_HIER + bool "Noop Hierarchical Scheduling support" + depends on IOSCHED_NOOP && CGROUPS + select ELV_FAIR_QUEUING + select GROUP_IOSCHED + default n + ---help--- + Enable hierarhical scheduling in noop. In this mode noop keeps + one IO queue per cgroup instead of a global queue. Elevator + fair queuing logic ensures fairness among various queues. + config IOSCHED_AS tristate "Anticipatory I/O scheduler" default y
block/noop-iosched.c +14 -0
@@ -6,6 +6,7 @@#include <linux/bio.h> #include <linux/module.h> #include <linux/init.h> +#include "elevator-fq.h" struct noop_queue {struct list_head queue;
@@ -82,6 +83,15 @@ static void noop_free_noop_queue(struct elevator_queue *e, void *sched_queue)kfree (nq); } +#ifdef CONFIG_IOSCHED_NOOP_HIER +static struct elv_fs_entry noop_attrs[] = { +ELV_ATTR (fairness), +ELV_ATTR (slice_sync), +ELV_ATTR (group_idle), + __ATTR_NULL +}; +#endif +static struct elevator_type elevator_noop = { .ops = { .elevator_merge_req_fn = noop_merged_requests,
@@ -92,6 +102,10 @@ static struct elevator_type elevator_noop = { .elevator_alloc_sched_queue_fn = noop_alloc_noop_queue, .elevator_free_sched_queue_fn = noop_free_noop_queue, }, +#ifdef CONFIG_IOSCHED_NOOP_HIER + .elevator_features = ELV_IOSCHED_NEED_FQ | ELV_IOSCHED_SINGLE_IOQ, + .elevator_attrs = noop_attrs, +#endif .elevator_name = "noop", .elevator_owner = THIS_MODULE, };
Signature · 1 line
1.6.0.6
Re: [PATCH 15/23] io-controller: noop changes for hierarchical fair queuing
Quoted 5 lines · Vivek Goyal
This patch changes noop to use queue scheduling code from elevator layer. One can go back to old noop by deselecting
CONFIG_IOSCHED_NOOP_HIER.Signed-off-by Nauman Rafique <[email protected]>Signed-off-by Vivek Goyal <[email protected]>
Not sure why noop needs hierarchical fair queueing support, but this patch is so small we might as well take it to keep things consistent between schedulers.
Signature · 1 line
All rights reversed.
Re: [PATCH 15/23] io-controller: noop changes for hierarchical fair queuing
Quoted 11 lines · Rik van Riel
Quoted 5 lines · Vivek Goyal
This patch changes noop to use queue scheduling code from elevator layer. One can go back to old noop by deselecting
CONFIG_IOSCHED_NOOP_HIER.Signed-off-by Nauman Rafique <[email protected]>Signed-off-by Vivek Goyal <[email protected]>Not sure why noop needs hierarchical fair queueing support, but this patch is so small we might as well take it to keep things consistent between schedulers.
Thinking more about it. It probably can be useful for the case ryo is pointing out where fast SSD drivers are not making use of kernel IO schedulers.
If they want group io schduling support on these SSDs, they can modify their driver to make use of hierarchical noop.
Quoted 1 line
Acked-by Rik van Riel <[email protected]>
Thanks
Vivek
Quoted 3 lines
Signature · 2 lines
-- All rights reversed.
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH 15/23] io-controller: noop changes for hierarchical fair queuing
Quoted 11 lines · Rik van Riel
Quoted 5 lines · Vivek Goyal
This patch changes noop to use queue scheduling code from elevator layer. One can go back to old noop by deselecting
CONFIG_IOSCHED_NOOP_HIER.Signed-off-by Nauman Rafique <[email protected]>Signed-off-by Vivek Goyal <[email protected]>Not sure why noop needs hierarchical fair queueing support, but this patch is so small we might as well take it to keep things consistent between schedulers.
Thinking more about it. It probably can be useful for the case ryo is pointing out where fast SSD drivers are not making use of kernel IO schedulers.
If they want group io schduling support on these SSDs, they can modify their driver to make use of hierarchical noop.
Quoted 1 line
Acked-by Rik van Riel <[email protected]>
Thanks
Vivek
Quoted 3 lines
Signature · 2 lines
-- All rights reversed.
A message that is not in the archive (sent off-list or not yet received): 1251495072-7780-16-git-send-email-vgoyal-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org
Rik van Riel
Acked-byRe: [PATCH 15/23] io-controller: noop changes for hierarchical fair queuing
Quoted 5 lines · Vivek Goyal
This patch changes noop to use queue scheduling code from elevator layer. One can go back to old noop by deselecting
CONFIG_IOSCHED_NOOP_HIER.Signed-off-by Nauman Rafique <[email protected]>Signed-off-by Vivek Goyal <[email protected]>
Not sure why noop needs hierarchical fair queueing support, but this patch is so small we might as well take it to keep things consistent between schedulers.
Signature · 1 line
All rights reversed.
PATCH 16/23 io-controller: deadline changes for hierarchical fair queuing
In reply to Vivek GoyalThis patch changes deadline to use queue scheduling code from elevator layer. One can go back to old deadline by selecting CONFIG_IOSCHED_DEADLINE_HIER.
--- block/Kconfig.iosched | 11 +++++++++++ block/deadline-iosched.c | 9 +++++++++ 2 files changed, 20 insertions(+), 0 deletions(-)
block/Kconfig.iosched +11 -0
@@ -55,6 +55,17 @@ config IOSCHED_DEADLINE a disk at any one time, its behaviour is almost identical to the anticipatory I/O scheduler and so is a good choice. +config IOSCHED_DEADLINE_HIER + bool "Deadline Hierarchical Scheduling support" + depends on IOSCHED_DEADLINE && CGROUPS + select ELV_FAIR_QUEUING + select GROUP_IOSCHED + default n + ---help--- + Enable hierarhical scheduling in deadline. In this mode deadline keeps + one IO queue per cgroup instead of a global queue. Elevator + fair queuing logic ensures fairness among various queues. + config IOSCHED_CFQ tristate "CFQ I/O scheduler" select ELV_FAIR_QUEUING
block/deadline-iosched.c +9 -0
@@ -13,6 +13,7 @@#include <linux/init.h> #include <linux/compiler.h> #include <linux/rbtree.h> +#include "elevator-fq.h" /* * See Documentation/block/deadline-iosched.txt
@@ -461,6 +462,11 @@ static struct elv_fs_entry deadline_attrs[] = {DD_ATTR (writes_starved),DD_ATTR (front_merges),DD_ATTR (fifo_batch), +#ifdef CONFIG_IOSCHED_DEADLINE_HIER +ELV_ATTR (fairness), +ELV_ATTR (slice_sync), +ELV_ATTR (group_idle), +#endif __ATTR_NULL };
@@ -478,6 +484,9 @@ static struct elevator_type iosched_deadline = { .elevator_alloc_sched_queue_fn = deadline_alloc_deadline_queue, .elevator_free_sched_queue_fn = deadline_free_deadline_queue, }, +#ifdef CONFIG_IOSCHED_DEADLINE_HIER + .elevator_features = ELV_IOSCHED_NEED_FQ | ELV_IOSCHED_SINGLE_IOQ, +#endif .elevator_attrs = deadline_attrs, .elevator_name = "deadline", .elevator_owner = THIS_MODULE,
Signature · 1 line
1.6.0.6
Re: [PATCH 16/23] io-controller: deadline changes for hierarchical fair queuing
Quoted 2 lines · Vivek Goyal
This patch changes deadline to use queue scheduling code from elevator layer. One can go back to old deadline by selecting
CONFIG_IOSCHED_DEADLINE_HIER.
^ deselecting ?
Quoted 2 lines
Signed-off-by Nauman Rafique <[email protected]>Signed-off-by Vivek Goyal <[email protected]>
Signature · 1 line
All rights reversed.
Re: [PATCH 16/23] io-controller: deadline changes for hierarchical fair queuing
Quoted 5 lines · Rik van Riel
Quoted 2 lines · Vivek Goyal
This patch changes deadline to use queue scheduling code from elevator layer. One can go back to old deadline by selecting
CONFIG_IOSCHED_DEADLINE_HIER.^ deselecting ?
Yes. Its a typo. Should be "deselecting". Will fix the comment.
Thanks
Vivek
Quoted 7 lines
Quoted 2 lines
Signed-off-by Nauman Rafique <[email protected]>Signed-off-by Vivek Goyal <[email protected]>Acked-by Rik van Riel <[email protected]>Signature · 1 line
All rights reversed.
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH 16/23] io-controller: deadline changes for hierarchical fair queuing
Quoted 5 lines · Rik van Riel
Quoted 2 lines · Vivek Goyal
This patch changes deadline to use queue scheduling code from elevator layer. One can go back to old deadline by selecting
CONFIG_IOSCHED_DEADLINE_HIER.^ deselecting ?
Yes. Its a typo. Should be "deselecting". Will fix the comment.
Thanks
Vivek
Quoted 7 lines
Quoted 2 lines
Signed-off-by Nauman Rafique <[email protected]>Signed-off-by Vivek Goyal <[email protected]>Acked-by Rik van Riel <[email protected]>Signature · 1 line
All rights reversed.
A message that is not in the archive (sent off-list or not yet received): 1251495072-7780-17-git-send-email-vgoyal-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org
Rik van Riel
Acked-byRe: [PATCH 16/23] io-controller: deadline changes for hierarchical fair queuing
Quoted 2 lines · Vivek Goyal
This patch changes deadline to use queue scheduling code from elevator layer. One can go back to old deadline by selecting
CONFIG_IOSCHED_DEADLINE_HIER.
^ deselecting ?
Quoted 2 lines
Signed-off-by Nauman Rafique <[email protected]>Signed-off-by Vivek Goyal <[email protected]>
Signature · 1 line
All rights reversed.
PATCH 17/23 io-controller: anticipatory changes for hierarchical fair queuing
In reply to Vivek GoyalThis patch changes anticipatory scheduler to use queue scheduling code from elevator layer. One can go back to old as by deselecting CONFIG_IOSCHED_AS_HIER. Even with CONFIG_IOSCHED_AS_HIER=y, with-out any other cgroup created, AS behavior should remain the same as old.
o AS is a single queue ioschduler, that means there is one AS queue per group.
o common layer code select the queue to dispatch from based on fairness, and then AS code selects the request with-in group.
o AS runs reads and writes batches with-in group. So common layer runs timed group queues and with-in group time, AS runs timed batches of reads and writes.
o Note: Previously AS write batch length was adjusted synamically whenever a W->R batch data direction took place and when first request from the read batch completed.
Now write batch updation takes place when last request from the write batch has finished during W->R transition.
o AS runs its own anticipation logic to anticipate on reads. common layer also does the anticipation on the group if think time of the group is with-in slice_idle.
o Introduced few debugging messages in AS.
--- block/Kconfig.iosched | 12 ++ block/as-iosched.c | 295 +++++++++++++++++++++++++++++++++++++++++++++- block/elevator-fq.c | 89 ++++++++++++-- block/elevator-fq.h | 2 + include/linux/elevator.h | 2 + 5 files changed, 382 insertions(+), 18 deletions(-)
block/Kconfig.iosched +12 -0
@@ -45,6 +45,18 @@ config IOSCHED_AS deadline I/O scheduler, it can also be slower in some cases especially some database loads. +config IOSCHED_AS_HIER + bool "Anticipatory Hierarchical Scheduling support" + depends on IOSCHED_AS && CGROUPS + select ELV_FAIR_QUEUING + select GROUP_IOSCHED + default n + ---help--- + Enable hierarhical scheduling in anticipatory. In this mode + anticipatory keeps one IO queue per cgroup instead of a global + queue. Elevator fair queuing logic ensures fairness among various + queues. + config IOSCHED_DEADLINE tristate "Deadline I/O scheduler" default y
block/as-iosched.c +289 -6
497 lines not on this page · raw message
block/elevator-fq.c +77 -12
150 lines not on this page · raw message
block/elevator-fq.h +2 -0
@@ -229,6 +229,7 @@ enum elv_queue_state_flags { ELV_QUEUE_FLAG_idle_window,/* elevator slice idling enabled */ ELV_QUEUE_FLAG_slice_new,/* no requests dispatched in slice */ ELV_QUEUE_FLAG_sync,/* synchronous queue */ + ELV_QUEUE_FLAG_must_expire, /* expire queue even slice is left */ };#define ELV_IO_QUEUE_FLAG_FNS (name) \
@@ -251,6 +252,7 @@ ELV_IO_QUEUE_FLAG_FNS(must_dispatch) ELV_IO_QUEUE_FLAG_FNS(idle_window) ELV_IO_QUEUE_FLAG_FNS(slice_new) ELV_IO_QUEUE_FLAG_FNS(sync) +ELV_IO_QUEUE_FLAG_FNS(must_expire)#ifdef CONFIG_GROUP_IOSCHED
include/linux/elevator.h +2 -0
@@ -41,6 +41,7 @@ typedef int (elevator_should_preempt_fn) (struct request_queue*, void*, struct request*);typedef struct io_queue * (elevator_close_cooperator_fn ) (struct request_queue *,void *); +typedef int (elevator_expire_ioq_fn ) (struct request_queue *,void *,int ,int );#endif struct elevator_ops
@@ -79,6 +80,7 @@ struct elevator_opselevator_arm_slice_timer_fn *elevator_arm_slice_timer_fn;elevator_should_preempt_fn *elevator_should_preempt_fn;elevator_close_cooperator_fn *elevator_close_cooperator_fn; +elevator_expire_ioq_fn *elevator_expire_ioq_fn;#endif };
Signature · 1 line
1.6.0.6
Re: [PATCH 17/23] io-controller: anticipatory changes for hierarchical fair queuing
Quoted 21 lines · Vivek Goyal
This patch changes anticipatory scheduler to use queue scheduling code from elevator layer. One can go back to old as by deselecting
CONFIG_IOSCHED_AS_HIER. Even withCONFIG_IOSCHED_AS_HIER=y, with-out any other cgroup created, AS behavior should remain the same as old.o AS is a single queue ioschduler, that means there is one AS queue per group.
o common layer code select the queue to dispatch from based on fairness, and then AS code selects the request with-in group.
o AS runs reads and writes batches with-in group. So common layer runs timed group queues and with-in group time, AS runs timed batches of reads and writes.
o Note: Previously AS write batch length was adjusted synamically whenever a
W->Rbatch data direction took place and when first request from the read batch completed.Now write batch updation takes place when last request from the write batch has finished during
W->Rtransition.
Quoted 8 lines
o AS runs its own anticipation logic to anticipate on reads. common layer also does the anticipation on the group if think time of the group is with-in
slice_idle.o Introduced few debugging messages in AS.
Signed-off-by Nauman Rafique <[email protected]>Signed-off-by Vivek Goyal <[email protected]>
A message that is not in the archive (sent off-list or not yet received): 1251495072-7780-18-git-send-email-vgoyal-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org
Rik van Riel
Acked-byRe: [PATCH 17/23] io-controller: anticipatory changes for hierarchical fair queuing
Quoted 21 lines · Vivek Goyal
This patch changes anticipatory scheduler to use queue scheduling code from elevator layer. One can go back to old as by deselecting
CONFIG_IOSCHED_AS_HIER. Even withCONFIG_IOSCHED_AS_HIER=y, with-out any other cgroup created, AS behavior should remain the same as old.o AS is a single queue ioschduler, that means there is one AS queue per group.
o common layer code select the queue to dispatch from based on fairness, and then AS code selects the request with-in group.
o AS runs reads and writes batches with-in group. So common layer runs timed group queues and with-in group time, AS runs timed batches of reads and writes.
o Note: Previously AS write batch length was adjusted synamically whenever a
W->Rbatch data direction took place and when first request from the read batch completed.Now write batch updation takes place when last request from the write batch has finished during
W->Rtransition.
Quoted 8 lines
o AS runs its own anticipation logic to anticipate on reads. common layer also does the anticipation on the group if think time of the group is with-in
slice_idle.o Introduced few debugging messages in AS.
Signed-off-by Nauman Rafique <[email protected]>Signed-off-by Vivek Goyal <[email protected]>
PATCH 18/23 io-controller: blkio_cgroup patches from Ryo to track async bios.
In reply to Vivek Goyalo blkio_cgroup patches from Ryo to track async bios.
o This functionality is used to determine the group of async IO from page instead of context of submitting task.
--- block/blk-ioc.c | 36 +++--- fs/buffer.c | 2 + fs/direct-io.c | 2 + include/linux/biotrack.h | 100 ++++++++++++++ include/linux/cgroup_subsys.h | 6 + include/linux/iocontext.h | 1 + include/linux/memcontrol.h | 6 + include/linux/mmzone.h | 4 +- include/linux/page_cgroup.h | 5 +- init/Kconfig | 16 +++ mm/Makefile | 4 +- mm/biotrack.c | 293 +++++++++++++++++++++++++++++++++++++++++ mm/bounce.c | 2 + mm/filemap.c | 2 + mm/memcontrol.c | 6 + mm/memory.c | 5 + mm/page-writeback.c | 2 + mm/page_cgroup.c | 23 ++-- mm/swap_state.c | 2 + 19 files changed, 486 insertions(+), 31 deletions(-) create mode 100644 include/linux/biotrack.h create mode 100644 mm/biotrack.c
block/blk-ioc.c +20 -16
@@ -84,27 +84,31 @@ void exit_io_context(void) } } +void init_io_context (struct io_context *ioc) +{ +atomic_long_set (&ioc->refcount,1 ); +atomic_set (&ioc->nr_tasks,1 ); +spin_lock_init (&ioc->lock); + ioc->ioprio_changed =0 ; + ioc->ioprio =0 ; +#ifdef CONFIG_GROUP_IOSCHED + ioc->cgroup_changed =0 ; +#endif + ioc->last_waited = jiffies;/* doesn't matter... */ + ioc->nr_batch_requests =0 ;/* because this is 0 */ + ioc->aic =NULL ; +INIT_RADIX_TREE (&ioc->radix_root,GFP_ATOMIC | __GFP_HIGH); +INIT_HLIST_HEAD (&ioc->cic_list); + ioc->ioc_data =NULL ; +} +struct io_context *alloc_io_context (gfp_t gfp_flags,int node) {struct io_context *ret; ret =kmem_cache_alloc_node (iocontext_cachep, gfp_flags, node); -if (ret) { -atomic_long_set (&ret->refcount,1 ); -atomic_set (&ret->nr_tasks,1 ); -spin_lock_init (&ret->lock); - ret->ioprio_changed =0 ; - ret->ioprio =0 ; -#ifdef CONFIG_GROUP_IOSCHED - ret->cgroup_changed =0 ; -#endif - ret->last_waited = jiffies;/* doesn't matter... */ - ret->nr_batch_requests =0 ;/* because this is 0 */ - ret->aic =NULL ; -INIT_RADIX_TREE (&ret->radix_root,GFP_ATOMIC | __GFP_HIGH); -INIT_HLIST_HEAD (&ret->cic_list); - ret->ioc_data =NULL ; - } +if (ret) +init_io_context (ret);return ret; }
fs/buffer.c +2 -0
@@ -36,6 +36,7 @@#include <linux/buffer_head.h> #include <linux/task_io_accounting_ops.h> #include <linux/bio.h> +#include <linux/biotrack.h> #include <linux/notifier.h> #include <linux/cpu.h> #include <linux/bitops.h>
@@ -668,6 +669,7 @@ static void __set_page_dirty(struct page *page,if (page->mapping) {/* Race with truncate? */ WARN_ON_ONCE (warn && !PageUptodate (page));account_page_dirtied (page, mapping); +blkio_cgroup_reset_owner_pagedirty (page, current->mm);radix_tree_tag_set (&mapping->page_tree,page_index (page),PAGECACHE_TAG_DIRTY ); }
fs/direct-io.c +2 -0
@@ -33,6 +33,7 @@#include <linux/err.h> #include <linux/blkdev.h> #include <linux/buffer_head.h> +#include <linux/biotrack.h> #include <linux/rwsem.h> #include <linux/uio.h> #include <asm/atomic.h>
@@ -797,6 +798,7 @@ static int do_direct_IO(struct dio *dio) ret =PTR_ERR (page); goto out; } +blkio_cgroup_reset_owner (page, current->mm);while (block_in_page < blocks_per_page) {unsigned offset_in_page = block_in_page << blkbits;
include/linux/biotrack.h new file +100 -0
100 lines not on this page · raw message
include/linux/cgroup_subsys.h +6 -0
@@ -43,6 +43,12 @@ SUBSYS(mem_cgroup)/* */ +#ifdef CONFIG_CGROUP_BLKIO +SUBSYS(blkio_cgroup) +#endif + +/* */ +#ifdef CONFIG_CGROUP_DEVICE SUBSYS(devices)#endif
include/linux/iocontext.h +1 -0
@@ -109,6 +109,7 @@ int put_io_context(struct io_context *ioc);void exit_io_context (void );struct io_context *get_io_context (gfp_t gfp_flags,int node);struct io_context *alloc_io_context (gfp_t gfp_flags,int node); +void init_io_context (struct io_context *ioc);void copy_io_context (struct io_context **pdst,struct io_context **psrc);#else static inline void exit_io_context (void )
include/linux/memcontrol.h +6 -0
@@ -37,6 +37,8 @@ struct mm_struct;* (Of course, if memcg does memory allocation in future, GFP_KERNEL is sane.) */ +extern void __init_mem_page_cgroup (struct page_cgroup *pc); +extern int mem_cgroup_newpage_charge (struct page *page,struct mm_struct *mm,gfp_t gfp_mask);/* for swap handling */
@@ -121,6 +123,10 @@ void mem_cgroup_update_mapped_file_stat(struct page *page, int val);#else /* CONFIG_CGROUP_MEM_RES_CTLR */ struct mem_cgroup ; +static inline void __init_mem_page_cgroup (struct page_cgroup *pc) +{ +} +static inline int mem_cgroup_newpage_charge (struct page *page,struct mm_struct *mm,gfp_t gfp_mask) {
include/linux/mmzone.h +2 -2
@@ -605,7 +605,7 @@ typedef struct pglist_data {int nr_zones;#ifdef CONFIG_FLAT_NODE_MEM_MAP /* means !SPARSEMEM */ struct page *node_mem_map; -#ifdef CONFIG_CGROUP_MEM_RES_CTLR +#ifdef CONFIG_CGROUP_PAGE struct page_cgroup *node_page_cgroup;#endif #endif
@@ -956,7 +956,7 @@ struct mem_section {/* See declaration of similar field in struct zone */ unsigned long *pageblock_flags; -#ifdef CONFIG_CGROUP_MEM_RES_CTLR +#ifdef CONFIG_CGROUP_PAGE /* * If !SPARSEMEM, pgdat doesn't have page_cgroup pointer. We use * section. (see memcontrol.h/page_cgroup.h about this.)
include/linux/page_cgroup.h +3 -2
@@ -1,7 +1,7 @@#ifndef __LINUX_PAGE_CGROUP_H#define __LINUX_PAGE_CGROUP_H -#ifdef CONFIG_CGROUP_MEM_RES_CTLR +#ifdef CONFIG_CGROUP_PAGE #include <linux/bit_spinlock.h> /* * Page Cgroup can be considered as an extended mem_map.
@@ -14,6 +14,7 @@ struct page_cgroup {unsigned long flags;struct mem_cgroup *mem_cgroup;struct page *page; +unsigned long blkio_cgroup_id;struct list_head lru;/* per cgroup LRU list */ };
@@ -83,7 +84,7 @@ static inline void unlock_page_cgroup(struct page_cgroup *pc) bit_spin_unlock(PCG_LOCK, &pc->flags); } -#else /* CONFIG_CGROUP_MEM_RES_CTLR */ +#else /* CONFIG_CGROUP_PAGE */ struct page_cgroup ;static inline void __meminit pgdat_page_cgroup_init (struct pglist_data *pgdat)
init/Kconfig +16 -0
@@ -622,6 +622,22 @@ config GROUP_IOSCHED endif # CGROUPS +config CGROUP_BLKIO + bool + depends on CGROUPS && BLOCK + select MM_OWNER + default n + ---help--- + Provides a Resource Controller which enables to track the onwner + of every Block I/O requests. + The information this subsystem provides can be used from any + kind of module such as dm-ioband device mapper modules or + the cfq-scheduler. + +config CGROUP_PAGE + def_bool y + depends on CGROUP_MEM_RES_CTLR || CGROUP_BLKIO + config MM_OWNER bool
mm/Makefile +3 -1
@@ -39,6 +39,8 @@ else obj-$(CONFIG_SMP) += allocpercpu.o endif obj-$(CONFIG_QUICKLIST) += quicklist.o -obj-$(CONFIG_CGROUP_MEM_RES_CTLR) += memcontrol.o page_cgroup.o +obj-$(CONFIG_CGROUP_MEM_RES_CTLR) += memcontrol.o +obj-$(CONFIG_CGROUP_PAGE) += page_cgroup.o +obj-$(CONFIG_CGROUP_BLKIO) += biotrack.o obj-$(CONFIG_DEBUG_KMEMLEAK) += kmemleak.o obj-$(CONFIG_DEBUG_KMEMLEAK_TEST) += kmemleak-test.o
mm/biotrack.c new file +293 -0
293 lines not on this page · raw message
mm/bounce.c +2 -0
@@ -13,6 +13,7 @@#include <linux/init.h> #include <linux/hash.h> #include <linux/highmem.h> +#include <linux/biotrack.h> #include <asm/tlbflush.h> #include <trace/events/block.h>
@@ -210,6 +211,7 @@ static void __blk_queue_bounce(struct request_queue *q, struct bio **bio_orig, to->bv_len = from->bv_len; to->bv_offset = from->bv_offset;inc_zone_page_state (to->bv_page,NR_BOUNCE ); +blkio_cgroup_copy_owner (to->bv_page, page);if (rw ==WRITE ) {char *vto, *vfrom;
mm/filemap.c +2 -0
@@ -33,6 +33,7 @@#include <linux/cpuset.h> #include <linux/hardirq.h> /* for BUG_ON(!in_atomic()) only */ #include <linux/memcontrol.h> +#include <linux/biotrack.h> #include <linux/mm_inline.h> /* for page_is_file_cache() */ #include "internal.h"
@@ -464,6 +465,7 @@ int add_to_page_cache_locked(struct page *page, struct address_space *mapping, gfp_mask & GFP_RECLAIM_MASK);if (error) goto out; +blkio_cgroup_set_owner (page, current->mm); error =radix_tree_preload (gfp_mask & ~__GFP_HIGHMEM);if (error ==0 ) {
mm/memcontrol.c +6 -0
@@ -129,6 +129,12 @@ struct mem_cgroup_lru_info {struct mem_cgroup_per_node *nodeinfo[MAX_NUMNODES ]; }; +void __meminit __init_mem_page_cgroup (struct page_cgroup *pc) +{ + pc->mem_cgroup =NULL ; +INIT_LIST_HEAD (&pc->lru); +} +/* * The memory controller data structure. The memory controller controls both * page cache and RSS per cgroup. We would eventually like to provide
mm/memory.c +5 -0
@@ -51,6 +51,7 @@#include <linux/init.h> #include <linux/writeback.h> #include <linux/memcontrol.h> +#include <linux/biotrack.h> #include <linux/mmu_notifier.h> #include <linux/kallsyms.h> #include <linux/swapops.h>
@@ -2116,6 +2117,7 @@ gotten:*/ ptep_clear_flush_notify (vma, address, page_table);page_add_new_anon_rmap (new_page, vma, address); +blkio_cgroup_set_owner (new_page, mm);set_pte_at (mm, address, page_table, entry);update_mmu_cache (vma, address, entry);if (old_page) {
@@ -2581,6 +2583,7 @@ static int do_swap_page(struct mm_struct *mm, struct vm_area_struct *vma,flush_icache_page (vma, page);set_pte_at (mm, address, page_table, pte);page_add_anon_rmap (page, vma, address); +blkio_cgroup_reset_owner (page, mm);/* It's better to call commit-charge after rmap is established */ mem_cgroup_commit_charge_swapin (page, ptr);
@@ -2645,6 +2648,7 @@ static int do_anonymous_page(struct mm_struct *mm, struct vm_area_struct *vma, goto release;inc_mm_counter (mm, anon_rss);page_add_new_anon_rmap (page, vma, address); +blkio_cgroup_set_owner (page, mm);set_pte_at (mm, address, page_table, entry);/* No need to invalidate - it was non-present before */
@@ -2792,6 +2796,7 @@ static int __do_fault(struct mm_struct *mm, struct vm_area_struct *vma,if (anon) {inc_mm_counter (mm, anon_rss);page_add_new_anon_rmap (page, vma, address); +blkio_cgroup_set_owner (page, mm); }else {inc_mm_counter (mm, file_rss);page_add_file_rmap (page);
mm/page-writeback.c +2 -0
@@ -23,6 +23,7 @@#include <linux/init.h> #include <linux/backing-dev.h> #include <linux/task_io_accounting_ops.h> +#include <linux/biotrack.h> #include <linux/blkdev.h> #include <linux/mpage.h> #include <linux/rmap.h>
@@ -1247,6 +1248,7 @@ int __set_page_dirty_nobuffers(struct page *page)BUG_ON (mapping2 != mapping);WARN_ON_ONCE (!PagePrivate (page) && !PageUptodate (page));account_page_dirtied (page, mapping); +blkio_cgroup_reset_owner_pagedirty (page, current->mm);radix_tree_tag_set (&mapping->page_tree,page_index (page),PAGECACHE_TAG_DIRTY ); }
mm/page_cgroup.c +13 -10
@@ -9,14 +9,15 @@#include <linux/vmalloc.h> #include <linux/cgroup.h> #include <linux/swapops.h> +#include <linux/biotrack.h> static void __meminit __init_page_cgroup (struct page_cgroup *pc,unsigned long pfn) { pc->flags =0 ; - pc->mem_cgroup =NULL ; pc->page =pfn_to_page (pfn); -INIT_LIST_HEAD (&pc->lru); +__init_mem_page_cgroup (pc); +__init_blkio_page_cgroup (pc); }static unsigned long total_usage;
@@ -74,7 +75,7 @@ void __init page_cgroup_init_flatmem(void)int nid, fail; -if (mem_cgroup_disabled ()) +if (mem_cgroup_disabled () &&blkio_cgroup_disabled ())return ;for_each_online_node (nid) {
@@ -83,12 +84,13 @@ void __init page_cgroup_init_flatmem(void) goto fail; }printk (KERN_INFO "allocated %ld bytes of page_cgroup\n" , total_usage); -printk (KERN_INFO "please try 'cgroup_disable=memory' option if you" -" don't want memory cgroups\n" ); +printk (KERN_INFO "please try 'cgroup_disable=memory,blkio' option" +" if you don't want memory and blkio cgroups\n" );return ; fail:printk (KERN_CRIT "allocation of page_cgroup failed.\n" ); -printk (KERN_CRIT "please try 'cgroup_disable=memory' boot option\n" ); +printk (KERN_CRIT +"please try 'cgroup_disable=memory,blkio' boot option\n" );panic ("Out of memory" ); }
@@ -245,7 +247,7 @@ void __init page_cgroup_init(void)unsigned long pfn;int fail =0 ; -if (mem_cgroup_disabled ()) +if (mem_cgroup_disabled () &&blkio_cgroup_disabled ())return ;for (pfn =0 ; !fail && pfn < max_pfn; pfn +=PAGES_PER_SECTION ) {
@@ -254,14 +256,15 @@ void __init page_cgroup_init(void) fail =init_section_page_cgroup (pfn); }if (fail) { -printk (KERN_CRIT "try 'cgroup_disable=memory' boot option\n" ); +printk (KERN_CRIT +"try 'cgroup_disable=memory,blkio' boot option\n" );panic ("Out of memory" ); }else {hotplug_memory_notifier (page_cgroup_callback,0 ); }printk (KERN_INFO "allocated %ld bytes of page_cgroup\n" , total_usage); -printk (KERN_INFO "please try 'cgroup_disable=memory' option if you don't" -" want memory cgroups\n" ); +printk (KERN_INFO "please try 'cgroup_disable=memory,blkio' option" +" if you don't want memory and blkio cgroups\n" ); }void __meminit pgdat_page_cgroup_init (struct pglist_data *pgdat)
mm/swap_state.c +2 -0
@@ -18,6 +18,7 @@#include <linux/pagevec.h> #include <linux/migrate.h> #include <linux/page_cgroup.h> +#include <linux/biotrack.h> #include <asm/pgtable.h>
@@ -307,6 +308,7 @@ struct page *read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask,*/ __set_page_locked (new_page);SetPageSwapBacked (new_page); +blkio_cgroup_set_owner (new_page, current->mm); err =add_to_swap_cache (new_page, entry, gfp_mask &GFP_KERNEL );if (likely (!err)) {/*
Signature · 1 line
1.6.0.6
Re: [PATCH 18/23] io-controller: blkio_cgroup patches from Ryo to track async bios.
Quoted 8 lines · Vivek Goyal
o
blkio_cgrouppatches from Ryo to track async bios.o This functionality is used to determine the group of async IO from page instead of context of submitting task.
Signed-off-by Hirokazu Takahashi <[email protected]>Signed-off-by Ryo Tsuruta <[email protected]>Signed-off-by Vivek Goyal <[email protected]>
This seems to be the most complex part of the code so far, but I see why this code is necessary.
Re: [PATCH 18/23] io-controller: blkio_cgroup patches from Ryo to track async bios.
Quoted 6 lines · Rik van Riel
Quoted 5 lines · Vivek Goyal
o
blkio_cgrouppatches from Ryo to track async bios.o This functionality is used to determine the group of async IO from page instead of context of submitting task.
Quoted 7 lines
Quoted 3 lines
Signed-off-by Hirokazu Takahashi <[email protected]>Signed-off-by Ryo Tsuruta <[email protected]>Signed-off-by Vivek Goyal <[email protected]>This seems to be the most complex part of the code so far, but I see why this code is necessary.
Hi Rik,
Thanks for reviewing the patches. I wanted to have better understanding of where all does it help to associate a bio to the group of process who created/owned the page. Hence few thoughts.
When a bio is submitted to IO scheduler, it needs to determine the group bio belongs to and group which should be charged to. There seem to be two methods.
- Attribute the bio to cgroup submitting process belongs to.
- For async requests, track the original owner hence cgroup of the page and charge that group for the bio.
One can think of pros/cons of both the approaches.
- The primary use case of tracking async context seems be that if a process T1 in group G1 mmaps a big file and then another process T2 in group G2, asks for memory and triggers reclaim and generates writes of the file pages mapped by T1, then these writes should not be charged to T2, hence blkio_cgroup pages.
But the flip side of this might be that group G2 is a low weight group and probably too busy also right now, which will delay the write out and possibly T2 will wait longer for memory to be allocated.
- At one point of time Andrew mentioned that buffered writes are generally a big problem and one needs to map these to owner's group. Though I am not very sure what specific problem he was referring to. Can we attribute buffered writes to pdflush threads and move all pdflush threads in a cgroup to limit system wide write out activity?
- Somebody also gave an example where there is a memory hogging process and possibly pushes out some processes to swap. It does not sound fair to charge those proccess for that swap writeout. These processes never requested swap IO.
- If there are multiple buffered writers in the system, then those writers can also be forced to writeout some pages to disk before they are allowed to dirty more pages. As per the page cache design, any writer can pick any inode and start writing out pages. So it can happen a weight group task is writting out pages dirtied by a lower weight group task. If, async bio is mapped to owner's group, it might happen that higher weight group task might be made to sleep on lower weight group task because request descriptors are all consumed up.
It looks like there does not seem to be a clean way which covers all the cases without issues. I am just trying to think, what is a simple way which covers most of the cases. Can we just stick to using submitting task context to determine a bio's group (as cfq does). Which can result in following.
- Less code and reduced complexity.
- Buffered writes will be charged to pdflush and its group. If one wish to limit buffered write activity for pdflush, one can move all the pdflush threads into a group and assign desired weight. Writes submitted in process context will continue to be charged to that process irrespective of the fact who dirtied that page.
- swap activity will be charged to kswapd and its group. If swap writes are coming from process context, it gets charged to process and its group.
- If one is worried about the case of one process being charged for write out of file mapped by another process during reclaim, then we can probably make use of memory controller and mount memory controller and io controller together on same hierarchy. I am told that with memory controller, group's memory will be reclaimed by the process requesting more memory. If that's the case, then IO will automatically be charged to right group if we use submitting task context.
I just wanted to bring this point forward for more discussions to know what is the right thing to do? Use bio tracking or not.
Ryo, any thoughts on this?
Thanks
Vivek
Quoted 1 line
Acked-by Rik van Riel <[email protected]>
Re: [PATCH 18/23] io-controller: blkio_cgroup patches from Ryo to track async bios.
Quoted 67 lines · Vivek Goyal
Quoted 6 lines · Rik van Riel
Quoted 5 lines · Vivek Goyal
o
blkio_cgrouppatches from Ryo to track async bios.o This functionality is used to determine the group of async IO from page instead of context of submitting task.
Quoted 7 lines
Quoted 3 lines
Signed-off-by Hirokazu Takahashi <[email protected]>Signed-off-by Ryo Tsuruta <[email protected]>Signed-off-by Vivek Goyal <[email protected]>This seems to be the most complex part of the code so far, but I see why this code is necessary.
Hi Rik,
Thanks for reviewing the patches. I wanted to have better understanding of where all does it help to associate a bio to the group of process who created/owned the page. Hence few thoughts.
When a bio is submitted to IO scheduler, it needs to determine the group bio belongs to and group which should be charged to. There seem to be two methods.
- Attribute the bio to cgroup submitting process belongs to.
- For async requests, track the original owner hence cgroup of the page and charge that group for the bio.One can think of pros/cons of both the approaches.
- The primary use case of tracking async context seems be that if a process T1 in group G1 mmaps a big file and then another process T2 in group G2, asks for memory and triggers reclaim and generates writes of the file pages mapped by T1, then these writes should not be charged to T2, hence
blkio_cgrouppages.But the flip side of this might be that group G2 is a low weight group and probably too busy also right now, which will delay the write out and possibly T2 will wait longer for memory to be allocated.
- At one point of time Andrew mentioned that buffered writes are generally a big problem and one needs to map these to owner's group. Though I am not very sure what specific problem he was referring to. Can we attribute buffered writes to pdflush threads and move all pdflush threads in a cgroup to limit system wide write out activity?
- Somebody also gave an example where there is a memory hogging process and possibly pushes out some processes to swap. It does not sound fair to charge those proccess for that swap writeout. These processes never requested swap IO.
- If there are multiple buffered writers in the system, then those writers can also be forced to writeout some pages to disk before they are allowed to dirty more pages. As per the page cache design, any writer can pick any inode and start writing out pages. So it can happen a weight group task is writting out pages dirtied by a lower weight group task. If, async bio is mapped to owner's group, it might happen that higher weight group task might be made to sleep on lower weight group task because request descriptors are all consumed up.
It looks like there does not seem to be a clean way which covers all the cases without issues. I am just trying to think, what is a simple way which covers most of the cases. Can we just stick to using submitting task context to determine a bio's group (as cfq does). Which can result in following.
Quoted 7 lines
- Less code and reduced complexity.
- Buffered writes will be charged to pdflush and its group. If one wish to limit buffered write activity for pdflush, one can move all the pdflush threads into a group and assign desired weight. Writes submitted in process context will continue to be charged to that process irrespective of the fact who dirtied that page.
What if we wanted to control buffered write activity per group? If a group keeps dirtying pages, we wouldn't want it to dominate the disk IO capacity at the expense of other cgroups (by dominating the writes sent down by pdflush).
Quoted 23 lines
- swap activity will be charged to kswapd and its group. If swap writes are coming from process context, it gets charged to process and its group.
- If one is worried about the case of one process being charged for write out of file mapped by another process during reclaim, then we can probably make use of memory controller and mount memory controller and io controller together on same hierarchy. I am told that with memory controller, group's memory will be reclaimed by the process requesting more memory. If that's the case, then IO will automatically be charged to right group if we use submitting task context.
I just wanted to bring this point forward for more discussions to know what is the right thing to do? Use bio tracking or not.
Ryo, any thoughts on this?
Thanks
VivekQuoted 1 line
Acked-by Rik van Riel <[email protected]>
Re: [PATCH 18/23] io-controller: blkio_cgroup patches from Ryo to track async bios.
Hi,
Quoted 22 lines
Quoted 22 lines
Hi Rik,
Thanks for reviewing the patches. I wanted to have better understanding of where all does it help to associate a bio to the group of process who created/owned the page. Hence few thoughts.
When a bio is submitted to IO scheduler, it needs to determine the group bio belongs to and group which should be charged to. There seem to be two methods.
- Attribute the bio to cgroup submitting process belongs to.
- For async requests, track the original owner hence cgroup of the page and charge that group for the bio.One can think of pros/cons of both the approaches.
- The primary use case of tracking async context seems be that if a process T1 in group G1 mmaps a big file and then another process T2 in group G2, asks for memory and triggers reclaim and generates writes of the file pages mapped by T1, then these writes should not be charged to T2, hence
blkio_cgrouppages.
Quoted 3 lines
Quoted 3 lines
But the flip side of this might be that group G2 is a low weight group and probably too busy also right now, which will delay the write out and possibly T2 will wait longer for memory to be allocated.
In order to avoid this wait, dm-ioband issues IO which has a page with PG_Reclaim as early as possible.
Quoted 5 lines
Quoted 5 lines
- At one point of time Andrew mentioned that buffered writes are generally a big problem and one needs to map these to owner's group. Though I am not very sure what specific problem he was referring to. Can we attribute buffered writes to pdflush threads and move all pdflush threads in a cgroup to limit system wide write out activity?
I think that buffered writes also should be controlled per cgroup as well as synchronous writes.
Quoted 4 lines
Quoted 4 lines
- Somebody also gave an example where there is a memory hogging process and possibly pushes out some processes to swap. It does not sound fair to charge those proccess for that swap writeout. These processes never requested swap IO.
I think that swap writeouts should be charged to the memory hogging process, because the process consumes more resources and it should get a penalty.
Quoted 8 lines
Quoted 8 lines
- If there are multiple buffered writers in the system, then those writers can also be forced to writeout some pages to disk before they are allowed to dirty more pages. As per the page cache design, any writer can pick any inode and start writing out pages. So it can happen a weight group task is writting out pages dirtied by a lower weight group task. If, async bio is mapped to owner's group, it might happen that higher weight group task might be made to sleep on lower weight group task because request descriptors are all consumed up.
As mentioned above, in dm-ioband, the bio is charged to the page owner and issued immediately.
Quoted 14 lines
Quoted 6 lines
It looks like there does not seem to be a clean way which covers all the cases without issues. I am just trying to think, what is a simple way which covers most of the cases. Can we just stick to using submitting task context to determine a bio's group (as cfq does). Which can result in following.
Quoted 7 lines
- Less code and reduced complexity.
- Buffered writes will be charged to pdflush and its group. If one wish to limit buffered write activity for pdflush, one can move all the pdflush threads into a group and assign desired weight. Writes submitted in process context will continue to be charged to that process irrespective of the fact who dirtied that page.
Quoted 4 lines
What if we wanted to control buffered write activity per group? If a group keeps dirtying pages, we wouldn't want it to dominate the disk IO capacity at the expense of other cgroups (by dominating the writes sent down by pdflush).
Yes, I think that is true.
Quoted 12 lines
Quoted 12 lines
- swap activity will be charged to kswapd and its group. If swap writes are coming from process context, it gets charged to process and its group.
- If one is worried about the case of one process being charged for write out of file mapped by another process during reclaim, then we can probably make use of memory controller and mount memory controller and io controller together on same hierarchy. I am told that with memory controller, group's memory will be reclaimed by the process requesting more memory. If that's the case, then IO will automatically be charged to right group if we use submitting task context.
Quoted 2 lines
Quoted 2 lines
I just wanted to bring this point forward for more discussions to know what is the right thing to do? Use bio tracking or not.
Thanks for bringing it forward.
Thanks,
Ryo Tsuruta
Re: [PATCH 18/23] io-controller: blkio_cgroup patches from Ryo to track async bios.
Quoted 24 lines · Ryo Tsuruta
Hi,
Quoted 22 lines
Quoted 22 lines
Hi Rik,
Thanks for reviewing the patches. I wanted to have better understanding of where all does it help to associate a bio to the group of process who created/owned the page. Hence few thoughts.
When a bio is submitted to IO scheduler, it needs to determine the group bio belongs to and group which should be charged to. There seem to be two methods.
- Attribute the bio to cgroup submitting process belongs to.
- For async requests, track the original owner hence cgroup of the page and charge that group for the bio.One can think of pros/cons of both the approaches.
- The primary use case of tracking async context seems be that if a process T1 in group G1 mmaps a big file and then another process T2 in group G2, asks for memory and triggers reclaim and generates writes of the file pages mapped by T1, then these writes should not be charged to T2, hence
blkio_cgrouppages.
Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
But the flip side of this might be that group G2 is a low weight group and probably too busy also right now, which will delay the write out and possibly T2 will wait longer for memory to be allocated.
In order to avoid this wait, dm-ioband issues IO which has a page with
PG_Reclaimas early as possible.
So in above case IO is still charged to G2 but you keep a track if page is PG_Reclaim then releae the this bio before other bios queued up in the group?
Quoted 9 lines
Quoted 5 lines
Quoted 5 lines
- At one point of time Andrew mentioned that buffered writes are generally a big problem and one needs to map these to owner's group. Though I am not very sure what specific problem he was referring to. Can we attribute buffered writes to pdflush threads and move all pdflush threads in a cgroup to limit system wide write out activity?
I think that buffered writes also should be controlled per cgroup as well as synchronous writes.
But it is hard to achieve fairness for buffered writes becase we don't create complete parallel IO paths and not necessarily higher weight process dispatches more buffered writes to IO scheduler. (Due to page cache buffered write logic).
So in some cases we might see buffered write fairness and in other cases not. For example, run two dd processes in two groups doing buffered writes and it is hard to achieve fairness between these.
That's why the idea that if we can't ensure Buffered write vs Buffered write fairness in all the cases, then does it make sense to attribute buffered writes to pdflush and put pdflush threads into a separate group to limit system wide write out activity.
Quoted 9 lines
Quoted 4 lines
Quoted 4 lines
- Somebody also gave an example where there is a memory hogging process and possibly pushes out some processes to swap. It does not sound fair to charge those proccess for that swap writeout. These processes never requested swap IO.
I think that swap writeouts should be charged to the memory hogging process, because the process consumes more resources and it should get a penalty.
A process requesting memory gets IO penalty? IMHO, swapping is a kernel mechanism and kernel's way of providing extended RAM. If we want to solve the issue of memory hogging by a process then right way to solve is to use memory controller and not by charging the process for IO activity. Instead, proabably a more suitable way is to charge swap activity to root group (where by default all the kernel related activity goes).
Quoted 11 lines
Quoted 8 lines
Quoted 8 lines
- If there are multiple buffered writers in the system, then those writers can also be forced to writeout some pages to disk before they are allowed to dirty more pages. As per the page cache design, any writer can pick any inode and start writing out pages. So it can happen a weight group task is writting out pages dirtied by a lower weight group task. If, async bio is mapped to owner's group, it might happen that higher weight group task might be made to sleep on lower weight group task because request descriptors are all consumed up.
As mentioned above, in dm-ioband, the bio is charged to the page owner and issued immediately.
But you are doing it only for selected pages and not for all buffered writes?
Quoted 15 lines
Quoted 15 lines
Quoted 14 lines
It looks like there does not seem to be a clean way which covers all the cases without issues. I am just trying to think, what is a simple way which covers most of the cases. Can we just stick to using submitting task context to determine a bio's group (as cfq does). Which can result in following.
- Less code and reduced complexity.
- Buffered writes will be charged to pdflush and its group. If one wish to limit buffered write activity for pdflush, one can move all the pdflush threads into a group and assign desired weight. Writes submitted in process context will continue to be charged to that process irrespective of the fact who dirtied that page.
Quoted 7 lines
Quoted 4 lines
What if we wanted to control buffered write activity per group? If a group keeps dirtying pages, we wouldn't want it to dominate the disk IO capacity at the expense of other cgroups (by dominating the writes sent down by pdflush).
Yes, I think that is true.
But anyway we are not able to gurantee this isolation in all the cases. Again I go back to example of two dd threads doing buffered writes in two groups.
I don't mind keeping it. Just wanted to make sure that we agree and understand that keeping it does not mean that we get buffered write vs buffered write isolation/fairness in all the cases.
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
- swap activity will be charged to kswapd and its group. If swap writes are coming from process context, it gets charged to process and its group.
- If one is worried about the case of one process being charged for write out of file mapped by another process during reclaim, then we can probably make use of memory controller and mount memory controller and io controller together on same hierarchy. I am told that with memory controller, group's memory will be reclaimed by the process requesting more memory. If that's the case, then IO will automatically be charged to right group if we use submitting task context.
Quoted 5 lines
Quoted 2 lines
Quoted 2 lines
I just wanted to bring this point forward for more discussions to know what is the right thing to do? Use bio tracking or not.
Thanks for bringing it forward.
Thanks
Vivek
Re: [PATCH 18/23] io-controller: blkio_cgroup patches from Ryo to track async bios.
Quoted 7 lines · Vivek Goyal
Quoted 3 lines · Ryo Tsuruta
I think that swap writeouts should be charged to the memory hogging process, because the process consumes more resources and it should get a penalty.
A process requesting memory gets IO penalty?
There is no easy answer here.
On the one hand, you want to charge the process that uses the resources.
On the other hand, if a lower resource use / higher priority process tries to free up some of those resources, it should not have its IO requests penalized (and get slowed down) because of something the first process did...
Signature · 1 line
All rights reversed.
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH 18/23] io-controller: blkio_cgroup patches from Ryo to track async bios.
Quoted 7 lines · Vivek Goyal
Quoted 3 lines · Ryo Tsuruta
I think that swap writeouts should be charged to the memory hogging process, because the process consumes more resources and it should get a penalty.
A process requesting memory gets IO penalty?
There is no easy answer here.
On the one hand, you want to charge the process that uses the resources.
On the other hand, if a lower resource use / higher priority process tries to free up some of those resources, it should not have its IO requests penalized (and get slowed down) because of something the first process did...
Signature · 1 line
All rights reversed.
Re: [PATCH 18/23] io-controller: blkio_cgroup patches from Ryo to track async bios.
In reply to a message not in the archiveQuoted 47 lines · Vivek Goyal
Quoted 24 lines · Ryo Tsuruta
Hi,
Quoted 22 lines
Quoted 22 lines
Hi Rik,
Thanks for reviewing the patches. I wanted to have better understanding of where all does it help to associate a bio to the group of process who created/owned the page. Hence few thoughts.
When a bio is submitted to IO scheduler, it needs to determine the group bio belongs to and group which should be charged to. There seem to be two methods.
- Attribute the bio to cgroup submitting process belongs to.
- For async requests, track the original owner hence cgroup of the page áand charge that group for the bio.One can think of pros/cons of both the approaches.
- The primary use case of tracking async context seems be that if a áprocess T1 in group G1 mmaps a big file and then another process T2 in ágroup G2, asks for memory and triggers reclaim and generates writes of áthe file pages mapped by T1, then these writes should not be charged to áT2, hence
blkio_cgrouppages.Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
áBut the flip side of this might be that group G2 is a low weight group áand probably too busy also right now, which will delay the write out áand possibly T2 will wait longer for memory to be allocated.
In order to avoid this wait, dm-ioband issues IO which has a page with
PG_Reclaimas early as possible.So in above case IO is still charged to G2 but you keep a track if page is
PG_Reclaimthen releae the this bio before other bios queued up in the group?Quoted 9 lines
Quoted 5 lines
Quoted 5 lines
- At one point of time Andrew mentioned that buffered writes are generally a ábig problem and one needs to map these to owner's group. Though I am not ávery sure what specific problem he was referring to. Can we attribute ábuffered writes to pdflush threads and move all pdflush threads in a ácgroup to limit system wide write out activity?
I think that buffered writes also should be controlled per cgroup as well as synchronous writes.
Quoted 8 lines
But it is hard to achieve fairness for buffered writes becase we don't create complete parallel IO paths and not necessarily higher weight process dispatches more buffered writes to IO scheduler. (Due to page cache buffered write logic).
So in some cases we might see buffered write fairness and in other cases not. For example, run two dd processes in two groups doing buffered writes and it is hard to achieve fairness between these.
If something is broken, we don't necessarily have to break it further. Instead, we should be thinking about why its hard to achieve fairness with buffered write back. Is there a way to change the writeback path to send down a constant stream of IO, instead of sending down bursts?
Quoted 90 lines
That's why the idea that if we can't ensure Buffered write vs Buffered write fairness in all the cases, then does it make sense to attribute buffered writes to pdflush and put pdflush threads into a separate group to limit system wide write out activity.
Quoted 9 lines
Quoted 4 lines
Quoted 4 lines
- Somebody also gave an example where there is a memory hogging process and ápossibly pushes out some processes to swap. It does not sound fair to ácharge those proccess for that swap writeout. These processes never árequested swap IO.
I think that swap writeouts should be charged to the memory hogging process, because the process consumes more resources and it should get a penalty.
A process requesting memory gets IO penalty? IMHO, swapping is a kernel mechanism and kernel's way of providing extended RAM. If we want to solve the issue of memory hogging by a process then right way to solve is to use memory controller and not by charging the process for IO activity. Instead, proabably a more suitable way is to charge swap activity to root group (where by default all the kernel related activity goes).
Quoted 11 lines
Quoted 8 lines
Quoted 8 lines
- If there are multiple buffered writers in the system, then those writers ácan also be forced to writeout some pages to disk before they are áallowed to dirty more pages. As per the page cache design, any writer ácan pick any inode and start writing out pages. So it can happen a áweight group task is writting out pages dirtied by a lower weight group átask. If, async bio is mapped to owner's group, it might happen that áhigher weight group task might be made to sleep on lower weight group átask because request descriptors are all consumed up.
As mentioned above, in dm-ioband, the bio is charged to the page owner and issued immediately.
But you are doing it only for selected pages and not for all buffered writes?
Quoted 15 lines
Quoted 15 lines
Quoted 14 lines
It looks like there does not seem to be a clean way which covers all the cases without issues. I am just trying to think, what is a simple way which covers most of the cases. Can we just stick to using submitting task context to determine a bio's group (as cfq does). Which can result in following.
- Less code and reduced complexity.
- Buffered writes will be charged to pdflush and its group. If one wish to álimit buffered write activity for pdflush, one can move all the pdflush áthreads into a group and assign desired weight. Writes submitted in áprocess context will continue to be charged to that process irrespective áof the fact who dirtied that page.
Quoted 7 lines
Quoted 4 lines
What if we wanted to control buffered write activity per group? If a group keeps dirtying pages, we wouldn't want it to dominate the disk IO capacity at the expense of other cgroups (by dominating the writes sent down by pdflush).
Yes, I think that is true.
But anyway we are not able to gurantee this isolation in all the cases. Again I go back to example of two dd threads doing buffered writes in two groups.
I don't mind keeping it. Just wanted to make sure that we agree and understand that keeping it does not mean that we get buffered write vs buffered write isolation/fairness in all the cases.
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
- swap activity will be charged to kswapd and its group. If swap writes áare coming from process context, it gets charged to process and its ágroup.
- If one is worried about the case of one process being charged for write áout of file mapped by another process during reclaim, then we can áprobably make use of memory controller and mount memory controller and áio controller together on same hierarchy. I am told that with memory ácontroller, group's memory will be reclaimed by the process requesting ámore memory. If that's the case, then IO will automatically be charged áto right group if we use submitting task context.
Quoted 5 lines
Quoted 2 lines
Quoted 2 lines
I just wanted to bring this point forward for more discussions to know what is the right thing to do? Use bio tracking or not.
Thanks for bringing it forward.
Thanks
Vivek
Re: [PATCH 18/23] io-controller: blkio_cgroup patches from Ryo to track async bios.
In reply to a message not in the archiveQuoted 10 lines · On Tue, 1 Sep 2009 10:11:42 -0400 Vivek Goyal <[email protected]> wrote:
Quoted 9 lines
Quoted 4 lines
Quoted 4 lines
- Somebody also gave an example where there is a memory hogging process and possibly pushes out some processes to swap. It does not sound fair to charge those proccess for that swap writeout. These processes never requested swap IO.
I think that swap writeouts should be charged to the memory hogging process, because the process consumes more resources and it should get a penalty.
Quoted 7 lines
A process requesting memory gets IO penalty? IMHO, swapping is a kernel mechanism and kernel's way of providing extended RAM. If we want to solve the issue of memory hogging by a process then right way to solve is to use memory controller and not by charging the process for IO activity. Instead, proabably a more suitable way is to charge swap activity to root group (where by default all the kernel related activity goes).
I agree. It't memcg's job.
(Support dirty_ratio in memcg is necessary, I think)
background-write-out-to-swap-for-memory-shortage should be handled as kernel I/O. If swap-out-by-memcg bacause of its limit is a problem, dirty_ratio for memcg should be implemetned.
Thanks,
-Kame
_______________________________________________
Containers mailing list
[email protected]
https://lists.linux-foundation.org/mailman/listinfo/containers
Re: [PATCH 18/23] io-controller: blkio_cgroup patches from Ryo to track async bios.
In reply to a message not in the archiveHi Vivek,
Quoted 10 lines
Quoted 10 lines
Quoted 9 lines
Quoted 9 lines
- The primary use case of tracking async context seems be that if a process T1 in group G1 mmaps a big file and then another process T2 in group G2, asks for memory and triggers reclaim and generates writes of the file pages mapped by T1, then these writes should not be charged to T2, hence
blkio_cgrouppages.But the flip side of this might be that group G2 is a low weight group and probably too busy also right now, which will delay the write out and possibly T2 will wait longer for memory to be allocated.
Quoted 7 lines
Quoted 3 lines
In order to avoid this wait, dm-ioband issues IO which has a page with
PG_Reclaimas early as possible.So in above case IO is still charged to G2 but you keep a track if page is
PG_Reclaimthen releae the this bio before other bios queued up in the group?
Yes, the bio with PG_Reclaim page is given priority over the other bios.
Quoted 15 lines
Quoted 9 lines
Quoted 5 lines
Quoted 5 lines
- At one point of time Andrew mentioned that buffered writes are generally a big problem and one needs to map these to owner's group. Though I am not very sure what specific problem he was referring to. Can we attribute buffered writes to pdflush threads and move all pdflush threads in a cgroup to limit system wide write out activity?
I think that buffered writes also should be controlled per cgroup as well as synchronous writes.
But it is hard to achieve fairness for buffered writes becase we don't create complete parallel IO paths and not necessarily higher weight process dispatches more buffered writes to IO scheduler. (Due to page cache buffered write logic).
Quoted 8 lines
So in some cases we might see buffered write fairness and in other cases not. For example, run two dd processes in two groups doing buffered writes and it is hard to achieve fairness between these.
That's why the idea that if we can't ensure Buffered write vs Buffered write fairness in all the cases, then does it make sense to attribute buffered writes to pdflush and put pdflush threads into a separate group to limit system wide write out activity.
If all buffered writes are treated as system wide activities, it does not mean that bandwidth is being controlled. It is true that pdflush doesn't do I/O according to weight, but bandwidth (including for bufferd writes) should be reserved for each cgroup.
Quoted 10 lines
Quoted 9 lines
Quoted 4 lines
Quoted 4 lines
- Somebody also gave an example where there is a memory hogging process and possibly pushes out some processes to swap. It does not sound fair to charge those proccess for that swap writeout. These processes never requested swap IO.
I think that swap writeouts should be charged to the memory hogging process, because the process consumes more resources and it should get a penalty.
Quoted 6 lines
A process requesting memory gets IO penalty? IMHO, swapping is a kernel mechanism and kernel's way of providing extended RAM. If we want to solve the issue of memory hogging by a process then right way to solve is to use memory controller and not by charging the process for IO activity. Instead, proabably a more suitable way is to charge swap activity to root group (where by default all the kernel related activity goes).
No. In the current blkio-cgroup, a process which uses a large amount of memory gets penalty, not a memory requester.
As you wrote, using both io-controller and memory controller are required to prevent swap-out caused by memory consumption on another cgroup.
Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
Quoted 8 lines
- If there are multiple buffered writers in the system, then those writers can also be forced to writeout some pages to disk before they are allowed to dirty more pages. As per the page cache design, any writer can pick any inode and start writing out pages. So it can happen a weight group task is writting out pages dirtied by a lower weight group task. If, async bio is mapped to owner's group, it might happen that higher weight group task might be made to sleep on lower weight group task because request descriptors are all consumed up.
Quoted 5 lines
Quoted 2 lines
As mentioned above, in dm-ioband, the bio is charged to the page owner and issued immediately.
But you are doing it only for selected pages and not for all buffered writes?
I'm sorry, I wrote wrong on the previous mail, IO for writing out page-cache pages is not issued immediately, it is throttled by dm-ioband.
Anyway, there is a case where a higher weight group task is made to sleep, but if we reserve the memory for each cgroup by memory controller in advance, we can avoid the task put to sleep.
Thanks,
Ryo Tsuruta
Re: [PATCH 18/23] io-controller: blkio_cgroup patches from Ryo to track async bios.
In reply to Vivek GoyalQuoted 47 lines · Vivek Goyal
Quoted 24 lines · Ryo Tsuruta
Hi,
Quoted 22 lines
Quoted 22 lines
Hi Rik,
Thanks for reviewing the patches. I wanted to have better understanding of where all does it help to associate a bio to the group of process who created/owned the page. Hence few thoughts.
When a bio is submitted to IO scheduler, it needs to determine the group bio belongs to and group which should be charged to. There seem to be two methods.
- Attribute the bio to cgroup submitting process belongs to.
- For async requests, track the original owner hence cgroup of the page áand charge that group for the bio.One can think of pros/cons of both the approaches.
- The primary use case of tracking async context seems be that if a áprocess T1 in group G1 mmaps a big file and then another process T2 in ágroup G2, asks for memory and triggers reclaim and generates writes of áthe file pages mapped by T1, then these writes should not be charged to áT2, hence
blkio_cgrouppages.Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
áBut the flip side of this might be that group G2 is a low weight group áand probably too busy also right now, which will delay the write out áand possibly T2 will wait longer for memory to be allocated.
In order to avoid this wait, dm-ioband issues IO which has a page with
PG_Reclaimas early as possible.So in above case IO is still charged to G2 but you keep a track if page is
PG_Reclaimthen releae the this bio before other bios queued up in the group?Quoted 9 lines
Quoted 5 lines
Quoted 5 lines
- At one point of time Andrew mentioned that buffered writes are generally a ábig problem and one needs to map these to owner's group. Though I am not ávery sure what specific problem he was referring to. Can we attribute ábuffered writes to pdflush threads and move all pdflush threads in a ácgroup to limit system wide write out activity?
I think that buffered writes also should be controlled per cgroup as well as synchronous writes.
Quoted 8 lines
But it is hard to achieve fairness for buffered writes becase we don't create complete parallel IO paths and not necessarily higher weight process dispatches more buffered writes to IO scheduler. (Due to page cache buffered write logic).
So in some cases we might see buffered write fairness and in other cases not. For example, run two dd processes in two groups doing buffered writes and it is hard to achieve fairness between these.
If something is broken, we don't necessarily have to break it further. Instead, we should be thinking about why its hard to achieve fairness with buffered write back. Is there a way to change the writeback path to send down a constant stream of IO, instead of sending down bursts?
Quoted 90 lines
That's why the idea that if we can't ensure Buffered write vs Buffered write fairness in all the cases, then does it make sense to attribute buffered writes to pdflush and put pdflush threads into a separate group to limit system wide write out activity.
Quoted 9 lines
Quoted 4 lines
Quoted 4 lines
- Somebody also gave an example where there is a memory hogging process and ápossibly pushes out some processes to swap. It does not sound fair to ácharge those proccess for that swap writeout. These processes never árequested swap IO.
I think that swap writeouts should be charged to the memory hogging process, because the process consumes more resources and it should get a penalty.
A process requesting memory gets IO penalty? IMHO, swapping is a kernel mechanism and kernel's way of providing extended RAM. If we want to solve the issue of memory hogging by a process then right way to solve is to use memory controller and not by charging the process for IO activity. Instead, proabably a more suitable way is to charge swap activity to root group (where by default all the kernel related activity goes).
Quoted 11 lines
Quoted 8 lines
Quoted 8 lines
- If there are multiple buffered writers in the system, then those writers ácan also be forced to writeout some pages to disk before they are áallowed to dirty more pages. As per the page cache design, any writer ácan pick any inode and start writing out pages. So it can happen a áweight group task is writting out pages dirtied by a lower weight group átask. If, async bio is mapped to owner's group, it might happen that áhigher weight group task might be made to sleep on lower weight group átask because request descriptors are all consumed up.
As mentioned above, in dm-ioband, the bio is charged to the page owner and issued immediately.
But you are doing it only for selected pages and not for all buffered writes?
Quoted 15 lines
Quoted 15 lines
Quoted 14 lines
It looks like there does not seem to be a clean way which covers all the cases without issues. I am just trying to think, what is a simple way which covers most of the cases. Can we just stick to using submitting task context to determine a bio's group (as cfq does). Which can result in following.
- Less code and reduced complexity.
- Buffered writes will be charged to pdflush and its group. If one wish to álimit buffered write activity for pdflush, one can move all the pdflush áthreads into a group and assign desired weight. Writes submitted in áprocess context will continue to be charged to that process irrespective áof the fact who dirtied that page.
Quoted 7 lines
Quoted 4 lines
What if we wanted to control buffered write activity per group? If a group keeps dirtying pages, we wouldn't want it to dominate the disk IO capacity at the expense of other cgroups (by dominating the writes sent down by pdflush).
Yes, I think that is true.
But anyway we are not able to gurantee this isolation in all the cases. Again I go back to example of two dd threads doing buffered writes in two groups.
I don't mind keeping it. Just wanted to make sure that we agree and understand that keeping it does not mean that we get buffered write vs buffered write isolation/fairness in all the cases.
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
- swap activity will be charged to kswapd and its group. If swap writes áare coming from process context, it gets charged to process and its ágroup.
- If one is worried about the case of one process being charged for write áout of file mapped by another process during reclaim, then we can áprobably make use of memory controller and mount memory controller and áio controller together on same hierarchy. I am told that with memory ácontroller, group's memory will be reclaimed by the process requesting ámore memory. If that's the case, then IO will automatically be charged áto right group if we use submitting task context.
Quoted 5 lines
Quoted 2 lines
Quoted 2 lines
I just wanted to bring this point forward for more discussions to know what is the right thing to do? Use bio tracking or not.
Thanks for bringing it forward.
Thanks
Vivek
Re: [PATCH 18/23] io-controller: blkio_cgroup patches from Ryo to track async bios.
In reply to Vivek GoyalQuoted 10 lines · On Tue, 1 Sep 2009 10:11:42 -0400 Vivek Goyal <[email protected]> wrote:
Quoted 9 lines
Quoted 4 lines
Quoted 4 lines
- Somebody also gave an example where there is a memory hogging process and possibly pushes out some processes to swap. It does not sound fair to charge those proccess for that swap writeout. These processes never requested swap IO.
I think that swap writeouts should be charged to the memory hogging process, because the process consumes more resources and it should get a penalty.
Quoted 7 lines
A process requesting memory gets IO penalty? IMHO, swapping is a kernel mechanism and kernel's way of providing extended RAM. If we want to solve the issue of memory hogging by a process then right way to solve is to use memory controller and not by charging the process for IO activity. Instead, proabably a more suitable way is to charge swap activity to root group (where by default all the kernel related activity goes).
I agree. It't memcg's job.
(Support dirty_ratio in memcg is necessary, I think)
background-write-out-to-swap-for-memory-shortage should be handled as kernel I/O. If swap-out-by-memcg bacause of its limit is a problem, dirty_ratio for memcg should be implemetned.
Thanks,
-Kame
Re: [PATCH 18/23] io-controller: blkio_cgroup patches from Ryo to track async bios.
Quoted 20 lines · KAMEZAWA Hiroyuki
Quoted 10 lines · On Tue, 1 Sep 2009 10:11:42 -0400 Vivek Goyal <[email protected]> wrote:
Quoted 9 lines
Quoted 4 lines
Quoted 4 lines
- Somebody also gave an example where there is a memory hogging process and possibly pushes out some processes to swap. It does not sound fair to charge those proccess for that swap writeout. These processes never requested swap IO.
I think that swap writeouts should be charged to the memory hogging process, because the process consumes more resources and it should get a penalty.
Quoted 7 lines
A process requesting memory gets IO penalty? IMHO, swapping is a kernel mechanism and kernel's way of providing extended RAM. If we want to solve the issue of memory hogging by a process then right way to solve is to use memory controller and not by charging the process for IO activity. Instead, proabably a more suitable way is to charge swap activity to root group (where by default all the kernel related activity goes).
Quoted 6 lines
I agree. It't memcg's job.
(Supportdirty_ratioin memcg is necessary, I think)background-write-out-to-swap-for-memory-shortage should be handled as kernel I/O. If swap-out-by-memcg bacause of its limit is a problem,
dirty_ratiofor memcg should be implemetned.
I tend to agree, looks like dirty_ratio will become important along with overcommit support in the future.
Balbir Singh.
A message that is not in the archive (sent off-list or not yet received): 20090902095912.cdf8a55e.kamezawa.hiroyu-+CUm20s59erQFUHtdCDX3A@public.gmane.org
Re: [PATCH 18/23] io-controller: blkio_cgroup patches from Ryo to track async bios.
Quoted 20 lines · KAMEZAWA Hiroyuki
Quoted 10 lines · On Tue, 1 Sep 2009 10:11:42 -0400 Vivek Goyal…
Quoted 9 lines
Quoted 4 lines
Quoted 4 lines
- Somebody also gave an example where there is a memory hogging process and possibly pushes out some processes to swap. It does not sound fair to charge those proccess for that swap writeout. These processes never requested swap IO.
I think that swap writeouts should be charged to the memory hogging process, because the process consumes more resources and it should get a penalty.
Quoted 7 lines
A process requesting memory gets IO penalty? IMHO, swapping is a kernel mechanism and kernel's way of providing extended RAM. If we want to solve the issue of memory hogging by a process then right way to solve is to use memory controller and not by charging the process for IO activity. Instead, proabably a more suitable way is to charge swap activity to root group (where by default all the kernel related activity goes).
Quoted 6 lines
I agree. It't memcg's job.
(Supportdirty_ratioin memcg is necessary, I think)background-write-out-to-swap-for-memory-shortage should be handled as kernel I/O. If swap-out-by-memcg bacause of its limit is a problem,
dirty_ratiofor memcg should be implemetned.
I tend to agree, looks like dirty_ratio will become important along with overcommit support in the future.
Balbir Singh.
Re: [PATCH 18/23] io-controller: blkio_cgroup patches from Ryo to track async bios.
In reply to Vivek GoyalHi Vivek,
Quoted 10 lines
Quoted 10 lines
Quoted 9 lines
Quoted 9 lines
- The primary use case of tracking async context seems be that if a process T1 in group G1 mmaps a big file and then another process T2 in group G2, asks for memory and triggers reclaim and generates writes of the file pages mapped by T1, then these writes should not be charged to T2, hence
blkio_cgrouppages.But the flip side of this might be that group G2 is a low weight group and probably too busy also right now, which will delay the write out and possibly T2 will wait longer for memory to be allocated.
Quoted 7 lines
Quoted 3 lines
In order to avoid this wait, dm-ioband issues IO which has a page with
PG_Reclaimas early as possible.So in above case IO is still charged to G2 but you keep a track if page is
PG_Reclaimthen releae the this bio before other bios queued up in the group?
Yes, the bio with PG_Reclaim page is given priority over the other bios.
Quoted 15 lines
Quoted 9 lines
Quoted 5 lines
Quoted 5 lines
- At one point of time Andrew mentioned that buffered writes are generally a big problem and one needs to map these to owner's group. Though I am not very sure what specific problem he was referring to. Can we attribute buffered writes to pdflush threads and move all pdflush threads in a cgroup to limit system wide write out activity?
I think that buffered writes also should be controlled per cgroup as well as synchronous writes.
But it is hard to achieve fairness for buffered writes becase we don't create complete parallel IO paths and not necessarily higher weight process dispatches more buffered writes to IO scheduler. (Due to page cache buffered write logic).
Quoted 8 lines
So in some cases we might see buffered write fairness and in other cases not. For example, run two dd processes in two groups doing buffered writes and it is hard to achieve fairness between these.
That's why the idea that if we can't ensure Buffered write vs Buffered write fairness in all the cases, then does it make sense to attribute buffered writes to pdflush and put pdflush threads into a separate group to limit system wide write out activity.
If all buffered writes are treated as system wide activities, it does not mean that bandwidth is being controlled. It is true that pdflush doesn't do I/O according to weight, but bandwidth (including for bufferd writes) should be reserved for each cgroup.
Quoted 10 lines
Quoted 9 lines
Quoted 4 lines
Quoted 4 lines
- Somebody also gave an example where there is a memory hogging process and possibly pushes out some processes to swap. It does not sound fair to charge those proccess for that swap writeout. These processes never requested swap IO.
I think that swap writeouts should be charged to the memory hogging process, because the process consumes more resources and it should get a penalty.
Quoted 6 lines
A process requesting memory gets IO penalty? IMHO, swapping is a kernel mechanism and kernel's way of providing extended RAM. If we want to solve the issue of memory hogging by a process then right way to solve is to use memory controller and not by charging the process for IO activity. Instead, proabably a more suitable way is to charge swap activity to root group (where by default all the kernel related activity goes).
No. In the current blkio-cgroup, a process which uses a large amount of memory gets penalty, not a memory requester.
As you wrote, using both io-controller and memory controller are required to prevent swap-out caused by memory consumption on another cgroup.
Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
Quoted 8 lines
- If there are multiple buffered writers in the system, then those writers can also be forced to writeout some pages to disk before they are allowed to dirty more pages. As per the page cache design, any writer can pick any inode and start writing out pages. So it can happen a weight group task is writting out pages dirtied by a lower weight group task. If, async bio is mapped to owner's group, it might happen that higher weight group task might be made to sleep on lower weight group task because request descriptors are all consumed up.
Quoted 5 lines
Quoted 2 lines
As mentioned above, in dm-ioband, the bio is charged to the page owner and issued immediately.
But you are doing it only for selected pages and not for all buffered writes?
I'm sorry, I wrote wrong on the previous mail, IO for writing out page-cache pages is not issued immediately, it is throttled by dm-ioband.
Anyway, there is a case where a higher weight group task is made to sleep, but if we reserve the memory for each cgroup by memory controller in advance, we can avoid the task put to sleep.
Thanks,
Ryo Tsuruta
Re: [PATCH 18/23] io-controller: blkio_cgroup patches from Ryo to track async bios.
Quoted 68 lines · Ryo Tsuruta
Hi Vivek,
Quoted 10 lines
Quoted 10 lines
Quoted 9 lines
Quoted 9 lines
- The primary use case of tracking async context seems be that if a process T1 in group G1 mmaps a big file and then another process T2 in group G2, asks for memory and triggers reclaim and generates writes of the file pages mapped by T1, then these writes should not be charged to T2, hence
blkio_cgrouppages.But the flip side of this might be that group G2 is a low weight group and probably too busy also right now, which will delay the write out and possibly T2 will wait longer for memory to be allocated.
Quoted 7 lines
Quoted 3 lines
In order to avoid this wait, dm-ioband issues IO which has a page with
PG_Reclaimas early as possible.So in above case IO is still charged to G2 but you keep a track if page is
PG_Reclaimthen releae the this bio before other bios queued up in the group?Yes, the bio with
PG_Reclaimpage is given priority over the other bios.Quoted 15 lines
Quoted 9 lines
Quoted 5 lines
Quoted 5 lines
- At one point of time Andrew mentioned that buffered writes are generally a big problem and one needs to map these to owner's group. Though I am not very sure what specific problem he was referring to. Can we attribute buffered writes to pdflush threads and move all pdflush threads in a cgroup to limit system wide write out activity?
I think that buffered writes also should be controlled per cgroup as well as synchronous writes.
But it is hard to achieve fairness for buffered writes becase we don't create complete parallel IO paths and not necessarily higher weight process dispatches more buffered writes to IO scheduler. (Due to page cache buffered write logic).
Quoted 8 lines
So in some cases we might see buffered write fairness and in other cases not. For example, run two dd processes in two groups doing buffered writes and it is hard to achieve fairness between these.
That's why the idea that if we can't ensure Buffered write vs Buffered write fairness in all the cases, then does it make sense to attribute buffered writes to pdflush and put pdflush threads into a separate group to limit system wide write out activity.
If all buffered writes are treated as system wide activities, it does not mean that bandwidth is being controlled. It is true that pdflush doesn't do I/O according to weight, but bandwidth (including for bufferd writes) should be reserved for each cgroup.
Quoted 10 lines
Quoted 9 lines
Quoted 4 lines
Quoted 4 lines
- Somebody also gave an example where there is a memory hogging process and possibly pushes out some processes to swap. It does not sound fair to charge those proccess for that swap writeout. These processes never requested swap IO.
I think that swap writeouts should be charged to the memory hogging process, because the process consumes more resources and it should get a penalty.
Quoted 6 lines
A process requesting memory gets IO penalty? IMHO, swapping is a kernel mechanism and kernel's way of providing extended RAM. If we want to solve the issue of memory hogging by a process then right way to solve is to use memory controller and not by charging the process for IO activity. Instead, proabably a more suitable way is to charge swap activity to root group (where by default all the kernel related activity goes).
Quoted 3 lines
No. In the current blkio-cgroup, a process which uses a large amount of memory gets penalty, not a memory requester.
At ioband level you just get to see bio and page. How do you decide wheter this bio is being issued by a process which is a memory hog?
In fact requester of memory could be anybody. It could be memory hog or a different process. So are you saying that you got a mechanism where you can detect that a process is memory hog and charge swap activity to it. IOW, if there are two processes A and B and assume A is the memory hog and then B requests for memory which triggers lot of swap IO, then you can charge all that IO to memory hog A?
Can you please point me to the relevant code in dm-ioband?
IMHO, to keep things simple, all swapping activity should be charged to root group and be considered as kernel activity and user space not be charged for that.
Thanks
Vivek
Quoted 29 lines
As you wrote, using both io-controller and memory controller are required to prevent swap-out caused by memory consumption on another cgroup.
Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
Quoted 8 lines
- If there are multiple buffered writers in the system, then those writers can also be forced to writeout some pages to disk before they are allowed to dirty more pages. As per the page cache design, any writer can pick any inode and start writing out pages. So it can happen a weight group task is writting out pages dirtied by a lower weight group task. If, async bio is mapped to owner's group, it might happen that higher weight group task might be made to sleep on lower weight group task because request descriptors are all consumed up.
Quoted 5 lines
Quoted 2 lines
As mentioned above, in dm-ioband, the bio is charged to the page owner and issued immediately.
But you are doing it only for selected pages and not for all buffered writes?
I'm sorry, I wrote wrong on the previous mail, IO for writing out page-cache pages is not issued immediately, it is throttled by dm-ioband.
Anyway, there is a case where a higher weight group task is made to sleep, but if we reserve the memory for each cgroup by memory controller in advance, we can avoid the task put to sleep.
Thanks,
Ryo Tsuruta
Re: [PATCH 18/23] io-controller: blkio_cgroup patches from Ryo to track async bios.
Hi Vivek,
Quoted 24 lines · Vivek Goyal
Quoted 17 lines
Quoted 10 lines
Quoted 9 lines
Quoted 4 lines
Quoted 4 lines
- Somebody also gave an example where there is a memory hogging process and possibly pushes out some processes to swap. It does not sound fair to charge those proccess for that swap writeout. These processes never requested swap IO.
I think that swap writeouts should be charged to the memory hogging process, because the process consumes more resources and it should get a penalty.
Quoted 6 lines
A process requesting memory gets IO penalty? IMHO, swapping is a kernel mechanism and kernel's way of providing extended RAM. If we want to solve the issue of memory hogging by a process then right way to solve is to use memory controller and not by charging the process for IO activity. Instead, proabably a more suitable way is to charge swap activity to root group (where by default all the kernel related activity goes).
Quoted 3 lines
No. In the current blkio-cgroup, a process which uses a large amount of memory gets penalty, not a memory requester.
At ioband level you just get to see bio and page. How do you decide wheter this bio is being issued by a process which is a memory hog?
Quoted 6 lines
In fact requester of memory could be anybody. It could be memory hog or a different process. So are you saying that you got a mechanism where you can detect that a process is memory hog and charge swap activity to it. IOW, if there are two processes A and B and assume A is the memory hog and then B requests for memory which triggers lot of swap IO, then you can charge all that IO to memory hog A?
When an annoymou page is allocated, blkio-cgroup sets an ID to the page. And then when the page is going to swap out, dm-ioband can know who the owner of the page is by retrieving ID from the page.
In the above case, since the pages of the process A are swapped-out, dm-ioband charges swap IO to the process A.
Quoted 5 lines
Can you please point me to the relevant code in dm-ioband?
IMHO, to keep things simple, all swapping activity should be charged to root group and be considered as kernel activity and user space not be charged for that.
Thanks,
Ryo Tsuruta
Re: [PATCH 18/23] io-controller: blkio_cgroup patches from Ryo to track async bios.
Quoted 34 lines · Ryo Tsuruta
Hi Vivek,
Quoted 24 lines · Vivek Goyal
Quoted 17 lines
Quoted 10 lines
Quoted 9 lines
Quoted 4 lines
Quoted 4 lines
- Somebody also gave an example where there is a memory hogging process and possibly pushes out some processes to swap. It does not sound fair to charge those proccess for that swap writeout. These processes never requested swap IO.
I think that swap writeouts should be charged to the memory hogging process, because the process consumes more resources and it should get a penalty.
Quoted 6 lines
A process requesting memory gets IO penalty? IMHO, swapping is a kernel mechanism and kernel's way of providing extended RAM. If we want to solve the issue of memory hogging by a process then right way to solve is to use memory controller and not by charging the process for IO activity. Instead, proabably a more suitable way is to charge swap activity to root group (where by default all the kernel related activity goes).
Quoted 3 lines
No. In the current blkio-cgroup, a process which uses a large amount of memory gets penalty, not a memory requester.
At ioband level you just get to see bio and page. How do you decide wheter this bio is being issued by a process which is a memory hog?
Quoted 6 lines
In fact requester of memory could be anybody. It could be memory hog or a different process. So are you saying that you got a mechanism where you can detect that a process is memory hog and charge swap activity to it. IOW, if there are two processes A and B and assume A is the memory hog and then B requests for memory which triggers lot of swap IO, then you can charge all that IO to memory hog A?
Quoted 7 lines
When an annoymou page is allocated, blkio-cgroup sets an ID to the page. And then when the page is going to swap out, dm-ioband can know who the owner of the page is by retrieving ID from the page.
In the above case, since the pages of the process A are swapped-out, dm-ioband charges swap IO to the process A.
But this does not mean that in all cases memory hog is being charged for swap IO, as you have said. So if a process A has done some anonymous page allocations and later a memory hog B comes in and forces swap out of A, you will charge A for swap activity which does not seem fair as B is memory hog here?
Thanks
Vivek
Quoted 8 lines
Quoted 5 lines
Can you please point me to the relevant code in dm-ioband?
IMHO, to keep things simple, all swapping activity should be charged to root group and be considered as kernel activity and user space not be charged for that.
Thanks,
Ryo Tsuruta
Re: [PATCH 18/23] io-controller: blkio_cgroup patches from Ryo to track async bios.
Hi Vivek,
Quoted 18 lines · Vivek Goyal
Quoted 10 lines
Quoted 9 lines
At ioband level you just get to see bio and page. How do you decide wheter this bio is being issued by a process which is a memory hog?
In fact requester of memory could be anybody. It could be memory hog or a different process. So are you saying that you got a mechanism where you can detect that a process is memory hog and charge swap activity to it. IOW, if there are two processes A and B and assume A is the memory hog and then B requests for memory which triggers lot of swap IO, then you can charge all that IO to memory hog A?
Quoted 7 lines
When an annoymou page is allocated, blkio-cgroup sets an ID to the page. And then when the page is going to swap out, dm-ioband can know who the owner of the page is by retrieving ID from the page.
In the above case, since the pages of the process A are swapped-out, dm-ioband charges swap IO to the process A.
Quoted 5 lines
But this does not mean that in all cases memory hog is being charged for swap IO, as you have said. So if a process A has done some anonymous page allocations and later a memory hog B comes in and forces swap out of A, you will charge A for swap activity which does not seem fair as B is memory hog here?
I think this charging policy is not bad, but I think I can understand you think it's not fair. Do you think it's fair if all IO is carged to B?
We should use both io and memory controller together, as you wrote,
Thanks,
Ryo Tsuruta
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH 18/23] io-controller: blkio_cgroup patches from Ryo to track async bios.
Hi Vivek,
Quoted 18 lines · Vivek Goyal
Quoted 10 lines
Quoted 9 lines
At ioband level you just get to see bio and page. How do you decide wheter this bio is being issued by a process which is a memory hog?
In fact requester of memory could be anybody. It could be memory hog or a different process. So are you saying that you got a mechanism where you can detect that a process is memory hog and charge swap activity to it. IOW, if there are two processes A and B and assume A is the memory hog and then B requests for memory which triggers lot of swap IO, then you can charge all that IO to memory hog A?
Quoted 7 lines
When an annoymou page is allocated, blkio-cgroup sets an ID to the page. And then when the page is going to swap out, dm-ioband can know who the owner of the page is by retrieving ID from the page.
In the above case, since the pages of the process A are swapped-out, dm-ioband charges swap IO to the process A.
Quoted 5 lines
But this does not mean that in all cases memory hog is being charged for swap IO, as you have said. So if a process A has done some anonymous page allocations and later a memory hog B comes in and forces swap out of A, you will charge A for swap activity which does not seem fair as B is memory hog here?
I think this charging policy is not bad, but I think I can understand you think it's not fair. Do you think it's fair if all IO is carged to B?
We should use both io and memory controller together, as you wrote,
Thanks,
Ryo Tsuruta
A message that is not in the archive (sent off-list or not yet received): 20090903.112423.226782505.ryov-jCdQPDEk3idL9jVzuh4AOg@public.gmane.org
Re: [PATCH 18/23] io-controller: blkio_cgroup patches from Ryo to track async bios.
Quoted 34 lines · Ryo Tsuruta
Hi Vivek,
Quoted 24 lines · Vivek Goyal
Quoted 17 lines
Quoted 10 lines
Quoted 9 lines
Quoted 4 lines
Quoted 4 lines
- Somebody also gave an example where there is a memory hogging process and possibly pushes out some processes to swap. It does not sound fair to charge those proccess for that swap writeout. These processes never requested swap IO.
I think that swap writeouts should be charged to the memory hogging process, because the process consumes more resources and it should get a penalty.
Quoted 6 lines
A process requesting memory gets IO penalty? IMHO, swapping is a kernel mechanism and kernel's way of providing extended RAM. If we want to solve the issue of memory hogging by a process then right way to solve is to use memory controller and not by charging the process for IO activity. Instead, proabably a more suitable way is to charge swap activity to root group (where by default all the kernel related activity goes).
Quoted 3 lines
No. In the current blkio-cgroup, a process which uses a large amount of memory gets penalty, not a memory requester.
At ioband level you just get to see bio and page. How do you decide wheter this bio is being issued by a process which is a memory hog?
Quoted 6 lines
In fact requester of memory could be anybody. It could be memory hog or a different process. So are you saying that you got a mechanism where you can detect that a process is memory hog and charge swap activity to it. IOW, if there are two processes A and B and assume A is the memory hog and then B requests for memory which triggers lot of swap IO, then you can charge all that IO to memory hog A?
Quoted 7 lines
When an annoymou page is allocated, blkio-cgroup sets an ID to the page. And then when the page is going to swap out, dm-ioband can know who the owner of the page is by retrieving ID from the page.
In the above case, since the pages of the process A are swapped-out, dm-ioband charges swap IO to the process A.
But this does not mean that in all cases memory hog is being charged for swap IO, as you have said. So if a process A has done some anonymous page allocations and later a memory hog B comes in and forces swap out of A, you will charge A for swap activity which does not seem fair as B is memory hog here?
Thanks
Vivek
Quoted 8 lines
Quoted 5 lines
Can you please point me to the relevant code in dm-ioband?
IMHO, to keep things simple, all swapping activity should be charged to root group and be considered as kernel activity and user space not be charged for that.
Thanks,
Ryo Tsuruta
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH 18/23] io-controller: blkio_cgroup patches from Ryo to track async bios.
Hi Vivek,
Quoted 24 lines · Vivek Goyal
Quoted 17 lines
Quoted 10 lines
Quoted 9 lines
Quoted 4 lines
Quoted 4 lines
- Somebody also gave an example where there is a memory hogging process and possibly pushes out some processes to swap. It does not sound fair to charge those proccess for that swap writeout. These processes never requested swap IO.
I think that swap writeouts should be charged to the memory hogging process, because the process consumes more resources and it should get a penalty.
Quoted 6 lines
A process requesting memory gets IO penalty? IMHO, swapping is a kernel mechanism and kernel's way of providing extended RAM. If we want to solve the issue of memory hogging by a process then right way to solve is to use memory controller and not by charging the process for IO activity. Instead, proabably a more suitable way is to charge swap activity to root group (where by default all the kernel related activity goes).
Quoted 3 lines
No. In the current blkio-cgroup, a process which uses a large amount of memory gets penalty, not a memory requester.
At ioband level you just get to see bio and page. How do you decide wheter this bio is being issued by a process which is a memory hog?
Quoted 6 lines
In fact requester of memory could be anybody. It could be memory hog or a different process. So are you saying that you got a mechanism where you can detect that a process is memory hog and charge swap activity to it. IOW, if there are two processes A and B and assume A is the memory hog and then B requests for memory which triggers lot of swap IO, then you can charge all that IO to memory hog A?
When an annoymou page is allocated, blkio-cgroup sets an ID to the page. And then when the page is going to swap out, dm-ioband can know who the owner of the page is by retrieving ID from the page.
In the above case, since the pages of the process A are swapped-out, dm-ioband charges swap IO to the process A.
Quoted 5 lines
Can you please point me to the relevant code in dm-ioband?
IMHO, to keep things simple, all swapping activity should be charged to root group and be considered as kernel activity and user space not be charged for that.
Thanks,
Ryo Tsuruta
A message that is not in the archive (sent off-list or not yet received): 20090902.185251.193693849.ryov-jCdQPDEk3idL9jVzuh4AOg@public.gmane.org