IO Controller
Subsystems (3)
Re: IO Controller per cgroup request descriptors (Re: [PATCH 01/10] Documentation)
In reply to a message not in the archiveQuoted 39 lines · Vivek Goyal
Quoted 30 lines · Munehiro
Quoted 25 lines · Vivek Goyal
Quoted 7 lines
Quoted 6 lines
+TODO
+==== +- Lots of cleanups, testing, bug fixing, optimizations, benchmarking etc... +- Convert cgroup ioprio to notion of weight. +- Anticipatory code will need more work. It is not working properly currently+ and needs more thought.
What are the problems with the code?
Have not got a chance to look into the issues in detail yet. Just a crude run saw drop in performance. Will debug it later the moment I have got async writes handled...
Quoted 6 lines
Quoted 1 line
+- Use of bio-cgroup patches.
I saw these posted as well
Quoted 2 lines
+- Use of Nauman's per cgroup request descriptor patches. +
More details would be nice, I am not sure I understand
Currently the number of request descriptors which can be allocated per device/request queue are fixed by a sysfs tunable (
q->nr_requests). So if there is lots of IO going on from one cgroup then it will consume all the available request descriptors and other cgroup might starve and not get its fair share.Quoted 3 lines
Hence we also need to introduce the notion of request descriptor limit per cgroup so that if request descriptors from one group are exhausted, then it does not impact the IO of other cgroup.
Quoted 7 lines
Unfortunately I couldn't find and I've never seen the Nauman's patches. So I tried to make a patch below against this todo. The reason why I'm posting this despite this is just a quick and ugly hack (and it might be a reinvention of wheel) is that I would like to discuss how we should define the limitation of requests per cgroup.
This patch should be applied on Vivek's I/O controller patches posted on Mar 11.
Quoted 4 lines
Hi IKEDA,
Sorry for the confusion here. Actually Nauman had sent a patch to select group of people who were initially copied on the mail thread.
I am sorry about that. Since I dropped my whole patch set in favor of Vivek's stuff, this stuff fell through the cracks.
Quoted 33 lines
Quoted 9 lines
This patch temporarily distribute
q->nr_requeststo each cgroup. I think the number should be weighted like BFQ's budget. But in this case, if the hierarchy of cgroup is deep, leaf cgroups are allowed to allocate very few numbers of requests. I don't think this is reasonable...but I don't have specific idea to solve this problem. Does anyone have the good idea?Thanks for the patch. Yes, ideally one would expect the request descriptor to be allocated also in proportion to the weight but I guess that would become very comlicated.
In terms of simpler things, two thoughts come to mind.
- First approach is to make
q->nr_requestsper group. So every group is entitled forq->nr_requestsas set by the user. This is what your patch seems to have done.I had some concerns with this approach. First of all it does not seem to have an upper bound on number of request descriptors allocated per queue because if a user creates more cgroups, total number of request descriptors increase.
- Second approach can be that we retain the meaning of
q->nr_requestswhich defines the total number of request descriptors on the queue (with the exception of 50% more descriptors for batching processes). And we define a new per group limitq->nr_group_requestswhich defines how many requests per group can be assigned. Soq->nr_requestsdefines total pool size on the queue andq->nr_group_requestswill define how many requests each group can allocate out of that pool.
Quoted 8 lines
Here the issue is that a user shall have to balance the
q->nr_group_requestsandq->nr_requestsproperly.To experiment, I have implemented the second approach. I am attaching the patch which is in my current tree. It probably will not apply on my march 11 posting as since then patches have changed. But posting it here so that at least it will give an idea behind the thought process.
Ideas are welcome...
I had started with the first option, but the second option sounds good too. But one problem that comes to mind is how we deal with hierarchies? The sys admin can limit the root level cgroups to specific number of request descriptors, but if applications running in a cgroup are allowed to create their own cgroups, then the total request descriptors of all child cgroups should be capped by the number assigned to parent cgroups.
Quoted 1,385 lines
Thanks
Viveko Currently a request queue has got fixed number of request descriptors for sync and async requests. Once the request descriptors are consumed, new processes are put to sleep and they effectively become serialized. Because sync and async queues are separate, async requests don't impact sync ones but if one is looking for fairness between async requests, that is not achievable if request queue descriptors become bottleneck.
o Make request descriptor's per io group so that if there is lots of IO going on in one cgroup, it does not impact the IO of other group.
o This patch implements the per cgroup request descriptors. request pool per queue is still common but every group will have its own wait list and its own count of request descriptors allocated to that group for sync and async queues. So effectively
request_listbecomes per io group property and not a global request queue feature.o Currently one can define
q->nr_requeststo limit request descriptors allocated for the queue. Now there is another tunableq->nr_group_requestswhich controls the requests descriptr limit per group.q->nr_requestssupercedesq->nr_group_requeststo make sure if there are lots of groups present, we don't end up allocating too many request descriptors on the queue.o Issues: Currently notion of congestion is per queue. With per group request descriptor it is possible that queue is not congested but the group bio will go into is congested.
Signed-off-by Nauman Rafique <[email protected]>Signed-off-by Vivek Goyal <[email protected]>---
block/blk-core.c | 216 ++++++++++++++++++++++++++++++++++---------------block/blk-settings.c| 3
block/blk-sysfs.c | 57 ++++++++++--
block/elevator-fq.c | 15 +++
block/elevator-fq.h | 8 +
block/elevator.c | 6 -
include/linux/blkdev.h | 62 +++++++++++++-7 files changed, 287 insertions(+), 80 deletions(-)Index:
linux9/include/linux/blkdev.h
===================================================================include/linux/blkdev.h 2009-04-30 16:18:29.000000000 -0400 renamed from include/linux/blkdev.h 2009-04-30 15:43:53.000000000 -0400 +36
-2@@ -32,21 +32,51 @@ struct request; struct sg_io_hdr; #define BLKDEV_MIN_RQ 4 + +#ifdef CONFIG_GROUP_IOSCHED +#define BLKDEV_MAX_RQ 256 /* Default maximum */ +#define BLKDEV_MAX_GROUP_RQ 64 /* Default maximum */ +#else #define BLKDEV_MAX_RQ 128 /* Default maximum */ +/* + * This is eqivalent to case of only one group present (root group). Let + * it consume all the request descriptors available on the queue . + */ +#define BLKDEV_MAX_GROUP_RQ BLKDEV_MAX_RQ /* Default maximum */ +#endif struct request; typedef void (rq_end_io_fn)(struct request *, int); struct request_list { /* - * count[], starved[], and wait[] are indexed by + * count[], starved and wait[] are indexed by * BLK_RW_SYNC/BLK_RW_ASYNC */ int count[2]; int starved[2]; + wait_queue_head_t wait[2]; +}; + +/* + * This data structures keeps track of mempool of requests for the queue + * and some overall statistics. + */ +struct request_data { + /* + * Per queue request descriptor count. This is in addition to per + * cgroup count + */ + int count[2]; int elvpriv; mempool_t *rq_pool; - wait_queue_head_t wait[2]; + int starved; + /* + * Global list for starved tasks. A task will be queued here if + * it could not allocate request descriptor and the associated + * group request list does not have any requests pending. + */ + wait_queue_head_t starved_wait; }; /*@@ -251,6 +281,7 @@ struct request { #ifdef CONFIG_GROUP_IOSCHED /* io group request belongs to */ struct io_group *iog; + struct request_list *rl; #endif /* GROUP_IOSCHED */ #endif /* ELV_FAIR_QUEUING */ };@@ -340,6 +371,9 @@ struct request_queue */ struct request_list rq; + /* Contains request pool and other data like starved data */ + struct request_data rq_data; + request_fn_proc *request_fn; make_request_fn *make_request_fn; prep_rq_fn *prep_rq_fn;@@ -402,6 +436,8 @@ struct request_queue * queue settings */unsigned long
nr_requests; /* Max # of requests */
+ /* Max # of per io group requests */
+ unsigned longnr_group_requests;
unsigned int nr_congestion_on;
unsigned int nr_congestion_off;
unsigned int nr_batching;include/linux/blkdev.h 2009-04-30 16:18:29.000000000 -0400 renamed from include/linux/blkdev.h 2009-04-30 15:43:53.000000000 -0400 +22
-0@@ -773,6 +809,28 @@ extern int scsi_cmd_ioctl(struct request extern int sg_scsi_ioctl(struct request_queue *, struct gendisk *, fmode_t, struct scsi_ioctl_command __user *); +extern void blk_init_request_list(struct request_list *rl); + +static inline struct request_list *blk_get_request_list(struct request_queue *q, + struct bio *bio) +{ +#ifdef CONFIG_GROUP_IOSCHED + return io_group_get_request_list(q, bio); +#else + return &q->rq; +#endif +} + +static inline struct request_list *rq_rl(struct request_queue *q, + struct request *rq) +{ +#ifdef CONFIG_GROUP_IOSCHED + return rq->rl; +#else + return blk_get_request_list(q, NULL); +#endif +} + /* * Temporary export, until SCSI gets fixed up. */Index:
linux9/block/elevator.c
===================================================================block/elevator.c 2009-04-30 16:18:29.000000000 -0400 renamed from block/elevator.c 2009-04-30 16:17:53.000000000 -0400 +3
-3@@ -664,7 +664,7 @@ void elv_quiesce_start(struct request_qu * make sure we don't have any requests in flight */ elv_drain_elevator(q); - while (q->rq.elvpriv) { + while (q->rq_data.elvpriv) { blk_start_queueing(q); spin_unlock_irq(q->queue_lock); msleep(10);@@ -764,8 +764,8 @@ void elv_insert(struct request_queue *q, } if (unplug_it && blk_queue_plugged(q)) { - int nrq = q->rq.count[BLK_RW_SYNC] + q->rq.count[BLK_RW_ASYNC] - - q->in_flight; + int nrq = q->rq_data.count[BLK_RW_SYNC] + + q->rq_data.count[BLK_RW_ASYNC] - q->in_flight; if (nrq >= q->unplug_thresh) __generic_unplug_device(q);Index:
linux9/block/blk-core.c
===================================================================block/blk-core.c 2009-04-30 16:18:29.000000000 -0400 renamed from block/blk-core.c 2009-04-30 16:17:53.000000000 -0400 +22
-8@@ -480,20 +480,31 @@ void blk_cleanup_queue(struct request_qu } EXPORT_SYMBOL(blk_cleanup_queue); -static int blk_init_free_list(struct request_queue *q) +void blk_init_request_list(struct request_list *rl) { - struct request_list *rl = &q->rq; rl->count[BLK_RW_SYNC] = rl->count[BLK_RW_ASYNC] = 0; - rl->starved[BLK_RW_SYNC] = rl->starved[BLK_RW_ASYNC] = 0; - rl->elvpriv = 0; init_waitqueue_head(&rl->wait[BLK_RW_SYNC]); init_waitqueue_head(&rl->wait[BLK_RW_ASYNC]); +} - rl->rq_pool = mempool_create_node(BLKDEV_MIN_RQ, mempool_alloc_slab, - mempool_free_slab, request_cachep, q->node); +static int blk_init_free_list(struct request_queue *q) +{ +#ifndef CONFIG_GROUP_IOSCHED + struct request_list *rl = blk_get_request_list(q, NULL); + + /* + * In case of group scheduling, request list is inside the associated + * group and when that group is instanciated, it takes care of + * initializing the request list also. + */ + blk_init_request_list(rl); +#endif + q->rq_data.rq_pool = mempool_create_node(BLKDEV_MIN_RQ, + mempool_alloc_slab, mempool_free_slab, + request_cachep, q->node); - if (!rl->rq_pool) + if (!q->rq_data.rq_pool) return -ENOMEM; return 0;@@ -590,6 +601,9 @@ blk_init_queue_node(request_fn_proc *rfn return NULL; } + /* init starved waiter wait queue */ + init_waitqueue_head(&q->rq_data.starved_wait); + /* * if caller didn't supply a lock, they get per-queue locking with * our embedded lock@@ -639,14 +653,14 @@ static inline void blk_free_request(stru { if (rq->cmd_flags & REQ_ELVPRIV) elv_put_request(q, rq); - mempool_free(rq, q->rq.rq_pool); + mempool_free(rq, q->rq_data.rq_pool); }static struct request *
blk_alloc_request(structrequest_queue*q, struct bio *bio, int rw, int priv,gfp_tgfp_mask)
{
- struct request *rq =mempool_alloc(q->rq.rq_pool, gfp_mask);
+ struct request *rq =mempool_alloc(q->rq_data.rq_pool, gfp_mask);if (!rq)
return NULL;block/blk-core.c 2009-04-30 16:18:29.000000000 -0400 renamed from block/blk-core.c 2009-04-30 16:17:53.000000000 -0400 +81
-42@@ -657,7 +671,7 @@ blk_alloc_request(struct request_queue * if (priv) { if (unlikely(elv_set_request(q, rq, bio, gfp_mask))) { - mempool_free(rq, q->rq.rq_pool); + mempool_free(rq, q->rq_data.rq_pool); return NULL; } rq->cmd_flags |= REQ_ELVPRIV;@@ -700,18 +714,18 @@ static void ioc_set_batching(struct requ ioc->last_waited = jiffies; } -static void __freed_request(struct request_queue *q, int sync) +static void __freed_request(struct request_queue *q, int sync, + struct request_list *rl) { - struct request_list *rl = &q->rq; - - if (rl->count[sync] < queue_congestion_off_threshold(q)) + if (q->rq_data.count[sync] < queue_congestion_off_threshold(q)) blk_clear_queue_congested(q, sync); - if (rl->count[sync] + 1 <= q->nr_requests) { + if (q->rq_data.count[sync] + 1 <= q->nr_requests) + blk_clear_queue_full(q, sync); + + if (rl->count[sync] + 1 <= q->nr_group_requests) { if (waitqueue_active(&rl->wait[sync])) wake_up(&rl->wait[sync]); - - blk_clear_queue_full(q, sync); } }@@ -719,18 +733,29 @@ static void __freed_request(struct reque * A request has just been released. Account for it, update the full and * congestion status, wake up any waiters. Called under q->queue_lock. */ -static void freed_request(struct request_queue *q, int sync, int priv) +static void freed_request(struct request_queue *q, int sync, int priv, + struct request_list *rl) { - struct request_list *rl = &q->rq; - + BUG_ON(!rl->count[sync]); rl->count[sync]--; + + BUG_ON(!q->rq_data.count[sync]); + q->rq_data.count[sync]--; + if (priv) - rl->elvpriv--; + q->rq_data.elvpriv--; - __freed_request(q, sync); + __freed_request(q, sync, rl); if (unlikely(rl->starved[sync ^ 1])) - __freed_request(q, sync ^ 1); + __freed_request(q, sync ^ 1, rl); + + /* Wake up the starved process on global list, if any */ + if (unlikely(q->rq_data.starved)) { + if (waitqueue_active(&q->rq_data.starved_wait)) + wake_up(&q->rq_data.starved_wait); + q->rq_data.starved--; + } } /*@@ -739,10 +764,9 @@ static void freed_request(struct request * Returns !NULL on success, with queue_lock *not held*. */ static struct request *get_request(struct request_queue *q, int rw_flags, - struct bio *bio, gfp_t gfp_mask) + struct bio *bio, gfp_t gfp_mask, struct request_list *rl) { struct request *rq = NULL; - struct request_list *rl = &q->rq; struct io_context *ioc = NULL; const bool is_sync = rw_is_sync(rw_flags) != 0; int may_queue, priv;@@ -751,31 +775,38 @@ static struct request *get_request(struc if (may_queue == ELV_MQUEUE_NO) goto rq_starved; - if (rl->count[is_sync]+1 >= queue_congestion_on_threshold(q)) { - if (rl->count[is_sync]+1 >= q->nr_requests) { - ioc = current_io_context(GFP_ATOMIC, q->node); - /* - * The queue will fill after this allocation, so set - * it as full, and mark this process as "batching". - * This process will be allowed to complete a batch of - * requests, others will be blocked. - */ - if (!blk_queue_full(q, is_sync)) { - ioc_set_batching(q, ioc); - blk_set_queue_full(q, is_sync); - } else { - if (may_queue != ELV_MQUEUE_MUST - && !ioc_batching(q, ioc)) { - /* - * The queue is full and the allocating - * process is not a "batcher", and not - * exempted by the IO scheduler - */ - goto out; - } + if (q->rq_data.count[is_sync]+1 >= queue_congestion_on_threshold(q)) + blk_set_queue_congested(q, is_sync); + + /* + * Looks like there is no user of queue full now. + * Keeping it for time being. + */ + if (q->rq_data.count[is_sync]+1 >= q->nr_requests) + blk_set_queue_full(q, is_sync); + + if (rl->count[is_sync]+1 >= q->nr_group_requests) { + ioc = current_io_context(GFP_ATOMIC, q->node); + /* + * The queue request descriptor group will fill after this + * allocation, so set + * it as full, and mark this process as "batching". + * This process will be allowed to complete a batch of + * requests, others will be blocked. + */ + if (rl->count[is_sync] <= q->nr_group_requests) + ioc_set_batching(q, ioc); + else { + if (may_queue != ELV_MQUEUE_MUST + && !ioc_batching(q, ioc)) { + /* + * The queue is full and the allocating + * process is not a "batcher", and not + * exempted by the IO scheduler + */ + goto out; } } - blk_set_queue_congested(q, is_sync); } /*@@ -783,19 +814,41 @@ static struct request *get_request(struc * limit of requests, otherwise we could have thousands of requests * allocated with any setting of ->nr_requests */ - if (rl->count[is_sync] >= (3 * q->nr_requests / 2)) + + if (q->rq_data.count[is_sync] >= (3 * q->nr_requests / 2)) + goto out; + + /* + * Allocation of request is allowed from queue perspective. Now check + * from per group request list + */ + + if (rl->count[is_sync] >= (3 * q->nr_group_requests / 2)) goto out; rl->count[is_sync]++; rl->starved[is_sync] = 0; + q->rq_data.count[is_sync]++; + priv = !test_bit(QUEUE_FLAG_ELVSWITCH, &q->queue_flags); if (priv) - rl->elvpriv++; + q->rq_data.elvpriv++; spin_unlock_irq(q->queue_lock); rq = blk_alloc_request(q, bio, rw_flags, priv, gfp_mask); + +#ifdef CONFIG_GROUP_IOSCHED + if (rq) { + /* + * TODO. Implement group reference counting and take the + * reference to the group to make sure group hence request + * list does not go away till rq finishes. + */ + rq->rl = rl; + } +#endif if (unlikely(!rq)) { /* * Allocation failed presumably due to memory. Undo anything@@ -805,7 +858,7 @@ static struct request *get_request(struc * wait queue, but this is pretty rare. */ spin_lock_irq(q->queue_lock); - freed_request(q, is_sync, priv); + freed_request(q, is_sync, priv, rl); /* * in the very unlikely event that allocation failed and no@@ -815,10 +868,26 @@ static struct request *get_request(struc * rq mempool into READ and WRITE */rq_starved:
- if (unlikely(rl->count[is_sync] == 0))
- rl->starved[is_sync] = 1;
-
- goto out;
+ if (unlikely(rl->count[is_sync] == 0)) {
+ /*
+ * If there is a request pending in other direction
+ * in same io group, then set the starved flag of
+ * the group request list. Otherwise, we need to
+ * make this process sleep in global starved list
+ * to make sure it will not sleep indefinitely.
+ */
+ if (rl->count[is_sync ^ 1] != 0) {
+ rl->starved[is_sync] = 1;
+ goto out;
+ } else {
+ /*
+ * It indicates to calling function to put
+ * task on global starved list. Not the best
+ * way
+ */
+ returnERR_PTR(-ENOMEM);
+ }
+ }
}/*
block/blk-core.c 2009-04-30 16:18:29.000000000 -0400 renamed from block/blk-core.c 2009-04-30 16:17:53.000000000 -0400 +29
-8@@ -846,15 +915,29 @@ static struct request *get_request_wait( { const bool is_sync = rw_is_sync(rw_flags) != 0; struct request *rq; + struct request_list *rl = blk_get_request_list(q, bio); - rq = get_request(q, rw_flags, bio, GFP_NOIO); - while (!rq) { + rq = get_request(q, rw_flags, bio, GFP_NOIO, rl); + while (!rq || (IS_ERR(rq) && PTR_ERR(rq) == -ENOMEM)) { DEFINE_WAIT(wait); struct io_context *ioc; - struct request_list *rl = &q->rq; - prepare_to_wait_exclusive(&rl->wait[is_sync], &wait, - TASK_UNINTERRUPTIBLE); + if (IS_ERR(rq) && PTR_ERR(rq) == -ENOMEM) { + /* + * Task failed allocation and needs to wait and + * try again. There are no requests pending from + * the io group hence need to sleep on global + * wait queue. Most likely the allocation failed + * because of memory issues. + */ + + q->rq_data.starved++; + prepare_to_wait_exclusive(&q->rq_data.starved_wait, + &wait, TASK_UNINTERRUPTIBLE); + } else { + prepare_to_wait_exclusive(&rl->wait[is_sync], &wait, + TASK_UNINTERRUPTIBLE); + } trace_block_sleeprq(q, bio, rw_flags & 1);@@ -874,7 +957,12 @@ static struct request *get_request_wait( spin_lock_irq(q->queue_lock); finish_wait(&rl->wait[is_sync], &wait); - rq = get_request(q, rw_flags, bio, GFP_NOIO); + /* + * After the sleep check the rl again in case cgrop bio + * belonged to is gone and it is mapped to root group now + */ + rl = blk_get_request_list(q, bio); + rq = get_request(q, rw_flags, bio, GFP_NOIO, rl); }; return rq;@@ -883,6 +971,7 @@ static struct request *get_request_wait( struct request *blk_get_request(struct request_queue *q, int rw, gfp_t gfp_mask) { struct request *rq; + struct request_list *rl = blk_get_request_list(q, NULL); BUG_ON(rw != READ && rw != WRITE);@@ -890,7 +979,7 @@ struct request *blk_get_request(struct r if (gfp_mask & __GFP_WAIT) { rq = get_request_wait(q, rw, NULL); } else { - rq = get_request(q, rw, NULL, gfp_mask); + rq = get_request(q, rw, NULL, gfp_mask, rl); if (!rq) spin_unlock_irq(q->queue_lock); }@@ -1073,12 +1162,13 @@ void __blk_put_request(struct request_qu if (req->cmd_flags & REQ_ALLOCED) { int is_sync = rq_is_sync(req) != 0; int priv = req->cmd_flags & REQ_ELVPRIV; + struct request_list *rl = rq_rl(q, req); BUG_ON(!list_empty(&req->queuelist)); BUG_ON(!hlist_unhashed(&req->hash)); blk_free_request(q, req); - freed_request(q, is_sync, priv); + freed_request(q, is_sync, priv, rl); } } EXPORT_SYMBOL_GPL(__blk_put_request);Index:
linux9/block/blk-sysfs.c
===================================================================block/blk-sysfs.c 2009-04-30 16:18:29.000000000 -0400 renamed from block/blk-sysfs.c 2009-04-30 16:18:27.000000000 -0400 +0
-0@@ -38,7 +38,7 @@ static ssize_t queue_requests_show(strucstatic
ssize_t
queue_requests_store(structrequest_queue*q, const char *page,size_tcount) {
- structrequest_list*rl =&q->rq;
+ structrequest_list*rl =blk_get_request_list(q, NULL); unsigned long nr;
int ret =queue_var_store(&nr, page, count);
if (nr <BLKDEV_MIN_RQ)block/blk-sysfs.c 2009-04-30 16:18:29.000000000 -0400 renamed from block/blk-sysfs.c 2009-04-30 16:18:27.000000000 -0400 +42
-8@@ -48,32 +48,55 @@ queue_requests_store(struct request_queu q->nr_requests = nr; blk_queue_congestion_threshold(q); - if (rl->count[BLK_RW_SYNC] >= queue_congestion_on_threshold(q)) + if (q->rq_data.count[BLK_RW_SYNC] >= queue_congestion_on_threshold(q)) blk_set_queue_congested(q, BLK_RW_SYNC); - else if (rl->count[BLK_RW_SYNC] < queue_congestion_off_threshold(q)) + else if (q->rq_data.count[BLK_RW_SYNC] < + queue_congestion_off_threshold(q)) blk_clear_queue_congested(q, BLK_RW_SYNC); - if (rl->count[BLK_RW_ASYNC] >= queue_congestion_on_threshold(q)) + if (q->rq_data.count[BLK_RW_ASYNC] >= queue_congestion_on_threshold(q)) blk_set_queue_congested(q, BLK_RW_ASYNC); - else if (rl->count[BLK_RW_ASYNC] < queue_congestion_off_threshold(q)) + else if (q->rq_data.count[BLK_RW_ASYNC] < + queue_congestion_off_threshold(q)) blk_clear_queue_congested(q, BLK_RW_ASYNC); - if (rl->count[BLK_RW_SYNC] >= q->nr_requests) { + if (q->rq_data.count[BLK_RW_SYNC] >= q->nr_requests) { blk_set_queue_full(q, BLK_RW_SYNC); - } else if (rl->count[BLK_RW_SYNC]+1 <= q->nr_requests) { + } else if (q->rq_data.count[BLK_RW_SYNC]+1 <= q->nr_requests) { blk_clear_queue_full(q, BLK_RW_SYNC); wake_up(&rl->wait[BLK_RW_SYNC]); } - if (rl->count[BLK_RW_ASYNC] >= q->nr_requests) { + if (q->rq_data.count[BLK_RW_ASYNC] >= q->nr_requests) { blk_set_queue_full(q, BLK_RW_ASYNC); - } else if (rl->count[BLK_RW_ASYNC]+1 <= q->nr_requests) { + } else if (q->rq_data.count[BLK_RW_ASYNC]+1 <= q->nr_requests) { blk_clear_queue_full(q, BLK_RW_ASYNC); wake_up(&rl->wait[BLK_RW_ASYNC]); } spin_unlock_irq(q->queue_lock); return ret; } +#ifdef CONFIG_GROUP_IOSCHED +static ssize_t queue_group_requests_show(struct request_queue *q, char *page) +{ + return queue_var_show(q->nr_group_requests, (page)); +} + +static ssize_t +queue_group_requests_store(struct request_queue *q, const char *page, + size_t count) +{ + unsigned long nr; + int ret = queue_var_store(&nr, page, count); + if (nr < BLKDEV_MIN_RQ) + nr = BLKDEV_MIN_RQ; + + spin_lock_irq(q->queue_lock); + q->nr_group_requests = nr; + spin_unlock_irq(q->queue_lock); + return ret; +} +#endif static ssize_t queue_ra_show(struct request_queue *q, char *page) {@@ -228,6 +251,14 @@ static struct queue_sysfs_entry queue_re .store = queue_requests_store, }; +#ifdef CONFIG_GROUP_IOSCHED +static struct queue_sysfs_entry queue_group_requests_entry = { + .attr = {.name = "nr_group_requests", .mode = S_IRUGO | S_IWUSR }, + .show = queue_group_requests_show, + .store = queue_group_requests_store, +}; +#endif + static struct queue_sysfs_entry queue_ra_entry = { .attr = {.name = "read_ahead_kb", .mode = S_IRUGO | S_IWUSR }, .show = queue_ra_show,@@ -308,6 +339,9 @@ static struct queue_sysfs_entry queue_sl static struct attribute *default_attrs[] = { &queue_requests_entry.attr, +#ifdef CONFIG_GROUP_IOSCHED + &queue_group_requests_entry.attr, +#endif &queue_ra_entry.attr, &queue_max_hw_sectors_entry.attr, &queue_max_sectors_entry.attr,@@ -389,12 +423,11 @@ static void blk_release_queue(struct kob {struct
request_queue*q =
container_of(kobj, structrequest_queue, kobj);
- structrequest_list*rl =&q->rq;blk_sync_queue(q);
- if (
rl->rq_pool)
-mempool_destroy(rl->rq_pool);
+ if (q->rq_data.rq_pool)
+mempool_destroy(q->rq_data.rq_pool);if (
q->queue_tags)
__blk_queue_free_tags(q);
Index:linux9/block/blk-settings.c
===================================================================block/blk-settings.c 2009-04-30 16:18:29.000000000 -0400 renamed from block/blk-settings.c 2009-04-30 15:43:53.000000000 -0400 +3
-0@@ -123,6 +123,9 @@ void blk_queue_make_request(struct reque * set defaults */ q->nr_requests = BLKDEV_MAX_RQ; +#ifdef CONFIG_GROUP_IOSCHED + q->nr_group_requests = BLKDEV_MAX_GROUP_RQ; +#endif blk_queue_max_phys_segments(q, MAX_PHYS_SEGMENTS); blk_queue_max_hw_segments(q, MAX_HW_SEGMENTS); blk_queue_segment_boundary(q, BLK_SEG_BOUNDARY_MASK);Index:
linux9/block/elevator-fq.c
===================================================================block/elevator-fq.c 2009-04-30 16:18:29.000000000 -0400 renamed from block/elevator-fq.c 2009-04-30 16:18:27.000000000 -0400 +15
-0@@ -954,6 +954,17 @@ struct io_cgroup *cgroup_to_io_cgroup(st struct io_cgroup, css); } +struct request_list *io_group_get_request_list(struct request_queue *q, + struct bio *bio) +{ + struct io_group *iog; + + iog = io_get_io_group_bio(q, bio, 1); + BUG_ON(!iog); +out: + return &iog->rl; +} + /* * Search the bfq_group for bfqd into the hash table (by now only a list) * of bgrp. Must be called under rcu_read_lock().@@ -1203,6 +1214,8 @@ struct io_group *io_group_chain_alloc(st io_group_init_entity(iocg, iog); iog->my_entity = &iog->entity; + blk_init_request_list(&iog->rl); + if (leaf == NULL) { leaf = iog; prev = leaf;@@ -1446,6 +1459,8 @@ struct io_group *io_alloc_root_group(str for (i = 0; i < IO_IOPRIO_CLASSES; i++) iog->sched_data.service_tree[i] = IO_SERVICE_TREE_INIT; + blk_init_request_list(&iog->rl); + iocg = &io_root_cgroup; spin_lock_irq(&iocg->lock); rcu_assign_pointer(iog->key, key);Index:
linux9/block/elevator-fq.h
===================================================================block/elevator-fq.h 2009-04-30 16:18:29.000000000 -0400 renamed from block/elevator-fq.h 2009-04-30 16:18:27.000000000 -0400 +8
-0@@ -239,8 +239,14 @@ struct io_group { /* Single ioq per group, used for noop, deadline, anticipatory */ struct io_queue *ioq; + + /* request list associated with the group */ + struct request_list rl; }; +#define IOG_FLAG_READFULL 1 /* read queue has been filled */ +#define IOG_FLAG_WRITEFULL 2 /* write queue has been filled */ + /** * struct bfqio_cgroup - bfq cgroup data structure. * @css: subsystem state for bfq in the containing cgroup.@@ -517,6 +523,8 @@ extern void elv_fq_unset_request_ioq(str extern struct io_queue *elv_lookup_ioq_current(struct request_queue *q); extern struct io_queue *elv_lookup_ioq_bio(struct request_queue *q, struct bio *bio); +extern struct request_list *io_group_get_request_list(struct request_queue *q, + struct bio *bio); /* Returns single ioq associated with the io group. */ static inline struct io_queue *io_group_ioq(struct io_group *iog)Thanks
VivekQuoted 619 lines
---
block/blk-core.c| 36 +++++++--block/blk-sysfs.c| 22 ++++--block/elevator-fq.c| 133 ++++++++++++++++++++++++++++++++--block/elevator-fq.h| 201 +++++++++++++++++++++++++++++++++++++++++++++++++++4 files changed, 371 insertions(+), 21 deletions(-)block/blk-core.c +28
-8@@ -705,11 +705,15 @@ static void ioc_set_batching(struct request_queue *q, struct io_context *ioc)static void __freed_request (struct request_queue *q,int rw) { struct request_list *rl = &q->rq; - - if (rl->count[rw] < queue_congestion_off_threshold(q)) + struct io_group *congested_iog, *full_iog; + + congested_iog = io_congested_io_group(q, rw); + if (rl->count[rw] < queue_congestion_off_threshold(q) && + !congested_iog) blk_clear_queue_congested(q, rw); - if (rl->count[rw] + 1 <= q->nr_requests) { + full_iog = io_full_io_group(q, rw); + if (rl->count[rw] + 1 <= q->nr_requests && !full_iog) { if (waitqueue_active(&rl->wait[rw])) wake_up(&rl->wait[rw]);@@ -721,13 +725,16 @@ static void __freed_request(struct request_queue *q, int rw)* A request has just been released. Account for it, update the full and * congestion status, wake up any waiters. Called under q->queue_lock. */ -static void freed_request (struct request_queue *q,int rw,int priv) +static void freed_request (struct request_queue *q,struct io_group *iog, + int rw, int priv) { struct request_list *rl = &q->rq; rl->count[rw]--; if (priv) rl->elvpriv--; + if (iog) + io_group_dec_count(iog, rw); __freed_request(q, rw);@@ -746,16 +753,21 @@ static struct request *get_request(struct request_queue *q, int rw_flags, { struct request *rq = NULL; struct request_list *rl = &q->rq; + struct io_group *iog; struct io_context *ioc = NULL; const int rw = rw_flags & 0x01; int may_queue, priv; + iog = __io_get_io_group(q); + may_queue = elv_may_queue(q, rw_flags); if (may_queue == ELV_MQUEUE_NO) goto rq_starved; - if (rl->count[rw]+1 >= queue_congestion_on_threshold(q)) { - if (rl->count[rw]+1 >= q->nr_requests) { + if (rl->count[rw]+1 >= queue_congestion_on_threshold(q) || + io_group_congestion_on(iog, rw)) { + if (rl->count[rw]+1 >= q->nr_requests || + io_group_full(iog, rw)) { ioc = current_io_context(GFP_ATOMIC, q->node); /** The queue will fill after this allocation, so set @@ -789,8 +801,15 @@ static struct request *get_request(struct request_queue *q, int rw_flags, if (rl->count[rw] >= (3 * q->nr_requests / 2)) goto out; + if (iog) + if (io_group_count(iog, rw) >= + (3 * io_group_nr_requests(iog) / 2)) + goto out; + rl->count[rw]++; rl->starved[rw] = 0; + if (iog) + io_group_inc_count(iog, rw); priv = !test_bit(QUEUE_FLAG_ELVSWITCH, &q->queue_flags); if (priv)@@ -808,7 +827,7 @@ static struct request *get_request(struct request_queue *q, int rw_flags,* wait queue, but this is pretty rare. */ spin_lock_irq(q->queue_lock); - freed_request(q, rw, priv); + freed_request(q, iog, rw, priv); /** in the very unlikely event that allocation failed and no @@ -1073,12 +1092,13 @@ void __blk_put_request(struct request_queue *q, struct request *req) if (req->cmd_flags & REQ_ALLOCED) { int rw = rq_data_dir(req); int priv = req->cmd_flags & REQ_ELVPRIV; + struct io_group *iog = io_request_io_group(req); BUG_ON(!list_empty(&req->queuelist)); BUG_ON(!hlist_unhashed(&req->hash)); blk_free_request(q, req); - freed_request(q, rw, priv); + freed_request(q, iog, rw, priv); } }EXPORT_SYMBOL_GPL (__blk_put_request);block/blk-sysfs.c +14
-8@@ -40,6 +40,7 @@ queue_requests_store(struct request_queue *q, const char *page, size_t count) { struct request_list *rl = &q->rq; unsigned long nr; + int iog_congested[2], iog_full[2]; int ret = queue_var_store(&nr, page, count); if (nr < BLKDEV_MIN_RQ) nr = BLKDEV_MIN_RQ;@@ -47,27 +48,32 @@ queue_requests_store(struct request_queue *q, const char *page, size_t count) spin_lock_irq(q->queue_lock); q->nr_requests = nr; blk_queue_congestion_threshold(q); + io_group_set_nrq_all(q, nr, iog_congested, iog_full); - if (rl->count[READ] >= queue_congestion_on_threshold(q)) + if (rl->count[READ] >= queue_congestion_on_threshold(q) || + iog_congested[READ]) blk_set_queue_congested(q, READ); - else if (rl->count[READ] < queue_congestion_off_threshold(q)) + else if (rl->count[READ] < queue_congestion_off_threshold(q) && + !iog_congested[READ]) blk_clear_queue_congested(q, READ); - if (rl->count[WRITE] >= queue_congestion_on_threshold(q)) + if (rl->count[WRITE] >= queue_congestion_on_threshold(q) || + iog_congested[WRITE]) blk_set_queue_congested(q, WRITE); - else if (rl->count[WRITE] < queue_congestion_off_threshold(q)) + else if (rl->count[WRITE] < queue_congestion_off_threshold(q) && + !iog_congested[WRITE]) blk_clear_queue_congested(q, WRITE); - if (rl->count[READ] >= q->nr_requests) { + if (rl->count[READ] >= q->nr_requests || iog_full[READ]) { blk_set_queue_full(q, READ); - } else if (rl->count[READ]+1 <= q->nr_requests) { + } else if (rl->count[READ]+1 <= q->nr_requests && !iog_full[READ]) { blk_clear_queue_full(q, READ); wake_up(&rl->wait[READ]); } - if (rl->count[WRITE] >= q->nr_requests) { + if (rl->count[WRITE] >= q->nr_requests || iog_full[WRITE]) { blk_set_queue_full(q, WRITE); - } else if (rl->count[WRITE]+1 <= q->nr_requests) { + } else if (rl->count[WRITE]+1 <= q->nr_requests && !iog_full[WRITE]) { blk_clear_queue_full(q, WRITE); wake_up(&rl->wait[WRITE]); }block/elevator-fq.c +128
-5@@ -924,6 +924,111 @@ struct io_group *io_lookup_io_group_current(struct request_queue *q) }EXPORT_SYMBOL (io_lookup_io_group_current); +/* +* TODO +* This is complete dupulication of blk_queue_congestion_threshold() +* except for the argument type and name. Can we merge them? +*/ +static void io_group_nrq_congestion_threshold (struct io_group_nrq *nrq) +{ + int nr; + + nr = nrq->nr_requests - (nrq->nr_requests / 8) + 1; + if (nr > nrq->nr_requests) + nr = nrq->nr_requests; + nrq->nr_congestion_on = nr; + + nr = nrq->nr_requests - (nrq->nr_requests / 8) + - (nrq->nr_requests / 16) - 1; + if (nr < 1) + nr = 1; + nrq->nr_congestion_off = nr; +} + +static void io_group_set_nrq (struct io_group_nrq *nrq,int nr_requests, + int *congested, int *full) +{ + int i; + + BUG_ON(nr_requests < 0); + + nrq->nr_requests = nr_requests; + io_group_nrq_congestion_threshold(nrq); + + for (i=0; i<2; i++) { + if (nrq->count[i] >= nrq->nr_congestion_on) + congested[i] = 1; + else if (nrq->count[i] < nrq->nr_congestion_off) + congested[i] = 0; + + if (nrq->count[i] >= nrq->nr_requests) + full[i] = 1; + else if (nrq->count[i]+1 <= nrq->nr_requests) + full[i] = 0; + } +} + +void io_group_set_nrq_all (struct request_queue *q,int nr, + int *congested, int *full) +{ + struct elv_fq_data *efqd = &q->elevator->efqd; + struct hlist_head *head = &efqd->group_list; + struct io_group *root = efqd->root_group; + struct hlist_node *n; + struct io_group *iog; + struct io_group_nrq *nrq; + int nrq_congested[2]; + int nrq_full[2]; + int i; + + for (i=0; i<2; i++) + *(congested + i) = *(full + i) = 0; + + nrq = &root->nrq; + io_group_set_nrq(nrq, nr, nrq_congested, nrq_full); + for (i=0; i<2; i++) { + *(congested + i) |= nrq_congested[i]; + *(full + i) |= nrq_full[i]; + } + + hlist_for_each_entry(iog, n, head, elv_data_node) { + nrq = &iog->nrq; + io_group_set_nrq(nrq, nr, nrq_congested, nrq_full); + for (i=0; i<2; i++) { + *(congested + i) |= nrq_congested[i]; + *(full + i) |= nrq_full[i]; + } + } +} + +struct io_group *io_congested_io_group (struct request_queue *q,int rw) +{ + struct hlist_head *head = &q->elevator->efqd.group_list; + struct hlist_node *n; + struct io_group *iog; + + hlist_for_each_entry(iog, n, head, elv_data_node) { + struct io_group_nrq *nrq = &iog->nrq; + if (nrq->count[rw] >= nrq->nr_congestion_off) + return iog; + } + return NULL; +} + +struct io_group *io_full_io_group (struct request_queue *q,int rw) +{ + struct hlist_head *head = &q->elevator->efqd.group_list; + struct hlist_node *n; + struct io_group *iog; + + hlist_for_each_entry(iog, n, head, elv_data_node) { + struct io_group_nrq *nrq = &iog->nrq; + if (nrq->count[rw] >= nrq->nr_requests) + return iog; + } + return NULL; +} +void io_group_init_entity (struct io_cgroup *iocg,struct io_group *iog) { struct io_entity *entity = &iog->entity;@@ -934,6 +1039,12 @@ void io_group_init_entity(struct io_cgroup *iocg, struct io_group *iog) entity->my_sched_data = &iog->sched_data; } +static void io_group_init_nrq (struct request_queue *q,struct io_group_nrq *nrq) +{ + nrq->nr_requests = q->nr_requests; + io_group_nrq_congestion_threshold(nrq); +} +void io_group_set_parent (struct io_group *iog,struct io_group *parent) { struct io_entity *entity;@@ -1053,6 +1164,8 @@ struct io_group *io_group_chain_alloc(struct request_queue *q, void *key, io_group_init_entity(iocg, iog); iog->my_entity = &iog->entity; + io_group_init_nrq(q, &iog->nrq); + if (leaf == NULL) { leaf = iog; prev = leaf;@@ -1176,7 +1289,7 @@ struct io_group *io_find_alloc_group(struct request_queue *q,* Generic function to make sure cgroup hierarchy is all setup once a request * from a cgroup is received by the io scheduler. */ -struct io_group *io_get_io_group (struct request_queue *q) +struct io_group *__io_get_io_group (struct request_queue *q) { struct cgroup *cgroup; struct io_group *iog;@@ -1192,6 +1305,19 @@ struct io_group *io_get_io_group(struct request_queue *q) return iog; } +struct io_group *io_get_io_group (struct request_queue *q) +{ + struct io_group *iog; + unsigned long flags; + + spin_lock_irqsave(q->queue_lock, flags); + iog = __io_get_io_group(q); + spin_unlock_irqrestore(q->queue_lock, flags); + BUG_ON(!iog); + + return iog; +} +void io_free_root_group (struct elevator_queue *e) { struct io_cgroup *iocg = &io_root_cgroup;@@ -1220,6 +1346,7 @@ struct io_group *io_alloc_root_group(struct request_queue *q, iog->entity.parent = NULL; for (i = 0; i < IO_IOPRIO_CLASSES; i++) iog->sched_data.service_tree[i] = IO_SERVICE_TREE_INIT; + io_group_init_nrq(q, &iog->nrq); iocg = &io_root_cgroup; spin_lock_irq(&iocg->lock);@@ -1533,15 +1660,11 @@ void elv_fq_set_request_io_group(struct request_queue *q, struct request *rq) { struct io_group *iog; - unsigned long flags; /* Make sure io group hierarchy has been setup and also set the * io group to which rq belongs. Later we should make use of * bio cgroup patches to determine the io group */ - spin_lock_irqsave(q->queue_lock, flags); iog = io_get_io_group(q); - spin_unlock_irqrestore(q->queue_lock, flags); - BUG_ON(!iog); /* Store iog in rq. TODO: take care of referencing */ rq->iog = iog;block/elevator-fq.h +201
-1@@ -187,6 +187,22 @@ struct io_queue {#ifdef CONFIG_GROUP_IOSCHED /** +* struct io_group_nrq - structure to store allocated requests info +* @nr_requests: maximun num of requests for the io_group +* @nr_congestion_on: threshold to determin the io_group is cogested. +* @nr_congestion_off: threshold to determin the io_group is not congested. +* @count: num of allocated requests. +* +* All fields are protected by queue_lock. +*/ +struct io_group_nrq { + unsigned long nr_requests; + unsigned int nr_congestion_on; + unsigned int nr_congestion_off; + int count[2]; +}; + +/** * struct bfq_group - per (device, cgroup) data structure. * @entity: schedulable entity to insert into the parent group sched_data. * @sched_data: own sched_data, to contain child entities (they may be @@ -235,6 +251,8 @@ struct io_group { /* Single ioq per group, used for noop, deadline, anticipatory */ struct io_queue *ioq; + + struct io_group_nrq nrq; };/** @@ -469,6 +487,11 @@ extern int elv_fq_set_request_ioq(struct request_queue *q, struct request *rq,extern void elv_fq_unset_request_ioq (struct request_queue *q, struct request *rq);extern struct io_queue *elv_lookup_ioq_current (struct request_queue *q); +extern void io_group_set_nrq_all (struct request_queue *q,int nr, + int *congested, int *full); +extern struct io_group *io_congested_io_group (struct request_queue *q,int rw); +extern struct io_group *io_full_io_group (struct request_queue *q,int rw); +extern struct io_group *__io_get_io_group (struct request_queue *q);/* Returns single ioq associated with the io group. */ static inline struct io_queue *io_group_ioq (struct io_group *iog)@@ -486,6 +509,52 @@ static inline void io_group_set_ioq(struct io_group *iog, struct io_queue *ioq) iog->ioq = ioq; } +static inline struct io_group *io_request_io_group (struct request *rq) +{ + return rq->iog; +} + +static inline unsigned long io_group_nr_requests (struct io_group *iog) +{ + BUG_ON(!iog); + return iog->nrq.nr_requests; +} + +static inline int io_group_inc_count (struct io_group *iog,int rw) +{ + BUG_ON(!iog); + return iog->nrq.count[rw]++; +} + +static inline int io_group_dec_count (struct io_group *iog,int rw) +{ + BUG_ON(!iog); + return iog->nrq.count[rw]--; +} + +static inline int io_group_count (struct io_group *iog,int rw) +{ + BUG_ON(!iog); + return iog->nrq.count[rw]; +} + +static inline int io_group_congestion_on (struct io_group *iog,int rw) +{ + BUG_ON(!iog); + return iog->nrq.count[rw] + 1 >= iog->nrq.nr_congestion_on; +} + +static inline int io_group_congestion_off (struct io_group *iog,int rw) +{ + BUG_ON(!iog); + return iog->nrq.count[rw] < iog->nrq.nr_congestion_off; +} + +static inline int io_group_full (struct io_group *iog,int rw) +{ + BUG_ON(!iog); + return iog->nrq.count[rw] + 1 >= iog->nrq.nr_requests; +}#else /* !GROUP_IOSCHED */ /* * No ioq movement is needed in case of flat setup. root io group gets cleaned @@ -537,6 +606,71 @@ static inline struct io_queue *elv_lookup_ioq_current(struct request_queue *q) return NULL; } +static inline void io_group_set_nrq_all (struct request_queue *q,int nr, + int *congested, int *full) +{ + int i; + for (i=0; i<2; i++) + *(congested + i) = *(full + i) = 0; +} + +static inline struct io_group * +io_congested_io_group (struct request_queue *q,int rw) +{ + return NULL; +} + +static inline struct io_group * +io_full_io_group (struct request_queue *q,int rw) +{ + return NULL; +} + +static inline struct io_group *__io_get_io_group (struct request_queue *q) +{ + return NULL; +} + +static inline struct io_group *io_request_io_group (struct request *rq) +{ + return NULL; +} + +static inline unsigned long io_group_nr_requests (struct io_group *iog) +{ + return 0; +} + +static inline int io_group_inc_count (struct io_group *iog,int rw) +{ + return 0; +} + +static inline int io_group_dec_count (struct io_group *iog,int rw) +{ + return 0; +} + +static inline int io_group_count (struct io_group *iog,int rw) +{ + return 0; +} + +static inline int io_group_congestion_on (struct io_group *iog,int rw) +{ + return 0; +} + +static inline int io_group_congestion_off (struct io_group *iog,int rw) +{ + return 1; +} + +static inline int io_group_full (struct io_group *iog,int rw) +{ + return 0; +} +#endif /* GROUP_IOSCHED */ /* Functions used by blksysfs.c */ @@ -589,6 +723,9 @@ extern void elv_free_ioq(struct io_queue *ioq);#else /* CONFIG_ELV_FAIR_QUEUING */ +struct io_group { +}; +static inline int elv_init_fq_data (struct request_queue *q, struct elevator_queue *e) {@@ -655,5 +792,69 @@ static inline struct io_queue *elv_lookup_ioq_current(struct request_queue *q) return NULL; } +static inline void io_group_set_nrq_all (struct request_queue *q,int nr, + int *congested, int *full) +{ + int i; + for (i=0; i<2; i++) + *(congested + i) = *(full + i) = 0; +} + +static inline struct io_group * +io_congested_io_group (struct request_queue *q,int rw) +{ + return NULL; +} + +static inline struct io_group * +io_full_io_group (struct request_queue *q,int rw) +{ + return NULL; +} + +static inline struct io_group *__io_get_io_group (struct request_queue *q) +{ + return NULL; +} + +static inline struct io_group *io_request_io_group (struct request *rq) +{ + return NULL; +} + +static inline unsigned long io_group_nr_requests (struct io_group *iog) +{ + return 0; +} + +static inline int io_group_inc_count (struct io_group *iog,int rw) +{ + return 0; +} + +static inline int io_group_dec_count (struct io_group *iog,int rw) +{ + return 0; +} + +static inline int io_group_count (struct io_group *iog,int rw) +{ + return 0; +} + +static inline int io_group_congestion_on (struct io_group *iog,int rw) +{ + return 0; +} + +static inline int io_group_congestion_off (struct io_group *iog,int rw) +{ + return 1; +} + +static inline int io_group_full (struct io_group *iog,int rw) +{ + return 0; +}#endif /* CONFIG_ELV_FAIR_QUEUING */ #endif /* _BFQ_SCHED_H */ --1.5.4.3
--
IKEDA, Munehiro
NEC Corporation of America
m-ikeda-MDRzhb/z0dd8UrSeD/[email protected]
Re: IO Controller per cgroup request descriptors (Re: [PATCH 01/10] Documentation)
In reply to Vivek GoyalQuoted 39 lines · Vivek Goyal
Quoted 30 lines · Munehiro
Quoted 25 lines · Vivek Goyal
Quoted 7 lines
Quoted 6 lines
+TODO
+==== +- Lots of cleanups, testing, bug fixing, optimizations, benchmarking etc... +- Convert cgroup ioprio to notion of weight. +- Anticipatory code will need more work. It is not working properly currently+ and needs more thought.
What are the problems with the code?
Have not got a chance to look into the issues in detail yet. Just a crude run saw drop in performance. Will debug it later the moment I have got async writes handled...
Quoted 6 lines
Quoted 1 line
+- Use of bio-cgroup patches.
I saw these posted as well
Quoted 2 lines
+- Use of Nauman's per cgroup request descriptor patches. +
More details would be nice, I am not sure I understand
Currently the number of request descriptors which can be allocated per device/request queue are fixed by a sysfs tunable (
q->nr_requests). So if there is lots of IO going on from one cgroup then it will consume all the available request descriptors and other cgroup might starve and not get its fair share.Quoted 3 lines
Hence we also need to introduce the notion of request descriptor limit per cgroup so that if request descriptors from one group are exhausted, then it does not impact the IO of other cgroup.
Quoted 7 lines
Unfortunately I couldn't find and I've never seen the Nauman's patches. So I tried to make a patch below against this todo. The reason why I'm posting this despite this is just a quick and ugly hack (and it might be a reinvention of wheel) is that I would like to discuss how we should define the limitation of requests per cgroup.
This patch should be applied on Vivek's I/O controller patches posted on Mar 11.
Quoted 4 lines
Hi IKEDA,
Sorry for the confusion here. Actually Nauman had sent a patch to select group of people who were initially copied on the mail thread.
I am sorry about that. Since I dropped my whole patch set in favor of Vivek's stuff, this stuff fell through the cracks.
Quoted 33 lines
Quoted 9 lines
This patch temporarily distribute
q->nr_requeststo each cgroup. I think the number should be weighted like BFQ's budget. But in this case, if the hierarchy of cgroup is deep, leaf cgroups are allowed to allocate very few numbers of requests. I don't think this is reasonable...but I don't have specific idea to solve this problem. Does anyone have the good idea?Thanks for the patch. Yes, ideally one would expect the request descriptor to be allocated also in proportion to the weight but I guess that would become very comlicated.
In terms of simpler things, two thoughts come to mind.
- First approach is to make
q->nr_requestsper group. So every group is entitled forq->nr_requestsas set by the user. This is what your patch seems to have done.I had some concerns with this approach. First of all it does not seem to have an upper bound on number of request descriptors allocated per queue because if a user creates more cgroups, total number of request descriptors increase.
- Second approach can be that we retain the meaning of
q->nr_requestswhich defines the total number of request descriptors on the queue (with the exception of 50% more descriptors for batching processes). And we define a new per group limitq->nr_group_requestswhich defines how many requests per group can be assigned. Soq->nr_requestsdefines total pool size on the queue andq->nr_group_requestswill define how many requests each group can allocate out of that pool.
Quoted 8 lines
Here the issue is that a user shall have to balance the
q->nr_group_requestsandq->nr_requestsproperly.To experiment, I have implemented the second approach. I am attaching the patch which is in my current tree. It probably will not apply on my march 11 posting as since then patches have changed. But posting it here so that at least it will give an idea behind the thought process.
Ideas are welcome...
I had started with the first option, but the second option sounds good too. But one problem that comes to mind is how we deal with hierarchies? The sys admin can limit the root level cgroups to specific number of request descriptors, but if applications running in a cgroup are allowed to create their own cgroups, then the total request descriptors of all child cgroups should be capped by the number assigned to parent cgroups.
Quoted 1,385 lines
Thanks
Viveko Currently a request queue has got fixed number of request descriptors for sync and async requests. Once the request descriptors are consumed, new processes are put to sleep and they effectively become serialized. Because sync and async queues are separate, async requests don't impact sync ones but if one is looking for fairness between async requests, that is not achievable if request queue descriptors become bottleneck.
o Make request descriptor's per io group so that if there is lots of IO going on in one cgroup, it does not impact the IO of other group.
o This patch implements the per cgroup request descriptors. request pool per queue is still common but every group will have its own wait list and its own count of request descriptors allocated to that group for sync and async queues. So effectively
request_listbecomes per io group property and not a global request queue feature.o Currently one can define
q->nr_requeststo limit request descriptors allocated for the queue. Now there is another tunableq->nr_group_requestswhich controls the requests descriptr limit per group.q->nr_requestssupercedesq->nr_group_requeststo make sure if there are lots of groups present, we don't end up allocating too many request descriptors on the queue.o Issues: Currently notion of congestion is per queue. With per group request descriptor it is possible that queue is not congested but the group bio will go into is congested.
Signed-off-by Nauman Rafique <[email protected]>Signed-off-by Vivek Goyal <[email protected]>---
block/blk-core.c | 216 ++++++++++++++++++++++++++++++++++---------------block/blk-settings.c| 3
block/blk-sysfs.c | 57 ++++++++++--
block/elevator-fq.c | 15 +++
block/elevator-fq.h | 8 +
block/elevator.c | 6 -
include/linux/blkdev.h | 62 +++++++++++++-7 files changed, 287 insertions(+), 80 deletions(-)Index:
linux9/include/linux/blkdev.h
===================================================================include/linux/blkdev.h 2009-04-30 16:18:29.000000000 -0400 renamed from include/linux/blkdev.h 2009-04-30 15:43:53.000000000 -0400 +36
-2@@ -32,21 +32,51 @@ struct request; struct sg_io_hdr; #define BLKDEV_MIN_RQ 4 + +#ifdef CONFIG_GROUP_IOSCHED +#define BLKDEV_MAX_RQ 256 /* Default maximum */ +#define BLKDEV_MAX_GROUP_RQ 64 /* Default maximum */ +#else #define BLKDEV_MAX_RQ 128 /* Default maximum */ +/* + * This is eqivalent to case of only one group present (root group). Let + * it consume all the request descriptors available on the queue . + */ +#define BLKDEV_MAX_GROUP_RQ BLKDEV_MAX_RQ /* Default maximum */ +#endif struct request; typedef void (rq_end_io_fn)(struct request *, int); struct request_list { /* - * count[], starved[], and wait[] are indexed by + * count[], starved and wait[] are indexed by * BLK_RW_SYNC/BLK_RW_ASYNC */ int count[2]; int starved[2]; + wait_queue_head_t wait[2]; +}; + +/* + * This data structures keeps track of mempool of requests for the queue + * and some overall statistics. + */ +struct request_data { + /* + * Per queue request descriptor count. This is in addition to per + * cgroup count + */ + int count[2]; int elvpriv; mempool_t *rq_pool; - wait_queue_head_t wait[2]; + int starved; + /* + * Global list for starved tasks. A task will be queued here if + * it could not allocate request descriptor and the associated + * group request list does not have any requests pending. + */ + wait_queue_head_t starved_wait; }; /*@@ -251,6 +281,7 @@ struct request { #ifdef CONFIG_GROUP_IOSCHED /* io group request belongs to */ struct io_group *iog; + struct request_list *rl; #endif /* GROUP_IOSCHED */ #endif /* ELV_FAIR_QUEUING */ };@@ -340,6 +371,9 @@ struct request_queue */ struct request_list rq; + /* Contains request pool and other data like starved data */ + struct request_data rq_data; + request_fn_proc *request_fn; make_request_fn *make_request_fn; prep_rq_fn *prep_rq_fn;@@ -402,6 +436,8 @@ struct request_queue * queue settings */unsigned long
nr_requests; /* Max # of requests */
+ /* Max # of per io group requests */
+ unsigned longnr_group_requests;
unsigned int nr_congestion_on;
unsigned int nr_congestion_off;
unsigned int nr_batching;include/linux/blkdev.h 2009-04-30 16:18:29.000000000 -0400 renamed from include/linux/blkdev.h 2009-04-30 15:43:53.000000000 -0400 +22
-0@@ -773,6 +809,28 @@ extern int scsi_cmd_ioctl(struct request extern int sg_scsi_ioctl(struct request_queue *, struct gendisk *, fmode_t, struct scsi_ioctl_command __user *); +extern void blk_init_request_list(struct request_list *rl); + +static inline struct request_list *blk_get_request_list(struct request_queue *q, + struct bio *bio) +{ +#ifdef CONFIG_GROUP_IOSCHED + return io_group_get_request_list(q, bio); +#else + return &q->rq; +#endif +} + +static inline struct request_list *rq_rl(struct request_queue *q, + struct request *rq) +{ +#ifdef CONFIG_GROUP_IOSCHED + return rq->rl; +#else + return blk_get_request_list(q, NULL); +#endif +} + /* * Temporary export, until SCSI gets fixed up. */Index:
linux9/block/elevator.c
===================================================================block/elevator.c 2009-04-30 16:18:29.000000000 -0400 renamed from block/elevator.c 2009-04-30 16:17:53.000000000 -0400 +3
-3@@ -664,7 +664,7 @@ void elv_quiesce_start(struct request_qu * make sure we don't have any requests in flight */ elv_drain_elevator(q); - while (q->rq.elvpriv) { + while (q->rq_data.elvpriv) { blk_start_queueing(q); spin_unlock_irq(q->queue_lock); msleep(10);@@ -764,8 +764,8 @@ void elv_insert(struct request_queue *q, } if (unplug_it && blk_queue_plugged(q)) { - int nrq = q->rq.count[BLK_RW_SYNC] + q->rq.count[BLK_RW_ASYNC] - - q->in_flight; + int nrq = q->rq_data.count[BLK_RW_SYNC] + + q->rq_data.count[BLK_RW_ASYNC] - q->in_flight; if (nrq >= q->unplug_thresh) __generic_unplug_device(q);Index:
linux9/block/blk-core.c
===================================================================block/blk-core.c 2009-04-30 16:18:29.000000000 -0400 renamed from block/blk-core.c 2009-04-30 16:17:53.000000000 -0400 +22
-8@@ -480,20 +480,31 @@ void blk_cleanup_queue(struct request_qu } EXPORT_SYMBOL(blk_cleanup_queue); -static int blk_init_free_list(struct request_queue *q) +void blk_init_request_list(struct request_list *rl) { - struct request_list *rl = &q->rq; rl->count[BLK_RW_SYNC] = rl->count[BLK_RW_ASYNC] = 0; - rl->starved[BLK_RW_SYNC] = rl->starved[BLK_RW_ASYNC] = 0; - rl->elvpriv = 0; init_waitqueue_head(&rl->wait[BLK_RW_SYNC]); init_waitqueue_head(&rl->wait[BLK_RW_ASYNC]); +} - rl->rq_pool = mempool_create_node(BLKDEV_MIN_RQ, mempool_alloc_slab, - mempool_free_slab, request_cachep, q->node); +static int blk_init_free_list(struct request_queue *q) +{ +#ifndef CONFIG_GROUP_IOSCHED + struct request_list *rl = blk_get_request_list(q, NULL); + + /* + * In case of group scheduling, request list is inside the associated + * group and when that group is instanciated, it takes care of + * initializing the request list also. + */ + blk_init_request_list(rl); +#endif + q->rq_data.rq_pool = mempool_create_node(BLKDEV_MIN_RQ, + mempool_alloc_slab, mempool_free_slab, + request_cachep, q->node); - if (!rl->rq_pool) + if (!q->rq_data.rq_pool) return -ENOMEM; return 0;@@ -590,6 +601,9 @@ blk_init_queue_node(request_fn_proc *rfn return NULL; } + /* init starved waiter wait queue */ + init_waitqueue_head(&q->rq_data.starved_wait); + /* * if caller didn't supply a lock, they get per-queue locking with * our embedded lock@@ -639,14 +653,14 @@ static inline void blk_free_request(stru { if (rq->cmd_flags & REQ_ELVPRIV) elv_put_request(q, rq); - mempool_free(rq, q->rq.rq_pool); + mempool_free(rq, q->rq_data.rq_pool); }static struct request *
blk_alloc_request(structrequest_queue*q, struct bio *bio, int rw, int priv,gfp_tgfp_mask)
{
- struct request *rq =mempool_alloc(q->rq.rq_pool, gfp_mask);
+ struct request *rq =mempool_alloc(q->rq_data.rq_pool, gfp_mask);if (!rq)
return NULL;block/blk-core.c 2009-04-30 16:18:29.000000000 -0400 renamed from block/blk-core.c 2009-04-30 16:17:53.000000000 -0400 +81
-42@@ -657,7 +671,7 @@ blk_alloc_request(struct request_queue * if (priv) { if (unlikely(elv_set_request(q, rq, bio, gfp_mask))) { - mempool_free(rq, q->rq.rq_pool); + mempool_free(rq, q->rq_data.rq_pool); return NULL; } rq->cmd_flags |= REQ_ELVPRIV;@@ -700,18 +714,18 @@ static void ioc_set_batching(struct requ ioc->last_waited = jiffies; } -static void __freed_request(struct request_queue *q, int sync) +static void __freed_request(struct request_queue *q, int sync, + struct request_list *rl) { - struct request_list *rl = &q->rq; - - if (rl->count[sync] < queue_congestion_off_threshold(q)) + if (q->rq_data.count[sync] < queue_congestion_off_threshold(q)) blk_clear_queue_congested(q, sync); - if (rl->count[sync] + 1 <= q->nr_requests) { + if (q->rq_data.count[sync] + 1 <= q->nr_requests) + blk_clear_queue_full(q, sync); + + if (rl->count[sync] + 1 <= q->nr_group_requests) { if (waitqueue_active(&rl->wait[sync])) wake_up(&rl->wait[sync]); - - blk_clear_queue_full(q, sync); } }@@ -719,18 +733,29 @@ static void __freed_request(struct reque * A request has just been released. Account for it, update the full and * congestion status, wake up any waiters. Called under q->queue_lock. */ -static void freed_request(struct request_queue *q, int sync, int priv) +static void freed_request(struct request_queue *q, int sync, int priv, + struct request_list *rl) { - struct request_list *rl = &q->rq; - + BUG_ON(!rl->count[sync]); rl->count[sync]--; + + BUG_ON(!q->rq_data.count[sync]); + q->rq_data.count[sync]--; + if (priv) - rl->elvpriv--; + q->rq_data.elvpriv--; - __freed_request(q, sync); + __freed_request(q, sync, rl); if (unlikely(rl->starved[sync ^ 1])) - __freed_request(q, sync ^ 1); + __freed_request(q, sync ^ 1, rl); + + /* Wake up the starved process on global list, if any */ + if (unlikely(q->rq_data.starved)) { + if (waitqueue_active(&q->rq_data.starved_wait)) + wake_up(&q->rq_data.starved_wait); + q->rq_data.starved--; + } } /*@@ -739,10 +764,9 @@ static void freed_request(struct request * Returns !NULL on success, with queue_lock *not held*. */ static struct request *get_request(struct request_queue *q, int rw_flags, - struct bio *bio, gfp_t gfp_mask) + struct bio *bio, gfp_t gfp_mask, struct request_list *rl) { struct request *rq = NULL; - struct request_list *rl = &q->rq; struct io_context *ioc = NULL; const bool is_sync = rw_is_sync(rw_flags) != 0; int may_queue, priv;@@ -751,31 +775,38 @@ static struct request *get_request(struc if (may_queue == ELV_MQUEUE_NO) goto rq_starved; - if (rl->count[is_sync]+1 >= queue_congestion_on_threshold(q)) { - if (rl->count[is_sync]+1 >= q->nr_requests) { - ioc = current_io_context(GFP_ATOMIC, q->node); - /* - * The queue will fill after this allocation, so set - * it as full, and mark this process as "batching". - * This process will be allowed to complete a batch of - * requests, others will be blocked. - */ - if (!blk_queue_full(q, is_sync)) { - ioc_set_batching(q, ioc); - blk_set_queue_full(q, is_sync); - } else { - if (may_queue != ELV_MQUEUE_MUST - && !ioc_batching(q, ioc)) { - /* - * The queue is full and the allocating - * process is not a "batcher", and not - * exempted by the IO scheduler - */ - goto out; - } + if (q->rq_data.count[is_sync]+1 >= queue_congestion_on_threshold(q)) + blk_set_queue_congested(q, is_sync); + + /* + * Looks like there is no user of queue full now. + * Keeping it for time being. + */ + if (q->rq_data.count[is_sync]+1 >= q->nr_requests) + blk_set_queue_full(q, is_sync); + + if (rl->count[is_sync]+1 >= q->nr_group_requests) { + ioc = current_io_context(GFP_ATOMIC, q->node); + /* + * The queue request descriptor group will fill after this + * allocation, so set + * it as full, and mark this process as "batching". + * This process will be allowed to complete a batch of + * requests, others will be blocked. + */ + if (rl->count[is_sync] <= q->nr_group_requests) + ioc_set_batching(q, ioc); + else { + if (may_queue != ELV_MQUEUE_MUST + && !ioc_batching(q, ioc)) { + /* + * The queue is full and the allocating + * process is not a "batcher", and not + * exempted by the IO scheduler + */ + goto out; } } - blk_set_queue_congested(q, is_sync); } /*@@ -783,19 +814,41 @@ static struct request *get_request(struc * limit of requests, otherwise we could have thousands of requests * allocated with any setting of ->nr_requests */ - if (rl->count[is_sync] >= (3 * q->nr_requests / 2)) + + if (q->rq_data.count[is_sync] >= (3 * q->nr_requests / 2)) + goto out; + + /* + * Allocation of request is allowed from queue perspective. Now check + * from per group request list + */ + + if (rl->count[is_sync] >= (3 * q->nr_group_requests / 2)) goto out; rl->count[is_sync]++; rl->starved[is_sync] = 0; + q->rq_data.count[is_sync]++; + priv = !test_bit(QUEUE_FLAG_ELVSWITCH, &q->queue_flags); if (priv) - rl->elvpriv++; + q->rq_data.elvpriv++; spin_unlock_irq(q->queue_lock); rq = blk_alloc_request(q, bio, rw_flags, priv, gfp_mask); + +#ifdef CONFIG_GROUP_IOSCHED + if (rq) { + /* + * TODO. Implement group reference counting and take the + * reference to the group to make sure group hence request + * list does not go away till rq finishes. + */ + rq->rl = rl; + } +#endif if (unlikely(!rq)) { /* * Allocation failed presumably due to memory. Undo anything@@ -805,7 +858,7 @@ static struct request *get_request(struc * wait queue, but this is pretty rare. */ spin_lock_irq(q->queue_lock); - freed_request(q, is_sync, priv); + freed_request(q, is_sync, priv, rl); /* * in the very unlikely event that allocation failed and no@@ -815,10 +868,26 @@ static struct request *get_request(struc * rq mempool into READ and WRITE */rq_starved:
- if (unlikely(rl->count[is_sync] == 0))
- rl->starved[is_sync] = 1;
-
- goto out;
+ if (unlikely(rl->count[is_sync] == 0)) {
+ /*
+ * If there is a request pending in other direction
+ * in same io group, then set the starved flag of
+ * the group request list. Otherwise, we need to
+ * make this process sleep in global starved list
+ * to make sure it will not sleep indefinitely.
+ */
+ if (rl->count[is_sync ^ 1] != 0) {
+ rl->starved[is_sync] = 1;
+ goto out;
+ } else {
+ /*
+ * It indicates to calling function to put
+ * task on global starved list. Not the best
+ * way
+ */
+ returnERR_PTR(-ENOMEM);
+ }
+ }
}/*
block/blk-core.c 2009-04-30 16:18:29.000000000 -0400 renamed from block/blk-core.c 2009-04-30 16:17:53.000000000 -0400 +29
-8@@ -846,15 +915,29 @@ static struct request *get_request_wait( { const bool is_sync = rw_is_sync(rw_flags) != 0; struct request *rq; + struct request_list *rl = blk_get_request_list(q, bio); - rq = get_request(q, rw_flags, bio, GFP_NOIO); - while (!rq) { + rq = get_request(q, rw_flags, bio, GFP_NOIO, rl); + while (!rq || (IS_ERR(rq) && PTR_ERR(rq) == -ENOMEM)) { DEFINE_WAIT(wait); struct io_context *ioc; - struct request_list *rl = &q->rq; - prepare_to_wait_exclusive(&rl->wait[is_sync], &wait, - TASK_UNINTERRUPTIBLE); + if (IS_ERR(rq) && PTR_ERR(rq) == -ENOMEM) { + /* + * Task failed allocation and needs to wait and + * try again. There are no requests pending from + * the io group hence need to sleep on global + * wait queue. Most likely the allocation failed + * because of memory issues. + */ + + q->rq_data.starved++; + prepare_to_wait_exclusive(&q->rq_data.starved_wait, + &wait, TASK_UNINTERRUPTIBLE); + } else { + prepare_to_wait_exclusive(&rl->wait[is_sync], &wait, + TASK_UNINTERRUPTIBLE); + } trace_block_sleeprq(q, bio, rw_flags & 1);@@ -874,7 +957,12 @@ static struct request *get_request_wait( spin_lock_irq(q->queue_lock); finish_wait(&rl->wait[is_sync], &wait); - rq = get_request(q, rw_flags, bio, GFP_NOIO); + /* + * After the sleep check the rl again in case cgrop bio + * belonged to is gone and it is mapped to root group now + */ + rl = blk_get_request_list(q, bio); + rq = get_request(q, rw_flags, bio, GFP_NOIO, rl); }; return rq;@@ -883,6 +971,7 @@ static struct request *get_request_wait( struct request *blk_get_request(struct request_queue *q, int rw, gfp_t gfp_mask) { struct request *rq; + struct request_list *rl = blk_get_request_list(q, NULL); BUG_ON(rw != READ && rw != WRITE);@@ -890,7 +979,7 @@ struct request *blk_get_request(struct r if (gfp_mask & __GFP_WAIT) { rq = get_request_wait(q, rw, NULL); } else { - rq = get_request(q, rw, NULL, gfp_mask); + rq = get_request(q, rw, NULL, gfp_mask, rl); if (!rq) spin_unlock_irq(q->queue_lock); }@@ -1073,12 +1162,13 @@ void __blk_put_request(struct request_qu if (req->cmd_flags & REQ_ALLOCED) { int is_sync = rq_is_sync(req) != 0; int priv = req->cmd_flags & REQ_ELVPRIV; + struct request_list *rl = rq_rl(q, req); BUG_ON(!list_empty(&req->queuelist)); BUG_ON(!hlist_unhashed(&req->hash)); blk_free_request(q, req); - freed_request(q, is_sync, priv); + freed_request(q, is_sync, priv, rl); } } EXPORT_SYMBOL_GPL(__blk_put_request);Index:
linux9/block/blk-sysfs.c
===================================================================block/blk-sysfs.c 2009-04-30 16:18:29.000000000 -0400 renamed from block/blk-sysfs.c 2009-04-30 16:18:27.000000000 -0400 +0
-0@@ -38,7 +38,7 @@ static ssize_t queue_requests_show(strucstatic
ssize_t
queue_requests_store(structrequest_queue*q, const char *page,size_tcount) {
- structrequest_list*rl =&q->rq;
+ structrequest_list*rl =blk_get_request_list(q, NULL); unsigned long nr;
int ret =queue_var_store(&nr, page, count);
if (nr <BLKDEV_MIN_RQ)block/blk-sysfs.c 2009-04-30 16:18:29.000000000 -0400 renamed from block/blk-sysfs.c 2009-04-30 16:18:27.000000000 -0400 +42
-8@@ -48,32 +48,55 @@ queue_requests_store(struct request_queu q->nr_requests = nr; blk_queue_congestion_threshold(q); - if (rl->count[BLK_RW_SYNC] >= queue_congestion_on_threshold(q)) + if (q->rq_data.count[BLK_RW_SYNC] >= queue_congestion_on_threshold(q)) blk_set_queue_congested(q, BLK_RW_SYNC); - else if (rl->count[BLK_RW_SYNC] < queue_congestion_off_threshold(q)) + else if (q->rq_data.count[BLK_RW_SYNC] < + queue_congestion_off_threshold(q)) blk_clear_queue_congested(q, BLK_RW_SYNC); - if (rl->count[BLK_RW_ASYNC] >= queue_congestion_on_threshold(q)) + if (q->rq_data.count[BLK_RW_ASYNC] >= queue_congestion_on_threshold(q)) blk_set_queue_congested(q, BLK_RW_ASYNC); - else if (rl->count[BLK_RW_ASYNC] < queue_congestion_off_threshold(q)) + else if (q->rq_data.count[BLK_RW_ASYNC] < + queue_congestion_off_threshold(q)) blk_clear_queue_congested(q, BLK_RW_ASYNC); - if (rl->count[BLK_RW_SYNC] >= q->nr_requests) { + if (q->rq_data.count[BLK_RW_SYNC] >= q->nr_requests) { blk_set_queue_full(q, BLK_RW_SYNC); - } else if (rl->count[BLK_RW_SYNC]+1 <= q->nr_requests) { + } else if (q->rq_data.count[BLK_RW_SYNC]+1 <= q->nr_requests) { blk_clear_queue_full(q, BLK_RW_SYNC); wake_up(&rl->wait[BLK_RW_SYNC]); } - if (rl->count[BLK_RW_ASYNC] >= q->nr_requests) { + if (q->rq_data.count[BLK_RW_ASYNC] >= q->nr_requests) { blk_set_queue_full(q, BLK_RW_ASYNC); - } else if (rl->count[BLK_RW_ASYNC]+1 <= q->nr_requests) { + } else if (q->rq_data.count[BLK_RW_ASYNC]+1 <= q->nr_requests) { blk_clear_queue_full(q, BLK_RW_ASYNC); wake_up(&rl->wait[BLK_RW_ASYNC]); } spin_unlock_irq(q->queue_lock); return ret; } +#ifdef CONFIG_GROUP_IOSCHED +static ssize_t queue_group_requests_show(struct request_queue *q, char *page) +{ + return queue_var_show(q->nr_group_requests, (page)); +} + +static ssize_t +queue_group_requests_store(struct request_queue *q, const char *page, + size_t count) +{ + unsigned long nr; + int ret = queue_var_store(&nr, page, count); + if (nr < BLKDEV_MIN_RQ) + nr = BLKDEV_MIN_RQ; + + spin_lock_irq(q->queue_lock); + q->nr_group_requests = nr; + spin_unlock_irq(q->queue_lock); + return ret; +} +#endif static ssize_t queue_ra_show(struct request_queue *q, char *page) {@@ -228,6 +251,14 @@ static struct queue_sysfs_entry queue_re .store = queue_requests_store, }; +#ifdef CONFIG_GROUP_IOSCHED +static struct queue_sysfs_entry queue_group_requests_entry = { + .attr = {.name = "nr_group_requests", .mode = S_IRUGO | S_IWUSR }, + .show = queue_group_requests_show, + .store = queue_group_requests_store, +}; +#endif + static struct queue_sysfs_entry queue_ra_entry = { .attr = {.name = "read_ahead_kb", .mode = S_IRUGO | S_IWUSR }, .show = queue_ra_show,@@ -308,6 +339,9 @@ static struct queue_sysfs_entry queue_sl static struct attribute *default_attrs[] = { &queue_requests_entry.attr, +#ifdef CONFIG_GROUP_IOSCHED + &queue_group_requests_entry.attr, +#endif &queue_ra_entry.attr, &queue_max_hw_sectors_entry.attr, &queue_max_sectors_entry.attr,@@ -389,12 +423,11 @@ static void blk_release_queue(struct kob {struct
request_queue*q =
container_of(kobj, structrequest_queue, kobj);
- structrequest_list*rl =&q->rq;blk_sync_queue(q);
- if (
rl->rq_pool)
-mempool_destroy(rl->rq_pool);
+ if (q->rq_data.rq_pool)
+mempool_destroy(q->rq_data.rq_pool);if (
q->queue_tags)
__blk_queue_free_tags(q);
Index:linux9/block/blk-settings.c
===================================================================block/blk-settings.c 2009-04-30 16:18:29.000000000 -0400 renamed from block/blk-settings.c 2009-04-30 15:43:53.000000000 -0400 +3
-0@@ -123,6 +123,9 @@ void blk_queue_make_request(struct reque * set defaults */ q->nr_requests = BLKDEV_MAX_RQ; +#ifdef CONFIG_GROUP_IOSCHED + q->nr_group_requests = BLKDEV_MAX_GROUP_RQ; +#endif blk_queue_max_phys_segments(q, MAX_PHYS_SEGMENTS); blk_queue_max_hw_segments(q, MAX_HW_SEGMENTS); blk_queue_segment_boundary(q, BLK_SEG_BOUNDARY_MASK);Index:
linux9/block/elevator-fq.c
===================================================================block/elevator-fq.c 2009-04-30 16:18:29.000000000 -0400 renamed from block/elevator-fq.c 2009-04-30 16:18:27.000000000 -0400 +15
-0@@ -954,6 +954,17 @@ struct io_cgroup *cgroup_to_io_cgroup(st struct io_cgroup, css); } +struct request_list *io_group_get_request_list(struct request_queue *q, + struct bio *bio) +{ + struct io_group *iog; + + iog = io_get_io_group_bio(q, bio, 1); + BUG_ON(!iog); +out: + return &iog->rl; +} + /* * Search the bfq_group for bfqd into the hash table (by now only a list) * of bgrp. Must be called under rcu_read_lock().@@ -1203,6 +1214,8 @@ struct io_group *io_group_chain_alloc(st io_group_init_entity(iocg, iog); iog->my_entity = &iog->entity; + blk_init_request_list(&iog->rl); + if (leaf == NULL) { leaf = iog; prev = leaf;@@ -1446,6 +1459,8 @@ struct io_group *io_alloc_root_group(str for (i = 0; i < IO_IOPRIO_CLASSES; i++) iog->sched_data.service_tree[i] = IO_SERVICE_TREE_INIT; + blk_init_request_list(&iog->rl); + iocg = &io_root_cgroup; spin_lock_irq(&iocg->lock); rcu_assign_pointer(iog->key, key);Index:
linux9/block/elevator-fq.h
===================================================================block/elevator-fq.h 2009-04-30 16:18:29.000000000 -0400 renamed from block/elevator-fq.h 2009-04-30 16:18:27.000000000 -0400 +8
-0@@ -239,8 +239,14 @@ struct io_group { /* Single ioq per group, used for noop, deadline, anticipatory */ struct io_queue *ioq; + + /* request list associated with the group */ + struct request_list rl; }; +#define IOG_FLAG_READFULL 1 /* read queue has been filled */ +#define IOG_FLAG_WRITEFULL 2 /* write queue has been filled */ + /** * struct bfqio_cgroup - bfq cgroup data structure. * @css: subsystem state for bfq in the containing cgroup.@@ -517,6 +523,8 @@ extern void elv_fq_unset_request_ioq(str extern struct io_queue *elv_lookup_ioq_current(struct request_queue *q); extern struct io_queue *elv_lookup_ioq_bio(struct request_queue *q, struct bio *bio); +extern struct request_list *io_group_get_request_list(struct request_queue *q, + struct bio *bio); /* Returns single ioq associated with the io group. */ static inline struct io_queue *io_group_ioq(struct io_group *iog)Thanks
VivekQuoted 619 lines
Signed-off-by Munehiro "Muuhh" Ikeda <[email protected]>---
block/blk-core.c| 36 +++++++--block/blk-sysfs.c| 22 ++++--block/elevator-fq.c| 133 ++++++++++++++++++++++++++++++++--block/elevator-fq.h| 201 +++++++++++++++++++++++++++++++++++++++++++++++++++4 files changed, 371 insertions(+), 21 deletions(-)block/blk-core.c +28
-8@@ -705,11 +705,15 @@ static void ioc_set_batching(struct request_queue *q, struct io_context *ioc)static void __freed_request (struct request_queue *q,int rw) { struct request_list *rl = &q->rq; - - if (rl->count[rw] < queue_congestion_off_threshold(q)) + struct io_group *congested_iog, *full_iog; + + congested_iog = io_congested_io_group(q, rw); + if (rl->count[rw] < queue_congestion_off_threshold(q) && + !congested_iog) blk_clear_queue_congested(q, rw); - if (rl->count[rw] + 1 <= q->nr_requests) { + full_iog = io_full_io_group(q, rw); + if (rl->count[rw] + 1 <= q->nr_requests && !full_iog) { if (waitqueue_active(&rl->wait[rw])) wake_up(&rl->wait[rw]);@@ -721,13 +725,16 @@ static void __freed_request(struct request_queue *q, int rw)* A request has just been released. Account for it, update the full and * congestion status, wake up any waiters. Called under q->queue_lock. */ -static void freed_request (struct request_queue *q,int rw,int priv) +static void freed_request (struct request_queue *q,struct io_group *iog, + int rw, int priv) { struct request_list *rl = &q->rq; rl->count[rw]--; if (priv) rl->elvpriv--; + if (iog) + io_group_dec_count(iog, rw); __freed_request(q, rw);@@ -746,16 +753,21 @@ static struct request *get_request(struct request_queue *q, int rw_flags, { struct request *rq = NULL; struct request_list *rl = &q->rq; + struct io_group *iog; struct io_context *ioc = NULL; const int rw = rw_flags & 0x01; int may_queue, priv; + iog = __io_get_io_group(q); + may_queue = elv_may_queue(q, rw_flags); if (may_queue == ELV_MQUEUE_NO) goto rq_starved; - if (rl->count[rw]+1 >= queue_congestion_on_threshold(q)) { - if (rl->count[rw]+1 >= q->nr_requests) { + if (rl->count[rw]+1 >= queue_congestion_on_threshold(q) || + io_group_congestion_on(iog, rw)) { + if (rl->count[rw]+1 >= q->nr_requests || + io_group_full(iog, rw)) { ioc = current_io_context(GFP_ATOMIC, q->node); /** The queue will fill after this allocation, so set @@ -789,8 +801,15 @@ static struct request *get_request(struct request_queue *q, int rw_flags, if (rl->count[rw] >= (3 * q->nr_requests / 2)) goto out; + if (iog) + if (io_group_count(iog, rw) >= + (3 * io_group_nr_requests(iog) / 2)) + goto out; + rl->count[rw]++; rl->starved[rw] = 0; + if (iog) + io_group_inc_count(iog, rw); priv = !test_bit(QUEUE_FLAG_ELVSWITCH, &q->queue_flags); if (priv)@@ -808,7 +827,7 @@ static struct request *get_request(struct request_queue *q, int rw_flags,* wait queue, but this is pretty rare. */ spin_lock_irq(q->queue_lock); - freed_request(q, rw, priv); + freed_request(q, iog, rw, priv); /** in the very unlikely event that allocation failed and no @@ -1073,12 +1092,13 @@ void __blk_put_request(struct request_queue *q, struct request *req) if (req->cmd_flags & REQ_ALLOCED) { int rw = rq_data_dir(req); int priv = req->cmd_flags & REQ_ELVPRIV; + struct io_group *iog = io_request_io_group(req); BUG_ON(!list_empty(&req->queuelist)); BUG_ON(!hlist_unhashed(&req->hash)); blk_free_request(q, req); - freed_request(q, rw, priv); + freed_request(q, iog, rw, priv); } }EXPORT_SYMBOL_GPL (__blk_put_request);block/blk-sysfs.c +14
-8@@ -40,6 +40,7 @@ queue_requests_store(struct request_queue *q, const char *page, size_t count) { struct request_list *rl = &q->rq; unsigned long nr; + int iog_congested[2], iog_full[2]; int ret = queue_var_store(&nr, page, count); if (nr < BLKDEV_MIN_RQ) nr = BLKDEV_MIN_RQ;@@ -47,27 +48,32 @@ queue_requests_store(struct request_queue *q, const char *page, size_t count) spin_lock_irq(q->queue_lock); q->nr_requests = nr; blk_queue_congestion_threshold(q); + io_group_set_nrq_all(q, nr, iog_congested, iog_full); - if (rl->count[READ] >= queue_congestion_on_threshold(q)) + if (rl->count[READ] >= queue_congestion_on_threshold(q) || + iog_congested[READ]) blk_set_queue_congested(q, READ); - else if (rl->count[READ] < queue_congestion_off_threshold(q)) + else if (rl->count[READ] < queue_congestion_off_threshold(q) && + !iog_congested[READ]) blk_clear_queue_congested(q, READ); - if (rl->count[WRITE] >= queue_congestion_on_threshold(q)) + if (rl->count[WRITE] >= queue_congestion_on_threshold(q) || + iog_congested[WRITE]) blk_set_queue_congested(q, WRITE); - else if (rl->count[WRITE] < queue_congestion_off_threshold(q)) + else if (rl->count[WRITE] < queue_congestion_off_threshold(q) && + !iog_congested[WRITE]) blk_clear_queue_congested(q, WRITE); - if (rl->count[READ] >= q->nr_requests) { + if (rl->count[READ] >= q->nr_requests || iog_full[READ]) { blk_set_queue_full(q, READ); - } else if (rl->count[READ]+1 <= q->nr_requests) { + } else if (rl->count[READ]+1 <= q->nr_requests && !iog_full[READ]) { blk_clear_queue_full(q, READ); wake_up(&rl->wait[READ]); } - if (rl->count[WRITE] >= q->nr_requests) { + if (rl->count[WRITE] >= q->nr_requests || iog_full[WRITE]) { blk_set_queue_full(q, WRITE); - } else if (rl->count[WRITE]+1 <= q->nr_requests) { + } else if (rl->count[WRITE]+1 <= q->nr_requests && !iog_full[WRITE]) { blk_clear_queue_full(q, WRITE); wake_up(&rl->wait[WRITE]); }block/elevator-fq.c +128
-5@@ -924,6 +924,111 @@ struct io_group *io_lookup_io_group_current(struct request_queue *q) }EXPORT_SYMBOL (io_lookup_io_group_current); +/* +* TODO +* This is complete dupulication of blk_queue_congestion_threshold() +* except for the argument type and name. Can we merge them? +*/ +static void io_group_nrq_congestion_threshold (struct io_group_nrq *nrq) +{ + int nr; + + nr = nrq->nr_requests - (nrq->nr_requests / 8) + 1; + if (nr > nrq->nr_requests) + nr = nrq->nr_requests; + nrq->nr_congestion_on = nr; + + nr = nrq->nr_requests - (nrq->nr_requests / 8) + - (nrq->nr_requests / 16) - 1; + if (nr < 1) + nr = 1; + nrq->nr_congestion_off = nr; +} + +static void io_group_set_nrq (struct io_group_nrq *nrq,int nr_requests, + int *congested, int *full) +{ + int i; + + BUG_ON(nr_requests < 0); + + nrq->nr_requests = nr_requests; + io_group_nrq_congestion_threshold(nrq); + + for (i=0; i<2; i++) { + if (nrq->count[i] >= nrq->nr_congestion_on) + congested[i] = 1; + else if (nrq->count[i] < nrq->nr_congestion_off) + congested[i] = 0; + + if (nrq->count[i] >= nrq->nr_requests) + full[i] = 1; + else if (nrq->count[i]+1 <= nrq->nr_requests) + full[i] = 0; + } +} + +void io_group_set_nrq_all (struct request_queue *q,int nr, + int *congested, int *full) +{ + struct elv_fq_data *efqd = &q->elevator->efqd; + struct hlist_head *head = &efqd->group_list; + struct io_group *root = efqd->root_group; + struct hlist_node *n; + struct io_group *iog; + struct io_group_nrq *nrq; + int nrq_congested[2]; + int nrq_full[2]; + int i; + + for (i=0; i<2; i++) + *(congested + i) = *(full + i) = 0; + + nrq = &root->nrq; + io_group_set_nrq(nrq, nr, nrq_congested, nrq_full); + for (i=0; i<2; i++) { + *(congested + i) |= nrq_congested[i]; + *(full + i) |= nrq_full[i]; + } + + hlist_for_each_entry(iog, n, head, elv_data_node) { + nrq = &iog->nrq; + io_group_set_nrq(nrq, nr, nrq_congested, nrq_full); + for (i=0; i<2; i++) { + *(congested + i) |= nrq_congested[i]; + *(full + i) |= nrq_full[i]; + } + } +} + +struct io_group *io_congested_io_group (struct request_queue *q,int rw) +{ + struct hlist_head *head = &q->elevator->efqd.group_list; + struct hlist_node *n; + struct io_group *iog; + + hlist_for_each_entry(iog, n, head, elv_data_node) { + struct io_group_nrq *nrq = &iog->nrq; + if (nrq->count[rw] >= nrq->nr_congestion_off) + return iog; + } + return NULL; +} + +struct io_group *io_full_io_group (struct request_queue *q,int rw) +{ + struct hlist_head *head = &q->elevator->efqd.group_list; + struct hlist_node *n; + struct io_group *iog; + + hlist_for_each_entry(iog, n, head, elv_data_node) { + struct io_group_nrq *nrq = &iog->nrq; + if (nrq->count[rw] >= nrq->nr_requests) + return iog; + } + return NULL; +} +void io_group_init_entity (struct io_cgroup *iocg,struct io_group *iog) { struct io_entity *entity = &iog->entity;@@ -934,6 +1039,12 @@ void io_group_init_entity(struct io_cgroup *iocg, struct io_group *iog) entity->my_sched_data = &iog->sched_data; } +static void io_group_init_nrq (struct request_queue *q,struct io_group_nrq *nrq) +{ + nrq->nr_requests = q->nr_requests; + io_group_nrq_congestion_threshold(nrq); +} +void io_group_set_parent (struct io_group *iog,struct io_group *parent) { struct io_entity *entity;@@ -1053,6 +1164,8 @@ struct io_group *io_group_chain_alloc(struct request_queue *q, void *key, io_group_init_entity(iocg, iog); iog->my_entity = &iog->entity; + io_group_init_nrq(q, &iog->nrq); + if (leaf == NULL) { leaf = iog; prev = leaf;@@ -1176,7 +1289,7 @@ struct io_group *io_find_alloc_group(struct request_queue *q,* Generic function to make sure cgroup hierarchy is all setup once a request * from a cgroup is received by the io scheduler. */ -struct io_group *io_get_io_group (struct request_queue *q) +struct io_group *__io_get_io_group (struct request_queue *q) { struct cgroup *cgroup; struct io_group *iog;@@ -1192,6 +1305,19 @@ struct io_group *io_get_io_group(struct request_queue *q) return iog; } +struct io_group *io_get_io_group (struct request_queue *q) +{ + struct io_group *iog; + unsigned long flags; + + spin_lock_irqsave(q->queue_lock, flags); + iog = __io_get_io_group(q); + spin_unlock_irqrestore(q->queue_lock, flags); + BUG_ON(!iog); + + return iog; +} +void io_free_root_group (struct elevator_queue *e) { struct io_cgroup *iocg = &io_root_cgroup;@@ -1220,6 +1346,7 @@ struct io_group *io_alloc_root_group(struct request_queue *q, iog->entity.parent = NULL; for (i = 0; i < IO_IOPRIO_CLASSES; i++) iog->sched_data.service_tree[i] = IO_SERVICE_TREE_INIT; + io_group_init_nrq(q, &iog->nrq); iocg = &io_root_cgroup; spin_lock_irq(&iocg->lock);@@ -1533,15 +1660,11 @@ void elv_fq_set_request_io_group(struct request_queue *q, struct request *rq) { struct io_group *iog; - unsigned long flags; /* Make sure io group hierarchy has been setup and also set the * io group to which rq belongs. Later we should make use of * bio cgroup patches to determine the io group */ - spin_lock_irqsave(q->queue_lock, flags); iog = io_get_io_group(q); - spin_unlock_irqrestore(q->queue_lock, flags); - BUG_ON(!iog); /* Store iog in rq. TODO: take care of referencing */ rq->iog = iog;block/elevator-fq.h +201
-1@@ -187,6 +187,22 @@ struct io_queue {#ifdef CONFIG_GROUP_IOSCHED /** +* struct io_group_nrq - structure to store allocated requests info +* @nr_requests: maximun num of requests for the io_group +* @nr_congestion_on: threshold to determin the io_group is cogested. +* @nr_congestion_off: threshold to determin the io_group is not congested. +* @count: num of allocated requests. +* +* All fields are protected by queue_lock. +*/ +struct io_group_nrq { + unsigned long nr_requests; + unsigned int nr_congestion_on; + unsigned int nr_congestion_off; + int count[2]; +}; + +/** * struct bfq_group - per (device, cgroup) data structure. * @entity: schedulable entity to insert into the parent group sched_data. * @sched_data: own sched_data, to contain child entities (they may be @@ -235,6 +251,8 @@ struct io_group { /* Single ioq per group, used for noop, deadline, anticipatory */ struct io_queue *ioq; + + struct io_group_nrq nrq; };/** @@ -469,6 +487,11 @@ extern int elv_fq_set_request_ioq(struct request_queue *q, struct request *rq,extern void elv_fq_unset_request_ioq (struct request_queue *q, struct request *rq);extern struct io_queue *elv_lookup_ioq_current (struct request_queue *q); +extern void io_group_set_nrq_all (struct request_queue *q,int nr, + int *congested, int *full); +extern struct io_group *io_congested_io_group (struct request_queue *q,int rw); +extern struct io_group *io_full_io_group (struct request_queue *q,int rw); +extern struct io_group *__io_get_io_group (struct request_queue *q);/* Returns single ioq associated with the io group. */ static inline struct io_queue *io_group_ioq (struct io_group *iog)@@ -486,6 +509,52 @@ static inline void io_group_set_ioq(struct io_group *iog, struct io_queue *ioq) iog->ioq = ioq; } +static inline struct io_group *io_request_io_group (struct request *rq) +{ + return rq->iog; +} + +static inline unsigned long io_group_nr_requests (struct io_group *iog) +{ + BUG_ON(!iog); + return iog->nrq.nr_requests; +} + +static inline int io_group_inc_count (struct io_group *iog,int rw) +{ + BUG_ON(!iog); + return iog->nrq.count[rw]++; +} + +static inline int io_group_dec_count (struct io_group *iog,int rw) +{ + BUG_ON(!iog); + return iog->nrq.count[rw]--; +} + +static inline int io_group_count (struct io_group *iog,int rw) +{ + BUG_ON(!iog); + return iog->nrq.count[rw]; +} + +static inline int io_group_congestion_on (struct io_group *iog,int rw) +{ + BUG_ON(!iog); + return iog->nrq.count[rw] + 1 >= iog->nrq.nr_congestion_on; +} + +static inline int io_group_congestion_off (struct io_group *iog,int rw) +{ + BUG_ON(!iog); + return iog->nrq.count[rw] < iog->nrq.nr_congestion_off; +} + +static inline int io_group_full (struct io_group *iog,int rw) +{ + BUG_ON(!iog); + return iog->nrq.count[rw] + 1 >= iog->nrq.nr_requests; +}#else /* !GROUP_IOSCHED */ /* * No ioq movement is needed in case of flat setup. root io group gets cleaned @@ -537,6 +606,71 @@ static inline struct io_queue *elv_lookup_ioq_current(struct request_queue *q) return NULL; } +static inline void io_group_set_nrq_all (struct request_queue *q,int nr, + int *congested, int *full) +{ + int i; + for (i=0; i<2; i++) + *(congested + i) = *(full + i) = 0; +} + +static inline struct io_group * +io_congested_io_group (struct request_queue *q,int rw) +{ + return NULL; +} + +static inline struct io_group * +io_full_io_group (struct request_queue *q,int rw) +{ + return NULL; +} + +static inline struct io_group *__io_get_io_group (struct request_queue *q) +{ + return NULL; +} + +static inline struct io_group *io_request_io_group (struct request *rq) +{ + return NULL; +} + +static inline unsigned long io_group_nr_requests (struct io_group *iog) +{ + return 0; +} + +static inline int io_group_inc_count (struct io_group *iog,int rw) +{ + return 0; +} + +static inline int io_group_dec_count (struct io_group *iog,int rw) +{ + return 0; +} + +static inline int io_group_count (struct io_group *iog,int rw) +{ + return 0; +} + +static inline int io_group_congestion_on (struct io_group *iog,int rw) +{ + return 0; +} + +static inline int io_group_congestion_off (struct io_group *iog,int rw) +{ + return 1; +} + +static inline int io_group_full (struct io_group *iog,int rw) +{ + return 0; +} +#endif /* GROUP_IOSCHED */ /* Functions used by blksysfs.c */ @@ -589,6 +723,9 @@ extern void elv_free_ioq(struct io_queue *ioq);#else /* CONFIG_ELV_FAIR_QUEUING */ +struct io_group { +}; +static inline int elv_init_fq_data (struct request_queue *q, struct elevator_queue *e) {@@ -655,5 +792,69 @@ static inline struct io_queue *elv_lookup_ioq_current(struct request_queue *q) return NULL; } +static inline void io_group_set_nrq_all (struct request_queue *q,int nr, + int *congested, int *full) +{ + int i; + for (i=0; i<2; i++) + *(congested + i) = *(full + i) = 0; +} + +static inline struct io_group * +io_congested_io_group (struct request_queue *q,int rw) +{ + return NULL; +} + +static inline struct io_group * +io_full_io_group (struct request_queue *q,int rw) +{ + return NULL; +} + +static inline struct io_group *__io_get_io_group (struct request_queue *q) +{ + return NULL; +} + +static inline struct io_group *io_request_io_group (struct request *rq) +{ + return NULL; +} + +static inline unsigned long io_group_nr_requests (struct io_group *iog) +{ + return 0; +} + +static inline int io_group_inc_count (struct io_group *iog,int rw) +{ + return 0; +} + +static inline int io_group_dec_count (struct io_group *iog,int rw) +{ + return 0; +} + +static inline int io_group_count (struct io_group *iog,int rw) +{ + return 0; +} + +static inline int io_group_congestion_on (struct io_group *iog,int rw) +{ + return 0; +} + +static inline int io_group_congestion_off (struct io_group *iog,int rw) +{ + return 1; +} + +static inline int io_group_full (struct io_group *iog,int rw) +{ + return 0; +}#endif /* CONFIG_ELV_FAIR_QUEUING */ #endif /* _BFQ_SCHED_H */ --1.5.4.3
--
IKEDA, Munehiro
NEC Corporation of America
[email protected]
A message that is not in the archive (sent off-list or not yet received): e98e18940905011639o63c048f1n79c7e7648441a06d-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org
Re: IO Controller per cgroup request descriptors (Re: [PATCH 01/10] Documentation)
Quoted 36 lines · Nauman Rafique
Quoted 35 lines · Vivek Goyal
Quoted 34 lines · Munehiro
Quoted 23 lines · Vivek Goyal
Quoted 7 lines
Review
Quoted 6 lines
(quoted diff) +6
-0+TODO +==== +- Lots of cleanups, testing, bug fixing, optimizations, benchmarking etc... +- Convert cgroup ioprio to notion of weight. +- Anticipatory code will need more work. It is not working properly currently + and needs more thought.What are the problems with the code?
Have not got a chance to look into the issues in detail yet. Just a crude run saw drop in performance. Will debug it later the moment I have got async writes handled...
Quoted 6 lines
Quoted 1 line
+- Use of bio-cgroup patches.
I saw these posted as well
Quoted 2 lines
+- Use of Nauman's per cgroup request descriptor patches. +
More details would be nice, I am not sure I understand
Currently the number of request descriptors which can be allocated per device/request queue are fixed by a sysfs tunable (
q->nr_requests). So if there is lots of IO going on from one cgroup then it will consume all the available request descriptors and other cgroup might starve and not get its fair share.Quoted 3 lines
Hence we also need to introduce the notion of request descriptor limit per cgroup so that if request descriptors from one group are exhausted, then it does not impact the IO of other cgroup.
Unfortunately I couldn't find and I've never seen the Nauman's patches. So I tried to make a patch below against this todo. The reason why I'm posting this despite this is just a quick and ugly hack (and it might be a reinvention of wheel) is that I would like to discuss how we should define the limitation of requests per cgroup.
This patch should be applied on Vivek's I/O controller patches posted on Mar 11.
Quoted 7 lines
Quoted 4 lines
Hi IKEDA,
Sorry for the confusion here. Actually Nauman had sent a patch to select group of people who were initially copied on the mail thread.
I am sorry about that. Since I dropped my whole patch set in favor of Vivek's stuff, this stuff fell through the cracks.
No problem at all guys. I'm glad to see your patch Vivek sent, thanks.
Quoted 39 lines
Quoted 30 lines
Quoted 7 lines
This patch temporarily distribute
q->nr_requeststo each cgroup. I think the number should be weighted like BFQ's budget. But in this case, if the hierarchy of cgroup is deep, leaf cgroups are allowed to allocate very few numbers of requests. I don't think this is reasonable...but I don't have specific idea to solve this problem. Does anyone have the good idea?Thanks for the patch. Yes, ideally one would expect the request descriptor to be allocated also in proportion to the weight but I guess that would become very comlicated.
In terms of simpler things, two thoughts come to mind.
- First approach is to make
q->nr_requestsper group. So every group is entitled forq->nr_requestsas set by the user. This is what your patch seems to have done.I had some concerns with this approach. First of all it does not seem to have an upper bound on number of request descriptors allocated per queue because if a user creates more cgroups, total number of request descriptors increase.
- Second approach can be that we retain the meaning of
q->nr_requestswhich defines the total number of request descriptors on the queue (with the exception of 50% more descriptors for batching processes). And we define a new per group limitq->nr_group_requestswhich defines how many requests per group can be assigned. Soq->nr_requestsdefines total pool size on the queue andq->nr_group_requestswill define how many requests each group can allocate out of that pool.Quoted 8 lines
Here the issue is that a user shall have to balance the
q->nr_group_requestsandq->nr_requestsproperly.To experiment, I have implemented the second approach. I am attaching the patch which is in my current tree. It probably will not apply on my march 11 posting as since then patches have changed. But posting it here so that at least it will give an idea behind the thought process.
Ideas are welcome...
Quoted 7 lines
I had started with the first option, but the second option sounds good too. But one problem that comes to mind is how we deal with hierarchies? The sys admin can limit the root level cgroups to specific number of request descriptors, but if applications running in a cgroup are allowed to create their own cgroups, then the total request descriptors of all child cgroups should be capped by the number assigned to parent cgroups.
I think the second option cannot coexist with hierarchy support of per cgroup request descriptors limitation. I guess the fundamental idea of the second approach is to make logic simpler by giving up hierarchy, isn't it correct?
IIUC, for hierarchy support, we need to have some good idea to solve the issue that a cgroup which belongs to deep hierarchy can have only few requests, as I mentioned.
Anyway, I keep my eyes on this issue. I'm looking forward to Vivek's next version patches.
(snip)
Quoted 13 lines
Quoted 13 lines
block/blk-sysfs.c +13
-0+#ifdef CONFIG_GROUP_IOSCHED +static ssize_t queue_group_requests_show (struct request_queue *q,char *page) +{ +return queue_var_show (q->nr_group_requests, (page)); +} + +static ssize_t +queue_group_requests_store (struct request_queue *q,const char *page, +size_t count) +{ +unsigned long nr; +int ret =queue_var_store (&nr, page, count); +if (nr <BLKDEV_MIN_RQ )
Quoted 8 lines
block/blk-sysfs.c queue_requests_store(struct request_queu · line 99
Quoted 8 lines
block/blk-sysfs.c +8
-0+ nr =BLKDEV_MIN_RQ ; + +spin_lock_irq (q->queue_lock); + q->nr_group_requests = nr; +spin_unlock_irq (q->queue_lock); + return ret; +} +#endif
(snip)
Unchanging io_context "batching" status is on purpose?
Thanks,
Muuhh
Signature · 3 lines
IKEDA, Munehiro
NEC Corporation of America
m-ikeda-MDRzhb/z0dd8UrSeD/[email protected]Re: IO Controller per cgroup request descriptors (Re: [PATCH 01/10] Documentation)
In reply to Nauman RafiqueQuoted 36 lines · Nauman Rafique
Quoted 35 lines · Vivek Goyal
Quoted 34 lines · Munehiro
Quoted 23 lines · Vivek Goyal
Quoted 7 lines
Review
Quoted 6 lines
(quoted diff) +6
-0+TODO +==== +- Lots of cleanups, testing, bug fixing, optimizations, benchmarking etc... +- Convert cgroup ioprio to notion of weight. +- Anticipatory code will need more work. It is not working properly currently + and needs more thought.What are the problems with the code?
Have not got a chance to look into the issues in detail yet. Just a crude run saw drop in performance. Will debug it later the moment I have got async writes handled...
Quoted 6 lines
Quoted 1 line
+- Use of bio-cgroup patches.
I saw these posted as well
Quoted 2 lines
+- Use of Nauman's per cgroup request descriptor patches. +
More details would be nice, I am not sure I understand
Currently the number of request descriptors which can be allocated per device/request queue are fixed by a sysfs tunable (
q->nr_requests). So if there is lots of IO going on from one cgroup then it will consume all the available request descriptors and other cgroup might starve and not get its fair share.Quoted 3 lines
Hence we also need to introduce the notion of request descriptor limit per cgroup so that if request descriptors from one group are exhausted, then it does not impact the IO of other cgroup.
Unfortunately I couldn't find and I've never seen the Nauman's patches. So I tried to make a patch below against this todo. The reason why I'm posting this despite this is just a quick and ugly hack (and it might be a reinvention of wheel) is that I would like to discuss how we should define the limitation of requests per cgroup.
This patch should be applied on Vivek's I/O controller patches posted on Mar 11.
Quoted 7 lines
Quoted 4 lines
Hi IKEDA,
Sorry for the confusion here. Actually Nauman had sent a patch to select group of people who were initially copied on the mail thread.
I am sorry about that. Since I dropped my whole patch set in favor of Vivek's stuff, this stuff fell through the cracks.
No problem at all guys. I'm glad to see your patch Vivek sent, thanks.
Quoted 39 lines
Quoted 30 lines
Quoted 7 lines
This patch temporarily distribute
q->nr_requeststo each cgroup. I think the number should be weighted like BFQ's budget. But in this case, if the hierarchy of cgroup is deep, leaf cgroups are allowed to allocate very few numbers of requests. I don't think this is reasonable...but I don't have specific idea to solve this problem. Does anyone have the good idea?Thanks for the patch. Yes, ideally one would expect the request descriptor to be allocated also in proportion to the weight but I guess that would become very comlicated.
In terms of simpler things, two thoughts come to mind.
- First approach is to make
q->nr_requestsper group. So every group is entitled forq->nr_requestsas set by the user. This is what your patch seems to have done.I had some concerns with this approach. First of all it does not seem to have an upper bound on number of request descriptors allocated per queue because if a user creates more cgroups, total number of request descriptors increase.
- Second approach can be that we retain the meaning of
q->nr_requestswhich defines the total number of request descriptors on the queue (with the exception of 50% more descriptors for batching processes). And we define a new per group limitq->nr_group_requestswhich defines how many requests per group can be assigned. Soq->nr_requestsdefines total pool size on the queue andq->nr_group_requestswill define how many requests each group can allocate out of that pool.Quoted 8 lines
Here the issue is that a user shall have to balance the
q->nr_group_requestsandq->nr_requestsproperly.To experiment, I have implemented the second approach. I am attaching the patch which is in my current tree. It probably will not apply on my march 11 posting as since then patches have changed. But posting it here so that at least it will give an idea behind the thought process.
Ideas are welcome...
Quoted 7 lines
I had started with the first option, but the second option sounds good too. But one problem that comes to mind is how we deal with hierarchies? The sys admin can limit the root level cgroups to specific number of request descriptors, but if applications running in a cgroup are allowed to create their own cgroups, then the total request descriptors of all child cgroups should be capped by the number assigned to parent cgroups.
I think the second option cannot coexist with hierarchy support of per cgroup request descriptors limitation. I guess the fundamental idea of the second approach is to make logic simpler by giving up hierarchy, isn't it correct?
IIUC, for hierarchy support, we need to have some good idea to solve the issue that a cgroup which belongs to deep hierarchy can have only few requests, as I mentioned.
Anyway, I keep my eyes on this issue. I'm looking forward to Vivek's next version patches.
(snip)
Quoted 13 lines
Quoted 13 lines
block/blk-sysfs.c +13
-0+#ifdef CONFIG_GROUP_IOSCHED +static ssize_t queue_group_requests_show (struct request_queue *q,char *page) +{ +return queue_var_show (q->nr_group_requests, (page)); +} + +static ssize_t +queue_group_requests_store (struct request_queue *q,const char *page, +size_t count) +{ +unsigned long nr; +int ret =queue_var_store (&nr, page, count); +if (nr <BLKDEV_MIN_RQ )
Quoted 8 lines
block/blk-sysfs.c queue_requests_store(struct request_queu · line 99
Quoted 8 lines
block/blk-sysfs.c +8
-0+ nr =BLKDEV_MIN_RQ ; + +spin_lock_irq (q->queue_lock); + q->nr_group_requests = nr; +spin_unlock_irq (q->queue_lock); + return ret; +} +#endif
(snip)
Unchanging io_context "batching" status is on purpose?
Thanks,
Muuhh
Signature · 3 lines
IKEDA, Munehiro NEC Corporation of America [email protected]
A message that is not in the archive (sent off-list or not yet received): 1236823015-4183-1-git-send-email-vgoyal-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org
PATCH 2/10 Common flat fair queuing code in elevaotor layer
This is common fair queuing code in elevator layer. This is controlled by config option CONFIG_ELV_FAIR_QUEUING. This patch initially only introduces flat fair queuing support where there is only one group, "root group" and all the tasks belong to root group.
This elevator layer changes are backward compatible. That means any ioscheduler using old interfaces will continue to work.
--- block/Kconfig.iosched | 13 + block/Makefile | 1 + block/blk-sysfs.c | 10 + block/elevator-fq.c | 1882 ++++++++++++++++++++++++++++++++++++++++++++++ block/elevator-fq.h | 479 ++++++++++++ block/elevator.c | 46 +- include/linux/blkdev.h | 5 + include/linux/elevator.h | 48 ++ 8 files changed, 2473 insertions(+), 11 deletions(-) create mode 100644 block/elevator-fq.c create mode 100644 block/elevator-fq.h
block/Kconfig.iosched +13 -0
@@ -2,6 +2,19 @@ if BLOCK menu "IO Schedulers" +config ELV_FAIR_QUEUING + bool "Elevator Fair Queuing Support" + default n + ---help--- + Traditionally only cfq had notion of multiple queues and it did + fair queuing at its own. With the cgroups and need of controlling + IO, now even the simple io schedulers like noop, deadline, as will + have one queue per cgroup and will need hierarchical fair queuing. + Instead of every io scheduler implementing its own fair queuing + logic, this option enables fair queuing in elevator layer so that + other ioschedulers can make use of it. + If unsure, say N. + config IOSCHED_NOOP bool default y
block/Makefile +1 -0
@@ -16,3 +16,4 @@ obj-$(CONFIG_IOSCHED_CFQ) += cfq-iosched.o obj-$(CONFIG_BLK_DEV_IO_TRACE) += blktrace.o obj-$(CONFIG_BLOCK_COMPAT) += compat_ioctl.o obj-$(CONFIG_BLK_DEV_INTEGRITY) += blk-integrity.o +obj-$(CONFIG_ELV_FAIR_QUEUING) += elevator-fq.o
block/blk-sysfs.c +10 -0
@@ -276,6 +276,13 @@ static struct queue_sysfs_entry queue_iostats_entry = { .store = queue_iostats_store, }; +#ifdef CONFIG_ELV_FAIR_QUEUING +static struct queue_sysfs_entry queue_slice_idle_entry = { + .attr = {.name ="slice_idle" , .mode =S_IRUGO |S_IWUSR }, + .show = elv_slice_idle_show, + .store = elv_slice_idle_store, +}; +#endif static struct attribute *default_attrs[] = { &queue_requests_entry.attr, &queue_ra_entry.attr,
@@ -287,6 +294,9 @@ static struct attribute *default_attrs[] = { &queue_nomerges_entry.attr, &queue_rq_affinity_entry.attr, &queue_iostats_entry.attr, +#ifdef CONFIG_ELV_FAIR_QUEUING + &queue_slice_idle_entry.attr, +#endif NULL, };
block/elevator-fq.c new file +1882 -0
1,882 lines not on this page · raw message
block/elevator-fq.h new file +479 -0
479 lines not on this page · raw message
block/elevator.c +35 -11
116 lines not on this page · raw message
include/linux/blkdev.h +5 -0
@@ -234,6 +234,11 @@ struct request {/* for bidi */ struct request *next_rq; + +#ifdef CONFIG_ELV_FAIR_QUEUING +/* io queue request belongs to */ +struct io_queue *ioq; +#endif };static inline unsigned short req_get_ioprio (struct request *req)
include/linux/elevator.h +48 -0
83 lines not on this page · raw message
Signature · 1 line
1.6.0.1
PATCH 3/10 Modify cfq to make use of flat elevator fair queuing
In reply to a message not in the archiveThis patch changes cfq to use fair queuing code from elevator layer.
o must_dispatch logic sounds like a dead code. Nobody seems to be making use of that flag. Retaining it for the time being.
--- block/Kconfig.iosched | 3 +- block/cfq-iosched.c | 1082 +++++++++++-------------------------------------- 2 files changed, 232 insertions(+), 853 deletions(-)
block/Kconfig.iosched +2 -1
@@ -3,7 +3,7 @@ if BLOCK menu "IO Schedulers" config ELV_FAIR_QUEUING - bool "Elevator Fair Queuing Support" + bool default n ---help--- Traditionally only cfq had notion of multiple queues and it did
@@ -46,6 +46,7 @@ config IOSCHED_DEADLINE config IOSCHED_CFQ tristate "CFQ I/O scheduler" + select ELV_FAIR_QUEUING default y ---help--- The CFQ I/O scheduler tries to distribute bandwidth equally
block/cfq-iosched.c +230 -852
1,757 lines not on this page · raw message
Signature · 1 line
1.6.0.1
PATCH 7/10 Prepare elevator layer for single queue schedulers
In reply to a message not in the archiveElevator layer now has support for hierarchical fair queuing. cfq has been migrated to make use of it and now it is time to do groundwork for noop, deadline and AS.
noop deadline and AS don't maintain separate queues for different processes. There is only one single queue. Effectively one can think that in hierarchical setup, there will be one queue per cgroup where requests from all the processes in the cgroup will be queued.
Generally io scheduler takes care of creating queues. Because there is only one queue here, we have modified common layer to take care of queue creation and some other functionality. This special casing helps in keeping the changes to noop, deadline and AS to the minimum.
--- block/elevator-fq.c | 153 ++++++++++++++++++++++++++++++++++++++++++++++ block/elevator-fq.h | 67 ++++++++++++++++++++ block/elevator.c | 35 ++++++++++- include/linux/elevator.h | 14 ++++ 4 files changed, 268 insertions(+), 1 deletions(-)
block/elevator-fq.c +153 -0
@@ -857,6 +857,12 @@ void io_put_io_group_queues(struct elevator_queue *e, struct io_group *iog)/* Free up async idle queue */ elv_release_ioq (e, &iog->async_idle_queue); + +#ifdef CONFIG_GROUP_IOSCHED +/* Optimization for io schedulers having single ioq */ +if (elv_iosched_single_ioq (e)) +elv_release_ioq (e, &iog->ioq); +#endif }
@@ -1538,6 +1544,153 @@ void elv_fq_set_request_io_group(struct request_queue *q, rq->iog = iog; } +/* +* Find/Create the io queue the rq should go in. This is an optimization +* for the io schedulers (noop, deadline and AS) which maintain only single +* io queue per cgroup. In this case common layer can just maintain a +* pointer in group data structure and keeps track of it. +* +* For the io schdulers like cfq, which maintain multiple io queues per +* cgroup, and decide the io queue of request based on process, this +* function is not invoked. +*/ +int elv_fq_set_request_ioq (struct request_queue *q,struct request *rq, +gfp_t gfp_mask) +{ +struct elevator_queue *e = q->elevator; +unsigned long flags; +struct io_queue *ioq =NULL , *new_ioq =NULL ; +struct io_group *iog; +void *sched_q =NULL , *new_sched_q =NULL ; + +if (!elv_iosched_fair_queuing_enabled (e)) +return 0 ; + +might_sleep_if (gfp_mask & __GFP_WAIT); +spin_lock_irqsave (q->queue_lock, flags); + +/* Determine the io group request belongs to */ + iog = rq->iog; +BUG_ON (!iog); + +retry: +/* Get the iosched queue */ + ioq =io_group_ioq (iog); +if (!ioq) { +/* io queue and sched_queue needs to be allocated */ +BUG_ON (!e->ops->elevator_alloc_sched_queue_fn); + +if (new_sched_q) { + goto alloc_ioq; + }else if (gfp_mask & __GFP_WAIT) { +/* +* Inform the allocator of the fact that we will +* just repeat this allocation if it fails, to allow +* the allocator to do whatever it needs to attempt to +* free memory. +*/ +spin_unlock_irq (q->queue_lock); +/* Call io scheduer to create scheduler queue */ + new_sched_q = e->ops->elevator_alloc_sched_queue_fn (q, + e, gfp_mask | __GFP_NOFAIL + | __GFP_ZERO); +spin_lock_irq (q->queue_lock); + goto retry; + }else { + sched_q = e->ops->elevator_alloc_sched_queue_fn (q, e, + gfp_mask | __GFP_ZERO); +if (!sched_q) + goto queue_fail; + } + +alloc_ioq: +if (new_ioq) { + ioq = new_ioq; + new_ioq =NULL ; + sched_q = new_sched_q; + new_sched_q =NULL ; + }else if (gfp_mask & __GFP_WAIT) { +/* +* Inform the allocator of the fact that we will +* just repeat this allocation if it fails, to allow +* the allocator to do whatever it needs to attempt to +* free memory. +*/ +spin_unlock_irq (q->queue_lock); + new_ioq =elv_alloc_ioq (q, gfp_mask | __GFP_NOFAIL + | __GFP_ZERO); +spin_lock_irq (q->queue_lock); + goto retry; + }else { + ioq =elv_alloc_ioq (q, gfp_mask | __GFP_ZERO); +if (!ioq) { + e->ops->elevator_free_sched_queue_fn (e, + sched_q); + sched_q =NULL ; + goto queue_fail; + } + } + +elv_init_ioq (e, ioq, sched_q,IOPRIO_CLASS_BE ,4 ,1 ); +io_group_set_ioq (iog, ioq); +elv_mark_ioq_sync (ioq); + } + +if (new_sched_q) + e->ops->elevator_free_sched_queue_fn (q->elevator, sched_q); + +if (new_ioq) +elv_free_ioq (new_ioq); + +/* Request reference */ +elv_get_ioq (ioq); + rq->ioq = ioq; +spin_unlock_irqrestore (q->queue_lock, flags); +return 0 ; + +queue_fail: +WARN_ON ((gfp_mask & __GFP_WAIT) && !ioq); +elv_schedule_dispatch (q); +spin_unlock_irqrestore (q->queue_lock, flags); +return 1 ; +} + +/* +* Find out the io queue of current task. Optimization for single ioq +* per io group io schedulers. +*/ +struct io_queue *elv_lookup_ioq_current (struct request_queue *q) +{ +struct io_group *iog; + +/* Determine the io group and io queue of the bio submitting task */ + iog =io_lookup_io_group_current (q); +if (!iog) { +/* May be task belongs to a cgroup for which io group has +* not been setup yet. */ +return NULL ; + } +return io_group_ioq (iog); +} + +/* +* This request has been serviced. Clean up ioq info and drop the reference. +* Again this is called only for single queue per cgroup schedulers (noop, +* deadline, AS). +*/ +void elv_fq_unset_request_ioq (struct request_queue *q,struct request *rq) +{ +struct io_queue *ioq = rq->ioq; + +if (!elv_iosched_fair_queuing_enabled (q->elevator)) +return ; + +if (ioq) { + rq->ioq =NULL ; +elv_put_ioq (ioq); + } +} +#else /* GROUP_IOSCHED */ void bfq_init_entity (struct io_entity *entity,struct io_group *iog) {
block/elevator-fq.h +67 -0
90 lines not on this page · raw message
block/elevator.c +34 -1
@@ -207,6 +207,14 @@ static void *elevator_alloc_sched_queue(struct request_queue *q, {void *sched_queue =NULL ; +/* +* If fair queuing is enabled, then queue allocation takes place +* during set_request() functions when request actually comes +* in. +*/ +if (elv_iosched_fair_queuing_enabled (eq)) +return NULL ; +if (eq->ops->elevator_alloc_sched_queue_fn) { sched_queue = eq->ops->elevator_alloc_sched_queue_fn (q, eq,GFP_KERNEL );
@@ -936,6 +944,13 @@ int elv_set_request(struct request_queue *q, struct request *rq, gfp_t gfp_mask)elv_fq_set_request_io_group (q, rq); +/* +* Optimization for noop, deadline and AS which maintain only single +* ioq per io group +*/ +if (elv_iosched_single_ioq (e)) +return elv_fq_set_request_ioq (q, rq, gfp_mask); +if (e->ops->elevator_set_req_fn)return e->ops->elevator_set_req_fn (q, rq, gfp_mask);
@@ -947,6 +962,15 @@ void elv_put_request(struct request_queue *q, struct request *rq) {struct elevator_queue *e = q->elevator; +/* +* Optimization for noop, deadline and AS which maintain only single +* ioq per io group +*/ +if (elv_iosched_single_ioq (e)) { +elv_fq_unset_request_ioq (q, rq); +return ; + } +if (e->ops->elevator_put_req_fn) e->ops->elevator_put_req_fn (rq); }
@@ -1329,9 +1353,18 @@ EXPORT_SYMBOL(elv_select_sched_queue);/* * Get the io scheduler queue pointer for current task. +* +* If fair queuing is enabled, determine the io group of task and retrieve +* the ioq pointer from that. This is used by only single queue ioschedulers +* for retrieving the queue associated with the group to decide whether the +* new bio can do a front merge or not. */ void *elv_get_sched_queue_current (struct request_queue *q) { -return q->elevator->sched_queue; +/* Fair queuing is not enabled. There is only one queue. */ +if (!elv_iosched_fair_queuing_enabled (q->elevator)) +return q->elevator->sched_queue; + +return ioq_sched_queue (elv_lookup_ioq_current (q)); }EXPORT_SYMBOL (elv_get_sched_queue_current);
include/linux/elevator.h +14 -0
@@ -245,17 +245,31 @@ enum {/* iosched wants to use fq logic of elevator layer */ #define ELV_IOSCHED_NEED_FQ 1 +/* iosched maintains only single ioq per group.*/ +#define ELV_IOSCHED_SINGLE_IOQ 2 +static inline int elv_iosched_fair_queuing_enabled (struct elevator_queue *e) {return (e->elevator_type->elevator_features) &ELV_IOSCHED_NEED_FQ ; } +static inline int elv_iosched_single_ioq (struct elevator_queue *e) +{ +return (e->elevator_type->elevator_features) &ELV_IOSCHED_SINGLE_IOQ ; +} +#else /* ELV_IOSCHED_FAIR_QUEUING */ static inline int elv_iosched_fair_queuing_enabled (struct elevator_queue *e) {return 0 ; } + +static inline int elv_iosched_single_ioq (struct elevator_queue *e) +{ +return 0 ; +} +#endif /* ELV_IOSCHED_FAIR_QUEUING */ extern void *elv_get_sched_queue (struct request_queue *q,struct request *rq);extern void *elv_select_sched_queue (struct request_queue *q,int force);
Signature · 1 line
1.6.0.1
Hi Vivek,
Could you tell me to which kernel I can apply your patches?
# latest mm? I would like to test your controller.
Thank you,
Takuya Yoshikawa
Quoted 49 lines · Vivek Goyal
Hi All,
Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.
http://lkml.org/lkml/2008/11/6/227
One of the takeaway from the discussion in this thread was that let us implement a common layer which contains the proportional weight scheduling code which can be shared by all the IO schedulers.
Implementing IO controller will not cover the devices which don't use IO schedulers but it should cover the common case.
There were more discussions regarding 2 level vs 1 level IO control at following link.
https://lists.linux-foundation.org/pipermail/containers/2009-January/015402.html
So in the mean time we took the discussion off the list and spent time on making the 1 level control apporoach work where majority of the proportional weight control is shared by the four schedulers instead of each one having to replicate the code. We make use of BFQ code for fair queuing as posted by Paolo and Fabio here.
http://lkml.org/lkml/2008/11/11/148
Details about design and howto have been put in documentation patch.
I have done very basic testing of running 2 or 3 "dd" threads in different cgroups. Wanted to get the patchset out for feedback/review before we dive into more bug fixing, benchmarking, optimizations etc.
Your feedback/comments are welcome.
Patch series contains 10 patches. It should be compilable and bootable after every patch. Intial 2 patches implement flat fair queuing (no cgroup support) and make cfq to use that. Later patches introduce hierarchical fair queuing support in elevator layer and modify other IO schdulers to use that.
Thanks
Vivek
_______________________________________________
Containers mailing list Containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
https://lists.linux-foundation.org/mailman/listinfo/containers
Quoted 11 lines · Vivek Goyal
Hi All,
Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.
http://lkml.org/lkml/2008/11/6/227
One of the takeaway from the discussion in this thread was that let us implement a common layer which contains the proportional weight scheduling code which can be shared by all the IO schedulers.
Hi Vivek,
I did some tests on my *old* i386 box(with two concurrent dd running), and notice that IO Controller doesn't work fine in such situation. But it can work perfectly in my *new* x86 box. I dig into this problem, and i guess the major reason is that my *old* i386 box is too slow, it can't ensure two running ioqs are always backlogged. If that is the case, I happens to have a thought. when an ioq uses up it time slice, we don't expire it immediately. May be we can give a piece of bonus time for idling to wait new requests if this ioq's finish time and its ancestor's finish time are all much smaller than other entities in each corresponding service tree.
Signature · 2 lines
Regards Gui Jianfeng
Quoted 4 lines · Vivek Goyal
Hi All,
Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.
Hi Vivek,
I got the following OOPS when testing, can't reproduce again :(
kernel BUG at block/elevator-fq.c:1396!
invalid opcode: 0000 [#1] SMP
last sysfs file: /sys/block/hdb/queue/scheduler
Modules linked in: ipv6 cpufreq_ondemand acpi_cpufreq dm_mirror dm_multipath sbd
Pid: 5032, comm: rmdir Not tainted (2.6.29-rc7-vivek #17) Veriton M460
EIP: 0060:[<c04ec7de>] EFLAGS: 00010082 CPU: 0
EIP is at iocg_destroy+0xdc/0x14e
EAX: 00000000 EBX: f62278b4 ECX: f6207800 EDX: f6227904
ESI: f6227800 EDI: f62278a0 EBP: 00000003 ESP: c8790f00
DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
Process rmdir (pid: 5032, ti=c8790000 task=f6636960 task.ti=c8790000)
Stack: f53cc5c0 f62b7258 f10991c8 00000282 f6227800 c0733fa0 ec1c5140 edfc6d34 c8790000 c04463ce f6a4c84c edfc6d34 0804c840 c048883c f6a4c84c f6a4c84c c04888d6 f6a4c84c 00000000 c04897de f6a4c84c c048504c f115f4c0 ebc37954 Call Trace: [<c04463ce>] cgroup_diput+0x41/0x8c [<c048883c>] dentry_iput+0x45/0x5e [<c04888d6>] d_kill+0x19/0x32 [<c04897de>] dput+0xd8/0xdf [<c048504c>] do_rmdir+0x8f/0xb6 [<c06330fc>] do_page_fault+0x2a2/0x579 [<c0402fc1>] sysenter_do_call+0x12/0x21 [<c0630000>] schedule+0x641/0x830 Code: 08 00 74 04 0f 0b eb fe 83 7f 04 00 74 04 0f 0b eb fe 45 83 c3 1c 83 fd 0 EIP: [<c04ec7de>] iocg_destroy+0xdc/0x14e SS:ESP 0068:c8790f00
Signature · 2 lines
Regards Gui Jianfeng
Quoted 35 lines · Vivek Goyal
Hi All,
Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.
http://lkml.org/lkml/2008/11/6/227
One of the takeaway from the discussion in this thread was that let us implement a common layer which contains the proportional weight scheduling code which can be shared by all the IO schedulers.
Implementing IO controller will not cover the devices which don't use IO schedulers but it should cover the common case.
There were more discussions regarding 2 level vs 1 level IO control at following link.
https://lists.linux-foundation.org/pipermail/containers/2009-January/015402.html
So in the mean time we took the discussion off the list and spent time on making the 1 level control apporoach work where majority of the proportional weight control is shared by the four schedulers instead of each one having to replicate the code. We make use of BFQ code for fair queuing as posted by Paolo and Fabio here.
http://lkml.org/lkml/2008/11/11/148
Details about design and howto have been put in documentation patch.
I have done very basic testing of running 2 or 3 "dd" threads in different cgroups. Wanted to get the patchset out for feedback/review before we dive into more bug fixing, benchmarking, optimizations etc.
Your feedback/comments are welcome.
Quoted 8 lines
Patch series contains 10 patches. It should be compilable and bootable after every patch. Intial 2 patches implement flat fair queuing (no cgroup support) and make cfq to use that. Later patches introduce hierarchical fair queuing support in elevator layer and modify other IO schdulers to use that.
Thanks
Vivek
Hi Vivek,
While testing these patches along with the bio-cgroup patches I noticed that for the case of 2 buffered writers (dd) with different weights, one of them would be able to use up a very large timeslice (I've seen upto 500ms) when the other queue is empty and not be accounted for it. This is due to the check in cfq_dispatch_requests() where a given cgroup can empty its entire queue (100 IOs or more) within its timeslice and have them sit in the dispatch queue ready for the disk driver to pick up. Moreover, this huge timeslice is not accounted for as this cgroup is charged only for the length of the intended timeslice and not the actual time taken.
The following patch fixes this by not optimizing on the single busy queue fact inside cfq_dispatch_requests. Note that this does not hurt throughput in any sense but just causes more IOs to be dispatched only when the drive is ready for them thus leading to better accounting too.
Fix bug where a given ioq can run through all its requests at once.
---
2.6.26/block/cfq-iosched.c +0 -0
@@ -848,8 +848,7 @@ static int cfq_dispatch_requests(struct request_queue *q, int force)
if (cfq_class_idle(cfqq)) max_dispatch = 1;
- if (elv_ioq_nr_dispatched(cfqq->ioq) >= max_dispatch &&
- elv_nr_busy_ioq(q->elevator) > 1)
+ if (elv_ioq_nr_dispatched(cfqq->ioq) >= max_dispatch)
break; if (cfqd->sync_flight && !cfq_cfqq_sync(cfqq))
PATCH 2/10 Common flat fair queuing code in elevaotor layer
In reply to Vivek GoyalThis is common fair queuing code in elevator layer. This is controlled by config option CONFIG_ELV_FAIR_QUEUING. This patch initially only introduces flat fair queuing support where there is only one group, "root group" and all the tasks belong to root group.
This elevator layer changes are backward compatible. That means any ioscheduler using old interfaces will continue to work.
--- block/Kconfig.iosched | 13 + block/Makefile | 1 + block/blk-sysfs.c | 10 + block/elevator-fq.c | 1882 ++++++++++++++++++++++++++++++++++++++++++++++ block/elevator-fq.h | 479 ++++++++++++ block/elevator.c | 46 +- include/linux/blkdev.h | 5 + include/linux/elevator.h | 48 ++ 8 files changed, 2473 insertions(+), 11 deletions(-) create mode 100644 block/elevator-fq.c create mode 100644 block/elevator-fq.h
block/Kconfig.iosched +13 -0
@@ -2,6 +2,19 @@ if BLOCK menu "IO Schedulers" +config ELV_FAIR_QUEUING + bool "Elevator Fair Queuing Support" + default n + ---help--- + Traditionally only cfq had notion of multiple queues and it did + fair queuing at its own. With the cgroups and need of controlling + IO, now even the simple io schedulers like noop, deadline, as will + have one queue per cgroup and will need hierarchical fair queuing. + Instead of every io scheduler implementing its own fair queuing + logic, this option enables fair queuing in elevator layer so that + other ioschedulers can make use of it. + If unsure, say N. + config IOSCHED_NOOP bool default y
block/Makefile +1 -0
@@ -16,3 +16,4 @@ obj-$(CONFIG_IOSCHED_CFQ) += cfq-iosched.o obj-$(CONFIG_BLK_DEV_IO_TRACE) += blktrace.o obj-$(CONFIG_BLOCK_COMPAT) += compat_ioctl.o obj-$(CONFIG_BLK_DEV_INTEGRITY) += blk-integrity.o +obj-$(CONFIG_ELV_FAIR_QUEUING) += elevator-fq.o
block/blk-sysfs.c +10 -0
@@ -276,6 +276,13 @@ static struct queue_sysfs_entry queue_iostats_entry = { .store = queue_iostats_store, }; +#ifdef CONFIG_ELV_FAIR_QUEUING +static struct queue_sysfs_entry queue_slice_idle_entry = { + .attr = {.name ="slice_idle" , .mode =S_IRUGO |S_IWUSR }, + .show = elv_slice_idle_show, + .store = elv_slice_idle_store, +}; +#endif static struct attribute *default_attrs[] = { &queue_requests_entry.attr, &queue_ra_entry.attr,
@@ -287,6 +294,9 @@ static struct attribute *default_attrs[] = { &queue_nomerges_entry.attr, &queue_rq_affinity_entry.attr, &queue_iostats_entry.attr, +#ifdef CONFIG_ELV_FAIR_QUEUING + &queue_slice_idle_entry.attr, +#endif NULL, };
block/elevator-fq.c new file +1882 -0
1,882 lines not on this page · raw message
block/elevator-fq.h new file +479 -0
479 lines not on this page · raw message
block/elevator.c +35 -11
116 lines not on this page · raw message
include/linux/blkdev.h +5 -0
@@ -234,6 +234,11 @@ struct request {/* for bidi */ struct request *next_rq; + +#ifdef CONFIG_ELV_FAIR_QUEUING +/* io queue request belongs to */ +struct io_queue *ioq; +#endif };static inline unsigned short req_get_ioprio (struct request *req)
include/linux/elevator.h +48 -0
83 lines not on this page · raw message
Signature · 1 line
1.6.0.1
A message that is not in the archive (sent off-list or not yet received): 1236823015-4183-3-git-send-email-vgoyal-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org
Re: [PATCH 02/10] Common flat fair queuing code in elevaotor layer
Vivek Goyal wrote:
...
Quoted 5 lines
block/elevator-fq.c new file +5
-0+ +int elv_init_ioq (struct elevator_queue *eq,struct io_queue *ioq, +void *sched_queue,int ioprio_class,int ioprio, +int is_sync) +{
block/elevator-fq.c line 965
Quoted 10 lines
block/elevator-fq.c new file +10
-0+struct elv_fq_data *efqd = &eq->efqd; +struct io_group *iog =io_lookup_io_group_current (efqd->queue); + +RB_CLEAR_NODE (&ioq->entity.rb_node); +atomic_set (&ioq->ref,0 ); + ioq->efqd = efqd; + ioq->entity.budget = efqd->elv_slice[is_sync]; +elv_ioq_set_ioprio_class (ioq, ioprio_class); +elv_ioq_set_ioprio (ioq, ioprio); + ioq->pid = current->pid;
Hi Vivek,
If we are using a scheduler other than cfq, IOW single ioq is used. that storing a pid to the ioq makes no sense, it just stores the first serviced task.
Quoted 6 lines
block/elevator-fq.c new file +6
-0+ ioq->sched_queue = sched_queue; +elv_mark_ioq_idle_window (ioq); +bfq_init_entity (&ioq->entity, iog); +return 0 ; +} +EXPORT_SYMBOL (elv_init_ioq);
...
block/elevator-fq.h line 239
Quoted 7 lines
block/elevator-fq.h new file +7
-0+ +extern int elv_slice_idle; +extern int elv_slice_async; + +/* Logging facilities. */ +#define elv_log_ioq(efqd, ioq, fmt, args...) \ + blk_add_trace_msg((efqd)->queue, "%d" fmt, (ioq)->pid, ##args)
Maybe we need to use current->pid instead, for ioq->pid is not valid sometimes.
Quoted 6 lines
block/elevator-fq.h new file +6
-0+ +#define elv_log(efqd, fmt, args...) \ + blk_add_trace_msg((efqd)->queue, "" fmt, ##args) + +#define ioq_sample_valid (samples) ((samples) > 80) +
Signature · 2 lines
Regards Gui Jianfeng
PATCH 1/1 IO Controller: Don't store the pid in single queue circumstances
In reply to a message not in the archiveVivek Goyal wrote:
...
Quoted 4 lines
(quoted diff) +4
-0+int elv_init_ioq(struct elevator_queue *eq, struct io_queue *ioq, + void *sched_queue, int ioprio_class, int ioprio, + int is_sync) +{
Review
Quoted 10 lines
(quoted diff) +10
-0+ struct elv_fq_data *efqd = &eq->efqd; + struct io_group *iog = io_lookup_io_group_current(efqd->queue); + + RB_CLEAR_NODE(&ioq->entity.rb_node); + atomic_set(&ioq->ref, 0); + ioq->efqd = efqd; + ioq->entity.budget = efqd->elv_slice[is_sync]; + elv_ioq_set_ioprio_class(ioq, ioprio_class); + elv_ioq_set_ioprio(ioq, ioprio); + ioq->pid = current->pid;
Hi Vivek,
Storing a pid in single queue circumstances doesn't make sence. So just store the pid when cfq is used.
---
block/elevator-fq.c | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-)
block/elevator-fq.c +4 -1
@@ -1988,7 +1988,10 @@ int elv_init_ioq(struct elevator_queue *eq, struct io_queue *ioq, ioq->entity.budget = efqd->elv_slice[is_sync];elv_ioq_set_ioprio_class (ioq, ioprio_class);elv_ioq_set_ioprio (ioq, ioprio); - ioq->pid = current->pid; +if (elv_iosched_single_ioq (eq)) + ioq->pid =0 ; +else + ioq->pid = current->pid; ioq->sched_queue = sched_queue;/* If generic idle logic is enabled, mark it */
Signature · 1 line
1.5.4.rc3
Re: [PATCH 02/10] Common flat fair queuing code in elevaotor layer
In reply to a message not in the archiveQuoted 18 lines · Vivek Goyal
block/elevator-fq.c new file +17
-0+/* +* elv_exit_fq_data is called before we call elevator_exit_fn. Before +* we ask elevator to cleanup its queues, we do the cleanup here so +* that all the group and idle tree references to ioq are dropped. Later +* during elevator cleanup, ioc reference will be dropped which will lead +* to removal of ioscheduler queue as well as associated ioq object. +*/ +void elv_exit_fq_data (struct elevator_queue *e) +{ +struct elv_fq_data *efqd = &e->efqd; +struct request_queue *q = efqd->queue; + +if (!elv_iosched_fair_queuing_enabled (e)) +return ; + +elv_shutdown_timer_wq (e); +
block/elevator-fq.c line 1842
Quoted 10 lines
block/elevator-fq.c new file +10
-0+spin_lock_irq (q->queue_lock); +/* This should drop all the idle tree references of ioq */ +elv_free_idle_ioq_list (e); +spin_unlock_irq (q->queue_lock); + +elv_shutdown_timer_wq (e); + +BUG_ON (timer_pending (&efqd->idle_slice_timer)); +io_free_root_group (e); +}
Hi Vivek,
When cleaning up the elv_fq_data and ioqs for the iogs
associated with a device on elv_exit(), I don't see any iogs except the root group being freed. In io_disconnect_groups() you remove the ioqs from each of the iog and move them to the root iog and then delete the root iog. Am I missing something here or are there leftover iogs at elv_exit?
-Divyesh
Re: [PATCH 02/10] Common flat fair queuing code in elevaotor layer
In reply to Vivek GoyalVivek Goyal wrote:
...
Quoted 5 lines
block/elevator-fq.c new file +5
-0+ +int elv_init_ioq (struct elevator_queue *eq,struct io_queue *ioq, +void *sched_queue,int ioprio_class,int ioprio, +int is_sync) +{
block/elevator-fq.c line 965
Quoted 10 lines
block/elevator-fq.c new file +10
-0+struct elv_fq_data *efqd = &eq->efqd; +struct io_group *iog =io_lookup_io_group_current (efqd->queue); + +RB_CLEAR_NODE (&ioq->entity.rb_node); +atomic_set (&ioq->ref,0 ); + ioq->efqd = efqd; + ioq->entity.budget = efqd->elv_slice[is_sync]; +elv_ioq_set_ioprio_class (ioq, ioprio_class); +elv_ioq_set_ioprio (ioq, ioprio); + ioq->pid = current->pid;
Hi Vivek,
If we are using a scheduler other than cfq, IOW single ioq is used. that storing a pid to the ioq makes no sense, it just stores the first serviced task.
Quoted 6 lines
block/elevator-fq.c new file +6
-0+ ioq->sched_queue = sched_queue; +elv_mark_ioq_idle_window (ioq); +bfq_init_entity (&ioq->entity, iog); +return 0 ; +} +EXPORT_SYMBOL (elv_init_ioq);
...
block/elevator-fq.h line 239
Quoted 7 lines
block/elevator-fq.h new file +7
-0+ +extern int elv_slice_idle; +extern int elv_slice_async; + +/* Logging facilities. */ +#define elv_log_ioq(efqd, ioq, fmt, args...) \ + blk_add_trace_msg((efqd)->queue, "%d" fmt, (ioq)->pid, ##args)
Maybe we need to use current->pid instead, for ioq->pid is not valid sometimes.
Quoted 6 lines
block/elevator-fq.h new file +6
-0+ +#define elv_log(efqd, fmt, args...) \ + blk_add_trace_msg((efqd)->queue, "" fmt, ##args) + +#define ioq_sample_valid (samples) ((samples) > 80) +
Signature · 2 lines
Regards Gui Jianfeng
PATCH 1/1 IO Controller: Don't store the pid in single queue circumstances
In reply to Vivek GoyalVivek Goyal wrote:
...
Quoted 4 lines
(quoted diff) +4
-0+int elv_init_ioq(struct elevator_queue *eq, struct io_queue *ioq, + void *sched_queue, int ioprio_class, int ioprio, + int is_sync) +{
Review
Quoted 10 lines
(quoted diff) +10
-0+ struct elv_fq_data *efqd = &eq->efqd; + struct io_group *iog = io_lookup_io_group_current(efqd->queue); + + RB_CLEAR_NODE(&ioq->entity.rb_node); + atomic_set(&ioq->ref, 0); + ioq->efqd = efqd; + ioq->entity.budget = efqd->elv_slice[is_sync]; + elv_ioq_set_ioprio_class(ioq, ioprio_class); + elv_ioq_set_ioprio(ioq, ioprio); + ioq->pid = current->pid;
Hi Vivek,
Storing a pid in single queue circumstances doesn't make sence. So just store the pid when cfq is used.
---
block/elevator-fq.c | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-)
block/elevator-fq.c +4 -1
@@ -1988,7 +1988,10 @@ int elv_init_ioq(struct elevator_queue *eq, struct io_queue *ioq, ioq->entity.budget = efqd->elv_slice[is_sync];elv_ioq_set_ioprio_class (ioq, ioprio_class);elv_ioq_set_ioprio (ioq, ioprio); - ioq->pid = current->pid; +if (elv_iosched_single_ioq (eq)) + ioq->pid =0 ; +else + ioq->pid = current->pid; ioq->sched_queue = sched_queue;/* If generic idle logic is enabled, mark it */
Signature · 1 line
1.5.4.rc3
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH] IO Controller: Don't store the pid in single queue circumstances
Quoted 32 lines · Gui Jianfeng
Vivek Goyal wrote:
...Quoted 4 lines
(quoted diff) +4
-0+int elv_init_ioq(struct elevator_queue *eq, struct io_queue *ioq, + void *sched_queue, int ioprio_class, int ioprio, + int is_sync) +{Review
Quoted 10 lines
(quoted diff) +10
-0+ struct elv_fq_data *efqd = &eq->efqd; + struct io_group *iog = io_lookup_io_group_current(efqd->queue); + + RB_CLEAR_NODE(&ioq->entity.rb_node); + atomic_set(&ioq->ref, 0); + ioq->efqd = efqd; + ioq->entity.budget = efqd->elv_slice[is_sync]; + elv_ioq_set_ioprio_class(ioq, ioprio_class); + elv_ioq_set_ioprio(ioq, ioprio); + ioq->pid = current->pid;Hi Vivek,
Storing a pid in single queue circumstances doesn't make sence. So just store the pid when cfq is used.
Signed-off-by Gui Jianfeng <[email protected]>---
block/elevator-fq.c | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-)block/elevator-fq.c +0
-0
block/elevator-fq.c int elv_init_ioq(struct elevator_queue *eq, struct io_queue *ioq, · line 1995
Quoted 9 lines
block/elevator-fq.c +4
-1@@ -1988,7 +1988,10 @@ int elv_init_ioq(struct elevator_queue *eq, struct io_queue *ioq, ioq->entity.budget = efqd->elv_slice[is_sync];elv_ioq_set_ioprio_class (ioq, ioprio_class);elv_ioq_set_ioprio (ioq, ioprio); - ioq->pid = current->pid; +if (elv_iosched_single_ioq (eq)) + ioq->pid =0 ; +else + ioq->pid = current->pid; ioq->sched_queue = sched_queue;
Thanks Gui. Yes, if there is a single ioq, this pid will reflect the pid of the process who caused the creation of the io queue and later requests from all the other processess will go into same queue.
In fact cfq also has the same issue for async queues where async queue will store the pid when it is created later all other processes of same prio level will use it.
So if you think displaying "0" is better than displaying the the pid of the process who created the queue, then I will include this patch. Right now I don't have very strong opinion about it.
Thanks
Vivek
Re: [PATCH] IO Controller: Don't store the pid in single queue circumstances
In reply to Gui JianfengQuoted 32 lines · Gui Jianfeng
Vivek Goyal wrote:
...Quoted 4 lines
(quoted diff) +4
-0+int elv_init_ioq(struct elevator_queue *eq, struct io_queue *ioq, + void *sched_queue, int ioprio_class, int ioprio, + int is_sync) +{Review
Quoted 10 lines
(quoted diff) +10
-0+ struct elv_fq_data *efqd = &eq->efqd; + struct io_group *iog = io_lookup_io_group_current(efqd->queue); + + RB_CLEAR_NODE(&ioq->entity.rb_node); + atomic_set(&ioq->ref, 0); + ioq->efqd = efqd; + ioq->entity.budget = efqd->elv_slice[is_sync]; + elv_ioq_set_ioprio_class(ioq, ioprio_class); + elv_ioq_set_ioprio(ioq, ioprio); + ioq->pid = current->pid;Hi Vivek,
Storing a pid in single queue circumstances doesn't make sence. So just store the pid when cfq is used.
Signed-off-by Gui Jianfeng <[email protected]>---
block/elevator-fq.c | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-)block/elevator-fq.c +0
-0
block/elevator-fq.c int elv_init_ioq(struct elevator_queue *eq, struct io_queue *ioq, · line 1995
Quoted 9 lines
block/elevator-fq.c +4
-1@@ -1988,7 +1988,10 @@ int elv_init_ioq(struct elevator_queue *eq, struct io_queue *ioq, ioq->entity.budget = efqd->elv_slice[is_sync];elv_ioq_set_ioprio_class (ioq, ioprio_class);elv_ioq_set_ioprio (ioq, ioprio); - ioq->pid = current->pid; +if (elv_iosched_single_ioq (eq)) + ioq->pid =0 ; +else + ioq->pid = current->pid; ioq->sched_queue = sched_queue;
Thanks Gui. Yes, if there is a single ioq, this pid will reflect the pid of the process who caused the creation of the io queue and later requests from all the other processess will go into same queue.
In fact cfq also has the same issue for async queues where async queue will store the pid when it is created later all other processes of same prio level will use it.
So if you think displaying "0" is better than displaying the the pid of the process who created the queue, then I will include this patch. Right now I don't have very strong opinion about it.
Thanks
Vivek
Re: [PATCH 02/10] Common flat fair queuing code in elevaotor layer
In reply to Vivek GoyalQuoted 18 lines · Vivek Goyal
block/elevator-fq.c new file +17
-0+/* +* elv_exit_fq_data is called before we call elevator_exit_fn. Before +* we ask elevator to cleanup its queues, we do the cleanup here so +* that all the group and idle tree references to ioq are dropped. Later +* during elevator cleanup, ioc reference will be dropped which will lead +* to removal of ioscheduler queue as well as associated ioq object. +*/ +void elv_exit_fq_data (struct elevator_queue *e) +{ +struct elv_fq_data *efqd = &e->efqd; +struct request_queue *q = efqd->queue; + +if (!elv_iosched_fair_queuing_enabled (e)) +return ; + +elv_shutdown_timer_wq (e); +
block/elevator-fq.c line 1842
Quoted 10 lines
block/elevator-fq.c new file +10
-0+spin_lock_irq (q->queue_lock); +/* This should drop all the idle tree references of ioq */ +elv_free_idle_ioq_list (e); +spin_unlock_irq (q->queue_lock); + +elv_shutdown_timer_wq (e); + +BUG_ON (timer_pending (&efqd->idle_slice_timer)); +io_free_root_group (e); +}
Hi Vivek,
When cleaning up the elv_fq_data and ioqs for the iogs
associated with a device on elv_exit(), I don't see any iogs except the root group being freed. In io_disconnect_groups() you remove the ioqs from each of the iog and move them to the root iog and then delete the root iog. Am I missing something here or are there leftover iogs at elv_exit?
-Divyesh
A message that is not in the archive (sent off-list or not yet received): af41c7c40904012106h41d3cb50i2eeab2a02277a4c9-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org
Re: [PATCH 02/10] Common flat fair queuing code in elevaotor layer
Quoted 30 lines · Divyesh Shah
Quoted 18 lines · Vivek Goyal
block/elevator-fq.c new file +17
-0+/* +* elv_exit_fq_data is called before we call elevator_exit_fn. Before +* we ask elevator to cleanup its queues, we do the cleanup here so +* that all the group and idle tree references to ioq are dropped. Later +* during elevator cleanup, ioc reference will be dropped which will lead +* to removal of ioscheduler queue as well as associated ioq object. +*/ +void elv_exit_fq_data (struct elevator_queue *e) +{ +struct elv_fq_data *efqd = &e->efqd; +struct request_queue *q = efqd->queue; + +if (!elv_iosched_fair_queuing_enabled (e)) +return ; + +elv_shutdown_timer_wq (e); +block/elevator-fq.c line 1842
Quoted 10 lines
block/elevator-fq.c new file +10
-0+spin_lock_irq (q->queue_lock); +/* This should drop all the idle tree references of ioq */ +elv_free_idle_ioq_list (e); +spin_unlock_irq (q->queue_lock); + +elv_shutdown_timer_wq (e); + +BUG_ON (timer_pending (&efqd->idle_slice_timer)); +io_free_root_group (e); +}
Quoted 7 lines
Hi Vivek,
When cleaning up theelv_fq_dataand ioqs for the iogs
associated with a device onelv_exit(), I don't see any iogs except the root group being freed. Inio_disconnect_groups()you remove the ioqs from each of the iog and move them to the root iog and then delete the root iog. Am I missing something here or are there leftover iogs atelv_exit?
Hi Divyesh,
io_groups are linked in two lists. One list is maintained by io_cgroup to keep track of how many io_groups are there associated with this cgroup and other list is maintained in elv_fq_data to keep track of how many io_groups are actually doing IO to this device.
Upon elevator exit, we remove the io_groups (io_disconnect_groups())from the list maintained by elv_fq_data but we don't free them up. Freeing up is finally done when cgroup is being deleted (iocg_destroy()).
Thanks
Vivek
Re: [PATCH 02/10] Common flat fair queuing code in elevaotor layer
In reply to Divyesh ShahQuoted 30 lines · Divyesh Shah
Quoted 18 lines · Vivek Goyal
block/elevator-fq.c new file +17
-0+/* +* elv_exit_fq_data is called before we call elevator_exit_fn. Before +* we ask elevator to cleanup its queues, we do the cleanup here so +* that all the group and idle tree references to ioq are dropped. Later +* during elevator cleanup, ioc reference will be dropped which will lead +* to removal of ioscheduler queue as well as associated ioq object. +*/ +void elv_exit_fq_data (struct elevator_queue *e) +{ +struct elv_fq_data *efqd = &e->efqd; +struct request_queue *q = efqd->queue; + +if (!elv_iosched_fair_queuing_enabled (e)) +return ; + +elv_shutdown_timer_wq (e); +block/elevator-fq.c line 1842
Quoted 10 lines
block/elevator-fq.c new file +10
-0+spin_lock_irq (q->queue_lock); +/* This should drop all the idle tree references of ioq */ +elv_free_idle_ioq_list (e); +spin_unlock_irq (q->queue_lock); + +elv_shutdown_timer_wq (e); + +BUG_ON (timer_pending (&efqd->idle_slice_timer)); +io_free_root_group (e); +}
Quoted 7 lines
Hi Vivek,
When cleaning up theelv_fq_dataand ioqs for the iogs
associated with a device onelv_exit(), I don't see any iogs except the root group being freed. Inio_disconnect_groups()you remove the ioqs from each of the iog and move them to the root iog and then delete the root iog. Am I missing something here or are there leftover iogs atelv_exit?
Hi Divyesh,
io_groups are linked in two lists. One list is maintained by io_cgroup to keep track of how many io_groups are there associated with this cgroup and other list is maintained in elv_fq_data to keep track of how many io_groups are actually doing IO to this device.
Upon elevator exit, we remove the io_groups (io_disconnect_groups())from the list maintained by elv_fq_data but we don't free them up. Freeing up is finally done when cgroup is being deleted (iocg_destroy()).
Thanks
Vivek
PATCH 3/10 Modify cfq to make use of flat elevator fair queuing
In reply to Vivek GoyalThis patch changes cfq to use fair queuing code from elevator layer.
o must_dispatch logic sounds like a dead code. Nobody seems to be making use of that flag. Retaining it for the time being.
--- block/Kconfig.iosched | 3 +- block/cfq-iosched.c | 1082 +++++++++++-------------------------------------- 2 files changed, 232 insertions(+), 853 deletions(-)
block/Kconfig.iosched +2 -1
@@ -3,7 +3,7 @@ if BLOCK menu "IO Schedulers" config ELV_FAIR_QUEUING - bool "Elevator Fair Queuing Support" + bool default n ---help--- Traditionally only cfq had notion of multiple queues and it did
@@ -46,6 +46,7 @@ config IOSCHED_DEADLINE config IOSCHED_CFQ tristate "CFQ I/O scheduler" + select ELV_FAIR_QUEUING default y ---help--- The CFQ I/O scheduler tries to distribute bandwidth equally
block/cfq-iosched.c +230 -852
1,757 lines not on this page · raw message
Signature · 1 line
1.6.0.1
PATCH 4/10 Common hierarchical fair queuing code in elevaotor layer
In reply to Vivek GoyalThis patch enables hierarchical fair queuing in common layer. It is controlled by config option CONFIG_GROUP_IOSCHED.
--- block/blk-ioc.c | 3 + block/elevator-fq.c | 991 ++++++++++++++++++++++++++++++++++++++--- block/elevator-fq.h | 113 +++++ block/elevator.c | 2 + include/linux/blkdev.h | 7 +- include/linux/cgroup_subsys.h | 7 + include/linux/iocontext.h | 5 + init/Kconfig | 8 + 8 files changed, 1064 insertions(+), 72 deletions(-)
block/blk-ioc.c +3 -0
@@ -95,6 +95,9 @@ struct io_context *alloc_io_context(gfp_t gfp_flags, int node)spin_lock_init (&ret->lock); ret->ioprio_changed =0 ; ret->ioprio =0 ; +#ifdef CONFIG_GROUP_IOSCHED + ret->cgroup_changed =0 ; +#endif ret->last_waited = jiffies;/* doesn't matter... */ ret->nr_batch_requests =0 ;/* because this is 0 */ ret->aic =NULL ;
block/elevator-fq.c +920 -71
1,109 lines not on this page · raw message
block/elevator-fq.h +113 -0
153 lines not on this page · raw message
block/elevator.c +2 -0
@@ -888,6 +888,8 @@ int elv_set_request(struct request_queue *q, struct request *rq, gfp_t gfp_mask) {struct elevator_queue *e = q->elevator; +elv_fq_set_request_io_group (q, rq); +if (e->ops->elevator_set_req_fn)return e->ops->elevator_set_req_fn (q, rq, gfp_mask);
include/linux/blkdev.h +6 -1
@@ -238,7 +238,12 @@ struct request {#ifdef CONFIG_ELV_FAIR_QUEUING /* io queue request belongs to */ struct io_queue *ioq; -#endif + +#ifdef CONFIG_GROUP_IOSCHED +/* io group request belongs to */ +struct io_group *iog; +#endif /* GROUP_IOSCHED */ +#endif /* ELV_FAIR_QUEUING */ };static inline unsigned short req_get_ioprio (struct request *req)
include/linux/cgroup_subsys.h +7 -0
@@ -60,3 +60,10 @@ SUBSYS(net_cls)#endif /* */ + +#ifdef CONFIG_GROUP_IOSCHED +SUBSYS(io) +#endif + +/* */ +
include/linux/iocontext.h +5 -0
@@ -73,6 +73,11 @@ struct io_context {unsigned short ioprio;unsigned short ioprio_changed; +#ifdef CONFIG_GROUP_IOSCHED +/* If task changes the cgroup, elevator processes it asynchronously */ +unsigned short cgroup_changed; +#endif +/* * For request batching */
init/Kconfig +8 -0
@@ -538,6 +538,14 @@ config CGROUP_MEM_RES_CTLR_SWAP there will be no overhead from this. Even when you set this config=y, if boot option "noswapaccount" is set, swap will not be accounted. +config GROUP_IOSCHED + bool "Group IO Scheduler" + depends on CGROUPS && ELV_FAIR_QUEUING + default n + ---help--- + This feature lets IO scheduler recognize task groups and control + disk bandwidth allocation to such task groups. + endif # CGROUPS config MM_OWNER
Signature · 1 line
1.6.0.1
PATCH 5/10 cfq changes to use hierarchical fair queuing code in elevaotor layer
In reply to Vivek GoyalMake cfq hierarhical.
--- block/Kconfig.iosched | 8 ++++++++ block/cfq-iosched.c | 48 ++++++++++++++++++++++++++++++++++++++++++++++++ init/Kconfig | 2 +- 3 files changed, 57 insertions(+), 1 deletions(-)
block/Kconfig.iosched +8 -0
@@ -54,6 +54,14 @@ config IOSCHED_CFQ working environment, suitable for desktop systems. This is the default I/O scheduler. +config IOSCHED_CFQ_HIER + bool "CFQ Hierarchical Scheduling support" + depends on IOSCHED_CFQ && CGROUPS + select GROUP_IOSCHED + default n + ---help--- + Enable hierarhical scheduling in cfq. + choice prompt "Default I/O scheduler" default DEFAULT_CFQ
block/cfq-iosched.c +48 -0
60 lines not on this page · raw message
init/Kconfig +1 -1
@@ -539,7 +539,7 @@ config CGROUP_MEM_RES_CTLR_SWAP if boot option "noswapaccount" is set, swap will not be accounted. config GROUP_IOSCHED - bool "Group IO Scheduler" + bool depends on CGROUPS && ELV_FAIR_QUEUING default n ---help---
Signature · 1 line
1.6.0.1
A message that is not in the archive (sent off-list or not yet received): 1236823015-4183-6-git-send-email-vgoyal-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org
PATCH 1/1 IO-Controller: Fix kernel panic after moving a task
Quoted 9 lines · Vivek Goyal
(quoted diff) +9
-0+#ifdef CONFIG_IOSCHED_CFQ_HIER +static void changed_cgroup(struct io_context *ioc, struct cfq_io_context *cic) +{ + struct cfq_queue *async_cfqq = cic_to_cfqq(cic, 0); + struct cfq_queue *sync_cfqq = cic_to_cfqq(cic, 1); + struct cfq_data *cfqd = cic->key; + struct io_group *iog, *__iog; + unsigned long flags; + struct request_queue *q;
Review
Quoted 10 lines
(quoted diff) +10
-0+ + if (unlikely(!cfqd)) + return; + + q = cfqd->q; + + spin_lock_irqsave(q->queue_lock, flags); + + iog = io_lookup_io_group_current(q); +
Hi Vivek,
I triggered another kernel panic when testing. When moving a task to another cgroup, the corresponding iog may not be setup properly all the time. "iog" might be NULL here. io_ioq_move() receives a NULL iog, kernel crash.
Consider the following piece of code:
941 int elv_set_request(struct request_queue *q, struct request *rq, gfp_t gfp_mask) 942 {
943 struct elevator_queue *e = q->elevator;
944
945 elv_fq_set_request_io_group(q, rq);
-->task moving to a new group is happenning here.
946
947 /*
948 * Optimization for noop, deadline and AS which maintain only single 949 * ioq per io group 950 */
951 if (elv_iosched_single_ioq(e))
952 return elv_fq_set_request_ioq(q, rq, gfp_mask);
953
954 if (e->ops->elevator_set_req_fn)
955 return e->ops->elevator_set_req_fn(q, rq, gfp_mask);
cfq_set_request() will finally call io_ioq_move(), but the iog is NULL, beacause the iogs in the hierarchy are not built yet. So kernel crashes.
956
957 rq->elevator_private = NULL;
958 return 0;
959 }
BUG: unable to handle kernel NULL pointer dereference at 000000bc IP: [<c04ebf8f>] io_ioq_move+0xf2/0x109 *pde = 6cc00067 Oops: 0000 [#1] SMP last sysfs file: /sys/block/hdb/queue/slice_idle Modules linked in: ipv6 cpufreq_ondemand acpi_cpufreq dm_mirror dm_multipath sbs sbshc battery ac lp snd_hda_codec_realtek snd_hda_intel snd_hda_codec snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss snd_pcm serio_raw snd_timer rtc_cmos parport_pc snd r8169 button rtc_core parport soundcore mii i2c_i801 rtc_lib snd_page_alloc pcspkr i2c_core dm_region_hash dm_log dm_mod ext3 jbd mbcache uhci_hcd ohci_hcd ehci_hcd
Pid: 5431, comm: dd Not tainted (2.6.29-rc7-vivek #19) Veriton M460
EIP: 0060:[<c04ebf8f>] EFLAGS: 00010046 CPU: 0
EIP is at io_ioq_move+0xf2/0x109
EAX: f6203a88 EBX: f6792c94 ECX: f6203a84 EDX: 00000006
ESI: 00000000 EDI: 00000000 EBP: f6203a60 ESP: f6304c28
DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
Process dd (pid: 5431, ti=f6304000 task=f669dae0 task.ti=f6304000)
Stack: f62478c0 0100dd40 f6247908 f62d995c 00000000 00000000 f675b54c c04e9182 f638e9b0 00000282 f62d99a4 f6325a2c c04e9113 f5a707c0 c04e7ae0 f675b000 f62d95fc f6325a2c c04e8501 00000010 f631e4e8 f675b000 00080000 ffffff10 Call Trace: [<c04e9182>] changed_cgroup+0x6f/0x8d [<c04e9113>] changed_cgroup+0x0/0x8d [<c04e7ae0>] __call_for_each_cic+0x1b/0x25 [<c04e8501>] cfq_set_request+0x158/0x2c7 [<c06316e6>] _spin_unlock_irqrestore+0x5/0x6 [<c04eb106>] elv_fq_set_request_io_group+0x2b/0x3e [<c04e83a9>] cfq_set_request+0x0/0x2c7 [<c04dddcb>] elv_set_request+0x3e/0x4e [<c04df3da>] get_request+0x1ed/0x29b [<c04df9bb>] get_request_wait+0xdf/0xf2 [<c04dfd89>] __make_request+0x2c6/0x372 [<c049bd76>] do_mpage_readpage+0x4fe/0x5e3 [<c04deba5>] generic_make_request+0x2d0/0x355 [<c04dff47>] submit_bio+0x92/0x97 [<c045bfcb>] add_to_page_cache_locked+0x8a/0xb7 [<c049bfa4>] mpage_end_io_read+0x0/0x50 [<c049b1b6>] mpage_bio_submit+0x19/0x1d [<c049bf9a>] mpage_readpages+0x9b/0xa5 [<f7dd18c7>] ext3_readpages+0x0/0x15 [ext3] [<c0462192>] __do_page_cache_readahead+0xea/0x154 [<f7dd2286>] ext3_get_block+0x0/0xbe [ext3] [<c045d34d>] generic_file_aio_read+0x276/0x569 [<c047cdd9>] do_sync_read+0xbf/0xfe [<c043a3f2>] getnstimeofday+0x51/0xdb [<c0434d3c>] autoremove_wake_function+0x0/0x2d [<c041bdc3>] sched_slice+0x61/0x6a [<c0423114>] task_tick_fair+0x3d/0x60 [<c04c1d79>] security_file_permission+0xc/0xd [<c047cd1a>] do_sync_read+0x0/0xfe [<c047d35a>] vfs_read+0x6c/0x8b [<c047d67e>] sys_read+0x3c/0x63 [<c0402fc1>] sysenter_do_call+0x12/0x21 [<c0630000>] schedule+0x551/0x830 Code: 08 31 c9 89 da e8 77 fc ff ff 8b 86 bc 00 00 00 85 ff 89 43 38 8d 46 60 89 43 40 74 1d 83 c4 0c 89 d8 5b 5e 5f 5d e9 aa f9 ff ff <8b> 86 bc 00 00 00 89 43 38 8d 46 60 89 43 40 83 c4 0c 5b 5e 5f EIP: [<c04ebf8f>] io_ioq_move+0xf2/0x109 SS:ESP 0068:f6304c28
Changelog:
Make sure iogs in the hierarchy are built properly after moving a task to a new cgroup.
--- block/cfq-iosched.c | 4 +++- block/elevator-fq.c | 1 + block/elevator-fq.h | 1 + 3 files changed, 5 insertions(+), 1 deletions(-)
block/cfq-iosched.c +3 -1
@@ -12,6 +12,8 @@#include <linux/rbtree.h> #include <linux/ioprio.h> #include <linux/blktrace_api.h> +#include "elevator-fq.h" +/* * tunables */
@@ -1086,7 +1088,7 @@ static void changed_cgroup(struct io_context *ioc, struct cfq_io_context *cic)spin_lock_irqsave (q->queue_lock, flags); - iog =io_lookup_io_group_current (q); + iog =io_get_io_group (q);if (async_cfqq !=NULL ) { __iog =cfqq_to_io_group (async_cfqq);
block/elevator-fq.c +1 -0
@@ -1191,6 +1191,7 @@ struct io_group *io_get_io_group(struct request_queue *q)return iog; } +EXPORT_SYMBOL (io_get_io_group);void io_free_root_group (struct elevator_queue *e) {
block/elevator-fq.h +1 -0
@@ -459,6 +459,7 @@ static inline struct io_group *ioq_to_io_group(struct io_queue *ioq) }#ifdef CONFIG_GROUP_IOSCHED +extern struct io_group *io_get_io_group (struct request_queue *q);extern int io_group_allow_merge (struct request *rq,struct bio *bio);extern void io_ioq_move (struct elevator_queue *e,struct io_queue *ioq,struct io_group *iog);
Signature · 1 line
1.5.4.rc3
PATCH 1/1 IO-Controller: Fix kernel panic after moving a task
In reply to Vivek GoyalQuoted 9 lines · Vivek Goyal
(quoted diff) +9
-0+#ifdef CONFIG_IOSCHED_CFQ_HIER +static void changed_cgroup(struct io_context *ioc, struct cfq_io_context *cic) +{ + struct cfq_queue *async_cfqq = cic_to_cfqq(cic, 0); + struct cfq_queue *sync_cfqq = cic_to_cfqq(cic, 1); + struct cfq_data *cfqd = cic->key; + struct io_group *iog, *__iog; + unsigned long flags; + struct request_queue *q;
Review
Quoted 10 lines
(quoted diff) +10
-0+ + if (unlikely(!cfqd)) + return; + + q = cfqd->q; + + spin_lock_irqsave(q->queue_lock, flags); + + iog = io_lookup_io_group_current(q); +
Hi Vivek,
I triggered another kernel panic when testing. When moving a task to another cgroup, the corresponding iog may not be setup properly all the time. "iog" might be NULL here. io_ioq_move() receives a NULL iog, kernel crash.
Consider the following piece of code:
941 int elv_set_request(struct request_queue *q, struct request *rq, gfp_t gfp_mask) 942 {
943 struct elevator_queue *e = q->elevator;
944
945 elv_fq_set_request_io_group(q, rq);
-->task moving to a new group is happenning here.
946
947 /*
948 * Optimization for noop, deadline and AS which maintain only single 949 * ioq per io group 950 */
951 if (elv_iosched_single_ioq(e))
952 return elv_fq_set_request_ioq(q, rq, gfp_mask);
953
954 if (e->ops->elevator_set_req_fn)
955 return e->ops->elevator_set_req_fn(q, rq, gfp_mask);
cfq_set_request() will finally call io_ioq_move(), but the iog is NULL, beacause the iogs in the hierarchy are not built yet. So kernel crashes.
956
957 rq->elevator_private = NULL;
958 return 0;
959 }
BUG: unable to handle kernel NULL pointer dereference at 000000bc IP: [<c04ebf8f>] io_ioq_move+0xf2/0x109 *pde = 6cc00067 Oops: 0000 [#1] SMP last sysfs file: /sys/block/hdb/queue/slice_idle Modules linked in: ipv6 cpufreq_ondemand acpi_cpufreq dm_mirror dm_multipath sbs sbshc battery ac lp snd_hda_codec_realtek snd_hda_intel snd_hda_codec snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss snd_pcm serio_raw snd_timer rtc_cmos parport_pc snd r8169 button rtc_core parport soundcore mii i2c_i801 rtc_lib snd_page_alloc pcspkr i2c_core dm_region_hash dm_log dm_mod ext3 jbd mbcache uhci_hcd ohci_hcd ehci_hcd
Pid: 5431, comm: dd Not tainted (2.6.29-rc7-vivek #19) Veriton M460
EIP: 0060:[<c04ebf8f>] EFLAGS: 00010046 CPU: 0
EIP is at io_ioq_move+0xf2/0x109
EAX: f6203a88 EBX: f6792c94 ECX: f6203a84 EDX: 00000006
ESI: 00000000 EDI: 00000000 EBP: f6203a60 ESP: f6304c28
DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
Process dd (pid: 5431, ti=f6304000 task=f669dae0 task.ti=f6304000)
Stack: f62478c0 0100dd40 f6247908 f62d995c 00000000 00000000 f675b54c c04e9182 f638e9b0 00000282 f62d99a4 f6325a2c c04e9113 f5a707c0 c04e7ae0 f675b000 f62d95fc f6325a2c c04e8501 00000010 f631e4e8 f675b000 00080000 ffffff10 Call Trace: [<c04e9182>] changed_cgroup+0x6f/0x8d [<c04e9113>] changed_cgroup+0x0/0x8d [<c04e7ae0>] __call_for_each_cic+0x1b/0x25 [<c04e8501>] cfq_set_request+0x158/0x2c7 [<c06316e6>] _spin_unlock_irqrestore+0x5/0x6 [<c04eb106>] elv_fq_set_request_io_group+0x2b/0x3e [<c04e83a9>] cfq_set_request+0x0/0x2c7 [<c04dddcb>] elv_set_request+0x3e/0x4e [<c04df3da>] get_request+0x1ed/0x29b [<c04df9bb>] get_request_wait+0xdf/0xf2 [<c04dfd89>] __make_request+0x2c6/0x372 [<c049bd76>] do_mpage_readpage+0x4fe/0x5e3 [<c04deba5>] generic_make_request+0x2d0/0x355 [<c04dff47>] submit_bio+0x92/0x97 [<c045bfcb>] add_to_page_cache_locked+0x8a/0xb7 [<c049bfa4>] mpage_end_io_read+0x0/0x50 [<c049b1b6>] mpage_bio_submit+0x19/0x1d [<c049bf9a>] mpage_readpages+0x9b/0xa5 [<f7dd18c7>] ext3_readpages+0x0/0x15 [ext3] [<c0462192>] __do_page_cache_readahead+0xea/0x154 [<f7dd2286>] ext3_get_block+0x0/0xbe [ext3] [<c045d34d>] generic_file_aio_read+0x276/0x569 [<c047cdd9>] do_sync_read+0xbf/0xfe [<c043a3f2>] getnstimeofday+0x51/0xdb [<c0434d3c>] autoremove_wake_function+0x0/0x2d [<c041bdc3>] sched_slice+0x61/0x6a [<c0423114>] task_tick_fair+0x3d/0x60 [<c04c1d79>] security_file_permission+0xc/0xd [<c047cd1a>] do_sync_read+0x0/0xfe [<c047d35a>] vfs_read+0x6c/0x8b [<c047d67e>] sys_read+0x3c/0x63 [<c0402fc1>] sysenter_do_call+0x12/0x21 [<c0630000>] schedule+0x551/0x830 Code: 08 31 c9 89 da e8 77 fc ff ff 8b 86 bc 00 00 00 85 ff 89 43 38 8d 46 60 89 43 40 74 1d 83 c4 0c 89 d8 5b 5e 5f 5d e9 aa f9 ff ff <8b> 86 bc 00 00 00 89 43 38 8d 46 60 89 43 40 83 c4 0c 5b 5e 5f EIP: [<c04ebf8f>] io_ioq_move+0xf2/0x109 SS:ESP 0068:f6304c28
Changelog:
Make sure iogs in the hierarchy are built properly after moving a task to a new cgroup.
--- block/cfq-iosched.c | 4 +++- block/elevator-fq.c | 1 + block/elevator-fq.h | 1 + 3 files changed, 5 insertions(+), 1 deletions(-)
block/cfq-iosched.c +3 -1
@@ -12,6 +12,8 @@#include <linux/rbtree.h> #include <linux/ioprio.h> #include <linux/blktrace_api.h> +#include "elevator-fq.h" +/* * tunables */
@@ -1086,7 +1088,7 @@ static void changed_cgroup(struct io_context *ioc, struct cfq_io_context *cic)spin_lock_irqsave (q->queue_lock, flags); - iog =io_lookup_io_group_current (q); + iog =io_get_io_group (q);if (async_cfqq !=NULL ) { __iog =cfqq_to_io_group (async_cfqq);
block/elevator-fq.c +1 -0
@@ -1191,6 +1191,7 @@ struct io_group *io_get_io_group(struct request_queue *q)return iog; } +EXPORT_SYMBOL (io_get_io_group);void io_free_root_group (struct elevator_queue *e) {
block/elevator-fq.h +1 -0
@@ -459,6 +459,7 @@ static inline struct io_group *ioq_to_io_group(struct io_queue *ioq) }#ifdef CONFIG_GROUP_IOSCHED +extern struct io_group *io_get_io_group (struct request_queue *q);extern int io_group_allow_merge (struct request *rq,struct bio *bio);extern void io_ioq_move (struct elevator_queue *e,struct io_queue *ioq,struct io_group *iog);
Signature · 1 line
1.5.4.rc3
Re: [PATCH] IO-Controller: Fix kernel panic after moving a task
Quoted 48 lines · Gui Jianfeng
Quoted 9 lines · Vivek Goyal
(quoted diff) +9
-0+#ifdef CONFIG_IOSCHED_CFQ_HIER +static void changed_cgroup(struct io_context *ioc, struct cfq_io_context *cic) +{ + struct cfq_queue *async_cfqq = cic_to_cfqq(cic, 0); + struct cfq_queue *sync_cfqq = cic_to_cfqq(cic, 1); + struct cfq_data *cfqd = cic->key; + struct io_group *iog, *__iog; + unsigned long flags; + struct request_queue *q;Review
Quoted 10 lines
(quoted diff) +10
-0+ + if (unlikely(!cfqd)) + return; + + q = cfqd->q; + + spin_lock_irqsave(q->queue_lock, flags); + + iog = io_lookup_io_group_current(q); +Hi Vivek,
I triggered another kernel panic when testing. When moving a task to another cgroup, the corresponding iog may not be setup properly all the time. "iog" might be NULL here.
io_ioq_move()receives a NULL iog, kernel crash.Consider the following piece of code:
941 int
elv_set_request(struct request_queue *q, struct request *rq, gfp_t gfp_mask)942 {
943 structelevator_queue*e =q->elevator;
944
945elv_fq_set_request_io_group(q, rq);-->task moving to a new group is happenning here.
946
947 /*948 * Optimization for noop, deadline and AS which maintain only single 949 * ioq per io group 950 */951 if (
elv_iosched_single_ioq(e))
952 returnelv_fq_set_request_ioq(q, rq, gfp_mask);
953
954 if (e->ops->elevator_set_req_fn)
955 returne->ops->elevator_set_req_fn(q, rq, gfp_mask);
Quoted 8 lines
cfq_set_request()will finally callio_ioq_move(), but the iog is NULL, beacause the iogs in the hierarchy are not built yet. So kernel crashes.956
957rq->elevator_private= NULL;
958 return 0;
959 }
Thanks Gui. Good catch.
Quoted 71 lines
BUG: unable to handle kernel NULL pointer dereference at 000000bc IP: [<c04ebf8f>] io_ioq_move+0xf2/0x109 *pde = 6cc00067 Oops: 0000 [#1] SMP last sysfs file: /sys/block/hdb/queue/slice_idle Modules linked in: ipv6 cpufreq_ondemand acpi_cpufreq dm_mirror dm_multipath sbs sbshc battery ac lp snd_hda_codec_realtek snd_hda_intel snd_hda_codec snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss snd_pcm serio_raw snd_timer rtc_cmos parport_pc snd r8169 button rtc_core parport soundcore mii i2c_i801 rtc_lib snd_page_alloc pcspkr i2c_core dm_region_hash dm_log dm_mod ext3 jbd mbcache uhci_hcd ohci_hcd ehci_hcdPid: 5431, comm: dd Not tainted (2.6.29-rc7-vivek #19) Veriton M460
EIP: 0060:[<c04ebf8f>] EFLAGS: 00010046 CPU: 0
EIP is atio_ioq_move+0xf2/0x109
EAX:f6203a88EBX:f6792c94ECX:f6203a84EDX: 00000006
ESI: 00000000 EDI: 00000000 EBP:f6203a60ESP:f6304c28
DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
Process dd (pid: 5431, ti=f6304000 task=f669dae0 task.ti=f6304000)Stack: f62478c0 0100dd40 f6247908 f62d995c 00000000 00000000 f675b54c c04e9182 f638e9b0 00000282 f62d99a4 f6325a2c c04e9113 f5a707c0 c04e7ae0 f675b000 f62d95fc f6325a2c c04e8501 00000010 f631e4e8 f675b000 00080000 ffffff10 Call Trace: [<c04e9182>] changed_cgroup+0x6f/0x8d [<c04e9113>] changed_cgroup+0x0/0x8d [<c04e7ae0>] __call_for_each_cic+0x1b/0x25 [<c04e8501>] cfq_set_request+0x158/0x2c7 [<c06316e6>] _spin_unlock_irqrestore+0x5/0x6 [<c04eb106>] elv_fq_set_request_io_group+0x2b/0x3e [<c04e83a9>] cfq_set_request+0x0/0x2c7 [<c04dddcb>] elv_set_request+0x3e/0x4e [<c04df3da>] get_request+0x1ed/0x29b [<c04df9bb>] get_request_wait+0xdf/0xf2 [<c04dfd89>] __make_request+0x2c6/0x372 [<c049bd76>] do_mpage_readpage+0x4fe/0x5e3 [<c04deba5>] generic_make_request+0x2d0/0x355 [<c04dff47>] submit_bio+0x92/0x97 [<c045bfcb>] add_to_page_cache_locked+0x8a/0xb7 [<c049bfa4>] mpage_end_io_read+0x0/0x50 [<c049b1b6>] mpage_bio_submit+0x19/0x1d [<c049bf9a>] mpage_readpages+0x9b/0xa5 [<f7dd18c7>] ext3_readpages+0x0/0x15 [ext3] [<c0462192>] __do_page_cache_readahead+0xea/0x154 [<f7dd2286>] ext3_get_block+0x0/0xbe [ext3] [<c045d34d>] generic_file_aio_read+0x276/0x569 [<c047cdd9>] do_sync_read+0xbf/0xfe [<c043a3f2>] getnstimeofday+0x51/0xdb [<c0434d3c>] autoremove_wake_function+0x0/0x2d [<c041bdc3>] sched_slice+0x61/0x6a [<c0423114>] task_tick_fair+0x3d/0x60 [<c04c1d79>] security_file_permission+0xc/0xd [<c047cd1a>] do_sync_read+0x0/0xfe [<c047d35a>] vfs_read+0x6c/0x8b [<c047d67e>] sys_read+0x3c/0x63 [<c0402fc1>] sysenter_do_call+0x12/0x21 [<c0630000>] schedule+0x551/0x830 Code: 08 31 c9 89 da e8 77 fc ff ff 8b 86 bc 00 00 00 85 ff 89 43 38 8d 46 60 89 43 40 74 1d 83 c4 0c 89 d8 5b 5e 5f 5d e9 aa f9 ff ff <8b> 86 bc 00 00 00 89 43 38 8d 46 60 89 43 40 83 c4 0c 5b 5e 5f EIP: [<c04ebf8f>] io_ioq_move+0xf2/0x109 SS:ESP 0068:f6304c28Changelog:
Make sure iogs in the hierarchy are built properly after moving a task to a new cgroup.
Signed-off-by Gui Jianfeng <[email protected]>--- block/cfq-iosched.c | 4 +++- block/elevator-fq.c | 1 + block/elevator-fq.h | 1 + 3 files changed, 5 insertions(+), 1 deletions(-)block/cfq-iosched.c +0
-0
block/cfq-iosched.c line 16
Quoted 5 lines
block/cfq-iosched.c +2
-0@@ -12,6 +12,8 @@#include <linux/rbtree.h> #include <linux/ioprio.h> #include <linux/blktrace_api.h> +#include "elevator-fq.h" +
I think above explicit inclusion of "elevator-fq.h" might be unnecessary as elevator.h includes elevator-fq.h and cfq-iosched.c is including elevator.h
block/cfq-iosched.c static void changed_cgroup(struct io_context *ioc, struct cfq_io_context *cic) · line 1091
Quoted 9 lines
block/cfq-iosched.c +1
-1/* * tunables */ @@ -1086,7 +1088,7 @@ static void changed_cgroup(struct io_context *ioc, struct cfq_io_context *cic)spin_lock_irqsave (q->queue_lock, flags); - iog =io_lookup_io_group_current (q); + iog =io_get_io_group (q);
A one line comment here explaining the need to get_io_group instead of lookup_io_group will be nice.
Thanks
Vivek
Quoted 31 lines
block/cfq-iosched.c +0
-0if (async_cfqq !=NULL ) { __iog =cfqq_to_io_group (async_cfqq);block/elevator-fq.c +1
-0@@ -1191,6 +1191,7 @@ struct io_group *io_get_io_group(struct request_queue *q)return iog; } +EXPORT_SYMBOL (io_get_io_group);void io_free_root_group (struct elevator_queue *e) {block/elevator-fq.h +1
-0@@ -459,6 +459,7 @@ static inline struct io_group *ioq_to_io_group(struct io_queue *ioq) }#ifdef CONFIG_GROUP_IOSCHED +extern struct io_group *io_get_io_group (struct request_queue *q);extern int io_group_allow_merge (struct request *rq,struct bio *bio);extern void io_ioq_move (struct elevator_queue *e,struct io_queue *ioq,struct io_group *iog);Signature · 1 line
1.5.4.rc3
PATCH 6/10 Separate out queue and data
In reply to Vivek Goyalo So far noop, deadline and AS had one common structure called *_data which contained both the queue information where requests are queued and also common data used for scheduling. This patch breaks down this common structure in two parts, *_queue and *_data. This is along the lines of cfq where all the reuquests are queued in queue and common data and tunables are part of data.
o It does not change the functionality but this re-organization helps once noop, deadline and AS are changed to use hierarchical fair queuing.
o looks like queue_empty function is not required and we can check for q->nr_sorted in elevator layer to see if ioscheduler queues are empty or not.
--- block/as-iosched.c | 209 ++++++++++++++++++++++++++-------------------- block/deadline-iosched.c | 117 ++++++++++++++++---------- block/elevator.c | 111 +++++++++++++++++++++---- block/noop-iosched.c | 59 ++++++------- include/linux/elevator.h | 8 ++- 5 files changed, 319 insertions(+), 185 deletions(-)
block/as-iosched.c +118 -91
@@ -79,13 +79,7 @@ enum anticipation_status {* or timed out */ }; -struct as_data { -/* -* run time data -*/ - -struct request_queue *q;/* the "owner" queue */ - +struct as_queue {/* * requests (as_rq s) are present on both sort_list and fifo_list */
@@ -93,6 +87,14 @@ struct as_data {struct list_head fifo_list[2 ];struct request *next_rq[2 ];/* next in sort order */ +unsigned long last_check_fifo[2 ]; +int write_batch_count;/* max # of reqs in a write batch */ +int current_write_count;/* how many requests left this batch */ +int write_batch_idled;/* has the write batch gone idle? */ +}; + +struct as_data { +struct request_queue *q;/* the "owner" queue */ sector_t last_sector[2 ];/* last REQ_SYNC & REQ_ASYNC sectors */ unsigned long exit_prob;/* probability a task will exit while
@@ -104,23 +106,19 @@ struct as_data {unsigned long new_ttime_mean;u64 new_seek_total;/* mean seek on new proc */ sector_t new_seek_mean; -unsigned long current_batch_expires; -unsigned long last_check_fifo[2 ]; +int changed_batch;/* 1: waiting for old batch to end */ int new_batch;/* 1: waiting on first read complete */ -int batch_data_dir;/* current batch REQ_SYNC / REQ_ASYNC */ -int write_batch_count;/* max # of reqs in a write batch */ -int current_write_count;/* how many requests left this batch */ -int write_batch_idled;/* has the write batch gone idle? */ enum anticipation_status antic_status;unsigned long antic_start;/* jiffies: when it started */ struct timer_list antic_timer;/* anticipatory scheduling timer */ -struct work_struct antic_work;/* Deferred unplugging */ +struct work_struct antic_work;/* Deferred unplugging */ struct io_context *io_context;/* Identify the expected process */ int ioc_finished;/* IO associated with io_context is finished */ int nr_dispatched; +int batch_data_dir;/* current batch REQ_SYNC / REQ_ASYNC */ /* * settings that change how the i/o scheduler behaves
@@ -261,13 +259,14 @@ static void as_put_io_context(struct request *rq)/* * rb tree support functions */ -#define RQ_RB_ROOT (ad, rq) (&(ad)->sort_list[rq_is_sync((rq))]) +#define RQ_RB_ROOT (asq, rq) (&(asq)->sort_list[rq_is_sync((rq))])static void as_add_rq_rb (struct as_data *ad,struct request *rq) {struct request *alias; +struct as_queue *asq =elv_get_sched_queue (ad->q, rq); -while ((unlikely (alias =elv_rb_add (RQ_RB_ROOT (ad, rq), rq)))) { +while ((unlikely (alias =elv_rb_add (RQ_RB_ROOT (asq, rq), rq)))) {as_move_to_dispatch (ad, alias);as_antic_stop (ad); }
@@ -275,7 +274,9 @@ static void as_add_rq_rb(struct as_data *ad, struct request *rq)static inline void as_del_rq_rb (struct as_data *ad,struct request *rq) { -elv_rb_del (RQ_RB_ROOT (ad, rq), rq); +struct as_queue *asq =elv_get_sched_queue (ad->q, rq); + +elv_rb_del (RQ_RB_ROOT (asq, rq), rq); }/*
@@ -369,7 +370,7 @@ as_choose_req(struct as_data *ad, struct request *rq1, struct request *rq2)* what request to process next. Anticipation works on top of this. */ static struct request * -as_find_next_rq (struct as_data *ad,struct request *last) +as_find_next_rq (struct as_data *ad,struct as_queue *asq,struct request *last) {struct rb_node *rbnext =rb_next (&last->rb_node);struct rb_node *rbprev =rb_prev (&last->rb_node);
@@ -385,7 +386,7 @@ as_find_next_rq(struct as_data *ad, struct request *last)else {const int data_dir =rq_is_sync (last); - rbnext =rb_first (&ad->sort_list[data_dir]); + rbnext =rb_first (&asq->sort_list[data_dir]);if (rbnext && rbnext != &last->rb_node) next =rb_entry_rq (rbnext); }
@@ -790,9 +791,10 @@ static int as_can_anticipate(struct as_data *ad, struct request *rq)static void as_update_rq (struct as_data *ad,struct request *rq) {const int data_dir =rq_is_sync (rq); +struct as_queue *asq =elv_get_sched_queue (ad->q, rq);/* keep the next_rq cache up to date */ - ad->next_rq[data_dir] =as_choose_req (ad, rq, ad->next_rq[data_dir]); + asq->next_rq[data_dir] =as_choose_req (ad, rq, asq->next_rq[data_dir]);/* * have we been anticipating this request?
@@ -813,25 +815,26 @@ static void update_write_batch(struct as_data *ad) {unsigned long batch = ad->batch_expire[REQ_ASYNC ];long write_time; +struct as_queue *asq =elv_get_sched_queue (ad->q,NULL ); write_time = (jiffies - ad->current_batch_expires) + batch;if (write_time <0 ) write_time =0 ; -if (write_time > batch && !ad->write_batch_idled) { +if (write_time > batch && !asq->write_batch_idled) {if (write_time > batch *3 ) - ad->write_batch_count /=2 ; + asq->write_batch_count /=2 ;else - ad->write_batch_count--; - }else if (write_time < batch && ad->current_write_count ==0 ) { + asq->write_batch_count--; + }else if (write_time < batch && asq->current_write_count ==0 ) {if (batch > write_time *3 ) - ad->write_batch_count *=2 ; + asq->write_batch_count *=2 ;else - ad->write_batch_count++; + asq->write_batch_count++; } -if (ad->write_batch_count <1 ) - ad->write_batch_count =1 ; +if (asq->write_batch_count <1 ) + asq->write_batch_count =1 ; }/*
@@ -902,6 +905,7 @@ static void as_remove_queued_request(struct request_queue *q,const int data_dir =rq_is_sync (rq);struct as_data *ad = q->elevator->elevator_data;struct io_context *ioc; +struct as_queue *asq =elv_get_sched_queue (q, rq);WARN_ON (RQ_STATE (rq) !=AS_RQ_QUEUED );
@@ -915,8 +919,8 @@ static void as_remove_queued_request(struct request_queue *q,* Update the "next_rq" cache if we are about to remove its * entry */ -if (ad->next_rq[data_dir] == rq) - ad->next_rq[data_dir] =as_find_next_rq (ad, rq); +if (asq->next_rq[data_dir] == rq) + asq->next_rq[data_dir] =as_find_next_rq (ad, asq, rq);rq_fifo_clear (rq);as_del_rq_rb (ad, rq);
@@ -930,23 +934,23 @@ static void as_remove_queued_request(struct request_queue *q,* * See as_antic_expired comment. */ -static int as_fifo_expired (struct as_data *ad,int adir) +static int as_fifo_expired (struct as_data *ad,struct as_queue *asq,int adir) {struct request *rq;long delta_jif; - delta_jif = jiffies - ad->last_check_fifo[adir]; + delta_jif = jiffies - asq->last_check_fifo[adir];if (unlikely (delta_jif <0 )) delta_jif = -delta_jif;if (delta_jif < ad->fifo_expire[adir])return 0 ; - ad->last_check_fifo[adir] = jiffies; + asq->last_check_fifo[adir] = jiffies; -if (list_empty (&ad->fifo_list[adir])) +if (list_empty (&asq->fifo_list[adir]))return 0 ; - rq =rq_entry_fifo (ad->fifo_list[adir].next); + rq =rq_entry_fifo (asq->fifo_list[adir].next);return time_after (jiffies,rq_fifo_time (rq)); }
@@ -955,7 +959,7 @@ static int as_fifo_expired(struct as_data *ad, int adir)* as_batch_expired returns true if the current batch has expired. A batch * is a set of reads or a set of writes. */ -static inline int as_batch_expired (struct as_data *ad) +static inline int as_batch_expired (struct as_data *ad,struct as_queue *asq) {if (ad->changed_batch || ad->new_batch)return 0 ;
@@ -965,7 +969,7 @@ static inline int as_batch_expired(struct as_data *ad)return time_after (jiffies, ad->current_batch_expires);return time_after (jiffies, ad->current_batch_expires) - || ad->current_write_count ==0 ; + || asq->current_write_count ==0 ; }/*
@@ -974,6 +978,7 @@ static inline int as_batch_expired(struct as_data *ad)static void as_move_to_dispatch (struct as_data *ad,struct request *rq) {const int data_dir =rq_is_sync (rq); +struct as_queue *asq =elv_get_sched_queue (ad->q, rq);BUG_ON (RB_EMPTY_NODE (&rq->rb_node));
@@ -996,12 +1001,12 @@ static void as_move_to_dispatch(struct as_data *ad, struct request *rq) ad->io_context =NULL ; } -if (ad->current_write_count !=0 ) - ad->current_write_count--; +if (asq->current_write_count !=0 ) + asq->current_write_count--; } ad->ioc_finished =0 ; - ad->next_rq[data_dir] =as_find_next_rq (ad, rq); + asq->next_rq[data_dir] =as_find_next_rq (ad, asq, rq);/* * take it off the sort and fifo list, add to dispatch queue
@@ -1025,10 +1030,16 @@ static void as_move_to_dispatch(struct as_data *ad, struct request *rq)static int as_dispatch_request (struct request_queue *q,int force) {struct as_data *ad = q->elevator->elevator_data; -const int reads = !list_empty (&ad->fifo_list[REQ_SYNC ]); -const int writes = !list_empty (&ad->fifo_list[REQ_ASYNC ]); +struct as_queue *asq =elv_select_sched_queue (q, force); +int reads, writes;struct request *rq; +if (!asq) +return 0 ; + + reads = !list_empty (&asq->fifo_list[REQ_SYNC ]); + writes = !list_empty (&asq->fifo_list[REQ_ASYNC ]); +if (unlikely (force)) {/* * Forced dispatch, accounting is useless. Reset
@@ -1043,25 +1054,25 @@ static int as_dispatch_request(struct request_queue *q, int force) ad->changed_batch =0 ; ad->new_batch =0 ; -while (ad->next_rq[REQ_SYNC ]) { -as_move_to_dispatch (ad, ad->next_rq[REQ_SYNC ]); +while (asq->next_rq[REQ_SYNC ]) { +as_move_to_dispatch (ad, asq->next_rq[REQ_SYNC ]); dispatched++; } - ad->last_check_fifo[REQ_SYNC ] = jiffies; + asq->last_check_fifo[REQ_SYNC ] = jiffies; -while (ad->next_rq[REQ_ASYNC ]) { -as_move_to_dispatch (ad, ad->next_rq[REQ_ASYNC ]); +while (asq->next_rq[REQ_ASYNC ]) { +as_move_to_dispatch (ad, asq->next_rq[REQ_ASYNC ]); dispatched++; } - ad->last_check_fifo[REQ_ASYNC ] = jiffies; + asq->last_check_fifo[REQ_ASYNC ] = jiffies;return dispatched; }/* Signal that the write batch was uncontended, so we can't time it */ if (ad->batch_data_dir ==REQ_ASYNC && !reads) { -if (ad->current_write_count ==0 || !writes) - ad->write_batch_idled =1 ; +if (asq->current_write_count ==0 || !writes) + asq->write_batch_idled =1 ; } if (!(reads || writes)
@@ -1070,14 +1081,14 @@ static int as_dispatch_request(struct request_queue *q, int force) || ad->changed_batch)return 0 ; -if (!(reads && writes &&as_batch_expired (ad))) { +if (!(reads && writes &&as_batch_expired (ad, asq))) {/* * batch is still running or no reads or no writes */ - rq = ad->next_rq[ad->batch_data_dir]; + rq = asq->next_rq[ad->batch_data_dir];if (ad->batch_data_dir ==REQ_SYNC && ad->antic_expire) { -if (as_fifo_expired (ad,REQ_SYNC )) +if (as_fifo_expired (ad, asq,REQ_SYNC )) goto fifo_expired;if (as_can_anticipate (ad, rq)) {
@@ -1101,7 +1112,7 @@ static int as_dispatch_request(struct request_queue *q, int force)*/ if (reads) { -BUG_ON (RB_EMPTY_ROOT (&ad->sort_list[REQ_SYNC ])); +BUG_ON (RB_EMPTY_ROOT (&asq->sort_list[REQ_SYNC ]));if (writes && ad->batch_data_dir ==REQ_SYNC )/*
@@ -1114,8 +1125,8 @@ static int as_dispatch_request(struct request_queue *q, int force) ad->changed_batch =1 ; } ad->batch_data_dir =REQ_SYNC ; - rq =rq_entry_fifo (ad->fifo_list[REQ_SYNC ].next); - ad->last_check_fifo[ad->batch_data_dir] = jiffies; + rq =rq_entry_fifo (asq->fifo_list[REQ_SYNC ].next); + asq->last_check_fifo[ad->batch_data_dir] = jiffies; goto dispatch_request; }
@@ -1125,7 +1136,7 @@ static int as_dispatch_request(struct request_queue *q, int force)if (writes) { dispatch_writes: -BUG_ON (RB_EMPTY_ROOT (&ad->sort_list[REQ_ASYNC ])); +BUG_ON (RB_EMPTY_ROOT (&asq->sort_list[REQ_ASYNC ]));if (ad->batch_data_dir ==REQ_SYNC ) { ad->changed_batch =1 ;
@@ -1138,10 +1149,10 @@ dispatch_writes: ad->new_batch =0 ; } ad->batch_data_dir =REQ_ASYNC ; - ad->current_write_count = ad->write_batch_count; - ad->write_batch_idled =0 ; - rq =rq_entry_fifo (ad->fifo_list[REQ_ASYNC ].next); - ad->last_check_fifo[REQ_ASYNC ] = jiffies; + asq->current_write_count = asq->write_batch_count; + asq->write_batch_idled =0 ; + rq =rq_entry_fifo (asq->fifo_list[REQ_ASYNC ].next); + asq->last_check_fifo[REQ_ASYNC ] = jiffies; goto dispatch_request; }
@@ -1153,9 +1164,9 @@ dispatch_request:* If a request has expired, service it. */ -if (as_fifo_expired (ad, ad->batch_data_dir)) { +if (as_fifo_expired (ad, asq, ad->batch_data_dir)) { fifo_expired: - rq =rq_entry_fifo (ad->fifo_list[ad->batch_data_dir].next); + rq =rq_entry_fifo (asq->fifo_list[ad->batch_data_dir].next); }if (ad->changed_batch) {
@@ -1188,6 +1199,7 @@ static void as_add_request(struct request_queue *q, struct request *rq) {struct as_data *ad = q->elevator->elevator_data;int data_dir; +struct as_queue *asq =elv_get_sched_queue (q, rq);RQ_SET_STATE (rq,AS_RQ_NEW );
@@ -1206,7 +1218,7 @@ static void as_add_request(struct request_queue *q, struct request *rq)* set expire time and add to fifo list */ rq_set_fifo_time (rq, jiffies + ad->fifo_expire[data_dir]); -list_add_tail (&rq->queuelist, &ad->fifo_list[data_dir]); +list_add_tail (&rq->queuelist, &asq->fifo_list[data_dir]);as_update_rq (ad, rq);/* keep state machine up to date */ RQ_SET_STATE (rq,AS_RQ_QUEUED );
@@ -1228,31 +1240,20 @@ static void as_deactivate_request(struct request_queue *q, struct request *rq)atomic_inc (&RQ_IOC (rq)->aic->nr_dispatched); } -/* -* as_queue_empty tells us if there are requests left in the device. It may -* not be the case that a driver can get the next request even if the queue -* is not empty - it is used in the block layer to check for plugging and -* merging opportunities -*/ -static int as_queue_empty (struct request_queue *q) -{ -struct as_data *ad = q->elevator->elevator_data; - -return list_empty (&ad->fifo_list[REQ_ASYNC ]) - &&list_empty (&ad->fifo_list[REQ_SYNC ]); -} -static int as_merge (struct request_queue *q,struct request **req,struct bio *bio) { -struct as_data *ad = q->elevator->elevator_data;sector_t rb_key = bio->bi_sector +bio_sectors (bio);struct request *__rq; +struct as_queue *asq =elv_get_sched_queue_current (q); + +if (!asq) +return ELEVATOR_NO_MERGE ;/* * check for front merge */ - __rq =elv_rb_find (&ad->sort_list[bio_data_dir (bio)], rb_key); + __rq =elv_rb_find (&asq->sort_list[bio_data_dir (bio)], rb_key);if (__rq &&elv_rq_merge_ok (__rq, bio)) { *req = __rq;return ELEVATOR_FRONT_MERGE ;
@@ -1339,6 +1340,41 @@ static int as_may_queue(struct request_queue *q, int rw)return ret; } +/* Called with queue lock held */ +static void *as_alloc_as_queue (struct request_queue *q, +struct elevator_queue *eq,gfp_t gfp_mask) +{ +struct as_queue *asq; +struct as_data *ad = eq->elevator_data; + + asq =kmalloc_node (sizeof (*asq), gfp_mask | __GFP_ZERO, q->node); +if (asq ==NULL ) + goto out; + +INIT_LIST_HEAD (&asq->fifo_list[REQ_SYNC ]); +INIT_LIST_HEAD (&asq->fifo_list[REQ_ASYNC ]); + asq->sort_list[REQ_SYNC ] =RB_ROOT ; + asq->sort_list[REQ_ASYNC ] =RB_ROOT ; +if (ad) + asq->write_batch_count = ad->batch_expire[REQ_ASYNC ] /10 ; +else + asq->write_batch_count = default_write_batch_expire /10 ; + +if (asq->write_batch_count <2 ) + asq->write_batch_count =2 ; +out: +return asq; +} + +static void as_free_as_queue (struct elevator_queue *e,void *sched_queue) +{ +struct as_queue *asq = sched_queue; + +BUG_ON (!list_empty (&asq->fifo_list[REQ_SYNC ])); +BUG_ON (!list_empty (&asq->fifo_list[REQ_ASYNC ])); +kfree (asq); +} +static void as_exit_queue (struct elevator_queue *e) {struct as_data *ad = e->elevator_data;
@@ -1346,9 +1382,6 @@ static void as_exit_queue(struct elevator_queue *e)del_timer_sync (&ad->antic_timer);cancel_work_sync (&ad->antic_work); -BUG_ON (!list_empty (&ad->fifo_list[REQ_SYNC ])); -BUG_ON (!list_empty (&ad->fifo_list[REQ_ASYNC ])); -put_io_context (ad->io_context);kfree (ad); }
@@ -1372,10 +1405,6 @@ static void *as_init_queue(struct request_queue *q)init_timer (&ad->antic_timer);INIT_WORK (&ad->antic_work, as_work_handler); -INIT_LIST_HEAD (&ad->fifo_list[REQ_SYNC ]); -INIT_LIST_HEAD (&ad->fifo_list[REQ_ASYNC ]); - ad->sort_list[REQ_SYNC ] =RB_ROOT ; - ad->sort_list[REQ_ASYNC ] =RB_ROOT ; ad->fifo_expire[REQ_SYNC ] = default_read_expire; ad->fifo_expire[REQ_ASYNC ] = default_write_expire; ad->antic_expire = default_antic_expire;
@@ -1383,9 +1412,6 @@ static void *as_init_queue(struct request_queue *q) ad->batch_expire[REQ_ASYNC ] = default_write_batch_expire; ad->current_batch_expires = jiffies + ad->batch_expire[REQ_SYNC ]; - ad->write_batch_count = ad->batch_expire[REQ_ASYNC ] /10 ; -if (ad->write_batch_count <2 ) - ad->write_batch_count =2 ;return ad; }
@@ -1482,7 +1508,6 @@ static struct elevator_type iosched_as = { .elevator_add_req_fn = as_add_request, .elevator_activate_req_fn = as_activate_request, .elevator_deactivate_req_fn = as_deactivate_request, - .elevator_queue_empty_fn = as_queue_empty, .elevator_completed_req_fn = as_completed_request, .elevator_former_req_fn = elv_rb_former_request, .elevator_latter_req_fn = elv_rb_latter_request,
@@ -1490,6 +1515,8 @@ static struct elevator_type iosched_as = { .elevator_init_fn = as_init_queue, .elevator_exit_fn = as_exit_queue, .trim = as_trim, + .elevator_alloc_sched_queue_fn = as_alloc_as_queue, + .elevator_free_sched_queue_fn = as_free_as_queue, }, .elevator_attrs = as_attrs,
block/deadline-iosched.c +71 -46
266 lines not on this page · raw message
block/elevator.c +96 -15
179 lines not on this page · raw message
block/noop-iosched.c +28 -31
111 lines not on this page · raw message
include/linux/elevator.h +6 -2
@@ -30,8 +30,9 @@ typedef void (elevator_deactivate_req_fn) (struct request_queue *, struct requestypedef void *(elevator_init_fn ) (struct request_queue *);typedef void (elevator_exit_fn ) (struct elevator_queue *); -#ifdef CONFIG_ELV_FAIR_QUEUING +typedef void * (elevator_alloc_sched_queue_fn ) (struct request_queue *q,struct elevator_queue *eq,gfp_t );typedef void (elevator_free_sched_queue_fn ) (struct elevator_queue *,void *); +#ifdef CONFIG_ELV_FAIR_QUEUING typedef void (elevator_active_ioq_set_fn ) (struct request_queue *,void *);typedef void (elevator_active_ioq_reset_fn ) (struct request_queue *);typedef void (elevator_arm_slice_timer_fn ) (struct request_queue *,void *);
@@ -68,8 +69,9 @@ struct elevator_opselevator_exit_fn *elevator_exit_fn;void (*trim)(struct io_context *); -#ifdef CONFIG_ELV_FAIR_QUEUING +elevator_alloc_sched_queue_fn *elevator_alloc_sched_queue_fn;elevator_free_sched_queue_fn *elevator_free_sched_queue_fn; +#ifdef CONFIG_ELV_FAIR_QUEUING elevator_active_ioq_set_fn *elevator_active_ioq_set_fn;elevator_active_ioq_reset_fn *elevator_active_ioq_reset_fn;
@@ -109,6 +111,7 @@ struct elevator_queue {struct elevator_ops *ops;void *elevator_data; +void *sched_queue;struct kobject kobj;struct elevator_type *elevator_type;struct mutex sysfs_lock;
@@ -256,5 +259,6 @@ static inline int elv_iosched_fair_queuing_enabled(struct elevator_queue *e)#endif /* ELV_IOSCHED_FAIR_QUEUING */ extern void *elv_get_sched_queue (struct request_queue *q,struct request *rq);extern void *elv_select_sched_queue (struct request_queue *q,int force); +extern void *elv_get_sched_queue_current (struct request_queue *q);#endif /* CONFIG_BLOCK */ #endif
Signature · 1 line
1.6.0.1
PATCH 7/10 Prepare elevator layer for single queue schedulers
In reply to Vivek GoyalElevator layer now has support for hierarchical fair queuing. cfq has been migrated to make use of it and now it is time to do groundwork for noop, deadline and AS.
noop deadline and AS don't maintain separate queues for different processes. There is only one single queue. Effectively one can think that in hierarchical setup, there will be one queue per cgroup where requests from all the processes in the cgroup will be queued.
Generally io scheduler takes care of creating queues. Because there is only one queue here, we have modified common layer to take care of queue creation and some other functionality. This special casing helps in keeping the changes to noop, deadline and AS to the minimum.
--- block/elevator-fq.c | 153 ++++++++++++++++++++++++++++++++++++++++++++++ block/elevator-fq.h | 67 ++++++++++++++++++++ block/elevator.c | 35 ++++++++++- include/linux/elevator.h | 14 ++++ 4 files changed, 268 insertions(+), 1 deletions(-)
block/elevator-fq.c +153 -0
@@ -857,6 +857,12 @@ void io_put_io_group_queues(struct elevator_queue *e, struct io_group *iog)/* Free up async idle queue */ elv_release_ioq (e, &iog->async_idle_queue); + +#ifdef CONFIG_GROUP_IOSCHED +/* Optimization for io schedulers having single ioq */ +if (elv_iosched_single_ioq (e)) +elv_release_ioq (e, &iog->ioq); +#endif }
@@ -1538,6 +1544,153 @@ void elv_fq_set_request_io_group(struct request_queue *q, rq->iog = iog; } +/* +* Find/Create the io queue the rq should go in. This is an optimization +* for the io schedulers (noop, deadline and AS) which maintain only single +* io queue per cgroup. In this case common layer can just maintain a +* pointer in group data structure and keeps track of it. +* +* For the io schdulers like cfq, which maintain multiple io queues per +* cgroup, and decide the io queue of request based on process, this +* function is not invoked. +*/ +int elv_fq_set_request_ioq (struct request_queue *q,struct request *rq, +gfp_t gfp_mask) +{ +struct elevator_queue *e = q->elevator; +unsigned long flags; +struct io_queue *ioq =NULL , *new_ioq =NULL ; +struct io_group *iog; +void *sched_q =NULL , *new_sched_q =NULL ; + +if (!elv_iosched_fair_queuing_enabled (e)) +return 0 ; + +might_sleep_if (gfp_mask & __GFP_WAIT); +spin_lock_irqsave (q->queue_lock, flags); + +/* Determine the io group request belongs to */ + iog = rq->iog; +BUG_ON (!iog); + +retry: +/* Get the iosched queue */ + ioq =io_group_ioq (iog); +if (!ioq) { +/* io queue and sched_queue needs to be allocated */ +BUG_ON (!e->ops->elevator_alloc_sched_queue_fn); + +if (new_sched_q) { + goto alloc_ioq; + }else if (gfp_mask & __GFP_WAIT) { +/* +* Inform the allocator of the fact that we will +* just repeat this allocation if it fails, to allow +* the allocator to do whatever it needs to attempt to +* free memory. +*/ +spin_unlock_irq (q->queue_lock); +/* Call io scheduer to create scheduler queue */ + new_sched_q = e->ops->elevator_alloc_sched_queue_fn (q, + e, gfp_mask | __GFP_NOFAIL + | __GFP_ZERO); +spin_lock_irq (q->queue_lock); + goto retry; + }else { + sched_q = e->ops->elevator_alloc_sched_queue_fn (q, e, + gfp_mask | __GFP_ZERO); +if (!sched_q) + goto queue_fail; + } + +alloc_ioq: +if (new_ioq) { + ioq = new_ioq; + new_ioq =NULL ; + sched_q = new_sched_q; + new_sched_q =NULL ; + }else if (gfp_mask & __GFP_WAIT) { +/* +* Inform the allocator of the fact that we will +* just repeat this allocation if it fails, to allow +* the allocator to do whatever it needs to attempt to +* free memory. +*/ +spin_unlock_irq (q->queue_lock); + new_ioq =elv_alloc_ioq (q, gfp_mask | __GFP_NOFAIL + | __GFP_ZERO); +spin_lock_irq (q->queue_lock); + goto retry; + }else { + ioq =elv_alloc_ioq (q, gfp_mask | __GFP_ZERO); +if (!ioq) { + e->ops->elevator_free_sched_queue_fn (e, + sched_q); + sched_q =NULL ; + goto queue_fail; + } + } + +elv_init_ioq (e, ioq, sched_q,IOPRIO_CLASS_BE ,4 ,1 ); +io_group_set_ioq (iog, ioq); +elv_mark_ioq_sync (ioq); + } + +if (new_sched_q) + e->ops->elevator_free_sched_queue_fn (q->elevator, sched_q); + +if (new_ioq) +elv_free_ioq (new_ioq); + +/* Request reference */ +elv_get_ioq (ioq); + rq->ioq = ioq; +spin_unlock_irqrestore (q->queue_lock, flags); +return 0 ; + +queue_fail: +WARN_ON ((gfp_mask & __GFP_WAIT) && !ioq); +elv_schedule_dispatch (q); +spin_unlock_irqrestore (q->queue_lock, flags); +return 1 ; +} + +/* +* Find out the io queue of current task. Optimization for single ioq +* per io group io schedulers. +*/ +struct io_queue *elv_lookup_ioq_current (struct request_queue *q) +{ +struct io_group *iog; + +/* Determine the io group and io queue of the bio submitting task */ + iog =io_lookup_io_group_current (q); +if (!iog) { +/* May be task belongs to a cgroup for which io group has +* not been setup yet. */ +return NULL ; + } +return io_group_ioq (iog); +} + +/* +* This request has been serviced. Clean up ioq info and drop the reference. +* Again this is called only for single queue per cgroup schedulers (noop, +* deadline, AS). +*/ +void elv_fq_unset_request_ioq (struct request_queue *q,struct request *rq) +{ +struct io_queue *ioq = rq->ioq; + +if (!elv_iosched_fair_queuing_enabled (q->elevator)) +return ; + +if (ioq) { + rq->ioq =NULL ; +elv_put_ioq (ioq); + } +} +#else /* GROUP_IOSCHED */ void bfq_init_entity (struct io_entity *entity,struct io_group *iog) {
block/elevator-fq.h +67 -0
90 lines not on this page · raw message
block/elevator.c +34 -1
@@ -207,6 +207,14 @@ static void *elevator_alloc_sched_queue(struct request_queue *q, {void *sched_queue =NULL ; +/* +* If fair queuing is enabled, then queue allocation takes place +* during set_request() functions when request actually comes +* in. +*/ +if (elv_iosched_fair_queuing_enabled (eq)) +return NULL ; +if (eq->ops->elevator_alloc_sched_queue_fn) { sched_queue = eq->ops->elevator_alloc_sched_queue_fn (q, eq,GFP_KERNEL );
@@ -936,6 +944,13 @@ int elv_set_request(struct request_queue *q, struct request *rq, gfp_t gfp_mask)elv_fq_set_request_io_group (q, rq); +/* +* Optimization for noop, deadline and AS which maintain only single +* ioq per io group +*/ +if (elv_iosched_single_ioq (e)) +return elv_fq_set_request_ioq (q, rq, gfp_mask); +if (e->ops->elevator_set_req_fn)return e->ops->elevator_set_req_fn (q, rq, gfp_mask);
@@ -947,6 +962,15 @@ void elv_put_request(struct request_queue *q, struct request *rq) {struct elevator_queue *e = q->elevator; +/* +* Optimization for noop, deadline and AS which maintain only single +* ioq per io group +*/ +if (elv_iosched_single_ioq (e)) { +elv_fq_unset_request_ioq (q, rq); +return ; + } +if (e->ops->elevator_put_req_fn) e->ops->elevator_put_req_fn (rq); }
@@ -1329,9 +1353,18 @@ EXPORT_SYMBOL(elv_select_sched_queue);/* * Get the io scheduler queue pointer for current task. +* +* If fair queuing is enabled, determine the io group of task and retrieve +* the ioq pointer from that. This is used by only single queue ioschedulers +* for retrieving the queue associated with the group to decide whether the +* new bio can do a front merge or not. */ void *elv_get_sched_queue_current (struct request_queue *q) { -return q->elevator->sched_queue; +/* Fair queuing is not enabled. There is only one queue. */ +if (!elv_iosched_fair_queuing_enabled (q->elevator)) +return q->elevator->sched_queue; + +return ioq_sched_queue (elv_lookup_ioq_current (q)); }EXPORT_SYMBOL (elv_get_sched_queue_current);
include/linux/elevator.h +14 -0
@@ -245,17 +245,31 @@ enum {/* iosched wants to use fq logic of elevator layer */ #define ELV_IOSCHED_NEED_FQ 1 +/* iosched maintains only single ioq per group.*/ +#define ELV_IOSCHED_SINGLE_IOQ 2 +static inline int elv_iosched_fair_queuing_enabled (struct elevator_queue *e) {return (e->elevator_type->elevator_features) &ELV_IOSCHED_NEED_FQ ; } +static inline int elv_iosched_single_ioq (struct elevator_queue *e) +{ +return (e->elevator_type->elevator_features) &ELV_IOSCHED_SINGLE_IOQ ; +} +#else /* ELV_IOSCHED_FAIR_QUEUING */ static inline int elv_iosched_fair_queuing_enabled (struct elevator_queue *e) {return 0 ; } + +static inline int elv_iosched_single_ioq (struct elevator_queue *e) +{ +return 0 ; +} +#endif /* ELV_IOSCHED_FAIR_QUEUING */ extern void *elv_get_sched_queue (struct request_queue *q,struct request *rq);extern void *elv_select_sched_queue (struct request_queue *q,int force);
Signature · 1 line
1.6.0.1
PATCH 8/10 noop changes for hierarchical fair queuing
In reply to Vivek GoyalThis patch changes noop to use queue scheduling code from elevator layer. One can go back to old noop by deselecting CONFIG_IOSCHED_NOOP_HIER.
--- block/Kconfig.iosched | 11 +++++++++++ block/noop-iosched.c | 3 +++ 2 files changed, 14 insertions(+), 0 deletions(-)
block/Kconfig.iosched +11 -0
@@ -25,6 +25,17 @@ config IOSCHED_NOOP that do their own scheduling and require only minimal assistance from the kernel. +config IOSCHED_NOOP_HIER + bool "Noop Hierarchical Scheduling support" + depends on IOSCHED_NOOP && CGROUPS + select ELV_FAIR_QUEUING + select GROUP_IOSCHED + default n + ---help--- + Enable hierarhical scheduling in noop. In this mode noop keeps + one IO queue per cgroup instead of a global queue. Elevator + fair queuing logic ensures fairness among various queues. + config IOSCHED_AS tristate "Anticipatory I/O scheduler" default y
block/noop-iosched.c +3 -0
@@ -92,6 +92,9 @@ static struct elevator_type elevator_noop = { .elevator_alloc_sched_queue_fn = noop_alloc_noop_queue, .elevator_free_sched_queue_fn = noop_free_noop_queue, }, +#ifdef CONFIG_IOSCHED_NOOP_HIER + .elevator_features = ELV_IOSCHED_NEED_FQ | ELV_IOSCHED_SINGLE_IOQ, +#endif .elevator_name = "noop", .elevator_owner = THIS_MODULE, };
Signature · 1 line
1.6.0.1
PATCH 9/10 deadline changes for hierarchical fair queuing
In reply to Vivek GoyalThis patch changes deadline to use queue scheduling code from elevator layer. One can go back to old deadline by selecting CONFIG_IOSCHED_DEADLINE_HIER.
--- block/Kconfig.iosched | 11 +++++++++++ block/deadline-iosched.c | 3 +++ 2 files changed, 14 insertions(+), 0 deletions(-)
block/Kconfig.iosched +11 -0
@@ -55,6 +55,17 @@ config IOSCHED_DEADLINE a disk at any one time, its behaviour is almost identical to the anticipatory I/O scheduler and so is a good choice. +config IOSCHED_DEADLINE_HIER + bool "Deadline Hierarchical Scheduling support" + depends on IOSCHED_DEADLINE && CGROUPS + select ELV_FAIR_QUEUING + select GROUP_IOSCHED + default n + ---help--- + Enable hierarhical scheduling in deadline. In this mode deadline keeps + one IO queue per cgroup instead of a global queue. Elevator + fair queuing logic ensures fairness among various queues. + config IOSCHED_CFQ tristate "CFQ I/O scheduler" select ELV_FAIR_QUEUING
block/deadline-iosched.c +3 -0
@@ -477,6 +477,9 @@ static struct elevator_type iosched_deadline = { .elevator_alloc_sched_queue_fn = deadline_alloc_deadline_queue, .elevator_free_sched_queue_fn = deadline_free_deadline_queue, }, +#ifdef CONFIG_IOSCHED_DEADLINE_HIER + .elevator_features = ELV_IOSCHED_NEED_FQ | ELV_IOSCHED_SINGLE_IOQ, +#endif .elevator_attrs = deadline_attrs, .elevator_name = "deadline", .elevator_owner = THIS_MODULE,
Signature · 1 line
1.6.0.1
PATCH 10/10 anticipatory changes for hierarchical fair queuing
In reply to Vivek GoyalThis patch changes anticipatory scheduler to use queue scheduling code from elevator layer. One can go back to old as by deselecting CONFIG_IOSCHED_AS_HIER.
TODO/Issues
===========
- AS anticipation logic does not seem to be sufficient to provide BW difference if two "dd" are going in two different cgroups. Needs to be looked into.
- AS write batch number of request adjustment happens upon every W->R batch direction switch. This automatic adjustment depends on how much time a read is taking after a W->R switch.
This does not gel very well when hierarhical scheduling is enabled and every io group can have its separate read/write batch. Now if io group switching takes place it creates issues.
Currently I have disabled write batch length adjustment in hierarchical mode.
- Currently performance seems to be very bad in hierarhical mode. Needs to be looked into.
- I think the whole idea of common layer doing time slice switching between queues and then queue in turn running timed batches is not very good. May be AS can maintain two queues (one for READS and other for WRITES) and let common layer do the time slice switching between these two queues.
--- block/Kconfig.iosched | 12 +++ block/as-iosched.c | 176 +++++++++++++++++++++++++++++++++++++++++++++- block/elevator-fq.c | 77 ++++++++++++++++---- include/linux/elevator.h | 16 ++++ 4 files changed, 265 insertions(+), 16 deletions(-)
block/Kconfig.iosched +12 -0
@@ -45,6 +45,18 @@ config IOSCHED_AS deadline I/O scheduler, it can also be slower in some cases especially some database loads. +config IOSCHED_AS_HIER + bool "Anticipatory Hierarchical Scheduling support" + depends on IOSCHED_AS && CGROUPS + select ELV_FAIR_QUEUING + select GROUP_IOSCHED + default n + ---help--- + Enable hierarhical scheduling in anticipatory. In this mode + anticipatory keeps one IO queue per cgroup instead of a global + queue. Elevator fair queuing logic ensures fairness among various + queues. + config IOSCHED_DEADLINE tristate "Deadline I/O scheduler" default y
block/as-iosched.c +175 -1
214 lines not on this page · raw message
block/elevator-fq.c +62 -15
125 lines not on this page · raw message
include/linux/elevator.h +16 -0
@@ -40,6 +40,7 @@ typedef int (elevator_should_preempt_fn) (struct request_queue*, void*, struct request*);typedef int (elevator_update_idle_window_fn ) (struct elevator_queue *,void *,struct request *); +typedef int (elevator_expire_ioq_fn ) (struct request_queue *,void *,int ,int );#endif struct elevator_ops
@@ -78,6 +79,7 @@ struct elevator_opselevator_arm_slice_timer_fn *elevator_arm_slice_timer_fn;elevator_should_preempt_fn *elevator_should_preempt_fn;elevator_update_idle_window_fn *elevator_update_idle_window_fn; +elevator_expire_ioq_fn *elevator_expire_ioq_fn;#endif };
@@ -248,6 +250,9 @@ enum {/* iosched maintains only single ioq per group.*/ #define ELV_IOSCHED_SINGLE_IOQ 2 +/* iosched does not need anticipation/idling logic support from common layer */ +#define ELV_IOSCHED_DONT_IDLE 4 +static inline int elv_iosched_fair_queuing_enabled (struct elevator_queue *e) {return (e->elevator_type->elevator_features) &ELV_IOSCHED_NEED_FQ ;
@@ -258,6 +263,12 @@ static inline int elv_iosched_single_ioq(struct elevator_queue *e) return (e->elevator_type->elevator_features) & ELV_IOSCHED_SINGLE_IOQ; } +/* returns 1 if elevator layer should enable its idling logic, 0 otherwise */ +static inline int elv_gen_idling_enabled (struct elevator_queue *e) +{ +return !((e->elevator_type->elevator_features) &ELV_IOSCHED_DONT_IDLE ); +} +#else /* ELV_IOSCHED_FAIR_QUEUING */ static inline int elv_iosched_fair_queuing_enabled (struct elevator_queue *e)
@@ -270,6 +281,11 @@ static inline int elv_iosched_single_ioq(struct elevator_queue *e) return 0; } +static inline int elv_gen_idling_enabled (struct elevator_queue *e) +{ +return 0 ; +} +#endif /* ELV_IOSCHED_FAIR_QUEUING */ extern void *elv_get_sched_queue (struct request_queue *q,struct request *rq);extern void *elv_select_sched_queue (struct request_queue *q,int force);
Signature · 1 line
1.6.0.1
A message that is not in the archive (sent off-list or not yet received): 1236823015-4183-11-git-send-email-vgoyal-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org
PATCH 1/1 IO Controller: No need to stop idling in as
Review
Quoted 8 lines · Vivek Goyal
(quoted diff) +4
-1.elevator_alloc_sched_queue_fn = as_alloc_as_queue, .elevator_free_sched_queue_fn = as_free_as_queue, +#ifdef CONFIG_IOSCHED_AS_HIER + .elevator_expire_ioq_fn = as_expire_ioq, + .elevator_active_ioq_set_fn = as_active_ioq_set, }, - + .elevator_features = ELV_IOSCHED_NEED_FQ | ELV_IOSCHED_SINGLE_IOQ | ELV_IOSCHED_DONT_IDLE,
Hi Vivek,
I found the IO Controller doesn't work in as.
I dig into this issue, and notice that you stop idling in as. IMHO, this will cause active ioq is always expired when tring to choose a new ioq to serve(elv_fq_select_ioq). Because idling is disabled, active ioq can't be kept anymore.
So i just get rid of ELV_IOSCHED_DONT_IDLE, and it works fine this time.
--- block/as-iosched.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
block/as-iosched.c +1 -1
@@ -1689,7 +1689,7 @@ static struct elevator_type iosched_as = { .elevator_expire_ioq_fn = as_expire_ioq, .elevator_active_ioq_set_fn = as_active_ioq_set, }, - .elevator_features = ELV_IOSCHED_NEED_FQ | ELV_IOSCHED_SINGLE_IOQ | ELV_IOSCHED_DONT_IDLE, + .elevator_features = ELV_IOSCHED_NEED_FQ | ELV_IOSCHED_SINGLE_IOQ, #else },#endif
Signature · 1 line
1.5.4.rc3
PATCH 1/1 IO Controller: No need to stop idling in as
In reply to Vivek GoyalReview
Quoted 8 lines · Vivek Goyal
(quoted diff) +4
-1.elevator_alloc_sched_queue_fn = as_alloc_as_queue, .elevator_free_sched_queue_fn = as_free_as_queue, +#ifdef CONFIG_IOSCHED_AS_HIER + .elevator_expire_ioq_fn = as_expire_ioq, + .elevator_active_ioq_set_fn = as_active_ioq_set, }, - + .elevator_features = ELV_IOSCHED_NEED_FQ | ELV_IOSCHED_SINGLE_IOQ | ELV_IOSCHED_DONT_IDLE,
Hi Vivek,
I found the IO Controller doesn't work in as.
I dig into this issue, and notice that you stop idling in as. IMHO, this will cause active ioq is always expired when tring to choose a new ioq to serve(elv_fq_select_ioq). Because idling is disabled, active ioq can't be kept anymore.
So i just get rid of ELV_IOSCHED_DONT_IDLE, and it works fine this time.
--- block/as-iosched.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
block/as-iosched.c +1 -1
@@ -1689,7 +1689,7 @@ static struct elevator_type iosched_as = { .elevator_expire_ioq_fn = as_expire_ioq, .elevator_active_ioq_set_fn = as_active_ioq_set, }, - .elevator_features = ELV_IOSCHED_NEED_FQ | ELV_IOSCHED_SINGLE_IOQ | ELV_IOSCHED_DONT_IDLE, + .elevator_features = ELV_IOSCHED_NEED_FQ | ELV_IOSCHED_SINGLE_IOQ, #else },#endif
Signature · 1 line
1.5.4.rc3
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH] IO Controller: No need to stop idling in as
Quoted 13 lines · Gui Jianfeng
Review
Quoted 8 lines · Vivek Goyal
(quoted diff) +4
-1.elevator_alloc_sched_queue_fn = as_alloc_as_queue, .elevator_free_sched_queue_fn = as_free_as_queue, +#ifdef CONFIG_IOSCHED_AS_HIER + .elevator_expire_ioq_fn = as_expire_ioq, + .elevator_active_ioq_set_fn = as_active_ioq_set, }, - + .elevator_features = ELV_IOSCHED_NEED_FQ | ELV_IOSCHED_SINGLE_IOQ | ELV_IOSCHED_DONT_IDLE,Hi Vivek,
Quoted 6 lines
I found the IO Controller doesn't work in as.
I dig into this issue, and notice that you stop idling in as. IMHO, this will cause active ioq is always expired when tring to choose a new ioq toserve(elv_fq_select_ioq). Because idling is disabled, active ioq can't be kept anymore.
So i just get rid ofELV_IOSCHED_DONT_IDLE, and it works fine this time.
Hi Gui,
Thanks for the testing. I have not enabled idling for AS in common layer becuase AS has its own idling/anticipation logic. I think we should not have anticipation going on at two places, common layer as well as individual io scheduler. That's the reason I have implemented a function elv_iosched_expire_ioq(), which calls io scheduler to find out if an ioq can be expired now.
So in elv_fq_select_ioq(), we call elv_iosched_expire_ioq() and if ioschduler denies expiration, then we don't expire the queue. In this case AS can deny the expiration if it is anticipating for next request.
Actually AS is very different and its little tricky to make it work with common layer, especially in terms of anticipation. This is on my TODO list but before I fix AS, I wanted to get other things right with common layer, cfq, noop and deadline.
Thanks
Vivek
Quoted 21 lines
Signed-off-by Gui Jianfeng <[email protected]>--- block/as-iosched.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)block/as-iosched.c +1
-1@@ -1689,7 +1689,7 @@ static struct elevator_type iosched_as = { .elevator_expire_ioq_fn = as_expire_ioq, .elevator_active_ioq_set_fn = as_active_ioq_set, }, - .elevator_features = ELV_IOSCHED_NEED_FQ | ELV_IOSCHED_SINGLE_IOQ | ELV_IOSCHED_DONT_IDLE, + .elevator_features = ELV_IOSCHED_NEED_FQ | ELV_IOSCHED_SINGLE_IOQ, #else },#endif Signature · 1 line
1.5.4.rc3
Re: [PATCH] IO Controller: No need to stop idling in as
In reply to Gui JianfengQuoted 13 lines · Gui Jianfeng
Review
Quoted 8 lines · Vivek Goyal
(quoted diff) +4
-1.elevator_alloc_sched_queue_fn = as_alloc_as_queue, .elevator_free_sched_queue_fn = as_free_as_queue, +#ifdef CONFIG_IOSCHED_AS_HIER + .elevator_expire_ioq_fn = as_expire_ioq, + .elevator_active_ioq_set_fn = as_active_ioq_set, }, - + .elevator_features = ELV_IOSCHED_NEED_FQ | ELV_IOSCHED_SINGLE_IOQ | ELV_IOSCHED_DONT_IDLE,Hi Vivek,
Quoted 6 lines
I found the IO Controller doesn't work in as.
I dig into this issue, and notice that you stop idling in as. IMHO, this will cause active ioq is always expired when tring to choose a new ioq toserve(elv_fq_select_ioq). Because idling is disabled, active ioq can't be kept anymore.
So i just get rid ofELV_IOSCHED_DONT_IDLE, and it works fine this time.
Hi Gui,
Thanks for the testing. I have not enabled idling for AS in common layer becuase AS has its own idling/anticipation logic. I think we should not have anticipation going on at two places, common layer as well as individual io scheduler. That's the reason I have implemented a function elv_iosched_expire_ioq(), which calls io scheduler to find out if an ioq can be expired now.
So in elv_fq_select_ioq(), we call elv_iosched_expire_ioq() and if ioschduler denies expiration, then we don't expire the queue. In this case AS can deny the expiration if it is anticipating for next request.
Actually AS is very different and its little tricky to make it work with common layer, especially in terms of anticipation. This is on my TODO list but before I fix AS, I wanted to get other things right with common layer, cfq, noop and deadline.
Thanks
Vivek
Quoted 21 lines
Signed-off-by Gui Jianfeng <[email protected]>--- block/as-iosched.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)block/as-iosched.c +1
-1@@ -1689,7 +1689,7 @@ static struct elevator_type iosched_as = { .elevator_expire_ioq_fn = as_expire_ioq, .elevator_active_ioq_set_fn = as_active_ioq_set, }, - .elevator_features = ELV_IOSCHED_NEED_FQ | ELV_IOSCHED_SINGLE_IOQ | ELV_IOSCHED_DONT_IDLE, + .elevator_features = ELV_IOSCHED_NEED_FQ | ELV_IOSCHED_SINGLE_IOQ, #else },#endif Signature · 1 line
1.5.4.rc3
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH] IO Controller: No need to stop idling in as
Quoted 20 lines · Vivek Goyal
Quoted 12 lines · Gui Jianfeng
Review
Quoted 8 lines · Vivek Goyal
(quoted diff) +4
-1.elevator_alloc_sched_queue_fn = as_alloc_as_queue, .elevator_free_sched_queue_fn = as_free_as_queue, +#ifdef CONFIG_IOSCHED_AS_HIER + .elevator_expire_ioq_fn = as_expire_ioq, + .elevator_active_ioq_set_fn = as_active_ioq_set, }, - + .elevator_features = ELV_IOSCHED_NEED_FQ | ELV_IOSCHED_SINGLE_IOQ | ELV_IOSCHED_DONT_IDLE,Hi Vivek,
Quoted 6 lines
I found the IO Controller doesn't work in as.
I dig into this issue, and notice that you stop idling in as. IMHO, this will cause active ioq is always expired when tring to choose a new ioq toserve(elv_fq_select_ioq). Because idling is disabled, active ioq can't be kept anymore.
So i just get rid ofELV_IOSCHED_DONT_IDLE, and it works fine this time.
Quoted 8 lines
Hi Gui,
Thanks for the testing. I have not enabled idling for AS in common layer becuase AS has its own idling/anticipation logic. I think we should not have anticipation going on at two places, common layer as well as individual io scheduler. That's the reason I have implemented a function
elv_iosched_expire_ioq(), which calls io scheduler to find out if an ioq can be expired now.
Hi Vivek,
If an user chooses fairness rather than throughput just like what your fairness patch is tring to do. Do we need to enable the common idling logic for as in this scenario.
Quoted 38 lines
So in
elv_fq_select_ioq(), we callelv_iosched_expire_ioq()and if ioschduler denies expiration, then we don't expire the queue. In this case AS can deny the expiration if it is anticipating for next request.Actually AS is very different and its little tricky to make it work with common layer, especially in terms of anticipation. This is on my TODO list but before I fix AS, I wanted to get other things right with common layer, cfq, noop and deadline.
Thanks
VivekQuoted 21 lines
Signed-off-by Gui Jianfeng <[email protected]>--- block/as-iosched.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)block/as-iosched.c +1
-1@@ -1689,7 +1689,7 @@ static struct elevator_type iosched_as = { .elevator_expire_ioq_fn = as_expire_ioq, .elevator_active_ioq_set_fn = as_active_ioq_set, }, - .elevator_features = ELV_IOSCHED_NEED_FQ | ELV_IOSCHED_SINGLE_IOQ | ELV_IOSCHED_DONT_IDLE, + .elevator_features = ELV_IOSCHED_NEED_FQ | ELV_IOSCHED_SINGLE_IOQ, #else },#endif Signature · 1 line
1.5.4.rc3
Signature · 2 lines
Regards Gui Jianfeng
Re: [PATCH] IO Controller: No need to stop idling in as
In reply to Vivek GoyalQuoted 20 lines · Vivek Goyal
Quoted 12 lines · Gui Jianfeng
Review
Quoted 8 lines · Vivek Goyal
(quoted diff) +4
-1.elevator_alloc_sched_queue_fn = as_alloc_as_queue, .elevator_free_sched_queue_fn = as_free_as_queue, +#ifdef CONFIG_IOSCHED_AS_HIER + .elevator_expire_ioq_fn = as_expire_ioq, + .elevator_active_ioq_set_fn = as_active_ioq_set, }, - + .elevator_features = ELV_IOSCHED_NEED_FQ | ELV_IOSCHED_SINGLE_IOQ | ELV_IOSCHED_DONT_IDLE,Hi Vivek,
Quoted 6 lines
I found the IO Controller doesn't work in as.
I dig into this issue, and notice that you stop idling in as. IMHO, this will cause active ioq is always expired when tring to choose a new ioq toserve(elv_fq_select_ioq). Because idling is disabled, active ioq can't be kept anymore.
So i just get rid ofELV_IOSCHED_DONT_IDLE, and it works fine this time.
Quoted 8 lines
Hi Gui,
Thanks for the testing. I have not enabled idling for AS in common layer becuase AS has its own idling/anticipation logic. I think we should not have anticipation going on at two places, common layer as well as individual io scheduler. That's the reason I have implemented a function
elv_iosched_expire_ioq(), which calls io scheduler to find out if an ioq can be expired now.
Hi Vivek,
If an user chooses fairness rather than throughput just like what your fairness patch is tring to do. Do we need to enable the common idling logic for as in this scenario.
Quoted 38 lines
So in
elv_fq_select_ioq(), we callelv_iosched_expire_ioq()and if ioschduler denies expiration, then we don't expire the queue. In this case AS can deny the expiration if it is anticipating for next request.Actually AS is very different and its little tricky to make it work with common layer, especially in terms of anticipation. This is on my TODO list but before I fix AS, I wanted to get other things right with common layer, cfq, noop and deadline.
Thanks
VivekQuoted 21 lines
Signed-off-by Gui Jianfeng <[email protected]>--- block/as-iosched.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)block/as-iosched.c +1
-1@@ -1689,7 +1689,7 @@ static struct elevator_type iosched_as = { .elevator_expire_ioq_fn = as_expire_ioq, .elevator_active_ioq_set_fn = as_active_ioq_set, }, - .elevator_features = ELV_IOSCHED_NEED_FQ | ELV_IOSCHED_SINGLE_IOQ | ELV_IOSCHED_DONT_IDLE, + .elevator_features = ELV_IOSCHED_NEED_FQ | ELV_IOSCHED_SINGLE_IOQ, #else },#endif Signature · 1 line
1.5.4.rc3
Signature · 2 lines
Regards Gui Jianfeng
Hi Vivek,
Could you tell me to which kernel I can apply your patches?
# latest mm? I would like to test your controller.
Thank you,
Takuya Yoshikawa
Quoted 49 lines · Vivek Goyal
Hi All,
Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.
http://lkml.org/lkml/2008/11/6/227
One of the takeaway from the discussion in this thread was that let us implement a common layer which contains the proportional weight scheduling code which can be shared by all the IO schedulers.
Implementing IO controller will not cover the devices which don't use IO schedulers but it should cover the common case.
There were more discussions regarding 2 level vs 1 level IO control at following link.
https://lists.linux-foundation.org/pipermail/containers/2009-January/015402.html
So in the mean time we took the discussion off the list and spent time on making the 1 level control apporoach work where majority of the proportional weight control is shared by the four schedulers instead of each one having to replicate the code. We make use of BFQ code for fair queuing as posted by Paolo and Fabio here.
http://lkml.org/lkml/2008/11/11/148
Details about design and howto have been put in documentation patch.
I have done very basic testing of running 2 or 3 "dd" threads in different cgroups. Wanted to get the patchset out for feedback/review before we dive into more bug fixing, benchmarking, optimizations etc.
Your feedback/comments are welcome.
Patch series contains 10 patches. It should be compilable and bootable after every patch. Intial 2 patches implement flat fair queuing (no cgroup support) and make cfq to use that. Later patches introduce hierarchical fair queuing support in elevator layer and modify other IO schdulers to use that.
Thanks
Vivek
_______________________________________________
Containers mailing list
[email protected]
https://lists.linux-foundation.org/mailman/listinfo/containers
A message that is not in the archive (sent off-list or not yet received): [email protected]
Hi Vivek,
It would be very appreciated if the patches can be based on 2.6.28.
Thanks a lot.
Anqin
Quoted 66 lines · Takuya Yoshikawa
Hi Vivek,
Could you tell me to which kernel I can apply your patches?
# latest mm? I would like to test your controller.Thank you,
Takuya YoshikawaQuoted 42 lines · Vivek Goyal
Hi All,
Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.
http://lkml.org/lkml/2008/11/6/227
One of the takeaway from the discussion in this thread was that let us implement a common layer which contains the proportional weight scheduling code which can be shared by all the IO schedulers.
Implementing IO controller will not cover the devices which don't use IO schedulers but it should cover the common case.
There were more discussions regarding 2 level vs 1 level IO control at following link.
https://lists.linux-foundation.org/pipermail/containers/2009-January/015402.html
So in the mean time we took the discussion off the list and spent time on making the 1 level control apporoach work where majority of the proportional weight control is shared by the four schedulers instead of each one having to replicate the code. We make use of BFQ code for fair queuing as posted by Paolo and Fabio here.
http://lkml.org/lkml/2008/11/11/148
Details about design and howto have been put in documentation patch.
I have done very basic testing of running 2 or 3 "dd" threads in different cgroups. Wanted to get the patchset out for feedback/review before we dive into more bug fixing, benchmarking, optimizations etc.
Your feedback/comments are welcome.
Patch series contains 10 patches. It should be compilable and bootable after every patch. Intial 2 patches implement flat fair queuing (no cgroup support) and make cfq to use that. Later patches introduce hierarchical fair queuing support in elevator layer and modify other IO schdulers to use that.
Quoted 7 lines
Thanks
Vivek
_______________________________________________
Containers mailing list Containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
https://lists.linux-foundation.org/mailman/listinfo/containers_______________________________________________
Containers mailing list Containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
https://lists.linux-foundation.org/mailman/listinfo/containers
Hi Vivek,
It would be very appreciated if the patches can be based on 2.6.28.
Thanks a lot.
Anqin
Quoted 66 lines · Takuya Yoshikawa
Hi Vivek,
Could you tell me to which kernel I can apply your patches?
# latest mm? I would like to test your controller.Thank you,
Takuya YoshikawaQuoted 42 lines · Vivek Goyal
Hi All,
Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.
http://lkml.org/lkml/2008/11/6/227
One of the takeaway from the discussion in this thread was that let us implement a common layer which contains the proportional weight scheduling code which can be shared by all the IO schedulers.
Implementing IO controller will not cover the devices which don't use IO schedulers but it should cover the common case.
There were more discussions regarding 2 level vs 1 level IO control at following link.
https://lists.linux-foundation.org/pipermail/containers/2009-January/015402.html
So in the mean time we took the discussion off the list and spent time on making the 1 level control apporoach work where majority of the proportional weight control is shared by the four schedulers instead of each one having to replicate the code. We make use of BFQ code for fair queuing as posted by Paolo and Fabio here.
http://lkml.org/lkml/2008/11/11/148
Details about design and howto have been put in documentation patch.
I have done very basic testing of running 2 or 3 "dd" threads in different cgroups. Wanted to get the patchset out for feedback/review before we dive into more bug fixing, benchmarking, optimizations etc.
Your feedback/comments are welcome.
Patch series contains 10 patches. It should be compilable and bootable after every patch. Intial 2 patches implement flat fair queuing (no cgroup support) and make cfq to use that. Later patches introduce hierarchical fair queuing support in elevator layer and modify other IO schdulers to use that.
Quoted 7 lines
Thanks
Vivek
_______________________________________________
Containers mailing list
[email protected]
https://lists.linux-foundation.org/mailman/listinfo/containers_______________________________________________
Containers mailing list
[email protected]
https://lists.linux-foundation.org/mailman/listinfo/containers
A message that is not in the archive (sent off-list or not yet received): d95d44a20903112340s3c77807dt465e68901747ad89-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org
(Please don't top-post...)
Quoted 4 lines · anqin
Hi Vivek,
It would be very appreciated if the patches can be based on 2.6.28.
Why? When this is ready to be merged, then it should be based on Jens' block-tree, or akpm's mm tree. And this version currently is based on 2.6.29-rc4, so if you want to try it out, just prepare a 2.6.29-rc4 kernel tree.
Quoted 14 lines
Thanks a lot.
Anqin
Quoted 8 lines · Takuya Yoshikawa
Hi Vivek,
Could you tell me to which kernel I can apply your patches?
# latest mm? I would like to test your controller.Thank you,
Takuya Yoshikawa
(Please don't top-post...)
Quoted 4 lines · anqin
Hi Vivek,
It would be very appreciated if the patches can be based on 2.6.28.
Why? When this is ready to be merged, then it should be based on Jens' block-tree, or akpm's mm tree. And this version currently is based on 2.6.29-rc4, so if you want to try it out, just prepare a 2.6.29-rc4 kernel tree.
Quoted 14 lines
Thanks a lot.
Anqin
Quoted 8 lines · Takuya Yoshikawa
Hi Vivek,
Could you tell me to which kernel I can apply your patches?
# latest mm? I would like to test your controller.Thank you,
Takuya Yoshikawa
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 4 lines
Why? When this is ready to be merged, then it should be based on Jens' block-tree, or akpm's mm tree. And this version currently is based on 2.6.29-rc4, so if you want to try it out, just prepare a 2.6.29-rc4 kernel tree.
I have checked the LKML and see these patches (on web pages) are based on 2.6.27. It seemed too old.
You mean that the codes have new patch files in 2.6.29-rc4?
Quoted 4 lines
Why? When this is ready to be merged, then it should be based on Jens' block-tree, or akpm's mm tree. And this version currently is based on 2.6.29-rc4, so if you want to try it out, just prepare a 2.6.29-rc4 kernel tree.
I have checked the LKML and see these patches (on web pages) are based on 2.6.27. It seemed too old.
You mean that the codes have new patch files in 2.6.29-rc4?
Quoted 9 lines · anqin
Quoted 4 lines
Why? When this is ready to be merged, then it should be based on Jens' block-tree, or akpm's mm tree. And this version currently is based on 2.6.29-rc4, so if you want to try it out, just prepare a 2.6.29-rc4 kernel tree.
I have checked the LKML and see these patches (on web pages) are based on 2.6.27. It seemed too old.
You mean that the codes have new patch files in 2.6.29-rc4?
I guess my mistake that I did not specify the kernel version in my mail.
These patches should apply on top of 2.6.29-rc7.
Thanks
Vivek
Quoted 4 lines · anqin
Hi Vivek,
It would be very appreciated if the patches can be based on 2.6.28.
Hi Anquin,
I think most of the people want to test new patches on latest kernels so I will keep it that way. You can backport it to previous kernels if you really need to. For me it will become very difficult to maintain two versions.
Is there any reason why you can't move to latest kernels?
Thanks
Vivek
Quoted 72 lines
Thanks a lot.
Anqin
Quoted 66 lines · Takuya Yoshikawa
Hi Vivek,
Could you tell me to which kernel I can apply your patches?
# latest mm? I would like to test your controller.Thank you,
Takuya YoshikawaQuoted 42 lines · Vivek Goyal
Hi All,
Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.
http://lkml.org/lkml/2008/11/6/227
One of the takeaway from the discussion in this thread was that let us implement a common layer which contains the proportional weight scheduling code which can be shared by all the IO schedulers.
Implementing IO controller will not cover the devices which don't use IO schedulers but it should cover the common case.
There were more discussions regarding 2 level vs 1 level IO control at following link.
https://lists.linux-foundation.org/pipermail/containers/2009-January/015402.html
So in the mean time we took the discussion off the list and spent time on making the 1 level control apporoach work where majority of the proportional weight control is shared by the four schedulers instead of each one having to replicate the code. We make use of BFQ code for fair queuing as posted by Paolo and Fabio here.
http://lkml.org/lkml/2008/11/11/148
Details about design and howto have been put in documentation patch.
I have done very basic testing of running 2 or 3 "dd" threads in different cgroups. Wanted to get the patchset out for feedback/review before we dive into more bug fixing, benchmarking, optimizations etc.
Your feedback/comments are welcome.
Patch series contains 10 patches. It should be compilable and bootable after every patch. Intial 2 patches implement flat fair queuing (no cgroup support) and make cfq to use that. Later patches introduce hierarchical fair queuing support in elevator layer and modify other IO schdulers to use that.
Quoted 7 lines
Thanks
Vivek
_______________________________________________
Containers mailing list Containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
https://lists.linux-foundation.org/mailman/listinfo/containers_______________________________________________
Containers mailing list Containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
https://lists.linux-foundation.org/mailman/listinfo/containers
Quoted 6 lines · Takuya Yoshikawa
Hi Vivek,
Could you tell me to which kernel I can apply your patches?
# latest mm? I would like to test your controller.
Hi Takuya,
These apply on linus git tree (2.6.29-rc7).
Thanks
Vivek
Quoted 54 lines
Thank you,
Takuya YoshikawaQuoted 49 lines · Vivek Goyal
Hi All,
Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.
http://lkml.org/lkml/2008/11/6/227
One of the takeaway from the discussion in this thread was that let us implement a common layer which contains the proportional weight scheduling code which can be shared by all the IO schedulers.
Implementing IO controller will not cover the devices which don't use IO schedulers but it should cover the common case.
There were more discussions regarding 2 level vs 1 level IO control at following link.
https://lists.linux-foundation.org/pipermail/containers/2009-January/015402.html
So in the mean time we took the discussion off the list and spent time on making the 1 level control apporoach work where majority of the proportional weight control is shared by the four schedulers instead of each one having to replicate the code. We make use of BFQ code for fair queuing as posted by Paolo and Fabio here.
http://lkml.org/lkml/2008/11/11/148
Details about design and howto have been put in documentation patch.
I have done very basic testing of running 2 or 3 "dd" threads in different cgroups. Wanted to get the patchset out for feedback/review before we dive into more bug fixing, benchmarking, optimizations etc.
Your feedback/comments are welcome.
Patch series contains 10 patches. It should be compilable and bootable after every patch. Intial 2 patches implement flat fair queuing (no cgroup support) and make cfq to use that. Later patches introduce hierarchical fair queuing support in elevator layer and modify other IO schdulers to use that.
Thanks
Vivek
_______________________________________________
Containers mailing list Containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
https://lists.linux-foundation.org/mailman/listinfo/containers
Quoted 11 lines · Vivek Goyal
Hi All,
Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.
http://lkml.org/lkml/2008/11/6/227
One of the takeaway from the discussion in this thread was that let us implement a common layer which contains the proportional weight scheduling code which can be shared by all the IO schedulers.
Hi Vivek,
I did some tests on my *old* i386 box(with two concurrent dd running), and notice that IO Controller doesn't work fine in such situation. But it can work perfectly in my *new* x86 box. I dig into this problem, and i guess the major reason is that my *old* i386 box is too slow, it can't ensure two running ioqs are always backlogged. If that is the case, I happens to have a thought. when an ioq uses up it time slice, we don't expire it immediately. May be we can give a piece of bonus time for idling to wait new requests if this ioq's finish time and its ancestor's finish time are all much smaller than other entities in each corresponding service tree.
Signature · 2 lines
Regards Gui Jianfeng
Quoted 13 lines · Gui Jianfeng
Quoted 11 lines · Vivek Goyal
Hi All,
Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.
http://lkml.org/lkml/2008/11/6/227
One of the takeaway from the discussion in this thread was that let us implement a common layer which contains the proportional weight scheduling code which can be shared by all the IO schedulers.
Quoted 6 lines
Hi Vivek,
I did some tests on my *old* i386 box(with two concurrent dd running), and notice that IO Controller doesn't work fine in such situation. But it can work perfectly in my *new* x86 box. I dig into this problem, and i guess the major reason is that my *old* i386 box is too slow, it can't ensure two running ioqs are always backlogged.
Hi Gui,
Have you run top to see what's the percentage cpu usage. I suspect that cpu is not keeping up pace disk to enqueue enough requests. I think process might be blocked somewhere else so that it could not issue requests.
Quoted 4 lines
If that is the case, I happens to have a thought. when an ioq uses up it time slice, we don't expire it immediately. May be we can give a piece of bonus time for idling to wait new requests if this ioq's finish time and its ancestor's finish time are all much smaller than other entities in each corresponding service tree.
Have you tried it with "fairness" enabled? With "fairness" enabled, for sync queues I am waiting for one extra idle time slice "8ms" for queue to get backlogged again before I move to the next queue?
Otherwise try to increase the idle time length to higher value say "12ms" just to see if that has any impact.
Can you please also send me output of blkparse. It might give some idea how IO schedulers see IO pattern.
Thanks
Vivek
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 37 lines · Vivek Goyal
Quoted 13 lines · Gui Jianfeng
Quoted 11 lines · Vivek Goyal
Hi All,
Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.
http://lkml.org/lkml/2008/11/6/227
One of the takeaway from the discussion in this thread was that let us implement a common layer which contains the proportional weight scheduling code which can be shared by all the IO schedulers.
Quoted 6 lines
Hi Vivek,
I did some tests on my *old* i386 box(with two concurrent dd running), and notice that IO Controller doesn't work fine in such situation. But it can work perfectly in my *new* x86 box. I dig into this problem, and i guess the major reason is that my *old* i386 box is too slow, it can't ensure two running ioqs are always backlogged.
Hi Gui,
Have you run top to see what's the percentage cpu usage. I suspect that cpu is not keeping up pace disk to enqueue enough requests. I think process might be blocked somewhere else so that it could not issue requests.
Quoted 4 lines
If that is the case, I happens to have a thought. when an ioq uses up it time slice, we don't expire it immediately. May be we can give a piece of bonus time for idling to wait new requests if this ioq's finish time and its ancestor's finish time are all much smaller than other entities in each corresponding service tree.
Have you tried it with "fairness" enabled? With "fairness" enabled, for sync queues I am waiting for one extra idle time slice "8ms" for queue to get backlogged again before I move to the next queue?
Quoted 5 lines
Otherwise try to increase the idle time length to higher value say "12ms" just to see if that has any impact.
Can you please also send me output of blkparse. It might give some idea how IO schedulers see IO pattern.
Hi Vivek,
Sorry for the late reply, I tried the "fairness" patch, it seems not working. I'v also tried to extend the idle value, not working either.
The blktrace output is attached. It seems that the high priority ioq is deleting from busy tree too often due to lacking of requests. My box is single CPU and CPU speed is a little slow. May be two concurrent dd is contending CPU to submit requests, that's the reason for not always backlogged for ioqs.
Quoted 6 lines
Thanks
Vivek
Signature · 2 lines
Regards Gui Jianfeng
_______________________________________________
Containers mailing list Containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
https://lists.linux-foundation.org/mailman/listinfo/containers
Quoted 37 lines · Vivek Goyal
Quoted 13 lines · Gui Jianfeng
Quoted 11 lines · Vivek Goyal
Hi All,
Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.
http://lkml.org/lkml/2008/11/6/227
One of the takeaway from the discussion in this thread was that let us implement a common layer which contains the proportional weight scheduling code which can be shared by all the IO schedulers.
Quoted 6 lines
Hi Vivek,
I did some tests on my *old* i386 box(with two concurrent dd running), and notice that IO Controller doesn't work fine in such situation. But it can work perfectly in my *new* x86 box. I dig into this problem, and i guess the major reason is that my *old* i386 box is too slow, it can't ensure two running ioqs are always backlogged.
Hi Gui,
Have you run top to see what's the percentage cpu usage. I suspect that cpu is not keeping up pace disk to enqueue enough requests. I think process might be blocked somewhere else so that it could not issue requests.
Quoted 4 lines
If that is the case, I happens to have a thought. when an ioq uses up it time slice, we don't expire it immediately. May be we can give a piece of bonus time for idling to wait new requests if this ioq's finish time and its ancestor's finish time are all much smaller than other entities in each corresponding service tree.
Have you tried it with "fairness" enabled? With "fairness" enabled, for sync queues I am waiting for one extra idle time slice "8ms" for queue to get backlogged again before I move to the next queue?
Quoted 5 lines
Otherwise try to increase the idle time length to higher value say "12ms" just to see if that has any impact.
Can you please also send me output of blkparse. It might give some idea how IO schedulers see IO pattern.
Hi Vivek,
Sorry for the late reply, I tried the "fairness" patch, it seems not working. I'v also tried to extend the idle value, not working either.
The blktrace output is attached. It seems that the high priority ioq is deleting from busy tree too often due to lacking of requests. My box is single CPU and CPU speed is a little slow. May be two concurrent dd is contending CPU to submit requests, that's the reason for not always backlogged for ioqs.
Quoted 6 lines
Thanks
Vivek
Signature · 2 lines
Regards Gui Jianfeng
Quoted 42 lines · Gui Jianfeng
Quoted 35 lines · Vivek Goyal
Quoted 13 lines · Gui Jianfeng
Quoted 11 lines · Vivek Goyal
Hi All,
Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.
http://lkml.org/lkml/2008/11/6/227
One of the takeaway from the discussion in this thread was that let us implement a common layer which contains the proportional weight scheduling code which can be shared by all the IO schedulers.
Quoted 6 lines
Hi Vivek,
I did some tests on my *old* i386 box(with two concurrent dd running), and notice that IO Controller doesn't work fine in such situation. But it can work perfectly in my *new* x86 box. I dig into this problem, and i guess the major reason is that my *old* i386 box is too slow, it can't ensure two running ioqs are always backlogged.
Hi Gui,
Have you run top to see what's the percentage cpu usage. I suspect that cpu is not keeping up pace disk to enqueue enough requests. I think process might be blocked somewhere else so that it could not issue requests.
Quoted 4 lines
If that is the case, I happens to have a thought. when an ioq uses up it time slice, we don't expire it immediately. May be we can give a piece of bonus time for idling to wait new requests if this ioq's finish time and its ancestor's finish time are all much smaller than other entities in each corresponding service tree.
Have you tried it with "fairness" enabled? With "fairness" enabled, for sync queues I am waiting for one extra idle time slice "8ms" for queue to get backlogged again before I move to the next queue?
Quoted 5 lines
Otherwise try to increase the idle time length to higher value say "12ms" just to see if that has any impact.
Can you please also send me output of blkparse. It might give some idea how IO schedulers see IO pattern.
Quoted 8 lines
Hi Vivek,
Sorry for the late reply, I tried the "fairness" patch, it seems not working. I'v also tried to extend the idle value, not working either.
The blktrace output is attached. It seems that the high priority ioq is deleting from busy tree too often due to lacking of requests. My box is single CPU and CPU speed is a little slow. May be two concurrent dd is contending CPU to submit requests, that's the reason for not always backlogged for ioqs.
Hi Vivek,
Sorry for bothering, there were some configure errors when i tested, and got the improper result.
The "fairness" patch seems to work fine now! It makes the high priority ioq *always* backlogged :)
Quoted 7 lines
Quoted 6 lines
Thanks
Vivek
Signature · 2 lines
Regards Gui Jianfeng
Quoted 4 lines · Vivek Goyal
Hi All,
Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.
Hi Vivek,
I got the following OOPS when testing, can't reproduce again :(
kernel BUG at block/elevator-fq.c:1396!
invalid opcode: 0000 [#1] SMP
last sysfs file: /sys/block/hdb/queue/scheduler
Modules linked in: ipv6 cpufreq_ondemand acpi_cpufreq dm_mirror dm_multipath sbd
Pid: 5032, comm: rmdir Not tainted (2.6.29-rc7-vivek #17) Veriton M460
EIP: 0060:[<c04ec7de>] EFLAGS: 00010082 CPU: 0
EIP is at iocg_destroy+0xdc/0x14e
EAX: 00000000 EBX: f62278b4 ECX: f6207800 EDX: f6227904
ESI: f6227800 EDI: f62278a0 EBP: 00000003 ESP: c8790f00
DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
Process rmdir (pid: 5032, ti=c8790000 task=f6636960 task.ti=c8790000)
Stack: f53cc5c0 f62b7258 f10991c8 00000282 f6227800 c0733fa0 ec1c5140 edfc6d34 c8790000 c04463ce f6a4c84c edfc6d34 0804c840 c048883c f6a4c84c f6a4c84c c04888d6 f6a4c84c 00000000 c04897de f6a4c84c c048504c f115f4c0 ebc37954 Call Trace: [<c04463ce>] cgroup_diput+0x41/0x8c [<c048883c>] dentry_iput+0x45/0x5e [<c04888d6>] d_kill+0x19/0x32 [<c04897de>] dput+0xd8/0xdf [<c048504c>] do_rmdir+0x8f/0xb6 [<c06330fc>] do_page_fault+0x2a2/0x579 [<c0402fc1>] sysenter_do_call+0x12/0x21 [<c0630000>] schedule+0x641/0x830 Code: 08 00 74 04 0f 0b eb fe 83 7f 04 00 74 04 0f 0b eb fe 45 83 c3 1c 83 fd 0 EIP: [<c04ec7de>] iocg_destroy+0xdc/0x14e SS:ESP 0068:c8790f00
Signature · 2 lines
Regards Gui Jianfeng
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 21 lines · Gui Jianfeng
Quoted 4 lines · Vivek Goyal
Hi All,
Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.
Hi Vivek,
I got the following OOPS when testing, can't reproduce again :(
kernel BUG at
block/elevator-fq.c:1396!
invalid opcode: 0000 [#1] SMP
last sysfs file:/sys/block/hdb/queue/schedulerModules linked in: ipv6 cpufreq_ondemand acpi_cpufreq dm_mirror dm_multipath sbdPid: 5032, comm: rmdir Not tainted (2.6.29-rc7-vivek #17) Veriton M460
EIP: 0060:[<c04ec7de>] EFLAGS: 00010082 CPU: 0
EIP is atiocg_destroy+0xdc/0x14e
EAX: 00000000 EBX:f62278b4ECX:f6207800EDX:f6227904
ESI:f6227800EDI:f62278a0EBP: 00000003 ESP:c8790f00
DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
Process rmdir (pid: 5032, ti=c8790000 task=f6636960 task.ti=c8790000)
Quoted 15 lines
Stack: f53cc5c0 f62b7258 f10991c8 00000282 f6227800 c0733fa0 ec1c5140 edfc6d34 c8790000 c04463ce f6a4c84c edfc6d34 0804c840 c048883c f6a4c84c f6a4c84c c04888d6 f6a4c84c 00000000 c04897de f6a4c84c c048504c f115f4c0 ebc37954 Call Trace: [<c04463ce>] cgroup_diput+0x41/0x8c [<c048883c>] dentry_iput+0x45/0x5e [<c04888d6>] d_kill+0x19/0x32 [<c04897de>] dput+0xd8/0xdf [<c048504c>] do_rmdir+0x8f/0xb6 [<c06330fc>] do_page_fault+0x2a2/0x579 [<c0402fc1>] sysenter_do_call+0x12/0x21 [<c0630000>] schedule+0x641/0x830 Code: 08 00 74 04 0f 0b eb fe 83 7f 04 00 74 04 0f 0b eb fe 45 83 c3 1c 83 fd 0 EIP: [<c04ec7de>] iocg_destroy+0xdc/0x14e SS:ESP 0068:c8790f00
We have seen this too. And have been able to reproduce it. I did not get a chance to fix it so far, but my understanding is that one of the async queues was active when the cgroup was getting destroyed. We moved it to root cgroup, but did not deactivate it; so active_entity still points to the entity of the async queue which has now been moved to the root cgroup. I will send an update if I can verify this, or fix it.
Quoted 6 lines
--
Regards
Gui Jianfeng
Quoted 10 lines · Gui Jianfeng
Quoted 4 lines · Vivek Goyal
Hi All,
Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.
Hi Vivek,
I got the following OOPS when testing, can't reproduce again :(
Hi Gui,
Thanks for the report. Will look into it and see if I can reproduce it.
Thanks
Vivek
Quoted 30 lines
kernel BUG at
block/elevator-fq.c:1396!
invalid opcode: 0000 [#1] SMP
last sysfs file:/sys/block/hdb/queue/schedulerModules linked in: ipv6 cpufreq_ondemand acpi_cpufreq dm_mirror dm_multipath sbdPid: 5032, comm: rmdir Not tainted (2.6.29-rc7-vivek #17) Veriton M460
EIP: 0060:[<c04ec7de>] EFLAGS: 00010082 CPU: 0
EIP is atiocg_destroy+0xdc/0x14e
EAX: 00000000 EBX:f62278b4ECX:f6207800EDX:f6227904
ESI:f6227800EDI:f62278a0EBP: 00000003 ESP:c8790f00
DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
Process rmdir (pid: 5032, ti=c8790000 task=f6636960 task.ti=c8790000)Stack: f53cc5c0 f62b7258 f10991c8 00000282 f6227800 c0733fa0 ec1c5140 edfc6d34 c8790000 c04463ce f6a4c84c edfc6d34 0804c840 c048883c f6a4c84c f6a4c84c c04888d6 f6a4c84c 00000000 c04897de f6a4c84c c048504c f115f4c0 ebc37954 Call Trace: [<c04463ce>] cgroup_diput+0x41/0x8c [<c048883c>] dentry_iput+0x45/0x5e [<c04888d6>] d_kill+0x19/0x32 [<c04897de>] dput+0xd8/0xdf [<c048504c>] do_rmdir+0x8f/0xb6 [<c06330fc>] do_page_fault+0x2a2/0x579 [<c0402fc1>] sysenter_do_call+0x12/0x21 [<c0630000>] schedule+0x641/0x830 Code: 08 00 74 04 0f 0b eb fe 83 7f 04 00 74 04 0f 0b eb fe 45 83 c3 1c 83 fd 0 EIP: [<c04ec7de>] iocg_destroy+0xdc/0x14e SS:ESP 0068:c8790f00Signature · 2 lines
Regards Gui Jianfeng
Quoted 21 lines · Gui Jianfeng
Quoted 4 lines · Vivek Goyal
Hi All,
Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.
Hi Vivek,
I got the following OOPS when testing, can't reproduce again :(
kernel BUG at
block/elevator-fq.c:1396!
invalid opcode: 0000 [#1] SMP
last sysfs file:/sys/block/hdb/queue/schedulerModules linked in: ipv6 cpufreq_ondemand acpi_cpufreq dm_mirror dm_multipath sbdPid: 5032, comm: rmdir Not tainted (2.6.29-rc7-vivek #17) Veriton M460
EIP: 0060:[<c04ec7de>] EFLAGS: 00010082 CPU: 0
EIP is atiocg_destroy+0xdc/0x14e
EAX: 00000000 EBX:f62278b4ECX:f6207800EDX:f6227904
ESI:f6227800EDI:f62278a0EBP: 00000003 ESP:c8790f00
DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
Process rmdir (pid: 5032, ti=c8790000 task=f6636960 task.ti=c8790000)
Quoted 15 lines
Stack: f53cc5c0 f62b7258 f10991c8 00000282 f6227800 c0733fa0 ec1c5140 edfc6d34 c8790000 c04463ce f6a4c84c edfc6d34 0804c840 c048883c f6a4c84c f6a4c84c c04888d6 f6a4c84c 00000000 c04897de f6a4c84c c048504c f115f4c0 ebc37954 Call Trace: [<c04463ce>] cgroup_diput+0x41/0x8c [<c048883c>] dentry_iput+0x45/0x5e [<c04888d6>] d_kill+0x19/0x32 [<c04897de>] dput+0xd8/0xdf [<c048504c>] do_rmdir+0x8f/0xb6 [<c06330fc>] do_page_fault+0x2a2/0x579 [<c0402fc1>] sysenter_do_call+0x12/0x21 [<c0630000>] schedule+0x641/0x830 Code: 08 00 74 04 0f 0b eb fe 83 7f 04 00 74 04 0f 0b eb fe 45 83 c3 1c 83 fd 0 EIP: [<c04ec7de>] iocg_destroy+0xdc/0x14e SS:ESP 0068:c8790f00
We have seen this too. And have been able to reproduce it. I did not get a chance to fix it so far, but my understanding is that one of the async queues was active when the cgroup was getting destroyed. We moved it to root cgroup, but did not deactivate it; so active_entity still points to the entity of the async queue which has now been moved to the root cgroup. I will send an update if I can verify this, or fix it.
Quoted 6 lines
--
Regards
Gui Jianfeng
Quoted 10 lines · Gui Jianfeng
Quoted 4 lines · Vivek Goyal
Hi All,
Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.
Hi Vivek,
I got the following OOPS when testing, can't reproduce again :(
Hi Gui,
Thanks for the report. Will look into it and see if I can reproduce it.
Thanks
Vivek
Quoted 30 lines
kernel BUG at
block/elevator-fq.c:1396!
invalid opcode: 0000 [#1] SMP
last sysfs file:/sys/block/hdb/queue/schedulerModules linked in: ipv6 cpufreq_ondemand acpi_cpufreq dm_mirror dm_multipath sbdPid: 5032, comm: rmdir Not tainted (2.6.29-rc7-vivek #17) Veriton M460
EIP: 0060:[<c04ec7de>] EFLAGS: 00010082 CPU: 0
EIP is atiocg_destroy+0xdc/0x14e
EAX: 00000000 EBX:f62278b4ECX:f6207800EDX:f6227904
ESI:f6227800EDI:f62278a0EBP: 00000003 ESP:c8790f00
DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
Process rmdir (pid: 5032, ti=c8790000 task=f6636960 task.ti=c8790000)Stack: f53cc5c0 f62b7258 f10991c8 00000282 f6227800 c0733fa0 ec1c5140 edfc6d34 c8790000 c04463ce f6a4c84c edfc6d34 0804c840 c048883c f6a4c84c f6a4c84c c04888d6 f6a4c84c 00000000 c04897de f6a4c84c c048504c f115f4c0 ebc37954 Call Trace: [<c04463ce>] cgroup_diput+0x41/0x8c [<c048883c>] dentry_iput+0x45/0x5e [<c04888d6>] d_kill+0x19/0x32 [<c04897de>] dput+0xd8/0xdf [<c048504c>] do_rmdir+0x8f/0xb6 [<c06330fc>] do_page_fault+0x2a2/0x579 [<c0402fc1>] sysenter_do_call+0x12/0x21 [<c0630000>] schedule+0x641/0x830 Code: 08 00 74 04 0f 0b eb fe 83 7f 04 00 74 04 0f 0b eb fe 45 83 c3 1c 83 fd 0 EIP: [<c04ec7de>] iocg_destroy+0xdc/0x14e SS:ESP 0068:c8790f00Signature · 2 lines
Regards Gui Jianfeng
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 6 lines · Vivek Goyal
Quoted 5 lines · Gui Jianfeng
Quoted 4 lines · Vivek Goyal
Hi All,
Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.
Quoted 8 lines
Quoted 4 lines
Hi Vivek,
I got the following OOPS when testing, can't reproduce again :(
Hi Gui,
Thanks for the report. Will look into it and see if I can reproduce it.
Hi Vivek,
The following script can reproduce the bug in my box.
#!/bin/sh
mkdir /cgroup
mount -t cgroup -o io io /cgroup mkdir /cgroup/test1
mkdir /cgroup/test2
echo cfq > /sys/block/sda/queue/scheduler echo 7 > /cgroup/test1/io.ioprio echo 1 > /cgroup/test2/io.ioprio echo 1 > /proc/sys/vm/drop_caches dd if=1000M.1 of=/dev/null & pid1=$!
echo $pid1
echo $pid1 > /cgroup/test1/tasks
dd if=1000M.2 of=/dev/null
pid2=$!
echo $pid2
echo $pid2 > /cgroup/test2/tasks
rmdir /cgroup/test1
rmdir /cgroup/test2
umount /cgroup
rmdir /cgroup
Signature · 2 lines
Regards Gui Jianfeng
Quoted 6 lines · Vivek Goyal
Quoted 5 lines · Gui Jianfeng
Quoted 4 lines · Vivek Goyal
Hi All,
Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.
Quoted 8 lines
Quoted 4 lines
Hi Vivek,
I got the following OOPS when testing, can't reproduce again :(
Hi Gui,
Thanks for the report. Will look into it and see if I can reproduce it.
Hi Vivek,
The following script can reproduce the bug in my box.
#!/bin/sh
mkdir /cgroup
mount -t cgroup -o io io /cgroup mkdir /cgroup/test1
mkdir /cgroup/test2
echo cfq > /sys/block/sda/queue/scheduler echo 7 > /cgroup/test1/io.ioprio echo 1 > /cgroup/test2/io.ioprio echo 1 > /proc/sys/vm/drop_caches dd if=1000M.1 of=/dev/null & pid1=$!
echo $pid1
echo $pid1 > /cgroup/test1/tasks
dd if=1000M.2 of=/dev/null
pid2=$!
echo $pid2
echo $pid2 > /cgroup/test2/tasks
rmdir /cgroup/test1
rmdir /cgroup/test2
umount /cgroup
rmdir /cgroup
Signature · 2 lines
Regards Gui Jianfeng
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 41 lines · Gui Jianfeng
Quoted 6 lines · Vivek Goyal
Quoted 5 lines · Gui Jianfeng
Quoted 4 lines · Vivek Goyal
Hi All,
Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.
Quoted 8 lines
Quoted 4 lines
Hi Vivek,
I got the following OOPS when testing, can't reproduce again :(
Hi Gui,
Thanks for the report. Will look into it and see if I can reproduce it.
Hi Vivek,
The following script can reproduce the bug in my box.
#!/bin/sh
mkdir /cgroup
mount -t cgroup -o io io /cgroup mkdir /cgroup/test1
mkdir /cgroup/test2echo cfq >
/sys/block/sda/queue/schedulerecho 7 > /cgroup/test1/io.ioprio echo 1 > /cgroup/test2/io.ioprio echo 1 >/proc/sys/vm/drop_cachesdd if=1000M.1 of=/dev/null & pid1=$!
echo $pid1
echo $pid1 > /cgroup/test1/tasks
dd if=1000M.2 of=/dev/null
pid2=$!
echo $pid2
echo $pid2 > /cgroup/test2/tasks
Quoted 4 lines
rmdir /cgroup/test1
rmdir /cgroup/test2
umount /cgroup
rmdir /cgroup
Yes, this bug happens when we move a task from a cgroup to another one, and delete the cgroup. Since the actual move to the new cgroup is performed in a delayed fashion, if the cgroup is removed before another request from the task is seen (and the actual move is performed) , it results in a hit on BUG_ON. I am working on a patch that will solve this problem and a few others; basically it would do ref counting for io_group structure. I am having a few problems with it at the moment; will post the patch as soon as I can get it to work.
Quoted 8 lines
--
Regards
Gui Jianfeng
Quoted 41 lines · Gui Jianfeng
Quoted 6 lines · Vivek Goyal
Quoted 5 lines · Gui Jianfeng
Quoted 4 lines · Vivek Goyal
Hi All,
Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.
Quoted 8 lines
Quoted 4 lines
Hi Vivek,
I got the following OOPS when testing, can't reproduce again :(
Hi Gui,
Thanks for the report. Will look into it and see if I can reproduce it.
Hi Vivek,
The following script can reproduce the bug in my box.
#!/bin/sh
mkdir /cgroup
mount -t cgroup -o io io /cgroup mkdir /cgroup/test1
mkdir /cgroup/test2echo cfq >
/sys/block/sda/queue/schedulerecho 7 > /cgroup/test1/io.ioprio echo 1 > /cgroup/test2/io.ioprio echo 1 >/proc/sys/vm/drop_cachesdd if=1000M.1 of=/dev/null & pid1=$!
echo $pid1
echo $pid1 > /cgroup/test1/tasks
dd if=1000M.2 of=/dev/null
pid2=$!
echo $pid2
echo $pid2 > /cgroup/test2/tasks
Quoted 4 lines
rmdir /cgroup/test1
rmdir /cgroup/test2
umount /cgroup
rmdir /cgroup
Thanks Gui. We have got races with task movement and cgroup deletion. In the original bfq patch, Fabio had implemented the logic to migrate the task queue synchronously. It found the logic to be little complicated so I changed it to delayed movement of queue from old cgroup to new cgroup. Fabio later mentioned that it introduces a race where old cgroup is deleted before task queue has actually moved to new cgroup.
Nauman is currently implementing reference counting for io groups and that will solve this problem at the same time some other problems like movement of queue to root group during cgroup deletion and which can potentially result in unfair share for some time to that queue etc.
Thanks
Vivek
Quoted 41 lines · Gui Jianfeng
Quoted 6 lines · Vivek Goyal
Quoted 5 lines · Gui Jianfeng
Quoted 4 lines · Vivek Goyal
Hi All,
Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.
Quoted 8 lines
Quoted 4 lines
Hi Vivek,
I got the following OOPS when testing, can't reproduce again :(
Hi Gui,
Thanks for the report. Will look into it and see if I can reproduce it.
Hi Vivek,
The following script can reproduce the bug in my box.
#!/bin/sh
mkdir /cgroup
mount -t cgroup -o io io /cgroup mkdir /cgroup/test1
mkdir /cgroup/test2echo cfq >
/sys/block/sda/queue/schedulerecho 7 > /cgroup/test1/io.ioprio echo 1 > /cgroup/test2/io.ioprio echo 1 >/proc/sys/vm/drop_cachesdd if=1000M.1 of=/dev/null & pid1=$!
echo $pid1
echo $pid1 > /cgroup/test1/tasks
dd if=1000M.2 of=/dev/null
pid2=$!
echo $pid2
echo $pid2 > /cgroup/test2/tasks
Quoted 4 lines
rmdir /cgroup/test1
rmdir /cgroup/test2
umount /cgroup
rmdir /cgroup
Yes, this bug happens when we move a task from a cgroup to another one, and delete the cgroup. Since the actual move to the new cgroup is performed in a delayed fashion, if the cgroup is removed before another request from the task is seen (and the actual move is performed) , it results in a hit on BUG_ON. I am working on a patch that will solve this problem and a few others; basically it would do ref counting for io_group structure. I am having a few problems with it at the moment; will post the patch as soon as I can get it to work.
Quoted 8 lines
--
Regards
Gui Jianfeng
Quoted 41 lines · Gui Jianfeng
Quoted 6 lines · Vivek Goyal
Quoted 5 lines · Gui Jianfeng
Quoted 4 lines · Vivek Goyal
Hi All,
Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.
Quoted 8 lines
Quoted 4 lines
Hi Vivek,
I got the following OOPS when testing, can't reproduce again :(
Hi Gui,
Thanks for the report. Will look into it and see if I can reproduce it.
Hi Vivek,
The following script can reproduce the bug in my box.
#!/bin/sh
mkdir /cgroup
mount -t cgroup -o io io /cgroup mkdir /cgroup/test1
mkdir /cgroup/test2echo cfq >
/sys/block/sda/queue/schedulerecho 7 > /cgroup/test1/io.ioprio echo 1 > /cgroup/test2/io.ioprio echo 1 >/proc/sys/vm/drop_cachesdd if=1000M.1 of=/dev/null & pid1=$!
echo $pid1
echo $pid1 > /cgroup/test1/tasks
dd if=1000M.2 of=/dev/null
pid2=$!
echo $pid2
echo $pid2 > /cgroup/test2/tasks
Quoted 4 lines
rmdir /cgroup/test1
rmdir /cgroup/test2
umount /cgroup
rmdir /cgroup
Thanks Gui. We have got races with task movement and cgroup deletion. In the original bfq patch, Fabio had implemented the logic to migrate the task queue synchronously. It found the logic to be little complicated so I changed it to delayed movement of queue from old cgroup to new cgroup. Fabio later mentioned that it introduces a race where old cgroup is deleted before task queue has actually moved to new cgroup.
Nauman is currently implementing reference counting for io groups and that will solve this problem at the same time some other problems like movement of queue to root group during cgroup deletion and which can potentially result in unfair share for some time to that queue etc.
Thanks
Vivek
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 60 lines · Vivek Goyal
Quoted 45 lines · Gui Jianfeng
Quoted 9 lines · Vivek Goyal
Quoted 7 lines · Gui Jianfeng
Quoted 5 lines · Vivek Goyal
Hi All,
Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.
Quoted 9 lines
Quoted 5 lines
Hi Vivek,
I got the following OOPS when testing, can't reproduce again :(
Hi Gui,
Thanks for the report. Will look into it and see if I can reproduce it.
Hi Vivek,
The following script can reproduce the bug in my box.
#!/bin/sh
mkdir /cgroup
mount -t cgroup -o io io /cgroup mkdir /cgroup/test1
mkdir /cgroup/test2echo cfq >
/sys/block/sda/queue/schedulerecho 7 > /cgroup/test1/io.ioprio echo 1 > /cgroup/test2/io.ioprio echo 1 >/proc/sys/vm/drop_cachesdd if=1000M.1 of=/dev/null & pid1=$!
echo $pid1
echo $pid1 > /cgroup/test1/tasks
dd if=1000M.2 of=/dev/null
pid2=$!
echo $pid2
echo $pid2 > /cgroup/test2/tasksQuoted 5 lines
rmdir /cgroup/test1
rmdir /cgroup/test2
umount /cgroup
rmdir /cgroupThanks Gui. We have got races with task movement and cgroup deletion. In the original bfq patch, Fabio had implemented the logic to migrate the task queue synchronously. It found the logic to be little complicated so I changed it to delayed movement of queue from old cgroup to new cgroup. Fabio later mentioned that it introduces a race where old cgroup is deleted before task queue has actually moved to new cgroup.
Quoted 8 lines
Nauman is currently implementing reference counting for io groups and that will solve this problem at the same time some other problems like movement of queue to root group during cgroup deletion and which can potentially result in unfair share for some time to that queue etc.
Thanks
Vivek
Hi Gui,
This patch should solve the problems reported by you. Please let me know if it does not work. @Vivek, this has a few more changes after the patch I sent you separately.
DESC
Add ref counting for io_group.
EDESC
Reference counting for io_group solves many problems, most of which occured when we tried to delete the cgroup. Earlier, ioqs were being moved out of cgroup to root cgroup. That is problematic in many ways:
First, the pending requests in queues might get unfair service, and will also cause unfairness for other cgroups at the root level. This problem can become signficant if cgroups are created and destroyed relatively frequently. Second, moving queues to root cgroups was complicated and was causing many BUG_ON's to trigger. Third, there is a single io queue in AS, Deadline and Noop within a cgroup; and it does not make sense to move it to the root cgroup. The same is true of async queues.
Requests already keep a reference on ioq, so queues keep a reference on cgroup. For async queues in CFQ, and single ioq in other schedulers, io_group also keeps are reference on io_queue. This reference on ioq is dropped when the queue is released (elv_release_ioq). So the queue can be freed.
When a queue is released, it puts the reference to io_group and the io_group is released after all the queues are released. Child groups also take reference on parent groups, and release it when they are destroyed.
Also we no longer need to maintain a seprate linked list of idle entities, which was maintained only to help release the ioq references during elevator switch. The code for releasing io_groups is reused for elevator switch, resulting in simpler and tight code.
block/cfq-iosched.c +11 -6
@@ -1090,8 +1090,8 @@ static void changed_cgroup(struct io_context *ioc, struct cfq_io_context *cic)if (async_cfqq !=NULL ) { __iog =cfqq_to_io_group (async_cfqq); -if (iog != __iog) { +/* Cgroup has changed, drop the reference to async queue */ cic_set_cfqq (cic,NULL ,0 );cfq_put_queue (async_cfqq); }
@@ -1099,8 +1099,10 @@ static void changed_cgroup(struct io_context *ioc, struct cfq_io_context *cic)if (sync_cfqq !=NULL ) { __iog =cfqq_to_io_group (sync_cfqq); -if (iog != __iog) -io_ioq_move (q->elevator, sync_cfqq->ioq, iog); +if (iog != __iog) { +cic_set_cfqq (cic,NULL ,1 ); +cfq_put_queue (sync_cfqq); + } }spin_unlock_irqrestore (q->queue_lock, flags);
@@ -1114,8 +1116,8 @@ static void cfq_ioc_set_cgroup(struct io_context *ioc)#endif /* CONFIG_IOSCHED_CFQ_HIER */ static struct cfq_queue * -cfq_find_alloc_queue (struct cfq_data *cfqd,int is_sync, -struct io_context *ioc,gfp_t gfp_mask) +cfq_find_alloc_queue (struct cfq_data *cfqd,struct io_group *iog, +int is_sync,struct io_context *ioc,gfp_t gfp_mask) {struct cfq_queue *cfqq, *new_cfqq =NULL ;struct cfq_io_context *cic;
@@ -1198,6 +1200,8 @@ alloc_ioq:elv_mark_ioq_sync (cfqq->ioq); } cfqq->pid = current->pid; +/* ioq reference on iog */ +elv_get_iog (iog);cfq_log_cfqq (cfqd, cfqq,"alloced" ); }
@@ -1229,7 +1233,8 @@ cfq_get_queue(struct cfq_data *cfqd, int is_sync, struct io_context *ioc, }if (!cfqq) { - cfqq =cfq_find_alloc_queue (cfqd, is_sync, ioc, gfp_mask); + cfqq =cfq_find_alloc_queue (cfqd, iog, + is_sync, ioc, gfp_mask);if (!cfqq)return NULL ; }
block/elevator-fq.c +98 -108
384 lines not on this page · raw message
block/elevator-fq.h +19 -0
@@ -213,6 +213,7 @@ struct io_group {struct hlist_node elv_data_node;struct hlist_node group_node;struct io_sched_data sched_data; +atomic_t ref;struct io_entity *my_entity;
@@ -229,6 +230,8 @@ struct io_group {/* Single ioq per group, used for noop, deadline, anticipatory */ struct io_queue *ioq; + +int deleting; };/**
@@ -462,6 +465,8 @@ extern int elv_fq_set_request_ioq(struct request_queue *q, struct request *rq, gfp_t gfp_mask);extern void elv_fq_unset_request_ioq (struct request_queue *q,struct request *rq); +extern void elv_put_iog (struct io_group *iog); +extern struct io_queue *elv_lookup_ioq_current (struct request_queue *q);/* Returns single ioq associated with the io group. */
@@ -480,6 +485,11 @@ static inline void io_group_set_ioq(struct io_group *iog, struct io_queue *ioq) iog->ioq = ioq; } +static inline void elv_get_iog (struct io_group *iog) +{ +atomic_inc (&iog->ref); +} +#else /* !GROUP_IOSCHED */ /* * No ioq movement is needed in case of flat setup. root io group gets cleaned
@@ -531,6 +541,15 @@ static inline struct io_queue *elv_lookup_ioq_current(struct request_queue *q) return NULL; } +static inline void elv_get_iog (struct io_group *iog) +{ +} + +static inline void elv_put_iog (struct io_group *iog) +{ +} + +#endif /* GROUP_IOSCHED */ /* Functions used by blksysfs.c */
Quoted 60 lines · Vivek Goyal
Quoted 45 lines · Gui Jianfeng
Quoted 9 lines · Vivek Goyal
Quoted 7 lines · Gui Jianfeng
Quoted 5 lines · Vivek Goyal
Hi All,
Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.
Quoted 9 lines
Quoted 5 lines
Hi Vivek,
I got the following OOPS when testing, can't reproduce again :(
Hi Gui,
Thanks for the report. Will look into it and see if I can reproduce it.
Hi Vivek,
The following script can reproduce the bug in my box.
#!/bin/sh
mkdir /cgroup
mount -t cgroup -o io io /cgroup mkdir /cgroup/test1
mkdir /cgroup/test2echo cfq >
/sys/block/sda/queue/schedulerecho 7 > /cgroup/test1/io.ioprio echo 1 > /cgroup/test2/io.ioprio echo 1 >/proc/sys/vm/drop_cachesdd if=1000M.1 of=/dev/null & pid1=$!
echo $pid1
echo $pid1 > /cgroup/test1/tasks
dd if=1000M.2 of=/dev/null
pid2=$!
echo $pid2
echo $pid2 > /cgroup/test2/tasksQuoted 5 lines
rmdir /cgroup/test1
rmdir /cgroup/test2
umount /cgroup
rmdir /cgroupThanks Gui. We have got races with task movement and cgroup deletion. In the original bfq patch, Fabio had implemented the logic to migrate the task queue synchronously. It found the logic to be little complicated so I changed it to delayed movement of queue from old cgroup to new cgroup. Fabio later mentioned that it introduces a race where old cgroup is deleted before task queue has actually moved to new cgroup.
Quoted 8 lines
Nauman is currently implementing reference counting for io groups and that will solve this problem at the same time some other problems like movement of queue to root group during cgroup deletion and which can potentially result in unfair share for some time to that queue etc.
Thanks
Vivek
Hi Gui,
This patch should solve the problems reported by you. Please let me know if it does not work. @Vivek, this has a few more changes after the patch I sent you separately.
DESC
Add ref counting for io_group.
EDESC
Reference counting for io_group solves many problems, most of which occured when we tried to delete the cgroup. Earlier, ioqs were being moved out of cgroup to root cgroup. That is problematic in many ways:
First, the pending requests in queues might get unfair service, and will also cause unfairness for other cgroups at the root level. This problem can become signficant if cgroups are created and destroyed relatively frequently. Second, moving queues to root cgroups was complicated and was causing many BUG_ON's to trigger. Third, there is a single io queue in AS, Deadline and Noop within a cgroup; and it does not make sense to move it to the root cgroup. The same is true of async queues.
Requests already keep a reference on ioq, so queues keep a reference on cgroup. For async queues in CFQ, and single ioq in other schedulers, io_group also keeps are reference on io_queue. This reference on ioq is dropped when the queue is released (elv_release_ioq). So the queue can be freed.
When a queue is released, it puts the reference to io_group and the io_group is released after all the queues are released. Child groups also take reference on parent groups, and release it when they are destroyed.
Also we no longer need to maintain a seprate linked list of idle entities, which was maintained only to help release the ioq references during elevator switch. The code for releasing io_groups is reused for elevator switch, resulting in simpler and tight code.
block/cfq-iosched.c +11 -6
@@ -1090,8 +1090,8 @@ static void changed_cgroup(struct io_context *ioc, struct cfq_io_context *cic)if (async_cfqq !=NULL ) { __iog =cfqq_to_io_group (async_cfqq); -if (iog != __iog) { +/* Cgroup has changed, drop the reference to async queue */ cic_set_cfqq (cic,NULL ,0 );cfq_put_queue (async_cfqq); }
@@ -1099,8 +1099,10 @@ static void changed_cgroup(struct io_context *ioc, struct cfq_io_context *cic)if (sync_cfqq !=NULL ) { __iog =cfqq_to_io_group (sync_cfqq); -if (iog != __iog) -io_ioq_move (q->elevator, sync_cfqq->ioq, iog); +if (iog != __iog) { +cic_set_cfqq (cic,NULL ,1 ); +cfq_put_queue (sync_cfqq); + } }spin_unlock_irqrestore (q->queue_lock, flags);
@@ -1114,8 +1116,8 @@ static void cfq_ioc_set_cgroup(struct io_context *ioc)#endif /* CONFIG_IOSCHED_CFQ_HIER */ static struct cfq_queue * -cfq_find_alloc_queue (struct cfq_data *cfqd,int is_sync, -struct io_context *ioc,gfp_t gfp_mask) +cfq_find_alloc_queue (struct cfq_data *cfqd,struct io_group *iog, +int is_sync,struct io_context *ioc,gfp_t gfp_mask) {struct cfq_queue *cfqq, *new_cfqq =NULL ;struct cfq_io_context *cic;
@@ -1198,6 +1200,8 @@ alloc_ioq:elv_mark_ioq_sync (cfqq->ioq); } cfqq->pid = current->pid; +/* ioq reference on iog */ +elv_get_iog (iog);cfq_log_cfqq (cfqd, cfqq,"alloced" ); }
@@ -1229,7 +1233,8 @@ cfq_get_queue(struct cfq_data *cfqd, int is_sync, struct io_context *ioc, }if (!cfqq) { - cfqq =cfq_find_alloc_queue (cfqd, is_sync, ioc, gfp_mask); + cfqq =cfq_find_alloc_queue (cfqd, iog, + is_sync, ioc, gfp_mask);if (!cfqq)return NULL ; }
block/elevator-fq.c +98 -108
384 lines not on this page · raw message
block/elevator-fq.h +19 -0
@@ -213,6 +213,7 @@ struct io_group {struct hlist_node elv_data_node;struct hlist_node group_node;struct io_sched_data sched_data; +atomic_t ref;struct io_entity *my_entity;
@@ -229,6 +230,8 @@ struct io_group {/* Single ioq per group, used for noop, deadline, anticipatory */ struct io_queue *ioq; + +int deleting; };/**
@@ -462,6 +465,8 @@ extern int elv_fq_set_request_ioq(struct request_queue *q, struct request *rq, gfp_t gfp_mask);extern void elv_fq_unset_request_ioq (struct request_queue *q,struct request *rq); +extern void elv_put_iog (struct io_group *iog); +extern struct io_queue *elv_lookup_ioq_current (struct request_queue *q);/* Returns single ioq associated with the io group. */
@@ -480,6 +485,11 @@ static inline void io_group_set_ioq(struct io_group *iog, struct io_queue *ioq) iog->ioq = ioq; } +static inline void elv_get_iog (struct io_group *iog) +{ +atomic_inc (&iog->ref); +} +#else /* !GROUP_IOSCHED */ /* * No ioq movement is needed in case of flat setup. root io group gets cleaned
@@ -531,6 +541,15 @@ static inline struct io_queue *elv_lookup_ioq_current(struct request_queue *q) return NULL; } +static inline void elv_get_iog (struct io_group *iog) +{ +} + +static inline void elv_put_iog (struct io_group *iog) +{ +} + +#endif /* GROUP_IOSCHED */ /* Functions used by blksysfs.c */
A message that is not in the archive (sent off-list or not yet received): [email protected]
Nauman Rafique wrote:
...
Quoted 4 lines
Hi Gui,
This patch should solve the problems reported by you. Please let me know if it does not work. @Vivek, this has a few more changes after the patch I sent you separately.
Hi Nauman,
I'v tried your patch, seems the bug has been fixed. thanks!
Quoted 32 lines
DESC
Add ref counting forio_group.
EDESCReference counting for
io_groupsolves many problems, most of which occured when we tried to delete the cgroup. Earlier, ioqs were being moved out of cgroup to root cgroup. That is problematic in many ways:
First, the pending requests in queues might get unfair service, and will also cause unfairness for other cgroups at the root level. This problem can become signficant if cgroups are created and destroyed relatively frequently. Second, moving queues to root cgroups was complicated and was causing many BUG_ON's to trigger. Third, there is a single io queue in AS, Deadline and Noop within a cgroup; and it does not make sense to move it to the root cgroup. The same is true of async queues.Requests already keep a reference on ioq, so queues keep a reference on cgroup. For async queues in CFQ, and single ioq in other schedulers,
io_groupalso keeps are reference onio_queue. This reference on ioq is dropped when the queue is released (elv_release_ioq). So the queue can be freed.When a queue is released, it puts the reference to
io_groupand theio_groupis released after all the queues are released. Child groups also take reference on parent groups, and release it when they are destroyed.Also we no longer need to maintain a seprate linked list of idle entities, which was maintained only to help release the ioq references during elevator switch. The code for releasing
io_groupsis reused for elevator switch, resulting in simpler and tight code.
Signature · 2 lines
Regards Gui Jianfeng
Nauman Rafique wrote:
...
Quoted 4 lines
Hi Gui,
This patch should solve the problems reported by you. Please let me know if it does not work. @Vivek, this has a few more changes after the patch I sent you separately.
Hi Nauman,
I'v tried your patch, seems the bug has been fixed. thanks!
Quoted 32 lines
DESC
Add ref counting forio_group.
EDESCReference counting for
io_groupsolves many problems, most of which occured when we tried to delete the cgroup. Earlier, ioqs were being moved out of cgroup to root cgroup. That is problematic in many ways:
First, the pending requests in queues might get unfair service, and will also cause unfairness for other cgroups at the root level. This problem can become signficant if cgroups are created and destroyed relatively frequently. Second, moving queues to root cgroups was complicated and was causing many BUG_ON's to trigger. Third, there is a single io queue in AS, Deadline and Noop within a cgroup; and it does not make sense to move it to the root cgroup. The same is true of async queues.Requests already keep a reference on ioq, so queues keep a reference on cgroup. For async queues in CFQ, and single ioq in other schedulers,
io_groupalso keeps are reference onio_queue. This reference on ioq is dropped when the queue is released (elv_release_ioq). So the queue can be freed.When a queue is released, it puts the reference to
io_groupand theio_groupis released after all the queues are released. Child groups also take reference on parent groups, and release it when they are destroyed.Also we no longer need to maintain a seprate linked list of idle entities, which was maintained only to help release the ioq references during elevator switch. The code for releasing
io_groupsis reused for elevator switch, resulting in simpler and tight code.
Signature · 2 lines
Regards Gui Jianfeng
Quoted 35 lines · Vivek Goyal
Hi All,
Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.
http://lkml.org/lkml/2008/11/6/227
One of the takeaway from the discussion in this thread was that let us implement a common layer which contains the proportional weight scheduling code which can be shared by all the IO schedulers.
Implementing IO controller will not cover the devices which don't use IO schedulers but it should cover the common case.
There were more discussions regarding 2 level vs 1 level IO control at following link.
https://lists.linux-foundation.org/pipermail/containers/2009-January/015402.html
So in the mean time we took the discussion off the list and spent time on making the 1 level control apporoach work where majority of the proportional weight control is shared by the four schedulers instead of each one having to replicate the code. We make use of BFQ code for fair queuing as posted by Paolo and Fabio here.
http://lkml.org/lkml/2008/11/11/148
Details about design and howto have been put in documentation patch.
I have done very basic testing of running 2 or 3 "dd" threads in different cgroups. Wanted to get the patchset out for feedback/review before we dive into more bug fixing, benchmarking, optimizations etc.
Your feedback/comments are welcome.
Quoted 8 lines
Patch series contains 10 patches. It should be compilable and bootable after every patch. Intial 2 patches implement flat fair queuing (no cgroup support) and make cfq to use that. Later patches introduce hierarchical fair queuing support in elevator layer and modify other IO schdulers to use that.
Thanks
Vivek
Hi Vivek,
While testing these patches along with the bio-cgroup patches I noticed that for the case of 2 buffered writers (dd) with different weights, one of them would be able to use up a very large timeslice (I've seen upto 500ms) when the other queue is empty and not be accounted for it. This is due to the check in cfq_dispatch_requests() where a given cgroup can empty its entire queue (100 IOs or more) within its timeslice and have them sit in the dispatch queue ready for the disk driver to pick up. Moreover, this huge timeslice is not accounted for as this cgroup is charged only for the length of the intended timeslice and not the actual time taken.
The following patch fixes this by not optimizing on the single busy queue fact inside cfq_dispatch_requests. Note that this does not hurt throughput in any sense but just causes more IOs to be dispatched only when the drive is ready for them thus leading to better accounting too.
Fix bug where a given ioq can run through all its requests at once.
---
2.6.26/block/cfq-iosched.c +0 -0
@@ -848,8 +848,7 @@ static int cfq_dispatch_requests(struct request_queue *q, int force)
if (cfq_class_idle(cfqq)) max_dispatch = 1;
- if (elv_ioq_nr_dispatched(cfqq->ioq) >= max_dispatch &&
- elv_nr_busy_ioq(q->elevator) > 1)
+ if (elv_ioq_nr_dispatched(cfqq->ioq) >= max_dispatch)
break; if (cfqd->sync_flight && !cfq_cfqq_sync(cfqq))
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 42 lines · Divyesh Shah
Quoted 41 lines · Vivek Goyal
Hi All,
Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.
http://lkml.org/lkml/2008/11/6/227
One of the takeaway from the discussion in this thread was that let us implement a common layer which contains the proportional weight scheduling code which can be shared by all the IO schedulers.
Implementing IO controller will not cover the devices which don't use IO schedulers but it should cover the common case.
There were more discussions regarding 2 level vs 1 level IO control at following link.
https://lists.linux-foundation.org/pipermail/containers/2009-January/015402.html
So in the mean time we took the discussion off the list and spent time on making the 1 level control apporoach work where majority of the proportional weight control is shared by the four schedulers instead of each one having to replicate the code. We make use of BFQ code for fair queuing as posted by Paolo and Fabio here.
http://lkml.org/lkml/2008/11/11/148
Details about design and howto have been put in documentation patch.
I have done very basic testing of running 2 or 3 "dd" threads in different cgroups. Wanted to get the patchset out for feedback/review before we dive into more bug fixing, benchmarking, optimizations etc.
Your feedback/comments are welcome.
Patch series contains 10 patches. It should be compilable and bootable after every patch. Intial 2 patches implement flat fair queuing (no cgroup support) and make cfq to use that. Later patches introduce hierarchical fair queuing support in elevator layer and modify other IO schdulers to use that.
Quoted 6 lines
Quoted 2 lines
Thanks
VivekHi Vivek,
While testing these patches along with the bio-cgroup patches I noticed that for the case of 2 buffered writers (dd) with different weights, one of them would be able to use up a very large timeslice (I've seen upto 500ms) when the other queue is empty and not be accounted for it. This is due to the check incfq_dispatch_requests()where a given cgroup can empty its entire queue (100 IOs or more) within its timeslice and have them sit in the dispatch queue ready for the disk driver to pick up. Moreover, this huge timeslice is not accounted for as this cgroup is charged only for the length of the intended timeslice and not the actual time taken.
The following patch fixes this by not optimizing on the single busy queue fact insidecfq_dispatch_requests. Note that this does not hurt throughput in any sense but just causes more IOs to be dispatched only when the drive is ready for them thus leading to better accounting too.
Hi Divyesh,
Thanks for the testing and noticing the issue. I also had noticed this issue.
Couple of points.
- In 30-rc3 jens has fixed the huge dispatch problem. Now in case of single ioq doing dispatch, in one round upto 4*quantum request can be dispatched. So that means in default configuration with single queue, maximum request on diaptch list can be 16.
- Secondly, in my tree, now I have modified the patches to charge for actual consumption of the slice instead of capping it to budget. In a week's time I should be able to post V2 of the patches. Please do try it out then
Thanks
Vivek
Quoted 19 lines
Fix bug where a given ioq can run through all its requests at once.
Signed-off-by Divyesh Shah <[email protected]>---
2.6.26/block/cfq-iosched.c +1
-2@@ -848,8 +848,7 @@ static int cfq_dispatch_requests(struct request_queue *q, int force)if (cfq_class_idle (cfqq)) max_dispatch =1 ; -if (elv_ioq_nr_dispatched (cfqq->ioq) >= max_dispatch && -elv_nr_busy_ioq (q->elevator) >1 ) +if (elv_ioq_nr_dispatched (cfqq->ioq) >= max_dispatch)break ;if (cfqd->sync_flight && !cfq_cfqq_sync (cfqq))
Quoted 42 lines · Divyesh Shah
Quoted 41 lines · Vivek Goyal
Hi All,
Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.
http://lkml.org/lkml/2008/11/6/227
One of the takeaway from the discussion in this thread was that let us implement a common layer which contains the proportional weight scheduling code which can be shared by all the IO schedulers.
Implementing IO controller will not cover the devices which don't use IO schedulers but it should cover the common case.
There were more discussions regarding 2 level vs 1 level IO control at following link.
https://lists.linux-foundation.org/pipermail/containers/2009-January/015402.html
So in the mean time we took the discussion off the list and spent time on making the 1 level control apporoach work where majority of the proportional weight control is shared by the four schedulers instead of each one having to replicate the code. We make use of BFQ code for fair queuing as posted by Paolo and Fabio here.
http://lkml.org/lkml/2008/11/11/148
Details about design and howto have been put in documentation patch.
I have done very basic testing of running 2 or 3 "dd" threads in different cgroups. Wanted to get the patchset out for feedback/review before we dive into more bug fixing, benchmarking, optimizations etc.
Your feedback/comments are welcome.
Patch series contains 10 patches. It should be compilable and bootable after every patch. Intial 2 patches implement flat fair queuing (no cgroup support) and make cfq to use that. Later patches introduce hierarchical fair queuing support in elevator layer and modify other IO schdulers to use that.
Quoted 6 lines
Quoted 2 lines
Thanks
VivekHi Vivek,
While testing these patches along with the bio-cgroup patches I noticed that for the case of 2 buffered writers (dd) with different weights, one of them would be able to use up a very large timeslice (I've seen upto 500ms) when the other queue is empty and not be accounted for it. This is due to the check incfq_dispatch_requests()where a given cgroup can empty its entire queue (100 IOs or more) within its timeslice and have them sit in the dispatch queue ready for the disk driver to pick up. Moreover, this huge timeslice is not accounted for as this cgroup is charged only for the length of the intended timeslice and not the actual time taken.
The following patch fixes this by not optimizing on the single busy queue fact insidecfq_dispatch_requests. Note that this does not hurt throughput in any sense but just causes more IOs to be dispatched only when the drive is ready for them thus leading to better accounting too.
Hi Divyesh,
Thanks for the testing and noticing the issue. I also had noticed this issue.
Couple of points.
- In 30-rc3 jens has fixed the huge dispatch problem. Now in case of single ioq doing dispatch, in one round upto 4*quantum request can be dispatched. So that means in default configuration with single queue, maximum request on diaptch list can be 16.
- Secondly, in my tree, now I have modified the patches to charge for actual consumption of the slice instead of capping it to budget. In a week's time I should be able to post V2 of the patches. Please do try it out then
Thanks
Vivek
Quoted 19 lines
Fix bug where a given ioq can run through all its requests at once.
Signed-off-by Divyesh Shah <[email protected]>---
2.6.26/block/cfq-iosched.c +1
-2@@ -848,8 +848,7 @@ static int cfq_dispatch_requests(struct request_queue *q, int force)if (cfq_class_idle (cfqq)) max_dispatch =1 ; -if (elv_ioq_nr_dispatched (cfqq->ioq) >= max_dispatch && -elv_nr_busy_ioq (q->elevator) >1 ) +if (elv_ioq_nr_dispatched (cfqq->ioq) >= max_dispatch)break ;if (cfqd->sync_flight && !cfq_cfqq_sync (cfqq))
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 54 lines · Vivek Goyal
Quoted 42 lines · Divyesh Shah
Quoted 41 lines · Vivek Goyal
Hi All,
Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.
http://lkml.org/lkml/2008/11/6/227
One of the takeaway from the discussion in this thread was that let us implement a common layer which contains the proportional weight scheduling code which can be shared by all the IO schedulers.
Implementing IO controller will not cover the devices which don't use IO schedulers but it should cover the common case.
There were more discussions regarding 2 level vs 1 level IO control at following link.
https://lists.linux-foundation.org/pipermail/containers/2009-January/015402.html
So in the mean time we took the discussion off the list and spent time on making the 1 level control apporoach work where majority of the proportional weight control is shared by the four schedulers instead of each one having to replicate the code. We make use of BFQ code for fair queuing as posted by Paolo and Fabio here.
http://lkml.org/lkml/2008/11/11/148
Details about design and howto have been put in documentation patch.
I have done very basic testing of running 2 or 3 "dd" threads in different cgroups. Wanted to get the patchset out for feedback/review before we dive into more bug fixing, benchmarking, optimizations etc.
Your feedback/comments are welcome.
Patch series contains 10 patches. It should be compilable and bootable after every patch. Intial 2 patches implement flat fair queuing (no cgroup support) and make cfq to use that. Later patches introduce hierarchical fair queuing support in elevator layer and modify other IO schdulers to use that.
Quoted 5 lines
Quoted 2 lines
Thanks
VivekHi Vivek,
While testing these patches along with the bio-cgroup patches I noticed that for the case of 2 buffered writers (dd) with different weights, one of them would be able to use up a very large timeslice (I've seen upto 500ms) when the other queue is empty and not be accounted for it. This is due to the check incfq_dispatch_requests()where a given cgroup can empty its entire queue (100 IOs or more) within its timeslice and have them sit in the dispatch queue ready for the disk driver to pick up. Moreover, this huge timeslice is not accounted for as this cgroup is charged only for the length of the intended timeslice and not the actual time taken.
The following patch fixes this by not optimizing on the single busy queue fact insidecfq_dispatch_requests. Note that this does not hurt throughput in any sense but just causes more IOs to be dispatched only when the drive is ready for them thus leading to better accounting too.Hi Divyesh,
Thanks for the testing and noticing the issue. I also had noticed this issue.
Quoted 6 lines
Couple of points.
- In 30-rc3 jens has fixed the huge dispatch problem. Now in case of single ioq doing dispatch, in one round upto 4*quantum request can be dispatched. So that means in default configuration with single queue, maximum request on diaptch list can be 16.
I just synced my git tree and I see Jens changes. That makes this much cleaner!
Quoted 5 lines
- Secondly, in my tree, now I have modified the patches to charge for actual consumption of the slice instead of capping it to budget. In a week's time I should be able to post V2 of the patches. Please do try it out then
I have that in my tree as well and was going to send that out. No need now :)
Quoted 22 lines
Thanks
VivekQuoted 18 lines
Fix bug where a given ioq can run through all its requests at once.
Signed-off-by Divyesh Shah <[email protected]>---
2.6.26/block/cfq-iosched.c +1
-2@@ -848,8 +848,7 @@ static int cfq_dispatch_requests(struct request_queue *q, int force)if (cfq_class_idle (cfqq)) max_dispatch =1 ; -if (elv_ioq_nr_dispatched (cfqq->ioq) >= max_dispatch && -elv_nr_busy_ioq (q->elevator) >1 ) +if (elv_ioq_nr_dispatched (cfqq->ioq) >= max_dispatch)break ;if (cfqd->sync_flight && !cfq_cfqq_sync (cfqq))
Quoted 54 lines · Vivek Goyal
Quoted 42 lines · Divyesh Shah
Quoted 41 lines · Vivek Goyal
Hi All,
Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.
http://lkml.org/lkml/2008/11/6/227
One of the takeaway from the discussion in this thread was that let us implement a common layer which contains the proportional weight scheduling code which can be shared by all the IO schedulers.
Implementing IO controller will not cover the devices which don't use IO schedulers but it should cover the common case.
There were more discussions regarding 2 level vs 1 level IO control at following link.
https://lists.linux-foundation.org/pipermail/containers/2009-January/015402.html
So in the mean time we took the discussion off the list and spent time on making the 1 level control apporoach work where majority of the proportional weight control is shared by the four schedulers instead of each one having to replicate the code. We make use of BFQ code for fair queuing as posted by Paolo and Fabio here.
http://lkml.org/lkml/2008/11/11/148
Details about design and howto have been put in documentation patch.
I have done very basic testing of running 2 or 3 "dd" threads in different cgroups. Wanted to get the patchset out for feedback/review before we dive into more bug fixing, benchmarking, optimizations etc.
Your feedback/comments are welcome.
Patch series contains 10 patches. It should be compilable and bootable after every patch. Intial 2 patches implement flat fair queuing (no cgroup support) and make cfq to use that. Later patches introduce hierarchical fair queuing support in elevator layer and modify other IO schdulers to use that.
Quoted 5 lines
Quoted 2 lines
Thanks
VivekHi Vivek,
While testing these patches along with the bio-cgroup patches I noticed that for the case of 2 buffered writers (dd) with different weights, one of them would be able to use up a very large timeslice (I've seen upto 500ms) when the other queue is empty and not be accounted for it. This is due to the check incfq_dispatch_requests()where a given cgroup can empty its entire queue (100 IOs or more) within its timeslice and have them sit in the dispatch queue ready for the disk driver to pick up. Moreover, this huge timeslice is not accounted for as this cgroup is charged only for the length of the intended timeslice and not the actual time taken.
The following patch fixes this by not optimizing on the single busy queue fact insidecfq_dispatch_requests. Note that this does not hurt throughput in any sense but just causes more IOs to be dispatched only when the drive is ready for them thus leading to better accounting too.Hi Divyesh,
Thanks for the testing and noticing the issue. I also had noticed this issue.
Quoted 6 lines
Couple of points.
- In 30-rc3 jens has fixed the huge dispatch problem. Now in case of single ioq doing dispatch, in one round upto 4*quantum request can be dispatched. So that means in default configuration with single queue, maximum request on diaptch list can be 16.
I just synced my git tree and I see Jens changes. That makes this much cleaner!
Quoted 5 lines
- Secondly, in my tree, now I have modified the patches to charge for actual consumption of the slice instead of capping it to budget. In a week's time I should be able to post V2 of the patches. Please do try it out then
I have that in my tree as well and was going to send that out. No need now :)
Quoted 22 lines
Thanks
VivekQuoted 18 lines
Fix bug where a given ioq can run through all its requests at once.
Signed-off-by Divyesh Shah <[email protected]>---
2.6.26/block/cfq-iosched.c +1
-2@@ -848,8 +848,7 @@ static int cfq_dispatch_requests(struct request_queue *q, int force)if (cfq_class_idle (cfqq)) max_dispatch =1 ; -if (elv_ioq_nr_dispatched (cfqq->ioq) >= max_dispatch && -elv_nr_busy_ioq (q->elevator) >1 ) +if (elv_ioq_nr_dispatched (cfqq->ioq) >= max_dispatch)break ;if (cfqd->sync_flight && !cfq_cfqq_sync (cfqq))