Skip to contentSkip to conversation

IO Controller

RFC In review Vivek Goyal 31 participants 160 messages Mar 12, 2009 – May 5, 2009 42 messages not in the archive Page 2 of 2

Subsystems (3)
lore
Share & tools
lore
#108 lore raw

Re: IO Controller per cgroup request descriptors (Re: [PATCH 01/10] Documentation)

In reply to a message not in the archive
Quoted 39 lines · Vivek Goyal
Quoted 30 lines · Munehiro
Quoted 25 lines · Vivek Goyal
Quoted 7 lines
Quoted 6 lines

+TODO

+====
+- Lots of cleanups, testing, bug fixing, optimizations, benchmarking etc...
+- Convert cgroup ioprio to notion of weight.
+- Anticipatory code will need more work. It is not working properly currently

+  and needs more thought.

What are the problems with the code?

Have not got a chance to look into the issues in detail yet. Just a crude run saw drop in performance. Will debug it later the moment I have got async writes handled...

Quoted 6 lines
Quoted 1 line

+- Use of bio-cgroup patches.

I saw these posted as well

Quoted 2 lines

+- Use of Nauman's per cgroup request descriptor patches. +

More details would be nice, I am not sure I understand

Currently the number of request descriptors which can be allocated per device/request queue are fixed by a sysfs tunable (q->nr_requests). So if there is lots of IO going on from one cgroup then it will consume all the available request descriptors and other cgroup might starve and not get its fair share.

Quoted 3 lines

Hence we also need to introduce the notion of request descriptor limit per cgroup so that if request descriptors from one group are exhausted, then it does not impact the IO of other cgroup.

Quoted 7 lines

Unfortunately I couldn't find and I've never seen the Nauman's patches. So I tried to make a patch below against this todo.  The reason why I'm posting this despite this is just a quick and ugly hack (and it might be a reinvention of wheel) is that I would like to discuss how we should define the limitation of requests per cgroup.
This patch should be applied on Vivek's I/O controller patches posted on Mar 11.

Quoted 4 lines

Hi IKEDA,

Sorry for the confusion here. Actually Nauman had sent a patch to select group of people who were initially copied on the mail thread.

I am sorry about that. Since I dropped my whole patch set in favor of Vivek's stuff, this stuff fell through the cracks.

Quoted 33 lines
Quoted 9 lines

This patch temporarily distribute q->nr_requests to each cgroup. I think the number should be weighted like BFQ's budget.  But in this case, if the hierarchy of cgroup is deep, leaf cgroups are allowed to allocate very few numbers of requests.  I don't think this is reasonable...but I don't have specific idea to solve this problem.  Does anyone have the good idea?

Thanks for the patch. Yes, ideally one would expect the request descriptor to be allocated also in proportion to the weight but I guess that would become very comlicated.

In terms of simpler things, two thoughts come to mind.

- First approach is to make q->nr_requests per group. So every group is entitled for q->nr_requests as set by the user. This is what your patch seems to have done.

 I had some concerns with this approach. First of all it does not seem to have an upper bound on number of request descriptors allocated per queue because if a user creates more cgroups, total number of request descriptors increase.

- Second approach can be that we retain the meaning of q->nr_requests which defines the total number of request descriptors on the queue (with the exception of 50% more descriptors for batching processes). And we define a new per group limit q->nr_group_requests which defines how many requests per group can be assigned. So q->nr_requests defines total pool size on the queue and q->nr_group_requests will define how many requests each group can allocate out of that pool.

Quoted 8 lines

 Here the issue is that a user shall have to balance the q->nr_group_requests    and q->nr_requests properly.

To experiment, I have implemented the second approach. I am attaching the patch which is in my current tree. It probably will not apply on my march 11 posting as since then patches have changed. But posting it here so that at least it will give an idea behind the thought process.

Ideas are welcome...

I had started with the first option, but the second option sounds good too. But one problem that comes to mind is how we deal with hierarchies? The sys admin can limit the root level cgroups to specific number of request descriptors, but if applications running in a cgroup are allowed to create their own cgroups, then the total request descriptors of all child cgroups should be capped by the number assigned to parent cgroups.

Quoted 1,385 lines

Thanks
Vivek

o Currently a request queue has got fixed number of request descriptors for sync and async requests. Once the request descriptors are consumed, new processes are put to sleep and they effectively become serialized. Because sync and async queues are separate, async requests don't impact sync ones but if one is looking for fairness between async requests, that is not achievable if request queue descriptors become bottleneck.

o Make request descriptor's per io group so that if there is lots of IO going on in one cgroup, it does not impact the IO of other group.

o This patch implements the per cgroup request descriptors. request pool per queue is still common but every group will have its own wait list and its own count of request descriptors allocated to that group for sync and async queues. So effectively request_list becomes per io group property and not a global request queue feature.

o Currently one can define q->nr_requests to limit request descriptors allocated for the queue. Now there is another tunable q->nr_group_requests which controls the requests descriptr limit per group. q->nr_requests supercedes q->nr_group_requests to make sure if there are lots of groups present, we don't end up allocating too many request descriptors on the queue.

o Issues: Currently notion of congestion is per queue. With per group request descriptor it is possible that queue is not congested but the group bio will go into is congested.

---
 block/blk-core.c       |  216 ++++++++++++++++++++++++++++++++++--------------- block/blk-settings.c   |    3
 block/blk-sysfs.c      |   57 ++++++++++--
 block/elevator-fq.c    |   15 +++
 block/elevator-fq.h    |    8 +
 block/elevator.c       |    6 -
 include/linux/blkdev.h |   62 +++++++++++++-

 7 files changed, 287 insertions(+), 80 deletions(-)

Index: linux9/include/linux/blkdev.h
===================================================================

include/linux/blkdev.h       2009-04-30 16:18:29.000000000 -0400 renamed from include/linux/blkdev.h  2009-04-30 15:43:53.000000000 -0400 +36 -2
@@ -32,21 +32,51 @@ struct request;
  struct sg_io_hdr;
 
  #define BLKDEV_MIN_RQ  4
+
+#ifdef CONFIG_GROUP_IOSCHED
+#define BLKDEV_MAX_RQ  256     /* Default maximum */
+#define BLKDEV_MAX_GROUP_RQ    64      /* Default maximum */
+#else
  #define BLKDEV_MAX_RQ  128     /* Default maximum */
+/*
+ * This is eqivalent to case of only one group present (root group). Let
+ * it consume all the request descriptors available on the queue .
+ */
+#define BLKDEV_MAX_GROUP_RQ    BLKDEV_MAX_RQ      /* Default maximum */
+#endif
 
  struct request;
  typedef void (rq_end_io_fn)(struct request *, int);
 
  struct request_list {
        /*
-        * count[], starved[], and wait[] are indexed by
+        * count[], starved and wait[] are indexed by
         * BLK_RW_SYNC/BLK_RW_ASYNC
         */
        int count[2];
        int starved[2];
+       wait_queue_head_t wait[2];
+};
+
+/*
+ * This data structures keeps track of mempool of requests for the queue
+ * and some overall statistics.
+ */
+struct request_data {
+       /*
+        * Per queue request descriptor count. This is in addition to per
+        * cgroup count
+        */
+       int count[2];
        int elvpriv;
        mempool_t *rq_pool;
-       wait_queue_head_t wait[2];
+       int starved;
+       /*
+        * Global list for starved tasks. A task will be queued here if
+        * it could not allocate request descriptor and the associated
+        * group request list does not have any requests pending.
+        */
+       wait_queue_head_t starved_wait;
  };
 
  /*
@@ -251,6 +281,7 @@ struct request {
  #ifdef CONFIG_GROUP_IOSCHED
        /* io group request belongs to */
        struct io_group *iog;
+       struct request_list *rl;
  #endif /* GROUP_IOSCHED */
  #endif /* ELV_FAIR_QUEUING */
  };
@@ -340,6 +371,9 @@ struct request_queue
         */
        struct request_list     rq;
 
+       /* Contains request pool and other data like starved data */
+       struct request_data     rq_data;
+
        request_fn_proc         *request_fn;
        make_request_fn         *make_request_fn;
        prep_rq_fn              *prep_rq_fn;
@@ -402,6 +436,8 @@ struct request_queue
         * queue settings
         */

       unsigned long           nr_requests;    /* Max # of requests */
+       /* Max # of per io group requests */
+       unsigned long           nr_group_requests;
       unsigned int            nr_congestion_on;
       unsigned int            nr_congestion_off;
       unsigned int            nr_batching;

include/linux/blkdev.h       2009-04-30 16:18:29.000000000 -0400 renamed from include/linux/blkdev.h  2009-04-30 15:43:53.000000000 -0400 +22 -0
@@ -773,6 +809,28 @@ extern int scsi_cmd_ioctl(struct request
  extern int sg_scsi_ioctl(struct request_queue *, struct gendisk *, fmode_t,
                         struct scsi_ioctl_command __user *);
 
+extern void blk_init_request_list(struct request_list *rl);
+
+static inline struct request_list *blk_get_request_list(struct request_queue *q,
+                                               struct bio *bio)
+{
+#ifdef CONFIG_GROUP_IOSCHED
+       return io_group_get_request_list(q, bio);
+#else
+       return &q->rq;
+#endif
+}
+
+static inline struct request_list *rq_rl(struct request_queue *q,
+                                               struct request *rq)
+{
+#ifdef CONFIG_GROUP_IOSCHED
+       return rq->rl;
+#else
+       return blk_get_request_list(q, NULL);
+#endif
+}
+
  /*
  * Temporary export, until SCSI gets fixed up.
  */

Index: linux9/block/elevator.c
===================================================================

block/elevator.c     2009-04-30 16:18:29.000000000 -0400 renamed from block/elevator.c        2009-04-30 16:17:53.000000000 -0400 +3 -3
@@ -664,7 +664,7 @@ void elv_quiesce_start(struct request_qu
         * make sure we don't have any requests in flight
         */
        elv_drain_elevator(q);
-       while (q->rq.elvpriv) {
+       while (q->rq_data.elvpriv) {
                blk_start_queueing(q);
                spin_unlock_irq(q->queue_lock);
                msleep(10);
@@ -764,8 +764,8 @@ void elv_insert(struct request_queue *q,
        }
 
        if (unplug_it && blk_queue_plugged(q)) {
-               int nrq = q->rq.count[BLK_RW_SYNC] + q->rq.count[BLK_RW_ASYNC]
-                       - q->in_flight;
+               int nrq = q->rq_data.count[BLK_RW_SYNC] +
+                               q->rq_data.count[BLK_RW_ASYNC] - q->in_flight;
 
                if (nrq >= q->unplug_thresh)
                        __generic_unplug_device(q);

Index: linux9/block/blk-core.c
===================================================================

block/blk-core.c     2009-04-30 16:18:29.000000000 -0400 renamed from block/blk-core.c        2009-04-30 16:17:53.000000000 -0400 +22 -8
@@ -480,20 +480,31 @@ void blk_cleanup_queue(struct request_qu
  }
  EXPORT_SYMBOL(blk_cleanup_queue);
 
-static int blk_init_free_list(struct request_queue *q)
+void blk_init_request_list(struct request_list *rl)
  {
-       struct request_list *rl = &q->rq;
 
        rl->count[BLK_RW_SYNC] = rl->count[BLK_RW_ASYNC] = 0;
-       rl->starved[BLK_RW_SYNC] = rl->starved[BLK_RW_ASYNC] = 0;
-       rl->elvpriv = 0;
        init_waitqueue_head(&rl->wait[BLK_RW_SYNC]);
        init_waitqueue_head(&rl->wait[BLK_RW_ASYNC]);
+}
 
-       rl->rq_pool = mempool_create_node(BLKDEV_MIN_RQ, mempool_alloc_slab,
-                               mempool_free_slab, request_cachep, q->node);
+static int blk_init_free_list(struct request_queue *q)
+{
+#ifndef CONFIG_GROUP_IOSCHED
+       struct request_list *rl = blk_get_request_list(q, NULL);
+
+       /*
+        * In case of group scheduling, request list is inside the associated
+        * group and when that group is instanciated, it takes care of
+        * initializing the request list also.
+        */
+       blk_init_request_list(rl);
+#endif
+       q->rq_data.rq_pool = mempool_create_node(BLKDEV_MIN_RQ,
+                               mempool_alloc_slab, mempool_free_slab,
+                               request_cachep, q->node);
 
-       if (!rl->rq_pool)
+       if (!q->rq_data.rq_pool)
                return -ENOMEM;
 
        return 0;
@@ -590,6 +601,9 @@ blk_init_queue_node(request_fn_proc *rfn
                return NULL;
        }
 
+       /* init starved waiter wait queue */
+       init_waitqueue_head(&q->rq_data.starved_wait);
+
        /*
         * if caller didn't supply a lock, they get per-queue locking with
         * our embedded lock
@@ -639,14 +653,14 @@ static inline void blk_free_request(stru
  {
        if (rq->cmd_flags & REQ_ELVPRIV)
                elv_put_request(q, rq);
-       mempool_free(rq, q->rq.rq_pool);
+       mempool_free(rq, q->rq_data.rq_pool);
  }
 

 static struct request *
 blk_alloc_request(struct request_queue *q, struct bio *bio, int rw, int priv, gfp_t gfp_mask)
 {
-       struct request *rq = mempool_alloc(q->rq.rq_pool, gfp_mask);
+       struct request *rq = mempool_alloc(q->rq_data.rq_pool, gfp_mask);

       if (!rq)
               return NULL;

block/blk-core.c     2009-04-30 16:18:29.000000000 -0400 renamed from block/blk-core.c        2009-04-30 16:17:53.000000000 -0400 +81 -42
@@ -657,7 +671,7 @@ blk_alloc_request(struct request_queue *
 
        if (priv) {
                if (unlikely(elv_set_request(q, rq, bio, gfp_mask))) {
-                       mempool_free(rq, q->rq.rq_pool);
+                       mempool_free(rq, q->rq_data.rq_pool);
                        return NULL;
                }
                rq->cmd_flags |= REQ_ELVPRIV;
@@ -700,18 +714,18 @@ static void ioc_set_batching(struct requ
        ioc->last_waited = jiffies;
  }
 
-static void __freed_request(struct request_queue *q, int sync)
+static void __freed_request(struct request_queue *q, int sync,
+                                       struct request_list *rl)
  {
-       struct request_list *rl = &q->rq;
-
-       if (rl->count[sync] < queue_congestion_off_threshold(q))
+       if (q->rq_data.count[sync] < queue_congestion_off_threshold(q))
                blk_clear_queue_congested(q, sync);
 
-       if (rl->count[sync] + 1 <= q->nr_requests) {
+       if (q->rq_data.count[sync] + 1 <= q->nr_requests)
+               blk_clear_queue_full(q, sync);
+
+       if (rl->count[sync] + 1 <= q->nr_group_requests) {
                if (waitqueue_active(&rl->wait[sync]))
                        wake_up(&rl->wait[sync]);
-
-               blk_clear_queue_full(q, sync);
        }
  }
 
@@ -719,18 +733,29 @@ static void __freed_request(struct reque
  * A request has just been released.  Account for it, update the full and
  * congestion status, wake up any waiters.   Called under q->queue_lock.
  */
-static void freed_request(struct request_queue *q, int sync, int priv)
+static void freed_request(struct request_queue *q, int sync, int priv,
+                                       struct request_list *rl)
  {
-       struct request_list *rl = &q->rq;
-
+       BUG_ON(!rl->count[sync]);
        rl->count[sync]--;
+
+       BUG_ON(!q->rq_data.count[sync]);
+       q->rq_data.count[sync]--;
+
        if (priv)
-               rl->elvpriv--;
+               q->rq_data.elvpriv--;
 
-       __freed_request(q, sync);
+       __freed_request(q, sync, rl);
 
        if (unlikely(rl->starved[sync ^ 1]))
-               __freed_request(q, sync ^ 1);
+               __freed_request(q, sync ^ 1, rl);
+
+       /* Wake up the starved process on global list, if any */
+       if (unlikely(q->rq_data.starved)) {
+               if (waitqueue_active(&q->rq_data.starved_wait))
+                       wake_up(&q->rq_data.starved_wait);
+               q->rq_data.starved--;
+       }
  }
 
  /*
@@ -739,10 +764,9 @@ static void freed_request(struct request
  * Returns !NULL on success, with queue_lock *not held*.
  */
  static struct request *get_request(struct request_queue *q, int rw_flags,
-                                  struct bio *bio, gfp_t gfp_mask)
+                  struct bio *bio, gfp_t gfp_mask, struct request_list *rl)
  {
        struct request *rq = NULL;
-       struct request_list *rl = &q->rq;
        struct io_context *ioc = NULL;
        const bool is_sync = rw_is_sync(rw_flags) != 0;
        int may_queue, priv;
@@ -751,31 +775,38 @@ static struct request *get_request(struc
        if (may_queue == ELV_MQUEUE_NO)
                goto rq_starved;
 
-       if (rl->count[is_sync]+1 >= queue_congestion_on_threshold(q)) {
-               if (rl->count[is_sync]+1 >= q->nr_requests) {
-                       ioc = current_io_context(GFP_ATOMIC, q->node);
-                       /*
-                        * The queue will fill after this allocation, so set
-                        * it as full, and mark this process as "batching".
-                        * This process will be allowed to complete a batch of
-                        * requests, others will be blocked.
-                        */
-                       if (!blk_queue_full(q, is_sync)) {
-                               ioc_set_batching(q, ioc);
-                               blk_set_queue_full(q, is_sync);
-                       } else {
-                               if (may_queue != ELV_MQUEUE_MUST
-                                               && !ioc_batching(q, ioc)) {
-                                       /*
-                                        * The queue is full and the allocating
-                                        * process is not a "batcher", and not
-                                        * exempted by the IO scheduler
-                                        */
-                                       goto out;
-                               }
+       if (q->rq_data.count[is_sync]+1 >= queue_congestion_on_threshold(q))
+               blk_set_queue_congested(q, is_sync);
+
+       /*
+        * Looks like there is no user of queue full now.
+        * Keeping it for time being.
+        */
+       if (q->rq_data.count[is_sync]+1 >= q->nr_requests)
+               blk_set_queue_full(q, is_sync);
+
+       if (rl->count[is_sync]+1 >= q->nr_group_requests) {
+               ioc = current_io_context(GFP_ATOMIC, q->node);
+               /*
+                * The queue request descriptor group will fill after this
+                * allocation, so set
+                * it as full, and mark this process as "batching".
+                * This process will be allowed to complete a batch of
+                * requests, others will be blocked.
+                */
+               if (rl->count[is_sync] <= q->nr_group_requests)
+                       ioc_set_batching(q, ioc);
+               else {
+                       if (may_queue != ELV_MQUEUE_MUST
+                                       && !ioc_batching(q, ioc)) {
+                               /*
+                                * The queue is full and the allocating
+                                * process is not a "batcher", and not
+                                * exempted by the IO scheduler
+                                */
+                               goto out;
                        }
                }
-               blk_set_queue_congested(q, is_sync);
        }
 
        /*
@@ -783,19 +814,41 @@ static struct request *get_request(struc
         * limit of requests, otherwise we could have thousands of requests
         * allocated with any setting of ->nr_requests
         */
-       if (rl->count[is_sync] >= (3 * q->nr_requests / 2))
+
+       if (q->rq_data.count[is_sync] >= (3 * q->nr_requests / 2))
+               goto out;
+
+       /*
+        * Allocation of request is allowed from queue perspective. Now check
+        * from per group request list
+        */
+
+       if (rl->count[is_sync] >= (3 * q->nr_group_requests / 2))
                goto out;
 
        rl->count[is_sync]++;
        rl->starved[is_sync] = 0;
 
+       q->rq_data.count[is_sync]++;
+
        priv = !test_bit(QUEUE_FLAG_ELVSWITCH, &q->queue_flags);
        if (priv)
-               rl->elvpriv++;
+               q->rq_data.elvpriv++;
 
        spin_unlock_irq(q->queue_lock);
 
        rq = blk_alloc_request(q, bio, rw_flags, priv, gfp_mask);
+
+#ifdef CONFIG_GROUP_IOSCHED
+       if (rq) {
+               /*
+                * TODO. Implement group reference counting and take the
+                * reference to the group to make sure group hence request
+                * list does not go away till rq finishes.
+                */
+               rq->rl = rl;
+       }
+#endif
        if (unlikely(!rq)) {
                /*
                 * Allocation failed presumably due to memory. Undo anything
@@ -805,7 +858,7 @@ static struct request *get_request(struc
                 * wait queue, but this is pretty rare.
                 */
                spin_lock_irq(q->queue_lock);
-               freed_request(q, is_sync, priv);
+               freed_request(q, is_sync, priv, rl);
 
                /*
                 * in the very unlikely event that allocation failed and no
@@ -815,10 +868,26 @@ static struct request *get_request(struc
                 * rq mempool into READ and WRITE
                 */

 rq_starved:
-               if (unlikely(rl->count[is_sync] == 0))
-                       rl->starved[is_sync] = 1;
-
-               goto out;
+               if (unlikely(rl->count[is_sync] == 0)) {
+                       /*
+                        * If there is a request pending in other direction
+                        * in same io group, then set the starved flag of
+                        * the group request list. Otherwise, we need to
+                        * make this process sleep in global starved list
+                        * to make sure it will not sleep indefinitely.
+                        */
+                       if (rl->count[is_sync ^ 1] != 0) {
+                               rl->starved[is_sync] = 1;
+                               goto out;
+                       } else {
+                               /*
+                                * It indicates to calling function to put
+                                * task on global starved list. Not the best
+                                * way
+                                */
+                               return ERR_PTR(-ENOMEM);
+                       }
+               }
       }

       /*

block/blk-core.c     2009-04-30 16:18:29.000000000 -0400 renamed from block/blk-core.c        2009-04-30 16:17:53.000000000 -0400 +29 -8
@@ -846,15 +915,29 @@ static struct request *get_request_wait(
  {
        const bool is_sync = rw_is_sync(rw_flags) != 0;
        struct request *rq;
+       struct request_list *rl = blk_get_request_list(q, bio);
 
-       rq = get_request(q, rw_flags, bio, GFP_NOIO);
-       while (!rq) {
+       rq = get_request(q, rw_flags, bio, GFP_NOIO, rl);
+       while (!rq || (IS_ERR(rq) && PTR_ERR(rq) == -ENOMEM)) {
                DEFINE_WAIT(wait);
                struct io_context *ioc;
-               struct request_list *rl = &q->rq;
 
-               prepare_to_wait_exclusive(&rl->wait[is_sync], &wait,
-                               TASK_UNINTERRUPTIBLE);
+               if (IS_ERR(rq) && PTR_ERR(rq) == -ENOMEM) {
+                       /*
+                        * Task failed allocation and needs to wait and
+                        * try again. There are no requests pending from
+                        * the io group hence need to sleep on global
+                        * wait queue. Most likely the allocation failed
+                        * because of memory issues.
+                        */
+
+                       q->rq_data.starved++;
+                       prepare_to_wait_exclusive(&q->rq_data.starved_wait,
+                                       &wait, TASK_UNINTERRUPTIBLE);
+               } else {
+                       prepare_to_wait_exclusive(&rl->wait[is_sync], &wait,
+                                       TASK_UNINTERRUPTIBLE);
+               }
 
                trace_block_sleeprq(q, bio, rw_flags & 1);
 
@@ -874,7 +957,12 @@ static struct request *get_request_wait(
                spin_lock_irq(q->queue_lock);
                finish_wait(&rl->wait[is_sync], &wait);
 
-               rq = get_request(q, rw_flags, bio, GFP_NOIO);
+               /*
+                * After the sleep check the rl again in case cgrop bio
+                * belonged to is gone and it is mapped to root group now
+                */
+               rl = blk_get_request_list(q, bio);
+               rq = get_request(q, rw_flags, bio, GFP_NOIO, rl);
        };
 
        return rq;
@@ -883,6 +971,7 @@ static struct request *get_request_wait(
  struct request *blk_get_request(struct request_queue *q, int rw, gfp_t gfp_mask)
  {
        struct request *rq;
+       struct request_list *rl = blk_get_request_list(q, NULL);
 
        BUG_ON(rw != READ && rw != WRITE);
 
@@ -890,7 +979,7 @@ struct request *blk_get_request(struct r
        if (gfp_mask & __GFP_WAIT) {
                rq = get_request_wait(q, rw, NULL);
        } else {
-               rq = get_request(q, rw, NULL, gfp_mask);
+               rq = get_request(q, rw, NULL, gfp_mask, rl);
                if (!rq)
                        spin_unlock_irq(q->queue_lock);
        }
@@ -1073,12 +1162,13 @@ void __blk_put_request(struct request_qu
        if (req->cmd_flags & REQ_ALLOCED) {
                int is_sync = rq_is_sync(req) != 0;
                int priv = req->cmd_flags & REQ_ELVPRIV;
+               struct request_list *rl = rq_rl(q, req);
 
                BUG_ON(!list_empty(&req->queuelist));
                BUG_ON(!hlist_unhashed(&req->hash));
 
                blk_free_request(q, req);
-               freed_request(q, is_sync, priv);
+               freed_request(q, is_sync, priv, rl);
        }
  }
  EXPORT_SYMBOL_GPL(__blk_put_request);

Index: linux9/block/blk-sysfs.c
===================================================================

block/blk-sysfs.c    2009-04-30 16:18:29.000000000 -0400 renamed from block/blk-sysfs.c       2009-04-30 16:18:27.000000000 -0400 +0 -0
@@ -38,7 +38,7 @@ static ssize_t queue_requests_show(struc

 static ssize_t
 queue_requests_store(struct request_queue *q, const char *page, size_t count) {
-       struct request_list *rl = &q->rq;
+       struct request_list *rl = blk_get_request_list(q, NULL); unsigned long nr;
       int ret = queue_var_store(&nr, page, count);
       if (nr < BLKDEV_MIN_RQ)

block/blk-sysfs.c    2009-04-30 16:18:29.000000000 -0400 renamed from block/blk-sysfs.c       2009-04-30 16:18:27.000000000 -0400 +42 -8
@@ -48,32 +48,55 @@ queue_requests_store(struct request_queu
        q->nr_requests = nr;
        blk_queue_congestion_threshold(q);
 
-       if (rl->count[BLK_RW_SYNC] >= queue_congestion_on_threshold(q))
+       if (q->rq_data.count[BLK_RW_SYNC] >= queue_congestion_on_threshold(q))
                blk_set_queue_congested(q, BLK_RW_SYNC);
-       else if (rl->count[BLK_RW_SYNC] < queue_congestion_off_threshold(q))
+       else if (q->rq_data.count[BLK_RW_SYNC] <
+                               queue_congestion_off_threshold(q))
                blk_clear_queue_congested(q, BLK_RW_SYNC);
 
-       if (rl->count[BLK_RW_ASYNC] >= queue_congestion_on_threshold(q))
+       if (q->rq_data.count[BLK_RW_ASYNC] >= queue_congestion_on_threshold(q))
                blk_set_queue_congested(q, BLK_RW_ASYNC);
-       else if (rl->count[BLK_RW_ASYNC] < queue_congestion_off_threshold(q))
+       else if (q->rq_data.count[BLK_RW_ASYNC] <
+                               queue_congestion_off_threshold(q))
                blk_clear_queue_congested(q, BLK_RW_ASYNC);
 
-       if (rl->count[BLK_RW_SYNC] >= q->nr_requests) {
+       if (q->rq_data.count[BLK_RW_SYNC] >= q->nr_requests) {
                blk_set_queue_full(q, BLK_RW_SYNC);
-       } else if (rl->count[BLK_RW_SYNC]+1 <= q->nr_requests) {
+       } else if (q->rq_data.count[BLK_RW_SYNC]+1 <= q->nr_requests) {
                blk_clear_queue_full(q, BLK_RW_SYNC);
                wake_up(&rl->wait[BLK_RW_SYNC]);
        }
 
-       if (rl->count[BLK_RW_ASYNC] >= q->nr_requests) {
+       if (q->rq_data.count[BLK_RW_ASYNC] >= q->nr_requests) {
                blk_set_queue_full(q, BLK_RW_ASYNC);
-       } else if (rl->count[BLK_RW_ASYNC]+1 <= q->nr_requests) {
+       } else if (q->rq_data.count[BLK_RW_ASYNC]+1 <= q->nr_requests) {
                blk_clear_queue_full(q, BLK_RW_ASYNC);
                wake_up(&rl->wait[BLK_RW_ASYNC]);
        }
        spin_unlock_irq(q->queue_lock);
        return ret;
  }
+#ifdef CONFIG_GROUP_IOSCHED
+static ssize_t queue_group_requests_show(struct request_queue *q, char *page)
+{
+       return queue_var_show(q->nr_group_requests, (page));
+}
+
+static ssize_t
+queue_group_requests_store(struct request_queue *q, const char *page,
+                                       size_t count)
+{
+       unsigned long nr;
+       int ret = queue_var_store(&nr, page, count);
+       if (nr < BLKDEV_MIN_RQ)
+               nr = BLKDEV_MIN_RQ;
+
+       spin_lock_irq(q->queue_lock);
+       q->nr_group_requests = nr;
+       spin_unlock_irq(q->queue_lock);
+       return ret;
+}
+#endif
 
  static ssize_t queue_ra_show(struct request_queue *q, char *page)
  {
@@ -228,6 +251,14 @@ static struct queue_sysfs_entry queue_re
        .store = queue_requests_store,
  };
 
+#ifdef CONFIG_GROUP_IOSCHED
+static struct queue_sysfs_entry queue_group_requests_entry = {
+       .attr = {.name = "nr_group_requests", .mode = S_IRUGO | S_IWUSR },
+       .show = queue_group_requests_show,
+       .store = queue_group_requests_store,
+};
+#endif
+
  static struct queue_sysfs_entry queue_ra_entry = {
        .attr = {.name = "read_ahead_kb", .mode = S_IRUGO | S_IWUSR },
        .show = queue_ra_show,
@@ -308,6 +339,9 @@ static struct queue_sysfs_entry queue_sl
 
  static struct attribute *default_attrs[] = {
        &queue_requests_entry.attr,
+#ifdef CONFIG_GROUP_IOSCHED
+       &queue_group_requests_entry.attr,
+#endif
        &queue_ra_entry.attr,
        &queue_max_hw_sectors_entry.attr,
        &queue_max_sectors_entry.attr,
@@ -389,12 +423,11 @@ static void blk_release_queue(struct kob
  {

       struct request_queue *q =
               container_of(kobj, struct request_queue, kobj);
-       struct request_list *rl = &q->rq;

       blk_sync_queue(q);

-       if (rl->rq_pool)
-               mempool_destroy(rl->rq_pool);
+       if (q->rq_data.rq_pool)
+               mempool_destroy(q->rq_data.rq_pool);

       if (q->queue_tags)
               __blk_queue_free_tags(q);
Index: linux9/block/blk-settings.c
===================================================================

block/blk-settings.c 2009-04-30 16:18:29.000000000 -0400 renamed from block/blk-settings.c    2009-04-30 15:43:53.000000000 -0400 +3 -0
@@ -123,6 +123,9 @@ void blk_queue_make_request(struct reque
         * set defaults
         */
        q->nr_requests = BLKDEV_MAX_RQ;
+#ifdef CONFIG_GROUP_IOSCHED
+       q->nr_group_requests = BLKDEV_MAX_GROUP_RQ;
+#endif
        blk_queue_max_phys_segments(q, MAX_PHYS_SEGMENTS);
        blk_queue_max_hw_segments(q, MAX_HW_SEGMENTS);
        blk_queue_segment_boundary(q, BLK_SEG_BOUNDARY_MASK);

Index: linux9/block/elevator-fq.c
===================================================================

block/elevator-fq.c  2009-04-30 16:18:29.000000000 -0400 renamed from block/elevator-fq.c     2009-04-30 16:18:27.000000000 -0400 +15 -0
@@ -954,6 +954,17 @@ struct io_cgroup *cgroup_to_io_cgroup(st
                            struct io_cgroup, css);
  }
 
+struct request_list *io_group_get_request_list(struct request_queue *q,
+                                               struct bio *bio)
+{
+       struct io_group *iog;
+
+       iog = io_get_io_group_bio(q, bio, 1);
+       BUG_ON(!iog);
+out:
+       return &iog->rl;
+}
+
  /*
  * Search the bfq_group for bfqd into the hash table (by now only a list)
  * of bgrp.  Must be called under rcu_read_lock().
@@ -1203,6 +1214,8 @@ struct io_group *io_group_chain_alloc(st
                io_group_init_entity(iocg, iog);
                iog->my_entity = &iog->entity;
 
+               blk_init_request_list(&iog->rl);
+
                if (leaf == NULL) {
                        leaf = iog;
                        prev = leaf;
@@ -1446,6 +1459,8 @@ struct io_group *io_alloc_root_group(str
        for (i = 0; i < IO_IOPRIO_CLASSES; i++)
                iog->sched_data.service_tree[i] = IO_SERVICE_TREE_INIT;
 
+       blk_init_request_list(&iog->rl);
+
        iocg = &io_root_cgroup;
        spin_lock_irq(&iocg->lock);
        rcu_assign_pointer(iog->key, key);

Index: linux9/block/elevator-fq.h
===================================================================

block/elevator-fq.h  2009-04-30 16:18:29.000000000 -0400 renamed from block/elevator-fq.h     2009-04-30 16:18:27.000000000 -0400 +8 -0
@@ -239,8 +239,14 @@ struct io_group {
 
        /* Single ioq per group, used for noop, deadline, anticipatory */
        struct io_queue *ioq;
+
+       /* request list associated with the group */
+       struct request_list rl;
  };
 
+#define IOG_FLAG_READFULL      1       /* read queue has been filled */
+#define IOG_FLAG_WRITEFULL     2       /* write queue has been filled */
+
  /**
  * struct bfqio_cgroup - bfq cgroup data structure.
  * @css: subsystem state for bfq in the containing cgroup.
@@ -517,6 +523,8 @@ extern void elv_fq_unset_request_ioq(str
  extern struct io_queue *elv_lookup_ioq_current(struct request_queue *q);
  extern struct io_queue *elv_lookup_ioq_bio(struct request_queue *q,
                                                struct bio *bio);
+extern struct request_list *io_group_get_request_list(struct request_queue *q,
+                                               struct bio *bio);
 
  /* Returns single ioq associated with the io group. */
  static inline struct io_queue *io_group_ioq(struct io_group *iog)

Thanks
Vivek

Quoted 619 lines

---
block/blk-core.c    |   36 +++++++--
block/blk-sysfs.c   |   22 ++++--
block/elevator-fq.c |  133 ++++++++++++++++++++++++++++++++-- block/elevator-fq.h |  201 +++++++++++++++++++++++++++++++++++++++++++++++++++

4 files changed, 371 insertions(+), 21 deletions(-)
block/blk-core.c +28 -8
index 29bcfac..21023f7 100644
@@ -705,11 +705,15 @@ static void ioc_set_batching(struct request_queue *q, struct io_context *ioc)
 static void __freed_request(struct request_queue *q, int rw)
 {
       struct request_list *rl = &q->rq;
-
-     if (rl->count[rw] < queue_congestion_off_threshold(q))
+     struct io_group *congested_iog, *full_iog;
+
+     congested_iog = io_congested_io_group(q, rw);
+     if (rl->count[rw] < queue_congestion_off_threshold(q) &&
+         !congested_iog)
               blk_clear_queue_congested(q, rw);
 
-     if (rl->count[rw] + 1 <= q->nr_requests) {
+     full_iog = io_full_io_group(q, rw);
+     if (rl->count[rw] + 1 <= q->nr_requests && !full_iog) {
               if (waitqueue_active(&rl->wait[rw]))
                       wake_up(&rl->wait[rw]);
 
@@ -721,13 +725,16 @@ static void __freed_request(struct request_queue *q, int rw)
  * A request has just been released.  Account for it, update the full and
  * congestion status, wake up any waiters.   Called under q->queue_lock.
  */
-static void freed_request(struct request_queue *q, int rw, int priv)
+static void freed_request(struct request_queue *q, struct io_group *iog,
+                       int rw, int priv)
 {
       struct request_list *rl = &q->rq;
 
       rl->count[rw]--;
       if (priv)
               rl->elvpriv--;
+     if (iog)
+             io_group_dec_count(iog, rw);
 
       __freed_request(q, rw);
 
@@ -746,16 +753,21 @@ static struct request *get_request(struct request_queue *q, int rw_flags,
 {
       struct request *rq = NULL;
       struct request_list *rl = &q->rq;
+     struct io_group *iog;
       struct io_context *ioc = NULL;
       const int rw = rw_flags & 0x01;
       int may_queue, priv;
 
+     iog = __io_get_io_group(q);
+
       may_queue = elv_may_queue(q, rw_flags);
       if (may_queue == ELV_MQUEUE_NO)
               goto rq_starved;
 
-     if (rl->count[rw]+1 >= queue_congestion_on_threshold(q)) {
-             if (rl->count[rw]+1 >= q->nr_requests) {
+     if (rl->count[rw]+1 >= queue_congestion_on_threshold(q) ||
+         io_group_congestion_on(iog, rw)) {
+             if (rl->count[rw]+1 >= q->nr_requests ||
+                 io_group_full(iog, rw)) {
                       ioc = current_io_context(GFP_ATOMIC, q->node);
                       /*
                        * The queue will fill after this allocation, so set
@@ -789,8 +801,15 @@ static struct request *get_request(struct request_queue *q, int rw_flags,
       if (rl->count[rw] >= (3 * q->nr_requests / 2))
               goto out;
 
+     if (iog)
+             if (io_group_count(iog, rw) >=
+                (3 * io_group_nr_requests(iog) / 2))
+                     goto out;
+
       rl->count[rw]++;
       rl->starved[rw] = 0;
+     if (iog)
+             io_group_inc_count(iog, rw);
 
       priv = !test_bit(QUEUE_FLAG_ELVSWITCH, &q->queue_flags);
       if (priv)
@@ -808,7 +827,7 @@ static struct request *get_request(struct request_queue *q, int rw_flags,
                * wait queue, but this is pretty rare.
                */
               spin_lock_irq(q->queue_lock);
-             freed_request(q, rw, priv);
+             freed_request(q, iog, rw, priv);
 
               /*
                * in the very unlikely event that allocation failed and no
@@ -1073,12 +1092,13 @@ void __blk_put_request(struct request_queue *q, struct request *req)
       if (req->cmd_flags & REQ_ALLOCED) {
               int rw = rq_data_dir(req);
               int priv = req->cmd_flags & REQ_ELVPRIV;
+             struct io_group *iog = io_request_io_group(req);
 
               BUG_ON(!list_empty(&req->queuelist));
               BUG_ON(!hlist_unhashed(&req->hash));
 
               blk_free_request(q, req);
-             freed_request(q, rw, priv);
+             freed_request(q, iog, rw, priv);
       }
 }
 EXPORT_SYMBOL_GPL(__blk_put_request);
block/blk-sysfs.c +14 -8
index 0d98c96..af5191c 100644
@@ -40,6 +40,7 @@ queue_requests_store(struct request_queue *q, const char *page, size_t count)
 {
       struct request_list *rl = &q->rq;
       unsigned long nr;
+     int iog_congested[2], iog_full[2];
       int ret = queue_var_store(&nr, page, count);
       if (nr < BLKDEV_MIN_RQ)
               nr = BLKDEV_MIN_RQ;
@@ -47,27 +48,32 @@ queue_requests_store(struct request_queue *q, const char *page, size_t count)
       spin_lock_irq(q->queue_lock);
       q->nr_requests = nr;
       blk_queue_congestion_threshold(q);
+     io_group_set_nrq_all(q, nr, iog_congested, iog_full);
 
-     if (rl->count[READ] >= queue_congestion_on_threshold(q))
+     if (rl->count[READ] >= queue_congestion_on_threshold(q) ||
+         iog_congested[READ])
               blk_set_queue_congested(q, READ);
-     else if (rl->count[READ] < queue_congestion_off_threshold(q))
+     else if (rl->count[READ] < queue_congestion_off_threshold(q) &&
+              !iog_congested[READ])
               blk_clear_queue_congested(q, READ);
 
-     if (rl->count[WRITE] >= queue_congestion_on_threshold(q))
+     if (rl->count[WRITE] >= queue_congestion_on_threshold(q) ||
+         iog_congested[WRITE])
               blk_set_queue_congested(q, WRITE);
-     else if (rl->count[WRITE] < queue_congestion_off_threshold(q))
+     else if (rl->count[WRITE] < queue_congestion_off_threshold(q) &&
+              !iog_congested[WRITE])
               blk_clear_queue_congested(q, WRITE);
 
-     if (rl->count[READ] >= q->nr_requests) {
+     if (rl->count[READ] >= q->nr_requests || iog_full[READ]) {
               blk_set_queue_full(q, READ);
-     } else if (rl->count[READ]+1 <= q->nr_requests) {
+     } else if (rl->count[READ]+1 <= q->nr_requests && !iog_full[READ]) {
               blk_clear_queue_full(q, READ);
               wake_up(&rl->wait[READ]);
       }
 
-     if (rl->count[WRITE] >= q->nr_requests) {
+     if (rl->count[WRITE] >= q->nr_requests || iog_full[WRITE]) {
               blk_set_queue_full(q, WRITE);
-     } else if (rl->count[WRITE]+1 <= q->nr_requests) {
+     } else if (rl->count[WRITE]+1 <= q->nr_requests && !iog_full[WRITE]) {
               blk_clear_queue_full(q, WRITE);
               wake_up(&rl->wait[WRITE]);
       }
block/elevator-fq.c +128 -5
index df53418..3b021f3 100644
@@ -924,6 +924,111 @@ struct io_group *io_lookup_io_group_current(struct request_queue *q)
 }
 EXPORT_SYMBOL(io_lookup_io_group_current);
 
+/*
+ * TODO
+ * This is complete dupulication of blk_queue_congestion_threshold()
+ * except for the argument type and name.  Can we merge them?
+ */
+static void io_group_nrq_congestion_threshold(struct io_group_nrq *nrq)
+{
+     int nr;
+
+     nr = nrq->nr_requests - (nrq->nr_requests / 8) + 1;
+     if (nr > nrq->nr_requests)
+             nr = nrq->nr_requests;
+     nrq->nr_congestion_on = nr;
+
+     nr = nrq->nr_requests - (nrq->nr_requests / 8)
+             - (nrq->nr_requests / 16) - 1;
+     if (nr < 1)
+             nr = 1;
+     nrq->nr_congestion_off = nr;
+}
+
+static void io_group_set_nrq(struct io_group_nrq *nrq, int nr_requests,
+                      int *congested, int *full)
+{
+     int i;
+
+     BUG_ON(nr_requests < 0);
+
+     nrq->nr_requests = nr_requests;
+     io_group_nrq_congestion_threshold(nrq);
+
+     for (i=0; i<2; i++) {
+             if (nrq->count[i] >= nrq->nr_congestion_on)
+                     congested[i] = 1;
+             else if (nrq->count[i] < nrq->nr_congestion_off)
+                     congested[i] = 0;
+
+             if (nrq->count[i] >= nrq->nr_requests)
+                     full[i] = 1;
+             else if (nrq->count[i]+1 <= nrq->nr_requests)
+                     full[i] = 0;
+     }
+}
+
+void io_group_set_nrq_all(struct request_queue *q, int nr,
+                         int *congested, int *full)
+{
+     struct elv_fq_data *efqd = &q->elevator->efqd;
+     struct hlist_head *head = &efqd->group_list;
+     struct io_group *root = efqd->root_group;
+     struct hlist_node *n;
+     struct io_group *iog;
+     struct io_group_nrq *nrq;
+     int nrq_congested[2];
+     int nrq_full[2];
+     int i;
+
+     for (i=0; i<2; i++)
+             *(congested + i) = *(full + i) = 0;
+
+     nrq = &root->nrq;
+     io_group_set_nrq(nrq, nr, nrq_congested, nrq_full);
+     for (i=0; i<2; i++) {
+             *(congested + i) |= nrq_congested[i];
+             *(full + i) |= nrq_full[i];
+     }
+
+     hlist_for_each_entry(iog, n, head, elv_data_node) {
+             nrq = &iog->nrq;
+             io_group_set_nrq(nrq, nr, nrq_congested, nrq_full);
+             for (i=0; i<2; i++) {
+                     *(congested + i) |= nrq_congested[i];
+                     *(full + i) |= nrq_full[i];
+             }
+     }
+}
+
+struct io_group *io_congested_io_group(struct request_queue *q, int rw)
+{
+     struct hlist_head *head = &q->elevator->efqd.group_list;
+     struct hlist_node *n;
+     struct io_group *iog;
+
+     hlist_for_each_entry(iog, n, head, elv_data_node) {
+             struct io_group_nrq *nrq = &iog->nrq;
+             if (nrq->count[rw] >= nrq->nr_congestion_off)
+                     return iog;
+     }
+     return NULL;
+}
+
+struct io_group *io_full_io_group(struct request_queue *q, int rw)
+{
+     struct hlist_head *head = &q->elevator->efqd.group_list;
+     struct hlist_node *n;
+     struct io_group *iog;
+
+     hlist_for_each_entry(iog, n, head, elv_data_node) {
+             struct io_group_nrq *nrq = &iog->nrq;
+             if (nrq->count[rw] >= nrq->nr_requests)
+                     return iog;
+     }
+     return NULL;
+}
+
 void io_group_init_entity(struct io_cgroup *iocg, struct io_group *iog)
 {
       struct io_entity *entity = &iog->entity;
@@ -934,6 +1039,12 @@ void io_group_init_entity(struct io_cgroup *iocg, struct io_group *iog)
       entity->my_sched_data = &iog->sched_data;
 }
 
+static void io_group_init_nrq(struct request_queue *q, struct io_group_nrq *nrq)
+{
+     nrq->nr_requests = q->nr_requests;
+     io_group_nrq_congestion_threshold(nrq);
+}
+
 void io_group_set_parent(struct io_group *iog, struct io_group *parent)
 {
       struct io_entity *entity;
@@ -1053,6 +1164,8 @@ struct io_group *io_group_chain_alloc(struct request_queue *q, void *key,
               io_group_init_entity(iocg, iog);
               iog->my_entity = &iog->entity;
 
+             io_group_init_nrq(q, &iog->nrq);
+
               if (leaf == NULL) {
                       leaf = iog;
                       prev = leaf;
@@ -1176,7 +1289,7 @@ struct io_group *io_find_alloc_group(struct request_queue *q,
  * Generic function to make sure cgroup hierarchy is all setup once a request
  * from a cgroup is received by the io scheduler.
  */
-struct io_group *io_get_io_group(struct request_queue *q)
+struct io_group *__io_get_io_group(struct request_queue *q)
 {
       struct cgroup *cgroup;
       struct io_group *iog;
@@ -1192,6 +1305,19 @@ struct io_group *io_get_io_group(struct request_queue *q)
       return iog;
 }
 
+struct io_group *io_get_io_group(struct request_queue *q)
+{
+     struct io_group *iog;
+     unsigned long flags;
+
+     spin_lock_irqsave(q->queue_lock, flags);
+     iog = __io_get_io_group(q);
+     spin_unlock_irqrestore(q->queue_lock, flags);
+     BUG_ON(!iog);
+
+     return iog;
+}
+
 void io_free_root_group(struct elevator_queue *e)
 {
       struct io_cgroup *iocg = &io_root_cgroup;
@@ -1220,6 +1346,7 @@ struct io_group *io_alloc_root_group(struct request_queue *q,
       iog->entity.parent = NULL;
       for (i = 0; i < IO_IOPRIO_CLASSES; i++)
               iog->sched_data.service_tree[i] = IO_SERVICE_TREE_INIT;
+     io_group_init_nrq(q, &iog->nrq);
 
       iocg = &io_root_cgroup;
       spin_lock_irq(&iocg->lock);
@@ -1533,15 +1660,11 @@ void elv_fq_set_request_io_group(struct request_queue *q,
                                               struct request *rq)
 {
       struct io_group *iog;
-     unsigned long flags;
 
       /* Make sure io group hierarchy has been setup and also set the
        * io group to which rq belongs. Later we should make use of
        * bio cgroup patches to determine the io group */
-     spin_lock_irqsave(q->queue_lock, flags);
       iog = io_get_io_group(q);
-     spin_unlock_irqrestore(q->queue_lock, flags);
-     BUG_ON(!iog);
 
       /* Store iog in rq. TODO: take care of referencing */
       rq->iog = iog;
block/elevator-fq.h +201 -1
index fc4110d..f8eabd4 100644
@@ -187,6 +187,22 @@ struct io_queue {
 
 #ifdef CONFIG_GROUP_IOSCHED
 /**
+ * struct io_group_nrq - structure to store allocated requests info
+ * @nr_requests: maximun num of requests for the io_group
+ * @nr_congestion_on: threshold to determin the io_group is cogested.
+ * @nr_congestion_off: threshold to determin the io_group is not congested.
+ * @count: num of allocated requests.
+ *
+ * All fields are protected by queue_lock.
+ */
+struct io_group_nrq {
+     unsigned long nr_requests;
+     unsigned int nr_congestion_on;
+     unsigned int nr_congestion_off;
+     int count[2];
+};
+
+/**
  * struct bfq_group - per (device, cgroup) data structure.
  * @entity: schedulable entity to insert into the parent group sched_data.
  * @sched_data: own sched_data, to contain child entities (they may be
@@ -235,6 +251,8 @@ struct io_group {
 
       /* Single ioq per group, used for noop, deadline, anticipatory */
       struct io_queue *ioq;
+
+     struct io_group_nrq nrq;
 };
 
 /**
@@ -469,6 +487,11 @@ extern int elv_fq_set_request_ioq(struct request_queue *q, struct request *rq,
 extern void elv_fq_unset_request_ioq(struct request_queue *q,
                                       struct request *rq);
 extern struct io_queue *elv_lookup_ioq_current(struct request_queue *q);
+extern void io_group_set_nrq_all(struct request_queue *q, int nr,
+                         int *congested, int *full);
+extern struct io_group *io_congested_io_group(struct request_queue *q, int rw);
+extern struct io_group *io_full_io_group(struct request_queue *q, int rw);
+extern struct io_group *__io_get_io_group(struct request_queue *q);
 
 /* Returns single ioq associated with the io group. */
 static inline struct io_queue *io_group_ioq(struct io_group *iog)
@@ -486,6 +509,52 @@ static inline void io_group_set_ioq(struct io_group *iog, struct io_queue *ioq)
       iog->ioq = ioq;
 }
 
+static inline struct io_group *io_request_io_group(struct request *rq)
+{
+     return rq->iog;
+}
+
+static inline unsigned long io_group_nr_requests(struct io_group *iog)
+{
+     BUG_ON(!iog);
+     return iog->nrq.nr_requests;
+}
+
+static inline int io_group_inc_count(struct io_group *iog, int rw)
+{
+     BUG_ON(!iog);
+     return iog->nrq.count[rw]++;
+}
+
+static inline int io_group_dec_count(struct io_group *iog, int rw)
+{
+     BUG_ON(!iog);
+     return iog->nrq.count[rw]--;
+}
+
+static inline int io_group_count(struct io_group *iog, int rw)
+{
+     BUG_ON(!iog);
+     return iog->nrq.count[rw];
+}
+
+static inline int io_group_congestion_on(struct io_group *iog, int rw)
+{
+     BUG_ON(!iog);
+     return iog->nrq.count[rw] + 1 >= iog->nrq.nr_congestion_on;
+}
+
+static inline int io_group_congestion_off(struct io_group *iog, int rw)
+{
+     BUG_ON(!iog);
+     return iog->nrq.count[rw] < iog->nrq.nr_congestion_off;
+}
+
+static inline int io_group_full(struct io_group *iog, int rw)
+{
+     BUG_ON(!iog);
+     return iog->nrq.count[rw] + 1 >= iog->nrq.nr_requests;
+}
 #else /* !GROUP_IOSCHED */
 /*
  * No ioq movement is needed in case of flat setup. root io group gets cleaned
@@ -537,6 +606,71 @@ static inline struct io_queue *elv_lookup_ioq_current(struct request_queue *q)
       return NULL;
 }
 
+static inline void io_group_set_nrq_all(struct request_queue *q, int nr,
+                                     int *congested, int *full)
+{
+     int i;
+     for (i=0; i<2; i++)
+             *(congested + i) = *(full + i) = 0;
+}
+
+static inline struct io_group *
+io_congested_io_group(struct request_queue *q, int rw)
+{
+     return NULL;
+}
+
+static inline struct io_group *
+io_full_io_group(struct request_queue *q, int rw)
+{
+     return NULL;
+}
+
+static inline struct io_group *__io_get_io_group(struct request_queue *q)
+{
+     return NULL;
+}
+
+static inline struct io_group *io_request_io_group(struct request *rq)
+{
+     return NULL;
+}
+
+static inline unsigned long io_group_nr_requests(struct io_group *iog)
+{
+     return 0;
+}
+
+static inline int io_group_inc_count(struct io_group *iog, int rw)
+{
+     return 0;
+}
+
+static inline int io_group_dec_count(struct io_group *iog, int rw)
+{
+     return 0;
+}
+
+static inline int io_group_count(struct io_group *iog, int rw)
+{
+     return 0;
+}
+
+static inline int io_group_congestion_on(struct io_group *iog, int rw)
+{
+     return 0;
+}
+
+static inline int io_group_congestion_off(struct io_group *iog, int rw)
+{
+     return 1;
+}
+
+static inline int io_group_full(struct io_group *iog, int rw)
+{
+     return 0;
+}
+
 #endif /* GROUP_IOSCHED */
 
 /* Functions used by blksysfs.c */
@@ -589,6 +723,9 @@ extern void elv_free_ioq(struct io_queue *ioq);
 
 #else /* CONFIG_ELV_FAIR_QUEUING */
 
+struct io_group {
+};
+
 static inline int elv_init_fq_data(struct request_queue *q,
                                       struct elevator_queue *e)
 {
@@ -655,5 +792,69 @@ static inline struct io_queue *elv_lookup_ioq_current(struct request_queue *q)
       return NULL;
 }
 
+static inline void io_group_set_nrq_all(struct request_queue *q, int nr,
+                                     int *congested, int *full)
+{
+     int i;
+     for (i=0; i<2; i++)
+             *(congested + i) = *(full + i) = 0;
+}
+
+static inline struct io_group *
+io_congested_io_group(struct request_queue *q, int rw)
+{
+     return NULL;
+}
+
+static inline struct io_group *
+io_full_io_group(struct request_queue *q, int rw)
+{
+     return NULL;
+}
+
+static inline struct io_group *__io_get_io_group(struct request_queue *q)
+{
+     return NULL;
+}
+
+static inline struct io_group *io_request_io_group(struct request *rq)
+{
+     return NULL;
+}
+
+static inline unsigned long io_group_nr_requests(struct io_group *iog)
+{
+     return 0;
+}
+
+static inline int io_group_inc_count(struct io_group *iog, int rw)
+{
+     return 0;
+}
+
+static inline int io_group_dec_count(struct io_group *iog, int rw)
+{
+     return 0;
+}
+
+static inline int io_group_count(struct io_group *iog, int rw)
+{
+     return 0;
+}
+
+static inline int io_group_congestion_on(struct io_group *iog, int rw)
+{
+     return 0;
+}
+
+static inline int io_group_congestion_off(struct io_group *iog, int rw)
+{
+     return 1;
+}
+
+static inline int io_group_full(struct io_group *iog, int rw)
+{
+     return 0;
+}
 #endif /* CONFIG_ELV_FAIR_QUEUING */
 #endif /* _BFQ_SCHED_H */
--

1.5.4.3

--
IKEDA, Munehiro
NEC Corporation of America
  m-ikeda-MDRzhb/z0dd8UrSeD/[email protected]

#109 lore raw

Re: IO Controller per cgroup request descriptors (Re: [PATCH 01/10] Documentation)

In reply to Vivek Goyal
Quoted 39 lines · Vivek Goyal
Quoted 30 lines · Munehiro
Quoted 25 lines · Vivek Goyal
Quoted 7 lines
Quoted 6 lines

+TODO

+====
+- Lots of cleanups, testing, bug fixing, optimizations, benchmarking etc...
+- Convert cgroup ioprio to notion of weight.
+- Anticipatory code will need more work. It is not working properly currently

+  and needs more thought.

What are the problems with the code?

Have not got a chance to look into the issues in detail yet. Just a crude run saw drop in performance. Will debug it later the moment I have got async writes handled...

Quoted 6 lines
Quoted 1 line

+- Use of bio-cgroup patches.

I saw these posted as well

Quoted 2 lines

+- Use of Nauman's per cgroup request descriptor patches. +

More details would be nice, I am not sure I understand

Currently the number of request descriptors which can be allocated per device/request queue are fixed by a sysfs tunable (q->nr_requests). So if there is lots of IO going on from one cgroup then it will consume all the available request descriptors and other cgroup might starve and not get its fair share.

Quoted 3 lines

Hence we also need to introduce the notion of request descriptor limit per cgroup so that if request descriptors from one group are exhausted, then it does not impact the IO of other cgroup.

Quoted 7 lines

Unfortunately I couldn't find and I've never seen the Nauman's patches. So I tried to make a patch below against this todo.  The reason why I'm posting this despite this is just a quick and ugly hack (and it might be a reinvention of wheel) is that I would like to discuss how we should define the limitation of requests per cgroup.
This patch should be applied on Vivek's I/O controller patches posted on Mar 11.

Quoted 4 lines

Hi IKEDA,

Sorry for the confusion here. Actually Nauman had sent a patch to select group of people who were initially copied on the mail thread.

I am sorry about that. Since I dropped my whole patch set in favor of Vivek's stuff, this stuff fell through the cracks.

Quoted 33 lines
Quoted 9 lines

This patch temporarily distribute q->nr_requests to each cgroup. I think the number should be weighted like BFQ's budget.  But in this case, if the hierarchy of cgroup is deep, leaf cgroups are allowed to allocate very few numbers of requests.  I don't think this is reasonable...but I don't have specific idea to solve this problem.  Does anyone have the good idea?

Thanks for the patch. Yes, ideally one would expect the request descriptor to be allocated also in proportion to the weight but I guess that would become very comlicated.

In terms of simpler things, two thoughts come to mind.

- First approach is to make q->nr_requests per group. So every group is entitled for q->nr_requests as set by the user. This is what your patch seems to have done.

 I had some concerns with this approach. First of all it does not seem to have an upper bound on number of request descriptors allocated per queue because if a user creates more cgroups, total number of request descriptors increase.

- Second approach can be that we retain the meaning of q->nr_requests which defines the total number of request descriptors on the queue (with the exception of 50% more descriptors for batching processes). And we define a new per group limit q->nr_group_requests which defines how many requests per group can be assigned. So q->nr_requests defines total pool size on the queue and q->nr_group_requests will define how many requests each group can allocate out of that pool.

Quoted 8 lines

 Here the issue is that a user shall have to balance the q->nr_group_requests    and q->nr_requests properly.

To experiment, I have implemented the second approach. I am attaching the patch which is in my current tree. It probably will not apply on my march 11 posting as since then patches have changed. But posting it here so that at least it will give an idea behind the thought process.

Ideas are welcome...

I had started with the first option, but the second option sounds good too. But one problem that comes to mind is how we deal with hierarchies? The sys admin can limit the root level cgroups to specific number of request descriptors, but if applications running in a cgroup are allowed to create their own cgroups, then the total request descriptors of all child cgroups should be capped by the number assigned to parent cgroups.

Quoted 1,385 lines

Thanks
Vivek

o Currently a request queue has got fixed number of request descriptors for sync and async requests. Once the request descriptors are consumed, new processes are put to sleep and they effectively become serialized. Because sync and async queues are separate, async requests don't impact sync ones but if one is looking for fairness between async requests, that is not achievable if request queue descriptors become bottleneck.

o Make request descriptor's per io group so that if there is lots of IO going on in one cgroup, it does not impact the IO of other group.

o This patch implements the per cgroup request descriptors. request pool per queue is still common but every group will have its own wait list and its own count of request descriptors allocated to that group for sync and async queues. So effectively request_list becomes per io group property and not a global request queue feature.

o Currently one can define q->nr_requests to limit request descriptors allocated for the queue. Now there is another tunable q->nr_group_requests which controls the requests descriptr limit per group. q->nr_requests supercedes q->nr_group_requests to make sure if there are lots of groups present, we don't end up allocating too many request descriptors on the queue.

o Issues: Currently notion of congestion is per queue. With per group request descriptor it is possible that queue is not congested but the group bio will go into is congested.

---
 block/blk-core.c       |  216 ++++++++++++++++++++++++++++++++++--------------- block/blk-settings.c   |    3
 block/blk-sysfs.c      |   57 ++++++++++--
 block/elevator-fq.c    |   15 +++
 block/elevator-fq.h    |    8 +
 block/elevator.c       |    6 -
 include/linux/blkdev.h |   62 +++++++++++++-

 7 files changed, 287 insertions(+), 80 deletions(-)

Index: linux9/include/linux/blkdev.h
===================================================================

include/linux/blkdev.h       2009-04-30 16:18:29.000000000 -0400 renamed from include/linux/blkdev.h  2009-04-30 15:43:53.000000000 -0400 +36 -2
@@ -32,21 +32,51 @@ struct request;
  struct sg_io_hdr;
 
  #define BLKDEV_MIN_RQ  4
+
+#ifdef CONFIG_GROUP_IOSCHED
+#define BLKDEV_MAX_RQ  256     /* Default maximum */
+#define BLKDEV_MAX_GROUP_RQ    64      /* Default maximum */
+#else
  #define BLKDEV_MAX_RQ  128     /* Default maximum */
+/*
+ * This is eqivalent to case of only one group present (root group). Let
+ * it consume all the request descriptors available on the queue .
+ */
+#define BLKDEV_MAX_GROUP_RQ    BLKDEV_MAX_RQ      /* Default maximum */
+#endif
 
  struct request;
  typedef void (rq_end_io_fn)(struct request *, int);
 
  struct request_list {
        /*
-        * count[], starved[], and wait[] are indexed by
+        * count[], starved and wait[] are indexed by
         * BLK_RW_SYNC/BLK_RW_ASYNC
         */
        int count[2];
        int starved[2];
+       wait_queue_head_t wait[2];
+};
+
+/*
+ * This data structures keeps track of mempool of requests for the queue
+ * and some overall statistics.
+ */
+struct request_data {
+       /*
+        * Per queue request descriptor count. This is in addition to per
+        * cgroup count
+        */
+       int count[2];
        int elvpriv;
        mempool_t *rq_pool;
-       wait_queue_head_t wait[2];
+       int starved;
+       /*
+        * Global list for starved tasks. A task will be queued here if
+        * it could not allocate request descriptor and the associated
+        * group request list does not have any requests pending.
+        */
+       wait_queue_head_t starved_wait;
  };
 
  /*
@@ -251,6 +281,7 @@ struct request {
  #ifdef CONFIG_GROUP_IOSCHED
        /* io group request belongs to */
        struct io_group *iog;
+       struct request_list *rl;
  #endif /* GROUP_IOSCHED */
  #endif /* ELV_FAIR_QUEUING */
  };
@@ -340,6 +371,9 @@ struct request_queue
         */
        struct request_list     rq;
 
+       /* Contains request pool and other data like starved data */
+       struct request_data     rq_data;
+
        request_fn_proc         *request_fn;
        make_request_fn         *make_request_fn;
        prep_rq_fn              *prep_rq_fn;
@@ -402,6 +436,8 @@ struct request_queue
         * queue settings
         */

       unsigned long           nr_requests;    /* Max # of requests */
+       /* Max # of per io group requests */
+       unsigned long           nr_group_requests;
       unsigned int            nr_congestion_on;
       unsigned int            nr_congestion_off;
       unsigned int            nr_batching;

include/linux/blkdev.h       2009-04-30 16:18:29.000000000 -0400 renamed from include/linux/blkdev.h  2009-04-30 15:43:53.000000000 -0400 +22 -0
@@ -773,6 +809,28 @@ extern int scsi_cmd_ioctl(struct request
  extern int sg_scsi_ioctl(struct request_queue *, struct gendisk *, fmode_t,
                         struct scsi_ioctl_command __user *);
 
+extern void blk_init_request_list(struct request_list *rl);
+
+static inline struct request_list *blk_get_request_list(struct request_queue *q,
+                                               struct bio *bio)
+{
+#ifdef CONFIG_GROUP_IOSCHED
+       return io_group_get_request_list(q, bio);
+#else
+       return &q->rq;
+#endif
+}
+
+static inline struct request_list *rq_rl(struct request_queue *q,
+                                               struct request *rq)
+{
+#ifdef CONFIG_GROUP_IOSCHED
+       return rq->rl;
+#else
+       return blk_get_request_list(q, NULL);
+#endif
+}
+
  /*
  * Temporary export, until SCSI gets fixed up.
  */

Index: linux9/block/elevator.c
===================================================================

block/elevator.c     2009-04-30 16:18:29.000000000 -0400 renamed from block/elevator.c        2009-04-30 16:17:53.000000000 -0400 +3 -3
@@ -664,7 +664,7 @@ void elv_quiesce_start(struct request_qu
         * make sure we don't have any requests in flight
         */
        elv_drain_elevator(q);
-       while (q->rq.elvpriv) {
+       while (q->rq_data.elvpriv) {
                blk_start_queueing(q);
                spin_unlock_irq(q->queue_lock);
                msleep(10);
@@ -764,8 +764,8 @@ void elv_insert(struct request_queue *q,
        }
 
        if (unplug_it && blk_queue_plugged(q)) {
-               int nrq = q->rq.count[BLK_RW_SYNC] + q->rq.count[BLK_RW_ASYNC]
-                       - q->in_flight;
+               int nrq = q->rq_data.count[BLK_RW_SYNC] +
+                               q->rq_data.count[BLK_RW_ASYNC] - q->in_flight;
 
                if (nrq >= q->unplug_thresh)
                        __generic_unplug_device(q);

Index: linux9/block/blk-core.c
===================================================================

block/blk-core.c     2009-04-30 16:18:29.000000000 -0400 renamed from block/blk-core.c        2009-04-30 16:17:53.000000000 -0400 +22 -8
@@ -480,20 +480,31 @@ void blk_cleanup_queue(struct request_qu
  }
  EXPORT_SYMBOL(blk_cleanup_queue);
 
-static int blk_init_free_list(struct request_queue *q)
+void blk_init_request_list(struct request_list *rl)
  {
-       struct request_list *rl = &q->rq;
 
        rl->count[BLK_RW_SYNC] = rl->count[BLK_RW_ASYNC] = 0;
-       rl->starved[BLK_RW_SYNC] = rl->starved[BLK_RW_ASYNC] = 0;
-       rl->elvpriv = 0;
        init_waitqueue_head(&rl->wait[BLK_RW_SYNC]);
        init_waitqueue_head(&rl->wait[BLK_RW_ASYNC]);
+}
 
-       rl->rq_pool = mempool_create_node(BLKDEV_MIN_RQ, mempool_alloc_slab,
-                               mempool_free_slab, request_cachep, q->node);
+static int blk_init_free_list(struct request_queue *q)
+{
+#ifndef CONFIG_GROUP_IOSCHED
+       struct request_list *rl = blk_get_request_list(q, NULL);
+
+       /*
+        * In case of group scheduling, request list is inside the associated
+        * group and when that group is instanciated, it takes care of
+        * initializing the request list also.
+        */
+       blk_init_request_list(rl);
+#endif
+       q->rq_data.rq_pool = mempool_create_node(BLKDEV_MIN_RQ,
+                               mempool_alloc_slab, mempool_free_slab,
+                               request_cachep, q->node);
 
-       if (!rl->rq_pool)
+       if (!q->rq_data.rq_pool)
                return -ENOMEM;
 
        return 0;
@@ -590,6 +601,9 @@ blk_init_queue_node(request_fn_proc *rfn
                return NULL;
        }
 
+       /* init starved waiter wait queue */
+       init_waitqueue_head(&q->rq_data.starved_wait);
+
        /*
         * if caller didn't supply a lock, they get per-queue locking with
         * our embedded lock
@@ -639,14 +653,14 @@ static inline void blk_free_request(stru
  {
        if (rq->cmd_flags & REQ_ELVPRIV)
                elv_put_request(q, rq);
-       mempool_free(rq, q->rq.rq_pool);
+       mempool_free(rq, q->rq_data.rq_pool);
  }
 

 static struct request *
 blk_alloc_request(struct request_queue *q, struct bio *bio, int rw, int priv, gfp_t gfp_mask)
 {
-       struct request *rq = mempool_alloc(q->rq.rq_pool, gfp_mask);
+       struct request *rq = mempool_alloc(q->rq_data.rq_pool, gfp_mask);

       if (!rq)
               return NULL;

block/blk-core.c     2009-04-30 16:18:29.000000000 -0400 renamed from block/blk-core.c        2009-04-30 16:17:53.000000000 -0400 +81 -42
@@ -657,7 +671,7 @@ blk_alloc_request(struct request_queue *
 
        if (priv) {
                if (unlikely(elv_set_request(q, rq, bio, gfp_mask))) {
-                       mempool_free(rq, q->rq.rq_pool);
+                       mempool_free(rq, q->rq_data.rq_pool);
                        return NULL;
                }
                rq->cmd_flags |= REQ_ELVPRIV;
@@ -700,18 +714,18 @@ static void ioc_set_batching(struct requ
        ioc->last_waited = jiffies;
  }
 
-static void __freed_request(struct request_queue *q, int sync)
+static void __freed_request(struct request_queue *q, int sync,
+                                       struct request_list *rl)
  {
-       struct request_list *rl = &q->rq;
-
-       if (rl->count[sync] < queue_congestion_off_threshold(q))
+       if (q->rq_data.count[sync] < queue_congestion_off_threshold(q))
                blk_clear_queue_congested(q, sync);
 
-       if (rl->count[sync] + 1 <= q->nr_requests) {
+       if (q->rq_data.count[sync] + 1 <= q->nr_requests)
+               blk_clear_queue_full(q, sync);
+
+       if (rl->count[sync] + 1 <= q->nr_group_requests) {
                if (waitqueue_active(&rl->wait[sync]))
                        wake_up(&rl->wait[sync]);
-
-               blk_clear_queue_full(q, sync);
        }
  }
 
@@ -719,18 +733,29 @@ static void __freed_request(struct reque
  * A request has just been released.  Account for it, update the full and
  * congestion status, wake up any waiters.   Called under q->queue_lock.
  */
-static void freed_request(struct request_queue *q, int sync, int priv)
+static void freed_request(struct request_queue *q, int sync, int priv,
+                                       struct request_list *rl)
  {
-       struct request_list *rl = &q->rq;
-
+       BUG_ON(!rl->count[sync]);
        rl->count[sync]--;
+
+       BUG_ON(!q->rq_data.count[sync]);
+       q->rq_data.count[sync]--;
+
        if (priv)
-               rl->elvpriv--;
+               q->rq_data.elvpriv--;
 
-       __freed_request(q, sync);
+       __freed_request(q, sync, rl);
 
        if (unlikely(rl->starved[sync ^ 1]))
-               __freed_request(q, sync ^ 1);
+               __freed_request(q, sync ^ 1, rl);
+
+       /* Wake up the starved process on global list, if any */
+       if (unlikely(q->rq_data.starved)) {
+               if (waitqueue_active(&q->rq_data.starved_wait))
+                       wake_up(&q->rq_data.starved_wait);
+               q->rq_data.starved--;
+       }
  }
 
  /*
@@ -739,10 +764,9 @@ static void freed_request(struct request
  * Returns !NULL on success, with queue_lock *not held*.
  */
  static struct request *get_request(struct request_queue *q, int rw_flags,
-                                  struct bio *bio, gfp_t gfp_mask)
+                  struct bio *bio, gfp_t gfp_mask, struct request_list *rl)
  {
        struct request *rq = NULL;
-       struct request_list *rl = &q->rq;
        struct io_context *ioc = NULL;
        const bool is_sync = rw_is_sync(rw_flags) != 0;
        int may_queue, priv;
@@ -751,31 +775,38 @@ static struct request *get_request(struc
        if (may_queue == ELV_MQUEUE_NO)
                goto rq_starved;
 
-       if (rl->count[is_sync]+1 >= queue_congestion_on_threshold(q)) {
-               if (rl->count[is_sync]+1 >= q->nr_requests) {
-                       ioc = current_io_context(GFP_ATOMIC, q->node);
-                       /*
-                        * The queue will fill after this allocation, so set
-                        * it as full, and mark this process as "batching".
-                        * This process will be allowed to complete a batch of
-                        * requests, others will be blocked.
-                        */
-                       if (!blk_queue_full(q, is_sync)) {
-                               ioc_set_batching(q, ioc);
-                               blk_set_queue_full(q, is_sync);
-                       } else {
-                               if (may_queue != ELV_MQUEUE_MUST
-                                               && !ioc_batching(q, ioc)) {
-                                       /*
-                                        * The queue is full and the allocating
-                                        * process is not a "batcher", and not
-                                        * exempted by the IO scheduler
-                                        */
-                                       goto out;
-                               }
+       if (q->rq_data.count[is_sync]+1 >= queue_congestion_on_threshold(q))
+               blk_set_queue_congested(q, is_sync);
+
+       /*
+        * Looks like there is no user of queue full now.
+        * Keeping it for time being.
+        */
+       if (q->rq_data.count[is_sync]+1 >= q->nr_requests)
+               blk_set_queue_full(q, is_sync);
+
+       if (rl->count[is_sync]+1 >= q->nr_group_requests) {
+               ioc = current_io_context(GFP_ATOMIC, q->node);
+               /*
+                * The queue request descriptor group will fill after this
+                * allocation, so set
+                * it as full, and mark this process as "batching".
+                * This process will be allowed to complete a batch of
+                * requests, others will be blocked.
+                */
+               if (rl->count[is_sync] <= q->nr_group_requests)
+                       ioc_set_batching(q, ioc);
+               else {
+                       if (may_queue != ELV_MQUEUE_MUST
+                                       && !ioc_batching(q, ioc)) {
+                               /*
+                                * The queue is full and the allocating
+                                * process is not a "batcher", and not
+                                * exempted by the IO scheduler
+                                */
+                               goto out;
                        }
                }
-               blk_set_queue_congested(q, is_sync);
        }
 
        /*
@@ -783,19 +814,41 @@ static struct request *get_request(struc
         * limit of requests, otherwise we could have thousands of requests
         * allocated with any setting of ->nr_requests
         */
-       if (rl->count[is_sync] >= (3 * q->nr_requests / 2))
+
+       if (q->rq_data.count[is_sync] >= (3 * q->nr_requests / 2))
+               goto out;
+
+       /*
+        * Allocation of request is allowed from queue perspective. Now check
+        * from per group request list
+        */
+
+       if (rl->count[is_sync] >= (3 * q->nr_group_requests / 2))
                goto out;
 
        rl->count[is_sync]++;
        rl->starved[is_sync] = 0;
 
+       q->rq_data.count[is_sync]++;
+
        priv = !test_bit(QUEUE_FLAG_ELVSWITCH, &q->queue_flags);
        if (priv)
-               rl->elvpriv++;
+               q->rq_data.elvpriv++;
 
        spin_unlock_irq(q->queue_lock);
 
        rq = blk_alloc_request(q, bio, rw_flags, priv, gfp_mask);
+
+#ifdef CONFIG_GROUP_IOSCHED
+       if (rq) {
+               /*
+                * TODO. Implement group reference counting and take the
+                * reference to the group to make sure group hence request
+                * list does not go away till rq finishes.
+                */
+               rq->rl = rl;
+       }
+#endif
        if (unlikely(!rq)) {
                /*
                 * Allocation failed presumably due to memory. Undo anything
@@ -805,7 +858,7 @@ static struct request *get_request(struc
                 * wait queue, but this is pretty rare.
                 */
                spin_lock_irq(q->queue_lock);
-               freed_request(q, is_sync, priv);
+               freed_request(q, is_sync, priv, rl);
 
                /*
                 * in the very unlikely event that allocation failed and no
@@ -815,10 +868,26 @@ static struct request *get_request(struc
                 * rq mempool into READ and WRITE
                 */

 rq_starved:
-               if (unlikely(rl->count[is_sync] == 0))
-                       rl->starved[is_sync] = 1;
-
-               goto out;
+               if (unlikely(rl->count[is_sync] == 0)) {
+                       /*
+                        * If there is a request pending in other direction
+                        * in same io group, then set the starved flag of
+                        * the group request list. Otherwise, we need to
+                        * make this process sleep in global starved list
+                        * to make sure it will not sleep indefinitely.
+                        */
+                       if (rl->count[is_sync ^ 1] != 0) {
+                               rl->starved[is_sync] = 1;
+                               goto out;
+                       } else {
+                               /*
+                                * It indicates to calling function to put
+                                * task on global starved list. Not the best
+                                * way
+                                */
+                               return ERR_PTR(-ENOMEM);
+                       }
+               }
       }

       /*

block/blk-core.c     2009-04-30 16:18:29.000000000 -0400 renamed from block/blk-core.c        2009-04-30 16:17:53.000000000 -0400 +29 -8
@@ -846,15 +915,29 @@ static struct request *get_request_wait(
  {
        const bool is_sync = rw_is_sync(rw_flags) != 0;
        struct request *rq;
+       struct request_list *rl = blk_get_request_list(q, bio);
 
-       rq = get_request(q, rw_flags, bio, GFP_NOIO);
-       while (!rq) {
+       rq = get_request(q, rw_flags, bio, GFP_NOIO, rl);
+       while (!rq || (IS_ERR(rq) && PTR_ERR(rq) == -ENOMEM)) {
                DEFINE_WAIT(wait);
                struct io_context *ioc;
-               struct request_list *rl = &q->rq;
 
-               prepare_to_wait_exclusive(&rl->wait[is_sync], &wait,
-                               TASK_UNINTERRUPTIBLE);
+               if (IS_ERR(rq) && PTR_ERR(rq) == -ENOMEM) {
+                       /*
+                        * Task failed allocation and needs to wait and
+                        * try again. There are no requests pending from
+                        * the io group hence need to sleep on global
+                        * wait queue. Most likely the allocation failed
+                        * because of memory issues.
+                        */
+
+                       q->rq_data.starved++;
+                       prepare_to_wait_exclusive(&q->rq_data.starved_wait,
+                                       &wait, TASK_UNINTERRUPTIBLE);
+               } else {
+                       prepare_to_wait_exclusive(&rl->wait[is_sync], &wait,
+                                       TASK_UNINTERRUPTIBLE);
+               }
 
                trace_block_sleeprq(q, bio, rw_flags & 1);
 
@@ -874,7 +957,12 @@ static struct request *get_request_wait(
                spin_lock_irq(q->queue_lock);
                finish_wait(&rl->wait[is_sync], &wait);
 
-               rq = get_request(q, rw_flags, bio, GFP_NOIO);
+               /*
+                * After the sleep check the rl again in case cgrop bio
+                * belonged to is gone and it is mapped to root group now
+                */
+               rl = blk_get_request_list(q, bio);
+               rq = get_request(q, rw_flags, bio, GFP_NOIO, rl);
        };
 
        return rq;
@@ -883,6 +971,7 @@ static struct request *get_request_wait(
  struct request *blk_get_request(struct request_queue *q, int rw, gfp_t gfp_mask)
  {
        struct request *rq;
+       struct request_list *rl = blk_get_request_list(q, NULL);
 
        BUG_ON(rw != READ && rw != WRITE);
 
@@ -890,7 +979,7 @@ struct request *blk_get_request(struct r
        if (gfp_mask & __GFP_WAIT) {
                rq = get_request_wait(q, rw, NULL);
        } else {
-               rq = get_request(q, rw, NULL, gfp_mask);
+               rq = get_request(q, rw, NULL, gfp_mask, rl);
                if (!rq)
                        spin_unlock_irq(q->queue_lock);
        }
@@ -1073,12 +1162,13 @@ void __blk_put_request(struct request_qu
        if (req->cmd_flags & REQ_ALLOCED) {
                int is_sync = rq_is_sync(req) != 0;
                int priv = req->cmd_flags & REQ_ELVPRIV;
+               struct request_list *rl = rq_rl(q, req);
 
                BUG_ON(!list_empty(&req->queuelist));
                BUG_ON(!hlist_unhashed(&req->hash));
 
                blk_free_request(q, req);
-               freed_request(q, is_sync, priv);
+               freed_request(q, is_sync, priv, rl);
        }
  }
  EXPORT_SYMBOL_GPL(__blk_put_request);

Index: linux9/block/blk-sysfs.c
===================================================================

block/blk-sysfs.c    2009-04-30 16:18:29.000000000 -0400 renamed from block/blk-sysfs.c       2009-04-30 16:18:27.000000000 -0400 +0 -0
@@ -38,7 +38,7 @@ static ssize_t queue_requests_show(struc

 static ssize_t
 queue_requests_store(struct request_queue *q, const char *page, size_t count) {
-       struct request_list *rl = &q->rq;
+       struct request_list *rl = blk_get_request_list(q, NULL); unsigned long nr;
       int ret = queue_var_store(&nr, page, count);
       if (nr < BLKDEV_MIN_RQ)

block/blk-sysfs.c    2009-04-30 16:18:29.000000000 -0400 renamed from block/blk-sysfs.c       2009-04-30 16:18:27.000000000 -0400 +42 -8
@@ -48,32 +48,55 @@ queue_requests_store(struct request_queu
        q->nr_requests = nr;
        blk_queue_congestion_threshold(q);
 
-       if (rl->count[BLK_RW_SYNC] >= queue_congestion_on_threshold(q))
+       if (q->rq_data.count[BLK_RW_SYNC] >= queue_congestion_on_threshold(q))
                blk_set_queue_congested(q, BLK_RW_SYNC);
-       else if (rl->count[BLK_RW_SYNC] < queue_congestion_off_threshold(q))
+       else if (q->rq_data.count[BLK_RW_SYNC] <
+                               queue_congestion_off_threshold(q))
                blk_clear_queue_congested(q, BLK_RW_SYNC);
 
-       if (rl->count[BLK_RW_ASYNC] >= queue_congestion_on_threshold(q))
+       if (q->rq_data.count[BLK_RW_ASYNC] >= queue_congestion_on_threshold(q))
                blk_set_queue_congested(q, BLK_RW_ASYNC);
-       else if (rl->count[BLK_RW_ASYNC] < queue_congestion_off_threshold(q))
+       else if (q->rq_data.count[BLK_RW_ASYNC] <
+                               queue_congestion_off_threshold(q))
                blk_clear_queue_congested(q, BLK_RW_ASYNC);
 
-       if (rl->count[BLK_RW_SYNC] >= q->nr_requests) {
+       if (q->rq_data.count[BLK_RW_SYNC] >= q->nr_requests) {
                blk_set_queue_full(q, BLK_RW_SYNC);
-       } else if (rl->count[BLK_RW_SYNC]+1 <= q->nr_requests) {
+       } else if (q->rq_data.count[BLK_RW_SYNC]+1 <= q->nr_requests) {
                blk_clear_queue_full(q, BLK_RW_SYNC);
                wake_up(&rl->wait[BLK_RW_SYNC]);
        }
 
-       if (rl->count[BLK_RW_ASYNC] >= q->nr_requests) {
+       if (q->rq_data.count[BLK_RW_ASYNC] >= q->nr_requests) {
                blk_set_queue_full(q, BLK_RW_ASYNC);
-       } else if (rl->count[BLK_RW_ASYNC]+1 <= q->nr_requests) {
+       } else if (q->rq_data.count[BLK_RW_ASYNC]+1 <= q->nr_requests) {
                blk_clear_queue_full(q, BLK_RW_ASYNC);
                wake_up(&rl->wait[BLK_RW_ASYNC]);
        }
        spin_unlock_irq(q->queue_lock);
        return ret;
  }
+#ifdef CONFIG_GROUP_IOSCHED
+static ssize_t queue_group_requests_show(struct request_queue *q, char *page)
+{
+       return queue_var_show(q->nr_group_requests, (page));
+}
+
+static ssize_t
+queue_group_requests_store(struct request_queue *q, const char *page,
+                                       size_t count)
+{
+       unsigned long nr;
+       int ret = queue_var_store(&nr, page, count);
+       if (nr < BLKDEV_MIN_RQ)
+               nr = BLKDEV_MIN_RQ;
+
+       spin_lock_irq(q->queue_lock);
+       q->nr_group_requests = nr;
+       spin_unlock_irq(q->queue_lock);
+       return ret;
+}
+#endif
 
  static ssize_t queue_ra_show(struct request_queue *q, char *page)
  {
@@ -228,6 +251,14 @@ static struct queue_sysfs_entry queue_re
        .store = queue_requests_store,
  };
 
+#ifdef CONFIG_GROUP_IOSCHED
+static struct queue_sysfs_entry queue_group_requests_entry = {
+       .attr = {.name = "nr_group_requests", .mode = S_IRUGO | S_IWUSR },
+       .show = queue_group_requests_show,
+       .store = queue_group_requests_store,
+};
+#endif
+
  static struct queue_sysfs_entry queue_ra_entry = {
        .attr = {.name = "read_ahead_kb", .mode = S_IRUGO | S_IWUSR },
        .show = queue_ra_show,
@@ -308,6 +339,9 @@ static struct queue_sysfs_entry queue_sl
 
  static struct attribute *default_attrs[] = {
        &queue_requests_entry.attr,
+#ifdef CONFIG_GROUP_IOSCHED
+       &queue_group_requests_entry.attr,
+#endif
        &queue_ra_entry.attr,
        &queue_max_hw_sectors_entry.attr,
        &queue_max_sectors_entry.attr,
@@ -389,12 +423,11 @@ static void blk_release_queue(struct kob
  {

       struct request_queue *q =
               container_of(kobj, struct request_queue, kobj);
-       struct request_list *rl = &q->rq;

       blk_sync_queue(q);

-       if (rl->rq_pool)
-               mempool_destroy(rl->rq_pool);
+       if (q->rq_data.rq_pool)
+               mempool_destroy(q->rq_data.rq_pool);

       if (q->queue_tags)
               __blk_queue_free_tags(q);
Index: linux9/block/blk-settings.c
===================================================================

block/blk-settings.c 2009-04-30 16:18:29.000000000 -0400 renamed from block/blk-settings.c    2009-04-30 15:43:53.000000000 -0400 +3 -0
@@ -123,6 +123,9 @@ void blk_queue_make_request(struct reque
         * set defaults
         */
        q->nr_requests = BLKDEV_MAX_RQ;
+#ifdef CONFIG_GROUP_IOSCHED
+       q->nr_group_requests = BLKDEV_MAX_GROUP_RQ;
+#endif
        blk_queue_max_phys_segments(q, MAX_PHYS_SEGMENTS);
        blk_queue_max_hw_segments(q, MAX_HW_SEGMENTS);
        blk_queue_segment_boundary(q, BLK_SEG_BOUNDARY_MASK);

Index: linux9/block/elevator-fq.c
===================================================================

block/elevator-fq.c  2009-04-30 16:18:29.000000000 -0400 renamed from block/elevator-fq.c     2009-04-30 16:18:27.000000000 -0400 +15 -0
@@ -954,6 +954,17 @@ struct io_cgroup *cgroup_to_io_cgroup(st
                            struct io_cgroup, css);
  }
 
+struct request_list *io_group_get_request_list(struct request_queue *q,
+                                               struct bio *bio)
+{
+       struct io_group *iog;
+
+       iog = io_get_io_group_bio(q, bio, 1);
+       BUG_ON(!iog);
+out:
+       return &iog->rl;
+}
+
  /*
  * Search the bfq_group for bfqd into the hash table (by now only a list)
  * of bgrp.  Must be called under rcu_read_lock().
@@ -1203,6 +1214,8 @@ struct io_group *io_group_chain_alloc(st
                io_group_init_entity(iocg, iog);
                iog->my_entity = &iog->entity;
 
+               blk_init_request_list(&iog->rl);
+
                if (leaf == NULL) {
                        leaf = iog;
                        prev = leaf;
@@ -1446,6 +1459,8 @@ struct io_group *io_alloc_root_group(str
        for (i = 0; i < IO_IOPRIO_CLASSES; i++)
                iog->sched_data.service_tree[i] = IO_SERVICE_TREE_INIT;
 
+       blk_init_request_list(&iog->rl);
+
        iocg = &io_root_cgroup;
        spin_lock_irq(&iocg->lock);
        rcu_assign_pointer(iog->key, key);

Index: linux9/block/elevator-fq.h
===================================================================

block/elevator-fq.h  2009-04-30 16:18:29.000000000 -0400 renamed from block/elevator-fq.h     2009-04-30 16:18:27.000000000 -0400 +8 -0
@@ -239,8 +239,14 @@ struct io_group {
 
        /* Single ioq per group, used for noop, deadline, anticipatory */
        struct io_queue *ioq;
+
+       /* request list associated with the group */
+       struct request_list rl;
  };
 
+#define IOG_FLAG_READFULL      1       /* read queue has been filled */
+#define IOG_FLAG_WRITEFULL     2       /* write queue has been filled */
+
  /**
  * struct bfqio_cgroup - bfq cgroup data structure.
  * @css: subsystem state for bfq in the containing cgroup.
@@ -517,6 +523,8 @@ extern void elv_fq_unset_request_ioq(str
  extern struct io_queue *elv_lookup_ioq_current(struct request_queue *q);
  extern struct io_queue *elv_lookup_ioq_bio(struct request_queue *q,
                                                struct bio *bio);
+extern struct request_list *io_group_get_request_list(struct request_queue *q,
+                                               struct bio *bio);
 
  /* Returns single ioq associated with the io group. */
  static inline struct io_queue *io_group_ioq(struct io_group *iog)

Thanks
Vivek

Quoted 619 lines

---
block/blk-core.c    |   36 +++++++--
block/blk-sysfs.c   |   22 ++++--
block/elevator-fq.c |  133 ++++++++++++++++++++++++++++++++-- block/elevator-fq.h |  201 +++++++++++++++++++++++++++++++++++++++++++++++++++

4 files changed, 371 insertions(+), 21 deletions(-)
block/blk-core.c +28 -8
index 29bcfac..21023f7 100644
@@ -705,11 +705,15 @@ static void ioc_set_batching(struct request_queue *q, struct io_context *ioc)
 static void __freed_request(struct request_queue *q, int rw)
 {
       struct request_list *rl = &q->rq;
-
-     if (rl->count[rw] < queue_congestion_off_threshold(q))
+     struct io_group *congested_iog, *full_iog;
+
+     congested_iog = io_congested_io_group(q, rw);
+     if (rl->count[rw] < queue_congestion_off_threshold(q) &&
+         !congested_iog)
               blk_clear_queue_congested(q, rw);
 
-     if (rl->count[rw] + 1 <= q->nr_requests) {
+     full_iog = io_full_io_group(q, rw);
+     if (rl->count[rw] + 1 <= q->nr_requests && !full_iog) {
               if (waitqueue_active(&rl->wait[rw]))
                       wake_up(&rl->wait[rw]);
 
@@ -721,13 +725,16 @@ static void __freed_request(struct request_queue *q, int rw)
  * A request has just been released.  Account for it, update the full and
  * congestion status, wake up any waiters.   Called under q->queue_lock.
  */
-static void freed_request(struct request_queue *q, int rw, int priv)
+static void freed_request(struct request_queue *q, struct io_group *iog,
+                       int rw, int priv)
 {
       struct request_list *rl = &q->rq;
 
       rl->count[rw]--;
       if (priv)
               rl->elvpriv--;
+     if (iog)
+             io_group_dec_count(iog, rw);
 
       __freed_request(q, rw);
 
@@ -746,16 +753,21 @@ static struct request *get_request(struct request_queue *q, int rw_flags,
 {
       struct request *rq = NULL;
       struct request_list *rl = &q->rq;
+     struct io_group *iog;
       struct io_context *ioc = NULL;
       const int rw = rw_flags & 0x01;
       int may_queue, priv;
 
+     iog = __io_get_io_group(q);
+
       may_queue = elv_may_queue(q, rw_flags);
       if (may_queue == ELV_MQUEUE_NO)
               goto rq_starved;
 
-     if (rl->count[rw]+1 >= queue_congestion_on_threshold(q)) {
-             if (rl->count[rw]+1 >= q->nr_requests) {
+     if (rl->count[rw]+1 >= queue_congestion_on_threshold(q) ||
+         io_group_congestion_on(iog, rw)) {
+             if (rl->count[rw]+1 >= q->nr_requests ||
+                 io_group_full(iog, rw)) {
                       ioc = current_io_context(GFP_ATOMIC, q->node);
                       /*
                        * The queue will fill after this allocation, so set
@@ -789,8 +801,15 @@ static struct request *get_request(struct request_queue *q, int rw_flags,
       if (rl->count[rw] >= (3 * q->nr_requests / 2))
               goto out;
 
+     if (iog)
+             if (io_group_count(iog, rw) >=
+                (3 * io_group_nr_requests(iog) / 2))
+                     goto out;
+
       rl->count[rw]++;
       rl->starved[rw] = 0;
+     if (iog)
+             io_group_inc_count(iog, rw);
 
       priv = !test_bit(QUEUE_FLAG_ELVSWITCH, &q->queue_flags);
       if (priv)
@@ -808,7 +827,7 @@ static struct request *get_request(struct request_queue *q, int rw_flags,
                * wait queue, but this is pretty rare.
                */
               spin_lock_irq(q->queue_lock);
-             freed_request(q, rw, priv);
+             freed_request(q, iog, rw, priv);
 
               /*
                * in the very unlikely event that allocation failed and no
@@ -1073,12 +1092,13 @@ void __blk_put_request(struct request_queue *q, struct request *req)
       if (req->cmd_flags & REQ_ALLOCED) {
               int rw = rq_data_dir(req);
               int priv = req->cmd_flags & REQ_ELVPRIV;
+             struct io_group *iog = io_request_io_group(req);
 
               BUG_ON(!list_empty(&req->queuelist));
               BUG_ON(!hlist_unhashed(&req->hash));
 
               blk_free_request(q, req);
-             freed_request(q, rw, priv);
+             freed_request(q, iog, rw, priv);
       }
 }
 EXPORT_SYMBOL_GPL(__blk_put_request);
block/blk-sysfs.c +14 -8
index 0d98c96..af5191c 100644
@@ -40,6 +40,7 @@ queue_requests_store(struct request_queue *q, const char *page, size_t count)
 {
       struct request_list *rl = &q->rq;
       unsigned long nr;
+     int iog_congested[2], iog_full[2];
       int ret = queue_var_store(&nr, page, count);
       if (nr < BLKDEV_MIN_RQ)
               nr = BLKDEV_MIN_RQ;
@@ -47,27 +48,32 @@ queue_requests_store(struct request_queue *q, const char *page, size_t count)
       spin_lock_irq(q->queue_lock);
       q->nr_requests = nr;
       blk_queue_congestion_threshold(q);
+     io_group_set_nrq_all(q, nr, iog_congested, iog_full);
 
-     if (rl->count[READ] >= queue_congestion_on_threshold(q))
+     if (rl->count[READ] >= queue_congestion_on_threshold(q) ||
+         iog_congested[READ])
               blk_set_queue_congested(q, READ);
-     else if (rl->count[READ] < queue_congestion_off_threshold(q))
+     else if (rl->count[READ] < queue_congestion_off_threshold(q) &&
+              !iog_congested[READ])
               blk_clear_queue_congested(q, READ);
 
-     if (rl->count[WRITE] >= queue_congestion_on_threshold(q))
+     if (rl->count[WRITE] >= queue_congestion_on_threshold(q) ||
+         iog_congested[WRITE])
               blk_set_queue_congested(q, WRITE);
-     else if (rl->count[WRITE] < queue_congestion_off_threshold(q))
+     else if (rl->count[WRITE] < queue_congestion_off_threshold(q) &&
+              !iog_congested[WRITE])
               blk_clear_queue_congested(q, WRITE);
 
-     if (rl->count[READ] >= q->nr_requests) {
+     if (rl->count[READ] >= q->nr_requests || iog_full[READ]) {
               blk_set_queue_full(q, READ);
-     } else if (rl->count[READ]+1 <= q->nr_requests) {
+     } else if (rl->count[READ]+1 <= q->nr_requests && !iog_full[READ]) {
               blk_clear_queue_full(q, READ);
               wake_up(&rl->wait[READ]);
       }
 
-     if (rl->count[WRITE] >= q->nr_requests) {
+     if (rl->count[WRITE] >= q->nr_requests || iog_full[WRITE]) {
               blk_set_queue_full(q, WRITE);
-     } else if (rl->count[WRITE]+1 <= q->nr_requests) {
+     } else if (rl->count[WRITE]+1 <= q->nr_requests && !iog_full[WRITE]) {
               blk_clear_queue_full(q, WRITE);
               wake_up(&rl->wait[WRITE]);
       }
block/elevator-fq.c +128 -5
index df53418..3b021f3 100644
@@ -924,6 +924,111 @@ struct io_group *io_lookup_io_group_current(struct request_queue *q)
 }
 EXPORT_SYMBOL(io_lookup_io_group_current);
 
+/*
+ * TODO
+ * This is complete dupulication of blk_queue_congestion_threshold()
+ * except for the argument type and name.  Can we merge them?
+ */
+static void io_group_nrq_congestion_threshold(struct io_group_nrq *nrq)
+{
+     int nr;
+
+     nr = nrq->nr_requests - (nrq->nr_requests / 8) + 1;
+     if (nr > nrq->nr_requests)
+             nr = nrq->nr_requests;
+     nrq->nr_congestion_on = nr;
+
+     nr = nrq->nr_requests - (nrq->nr_requests / 8)
+             - (nrq->nr_requests / 16) - 1;
+     if (nr < 1)
+             nr = 1;
+     nrq->nr_congestion_off = nr;
+}
+
+static void io_group_set_nrq(struct io_group_nrq *nrq, int nr_requests,
+                      int *congested, int *full)
+{
+     int i;
+
+     BUG_ON(nr_requests < 0);
+
+     nrq->nr_requests = nr_requests;
+     io_group_nrq_congestion_threshold(nrq);
+
+     for (i=0; i<2; i++) {
+             if (nrq->count[i] >= nrq->nr_congestion_on)
+                     congested[i] = 1;
+             else if (nrq->count[i] < nrq->nr_congestion_off)
+                     congested[i] = 0;
+
+             if (nrq->count[i] >= nrq->nr_requests)
+                     full[i] = 1;
+             else if (nrq->count[i]+1 <= nrq->nr_requests)
+                     full[i] = 0;
+     }
+}
+
+void io_group_set_nrq_all(struct request_queue *q, int nr,
+                         int *congested, int *full)
+{
+     struct elv_fq_data *efqd = &q->elevator->efqd;
+     struct hlist_head *head = &efqd->group_list;
+     struct io_group *root = efqd->root_group;
+     struct hlist_node *n;
+     struct io_group *iog;
+     struct io_group_nrq *nrq;
+     int nrq_congested[2];
+     int nrq_full[2];
+     int i;
+
+     for (i=0; i<2; i++)
+             *(congested + i) = *(full + i) = 0;
+
+     nrq = &root->nrq;
+     io_group_set_nrq(nrq, nr, nrq_congested, nrq_full);
+     for (i=0; i<2; i++) {
+             *(congested + i) |= nrq_congested[i];
+             *(full + i) |= nrq_full[i];
+     }
+
+     hlist_for_each_entry(iog, n, head, elv_data_node) {
+             nrq = &iog->nrq;
+             io_group_set_nrq(nrq, nr, nrq_congested, nrq_full);
+             for (i=0; i<2; i++) {
+                     *(congested + i) |= nrq_congested[i];
+                     *(full + i) |= nrq_full[i];
+             }
+     }
+}
+
+struct io_group *io_congested_io_group(struct request_queue *q, int rw)
+{
+     struct hlist_head *head = &q->elevator->efqd.group_list;
+     struct hlist_node *n;
+     struct io_group *iog;
+
+     hlist_for_each_entry(iog, n, head, elv_data_node) {
+             struct io_group_nrq *nrq = &iog->nrq;
+             if (nrq->count[rw] >= nrq->nr_congestion_off)
+                     return iog;
+     }
+     return NULL;
+}
+
+struct io_group *io_full_io_group(struct request_queue *q, int rw)
+{
+     struct hlist_head *head = &q->elevator->efqd.group_list;
+     struct hlist_node *n;
+     struct io_group *iog;
+
+     hlist_for_each_entry(iog, n, head, elv_data_node) {
+             struct io_group_nrq *nrq = &iog->nrq;
+             if (nrq->count[rw] >= nrq->nr_requests)
+                     return iog;
+     }
+     return NULL;
+}
+
 void io_group_init_entity(struct io_cgroup *iocg, struct io_group *iog)
 {
       struct io_entity *entity = &iog->entity;
@@ -934,6 +1039,12 @@ void io_group_init_entity(struct io_cgroup *iocg, struct io_group *iog)
       entity->my_sched_data = &iog->sched_data;
 }
 
+static void io_group_init_nrq(struct request_queue *q, struct io_group_nrq *nrq)
+{
+     nrq->nr_requests = q->nr_requests;
+     io_group_nrq_congestion_threshold(nrq);
+}
+
 void io_group_set_parent(struct io_group *iog, struct io_group *parent)
 {
       struct io_entity *entity;
@@ -1053,6 +1164,8 @@ struct io_group *io_group_chain_alloc(struct request_queue *q, void *key,
               io_group_init_entity(iocg, iog);
               iog->my_entity = &iog->entity;
 
+             io_group_init_nrq(q, &iog->nrq);
+
               if (leaf == NULL) {
                       leaf = iog;
                       prev = leaf;
@@ -1176,7 +1289,7 @@ struct io_group *io_find_alloc_group(struct request_queue *q,
  * Generic function to make sure cgroup hierarchy is all setup once a request
  * from a cgroup is received by the io scheduler.
  */
-struct io_group *io_get_io_group(struct request_queue *q)
+struct io_group *__io_get_io_group(struct request_queue *q)
 {
       struct cgroup *cgroup;
       struct io_group *iog;
@@ -1192,6 +1305,19 @@ struct io_group *io_get_io_group(struct request_queue *q)
       return iog;
 }
 
+struct io_group *io_get_io_group(struct request_queue *q)
+{
+     struct io_group *iog;
+     unsigned long flags;
+
+     spin_lock_irqsave(q->queue_lock, flags);
+     iog = __io_get_io_group(q);
+     spin_unlock_irqrestore(q->queue_lock, flags);
+     BUG_ON(!iog);
+
+     return iog;
+}
+
 void io_free_root_group(struct elevator_queue *e)
 {
       struct io_cgroup *iocg = &io_root_cgroup;
@@ -1220,6 +1346,7 @@ struct io_group *io_alloc_root_group(struct request_queue *q,
       iog->entity.parent = NULL;
       for (i = 0; i < IO_IOPRIO_CLASSES; i++)
               iog->sched_data.service_tree[i] = IO_SERVICE_TREE_INIT;
+     io_group_init_nrq(q, &iog->nrq);
 
       iocg = &io_root_cgroup;
       spin_lock_irq(&iocg->lock);
@@ -1533,15 +1660,11 @@ void elv_fq_set_request_io_group(struct request_queue *q,
                                               struct request *rq)
 {
       struct io_group *iog;
-     unsigned long flags;
 
       /* Make sure io group hierarchy has been setup and also set the
        * io group to which rq belongs. Later we should make use of
        * bio cgroup patches to determine the io group */
-     spin_lock_irqsave(q->queue_lock, flags);
       iog = io_get_io_group(q);
-     spin_unlock_irqrestore(q->queue_lock, flags);
-     BUG_ON(!iog);
 
       /* Store iog in rq. TODO: take care of referencing */
       rq->iog = iog;
block/elevator-fq.h +201 -1
index fc4110d..f8eabd4 100644
@@ -187,6 +187,22 @@ struct io_queue {
 
 #ifdef CONFIG_GROUP_IOSCHED
 /**
+ * struct io_group_nrq - structure to store allocated requests info
+ * @nr_requests: maximun num of requests for the io_group
+ * @nr_congestion_on: threshold to determin the io_group is cogested.
+ * @nr_congestion_off: threshold to determin the io_group is not congested.
+ * @count: num of allocated requests.
+ *
+ * All fields are protected by queue_lock.
+ */
+struct io_group_nrq {
+     unsigned long nr_requests;
+     unsigned int nr_congestion_on;
+     unsigned int nr_congestion_off;
+     int count[2];
+};
+
+/**
  * struct bfq_group - per (device, cgroup) data structure.
  * @entity: schedulable entity to insert into the parent group sched_data.
  * @sched_data: own sched_data, to contain child entities (they may be
@@ -235,6 +251,8 @@ struct io_group {
 
       /* Single ioq per group, used for noop, deadline, anticipatory */
       struct io_queue *ioq;
+
+     struct io_group_nrq nrq;
 };
 
 /**
@@ -469,6 +487,11 @@ extern int elv_fq_set_request_ioq(struct request_queue *q, struct request *rq,
 extern void elv_fq_unset_request_ioq(struct request_queue *q,
                                       struct request *rq);
 extern struct io_queue *elv_lookup_ioq_current(struct request_queue *q);
+extern void io_group_set_nrq_all(struct request_queue *q, int nr,
+                         int *congested, int *full);
+extern struct io_group *io_congested_io_group(struct request_queue *q, int rw);
+extern struct io_group *io_full_io_group(struct request_queue *q, int rw);
+extern struct io_group *__io_get_io_group(struct request_queue *q);
 
 /* Returns single ioq associated with the io group. */
 static inline struct io_queue *io_group_ioq(struct io_group *iog)
@@ -486,6 +509,52 @@ static inline void io_group_set_ioq(struct io_group *iog, struct io_queue *ioq)
       iog->ioq = ioq;
 }
 
+static inline struct io_group *io_request_io_group(struct request *rq)
+{
+     return rq->iog;
+}
+
+static inline unsigned long io_group_nr_requests(struct io_group *iog)
+{
+     BUG_ON(!iog);
+     return iog->nrq.nr_requests;
+}
+
+static inline int io_group_inc_count(struct io_group *iog, int rw)
+{
+     BUG_ON(!iog);
+     return iog->nrq.count[rw]++;
+}
+
+static inline int io_group_dec_count(struct io_group *iog, int rw)
+{
+     BUG_ON(!iog);
+     return iog->nrq.count[rw]--;
+}
+
+static inline int io_group_count(struct io_group *iog, int rw)
+{
+     BUG_ON(!iog);
+     return iog->nrq.count[rw];
+}
+
+static inline int io_group_congestion_on(struct io_group *iog, int rw)
+{
+     BUG_ON(!iog);
+     return iog->nrq.count[rw] + 1 >= iog->nrq.nr_congestion_on;
+}
+
+static inline int io_group_congestion_off(struct io_group *iog, int rw)
+{
+     BUG_ON(!iog);
+     return iog->nrq.count[rw] < iog->nrq.nr_congestion_off;
+}
+
+static inline int io_group_full(struct io_group *iog, int rw)
+{
+     BUG_ON(!iog);
+     return iog->nrq.count[rw] + 1 >= iog->nrq.nr_requests;
+}
 #else /* !GROUP_IOSCHED */
 /*
  * No ioq movement is needed in case of flat setup. root io group gets cleaned
@@ -537,6 +606,71 @@ static inline struct io_queue *elv_lookup_ioq_current(struct request_queue *q)
       return NULL;
 }
 
+static inline void io_group_set_nrq_all(struct request_queue *q, int nr,
+                                     int *congested, int *full)
+{
+     int i;
+     for (i=0; i<2; i++)
+             *(congested + i) = *(full + i) = 0;
+}
+
+static inline struct io_group *
+io_congested_io_group(struct request_queue *q, int rw)
+{
+     return NULL;
+}
+
+static inline struct io_group *
+io_full_io_group(struct request_queue *q, int rw)
+{
+     return NULL;
+}
+
+static inline struct io_group *__io_get_io_group(struct request_queue *q)
+{
+     return NULL;
+}
+
+static inline struct io_group *io_request_io_group(struct request *rq)
+{
+     return NULL;
+}
+
+static inline unsigned long io_group_nr_requests(struct io_group *iog)
+{
+     return 0;
+}
+
+static inline int io_group_inc_count(struct io_group *iog, int rw)
+{
+     return 0;
+}
+
+static inline int io_group_dec_count(struct io_group *iog, int rw)
+{
+     return 0;
+}
+
+static inline int io_group_count(struct io_group *iog, int rw)
+{
+     return 0;
+}
+
+static inline int io_group_congestion_on(struct io_group *iog, int rw)
+{
+     return 0;
+}
+
+static inline int io_group_congestion_off(struct io_group *iog, int rw)
+{
+     return 1;
+}
+
+static inline int io_group_full(struct io_group *iog, int rw)
+{
+     return 0;
+}
+
 #endif /* GROUP_IOSCHED */
 
 /* Functions used by blksysfs.c */
@@ -589,6 +723,9 @@ extern void elv_free_ioq(struct io_queue *ioq);
 
 #else /* CONFIG_ELV_FAIR_QUEUING */
 
+struct io_group {
+};
+
 static inline int elv_init_fq_data(struct request_queue *q,
                                       struct elevator_queue *e)
 {
@@ -655,5 +792,69 @@ static inline struct io_queue *elv_lookup_ioq_current(struct request_queue *q)
       return NULL;
 }
 
+static inline void io_group_set_nrq_all(struct request_queue *q, int nr,
+                                     int *congested, int *full)
+{
+     int i;
+     for (i=0; i<2; i++)
+             *(congested + i) = *(full + i) = 0;
+}
+
+static inline struct io_group *
+io_congested_io_group(struct request_queue *q, int rw)
+{
+     return NULL;
+}
+
+static inline struct io_group *
+io_full_io_group(struct request_queue *q, int rw)
+{
+     return NULL;
+}
+
+static inline struct io_group *__io_get_io_group(struct request_queue *q)
+{
+     return NULL;
+}
+
+static inline struct io_group *io_request_io_group(struct request *rq)
+{
+     return NULL;
+}
+
+static inline unsigned long io_group_nr_requests(struct io_group *iog)
+{
+     return 0;
+}
+
+static inline int io_group_inc_count(struct io_group *iog, int rw)
+{
+     return 0;
+}
+
+static inline int io_group_dec_count(struct io_group *iog, int rw)
+{
+     return 0;
+}
+
+static inline int io_group_count(struct io_group *iog, int rw)
+{
+     return 0;
+}
+
+static inline int io_group_congestion_on(struct io_group *iog, int rw)
+{
+     return 0;
+}
+
+static inline int io_group_congestion_off(struct io_group *iog, int rw)
+{
+     return 1;
+}
+
+static inline int io_group_full(struct io_group *iog, int rw)
+{
+     return 0;
+}
 #endif /* CONFIG_ELV_FAIR_QUEUING */
 #endif /* _BFQ_SCHED_H */
--

1.5.4.3

--
IKEDA, Munehiro
NEC Corporation of America
  [email protected]

A message that is not in the archive (sent off-list or not yet received): e98e18940905011639o63c048f1n79c7e7648441a06d-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org

#111 lore raw

Re: IO Controller per cgroup request descriptors (Re: [PATCH 01/10] Documentation)

Quoted 36 lines · Nauman Rafique
Quoted 35 lines · Vivek Goyal
Quoted 34 lines · Munehiro
Quoted 23 lines · Vivek Goyal
Quoted 7 lines

Review

Quoted 6 lines
(quoted diff) +6 -0
+TODO
+====
+- Lots of cleanups, testing, bug fixing, optimizations, benchmarking etc...
+- Convert cgroup ioprio to notion of weight.
+- Anticipatory code will need more work. It is not working properly currently
+  and needs more thought.

What are the problems with the code?

Have not got a chance to look into the issues in detail yet. Just a crude run saw drop in performance. Will debug it later the moment I have got async writes handled...

Quoted 6 lines
Quoted 1 line

+- Use of bio-cgroup patches.

I saw these posted as well

Quoted 2 lines

+- Use of Nauman's per cgroup request descriptor patches. +

More details would be nice, I am not sure I understand

Currently the number of request descriptors which can be allocated per device/request queue are fixed by a sysfs tunable (q->nr_requests). So if there is lots of IO going on from one cgroup then it will consume all the available request descriptors and other cgroup might starve and not get its fair share.

Quoted 3 lines

Hence we also need to introduce the notion of request descriptor limit per cgroup so that if request descriptors from one group are exhausted, then it does not impact the IO of other cgroup.

Unfortunately I couldn't find and I've never seen the Nauman's patches. So I tried to make a patch below against this todo. The reason why I'm posting this despite this is just a quick and ugly hack (and it might be a reinvention of wheel) is that I would like to discuss how we should define the limitation of requests per cgroup.
This patch should be applied on Vivek's I/O controller patches posted on Mar 11.

Quoted 7 lines
Quoted 4 lines

Hi IKEDA,

Sorry for the confusion here. Actually Nauman had sent a patch to select group of people who were initially copied on the mail thread.

I am sorry about that. Since I dropped my whole patch set in favor of Vivek's stuff, this stuff fell through the cracks.

No problem at all guys. I'm glad to see your patch Vivek sent, thanks.

Quoted 39 lines
Quoted 30 lines
Quoted 7 lines

This patch temporarily distribute q->nr_requests to each cgroup. I think the number should be weighted like BFQ's budget. But in this case, if the hierarchy of cgroup is deep, leaf cgroups are allowed to allocate very few numbers of requests. I don't think this is reasonable...but I don't have specific idea to solve this problem. Does anyone have the good idea?

Thanks for the patch. Yes, ideally one would expect the request descriptor to be allocated also in proportion to the weight but I guess that would become very comlicated.

In terms of simpler things, two thoughts come to mind.

- First approach is to make q->nr_requests per group. So every group is entitled for q->nr_requests as set by the user. This is what your patch seems to have done.

I had some concerns with this approach. First of all it does not seem to have an upper bound on number of request descriptors allocated per queue because if a user creates more cgroups, total number of request descriptors increase.

- Second approach can be that we retain the meaning of q->nr_requests which defines the total number of request descriptors on the queue (with the exception of 50% more descriptors for batching processes). And we define a new per group limit q->nr_group_requests which defines how many requests per group can be assigned. So q->nr_requests defines total pool size on the queue and q->nr_group_requests will define how many requests each group can allocate out of that pool.

Quoted 8 lines

Here the issue is that a user shall have to balance the q->nr_group_requests and q->nr_requests properly.

To experiment, I have implemented the second approach. I am attaching the patch which is in my current tree. It probably will not apply on my march 11 posting as since then patches have changed. But posting it here so that at least it will give an idea behind the thought process.

Ideas are welcome...

Quoted 7 lines

I had started with the first option, but the second option sounds good too. But one problem that comes to mind is how we deal with hierarchies? The sys admin can limit the root level cgroups to specific number of request descriptors, but if applications running in a cgroup are allowed to create their own cgroups, then the total request descriptors of all child cgroups should be capped by the number assigned to parent cgroups.

I think the second option cannot coexist with hierarchy support of per cgroup request descriptors limitation. I guess the fundamental idea of the second approach is to make logic simpler by giving up hierarchy, isn't it correct?
IIUC, for hierarchy support, we need to have some good idea to solve the issue that a cgroup which belongs to deep hierarchy can have only few requests, as I mentioned.
Anyway, I keep my eyes on this issue. I'm looking forward to Vivek's next version patches.

(snip)

Quoted 13 lines
Quoted 13 lines
block/blk-sysfs.c +13 -0
+#ifdef CONFIG_GROUP_IOSCHED
+static ssize_t queue_group_requests_show(struct request_queue *q, char *page)
+{
+       return queue_var_show(q->nr_group_requests, (page));
+}
+
+static ssize_t
+queue_group_requests_store(struct request_queue *q, const char *page,
+                                       size_t count)
+{
+       unsigned long nr;
+       int ret = queue_var_store(&nr, page, count);
+       if (nr < BLKDEV_MIN_RQ)
Quoted 8 lines

block/blk-sysfs.c queue_requests_store(struct request_queu · line 99

Quoted 8 lines
block/blk-sysfs.c +8 -0
+               nr = BLKDEV_MIN_RQ;
+
+       spin_lock_irq(q->queue_lock);
+       q->nr_group_requests = nr;
+       spin_unlock_irq(q->queue_lock);
+       return ret;
+}
+#endif

(snip)

Unchanging io_context "batching" status is on purpose?

Thanks,
Muuhh

Signature · 3 lines
IKEDA, Munehiro
  NEC Corporation of America
    m-ikeda-MDRzhb/z0dd8UrSeD/[email protected]
#112 lore raw

Re: IO Controller per cgroup request descriptors (Re: [PATCH 01/10] Documentation)

In reply to Nauman Rafique
Quoted 36 lines · Nauman Rafique
Quoted 35 lines · Vivek Goyal
Quoted 34 lines · Munehiro
Quoted 23 lines · Vivek Goyal
Quoted 7 lines

Review

Quoted 6 lines
(quoted diff) +6 -0
+TODO
+====
+- Lots of cleanups, testing, bug fixing, optimizations, benchmarking etc...
+- Convert cgroup ioprio to notion of weight.
+- Anticipatory code will need more work. It is not working properly currently
+  and needs more thought.

What are the problems with the code?

Have not got a chance to look into the issues in detail yet. Just a crude run saw drop in performance. Will debug it later the moment I have got async writes handled...

Quoted 6 lines
Quoted 1 line

+- Use of bio-cgroup patches.

I saw these posted as well

Quoted 2 lines

+- Use of Nauman's per cgroup request descriptor patches. +

More details would be nice, I am not sure I understand

Currently the number of request descriptors which can be allocated per device/request queue are fixed by a sysfs tunable (q->nr_requests). So if there is lots of IO going on from one cgroup then it will consume all the available request descriptors and other cgroup might starve and not get its fair share.

Quoted 3 lines

Hence we also need to introduce the notion of request descriptor limit per cgroup so that if request descriptors from one group are exhausted, then it does not impact the IO of other cgroup.

Unfortunately I couldn't find and I've never seen the Nauman's patches. So I tried to make a patch below against this todo. The reason why I'm posting this despite this is just a quick and ugly hack (and it might be a reinvention of wheel) is that I would like to discuss how we should define the limitation of requests per cgroup.
This patch should be applied on Vivek's I/O controller patches posted on Mar 11.

Quoted 7 lines
Quoted 4 lines

Hi IKEDA,

Sorry for the confusion here. Actually Nauman had sent a patch to select group of people who were initially copied on the mail thread.

I am sorry about that. Since I dropped my whole patch set in favor of Vivek's stuff, this stuff fell through the cracks.

No problem at all guys. I'm glad to see your patch Vivek sent, thanks.

Quoted 39 lines
Quoted 30 lines
Quoted 7 lines

This patch temporarily distribute q->nr_requests to each cgroup. I think the number should be weighted like BFQ's budget. But in this case, if the hierarchy of cgroup is deep, leaf cgroups are allowed to allocate very few numbers of requests. I don't think this is reasonable...but I don't have specific idea to solve this problem. Does anyone have the good idea?

Thanks for the patch. Yes, ideally one would expect the request descriptor to be allocated also in proportion to the weight but I guess that would become very comlicated.

In terms of simpler things, two thoughts come to mind.

- First approach is to make q->nr_requests per group. So every group is entitled for q->nr_requests as set by the user. This is what your patch seems to have done.

I had some concerns with this approach. First of all it does not seem to have an upper bound on number of request descriptors allocated per queue because if a user creates more cgroups, total number of request descriptors increase.

- Second approach can be that we retain the meaning of q->nr_requests which defines the total number of request descriptors on the queue (with the exception of 50% more descriptors for batching processes). And we define a new per group limit q->nr_group_requests which defines how many requests per group can be assigned. So q->nr_requests defines total pool size on the queue and q->nr_group_requests will define how many requests each group can allocate out of that pool.

Quoted 8 lines

Here the issue is that a user shall have to balance the q->nr_group_requests and q->nr_requests properly.

To experiment, I have implemented the second approach. I am attaching the patch which is in my current tree. It probably will not apply on my march 11 posting as since then patches have changed. But posting it here so that at least it will give an idea behind the thought process.

Ideas are welcome...

Quoted 7 lines

I had started with the first option, but the second option sounds good too. But one problem that comes to mind is how we deal with hierarchies? The sys admin can limit the root level cgroups to specific number of request descriptors, but if applications running in a cgroup are allowed to create their own cgroups, then the total request descriptors of all child cgroups should be capped by the number assigned to parent cgroups.

I think the second option cannot coexist with hierarchy support of per cgroup request descriptors limitation. I guess the fundamental idea of the second approach is to make logic simpler by giving up hierarchy, isn't it correct?
IIUC, for hierarchy support, we need to have some good idea to solve the issue that a cgroup which belongs to deep hierarchy can have only few requests, as I mentioned.
Anyway, I keep my eyes on this issue. I'm looking forward to Vivek's next version patches.

(snip)

Quoted 13 lines
Quoted 13 lines
block/blk-sysfs.c +13 -0
+#ifdef CONFIG_GROUP_IOSCHED
+static ssize_t queue_group_requests_show(struct request_queue *q, char *page)
+{
+       return queue_var_show(q->nr_group_requests, (page));
+}
+
+static ssize_t
+queue_group_requests_store(struct request_queue *q, const char *page,
+                                       size_t count)
+{
+       unsigned long nr;
+       int ret = queue_var_store(&nr, page, count);
+       if (nr < BLKDEV_MIN_RQ)
Quoted 8 lines

block/blk-sysfs.c queue_requests_store(struct request_queu · line 99

Quoted 8 lines
block/blk-sysfs.c +8 -0
+               nr = BLKDEV_MIN_RQ;
+
+       spin_lock_irq(q->queue_lock);
+       q->nr_group_requests = nr;
+       spin_unlock_irq(q->queue_lock);
+       return ret;
+}
+#endif

(snip)

Unchanging io_context "batching" status is on purpose?

Thanks,
Muuhh

Signature · 3 lines
IKEDA, Munehiro
 NEC Corporation of America
   [email protected]

A message that is not in the archive (sent off-list or not yet received): 1236823015-4183-1-git-send-email-vgoyal-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org

#114 lore raw

PATCH 2/10 Common flat fair queuing code in elevaotor layer

This is common fair queuing code in elevator layer. This is controlled by config option CONFIG_ELV_FAIR_QUEUING. This patch initially only introduces flat fair queuing support where there is only one group, "root group" and all the tasks belong to root group.

This elevator layer changes are backward compatible. That means any ioscheduler using old interfaces will continue to work.

---
 block/Kconfig.iosched    |   13 +
 block/Makefile           |    1 +
 block/blk-sysfs.c        |   10 +
 block/elevator-fq.c      | 1882 ++++++++++++++++++++++++++++++++++++++++++++++
 block/elevator-fq.h      |  479 ++++++++++++
 block/elevator.c         |   46 +-
 include/linux/blkdev.h   |    5 +
 include/linux/elevator.h |   48 ++
 8 files changed, 2473 insertions(+), 11 deletions(-)
 create mode 100644 block/elevator-fq.c
 create mode 100644 block/elevator-fq.h
block/Kconfig.iosched +13 -0
index 7e803fc..3398134 100644
@@ -2,6 +2,19 @@ if BLOCK
 
 menu "IO Schedulers"
 
+config ELV_FAIR_QUEUING
+	bool "Elevator Fair Queuing Support"
+	default n
+	---help---
+	  Traditionally only cfq had notion of multiple queues and it did
+	  fair queuing at its own. With the cgroups and need of controlling
+	  IO, now even the simple io schedulers like noop, deadline, as will
+	  have one queue per cgroup and will need hierarchical fair queuing.
+	  Instead of every io scheduler implementing its own fair queuing
+	  logic, this option enables fair queuing in elevator layer so that
+	  other ioschedulers can make use of it.
+	  If unsure, say N.
+
 config IOSCHED_NOOP
 	bool
 	default y
block/Makefile +1 -0
index bfe7304..6f410d5 100644
@@ -16,3 +16,4 @@ obj-$(CONFIG_IOSCHED_CFQ)	+= cfq-iosched.o
 obj-$(CONFIG_BLK_DEV_IO_TRACE)	+= blktrace.o
 obj-$(CONFIG_BLOCK_COMPAT)	+= compat_ioctl.o
 obj-$(CONFIG_BLK_DEV_INTEGRITY)	+= blk-integrity.o
+obj-$(CONFIG_ELV_FAIR_QUEUING)	+= elevator-fq.o
block/blk-sysfs.c +10 -0
index e29ddfc..0d98c96 100644
@@ -276,6 +276,13 @@ static struct queue_sysfs_entry queue_iostats_entry = {
 	.store = queue_iostats_store,
 };
 
+#ifdef CONFIG_ELV_FAIR_QUEUING
+static struct queue_sysfs_entry queue_slice_idle_entry = {
+	.attr = {.name = "slice_idle", .mode = S_IRUGO | S_IWUSR },
+	.show = elv_slice_idle_show,
+	.store = elv_slice_idle_store,
+};
+#endif
 static struct attribute *default_attrs[] = {
 	&queue_requests_entry.attr,
 	&queue_ra_entry.attr,
@@ -287,6 +294,9 @@ static struct attribute *default_attrs[] = {
 	&queue_nomerges_entry.attr,
 	&queue_rq_affinity_entry.attr,
 	&queue_iostats_entry.attr,
+#ifdef CONFIG_ELV_FAIR_QUEUING
+	&queue_slice_idle_entry.attr,
+#endif
 	NULL,
 };
 
block/elevator-fq.c new file +1882 -0

1,882 lines not on this page · raw message

block/elevator-fq.h new file +479 -0

479 lines not on this page · raw message

block/elevator.c +35 -11

116 lines not on this page · raw message

include/linux/blkdev.h +5 -0
index 465d6ba..cf02216 100644
@@ -234,6 +234,11 @@ struct request {
 
 	/* for bidi */
 	struct request *next_rq;
+
+#ifdef CONFIG_ELV_FAIR_QUEUING
+	/* io queue request belongs to */
+	struct io_queue *ioq;
+#endif
 };
 
 static inline unsigned short req_get_ioprio(struct request *req)
include/linux/elevator.h +48 -0

83 lines not on this page · raw message

Signature · 1 line
1.6.0.1
#115 lore raw

PATCH 3/10 Modify cfq to make use of flat elevator fair queuing

In reply to a message not in the archive

This patch changes cfq to use fair queuing code from elevator layer.

o must_dispatch logic sounds like a dead code. Nobody seems to be making use of that flag. Retaining it for the time being.

---
 block/Kconfig.iosched |    3 +-
 block/cfq-iosched.c   | 1082 +++++++++++--------------------------------------
 2 files changed, 232 insertions(+), 853 deletions(-)
block/Kconfig.iosched +2 -1
index 3398134..dd5224d 100644
@@ -3,7 +3,7 @@ if BLOCK
 menu "IO Schedulers"
 
 config ELV_FAIR_QUEUING
-	bool "Elevator Fair Queuing Support"
+	bool
 	default n
 	---help---
 	  Traditionally only cfq had notion of multiple queues and it did
@@ -46,6 +46,7 @@ config IOSCHED_DEADLINE
 
 config IOSCHED_CFQ
 	tristate "CFQ I/O scheduler"
+	select ELV_FAIR_QUEUING
 	default y
 	---help---
 	  The CFQ I/O scheduler tries to distribute bandwidth equally
block/cfq-iosched.c +230 -852

1,757 lines not on this page · raw message

Signature · 1 line
1.6.0.1
#116 lore raw

PATCH 7/10 Prepare elevator layer for single queue schedulers

In reply to a message not in the archive

Elevator layer now has support for hierarchical fair queuing. cfq has been migrated to make use of it and now it is time to do groundwork for noop, deadline and AS.

noop deadline and AS don't maintain separate queues for different processes. There is only one single queue. Effectively one can think that in hierarchical setup, there will be one queue per cgroup where requests from all the processes in the cgroup will be queued.

Generally io scheduler takes care of creating queues. Because there is only one queue here, we have modified common layer to take care of queue creation and some other functionality. This special casing helps in keeping the changes to noop, deadline and AS to the minimum.

---
 block/elevator-fq.c      |  153 ++++++++++++++++++++++++++++++++++++++++++++++
 block/elevator-fq.h      |   67 ++++++++++++++++++++
 block/elevator.c         |   35 ++++++++++-
 include/linux/elevator.h |   14 ++++
 4 files changed, 268 insertions(+), 1 deletions(-)
block/elevator-fq.c +153 -0
index 389f68e..172f9e3 100644
@@ -857,6 +857,12 @@ void io_put_io_group_queues(struct elevator_queue *e, struct io_group *iog)
 
 	/* Free up async idle queue */
 	elv_release_ioq(e, &iog->async_idle_queue);
+
+#ifdef CONFIG_GROUP_IOSCHED
+	/* Optimization for io schedulers having single ioq */
+	if (elv_iosched_single_ioq(e))
+		elv_release_ioq(e, &iog->ioq);
+#endif
 }
 
 
@@ -1538,6 +1544,153 @@ void elv_fq_set_request_io_group(struct request_queue *q,
 	rq->iog = iog;
 }
 
+/*
+ * Find/Create the io queue the rq should go in. This is an optimization
+ * for the io schedulers (noop, deadline and AS) which maintain only single
+ * io queue per cgroup. In this case common layer can just maintain a
+ * pointer in group data structure and keeps track of it.
+ *
+ * For the io schdulers like cfq, which maintain multiple io queues per
+ * cgroup, and decide the io queue  of request based on process, this
+ * function is not invoked.
+ */
+int elv_fq_set_request_ioq(struct request_queue *q, struct request *rq,
+					gfp_t gfp_mask)
+{
+	struct elevator_queue *e = q->elevator;
+	unsigned long flags;
+	struct io_queue *ioq = NULL, *new_ioq = NULL;
+	struct io_group *iog;
+	void *sched_q = NULL, *new_sched_q = NULL;
+
+	if (!elv_iosched_fair_queuing_enabled(e))
+		return 0;
+
+	might_sleep_if(gfp_mask & __GFP_WAIT);
+	spin_lock_irqsave(q->queue_lock, flags);
+
+	/* Determine the io group request belongs to */
+	iog = rq->iog;
+	BUG_ON(!iog);
+
+retry:
+	/* Get the iosched queue */
+	ioq = io_group_ioq(iog);
+	if (!ioq) {
+		/* io queue and sched_queue needs to be allocated */
+		BUG_ON(!e->ops->elevator_alloc_sched_queue_fn);
+
+		if (new_sched_q) {
+			goto alloc_ioq;
+		} else if (gfp_mask & __GFP_WAIT) {
+			/*
+			 * Inform the allocator of the fact that we will
+			 * just repeat this allocation if it fails, to allow
+			 * the allocator to do whatever it needs to attempt to
+			 * free memory.
+			 */
+			spin_unlock_irq(q->queue_lock);
+			/* Call io scheduer to create scheduler queue */
+			new_sched_q = e->ops->elevator_alloc_sched_queue_fn(q,
+					e, gfp_mask | __GFP_NOFAIL
+					| __GFP_ZERO);
+			spin_lock_irq(q->queue_lock);
+			goto retry;
+		} else {
+			sched_q = e->ops->elevator_alloc_sched_queue_fn(q, e,
+						gfp_mask | __GFP_ZERO);
+			if (!sched_q)
+				goto queue_fail;
+		}
+
+alloc_ioq:
+		if (new_ioq) {
+			ioq = new_ioq;
+			new_ioq = NULL;
+			sched_q = new_sched_q;
+			new_sched_q = NULL;
+		} else if (gfp_mask & __GFP_WAIT) {
+			/*
+			 * Inform the allocator of the fact that we will
+			 * just repeat this allocation if it fails, to allow
+			 * the allocator to do whatever it needs to attempt to
+			 * free memory.
+			 */
+			spin_unlock_irq(q->queue_lock);
+			new_ioq = elv_alloc_ioq(q, gfp_mask | __GFP_NOFAIL
+							| __GFP_ZERO);
+			spin_lock_irq(q->queue_lock);
+			goto retry;
+		} else {
+			ioq = elv_alloc_ioq(q, gfp_mask | __GFP_ZERO);
+			if (!ioq) {
+				e->ops->elevator_free_sched_queue_fn(e,
+							sched_q);
+				sched_q = NULL;
+				goto queue_fail;
+			}
+		}
+
+		elv_init_ioq(e, ioq, sched_q, IOPRIO_CLASS_BE, 4, 1);
+		io_group_set_ioq(iog, ioq);
+		elv_mark_ioq_sync(ioq);
+	}
+
+	if (new_sched_q)
+		e->ops->elevator_free_sched_queue_fn(q->elevator, sched_q);
+
+	if (new_ioq)
+		elv_free_ioq(new_ioq);
+
+	/* Request reference */
+	elv_get_ioq(ioq);
+	rq->ioq = ioq;
+	spin_unlock_irqrestore(q->queue_lock, flags);
+	return 0;
+
+queue_fail:
+	WARN_ON((gfp_mask & __GFP_WAIT) && !ioq);
+	elv_schedule_dispatch(q);
+	spin_unlock_irqrestore(q->queue_lock, flags);
+	return 1;
+}
+
+/*
+ * Find out the io queue of current task. Optimization for single ioq
+ * per io group io schedulers.
+ */
+struct io_queue *elv_lookup_ioq_current(struct request_queue *q)
+{
+	struct io_group *iog;
+
+	/* Determine the io group and io queue of the bio submitting task */
+	iog = io_lookup_io_group_current(q);
+	if (!iog) {
+		/* May be task belongs to a cgroup for which io group has
+		 * not been setup yet. */
+		return NULL;
+	}
+	return io_group_ioq(iog);
+}
+
+/*
+ * This request has been serviced. Clean up ioq info and drop the reference.
+ * Again this is called only for single queue per cgroup schedulers (noop,
+ * deadline, AS).
+ */
+void elv_fq_unset_request_ioq(struct request_queue *q, struct request *rq)
+{
+	struct io_queue *ioq = rq->ioq;
+
+	if (!elv_iosched_fair_queuing_enabled(q->elevator))
+		return;
+
+	if (ioq) {
+		rq->ioq = NULL;
+		elv_put_ioq(ioq);
+	}
+}
+
 #else /* GROUP_IOSCHED */
 void bfq_init_entity(struct io_entity *entity, struct io_group *iog)
 {
block/elevator-fq.h +67 -0

90 lines not on this page · raw message

block/elevator.c +34 -1
index 5df13c4..bce4421 100644
@@ -207,6 +207,14 @@ static void *elevator_alloc_sched_queue(struct request_queue *q,
 {
 	void *sched_queue = NULL;
 
+	/*
+	 * If fair queuing is enabled, then queue allocation takes place
+	 * during set_request() functions when request actually comes
+	 * in.
+	 */
+	if (elv_iosched_fair_queuing_enabled(eq))
+		return NULL;
+
 	if (eq->ops->elevator_alloc_sched_queue_fn) {
 		sched_queue = eq->ops->elevator_alloc_sched_queue_fn(q, eq,
 								GFP_KERNEL);
@@ -936,6 +944,13 @@ int elv_set_request(struct request_queue *q, struct request *rq, gfp_t gfp_mask)
 
 	elv_fq_set_request_io_group(q, rq);
 
+	/*
+	 * Optimization for noop, deadline and AS which maintain only single
+	 * ioq per io group
+	 */
+	if (elv_iosched_single_ioq(e))
+		return elv_fq_set_request_ioq(q, rq, gfp_mask);
+
 	if (e->ops->elevator_set_req_fn)
 		return e->ops->elevator_set_req_fn(q, rq, gfp_mask);
 
@@ -947,6 +962,15 @@ void elv_put_request(struct request_queue *q, struct request *rq)
 {
 	struct elevator_queue *e = q->elevator;
 
+	/*
+	 * Optimization for noop, deadline and AS which maintain only single
+	 * ioq per io group
+	 */
+	if (elv_iosched_single_ioq(e)) {
+		elv_fq_unset_request_ioq(q, rq);
+		return;
+	}
+
 	if (e->ops->elevator_put_req_fn)
 		e->ops->elevator_put_req_fn(rq);
 }
@@ -1329,9 +1353,18 @@ EXPORT_SYMBOL(elv_select_sched_queue);
 
 /*
  * Get the io scheduler queue pointer for current task.
+ *
+ * If fair queuing is enabled, determine the io group of task and retrieve
+ * the ioq pointer from that. This is used by only single queue ioschedulers
+ * for retrieving the queue associated with the group to decide whether the
+ * new bio can do a front merge or not.
  */
 void *elv_get_sched_queue_current(struct request_queue *q)
 {
-	return q->elevator->sched_queue;
+	/* Fair queuing is not enabled. There is only one queue. */
+	if (!elv_iosched_fair_queuing_enabled(q->elevator))
+		return q->elevator->sched_queue;
+
+	return ioq_sched_queue(elv_lookup_ioq_current(q));
 }
 EXPORT_SYMBOL(elv_get_sched_queue_current);
include/linux/elevator.h +14 -0
index bb5ae3a..8cee877 100644
@@ -245,17 +245,31 @@ enum {
 /* iosched wants to use fq logic of elevator layer */
 #define	ELV_IOSCHED_NEED_FQ	1
 
+/* iosched maintains only single ioq per group.*/
+#define ELV_IOSCHED_SINGLE_IOQ        2
+
 static inline int elv_iosched_fair_queuing_enabled(struct elevator_queue *e)
 {
 	return (e->elevator_type->elevator_features) & ELV_IOSCHED_NEED_FQ;
 }
 
+static inline int elv_iosched_single_ioq(struct elevator_queue *e)
+{
+	return (e->elevator_type->elevator_features) & ELV_IOSCHED_SINGLE_IOQ;
+}
+
 #else /* ELV_IOSCHED_FAIR_QUEUING */
 
 static inline int elv_iosched_fair_queuing_enabled(struct elevator_queue *e)
 {
 	return 0;
 }
+
+static inline int elv_iosched_single_ioq(struct elevator_queue *e)
+{
+	return 0;
+}
+
 #endif /* ELV_IOSCHED_FAIR_QUEUING */
 extern void *elv_get_sched_queue(struct request_queue *q, struct request *rq);
 extern void *elv_select_sched_queue(struct request_queue *q, int force);
Signature · 1 line
1.6.0.1
#117 lore raw
In reply to a message not in the archive

Hi Vivek,

Could you tell me to which kernel I can apply your patches?

   # latest mm?
I would like to test your controller.

Thank you,
Takuya Yoshikawa

Quoted 49 lines · Vivek Goyal

Hi All,

Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.

http://lkml.org/lkml/2008/11/6/227

One of the takeaway from the discussion in this thread was that let us implement a common layer which contains the proportional weight scheduling code which can be shared by all the IO schedulers.

Implementing IO controller will not cover the devices which don't use IO schedulers but it should cover the common case.

There were more discussions regarding 2 level vs 1 level IO control at following link.

https://lists.linux-foundation.org/pipermail/containers/2009-January/015402.html

So in the mean time we took the discussion off the list and spent time on making the 1 level control apporoach work where majority of the proportional weight control is shared by the four schedulers instead of each one having to replicate the code. We make use of BFQ code for fair queuing as posted by Paolo and Fabio here.

http://lkml.org/lkml/2008/11/11/148

Details about design and howto have been put in documentation patch.

I have done very basic testing of running 2 or 3 "dd" threads in different cgroups. Wanted to get the patchset out for feedback/review before we dive into more bug fixing, benchmarking, optimizations etc.

Your feedback/comments are welcome.

Patch series contains 10 patches. It should be compilable and bootable after every patch. Intial 2 patches implement flat fair queuing (no cgroup support) and make cfq to use that. Later patches introduce hierarchical fair queuing support in elevator layer and modify other IO schdulers to use that.

Thanks
Vivek
_______________________________________________
Containers mailing list Containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
https://lists.linux-foundation.org/mailman/listinfo/containers

#118 lore raw
In reply to a message not in the archive
Quoted 11 lines · Vivek Goyal

Hi All,

Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.

http://lkml.org/lkml/2008/11/6/227

One of the takeaway from the discussion in this thread was that let us implement a common layer which contains the proportional weight scheduling code which can be shared by all the IO schedulers.

Hi Vivek,

I did some tests on my *old* i386 box(with two concurrent dd running), and notice that IO Controller doesn't work fine in such situation. But it can work perfectly in my *new* x86 box. I dig into this problem, and i guess the major reason is that my *old* i386 box is too slow, it can't ensure two running ioqs are always backlogged. If that is the case, I happens to have a thought. when an ioq uses up it time slice, we don't expire it immediately. May be we can give a piece of bonus time for idling to wait new requests if this ioq's finish time and its ancestor's finish time are all much smaller than other entities in each corresponding service tree.

Signature · 2 lines
Regards
Gui Jianfeng
#119 lore raw
In reply to a message not in the archive
Quoted 4 lines · Vivek Goyal

Hi All,

Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.

Hi Vivek,

I got the following OOPS when testing, can't reproduce again :(

kernel BUG at block/elevator-fq.c:1396!
invalid opcode: 0000 [#1] SMP
last sysfs file: /sys/block/hdb/queue/scheduler

Modules linked in: ipv6 cpufreq_ondemand acpi_cpufreq dm_mirror dm_multipath sbd

Pid: 5032, comm: rmdir Not tainted (2.6.29-rc7-vivek #17) Veriton M460
EIP: 0060:[<c04ec7de>] EFLAGS: 00010082 CPU: 0
EIP is at iocg_destroy+0xdc/0x14e
EAX: 00000000 EBX: f62278b4 ECX: f6207800 EDX: f6227904
ESI: f6227800 EDI: f62278a0 EBP: 00000003 ESP: c8790f00
DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
Process rmdir (pid: 5032, ti=c8790000 task=f6636960 task.ti=c8790000)

Stack:
 f53cc5c0 f62b7258 f10991c8 00000282 f6227800 c0733fa0 ec1c5140 edfc6d34
 c8790000 c04463ce f6a4c84c edfc6d34 0804c840 c048883c f6a4c84c f6a4c84c
 c04888d6 f6a4c84c 00000000 c04897de f6a4c84c c048504c f115f4c0 ebc37954
Call Trace:
 [<c04463ce>] cgroup_diput+0x41/0x8c
 [<c048883c>] dentry_iput+0x45/0x5e
 [<c04888d6>] d_kill+0x19/0x32
 [<c04897de>] dput+0xd8/0xdf
 [<c048504c>] do_rmdir+0x8f/0xb6
 [<c06330fc>] do_page_fault+0x2a2/0x579
 [<c0402fc1>] sysenter_do_call+0x12/0x21
 [<c0630000>] schedule+0x641/0x830
Code: 08 00 74 04 0f 0b eb fe 83 7f 04 00 74 04 0f 0b eb fe 45 83 c3 1c 83 fd 0
EIP: [<c04ec7de>] iocg_destroy+0xdc/0x14e SS:ESP 0068:c8790f00
Signature · 2 lines
Regards
Gui Jianfeng
#120 lore raw
In reply to a message not in the archive
Quoted 35 lines · Vivek Goyal

Hi All,

Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.

http://lkml.org/lkml/2008/11/6/227

One of the takeaway from the discussion in this thread was that let us implement a common layer which contains the proportional weight scheduling code which can be shared by all the IO schedulers.

Implementing IO controller will not cover the devices which don't use IO schedulers but it should cover the common case.

There were more discussions regarding 2 level vs 1 level IO control at following link.

https://lists.linux-foundation.org/pipermail/containers/2009-January/015402.html

So in the mean time we took the discussion off the list and spent time on making the 1 level control apporoach work where majority of the proportional weight control is shared by the four schedulers instead of each one having to replicate the code. We make use of BFQ code for fair queuing as posted by Paolo and Fabio here.

http://lkml.org/lkml/2008/11/11/148

Details about design and howto have been put in documentation patch.

I have done very basic testing of running 2 or 3 "dd" threads in different cgroups. Wanted to get the patchset out for feedback/review before we dive into more bug fixing, benchmarking, optimizations etc.

Your feedback/comments are welcome.

Quoted 8 lines

Patch series contains 10 patches. It should be compilable and bootable after every patch. Intial 2 patches implement flat fair queuing (no cgroup support) and make cfq to use that. Later patches introduce hierarchical fair queuing support in elevator layer and modify other IO schdulers to use that.

Thanks
Vivek

Hi Vivek,
While testing these patches along with the bio-cgroup patches I noticed that for the case of 2 buffered writers (dd) with different weights, one of them would be able to use up a very large timeslice (I've seen upto 500ms) when the other queue is empty and not be accounted for it. This is due to the check in cfq_dispatch_requests() where a given cgroup can empty its entire queue (100 IOs or more) within its timeslice and have them sit in the dispatch queue ready for the disk driver to pick up. Moreover, this huge timeslice is not accounted for as this cgroup is charged only for the length of the intended timeslice and not the actual time taken.
The following patch fixes this by not optimizing on the single busy queue fact inside cfq_dispatch_requests. Note that this does not hurt throughput in any sense but just causes more IOs to be dispatched only when the drive is ready for them thus leading to better accounting too.

Fix bug where a given ioq can run through all its requests at once.

---

2.6.26/block/cfq-iosched.c +0 -0
index 5a275a2..c0199a6 100644
@@ -848,8 +848,7 @@ static int cfq_dispatch_requests(struct request_queue *q, int force)
		if (cfq_class_idle(cfqq))
			max_dispatch = 1;

- if (elv_ioq_nr_dispatched(cfqq->ioq) >= max_dispatch &&
- elv_nr_busy_ioq(q->elevator) > 1)
+ if (elv_ioq_nr_dispatched(cfqq->ioq) >= max_dispatch)

			break;

		if (cfqd->sync_flight && !cfq_cfqq_sync(cfqq))
#121 lore raw

PATCH 2/10 Common flat fair queuing code in elevaotor layer

In reply to Vivek Goyal

This is common fair queuing code in elevator layer. This is controlled by config option CONFIG_ELV_FAIR_QUEUING. This patch initially only introduces flat fair queuing support where there is only one group, "root group" and all the tasks belong to root group.

This elevator layer changes are backward compatible. That means any ioscheduler using old interfaces will continue to work.

---
 block/Kconfig.iosched    |   13 +
 block/Makefile           |    1 +
 block/blk-sysfs.c        |   10 +
 block/elevator-fq.c      | 1882 ++++++++++++++++++++++++++++++++++++++++++++++
 block/elevator-fq.h      |  479 ++++++++++++
 block/elevator.c         |   46 +-
 include/linux/blkdev.h   |    5 +
 include/linux/elevator.h |   48 ++
 8 files changed, 2473 insertions(+), 11 deletions(-)
 create mode 100644 block/elevator-fq.c
 create mode 100644 block/elevator-fq.h
block/Kconfig.iosched +13 -0
index 7e803fc..3398134 100644
@@ -2,6 +2,19 @@ if BLOCK
 
 menu "IO Schedulers"
 
+config ELV_FAIR_QUEUING
+	bool "Elevator Fair Queuing Support"
+	default n
+	---help---
+	  Traditionally only cfq had notion of multiple queues and it did
+	  fair queuing at its own. With the cgroups and need of controlling
+	  IO, now even the simple io schedulers like noop, deadline, as will
+	  have one queue per cgroup and will need hierarchical fair queuing.
+	  Instead of every io scheduler implementing its own fair queuing
+	  logic, this option enables fair queuing in elevator layer so that
+	  other ioschedulers can make use of it.
+	  If unsure, say N.
+
 config IOSCHED_NOOP
 	bool
 	default y
block/Makefile +1 -0
index bfe7304..6f410d5 100644
@@ -16,3 +16,4 @@ obj-$(CONFIG_IOSCHED_CFQ)	+= cfq-iosched.o
 obj-$(CONFIG_BLK_DEV_IO_TRACE)	+= blktrace.o
 obj-$(CONFIG_BLOCK_COMPAT)	+= compat_ioctl.o
 obj-$(CONFIG_BLK_DEV_INTEGRITY)	+= blk-integrity.o
+obj-$(CONFIG_ELV_FAIR_QUEUING)	+= elevator-fq.o
block/blk-sysfs.c +10 -0
index e29ddfc..0d98c96 100644
@@ -276,6 +276,13 @@ static struct queue_sysfs_entry queue_iostats_entry = {
 	.store = queue_iostats_store,
 };
 
+#ifdef CONFIG_ELV_FAIR_QUEUING
+static struct queue_sysfs_entry queue_slice_idle_entry = {
+	.attr = {.name = "slice_idle", .mode = S_IRUGO | S_IWUSR },
+	.show = elv_slice_idle_show,
+	.store = elv_slice_idle_store,
+};
+#endif
 static struct attribute *default_attrs[] = {
 	&queue_requests_entry.attr,
 	&queue_ra_entry.attr,
@@ -287,6 +294,9 @@ static struct attribute *default_attrs[] = {
 	&queue_nomerges_entry.attr,
 	&queue_rq_affinity_entry.attr,
 	&queue_iostats_entry.attr,
+#ifdef CONFIG_ELV_FAIR_QUEUING
+	&queue_slice_idle_entry.attr,
+#endif
 	NULL,
 };
 
block/elevator-fq.c new file +1882 -0

1,882 lines not on this page · raw message

block/elevator-fq.h new file +479 -0

479 lines not on this page · raw message

block/elevator.c +35 -11

116 lines not on this page · raw message

include/linux/blkdev.h +5 -0
index 465d6ba..cf02216 100644
@@ -234,6 +234,11 @@ struct request {
 
 	/* for bidi */
 	struct request *next_rq;
+
+#ifdef CONFIG_ELV_FAIR_QUEUING
+	/* io queue request belongs to */
+	struct io_queue *ioq;
+#endif
 };
 
 static inline unsigned short req_get_ioprio(struct request *req)
include/linux/elevator.h +48 -0

83 lines not on this page · raw message

Signature · 1 line
1.6.0.1

A message that is not in the archive (sent off-list or not yet received): 1236823015-4183-3-git-send-email-vgoyal-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org

#123 lore raw

Re: [PATCH 02/10] Common flat fair queuing code in elevaotor layer

Vivek Goyal wrote:
...

Quoted 5 lines
block/elevator-fq.c new file +5 -0
+
+int elv_init_ioq(struct elevator_queue *eq, struct io_queue *ioq,
+			void *sched_queue, int ioprio_class, int ioprio,
+			int is_sync)
+{

block/elevator-fq.c line 965

Quoted 10 lines
block/elevator-fq.c new file +10 -0
+	struct elv_fq_data *efqd = &eq->efqd;
+	struct io_group *iog = io_lookup_io_group_current(efqd->queue);
+
+	RB_CLEAR_NODE(&ioq->entity.rb_node);
+	atomic_set(&ioq->ref, 0);
+	ioq->efqd = efqd;
+	ioq->entity.budget = efqd->elv_slice[is_sync];
+	elv_ioq_set_ioprio_class(ioq, ioprio_class);
+	elv_ioq_set_ioprio(ioq, ioprio);
+	ioq->pid = current->pid;

Hi Vivek,

If we are using a scheduler other than cfq, IOW single ioq is used. that storing a pid to the ioq makes no sense, it just stores the first serviced task.

Quoted 6 lines
block/elevator-fq.c new file +6 -0
+	ioq->sched_queue = sched_queue;
+	elv_mark_ioq_idle_window(ioq);
+	bfq_init_entity(&ioq->entity, iog);
+	return 0;
+}
+EXPORT_SYMBOL(elv_init_ioq);

...

block/elevator-fq.h line 239

Quoted 7 lines
block/elevator-fq.h new file +7 -0
+
+extern int elv_slice_idle;
+extern int elv_slice_async;
+
+/* Logging facilities. */
+#define elv_log_ioq(efqd, ioq, fmt, args...) \
+	blk_add_trace_msg((efqd)->queue, "%d" fmt, (ioq)->pid, ##args)

Maybe we need to use current->pid instead, for ioq->pid is not valid sometimes.

Quoted 6 lines
block/elevator-fq.h new file +6 -0
+
+#define elv_log(efqd, fmt, args...) \
+	blk_add_trace_msg((efqd)->queue, "" fmt, ##args)
+
+#define ioq_sample_valid(samples)   ((samples) > 80)
+
Signature · 2 lines
Regards
Gui Jianfeng
#124 lore raw

PATCH 1/1 IO Controller: Don't store the pid in single queue circumstances

In reply to a message not in the archive

Vivek Goyal wrote:
...

Quoted 4 lines
(quoted diff) +4 -0
+int elv_init_ioq(struct elevator_queue *eq, struct io_queue *ioq,
+			void *sched_queue, int ioprio_class, int ioprio,
+			int is_sync)
+{

Review

Quoted 10 lines
(quoted diff) +10 -0
+	struct elv_fq_data *efqd = &eq->efqd;
+	struct io_group *iog = io_lookup_io_group_current(efqd->queue);
+
+	RB_CLEAR_NODE(&ioq->entity.rb_node);
+	atomic_set(&ioq->ref, 0);
+	ioq->efqd = efqd;
+	ioq->entity.budget = efqd->elv_slice[is_sync];
+	elv_ioq_set_ioprio_class(ioq, ioprio_class);
+	elv_ioq_set_ioprio(ioq, ioprio);
+	ioq->pid = current->pid;

Hi Vivek,

Storing a pid in single queue circumstances doesn't make sence. So just store the pid when cfq is used.

---

 block/elevator-fq.c |    5 ++++-
 1 files changed, 4 insertions(+), 1 deletions(-)
block/elevator-fq.c +4 -1
index df53418..c72f7e6 100644
@@ -1988,7 +1988,10 @@ int elv_init_ioq(struct elevator_queue *eq, struct io_queue *ioq,
 	ioq->entity.budget = efqd->elv_slice[is_sync];
 	elv_ioq_set_ioprio_class(ioq, ioprio_class);
 	elv_ioq_set_ioprio(ioq, ioprio);
-	ioq->pid = current->pid;
+	if (elv_iosched_single_ioq(eq))
+		ioq->pid = 0;
+	else
+		ioq->pid = current->pid;
 	ioq->sched_queue = sched_queue;
 
 	/* If generic idle logic is enabled, mark it */
Signature · 1 line
1.5.4.rc3
#125 lore raw

Re: [PATCH 02/10] Common flat fair queuing code in elevaotor layer

In reply to a message not in the archive
Quoted 18 lines · Vivek Goyal
block/elevator-fq.c new file +17 -0
+/*
+ * elv_exit_fq_data is called before we call elevator_exit_fn. Before
+ * we ask elevator to cleanup its queues, we do the cleanup here so
+ * that all the group and idle tree references to ioq are dropped. Later
+ * during elevator cleanup, ioc reference will be dropped which will lead
+ * to removal of ioscheduler queue as well as associated ioq object.
+ */
+void elv_exit_fq_data(struct elevator_queue *e)
+{
+       struct elv_fq_data *efqd = &e->efqd;
+       struct request_queue *q = efqd->queue;
+
+       if (!elv_iosched_fair_queuing_enabled(e))
+               return;
+
+       elv_shutdown_timer_wq(e);
+

block/elevator-fq.c line 1842

Quoted 10 lines
block/elevator-fq.c new file +10 -0
+       spin_lock_irq(q->queue_lock);
+       /* This should drop all the idle tree references of ioq */
+       elv_free_idle_ioq_list(e);
+       spin_unlock_irq(q->queue_lock);
+
+       elv_shutdown_timer_wq(e);
+
+       BUG_ON(timer_pending(&efqd->idle_slice_timer));
+       io_free_root_group(e);
+}

Hi Vivek,
When cleaning up the elv_fq_data and ioqs for the iogs
associated with a device on elv_exit(), I don't see any iogs except the root group being freed. In io_disconnect_groups() you remove the ioqs from each of the iog and move them to the root iog and then delete the root iog. Am I missing something here or are there leftover iogs at elv_exit?

-Divyesh

#126 lore raw

Re: [PATCH 02/10] Common flat fair queuing code in elevaotor layer

In reply to Vivek Goyal

Vivek Goyal wrote:
...

Quoted 5 lines
block/elevator-fq.c new file +5 -0
+
+int elv_init_ioq(struct elevator_queue *eq, struct io_queue *ioq,
+			void *sched_queue, int ioprio_class, int ioprio,
+			int is_sync)
+{

block/elevator-fq.c line 965

Quoted 10 lines
block/elevator-fq.c new file +10 -0
+	struct elv_fq_data *efqd = &eq->efqd;
+	struct io_group *iog = io_lookup_io_group_current(efqd->queue);
+
+	RB_CLEAR_NODE(&ioq->entity.rb_node);
+	atomic_set(&ioq->ref, 0);
+	ioq->efqd = efqd;
+	ioq->entity.budget = efqd->elv_slice[is_sync];
+	elv_ioq_set_ioprio_class(ioq, ioprio_class);
+	elv_ioq_set_ioprio(ioq, ioprio);
+	ioq->pid = current->pid;

Hi Vivek,

If we are using a scheduler other than cfq, IOW single ioq is used. that storing a pid to the ioq makes no sense, it just stores the first serviced task.

Quoted 6 lines
block/elevator-fq.c new file +6 -0
+	ioq->sched_queue = sched_queue;
+	elv_mark_ioq_idle_window(ioq);
+	bfq_init_entity(&ioq->entity, iog);
+	return 0;
+}
+EXPORT_SYMBOL(elv_init_ioq);

...

block/elevator-fq.h line 239

Quoted 7 lines
block/elevator-fq.h new file +7 -0
+
+extern int elv_slice_idle;
+extern int elv_slice_async;
+
+/* Logging facilities. */
+#define elv_log_ioq(efqd, ioq, fmt, args...) \
+	blk_add_trace_msg((efqd)->queue, "%d" fmt, (ioq)->pid, ##args)

Maybe we need to use current->pid instead, for ioq->pid is not valid sometimes.

Quoted 6 lines
block/elevator-fq.h new file +6 -0
+
+#define elv_log(efqd, fmt, args...) \
+	blk_add_trace_msg((efqd)->queue, "" fmt, ##args)
+
+#define ioq_sample_valid(samples)   ((samples) > 80)
+
Signature · 2 lines
Regards
Gui Jianfeng
#127 lore raw

PATCH 1/1 IO Controller: Don't store the pid in single queue circumstances

In reply to Vivek Goyal

Vivek Goyal wrote:
...

Quoted 4 lines
(quoted diff) +4 -0
+int elv_init_ioq(struct elevator_queue *eq, struct io_queue *ioq,
+			void *sched_queue, int ioprio_class, int ioprio,
+			int is_sync)
+{

Review

Quoted 10 lines
(quoted diff) +10 -0
+	struct elv_fq_data *efqd = &eq->efqd;
+	struct io_group *iog = io_lookup_io_group_current(efqd->queue);
+
+	RB_CLEAR_NODE(&ioq->entity.rb_node);
+	atomic_set(&ioq->ref, 0);
+	ioq->efqd = efqd;
+	ioq->entity.budget = efqd->elv_slice[is_sync];
+	elv_ioq_set_ioprio_class(ioq, ioprio_class);
+	elv_ioq_set_ioprio(ioq, ioprio);
+	ioq->pid = current->pid;

Hi Vivek,

Storing a pid in single queue circumstances doesn't make sence. So just store the pid when cfq is used.

---

 block/elevator-fq.c |    5 ++++-
 1 files changed, 4 insertions(+), 1 deletions(-)
block/elevator-fq.c +4 -1
index df53418..c72f7e6 100644
@@ -1988,7 +1988,10 @@ int elv_init_ioq(struct elevator_queue *eq, struct io_queue *ioq,
 	ioq->entity.budget = efqd->elv_slice[is_sync];
 	elv_ioq_set_ioprio_class(ioq, ioprio_class);
 	elv_ioq_set_ioprio(ioq, ioprio);
-	ioq->pid = current->pid;
+	if (elv_iosched_single_ioq(eq))
+		ioq->pid = 0;
+	else
+		ioq->pid = current->pid;
 	ioq->sched_queue = sched_queue;
 
 	/* If generic idle logic is enabled, mark it */
Signature · 1 line
1.5.4.rc3

A message that is not in the archive (sent off-list or not yet received): [email protected]

#129 lore raw

Re: [PATCH] IO Controller: Don't store the pid in single queue circumstances

Quoted 32 lines · Gui Jianfeng

Vivek Goyal wrote:
...

Quoted 4 lines
(quoted diff) +4 -0
+int elv_init_ioq(struct elevator_queue *eq, struct io_queue *ioq,
+			void *sched_queue, int ioprio_class, int ioprio,
+			int is_sync)
+{

Review

Quoted 10 lines
(quoted diff) +10 -0
+	struct elv_fq_data *efqd = &eq->efqd;
+	struct io_group *iog = io_lookup_io_group_current(efqd->queue);
+
+	RB_CLEAR_NODE(&ioq->entity.rb_node);
+	atomic_set(&ioq->ref, 0);
+	ioq->efqd = efqd;
+	ioq->entity.budget = efqd->elv_slice[is_sync];
+	elv_ioq_set_ioprio_class(ioq, ioprio_class);
+	elv_ioq_set_ioprio(ioq, ioprio);
+	ioq->pid = current->pid;

Hi Vivek,

Storing a pid in single queue circumstances doesn't make sence. So just store the pid when cfq is used.

---

 block/elevator-fq.c |    5 ++++-
 1 files changed, 4 insertions(+), 1 deletions(-)
block/elevator-fq.c +0 -0
index df53418..c72f7e6 100644

block/elevator-fq.c int elv_init_ioq(struct elevator_queue *eq, struct io_queue *ioq, · line 1995

Quoted 9 lines
block/elevator-fq.c +4 -1
@@ -1988,7 +1988,10 @@ int elv_init_ioq(struct elevator_queue *eq, struct io_queue *ioq,
 	ioq->entity.budget = efqd->elv_slice[is_sync];
 	elv_ioq_set_ioprio_class(ioq, ioprio_class);
 	elv_ioq_set_ioprio(ioq, ioprio);
-	ioq->pid = current->pid;
+	if (elv_iosched_single_ioq(eq))
+		ioq->pid = 0;
+	else
+		ioq->pid = current->pid;
 	ioq->sched_queue = sched_queue;

Thanks Gui. Yes, if there is a single ioq, this pid will reflect the pid of the process who caused the creation of the io queue and later requests from all the other processess will go into same queue.

In fact cfq also has the same issue for async queues where async queue will store the pid when it is created later all other processes of same prio level will use it.

So if you think displaying "0" is better than displaying the the pid of the process who created the queue, then I will include this patch. Right now I don't have very strong opinion about it.

Thanks
Vivek

#130 lore raw

Re: [PATCH] IO Controller: Don't store the pid in single queue circumstances

In reply to Gui Jianfeng
Quoted 32 lines · Gui Jianfeng

Vivek Goyal wrote:
...

Quoted 4 lines
(quoted diff) +4 -0
+int elv_init_ioq(struct elevator_queue *eq, struct io_queue *ioq,
+			void *sched_queue, int ioprio_class, int ioprio,
+			int is_sync)
+{

Review

Quoted 10 lines
(quoted diff) +10 -0
+	struct elv_fq_data *efqd = &eq->efqd;
+	struct io_group *iog = io_lookup_io_group_current(efqd->queue);
+
+	RB_CLEAR_NODE(&ioq->entity.rb_node);
+	atomic_set(&ioq->ref, 0);
+	ioq->efqd = efqd;
+	ioq->entity.budget = efqd->elv_slice[is_sync];
+	elv_ioq_set_ioprio_class(ioq, ioprio_class);
+	elv_ioq_set_ioprio(ioq, ioprio);
+	ioq->pid = current->pid;

Hi Vivek,

Storing a pid in single queue circumstances doesn't make sence. So just store the pid when cfq is used.

---

 block/elevator-fq.c |    5 ++++-
 1 files changed, 4 insertions(+), 1 deletions(-)
block/elevator-fq.c +0 -0
index df53418..c72f7e6 100644

block/elevator-fq.c int elv_init_ioq(struct elevator_queue *eq, struct io_queue *ioq, · line 1995

Quoted 9 lines
block/elevator-fq.c +4 -1
@@ -1988,7 +1988,10 @@ int elv_init_ioq(struct elevator_queue *eq, struct io_queue *ioq,
 	ioq->entity.budget = efqd->elv_slice[is_sync];
 	elv_ioq_set_ioprio_class(ioq, ioprio_class);
 	elv_ioq_set_ioprio(ioq, ioprio);
-	ioq->pid = current->pid;
+	if (elv_iosched_single_ioq(eq))
+		ioq->pid = 0;
+	else
+		ioq->pid = current->pid;
 	ioq->sched_queue = sched_queue;

Thanks Gui. Yes, if there is a single ioq, this pid will reflect the pid of the process who caused the creation of the io queue and later requests from all the other processess will go into same queue.

In fact cfq also has the same issue for async queues where async queue will store the pid when it is created later all other processes of same prio level will use it.

So if you think displaying "0" is better than displaying the the pid of the process who created the queue, then I will include this patch. Right now I don't have very strong opinion about it.

Thanks
Vivek

#131 lore raw

Re: [PATCH 02/10] Common flat fair queuing code in elevaotor layer

In reply to Vivek Goyal
Quoted 18 lines · Vivek Goyal
block/elevator-fq.c new file +17 -0
+/*
+ * elv_exit_fq_data is called before we call elevator_exit_fn. Before
+ * we ask elevator to cleanup its queues, we do the cleanup here so
+ * that all the group and idle tree references to ioq are dropped. Later
+ * during elevator cleanup, ioc reference will be dropped which will lead
+ * to removal of ioscheduler queue as well as associated ioq object.
+ */
+void elv_exit_fq_data(struct elevator_queue *e)
+{
+       struct elv_fq_data *efqd = &e->efqd;
+       struct request_queue *q = efqd->queue;
+
+       if (!elv_iosched_fair_queuing_enabled(e))
+               return;
+
+       elv_shutdown_timer_wq(e);
+

block/elevator-fq.c line 1842

Quoted 10 lines
block/elevator-fq.c new file +10 -0
+       spin_lock_irq(q->queue_lock);
+       /* This should drop all the idle tree references of ioq */
+       elv_free_idle_ioq_list(e);
+       spin_unlock_irq(q->queue_lock);
+
+       elv_shutdown_timer_wq(e);
+
+       BUG_ON(timer_pending(&efqd->idle_slice_timer));
+       io_free_root_group(e);
+}

Hi Vivek,
When cleaning up the elv_fq_data and ioqs for the iogs
associated with a device on elv_exit(), I don't see any iogs except the root group being freed. In io_disconnect_groups() you remove the ioqs from each of the iog and move them to the root iog and then delete the root iog. Am I missing something here or are there leftover iogs at elv_exit?

-Divyesh

A message that is not in the archive (sent off-list or not yet received): af41c7c40904012106h41d3cb50i2eeab2a02277a4c9-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org

#133 lore raw

Re: [PATCH 02/10] Common flat fair queuing code in elevaotor layer

Quoted 30 lines · Divyesh Shah
Quoted 18 lines · Vivek Goyal
block/elevator-fq.c new file +17 -0
+/*
+ * elv_exit_fq_data is called before we call elevator_exit_fn. Before
+ * we ask elevator to cleanup its queues, we do the cleanup here so
+ * that all the group and idle tree references to ioq are dropped. Later
+ * during elevator cleanup, ioc reference will be dropped which will lead
+ * to removal of ioscheduler queue as well as associated ioq object.
+ */
+void elv_exit_fq_data(struct elevator_queue *e)
+{
+       struct elv_fq_data *efqd = &e->efqd;
+       struct request_queue *q = efqd->queue;
+
+       if (!elv_iosched_fair_queuing_enabled(e))
+               return;
+
+       elv_shutdown_timer_wq(e);
+

block/elevator-fq.c line 1842

Quoted 10 lines
block/elevator-fq.c new file +10 -0
+       spin_lock_irq(q->queue_lock);
+       /* This should drop all the idle tree references of ioq */
+       elv_free_idle_ioq_list(e);
+       spin_unlock_irq(q->queue_lock);
+
+       elv_shutdown_timer_wq(e);
+
+       BUG_ON(timer_pending(&efqd->idle_slice_timer));
+       io_free_root_group(e);
+}
Quoted 7 lines

Hi Vivek,
When cleaning up the elv_fq_data and ioqs for the iogs
associated with a device on elv_exit(), I don't see any iogs except the root group being freed. In io_disconnect_groups() you remove the ioqs from each of the iog and move them to the root iog and then delete the root iog. Am I missing something here or are there leftover iogs at elv_exit?

Hi Divyesh,

io_groups are linked in two lists. One list is maintained by io_cgroup to keep track of how many io_groups are there associated with this cgroup and other list is maintained in elv_fq_data to keep track of how many io_groups are actually doing IO to this device.

Upon elevator exit, we remove the io_groups (io_disconnect_groups())from the list maintained by elv_fq_data but we don't free them up. Freeing up is finally done when cgroup is being deleted (iocg_destroy()).

Thanks
Vivek

#134 lore raw

Re: [PATCH 02/10] Common flat fair queuing code in elevaotor layer

In reply to Divyesh Shah
Quoted 30 lines · Divyesh Shah
Quoted 18 lines · Vivek Goyal
block/elevator-fq.c new file +17 -0
+/*
+ * elv_exit_fq_data is called before we call elevator_exit_fn. Before
+ * we ask elevator to cleanup its queues, we do the cleanup here so
+ * that all the group and idle tree references to ioq are dropped. Later
+ * during elevator cleanup, ioc reference will be dropped which will lead
+ * to removal of ioscheduler queue as well as associated ioq object.
+ */
+void elv_exit_fq_data(struct elevator_queue *e)
+{
+       struct elv_fq_data *efqd = &e->efqd;
+       struct request_queue *q = efqd->queue;
+
+       if (!elv_iosched_fair_queuing_enabled(e))
+               return;
+
+       elv_shutdown_timer_wq(e);
+

block/elevator-fq.c line 1842

Quoted 10 lines
block/elevator-fq.c new file +10 -0
+       spin_lock_irq(q->queue_lock);
+       /* This should drop all the idle tree references of ioq */
+       elv_free_idle_ioq_list(e);
+       spin_unlock_irq(q->queue_lock);
+
+       elv_shutdown_timer_wq(e);
+
+       BUG_ON(timer_pending(&efqd->idle_slice_timer));
+       io_free_root_group(e);
+}
Quoted 7 lines

Hi Vivek,
When cleaning up the elv_fq_data and ioqs for the iogs
associated with a device on elv_exit(), I don't see any iogs except the root group being freed. In io_disconnect_groups() you remove the ioqs from each of the iog and move them to the root iog and then delete the root iog. Am I missing something here or are there leftover iogs at elv_exit?

Hi Divyesh,

io_groups are linked in two lists. One list is maintained by io_cgroup to keep track of how many io_groups are there associated with this cgroup and other list is maintained in elv_fq_data to keep track of how many io_groups are actually doing IO to this device.

Upon elevator exit, we remove the io_groups (io_disconnect_groups())from the list maintained by elv_fq_data but we don't free them up. Freeing up is finally done when cgroup is being deleted (iocg_destroy()).

Thanks
Vivek

#135 lore raw

PATCH 3/10 Modify cfq to make use of flat elevator fair queuing

In reply to Vivek Goyal

This patch changes cfq to use fair queuing code from elevator layer.

o must_dispatch logic sounds like a dead code. Nobody seems to be making use of that flag. Retaining it for the time being.

---
 block/Kconfig.iosched |    3 +-
 block/cfq-iosched.c   | 1082 +++++++++++--------------------------------------
 2 files changed, 232 insertions(+), 853 deletions(-)
block/Kconfig.iosched +2 -1
index 3398134..dd5224d 100644
@@ -3,7 +3,7 @@ if BLOCK
 menu "IO Schedulers"
 
 config ELV_FAIR_QUEUING
-	bool "Elevator Fair Queuing Support"
+	bool
 	default n
 	---help---
 	  Traditionally only cfq had notion of multiple queues and it did
@@ -46,6 +46,7 @@ config IOSCHED_DEADLINE
 
 config IOSCHED_CFQ
 	tristate "CFQ I/O scheduler"
+	select ELV_FAIR_QUEUING
 	default y
 	---help---
 	  The CFQ I/O scheduler tries to distribute bandwidth equally
block/cfq-iosched.c +230 -852

1,757 lines not on this page · raw message

Signature · 1 line
1.6.0.1
#136 lore raw

PATCH 4/10 Common hierarchical fair queuing code in elevaotor layer

In reply to Vivek Goyal

This patch enables hierarchical fair queuing in common layer. It is controlled by config option CONFIG_GROUP_IOSCHED.

---
 block/blk-ioc.c               |    3 +
 block/elevator-fq.c           |  991 ++++++++++++++++++++++++++++++++++++++---
 block/elevator-fq.h           |  113 +++++
 block/elevator.c              |    2 +
 include/linux/blkdev.h        |    7 +-
 include/linux/cgroup_subsys.h |    7 +
 include/linux/iocontext.h     |    5 +
 init/Kconfig                  |    8 +
 8 files changed, 1064 insertions(+), 72 deletions(-)
block/blk-ioc.c +3 -0
index 012f065..8f0f6cf 100644
@@ -95,6 +95,9 @@ struct io_context *alloc_io_context(gfp_t gfp_flags, int node)
 		spin_lock_init(&ret->lock);
 		ret->ioprio_changed = 0;
 		ret->ioprio = 0;
+#ifdef CONFIG_GROUP_IOSCHED
+		ret->cgroup_changed = 0;
+#endif
 		ret->last_waited = jiffies; /* doesn't matter... */
 		ret->nr_batch_requests = 0; /* because this is 0 */
 		ret->aic = NULL;
block/elevator-fq.c +920 -71

1,109 lines not on this page · raw message

block/elevator-fq.h +113 -0

153 lines not on this page · raw message

block/elevator.c +2 -0
index 7a3a7e9..27889bc 100644
@@ -888,6 +888,8 @@ int elv_set_request(struct request_queue *q, struct request *rq, gfp_t gfp_mask)
 {
 	struct elevator_queue *e = q->elevator;
 
+	elv_fq_set_request_io_group(q, rq);
+
 	if (e->ops->elevator_set_req_fn)
 		return e->ops->elevator_set_req_fn(q, rq, gfp_mask);
 
include/linux/blkdev.h +6 -1
index cf02216..0baeb8e 100644
@@ -238,7 +238,12 @@ struct request {
 #ifdef CONFIG_ELV_FAIR_QUEUING
 	/* io queue request belongs to */
 	struct io_queue *ioq;
-#endif
+
+#ifdef CONFIG_GROUP_IOSCHED
+	/* io group request belongs to */
+	struct io_group *iog;
+#endif /* GROUP_IOSCHED */
+#endif /* ELV_FAIR_QUEUING */
 };
 
 static inline unsigned short req_get_ioprio(struct request *req)
include/linux/cgroup_subsys.h +7 -0
index 9c8d31b..68ea6bd 100644
@@ -60,3 +60,10 @@ SUBSYS(net_cls)
 #endif
 
 /* */
+
+#ifdef CONFIG_GROUP_IOSCHED
+SUBSYS(io)
+#endif
+
+/* */
+
include/linux/iocontext.h +5 -0
index 08b987b..51664bb 100644
@@ -73,6 +73,11 @@ struct io_context {
 	unsigned short ioprio;
 	unsigned short ioprio_changed;
 
+#ifdef CONFIG_GROUP_IOSCHED
+	/* If task changes the cgroup, elevator processes it asynchronously */
+	unsigned short cgroup_changed;
+#endif
+
 	/*
 	 * For request batching
 	 */
init/Kconfig +8 -0
index 6a5c5fe..66c2310 100644
@@ -538,6 +538,14 @@ config CGROUP_MEM_RES_CTLR_SWAP
 	  there will be no overhead from this. Even when you set this config=y,
 	  if boot option "noswapaccount" is set, swap will not be accounted.
 
+config GROUP_IOSCHED
+	bool "Group IO Scheduler"
+	depends on CGROUPS && ELV_FAIR_QUEUING
+	default n
+	---help---
+	  This feature lets IO scheduler recognize task groups and control
+	  disk bandwidth allocation to such task groups.
+
 endif # CGROUPS
 
 config MM_OWNER
Signature · 1 line
1.6.0.1
#137 lore raw

PATCH 5/10 cfq changes to use hierarchical fair queuing code in elevaotor layer

In reply to Vivek Goyal

Make cfq hierarhical.

---
 block/Kconfig.iosched |    8 ++++++++
 block/cfq-iosched.c   |   48 ++++++++++++++++++++++++++++++++++++++++++++++++
 init/Kconfig          |    2 +-
 3 files changed, 57 insertions(+), 1 deletions(-)
block/Kconfig.iosched +8 -0
index dd5224d..a91a807 100644
@@ -54,6 +54,14 @@ config IOSCHED_CFQ
 	  working environment, suitable for desktop systems.
 	  This is the default I/O scheduler.
 
+config IOSCHED_CFQ_HIER
+	bool "CFQ Hierarchical Scheduling support"
+	depends on IOSCHED_CFQ && CGROUPS
+	select GROUP_IOSCHED
+	default n
+	---help---
+	  Enable hierarhical scheduling in cfq.
+
 choice
 	prompt "Default I/O scheduler"
 	default DEFAULT_CFQ
block/cfq-iosched.c +48 -0

60 lines not on this page · raw message

init/Kconfig +1 -1
index 66c2310..d7bc054 100644
@@ -539,7 +539,7 @@ config CGROUP_MEM_RES_CTLR_SWAP
 	  if boot option "noswapaccount" is set, swap will not be accounted.
 
 config GROUP_IOSCHED
-	bool "Group IO Scheduler"
+	bool
 	depends on CGROUPS && ELV_FAIR_QUEUING
 	default n
 	---help---
Signature · 1 line
1.6.0.1

A message that is not in the archive (sent off-list or not yet received): 1236823015-4183-6-git-send-email-vgoyal-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org

#139 lore raw

PATCH 1/1 IO-Controller: Fix kernel panic after moving a task

Quoted 9 lines · Vivek Goyal
(quoted diff) +9 -0
+#ifdef CONFIG_IOSCHED_CFQ_HIER
+static void changed_cgroup(struct io_context *ioc, struct cfq_io_context *cic)
+{
+	struct cfq_queue *async_cfqq = cic_to_cfqq(cic, 0);
+	struct cfq_queue *sync_cfqq = cic_to_cfqq(cic, 1);
+	struct cfq_data *cfqd = cic->key;
+	struct io_group *iog, *__iog;
+	unsigned long flags;
+	struct request_queue *q;

Review

Quoted 10 lines
(quoted diff) +10 -0
+
+	if (unlikely(!cfqd))
+		return;
+
+	q = cfqd->q;
+
+	spin_lock_irqsave(q->queue_lock, flags);
+
+	iog = io_lookup_io_group_current(q);
+

Hi Vivek,

I triggered another kernel panic when testing. When moving a task to another cgroup, the corresponding iog may not be setup properly all the time. "iog" might be NULL here. io_ioq_move() receives a NULL iog, kernel crash.

Consider the following piece of code:

941 int elv_set_request(struct request_queue *q, struct request *rq, gfp_t gfp_mask) 942 {
943 struct elevator_queue *e = q->elevator;
944
945 elv_fq_set_request_io_group(q, rq);

-->task moving to a new group is happenning here.

946
947 /*

 948          * Optimization for noop, deadline and AS which maintain only single
 949          * ioq per io group
 950          */

951 if (elv_iosched_single_ioq(e))
952 return elv_fq_set_request_ioq(q, rq, gfp_mask);
953
954 if (e->ops->elevator_set_req_fn)
955 return e->ops->elevator_set_req_fn(q, rq, gfp_mask);

cfq_set_request() will finally call io_ioq_move(), but the iog is NULL, beacause the iogs in the hierarchy are not built yet. So kernel crashes.

956
957 rq->elevator_private = NULL;
958 return 0;
959 }

BUG: unable to handle kernel NULL pointer dereference at 000000bc
IP: [<c04ebf8f>] io_ioq_move+0xf2/0x109
*pde = 6cc00067
Oops: 0000 [#1] SMP
last sysfs file: /sys/block/hdb/queue/slice_idle
Modules linked in: ipv6 cpufreq_ondemand acpi_cpufreq dm_mirror dm_multipath sbs sbshc battery ac lp snd_hda_codec_realtek snd_hda_intel snd_hda_codec snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss snd_pcm serio_raw snd_timer rtc_cmos parport_pc snd r8169 button rtc_core parport soundcore mii i2c_i801 rtc_lib snd_page_alloc pcspkr i2c_core dm_region_hash dm_log dm_mod ext3 jbd mbcache uhci_hcd ohci_hcd ehci_hcd

Pid: 5431, comm: dd Not tainted (2.6.29-rc7-vivek #19) Veriton M460
EIP: 0060:[<c04ebf8f>] EFLAGS: 00010046 CPU: 0
EIP is at io_ioq_move+0xf2/0x109
EAX: f6203a88 EBX: f6792c94 ECX: f6203a84 EDX: 00000006
ESI: 00000000 EDI: 00000000 EBP: f6203a60 ESP: f6304c28
DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
Process dd (pid: 5431, ti=f6304000 task=f669dae0 task.ti=f6304000)

Stack:
 f62478c0 0100dd40 f6247908 f62d995c 00000000 00000000 f675b54c c04e9182
 f638e9b0 00000282 f62d99a4 f6325a2c c04e9113 f5a707c0 c04e7ae0 f675b000
 f62d95fc f6325a2c c04e8501 00000010 f631e4e8 f675b000 00080000 ffffff10
Call Trace:
 [<c04e9182>] changed_cgroup+0x6f/0x8d
 [<c04e9113>] changed_cgroup+0x0/0x8d
 [<c04e7ae0>] __call_for_each_cic+0x1b/0x25
 [<c04e8501>] cfq_set_request+0x158/0x2c7
 [<c06316e6>] _spin_unlock_irqrestore+0x5/0x6
 [<c04eb106>] elv_fq_set_request_io_group+0x2b/0x3e
 [<c04e83a9>] cfq_set_request+0x0/0x2c7
 [<c04dddcb>] elv_set_request+0x3e/0x4e
 [<c04df3da>] get_request+0x1ed/0x29b
 [<c04df9bb>] get_request_wait+0xdf/0xf2
 [<c04dfd89>] __make_request+0x2c6/0x372
 [<c049bd76>] do_mpage_readpage+0x4fe/0x5e3
 [<c04deba5>] generic_make_request+0x2d0/0x355
 [<c04dff47>] submit_bio+0x92/0x97
 [<c045bfcb>] add_to_page_cache_locked+0x8a/0xb7
 [<c049bfa4>] mpage_end_io_read+0x0/0x50
 [<c049b1b6>] mpage_bio_submit+0x19/0x1d
 [<c049bf9a>] mpage_readpages+0x9b/0xa5
 [<f7dd18c7>] ext3_readpages+0x0/0x15 [ext3]
 [<c0462192>] __do_page_cache_readahead+0xea/0x154
 [<f7dd2286>] ext3_get_block+0x0/0xbe [ext3]
 [<c045d34d>] generic_file_aio_read+0x276/0x569
 [<c047cdd9>] do_sync_read+0xbf/0xfe
 [<c043a3f2>] getnstimeofday+0x51/0xdb
 [<c0434d3c>] autoremove_wake_function+0x0/0x2d
 [<c041bdc3>] sched_slice+0x61/0x6a
 [<c0423114>] task_tick_fair+0x3d/0x60
 [<c04c1d79>] security_file_permission+0xc/0xd
 [<c047cd1a>] do_sync_read+0x0/0xfe
 [<c047d35a>] vfs_read+0x6c/0x8b
 [<c047d67e>] sys_read+0x3c/0x63
 [<c0402fc1>] sysenter_do_call+0x12/0x21
 [<c0630000>] schedule+0x551/0x830
Code: 08 31 c9 89 da e8 77 fc ff ff 8b 86 bc 00 00 00 85 ff 89 43 38 8d 46 60 89 43 40 74 1d 83 c4 0c 89 d8 5b 5e 5f 5d e9 aa f9 ff ff <8b> 86 bc 00 00 00 89 43 38 8d 46 60 89 43 40 83 c4 0c 5b 5e 5f
EIP: [<c04ebf8f>] io_ioq_move+0xf2/0x109 SS:ESP 0068:f6304c28

Changelog:

Make sure iogs in the hierarchy are built properly after moving a task to a new cgroup.

---
 block/cfq-iosched.c |    4 +++-
 block/elevator-fq.c |    1 +
 block/elevator-fq.h |    1 +
 3 files changed, 5 insertions(+), 1 deletions(-)
block/cfq-iosched.c +3 -1
index 0ecf7c7..6d7bb8a 100644
@@ -12,6 +12,8 @@
 #include <linux/rbtree.h>
 #include <linux/ioprio.h>
 #include <linux/blktrace_api.h>
+#include "elevator-fq.h"
+
 /*
  * tunables
  */
@@ -1086,7 +1088,7 @@ static void changed_cgroup(struct io_context *ioc, struct cfq_io_context *cic)
 
 	spin_lock_irqsave(q->queue_lock, flags);
 
-	iog = io_lookup_io_group_current(q);
+	iog = io_get_io_group(q);
 
 	if (async_cfqq != NULL) {
 		__iog = cfqq_to_io_group(async_cfqq);
block/elevator-fq.c +1 -0
index df53418..f81cf6a 100644
@@ -1191,6 +1191,7 @@ struct io_group *io_get_io_group(struct request_queue *q)
 
 	return iog;
 }
+EXPORT_SYMBOL(io_get_io_group);
 
 void io_free_root_group(struct elevator_queue *e)
 {
block/elevator-fq.h +1 -0
index fc4110d..f17e425 100644
@@ -459,6 +459,7 @@ static inline struct io_group *ioq_to_io_group(struct io_queue *ioq)
 }
 
 #ifdef CONFIG_GROUP_IOSCHED
+extern struct io_group *io_get_io_group(struct request_queue *q);
 extern int io_group_allow_merge(struct request *rq, struct bio *bio);
 extern void io_ioq_move(struct elevator_queue *e, struct io_queue *ioq,
 					struct io_group *iog);
Signature · 1 line
1.5.4.rc3
#140 lore raw

PATCH 1/1 IO-Controller: Fix kernel panic after moving a task

In reply to Vivek Goyal
Quoted 9 lines · Vivek Goyal
(quoted diff) +9 -0
+#ifdef CONFIG_IOSCHED_CFQ_HIER
+static void changed_cgroup(struct io_context *ioc, struct cfq_io_context *cic)
+{
+	struct cfq_queue *async_cfqq = cic_to_cfqq(cic, 0);
+	struct cfq_queue *sync_cfqq = cic_to_cfqq(cic, 1);
+	struct cfq_data *cfqd = cic->key;
+	struct io_group *iog, *__iog;
+	unsigned long flags;
+	struct request_queue *q;

Review

Quoted 10 lines
(quoted diff) +10 -0
+
+	if (unlikely(!cfqd))
+		return;
+
+	q = cfqd->q;
+
+	spin_lock_irqsave(q->queue_lock, flags);
+
+	iog = io_lookup_io_group_current(q);
+

Hi Vivek,

I triggered another kernel panic when testing. When moving a task to another cgroup, the corresponding iog may not be setup properly all the time. "iog" might be NULL here. io_ioq_move() receives a NULL iog, kernel crash.

Consider the following piece of code:

941 int elv_set_request(struct request_queue *q, struct request *rq, gfp_t gfp_mask) 942 {
943 struct elevator_queue *e = q->elevator;
944
945 elv_fq_set_request_io_group(q, rq);

-->task moving to a new group is happenning here.

946
947 /*

 948          * Optimization for noop, deadline and AS which maintain only single
 949          * ioq per io group
 950          */

951 if (elv_iosched_single_ioq(e))
952 return elv_fq_set_request_ioq(q, rq, gfp_mask);
953
954 if (e->ops->elevator_set_req_fn)
955 return e->ops->elevator_set_req_fn(q, rq, gfp_mask);

cfq_set_request() will finally call io_ioq_move(), but the iog is NULL, beacause the iogs in the hierarchy are not built yet. So kernel crashes.

956
957 rq->elevator_private = NULL;
958 return 0;
959 }

BUG: unable to handle kernel NULL pointer dereference at 000000bc
IP: [<c04ebf8f>] io_ioq_move+0xf2/0x109
*pde = 6cc00067
Oops: 0000 [#1] SMP
last sysfs file: /sys/block/hdb/queue/slice_idle
Modules linked in: ipv6 cpufreq_ondemand acpi_cpufreq dm_mirror dm_multipath sbs sbshc battery ac lp snd_hda_codec_realtek snd_hda_intel snd_hda_codec snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss snd_pcm serio_raw snd_timer rtc_cmos parport_pc snd r8169 button rtc_core parport soundcore mii i2c_i801 rtc_lib snd_page_alloc pcspkr i2c_core dm_region_hash dm_log dm_mod ext3 jbd mbcache uhci_hcd ohci_hcd ehci_hcd

Pid: 5431, comm: dd Not tainted (2.6.29-rc7-vivek #19) Veriton M460
EIP: 0060:[<c04ebf8f>] EFLAGS: 00010046 CPU: 0
EIP is at io_ioq_move+0xf2/0x109
EAX: f6203a88 EBX: f6792c94 ECX: f6203a84 EDX: 00000006
ESI: 00000000 EDI: 00000000 EBP: f6203a60 ESP: f6304c28
DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
Process dd (pid: 5431, ti=f6304000 task=f669dae0 task.ti=f6304000)

Stack:
 f62478c0 0100dd40 f6247908 f62d995c 00000000 00000000 f675b54c c04e9182
 f638e9b0 00000282 f62d99a4 f6325a2c c04e9113 f5a707c0 c04e7ae0 f675b000
 f62d95fc f6325a2c c04e8501 00000010 f631e4e8 f675b000 00080000 ffffff10
Call Trace:
 [<c04e9182>] changed_cgroup+0x6f/0x8d
 [<c04e9113>] changed_cgroup+0x0/0x8d
 [<c04e7ae0>] __call_for_each_cic+0x1b/0x25
 [<c04e8501>] cfq_set_request+0x158/0x2c7
 [<c06316e6>] _spin_unlock_irqrestore+0x5/0x6
 [<c04eb106>] elv_fq_set_request_io_group+0x2b/0x3e
 [<c04e83a9>] cfq_set_request+0x0/0x2c7
 [<c04dddcb>] elv_set_request+0x3e/0x4e
 [<c04df3da>] get_request+0x1ed/0x29b
 [<c04df9bb>] get_request_wait+0xdf/0xf2
 [<c04dfd89>] __make_request+0x2c6/0x372
 [<c049bd76>] do_mpage_readpage+0x4fe/0x5e3
 [<c04deba5>] generic_make_request+0x2d0/0x355
 [<c04dff47>] submit_bio+0x92/0x97
 [<c045bfcb>] add_to_page_cache_locked+0x8a/0xb7
 [<c049bfa4>] mpage_end_io_read+0x0/0x50
 [<c049b1b6>] mpage_bio_submit+0x19/0x1d
 [<c049bf9a>] mpage_readpages+0x9b/0xa5
 [<f7dd18c7>] ext3_readpages+0x0/0x15 [ext3]
 [<c0462192>] __do_page_cache_readahead+0xea/0x154
 [<f7dd2286>] ext3_get_block+0x0/0xbe [ext3]
 [<c045d34d>] generic_file_aio_read+0x276/0x569
 [<c047cdd9>] do_sync_read+0xbf/0xfe
 [<c043a3f2>] getnstimeofday+0x51/0xdb
 [<c0434d3c>] autoremove_wake_function+0x0/0x2d
 [<c041bdc3>] sched_slice+0x61/0x6a
 [<c0423114>] task_tick_fair+0x3d/0x60
 [<c04c1d79>] security_file_permission+0xc/0xd
 [<c047cd1a>] do_sync_read+0x0/0xfe
 [<c047d35a>] vfs_read+0x6c/0x8b
 [<c047d67e>] sys_read+0x3c/0x63
 [<c0402fc1>] sysenter_do_call+0x12/0x21
 [<c0630000>] schedule+0x551/0x830
Code: 08 31 c9 89 da e8 77 fc ff ff 8b 86 bc 00 00 00 85 ff 89 43 38 8d 46 60 89 43 40 74 1d 83 c4 0c 89 d8 5b 5e 5f 5d e9 aa f9 ff ff <8b> 86 bc 00 00 00 89 43 38 8d 46 60 89 43 40 83 c4 0c 5b 5e 5f
EIP: [<c04ebf8f>] io_ioq_move+0xf2/0x109 SS:ESP 0068:f6304c28

Changelog:

Make sure iogs in the hierarchy are built properly after moving a task to a new cgroup.

---
 block/cfq-iosched.c |    4 +++-
 block/elevator-fq.c |    1 +
 block/elevator-fq.h |    1 +
 3 files changed, 5 insertions(+), 1 deletions(-)
block/cfq-iosched.c +3 -1
index 0ecf7c7..6d7bb8a 100644
@@ -12,6 +12,8 @@
 #include <linux/rbtree.h>
 #include <linux/ioprio.h>
 #include <linux/blktrace_api.h>
+#include "elevator-fq.h"
+
 /*
  * tunables
  */
@@ -1086,7 +1088,7 @@ static void changed_cgroup(struct io_context *ioc, struct cfq_io_context *cic)
 
 	spin_lock_irqsave(q->queue_lock, flags);
 
-	iog = io_lookup_io_group_current(q);
+	iog = io_get_io_group(q);
 
 	if (async_cfqq != NULL) {
 		__iog = cfqq_to_io_group(async_cfqq);
block/elevator-fq.c +1 -0
index df53418..f81cf6a 100644
@@ -1191,6 +1191,7 @@ struct io_group *io_get_io_group(struct request_queue *q)
 
 	return iog;
 }
+EXPORT_SYMBOL(io_get_io_group);
 
 void io_free_root_group(struct elevator_queue *e)
 {
block/elevator-fq.h +1 -0
index fc4110d..f17e425 100644
@@ -459,6 +459,7 @@ static inline struct io_group *ioq_to_io_group(struct io_queue *ioq)
 }
 
 #ifdef CONFIG_GROUP_IOSCHED
+extern struct io_group *io_get_io_group(struct request_queue *q);
 extern int io_group_allow_merge(struct request *rq, struct bio *bio);
 extern void io_ioq_move(struct elevator_queue *e, struct io_queue *ioq,
 					struct io_group *iog);
Signature · 1 line
1.5.4.rc3
#141 lore raw

Re: [PATCH] IO-Controller: Fix kernel panic after moving a task

Quoted 48 lines · Gui Jianfeng
Quoted 9 lines · Vivek Goyal
(quoted diff) +9 -0
+#ifdef CONFIG_IOSCHED_CFQ_HIER
+static void changed_cgroup(struct io_context *ioc, struct cfq_io_context *cic)
+{
+	struct cfq_queue *async_cfqq = cic_to_cfqq(cic, 0);
+	struct cfq_queue *sync_cfqq = cic_to_cfqq(cic, 1);
+	struct cfq_data *cfqd = cic->key;
+	struct io_group *iog, *__iog;
+	unsigned long flags;
+	struct request_queue *q;

Review

Quoted 10 lines
(quoted diff) +10 -0
+
+	if (unlikely(!cfqd))
+		return;
+
+	q = cfqd->q;
+
+	spin_lock_irqsave(q->queue_lock, flags);
+
+	iog = io_lookup_io_group_current(q);
+

Hi Vivek,

I triggered another kernel panic when testing. When moving a task to another cgroup, the corresponding iog may not be setup properly all the time. "iog" might be NULL here. io_ioq_move() receives a NULL iog, kernel crash.

Consider the following piece of code:

941 int elv_set_request(struct request_queue *q, struct request *rq, gfp_t gfp_mask) 942 {
943 struct elevator_queue *e = q->elevator;
944
945 elv_fq_set_request_io_group(q, rq);

-->task moving to a new group is happenning here.

946
947 /*

 948          * Optimization for noop, deadline and AS which maintain only single
 949          * ioq per io group
 950          */

951 if (elv_iosched_single_ioq(e))
952 return elv_fq_set_request_ioq(q, rq, gfp_mask);
953
954 if (e->ops->elevator_set_req_fn)
955 return e->ops->elevator_set_req_fn(q, rq, gfp_mask);

Quoted 8 lines

cfq_set_request() will finally call io_ioq_move(), but the iog is NULL, beacause the iogs in the hierarchy are not built yet. So kernel crashes.

956
957 rq->elevator_private = NULL;
958 return 0;
959 }

Thanks Gui. Good catch.

Quoted 71 lines
BUG: unable to handle kernel NULL pointer dereference at 000000bc
IP: [<c04ebf8f>] io_ioq_move+0xf2/0x109
*pde = 6cc00067
Oops: 0000 [#1] SMP
last sysfs file: /sys/block/hdb/queue/slice_idle
Modules linked in: ipv6 cpufreq_ondemand acpi_cpufreq dm_mirror dm_multipath sbs sbshc battery ac lp snd_hda_codec_realtek snd_hda_intel snd_hda_codec snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss snd_pcm serio_raw snd_timer rtc_cmos parport_pc snd r8169 button rtc_core parport soundcore mii i2c_i801 rtc_lib snd_page_alloc pcspkr i2c_core dm_region_hash dm_log dm_mod ext3 jbd mbcache uhci_hcd ohci_hcd ehci_hcd

Pid: 5431, comm: dd Not tainted (2.6.29-rc7-vivek #19) Veriton M460
EIP: 0060:[<c04ebf8f>] EFLAGS: 00010046 CPU: 0
EIP is at io_ioq_move+0xf2/0x109
EAX: f6203a88 EBX: f6792c94 ECX: f6203a84 EDX: 00000006
ESI: 00000000 EDI: 00000000 EBP: f6203a60 ESP: f6304c28
DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
Process dd (pid: 5431, ti=f6304000 task=f669dae0 task.ti=f6304000)

Stack:
 f62478c0 0100dd40 f6247908 f62d995c 00000000 00000000 f675b54c c04e9182
 f638e9b0 00000282 f62d99a4 f6325a2c c04e9113 f5a707c0 c04e7ae0 f675b000
 f62d95fc f6325a2c c04e8501 00000010 f631e4e8 f675b000 00080000 ffffff10
Call Trace:
 [<c04e9182>] changed_cgroup+0x6f/0x8d
 [<c04e9113>] changed_cgroup+0x0/0x8d
 [<c04e7ae0>] __call_for_each_cic+0x1b/0x25
 [<c04e8501>] cfq_set_request+0x158/0x2c7
 [<c06316e6>] _spin_unlock_irqrestore+0x5/0x6
 [<c04eb106>] elv_fq_set_request_io_group+0x2b/0x3e
 [<c04e83a9>] cfq_set_request+0x0/0x2c7
 [<c04dddcb>] elv_set_request+0x3e/0x4e
 [<c04df3da>] get_request+0x1ed/0x29b
 [<c04df9bb>] get_request_wait+0xdf/0xf2
 [<c04dfd89>] __make_request+0x2c6/0x372
 [<c049bd76>] do_mpage_readpage+0x4fe/0x5e3
 [<c04deba5>] generic_make_request+0x2d0/0x355
 [<c04dff47>] submit_bio+0x92/0x97
 [<c045bfcb>] add_to_page_cache_locked+0x8a/0xb7
 [<c049bfa4>] mpage_end_io_read+0x0/0x50
 [<c049b1b6>] mpage_bio_submit+0x19/0x1d
 [<c049bf9a>] mpage_readpages+0x9b/0xa5
 [<f7dd18c7>] ext3_readpages+0x0/0x15 [ext3]
 [<c0462192>] __do_page_cache_readahead+0xea/0x154
 [<f7dd2286>] ext3_get_block+0x0/0xbe [ext3]
 [<c045d34d>] generic_file_aio_read+0x276/0x569
 [<c047cdd9>] do_sync_read+0xbf/0xfe
 [<c043a3f2>] getnstimeofday+0x51/0xdb
 [<c0434d3c>] autoremove_wake_function+0x0/0x2d
 [<c041bdc3>] sched_slice+0x61/0x6a
 [<c0423114>] task_tick_fair+0x3d/0x60
 [<c04c1d79>] security_file_permission+0xc/0xd
 [<c047cd1a>] do_sync_read+0x0/0xfe
 [<c047d35a>] vfs_read+0x6c/0x8b
 [<c047d67e>] sys_read+0x3c/0x63
 [<c0402fc1>] sysenter_do_call+0x12/0x21
 [<c0630000>] schedule+0x551/0x830
Code: 08 31 c9 89 da e8 77 fc ff ff 8b 86 bc 00 00 00 85 ff 89 43 38 8d 46 60 89 43 40 74 1d 83 c4 0c 89 d8 5b 5e 5f 5d e9 aa f9 ff ff <8b> 86 bc 00 00 00 89 43 38 8d 46 60 89 43 40 83 c4 0c 5b 5e 5f
EIP: [<c04ebf8f>] io_ioq_move+0xf2/0x109 SS:ESP 0068:f6304c28

Changelog:

Make sure iogs in the hierarchy are built properly after moving a task to a new cgroup.

---
 block/cfq-iosched.c |    4 +++-
 block/elevator-fq.c |    1 +
 block/elevator-fq.h |    1 +
 3 files changed, 5 insertions(+), 1 deletions(-)
block/cfq-iosched.c +0 -0
index 0ecf7c7..6d7bb8a 100644

block/cfq-iosched.c line 16

Quoted 5 lines
block/cfq-iosched.c +2 -0
@@ -12,6 +12,8 @@
 #include <linux/rbtree.h>
 #include <linux/ioprio.h>
 #include <linux/blktrace_api.h>
+#include "elevator-fq.h"
+

I think above explicit inclusion of "elevator-fq.h" might be unnecessary as elevator.h includes elevator-fq.h and cfq-iosched.c is including elevator.h

block/cfq-iosched.c static void changed_cgroup(struct io_context *ioc, struct cfq_io_context *cic) · line 1091

Quoted 9 lines
block/cfq-iosched.c +1 -1
 /*
  * tunables
  */
@@ -1086,7 +1088,7 @@ static void changed_cgroup(struct io_context *ioc, struct cfq_io_context *cic)
 
 	spin_lock_irqsave(q->queue_lock, flags);
 
-	iog = io_lookup_io_group_current(q);
+	iog = io_get_io_group(q);

A one line comment here explaining the need to get_io_group instead of lookup_io_group will be nice.

Thanks
Vivek

Quoted 31 lines
block/cfq-iosched.c +0 -0
 	if (async_cfqq != NULL) {
 		__iog = cfqq_to_io_group(async_cfqq);
block/elevator-fq.c +1 -0
index df53418..f81cf6a 100644
@@ -1191,6 +1191,7 @@ struct io_group *io_get_io_group(struct request_queue *q)
 
 	return iog;
 }
+EXPORT_SYMBOL(io_get_io_group);
 
 void io_free_root_group(struct elevator_queue *e)
 {
block/elevator-fq.h +1 -0
index fc4110d..f17e425 100644
@@ -459,6 +459,7 @@ static inline struct io_group *ioq_to_io_group(struct io_queue *ioq)
 }
 
 #ifdef CONFIG_GROUP_IOSCHED
+extern struct io_group *io_get_io_group(struct request_queue *q);
 extern int io_group_allow_merge(struct request *rq, struct bio *bio);
 extern void io_ioq_move(struct elevator_queue *e, struct io_queue *ioq,
 					struct io_group *iog);
Signature · 1 line
1.5.4.rc3
#142 lore raw

PATCH 6/10 Separate out queue and data

In reply to Vivek Goyal

o So far noop, deadline and AS had one common structure called *_data which contained both the queue information where requests are queued and also common data used for scheduling. This patch breaks down this common structure in two parts, *_queue and *_data. This is along the lines of cfq where all the reuquests are queued in queue and common data and tunables are part of data.

o It does not change the functionality but this re-organization helps once noop, deadline and AS are changed to use hierarchical fair queuing.

o looks like queue_empty function is not required and we can check for q->nr_sorted in elevator layer to see if ioscheduler queues are empty or not.

---
 block/as-iosched.c       |  209 ++++++++++++++++++++++++++--------------------
 block/deadline-iosched.c |  117 ++++++++++++++++----------
 block/elevator.c         |  111 +++++++++++++++++++++----
 block/noop-iosched.c     |   59 ++++++-------
 include/linux/elevator.h |    8 ++-
 5 files changed, 319 insertions(+), 185 deletions(-)
block/as-iosched.c +118 -91
index 631f6f4..6d2890c 100644
@@ -79,13 +79,7 @@ enum anticipation_status {
 				 * or timed out */
 };
 
-struct as_data {
-	/*
-	 * run time data
-	 */
-
-	struct request_queue *q;	/* the "owner" queue */
-
+struct as_queue {
 	/*
 	 * requests (as_rq s) are present on both sort_list and fifo_list
 	 */
@@ -93,6 +87,14 @@ struct as_data {
 	struct list_head fifo_list[2];
 
 	struct request *next_rq[2];	/* next in sort order */
+	unsigned long last_check_fifo[2];
+	int write_batch_count;		/* max # of reqs in a write batch */
+	int current_write_count;	/* how many requests left this batch */
+	int write_batch_idled;		/* has the write batch gone idle? */
+};
+
+struct as_data {
+	struct request_queue *q;	/* the "owner" queue */
 	sector_t last_sector[2];	/* last REQ_SYNC & REQ_ASYNC sectors */
 
 	unsigned long exit_prob;	/* probability a task will exit while
@@ -104,23 +106,19 @@ struct as_data {
 	unsigned long new_ttime_mean;
 	u64 new_seek_total;		/* mean seek on new proc */
 	sector_t new_seek_mean;
-
 	unsigned long current_batch_expires;
-	unsigned long last_check_fifo[2];
+
 	int changed_batch;		/* 1: waiting for old batch to end */
 	int new_batch;			/* 1: waiting on first read complete */
-	int batch_data_dir;		/* current batch REQ_SYNC / REQ_ASYNC */
-	int write_batch_count;		/* max # of reqs in a write batch */
-	int current_write_count;	/* how many requests left this batch */
-	int write_batch_idled;		/* has the write batch gone idle? */
 
 	enum anticipation_status antic_status;
 	unsigned long antic_start;	/* jiffies: when it started */
 	struct timer_list antic_timer;	/* anticipatory scheduling timer */
-	struct work_struct antic_work;	/* Deferred unplugging */
+	struct work_struct antic_work;  /* Deferred unplugging */
 	struct io_context *io_context;	/* Identify the expected process */
 	int ioc_finished; /* IO associated with io_context is finished */
 	int nr_dispatched;
+	int batch_data_dir;		/* current batch REQ_SYNC / REQ_ASYNC */
 
 	/*
 	 * settings that change how the i/o scheduler behaves
@@ -261,13 +259,14 @@ static void as_put_io_context(struct request *rq)
 /*
  * rb tree support functions
  */
-#define RQ_RB_ROOT(ad, rq)	(&(ad)->sort_list[rq_is_sync((rq))])
+#define RQ_RB_ROOT(asq, rq)	(&(asq)->sort_list[rq_is_sync((rq))])
 
 static void as_add_rq_rb(struct as_data *ad, struct request *rq)
 {
 	struct request *alias;
+	struct as_queue *asq = elv_get_sched_queue(ad->q, rq);
 
-	while ((unlikely(alias = elv_rb_add(RQ_RB_ROOT(ad, rq), rq)))) {
+	while ((unlikely(alias = elv_rb_add(RQ_RB_ROOT(asq, rq), rq)))) {
 		as_move_to_dispatch(ad, alias);
 		as_antic_stop(ad);
 	}
@@ -275,7 +274,9 @@ static void as_add_rq_rb(struct as_data *ad, struct request *rq)
 
 static inline void as_del_rq_rb(struct as_data *ad, struct request *rq)
 {
-	elv_rb_del(RQ_RB_ROOT(ad, rq), rq);
+	struct as_queue *asq = elv_get_sched_queue(ad->q, rq);
+
+	elv_rb_del(RQ_RB_ROOT(asq, rq), rq);
 }
 
 /*
@@ -369,7 +370,7 @@ as_choose_req(struct as_data *ad, struct request *rq1, struct request *rq2)
  * what request to process next. Anticipation works on top of this.
  */
 static struct request *
-as_find_next_rq(struct as_data *ad, struct request *last)
+as_find_next_rq(struct as_data *ad, struct as_queue *asq, struct request *last)
 {
 	struct rb_node *rbnext = rb_next(&last->rb_node);
 	struct rb_node *rbprev = rb_prev(&last->rb_node);
@@ -385,7 +386,7 @@ as_find_next_rq(struct as_data *ad, struct request *last)
 	else {
 		const int data_dir = rq_is_sync(last);
 
-		rbnext = rb_first(&ad->sort_list[data_dir]);
+		rbnext = rb_first(&asq->sort_list[data_dir]);
 		if (rbnext && rbnext != &last->rb_node)
 			next = rb_entry_rq(rbnext);
 	}
@@ -790,9 +791,10 @@ static int as_can_anticipate(struct as_data *ad, struct request *rq)
 static void as_update_rq(struct as_data *ad, struct request *rq)
 {
 	const int data_dir = rq_is_sync(rq);
+	struct as_queue *asq = elv_get_sched_queue(ad->q, rq);
 
 	/* keep the next_rq cache up to date */
-	ad->next_rq[data_dir] = as_choose_req(ad, rq, ad->next_rq[data_dir]);
+	asq->next_rq[data_dir] = as_choose_req(ad, rq, asq->next_rq[data_dir]);
 
 	/*
 	 * have we been anticipating this request?
@@ -813,25 +815,26 @@ static void update_write_batch(struct as_data *ad)
 {
 	unsigned long batch = ad->batch_expire[REQ_ASYNC];
 	long write_time;
+	struct as_queue *asq = elv_get_sched_queue(ad->q, NULL);
 
 	write_time = (jiffies - ad->current_batch_expires) + batch;
 	if (write_time < 0)
 		write_time = 0;
 
-	if (write_time > batch && !ad->write_batch_idled) {
+	if (write_time > batch && !asq->write_batch_idled) {
 		if (write_time > batch * 3)
-			ad->write_batch_count /= 2;
+			asq->write_batch_count /= 2;
 		else
-			ad->write_batch_count--;
-	} else if (write_time < batch && ad->current_write_count == 0) {
+			asq->write_batch_count--;
+	} else if (write_time < batch && asq->current_write_count == 0) {
 		if (batch > write_time * 3)
-			ad->write_batch_count *= 2;
+			asq->write_batch_count *= 2;
 		else
-			ad->write_batch_count++;
+			asq->write_batch_count++;
 	}
 
-	if (ad->write_batch_count < 1)
-		ad->write_batch_count = 1;
+	if (asq->write_batch_count < 1)
+		asq->write_batch_count = 1;
 }
 
 /*
@@ -902,6 +905,7 @@ static void as_remove_queued_request(struct request_queue *q,
 	const int data_dir = rq_is_sync(rq);
 	struct as_data *ad = q->elevator->elevator_data;
 	struct io_context *ioc;
+	struct as_queue *asq = elv_get_sched_queue(q, rq);
 
 	WARN_ON(RQ_STATE(rq) != AS_RQ_QUEUED);
 
@@ -915,8 +919,8 @@ static void as_remove_queued_request(struct request_queue *q,
 	 * Update the "next_rq" cache if we are about to remove its
 	 * entry
 	 */
-	if (ad->next_rq[data_dir] == rq)
-		ad->next_rq[data_dir] = as_find_next_rq(ad, rq);
+	if (asq->next_rq[data_dir] == rq)
+		asq->next_rq[data_dir] = as_find_next_rq(ad, asq, rq);
 
 	rq_fifo_clear(rq);
 	as_del_rq_rb(ad, rq);
@@ -930,23 +934,23 @@ static void as_remove_queued_request(struct request_queue *q,
  *
  * See as_antic_expired comment.
  */
-static int as_fifo_expired(struct as_data *ad, int adir)
+static int as_fifo_expired(struct as_data *ad, struct as_queue *asq, int adir)
 {
 	struct request *rq;
 	long delta_jif;
 
-	delta_jif = jiffies - ad->last_check_fifo[adir];
+	delta_jif = jiffies - asq->last_check_fifo[adir];
 	if (unlikely(delta_jif < 0))
 		delta_jif = -delta_jif;
 	if (delta_jif < ad->fifo_expire[adir])
 		return 0;
 
-	ad->last_check_fifo[adir] = jiffies;
+	asq->last_check_fifo[adir] = jiffies;
 
-	if (list_empty(&ad->fifo_list[adir]))
+	if (list_empty(&asq->fifo_list[adir]))
 		return 0;
 
-	rq = rq_entry_fifo(ad->fifo_list[adir].next);
+	rq = rq_entry_fifo(asq->fifo_list[adir].next);
 
 	return time_after(jiffies, rq_fifo_time(rq));
 }
@@ -955,7 +959,7 @@ static int as_fifo_expired(struct as_data *ad, int adir)
  * as_batch_expired returns true if the current batch has expired. A batch
  * is a set of reads or a set of writes.
  */
-static inline int as_batch_expired(struct as_data *ad)
+static inline int as_batch_expired(struct as_data *ad, struct as_queue *asq)
 {
 	if (ad->changed_batch || ad->new_batch)
 		return 0;
@@ -965,7 +969,7 @@ static inline int as_batch_expired(struct as_data *ad)
 		return time_after(jiffies, ad->current_batch_expires);
 
 	return time_after(jiffies, ad->current_batch_expires)
-		|| ad->current_write_count == 0;
+		|| asq->current_write_count == 0;
 }
 
 /*
@@ -974,6 +978,7 @@ static inline int as_batch_expired(struct as_data *ad)
 static void as_move_to_dispatch(struct as_data *ad, struct request *rq)
 {
 	const int data_dir = rq_is_sync(rq);
+	struct as_queue *asq = elv_get_sched_queue(ad->q, rq);
 
 	BUG_ON(RB_EMPTY_NODE(&rq->rb_node));
 
@@ -996,12 +1001,12 @@ static void as_move_to_dispatch(struct as_data *ad, struct request *rq)
 			ad->io_context = NULL;
 		}
 
-		if (ad->current_write_count != 0)
-			ad->current_write_count--;
+		if (asq->current_write_count != 0)
+			asq->current_write_count--;
 	}
 	ad->ioc_finished = 0;
 
-	ad->next_rq[data_dir] = as_find_next_rq(ad, rq);
+	asq->next_rq[data_dir] = as_find_next_rq(ad, asq, rq);
 
 	/*
 	 * take it off the sort and fifo list, add to dispatch queue
@@ -1025,10 +1030,16 @@ static void as_move_to_dispatch(struct as_data *ad, struct request *rq)
 static int as_dispatch_request(struct request_queue *q, int force)
 {
 	struct as_data *ad = q->elevator->elevator_data;
-	const int reads = !list_empty(&ad->fifo_list[REQ_SYNC]);
-	const int writes = !list_empty(&ad->fifo_list[REQ_ASYNC]);
+	struct as_queue *asq = elv_select_sched_queue(q, force);
+	int reads, writes;
 	struct request *rq;
 
+	if (!asq)
+		return 0;
+
+	reads = !list_empty(&asq->fifo_list[REQ_SYNC]);
+	writes = !list_empty(&asq->fifo_list[REQ_ASYNC]);
+
 	if (unlikely(force)) {
 		/*
 		 * Forced dispatch, accounting is useless.  Reset
@@ -1043,25 +1054,25 @@ static int as_dispatch_request(struct request_queue *q, int force)
 		ad->changed_batch = 0;
 		ad->new_batch = 0;
 
-		while (ad->next_rq[REQ_SYNC]) {
-			as_move_to_dispatch(ad, ad->next_rq[REQ_SYNC]);
+		while (asq->next_rq[REQ_SYNC]) {
+			as_move_to_dispatch(ad, asq->next_rq[REQ_SYNC]);
 			dispatched++;
 		}
-		ad->last_check_fifo[REQ_SYNC] = jiffies;
+		asq->last_check_fifo[REQ_SYNC] = jiffies;
 
-		while (ad->next_rq[REQ_ASYNC]) {
-			as_move_to_dispatch(ad, ad->next_rq[REQ_ASYNC]);
+		while (asq->next_rq[REQ_ASYNC]) {
+			as_move_to_dispatch(ad, asq->next_rq[REQ_ASYNC]);
 			dispatched++;
 		}
-		ad->last_check_fifo[REQ_ASYNC] = jiffies;
+		asq->last_check_fifo[REQ_ASYNC] = jiffies;
 
 		return dispatched;
 	}
 
 	/* Signal that the write batch was uncontended, so we can't time it */
 	if (ad->batch_data_dir == REQ_ASYNC && !reads) {
-		if (ad->current_write_count == 0 || !writes)
-			ad->write_batch_idled = 1;
+		if (asq->current_write_count == 0 || !writes)
+			asq->write_batch_idled = 1;
 	}
 
 	if (!(reads || writes)
@@ -1070,14 +1081,14 @@ static int as_dispatch_request(struct request_queue *q, int force)
 		|| ad->changed_batch)
 		return 0;
 
-	if (!(reads && writes && as_batch_expired(ad))) {
+	if (!(reads && writes && as_batch_expired(ad, asq))) {
 		/*
 		 * batch is still running or no reads or no writes
 		 */
-		rq = ad->next_rq[ad->batch_data_dir];
+		rq = asq->next_rq[ad->batch_data_dir];
 
 		if (ad->batch_data_dir == REQ_SYNC && ad->antic_expire) {
-			if (as_fifo_expired(ad, REQ_SYNC))
+			if (as_fifo_expired(ad, asq, REQ_SYNC))
 				goto fifo_expired;
 
 			if (as_can_anticipate(ad, rq)) {
@@ -1101,7 +1112,7 @@ static int as_dispatch_request(struct request_queue *q, int force)
 	 */
 
 	if (reads) {
-		BUG_ON(RB_EMPTY_ROOT(&ad->sort_list[REQ_SYNC]));
+		BUG_ON(RB_EMPTY_ROOT(&asq->sort_list[REQ_SYNC]));
 
 		if (writes && ad->batch_data_dir == REQ_SYNC)
 			/*
@@ -1114,8 +1125,8 @@ static int as_dispatch_request(struct request_queue *q, int force)
 			ad->changed_batch = 1;
 		}
 		ad->batch_data_dir = REQ_SYNC;
-		rq = rq_entry_fifo(ad->fifo_list[REQ_SYNC].next);
-		ad->last_check_fifo[ad->batch_data_dir] = jiffies;
+		rq = rq_entry_fifo(asq->fifo_list[REQ_SYNC].next);
+		asq->last_check_fifo[ad->batch_data_dir] = jiffies;
 		goto dispatch_request;
 	}
 
@@ -1125,7 +1136,7 @@ static int as_dispatch_request(struct request_queue *q, int force)
 
 	if (writes) {
 dispatch_writes:
-		BUG_ON(RB_EMPTY_ROOT(&ad->sort_list[REQ_ASYNC]));
+		BUG_ON(RB_EMPTY_ROOT(&asq->sort_list[REQ_ASYNC]));
 
 		if (ad->batch_data_dir == REQ_SYNC) {
 			ad->changed_batch = 1;
@@ -1138,10 +1149,10 @@ dispatch_writes:
 			ad->new_batch = 0;
 		}
 		ad->batch_data_dir = REQ_ASYNC;
-		ad->current_write_count = ad->write_batch_count;
-		ad->write_batch_idled = 0;
-		rq = rq_entry_fifo(ad->fifo_list[REQ_ASYNC].next);
-		ad->last_check_fifo[REQ_ASYNC] = jiffies;
+		asq->current_write_count = asq->write_batch_count;
+		asq->write_batch_idled = 0;
+		rq = rq_entry_fifo(asq->fifo_list[REQ_ASYNC].next);
+		asq->last_check_fifo[REQ_ASYNC] = jiffies;
 		goto dispatch_request;
 	}
 
@@ -1153,9 +1164,9 @@ dispatch_request:
 	 * If a request has expired, service it.
 	 */
 
-	if (as_fifo_expired(ad, ad->batch_data_dir)) {
+	if (as_fifo_expired(ad, asq, ad->batch_data_dir)) {
 fifo_expired:
-		rq = rq_entry_fifo(ad->fifo_list[ad->batch_data_dir].next);
+		rq = rq_entry_fifo(asq->fifo_list[ad->batch_data_dir].next);
 	}
 
 	if (ad->changed_batch) {
@@ -1188,6 +1199,7 @@ static void as_add_request(struct request_queue *q, struct request *rq)
 {
 	struct as_data *ad = q->elevator->elevator_data;
 	int data_dir;
+	struct as_queue *asq = elv_get_sched_queue(q, rq);
 
 	RQ_SET_STATE(rq, AS_RQ_NEW);
 
@@ -1206,7 +1218,7 @@ static void as_add_request(struct request_queue *q, struct request *rq)
 	 * set expire time and add to fifo list
 	 */
 	rq_set_fifo_time(rq, jiffies + ad->fifo_expire[data_dir]);
-	list_add_tail(&rq->queuelist, &ad->fifo_list[data_dir]);
+	list_add_tail(&rq->queuelist, &asq->fifo_list[data_dir]);
 
 	as_update_rq(ad, rq); /* keep state machine up to date */
 	RQ_SET_STATE(rq, AS_RQ_QUEUED);
@@ -1228,31 +1240,20 @@ static void as_deactivate_request(struct request_queue *q, struct request *rq)
 		atomic_inc(&RQ_IOC(rq)->aic->nr_dispatched);
 }
 
-/*
- * as_queue_empty tells us if there are requests left in the device. It may
- * not be the case that a driver can get the next request even if the queue
- * is not empty - it is used in the block layer to check for plugging and
- * merging opportunities
- */
-static int as_queue_empty(struct request_queue *q)
-{
-	struct as_data *ad = q->elevator->elevator_data;
-
-	return list_empty(&ad->fifo_list[REQ_ASYNC])
-		&& list_empty(&ad->fifo_list[REQ_SYNC]);
-}
-
 static int
 as_merge(struct request_queue *q, struct request **req, struct bio *bio)
 {
-	struct as_data *ad = q->elevator->elevator_data;
 	sector_t rb_key = bio->bi_sector + bio_sectors(bio);
 	struct request *__rq;
+	struct as_queue *asq = elv_get_sched_queue_current(q);
+
+	if (!asq)
+		return ELEVATOR_NO_MERGE;
 
 	/*
 	 * check for front merge
 	 */
-	__rq = elv_rb_find(&ad->sort_list[bio_data_dir(bio)], rb_key);
+	__rq = elv_rb_find(&asq->sort_list[bio_data_dir(bio)], rb_key);
 	if (__rq && elv_rq_merge_ok(__rq, bio)) {
 		*req = __rq;
 		return ELEVATOR_FRONT_MERGE;
@@ -1339,6 +1340,41 @@ static int as_may_queue(struct request_queue *q, int rw)
 	return ret;
 }
 
+/* Called with queue lock held */
+static void *as_alloc_as_queue(struct request_queue *q,
+				struct elevator_queue *eq, gfp_t gfp_mask)
+{
+	struct as_queue *asq;
+	struct as_data *ad = eq->elevator_data;
+
+	asq = kmalloc_node(sizeof(*asq), gfp_mask | __GFP_ZERO, q->node);
+	if (asq == NULL)
+		goto out;
+
+	INIT_LIST_HEAD(&asq->fifo_list[REQ_SYNC]);
+	INIT_LIST_HEAD(&asq->fifo_list[REQ_ASYNC]);
+	asq->sort_list[REQ_SYNC] = RB_ROOT;
+	asq->sort_list[REQ_ASYNC] = RB_ROOT;
+	if (ad)
+		asq->write_batch_count = ad->batch_expire[REQ_ASYNC] / 10;
+	else
+		asq->write_batch_count = default_write_batch_expire / 10;
+
+	if (asq->write_batch_count < 2)
+		asq->write_batch_count = 2;
+out:
+	return asq;
+}
+
+static void as_free_as_queue(struct elevator_queue *e, void *sched_queue)
+{
+	struct as_queue *asq = sched_queue;
+
+	BUG_ON(!list_empty(&asq->fifo_list[REQ_SYNC]));
+	BUG_ON(!list_empty(&asq->fifo_list[REQ_ASYNC]));
+	kfree(asq);
+}
+
 static void as_exit_queue(struct elevator_queue *e)
 {
 	struct as_data *ad = e->elevator_data;
@@ -1346,9 +1382,6 @@ static void as_exit_queue(struct elevator_queue *e)
 	del_timer_sync(&ad->antic_timer);
 	cancel_work_sync(&ad->antic_work);
 
-	BUG_ON(!list_empty(&ad->fifo_list[REQ_SYNC]));
-	BUG_ON(!list_empty(&ad->fifo_list[REQ_ASYNC]));
-
 	put_io_context(ad->io_context);
 	kfree(ad);
 }
@@ -1372,10 +1405,6 @@ static void *as_init_queue(struct request_queue *q)
 	init_timer(&ad->antic_timer);
 	INIT_WORK(&ad->antic_work, as_work_handler);
 
-	INIT_LIST_HEAD(&ad->fifo_list[REQ_SYNC]);
-	INIT_LIST_HEAD(&ad->fifo_list[REQ_ASYNC]);
-	ad->sort_list[REQ_SYNC] = RB_ROOT;
-	ad->sort_list[REQ_ASYNC] = RB_ROOT;
 	ad->fifo_expire[REQ_SYNC] = default_read_expire;
 	ad->fifo_expire[REQ_ASYNC] = default_write_expire;
 	ad->antic_expire = default_antic_expire;
@@ -1383,9 +1412,6 @@ static void *as_init_queue(struct request_queue *q)
 	ad->batch_expire[REQ_ASYNC] = default_write_batch_expire;
 
 	ad->current_batch_expires = jiffies + ad->batch_expire[REQ_SYNC];
-	ad->write_batch_count = ad->batch_expire[REQ_ASYNC] / 10;
-	if (ad->write_batch_count < 2)
-		ad->write_batch_count = 2;
 
 	return ad;
 }
@@ -1482,7 +1508,6 @@ static struct elevator_type iosched_as = {
 		.elevator_add_req_fn =		as_add_request,
 		.elevator_activate_req_fn =	as_activate_request,
 		.elevator_deactivate_req_fn = 	as_deactivate_request,
-		.elevator_queue_empty_fn =	as_queue_empty,
 		.elevator_completed_req_fn =	as_completed_request,
 		.elevator_former_req_fn =	elv_rb_former_request,
 		.elevator_latter_req_fn =	elv_rb_latter_request,
@@ -1490,6 +1515,8 @@ static struct elevator_type iosched_as = {
 		.elevator_init_fn =		as_init_queue,
 		.elevator_exit_fn =		as_exit_queue,
 		.trim =				as_trim,
+		.elevator_alloc_sched_queue_fn = as_alloc_as_queue,
+		.elevator_free_sched_queue_fn = as_free_as_queue,
 	},
 
 	.elevator_attrs = as_attrs,
block/deadline-iosched.c +71 -46

266 lines not on this page · raw message

block/elevator.c +96 -15

179 lines not on this page · raw message

block/noop-iosched.c +28 -31

111 lines not on this page · raw message

include/linux/elevator.h +6 -2
index 6f2dea5..bb5ae3a 100644
@@ -30,8 +30,9 @@ typedef void (elevator_deactivate_req_fn) (struct request_queue *, struct reques
 
 typedef void *(elevator_init_fn) (struct request_queue *);
 typedef void (elevator_exit_fn) (struct elevator_queue *);
-#ifdef CONFIG_ELV_FAIR_QUEUING
+typedef void* (elevator_alloc_sched_queue_fn) (struct request_queue *q, struct elevator_queue *eq, gfp_t);
 typedef void (elevator_free_sched_queue_fn) (struct elevator_queue*, void *);
+#ifdef CONFIG_ELV_FAIR_QUEUING
 typedef void (elevator_active_ioq_set_fn) (struct request_queue*, void *);
 typedef void (elevator_active_ioq_reset_fn) (struct request_queue *);
 typedef void (elevator_arm_slice_timer_fn) (struct request_queue*, void*);
@@ -68,8 +69,9 @@ struct elevator_ops
 	elevator_exit_fn *elevator_exit_fn;
 	void (*trim)(struct io_context *);
 
-#ifdef CONFIG_ELV_FAIR_QUEUING
+	elevator_alloc_sched_queue_fn *elevator_alloc_sched_queue_fn;
 	elevator_free_sched_queue_fn *elevator_free_sched_queue_fn;
+#ifdef CONFIG_ELV_FAIR_QUEUING
 	elevator_active_ioq_set_fn *elevator_active_ioq_set_fn;
 	elevator_active_ioq_reset_fn *elevator_active_ioq_reset_fn;
 
@@ -109,6 +111,7 @@ struct elevator_queue
 {
 	struct elevator_ops *ops;
 	void *elevator_data;
+	void *sched_queue;
 	struct kobject kobj;
 	struct elevator_type *elevator_type;
 	struct mutex sysfs_lock;
@@ -256,5 +259,6 @@ static inline int elv_iosched_fair_queuing_enabled(struct elevator_queue *e)
 #endif /* ELV_IOSCHED_FAIR_QUEUING */
 extern void *elv_get_sched_queue(struct request_queue *q, struct request *rq);
 extern void *elv_select_sched_queue(struct request_queue *q, int force);
+extern void *elv_get_sched_queue_current(struct request_queue *q);
 #endif /* CONFIG_BLOCK */
 #endif
Signature · 1 line
1.6.0.1
#143 lore raw

PATCH 7/10 Prepare elevator layer for single queue schedulers

In reply to Vivek Goyal

Elevator layer now has support for hierarchical fair queuing. cfq has been migrated to make use of it and now it is time to do groundwork for noop, deadline and AS.

noop deadline and AS don't maintain separate queues for different processes. There is only one single queue. Effectively one can think that in hierarchical setup, there will be one queue per cgroup where requests from all the processes in the cgroup will be queued.

Generally io scheduler takes care of creating queues. Because there is only one queue here, we have modified common layer to take care of queue creation and some other functionality. This special casing helps in keeping the changes to noop, deadline and AS to the minimum.

---
 block/elevator-fq.c      |  153 ++++++++++++++++++++++++++++++++++++++++++++++
 block/elevator-fq.h      |   67 ++++++++++++++++++++
 block/elevator.c         |   35 ++++++++++-
 include/linux/elevator.h |   14 ++++
 4 files changed, 268 insertions(+), 1 deletions(-)
block/elevator-fq.c +153 -0
index 389f68e..172f9e3 100644
@@ -857,6 +857,12 @@ void io_put_io_group_queues(struct elevator_queue *e, struct io_group *iog)
 
 	/* Free up async idle queue */
 	elv_release_ioq(e, &iog->async_idle_queue);
+
+#ifdef CONFIG_GROUP_IOSCHED
+	/* Optimization for io schedulers having single ioq */
+	if (elv_iosched_single_ioq(e))
+		elv_release_ioq(e, &iog->ioq);
+#endif
 }
 
 
@@ -1538,6 +1544,153 @@ void elv_fq_set_request_io_group(struct request_queue *q,
 	rq->iog = iog;
 }
 
+/*
+ * Find/Create the io queue the rq should go in. This is an optimization
+ * for the io schedulers (noop, deadline and AS) which maintain only single
+ * io queue per cgroup. In this case common layer can just maintain a
+ * pointer in group data structure and keeps track of it.
+ *
+ * For the io schdulers like cfq, which maintain multiple io queues per
+ * cgroup, and decide the io queue  of request based on process, this
+ * function is not invoked.
+ */
+int elv_fq_set_request_ioq(struct request_queue *q, struct request *rq,
+					gfp_t gfp_mask)
+{
+	struct elevator_queue *e = q->elevator;
+	unsigned long flags;
+	struct io_queue *ioq = NULL, *new_ioq = NULL;
+	struct io_group *iog;
+	void *sched_q = NULL, *new_sched_q = NULL;
+
+	if (!elv_iosched_fair_queuing_enabled(e))
+		return 0;
+
+	might_sleep_if(gfp_mask & __GFP_WAIT);
+	spin_lock_irqsave(q->queue_lock, flags);
+
+	/* Determine the io group request belongs to */
+	iog = rq->iog;
+	BUG_ON(!iog);
+
+retry:
+	/* Get the iosched queue */
+	ioq = io_group_ioq(iog);
+	if (!ioq) {
+		/* io queue and sched_queue needs to be allocated */
+		BUG_ON(!e->ops->elevator_alloc_sched_queue_fn);
+
+		if (new_sched_q) {
+			goto alloc_ioq;
+		} else if (gfp_mask & __GFP_WAIT) {
+			/*
+			 * Inform the allocator of the fact that we will
+			 * just repeat this allocation if it fails, to allow
+			 * the allocator to do whatever it needs to attempt to
+			 * free memory.
+			 */
+			spin_unlock_irq(q->queue_lock);
+			/* Call io scheduer to create scheduler queue */
+			new_sched_q = e->ops->elevator_alloc_sched_queue_fn(q,
+					e, gfp_mask | __GFP_NOFAIL
+					| __GFP_ZERO);
+			spin_lock_irq(q->queue_lock);
+			goto retry;
+		} else {
+			sched_q = e->ops->elevator_alloc_sched_queue_fn(q, e,
+						gfp_mask | __GFP_ZERO);
+			if (!sched_q)
+				goto queue_fail;
+		}
+
+alloc_ioq:
+		if (new_ioq) {
+			ioq = new_ioq;
+			new_ioq = NULL;
+			sched_q = new_sched_q;
+			new_sched_q = NULL;
+		} else if (gfp_mask & __GFP_WAIT) {
+			/*
+			 * Inform the allocator of the fact that we will
+			 * just repeat this allocation if it fails, to allow
+			 * the allocator to do whatever it needs to attempt to
+			 * free memory.
+			 */
+			spin_unlock_irq(q->queue_lock);
+			new_ioq = elv_alloc_ioq(q, gfp_mask | __GFP_NOFAIL
+							| __GFP_ZERO);
+			spin_lock_irq(q->queue_lock);
+			goto retry;
+		} else {
+			ioq = elv_alloc_ioq(q, gfp_mask | __GFP_ZERO);
+			if (!ioq) {
+				e->ops->elevator_free_sched_queue_fn(e,
+							sched_q);
+				sched_q = NULL;
+				goto queue_fail;
+			}
+		}
+
+		elv_init_ioq(e, ioq, sched_q, IOPRIO_CLASS_BE, 4, 1);
+		io_group_set_ioq(iog, ioq);
+		elv_mark_ioq_sync(ioq);
+	}
+
+	if (new_sched_q)
+		e->ops->elevator_free_sched_queue_fn(q->elevator, sched_q);
+
+	if (new_ioq)
+		elv_free_ioq(new_ioq);
+
+	/* Request reference */
+	elv_get_ioq(ioq);
+	rq->ioq = ioq;
+	spin_unlock_irqrestore(q->queue_lock, flags);
+	return 0;
+
+queue_fail:
+	WARN_ON((gfp_mask & __GFP_WAIT) && !ioq);
+	elv_schedule_dispatch(q);
+	spin_unlock_irqrestore(q->queue_lock, flags);
+	return 1;
+}
+
+/*
+ * Find out the io queue of current task. Optimization for single ioq
+ * per io group io schedulers.
+ */
+struct io_queue *elv_lookup_ioq_current(struct request_queue *q)
+{
+	struct io_group *iog;
+
+	/* Determine the io group and io queue of the bio submitting task */
+	iog = io_lookup_io_group_current(q);
+	if (!iog) {
+		/* May be task belongs to a cgroup for which io group has
+		 * not been setup yet. */
+		return NULL;
+	}
+	return io_group_ioq(iog);
+}
+
+/*
+ * This request has been serviced. Clean up ioq info and drop the reference.
+ * Again this is called only for single queue per cgroup schedulers (noop,
+ * deadline, AS).
+ */
+void elv_fq_unset_request_ioq(struct request_queue *q, struct request *rq)
+{
+	struct io_queue *ioq = rq->ioq;
+
+	if (!elv_iosched_fair_queuing_enabled(q->elevator))
+		return;
+
+	if (ioq) {
+		rq->ioq = NULL;
+		elv_put_ioq(ioq);
+	}
+}
+
 #else /* GROUP_IOSCHED */
 void bfq_init_entity(struct io_entity *entity, struct io_group *iog)
 {
block/elevator-fq.h +67 -0

90 lines not on this page · raw message

block/elevator.c +34 -1
index 5df13c4..bce4421 100644
@@ -207,6 +207,14 @@ static void *elevator_alloc_sched_queue(struct request_queue *q,
 {
 	void *sched_queue = NULL;
 
+	/*
+	 * If fair queuing is enabled, then queue allocation takes place
+	 * during set_request() functions when request actually comes
+	 * in.
+	 */
+	if (elv_iosched_fair_queuing_enabled(eq))
+		return NULL;
+
 	if (eq->ops->elevator_alloc_sched_queue_fn) {
 		sched_queue = eq->ops->elevator_alloc_sched_queue_fn(q, eq,
 								GFP_KERNEL);
@@ -936,6 +944,13 @@ int elv_set_request(struct request_queue *q, struct request *rq, gfp_t gfp_mask)
 
 	elv_fq_set_request_io_group(q, rq);
 
+	/*
+	 * Optimization for noop, deadline and AS which maintain only single
+	 * ioq per io group
+	 */
+	if (elv_iosched_single_ioq(e))
+		return elv_fq_set_request_ioq(q, rq, gfp_mask);
+
 	if (e->ops->elevator_set_req_fn)
 		return e->ops->elevator_set_req_fn(q, rq, gfp_mask);
 
@@ -947,6 +962,15 @@ void elv_put_request(struct request_queue *q, struct request *rq)
 {
 	struct elevator_queue *e = q->elevator;
 
+	/*
+	 * Optimization for noop, deadline and AS which maintain only single
+	 * ioq per io group
+	 */
+	if (elv_iosched_single_ioq(e)) {
+		elv_fq_unset_request_ioq(q, rq);
+		return;
+	}
+
 	if (e->ops->elevator_put_req_fn)
 		e->ops->elevator_put_req_fn(rq);
 }
@@ -1329,9 +1353,18 @@ EXPORT_SYMBOL(elv_select_sched_queue);
 
 /*
  * Get the io scheduler queue pointer for current task.
+ *
+ * If fair queuing is enabled, determine the io group of task and retrieve
+ * the ioq pointer from that. This is used by only single queue ioschedulers
+ * for retrieving the queue associated with the group to decide whether the
+ * new bio can do a front merge or not.
  */
 void *elv_get_sched_queue_current(struct request_queue *q)
 {
-	return q->elevator->sched_queue;
+	/* Fair queuing is not enabled. There is only one queue. */
+	if (!elv_iosched_fair_queuing_enabled(q->elevator))
+		return q->elevator->sched_queue;
+
+	return ioq_sched_queue(elv_lookup_ioq_current(q));
 }
 EXPORT_SYMBOL(elv_get_sched_queue_current);
include/linux/elevator.h +14 -0
index bb5ae3a..8cee877 100644
@@ -245,17 +245,31 @@ enum {
 /* iosched wants to use fq logic of elevator layer */
 #define	ELV_IOSCHED_NEED_FQ	1
 
+/* iosched maintains only single ioq per group.*/
+#define ELV_IOSCHED_SINGLE_IOQ        2
+
 static inline int elv_iosched_fair_queuing_enabled(struct elevator_queue *e)
 {
 	return (e->elevator_type->elevator_features) & ELV_IOSCHED_NEED_FQ;
 }
 
+static inline int elv_iosched_single_ioq(struct elevator_queue *e)
+{
+	return (e->elevator_type->elevator_features) & ELV_IOSCHED_SINGLE_IOQ;
+}
+
 #else /* ELV_IOSCHED_FAIR_QUEUING */
 
 static inline int elv_iosched_fair_queuing_enabled(struct elevator_queue *e)
 {
 	return 0;
 }
+
+static inline int elv_iosched_single_ioq(struct elevator_queue *e)
+{
+	return 0;
+}
+
 #endif /* ELV_IOSCHED_FAIR_QUEUING */
 extern void *elv_get_sched_queue(struct request_queue *q, struct request *rq);
 extern void *elv_select_sched_queue(struct request_queue *q, int force);
Signature · 1 line
1.6.0.1
#144 lore raw

PATCH 8/10 noop changes for hierarchical fair queuing

In reply to Vivek Goyal

This patch changes noop to use queue scheduling code from elevator layer. One can go back to old noop by deselecting CONFIG_IOSCHED_NOOP_HIER.

---
 block/Kconfig.iosched |   11 +++++++++++
 block/noop-iosched.c  |    3 +++
 2 files changed, 14 insertions(+), 0 deletions(-)
block/Kconfig.iosched +11 -0
index a91a807..9da6657 100644
@@ -25,6 +25,17 @@ config IOSCHED_NOOP
 	  that do their own scheduling and require only minimal assistance from
 	  the kernel.
 
+config IOSCHED_NOOP_HIER
+	bool "Noop Hierarchical Scheduling support"
+	depends on IOSCHED_NOOP && CGROUPS
+	select ELV_FAIR_QUEUING
+	select GROUP_IOSCHED
+	default n
+	---help---
+	  Enable hierarhical scheduling in noop. In this mode noop keeps
+	  one IO queue per cgroup instead of a global queue. Elevator
+	  fair queuing logic ensures fairness among various queues.
+
 config IOSCHED_AS
 	tristate "Anticipatory I/O scheduler"
 	default y
block/noop-iosched.c +3 -0
index d587832..73e571d 100644
@@ -92,6 +92,9 @@ static struct elevator_type elevator_noop = {
 		.elevator_alloc_sched_queue_fn	= noop_alloc_noop_queue,
 		.elevator_free_sched_queue_fn	= noop_free_noop_queue,
 	},
+#ifdef CONFIG_IOSCHED_NOOP_HIER
+	.elevator_features = ELV_IOSCHED_NEED_FQ | ELV_IOSCHED_SINGLE_IOQ,
+#endif
 	.elevator_name = "noop",
 	.elevator_owner = THIS_MODULE,
 };
Signature · 1 line
1.6.0.1
#145 lore raw

PATCH 9/10 deadline changes for hierarchical fair queuing

In reply to Vivek Goyal

This patch changes deadline to use queue scheduling code from elevator layer. One can go back to old deadline by selecting CONFIG_IOSCHED_DEADLINE_HIER.

---
 block/Kconfig.iosched    |   11 +++++++++++
 block/deadline-iosched.c |    3 +++
 2 files changed, 14 insertions(+), 0 deletions(-)
block/Kconfig.iosched +11 -0
index 9da6657..3a9e7d7 100644
@@ -55,6 +55,17 @@ config IOSCHED_DEADLINE
 	  a disk at any one time, its behaviour is almost identical to the
 	  anticipatory I/O scheduler and so is a good choice.
 
+config IOSCHED_DEADLINE_HIER
+	bool "Deadline Hierarchical Scheduling support"
+	depends on IOSCHED_DEADLINE && CGROUPS
+	select ELV_FAIR_QUEUING
+	select GROUP_IOSCHED
+	default n
+	---help---
+	  Enable hierarhical scheduling in deadline. In this mode deadline keeps
+	  one IO queue per cgroup instead of a global queue. Elevator
+	  fair queuing logic ensures fairness among various queues.
+
 config IOSCHED_CFQ
 	tristate "CFQ I/O scheduler"
 	select ELV_FAIR_QUEUING
block/deadline-iosched.c +3 -0
index 5e65041..27b77b9 100644
@@ -477,6 +477,9 @@ static struct elevator_type iosched_deadline = {
 		.elevator_alloc_sched_queue_fn = deadline_alloc_deadline_queue,
 		.elevator_free_sched_queue_fn = deadline_free_deadline_queue,
 	},
+#ifdef CONFIG_IOSCHED_DEADLINE_HIER
+	.elevator_features = ELV_IOSCHED_NEED_FQ | ELV_IOSCHED_SINGLE_IOQ,
+#endif
 	.elevator_attrs = deadline_attrs,
 	.elevator_name = "deadline",
 	.elevator_owner = THIS_MODULE,
Signature · 1 line
1.6.0.1
#146 lore raw

PATCH 10/10 anticipatory changes for hierarchical fair queuing

In reply to Vivek Goyal

This patch changes anticipatory scheduler to use queue scheduling code from elevator layer. One can go back to old as by deselecting CONFIG_IOSCHED_AS_HIER.

TODO/Issues
===========
- AS anticipation logic does not seem to be sufficient to provide BW difference if two "dd" are going in two different cgroups. Needs to be looked into.

- AS write batch number of request adjustment happens upon every W->R batch direction switch. This automatic adjustment depends on how much time a read is taking after a W->R switch.

This does not gel very well when hierarhical scheduling is enabled and every io group can have its separate read/write batch. Now if io group switching takes place it creates issues.

Currently I have disabled write batch length adjustment in hierarchical mode.

- Currently performance seems to be very bad in hierarhical mode. Needs to be looked into.

- I think the whole idea of common layer doing time slice switching between queues and then queue in turn running timed batches is not very good. May be AS can maintain two queues (one for READS and other for WRITES) and let common layer do the time slice switching between these two queues.

---
 block/Kconfig.iosched    |   12 +++
 block/as-iosched.c       |  176 +++++++++++++++++++++++++++++++++++++++++++++-
 block/elevator-fq.c      |   77 ++++++++++++++++----
 include/linux/elevator.h |   16 ++++
 4 files changed, 265 insertions(+), 16 deletions(-)
block/Kconfig.iosched +12 -0
index 3a9e7d7..77fc786 100644
@@ -45,6 +45,18 @@ config IOSCHED_AS
 	  deadline I/O scheduler, it can also be slower in some cases
 	  especially some database loads.
 
+config IOSCHED_AS_HIER
+	bool "Anticipatory Hierarchical Scheduling support"
+	depends on IOSCHED_AS && CGROUPS
+	select ELV_FAIR_QUEUING
+	select GROUP_IOSCHED
+	default n
+	---help---
+	  Enable hierarhical scheduling in anticipatory. In this mode
+	  anticipatory keeps one IO queue per cgroup instead of a global
+	  queue. Elevator fair queuing logic ensures fairness among various
+	  queues.
+
 config IOSCHED_DEADLINE
 	tristate "Deadline I/O scheduler"
 	default y
block/as-iosched.c +175 -1

214 lines not on this page · raw message

block/elevator-fq.c +62 -15

125 lines not on this page · raw message

include/linux/elevator.h +16 -0
index 8cee877..9b5c9b9 100644
@@ -40,6 +40,7 @@ typedef int (elevator_should_preempt_fn) (struct request_queue*, void*,
 						struct request*);
 typedef int (elevator_update_idle_window_fn) (struct elevator_queue*, void*,
 						struct request*);
+typedef int (elevator_expire_ioq_fn) (struct request_queue*, void *, int, int);
 #endif
 
 struct elevator_ops
@@ -78,6 +79,7 @@ struct elevator_ops
 	elevator_arm_slice_timer_fn *elevator_arm_slice_timer_fn;
 	elevator_should_preempt_fn *elevator_should_preempt_fn;
 	elevator_update_idle_window_fn *elevator_update_idle_window_fn;
+	elevator_expire_ioq_fn  *elevator_expire_ioq_fn;
 #endif
 };
 
@@ -248,6 +250,9 @@ enum {
 /* iosched maintains only single ioq per group.*/
 #define ELV_IOSCHED_SINGLE_IOQ        2
 
+/* iosched does not need anticipation/idling logic support from common layer */
+#define ELV_IOSCHED_DONT_IDLE	4
+
 static inline int elv_iosched_fair_queuing_enabled(struct elevator_queue *e)
 {
 	return (e->elevator_type->elevator_features) & ELV_IOSCHED_NEED_FQ;
@@ -258,6 +263,12 @@ static inline int elv_iosched_single_ioq(struct elevator_queue *e)
 	return (e->elevator_type->elevator_features) & ELV_IOSCHED_SINGLE_IOQ;
 }
 
+/* returns 1 if elevator layer should enable its idling logic, 0 otherwise */
+static inline int elv_gen_idling_enabled(struct elevator_queue *e)
+{
+	return !((e->elevator_type->elevator_features) & ELV_IOSCHED_DONT_IDLE);
+}
+
 #else /* ELV_IOSCHED_FAIR_QUEUING */
 
 static inline int elv_iosched_fair_queuing_enabled(struct elevator_queue *e)
@@ -270,6 +281,11 @@ static inline int elv_iosched_single_ioq(struct elevator_queue *e)
 	return 0;
 }
 
+static inline int elv_gen_idling_enabled(struct elevator_queue *e)
+{
+	return 0;
+}
+
 #endif /* ELV_IOSCHED_FAIR_QUEUING */
 extern void *elv_get_sched_queue(struct request_queue *q, struct request *rq);
 extern void *elv_select_sched_queue(struct request_queue *q, int force);
Signature · 1 line
1.6.0.1

A message that is not in the archive (sent off-list or not yet received): 1236823015-4183-11-git-send-email-vgoyal-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org

#148 lore raw

PATCH 1/1 IO Controller: No need to stop idling in as

Review

Quoted 8 lines · Vivek Goyal
(quoted diff) +4 -1
 		.elevator_alloc_sched_queue_fn = as_alloc_as_queue,
 		.elevator_free_sched_queue_fn = as_free_as_queue,
+#ifdef CONFIG_IOSCHED_AS_HIER
+		.elevator_expire_ioq_fn =       as_expire_ioq,
+		.elevator_active_ioq_set_fn =   as_active_ioq_set,
 	},
-
+	.elevator_features = ELV_IOSCHED_NEED_FQ | ELV_IOSCHED_SINGLE_IOQ | ELV_IOSCHED_DONT_IDLE,

Hi Vivek,

I found the IO Controller doesn't work in as.
I dig into this issue, and notice that you stop idling in as. IMHO, this will cause active ioq is always expired when tring to choose a new ioq to serve(elv_fq_select_ioq). Because idling is disabled, active ioq can't be kept anymore.
So i just get rid of ELV_IOSCHED_DONT_IDLE, and it works fine this time.

---
 block/as-iosched.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
block/as-iosched.c +1 -1
index 27c14a7..499c521 100644
@@ -1689,7 +1689,7 @@ static struct elevator_type iosched_as = {
 		.elevator_expire_ioq_fn =       as_expire_ioq,
 		.elevator_active_ioq_set_fn =   as_active_ioq_set,
 	},
-	.elevator_features = ELV_IOSCHED_NEED_FQ | ELV_IOSCHED_SINGLE_IOQ | ELV_IOSCHED_DONT_IDLE,
+	.elevator_features = ELV_IOSCHED_NEED_FQ | ELV_IOSCHED_SINGLE_IOQ,
 #else
 	},
 #endif
Signature · 1 line
1.5.4.rc3
#149 lore raw

PATCH 1/1 IO Controller: No need to stop idling in as

In reply to Vivek Goyal

Review

Quoted 8 lines · Vivek Goyal
(quoted diff) +4 -1
 		.elevator_alloc_sched_queue_fn = as_alloc_as_queue,
 		.elevator_free_sched_queue_fn = as_free_as_queue,
+#ifdef CONFIG_IOSCHED_AS_HIER
+		.elevator_expire_ioq_fn =       as_expire_ioq,
+		.elevator_active_ioq_set_fn =   as_active_ioq_set,
 	},
-
+	.elevator_features = ELV_IOSCHED_NEED_FQ | ELV_IOSCHED_SINGLE_IOQ | ELV_IOSCHED_DONT_IDLE,

Hi Vivek,

I found the IO Controller doesn't work in as.
I dig into this issue, and notice that you stop idling in as. IMHO, this will cause active ioq is always expired when tring to choose a new ioq to serve(elv_fq_select_ioq). Because idling is disabled, active ioq can't be kept anymore.
So i just get rid of ELV_IOSCHED_DONT_IDLE, and it works fine this time.

---
 block/as-iosched.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
block/as-iosched.c +1 -1
index 27c14a7..499c521 100644
@@ -1689,7 +1689,7 @@ static struct elevator_type iosched_as = {
 		.elevator_expire_ioq_fn =       as_expire_ioq,
 		.elevator_active_ioq_set_fn =   as_active_ioq_set,
 	},
-	.elevator_features = ELV_IOSCHED_NEED_FQ | ELV_IOSCHED_SINGLE_IOQ | ELV_IOSCHED_DONT_IDLE,
+	.elevator_features = ELV_IOSCHED_NEED_FQ | ELV_IOSCHED_SINGLE_IOQ,
 #else
 	},
 #endif
Signature · 1 line
1.5.4.rc3

A message that is not in the archive (sent off-list or not yet received): [email protected]

#151 lore raw

Re: [PATCH] IO Controller: No need to stop idling in as

Quoted 13 lines · Gui Jianfeng

Review

Quoted 8 lines · Vivek Goyal
(quoted diff) +4 -1
 		.elevator_alloc_sched_queue_fn = as_alloc_as_queue,
 		.elevator_free_sched_queue_fn = as_free_as_queue,
+#ifdef CONFIG_IOSCHED_AS_HIER
+		.elevator_expire_ioq_fn =       as_expire_ioq,
+		.elevator_active_ioq_set_fn =   as_active_ioq_set,
 	},
-
+	.elevator_features = ELV_IOSCHED_NEED_FQ | ELV_IOSCHED_SINGLE_IOQ | ELV_IOSCHED_DONT_IDLE,

Hi Vivek,

Quoted 6 lines

I found the IO Controller doesn't work in as.
I dig into this issue, and notice that you stop idling in as. IMHO, this will cause active ioq is always expired when tring to choose a new ioq to serve(elv_fq_select_ioq). Because idling is disabled, active ioq can't be kept anymore.
So i just get rid of ELV_IOSCHED_DONT_IDLE, and it works fine this time.

Hi Gui,

Thanks for the testing. I have not enabled idling for AS in common layer becuase AS has its own idling/anticipation logic. I think we should not have anticipation going on at two places, common layer as well as individual io scheduler. That's the reason I have implemented a function elv_iosched_expire_ioq(), which calls io scheduler to find out if an ioq can be expired now.

So in elv_fq_select_ioq(), we call elv_iosched_expire_ioq() and if ioschduler denies expiration, then we don't expire the queue. In this case AS can deny the expiration if it is anticipating for next request.

Actually AS is very different and its little tricky to make it work with common layer, especially in terms of anticipation. This is on my TODO list but before I fix AS, I wanted to get other things right with common layer, cfq, noop and deadline.

Thanks
Vivek

Quoted 21 lines
---
 block/as-iosched.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
block/as-iosched.c +1 -1
index 27c14a7..499c521 100644
@@ -1689,7 +1689,7 @@ static struct elevator_type iosched_as = {
 		.elevator_expire_ioq_fn =       as_expire_ioq,
 		.elevator_active_ioq_set_fn =   as_active_ioq_set,
 	},
-	.elevator_features = ELV_IOSCHED_NEED_FQ | ELV_IOSCHED_SINGLE_IOQ | ELV_IOSCHED_DONT_IDLE,
+	.elevator_features = ELV_IOSCHED_NEED_FQ | ELV_IOSCHED_SINGLE_IOQ,
 #else
 	},
 #endif
Signature · 1 line
1.5.4.rc3
#152 lore raw

Re: [PATCH] IO Controller: No need to stop idling in as

In reply to Gui Jianfeng
Quoted 13 lines · Gui Jianfeng

Review

Quoted 8 lines · Vivek Goyal
(quoted diff) +4 -1
 		.elevator_alloc_sched_queue_fn = as_alloc_as_queue,
 		.elevator_free_sched_queue_fn = as_free_as_queue,
+#ifdef CONFIG_IOSCHED_AS_HIER
+		.elevator_expire_ioq_fn =       as_expire_ioq,
+		.elevator_active_ioq_set_fn =   as_active_ioq_set,
 	},
-
+	.elevator_features = ELV_IOSCHED_NEED_FQ | ELV_IOSCHED_SINGLE_IOQ | ELV_IOSCHED_DONT_IDLE,

Hi Vivek,

Quoted 6 lines

I found the IO Controller doesn't work in as.
I dig into this issue, and notice that you stop idling in as. IMHO, this will cause active ioq is always expired when tring to choose a new ioq to serve(elv_fq_select_ioq). Because idling is disabled, active ioq can't be kept anymore.
So i just get rid of ELV_IOSCHED_DONT_IDLE, and it works fine this time.

Hi Gui,

Thanks for the testing. I have not enabled idling for AS in common layer becuase AS has its own idling/anticipation logic. I think we should not have anticipation going on at two places, common layer as well as individual io scheduler. That's the reason I have implemented a function elv_iosched_expire_ioq(), which calls io scheduler to find out if an ioq can be expired now.

So in elv_fq_select_ioq(), we call elv_iosched_expire_ioq() and if ioschduler denies expiration, then we don't expire the queue. In this case AS can deny the expiration if it is anticipating for next request.

Actually AS is very different and its little tricky to make it work with common layer, especially in terms of anticipation. This is on my TODO list but before I fix AS, I wanted to get other things right with common layer, cfq, noop and deadline.

Thanks
Vivek

Quoted 21 lines
---
 block/as-iosched.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
block/as-iosched.c +1 -1
index 27c14a7..499c521 100644
@@ -1689,7 +1689,7 @@ static struct elevator_type iosched_as = {
 		.elevator_expire_ioq_fn =       as_expire_ioq,
 		.elevator_active_ioq_set_fn =   as_active_ioq_set,
 	},
-	.elevator_features = ELV_IOSCHED_NEED_FQ | ELV_IOSCHED_SINGLE_IOQ | ELV_IOSCHED_DONT_IDLE,
+	.elevator_features = ELV_IOSCHED_NEED_FQ | ELV_IOSCHED_SINGLE_IOQ,
 #else
 	},
 #endif
Signature · 1 line
1.5.4.rc3

A message that is not in the archive (sent off-list or not yet received): [email protected]

#154 lore raw

Re: [PATCH] IO Controller: No need to stop idling in as

Quoted 20 lines · Vivek Goyal
Quoted 12 lines · Gui Jianfeng

Review

Quoted 8 lines · Vivek Goyal
(quoted diff) +4 -1
 		.elevator_alloc_sched_queue_fn = as_alloc_as_queue,
 		.elevator_free_sched_queue_fn = as_free_as_queue,
+#ifdef CONFIG_IOSCHED_AS_HIER
+		.elevator_expire_ioq_fn =       as_expire_ioq,
+		.elevator_active_ioq_set_fn =   as_active_ioq_set,
 	},
-
+	.elevator_features = ELV_IOSCHED_NEED_FQ | ELV_IOSCHED_SINGLE_IOQ | ELV_IOSCHED_DONT_IDLE,

Hi Vivek,

Quoted 6 lines

I found the IO Controller doesn't work in as.
I dig into this issue, and notice that you stop idling in as. IMHO, this will cause active ioq is always expired when tring to choose a new ioq to serve(elv_fq_select_ioq). Because idling is disabled, active ioq can't be kept anymore.
So i just get rid of ELV_IOSCHED_DONT_IDLE, and it works fine this time.

Quoted 8 lines

Hi Gui,

Thanks for the testing. I have not enabled idling for AS in common layer becuase AS has its own idling/anticipation logic. I think we should not have anticipation going on at two places, common layer as well as individual io scheduler. That's the reason I have implemented a function elv_iosched_expire_ioq(), which calls io scheduler to find out if an ioq can be expired now.

Hi Vivek,

If an user chooses fairness rather than throughput just like what your fairness patch is tring to do. Do we need to enable the common idling logic for as in this scenario.

Quoted 38 lines

So in elv_fq_select_ioq(), we call elv_iosched_expire_ioq() and if ioschduler denies expiration, then we don't expire the queue. In this case AS can deny the expiration if it is anticipating for next request.

Actually AS is very different and its little tricky to make it work with common layer, especially in terms of anticipation. This is on my TODO list but before I fix AS, I wanted to get other things right with common layer, cfq, noop and deadline.

Thanks
Vivek

Quoted 21 lines
---
 block/as-iosched.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
block/as-iosched.c +1 -1
index 27c14a7..499c521 100644
@@ -1689,7 +1689,7 @@ static struct elevator_type iosched_as = {
 		.elevator_expire_ioq_fn =       as_expire_ioq,
 		.elevator_active_ioq_set_fn =   as_active_ioq_set,
 	},
-	.elevator_features = ELV_IOSCHED_NEED_FQ | ELV_IOSCHED_SINGLE_IOQ | ELV_IOSCHED_DONT_IDLE,
+	.elevator_features = ELV_IOSCHED_NEED_FQ | ELV_IOSCHED_SINGLE_IOQ,
 #else
 	},
 #endif
Signature · 1 line
1.5.4.rc3
Signature · 2 lines
Regards
Gui Jianfeng
#155 lore raw

Re: [PATCH] IO Controller: No need to stop idling in as

In reply to Vivek Goyal
Quoted 20 lines · Vivek Goyal
Quoted 12 lines · Gui Jianfeng

Review

Quoted 8 lines · Vivek Goyal
(quoted diff) +4 -1
 		.elevator_alloc_sched_queue_fn = as_alloc_as_queue,
 		.elevator_free_sched_queue_fn = as_free_as_queue,
+#ifdef CONFIG_IOSCHED_AS_HIER
+		.elevator_expire_ioq_fn =       as_expire_ioq,
+		.elevator_active_ioq_set_fn =   as_active_ioq_set,
 	},
-
+	.elevator_features = ELV_IOSCHED_NEED_FQ | ELV_IOSCHED_SINGLE_IOQ | ELV_IOSCHED_DONT_IDLE,

Hi Vivek,

Quoted 6 lines

I found the IO Controller doesn't work in as.
I dig into this issue, and notice that you stop idling in as. IMHO, this will cause active ioq is always expired when tring to choose a new ioq to serve(elv_fq_select_ioq). Because idling is disabled, active ioq can't be kept anymore.
So i just get rid of ELV_IOSCHED_DONT_IDLE, and it works fine this time.

Quoted 8 lines

Hi Gui,

Thanks for the testing. I have not enabled idling for AS in common layer becuase AS has its own idling/anticipation logic. I think we should not have anticipation going on at two places, common layer as well as individual io scheduler. That's the reason I have implemented a function elv_iosched_expire_ioq(), which calls io scheduler to find out if an ioq can be expired now.

Hi Vivek,

If an user chooses fairness rather than throughput just like what your fairness patch is tring to do. Do we need to enable the common idling logic for as in this scenario.

Quoted 38 lines

So in elv_fq_select_ioq(), we call elv_iosched_expire_ioq() and if ioschduler denies expiration, then we don't expire the queue. In this case AS can deny the expiration if it is anticipating for next request.

Actually AS is very different and its little tricky to make it work with common layer, especially in terms of anticipation. This is on my TODO list but before I fix AS, I wanted to get other things right with common layer, cfq, noop and deadline.

Thanks
Vivek

Quoted 21 lines
---
 block/as-iosched.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
block/as-iosched.c +1 -1
index 27c14a7..499c521 100644
@@ -1689,7 +1689,7 @@ static struct elevator_type iosched_as = {
 		.elevator_expire_ioq_fn =       as_expire_ioq,
 		.elevator_active_ioq_set_fn =   as_active_ioq_set,
 	},
-	.elevator_features = ELV_IOSCHED_NEED_FQ | ELV_IOSCHED_SINGLE_IOQ | ELV_IOSCHED_DONT_IDLE,
+	.elevator_features = ELV_IOSCHED_NEED_FQ | ELV_IOSCHED_SINGLE_IOQ,
 #else
 	},
 #endif
Signature · 1 line
1.5.4.rc3
Signature · 2 lines
Regards
Gui Jianfeng
#156 lore raw
In reply to Vivek Goyal

Hi Vivek,

Could you tell me to which kernel I can apply your patches?

  # latest mm?
I would like to test your controller.

Thank you,
Takuya Yoshikawa

Quoted 49 lines · Vivek Goyal

Hi All,

Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.

http://lkml.org/lkml/2008/11/6/227

One of the takeaway from the discussion in this thread was that let us implement a common layer which contains the proportional weight scheduling code which can be shared by all the IO schedulers.

Implementing IO controller will not cover the devices which don't use IO schedulers but it should cover the common case.

There were more discussions regarding 2 level vs 1 level IO control at following link.

https://lists.linux-foundation.org/pipermail/containers/2009-January/015402.html

So in the mean time we took the discussion off the list and spent time on making the 1 level control apporoach work where majority of the proportional weight control is shared by the four schedulers instead of each one having to replicate the code. We make use of BFQ code for fair queuing as posted by Paolo and Fabio here.

http://lkml.org/lkml/2008/11/11/148

Details about design and howto have been put in documentation patch.

I have done very basic testing of running 2 or 3 "dd" threads in different cgroups. Wanted to get the patchset out for feedback/review before we dive into more bug fixing, benchmarking, optimizations etc.

Your feedback/comments are welcome.

Patch series contains 10 patches. It should be compilable and bootable after every patch. Intial 2 patches implement flat fair queuing (no cgroup support) and make cfq to use that. Later patches introduce hierarchical fair queuing support in elevator layer and modify other IO schdulers to use that.

Thanks
Vivek
_______________________________________________
Containers mailing list
[email protected]
https://lists.linux-foundation.org/mailman/listinfo/containers

A message that is not in the archive (sent off-list or not yet received): [email protected]

#158 lore raw

Hi Vivek,

It would be very appreciated if the patches can be based on 2.6.28.

Thanks a lot.

Anqin

Quoted 66 lines · Takuya Yoshikawa

Hi Vivek,

Could you tell me to which kernel I can apply your patches?

  # latest mm?
I would like to test your controller.

Thank you,
Takuya Yoshikawa

Quoted 42 lines · Vivek Goyal

Hi All,

Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.

http://lkml.org/lkml/2008/11/6/227

One of the takeaway from the discussion in this thread was that let us implement a common layer which contains the proportional weight scheduling code which can be shared by all the IO schedulers.

Implementing IO controller will not cover the devices which don't use IO schedulers but it should cover the common case.

There were more discussions regarding 2 level vs 1 level IO control at following link.

https://lists.linux-foundation.org/pipermail/containers/2009-January/015402.html

So in the mean time we took the discussion off the list and spent time on making the 1 level control apporoach work where majority of the proportional weight control is shared by the four schedulers instead of each one having to replicate the code. We make use of BFQ code for fair queuing as posted by Paolo and Fabio here.

http://lkml.org/lkml/2008/11/11/148

Details about design and howto have been put in documentation patch.

I have done very basic testing of running 2 or 3 "dd" threads in different cgroups. Wanted to get the patchset out for feedback/review before we dive into more bug fixing, benchmarking, optimizations etc.

Your feedback/comments are welcome.

Patch series contains 10 patches. It should be compilable and bootable after every patch. Intial 2 patches implement flat fair queuing (no cgroup support) and make cfq to use that. Later patches introduce hierarchical fair queuing support in elevator layer and modify other IO schdulers to use that.

Quoted 7 lines

Thanks
Vivek
_______________________________________________
Containers mailing list Containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
https://lists.linux-foundation.org/mailman/listinfo/containers

_______________________________________________
Containers mailing list Containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
https://lists.linux-foundation.org/mailman/listinfo/containers

#159 lore raw
In reply to Takuya Yoshikawa

Hi Vivek,

It would be very appreciated if the patches can be based on 2.6.28.

Thanks a lot.

Anqin

Quoted 66 lines · Takuya Yoshikawa

Hi Vivek,

Could you tell me to which kernel I can apply your patches?

  # latest mm?
I would like to test your controller.

Thank you,
Takuya Yoshikawa

Quoted 42 lines · Vivek Goyal

Hi All,

Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.

http://lkml.org/lkml/2008/11/6/227

One of the takeaway from the discussion in this thread was that let us implement a common layer which contains the proportional weight scheduling code which can be shared by all the IO schedulers.

Implementing IO controller will not cover the devices which don't use IO schedulers but it should cover the common case.

There were more discussions regarding 2 level vs 1 level IO control at following link.

https://lists.linux-foundation.org/pipermail/containers/2009-January/015402.html

So in the mean time we took the discussion off the list and spent time on making the 1 level control apporoach work where majority of the proportional weight control is shared by the four schedulers instead of each one having to replicate the code. We make use of BFQ code for fair queuing as posted by Paolo and Fabio here.

http://lkml.org/lkml/2008/11/11/148

Details about design and howto have been put in documentation patch.

I have done very basic testing of running 2 or 3 "dd" threads in different cgroups. Wanted to get the patchset out for feedback/review before we dive into more bug fixing, benchmarking, optimizations etc.

Your feedback/comments are welcome.

Patch series contains 10 patches. It should be compilable and bootable after every patch. Intial 2 patches implement flat fair queuing (no cgroup support) and make cfq to use that. Later patches introduce hierarchical fair queuing support in elevator layer and modify other IO schdulers to use that.

Quoted 7 lines

Thanks
Vivek
_______________________________________________
Containers mailing list
[email protected]
https://lists.linux-foundation.org/mailman/listinfo/containers

_______________________________________________
Containers mailing list
[email protected]
https://lists.linux-foundation.org/mailman/listinfo/containers

A message that is not in the archive (sent off-list or not yet received): d95d44a20903112340s3c77807dt465e68901747ad89-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org

#161 lore raw

(Please don't top-post...)

Quoted 4 lines · anqin

Hi Vivek,

It would be very appreciated if the patches can be based on 2.6.28.

Why? When this is ready to be merged, then it should be based on Jens' block-tree, or akpm's mm tree. And this version currently is based on 2.6.29-rc4, so if you want to try it out, just prepare a 2.6.29-rc4 kernel tree.

Quoted 14 lines

Thanks a lot.

Anqin

Quoted 8 lines · Takuya Yoshikawa

Hi Vivek,

Could you tell me to which kernel I can apply your patches?

  # latest mm?
I would like to test your controller.

Thank you,
Takuya Yoshikawa

#162 lore raw
In reply to anqin

(Please don't top-post...)

Quoted 4 lines · anqin

Hi Vivek,

It would be very appreciated if the patches can be based on 2.6.28.

Why? When this is ready to be merged, then it should be based on Jens' block-tree, or akpm's mm tree. And this version currently is based on 2.6.29-rc4, so if you want to try it out, just prepare a 2.6.29-rc4 kernel tree.

Quoted 14 lines

Thanks a lot.

Anqin

Quoted 8 lines · Takuya Yoshikawa

Hi Vivek,

Could you tell me to which kernel I can apply your patches?

  # latest mm?
I would like to test your controller.

Thank you,
Takuya Yoshikawa

A message that is not in the archive (sent off-list or not yet received): [email protected]

#164 lore raw
Quoted 4 lines

Why? When this is ready to be merged, then it should be based on Jens' block-tree, or akpm's mm tree. And this version currently is based on 2.6.29-rc4, so if you want to try it out, just prepare a 2.6.29-rc4 kernel tree.

I have checked the LKML and see these patches (on web pages) are based on 2.6.27. It seemed too old.

You mean that the codes have new patch files in 2.6.29-rc4?

#165 lore raw
In reply to Li Zefan
Quoted 4 lines

Why? When this is ready to be merged, then it should be based on Jens' block-tree, or akpm's mm tree. And this version currently is based on 2.6.29-rc4, so if you want to try it out, just prepare a 2.6.29-rc4 kernel tree.

I have checked the LKML and see these patches (on web pages) are based on 2.6.27. It seemed too old.

You mean that the codes have new patch files in 2.6.29-rc4?

#166 lore raw
Quoted 9 lines · anqin
Quoted 4 lines

Why? When this is ready to be merged, then it should be based on Jens' block-tree, or akpm's mm tree. And this version currently is based on 2.6.29-rc4, so if you want to try it out, just prepare a 2.6.29-rc4 kernel tree.

I have checked the LKML and see these patches (on web pages) are based on 2.6.27. It seemed too old.

You mean that the codes have new patch files in 2.6.29-rc4?

I guess my mistake that I did not specify the kernel version in my mail.

These patches should apply on top of 2.6.29-rc7.

Thanks
Vivek

#167 lore raw
In reply to anqin
Quoted 4 lines · anqin

Hi Vivek,

It would be very appreciated if the patches can be based on 2.6.28.

Hi Anquin,

I think most of the people want to test new patches on latest kernels so I will keep it that way. You can backport it to previous kernels if you really need to. For me it will become very difficult to maintain two versions.

Is there any reason why you can't move to latest kernels?

Thanks
Vivek

Quoted 72 lines

Thanks a lot.

Anqin

Quoted 66 lines · Takuya Yoshikawa

Hi Vivek,

Could you tell me to which kernel I can apply your patches?

  # latest mm?
I would like to test your controller.

Thank you,
Takuya Yoshikawa

Quoted 42 lines · Vivek Goyal

Hi All,

Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.

http://lkml.org/lkml/2008/11/6/227

One of the takeaway from the discussion in this thread was that let us implement a common layer which contains the proportional weight scheduling code which can be shared by all the IO schedulers.

Implementing IO controller will not cover the devices which don't use IO schedulers but it should cover the common case.

There were more discussions regarding 2 level vs 1 level IO control at following link.

https://lists.linux-foundation.org/pipermail/containers/2009-January/015402.html

So in the mean time we took the discussion off the list and spent time on making the 1 level control apporoach work where majority of the proportional weight control is shared by the four schedulers instead of each one having to replicate the code. We make use of BFQ code for fair queuing as posted by Paolo and Fabio here.

http://lkml.org/lkml/2008/11/11/148

Details about design and howto have been put in documentation patch.

I have done very basic testing of running 2 or 3 "dd" threads in different cgroups. Wanted to get the patchset out for feedback/review before we dive into more bug fixing, benchmarking, optimizations etc.

Your feedback/comments are welcome.

Patch series contains 10 patches. It should be compilable and bootable after every patch. Intial 2 patches implement flat fair queuing (no cgroup support) and make cfq to use that. Later patches introduce hierarchical fair queuing support in elevator layer and modify other IO schdulers to use that.

Quoted 7 lines

Thanks
Vivek
_______________________________________________
Containers mailing list Containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
https://lists.linux-foundation.org/mailman/listinfo/containers

_______________________________________________
Containers mailing list Containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
https://lists.linux-foundation.org/mailman/listinfo/containers

#168 lore raw
In reply to Takuya Yoshikawa
Quoted 6 lines · Takuya Yoshikawa

Hi Vivek,

Could you tell me to which kernel I can apply your patches?

  # latest mm?
I would like to test your controller.

Hi Takuya,

These apply on linus git tree (2.6.29-rc7).

Thanks
Vivek

Quoted 54 lines

Thank you,
Takuya Yoshikawa

Quoted 49 lines · Vivek Goyal

Hi All,

Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.

http://lkml.org/lkml/2008/11/6/227

One of the takeaway from the discussion in this thread was that let us implement a common layer which contains the proportional weight scheduling code which can be shared by all the IO schedulers.

Implementing IO controller will not cover the devices which don't use IO schedulers but it should cover the common case.

There were more discussions regarding 2 level vs 1 level IO control at following link.

https://lists.linux-foundation.org/pipermail/containers/2009-January/015402.html

So in the mean time we took the discussion off the list and spent time on making the 1 level control apporoach work where majority of the proportional weight control is shared by the four schedulers instead of each one having to replicate the code. We make use of BFQ code for fair queuing as posted by Paolo and Fabio here.

http://lkml.org/lkml/2008/11/11/148

Details about design and howto have been put in documentation patch.

I have done very basic testing of running 2 or 3 "dd" threads in different cgroups. Wanted to get the patchset out for feedback/review before we dive into more bug fixing, benchmarking, optimizations etc.

Your feedback/comments are welcome.

Patch series contains 10 patches. It should be compilable and bootable after every patch. Intial 2 patches implement flat fair queuing (no cgroup support) and make cfq to use that. Later patches introduce hierarchical fair queuing support in elevator layer and modify other IO schdulers to use that.

Thanks
Vivek
_______________________________________________
Containers mailing list Containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
https://lists.linux-foundation.org/mailman/listinfo/containers

#169 lore raw
In reply to Vivek Goyal
Quoted 11 lines · Vivek Goyal

Hi All,

Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.

http://lkml.org/lkml/2008/11/6/227

One of the takeaway from the discussion in this thread was that let us implement a common layer which contains the proportional weight scheduling code which can be shared by all the IO schedulers.

Hi Vivek,

I did some tests on my *old* i386 box(with two concurrent dd running), and notice that IO Controller doesn't work fine in such situation. But it can work perfectly in my *new* x86 box. I dig into this problem, and i guess the major reason is that my *old* i386 box is too slow, it can't ensure two running ioqs are always backlogged. If that is the case, I happens to have a thought. when an ioq uses up it time slice, we don't expire it immediately. May be we can give a piece of bonus time for idling to wait new requests if this ioq's finish time and its ancestor's finish time are all much smaller than other entities in each corresponding service tree.

Signature · 2 lines
Regards
Gui Jianfeng
#170 lore raw
Quoted 13 lines · Gui Jianfeng
Quoted 11 lines · Vivek Goyal

Hi All,

Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.

http://lkml.org/lkml/2008/11/6/227

One of the takeaway from the discussion in this thread was that let us implement a common layer which contains the proportional weight scheduling code which can be shared by all the IO schedulers.

Quoted 6 lines

Hi Vivek,

I did some tests on my *old* i386 box(with two concurrent dd running), and notice that IO Controller doesn't work fine in such situation. But it can work perfectly in my *new* x86 box. I dig into this problem, and i guess the major reason is that my *old* i386 box is too slow, it can't ensure two running ioqs are always backlogged.

Hi Gui,

Have you run top to see what's the percentage cpu usage. I suspect that cpu is not keeping up pace disk to enqueue enough requests. I think process might be blocked somewhere else so that it could not issue requests.

Quoted 4 lines

If that is the case, I happens to have a thought. when an ioq uses up it time slice, we don't expire it immediately. May be we can give a piece of bonus time for idling to wait new requests if this ioq's finish time and its ancestor's finish time are all much smaller than other entities in each corresponding service tree.

Have you tried it with "fairness" enabled? With "fairness" enabled, for sync queues I am waiting for one extra idle time slice "8ms" for queue to get backlogged again before I move to the next queue?

Otherwise try to increase the idle time length to higher value say "12ms" just to see if that has any impact.

Can you please also send me output of blkparse. It might give some idea how IO schedulers see IO pattern.

Thanks
Vivek

A message that is not in the archive (sent off-list or not yet received): [email protected]

#172 lore raw
Quoted 37 lines · Vivek Goyal
Quoted 13 lines · Gui Jianfeng
Quoted 11 lines · Vivek Goyal

Hi All,

Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.

http://lkml.org/lkml/2008/11/6/227

One of the takeaway from the discussion in this thread was that let us implement a common layer which contains the proportional weight scheduling code which can be shared by all the IO schedulers.

Quoted 6 lines

Hi Vivek,

I did some tests on my *old* i386 box(with two concurrent dd running), and notice that IO Controller doesn't work fine in such situation. But it can work perfectly in my *new* x86 box. I dig into this problem, and i guess the major reason is that my *old* i386 box is too slow, it can't ensure two running ioqs are always backlogged.

Hi Gui,

Have you run top to see what's the percentage cpu usage. I suspect that cpu is not keeping up pace disk to enqueue enough requests. I think process might be blocked somewhere else so that it could not issue requests.

Quoted 4 lines

If that is the case, I happens to have a thought. when an ioq uses up it time slice, we don't expire it immediately. May be we can give a piece of bonus time for idling to wait new requests if this ioq's finish time and its ancestor's finish time are all much smaller than other entities in each corresponding service tree.

Have you tried it with "fairness" enabled? With "fairness" enabled, for sync queues I am waiting for one extra idle time slice "8ms" for queue to get backlogged again before I move to the next queue?

Quoted 5 lines

Otherwise try to increase the idle time length to higher value say "12ms" just to see if that has any impact.

Can you please also send me output of blkparse. It might give some idea how IO schedulers see IO pattern.

Hi Vivek,

Sorry for the late reply, I tried the "fairness" patch, it seems not working. I'v also tried to extend the idle value, not working either.
The blktrace output is attached. It seems that the high priority ioq is deleting from busy tree too often due to lacking of requests. My box is single CPU and CPU speed is a little slow. May be two concurrent dd is contending CPU to submit requests, that's the reason for not always backlogged for ioqs.

Quoted 6 lines

Thanks
Vivek

Signature · 2 lines
Regards
Gui Jianfeng

_______________________________________________
Containers mailing list Containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
https://lists.linux-foundation.org/mailman/listinfo/containers

#173 lore raw
In reply to Vivek Goyal
Quoted 37 lines · Vivek Goyal
Quoted 13 lines · Gui Jianfeng
Quoted 11 lines · Vivek Goyal

Hi All,

Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.

http://lkml.org/lkml/2008/11/6/227

One of the takeaway from the discussion in this thread was that let us implement a common layer which contains the proportional weight scheduling code which can be shared by all the IO schedulers.

Quoted 6 lines

Hi Vivek,

I did some tests on my *old* i386 box(with two concurrent dd running), and notice that IO Controller doesn't work fine in such situation. But it can work perfectly in my *new* x86 box. I dig into this problem, and i guess the major reason is that my *old* i386 box is too slow, it can't ensure two running ioqs are always backlogged.

Hi Gui,

Have you run top to see what's the percentage cpu usage. I suspect that cpu is not keeping up pace disk to enqueue enough requests. I think process might be blocked somewhere else so that it could not issue requests.

Quoted 4 lines

If that is the case, I happens to have a thought. when an ioq uses up it time slice, we don't expire it immediately. May be we can give a piece of bonus time for idling to wait new requests if this ioq's finish time and its ancestor's finish time are all much smaller than other entities in each corresponding service tree.

Have you tried it with "fairness" enabled? With "fairness" enabled, for sync queues I am waiting for one extra idle time slice "8ms" for queue to get backlogged again before I move to the next queue?

Quoted 5 lines

Otherwise try to increase the idle time length to higher value say "12ms" just to see if that has any impact.

Can you please also send me output of blkparse. It might give some idea how IO schedulers see IO pattern.

Hi Vivek,

Sorry for the late reply, I tried the "fairness" patch, it seems not working. I'v also tried to extend the idle value, not working either.
The blktrace output is attached. It seems that the high priority ioq is deleting from busy tree too often due to lacking of requests. My box is single CPU and CPU speed is a little slow. May be two concurrent dd is contending CPU to submit requests, that's the reason for not always backlogged for ioqs.

Quoted 6 lines

Thanks
Vivek

Signature · 2 lines
Regards
Gui Jianfeng
#174 lore raw
Quoted 42 lines · Gui Jianfeng
Quoted 35 lines · Vivek Goyal
Quoted 13 lines · Gui Jianfeng
Quoted 11 lines · Vivek Goyal

Hi All,

Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.

http://lkml.org/lkml/2008/11/6/227

One of the takeaway from the discussion in this thread was that let us implement a common layer which contains the proportional weight scheduling code which can be shared by all the IO schedulers.

Quoted 6 lines

Hi Vivek,

I did some tests on my *old* i386 box(with two concurrent dd running), and notice that IO Controller doesn't work fine in such situation. But it can work perfectly in my *new* x86 box. I dig into this problem, and i guess the major reason is that my *old* i386 box is too slow, it can't ensure two running ioqs are always backlogged.

Hi Gui,

Have you run top to see what's the percentage cpu usage. I suspect that cpu is not keeping up pace disk to enqueue enough requests. I think process might be blocked somewhere else so that it could not issue requests.

Quoted 4 lines

If that is the case, I happens to have a thought. when an ioq uses up it time slice, we don't expire it immediately. May be we can give a piece of bonus time for idling to wait new requests if this ioq's finish time and its ancestor's finish time are all much smaller than other entities in each corresponding service tree.

Have you tried it with "fairness" enabled? With "fairness" enabled, for sync queues I am waiting for one extra idle time slice "8ms" for queue to get backlogged again before I move to the next queue?

Quoted 5 lines

Otherwise try to increase the idle time length to higher value say "12ms" just to see if that has any impact.

Can you please also send me output of blkparse. It might give some idea how IO schedulers see IO pattern.

Quoted 8 lines

Hi Vivek,

Sorry for the late reply, I tried the "fairness" patch, it seems not working. I'v also tried to extend the idle value, not working either.
The blktrace output is attached. It seems that the high priority ioq is deleting from busy tree too often due to lacking of requests. My box is single CPU and CPU speed is a little slow. May be two concurrent dd is contending CPU to submit requests, that's the reason for not always backlogged for ioqs.

Hi Vivek,

Sorry for bothering, there were some configure errors when i tested, and got the improper result.
The "fairness" patch seems to work fine now! It makes the high priority ioq *always* backlogged :)

Quoted 7 lines
Quoted 6 lines

Thanks
Vivek

Signature · 2 lines
Regards
Gui Jianfeng
#175 lore raw
In reply to Vivek Goyal
Quoted 4 lines · Vivek Goyal

Hi All,

Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.

Hi Vivek,

I got the following OOPS when testing, can't reproduce again :(

kernel BUG at block/elevator-fq.c:1396!
invalid opcode: 0000 [#1] SMP
last sysfs file: /sys/block/hdb/queue/scheduler

Modules linked in: ipv6 cpufreq_ondemand acpi_cpufreq dm_mirror dm_multipath sbd

Pid: 5032, comm: rmdir Not tainted (2.6.29-rc7-vivek #17) Veriton M460
EIP: 0060:[<c04ec7de>] EFLAGS: 00010082 CPU: 0
EIP is at iocg_destroy+0xdc/0x14e
EAX: 00000000 EBX: f62278b4 ECX: f6207800 EDX: f6227904
ESI: f6227800 EDI: f62278a0 EBP: 00000003 ESP: c8790f00
DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
Process rmdir (pid: 5032, ti=c8790000 task=f6636960 task.ti=c8790000)

Stack:
 f53cc5c0 f62b7258 f10991c8 00000282 f6227800 c0733fa0 ec1c5140 edfc6d34
 c8790000 c04463ce f6a4c84c edfc6d34 0804c840 c048883c f6a4c84c f6a4c84c
 c04888d6 f6a4c84c 00000000 c04897de f6a4c84c c048504c f115f4c0 ebc37954
Call Trace:
 [<c04463ce>] cgroup_diput+0x41/0x8c
 [<c048883c>] dentry_iput+0x45/0x5e
 [<c04888d6>] d_kill+0x19/0x32
 [<c04897de>] dput+0xd8/0xdf
 [<c048504c>] do_rmdir+0x8f/0xb6
 [<c06330fc>] do_page_fault+0x2a2/0x579
 [<c0402fc1>] sysenter_do_call+0x12/0x21
 [<c0630000>] schedule+0x641/0x830
Code: 08 00 74 04 0f 0b eb fe 83 7f 04 00 74 04 0f 0b eb fe 45 83 c3 1c 83 fd 0
EIP: [<c04ec7de>] iocg_destroy+0xdc/0x14e SS:ESP 0068:c8790f00
Signature · 2 lines
Regards
Gui Jianfeng

A message that is not in the archive (sent off-list or not yet received): [email protected]

#177 lore raw
Quoted 21 lines · Gui Jianfeng
Quoted 4 lines · Vivek Goyal

Hi All,

Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.

 Hi Vivek,

 I got the following OOPS when testing, can't reproduce again :(

kernel BUG at block/elevator-fq.c:1396!
invalid opcode: 0000 [#1] SMP
last sysfs file: /sys/block/hdb/queue/scheduler

Modules linked in: ipv6 cpufreq_ondemand acpi_cpufreq dm_mirror dm_multipath sbd

Pid: 5032, comm: rmdir Not tainted (2.6.29-rc7-vivek #17) Veriton M460
EIP: 0060:[<c04ec7de>] EFLAGS: 00010082 CPU: 0
EIP is at iocg_destroy+0xdc/0x14e
EAX: 00000000 EBX: f62278b4 ECX: f6207800 EDX: f6227904
ESI: f6227800 EDI: f62278a0 EBP: 00000003 ESP: c8790f00
 DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
Process rmdir (pid: 5032, ti=c8790000 task=f6636960 task.ti=c8790000)

Quoted 15 lines
Stack:
 f53cc5c0 f62b7258 f10991c8 00000282 f6227800 c0733fa0 ec1c5140 edfc6d34
 c8790000 c04463ce f6a4c84c edfc6d34 0804c840 c048883c f6a4c84c f6a4c84c
 c04888d6 f6a4c84c 00000000 c04897de f6a4c84c c048504c f115f4c0 ebc37954
Call Trace:
 [<c04463ce>] cgroup_diput+0x41/0x8c
 [<c048883c>] dentry_iput+0x45/0x5e
 [<c04888d6>] d_kill+0x19/0x32
 [<c04897de>] dput+0xd8/0xdf
 [<c048504c>] do_rmdir+0x8f/0xb6
 [<c06330fc>] do_page_fault+0x2a2/0x579
 [<c0402fc1>] sysenter_do_call+0x12/0x21
 [<c0630000>] schedule+0x641/0x830
Code: 08 00 74 04 0f 0b eb fe 83 7f 04 00 74 04 0f 0b eb fe 45 83 c3 1c 83 fd 0
EIP: [<c04ec7de>] iocg_destroy+0xdc/0x14e SS:ESP 0068:c8790f00

We have seen this too. And have been able to reproduce it. I did not get a chance to fix it so far, but my understanding is that one of the async queues was active when the cgroup was getting destroyed. We moved it to root cgroup, but did not deactivate it; so active_entity still points to the entity of the async queue which has now been moved to the root cgroup. I will send an update if I can verify this, or fix it.

Quoted 6 lines

--
Regards
Gui Jianfeng

#178 lore raw
In reply to a message not in the archive
Quoted 10 lines · Gui Jianfeng
Quoted 4 lines · Vivek Goyal

Hi All,

Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.

Hi Vivek,

I got the following OOPS when testing, can't reproduce again :(

Hi Gui,

Thanks for the report. Will look into it and see if I can reproduce it.

Thanks
Vivek

Quoted 30 lines

kernel BUG at block/elevator-fq.c:1396!
invalid opcode: 0000 [#1] SMP
last sysfs file: /sys/block/hdb/queue/scheduler

Modules linked in: ipv6 cpufreq_ondemand acpi_cpufreq dm_mirror dm_multipath sbd

Pid: 5032, comm: rmdir Not tainted (2.6.29-rc7-vivek #17) Veriton M460
EIP: 0060:[<c04ec7de>] EFLAGS: 00010082 CPU: 0
EIP is at iocg_destroy+0xdc/0x14e
EAX: 00000000 EBX: f62278b4 ECX: f6207800 EDX: f6227904
ESI: f6227800 EDI: f62278a0 EBP: 00000003 ESP: c8790f00
DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
Process rmdir (pid: 5032, ti=c8790000 task=f6636960 task.ti=c8790000)

Stack:
 f53cc5c0 f62b7258 f10991c8 00000282 f6227800 c0733fa0 ec1c5140 edfc6d34
 c8790000 c04463ce f6a4c84c edfc6d34 0804c840 c048883c f6a4c84c f6a4c84c
 c04888d6 f6a4c84c 00000000 c04897de f6a4c84c c048504c f115f4c0 ebc37954
Call Trace:
 [<c04463ce>] cgroup_diput+0x41/0x8c
 [<c048883c>] dentry_iput+0x45/0x5e
 [<c04888d6>] d_kill+0x19/0x32
 [<c04897de>] dput+0xd8/0xdf
 [<c048504c>] do_rmdir+0x8f/0xb6
 [<c06330fc>] do_page_fault+0x2a2/0x579
 [<c0402fc1>] sysenter_do_call+0x12/0x21
 [<c0630000>] schedule+0x641/0x830
Code: 08 00 74 04 0f 0b eb fe 83 7f 04 00 74 04 0f 0b eb fe 45 83 c3 1c 83 fd 0
EIP: [<c04ec7de>] iocg_destroy+0xdc/0x14e SS:ESP 0068:c8790f00
Signature · 2 lines
Regards
Gui Jianfeng
#179 lore raw
In reply to Gui Jianfeng
Quoted 21 lines · Gui Jianfeng
Quoted 4 lines · Vivek Goyal

Hi All,

Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.

 Hi Vivek,

 I got the following OOPS when testing, can't reproduce again :(

kernel BUG at block/elevator-fq.c:1396!
invalid opcode: 0000 [#1] SMP
last sysfs file: /sys/block/hdb/queue/scheduler

Modules linked in: ipv6 cpufreq_ondemand acpi_cpufreq dm_mirror dm_multipath sbd

Pid: 5032, comm: rmdir Not tainted (2.6.29-rc7-vivek #17) Veriton M460
EIP: 0060:[<c04ec7de>] EFLAGS: 00010082 CPU: 0
EIP is at iocg_destroy+0xdc/0x14e
EAX: 00000000 EBX: f62278b4 ECX: f6207800 EDX: f6227904
ESI: f6227800 EDI: f62278a0 EBP: 00000003 ESP: c8790f00
 DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
Process rmdir (pid: 5032, ti=c8790000 task=f6636960 task.ti=c8790000)

Quoted 15 lines
Stack:
 f53cc5c0 f62b7258 f10991c8 00000282 f6227800 c0733fa0 ec1c5140 edfc6d34
 c8790000 c04463ce f6a4c84c edfc6d34 0804c840 c048883c f6a4c84c f6a4c84c
 c04888d6 f6a4c84c 00000000 c04897de f6a4c84c c048504c f115f4c0 ebc37954
Call Trace:
 [<c04463ce>] cgroup_diput+0x41/0x8c
 [<c048883c>] dentry_iput+0x45/0x5e
 [<c04888d6>] d_kill+0x19/0x32
 [<c04897de>] dput+0xd8/0xdf
 [<c048504c>] do_rmdir+0x8f/0xb6
 [<c06330fc>] do_page_fault+0x2a2/0x579
 [<c0402fc1>] sysenter_do_call+0x12/0x21
 [<c0630000>] schedule+0x641/0x830
Code: 08 00 74 04 0f 0b eb fe 83 7f 04 00 74 04 0f 0b eb fe 45 83 c3 1c 83 fd 0
EIP: [<c04ec7de>] iocg_destroy+0xdc/0x14e SS:ESP 0068:c8790f00

We have seen this too. And have been able to reproduce it. I did not get a chance to fix it so far, but my understanding is that one of the async queues was active when the cgroup was getting destroyed. We moved it to root cgroup, but did not deactivate it; so active_entity still points to the entity of the async queue which has now been moved to the root cgroup. I will send an update if I can verify this, or fix it.

Quoted 6 lines

--
Regards
Gui Jianfeng

#180 lore raw
In reply to Gui Jianfeng
Quoted 10 lines · Gui Jianfeng
Quoted 4 lines · Vivek Goyal

Hi All,

Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.

Hi Vivek,

I got the following OOPS when testing, can't reproduce again :(

Hi Gui,

Thanks for the report. Will look into it and see if I can reproduce it.

Thanks
Vivek

Quoted 30 lines

kernel BUG at block/elevator-fq.c:1396!
invalid opcode: 0000 [#1] SMP
last sysfs file: /sys/block/hdb/queue/scheduler

Modules linked in: ipv6 cpufreq_ondemand acpi_cpufreq dm_mirror dm_multipath sbd

Pid: 5032, comm: rmdir Not tainted (2.6.29-rc7-vivek #17) Veriton M460
EIP: 0060:[<c04ec7de>] EFLAGS: 00010082 CPU: 0
EIP is at iocg_destroy+0xdc/0x14e
EAX: 00000000 EBX: f62278b4 ECX: f6207800 EDX: f6227904
ESI: f6227800 EDI: f62278a0 EBP: 00000003 ESP: c8790f00
DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
Process rmdir (pid: 5032, ti=c8790000 task=f6636960 task.ti=c8790000)

Stack:
 f53cc5c0 f62b7258 f10991c8 00000282 f6227800 c0733fa0 ec1c5140 edfc6d34
 c8790000 c04463ce f6a4c84c edfc6d34 0804c840 c048883c f6a4c84c f6a4c84c
 c04888d6 f6a4c84c 00000000 c04897de f6a4c84c c048504c f115f4c0 ebc37954
Call Trace:
 [<c04463ce>] cgroup_diput+0x41/0x8c
 [<c048883c>] dentry_iput+0x45/0x5e
 [<c04888d6>] d_kill+0x19/0x32
 [<c04897de>] dput+0xd8/0xdf
 [<c048504c>] do_rmdir+0x8f/0xb6
 [<c06330fc>] do_page_fault+0x2a2/0x579
 [<c0402fc1>] sysenter_do_call+0x12/0x21
 [<c0630000>] schedule+0x641/0x830
Code: 08 00 74 04 0f 0b eb fe 83 7f 04 00 74 04 0f 0b eb fe 45 83 c3 1c 83 fd 0
EIP: [<c04ec7de>] iocg_destroy+0xdc/0x14e SS:ESP 0068:c8790f00
Signature · 2 lines
Regards
Gui Jianfeng

A message that is not in the archive (sent off-list or not yet received): [email protected]

#182 lore raw
Quoted 6 lines · Vivek Goyal
Quoted 5 lines · Gui Jianfeng
Quoted 4 lines · Vivek Goyal

Hi All,

Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.

Quoted 8 lines
Quoted 4 lines

Hi Vivek,

I got the following OOPS when testing, can't reproduce again :(

Hi Gui,

Thanks for the report. Will look into it and see if I can reproduce it.

Hi Vivek,

The following script can reproduce the bug in my box.

#!/bin/sh

mkdir /cgroup
mount -t cgroup -o io io /cgroup mkdir /cgroup/test1
mkdir /cgroup/test2

echo cfq > /sys/block/sda/queue/scheduler echo 7 > /cgroup/test1/io.ioprio echo 1 > /cgroup/test2/io.ioprio echo 1 > /proc/sys/vm/drop_caches dd if=1000M.1 of=/dev/null & pid1=$!
echo $pid1
echo $pid1 > /cgroup/test1/tasks
dd if=1000M.2 of=/dev/null
pid2=$!
echo $pid2
echo $pid2 > /cgroup/test2/tasks

rmdir /cgroup/test1
rmdir /cgroup/test2
umount /cgroup
rmdir /cgroup

Signature · 2 lines
Regards
Gui Jianfeng
#183 lore raw
In reply to Vivek Goyal
Quoted 6 lines · Vivek Goyal
Quoted 5 lines · Gui Jianfeng
Quoted 4 lines · Vivek Goyal

Hi All,

Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.

Quoted 8 lines
Quoted 4 lines

Hi Vivek,

I got the following OOPS when testing, can't reproduce again :(

Hi Gui,

Thanks for the report. Will look into it and see if I can reproduce it.

Hi Vivek,

The following script can reproduce the bug in my box.

#!/bin/sh

mkdir /cgroup
mount -t cgroup -o io io /cgroup mkdir /cgroup/test1
mkdir /cgroup/test2

echo cfq > /sys/block/sda/queue/scheduler echo 7 > /cgroup/test1/io.ioprio echo 1 > /cgroup/test2/io.ioprio echo 1 > /proc/sys/vm/drop_caches dd if=1000M.1 of=/dev/null & pid1=$!
echo $pid1
echo $pid1 > /cgroup/test1/tasks
dd if=1000M.2 of=/dev/null
pid2=$!
echo $pid2
echo $pid2 > /cgroup/test2/tasks

rmdir /cgroup/test1
rmdir /cgroup/test2
umount /cgroup
rmdir /cgroup

Signature · 2 lines
Regards
Gui Jianfeng

A message that is not in the archive (sent off-list or not yet received): [email protected]

#185 lore raw
Quoted 41 lines · Gui Jianfeng
Quoted 6 lines · Vivek Goyal
Quoted 5 lines · Gui Jianfeng
Quoted 4 lines · Vivek Goyal

Hi All,

Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.

Quoted 8 lines
Quoted 4 lines

  Hi Vivek,

  I got the following OOPS when testing, can't reproduce again :(

Hi Gui,

Thanks for the report. Will look into it and see if I can reproduce it.

 Hi Vivek,

 The following script can reproduce the bug in my box.

#!/bin/sh

mkdir /cgroup
mount -t cgroup -o io io /cgroup mkdir /cgroup/test1
mkdir /cgroup/test2

echo cfq > /sys/block/sda/queue/scheduler echo 7 > /cgroup/test1/io.ioprio echo 1 > /cgroup/test2/io.ioprio echo 1 > /proc/sys/vm/drop_caches dd if=1000M.1 of=/dev/null & pid1=$!
echo $pid1
echo $pid1 > /cgroup/test1/tasks
dd if=1000M.2 of=/dev/null
pid2=$!
echo $pid2
echo $pid2 > /cgroup/test2/tasks

Quoted 4 lines

rmdir /cgroup/test1
rmdir /cgroup/test2
umount /cgroup
rmdir /cgroup

Yes, this bug happens when we move a task from a cgroup to another one, and delete the cgroup. Since the actual move to the new cgroup is performed in a delayed fashion, if the cgroup is removed before another request from the task is seen (and the actual move is performed) , it results in a hit on BUG_ON. I am working on a patch that will solve this problem and a few others; basically it would do ref counting for io_group structure. I am having a few problems with it at the moment; will post the patch as soon as I can get it to work.

Quoted 8 lines

--
Regards
Gui Jianfeng

#186 lore raw
In reply to a message not in the archive
Quoted 41 lines · Gui Jianfeng
Quoted 6 lines · Vivek Goyal
Quoted 5 lines · Gui Jianfeng
Quoted 4 lines · Vivek Goyal

Hi All,

Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.

Quoted 8 lines
Quoted 4 lines

Hi Vivek,

I got the following OOPS when testing, can't reproduce again :(

Hi Gui,

Thanks for the report. Will look into it and see if I can reproduce it.

Hi Vivek,

The following script can reproduce the bug in my box.

#!/bin/sh

mkdir /cgroup
mount -t cgroup -o io io /cgroup mkdir /cgroup/test1
mkdir /cgroup/test2

echo cfq > /sys/block/sda/queue/scheduler echo 7 > /cgroup/test1/io.ioprio echo 1 > /cgroup/test2/io.ioprio echo 1 > /proc/sys/vm/drop_caches dd if=1000M.1 of=/dev/null & pid1=$!
echo $pid1
echo $pid1 > /cgroup/test1/tasks
dd if=1000M.2 of=/dev/null
pid2=$!
echo $pid2
echo $pid2 > /cgroup/test2/tasks

Quoted 4 lines

rmdir /cgroup/test1
rmdir /cgroup/test2
umount /cgroup
rmdir /cgroup

Thanks Gui. We have got races with task movement and cgroup deletion. In the original bfq patch, Fabio had implemented the logic to migrate the task queue synchronously. It found the logic to be little complicated so I changed it to delayed movement of queue from old cgroup to new cgroup. Fabio later mentioned that it introduces a race where old cgroup is deleted before task queue has actually moved to new cgroup.

Nauman is currently implementing reference counting for io groups and that will solve this problem at the same time some other problems like movement of queue to root group during cgroup deletion and which can potentially result in unfair share for some time to that queue etc.

Thanks
Vivek

#187 lore raw
In reply to Gui Jianfeng
Quoted 41 lines · Gui Jianfeng
Quoted 6 lines · Vivek Goyal
Quoted 5 lines · Gui Jianfeng
Quoted 4 lines · Vivek Goyal

Hi All,

Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.

Quoted 8 lines
Quoted 4 lines

  Hi Vivek,

  I got the following OOPS when testing, can't reproduce again :(

Hi Gui,

Thanks for the report. Will look into it and see if I can reproduce it.

 Hi Vivek,

 The following script can reproduce the bug in my box.

#!/bin/sh

mkdir /cgroup
mount -t cgroup -o io io /cgroup mkdir /cgroup/test1
mkdir /cgroup/test2

echo cfq > /sys/block/sda/queue/scheduler echo 7 > /cgroup/test1/io.ioprio echo 1 > /cgroup/test2/io.ioprio echo 1 > /proc/sys/vm/drop_caches dd if=1000M.1 of=/dev/null & pid1=$!
echo $pid1
echo $pid1 > /cgroup/test1/tasks
dd if=1000M.2 of=/dev/null
pid2=$!
echo $pid2
echo $pid2 > /cgroup/test2/tasks

Quoted 4 lines

rmdir /cgroup/test1
rmdir /cgroup/test2
umount /cgroup
rmdir /cgroup

Yes, this bug happens when we move a task from a cgroup to another one, and delete the cgroup. Since the actual move to the new cgroup is performed in a delayed fashion, if the cgroup is removed before another request from the task is seen (and the actual move is performed) , it results in a hit on BUG_ON. I am working on a patch that will solve this problem and a few others; basically it would do ref counting for io_group structure. I am having a few problems with it at the moment; will post the patch as soon as I can get it to work.

Quoted 8 lines

--
Regards
Gui Jianfeng

#188 lore raw
In reply to Gui Jianfeng
Quoted 41 lines · Gui Jianfeng
Quoted 6 lines · Vivek Goyal
Quoted 5 lines · Gui Jianfeng
Quoted 4 lines · Vivek Goyal

Hi All,

Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.

Quoted 8 lines
Quoted 4 lines

Hi Vivek,

I got the following OOPS when testing, can't reproduce again :(

Hi Gui,

Thanks for the report. Will look into it and see if I can reproduce it.

Hi Vivek,

The following script can reproduce the bug in my box.

#!/bin/sh

mkdir /cgroup
mount -t cgroup -o io io /cgroup mkdir /cgroup/test1
mkdir /cgroup/test2

echo cfq > /sys/block/sda/queue/scheduler echo 7 > /cgroup/test1/io.ioprio echo 1 > /cgroup/test2/io.ioprio echo 1 > /proc/sys/vm/drop_caches dd if=1000M.1 of=/dev/null & pid1=$!
echo $pid1
echo $pid1 > /cgroup/test1/tasks
dd if=1000M.2 of=/dev/null
pid2=$!
echo $pid2
echo $pid2 > /cgroup/test2/tasks

Quoted 4 lines

rmdir /cgroup/test1
rmdir /cgroup/test2
umount /cgroup
rmdir /cgroup

Thanks Gui. We have got races with task movement and cgroup deletion. In the original bfq patch, Fabio had implemented the logic to migrate the task queue synchronously. It found the logic to be little complicated so I changed it to delayed movement of queue from old cgroup to new cgroup. Fabio later mentioned that it introduces a race where old cgroup is deleted before task queue has actually moved to new cgroup.

Nauman is currently implementing reference counting for io groups and that will solve this problem at the same time some other problems like movement of queue to root group during cgroup deletion and which can potentially result in unfair share for some time to that queue etc.

Thanks
Vivek

A message that is not in the archive (sent off-list or not yet received): [email protected]

#190 lore raw
Quoted 60 lines · Vivek Goyal
Quoted 45 lines · Gui Jianfeng
Quoted 9 lines · Vivek Goyal
Quoted 7 lines · Gui Jianfeng
Quoted 5 lines · Vivek Goyal

Hi All,

Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.

Quoted 9 lines
Quoted 5 lines

Hi Vivek,

I got the following OOPS when testing, can't reproduce again :(

Hi Gui,

Thanks for the report. Will look into it and see if I can reproduce it.

Hi Vivek,

The following script can reproduce the bug in my box.

#!/bin/sh

mkdir /cgroup
mount -t cgroup -o io io /cgroup mkdir /cgroup/test1
mkdir /cgroup/test2

echo cfq > /sys/block/sda/queue/scheduler echo 7 > /cgroup/test1/io.ioprio echo 1 > /cgroup/test2/io.ioprio echo 1 > /proc/sys/vm/drop_caches dd if=1000M.1 of=/dev/null & pid1=$!
echo $pid1
echo $pid1 > /cgroup/test1/tasks
dd if=1000M.2 of=/dev/null
pid2=$!
echo $pid2
echo $pid2 > /cgroup/test2/tasks

Quoted 5 lines

rmdir /cgroup/test1
rmdir /cgroup/test2
umount /cgroup
rmdir /cgroup

Thanks Gui. We have got races with task movement and cgroup deletion. In the original bfq patch, Fabio had implemented the logic to migrate the task queue synchronously. It found the logic to be little complicated so I changed it to delayed movement of queue from old cgroup to new cgroup. Fabio later mentioned that it introduces a race where old cgroup is deleted before task queue has actually moved to new cgroup.

Quoted 8 lines

Nauman is currently implementing reference counting for io groups and that will solve this problem at the same time some other problems like movement of queue to root group during cgroup deletion and which can potentially result in unfair share for some time to that queue etc.

Thanks
Vivek

Hi Gui,
This patch should solve the problems reported by you. Please let me know if it does not work. @Vivek, this has a few more changes after the patch I sent you separately.

DESC
Add ref counting for io_group.
EDESC

Reference counting for io_group solves many problems, most of which occured when we tried to delete the cgroup. Earlier, ioqs were being moved out of cgroup to root cgroup. That is problematic in many ways:
First, the pending requests in queues might get unfair service, and will also cause unfairness for other cgroups at the root level. This problem can become signficant if cgroups are created and destroyed relatively frequently. Second, moving queues to root cgroups was complicated and was causing many BUG_ON's to trigger. Third, there is a single io queue in AS, Deadline and Noop within a cgroup; and it does not make sense to move it to the root cgroup. The same is true of async queues.

Requests already keep a reference on ioq, so queues keep a reference on cgroup. For async queues in CFQ, and single ioq in other schedulers, io_group also keeps are reference on io_queue. This reference on ioq is dropped when the queue is released (elv_release_ioq). So the queue can be freed.

When a queue is released, it puts the reference to io_group and the io_group is released after all the queues are released. Child groups also take reference on parent groups, and release it when they are destroyed.

Also we no longer need to maintain a seprate linked list of idle entities, which was maintained only to help release the ioq references during elevator switch. The code for releasing io_groups is reused for elevator switch, resulting in simpler and tight code.

block/cfq-iosched.c +11 -6
index 0ecf7c7..21e8ab8 100644
@@ -1090,8 +1090,8 @@ static void changed_cgroup(struct io_context *ioc, struct cfq_io_context *cic)
 
 	if (async_cfqq != NULL) {
 		__iog = cfqq_to_io_group(async_cfqq);
-
 		if (iog != __iog) {
+			/* Cgroup has changed, drop the reference to async queue */
 			cic_set_cfqq(cic, NULL, 0);
 			cfq_put_queue(async_cfqq);
 		}
@@ -1099,8 +1099,10 @@ static void changed_cgroup(struct io_context *ioc, struct cfq_io_context *cic)
 
 	if (sync_cfqq != NULL) {
 		__iog = cfqq_to_io_group(sync_cfqq);
-		if (iog != __iog)
-			io_ioq_move(q->elevator, sync_cfqq->ioq, iog);
+		if (iog != __iog) {
+			cic_set_cfqq(cic, NULL, 1);
+			cfq_put_queue(sync_cfqq);
+		}
 	}
 
 	spin_unlock_irqrestore(q->queue_lock, flags);
@@ -1114,8 +1116,8 @@ static void cfq_ioc_set_cgroup(struct io_context *ioc)
 #endif  /* CONFIG_IOSCHED_CFQ_HIER */
 
 static struct cfq_queue *
-cfq_find_alloc_queue(struct cfq_data *cfqd, int is_sync,
-				struct io_context *ioc, gfp_t gfp_mask)
+cfq_find_alloc_queue(struct cfq_data *cfqd, struct io_group *iog,
+		     int is_sync, struct io_context *ioc, gfp_t gfp_mask)
 {
 	struct cfq_queue *cfqq, *new_cfqq = NULL;
 	struct cfq_io_context *cic;
@@ -1198,6 +1200,8 @@ alloc_ioq:
 			elv_mark_ioq_sync(cfqq->ioq);
 		}
 		cfqq->pid = current->pid;
+		/* ioq reference on iog */
+		elv_get_iog(iog);
 		cfq_log_cfqq(cfqd, cfqq, "alloced");
 	}
 
@@ -1229,7 +1233,8 @@ cfq_get_queue(struct cfq_data *cfqd, int is_sync, struct io_context *ioc,
 	}
 
 	if (!cfqq) {
-		cfqq = cfq_find_alloc_queue(cfqd, is_sync, ioc, gfp_mask);
+		cfqq = cfq_find_alloc_queue(cfqd, iog, 
+					    is_sync, ioc, gfp_mask);
 		if (!cfqq)
 			return NULL;
 	}
block/elevator-fq.c +98 -108

384 lines not on this page · raw message

block/elevator-fq.h +19 -0
index 62b2ee2..7622b28 100644
@@ -213,6 +213,7 @@ struct io_group {
 	struct hlist_node elv_data_node;
 	struct hlist_node group_node;
 	struct io_sched_data sched_data;
+	atomic_t ref;
 
 	struct io_entity *my_entity;
 
@@ -229,6 +230,8 @@ struct io_group {
 
 	/* Single ioq per group, used for noop, deadline, anticipatory */
 	struct io_queue *ioq;
+
+	int deleting;
 };
 
 /**
@@ -462,6 +465,8 @@ extern int elv_fq_set_request_ioq(struct request_queue *q, struct request *rq,
 					gfp_t gfp_mask);
 extern void elv_fq_unset_request_ioq(struct request_queue *q,
 					struct request *rq);
+extern void elv_put_iog(struct io_group *iog);
+
 extern struct io_queue *elv_lookup_ioq_current(struct request_queue *q);
 
 /* Returns single ioq associated with the io group. */
@@ -480,6 +485,11 @@ static inline void io_group_set_ioq(struct io_group *iog, struct io_queue *ioq)
 	iog->ioq = ioq;
 }
 
+static inline void elv_get_iog(struct io_group *iog)
+{
+	atomic_inc(&iog->ref);
+}
+
 #else /* !GROUP_IOSCHED */
 /*
  * No ioq movement is needed in case of flat setup. root io group gets cleaned
@@ -531,6 +541,15 @@ static inline struct io_queue *elv_lookup_ioq_current(struct request_queue *q)
 	return NULL;
 }
 
+static inline void elv_get_iog(struct io_group *iog)
+{
+}
+
+static inline void elv_put_iog(struct io_group *iog)
+{
+}
+
+
 #endif /* GROUP_IOSCHED */
 
 /* Functions used by blksysfs.c */
#191 lore raw
In reply to Vivek Goyal
Quoted 60 lines · Vivek Goyal
Quoted 45 lines · Gui Jianfeng
Quoted 9 lines · Vivek Goyal
Quoted 7 lines · Gui Jianfeng
Quoted 5 lines · Vivek Goyal

Hi All,

Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.

Quoted 9 lines
Quoted 5 lines

Hi Vivek,

I got the following OOPS when testing, can't reproduce again :(

Hi Gui,

Thanks for the report. Will look into it and see if I can reproduce it.

Hi Vivek,

The following script can reproduce the bug in my box.

#!/bin/sh

mkdir /cgroup
mount -t cgroup -o io io /cgroup mkdir /cgroup/test1
mkdir /cgroup/test2

echo cfq > /sys/block/sda/queue/scheduler echo 7 > /cgroup/test1/io.ioprio echo 1 > /cgroup/test2/io.ioprio echo 1 > /proc/sys/vm/drop_caches dd if=1000M.1 of=/dev/null & pid1=$!
echo $pid1
echo $pid1 > /cgroup/test1/tasks
dd if=1000M.2 of=/dev/null
pid2=$!
echo $pid2
echo $pid2 > /cgroup/test2/tasks

Quoted 5 lines

rmdir /cgroup/test1
rmdir /cgroup/test2
umount /cgroup
rmdir /cgroup

Thanks Gui. We have got races with task movement and cgroup deletion. In the original bfq patch, Fabio had implemented the logic to migrate the task queue synchronously. It found the logic to be little complicated so I changed it to delayed movement of queue from old cgroup to new cgroup. Fabio later mentioned that it introduces a race where old cgroup is deleted before task queue has actually moved to new cgroup.

Quoted 8 lines

Nauman is currently implementing reference counting for io groups and that will solve this problem at the same time some other problems like movement of queue to root group during cgroup deletion and which can potentially result in unfair share for some time to that queue etc.

Thanks
Vivek

Hi Gui,
This patch should solve the problems reported by you. Please let me know if it does not work. @Vivek, this has a few more changes after the patch I sent you separately.

DESC
Add ref counting for io_group.
EDESC

Reference counting for io_group solves many problems, most of which occured when we tried to delete the cgroup. Earlier, ioqs were being moved out of cgroup to root cgroup. That is problematic in many ways:
First, the pending requests in queues might get unfair service, and will also cause unfairness for other cgroups at the root level. This problem can become signficant if cgroups are created and destroyed relatively frequently. Second, moving queues to root cgroups was complicated and was causing many BUG_ON's to trigger. Third, there is a single io queue in AS, Deadline and Noop within a cgroup; and it does not make sense to move it to the root cgroup. The same is true of async queues.

Requests already keep a reference on ioq, so queues keep a reference on cgroup. For async queues in CFQ, and single ioq in other schedulers, io_group also keeps are reference on io_queue. This reference on ioq is dropped when the queue is released (elv_release_ioq). So the queue can be freed.

When a queue is released, it puts the reference to io_group and the io_group is released after all the queues are released. Child groups also take reference on parent groups, and release it when they are destroyed.

Also we no longer need to maintain a seprate linked list of idle entities, which was maintained only to help release the ioq references during elevator switch. The code for releasing io_groups is reused for elevator switch, resulting in simpler and tight code.

block/cfq-iosched.c +11 -6
index 0ecf7c7..21e8ab8 100644
@@ -1090,8 +1090,8 @@ static void changed_cgroup(struct io_context *ioc, struct cfq_io_context *cic)
 
 	if (async_cfqq != NULL) {
 		__iog = cfqq_to_io_group(async_cfqq);
-
 		if (iog != __iog) {
+			/* Cgroup has changed, drop the reference to async queue */
 			cic_set_cfqq(cic, NULL, 0);
 			cfq_put_queue(async_cfqq);
 		}
@@ -1099,8 +1099,10 @@ static void changed_cgroup(struct io_context *ioc, struct cfq_io_context *cic)
 
 	if (sync_cfqq != NULL) {
 		__iog = cfqq_to_io_group(sync_cfqq);
-		if (iog != __iog)
-			io_ioq_move(q->elevator, sync_cfqq->ioq, iog);
+		if (iog != __iog) {
+			cic_set_cfqq(cic, NULL, 1);
+			cfq_put_queue(sync_cfqq);
+		}
 	}
 
 	spin_unlock_irqrestore(q->queue_lock, flags);
@@ -1114,8 +1116,8 @@ static void cfq_ioc_set_cgroup(struct io_context *ioc)
 #endif  /* CONFIG_IOSCHED_CFQ_HIER */
 
 static struct cfq_queue *
-cfq_find_alloc_queue(struct cfq_data *cfqd, int is_sync,
-				struct io_context *ioc, gfp_t gfp_mask)
+cfq_find_alloc_queue(struct cfq_data *cfqd, struct io_group *iog,
+		     int is_sync, struct io_context *ioc, gfp_t gfp_mask)
 {
 	struct cfq_queue *cfqq, *new_cfqq = NULL;
 	struct cfq_io_context *cic;
@@ -1198,6 +1200,8 @@ alloc_ioq:
 			elv_mark_ioq_sync(cfqq->ioq);
 		}
 		cfqq->pid = current->pid;
+		/* ioq reference on iog */
+		elv_get_iog(iog);
 		cfq_log_cfqq(cfqd, cfqq, "alloced");
 	}
 
@@ -1229,7 +1233,8 @@ cfq_get_queue(struct cfq_data *cfqd, int is_sync, struct io_context *ioc,
 	}
 
 	if (!cfqq) {
-		cfqq = cfq_find_alloc_queue(cfqd, is_sync, ioc, gfp_mask);
+		cfqq = cfq_find_alloc_queue(cfqd, iog, 
+					    is_sync, ioc, gfp_mask);
 		if (!cfqq)
 			return NULL;
 	}
block/elevator-fq.c +98 -108

384 lines not on this page · raw message

block/elevator-fq.h +19 -0
index 62b2ee2..7622b28 100644
@@ -213,6 +213,7 @@ struct io_group {
 	struct hlist_node elv_data_node;
 	struct hlist_node group_node;
 	struct io_sched_data sched_data;
+	atomic_t ref;
 
 	struct io_entity *my_entity;
 
@@ -229,6 +230,8 @@ struct io_group {
 
 	/* Single ioq per group, used for noop, deadline, anticipatory */
 	struct io_queue *ioq;
+
+	int deleting;
 };
 
 /**
@@ -462,6 +465,8 @@ extern int elv_fq_set_request_ioq(struct request_queue *q, struct request *rq,
 					gfp_t gfp_mask);
 extern void elv_fq_unset_request_ioq(struct request_queue *q,
 					struct request *rq);
+extern void elv_put_iog(struct io_group *iog);
+
 extern struct io_queue *elv_lookup_ioq_current(struct request_queue *q);
 
 /* Returns single ioq associated with the io group. */
@@ -480,6 +485,11 @@ static inline void io_group_set_ioq(struct io_group *iog, struct io_queue *ioq)
 	iog->ioq = ioq;
 }
 
+static inline void elv_get_iog(struct io_group *iog)
+{
+	atomic_inc(&iog->ref);
+}
+
 #else /* !GROUP_IOSCHED */
 /*
  * No ioq movement is needed in case of flat setup. root io group gets cleaned
@@ -531,6 +541,15 @@ static inline struct io_queue *elv_lookup_ioq_current(struct request_queue *q)
 	return NULL;
 }
 
+static inline void elv_get_iog(struct io_group *iog)
+{
+}
+
+static inline void elv_put_iog(struct io_group *iog)
+{
+}
+
+
 #endif /* GROUP_IOSCHED */
 
 /* Functions used by blksysfs.c */

A message that is not in the archive (sent off-list or not yet received): [email protected]

#193 lore raw

Nauman Rafique wrote:
...

Quoted 4 lines

Hi Gui,
This patch should solve the problems reported by you. Please let me know if it does not work. @Vivek, this has a few more changes after the patch I sent you separately.

Hi Nauman,

I'v tried your patch, seems the bug has been fixed. thanks!

Quoted 32 lines

DESC
Add ref counting for io_group.
EDESC

Reference counting for io_group solves many problems, most of which occured when we tried to delete the cgroup. Earlier, ioqs were being moved out of cgroup to root cgroup. That is problematic in many ways:
First, the pending requests in queues might get unfair service, and will also cause unfairness for other cgroups at the root level. This problem can become signficant if cgroups are created and destroyed relatively frequently. Second, moving queues to root cgroups was complicated and was causing many BUG_ON's to trigger. Third, there is a single io queue in AS, Deadline and Noop within a cgroup; and it does not make sense to move it to the root cgroup. The same is true of async queues.

Requests already keep a reference on ioq, so queues keep a reference on cgroup. For async queues in CFQ, and single ioq in other schedulers, io_group also keeps are reference on io_queue. This reference on ioq is dropped when the queue is released (elv_release_ioq). So the queue can be freed.

When a queue is released, it puts the reference to io_group and the io_group is released after all the queues are released. Child groups also take reference on parent groups, and release it when they are destroyed.

Also we no longer need to maintain a seprate linked list of idle entities, which was maintained only to help release the ioq references during elevator switch. The code for releasing io_groups is reused for elevator switch, resulting in simpler and tight code.

Signature · 2 lines
Regards
Gui Jianfeng
#194 lore raw
In reply to Nauman Rafique

Nauman Rafique wrote:
...

Quoted 4 lines

Hi Gui,
This patch should solve the problems reported by you. Please let me know if it does not work. @Vivek, this has a few more changes after the patch I sent you separately.

Hi Nauman,

I'v tried your patch, seems the bug has been fixed. thanks!

Quoted 32 lines

DESC
Add ref counting for io_group.
EDESC

Reference counting for io_group solves many problems, most of which occured when we tried to delete the cgroup. Earlier, ioqs were being moved out of cgroup to root cgroup. That is problematic in many ways:
First, the pending requests in queues might get unfair service, and will also cause unfairness for other cgroups at the root level. This problem can become signficant if cgroups are created and destroyed relatively frequently. Second, moving queues to root cgroups was complicated and was causing many BUG_ON's to trigger. Third, there is a single io queue in AS, Deadline and Noop within a cgroup; and it does not make sense to move it to the root cgroup. The same is true of async queues.

Requests already keep a reference on ioq, so queues keep a reference on cgroup. For async queues in CFQ, and single ioq in other schedulers, io_group also keeps are reference on io_queue. This reference on ioq is dropped when the queue is released (elv_release_ioq). So the queue can be freed.

When a queue is released, it puts the reference to io_group and the io_group is released after all the queues are released. Child groups also take reference on parent groups, and release it when they are destroyed.

Also we no longer need to maintain a seprate linked list of idle entities, which was maintained only to help release the ioq references during elevator switch. The code for releasing io_groups is reused for elevator switch, resulting in simpler and tight code.

Signature · 2 lines
Regards
Gui Jianfeng
#195 lore raw
In reply to Vivek Goyal
Quoted 35 lines · Vivek Goyal

Hi All,

Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.

http://lkml.org/lkml/2008/11/6/227

One of the takeaway from the discussion in this thread was that let us implement a common layer which contains the proportional weight scheduling code which can be shared by all the IO schedulers.

Implementing IO controller will not cover the devices which don't use IO schedulers but it should cover the common case.

There were more discussions regarding 2 level vs 1 level IO control at following link.

https://lists.linux-foundation.org/pipermail/containers/2009-January/015402.html

So in the mean time we took the discussion off the list and spent time on making the 1 level control apporoach work where majority of the proportional weight control is shared by the four schedulers instead of each one having to replicate the code. We make use of BFQ code for fair queuing as posted by Paolo and Fabio here.

http://lkml.org/lkml/2008/11/11/148

Details about design and howto have been put in documentation patch.

I have done very basic testing of running 2 or 3 "dd" threads in different cgroups. Wanted to get the patchset out for feedback/review before we dive into more bug fixing, benchmarking, optimizations etc.

Your feedback/comments are welcome.

Quoted 8 lines

Patch series contains 10 patches. It should be compilable and bootable after every patch. Intial 2 patches implement flat fair queuing (no cgroup support) and make cfq to use that. Later patches introduce hierarchical fair queuing support in elevator layer and modify other IO schdulers to use that.

Thanks
Vivek

Hi Vivek,
While testing these patches along with the bio-cgroup patches I noticed that for the case of 2 buffered writers (dd) with different weights, one of them would be able to use up a very large timeslice (I've seen upto 500ms) when the other queue is empty and not be accounted for it. This is due to the check in cfq_dispatch_requests() where a given cgroup can empty its entire queue (100 IOs or more) within its timeslice and have them sit in the dispatch queue ready for the disk driver to pick up. Moreover, this huge timeslice is not accounted for as this cgroup is charged only for the length of the intended timeslice and not the actual time taken.
The following patch fixes this by not optimizing on the single busy queue fact inside cfq_dispatch_requests. Note that this does not hurt throughput in any sense but just causes more IOs to be dispatched only when the drive is ready for them thus leading to better accounting too.

Fix bug where a given ioq can run through all its requests at once.

---

2.6.26/block/cfq-iosched.c +0 -0
index 5a275a2..c0199a6 100644
@@ -848,8 +848,7 @@ static int cfq_dispatch_requests(struct request_queue *q, int force)
		if (cfq_class_idle(cfqq))
			max_dispatch = 1;

- if (elv_ioq_nr_dispatched(cfqq->ioq) >= max_dispatch &&
- elv_nr_busy_ioq(q->elevator) > 1)
+ if (elv_ioq_nr_dispatched(cfqq->ioq) >= max_dispatch)

			break;

		if (cfqd->sync_flight && !cfq_cfqq_sync(cfqq))

A message that is not in the archive (sent off-list or not yet received): [email protected]

#197 lore raw
Quoted 42 lines · Divyesh Shah
Quoted 41 lines · Vivek Goyal

Hi All,

Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.

http://lkml.org/lkml/2008/11/6/227

One of the takeaway from the discussion in this thread was that let us implement a common layer which contains the proportional weight scheduling code which can be shared by all the IO schedulers.

Implementing IO controller will not cover the devices which don't use IO schedulers but it should cover the common case.

There were more discussions regarding 2 level vs 1 level IO control at following link.

https://lists.linux-foundation.org/pipermail/containers/2009-January/015402.html

So in the mean time we took the discussion off the list and spent time on making the 1 level control apporoach work where majority of the proportional weight control is shared by the four schedulers instead of each one having to replicate the code. We make use of BFQ code for fair queuing as posted by Paolo and Fabio here.

http://lkml.org/lkml/2008/11/11/148

Details about design and howto have been put in documentation patch.

I have done very basic testing of running 2 or 3 "dd" threads in different cgroups. Wanted to get the patchset out for feedback/review before we dive into more bug fixing, benchmarking, optimizations etc.

Your feedback/comments are welcome.

Patch series contains 10 patches. It should be compilable and bootable after every patch. Intial 2 patches implement flat fair queuing (no cgroup support) and make cfq to use that. Later patches introduce hierarchical fair queuing support in elevator layer and modify other IO schdulers to use that.

Quoted 6 lines
Quoted 2 lines

Thanks
Vivek

Hi Vivek,
While testing these patches along with the bio-cgroup patches I noticed that for the case of 2 buffered writers (dd) with different weights, one of them would be able to use up a very large timeslice (I've seen upto 500ms) when the other queue is empty and not be accounted for it. This is due to the check in cfq_dispatch_requests() where a given cgroup can empty its entire queue (100 IOs or more) within its timeslice and have them sit in the dispatch queue ready for the disk driver to pick up. Moreover, this huge timeslice is not accounted for as this cgroup is charged only for the length of the intended timeslice and not the actual time taken.
The following patch fixes this by not optimizing on the single busy queue fact inside cfq_dispatch_requests. Note that this does not hurt throughput in any sense but just causes more IOs to be dispatched only when the drive is ready for them thus leading to better accounting too.

Hi Divyesh,

Thanks for the testing and noticing the issue. I also had noticed this issue.

Couple of points.

- In 30-rc3 jens has fixed the huge dispatch problem. Now in case of single ioq doing dispatch, in one round upto 4*quantum request can be dispatched. So that means in default configuration with single queue, maximum request on diaptch list can be 16.

- Secondly, in my tree, now I have modified the patches to charge for actual consumption of the slice instead of capping it to budget. In a week's time I should be able to post V2 of the patches. Please do try it out then

Thanks
Vivek

Quoted 19 lines

Fix bug where a given ioq can run through all its requests at once.

---

2.6.26/block/cfq-iosched.c +1 -2
index 5a275a2..c0199a6 100644
@@ -848,8 +848,7 @@ static int cfq_dispatch_requests(struct request_queue *q, int force)
 		if (cfq_class_idle(cfqq))
 			max_dispatch = 1;
 
-		if (elv_ioq_nr_dispatched(cfqq->ioq) >= max_dispatch &&
-			elv_nr_busy_ioq(q->elevator) > 1)
+		if (elv_ioq_nr_dispatched(cfqq->ioq) >= max_dispatch)
 			break;
 
 		if (cfqd->sync_flight && !cfq_cfqq_sync(cfqq))
#198 lore raw
In reply to Divyesh Shah
Quoted 42 lines · Divyesh Shah
Quoted 41 lines · Vivek Goyal

Hi All,

Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.

http://lkml.org/lkml/2008/11/6/227

One of the takeaway from the discussion in this thread was that let us implement a common layer which contains the proportional weight scheduling code which can be shared by all the IO schedulers.

Implementing IO controller will not cover the devices which don't use IO schedulers but it should cover the common case.

There were more discussions regarding 2 level vs 1 level IO control at following link.

https://lists.linux-foundation.org/pipermail/containers/2009-January/015402.html

So in the mean time we took the discussion off the list and spent time on making the 1 level control apporoach work where majority of the proportional weight control is shared by the four schedulers instead of each one having to replicate the code. We make use of BFQ code for fair queuing as posted by Paolo and Fabio here.

http://lkml.org/lkml/2008/11/11/148

Details about design and howto have been put in documentation patch.

I have done very basic testing of running 2 or 3 "dd" threads in different cgroups. Wanted to get the patchset out for feedback/review before we dive into more bug fixing, benchmarking, optimizations etc.

Your feedback/comments are welcome.

Patch series contains 10 patches. It should be compilable and bootable after every patch. Intial 2 patches implement flat fair queuing (no cgroup support) and make cfq to use that. Later patches introduce hierarchical fair queuing support in elevator layer and modify other IO schdulers to use that.

Quoted 6 lines
Quoted 2 lines

Thanks
Vivek

Hi Vivek,
While testing these patches along with the bio-cgroup patches I noticed that for the case of 2 buffered writers (dd) with different weights, one of them would be able to use up a very large timeslice (I've seen upto 500ms) when the other queue is empty and not be accounted for it. This is due to the check in cfq_dispatch_requests() where a given cgroup can empty its entire queue (100 IOs or more) within its timeslice and have them sit in the dispatch queue ready for the disk driver to pick up. Moreover, this huge timeslice is not accounted for as this cgroup is charged only for the length of the intended timeslice and not the actual time taken.
The following patch fixes this by not optimizing on the single busy queue fact inside cfq_dispatch_requests. Note that this does not hurt throughput in any sense but just causes more IOs to be dispatched only when the drive is ready for them thus leading to better accounting too.

Hi Divyesh,

Thanks for the testing and noticing the issue. I also had noticed this issue.

Couple of points.

- In 30-rc3 jens has fixed the huge dispatch problem. Now in case of single ioq doing dispatch, in one round upto 4*quantum request can be dispatched. So that means in default configuration with single queue, maximum request on diaptch list can be 16.

- Secondly, in my tree, now I have modified the patches to charge for actual consumption of the slice instead of capping it to budget. In a week's time I should be able to post V2 of the patches. Please do try it out then

Thanks
Vivek

Quoted 19 lines

Fix bug where a given ioq can run through all its requests at once.

---

2.6.26/block/cfq-iosched.c +1 -2
index 5a275a2..c0199a6 100644
@@ -848,8 +848,7 @@ static int cfq_dispatch_requests(struct request_queue *q, int force)
 		if (cfq_class_idle(cfqq))
 			max_dispatch = 1;
 
-		if (elv_ioq_nr_dispatched(cfqq->ioq) >= max_dispatch &&
-			elv_nr_busy_ioq(q->elevator) > 1)
+		if (elv_ioq_nr_dispatched(cfqq->ioq) >= max_dispatch)
 			break;
 
 		if (cfqd->sync_flight && !cfq_cfqq_sync(cfqq))

A message that is not in the archive (sent off-list or not yet received): [email protected]

#200 lore raw
Quoted 54 lines · Vivek Goyal
Quoted 42 lines · Divyesh Shah
Quoted 41 lines · Vivek Goyal

Hi All,

Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.

http://lkml.org/lkml/2008/11/6/227

One of the takeaway from the discussion in this thread was that let us implement a common layer which contains the proportional weight scheduling code which can be shared by all the IO schedulers.

Implementing IO controller will not cover the devices which don't use IO schedulers but it should cover the common case.

There were more discussions regarding 2 level vs 1 level IO control at following link.

https://lists.linux-foundation.org/pipermail/containers/2009-January/015402.html

So in the mean time we took the discussion off the list and spent time on making the 1 level control apporoach work where majority of the proportional weight control is shared by the four schedulers instead of each one having to replicate the code. We make use of BFQ code for fair queuing as posted by Paolo and Fabio here.

http://lkml.org/lkml/2008/11/11/148

Details about design and howto have been put in documentation patch.

I have done very basic testing of running 2 or 3 "dd" threads in different cgroups. Wanted to get the patchset out for feedback/review before we dive into more bug fixing, benchmarking, optimizations etc.

Your feedback/comments are welcome.

Patch series contains 10 patches. It should be compilable and bootable after every patch. Intial 2 patches implement flat fair queuing (no cgroup support) and make cfq to use that. Later patches introduce hierarchical fair queuing support in elevator layer and modify other IO schdulers to use that.

Quoted 5 lines
Quoted 2 lines

Thanks
Vivek

Hi Vivek,
While testing these patches along with the bio-cgroup patches I noticed that for the case of 2 buffered writers (dd) with different weights, one of them would be able to use up a very large timeslice (I've seen upto 500ms) when the other queue is empty and not be accounted for it. This is due to the check in cfq_dispatch_requests() where a given cgroup can empty its entire queue (100 IOs or more) within its timeslice and have them sit in the dispatch queue ready for the disk driver to pick up. Moreover, this huge timeslice is not accounted for as this cgroup is charged only for the length of the intended timeslice and not the actual time taken.
The following patch fixes this by not optimizing on the single busy queue fact inside cfq_dispatch_requests. Note that this does not hurt throughput in any sense but just causes more IOs to be dispatched only when the drive is ready for them thus leading to better accounting too.

Hi Divyesh,

Thanks for the testing and noticing the issue. I also had noticed this issue.

Quoted 6 lines

Couple of points.

- In 30-rc3 jens has fixed the huge dispatch problem. Now in case of single ioq doing dispatch, in one round upto 4*quantum request can be dispatched. So that means in default configuration with single queue, maximum request on diaptch list can be 16.

I just synced my git tree and I see Jens changes. That makes this much cleaner!

Quoted 5 lines

- Secondly, in my tree, now I have modified the patches to charge for actual consumption of the slice instead of capping it to budget. In a week's time I should be able to post V2 of the patches. Please do try it out then

I have that in my tree as well and was going to send that out. No need now :)

Quoted 22 lines

Thanks
Vivek

Quoted 18 lines

Fix bug where a given ioq can run through all its requests at once.

---

2.6.26/block/cfq-iosched.c +1 -2
index 5a275a2..c0199a6 100644
@@ -848,8 +848,7 @@ static int cfq_dispatch_requests(struct request_queue *q, int force)
 		if (cfq_class_idle(cfqq))
 			max_dispatch = 1;
 
-		if (elv_ioq_nr_dispatched(cfqq->ioq) >= max_dispatch &&
-			elv_nr_busy_ioq(q->elevator) > 1)
+		if (elv_ioq_nr_dispatched(cfqq->ioq) >= max_dispatch)
 			break;
 
 		if (cfqd->sync_flight && !cfq_cfqq_sync(cfqq))
#201 lore raw
In reply to Vivek Goyal
Quoted 54 lines · Vivek Goyal
Quoted 42 lines · Divyesh Shah
Quoted 41 lines · Vivek Goyal

Hi All,

Here is another posting for IO controller patches. Last time I had posted RFC patches for an IO controller which did bio control per cgroup.

http://lkml.org/lkml/2008/11/6/227

One of the takeaway from the discussion in this thread was that let us implement a common layer which contains the proportional weight scheduling code which can be shared by all the IO schedulers.

Implementing IO controller will not cover the devices which don't use IO schedulers but it should cover the common case.

There were more discussions regarding 2 level vs 1 level IO control at following link.

https://lists.linux-foundation.org/pipermail/containers/2009-January/015402.html

So in the mean time we took the discussion off the list and spent time on making the 1 level control apporoach work where majority of the proportional weight control is shared by the four schedulers instead of each one having to replicate the code. We make use of BFQ code for fair queuing as posted by Paolo and Fabio here.

http://lkml.org/lkml/2008/11/11/148

Details about design and howto have been put in documentation patch.

I have done very basic testing of running 2 or 3 "dd" threads in different cgroups. Wanted to get the patchset out for feedback/review before we dive into more bug fixing, benchmarking, optimizations etc.

Your feedback/comments are welcome.

Patch series contains 10 patches. It should be compilable and bootable after every patch. Intial 2 patches implement flat fair queuing (no cgroup support) and make cfq to use that. Later patches introduce hierarchical fair queuing support in elevator layer and modify other IO schdulers to use that.

Quoted 5 lines
Quoted 2 lines

Thanks
Vivek

Hi Vivek,
While testing these patches along with the bio-cgroup patches I noticed that for the case of 2 buffered writers (dd) with different weights, one of them would be able to use up a very large timeslice (I've seen upto 500ms) when the other queue is empty and not be accounted for it. This is due to the check in cfq_dispatch_requests() where a given cgroup can empty its entire queue (100 IOs or more) within its timeslice and have them sit in the dispatch queue ready for the disk driver to pick up. Moreover, this huge timeslice is not accounted for as this cgroup is charged only for the length of the intended timeslice and not the actual time taken.
The following patch fixes this by not optimizing on the single busy queue fact inside cfq_dispatch_requests. Note that this does not hurt throughput in any sense but just causes more IOs to be dispatched only when the drive is ready for them thus leading to better accounting too.

Hi Divyesh,

Thanks for the testing and noticing the issue. I also had noticed this issue.

Quoted 6 lines

Couple of points.

- In 30-rc3 jens has fixed the huge dispatch problem. Now in case of single ioq doing dispatch, in one round upto 4*quantum request can be dispatched. So that means in default configuration with single queue, maximum request on diaptch list can be 16.

I just synced my git tree and I see Jens changes. That makes this much cleaner!

Quoted 5 lines

- Secondly, in my tree, now I have modified the patches to charge for actual consumption of the slice instead of capping it to budget. In a week's time I should be able to post V2 of the patches. Please do try it out then

I have that in my tree as well and was going to send that out. No need now :)

Quoted 22 lines

Thanks
Vivek

Quoted 18 lines

Fix bug where a given ioq can run through all its requests at once.

---

2.6.26/block/cfq-iosched.c +1 -2
index 5a275a2..c0199a6 100644
@@ -848,8 +848,7 @@ static int cfq_dispatch_requests(struct request_queue *q, int force)
 		if (cfq_class_idle(cfqq))
 			max_dispatch = 1;
 
-		if (elv_ioq_nr_dispatched(cfqq->ioq) >= max_dispatch &&
-			elv_nr_busy_ioq(q->elevator) > 1)
+		if (elv_ioq_nr_dispatched(cfqq->ioq) >= max_dispatch)
 			break;
 
 		if (cfqd->sync_flight && !cfq_cfqq_sync(cfqq))