Make io_submit non-blocking

Currently, io_submit tries to execute the io requests on the same thread, which could block because of various reaons (eg. allocation of disk blocks). So, essentially, io_submit ends up being a blocking call.

With this patch, io_submit prepares all the kiocbs and then adds (kicks) them to ctx->run_list (kicked) in one go and then schedules the workqueue. The actual operations are not executed on io_submit's process context, so it can return very quickly.

This run_list is processed either on a workqueue or in response to an io_getevents call. This utilizes the existing retry infrastructure.

It uses override_creds/revert_creds to use the submitting process' credentials when processing the iocb request from the workqueue. This is required for proper support of quota and reserved block access.

Currently, we use block plugging in io_submit, since most of the IO was being done there itself. This patch moves it to aio_kick_handler and aio_run_all_iocbs, where the IO gets submitted.

All the tests were run with ext4.

I tested the patch with fio
(fio rand-rw-disk.fio --max-jobs=2 --latency-log
--bandwidth-log)

**Unpatched**
read : io=102120KB, bw=618740 B/s, iops=151 , runt=169006msec slat (usec): min=275 , max=87560 , avg=6571.88, stdev=2799.57

write: io=102680KB, bw=622133 B/s, iops=151 , runt=169006msec slat (usec): min=2 , max=196 , avg=24.66, stdev=20.35

**Patched**
read : io=102864KB, bw=504885 B/s, iops=123 , runt=208627msec slat (usec): min=0 , max=120 , avg= 1.65, stdev= 3.46

write: io=101936KB, bw=500330 B/s, iops=122 , runt=208627msec slat (usec): min=0 , max=131 , avg= 1.85, stdev= 3.27

patch. The full fio results for the "old"(unpatched) and "new"(patched) cases are attached. Results with both ramdisk (*rd*) and disk attached, and also the corresponding fio files.

Some variations I tried:

1. I tried to submit one iocb at a time (lock/unlock ctx->lock), and that had good performance for a regular disk, but when I tested with a ramdisk (to simulate very fast disk), performance was extremely bad. Submitting all the iocbs from an io_submit in one go, restored the performance (latencies+bandwidth).

2. I was earlier trying to use queue_delayed_work with 0 timeout, but that worsened the submit latencies a bit but improved bandwidth.

3. Also, I tried not using aio_queue_work from io_submit call, and instead depending on an already scheduled one or the iocbs being run when io_getevents gets called. This seemed to give improved perfomance. But does this constitute as change of api semantics?

--

index 71f613c..79801096b 100644
@@ -563,6 +563,11 @@ static int __aio_put_req(struct kioctx *ctx, struct kiocb *req)
 	req->ki_cancel = NULL;
 	req->ki_retry = NULL;
 
+	if (likely(req->submitter_cred)) {
+		put_cred(req->submitter_cred);
+		req->submitter_cred = NULL;
+	}
+
 	fput(req->ki_filp);
 	req->ki_filp = NULL;
 	really_put_req(ctx, req);

@@ -659,6 +664,7 @@ static ssize_t aio_run_iocb(struct kiocb *iocb)
 	struct kioctx	*ctx = iocb->ki_ctx;
 	ssize_t (*retry)(struct kiocb *);
 	ssize_t ret;
+	const struct cred *old_cred = NULL;
 
 	if (!(retry = iocb->ki_retry)) {
 		printk("aio_run_iocb: iocb->ki_retry = NULL\n");

@@ -703,12 +709,19 @@ static ssize_t aio_run_iocb(struct kiocb *iocb)
 		goto out;
 	}
 
+	if (iocb->submitter_cred)
+		/* setup creds */
+		old_cred = override_creds(iocb->submitter_cred);
+
 	/*
 	 * Now we are all set to call the retry method in async
 	 * context.
 	 */
 	ret = retry(iocb);
 
+	if (old_cred)
+		revert_creds(old_cred);
+
 	if (ret != -EIOCBRETRY && ret != -EIOCBQUEUED) {
 		/*
 		 * There's no easy way to restart the syscall since other AIO's

@@ -804,10 +817,14 @@ static void aio_queue_work(struct kioctx * ctx)
  */
 static inline void aio_run_all_iocbs(struct kioctx *ctx)
 {
+	struct blk_plug plug;
+
+	blk_start_plug(&plug);
 	spin_lock_irq(&ctx->ctx_lock);
 	while (__aio_run_iocbs(ctx))
 		;
 	spin_unlock_irq(&ctx->ctx_lock);
+	blk_finish_plug(&plug);
 }
 
 /*

@@ -825,13 +842,16 @@ static void aio_kick_handler(struct work_struct *work)
 	mm_segment_t oldfs = get_fs();
 	struct mm_struct *mm;
 	int requeue;
+	struct blk_plug plug;
 
 	set_fs(USER_DS);
 	use_mm(ctx->mm);
+	blk_start_plug(&plug);
 	spin_lock_irq(&ctx->ctx_lock);
 	requeue =__aio_run_iocbs(ctx);
 	mm = ctx->mm;
 	spin_unlock_irq(&ctx->ctx_lock);
+	blk_finish_plug(&plug);
  	unuse_mm(mm);
 	set_fs(oldfs);
 	/*

@@ -1506,12 +1526,14 @@ static ssize_t aio_setup_iocb(struct kiocb *kiocb, bool compat)
 
 static int io_submit_one(struct kioctx *ctx, struct iocb __user *user_iocb,
 			 struct iocb *iocb, struct kiocb_batch *batch,
-			 bool compat)
+			 bool compat, struct kiocb **req_entry)
 {
 	struct kiocb *req;
 	struct file *file;
 	ssize_t ret;
 
+	*req_entry = NULL;
+
 	/* enforce forwards compatibility on users */
 	if (unlikely(iocb->aio_reserved1 || iocb->aio_reserved2)) {
 		pr_debug("EINVAL: io_submit: reserve field set\n");

@@ -1537,6 +1559,7 @@ static int io_submit_one(struct kioctx *ctx, struct iocb __user *user_iocb,
 		fput(file);
 		return -EAGAIN;
 	}
+
 	req->ki_filp = file;
 	if (iocb->aio_flags & IOCB_FLAG_RESFD) {
 		/*

@@ -1567,38 +1590,16 @@ static int io_submit_one(struct kioctx *ctx, struct iocb __user *user_iocb,
 	req->ki_left = req->ki_nbytes = iocb->aio_nbytes;
 	req->ki_opcode = iocb->aio_lio_opcode;
 
+	req->submitter_cred = get_current_cred();
+
 	ret = aio_setup_iocb(req, compat);
 
 	if (ret)
 		goto out_put_req;
 
-	spin_lock_irq(&ctx->ctx_lock);
-	/*
-	 * We could have raced with io_destroy() and are currently holding a
-	 * reference to ctx which should be destroyed. We cannot submit IO
-	 * since ctx gets freed as soon as io_submit() puts its reference.  The
-	 * check here is reliable: io_destroy() sets ctx->dead before waiting
-	 * for outstanding IO and the barrier between these two is realized by
-	 * unlock of mm->ioctx_lock and lock of ctx->ctx_lock.  Analogously we
-	 * increment ctx->reqs_active before checking for ctx->dead and the
-	 * barrier is realized by unlock and lock of ctx->ctx_lock. Thus if we
-	 * don't see ctx->dead set here, io_destroy() waits for our IO to
-	 * finish.
-	 */
-	if (ctx->dead) {
-		spin_unlock_irq(&ctx->ctx_lock);
-		ret = -EINVAL;
-		goto out_put_req;
-	}
-	aio_run_iocb(req);
-	if (!list_empty(&ctx->run_list)) {
-		/* drain the run list */
-		while (__aio_run_iocbs(ctx))
-			;
-	}
-	spin_unlock_irq(&ctx->ctx_lock);
-
 	aio_put_req(req);	/* drop extra ref to req */
+
+	*req_entry = req;
 	return 0;
 
 out_put_req:

@@ -1613,8 +1614,10 @@ long do_io_submit(aio_context_t ctx_id, long nr,
 	struct kioctx *ctx;
 	long ret = 0;
 	int i = 0;
-	struct blk_plug plug;
 	struct kiocb_batch batch;
+	struct kiocb **req_arr = NULL;
+	int nr_submitted = 0;
+	int req_arr_cnt = 0;
 
 	if (unlikely(nr < 0))
 		return -EINVAL;

@@ -1632,8 +1635,8 @@ long do_io_submit(aio_context_t ctx_id, long nr,
 	}
 
 	kiocb_batch_init(&batch, nr);
-
-	blk_start_plug(&plug);
+	req_arr = kmalloc(sizeof(struct kiocb *) * nr, GFP_KERNEL);
+	memset(req_arr, 0, sizeof(req_arr));
 
 	/*
 	 * AKPM: should this return a partial result if some of the IOs were

@@ -1653,15 +1656,51 @@ long do_io_submit(aio_context_t ctx_id, long nr,
 			break;
 		}
 
-		ret = io_submit_one(ctx, user_iocb, &tmp, &batch, compat);
+		ret = io_submit_one(ctx, user_iocb, &tmp, &batch, compat,
+				&req_arr[i]);
 		if (ret)
 			break;
+		req_arr_cnt++;
 	}
-	blk_finish_plug(&plug);
 
+	spin_lock_irq(&ctx->ctx_lock);
+	/*
+	 * We could have raced with io_destroy() and are currently holding a
+	 * reference to ctx which should be destroyed. We cannot submit IO
+	 * since ctx gets freed as soon as io_submit() puts its reference.  The
+	 * check here is reliable: io_destroy() sets ctx->dead before waiting
+	 * for outstanding IO and the barrier between these two is realized by
+	 * unlock of mm->ioctx_lock and lock of ctx->ctx_lock.  Analogously we
+	 * increment ctx->reqs_active before checking for ctx->dead and the
+	 * barrier is realized by unlock and lock of ctx->ctx_lock. Thus if we
+	 * don't see ctx->dead set here, io_destroy() waits for our IO to
+	 * finish.
+	 */
+	if (ctx->dead) {
+		spin_unlock_irq(&ctx->ctx_lock);
+		for (i = 0; i < req_arr_cnt; i++)
+			/* drop i/o ref to the req */
+			__aio_put_req(ctx, req_arr[i]);
+
+		ret = -EINVAL;
+		goto out;
+	}
+
+	for (i = 0; i < req_arr_cnt; i++) {
+		struct kiocb *req = req_arr[i];
+		if (likely(!kiocbTryKick(req)))
+			__queue_kicked_iocb(req);
+		nr_submitted++;
+	}
+	if (likely(nr_submitted > 0))
+		aio_queue_work(ctx);
+	spin_unlock_irq(&ctx->ctx_lock);
+
+out:
 	kiocb_batch_free(ctx, &batch);
+	kfree(req_arr);
 	put_ioctx(ctx);
-	return i ? i : ret;
+	return nr_submitted ? nr_submitted : ret;
 }
 
 /* sys_io_submit:

index b1a520e..bcd6a5e 100644
@@ -124,6 +124,8 @@ struct kiocb {
 	 * this is the underlying eventfd context to deliver events to.
 	 */
 	struct eventfd_ctx	*ki_eventfd;
+
+	const struct cred	*submitter_cred;
 };
 
 #define is_sync_kiocb(iocb)	((iocb)->ki_key == KIOCB_SYNC_KEY)

Ankit Jain
SUSE Labs

random_rw: (g=0): rw=randrw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=32 fio-2.0.8-9-gfb9f0
Starting 1 process

random_rw: (groupid=0, jobs=1): err= 0: pid=2021: Tue Jul 24 15:37:25 2012
read : io=102864KB, bw=504885 B/s, iops=123 , runt=208627msec

    slat (usec): min=0 , max=120 , avg= 1.65, stdev= 3.46
    clat (msec): min=29 , max=619 , avg=134.15, stdev=53.04
     lat (msec): min=29 , max=619 , avg=134.15, stdev=53.04
    clat percentiles (msec):
     |  1.00th=[   63],  5.00th=[   78], 10.00th=[   87], 20.00th=[   98],
     | 30.00th=[  106], 40.00th=[  114], 50.00th=[  123], 60.00th=[  130],
     | 70.00th=[  143], 80.00th=[  163], 90.00th=[  198], 95.00th=[  231],
     | 99.00th=[  302], 99.50th=[  363], 99.90th=[  619], 99.95th=[  619],
     | 99.99th=[  619]
    bw (KB/s)  : min=  153, max=  761, per=100.00%, avg=497.88, stdev=128.35
  write: io=101936KB, bw=500330 B/s, iops=122 , runt=208627msec
    slat (usec): min=0 , max=131 , avg= 1.85, stdev= 3.27
    clat (usec): min=97 , max=619552 , avg=126590.22, stdev=50338.80
     lat (usec): min=108 , max=619553 , avg=126592.26, stdev=50338.83
    clat percentiles (msec):
     |  1.00th=[   59],  5.00th=[   72], 10.00th=[   80], 20.00th=[   91],
     | 30.00th=[  100], 40.00th=[  108], 50.00th=[  116], 60.00th=[  125],
     | 70.00th=[  137], 80.00th=[  155], 90.00th=[  188], 95.00th=[  225],
     | 99.00th=[  293], 99.50th=[  334], 99.90th=[  578], 99.95th=[  619],
     | 99.99th=[  619]
    bw (KB/s)  : min=  127, max=  842, per=100.00%, avg=493.55, stdev=140.80
    lat (usec) : 100=0.01%, 250=0.01%
    lat (msec) : 50=0.25%, 100=26.26%, 250=70.61%, 500=2.69%, 750=0.19%
  cpu          : usr=0.07%, sys=1.25%, ctx=27318, majf=0, minf=24
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=99.9%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
     issued    : total=r=25716/w=25484/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=102864KB, aggrb=493KB/s, minb=493KB/s, maxb=493KB/s, mint=208627msec, maxt=208627msec
WRITE: io=101936KB, aggrb=488KB/s, minb=488KB/s, maxb=488KB/s, mint=208627msec, maxt=208627msec

Disk stats (read/write):
sda: ios=25681/22647, merge=0/70, ticks=204770/98179832, in_queue=99369560, util=98.97%
fio rand-rw-disk-2.fio --output=/home/radical/src/play/ios-test/new-logs/ext4-disk-2-b73147d.log --max-jobs=2 --latency-log --bandwidth-log b73147d remove unused ki_colln

random_rw: (g=0): rw=randrw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=32 fio-2.0.8-9-gfb9f0
Starting 1 process
random_rw: Laying out IO file(s) (1 file(s) / 200MB)

random_rw: (groupid=0, jobs=1): err= 0: pid=2011: Tue Jul 24 01:00:02 2012
read : io=102120KB, bw=618740 B/s, iops=151 , runt=169006msec

    slat (usec): min=275 , max=87560 , avg=6571.88, stdev=2799.57
    clat (usec): min=294 , max=214745 , avg=102349.31, stdev=21957.26
     lat (msec): min=9 , max=225 , avg=108.92, stdev=22.21
    clat percentiles (msec):
     |  1.00th=[   54],  5.00th=[   68], 10.00th=[   76], 20.00th=[   84],
     | 30.00th=[   91], 40.00th=[   96], 50.00th=[  102], 60.00th=[  108],
     | 70.00th=[  114], 80.00th=[  121], 90.00th=[  131], 95.00th=[  141],
     | 99.00th=[  157], 99.50th=[  161], 99.90th=[  182], 99.95th=[  198],
     | 99.99th=[  210]
    bw (KB/s)  : min=  474, max=  817, per=99.90%, avg=603.38, stdev=47.87
  write: io=102680KB, bw=622133 B/s, iops=151 , runt=169006msec
    slat (usec): min=2 , max=196 , avg=24.66, stdev=20.35
    clat (usec): min=42 , max=221533 , avg=102260.25, stdev=21825.38
     lat (usec): min=85 , max=221542 , avg=102285.38, stdev=21825.25
    clat percentiles (msec):
     |  1.00th=[   53],  5.00th=[   69], 10.00th=[   76], 20.00th=[   85],
     | 30.00th=[   91], 40.00th=[   96], 50.00th=[  102], 60.00th=[  108],
     | 70.00th=[  114], 80.00th=[  121], 90.00th=[  131], 95.00th=[  139],
     | 99.00th=[  157], 99.50th=[  163], 99.90th=[  184], 99.95th=[  206],
     | 99.99th=[  215]
    bw (KB/s)  : min=  318, max=  936, per=99.86%, avg=606.14, stdev=100.63
    lat (usec) : 50=0.01%, 250=0.01%, 500=0.01%
    lat (msec) : 10=0.01%, 20=0.02%, 50=0.60%, 100=46.62%, 250=52.75%
  cpu          : usr=0.41%, sys=1.58%, ctx=27474, majf=0, minf=22
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=99.9%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
     issued    : total=r=25530/w=25670/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=102120KB, aggrb=604KB/s, minb=604KB/s, maxb=604KB/s, mint=169006msec, maxt=169006msec
WRITE: io=102680KB, aggrb=607KB/s, minb=607KB/s, maxb=607KB/s, mint=169006msec, maxt=169006msec

Disk stats (read/write):
sda: ios=25533/4, merge=23/18, ticks=164781/112, in_queue=164823, util=97.51%

random_rw: (g=0): rw=randrw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=32 fio-2.0.8-9-gfb9f0
Starting 1 process
random_rw: Laying out IO file(s) (1 file(s) / 1700MB)

random_rw: (groupid=0, jobs=1): err= 0: pid=2002: Tue Jul 24 15:31:47 2012
read : io=870504KB, bw=558373KB/s, iops=139593 , runt= 1559msec

    slat (usec): min=0 , max=32 , avg= 0.38, stdev= 0.52
    clat (usec): min=62 , max=597 , avg=114.05, stdev=18.96
     lat (usec): min=63 , max=599 , avg=114.49, stdev=19.03
    clat percentiles (usec):
     |  1.00th=[  103],  5.00th=[  105], 10.00th=[  106], 20.00th=[  107],
     | 30.00th=[  108], 40.00th=[  108], 50.00th=[  109], 60.00th=[  110],
     | 70.00th=[  115], 80.00th=[  123], 90.00th=[  126], 95.00th=[  129],
     | 99.00th=[  145], 99.50th=[  151], 99.90th=[  438], 99.95th=[  462],
     | 99.99th=[  580]
    bw (KB/s)  : min=550016, max=572568, per=100.00%, avg=560584.00, stdev=11342.49
  write: io=870296KB, bw=558240KB/s, iops=139559 , runt=  1559msec
    slat (usec): min=0 , max=65 , avg= 0.42, stdev= 0.53
    clat (usec): min=62 , max=595 , avg=113.45, stdev=18.91
     lat (usec): min=63 , max=597 , avg=113.93, stdev=18.99
    clat percentiles (usec):
     |  1.00th=[  103],  5.00th=[  104], 10.00th=[  105], 20.00th=[  106],
     | 30.00th=[  107], 40.00th=[  108], 50.00th=[  108], 60.00th=[  110],
     | 70.00th=[  115], 80.00th=[  122], 90.00th=[  125], 95.00th=[  129],
     | 99.00th=[  145], 99.50th=[  149], 99.90th=[  438], 99.95th=[  462],
     | 99.99th=[  564]
    bw (KB/s)  : min=547456, max=575144, per=100.00%, avg=559728.00, stdev=14109.21
    lat (usec) : 100=0.02%, 250=99.72%, 500=0.24%, 750=0.03%
  cpu          : usr=18.73%, sys=80.76%, ctx=175, majf=0, minf=22
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
     issued    : total=r=217626/w=217574/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=870504KB, aggrb=558373KB/s, minb=558373KB/s, maxb=558373KB/s, mint=1559msec, maxt=1559msec
WRITE: io=870296KB, aggrb=558239KB/s, minb=558239KB/s, maxb=558239KB/s, mint=1559msec, maxt=1559msec

Disk stats (read/write):
ram0: ios=0/0, merge=0/0, ticks=0/0, in_queue=0, util=0.00%
fio rand-rw-rd.fio --output=/home/radical/src/play/ios-test/new-logs/ext4-rd-b73147d.log --max-jobs=2 --latency-log --bandwidth-log b73147d remove unused ki_colln

random_rw: (g=0): rw=randrw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=32 fio-2.0.8-9-gfb9f0
Starting 1 process
random_rw: Laying out IO file(s) (1 file(s) / 1700MB)

random_rw: (groupid=0, jobs=1): err= 0: pid=1999: Tue Jul 24 00:55:56 2012
read : io=869872KB, bw=489517KB/s, iops=122379 , runt= 1777msec

    slat (usec): min=2 , max=78 , avg= 3.46, stdev= 0.87
    clat (usec): min=16 , max=604 , avg=126.83, stdev=16.35
     lat (usec): min=19 , max=617 , avg=130.40, stdev=16.74
    clat percentiles (usec):
     |  1.00th=[  119],  5.00th=[  121], 10.00th=[  122], 20.00th=[  123],
     | 30.00th=[  124], 40.00th=[  124], 50.00th=[  125], 60.00th=[  126],
     | 70.00th=[  127], 80.00th=[  129], 90.00th=[  133], 95.00th=[  135],
     | 99.00th=[  149], 99.50th=[  155], 99.90th=[  490], 99.95th=[  524],
     | 99.99th=[  548]
    bw (KB/s)  : min=487648, max=493224, per=100.00%, avg=490160.00, stdev=2828.69
  write: io=870928KB, bw=490111KB/s, iops=122527 , runt=  1777msec
    slat (usec): min=2 , max=171 , avg= 2.78, stdev= 0.90
    clat (usec): min=23 , max=606 , avg=126.77, stdev=16.22
     lat (usec): min=26 , max=616 , avg=129.65, stdev=16.57
    clat percentiles (usec):
     |  1.00th=[  118],  5.00th=[  120], 10.00th=[  121], 20.00th=[  123],
     | 30.00th=[  124], 40.00th=[  124], 50.00th=[  125], 60.00th=[  126],
     | 70.00th=[  127], 80.00th=[  129], 90.00th=[  133], 95.00th=[  135],
     | 99.00th=[  149], 99.50th=[  155], 99.90th=[  490], 99.95th=[  516],
     | 99.99th=[  548]
    bw (KB/s)  : min=484072, max=496464, per=100.00%, avg=490920.00, stdev=6298.07
    lat (usec) : 20=0.01%, 50=0.01%, 100=0.01%, 250=99.80%, 500=0.11%
    lat (usec) : 750=0.08%
  cpu          : usr=24.21%, sys=75.34%, ctx=185, majf=0, minf=22
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
     issued    : total=r=217468/w=217732/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=869872KB, aggrb=489517KB/s, minb=489517KB/s, maxb=489517KB/s, mint=1777msec, maxt=1777msec
WRITE: io=870928KB, aggrb=490111KB/s, minb=490111KB/s, maxb=490111KB/s, mint=1777msec, maxt=1777msec

Disk stats (read/write):
ram0: ios=0/0, merge=0/0, ticks=0/0, in_queue=0, util=0.00%

[random_rw]
rw=randrw
size=200m
directory=/misc/rd
ioengine=libaio
iodepth=32

[random_rw]
rw=randrw
size=1700m
directory=/mnt/rd
ioengine=libaio
iodepth=32

Hi Ankit,

Ideally filesystem should take care of it e.g. by deferring such time consuming allocations and return -EIOCBQUEUED immediately. But have you seen such cases?

With lots of application threads firing continuous IOs, workqueue threads might become bottleneck and you might have to eventually develop a priority scheduling. This workqueue was originally designed for IO retries which is an error path, now consumers of workqueue might easily increase by 100x.

I once have observed latency issues with aio_queue_work with lesser number of threads when I was trying to resubmit IOs on a ramdisk, as this function introduces a mandatory delay if nobody is waiting on this iocb. The latencies were high but with large number of threads, effect was not prominent.

--
Rajat Sharma

--
To unsubscribe, send a message with 'unsubscribe linux-aio' in the body to [email protected]. For more info on Linux AIO,
see: http://www.kvack.org/aio/
Don't email: <a href=mailto:"[email protected]">[email protected]</a>

Oh, it happens all the time if you are using AIO. If the file system needs to read or write any metadata block, AIO can become distinctly non-"A". The workaround that I've chosen is to create a way to cache the information needed for the bmap() operation, triggered via an ioctl() issued at open time, so that this is not an issue, but that only works if the file is pre-allocated, and there is no need to do any block allocations.

It's all very well and good to say, "the file system should handle it", but that just pushes the problem onto the file system. And since you need to potentially issue block I/O requests, which you can't do from an interrupt context (i.e., a block I/O completion handler), you really do need to create a workqueue in order to make things work.

If you do it in the fs/direct_io.c layer, at least that way you can solve the problem once for all file systems....

Yes, you definitely need to throttle how many outstanding AIO's can be allowed to be outstanding, either globally, or on a per-superblock/process/user/cgroup basis, and return EAGAIN if there are too many outstanding requests.

Speaking of cgroups, one of the other challenges with running the AIO out of a workqueue is trying to respect cgroup restrictions. In particular, the io-throttle cgroup (which is needed to provide Proportional I/O support), but also the memory cgroup.

All of these complications is why I decided to simply go with the "pin metadata" approach, since I didn't need to worry (at least initially) with the allocating write case. (These patches to ext4 haven't yet been published upstream, mainly because they need a lot of cleanup work and I haven't had time to do that cleanup; my intention is to get the "big extents" patchset upstream, though.)

- Ted

--
To unsubscribe, send a message with 'unsubscribe linux-aio' in the body to [email protected]. For more info on Linux AIO,
see: http://www.kvack.org/aio/
Don't email: <a href=mailto:"[email protected]">[email protected]</a>

Hmmm, I had to check the numbers twice - that's only 600KB/s.

Perhaps you need to test on something more than a single piece of spinning rust. Optimising AIO for SSD rates (say 100k 4k write IOPS) is probably more relevant to the majority of AIO users....

So you made ext4 20% slower at random 4k writes with worst case latencies only improving by about 30%. That, I think, is a non-starter....

Also, you added a memory allocation in the io submit code. Worse case latency will still be effectively undefined - what happens to latencies if you generate memory pressure while the test is running?

FWIW, if you are going to change generic code, you need to present results for other filesystems as well (xfs, btrfs are typical), as they may not have the same problems as ext4 or react the same way to your change. The result might simply be "it is 20% slower"....

Cheers,

Dave.

Dave Chinner
[email protected]

And most importantly block devices, as they are one of the biggest use cases of AIO. With an almost no-op get_blocks callback I can't see how this change would provide any gain there.

--
To unsubscribe, send a message with 'unsubscribe linux-aio' in the body to [email protected]. For more info on Linux AIO,
see: http://www.kvack.org/aio/
Don't email: <a href=mailto:"[email protected]">[email protected]</a>

Historically we'd often see submission stuck waiting for requests. Tasks often try to submit way more aio than the block layer is happy to have in flight.

Dunno if that's still a problem these days.

- z

--
To unsubscribe, send a message with 'unsubscribe linux-aio' in the body to [email protected]. For more info on Linux AIO,
see: http://www.kvack.org/aio/
Don't email: <a href=mailto:"[email protected]">[email protected]</a>

I tried running fio against a block device, disk partition and a ramdisk. I ran this with a single job though. For disks, bandwidth seems to stay nearly the same with submit latencies getting better. And for ramdisk, bandwidth also sees improvement. I should probably be doing better tests, any suggestions on what or how I can test? For block devices, if the patch doesn't make it worse, at least, then that should be good enough?

------ disk -------
submit latencies(usec)
B/w iops runtime min max avg std dev
Read :

Old:  417335 B/s  101   252668msec     4  231  40.03  21.66
New:  419099 B/s  102   251282msec     0  169   8.20   6.95

Write:

Old:  412667 B/s  100   252668msec     3  272  47.65  24.58
New:  415481 B/s  101   251282msec     0  134   7.95   7.11

------ ramdisk -------
submit latencies(usec)
B/w iops runtime min max avg std dev
Read:

Old:  708235KB/s  177058   1227msec     1   51   1.61  0.72
New:  822157KB/s  205539   1059msec     0   14   0.38  0.52

Write:

Old:  710510KB/s  177627   1227msec     2   46   2.33  0.81
New:  821658KB/s  205414   1059msec     0   24   0.40  0.53

Full fio results are attached, and I dropped cache before running the tests.

Ankit Jain
SUSE Labs

random_rw: (g=0): rw=randrw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=32 fio-2.0.8-9-gfb9f0
Starting 1 process

random_rw: (groupid=0, jobs=1): err= 0: pid=2109: Thu Jul 26 17:14:55 2012
read : io=102844KB, bw=419099 B/s, iops=102 , runt=251282msec

    slat (usec): min=0 , max=169 , avg= 8.20, stdev= 6.95
    clat (usec): min=335 , max=3356.7K, avg=255054.47, stdev=158234.29
     lat (usec): min=342 , max=3356.7K, avg=255063.32, stdev=158234.33
    clat percentiles (msec):
     |  1.00th=[    8],  5.00th=[   50], 10.00th=[   84], 20.00th=[  130],
     | 30.00th=[  169], 40.00th=[  204], 50.00th=[  237], 60.00th=[  269],
     | 70.00th=[  306], 80.00th=[  351], 90.00th=[  437], 95.00th=[  529],
     | 99.00th=[  791], 99.50th=[  914], 99.90th=[ 1237], 99.95th=[ 1483],
     | 99.99th=[ 2073]
    bw (KB/s)  : min=  111, max=  646, per=100.00%, avg=410.90, stdev=84.69
  write: io=101956KB, bw=415481 B/s, iops=101 , runt=251282msec
    slat (usec): min=0 , max=134 , avg= 7.95, stdev= 7.11
    clat (usec): min=189 , max=928209 , avg=58138.79, stdev=76776.72
     lat (usec): min=194 , max=928221 , avg=58147.37, stdev=76776.86
    clat percentiles (usec):
     |  1.00th=[  498],  5.00th=[  828], 10.00th=[ 1624], 20.00th=[ 4960],
     | 30.00th=[12352], 40.00th=[22144], 50.00th=[33536], 60.00th=[46848],
     | 70.00th=[63232], 80.00th=[90624], 90.00th=[148480], 95.00th=[203776],
     | 99.00th=[370688], 99.50th=[460800], 99.90th=[643072], 99.95th=[716800],
     | 99.99th=[831488]
    bw (KB/s)  : min=   31, max=  864, per=100.00%, avg=408.11, stdev=111.34
    lat (usec) : 250=0.02%, 500=0.54%, 750=1.27%, 1000=1.51%
    lat (msec) : 2=2.39%, 4=3.60%, 10=4.63%, 20=5.96%, 50=13.51%
    lat (msec) : 100=14.18%, 250=27.95%, 500=21.04%, 750=2.78%, 1000=0.46%
    lat (msec) : 2000=0.15%, >=2000=0.01%
  cpu          : usr=0.51%, sys=1.52%, ctx=52135, majf=0, minf=23
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=99.9%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
     issued    : total=r=25711/w=25489/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=102844KB, aggrb=409KB/s, minb=409KB/s, maxb=409KB/s, mint=251282msec, maxt=251282msec
WRITE: io=101956KB, aggrb=405KB/s, minb=405KB/s, maxb=405KB/s, mint=251282msec, maxt=251282msec
fio rand-rw-disk-2-raw.fio --output=/home/radical/src/play/ios-test/logs-with-drop-cache/ad6d29a/raw-disk-2-raw-ad6d29a.log --max-jobs=2 --latency-log --bandwidth-log ad6d29a sent upstream

random_rw: (g=0): rw=randrw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=32 fio-2.0.8-9-gfb9f0
Starting 1 process

random_rw: (groupid=0, jobs=1): err= 0: pid=2117: Thu Jul 26 17:53:41 2012
read : io=102976KB, bw=417335 B/s, iops=101 , runt=252668msec

    slat (usec): min=4 , max=231 , avg=40.03, stdev=21.66
    clat (usec): min=236 , max=4075.6K, avg=254175.39, stdev=158853.64
     lat (usec): min=339 , max=4075.7K, avg=254216.22, stdev=158853.33
    clat percentiles (msec):
     |  1.00th=[    7],  5.00th=[   51], 10.00th=[   85], 20.00th=[  131],
     | 30.00th=[  169], 40.00th=[  204], 50.00th=[  237], 60.00th=[  269],
     | 70.00th=[  306], 80.00th=[  351], 90.00th=[  433], 95.00th=[  529],
     | 99.00th=[  766], 99.50th=[  906], 99.90th=[ 1270], 99.95th=[ 1500],
     | 99.99th=[ 3261]
    bw (KB/s)  : min=   83, max=  624, per=100.00%, avg=409.49, stdev=91.68
  write: io=101824KB, bw=412667 B/s, iops=100 , runt=252668msec
    slat (usec): min=3 , max=272 , avg=47.65, stdev=24.58
    clat (usec): min=139 , max=1248.8K, avg=60442.70, stdev=82817.74
     lat (usec): min=198 , max=1248.9K, avg=60491.15, stdev=82817.11
    clat percentiles (usec):
     |  1.00th=[  438],  5.00th=[  812], 10.00th=[ 1704], 20.00th=[ 5280],
     | 30.00th=[13376], 40.00th=[23168], 50.00th=[34560], 60.00th=[47872],
     | 70.00th=[66048], 80.00th=[91648], 90.00th=[150528], 95.00th=[209920],
     | 99.00th=[403456], 99.50th=[505856], 99.90th=[798720], 99.95th=[897024],
     | 99.99th=[1073152]
    bw (KB/s)  : min=    7, max=  808, per=100.00%, avg=405.72, stdev=121.51
    lat (usec) : 250=0.03%, 500=0.79%, 750=1.35%, 1000=1.21%
    lat (msec) : 2=2.29%, 4=3.59%, 10=4.36%, 20=5.90%, 50=13.45%
    lat (msec) : 100=14.37%, 250=28.08%, 500=21.27%, 750=2.73%, 1000=0.42%
    lat (msec) : 2000=0.17%, >=2000=0.01%
  cpu          : usr=0.54%, sys=1.44%, ctx=52211, majf=0, minf=23
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=99.9%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
     issued    : total=r=25744/w=25456/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=102976KB, aggrb=407KB/s, minb=407KB/s, maxb=407KB/s, mint=252668msec, maxt=252668msec
WRITE: io=101824KB, aggrb=402KB/s, minb=402KB/s, maxb=402KB/s, mint=252668msec, maxt=252668msec
fio rand-rw-disk-2-raw.fio --output=/home/radical/src/play/ios-test/logs-with-drop-cache/918227b/raw-disk-2-raw-918227b.log --max-jobs=2 --latency-log --bandwidth-log 918227b Merge tag 'fbdev-fixes-for-3.5-2' of git://github.com/schandinat/linux-2.6

random_rw: (g=0): rw=randrw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=32 fio-2.0.8-9-gfb9f0
Starting 1 process

random_rw: (groupid=0, jobs=1): err= 0: pid=2179: Thu Jul 26 17:14:57 2012
read : io=870664KB, bw=822157KB/s, iops=205539 , runt= 1059msec

    slat (usec): min=0 , max=14 , avg= 0.38, stdev= 0.52
    clat (usec): min=44 , max=294 , avg=76.67, stdev=12.77
     lat (usec): min=45 , max=296 , avg=77.10, stdev=12.84
    clat percentiles (usec):
     |  1.00th=[   70],  5.00th=[   71], 10.00th=[   72], 20.00th=[   73],
     | 30.00th=[   74], 40.00th=[   74], 50.00th=[   75], 60.00th=[   76],
     | 70.00th=[   76], 80.00th=[   77], 90.00th=[   79], 95.00th=[   86],
     | 99.00th=[   97], 99.50th=[  107], 99.90th=[  255], 99.95th=[  266],
     | 99.99th=[  286]
    bw (KB/s)  : min=819368, max=826656, per=100.00%, avg=823012.00, stdev=5153.39
  write: io=870136KB, bw=821658KB/s, iops=205414 , runt=  1059msec
    slat (usec): min=0 , max=24 , avg= 0.40, stdev= 0.53
    clat (usec): min=42 , max=292 , avg=77.34, stdev=12.81
     lat (usec): min=43 , max=293 , avg=77.79, stdev=12.89
    clat percentiles (usec):
     |  1.00th=[   70],  5.00th=[   72], 10.00th=[   73], 20.00th=[   74],
     | 30.00th=[   74], 40.00th=[   75], 50.00th=[   76], 60.00th=[   76],
     | 70.00th=[   77], 80.00th=[   78], 90.00th=[   80], 95.00th=[   87],
     | 99.00th=[   98], 99.50th=[  107], 99.90th=[  262], 99.95th=[  270],
     | 99.99th=[  286]
    bw (KB/s)  : min=819368, max=825328, per=100.00%, avg=822348.00, stdev=4214.36
    lat (usec) : 50=0.01%, 100=99.35%, 250=0.41%, 500=0.24%
  cpu          : usr=24.76%, sys=74.76%, ctx=114, majf=0, minf=24
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
     issued    : total=r=217666/w=217534/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=870664KB, aggrb=822156KB/s, minb=822156KB/s, maxb=822156KB/s, mint=1059msec, maxt=1059msec
WRITE: io=870136KB, aggrb=821658KB/s, minb=821658KB/s, maxb=821658KB/s, mint=1059msec, maxt=1059msec

Disk stats (read/write):
ram0: ios=0/0, merge=0/0, ticks=0/0, in_queue=0, util=0.00%
fio rand-rw-rd-raw.fio --output=/home/radical/src/play/ios-test/logs-with-drop-cache/ad6d29a/raw-rd-raw-ad6d29a.log --max-jobs=2 --latency-log --bandwidth-log ad6d29a sent upstream

random_rw: (g=0): rw=randrw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=32 fio-2.0.8-9-gfb9f0
Starting 1 process

random_rw: (groupid=0, jobs=1): err= 0: pid=2192: Thu Jul 26 17:53:52 2012
read : io=869004KB, bw=708235KB/s, iops=177058 , runt= 1227msec

    slat (usec): min=1 , max=51 , avg= 1.61, stdev= 0.72
    clat (usec): min=17 , max=425 , avg=87.51, stdev=17.51
     lat (usec): min=19 , max=432 , avg=89.20, stdev=17.83
    clat percentiles (usec):
     |  1.00th=[   79],  5.00th=[   81], 10.00th=[   82], 20.00th=[   83],
     | 30.00th=[   84], 40.00th=[   85], 50.00th=[   86], 60.00th=[   86],
     | 70.00th=[   87], 80.00th=[   89], 90.00th=[   91], 95.00th=[   98],
     | 99.00th=[  111], 99.50th=[  118], 99.90th=[  374], 99.95th=[  390],
     | 99.99th=[  406]
    bw (KB/s)  : min=707912, max=711328, per=100.00%, avg=709620.00, stdev=2415.48
  write: io=871796KB, bw=710510KB/s, iops=177627 , runt=  1227msec
    slat (usec): min=2 , max=46 , avg= 2.33, stdev= 0.81
    clat (usec): min=14 , max=425 , avg=87.62, stdev=17.61
     lat (usec): min=16 , max=435 , avg=90.05, stdev=18.07
    clat percentiles (usec):
     |  1.00th=[   79],  5.00th=[   81], 10.00th=[   82], 20.00th=[   83],
     | 30.00th=[   84], 40.00th=[   85], 50.00th=[   86], 60.00th=[   87],
     | 70.00th=[   87], 80.00th=[   89], 90.00th=[   91], 95.00th=[   98],
     | 99.00th=[  111], 99.50th=[  118], 99.90th=[  378], 99.95th=[  390],
     | 99.99th=[  406]
    bw (KB/s)  : min=709360, max=717872, per=100.00%, avg=713616.00, stdev=6018.89
    lat (usec) : 20=0.01%, 50=0.01%, 100=95.59%, 250=4.04%, 500=0.36%
  cpu          : usr=31.57%, sys=67.94%, ctx=125, majf=0, minf=24
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
     issued    : total=r=217251/w=217949/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=869004KB, aggrb=708234KB/s, minb=708234KB/s, maxb=708234KB/s, mint=1227msec, maxt=1227msec
WRITE: io=871796KB, aggrb=710510KB/s, minb=710510KB/s, maxb=710510KB/s, mint=1227msec, maxt=1227msec

Disk stats (read/write):
ram0: ios=0/0, merge=0/0, ticks=0/0, in_queue=0, util=0.00%
fio rand-rw-rd-raw.fio --output=/home/radical/src/play/ios-test/logs-with-drop-cache/918227b/raw-rd-raw-918227b.log --max-jobs=2 --latency-log --bandwidth-log 918227b Merge tag 'fbdev-fixes-for-3.5-2' of git://github.com/schandinat/linux-2.6

Well, is the test actually *doing* anything with these IOs?

Calling io_submit() and then immediately waiting for completion is the best case for offloading work to threads inside io_submit(). It's likely that the kernel thread will then get a chance to run and submit the IO and you won't have lost much time since the io_submit() queued the work.

So try saturating the cpus while the tests are running. Give the kernel aio submission threads some competition for run time on the cpus.

Maybe with the cpuio bits of fio? I haven't used that myself but the description of it in its README/HOWTO files are using all the right words :).

- z

[snip]

I tested with a ramdisk to "simulate" a fast disk and had attached the results. I'll try to get hold of a SSD and then test with that also. Meanwhile, I ran the tests again, with ext3/ext4/xfs/btrfs and not sure what I had screwed up when I did that previous test, but the numbers look proper (as I was getting in my earlier testing) now:

For disk, I tested on a separate partition formatted with the fs, and then run fio on it, with 1 job. Here "Old" is 3.5-rc7 (918227b).

------ disk -------
====== ext3 ======
submit latencies(usec)
B/w iops runtime min max avg std dev

ext3-read :

Old:  453352 B/s  110   231050msec    3  283048  170.28 5183.28
New:  451298 B/s  110   232555msec    0     444    8.18    7.95
ext3-write:
Old:  454309 B/s  110   231050msec    2  304614  232.72 6549.82
New:  450488 B/s  109   232555msec    0     233    7.94    7.23

====== ext4 ======
ext4-read :

Old:  459824 B/s  112   228635msec    2  260051  121.40 3569.78
New:  422700 B/s  103   247097msec    0     165    8.18    7.87
ext4-write:
Old:  457424 B/s  111   228635msec    3  312958  166.75 4616.58
New:  426015 B/s  104   247097msec    0     169    8.00    8.08

====== xfs ======
xfs-read :

Old:  467330 B/s  114   224516msec    3     272   46.45   25.35
New:  417049 B/s  101   252262msec    0     165    7.84    7.87
xfs-write:
Old:  466746 B/s  113   224516msec    3     265   52.52   28.13
New:  414289 B/s  101   252262msec    0     143    7.58    7.66

====== btrfs ======
btrfs-read :

Old:  1027.1KB/s  256    99918msec    5   84457   62.15  527.24
New:  1054.5KB/s  263    97542msec    0     121    9.72    7.05
btrfs-write:
Old:  1021.8KB/s  255    99918msec    10 139473   84.96  899.99
New:  1045.2KB/s  261    97542msec    0     248    9.55    7.02

These are the figures with a ramdisk:

------ ramdisk -------
====== ext3 ======
submit latencies (usec)
B/w iops runtime min max avg std dev

ext3-read :

Old:  430312KB/s  107577     2026msec    1  7072   3.85 15.17
New:  491251KB/s  122812     1772msec    0    22   0.39  0.52
ext3-write:
Old:  428918KB/s  107229     2026msec    2    61   3.46  0.85
New:  491142KB/s  122785     1772msec    0    62   0.43  0.55

====== ext4 ======
ext4-read :

Old:  466132KB/s  116532     1869msec    2   133   3.66  1.04
New:  542337KB/s  135584     1607msec    0    67   0.40  0.54
ext4-write:
Old:  465276KB/s  116318     1869msec    2   127   2.96  0.94
New:  540923KB/s  135230     1607msec    0    73   0.43  0.55

====== xfs ======
xfs-read :

Old:  485556KB/s  121389     1794msec    2   160   3.58  1.22
New:  581477KB/s  145369     1495msec    0    19   0.39  0.51
xfs-write:
Old:  484789KB/s  121197     1794msec    1    87   2.68  0.99
New:  582938KB/s  145734     1495msec    0    56   0.43  0.55

====== btrfs ======
I had trouble with btrfs on a ramdisk though, it complained about space during preallocation. This was with a 4gig ramdisk and fio set to write 1700mb file, so these numbers are from that partial run. Btrfs ran fine on a regular disk though.

btrfs-read :

Old:  107519KB/s  26882     2579msec    13  1492  17.03  9.23
New:  109878KB/s  27469     4665msec    0     29   0.45  0.55
btrfs-write:
Old:  108047KB/s  27020     2579msec    1  64963  17.21 823.88
New:  109413KB/s  27357     4665msec    0     32   0.48   0.56

Also, I dropped caches ("echo 3 > /proc/vm/sys/drop_cache") and sync'ed before running each test. All the fio log files are attached.

Any suggestions on how I might test this better, other than the SSD suggestion ofcourse.

[snip]

I'll try to fix this.

Ankit Jain
SUSE Labs

Yup, sadly that's how its built. A blocking submission phase that returns once completion doesn't need the submitters's context. It happens to mostly work for O_DIRECT block IO most of the time.

Strong nack; this isn't safe without having done the work to ensure that all the task_struct references under the f_op->aio_*() paths won't be horribly confused to find a kernel thread instead of the process that called io_submit().

The one-off handling of the submitters's cred is an indication that there might be other cases to worry about :).

You can't rely on io_getevents() being called for forward progress. Its perfectly reasonable for a task to wait for io completion by polling an eventfd that aio_complete() notifies, for instance.

- z

--
To unsubscribe, send a message with 'unsubscribe linux-aio' in the body to [email protected]. For more info on Linux AIO,
see: http://www.kvack.org/aio/
Don't email: <a href=mailto:"[email protected]">[email protected]</a>

[snip]

Makes sense, I will try to look into this.

Ah okay, didn't realize that.

Thanks,

Ankit Jain
SUSE Labs


--
To unsubscribe, send a message with 'unsubscribe linux-aio' in
the body to [email protected].  For more info on Linux AIO,
see: http://www.kvack.org/aio/
Don't email: <a href=mailto:"[email protected]">[email protected]</a>