Skip to contentSkip to conversation

aoe: add barrier support

RFC New Ed Cashin 1 participant 1 message Oct 9, 2009

Subsystems (2)
lore
Share & tools
lore

Ed Cashin

author
#0 lore raw

RFC 1/1 aoe: add barrier support

This patch allows the aoe driver to support barrier bios by draining the current set of outstanding AoE commands and then issuing an ATA flush command. If the barrier contains I/O, that I/O is then performed, followed by a final ATA flush command.

This aoe driver differs from most block device drivers in that it does not handle I/O requests but handles bios, providing a make_request_fn to the block layer.

The implementation makes the make_request_fn sleep to wait for any in-progress barrier to finish, and it sleeps waiting for the ATA flush to complete. I expect the process using make_request_fn to be something like "cp", in which case sleeping will not interfere with the performance characteristics of any unrelated aoe devices in the system. That hasn't been tested yet, though, and I'm concerned that putting pdflush to sleep could interfere with dirty data flushing on other aoe devices. Any comments about this issue would be appreciated.

Some debugging code remains in this patch for testing purposes, marked with "XXXdebug". This code allows barrier handling to be turned off and on and to be traced. Turning barrier support on and off is only supported on module load. This testing feature will not be a part of the final barrier support for aoe.

Jens Axboe suggests that code that we know can sleep should use spin_lock_irq instead of spin_lock_irqsave. Even though the latter is harmless, it also adds no value. This patch sneaks a few such lock changes in. Please comment if you think the change from spin_lock_irqsave to spin_lock_irq should be split out of the final version of this patch.

---
 drivers/block/aoe/aoe.h    |    8 ++-
 drivers/block/aoe/aoeblk.c |  122 ++++++++++++++++++++++++++++++++++++----
 drivers/block/aoe/aoecmd.c |  133 +++++++++++++++++++++++++++++++++++++++-----
 drivers/block/aoe/aoedev.c |    5 +-
 4 files changed, 240 insertions(+), 28 deletions(-)
drivers/block/aoe/aoe.h +6 -2
index db195ab..40058c0 100644
@@ -79,6 +79,7 @@ enum {
 	DEVFL_GDALLOC = (1<<4),	/* need to alloc gendisk */
 	DEVFL_KICKME = (1<<5),	/* slow polling network card catch */
 	DEVFL_NEWSIZE = (1<<6),	/* need to update dev size in block layer */
+	DEVFL_BARFAIL = (1<<7), /* barrier failed */
 
 	BUFFL_FAIL = 1,
 };
@@ -107,8 +108,8 @@ struct buf {
 	ulong bv_resid;
 	ulong bv_off;
 	sector_t sector;
-	struct bio *bio;
-	struct bio_vec *bv;
+	struct bio *bio;	/* NULL for ATA flush */
+	struct bio_vec *bv;	/* points to completion when bio is NULL */
 };
 
 struct frame {
@@ -168,6 +169,7 @@ struct aoedev {
 	struct aoetgt *targets[NTARGETS];
 	struct aoetgt **tgt;	/* target in use when working */
 	struct aoetgt **htgt;	/* target needing rexmit assistance */
+	struct bio *barrier;
 };
 
 
@@ -175,6 +177,7 @@ int aoeblk_init(void);
 void aoeblk_exit(void);
 void aoeblk_gdalloc(void *);
 void aoedisk_rm_sysfs(struct aoedev *d);
+void aoeblk_drain_check(struct aoedev *d);
 
 int aoechr_init(void);
 void aoechr_exit(void);
@@ -187,6 +190,7 @@ void aoecmd_cfg_rsp(struct sk_buff *);
 void aoecmd_sleepwork(struct work_struct *);
 void aoecmd_cleanslate(struct aoedev *);
 struct sk_buff *aoecmd_ata_id(struct aoedev *);
+void aoecmd_ata_flush(struct aoedev *d, struct completion *work);
 
 int aoedev_init(void);
 void aoedev_exit(void);
drivers/block/aoe/aoeblk.c +111 -11

173 lines not on this page · raw message

drivers/block/aoe/aoecmd.c +119 -14

214 lines not on this page · raw message

drivers/block/aoe/aoedev.c +4 -1
index fa67027..ea5d731 100644
@@ -67,7 +67,10 @@ aoedev_downdev(struct aoedev *d)
 			if (--buf->nframesout == 0
 			&& buf != d->inprocess) {
 				mempool_free(buf, d->bufpool);
-				bio_endio(bio, -EIO);
+				if (bio == d->barrier)
+					d->flags |= DEVFL_BARFAIL;
+				else
+					bio_endio(bio, -EIO);
 			}
 		}
 		(*t)->maxout = (*t)->nframes;
Signature · 1 line
1.5.6.5