blk: reimplementation of I/O barrier
Subsystems (9)
cover blk: reimplementation of I/O barrier
Hello, Jens, James, Jeff and Bartlomiej.
This is the fifth posting of blk ordered reimplementation. The last posting was on 19th, Oct.
http://marc.theaimsgroup.com/?l=linux-kernel&m=112972609907761&w=2
The generic dispatch queue patchset which is required for this ordered patchset has made it into the main tree.
http://marc.theaimsgroup.com/?l=linux-kernel&m=112972555921965&w=2
Changes from the previous post are...
* As core blk files have moved from drivers/block/ to block/, patches are updated accordingly.
* In 04_blk_scsi-update-ordered, sd_prepare_flush() is updated to initialize rq->cmd_len properly. This used to work like this previously but something broke it. Note that the original flush code is broken too. James, can you please verify this?
* libata has gone through some changes since last posting. libata updates are rewritten. Jeff, please review #07 and #08.
Other than above three, nothing has changed. Jens, after reviewing, can we push these into -mm?
Thanks. :-)
[ Start of patch descriptions ]
01_blk_add-uptodate-to-end_that_request_last.patch
: add @uptodate to end_that_request_last() and @error to rq_end_io_fn()
Add @uptodate argument to end_that_request_last() and @error to rq_end_io_fn(). There's no generic way to pass error code to request completion function, making generic error handling of non-fs request difficult (rq->errors is driver-specific and each driver uses it differently). This patch adds @uptodate to end_that_request_last() and @error to rq_end_io_fn().
For fs requests, this doesn't really matter, so just using the same uptodate argument used in the last call to end_that_request_first() should suffice. IMHO, this can also help the generic command-carrying request Jens is working on.
02_blk_implement-init_request_from_bio.patch
: separate out bio init part from __make_request
Separate out bio initialization part from __make_request. It will be used by the following blk_ordered_reimpl.
03_blk_reimplement-ordered.patch
: reimplement handling of barrier request
Reimplement handling of barrier requests.
* Flexible handling to deal with various capabilities of target devices.
* Retry support for falling back.
* Tagged queues which don't support ordered tag can do ordered.
04_blk_scsi-update-ordered.patch
: update SCSI to use new blk_ordered
All ordered request related stuff delegated to HLD. Midlayer now doens't deal with ordered setting or prepare_flush callback. sd.c updated to deal with blk_queue_ordered setting. Currently, ordered tag isn't used as SCSI midlayer cannot guarantee request ordering.
05_blk_scsi-add-fua-support.patch
: add FUA support to SCSI disk
Add FUA support to SCSI disk.
06_blk_libata-update-ordered.patch
: update libata to use new blk_ordered
Reflect changes in SCSI midlayer and updated to use new ordered request implementation
07_blk_libata-add-fua-support.patch
: add FUA support to libata
Add FUA support to libata.
08_blk_ide-update-ordered.patch
: update IDE to use new blk_ordered
Update IDE to use new blk_ordered.
09_blk_ide-add-fua-support.patch
: add FUA support to IDE
Add FUA support to IDE
10_blk_add-barrier-doc.patch
: I/O barrier documentation
I/O barrier documentation
[ End of patch descriptions ]
--
tejun
Re: [PATCH linux-2.6-block:post-2.6.15 01/10] blk: add @uptodate to end_that_request_last() and @error to rq_end_io_fn()
01_blk_add-uptodate-to-end_that_request_last.patch
Add @uptodate argument to end_that_request_last() and @error to rq_end_io_fn(). There's no generic way to pass error code to request completion function, making generic error handling of non-fs request difficult (rq->errors is driver-specific and each driver uses it differently). This patch adds @uptodate to end_that_request_last() and @error to rq_end_io_fn().
For fs requests, this doesn't really matter, so just using the same uptodate argument used in the last call to end_that_request_first() should suffice. IMHO, this can also help the generic command-carrying request Jens is working on.
block/elevator.c | 2 +- block/ll_rw_blk.c | 22 +++++++++++++++------- drivers/block/DAC960.c | 2 +- drivers/block/cciss.c | 2 +- drivers/block/cpqarray.c | 2 +- drivers/block/floppy.c | 2 +- drivers/block/nbd.c | 2 +- drivers/block/sx8.c | 2 +- drivers/block/ub.c | 2 +- drivers/block/viodasd.c | 2 +- drivers/cdrom/cdu31a.c | 2 +- drivers/ide/ide-cd.c | 4 ++-- drivers/ide/ide-io.c | 6 +++--- drivers/message/i2o/i2o_block.c | 2 +- drivers/mmc/mmc_block.c | 4 ++-- drivers/s390/block/dasd.c | 2 +- drivers/s390/char/tape_block.c | 2 +- drivers/scsi/ide-scsi.c | 4 ++-- drivers/scsi/scsi_lib.c | 2 +- drivers/scsi/sd.c | 2 +- include/linux/blkdev.h | 6 +++--- 21 files changed, 42 insertions(+), 34 deletions(-)
drivers/block/DAC960.c +1 -1
@@ -3471,7 +3471,7 @@ static inline boolean DAC960_ProcessCompif (!end_that_request_first (Request, UpToDate, Command->BlockCount)) { -end_that_request_last (Request); +end_that_request_last (Request, UpToDate);if (Command->Completion) {complete (Command->Completion);
drivers/block/cciss.c +1 -1
@@ -2299,7 +2299,7 @@ static inline void complete_command( ctlprintk ("Done with %p\n" , cmd->rq);#endif /* CCISS_DEBUG */ -end_that_request_last (cmd->rq); +end_that_request_last (cmd->rq, status ?1 : -EIO );cmd_free (h,cmd,1 ); }
drivers/block/cpqarray.c +1 -1
@@ -1036,7 +1036,7 @@ static inline void complete_command(cmdlcomplete_buffers (cmd->rq->bio, ok); DBGPX(printk("Done with %p\n", cmd->rq);); -end_that_request_last (cmd->rq); +end_that_request_last (cmd->rq, ok ?1 : -EIO ); }/*
drivers/block/floppy.c +1 -1
@@ -2301,7 +2301,7 @@ static void floppy_end_request(struct readd_disk_randomness (req->rq_disk);floppy_off ((long )req->rq_disk->private_data);blkdev_dequeue_request (req); -end_that_request_last (req); +end_that_request_last (req, uptodate);/* We're done with the request */ current_req =NULL ;
drivers/block/nbd.c +1 -1
@@ -136,7 +136,7 @@ static void nbd_end_request(struct requespin_lock_irqsave (q->queue_lock, flags);if (!end_that_request_first (req, uptodate, req->nr_sectors)) { -end_that_request_last (req); +end_that_request_last (req, uptodate); }spin_unlock_irqrestore (q->queue_lock, flags); }
drivers/block/sx8.c +1 -1
@@ -770,7 +770,7 @@ static inline void carm_end_request_queu rc =end_that_request_first (req, uptodate, req->hard_nr_sectors);assert (rc ==0 ); -end_that_request_last (req); +end_that_request_last (req, uptodate); rc =carm_put_request (host, crq);assert (rc ==0 );
drivers/block/ub.c +1 -1
@@ -942,7 +942,7 @@ static void ub_end_rq(struct request *rq rc =end_that_request_first (rq, uptodate, rq->hard_nr_sectors);// assert(rc == 0); -end_that_request_last (rq); +end_that_request_last (rq, uptodate); }/*
drivers/block/viodasd.c +1 -1
@@ -305,7 +305,7 @@ static void viodasd_end_request(struct rif (end_that_request_first (req, uptodate, num_sectors))return ;add_disk_randomness (req->rq_disk); -end_that_request_last (req); +end_that_request_last (req, uptodate); }/*
drivers/cdrom/cdu31a.c +1 -1
@@ -1402,7 +1402,7 @@ static void do_cdu31a_request(request_quif (!end_that_request_first (req,1 , nblock)) {spin_lock_irq (q->queue_lock);blkdev_dequeue_request (req); -end_that_request_last (req); +end_that_request_last (req,1 );spin_unlock_irq (q->queue_lock); }continue ;
drivers/ide/ide-cd.c +2 -2
@@ -614,7 +614,7 @@ static void cdrom_end_request (ide_drive*/ spin_lock_irqsave (&ide_lock, flags);end_that_request_chunk (failed,0 , failed->data_len); -end_that_request_last (failed); +end_that_request_last (failed,0 );spin_unlock_irqrestore (&ide_lock, flags); }
@@ -1739,7 +1739,7 @@ end_request:spin_lock_irqsave (&ide_lock, flags);blkdev_dequeue_request (rq); -end_that_request_last (rq); +end_that_request_last (rq,1 );HWGROUP (drive)->rq =NULL ;spin_unlock_irqrestore (&ide_lock, flags);return ide_stopped;
drivers/ide/ide-io.c +3 -3
@@ -89,7 +89,7 @@ int __ide_end_request(ide_drive_t *driveblkdev_dequeue_request (rq);HWGROUP (drive)->rq =NULL ; -end_that_request_last (rq); +end_that_request_last (rq, uptodate); ret =0 ; }return ret;
@@ -247,7 +247,7 @@ static void ide_complete_pm_request (ide }blkdev_dequeue_request (rq);HWGROUP (drive)->rq =NULL ; -end_that_request_last (rq); +end_that_request_last (rq,1 );spin_unlock_irqrestore (&ide_lock, flags); }
@@ -379,7 +379,7 @@ void ide_end_drive_cmd (ide_drive_t *driblkdev_dequeue_request (rq);HWGROUP (drive)->rq =NULL ; rq->errors = err; -end_that_request_last (rq); +end_that_request_last (rq, !rq->errors);spin_unlock_irqrestore (&ide_lock, flags); }
drivers/message/i2o/i2o_block.c +1 -1
@@ -466,7 +466,7 @@ static void i2o_block_end_request(structspin_lock_irqsave (q->queue_lock, flags); -end_that_request_last (req); +end_that_request_last (req, uptodate);if (likely (dev)) { dev->open_queue_depth--;
drivers/mmc/mmc_block.c +2 -2
@@ -263,7 +263,7 @@ static int mmc_blk_issue_rq(struct mmc_q*/ add_disk_randomness (req->rq_disk);blkdev_dequeue_request (req); -end_that_request_last (req); +end_that_request_last (req,1 ); }spin_unlock_irq (&md->lock); }while (ret);
@@ -289,7 +289,7 @@ static int mmc_blk_issue_rq(struct mmc_qadd_disk_randomness (req->rq_disk);blkdev_dequeue_request (req); -end_that_request_last (req); +end_that_request_last (req,0 );spin_unlock_irq (&md->lock);return 0 ;
drivers/s390/block/dasd.c +1 -1
@@ -1035,7 +1035,7 @@ dasd_end_request(struct request *req, inif (end_that_request_first (req, uptodate, req->hard_nr_sectors))BUG ();add_disk_randomness (req->rq_disk); -end_that_request_last (req); +end_that_request_last (req, uptodate); }/*
drivers/s390/char/tape_block.c +1 -1
@@ -78,7 +78,7 @@ tapeblock_end_request(struct request *re {if (end_that_request_first (req, uptodate, req->hard_nr_sectors))BUG (); -end_that_request_last (req); +end_that_request_last (req, uptodate); } static void
drivers/scsi/ide-scsi.c +2 -2
@@ -1046,7 +1046,7 @@ static int idescsi_eh_reset (struct scsi/* kill current request */ blkdev_dequeue_request (req); -end_that_request_last (req); +end_that_request_last (req,0 );if (req->flags &REQ_SENSE )kfree (scsi->pc->buffer);kfree (scsi->pc);
@@ -1056,7 +1056,7 @@ static int idescsi_eh_reset (struct scsi/* now nuke the drive queue */ while ((req =elv_next_request (drive->queue))) {blkdev_dequeue_request (req); -end_that_request_last (req); +end_that_request_last (req,0 ); }HWGROUP (drive)->rq =NULL ;
drivers/scsi/scsi_lib.c +1 -1
@@ -624,7 +624,7 @@ static struct scsi_cmnd *scsi_end_requesspin_lock_irqsave (q->queue_lock, flags);if (blk_rq_tagged (req))blk_queue_end_tag (q, req); -end_that_request_last (req); +end_that_request_last (req, uptodate);spin_unlock_irqrestore (q->queue_lock, flags);/*
drivers/scsi/sd.c +1 -1
@@ -762,7 +762,7 @@ static void sd_end_flush(request_queue_t* force journal abort of barriers */ end_that_request_first (rq, -EOPNOTSUPP , rq->hard_nr_sectors); -end_that_request_last (rq); +end_that_request_last (rq, -EOPNOTSUPP ); } }
include/linux/blkdev.h +3 -3
@@ -102,7 +102,7 @@ void copy_io_context(struct io_context *void swap_io_context (struct io_context **ioc1,struct io_context **ioc2);struct request ; -typedef void (rq_end_io_fn )(struct request *); +typedef void (rq_end_io_fn )(struct request *,int );struct request_list {int count[2 ];
@@ -557,7 +557,7 @@ extern void blk_unregister_queue(structextern void register_disk (struct gendisk *dev);extern void generic_make_request (struct bio *bio);extern void blk_put_request (struct request *); -extern void blk_end_sync_rq (struct request *rq); +extern void blk_end_sync_rq (struct request *rq,int error);extern void blk_attempt_remerge (request_queue_t *,struct request *);extern struct request *blk_get_request (request_queue_t *,int ,gfp_t );extern void blk_insert_request (request_queue_t *,struct request *,int ,void *);
@@ -607,7 +607,7 @@ static inline void blk_run_address_space*/ extern int end_that_request_first (struct request *,int ,int );extern int end_that_request_chunk (struct request *,int ,int ); -extern void end_that_request_last (struct request *); +extern void end_that_request_last (struct request *,int );extern void end_request (struct request *req,int uptodate);/*
block/elevator.c +1 -1
@@ -484,7 +484,7 @@ struct request *elv_next_request(requestblkdev_dequeue_request (rq); rq->flags |=REQ_QUIET ;end_that_request_chunk (rq,0 , nr_bytes); -end_that_request_last (rq); +end_that_request_last (rq,0 ); }else {printk (KERN_ERR "%s: bad return=%d\n" , __FUNCTION__, ret);
block/ll_rw_blk.c +15 -7
@@ -346,7 +346,7 @@ EXPORT_SYMBOL(blk_queue_issue_flush_fn);/* * Cache flushing for ordered writes handling */ -static void blk_pre_flush_end_io (struct request *flush_rq) +static void blk_pre_flush_end_io (struct request *flush_rq,int error) {struct request *rq = flush_rq->end_io_data;request_queue_t *q = rq->q;
@@ -364,7 +364,7 @@ static void blk_pre_flush_end_io(struct } } -static void blk_post_flush_end_io (struct request *flush_rq) +static void blk_post_flush_end_io (struct request *flush_rq,int error) {struct request *rq = flush_rq->end_io_data;request_queue_t *q = rq->q;
@@ -2301,7 +2301,7 @@ EXPORT_SYMBOL(blk_rq_map_kern);*/ void blk_execute_rq_nowait (request_queue_t *q,struct gendisk *bd_disk,struct request *rq,int at_head, -void (*done)(struct request *)) +rq_end_io_fn *done) {int where = at_head ?ELEVATOR_INSERT_FRONT :ELEVATOR_INSERT_BACK ;
@@ -2501,7 +2501,7 @@ EXPORT_SYMBOL(blk_put_request);* blk_end_sync_rq - executes a completion event on a request * @rq: request to complete */ -void blk_end_sync_rq (struct request *rq) +void blk_end_sync_rq (struct request *rq,int error) {struct completion *waiting = rq->waiting;
@@ -3163,9 +3163,17 @@ EXPORT_SYMBOL(end_that_request_chunk);/* * queue lock must be held */ -void end_that_request_last (struct request *req) +void end_that_request_last (struct request *req,int uptodate) {struct gendisk *disk = req->rq_disk; +int error; + +/* +* extend uptodate bool to allow < 0 value to be direct io error +*/ + error =0 ; +if (end_io_error (uptodate)) + error = !uptodate ? -EIO : uptodate;if (unlikely (laptop_mode) &&blk_fs_request (req))laptop_io_completion ();
@@ -3180,7 +3188,7 @@ void end_that_request_last(struct reques disk->in_flight--; }if (req->end_io) - req->end_io (req); + req->end_io (req, error);else __blk_put_request (req->q, req); }
@@ -3192,7 +3200,7 @@ void end_request(struct request *req, inif (!end_that_request_first (req, uptodate, req->hard_cur_sectors)) {add_disk_randomness (req->rq_disk);blkdev_dequeue_request (req); -end_that_request_last (req); +end_that_request_last (req, uptodate); } }
Re: [PATCH linux-2.6-block:post-2.6.15 02/10] blk: separate out bio init part from __make_request
In reply to Tejun Heo02_blk_implement-init_request_from_bio.patch
Separate out bio initialization part from __make_request. It will be used by the following blk_ordered_reimpl.
ll_rw_blk.c | 62 +++++++++++++++++++++++++++++++----------------------------- 1 file changed, 33 insertions(+), 29 deletions(-)
block/ll_rw_blk.c +33 -29
@@ -38,6 +38,8 @@static void blk_unplug_work (void *data);static void blk_unplug_timeout (unsigned long data);static void drive_stat_acct (struct request *rq,int nr_sectors,int new_io); +static void init_request_from_bio (struct request *req,struct bio *bio); +static int __make_request (request_queue_t *q,struct bio *bio);/* * For the allocated request tables
@@ -1651,8 +1653,6 @@ static int blk_init_free_list(request_qureturn 0 ; } -static int __make_request (request_queue_t *,struct bio *); -request_queue_t *blk_alloc_queue (gfp_t gfp_mask) {return blk_alloc_queue_node (gfp_mask,-1 );
@@ -2639,6 +2639,36 @@ void blk_attempt_remerge(request_queue_tEXPORT_SYMBOL (blk_attempt_remerge); +static void init_request_from_bio (struct request *req,struct bio *bio) +{ + req->flags |=REQ_CMD ; + +/* +* inherit FAILFAST from bio (for read-ahead, and explicit FAILFAST) +*/ +if (bio_rw_ahead (bio) ||bio_failfast (bio)) + req->flags |=REQ_FAILFAST ; + +/* +* REQ_BARRIER implies no merging, but lets make it explicit +*/ +if (unlikely (bio_barrier (bio))) + req->flags |= (REQ_HARDBARRIER |REQ_NOMERGE ); + + req->errors =0 ; + req->hard_sector = req->sector = bio->bi_sector; + req->hard_nr_sectors = req->nr_sectors =bio_sectors (bio); + req->current_nr_sectors = req->hard_cur_sectors =bio_cur_sectors (bio); + req->nr_phys_segments =bio_phys_segments (req->q, bio); + req->nr_hw_segments =bio_hw_segments (req->q, bio); + req->buffer =bio_data (bio);/* see ->buffer comment above */ + req->waiting =NULL ; + req->bio = req->biotail = bio; + req->ioprio =bio_prio (bio); + req->rq_disk = bio->bi_bdev->bd_disk; + req->start_time = jiffies; +} +static int __make_request (request_queue_t *q,struct bio *bio) {struct request *req;
@@ -2734,33 +2764,7 @@ get_rq:* We don't worry about that case for efficiency. It won't happen * often, and the elevators are able to handle it. */ - - req->flags |=REQ_CMD ; - -/* -* inherit FAILFAST from bio (for read-ahead, and explicit FAILFAST) -*/ -if (bio_rw_ahead (bio) ||bio_failfast (bio)) - req->flags |=REQ_FAILFAST ; - -/* -* REQ_BARRIER implies no merging, but lets make it explicit -*/ -if (unlikely (barrier)) - req->flags |= (REQ_HARDBARRIER |REQ_NOMERGE ); - - req->errors =0 ; - req->hard_sector = req->sector = sector; - req->hard_nr_sectors = req->nr_sectors = nr_sectors; - req->current_nr_sectors = req->hard_cur_sectors = cur_nr_sectors; - req->nr_phys_segments =bio_phys_segments (q, bio); - req->nr_hw_segments =bio_hw_segments (q, bio); - req->buffer =bio_data (bio);/* see ->buffer comment above */ - req->waiting =NULL ; - req->bio = req->biotail = bio; - req->ioprio = prio; - req->rq_disk = bio->bi_bdev->bd_disk; - req->start_time = jiffies; +init_request_from_bio (req, bio);spin_lock_irq (q->queue_lock);if (elv_queue_empty (q))
Re: [PATCH linux-2.6-block:post-2.6.15 06/10] blk: update libata to use new blk_ordered
In reply to Tejun Heo06_blk_libata-update-ordered.patch
Reflect changes in SCSI midlayer and updated to use new ordered request implementation
ahci.c | 1 - ata_piix.c | 1 - sata_mv.c | 1 - sata_nv.c | 1 - sata_promise.c | 1 - sata_sil.c | 1 - sata_sil24.c | 1 - sata_sis.c | 1 - sata_svw.c | 1 - sata_sx4.c | 1 - sata_uli.c | 1 - sata_via.c | 1 - sata_vsc.c | 1 - 13 files changed, 13 deletions(-)
drivers/scsi/ahci.c +0 -1
@@ -213,7 +213,6 @@ static struct scsi_host_template ahci_sh .dma_boundary = AHCI_DMA_BOUNDARY, .slave_configure = ata_scsi_slave_config, .bios_param = ata_std_bios_param, - .ordered_flush = 1, };static const struct ata_port_operations ahci_ops = {
drivers/scsi/ata_piix.c +0 -1
@@ -144,7 +144,6 @@ static struct scsi_host_template piix_sh .dma_boundary = ATA_DMA_BOUNDARY, .slave_configure = ata_scsi_slave_config, .bios_param = ata_std_bios_param, - .ordered_flush = 1, };static const struct ata_port_operations piix_pata_ops = {
drivers/scsi/sata_mv.c +0 -1
@@ -287,7 +287,6 @@ static struct scsi_host_template mv_sht .dma_boundary = MV_DMA_BOUNDARY, .slave_configure = ata_scsi_slave_config, .bios_param = ata_std_bios_param, - .ordered_flush = 1, };static const struct ata_port_operations mv_ops = {
drivers/scsi/sata_nv.c +0 -1
@@ -235,7 +235,6 @@ static struct scsi_host_template nv_sht .dma_boundary = ATA_DMA_BOUNDARY, .slave_configure = ata_scsi_slave_config, .bios_param = ata_std_bios_param, - .ordered_flush = 1, };static const struct ata_port_operations nv_ops = {
drivers/scsi/sata_promise.c +0 -1
@@ -111,7 +111,6 @@ static struct scsi_host_template pdc_ata .dma_boundary = ATA_DMA_BOUNDARY, .slave_configure = ata_scsi_slave_config, .bios_param = ata_std_bios_param, - .ordered_flush = 1, };static const struct ata_port_operations pdc_sata_ops = {
drivers/scsi/sata_sil.c +0 -1
@@ -147,7 +147,6 @@ static struct scsi_host_template sil_sht .dma_boundary = ATA_DMA_BOUNDARY, .slave_configure = ata_scsi_slave_config, .bios_param = ata_std_bios_param, - .ordered_flush = 1, };static const struct ata_port_operations sil_ops = {
drivers/scsi/sata_sis.c +0 -1
@@ -99,7 +99,6 @@ static struct scsi_host_template sis_sht .dma_boundary = ATA_DMA_BOUNDARY, .slave_configure = ata_scsi_slave_config, .bios_param = ata_std_bios_param, - .ordered_flush = 1, };static const struct ata_port_operations sis_ops = {
drivers/scsi/sata_svw.c +0 -1
@@ -303,7 +303,6 @@ static struct scsi_host_template k2_sata .proc_info = k2_sata_proc_info, #endif .bios_param = ata_std_bios_param, - .ordered_flush = 1, };
drivers/scsi/sata_sx4.c +0 -1
@@ -194,7 +194,6 @@ static struct scsi_host_template pdc_sat .dma_boundary = ATA_DMA_BOUNDARY, .slave_configure = ata_scsi_slave_config, .bios_param = ata_std_bios_param, - .ordered_flush = 1, };static const struct ata_port_operations pdc_20621_ops = {
drivers/scsi/sata_uli.c +0 -1
@@ -87,7 +87,6 @@ static struct scsi_host_template uli_sht .dma_boundary = ATA_DMA_BOUNDARY, .slave_configure = ata_scsi_slave_config, .bios_param = ata_std_bios_param, - .ordered_flush = 1, };static const struct ata_port_operations uli_ops = {
drivers/scsi/sata_via.c +0 -1
@@ -106,7 +106,6 @@ static struct scsi_host_template svia_sh .dma_boundary = ATA_DMA_BOUNDARY, .slave_configure = ata_scsi_slave_config, .bios_param = ata_std_bios_param, - .ordered_flush = 1, };static const struct ata_port_operations svia_sata_ops = {
drivers/scsi/sata_vsc.c +0 -1
@@ -235,7 +235,6 @@ static struct scsi_host_template vsc_sat .dma_boundary = ATA_DMA_BOUNDARY, .slave_configure = ata_scsi_slave_config, .bios_param = ata_std_bios_param, - .ordered_flush = 1, };
drivers/scsi/sata_sil24.c +0 -1
@@ -272,7 +272,6 @@ static struct scsi_host_template sil24_s .dma_boundary = ATA_DMA_BOUNDARY, .slave_configure = ata_scsi_slave_config, .bios_param = ata_std_bios_param, - .ordered_flush = 1, /* NCQ not supported yet */ };static const struct ata_port_operations sil24_ops = {
Re: [PATCH linux-2.6-block:post-2.6.15 04/10] blk: update SCSI to use new blk_ordered
In reply to Tejun Heo04_blk_scsi-update-ordered.patch
All ordered request related stuff delegated to HLD. Midlayer now doens't deal with ordered setting or prepare_flush callback. sd.c updated to deal with blk_queue_ordered setting. Currently, ordered tag isn't used as SCSI midlayer cannot guarantee request ordering.
drivers/scsi/hosts.c | 9 ------ drivers/scsi/scsi_lib.c | 46 ---------------------------------- drivers/scsi/sd.c | 60 ++++++++++++++++----------------------------- include/scsi/scsi_driver.h | 1 include/scsi/scsi_host.h | 1 5 files changed, 22 insertions(+), 95 deletions(-)
drivers/scsi/hosts.c +0 -9
@@ -347,17 +347,8 @@ struct Scsi_Host *scsi_host_alloc(struct shost->cmd_per_lun = sht->cmd_per_lun; shost->unchecked_isa_dma = sht->unchecked_isa_dma; shost->use_clustering = sht->use_clustering; - shost->ordered_flush = sht->ordered_flush; shost->ordered_tag = sht->ordered_tag; -/* -* hosts/devices that do queueing must support ordered tags -*/ -if (shost->can_queue >1 && shost->ordered_flush) { -printk (KERN_ERR "scsi: ordered flushes don't support queueing\n" ); - shost->ordered_flush =0 ; - } -if (sht->max_host_blocked) shost->max_host_blocked = sht->max_host_blocked;else
drivers/scsi/scsi_lib.c +0 -46
64 lines not on this page · raw message
drivers/scsi/sd.c +22 -38
112 lines not on this page · raw message
include/scsi/scsi_driver.h +0 -1
@@ -15,7 +15,6 @@ struct scsi_driver {void (*rescan)(struct device *);int (*issue_flush)(struct device *,sector_t *);int (*prepare_flush)(struct request_queue *,struct request *); -void (*end_flush)(struct request_queue *,struct request *); };#define to_scsi_driver (drv) \ container_of((drv), struct scsi_driver, gendrv)
include/scsi/scsi_host.h +0 -1
@@ -392,7 +392,6 @@ struct scsi_host_template {/* * ordered write support */ - unsigned ordered_flush:1; unsigned ordered_tag:1;/*
Re: [PATCH linux-2.6-block:post-2.6.15 05/10] blk: add FUA support to SCSI disk
In reply to Tejun Heo05_blk_scsi-add-fua-support.patch
Add FUA support to SCSI disk.
sd.c | 29 ++++++++++++++++++++++++++--- 1 file changed, 26 insertions(+), 3 deletions(-)
drivers/scsi/sd.c +26 -3
@@ -102,6 +102,7 @@ struct scsi_disk {u8 write_prot; unsigned WCE : 1; /* state of disk WCE bit */ unsigned RCD : 1; /* state of disk RCD bit, unused */ + unsigned DPOFUA : 1; /* state of disk DPOFUA bit */ };static DEFINE_IDR (sd_index_idr);
@@ -357,6 +358,7 @@ static int sd_init_command(struct scsi_cif (block >0xffffffff ) { SCpnt->cmnd[0 ] +=READ_16 -READ_6 ; + SCpnt->cmnd[1 ] |=blk_fua_rq (rq) ?0x8 :0 ; SCpnt->cmnd[2 ] =sizeof (block) >4 ? (unsigned char ) (block >>56 ) &0xff :0 ; SCpnt->cmnd[3 ] =sizeof (block) >4 ? (unsigned char ) (block >>48 ) &0xff :0 ; SCpnt->cmnd[4 ] =sizeof (block) >4 ? (unsigned char ) (block >>40 ) &0xff :0 ;
@@ -376,6 +378,7 @@ static int sd_init_command(struct scsi_c this_count =0xffff ; SCpnt->cmnd[0 ] +=READ_10 -READ_6 ; + SCpnt->cmnd[1 ] |=blk_fua_rq (rq) ?0x8 :0 ; SCpnt->cmnd[2 ] = (unsigned char ) (block >>24 ) &0xff ; SCpnt->cmnd[3 ] = (unsigned char ) (block >>16 ) &0xff ; SCpnt->cmnd[4 ] = (unsigned char ) (block >>8 ) &0xff ;
@@ -384,6 +387,17 @@ static int sd_init_command(struct scsi_c SCpnt->cmnd[7 ] = (unsigned char ) (this_count >>8 ) &0xff ; SCpnt->cmnd[8 ] = (unsigned char ) this_count &0xff ; }else { +if (unlikely (blk_fua_rq (rq))) { +/* +* This happens only if this drive failed +* 10byte rw command with ILLEGAL_REQUEST +* during operation and thus turned off +* use_10_for_rw. +*/ +printk (KERN_ERR "sd: FUA write on READ/WRITE(6) drive\n" ); +return 0 ; + } + SCpnt->cmnd[1 ] |= (unsigned char ) ((block >>16 ) &0x1f ); SCpnt->cmnd[2 ] = (unsigned char ) ((block >>8 ) &0xff ); SCpnt->cmnd[3 ] = (unsigned char ) block &0xff ;
@@ -1409,10 +1423,18 @@ sd_read_cache_type(struct scsi_disk *sdk sdkp->RCD =0 ; } + sdkp->DPOFUA = (data.device_specific &0x10 ) !=0 ; +if (sdkp->DPOFUA && !sdkp->device->use_10_for_rw) { +printk (KERN_NOTICE "SCSI device %s: uses " +"READ/WRITE(6), disabling FUA\n" , diskname); + sdkp->DPOFUA =0 ; + } + ct = sdkp->RCD +2 *sdkp->WCE; -printk (KERN_NOTICE "SCSI device %s: drive cache: %s\n" , - diskname, types[ct]); +printk (KERN_NOTICE "SCSI device %s: drive cache: %s%s\n" , + diskname, types[ct], + sdkp->DPOFUA ?" w/ FUA" :"" );return ; }
@@ -1491,7 +1513,8 @@ static int sd_revalidate_disk(struct gen* QUEUE_ORDERED_TAG_* even when ordered tag is supported. */ if (sdkp->WCE) - ordered =QUEUE_ORDERED_DRAIN_FLUSH ; + ordered = sdkp->DPOFUA + ?QUEUE_ORDERED_DRAIN_FUA :QUEUE_ORDERED_DRAIN_FLUSH ;else ordered =QUEUE_ORDERED_DRAIN ;
Re: [PATCH linux-2.6-block:post-2.6.15 07/10] blk: add FUA support to libata
In reply to Tejun Heo07_blk_libata-add-fua-support.patch
Add FUA support to libata.
drivers/scsi/libata-core.c | 31 +++++++++++++++++++++++++------ drivers/scsi/libata-scsi.c | 32 ++++++++++++++++++++++++++------ drivers/scsi/libata.h | 4 +++- include/linux/ata.h | 6 +++++- include/linux/libata.h | 3 ++- 5 files changed, 61 insertions(+), 15 deletions(-)
drivers/scsi/libata-core.c +25 -6
@@ -563,16 +563,28 @@ static const u8 ata_rw_cmds[] = {ATA_CMD_WRITE_MULTI ,ATA_CMD_READ_MULTI_EXT ,ATA_CMD_WRITE_MULTI_EXT , +0 , +0 , +0 , +ATA_CMD_WRITE_MULTI_FUA_EXT ,/* pio */ ATA_CMD_PIO_READ ,ATA_CMD_PIO_WRITE ,ATA_CMD_PIO_READ_EXT ,ATA_CMD_PIO_WRITE_EXT , +0 , +0 , +0 , +0 ,/* dma */ ATA_CMD_READ ,ATA_CMD_WRITE ,ATA_CMD_READ_EXT , -ATA_CMD_WRITE_EXT +ATA_CMD_WRITE_EXT , +0 , +0 , +0 , +ATA_CMD_WRITE_FUA_EXT };/**
@@ -585,25 +597,32 @@ static const u8 ata_rw_cmds[] = {* LOCKING: * caller. */ -void ata_rwcmd_protocol (struct ata_queued_cmd *qc) +int ata_rwcmd_protocol (struct ata_queued_cmd *qc) {struct ata_taskfile *tf = &qc->tf;struct ata_device *dev = qc->dev; +u8 cmd; -int index, lba48, write; +int index, fua, lba48, write; + fua = (tf->flags &ATA_TFLAG_FUA ) ?4 :0 ; lba48 = (tf->flags &ATA_TFLAG_LBA48 ) ?2 :0 ; write = (tf->flags &ATA_TFLAG_WRITE ) ?1 :0 ;if (dev->flags &ATA_DFLAG_PIO ) { tf->protocol =ATA_PROT_PIO ; - index = dev->multi_count ?0 :4 ; + index = dev->multi_count ?0 :8 ; }else { tf->protocol =ATA_PROT_DMA ; - index =8 ; + index =16 ; } - tf->command = ata_rw_cmds[index + lba48 + write]; + cmd = ata_rw_cmds[index + fua + lba48 + write]; +if (cmd) { + tf->command = cmd; +return 0 ; + } +return -1 ; }static const char * xfer_mode_str[] = {
include/linux/ata.h +5 -1
@@ -129,6 +129,7 @@ enum {ATA_CMD_READ_EXT =0x25 ,ATA_CMD_WRITE =0xCA ,ATA_CMD_WRITE_EXT =0x35 , +ATA_CMD_WRITE_FUA_EXT =0x3D ,ATA_CMD_PIO_READ =0x20 ,ATA_CMD_PIO_READ_EXT =0x24 , ATA_CMD_PIO_WRITE = 0x30,
@@ -137,6 +138,7 @@ enum {ATA_CMD_READ_MULTI_EXT =0x29 ,ATA_CMD_WRITE_MULTI =0xC5 ,ATA_CMD_WRITE_MULTI_EXT =0x39 , +ATA_CMD_WRITE_MULTI_FUA_EXT =0xCE ,ATA_CMD_SET_FEATURES =0xEF ,ATA_CMD_PACKET =0xA0 , ATA_CMD_VERIFY = 0x40,
@@ -192,6 +194,7 @@ enum {ATA_TFLAG_DEVICE = (1 <<2 ),/* enable r/w to device reg */ ATA_TFLAG_WRITE = (1 <<3 ),/* data dir: host->dev==1 (write) */ ATA_TFLAG_LBA = (1 <<4 ),/* enable LBA */ + ATA_TFLAG_FUA = (1 << 5), /* enable FUA */ }; enum ata_tf_protocols {
@@ -245,7 +248,8 @@ struct ata_taskfile {#define ata_id_is_sata (id) ((id)[93] == 0)#define ata_id_rahead_enabled (id) ((id)[85] & (1 << 6))#define ata_id_wcache_enabled (id) ((id)[85] & (1 << 5)) -#define ata_id_has_flush (id) ((id)[83] & (1 << 12)) +#define ata_id_has_fua (id) ((id)[84] & (1 << 6)) +#define ata_id_has_flush (id) ((id)[83] & (1 << 12))#define ata_id_has_flush_ext (id) ((id)[83] & (1 << 13))#define ata_id_has_lba48 (id) ((id)[83] & (1 << 10))#define ata_id_has_wcache (id) ((id)[82] & (1 << 5))
drivers/scsi/libata-scsi.c +26 -6
@@ -1080,11 +1080,13 @@ static unsigned int ata_scsi_rw_xlat(str scsicmd[0] == WRITE_16) tf->flags |=ATA_TFLAG_WRITE ; -/* Calculate the SCSI LBA and transfer length. */ +/* Calculate the SCSI LBA, transfer length and FUA. */ switch (scsicmd[0 ]) {case READ_10 :case WRITE_10 :scsi_10_lba_len (scsicmd, &block, &n_block); +if (unlikely (scsicmd[1 ] & (1 <<3 ))) + tf->flags |=ATA_TFLAG_FUA ;break ;case READ_6 :case WRITE_6 :
@@ -1099,6 +1101,8 @@ static unsigned int ata_scsi_rw_xlat(strcase READ_16 :case WRITE_16 :scsi_16_lba_len (scsicmd, &block, &n_block); +if (unlikely (scsicmd[1 ] & (1 <<3 ))) + tf->flags |=ATA_TFLAG_FUA ;break ;default :DPRINTK ("no-byte command\n" );
@@ -1142,7 +1146,8 @@ static unsigned int ata_scsi_rw_xlat(str tf->device |= (block >>24 ) &0xf ; } -ata_rwcmd_protocol (qc); +if (unlikely (ata_rwcmd_protocol (qc) <0 )) + goto invalid_fld; qc->nsect = n_block; tf->nsect = n_block &0xff ;
@@ -1160,7 +1165,8 @@ static unsigned int ata_scsi_rw_xlat(strif ((block >>28 ) || (n_block >256 )) goto out_of_range; -ata_rwcmd_protocol (qc); +if (unlikely (ata_rwcmd_protocol (qc) <0 )) + goto invalid_fld;/* Convert LBA to CHS */ track = (u32 )block / dev->sectors;
@@ -1696,6 +1702,7 @@ static unsigned int ata_msense_rw_recoveunsigned int ata_scsiop_mode_sense (struct ata_scsi_args *args,u8 *rbuf,unsigned int buflen) { +struct ata_device *dev = args->dev;u8 *scsicmd = args->cmd->cmnd, *p, *last;const u8 sat_blk_desc[] = {0 ,0 ,0 ,0 ,/* number of blocks: sat unspecified */
@@ -1704,6 +1711,7 @@ unsigned int ata_scsiop_mode_sense(struc };u8 pg, spg;unsigned int ebd, page_control, six_byte, output_len, alloc_len, minlen; +u8 dpofua;VPRINTK ("ENTER\n" );
@@ -1772,9 +1780,17 @@ unsigned int ata_scsiop_mode_sense(strucif (minlen <1 )return 0 ; + + dpofua =0 ; +if (ata_id_has_fua (args->id) && dev->flags &ATA_DFLAG_LBA48 && + (!(dev->flags &ATA_DFLAG_PIO ) || dev->multi_count)) + dpofua =1 <<4 ; +if (six_byte) { output_len--; rbuf[0 ] = output_len; +if (minlen >2 ) + rbuf[2 ] |= dpofua;if (ebd) {if (minlen >3 ) rbuf[3 ] =sizeof (sat_blk_desc);
@@ -1787,6 +1803,8 @@ unsigned int ata_scsiop_mode_sense(struc rbuf[0 ] = output_len >>8 ;if (minlen >1 ) rbuf[1 ] = output_len; +if (minlen >3 ) + rbuf[3 ] |= dpofua;if (ebd) {if (minlen >7 ) rbuf[7 ] =sizeof (sat_blk_desc);
@@ -2424,7 +2442,7 @@ int ata_scsi_queuecmd(struct scsi_cmnd *if (xlat_func)ata_scsi_translate (ap, dev, cmd, done, xlat_func);else -ata_scsi_simulate (dev->id, cmd, done); +ata_scsi_simulate (ap, dev, cmd, done); }else ata_scsi_translate (ap, dev, cmd, done, atapi_xlat);
@@ -2447,14 +2465,16 @@ out_unlock:* spin_lock_irqsave(host_set lock) */ -void ata_scsi_simulate (u16 *id, +void ata_scsi_simulate (struct ata_port *ap,struct ata_device *dev,struct scsi_cmnd *cmd,void (*done)(struct scsi_cmnd *)) {struct ata_scsi_args args;const u8 *scsicmd = cmd->cmnd; - args.id = id; + args.ap = ap; + args.dev = dev; + args.id = dev->id; args.cmd = cmd; args.done = done;
drivers/scsi/libata.h +3 -1
@@ -32,6 +32,8 @@#define DRV_VERSION "1.12"/* must be exactly four chars */ struct ata_scsi_args { +struct ata_port *ap; +struct ata_device *dev;u16 *id;struct scsi_cmnd *cmd;void (*done)(struct scsi_cmnd *);
@@ -42,7 +44,7 @@ extern int atapi_enabled;extern int ata_qc_complete_noop (struct ata_queued_cmd *qc,unsigned int err_mask);extern struct ata_queued_cmd *ata_qc_new_init (struct ata_port *ap,struct ata_device *dev); -extern void ata_rwcmd_protocol (struct ata_queued_cmd *qc); +extern int ata_rwcmd_protocol (struct ata_queued_cmd *qc);extern void ata_qc_free (struct ata_queued_cmd *qc);extern int ata_qc_issue (struct ata_queued_cmd *qc);extern int ata_check_atapi_dma (struct ata_queued_cmd *qc);
include/linux/libata.h +2 -1
@@ -476,7 +476,8 @@ extern u8 ata_bmdma_status(struct ata_extern void ata_bmdma_irq_clear (struct ata_port *ap);extern void ata_qc_complete (struct ata_queued_cmd *qc,unsigned int err_mask);extern void ata_eng_timeout (struct ata_port *ap); -extern void ata_scsi_simulate (u16 *id,struct scsi_cmnd *cmd, +extern void ata_scsi_simulate (struct ata_port *ap,struct ata_device *dev, +struct scsi_cmnd *cmd,void (*done)(struct scsi_cmnd *)); extern int ata_std_bios_param(struct scsi_device *sdev, struct block_device *bdev,
Re: [PATCH linux-2.6-block:post-2.6.15 10/10] blk: I/O barrier documentation
In reply to Tejun Heo10_blk_add-barrier-doc.patch
I/O barrier documentation
barrier.txt | 271 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ biodoc.txt | 10 -- 2 files changed, 273 insertions(+), 8 deletions(-)
Documentation/block/barrier.txt new file +271 -0
@@ -0,0 +1,271 @@ +I/O Barriers +============ +Tejun Heo <[email protected]>, July 22 2005 + +I/O barrier requests are used to guarantee ordering around the barrier +requests. Unless you're crazy enough to use disk drives for +implementing synchronization constructs (wow, sounds interesting...), +the ordering is meaningful only for write requests for things like +journal checkpoints. All requests queued before a barrier request +must be finished (made it to the physical medium) before the barrier +request is started, and all requests queued after the barrier request +must be started only after the barrier request is finished (again, +made it to the physical medium). + +In other words, I/O barrier requests have the following two properties. + +1. Request ordering + +Requests cannot pass the barrier request. Preceding requests are +processed before the barrier and following requests after. + +Depending on what features a drive supports, this can be done in one +of the following three ways. + +i. For devices which have queue depth greater than 1 (TCQ devices) and +support ordered tags, block layer can just issue the barrier as an +ordered request and the lower level driver, controller and drive +itself are responsible for making sure that the ordering contraint is +met. Most modern SCSI controllers/drives should support this. + +NOTE: SCSI ordered tag isn't currently used due to limitation in the + SCSI midlayer, see the following random notes section. + +ii. For devices which have queue depth greater than 1 but don't +support ordered tags, block layer ensures that the requests preceding +a barrier request finishes before issuing the barrier request. Also, +it defers requests following the barrier until the barrier request is +finished. Older SCSI controllers/drives and SATA drives fall in this +category. + +iii. Devices which have queue depth of 1. This is a degenerate case +of ii. Just keeping issue order suffices. Ancient SCSI +controllers/drives and IDE drives are in this category. + +2. Forced flushing to physcial medium + +Again, if you're not gonna do synchronization with disk drives (dang, +it sounds even more appealing now!), the reason you use I/O barriers +is mainly to protect filesystem integrity when power failure or some +other events abruptly stop the drive from operating and possibly make +the drive lose data in its cache. So, I/O barriers need to guarantee +that requests actually get written to non-volatile medium in order. + +There are four cases, + +i. No write-back cache. Keeping requests ordered is enough. + +ii. Write-back cache but no flush operation. There's no way to +gurantee physical-medium commit order. This kind of devices can't to +I/O barriers. + +iii. Write-back cache and flush operation but no FUA (forced unit +access). We need two cache flushes - before and after the barrier +request. + +iv. Write-back cache, flush operation and FUA. We still need one +flush to make sure requests preceding a barrier are written to medium, +but post-barrier flush can be avoided by using FUA write on the +barrier itself. + + +How to support barrier requests in drivers +------------------------------------------ + +All barrier handling is done inside block layer proper. All low level +drivers have to are implementing its prepare_flush_fn and using one +the following two functions to indicate what barrier type it supports +and how to prepare flush requests. Note that the term 'ordered' is +used to indicate the whole sequence of performing barrier requests +including draining and flushing. + +typedef void (prepare_flush_fn)(request_queue_t *q, struct request *rq); + +int blk_queue_ordered(request_queue_t *q, unsigned ordered, + prepare_flush_fn *prepare_flush_fn, + unsigned gfp_mask); + +int blk_queue_ordered_locked(request_queue_t *q, unsigned ordered, + prepare_flush_fn *prepare_flush_fn, + unsigned gfp_mask); + +The only difference between the two functions is whether or not the +caller is holding q->queue_lock on entry. The latter expects the +caller is holding the lock. + +@q : the queue in question +@ordered : the ordered mode the driver/device supports +@prepare_flush_fn : this function should prepare @rq such that it + flushes cache to physical medium when executed +@gfp_mask : gfp_mask used when allocating data structures + for ordered processing + +For example, SCSI disk driver's prepare_flush_fn looks like the +following. + +static void sd_prepare_flush(request_queue_t *q, struct request *rq) +{ + memset(rq->cmd, 0, sizeof(rq->cmd)); + rq->flags |= REQ_BLOCK_PC; + rq->timeout = SD_TIMEOUT; + rq->cmd[0] = SYNCHRONIZE_CACHE; +} + +The following seven ordered modes are supported. The following table +shows which mode should be used depending on what features a +device/driver supports. In the leftmost column of table, +QUEUE_ORDERED_ prefix is omitted from the mode names to save space. + +The table is followed by description of each mode. Note that in the +descriptions of QUEUE_ORDERED_DRAIN*, '=>' is used whereas '->' is +used for QUEUE_ORDERED_TAG* descriptions. '=>' indicates that the +preceding step must be complete before proceeding to the next step. +'->' indicates that the next step can start as soon as the previous +step is issued. + + write-back cache ordered tag flush FUA +----------------------------------------------------------------------- +NONE yes/no N/A no N/A +DRAIN no no N/A N/A +DRAIN_FLUSH yes no yes no +DRAIN_FUA yes no yes yes +TAG no yes N/A N/A +TAG_FLUSH yes yes yes no +TAG_FUA yes yes yes yes + + +QUEUE_ORDERED_NONE + I/O barriers are not needed and/or supported. + + Sequence: N/A + +QUEUE_ORDERED_DRAIN + Requests are ordered by draining the request queue and cache + flushing isn't needed. + + Sequence: drain => barrier + +QUEUE_ORDERED_DRAIN_FLUSH + Requests are ordered by draining the request queue and both + pre-barrier and post-barrier cache flushings are needed. + + Sequence: drain => preflush => barrier => postflush + +QUEUE_ORDERED_DRAIN_FUA + Requests are ordered by draining the request queue and + pre-barrier cache flushing is needed. By using FUA on barrier + request, post-barrier flushing can be skipped. + + Sequence: drain => preflush => barrier + +QUEUE_ORDERED_TAG + Requests are ordered by ordered tag and cache flushing isn't + needed. + + Sequence: barrier + +QUEUE_ORDERED_TAG_FLUSH + Requests are ordered by ordered tag and both pre-barrier and + post-barrier cache flushings are needed. + + Sequence: preflush -> barrier -> postflush + +QUEUE_ORDERED_TAG_FUA + Requests are ordered by ordered tag and pre-barrier cache + flushing is needed. By using FUA on barrier request, + post-barrier flushing can be skipped. + + Sequence: preflush -> barrier + + +Random notes/caveats +-------------------- + +* SCSI layer currently can't use TAG ordering even if the drive, +controller and driver support it. The problem is that SCSI midlayer +request dispatch function is not atomic. It releases queue lock and +switch to SCSI host lock during issue and it's possible and likely to +happen in time that requests change their relative positions. Once +this problem is solved, TAG ordering can be enabled. + +* Currently, no matter which ordered mode is used, there can be only +one barrier request in progress. All I/O barriers are held off by +block layer until the previous I/O barrier is complete. This doesn't +make any difference for DRAIN ordered devices, but, for TAG ordered +devices with very high command latency, passing multiple I/O barriers +to low level *might* be helpful if they are very frequent. Well, this +certainly is a non-issue. I'm writing this just to make clear that no +two I/O barrier is ever passed to low-level driver. + +* Completion order. Requests in ordered sequence are issued in order +but not required to finish in order. Barrier implementation can +handle out-of-order completion of ordered sequence. IOW, the requests +MUST be processed in order but the hardware/software completion paths +are allowed to reorder completion notifications - eg. current SCSI +midlayer doesn't preserve completion order during error handling. + +* Requeueing order. Low-level drivers are free to requeue any request +after they removed it from the request queue with +blkdev_dequeue_request(). As barrier sequence should be kept in order +when requeued, generic elevator code takes care of putting requests in +order around barrier. See blk_ordered_req_seq() and +ELEVATOR_INSERT_REQUEUE handling in __elv_add_request() for details. + +Note that block drivers must not requeue preceding requests while +completing latter requests in an ordered sequence. Currently, no +error checking is done against this. + +* Error handling. Currently, block layer will report error to upper +layer if any of requests in an ordered sequence fails. Unfortunately, +this doesn't seem to be enough. Look at the following request flow. +QUEUE_ORDERED_TAG_FLUSH is in use. + + [0] [1] [2] [3] [pre] [barrier] [post] < [4] [5] [6] ... > + still in elevator + +Let's say request [2], [3] are write requests to update file system +metadata (journal or whatever) and [barrier] is used to mark that +those updates are valid. Consider the following sequence. + + i. Requests [0] ~ [post] leaves the request queue and enters + low-level driver. + ii. After a while, unfortunately, something goes wrong and the + drive fails [2]. Note that any of [0], [1] and [3] could have + completed by this time, but [pre] couldn't have been finished + as the drive must process it in order and it failed before + processing that command. + iii. Error handling kicks in and determines that the error is + unrecoverable and fails [2], and resumes operation. + iv. [pre] [barrier] [post] gets processed. + v. *BOOM* power fails + +The problem here is that the barrier request is *supposed* to indicate +that filesystem update requests [2] and [3] made it safely to the +physical medium and, if the machine crashes after the barrier is +written, filesystem recovery code can depend on that. Sadly, that +isn't true in this case anymore. IOW, the success of a I/O barrier +should also be dependent on success of some of the preceding requests, +where only upper layer (filesystem) knows what 'some' is. + +This can be solved by implementing a way to tell the block layer which +requests affect the success of the following barrier request and +making lower lever drivers to resume operation on error only after +block layer tells it to do so. + +As the probability of this happening is very low and the drive should +be faulty, implementing the fix is probably an overkill. But, still, +it's there. + +* In previous drafts of barrier implementation, there was fallback +mechanism such that, if FUA or ordered TAG fails, less fancy ordered +mode can be selected and the failed barrier request is retried +automatically. The rationale for this feature was that as FUA is +pretty new in ATA world and ordered tag was never used widely, there +could be devices which report to support those features but choke when +actually given such requests. + + This was removed for two reasons 1. it's an overkill 2. it's +impossible to implement properly when TAG ordering is used as low +level drivers resume after an error automatically. If it's ever +needed adding it back and modifying low level drivers accordingly +shouldn't be difficult.
Documentation/block/biodoc.txt +2 -8
@@ -263,14 +263,8 @@ A flag in the bio structure, BIO_BARRIER The generic i/o scheduler would make sure that it places the barrier request and all other requests coming after it after all the previous requests in the queue. Barriers may be implemented in different ways depending on the -driver. A SCSI driver for example could make use of ordered tags to -preserve the necessary ordering with a lower impact on throughput. For IDE -this might be two sync cache flush: a pre and post flush when encountering -a barrier write. - -There is a provision for queues to indicate what kind of barriers they -can provide. This is as of yet unmerged, details will be added here once it -is in the kernel. +driver. For more details regarding I/O barriers, please read barrier.txt +in this directory. 1.2.2 Request Priority/Latency
Re: [PATCH linux-2.6-block:post-2.6.15 03/10] blk: reimplement handling of barrier request
In reply to Tejun Heo03_blk_reimplement-ordered.patch
Reimplement handling of barrier requests.
* Flexible handling to deal with various capabilities of target devices.
* Retry support for falling back.
* Tagged queues which don't support ordered tag can do ordered.
block/elevator.c | 83 +++++--- block/ll_rw_blk.c | 468 ++++++++++++++++++++++++++++++++--------------- include/linux/blkdev.h | 83 +++++--- include/linux/elevator.h | 1 4 files changed, 443 insertions(+), 192 deletions(-)
block/elevator.c +57 -26
@@ -303,22 +303,24 @@ void elv_requeue_request(request_queue_t rq->flags &= ~REQ_STARTED ; -/* -* if this is the flush, requeue the original instead and drop the flush -*/ -if (rq->flags &REQ_BAR_FLUSH ) { -clear_bit (QUEUE_FLAG_FLUSH , &q->queue_flags); - rq = rq->end_io_data; - } - -__elv_add_request (q, rq,ELEVATOR_INSERT_FRONT ,0 ); +__elv_add_request (q, rq,ELEVATOR_INSERT_REQUEUE ,0 ); }void __elv_add_request (request_queue_t *q,struct request *rq,int where,int plug) { +struct list_head *pos; +unsigned ordseq; + + rq->flags |= q->ordcolor ?REQ_ORDERED_COLOR :0 ; +if (rq->flags & (REQ_SOFTBARRIER |REQ_HARDBARRIER )) {/* +* toggle ordered color +*/ + q->ordcolor ^=1 ; + +/* * barriers implicitly indicate back insertion */ if (where ==ELEVATOR_INSERT_SORT )
@@ -379,6 +381,30 @@ void __elv_add_request(request_queue_t * q->elevator->ops->elevator_add_req_fn (q, rq);break ; +case ELEVATOR_INSERT_REQUEUE : +/* +* If ordered flush isn't in progress, we do front +* insertion; otherwise, requests should be requeued +* in ordseq order. +*/ + rq->flags |=REQ_SOFTBARRIER ; + +if (q->ordseq ==0 ) { +list_add (&rq->queuelist, &q->queue_head); +break ; + } + + ordseq =blk_ordered_req_seq (rq); + +list_for_each (pos, &q->queue_head) { +struct request *pos_rq =list_entry_rq (pos); +if (ordseq <=blk_ordered_req_seq (pos_rq)) +break ; + } + +list_add_tail (&rq->queuelist, pos); +break ; +default :printk (KERN_ERR "%s: bad insertion point %d\n" , __FUNCTION__, where);
@@ -408,25 +434,16 @@ static inline struct request *__elv_next {struct request *rq; -if (unlikely (list_empty (&q->queue_head) && - !q->elevator->ops->elevator_dispatch_fn (q,0 ))) -return NULL ; - - rq =list_entry_rq (q->queue_head.next); - -/* -* if this is a barrier write and the device has to issue a -* flush sequence to support it, check how far we are -*/ -if (blk_fs_request (rq) &&blk_barrier_rq (rq)) { -BUG_ON (q->ordered ==QUEUE_ORDERED_NONE ); +while (1 ) { +while (!list_empty (&q->queue_head)) { + rq =list_entry_rq (q->queue_head.next); +if (blk_do_ordered (q, &rq)) +return rq; + } -if (q->ordered ==QUEUE_ORDERED_FLUSH && - !blk_barrier_preflush (rq)) - rq =blk_start_pre_flush (q, rq); +if (!q->elevator->ops->elevator_dispatch_fn (q,0 )) +return NULL ; } - -return rq; }struct request *elv_next_request (request_queue_t *q)
@@ -593,7 +610,21 @@ void elv_completed_request(request_queue* request is released from the driver, io must be done */ if (blk_account_rq (rq)) { +struct request *first_rq =list_entry_rq (q->queue_head.next); + q->in_flight--; + +/* +* Check if the queue is waiting for fs requests to be +* drained for flush sequence. +*/ +if (q->ordseq && q->in_flight ==0 && +blk_ordered_cur_seq (q) ==QUEUE_ORDSEQ_DRAIN && +blk_ordered_req_seq (first_rq) >QUEUE_ORDSEQ_DRAIN ) { +blk_ordered_complete_seq (q,QUEUE_ORDSEQ_DRAIN ,0 ); + q->request_fn (q); + } +if (blk_sorted_rq (rq) && e->ops->elevator_completed_req_fn) e->ops->elevator_completed_req_fn (q, rq); }
block/ll_rw_blk.c +327 -141
571 lines not on this page · raw message
include/linux/blkdev.h +58 -25
148 lines not on this page · raw message
include/linux/elevator.h +1 -0
@@ -130,6 +130,7 @@ extern int elv_try_last_merge(request_qu#define ELEVATOR_INSERT_FRONT 1#define ELEVATOR_INSERT_BACK 2#define ELEVATOR_INSERT_SORT 3 +#define ELEVATOR_INSERT_REQUEUE 4/* * return values from elevator_may_queue_fn
Re: [PATCH linux-2.6-block:post-2.6.15 09/10] blk: add FUA support to IDE
In reply to Tejun Heo09_blk_ide-add-fua-support.patch
Add FUA support to IDE
drivers/ide/ide-disk.c | 57 +++++++++++++++++++++++++++++++++++++++++-------- include/linux/hdreg.h | 16 ++++++++++++- include/linux/ide.h | 3 ++ 3 files changed, 65 insertions(+), 11 deletions(-)
drivers/ide/ide-disk.c +48 -9
@@ -164,13 +164,14 @@ static ide_startstop_t __ide_do_rw_disk(ide_hwif_t *hwif =HWIF (drive);unsigned int dma = drive->using_dma;u8 lba48 = (drive->addressing ==1 ) ?1 :0 ; +int fua =blk_fua_rq (rq);task_ioreg_t command =WIN_NOP ;ata_nsector_t nsectors; nsectors.all = (u16 ) rq->nr_sectors;if (hwif->no_lba48_dma && lba48 && dma) { -if (block + rq->nr_sectors >1ULL <<28 ) +if (block + rq->nr_sectors >1ULL <<28 || fua) dma =0 ;else lba48 =0 ;
@@ -226,6 +227,16 @@ static ide_startstop_t __ide_do_rw_disk( hwif->OUTB (tasklets[6 ],IDE_HCYL_REG ); hwif->OUTB (0x00 |drive->select.all,IDE_SELECT_REG ); }else { +if (unlikely (fua)) { +/* +* This happens if LBA48 addressing is +* turned off during operation. +*/ +printk (KERN_ERR "%s: FUA write but LBA48 off\n" , + drive->name); + goto fail; + } + hwif->OUTB (0x00 ,IDE_FEATURE_REG ); hwif->OUTB (nsectors.b.low,IDE_NSECTOR_REG ); hwif->OUTB (block,IDE_SECTOR_REG );
@@ -253,9 +264,12 @@ static ide_startstop_t __ide_do_rw_disk(if (dma) {if (!hwif->dma_setup (drive)) {if (rq_data_dir (rq)) { - command = lba48 ?WIN_WRITEDMA_EXT :WIN_WRITEDMA ; -if (drive->vdma) - command = lba48 ?WIN_WRITE_EXT :WIN_WRITE ; +if (!fua) { + command = lba48 ?WIN_WRITEDMA_EXT :WIN_WRITEDMA ; +if (drive->vdma) + command = lba48 ?WIN_WRITE_EXT :WIN_WRITE ; + }else + command =ATA_CMD_WRITE_FUA_EXT ; }else { command = lba48 ?WIN_READDMA_EXT :WIN_READDMA ;if (drive->vdma)
@@ -284,8 +298,20 @@ static ide_startstop_t __ide_do_rw_disk( }else {if (drive->mult_count) { hwif->data_phase =TASKFILE_MULTI_OUT ; - command = lba48 ?WIN_MULTWRITE_EXT :WIN_MULTWRITE ; +if (!fua) + command = lba48 ?WIN_MULTWRITE_EXT :WIN_MULTWRITE ; +else + command =ATA_CMD_WRITE_MULTI_FUA_EXT ; }else { +if (unlikely (fua)) { +/* +* This happens if multisector PIO is +* turned off during operation. +*/ +printk (KERN_ERR "%s: FUA write but in single " +"sector PIO mode\n" , drive->name); + goto fail; + } hwif->data_phase =TASKFILE_OUT ; command = lba48 ?WIN_WRITE_EXT :WIN_WRITE ; }
@@ -295,6 +321,10 @@ static ide_startstop_t __ide_do_rw_disk(return pre_task_out_intr (drive, rq); } + + fail: +ide_end_request (drive,0 ,0 ); +return ide_stopped; }/*
@@ -846,7 +876,7 @@ static void idedisk_setup (ide_drive_t * {struct hd_driveid *id = drive->id;unsigned long long capacity; -int barrier; +int barrier, fua;idedisk_add_settings (drive);
@@ -967,10 +997,19 @@ static void idedisk_setup (ide_drive_t * barrier =0 ; } -printk (KERN_INFO "%s: cache flushes %ssupported\n" , - drive->name, barrier ?"" :"not " ); + fua = barrier &&idedisk_supports_lba48 (id) &&ide_id_has_fua (id); +/* When using PIO, FUA needs multisector. */ +if ((!drive->using_dma || drive->hwif->no_lba48_dma) && + drive->mult_count ==0 ) + fua =0 ; + +printk (KERN_INFO "%s: cache flushes %ssupported%s\n" , + drive->name, barrier ?"" :"not " , + fua ?" w/ FUA" :"" );if (barrier) { -blk_queue_ordered (drive->queue,QUEUE_ORDERED_DRAIN_FLUSH , +unsigned ordered = fua ?QUEUE_ORDERED_DRAIN_FUA + :QUEUE_ORDERED_DRAIN_FLUSH ; +blk_queue_ordered (drive->queue, ordered, idedisk_prepare_flush,GFP_KERNEL );blk_queue_issue_flush_fn (drive->queue, idedisk_issue_flush); }else if (!drive->wcache)
include/linux/hdreg.h +14 -2
@@ -550,7 +550,13 @@ struct hd_driveid {* cmd set-feature supported extensions * 15: Shall be ZERO * 14: Shall be ONE -* 13:6 reserved +* 13: IDLE IMMEDIATE w/ UNLOAD FEATURE +* 12:11 reserved for technical report +* 10: URG for WRITE STREAM +* 9: URG for READ STREAM +* 8: 64-bit World wide name +* 7: WRITE DMA QUEUED FUA EXT +* 6: WRITE DMA/MULTIPLE FUA EXT * 5: General Purpose Logging * 4: Streaming Feature Set * 3: Media Card Pass Through
@@ -600,7 +606,13 @@ struct hd_driveid {* command set-feature default * 15: Shall be ZERO * 14: Shall be ONE -* 13:6 reserved +* 13: IDLE IMMEDIATE w/ UNLOAD FEATURE +* 12:11 reserved for technical report +* 10: URG for WRITE STREAM +* 9: URG for READ STREAM +* 8: 64-bit World wide name +* 7: WRITE DMA QUEUED FUA EXT +* 6: WRITE DMA/MULTIPLE FUA EXT * 5: General Purpose Logging enabled * 4: Valid CONFIGURE STREAM executed * 3: Media Card Pass Through enabled
include/linux/ide.h +3 -0
@@ -1503,6 +1503,9 @@ extern struct bus_type ide_bus_type;/* check if CACHE FLUSH (EXT) command is supported (bits defined in ATA-6) */ #define ide_id_has_flush_cache (id) ((id)->cfs_enable_2 & 0x3000) +/* check if WRITE DMA FUA EXT command is supported (defined in ATA-8) */ +#define ide_id_has_fua (id) ((id)->cfsse & 0x0040) +/* some Maxtor disks have bit 13 defined incorrectly so check bit 10 too */ #define ide_id_has_flush_cache_ext (id) \ (((id)->cfs_enable_2 & 0x2400) == 0x2400)
Re: [PATCH linux-2.6-block:post-2.6.15 09/10] blk: add FUA support to IDE
Quoted 51 lines · Tejun Heo
09_blk_ide-add-fua-support.patch
Add FUA support to IDE
Signed-off-by Tejun Heo <[email protected]>drivers/ide/ide-disk.c | 57 +++++++++++++++++++++++++++++++++++++++++-------- include/linux/hdreg.h | 16 ++++++++++++- include/linux/ide.h | 3 ++ 3 files changed, 65 insertions(+), 11 deletions(-)Index:
work/drivers/ide/ide-disk.c
===================================================================drivers/ide/ide-disk.c 2005-11-18 00:35:07.000000000 +0900 renamed from drivers/ide/ide-disk.c 2005-11-18 00:35:06.000000000 +0900 +12
-1@@ -164,13 +164,14 @@ static ide_startstop_t __ide_do_rw_disk( ide_hwif_t *hwif = HWIF(drive); unsigned int dma = drive->using_dma; u8 lba48 = (drive->addressing == 1) ? 1 : 0; + int fua = blk_fua_rq(rq); task_ioreg_t command = WIN_NOP; ata_nsector_t nsectors; nsectors.all = (u16) rq->nr_sectors; if (hwif->no_lba48_dma && lba48 && dma) { - if (block + rq->nr_sectors > 1ULL << 28) + if (block + rq->nr_sectors > 1ULL << 28 || fua) dma = 0; else lba48 = 0;@@ -226,6 +227,16 @@ static ide_startstop_t __ide_do_rw_disk( hwif->OUTB(tasklets[6], IDE_HCYL_REG); hwif->OUTB(0x00|drive->select.all,IDE_SELECT_REG); } else { + if (unlikely(fua)) { + /* + * This happens if LBA48 addressing is + * turned off during operation. + */ + printk(KERN_ERR "%s: FUA write but LBA48 off\n", + drive->name); + goto fail; + } + hwif->OUTB(0x00, IDE_FEATURE_REG); hwif->OUTB(nsectors.b.low, IDE_NSECTOR_REG); hwif->OUTB(block, IDE_SECTOR_REG);@@ -253,9 +264,12 @@ static ide_startstop_t __ide_do_rw_disk( if (dma) { if (!hwif->dma_setup(drive)) {
drivers/ide/ide-disk.c 2005-11-18 00:35:07.000000000 +0900 static ide_startstop_t __ide_do_rw_disk( · line 272
Quoted 10 lines
drivers/ide/ide-disk.c 2005-11-18 00:35:07.000000000 +0900 renamed from drivers/ide/ide-disk.c 2005-11-18 00:35:06.000000000 +0900 +6
-3if (rq_data_dir(rq)) { - command = lba48 ? WIN_WRITEDMA_EXT : WIN_WRITEDMA; - if (drive->vdma) - command = lba48 ? WIN_WRITE_EXT: WIN_WRITE; + if (!fua) { + command = lba48 ? WIN_WRITEDMA_EXT : WIN_WRITEDMA; + if (drive->vdma) + command = lba48 ? WIN_WRITE_EXT: WIN_WRITE; + } else + command = ATA_CMD_WRITE_FUA_EXT;
What does happen for fua && drive->vdma case?
Quoted 12 lines
drivers/ide/ide-disk.c 2005-11-18 00:35:07.000000000 +0900 renamed from drivers/ide/ide-disk.c 2005-11-18 00:35:06.000000000 +0900 +4
-1} else { command = lba48 ? WIN_READDMA_EXT : WIN_READDMA; if (drive->vdma)@@ -284,8 +298,20 @@ static ide_startstop_t __ide_do_rw_disk( } else { if (drive->mult_count) { hwif->data_phase = TASKFILE_MULTI_OUT; - command = lba48 ? WIN_MULTWRITE_EXT : WIN_MULTWRITE; + if (!fua) + command = lba48 ? WIN_MULTWRITE_EXT : WIN_MULTWRITE; + else + command = ATA_CMD_WRITE_MULTI_FUA_EXT;
drivers/ide/ide-disk.c 2005-11-18 00:35:07.000000000 +0900 static ide_startstop_t __ide_do_rw_disk( · line 314
Quoted 10 lines
drivers/ide/ide-disk.c 2005-11-18 00:35:07.000000000 +0900 renamed from drivers/ide/ide-disk.c 2005-11-18 00:35:06.000000000 +0900 +9
-0} else { + if (unlikely(fua)) { + /* + * This happens if multisector PIO is + * turned off during operation. + */ + printk(KERN_ERR "%s: FUA write but in single " + "sector PIO mode\n", drive->name); + goto fail; + }
Wouldn't it be better to do the following check at the beginning of __ide_do_rw_disk() (after checking for dma vs lba48):
if (fua) {
if (!lba48 || ((!dma || drive->vdma) && !drive->mult_count))
goto fail_fua;
}...
and fail the request if needed *before* actually touching any hardware registers?
fail_fua:
printk(KERN_ERR "%s: FUA write unsupported (lba48=%u dma=%u"
" vdma=%u mult_count=%u)\n", drive->name,
lba48, dma, drive->vdma,
drive->mult_count);
ide_end_request(drive, 0, 0);
return ide_stopped;Quoted 24 lines
drivers/ide/ide-disk.c 2005-11-18 00:35:07.000000000 +0900 renamed from drivers/ide/ide-disk.c 2005-11-18 00:35:06.000000000 +0900 +5
-1hwif->data_phase = TASKFILE_OUT; command = lba48 ? WIN_WRITE_EXT : WIN_WRITE; }@@ -295,6 +321,10 @@ static ide_startstop_t __ide_do_rw_disk( return pre_task_out_intr(drive, rq); } + + fail: + ide_end_request(drive, 0, 0); + return ide_stopped; } /*@@ -846,7 +876,7 @@ static void idedisk_setup (ide_drive_t * { struct hd_driveid *id = drive->id; unsigned long long capacity; - int barrier; + int barrier, fua; idedisk_add_settings(drive);
drivers/ide/ide-disk.c 2005-11-18 00:35:07.000000000 +0900 static void idedisk_setup (ide_drive_t * · line 1004
Quoted 10 lines
drivers/ide/ide-disk.c 2005-11-18 00:35:07.000000000 +0900 renamed from drivers/ide/ide-disk.c 2005-11-18 00:35:06.000000000 +0900 +5
-2@@ -967,10 +997,19 @@ static void idedisk_setup (ide_drive_t * barrier = 0; } - printk(KERN_INFO "%s: cache flushes %ssupported\n", - drive->name, barrier ? "" : "not "); + fua = barrier && idedisk_supports_lba48(id) && ide_id_has_fua(id); + /* When using PIO, FUA needs multisector. */ + if ((!drive->using_dma || drive->hwif->no_lba48_dma) && + drive->mult_count == 0) + fua = 0;
Shouldn't this check also for drive->vdma?
Quoted 62 lines
drivers/ide/ide-disk.c 2005-11-18 00:35:07.000000000 +0900 renamed from drivers/ide/ide-disk.c 2005-11-18 00:35:06.000000000 +0900 +7
-1+ + printk(KERN_INFO "%s: cache flushes %ssupported%s\n", + drive->name, barrier ? "" : "not ", + fua ? " w/ FUA" : ""); if (barrier) { - blk_queue_ordered(drive->queue, QUEUE_ORDERED_DRAIN_FLUSH, + unsigned ordered = fua ? QUEUE_ORDERED_DRAIN_FUA + : QUEUE_ORDERED_DRAIN_FLUSH; + blk_queue_ordered(drive->queue, ordered, idedisk_prepare_flush, GFP_KERNEL); blk_queue_issue_flush_fn(drive->queue, idedisk_issue_flush); } else if (!drive->wcache)Index:
work/include/linux/hdreg.h
===================================================================include/linux/hdreg.h 2005-11-18 00:35:07.000000000 +0900 renamed from include/linux/hdreg.h 2005-11-18 00:06:46.000000000 +0900 +14
-2@@ -550,7 +550,13 @@ struct hd_driveid { * cmd set-feature supported extensions * 15: Shall be ZERO * 14: Shall be ONE - * 13:6 reserved + * 13: IDLE IMMEDIATE w/ UNLOAD FEATURE + * 12:11 reserved for technical report + * 10: URG for WRITE STREAM + * 9: URG for READ STREAM + * 8: 64-bit World wide name + * 7: WRITE DMA QUEUED FUA EXT + * 6: WRITE DMA/MULTIPLE FUA EXT * 5: General Purpose Logging * 4: Streaming Feature Set * 3: Media Card Pass Through@@ -600,7 +606,13 @@ struct hd_driveid { * command set-feature default * 15: Shall be ZERO * 14: Shall be ONE - * 13:6 reserved + * 13: IDLE IMMEDIATE w/ UNLOAD FEATURE + * 12:11 reserved for technical report + * 10: URG for WRITE STREAM + * 9: URG for READ STREAM + * 8: 64-bit World wide name + * 7: WRITE DMA QUEUED FUA EXT + * 6: WRITE DMA/MULTIPLE FUA EXT * 5: General Purpose Logging enabled * 4: Valid CONFIGURE STREAM executed * 3: Media Card Pass Through enabledIndex:
work/include/linux/ide.h
===================================================================include/linux/ide.h 2005-11-18 00:35:07.000000000 +0900 renamed from include/linux/ide.h 2005-11-18 00:14:29.000000000 +0900 +3
-0@@ -1503,6 +1503,9 @@ extern struct bus_type ide_bus_type; /* check if CACHE FLUSH (EXT) command is supported (bits defined in ATA-6) */ #define ide_id_has_flush_cache(id) ((id)->cfs_enable_2 & 0x3000) +/* check if WRITE DMA FUA EXT command is supported (defined in ATA-8) */ +#define ide_id_has_fua(id) ((id)->cfsse & 0x0040) + /* some Maxtor disks have bit 13 defined incorrectly so check bit 10 too */ #define ide_id_has_flush_cache_ext(id) \ (((id)->cfs_enable_2 & 0x2400) == 0x2400)
Re: [PATCH linux-2.6-block:post-2.6.15 09/10] blk: add FUA support to IDE
Hi, Bartlomiej.
Quoted 4 lines · Bartlomiej Zolnierkiewicz
On 11/17/05, Tejun Heo <[email protected]> wrote:
What does happen for fua &&
drive->vdmacase?
Thanks for pointing out, wasn't thinking about that. Hmmm... When using vdma, single-sector PIO commands are issued instead but there's no single-sector FUA PIO command. Would issuing ATA_CMD_WRITE_MULTI_FUA_EXT instead of ATA_CMD_WRITE_FUA_EXT work? Or should I just disable FUA on vdma case?
Quoted 38 lines
Quoted 13 lines
} else { command = lba48 ? WIN_READDMA_EXT : WIN_READDMA; if (drive->vdma)drivers/ide/ide-disk.c +4
-1@@ -284,8 +298,20 @@ static ide_startstop_t __ide_do_rw_disk( }else {if (drive->mult_count) { hwif->data_phase =TASKFILE_MULTI_OUT ; - command = lba48 ?WIN_MULTWRITE_EXT :WIN_MULTWRITE ; +if (!fua) + command = lba48 ?WIN_MULTWRITE_EXT :WIN_MULTWRITE ; +else + command =ATA_CMD_WRITE_MULTI_FUA_EXT ;drivers/ide/ide-disk.c static ide_startstop_t __ide_do_rw_disk( · line 314
Quoted 10 lines
drivers/ide/ide-disk.c +9
-0}else { +if (unlikely (fua)) { +/* +* This happens if multisector PIO is +* turned off during operation. +*/ +printk (KERN_ERR "%s: FUA write but in single " +"sector PIO mode\n" , drive->name); + goto fail; + }Wouldn't it be better to do the following check at the beginning of
__ide_do_rw_disk()(after checking for dma vs lba48):if (fua) { if (!lba48 || ((!dma || drive->vdma) && !drive->mult_count)) goto fail_fua; }...
and fail the request if needed *before* actually touching any hardware registers?
Quoted 8 lines
fail_fua:printk(KERN_ERR "%s: FUA write unsupported (lba48=%u dma=%u" " vdma=%u mult_count=%u)\n", drive->name, lba48, dma, drive->vdma, drive->mult_count); ide_end_request(drive, 0, 0); return ide_stopped;
Hmmm... The thing is that those failure cases will happen extremely rarely if at all. Remember this post?
http://marc.theaimsgroup.com/?l=linux-kernel&m=111798102108338&w=3
It's mostly guaranteed that those failure cases don't occur, so I thought avoiding IO on failure case wasn't that helpful.
Quoted 28 lines
Quoted 28 lines
drivers/ide/ide-disk.c +5
-1hwif->data_phase =TASKFILE_OUT ; command = lba48 ?WIN_WRITE_EXT :WIN_WRITE ; }@@ -295,6 +321,10 @@ static ide_startstop_t __ide_do_rw_disk(return pre_task_out_intr (drive, rq); } + + fail: +ide_end_request (drive,0 ,0 ); +return ide_stopped; }/* @@ -846,7 +876,7 @@ static void idedisk_setup (ide_drive_t * {struct hd_driveid *id = drive->id;unsigned long long capacity; -int barrier; +int barrier, fua;idedisk_add_settings (drive);@@ -967,10 +997,19 @@ static void idedisk_setup (ide_drive_t * barrier =0 ; }
Quoted 11 lines
drivers/ide/ide-disk.c static void idedisk_setup (ide_drive_t * · line 1004
Quoted 7 lines
drivers/ide/ide-disk.c +5
-2-printk (KERN_INFO "%s: cache flushes %ssupported\n" , - drive->name, barrier ?"" :"not " ); + fua = barrier &&idedisk_supports_lba48 (id) &&ide_id_has_fua (id); +/* When using PIO, FUA needs multisector. */ +if ((!drive->using_dma || drive->hwif->no_lba48_dma) && + drive->mult_count ==0 ) + fua =0 ;Shouldn't this check also for
drive->vdma?
Yes, it does. Thanks for pointing out. One question though. FUA support should be changed if using_dma/mult_count settings are changed.
As using_dma configuration is handled by IDE midlayer, we might need
to add a callback there. What do you think?
Signature · 1 line
tejun
Re: [PATCH linux-2.6-block:post-2.6.15 09/10] blk: add FUA support to IDE
Quoted 5 lines · Tejun Heo
Hi, Bartlomiej.
Quoted 2 lines · Bartlomiej Zolnierkiewicz
On 11/17/05, Tejun Heo <[email protected]> wrote:
Quoted 8 lines
Quoted 2 lines
What does happen for fua &&
drive->vdmacase?Thanks for pointing out, wasn't thinking about that. Hmmm... When using vdma, single-sector PIO commands are issued instead but there's no single-sector FUA PIO command. Would issuing
ATA_CMD_WRITE_MULTI_FUA_EXTinstead ofATA_CMD_WRITE_FUA_EXTwork? Or should I just disable FUA on vdma case?
Probably it should work fine given that drive->mult_count is on.
The only controller using drive->vdma in the current tree is cs5520 so you should confirm this with Mark Lord & Alan Cox.
Quoted 47 lines
Quoted 38 lines
Quoted 13 lines
} else { command = lba48 ? WIN_READDMA_EXT : WIN_READDMA; if (drive->vdma)drivers/ide/ide-disk.c +4
-1@@ -284,8 +298,20 @@ static ide_startstop_t __ide_do_rw_disk( }else {if (drive->mult_count) { hwif->data_phase =TASKFILE_MULTI_OUT ; - command = lba48 ?WIN_MULTWRITE_EXT :WIN_MULTWRITE ; +if (!fua) + command = lba48 ?WIN_MULTWRITE_EXT :WIN_MULTWRITE ; +else + command =ATA_CMD_WRITE_MULTI_FUA_EXT ;drivers/ide/ide-disk.c static ide_startstop_t __ide_do_rw_disk( · line 314
Quoted 10 lines
drivers/ide/ide-disk.c +9
-0}else { +if (unlikely (fua)) { +/* +* This happens if multisector PIO is +* turned off during operation. +*/ +printk (KERN_ERR "%s: FUA write but in single " +"sector PIO mode\n" , drive->name); + goto fail; + }Wouldn't it be better to do the following check at the beginning of
__ide_do_rw_disk()(after checking for dma vs lba48):if (fua) { if (!lba48 || ((!dma || drive->vdma) && !drive->mult_count)) goto fail_fua; }...
and fail the request if needed *before* actually touching any hardware registers?
Quoted 8 lines
fail_fua:printk(KERN_ERR "%s: FUA write unsupported (lba48=%u dma=%u" " vdma=%u mult_count=%u)\n", drive->name, lba48, dma, drive->vdma, drive->mult_count); ide_end_request(drive, 0, 0); return ide_stopped;
Quoted 7 lines
Hmmm... The thing is that those failure cases will happen extremely rarely if at all. Remember this post?
http://marc.theaimsgroup.com/?l=linux-kernel&m=111798102108338&w=3
It's mostly guaranteed that those failure cases don't occur, so I thought avoiding IO on failure case wasn't that helpful.
There are two problems with this approach:
* configuration can change dynamically
* locking for configuration changes is flakey
so it can happen that FUA request will slip into __ide_do_rw_disk().
The best way to deal with such case is to kill it early without touching I/O registers and/or DMA engine and causing big havoc.
Quoted 16 lines
Quoted 4 lines
Quoted 4 lines
drivers/ide/ide-disk.c +0
-0@@ -967,10 +997,19 @@ static void idedisk_setup (ide_drive_t * barrier =0 ; }Quoted 11 lines
drivers/ide/ide-disk.c static void idedisk_setup (ide_drive_t * · line 1004
Quoted 7 lines
drivers/ide/ide-disk.c +5
-2-printk (KERN_INFO "%s: cache flushes %ssupported\n" , - drive->name, barrier ?"" :"not " ); + fua = barrier &&idedisk_supports_lba48 (id) &&ide_id_has_fua (id); +/* When using PIO, FUA needs multisector. */ +if ((!drive->using_dma || drive->hwif->no_lba48_dma) && + drive->mult_count ==0 ) + fua =0 ;Shouldn't this check also for
drive->vdma?
Quoted 4 lines
Yes, it does. Thanks for pointing out. One question though. FUA support should be changed if using_dma/mult_count settings are changed.
Asusing_dmaconfiguration is handled by IDE midlayer, we might need
to add a callback there. What do you think?
It seems it is needed nowadays as there are multiple I/O barrier methods.
Maybe the other alternative is to add ->rq_select_barrier() hook to the block layer and for each request check what kind of barrier should be issued (it still won't help for flakey configuration locking but you won't have to add any callbacks etc).
Long-term we should see if it is possible to remove dynamic IDE drive configuration and always just use the best possible settings.
Thanks,
Bartlomiej
Re: [PATCH linux-2.6-block:post-2.6.15 09/10] blk: add FUA support to IDE
Quoted 4 lines · Bartlomiej Zolnierkiewicz
Probably it should work fine given that
drive->mult_countis on.The only controller using
drive->vdmain the current tree is cs5520 so you should confirm this with Mark Lord & Alan Cox.
The CS5520 VDMA performs PIO commands with controller driven DMA to PIO of the data blocks. Thus it can do any PIO command with one data in or out phase as if it were DMA.
Quoted 2 lines
Long-term we should see if it is possible to remove dynamic IDE drive configuration and always just use the best possible settings.
Hotplug rather prevents that. For the lifetime of a drive connection most of the settings ought to be settable once. Speeds are the obvious exception.
Re: [PATCH linux-2.6-block:post-2.6.15 09/10] blk: add FUA support to IDE
Quoted 9 lines · Alan Cox
Quoted 4 lines · Bartlomiej Zolnierkiewicz
Probably it should work fine given that
drive->mult_countis on.The only controller using
drive->vdmain the current tree is cs5520 so you should confirm this with Mark Lord & Alan Cox.The CS5520 VDMA performs PIO commands with controller driven DMA to PIO of the data blocks. Thus it can do any PIO command with one data in or out phase as if it were DMA.
Therefore doing ATA_CMD_WRITE_MULTI_FUA_EXT w/ VDMA should be just fine as is WIN_WRITE_EXT w/ VDMA currently?
Quoted 6 lines
Quoted 2 lines
Long-term we should see if it is possible to remove dynamic IDE drive configuration and always just use the best possible settings.
Hotplug rather prevents that. For the lifetime of a drive connection most of the settings ought to be settable once. Speeds are the obvious exception.
I'm talking only about lifetime of a drive connection here so I think that we fully agree on this one.
Thanks,
Bartlomiej
Re: [PATCH linux-2.6-block:post-2.6.15 09/10] blk: add FUA support to IDE
Quoted 7 lines · Bartlomiej Zolnierkiewicz
Quoted 6 lines · Alan Cox
Quoted 4 lines · Bartlomiej Zolnierkiewicz
Probably it should work fine given that
drive->mult_countis on.The only controller using
drive->vdmain the current tree is cs5520 so you should confirm this with Mark Lord & Alan Cox.
Quoted 6 lines
Quoted 3 lines
The CS5520 VDMA performs PIO commands with controller driven DMA to PIO of the data blocks. Thus it can do any PIO command with one data in or out phase as if it were DMA.
Therefore doing
ATA_CMD_WRITE_MULTI_FUA_EXTw/ VDMA should be just fine as isWIN_WRITE_EXTw/ VDMA currently?
As I understand it yes
Re: [PATCH linux-2.6-block:post-2.6.15 09/10] blk: add FUA support to IDE
In reply to Bartlomiej ZolnierkiewiczHi, Bartlomiej.
Quoted 18 lines · Bartlomiej Zolnierkiewicz
Quoted 6 lines · Tejun Heo
Hi, Bartlomiej.
Quoted 2 lines · Bartlomiej Zolnierkiewicz
On 11/17/05, Tejun Heo <[email protected]> wrote:
Quoted 8 lines
Quoted 2 lines
What does happen for fua &&
drive->vdmacase?Thanks for pointing out, wasn't thinking about that. Hmmm... When using vdma, single-sector PIO commands are issued instead but there's no single-sector FUA PIO command. Would issuing
ATA_CMD_WRITE_MULTI_FUA_EXTinstead ofATA_CMD_WRITE_FUA_EXTwork? Or should I just disable FUA on vdma case?
Quoted 5 lines
Probably it should work fine given that
drive->mult_countis on.The only controller using
drive->vdmain the current tree is cs5520 so you should confirm this with Mark Lord & Alan Cox.
This is done without too much trouble.
Quoted 58 lines
Quoted 47 lines
Quoted 38 lines
Quoted 13 lines
} else { command = lba48 ? WIN_READDMA_EXT : WIN_READDMA; if (drive->vdma)drivers/ide/ide-disk.c +4
-1@@ -284,8 +298,20 @@ static ide_startstop_t __ide_do_rw_disk( }else {if (drive->mult_count) { hwif->data_phase =TASKFILE_MULTI_OUT ; - command = lba48 ?WIN_MULTWRITE_EXT :WIN_MULTWRITE ; +if (!fua) + command = lba48 ?WIN_MULTWRITE_EXT :WIN_MULTWRITE ; +else + command =ATA_CMD_WRITE_MULTI_FUA_EXT ;drivers/ide/ide-disk.c static ide_startstop_t __ide_do_rw_disk( · line 314
Quoted 10 lines
drivers/ide/ide-disk.c +9
-0}else { +if (unlikely (fua)) { +/* +* This happens if multisector PIO is +* turned off during operation. +*/ +printk (KERN_ERR "%s: FUA write but in single " +"sector PIO mode\n" , drive->name); + goto fail; + }Wouldn't it be better to do the following check at the beginning of
__ide_do_rw_disk()(after checking for dma vs lba48):if (fua) { if (!lba48 || ((!dma || drive->vdma) && !drive->mult_count)) goto fail_fua; }...
and fail the request if needed *before* actually touching any hardware registers?
Quoted 8 lines
fail_fua:printk(KERN_ERR "%s: FUA write unsupported (lba48=%u dma=%u" " vdma=%u mult_count=%u)\n", drive->name, lba48, dma, drive->vdma, drive->mult_count); ide_end_request(drive, 0, 0); return ide_stopped;Quoted 7 lines
Hmmm... The thing is that those failure cases will happen extremely rarely if at all. Remember this post?
http://marc.theaimsgroup.com/?l=linux-kernel&m=111798102108338&w=3
It's mostly guaranteed that those failure cases don't occur, so I thought avoiding IO on failure case wasn't that helpful.
There are two problems with this approach:
Quoted 8 lines
* configuration can change dynamically
* locking for configuration changes is flakeyso it can happen that FUA request will slip into
__ide_do_rw_disk().The best way to deal with such case is to kill it early without touching I/O registers and/or DMA engine and causing big havoc.
I've taken your advice and added the test at the top.
Quoted 25 lines
Quoted 17 lines
Quoted 5 lines
Quoted 5 lines
drivers/ide/ide-disk.c +0
-0@@ -967,10 +997,19 @@ static void idedisk_setup (ide_drive_t * barrier =0 ; }Quoted 11 lines
drivers/ide/ide-disk.c static void idedisk_setup (ide_drive_t * · line 1004
Quoted 7 lines
drivers/ide/ide-disk.c +5
-2-printk (KERN_INFO "%s: cache flushes %ssupported\n" , - drive->name, barrier ?"" :"not " ); + fua = barrier &&idedisk_supports_lba48 (id) &&ide_id_has_fua (id); +/* When using PIO, FUA needs multisector. */ +if ((!drive->using_dma || drive->hwif->no_lba48_dma) && + drive->mult_count ==0 ) + fua =0 ;Shouldn't this check also for
drive->vdma?Quoted 4 lines
Yes, it does. Thanks for pointing out. One question though. FUA support should be changed if using_dma/mult_count settings are changed.
Asusing_dmaconfiguration is handled by IDE midlayer, we might need
to add a callback there. What do you think?It seems it is needed nowadays as there are multiple I/O barrier methods.
Quoted 7 lines
Maybe the other alternative is to add ->rq_select_barrier() hook to the block layer and for each request check what kind of barrier should be issued (it still won't help for flakey configuration locking but you won't have to add any callbacks etc).
Long-term we should see if it is possible to remove dynamic IDE drive configuration and always just use the best possible settings.
Well, this one is quite a pain in the ass.
I'm not very fond of ->rq_select_barrier() approach for the following reasons.
* That removes possibility of correct synchronization. With blk_queue_ordered() approach, we can later add blk_queue_[un]lock_ordered() to achieve correct synchronization if that becomes necessary, but with ->rq_select_barrier() approach, the low-level driver ends up having less control over what's gonna happen when.
* Changing ordered mode is not supposed to be a frequent operation and the blk_queue_ordered() interface makes that explicit.
So, I added ide_driver_t->protocol_changed() callback which gets called whenever dma/multimode changes occur. Unfortunately, dma/multmode changes can be committed with or without context, and with or without queue lock. As blk_queue_ordered uses the queue lock for synchronization, this becomes issue.
I tried to distinguish places where the changes occur while queue lock is held from the other. Not only was it highly error-prone, it couldn't be done without modifying/auditing all low-level drivers as some drivers (cs5520) use the same function which touches dma setting (cs5520_tune_chipset) from both ->speedproc (called with queuelock) and ->ide_dma_check (called without queuelock).
One alternative I'm thinking of is using a workqueue to call blk_queue_ordered, such that we don't have to guess whether or not we're called with queuelock held. Unfortunately, this will give us a small window where wrong barrier requests can hit the drive.
Bartlomiej, any ideas?
Jens, as this one seems to need some time to settle, I'm gonna post updated patchset for post-2.6.15 without ide-fua patch, so that the other stuff can be pushed into -mm. I think we can live without ide-fua for a while. :-)
Thanks.
Signature · 1 line
tejun
Re: [PATCH linux-2.6-block:post-2.6.15 09/10] blk: add FUA support to IDE
Oops, I was delusional again.
Quoted 6 lines · Tejun Heo
Well, this one is quite a pain in the ass.
I'm not very fond of ->rq_select_barrier() approach for the following reasons.
* That removes possibility of correct synchronization. With
blk_queue_ordered()approach, we can later add blk_queue_[un]lock_ordered() to achieve correct synchronization if that becomes necessary, but with ->rq_select_barrier() approach, the low-level driver ends up having less control over what's gonna happen when.
Of course, we can do the same lock/unlock dance with ->rq_select_barrier() approach. I wasn't thinking straight. Forget this rationale.
Quoted 9 lines
* Changing ordered mode is not supposed to be a frequent operation and the
blk_queue_ordered()interface makes that explicit.So, I added
ide_driver_t->protocol_changed()callback which gets called whenever dma/multimode changes occur. Unfortunately, dma/multmode changes can be committed with or without context, and with or without queue lock. Asblk_queue_ordereduses the queue lock for synchronization, this becomes issue.I tried to distinguish places where the changes occur while queue lock is held from the other. Not only was it highly error-prone, it couldn't be done without modifying/auditing all low-level drivers as some drivers (cs5520) use the same function which touches dma setting (
cs5520_tune_chipset) from both ->speedproc (called with queuelock) and ->ide_dma_check (called without queuelock).One alternative I'm thinking of is using a workqueue to call
blk_queue_ordered, such that we don't have to guess whether or not we're called with queuelock held. Unfortunately, this will give us a small window where wrong barrier requests can hit the drive.
One thing I wanna add here is that using ->rq_select_barrier() would have similar race window. The race windows is just hidden there in the request queue.
Quoted 7 lines
Bartlomiej, any ideas?
Jens, as this one seems to need some time to settle, I'm gonna post updated patchset for post-2.6.15 without ide-fua patch, so that the other stuff can be pushed into -mm. I think we can live without ide-fua for a while. :-)
Thanks.
Signature · 1 line
tejun
Re: [PATCH linux-2.6-block:post-2.6.15 09/10] blk: add FUA support to IDE
Quoted 37 lines · Tejun Heo
Oops, I was delusional again.
Quoted 11 lines · Tejun Heo
Well, this one is quite a pain in the ass.
I'm not very fond of ->rq_select_barrier() approach for the following reasons.
* That removes possibility of correct synchronization. With
blk_queue_ordered()approach, we can later add blk_queue_[un]lock_ordered() to achieve correct synchronization if that becomes necessary, but with ->rq_select_barrier() approach, the low-level driver ends up having less control over what's gonna happen when.Of course, we can do the same lock/unlock dance with ->rq_select_barrier() approach. I wasn't thinking straight. Forget this rationale.
Quoted 17 lines
* Changing ordered mode is not supposed to be a frequent operation and the
blk_queue_ordered()interface makes that explicit.So, I added
ide_driver_t->protocol_changed()callback which gets called whenever dma/multimode changes occur. Unfortunately, dma/multmode changes can be committed with or without context, and with or without queue lock. Asblk_queue_ordereduses the queue lock for synchronization, this becomes issue.I tried to distinguish places where the changes occur while queue lock is held from the other. Not only was it highly error-prone, it couldn't be done without modifying/auditing all low-level drivers as some drivers (cs5520) use the same function which touches dma setting (
cs5520_tune_chipset) from both ->speedproc (called with queuelock) and ->ide_dma_check (called without queuelock).
Quoted 11 lines
Quoted 4 lines
One alternative I'm thinking of is using a workqueue to call
blk_queue_ordered, such that we don't have to guess whether or not we're called with queuelock held. Unfortunately, this will give us a small window where wrong barrier requests can hit the drive.One thing I wanna add here is that using ->rq_select_barrier() would have similar race window. The race windows is just hidden there in the request queue.
Quoted 2 lines
Bartlomiej, any ideas?
Wouldn't the whole thing get magically fixed with conversion of IDE settings to use REQ_DRIVER and making them pass through request queue (old idea everybody seems to like but nobody wants to implement :)?
Quoted 4 lines
Quoted 4 lines
Jens, as this one seems to need some time to settle, I'm gonna post updated patchset for post-2.6.15 without ide-fua patch, so that the other stuff can be pushed into -mm. I think we can live without ide-fua for a while. :-)
You can as well post updated ide-fua patch.
I would like to see it, it may be OK to put it in -mm for now.
Thanks,
Bartlomiej
Re: [PATCH linux-2.6-block:post-2.6.15 08/10] blk: update IDE to use new blk_ordered
In reply to Tejun Heo08_blk_ide-update-ordered.patch
Update IDE to use new blk_ordered.
ide-disk.c | 57 +++++++++------------------------------------------------ ide-io.c | 5 +---- 2 files changed, 10 insertions(+), 52 deletions(-)
drivers/ide/ide-disk.c +9 -48
@@ -681,50 +681,9 @@ static ide_proc_entry_t idedisk_proc[] =#endif /* CONFIG_PROC_FS */ -static void idedisk_end_flush (request_queue_t *q,struct request *flush_rq) +static void idedisk_prepare_flush (request_queue_t *q,struct request *rq) {ide_drive_t *drive = q->queuedata; -struct request *rq = flush_rq->end_io_data; -int good_sectors = rq->hard_nr_sectors; -int bad_sectors; -sector_t sector; - -if (flush_rq->errors &ABRT_ERR ) { -printk (KERN_ERR "%s: barrier support doesn't work\n" , drive->name); -blk_queue_ordered (drive->queue,QUEUE_ORDERED_NONE ); -blk_queue_issue_flush_fn (drive->queue,NULL ); - good_sectors =0 ; - }else if (flush_rq->errors) { - good_sectors =0 ; -if (blk_barrier_preflush (rq)) { - sector =ide_get_error_location (drive,flush_rq->buffer); -if ((sector >= rq->hard_sector) && - (sector < rq->hard_sector + rq->hard_nr_sectors)) - good_sectors = sector - rq->hard_sector; - } - } - -if (flush_rq->errors) -printk (KERN_ERR "%s: failed barrier write: " -"sector=%Lx(good=%d/bad=%d)\n" , - drive->name, (unsigned long long )rq->sector, - good_sectors, - (int ) (rq->hard_nr_sectors-good_sectors)); - - bad_sectors = rq->hard_nr_sectors - good_sectors; - -if (good_sectors) -__ide_end_request (drive, rq,1 , good_sectors); -if (bad_sectors) -__ide_end_request (drive, rq,0 , bad_sectors); -} - -static int idedisk_prepare_flush (request_queue_t *q,struct request *rq) -{ -ide_drive_t *drive = q->queuedata; - -if (!drive->wcache) -return 0 ;memset (rq->cmd,0 ,sizeof (rq->cmd));
@@ -735,9 +694,8 @@ static int idedisk_prepare_flush(request rq->cmd[0 ] =WIN_FLUSH_CACHE ; - rq->flags |=REQ_DRIVE_TASK |REQ_SOFTBARRIER ; + rq->flags |=REQ_DRIVE_TASK ; rq->buffer = rq->cmd; -return 1 ; } static int idedisk_issue_flush(request_queue_t *q, struct gendisk *disk,
@@ -1012,11 +970,12 @@ static void idedisk_setup (ide_drive_t *printk (KERN_INFO "%s: cache flushes %ssupported\n" , drive->name, barrier ?"" :"not " );if (barrier) { -blk_queue_ordered (drive->queue,QUEUE_ORDERED_FLUSH ); - drive->queue->prepare_flush_fn = idedisk_prepare_flush; - drive->queue->end_flush_fn = idedisk_end_flush; +blk_queue_ordered (drive->queue,QUEUE_ORDERED_DRAIN_FLUSH , + idedisk_prepare_flush,GFP_KERNEL );blk_queue_issue_flush_fn (drive->queue, idedisk_issue_flush); - } + }else if (!drive->wcache) +blk_queue_ordered (drive->queue,QUEUE_ORDERED_DRAIN , +NULL ,GFP_KERNEL ); }static void ide_cacheflush_p (ide_drive_t *drive)
@@ -1034,6 +993,8 @@ static int ide_disk_remove(struct devicestruct ide_disk_obj *idkp = drive->driver_data;struct gendisk *g = idkp->disk; +blk_queue_ordered (drive->queue,QUEUE_ORDERED_NONE ,NULL ,0 ); +ide_cacheflush_p (drive);ide_unregister_subdriver (drive, idkp->driver);
drivers/ide/ide-io.c +1 -4
@@ -119,10 +119,7 @@ int ide_end_request (ide_drive_t *drive,if (!nr_sectors) nr_sectors = rq->hard_cur_sectors; -if (blk_complete_barrier_rq_locked (drive->queue, rq, nr_sectors)) - ret = rq->nr_sectors !=0 ; -else - ret =__ide_end_request (drive, rq, uptodate, nr_sectors); + ret =__ide_end_request (drive, rq, uptodate, nr_sectors);spin_unlock_irqrestore (&ide_lock, flags);return ret;
Re: [PATCH linux-2.6-block:post-2.6.15 08/10] blk: update IDE to use new blk_ordered
Hi,
Quoted 46 lines · Tejun Heo
08_blk_ide-update-ordered.patch
Update IDE to use new
blk_ordered.Signed-off-by Tejun Heo <[email protected]>ide-disk.c | 57 +++++++++------------------------------------------------ ide-io.c | 5 +---- 2 files changed, 10 insertions(+), 52 deletions(-)Index:
work/drivers/ide/ide-disk.c
===================================================================drivers/ide/ide-disk.c 2005-11-18 00:35:06.000000000 +0900 renamed from drivers/ide/ide-disk.c 2005-11-18 00:14:21.000000000 +0900 +1
-25@@ -681,50 +681,9 @@ static ide_proc_entry_t idedisk_proc[] = #endif /* CONFIG_PROC_FS */ -static void idedisk_end_flush(request_queue_t *q, struct request *flush_rq) +static void idedisk_prepare_flush(request_queue_t *q, struct request *rq) { ide_drive_t *drive = q->queuedata; - struct request *rq = flush_rq->end_io_data; - int good_sectors = rq->hard_nr_sectors; - int bad_sectors; - sector_t sector; - - if (flush_rq->errors & ABRT_ERR) { - printk(KERN_ERR "%s: barrier support doesn't work\n", drive->name); - blk_queue_ordered(drive->queue, QUEUE_ORDERED_NONE); - blk_queue_issue_flush_fn(drive->queue, NULL); - good_sectors = 0; - } else if (flush_rq->errors) { - good_sectors = 0; - if (blk_barrier_preflush(rq)) { - sector = ide_get_error_location(drive,flush_rq->buffer); - if ((sector >= rq->hard_sector) && - (sector < rq->hard_sector + rq->hard_nr_sectors)) - good_sectors = sector - rq->hard_sector; - } - } - - if (flush_rq->errors) - printk(KERN_ERR "%s: failed barrier write: " - "sector=%Lx(good=%d/bad=%d)\n", - drive->name, (unsigned long long)rq->sector,
drivers/ide/ide-disk.c 2005-11-18 00:35:06.000000000 +0900 static ide_proc_entry_t idedisk_proc[] = · line 720
Quoted 10 lines
drivers/ide/ide-disk.c 2005-11-18 00:35:06.000000000 +0900 renamed from drivers/ide/ide-disk.c 2005-11-18 00:14:21.000000000 +0900 +0
-10- good_sectors, - (int) (rq->hard_nr_sectors-good_sectors)); - - bad_sectors = rq->hard_nr_sectors - good_sectors; - - if (good_sectors) - __ide_end_request(drive, rq, 1, good_sectors); - if (bad_sectors) - __ide_end_request(drive, rq, 0, bad_sectors); -}
I fail to see how the partial completions (good + bad sectors) are done in your new scheme, please explain.
drivers/ide/ide-disk.c 2005-11-18 00:35:06.000000000 +0900 static ide_proc_entry_t idedisk_proc[] = · line 727
Quoted 7 lines
drivers/ide/ide-disk.c 2005-11-18 00:35:06.000000000 +0900 renamed from drivers/ide/ide-disk.c 2005-11-18 00:14:21.000000000 +0900 +0
-7- -static int idedisk_prepare_flush(request_queue_t *q, struct request *rq) -{ - ide_drive_t *drive = q->queuedata; - - if (!drive->wcache) - return 0;
What does happen if somebody disables drive->wcache later?
Quoted 17 lines
drivers/ide/ide-disk.c 2005-11-18 00:35:06.000000000 +0900 renamed from drivers/ide/ide-disk.c 2005-11-18 00:14:21.000000000 +0900 +1
-2memset(rq->cmd, 0, sizeof(rq->cmd));@@ -735,9 +694,8 @@ static int idedisk_prepare_flush(request rq->cmd[0] = WIN_FLUSH_CACHE; - rq->flags |= REQ_DRIVE_TASK | REQ_SOFTBARRIER; + rq->flags |= REQ_DRIVE_TASK; rq->buffer = rq->cmd; - return 1; } static int idedisk_issue_flush(request_queue_t *q, struct gendisk *disk,@@ -1012,11 +970,12 @@ static void idedisk_setup (ide_drive_t * printk(KERN_INFO "%s: cache flushes %ssupported\n", drive->name, barrier ? "" : "not "); if (barrier) {
drivers/ide/ide-disk.c 2005-11-18 00:35:06.000000000 +0900 static void idedisk_setup (ide_drive_t * · line 978
Quoted 10 lines
drivers/ide/ide-disk.c 2005-11-18 00:35:06.000000000 +0900 renamed from drivers/ide/ide-disk.c 2005-11-18 00:14:21.000000000 +0900 +5
-4- blk_queue_ordered(drive->queue, QUEUE_ORDERED_FLUSH); - drive->queue->prepare_flush_fn = idedisk_prepare_flush; - drive->queue->end_flush_fn = idedisk_end_flush; + blk_queue_ordered(drive->queue, QUEUE_ORDERED_DRAIN_FLUSH, + idedisk_prepare_flush, GFP_KERNEL); blk_queue_issue_flush_fn(drive->queue, idedisk_issue_flush); - } + } else if (!drive->wcache) + blk_queue_ordered(drive->queue, QUEUE_ORDERED_DRAIN, + NULL, GFP_KERNEL);
What does happen if somebody enables drive->wcache later?
drivers/ide/ide-disk.c 2005-11-18 00:35:06.000000000 +0900 static int ide_disk_remove(struct device · line 997
Quoted 9 lines
drivers/ide/ide-disk.c 2005-11-18 00:35:06.000000000 +0900 renamed from drivers/ide/ide-disk.c 2005-11-18 00:14:21.000000000 +0900 +2
-0} static void ide_cacheflush_p(ide_drive_t *drive)@@ -1034,6 +993,8 @@ static int ide_disk_remove(struct device struct ide_disk_obj *idkp = drive->driver_data; struct gendisk *g = idkp->disk; + blk_queue_ordered(drive->queue, QUEUE_ORDERED_NONE, NULL, 0); +
Shouldn't this be done in ide_disk_release()?
Quoted 20 lines
drivers/ide/ide-disk.c 2005-11-18 00:35:06.000000000 +0900 renamed from drivers/ide/ide-disk.c 2005-11-18 00:14:21.000000000 +0900 +0
-0ide_cacheflush_p(drive); ide_unregister_subdriver(drive, idkp->driver);Index:
work/drivers/ide/ide-io.c
===================================================================drivers/ide/ide-io.c 2005-11-18 00:35:06.000000000 +0900 renamed from drivers/ide/ide-io.c 2005-11-18 00:35:04.000000000 +0900 +1
-4@@ -119,10 +119,7 @@ int ide_end_request (ide_drive_t *drive, if (!nr_sectors) nr_sectors = rq->hard_cur_sectors; - if (blk_complete_barrier_rq_locked(drive->queue, rq, nr_sectors)) - ret = rq->nr_sectors != 0; - else - ret = __ide_end_request(drive, rq, uptodate, nr_sectors); + ret = __ide_end_request(drive, rq, uptodate, nr_sectors); spin_unlock_irqrestore(&ide_lock, flags); return ret;
Re: [PATCH linux-2.6-block:post-2.6.15 08/10] blk: update IDE to use new blk_ordered
Hello, Bartlomiej.
Quoted 5 lines · Bartlomiej Zolnierkiewicz
On 11/17/05, Tejun Heo <[email protected]> wrote:
I fail to see how the partial completions (good + bad sectors) are done in your new scheme, please explain.
It doesn't. I've noted this way back when I posted this patchset the second time.
http://marc.theaimsgroup.com/?l=linux-kernel&m=111795127124020&w=2
Rationales
* The actual barrier IO request is issued as a part of ordered sequence. When any part of this sequence fails (any of leading flush, barrier IO
or post flush), the whole sequence should be considered to have failed.
e.g. if leading flush fails, there's no point in reporting partial or full success of barrier IO. Ditto for tailing flush. We can special case when only part of barrier IO fails and report partial barrier success, but 1. benefits are doubtful 2. even if it's implemented, it wouldn't work (see next rationale)
* Barrier requests are not mergeable. ie. Each barrier bio is turned into one barrier request and partially completing the request doesn't result in any successfully completed bio.
* SCSI doesn't handle partial completion of barrier IOs.
Quoted 12 lines
drivers/ide/ide-disk.c static ide_proc_entry_t idedisk_proc[] = · line 727
Quoted 8 lines
drivers/ide/ide-disk.c +0
-7- -static int idedisk_prepare_flush (request_queue_t *q,struct request *rq) -{ -ide_drive_t *drive = q->queuedata; - -if (!drive->wcache) -return 0 ;What does happen if somebody disables
drive->wcachelater?
Thanks for pointing out. I've moved ordered configuration into write_cache such that ordered is reconfigured when write_cache changes.
There can be in-flight barrier requests which are inconsistent with the newly updated setting, but 1. it's not too unfair to assume that user is responsible for that synchronization 2. the original implementation had the same issue 3. the consequence is not catastrophic.
Quoted 23 lines
Quoted 23 lines
memset(rq->cmd, 0, sizeof(rq->cmd));drivers/ide/ide-disk.c +3
-5@@ -735,9 +694,8 @@ static int idedisk_prepare_flush(request rq->cmd[0 ] =WIN_FLUSH_CACHE ; - rq->flags |=REQ_DRIVE_TASK |REQ_SOFTBARRIER ; + rq->flags |=REQ_DRIVE_TASK ; rq->buffer = rq->cmd; -return 1 ; } static int idedisk_issue_flush(request_queue_t *q, struct gendisk *disk,@@ -1012,11 +970,12 @@ static void idedisk_setup (ide_drive_t *printk (KERN_INFO "%s: cache flushes %ssupported\n" , drive->name, barrier ?"" :"not " );if (barrier) { -blk_queue_ordered (drive->queue,QUEUE_ORDERED_FLUSH ); - drive->queue->prepare_flush_fn = idedisk_prepare_flush; - drive->queue->end_flush_fn = idedisk_end_flush; +blk_queue_ordered (drive->queue,QUEUE_ORDERED_DRAIN_FLUSH , + idedisk_prepare_flush,GFP_KERNEL );
Quoted 9 lines
drivers/ide/ide-disk.c static void idedisk_setup (ide_drive_t * · line 978
Quoted 5 lines
drivers/ide/ide-disk.c +3
-1blk_queue_issue_flush_fn (drive->queue, idedisk_issue_flush); - } + }else if (!drive->wcache) +blk_queue_ordered (drive->queue,QUEUE_ORDERED_DRAIN , +NULL ,GFP_KERNEL );What does happen if somebody enables
drive->wcachelater?
ditto.
Quoted 8 lines
Quoted 8 lines
drivers/ide/ide-disk.c +0
-0}static void ide_cacheflush_p (ide_drive_t *drive)@@ -1034,6 +993,8 @@ static int ide_disk_remove(struct devicestruct ide_disk_obj *idkp = drive->driver_data;struct gendisk *g = idkp->disk;
Quoted 5 lines
drivers/ide/ide-disk.c static int ide_disk_remove(struct device · line 997
Quoted 2 lines
drivers/ide/ide-disk.c +2
-0+blk_queue_ordered (drive->queue,QUEUE_ORDERED_NONE ,NULL ,0 ); +Shouldn't this be done in
ide_disk_release()?
Hmmm... The thing is that, AFAIK, requests are not supposed to be issued after ->remove is called (->remove is called only on module unload unless hardware is hot-unplugged and HL driver cannot be unloaded while it's still opened). I think that's why both sd and ide-disk issue the last cache flush in ->remove callbacks but not in ->release.
I think the most natural place to put ordered deconfiguration is right above the last cache flush. Hmmm... If above is not true, I think we should move both ordered deconfig and the last cache flushes to ->release callbacks. What do you think?
Thanks.
Signature · 1 line
tejun
Re: [PATCH linux-2.6-block:post-2.6.15 08/10] blk: update IDE to use new blk_ordered
Hi,
Quoted 11 lines · Tejun Heo
Hello, Bartlomiej.
Quoted 5 lines · Bartlomiej Zolnierkiewicz
On 11/17/05, Tejun Heo <[email protected]> wrote:
I fail to see how the partial completions (good + bad sectors) are done in your new scheme, please explain.
It doesn't. I've noted this way back when I posted this patchset the second time.
This should be noted in the patch description not in the announcement.
Quoted 14 lines
http://marc.theaimsgroup.com/?l=linux-kernel&m=111795127124020&w=2
Rationales
* The actual barrier IO request is issued as a part of ordered sequence. When any part of this sequence fails (any of leading flush, barrier IO
or post flush), the whole sequence should be considered to have failed.e.g. if leading flush fails, there's no point in reporting partial or full success of barrier IO. Ditto for tailing flush. We can special case when only part of barrier IO fails and report partial barrier success, but 1. benefits are doubtful 2. even if it's implemented, it wouldn't work (see next rationale)
Quoted 3 lines
* Barrier requests are not mergeable. ie. Each barrier bio is turned into one barrier request and partially completing the request doesn't result in any successfully completed bio.
However your flush request can fail on the sector _completely_ unrelated to the barrier request so in this case old code worked differently. Anyway I'm fine with this change (previous logic was too complicated).
Quoted 15 lines
* SCSI doesn't handle partial completion of barrier IOs.
Quoted 12 lines
drivers/ide/ide-disk.c static ide_proc_entry_t idedisk_proc[] = · line 727
Quoted 8 lines
drivers/ide/ide-disk.c +0
-7- -static int idedisk_prepare_flush (request_queue_t *q,struct request *rq) -{ -ide_drive_t *drive = q->queuedata; - -if (!drive->wcache) -return 0 ;What does happen if somebody disables
drive->wcachelater?
Quoted 6 lines
Thanks for pointing out. I've moved ordered configuration into
write_cachesuch that ordered is reconfigured whenwrite_cachechanges. There can be in-flight barrier requests which are inconsistent with the newly updated setting, but 1. it's not too unfair to assume that user is responsible for that synchronization 2. the original implementation had the same issue 3. the consequence is not catastrophic.
The consequence could be increased number of bugreports about failed IDE commands which wasn't the case with !drive->wcache check in place - please leave as it was.
Quoted 11 lines
Quoted 10 lines
drivers/ide/ide-disk.c static int ide_disk_remove(struct device · line 997
Quoted 7 lines
static void
ide_cacheflush_p(ide_drive_t *drive)drivers/ide/ide-disk.c +2
-0@@ -1034,6 +993,8 @@ static int ide_disk_remove(struct devicestruct ide_disk_obj *idkp = drive->driver_data;struct gendisk *g = idkp->disk; +blk_queue_ordered (drive->queue,QUEUE_ORDERED_NONE ,NULL ,0 ); +Shouldn't this be done in
ide_disk_release()?
Quoted 5 lines
Hmmm... The thing is that, AFAIK, requests are not supposed to be issued after ->remove is called (->remove is called only on module unload unless hardware is hot-unplugged and HL driver cannot be unloaded while it's still opened). I think that's why both sd and ide-disk issue the last cache flush in ->remove callbacks but not in ->release.
Are you sure? I think that only calling del_gendisk() assures you that there won't be outstanding fs requests?
I have also noticed bug in ide_disk_remove() - ide_cacheflush_p() should be called after del_gendisk() - I will fix it later.
BTW Nowadays you can dynamically dettach/attach driver from/to device using sysfs interface.
Thanks,
Bartlomiej
Re: [PATCH linux-2.6-block:post-2.6.15 08/10] blk: update IDE to use new blk_ordered
Quoted 8 lines · Bartlomiej Zolnierkiewicz
Hi,
Quoted 5 lines · Tejun Heo
Hello, Bartlomiej.
Quoted 2 lines · Bartlomiej Zolnierkiewicz
On 11/17/05, Tejun Heo <[email protected]> wrote:
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
I fail to see how the partial completions (good + bad sectors) are done in your new scheme, please explain.
It doesn't. I've noted this way back when I posted this patchset the second time.
This should be noted in the patch description not in the announcement.
Will add it to patch description.
Quoted 15 lines
Quoted 15 lines
http://marc.theaimsgroup.com/?l=linux-kernel&m=111795127124020&w=2
Rationales
* The actual barrier IO request is issued as a part of ordered sequence. When any part of this sequence fails (any of leading flush, barrier IO
or post flush), the whole sequence should be considered to have failed.e.g. if leading flush fails, there's no point in reporting partial or full success of barrier IO. Ditto for tailing flush. We can special case when only part of barrier IO fails and report partial barrier success, but 1. benefits are doubtful 2. even if it's implemented, it wouldn't work (see next rationale)
Quoted 8 lines
Quoted 3 lines
* Barrier requests are not mergeable. ie. Each barrier bio is turned into one barrier request and partially completing the request doesn't result in any successfully completed bio.
However your flush request can fail on the sector _completely_ unrelated to the barrier request so in this case old code worked differently. Anyway I'm fine with this change (previous logic was too complicated).
Hmmm... Ordered sequence should fail even when flush request fails on a completely unrelated to the barrier request. The barrier request must make sure that all requests issued prior to it have made to the physical medium successfully.
Anyways, you're right in that it acts differently from the original code and it should be noted in the patch description. I'll update the patch description.
Quoted 23 lines
Quoted 16 lines
* SCSI doesn't handle partial completion of barrier IOs.
Quoted 12 lines
drivers/ide/ide-disk.c static ide_proc_entry_t idedisk_proc[] = · line 727
Quoted 8 lines
drivers/ide/ide-disk.c +0
-7- -static int idedisk_prepare_flush (request_queue_t *q,struct request *rq) -{ -ide_drive_t *drive = q->queuedata; - -if (!drive->wcache) -return 0 ;What does happen if somebody disables
drive->wcachelater?Quoted 6 lines
Thanks for pointing out. I've moved ordered configuration into
write_cachesuch that ordered is reconfigured whenwrite_cachechanges. There can be in-flight barrier requests which are inconsistent with the newly updated setting, but 1. it's not too unfair to assume that user is responsible for that synchronization 2. the original implementation had the same issue 3. the consequence is not catastrophic.
Quoted 3 lines
The consequence could be increased number of bugreports about failed IDE commands which wasn't the case with !drive->wcache check in place - please leave as it was.
Ordered requests are processed in the following order.
1. barrier bio reaches blk queue
2. barrier req queued in order
3. when barrier req reaches the head of the request queue, it gets interpreted into preflush-barrier-postflush requests sequence and queued. ->prepare_flush_fn is called in this step.
4. When all three requests complete, the ordered sequence ends.
Adding !drive->wcache test to idedisk_prepare_flush, which in turn requires adding ->prepare_flush_fn error handling to blk ordered handling, prevents flushes for barrier requests between step#1 and step#3. We can still have flush reqeuests between #3 and #4 after wcache is turned off.
Please also note that any of above happens only if a user turns off ->wcache setting while a fs is actively performing a barrier.
I'm not sure the benefit justifies added complexity. Do you still think adding ->wcache test is necessary?
Quoted 18 lines
Quoted 12 lines
Quoted 11 lines
drivers/ide/ide-disk.c static int ide_disk_remove(struct device · line 997
Quoted 8 lines
static void
ide_cacheflush_p(ide_drive_t *drive)drivers/ide/ide-disk.c +2
-0@@ -1034,6 +993,8 @@ static int ide_disk_remove(struct devicestruct ide_disk_obj *idkp = drive->driver_data;struct gendisk *g = idkp->disk; +blk_queue_ordered (drive->queue,QUEUE_ORDERED_NONE ,NULL ,0 ); +Shouldn't this be done in
ide_disk_release()?Quoted 5 lines
Hmmm... The thing is that, AFAIK, requests are not supposed to be issued after ->remove is called (->remove is called only on module unload unless hardware is hot-unplugged and HL driver cannot be unloaded while it's still opened). I think that's why both sd and ide-disk issue the last cache flush in ->remove callbacks but not in ->release.
Quoted 8 lines
Are you sure? I think that only calling
del_gendisk()assures you that there won't be outstanding fs requests?I have also noticed bug in
ide_disk_remove()-ide_cacheflush_p()should be called afterdel_gendisk()- I will fix it later.BTW Nowadays you can dynamically dettach/attach driver from/to device using sysfs interface.
I agree that it should go into ->release, but I am still a bit scared about issuing commands in ->release (it might access some data structure which might be gone by then). Also, the correct order seems to be 'turning off ordered' and then 'perform the last cache flush'. So, how about adding blk_queue_ordered right above the last ide_cacheflush_p() now and move both to ->release in a separate patch for both IDE and SCSI?
Thanks.
Signature · 1 line
tejun
Re: [PATCH linux-2.6-block:post-2.6.15 08/10] blk: update IDE to use new blk_ordered
Quoted 25 lines · Tejun Heo
Quoted 14 lines · Bartlomiej Zolnierkiewicz
Quoted 14 lines
http://marc.theaimsgroup.com/?l=linux-kernel&m=111795127124020&w=2
Rationales
* The actual barrier IO request is issued as a part of ordered sequence. When any part of this sequence fails (any of leading flush, barrier IO
or post flush), the whole sequence should be considered to have failed.e.g. if leading flush fails, there's no point in reporting partial or full success of barrier IO. Ditto for tailing flush. We can special case when only part of barrier IO fails and report partial barrier success, but 1. benefits are doubtful 2. even if it's implemented, it wouldn't work (see next rationale)
Quoted 8 lines
Quoted 3 lines
* Barrier requests are not mergeable. ie. Each barrier bio is turned into one barrier request and partially completing the request doesn't result in any successfully completed bio.
However your flush request can fail on the sector _completely_ unrelated to the barrier request so in this case old code worked differently. Anyway I'm fine with this change (previous logic was too complicated).
Quoted 8 lines
Hmmm... Ordered sequence should fail even when flush request fails on a completely unrelated to the barrier request. The barrier request must make sure that all requests issued prior to it have made to the physical medium successfully.
Anyways, you're right in that it acts differently from the original code and it should be noted in the patch description. I'll update the patch description.
Thanks.
Quoted 36 lines
Quoted 22 lines
Quoted 15 lines
* SCSI doesn't handle partial completion of barrier IOs.
Quoted 12 lines
drivers/ide/ide-disk.c static ide_proc_entry_t idedisk_proc[] = · line 727
Quoted 8 lines
drivers/ide/ide-disk.c +0
-7- -static int idedisk_prepare_flush (request_queue_t *q,struct request *rq) -{ -ide_drive_t *drive = q->queuedata; - -if (!drive->wcache) -return 0 ;What does happen if somebody disables
drive->wcachelater?Quoted 6 lines
Thanks for pointing out. I've moved ordered configuration into
write_cachesuch that ordered is reconfigured whenwrite_cachechanges. There can be in-flight barrier requests which are inconsistent with the newly updated setting, but 1. it's not too unfair to assume that user is responsible for that synchronization 2. the original implementation had the same issue 3. the consequence is not catastrophic.Quoted 3 lines
The consequence could be increased number of bugreports about failed IDE commands which wasn't the case with !drive->wcache check in place - please leave as it was.
Ordered requests are processed in the following order.
1. barrier bio reaches blk queue
2. barrier req queued in order
3. when barrier req reaches the head of the request queue, it gets interpreted into preflush-barrier-postflush requests sequence and queued. ->prepare_flush_fn is called in this step.
Quoted 5 lines
4. When all three requests complete, the ordered sequence ends.
Adding !drive->wcache test to
idedisk_prepare_flush, which in turn requires adding ->prepare_flush_fn error handling to blk ordered handling, prevents flushes for barrier requests between step#1 and
Why for !drive->wcache flush can't be consider as successful like it was before these changes...
Quoted 2 lines
step#3. We can still have flush reqeuests between #3 and #4 after wcache is turned off.
ditto
Quoted 33 lines
Please also note that any of above happens only if a user turns off ->wcache setting while a fs is actively performing a barrier.
I'm not sure the benefit justifies added complexity. Do you still think adding ->wcache test is necessary?
Quoted 18 lines
Quoted 12 lines
Quoted 11 lines
drivers/ide/ide-disk.c static int ide_disk_remove(struct device · line 997
Quoted 8 lines
static void
ide_cacheflush_p(ide_drive_t *drive)drivers/ide/ide-disk.c +2
-0@@ -1034,6 +993,8 @@ static int ide_disk_remove(struct devicestruct ide_disk_obj *idkp = drive->driver_data;struct gendisk *g = idkp->disk; +blk_queue_ordered (drive->queue,QUEUE_ORDERED_NONE ,NULL ,0 ); +Shouldn't this be done in
ide_disk_release()?Quoted 5 lines
Hmmm... The thing is that, AFAIK, requests are not supposed to be issued after ->remove is called (->remove is called only on module unload unless hardware is hot-unplugged and HL driver cannot be unloaded while it's still opened). I think that's why both sd and ide-disk issue the last cache flush in ->remove callbacks but not in ->release.
Quoted 8 lines
Are you sure? I think that only calling
del_gendisk()assures you that there won't be outstanding fs requests?I have also noticed bug in
ide_disk_remove()-ide_cacheflush_p()should be called afterdel_gendisk()- I will fix it later.BTW Nowadays you can dynamically dettach/attach driver from/to device using sysfs interface.
Quoted 7 lines
I agree that it should go into ->release, but I am still a bit scared about issuing commands in ->release (it might access some data structure which might be gone by then). Also, the correct order seems to be 'turning off ordered' and then 'perform the last cache flush'. So, how about adding
blk_queue_orderedright above the lastide_cacheflush_p()now and move both to ->release in a separate patch for both IDE and SCSI?
Not needed, when ide-disk is fixed to call del_gendisk() after ide_cacheflush_p(), we can add blk_queue_orderer() before the latter and then everything should be OK.
Thanks,
Bartlomiej
Re: [PATCH linux-2.6-block:post-2.6.15 08/10] blk: update IDE to use new blk_ordered
thinko
Quoted 34 lines · Bartlomiej Zolnierkiewicz
Quoted 26 lines
Quoted 17 lines
Quoted 11 lines
Quoted 10 lines
drivers/ide/ide-disk.c static int ide_disk_remove(struct device · line 997
Quoted 7 lines
static void
ide_cacheflush_p(ide_drive_t *drive)drivers/ide/ide-disk.c +2
-0@@ -1034,6 +993,8 @@ static int ide_disk_remove(struct devicestruct ide_disk_obj *idkp = drive->driver_data;struct gendisk *g = idkp->disk; +blk_queue_ordered (drive->queue,QUEUE_ORDERED_NONE ,NULL ,0 ); +Shouldn't this be done in
ide_disk_release()?Quoted 5 lines
Hmmm... The thing is that, AFAIK, requests are not supposed to be issued after ->remove is called (->remove is called only on module unload unless hardware is hot-unplugged and HL driver cannot be unloaded while it's still opened). I think that's why both sd and ide-disk issue the last cache flush in ->remove callbacks but not in ->release.
Quoted 8 lines
Are you sure? I think that only calling
del_gendisk()assures you that there won't be outstanding fs requests?I have also noticed bug in
ide_disk_remove()-ide_cacheflush_p()should be called afterdel_gendisk()- I will fix it later.BTW Nowadays you can dynamically dettach/attach driver from/to device using sysfs interface.
Quoted 7 lines
I agree that it should go into ->release, but I am still a bit scared about issuing commands in ->release (it might access some data structure which might be gone by then). Also, the correct order seems to be 'turning off ordered' and then 'perform the last cache flush'. So, how about adding
blk_queue_orderedright above the lastide_cacheflush_p()now and move both to ->release in a separate patch for both IDE and SCSI?
Quoted 2 lines
Not needed, when ide-disk is fixed to call
del_gendisk()afteride_cacheflush_p(), we can addblk_queue_orderer()before
ide_cacheflush_p() after del_gendisk()
Quoted 1 line
the latter and then everything should be OK.
blk_queue_ordererd() before ide_cache_flush_p()
:)
Re: [PATCH linux-2.6-block:post-2.6.15 08/10] blk: update IDE to use new blk_ordered
In reply to Bartlomiej ZolnierkiewiczOn Tue, Nov 22, 2005 at 09:36:09AM +0100, Bartlomiej Zolnierkiewicz wrote:
[--snip--]
Quoted 17 lines
Quoted 11 lines
Ordered requests are processed in the following order.
1. barrier bio reaches blk queue
2. barrier req queued in order
3. when barrier req reaches the head of the request queue, it gets interpreted into preflush-barrier-postflush requests sequence and queued. ->prepare_flush_fn is called in this step.
Quoted 5 lines
4. When all three requests complete, the ordered sequence ends.
Adding !drive->wcache test to
idedisk_prepare_flush, which in turn requires adding ->prepare_flush_fn error handling to blk ordered handling, prevents flushes for barrier requests between step#1 and
Quoted 8 lines
Why for !drive->wcache flush can't be consider as successful like it was before these changes...
Quoted 2 lines
step#3. We can still have flush reqeuests between #3 and #4 after wcache is turned off.
ditto
I think we have two alternatives here - both have some problems.
1. make ->prepare_flush_fn return some code to tell blk layer skip the flush as the original code did.
This is what you're proposing, I guess. The reason why I'm reluctant to take this approach is that there still remains window of error between #3 and #4. The flush requests could already be prepared and in the queue when ->wcache is turned off. AFAICS, the original code had the same problem, although the window was narrower.
2. complete flush commands successfully in execute_drive_cmd() if wcache is not enabled.
This approach fixes all cases but the implementation would be a bit hackish. execute_drive_cmd() will have to look at the command and ide_disk->wcache to determine if a special command should be completed successfully without actually executing it. Maybe we can add a per-HL-driver callback for that.
Bartlomiej, I'm not really sure about both of above approaches. My humble opinion is that benefits brought by both of above aren't worth the added complexity. The worst can happen is a few IDE command failures caused by executing flush command on a wcache disabled drive. And that would happen only when the user turns off wcache while barrier sequence is *in progress*.
Hmmm... What do you think? It's your call. I'll do what you choose.
[--snip--]
Quoted 11 lines
Quoted 7 lines
I agree that it should go into ->release, but I am still a bit scared about issuing commands in ->release (it might access some data structure which might be gone by then). Also, the correct order seems to be 'turning off ordered' and then 'perform the last cache flush'. So, how about adding
blk_queue_orderedright above the lastide_cacheflush_p()now and move both to ->release in a separate patch for both IDE and SCSI?Not needed, when ide-disk is fixed to call
del_gendisk()afteride_cacheflush_p(), we can addblk_queue_orderer()before the latter and then everything should be OK.
Can you get the patch into the post-2.6.15 branch of linux-2.6-block tree?
Thanks. :-)
Signature · 1 line
tejun
Re: [PATCH linux-2.6-block:post-2.6.15 08/10] blk: update IDE to use new blk_ordered
Quoted 48 lines · Tejun Heo
On Tue, Nov 22, 2005 at 09:36:09AM +0100, Bartlomiej Zolnierkiewicz wrote:
[--snip--]Quoted 17 lines
Quoted 11 lines
Ordered requests are processed in the following order.
1. barrier bio reaches blk queue
2. barrier req queued in order
3. when barrier req reaches the head of the request queue, it gets interpreted into preflush-barrier-postflush requests sequence and queued. ->prepare_flush_fn is called in this step.
Quoted 5 lines
4. When all three requests complete, the ordered sequence ends.
Adding !drive->wcache test to
idedisk_prepare_flush, which in turn requires adding ->prepare_flush_fn error handling to blk ordered handling, prevents flushes for barrier requests between step#1 andQuoted 8 lines
Why for !drive->wcache flush can't be consider as successful like it was before these changes...
Quoted 2 lines
step#3. We can still have flush reqeuests between #3 and #4 after wcache is turned off.
ditto
I think we have two alternatives here - both have some problems.
1. make ->prepare_flush_fn return some code to tell blk layer skip the flush as the original code did.
This is what you're proposing, I guess. The reason why I'm reluctant to take this approach is that there still remains window of error between #3 and #4. The flush requests could already be prepared and in the queue when ->wcache is turned off. AFAICS, the original code had the same problem, although the window was narrower.
2. complete flush commands successfully in
execute_drive_cmd()if wcache is not enabled.This approach fixes all cases but the implementation would be a bit hackish.
execute_drive_cmd()will have to look at the command andide_disk->wcacheto determine if a special command should be completed successfully without actually executing it. Maybe we can add a per-HL-driver callback for that.
Quoted 8 lines
Bartlomiej, I'm not really sure about both of above approaches. My humble opinion is that benefits brought by both of above aren't worth the added complexity. The worst can happen is a few IDE command failures caused by executing flush command on a wcache disabled drive. And that would happen only when the user turns off wcache while barrier sequence is *in progress*.
Hmmm... What do you think? It's your call. I'll do what you choose.
Hmm... both solutions sucks. After second thought I agree with you w.r.t to original changes (just remember to document them in the patch description).
Bartlomiej
Re: [PATCH linux-2.6-block:post-2.6.15 08/10] blk: update IDE to use new blk_ordered
Quoted 56 lines · Bartlomiej Zolnierkiewicz
Quoted 55 lines · Tejun Heo
On Tue, Nov 22, 2005 at 09:36:09AM +0100, Bartlomiej Zolnierkiewicz wrote:
[--snip--]Quoted 17 lines
Quoted 11 lines
Ordered requests are processed in the following order.
1. barrier bio reaches blk queue
2. barrier req queued in order
3. when barrier req reaches the head of the request queue, it gets interpreted into preflush-barrier-postflush requests sequence and queued. ->prepare_flush_fn is called in this step.
Quoted 5 lines
4. When all three requests complete, the ordered sequence ends.
Adding !drive->wcache test to
idedisk_prepare_flush, which in turn requires adding ->prepare_flush_fn error handling to blk ordered handling, prevents flushes for barrier requests between step#1 andQuoted 8 lines
Why for !drive->wcache flush can't be consider as successful like it was before these changes...
Quoted 2 lines
step#3. We can still have flush reqeuests between #3 and #4 after wcache is turned off.
ditto
I think we have two alternatives here - both have some problems.
1. make ->prepare_flush_fn return some code to tell blk layer skip the flush as the original code did.
This is what you're proposing, I guess. The reason why I'm reluctant to take this approach is that there still remains window of error between #3 and #4. The flush requests could already be prepared and in the queue when ->wcache is turned off. AFAICS, the original code had the same problem, although the window was narrower.
2. complete flush commands successfully in
execute_drive_cmd()if wcache is not enabled.This approach fixes all cases but the implementation would be a bit hackish.
execute_drive_cmd()will have to look at the command andide_disk->wcacheto determine if a special command should be completed successfully without actually executing it. Maybe we can add a per-HL-driver callback for that.Bartlomiej, I'm not really sure about both of above approaches. My humble opinion is that benefits brought by both of above aren't worth the added complexity. The worst can happen is a few IDE command failures caused by executing flush command on a wcache disabled drive. And that would happen only when the user turns off wcache while barrier sequence is *in progress*.
Quoted 5 lines
Quoted 1 line
Hmmm... What do you think? It's your call. I'll do what you choose.
Hmm... both solutions sucks. After second thought I agree with you w.r.t to original changes (just remember to document them in the patch description).
Me too. Plus on most drives a flush cache command on a drive with write back caching disabled will succeed, not fail. t13 is about to mandate that behaviour as well, and honestly I think this is the logical way to implement it in a drive (the flush just becomes a noop).
So Tejun, where do we stand with this patch series? Any changes over what you posted last week? I'd like to get this merged in the block tree, get it in -mm and merged for 2.6.16 if things work out.
Signature · 1 line
Jens Axboe
Re: [PATCH linux-2.6-block:post-2.6.15 08/10] blk: update IDE to use new blk_ordered
Quoted 60 lines · Jens Axboe
Quoted 58 lines · Bartlomiej Zolnierkiewicz
Quoted 56 lines · Tejun Heo
On Tue, Nov 22, 2005 at 09:36:09AM +0100, Bartlomiej Zolnierkiewicz wrote:
[--snip--]Quoted 16 lines
Quoted 10 lines
Ordered requests are processed in the following order.
1. barrier bio reaches blk queue
2. barrier req queued in order
3. when barrier req reaches the head of the request queue, it gets interpreted into preflush-barrier-postflush requests sequence and queued. ->prepare_flush_fn is called in this step.
Quoted 5 lines
4. When all three requests complete, the ordered sequence ends.
Adding !drive->wcache test to
idedisk_prepare_flush, which in turn requires adding ->prepare_flush_fn error handling to blk ordered handling, prevents flushes for barrier requests between step#1 andQuoted 9 lines
Why for !drive->wcache flush can't be consider as successful like it was before these changes...
Quoted 2 lines
step#3. We can still have flush reqeuests between #3 and #4 after wcache is turned off.
ditto
I think we have two alternatives here - both have some problems.
1. make ->prepare_flush_fn return some code to tell blk layer skip the flush as the original code did.
This is what you're proposing, I guess. The reason why I'm reluctant to take this approach is that there still remains window of error between #3 and #4. The flush requests could already be prepared and in the queue when ->wcache is turned off. AFAICS, the original code had the same problem, although the window was narrower.
2. complete flush commands successfully in
execute_drive_cmd()if wcache is not enabled.This approach fixes all cases but the implementation would be a bit hackish.
execute_drive_cmd()will have to look at the command andide_disk->wcacheto determine if a special command should be completed successfully without actually executing it. Maybe we can add a per-HL-driver callback for that.Bartlomiej, I'm not really sure about both of above approaches. My humble opinion is that benefits brought by both of above aren't worth the added complexity. The worst can happen is a few IDE command failures caused by executing flush command on a wcache disabled drive. And that would happen only when the user turns off wcache while barrier sequence is *in progress*.
Quoted 6 lines
Quoted 5 lines
Quoted 1 line
Hmmm... What do you think? It's your call. I'll do what you choose.
Hmm... both solutions sucks. After second thought I agree with you w.r.t to original changes (just remember to document them in the patch description).
Yeap, both suck. Glad to hear that. :-)
Quoted 10 lines
Me too. Plus on most drives a flush cache command on a drive with write back caching disabled will succeed, not fail. t13 is about to mandate that behaviour as well, and honestly I think this is the logical way to implement it in a drive (the flush just becomes a noop).
So Tejun, where do we stand with this patch series? Any changes over what you posted last week? I'd like to get this merged in the block tree, get it in -mm and merged for 2.6.16 if things work out.
Hi, Bartlomiej. Hi, Jens.
Currently, there are only two more things to do before getting this thing into the -mm tree.
1. Adjusting this patch (update-ide) after merging Bartlomiej's del_gendisk() fix patch.
2. Update ide-fua patch as discussed and get it reviewed by Bartlomiej.
Other than above two, I think we're ready. I'll post update ide-fua patch later today.
Thanks.
Signature · 1 line
tejun
Re: [PATCH linux-2.6-block:post-2.6.15 08/10] blk: update IDE to use new blk_ordered
On 11/23/05, Tejun <[email protected]> wrote:
<...>
Quoted 7 lines
Hi, Bartlomiej. Hi, Jens.
Currently, there are only two more things to do before getting this thing into the -mm tree.
1. Adjusting this patch (update-ide) after merging Bartlomiej's
del_gendisk()fix patch.
This should be in the mainline today/tomorrow as it is obviously correct and I need to push another IDE update anyway (new HW support).
Quoted 9 lines
2. Update ide-fua patch as discussed and get it reviewed by Bartlomiej.
Other than above two, I think we're ready. I'll post update ide-fua patch later today.
Thanks.
--
tejun
Quoted 2 lines · Tejun Heo
* libata has gone through some changes since last posting. libata updates are rewritten. Jeff, please review #07 and #08.
Look OK to me.
Jeff