Skip to contentSkip to conversation

libata: eliminate the home grown dma padding in favour of that provided by the block layer

PATCH Superseded James Bottomley 4 participants 33 messages Dec 31, 2007 – Feb 5, 2008

Subsystems (7)
SCSI Block layer libata SCSI libsas +3 more subsystems
lore
Share & tools
lore

James Bottomley

author maintainer
#0 lore raw

PATCH 1/1 libata: eliminate the home grown dma padding in favour of that provided by the block layer

ATA requires that all DMA transfers begin and end on word boundaries. Because of this, a large amount of machinery grew up in ide to adjust scatterlists on this basis. However, as of 2.5, the block layer has a dma_alignment variable which ensures both the beginning and length of a DMA transfer are aligned on the dma_alignment boundary. Although the block layer does adjust the beginning of the transfer to ensure this happens, it doesn't actually adjust the length, it merely makes sure that space is allocated for transfers beyond the declared length. The upshot of this is that scatterlists may be padded to any size between the actual length and the length adjusted to the dma_alignment safely knowing that memory is allocated in this region.

Right at the moment, SCSI takes the default dma_aligment which is on a 512 byte boundary. Note that this aligment only applies to transfers coming in from user space. However, since all kernel allocations are automatically aligned on a minimum of 32 byte boundaries, it is safe to adjust them in this manner as well.

---

This also fixes a current panic in libsas with SATAPI devices. I was trying to trace a bad interaction between the libata padding and the new sglist code which was the root cause, and ended up concluding that the whole libata padding thing was redundant.

In a future improvement, I think we can relax SCSI dma_alignemnt (it's causing almost every user space command to be copied instead of directly passed through) and allow devices and transports to build up their own requirements instead.

James

 drivers/ata/ahci.c            |    5 -
 drivers/ata/libata-core.c     |  126 +++---------------------------------------
 drivers/ata/libata-scsi.c     |   18 +-----
 drivers/ata/pata_icside.c     |    8 --
 drivers/ata/sata_fsl.c        |   17 -----
 drivers/ata/sata_mv.c         |    5 -
 drivers/ata/sata_nv.c         |    2 
 drivers/ata/sata_promise.c    |    2 
 drivers/ata/sata_qstor.c      |    2 
 drivers/ata/sata_sil24.c      |    5 -
 drivers/scsi/ipr.c            |    4 -
 drivers/scsi/libsas/sas_ata.c |    4 -
 include/linux/libata.h        |   35 -----------
 13 files changed, 25 insertions(+), 208 deletions(-)
drivers/ata/ahci.c +0 -5
index 54f38c2..4dd318d 100644
@@ -1965,16 +1965,11 @@ static int ahci_port_start(struct ata_port *ap)
 	struct ahci_port_priv *pp;
 	void *mem;
 	dma_addr_t mem_dma;
-	int rc;
 
 	pp = devm_kzalloc(dev, sizeof(*pp), GFP_KERNEL);
 	if (!pp)
 		return -ENOMEM;
 
-	rc = ata_pad_alloc(ap, dev);
-	if (rc)
-		return rc;
-
 	mem = dmam_alloc_coherent(dev, AHCI_PORT_PRIV_DMA_SZ, &mem_dma,
 				  GFP_KERNEL);
 	if (!mem)
drivers/ata/libata-core.c +11 -115

222 lines not on this page · raw message

drivers/ata/libata-scsi.c +3 -15
index a883bb0..571fba5 100644
@@ -832,15 +832,6 @@ static void ata_scsi_dev_config(struct scsi_device *sdev,
 	/* configure max sectors */
 	blk_queue_max_sectors(sdev->request_queue, dev->max_sectors);
 
-	/* SATA DMA transfers must be multiples of 4 byte, so
-	 * we need to pad ATAPI transfers using an extra sg.
-	 * Decrement max hw segments accordingly.
-	 */
-	if (dev->class == ATA_DEV_ATAPI) {
-		struct request_queue *q = sdev->request_queue;
-		blk_queue_max_hw_segments(q, q->max_hw_segments - 1);
-	}
-
 	if (dev->flags & ATA_DFLAG_AN)
 		set_bit(SDEV_EVT_MEDIA_CHANGE, sdev->supported_events);
 
@@ -3527,7 +3518,7 @@ EXPORT_SYMBOL_GPL(ata_sas_port_alloc);
  *	@ap: Port to initialize
  *
  *	Called just after data structures for each port are
- *	initialized.  Allocates DMA pad.
+ *	initialized.
  *
  *	May be used as the port_start() entry in ata_port_operations.
  *
@@ -3536,7 +3527,7 @@ EXPORT_SYMBOL_GPL(ata_sas_port_alloc);
  */
 int ata_sas_port_start(struct ata_port *ap)
 {
-	return ata_pad_alloc(ap, ap->dev);
+	return 0;
 }
 EXPORT_SYMBOL_GPL(ata_sas_port_start);
 
@@ -3544,8 +3535,6 @@ EXPORT_SYMBOL_GPL(ata_sas_port_start);
  *	ata_port_stop - Undo ata_sas_port_start()
  *	@ap: Port to shut down
  *
- *	Frees the DMA pad.
- *
  *	May be used as the port_stop() entry in ata_port_operations.
  *
  *	LOCKING:
@@ -3554,7 +3543,6 @@ EXPORT_SYMBOL_GPL(ata_sas_port_start);
 
 void ata_sas_port_stop(struct ata_port *ap)
 {
-	ata_pad_free(ap, ap->dev);
 }
 EXPORT_SYMBOL_GPL(ata_sas_port_stop);
 
@@ -3603,7 +3591,7 @@ EXPORT_SYMBOL_GPL(ata_sas_port_destroy);
  *
  *	RETURNS:
  *	Zero.
- */
+< */
 
 int ata_sas_slave_configure(struct scsi_device *sdev, struct ata_port *ap)
 {
drivers/ata/pata_icside.c +0 -8
index 842fe08..d53809c 100644
@@ -303,12 +303,6 @@ static int icside_dma_init(struct pata_icside_info *info)
 }
 
 
-static int pata_icside_port_start(struct ata_port *ap)
-{
-	/* No PRD to alloc */
-	return ata_pad_alloc(ap, ap->dev);
-}
-
 static struct scsi_host_template pata_icside_sht = {
 	.module			= THIS_MODULE,
 	.name			= DRV_NAME,
@@ -388,8 +382,6 @@ static struct ata_port_operations pata_icside_port_ops = {
 	.irq_clear		= ata_dummy_noret,
 	.irq_on			= ata_irq_on,
 
-	.port_start		= pata_icside_port_start,
-
 	.bmdma_stop		= pata_icside_bmdma_stop,
 	.bmdma_status		= pata_icside_bmdma_status,
 };
drivers/ata/sata_fsl.c +2 -15
index d015b4a..7fdab46 100644
@@ -354,8 +354,8 @@ static unsigned int sata_fsl_fill_sg(struct ata_queued_cmd *qc, void *cmd_desc,
 			ata_port_printk(qc->ap, KERN_ERR,
 					"s/g len unaligned : 0x%x\n", sg_len);
 
-		if ((num_prde == (SATA_FSL_MAX_PRD_DIRECT - 1)) &&
-		    (qc->n_iter + 1 != qc->n_elem)) {
+		if (num_prde == (SATA_FSL_MAX_PRD_DIRECT - 1) &&
+		    sg_next(sg) != NULL) {
 			VPRINTK("setting indirect prde\n");
 			prd_ptr_to_indirect_ext = prd;
 			prd->dba = cpu_to_le32(indirect_ext_segment_paddr);
@@ -600,21 +600,9 @@ static int sata_fsl_port_start(struct ata_port *ap)
 	if (!pp)
 		return -ENOMEM;
 
-	/*
-	 * allocate per command dma alignment pad buffer, which is used
-	 * internally by libATA to ensure that all transfers ending on
-	 * unaligned boundaries are padded, to align on Dword boundaries
-	 */
-	retval = ata_pad_alloc(ap, dev);
-	if (retval) {
-		kfree(pp);
-		return retval;
-	}
-
 	mem = dma_alloc_coherent(dev, SATA_FSL_PORT_PRIV_DMA_SZ, &mem_dma,
 				 GFP_KERNEL);
 	if (!mem) {
-		ata_pad_free(ap, dev);
 		kfree(pp);
 		return -ENOMEM;
 	}
@@ -693,7 +681,6 @@ static void sata_fsl_port_stop(struct ata_port *ap)
 	dma_free_coherent(dev, SATA_FSL_PORT_PRIV_DMA_SZ,
 			  pp->cmdslot, pp->cmdslot_paddr);
 
-	ata_pad_free(ap, dev);
 	kfree(pp);
 }
 
drivers/ata/sata_mv.c +0 -5
index 37b850a..94bf504 100644
@@ -1054,7 +1054,6 @@ static int mv_port_start(struct ata_port *ap)
 	void *mem;
 	dma_addr_t mem_dma;
 	unsigned long flags;
-	int rc;
 
 	pp = devm_kzalloc(dev, sizeof(*pp), GFP_KERNEL);
 	if (!pp)
@@ -1066,10 +1065,6 @@ static int mv_port_start(struct ata_port *ap)
 		return -ENOMEM;
 	memset(mem, 0, MV_PORT_PRIV_DMA_SZ);
 
-	rc = ata_pad_alloc(ap, dev);
-	if (rc)
-		return rc;
-
 	/* First item in chunk of DMA memory:
 	 * 32-slot command request table (CRQB), 32 bytes each in size
 	 */
drivers/ata/sata_nv.c +1 -1
index ed5dc7c..7eaacc7 100644
@@ -2000,7 +2000,7 @@ static void nv_swncq_fill_sg(struct ata_queued_cmd *qc)
 	struct ata_prd *prd;
 
 	WARN_ON(qc->__sg == NULL);
-	WARN_ON(qc->n_elem == 0 && qc->pad_len == 0);
+	WARN_ON(qc->n_elem == 0);
 
 	prd = pp->prd + ATA_MAX_PRD * qc->tag;
 
drivers/ata/sata_promise.c +1 -1
index 7914def..242949c 100644
@@ -548,7 +548,7 @@ static void pdc_fill_sg(struct ata_queued_cmd *qc)
 		return;
 
 	WARN_ON(qc->__sg == NULL);
-	WARN_ON(qc->n_elem == 0 && qc->pad_len == 0);
+	WARN_ON(qc->n_elem == 0);
 
 	idx = 0;
 	ata_for_each_sg(sg, qc) {
drivers/ata/sata_qstor.c +1 -1
index 2f1de6e..258c482 100644
@@ -291,7 +291,7 @@ static unsigned int qs_fill_sg(struct ata_queued_cmd *qc)
 	u8 *prd = pp->pkt + QS_CPB_BYTES;
 
 	WARN_ON(qc->__sg == NULL);
-	WARN_ON(qc->n_elem == 0 && qc->pad_len == 0);
+	WARN_ON(qc->n_elem == 0);
 
 	nelem = 0;
 	ata_for_each_sg(sg, qc) {
drivers/ata/sata_sil24.c +0 -5
index 96fd526..228c36c 100644
@@ -1232,7 +1232,6 @@ static int sil24_port_start(struct ata_port *ap)
 	union sil24_cmd_block *cb;
 	size_t cb_size = sizeof(*cb) * SIL24_MAX_CMDS;
 	dma_addr_t cb_dma;
-	int rc;
 
 	pp = devm_kzalloc(dev, sizeof(*pp), GFP_KERNEL);
 	if (!pp)
@@ -1245,10 +1244,6 @@ static int sil24_port_start(struct ata_port *ap)
 		return -ENOMEM;
 	memset(cb, 0, cb_size);
 
-	rc = ata_pad_alloc(ap, dev);
-	if (rc)
-		return rc;
-
 	pp->cmd_block = cb;
 	pp->cmd_block_dma = cb_dma;
 
drivers/scsi/ipr.c +2 -2
index 9018ee8..987f44a 100644
@@ -5140,7 +5140,7 @@ static void ipr_build_ata_ioadl(struct ipr_cmnd *ipr_cmd,
 	struct ipr_ioarcb *ioarcb = &ipr_cmd->ioarcb;
 	struct ipr_ioadl_desc *ioadl = ipr_cmd->ioadl;
 	struct ipr_ioadl_desc *last_ioadl = NULL;
-	int len = qc->nbytes + qc->pad_len;
+	int len = qc->nbytes;
 	struct scatterlist *sg;
 
 	if (len == 0)
@@ -5205,7 +5205,7 @@ static unsigned int ipr_qc_issue(struct ata_queued_cmd *qc)
 	ioarcb->cmd_pkt.request_type = IPR_RQTYPE_ATA_PASSTHRU;
 	ioarcb->cmd_pkt.flags_hi |= IPR_FLAGS_HI_NO_LINK_DESC;
 	ioarcb->cmd_pkt.flags_hi |= IPR_FLAGS_HI_NO_ULEN_CHK;
-	ipr_cmd->dma_use_sg = qc->pad_len ? qc->n_elem + 1 : qc->n_elem;
+	ipr_cmd->dma_use_sg = qc->n_elem;
 
 	ipr_build_ata_ioadl(ipr_cmd, qc);
 	regs->flags |= IPR_ATA_FLAG_STATUS_ON_GOOD_COMPLETION;
drivers/scsi/libsas/sas_ata.c +2 -2
index 0829b55..78bd2f3 100644
@@ -178,8 +178,8 @@ static unsigned int sas_ata_qc_issue(struct ata_queued_cmd *qc)
 	task->uldd_task = qc;
 	if (is_atapi_taskfile(&qc->tf)) {
 		memcpy(task->ata_task.atapi_packet, qc->cdb, qc->dev->cdb_len);
-		task->total_xfer_len = qc->nbytes + qc->pad_len;
-		task->num_scatter = qc->pad_len ? qc->n_elem + 1 : qc->n_elem;
+		task->total_xfer_len = qc->nbytes;
+		task->num_scatter = qc->n_elem;
 	} else {
 		ata_for_each_sg(sg, qc) {
 			num++;
include/linux/libata.h +2 -33
index 124033c..2f40d57 100644
@@ -282,7 +282,7 @@ enum {
 
 	/* size of buffer to pad xfers ending on unaligned boundaries */
 	ATA_DMA_PAD_SZ		= 4,
-	ATA_DMA_PAD_BUF_SZ	= ATA_DMA_PAD_SZ * ATA_MAX_QUEUE,
+	ATA_DMA_PAD_MASK	= ATA_DMA_PAD_SZ - 1,
 
 	/* ering size */
 	ATA_ERING_SIZE		= 32,
@@ -446,12 +446,9 @@ struct ata_queued_cmd {
 	unsigned long		flags;		/* ATA_QCFLAG_xxx */
 	unsigned int		tag;
 	unsigned int		n_elem;
-	unsigned int		n_iter;
-	unsigned int		orig_n_elem;
 
 	int			dma_dir;
 
-	unsigned int		pad_len;
 	unsigned int		sect_size;
 
 	unsigned int		nbytes;
@@ -461,7 +458,6 @@ struct ata_queued_cmd {
 	unsigned int		cursg_ofs;
 
 	struct scatterlist	sgent;
-	struct scatterlist	pad_sgent;
 	void			*buf_virt;
 
 	/* DO NOT iterate over __sg manually, use ata_for_each_sg() */
@@ -606,9 +602,6 @@ struct ata_port {
 	struct ata_prd		*prd;	 /* our SG list */
 	dma_addr_t		prd_dma; /* and its DMA mapping */
 
-	void			*pad;	/* array of DMA pad buffers */
-	dma_addr_t		pad_dma;
-
 	struct ata_ioports	ioaddr;	/* ATA cmd/ctl/dma register blocks */
 
 	u8			ctl;	/* cache of ATA control register */
@@ -1080,24 +1073,15 @@ extern void ata_port_pbar_desc(struct ata_port *ap, int bar, ssize_t offset,
 static inline struct scatterlist *
 ata_qc_first_sg(struct ata_queued_cmd *qc)
 {
-	qc->n_iter = 0;
 	if (qc->n_elem)
 		return qc->__sg;
-	if (qc->pad_len)
-		return &qc->pad_sgent;
 	return NULL;
 }
 
 static inline struct scatterlist *
 ata_qc_next_sg(struct scatterlist *sg, struct ata_queued_cmd *qc)
 {
-	if (sg == &qc->pad_sgent)
-		return NULL;
-	if (++qc->n_iter < qc->n_elem)
-		return sg_next(sg);
-	if (qc->pad_len)
-		return &qc->pad_sgent;
-	return NULL;
+	return sg_next(sg);
 }
 
 #define ata_for_each_sg(sg, qc) \
@@ -1343,9 +1327,7 @@ static inline void ata_qc_reinit(struct ata_queued_cmd *qc)
 	qc->cursg_ofs = 0;
 	qc->nbytes = qc->curbytes = 0;
 	qc->n_elem = 0;
-	qc->n_iter = 0;
 	qc->err_mask = 0;
-	qc->pad_len = 0;
 	qc->sect_size = ATA_SECT_SIZE;
 
 	ata_tf_init(qc->dev, &qc->tf);
@@ -1379,19 +1361,6 @@ static inline unsigned int __ac_err_mask(u8 status)
 	return mask;
 }
 
-static inline int ata_pad_alloc(struct ata_port *ap, struct device *dev)
-{
-	ap->pad_dma = 0;
-	ap->pad = dmam_alloc_coherent(dev, ATA_DMA_PAD_BUF_SZ,
-				      &ap->pad_dma, GFP_KERNEL);
-	return (ap->pad == NULL) ? -ENOMEM : 0;
-}
-
-static inline void ata_pad_free(struct ata_port *ap, struct device *dev)
-{
-	dmam_free_coherent(dev, ATA_DMA_PAD_BUF_SZ, ap->pad, ap->pad_dma);
-}
-
 static inline struct ata_port *ata_shost_to_port(struct Scsi_Host *host)
 {
 	return *(struct ata_port **)&host->hostdata[0];
#1 lore raw
Quoted 27 lines · James Bottomley

ATA requires that all DMA transfers begin and end on word boundaries. Because of this, a large amount of machinery grew up in ide to adjust scatterlists on this basis. However, as of 2.5, the block layer has a dma_alignment variable which ensures both the beginning and length of a DMA transfer are aligned on the dma_alignment boundary. Although the block layer does adjust the beginning of the transfer to ensure this happens, it doesn't actually adjust the length, it merely makes sure that space is allocated for transfers beyond the declared length. The upshot of this is that scatterlists may be padded to any size between the actual length and the length adjusted to the dma_alignment safely knowing that memory is allocated in this region.

Right at the moment, SCSI takes the default dma_aligment which is on a 512 byte boundary. Note that this aligment only applies to transfers coming in from user space. However, since all kernel allocations are automatically aligned on a minimum of 32 byte boundaries, it is safe to adjust them in this manner as well.

---

This also fixes a current panic in libsas with SATAPI devices. I was trying to trace a bad interaction between the libata padding and the new sglist code which was the root cause, and ended up concluding that the whole libata padding thing was redundant.

Quoted 4 lines

In a future improvement, I think we can relax SCSI dma_alignemnt (it's causing almost every user space command to be copied instead of directly passed through) and allow devices and transports to build up their own requirements instead.

Initial read: ACK, with thanks. I'll queue this into libata#upstream, unless you have other suggestions.

This is definitely the sort of thing that low-level drivers (and their helper libs, e.g. libata) should not need to do.

Jeff

#2 lore raw
In reply to James Bottomley
Quoted 11 lines · On Mon, 31 Dec 2007 15:56:08 -0600 James Bottomley…

ATA requires that all DMA transfers begin and end on word boundaries. Because of this, a large amount of machinery grew up in ide to adjust scatterlists on this basis. However, as of 2.5, the block layer has a dma_alignment variable which ensures both the beginning and length of a DMA transfer are aligned on the dma_alignment boundary. Although the block layer does adjust the beginning of the transfer to ensure this happens, it doesn't actually adjust the length, it merely makes sure that space is allocated for transfers beyond the declared length. The upshot of this is that scatterlists may be padded to any size between the actual length and the length adjusted to the dma_alignment safely knowing that memory is allocated in this region.

Great!

Quoted 60 lines
include/linux/libata.h +1 -12
index 124033c..2f40d57 100644
@@ -282,7 +282,7 @@ enum {
 
 	/* size of buffer to pad xfers ending on unaligned boundaries */
 	ATA_DMA_PAD_SZ		= 4,
-	ATA_DMA_PAD_BUF_SZ	= ATA_DMA_PAD_SZ * ATA_MAX_QUEUE,
+	ATA_DMA_PAD_MASK	= ATA_DMA_PAD_SZ - 1,
 
 	/* ering size */
 	ATA_ERING_SIZE		= 32,
@@ -446,12 +446,9 @@ struct ata_queued_cmd {
 	unsigned long		flags;		/* ATA_QCFLAG_xxx */
 	unsigned int		tag;
 	unsigned int		n_elem;
-	unsigned int		n_iter;
-	unsigned int		orig_n_elem;
 
 	int			dma_dir;
 
-	unsigned int		pad_len;
 	unsigned int		sect_size;
 
 	unsigned int		nbytes;
@@ -461,7 +458,6 @@ struct ata_queued_cmd {
 	unsigned int		cursg_ofs;
 
 	struct scatterlist	sgent;
-	struct scatterlist	pad_sgent;
 	void			*buf_virt;
 
 	/* DO NOT iterate over __sg manually, use ata_for_each_sg() */
@@ -606,9 +602,6 @@ struct ata_port {
 	struct ata_prd		*prd;	 /* our SG list */
 	dma_addr_t		prd_dma; /* and its DMA mapping */
 
-	void			*pad;	/* array of DMA pad buffers */
-	dma_addr_t		pad_dma;
-
 	struct ata_ioports	ioaddr;	/* ATA cmd/ctl/dma register blocks */
 
 	u8			ctl;	/* cache of ATA control register */
@@ -1080,24 +1073,15 @@ extern void ata_port_pbar_desc(struct ata_port *ap, int bar, ssize_t offset,
 static inline struct scatterlist *
 ata_qc_first_sg(struct ata_queued_cmd *qc)
 {
-	qc->n_iter = 0;
 	if (qc->n_elem)
 		return qc->__sg;
-	if (qc->pad_len)
-		return &qc->pad_sgent;
 	return NULL;
 }
 
 static inline struct scatterlist *
 ata_qc_next_sg(struct scatterlist *sg, struct ata_queued_cmd *qc)
 {
-	if (sg == &qc->pad_sgent)

include/linux/libata.h extern void ata_port_pbar_desc(struct ata_port *ap, int bar, ssize_t offset, · line 1087

Quoted 10 lines
include/linux/libata.h +1 -6
-		return NULL;
-	if (++qc->n_iter < qc->n_elem)
-		return sg_next(sg);
-	if (qc->pad_len)
-		return &qc->pad_sgent;
-	return NULL;
+	return sg_next(sg);
 }
 
 #define ata_for_each_sg(sg, qc) \

How about removing ata_qc_first_sg and ata_qc_next_sg completely?

Now we can just replace ata_qc_next_sg with sg_next. qc->__sg seems to be always initialized to NULL so we can remove ata_qc_first_sg too.

include/linux/libata.h +1 -18
index 4f6404c..2774882 100644
@@ -1054,25 +1054,8 @@ extern void ata_port_pbar_desc(struct ata_port *ap, int bar, ssize_t offset,
 			       const char *name);
 #endif
 
-/*
- * qc helpers
- */
-static inline struct scatterlist *
-ata_qc_first_sg(struct ata_queued_cmd *qc)
-{
-	if (qc->n_elem)
-		return qc->__sg;
-	return NULL;
-}
-
-static inline struct scatterlist *
-ata_qc_next_sg(struct scatterlist *sg, struct ata_queued_cmd *qc)
-{
-	return sg_next(sg);
-}
-
 #define ata_for_each_sg(sg, qc) \
-	for (sg = ata_qc_first_sg(qc); sg; sg = ata_qc_next_sg(sg, qc))
+	for (sg = qc->__sg; sg; sg = sg_next(sg))
 
 static inline unsigned int ata_tag_valid(unsigned int tag)
 {
author maintainer
#3 lore raw
Quoted 27 lines · FUJITA Tomonori
Quoted 25 lines · On Mon, 31 Dec 2007 15:56:08 -0600 James Bottomley…
(quoted diff) +1 -10
@@ -1080,24 +1073,15 @@ extern void ata_port_pbar_desc(struct ata_port *ap, int bar, ssize_t offset,
 static inline struct scatterlist *
 ata_qc_first_sg(struct ata_queued_cmd *qc)
 {
-	qc->n_iter = 0;
 	if (qc->n_elem)
 		return qc->__sg;
-	if (qc->pad_len)
-		return &qc->pad_sgent;
 	return NULL;
 }
 
 static inline struct scatterlist *
 ata_qc_next_sg(struct scatterlist *sg, struct ata_queued_cmd *qc)
 {
-	if (sg == &qc->pad_sgent)
-		return NULL;
-	if (++qc->n_iter < qc->n_elem)
-		return sg_next(sg);
-	if (qc->pad_len)
-		return &qc->pad_sgent;
-	return NULL;
+	return sg_next(sg);
 }
 
Quoted 6 lines

include/linux/libata.h extern void ata_port_pbar_desc(struct ata_port *ap, int bar, ssize_t offset, · line 1057

Quoted 1 line
include/linux/libata.h +0 -0
 #define ata_for_each_sg(sg, qc) \

How about removing ata_qc_first_sg and ata_qc_next_sg completely?

Now we can just replace ata_qc_next_sg with sg_next. qc->__sg seems to be always initialized to NULL so we can remove ata_qc_first_sg too.

Sure ... I assumed (without actually looking) that the inlines were there because they were an API used throughout the drivers. Actually, grep tells me they're only used in the ata_for_each_sg macro, so this patch looks good.

Thanks,

James

Quoted 35 lines
include/linux/libata.h +1 -19
index 4f6404c..2774882 100644
@@ -1054,25 +1054,8 @@ extern void ata_port_pbar_desc(struct ata_port *ap, int bar, ssize_t offset,
 			       const char *name);
 #endif
 
-/*
- * qc helpers
- */
-static inline struct scatterlist *
-ata_qc_first_sg(struct ata_queued_cmd *qc)
-{
-	if (qc->n_elem)
-		return qc->__sg;
-	return NULL;
-}
-
-static inline struct scatterlist *
-ata_qc_next_sg(struct scatterlist *sg, struct ata_queued_cmd *qc)
-{
-	return sg_next(sg);
-}
-
 #define ata_for_each_sg(sg, qc) \
-	for (sg = ata_qc_first_sg(qc); sg; sg = ata_qc_next_sg(sg, qc))
+	for (sg = qc->__sg; sg; sg = sg_next(sg))
 
 static inline unsigned int ata_tag_valid(unsigned int tag)
 {
-

To unsubscribe from this list: send the line "unsubscribe linux-ide" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#4 lore raw
In reply to James Bottomley
Quoted 27 lines · James Bottomley

ATA requires that all DMA transfers begin and end on word boundaries. Because of this, a large amount of machinery grew up in ide to adjust scatterlists on this basis. However, as of 2.5, the block layer has a dma_alignment variable which ensures both the beginning and length of a DMA transfer are aligned on the dma_alignment boundary. Although the block layer does adjust the beginning of the transfer to ensure this happens, it doesn't actually adjust the length, it merely makes sure that space is allocated for transfers beyond the declared length. The upshot of this is that scatterlists may be padded to any size between the actual length and the length adjusted to the dma_alignment safely knowing that memory is allocated in this region.

Right at the moment, SCSI takes the default dma_aligment which is on a 512 byte boundary. Note that this aligment only applies to transfers coming in from user space. However, since all kernel allocations are automatically aligned on a minimum of 32 byte boundaries, it is safe to adjust them in this manner as well.

---

This also fixes a current panic in libsas with SATAPI devices. I was trying to trace a bad interaction between the libata padding and the new sglist code which was the root cause, and ended up concluding that the whole libata padding thing was redundant.

Quoted 4 lines

In a future improvement, I think we can relax SCSI dma_alignemnt (it's causing almost every user space command to be copied instead of directly passed through) and allow devices and transports to build up their own requirements instead.

Great but this intersects with currently queued and pending changes for libata-dev#upstream and needs to be merged. Also, DMA alignment at block layer isn't enough for ATA. ATA needs drain buffers for ATAPI commands with variable length response. :-(

Signature · 1 line
tejun
author maintainer
#5 lore raw
Quoted 28 lines · Tejun Heo
Quoted 27 lines · James Bottomley

ATA requires that all DMA transfers begin and end on word boundaries. Because of this, a large amount of machinery grew up in ide to adjust scatterlists on this basis. However, as of 2.5, the block layer has a dma_alignment variable which ensures both the beginning and length of a DMA transfer are aligned on the dma_alignment boundary. Although the block layer does adjust the beginning of the transfer to ensure this happens, it doesn't actually adjust the length, it merely makes sure that space is allocated for transfers beyond the declared length. The upshot of this is that scatterlists may be padded to any size between the actual length and the length adjusted to the dma_alignment safely knowing that memory is allocated in this region.

Right at the moment, SCSI takes the default dma_aligment which is on a 512 byte boundary. Note that this aligment only applies to transfers coming in from user space. However, since all kernel allocations are automatically aligned on a minimum of 32 byte boundaries, it is safe to adjust them in this manner as well.

---

This also fixes a current panic in libsas with SATAPI devices. I was trying to trace a bad interaction between the libata padding and the new sglist code which was the root cause, and ended up concluding that the whole libata padding thing was redundant.

Quoted 7 lines
Quoted 4 lines

In a future improvement, I think we can relax SCSI dma_alignemnt (it's causing almost every user space command to be copied instead of directly passed through) and allow devices and transports to build up their own requirements instead.

Great but this intersects with currently queued and pending changes for libata-dev#upstream and needs to be merged.

That's OK, I can do the merge .. that's what git is for.

Quoted 3 lines

Also, DMA alignment at
block layer isn't enough for ATA. ATA needs drain buffers for ATAPI commands with variable length response. :-(

OK, where is this in the libata code? The dma_pad size is only 4 bytes, so this drain, I assume is only a word long? Given the word alignment requirements of ATA doesn't this still mean it's only draining up to the word boundary anyway (so the code is still correct)?

James

#6 lore raw
Quoted 8 lines · James Bottomley
Quoted 3 lines

Also, DMA alignment at
block layer isn't enough for ATA. ATA needs drain buffers for ATAPI commands with variable length response. :-(

OK, where is this in the libata code? The dma_pad size is only 4 bytes, so this drain, I assume is only a word long? Given the word alignment requirements of ATA doesn't this still mean it's only draining up to the word boundary anyway (so the code is still correct)?

Patch is acked but not merged yet.

http://git.kernel.org/?p=linux/kernel/git/tj/libata-dev.git;a=commit;h=6cd22a0febc74bebe52d58eb22271b8770892a2d

The full function can be read from the following. It's ata_sg_setup_extra().

http://git.kernel.org/?p=linux/kernel/git/tj/libata-dev.git;a=blob;f=drivers/ata/libata-core.c;h=d763c072e6cefc724ea24cb68a7adf47b340f054;hb=6cd22a0febc74bebe52d58eb22271b8770892a2d

Signature · 1 line
tejun
author maintainer
#7 lore raw
Quoted 10 lines · Tejun Heo
Quoted 8 lines · James Bottomley
Quoted 3 lines

Also, DMA alignment at
block layer isn't enough for ATA. ATA needs drain buffers for ATAPI commands with variable length response. :-(

OK, where is this in the libata code? The dma_pad size is only 4 bytes, so this drain, I assume is only a word long? Given the word alignment requirements of ATA doesn't this still mean it's only draining up to the word boundary anyway (so the code is still correct)?

Quoted 8 lines

Patch is acked but not merged yet.

http://git.kernel.org/?p=linux/kernel/git/tj/libata-dev.git;a=commit;h=6cd22a0febc74bebe52d58eb22271b8770892a2d

The full function can be read from the following. It's ata_sg_setup_extra().

http://git.kernel.org/?p=linux/kernel/git/tj/libata-dev.git;a=blob;f=drivers/ata/libata-core.c;h=d763c072e6cefc724ea24cb68a7adf47b340f054;hb=6cd22a0febc74bebe52d58eb22271b8770892a2d

OK, but that patch was sent to the mailing list on 4 Jan ... five days after this one. It's a little hard to take unposted patches into account ...

It's a fairly comprehensive merge clash ... where is this drain patch on the upstream track? because currently ipr and aic94xx panic the system without the dma padding removal (I just figured -rc6 was a bit late for major surgery like this, but I was planning a backport if it stood up in 2.6.24).

James

author maintainer
#8 lore raw

PATCH RESEND 1/1 libata: eliminate the home grown dma padding in favour of that provided by the block layer

In reply to James Bottomley

ATA requires that all DMA transfers begin and end on word boundaries. Because of this, a large amount of machinery grew up in ide to adjust scatterlists on this basis. However, as of 2.5, the block layer has a dma_alignment variable which ensures both the beginning and length of a DMA transfer are aligned on the dma_alignment boundary. Although the block layer does adjust the beginning of the transfer to ensure this happens, it doesn't actually adjust the length, it merely makes sure that space is allocated for transfers beyond the declared length. The upshot of this is that scatterlists may be padded to any size between the actual length and the length adjusted to the dma_alignment safely knowing that memory is allocated in this region.

Right at the moment, SCSI takes the default dma_aligment which is on a 512 byte boundary. Note that this aligment only applies to transfers coming in from user space. However, since all kernel allocations are automatically aligned on a minimum of 32 byte boundaries, it is safe to adjust them in this manner as well.

---
 drivers/ata/ahci.c            |    5 --
 drivers/ata/libata-core.c     |  146 +++++-----------------------------------
 drivers/ata/libata-scsi.c     |   16 +----
 drivers/ata/pata_icside.c     |    8 --
 drivers/ata/sata_fsl.c        |   17 +----
 drivers/ata/sata_mv.c         |    5 --
 drivers/ata/sata_sil24.c      |    5 --
 drivers/scsi/ipr.c            |    4 +-
 drivers/scsi/libsas/sas_ata.c |    4 +-
 include/linux/libata.h        |   28 +--------
 10 files changed, 28 insertions(+), 210 deletions(-)
drivers/ata/ahci.c +0 -5
index 49761bc..9b363ea 100644
@@ -1964,16 +1964,11 @@ static int ahci_port_start(struct ata_port *ap)
 	struct ahci_port_priv *pp;
 	void *mem;
 	dma_addr_t mem_dma;
-	int rc;
 
 	pp = devm_kzalloc(dev, sizeof(*pp), GFP_KERNEL);
 	if (!pp)
 		return -ENOMEM;
 
-	rc = ata_pad_alloc(ap, dev);
-	if (rc)
-		return rc;
-
 	mem = dmam_alloc_coherent(dev, AHCI_PORT_PRIV_DMA_SZ, &mem_dma,
 				  GFP_KERNEL);
 	if (!mem)
drivers/ata/libata-core.c +19 -127

195 lines not on this page · raw message

drivers/ata/libata-scsi.c +2 -14
index 58f0208..c4aeaab 100644
@@ -835,15 +835,6 @@ static void ata_scsi_dev_config(struct scsi_device *sdev,
 	/* configure max sectors */
 	blk_queue_max_sectors(sdev->request_queue, dev->max_sectors);
 
-	/* SATA DMA transfers must be multiples of 4 byte, so
-	 * we need to pad ATAPI transfers using an extra sg.
-	 * Decrement max hw segments accordingly.
-	 */
-	if (dev->class == ATA_DEV_ATAPI) {
-		struct request_queue *q = sdev->request_queue;
-		blk_queue_max_hw_segments(q, q->max_hw_segments - 1);
-	}
-
 	if (dev->class == ATA_DEV_ATA)
 		sdev->manage_start_stop = 1;
 
@@ -3551,7 +3542,7 @@ EXPORT_SYMBOL_GPL(ata_sas_port_alloc);
  *	@ap: Port to initialize
  *
  *	Called just after data structures for each port are
- *	initialized.  Allocates DMA pad.
+ *	initialized.
  *
  *	May be used as the port_start() entry in ata_port_operations.
  *
@@ -3560,7 +3551,7 @@ EXPORT_SYMBOL_GPL(ata_sas_port_alloc);
  */
 int ata_sas_port_start(struct ata_port *ap)
 {
-	return ata_pad_alloc(ap, ap->dev);
+	return 0;
 }
 EXPORT_SYMBOL_GPL(ata_sas_port_start);
 
@@ -3568,8 +3559,6 @@ EXPORT_SYMBOL_GPL(ata_sas_port_start);
  *	ata_port_stop - Undo ata_sas_port_start()
  *	@ap: Port to shut down
  *
- *	Frees the DMA pad.
- *
  *	May be used as the port_stop() entry in ata_port_operations.
  *
  *	LOCKING:
@@ -3578,7 +3567,6 @@ EXPORT_SYMBOL_GPL(ata_sas_port_start);
 
 void ata_sas_port_stop(struct ata_port *ap)
 {
-	ata_pad_free(ap, ap->dev);
 }
 EXPORT_SYMBOL_GPL(ata_sas_port_stop);
 
drivers/ata/pata_icside.c +0 -8
index 5b8586d..f97068b 100644
@@ -304,12 +304,6 @@ static int icside_dma_init(struct pata_icside_info *info)
 }
 
 
-static int pata_icside_port_start(struct ata_port *ap)
-{
-	/* No PRD to alloc */
-	return ata_pad_alloc(ap, ap->dev);
-}
-
 static struct scsi_host_template pata_icside_sht = {
 	.module			= THIS_MODULE,
 	.name			= DRV_NAME,
@@ -389,8 +383,6 @@ static struct ata_port_operations pata_icside_port_ops = {
 	.irq_clear		= ata_dummy_noret,
 	.irq_on			= ata_irq_on,
 
-	.port_start		= pata_icside_port_start,
-
 	.bmdma_stop		= pata_icside_bmdma_stop,
 	.bmdma_status		= pata_icside_bmdma_status,
 };
drivers/ata/sata_fsl.c +2 -15
index d041709..5d40e4b 100644
@@ -356,8 +356,8 @@ static unsigned int sata_fsl_fill_sg(struct ata_queued_cmd *qc, void *cmd_desc,
 			ata_port_printk(qc->ap, KERN_ERR,
 					"s/g len unaligned : 0x%x\n", sg_len);
 
-		if ((num_prde == (SATA_FSL_MAX_PRD_DIRECT - 1)) &&
-		    (qc->n_iter + 1 != qc->n_elem)) {
+		if (num_prde == (SATA_FSL_MAX_PRD_DIRECT - 1) &&
+		    sg_next(sg) != NULL) {
 			VPRINTK("setting indirect prde\n");
 			prd_ptr_to_indirect_ext = prd;
 			prd->dba = cpu_to_le32(indirect_ext_segment_paddr);
@@ -602,21 +602,9 @@ static int sata_fsl_port_start(struct ata_port *ap)
 	if (!pp)
 		return -ENOMEM;
 
-	/*
-	 * allocate per command dma alignment pad buffer, which is used
-	 * internally by libATA to ensure that all transfers ending on
-	 * unaligned boundaries are padded, to align on Dword boundaries
-	 */
-	retval = ata_pad_alloc(ap, dev);
-	if (retval) {
-		kfree(pp);
-		return retval;
-	}
-
 	mem = dma_alloc_coherent(dev, SATA_FSL_PORT_PRIV_DMA_SZ, &mem_dma,
 				 GFP_KERNEL);
 	if (!mem) {
-		ata_pad_free(ap, dev);
 		kfree(pp);
 		return -ENOMEM;
 	}
@@ -695,7 +683,6 @@ static void sata_fsl_port_stop(struct ata_port *ap)
 	dma_free_coherent(dev, SATA_FSL_PORT_PRIV_DMA_SZ,
 			  pp->cmdslot, pp->cmdslot_paddr);
 
-	ata_pad_free(ap, dev);
 	kfree(pp);
 }
 
drivers/ata/sata_mv.c +0 -5
index 7e72463..73fb2ba 100644
@@ -1054,7 +1054,6 @@ static int mv_port_start(struct ata_port *ap)
 	void *mem;
 	dma_addr_t mem_dma;
 	unsigned long flags;
-	int rc;
 
 	pp = devm_kzalloc(dev, sizeof(*pp), GFP_KERNEL);
 	if (!pp)
@@ -1066,10 +1065,6 @@ static int mv_port_start(struct ata_port *ap)
 		return -ENOMEM;
 	memset(mem, 0, MV_PORT_PRIV_DMA_SZ);
 
-	rc = ata_pad_alloc(ap, dev);
-	if (rc)
-		return rc;
-
 	/* First item in chunk of DMA memory:
 	 * 32-slot command request table (CRQB), 32 bytes each in size
 	 */
drivers/ata/sata_sil24.c +0 -5
index b4b1f91..df7988d 100644
@@ -1234,7 +1234,6 @@ static int sil24_port_start(struct ata_port *ap)
 	union sil24_cmd_block *cb;
 	size_t cb_size = sizeof(*cb) * SIL24_MAX_CMDS;
 	dma_addr_t cb_dma;
-	int rc;
 
 	pp = devm_kzalloc(dev, sizeof(*pp), GFP_KERNEL);
 	if (!pp)
@@ -1247,10 +1246,6 @@ static int sil24_port_start(struct ata_port *ap)
 		return -ENOMEM;
 	memset(cb, 0, cb_size);
 
-	rc = ata_pad_alloc(ap, dev);
-	if (rc)
-		return rc;
-
 	pp->cmd_block = cb;
 	pp->cmd_block_dma = cb_dma;
 
drivers/scsi/ipr.c +2 -2
index 73270ff..6762e89 100644
@@ -5140,7 +5140,7 @@ static void ipr_build_ata_ioadl(struct ipr_cmnd *ipr_cmd,
 	struct ipr_ioarcb *ioarcb = &ipr_cmd->ioarcb;
 	struct ipr_ioadl_desc *ioadl = ipr_cmd->ioadl;
 	struct ipr_ioadl_desc *last_ioadl = NULL;
-	int len = qc->nbytes + qc->pad_len;
+	int len = qc->nbytes;
 	struct scatterlist *sg;
 	unsigned int si;
 
@@ -5206,7 +5206,7 @@ static unsigned int ipr_qc_issue(struct ata_queued_cmd *qc)
 	ioarcb->cmd_pkt.request_type = IPR_RQTYPE_ATA_PASSTHRU;
 	ioarcb->cmd_pkt.flags_hi |= IPR_FLAGS_HI_NO_LINK_DESC;
 	ioarcb->cmd_pkt.flags_hi |= IPR_FLAGS_HI_NO_ULEN_CHK;
-	ipr_cmd->dma_use_sg = qc->pad_len ? qc->n_elem + 1 : qc->n_elem;
+	ipr_cmd->dma_use_sg = qc->n_elem;
 
 	ipr_build_ata_ioadl(ipr_cmd, qc);
 	regs->flags |= IPR_ATA_FLAG_STATUS_ON_GOOD_COMPLETION;
drivers/scsi/libsas/sas_ata.c +2 -2
index 0996f86..7cd05b5 100644
@@ -178,8 +178,8 @@ static unsigned int sas_ata_qc_issue(struct ata_queued_cmd *qc)
 	task->uldd_task = qc;
 	if (ata_is_atapi(qc->tf.protocol)) {
 		memcpy(task->ata_task.atapi_packet, qc->cdb, qc->dev->cdb_len);
-		task->total_xfer_len = qc->nbytes + qc->pad_len;
-		task->num_scatter = qc->pad_len ? qc->n_elem + 1 : qc->n_elem;
+		task->total_xfer_len = qc->nbytes;
+		task->num_scatter = qc->n_elem;
 	} else {
 		for_each_sg(qc->sg, sg, qc->n_elem, si)
 			xfer += sg->length;
include/linux/libata.h +1 -27
index ccb0556..8ee6dfc 100644
@@ -278,7 +278,7 @@ enum {
 
 	/* size of buffer to pad xfers ending on unaligned boundaries */
 	ATA_DMA_PAD_SZ		= 4,
-	ATA_DMA_PAD_BUF_SZ	= ATA_DMA_PAD_SZ * ATA_MAX_QUEUE,
+	ATA_DMA_PAD_MASK	= ATA_DMA_PAD_SZ - 1,
 
 	/* ering size */
 	ATA_ERING_SIZE		= 32,
@@ -457,12 +457,9 @@ struct ata_queued_cmd {
 	unsigned long		flags;		/* ATA_QCFLAG_xxx */
 	unsigned int		tag;
 	unsigned int		n_elem;
-	unsigned int		n_iter;
-	unsigned int		mapped_n_elem;
 
 	int			dma_dir;
 
-	unsigned int		pad_len;
 	unsigned int		sect_size;
 
 	unsigned int		nbytes;
@@ -472,10 +469,7 @@ struct ata_queued_cmd {
 	struct scatterlist	*cursg;
 	unsigned int		cursg_ofs;
 
-	struct scatterlist	*last_sg;
-	struct scatterlist	saved_last_sg;
 	struct scatterlist	sgent;
-	struct scatterlist	extra_sg[2];
 
 	struct scatterlist	*sg;
 
@@ -620,9 +614,6 @@ struct ata_port {
 	struct ata_prd		*prd;	 /* our SG list */
 	dma_addr_t		prd_dma; /* and its DMA mapping */
 
-	void			*pad;	/* array of DMA pad buffers */
-	dma_addr_t		pad_dma;
-
 	struct ata_ioports	ioaddr;	/* ATA cmd/ctl/dma register blocks */
 
 	u8			ctl;	/* cache of ATA control register */
@@ -1363,11 +1354,7 @@ static inline void ata_qc_reinit(struct ata_queued_cmd *qc)
 	qc->cursg_ofs = 0;
 	qc->nbytes = qc->raw_nbytes = qc->curbytes = 0;
 	qc->n_elem = 0;
-	qc->mapped_n_elem = 0;
-	qc->n_iter = 0;
 	qc->err_mask = 0;
-	qc->pad_len = 0;
-	qc->last_sg = NULL;
 	qc->sect_size = ATA_SECT_SIZE;
 
 	ata_tf_init(qc->dev, &qc->tf);
@@ -1422,19 +1409,6 @@ static inline unsigned int __ac_err_mask(u8 status)
 	return mask;
 }
 
-static inline int ata_pad_alloc(struct ata_port *ap, struct device *dev)
-{
-	ap->pad_dma = 0;
-	ap->pad = dmam_alloc_coherent(dev, ATA_DMA_PAD_BUF_SZ,
-				      &ap->pad_dma, GFP_KERNEL);
-	return (ap->pad == NULL) ? -ENOMEM : 0;
-}
-
-static inline void ata_pad_free(struct ata_port *ap, struct device *dev)
-{
-	dmam_free_coherent(dev, ATA_DMA_PAD_BUF_SZ, ap->pad, ap->pad_dma);
-}
-
 static inline struct ata_port *ata_shost_to_port(struct Scsi_Host *host)
 {
 	return *(struct ata_port **)&host->hostdata[0];
Signature · 1 line
1.5.3.7
author maintainer
#9 lore raw

PATCH RESEND 1/1 libata: eliminate the home grown dma padding in favour of that provided by the block layer

Could we please get this in ... I thought I mentioned several times that it fixes a fatal oops in both aic94xx and ipr.

James

---

From: James Bottomley <[email protected]>
Date: Mon, 31 Dec 2007 15:56:08 -0600
Subject: libata: eliminate the home grown dma padding in favour of that provided by the block layer

ATA requires that all DMA transfers begin and end on word boundaries. Because of this, a large amount of machinery grew up in ide to adjust scatterlists on this basis. However, as of 2.5, the block layer has a dma_alignment variable which ensures both the beginning and length of a DMA transfer are aligned on the dma_alignment boundary. Although the block layer does adjust the beginning of the transfer to ensure this happens, it doesn't actually adjust the length, it merely makes sure that space is allocated for transfers beyond the declared length. The upshot of this is that scatterlists may be padded to any size between the actual length and the length adjusted to the dma_alignment safely knowing that memory is allocated in this region.

Right at the moment, SCSI takes the default dma_aligment which is on a 512 byte boundary. Note that this aligment only applies to transfers coming in from user space. However, since all kernel allocations are automatically aligned on a minimum of 32 byte boundaries, it is safe to adjust them in this manner as well.

---
 drivers/ata/ahci.c            |    5 --
 drivers/ata/libata-core.c     |  146 +++++-----------------------------------
 drivers/ata/libata-scsi.c     |   23 +-----
 drivers/ata/pata_icside.c     |    8 --
 drivers/ata/sata_fsl.c        |   17 +----
 drivers/ata/sata_mv.c         |    5 --
 drivers/ata/sata_sil24.c      |    5 --
 drivers/scsi/ipr.c            |    4 +-
 drivers/scsi/libsas/sas_ata.c |    4 +-
 include/linux/libata.h        |   28 +--------
 10 files changed, 30 insertions(+), 215 deletions(-)
drivers/ata/ahci.c +0 -5
index 6f089b8..2a5a77a 100644
@@ -1977,16 +1977,11 @@ static int ahci_port_start(struct ata_port *ap)
 	struct ahci_port_priv *pp;
 	void *mem;
 	dma_addr_t mem_dma;
-	int rc;
 
 	pp = devm_kzalloc(dev, sizeof(*pp), GFP_KERNEL);
 	if (!pp)
 		return -ENOMEM;
 
-	rc = ata_pad_alloc(ap, dev);
-	if (rc)
-		return rc;
-
 	mem = dmam_alloc_coherent(dev, AHCI_PORT_PRIV_DMA_SZ, &mem_dma,
 				  GFP_KERNEL);
 	if (!mem)
drivers/ata/libata-core.c +19 -127

195 lines not on this page · raw message

drivers/ata/libata-scsi.c +4 -19
index c02c490..8ffff44 100644
@@ -832,24 +832,12 @@ static void ata_scsi_dev_config(struct scsi_device *sdev,
 	/* configure max sectors */
 	blk_queue_max_sectors(sdev->request_queue, dev->max_sectors);
 
-	/* SATA DMA transfers must be multiples of 4 byte, so
-	 * we need to pad ATAPI transfers using an extra sg.
-	 * Decrement max hw segments accordingly.
-	 */
-	if (dev->class == ATA_DEV_ATAPI) {
-		struct request_queue *q = sdev->request_queue;
-		blk_queue_max_hw_segments(q, q->max_hw_segments - 1);
-
-		/* set the min alignment */
-		blk_queue_update_dma_alignment(sdev->request_queue,
-					       ATA_DMA_PAD_SZ - 1);
-	} else
+	if (dev->class == ATA_DEV_ATA) {
 		/* ATA devices must be sector aligned */
 		blk_queue_update_dma_alignment(sdev->request_queue,
 					       ATA_SECT_SIZE - 1);
-
-	if (dev->class == ATA_DEV_ATA)
 		sdev->manage_start_stop = 1;
+	}
 
 	if (dev->flags & ATA_DFLAG_AN)
 		set_bit(SDEV_EVT_MEDIA_CHANGE, sdev->supported_events);
@@ -3555,7 +3543,7 @@ EXPORT_SYMBOL_GPL(ata_sas_port_alloc);
  *	@ap: Port to initialize
  *
  *	Called just after data structures for each port are
- *	initialized.  Allocates DMA pad.
+ *	initialized.
  *
  *	May be used as the port_start() entry in ata_port_operations.
  *
@@ -3564,7 +3552,7 @@ EXPORT_SYMBOL_GPL(ata_sas_port_alloc);
  */
 int ata_sas_port_start(struct ata_port *ap)
 {
-	return ata_pad_alloc(ap, ap->dev);
+	return 0;
 }
 EXPORT_SYMBOL_GPL(ata_sas_port_start);
 
@@ -3572,8 +3560,6 @@ EXPORT_SYMBOL_GPL(ata_sas_port_start);
  *	ata_port_stop - Undo ata_sas_port_start()
  *	@ap: Port to shut down
  *
- *	Frees the DMA pad.
- *
  *	May be used as the port_stop() entry in ata_port_operations.
  *
  *	LOCKING:
@@ -3582,7 +3568,6 @@ EXPORT_SYMBOL_GPL(ata_sas_port_start);
 
 void ata_sas_port_stop(struct ata_port *ap)
 {
-	ata_pad_free(ap, ap->dev);
 }
 EXPORT_SYMBOL_GPL(ata_sas_port_stop);
 
drivers/ata/pata_icside.c +0 -8
index 5b8586d..f97068b 100644
@@ -304,12 +304,6 @@ static int icside_dma_init(struct pata_icside_info *info)
 }
 
 
-static int pata_icside_port_start(struct ata_port *ap)
-{
-	/* No PRD to alloc */
-	return ata_pad_alloc(ap, ap->dev);
-}
-
 static struct scsi_host_template pata_icside_sht = {
 	.module			= THIS_MODULE,
 	.name			= DRV_NAME,
@@ -389,8 +383,6 @@ static struct ata_port_operations pata_icside_port_ops = {
 	.irq_clear		= ata_dummy_noret,
 	.irq_on			= ata_irq_on,
 
-	.port_start		= pata_icside_port_start,
-
 	.bmdma_stop		= pata_icside_bmdma_stop,
 	.bmdma_status		= pata_icside_bmdma_status,
 };
drivers/ata/sata_fsl.c +2 -15
index 922d7b2..9323dd0 100644
@@ -355,8 +355,8 @@ static unsigned int sata_fsl_fill_sg(struct ata_queued_cmd *qc, void *cmd_desc,
 			ata_port_printk(qc->ap, KERN_ERR,
 					"s/g len unaligned : 0x%x\n", sg_len);
 
-		if ((num_prde == (SATA_FSL_MAX_PRD_DIRECT - 1)) &&
-		    (qc->n_iter + 1 != qc->n_elem)) {
+		if (num_prde == (SATA_FSL_MAX_PRD_DIRECT - 1) &&
+		    sg_next(sg) != NULL) {
 			VPRINTK("setting indirect prde\n");
 			prd_ptr_to_indirect_ext = prd;
 			prd->dba = cpu_to_le32(indirect_ext_segment_paddr);
@@ -601,21 +601,9 @@ static int sata_fsl_port_start(struct ata_port *ap)
 	if (!pp)
 		return -ENOMEM;
 
-	/*
-	 * allocate per command dma alignment pad buffer, which is used
-	 * internally by libATA to ensure that all transfers ending on
-	 * unaligned boundaries are padded, to align on Dword boundaries
-	 */
-	retval = ata_pad_alloc(ap, dev);
-	if (retval) {
-		kfree(pp);
-		return retval;
-	}
-
 	mem = dma_alloc_coherent(dev, SATA_FSL_PORT_PRIV_DMA_SZ, &mem_dma,
 				 GFP_KERNEL);
 	if (!mem) {
-		ata_pad_free(ap, dev);
 		kfree(pp);
 		return -ENOMEM;
 	}
@@ -694,7 +682,6 @@ static void sata_fsl_port_stop(struct ata_port *ap)
 	dma_free_coherent(dev, SATA_FSL_PORT_PRIV_DMA_SZ,
 			  pp->cmdslot, pp->cmdslot_paddr);
 
-	ata_pad_free(ap, dev);
 	kfree(pp);
 }
 
drivers/ata/sata_mv.c +0 -5
index 7e72463..73fb2ba 100644
@@ -1054,7 +1054,6 @@ static int mv_port_start(struct ata_port *ap)
 	void *mem;
 	dma_addr_t mem_dma;
 	unsigned long flags;
-	int rc;
 
 	pp = devm_kzalloc(dev, sizeof(*pp), GFP_KERNEL);
 	if (!pp)
@@ -1066,10 +1065,6 @@ static int mv_port_start(struct ata_port *ap)
 		return -ENOMEM;
 	memset(mem, 0, MV_PORT_PRIV_DMA_SZ);
 
-	rc = ata_pad_alloc(ap, dev);
-	if (rc)
-		return rc;
-
 	/* First item in chunk of DMA memory:
 	 * 32-slot command request table (CRQB), 32 bytes each in size
 	 */
drivers/ata/sata_sil24.c +0 -5
index b4b1f91..df7988d 100644
@@ -1234,7 +1234,6 @@ static int sil24_port_start(struct ata_port *ap)
 	union sil24_cmd_block *cb;
 	size_t cb_size = sizeof(*cb) * SIL24_MAX_CMDS;
 	dma_addr_t cb_dma;
-	int rc;
 
 	pp = devm_kzalloc(dev, sizeof(*pp), GFP_KERNEL);
 	if (!pp)
@@ -1247,10 +1246,6 @@ static int sil24_port_start(struct ata_port *ap)
 		return -ENOMEM;
 	memset(cb, 0, cb_size);
 
-	rc = ata_pad_alloc(ap, dev);
-	if (rc)
-		return rc;
-
 	pp->cmd_block = cb;
 	pp->cmd_block_dma = cb_dma;
 
drivers/scsi/ipr.c +2 -2
index 73270ff..6762e89 100644
@@ -5140,7 +5140,7 @@ static void ipr_build_ata_ioadl(struct ipr_cmnd *ipr_cmd,
 	struct ipr_ioarcb *ioarcb = &ipr_cmd->ioarcb;
 	struct ipr_ioadl_desc *ioadl = ipr_cmd->ioadl;
 	struct ipr_ioadl_desc *last_ioadl = NULL;
-	int len = qc->nbytes + qc->pad_len;
+	int len = qc->nbytes;
 	struct scatterlist *sg;
 	unsigned int si;
 
@@ -5206,7 +5206,7 @@ static unsigned int ipr_qc_issue(struct ata_queued_cmd *qc)
 	ioarcb->cmd_pkt.request_type = IPR_RQTYPE_ATA_PASSTHRU;
 	ioarcb->cmd_pkt.flags_hi |= IPR_FLAGS_HI_NO_LINK_DESC;
 	ioarcb->cmd_pkt.flags_hi |= IPR_FLAGS_HI_NO_ULEN_CHK;
-	ipr_cmd->dma_use_sg = qc->pad_len ? qc->n_elem + 1 : qc->n_elem;
+	ipr_cmd->dma_use_sg = qc->n_elem;
 
 	ipr_build_ata_ioadl(ipr_cmd, qc);
 	regs->flags |= IPR_ATA_FLAG_STATUS_ON_GOOD_COMPLETION;
drivers/scsi/libsas/sas_ata.c +2 -2
index 0996f86..7cd05b5 100644
@@ -178,8 +178,8 @@ static unsigned int sas_ata_qc_issue(struct ata_queued_cmd *qc)
 	task->uldd_task = qc;
 	if (ata_is_atapi(qc->tf.protocol)) {
 		memcpy(task->ata_task.atapi_packet, qc->cdb, qc->dev->cdb_len);
-		task->total_xfer_len = qc->nbytes + qc->pad_len;
-		task->num_scatter = qc->pad_len ? qc->n_elem + 1 : qc->n_elem;
+		task->total_xfer_len = qc->nbytes;
+		task->num_scatter = qc->n_elem;
 	} else {
 		for_each_sg(qc->sg, sg, qc->n_elem, si)
 			xfer += sg->length;
include/linux/libata.h +1 -27
index 4374c42..be2ace3 100644
@@ -278,7 +278,7 @@ enum {
 
 	/* size of buffer to pad xfers ending on unaligned boundaries */
 	ATA_DMA_PAD_SZ		= 4,
-	ATA_DMA_PAD_BUF_SZ	= ATA_DMA_PAD_SZ * ATA_MAX_QUEUE,
+	ATA_DMA_PAD_MASK	= ATA_DMA_PAD_SZ - 1,
 
 	/* ering size */
 	ATA_ERING_SIZE		= 32,
@@ -457,12 +457,9 @@ struct ata_queued_cmd {
 	unsigned long		flags;		/* ATA_QCFLAG_xxx */
 	unsigned int		tag;
 	unsigned int		n_elem;
-	unsigned int		n_iter;
-	unsigned int		mapped_n_elem;
 
 	int			dma_dir;
 
-	unsigned int		pad_len;
 	unsigned int		sect_size;
 
 	unsigned int		nbytes;
@@ -472,10 +469,7 @@ struct ata_queued_cmd {
 	struct scatterlist	*cursg;
 	unsigned int		cursg_ofs;
 
-	struct scatterlist	*last_sg;
-	struct scatterlist	saved_last_sg;
 	struct scatterlist	sgent;
-	struct scatterlist	extra_sg[2];
 
 	struct scatterlist	*sg;
 
@@ -620,9 +614,6 @@ struct ata_port {
 	struct ata_prd		*prd;	 /* our SG list */
 	dma_addr_t		prd_dma; /* and its DMA mapping */
 
-	void			*pad;	/* array of DMA pad buffers */
-	dma_addr_t		pad_dma;
-
 	struct ata_ioports	ioaddr;	/* ATA cmd/ctl/dma register blocks */
 
 	u8			ctl;	/* cache of ATA control register */
@@ -1366,11 +1357,7 @@ static inline void ata_qc_reinit(struct ata_queued_cmd *qc)
 	qc->cursg_ofs = 0;
 	qc->nbytes = qc->raw_nbytes = qc->curbytes = 0;
 	qc->n_elem = 0;
-	qc->mapped_n_elem = 0;
-	qc->n_iter = 0;
 	qc->err_mask = 0;
-	qc->pad_len = 0;
-	qc->last_sg = NULL;
 	qc->sect_size = ATA_SECT_SIZE;
 
 	ata_tf_init(qc->dev, &qc->tf);
@@ -1425,19 +1412,6 @@ static inline unsigned int __ac_err_mask(u8 status)
 	return mask;
 }
 
-static inline int ata_pad_alloc(struct ata_port *ap, struct device *dev)
-{
-	ap->pad_dma = 0;
-	ap->pad = dmam_alloc_coherent(dev, ATA_DMA_PAD_BUF_SZ,
-				      &ap->pad_dma, GFP_KERNEL);
-	return (ap->pad == NULL) ? -ENOMEM : 0;
-}
-
-static inline void ata_pad_free(struct ata_port *ap, struct device *dev)
-{
-	dmam_free_coherent(dev, ATA_DMA_PAD_BUF_SZ, ap->pad, ap->pad_dma);
-}
-
 static inline struct ata_port *ata_shost_to_port(struct Scsi_Host *host)
 {
 	return *(struct ata_port **)&host->hostdata[0];
Signature · 1 line
1.5.3.8
#10 lore raw
Quoted 2 lines · James Bottomley

Could we please get this in ... I thought I mentioned several times that it fixes a fatal oops in both aic94xx and ipr.

Tejun has a persistent objection... see other email.

Jeff

author maintainer
#11 lore raw
Quoted 5 lines · Jeff Garzik
Quoted 2 lines · James Bottomley

Could we please get this in ... I thought I mentioned several times that it fixes a fatal oops in both aic94xx and ipr.

Tejun has a persistent objection... see other email.

Actually, see other email .. I meant that this patch (eliminate dma padding) is independent of the drain one.

James

#12 lore raw
Quoted 8 lines · James Bottomley
Quoted 4 lines · Jeff Garzik
Quoted 2 lines · James Bottomley

Could we please get this in ... I thought I mentioned several times that it fixes a fatal oops in both aic94xx and ipr.

Tejun has a persistent objection... see other email.

Actually, see other email .. I meant that this patch (eliminate dma padding) is independent of the drain one.

Sorry about the delay.

There's a problem here. For the blk layer dma padding itself, it's okay but the problem is that it blocks the pending draining patch without supplying usable alternative at the moment. I agree that the long term solution should be in the block layer && I understand that it causes problem for SAS controllers but for the moment if we don't include the existing draining patch, far more ATAPI devices are affected. So, it's catch-22 situation.

I think the best solution is to update block layer draining such that it can be included together before the merge window closes. I'll dig into it.

Thanks.

Signature · 1 line
tejun
author maintainer
#13 lore raw
Quoted 7 lines · Tejun Heo
Quoted 6 lines · James Bottomley
Quoted 4 lines · Jeff Garzik
Quoted 2 lines · James Bottomley

Could we please get this in ... I thought I mentioned several times that it fixes a fatal oops in both aic94xx and ipr.

Tejun has a persistent objection... see other email.

Quoted 8 lines
Quoted 2 lines

Actually, see other email .. I meant that this patch (eliminate dma padding) is independent of the drain one.

Sorry about the delay.

There's a problem here. For the blk layer dma padding itself, it's okay but the problem is that it blocks the pending draining patch without supplying usable alternative at the moment.

The alternative is already there. The current drain infrastructure is already in the block layer. The patch I sent merely activates it for ATA. Is there some issue with this that I haven't forseen? My analysis is that it should do everything your patch does (except at the block layer and in a manner that doesn't trigger the aic94xx panic).

Quoted 8 lines

I agree that the long term
solution should be in the block layer && I understand that it causes problem for SAS controllers but for the moment if we don't include the existing draining patch, far more ATAPI devices are affected. So, it's catch-22 situation.

I think the best solution is to update block layer draining such that it can be included together before the merge window closes. I'll dig into it.

Like I said, the block layer pieces are already upstream. All we need is the ATA bits and I think it should all work ... unless there's some part I haven't though of?

James

#14 lore raw
Quoted 6 lines · James Bottomley
Quoted 2 lines

I think the best solution is to update block layer draining such that it can be included together before the merge window closes. I'll dig into it.

Like I said, the block layer pieces are already upstream. All we need is the ATA bits and I think it should all work ... unless there's some part I haven't though of?

Yeah, I think we'll probably need to add rq->raw_data_len and I'd really like to map longer drain area but I think the basics are there. What's needed is updating libata accordingly and testing it. I'm currently away from all my toys. I'll implement the ATA part, test it and submit the patch on Monday.

Thanks.

Signature · 1 line
tejun
author maintainer
#15 lore raw
Quoted 10 lines · Tejun Heo
Quoted 6 lines · James Bottomley
Quoted 2 lines

I think the best solution is to update block layer draining such that it can be included together before the merge window closes. I'll dig into it.

Like I said, the block layer pieces are already upstream. All we need is the ATA bits and I think it should all work ... unless there's some part I haven't though of?

Yeah, I think we'll probably need to add rq->raw_data_len and I'd really like to map longer drain area but I think the basics are there.

I'm reluctant to add another parameter to the request, but this one you can calculate: you just do it wherever you work out the size of the request. If data_len is the true data length and total_data_len is the data length plus the drain length, the calculation fragment is

if (blk_pc_request(req))

	data_len = req->data_len;
else
	data_len = req->nr_sectors << 9;

total_data_len = data_len + req->q->dma_drain_size;

If the request has already been mapped by scsi, then data_len is actually scsi_cmnd->sdb.length

Quoted 1 line

What's needed is updating libata accordingly and testing it.

Actually, I sent the patch to do this a few days ago:

http://marc.info/?l=linux-ide&m=120189565418258

But I've attached it again.

Quoted 2 lines

I'm currently away from all my toys. I'll implement the ATA part, test it and submit the patch on Monday.

Great, will look forward to the results. Like I said, I think the turn on draining in slave configure should be narrowed from all ATAPI devices to all AHCI ATAPI devices if there's no evidence that any other implementation that uses DMA to transfer PIO isn't similarly broken (I know the aic94xx works fine, for instance ...

James

---

From: James Bottomley <[email protected]>
Date: Thu, 10 Jan 2008 11:42:50 -0600
Subject: libata: implement drain buffers

This just updates the libata slave configure routine to take advantage of the block layer drain buffers.

I suspect I should also be checking for AHCI as well as ATA_DEV_ATAPI, but I couldn't see how to do that easily.

---
 drivers/ata/libata-scsi.c |   26 ++++++++++++++++++++++----
 1 files changed, 22 insertions(+), 4 deletions(-)
drivers/ata/libata-scsi.c +22 -4
index 8ffff44..a28351c 100644
@@ -826,8 +826,8 @@ static void ata_scsi_sdev_config(struct scsi_device *sdev)
 	sdev->max_device_blocked = 1;
 }
 
-static void ata_scsi_dev_config(struct scsi_device *sdev,
-				struct ata_device *dev)
+static int ata_scsi_dev_config(struct scsi_device *sdev,
+			       struct ata_device *dev)
 {
 	/* configure max sectors */
 	blk_queue_max_sectors(sdev->request_queue, dev->max_sectors);
@@ -839,6 +839,16 @@ static void ata_scsi_dev_config(struct scsi_device *sdev,
 		sdev->manage_start_stop = 1;
 	}
 
+	if (dev->class == ATA_DEV_ATAPI) {
+		struct request_queue *q = sdev->request_queue;
+		void *buf = kmalloc(ATAPI_MAX_DRAIN, GFP_KERNEL);
+		if (!buf) {
+			sdev_printk(KERN_ERR, sdev, "drain buffer allocation failed\n");
+			return -ENOMEM;
+		}
+		blk_queue_dma_drain(q, buf, ATAPI_MAX_DRAIN);
+	}
+
 	if (dev->flags & ATA_DFLAG_AN)
 		set_bit(SDEV_EVT_MEDIA_CHANGE, sdev->supported_events);
 
@@ -849,6 +859,8 @@ static void ata_scsi_dev_config(struct scsi_device *sdev,
 		depth = min(ATA_MAX_QUEUE - 1, depth);
 		scsi_adjust_queue_depth(sdev, MSG_SIMPLE_TAG, depth);
 	}
+
+	return 0;
 }
 
 /**
@@ -867,13 +879,14 @@ int ata_scsi_slave_config(struct scsi_device *sdev)
 {
 	struct ata_port *ap = ata_shost_to_port(sdev->host);
 	struct ata_device *dev = __ata_scsi_find_dev(ap, sdev);
+	int rc = 0;
 
 	ata_scsi_sdev_config(sdev);
 
 	if (dev)
-		ata_scsi_dev_config(sdev, dev);
+		rc = ata_scsi_dev_config(sdev, dev);
 
-	return 0;
+	return rc;
 }
 
 /**
@@ -895,6 +908,7 @@ void ata_scsi_slave_destroy(struct scsi_device *sdev)
 	struct ata_port *ap = ata_shost_to_port(sdev->host);
 	unsigned long flags;
 	struct ata_device *dev;
+	struct request_queue *q = sdev->request_queue;
 
 	if (!ap->ops->error_handler)
 		return;
@@ -908,6 +922,10 @@ void ata_scsi_slave_destroy(struct scsi_device *sdev)
 		ata_port_schedule_eh(ap);
 	}
 	spin_unlock_irqrestore(ap->lock, flags);
+
+	kfree(q->dma_drain_buffer);
+	q->dma_drain_buffer = NULL;
+	q->dma_drain_size = 0;
 }
 
 /**
Signature · 1 line
1.5.3.8
#16 lore raw
Quoted 5 lines · James Bottomley

I'm reluctant to add another parameter to the request, but this one you can calculate: you just do it wherever you work out the size of the request. If data_len is the true data length and total_data_len is the data length plus the drain length, the calculation fragment is

Quoted 8 lines

if (blk_pc_request(req))

	data_len = req->data_len;
else
	data_len = req->nr_sectors << 9;

total_data_len = data_len + req->q->dma_drain_size;

If the request has already been mapped by scsi, then data_len is actually scsi_cmnd->sdb.length

We either need to add a field or a helper and rq->data_len should probably record the size with drain buffer attached and either add raw_data_len or blk_rq_raw_data_len(). That size is the length of data in sg and should be programmed into the controller etc... For ATAPI the raw size is only used to program the chunk size for odd devices.

Quoted 7 lines
Quoted 1 line

What's needed is updating libata accordingly and testing it.

Actually, I sent the patch to do this a few days ago:

http://marc.info/?l=linux-ide&m=120189565418258

But I've attached it again.

Thanks a lot.

Quoted 8 lines
Quoted 2 lines

I'm currently away from all my toys. I'll implement the ATA part, test it and submit the patch on Monday.

Great, will look forward to the results. Like I said, I think the turn on draining in slave configure should be narrowed from all ATAPI devices to all AHCI ATAPI devices if there's no evidence that any other implementation that uses DMA to transfer PIO isn't similarly broken (I know the aic94xx works fine, for instance ...

What do you mean by aic94xx working fine? Does the controller automatically throw away extra data FISes?

Off to bed now. See you tomorrow.

Signature · 1 line
tejun
author maintainer
#17 lore raw
Quoted 12 lines · Tejun Heo
Quoted 11 lines · James Bottomley

I'm reluctant to add another parameter to the request, but this one you can calculate: you just do it wherever you work out the size of the request. If data_len is the true data length and total_data_len is the data length plus the drain length, the calculation fragment is

if (blk_pc_request(req))

	data_len = req->data_len;
else
	data_len = req->nr_sectors << 9;

total_data_len = data_len + req->q->dma_drain_size;

Quoted 8 lines
Quoted 2 lines

If the request has already been mapped by scsi, then data_len is actually scsi_cmnd->sdb.length

We either need to add a field or a helper and rq->data_len should probably record the size with drain buffer attached and either add raw_data_len or blk_rq_raw_data_len(). That size is the length of data in sg and should be programmed into the controller etc... For ATAPI the raw size is only used to program the chunk size for odd devices.

OK, could you show me an example of where you need it and I'll come up with the macro ... that should also help us decide whether it needs to be in block or in libata alone. Note that aic94xx only wants the true size (we effectively treat the drain element as non existent), and I anticipate this being true of most conforming implementations. It's only the problem HBAs that need to know how much slack they have for DMA overruns.

Quoted 13 lines
Quoted 7 lines
Quoted 1 line

What's needed is updating libata accordingly and testing it.

Actually, I sent the patch to do this a few days ago:

http://marc.info/?l=linux-ide&m=120189565418258

But I've attached it again.

Thanks a lot.

Quoted 3 lines
Quoted 2 lines

I'm currently away from all my toys. I'll implement the ATA part, test it and submit the patch on Monday.

Quoted 8 lines
Quoted 5 lines

Great, will look forward to the results. Like I said, I think the turn on draining in slave configure should be narrowed from all ATAPI devices to all AHCI ATAPI devices if there's no evidence that any other implementation that uses DMA to transfer PIO isn't similarly broken (I know the aic94xx works fine, for instance ...

What do you mean by aic94xx working fine? Does the controller automatically throw away extra data FISes?

The aic94xx sequencer has a very finely honed sense of DMA transfers. It's fully automated, and handles both ATA DMA and ATA PIO in the sequencer engine (so all the driver sees is DMA).

It reports both underrun and overrun conditions. For DMA underrun (device transfers less than expected, it just returns what it has and how much was missing as the residual) for DMA overrun (as in device tried to take more than it was programmed to send on either read or write) for PIO it does seem to zero fill or discard and then simply report task complete with overrun and let libsas sort it out. I suspect for DMA it first tries DMAT before taking other actions, but I'd need a protocol analyser (or the sequencer docs) to be sure.

We handle overruns as error conditions in both SAS and ATA at the moment, but the point is that the ATAPI device is fully happy and quiesced when we do this.

James

#18 lore raw
Quoted 3 lines · James Bottomley

The aic94xx sequencer has a very finely honed sense of DMA transfers. It's fully automated, and handles both ATA DMA and ATA PIO in the sequencer engine (so all the driver sees is DMA).

ditto AHCI, and most other DMA engines

Quoted 1 line

It reports both underrun and overrun conditions. For DMA underrun

ditto AHCI, and most other DMA engines

Quoted 7 lines

(device transfers less than expected, it just returns what it has and how much was missing as the residual) for DMA overrun (as in device tried to take more than it was programmed to send on either read or write) for PIO it does seem to zero fill or discard and then simply report task complete with overrun and let libsas sort it out. I suspect for DMA it first tries DMAT before taking other actions, but I'd need a protocol analyser (or the sequencer docs) to be sure.

Almost every other DMA engine on the planet besides aic94xx is pretty much the same... you set up an s/g tables, and it reports overrun or underrun via an interrupt + status register bit.

It sounds like aic94xx might do more work in the firmware -- that counts as "advanced", since some of the DMA engine cleanup clearly occurs in firmware, rather than pushed to kernel software.

Nowhere do I see anything about AHCI that is "broken." It has standard DMA engine behavior found in storage and non-storage hardware.

Quoted 3 lines

We handle overruns as error conditions in both SAS and ATA at the moment, but the point is that the ATAPI device is fully happy and quiesced when we do this.

That may be the result of aic94xx handling extra FIS's in the firmware, something we cannot depend on for purely silicon-based devices.

mvsas, broadsas, ahci, sata_sil24, and others behave similarly... Please don't mistake lack of firmware cleanup as "broken hardware."

Jeff

author maintainer
#19 lore raw
Quoted 25 lines · Jeff Garzik
Quoted 3 lines · James Bottomley

The aic94xx sequencer has a very finely honed sense of DMA transfers. It's fully automated, and handles both ATA DMA and ATA PIO in the sequencer engine (so all the driver sees is DMA).

ditto AHCI, and most other DMA engines

Quoted 1 line

It reports both underrun and overrun conditions. For DMA underrun

ditto AHCI, and most other DMA engines

Quoted 7 lines

(device transfers less than expected, it just returns what it has and how much was missing as the residual) for DMA overrun (as in device tried to take more than it was programmed to send on either read or write) for PIO it does seem to zero fill or discard and then simply report task complete with overrun and let libsas sort it out. I suspect for DMA it first tries DMAT before taking other actions, but I'd need a protocol analyser (or the sequencer docs) to be sure.

Almost every other DMA engine on the planet besides aic94xx is pretty much the same... you set up an s/g tables, and it reports overrun or underrun via an interrupt + status register bit.

Quoted 6 lines

It sounds like aic94xx might do more work in the firmware -- that counts as "advanced", since some of the DMA engine cleanup clearly occurs in firmware, rather than pushed to kernel software.

Nowhere do I see anything about AHCI that is "broken." It has standard DMA engine behavior found in storage and non-storage hardware.

I'm only really going by what Tejun says about AHCI. The problem as I understand it is data overrun on PIO mode commands. AHCI apparently (like aic94xx) processes these internally and doesn't actually use the libata pio handlers, so it just uses an internal buffer to receive the PIO and DMA it into memory. However, Tejun says (and he'll correct me if I'm paraphrasing wrongly, since this was on IRC a while ago) that jmb ahci and sata_sil24 both error out in different (but fairly nasty) ways if they get extra PIO data that there's no place in the SG list to deposit. This seems to be why he wants to introduce a DMA drain in addition to the existing PIO drain. I'd certainly characterise this behaviour as "broken" ... especially as not all AHCI implementations apparently have the bug ... some do exactly the right thing on PIO overruns and don't need the drain element.

Quoted 9 lines
Quoted 3 lines

We handle overruns as error conditions in both SAS and ATA at the moment, but the point is that the ATAPI device is fully happy and quiesced when we do this.

That may be the result of aic94xx handling extra FIS's in the firmware, something we cannot depend on for purely silicon-based devices.

mvsas, broadsas, ahci, sata_sil24, and others behave similarly... Please don't mistake lack of firmware cleanup as "broken hardware."

OK, well this is probably me coming from the SCSI world. Data overruns happen (especially if you allow users to control the command parameters), and just discarding the overrun has been common practice for decades.

Correct me if I'm wrong, but I think ATAPI7v3 covers the PIO overrun case in 20.4 and indicates precisely how the HBA should handle the problem. ACHI also includes the PxIS.OFS bits precisly for this case. The problem seems to come when using this mechanism hangs the device (as it apparently does in a limited number of ACHI implementations) and we just have to take the data in via DMA from the HBA (hence the need for a drain buffer).

James

#20 lore raw
Quoted 5 lines · James Bottomley

I'm only really going by what Tejun says about AHCI. The problem as I understand it is data overrun on PIO mode commands. AHCI apparently (like aic94xx) processes these internally and doesn't actually use the libata pio handlers, so it just uses an internal buffer to receive the PIO and DMA it into memory. However, Tejun says (and he'll correct me

Quoted 8 lines

if I'm paraphrasing wrongly, since this was on IRC a while ago) that jmb ahci and sata_sil24 both error out in different (but fairly nasty) ways if they get extra PIO data that there's no place in the SG list to deposit. This seems to be why he wants to introduce a DMA drain in addition to the existing PIO drain. I'd certainly characterise this behaviour as "broken" ... especially as not all AHCI implementations apparently have the bug ... some do exactly the right thing on PIO overruns and don't need the drain element.

JMB ahci triggers internal error (or was it timeout? my memory is a bit blurry now) on overflow. ICHx ahci and sata_sil24 corrupt data by offsetting the last FIS containing odd bytes by a byte if data buffer is not aligned to 4bytes under certain conditions.

[and some explanations on why the aligning and draining stuff]

Also note that ignoring overflow and/or appending draining buffer shouldn't be applied to READ/WRITE. Over/underflow should just cause HSM violation on RWs and friends. We can do this for each driver or rather each controller by enabling OFS (ahci), DRD (sil24) but the catch is that it's pretty darn difficult to verify it actually works. It not only depends on specific controller but also on which ATAPI device is attached and how it behaves depending on chunk size, transfer size in CDB and command protocol including where it splits data FISes.

For example, IIRC, the above offset-by-one condition occurs on ICHx ahci's (I've tested 7, 8 and 9), under PIO protocol, when the device determines to use a separate data FIS for the last three bytes and I don't know why it doesn't happen for DMA. It's probably because the ATAPI devices I have don't split FIS there but who knows?

So, I think we're far better off implementing a generic mechanism at some higher layer. libata core was okay. Block layer is much better. The overhead is insignificant and the aligning and draining aren't needed for hot path commands anyway.

Thanks.

Signature · 1 line
tejun
#21 lore raw
In reply to James Bottomley
Quoted 21 lines · James Bottomley
Quoted 12 lines · Tejun Heo
Quoted 11 lines · James Bottomley

I'm reluctant to add another parameter to the request, but this one you can calculate: you just do it wherever you work out the size of the request. If data_len is the true data length and total_data_len is the data length plus the drain length, the calculation fragment is

if (blk_pc_request(req))

	data_len = req->data_len;
else
	data_len = req->nr_sectors << 9;

total_data_len = data_len + req->q->dma_drain_size;

Quoted 7 lines
Quoted 2 lines

If the request has already been mapped by scsi, then data_len is actually scsi_cmnd->sdb.length

We either need to add a field or a helper and rq->data_len should probably record the size with drain buffer attached and either add raw_data_len or blk_rq_raw_data_len(). That size is the length of data in sg and should be programmed into the controller etc... For ATAPI the raw size is only used to program the chunk size for odd devices.

Quoted 7 lines

OK, could you show me an example of where you need it and I'll come up with the macro ... that should also help us decide whether it needs to be in block or in libata alone. Note that aic94xx only wants the true size (we effectively treat the drain element as non existent), and I anticipate this being true of most conforming implementations. It's only the problem HBAs that need to know how much slack they have for DMA overruns.

Some ATA controllers including SFF BMDMA and libata PIO HSM need the number of bytes mapped in the sg table. Yeah, it can be calculated with a simple macro but it also is a fundamentally confusing dual-sizing which should be made as clear as possible. Plus, it can be difficult to find out when somebody used the wrong thing, so what I'm saying is that we need to make it easy. Anyways, please lemme work on it a bit. I'll get back to you guys soon.

Thanks.

Signature · 1 line
tejun
#22 lore raw
Quoted 7 lines · Tejun Heo

Some ATA controllers including SFF BMDMA and libata PIO HSM need the number of bytes mapped in the sg table. Yeah, it can be calculated with a simple macro but it also is a fundamentally confusing dual-sizing which should be made as clear as possible. Plus, it can be difficult to find out when somebody used the wrong thing, so what I'm saying is that we need to make it easy. Anyways, please lemme work on it a bit. I'll get back to you guys soon.

Okay, here's first draft combined patch. Only compile tested (expect it to be broken) but it should be functionally equivalent to ata_sg_setup_extra() based implementation albeit with shorter drain buffer size. Several things to note...

* fsl last sg check isn't included here. Will split it out and post it separately.

* rq->raw_data_len added. Rationales...

- All these padding and draining are to prevent controllers from crapping themselves when data buffer is shorter than it likes it to be. Any controller which talks MMC (or SPC for that matter) should be ready for transfers shorter than buffer so feeding enlarged buffer size is inherently safter than feeding the length, so the primary data length field, rq->data_len, contains the adjusted length.

- raw_data_len can't be easily deduced from data_len. The other way is possible but with both aligning and draining and command filtering, calculating it later is messy.

* Draining configuration is done in sr as it's the driver for MMC. It can move both ways - either into SCSI midlayer as SPC and other commands do variable length responses too or into libata if all non-ATA controllers are happy without such workarounds. If you ask me, I'm inclined to move it into SCSI midlayer as the added overhead is insignificant (especially with drain_needed added) and it won't break anything (well, theoretically, at least).

* Padding via alinging seems a bit too hacky to me. It doesn't even cover all sg cases. I think we'll need improvements there, well, but for the time being, this should do.

I'll test and report in a few hours.

Thanks.

block/blk-core.c +2 -0
Index: work/block/blk-core.c
===================================================================
@@ -116,6 +116,7 @@ void rq_init(struct request_queue *q, st
 	rq->ref_count = 1;
 	rq->q = q;
 	rq->special = NULL;
+	rq->raw_data_len = 0;
 	rq->data_len = 0;
 	rq->data = NULL;
 	rq->nr_phys_segments = 0;
@@ -1982,6 +1983,7 @@ void blk_rq_bio_prep(struct request_queu
 	rq->hard_cur_sectors = rq->current_nr_sectors;
 	rq->hard_nr_sectors = rq->nr_sectors = bio_sectors(bio);
 	rq->buffer = bio_data(bio);
+	rq->raw_data_len = bio->bi_size;
 	rq->data_len = bio->bi_size;
 
 	rq->bio = rq->biotail = bio;
block/blk-map.c +4 -1
Index: work/block/blk-map.c
===================================================================
@@ -19,6 +19,7 @@ int blk_rq_append_bio(struct request_que
 		rq->biotail->bi_next = bio;
 		rq->biotail = bio;
 
+		rq->raw_data_len += bio->bi_size;
 		rq->data_len += bio->bi_size;
 	}
 	return 0;
@@ -56,8 +57,10 @@ static int __blk_rq_map_user(struct requ
 	if (!(uaddr & queue_dma_alignment(q)) &&
 	    !(len & queue_dma_alignment(q)))
 		bio = bio_map_user(q, NULL, uaddr, len, reading);
-	else
+	else {
 		bio = bio_copy_user(q, uaddr, len, reading);
+		rq->data_len = roundup(len, queue_dma_alignment(q));
+	}
 
 	if (IS_ERR(bio))
 		return PTR_ERR(bio);
include/linux/blkdev.h +6 -2
Index: work/include/linux/blkdev.h
===================================================================
@@ -214,6 +214,7 @@ struct request {
 	unsigned int cmd_len;
 	unsigned char cmd[BLK_MAX_CDB];
 
+	unsigned int raw_data_len;
 	unsigned int data_len;
 	unsigned int sense_len;
 	void *data;
@@ -256,6 +257,7 @@ struct bio_vec;
 typedef int (merge_bvec_fn) (struct request_queue *, struct bio *, struct bio_vec *);
 typedef void (prepare_flush_fn) (struct request_queue *, struct request *);
 typedef void (softirq_done_fn)(struct request *);
+typedef int (dma_drain_needed_fn)(struct request *);
 
 enum blk_queue_state {
 	Queue_down,
@@ -292,6 +294,7 @@ struct request_queue
 	merge_bvec_fn		*merge_bvec_fn;
 	prepare_flush_fn	*prepare_flush_fn;
 	softirq_done_fn		*softirq_done_fn;
+	dma_drain_needed_fn	*dma_drain_needed;
 
 	/*
 	 * Dispatch queue sorting
@@ -696,8 +699,9 @@ extern void blk_queue_max_hw_segments(st
 extern void blk_queue_max_segment_size(struct request_queue *, unsigned int);
 extern void blk_queue_hardsect_size(struct request_queue *, unsigned short);
 extern void blk_queue_stack_limits(struct request_queue *t, struct request_queue *b);
-extern int blk_queue_dma_drain(struct request_queue *q, void *buf,
-			       unsigned int size);
+extern int blk_queue_dma_drain(struct request_queue *q,
+			       dma_drain_needed_fn *dma_drain_needed,
+			       void *buf, unsigned int size);
 extern void blk_queue_segment_boundary(struct request_queue *, unsigned long);
 extern void blk_queue_prep_rq(struct request_queue *, prep_rq_fn *pfn);
 extern void blk_queue_merge_bvec(struct request_queue *, merge_bvec_fn *);
block/blk-merge.c +2 -1
Index: work/block/blk-merge.c
===================================================================
@@ -220,7 +220,7 @@ new_segment:
 		bvprv = bvec;
 	} /* segments in rq */
 
-	if (q->dma_drain_size) {
+	if (q->dma_drain_size && q->dma_drain_needed(rq)) {
 		sg->page_link &= ~0x02;
 		sg = sg_next(sg);
 		sg_set_page(sg, virt_to_page(q->dma_drain_buffer),
@@ -228,6 +228,7 @@ new_segment:
 			    ((unsigned long)q->dma_drain_buffer) &
 			    (PAGE_SIZE - 1));
 		nsegs++;
+		rq->data_len += q->dma_drain_size;
 	}
 
 	if (sg)
block/bsg.c +4 -4
Index: work/block/bsg.c
===================================================================
@@ -437,14 +437,14 @@ static int blk_complete_sgv4_hdr_rq(stru
 	}
 
 	if (rq->next_rq) {
-		hdr->dout_resid = rq->data_len;
-		hdr->din_resid = rq->next_rq->data_len;
+		hdr->dout_resid = rq->raw_data_len;
+		hdr->din_resid = rq->next_rq->raw_data_len;
 		blk_rq_unmap_user(bidi_bio);
 		blk_put_request(rq->next_rq);
 	} else if (rq_data_dir(rq) == READ)
-		hdr->din_resid = rq->data_len;
+		hdr->din_resid = rq->raw_data_len;
 	else
-		hdr->dout_resid = rq->data_len;
+		hdr->dout_resid = rq->raw_data_len;
 
 	/*
 	 * If the request generated a negative error number, return it
block/scsi_ioctl.c +2 -1
Index: work/block/scsi_ioctl.c
===================================================================
@@ -266,7 +266,7 @@ static int blk_complete_sghdr_rq(struct 
 	hdr->info = 0;
 	if (hdr->masked_status || hdr->host_status || hdr->driver_status)
 		hdr->info |= SG_INFO_CHECK;
-	hdr->resid = rq->data_len;
+	hdr->resid = rq->raw_data_len;
 	hdr->sb_len_wr = 0;
 
 	if (rq->sense_len && hdr->sbp) {
@@ -528,6 +528,7 @@ static int __blk_send_generic(struct req
 	rq = blk_get_request(q, WRITE, __GFP_WAIT);
 	rq->cmd_type = REQ_TYPE_BLOCK_PC;
 	rq->data = NULL;
+	rq->raw_data_len = 0;
 	rq->data_len = 0;
 	rq->timeout = BLK_DEFAULT_SG_TIMEOUT;
 	memset(rq->cmd, 0, sizeof(rq->cmd));
block/blk-settings.c +5 -2
Index: work/block/blk-settings.c
===================================================================
@@ -296,6 +296,7 @@ EXPORT_SYMBOL(blk_queue_stack_limits);
  * blk_queue_dma_drain - Set up a drain buffer for excess dma.
  *
  * @q:  the request queue for the device
+ * @drain_needed: fn which returns non-zero if drain is needed for the request
  * @buf:	physically contiguous buffer
  * @size:	size of the buffer in bytes
  *
@@ -315,14 +316,16 @@ EXPORT_SYMBOL(blk_queue_stack_limits);
  * device can support otherwise there won't be room for the drain
  * buffer.
  */
-int blk_queue_dma_drain(struct request_queue *q, void *buf,
-				unsigned int size)
+extern int blk_queue_dma_drain(struct request_queue *q,
+			       dma_drain_needed_fn *dma_drain_needed,
+			       void *buf, unsigned int size)
 {
 	if (q->max_hw_segments < 2 || q->max_phys_segments < 2)
 		return -EINVAL;
 	/* make room for appending the drain */
 	--q->max_hw_segments;
 	--q->max_phys_segments;
+	q->dma_drain_needed = dma_drain_needed;
 	q->dma_drain_buffer = buf;
 	q->dma_drain_size = size;
 
drivers/scsi/sr.c +36 -6

86 lines not on this page · raw message

drivers/scsi/sr.h +1 -0
Index: work/drivers/scsi/sr.h
===================================================================
@@ -22,6 +22,7 @@
 
 #define MAX_RETRIES	3
 #define SR_TIMEOUT	(30 * HZ)
+#define SR_DRAIN_SIZE	PAGE_SIZE
 
 struct scsi_device;
 
drivers/ata/libata-core.c +7 -133

179 lines not on this page · raw message

drivers/ata/libata-scsi.c +4 -12
Index: work/drivers/ata/libata-scsi.c
===================================================================
@@ -832,18 +832,11 @@ static void ata_scsi_dev_config(struct s
 	/* configure max sectors */
 	blk_queue_max_sectors(sdev->request_queue, dev->max_sectors);
 
-	/* SATA DMA transfers must be multiples of 4 byte, so
-	 * we need to pad ATAPI transfers using an extra sg.
-	 * Decrement max hw segments accordingly.
-	 */
-	if (dev->class == ATA_DEV_ATAPI) {
-		struct request_queue *q = sdev->request_queue;
-		blk_queue_max_hw_segments(q, q->max_hw_segments - 1);
-
+	if (dev->class == ATA_DEV_ATAPI)
 		/* set the min alignment */
 		blk_queue_update_dma_alignment(sdev->request_queue,
 					       ATA_DMA_PAD_SZ - 1);
-	} else
+	else
 		/* ATA devices must be sector aligned */
 		blk_queue_update_dma_alignment(sdev->request_queue,
 					       ATA_SECT_SIZE - 1);
@@ -2500,7 +2493,7 @@ static unsigned int atapi_xlat(struct at
 	 * want to set it properly, and for DMA where it is
 	 * effectively meaningless.
 	 */
-	nbytes = min(qc->nbytes, (unsigned int)63 * 1024);
+	nbytes = min(scmd->request->raw_data_len, (unsigned int)63 * 1024);
 
 	/* Most ATAPI devices which honor transfer chunk size don't
 	 * behave according to the spec when odd chunk size which
@@ -3564,7 +3557,7 @@ EXPORT_SYMBOL_GPL(ata_sas_port_alloc);
  */
 int ata_sas_port_start(struct ata_port *ap)
 {
-	return ata_pad_alloc(ap, ap->dev);
+	return 0;
 }
 EXPORT_SYMBOL_GPL(ata_sas_port_start);
 
@@ -3582,7 +3575,6 @@ EXPORT_SYMBOL_GPL(ata_sas_port_start);
 
 void ata_sas_port_stop(struct ata_port *ap)
 {
-	ata_pad_free(ap, ap->dev);
 }
 EXPORT_SYMBOL_GPL(ata_sas_port_stop);
 
drivers/ata/sata_fsl.c +0 -13
Index: work/drivers/ata/sata_fsl.c
===================================================================
@@ -601,21 +601,9 @@ static int sata_fsl_port_start(struct at
 	if (!pp)
 		return -ENOMEM;
 
-	/*
-	 * allocate per command dma alignment pad buffer, which is used
-	 * internally by libATA to ensure that all transfers ending on
-	 * unaligned boundaries are padded, to align on Dword boundaries
-	 */
-	retval = ata_pad_alloc(ap, dev);
-	if (retval) {
-		kfree(pp);
-		return retval;
-	}
-
 	mem = dma_alloc_coherent(dev, SATA_FSL_PORT_PRIV_DMA_SZ, &mem_dma,
 				 GFP_KERNEL);
 	if (!mem) {
-		ata_pad_free(ap, dev);
 		kfree(pp);
 		return -ENOMEM;
 	}
@@ -694,7 +682,6 @@ static void sata_fsl_port_stop(struct at
 	dma_free_coherent(dev, SATA_FSL_PORT_PRIV_DMA_SZ,
 			  pp->cmdslot, pp->cmdslot_paddr);
 
-	ata_pad_free(ap, dev);
 	kfree(pp);
 }
 
drivers/ata/sata_mv.c +1 -5
Index: work/drivers/ata/sata_mv.c
===================================================================
@@ -1157,17 +1157,13 @@ static int mv_port_start(struct ata_port
 	struct mv_port_priv *pp;
 	void __iomem *port_mmio = mv_ap_base(ap);
 	unsigned long flags;
-	int tag, rc;
+	int tag;
 
 	pp = devm_kzalloc(dev, sizeof(*pp), GFP_KERNEL);
 	if (!pp)
 		return -ENOMEM;
 	ap->private_data = pp;
 
-	rc = ata_pad_alloc(ap, dev);
-	if (rc)
-		return rc;
-
 	pp->crqb = dma_pool_alloc(hpriv->crqb_pool, GFP_KERNEL, &pp->crqb_dma);
 	if (!pp->crqb)
 		return -ENOMEM;
drivers/ata/sata_sil24.c +0 -5
Index: work/drivers/ata/sata_sil24.c
===================================================================
@@ -1234,7 +1234,6 @@ static int sil24_port_start(struct ata_p
 	union sil24_cmd_block *cb;
 	size_t cb_size = sizeof(*cb) * SIL24_MAX_CMDS;
 	dma_addr_t cb_dma;
-	int rc;
 
 	pp = devm_kzalloc(dev, sizeof(*pp), GFP_KERNEL);
 	if (!pp)
@@ -1247,10 +1246,6 @@ static int sil24_port_start(struct ata_p
 		return -ENOMEM;
 	memset(cb, 0, cb_size);
 
-	rc = ata_pad_alloc(ap, dev);
-	if (rc)
-		return rc;
-
 	pp->cmd_block = cb;
 	pp->cmd_block_dma = cb_dma;
 
include/linux/libata.h +0 -23
Index: work/include/linux/libata.h
===================================================================
@@ -278,7 +278,6 @@ enum {
 
 	/* size of buffer to pad xfers ending on unaligned boundaries */
 	ATA_DMA_PAD_SZ		= 4,
-	ATA_DMA_PAD_BUF_SZ	= ATA_DMA_PAD_SZ * ATA_MAX_QUEUE,
 
 	/* ering size */
 	ATA_ERING_SIZE		= 32,
@@ -458,11 +457,9 @@ struct ata_queued_cmd {
 	unsigned int		tag;
 	unsigned int		n_elem;
 	unsigned int		n_iter;
-	unsigned int		mapped_n_elem;
 
 	int			dma_dir;
 
-	unsigned int		pad_len;
 	unsigned int		sect_size;
 
 	unsigned int		nbytes;
@@ -473,9 +470,7 @@ struct ata_queued_cmd {
 	unsigned int		cursg_ofs;
 
 	struct scatterlist	*last_sg;
-	struct scatterlist	saved_last_sg;
 	struct scatterlist	sgent;
-	struct scatterlist	extra_sg[2];
 
 	struct scatterlist	*sg;
 
@@ -620,9 +615,6 @@ struct ata_port {
 	struct ata_prd		*prd;	 /* our SG list */
 	dma_addr_t		prd_dma; /* and its DMA mapping */
 
-	void			*pad;	/* array of DMA pad buffers */
-	dma_addr_t		pad_dma;
-
 	struct ata_ioports	ioaddr;	/* ATA cmd/ctl/dma register blocks */
 
 	u8			ctl;	/* cache of ATA control register */
@@ -1366,10 +1358,8 @@ static inline void ata_qc_reinit(struct 
 	qc->cursg_ofs = 0;
 	qc->nbytes = qc->raw_nbytes = qc->curbytes = 0;
 	qc->n_elem = 0;
-	qc->mapped_n_elem = 0;
 	qc->n_iter = 0;
 	qc->err_mask = 0;
-	qc->pad_len = 0;
 	qc->last_sg = NULL;
 	qc->sect_size = ATA_SECT_SIZE;
 
@@ -1425,19 +1415,6 @@ static inline unsigned int __ac_err_mask
 	return mask;
 }
 
-static inline int ata_pad_alloc(struct ata_port *ap, struct device *dev)
-{
-	ap->pad_dma = 0;
-	ap->pad = dmam_alloc_coherent(dev, ATA_DMA_PAD_BUF_SZ,
-				      &ap->pad_dma, GFP_KERNEL);
-	return (ap->pad == NULL) ? -ENOMEM : 0;
-}
-
-static inline void ata_pad_free(struct ata_port *ap, struct device *dev)
-{
-	dmam_free_coherent(dev, ATA_DMA_PAD_BUF_SZ, ap->pad, ap->pad_dma);
-}
-
 static inline struct ata_port *ata_shost_to_port(struct Scsi_Host *host)
 {
 	return *(struct ata_port **)&host->hostdata[0];
drivers/scsi/libsas/sas_ata.c +2 -2
Index: work/drivers/scsi/libsas/sas_ata.c
===================================================================
@@ -178,8 +178,8 @@ static unsigned int sas_ata_qc_issue(str
 	task->uldd_task = qc;
 	if (ata_is_atapi(qc->tf.protocol)) {
 		memcpy(task->ata_task.atapi_packet, qc->cdb, qc->dev->cdb_len);
-		task->total_xfer_len = qc->nbytes + qc->pad_len;
-		task->num_scatter = qc->pad_len ? qc->n_elem + 1 : qc->n_elem;
+		task->total_xfer_len = qc->nbytes;
+		task->num_scatter = qc->n_elem;
 	} else {
 		for_each_sg(qc->sg, sg, qc->n_elem, si)
 			xfer += sg->length;
drivers/scsi/ipr.c +2 -2
Index: work/drivers/scsi/ipr.c
===================================================================
@@ -5140,7 +5140,7 @@ static void ipr_build_ata_ioadl(struct i
 	struct ipr_ioarcb *ioarcb = &ipr_cmd->ioarcb;
 	struct ipr_ioadl_desc *ioadl = ipr_cmd->ioadl;
 	struct ipr_ioadl_desc *last_ioadl = NULL;
-	int len = qc->nbytes + qc->pad_len;
+	int len = qc->nbytes;
 	struct scatterlist *sg;
 	unsigned int si;
 
@@ -5206,7 +5206,7 @@ static unsigned int ipr_qc_issue(struct 
 	ioarcb->cmd_pkt.request_type = IPR_RQTYPE_ATA_PASSTHRU;
 	ioarcb->cmd_pkt.flags_hi |= IPR_FLAGS_HI_NO_LINK_DESC;
 	ioarcb->cmd_pkt.flags_hi |= IPR_FLAGS_HI_NO_ULEN_CHK;
-	ipr_cmd->dma_use_sg = qc->pad_len ? qc->n_elem + 1 : qc->n_elem;
+	ipr_cmd->dma_use_sg = qc->n_elem;
 
 	ipr_build_ata_ioadl(ipr_cmd, qc);
 	regs->flags |= IPR_ATA_FLAG_STATUS_ON_GOOD_COMPLETION;
drivers/ata/ahci.c +0 -5
Index: work/drivers/ata/ahci.c
===================================================================
@@ -1979,16 +1979,11 @@ static int ahci_port_start(struct ata_po
 	struct ahci_port_priv *pp;
 	void *mem;
 	dma_addr_t mem_dma;
-	int rc;
 
 	pp = devm_kzalloc(dev, sizeof(*pp), GFP_KERNEL);
 	if (!pp)
 		return -ENOMEM;
 
-	rc = ata_pad_alloc(ap, dev);
-	if (rc)
-		return rc;
-
 	mem = dmam_alloc_coherent(dev, AHCI_PORT_PRIV_DMA_SZ, &mem_dma,
 				  GFP_KERNEL);
 	if (!mem)
drivers/ata/pata_icside.c +0 -8
Index: work/drivers/ata/pata_icside.c
===================================================================
@@ -304,12 +304,6 @@ static int icside_dma_init(struct pata_i
 }
 
 
-static int pata_icside_port_start(struct ata_port *ap)
-{
-	/* No PRD to alloc */
-	return ata_pad_alloc(ap, ap->dev);
-}
-
 static struct scsi_host_template pata_icside_sht = {
 	.module			= THIS_MODULE,
 	.name			= DRV_NAME,
@@ -389,8 +383,6 @@ static struct ata_port_operations pata_i
 	.irq_clear		= ata_dummy_noret,
 	.irq_on			= ata_irq_on,
 
-	.port_start		= pata_icside_port_start,
-
 	.bmdma_stop		= pata_icside_bmdma_stop,
 	.bmdma_status		= pata_icside_bmdma_status,
 };
#23 lore raw

And, here's working version. I'll splite and post them tomorrow.

Thanks.

block/blk-core.c +2 -0
Index: work/block/blk-core.c
===================================================================
@@ -116,6 +116,7 @@ void rq_init(struct request_queue *q, st
 	rq->ref_count = 1;
 	rq->q = q;
 	rq->special = NULL;
+	rq->raw_data_len = 0;
 	rq->data_len = 0;
 	rq->data = NULL;
 	rq->nr_phys_segments = 0;
@@ -1982,6 +1983,7 @@ void blk_rq_bio_prep(struct request_queu
 	rq->hard_cur_sectors = rq->current_nr_sectors;
 	rq->hard_nr_sectors = rq->nr_sectors = bio_sectors(bio);
 	rq->buffer = bio_data(bio);
+	rq->raw_data_len = bio->bi_size;
 	rq->data_len = bio->bi_size;
 
 	rq->bio = rq->biotail = bio;
block/blk-map.c +20 -0
Index: work/block/blk-map.c
===================================================================
@@ -19,6 +19,7 @@ int blk_rq_append_bio(struct request_que
 		rq->biotail->bi_next = bio;
 		rq->biotail = bio;
 
+		rq->raw_data_len += bio->bi_size;
 		rq->data_len += bio->bi_size;
 	}
 	return 0;
@@ -139,6 +140,25 @@ int blk_rq_map_user(struct request_queue
 		ubuf += ret;
 	}
 
+	/*
+	 * __blk_rq_map_user() copies the buffers if starting address
+	 * or length aren't aligned.  As the copied buffer is always
+	 * page aligned, we know for a fact that there's enough room
+	 * for padding.  Extend the last bio and update rq->data_len
+	 * accordingly.
+	 *
+	 * On unmap, bio_uncopy_user() will use unmodified
+	 * bio_map_data pointed to by bio->bi_private.
+	 */
+	if (len & queue_dma_alignment(q)) {
+		unsigned int pad_len = (queue_dma_alignment(q) & ~len) + 1;
+		struct bio *bio = rq->biotail;
+
+		bio->bi_io_vec[bio->bi_vcnt - 1].bv_len += pad_len;
+		bio->bi_size += pad_len;
+		rq->data_len += pad_len;
+	}
+
 	rq->buffer = rq->data = NULL;
 	return 0;
 unmap_rq:
include/linux/blkdev.h +6 -2
Index: work/include/linux/blkdev.h
===================================================================
@@ -214,6 +214,7 @@ struct request {
 	unsigned int cmd_len;
 	unsigned char cmd[BLK_MAX_CDB];
 
+	unsigned int raw_data_len;
 	unsigned int data_len;
 	unsigned int sense_len;
 	void *data;
@@ -256,6 +257,7 @@ struct bio_vec;
 typedef int (merge_bvec_fn) (struct request_queue *, struct bio *, struct bio_vec *);
 typedef void (prepare_flush_fn) (struct request_queue *, struct request *);
 typedef void (softirq_done_fn)(struct request *);
+typedef int (dma_drain_needed_fn)(struct request *);
 
 enum blk_queue_state {
 	Queue_down,
@@ -292,6 +294,7 @@ struct request_queue
 	merge_bvec_fn		*merge_bvec_fn;
 	prepare_flush_fn	*prepare_flush_fn;
 	softirq_done_fn		*softirq_done_fn;
+	dma_drain_needed_fn	*dma_drain_needed;
 
 	/*
 	 * Dispatch queue sorting
@@ -696,8 +699,9 @@ extern void blk_queue_max_hw_segments(st
 extern void blk_queue_max_segment_size(struct request_queue *, unsigned int);
 extern void blk_queue_hardsect_size(struct request_queue *, unsigned short);
 extern void blk_queue_stack_limits(struct request_queue *t, struct request_queue *b);
-extern int blk_queue_dma_drain(struct request_queue *q, void *buf,
-			       unsigned int size);
+extern int blk_queue_dma_drain(struct request_queue *q,
+			       dma_drain_needed_fn *dma_drain_needed,
+			       void *buf, unsigned int size);
 extern void blk_queue_segment_boundary(struct request_queue *, unsigned long);
 extern void blk_queue_prep_rq(struct request_queue *, prep_rq_fn *pfn);
 extern void blk_queue_merge_bvec(struct request_queue *, merge_bvec_fn *);
block/blk-merge.c +2 -1
Index: work/block/blk-merge.c
===================================================================
@@ -220,7 +220,7 @@ new_segment:
 		bvprv = bvec;
 	} /* segments in rq */
 
-	if (q->dma_drain_size) {
+	if (q->dma_drain_size && q->dma_drain_needed(rq)) {
 		sg->page_link &= ~0x02;
 		sg = sg_next(sg);
 		sg_set_page(sg, virt_to_page(q->dma_drain_buffer),
@@ -228,6 +228,7 @@ new_segment:
 			    ((unsigned long)q->dma_drain_buffer) &
 			    (PAGE_SIZE - 1));
 		nsegs++;
+		rq->data_len += q->dma_drain_size;
 	}
 
 	if (sg)
block/bsg.c +4 -4
Index: work/block/bsg.c
===================================================================
@@ -437,14 +437,14 @@ static int blk_complete_sgv4_hdr_rq(stru
 	}
 
 	if (rq->next_rq) {
-		hdr->dout_resid = rq->data_len;
-		hdr->din_resid = rq->next_rq->data_len;
+		hdr->dout_resid = rq->raw_data_len;
+		hdr->din_resid = rq->next_rq->raw_data_len;
 		blk_rq_unmap_user(bidi_bio);
 		blk_put_request(rq->next_rq);
 	} else if (rq_data_dir(rq) == READ)
-		hdr->din_resid = rq->data_len;
+		hdr->din_resid = rq->raw_data_len;
 	else
-		hdr->dout_resid = rq->data_len;
+		hdr->dout_resid = rq->raw_data_len;
 
 	/*
 	 * If the request generated a negative error number, return it
block/scsi_ioctl.c +2 -1
Index: work/block/scsi_ioctl.c
===================================================================
@@ -266,7 +266,7 @@ static int blk_complete_sghdr_rq(struct 
 	hdr->info = 0;
 	if (hdr->masked_status || hdr->host_status || hdr->driver_status)
 		hdr->info |= SG_INFO_CHECK;
-	hdr->resid = rq->data_len;
+	hdr->resid = rq->raw_data_len;
 	hdr->sb_len_wr = 0;
 
 	if (rq->sense_len && hdr->sbp) {
@@ -528,6 +528,7 @@ static int __blk_send_generic(struct req
 	rq = blk_get_request(q, WRITE, __GFP_WAIT);
 	rq->cmd_type = REQ_TYPE_BLOCK_PC;
 	rq->data = NULL;
+	rq->raw_data_len = 0;
 	rq->data_len = 0;
 	rq->timeout = BLK_DEFAULT_SG_TIMEOUT;
 	memset(rq->cmd, 0, sizeof(rq->cmd));
drivers/scsi/scsi_lib.c +4 -4
Index: work/drivers/scsi/scsi_lib.c
===================================================================
@@ -1015,10 +1015,6 @@ static int scsi_init_sgtable(struct requ
 	}
 
 	req->buffer = NULL;
-	if (blk_pc_request(req))
-		sdb->length = req->data_len;
-	else
-		sdb->length = req->nr_sectors << 9;
 
 	/* 
 	 * Next, walk the list, and fill in the addresses and sizes of
@@ -1027,6 +1023,10 @@ static int scsi_init_sgtable(struct requ
 	count = blk_rq_map_sg(req->q, req, sdb->table.sgl);
 	BUG_ON(count > sdb->table.nents);
 	sdb->table.nents = count;
+	if (blk_pc_request(req))
+		sdb->length = req->data_len;
+	else
+		sdb->length = req->nr_sectors << 9;
 	return BLKPREP_OK;
 }
 
block/blk-settings.c +5 -2
Index: work/block/blk-settings.c
===================================================================
@@ -296,6 +296,7 @@ EXPORT_SYMBOL(blk_queue_stack_limits);
  * blk_queue_dma_drain - Set up a drain buffer for excess dma.
  *
  * @q:  the request queue for the device
+ * @dma_drain_needed: fn which returns non-zero if drain is necessary
  * @buf:	physically contiguous buffer
  * @size:	size of the buffer in bytes
  *
@@ -315,14 +316,16 @@ EXPORT_SYMBOL(blk_queue_stack_limits);
  * device can support otherwise there won't be room for the drain
  * buffer.
  */
-int blk_queue_dma_drain(struct request_queue *q, void *buf,
-				unsigned int size)
+extern int blk_queue_dma_drain(struct request_queue *q,
+			       dma_drain_needed_fn *dma_drain_needed,
+			       void *buf, unsigned int size)
 {
 	if (q->max_hw_segments < 2 || q->max_phys_segments < 2)
 		return -EINVAL;
 	/* make room for appending the drain */
 	--q->max_hw_segments;
 	--q->max_phys_segments;
+	q->dma_drain_needed = dma_drain_needed;
 	q->dma_drain_buffer = buf;
 	q->dma_drain_size = size;
 
drivers/scsi/sr.c +36 -6

86 lines not on this page · raw message

drivers/scsi/sr.h +1 -0
Index: work/drivers/scsi/sr.h
===================================================================
@@ -22,6 +22,7 @@
 
 #define MAX_RETRIES	3
 #define SR_TIMEOUT	(30 * HZ)
+#define SR_DRAIN_SIZE	PAGE_SIZE
 
 struct scsi_device;
 
drivers/ata/libata-core.c +18 -135

199 lines not on this page · raw message

drivers/ata/libata-scsi.c +6 -12
Index: work/drivers/ata/libata-scsi.c
===================================================================
@@ -832,18 +832,11 @@ static void ata_scsi_dev_config(struct s
 	/* configure max sectors */
 	blk_queue_max_sectors(sdev->request_queue, dev->max_sectors);
 
-	/* SATA DMA transfers must be multiples of 4 byte, so
-	 * we need to pad ATAPI transfers using an extra sg.
-	 * Decrement max hw segments accordingly.
-	 */
-	if (dev->class == ATA_DEV_ATAPI) {
-		struct request_queue *q = sdev->request_queue;
-		blk_queue_max_hw_segments(q, q->max_hw_segments - 1);
-
+	if (dev->class == ATA_DEV_ATAPI)
 		/* set the min alignment */
 		blk_queue_update_dma_alignment(sdev->request_queue,
 					       ATA_DMA_PAD_SZ - 1);
-	} else
+	else
 		/* ATA devices must be sector aligned */
 		blk_queue_update_dma_alignment(sdev->request_queue,
 					       ATA_SECT_SIZE - 1);
@@ -2500,7 +2493,9 @@ static unsigned int atapi_xlat(struct at
 	 * want to set it properly, and for DMA where it is
 	 * effectively meaningless.
 	 */
-	nbytes = min(qc->nbytes, (unsigned int)63 * 1024);
+	nbytes = min(scmd->request->raw_data_len, (unsigned int)63 * 1024);
+	ata_dev_printk(qc->dev, KERN_INFO, "XXX raw_data_len=%u\n",
+		       scmd->request->raw_data_len);
 
 	/* Most ATAPI devices which honor transfer chunk size don't
 	 * behave according to the spec when odd chunk size which
@@ -3564,7 +3559,7 @@ EXPORT_SYMBOL_GPL(ata_sas_port_alloc);
  */
 int ata_sas_port_start(struct ata_port *ap)
 {
-	return ata_pad_alloc(ap, ap->dev);
+	return 0;
 }
 EXPORT_SYMBOL_GPL(ata_sas_port_start);
 
@@ -3582,7 +3577,6 @@ EXPORT_SYMBOL_GPL(ata_sas_port_start);
 
 void ata_sas_port_stop(struct ata_port *ap)
 {
-	ata_pad_free(ap, ap->dev);
 }
 EXPORT_SYMBOL_GPL(ata_sas_port_stop);
 
drivers/ata/sata_fsl.c +0 -13
Index: work/drivers/ata/sata_fsl.c
===================================================================
@@ -601,21 +601,9 @@ static int sata_fsl_port_start(struct at
 	if (!pp)
 		return -ENOMEM;
 
-	/*
-	 * allocate per command dma alignment pad buffer, which is used
-	 * internally by libATA to ensure that all transfers ending on
-	 * unaligned boundaries are padded, to align on Dword boundaries
-	 */
-	retval = ata_pad_alloc(ap, dev);
-	if (retval) {
-		kfree(pp);
-		return retval;
-	}
-
 	mem = dma_alloc_coherent(dev, SATA_FSL_PORT_PRIV_DMA_SZ, &mem_dma,
 				 GFP_KERNEL);
 	if (!mem) {
-		ata_pad_free(ap, dev);
 		kfree(pp);
 		return -ENOMEM;
 	}
@@ -694,7 +682,6 @@ static void sata_fsl_port_stop(struct at
 	dma_free_coherent(dev, SATA_FSL_PORT_PRIV_DMA_SZ,
 			  pp->cmdslot, pp->cmdslot_paddr);
 
-	ata_pad_free(ap, dev);
 	kfree(pp);
 }
 
drivers/ata/sata_mv.c +1 -5
Index: work/drivers/ata/sata_mv.c
===================================================================
@@ -1157,17 +1157,13 @@ static int mv_port_start(struct ata_port
 	struct mv_port_priv *pp;
 	void __iomem *port_mmio = mv_ap_base(ap);
 	unsigned long flags;
-	int tag, rc;
+	int tag;
 
 	pp = devm_kzalloc(dev, sizeof(*pp), GFP_KERNEL);
 	if (!pp)
 		return -ENOMEM;
 	ap->private_data = pp;
 
-	rc = ata_pad_alloc(ap, dev);
-	if (rc)
-		return rc;
-
 	pp->crqb = dma_pool_alloc(hpriv->crqb_pool, GFP_KERNEL, &pp->crqb_dma);
 	if (!pp->crqb)
 		return -ENOMEM;
drivers/ata/sata_sil24.c +0 -5
Index: work/drivers/ata/sata_sil24.c
===================================================================
@@ -1234,7 +1234,6 @@ static int sil24_port_start(struct ata_p
 	union sil24_cmd_block *cb;
 	size_t cb_size = sizeof(*cb) * SIL24_MAX_CMDS;
 	dma_addr_t cb_dma;
-	int rc;
 
 	pp = devm_kzalloc(dev, sizeof(*pp), GFP_KERNEL);
 	if (!pp)
@@ -1247,10 +1246,6 @@ static int sil24_port_start(struct ata_p
 		return -ENOMEM;
 	memset(cb, 0, cb_size);
 
-	rc = ata_pad_alloc(ap, dev);
-	if (rc)
-		return rc;
-
 	pp->cmd_block = cb;
 	pp->cmd_block_dma = cb_dma;
 
include/linux/libata.h +1 -25
Index: work/include/linux/libata.h
===================================================================
@@ -278,7 +278,6 @@ enum {
 
 	/* size of buffer to pad xfers ending on unaligned boundaries */
 	ATA_DMA_PAD_SZ		= 4,
-	ATA_DMA_PAD_BUF_SZ	= ATA_DMA_PAD_SZ * ATA_MAX_QUEUE,
 
 	/* ering size */
 	ATA_ERING_SIZE		= 32,
@@ -458,24 +457,19 @@ struct ata_queued_cmd {
 	unsigned int		tag;
 	unsigned int		n_elem;
 	unsigned int		n_iter;
-	unsigned int		mapped_n_elem;
 
 	int			dma_dir;
 
-	unsigned int		pad_len;
 	unsigned int		sect_size;
 
 	unsigned int		nbytes;
-	unsigned int		raw_nbytes;
 	unsigned int		curbytes;
 
 	struct scatterlist	*cursg;
 	unsigned int		cursg_ofs;
 
 	struct scatterlist	*last_sg;
-	struct scatterlist	saved_last_sg;
 	struct scatterlist	sgent;
-	struct scatterlist	extra_sg[2];
 
 	struct scatterlist	*sg;
 
@@ -620,9 +614,6 @@ struct ata_port {
 	struct ata_prd		*prd;	 /* our SG list */
 	dma_addr_t		prd_dma; /* and its DMA mapping */
 
-	void			*pad;	/* array of DMA pad buffers */
-	dma_addr_t		pad_dma;
-
 	struct ata_ioports	ioaddr;	/* ATA cmd/ctl/dma register blocks */
 
 	u8			ctl;	/* cache of ATA control register */
@@ -1364,12 +1355,10 @@ static inline void ata_qc_reinit(struct 
 	qc->flags = 0;
 	qc->cursg = NULL;
 	qc->cursg_ofs = 0;
-	qc->nbytes = qc->raw_nbytes = qc->curbytes = 0;
+	qc->nbytes = qc->curbytes = 0;
 	qc->n_elem = 0;
-	qc->mapped_n_elem = 0;
 	qc->n_iter = 0;
 	qc->err_mask = 0;
-	qc->pad_len = 0;
 	qc->last_sg = NULL;
 	qc->sect_size = ATA_SECT_SIZE;
 
@@ -1425,19 +1414,6 @@ static inline unsigned int __ac_err_mask
 	return mask;
 }
 
-static inline int ata_pad_alloc(struct ata_port *ap, struct device *dev)
-{
-	ap->pad_dma = 0;
-	ap->pad = dmam_alloc_coherent(dev, ATA_DMA_PAD_BUF_SZ,
-				      &ap->pad_dma, GFP_KERNEL);
-	return (ap->pad == NULL) ? -ENOMEM : 0;
-}
-
-static inline void ata_pad_free(struct ata_port *ap, struct device *dev)
-{
-	dmam_free_coherent(dev, ATA_DMA_PAD_BUF_SZ, ap->pad, ap->pad_dma);
-}
-
 static inline struct ata_port *ata_shost_to_port(struct Scsi_Host *host)
 {
 	return *(struct ata_port **)&host->hostdata[0];
drivers/scsi/libsas/sas_ata.c +2 -2
Index: work/drivers/scsi/libsas/sas_ata.c
===================================================================
@@ -178,8 +178,8 @@ static unsigned int sas_ata_qc_issue(str
 	task->uldd_task = qc;
 	if (ata_is_atapi(qc->tf.protocol)) {
 		memcpy(task->ata_task.atapi_packet, qc->cdb, qc->dev->cdb_len);
-		task->total_xfer_len = qc->nbytes + qc->pad_len;
-		task->num_scatter = qc->pad_len ? qc->n_elem + 1 : qc->n_elem;
+		task->total_xfer_len = qc->nbytes;
+		task->num_scatter = qc->n_elem;
 	} else {
 		for_each_sg(qc->sg, sg, qc->n_elem, si)
 			xfer += sg->length;
drivers/scsi/ipr.c +2 -2
Index: work/drivers/scsi/ipr.c
===================================================================
@@ -5140,7 +5140,7 @@ static void ipr_build_ata_ioadl(struct i
 	struct ipr_ioarcb *ioarcb = &ipr_cmd->ioarcb;
 	struct ipr_ioadl_desc *ioadl = ipr_cmd->ioadl;
 	struct ipr_ioadl_desc *last_ioadl = NULL;
-	int len = qc->nbytes + qc->pad_len;
+	int len = qc->nbytes;
 	struct scatterlist *sg;
 	unsigned int si;
 
@@ -5206,7 +5206,7 @@ static unsigned int ipr_qc_issue(struct 
 	ioarcb->cmd_pkt.request_type = IPR_RQTYPE_ATA_PASSTHRU;
 	ioarcb->cmd_pkt.flags_hi |= IPR_FLAGS_HI_NO_LINK_DESC;
 	ioarcb->cmd_pkt.flags_hi |= IPR_FLAGS_HI_NO_ULEN_CHK;
-	ipr_cmd->dma_use_sg = qc->pad_len ? qc->n_elem + 1 : qc->n_elem;
+	ipr_cmd->dma_use_sg = qc->n_elem;
 
 	ipr_build_ata_ioadl(ipr_cmd, qc);
 	regs->flags |= IPR_ATA_FLAG_STATUS_ON_GOOD_COMPLETION;
drivers/ata/ahci.c +0 -5
Index: work/drivers/ata/ahci.c
===================================================================
@@ -1979,16 +1979,11 @@ static int ahci_port_start(struct ata_po
 	struct ahci_port_priv *pp;
 	void *mem;
 	dma_addr_t mem_dma;
-	int rc;
 
 	pp = devm_kzalloc(dev, sizeof(*pp), GFP_KERNEL);
 	if (!pp)
 		return -ENOMEM;
 
-	rc = ata_pad_alloc(ap, dev);
-	if (rc)
-		return rc;
-
 	mem = dmam_alloc_coherent(dev, AHCI_PORT_PRIV_DMA_SZ, &mem_dma,
 				  GFP_KERNEL);
 	if (!mem)
drivers/ata/pata_icside.c +0 -8
Index: work/drivers/ata/pata_icside.c
===================================================================
@@ -304,12 +304,6 @@ static int icside_dma_init(struct pata_i
 }
 
 
-static int pata_icside_port_start(struct ata_port *ap)
-{
-	/* No PRD to alloc */
-	return ata_pad_alloc(ap, ap->dev);
-}
-
 static struct scsi_host_template pata_icside_sht = {
 	.module			= THIS_MODULE,
 	.name			= DRV_NAME,
@@ -389,8 +383,6 @@ static struct ata_port_operations pata_i
 	.irq_clear		= ata_dummy_noret,
 	.irq_on			= ata_irq_on,
 
-	.port_start		= pata_icside_port_start,
-
 	.bmdma_stop		= pata_icside_bmdma_stop,
 	.bmdma_status		= pata_icside_bmdma_status,
 };
author maintainer
#24 lore raw
Quoted 67 lines · Tejun Heo

And, here's working version. I'll splite and post them tomorrow.

Thanks.

Index: work/block/blk-core.c
===================================================================

block/blk-core.c +2 -0
@@ -116,6 +116,7 @@ void rq_init(struct request_queue *q, st
 	rq->ref_count = 1;
 	rq->q = q;
 	rq->special = NULL;
+	rq->raw_data_len = 0;
 	rq->data_len = 0;
 	rq->data = NULL;
 	rq->nr_phys_segments = 0;
@@ -1982,6 +1983,7 @@ void blk_rq_bio_prep(struct request_queu
 	rq->hard_cur_sectors = rq->current_nr_sectors;
 	rq->hard_nr_sectors = rq->nr_sectors = bio_sectors(bio);
 	rq->buffer = bio_data(bio);
+	rq->raw_data_len = bio->bi_size;
 	rq->data_len = bio->bi_size;
 
 	rq->bio = rq->biotail = bio;

Index: work/block/blk-map.c
===================================================================

block/blk-map.c +20 -0
@@ -19,6 +19,7 @@ int blk_rq_append_bio(struct request_que
 		rq->biotail->bi_next = bio;
 		rq->biotail = bio;
 
+		rq->raw_data_len += bio->bi_size;
 		rq->data_len += bio->bi_size;
 	}
 	return 0;
@@ -139,6 +140,25 @@ int blk_rq_map_user(struct request_queue
 		ubuf += ret;
 	}
 
+	/*
+	 * __blk_rq_map_user() copies the buffers if starting address
+	 * or length aren't aligned.  As the copied buffer is always
+	 * page aligned, we know for a fact that there's enough room
+	 * for padding.  Extend the last bio and update rq->data_len
+	 * accordingly.
+	 *
+	 * On unmap, bio_uncopy_user() will use unmodified
+	 * bio_map_data pointed to by bio->bi_private.
+	 */
+	if (len & queue_dma_alignment(q)) {
+		unsigned int pad_len = (queue_dma_alignment(q) & ~len) + 1;
+		struct bio *bio = rq->biotail;
+
+		bio->bi_io_vec[bio->bi_vcnt - 1].bv_len += pad_len;
+		bio->bi_size += pad_len;
+		rq->data_len += pad_len;
+	}
+
 	rq->buffer = rq->data = NULL;
 	return 0;
 unmap_rq:

Index: work/include/linux/blkdev.h
===================================================================

include/linux/blkdev.h struct request { · line 220

Quoted 7 lines
include/linux/blkdev.h +1 -0
@@ -214,6 +214,7 @@ struct request {
 	unsigned int cmd_len;
 	unsigned char cmd[BLK_MAX_CDB];
 
+	unsigned int raw_data_len;
 	unsigned int data_len;
 	unsigned int sense_len;
 	void *data;

Please, no ... none of this is necessary. You're wasting four bytes in every request from a quantity we already know in the queue.

The point of the original code is that the drain is always waiting for you silently in the sg list, but never shown in the length. That means it's entirely up to you whether you use it or ignore it. It also means that there's no need for the discriminating function in block, you either do or don't use the drain element.

include/linux/blkdev.h struct request_queue · line 297

Quoted 13 lines
include/linux/blkdev.h +2 -0
@@ -256,6 +257,7 @@ struct bio_vec;
 typedef int (merge_bvec_fn) (struct request_queue *, struct bio *, struct bio_vec *);
 typedef void (prepare_flush_fn) (struct request_queue *, struct request *);
 typedef void (softirq_done_fn)(struct request *);
+typedef int (dma_drain_needed_fn)(struct request *);
 
 enum blk_queue_state {
 	Queue_down,
@@ -292,6 +294,7 @@ struct request_queue
 	merge_bvec_fn		*merge_bvec_fn;
 	prepare_flush_fn	*prepare_flush_fn;
 	softirq_done_fn		*softirq_done_fn;
+	dma_drain_needed_fn	*dma_drain_needed;

Like I said, not necessary because the MMC devices are all single command queues, so for them, you simply apply the drain buffer the whole time in block and decide whether to use it in the issue layer

Quoted 125 lines
include/linux/blkdev.h +3 -2
 	/*
 	 * Dispatch queue sorting
@@ -696,8 +699,9 @@ extern void blk_queue_max_hw_segments(st
 extern void blk_queue_max_segment_size(struct request_queue *, unsigned int);
 extern void blk_queue_hardsect_size(struct request_queue *, unsigned short);
 extern void blk_queue_stack_limits(struct request_queue *t, struct request_queue *b);
-extern int blk_queue_dma_drain(struct request_queue *q, void *buf,
-			       unsigned int size);
+extern int blk_queue_dma_drain(struct request_queue *q,
+			       dma_drain_needed_fn *dma_drain_needed,
+			       void *buf, unsigned int size);
 extern void blk_queue_segment_boundary(struct request_queue *, unsigned long);
 extern void blk_queue_prep_rq(struct request_queue *, prep_rq_fn *pfn);
 extern void blk_queue_merge_bvec(struct request_queue *, merge_bvec_fn *);

Index: work/block/blk-merge.c
===================================================================

block/blk-merge.c +2 -1
@@ -220,7 +220,7 @@ new_segment:
 		bvprv = bvec;
 	} /* segments in rq */
 
-	if (q->dma_drain_size) {
+	if (q->dma_drain_size && q->dma_drain_needed(rq)) {
 		sg->page_link &= ~0x02;
 		sg = sg_next(sg);
 		sg_set_page(sg, virt_to_page(q->dma_drain_buffer),
@@ -228,6 +228,7 @@ new_segment:
 			    ((unsigned long)q->dma_drain_buffer) &
 			    (PAGE_SIZE - 1));
 		nsegs++;
+		rq->data_len += q->dma_drain_size;
 	}
 
 	if (sg)

Index: work/block/bsg.c
===================================================================

block/bsg.c +4 -4
@@ -437,14 +437,14 @@ static int blk_complete_sgv4_hdr_rq(stru
 	}
 
 	if (rq->next_rq) {
-		hdr->dout_resid = rq->data_len;
-		hdr->din_resid = rq->next_rq->data_len;
+		hdr->dout_resid = rq->raw_data_len;
+		hdr->din_resid = rq->next_rq->raw_data_len;
 		blk_rq_unmap_user(bidi_bio);
 		blk_put_request(rq->next_rq);
 	} else if (rq_data_dir(rq) == READ)
-		hdr->din_resid = rq->data_len;
+		hdr->din_resid = rq->raw_data_len;
 	else
-		hdr->dout_resid = rq->data_len;
+		hdr->dout_resid = rq->raw_data_len;
 
 	/*
 	 * If the request generated a negative error number, return it

Index: work/block/scsi_ioctl.c
===================================================================

block/scsi_ioctl.c +2 -1
@@ -266,7 +266,7 @@ static int blk_complete_sghdr_rq(struct 
 	hdr->info = 0;
 	if (hdr->masked_status || hdr->host_status || hdr->driver_status)
 		hdr->info |= SG_INFO_CHECK;
-	hdr->resid = rq->data_len;
+	hdr->resid = rq->raw_data_len;
 	hdr->sb_len_wr = 0;
 
 	if (rq->sense_len && hdr->sbp) {
@@ -528,6 +528,7 @@ static int __blk_send_generic(struct req
 	rq = blk_get_request(q, WRITE, __GFP_WAIT);
 	rq->cmd_type = REQ_TYPE_BLOCK_PC;
 	rq->data = NULL;
+	rq->raw_data_len = 0;
 	rq->data_len = 0;
 	rq->timeout = BLK_DEFAULT_SG_TIMEOUT;
 	memset(rq->cmd, 0, sizeof(rq->cmd));

Index: work/drivers/scsi/scsi_lib.c
===================================================================

drivers/scsi/scsi_lib.c +4 -4
@@ -1015,10 +1015,6 @@ static int scsi_init_sgtable(struct requ
 	}
 
 	req->buffer = NULL;
-	if (blk_pc_request(req))
-		sdb->length = req->data_len;
-	else
-		sdb->length = req->nr_sectors << 9;
 
 	/* 
 	 * Next, walk the list, and fill in the addresses and sizes of
@@ -1027,6 +1023,10 @@ static int scsi_init_sgtable(struct requ
 	count = blk_rq_map_sg(req->q, req, sdb->table.sgl);
 	BUG_ON(count > sdb->table.nents);
 	sdb->table.nents = count;
+	if (blk_pc_request(req))
+		sdb->length = req->data_len;
+	else
+		sdb->length = req->nr_sectors << 9;
 	return BLKPREP_OK;
 }
 

Index: work/block/blk-settings.c
===================================================================

block/blk-settings.c +3 -2
@@ -296,6 +296,7 @@ EXPORT_SYMBOL(blk_queue_stack_limits);
  * blk_queue_dma_drain - Set up a drain buffer for excess dma.
  *
  * @q:  the request queue for the device
+ * @dma_drain_needed: fn which returns non-zero if drain is necessary
  * @buf:	physically contiguous buffer
  * @size:	size of the buffer in bytes
  *
@@ -315,14 +316,16 @@ EXPORT_SYMBOL(blk_queue_stack_limits);
  * device can support otherwise there won't be room for the drain
  * buffer.
  */
-int blk_queue_dma_drain(struct request_queue *q, void *buf,
-				unsigned int size)
+extern int blk_queue_dma_drain(struct request_queue *q,
+			       dma_drain_needed_fn *dma_drain_needed,

block/blk-settings.c EXPORT_SYMBOL(blk_queue_stack_limits); · line 330

Quoted 10 lines
block/blk-settings.c +2 -0
+			       void *buf, unsigned int size)
 {
 	if (q->max_hw_segments < 2 || q->max_phys_segments < 2)
 		return -EINVAL;
 	/* make room for appending the drain */
 	--q->max_hw_segments;
 	--q->max_phys_segments;
+	q->dma_drain_needed = dma_drain_needed;
 	q->dma_drain_buffer = buf;
 	q->dma_drain_size = size;

All of these block changes go away if you simply use the drain where it's needed

Quoted 84 lines

Index: work/drivers/scsi/sr.c
===================================================================

drivers/scsi/sr.c +30 -6
@@ -557,11 +557,30 @@ static void sr_release(struct cdrom_devi
 
 }
 
+static int sr_drain_needed(struct request *rq)
+{
+	if (likely(!blk_pc_request(rq)))
+		return 0;
+
+	switch (rq->cmd[0]) {
+	case GPCMD_READ_10:
+	case GPCMD_READ_12:
+	case GPCMD_WRITE_10:
+	case GPCMD_WRITE_12:
+	case GPCMD_WRITE_AND_VERIFY_10:
+		return 0;
+	}
+
+	return 1;
+}
+
 static int sr_probe(struct device *dev)
 {
 	struct scsi_device *sdev = to_scsi_device(dev);
+	struct request_queue *queue = sdev->request_queue;
 	struct gendisk *disk;
-	struct scsi_cd *cd;
+	struct scsi_cd *cd = NULL;
+	void *drain_buf = NULL;
 	int minor, error;
 
 	error = -ENODEV;
@@ -573,11 +592,15 @@ static int sr_probe(struct device *dev)
 	if (!cd)
 		goto fail;
 
+	drain_buf = kmalloc(SR_DRAIN_SIZE, queue->bounce_gfp | GFP_KERNEL);
+	if (!drain_buf)
+		goto fail;
+
 	kref_init(&cd->kref);
 
 	disk = alloc_disk(1);
 	if (!disk)
-		goto fail_free;
+		goto fail;
 
 	spin_lock(&sr_index_lock);
 	minor = find_first_zero_bit(sr_index_bits, SR_DISKS);
@@ -615,13 +638,14 @@ static int sr_probe(struct device *dev)
 
 	/* FIXME: need to handle a get_capabilities failure properly ?? */
 	get_capabilities(cd);
-	blk_queue_prep_rq(sdev->request_queue, sr_prep_fn);
+	blk_queue_prep_rq(queue, sr_prep_fn);
+	blk_queue_dma_drain(queue, sr_drain_needed, drain_buf, SR_DRAIN_SIZE);
 	sr_vendor_init(cd);
 
 	disk->driverfs_dev = &sdev->sdev_gendev;
 	set_capacity(disk, cd->capacity);
 	disk->private_data = &cd->driver;
-	disk->queue = sdev->request_queue;
+	disk->queue = queue;
 	cd->cdi.disk = disk;
 
 	if (register_cdrom(&cd->cdi))
@@ -637,9 +661,9 @@ static int sr_probe(struct device *dev)
 
 fail_put:
 	put_disk(disk);
-fail_free:
-	kfree(cd);
 fail:
+	kfree(cd);
+	kfree(drain_buf);
 	return error;
 }
 
@@ -894,6 +918,12 @@ static void sr_kref_release(struct kref 
 static int sr_remove(struct device *dev)

drivers/scsi/sr.c static void sr_kref_release(struct kref · line 928

Quoted 10 lines
drivers/scsi/sr.c +6 -0
 {
 	struct scsi_cd *cd = dev_get_drvdata(dev);
+	struct scsi_device *sdev = to_scsi_device(dev);
+	struct request_queue *queue = sdev->request_queue;
+
+	kfree(queue->dma_drain_buffer);
+	queue->dma_drain_buffer = NULL;
+	queue->dma_drain_size = 0;
 
 	del_gendisk(cd->disk);

This is basically all setup. There's no actual changes for use. Since the only consumer of this is libata, all of this needs to be gated by whether it's a libata device ... which really suggests it needs to be done in libata.

Plus, tape devices are also ATAPI and since the problem seems to be handling of ATAPI pio commands, they need all of this too. It really is, I think, better just to do the setup in the libata slave configure if the device is atapi.

For all the rest, I think code demonstrates better ... well, except the sata_fsl.c update ... you leave in qc->n_iter, but it's only ever set to zero ... doesn't that cause this driver to break?

I analysed all the places you use the expanded data length. To keep the true length in the request and add the drain if necessary, I think requires this update over my published libata drain patch. I've also added the request accessor to the drain buffer. Could you verify this actually works? If it does, it's better than all the additions to block and sr. It would probably help to resend the series rebased against git current.

James

---

drivers/ata/libata-scsi.c +4 -0
index a28351c..acf6a8b 100644
@@ -2496,6 +2496,8 @@ static unsigned int atapi_xlat(struct ata_queued_cmd *qc)
 
 	qc->tf.command = ATA_CMD_PACKET;
 	qc->nbytes = scsi_bufflen(scmd);
+	if (blk_pc_request(scmd->request))
+		qc->nbytes += blk_rq_drain_size(scmd->request);
 
 	/* check whether ATAPI DMA is safe */
 	if (!using_pio && ata_check_atapi_dma(qc))
@@ -2832,6 +2834,8 @@ static unsigned int ata_scsi_pass_thru(struct ata_queued_cmd *qc)
 	 *       cover scatter/gather case.
 	 */
 	qc->nbytes = scsi_bufflen(scmd);
+	if (ata_is_atapi(qc->tf.protocol))
+		qc->nbytes += blk_rq_drain_size(scmd->request);
 
 	/* request result TF and be quiet about device error */
 	qc->flags |= ATA_QCFLAG_RESULT_TF | ATA_QCFLAG_QUIET;
include/linux/blkdev.h +5 -0
index 90392a9..a526066 100644
@@ -676,6 +676,11 @@ extern void blk_complete_request(struct request *);
 extern unsigned int blk_rq_bytes(struct request *rq);
 extern unsigned int blk_rq_cur_bytes(struct request *rq);
 
+static inline int blk_rq_drain_size(struct request *rq)
+{
+	return rq->q->dma_drain_size;
+}
+
 static inline void blkdev_dequeue_request(struct request *req)
 {
 	elv_dequeue_request(req->q, req);
#25 lore raw
Quoted 4 lines · James Bottomley

Plus, tape devices are also ATAPI and since the problem seems to be handling of ATAPI pio commands, they need all of this too. It really is, I think, better just to do the setup in the libata slave configure if the device is atapi.

If no SCSI HBA needs it, fine.

Quoted 3 lines

For all the rest, I think code demonstrates better ... well, except the sata_fsl.c update ... you leave in qc->n_iter, but it's only ever set to zero ... doesn't that cause this driver to break?

Probably. I was gonna cross check with your patch while splitting. They seem mostly identical so I'll probably use your patch with some modifications.

Quoted 7 lines

I analysed all the places you use the expanded data length. To keep the true length in the request and add the drain if necessary, I think requires this update over my published libata drain patch. I've also added the request accessor to the drain buffer. Could you verify this actually works? If it does, it's better than all the additions to block and sr. It would probably help to resend the series rebased against git current.

No, it doesn't. Drain needs to extend the sg table too and it makes controllers lax about buffer overruns for commands they shouldn't be. Gee.. What's the deal with extra four bytes? If you're worried about the size of struct request, chopping off PC request part into a separate struct would be far better strategy than making what's already confusing more confusing.

Thanks.

Signature · 1 line
tejun
author maintainer
#26 lore raw
Quoted 24 lines · Tejun Heo
Quoted 4 lines · James Bottomley

Plus, tape devices are also ATAPI and since the problem seems to be handling of ATAPI pio commands, they need all of this too. It really is, I think, better just to do the setup in the libata slave configure if the device is atapi.

If no SCSI HBA needs it, fine.

Quoted 3 lines

For all the rest, I think code demonstrates better ... well, except the sata_fsl.c update ... you leave in qc->n_iter, but it's only ever set to zero ... doesn't that cause this driver to break?

Probably. I was gonna cross check with your patch while splitting. They seem mostly identical so I'll probably use your patch with some modifications.

Quoted 7 lines

I analysed all the places you use the expanded data length. To keep the true length in the request and add the drain if necessary, I think requires this update over my published libata drain patch. I've also added the request accessor to the drain buffer. Could you verify this actually works? If it does, it's better than all the additions to block and sr. It would probably help to resend the series rebased against git current.

Quoted 1 line

No, it doesn't. Drain needs to extend the sg table too

The patches do extend the sg table.

Quoted 1 line

and it makes controllers lax about buffer overruns for commands they shouldn't be.

So adjust the qc->nbytes to show no drain element in libata ... although the extra element gets mapped, the HBA will be none the wiser if there's zero length allocated to it. Thus the entire system behaves exactly like the drain element isn't present. The block layer is really just making the drain element available ... you don't have to use it.

Quoted 4 lines

Gee.. What's the deal with extra four bytes? If you're worried about the size of struct request, chopping off PC request part into a separate struct would be far better strategy than making what's already confusing more confusing.

Well, we are trying to slim down the request structure and scsi_cmnd structure. However, there's no inherent advantage to counting this all in the block layer. Think of it just as a transparent passthrough. You've told the block layer to make sure you have room for a drain element when it constructs the SG list, which it does. Whether you use it (by adding blk_rq_drain_size() on to the actual length you hand down) or not is entirely up to you ... there's no need to push the decision on that one into the block layer, since use or not is pretty much a cost free thing.

James

#27 lore raw

Hello,

Quoted 3 lines · James Bottomley
Quoted 1 line

No, it doesn't. Drain needs to extend the sg table too

The patches do extend the sg table.

Hmm... Where?

Quoted 7 lines
Quoted 1 line

and it makes controllers lax about buffer overruns for commands they shouldn't be.

So adjust the qc->nbytes to show no drain element in libata ... although the extra element gets mapped, the HBA will be none the wiser if there's zero length allocated to it. Thus the entire system behaves exactly like the drain element isn't present. The block layer is really just making the drain element available ... you don't have to use it.

Adjusting qc->nbytes isn't enough. libata will have to trim at the end of sglist. With both draining and padding, it means libata will have to trim one sg entry and a few more bytes. Block layer is behaving differently and in a very noticeable way, it's sending inconsistent values in data_len and sg list.

Quoted 5 lines
Quoted 4 lines

Gee.. What's the deal with extra four bytes? If you're worried about the size of struct request, chopping off PC request part into a separate struct would be far better strategy than making what's already confusing more confusing.

Quoted 9 lines

Well, we are trying to slim down the request structure and scsi_cmnd structure. However, there's no inherent advantage to counting this all in the block layer. Think of it just as a transparent passthrough. You've told the block layer to make sure you have room for a drain element when it constructs the SG list, which it does. Whether you use it (by adding blk_rq_drain_size() on to the actual length you hand down) or not is entirely up to you ... there's no need to push the decision on that one into the block layer, since use or not is pretty much a cost free thing.

Okay, I see your point. The biggest problem is that by doing that blk layer breaks a pretty important rule - keeping data len consistent with sg list and the bugs it's gonna cause will be very subtle and dangerous.
This is definitely where we can spend four extra bytes.

Thanks.

Signature · 1 line
tejun
author maintainer
#28 lore raw
Quoted 8 lines · Tejun Heo

Hello,

Quoted 3 lines · James Bottomley
Quoted 1 line

No, it doesn't. Drain needs to extend the sg table too

The patches do extend the sg table.

Hmm... Where?

It's the block component; it's already in git head with this patch:

commit fa0ccd837e3dddb44c7db2f128a8bb7e4eabc21a

Author: James Bottomley <[email protected]>
Date:   Thu Jan 10 11:30:36 2008 -0600

block: implement drain buffers

The description is pretty reasonable. The way it works is by making block add the drain buffer to the sg table as the last element.

The way it works is if you simply use the length block gives you, you find yourself doing DMA to every element in the sg table except the last one. If you expand the length to anything up to blk_rq_drain_size() you then use the last element for the drain.

Quoted 8 lines
Quoted 7 lines
Quoted 1 line

and it makes controllers lax about buffer overruns for commands they shouldn't be.

So adjust the qc->nbytes to show no drain element in libata ... although the extra element gets mapped, the HBA will be none the wiser if there's zero length allocated to it. Thus the entire system behaves exactly like the drain element isn't present. The block layer is really just making the drain element available ... you don't have to use it.

Quoted 5 lines

Adjusting qc->nbytes isn't enough. libata will have to trim at the end of sglist. With both draining and padding, it means libata will have to trim one sg entry and a few more bytes. Block layer is behaving differently and in a very noticeable way, it's sending inconsistent values in data_len and sg list.

No, that's the point. With all the space available, you don't need to trim at all ... you simply don't use it. The rationale is the way the sg element stuff works. If you program a DMA table (PRD table in IDE speak) into a controller, it doesn't care if you don't use the last few pieces. With the default length set, the DMA controller won't use the last programmed element, but it's their if you expand the length or tell it to for an overrun.

Quoted 15 lines
Quoted 5 lines
Quoted 4 lines

Gee.. What's the deal with extra four bytes? If you're worried about the size of struct request, chopping off PC request part into a separate struct would be far better strategy than making what's already confusing more confusing.

Quoted 9 lines

Well, we are trying to slim down the request structure and scsi_cmnd structure. However, there's no inherent advantage to counting this all in the block layer. Think of it just as a transparent passthrough. You've told the block layer to make sure you have room for a drain element when it constructs the SG list, which it does. Whether you use it (by adding blk_rq_drain_size() on to the actual length you hand down) or not is entirely up to you ... there's no need to push the decision on that one into the block layer, since use or not is pretty much a cost free thing.

Quoted 4 lines

Okay, I see your point. The biggest problem is that by doing that blk layer breaks a pretty important rule - keeping data len consistent with sg list and the bugs it's gonna cause will be very subtle and dangerous.
This is definitely where we can spend four extra bytes.

The lengths don't have to be consistent (otherwise we wouldn't need to calculate them separately) the request reported length just has to be less than the sg table actual length. We even make use of this property in SCSI when we can't fit a request into a single command ... we are allowed to chunk up the request as we complete it. I'm not saying it's best practice, but it is acceptable behaviour.

James

#29 lore raw
Quoted 12 lines · James Bottomley
Quoted 3 lines
Quoted 2 lines
Quoted 1 line

No, it doesn't. Drain needs to extend the sg table too

The patches do extend the sg table.

Hmm... Where?

It's the block component; it's already in git head with this patch:

commit fa0ccd837e3dddb44c7db2f128a8bb7e4eabc21a

Author: James Bottomley <[email protected]>
Date:   Thu Jan 10 11:30:36 2008 -0600

block: implement drain buffers

Quoted 7 lines

The description is pretty reasonable. The way it works is by making block add the drain buffer to the sg table as the last element.

The way it works is if you simply use the length block gives you, you find yourself doing DMA to every element in the sg table except the last one. If you expand the length to anything up to blk_rq_drain_size() you then use the last element for the drain.

Oops, I was talking about padding. Sorry about that.

Quoted 6 lines
Quoted 5 lines

Adjusting qc->nbytes isn't enough. libata will have to trim at the end of sglist. With both draining and padding, it means libata will have to trim one sg entry and a few more bytes. Block layer is behaving differently and in a very noticeable way, it's sending inconsistent values in data_len and sg list.

Quoted 7 lines

No, that's the point. With all the space available, you don't need to trim at all ... you simply don't use it. The rationale is the way the sg element stuff works. If you program a DMA table (PRD table in IDE speak) into a controller, it doesn't care if you don't use the last few pieces. With the default length set, the DMA controller won't use the last programmed element, but it's their if you expand the length or tell it to for an overrun.

Not all controllers have a place to set 'default length'. They just run till sg list terminates.

Quoted 11 lines
Quoted 4 lines

Okay, I see your point. The biggest problem is that by doing that blk layer breaks a pretty important rule - keeping data len consistent with sg list and the bugs it's gonna cause will be very subtle and dangerous.
This is definitely where we can spend four extra bytes.

The lengths don't have to be consistent (otherwise we wouldn't need to calculate them separately) the request reported length just has to be less than the sg table actual length. We even make use of this property in SCSI when we can't fit a request into a single command ... we are allowed to chunk up the request as we complete it. I'm not saying it's best practice, but it is acceptable behaviour.

libata definitely isn't ready for that and I don't think expanding the behavior to whole block layer is a wise thing to do when the best practice can be bought at the cost of 4 bytes per request. No?

Signature · 1 line
tejun
author maintainer
#30 lore raw
Quoted 16 lines · Tejun Heo
Quoted 15 lines · James Bottomley
Quoted 3 lines
Quoted 2 lines
Quoted 1 line

No, it doesn't. Drain needs to extend the sg table too

The patches do extend the sg table.

Hmm... Where?

It's the block component; it's already in git head with this patch:

commit fa0ccd837e3dddb44c7db2f128a8bb7e4eabc21a

Author: James Bottomley <[email protected]>
Date:   Thu Jan 10 11:30:36 2008 -0600

block: implement drain buffers

The description is pretty reasonable. The way it works is by making block add the drain buffer to the sg table as the last element.

Quoted 6 lines
Quoted 4 lines

The way it works is if you simply use the length block gives you, you find yourself doing DMA to every element in the sg table except the last one. If you expand the length to anything up to blk_rq_drain_size() you then use the last element for the drain.

Oops, I was talking about padding. Sorry about that.

Oh, OK ... the I thought the changelog was pretty explicit. However, it works because the dma_aligment parameter of the block layer ensures that all elements of the sg list begin and end on an address that conforms to the dma_alignment. It does this basically by forcing a copy of user incoming commands if they don't conform. It does nothing with kernel based commands, but you can always assume for direct DMA that they begin and end on the cache line boundary ... and if they don't, there's always a spare allocation to expand them.

Quoted 14 lines
Quoted 6 lines
Quoted 5 lines

Adjusting qc->nbytes isn't enough. libata will have to trim at the end of sglist. With both draining and padding, it means libata will have to trim one sg entry and a few more bytes. Block layer is behaving differently and in a very noticeable way, it's sending inconsistent values in data_len and sg list.

Quoted 7 lines

No, that's the point. With all the space available, you don't need to trim at all ... you simply don't use it. The rationale is the way the sg element stuff works. If you program a DMA table (PRD table in IDE speak) into a controller, it doesn't care if you don't use the last few pieces. With the default length set, the DMA controller won't use the last programmed element, but it's their if you expand the length or tell it to for an overrun.

Quoted 2 lines

Not all controllers have a place to set 'default length'. They just run till sg list terminates.

True ... but the traversal of the sg list terminates when the data transfer finishes ... this is the basis of your drain patch (and mine). Basically we don't require that anything transfer into the drain, but it's there just in case the transfer overruns and it needs to be used.

Quoted 12 lines
Quoted 11 lines
Quoted 4 lines

Okay, I see your point. The biggest problem is that by doing that blk layer breaks a pretty important rule - keeping data len consistent with sg list and the bugs it's gonna cause will be very subtle and dangerous.
This is definitely where we can spend four extra bytes.

The lengths don't have to be consistent (otherwise we wouldn't need to calculate them separately) the request reported length just has to be less than the sg table actual length. We even make use of this property in SCSI when we can't fit a request into a single command ... we are allowed to chunk up the request as we complete it. I'm not saying it's best practice, but it is acceptable behaviour.

Quoted 3 lines

libata definitely isn't ready for that and I don't think expanding the behavior to whole block layer is a wise thing to do when the best practice can be bought at the cost of 4 bytes per request. No?

but the whole basis of both patches is inconsistent lengths: we're expecting a transfer of X, but we program the element lists for Y (>X) and see if the transfer actually needs >X (but hopefully <Y).

James

#31 lore raw
Quoted 10 lines · James Bottomley
Quoted 1 line

Oops, I was talking about padding. Sorry about that.

Oh, OK ... the I thought the changelog was pretty explicit. However, it works because the dma_aligment parameter of the block layer ensures that all elements of the sg list begin and end on an address that conforms to the dma_alignment. It does this basically by forcing a copy of user incoming commands if they don't conform. It does nothing with kernel based commands, but you can always assume for direct DMA that they begin and end on the cache line boundary ... and if they don't, there's always a spare allocation to expand them.

Yeah, the area is expanded but the bio list isn't updated accordingly. In turn the sg list won't be. I found where you worked around this. The following chunk is added to ata_sg_setup().

+ /* needs padding? */
+ if (pad) {
+ struct scatterlist *sg = sg_last(qc->sg, n_elem);
+ qc->nbytes += ATA_DMA_PAD_SZ - pad;
+ sg->length += ATA_DMA_PAD_SZ - pad;
+ }

So, you add the area way up in the stack and adjust sg way down the stack. This is too fragile. Actually, it's already broken and demonstrates very well why block layer shouldn't do half of the job and defer the rest to lower layer.

With draining, the above code ends up extending the sg entry for drain page by pad bytes and there's no extra byte allocated after the drain page. The controller will end up contaminating pad bytes of the next page and it will be extremely difficult to reproduce and track down.

Quoted 7 lines
Quoted 2 lines

Not all controllers have a place to set 'default length'. They just run till sg list terminates.

True ... but the traversal of the sg list terminates when the data transfer finishes ... this is the basis of your drain patch (and mine). Basically we don't require that anything transfer into the drain, but it's there just in case the transfer overruns and it needs to be used.

The thing is there is no way to detect data overrun if sg list doesn't terminate. For data transfer commands, sg list must terminate precisely at the request boundary. If 4k drain page is automatically appended, the controller will happily overflow into that 4k without notifying anyone. Or worse, the device can suck in few extra blocks without triggering error. sgl should never contain extra entries for any command which can affect data integrity.

Quoted 7 lines
Quoted 3 lines

libata definitely isn't ready for that and I don't think expanding the behavior to whole block layer is a wise thing to do when the best practice can be bought at the cost of 4 bytes per request. No?

but the whole basis of both patches is inconsistent lengths: we're expecting a transfer of X, but we program the element lists for Y (>X) and see if the transfer actually needs >X (but hopefully <Y).

Yes, it's dual-sizing. There's the size upper layer requested and there's slightly different size lower layer wants to see (most of the time). Block layer sits inbetween and matches the impedance by extending the request necessary but from either side the picture is still consistent. Lower layer (mostly) just sees ->data_len and matching sgl. Upper layer sends and receives the data length it requested.

The difference is that with your approach what lower layer sees becomes inconsistent in itself and half of the impedance matching is left to lower layer and it will cause nasty problems down the road.

Thanks.

Signature · 1 line
tejun
author maintainer
#32 lore raw
In reply to Tejun Heo
Quoted 9 lines · Tejun Heo
Quoted 7 lines · Tejun Heo

Some ATA controllers including SFF BMDMA and libata PIO HSM need the number of bytes mapped in the sg table. Yeah, it can be calculated with a simple macro but it also is a fundamentally confusing dual-sizing which should be made as clear as possible. Plus, it can be difficult to find out when somebody used the wrong thing, so what I'm saying is that we need to make it easy. Anyways, please lemme work on it a bit. I'll get back to you guys soon.

Quoted 4 lines

Okay, here's first draft combined patch. Only compile tested (expect it to be broken) but it should be functionally equivalent to ata_sg_setup_extra() based implementation albeit with shorter drain buffer size. Several things to note...

I noticed you posted an update ... I'll go over the code in that one.

Quoted 5 lines

* fsl last sg check isn't included here. Will split it out and post it separately.

* rq->raw_data_len added. Rationales...

Quoted 7 lines

- All these padding and draining are to prevent controllers from crapping themselves when data buffer is shorter than it likes it to be. Any controller which talks MMC (or SPC for that matter) should be ready for transfers shorter than buffer so feeding enlarged buffer size is inherently safter than feeding the length, so the primary data length field, rq->data_len, contains the adjusted length.

Actually, no they don't. Overrun termination is pretty much standard in most HBAs that do MMC (including the strange block one). Allowing an overrun is simply unsafe and a security risk, so all apart from the ATA ones allow us to terminate the transaction safely.

Quoted 3 lines

- raw_data_len can't be easily deduced from data_len. The other way is possible but with both aligning and draining and command filtering, calculating it later is messy.

Like I said, it's simply the command size plus the drain buffer which you get from the queue. I'll redo your patches to show how it can be done without adding the extra space to every request.

Quoted 7 lines

* Draining configuration is done in sr as it's the driver for MMC. It can move both ways - either into SCSI midlayer as SPC and other commands do variable length responses too or into libata if all non-ATA controllers are happy without such workarounds. If you ask me, I'm inclined to move it into SCSI midlayer as the added overhead is insignificant (especially with drain_needed added) and it won't break anything (well, theoretically, at least).

Let me think about this one. The only consumer of draining is libata. It might make sense to process the drains in sr.c but it certainly doesn't make sense to turn them on indiscriminately in there because *none* of the SCSI users needs them.

Quoted 3 lines

* Padding via alinging seems a bit too hacky to me. It doesn't even cover all sg cases. I think we'll need improvements there, well, but for the time being, this should do.

That's essentially what the dma_aligment parameter was designed for ... we can always make the way it's handled differently; I just updated block to use what exists already.

Quoted 1 line

I'll test and report in a few hours.

Great, thanks.

James