Skip to contentSkip to conversation

mmc: Add sdhci workaround stability enhencement

PATCH In review Jun Nie 6 participants 24 messages Dec 2, 2019 – Dec 17, 2019

Subsystems (5)
lore
Share & tools
lore

Jun Nie

author
#0 lore raw

cover mmc: Add sdhci workaround stability enhencement

This patch set made three changes:
1. set enhence full power cycle stability.
2. Add dt property to support DMA memory address boundary workaround.
3. Add dt property to non-standard HS400 mode value in ctrl register.

Jun Nie (4):
mmc: sdhci: Add delay after power off
mmc: sdhci: dt: Add DMA boundary and HS400 properties
mmc: sdhci: Set ctrl_hs400 value in dts
mmc: sdhci: Add DMA memory boundary workaround

 .../devicetree/bindings/mmc/sdhci.txt         |  8 +++++
 drivers/mmc/host/sdhci.c                      | 36 +++++++++++++++++--
 drivers/mmc/host/sdhci.h                      |  9 ++++-
 3 files changed, 50 insertions(+), 3 deletions(-)
Signature · 1 line
2.17.1
author
#1 lore raw

PATCH 1/4 mmc: sdhci: Add delay after power off

Add delay after power off to ensure that full power cycle is successful. Otherwise, some controllers, at lease for Hisilicon eMMC controller, may not be unstable sometimes for full power cycle operation.

---
 drivers/mmc/host/sdhci.c | 7 +++++++
 drivers/mmc/host/sdhci.h | 2 ++
 2 files changed, 9 insertions(+)
drivers/mmc/host/sdhci.c +7 -0
index 3140fe2e5dba..a654f0aeb438 100644
@@ -1761,6 +1761,13 @@ void sdhci_set_power_noreg(struct sdhci_host *host, unsigned char mode,
 		sdhci_writeb(host, 0, SDHCI_POWER_CONTROL);
 		if (host->quirks2 & SDHCI_QUIRK2_CARD_ON_NEEDS_BUS_ON)
 			sdhci_runtime_pm_bus_off(host);
+
+		/*
+		 * Some controllers need an extra 100ms delay to secure
+		 * full power cycle is successful.
+		 */
+		if (host->quirks2 & SDHCI_QUIRK2_DELAY_AFTER_POWER_OFF)
+			msleep(100);
 	} else {
 		/*
 		 * Spec says that we should clear the power reg before setting
drivers/mmc/host/sdhci.h +2 -0
index 0ed3e0eaef5f..0e6f97eaa796 100644
@@ -482,6 +482,8 @@ struct sdhci_host {
  * block count.
  */
 #define SDHCI_QUIRK2_USE_32BIT_BLK_CNT			(1<<18)
+/* Controllers need an extra 100ms delay to make sure power off completely */
+#define SDHCI_QUIRK2_DELAY_AFTER_POWER_OFF		(1<<19)
 
 	int irq;		/* Device IRQ */
 	void __iomem *ioaddr;	/* Mapped address */
Signature · 1 line
2.17.1
maintainer
#2 lore raw

Re: [PATCH 1/4] mmc: sdhci: Add delay after power off

Quoted 16 lines · Jun Nie

Add delay after power off to ensure that full power cycle is successful. Otherwise, some controllers, at lease for Hisilicon eMMC controller, may not be unstable sometimes for full power cycle operation.

---
 drivers/mmc/host/sdhci.c | 7 +++++++
 drivers/mmc/host/sdhci.h | 2 ++
 2 files changed, 9 insertions(+)
drivers/mmc/host/sdhci.c +0 -0
index 3140fe2e5dba..a654f0aeb438 100644

drivers/mmc/host/sdhci.c void sdhci_set_power_noreg(struct sdhci_host *host, unsigned char mode, · line 1770

Quoted 10 lines
drivers/mmc/host/sdhci.c +7 -0
@@ -1761,6 +1761,13 @@ void sdhci_set_power_noreg(struct sdhci_host *host, unsigned char mode,
 		sdhci_writeb(host, 0, SDHCI_POWER_CONTROL);
 		if (host->quirks2 & SDHCI_QUIRK2_CARD_ON_NEEDS_BUS_ON)
 			sdhci_runtime_pm_bus_off(host);
+
+		/*
+		 * Some controllers need an extra 100ms delay to secure
+		 * full power cycle is successful.
+		 */
+		if (host->quirks2 & SDHCI_QUIRK2_DELAY_AFTER_POWER_OFF)
+			msleep(100);

Please use the ->set_power() callback and do this in your own driver.

Quoted 17 lines
drivers/mmc/host/sdhci.c +0 -0
 	} else {
 		/*
 		 * Spec says that we should clear the power reg before setting
drivers/mmc/host/sdhci.h +2 -0
index 0ed3e0eaef5f..0e6f97eaa796 100644
@@ -482,6 +482,8 @@ struct sdhci_host {
  * block count.
  */
 #define SDHCI_QUIRK2_USE_32BIT_BLK_CNT			(1<<18)
+/* Controllers need an extra 100ms delay to make sure power off completely */
+#define SDHCI_QUIRK2_DELAY_AFTER_POWER_OFF		(1<<19)
 
 	int irq;		/* Device IRQ */
 	void __iomem *ioaddr;	/* Mapped address */
author
#3 lore raw

PATCH 2/4 mmc: sdhci: dt: Add DMA boundary and HS400 properties

In reply to Jun Nie

DMA memory cannot cross specific boundary on some controller, such as 128MB on SDHCI Designware. Add sdhci-dma-mem-boundary property to split DMA operation in such case.

sdhci-ctrl-hs400 specify the HS400 mode setting for register SDHCI_HOST_CONTROL2(offset 0x3E:bit[2:0]). Because this value is not defined in SDHC Standard specification.

---
 Documentation/devicetree/bindings/mmc/sdhci.txt | 8 ++++++++
 1 file changed, 8 insertions(+)
Documentation/devicetree/bindings/mmc/sdhci.txt +8 -0
index 0e9923a64024..e6d7feb9a741 100644
@@ -11,3 +11,11 @@ Optional properties:
 - sdhci-caps: The sdhci capabilities register is incorrect. This 64bit
   property corresponds to the bits in the sdhci capability register. If the
   bit is on in the property then the bit should be turned on.
+- sdhci-dma-mem-boundary: The sdhci controller DMA memory space boundary.
+  If the controller's DMA cannot cross a specific memory space boundary,
+  such as 128MB, set this value in dt and driver will split the DMA
+  operation when crossing such boundary.
+- sdhci-ctrl-hs400: The HS400 is not defined in SDHC Standard specification
+  for SDHCI_HOST_CONTROL2(offset 0x3E:bit[2:0]). Different controllers have
+  have different value for HS400 mode. If 0x5 is not the HS400 mode value
+  for your controller, you should specify the value with this property.
Signature · 1 line
2.17.1
maintainer
#4 lore raw

Re: [PATCH 2/4] mmc: sdhci: dt: Add DMA boundary and HS400 properties

Quoted 18 lines · Jun Nie

DMA memory cannot cross specific boundary on some controller, such as 128MB on SDHCI Designware. Add sdhci-dma-mem-boundary property to split DMA operation in such case.

sdhci-ctrl-hs400 specify the HS400 mode setting for register SDHCI_HOST_CONTROL2(offset 0x3E:bit[2:0]). Because this value is not defined in SDHC Standard specification.

---
 Documentation/devicetree/bindings/mmc/sdhci.txt | 8 ++++++++
 1 file changed, 8 insertions(+)
Documentation/devicetree/bindings/mmc/sdhci.txt +0 -0
index 0e9923a64024..e6d7feb9a741 100644

Documentation/devicetree/bindings/mmc/sdhci.txt Optional properties: · line 17

Quoted 7 lines
Documentation/devicetree/bindings/mmc/sdhci.txt +4 -0
@@ -11,3 +11,11 @@ Optional properties:
 - sdhci-caps: The sdhci capabilities register is incorrect. This 64bit
   property corresponds to the bits in the sdhci capability register. If the
   bit is on in the property then the bit should be turned on.
+- sdhci-dma-mem-boundary: The sdhci controller DMA memory space boundary.
+  If the controller's DMA cannot cross a specific memory space boundary,
+  such as 128MB, set this value in dt and driver will split the DMA
+  operation when crossing such boundary.

This should be implied by the compatible string.

Documentation/devicetree/bindings/mmc/sdhci.txt Optional properties: · line 21

Quoted 4 lines
Documentation/devicetree/bindings/mmc/sdhci.txt +4 -0
+- sdhci-ctrl-hs400: The HS400 is not defined in SDHC Standard specification
+  for SDHCI_HOST_CONTROL2(offset 0x3E:bit[2:0]). Different controllers have
+  have different value for HS400 mode. If 0x5 is not the HS400 mode value
+  for your controller, you should specify the value with this property.

This too, unless it needs to be tuned per board.

Can you be more specific as to what the possible values are and what they do?

Rob

author
#5 lore raw

Re: [PATCH 2/4] mmc: sdhci: dt: Add DMA boundary and HS400 properties

Quoted 35 lines · Rob Herring <[email protected]> 于2019年12月14日周六 上午7:01写道:
Quoted 18 lines · Jun Nie

DMA memory cannot cross specific boundary on some controller, such as 128MB on SDHCI Designware. Add sdhci-dma-mem-boundary property to split DMA operation in such case.

sdhci-ctrl-hs400 specify the HS400 mode setting for register SDHCI_HOST_CONTROL2(offset 0x3E:bit[2:0]). Because this value is not defined in SDHC Standard specification.

---
 Documentation/devicetree/bindings/mmc/sdhci.txt | 8 ++++++++
 1 file changed, 8 insertions(+)
Documentation/devicetree/bindings/mmc/sdhci.txt +0 -0
index 0e9923a64024..e6d7feb9a741 100644

Documentation/devicetree/bindings/mmc/sdhci.txt Optional properties: · line 17

Quoted 7 lines
Documentation/devicetree/bindings/mmc/sdhci.txt +4 -0
@@ -11,3 +11,11 @@ Optional properties:
 - sdhci-caps: The sdhci capabilities register is incorrect. This 64bit
   property corresponds to the bits in the sdhci capability register. If the
   bit is on in the property then the bit should be turned on.
+- sdhci-dma-mem-boundary: The sdhci controller DMA memory space boundary.
+  If the controller's DMA cannot cross a specific memory space boundary,
+  such as 128MB, set this value in dt and driver will split the DMA
+  operation when crossing such boundary.

This should be implied by the compatible string.

Documentation/devicetree/bindings/mmc/sdhci.txt Optional properties: · line 21

Quoted 4 lines
Documentation/devicetree/bindings/mmc/sdhci.txt +4 -0
+- sdhci-ctrl-hs400: The HS400 is not defined in SDHC Standard specification
+  for SDHCI_HOST_CONTROL2(offset 0x3E:bit[2:0]). Different controllers have
+  have different value for HS400 mode. If 0x5 is not the HS400 mode value
+  for your controller, you should specify the value with this property.
Quoted 4 lines

This too, unless it needs to be tuned per board.

Can you be more specific as to what the possible values are and what they do?

It is specific to SoC or specific to controller. HS400 mode value on DWC3 of new Hisilicon
SoC is 7, not 5. This same is to DMA buffer memory address boundary.
Do you mean you
want the boundary value and HS400 mode value should bundled with compatible, ie. specific SoC or controller, to set a value in sdhci layer from platform glue driver?

Quoted 2 lines

Rob

author
#6 lore raw

PATCH 3/4 mmc: sdhci: Set ctrl_hs400 value in dts

In reply to Jun Nie

Because ctrl_hs400 value is non-standard, add support to set it in dts.

---
 drivers/mmc/host/sdhci.c | 9 ++++++++-
 drivers/mmc/host/sdhci.h | 6 +++++-
 2 files changed, 13 insertions(+), 2 deletions(-)
drivers/mmc/host/sdhci.c +8 -1
index a654f0aeb438..d8a6c1c91448 100644
@@ -1899,7 +1899,7 @@ void sdhci_set_uhs_signaling(struct sdhci_host *host, unsigned timing)
 		 (timing == MMC_TIMING_MMC_DDR52))
 		ctrl_2 |= SDHCI_CTRL_UHS_DDR50;
 	else if (timing == MMC_TIMING_MMC_HS400)
-		ctrl_2 |= SDHCI_CTRL_HS400; /* Non-standard */
+		ctrl_2 |= host->sdhci_ctrl_hs400; /* Non-standard */
 	sdhci_writew(host, ctrl_2, SDHCI_HOST_CONTROL2);
 }
 EXPORT_SYMBOL_GPL(sdhci_set_uhs_signaling);
@@ -3635,6 +3635,13 @@ void __sdhci_read_caps(struct sdhci_host *host, const u16 *ver,
 	of_property_read_u64(mmc_dev(host->mmc)->of_node,
 			     "sdhci-caps", &dt_caps);
 
+	if (of_property_read_u32(mmc_dev(host->mmc)->of_node,
+				 "sdhci-ctrl-hs400", &host->sdhci_ctrl_hs400))
+		host->sdhci_ctrl_hs400 = SDHCI_CTRL_HS400;
+	else
+		WARN_ON(host->sdhci_ctrl_hs400 > 7
+			|| host->sdhci_ctrl_hs400 < 5);
+
 	v = ver ? *ver : sdhci_readw(host, SDHCI_HOST_VERSION);
 	host->version = (v & SDHCI_SPEC_VER_MASK) >> SDHCI_SPEC_VER_SHIFT;
 
drivers/mmc/host/sdhci.h +5 -1
index 0e6f97eaa796..cac4d819f62c 100644
@@ -184,7 +184,8 @@
 #define   SDHCI_CTRL_UHS_SDR50		0x0002
 #define   SDHCI_CTRL_UHS_SDR104		0x0003
 #define   SDHCI_CTRL_UHS_DDR50		0x0004
-#define   SDHCI_CTRL_HS400		0x0005 /* Non-standard */
+/* SDHCI_CTRL_HS400 is non-standard value, can change it in dts */
+#define   SDHCI_CTRL_HS400		0x0005
 #define  SDHCI_CTRL_VDD_180		0x0008
 #define  SDHCI_CTRL_DRV_TYPE_MASK	0x0030
 #define   SDHCI_CTRL_DRV_TYPE_B		0x0000
@@ -605,6 +606,9 @@ struct sdhci_host {
 
 	u64			data_timeout;
 
+	/* SDHCI_CTRL_HS400 value */
+	u32			sdhci_ctrl_hs400;
+
 	unsigned long private[0] ____cacheline_aligned;
 };
 
Signature · 1 line
2.17.1
maintainer
#7 lore raw

Re: [PATCH 3/4] mmc: sdhci: Set ctrl_hs400 value in dts

Quoted 14 lines · Jun Nie

Because ctrl_hs400 value is non-standard, add support to set it in dts.

---
 drivers/mmc/host/sdhci.c | 9 ++++++++-
 drivers/mmc/host/sdhci.h | 6 +++++-
 2 files changed, 13 insertions(+), 2 deletions(-)
drivers/mmc/host/sdhci.c +0 -0
index a654f0aeb438..d8a6c1c91448 100644

drivers/mmc/host/sdhci.c void sdhci_set_uhs_signaling(struct sdhci_host *host, unsigned timing) · line 1902

Quoted 5 lines
drivers/mmc/host/sdhci.c +1 -1
@@ -1899,7 +1899,7 @@ void sdhci_set_uhs_signaling(struct sdhci_host *host, unsigned timing)
 		 (timing == MMC_TIMING_MMC_DDR52))
 		ctrl_2 |= SDHCI_CTRL_UHS_DDR50;
 	else if (timing == MMC_TIMING_MMC_HS400)
-		ctrl_2 |= SDHCI_CTRL_HS400; /* Non-standard */
+		ctrl_2 |= host->sdhci_ctrl_hs400; /* Non-standard */

Let's limit this to the correct bits i.e.

		ctrl_2 |= host->sdhci_ctrl_hs400 & 7; /* Non-standard */

drivers/mmc/host/sdhci.c void __sdhci_read_caps(struct sdhci_host *host, const u16 *ver, · line 3636

Quoted 6 lines
drivers/mmc/host/sdhci.c +0 -0
 	sdhci_writew(host, ctrl_2, SDHCI_HOST_CONTROL2);
 }
 EXPORT_SYMBOL_GPL(sdhci_set_uhs_signaling);
@@ -3635,6 +3635,13 @@ void __sdhci_read_caps(struct sdhci_host *host, const u16 *ver,
 	of_property_read_u64(mmc_dev(host->mmc)->of_node,
 			     "sdhci-caps", &dt_caps);

Would you mind changing these APIs from 'of' to 'dev' properties? Needs to be a separate patch.

drivers/mmc/host/sdhci.c void __sdhci_read_caps(struct sdhci_host *host, const u16 *ver, · line 3639

Quoted 3 lines
drivers/mmc/host/sdhci.c +2 -0
+	if (of_property_read_u32(mmc_dev(host->mmc)->of_node,
+				 "sdhci-ctrl-hs400", &host->sdhci_ctrl_hs400))

And then use a 'dev' property API here.

drivers/mmc/host/sdhci.c void __sdhci_read_caps(struct sdhci_host *host, const u16 *ver, · line 3643

Quoted 4 lines
drivers/mmc/host/sdhci.c +4 -0
+		host->sdhci_ctrl_hs400 = SDHCI_CTRL_HS400;
+	else
+		WARN_ON(host->sdhci_ctrl_hs400 > 7
+			|| host->sdhci_ctrl_hs400 < 5);

Please remove this warning.

Quoted 29 lines
drivers/mmc/host/sdhci.c +1 -0
+
 	v = ver ? *ver : sdhci_readw(host, SDHCI_HOST_VERSION);
 	host->version = (v & SDHCI_SPEC_VER_MASK) >> SDHCI_SPEC_VER_SHIFT;
 
drivers/mmc/host/sdhci.h +5 -1
index 0e6f97eaa796..cac4d819f62c 100644
@@ -184,7 +184,8 @@
 #define   SDHCI_CTRL_UHS_SDR50		0x0002
 #define   SDHCI_CTRL_UHS_SDR104		0x0003
 #define   SDHCI_CTRL_UHS_DDR50		0x0004
-#define   SDHCI_CTRL_HS400		0x0005 /* Non-standard */
+/* SDHCI_CTRL_HS400 is non-standard value, can change it in dts */
+#define   SDHCI_CTRL_HS400		0x0005
 #define  SDHCI_CTRL_VDD_180		0x0008
 #define  SDHCI_CTRL_DRV_TYPE_MASK	0x0030
 #define   SDHCI_CTRL_DRV_TYPE_B		0x0000
@@ -605,6 +606,9 @@ struct sdhci_host {
 
 	u64			data_timeout;
 
+	/* SDHCI_CTRL_HS400 value */
+	u32			sdhci_ctrl_hs400;
+
 	unsigned long private[0] ____cacheline_aligned;
 };
 
author
#8 lore raw

PATCH 4/4 mmc: sdhci: Add DMA memory boundary workaround

In reply to Jun Nie

DMA memory cannot cross specific boundary for some SDHCI controller, such as DesignWare SDHCI controller. Add DMA memory boundary dt property and workaround the limitation.

---
 drivers/mmc/host/sdhci.c | 20 +++++++++++++++++++-
 drivers/mmc/host/sdhci.h |  1 +
 2 files changed, 20 insertions(+), 1 deletion(-)
drivers/mmc/host/sdhci.c +19 -1
index d8a6c1c91448..56c53fbadd9d 100644
@@ -763,9 +763,25 @@ static void sdhci_adma_table_pre(struct sdhci_host *host,
 		BUG_ON(len > 65536);
 
 		/* tran, valid */
-		if (len)
+		if (len) {
+			unsigned int boundary = host->dma_mem_boundary;
+			/*
+			 * work around for buffer across mem boundary, split
+			 * the buffer.
+			 */
+			if (boundary &&
+			    ((addr & (boundary - 1)) + len) > boundary) {
+				offset = boundary - (addr & (boundary - 1));
+				__sdhci_adma_write_desc(host, &desc,
+							addr, offset,
+							ADMA2_TRAN_VALID);
+				addr += offset;
+				len -= offset;
+			}
+
 			__sdhci_adma_write_desc(host, &desc, addr, len,
 						ADMA2_TRAN_VALID);
+		}
 
 		/*
 		 * If this triggers then we have a calculation bug
@@ -3634,6 +3650,8 @@ void __sdhci_read_caps(struct sdhci_host *host, const u16 *ver,
 			     "sdhci-caps-mask", &dt_caps_mask);
 	of_property_read_u64(mmc_dev(host->mmc)->of_node,
 			     "sdhci-caps", &dt_caps);
+	of_property_read_u32(mmc_dev(host->mmc)->of_node,
+			     "sdhci-dma-mem-boundary", &host->dma_mem_boundary);
 
 	if (of_property_read_u32(mmc_dev(host->mmc)->of_node,
 				 "sdhci-ctrl-hs400", &host->sdhci_ctrl_hs400))
drivers/mmc/host/sdhci.h +1 -0
index cac4d819f62c..954ac08c4fb0 100644
@@ -608,6 +608,7 @@ struct sdhci_host {
 
 	/* SDHCI_CTRL_HS400 value */
 	u32			sdhci_ctrl_hs400;
+	u32			dma_mem_boundary;
 
 	unsigned long private[0] ____cacheline_aligned;
 };
Signature · 1 line
2.17.1
#9 lore raw

Re: [PATCH 4/4] mmc: sdhci: Add DMA memory boundary workaround

Quoted 3 lines · Jun Nie

DMA memory cannot cross specific boundary for some SDHCI controller, such as DesignWare SDHCI controller. Add DMA memory boundary dt property and workaround the limitation.

If you use blk_queue_segment_boundary to tell the block layer the segment boundary restrictions it won't ever send you segments that don't fit. With just the workaround in this patch you'll run into max_segments accounting issues, don't you?

author
#10 lore raw

Re: [PATCH 4/4] mmc: sdhci: Add DMA memory boundary workaround

Quoted 10 lines · Christoph Hellwig <[email protected]> 于2019年12月3日周二 上午1:52写道:
Quoted 3 lines · Jun Nie

DMA memory cannot cross specific boundary for some SDHCI controller, such as DesignWare SDHCI controller. Add DMA memory boundary dt property and workaround the limitation.

If you use blk_queue_segment_boundary to tell the block layer the segment boundary restrictions it won't ever send you segments that don't fit. With just the workaround in this patch you'll run into max_segments accounting issues, don't you?

Thanks for the reminder! So I need to parse the segment_boundary from device tree and use below code to set it, right?
For the max_segments accounting error, I did not see it so far though I believe it is true in theory. Maybe it is due to segment boundary value is very large.

+++ b/drivers/mmc/core/queue.c
@@ -374,6 +374,7 @@ static void mmc_setup_queue(struct mmc_queue *mq, struct mmc_card *card)

                WARN(!blk_queue_can_use_dma_map_merging(mq->queue,
                                                        mmc_dev(host)),
                     "merging was advertised but not possible");
+       blk_queue_segment_boundary(mq->queue, mmc->segment_boundary);
        blk_queue_max_segments(mq->queue, mmc_get_max_segments(host));
#11 lore raw

Re: [PATCH 4/4] mmc: sdhci: Add DMA memory boundary workaround

Quoted 8 lines · Jun Nie

Thanks for the reminder! So I need to parse the segment_boundary from device tree and use below code to set it, right?
For the max_segments accounting error, I did not see it so far though I believe it is true in theory. Maybe it is due to segment boundary value is very large.

+++ b/drivers/mmc/core/queue.c

Review static void mmc_setup_queue(struct mmc_queue *mq, · line 379

Quoted 6 lines
(quoted diff) +1 -0
@@ -374,6 +374,7 @@ static void mmc_setup_queue(struct mmc_queue *mq,
 struct mmc_card *card)
                WARN(!blk_queue_can_use_dma_map_merging(mq->queue,
                                                        mmc_dev(host)),
                     "merging was advertised but not possible");
+       blk_queue_segment_boundary(mq->queue, mmc->segment_boundary);
        blk_queue_max_segments(mq->queue, mmc_get_max_segments(host));

Yes, I think should do it. Maybe modulo a check if the low-level driver actually sets a segment boundary.

author
#12 lore raw

Re: [PATCH 4/4] mmc: sdhci: Add DMA memory boundary workaround

Quoted 8 lines · Christoph Hellwig <[email protected]> 于2019年12月3日周二 下午3:36写道:
Quoted 6 lines · Jun Nie

Thanks for the reminder! So I need to parse the segment_boundary from device tree and use below code to set it, right?
For the max_segments accounting error, I did not see it so far though I believe it is true in theory. Maybe it is due to segment boundary value is very large.

Quoted 11 lines

Review static void mmc_setup_queue(struct mmc_queue *mq, · line 379

Quoted 8 lines

+++ b/drivers/mmc/core/queue.c

(quoted diff) +1 -0
@@ -374,6 +374,7 @@ static void mmc_setup_queue(struct mmc_queue *mq,
 struct mmc_card *card)
                WARN(!blk_queue_can_use_dma_map_merging(mq->queue,
                                                        mmc_dev(host)),
                     "merging was advertised but not possible");
+       blk_queue_segment_boundary(mq->queue, mmc->segment_boundary);
        blk_queue_max_segments(mq->queue, mmc_get_max_segments(host));

Yes, I think should do it. Maybe modulo a check if the low-level driver actually sets a segment boundary.

For the block device, such as SD card, it is right solution. But I have concern on SDIO case. Maybe we should add workaround together with block layer segment boundary restriction. How do you think about it?

Jun

#13 lore raw

Re: [PATCH 4/4] mmc: sdhci: Add DMA memory boundary workaround

Quoted 23 lines · On Wed, 4 Dec 2019 14:00:08 +0800 Jun Nie wrote:
Quoted 8 lines · Christoph Hellwig <[email protected]> 于2019年12月3日周二 下午3:36写道:
Quoted 6 lines · Jun Nie

Thanks for the reminder! So I need to parse the segment_boundary from device tree and use below code to set it, right?
For the max_segments accounting error, I did not see it so far though I believe it is true in theory. Maybe it is due to segment boundary value is very large.

Quoted 11 lines

Review static void mmc_setup_queue(struct mmc_queue *mq, · line 379

Quoted 8 lines

+++ b/drivers/mmc/core/queue.c

(quoted diff) +1 -0
@@ -374,6 +374,7 @@ static void mmc_setup_queue(struct mmc_queue *mq,
 struct mmc_card *card)
                WARN(!blk_queue_can_use_dma_map_merging(mq->queue,
                                                        mmc_dev(host)),
                     "merging was advertised but not possible");
+       blk_queue_segment_boundary(mq->queue, mmc->segment_boundary);
        blk_queue_max_segments(mq->queue, mmc_get_max_segments(host));  

Yes, I think should do it. Maybe modulo a check if the low-level driver actually sets a segment boundary.

Quoted 5 lines

For the block device, such as SD card, it is right solution. But I have concern on SDIO case. Maybe we should add workaround together with block layer segment boundary restriction. How do you think about it?

Another trouble is how to workaround if the sg is constructed by mmc and no block layer interactions at all. e.g __mmc_blk_ioctl_cmd(), and all those sgs in mmc_test.c

Thanks

#14 lore raw

Re: [PATCH 4/4] mmc: sdhci: Add DMA memory boundary workaround

Quoted 26 lines · Jisheng Zhang
Quoted 23 lines · On Wed, 4 Dec 2019 14:00:08 +0800 Jun Nie wrote:
Quoted 19 lines · Christoph Hellwig <[email protected]> 于2019年12月3日周二 下午3:36写道:
Quoted 8 lines · Jun Nie

Thanks for the reminder! So I need to parse the segment_boundary from device tree and use below code to set it, right?
For the max_segments accounting error, I did not see it so far though I believe it is true in theory. Maybe it is due to segment boundary value is very large.

+++ b/drivers/mmc/core/queue.c

Review static void mmc_setup_queue(struct mmc_queue *mq, · line 379

Quoted 6 lines
(quoted diff) +1 -0
@@ -374,6 +374,7 @@ static void mmc_setup_queue(struct mmc_queue *mq,
 struct mmc_card *card)
                WARN(!blk_queue_can_use_dma_map_merging(mq->queue,
                                                        mmc_dev(host)),
                     "merging was advertised but not possible");
+       blk_queue_segment_boundary(mq->queue, mmc->segment_boundary);
        blk_queue_max_segments(mq->queue, mmc_get_max_segments(host));

Yes, I think should do it. Maybe modulo a check if the low-level driver actually sets a segment boundary.

Quoted 5 lines
Quoted 5 lines

For the block device, such as SD card, it is right solution. But I have concern on SDIO case. Maybe we should add workaround together with block layer segment boundary restriction. How do you think about it?

Yes, buffers for SDIO are a consern. Especially since those buffers are allocated by SDIO func drivers or even from upper layers, such as the network stacks, for example.

I think SDIO func drivers simply need to respect the constraints the host has set via the .segment_boundary, .max_seg_size, .max_segs, etc. We should export SDIO func APIs to make the information available for the SDIO func drivers.

Quoted 4 lines

Another trouble is how to workaround if the sg is constructed by mmc and no block layer interactions at all. e.g __mmc_blk_ioctl_cmd(), and all those sgs in mmc_test.c

Those should be easier to fix, as the buffer/sg allocation can be fixed internally by mmc core. Just post some patches. :-)

I am more worried about SDIO, as those buffers are not that easy to control.

Note that, there have been suggestions on adding an SDIO interface where an sg can be passed [1]. Unfurtunate those patches got stuck and didn't make it.

Kind regards
Uffe

[1]
https://patchwork.kernel.org/patch/10123143/

#15 lore raw

Re: [PATCH 4/4] mmc: sdhci: Add DMA memory boundary workaround

In reply to Jun Nie
Quoted 5 lines · On Mon, 2 Dec 2019 22:41:04 +0800 Jun Nie wrote:

DMA memory cannot cross specific boundary for some SDHCI controller, such as DesignWare SDHCI controller. Add DMA memory boundary dt property and workaround the limitation.

IMHO, the workaround could be implemented in each SDHCI host driver.

eg. drivers/mmc/host/sdhci-of-dwcmshc.c

thanks

Quoted 62 lines
---
 drivers/mmc/host/sdhci.c | 20 +++++++++++++++++++-
 drivers/mmc/host/sdhci.h |  1 +
 2 files changed, 20 insertions(+), 1 deletion(-)
drivers/mmc/host/sdhci.c +19 -1
index d8a6c1c91448..56c53fbadd9d 100644
@@ -763,9 +763,25 @@ static void sdhci_adma_table_pre(struct sdhci_host *host,
                BUG_ON(len > 65536);
 
                /* tran, valid */
-               if (len)
+               if (len) {
+                       unsigned int boundary = host->dma_mem_boundary;
+                       /*
+                        * work around for buffer across mem boundary, split
+                        * the buffer.
+                        */
+                       if (boundary &&
+                           ((addr & (boundary - 1)) + len) > boundary) {
+                               offset = boundary - (addr & (boundary - 1));
+                               __sdhci_adma_write_desc(host, &desc,
+                                                       addr, offset,
+                                                       ADMA2_TRAN_VALID);
+                               addr += offset;
+                               len -= offset;
+                       }
+
                        __sdhci_adma_write_desc(host, &desc, addr, len,
                                                ADMA2_TRAN_VALID);
+               }
 
                /*
                 * If this triggers then we have a calculation bug
@@ -3634,6 +3650,8 @@ void __sdhci_read_caps(struct sdhci_host *host, const u16 *ver,
                             "sdhci-caps-mask", &dt_caps_mask);
        of_property_read_u64(mmc_dev(host->mmc)->of_node,
                             "sdhci-caps", &dt_caps);
+       of_property_read_u32(mmc_dev(host->mmc)->of_node,
+                            "sdhci-dma-mem-boundary", &host->dma_mem_boundary);
 
        if (of_property_read_u32(mmc_dev(host->mmc)->of_node,
                                 "sdhci-ctrl-hs400", &host->sdhci_ctrl_hs400))
drivers/mmc/host/sdhci.h +1 -1
index cac4d819f62c..954ac08c4fb0 100644
@@ -608,6 +608,7 @@ struct sdhci_host {
 
        /* SDHCI_CTRL_HS400 value */
        u32                     sdhci_ctrl_hs400;
+       u32                     dma_mem_boundary;
 
        unsigned long private[0] ____cacheline_aligned;
 };
--

2.17.1

author
#16 lore raw

Re: [PATCH 4/4] mmc: sdhci: Add DMA memory boundary workaround

Quoted 6 lines · Jisheng Zhang <[email protected]> 于2019年12月3日周二 上午10:47写道:
Quoted 2 lines · On Mon, 2 Dec 2019 22:41:04 +0800 Jun Nie wrote:
Quoted 8 lines
Quoted 3 lines

DMA memory cannot cross specific boundary for some SDHCI controller, such as DesignWare SDHCI controller. Add DMA memory boundary dt property and workaround the limitation.

IMHO, the workaround could be implemented in each SDHCI host driver.

eg. drivers/mmc/host/sdhci-of-dwcmshc.c

Thanks for the suggestion! Christoph's suggestion can prevent the the issue from the block layer, thus the code can be shared across all controllers. I prefer
his suggestions.

Jun

#17 lore raw

Re: [PATCH 4/4] mmc: sdhci: Add DMA memory boundary workaround

+ Christoph

Quoted 13 lines · On Tue, 3 Dec 2019 11:33:15 +0800 Jun Nie wrote:
Quoted 10 lines · Jisheng Zhang <[email protected]> 于2019年12月3日周二 上午10:47写道:
Quoted 5 lines · On Mon, 2 Dec 2019 22:41:04 +0800 Jun Nie wrote:

DMA memory cannot cross specific boundary for some SDHCI controller, such as DesignWare SDHCI controller. Add DMA memory boundary dt property and workaround the limitation.

Quoted 6 lines
Quoted 4 lines

IMHO, the workaround could be implemented in each SDHCI host driver.

eg. drivers/mmc/host/sdhci-of-dwcmshc.c

Thanks for the suggestion! Christoph's suggestion can prevent the the issue from the block layer, thus the code can be shared across all

To be honest, I did consider similar solution from block layer, I.E set the seg_boundary_mask, when submitting the workaround last year, but per my understanding, SDHCI limitation is the physical DMA addr can't span one specific boundary, so setting seg_boundary_mask w/ blk_queue_segment_boundary can't work. I'm not sure I understand blk_queue_segment_boundary() properly. May Christoph help to clarify?

From another side, drivers/ata/libata-sff.c also workaround the 64K phy DMA boundary limitation itself rather than from block layer.

Thanks

Quoted 4 lines

controllers. I prefer
his suggestions.

Jun

#18 lore raw

Re: [PATCH 4/4] mmc: sdhci: Add DMA memory boundary workaround

Quoted 10 lines · Jisheng Zhang
Quoted 5 lines
Quoted 3 lines

eg. drivers/mmc/host/sdhci-of-dwcmshc.c

Thanks for the suggestion! Christoph's suggestion can prevent the the issue from the block layer, thus the code can be shared across all

To be honest, I did consider similar solution from block layer, I.E set the seg_boundary_mask, when submitting the workaround last year, but per my understanding, SDHCI limitation is the physical DMA addr can't span one specific boundary,

As in exactly one boundary and not an alignment? Where the one boundary is not a power of two and thus can't be expressed?

Quoted 6 lines

so setting seg_boundary_mask w/ blk_queue_segment_boundary
can't work. I'm not sure I understand blk_queue_segment_boundary() properly. May Christoph help to clarify?

From another side, drivers/ata/libata-sff.c also workaround the 64K phy DMA boundary limitation itself rather than from block layer.

As far as I can tell that workaround should use the segment boundary setting as well.

#19 lore raw

Re: [PATCH 4/4] mmc: sdhci: Add DMA memory boundary workaround

Quoted 9 lines · On Tue, 3 Dec 2019 01:18:24 -0800 Christoph Hellwig <[email protected]> wrote:
Quoted 6 lines · Jisheng Zhang
Quoted 5 lines
Quoted 3 lines

eg. drivers/mmc/host/sdhci-of-dwcmshc.c

Thanks for the suggestion! Christoph's suggestion can prevent the the issue from the block layer, thus the code can be shared across all

Quoted 7 lines
Quoted 4 lines

To be honest, I did consider similar solution from block layer, I.E set the seg_boundary_mask, when submitting the workaround last year, but per my understanding, SDHCI limitation is the physical DMA addr can't span one specific boundary,

As in exactly one boundary and not an alignment? Where the one boundary is not a power of two and thus can't be expressed?

Take drivers/mmc/host/sdhci-of-dwcmshc.c for example, target physical DMA addr can't span 128MB, 256MB, 128*3MB, ...128*nMB

I'm not sure whether blk_queue_segment_boundary could solve this limitation.

Quoted 11 lines
Quoted 8 lines

so setting seg_boundary_mask w/ blk_queue_segment_boundary
can't work. I'm not sure I understand blk_queue_segment_boundary() properly. May Christoph help to clarify?

From another side, drivers/ata/libata-sff.c also workaround the 64K phy DMA boundary limitation itself rather than from block layer.

As far as I can tell that workaround should use the segment boundary setting as well.

#20 lore raw

Re: [PATCH 4/4] mmc: sdhci: Add DMA memory boundary workaround

Quoted 7 lines · Jisheng Zhang
Quoted 2 lines

As in exactly one boundary and not an alignment? Where the one boundary is not a power of two and thus can't be expressed?

Take drivers/mmc/host/sdhci-of-dwcmshc.c for example, target physical DMA addr can't span 128MB, 256MB, 128*3MB, ...128*nMB

I'm not sure whether blk_queue_segment_boundary could solve this limitation.

That is exaxtly the kind of limitation blk_queue_segment_boundary is intended for.

#21 lore raw

Re: [PATCH 4/4] mmc: sdhci: Add DMA memory boundary workaround

Hi Christoph

Quoted 5 lines · On Tue, 3 Dec 2019 05:06:09 -0800 Christoph Hellwig wrote:
Quoted 3 lines · Jisheng Zhang
Quoted 2 lines

As in exactly one boundary and not an alignment? Where the one boundary is not a power of two and thus can't be expressed?

Quoted 7 lines
Quoted 4 lines

Take drivers/mmc/host/sdhci-of-dwcmshc.c for example, target physical DMA addr can't span 128MB, 256MB, 128*3MB, ...128*nMB

I'm not sure whether blk_queue_segment_boundary could solve this limitation.

That is exaxtly the kind of limitation blk_queue_segment_boundary is intended for.

Until after dma_map_sg(), we can't know the physical DMA address range, so how does block layer know and check the DMA range beforehand? I'm a newbie on block layer, could you please teach me? At the same time
I'm reading the code as well.

Thanks in advance

#22 lore raw

Re: [PATCH 4/4] mmc: sdhci: Add DMA memory boundary workaround

In reply to Jisheng Zhang
Quoted 25 lines · On Tue, 3 Dec 2019 09:05:23 +0000 Jisheng Zhang wrote:

+ Christoph

Quoted 13 lines · On Tue, 3 Dec 2019 11:33:15 +0800 Jun Nie wrote:
Quoted 10 lines · Jisheng Zhang <[email protected]> 于2019年12月3日周二 上午10:47写道:
Quoted 5 lines · On Mon, 2 Dec 2019 22:41:04 +0800 Jun Nie wrote:

DMA memory cannot cross specific boundary for some SDHCI controller, such as DesignWare SDHCI controller. Add DMA memory boundary dt property and workaround the limitation.

Quoted 6 lines
Quoted 4 lines

IMHO, the workaround could be implemented in each SDHCI host driver.

eg. drivers/mmc/host/sdhci-of-dwcmshc.c

Thanks for the suggestion! Christoph's suggestion can prevent the the issue from the block layer, thus the code can be shared across all

Quoted 6 lines

To be honest, I did consider similar solution from block layer, I.E set the seg_boundary_mask, when submitting the workaround last year, but per my understanding, SDHCI limitation is the physical DMA addr can't span one specific boundary, so setting seg_boundary_mask w/ blk_queue_segment_boundary can't work. I'm not sure I understand blk_queue_segment_boundary() properly. May Christoph help to clarify?

what's more, not all scatterlist in mmc are from block layer, for example, __mmc_blk_ioctl_cmd(), mmc_test.c etc..

How do we ensure the boundary is fine in these cases?

Quoted 11 lines

From another side, drivers/ata/libata-sff.c also workaround the 64K phy DMA boundary limitation itself rather than from block layer.

Thanks

Quoted 4 lines

controllers. I prefer
his suggestions.

Jun

#23 lore raw

Re: [PATCH 4/4] mmc: sdhci: Add DMA memory boundary workaround

Quoted 7 lines · Jisheng Zhang
Quoted 2 lines

can't work. I'm not sure I understand blk_queue_segment_boundary() properly. May Christoph help to clarify?

what's more, not all scatterlist in mmc are from block layer, for example, __mmc_blk_ioctl_cmd(), mmc_test.c etc..

How do we ensure the boundary is fine in these cases?

By using block layer passthrough requests (REQ_OP_DRV*). Otherwise any other I/O limit imposed by the driver is ignored as well.