Skip to contentSkip to conversation

mtd: gpmi: serialize all the dma operations

PATCH New Huang Shijie 3 participants 9 messages Nov 6, 2013 – Nov 12, 2013

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 mtd: gpmi: serialize all the dma operations

[1] The gpmi uses the nand_command_lp to issue the commands to NAND chips. It will issue a DMA operation when it handles a NAND_CMD_NONE control command. So when we read a page(NAND_CMD_READ0) from the NAND, we may send two DMA operations back-to-back.

If we do not serialize the two DMA operations, we will meet a bug when

1.1) we enable CONFIG_DMA_API_DEBUG, CONFIG_DMADEVICES_DEBUG,
and CONFIG_DEBUG_SG.

1.2) Use the following commands in an UART console and a SSH console:
cmd 1: while true;do dd if=/dev/mtd0 of=/dev/null;done cmd 1: while true;do dd if=/dev/mmcblk0 of=/dev/null;done

The kernel log shows below:
-----------------------------------------------------------------
kernel BUG at lib/scatterlist.c:28!
Unable to handle kernel NULL pointer dereference at virtual address 00000000
.........................
[<80044a0c>] (__bug+0x18/0x24) from [<80249b74>] (sg_next+0x48/0x4c) [<80249b74>] (sg_next+0x48/0x4c) from [<80255398>] (debug_dma_unmap_sg+0x170/0x1a4) [<80255398>] (debug_dma_unmap_sg+0x170/0x1a4) from [<8004af58>] (dma_unmap_sg+0x14/0x6c) [<8004af58>] (dma_unmap_sg+0x14/0x6c) from [<8027e594>] (mxs_dma_tasklet+0x18/0x1c) [<8027e594>] (mxs_dma_tasklet+0x18/0x1c) from [<8007d444>] (tasklet_action+0x114/0x164)
-----------------------------------------------------------------

1.3) Assume the two DMA operations is X (first) and Y (second).
The root cause of the bug:
X's tasklet mxs_dma_tasklet trid to unmap the scatterlist, while Y is trying to set up a new DMA operation with the _SAME_ scatterlist in another ARM core.

[2] This patch adds a wait queue and two helpers gpmi_enter_dma/gpmi_exit_dma to serialize all the DMA operations.

Cc stable at vger.kernel.org
---
 drivers/mtd/nand/gpmi-nand/gpmi-nand.c |   24 ++++++++++++++++++++++++
 drivers/mtd/nand/gpmi-nand/gpmi-nand.h |    2 ++
 2 files changed, 26 insertions(+), 0 deletions(-)
drivers/mtd/nand/gpmi-nand/gpmi-nand.c +24 -0
index 71df69e..b849b92 100644
@@ -392,6 +392,20 @@ void prepare_data_dma(struct gpmi_nand_data *this, enum dma_data_direction dr)
 	}
 }
 
+static void gpmi_enter_dma(struct gpmi_nand_data *this)
+{
+	/* Wait until the previous DMA is finished. */
+	wait_event(this->dma_wait, !this->dma_is_working);
+
+	this->dma_is_working = true;
+}
+
+static void gpmi_exit_dma(struct gpmi_nand_data *this)
+{
+	this->dma_is_working = false;
+	wake_up(&this->dma_wait);
+}
+
 /* This will be called after the DMA operation is finished. */
 static void dma_irq_callback(void *param)
 {
@@ -424,6 +438,7 @@ static void dma_irq_callback(void *param)
 	default:
 		pr_err("in wrong DMA operation.\n");
 	}
+	gpmi_exit_dma(this);
 }
 
 int start_dma_without_bch_irq(struct gpmi_nand_data *this,
@@ -906,6 +921,8 @@ static void gpmi_cmd_ctrl(struct mtd_info *mtd, int data, unsigned int ctrl)
 	if (!this->command_length)
 		return;
 
+	gpmi_enter_dma(this);
+
 	ret = gpmi_send_command(this);
 	if (ret)
 		pr_err("Chip: %u, Error %d\n", this->current_chip, ret);
@@ -943,6 +960,8 @@ static void gpmi_read_buf(struct mtd_info *mtd, uint8_t *buf, int len)
 	this->upper_buf	= buf;
 	this->upper_len	= len;
 
+	gpmi_enter_dma(this);
+
 	gpmi_read_data(this);
 }
 
@@ -955,6 +974,8 @@ static void gpmi_write_buf(struct mtd_info *mtd, const uint8_t *buf, int len)
 	this->upper_buf	= (uint8_t *)buf;
 	this->upper_len	= len;
 
+	gpmi_enter_dma(this);
+
 	gpmi_send_data(this);
 }
 
@@ -1031,6 +1052,7 @@ static int gpmi_ecc_read_page(struct mtd_info *mtd, struct nand_chip *chip,
 	int           ret;
 
 	pr_debug("page number is : %d\n", page);
+	gpmi_enter_dma(this);
 	ret = read_page_prepare(this, buf, mtd->writesize,
 					this->payload_virt, this->payload_phys,
 					nfc_geo->payload_size,
@@ -1107,6 +1129,7 @@ static int gpmi_ecc_write_page(struct mtd_info *mtd, struct nand_chip *chip,
 	int        ret;
 
 	pr_debug("ecc write page.\n");
+	gpmi_enter_dma(this);
 	if (this->swap_block_mark) {
 		/*
 		 * If control arrives here, we're doing block mark swapping.
@@ -1745,6 +1768,7 @@ static int gpmi_nand_probe(struct platform_device *pdev)
 	platform_set_drvdata(pdev, this);
 	this->pdev  = pdev;
 	this->dev   = &pdev->dev;
+	init_waitqueue_head(&this->dma_wait);
 
 	ret = acquire_resources(this);
 	if (ret)
drivers/mtd/nand/gpmi-nand/gpmi-nand.h +2 -0
index a7685e3..9597615 100644
@@ -160,6 +160,8 @@ struct gpmi_nand_data {
 
 	/* for DMA operations */
 	bool			direct_dma_map_ok;
+	bool			dma_is_working;
+	wait_queue_head_t	dma_wait;
 
 	struct scatterlist	cmd_sgl;
 	char			*cmd_buffer;
Signature · 1 line
1.7.2.rc3
#1 lore raw

PATCH 1/1 mtd: gpmi: serialize all the dma operations

Hi Huang,

Quoted 5 lines · Huang Shijie

[1] The gpmi uses the nand_command_lp to issue the commands to NAND chips. It will issue a DMA operation when it handles a NAND_CMD_NONE control command. So when we read a page(NAND_CMD_READ0) from the NAND, we may send two DMA operations back-to-back.

Quoted 8 lines

If we do not serialize the two DMA operations, we will meet a bug when

1.1) we enable CONFIG_DMA_API_DEBUG, CONFIG_DMADEVICES_DEBUG,
and CONFIG_DEBUG_SG.

1.2) Use the following commands in an UART console and a SSH console:
cmd 1: while true;do dd if=/dev/mtd0 of=/dev/null;done cmd 1: while true;do dd if=/dev/mmcblk0 of=/dev/null;done

How does mmcblk0 have anything to do with the GPMI NAND DMA?

Quoted 12 lines

The kernel log shows below:
-----------------------------------------------------------------
kernel BUG at lib/scatterlist.c:28!
Unable to handle kernel NULL pointer dereference at virtual address 00000000
.........................
[<80044a0c>] (__bug+0x18/0x24) from [<80249b74>] (sg_next+0x48/0x4c) [<80249b74>] (sg_next+0x48/0x4c) from [<80255398>] (debug_dma_unmap_sg+0x170/0x1a4) [<80255398>] (debug_dma_unmap_sg+0x170/0x1a4) from [<8004af58>] (dma_unmap_sg+0x14/0x6c) [<8004af58>] (dma_unmap_sg+0x14/0x6c) from [<8027e594>] (mxs_dma_tasklet+0x18/0x1c) [<8027e594>] (mxs_dma_tasklet+0x18/0x1c) from [<8007d444>] (tasklet_action+0x114/0x164)
-----------------------------------------------------------------

Quoted 5 lines

1.3) Assume the two DMA operations is X (first) and Y (second).
The root cause of the bug:
X's tasklet mxs_dma_tasklet trid to unmap the scatterlist, while Y is trying to set up a new DMA operation with the _SAME_ scatterlist in another ARM core.

How are X and Y occurring concurrently? MTD/NAND has locking such that only one I/O operation is working on the chip at one time, right?

Quoted 2 lines

[2] This patch adds a wait queue and two helpers gpmi_enter_dma/gpmi_exit_dma to serialize all the DMA operations.

If you really need this serialization, wouldn't a spinlock or mutex suffice?

Quoted 2 lines
Cc stable at vger.kernel.org

Perhaps I'm missing some things, but I don't feel like the problem is sufficiently well described, and I'm not sure this is the right solution. But please, educate me.

Brian

#2 lore raw

PATCH 1/1 mtd: gpmi: serialize all the dma operations

? 2013?11?08? 05:12, Brian Norris ??:

Quoted 10 lines

Hi Huang,

Quoted 7 lines · Huang Shijie

[1] The gpmi uses the nand_command_lp to issue the commands to NAND chips. It will issue a DMA operation when it handles a NAND_CMD_NONE control command. So when we read a page(NAND_CMD_READ0) from the NAND, we may send two DMA operations back-to-back.

If we do not serialize the two DMA operations, we will meet a bug when

Quoted 7 lines
Quoted 6 lines

1.1) we enable CONFIG_DMA_API_DEBUG, CONFIG_DMADEVICES_DEBUG,
and CONFIG_DEBUG_SG.

1.2) Use the following commands in an UART console and a SSH console:
cmd 1: while true;do dd if=/dev/mtd0 of=/dev/null;done cmd 1: while true;do dd if=/dev/mmcblk0 of=/dev/null;done

How does mmcblk0 have anything to do with the GPMI NAND DMA?

Reading the mmcblk0 is just to heavy the arm core loading. Reading the mmcblk0 also revokes the
tasklets, and so make the tasklets of GPMI DMA not handled quickly enough.

Quoted 12 lines
Quoted 12 lines
     The kernel log shows below:
     -----------------------------------------------------------------
     kernel BUG at lib/scatterlist.c:28!
     Unable to handle kernel NULL pointer dereference at virtual address 00000000
       .........................
     [<80044a0c>] (__bug+0x18/0x24) from [<80249b74>] (sg_next+0x48/0x4c)
     [<80249b74>] (sg_next+0x48/0x4c) from [<80255398>] (debug_dma_unmap_sg+0x170/0x1a4)
     [<80255398>] (debug_dma_unmap_sg+0x170/0x1a4) from [<8004af58>] (dma_unmap_sg+0x14/0x6c)
     [<8004af58>] (dma_unmap_sg+0x14/0x6c) from [<8027e594>] (mxs_dma_tasklet+0x18/0x1c)
     [<8027e594>] (mxs_dma_tasklet+0x18/0x1c) from [<8007d444>] (tasklet_action+0x114/0x164)
     -----------------------------------------------------------------
Quoted 7 lines
Quoted 5 lines

1.3) Assume the two DMA operations is X (first) and Y (second).
The root cause of the bug:
X's tasklet mxs_dma_tasklet trid to unmap the scatterlist, while Y is trying to set up a new DMA operation with the _SAME_ scatterlist in another ARM core.

How are X and Y occurring concurrently? MTD/NAND has locking such that only one I/O operation is working on the chip at one time, right?

X and Y can not occur concurrently. X and Y is issue when the nand_command_lp calls the:
chip->cmd_ctrl(mtd, NAND_CMD_NONE, ....);

When we read a page from the NAND, the X and Y is issued back to back.

Quoted 5 lines
Quoted 2 lines

[2] This patch adds a wait queue and two helpers gpmi_enter_dma/gpmi_exit_dma to serialize all the DMA operations.

If you really need this serialization, wouldn't a spinlock or mutex suffice?

spinlock is too short for this case.

mutex_unlock can not be call in the interrupt context, such as the in the tasklet.

Quoted 6 lines
Quoted 2 lines
Cc stable at vger.kernel.org

Perhaps I'm missing some things, but I don't feel like the problem is sufficiently well described, and I'm not sure this is the right solution. But please, educate me.

should i add more description in the section of the root cause?

thanks
Huang Shijie

#3 lore raw

PATCH 1/1 mtd: gpmi: serialize all the dma operations

In reply to Brian Norris
Quoted 2 lines · Brian Norris

sufficiently well described, and I'm not sure this is the right solution. But please, educate me.

I find a better way to synchronize the two DMA operations. I will send out the new patch after i test it.

thanks
Huang Shijie

#4 lore raw

PATCH 1/1 mtd: gpmi: wake up the process at the end of the DMA callback

In reply to Huang Shijie

[1] The gpmi uses the nand_command_lp to issue the commands to NAND chips. The gpmi issues a DMA operation with gpmi_cmd_ctrl when it handles a NAND_CMD_NONE control command. So when we read a page(NAND_CMD_READ0) from the NAND, we may send two DMA operations back-to-back.

If we do not serialize the two DMA operations, we will meet a bug when

1.1) we enable CONFIG_DMA_API_DEBUG, CONFIG_DMADEVICES_DEBUG,
and CONFIG_DEBUG_SG.

1.2) Use the following commands in an UART console and a SSH console:
cmd 1: while true;do dd if=/dev/mtd0 of=/dev/null;done cmd 1: while true;do dd if=/dev/mmcblk0 of=/dev/null;done

The kernel log shows below:
-----------------------------------------------------------------
kernel BUG at lib/scatterlist.c:28!
Unable to handle kernel NULL pointer dereference at virtual address 00000000
.........................
[<80044a0c>] (__bug+0x18/0x24) from [<80249b74>] (sg_next+0x48/0x4c) [<80249b74>] (sg_next+0x48/0x4c) from [<80255398>] (debug_dma_unmap_sg+0x170/0x1a4) [<80255398>] (debug_dma_unmap_sg+0x170/0x1a4) from [<8004af58>] (dma_unmap_sg+0x14/0x6c) [<8004af58>] (dma_unmap_sg+0x14/0x6c) from [<8027e594>] (mxs_dma_tasklet+0x18/0x1c) [<8027e594>] (mxs_dma_tasklet+0x18/0x1c) from [<8007d444>] (tasklet_action+0x114/0x164)
-----------------------------------------------------------------

1.3) Assume the two DMA operations is X (first) and Y (second).

The root cause of the bug:
Assume process P issues DMA X, and sleep on the completion
@this->dma_done. X's tasklet callback is dma_irq_callback. It firstly wake up the process sleeping on the completion @this->dma_done, and then trid to unmap the scatterlist S. The waked process P will issue Y in another ARM core. Y initializes S->sg_magic to zero with sg_init_one(), while dma_irq_callback is unmapping S at the same time.

See the diagram:

ARM core 0 | ARM core 1
-------------------------------------------------------------
(P issues DMA X, then sleep) --> |
|
(X's tasklet wakes P) --> |
|
| <-- (P begin to issue DMA Y) |
(X's tasklet unmap the |
scatterlist S with dma_unmap_sg) --> | <-- (Y calls sg_init_one() to init
| scatterlist S)
|

[2] This patch serialize both the X and Y in the following way:

     Unmap the DMA scatterlist S firstly, and wake up the process at the end
     of the DMA callback, in such a way, Y will be executed after X.

     After this patch:

                   ARM core 0              |         ARM core 1
	 -------------------------------------------------------------
         (P issues DMA X, then sleep)  --> |
                                           |
         (X's tasklet wakes P)         --> |
                                           |
         (X's tasklet unmap the            |
      scatterlist S with dma_unmap_sg) --> |
                                           |
                                           | <-- (P begin to issue DMA Y)
                                           |
                                           | <-- (Y calls sg_init_one() to init
                                           |     scatterlist S)
                                           |
---
 drivers/mtd/nand/gpmi-nand/gpmi-nand.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)
drivers/mtd/nand/gpmi-nand/gpmi-nand.c +2 -2
index 4b6d802..f8dc560 100644
@@ -394,8 +394,6 @@ static void dma_irq_callback(void *param)
 	struct gpmi_nand_data *this = param;
 	struct completion *dma_c = &this->dma_done;
 
-	complete(dma_c);
-
 	switch (this->dma_type) {
 	case DMA_FOR_COMMAND:
 		dma_unmap_sg(this->dev, &this->cmd_sgl, 1, DMA_TO_DEVICE);
@@ -420,6 +418,8 @@ static void dma_irq_callback(void *param)
 	default:
 		pr_err("in wrong DMA operation.\n");
 	}
+
+	complete(dma_c);
 }
 
 int start_dma_without_bch_irq(struct gpmi_nand_data *this,
Signature · 1 line
1.7.4.4
#5 lore raw

PATCH 1/1 mtd: gpmi: wake up the process at the end of the DMA callback

[1] The gpmi uses the nand_command_lp to issue the commands to NAND chips. The gpmi issues a DMA operation with gpmi_cmd_ctrl when it handles a NAND_CMD_NONE control command. So when we read a page(NAND_CMD_READ0) from the NAND, we may send two DMA operations back-to-back.

If we do not serialize the two DMA operations, we will meet a bug when

1.1) we enable CONFIG_DMA_API_DEBUG, CONFIG_DMADEVICES_DEBUG,
and CONFIG_DEBUG_SG.

1.2) Use the following commands in an UART console and a SSH console:
cmd 1: while true;do dd if=/dev/mtd0 of=/dev/null;done cmd 1: while true;do dd if=/dev/mmcblk0 of=/dev/null;done

The kernel log shows below:
-----------------------------------------------------------------
kernel BUG at lib/scatterlist.c:28!
Unable to handle kernel NULL pointer dereference at virtual address 00000000
.........................
[<80044a0c>] (__bug+0x18/0x24) from [<80249b74>] (sg_next+0x48/0x4c) [<80249b74>] (sg_next+0x48/0x4c) from [<80255398>] (debug_dma_unmap_sg+0x170/0x1a4) [<80255398>] (debug_dma_unmap_sg+0x170/0x1a4) from [<8004af58>] (dma_unmap_sg+0x14/0x6c) [<8004af58>] (dma_unmap_sg+0x14/0x6c) from [<8027e594>] (mxs_dma_tasklet+0x18/0x1c) [<8027e594>] (mxs_dma_tasklet+0x18/0x1c) from [<8007d444>] (tasklet_action+0x114/0x164)
-----------------------------------------------------------------

1.3) Assume the two DMA operations is X (first) and Y (second).

The root cause of the bug:
Assume process P issues DMA X, and sleep on the completion
@this->dma_done. X's tasklet callback is dma_irq_callback. It firstly wake up the process sleeping on the completion @this->dma_done, and then trid to unmap the scatterlist S. The waked process P will issue Y in another ARM core. Y initializes S->sg_magic to zero with sg_init_one(), while dma_irq_callback is unmapping S at the same time.

See the diagram:

ARM core 0 | ARM core 1
-------------------------------------------------------------
(P issues DMA X, then sleep) --> |
|
(X's tasklet wakes P) --> |
|
| <-- (P begin to issue DMA Y) |
(X's tasklet unmap the |
scatterlist S with dma_unmap_sg) --> | <-- (Y calls sg_init_one() to init
| scatterlist S)
|

[2] This patch serialize both the X and Y in the following way:

     Unmap the DMA scatterlist S firstly, and wake up the process at the end
     of the DMA callback, in such a way, Y will be executed after X.

     After this patch:

                   ARM core 0              |         ARM core 1
	 -------------------------------------------------------------
         (P issues DMA X, then sleep)  --> |
                                           |
         (X's tasklet unmap the            |
      scatterlist S with dma_unmap_sg) --> |
                                           |
         (X's tasklet wakes P)         --> |
                                           |
                                           | <-- (P begin to issue DMA Y)
                                           |
                                           | <-- (Y calls sg_init_one() to init
                                           |     scatterlist S)
                                           |

[email protected]

---
fix the wrong diagram.

---
 drivers/mtd/nand/gpmi-nand/gpmi-nand.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)
drivers/mtd/nand/gpmi-nand/gpmi-nand.c +2 -2
index 7ac2280..f89b4fd 100644
@@ -394,8 +394,6 @@ static void dma_irq_callback(void *param)
 	struct gpmi_nand_data *this = param;
 	struct completion *dma_c = &this->dma_done;
 
-	complete(dma_c);
-
 	switch (this->dma_type) {
 	case DMA_FOR_COMMAND:
 		dma_unmap_sg(this->dev, &this->cmd_sgl, 1, DMA_TO_DEVICE);
@@ -420,6 +418,8 @@ static void dma_irq_callback(void *param)
 	default:
 		pr_err("in wrong DMA operation.\n");
 	}
+
+	complete(dma_c);
 }
 
 int start_dma_without_bch_irq(struct gpmi_nand_data *this,
Signature · 1 line
1.7.2.rc3
#6 lore raw

PATCH 1/1 mtd: gpmi: wake up the process at the end of the DMA callback

[1] The gpmi uses the nand_command_lp to issue the commands to NAND chips. The gpmi issues a DMA operation with gpmi_cmd_ctrl when it handles a NAND_CMD_NONE control command. So when we read a page(NAND_CMD_READ0) from the NAND, we may send two DMA operations back-to-back.

If we do not serialize the two DMA operations, we will meet a bug when

1.1) we enable CONFIG_DMA_API_DEBUG, CONFIG_DMADEVICES_DEBUG,
and CONFIG_DEBUG_SG.

1.2) Use the following commands in an UART console and a SSH console:
cmd 1: while true;do dd if=/dev/mtd0 of=/dev/null;done cmd 1: while true;do dd if=/dev/mmcblk0 of=/dev/null;done

The kernel log shows below:
-----------------------------------------------------------------
kernel BUG at lib/scatterlist.c:28!
Unable to handle kernel NULL pointer dereference at virtual address 00000000
.........................
[<80044a0c>] (__bug+0x18/0x24) from [<80249b74>] (sg_next+0x48/0x4c) [<80249b74>] (sg_next+0x48/0x4c) from [<80255398>] (debug_dma_unmap_sg+0x170/0x1a4) [<80255398>] (debug_dma_unmap_sg+0x170/0x1a4) from [<8004af58>] (dma_unmap_sg+0x14/0x6c) [<8004af58>] (dma_unmap_sg+0x14/0x6c) from [<8027e594>] (mxs_dma_tasklet+0x18/0x1c) [<8027e594>] (mxs_dma_tasklet+0x18/0x1c) from [<8007d444>] (tasklet_action+0x114/0x164)
-----------------------------------------------------------------

1.3) Assume the two DMA operations is X (first) and Y (second).

The root cause of the bug:
Assume process P issues DMA X, and sleep on the completion
@this->dma_done. X's tasklet callback is dma_irq_callback. It firstly wake up the process sleeping on the completion @this->dma_done, and then trid to unmap the scatterlist S. The waked process P will issue Y in another ARM core. Y initializes S->sg_magic to zero with sg_init_one(), while dma_irq_callback is unmapping S at the same time.

See the diagram:

ARM core 0 | ARM core 1
-------------------------------------------------------------
(P issues DMA X, then sleep) --> |
|
(X's tasklet wakes P) --> |
|
| <-- (P begin to issue DMA Y) |
(X's tasklet unmap the |
scatterlist S with dma_unmap_sg) --> | <-- (Y calls sg_init_one() to init
| scatterlist S)
|

[2] This patch serialize both the X and Y in the following way:

     Unmap the DMA scatterlist S firstly, and wake up the process at the end
     of the DMA callback, in such a way, Y will be executed after X.

     After this patch:

                   ARM core 0              |         ARM core 1
	 -------------------------------------------------------------
         (P issues DMA X, then sleep)  --> |
                                           |
         (X's tasklet unmap the            |
      scatterlist S with dma_unmap_sg) --> |
                                           |
         (X's tasklet wakes P)         --> |
                                           |
                                           | <-- (P begin to issue DMA Y)
                                           |
                                           | <-- (Y calls sg_init_one() to init
                                           |     scatterlist S)
                                           |

---
fix the wrong diagram.

Resend this patch, CC to stable.

---
 drivers/mtd/nand/gpmi-nand/gpmi-nand.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)
drivers/mtd/nand/gpmi-nand/gpmi-nand.c +2 -2
index 7ac2280..f89b4fd 100644
@@ -394,8 +394,6 @@ static void dma_irq_callback(void *param)
 	struct gpmi_nand_data *this = param;
 	struct completion *dma_c = &this->dma_done;
 
-	complete(dma_c);
-
 	switch (this->dma_type) {
 	case DMA_FOR_COMMAND:
 		dma_unmap_sg(this->dev, &this->cmd_sgl, 1, DMA_TO_DEVICE);
@@ -420,6 +418,8 @@ static void dma_irq_callback(void *param)
 	default:
 		pr_err("in wrong DMA operation.\n");
 	}
+
+	complete(dma_c);
 }
 
 int start_dma_without_bch_irq(struct gpmi_nand_data *this,
Signature · 1 line
1.7.2.rc3
#7 lore raw

Re: [PATCH fix] mtd: gpmi: wake up the process at the end of the DMA callback

Hi Huang,

Quoted 73 lines · Huang Shijie

[1] The gpmi uses the nand_command_lp to issue the commands to NAND chips. The gpmi issues a DMA operation with gpmi_cmd_ctrl when it handles a NAND_CMD_NONE control command. So when we read a page(NAND_CMD_READ0) from the NAND, we may send two DMA operations back-to-back.

If we do not serialize the two DMA operations, we will meet a bug when

1.1) we enable CONFIG_DMA_API_DEBUG, CONFIG_DMADEVICES_DEBUG,
and CONFIG_DEBUG_SG.

1.2) Use the following commands in an UART console and a SSH console:
cmd 1: while true;do dd if=/dev/mtd0 of=/dev/null;done cmd 1: while true;do dd if=/dev/mmcblk0 of=/dev/null;done

The kernel log shows below:
-----------------------------------------------------------------
kernel BUG at lib/scatterlist.c:28!
Unable to handle kernel NULL pointer dereference at virtual address 00000000
.........................
[<80044a0c>] (__bug+0x18/0x24) from [<80249b74>] (sg_next+0x48/0x4c) [<80249b74>] (sg_next+0x48/0x4c) from [<80255398>] (debug_dma_unmap_sg+0x170/0x1a4) [<80255398>] (debug_dma_unmap_sg+0x170/0x1a4) from [<8004af58>] (dma_unmap_sg+0x14/0x6c) [<8004af58>] (dma_unmap_sg+0x14/0x6c) from [<8027e594>] (mxs_dma_tasklet+0x18/0x1c) [<8027e594>] (mxs_dma_tasklet+0x18/0x1c) from [<8007d444>] (tasklet_action+0x114/0x164)
-----------------------------------------------------------------

1.3) Assume the two DMA operations is X (first) and Y (second).

The root cause of the bug:
Assume process P issues DMA X, and sleep on the completion
@this->dma_done. X's tasklet callback is dma_irq_callback. It firstly wake up the process sleeping on the completion @this->dma_done, and then trid to unmap the scatterlist S. The waked process P will issue Y in another ARM core. Y initializes S->sg_magic to zero with sg_init_one(), while dma_irq_callback is unmapping S at the same time.

See the diagram:

ARM core 0 | ARM core 1
-------------------------------------------------------------
(P issues DMA X, then sleep) --> |
|
(X's tasklet wakes P) --> |
|
| <-- (P begin to issue DMA Y) |
(X's tasklet unmap the |
scatterlist S with dma_unmap_sg) --> | <-- (Y calls sg_init_one() to init
| scatterlist S)
|

[2] This patch serialize both the X and Y in the following way:

     Unmap the DMA scatterlist S firstly, and wake up the process at the end
     of the DMA callback, in such a way, Y will be executed after X.

     After this patch:

                   ARM core 0              |         ARM core 1
	 -------------------------------------------------------------
         (P issues DMA X, then sleep)  --> |
                                           |
         (X's tasklet unmap the            |
      scatterlist S with dma_unmap_sg) --> |
                                           |
         (X's tasklet wakes P)         --> |
                                           |
                                           | <-- (P begin to issue DMA Y)
                                           |
                                           | <-- (Y calls sg_init_one() to init
                                           |     scatterlist S)
                                           |
Quoted 4 lines

---
fix the wrong diagram.

Thanks for improving this with (1) a much simpler patch and (2) a very good diagram. I think it hits at the real heart of the problem without adding extra concurrency primitives which obscure the code.

Quoted 1 line

Resend this patch, CC to stable.

It looks like this is long-standing bug (since the driver was introduced in v3.2), right? I'll mark it with v3.2+.

Quoted 18 lines
---
 drivers/mtd/nand/gpmi-nand/gpmi-nand.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)
drivers/mtd/nand/gpmi-nand/gpmi-nand.c +0 -2
index 7ac2280..f89b4fd 100644
@@ -394,8 +394,6 @@ static void dma_irq_callback(void *param)
 	struct gpmi_nand_data *this = param;
 	struct completion *dma_c = &this->dma_done;
 
-	complete(dma_c);
-
 	switch (this->dma_type) {
 	case DMA_FOR_COMMAND:
 		dma_unmap_sg(this->dev, &this->cmd_sgl, 1, DMA_TO_DEVICE);

drivers/mtd/nand/gpmi-nand/gpmi-nand.c static void dma_irq_callback(void *param) · line 425

Quoted 8 lines
drivers/mtd/nand/gpmi-nand/gpmi-nand.c +2 -0
@@ -420,6 +418,8 @@ static void dma_irq_callback(void *param)
 	default:
 		pr_err("in wrong DMA operation.\n");
 	}
+
+	complete(dma_c);
 }
 
 int start_dma_without_bch_irq(struct gpmi_nand_data *this,

I fixed the subject to describe the problem better ("mtd: gpmi: fix kernel BUG due to racing DMA operation") and pushed it to l2-mtd.git. Thanks!

Brian

#8 lore raw

Re: [PATCH fix] mtd: gpmi: wake up the process at the end of the DMA callback

Quoted 2 lines · Brian Norris

I fixed the subject to describe the problem better ("mtd: gpmi: fix kernel BUG due to racing DMA operation") and pushed it to l2-mtd.git.

thanks!

Huang Shijie