Skip to contentSkip to conversation

mmc: block: enable cache-flushing when mmc cache is on

PATCH Superseded Michael Wu 5 participants 21 messages Mar 12, 2022 – Mar 29, 2022 MMC, SD and SDIO

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Michael Wu

author
#0 lore raw

PATCH 1/1 mmc: block: enable cache-flushing when mmc cache is on

The mmc core enable cache on default. But it only enables cache-flushing when host supports cmd23 and eMMC supports reliable write. For hosts which do not support cmd23 or eMMCs which do not support reliable write, the cache can not be flushed by sync command. This may leads to cache data lost.
This patch enables cache-flushing as long as cache is enabled, no matter host supports cmd23 and/or eMMC supports reliable write or not.

---
 drivers/mmc/core/block.c | 20 ++++++++++++++------
 1 file changed, 14 insertions(+), 6 deletions(-)
drivers/mmc/core/block.c +14 -6
index 689eb9afeeed..1e508c079c1e 100644
@@ -2279,6 +2279,8 @@ static struct mmc_blk_data *mmc_blk_alloc_req(struct mmc_card *card,
 	struct mmc_blk_data *md;
 	int devidx, ret;
 	char cap_str[10];
+	bool enable_cache = false;
+	bool enable_fua = false;
 
 	devidx = ida_simple_get(&mmc_blk_ida, 0, max_devices, GFP_KERNEL);
 	if (devidx < 0) {
@@ -2375,12 +2377,18 @@ static struct mmc_blk_data *mmc_blk_alloc_req(struct mmc_card *card,
 			md->flags |= MMC_BLK_CMD23;
 	}
 
-	if (mmc_card_mmc(card) &&
-	    md->flags & MMC_BLK_CMD23 &&
-	    ((card->ext_csd.rel_param & EXT_CSD_WR_REL_PARAM_EN) ||
-	     card->ext_csd.rel_sectors)) {
-		md->flags |= MMC_BLK_REL_WR;
-		blk_queue_write_cache(md->queue.queue, true, true);
+	if (mmc_card_mmc(card)) {
+		if (md->flags & MMC_BLK_CMD23 &&
+			((card->ext_csd.rel_param & EXT_CSD_WR_REL_PARAM_EN) ||
+			card->ext_csd.rel_sectors)) {
+			md->flags |= MMC_BLK_REL_WR;
+			enable_fua = true;
+		}
+
+		if (mmc_cache_enabled(card->host))
+			enable_cache = true;
+
+		blk_queue_write_cache(md->queue.queue, enable_cache, enable_fua);
 	}
 
 	string_get_size((u64)size, 512, STRING_UNITS_2,
Signature · 1 line
2.29.0
#1 lore raw
Quoted 8 lines · Michael Wu

The mmc core enable cache on default. But it only enables cache-flushing when host supports cmd23 and eMMC supports reliable write. For hosts which do not support cmd23 or eMMCs which do not support reliable write, the cache can not be flushed by sync command. This may leads to cache data lost.
This patch enables cache-flushing as long as cache is enabled, no matter host supports cmd23 and/or eMMC supports reliable write or not.

Fixes tag?

Quoted 31 lines
---
 drivers/mmc/core/block.c | 20 ++++++++++++++------
 1 file changed, 14 insertions(+), 6 deletions(-)
drivers/mmc/core/block.c +5 -6
index 689eb9afeeed..1e508c079c1e 100644
@@ -2279,6 +2279,8 @@ static struct mmc_blk_data *mmc_blk_alloc_req(struct mmc_card *card,
 	struct mmc_blk_data *md;
 	int devidx, ret;
 	char cap_str[10];
+	bool enable_cache = false;
+	bool enable_fua = false;
 
 	devidx = ida_simple_get(&mmc_blk_ida, 0, max_devices, GFP_KERNEL);
 	if (devidx < 0) {
@@ -2375,12 +2377,18 @@ static struct mmc_blk_data *mmc_blk_alloc_req(struct mmc_card *card,
 			md->flags |= MMC_BLK_CMD23;
 	}
 
-	if (mmc_card_mmc(card) &&
-	    md->flags & MMC_BLK_CMD23 &&
-	    ((card->ext_csd.rel_param & EXT_CSD_WR_REL_PARAM_EN) ||
-	     card->ext_csd.rel_sectors)) {
-		md->flags |= MMC_BLK_REL_WR;
-		blk_queue_write_cache(md->queue.queue, true, true);
+	if (mmc_card_mmc(card)) {
+		if (md->flags & MMC_BLK_CMD23 &&
+			((card->ext_csd.rel_param & EXT_CSD_WR_REL_PARAM_EN) ||

drivers/mmc/core/block.c static struct mmc_blk_data *mmc_blk_alloc_req(struct mmc_card *card, · line 2392

Quoted 10 lines
drivers/mmc/core/block.c +9 -0
+			card->ext_csd.rel_sectors)) {
+			md->flags |= MMC_BLK_REL_WR;
+			enable_fua = true;
+		}
+
+		if (mmc_cache_enabled(card->host))
+			enable_cache = true;
+
+		blk_queue_write_cache(md->queue.queue, enable_cache, enable_fua);
 	}

Seems like we should inform block layer about SD card cache also

Quoted 2 lines
drivers/mmc/core/block.c +0 -0
 	string_get_size((u64)size, 512, STRING_UNITS_2,
author
#2 lore raw
Quoted 10 lines · Adrian Hunter
Quoted 8 lines · Michael Wu

The mmc core enable cache on default. But it only enables cache-flushing when host supports cmd23 and eMMC supports reliable write. For hosts which do not support cmd23 or eMMCs which do not support reliable write, the cache can not be flushed by sync command. This may leads to cache data lost.
This patch enables cache-flushing as long as cache is enabled, no matter host supports cmd23 and/or eMMC supports reliable write or not.

Quoted 2 lines

Fixes tag?

Hi Adrian,
My patch intend to fix the cache problem brought by the following two patches:

Fixes d0c97cfb81ebc ("mmc: core: Use CMD23 for multiblock transfers when we can.")
Fixes e9d5c746246c8 ("mmc/block: switch to using blk_queue_write_cache()")

I'm not sure if this is what you referred to ("Fixes tag"). Please correct me if I misunderstood.

Quoted 35 lines
Quoted 35 lines
---
  drivers/mmc/core/block.c | 20 ++++++++++++++------
  1 file changed, 14 insertions(+), 6 deletions(-)
drivers/mmc/core/block.c +9 -6
index 689eb9afeeed..1e508c079c1e 100644
@@ -2279,6 +2279,8 @@ static struct mmc_blk_data *mmc_blk_alloc_req(struct mmc_card *card,
  	struct mmc_blk_data *md;
  	int devidx, ret;
  	char cap_str[10];
+	bool enable_cache = false;
+	bool enable_fua = false;
 
  	devidx = ida_simple_get(&mmc_blk_ida, 0, max_devices, GFP_KERNEL);
  	if (devidx < 0) {
@@ -2375,12 +2377,18 @@ static struct mmc_blk_data *mmc_blk_alloc_req(struct mmc_card *card,
  			md->flags |= MMC_BLK_CMD23;
  	}
 
-	if (mmc_card_mmc(card) &&
-	    md->flags & MMC_BLK_CMD23 &&
-	    ((card->ext_csd.rel_param & EXT_CSD_WR_REL_PARAM_EN) ||
-	     card->ext_csd.rel_sectors)) {
-		md->flags |= MMC_BLK_REL_WR;
-		blk_queue_write_cache(md->queue.queue, true, true);
+	if (mmc_card_mmc(card)) {
+		if (md->flags & MMC_BLK_CMD23 &&
+			((card->ext_csd.rel_param & EXT_CSD_WR_REL_PARAM_EN) ||
+			card->ext_csd.rel_sectors)) {
+			md->flags |= MMC_BLK_REL_WR;
+			enable_fua = true;
+		}
Quoted 9 lines

drivers/mmc/core/block.c static struct mmc_blk_data *mmc_blk_alloc_req(struct mmc_card *card, · line 2392

Quoted 6 lines
drivers/mmc/core/block.c +5 -0
+
+		if (mmc_cache_enabled(card->host))
+			enable_cache = true;
+
+		blk_queue_write_cache(md->queue.queue, enable_cache, enable_fua);
  	}

Seems like we should inform block layer about SD card cache also

I saw another mail by Avri Altman, which says few days will be needed to ask internally. Shall I wait or make another change here on 'inform block layer about SD card cache'?

Quoted 2 lines
Quoted 2 lines
drivers/mmc/core/block.c +0 -0
  	string_get_size((u64)size, 512, STRING_UNITS_2,
Signature · 2 lines
Best Regards,
Michael Wu
#3 lore raw
Quoted 75 lines · Adrian Hunter
Quoted 3 lines · Michael Wu
Quoted 2 lines

The mmc core enable cache on default. But it only enables cache-flushing when host supports cmd23 and eMMC supports reliable

write.

Quoted 9 lines
Quoted 6 lines

For hosts which do not support cmd23 or eMMCs which do not support reliable write, the cache can not be flushed by sync command. This may leads to cache data lost.
This patch enables cache-flushing as long as cache is enabled, no matter host supports cmd23 and/or eMMC supports reliable write or not.

Fixes tag?

Hi Adrian,
My patch intend to fix the cache problem brought by the following two patches:

Fixes d0c97cfb81ebc ("mmc: core: Use CMD23 for multiblock transfers when

we can.")

Fixes e9d5c746246c8 ("mmc/block: switch to using blk_queue_write_cache()")

I'm not sure if this is what you referred to ("Fixes tag"). Please correct me if I misunderstood.

Quoted 10 lines

drivers/mmc/core/block.c static struct mmc_blk_data

Quoted 10 lines
---
  drivers/mmc/core/block.c | 20 ++++++++++++++------
  1 file changed, 14 insertions(+), 6 deletions(-)
drivers/mmc/core/block.c +0 -0
index 689eb9afeeed..1e508c079c1e 100644
@@ -2279,6 +2279,8 @@ static struct mmc_blk_data

*mmc_blk_alloc_req(struct mmc_card *card,

Quoted 9 lines

drivers/mmc/core/block.c static struct mmc_blk_data

Quoted 9 lines
drivers/mmc/core/block.c +2 -0
     struct mmc_blk_data *md;
     int devidx, ret;
     char cap_str[10];
+    bool enable_cache = false;
+    bool enable_fua = false;
 
     devidx = ida_simple_get(&mmc_blk_ida, 0, max_devices, GFP_KERNEL);
     if (devidx < 0) {
@@ -2375,12 +2377,18 @@ static struct mmc_blk_data

*mmc_blk_alloc_req(struct mmc_card *card,

Quoted 12 lines

drivers/mmc/core/block.c static struct mmc_blk_data *mmc_blk_alloc_req(struct mmc_card *card,

Quoted 12 lines
drivers/mmc/core/block.c +3 -6
                     md->flags |= MMC_BLK_CMD23;
     }
 
-    if (mmc_card_mmc(card) &&
-        md->flags & MMC_BLK_CMD23 &&
-        ((card->ext_csd.rel_param & EXT_CSD_WR_REL_PARAM_EN) ||
-         card->ext_csd.rel_sectors)) {
-            md->flags |= MMC_BLK_REL_WR;
-            blk_queue_write_cache(md->queue.queue, true, true);
+    if (mmc_card_mmc(card)) {
+            if (md->flags & MMC_BLK_CMD23 &&
+                    ((card->ext_csd.rel_param & EXT_CSD_WR_REL_PARAM_EN)
||
>> +                    card->ext_csd.rel_sectors)) {
>> +                    md->flags |= MMC_BLK_REL_WR;
>> +                    enable_fua = true;
>> +            }
Quoted 10 lines

drivers/mmc/core/block.c static struct mmc_blk_data *mmc_blk_alloc_req(struct mmc_card *card, · line 2392

Quoted 7 lines
drivers/mmc/core/block.c +6 -0
+
+            if (mmc_cache_enabled(card->host))
+                    enable_cache = true;
+
+            blk_queue_write_cache(md->queue.queue, enable_cache,
+ enable_fua);
     }

Seems like we should inform block layer about SD card cache also

Quoted 3 lines

I saw another mail by Avri Altman, which says few days will be needed to ask internally. Shall I wait or make another change here on 'inform block layer about SD card cache'?

Please don't wait.

Thanks,
Avri

Quoted 7 lines
Quoted 3 lines

drivers/mmc/core/block.c static struct mmc_blk_data *mmc_blk_alloc_req(struct mmc_card *card, · line 2394

Quoted 3 lines
drivers/mmc/core/block.c +0 -0
     string_get_size((u64)size, 512, STRING_UNITS_2,

--
Best Regards,
Michael Wu

author
#4 lore raw
Quoted 75 lines · Avri Altman
Quoted 75 lines · Adrian Hunter
Quoted 3 lines · Michael Wu
Quoted 2 lines

The mmc core enable cache on default. But it only enables cache-flushing when host supports cmd23 and eMMC supports reliable

write.

Quoted 9 lines
Quoted 6 lines

For hosts which do not support cmd23 or eMMCs which do not support reliable write, the cache can not be flushed by sync command. This may leads to cache data lost.
This patch enables cache-flushing as long as cache is enabled, no matter host supports cmd23 and/or eMMC supports reliable write or not.

Fixes tag?

Hi Adrian,
My patch intend to fix the cache problem brought by the following two patches:

Fixes d0c97cfb81ebc ("mmc: core: Use CMD23 for multiblock transfers when

we can.")

Fixes e9d5c746246c8 ("mmc/block: switch to using blk_queue_write_cache()")

I'm not sure if this is what you referred to ("Fixes tag"). Please correct me if I misunderstood.

Quoted 10 lines

drivers/mmc/core/block.c static struct mmc_blk_data

Quoted 10 lines
---
   drivers/mmc/core/block.c | 20 ++++++++++++++------
   1 file changed, 14 insertions(+), 6 deletions(-)
drivers/mmc/core/block.c +0 -0
index 689eb9afeeed..1e508c079c1e 100644
@@ -2279,6 +2279,8 @@ static struct mmc_blk_data

*mmc_blk_alloc_req(struct mmc_card *card,

Quoted 9 lines

drivers/mmc/core/block.c static struct mmc_blk_data

Quoted 9 lines
drivers/mmc/core/block.c +2 -0
      struct mmc_blk_data *md;
      int devidx, ret;
      char cap_str[10];
+    bool enable_cache = false;
+    bool enable_fua = false;
 
      devidx = ida_simple_get(&mmc_blk_ida, 0, max_devices, GFP_KERNEL);
      if (devidx < 0) {
@@ -2375,12 +2377,18 @@ static struct mmc_blk_data

*mmc_blk_alloc_req(struct mmc_card *card,

Quoted 12 lines

drivers/mmc/core/block.c static struct mmc_blk_data *mmc_blk_alloc_req(struct mmc_card *card,

Quoted 12 lines
drivers/mmc/core/block.c +3 -6
                      md->flags |= MMC_BLK_CMD23;
      }
 
-    if (mmc_card_mmc(card) &&
-        md->flags & MMC_BLK_CMD23 &&
-        ((card->ext_csd.rel_param & EXT_CSD_WR_REL_PARAM_EN) ||
-         card->ext_csd.rel_sectors)) {
-            md->flags |= MMC_BLK_REL_WR;
-            blk_queue_write_cache(md->queue.queue, true, true);
+    if (mmc_card_mmc(card)) {
+            if (md->flags & MMC_BLK_CMD23 &&
+                    ((card->ext_csd.rel_param & EXT_CSD_WR_REL_PARAM_EN)
||
>> +                    card->ext_csd.rel_sectors)) {
>> +                    md->flags |= MMC_BLK_REL_WR;
>> +                    enable_fua = true;
>> +            }
Quoted 10 lines

drivers/mmc/core/block.c static struct mmc_blk_data *mmc_blk_alloc_req(struct mmc_card *card, · line 2392

Quoted 7 lines
drivers/mmc/core/block.c +6 -0
+
+            if (mmc_cache_enabled(card->host))
+                    enable_cache = true;
+
+            blk_queue_write_cache(md->queue.queue, enable_cache,
+ enable_fua);
      }

Seems like we should inform block layer about SD card cache also

Quoted 15 lines
Quoted 3 lines

I saw another mail by Avri Altman, which says few days will be needed to ask internally. Shall I wait or make another change here on 'inform block layer about SD card cache'?

Please don't wait.

Thanks,
Avri

Quoted 7 lines
Quoted 3 lines

drivers/mmc/core/block.c static struct mmc_blk_data *mmc_blk_alloc_req(struct mmc_card *card, · line 2394

Quoted 3 lines
drivers/mmc/core/block.c +0 -0
      string_get_size((u64)size, 512, STRING_UNITS_2,

--
Best Regards,
Michael Wu

Hi Avril & Adrian,
Thanks for your efforts. Could we have an agreement now --

1. enabling-cache and cmd23/reliable-write should be independent;

Quoted 23 lines · Adrian Hunter
Quoted 22 lines · Avri Altman
Quoted 9 lines

Hi,

Quoted 8 lines

The mmc core enable cache on default. But it only enables cache-flushing when host supports cmd23 and eMMC supports reliable write. For hosts which do not support cmd23 or eMMCs which do not support reliable write, the cache can not be flushed by sync command. This may leads to cache data lost.
This patch enables cache-flushing as long as cache is enabled, no matter host supports cmd23 and/or eMMC supports reliable write or not.

Quoted 3 lines

I looked in the spec and indeed couldn't find why enabling cache is dependent of cmd23/reliable write.
Nor I was able to find the original commit log.

Reliable write was added first, so it might have been an oversight:

commit 881d1c25f765938a95def5afe39486ce39f9fc96

Author: Seungwon Jeon <[email protected]>
Date:   Fri Oct 14 14:03:21 2011 +0900

mmc: core: Add cache control for eMMC4.5 device

Quoted 7 lines
Quoted 7 lines

This patch adds cache feature of eMMC4.5 Spec. If device supports cache capability, host can utilize some specific operations.

Here's what I found in the spec JESD84-B51:

Quoted 6 lines

6.6.31 Cache
Caching of data shall apply only for the single block read/write(CMD17/24), pre-defined multiple block read/write(CMD23+CMD18/25) and open ended multiple block read/write(CMD18/25+CMD12) commands and excludes any other access e.g., to the register space(e.g., CMD6).

Which means with CMD18/25+CMD12 (without using CMD23), the cache can also be enabled. Maybe this could be an evidence of the independence between enabling-cache and cmd23/reliable-write?

2. We don't consider supporting SD in this change.

Quoted 6 lines · Avri Altman
Quoted 5 lines

Here is what our SD system guys wrote:
" In SD we don’t support reliable write and this eMMC driver may not >> be utilizing the cache feature we added in SD5.0.
The method of cache flush is different between SD and eMMC."

So adding SD seems to be out of scope of this change.

Is there anything else I can do about this patch? Thanks again.

Signature · 2 lines
Best Regards,
Michael Wu
Acked-by
#5 lore raw
Quoted 12 lines

Hi Avril & Adrian,
Thanks for your efforts. Could we have an agreement now --

1. enabling-cache and cmd23/reliable-write should be independent;

Here's what I found in the spec JESD84-B51:

Quoted 6 lines

6.6.31 Cache
Caching of data shall apply only for the single block read/write(CMD17/24), pre-defined multiple block read/write(CMD23+CMD18/25) and open ended multiple block read/write(CMD18/25+CMD12) commands and excludes any other access e.g., to the register space(e.g., CMD6).

Quoted 3 lines

Which means with CMD18/25+CMD12 (without using CMD23), the cache can also be enabled. Maybe this could be an evidence of the independence between enabling-cache and cmd23/reliable-write?

Thanks,
Avri

Quoted 16 lines

2. We don't consider supporting SD in this change.

Quoted 7 lines · Avri Altman
Quoted 6 lines

Here is what our SD system guys wrote:
" In SD we don’t support reliable write and this eMMC driver may not
be utilizing the cache feature we added in SD5.0.
The method of cache flush is different between SD and eMMC."

So adding SD seems to be out of scope of this change.

Is there anything else I can do about this patch? Thanks again.

--
Best Regards,
Michael Wu

#6 lore raw
Quoted 15 lines · Avri Altman
Quoted 12 lines

Hi Avril & Adrian,
Thanks for your efforts. Could we have an agreement now --

1. enabling-cache and cmd23/reliable-write should be independent;

Here's what I found in the spec JESD84-B51:

Quoted 6 lines

6.6.31 Cache
Caching of data shall apply only for the single block read/write(CMD17/24), pre-defined multiple block read/write(CMD23+CMD18/25) and open ended multiple block read/write(CMD18/25+CMD12) commands and excludes any other access e.g., to the register space(e.g., CMD6).

Quoted 3 lines

Which means with CMD18/25+CMD12 (without using CMD23), the cache can also be enabled. Maybe this could be an evidence of the independence between enabling-cache and cmd23/reliable-write?

Quoted 17 lines

Thanks,
Avri

Quoted 12 lines

2. We don't consider supporting SD in this change.

Quoted 7 lines · Avri Altman
Quoted 6 lines

Here is what our SD system guys wrote:
" In SD we don’t support reliable write and this eMMC driver may not
be utilizing the cache feature we added in SD5.0.
The method of cache flush is different between SD and eMMC."

So adding SD seems to be out of scope of this change.

Is there anything else I can do about this patch? Thanks again.

So we are not going to let the block layer know about SD cache? Or is it a separate change?

#7 lore raw
Quoted 2 lines

So we are not going to let the block layer know about SD cache? Or is it a separate change?

I have some code for this laying around, but as it requires reading, parsing and writing Function Registers,
in particular PEH, it's a lot of boilerplate code to get the functionality, but I'll clean it up and send a patch in the coming weeks.

From: Adrian Hunter <[email protected]>
Sent: Wednesday, March 16, 2022 12:28 PM
To: Avri Altman; Michael Wu; [email protected]; [email protected]; [email protected]
Cc: [email protected]; [email protected]; allwinner-opensource-support
Subject: Re: [PATCH] mmc: block: enable cache-flushing when mmc cache is on

Quoted 15 lines · Avri Altman
Quoted 5 lines

Hi Avril & Adrian,
Thanks for your efforts. Could we have an agreement now --

1. enabling-cache and cmd23/reliable-write should be independent;

Quoted 10 lines

Here's what I found in the spec JESD84-B51:
  > 6.6.31 Cache
  > Caching of data shall apply only for the single block > read/write(CMD17/24), pre-defined multiple block
  > read/write(CMD23+CMD18/25) and open ended multiple block > read/write(CMD18/25+CMD12) commands and excludes any other access > e.g., to the register space(e.g., CMD6).
Which means with CMD18/25+CMD12 (without using CMD23), the cache can also be enabled. Maybe this could be an evidence of the independence between enabling-cache and cmd23/reliable-write?

Quoted 17 lines

Thanks,
Avri

Quoted 12 lines

2. We don't consider supporting SD in this change.

  > On 14/03/2022 19:10, Avri Altman wrote:
  >> Here is what our SD system guys wrote:
  >> " In SD we don’t support reliable write and this eMMC driver may not >>    be utilizing the cache feature we added in SD5.0.
  >>   The method of cache flush is different between SD and eMMC." >>
  >> So adding SD seems to be out of scope of this change.

Is there anything else I can do about this patch? Thanks again.

So we are not going to let the block layer know about SD cache? Or is it a separate change?

    =

Hyperstone GmbH | Reichenaustr. 39a | 78467 Konstanz Managing Director: Dr. Jan Peter Berns. Commercial register of local courts: Freiburg HRB381782

#8 lore raw
Quoted 6 lines · Christian Löhle
Quoted 2 lines

So we are not going to let the block layer know about SD cache? Or is it a separate change?

I have some code for this laying around, but as it requires reading, parsing and writing Function Registers,
in particular PEH, it's a lot of boilerplate code to get the functionality, but I'll clean it up and send a patch in the coming weeks.

We have the sd cache flush. We would presumably just need to call blk_queue_write_cache() for the !mmc_card_mmc(card) case e.g.

	if (mmc_has_reliable_write(card)) {
		md->flags |= MMC_BLK_REL_WR;
		enable_fua = true;
	}

	if (mmc_cache_enabled(card->host))
		enable_cache = true;

	blk_queue_write_cache(md->queue.queue, enable_cache, enable_fua);

Avri, were you objecting to that?

Quoted 51 lines

From: Adrian Hunter <[email protected]>
Sent: Wednesday, March 16, 2022 12:28 PM
To: Avri Altman; Michael Wu; [email protected]; [email protected]; [email protected]
Cc: [email protected]; [email protected]; allwinner-opensource-support
Subject: Re: [PATCH] mmc: block: enable cache-flushing when mmc cache is on

Quoted 15 lines · Avri Altman
Quoted 5 lines

Hi Avril & Adrian,
Thanks for your efforts. Could we have an agreement now --

1. enabling-cache and cmd23/reliable-write should be independent;

Quoted 10 lines

Here's what I found in the spec JESD84-B51:
  > 6.6.31 Cache
  > Caching of data shall apply only for the single block > read/write(CMD17/24), pre-defined multiple block
  > read/write(CMD23+CMD18/25) and open ended multiple block > read/write(CMD18/25+CMD12) commands and excludes any other access > e.g., to the register space(e.g., CMD6).
Which means with CMD18/25+CMD12 (without using CMD23), the cache can also be enabled. Maybe this could be an evidence of the independence between enabling-cache and cmd23/reliable-write?

Quoted 17 lines

Thanks,
Avri

Quoted 12 lines

2. We don't consider supporting SD in this change.

  > On 14/03/2022 19:10, Avri Altman wrote:
  >> Here is what our SD system guys wrote:
  >> " In SD we don’t support reliable write and this eMMC driver may not >>    be utilizing the cache feature we added in SD5.0.
  >>   The method of cache flush is different between SD and eMMC." >>
  >> So adding SD seems to be out of scope of this change.

Is there anything else I can do about this patch? Thanks again.

So we are not going to let the block layer know about SD cache? Or is it a separate change?

    =

Hyperstone GmbH | Reichenaustr. 39a | 78467 Konstanz Managing Director: Dr. Jan Peter Berns. Commercial register of local courts: Freiburg HRB381782

#9 lore raw
Quoted 12 lines · Adrian Hunter
Quoted 6 lines · Christian Löhle
Quoted 2 lines

So we are not going to let the block layer know about SD cache? Or is it a separate change?

I have some code for this laying around, but as it requires reading, parsing and writing Function Registers,
in particular PEH, it's a lot of boilerplate code to get the functionality, but I'll clean it up and send a patch in the coming weeks.

We have the sd cache flush. We would presumably just need to call blk_queue_write_cache() for the !mmc_card_mmc(card) case e.g.

Quoted 9 lines
        if (mmc_has_reliable_write(card)) {
                md->flags |= MMC_BLK_REL_WR;
                enable_fua = true;
        }

        if (mmc_cache_enabled(card->host))
                enable_cache = true;

        blk_queue_write_cache(md->queue.queue, enable_cache, enable_fua);

To me, this seems like the most reasonable thing to do.

However, I have to admit that it's not clear to me, if there was a good reason to why commit f4c5522b0a88 ("mmc: Reliable write support.") also added support for REQ_FLUSH (write back cache) and why not only REQ_FUA. I assumed this was wrong too, right?

When it comes to patches for stable kernels. mmc_cache_enabled() was introduced quite recently in v5.13, so for older kernels that call needs to be replaced with something else.

In any case, the relevant commits that can be considered as needs to be fixed seems like these:
commit f4c5522b0a88 ("mmc: Reliable write support.")
commit 881d1c25f765 ("mmc: core: Add cache control for eMMC4.5 device") commit 130206a615a9 ("mmc: core: Add support for cache ctrl for SD cards")

[...]

Kind regards
Uffe

#10 lore raw
Quoted 35 lines · Ulf Hansson
Quoted 12 lines · Adrian Hunter
Quoted 6 lines · Christian Löhle
Quoted 2 lines

So we are not going to let the block layer know about SD cache? Or is it a separate change?

I have some code for this laying around, but as it requires reading, parsing and writing Function Registers,
in particular PEH, it's a lot of boilerplate code to get the functionality, but I'll clean it up and send a patch in the coming weeks.

We have the sd cache flush. We would presumably just need to call blk_queue_write_cache() for the !mmc_card_mmc(card) case e.g.

Quoted 9 lines
        if (mmc_has_reliable_write(card)) {
                md->flags |= MMC_BLK_REL_WR;
                enable_fua = true;
        }

        if (mmc_cache_enabled(card->host))
                enable_cache = true;

        blk_queue_write_cache(md->queue.queue, enable_cache, enable_fua);

To me, this seems like the most reasonable thing to do.

However, I have to admit that it's not clear to me, if there was a good reason to why commit f4c5522b0a88 ("mmc: Reliable write support.") also added support for REQ_FLUSH (write back cache) and why not only REQ_FUA. I assumed this was wrong too, right?

When it comes to patches for stable kernels. mmc_cache_enabled() was introduced quite recently in v5.13, so for older kernels that call needs to be replaced with something else.

Quoted 7 lines

In any case, the relevant commits that can be considered as needs to be fixed seems like these:
commit f4c5522b0a88 ("mmc: Reliable write support.")
commit 881d1c25f765 ("mmc: core: Add cache control for eMMC4.5 device") commit 130206a615a9 ("mmc: core: Add support for cache ctrl for SD cards")

[...]

Michael, are you planning to send a v2 for this? Or are there any parts that are still unclear to you?

Kind regards
Uffe

author
#11 lore raw
Quoted 25 lines · Ulf Hansson
Quoted 24 lines · Ulf Hansson
Quoted 21 lines · Adrian Hunter
Quoted 6 lines · Christian Löhle
Quoted 2 lines

So we are not going to let the block layer know about SD cache? Or is it a separate change?

I have some code for this laying around, but as it requires reading, parsing and writing Function Registers,
in particular PEH, it's a lot of boilerplate code to get the functionality, but I'll clean it up and send a patch in the coming weeks.

We have the sd cache flush. We would presumably just need to call blk_queue_write_cache() for the !mmc_card_mmc(card) case e.g.

         if (mmc_has_reliable_write(card)) {
                 md->flags |= MMC_BLK_REL_WR;
                 enable_fua = true;
         }

         if (mmc_cache_enabled(card->host))
                 enable_cache = true;

         blk_queue_write_cache(md->queue.queue, enable_cache, enable_fua);
Quoted 7 lines
Quoted 7 lines

To me, this seems like the most reasonable thing to do.

However, I have to admit that it's not clear to me, if there was a good reason to why commit f4c5522b0a88 ("mmc: Reliable write support.") also added support for REQ_FLUSH (write back cache) and why not only REQ_FUA. I assumed this was wrong too, right?

Hi Ulf,

1. I've found the reason. If we only enable REQ_FUA, there won't be any effect -- The block layer won't send any request with FUA flag to the driver. If we want REQ_FUA to take effect, we must enable REQ_FLUSH. But on the contrary, REQ_FLUSH does not rely on REQ_FUA.
In the previous patch(commit f4c5522b0a88 ("mmc: Reliable write support.")), REQ_FLUSH was added to make REQ_FUA effective. I've done experiments to prove this.

2. Why block layer requires REQ_FLUSH to make REQ_FUA effective? I did not find the reason. Does anyone know about this? Thank you.

Quoted 10 lines
Quoted 10 lines

When it comes to patches for stable kernels. mmc_cache_enabled() was introduced quite recently in v5.13, so for older kernels that call needs to be replaced with something else.

In any case, the relevant commits that can be considered as needs to be fixed seems like these:
commit f4c5522b0a88 ("mmc: Reliable write support.")
commit 881d1c25f765 ("mmc: core: Add cache control for eMMC4.5 device") commit 130206a615a9 ("mmc: core: Add support for cache ctrl for SD cards")

Quoted 4 lines
Quoted 1 line

[...]

Michael, are you planning to send a v2 for this? Or are there any parts that are still unclear to you?

Dear Ulf, Sorry for the delay. I was trying to figure out the SD cache stuff, so a few day was taken...

Quoted 3 lines

Kind regards
Uffe

Signature · 2 lines
Best Regards,
Michael Wu
#12 lore raw
Quoted 37 lines · Michael Wu
Quoted 25 lines · Ulf Hansson
Quoted 24 lines · Ulf Hansson
Quoted 21 lines · Adrian Hunter
Quoted 6 lines · Christian Löhle
Quoted 2 lines

So we are not going to let the block layer know about SD cache? Or is it a separate change?

I have some code for this laying around, but as it requires reading, parsing and writing Function Registers,
in particular PEH, it's a lot of boilerplate code to get the functionality, but I'll clean it up and send a patch in the coming weeks.

We have the sd cache flush. We would presumably just need to call blk_queue_write_cache() for the !mmc_card_mmc(card) case e.g.

         if (mmc_has_reliable_write(card)) {
                 md->flags |= MMC_BLK_REL_WR;
                 enable_fua = true;
         }

         if (mmc_cache_enabled(card->host))
                 enable_cache = true;

         blk_queue_write_cache(md->queue.queue, enable_cache, enable_fua);
Quoted 7 lines
Quoted 7 lines

To me, this seems like the most reasonable thing to do.

However, I have to admit that it's not clear to me, if there was a good reason to why commit f4c5522b0a88 ("mmc: Reliable write support.") also added support for REQ_FLUSH (write back cache) and why not only REQ_FUA. I assumed this was wrong too, right?

Hi Ulf,

Quoted 8 lines

1. I've found the reason. If we only enable REQ_FUA, there won't be any effect -- The block layer won't send any request with FUA flag to the driver.
If we want REQ_FUA to take effect, we must enable REQ_FLUSH. But on the contrary, REQ_FLUSH does not rely on REQ_FUA.
In the previous patch(commit f4c5522b0a88 ("mmc: Reliable write support.")), REQ_FLUSH was added to make REQ_FUA effective. I've done experiments to prove this.

Thanks for doing the research and for confirming.

Note that this is also pretty well documented in
Documentation/block/writeback_cache_control.rst.

Quoted 3 lines

2. Why block layer requires REQ_FLUSH to make REQ_FUA effective? I did not find the reason. Does anyone know about this? Thank you.

The REQ_FLUSH indicates that the storage device has a write back cache, which also can be flushed in some device specific way.

The REQ_FUA (Force Unit Access), tells that the data can be written to the storage device, in a way that when the I/O request is completed, the data is fully written to the device (the data must not be left in the write back cache). In other words, REQ_FUA doesn't make sense unless REQ_FLUSH is supported too.

$subject patch should also conform to this pattern.

However, it's still questionable to me whether we want to support REQ_FUA through the eMMC reliable write command - in case we also have support for managing the eMMC cache separately. It looks to me that the reason why we currently support REQ_FUA, is because back in the days when there was only the eMMC reliable write command available, it was simply the best we could do. But it was never really a good fit.

I am starting to think that we may consider dropping REQ_FUA, if we have the option to manage the eMMC cache separately - no matter whether the eMMC reliable write command is supported or not. In this case, REQ_FLUSH is sufficient and also a better match to what we really can support.

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

When it comes to patches for stable kernels. mmc_cache_enabled() was introduced quite recently in v5.13, so for older kernels that call needs to be replaced with something else.

In any case, the relevant commits that can be considered as needs to be fixed seems like these:
commit f4c5522b0a88 ("mmc: Reliable write support.")
commit 881d1c25f765 ("mmc: core: Add cache control for eMMC4.5 device") commit 130206a615a9 ("mmc: core: Add support for cache ctrl for SD cards")

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

[...]

Michael, are you planning to send a v2 for this? Or are there any parts that are still unclear to you?

Dear Ulf, Sorry for the delay. I was trying to figure out the SD cache stuff, so a few day was taken...

No problem, I have been busy too. :-)

Kind regards
Uffe

author
#13 lore raw
Quoted 47 lines · Ulf Hansson
Quoted 37 lines · Michael Wu
Quoted 25 lines · Ulf Hansson
Quoted 24 lines · Ulf Hansson
Quoted 21 lines · Adrian Hunter
Quoted 6 lines · Christian Löhle
Quoted 2 lines

So we are not going to let the block layer know about SD cache? Or is it a separate change?

I have some code for this laying around, but as it requires reading, parsing and writing Function Registers,
in particular PEH, it's a lot of boilerplate code to get the functionality, but I'll clean it up and send a patch in the coming weeks.

We have the sd cache flush. We would presumably just need to call blk_queue_write_cache() for the !mmc_card_mmc(card) case e.g.

          if (mmc_has_reliable_write(card)) {
                  md->flags |= MMC_BLK_REL_WR;
                  enable_fua = true;
          }

          if (mmc_cache_enabled(card->host))
                  enable_cache = true;

          blk_queue_write_cache(md->queue.queue, enable_cache, enable_fua);
Quoted 7 lines
Quoted 7 lines

To me, this seems like the most reasonable thing to do.

However, I have to admit that it's not clear to me, if there was a good reason to why commit f4c5522b0a88 ("mmc: Reliable write support.") also added support for REQ_FLUSH (write back cache) and why not only REQ_FUA. I assumed this was wrong too, right?

Hi Ulf,

Quoted 8 lines

1. I've found the reason. If we only enable REQ_FUA, there won't be any effect -- The block layer won't send any request with FUA flag to the driver.
If we want REQ_FUA to take effect, we must enable REQ_FLUSH. But on the contrary, REQ_FLUSH does not rely on REQ_FUA.
In the previous patch(commit f4c5522b0a88 ("mmc: Reliable write support.")), REQ_FLUSH was added to make REQ_FUA effective. I've done experiments to prove this.

Quoted 4 lines

Thanks for doing the research and for confirming.

Note that this is also pretty well documented in
Documentation/block/writeback_cache_control.rst.

Thanks for reminding. I'm clear now.

Quoted 5 lines
Quoted 4 lines

2. Why block layer requires REQ_FLUSH to make REQ_FUA effective? I did not find the reason. Does anyone know about this? Thank you.

Quoted 9 lines

The REQ_FLUSH indicates that the storage device has a write back cache, which also can be flushed in some device specific way.

The REQ_FUA (Force Unit Access), tells that the data can be written to the storage device, in a way that when the I/O request is completed, the data is fully written to the device (the data must not be left in the write back cache). In other words, REQ_FUA doesn't make sense unless REQ_FLUSH is supported too.

Thank you for your answer.

Quoted 1 line

$subject patch should also conform to this pattern.

I'm not sure if I understood this in a right way... Did you mean I should modify the subject of this mail/patch?

Quoted 8 lines

However, it's still questionable to me whether we want to support REQ_FUA through the eMMC reliable write command - in case we also have support for managing the eMMC cache separately. It looks to me that the reason why we currently support REQ_FUA, is because back in the days when there was only the eMMC reliable write command available, it was simply the best we could do. But it was never really a good fit.

Quoted 5 lines

I am starting to think that we may consider dropping REQ_FUA, if we have the option to manage the eMMC cache separately - no matter whether the eMMC reliable write command is supported or not. In this case, REQ_FLUSH is sufficient and also a better match to what we really can support.

Hi Ulf,
As to dropping REQ_FUA, I don't know if it is a good idea, but generally we are facing three possible situations:

1. If both cache and reliable-write are available, both REQ_FUA and REQ_FLUSH can be supported at the same time. In this case, with available cache, the behavior of reliable-write is to write eMMC while skipping cache, which is consistent with the current kernel's definition of REQ_FUA. What's more, most eMMCs now support both cache and reliable-write command.
2. If only reliable-write is available, REQ_FUA should not be supported, which is consistent with the current standard in another way. But I don't think eMMCs that only support reliable-write can be easily found nowadays.
3. If only cache is available, we just use REQ_FLUSH. It is not in conflict with keeping REQ_FUA.

Maybe, is it more reasonable to reserve FUA and use if/else to pick it up or down, considering the compatibility? I mean, in most cases, FUA and FLUSH are complementary. So it seems more feasible with branch to choose.

Quoted 23 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

When it comes to patches for stable kernels. mmc_cache_enabled() was introduced quite recently in v5.13, so for older kernels that call needs to be replaced with something else.

In any case, the relevant commits that can be considered as needs to be fixed seems like these:
commit f4c5522b0a88 ("mmc: Reliable write support.")
commit 881d1c25f765 ("mmc: core: Add cache control for eMMC4.5 device") commit 130206a615a9 ("mmc: core: Add support for cache ctrl for SD cards")

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

[...]

Michael, are you planning to send a v2 for this? Or are there any parts that are still unclear to you?

Dear Ulf, Sorry for the delay. I was trying to figure out the SD cache stuff, so a few day was taken...

No problem, I have been busy too. :-)

Kind regards
Uffe

Signature · 2 lines
Best Regards,
Michael Wu
#14 lore raw
Quoted 71 lines · Michael Wu
Quoted 47 lines · Ulf Hansson
Quoted 37 lines · Michael Wu
Quoted 25 lines · Ulf Hansson
Quoted 24 lines · Ulf Hansson
Quoted 21 lines · Adrian Hunter
Quoted 6 lines · Christian Löhle
Quoted 2 lines

So we are not going to let the block layer know about SD cache? Or is it a separate change?

I have some code for this laying around, but as it requires reading, parsing and writing Function Registers,
in particular PEH, it's a lot of boilerplate code to get the functionality, but I'll clean it up and send a patch in the coming weeks.

We have the sd cache flush. We would presumably just need to call blk_queue_write_cache() for the !mmc_card_mmc(card) case e.g.

          if (mmc_has_reliable_write(card)) {
                  md->flags |= MMC_BLK_REL_WR;
                  enable_fua = true;
          }

          if (mmc_cache_enabled(card->host))
                  enable_cache = true;

          blk_queue_write_cache(md->queue.queue, enable_cache, enable_fua);
Quoted 7 lines
Quoted 7 lines

To me, this seems like the most reasonable thing to do.

However, I have to admit that it's not clear to me, if there was a good reason to why commit f4c5522b0a88 ("mmc: Reliable write support.") also added support for REQ_FLUSH (write back cache) and why not only REQ_FUA. I assumed this was wrong too, right?

Hi Ulf,

Quoted 8 lines

1. I've found the reason. If we only enable REQ_FUA, there won't be any effect -- The block layer won't send any request with FUA flag to the driver.
If we want REQ_FUA to take effect, we must enable REQ_FLUSH. But on the contrary, REQ_FLUSH does not rely on REQ_FUA.
In the previous patch(commit f4c5522b0a88 ("mmc: Reliable write support.")), REQ_FLUSH was added to make REQ_FUA effective. I've done experiments to prove this.

Quoted 4 lines

Thanks for doing the research and for confirming.

Note that this is also pretty well documented in
Documentation/block/writeback_cache_control.rst.

Thanks for reminding. I'm clear now.

Quoted 5 lines
Quoted 4 lines

2. Why block layer requires REQ_FLUSH to make REQ_FUA effective? I did not find the reason. Does anyone know about this? Thank you.

Quoted 9 lines

The REQ_FLUSH indicates that the storage device has a write back cache, which also can be flushed in some device specific way.

The REQ_FUA (Force Unit Access), tells that the data can be written to the storage device, in a way that when the I/O request is completed, the data is fully written to the device (the data must not be left in the write back cache). In other words, REQ_FUA doesn't make sense unless REQ_FLUSH is supported too.

Quoted 6 lines

Thank you for your answer.

Quoted 1 line

$subject patch should also conform to this pattern.

I'm not sure if I understood this in a right way... Did you mean I should modify the subject of this mail/patch?

No, I just meant that the code in the patch should conform to this.

If REQ_FUA is set, REQ_FLUSH must be set too.

Quoted 19 lines
Quoted 9 lines

However, it's still questionable to me whether we want to support REQ_FUA through the eMMC reliable write command - in case we also have support for managing the eMMC cache separately. It looks to me that the reason why we currently support REQ_FUA, is because back in the days when there was only the eMMC reliable write command available, it was simply the best we could do. But it was never really a good fit.

Quoted 5 lines

I am starting to think that we may consider dropping REQ_FUA, if we have the option to manage the eMMC cache separately - no matter whether the eMMC reliable write command is supported or not. In this case, REQ_FLUSH is sufficient and also a better match to what we really can support.

Hi Ulf,
As to dropping REQ_FUA, I don't know if it is a good idea, but generally we are facing three possible situations:

Quoted 6 lines

1. If both cache and reliable-write are available, both REQ_FUA and REQ_FLUSH can be supported at the same time. In this case, with available cache, the behavior of reliable-write is to write eMMC while skipping cache, which is consistent with the current kernel's definition of REQ_FUA. What's more, most eMMCs now support both cache and reliable-write command.

Yes, this seems reasonable.

Quoted 4 lines

2. If only reliable-write is available, REQ_FUA should not be supported, which is consistent with the current standard in another way. But I don't think eMMCs that only support reliable-write can be easily found nowadays.

If we drop REQ_FUA for this case, I am worried that we might break use cases for those older eMMC devices.

So, no, let's keep REQ_FUA and REQ_FLUSH if reliable-write is supported.

Quoted 2 lines

3. If only cache is available, we just use REQ_FLUSH. It is not in conflict with keeping REQ_FUA.

Right.

Quoted 5 lines

Maybe, is it more reasonable to reserve FUA and use if/else to pick it up or down, considering the compatibility? I mean, in most cases, FUA and FLUSH are complementary. So it seems more feasible with branch to choose.

Let's summarize what I think we should do then:

 if (reliable-write supported) {
     enable_fua = true;
     enable_cache = true;
}

if (mmc_cache_enabled)

     enable_cache = true;

blk_queue_write_cache(md->queue.queue, enable_cache, enable_fua);

Does this seem reasonable to you?

[...]

Kind regards
Uffe

author
#15 lore raw
Quoted 72 lines · Ulf Hansson
Quoted 71 lines · Michael Wu
Quoted 47 lines · Ulf Hansson
Quoted 37 lines · Michael Wu
Quoted 25 lines · Ulf Hansson
Quoted 24 lines · Ulf Hansson
Quoted 21 lines · Adrian Hunter
Quoted 6 lines · Christian Löhle
Quoted 2 lines

So we are not going to let the block layer know about SD cache? Or is it a separate change?

I have some code for this laying around, but as it requires reading, parsing and writing Function Registers,
in particular PEH, it's a lot of boilerplate code to get the functionality, but I'll clean it up and send a patch in the coming weeks.

We have the sd cache flush. We would presumably just need to call blk_queue_write_cache() for the !mmc_card_mmc(card) case e.g.

           if (mmc_has_reliable_write(card)) {
                   md->flags |= MMC_BLK_REL_WR;
                   enable_fua = true;
           }

           if (mmc_cache_enabled(card->host))
                   enable_cache = true;

           blk_queue_write_cache(md->queue.queue, enable_cache, enable_fua);
Quoted 7 lines
Quoted 7 lines

To me, this seems like the most reasonable thing to do.

However, I have to admit that it's not clear to me, if there was a good reason to why commit f4c5522b0a88 ("mmc: Reliable write support.") also added support for REQ_FLUSH (write back cache) and why not only REQ_FUA. I assumed this was wrong too, right?

Hi Ulf,

Quoted 8 lines

1. I've found the reason. If we only enable REQ_FUA, there won't be any effect -- The block layer won't send any request with FUA flag to the driver.
If we want REQ_FUA to take effect, we must enable REQ_FLUSH. But on the contrary, REQ_FLUSH does not rely on REQ_FUA.
In the previous patch(commit f4c5522b0a88 ("mmc: Reliable write support.")), REQ_FLUSH was added to make REQ_FUA effective. I've done experiments to prove this.

Quoted 4 lines

Thanks for doing the research and for confirming.

Note that this is also pretty well documented in
Documentation/block/writeback_cache_control.rst.

Thanks for reminding. I'm clear now.

Quoted 14 lines
Quoted 4 lines

2. Why block layer requires REQ_FLUSH to make REQ_FUA effective? I did not find the reason. Does anyone know about this? Thank you.

The REQ_FLUSH indicates that the storage device has a write back cache, which also can be flushed in some device specific way.

The REQ_FUA (Force Unit Access), tells that the data can be written to the storage device, in a way that when the I/O request is completed, the data is fully written to the device (the data must not be left in the write back cache). In other words, REQ_FUA doesn't make sense unless REQ_FLUSH is supported too.

Quoted 8 lines
Quoted 6 lines

Thank you for your answer.

Quoted 1 line

$subject patch should also conform to this pattern.

I'm not sure if I understood this in a right way... Did you mean I should modify the subject of this mail/patch?

No, I just meant that the code in the patch should conform to this.

No problem. Please have a look at the code below.

Quoted 37 lines

If REQ_FUA is set, REQ_FLUSH must be set too.

Quoted 19 lines
Quoted 9 lines

However, it's still questionable to me whether we want to support REQ_FUA through the eMMC reliable write command - in case we also have support for managing the eMMC cache separately. It looks to me that the reason why we currently support REQ_FUA, is because back in the days when there was only the eMMC reliable write command available, it was simply the best we could do. But it was never really a good fit.

Quoted 5 lines

I am starting to think that we may consider dropping REQ_FUA, if we have the option to manage the eMMC cache separately - no matter whether the eMMC reliable write command is supported or not. In this case, REQ_FLUSH is sufficient and also a better match to what we really can support.

Hi Ulf,
As to dropping REQ_FUA, I don't know if it is a good idea, but generally we are facing three possible situations:

Quoted 6 lines

1. If both cache and reliable-write are available, both REQ_FUA and REQ_FLUSH can be supported at the same time. In this case, with available cache, the behavior of reliable-write is to write eMMC while skipping cache, which is consistent with the current kernel's definition of REQ_FUA. What's more, most eMMCs now support both cache and reliable-write command.

Yes, this seems reasonable.

Quoted 4 lines

2. If only reliable-write is available, REQ_FUA should not be supported, which is consistent with the current standard in another way. But I don't think eMMCs that only support reliable-write can be easily found nowadays.

Quoted 4 lines

If we drop REQ_FUA for this case, I am worried that we might break use cases for those older eMMC devices.

So, no, let's keep REQ_FUA and REQ_FLUSH if reliable-write is supported.

OK. Let's keep them.

Quoted 19 lines
Quoted 3 lines

3. If only cache is available, we just use REQ_FLUSH. It is not in conflict with keeping REQ_FUA.

Right.

Quoted 5 lines

Maybe, is it more reasonable to reserve FUA and use if/else to pick it up or down, considering the compatibility? I mean, in most cases, FUA and FLUSH are complementary. So it seems more feasible with branch to choose.

Let's summarize what I think we should do then:

  if (reliable-write supported) {
      enable_fua = true;
      enable_cache = true;
}
Quoted 6 lines

if (mmc_cache_enabled)

      enable_cache = true;

blk_queue_write_cache(md->queue.queue, enable_cache, enable_fua);

Does this seem reasonable to you?

Yes. Let me attach the complete code here:

    if (md->flags & MMC_BLK_CMD23 &&
        ((card->ext_csd.rel_param & EXT_CSD_WR_REL_PARAM_EN) ||
        card->ext_csd.rel_sectors)) {
        md->flags |= MMC_BLK_REL_WR;
        enable_fua = true;
        enable_cache = true;
    }

    if (mmc_cache_enabled(card->host))
        enable_cache = true;

    blk_queue_write_cache(md->queue.queue, enable_cache, enable_fua);

If this is good, I'll submit a patch-v2 soon.

Quoted 5 lines

[...]

Kind regards
Uffe

Signature · 2 lines
Best Regards,
Michael Wu
#16 lore raw
Quoted 171 lines · Michael Wu
Quoted 72 lines · Ulf Hansson
Quoted 71 lines · Michael Wu
Quoted 47 lines · Ulf Hansson
Quoted 37 lines · Michael Wu
Quoted 25 lines · Ulf Hansson
Quoted 24 lines · Ulf Hansson
Quoted 21 lines · Adrian Hunter
Quoted 6 lines · Christian Löhle
Quoted 2 lines

So we are not going to let the block layer know about SD cache? Or is it a separate change?

I have some code for this laying around, but as it requires reading, parsing and writing Function Registers,
in particular PEH, it's a lot of boilerplate code to get the functionality, but I'll clean it up and send a patch in the coming weeks.

We have the sd cache flush. We would presumably just need to call blk_queue_write_cache() for the !mmc_card_mmc(card) case e.g.

           if (mmc_has_reliable_write(card)) {
                   md->flags |= MMC_BLK_REL_WR;
                   enable_fua = true;
           }

           if (mmc_cache_enabled(card->host))
                   enable_cache = true;

           blk_queue_write_cache(md->queue.queue, enable_cache, enable_fua);
Quoted 7 lines
Quoted 7 lines

To me, this seems like the most reasonable thing to do.

However, I have to admit that it's not clear to me, if there was a good reason to why commit f4c5522b0a88 ("mmc: Reliable write support.") also added support for REQ_FLUSH (write back cache) and why not only REQ_FUA. I assumed this was wrong too, right?

Hi Ulf,

Quoted 8 lines

1. I've found the reason. If we only enable REQ_FUA, there won't be any effect -- The block layer won't send any request with FUA flag to the driver.
If we want REQ_FUA to take effect, we must enable REQ_FLUSH. But on the contrary, REQ_FLUSH does not rely on REQ_FUA.
In the previous patch(commit f4c5522b0a88 ("mmc: Reliable write support.")), REQ_FLUSH was added to make REQ_FUA effective. I've done experiments to prove this.

Quoted 4 lines

Thanks for doing the research and for confirming.

Note that this is also pretty well documented in
Documentation/block/writeback_cache_control.rst.

Thanks for reminding. I'm clear now.

Quoted 14 lines
Quoted 4 lines

2. Why block layer requires REQ_FLUSH to make REQ_FUA effective? I did not find the reason. Does anyone know about this? Thank you.

The REQ_FLUSH indicates that the storage device has a write back cache, which also can be flushed in some device specific way.

The REQ_FUA (Force Unit Access), tells that the data can be written to the storage device, in a way that when the I/O request is completed, the data is fully written to the device (the data must not be left in the write back cache). In other words, REQ_FUA doesn't make sense unless REQ_FLUSH is supported too.

Quoted 8 lines
Quoted 6 lines

Thank you for your answer.

Quoted 1 line

$subject patch should also conform to this pattern.

I'm not sure if I understood this in a right way... Did you mean I should modify the subject of this mail/patch?

No, I just meant that the code in the patch should conform to this.

No problem. Please have a look at the code below.

Quoted 37 lines

If REQ_FUA is set, REQ_FLUSH must be set too.

Quoted 19 lines
Quoted 9 lines

However, it's still questionable to me whether we want to support REQ_FUA through the eMMC reliable write command - in case we also have support for managing the eMMC cache separately. It looks to me that the reason why we currently support REQ_FUA, is because back in the days when there was only the eMMC reliable write command available, it was simply the best we could do. But it was never really a good fit.

Quoted 5 lines

I am starting to think that we may consider dropping REQ_FUA, if we have the option to manage the eMMC cache separately - no matter whether the eMMC reliable write command is supported or not. In this case, REQ_FLUSH is sufficient and also a better match to what we really can support.

Hi Ulf,
As to dropping REQ_FUA, I don't know if it is a good idea, but generally we are facing three possible situations:

Quoted 6 lines

1. If both cache and reliable-write are available, both REQ_FUA and REQ_FLUSH can be supported at the same time. In this case, with available cache, the behavior of reliable-write is to write eMMC while skipping cache, which is consistent with the current kernel's definition of REQ_FUA. What's more, most eMMCs now support both cache and reliable-write command.

Yes, this seems reasonable.

Quoted 4 lines

2. If only reliable-write is available, REQ_FUA should not be supported, which is consistent with the current standard in another way. But I don't think eMMCs that only support reliable-write can be easily found nowadays.

Quoted 4 lines

If we drop REQ_FUA for this case, I am worried that we might break use cases for those older eMMC devices.

So, no, let's keep REQ_FUA and REQ_FLUSH if reliable-write is supported.

OK. Let's keep them.

Quoted 19 lines
Quoted 3 lines

3. If only cache is available, we just use REQ_FLUSH. It is not in conflict with keeping REQ_FUA.

Right.

Quoted 5 lines

Maybe, is it more reasonable to reserve FUA and use if/else to pick it up or down, considering the compatibility? I mean, in most cases, FUA and FLUSH are complementary. So it seems more feasible with branch to choose.

Let's summarize what I think we should do then:

  if (reliable-write supported) {
      enable_fua = true;
      enable_cache = true;
}
Quoted 6 lines

if (mmc_cache_enabled)

      enable_cache = true;

blk_queue_write_cache(md->queue.queue, enable_cache, enable_fua);

Does this seem reasonable to you?

Yes. Let me attach the complete code here:

     if (md->flags & MMC_BLK_CMD23 &&
         ((card->ext_csd.rel_param & EXT_CSD_WR_REL_PARAM_EN) ||
         card->ext_csd.rel_sectors)) {
         md->flags |= MMC_BLK_REL_WR;
         enable_fua = true;
         enable_cache = true;
     }

     if (mmc_cache_enabled(card->host))
         enable_cache = true;

     blk_queue_write_cache(md->queue.queue, enable_cache, enable_fua);
Quoted 1 line

If this is good, I'll submit a patch-v2 soon.

Yes, this looks good to me!

Kind regards
Uffe

#17 lore raw
In reply to Adrian Hunter
Quoted 24 lines · Christian Löhle
Quoted 5 lines
Quoted 2 lines

So we are not going to let the block layer know about SD cache? Or is it a separate change?

I have some code for this laying around, but as it requires reading, parsing and writing Function Registers, in particular PEH, it's a lot of

boilerplate code to get the functionality, but I'll clean it up and send a patch in the coming weeks.

Quoted 1 line

We have the sd cache flush. We would presumably just need to call blk_queue_write_cache() for the !mmc_card_mmc(card) case e.g.

        if (mmc_has_reliable_write(card)) {
                md->flags |= MMC_BLK_REL_WR;
                enable_fua = true;
        }

        if (mmc_cache_enabled(card->host))
                enable_cache = true;

        blk_queue_write_cache(md->queue.queue, enable_cache, enable_fua);
Quoted 1 line

Avri, were you objecting to that?

Hi, Sorry for my late response.
Yes - That's fine.

Thanks,
Avri

Quoted 57 lines
Quoted 14 lines

From: Adrian Hunter <[email protected]>
Sent: Wednesday, March 16, 2022 12:28 PM
To: Avri Altman; Michael Wu; [email protected]; [email protected]; [email protected]
Cc: [email protected]; [email protected]; allwinner-opensource-support
Subject: Re: [PATCH] mmc: block: enable cache-flushing when mmc cache is on

Quoted 12 lines
Quoted 11 lines · Avri Altman
Quoted 11 lines

Hi Avril & Adrian,
Thanks for your efforts. Could we have an agreement now --

1. enabling-cache and cmd23/reliable-write should be independent;

Here's what I found in the spec JESD84-B51:

Quoted 5 lines

6.6.31 Cache
Caching of data shall apply only for the single block read/write(CMD17/24), pre-defined multiple block read/write(CMD23+CMD18/25) and open ended multiple block read/write(CMD18/25+CMD12) commands and excludes any other

access

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 1 line

e.g., to the register space(e.g., CMD6).

Which means with CMD18/25+CMD12 (without using CMD23), the cache

can

Quoted 27 lines
Quoted 19 lines
Quoted 2 lines

also be enabled. Maybe this could be an evidence of the independence between enabling-cache and cmd23/reliable-write?

Thanks,
Avri

Quoted 12 lines

2. We don't consider supporting SD in this change.

Quoted 7 lines · Avri Altman
Quoted 6 lines

Here is what our SD system guys wrote:
" In SD we don’t support reliable write and this eMMC driver may not be utilizing the cache feature we added in SD5.0.
   The method of cache flush is different between SD and eMMC."

So adding SD seems to be out of scope of this change.

Is there anything else I can do about this patch? Thanks again.

So we are not going to let the block layer know about SD cache? Or is it a separate change?

    =

Hyperstone GmbH | Reichenaustr. 39a | 78467 Konstanz Managing
Director: Dr. Jan Peter Berns.
Commercial register of local courts: Freiburg HRB381782

#18 lore raw
In reply to Michael Wu
Quoted 6 lines
Quoted 2 lines

Seems like we should inform block layer about SD card cache also

I saw another mail by Avri Altman, which says few days will be needed to ask internally. Shall I wait or make another change here on 'inform block layer about SD card cache'?

Here is what our SD system guys wrote:
" In SD we don’t support reliable write and this eMMC driver may not be utilizing the cache feature we added in SD5.0.
The method of cache flush is different between SD and eMMC."

So adding SD seems to be out of scope of this change.

Thanks,
Avri

Quoted 7 lines
Quoted 3 lines
Quoted 3 lines

     string_get_size((u64)size, 512, STRING_UNITS_2,

--
Best Regards,
Michael Wu

#19 lore raw
In reply to Michael Wu

Hi,

Quoted 7 lines

The mmc core enable cache on default. But it only enables cache-flushing when host supports cmd23 and eMMC supports reliable write.
For hosts which do not support cmd23 or eMMCs which do not support reliable write, the cache can not be flushed by sync command. This may leads to cache data lost.
This patch enables cache-flushing as long as cache is enabled, no matter host supports cmd23 and/or eMMC supports reliable write or not.

I looked in the spec and indeed couldn't find why enabling cache is dependent of cmd23/reliable write. Nor I was able to find the original commit log.

Please allow few days to ask internally.

Thanks,
Avri

#20 lore raw
Quoted 10 lines · Avri Altman

Hi,

Quoted 7 lines

The mmc core enable cache on default. But it only enables cache-flushing when host supports cmd23 and eMMC supports reliable write.
For hosts which do not support cmd23 or eMMCs which do not support reliable write, the cache can not be flushed by sync command. This may leads to cache data lost.
This patch enables cache-flushing as long as cache is enabled, no matter host supports cmd23 and/or eMMC supports reliable write or not.

I looked in the spec and indeed couldn't find why enabling cache is dependent of cmd23/reliable write. Nor I was able to find the original commit log.

Reliable write was added first, so it might have been an oversight:

commit 881d1c25f765938a95def5afe39486ce39f9fc96

Author: Seungwon Jeon <[email protected]>
Date:   Fri Oct 14 14:03:21 2011 +0900

mmc: core: Add cache control for eMMC4.5 device

This patch adds cache feature of eMMC4.5 Spec.
If device supports cache capability, host can utilize some specific operations.

Quoted 5 lines

Please allow few days to ask internally.

Thanks,
Avri