Skip to contentSkip to conversation

block layer change necessary for bcache zoned device support

RFC PATCH In review Coly Li 5 participants 26 messages May 16, 2020 – May 18, 2020

Subsystems (3)
lore
Share & tools
lore

Coly Li

author
#0 lore raw

v2 cover block layer change necessary for bcache zoned device support

Hi folks,

Recently I am working on supprt bcache to be created on zoned devices e.g. host managed SMR hard drives, then frequent random READ I/Os on these SMR drives can be accelerated.

To make the bcache code work correctly, there are some small but maybe important changes of block layer code are necessary.

Thanks for the review comments from Keith Busch, I fix the typo and post the v2 series for your review and comments.

Thank you all in advance.

Coly Li
---

Coly Li (4):
block: change REQ_OP_ZONE_RESET from 6 to 13
block: block: change REQ_OP_ZONE_RESET_ALL from 8 to 15
block: remove queue_is_mq restriction from blk_revalidate_disk_zones()
block: set bi_size to REQ_OP_ZONE_RESET bio

 block/blk-zoned.c         | 6 ++++--
 include/linux/blk_types.h | 8 ++++----
 2 files changed, 8 insertions(+), 6 deletions(-)
Signature · 1 line
2.25.0
author
#1 lore raw

RFC PATCH v2 1/4 block: change REQ_OP_ZONE_RESET from 6 to 13

For a zoned device, e.g. host managed SMR hard drive, REQ_OP_ZONE_RESET is to reset the LBA of a zone's write pointer back to the start LBA of this zone. After the write point is reset, all previously stored data in this zone is invalid and unaccessible anymore. Therefore, this op code changes on disk data, belongs to a WRITE request op code.

Current REQ_OP_ZONE_RESET is defined as number 6, but the convention of the op code is, READ requests are even numbers, and WRITE requests are odd numbers. See how op_is_write defined,

397 static inline bool op_is_write(unsigned int op) 398 {
399 return (op & 1);
400 }

When create bcache device on top of the zoned SMR drive, bcache driver has to handle the REQ_OP_ZONE_RESET bio by invalidate all cached data belongs to the LBA range of the restting zone. A wrong op code value will make the this zone management bio goes into bcache' read requests code path and causes undefined result.

This patch changes REQ_OP_ZONE_RESET value from 6 to 13, the new odd number will make bcache driver handle this zone management bio properly in write requests code path.

Fixes 87374179c535 ("block: add a proper block layer data direction encoding")
Cc Christoph Hellwig <[email protected]>
Cc Damien Le Moal <[email protected]>
Cc Hannes Reinecke <[email protected]>
Cc Jens Axboe <[email protected]>
Cc Johannes Thumshirn <[email protected]>
Cc Keith Busch <[email protected]>
Cc Shaun Tancheff <[email protected]>
---
 include/linux/blk_types.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
include/linux/blk_types.h +2 -2
index 31eb92876be7..8f7bc15a6de3 100644
@@ -282,8 +282,6 @@ enum req_opf {
 	REQ_OP_DISCARD		= 3,
 	/* securely erase sectors */
 	REQ_OP_SECURE_ERASE	= 5,
-	/* reset a zone write pointer */
-	REQ_OP_ZONE_RESET	= 6,
 	/* write the same sector many times */
 	REQ_OP_WRITE_SAME	= 7,
 	/* reset all the zone present on the device */
@@ -296,6 +294,8 @@ enum req_opf {
 	REQ_OP_ZONE_CLOSE	= 11,
 	/* Transition a zone to full */
 	REQ_OP_ZONE_FINISH	= 12,
+	/* reset a zone write pointer */
+	REQ_OP_ZONE_RESET	= 13,
 
 	/* SCSI passthrough using struct scsi_request */
 	REQ_OP_SCSI_IN		= 32,
Signature · 1 line
2.25.0
#2 lore raw

Re: [RFC PATCH v2 1/4] block: change REQ_OP_ZONE_RESET from 6 to 13

include/linux/blk_types.h enum req_opf { · line 298

Quoted 3 lines · Coly Li

REQ_OP_ZONE_FINISH = 12,

include/linux/blk_types.h +2 -0
+	/* reset a zone write pointer */
+	REQ_OP_ZONE_RESET	= 13,

I think 13 is taken, I found this in block tree branch :- for-5.8/block

316         REQ_OP_ZONE_FINISH      = 12,
317         /* write data at the current zone write pointer */
318         REQ_OP_ZONE_APPEND      = 13,
author
#3 lore raw

Re: [RFC PATCH v2 1/4] block: change REQ_OP_ZONE_RESET from 6 to 13

Quoted 8 lines · Chaitanya Kulkarni

include/linux/blk_types.h enum req_opf { · line 298

Quoted 3 lines · Coly Li

REQ_OP_ZONE_FINISH = 12,

include/linux/blk_types.h +2 -0
+	/* reset a zone write pointer */
+	REQ_OP_ZONE_RESET	= 13,

I think 13 is taken, I found this in block tree branch :- for-5.8/block

Quoted 4 lines
316         REQ_OP_ZONE_FINISH      = 12,
317         /* write data at the current zone write pointer */
318         REQ_OP_ZONE_APPEND      = 13,

Aha, sure. Then I have to look at 15.

Thanks for the hint.

Coly Li

#4 lore raw

Re: [RFC PATCH v2 1/4] block: change REQ_OP_ZONE_RESET from 6 to 13

In reply to Coly Li
Quoted 9 lines · Coly Li

For a zoned device, e.g. host managed SMR hard drive, REQ_OP_ZONE_RESET is to reset the LBA of a zone's write pointer back to the start LBA of this zone. After the write point is reset, all previously stored data in this zone is invalid and unaccessible anymore. Therefore, this op code changes on disk data, belongs to a WRITE request op code.

Current REQ_OP_ZONE_RESET is defined as number 6, but the convention of the op code is, READ requests are even numbers, and WRITE requests are odd numbers. See how op_is_write defined,

The convention is all about data transfer, and zone reset does not transfer any data.

author
#5 lore raw

Re: [RFC PATCH v2 1/4] block: change REQ_OP_ZONE_RESET from 6 to 13

Quoted 7 lines · Christoph Hellwig
Quoted 6 lines · Coly Li

For a zoned device, e.g. host managed SMR hard drive, REQ_OP_ZONE_RESET is to reset the LBA of a zone's write pointer back to the start LBA of this zone. After the write point is reset, all previously stored data in this zone is invalid and unaccessible anymore. Therefore, this op code changes on disk data, belongs to a WRITE request op code.

Quoted 7 lines
Quoted 3 lines

Current REQ_OP_ZONE_RESET is defined as number 6, but the convention of the op code is, READ requests are even numbers, and WRITE requests are odd numbers. See how op_is_write defined,

The convention is all about data transfer, and zone reset does not transfer any data.

Yes you are right, just like REQ_OP_DISCARD which does not transfer any data but changes the data on device. If the request changes the stored data, it does transfer data.

This is why in patch "block: set bi_size to REQ_OP_ZONE_RESET bio" I set bi_size to the REQ_OP_ZONE_RESET and REQ_OP_ZONE_RESET_ALL bios.

Coly Li

#6 lore raw

Re: [RFC PATCH v2 1/4] block: change REQ_OP_ZONE_RESET from 6 to 13

Quoted 3 lines · Coly Li

Yes you are right, just like REQ_OP_DISCARD which does not transfer any data but changes the data on device. If the request changes the stored data, it does transfer data.

REQ_OP_DISCARD is a special case, because most implementation end up transferring data, it just gets attached in the low-level driver.

author
#7 lore raw

Re: [RFC PATCH v2 1/4] block: change REQ_OP_ZONE_RESET from 6 to 13

Quoted 8 lines · Christoph Hellwig
Quoted 3 lines · Coly Li

Yes you are right, just like REQ_OP_DISCARD which does not transfer any data but changes the data on device. If the request changes the stored data, it does transfer data.

REQ_OP_DISCARD is a special case, because most implementation end up transferring data, it just gets attached in the low-level driver.

Yes, REQ_OP_ZONE_RESET and REQ_OP_ZONE_RESET_ALL are quite similar to REQ_OP_DISCARD. Data read from the LBA range of reset zone is not suggested and the content is undefined.

For bcache, similar to REQ_OP_DISCARD, REQ_OP_ZONE_RESET and REQ_OP_ZONE_RESET_ALL are handled in same way: If the backing device supports discard/zone_reset, and the operation successes, then cached data on SSD covered by the LBA range should be invalid, otherwise users will read outdated and garbage data.

We should treat REQ_OP_ZONE_RESET and REQ_OP_ZONE_RESET_ALL being in WRITE direction.

Coly Li

#8 lore raw

Re: [RFC PATCH v2 1/4] block: change REQ_OP_ZONE_RESET from 6 to 13

Quoted 14 lines · Coly Li
Quoted 8 lines · Christoph Hellwig
Quoted 3 lines · Coly Li

Yes you are right, just like REQ_OP_DISCARD which does not transfer any data but changes the data on device. If the request changes the stored data, it does transfer data.

REQ_OP_DISCARD is a special case, because most implementation end up transferring data, it just gets attached in the low-level driver.

Yes, REQ_OP_ZONE_RESET and REQ_OP_ZONE_RESET_ALL are quite similar to REQ_OP_DISCARD. Data read from the LBA range of reset zone is not suggested and the content is undefined.

Quoted 8 lines

For bcache, similar to REQ_OP_DISCARD, REQ_OP_ZONE_RESET and REQ_OP_ZONE_RESET_ALL are handled in same way: If the backing device supports discard/zone_reset, and the operation successes, then cached data on SSD covered by the LBA range should be invalid, otherwise users will read outdated and garbage data.

We should treat REQ_OP_ZONE_RESET and REQ_OP_ZONE_RESET_ALL being in WRITE direction.

No, the difference is that the underlying SCSI/ATA/NVMe command tend to usually actually transfer data. Take a look at the special_vec field in struct request and the RQF_SPECIAL_PAYLOAD flag.

author
#9 lore raw

Re: [RFC PATCH v2 1/4] block: change REQ_OP_ZONE_RESET from 6 to 13

Quoted 21 lines · Christoph Hellwig
Quoted 20 lines · Coly Li
Quoted 8 lines · Christoph Hellwig
Quoted 3 lines · Coly Li

Yes you are right, just like REQ_OP_DISCARD which does not transfer any data but changes the data on device. If the request changes the stored data, it does transfer data.

REQ_OP_DISCARD is a special case, because most implementation end up transferring data, it just gets attached in the low-level driver.

Yes, REQ_OP_ZONE_RESET and REQ_OP_ZONE_RESET_ALL are quite similar to REQ_OP_DISCARD. Data read from the LBA range of reset zone is not suggested and the content is undefined.

For bcache, similar to REQ_OP_DISCARD, REQ_OP_ZONE_RESET and REQ_OP_ZONE_RESET_ALL are handled in same way: If the backing device supports discard/zone_reset, and the operation successes, then cached data on SSD covered by the LBA range should be invalid, otherwise users will read outdated and garbage data.

Quoted 7 lines
Quoted 2 lines

We should treat REQ_OP_ZONE_RESET and REQ_OP_ZONE_RESET_ALL being in WRITE direction.

No, the difference is that the underlying SCSI/ATA/NVMe command tend to usually actually transfer data. Take a look at the special_vec field in struct request and the RQF_SPECIAL_PAYLOAD flag.

Then bio_data_dir() will be problematic, as it is defined,
52 /*
53 * Return the data direction, READ or WRITE. 54 */
55 #define bio_data_dir(bio) \
56 (op_is_write(bio_op(bio)) ? WRITE : READ)

For the REQ_OP_ZONE_RESET bio, bio_data_dir() will report it as READ.

Since the author is you, how to fix this ?

Coly Li

#10 lore raw

Re: [RFC PATCH v2 1/4] block: change REQ_OP_ZONE_RESET from 6 to 13

Quoted 37 lines · Coly Li
Quoted 21 lines · Christoph Hellwig
Quoted 20 lines · Coly Li
Quoted 8 lines · Christoph Hellwig
Quoted 3 lines · Coly Li

Yes you are right, just like REQ_OP_DISCARD which does not transfer any data but changes the data on device. If the request changes the stored data, it does transfer data.

REQ_OP_DISCARD is a special case, because most implementation end up transferring data, it just gets attached in the low-level driver.

Yes, REQ_OP_ZONE_RESET and REQ_OP_ZONE_RESET_ALL are quite similar to REQ_OP_DISCARD. Data read from the LBA range of reset zone is not suggested and the content is undefined.

For bcache, similar to REQ_OP_DISCARD, REQ_OP_ZONE_RESET and REQ_OP_ZONE_RESET_ALL are handled in same way: If the backing device supports discard/zone_reset, and the operation successes, then cached data on SSD covered by the LBA range should be invalid, otherwise users will read outdated and garbage data.

Quoted 7 lines
Quoted 2 lines

We should treat REQ_OP_ZONE_RESET and REQ_OP_ZONE_RESET_ALL being in WRITE direction.

No, the difference is that the underlying SCSI/ATA/NVMe command tend to usually actually transfer data. Take a look at the special_vec field in struct request and the RQF_SPECIAL_PAYLOAD flag.

Then bio_data_dir() will be problematic, as it is defined,
52 /*
53 * Return the data direction, READ or WRITE. 54 */
55 #define bio_data_dir(bio) \
56 (op_is_write(bio_op(bio)) ? WRITE : READ)

Quoted 4 lines

For the REQ_OP_ZONE_RESET bio, bio_data_dir() will report it as READ.

Since the author is you, how to fix this ?

Well, but I do think that Coly is right in that bio_data_dir() is very unconvincing, or at least improperly documented.

I can't quite follow Christoph's argument in that bio_data_dir() reflects whether data is _actually_ transferred (if so, why would REQ_OP_ZONE_CLOSE() be marked as WRITE?) However, in most cases bio_data_dir() will only come into effect if there are attached bvecs.

So the _correct_ usage for bio_data_dir() would be to check if there is data attached to the bio (eg by using bio_has_data()), and only call bio_data_dir() if that returns true. For false you'd need to add a switch evaluating the individual commands.

And, btw, bio_has_data() needs updating, too; all the zone commands are missing from there, too.

Cheers,

Hannes

Signature · 4 lines
Dr. Hannes Reinecke            Teamlead Storage & Networking
[email protected]                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer
#11 lore raw

Re: [RFC PATCH v2 1/4] block: change REQ_OP_ZONE_RESET from 6 to 13

Quoted 52 lines · Hannes Reinecke
Quoted 37 lines · Coly Li
Quoted 21 lines · Christoph Hellwig
Quoted 20 lines · Coly Li
Quoted 8 lines · Christoph Hellwig
Quoted 3 lines · Coly Li

Yes you are right, just like REQ_OP_DISCARD which does not transfer any data but changes the data on device. If the request changes the stored data, it does transfer data.

REQ_OP_DISCARD is a special case, because most implementation end up transferring data, it just gets attached in the low-level driver.

Yes, REQ_OP_ZONE_RESET and REQ_OP_ZONE_RESET_ALL are quite similar to REQ_OP_DISCARD. Data read from the LBA range of reset zone is not suggested and the content is undefined.

For bcache, similar to REQ_OP_DISCARD, REQ_OP_ZONE_RESET and REQ_OP_ZONE_RESET_ALL are handled in same way: If the backing device supports discard/zone_reset, and the operation successes, then cached data on SSD covered by the LBA range should be invalid, otherwise users will read outdated and garbage data.

Quoted 7 lines
Quoted 2 lines

We should treat REQ_OP_ZONE_RESET and REQ_OP_ZONE_RESET_ALL being in WRITE direction.

No, the difference is that the underlying SCSI/ATA/NVMe command tend to usually actually transfer data. Take a look at the special_vec field in struct request and the RQF_SPECIAL_PAYLOAD flag.

Then bio_data_dir() will be problematic, as it is defined,
52 /*
53 * Return the data direction, READ or WRITE. 54 */
55 #define bio_data_dir(bio) \
56 (op_is_write(bio_op(bio)) ? WRITE : READ)

Quoted 4 lines

For the REQ_OP_ZONE_RESET bio, bio_data_dir() will report it as READ.

Since the author is you, how to fix this ?

Well, but I do think that Coly is right in that bio_data_dir() is very unconvincing, or at least improperly documented.

I can't quite follow Christoph's argument in that bio_data_dir() reflects whether data is _actually_ transferred (if so, why would REQ_OP_ZONE_CLOSE() be marked as WRITE?)
However, in most cases bio_data_dir() will only come into effect if there are attached bvecs.

Quoted 7 lines

So the _correct_ usage for bio_data_dir() would be to check if there is data attached to the bio (eg by using bio_has_data()), and only call bio_data_dir() if that returns true. For false you'd need to add a switch evaluating the individual commands.

And, btw, bio_has_data() needs updating, too; all the zone commands are missing from there, too.

Not really. They all have 0 size, so bio_has_data() always return false for zone management ops. It will return true only for report zones.

Quoted 5 lines

Cheers,

Hannes

Signature · 2 lines
Damien Le Moal
Western Digital Research
#12 lore raw

Re: [RFC PATCH v2 1/4] block: change REQ_OP_ZONE_RESET from 6 to 13

In reply to Coly Li
Quoted 52 lines · Coly Li

For a zoned device, e.g. host managed SMR hard drive, REQ_OP_ZONE_RESET is to reset the LBA of a zone's write pointer back to the start LBA of this zone. After the write point is reset, all previously stored data in this zone is invalid and unaccessible anymore. Therefore, this op code changes on disk data, belongs to a WRITE request op code.

Current REQ_OP_ZONE_RESET is defined as number 6, but the convention of the op code is, READ requests are even numbers, and WRITE requests are odd numbers. See how op_is_write defined,

397 static inline bool op_is_write(unsigned int op) 398 {
399 return (op & 1);
400 }

When create bcache device on top of the zoned SMR drive, bcache driver has to handle the REQ_OP_ZONE_RESET bio by invalidate all cached data belongs to the LBA range of the restting zone. A wrong op code value will make the this zone management bio goes into bcache' read requests code path and causes undefined result.

This patch changes REQ_OP_ZONE_RESET value from 6 to 13, the new odd number will make bcache driver handle this zone management bio properly in write requests code path.

Fixes 87374179c535 ("block: add a proper block layer data direction encoding")
Cc Christoph Hellwig <[email protected]>
Cc Damien Le Moal <[email protected]>
Cc Hannes Reinecke <[email protected]>
Cc Jens Axboe <[email protected]>
Cc Johannes Thumshirn <[email protected]>
Cc Keith Busch <[email protected]>
Cc Shaun Tancheff <[email protected]>
---
 include/linux/blk_types.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
include/linux/blk_types.h +0 -2
index 31eb92876be7..8f7bc15a6de3 100644
@@ -282,8 +282,6 @@ enum req_opf {
 	REQ_OP_DISCARD		= 3,
 	/* securely erase sectors */
 	REQ_OP_SECURE_ERASE	= 5,
-	/* reset a zone write pointer */
-	REQ_OP_ZONE_RESET	= 6,
 	/* write the same sector many times */
 	REQ_OP_WRITE_SAME	= 7,
 	/* reset all the zone present on the device */

include/linux/blk_types.h enum req_opf { · line 298

Quoted 5 lines
include/linux/blk_types.h +2 -0
@@ -296,6 +294,8 @@ enum req_opf {
 	REQ_OP_ZONE_CLOSE	= 11,
 	/* Transition a zone to full */
 	REQ_OP_ZONE_FINISH	= 12,
+	/* reset a zone write pointer */
+	REQ_OP_ZONE_RESET	= 13,

As Chaitanya pointed out, this is already used. Please rebase on Jens block/for-5.8/block branch.

I do not see any problem with this change, but as Christoph commented, since zone reset does not transfer any data, I do not really see the necessity for this. Zone reset is indeed data-destructive, but it is not a "write" command. But following DISCARD and having the op number as an odd number is OK I think.

Quoted 4 lines
include/linux/blk_types.h +0 -0
 	/* SCSI passthrough using struct scsi_request */
 	REQ_OP_SCSI_IN		= 32,
 
Signature · 2 lines
Damien Le Moal
Western Digital Research
#13 lore raw

Re: [RFC PATCH v2 1/4] block: change REQ_OP_ZONE_RESET from 6 to 13

Quoted 7 lines · Damien Le Moal

As Chaitanya pointed out, this is already used. Please rebase on Jens block/for-5.8/block branch.

I do not see any problem with this change, but as Christoph commented, since zone reset does not transfer any data, I do not really see the necessity for this. Zone reset is indeed data-destructive, but it is not a "write" command. But following DISCARD and having the op number as an odd number is OK I think.

Let's keep reset consistent with discard and write-zeroes.

author
#14 lore raw

RFC PATCH v2 2/4 block: block: change REQ_OP_ZONE_RESET_ALL from 8 to 15

In reply to Coly Li

For a zoned device, e.g. host managed SMR hard drive, the request op REQ_OP_ZONE_RESET_ALL is to reset LBA of all zones' writer pointers to the start LBA of the zone they belong to. After all the write pointers are reset, all previously stored data is invalid and unaccessible. Therefore this op code changes on-disk data, belongs to WRITE request op code.

Similar to the problem of REQ_OP_ZONE_RESET, REQ_OP_ZONE_RESET_ALL is even number 8, it means bios with REQ_OP_ZONE_RESET_ALL in bio->bi_opf will be treated as READ request by op_is_write().

Same problem will happen when bcache device is created on top of zoned device like host managed SMR hard drive, bcache driver should invalid all cached data for the REQ_OP_ZONE_RESET_ALL request, but this zone management bio is mistakenly treated as READ request and go into bcache read code path, which will cause undefined behavior.

This patch changes REQ_OP_ZONE_RESET_ALL value from 8 to 15, this new odd number will make bcache driver handle this zone management bio in write request code path properly.

Fixes e84e8f066395 ("block: add req op to reset all zones and flag")
Cc Chaitanya Kulkarni <[email protected]>
Cc Damien Le Moal <[email protected]>
Cc Hannes Reinecke <[email protected]>
Cc Jens Axboe <[email protected]>
Cc Johannes Thumshirn <[email protected]>
Cc Keith Busch <[email protected]>

---
Changelog:
v2: fix typo for REQ_OP_ZONE_RESET_ALL.
v1: initial version.

 include/linux/blk_types.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
include/linux/blk_types.h +2 -2
index 8f7bc15a6de3..618032fa1b29 100644
@@ -284,8 +284,6 @@ enum req_opf {
 	REQ_OP_SECURE_ERASE	= 5,
 	/* write the same sector many times */
 	REQ_OP_WRITE_SAME	= 7,
-	/* reset all the zone present on the device */
-	REQ_OP_ZONE_RESET_ALL	= 8,
 	/* write the zero filled sector many times */
 	REQ_OP_WRITE_ZEROES	= 9,
 	/* Open a zone */
@@ -296,6 +294,8 @@ enum req_opf {
 	REQ_OP_ZONE_FINISH	= 12,
 	/* reset a zone write pointer */
 	REQ_OP_ZONE_RESET	= 13,
+	/* reset all the zone present on the device */
+	REQ_OP_ZONE_RESET_ALL	= 15,
 
 	/* SCSI passthrough using struct scsi_request */
 	REQ_OP_SCSI_IN		= 32,
Signature · 1 line
2.25.0
#15 lore raw

Re: [RFC PATCH v2 2/4] block: block: change REQ_OP_ZONE_RESET_ALL from 8 to 15

Quoted 51 lines · Coly Li

For a zoned device, e.g. host managed SMR hard drive, the request op REQ_OP_ZONE_RESET_ALL is to reset LBA of all zones' writer pointers to the start LBA of the zone they belong to. After all the write pointers are reset, all previously stored data is invalid and unaccessible. Therefore this op code changes on-disk data, belongs to WRITE request op code.

Similar to the problem of REQ_OP_ZONE_RESET, REQ_OP_ZONE_RESET_ALL is even number 8, it means bios with REQ_OP_ZONE_RESET_ALL in bio->bi_opf will be treated as READ request by op_is_write().

Same problem will happen when bcache device is created on top of zoned device like host managed SMR hard drive, bcache driver should invalid all cached data for the REQ_OP_ZONE_RESET_ALL request, but this zone management bio is mistakenly treated as READ request and go into bcache read code path, which will cause undefined behavior.

This patch changes REQ_OP_ZONE_RESET_ALL value from 8 to 15, this new odd number will make bcache driver handle this zone management bio in write request code path properly.

Fixes e84e8f066395 ("block: add req op to reset all zones and flag")
Cc Chaitanya Kulkarni <[email protected]>
Cc Damien Le Moal <[email protected]>
Cc Hannes Reinecke <[email protected]>
Cc Jens Axboe <[email protected]>
Cc Johannes Thumshirn <[email protected]>
Cc Keith Busch <[email protected]>

---
Changelog:
v2: fix typo for REQ_OP_ZONE_RESET_ALL.
v1: initial version.

 include/linux/blk_types.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
include/linux/blk_types.h +0 -2
index 8f7bc15a6de3..618032fa1b29 100644
@@ -284,8 +284,6 @@ enum req_opf {
 	REQ_OP_SECURE_ERASE	= 5,
 	/* write the same sector many times */
 	REQ_OP_WRITE_SAME	= 7,
-	/* reset all the zone present on the device */
-	REQ_OP_ZONE_RESET_ALL	= 8,
 	/* write the zero filled sector many times */
 	REQ_OP_WRITE_ZEROES	= 9,
 	/* Open a zone */

include/linux/blk_types.h enum req_opf { · line 298

Quoted 5 lines
include/linux/blk_types.h +2 -0
@@ -296,6 +294,8 @@ enum req_opf {
 	REQ_OP_ZONE_FINISH	= 12,
 	/* reset a zone write pointer */
 	REQ_OP_ZONE_RESET	= 13,
+	/* reset all the zone present on the device */
+	REQ_OP_ZONE_RESET_ALL	= 15,

Same comment as for patch 1. And you can probably squash this patch into patch 1 to avoid repeating mostly the same explanation twice in the commit message. REQ_OP_ZONE_RESET and REQ_OP_ZONE_RESET_ALL are very closely related. The latter may actually be processed using the former anyway, so the op number change for both should be a single patch to be consistent.

Quoted 4 lines
include/linux/blk_types.h +0 -0
 	/* SCSI passthrough using struct scsi_request */
 	REQ_OP_SCSI_IN		= 32,
 
Signature · 2 lines
Damien Le Moal
Western Digital Research
author
#16 lore raw

RFC PATCH v2 3/4 block: remove queue_is_mq restriction from blk_revalidate_disk_zones()

In reply to Coly Li

The bcache driver is bio based and NOT request based multiqueued driver, if a zoned SMR hard drive is used as backing device of a bcache device, calling blk_revalidate_disk_zones() for the bcache device will fail due to the following check in blk_revalidate_disk_zones(),
478 if (WARN_ON_ONCE(!queue_is_mq(q)))
479 return -EIO;

Now bcache is able to export the zoned information from the underlying zoned SMR drives and format zonefs on top of a bcache device, the resitriction that a zoned device should be multiqueued is unnecessary for now.

Although in commit ae58954d8734c ("block: don't handle bio based drivers in blk_revalidate_disk_zones") it is said that bio based drivers should not call blk_revalidate_disk_zones() and just manually update their own q->nr_zones, but this is inaccurate. The bio based drivers also need to set their zone size and initialize bitmaps for cnv and seq zones, it is necessary to call blk_revalidate_disk_zones() for bio based drivers.

This patch removes the above queue_is_mq() restriction to permit bcache driver calls blk_revalidate_disk_zones() for bcache device zoned information initialization.

Fixes ae58954d8734c ("block: don't handle bio based drivers in blk_revalidate_disk_zones")
Cc Christoph Hellwig <[email protected]>
Cc Damien Le Moal <[email protected]>
Cc Hannes Reinecke <[email protected]>
Cc Jens Axboe <[email protected]>
Cc Keith Busch <[email protected]>
---
 block/blk-zoned.c | 2 --
 1 file changed, 2 deletions(-)
block/blk-zoned.c +0 -2
index f87956e0dcaf..1e0708c68267 100644
@@ -475,8 +475,6 @@ int blk_revalidate_disk_zones(struct gendisk *disk)
 
 	if (WARN_ON_ONCE(!blk_queue_is_zoned(q)))
 		return -EIO;
-	if (WARN_ON_ONCE(!queue_is_mq(q)))
-		return -EIO;
 
 	/*
 	 * Ensure that all memory allocations in this context are done as if
Signature · 1 line
2.25.0
#17 lore raw

Re: [RFC PATCH v2 3/4] block: remove queue_is_mq restriction from blk_revalidate_disk_zones()

Quoted 12 lines · Coly Li

The bcache driver is bio based and NOT request based multiqueued driver, if a zoned SMR hard drive is used as backing device of a bcache device, calling blk_revalidate_disk_zones() for the bcache device will fail due to the following check in blk_revalidate_disk_zones(),
478 if (WARN_ON_ONCE(!queue_is_mq(q)))
479 return -EIO;

Now bcache is able to export the zoned information from the underlying zoned SMR drives and format zonefs on top of a bcache device, the resitriction that a zoned device should be multiqueued is unnecessary for now.

Quoted 6 lines

Although in commit ae58954d8734c ("block: don't handle bio based drivers in blk_revalidate_disk_zones") it is said that bio based drivers should not call blk_revalidate_disk_zones() and just manually update their own q->nr_zones, but this is inaccurate. The bio based drivers also need to set their zone size and initialize bitmaps for cnv and seq zones, it is necessary to call blk_revalidate_disk_zones() for bio based drivers.

Why would you need these bitmaps for bcache? There is no reason to serialize requests for stacking drivers, and you can already derive if a zone is sequential or not from whatever internal information you use.

So without a user that actually makes sense: NAK.

author
#18 lore raw

Re: [RFC PATCH v2 3/4] block: remove queue_is_mq restriction from blk_revalidate_disk_zones()

Quoted 20 lines · Christoph Hellwig
Quoted 12 lines · Coly Li

The bcache driver is bio based and NOT request based multiqueued driver, if a zoned SMR hard drive is used as backing device of a bcache device, calling blk_revalidate_disk_zones() for the bcache device will fail due to the following check in blk_revalidate_disk_zones(),
478 if (WARN_ON_ONCE(!queue_is_mq(q)))
479 return -EIO;

Now bcache is able to export the zoned information from the underlying zoned SMR drives and format zonefs on top of a bcache device, the resitriction that a zoned device should be multiqueued is unnecessary for now.

Quoted 6 lines

Although in commit ae58954d8734c ("block: don't handle bio based drivers in blk_revalidate_disk_zones") it is said that bio based drivers should not call blk_revalidate_disk_zones() and just manually update their own q->nr_zones, but this is inaccurate. The bio based drivers also need to set their zone size and initialize bitmaps for cnv and seq zones, it is necessary to call blk_revalidate_disk_zones() for bio based drivers.

Quoted 7 lines

Why would you need these bitmaps for bcache? There is no reason to serialize requests for stacking drivers, and you can already derive if a zone is sequential or not from whatever internal information you use.

So without a user that actually makes sense: NAK.

It is OK for me to set the zone_nr and zone size without calling blk_revalidate_disk_zones().

Coly Li

#19 lore raw

Re: [RFC PATCH v2 3/4] block: remove queue_is_mq restriction from blk_revalidate_disk_zones()

Quoted 2 lines · Coly Li

It is OK for me to set the zone_nr and zone size without calling blk_revalidate_disk_zones().

Yes. And without having seen your code I'm pretty sure it should get simpler by not using blk_revalidate_disk_zones.

#20 lore raw

Re: [RFC PATCH v2 3/4] block: remove queue_is_mq restriction from blk_revalidate_disk_zones()

In reply to Coly Li
Quoted 26 lines · Coly Li
Quoted 25 lines · Christoph Hellwig
Quoted 12 lines · Coly Li

The bcache driver is bio based and NOT request based multiqueued driver, if a zoned SMR hard drive is used as backing device of a bcache device, calling blk_revalidate_disk_zones() for the bcache device will fail due to the following check in blk_revalidate_disk_zones(),
478 if (WARN_ON_ONCE(!queue_is_mq(q)))
479 return -EIO;

Now bcache is able to export the zoned information from the underlying zoned SMR drives and format zonefs on top of a bcache device, the resitriction that a zoned device should be multiqueued is unnecessary for now.

Quoted 6 lines

Although in commit ae58954d8734c ("block: don't handle bio based drivers in blk_revalidate_disk_zones") it is said that bio based drivers should not call blk_revalidate_disk_zones() and just manually update their own q->nr_zones, but this is inaccurate. The bio based drivers also need to set their zone size and initialize bitmaps for cnv and seq zones, it is necessary to call blk_revalidate_disk_zones() for bio based drivers.

Why would you need these bitmaps for bcache? There is no reason to serialize requests for stacking drivers, and you can already derive if a zone is sequential or not from whatever internal information you use.

Quoted 5 lines
Quoted 2 lines

So without a user that actually makes sense: NAK.

It is OK for me to set the zone_nr and zone size without calling blk_revalidate_disk_zones().

Yes, no problem with that.

This is how device mapper BIO based targets handle zoned devices. See dm_set_device_limits() which uses bdev_stack_limits() for handling chunk_sectors (zone size) and directly set q->limits.zoned to blk_queue_zoned_model(q) of the the underlying device. For the number of zones, see dm_table_set_restrictions() which uses blkdev_nr_zones(t->md->disk) to set q->nr_zones. Note that blkdev_nr_zones() uses the gendisk capacity of the logical device (the bcache device in your case) and the zone size, so both must be set first before calling blkdev_nr_zones().

Quoted 3 lines

Coly Li

Signature · 2 lines
Damien Le Moal
Western Digital Research
#21 lore raw

Re: [RFC PATCH v2 3/4] block: remove queue_is_mq restriction from blk_revalidate_disk_zones()

In reply to Coly Li
Quoted 39 lines · Coly Li

The bcache driver is bio based and NOT request based multiqueued driver, if a zoned SMR hard drive is used as backing device of a bcache device, calling blk_revalidate_disk_zones() for the bcache device will fail due to the following check in blk_revalidate_disk_zones(),
478 if (WARN_ON_ONCE(!queue_is_mq(q)))
479 return -EIO;

Now bcache is able to export the zoned information from the underlying zoned SMR drives and format zonefs on top of a bcache device, the resitriction that a zoned device should be multiqueued is unnecessary for now.

Although in commit ae58954d8734c ("block: don't handle bio based drivers in blk_revalidate_disk_zones") it is said that bio based drivers should not call blk_revalidate_disk_zones() and just manually update their own q->nr_zones, but this is inaccurate. The bio based drivers also need to set their zone size and initialize bitmaps for cnv and seq zones, it is necessary to call blk_revalidate_disk_zones() for bio based drivers.

This patch removes the above queue_is_mq() restriction to permit bcache driver calls blk_revalidate_disk_zones() for bcache device zoned information initialization.

Fixes ae58954d8734c ("block: don't handle bio based drivers in blk_revalidate_disk_zones")
Cc Christoph Hellwig <[email protected]>
Cc Damien Le Moal <[email protected]>
Cc Hannes Reinecke <[email protected]>
Cc Jens Axboe <[email protected]>
Cc Keith Busch <[email protected]>
---
 block/blk-zoned.c | 2 --
 1 file changed, 2 deletions(-)
block/blk-zoned.c +0 -0
index f87956e0dcaf..1e0708c68267 100644

block/blk-zoned.c int blk_revalidate_disk_zones(struct gendisk *disk) · line 479

Quoted 5 lines
block/blk-zoned.c +0 -2
@@ -475,8 +475,6 @@ int blk_revalidate_disk_zones(struct gendisk *disk)
 
 	if (WARN_ON_ONCE(!blk_queue_is_zoned(q)))
 		return -EIO;
-	if (WARN_ON_ONCE(!queue_is_mq(q)))
-		return -EIO;

Same comment as Christoph: BIO drivers can set nr_zones and chunk_sectors themselves, manually. They also do not need the bitmaps as BIO devices do not have an elevator, and as Christoph said, can (if needed) look at the bitmaps of the underlying device (e.g. for zone conv vs seq zone type differentiation).

So I do not think this patch is needed, and would only cause more memory consumption for no good reason that I can see.

Quoted 4 lines
block/blk-zoned.c +0 -0
 	/*
 	 * Ensure that all memory allocations in this context are done as if
 
Signature · 2 lines
Damien Le Moal
Western Digital Research
author
#22 lore raw

RFC PATCH v2 4/4 block: set bi_size to REQ_OP_ZONE_RESET bio

In reply to Coly Li

Now for zoned device, zone management ioctl commands are converted into zone management bios and handled by blkdev_zone_mgmt(). There are 4 zone management bios are handled, their op code is,
- REQ_OP_ZONE_RESET
Reset the zone's writer pointer and empty all previously stored data.
- REQ_OP_ZONE_OPEN
Open the zones in the specified sector range, no influence on data.
- REQ_OP_ZONE_CLOSE
Close the zones in the specified sector range, no influence on data.
- REQ_OP_ZONE_FINISH
Mark the zone as full, no influence on data.
All the zone management bios has 0 byte size, a.k.a their bi_size is 0.

Exept for REQ_OP_ZONE_RESET request, zero length bio works fine for other zone management bio, before the zoned device e.g. host managed SMR hard drive can be created as a bcache device.

When a bcache device (virtual block device to forward bios like md raid drivers) can be created on top of the zoned device, and a fast SSD is attached as a cache device, bcache driver may cache the frequent random READ requests on fast SSD to accelerate hot data READ performance.

When bcache driver receives a zone management bio for REQ_OP_ZONE_RESET op, while forwarding the request to underlying zoned device e.g. host managed SMR hard drive, it should also invalidate all cached data from SSD for the resetting zone. Otherwise bcache will continue provide the outdated cached data to READ request and cause potential data storage inconsistency and corruption.

In order to invalidate outdated data from SSD for the reset zone, bcache needs to know not only the start LBA but also the range length of the resetting zone. Otherwise, bcache won't be able to accurately invalidate the outdated cached data.

Is it possible to simply set the bi_size inside bcache driver? The answer is NO. Although every REQ_OP_ZONE_RESET bio has exact length as zone size or q->limits.chunk_sectors, it is possible that some other layer stacking block driver (in the future) exists between bcache driver and blkdev_zone_mgmt() where the zone management bio is made.

The best location to set bi_size is where the zone management bio is composed in blkdev_zone_mgmt(), then no matter how this bio is split before bcache driver receives it, bcache driver can always correctly invalidate the resetting range.

This patch sets the bi_size of REQ_OP_ZONE_RESET bio for each resetting zone. Here REQ_OP_ZONE_RESET_ALL is special whose bi_size should be set as capacity of whole drive size, then bcache can invalidate all cached data from SSD for the zoned backing device.

With this change, now bcache code can handle REQ_OP_ZONE_RESET bio in the way very similar to REQ_OP_DISCARD bio with very little change.

Cc Ajay Joshi <[email protected]>
Cc Chaitanya Kulkarni <[email protected]>
Cc Christoph Hellwig <[email protected]>
Cc Damien Le Moal <[email protected]>
Cc Hannes Reinecke <[email protected]>
Cc Johannes Thumshirn <[email protected]>
Cc Keith Busch <[email protected]>

---
Changelog:
v2: fix typo for REQ_OP_ZONE_RESET_ALL.
v1: initial version.

 block/blk-zoned.c | 4 ++++
 1 file changed, 4 insertions(+)
block/blk-zoned.c +4 -0
index 1e0708c68267..01d91314399b 100644
@@ -227,11 +227,15 @@ int blkdev_zone_mgmt(struct block_device *bdev, enum req_opf op,
 		if (op == REQ_OP_ZONE_RESET &&
 		    blkdev_allow_reset_all_zones(bdev, sector, nr_sectors)) {
 			bio->bi_opf = REQ_OP_ZONE_RESET_ALL;
+			bio->bi_iter.bi_sector = sector;
+			bio->bi_iter.bi_size = nr_sectors;
 			break;
 		}
 
 		bio->bi_opf = op | REQ_SYNC;
 		bio->bi_iter.bi_sector = sector;
+		if (op == REQ_OP_ZONE_RESET)
+			bio->bi_iter.bi_size = zone_sectors;
 		sector += zone_sectors;
 
 		/* This may take a while, so be nice to others */
Signature · 1 line
2.25.0
#23 lore raw

Re: [RFC PATCH v2 4/4] block: set bi_size to REQ_OP_ZONE_RESET bio

This looks ok, but I want to hear Damiens opinion.

#24 lore raw

Re: [RFC PATCH v2 4/4] block: set bi_size to REQ_OP_ZONE_RESET bio

In reply to Coly Li
Quoted 34 lines · Coly Li

Now for zoned device, zone management ioctl commands are converted into zone management bios and handled by blkdev_zone_mgmt(). There are 4 zone management bios are handled, their op code is,
- REQ_OP_ZONE_RESET
Reset the zone's writer pointer and empty all previously stored data.
- REQ_OP_ZONE_OPEN
Open the zones in the specified sector range, no influence on data.
- REQ_OP_ZONE_CLOSE
Close the zones in the specified sector range, no influence on data.
- REQ_OP_ZONE_FINISH
Mark the zone as full, no influence on data.
All the zone management bios has 0 byte size, a.k.a their bi_size is 0.

Exept for REQ_OP_ZONE_RESET request, zero length bio works fine for other zone management bio, before the zoned device e.g. host managed SMR hard drive can be created as a bcache device.

When a bcache device (virtual block device to forward bios like md raid drivers) can be created on top of the zoned device, and a fast SSD is attached as a cache device, bcache driver may cache the frequent random READ requests on fast SSD to accelerate hot data READ performance.

When bcache driver receives a zone management bio for REQ_OP_ZONE_RESET op, while forwarding the request to underlying zoned device e.g. host managed SMR hard drive, it should also invalidate all cached data from SSD for the resetting zone. Otherwise bcache will continue provide the outdated cached data to READ request and cause potential data storage inconsistency and corruption.

In order to invalidate outdated data from SSD for the reset zone, bcache needs to know not only the start LBA but also the range length of the resetting zone. Otherwise, bcache won't be able to accurately invalidate the outdated cached data.

Quoted 5 lines

Is it possible to simply set the bi_size inside bcache driver? The answer is NO. Although every REQ_OP_ZONE_RESET bio has exact length as zone size or q->limits.chunk_sectors, it is possible that some other layer stacking block driver (in the future) exists between bcache driver and blkdev_zone_mgmt() where the zone management bio is made.

But bcache "knows" what underlying devices it is using, right ? So getting the SMR drive zone size using blk_queue_zone_sectors(), bdev_zone_sectors() or by directly accessing q->limits->chunk_sectors should not be a problem at all, no ?

Quoted 10 lines

The best location to set bi_size is where the zone management bio is composed in blkdev_zone_mgmt(), then no matter how this bio is split before bcache driver receives it, bcache driver can always correctly invalidate the resetting range.

This patch sets the bi_size of REQ_OP_ZONE_RESET bio for each resetting zone. Here REQ_OP_ZONE_RESET_ALL is special whose bi_size should be set as capacity of whole drive size, then bcache can invalidate all cached data from SSD for the zoned backing device.

NACK. The problem here is that struct bio_vec bv_len field and struct bvec_iter bi_size field are both "unsigned int". So they can describe at most 4G x 512B = 2TB ranges. For REQ_OP_ZONE_RESET_ALL, that is simply way too small (we already are at 20TB capacity for SMR). One cannot issue a BIO with a length that is the entire disk capacity.

I really think that bcache should handle the zone management ops as a special case. I understand the goal of trying to minimize that by integrating them as much as possible into the regular bcache IO path, but that really seems dangerous to me. Since these operations will need remapping in bcache anyway (for handling the first hidden zone containing the super block), all special processing can be done under a single "if" which should not impact too much performance of regular read/write commands in the hot path.

Device mapper has such code: see __split_and_process_bio() and its use of op_is_zone_mgmt() function to handle zone management requests slightly differently than other BIOs. That remove the need for relying on op_is_write() direction (patch 1 and 2 in this series) for reset zones too, which in the end will I think make your bcache code a lot more solid.

Quoted 41 lines

With this change, now bcache code can handle REQ_OP_ZONE_RESET bio in the way very similar to REQ_OP_DISCARD bio with very little change.

Cc Ajay Joshi <[email protected]>
Cc Chaitanya Kulkarni <[email protected]>
Cc Christoph Hellwig <[email protected]>
Cc Damien Le Moal <[email protected]>
Cc Hannes Reinecke <[email protected]>
Cc Johannes Thumshirn <[email protected]>
Cc Keith Busch <[email protected]>

---
Changelog:
v2: fix typo for REQ_OP_ZONE_RESET_ALL.
v1: initial version.

 block/blk-zoned.c | 4 ++++
 1 file changed, 4 insertions(+)
block/blk-zoned.c +4 -0
index 1e0708c68267..01d91314399b 100644
@@ -227,11 +227,15 @@ int blkdev_zone_mgmt(struct block_device *bdev, enum req_opf op,
 		if (op == REQ_OP_ZONE_RESET &&
 		    blkdev_allow_reset_all_zones(bdev, sector, nr_sectors)) {
 			bio->bi_opf = REQ_OP_ZONE_RESET_ALL;
+			bio->bi_iter.bi_sector = sector;
+			bio->bi_iter.bi_size = nr_sectors;
 			break;
 		}
 
 		bio->bi_opf = op | REQ_SYNC;
 		bio->bi_iter.bi_sector = sector;
+		if (op == REQ_OP_ZONE_RESET)
+			bio->bi_iter.bi_size = zone_sectors;
 		sector += zone_sectors;
 
 		/* This may take a while, so be nice to others */
Signature · 2 lines
Damien Le Moal
Western Digital Research
author
#25 lore raw

Re: [RFC PATCH v2 4/4] block: set bi_size to REQ_OP_ZONE_RESET bio

Quoted 70 lines · Damien Le Moal
Quoted 34 lines · Coly Li

Now for zoned device, zone management ioctl commands are converted into zone management bios and handled by blkdev_zone_mgmt(). There are 4 zone management bios are handled, their op code is,
- REQ_OP_ZONE_RESET
Reset the zone's writer pointer and empty all previously stored data.
- REQ_OP_ZONE_OPEN
Open the zones in the specified sector range, no influence on data.
- REQ_OP_ZONE_CLOSE
Close the zones in the specified sector range, no influence on data.
- REQ_OP_ZONE_FINISH
Mark the zone as full, no influence on data.
All the zone management bios has 0 byte size, a.k.a their bi_size is 0.

Exept for REQ_OP_ZONE_RESET request, zero length bio works fine for other zone management bio, before the zoned device e.g. host managed SMR hard drive can be created as a bcache device.

When a bcache device (virtual block device to forward bios like md raid drivers) can be created on top of the zoned device, and a fast SSD is attached as a cache device, bcache driver may cache the frequent random READ requests on fast SSD to accelerate hot data READ performance.

When bcache driver receives a zone management bio for REQ_OP_ZONE_RESET op, while forwarding the request to underlying zoned device e.g. host managed SMR hard drive, it should also invalidate all cached data from SSD for the resetting zone. Otherwise bcache will continue provide the outdated cached data to READ request and cause potential data storage inconsistency and corruption.

In order to invalidate outdated data from SSD for the reset zone, bcache needs to know not only the start LBA but also the range length of the resetting zone. Otherwise, bcache won't be able to accurately invalidate the outdated cached data.

Quoted 5 lines

Is it possible to simply set the bi_size inside bcache driver? The answer is NO. Although every REQ_OP_ZONE_RESET bio has exact length as zone size or q->limits.chunk_sectors, it is possible that some other layer stacking block driver (in the future) exists between bcache driver and blkdev_zone_mgmt() where the zone management bio is made.

But bcache "knows" what underlying devices it is using, right ? So getting the SMR drive zone size using blk_queue_zone_sectors(), bdev_zone_sectors() or by directly accessing q->limits->chunk_sectors should not be a problem at all, no ?

Quoted 10 lines

The best location to set bi_size is where the zone management bio is composed in blkdev_zone_mgmt(), then no matter how this bio is split before bcache driver receives it, bcache driver can always correctly invalidate the resetting range.

This patch sets the bi_size of REQ_OP_ZONE_RESET bio for each resetting zone. Here REQ_OP_ZONE_RESET_ALL is special whose bi_size should be set as capacity of whole drive size, then bcache can invalidate all cached data from SSD for the zoned backing device.

NACK. The problem here is that struct bio_vec bv_len field and struct bvec_iter bi_size field are both "unsigned int". So they can describe at most 4G x 512B = 2TB ranges. For REQ_OP_ZONE_RESET_ALL, that is simply way too small (we already are at 20TB capacity for SMR). One cannot issue a BIO with a length that is the entire disk capacity.

I really think that bcache should handle the zone management ops as a special case. I understand the goal of trying to minimize that by integrating them as much as possible into the regular bcache IO path, but that really seems dangerous to me. Since these operations will need remapping in bcache anyway (for handling the first hidden zone containing the super block), all special processing can be done under a single "if" which should not impact too much performance of regular read/write commands in the hot path.

Quoted 6 lines

Device mapper has such code: see __split_and_process_bio() and its use of op_is_zone_mgmt() function to handle zone management requests slightly differently than other BIOs. That remove the need for relying on op_is_write() direction (patch 1 and 2 in this series) for reset zones too, which in the end will I think make your bcache code a lot more solid.

Copied. I understand and agree with you and Christoph now. Let me reconstruct the bcache code and maybe the depended change of generic block layer code can be avoided.

Damien and Christoph, thank you all for the comments.

Coly Li