Skip to contentSkip to conversation

btt: fix block integrity

PATCH In review Keith Busch 8 participants 11 messages Aug 30, 2024 – Nov 15, 2024

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 btt: fix block integrity

From: Keith Busch <[email protected]>

bip is NULL before bio_integrity_prep().

---
 drivers/nvdimm/btt.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/nvdimm/btt.c +2 -1
index 423dcd1909061..13594fb712186 100644
@@ -1435,8 +1435,8 @@ static int btt_do_bvec(struct btt *btt, struct bio_integrity_payload *bip,
 
 static void btt_submit_bio(struct bio *bio)
 {
-	struct bio_integrity_payload *bip = bio_integrity(bio);
 	struct btt *btt = bio->bi_bdev->bd_disk->private_data;
+	struct bio_integrity_payload *bip;
 	struct bvec_iter iter;
 	unsigned long start;
 	struct bio_vec bvec;
@@ -1445,6 +1445,7 @@ static void btt_submit_bio(struct bio *bio)
 
 	if (!bio_integrity_prep(bio))
 		return;
+	bip = bio_integrity(bio);
 
 	do_acct = blk_queue_io_stat(bio->bi_bdev->bd_disk->queue);
 	if (do_acct)
Signature · 1 line
2.43.5
maintainer Reviewed-by on v1 1/1
#1 lore raw
Quoted 5 lines · Keith Busch

From: Keith Busch <[email protected]>

bip is NULL before bio_integrity_prep().

Quoted 26 lines
---
 drivers/nvdimm/btt.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/nvdimm/btt.c +2 -1
index 423dcd1909061..13594fb712186 100644
@@ -1435,8 +1435,8 @@ static int btt_do_bvec(struct btt *btt, struct bio_integrity_payload *bip,
 
 static void btt_submit_bio(struct bio *bio)
 {
-	struct bio_integrity_payload *bip = bio_integrity(bio);
 	struct btt *btt = bio->bi_bdev->bd_disk->private_data;
+	struct bio_integrity_payload *bip;
 	struct bvec_iter iter;
 	unsigned long start;
 	struct bio_vec bvec;
@@ -1445,6 +1445,7 @@ static void btt_submit_bio(struct bio *bio)
 
 	if (!bio_integrity_prep(bio))
 		return;
+	bip = bio_integrity(bio);
 
 	do_acct = blk_queue_io_stat(bio->bi_bdev->bd_disk->queue);
 	if (do_acct)
#2 lore raw
In reply to Keith Busch
Quoted 3 lines · Keith Busch

From: Keith Busch <[email protected]>

bip is NULL before bio_integrity_prep().

Did this fail in some way the user might see? How was this found?

I think the code is correct but should this be backported to stable or anything?

Ira

Quoted 32 lines
---
 drivers/nvdimm/btt.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/nvdimm/btt.c +2 -1
index 423dcd1909061..13594fb712186 100644
@@ -1435,8 +1435,8 @@ static int btt_do_bvec(struct btt *btt, struct bio_integrity_payload *bip,
 
 static void btt_submit_bio(struct bio *bio)
 {
-	struct bio_integrity_payload *bip = bio_integrity(bio);
 	struct btt *btt = bio->bi_bdev->bd_disk->private_data;
+	struct bio_integrity_payload *bip;
 	struct bvec_iter iter;
 	unsigned long start;
 	struct bio_vec bvec;
@@ -1445,6 +1445,7 @@ static void btt_submit_bio(struct bio *bio)
 
 	if (!bio_integrity_prep(bio))
 		return;
+	bip = bio_integrity(bio);
 
 	do_acct = blk_queue_io_stat(bio->bi_bdev->bd_disk->queue);
 	if (do_acct)
Signature · 1 line
2.43.5
#3 lore raw
Quoted 6 lines · Ira Weiny
Quoted 3 lines · Keith Busch

From: Keith Busch <[email protected]>

bip is NULL before bio_integrity_prep().

Did this fail in some way the user might see? How was this found?

I think this means no one ever used block integrity with btt. :)

I found this purely from code inspection because I was combing through bio_integrity for a completely unrelated problem. I was trying to make sense of how other drivers use it, and this one didn't make any.

Quoted 2 lines

I think the code is correct but should this be backported to stable or anything?

Up to you! It's not fixing a regression since it appears to have never worked before. You can also just delete support for it entirely if no one cares to use this feature.

#4 lore raw
Quoted 14 lines · Keith Busch
Quoted 6 lines · Ira Weiny
Quoted 3 lines · Keith Busch

From: Keith Busch <[email protected]>

bip is NULL before bio_integrity_prep().

Did this fail in some way the user might see? How was this found?

I think this means no one ever used block integrity with btt. :)

I found this purely from code inspection because I was combing through bio_integrity for a completely unrelated problem. I was trying to make sense of how other drivers use it, and this one didn't make any.

Quoted 6 lines
Quoted 2 lines

I think the code is correct but should this be backported to stable or anything?

Up to you! It's not fixing a regression since it appears to have never worked before. You can also just delete support for it entirely if no one cares to use this feature.

I think most people are just hoping that filesystem metadata checksums are catching torn writes and not using btt. For the few that do not have a checksumming filesystem and are using btt I only expect they are using 512 or 4K sector sizes and not the sizes with integrity metadata. For the ones that are using the odd sizes with integrity metadata they obviously do not care about the integrity data being transferred since that never apparently worked.

My vote is delete the integrity support, but keep the odd sector size support for compatibility.

#5 lore raw
Quoted 30 lines · Dan Williams
Quoted 14 lines · Keith Busch
Quoted 6 lines · Ira Weiny
Quoted 3 lines · Keith Busch

From: Keith Busch <[email protected]>

bip is NULL before bio_integrity_prep().

Did this fail in some way the user might see? How was this found?

I think this means no one ever used block integrity with btt. :)

I found this purely from code inspection because I was combing through bio_integrity for a completely unrelated problem. I was trying to make sense of how other drivers use it, and this one didn't make any.

Quoted 6 lines
Quoted 2 lines

I think the code is correct but should this be backported to stable or anything?

Up to you! It's not fixing a regression since it appears to have never worked before. You can also just delete support for it entirely if no one cares to use this feature.

I think most people are just hoping that filesystem metadata checksums are catching torn writes and not using btt. For the few that do not have a checksumming filesystem and are using btt I only expect they are using 512 or 4K sector sizes and not the sizes with integrity metadata. For the ones that are using the odd sizes with integrity metadata they obviously do not care about the integrity data being transferred since that never apparently worked.

Quoted 2 lines

My vote is delete the integrity support, but keep the odd sector size support for compatibility.

More generally, does anyone use btt? Should we start the deprecation process for it?

-Jeff

#6 lore raw
Quoted 32 lines · Jeff Moyer
Quoted 30 lines · Dan Williams
Quoted 14 lines · Keith Busch
Quoted 6 lines · Ira Weiny
Quoted 3 lines · Keith Busch

From: Keith Busch <[email protected]>

bip is NULL before bio_integrity_prep().

Did this fail in some way the user might see? How was this found?

I think this means no one ever used block integrity with btt. :)

I found this purely from code inspection because I was combing through bio_integrity for a completely unrelated problem. I was trying to make sense of how other drivers use it, and this one didn't make any.

Quoted 6 lines
Quoted 2 lines

I think the code is correct but should this be backported to stable or anything?

Up to you! It's not fixing a regression since it appears to have never worked before. You can also just delete support for it entirely if no one cares to use this feature.

I think most people are just hoping that filesystem metadata checksums are catching torn writes and not using btt. For the few that do not have a checksumming filesystem and are using btt I only expect they are using 512 or 4K sector sizes and not the sizes with integrity metadata. For the ones that are using the odd sizes with integrity metadata they obviously do not care about the integrity data being transferred since that never apparently worked.

Quoted 5 lines
Quoted 2 lines

My vote is delete the integrity support, but keep the odd sector size support for compatibility.

More generally, does anyone use btt? Should we start the deprecation process for it?

True, I think it is worth starting the deprecation process and see who screams. It is easy enough to revive if there are still users out there that have not since moved back to NVME for storage.

#7 lore raw
Quoted 3 lines · Dan Williams

True, I think it is worth starting the deprecation process and see who screams. It is easy enough to revive if there are still users out there that have not since moved back to NVME for storage.

Were there every any users? I've seen bug reports for all kinds of weird drivers, but never ever btt..

maintainer
#8 lore raw
In reply to Keith Busch
Quoted 28 lines · Keith Busch

From: Keith Busch <[email protected]>

bip is NULL before bio_integrity_prep().

---
 drivers/nvdimm/btt.c | 3 ++-

 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/nvdimm/btt.c +0 -0
index 423dcd1909061..13594fb712186 100644
@@ -1435,8 +1435,8 @@ static int btt_do_bvec(struct btt *btt, struct
 bio_integrity_payload *bip,


 static void btt_submit_bio(struct bio *bio)
 {

drivers/nvdimm/btt.c +0 -1
-	struct bio_integrity_payload *bip = bio_integrity(bio);

  struct btt *btt = bio->bi_bdev->bd_disk->private_data;

drivers/nvdimm/btt.c +1 -0
+	struct bio_integrity_payload *bip;

  struct bvec_iter iter;
  unsigned long start;
  struct bio_vec bvec;

drivers/nvdimm/btt.c +0 -0
@@ -1445,6 +1445,7 @@ static void btt_submit_bio(struct bio *bio)


  if (!bio_integrity_prep(bio))
  return;

drivers/nvdimm/btt.c static void btt_submit_bio(struct bio *bio) · line 1448

Quoted 1 line
drivers/nvdimm/btt.c +1 -0
+	bip = bio_integrity(bio);

Hi Keith,

I suspect this isn't actually needed, since the btt never generated its own protection payload. See the /* Already protected? */ case in bio_integrity_prep() - I think that's the only case we were trying to account for - i.e. 'some other layer' set the integrity payload, and we're just passing it on to it's right spot in pmem, and reading it back. The btt itself doesn't ever try to generate and set a protection payload of its own.

If you look at the original flow in 41cd8b70c37ace40077c8d6ec0b74b983178c192, btt never actually wants to call bio_integrity_prep and allocate the bip - if it has to do that, that's treated as an error.

Since some of the reworks then to eliminate bio_integrity_enabled, and other block level changes, this has changed to actually allocating bip and continuing instead of erroring, but coincidentially since we assign bip before the allocation (i.e. NULL as you point out), any future steps nicely ignore it, but if it was set by another subsystem, things should still 'work' - as in bio_integrity_prep() would return true, and bip would be non-NULL, and would get written/read as needed, and this is the happy path.

Does this makes sense or am I missing something?
It's probably irrelevant if we deprecate all of this soon anyway :)

Quoted 3 lines


  do_acct = blk_queue_io_stat(bio->bi_bdev->bd_disk->queue); if (do_acct)

#9 lore raw
Quoted 22 lines · Vishal L

I suspect this isn't actually needed, since the btt never generated its own protection payload. See the /* Already protected? */ case in bio_integrity_prep() - I think that's the only case we were trying to account for - i.e. 'some other layer' set the integrity payload, and we're just passing it on to it's right spot in pmem, and reading it back. The btt itself doesn't ever try to generate and set a protection payload of its own.

If you look at the original flow in 41cd8b70c37ace40077c8d6ec0b74b983178c192, btt never actually wants to call bio_integrity_prep and allocate the bip - if it has to do that, that's treated as an error.

Since some of the reworks then to eliminate bio_integrity_enabled, and other block level changes, this has changed to actually allocating bip and continuing instead of erroring, but coincidentially since we assign bip before the allocation (i.e. NULL as you point out), any future steps nicely ignore it, but if it was set by another subsystem, things should still 'work' - as in bio_integrity_prep() would return true, and bip would be non-NULL, and would get written/read as needed, and this is the happy path.

Quoted 1 line

Does this makes sense or am I missing something?

One of us might be missing something. :)

The only upper layers I find that pass an integrity payload to the lower level driver are the scsi and nvme targets. It may make sense to use btt as the backend for those, but is that the only use case this was trying to enable? Or is there some other path I didn't find?

maintainer
#10 lore raw
Quoted 23 lines · Keith Busch
Quoted 22 lines · Vishal L

I suspect this isn't actually needed, since the btt never generated its own protection payload. See the /* Already protected? */ case in bio_integrity_prep() - I think that's the only case we were trying to account for - i.e. 'some other layer' set the integrity payload, and we're just passing it on to it's right spot in pmem, and reading it back. The btt itself doesn't ever try to generate and set a protection payload of its own.

If you look at the original flow in 41cd8b70c37ace40077c8d6ec0b74b983178c192, btt never actually wants to call bio_integrity_prep and allocate the bip - if it has to do that, that's treated as an error.

Since some of the reworks then to eliminate bio_integrity_enabled, and other block level changes, this has changed to actually allocating bip and continuing instead of erroring, but coincidentially since we assign bip before the allocation (i.e. NULL as you point out), any future steps nicely ignore it, but if it was set by another subsystem, things should still 'work' - as in bio_integrity_prep() would return true, and bip would be non-NULL, and would get written/read as needed, and this is the happy path.

Quoted 8 lines
Quoted 1 line

Does this makes sense or am I missing something?

One of us might be missing something. :)

The only upper layers I find that pass an integrity payload to the lower level driver are the scsi and nvme targets. It may make sense to use btt as the backend for those, but is that the only use case this was trying to enable? Or is there some other path I didn't find?

Right, that was the only use case this was trying to enable. I'm not sure btt can even be a backend to scsi or nvme, I think one possibilyty could've been something like md, or something more custom. The goal with btt was to pass through any integrity payloads if it encounters them, but otherwise not generate one by itself.