fix virt_boundary_mask handling in SCSI
Subsystems (4)
Hi all,
this series fixes a corruption when drivers using virt_boundary_mask set a limited max_segment_size by accident, which Red Hat reported as causing data corruption with storvsc. I did audit the tree and also found that this can affect SRP and iSER as well.
Note that I've dropped the Tested-by from Laurence because the patch changed very slightly from the last version.
Diffstat:
infiniband/ulp/srp/ib_srp.c | 5 +++-- scsi/hosts.c | 20 +++++++++++++------- 2 files changed, 16 insertions(+), 9 deletions(-)
PATCH 1/2 RDMA/srp: don't set a max_segment_size when virt_boundary_mask is set
virt_boundary_mask implies an unlimited max_segment_size. Setting both can lead to data corruption, and we're going to check for this in the SCSI midlayer soon.
--- drivers/infiniband/ulp/srp/ib_srp.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
drivers/infiniband/ulp/srp/ib_srp.c +3 -2
@@ -3705,9 +3705,10 @@ static ssize_t add_target_store(struct device *dev, target_host->max_id =1 ; target_host->max_lun =-1LL ; target_host->max_cmd_len =sizeof ((struct srp_cmd *) (void *)0L )->cdb; - target_host->max_segment_size =ib_dma_max_seg_size (ibdev); -if (!(ibdev->attrs.kernel_cap_flags &IBK_SG_GAPS_REG )) +if (ibdev->attrs.kernel_cap_flags &IBK_SG_GAPS_REG ) + target_host->max_segment_size =ib_dma_max_seg_size (ibdev); +else target_host->virt_boundary_mask = ~srp_dev->mr_page_mask; target =host_to_target (target_host);
Signature · 1 line
2.47.2
Bart Van Assche
maintainerRe: [PATCH 1/2] RDMA/srp: don't set a max_segment_size when virt_boundary_mask is set
Quoted 3 lines · Christoph Hellwig
virt_boundary_maskimplies an unlimitedmax_segment_size. Setting both can lead to data corruption, and we're going to check for this in the SCSI midlayer soon.
Please make this patch description more detailed and mention that __blk_rq_map_sg() may split sg-lists such that the virt_boundary_mask is not respected if max_segment_size != UINT_MAX.
Thanks,
Bart.
Re: [PATCH 1/2] RDMA/srp: don't set a max_segment_size when virt_boundary_mask is set
In reply to Christoph HellwigQuoted 16 lines · Christoph Hellwig
virt_boundary_maskimplies an unlimitedmax_segment_size. Setting both can lead to data corruption, and we're going to check for this in the SCSI midlayer soon.Signed-off-by Christoph Hellwig <[email protected]>--- drivers/infiniband/ulp/srp/ib_srp.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)drivers/infiniband/ulp/srp/ib_srp.c +0
-0@@ -3705,9 +3705,10 @@ static ssize_t add_target_store(struct device *dev, target_host->max_id =1 ; target_host->max_lun =-1LL ;
drivers/infiniband/ulp/srp/ib_srp.c static ssize_t add_target_store(struct device *dev, · line 3714
Quoted 10 lines
drivers/infiniband/ulp/srp/ib_srp.c +3
-2target_host->max_cmd_len =sizeof ((struct srp_cmd *) (void *)0L )->cdb; - target_host->max_segment_size =ib_dma_max_seg_size (ibdev); -if (!(ibdev->attrs.kernel_cap_flags &IBK_SG_GAPS_REG )) +if (ibdev->attrs.kernel_cap_flags &IBK_SG_GAPS_REG ) + target_host->max_segment_size =ib_dma_max_seg_size (ibdev); +else target_host->virt_boundary_mask = ~srp_dev->mr_page_mask; target =host_to_target (target_host);
Cheers,
Hannes
Signature · 4 lines
Dr. Hannes Reinecke Kernel Storage Architect [email protected] +49 911 74053 688 SUSE Software Solutions GmbH, Frankenstr. 146, 90461 Nürnberg HRB 36809 (AG Nürnberg), GF: I. Totev, A. McDonald, W. Knoblich
PATCH 2/2 scsi: enforce unlimited max_segment_size when virt_boundary_mask is set
In reply to Christoph HellwigThe virt_boundary_mask limit requires an unlimited max_segment_size for bio splitting to not corrupt data. Historically, the block layer tried to validate this, although the check was half-hearted until the addition of the atomic queue limits API. The full blown check than triggered issues with stacked devices incorrectly inheriting limits such as the virt boundary and got disabled in commit b561ea56a264 ("block: allow device to have both virt_boundary_mask and max segment size") instead of fixing the issue properly.
Ensure that the SCSI mid layer doesn't set the default low max_segment_size limit for this case, and check for invalid max_segment_size values in the host template, similar to the original block layer check given that SCSI devices can't be stacked.
This fixes reported data corruption on storvsc, although as far as I can tell storvsc always failed to properly set the max_segment_size limit as the SCSI APIs historically applied that when setting up the host, while storvsc only set the virt_boundary_mask when configuring the scsi_device.
--- drivers/scsi/hosts.c | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-)
drivers/scsi/hosts.c +13 -7
@@ -473,10 +473,19 @@ struct Scsi_Host *scsi_host_alloc(const struct scsi_host_template *sht, int privelse shost->max_sectors =SCSI_DEFAULT_MAX_SECTORS ; -if (sht->max_segment_size) - shost->max_segment_size = sht->max_segment_size; -else - shost->max_segment_size =BLK_MAX_SEGMENT_SIZE ; +if (sht->virt_boundary_mask) + shost->virt_boundary_mask = sht->virt_boundary_mask; + +if (shost->virt_boundary_mask) { +WARN_ON_ONCE (sht->max_segment_size && + sht->max_segment_size !=UINT_MAX ); + shost->max_segment_size =UINT_MAX ; + }else { +if (sht->max_segment_size) + shost->max_segment_size = sht->max_segment_size; +else + shost->max_segment_size =BLK_MAX_SEGMENT_SIZE ; + }/* 32-byte (dword) is a common minimum for HBAs. */ if (sht->dma_alignment)
@@ -492,9 +501,6 @@ struct Scsi_Host *scsi_host_alloc(const struct scsi_host_template *sht, int privelse shost->dma_boundary =0xffffffff ; -if (sht->virt_boundary_mask) - shost->virt_boundary_mask = sht->virt_boundary_mask; -device_initialize (&shost->shost_gendev);dev_set_name (&shost->shost_gendev,"host%d" , shost->host_no); shost->shost_gendev.bus = &scsi_bus_type;
Signature · 1 line
2.47.2
Re: [PATCH 2/2] scsi: enforce unlimited max_segment_size when virt_boundary_mask is set
Quoted 19 lines · Christoph Hellwig
The
virt_boundary_masklimit requires an unlimitedmax_segment_sizefor bio splitting to not corrupt data. Historically, the block layer tried to validate this, although the check was half-hearted until the addition of the atomic queue limits API. The full blown check than triggered issues with stacked devices incorrectly inheriting limits such as the virt boundary and got disabled in commitb561ea56a264("block: allow device to have bothvirt_boundary_maskand max segment size") instead of fixing the issue properly.Ensure that the SCSI mid layer doesn't set the default low
max_segment_sizelimit for this case, and check for invalidmax_segment_sizevalues in the host template, similar to the original block layer check given that SCSI devices can't be stacked.This fixes reported data corruption on storvsc, although as far as I can tell storvsc always failed to properly set the
max_segment_sizelimit as the SCSI APIs historically applied that when setting up the host, while storvsc only set thevirt_boundary_maskwhen configuring thescsi_device.
Quoted 4 lines
Fixes 81988a0e6b03 ("storvsc: get rid of bounce buffer")Fixes b561ea56a264 ("block: allow device to have both virt_boundary_mask and max segment size")Reported-by Ming Lei <[email protected]>Signed-off-by Christoph Hellwig <[email protected]>
Regardless of nitpicking of coding style, FWIW:
Quoted 9 lines
--- drivers/scsi/hosts.c | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-)drivers/scsi/hosts.c +0
-0
drivers/scsi/hosts.c struct Scsi_Host *scsi_host_alloc(const struct scsi_host_template *sht, int priv · line 477
Quoted 9 lines
drivers/scsi/hosts.c +2
-4@@ -473,10 +473,19 @@ struct Scsi_Host *scsi_host_alloc(const struct scsi_host_template *sht, int privelse shost->max_sectors =SCSI_DEFAULT_MAX_SECTORS ; -if (sht->max_segment_size) - shost->max_segment_size = sht->max_segment_size; -else - shost->max_segment_size =BLK_MAX_SEGMENT_SIZE ; +if (sht->virt_boundary_mask) + shost->virt_boundary_mask = sht->virt_boundary_mask;
nit: you could just always set shost->virt_boundary_mask = sht->virt_boundary_mask
drivers/scsi/hosts.c struct Scsi_Host *scsi_host_alloc(const struct scsi_host_template *sht, int priv · line 479
Quoted 2 lines
drivers/scsi/hosts.c +2
-0+ +if (shost->virt_boundary_mask) {
Or combine into a single if-else statement.
drivers/scsi/hosts.c struct Scsi_Host *scsi_host_alloc(const struct scsi_host_template *sht, int priv · line 483
Quoted 4 lines
drivers/scsi/hosts.c +4
-0+WARN_ON_ONCE (sht->max_segment_size && + sht->max_segment_size !=UINT_MAX ); + shost->max_segment_size =UINT_MAX ; + }else {
else if might be nicer, by maybe not as (I think) {} should be used and that is not pretty for single line statements.
Quoted 17 lines
drivers/scsi/hosts.c +4
-3+if (sht->max_segment_size) + shost->max_segment_size = sht->max_segment_size;> + else + shost->max_segment_size =BLK_MAX_SEGMENT_SIZE ; + }/* 32-byte (dword) is a common minimum for HBAs. */ if (sht->dma_alignment)@@ -492,9 +501,6 @@ struct Scsi_Host *scsi_host_alloc(const struct scsi_host_template *sht, int privelse shost->dma_boundary =0xffffffff ; -if (sht->virt_boundary_mask) - shost->virt_boundary_mask = sht->virt_boundary_mask; -device_initialize (&shost->shost_gendev);dev_set_name (&shost->shost_gendev,"host%d" , shost->host_no); shost->shost_gendev.bus = &scsi_bus_type;
Re: [PATCH 2/2] scsi: enforce unlimited max_segment_size when virt_boundary_mask is set
Quoted 7 lines · John Garry
drivers/scsi/hosts.c struct Scsi_Host *scsi_host_alloc(const struct scsi_host_template *sht, int priv · line 477
Quoted 4 lines
drivers/scsi/hosts.c +2
-2-else - shost->max_segment_size =BLK_MAX_SEGMENT_SIZE ; +if (sht->virt_boundary_mask) + shost->virt_boundary_mask = sht->virt_boundary_mask;nit: you could just always set
shost->virt_boundary_mask=sht->virt_boundary_mask
I could, but it would change behavior and break drivers. The SCSI midlayer allows overriding the template provided values in the host itself after allocating and before adding it. For the virt_boundary_mask that features is used by iser and srp.
Quoted 11 lines
drivers/scsi/hosts.c struct Scsi_Host *scsi_host_alloc(const struct scsi_host_template *sht, int priv · line 479
Quoted 1 line
drivers/scsi/hosts.c +1
-0+if (shost->virt_boundary_mask) {Or combine into a single if-else statement.
drivers/scsi/hosts.c struct Scsi_Host *scsi_host_alloc(const struct scsi_host_template *sht, int priv · line 483
Quoted 4 lines
drivers/scsi/hosts.c +4
-0+WARN_ON_ONCE (sht->max_segment_size && + sht->max_segment_size !=UINT_MAX ); + shost->max_segment_size =UINT_MAX ; + }else {else if might be nicer, by maybe not as (I think) {} should be used and that is not pretty for single line statements.
I also really want to keep the virt boundary vs non virt boundary cases visually separe as that's the main branch.
Re: [PATCH 2/2] scsi: enforce unlimited max_segment_size when virt_boundary_mask is set
Quoted 11 lines · Christoph Hellwig
Quoted 6 lines · John Garry
drivers/scsi/hosts.c struct Scsi_Host *scsi_host_alloc(const struct scsi_host_template *sht, int priv · line 477
Quoted 4 lines
drivers/scsi/hosts.c +2
-2-else - shost->max_segment_size =BLK_MAX_SEGMENT_SIZE ; +if (sht->virt_boundary_mask) + shost->virt_boundary_mask = sht->virt_boundary_mask;nit: you could just always set
shost->virt_boundary_mask=sht->virt_boundary_maskI could, but it would change behavior and break drivers. The SCSI midlayer allows overriding the template provided values in the host itself after allocating and before adding it. For the
virt_boundary_maskthat features is used by iser and srp.
Since shost is zero-init'ed, I did not think that my suggestion for this minor simplification in scsi_host_alloc() logically changes anything.
Re: [PATCH 2/2] scsi: enforce unlimited max_segment_size when virt_boundary_mask is set
Quoted 8 lines · John Garry
Quoted 7 lines · Christoph Hellwig
Quoted 6 lines · John Garry
drivers/scsi/hosts.c struct Scsi_Host *scsi_host_alloc(const struct scsi_host_template *sht, int priv · line 477
Quoted 4 lines
drivers/scsi/hosts.c +2
-2-else - shost->max_segment_size =BLK_MAX_SEGMENT_SIZE ; +if (sht->virt_boundary_mask) + shost->virt_boundary_mask = sht->virt_boundary_mask;nit: you could just always set
shost->virt_boundary_mask=sht->virt_boundary_mask
Quoted 7 lines
Quoted 4 lines
I could, but it would change behavior and break drivers. The SCSI midlayer allows overriding the template provided values in the host itself after allocating and before adding it. For the
virt_boundary_maskthat features is used by iser and srp.Since shost is zero-init'ed, I did not think that my suggestion for this minor simplification in
scsi_host_alloc()logically changes anything.
Oh, you're right - I thought we did the sht assignments in scsi_add_host. So the changes would be fine. But that also means we don't catch conflicts added by the direct shost manipulation.
Re: [PATCH 2/2] scsi: enforce unlimited max_segment_size when virt_boundary_mask is set
In reply to Christoph HellwigQuoted 19 lines · Christoph Hellwig
The
virt_boundary_masklimit requires an unlimitedmax_segment_sizefor bio splitting to not corrupt data. Historically, the block layer tried to validate this, although the check was half-hearted until the addition of the atomic queue limits API. The full blown check than triggered issues with stacked devices incorrectly inheriting limits such as the virt boundary and got disabled in commitb561ea56a264("block: allow device to have bothvirt_boundary_maskand max segment size") instead of fixing the issue properly.Ensure that the SCSI mid layer doesn't set the default low
max_segment_sizelimit for this case, and check for invalidmax_segment_sizevalues in the host template, similar to the original block layer check given that SCSI devices can't be stacked.This fixes reported data corruption on storvsc, although as far as I can tell storvsc always failed to properly set the
max_segment_sizelimit as the SCSI APIs historically applied that when setting up the host, while storvsc only set thevirt_boundary_maskwhen configuring thescsi_device.
Quoted 4 lines
Fixes 81988a0e6b03 ("storvsc: get rid of bounce buffer")Fixes b561ea56a264 ("block: allow device to have both virt_boundary_mask and max segment size")Reported-by Ming Lei <[email protected]>Signed-off-by Christoph Hellwig <[email protected]>
Thanks,
Ming
Re: [PATCH 2/2] scsi: enforce unlimited max_segment_size when virt_boundary_mask is set
In reply to Christoph HellwigQuoted 19 lines · Christoph Hellwig
The
virt_boundary_masklimit requires an unlimitedmax_segment_sizefor bio splitting to not corrupt data. Historically, the block layer tried to validate this, although the check was half-hearted until the addition of the atomic queue limits API. The full blown check than triggered issues with stacked devices incorrectly inheriting limits such as the virt boundary and got disabled in commitb561ea56a264("block: allow device to have bothvirt_boundary_maskand max segment size") instead of fixing the issue properly.Ensure that the SCSI mid layer doesn't set the default low
max_segment_sizelimit for this case, and check for invalidmax_segment_sizevalues in the host template, similar to the original block layer check given that SCSI devices can't be stacked.This fixes reported data corruption on storvsc, although as far as I can tell storvsc always failed to properly set the
max_segment_sizelimit as the SCSI APIs historically applied that when setting up the host, while storvsc only set thevirt_boundary_maskwhen configuring thescsi_device.
Quoted 8 lines
Fixes 81988a0e6b03 ("storvsc: get rid of bounce buffer")Fixes b561ea56a264 ("block: allow device to have both virt_boundary_mask and max segment size")Reported-by Ming Lei <[email protected]>Signed-off-by Christoph Hellwig <[email protected]>--- drivers/scsi/hosts.c | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-)
Cheers,
Hannes
Signature · 4 lines
Dr. Hannes Reinecke Kernel Storage Architect [email protected] +49 911 74053 688 SUSE Software Solutions GmbH, Frankenstr. 146, 90461 Nürnberg HRB 36809 (AG Nürnberg), GF: I. Totev, A. McDonald, W. Knoblich
Quoted 10 lines · Christoph Hellwig
Hi all,
this series fixes a corruption when drivers using
virt_boundary_maskset
a limitedmax_segment_sizeby accident, which Red Hat reported as causing
data corruption with storvsc. I did audit the tree and also found that
this can affect SRP and iSER as well.
Quoted 8 lines
Note that I've dropped the Tested-by from Laurence because the patch changed very slightly from the last version.
Diffstat:
infiniband/ulp/srp/ib_srp.c | 5 +++-- scsi/hosts.c | 20 +++++++++++++-------2 files changed, 16 insertions(+), 9 deletions(-)
Grabbing latest and will test tomorrow and reply
Laurence Oberman
Tested-byQuoted 16 lines · Laurence Oberman
Quoted 15 lines · Christoph Hellwig
Hi all,
this series fixes a corruption when drivers using
virt_boundary_mask
set
a limitedmax_segment_sizeby accident, which Red Hat reported as causing
data corruption with storvsc. I did audit the tree and also found that
this can affect SRP and iSER as well.Note that I've dropped the Tested-by from Laurence because the patch
changed very slightly from the last version.
Quoted 7 lines
Quoted 5 lines
Diffstat:
infiniband/ulp/srp/ib_srp.c | 5 +++-- scsi/hosts.c | 20 +++++++++++++-------2 files changed, 16 insertions(+), 9 deletions(-)Grabbing latest and will test tomorrow and reply
For the series looks good.
Same testing shows no corruptions on storvsc for the REDO so passed. For SRP initiators generic testing done with fio and passed, unable to test SRP LUNS with Oracle REDO at this time.
Here it is, enough reviewers already so just the testing Patches were applied to a 9.6 kernel because I needed such a kernel for Oracle compatiility.
Quoted 31 lines · Laurence Oberman
Quoted 17 lines · Laurence Oberman
Quoted 16 lines · Christoph Hellwig
Hi all,
this series fixes a corruption when drivers using
virt_boundary_mask
set
a limitedmax_segment_sizeby accident, which Red Hat reported as causing
data corruption with storvsc. I did audit the tree and also found
that
this can affect SRP and iSER as well.Note that I've dropped the Tested-by from Laurence because the patch
changed very slightly from the last version.Quoted 7 lines
Quoted 5 lines
Diffstat:
infiniband/ulp/srp/ib_srp.c | 5 +++-- scsi/hosts.c | 20 +++++++++++++-------2 files changed, 16 insertions(+), 9 deletions(-)Grabbing latest and will test tomorrow and reply
For the series looks good.
Same testing shows no corruptions on storvsc for the REDO so passed. For SRP initiators generic testing done with fio and passed, unable to
test SRP LUNS with Oracle REDO at this time.
Quoted 9 lines
Here it is, enough reviewers already so just the testing Patches were applied to a 9.6 kernel because I needed such a kernel for
Oracle compatiility.Tested-by Laurence Oberman <[email protected]>
Nit, fix my email, dropped an l should be loberman@redhat. com of course