scsi_debug: fix data integrity support
cover scsi_debug: fix data integrity support
When I tried testing the data integrity support in scsi_debug on x86_32, I got CONFIG_DEBUG_HIGHMEM warnings and protection errors. This was triggered due to misused kmap_atomic/kunmap_atomic.
And then, while I was testing the fix of the above issue with several combination with module parameters dix and dif, I found that doing 'modprobe scsi_debug dif=0 dix=1' causes kernel crash.
This patch set includes these fixes and cleanup which is related to data integrity support.
Akinobu Mita (3):
scsi_debug: fix data integrity support on highmem machine
scsi_debug: fix NULL pointer dereference with parameters dif=0 dix=1
scsi_debug: simplify offset calculation for dif_storep
drivers/scsi/scsi_debug.c | 29 ++++++++++++----------------- 1 file changed, 12 insertions(+), 17 deletions(-)
Signature · 1 line
1.8.1.4
PATCH 1/3 scsi_debug: fix data integrity support on highmem machine
kmap_atomic() is now using stack based implementation and doesn't take the KM_type argument anymore. So nesting kmap_atomic() calls must be properly stacked.
This fixes nesting kmap_atomic() calls for scsi_sglist and scsi_prot_sglist in prot_verify_write().
This also fixes another issue that invalid kmap address is used for kunmap_atomic(): the kmap address 'daddr' is incremented in the loop for each data page, and it can reach the next page boundary.
These problems trigger CONFIG_DEBUG_HIGHMEM warnings, protection errors, and kernel crash when doing I/O for the storage installed by 'modprobe scsi_debug dif=1 dix=1' on x86_32 with highmem.
--- drivers/scsi/scsi_debug.c | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-)
drivers/scsi/scsi_debug.c +4 -5
@@ -1814,12 +1814,12 @@ static int prot_verify_write(struct scsi_cmnd *SCpnt, sector_t start_sec,BUG_ON (scsi_sg_count (SCpnt) ==0 );BUG_ON (scsi_prot_sg_count (SCpnt) ==0 ); - paddr =kmap_atomic (sg_page (psgl)) + psgl->offset; ppage_offset =0 ;/* For each data page */ scsi_for_each_sg (SCpnt, dsgl,scsi_sg_count (SCpnt), i) { daddr =kmap_atomic (sg_page (dsgl)) + dsgl->offset; + paddr =kmap_atomic (sg_page (psgl)) + psgl->offset;/* For each sector-sized chunk in data page */ for (j =0 ; j < dsgl->length ; j += scsi_debug_sector_size) {
@@ -1904,19 +1904,18 @@ static int prot_verify_write(struct scsi_cmnd *SCpnt, sector_t start_sec, ppage_offset +=sizeof (struct sd_dif_tuple ); } -kunmap_atomic (daddr); +kunmap_atomic (paddr); +kunmap_atomic (daddr - dsgl->length); } -kunmap_atomic (paddr); - dix_writes++;return 0 ; out: dif_errors++; -kunmap_atomic (daddr);kunmap_atomic (paddr); +kunmap_atomic (daddr);return ret; }
Signature · 1 line
1.8.1.4
Re: [PATCH 1/3] scsi_debug: fix data integrity support on highmem machine
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Akinobu" == Akinobu Mita <[email protected]> writes:
Akinobu> kmap_atomic() is now using stack based implementation and Akinobu> doesn't take the KM_type argument anymore. So nesting Akinobu> kmap_atomic() calls must be properly stacked.
Akinobu> This fixes nesting kmap_atomic() calls for scsi_sglist and Akinobu> scsi_prot_sglist in prot_verify_write().
I don't see how the prog_sglist is incorrectly nested? Also, with your code you also end up mapping and unmapping the protection page for every data segment. The two scatterlists have different cadence.
Akinobu> This also fixes another issue that invalid kmap address is used Akinobu> for kunmap_atomic(): the kmap address 'daddr' is incremented in Akinobu> the loop for each data page, and it can reach the next page Akinobu> boundary.
That fix is fine.
Signature · 1 line
Martin K. Petersen Oracle Linux Engineering
Re: [PATCH 1/3] scsi_debug: fix data integrity support on highmem machine
2013/4/25 Martin K. Petersen <[email protected]>:
Quoted 6 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Akinobu" == Akinobu Mita <[email protected]> writes:
Akinobu>
kmap_atomic()is now using stack based implementation and Akinobu> doesn't take theKM_typeargument anymore. So nesting Akinobu>kmap_atomic()calls must be properly stacked.
Quoted 6 lines
Akinobu> This fixes nesting
kmap_atomic()calls forscsi_sglistand Akinobu>scsi_prot_sglistinprot_verify_write().I don't see how the
prog_sglistis incorrectly nested? Also, with your code you also end up mapping and unmapping the protection page for every data segment. The two scatterlists have different cadence.
I meant that the unmapped address by kunmap_atomic() should be the address returned by the last kmap_atomic() call which is not yet unmapped. Because kmap_atomic was changed to stack based implementation.
Specifically,
/* For each data page */
scsi_for_each_sg(SCpnt, dsgl, scsi_sg_count(SCpnt), i) {
daddr = kmap_atomic(sg_page(dsgl)) + dsgl->offset;This kmapped address 'daddr' will not unmapped during the next for-loop.
/* For each sector-sized chunk in data page */
for (j = 0 ; j < dsgl->length ; j += scsi_debug_sector_size) {
/* If we're at the end of the current
* protection page advance to the next one
*/
if (ppage_offset >= psgl->length) {
kunmap_atomic(paddr);
...But this kunmap_atomic() for the first time in this loop is called. The last two kmap_atomic() and kunmap_atomic() do not correspond and it breaks kmap_atomic/kunmap_atomic stack.
PATCH 2/3 scsi_debug: fix NULL pointer dereference with parameters dif=0 dix=1
In reply to Akinobu MitaThe protection info dif_storep is allocated only when parameter dif is not zero. But it will be accessed when reading or writing to the storage installed with parameter dix is not zero.
So kernel crashes if scsi_debug module is loaded with parameters dix=1 and dif=0.
This fixes it by making dif_storep available if parameter dix is not zero instead of checking if parameter dif is not zero.
--- drivers/scsi/scsi_debug.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/scsi/scsi_debug.c +1 -1
@@ -3306,7 +3306,7 @@ static int __init scsi_debug_init(void)if (scsi_debug_num_parts >0 )sdebug_build_parts (fake_storep, sz); -if (scsi_debug_dif) { +if (scsi_debug_dix) {int dif_size; dif_size = sdebug_store_sectors *sizeof (struct sd_dif_tuple );
Signature · 1 line
1.8.1.4
Re: [PATCH 2/3] scsi_debug: fix NULL pointer dereference with parameters dif=0 dix=1
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Akinobu" == Akinobu Mita <[email protected]> writes:
Akinobu> The protection info dif_storep is allocated only when parameter Akinobu> dif is not zero. But it will be accessed when reading or Akinobu> writing to the storage installed with parameter dix is not Akinobu> zero.
Akinobu> So kernel crashes if scsi_debug module is loaded with Akinobu> parameters dix=1 and dif=0.
The full story is that scsi_debug does not support DIF and DIX correctly by virtue of simultaneously being the HBA and the target. And since there is no actual data transfer between the HBA and the target the notion of DIF is weak at best.
I did look into making scsi_debug do the right thing but it's quite a bit of code and I lost interest about halfway through the effort. If you'd like to fix this properly I can probably find the patch to use as baseline?
In this particular case I'm ok with your patch keying off of scsi_debug_dix. But it's very important to me that I'm able to set P_TYPE and the host protection mask as I see fit using the module parameters. Even when the combination of options don't make strict sense.
Signature · 1 line
Martin K. Petersen Oracle Linux Engineering
Re: [PATCH 2/3] scsi_debug: fix NULL pointer dereference with parameters dif=0 dix=1
2013/4/25 Martin K. Petersen <[email protected]>:
Quoted 15 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Akinobu" == Akinobu Mita <[email protected]> writes:
Akinobu> The protection info
dif_storepis allocated only when parameter Akinobu> dif is not zero. But it will be accessed when reading or Akinobu> writing to the storage installed with parameter dix is not Akinobu> zero.Akinobu> So kernel crashes if
scsi_debugmodule is loaded with Akinobu> parameters dix=1 and dif=0.The full story is that
scsi_debugdoes not support DIF and DIX correctly by virtue of simultaneously being the HBA and the target. And since there is no actual data transfer between the HBA and the target the notion of DIF is weak at best.
Quoted 4 lines
I did look into making
scsi_debugdo the right thing but it's quite a bit of code and I lost interest about halfway through the effort. If you'd like to fix this properly I can probably find the patch to use as baseline?
I'm interested in the patch. So could you provide it?
PATCH 3/3 scsi_debug: simplify offset calculation for dif_storep
In reply to Akinobu Mitadif_storep is declared as pointer to unsigned char type. But it is actually used to store vmalloced array of struct sd_dif_tuple.
This changes the type of dif_storep to the pointer to struct sd_dif_tuple. It simplifies offset calculation for dif_storep and enables to remove hardcoded size of struct sd_dif_tuple.
--- drivers/scsi/scsi_debug.c | 18 +++++++----------- 1 file changed, 7 insertions(+), 11 deletions(-)
drivers/scsi/scsi_debug.c +7 -11
@@ -258,7 +258,7 @@ struct sdebug_queued_cmd {static struct sdebug_queued_cmd queued_arr[SCSI_DEBUG_CANQUEUE ];static unsigned char * fake_storep;/* ramdisk storage */ -static unsigned char *dif_storep;/* protection info */ +static struct sd_dif_tuple *dif_storep;/* protection info */ static void *map_storep;/* provisioning map */ static unsigned long map_size;
@@ -277,11 +277,6 @@ static char sdebug_proc_name[] = "scsi_debug";static struct bus_type pseudo_lld_bus; -static inline sector_t dif_offset (sector_t sector) -{ -return sector <<3 ; -} -static struct device_driver sdebug_driverfs_driver = { .name = sdebug_proc_name, .bus = &pseudo_lld_bus,
@@ -1653,7 +1648,7 @@ static int prot_verify_read(struct scsi_cmnd *SCpnt, sector_t start_sec, start_sec =do_div (tmp_sec, sdebug_store_sectors); - sdt = (struct sd_dif_tuple *)(dif_storep +dif_offset (start_sec)); + sdt = dif_storep + start_sec;for (i =0 ; i < sectors ; i++) {u16 csum;
@@ -1708,16 +1703,17 @@ static int prot_verify_read(struct scsi_cmnd *SCpnt, sector_t start_sec, ei_lba++; } - resid = sectors *8 ;/* Bytes of protection data to copy into sgl */ +/* Bytes of protection data to copy into sgl */ + resid = sectors *sizeof (*dif_storep); sector = start_sec;scsi_for_each_prot_sg (SCpnt, psgl,scsi_prot_sg_count (SCpnt), i) {int len =min (psgl->length, resid); paddr =kmap_atomic (sg_page (psgl)) + psgl->offset; -memcpy (paddr, dif_storep +dif_offset (sector), len); +memcpy (paddr, dif_storep + sector, len); - sector += len >>3 ; + sector += len /sizeof (*dif_storep);if (sector >= sdebug_store_sectors) {/* Force wrap */ tmp_sec = sector;
@@ -1891,7 +1887,7 @@ static int prot_verify_write(struct scsi_cmnd *SCpnt, sector_t start_sec,* correctness we need to verify each sector * before writing it to "stable" storage */ -memcpy (dif_storep +dif_offset (sector), sdt,8 ); +memcpy (dif_storep + sector, sdt,sizeof (*sdt)); sector++;
Signature · 1 line
1.8.1.4
Re: [PATCH 3/3] scsi_debug: simplify offset calculation for dif_storep
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Akinobu" == Akinobu Mita <[email protected]> writes:
+ sector += len / sizeof(*dif_storep);
I'd rather see sizeof(struct scsi_dif_tuple) here. But that's just personal preference. Otherwise ok.
Signature · 1 line
Martin K. Petersen Oracle Linux Engineering
Quoted 16 lines · Akinobu Mita
When I tried testing the data integrity support in
scsi_debugonx86_32, I gotCONFIG_DEBUG_HIGHMEMwarnings and protection errors. This was triggered due to misused kmap_atomic/kunmap_atomic.And then, while I was testing the fix of the above issue with several combination with module parameters dix and dif, I found that doing 'modprobe
scsi_debugdif=0 dix=1' causes kernel crash.This patch set includes these fixes and cleanup which is related to data integrity support.
Cc "James E.J. Bottomley" <[email protected]>Cc Douglas Gilbert <[email protected]>Cc "Martin K. Petersen" <[email protected]>
Quoted 7 lines
Akinobu Mita (3):
scsi_debug: fix data integrity support on highmem machine
scsi_debug: fix NULL pointer dereference with parameters dif=0 dix=1
scsi_debug: simplify offset calculation fordif_storepdrivers/scsi/scsi_debug.c | 29 ++++++++++++----------------- 1 file changed, 12 insertions(+), 17 deletions(-)
Again, I'd like to see some feedback from Martin Petersen on this set of patches. For my part, for this patch series (1/3 to 3/3):