Fix unaligned queue_limit->io_opt on SAS and 4KN disk
cover Fix unaligned queue_limit->io_opt on SAS and 4KN disk
If there is an SAS attached disk we set the SCSI host opt_sectors in sas_host_setup() to either dma_opt_mapping_size() or host->max_sectors this value can be later assigned to the device queue_limit as io_opt, but there is no further check if the io_opt is aligned with the device limits.
Rounding down to device logical_sectors will keep the io_opt large enough to keep benefit of bigger IO but still aligned within the device limits.
Lukas Herbolt (1):
scsi: sd: Rounddown host->opt_sectors to logical sectors reported by
the disk.
drivers/scsi/sd.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
Signature · 1 line
2.51.1
PATCH 1/1 scsi: sd: Rounddown host->opt_sectors to logical
The host->opt_sectors are by default 512 bytes aligned if we have 4KN SAS disk reporting zero or smaller opt_xfer_block than the host->opt_sectors we will end up with unaligned queue_limit->io_opt.
--- drivers/scsi/sd.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/scsi/sd.c +2 -1
@@ -3787,7 +3787,8 @@ static int sd_revalidate_disk(struct gendisk *disk)* an impact on performance for when the size of a request exceeds this * host limit. */ - lim.io_opt = sdp->host->opt_sectors <<SECTOR_SHIFT ; + lim.io_opt =rounddown (lim.logical_block_size, + sdp->host->opt_sectors <<SECTOR_SHIFT );if (sd_validate_opt_xfer_size (sdkp, dev_max)) { lim.io_opt =min_not_zero (lim.io_opt,logical_to_bytes (sdp, sdkp->opt_xfer_blocks));
Signature · 1 line
2.51.1
Re: [PATCH 1/1] scsi: sd: Rounddown host->opt_sectors to logical
Lukas,
Quoted 3 lines
The
host->opt_sectorsare by default 512 bytes aligned if we have 4KN SAS disk reporting zero or smalleropt_xfer_blockthan thehost->opt_sectorswe will end up with unalignedqueue_limit->io_opt.
I am intrigued wrt. which controller returns a host->opt_sectors that would trigger this misalignment. What is the actual value reported?
drivers/scsi/sd.c static int sd_revalidate_disk(struct gendisk *disk) · line 3791
Quoted 2 lines
drivers/scsi/sd.c +2
-0+ lim.io_opt =rounddown (lim.logical_block_size, + sdp->host->opt_sectors <<SECTOR_SHIFT );
Those parameters would need to be reversed, I think. You want to round down opt_sectors and not the logical_block_size.
Signature · 1 line
Martin K. Petersen
Re: [PATCH 1/1] scsi: sd: Rounddown host->opt_sectors to logical
Quoted 9 lines · Martin K. Petersen
Lukas,
Quoted 3 lines
The
host->opt_sectorsare by default 512 bytes aligned if we have 4KN SAS disk reporting zero or smalleropt_xfer_blockthan thehost->opt_sectorswe will end up with unalignedqueue_limit->io_opt.I am intrigued wrt. which controller returns a
host->opt_sectorsthat would trigger this misalignment. What is the actual value reported?
I have seen it on mpt3sas, I do not have the hardware in hand, but I do not think is related to specific driver as it goes trough sas_host_setup()
static int sas_host_setup(struct transport_container *tc, struct device *dev,
struct device *cdev)
{
...
if (dma_dev->dma_mask) {
shost->opt_sectors = min_t(unsigned int,
shost->max_sectors,
dma_opt_mapping_size(dma_dev) >>
SECTOR_SHIFT);
}
return 0;
}I have vmcore from the system.
crash> scsishow | grep ^host | grep -v ahci
host0 mpt3sas ffff90b78c936000 ffff90b83ac92d20 ffff90b78c9368d0
crash> Scsi_Host.opt_sectors ffff90b78c936000
opt_sectors = 0x7fff
crash> pd 0x7fff<<9
$3 = 16776704
and lsblk output:
NAME ALIGNMENT MIN-IO OPT-IO PHY-SEC LOG-SEC ROTA SCHED RQ-SIZE RA WSAME
sdc 0 4096 16776704 4096 4096 1 mq-deadline
256 32764 0B
Quoted 5 lines
drivers/scsi/sd.c static int sd_revalidate_disk(struct gendisk *disk) · line 3791
Quoted 2 lines
drivers/scsi/sd.c +2
-0+ lim.io_opt =rounddown (lim.logical_block_size, + sdp->host->opt_sectors <<SECTOR_SHIFT );Those parameters would need to be reversed, I think. You want to round down
opt_sectorsand not thelogical_block_size.
Ugh! My bad! Will send v2.
Re: [PATCH 1/1] scsi: sd: Rounddown host->opt_sectors to logical
In reply to Martin K. PetersenQuoted 7 lines · Martin K. Petersen
Lukas,
Quoted 3 lines
The
host->opt_sectorsare by default 512 bytes aligned if we have 4KN SAS disk reporting zero or smalleropt_xfer_blockthan thehost->opt_sectorswe will end up with unalignedqueue_limit->io_opt.
Quoted 9 lines
I am intrigued wrt. which controller returns a
host->opt_sectorsthat would trigger this misalignment. What is the actual value reported?drivers/scsi/sd.c static int sd_revalidate_disk(struct gendisk *disk) · line 3791
Quoted 2 lines
drivers/scsi/sd.c +2
-0+ lim.io_opt =rounddown (lim.logical_block_size, + sdp->host->opt_sectors <<SECTOR_SHIFT );Those parameters would need to be reversed, I think. You want to round down
opt_sectorsand not thelogical_block_size.
In blk_validate_limits(), we already round down lim.io_opt to physical block size, and physical block size >= logical block size (so this extra rounding down should not be required). What am I missing?
Re: [PATCH 1/1] scsi: sd: Rounddown host->opt_sectors to logical
Quoted 11 lines · John Garry
Quoted 10 lines · Martin K. Petersen
Lukas,
Quoted 3 lines
The
host->opt_sectorsare by default 512 bytes aligned if we have 4KN SAS disk reporting zero or smalleropt_xfer_blockthan thehost->opt_sectorswe will end up with unalignedqueue_limit->io_opt.I am intrigued wrt. which controller returns a
host->opt_sectorsthat would trigger this misalignment. What is the actual value reported?
Quoted 8 lines
Quoted 6 lines
drivers/scsi/sd.c static int sd_revalidate_disk(struct gendisk *disk) · line 3791
Quoted 2 lines
drivers/scsi/sd.c +2
-0+ lim.io_opt =rounddown (lim.logical_block_size, + sdp->host->opt_sectors <<SECTOR_SHIFT );Those parameters would need to be reversed, I think. You want to round down
opt_sectorsand not thelogical_block_size.In
blk_validate_limits(), we already round down lim.io_opt to physical block size, and physical block size >= logical block size (so this extra rounding down should not be required). What am I missing?
Right, I overlooked it. The problematic kernel I was running is before this patch 9c0ba14828d64744ccd195c610594ba254a1a9ab which is setting the lim.io_opt to the proper value.
Thanks for the help!