scsi: ufs: Increase the maximum data buffer size
Subsystems (2)
Bart Van Assche
reviewerPATCH v3 1/1 scsi: ufs: Increase the maximum data buffer size
Measurements for one particular UFS controller + UFS device show a 25% higher read bandwidth if the maximum data buffer size is increased from 512 KiB to 1 MiB. Hence increase the maximum size of the data buffer associated with a single request from SCSI_DEFAULT_MAX_SECTORS (1024)
* 512 bytes = 512 KiB to 1 MiB.
Notes:
- The maximum data buffer size supported by the UFSHCI specification is 65535 * 256 KiB or about 16 GiB.
- The maximum data buffer size for READ(10) commands is 65535 logical blocks. To transfer more than 65535 * 4096 bytes = 255 MiB with a single SCSI command, the READ(16) command is required. Support for READ(16) is optional in the UFS 3.1 and UFS 4.0 standards.
---
Changes compared to v2: changed maximum transfer size 255 MiB to 1 MiB. Changes compared to v1: changed maximum transfer size from 1 GiB to 255 MiB.
drivers/ufs/core/ufshcd.c | 1 + 1 file changed, 1 insertion(+)
drivers/ufs/core/ufshcd.c +1 -0
@@ -8365,6 +8365,7 @@ static struct scsi_host_template ufshcd_driver_template = { .cmd_per_lun = UFSHCD_CMD_PER_LUN, .can_queue = UFSHCD_CAN_QUEUE, .max_segment_size = PRDT_DATA_BYTE_COUNT_MAX, + .max_sectors = (1 << 20) / SECTOR_SIZE, /* 1 MiB */ .max_host_blocked = 1, .track_queue_depth = 1, .sdev_groups = ufshcd_driver_groups,
Quoted 14 lines
Measurements for one particular UFS controller + UFS device show a 25% higher read bandwidth if the maximum data buffer size is increased from 512 KiB to 1 MiB. Hence increase the maximum size of the data buffer associated with a single request from
SCSI_DEFAULT_MAX_SECTORS(1024)
* 512 bytes = 512 KiB to 1 MiB.Notes:
- The maximum data buffer size supported by the UFSHCI specification is 65535 * 256 KiB or about 16 GiB.
- The maximum data buffer size for READ(10) commands is 65535 logical blocks. To transfer more than 65535 * 4096 bytes = 255 MiB with a single SCSI command, the READ(16) command is required. Support for READ(16) is optional in the UFS 3.1 and UFS 4.0 standards.
Quoted 5 lines
Cc Adrian Hunter <[email protected]>Cc Avri Altman <[email protected]>Cc Bean Huo <[email protected]>Cc Stanley Chu <[email protected]>Signed-off-by Bart Van Assche <[email protected]>
Martin K. Petersen
AppliedBart,
Quoted 5 lines
Measurements for one particular UFS controller + UFS device show a 25% higher read bandwidth if the maximum data buffer size is increased from 512 KiB to 1 MiB. Hence increase the maximum size of the data buffer associated with a single request from
SCSI_DEFAULT_MAX_SECTORS(1024)
* 512 bytes = 512 KiB to 1 MiB.
Applied to 5.20/scsi-staging, thanks!
Signature · 1 line
Martin K. Petersen Oracle Linux Engineering
Hi bart
Is it possible by adding only max_sector to increase the data buffer size? I think the data buffer will split to 512 KiB, because the sg_table size is SG_ALL
Thanks,
yohan
Quoted 50 lines
-----Original Message-----
From: Bart Van Assche <[email protected]>
Sent: Wednesday, July 27, 2022 7:52 AM
To: Martin K . Petersen <[email protected]>
Cc: Jaegeuk Kim <[email protected]>; [email protected]; Adrian Hunter <[email protected]>; Bart Van Assche <[email protected]>; Avri Altman <[email protected]>; Bean Huo <[email protected]>; Stanley Chu <[email protected]>; James E.J. Bottomley <[email protected]>; Matthias Brugger <[email protected]>
Subject: [PATCH v3] scsi: ufs: Increase the maximum data buffer sizeMeasurements for one particular UFS controller + UFS device show a 25% higher read bandwidth if the maximum data buffer size is increased from 512 KiB to 1 MiB. Hence increase the maximum size of the data buffer associated with a single request from
SCSI_DEFAULT_MAX_SECTORS(1024)
* 512 bytes = 512 KiB to 1 MiB.Notes:
- The maximum data buffer size supported by the UFSHCI specification is 65535 * 256 KiB or about 16 GiB.
- The maximum data buffer size for READ(10) commands is 65535 logical blocks. To transfer more than 65535 * 4096 bytes = 255 MiB with a single SCSI command, the READ(16) command is required. Support for READ(16) is optional in the UFS 3.1 and UFS 4.0 standards.Cc Adrian Hunter <[email protected]>Cc Avri Altman <[email protected]>Cc Bean Huo <[email protected]>Cc Stanley Chu <[email protected]>Signed-off-by Bart Van Assche <[email protected]>---
Changes compared to v2: changed maximum transfer size 255 MiB to 1 MiB. Changes compared to v1: changed maximum transfer size from 1 GiB to 255 MiB.drivers/ufs/core/ufshcd.c | 1 + 1 file changed, 1 insertion(+)drivers/ufs/core/ufshcd.c index renamed from drivers/ufs/core/ufshcd.c +0
-0
36b7212e9cb5..678bc8d6d6aa100644drivers/ufs/core/ufshcd.c +1
-0@@ -8365,6 +8365,7 @@ static struct scsi_host_template ufshcd_driver_template = { .cmd_per_lun = UFSHCD_CMD_PER_LUN, .can_queue = UFSHCD_CAN_QUEUE, .max_segment_size = PRDT_DATA_BYTE_COUNT_MAX, + .max_sectors = (1 << 20) / SECTOR_SIZE, /* 1 MiB */ .max_host_blocked = 1, .track_queue_depth = 1,.sdev_groups =
ufshcd_driver_groups,
Bart Van Assche
reviewerQuoted 1 line · On 8/2/22 16:40, [email protected] wrote:
Is it possible by adding only
max_sectorto increase the data buffer size?
Yes.
Quoted 1 line
I think the data buffer will split to 512 KiB, because the
sg_tablesize isSG_ALL
I don't think so. With this patch applied, the limits supported by the UFS driver are as follows:
.sg_tablesize = SG_ALL, /* 128 */ .max_segment_size = PRDT_DATA_BYTE_COUNT_MAX, /* 256 KiB*/ .max_sectors = (1 << 20) / SECTOR_SIZE, /* 1 MiB */
So the maximum data buffer size is min(max_sectors * 512, sg_tablesize * max_segment_size) = min(1 MiB, 128 * 256 KiB) = 1 MiB. On a system with 4 KiB pages, the data buffer size will be 128 * 4 KiB = 512 MiB if none of the pages involved in the I/O are contiguous.
Bart.
Quoted 11 lines · On 8/2/22 16:40, [email protected] wrote:
Quoted 1 line
Is it possible by adding only
max_sectorto increase the data buffer size?Yes.
Quoted 2 lines
I think the data buffer will split to 512 KiB, because the
sg_tablesize isSG_ALLI don't think so. With this patch applied, the limits supported by the UFS driver are as follows:
Quoted 8 lines
.sg_tablesize = SG_ALL, /* 128 */ .max_segment_size = PRDT_DATA_BYTE_COUNT_MAX, /* 256 KiB*/ .max_sectors = (1 << 20) / SECTOR_SIZE, /* 1 MiB */So the maximum data buffer size is
min(max_sectors * 512, sg_tablesize * max_segment_size)=min(1 MiB, 128 * 256 KiB)= 1 MiB. On a system with 4 KiB pages, the data buffer size will be 128 * 4 KiB = 512 MiB if none of the pages involved in the I/O are contiguous.
In block layer, max_segment_size is obtained from get_max_segment_size. seg_boundary_mask is set to PAGE_SIZE - 1 in the ufs driver.
The segment size is the PAGE size, and the max buffer size is segment size * max segment count ( PAGE SIZE * 128 ) = 512 KiB in block layer Right?
Quoted 2 lines
Bart.
Bart Van Assche
reviewerQuoted 5 lines · On 8/3/22 18:50, [email protected] wrote:
In block layer,
max_segment_sizeis obtained fromget_max_segment_size.seg_boundary_maskis set toPAGE_SIZE- 1 in the ufs driver.
The segment size is the PAGE size, and the max buffer size is segment size * max segment count ( PAGE SIZE * 128 ) = 512 KiB in block layer Right?
Thanks for having reported this. I had overlooked that the UFS host controller driver sets the dma_boundary member of the host_template field. Is my understanding correct that UFS host controllers should support DMA segments that consist of multiple physical pages?
Thanks,
Bart.
Quoted 10 lines
Quoted 5 lines
In block layer,
max_segment_sizeis obtained fromget_max_segment_size.seg_boundary_maskis set toPAGE_SIZE- 1 in the ufs driver. The segment size is the PAGE size, and the max buffer size is segment size * max segment count ( PAGE SIZE * 128 ) = 512 KiB in block layer Right?Thanks for having reported this. I had overlooked that the UFS host controller driver sets the
dma_boundarymember of thehost_templatefield. Is my understanding correct that UFS host controllers should support DMA segments that consist of multiple physical pages?
max value of data byte count is 256kb in PRDT.
256kb (data byte count) * 128 (max segments) = 32768kb
If physical pages are continuous, it seems that IO can be delivered up to 32mb. performance checks are required, but we don't have to consider saturation. because the device vendor can set opt_xfer_blocks .
Thanks,
yohan
Quoted 4 lines
Thanks,
Bart.
Quoted 27 lines · Bart Van Assche
Quoted 2 lines · On 8/2/22 16:40, [email protected] wrote:
Is it possible by adding only
max_sectorto increase the data buffer size?Yes.
Quoted 2 lines
I think the data buffer will split to 512 KiB, because the
sg_tablesize isSG_ALLI don't think so. With this patch applied, the limits supported by the
UFS driver are as follows:.sg_tablesize = SG_ALL, /* 128 */ .max_segment_size =
PRDT_DATA_BYTE_COUNT_MAX, /* 256 KiB*/
.max_sectors = (1 << 20) / SECTOR_SIZE, /* 1 MiB */So the maximum data buffer size is min(max_sectors * 512,
sg_tablesize*max_segment_size) =min(1 MiB, 128 * 256 KiB)= 1 MiB. On a system with
4 KiB pages, the data buffer size will be 128 * 4 KiB = 512 MiB if none
of the pages involved in the I/O are contiguous.
Quoted 1 line
Bart.
Bart,
This change just increases the shost->max_sectors limit from 501KB to 1Mb, but the final value will be overridden by the optimal transfer length defined in the VPD, right?
Kind regards,
Bean
Bart Van Assche
reviewerQuoted 28 lines · Bean Huo
Quoted 19 lines · Bart Van Assche
Quoted 2 lines · On 8/2/22 16:40, [email protected] wrote:
Is it possible by adding only
max_sectorto increase the data buffer size?Yes.
Quoted 2 lines
I think the data buffer will split to 512 KiB, because the
sg_tablesize isSG_ALLI don't think so. With this patch applied, the limits supported by the
UFS driver are as follows:.sg_tablesize = SG_ALL, /* 128 */ .max_segment_size =
PRDT_DATA_BYTE_COUNT_MAX, /* 256
KiB*/
.max_sectors = (1 << 20) / SECTOR_SIZE, /* 1 MiB
*/Quoted 7 lines
So the maximum data buffer size is min(max_sectors * 512,
sg_tablesize*max_segment_size) =min(1 MiB, 128 * 256 KiB)= 1 MiB. On a system with
4 KiB pages, the data buffer size will be 128 * 4 KiB = 512 MiB if none
of the pages involved in the I/O are contiguous.
Quoted 3 lines
This change just increases the
shost->max_sectorslimit from 501KB to 1Mb, but the final value will be overridden by the optimal transfer length defined in the VPD, right?
Hi Bean,
It seems to me that the block layer only uses the optimal transfer size (io_opt) to determine how much data to read ahead during sequential reads? See also disk_update_readahead().
The above patch increases max_sectors but is not sufficient to increase the maximum transfer size: SG_ALL (128) * 4 KiB (dma_boundary + 1) = 512 KiB. To increase the maximum transfer size, the dma_boundary parameter would have to be modified. I have not yet submitted a patch that modifies that parameter since on my test setup (Exynos host controller) the current value is the largest value supported.
Thanks,
Bart.
Quoted 8 lines · Bart Van Assche
Hi Bean,
It seems to me that the block layer only uses the optimal transfer size
(io_opt) to determine how much data to read ahead during sequential reads? See alsodisk_update_readahead().
Bart,
Sorry for later reply, Didn't notice a question here.
In the upstream standard Linux kernel, if the device supports a valid VPD, I think, your this change will also not change the read-ahead window.
....
sdkp->opt_xfer_blocks = get_unaligned_be32(&vpd->data[12]);
sd_revalidate_disk() {
...
if (sd_validate_opt_xfer_size(sdkp, dev_max)) {
q->limits.io_opt = logical_to_bytes(sdp, sdkp->opt_xfer_blocks);
rw_max = logical_to_sectors(sdp, sdkp->opt_xfer_blocks); }
...
}
device_add_disk()
disk_update_readahead(disk) {
disk->bdi->ra_pages =
max(queue_io_opt(q) * 2 / PAGE_SIZE,
VM_READAHEAD_PAGES);
}e.g., if device reports opt 512KB, and the maximum transfer length will be 512KB, and readahead window will be 1MB.
Kind regards,
Bean