block: remove artifical max_hw_sectors cap
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 4 lines · Stefan Priebe - Profihost AG
Hi,
since the below patch i've some problems with iscsi.
Quoted 8 lines
The LIO based iscsi Server is full of messages like this:
SCSI OP 2ah with too big sectors 8344 exceedsfabric_max_sectors: 8192And the client says:
Buffer I/O error on device sdd, logical block 38861907 lost page write due to I/O error on sddMay be some code fixes for iscsi client is missing?
No, the problem seems that the the target is enforcing a size limit without communicating it through the block limits EVPD page.
Nic, can you fix LIO to expose the proper max xfer size?
Hi Christoph,
Am 23.12.2014 um 09:28 schrieb Christoph Hellwig:
Quoted 12 lines · Stefan Priebe - Profihost AG
Quoted 11 lines
Hi,
since the below patch i've some problems with iscsi.
The LIO based iscsi Server is full of messages like this:
SCSI OP 2ah with too big sectors 8344 exceedsfabric_max_sectors: 8192And the client says:
Buffer I/O error on device sdd, logical block 38861907 lost page write due to I/O error on sdd
Quoted 6 lines
Quoted 1 line
May be some code fixes for iscsi client is missing?
No, the problem seems that the the target is enforcing a size limit without communicating it through the block limits EVPD page.
Nic, can you fix LIO to expose the proper max xfer size?
some more problems while running this patch.
My crucial m500 and m550 ssds (i have around 60 in 30 different cases and motherboards but all attached as jbods at LSI HBAs) print those errors:
Dec 23 03:14:56 cloud2-1375 kernel: [585489.194158] Write(10): 2a 00 04 f9 db f0 00 0c d8 00 Dec 23 03:14:56 cloud2-1375 kernel: [585489.194162] scsi target0:0:7: handle(0x000a), sas_address(0x4433221107000000), phy(7) Dec 23 03:14:56 cloud2-1375 kernel: [585489.194163] scsi target0:0:7: enclosure_logical_id(0x5003048016aee700), slot(5) Dec 23 03:14:56 cloud2-1375 kernel: [585489.194178] sd 0:0:7:0: task abort: SUCCESS scmd(ffff88018c6b4700) Dec 23 03:14:56 cloud2-1375 kernel: [585489.194178] sd 0:0:7:0: attempting task abort! scmd(ffff8807a12e6800) Dec 23 03:14:56 cloud2-1375 kernel: [585489.194179] sd 0:0:7:0: [sdh] CDB: Dec 23 03:14:56 cloud2-1375 kernel: [585489.194180] Read(10): 28 00 15 d4 f9 60 00 00 10 00 Dec 23 03:14:56 cloud2-1375 kernel: [585489.194183] scsi target0:0:7: handle(0x000a), sas_address(0x4433221107000000), phy(7) Dec 23 03:14:56 cloud2-1375 kernel: [585489.194184] scsi target0:0:7: enclosure_logical_id(0x5003048016aee700), slot(5) Dec 23 03:14:56 cloud2-1375 kernel: [585489.194199] sd 0:0:7:0: task abort: SUCCESS scmd(ffff8807a12e6800) Dec 23 03:14:56 cloud2-1375 kernel: [585489.194199] sd 0:0:7:0: attempting task abort! scmd(ffff88078d989800) Dec 23 03:14:56 cloud2-1375 kernel: [585489.194200] sd 0:0:7:0: [sdh] CDB: Dec 23 03:14:56 cloud2-1375 kernel: [585489.194201] Read(10): 28 00 15 aa d2 f9 00 00 20 00
Stefan
Quoted 6 lines · Stefan Priebe
Quoted 1 line
Nic, can you fix LIO to expose the proper max xfer size?
some more problems while running this patch.
My crucial m500 and m550 ssds (i have around 60 in 30 different cases and motherboards but all attached as jbods at LSI HBAs) print those errors:
What is the max_sectors_kb value for them?
Am 30.12.2014 um 12:32 schrieb Christoph Hellwig:
Quoted 10 lines · Stefan Priebe
Quoted 6 lines
Quoted 1 line
Nic, can you fix LIO to expose the proper max xfer size?
some more problems while running this patch.
My crucial m500 and m550 ssds (i have around 60 in 30 different cases and motherboards but all attached as jbods at LSI HBAs) print those errors:
What is the
max_sectors_kbvalue for them?
# cat /sys/block/sdi/queue/max_sectors_kb 16383
Greets,
Stefan
Quoted 4 lines · Stefan Priebe - Profihost AG
Quoted 2 lines
What is the
max_sectors_kbvalue for them?# cat /sys/block/sdi/queue/max_sectors_kb 16383
That's odd, it's half of what ATA disks should support. Is this device attached to the HBA or an expander?
Am 05.01.2015 um 18:17 schrieb Christoph Hellwig:
Quoted 8 lines · Stefan Priebe - Profihost AG
Quoted 4 lines
Quoted 2 lines
What is the
max_sectors_kbvalue for them?# cat /sys/block/sdi/queue/max_sectors_kb 16383That's odd, it's half of what ATA disks should support. Is this device attached to the HBA or an expander?
No the Board has:
10x SATA3 (6Gbps) via C612 from Intel.
Stefan
Quoted 12 lines · Christoph Hellwig
Quoted 11 lines · Stefan Priebe - Profihost AG
Hi,
since the below patch i've some problems with iscsi.
The LIO based iscsi Server is full of messages like this:
SCSI OP 2ah with too big sectors 8344 exceedsfabric_max_sectors: 8192And the client says:
Buffer I/O error on device sdd, logical block 38861907 lost page write due to I/O error on sdd
Quoted 6 lines
Quoted 1 line
May be some code fixes for iscsi client is missing?
No, the problem seems that the the target is enforcing a size limit without communicating it through the block limits EVPD page.
Nic, can you fix LIO to expose the proper max xfer size?
The fabric_max_sectors=8192 value is already being exposed by target in block limits EVPD as MAXIMUM TRANSFER LENGTH.
I'm guessing that since the host side support was not added until June 2014 in commit bcdb247c by MKP, Stefan is likely using an earlier initiator that is not honoring this value.
However, this same issue has been reported elsewhere [1] with an MSFT FC host sending I/Os of 8 MB that also doesn't appear to honor block limits MAXIMUM TRANSFER LENGTH either.
So that said, I think it's about time to go ahead and drop the arbitrary limitation of fabric_max_sectors, and simply enforce a sane limit (eg:
not UINT_MAX) for hw_max_sectors.
The only limitations for backends that I'm aware of is the FILEIO FD_MAX_BYTES limit for the number of iovecs per vfs_[writev,readv] call, and for IBLOCK what queue_max_hw_sectors() reports for raw block make_request() based drivers that aren't able to
I'll put together a patch soon to drop the fabric_max_sectors stuff, and use the existing hw_max_sectors to enforce any backend specific I/O size limitations.
--nab
Quoted 31 lines · Nicholas A. Bellinger
Quoted 12 lines · Christoph Hellwig
Quoted 11 lines · Stefan Priebe - Profihost AG
Hi,
since the below patch i've some problems with iscsi.
The LIO based iscsi Server is full of messages like this:
SCSI OP 2ah with too big sectors 8344 exceedsfabric_max_sectors: 8192And the client says:
Buffer I/O error on device sdd, logical block 38861907 lost page write due to I/O error on sddQuoted 6 lines
Quoted 1 line
May be some code fixes for iscsi client is missing?
No, the problem seems that the the target is enforcing a size limit without communicating it through the block limits EVPD page.
Nic, can you fix LIO to expose the proper max xfer size?
The fabric_max_sectors=8192 value is already being exposed by target in block limits EVPD as MAXIMUM TRANSFER LENGTH.
I'm guessing that since the host side support was not added until June 2014 in commit
bcdb247cby MKP, Stefan is likely using an earlier initiator that is not honoring this value.However, this same issue has been reported elsewhere [1] with an MSFT FC host sending I/Os of 8 MB that also doesn't appear to honor block limits MAXIMUM TRANSFER LENGTH either.
Quoted 9 lines
So that said, I think it's about time to go ahead and drop the arbitrary limitation of
fabric_max_sectors, and simply enforce a sane limit (eg:
notUINT_MAX) forhw_max_sectors.The only limitations for backends that I'm aware of is the FILEIO
FD_MAX_BYTESlimit for the number of iovecs per vfs_[writev,readv] call, and for IBLOCK whatqueue_max_hw_sectors()reports for raw blockmake_request()based drivers that aren't able to
... split large I/O requests into smaller ones.
Quoted 12 lines
I'll put together a patch soon to drop the
fabric_max_sectorsstuff, and use the existinghw_max_sectorsto enforce any backend specific I/O size limitations.--nab
[1] http://www.spinics.net/lists/target-devel/msg08024.html
--
To unsubscribe from this list: send the line "unsubscribe target-devel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 6 lines · Nicholas A. Bellinger
The fabric_max_sectors=8192 value is already being exposed by target in block limits EVPD as MAXIMUM TRANSFER LENGTH.
I'm guessing that since the host side support was not added until June 2014 in commit
bcdb247cby MKP, Stefan is likely using an earlier initiator that is not honoring this value.
If I understand Stefan correct he was using a recent 3.19-rc kernel, which contains my commit in the subject that stops capping the max_sectors used by the kernel below that which the hardware supports.
Given that LIO actually implements the block limits EVPD that suggests we're not properly using that information.
Stefant, what does:
sg_vpd -p bl /dev/sdX
for the LIO device tell?
Am 07.01.2015 um 15:31 schrieb Christoph Hellwig:
Quoted 12 lines · Nicholas A. Bellinger
Quoted 6 lines
The fabric_max_sectors=8192 value is already being exposed by target in block limits EVPD as MAXIMUM TRANSFER LENGTH.
I'm guessing that since the host side support was not added until June 2014 in commit
bcdb247cby MKP, Stefan is likely using an earlier initiator that is not honoring this value.If I understand Stefan correct he was using a recent 3.19-rc kernel, which contains my commit in the subject that stops capping the
max_sectorsused by the kernel below that which the hardware supports.
Quoted 8 lines
Given that LIO actually implements the block limits EVPD that suggests we're not properly using that information.
Stefant, what does:
sg_vpd-p bl/dev/sdXfor the LIO device tell?
I'm sorry i'm using a 3.10.63 vanilla kernel. So i'm missing a commit?
Stefan
Quoted 7 lines · Stefan Priebe - Profihost AG
Quoted 7 lines
If I understand Stefan correct he was using a recent 3.19-rc kernel, which contains my commit in the subject that stops capping the
max_sectorsused by the kernel below that which the hardware supports.Given that LIO actually implements the block limits EVPD that suggests we're not properly using that information.
Quoted 7 lines
Quoted 5 lines
Stefant, what does:
sg_vpd-p bl/dev/sdXfor the LIO device tell?
I'm sorry i'm using a 3.10.63 vanilla kernel. So i'm missing a commit?
You said "block: remove artifical max_hw_sectors cap" causes the I/O errors for you, which should only be in a 3.19-ish kernel.