mpt3sas: Set DMA_BIDIRECTIONAL for additional SCSI commands
Subsystems (2)
Ranjan Kumar
maintainerPATCH 1/1 mpt3sas: Set DMA_BIDIRECTIONAL for additional SCSI commands
Extend DMA direction override to handle additional SCSI commands (SECURITY_PROTOCOL_IN, SERVICE_ACTION_IN_16 with RELEASE) that require bidirectional DMA mapping, in addition to ZBC REPORT_ZONES. This avoids issues on platforms that perform strict DMA checks.
--- drivers/scsi/mpt3sas/mpt3sas_base.c | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-)
drivers/scsi/mpt3sas/mpt3sas_base.c +16 -2
@@ -2686,8 +2686,22 @@ static inline int _base_scsi_dma_map(struct scsi_cmnd *cmd)* (e.g. AMD hosts). Avoid such issue by making the report zones buffer * mapping bi-directional. */ -if (cmd->cmnd[0 ] ==ZBC_IN && cmd->cmnd[1 ] ==ZI_REPORT_ZONES ) - cmd->sc_data_direction =DMA_BIDIRECTIONAL ; + +switch (cmd->cmnd[0 ]) { +case SECURITY_PROTOCOL_IN : + cmd->sc_data_direction =DMA_BIDIRECTIONAL ; +break ; +case ZBC_IN : +if (cmd->cmnd[1 ] ==ZI_REPORT_ZONES ) + cmd->sc_data_direction =DMA_BIDIRECTIONAL ; +break ; +case SERVICE_ACTION_IN_16 : +if (cmd->cmnd[1 ] ==0x17 ) + cmd->sc_data_direction =DMA_BIDIRECTIONAL ; +break ; +default : +break ; + }return scsi_dma_map (cmd); }
Signature · 1 line
2.31.1
Quoted 26 lines · Ranjan Kumar
Extend DMA direction override to handle additional SCSI commands (
SECURITY_PROTOCOL_IN,SERVICE_ACTION_IN_16with RELEASE) that require bidirectional DMA mapping, in addition to ZBCREPORT_ZONES. This avoids issues on platforms that perform strict DMA checks.Signed-off-by Ranjan Kumar <[email protected]>--- drivers/scsi/mpt3sas/mpt3sas_base.c | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-)drivers/scsi/mpt3sas/mpt3sas_base.c +6
-2@@ -2686,8 +2686,22 @@ static inline int _base_scsi_dma_map(struct scsi_cmnd *cmd)* (e.g. AMD hosts). Avoid such issue by making the report zones buffer * mapping bi-directional. */ -if (cmd->cmnd[0 ] ==ZBC_IN && cmd->cmnd[1 ] ==ZI_REPORT_ZONES ) - cmd->sc_data_direction =DMA_BIDIRECTIONAL ; + +switch (cmd->cmnd[0 ]) { +case SECURITY_PROTOCOL_IN : + cmd->sc_data_direction =DMA_BIDIRECTIONAL ; +break ; +case ZBC_IN :
drivers/scsi/mpt3sas/mpt3sas_base.c static inline int _base_scsi_dma_map(struct scsi_cmnd *cmd) · line 2704
Quoted 10 lines
drivers/scsi/mpt3sas/mpt3sas_base.c +10
-0+if (cmd->cmnd[1 ] ==ZI_REPORT_ZONES ) + cmd->sc_data_direction =DMA_BIDIRECTIONAL ; +break ; +case SERVICE_ACTION_IN_16 : +if (cmd->cmnd[1 ] ==0x17 ) + cmd->sc_data_direction =DMA_BIDIRECTIONAL ; +break ; +default : +break ; + }
Might be my mail client, but indentation looks broken here.
Quoted 4 lines · Ranjan Kumar
Extend DMA direction override to handle additional SCSI commands (
SECURITY_PROTOCOL_IN,SERVICE_ACTION_IN_16with RELEASE) that require bidirectional DMA mapping, in addition to ZBCREPORT_ZONES. This avoids issues on platforms that perform strict DMA checks.
I think you are totally misstating the problem here.
The Broadcom MPT3SAS implementation, probablt the SATL is implemented gravely incorrectly, and rewrites data structure in place against the protocol requirements, which is cought by all but the dumbest IOMMUs, but Broadcome until now never bothered to actually rest the junk they ship out to customers fully.
Sathya Prakash Veerichetty
maintainerQuoted 7 lines · Christoph Hellwig
Quoted 4 lines · Ranjan Kumar
Extend DMA direction override to handle additional SCSI commands (
SECURITY_PROTOCOL_IN,SERVICE_ACTION_IN_16with RELEASE) that require bidirectional DMA mapping, in addition to ZBCREPORT_ZONES. This avoids issues on platforms that perform strict DMA checks.
Quoted 7 lines
I think you are totally misstating the problem here.
The Broadcom MPT3SAS implementation, probablt the SATL is implemented gravely incorrectly, and rewrites data structure in place against the protocol requirements, which is cought by all but the dumbest IOMMUs, but Broadcome until now never bothered to actually rest the junk they ship out to customers fully.
There is a resource issue which prevents SATL from doing double buffering and RMW of the contents to map the data from SATA drives to the host buffers for the specific commands. Our firmware changes leads to performance and functional issues, So we are left with either failing those commands in the driver or changing the mapping, We decided to go with changing the DMA mapping as we know those commands require both read/write access to the buffers.
Any functional issue of changing the DMA mapping?
Quoted 8 lines · Sathya Prakash Veerichetty
There is a resource issue which prevents SATL from doing double buffering and RMW of the contents to map the data from SATA drives to the host buffers for the specific commands. Our firmware changes leads to performance and functional issues, So we are left with either failing those commands in the driver or changing the mapping, We decided to go with changing the DMA mapping as we know those commands require both read/write access to the buffers.
Any functional issue of changing the DMA mapping?
Well, we have to change the mapping to make your broken hardware work. But these kinds of hacks in firmware are really problematic. When an application does a passthrough of a SCSI command it can very much expect that the payload is not changed by a call to SG_IO, and this breaks it.
Sathya Prakash Veerichetty
maintainerQuoted 11 lines · Christoph Hellwig
Quoted 8 lines · Sathya Prakash Veerichetty
There is a resource issue which prevents SATL from doing double buffering and RMW of the contents to map the data from SATA drives to the host buffers for the specific commands. Our firmware changes leads to performance and functional issues, So we are left with either failing those commands in the driver or changing the mapping, We decided to go with changing the DMA mapping as we know those commands require both read/write access to the buffers.
Any functional issue of changing the DMA mapping?
Quoted 6 lines
Well, we have to change the mapping to make your broken hardware work. But these kinds of hacks in firmware are really problematic. When an application does a passthrough of a SCSI command it can very much expect that the payload is not changed by a call to
SG_IO, and this breaks it.
Agree, that is the reason, we don't want to change the mapping outside the driver. In this case, the problem is with reading the payload (the write access was already available in the default mapping). We tried a couple of long term solutions in the firmware and with the current hardware capabilities they were not successful. The other option we have other than modifying the mapping is to fail the commands but that will result in functional failure. So we took the least impactful approach.
Quoted 26 lines · Ranjan Kumar
Extend DMA direction override to handle additional SCSI commands (
SECURITY_PROTOCOL_IN,SERVICE_ACTION_IN_16with RELEASE) that require bidirectional DMA mapping, in addition to ZBCREPORT_ZONES. This avoids issues on platforms that perform strict DMA checks.Signed-off-by Ranjan Kumar <[email protected]>--- drivers/scsi/mpt3sas/mpt3sas_base.c | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-)drivers/scsi/mpt3sas/mpt3sas_base.c +6
-2@@ -2686,8 +2686,22 @@ static inline int _base_scsi_dma_map(struct scsi_cmnd *cmd)* (e.g. AMD hosts). Avoid such issue by making the report zones buffer * mapping bi-directional. */ -if (cmd->cmnd[0 ] ==ZBC_IN && cmd->cmnd[1 ] ==ZI_REPORT_ZONES ) - cmd->sc_data_direction =DMA_BIDIRECTIONAL ; + +switch (cmd->cmnd[0 ]) { +case SECURITY_PROTOCOL_IN : + cmd->sc_data_direction =DMA_BIDIRECTIONAL ; +break ; +case ZBC_IN :
drivers/scsi/mpt3sas/mpt3sas_base.c static inline int _base_scsi_dma_map(struct scsi_cmnd *cmd) · line 2704
Quoted 10 lines
drivers/scsi/mpt3sas/mpt3sas_base.c +10
-0+if (cmd->cmnd[1 ] ==ZI_REPORT_ZONES ) + cmd->sc_data_direction =DMA_BIDIRECTIONAL ; +break ; +case SERVICE_ACTION_IN_16 : +if (cmd->cmnd[1 ] ==0x17 ) + cmd->sc_data_direction =DMA_BIDIRECTIONAL ; +break ; +default : +break ; + }
Very broken indentation. And the comment above this hunk would need to be updated too.
This really has to stop. The need for modifying a data buffer content when translating between SCSI and ATA is nothing new. HBA hardware designs that cannot handle that internally are simply broken and should be fixed. Or better:
make your HBA use libsas and rely on libata-scsi SAT.
The commands above are not "performance" commands as they are generally not issued in the hot path under heavy read/write workload. So even if they are a little slower because of the HBA double buffering, that's fine. Relying on unauthorized accesses to host memory to replace the HBA internal buffering is probably not faster at all anyway.
I was already very sad/disapointed to stumble on the report zones issue. Adding more commands to this is really not desired at all. Where does it stop ? After all, you are not doing anything like this for translating ATA log pages to mode/vpd pages, right ? So the HBA *is* capable of double buffering, no ?
Your commit message says nothing of the use case that this fixes. Is there an actual problem in the field without this modification ? Without a better justification, I am not in favor of this and would prefer the fixes to go into the HBA firmware.
Signature · 2 lines
Damien Le Moal Western Digital Research
Sathya Prakash Veerichetty
maintainerQuoted 33 lines · Damien Le Moal
Quoted 31 lines · Ranjan Kumar
Extend DMA direction override to handle additional SCSI commands (
SECURITY_PROTOCOL_IN,SERVICE_ACTION_IN_16with RELEASE) that require bidirectional DMA mapping, in addition to ZBCREPORT_ZONES. This avoids issues on platforms that perform strict DMA checks.Signed-off-by Ranjan Kumar <[email protected]>--- drivers/scsi/mpt3sas/mpt3sas_base.c | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-)drivers/scsi/mpt3sas/mpt3sas_base.c +11
-2@@ -2686,8 +2686,22 @@ static inline int _base_scsi_dma_map(struct scsi_cmnd *cmd)* (e.g. AMD hosts). Avoid such issue by making the report zones buffer * mapping bi-directional. */ -if (cmd->cmnd[0 ] ==ZBC_IN && cmd->cmnd[1 ] ==ZI_REPORT_ZONES ) - cmd->sc_data_direction =DMA_BIDIRECTIONAL ; + +switch (cmd->cmnd[0 ]) { +case SECURITY_PROTOCOL_IN : + cmd->sc_data_direction =DMA_BIDIRECTIONAL ; +break ; +case ZBC_IN : +if (cmd->cmnd[1 ] ==ZI_REPORT_ZONES ) + cmd->sc_data_direction =DMA_BIDIRECTIONAL ; +break ; +case SERVICE_ACTION_IN_16 : +if (cmd->cmnd[1 ] ==0x17 )
Quoted 8 lines
drivers/scsi/mpt3sas/mpt3sas_base.c static inline int _base_scsi_dma_map(struct scsi_cmnd *cmd) · line 2704
Quoted 5 lines
drivers/scsi/mpt3sas/mpt3sas_base.c +5
-0+ cmd->sc_data_direction =DMA_BIDIRECTIONAL ; +break ; +default : +break ; + }Very broken indentation. And the comment above this hunk would need to be updated too.
Will fix it in the next version of the patch.
Quoted 6 lines
This really has to stop. The need for modifying a data buffer content when translating between SCSI and ATA is nothing new. HBA hardware designs that cannot handle that internally are simply broken and should be fixed. Or better:
make your HBA use libsas and rely on libata-scsi SAT.
Quoted 6 lines
The commands above are not "performance" commands as they are generally not issued in the hot path under heavy read/write workload. So even if they are a little slower because of the HBA double buffering, that's fine. Relying on unauthorized accesses to host memory to replace the HBA internal buffering is probably not faster at all anyway.
There were multiple attempts to work this around in our firmware in the existing generation of the products for the last few months and with the limited memory resources available in the controller, performing double buffering of the commands for 1024 drives were resulting in resource unavailability and commands getting timed out and resets. So no controller firmware level solution is viable at this moment.
Quoted 9 lines
I was already very sad/disapointed to stumble on the report zones issue. Adding more commands to this is really not desired at all. Where does it stop ? After all, you are not doing anything like this for translating ATA log pages to mode/vpd pages, right ? So the HBA *is* capable of double buffering, no ?
Your commit message says nothing of the use case that this fixes. Is there an actual problem in the field without this modification ? Without a better justification, I am not in favor of this and would prefer the fixes to go into the HBA firmware.
There are no real use cases we have encountered. What we want to protect is if one such command is issued we shouldn't end up in IOMMU fault, with that said, we have two options
1. Change the dma mapping as mentioned in this patch
2. Modify the drivers to fail the specific commands as unsupported.
Quoted 5 lines
--
Damien Le Moal
Western Digital Research