Skip to contentSkip to conversation

ARM SMMU - Stall mode

DISCUSSION Will Deacon 2 participants 3 messages Mar 19, 2015 – Mar 23, 2015 2 messages not in the archive

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): 20150319065207.GE23658@toto

#1 lore raw
Quoted 1 line · Edgar E. Iglesias

Hi,

Hello Edgar,

Quoted 3 lines

I'm looking at adding models for the SMMU and it's stall feature to Xilinx ZynqMP QEMU models. The stall mode is quite problematic to model though. So I'm trying to figure out if I can get away without it :-)

Sure -- just make the context bank SCTLR.CFCFG fields RES0 and you'll be fine. The stall mode is optional in the architecture and not used by Linux.

Quoted 9 lines

I was trying to figure from the kernel sources if the SMMU Stall mode (for stalling and later resuming failed memory accesses) is used by the kernel.

AFAICT, the stall mode is enabled by the SMMU driver but the users of the report_iommu_fault() callbacks seem to all just do error reporting. Do I understand correctly in that Linux/KVM is currently not really depending on the stall function?

Quoted 6 lines

Are there any plans or ongoing work to implement paging features that would require the stall/resume feature for IOMMUs?
E.g swapping of DMA:able memory, CoW etc.

I noticed that AMD has something in their HSA/KFD framework around it but it seems quite AMD specific, please correct me if I'm wrong.

The main issue with the stall model is that it doesn't work nicely with PCI and is prone to deadlock on systems that haven't been designed with stalled transactions in mind. If/when Linux gets support for paged DMA memory this will probably be using ATS/PRI.

Will

A message that is not in the archive (sent off-list or not yet received): [email protected]

#3 lore raw
Quoted 26 lines · Will Deacon
Quoted 1 line · Edgar E. Iglesias

Hi,

Hello Edgar,

Quoted 3 lines

I'm looking at adding models for the SMMU and it's stall feature to Xilinx ZynqMP QEMU models. The stall mode is quite problematic to model though. So I'm trying to figure out if I can get away without it :-)

Sure -- just make the context bank SCTLR.CFCFG fields RES0 and you'll be fine. The stall mode is optional in the architecture and not used by Linux.

Quoted 13 lines

I was trying to figure from the kernel sources if the SMMU Stall mode (for stalling and later resuming failed memory accesses) is used by the kernel.

AFAICT, the stall mode is enabled by the SMMU driver but the users of the report_iommu_fault() callbacks seem to all just do error reporting. Do I understand correctly in that Linux/KVM is currently not really depending on the stall function?

Are there any plans or ongoing work to implement paging features that would require the stall/resume feature for IOMMUs?
E.g swapping of DMA:able memory, CoW etc.

Quoted 7 lines
Quoted 2 lines

I noticed that AMD has something in their HSA/KFD framework around it but it seems quite AMD specific, please correct me if I'm wrong.

The main issue with the stall model is that it doesn't work nicely with PCI and is prone to deadlock on systems that haven't been designed with stalled transactions in mind. If/when Linux gets support for paged DMA memory this will probably be using ATS/PRI.

Thanks Will,

What is the situation for platform devices (non-PCI) hanging off on-chip AXI interconnects for example, do you think the SMMUs stall mode would be useful to Linux to do paged DMA?
Or is the stall mode unlikely to be used at all with current versions of the SMMU?

Thanks again for your feedback!

Best regards,
Edgar

#4 lore raw
Quoted 31 lines · Edgar E. Iglesias
Quoted 22 lines · Will Deacon
Quoted 3 lines · Edgar E. Iglesias

I'm looking at adding models for the SMMU and it's stall feature to Xilinx ZynqMP QEMU models. The stall mode is quite problematic to model though. So I'm trying to figure out if I can get away without it :-)

Sure -- just make the context bank SCTLR.CFCFG fields RES0 and you'll be fine. The stall mode is optional in the architecture and not used by Linux.

Quoted 13 lines

I was trying to figure from the kernel sources if the SMMU Stall mode (for stalling and later resuming failed memory accesses) is used by the kernel.

AFAICT, the stall mode is enabled by the SMMU driver but the users of the report_iommu_fault() callbacks seem to all just do error reporting. Do I understand correctly in that Linux/KVM is currently not really depending on the stall function?

Are there any plans or ongoing work to implement paging features that would require the stall/resume feature for IOMMUs?
E.g swapping of DMA:able memory, CoW etc.

Quoted 7 lines
Quoted 2 lines

I noticed that AMD has something in their HSA/KFD framework around it but it seems quite AMD specific, please correct me if I'm wrong.

The main issue with the stall model is that it doesn't work nicely with PCI and is prone to deadlock on systems that haven't been designed with stalled transactions in mind. If/when Linux gets support for paged DMA memory this will probably be using ATS/PRI.

Quoted 5 lines

What is the situation for platform devices (non-PCI) hanging off on-chip AXI interconnects for example, do you think the SMMUs stall mode would be useful to Linux to do paged DMA?
Or is the stall mode unlikely to be used at all with current versions of the SMMU?

Well, I wouldn't rule anything out, but I don't know of anybody working on the stall model and it would need to be used on an opt-in basis given the potential for deadlocks.

Will