iommu/vt-d: Use the SIRTP when enabling remapping
RFC 1/1 iommu/vt-d: Use the SIRTP when enabling remapping
The previous change (iommu/vt-d: Don't store SIRTP request) to this area caused a crash in our simulator. In particular is seems that by the time a UART interrupt is sent through the system, we don't see interrupt remapping to be enabled. So the interrupt does not get translated to a logical interrupt and crashes.
OR'ing the SIRTP request to make sure it is seen but hopefully not sticky. This seems like a clean fix, at least on our simulator; if you don't agree, our simulator guy will take a closer look at our iommu model.
Found testing on our simulator, not real hardware.
--- drivers/iommu/intel_irq_remapping.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/iommu/intel_irq_remapping.c +1 -1
@@ -455,7 +455,7 @@ static void iommu_set_irq_remapping(struct intel_iommu *iommu, int mode)/* Enable interrupt-remapping */ iommu->gcmd |=DMA_GCMD_IRE ; iommu->gcmd &= ~DMA_GCMD_CFI ;/* Block compatibility-format MSIs */ -writel (iommu->gcmd, iommu->reg +DMAR_GCMD_REG ); +writel (iommu->gcmd |DMA_GCMD_SIRTP , iommu->reg +DMAR_GCMD_REG );IOMMU_WAIT_OP (iommu,DMAR_GSTS_REG , readl, (sts &DMA_GSTS_IRES ), sts);
Signature · 1 line
1.8.2.1
Quoted 9 lines · Nathan Zimmer
The previous change (iommu/vt-d: Don't store SIRTP request) to this area caused a crash in our simulator. In particular is seems that by the time a UART interrupt is sent through the system, we don't see interrupt remapping to be enabled. So the interrupt does not get translated to a logical interrupt and crashes.
OR'ing the SIRTP request to make sure it is seen but hopefully not sticky. This seems like a clean fix, at least on our simulator; if you don't agree, our simulator guy will take a closer look at our iommu model.
Check the VT-d spec (6.7, Set Interrupt Remapping Table Pointer Operation): "Software must always follow the interrupt-remapping table pointer set operation with a global invalidate of the IEC to ensure hardware references the new structures *before* enabling interrupt remapping." There is also (command register description): "If multiple control fields in this register need to be modified, software must serialize the modifications through multiple writes to this register." This, in combination with the command registers description of bits 24 and 25 strongly suggests that your model is broken.
Quoted 2 lines
Found testing on our simulator, not real hardware.
Funnily, the original issue was found by the QEMU model of VT-d that used to take the last cited sentence very strictly (I asked to remove it due to the preexisting Linux versions).
Jan
Signature · 2 lines
Siemens AG, Corporate Technology, CT RTC ITP SES-DE Corporate Competence Center Embedded Linux
Quoted 11 lines · Jan Kiszka
Quoted 9 lines · Nathan Zimmer
The previous change (iommu/vt-d: Don't store SIRTP request) to this area caused a crash in our simulator. In particular is seems that by the time a UART interrupt is sent through the system, we don't see interrupt remapping to be enabled. So the interrupt does not get translated to a logical interrupt and crashes.
OR'ing the SIRTP request to make sure it is seen but hopefully not sticky. This seems like a clean fix, at least on our simulator; if you don't agree, our simulator guy will take a closer look at our iommu model.
Quoted 9 lines
Check the VT-d spec (6.7, Set Interrupt Remapping Table Pointer Operation): "Software must always follow the interrupt-remapping table pointer set operation with a global invalidate of the IEC to ensure hardware references the new structures *before* enabling interrupt remapping." There is also (command register description): "If multiple control fields in this register need to be modified, software must serialize the modifications through multiple writes to this register." This, in combination with the command registers description of bits 24 and 25 strongly suggests that your model is broken.
Thanks, Jan. I'll read up on it. Still working out some kinks with the relatively new IOMMU/interrupt-remapping logic which we added over the last 18 months or so. Sorry for any distraction, and thanks for the help/info.
Gary
Quoted 10 lines
Quoted 3 lines
Found testing on our simulator, not real hardware.
Funnily, the original issue was found by the QEMU model of VT-d that used to take the last cited sentence very strictly (I asked to remove it due to the preexisting Linux versions).
Jan
I'm not really a kernel developer and don't know the proper etiquette for this, but just wanted to say thanks to Jan for the guidance. With your help, I was able to track our problem down to a much more fundamental problem of incorrect bit position of the fields in our VT-d GLBCMD register, attributable to our now-manual process of generating register information from Intel socket documentation. (We used to used Intel's SoftSDV to determine register fields, but that's no longer available and we have to use fallible human eyes! Thousands of registers in each new socket...)
Gary
Quoted 38 lines · Gary Kroening
Quoted 11 lines · Jan Kiszka
Quoted 9 lines · Nathan Zimmer
The previous change (iommu/vt-d: Don't store SIRTP request) to this area caused a crash in our simulator. In particular is seems that by the time a UART interrupt is sent through the system, we don't see interrupt remapping to be enabled. So the interrupt does not get translated to a logical interrupt and crashes.
OR'ing the SIRTP request to make sure it is seen but hopefully not sticky. This seems like a clean fix, at least on our simulator; if you don't agree, our simulator guy will take a closer look at our iommu model.
Quoted 9 lines
Check the VT-d spec (6.7, Set Interrupt Remapping Table Pointer Operation): "Software must always follow the interrupt-remapping table pointer set operation with a global invalidate of the IEC to ensure hardware references the new structures *before* enabling interrupt remapping." There is also (command register description): "If multiple control fields in this register need to be modified, software must serialize the modifications through multiple writes to this register." This, in combination with the command registers description of bits 24 and 25 strongly suggests that your model is broken.
Thanks, Jan. I'll read up on it. Still working out some kinks with the relatively new IOMMU/interrupt-remapping logic which we added over the last 18 months or so. Sorry for any distraction, and thanks for the help/info.
GaryQuoted 10 lines
Quoted 3 lines
Found testing on our simulator, not real hardware.
Funnily, the original issue was found by the QEMU model of VT-d that used to take the last cited sentence very strictly (I asked to remove it due to the preexisting Linux versions).
Jan