x86-64: Calgary IOMMU - introduce iommu_detected
PATCH 1/4 x86-64: Calgary IOMMU - introduce iommu_detected
swiotlb relies on the gart specific iommu_aperture variable to know if we discovered a hardware IOMMU before swiotlb initialization. Introduce iommu_detected to do the same thing, but in a HW IOMMU neutral manner, in preparation for adding the Calgary HW IOMMU.
Thanks,
Jon
arch/x86_64/kernel/aperture.c +1 -0
@@ -212,6 +212,7 @@if (read_pci_config (0 , num,3 ,0x00 ) !=NB_ID_3 )continue ; + iommu_detected =1 ; iommu_aperture =1 ; aper_order = (read_pci_config (0 , num,3 ,0x90 ) >>1 ) &7 ;
arch/x86_64/kernel/pci-dma.c +3 -0
@@ -32,6 +32,9 @@int panic_on_overflow__read_mostly =0 ;int force_iommu__read_mostly =0 ;#endif + +/* Set this to 1 if there is a HW IOMMU in the system */ +int iommu_detected__read_mostly =0 ;/* Dummy device used for NULL arguments (normally ISA). Better would be probably a smaller DMA mask, but this is bug-to-bug compatible
arch/x86_64/kernel/pci-gart.c +4 -0
@@ -624,6 +624,10 @@if (swiotlb)return -1 ; +/* Did we detect a different HW IOMMU? */ +if (iommu_detected && !iommu_aperture) +return -1 ; + if (no_iommu || (!force_iommu && end_pfn <= MAX_DMA32_PFN) || !iommu_aperture ||
arch/x86_64/kernel/pci-swiotlb.c +1 -1
@@ -31,7 +31,7 @@void pci_swiotlb_init (void ) {/* don't initialize swiotlb if iommu=off (no_iommu=1) */ -if (!iommu_aperture && !no_iommu && +if (!iommu_detected && !no_iommu && (end_pfn >MAX_DMA32_PFN || force_iommu)) swiotlb =1 ;if (swiotlb) {
include/asm-x86_64/proto.h +1 -0
@@ -116,6 +116,7 @@extern int acpi_ht;extern int acpi_disabled; +extern int iommu_detected;#ifdef CONFIG_GART_IOMMU extern int fallback_aper_order;extern int fallback_aper_force;
Quoted 4 lines · Jon Mason
swiotlb relies on the gart specific
iommu_aperturevariable to know if we discovered a hardware IOMMU before swiotlb initialization. Introduceiommu_detectedto do the same thing, but in a HW IOMMU neutral manner, in preparation for adding the Calgary HW IOMMU.
I applied them all.
But I think you broke the aperture setup. iommu_setup really needs to be called early, otherwise aperture.c doesn't get the right parameters. I undid that change.
And please next time send against the latest tree. It required quite some tweaking to apply.
-Andi
Quoted 7 lines · Andi Kleen
Quoted 4 lines · Jon Mason
swiotlb relies on the gart specific
iommu_aperturevariable to know if we discovered a hardware IOMMU before swiotlb initialization. Introduceiommu_detectedto do the same thing, but in a HW IOMMU neutral manner, in preparation for adding the Calgary HW IOMMU.I applied them all.
Fantastic!
Quoted 3 lines
But I think you broke the aperture setup.
iommu_setupreally needs to be called early, otherwiseaperture.cdoesn't get the right parameters. I undid that change.
I'll take a look at that, but I did boot test these patches on my opteron system and didn't notice anything wrong. Are the patches available for me to look at on firstfloor?
Quoted 2 lines
And please next time send against the latest tree. It required quite some tweaking to apply.
Sorry, I pulled a the latest mercurial tree this morning, but I guess that one is stale (or became stale over the day).
Quoted 1 line
-Andi
Quoted 10 lines · Jon Mason
Quoted 7 lines · Andi Kleen
Quoted 4 lines · Jon Mason
swiotlb relies on the gart specific
iommu_aperturevariable to know if we discovered a hardware IOMMU before swiotlb initialization. Introduceiommu_detectedto do the same thing, but in a HW IOMMU neutral manner, in preparation for adding the Calgary HW IOMMU.I applied them all.
Fantastic!
Seconded :-)
Quoted 7 lines
Quoted 3 lines
But I think you broke the aperture setup.
iommu_setupreally needs to be called early, otherwiseaperture.cdoesn't get the right parameters. I undid that change.I'll take a look at that, but I did boot test these patches on my opteron system and didn't notice anything wrong. Are the patches available for me to look at on firstfloor?
ftp://ftp.firstfloor.org/pub/ak/x86_64/quilt/patches/
Quoted 5 lines
Quoted 2 lines
And please next time send against the latest tree. It required quite some tweaking to apply.
Sorry, I pulled a the latest mercurial tree this morning, but I guess that one is stale (or became stale over the day).
I guess Andi means the firstfloor tree, which has a number of other IOMMU related changes that are not in mainline yet. Let's take it for a spin.
Cheers,
Muli
Quoted 11 lines · Andi Kleen
Quoted 4 lines · Jon Mason
swiotlb relies on the gart specific
iommu_aperturevariable to know if we discovered a hardware IOMMU before swiotlb initialization. Introduceiommu_detectedto do the same thing, but in a HW IOMMU neutral manner, in preparation for adding the Calgary HW IOMMU.I applied them all.
But I think you broke the aperture setup.
iommu_setupreally needs to be called early, otherwiseaperture.cdoesn't get the right parameters. I undid that change.
Actually, I didn't break aperture.c by moving iommu_setup because I moved iommu_hole_init after __setup. This change also enabled me to move back the calgary specific bootarg parsing (like you asked for in the original version of the path).
Also, in the reworked version of iommu-abstraction on ftp://ftp.firstfloor.org/pub/ak/x86_64/quilt/patches has a bug. When iommu_setup was kept in it's original form, the "+__setup("iommu=", iommu_setup);" wasn't removed, which gives 2 calls to the same function. I'll send updated versions of the patches here shortly which will apply cleanly inplace to that tree.
Quoted 3 lines
And please next time send against the latest tree. It required quite some tweaking to apply.
Sorry, I'll be happy to do it next time :)
Thanks,
Jon
Quoted 7 lines
Also, in the reworked version of iommu-abstraction on ftp://ftp.firstfloor.org/pub/ak/x86_64/quilt/patches has a bug. When
iommu_setupwas kept in it's original form, the "+__setup("iommu=",iommu_setup);" wasn't removed, which gives 2 calls to the same function. I'll send updated versions of the patches here shortly which will apply cleanly inplace to that tree.
That was intentional. All early boot options need a __setup too, otherwise they will appear as variables in init's environment.
It would be only a problem if the code couldn't run twice, but that is normally not the case.
-Andi