Questions about IOMMU & PCIe switch
Hello.
We have two kinds of PCIe cards:
A1 - based on chip B
A2 - based on same chip B but behind PCIe switch
Card A1 work flawlessly in any configuration, but card A2 work flawlessly only if system lack IOMMU or have disabled IOMMU
Questions:
Do we have to treat card with PCIe switch in special way? How?
Where (PCI bridge driver? card's driver?)?
Any examples in kernel?
Thank you.
Raimonds Cicans
Quoted 7 lines · Raimonds Cicans
We have two kinds of PCIe cards:
A1 - based on chip B
A2 - based on same chip B but behind PCIe switchCard A1 work flawlessly in any configuration, but card A2 work flawlessly only if system lack IOMMU or have disabled IOMMU
In theory, such a switch should be transparent.
Quoted 1 line
Do we have to treat card with PCIe switch in special way?
No. But the switch has to work correctly, and driver must be written correctly.
Which switch? Which driver?
Regards,
Clemens
Quoted 8 lines · Clemens Ladisch
Quoted 7 lines · Raimonds Cicans
We have two kinds of PCIe cards:
A1 - based on chip B
A2 - based on same chip B but behind PCIe switchCard A1 work flawlessly in any configuration, but card A2 work flawlessly only if system lack IOMMU or have disabled IOMMU
Quoted 7 lines
In theory, such a switch should be transparent.
Quoted 1 line
Do we have to treat card with PCIe switch in special way?
No. But the switch has to work correctly, and driver must be written correctly.
Which switch? Which driver?
We talk about TBS 6285 Quad DVB-T PCIe receiver card. Starting from version V20 manufacturer added Pericom PCIe switch PI7C9X2G304SL.
Card have 2 out of kernel drivers:
1) Semi open from TBS:
http://www.tbsdtv.com/download/document/common/tbs-linux-drivers_v141225.zip
2) Open from: https://github.com/ljalves/linux_media
If you want more information you can look at open ticket:
https://github.com/ljalves/linux_media/issues/66
Raimonds Cicans
Quoted 1 line · Raimonds Cicans
If the TBS driver works, why don't you use it?
The WARNING from the kernel log indicates a hardware bug in the PCIe bridge. Do you have the same card, and do you also get this warning with kernel 3.16?
I/O virtualization requires handling packets from different devices differently. Problems with DMAR might indicate that some PCIe device does not correctly identify itself as the source of PCIe packets.
Regards,
Clemens
Quoted 3 lines · Clemens Ladisch
Quoted 1 line · Raimonds Cicans
If the TBS driver works, why don't you use it?
1) driver is not stable in 24x7 setups
2) driver use old DVBAPI. This cause problems with some user space programs.
3) TBS recommends to use card in MSI interrupt mode but this mode on IOMMU systems do not work:
card is able to find transponders and tune to it but can not receive any data
4) I use GRSecurity patched kernels but external media drivers are incompatible with such kernels.
I am forced to use driver source code injection in kernel from project https://github.com/bas-t/saa716x-intree
which injects few drivers from https://github.com/ljalves/linux_media
in kernel source tree.
5) I prefer open source drivers, but TBS drivers are half way open - open back-end drivers and closed front-end drivers.
Quoted 4 lines
The WARNING from the kernel log indicates a hardware bug in the PCIe bridge. Do you have the same card, and do you also get this warning with kernel 3.16?
1) I have same card with same revision number
2) affected computer is Intel based but mine is AMD based
3) I can not test kernel 3.16 because I had IOMMU related regression in kernels 3.14-3.17 (AMD-Vi: Completion-Wait loop timed out) On other kernels I do not have such warning, but starting from kernel 3.17.7 I have IOMMU related messages with my other card (TBS 6981):
AMD-Vi: Event logged [IO_PAGE_FAULT device=08:00.0 domain=0x001c address=0x0000000001355000 flags=0x0000]
and
WARNING: CPU: 0 PID: 13204 at drivers/iommu/amd_iommu.c:2625 dma_ops_domain_unmap.part.9+0x4d/0x56()
Because TBS 6981 driver is included in kernel I contacted yesterday linux-media mailing list, but not received answer yet.
Raimonds Cicans
Quoted 4 lines · Raimonds Cicans
Quoted 3 lines · Clemens Ladisch
Quoted 1 line · Raimonds Cicans
Quoted 8 lines
Quoted 3 lines
Quoted 1 line
If the TBS driver works, why don't you use it?
1) driver is not stable in 24x7 setups
2) driver use old DVBAPI. This cause problems with some user space programs.
It shouldn't. It is backward compatible.
Quoted 5 lines
3) TBS recommends to use card in MSI interrupt mode but this mode on IOMMU systems do not work:
card is able to find transponders and tune to it but can not receive any data
TBS didn't even have the courtesy to say thanks, but still. They don't have any clue on the driver.
You shouldn't use MSI mode on most systems unless you are sure that your machine works perfectly well with MSI mode.
You should simply use standard interrupt mode, if you are unsure.
In any sense, that card looks a bit odd:
1. It doesn't need a PCIe switch, nor that CPLD in there for the fact that it is using a SAA7160, rather than an E version, which thereby support 4 FGPI inputs (4 simultaneous TS's) (Redundant components)
2. But in any sense, the PCIe switch is transparent, It doesn't need any programming. All what you can do with the PCIe switch is to reprogram the lines to it, nothing more. In any case, if your card is detected, there's no point in playing with the PCIe switch
3. The TS failure in a while can most likely be attributed to bad CPLD programming. On the cards that I have worked with, they've worked pretty well.
Regards,
Manu