Skip to contentSkip to conversation

Questions about IOMMU & PCIe switch

DISCUSSION Raimonds Cicans 3 participants 6 messages Jan 7, 2015 – Jan 8, 2015

lore
Share & tools
lore
#0 lore raw

Hello.

We have two kinds of PCIe cards:
A1 - based on chip B
A2 - based on same chip B but behind PCIe switch

Card A1 work flawlessly in any configuration, but card A2 work flawlessly only if system lack IOMMU or have disabled IOMMU

Questions:
Do we have to treat card with PCIe switch in special way? How?
Where (PCI bridge driver? card's driver?)?
Any examples in kernel?

Thank you.

Raimonds Cicans

#1 lore raw
Quoted 7 lines · Raimonds Cicans

We have two kinds of PCIe cards:
A1 - based on chip B
A2 - based on same chip B but behind PCIe switch

Card A1 work flawlessly in any configuration, but card A2 work flawlessly only if system lack IOMMU or have disabled IOMMU

In theory, such a switch should be transparent.

Quoted 1 line

Do we have to treat card with PCIe switch in special way?

No. But the switch has to work correctly, and driver must be written correctly.

Which switch? Which driver?

Regards,
Clemens

#2 lore raw
Quoted 8 lines · Clemens Ladisch
Quoted 7 lines · Raimonds Cicans

We have two kinds of PCIe cards:
A1 - based on chip B
A2 - based on same chip B but behind PCIe switch

Card A1 work flawlessly in any configuration, but card A2 work flawlessly only if system lack IOMMU or have disabled IOMMU

Quoted 7 lines

In theory, such a switch should be transparent.

Quoted 1 line

Do we have to treat card with PCIe switch in special way?

No. But the switch has to work correctly, and driver must be written correctly.

Which switch? Which driver?

We talk about TBS 6285 Quad DVB-T PCIe receiver card. Starting from version V20 manufacturer added Pericom PCIe switch PI7C9X2G304SL.

Card have 2 out of kernel drivers:
1) Semi open from TBS:
http://www.tbsdtv.com/download/document/common/tbs-linux-drivers_v141225.zip

2) Open from: https://github.com/ljalves/linux_media

If you want more information you can look at open ticket:
https://github.com/ljalves/linux_media/issues/66

Raimonds Cicans

#3 lore raw
Quoted 1 line · Raimonds Cicans

https://github.com/ljalves/linux_media/issues/66

If the TBS driver works, why don't you use it?

The WARNING from the kernel log indicates a hardware bug in the PCIe bridge. Do you have the same card, and do you also get this warning with kernel 3.16?

I/O virtualization requires handling packets from different devices differently. Problems with DMAR might indicate that some PCIe device does not correctly identify itself as the source of PCIe packets.

Regards,
Clemens

#4 lore raw
Quoted 3 lines · Clemens Ladisch
Quoted 1 line · Raimonds Cicans

https://github.com/ljalves/linux_media/issues/66

If the TBS driver works, why don't you use it?

1) driver is not stable in 24x7 setups

2) driver use old DVBAPI. This cause problems with some user space programs.

3) TBS recommends to use card in MSI interrupt mode but this mode on IOMMU systems do not work:
card is able to find transponders and tune to it but can not receive any data

4) I use GRSecurity patched kernels but external media drivers are incompatible with such kernels.
I am forced to use driver source code injection in kernel from project https://github.com/bas-t/saa716x-intree
which injects few drivers from https://github.com/ljalves/linux_media
in kernel source tree.

5) I prefer open source drivers, but TBS drivers are half way open - open back-end drivers and closed front-end drivers.

Quoted 4 lines

The WARNING from the kernel log indicates a hardware bug in the PCIe bridge. Do you have the same card, and do you also get this warning with kernel 3.16?

1) I have same card with same revision number

2) affected computer is Intel based but mine is AMD based

3) I can not test kernel 3.16 because I had IOMMU related regression in kernels 3.14-3.17 (AMD-Vi: Completion-Wait loop timed out) On other kernels I do not have such warning, but starting from kernel 3.17.7 I have IOMMU related messages with my other card (TBS 6981):

AMD-Vi: Event logged [IO_PAGE_FAULT device=08:00.0 domain=0x001c address=0x0000000001355000 flags=0x0000]

and

WARNING: CPU: 0 PID: 13204 at drivers/iommu/amd_iommu.c:2625 dma_ops_domain_unmap.part.9+0x4d/0x56()

Because TBS 6981 driver is included in kernel I contacted yesterday linux-media mailing list, but not received answer yet.

Raimonds Cicans

#5 lore raw
Quoted 4 lines · Raimonds Cicans
Quoted 3 lines · Clemens Ladisch
Quoted 1 line · Raimonds Cicans
Quoted 8 lines
Quoted 3 lines
Quoted 1 line

https://github.com/ljalves/linux_media/issues/66

If the TBS driver works, why don't you use it?

1) driver is not stable in 24x7 setups

2) driver use old DVBAPI. This cause problems with some user space programs.

It shouldn't. It is backward compatible.

Quoted 5 lines

3) TBS recommends to use card in MSI interrupt mode but this mode on IOMMU systems do not work:
card is able to find transponders and tune to it but can not receive any data

TBS didn't even have the courtesy to say thanks, but still. They don't have any clue on the driver.

You shouldn't use MSI mode on most systems unless you are sure that your machine works perfectly well with MSI mode.

You should simply use standard interrupt mode, if you are unsure.

In any sense, that card looks a bit odd:

1. It doesn't need a PCIe switch, nor that CPLD in there for the fact that it is using a SAA7160, rather than an E version, which thereby support 4 FGPI inputs (4 simultaneous TS's) (Redundant components)

2. But in any sense, the PCIe switch is transparent, It doesn't need any programming. All what you can do with the PCIe switch is to reprogram the lines to it, nothing more. In any case, if your card is detected, there's no point in playing with the PCIe switch

3. The TS failure in a while can most likely be attributed to bad CPLD programming. On the cards that I have worked with, they've worked pretty well.

Regards,

Manu