Skip to contentSkip to conversation

IOMMU groups: better with Intel root ports, or with PEX8606 switch?

DISCUSSION Dana Goyette 2 participants 16 messages Dec 29, 2013 – Jan 5, 2014

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

I have purchased both a SuperMicro X10SAE and an X10SAT, and I need to soon decide which one to keep.

The SuperMicro X10SAT has all the PCIe x1 slots hidden behind a PLX PEX8066 switch, which claims to support ACS. I'd expect the devices downstream of the PLX switch to be in separate groups.

With Linux 3.13-rc5 and "enable overrides for missing ACS capabilities" applied and set for the Intel root ports, the devices behind the switch remain stuck in the same group.

In terms of passing devices to different VMs, which is better: all devices on different root ports, or all devices behind the one ACS-supporting switch?

X10SAT IOMMU groups:

### Group 0 ###
00:00.0 Host bridge: Intel Corporation Xeon E3-1200 v3 Processor
DRAM Controller (rev 06)

### Group 1 ###

    00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v3/4th Gen Core 

Processor PCI Express x16 Controller (rev 06)
01:00.0 VGA compatible controller: Advanced Micro Devices, Inc.
[AMD/ATI] Juniper PRO [Radeon HD 5750]
01:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI]
Juniper HDMI Audio [Radeon HD 5700 Series]

### Group 2 ###
00:02.0 VGA compatible controller: Intel Corporation Xeon E3-1200
v3 Processor Integrated Graphics Controller (rev 06)

### Group 3 ###

    00:03.0 Audio device: Intel Corporation Xeon E3-1200 v3/4th Gen 

Core Processor HD Audio Controller (rev 06)

### Group 4 ###
00:14.0 USB controller: Intel Corporation 8 Series/C220 Series
Chipset Family USB xHCI (rev 05)

### Group 5 ###
00:16.0 Communication controller: Intel Corporation 8 Series/C220
Series Chipset Family MEI Controller #1 (rev 04)
00:16.3 Serial controller: Intel Corporation 8 Series/C220 Series
Chipset Family KT Controller (rev 04)

### Group 6 ###
00:19.0 Ethernet controller: Intel Corporation Ethernet Connection
I217-LM (rev 05)

### Group 7 ###
00:1a.0 USB controller: Intel Corporation 8 Series/C220 Series
Chipset Family USB EHCI #2 (rev 05)
### Group 8 ###
00:1b.0 Audio device: Intel Corporation 8 Series/C220 Series
Chipset High Definition Audio Controller (rev 05)

### Group 9 ###
00:1c.0 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset
Family PCI Express Root Port #1 (rev d5)

### Group 10 ###
00:1c.1 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset
Family PCI Express Root Port #2 (rev d5)

    03:00.0 PCI bridge: PLX Technology, Inc. PEX 8606 6 Lane, 6 Port 
PCI Express Gen 2 (5.0 GT/s) Switch (rev ba)
    04:01.0 PCI bridge: PLX Technology, Inc. PEX 8606 6 Lane, 6 Port 
PCI Express Gen 2 (5.0 GT/s) Switch (rev ba)
    04:04.0 PCI bridge: PLX Technology, Inc. PEX 8606 6 Lane, 6 Port 
PCI Express Gen 2 (5.0 GT/s) Switch (rev ba)
    04:05.0 PCI bridge: PLX Technology, Inc. PEX 8606 6 Lane, 6 Port 
PCI Express Gen 2 (5.0 GT/s) Switch (rev ba)
    04:07.0 PCI bridge: PLX Technology, Inc. PEX 8606 6 Lane, 6 Port 
PCI Express Gen 2 (5.0 GT/s) Switch (rev ba)
    04:09.0 PCI bridge: PLX Technology, Inc. PEX 8606 6 Lane, 6 Port 
PCI Express Gen 2 (5.0 GT/s) Switch (rev ba)
    08:00.0 USB controller: Renesas Technology Corp. uPD720201 USB 3.0 
Host Controller (rev 03)
    09:00.0 PCI bridge: Texas Instruments XIO2213A/B/XIO2221 PCI 
Express to PCI Bridge [Cheetah Express] (rev 01)
    0a:00.0 FireWire (IEEE 1394): Texas Instruments XIO2213A/B/XIO2221 

IEEE-1394b OHCI Controller [Cheetah Express] (rev 01)

### Group 11 ###
00:1c.3 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset
Family PCI Express Root Port #4 (rev d5)
### Group 12 ###
00:1c.4 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset
Family PCI Express Root Port #5 (rev d5)

### Group 13 ###
00:1d.0 USB controller: Intel Corporation 8 Series/C220 Series
Chipset Family USB EHCI #1 (rev 05)

### Group 14 ###
00:1f.0 ISA bridge: Intel Corporation C226 Series Chipset Family
Server Advanced SKU LPC Controller (rev 05)
00:1f.2 SATA controller: Intel Corporation 8 Series/C220 Series
Chipset Family 6-port SATA Controller 1 [AHCI mode] (rev 05)
00:1f.3 SMBus: Intel Corporation 8 Series/C220 Series Chipset
Family SMBus Controller (rev 05)
00:1f.6 Signal processing controller: Intel Corporation 8 Series
Chipset Family Thermal Management Controller (rev 05)

### Group 15 ###
02:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA
Controller (rev 01)

### Group 16 ###
0b:00.0 Ethernet controller: Intel Corporation I210 Gigabit Network
Connection (rev 03)

#1 lore raw
Quoted 7 lines · Dana Goyette

I have purchased both a SuperMicro X10SAE and an X10SAT, and I need to soon decide which one to keep.

The SuperMicro X10SAT has all the PCIe x1 slots hidden behind a PLX PEX8066 switch, which claims to support ACS. I'd expect the devices downstream of the PLX switch to be in separate groups.

Quoted 7 lines

With Linux 3.13-rc5 and "enable overrides for missing ACS capabilities" applied and set for the Intel root ports, the devices behind the switch remain stuck in the same group.

In terms of passing devices to different VMs, which is better: all devices on different root ports, or all devices behind the one ACS-supporting switch?

Can you provide lspci -vvv info? If you're getting that for groups either the switch has ACS capabilities, but doesn't support the features we need or we're doing something wrong. Thanks,

Alex

#2 lore raw
Quoted 20 lines · Alex Williamson
Quoted 7 lines · Dana Goyette

I have purchased both a SuperMicro X10SAE and an X10SAT, and I need to soon decide which one to keep.

The SuperMicro X10SAT has all the PCIe x1 slots hidden behind a PLX PEX8066 switch, which claims to support ACS. I'd expect the devices downstream of the PLX switch to be in separate groups.

Quoted 7 lines

With Linux 3.13-rc5 and "enable overrides for missing ACS capabilities" applied and set for the Intel root ports, the devices behind the switch remain stuck in the same group.

In terms of passing devices to different VMs, which is better: all devices on different root ports, or all devices behind the one ACS-supporting switch?

Can you provide lspci -vvv info? If you're getting that for groups either the switch has ACS capabilities, but doesn't support the features we need or we're doing something wrong. Thanks,

Quoted 7 lines

Alex

--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

I initially tried attaching the output as a .txt file, but it's too large. Anyway, here's the output of lspci -nnvvv (you may notice that I moved the Radeon to a different slot).

00:00.0 Host bridge [0600]: Intel Corporation Xeon E3-1200 v3 Processor DRAM Controller [8086:0c08] (rev 06)
Subsystem: Super Micro Computer Inc Device [15d9:0818]
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr+ Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort+ >SERR- <PERR- INTx-
Latency: 0
Capabilities: [e0] Vendor Specific Information: Len=0c <?>

00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x16 Controller [8086:0c01] (rev 06)

(prog-if 00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr+ Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-

        Latency: 0, Cache Line Size: 64 bytes
        Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
        I/O behind bridge: 0000f000-00000fff
        Memory behind bridge: fff00000-000fffff
        Prefetchable memory behind bridge: 

00000000fff00000-00000000000fffff
Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- <SERR- <PERR-
BridgeCtl: Parity+ SERR+ NoISA+ VGA- MAbort- >Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: [88] Subsystem: Intel Corporation Device [8086:2010]
Capabilities: [80] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA
PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [90] MSI: Enable+ Count=1/1 Maskable- 64bit-
Address: fee002b8 Data: 0000
Capabilities: [a0] Express (v2) Root Port (Slot+), MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s
<64ns, L1 <1us
ExtTag- RBE+ FLReset-
DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+
Unsupported+
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 256 bytes, MaxReadReq 128 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq-
AuxPwr- TransPend-
LnkCap: Port #2, Speed 8GT/s, Width x8, ASPM L0s L1,
Latency L0 <1us, L1 <8us
ClockPM- Surprise- LLActRep- BwNot+
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain-
CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x0, TrErr- Train- SlotClk+
DLActive- BWMgmt- ABWMgmt-
SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd-
HotPlug- Surprise-
Slot #1, PowerLimit 75.000W; Interlock- NoCompl+
SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt-
HPIrq- LinkChg-
Control: AttnInd Unknown, PwrInd Unknown,
Power- Interlock-
SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt-
PresDet- Interlock-
Changed: MRL- PresDet- LinkState-
RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal+
PMEIntEna- CRSVisible-
RootCap: CRSVisible-
RootSta: PME ReqID 0000, PMEStatus- PMEPending-
DevCap2: Completion Timeout: Not Supported, TimeoutDis-
ARIFwd-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-
ARIFwd-
LnkCtl2: Target Link Speed: 8GT/s, EnterCompliance-
SpeedDis-, Selectable De-emphasis: -3.5dB
Transmit Margin: Normal Operating Range,
EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB,
EqualizationComplete-, EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-,
LinkEqualizationRequest-
Capabilities: [100 v1] Virtual Channel
Caps: LPEVC=0 RefClk=100ns PATEntryBits=1
Arb: Fixed- WRR32- WRR64- WRR128-
Ctrl: ArbSelect=Fixed
Status: InProgress-
VC0: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
Arb: Fixed+ WRR32- WRR64- WRR128- TWRR128-
WRR256-
Ctrl: Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
Status: NegoPending- InProgress-
Capabilities: [140 v1] Root Complex Link
Desc: PortNumber=02 ComponentID=01 EltType=Config
Link0: Desc: TargetPort=00 TargetComponent=01
AssocRCRB- LinkType=MemMapped LinkValid+

                        Addr:   00000000fed19000
        Capabilities: [d94 v1] #19
        Kernel driver in use: pcieport

00:01.1 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x8 Controller [8086:0c05] (rev 06) (prog-

if 00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr+ Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Bus: primary=00, secondary=02, subordinate=02, sec-latency=0 I/O behind bridge: 0000e000-0000efff
Memory behind bridge: eee00000-eeefffff
Prefetchable memory behind bridge:
00000000c0000000-00000000cfffffff
Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- <SERR- <PERR-
BridgeCtl: Parity+ SERR+ NoISA- VGA+ MAbort- >Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: [88] Subsystem: Intel Corporation Device [8086:2010]
Capabilities: [80] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA
PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [90] MSI: Enable+ Count=1/1 Maskable- 64bit-
Address: fee002d8 Data: 0000
Capabilities: [a0] Express (v2) Root Port (Slot+), MSI 00
DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s
<64ns, L1 <1us
ExtTag- RBE+ FLReset-
DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+
Unsupported+
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq-
AuxPwr- TransPend-
LnkCap: Port #3, Speed 8GT/s, Width x8, ASPM L0s L1,
Latency L0 <256ns, L1 <8us
ClockPM- Surprise- LLActRep- BwNot+
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain-
CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 5GT/s, Width x8, TrErr- Train- SlotClk+
DLActive- BWMgmt+ ABWMgmt+
SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd-
HotPlug- Surprise-
Slot #2, PowerLimit 75.000W; Interlock- NoCompl+
SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt-
HPIrq- LinkChg-
Control: AttnInd Unknown, PwrInd Unknown,
Power- Interlock-
SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt-
PresDet+ Interlock-
Changed: MRL- PresDet- LinkState-
RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal+
PMEIntEna- CRSVisible-
RootCap: CRSVisible-
RootSta: PME ReqID 0000, PMEStatus- PMEPending-
DevCap2: Completion Timeout: Not Supported, TimeoutDis-
ARIFwd-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-
ARIFwd-
LnkCtl2: Target Link Speed: 8GT/s, EnterCompliance-
SpeedDis-, Selectable De-emphasis: -3.5dB
Transmit Margin: Normal Operating Range,
EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -3.5dB,
EqualizationComplete-, EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-,
LinkEqualizationRequest-
Capabilities: [100 v1] Virtual Channel
Caps: LPEVC=0 RefClk=100ns PATEntryBits=1
Arb: Fixed- WRR32- WRR64- WRR128-
Ctrl: ArbSelect=Fixed
Status: InProgress-
VC0: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
Arb: Fixed+ WRR32- WRR64- WRR128- TWRR128-
WRR256-
Ctrl: Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
Status: NegoPending- InProgress-
Capabilities: [140 v1] Root Complex Link
Desc: PortNumber=03 ComponentID=01 EltType=Config
Link0: Desc: TargetPort=00 TargetComponent=01
AssocRCRB- LinkType=MemMapped LinkValid+

                        Addr:   00000000fed19000
        Capabilities: [d94 v1] #19
        Kernel driver in use: pcieport

00:02.0 Display controller [0380]: Intel Corporation Xeon E3-1200 v3 Processor Integrated Graphics Controller [8086:041a] (rev 06)
Subsystem: Intel Corporation Device [8086:2010]
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr- Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-

        Latency: 0
        Interrupt: pin A routed to IRQ 69
        Region 0: Memory at ee400000 (64-bit, non-prefetchable) [size=4M]
        Region 2: Memory at 80000000 (64-bit, prefetchable) [size=256M]
        Region 4: I/O ports at f000 [size=64]
        Capabilities: [90] MSI: Enable+ Count=1/1 Maskable- 64bit-
                Address: fee00018  Data: 0000
        Capabilities: [d0] Power Management version 2
                Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA 

PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [a4] PCI Advanced Features
AFCap: TP+ FLR+
AFCtrl: FLR-
AFStatus: TP-
Kernel driver in use: i915

00:03.0 Audio device [0403]: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor HD Audio Controller [8086:0c0c] (rev 06)
Subsystem: Intel Corporation Device [8086:2010]
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr+ Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 70
Region 0: Memory at eef34000 (64-bit, non-prefetchable) [size=16K]
Capabilities: [50] Power Management version 2
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA
PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [60] MSI: Enable+ Count=1/1 Maskable- 64bit-
Address: fee005f8 Data: 0000
Capabilities: [70] Express (v1) Root Complex Integrated
Endpoint, MSI 00
DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s
unlimited, L1 unlimited
ExtTag- RBE- FLReset+
DevCtl: Report errors: Correctable- Non-Fatal- Fatal-
Unsupported-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
MaxPayload 128 bytes, MaxReadReq 128 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq-
AuxPwr- TransPend-
LnkCap: Port #0, Speed unknown, Width x0, ASPM unknown,
Latency L0 <64ns, L1 <1us
ClockPM- Surprise- LLActRep- BwNot-
LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed unknown, Width x0, TrErr- Train- SlotClk-
DLActive- BWMgmt- ABWMgmt-
Kernel driver in use: snd_hda_intel

00:14.0 USB controller [0c03]: Intel Corporation 8 Series/C220 Series Chipset Family USB xHCI [8086:8c31] (rev 05) (prog-if 30 [XHCI])
Subsystem: Super Micro Computer Inc Device [15d9:0818]
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr+ Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium

Quoted 1 line

TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-

Latency: 0
Interrupt: pin A routed to IRQ 50
Region 0: Memory at eef20000 (64-bit, non-prefetchable) [size=64K]
Capabilities: [70] Power Management version 2
Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA
PME(D0-,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [80] MSI: Enable+ Count=1/8 Maskable- 64bit+
Address: 00000000fee003f8 Data: 0000
Kernel driver in use: xhci_hcd

00:16.0 Communication controller [0780]: Intel Corporation 8 Series/C220 Series Chipset Family MEI Controller #1 [8086:8c3a] (rev 04)
Subsystem: Super Micro Computer Inc Device [15d9:0818]
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0
Interrupt: pin A routed to IRQ 11
Region 0: Memory at eef40000 (64-bit, non-prefetchable) [size=16]
Capabilities: [50] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA
PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [8c] MSI: Enable- Count=1/1 Maskable- 64bit+
Address: 0000000000000000 Data: 0000

00:16.3 Serial controller [0700]: Intel Corporation 8 Series/C220 Series Chipset Family KT Controller [8086:8c3d] (rev 04) (prog-if 02

[16550])
Subsystem: Super Micro Computer Inc Device [15d9:0818]
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0
Interrupt: pin B routed to IRQ 19
Region 0: I/O ports at f0c0 [size=8]
Region 1: Memory at eef3e000 (32-bit, non-prefetchable) [size=4K]
Capabilities: [c8] Power Management version 3
Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA
PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [d0] MSI: Enable- Count=1/1 Maskable- 64bit+
Address: 0000000000000000 Data: 0000
Kernel driver in use: serial

00:19.0 Ethernet controller [0200]: Intel Corporation Ethernet Connection I217-LM [8086:153a] (rev 05)
Subsystem: Super Micro Computer Inc Device [15d9:153a]
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr+ Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0
Interrupt: pin A routed to IRQ 61
Region 0: Memory at eef00000 (32-bit, non-prefetchable) [size=128K]
Region 1: Memory at eef3d000 (32-bit, non-prefetchable) [size=4K]
Region 2: I/O ports at f060 [size=32]
Capabilities: [c8] Power Management version 2
Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA
PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=1 PME-
Capabilities: [d0] MSI: Enable+ Count=1/1 Maskable- 64bit+
Address: 00000000fee00578 Data: 0000
Capabilities: [e0] PCI Advanced Features
AFCap: TP+ FLR+
AFCtrl: FLR-
AFStatus: TP-
Kernel driver in use: e1000e

00:1b.0 Audio device [0403]: Intel Corporation 8 Series/C220 Series Chipset High Definition Audio Controller [8086:8c20] (rev 05)
Subsystem: Super Micro Computer Inc Device [15d9:0818]
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr+ Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 67
Region 0: Memory at eef30000 (64-bit, non-prefetchable) [size=16K]
Capabilities: [50] Power Management version 2
Flags: PMEClk- DSI- D1- D2- AuxCurrent=55mA
PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [60] MSI: Enable+ Count=1/1 Maskable- 64bit+
Address: 00000000fee005b8 Data: 0000
Capabilities: [70] Express (v1) Root Complex Integrated
Endpoint, MSI 00
DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s
<64ns, L1 <1us
ExtTag- RBE- FLReset+
DevCtl: Report errors: Correctable- Non-Fatal- Fatal-
Unsupported-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
MaxPayload 128 bytes, MaxReadReq 128 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq-
AuxPwr+ TransPend-
LnkCap: Port #0, Speed unknown, Width x0, ASPM unknown,
Latency L0 <64ns, L1 <1us
ClockPM- Surprise- LLActRep- BwNot-
LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed unknown, Width x0, TrErr- Train- SlotClk-
DLActive- BWMgmt- ABWMgmt-
Capabilities: [100 v1] Virtual Channel
Caps: LPEVC=0 RefClk=100ns PATEntryBits=1
Arb: Fixed- WRR32- WRR64- WRR128-
Ctrl: ArbSelect=Fixed
Status: InProgress-
VC0: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
Arb: Fixed- WRR32- WRR64- WRR128- TWRR128-
WRR256-
Ctrl: Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
Status: NegoPending- InProgress-
VC1: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
Arb: Fixed- WRR32- WRR64- WRR128- TWRR128-
WRR256-
Ctrl: Enable+ ID=2 ArbSelect=Fixed TC/VC=00
Status: NegoPending- InProgress-
Kernel driver in use: snd_hda_intel

00:1c.0 PCI bridge [0604]: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #1 [8086:8c10] (rev d5) (prog-if 00

[Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr+ Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Bus: primary=00, secondary=03, subordinate=03, sec-latency=0 I/O behind bridge: 0000d000-0000dfff
Memory behind bridge: eed00000-eedfffff
Prefetchable memory behind bridge:
00000000fff00000-00000000000fffff
Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- <SERR- <PERR-
BridgeCtl: Parity+ SERR+ NoISA- VGA- MAbort- >Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: [40] Express (v2) Root Port (Slot+), MSI 00
DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s
<64ns, L1 <1us
ExtTag- RBE+ FLReset-
DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+
Unsupported-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq-
AuxPwr+ TransPend-
LnkCap: Port #1, Speed 5GT/s, Width x1, ASPM L0s L1,
Latency L0 <512ns, L1 <16us
ClockPM- Surprise- LLActRep+ BwNot+
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain-
CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 5GT/s, Width x1, TrErr- Train- SlotClk+
DLActive+ BWMgmt+ ABWMgmt-
SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd-
HotPlug- Surprise-
Slot #0, PowerLimit 10.000W; Interlock- NoCompl+
SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt-
HPIrq- LinkChg-
Control: AttnInd Unknown, PwrInd Unknown,
Power- Interlock-
SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt-
PresDet+ Interlock-
Changed: MRL- PresDet- LinkState-
RootCtl: ErrCorrectable+ ErrNon-Fatal+ ErrFatal+
PMEIntEna- CRSVisible-
RootCap: CRSVisible-
RootSta: PME ReqID 0000, PMEStatus- PMEPending-
DevCap2: Completion Timeout: Range ABC, TimeoutDis+ ARIFwd-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-
ARIFwd-
LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance-
SpeedDis-, Selectable De-emphasis: -6dB
Transmit Margin: Normal Operating Range,
EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB,
EqualizationComplete-, EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-,
LinkEqualizationRequest-

        Capabilities: [80] MSI: Enable- Count=1/1 Maskable- 64bit-
                Address: 00000000  Data: 0000
        Capabilities: [90] Subsystem: Super Micro Computer Inc Device 

[15d9:0818]
Capabilities: [a0] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA
PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Kernel driver in use: pcieport

00:1c.1 PCI bridge [0604]: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #2 [8086:8c12] (rev d5) (prog-if 00

[Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr+ Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Bus: primary=00, secondary=04, subordinate=0b, sec-latency=0 I/O behind bridge: 0000f000-00000fff
Memory behind bridge: ee800000-eebfffff
Prefetchable memory behind bridge:
00000000fff00000-00000000000fffff
Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort+ <SERR- <PERR-
BridgeCtl: Parity+ SERR+ NoISA- VGA- MAbort- >Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: [40] Express (v2) Root Port (Slot+), MSI 00
DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s
<64ns, L1 <1us
ExtTag- RBE+ FLReset-
DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+
Unsupported-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq-
AuxPwr+ TransPend-
LnkCap: Port #2, Speed 5GT/s, Width x1, ASPM L0s L1,
Latency L0 <1us, L1 <16us
ClockPM- Surprise- LLActRep+ BwNot+
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain-
CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 5GT/s, Width x1, TrErr- Train- SlotClk+
DLActive+ BWMgmt+ ABWMgmt+
SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd-
HotPlug- Surprise-
Slot #1, PowerLimit 10.000W; Interlock- NoCompl+
SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt-
HPIrq- LinkChg-
Control: AttnInd Unknown, PwrInd Unknown,
Power- Interlock-
SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt-
PresDet+ Interlock-
Changed: MRL- PresDet- LinkState-
RootCtl: ErrCorrectable+ ErrNon-Fatal+ ErrFatal+
PMEIntEna- CRSVisible-
RootCap: CRSVisible-
RootSta: PME ReqID 0000, PMEStatus- PMEPending-
DevCap2: Completion Timeout: Range ABC, TimeoutDis+ ARIFwd-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-
ARIFwd-
LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance-
SpeedDis-, Selectable De-emphasis: -6dB
Transmit Margin: Normal Operating Range,
EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB,
EqualizationComplete-, EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-,
LinkEqualizationRequest-

        Capabilities: [80] MSI: Enable- Count=1/1 Maskable- 64bit-
                Address: 00000000  Data: 0000
        Capabilities: [90] Subsystem: Super Micro Computer Inc Device 

[15d9:0818]
Capabilities: [a0] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA
PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Kernel driver in use: pcieport

00:1c.3 PCI bridge [0604]: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #4 [8086:8c16] (rev d5) (prog-if 00

[Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr+ Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Bus: primary=00, secondary=0c, subordinate=0c, sec-latency=0 I/O behind bridge: 0000c000-0000cfff
Memory behind bridge: eec00000-eecfffff
Prefetchable memory behind bridge:
00000000fff00000-00000000000fffff
Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- <SERR- <PERR-
BridgeCtl: Parity+ SERR+ NoISA- VGA- MAbort- >Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: [40] Express (v2) Root Port (Slot+), MSI 00
DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s
<64ns, L1 <1us
ExtTag- RBE+ FLReset-
DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+
Unsupported-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq-
AuxPwr+ TransPend-
LnkCap: Port #4, Speed 5GT/s, Width x1, ASPM L0s L1,
Latency L0 <512ns, L1 <16us
ClockPM- Surprise- LLActRep+ BwNot+
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain-
CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+
DLActive+ BWMgmt+ ABWMgmt-
SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd-
HotPlug- Surprise-
Slot #3, PowerLimit 10.000W; Interlock- NoCompl+
SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt-
HPIrq- LinkChg-
Control: AttnInd Unknown, PwrInd Unknown,
Power- Interlock-
SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt-
PresDet+ Interlock-
Changed: MRL- PresDet- LinkState-
RootCtl: ErrCorrectable+ ErrNon-Fatal+ ErrFatal+
PMEIntEna- CRSVisible-
RootCap: CRSVisible-
RootSta: PME ReqID 0000, PMEStatus- PMEPending-
DevCap2: Completion Timeout: Range ABC, TimeoutDis+ ARIFwd-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-
ARIFwd-
LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance-
SpeedDis-, Selectable De-emphasis: -6dB
Transmit Margin: Normal Operating Range,
EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -3.5dB,
EqualizationComplete-, EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-,
LinkEqualizationRequest-

        Capabilities: [80] MSI: Enable- Count=1/1 Maskable- 64bit-
                Address: 00000000  Data: 0000
        Capabilities: [90] Subsystem: Super Micro Computer Inc Device 

[15d9:0818]
Capabilities: [a0] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA
PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Kernel driver in use: pcieport

00:1c.4 PCI bridge [0604]: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #5 [8086:8c18] (rev d5) (prog-if 00

[Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr+ Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 128 bytes
Bus: primary=00, secondary=0d, subordinate=45, sec-latency=0 I/O behind bridge: 00002000-00002fff
Memory behind bridge: d8000000-ee0fffff
Prefetchable memory behind bridge:
0000000090000000-00000000b1ffffff
Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- <SERR- <PERR-
BridgeCtl: Parity+ SERR+ NoISA- VGA- MAbort- >Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: [40] Express (v2) Root Port (Slot+), MSI 00
DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s
<64ns, L1 <1us
ExtTag- RBE+ FLReset-
DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+
Unsupported-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes
DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq-
AuxPwr+ TransPend-
LnkCap: Port #5, Speed 5GT/s, Width x4, ASPM L0s L1,
Latency L0 <1us, L1 <16us
ClockPM- Surprise- LLActRep+ BwNot+
LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes Disabled-
Retrain- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x4, TrErr- Train- SlotClk+
DLActive- BWMgmt+ ABWMgmt-
SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd-
HotPlug+ Surprise+
Slot #4, PowerLimit 25.000W; Interlock- NoCompl+
SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt-
HPIrq- LinkChg-
Control: AttnInd Unknown, PwrInd Unknown,
Power- Interlock-
SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt-
PresDet- Interlock-
Changed: MRL- PresDet- LinkState-
RootCtl: ErrCorrectable+ ErrNon-Fatal+ ErrFatal+
PMEIntEna- CRSVisible-
RootCap: CRSVisible-
RootSta: PME ReqID 0000, PMEStatus- PMEPending-
DevCap2: Completion Timeout: Range ABC, TimeoutDis+ ARIFwd-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-
ARIFwd-
LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance-
SpeedDis-, Selectable De-emphasis: -6dB
Transmit Margin: Normal Operating Range,
EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -3.5dB,
EqualizationComplete-, EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-,
LinkEqualizationRequest-

        Capabilities: [80] MSI: Enable- Count=1/1 Maskable- 64bit-
                Address: 00000000  Data: 0000
        Capabilities: [90] Subsystem: Super Micro Computer Inc Device 

[15d9:0818]
Capabilities: [a0] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA
PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Kernel driver in use: pcieport

00:1d.0 USB controller [0c03]: Intel Corporation 8 Series/C220 Series Chipset Family USB EHCI #1 [8086:8c26] (rev 05) (prog-if 20 [EHCI])
Subsystem: Super Micro Computer Inc Device [15d9:0818]
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr+ Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium

Quoted 1 line

TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-

Latency: 0
Interrupt: pin A routed to IRQ 23
Region 0: Memory at eef3b000 (32-bit, non-prefetchable) [size=1K]
Capabilities: [50] Power Management version 2
Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA
PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [58] Debug port: BAR=1 offset=00a0
Capabilities: [98] PCI Advanced Features
AFCap: TP+ FLR+
AFCtrl: FLR-
AFStatus: TP-
Kernel driver in use: ehci-pci

00:1f.0 ISA bridge [0601]: Intel Corporation C226 Series Chipset Family Server Advanced SKU LPC Controller [8086:8c56] (rev 05)
Subsystem: Super Micro Computer Inc Device [15d9:0818]
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr+ Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium

Quoted 1 line

TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-

Latency: 0
Capabilities: [e0] Vendor Specific Information: Len=0c <?> Kernel driver in use: lpc_ich

00:1f.2 SATA controller [0106]: Intel Corporation 8 Series/C220 Series Chipset Family 6-port SATA Controller 1 [AHCI mode] [8086:8c02]

(rev 05) (prog-if 01 [AHCI 1.0])
Subsystem: Super Micro Computer Inc Device [15d9:0818]
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr+ Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium

Quoted 1 line

TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-

        Latency: 0
        Interrupt: pin B routed to IRQ 59
        Region 0: I/O ports at f0b0 [size=8]
        Region 1: I/O ports at f0a0 [size=4]
        Region 2: I/O ports at f090 [size=8]
        Region 3: I/O ports at f080 [size=4]
        Region 4: I/O ports at f040 [size=32]
        Region 5: Memory at eef3a000 (32-bit, non-prefetchable) [size=2K]
        Capabilities: [80] MSI: Enable+ Count=1/1 Maskable- 64bit-
                Address: fee00518  Data: 0000
        Capabilities: [70] Power Management version 3
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA 

PME(D0-,D1-,D2-,D3hot+,D3cold-)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [a8] SATA HBA v1.0 BAR4 Offset=00000004 Kernel driver in use: ahci

00:1f.3 SMBus [0c05]: Intel Corporation 8 Series/C220 Series Chipset Family SMBus Controller [8086:8c22] (rev 05)
Subsystem: Super Micro Computer Inc Device [15d9:0818]
Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop-
ParErr+ Stepping- SERR+ FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium

Quoted 1 line

TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-

        Interrupt: pin C routed to IRQ 5
        Region 0: Memory at eef39000 (64-bit, non-prefetchable) [size=256]
        Region 4: I/O ports at 0580 [size=32]

00:1f.6 Signal processing controller [1180]: Intel Corporation 8 Series Chipset Family Thermal Management Controller [8086:8c24] (rev 05)
Subsystem: Super Micro Computer Inc Device [15d9:0818]
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr- Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0
Interrupt: pin C routed to IRQ 5
Region 0: Memory at eef38000 (64-bit, non-prefetchable) [size=4K]
Capabilities: [50] Power Management version 3
Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA
PME(D0-,D1-,D2-,D3hot-,D3cold-)

                Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [80] MSI: Enable- Count=1/1 Maskable- 64bit-
                Address: 00000000  Data: 0000

02:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Juniper PRO [Radeon HD 5750] [1002:68be] (prog-if 00 [VGA

controller])
Subsystem: XFX Pine Group Inc. Device [1682:2980]
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr+ Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 71
Region 0: Memory at c0000000 (64-bit, prefetchable) [size=256M] Region 2: Memory at eee20000 (64-bit, non-prefetchable) [size=128K] Region 4: I/O ports at e000 [size=256]
Expansion ROM at eee00000 [disabled] [size=128K]
Capabilities: [50] Power Management version 3
Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA
PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [58] Express (v2) Legacy Endpoint, MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s
<4us, L1 unlimited
ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+
Unsupported-
RlxdOrd- ExtTag+ PhantFunc- AuxPwr- NoSnoop+ MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+
AuxPwr- TransPend-
LnkCap: Port #1, Speed 5GT/s, Width x16, ASPM L0s L1,
Latency L0 <64ns, L1 <1us
ClockPM- Surprise- LLActRep- BwNot-
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain-
CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 5GT/s, Width x8, TrErr- Train- SlotClk+
DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Not Supported, TimeoutDis-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-
LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance-
SpeedDis-, Selectable De-emphasis: -6dB
Transmit Margin: Normal Operating Range,
EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -3.5dB,
EqualizationComplete-, EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-,
LinkEqualizationRequest-
Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
Address: 00000000fee00718 Data: 0000
Capabilities: [100 v1] Vendor Specific Information: ID=0001
Rev=1 Len=010 <?>
Capabilities: [150 v1] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt-
UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt-
UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt-
UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout-
NonFatalErr+
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout-
NonFatalErr+
AERCap: First Error Pointer: 00, GenCap+ CGenEn-
ChkCap+ ChkEn-
Kernel driver in use: radeon

02:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Juniper HDMI Audio [Radeon HD 5700 Series] [1002:aa58]
Subsystem: XFX Pine Group Inc. Device [1682:aa58]
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr+ Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Interrupt: pin B routed to IRQ 68
Region 0: Memory at eee40000 (64-bit, non-prefetchable) [size=16K]
Capabilities: [50] Power Management version 3
Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA
PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [58] Express (v2) Legacy Endpoint, MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s
<4us, L1 unlimited
ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+
Unsupported-
RlxdOrd- ExtTag+ PhantFunc- AuxPwr- NoSnoop+
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+
AuxPwr- TransPend-
LnkCap: Port #1, Speed 5GT/s, Width x16, ASPM L0s L1,
Latency L0 <64ns, L1 <1us
ClockPM- Surprise- LLActRep- BwNot-
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain-
CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 5GT/s, Width x8, TrErr- Train- SlotClk+
DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Not Supported, TimeoutDis-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-
LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance-
SpeedDis-, Selectable De-emphasis: -6dB
Transmit Margin: Normal Operating Range,
EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -3.5dB,
EqualizationComplete-, EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-,
LinkEqualizationRequest-
Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
Address: 00000000fee005d8 Data: 0000
Capabilities: [100 v1] Vendor Specific Information: ID=0001
Rev=1 Len=010 <?>
Capabilities: [150 v1] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt-
UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt-
UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt-
UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout-
NonFatalErr+
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout-
NonFatalErr+
AERCap: First Error Pointer: 00, GenCap+ CGenEn-
ChkCap+ ChkEn-
Kernel driver in use: snd_hda_intel

03:00.0 SATA controller [0106]: ASMedia Technology Inc. ASM1062 Serial ATA Controller [1b21:0612] (rev 01) (prog-if 01 [AHCI 1.0])
Subsystem: Super Micro Computer Inc Device [15d9:0818]
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr+ Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-

        Latency: 0, Cache Line Size: 64 bytes
        Interrupt: pin A routed to IRQ 60
        Region 0: I/O ports at d050 [size=8]
        Region 1: I/O ports at d040 [size=4]
        Region 2: I/O ports at d030 [size=8]
        Region 3: I/O ports at d020 [size=4]
        Region 4: I/O ports at d000 [size=32]
        Region 5: Memory at eed00000 (32-bit, non-prefetchable) [size=512]
        Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit-
                Address: fee00538  Data: 0000
        Capabilities: [78] Power Management version 3
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA 

PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [80] Express (v2) Legacy Endpoint, MSI 00
DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s
<1us, L1 <8us
ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal-
Unsupported-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop+ MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq-
AuxPwr- TransPend-
LnkCap: Port #1, Speed 5GT/s, Width x1, ASPM unknown,
Latency L0 <512ns, L1 <2us
ClockPM- Surprise- LLActRep- BwNot-
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain-
CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 5GT/s, Width x1, TrErr- Train- SlotClk+
DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Range ABC, TimeoutDis+
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-
LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance-
SpeedDis-, Selectable De-emphasis: -6dB
Transmit Margin: Normal Operating Range,
EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB,
EqualizationComplete-, EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-,
LinkEqualizationRequest-
Capabilities: [100 v1] Virtual Channel
Caps: LPEVC=0 RefClk=100ns PATEntryBits=1
Arb: Fixed- WRR32- WRR64- WRR128-
Ctrl: ArbSelect=Fixed
Status: InProgress-
VC0: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
Arb: Fixed- WRR32- WRR64- WRR128- TWRR128-
WRR256-
Ctrl: Enable+ ID=0 ArbSelect=Fixed TC/VC=01
Status: NegoPending- InProgress-
Kernel driver in use: ahci

04:00.0 PCI bridge [0604]: PLX Technology, Inc. PEX 8606 6 Lane, 6 Port PCI Express Gen 2 (5.0 GT/s) Switch [10b5:8606] (rev ba) (prog-if

00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr+ Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR+ <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Region 0: Memory at eeb00000 (32-bit, non-prefetchable) [size=128K]
Bus: primary=04, secondary=05, subordinate=0b, sec-latency=0 I/O behind bridge: 0000f000-00000fff
Memory behind bridge: ee800000-eeafffff
Prefetchable memory behind bridge:
00000000fff00000-00000000000fffff
Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- <SERR- <PERR-
BridgeCtl: Parity+ SERR+ NoISA- VGA- MAbort- >Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: [40] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA
PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [48] MSI: Enable+ Count=1/4 Maskable+ 64bit+
Address: 00000000fee00318 Data: 0000
Masking: 00000001 Pending: 00000000
Capabilities: [68] Express (v2) Upstream Port, MSI 00
DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s
<64ns, L1 <1us
ExtTag- AttnBtn- AttnInd- PwrInd- RBE+
FLReset-SlotPowerLimit 10.000W
DevCtl: Report errors: Correctable- Non-Fatal- Fatal-
Unsupported-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes
DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+
AuxPwr- TransPend-
LnkCap: Port #0, Speed 5GT/s, Width x1, ASPM L0s L1,
Latency L0 <1us, L1 <2us
ClockPM- Surprise- LLActRep- BwNot-
LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 5GT/s, Width x1, TrErr- Train- SlotClk-
DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Not Supported, TimeoutDis-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-
LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance-
SpeedDis-, Selectable De-emphasis: -6dB
Transmit Margin: Normal Operating Range,
EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB,
EqualizationComplete-, EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-,
LinkEqualizationRequest-
Capabilities: [a4] Subsystem: PLX Technology, Inc. PEX 8606 6
Lane, 6 Port PCI Express Gen 2 (5.0 GT/s) Switch [10b5:8606]
Capabilities: [100 v1] Device Serial Number ba-86-01-10-b5-df-0e-00
Capabilities: [fb4 v1] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt-
UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt-
UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt-
UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout-
NonFatalErr+
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout-
NonFatalErr+
AERCap: First Error Pointer: 1f, GenCap+ CGenEn-
ChkCap+ ChkEn-
Capabilities: [138 v1] Power Budgeting <?>
Capabilities: [148 v1] Virtual Channel
Caps: LPEVC=1 RefClk=100ns PATEntryBits=4
Arb: Fixed+ WRR32- WRR64- WRR128-
Ctrl: ArbSelect=Fixed
Status: InProgress-
VC0: Caps: PATOffset=06 MaxTimeSlots=1 RejSnoopTrans-
Arb: Fixed- WRR32+ WRR64- WRR128- TWRR128-
WRR256-
Ctrl: Enable+ ID=0 ArbSelect=WRR32 TC/VC=01
Status: NegoPending- InProgress-
Port Arbitration Table <?>
VC1: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
Arb: Fixed+ WRR32- WRR64- WRR128- TWRR128-
WRR256-
Ctrl: Enable- ID=1 ArbSelect=Fixed TC/VC=00
Status: NegoPending- InProgress-
Capabilities: [448 v1] Vendor Specific Information: ID=0000
Rev=0 Len=0cc <?>
Capabilities: [950 v1] Vendor Specific Information: ID=0001
Rev=0 Len=010 <?>
Kernel driver in use: pcieport

05:01.0 PCI bridge [0604]: PLX Technology, Inc. PEX 8606 6 Lane, 6 Port PCI Express Gen 2 (5.0 GT/s) Switch [10b5:8606] (rev ba) (prog-if

00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr+ Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-

        Latency: 0, Cache Line Size: 64 bytes
        Bus: primary=05, secondary=06, subordinate=06, sec-latency=0
        I/O behind bridge: 0000f000-00000fff
        Memory behind bridge: fff00000-000fffff
        Prefetchable memory behind bridge: 

00000000fff00000-00000000000fffff
Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- <SERR- <PERR-
BridgeCtl: Parity+ SERR+ NoISA- VGA- MAbort- >Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: [40] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA
PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [48] MSI: Enable+ Count=1/4 Maskable+ 64bit+
Address: 00000000fee00358 Data: 0000
Masking: 00000001 Pending: 00000000
Capabilities: [68] Express (v2) Downstream Port (Slot+), MSI 00
DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s
<64ns, L1 <1us
ExtTag- RBE+ FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal-
Unsupported-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes
DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+
AuxPwr- TransPend-
LnkCap: Port #1, Speed 5GT/s, Width x1, ASPM L0s L1,
Latency L0 <2us, L1 <4us
ClockPM- Surprise+ LLActRep+ BwNot+
LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x0, TrErr- Train- SlotClk-
DLActive- BWMgmt- ABWMgmt-
SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd-
HotPlug- Surprise-
Slot #113, PowerLimit 25.000W; Interlock- NoCompl-
SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt-
HPIrq- LinkChg-
Control: AttnInd Off, PwrInd Off, Power- Interlock-
SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt-
PresDet- Interlock-
Changed: MRL- PresDet- LinkState-
DevCap2: Completion Timeout: Not Supported, TimeoutDis-
ARIFwd+
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-
ARIFwd-
LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance-
SpeedDis-, Selectable De-emphasis: -6dB
Transmit Margin: Normal Operating Range,
EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB,
EqualizationComplete-, EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-,
LinkEqualizationRequest-
Capabilities: [a4] Subsystem: PLX Technology, Inc. PEX 8606 6
Lane, 6 Port PCI Express Gen 2 (5.0 GT/s) Switch [10b5:8606]
Capabilities: [100 v1] Device Serial Number ba-86-01-10-b5-df-0e-00
Capabilities: [fb4 v1] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt-
UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt-
UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt-
UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout-
NonFatalErr+
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout-
NonFatalErr+
AERCap: First Error Pointer: 1f, GenCap+ CGenEn-
ChkCap+ ChkEn-
Capabilities: [148 v1] Virtual Channel
Caps: LPEVC=1 RefClk=100ns PATEntryBits=1
Arb: Fixed+ WRR32- WRR64- WRR128-
Ctrl: ArbSelect=Fixed
Status: InProgress-
VC0: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
Arb: Fixed+ WRR32- WRR64- WRR128- TWRR128-
WRR256-
Ctrl: Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
Status: NegoPending- InProgress-
VC1: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
Arb: Fixed+ WRR32- WRR64- WRR128- TWRR128-
WRR256-
Ctrl: Enable- ID=1 ArbSelect=Fixed TC/VC=00
Status: NegoPending- InProgress-
Capabilities: [520 v1] Access Control Services
ACSCap: SrcValid+ TransBlk+ ReqRedir+ CmpltRedir+
UpstreamFwd+ EgressCtrl+ DirectTrans+
ACSCtl: SrcValid+ TransBlk- ReqRedir+ CmpltRedir+
UpstreamFwd+ EgressCtrl- DirectTrans-
Capabilities: [950 v1] Vendor Specific Information: ID=0001
Rev=0 Len=010 <?>
Kernel driver in use: pcieport

05:04.0 PCI bridge [0604]: PLX Technology, Inc. PEX 8606 6 Lane, 6 Port PCI Express Gen 2 (5.0 GT/s) Switch [10b5:8606] (rev ba) (prog-if

00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr+ Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-

        Latency: 0, Cache Line Size: 64 bytes
        Bus: primary=05, secondary=07, subordinate=07, sec-latency=0
        I/O behind bridge: 0000f000-00000fff
        Memory behind bridge: fff00000-000fffff
        Prefetchable memory behind bridge: 

00000000fff00000-00000000000fffff
Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- <SERR- <PERR-
BridgeCtl: Parity+ SERR+ NoISA- VGA- MAbort- >Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: [40] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA
PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [48] MSI: Enable+ Count=1/4 Maskable+ 64bit+
Address: 00000000fee00378 Data: 0000
Masking: 00000001 Pending: 00000000
Capabilities: [68] Express (v2) Downstream Port (Slot+), MSI 00
DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s
<64ns, L1 <1us
ExtTag- RBE+ FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal-
Unsupported-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes
DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+
AuxPwr- TransPend-
LnkCap: Port #4, Speed 5GT/s, Width x1, ASPM L0s L1,
Latency L0 <2us, L1 <4us
ClockPM- Surprise+ LLActRep+ BwNot+
LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x0, TrErr- Train- SlotClk-
DLActive- BWMgmt- ABWMgmt-
SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd-
HotPlug- Surprise-
Slot #116, PowerLimit 25.000W; Interlock- NoCompl-
SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt-
HPIrq- LinkChg-
Control: AttnInd Off, PwrInd Off, Power- Interlock-
SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt-
PresDet- Interlock-
Changed: MRL- PresDet- LinkState-
DevCap2: Completion Timeout: Not Supported, TimeoutDis-
ARIFwd+
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-
ARIFwd-
LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance-
SpeedDis-, Selectable De-emphasis: -6dB
Transmit Margin: Normal Operating Range,
EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB,
EqualizationComplete-, EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-,
LinkEqualizationRequest-
Capabilities: [a4] Subsystem: PLX Technology, Inc. PEX 8606 6
Lane, 6 Port PCI Express Gen 2 (5.0 GT/s) Switch [10b5:8606]
Capabilities: [100 v1] Device Serial Number ba-86-01-10-b5-df-0e-00
Capabilities: [fb4 v1] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt-
UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt-
UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt-
UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout-
NonFatalErr+
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout-
NonFatalErr+
AERCap: First Error Pointer: 1f, GenCap+ CGenEn-
ChkCap+ ChkEn-
Capabilities: [148 v1] Virtual Channel
Caps: LPEVC=1 RefClk=100ns PATEntryBits=1
Arb: Fixed+ WRR32- WRR64- WRR128-
Ctrl: ArbSelect=Fixed
Status: InProgress-
VC0: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
Arb: Fixed+ WRR32- WRR64- WRR128- TWRR128-
WRR256-
Ctrl: Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
Status: NegoPending- InProgress-
VC1: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
Arb: Fixed+ WRR32- WRR64- WRR128- TWRR128-
WRR256-
Ctrl: Enable- ID=1 ArbSelect=Fixed TC/VC=00
Status: NegoPending- InProgress-
Capabilities: [520 v1] Access Control Services
ACSCap: SrcValid+ TransBlk+ ReqRedir+ CmpltRedir+
UpstreamFwd+ EgressCtrl+ DirectTrans+
ACSCtl: SrcValid+ TransBlk- ReqRedir+ CmpltRedir+
UpstreamFwd+ EgressCtrl- DirectTrans-
Capabilities: [950 v1] Vendor Specific Information: ID=0001
Rev=0 Len=010 <?>
Kernel driver in use: pcieport

05:05.0 PCI bridge [0604]: PLX Technology, Inc. PEX 8606 6 Lane, 6 Port PCI Express Gen 2 (5.0 GT/s) Switch [10b5:8606] (rev ba) (prog-if

00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr+ Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-

        Latency: 0, Cache Line Size: 64 bytes
        Bus: primary=05, secondary=08, subordinate=08, sec-latency=0
        I/O behind bridge: 0000f000-00000fff
        Memory behind bridge: fff00000-000fffff
        Prefetchable memory behind bridge: 

00000000fff00000-00000000000fffff
Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- <SERR- <PERR-
BridgeCtl: Parity+ SERR+ NoISA- VGA- MAbort- >Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: [40] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA
PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [48] MSI: Enable+ Count=1/4 Maskable+ 64bit+
Address: 00000000fee00398 Data: 0000
Masking: 00000001 Pending: 00000000
Capabilities: [68] Express (v2) Downstream Port (Slot+), MSI 00
DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s
<64ns, L1 <1us
ExtTag- RBE+ FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal-
Unsupported-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes
DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+
AuxPwr- TransPend-
LnkCap: Port #5, Speed 5GT/s, Width x1, ASPM L0s L1,
Latency L0 <2us, L1 <4us
ClockPM- Surprise+ LLActRep+ BwNot+
LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x0, TrErr- Train- SlotClk-
DLActive- BWMgmt- ABWMgmt-
SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd-
HotPlug- Surprise-
Slot #117, PowerLimit 25.000W; Interlock- NoCompl-
SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt-
HPIrq- LinkChg-
Control: AttnInd Off, PwrInd Off, Power- Interlock-
SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt-
PresDet- Interlock-
Changed: MRL- PresDet- LinkState-
DevCap2: Completion Timeout: Not Supported, TimeoutDis-
ARIFwd+
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-
ARIFwd-
LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance-
SpeedDis-, Selectable De-emphasis: -6dB
Transmit Margin: Normal Operating Range,
EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB,
EqualizationComplete-, EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-,
LinkEqualizationRequest-
Capabilities: [a4] Subsystem: PLX Technology, Inc. PEX 8606 6
Lane, 6 Port PCI Express Gen 2 (5.0 GT/s) Switch [10b5:8606]
Capabilities: [100 v1] Device Serial Number ba-86-01-10-b5-df-0e-00
Capabilities: [fb4 v1] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt-
UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt-
UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt-
UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout-
NonFatalErr+
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout-
NonFatalErr+
AERCap: First Error Pointer: 1f, GenCap+ CGenEn-
ChkCap+ ChkEn-
Capabilities: [148 v1] Virtual Channel
Caps: LPEVC=1 RefClk=100ns PATEntryBits=1
Arb: Fixed+ WRR32- WRR64- WRR128-
Ctrl: ArbSelect=Fixed
Status: InProgress-
VC0: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
Arb: Fixed+ WRR32- WRR64- WRR128- TWRR128-
WRR256-
Ctrl: Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
Status: NegoPending- InProgress-
VC1: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
Arb: Fixed+ WRR32- WRR64- WRR128- TWRR128-
WRR256-
Ctrl: Enable- ID=1 ArbSelect=Fixed TC/VC=00
Status: NegoPending- InProgress-
Capabilities: [520 v1] Access Control Services
ACSCap: SrcValid+ TransBlk+ ReqRedir+ CmpltRedir+
UpstreamFwd+ EgressCtrl+ DirectTrans+
ACSCtl: SrcValid+ TransBlk- ReqRedir+ CmpltRedir+
UpstreamFwd+ EgressCtrl- DirectTrans-
Capabilities: [950 v1] Vendor Specific Information: ID=0001
Rev=0 Len=010 <?>
Kernel driver in use: pcieport

05:07.0 PCI bridge [0604]: PLX Technology, Inc. PEX 8606 6 Lane, 6 Port PCI Express Gen 2 (5.0 GT/s) Switch [10b5:8606] (rev ba) (prog-if

00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr+ Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR+ <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Bus: primary=05, secondary=09, subordinate=09, sec-latency=0 I/O behind bridge: 0000f000-00000fff
Memory behind bridge: eea00000-eeafffff
Prefetchable memory behind bridge:
00000000fff00000-00000000000fffff
Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- <SERR- <PERR-
BridgeCtl: Parity+ SERR+ NoISA- VGA- MAbort- >Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: [40] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA
PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [48] MSI: Enable+ Count=1/4 Maskable+ 64bit+
Address: 00000000fee003b8 Data: 0000
Masking: 00000001 Pending: 00000000
Capabilities: [68] Express (v2) Downstream Port (Slot+), MSI 00
DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s
<64ns, L1 <1us
ExtTag- RBE+ FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal-
Unsupported-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes
DevSta: CorrErr+ UncorrErr+ FatalErr- UnsuppReq+
AuxPwr- TransPend-
LnkCap: Port #7, Speed 5GT/s, Width x1, ASPM L0s L1,
Latency L0 <1us, L1 <2us
ClockPM- Surprise+ LLActRep+ BwNot+
LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 5GT/s, Width x1, TrErr- Train- SlotClk-
DLActive+ BWMgmt+ ABWMgmt+
SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd-
HotPlug- Surprise-
Slot #119, PowerLimit 25.000W; Interlock- NoCompl-
SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt-
HPIrq- LinkChg-
Control: AttnInd Off, PwrInd Off, Power- Interlock-
SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt-
PresDet+ Interlock-
Changed: MRL- PresDet- LinkState-
DevCap2: Completion Timeout: Not Supported, TimeoutDis-
ARIFwd+
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-
ARIFwd-
LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance-
SpeedDis-, Selectable De-emphasis: -6dB
Transmit Margin: Normal Operating Range,
EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB,
EqualizationComplete-, EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-,
LinkEqualizationRequest-
Capabilities: [a4] Subsystem: PLX Technology, Inc. PEX 8606 6
Lane, 6 Port PCI Express Gen 2 (5.0 GT/s) Switch [10b5:8606]
Capabilities: [100 v1] Device Serial Number ba-86-01-10-b5-df-0e-00
Capabilities: [fb4 v1] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt-
UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt-
UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt-
UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout-
NonFatalErr+
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout-
NonFatalErr+
AERCap: First Error Pointer: 14, GenCap+ CGenEn-
ChkCap+ ChkEn-
Capabilities: [148 v1] Virtual Channel
Caps: LPEVC=1 RefClk=100ns PATEntryBits=1
Arb: Fixed+ WRR32- WRR64- WRR128-
Ctrl: ArbSelect=Fixed
Status: InProgress-
VC0: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
Arb: Fixed+ WRR32- WRR64- WRR128- TWRR128-
WRR256-
Ctrl: Enable+ ID=0 ArbSelect=Fixed TC/VC=01
Status: NegoPending- InProgress-
VC1: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
Arb: Fixed+ WRR32- WRR64- WRR128- TWRR128-
WRR256-
Ctrl: Enable- ID=1 ArbSelect=Fixed TC/VC=00
Status: NegoPending- InProgress-
Capabilities: [520 v1] Access Control Services
ACSCap: SrcValid+ TransBlk+ ReqRedir+ CmpltRedir+
UpstreamFwd+ EgressCtrl+ DirectTrans+
ACSCtl: SrcValid+ TransBlk- ReqRedir+ CmpltRedir+
UpstreamFwd+ EgressCtrl- DirectTrans-
Capabilities: [950 v1] Vendor Specific Information: ID=0001
Rev=0 Len=010 <?>
Kernel driver in use: pcieport

05:09.0 PCI bridge [0604]: PLX Technology, Inc. PEX 8606 6 Lane, 6 Port PCI Express Gen 2 (5.0 GT/s) Switch [10b5:8606] (rev ba) (prog-if

00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr+ Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Bus: primary=05, secondary=0a, subordinate=0b, sec-latency=0 I/O behind bridge: 0000f000-00000fff
Memory behind bridge: ee800000-ee9fffff
Prefetchable memory behind bridge:
00000000fff00000-00000000000fffff
Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- <SERR- <PERR-
BridgeCtl: Parity+ SERR+ NoISA- VGA- MAbort- >Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: [40] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA
PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [48] MSI: Enable+ Count=1/4 Maskable+ 64bit+
Address: 00000000fee003d8 Data: 0000
Masking: 00000001 Pending: 00000000
Capabilities: [68] Express (v2) Downstream Port (Slot+), MSI 00
DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s
<64ns, L1 <1us
ExtTag- RBE+ FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal-
Unsupported-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes
DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+
AuxPwr- TransPend-
LnkCap: Port #9, Speed 5GT/s, Width x1, ASPM L0s L1,
Latency L0 <2us, L1 <4us
ClockPM- Surprise+ LLActRep+ BwNot+
LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk-
DLActive+ BWMgmt+ ABWMgmt-
SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd-
HotPlug- Surprise-
Slot #121, PowerLimit 25.000W; Interlock- NoCompl-
SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt-
HPIrq- LinkChg-
Control: AttnInd Off, PwrInd Off, Power- Interlock-
SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt-
PresDet+ Interlock-
Changed: MRL- PresDet- LinkState-
DevCap2: Completion Timeout: Not Supported, TimeoutDis-
ARIFwd+
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-
ARIFwd-
LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance-
SpeedDis-, Selectable De-emphasis: -6dB
Transmit Margin: Normal Operating Range,
EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB,
EqualizationComplete-, EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-,
LinkEqualizationRequest-
Capabilities: [a4] Subsystem: PLX Technology, Inc. PEX 8606 6
Lane, 6 Port PCI Express Gen 2 (5.0 GT/s) Switch [10b5:8606]
Capabilities: [100 v1] Device Serial Number ba-86-01-10-b5-df-0e-00
Capabilities: [fb4 v1] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt-
UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt-
UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt-
UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout-
NonFatalErr+
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout-
NonFatalErr+
AERCap: First Error Pointer: 1f, GenCap+ CGenEn-
ChkCap+ ChkEn-
Capabilities: [148 v1] Virtual Channel
Caps: LPEVC=1 RefClk=100ns PATEntryBits=1
Arb: Fixed+ WRR32- WRR64- WRR128-
Ctrl: ArbSelect=Fixed
Status: InProgress-
VC0: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
Arb: Fixed+ WRR32- WRR64- WRR128- TWRR128-
WRR256-
Ctrl: Enable+ ID=0 ArbSelect=Fixed TC/VC=01
Status: NegoPending- InProgress-
VC1: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
Arb: Fixed+ WRR32- WRR64- WRR128- TWRR128-
WRR256-
Ctrl: Enable- ID=1 ArbSelect=Fixed TC/VC=00
Status: NegoPending- InProgress-
Capabilities: [520 v1] Access Control Services
ACSCap: SrcValid+ TransBlk+ ReqRedir+ CmpltRedir+
UpstreamFwd+ EgressCtrl+ DirectTrans+
ACSCtl: SrcValid+ TransBlk- ReqRedir+ CmpltRedir+
UpstreamFwd+ EgressCtrl- DirectTrans-
Capabilities: [950 v1] Vendor Specific Information: ID=0001
Rev=0 Len=010 <?>
Kernel driver in use: pcieport

09:00.0 USB controller [0c03]: Renesas Technology Corp. uPD720201 USB 3.0 Host Controller [1912:0014] (rev 03) (prog-if 30 [XHCI])
Subsystem: Super Micro Computer Inc Device [15d9:0818]
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr+ Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 16
Region 0: Memory at eea00000 (64-bit, non-prefetchable) [size=8K]
Capabilities: [50] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA
PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [70] MSI: Enable- Count=1/8 Maskable- 64bit+
Address: 0000000000000000 Data: 0000
Capabilities: [90] MSI-X: Enable+ Count=8 Masked-
Vector table: BAR=0 offset=00001000
PBA: BAR=0 offset=00001080
Capabilities: [a0] Express (v2) Endpoint, MSI 00
DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s
unlimited, L1 unlimited
ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+
Unsupported-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop+
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+
AuxPwr+ TransPend-
LnkCap: Port #0, Speed 5GT/s, Width x1, ASPM L0s L1,
Latency L0 <4us, L1 unlimited
ClockPM+ Surprise- LLActRep- BwNot-
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain-
CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 5GT/s, Width x1, TrErr- Train- SlotClk+
DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Not Supported, TimeoutDis+
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-
LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance-
SpeedDis-, Selectable De-emphasis: -6dB
Transmit Margin: Normal Operating Range,
EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB,
EqualizationComplete-, EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-,
LinkEqualizationRequest-
Capabilities: [100 v1] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt-
UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt-
UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt-
UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout-
NonFatalErr+
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout-
NonFatalErr+
AERCap: First Error Pointer: 00, GenCap- CGenEn-
ChkCap- ChkEn-
Capabilities: [150 v1] Latency Tolerance Reporting
Max snoop latency: 71680ns
Max no snoop latency: 71680ns
Kernel driver in use: xhci_hcd

0a:00.0 PCI bridge [0604]: Texas Instruments XIO2213A/B/XIO2221 PCI Express to PCI Bridge [Cheetah Express] [104c:823e] (rev 01) (prog-if

00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr+ Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Region 1: Memory at ee900000 (32-bit, non-prefetchable) [size=4K]
Bus: primary=0a, secondary=0b, subordinate=0b, sec-latency=0 I/O behind bridge: 0000f000-00000fff
Memory behind bridge: ee800000-ee8fffff
Prefetchable memory behind bridge:
00000000fff00000-00000000000fffff
Secondary status: 66MHz+ FastB2B+ ParErr- DEVSEL=medium

Quoted 1 line

TAbort- <TAbort- <MAbort+ <SERR- <PERR-

BridgeCtl: Parity+ SERR+ NoISA- VGA- MAbort- >Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: [50] Power Management version 3
Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA
PME(D0-,D1-,D2-,D3hot-,D3cold-)

                Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
                Bridge: PM- B3+
        Capabilities: [60] MSI: Enable- Count=1/16 Maskable- 64bit+
                Address: 0000000000000000  Data: 0000
        Capabilities: [80] Subsystem: Device [3412:7856]
        Capabilities: [90] Express (v1) PCI/PCI-X Bridge, MSI 00
                DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s 

<64ns, L1 <1us
ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+
Unsupported-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop+
BrConfRtry-
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+
AuxPwr- TransPend-

                LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, 

Latency L0 <1us, L1 <16us
ClockPM+ Surprise- LLActRep- BwNot-
LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+
DLActive- BWMgmt- ABWMgmt-
Capabilities: [100 v1] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt-
UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt-
UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt-
UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout-
NonFatalErr+
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout-
NonFatalErr+
AERCap: First Error Pointer: 14, GenCap+ CGenEn-
ChkCap+ ChkEn-

0b:00.0 FireWire (IEEE 1394) [0c00]: Texas Instruments XIO2213A/B/XIO2221 IEEE-1394b OHCI Controller [Cheetah Express] [104c:823f] (rev

01) (prog-if 10 [OHCI])
Subsystem: Device [3412:7856]
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr+ Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz+ UDF- FastB2B- ParErr- DEVSEL=medium

Quoted 1 line

TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-

Latency: 64 (500ns min, 1000ns max), Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 18
Region 0: Memory at ee804000 (32-bit, non-prefetchable) [size=2K] Region 1: Memory at ee800000 (32-bit, non-prefetchable) [size=16K]
Capabilities: [44] Power Management version 3
Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA
PME(D0+,D1+,D2+,D3hot+,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME+
Kernel driver in use: firewire_ohci

0c:00.0 Ethernet controller [0200]: Intel Corporation I210 Gigabit Network Connection [8086:1533] (rev 03)
Subsystem: Super Micro Computer Inc Device [15d9:1533]
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr+ Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 19
Region 0: Memory at eec00000 (32-bit, non-prefetchable) [size=512K]
Region 2: I/O ports at c000 [size=32]
Region 3: Memory at eec80000 (32-bit, non-prefetchable) [size=16K]
Capabilities: [40] Power Management version 3
Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA
PME(D0+,D1-,D2-,D3hot+,D3cold+)

                Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=1 PME-
        Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
                Address: 0000000000000000  Data: 0000
                Masking: 00000000  Pending: 00000000
        Capabilities: [70] MSI-X: Enable+ Count=5 Masked-
                Vector table: BAR=3 offset=00000000
                PBA: BAR=3 offset=00002000
        Capabilities: [a0] Express (v2) Endpoint, MSI 00
                DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s 

<512ns, L1 <64us
ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+
DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+
Unsupported+
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop+
FLReset-
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+
AuxPwr+ TransPend-

                LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, 

Latency L0 <2us, L1 <16us
ClockPM- Surprise- LLActRep- BwNot-
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain-
CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+
DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Range ABCD, TimeoutDis+
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-
LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance-
SpeedDis-, Selectable De-emphasis: -6dB
Transmit Margin: Normal Operating Range,
EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB,
EqualizationComplete-, EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-,
LinkEqualizationRequest-
Capabilities: [100 v2] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt-
UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt-
UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt-
UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout-
NonFatalErr+
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout-
NonFatalErr+
AERCap: First Error Pointer: 00, GenCap+ CGenEn-
ChkCap+ ChkEn-
Capabilities: [140 v1] Device Serial Number 00-25-90-ff-ff-d8-6d-cf
Capabilities: [1a0 v1] Transaction Processing Hints
Device specific mode supported
Steering table in TPH capability structure
Kernel driver in use: igb

#3 lore raw
Quoted 17 lines · Dana Goyette
Quoted 16 lines · Alex Williamson
Quoted 14 lines · Dana Goyette

I have purchased both a SuperMicro X10SAE and an X10SAT, and I need to soon decide which one to keep.

The SuperMicro X10SAT has all the PCIe x1 slots hidden behind a PLX PEX8066 switch, which claims to support ACS. I'd expect the devices downstream of the PLX switch to be in separate groups.

With Linux 3.13-rc5 and "enable overrides for missing ACS capabilities" applied and set for the Intel root ports, the devices behind the switch remain stuck in the same group.

In terms of passing devices to different VMs, which is better: all devices on different root ports, or all devices behind the one ACS-supporting switch?

Quoted 7 lines
Quoted 4 lines

Can you provide lspci -vvv info? If you're getting that for groups either the switch has ACS capabilities, but doesn't support the features we need or we're doing something wrong. Thanks,

I initially tried attaching the output as a .txt file, but it's too large. Anyway, here's the output of lspci -nnvvv (you may notice that I moved the Radeon to a different slot).

Well, something seems amiss since the downstream switch ports all seem to support and enable the correct set of ACS capabilities. I'm tending to suspect something wrong with the ACS override patch or how it's being used since your IOMMU group is still based at the root port. Each root port is isolated from the other root ports though, so something is happening with the override patch. Can you provide the kernel command line you use to enable ACS overrides and the override patch you're using, as it applies to 3.13-rc5? Thanks,

Alex

#4 lore raw

Re: IOMMU groups ... PEX8606 switch?

Quoted 35 lines · Alex Williamson
Quoted 17 lines · Dana Goyette
Quoted 16 lines · Alex Williamson
Quoted 14 lines · Dana Goyette

I have purchased both a SuperMicro X10SAE and an X10SAT, and I need to soon decide which one to keep.

The SuperMicro X10SAT has all the PCIe x1 slots hidden behind a PLX PEX8066 switch, which claims to support ACS. I'd expect the devices downstream of the PLX switch to be in separate groups.

With Linux 3.13-rc5 and "enable overrides for missing ACS capabilities" applied and set for the Intel root ports, the devices behind the switch remain stuck in the same group.

In terms of passing devices to different VMs, which is better: all devices on different root ports, or all devices behind the one ACS-supporting switch?

Quoted 7 lines
Quoted 4 lines

Can you provide lspci -vvv info? If you're getting that for groups either the switch has ACS capabilities, but doesn't support the features we need or we're doing something wrong. Thanks,

I initially tried attaching the output as a .txt file, but it's too large. Anyway, here's the output of lspci -nnvvv (you may notice that I moved the Radeon to a different slot).

Well, something seems amiss since the downstream switch ports all seem to support and enable the correct set of ACS capabilities. I'm tending to suspect something wrong with the ACS override patch or how it's being used since your IOMMU group is still based at the root port. Each root port is isolated from the other root ports though, so something is happening with the override patch. Can you provide the kernel command line you use to enable ACS overrides and the override patch you're using, as it applies to 3.13-rc5? Thanks,

Quoted 7 lines

Alex

--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

I'm using the original acs-override patch from this post:
https://lkml.org/lkml/2013/5/30/513

Kernel parameter is:
pcie_acs_override=id:8086:8c10,id:8086:8c12,id:8086:8c16,id:8086:8c18

When booting a kernel without the override patch, the following devices are all in the same group: Intel Root Ports 1, 2, 4, 5; ASMedia SATA controller; PLX PEX8606 switch; Renesas USB controller; TI Firewire controller; Intel I210 Ethernet controller.

#5 lore raw

Re: IOMMU groups ... PEX8606 switch?

Quoted 46 lines · Dana Goyette
Quoted 35 lines · Alex Williamson
Quoted 17 lines · Dana Goyette
Quoted 16 lines · Alex Williamson
Quoted 14 lines · Dana Goyette

I have purchased both a SuperMicro X10SAE and an X10SAT, and I need to soon decide which one to keep.

The SuperMicro X10SAT has all the PCIe x1 slots hidden behind a PLX PEX8066 switch, which claims to support ACS. I'd expect the devices downstream of the PLX switch to be in separate groups.

With Linux 3.13-rc5 and "enable overrides for missing ACS capabilities" applied and set for the Intel root ports, the devices behind the switch remain stuck in the same group.

In terms of passing devices to different VMs, which is better: all devices on different root ports, or all devices behind the one ACS-supporting switch?

Quoted 7 lines
Quoted 4 lines

Can you provide lspci -vvv info? If you're getting that for groups either the switch has ACS capabilities, but doesn't support the features we need or we're doing something wrong. Thanks,

I initially tried attaching the output as a .txt file, but it's too large. Anyway, here's the output of lspci -nnvvv (you may notice that I moved the Radeon to a different slot).

Well, something seems amiss since the downstream switch ports all seem to support and enable the correct set of ACS capabilities. I'm tending to suspect something wrong with the ACS override patch or how it's being used since your IOMMU group is still based at the root port. Each root port is isolated from the other root ports though, so something is happening with the override patch. Can you provide the kernel command line you use to enable ACS overrides and the override patch you're using, as it applies to 3.13-rc5? Thanks,

Quoted 7 lines

Alex

--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

I'm using the original acs-override patch from this post:
https://lkml.org/lkml/2013/5/30/513

Quoted 7 lines

Kernel parameter is:
pcie_acs_override=id:8086:8c10,id:8086:8c12,id:8086:8c16,id:8086:8c18

When booting a kernel without the override patch, the following devices are all in the same group: Intel Root Ports 1, 2, 4, 5; ASMedia SATA controller; PLX PEX8606 switch; Renesas USB controller; TI Firewire controller; Intel I210 Ethernet controller.

Ok, here's my shot in the dark; we must be detecting something about the upstream switch port to make it fail the ACS test and the only thing I can find that might do this is if the PCI config header on the upstream switch reported itself as a multifunction device. Multifunction upstream switch ports do need ACS capabilities to make sure that traffic isn't routed back through other functions. Single function devices do not. To test that theory, please provide 'lspci -vxs 4:00.0'. We're looking to see whether the byte at 0xe has the MSB set. If it does, it lies that it's a multifunction device. If it doesn't I'll have to get the dart board back out.

FWIW, you should be able to work around this by adding id:10b5:8606 to your list of overrides. Long term, if this is the problem, we'll want to add a quirk to sanitize the multifunction device flag. Thanks,

Alex

#6 lore raw

Re: IOMMU groups ... PEX8606 switch?

Quoted 66 lines · Alex Williamson
Quoted 46 lines · Dana Goyette
Quoted 35 lines · Alex Williamson
Quoted 17 lines · Dana Goyette
Quoted 16 lines · Alex Williamson
Quoted 14 lines · Dana Goyette

I have purchased both a SuperMicro X10SAE and an X10SAT, and I need to soon decide which one to keep.

The SuperMicro X10SAT has all the PCIe x1 slots hidden behind a PLX PEX8066 switch, which claims to support ACS. I'd expect the devices downstream of the PLX switch to be in separate groups.

With Linux 3.13-rc5 and "enable overrides for missing ACS capabilities" applied and set for the Intel root ports, the devices behind the switch remain stuck in the same group.

In terms of passing devices to different VMs, which is better: all devices on different root ports, or all devices behind the one ACS-supporting switch?

Quoted 7 lines
Quoted 4 lines

Can you provide lspci -vvv info? If you're getting that for groups either the switch has ACS capabilities, but doesn't support the features we need or we're doing something wrong. Thanks,

I initially tried attaching the output as a .txt file, but it's too large. Anyway, here's the output of lspci -nnvvv (you may notice that I moved the Radeon to a different slot).

Well, something seems amiss since the downstream switch ports all seem to support and enable the correct set of ACS capabilities. I'm tending to suspect something wrong with the ACS override patch or how it's being used since your IOMMU group is still based at the root port. Each root port is isolated from the other root ports though, so something is happening with the override patch. Can you provide the kernel command line you use to enable ACS overrides and the override patch you're using, as it applies to 3.13-rc5? Thanks,

Quoted 7 lines

Alex

--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

I'm using the original acs-override patch from this post:
https://lkml.org/lkml/2013/5/30/513

Quoted 7 lines

Kernel parameter is:
pcie_acs_override=id:8086:8c10,id:8086:8c12,id:8086:8c16,id:8086:8c18

When booting a kernel without the override patch, the following devices are all in the same group: Intel Root Ports 1, 2, 4, 5; ASMedia SATA controller; PLX PEX8606 switch; Renesas USB controller; TI Firewire controller; Intel I210 Ethernet controller.

Ok, here's my shot in the dark; we must be detecting something about the upstream switch port to make it fail the ACS test and the only thing I can find that might do this is if the PCI config header on the upstream switch reported itself as a multifunction device. Multifunction upstream switch ports do need ACS capabilities to make sure that traffic isn't routed back through other functions. Single function devices do not. To test that theory, please provide 'lspci -vxs 4:00.0'. We're looking to see whether the byte at 0xe has the MSB set. If it does, it lies that it's a multifunction device. If it doesn't I'll have to get the dart board back out.

Quoted 6 lines

FWIW, you should be able to work around this by adding id:10b5:8606 to your list of overrides. Long term, if this is the problem, we'll want to add a quirk to sanitize the multifunction device flag. Thanks,

Alex

04:00.0 is the ASMedia SATA controller; I'll assume you meant the upstream port of the PLX switch.

05:00.0 PCI bridge: PLX Technology, Inc. PEX 8606 6 Lane, 6 Port PCI Express Gen 2 (5.0 GT/s) Switch (rev ba) (prog-if 00 [Normal decode])
Flags: bus master, fast devsel, latency 0
Memory at eeb00000 (32-bit, non-prefetchable) [size=128K]
Bus: primary=05, secondary=06, subordinate=0c, sec-latency=0
Memory behind bridge: ee800000-eeafffff
Capabilities: [40] Power Management version 3
Capabilities: [48] MSI: Enable+ Count=1/4 Maskable+ 64bit+
Capabilities: [68] Express Upstream Port, MSI 00
Capabilities: [a4] Subsystem: PLX Technology, Inc. PEX 8606 6 Lane, 6
Port PCI Express Gen 2 (5.0 GT/s) Switch
Capabilities: [100] Device Serial Number ba-86-01-10-b5-df-0e-00
Capabilities: [fb4] Advanced Error Reporting
Capabilities: [138] Power Budgeting <?>
Capabilities: [148] Virtual Channel
Capabilities: [448] Vendor Specific Information: ID=0000 Rev=0 Len=0cc <?>
Capabilities: [950] Vendor Specific Information: ID=0001 Rev=0 Len=010 <?>
Kernel driver in use: pcieport
00: b5 10 06 86 47 05 10 40 ba 00 04 06 10 00 01 00
10: 00 00 b0 ee 00 00 00 00 05 06 0c 00 f1 01 00 00
20: 80 ee a0 ee f1 ff 01 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 40 00 00 00 00 00 00 00 0a 01 13 00

#7 lore raw

Re: IOMMU groups ... PEX8606 switch?

Quoted 71 lines · Dana Goyette
Quoted 70 lines · Alex Williamson
Quoted 46 lines · Dana Goyette
Quoted 35 lines · Alex Williamson
Quoted 17 lines · Dana Goyette
Quoted 16 lines · Alex Williamson
Quoted 14 lines · Dana Goyette

I have purchased both a SuperMicro X10SAE and an X10SAT, and I need to soon decide which one to keep.

The SuperMicro X10SAT has all the PCIe x1 slots hidden behind a PLX PEX8066 switch, which claims to support ACS. I'd expect the devices downstream of the PLX switch to be in separate groups.

With Linux 3.13-rc5 and "enable overrides for missing ACS capabilities" applied and set for the Intel root ports, the devices behind the switch remain stuck in the same group.

In terms of passing devices to different VMs, which is better: all devices on different root ports, or all devices behind the one ACS-supporting switch?

Quoted 7 lines
Quoted 4 lines

Can you provide lspci -vvv info? If you're getting that for groups either the switch has ACS capabilities, but doesn't support the features we need or we're doing something wrong. Thanks,

I initially tried attaching the output as a .txt file, but it's too large. Anyway, here's the output of lspci -nnvvv (you may notice that I moved the Radeon to a different slot).

Well, something seems amiss since the downstream switch ports all seem to support and enable the correct set of ACS capabilities. I'm tending to suspect something wrong with the ACS override patch or how it's being used since your IOMMU group is still based at the root port. Each root port is isolated from the other root ports though, so something is happening with the override patch. Can you provide the kernel command line you use to enable ACS overrides and the override patch you're using, as it applies to 3.13-rc5? Thanks,

Quoted 7 lines

Alex

--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

I'm using the original acs-override patch from this post:
https://lkml.org/lkml/2013/5/30/513

Quoted 7 lines

Kernel parameter is:
pcie_acs_override=id:8086:8c10,id:8086:8c12,id:8086:8c16,id:8086:8c18

When booting a kernel without the override patch, the following devices are all in the same group: Intel Root Ports 1, 2, 4, 5; ASMedia SATA controller; PLX PEX8606 switch; Renesas USB controller; TI Firewire controller; Intel I210 Ethernet controller.

Ok, here's my shot in the dark; we must be detecting something about the upstream switch port to make it fail the ACS test and the only thing I can find that might do this is if the PCI config header on the upstream switch reported itself as a multifunction device. Multifunction upstream switch ports do need ACS capabilities to make sure that traffic isn't routed back through other functions. Single function devices do not. To test that theory, please provide 'lspci -vxs 4:00.0'. We're looking to see whether the byte at 0xe has the MSB set. If it does, it lies that it's a multifunction device. If it doesn't I'll have to get the dart board back out.

FWIW, you should be able to work around this by adding id:10b5:8606 to your list of overrides. Long term, if this is the problem, we'll want to add a quirk to sanitize the multifunction device flag. Thanks,

Quoted 5 lines
Quoted 2 lines

Alex

04:00.0 is the ASMedia SATA controller; I'll assume you meant the upstream port of the PLX switch.

Yes, it was 04:00.0 in the previous lspci listing.

Quoted 11 lines

05:00.0 PCI bridge: PLX Technology, Inc. PEX 8606 6 Lane, 6 Port PCI Express Gen 2 (5.0 GT/s) Switch (rev ba) (prog-if 00 [Normal decode])
Flags: bus master, fast devsel, latency 0
Memory at eeb00000 (32-bit, non-prefetchable) [size=128K]
Bus: primary=05, secondary=06, subordinate=0c, sec-latency=0 Memory behind bridge: ee800000-eeafffff
Capabilities: [40] Power Management version 3
Capabilities: [48] MSI: Enable+ Count=1/4 Maskable+ 64bit+
Capabilities: [68] Express Upstream Port, MSI 00
Capabilities: [a4] Subsystem: PLX Technology, Inc. PEX 8606 6 Lane, 6
Port PCI Express Gen 2 (5.0 GT/s) Switch

Quoted 8 lines

Capabilities: [100] Device Serial Number ba-86-01-10-b5-df-0e-00
Capabilities: [fb4] Advanced Error Reporting
Capabilities: [138] Power Budgeting <?>
Capabilities: [148] Virtual Channel
Capabilities: [448] Vendor Specific Information: ID=0000 Rev=0 Len=0cc <?>
Capabilities: [950] Vendor Specific Information: ID=0001 Rev=0 Len=010 <?> Kernel driver in use: pcieport
00: b5 10 06 86 47 05 10 40 ba 00 04 06 10 00 01 00

Well, that wasn't it. I'll keep looking and probably send you a patch to figure out what's going wrong. Thanks,

Alex

#8 lore raw

Re: IOMMU groups ... PEX8606 switch?

In reply to Dana Goyette
Quoted 108 lines · Dana Goyette
Quoted 72 lines · Alex Williamson
Quoted 52 lines · Dana Goyette
Quoted 41 lines · Alex Williamson
Quoted 25 lines · Dana Goyette
Quoted 15 lines · Alex Williamson
Quoted 14 lines · Dana Goyette

I have purchased both a SuperMicro X10SAE and an X10SAT, and I need to
soon decide which one to keep.

The SuperMicro X10SAT has all the PCIe x1 slots hidden behind a PLX PEX8066 switch, which claims to support ACS. I'd expect the devices downstream of the PLX switch to be in separate groups.

With Linux 3.13-rc5 and "enable overrides for missing ACS capabilities"
applied and set for the Intel root ports, the devices behind the switch
remain stuck in the same group.

Quoted 9 lines
Quoted 3 lines

In terms of passing devices to different VMs, which is better: all devices on different root ports, or all devices behind the one ACS-supporting switch?

Can you provide lspci -vvv info? If you're getting that for groups either the switch has ACS capabilities, but doesn't support the features
we need or we're doing something wrong. Thanks,

Quoted 4 lines

I initially tried attaching the output as a .txt file, but it's too large. Anyway, here's the output of lspci -nnvvv (you may notice that I
moved the Radeon to a different slot).

Well, something seems amiss since the downstream switch ports all seem to support and enable the correct set of ACS capabilities. I'm tending to suspect something wrong with the ACS override patch or how it's being
used since your IOMMU group is still based at the root port. Each root port is isolated from the other root ports though, so something is happening with the override patch. Can you provide the kernel command line you use to enable ACS overrides and the override patch you're using, as it applies to 3.13-rc5? Thanks,

Quoted 7 lines

Alex

--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

I'm using the original acs-override patch from this post:
https://lkml.org/lkml/2013/5/30/513

Quoted 7 lines

Kernel parameter is:
pcie_acs_override=id:8086:8c10,id:8086:8c12,id:8086:8c16,id:8086:8c18

When booting a kernel without the override patch, the following devices are all in the same group: Intel Root Ports 1, 2, 4, 5; ASMedia SATA controller; PLX PEX8606 switch; Renesas USB controller; TI Firewire controller; Intel I210 Ethernet controller.

Ok, here's my shot in the dark; we must be detecting something about the upstream switch port to make it fail the ACS test and the only thing I can find that might do this is if the PCI config header on the upstream switch reported itself as a multifunction device. Multifunction upstream switch ports do need ACS capabilities to make sure that traffic isn't routed back through other functions. Single function devices do not. To test that theory, please provide 'lspci -vxs 4:00.0'. We're looking to see whether the byte at 0xe has the MSB set. If it does, it lies that it's a multifunction device. If it doesn't I'll have to get the dart board back out.

Quoted 6 lines

FWIW, you should be able to work around this by adding id:10b5:8606 to your list of overrides. Long term, if this is the problem, we'll want to add a quirk to sanitize the multifunction device flag. Thanks,

Alex

04:00.0 is the ASMedia SATA controller; I'll assume you meant the upstream port of the PLX switch.

05:00.0 PCI bridge: PLX Technology, Inc. PEX 8606 6 Lane, 6 Port PCI Express Gen 2 (5.0 GT/s) Switch (rev ba) (prog-if 00 [Normal decode])
Flags: bus master, fast devsel, latency 0
Memory at eeb00000 (32-bit, non-prefetchable) [size=128K]
Bus: primary=05, secondary=06, subordinate=0c, sec-latency=0 Memory behind bridge: ee800000-eeafffff
Capabilities: [40] Power Management version 3
Capabilities: [48] MSI: Enable+ Count=1/4 Maskable+ 64bit+
Capabilities: [68] Express Upstream Port, MSI 00
Capabilities: [a4] Subsystem: PLX Technology, Inc. PEX 8606 6 Lane,
6 Port PCI Express Gen 2 (5.0 GT/s) Switch
Capabilities: [100] Device Serial Number ba-86-01-10-b5-df-0e-00
Capabilities: [fb4] Advanced Error Reporting
Capabilities: [138] Power Budgeting <?>
Capabilities: [148] Virtual Channel
Capabilities: [448] Vendor Specific Information: ID=0000 Rev=0
Len=0cc <?>
Capabilities: [950] Vendor Specific Information: ID=0001 Rev=0
Len=010 <?>
Kernel driver in use: pcieport
00: b5 10 06 86 47 05 10 40 ba 00 04 06 10 00 01 00
10: 00 00 b0 ee 00 00 00 00 05 06 0c 00 f1 01 00 00
20: 80 ee a0 ee f1 ff 01 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 40 00 00 00 00 00 00 00 0a 01 13 00

Quoted 5 lines

--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Just in case:

The root port:
00:1c.1 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #2 (rev d5) (prog-if 00 [Normal decode])
Flags: bus master, fast devsel, latency 0
Bus: primary=00, secondary=05, subordinate=0c, sec-latency=0
Memory behind bridge: ee800000-eebfffff
Capabilities: [40] Express Root Port (Slot+), MSI 00
Capabilities: [80] MSI: Enable- Count=1/1 Maskable- 64bit-
Capabilities: [90] Subsystem: Super Micro Computer Inc Device 0818
Capabilities: [a0] Power Management version 3
Kernel driver in use: pcieport
00: 86 80 12 8c 47 01 10 00 d5 00 04 06 10 00 81 00
10: 00 00 00 00 00 00 00 00 00 05 0c 00 f0 00 00 20
20: 80 ee b0 ee f1 ff 01 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 40 00 00 00 00 00 00 00 0a 02 13 00

The first downstream port:

06:01.0 PCI bridge: PLX Technology, Inc. PEX 8606 6 Lane, 6 Port PCI Express Gen 2 (5.0 GT/s) Switch (rev ba) (prog-if 00 [Normal decode])
Flags: bus master, fast devsel, latency 0
Bus: primary=06, secondary=07, subordinate=07, sec-latency=0
Capabilities: [40] Power Management version 3
Capabilities: [48] MSI: Enable+ Count=1/4 Maskable+ 64bit+
Capabilities: [68] Express Downstream Port (Slot+), MSI 00
Capabilities: [a4] Subsystem: PLX Technology, Inc. PEX 8606 6 Lane, 6
Port PCI Express Gen 2 (5.0 GT/s) Switch
Capabilities: [100] Device Serial Number ba-86-01-10-b5-df-0e-00
Capabilities: [fb4] Advanced Error Reporting
Capabilities: [148] Virtual Channel
Capabilities: [520] Access Control Services
Capabilities: [950] Vendor Specific Information: ID=0001 Rev=0 Len=010 <?>
Kernel driver in use: pcieport
00: b5 10 06 86 47 05 10 00 ba 00 04 06 10 00 01 00
10: 00 00 00 00 00 00 00 00 06 07 07 00 f1 01 00 00
20: f0 ff 00 00 f1 ff 01 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 40 00 00 00 00 00 00 00 0a 01 13 00

#9 lore raw

Re: IOMMU groups ... PEX8606 switch?

Quoted 146 lines · Dana Goyette
Quoted 108 lines · Dana Goyette
Quoted 72 lines · Alex Williamson
Quoted 52 lines · Dana Goyette
Quoted 41 lines · Alex Williamson
Quoted 25 lines · Dana Goyette
Quoted 15 lines · Alex Williamson
Quoted 14 lines · Dana Goyette

I have purchased both a SuperMicro X10SAE and an X10SAT, and I need to
soon decide which one to keep.

The SuperMicro X10SAT has all the PCIe x1 slots hidden behind a PLX PEX8066 switch, which claims to support ACS. I'd expect the devices downstream of the PLX switch to be in separate groups.

With Linux 3.13-rc5 and "enable overrides for missing ACS capabilities"
applied and set for the Intel root ports, the devices behind the switch
remain stuck in the same group.

Quoted 9 lines
Quoted 3 lines

In terms of passing devices to different VMs, which is better: all devices on different root ports, or all devices behind the one ACS-supporting switch?

Can you provide lspci -vvv info? If you're getting that for groups either the switch has ACS capabilities, but doesn't support the features
we need or we're doing something wrong. Thanks,

Quoted 4 lines

I initially tried attaching the output as a .txt file, but it's too large. Anyway, here's the output of lspci -nnvvv (you may notice that I
moved the Radeon to a different slot).

Well, something seems amiss since the downstream switch ports all seem to support and enable the correct set of ACS capabilities. I'm tending to suspect something wrong with the ACS override patch or how it's being
used since your IOMMU group is still based at the root port. Each root port is isolated from the other root ports though, so something is happening with the override patch. Can you provide the kernel command line you use to enable ACS overrides and the override patch you're using, as it applies to 3.13-rc5? Thanks,

Quoted 7 lines

Alex

--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

I'm using the original acs-override patch from this post:
https://lkml.org/lkml/2013/5/30/513

Quoted 7 lines

Kernel parameter is:
pcie_acs_override=id:8086:8c10,id:8086:8c12,id:8086:8c16,id:8086:8c18

When booting a kernel without the override patch, the following devices are all in the same group: Intel Root Ports 1, 2, 4, 5; ASMedia SATA controller; PLX PEX8606 switch; Renesas USB controller; TI Firewire controller; Intel I210 Ethernet controller.

Ok, here's my shot in the dark; we must be detecting something about the upstream switch port to make it fail the ACS test and the only thing I can find that might do this is if the PCI config header on the upstream switch reported itself as a multifunction device. Multifunction upstream switch ports do need ACS capabilities to make sure that traffic isn't routed back through other functions. Single function devices do not. To test that theory, please provide 'lspci -vxs 4:00.0'. We're looking to see whether the byte at 0xe has the MSB set. If it does, it lies that it's a multifunction device. If it doesn't I'll have to get the dart board back out.

Quoted 6 lines

FWIW, you should be able to work around this by adding id:10b5:8606 to your list of overrides. Long term, if this is the problem, we'll want to add a quirk to sanitize the multifunction device flag. Thanks,

Alex

04:00.0 is the ASMedia SATA controller; I'll assume you meant the upstream port of the PLX switch.

05:00.0 PCI bridge: PLX Technology, Inc. PEX 8606 6 Lane, 6 Port PCI Express Gen 2 (5.0 GT/s) Switch (rev ba) (prog-if 00 [Normal decode])
Flags: bus master, fast devsel, latency 0
Memory at eeb00000 (32-bit, non-prefetchable) [size=128K]
Bus: primary=05, secondary=06, subordinate=0c, sec-latency=0 Memory behind bridge: ee800000-eeafffff
Capabilities: [40] Power Management version 3
Capabilities: [48] MSI: Enable+ Count=1/4 Maskable+ 64bit+
Capabilities: [68] Express Upstream Port, MSI 00
Capabilities: [a4] Subsystem: PLX Technology, Inc. PEX 8606 6 Lane,
6 Port PCI Express Gen 2 (5.0 GT/s) Switch
Capabilities: [100] Device Serial Number ba-86-01-10-b5-df-0e-00
Capabilities: [fb4] Advanced Error Reporting
Capabilities: [138] Power Budgeting <?>
Capabilities: [148] Virtual Channel
Capabilities: [448] Vendor Specific Information: ID=0000 Rev=0
Len=0cc <?>
Capabilities: [950] Vendor Specific Information: ID=0001 Rev=0
Len=010 <?>
Kernel driver in use: pcieport
00: b5 10 06 86 47 05 10 40 ba 00 04 06 10 00 01 00
10: 00 00 b0 ee 00 00 00 00 05 06 0c 00 f1 01 00 00
20: 80 ee a0 ee f1 ff 01 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 40 00 00 00 00 00 00 00 0a 01 13 00

Quoted 5 lines

--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Just in case:

The root port:
00:1c.1 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #2 (rev d5) (prog-if 00 [Normal decode])
Flags: bus master, fast devsel, latency 0
Bus: primary=00, secondary=05, subordinate=0c, sec-latency=0 Memory behind bridge: ee800000-eebfffff
Capabilities: [40] Express Root Port (Slot+), MSI 00
Capabilities: [80] MSI: Enable- Count=1/1 Maskable- 64bit-
Capabilities: [90] Subsystem: Super Micro Computer Inc Device 0818
Capabilities: [a0] Power Management version 3
Kernel driver in use: pcieport
00: 86 80 12 8c 47 01 10 00 d5 00 04 06 10 00 81 00
10: 00 00 00 00 00 00 00 00 00 05 0c 00 f0 00 00 20
20: 80 ee b0 ee f1 ff 01 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 40 00 00 00 00 00 00 00 0a 02 13 00

The first downstream port:

06:01.0 PCI bridge: PLX Technology, Inc. PEX 8606 6 Lane, 6 Port PCI Express Gen 2 (5.0 GT/s) Switch (rev ba) (prog-if 00 [Normal decode])
Flags: bus master, fast devsel, latency 0
Bus: primary=06, secondary=07, subordinate=07, sec-latency=0
Capabilities: [40] Power Management version 3
Capabilities: [48] MSI: Enable+ Count=1/4 Maskable+ 64bit+
Capabilities: [68] Express Downstream Port (Slot+), MSI 00
Capabilities: [a4] Subsystem: PLX Technology, Inc. PEX 8606 6 Lane, 6
Port PCI Express Gen 2 (5.0 GT/s) Switch
Capabilities: [100] Device Serial Number ba-86-01-10-b5-df-0e-00
Capabilities: [fb4] Advanced Error Reporting

Quoted 8 lines

Capabilities: [148] Virtual Channel
Capabilities: [520] Access Control Services
Capabilities: [950] Vendor Specific Information: ID=0001 Rev=0 Len=010 <?> Kernel driver in use: pcieport
00: b5 10 06 86 47 05 10 00 ba 00 04 06 10 00 01 00
10: 00 00 00 00 00 00 00 00 06 07 07 00 f1 01 00 00
20: f0 ff 00 00 f1 ff 01 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 40 00 00 00 00 00 00 00 0a 01 13 00

Thanks, can you confirm whether adding id:10b5:8606 to the overrides works?

Alex

#10 lore raw

Re: IOMMU groups ... PEX8606 switch?

Quoted 156 lines · Alex Williamson
Quoted 146 lines · Dana Goyette
Quoted 108 lines · Dana Goyette
Quoted 72 lines · Alex Williamson
Quoted 52 lines · Dana Goyette
Quoted 41 lines · Alex Williamson
Quoted 25 lines · Dana Goyette
Quoted 15 lines · Alex Williamson
Quoted 14 lines · Dana Goyette

I have purchased both a SuperMicro X10SAE and an X10SAT, and I need to
soon decide which one to keep.

The SuperMicro X10SAT has all the PCIe x1 slots hidden behind a PLX PEX8066 switch, which claims to support ACS. I'd expect the devices downstream of the PLX switch to be in separate groups.

With Linux 3.13-rc5 and "enable overrides for missing ACS capabilities"
applied and set for the Intel root ports, the devices behind the switch
remain stuck in the same group.

Quoted 9 lines
Quoted 3 lines

In terms of passing devices to different VMs, which is better: all devices on different root ports, or all devices behind the one ACS-supporting switch?

Can you provide lspci -vvv info? If you're getting that for groups either the switch has ACS capabilities, but doesn't support the features
we need or we're doing something wrong. Thanks,

Quoted 4 lines

I initially tried attaching the output as a .txt file, but it's too large. Anyway, here's the output of lspci -nnvvv (you may notice that I
moved the Radeon to a different slot).

Well, something seems amiss since the downstream switch ports all seem to support and enable the correct set of ACS capabilities. I'm tending to suspect something wrong with the ACS override patch or how it's being
used since your IOMMU group is still based at the root port. Each root port is isolated from the other root ports though, so something is happening with the override patch. Can you provide the kernel command line you use to enable ACS overrides and the override patch you're using, as it applies to 3.13-rc5? Thanks,

Quoted 7 lines

Alex

--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

I'm using the original acs-override patch from this post:
https://lkml.org/lkml/2013/5/30/513

Quoted 7 lines

Kernel parameter is:
pcie_acs_override=id:8086:8c10,id:8086:8c12,id:8086:8c16,id:8086:8c18

When booting a kernel without the override patch, the following devices are all in the same group: Intel Root Ports 1, 2, 4, 5; ASMedia SATA controller; PLX PEX8606 switch; Renesas USB controller; TI Firewire controller; Intel I210 Ethernet controller.

Ok, here's my shot in the dark; we must be detecting something about the upstream switch port to make it fail the ACS test and the only thing I can find that might do this is if the PCI config header on the upstream switch reported itself as a multifunction device. Multifunction upstream switch ports do need ACS capabilities to make sure that traffic isn't routed back through other functions. Single function devices do not. To test that theory, please provide 'lspci -vxs 4:00.0'. We're looking to see whether the byte at 0xe has the MSB set. If it does, it lies that it's a multifunction device. If it doesn't I'll have to get the dart board back out.

Quoted 6 lines

FWIW, you should be able to work around this by adding id:10b5:8606 to your list of overrides. Long term, if this is the problem, we'll want to add a quirk to sanitize the multifunction device flag. Thanks,

Alex

04:00.0 is the ASMedia SATA controller; I'll assume you meant the upstream port of the PLX switch.

05:00.0 PCI bridge: PLX Technology, Inc. PEX 8606 6 Lane, 6 Port PCI Express Gen 2 (5.0 GT/s) Switch (rev ba) (prog-if 00 [Normal decode])
Flags: bus master, fast devsel, latency 0
Memory at eeb00000 (32-bit, non-prefetchable) [size=128K]
Bus: primary=05, secondary=06, subordinate=0c, sec-latency=0 Memory behind bridge: ee800000-eeafffff
Capabilities: [40] Power Management version 3
Capabilities: [48] MSI: Enable+ Count=1/4 Maskable+ 64bit+
Capabilities: [68] Express Upstream Port, MSI 00
Capabilities: [a4] Subsystem: PLX Technology, Inc. PEX 8606 6 Lane,
6 Port PCI Express Gen 2 (5.0 GT/s) Switch
Capabilities: [100] Device Serial Number ba-86-01-10-b5-df-0e-00
Capabilities: [fb4] Advanced Error Reporting
Capabilities: [138] Power Budgeting <?>
Capabilities: [148] Virtual Channel
Capabilities: [448] Vendor Specific Information: ID=0000 Rev=0
Len=0cc <?>
Capabilities: [950] Vendor Specific Information: ID=0001 Rev=0
Len=010 <?>
Kernel driver in use: pcieport
00: b5 10 06 86 47 05 10 40 ba 00 04 06 10 00 01 00
10: 00 00 b0 ee 00 00 00 00 05 06 0c 00 f1 01 00 00
20: 80 ee a0 ee f1 ff 01 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 40 00 00 00 00 00 00 00 0a 01 13 00

Quoted 5 lines

--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Just in case:

The root port:
00:1c.1 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #2 (rev d5) (prog-if 00 [Normal decode])
Flags: bus master, fast devsel, latency 0
Bus: primary=00, secondary=05, subordinate=0c, sec-latency=0 Memory behind bridge: ee800000-eebfffff
Capabilities: [40] Express Root Port (Slot+), MSI 00
Capabilities: [80] MSI: Enable- Count=1/1 Maskable- 64bit-
Capabilities: [90] Subsystem: Super Micro Computer Inc Device 0818
Capabilities: [a0] Power Management version 3
Kernel driver in use: pcieport
00: 86 80 12 8c 47 01 10 00 d5 00 04 06 10 00 81 00
10: 00 00 00 00 00 00 00 00 00 05 0c 00 f0 00 00 20
20: 80 ee b0 ee f1 ff 01 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 40 00 00 00 00 00 00 00 0a 02 13 00

The first downstream port:

06:01.0 PCI bridge: PLX Technology, Inc. PEX 8606 6 Lane, 6 Port PCI Express Gen 2 (5.0 GT/s) Switch (rev ba) (prog-if 00 [Normal decode])
Flags: bus master, fast devsel, latency 0
Bus: primary=06, secondary=07, subordinate=07, sec-latency=0
Capabilities: [40] Power Management version 3
Capabilities: [48] MSI: Enable+ Count=1/4 Maskable+ 64bit+
Capabilities: [68] Express Downstream Port (Slot+), MSI 00
Capabilities: [a4] Subsystem: PLX Technology, Inc. PEX 8606 6 Lane, 6
Port PCI Express Gen 2 (5.0 GT/s) Switch
Capabilities: [100] Device Serial Number ba-86-01-10-b5-df-0e-00
Capabilities: [fb4] Advanced Error Reporting

Quoted 8 lines

Capabilities: [148] Virtual Channel
Capabilities: [520] Access Control Services
Capabilities: [950] Vendor Specific Information: ID=0001 Rev=0 Len=010 <?> Kernel driver in use: pcieport
00: b5 10 06 86 47 05 10 00 ba 00 04 06 10 00 01 00
10: 00 00 00 00 00 00 00 00 06 07 07 00 f1 01 00 00
20: f0 ff 00 00 f1 ff 01 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 40 00 00 00 00 00 00 00 0a 01 13 00

Quoted 5 lines

Thanks, can you confirm whether adding id:10b5:8606 to the overrides works?

Alex

Oddly, adding that ID doesn't seem to change anything.

\boot\vmlinuz-3.13.0-rc5 ro root=UUID=8414610a-91ff-41b9-80c2-cc49ee34cce6 console=ttyS4,115200n8 console=tty0 intel_iommu=on,igfx_off pcie_acs_override=id:8086:8c10,id:8086:8c16,id:8086:8c18,id:8086:ac1a,id:8086:8c1c,id:8086:8c1e,id:10b5:8606 initrd=boot\initrd.img-3.13.0-rc5

### Group 10 ###
00:1c.1 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset
Family PCI Express Root Port #2 (rev d5)

    05:00.0 PCI bridge: PLX Technology, Inc. PEX 8606 6 Lane, 6 Port 
PCI Express Gen 2 (5.0 GT/s) Switch (rev ba)
    06:01.0 PCI bridge: PLX Technology, Inc. PEX 8606 6 Lane, 6 Port 
PCI Express Gen 2 (5.0 GT/s) Switch (rev ba)
    06:04.0 PCI bridge: PLX Technology, Inc. PEX 8606 6 Lane, 6 Port 
PCI Express Gen 2 (5.0 GT/s) Switch (rev ba)
    06:05.0 PCI bridge: PLX Technology, Inc. PEX 8606 6 Lane, 6 Port 
PCI Express Gen 2 (5.0 GT/s) Switch (rev ba)
    06:07.0 PCI bridge: PLX Technology, Inc. PEX 8606 6 Lane, 6 Port 
PCI Express Gen 2 (5.0 GT/s) Switch (rev ba)
    06:09.0 PCI bridge: PLX Technology, Inc. PEX 8606 6 Lane, 6 Port 
PCI Express Gen 2 (5.0 GT/s) Switch (rev ba)
    0a:00.0 USB controller: Renesas Technology Corp. uPD720201 USB 3.0 
Host Controller (rev 03)
    0b:00.0 PCI bridge: Texas Instruments XIO2213A/B/XIO2221 PCI 
Express to PCI Bridge [Cheetah Express] (rev 01)
    0c:00.0 FireWire (IEEE 1394): Texas Instruments XIO2213A/B/XIO2221 

IEEE-1394b OHCI Controller [Cheetah Express] (rev 01)

#11 lore raw

Re: IOMMU groups ... PEX8606 switch?

In reply to Dana Goyette
Quoted 46 lines · Dana Goyette
Quoted 35 lines · Alex Williamson
Quoted 17 lines · Dana Goyette
Quoted 16 lines · Alex Williamson
Quoted 14 lines · Dana Goyette

I have purchased both a SuperMicro X10SAE and an X10SAT, and I need to soon decide which one to keep.

The SuperMicro X10SAT has all the PCIe x1 slots hidden behind a PLX PEX8066 switch, which claims to support ACS. I'd expect the devices downstream of the PLX switch to be in separate groups.

With Linux 3.13-rc5 and "enable overrides for missing ACS capabilities" applied and set for the Intel root ports, the devices behind the switch remain stuck in the same group.

In terms of passing devices to different VMs, which is better: all devices on different root ports, or all devices behind the one ACS-supporting switch?

Quoted 7 lines
Quoted 4 lines

Can you provide lspci -vvv info? If you're getting that for groups either the switch has ACS capabilities, but doesn't support the features we need or we're doing something wrong. Thanks,

I initially tried attaching the output as a .txt file, but it's too large. Anyway, here's the output of lspci -nnvvv (you may notice that I moved the Radeon to a different slot).

Well, something seems amiss since the downstream switch ports all seem to support and enable the correct set of ACS capabilities. I'm tending to suspect something wrong with the ACS override patch or how it's being used since your IOMMU group is still based at the root port. Each root port is isolated from the other root ports though, so something is happening with the override patch. Can you provide the kernel command line you use to enable ACS overrides and the override patch you're using, as it applies to 3.13-rc5? Thanks,

Quoted 7 lines

Alex

--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

I'm using the original acs-override patch from this post:
https://lkml.org/lkml/2013/5/30/513

Quoted 7 lines

Kernel parameter is:
pcie_acs_override=id:8086:8c10,id:8086:8c12,id:8086:8c16,id:8086:8c18

When booting a kernel without the override patch, the following devices are all in the same group: Intel Root Ports 1, 2, 4, 5; ASMedia SATA controller; PLX PEX8606 switch; Renesas USB controller; TI Firewire controller; Intel I210 Ethernet controller.

Could you please try the patch below and send dmesg for the system once booted. This applies directly to upstream and includes the acs override patch. Thanks,

Alex

Documentation/kernel-parameters.txt +10 -0
index b9e9bd8..2481c6c 100644
@@ -2473,6 +2473,16 @@ bytes respectively. Such letter suffixes can also be entirely omitted.
 		nomsi	Do not use MSI for native PCIe PME signaling (this makes
 			all PCIe root ports use INTx for all services).
 
+	pcie_acs_override =
+			[PCIE] Override missing PCIe ACS support for:
+		downstream
+			All downstream ports - full ACS capabilties
+		multifunction
+			All multifunction devices - multifunction ACS subset
+		id:nnnn:nnnn
+			Specfic device - full ACS capabilities
+			Specified as vid:did (vendor/device ID) in hex
+
 	pcmv=		[HW,PCMCIA] BadgePAD 4
 
 	pd.		[PARIDE]
drivers/iommu/intel-iommu.c +10 -0
index 43b9bfe..b31961d 100644
@@ -4166,8 +4166,10 @@ static int intel_iommu_add_device(struct device *dev)
 			     pdev->bus->number, pdev->devfn))
 		return -ENODEV;
 
+	pr_info("%s(%s)\n", __func__, pci_name(pdev));
 	bridge = pci_find_upstream_pcie_bridge(pdev);
 	if (bridge) {
+		pr_info("Upstream bridge %s\n", pci_name(bridge));
 		if (pci_is_pcie(bridge))
 			dma_pdev = pci_get_domain_bus_and_slot(
 						pci_domain_nr(pdev->bus),
@@ -4177,9 +4179,13 @@ static int intel_iommu_add_device(struct device *dev)
 	} else
 		dma_pdev = pci_dev_get(pdev);
 
+	pr_info("dma_pdev #1: %s\n", pci_name(dma_pdev));
+
 	/* Account for quirked devices */
 	swap_pci_ref(&dma_pdev, pci_get_dma_source(dma_pdev));
 
+	pr_info("dma_pdev #2: %s\n", pci_name(dma_pdev));
+
 	/*
 	 * If it's a multifunction device that does not support our
 	 * required ACS flags, add to the same group as lowest numbered
@@ -4204,6 +4210,8 @@ static int intel_iommu_add_device(struct device *dev)
 		}
 	}
 
+	pr_info("dma_pdev #3: %s\n", pci_name(dma_pdev));
+
 	/*
 	 * Devices on the root bus go through the iommu.  If that's not us,
 	 * find the next upstream device and test ACS up to the root bus.
@@ -4225,6 +4233,8 @@ static int intel_iommu_add_device(struct device *dev)
 		swap_pci_ref(&dma_pdev, pci_dev_get(bus->self));
 	}
 
+	pr_info("dma_pdev #4: %s\n", pci_name(dma_pdev));
+
 root_bus:
 	group = iommu_group_get(&dma_pdev->dev);
 	pci_dev_put(dma_pdev);
drivers/pci/pci.c +41 -14

123 lines not on this page · raw message

drivers/pci/quirks.c +119 -0

130 lines not on this page · raw message

#12 lore raw

Re: IOMMU groups ... PEX8606 switch?

Quoted 48 lines · Alex Williamson
Quoted 47 lines · Dana Goyette
Quoted 33 lines · Alex Williamson
Quoted 16 lines · Dana Goyette
Quoted 15 lines · Alex Williamson
Quoted 14 lines · Dana Goyette

I have purchased both a SuperMicro X10SAE and an X10SAT, and I need to soon decide which one to keep.

The SuperMicro X10SAT has all the PCIe x1 slots hidden behind a PLX PEX8066 switch, which claims to support ACS. I'd expect the devices downstream of the PLX switch to be in separate groups.

With Linux 3.13-rc5 and "enable overrides for missing ACS capabilities" applied and set for the Intel root ports, the devices behind the switch remain stuck in the same group.

In terms of passing devices to different VMs, which is better: all devices on different root ports, or all devices behind the one ACS-supporting switch?

Quoted 7 lines
Quoted 4 lines

Can you provide lspci -vvv info? If you're getting that for groups either the switch has ACS capabilities, but doesn't support the features we need or we're doing something wrong. Thanks,

I initially tried attaching the output as a .txt file, but it's too large. Anyway, here's the output of lspci -nnvvv (you may notice that I moved the Radeon to a different slot).

Well, something seems amiss since the downstream switch ports all seem to support and enable the correct set of ACS capabilities. I'm tending to suspect something wrong with the ACS override patch or how it's being used since your IOMMU group is still based at the root port. Each root port is isolated from the other root ports though, so something is happening with the override patch. Can you provide the kernel command line you use to enable ACS overrides and the override patch you're using, as it applies to 3.13-rc5? Thanks,

Quoted 7 lines

Alex

--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

I'm using the original acs-override patch from this post:
https://lkml.org/lkml/2013/5/30/513

Kernel parameter is:
pcie_acs_override=id:8086:8c10,id:8086:8c12,id:8086:8c16,id:8086:8c18

Quoted 7 lines
Quoted 4 lines

When booting a kernel without the override patch, the following devices are all in the same group: Intel Root Ports 1, 2, 4, 5; ASMedia SATA controller; PLX PEX8606 switch; Renesas USB controller; TI Firewire controller; Intel I210 Ethernet controller.

Could you please try the patch below and send dmesg for the system once booted. This applies directly to upstream and includes the acs override patch. Thanks,

(removed patch from quote.)

Here's the complete dmesg, with pcie_acs_override still set:

http://pastebin.com/YHuKnrTb

Most relevant section:

[    0.524362] DMAR: No ATSR found
[    0.524386] IOMMU 1 0xfed91000: using Queued invalidation
[    0.524389] IOMMU: Setting RMRR:
[    0.524398] IOMMU: Setting identity map for device 0000:00:1d.0 [0x7bea1000 - 0x7beaffff]
[    0.524423] IOMMU: Setting identity map for device 0000:00:1a.0 [0x7bea1000 - 0x7beaffff]
[    0.524441] IOMMU: Setting identity map for device 0000:00:14.0 [0x7bea1000 - 0x7beaffff]
[    0.524454] IOMMU: Prepare 0-16MiB unity mapping for LPC
[    0.524461] IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
[    0.524548] PCI-DMA: Intel(R) Virtualization Technology for Directed I/O
[    0.524551] intel_iommu_add_device(0000:00:00.0)
[    0.524552] dma_pdev #1: 0000:00:00.0
[    0.524553] dma_pdev #2: 0000:00:00.0
[    0.524554] dma_pdev #3: 0000:00:00.0
[    0.524554] dma_pdev #4: 0000:00:00.0
[    0.524565] intel_iommu_add_device(0000:00:01.0)
[    0.524566] dma_pdev #1: 0000:00:01.0
[    0.524567] dma_pdev #2: 0000:00:01.0
[    0.524569] pci_acs_enabled(0000:00:01.0, 001d)
[    0.524572] pci_acs_flags_enabled no ACS capability on 0000:00:01.0
[    0.524573] pci_acs_flags_enabled(0000:00:01.0, 001d) -> false
[    0.524574] -> false
[    0.524575] pci_acs_enabled(0000:00:01.0, 001d)
[    0.524577] pci_acs_flags_enabled no ACS capability on 0000:00:01.0
[    0.524578] pci_acs_flags_enabled(0000:00:01.0, 001d) -> false
[    0.524579] -> false
[    0.524580] dma_pdev #3: 0000:00:01.0
[    0.524581] dma_pdev #4: 0000:00:01.0
[    0.524584] intel_iommu_add_device(0000:00:01.1)
[    0.524586] dma_pdev #1: 0000:00:01.1
[    0.524586] dma_pdev #2: 0000:00:01.1
[    0.524587] pci_acs_enabled(0000:00:01.1, 001d)
[    0.524589] pci_acs_flags_enabled no ACS capability on 0000:00:01.1
[    0.524590] pci_acs_flags_enabled(0000:00:01.1, 001d) -> false
[    0.524591] -> false
[    0.524592] pci_acs_enabled(0000:00:01.0, 001d)
[    0.524593] pci_acs_flags_enabled no ACS capability on 0000:00:01.0
[    0.524595] pci_acs_flags_enabled(0000:00:01.0, 001d) -> false
[    0.524596] -> false
[    0.524596] dma_pdev #3: 0000:00:01.0
[    0.524597] dma_pdev #4: 0000:00:01.0
[    0.524599] intel_iommu_add_device(0000:00:02.0)
[    0.524601] dma_pdev #1: 0000:00:02.0
[    0.524601] dma_pdev #2: 0000:00:02.0
[    0.524602] dma_pdev #3: 0000:00:02.0
[    0.524603] dma_pdev #4: 0000:00:02.0
[    0.524606] intel_iommu_add_device(0000:00:03.0)
[    0.524607] dma_pdev #1: 0000:00:03.0
[    0.524608] dma_pdev #2: 0000:00:03.0
[    0.524609] dma_pdev #3: 0000:00:03.0
[    0.524610] dma_pdev #4: 0000:00:03.0
[    0.524613] intel_iommu_add_device(0000:00:14.0)
[    0.524614] dma_pdev #1: 0000:00:14.0
[    0.524615] dma_pdev #2: 0000:00:14.0
[    0.524616] dma_pdev #3: 0000:00:14.0
[    0.524617] dma_pdev #4: 0000:00:14.0
[    0.524620] intel_iommu_add_device(0000:00:16.0)
[    0.524621] dma_pdev #1: 0000:00:16.0
[    0.524622] dma_pdev #2: 0000:00:16.0
[    0.524623] pci_acs_enabled(0000:00:16.0, 001d)
[    0.524624] -> false
[    0.524625] pci_acs_enabled(0000:00:16.0, 001d)
[    0.524626] -> false
[    0.524626] dma_pdev #3: 0000:00:16.0
[    0.524627] dma_pdev #4: 0000:00:16.0
[    0.524630] intel_iommu_add_device(0000:00:16.3)
[    0.524631] dma_pdev #1: 0000:00:16.3
[    0.524632] dma_pdev #2: 0000:00:16.3
[    0.524633] pci_acs_enabled(0000:00:16.3, 001d)
[    0.524634] -> false
[    0.524635] pci_acs_enabled(0000:00:16.0, 001d)
[    0.524636] -> false
[    0.524637] dma_pdev #3: 0000:00:16.0
[    0.524638] dma_pdev #4: 0000:00:16.0
[    0.524640] intel_iommu_add_device(0000:00:19.0)
[    0.524641] dma_pdev #1: 0000:00:19.0
[    0.524642] dma_pdev #2: 0000:00:19.0
[    0.524643] dma_pdev #3: 0000:00:19.0
[    0.524643] dma_pdev #4: 0000:00:19.0
[    0.524646] intel_iommu_add_device(0000:00:1a.0)
[    0.524647] dma_pdev #1: 0000:00:1a.0
[    0.524648] dma_pdev #2: 0000:00:1a.0
[    0.524649] dma_pdev #3: 0000:00:1a.0
[    0.524650] dma_pdev #4: 0000:00:1a.0
[    0.524654] intel_iommu_add_device(0000:00:1b.0)
[    0.524655] dma_pdev #1: 0000:00:1b.0
[    0.524656] dma_pdev #2: 0000:00:1b.0
[    0.524657] dma_pdev #3: 0000:00:1b.0
[    0.524658] dma_pdev #4: 0000:00:1b.0
[    0.524661] intel_iommu_add_device(0000:00:1c.0)
[    0.524662] dma_pdev #1: 0000:00:1c.0
[    0.524663] dma_pdev #2: 0000:00:1c.0
[    0.524664] pci_acs_enabled(0000:00:1c.0, 001d)
[    0.524665] PCIe ACS override match for 0000:00:1c.0
[    0.524666] -> true
[    0.524667] dma_pdev #3: 0000:00:1c.0
[    0.524668] dma_pdev #4: 0000:00:1c.0
[    0.524671] intel_iommu_add_device(0000:00:1c.1)
[    0.524672] dma_pdev #1: 0000:00:1c.1
[    0.524673] dma_pdev #2: 0000:00:1c.1
[    0.524674] pci_acs_enabled(0000:00:1c.1, 001d)
[    0.524681] pci_acs_flags_enabled no ACS capability on 0000:00:1c.1
[    0.524682] pci_acs_flags_enabled(0000:00:1c.1, 001d) -> false
[    0.524683] -> false
[    0.524684] pci_acs_enabled(0000:00:1c.0, 001d)
[    0.524685] PCIe ACS override match for 0000:00:1c.0
[    0.524686] -> true
[    0.524687] pci_acs_enabled(0000:00:1c.1, 001d)
[    0.524694] pci_acs_flags_enabled no ACS capability on 0000:00:1c.1
[    0.524695] pci_acs_flags_enabled(0000:00:1c.1, 001d) -> false
[    0.524696] -> false
[    0.524697] dma_pdev #3: 0000:00:1c.1
[    0.524698] dma_pdev #4: 0000:00:1c.1
[    0.524700] intel_iommu_add_device(0000:00:1c.3)
[    0.524701] dma_pdev #1: 0000:00:1c.3
[    0.524702] dma_pdev #2: 0000:00:1c.3
[    0.524703] pci_acs_enabled(0000:00:1c.3, 001d)
[    0.524704] PCIe ACS override match for 0000:00:1c.3
[    0.524705] -> true
[    0.524706] dma_pdev #3: 0000:00:1c.3
[    0.524707] dma_pdev #4: 0000:00:1c.3
[    0.524710] intel_iommu_add_device(0000:00:1c.4)
[    0.524711] dma_pdev #1: 0000:00:1c.4
[    0.524712] dma_pdev #2: 0000:00:1c.4
[    0.524713] pci_acs_enabled(0000:00:1c.4, 001d)
[    0.524714] PCIe ACS override match for 0000:00:1c.4
[    0.524715] -> true
[    0.524715] dma_pdev #3: 0000:00:1c.4
[    0.524716] dma_pdev #4: 0000:00:1c.4
[    0.524719] intel_iommu_add_device(0000:00:1d.0)
[    0.524720] dma_pdev #1: 0000:00:1d.0
[    0.524721] dma_pdev #2: 0000:00:1d.0
[    0.524722] dma_pdev #3: 0000:00:1d.0
[    0.524723] dma_pdev #4: 0000:00:1d.0
[    0.524726] intel_iommu_add_device(0000:00:1f.0)
[    0.524727] dma_pdev #1: 0000:00:1f.0
[    0.524728] dma_pdev #2: 0000:00:1f.0
[    0.524728] pci_acs_enabled(0000:00:1f.0, 001d)
[    0.524729] -> false
[    0.524730] pci_acs_enabled(0000:00:1f.0, 001d)
[    0.524731] -> false
[    0.524732] dma_pdev #3: 0000:00:1f.0
[    0.524733] dma_pdev #4: 0000:00:1f.0
[    0.524736] intel_iommu_add_device(0000:00:1f.2)
[    0.524737] dma_pdev #1: 0000:00:1f.2
[    0.524738] dma_pdev #2: 0000:00:1f.2
[    0.524739] pci_acs_enabled(0000:00:1f.2, 001d)
[    0.524740] -> false
[    0.524741] pci_acs_enabled(0000:00:1f.0, 001d)
[    0.524741] -> false
[    0.524742] dma_pdev #3: 0000:00:1f.0
[    0.524743] dma_pdev #4: 0000:00:1f.0
[    0.524745] intel_iommu_add_device(0000:00:1f.3)
[    0.524746] dma_pdev #1: 0000:00:1f.3
[    0.524747] dma_pdev #2: 0000:00:1f.3
[    0.524748] pci_acs_enabled(0000:00:1f.3, 001d)
[    0.524749] -> false
[    0.524750] pci_acs_enabled(0000:00:1f.0, 001d)
[    0.524751] -> false
[    0.524752] dma_pdev #3: 0000:00:1f.0
[    0.524752] dma_pdev #4: 0000:00:1f.0
[    0.524754] intel_iommu_add_device(0000:00:1f.6)
[    0.524755] dma_pdev #1: 0000:00:1f.6
[    0.524756] dma_pdev #2: 0000:00:1f.6
[    0.524757] pci_acs_enabled(0000:00:1f.6, 001d)
[    0.524758] -> false
[    0.524759] pci_acs_enabled(0000:00:1f.0, 001d)
[    0.524760] -> false
[    0.524761] dma_pdev #3: 0000:00:1f.0
[    0.524762] dma_pdev #4: 0000:00:1f.0
[    0.524765] intel_iommu_add_device(0000:01:00.0)
[    0.524766] dma_pdev #1: 0000:01:00.0
[    0.524767] dma_pdev #2: 0000:01:00.0
[    0.524768] pci_acs_enabled(0000:01:00.0, 001d)
[    0.524771] pci_acs_flags_enabled no ACS capability on 0000:01:00.0
[    0.524772] pci_acs_flags_enabled(0000:01:00.0, 001d) -> false
[    0.524773] -> false
[    0.524774] pci_acs_enabled(0000:01:00.0, 001d)
[    0.524776] pci_acs_flags_enabled no ACS capability on 0000:01:00.0
[    0.524777] pci_acs_flags_enabled(0000:01:00.0, 001d) -> false
[    0.524778] -> false
[    0.524779] dma_pdev #3: 0000:01:00.0
[    0.524780] pci_acs_path_enabled(0000:00:01.0, NULL, 001d)
[    0.524781] pci_acs_enabled(0000:00:01.0, 001d)
[    0.524783] pci_acs_flags_enabled no ACS capability on 0000:00:01.0
[    0.524784] pci_acs_flags_enabled(0000:00:01.0, 001d) -> false
[    0.524785] -> false
[    0.524786] -> false
[    0.524787] dma_pdev #4: 0000:00:01.0
[    0.524789] intel_iommu_add_device(0000:01:00.1)
[    0.524790] dma_pdev #1: 0000:01:00.1
[    0.524791] dma_pdev #2: 0000:01:00.1
[    0.524792] pci_acs_enabled(0000:01:00.1, 001d)
[    0.524794] pci_acs_flags_enabled no ACS capability on 0000:01:00.1
[    0.524795] pci_acs_flags_enabled(0000:01:00.1, 001d) -> false
[    0.524796] -> false
[    0.524797] pci_acs_enabled(0000:01:00.0, 001d)
[    0.524799] pci_acs_flags_enabled no ACS capability on 0000:01:00.0
[    0.524800] pci_acs_flags_enabled(0000:01:00.0, 001d) -> false
[    0.524801] -> false
[    0.524802] dma_pdev #3: 0000:01:00.0
[    0.524803] pci_acs_path_enabled(0000:00:01.0, NULL, 001d)
[    0.524804] pci_acs_enabled(0000:00:01.0, 001d)
[    0.524806] pci_acs_flags_enabled no ACS capability on 0000:00:01.0
[    0.524807] pci_acs_flags_enabled(0000:00:01.0, 001d) -> false
[    0.524808] -> false
[    0.524809] -> false
[    0.524809] dma_pdev #4: 0000:00:01.0
[    0.524811] intel_iommu_add_device(0000:02:00.0)
[    0.524813] dma_pdev #1: 0000:02:00.0
[    0.524813] dma_pdev #2: 0000:02:00.0
[    0.524814] dma_pdev #3: 0000:02:00.0
[    0.524815] pci_acs_path_enabled(0000:00:01.1, NULL, 001d)
[    0.524816] pci_acs_enabled(0000:00:01.1, 001d)
[    0.524818] pci_acs_flags_enabled no ACS capability on 0000:00:01.1
[    0.524819] pci_acs_flags_enabled(0000:00:01.1, 001d) -> false
[    0.524820] -> false
[    0.524821] -> false
[    0.524822] dma_pdev #4: 0000:00:01.1
[    0.524824] intel_iommu_add_device(0000:03:04.0)
[    0.524825] Upstream bridge 0000:02:00.0
[    0.524830] dma_pdev #1: 0000:02:00.0
[    0.524831] dma_pdev #2: 0000:02:00.0
[    0.524832] dma_pdev #3: 0000:02:00.0
[    0.524833] pci_acs_path_enabled(0000:00:01.1, NULL, 001d)
[    0.524834] pci_acs_enabled(0000:00:01.1, 001d)
[    0.524835] pci_acs_flags_enabled no ACS capability on 0000:00:01.1
[    0.524837] pci_acs_flags_enabled(0000:00:01.1, 001d) -> false
[    0.524837] -> false
[    0.524838] -> false
[    0.524839] dma_pdev #4: 0000:00:01.1
[    0.524841] intel_iommu_add_device(0000:04:00.0)
[    0.524842] dma_pdev #1: 0000:04:00.0
[    0.524843] dma_pdev #2: 0000:04:00.0
[    0.524844] dma_pdev #3: 0000:04:00.0
[    0.524845] pci_acs_path_enabled(0000:00:1c.0, NULL, 001d)
[    0.524846] pci_acs_enabled(0000:00:1c.0, 001d)
[    0.524847] PCIe ACS override match for 0000:00:1c.0
[    0.524848] -> true
[    0.524849] -> true
[    0.524849] dma_pdev #4: 0000:04:00.0
[    0.524852] intel_iommu_add_device(0000:05:00.0)
[    0.524853] dma_pdev #1: 0000:05:00.0
[    0.524854] dma_pdev #2: 0000:05:00.0
[    0.524855] dma_pdev #3: 0000:05:00.0
[    0.524856] pci_acs_path_enabled(0000:00:1c.1, NULL, 001d)
[    0.524857] pci_acs_enabled(0000:00:1c.1, 001d)
[    0.524865] pci_acs_flags_enabled no ACS capability on 0000:00:1c.1
[    0.524866] pci_acs_flags_enabled(0000:00:1c.1, 001d) -> false
[    0.524867] -> false
[    0.524868] -> false
[    0.524869] dma_pdev #4: 0000:00:1c.1
[    0.524871] intel_iommu_add_device(0000:06:01.0)
[    0.524872] dma_pdev #1: 0000:06:01.0
[    0.524872] dma_pdev #2: 0000:06:01.0
[    0.524873] dma_pdev #3: 0000:06:01.0
[    0.524874] pci_acs_path_enabled(0000:05:00.0, NULL, 001d)
[    0.524875] pci_acs_enabled(0000:05:00.0, 001d)
[    0.524876] PCIe ACS override match for 0000:05:00.0
[    0.524877] -> true
[    0.524878] pci_acs_enabled(0000:00:1c.1, 001d)
[    0.524886] pci_acs_flags_enabled no ACS capability on 0000:00:1c.1
[    0.524887] pci_acs_flags_enabled(0000:00:1c.1, 001d) -> false
[    0.524888] -> false
[    0.524888] -> false
[    0.524889] pci_acs_path_enabled(0000:00:1c.1, NULL, 001d)
[    0.524891] pci_acs_enabled(0000:00:1c.1, 001d)
[    0.524898] pci_acs_flags_enabled no ACS capability on 0000:00:1c.1
[    0.524899] pci_acs_flags_enabled(0000:00:1c.1, 001d) -> false
[    0.524900] -> false
[    0.524901] -> false
[    0.524901] dma_pdev #4: 0000:00:1c.1
[    0.524903] intel_iommu_add_device(0000:06:04.0)
[    0.524904] dma_pdev #1: 0000:06:04.0
[    0.524905] dma_pdev #2: 0000:06:04.0
[    0.524906] dma_pdev #3: 0000:06:04.0
[    0.524907] pci_acs_path_enabled(0000:05:00.0, NULL, 001d)
[    0.524908] pci_acs_enabled(0000:05:00.0, 001d)
[    0.524909] PCIe ACS override match for 0000:05:00.0
[    0.524910] -> true
[    0.524911] pci_acs_enabled(0000:00:1c.1, 001d)
[    0.524913] pci_acs_flags_enabled no ACS capability on 0000:00:1c.1
[    0.524914] pci_acs_flags_enabled(0000:00:1c.1, 001d) -> false
[    0.524915] -> false
[    0.524916] -> false
[    0.524917] pci_acs_path_enabled(0000:00:1c.1, NULL, 001d)
[    0.524918] pci_acs_enabled(0000:00:1c.1, 001d)
[    0.524925] pci_acs_flags_enabled no ACS capability on 0000:00:1c.1
[    0.524926] pci_acs_flags_enabled(0000:00:1c.1, 001d) -> false
[    0.524927] -> false
[    0.524928] -> false
[    0.524929] dma_pdev #4: 0000:00:1c.1
[    0.524949] intel_iommu_add_device(0000:06:05.0)
[    0.524951] dma_pdev #1: 0000:06:05.0
[    0.524951] dma_pdev #2: 0000:06:05.0
[    0.524952] dma_pdev #3: 0000:06:05.0
[    0.524953] pci_acs_path_enabled(0000:05:00.0, NULL, 001d)
[    0.524954] pci_acs_enabled(0000:05:00.0, 001d)
[    0.524955] PCIe ACS override match for 0000:05:00.0
[    0.524957] -> true
[    0.524958] pci_acs_enabled(0000:00:1c.1, 001d)
[    0.524964] pci_acs_flags_enabled no ACS capability on 0000:00:1c.1
[    0.524965] pci_acs_flags_enabled(0000:00:1c.1, 001d) -> false
[    0.524966] -> false
[    0.524967] -> false
[    0.524968] pci_acs_path_enabled(0000:00:1c.1, NULL, 001d)
[    0.524969] pci_acs_enabled(0000:00:1c.1, 001d)
[    0.524976] pci_acs_flags_enabled no ACS capability on 0000:00:1c.1
[    0.524977] pci_acs_flags_enabled(0000:00:1c.1, 001d) -> false
[    0.524978] -> false
[    0.524979] -> false
[    0.524979] dma_pdev #4: 0000:00:1c.1
[    0.524983] intel_iommu_add_device(0000:06:07.0)
[    0.524984] dma_pdev #1: 0000:06:07.0
[    0.524985] dma_pdev #2: 0000:06:07.0
[    0.524986] dma_pdev #3: 0000:06:07.0
[    0.524987] pci_acs_path_enabled(0000:05:00.0, NULL, 001d)
[    0.524988] pci_acs_enabled(0000:05:00.0, 001d)
[    0.524989] PCIe ACS override match for 0000:05:00.0
[    0.524990] -> true
[    0.524991] pci_acs_enabled(0000:00:1c.1, 001d)
[    0.524997] pci_acs_flags_enabled no ACS capability on 0000:00:1c.1
[    0.524998] pci_acs_flags_enabled(0000:00:1c.1, 001d) -> false
[    0.524999] -> false
[    0.525000] -> false
[    0.525001] pci_acs_path_enabled(0000:00:1c.1, NULL, 001d)
[    0.525002] pci_acs_enabled(0000:00:1c.1, 001d)
[    0.525004] pci_acs_flags_enabled no ACS capability on 0000:00:1c.1
[    0.525006] pci_acs_flags_enabled(0000:00:1c.1, 001d) -> false
[    0.525007] -> false
[    0.525007] -> false
[    0.525008] dma_pdev #4: 0000:00:1c.1
[    0.525010] intel_iommu_add_device(0000:06:09.0)
[    0.525011] dma_pdev #1: 0000:06:09.0
[    0.525012] dma_pdev #2: 0000:06:09.0
[    0.525013] dma_pdev #3: 0000:06:09.0
[    0.525014] pci_acs_path_enabled(0000:05:00.0, NULL, 001d)
[    0.525015] pci_acs_enabled(0000:05:00.0, 001d)
[    0.525016] PCIe ACS override match for 0000:05:00.0
[    0.525017] -> true
[    0.525018] pci_acs_enabled(0000:00:1c.1, 001d)
[    0.525025] pci_acs_flags_enabled no ACS capability on 0000:00:1c.1
[    0.525026] pci_acs_flags_enabled(0000:00:1c.1, 001d) -> false
[    0.525027] -> false
[    0.525028] -> false
[    0.525029] pci_acs_path_enabled(0000:00:1c.1, NULL, 001d)
[    0.525030] pci_acs_enabled(0000:00:1c.1, 001d)
[    0.525037] pci_acs_flags_enabled no ACS capability on 0000:00:1c.1
[    0.525038] pci_acs_flags_enabled(0000:00:1c.1, 001d) -> false
[    0.525039] -> false
[    0.525040] -> false
[    0.525041] dma_pdev #4: 0000:00:1c.1
[    0.525043] intel_iommu_add_device(0000:0a:00.0)
[    0.525044] dma_pdev #1: 0000:0a:00.0
[    0.525044] dma_pdev #2: 0000:0a:00.0
[    0.525045] dma_pdev #3: 0000:0a:00.0
[    0.525046] pci_acs_path_enabled(0000:06:07.0, NULL, 001d)
[    0.525048] pci_acs_enabled(0000:06:07.0, 001d)
[    0.525048] PCIe ACS override match for 0000:06:07.0
[    0.525049] -> true
[    0.525050] pci_acs_enabled(0000:05:00.0, 001d)
[    0.525051] PCIe ACS override match for 0000:05:00.0
[    0.525052] -> true
[    0.525053] pci_acs_enabled(0000:00:1c.1, 001d)
[    0.525060] pci_acs_flags_enabled no ACS capability on 0000:00:1c.1
[    0.525062] pci_acs_flags_enabled(0000:00:1c.1, 001d) -> false
[    0.525062] -> false
[    0.525063] -> false
[    0.525064] pci_acs_path_enabled(0000:05:00.0, NULL, 001d)
[    0.525065] pci_acs_enabled(0000:05:00.0, 001d)
[    0.525066] PCIe ACS override match for 0000:05:00.0
[    0.525067] -> true
[    0.525068] pci_acs_enabled(0000:00:1c.1, 001d)
[    0.525075] pci_acs_flags_enabled no ACS capability on 0000:00:1c.1
[    0.525076] pci_acs_flags_enabled(0000:00:1c.1, 001d) -> false
[    0.525077] -> false
[    0.525078] -> false
[    0.525079] pci_acs_path_enabled(0000:00:1c.1, NULL, 001d)
[    0.525080] pci_acs_enabled(0000:00:1c.1, 001d)
[    0.525087] pci_acs_flags_enabled no ACS capability on 0000:00:1c.1
[    0.525088] pci_acs_flags_enabled(0000:00:1c.1, 001d) -> false
[    0.525089] -> false
[    0.525090] -> false
[    0.525091] dma_pdev #4: 0000:00:1c.1
[    0.525093] intel_iommu_add_device(0000:0b:00.0)
[    0.525094] dma_pdev #1: 0000:0b:00.0
[    0.525095] dma_pdev #2: 0000:0b:00.0
[    0.525096] dma_pdev #3: 0000:0b:00.0
[    0.525097] pci_acs_path_enabled(0000:06:09.0, NULL, 001d)
[    0.525098] pci_acs_enabled(0000:06:09.0, 001d)
[    0.525099] PCIe ACS override match for 0000:06:09.0
[    0.525100] -> true
[    0.525101] pci_acs_enabled(0000:05:00.0, 001d)
[    0.525102] PCIe ACS override match for 0000:05:00.0
[    0.525103] -> true
[    0.525103] pci_acs_enabled(0000:00:1c.1, 001d)
[    0.525111] pci_acs_flags_enabled no ACS capability on 0000:00:1c.1
[    0.525112] pci_acs_flags_enabled(0000:00:1c.1, 001d) -> false
[    0.525113] -> false
[    0.525114] -> false
[    0.525115] pci_acs_path_enabled(0000:05:00.0, NULL, 001d)
[    0.525116] pci_acs_enabled(0000:05:00.0, 001d)
[    0.525117] PCIe ACS override match for 0000:05:00.0
[    0.525118] -> true
[    0.525118] pci_acs_enabled(0000:00:1c.1, 001d)
[    0.525125] pci_acs_flags_enabled no ACS capability on 0000:00:1c.1
[    0.525126] pci_acs_flags_enabled(0000:00:1c.1, 001d) -> false
[    0.525127] -> false
[    0.525128] -> false
[    0.525129] pci_acs_path_enabled(0000:00:1c.1, NULL, 001d)
[    0.525130] pci_acs_enabled(0000:00:1c.1, 001d)
[    0.525137] pci_acs_flags_enabled no ACS capability on 0000:00:1c.1
[    0.525138] pci_acs_flags_enabled(0000:00:1c.1, 001d) -> false
[    0.525139] -> false
[    0.525140] -> false
[    0.525141] dma_pdev #4: 0000:00:1c.1
[    0.525143] intel_iommu_add_device(0000:0c:00.0)
[    0.525144] Upstream bridge 0000:0b:00.0
[    0.525148] dma_pdev #1: 0000:0c:00.0
[    0.525149] dma_pdev #2: 0000:0c:00.0
[    0.525150] dma_pdev #3: 0000:0c:00.0
[    0.525151] pci_acs_path_enabled(0000:0b:00.0, NULL, 001d)
[    0.525152] pci_acs_enabled(0000:0b:00.0, 001d)
[    0.525153] -> false
[    0.525154] -> false
[    0.525155] pci_acs_path_enabled(0000:06:09.0, NULL, 001d)
[    0.525156] pci_acs_enabled(0000:06:09.0, 001d)
[    0.525157] PCIe ACS override match for 0000:06:09.0
[    0.525158] -> true
[    0.525158] pci_acs_enabled(0000:05:00.0, 001d)
[    0.525159] PCIe ACS override match for 0000:05:00.0
[    0.525160] -> true
[    0.525161] pci_acs_enabled(0000:00:1c.1, 001d)
[    0.525168] pci_acs_flags_enabled no ACS capability on 0000:00:1c.1
[    0.525169] pci_acs_flags_enabled(0000:00:1c.1, 001d) -> false
[    0.525170] -> false
[    0.525171] -> false
[    0.525172] pci_acs_path_enabled(0000:05:00.0, NULL, 001d)
[    0.525173] pci_acs_enabled(0000:05:00.0, 001d)
[    0.525174] PCIe ACS override match for 0000:05:00.0
[    0.525175] -> true
[    0.525175] pci_acs_enabled(0000:00:1c.1, 001d)
[    0.525178] pci_acs_flags_enabled no ACS capability on 0000:00:1c.1
[    0.525179] pci_acs_flags_enabled(0000:00:1c.1, 001d) -> false
[    0.525180] -> false
[    0.525181] -> false
[    0.525182] pci_acs_path_enabled(0000:00:1c.1, NULL, 001d)
[    0.525183] pci_acs_enabled(0000:00:1c.1, 001d)
[    0.525190] pci_acs_flags_enabled no ACS capability on 0000:00:1c.1
[    0.525191] pci_acs_flags_enabled(0000:00:1c.1, 001d) -> false
[    0.525192] -> false
[    0.525193] -> false
[    0.525193] dma_pdev #4: 0000:00:1c.1
[    0.525195] intel_iommu_add_device(0000:0d:00.0)
[    0.525196] dma_pdev #1: 0000:0d:00.0
[    0.525197] dma_pdev #2: 0000:0d:00.0
[    0.525198] dma_pdev #3: 0000:0d:00.0
[    0.525199] pci_acs_path_enabled(0000:00:1c.3, NULL, 001d)
[    0.525200] pci_acs_enabled(0000:00:1c.3, 001d)
[    0.525201] PCIe ACS override match for 0000:00:1c.3
[    0.525202] -> true
[    0.525203] -> true
[    0.525204] dma_pdev #4: 0000:0d:00.0

lspci as slots are now (since Radeon is back in first x16 slot):

00:00.0 Host bridge [0600]: Intel Corporation Xeon E3-1200 v3 Processor DRAM Controller [8086:0c08] (rev 06)
00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x16 Controller [8086:0c01] (rev 06) 00:01.1 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x8 Controller [8086:0c05] (rev 06) 00:02.0 VGA compatible controller [0300]: Intel Corporation Xeon E3-1200 v3 Processor Integrated Graphics Controller [8086:041a] (rev 06) 00:03.0 Audio device [0403]: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor HD Audio Controller [8086:0c0c] (rev 06) 00:14.0 USB controller [0c03]: Intel Corporation 8 Series/C220 Series Chipset Family USB xHCI [8086:8c31] (rev 05) 00:16.0 Communication controller [0780]: Intel Corporation 8 Series/C220 Series Chipset Family MEI Controller #1 [8086:8c3a] (rev 04) 00:16.3 Serial controller [0700]: Intel Corporation 8 Series/C220 Series Chipset Family KT Controller [8086:8c3d] (rev 04) 00:19.0 Ethernet controller [0200]: Intel Corporation Ethernet Connection I217-LM [8086:153a] (rev 05)
00:1b.0 Audio device [0403]: Intel Corporation 8 Series/C220 Series Chipset High Definition Audio Controller [8086:8c20] (rev 05) 00:1c.0 PCI bridge [0604]: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #1 [8086:8c10] (rev d5) 00:1c.1 PCI bridge [0604]: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #2 [8086:8c12] (rev d5) 00:1c.3 PCI bridge [0604]: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #4 [8086:8c16] (rev d5) 00:1c.4 PCI bridge [0604]: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #5 [8086:8c18] (rev d5) 00:1d.0 USB controller [0c03]: Intel Corporation 8 Series/C220 Series Chipset Family USB EHCI #1 [8086:8c26] (rev 05) 00:1f.0 ISA bridge [0601]: Intel Corporation C226 Series Chipset Family Server Advanced SKU LPC Controller [8086:8c56] (rev 05) 00:1f.2 SATA controller [0106]: Intel Corporation 8 Series/C220 Series Chipset Family 6-port SATA Controller 1 [AHCI mode] [8086:8c02] (rev 05) 00:1f.3 SMBus [0c05]: Intel Corporation 8 Series/C220 Series Chipset Family SMBus Controller [8086:8c22] (rev 05) 00:1f.6 Signal processing controller [1180]: Intel Corporation 8 Series Chipset Family Thermal Management Controller [8086:8c24] (rev 05) 01:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Juniper PRO [Radeon HD 5750] [1002:68be] 01:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Juniper HDMI Audio [Radeon HD 5700 Series] [1002:aa58] 02:00.0 PCI bridge [0604]: PLX Technology, Inc. PEX8112 x1 Lane PCI Express-to-PCI Bridge [10b5:8112] (rev aa)
03:04.0 Multimedia audio controller [0401]: C-Media Electronics Inc CMI8788 [Oxygen HD Audio] [13f6:8788]
04:00.0 SATA controller [0106]: ASMedia Technology Inc. ASM1062 Serial ATA Controller [1b21:0612] (rev 01)
05:00.0 PCI bridge [0604]: PLX Technology, Inc. PEX 8606 6 Lane, 6 Port PCI Express Gen 2 (5.0 GT/s) Switch [10b5:8606] (rev ba) 06:01.0 PCI bridge [0604]: PLX Technology, Inc. PEX 8606 6 Lane, 6 Port PCI Express Gen 2 (5.0 GT/s) Switch [10b5:8606] (rev ba) 06:04.0 PCI bridge [0604]: PLX Technology, Inc. PEX 8606 6 Lane, 6 Port PCI Express Gen 2 (5.0 GT/s) Switch [10b5:8606] (rev ba) 06:05.0 PCI bridge [0604]: PLX Technology, Inc. PEX 8606 6 Lane, 6 Port PCI Express Gen 2 (5.0 GT/s) Switch [10b5:8606] (rev ba) 06:07.0 PCI bridge [0604]: PLX Technology, Inc. PEX 8606 6 Lane, 6 Port PCI Express Gen 2 (5.0 GT/s) Switch [10b5:8606] (rev ba) 06:09.0 PCI bridge [0604]: PLX Technology, Inc. PEX 8606 6 Lane, 6 Port PCI Express Gen 2 (5.0 GT/s) Switch [10b5:8606] (rev ba) 0a:00.0 USB controller [0c03]: Renesas Technology Corp. uPD720201 USB 3.0 Host Controller [1912:0014] (rev 03)
0b:00.0 PCI bridge [0604]: Texas Instruments XIO2213A/B/XIO2221 PCI Express to PCI Bridge [Cheetah Express] [104c:823e] (rev 01) 0c:00.0 FireWire (IEEE 1394) [0c00]: Texas Instruments XIO2213A/B/XIO2221 IEEE-1394b OHCI Controller [Cheetah Express] [104c:823f] (rev 01) 0d:00.0 Ethernet controller [0200]: Intel Corporation I210 Gigabit Network Connection [8086:1533] (rev 03)

#13 lore raw

Re: IOMMU groups ... PEX8606 switch?

Quoted 43 lines · Dana Goyette
Quoted 42 lines · Alex Williamson
Quoted 41 lines · Dana Goyette
Quoted 40 lines · Alex Williamson
Quoted 16 lines · Dana Goyette
Quoted 15 lines · Alex Williamson
Quoted 14 lines · Dana Goyette

I have purchased both a SuperMicro X10SAE and an X10SAT, and I need to soon decide which one to keep.

The SuperMicro X10SAT has all the PCIe x1 slots hidden behind a PLX PEX8066 switch, which claims to support ACS. I'd expect the devices downstream of the PLX switch to be in separate groups.

With Linux 3.13-rc5 and "enable overrides for missing ACS capabilities" applied and set for the Intel root ports, the devices behind the switch remain stuck in the same group.

In terms of passing devices to different VMs, which is better: all devices on different root ports, or all devices behind the one ACS-supporting switch?

Quoted 7 lines
Quoted 4 lines

Can you provide lspci -vvv info? If you're getting that for groups either the switch has ACS capabilities, but doesn't support the features we need or we're doing something wrong. Thanks,

I initially tried attaching the output as a .txt file, but it's too large. Anyway, here's the output of lspci -nnvvv (you may notice that I moved the Radeon to a different slot).

Well, something seems amiss since the downstream switch ports all seem to support and enable the correct set of ACS capabilities. I'm tending to suspect something wrong with the ACS override patch or how it's being used since your IOMMU group is still based at the root port. Each root port is isolated from the other root ports though, so something is happening with the override patch. Can you provide the kernel command line you use to enable ACS overrides and the override patch you're using, as it applies to 3.13-rc5? Thanks,

Alex

--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

I'm using the original acs-override patch from this post:
https://lkml.org/lkml/2013/5/30/513

Kernel parameter is:
pcie_acs_override=id:8086:8c10,id:8086:8c12,id:8086:8c16,id:8086:8c18

Actually, you're not:

pcie_acs_override=id:8086:8c10,id:8086:8c16,id:8086:8c18,id:8086:ac1a,id:8086:8c1c,id:8086:8c1e,id:10b5:8606

And we register all of them:

[    0.000000] PCIe ACS bypass added for 8086:8c10
[    0.000000] PCIe ACS bypass added for 8086:8c16
[    0.000000] PCIe ACS bypass added for 8086:8c18
[    0.000000] PCIe ACS bypass added for 8086:ac1a
[    0.000000] PCIe ACS bypass added for 8086:8c1c
[    0.000000] PCIe ACS bypass added for 8086:8c1e
[    0.000000] PCIe ACS bypass added for 10b5:8606

However, note that the root port causing you trouble is 8086:8c12, which isn't provided as an override, therefore the code is doing the right thing and grouping all devices behind that root port together.

Quoted 561 lines
Quoted 8 lines
Quoted 5 lines

When booting a kernel without the override patch, the following devices are all in the same group: Intel Root Ports 1, 2, 4, 5; ASMedia SATA controller; PLX PEX8606 switch; Renesas USB controller; TI Firewire controller; Intel I210 Ethernet controller.

Could you please try the patch below and send dmesg for the system once booted. This applies directly to upstream and includes the acs override patch. Thanks,

(removed patch from quote.)

Here's the complete dmesg, with pcie_acs_override still set:

http://pastebin.com/YHuKnrTb

Most relevant section:

[    0.524362] DMAR: No ATSR found
[    0.524386] IOMMU 1 0xfed91000: using Queued invalidation
[    0.524389] IOMMU: Setting RMRR:
[    0.524398] IOMMU: Setting identity map for device 0000:00:1d.0 
[0x7bea1000 - 0x7beaffff]
[    0.524423] IOMMU: Setting identity map for device 0000:00:1a.0 
[0x7bea1000 - 0x7beaffff]
[    0.524441] IOMMU: Setting identity map for device 0000:00:14.0 
[0x7bea1000 - 0x7beaffff]
[    0.524454] IOMMU: Prepare 0-16MiB unity mapping for LPC
[    0.524461] IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 
- 0xffffff]
[    0.524548] PCI-DMA: Intel(R) Virtualization Technology for Directed I/O
[    0.524551] intel_iommu_add_device(0000:00:00.0)
[    0.524552] dma_pdev #1: 0000:00:00.0
[    0.524553] dma_pdev #2: 0000:00:00.0
[    0.524554] dma_pdev #3: 0000:00:00.0
[    0.524554] dma_pdev #4: 0000:00:00.0
[    0.524565] intel_iommu_add_device(0000:00:01.0)
[    0.524566] dma_pdev #1: 0000:00:01.0
[    0.524567] dma_pdev #2: 0000:00:01.0
[    0.524569] pci_acs_enabled(0000:00:01.0, 001d)
[    0.524572] pci_acs_flags_enabled no ACS capability on 0000:00:01.0
[    0.524573] pci_acs_flags_enabled(0000:00:01.0, 001d) -> false
[    0.524574] -> false
[    0.524575] pci_acs_enabled(0000:00:01.0, 001d)
[    0.524577] pci_acs_flags_enabled no ACS capability on 0000:00:01.0
[    0.524578] pci_acs_flags_enabled(0000:00:01.0, 001d) -> false
[    0.524579] -> false
[    0.524580] dma_pdev #3: 0000:00:01.0
[    0.524581] dma_pdev #4: 0000:00:01.0
[    0.524584] intel_iommu_add_device(0000:00:01.1)
[    0.524586] dma_pdev #1: 0000:00:01.1
[    0.524586] dma_pdev #2: 0000:00:01.1
[    0.524587] pci_acs_enabled(0000:00:01.1, 001d)
[    0.524589] pci_acs_flags_enabled no ACS capability on 0000:00:01.1
[    0.524590] pci_acs_flags_enabled(0000:00:01.1, 001d) -> false
[    0.524591] -> false
[    0.524592] pci_acs_enabled(0000:00:01.0, 001d)
[    0.524593] pci_acs_flags_enabled no ACS capability on 0000:00:01.0
[    0.524595] pci_acs_flags_enabled(0000:00:01.0, 001d) -> false
[    0.524596] -> false
[    0.524596] dma_pdev #3: 0000:00:01.0
[    0.524597] dma_pdev #4: 0000:00:01.0
[    0.524599] intel_iommu_add_device(0000:00:02.0)
[    0.524601] dma_pdev #1: 0000:00:02.0
[    0.524601] dma_pdev #2: 0000:00:02.0
[    0.524602] dma_pdev #3: 0000:00:02.0
[    0.524603] dma_pdev #4: 0000:00:02.0
[    0.524606] intel_iommu_add_device(0000:00:03.0)
[    0.524607] dma_pdev #1: 0000:00:03.0
[    0.524608] dma_pdev #2: 0000:00:03.0
[    0.524609] dma_pdev #3: 0000:00:03.0
[    0.524610] dma_pdev #4: 0000:00:03.0
[    0.524613] intel_iommu_add_device(0000:00:14.0)
[    0.524614] dma_pdev #1: 0000:00:14.0
[    0.524615] dma_pdev #2: 0000:00:14.0
[    0.524616] dma_pdev #3: 0000:00:14.0
[    0.524617] dma_pdev #4: 0000:00:14.0
[    0.524620] intel_iommu_add_device(0000:00:16.0)
[    0.524621] dma_pdev #1: 0000:00:16.0
[    0.524622] dma_pdev #2: 0000:00:16.0
[    0.524623] pci_acs_enabled(0000:00:16.0, 001d)
[    0.524624] -> false
[    0.524625] pci_acs_enabled(0000:00:16.0, 001d)
[    0.524626] -> false
[    0.524626] dma_pdev #3: 0000:00:16.0
[    0.524627] dma_pdev #4: 0000:00:16.0
[    0.524630] intel_iommu_add_device(0000:00:16.3)
[    0.524631] dma_pdev #1: 0000:00:16.3
[    0.524632] dma_pdev #2: 0000:00:16.3
[    0.524633] pci_acs_enabled(0000:00:16.3, 001d)
[    0.524634] -> false
[    0.524635] pci_acs_enabled(0000:00:16.0, 001d)
[    0.524636] -> false
[    0.524637] dma_pdev #3: 0000:00:16.0
[    0.524638] dma_pdev #4: 0000:00:16.0
[    0.524640] intel_iommu_add_device(0000:00:19.0)
[    0.524641] dma_pdev #1: 0000:00:19.0
[    0.524642] dma_pdev #2: 0000:00:19.0
[    0.524643] dma_pdev #3: 0000:00:19.0
[    0.524643] dma_pdev #4: 0000:00:19.0
[    0.524646] intel_iommu_add_device(0000:00:1a.0)
[    0.524647] dma_pdev #1: 0000:00:1a.0
[    0.524648] dma_pdev #2: 0000:00:1a.0
[    0.524649] dma_pdev #3: 0000:00:1a.0
[    0.524650] dma_pdev #4: 0000:00:1a.0
[    0.524654] intel_iommu_add_device(0000:00:1b.0)
[    0.524655] dma_pdev #1: 0000:00:1b.0
[    0.524656] dma_pdev #2: 0000:00:1b.0
[    0.524657] dma_pdev #3: 0000:00:1b.0
[    0.524658] dma_pdev #4: 0000:00:1b.0
[    0.524661] intel_iommu_add_device(0000:00:1c.0)
[    0.524662] dma_pdev #1: 0000:00:1c.0
[    0.524663] dma_pdev #2: 0000:00:1c.0
[    0.524664] pci_acs_enabled(0000:00:1c.0, 001d)
[    0.524665] PCIe ACS override match for 0000:00:1c.0
[    0.524666] -> true
[    0.524667] dma_pdev #3: 0000:00:1c.0
[    0.524668] dma_pdev #4: 0000:00:1c.0
[    0.524671] intel_iommu_add_device(0000:00:1c.1)
[    0.524672] dma_pdev #1: 0000:00:1c.1
[    0.524673] dma_pdev #2: 0000:00:1c.1
[    0.524674] pci_acs_enabled(0000:00:1c.1, 001d)
[    0.524681] pci_acs_flags_enabled no ACS capability on 0000:00:1c.1
[    0.524682] pci_acs_flags_enabled(0000:00:1c.1, 001d) -> false
[    0.524683] -> false
[    0.524684] pci_acs_enabled(0000:00:1c.0, 001d)
[    0.524685] PCIe ACS override match for 0000:00:1c.0
[    0.524686] -> true
[    0.524687] pci_acs_enabled(0000:00:1c.1, 001d)
[    0.524694] pci_acs_flags_enabled no ACS capability on 0000:00:1c.1
[    0.524695] pci_acs_flags_enabled(0000:00:1c.1, 001d) -> false
[    0.524696] -> false
[    0.524697] dma_pdev #3: 0000:00:1c.1
[    0.524698] dma_pdev #4: 0000:00:1c.1
[    0.524700] intel_iommu_add_device(0000:00:1c.3)
[    0.524701] dma_pdev #1: 0000:00:1c.3
[    0.524702] dma_pdev #2: 0000:00:1c.3
[    0.524703] pci_acs_enabled(0000:00:1c.3, 001d)
[    0.524704] PCIe ACS override match for 0000:00:1c.3
[    0.524705] -> true
[    0.524706] dma_pdev #3: 0000:00:1c.3
[    0.524707] dma_pdev #4: 0000:00:1c.3
[    0.524710] intel_iommu_add_device(0000:00:1c.4)
[    0.524711] dma_pdev #1: 0000:00:1c.4
[    0.524712] dma_pdev #2: 0000:00:1c.4
[    0.524713] pci_acs_enabled(0000:00:1c.4, 001d)
[    0.524714] PCIe ACS override match for 0000:00:1c.4
[    0.524715] -> true
[    0.524715] dma_pdev #3: 0000:00:1c.4
[    0.524716] dma_pdev #4: 0000:00:1c.4
[    0.524719] intel_iommu_add_device(0000:00:1d.0)
[    0.524720] dma_pdev #1: 0000:00:1d.0
[    0.524721] dma_pdev #2: 0000:00:1d.0
[    0.524722] dma_pdev #3: 0000:00:1d.0
[    0.524723] dma_pdev #4: 0000:00:1d.0
[    0.524726] intel_iommu_add_device(0000:00:1f.0)
[    0.524727] dma_pdev #1: 0000:00:1f.0
[    0.524728] dma_pdev #2: 0000:00:1f.0
[    0.524728] pci_acs_enabled(0000:00:1f.0, 001d)
[    0.524729] -> false
[    0.524730] pci_acs_enabled(0000:00:1f.0, 001d)
[    0.524731] -> false
[    0.524732] dma_pdev #3: 0000:00:1f.0
[    0.524733] dma_pdev #4: 0000:00:1f.0
[    0.524736] intel_iommu_add_device(0000:00:1f.2)
[    0.524737] dma_pdev #1: 0000:00:1f.2
[    0.524738] dma_pdev #2: 0000:00:1f.2
[    0.524739] pci_acs_enabled(0000:00:1f.2, 001d)
[    0.524740] -> false
[    0.524741] pci_acs_enabled(0000:00:1f.0, 001d)
[    0.524741] -> false
[    0.524742] dma_pdev #3: 0000:00:1f.0
[    0.524743] dma_pdev #4: 0000:00:1f.0
[    0.524745] intel_iommu_add_device(0000:00:1f.3)
[    0.524746] dma_pdev #1: 0000:00:1f.3
[    0.524747] dma_pdev #2: 0000:00:1f.3
[    0.524748] pci_acs_enabled(0000:00:1f.3, 001d)
[    0.524749] -> false
[    0.524750] pci_acs_enabled(0000:00:1f.0, 001d)
[    0.524751] -> false
[    0.524752] dma_pdev #3: 0000:00:1f.0
[    0.524752] dma_pdev #4: 0000:00:1f.0
[    0.524754] intel_iommu_add_device(0000:00:1f.6)
[    0.524755] dma_pdev #1: 0000:00:1f.6
[    0.524756] dma_pdev #2: 0000:00:1f.6
[    0.524757] pci_acs_enabled(0000:00:1f.6, 001d)
[    0.524758] -> false
[    0.524759] pci_acs_enabled(0000:00:1f.0, 001d)
[    0.524760] -> false
[    0.524761] dma_pdev #3: 0000:00:1f.0
[    0.524762] dma_pdev #4: 0000:00:1f.0
[    0.524765] intel_iommu_add_device(0000:01:00.0)
[    0.524766] dma_pdev #1: 0000:01:00.0
[    0.524767] dma_pdev #2: 0000:01:00.0
[    0.524768] pci_acs_enabled(0000:01:00.0, 001d)
[    0.524771] pci_acs_flags_enabled no ACS capability on 0000:01:00.0
[    0.524772] pci_acs_flags_enabled(0000:01:00.0, 001d) -> false
[    0.524773] -> false
[    0.524774] pci_acs_enabled(0000:01:00.0, 001d)
[    0.524776] pci_acs_flags_enabled no ACS capability on 0000:01:00.0
[    0.524777] pci_acs_flags_enabled(0000:01:00.0, 001d) -> false
[    0.524778] -> false
[    0.524779] dma_pdev #3: 0000:01:00.0
[    0.524780] pci_acs_path_enabled(0000:00:01.0, NULL, 001d)
[    0.524781] pci_acs_enabled(0000:00:01.0, 001d)
[    0.524783] pci_acs_flags_enabled no ACS capability on 0000:00:01.0
[    0.524784] pci_acs_flags_enabled(0000:00:01.0, 001d) -> false
[    0.524785] -> false
[    0.524786] -> false
[    0.524787] dma_pdev #4: 0000:00:01.0
[    0.524789] intel_iommu_add_device(0000:01:00.1)
[    0.524790] dma_pdev #1: 0000:01:00.1
[    0.524791] dma_pdev #2: 0000:01:00.1
[    0.524792] pci_acs_enabled(0000:01:00.1, 001d)
[    0.524794] pci_acs_flags_enabled no ACS capability on 0000:01:00.1
[    0.524795] pci_acs_flags_enabled(0000:01:00.1, 001d) -> false
[    0.524796] -> false
[    0.524797] pci_acs_enabled(0000:01:00.0, 001d)
[    0.524799] pci_acs_flags_enabled no ACS capability on 0000:01:00.0
[    0.524800] pci_acs_flags_enabled(0000:01:00.0, 001d) -> false
[    0.524801] -> false
[    0.524802] dma_pdev #3: 0000:01:00.0
[    0.524803] pci_acs_path_enabled(0000:00:01.0, NULL, 001d)
[    0.524804] pci_acs_enabled(0000:00:01.0, 001d)
[    0.524806] pci_acs_flags_enabled no ACS capability on 0000:00:01.0
[    0.524807] pci_acs_flags_enabled(0000:00:01.0, 001d) -> false
[    0.524808] -> false
[    0.524809] -> false
[    0.524809] dma_pdev #4: 0000:00:01.0
[    0.524811] intel_iommu_add_device(0000:02:00.0)
[    0.524813] dma_pdev #1: 0000:02:00.0
[    0.524813] dma_pdev #2: 0000:02:00.0
[    0.524814] dma_pdev #3: 0000:02:00.0
[    0.524815] pci_acs_path_enabled(0000:00:01.1, NULL, 001d)
[    0.524816] pci_acs_enabled(0000:00:01.1, 001d)
[    0.524818] pci_acs_flags_enabled no ACS capability on 0000:00:01.1
[    0.524819] pci_acs_flags_enabled(0000:00:01.1, 001d) -> false
[    0.524820] -> false
[    0.524821] -> false
[    0.524822] dma_pdev #4: 0000:00:01.1
[    0.524824] intel_iommu_add_device(0000:03:04.0)
[    0.524825] Upstream bridge 0000:02:00.0
[    0.524830] dma_pdev #1: 0000:02:00.0
[    0.524831] dma_pdev #2: 0000:02:00.0
[    0.524832] dma_pdev #3: 0000:02:00.0
[    0.524833] pci_acs_path_enabled(0000:00:01.1, NULL, 001d)
[    0.524834] pci_acs_enabled(0000:00:01.1, 001d)
[    0.524835] pci_acs_flags_enabled no ACS capability on 0000:00:01.1
[    0.524837] pci_acs_flags_enabled(0000:00:01.1, 001d) -> false
[    0.524837] -> false
[    0.524838] -> false
[    0.524839] dma_pdev #4: 0000:00:01.1
[    0.524841] intel_iommu_add_device(0000:04:00.0)
[    0.524842] dma_pdev #1: 0000:04:00.0
[    0.524843] dma_pdev #2: 0000:04:00.0
[    0.524844] dma_pdev #3: 0000:04:00.0
[    0.524845] pci_acs_path_enabled(0000:00:1c.0, NULL, 001d)
[    0.524846] pci_acs_enabled(0000:00:1c.0, 001d)
[    0.524847] PCIe ACS override match for 0000:00:1c.0
[    0.524848] -> true
[    0.524849] -> true
[    0.524849] dma_pdev #4: 0000:04:00.0
[    0.524852] intel_iommu_add_device(0000:05:00.0)
[    0.524853] dma_pdev #1: 0000:05:00.0
[    0.524854] dma_pdev #2: 0000:05:00.0
[    0.524855] dma_pdev #3: 0000:05:00.0
[    0.524856] pci_acs_path_enabled(0000:00:1c.1, NULL, 001d)
[    0.524857] pci_acs_enabled(0000:00:1c.1, 001d)
[    0.524865] pci_acs_flags_enabled no ACS capability on 0000:00:1c.1
[    0.524866] pci_acs_flags_enabled(0000:00:1c.1, 001d) -> false
[    0.524867] -> false
[    0.524868] -> false
[    0.524869] dma_pdev #4: 0000:00:1c.1
[    0.524871] intel_iommu_add_device(0000:06:01.0)
[    0.524872] dma_pdev #1: 0000:06:01.0
[    0.524872] dma_pdev #2: 0000:06:01.0
[    0.524873] dma_pdev #3: 0000:06:01.0
[    0.524874] pci_acs_path_enabled(0000:05:00.0, NULL, 001d)
[    0.524875] pci_acs_enabled(0000:05:00.0, 001d)
[    0.524876] PCIe ACS override match for 0000:05:00.0
[    0.524877] -> true
[    0.524878] pci_acs_enabled(0000:00:1c.1, 001d)
[    0.524886] pci_acs_flags_enabled no ACS capability on 0000:00:1c.1
[    0.524887] pci_acs_flags_enabled(0000:00:1c.1, 001d) -> false
[    0.524888] -> false
[    0.524888] -> false
[    0.524889] pci_acs_path_enabled(0000:00:1c.1, NULL, 001d)
[    0.524891] pci_acs_enabled(0000:00:1c.1, 001d)
[    0.524898] pci_acs_flags_enabled no ACS capability on 0000:00:1c.1
[    0.524899] pci_acs_flags_enabled(0000:00:1c.1, 001d) -> false
[    0.524900] -> false
[    0.524901] -> false
[    0.524901] dma_pdev #4: 0000:00:1c.1
[    0.524903] intel_iommu_add_device(0000:06:04.0)
[    0.524904] dma_pdev #1: 0000:06:04.0
[    0.524905] dma_pdev #2: 0000:06:04.0
[    0.524906] dma_pdev #3: 0000:06:04.0
[    0.524907] pci_acs_path_enabled(0000:05:00.0, NULL, 001d)
[    0.524908] pci_acs_enabled(0000:05:00.0, 001d)
[    0.524909] PCIe ACS override match for 0000:05:00.0
[    0.524910] -> true
[    0.524911] pci_acs_enabled(0000:00:1c.1, 001d)
[    0.524913] pci_acs_flags_enabled no ACS capability on 0000:00:1c.1
[    0.524914] pci_acs_flags_enabled(0000:00:1c.1, 001d) -> false
[    0.524915] -> false
[    0.524916] -> false
[    0.524917] pci_acs_path_enabled(0000:00:1c.1, NULL, 001d)
[    0.524918] pci_acs_enabled(0000:00:1c.1, 001d)
[    0.524925] pci_acs_flags_enabled no ACS capability on 0000:00:1c.1
[    0.524926] pci_acs_flags_enabled(0000:00:1c.1, 001d) -> false
[    0.524927] -> false
[    0.524928] -> false
[    0.524929] dma_pdev #4: 0000:00:1c.1
[    0.524949] intel_iommu_add_device(0000:06:05.0)
[    0.524951] dma_pdev #1: 0000:06:05.0
[    0.524951] dma_pdev #2: 0000:06:05.0
[    0.524952] dma_pdev #3: 0000:06:05.0
[    0.524953] pci_acs_path_enabled(0000:05:00.0, NULL, 001d)
[    0.524954] pci_acs_enabled(0000:05:00.0, 001d)
[    0.524955] PCIe ACS override match for 0000:05:00.0
[    0.524957] -> true
[    0.524958] pci_acs_enabled(0000:00:1c.1, 001d)
[    0.524964] pci_acs_flags_enabled no ACS capability on 0000:00:1c.1
[    0.524965] pci_acs_flags_enabled(0000:00:1c.1, 001d) -> false
[    0.524966] -> false
[    0.524967] -> false
[    0.524968] pci_acs_path_enabled(0000:00:1c.1, NULL, 001d)
[    0.524969] pci_acs_enabled(0000:00:1c.1, 001d)
[    0.524976] pci_acs_flags_enabled no ACS capability on 0000:00:1c.1
[    0.524977] pci_acs_flags_enabled(0000:00:1c.1, 001d) -> false
[    0.524978] -> false
[    0.524979] -> false
[    0.524979] dma_pdev #4: 0000:00:1c.1
[    0.524983] intel_iommu_add_device(0000:06:07.0)
[    0.524984] dma_pdev #1: 0000:06:07.0
[    0.524985] dma_pdev #2: 0000:06:07.0
[    0.524986] dma_pdev #3: 0000:06:07.0
[    0.524987] pci_acs_path_enabled(0000:05:00.0, NULL, 001d)
[    0.524988] pci_acs_enabled(0000:05:00.0, 001d)
[    0.524989] PCIe ACS override match for 0000:05:00.0
[    0.524990] -> true
[    0.524991] pci_acs_enabled(0000:00:1c.1, 001d)
[    0.524997] pci_acs_flags_enabled no ACS capability on 0000:00:1c.1
[    0.524998] pci_acs_flags_enabled(0000:00:1c.1, 001d) -> false
[    0.524999] -> false
[    0.525000] -> false
[    0.525001] pci_acs_path_enabled(0000:00:1c.1, NULL, 001d)
[    0.525002] pci_acs_enabled(0000:00:1c.1, 001d)
[    0.525004] pci_acs_flags_enabled no ACS capability on 0000:00:1c.1
[    0.525006] pci_acs_flags_enabled(0000:00:1c.1, 001d) -> false
[    0.525007] -> false
[    0.525007] -> false
[    0.525008] dma_pdev #4: 0000:00:1c.1
[    0.525010] intel_iommu_add_device(0000:06:09.0)
[    0.525011] dma_pdev #1: 0000:06:09.0
[    0.525012] dma_pdev #2: 0000:06:09.0
[    0.525013] dma_pdev #3: 0000:06:09.0
[    0.525014] pci_acs_path_enabled(0000:05:00.0, NULL, 001d)
[    0.525015] pci_acs_enabled(0000:05:00.0, 001d)
[    0.525016] PCIe ACS override match for 0000:05:00.0
[    0.525017] -> true
[    0.525018] pci_acs_enabled(0000:00:1c.1, 001d)
[    0.525025] pci_acs_flags_enabled no ACS capability on 0000:00:1c.1
[    0.525026] pci_acs_flags_enabled(0000:00:1c.1, 001d) -> false
[    0.525027] -> false
[    0.525028] -> false
[    0.525029] pci_acs_path_enabled(0000:00:1c.1, NULL, 001d)
[    0.525030] pci_acs_enabled(0000:00:1c.1, 001d)
[    0.525037] pci_acs_flags_enabled no ACS capability on 0000:00:1c.1
[    0.525038] pci_acs_flags_enabled(0000:00:1c.1, 001d) -> false
[    0.525039] -> false
[    0.525040] -> false
[    0.525041] dma_pdev #4: 0000:00:1c.1
[    0.525043] intel_iommu_add_device(0000:0a:00.0)
[    0.525044] dma_pdev #1: 0000:0a:00.0
[    0.525044] dma_pdev #2: 0000:0a:00.0
[    0.525045] dma_pdev #3: 0000:0a:00.0
[    0.525046] pci_acs_path_enabled(0000:06:07.0, NULL, 001d)
[    0.525048] pci_acs_enabled(0000:06:07.0, 001d)
[    0.525048] PCIe ACS override match for 0000:06:07.0
[    0.525049] -> true
[    0.525050] pci_acs_enabled(0000:05:00.0, 001d)
[    0.525051] PCIe ACS override match for 0000:05:00.0
[    0.525052] -> true
[    0.525053] pci_acs_enabled(0000:00:1c.1, 001d)
[    0.525060] pci_acs_flags_enabled no ACS capability on 0000:00:1c.1
[    0.525062] pci_acs_flags_enabled(0000:00:1c.1, 001d) -> false
[    0.525062] -> false
[    0.525063] -> false
[    0.525064] pci_acs_path_enabled(0000:05:00.0, NULL, 001d)
[    0.525065] pci_acs_enabled(0000:05:00.0, 001d)
[    0.525066] PCIe ACS override match for 0000:05:00.0
[    0.525067] -> true
[    0.525068] pci_acs_enabled(0000:00:1c.1, 001d)
[    0.525075] pci_acs_flags_enabled no ACS capability on 0000:00:1c.1
[    0.525076] pci_acs_flags_enabled(0000:00:1c.1, 001d) -> false
[    0.525077] -> false
[    0.525078] -> false
[    0.525079] pci_acs_path_enabled(0000:00:1c.1, NULL, 001d)
[    0.525080] pci_acs_enabled(0000:00:1c.1, 001d)
[    0.525087] pci_acs_flags_enabled no ACS capability on 0000:00:1c.1
[    0.525088] pci_acs_flags_enabled(0000:00:1c.1, 001d) -> false
[    0.525089] -> false
[    0.525090] -> false
[    0.525091] dma_pdev #4: 0000:00:1c.1
[    0.525093] intel_iommu_add_device(0000:0b:00.0)
[    0.525094] dma_pdev #1: 0000:0b:00.0
[    0.525095] dma_pdev #2: 0000:0b:00.0
[    0.525096] dma_pdev #3: 0000:0b:00.0
[    0.525097] pci_acs_path_enabled(0000:06:09.0, NULL, 001d)
[    0.525098] pci_acs_enabled(0000:06:09.0, 001d)
[    0.525099] PCIe ACS override match for 0000:06:09.0
[    0.525100] -> true
[    0.525101] pci_acs_enabled(0000:05:00.0, 001d)
[    0.525102] PCIe ACS override match for 0000:05:00.0
[    0.525103] -> true
[    0.525103] pci_acs_enabled(0000:00:1c.1, 001d)
[    0.525111] pci_acs_flags_enabled no ACS capability on 0000:00:1c.1
[    0.525112] pci_acs_flags_enabled(0000:00:1c.1, 001d) -> false
[    0.525113] -> false
[    0.525114] -> false
[    0.525115] pci_acs_path_enabled(0000:05:00.0, NULL, 001d)
[    0.525116] pci_acs_enabled(0000:05:00.0, 001d)
[    0.525117] PCIe ACS override match for 0000:05:00.0
[    0.525118] -> true
[    0.525118] pci_acs_enabled(0000:00:1c.1, 001d)
[    0.525125] pci_acs_flags_enabled no ACS capability on 0000:00:1c.1
[    0.525126] pci_acs_flags_enabled(0000:00:1c.1, 001d) -> false
[    0.525127] -> false
[    0.525128] -> false
[    0.525129] pci_acs_path_enabled(0000:00:1c.1, NULL, 001d)
[    0.525130] pci_acs_enabled(0000:00:1c.1, 001d)
[    0.525137] pci_acs_flags_enabled no ACS capability on 0000:00:1c.1
[    0.525138] pci_acs_flags_enabled(0000:00:1c.1, 001d) -> false
[    0.525139] -> false
[    0.525140] -> false
[    0.525141] dma_pdev #4: 0000:00:1c.1
[    0.525143] intel_iommu_add_device(0000:0c:00.0)
[    0.525144] Upstream bridge 0000:0b:00.0
[    0.525148] dma_pdev #1: 0000:0c:00.0
[    0.525149] dma_pdev #2: 0000:0c:00.0
[    0.525150] dma_pdev #3: 0000:0c:00.0
[    0.525151] pci_acs_path_enabled(0000:0b:00.0, NULL, 001d)
[    0.525152] pci_acs_enabled(0000:0b:00.0, 001d)
[    0.525153] -> false
[    0.525154] -> false
[    0.525155] pci_acs_path_enabled(0000:06:09.0, NULL, 001d)
[    0.525156] pci_acs_enabled(0000:06:09.0, 001d)
[    0.525157] PCIe ACS override match for 0000:06:09.0
[    0.525158] -> true
[    0.525158] pci_acs_enabled(0000:05:00.0, 001d)
[    0.525159] PCIe ACS override match for 0000:05:00.0
[    0.525160] -> true
[    0.525161] pci_acs_enabled(0000:00:1c.1, 001d)
[    0.525168] pci_acs_flags_enabled no ACS capability on 0000:00:1c.1
[    0.525169] pci_acs_flags_enabled(0000:00:1c.1, 001d) -> false
[    0.525170] -> false
[    0.525171] -> false
[    0.525172] pci_acs_path_enabled(0000:05:00.0, NULL, 001d)
[    0.525173] pci_acs_enabled(0000:05:00.0, 001d)
[    0.525174] PCIe ACS override match for 0000:05:00.0
[    0.525175] -> true
[    0.525175] pci_acs_enabled(0000:00:1c.1, 001d)
[    0.525178] pci_acs_flags_enabled no ACS capability on 0000:00:1c.1
[    0.525179] pci_acs_flags_enabled(0000:00:1c.1, 001d) -> false
[    0.525180] -> false
[    0.525181] -> false
[    0.525182] pci_acs_path_enabled(0000:00:1c.1, NULL, 001d)
[    0.525183] pci_acs_enabled(0000:00:1c.1, 001d)
[    0.525190] pci_acs_flags_enabled no ACS capability on 0000:00:1c.1
[    0.525191] pci_acs_flags_enabled(0000:00:1c.1, 001d) -> false
[    0.525192] -> false
[    0.525193] -> false
[    0.525193] dma_pdev #4: 0000:00:1c.1
[    0.525195] intel_iommu_add_device(0000:0d:00.0)
[    0.525196] dma_pdev #1: 0000:0d:00.0
[    0.525197] dma_pdev #2: 0000:0d:00.0
[    0.525198] dma_pdev #3: 0000:0d:00.0
[    0.525199] pci_acs_path_enabled(0000:00:1c.3, NULL, 001d)
[    0.525200] pci_acs_enabled(0000:00:1c.3, 001d)
[    0.525201] PCIe ACS override match for 0000:00:1c.3
[    0.525202] -> true
[    0.525203] -> true
[    0.525204] dma_pdev #4: 0000:0d:00.0

lspci as slots are now (since Radeon is back in first x16 slot):

00:00.0 Host bridge [0600]: Intel Corporation Xeon E3-1200 v3 Processor DRAM Controller [8086:0c08] (rev 06)
00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x16 Controller [8086:0c01] (rev 06) 00:01.1 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x8 Controller [8086:0c05] (rev 06) 00:02.0 VGA compatible controller [0300]: Intel Corporation Xeon E3-1200 v3 Processor Integrated Graphics Controller [8086:041a] (rev 06) 00:03.0 Audio device [0403]: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor HD Audio Controller [8086:0c0c] (rev 06) 00:14.0 USB controller [0c03]: Intel Corporation 8 Series/C220 Series Chipset Family USB xHCI [8086:8c31] (rev 05)
00:16.0 Communication controller [0780]: Intel Corporation 8 Series/C220 Series Chipset Family MEI Controller #1 [8086:8c3a] (rev 04) 00:16.3 Serial controller [0700]: Intel Corporation 8 Series/C220 Series Chipset Family KT Controller [8086:8c3d] (rev 04)
00:19.0 Ethernet controller [0200]: Intel Corporation Ethernet Connection I217-LM [8086:153a] (rev 05)
00:1b.0 Audio device [0403]: Intel Corporation 8 Series/C220 Series Chipset High Definition Audio Controller [8086:8c20] (rev 05) 00:1c.0 PCI bridge [0604]: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #1 [8086:8c10] (rev d5) 00:1c.1 PCI bridge [0604]: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #2 [8086:8c12] (rev d5) 00:1c.3 PCI bridge [0604]: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #4 [8086:8c16] (rev d5) 00:1c.4 PCI bridge [0604]: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #5 [8086:8c18] (rev d5) 00:1d.0 USB controller [0c03]: Intel Corporation 8 Series/C220 Series Chipset Family USB EHCI #1 [8086:8c26] (rev 05)
00:1f.0 ISA bridge [0601]: Intel Corporation C226 Series Chipset Family Server Advanced SKU LPC Controller [8086:8c56] (rev 05) 00:1f.2 SATA controller [0106]: Intel Corporation 8 Series/C220 Series Chipset Family 6-port SATA Controller 1 [AHCI mode] [8086:8c02] (rev 05) 00:1f.3 SMBus [0c05]: Intel Corporation 8 Series/C220 Series Chipset Family SMBus Controller [8086:8c22] (rev 05)
00:1f.6 Signal processing controller [1180]: Intel Corporation 8 Series Chipset Family Thermal Management Controller [8086:8c24] (rev 05) 01:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Juniper PRO [Radeon HD 5750] [1002:68be]
01:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Juniper HDMI Audio [Radeon HD 5700 Series] [1002:aa58] 02:00.0 PCI bridge [0604]: PLX Technology, Inc. PEX8112 x1 Lane PCI Express-to-PCI Bridge [10b5:8112] (rev aa)
03:04.0 Multimedia audio controller [0401]: C-Media Electronics Inc CMI8788 [Oxygen HD Audio] [13f6:8788]
04:00.0 SATA controller [0106]: ASMedia Technology Inc. ASM1062 Serial ATA Controller [1b21:0612] (rev 01)
05:00.0 PCI bridge [0604]: PLX Technology, Inc. PEX 8606 6 Lane, 6 Port PCI Express Gen 2 (5.0 GT/s) Switch [10b5:8606] (rev ba) 06:01.0 PCI bridge [0604]: PLX Technology, Inc. PEX 8606 6 Lane, 6 Port PCI Express Gen 2 (5.0 GT/s) Switch [10b5:8606] (rev ba) 06:04.0 PCI bridge [0604]: PLX Technology, Inc. PEX 8606 6 Lane, 6 Port PCI Express Gen 2 (5.0 GT/s) Switch [10b5:8606] (rev ba) 06:05.0 PCI bridge [0604]: PLX Technology, Inc. PEX 8606 6 Lane, 6 Port PCI Express Gen 2 (5.0 GT/s) Switch [10b5:8606] (rev ba) 06:07.0 PCI bridge [0604]: PLX Technology, Inc. PEX 8606 6 Lane, 6 Port PCI Express Gen 2 (5.0 GT/s) Switch [10b5:8606] (rev ba) 06:09.0 PCI bridge [0604]: PLX Technology, Inc. PEX 8606 6 Lane, 6 Port PCI Express Gen 2 (5.0 GT/s) Switch [10b5:8606] (rev ba) 0a:00.0 USB controller [0c03]: Renesas Technology Corp. uPD720201 USB 3.0 Host Controller [1912:0014] (rev 03)
0b:00.0 PCI bridge [0604]: Texas Instruments XIO2213A/B/XIO2221 PCI Express to PCI Bridge [Cheetah Express] [104c:823e] (rev 01) 0c:00.0 FireWire (IEEE 1394) [0c00]: Texas Instruments XIO2213A/B/XIO2221 IEEE-1394b OHCI Controller [Cheetah Express] [104c:823f] (rev 01)
0d:00.0 Ethernet controller [0200]: Intel Corporation I210 Gigabit Network Connection [8086:1533] (rev 03)

--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#14 lore raw

Re: IOMMU groups ... PEX8606 switch?

Quoted 64 lines · Alex Williamson
Quoted 43 lines · Dana Goyette
Quoted 42 lines · Alex Williamson
Quoted 41 lines · Dana Goyette
Quoted 40 lines · Alex Williamson
Quoted 16 lines · Dana Goyette
Quoted 15 lines · Alex Williamson
Quoted 14 lines · Dana Goyette

I have purchased both a SuperMicro X10SAE and an X10SAT, and I need to soon decide which one to keep.

The SuperMicro X10SAT has all the PCIe x1 slots hidden behind a PLX PEX8066 switch, which claims to support ACS. I'd expect the devices downstream of the PLX switch to be in separate groups.

With Linux 3.13-rc5 and "enable overrides for missing ACS capabilities" applied and set for the Intel root ports, the devices behind the switch remain stuck in the same group.

In terms of passing devices to different VMs, which is better: all devices on different root ports, or all devices behind the one ACS-supporting switch?

Quoted 7 lines
Quoted 4 lines

Can you provide lspci -vvv info? If you're getting that for groups either the switch has ACS capabilities, but doesn't support the features we need or we're doing something wrong. Thanks,

I initially tried attaching the output as a .txt file, but it's too large. Anyway, here's the output of lspci -nnvvv (you may notice that I moved the Radeon to a different slot).

Well, something seems amiss since the downstream switch ports all seem to support and enable the correct set of ACS capabilities. I'm tending to suspect something wrong with the ACS override patch or how it's being used since your IOMMU group is still based at the root port. Each root port is isolated from the other root ports though, so something is happening with the override patch. Can you provide the kernel command line you use to enable ACS overrides and the override patch you're using, as it applies to 3.13-rc5? Thanks,

Alex

--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

I'm using the original acs-override patch from this post:
https://lkml.org/lkml/2013/5/30/513

Kernel parameter is:
pcie_acs_override=id:8086:8c10,id:8086:8c12,id:8086:8c16,id:8086:8c18

Actually, you're not:

pcie_acs_override=id:8086:8c10,id:8086:8c16,id:8086:8c18,id:8086:ac1a,id:8086:8c1c,id:8086:8c1e,id:10b5:8606

And we register all of them:

[    0.000000] PCIe ACS bypass added for 8086:8c10
[    0.000000] PCIe ACS bypass added for 8086:8c16
[    0.000000] PCIe ACS bypass added for 8086:8c18
[    0.000000] PCIe ACS bypass added for 8086:ac1a
[    0.000000] PCIe ACS bypass added for 8086:8c1c
[    0.000000] PCIe ACS bypass added for 8086:8c1e
[    0.000000] PCIe ACS bypass added for 10b5:8606
Quoted 4 lines

However, note that the root port causing you trouble is 8086:8c12, which isn't provided as an override, therefore the code is doing the right thing and grouping all devices behind that root port together.

Thanks for catching that -- I certainly missed it!
I've added the override for that root port and removed the override for the PLX switch; now all the ports are indeed in separate groups.

Do we yet know if it'll be possible to properly isolate the Intel root ports, without this ACS override?

#15 lore raw

Re: IOMMU groups ... PEX8606 switch?

In reply to Alex Williamson

(Re-sending to gmane.linux.kernel.pci so the solution ends up there, as
well.)

Quoted 64 lines · Alex Williamson
Quoted 43 lines · Dana Goyette
Quoted 42 lines · Alex Williamson
Quoted 41 lines · Dana Goyette
Quoted 40 lines · Alex Williamson
Quoted 16 lines · Dana Goyette
Quoted 15 lines · Alex Williamson
Quoted 14 lines · Dana Goyette

I have purchased both a SuperMicro X10SAE and an X10SAT, and I need to soon decide which one to keep.

The SuperMicro X10SAT has all the PCIe x1 slots hidden behind a PLX PEX8066 switch, which claims to support ACS. I'd expect the devices downstream of the PLX switch to be in separate groups.

With Linux 3.13-rc5 and "enable overrides for missing ACS capabilities" applied and set for the Intel root ports, the devices behind the switch remain stuck in the same group.

In terms of passing devices to different VMs, which is better: all devices on different root ports, or all devices behind the one ACS-supporting switch?

Quoted 7 lines
Quoted 4 lines

Can you provide lspci -vvv info? If you're getting that for groups either the switch has ACS capabilities, but doesn't support the features we need or we're doing something wrong. Thanks,

I initially tried attaching the output as a .txt file, but it's too large. Anyway, here's the output of lspci -nnvvv (you may notice that I moved the Radeon to a different slot).

Well, something seems amiss since the downstream switch ports all seem to support and enable the correct set of ACS capabilities. I'm tending to suspect something wrong with the ACS override patch or how it's being used since your IOMMU group is still based at the root port. Each root port is isolated from the other root ports though, so something is happening with the override patch. Can you provide the kernel command line you use to enable ACS overrides and the override patch you're using, as it applies to 3.13-rc5? Thanks,

Alex

--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

I'm using the original acs-override patch from this post:
https://lkml.org/lkml/2013/5/30/513

Kernel parameter is:
pcie_acs_override=id:8086:8c10,id:8086:8c12,id:8086:8c16,id:8086:8c18

Actually, you're not:

pcie_acs_override=id:8086:8c10,id:8086:8c16,id:8086:8c18,id:8086:ac1a,id:8086:8c1c,id:8086:8c1e,id:10b5:8606

And we register all of them:

[    0.000000] PCIe ACS bypass added for 8086:8c10
[    0.000000] PCIe ACS bypass added for 8086:8c16
[    0.000000] PCIe ACS bypass added for 8086:8c18
[    0.000000] PCIe ACS bypass added for 8086:ac1a
[    0.000000] PCIe ACS bypass added for 8086:8c1c
[    0.000000] PCIe ACS bypass added for 8086:8c1e
[    0.000000] PCIe ACS bypass added for 10b5:8606
Quoted 4 lines

However, note that the root port causing you trouble is 8086:8c12, which isn't provided as an override, therefore the code is doing the right thing and grouping all devices behind that root port together.

Thanks for catching that -- I certainly missed it!
I've added the override for that root port and removed the override for the PLX switch; now all the ports are indeed in separate groups.

Do we yet know if it'll be possible to properly isolate the Intel root ports, without this ACS override?

--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html