Dell Vostro 3550: pci_hotplug+acpiphp require 'pcie_aspm=force' on kernel command-line for hotplug to work
Hi,
I am following up on a former thread
Re: 3.2.11: PCI Express card cannot be re-detected withing cca 60sec timeframe about the same issue. I think I found some new info while playing with 3.7.1 kernel. It happened to me that my hotplug of express cards stopped working so it made me to to dive in a figure out what driver did I do to my .config, and what combinations of drivers and kernel command-line parameters work and which not. This email will cover just one case.
On this Dell Vostro 3550 express card slot works if kernel is without pciehp altogether and pci_hotplug+acpiphp are loaded as modules later on. The problem is that I must use pcie_aspm=off. Below is hopefully an answer why. I can provide dmesg/interrupts/config.gz/lsusb/lspci for each combination I tried. Here I have pasted just bits which I found only directly related. Please bear in mind previously it seemed that the hardware has broken PresDet hadling and therefore I faced 'Surprise removal' meesages from pciehp. This email is however about acpiphp and you can see that actually PresDet flag contains correct values. But I want to avoid here showing that pciehp has a bug. That will come in another thread. ;-)
The kernel binary has no pciehp at all but pci_hotplug and acpiphp are available as modules. I load them manually after saving dmesg/lspci/lsusb/interrupts/.config into a separate directory. I then poke through the collected data and run diff. I did everything under two different BIOS configs. I suspected the media card reader somehow interferes with the express slot card detection because it was sometime detected on the ehci but when I (un)plugged the express card into its slot. Am not sure yet, but FYI what is avalable for you off-the-list ;-)
# du -sh MediaCard_reader_* 12M MediaCard_reader_disabled_in_BIOS 7.5M MediaCard_reader_enabled_in_BIOS #
And now the results:
cold-boot-with-express_card_inserted__pcie_aspm-off/dmesg.notiming 2013-01-09 15:42:02.000000000 +0100 renamed from cold-boot-with-express_card_inserted/dmesg.notiming 2013-01-09 15:15:28.000000000 +0100 +9 -12
@@ -1,5 +1,5 @@ Linux version 3.7.1-default (root@vostro) (gcc version 4.6.3 (Gentoo 4.6.3 p1.8, pie-0.5.2) ) #30 SMP Wed Jan 9 14:45:10 MET 2013 -Command line: root=/dev/sda5 pciehp.pciehp_debug=1 slub_debug=AFPZ +Command line: root=/dev/sda5 pciehp.pciehp_debug=1 slub_debug=AFPZ pcie_aspm=off
@@ -374,7 +375,6 @@ pci 0000:11:00.0: reg 20: [io 0xc000-0xc07f] pci 0000:11:00.0: reg 30: [mem 0xf6c00000-0xf6c7ffff pref] pci 0000:11:00.0: supports D1 D2 -pci 0000:11:00.0: disabling ASPM on pre-1.1 PCIe device. You can enable it with 'pcie_aspm=force' pci_bus 0000:11: fixups for bus pci 0000:00:1c.7: PCI bridge to [bus 11-16] pci 0000:00:1c.7: bridge window [io 0xc000-0xdfff]
@@ -393,8 +393,7 @@ ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.RP04._PRT] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.RP05._PRT] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.RP08._PRT] - pci0000:00: Requesting ACPI _OSC control (0x1d) - pci0000:00: ACPI _OSC control (0x19) granted + pci0000:00: Unable to request _OSC control (_OSC support mask: 0x19) ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 5 6 10 *11 12 14 15) ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 *5 6 10 11 12 14 15) ACPI: PCI Interrupt Link [LNKC] (IRQs *3 4 5 6 10 11 12 14 15)
@@ -654,7 +653,6 @@ io scheduler noop registered io scheduler deadline registered (default) io scheduler cfq registered -pcieport 0000:00:1c.7: irq 40 for MSI/MSI-X intel_idle: MWAIT substates: 0x21120 intel_idle: v0.4 model 0x2A intel_idle: lapic_timer_reliable_states 0xffffffff
@@ -688,7 +686,7 @@ ACPI: If an ACPI driver is available for this device, you should use it instead of the native driver lpc_ich: Resource conflict(s) found affecting gpio_ich ahci 0000:00:1f.2: version 3.0 -ahci 0000:00:1f.2: irq 41 for MSI/MSI-X +ahci 0000:00:1f.2: irq 40 for MSI/MSI-X ahci: SSS flag set, parallel bus scan disabled ahci 0000:00:1f.2: AHCI 0001.0300 32 slots 6 ports 6 Gbps 0x31 impl SATA mode ahci 0000:00:1f.2: flags: 64bit ncq sntf stag pm led clo pio slum part ems sxs apst
@@ -699,18 +697,18 @@ scsi3 : ahci scsi4 : ahci scsi5 : ahci -ata1: SATA max UDMA/133 abar m2048@0xf7f06000 port 0xf7f06100 irq 41 +ata1: SATA max UDMA/133 abar m2048@0xf7f06000 port 0xf7f06100 irq 40 ata2: DUMMY ata3: DUMMY ata4: DUMMY -ata5: SATA max UDMA/133 abar m2048@0xf7f06000 port 0xf7f06300 irq 41 -ata6: SATA max UDMA/133 abar m2048@0xf7f06000 port 0xf7f06380 irq 41 +ata5: SATA max UDMA/133 abar m2048@0xf7f06000 port 0xf7f06300 irq 40 +ata6: SATA max UDMA/133 abar m2048@0xf7f06000 port 0xf7f06380 irq 40 tun: Universal TUN/TAP device driver, 1.6 tun: (C) 1999-2004 Max Krasnyansky <[email protected]> r8169 Gigabit Ethernet driver 2.3LK-NAPI loaded r8169 0000:05:00.0: enabling Mem-Wr-Inval -r8169 0000:05:00.0: irq 42 for MSI/MSI-X -r8169 0000:05:00.0 eth0: RTL8168e/8111e at 0xffffc9000184a000, 18:03:73:a0:6f:6a, XID 0c200000 IRQ 42 +r8169 0000:05:00.0: irq 41 for MSI/MSI-X +r8169 0000:05:00.0 eth0: RTL8168e/8111e at 0xffffc9000184a000, 18:03:73:a0:6f:6a, XID 0c200000 IRQ 41 r8169 0000:05:00.0 eth0: jumbo features [frames: 9200 bytes, tx checksumming: ko] VMware vmxnet3 virtual NIC driver - version 1.1.29.0-k-NAPI ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
@@ -791,9 +789,9 @@ xhci_hcd 0000:0b:00.0: enabling Mem-Wr-Inval xhci_hcd 0000:0b:00.0: supports USB remote wakeup xhci_hcd 0000:0b:00.0: irq 16, io mem 0xf7d00000 +xhci_hcd 0000:0b:00.0: irq 42 for MSI/MSI-X xhci_hcd 0000:0b:00.0: irq 43 for MSI/MSI-X xhci_hcd 0000:0b:00.0: irq 44 for MSI/MSI-X -xhci_hcd 0000:0b:00.0: irq 45 for MSI/MSI-X usb usb3: default language 0x0409 usb usb3: udev 1, busnum 3, minor = 256 usb usb3: New USB device found, idVendor=1d6b, idProduct=0002
[cut]
I am not much knowing what is this kernel stuff all about but why other devices retain MSI and only 0000:00:1c.7 lost it? Due to IRQ40 being not picked up by pcieport it was assigned to other devices in the row and therefore the diff just shows changed IRQ numbers for following devices. I will show below just a snippet of interrupt changes with a question "Why was the IRQ40 not reported when claimed by pcieport? The is no line starting as '-40'! Was it never reported through /proc/interrupts until 'pcie_aspm=off'? But dmesg showed that it was used, or thought to be assigned by the driver itself?
cold-boot-with-express_card_inserted__pcie_aspm-off/interrupts 2013-01-09 15:42:01.000000000 +0100 renamed from cold-boot-with-express_card_inserted/interrupts 2013-01-09 15:14:33.000000000 +0100 +8 -7
@@ -1,29 +1,29 @@ CPU0 CPU1 0: 36 0 IO-APIC-edge timer 1: 13 0 IO-APIC-edge i8042 - 8: 16 0 IO-APIC-edge rtc0 + 8: 60 0 IO-APIC-edge rtc0 9: 0 0 IO-APIC-fasteoi acpi 12: 241 0 IO-APIC-edge i8042 - 16: 53 0 IO-APIC-fasteoi ehci_hcd:usb1 - 19: 355 0 IO-APIC-fasteoi sata_sil24 - 23: 440 0 IO-APIC-fasteoi ehci_hcd:usb2 - 41: 5342 0 PCI-MSI-edge ahci - 42: 264 0 PCI-MSI-edge eth0 - 43: 1116 0 PCI-MSI-edge xhci_hcd + 16: 50 0 IO-APIC-fasteoi ehci_hcd:usb1 + 19: 356 0 IO-APIC-fasteoi sata_sil24 + 23: 345 0 IO-APIC-fasteoi ehci_hcd:usb2 + 40: 5368 0 PCI-MSI-edge ahci + 41: 205 0 PCI-MSI-edge eth0 + 42: 308 0 PCI-MSI-edge xhci_hcd + 43: 0 0 PCI-MSI-edge xhci_hcd
[cut]
--- MediaCard_reader_enabled_in_BIOS/cold-boot-with-express_card_inserted/lspci_vvxx 2013-01-09 15:14:26.000000000 +0100
+++ MediaCard_reader_enabled_in_BIOS/cold-boot-with-express_card_inserted__pcie_aspm-off/lspci_vvxx 2013-01-09 15:42:01.000000000 +0100
[cut]
@@ -284,7 +284,7 @@
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
LnkCap: Port #5, Speed 5GT/s, Width x1, ASPM L0s L1, Latency L0 <512ns, L1 <16us
ClockPM- Surprise- LLActRep+ BwNot-
- LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
+ LnkCtl: ASPM L1 Enabled; RCB 64 bytes Disabled- Retrain- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive+ BWMgmt+ ABWMgmt+
SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
@@ -316,7 +316,7 @@
30: 00 00 00 00 40 00 00 00 00 00 00 00 0b 01 10 00
00:1c.7 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 8 (rev b5) (prog-if 00 [Normal decode])
- Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
+ Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Bus: primary=00, secondary=11, subordinate=16, sec-latency=0
@@ -354,14 +354,14 @@
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
- Capabilities: [80] MSI: Enable+ Count=1/1 Maskable- 64bit-
- Address: fee0300c Data: 4191
+ Capabilities: [80] MSI: Enable- Count=1/1 Maskable- 64bit-
+ Address: 00000000 Data: 0000
Capabilities: [90] Subsystem: Dell Device 04b3
Capabilities: [a0] Power Management version 2
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Kernel driver in use: pcieport
-00: 86 80 1e 1c 07 04 10 00 b5 00 04 06 10 00 81 00
+00: 86 80 1e 1c 07 00 10 00 b5 00 04 06 10 00 81 00
10: 00 00 00 00 00 00 00 00 00 11 16 00 c0 d0 00 00
20: c0 f6 c0 f7 01 f0 01 f1 00 00 00 00 00 00 00 00
30: 00 00 00 00 40 00 00 00 00 00 00 00 0a 04 10 00
Loading the hotplug drivers does not yield the IRQ (I think actually the modprobe acpiphp replied that the device does not exist, but forced loading of pci_hotplug and of did load acpiphp):
@@ -1252,3 +1249,67 @@
perf_event_intel: PEBS enabled due to microcode update r8169 0000:05:00.0 eth0: link down
r8169 0000:05:00.0 eth0: link up
+usb 4-1: remote wakeup needed for autosuspend +pci_hotplug: PCI Hot Plug PCI Core version: 0.5 +acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 +pci_bus 0000:11: dev 00, created physical slot 1
I think the error that a device does not exist should be logged by kernel. Stderr is not a good storage media, nor my memory. ;-)
At least appears I really do have my card in the slot since cold boot. ;-) Probably due to the pci_hotplug driver working?
cold-boot-with-express_card_inserted__pcie_aspm-off__loaded_acpiphp__pci_hotplug/lspci_vvxx 2013-01-09 15:44:19.000000000 +0100 renamed from cold-boot-with-express_card_inserted__pcie_aspm-off/lspci_vvxx 2013-01-09 15:42:01.000000000 +0100 +1 -0
@@ -589,6 +589,7 @@ 11:00.0 Mass storage controller: Silicon Image, Inc. SiI 3132 Serial ATA Raid II Controller (rev 01) Subsystem: Silicon Image, Inc. SiI 3132 Serial ATA Raid II Controller + Physical Slot: 1 Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 64 bytes
Does the above make sense to you?
So, 'pcie_aspm=force' kills the 0000:11:00.0 device, probably because of pci0000:00: Unable to request _OSC control (_OSC support mask: 0x19)?
Below I am showing what *works*.
If I don't care about hotplug and do NOT pass 'pcie_aspm=force' here is shown how is the card detected during cold-boot through PCI/PCIe (no hotplug support needed, not even loaded!):
cold-boot-with-express_card_inserted/dmesg.notiming 2013-01-09 15:15:28.000000000 +0100 renamed from cold-boot-without-express_card_inserted/dmesg.notiming 2013-01-09 15:04:26.000000000 +0100 +17 -2
@@ -368,6 +368,13 @@ pci_bus 0000:0b: bus scan returning with max=0b pci 0000:00:1c.7: scanning [bus 11-16] behind bridge, pass 0 pci_bus 0000:11: scanning bus +pci 0000:11:00.0: [1095:3132] type 00 class 0x018000 +pci 0000:11:00.0: reg 10: [mem 0xf6c84000-0xf6c8407f 64bit] +pci 0000:11:00.0: reg 18: [mem 0xf6c80000-0xf6c83fff 64bit] +pci 0000:11:00.0: reg 20: [io 0xc000-0xc07f] +pci 0000:11:00.0: reg 30: [mem 0xf6c00000-0xf6c7ffff pref] +pci 0000:11:00.0: supports D1 D2 +pci 0000:11:00.0: disabling ASPM on pre-1.1 PCIe device. You can enable it with 'pcie_aspm=force' pci_bus 0000:11: fixups for bus pci 0000:00:1c.7: PCI bridge to [bus 11-16] pci 0000:00:1c.7: bridge window [io 0xc000-0xdfff]
@@ -433,6 +440,9 @@ pci 0000:09:00.0: BAR 0: reserving [mem 0xf7e00000-0xf7e01fff flags 0x140204] (d=0, p=0) pci 0000:0b:00.0: BAR 0: reserving [mem 0xf7d00000-0xf7d0ffff flags 0x140204] (d=0, p=0) pci 0000:0b:00.0: BAR 2: reserving [mem 0xf7d10000-0xf7d11fff flags 0x140204] (d=0, p=0) +pci 0000:11:00.0: BAR 0: reserving [mem 0xf6c84000-0xf6c8407f flags 0x140204] (d=0, p=0) +pci 0000:11:00.0: BAR 2: reserving [mem 0xf6c80000-0xf6c83fff flags 0x140204] (d=0, p=0) +pci 0000:11:00.0: BAR 4: reserving [io 0xc000-0xc07f flags 0x40101] (d=0, p=0) e820: reserve RAM buffer [mem 0x0009d400-0x0009ffff] e820: reserve RAM buffer [mem 0xda4e5000-0xdbffffff] e820: reserve RAM buffer [mem 0xda793000-0xdbffffff]
@@ -1152,10 +1162,15 @@ usbhid 3-1.4:1.0: usb_probe_interface - got id input: CHICONY USB Keyboard as /devices/pci0000:00/0000:00:1c.4/0000:0b:00.0/usb3/3-1/3-1.4/3-1.4:1.0/input/input13 hid-generic 0003:04F2:0116.0002: input,hidraw1: USB HID v1.10 Keyboard [CHICONY USB Keyboard] on usb-0000:0b:00.0-1.4/input0 -hub 2-1:1.0: state 7 ports 8 chg 0000 evt 0040 +hub 3-1:1.0: state 7 ports 4 chg 0000 evt 0008 scsi 6:0:0:0: Direct-Access Generic- Multi-Card 1.00 PQ: 0 ANSI: 0 CCS sd 6:0:0:0: Attached scsi generic sg2 type 0 -systemd-udevd[1934]: starting version 196 +systemd-udevd[1937]: starting version 196 +sata_sil24 0000:11:00.0: version 1.1 +scsi7 : sata_sil24 +scsi8 : sata_sil24 +ata7: SATA max UDMA/100 host m128@0xf6c84000 port 0xf6c80000 irq 19 +ata8: SATA max UDMA/100 host m128@0xf6c84000 port 0xf6c82000 irq 19 Intel(R) Wireless WiFi driver for Linux, in-tree:d Copyright(c) 2003-2012 Intel Corporation iwlwifi 0000:09:00.0: pci_resource_len = 0x00002000
[cut]
cold-boot-with-express_card_inserted/lspci_vvxx 2013-01-09 15:14:26.000000000 +0100 renamed from cold-boot-without-express_card_inserted/lspci_vvxx 2013-01-09 15:01:05.000000000 +0100 +44 -4
101 lines not on this page · raw message
cold-boot-with-express_card_inserted/interrupts 2013-01-09 15:14:33.000000000 +0100 renamed from cold-boot-without-express_card_inserted/interrupts 2013-01-09 15:01:29.000000000 +0100 +13 -12
@@ -1,28 +1,29 @@ CPU0 CPU1 0: 36 0 IO-APIC-edge timer 1: 13 0 IO-APIC-edge i8042 - 8: 22 0 IO-APIC-edge rtc0 + 8: 16 0 IO-APIC-edge rtc0 9: 0 0 IO-APIC-fasteoi acpi 12: 241 0 IO-APIC-edge i8042 16: 53 0 IO-APIC-fasteoi ehci_hcd:usb1 - 23: 386 0 IO-APIC-fasteoi ehci_hcd:usb2 - 41: 5381 0 PCI-MSI-edge ahci - 42: 309 0 PCI-MSI-edge eth0 - 43: 422 0 PCI-MSI-edge xhci_hcd + 19: 355 0 IO-APIC-fasteoi sata_sil24 + 23: 440 0 IO-APIC-fasteoi ehci_hcd:usb2 + 41: 5342 0 PCI-MSI-edge ahci + 42: 264 0 PCI-MSI-edge eth0 + 43: 1116 0 PCI-MSI-edge xhci_hcd 44: 0 0 PCI-MSI-edge xhci_hcd 45: 0 0 PCI-MSI-edge xhci_hcd - 46: 509 0 PCI-MSI-edge snd_hda_intel + 46: 506 0 PCI-MSI-edge snd_hda_intel 47: 1 0 PCI-MSI-edge iwlwifi 48: 13 0 PCI-MSI-edge i915 -NMI: 17 30 Non-maskable interrupts -LOC: 8163 6823 Local timer interrupts +NMI: 16 29 Non-maskable interrupts +LOC: 8724 6451 Local timer interrupts SPU: 0 0 Spurious interrupts -PMI: 17 30 Performance monitoring interrupts +PMI: 16 29 Performance monitoring interrupts IWI: 0 0 IRQ work interrupts RTR: 0 0 APIC ICR read retries -RES: 3121 3805 Rescheduling interrupts -CAL: 34 2857 Function call interrupts -TLB: 93 142 TLB shootdowns +RES: 4084 4027 Rescheduling interrupts +CAL: 23 2719 Function call interrupts +TLB: 104 50 TLB shootdowns TRM: 0 0 Thermal event interrupts THR: 0 0 Threshold APIC interrupts MCE: 0 0 Machine check exceptions
And showing, that acpiphp works here while 'pcie_aspm=off':
cold-boot-without-express_card_inserted__pcie_aspm-off__loaded_acpiphp__pci_hotplug__card_hotplugged/dmesg.notiming 2013-01-09 15:58:36.000000000 +0100 renamed from cold-boot-with-express_card_inserted__pcie_aspm-off/dmesg.notiming 2013-01-09 15:42:02.000000000 +0100 +1 -15
@@ -369,12 +369,6 @@ pci_bus 0000:0b: bus scan returning with max=0b pci 0000:00:1c.7: scanning [bus 11-16] behind bridge, pass 0 pci_bus 0000:11: scanning bus -pci 0000:11:00.0: [1095:3132] type 00 class 0x018000 -pci 0000:11:00.0: reg 10: [mem 0xf6c84000-0xf6c8407f 64bit] -pci 0000:11:00.0: reg 18: [mem 0xf6c80000-0xf6c83fff 64bit] -pci 0000:11:00.0: reg 20: [io 0xc000-0xc07f] -pci 0000:11:00.0: reg 30: [mem 0xf6c00000-0xf6c7ffff pref] -pci 0000:11:00.0: supports D1 D2 pci_bus 0000:11: fixups for bus pci 0000:00:1c.7: PCI bridge to [bus 11-16] pci 0000:00:1c.7: bridge window [io 0xc000-0xdfff]
@@ -439,9 +433,6 @@ pci 0000:09:00.0: BAR 0: reserving [mem 0xf7e00000-0xf7e01fff flags 0x140204] (d=0, p=0) pci 0000:0b:00.0: BAR 0: reserving [mem 0xf7d00000-0xf7d0ffff flags 0x140204] (d=0, p=0) pci 0000:0b:00.0: BAR 2: reserving [mem 0xf7d10000-0xf7d11fff flags 0x140204] (d=0, p=0) -pci 0000:11:00.0: BAR 0: reserving [mem 0xf6c84000-0xf6c8407f flags 0x140204] (d=0, p=0) -pci 0000:11:00.0: BAR 2: reserving [mem 0xf6c80000-0xf6c83fff flags 0x140204] (d=0, p=0) -pci 0000:11:00.0: BAR 4: reserving [io 0xc000-0xc07f flags 0x40101] (d=0, p=0) e820: reserve RAM buffer [mem 0x0009d400-0x0009ffff] e820: reserve RAM buffer [mem 0xda4e5000-0xdbffffff] e820: reserve RAM buffer [mem 0xda793000-0xdbffffff]
@@ -1162,12 +1153,7 @@ hub 3-1:1.0: state 7 ports 4 chg 0000 evt 0008 scsi 6:0:0:0: Direct-Access Generic- Multi-Card 1.00 PQ: 0 ANSI: 0 CCS sd 6:0:0:0: Attached scsi generic sg2 type 0 -systemd-udevd[1935]: starting version 196 -sata_sil24 0000:11:00.0: version 1.1 -scsi7 : sata_sil24 -scsi8 : sata_sil24 -ata7: SATA max UDMA/100 host m128@0xf6c84000 port 0xf6c80000 irq 19 -ata8: SATA max UDMA/100 host m128@0xf6c84000 port 0xf6c82000 irq 19 +systemd-udevd[1933]: starting version 196 Intel(R) Wireless WiFi driver for Linux, in-tree:d Copyright(c) 2003-2012 Intel Corporation iwlwifi 0000:09:00.0: pci_resource_len = 0x00002000
[cut]
@@ -1249,3 +1206,58 @@
perf_event_intel: PEBS enabled due to microcode update
r8169 0000:05:00.0 eth0: link down
r8169 0000:05:00.0 eth0: link up
+usb 4-1: remote wakeup needed for autosuspend
+pci_hotplug: PCI Hot Plug PCI Core version: 0.5
+acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
+pci_bus 0000:11: dev 00, created physical slot 1
+acpiphp: Slot [1] registered
+usb 4-1: remote wakeup needed for autosuspend
+pci 0000:11:00.0: [1095:3132] type 00 class 0x018000
+pci 0000:11:00.0: reg 10: [mem 0x00000000-0x0000007f 64bit]
+pci 0000:11:00.0: reg 18: [mem 0x00000000-0x00003fff 64bit]
+pci 0000:11:00.0: reg 20: [io 0x0000-0x007f]
+pci 0000:11:00.0: reg 30: [mem 0x00000000-0x0007ffff pref]
+pci 0000:11:00.0: supports D1 D2
+pci 0000:11:00.0: BAR 6: assigned [mem 0xf0000000-0xf007ffff pref]
+pci 0000:11:00.0: BAR 2: assigned [mem 0xf6c00000-0xf6c03fff 64bit]
+pci 0000:11:00.0: BAR 2: set to [mem 0xf6c00000-0xf6c03fff 64bit] (PCI address [0xf6c00000-0xf6c03fff]) +pci 0000:11:00.0: BAR 0: assigned [mem 0xf6c04000-0xf6c0407f 64bit]
+pci 0000:11:00.0: BAR 0: set to [mem 0xf6c04000-0xf6c0407f 64bit] (PCI address [0xf6c04000-0xf6c0407f]) +pci 0000:11:00.0: BAR 4: assigned [io 0xc000-0xc07f]
+pci 0000:11:00.0: BAR 4: set to [io 0xc000-0xc07f] (PCI address [0xc000-0xc07f]) +sata_sil24 0000:11:00.0: version 1.1
+sata_sil24 0000:11:00.0: enabling device (0100 -> 0103)
+sata_sil24 0000:11:00.0: enabling bus mastering
+scsi7 : sata_sil24
+scsi8 : sata_sil24
+ata7: SATA max UDMA/100 host m128@0xf6c04000 port 0xf6c00000 irq 19
+ata8: SATA max UDMA/100 host m128@0xf6c04000 port 0xf6c02000 irq 19
+ata7: SATA link up 3.0 Gbps (SStatus 123 SControl 0)
+ata7.00: ATA-8: WDC WD10TPVT-00HT5T0, 01.01A01, max UDMA/133
+ata7.00: 1953525168 sectors, multi 0: LBA48 NCQ (depth 31/32)
+ata7.00: failed to get Identify Device Data, Emask 0x1
+ata7.00: failed to get Identify Device Data, Emask 0x1
+ata7.00: configured for UDMA/100
+scsi 7:0:0:0: Direct-Access ATA WDC WD10TPVT-00H 01.0 PQ: 0 ANSI: 5
+sd 7:0:0:0: [sdc] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB)
+sd 7:0:0:0: [sdc] Write Protect is off
+sd 7:0:0:0: [sdc] Mode Sense: 00 3a 00 00
+sd 7:0:0:0: [sdc] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA +sd 7:0:0:0: Attached scsi generic sg3 type 0
+ sdc: sdc1
+sd 7:0:0:0: [sdc] Attached SCSI disk
+ata8: SATA link up 3.0 Gbps (SStatus 123 SControl 0)
+ata8.00: ATA-8: ST3000VX000-9YW166, CV12, max UDMA/133
+ata8.00: 5860533168 sectors, multi 0: LBA48 NCQ (depth 31/32)
+ata8.00: failed to get Identify Device Data, Emask 0x1
+ata8.00: failed to get Identify Device Data, Emask 0x1
+ata8.00: configured for UDMA/100
+scsi 8:0:0:0: Direct-Access ATA ST3000VX000-9YW1 CV12 PQ: 0 ANSI: 5
+sd 8:0:0:0: Attached scsi generic sg4 type 0
+sd 8:0:0:0: [sdd] 5860533168 512-byte logical blocks: (3.00 TB/2.72 TiB)
+sd 8:0:0:0: [sdd] 4096-byte physical blocks
+sd 8:0:0:0: [sdd] Write Protect is off
+sd 8:0:0:0: [sdd] Mode Sense: 00 3a 00 00
+sd 8:0:0:0: [sdd] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
+ sdd: sdd1
+sd 8:0:0:0: [sdd] Attached SCSI disk
cold-boot-without-express_card_inserted__pcie_aspm-off__loaded_acpiphp__pci_hotplug__card_hotplugged/lspci_vvxx 2013-01-09 15:58:36.000000000 +0100 renamed from cold-boot-with-express_card_inserted__pcie_aspm-off/lspci_vvxx 2013-01-09 15:42:01.000000000 +0100 +10 -9
@@ -343,7 +343,7 @@ SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg- Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock- SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock- - Changed: MRL- PresDet- LinkState- + Changed: MRL- PresDet- LinkState+ RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible- RootCap: CRSVisible- RootSta: PME ReqID 0000, PMEStatus- PMEPending-
@@ -589,14 +589,15 @@ 11:00.0 Mass storage controller: Silicon Image, Inc. SiI 3132 Serial ATA Raid II Controller (rev 01) Subsystem: Silicon Image, Inc. SiI 3132 Serial ATA Raid II Controller - Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- + Physical Slot: 1 + Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- - Latency: 0, Cache Line Size: 64 bytes + Latency: 0, Cache Line Size: 32 bytes Interrupt: pin A routed to IRQ 19 - Region 0: Memory at f6c84000 (64-bit, non-prefetchable) [size=128] - Region 2: Memory at f6c80000 (64-bit, non-prefetchable) [size=16K] + Region 0: Memory at f6c04000 (64-bit, non-prefetchable) [size=128] + Region 2: Memory at f6c00000 (64-bit, non-prefetchable) [size=16K] Region 4: I/O ports at c000 [size=128] - Expansion ROM at f6c00000 [disabled] [size=512K] + [virtual] Expansion ROM at f0000000 [disabled] [size=512K] Capabilities: [54] Power Management version 2 Flags: PMEClk- DSI+ D1+ D2+ AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=1 PME-
@@ -622,8 +623,8 @@ CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- AERCap: First Error Pointer: 14, GenCap+ CGenEn- ChkCap+ ChkEn- Kernel driver in use: sata_sil24 -00: 95 10 32 31 07 00 10 00 01 00 80 01 10 00 00 00 -10: 04 40 c8 f6 00 00 00 00 04 00 c8 f6 00 00 00 00 +00: 95 10 32 31 07 01 10 00 01 00 80 01 08 00 00 00 +10: 04 40 c0 f6 00 00 00 00 04 00 c0 f6 00 00 00 00 20: 01 c0 00 00 00 00 00 00 00 00 00 00 95 10 32 31 -30: 00 00 c0 f6 54 00 00 00 00 00 00 00 0a 01 00 00 +30: 00 00 00 00 54 00 00 00 00 00 00 00 00 01 00 00
cold-boot-without-express_card_inserted__pcie_aspm-off__loaded_acpiphp__pci_hotplug__card_hotplugged/interrupts 2013-01-09 15:58:36.000000000 +0100 renamed from cold-boot-with-express_card_inserted__pcie_aspm-off/interrupts 2013-01-09 15:42:01.000000000 +0100 +15 -15
@@ -1,29 +1,29 @@ CPU0 CPU1 0: 36 0 IO-APIC-edge timer 1: 13 0 IO-APIC-edge i8042 - 8: 60 0 IO-APIC-edge rtc0 - 9: 0 0 IO-APIC-fasteoi acpi + 8: 24 0 IO-APIC-edge rtc0 + 9: 1 0 IO-APIC-fasteoi acpi 12: 241 0 IO-APIC-edge i8042 - 16: 50 0 IO-APIC-fasteoi ehci_hcd:usb1 - 19: 356 0 IO-APIC-fasteoi sata_sil24 - 23: 345 0 IO-APIC-fasteoi ehci_hcd:usb2 - 40: 5368 0 PCI-MSI-edge ahci - 41: 205 0 PCI-MSI-edge eth0 - 42: 308 0 PCI-MSI-edge xhci_hcd + 16: 100 0 IO-APIC-fasteoi ehci_hcd:usb1 + 19: 325 0 IO-APIC-fasteoi sata_sil24 + 23: 873 0 IO-APIC-fasteoi ehci_hcd:usb2 + 40: 5637 0 PCI-MSI-edge ahci + 41: 375 0 PCI-MSI-edge eth0 + 42: 861 0 PCI-MSI-edge xhci_hcd 43: 0 0 PCI-MSI-edge xhci_hcd 44: 0 0 PCI-MSI-edge xhci_hcd - 45: 511 0 PCI-MSI-edge snd_hda_intel + 45: 504 0 PCI-MSI-edge snd_hda_intel 46: 1 0 PCI-MSI-edge iwlwifi 47: 13 0 PCI-MSI-edge i915 -NMI: 20 23 Non-maskable interrupts -LOC: 7196 5604 Local timer interrupts +NMI: 35 11 Non-maskable interrupts +LOC: 13694 9226 Local timer interrupts SPU: 0 0 Spurious interrupts -PMI: 20 23 Performance monitoring interrupts +PMI: 35 11 Performance monitoring interrupts IWI: 0 0 IRQ work interrupts RTR: 0 0 APIC ICR read retries -RES: 3203 3903 Rescheduling interrupts -CAL: 18 2272 Function call interrupts -TLB: 114 55 TLB shootdowns +RES: 4840 4130 Rescheduling interrupts +CAL: 27 2852 Function call interrupts +TLB: 210 133 TLB shootdowns TRM: 0 0 Thermal event interrupts THR: 0 0 Threshold APIC interrupts MCE: 0 0 Machine check exceptions
Thanks
Martin
Quoted 3 lines · Martin Mokrejs
- pci0000:00: Requesting ACPI
_OSCcontrol (0x1d)
- pci0000:00: ACPI_OSCcontrol (0x19) granted
+ pci0000:00: Unable to request_OSCcontrol (_OSCsupport mask:0x19)
according to _OSC related game in acpi_pci_root_add() it will query_osc_support with flags |=
OSC_EXT_PCI_CONFIG_SUPPORT \
| OSC_ACTIVE_STATE_PWR_SUPPORT \
| OSC_CLOCK_PWR_CAPABILITY_SUPPORT \
| OSC_MSI_SUPPORTand the firmware should not return ACPI FAILURE, and if it return failure, flags will get reset.
then if and only if flags keep there five bits, kernel will try to set control to _OSC for
OSC_PCI_EXPRESS_CAP_STRUCTURE_CONTROL
| OSC_PCI_EXPRESS_NATIVE_HP_CONTROL
| OSC_PCI_EXPRESS_PME_CONTROL;and may be AER.
that will let pciehp own the device <pciehp will claim that later...>
in acpiphp there is module that will check if port is owned by pciehp, and it will bail out early.
in device_is_managed_by_native_pciehp...
pcie_aspm=off will stop all _osc setting, like pciehp, pme and aer.
the checking in acpiphp is introduced by:
commit 0d52f54e2ef64c189dedc332e680b2eb4a34590a
Author: Rafael J. Wysocki <[email protected]> Date: Sat Oct 22 00:43:38 2011 +0200
PCI / ACPI: Make acpiphp ignore root bridges using PCIe native hotplug
so it is a regression.
Rafael,
can you fix that? otherwise user will have specify weird "pcie_aspm=off" to make acpiphp working.
looks like we need to have other way to do handshaking between pciehp and acpiphp.
Thanks
Yinghai
Hi Yinghai,
thank you for you answer, it is way too late here but a quick answer ...
Quoted 30 lines · Yinghai Lu
Quoted 3 lines · Martin Mokrejs
- pci0000:00: Requesting ACPI
_OSCcontrol (0x1d)
- pci0000:00: ACPI_OSCcontrol (0x19) granted
+ pci0000:00: Unable to request_OSCcontrol (_OSCsupport mask:0x19)according to
_OSCrelated game inacpi_pci_root_add()it willquery_osc_supportwith flags |=OSC_EXT_PCI_CONFIG_SUPPORT \ | OSC_ACTIVE_STATE_PWR_SUPPORT \ | OSC_CLOCK_PWR_CAPABILITY_SUPPORT \ | OSC_MSI_SUPPORTand the firmware should not return ACPI FAILURE, and if it return failure, flags will get reset.
then if and only if flags keep there five bits, kernel will try to set control to
_OSCforOSC_PCI_EXPRESS_CAP_STRUCTURE_CONTROL | OSC_PCI_EXPRESS_NATIVE_HP_CONTROL | OSC_PCI_EXPRESS_PME_CONTROL;and may be AER.
that will let pciehp own the device <pciehp will claim that later...>
in acpiphp there is module that will check if port is owned by pciehp, and it will bail out early.
indevice_is_managed_by_native_pciehp...pcie_aspm=off will stop all
_oscsetting, like pciehp, pme and aer.
Quoted 8 lines
the checking in acpiphp is introduced by:
commit0d52f54e2ef64c189dedc332e680b2eb4a34590aAuthor: Rafael J. Wysocki <[email protected]> Date: Sat Oct 22 00:43:38 2011 +0200PCI / ACPI: Make acpiphp ignore root bridges using PCIe native hotplug
so it is a regression.
It is true that since the time around 3.3.x when I reported the problem I had in grub.conf kept pcie_aspm=force. And as I fiddled with .config now for 3.7.1 the hotplug stopped working. Sure, because I used pciehp so far, now we know. ;-) Only now I realized that if I switch to acpiphp, the 'Surprise removal' message is gone (acpiphp maybe does not even try to print something similar to it?). But as you see, I convinced myself the BIOS/hardware is correct and does change PresDet properly under acpiphp. But in one case I saw a change between two seemingly same states, when in lspci output a difference was in "BadDLLP", whatever that is. Just in case you come across that in the diff output. I wish I could prove with contents that the builtin EHCI on the C6/C200 chip interferes with the express card slot and the card reader. That was the reason why I disabled the CardReader in BIOS, because from the diffs you can see the card reader was detected only after I plugged in an express card. I speculate this could be also why the PresDet behaves differently with various express cards.
Quoted 8 lines
Rafael,
can you fix that? otherwise user will have specify weird "pcie_aspm=off" to make acpiphp working.
looks like we need to have other way to do handshaking between pciehp and acpiphp.
I am pushing now out the collected files. I am terribly slow in making up the other bug reports. I sent out already those bugs reported by kmemleak but there were other crashes. So, if you have the time; do something along:
wget 195.113.57.32/~mmokrejs/tmp/PCIe_hotplug_scenarios.tar.bz2 unpack
ls -latr foo
ls -latr bar
The names of directories are long/ugly but should be clear also to you what I did.
for f in dmesg.notiming lspci_vvxx interrupts; do diff -u -w foo/$f bar/$f | less; done
You will see that when I enabled pciehp (dirnames with HOTPLUG_PCI_PCIE in the name) the PresDet+ is not changed to PresDet- when a card is removed. But it differs what card was removed. This seems to be broken only for the USB3 card based on the NEC chip. It looks the PresDet issue does NOT happen for OHCI firewire card, nor PL2303 based serial card (for the Prolific I did not collect the data).
Well, if you don't have time I will poke through during next days. But you might be faster in interpreting the diffs. Just follow the directory timestamps to have least diff output. Compare states when a slot was with a card plugged in together. Similarly look for those dirnames which end with "card_unplugged" to see whether the PresDet and were all set to minus.
Oh, yeah, I think with 3.[34] kernel series the PresDet on the "Changed:" line from lspci did not follow the actual change, inferable from the SltSta line (thread "linux-3.4-rc5: eSATA Sil3132 ExpressCard removal results in warn_slowpath_common"). Not sure if it still happens with 3.7.1, you will see. ;-)
Finally, I have no idea what was changed in the new BIOS release, now I have A11, previously had A9 and A10. I did not observe any fix in that but as I was using the pciehp module maybe I could not even realize PresDet is working now. But why not for all cards, thats is still unclear.
Cheers,
Martin
Quoted 39 lines · Martin Mokrejs
Hi Yinghai,
thank you for you answer, it is way too late here but a quick answer ...Quoted 35 lines · Yinghai Lu
Quoted 3 lines · Martin Mokrejs
- pci0000:00: Requesting ACPI
_OSCcontrol (0x1d)
- pci0000:00: ACPI_OSCcontrol (0x19) granted
+ pci0000:00: Unable to request_OSCcontrol (_OSCsupport mask:0x19)according to
_OSCrelated game inacpi_pci_root_add()it willquery_osc_supportwith flags |=OSC_EXT_PCI_CONFIG_SUPPORT \ | OSC_ACTIVE_STATE_PWR_SUPPORT \ | OSC_CLOCK_PWR_CAPABILITY_SUPPORT \ | OSC_MSI_SUPPORTand the firmware should not return ACPI FAILURE, and if it return failure, flags will get reset.
then if and only if flags keep there five bits, kernel will try to set control to
_OSCforOSC_PCI_EXPRESS_CAP_STRUCTURE_CONTROL | OSC_PCI_EXPRESS_NATIVE_HP_CONTROL | OSC_PCI_EXPRESS_PME_CONTROL;and may be AER.
that will let pciehp own the device <pciehp will claim that later...>
in acpiphp there is module that will check if port is owned by pciehp, and it will bail out early.
indevice_is_managed_by_native_pciehp...pcie_aspm=off will stop all
_oscsetting, like pciehp, pme and aer.the checking in acpiphp is introduced by:
commit0d52f54e2ef64c189dedc332e680b2eb4a34590aAuthor: Rafael J. Wysocki <[email protected]> Date: Sat Oct 22 00:43:38 2011 +0200
Quoted 6 lines
Quoted 3 lines
PCI / ACPI: Make acpiphp ignore root bridges using PCIe native hotplug
so it is a regression.
It is true that since the time around 3.3.x when I reported the problem I had in grub.conf kept pcie_aspm=force.
Have you tried pcie_ports=compat?
This is the command line option to use in such cases, in theory.
Thanks,
Rafael
Signature · 2 lines
I speak only for myself. Rafael J. Wysocki, Intel Open Source Technology Center.
Looks I never answered this ... :(
Quoted 42 lines · Rafael J. Wysocki
Quoted 41 lines · Martin Mokrejs
Hi Yinghai,
thank you for you answer, it is way too late here but a quick answer ...Quoted 37 lines · Yinghai Lu
Quoted 3 lines · Martin Mokrejs
- pci0000:00: Requesting ACPI
_OSCcontrol (0x1d)
- pci0000:00: ACPI_OSCcontrol (0x19) granted
+ pci0000:00: Unable to request_OSCcontrol (_OSCsupport mask:0x19)according to
_OSCrelated game inacpi_pci_root_add()it willquery_osc_supportwith flags |=OSC_EXT_PCI_CONFIG_SUPPORT \ | OSC_ACTIVE_STATE_PWR_SUPPORT \ | OSC_CLOCK_PWR_CAPABILITY_SUPPORT \ | OSC_MSI_SUPPORTand the firmware should not return ACPI FAILURE, and if it return failure, flags will get reset.
then if and only if flags keep there five bits, kernel will try to set control to
_OSCforOSC_PCI_EXPRESS_CAP_STRUCTURE_CONTROL | OSC_PCI_EXPRESS_NATIVE_HP_CONTROL | OSC_PCI_EXPRESS_PME_CONTROL;and may be AER.
that will let pciehp own the device <pciehp will claim that later...>
in acpiphp there is module that will check if port is owned by pciehp, and it will bail out early.
indevice_is_managed_by_native_pciehp...pcie_aspm=off will stop all
_oscsetting, like pciehp, pme and aer.the checking in acpiphp is introduced by:
commit0d52f54e2ef64c189dedc332e680b2eb4a34590aAuthor: Rafael J. Wysocki <[email protected]> Date: Sat Oct 22 00:43:38 2011 +0200PCI / ACPI: Make acpiphp ignore root bridges using PCIe native hotplug
Quoted 8 lines
Quoted 4 lines
Quoted 1 line
so it is a regression.
It is true that since the time around 3.3.x when I reported the problem I had in grub.conf kept pcie_aspm=force.
Have you tried pcie_ports=compat?
This is the command line option to use in such cases, in theory.
No, I did not know about this option. I can redo some tests if you want.
Martin
Bjorn Helgaas
maintainerQuoted 9 lines · Martin Mokrejs
Hi,
I am following up on a former thread
Re: 3.2.11: PCI Express card cannot be re-detected withing cca 60sec timeframe about the same issue. I think I found some new info while playing with 3.7.1 kernel. It happened to me that my hotplug of express cards stopped working so it made me to to dive in a figure out what driver did I do to my .config, and what combinations of drivers and kernel command-line parameters work and which not. This email will cover just one case.
Quoted 3 lines
On this Dell Vostro 3550 express card slot works if kernel is without pciehp altogether and pci_hotplug+acpiphp are loaded as modules later on. The problem is that I must use pcie_aspm=off.
I confess I am completely bewildered here. Something is clearly badly broken, but I'm having a hard time figuring out exactly what it is. I think I'm overwhelmed by all the data :)
Quoted 1 line
From your previous "3.2.11: PCI Express card cannot be re-detected
withing cca 60sec timeframe" thread, I think:
1) With pciehp, insertion and removal of an NEC uPD720200 USB3.0 card doesn't work correctly [1]. The insertion/removal events don't seem to be detected immediately.
2) Insertion/removal of firewire card works correctly [2]
3) Insertion/removal of AXAGO ECA-SP serial/parallel card works correctly [3]
4) When the xhci driver is not loaded, insertion/removal events of the NEC USB3.0 card *are* detected correctly [4]
The ExpressCard slot is below the 00:1c.7 root port, and this port supports native PCIe hotplug. When CONFIG_HOTPLUG_PCI_PCIE=y, Linux requests control over PCIe native hotplug, and I think your BIOS grants it.
In this thread ("Dell Vostro 3550: pci_hotplug+acpiphp require 'pcie_aspm=force' on kernel command-line for hotplug to work"), I think you are saying that if you disable pciehp and use acpiphp and the "pcie_aspm=off" parameter, the ExpressCard slot works perfectly.
But I think it's a bad idea to go down the road of using acpiphp. Native PCIe hotplug (pciehp) is the default when it is supported, and as far as I can tell, it *is* supported on this system. If we had some indication that it's not supported, e.g., if the BIOS declined to grant us control over PCIe native hotplug, then of course we would fall back to using acpiphp. But I don't think we do, so we should figure out how to make pciehp work.
If it's really true that pciehp works perfectly except when xhci is loaded, that seems like a good clue, and we should look for some interaction between xhci and pciehp.
Martin, can you confirm my assumptions above or correct any mistakes I made?
Bjorn
[1] http://marc.info/?l=linux-pci&m=133236584826563&w=1
[2] http://marc.info/?l=linux-pci&m=133457374507707&w=1
[3] http://marc.info/?l=linux-pci&m=133460527222076&w=1
[4] http://marc.info/?l=linux-kernel&m=133547823904339&w=1
Hi Bjorn,
thank you for your time on this issue.
Quoted 19 lines · Bjorn Helgaas
Quoted 9 lines · Martin Mokrejs
Hi,
I am following up on a former thread
Re: 3.2.11: PCI Express card cannot be re-detected withing cca 60sec timeframe about the same issue. I think I found some new info while playing with 3.7.1 kernel. It happened to me that my hotplug of express cards stopped working so it made me to to dive in a figure out what driver did I do to my .config, and what combinations of drivers and kernel command-line parameters work and which not. This email will cover just one case.Quoted 3 lines
On this Dell Vostro 3550 express card slot works if kernel is without pciehp altogether and pci_hotplug+acpiphp are loaded as modules later on. The problem is that I must use pcie_aspm=off.
I confess I am completely bewildered here. Something is clearly badly broken, but I'm having a hard time figuring out exactly what it is. I think I'm overwhelmed by all the data :)
Quoted 6 lines
Quoted 1 line
From your previous "3.2.11: PCI Express card cannot be re-detected
withing cca 60sec timeframe" thread, I think:
1) With pciehp, insertion and removal of an NEC uPD720200 USB3.0 card doesn't work correctly [1]. The insertion/removal events don't seem to be detected immediately.
... except for the FireWire, serial/parallel and eSATA port-providing cards in 2) and 3). The one doing SATA is based on Silicon Image3132 chip, using sata_sil24 driver.
Quoted 1 line
2) Insertion/removal of firewire card works correctly [2]
Yes.
Quoted 1 line
3) Insertion/removal of AXAGO ECA-SP serial/parallel card works correctly [3]
Yes.
Quoted 2 lines
4) When the xhci driver is not loaded, insertion/removal events of the NEC USB3.0 card *are* detected correctly [4]
And my suspicion was that when a USB device is attached to the ExpressCard USB3.0 controller the "usb-storage?" or whichever driver" realizes much earlier (in time) that the ExpressCard is gone and system thus behaves properly. I think something is delayed by xhci driver until a poll happens after every 60 seconds. With usb-storage the change gets visible "immediately". Or, the card is not PCI or PCIe hotplug capable [6]? Could that be the difference in behior?
Quoted 6 lines
The ExpressCard slot is below the 00:1c.7 root port, and this port supports native PCIe hotplug. When
CONFIG_HOTPLUG_PCI_PCIE=y, Linux requests control over PCIe native hotplug, and I think your BIOS grants it.
Quoted 6 lines
In this thread ("Dell Vostro 3550: pci_hotplug+acpiphp require 'pcie_aspm=force' on kernel command-line for hotplug to work"), I think you are saying that if you disable pciehp and use acpiphp and the "pcie_aspm=off" parameter, the ExpressCard slot works perfectly.
But I think it's a bad idea to go down the road of using acpiphp.
Unfortunately I was forced to switch to acpiphp because commit 0d52f54e2ef64c189dedc332e680b2eb4a34590a (as diagnosed by Yinghai) becuase pciehp stopped working (although it worked badly anyways). The commit went in about 3.5 kernel. For me, practially I had to switch from "pcie_aspm=force" (kernels 3.2 - 3.4) to "pcie_aspm=off" (3.7 for sure). See the note Yijing in [5].
Quoted 6 lines
Native PCIe hotplug (pciehp) is the default when it is supported, and as far as I can tell, it *is* supported on this system. If we had some indication that it's not supported, e.g., if the BIOS declined to grant us control over PCIe native hotplug, then of course we would fall back to using acpiphp. But I don't think we do, so we should figure out how to make pciehp work.
I do not understand the diffs between PCI hotplug, PCIe hotplug and their requriements by pciehp versus acpiphp so do not want to comment on that. I just linked to Yijing's comments in other threads [5, 6] and judge what is more generic option.
Yighai in the 3.2.x thread postulated it is a BIOS or silicon bug incorrectly providing PresDet status. I was glad that I can show that with acpiphp the PresDet is *always* correct (3.7. kernel) *provided* I disabled MediaCard reader in BIOS.
So although I do not believe into a hardware bug in case of PresDet reporting I smell there is one in cross-interaction between ExpressCard slot, EHCI port and MediaCard reader (via EHCI as well), all hooked up to SandyBridge C6/C200 intel chip. What I saw is that the MediaCard reader gets detected when an Expresscard is plugged into its slot (that is, much later after a bootup, and triggered only by express card insertion). I speculate further that the EHCI port sometimes resets again just due to some ExpressCard interference. I reported such problems in the past on usb mailing list we could not make any conclusion out of it except that my external USB hub might be bad. Buying a new one did not help. Instead, I disabled the MediaCard reader in BIOS.
Note: My motherboard was already exchanged and also the ExpressCard metalic slot thing.
Quoted 4 lines
If it's really true that pciehp works perfectly except when xhci is loaded, that seems like a good clue, and we should look for some interaction between xhci and pciehp.
And also, why is presence of Firewire, RS232/LPT and sata_sil24 cards is reported correctly by pciehp while not NEC-based USB3 card.
The (mis)behavior with the USB3 card was that it's removal was not notified. On subsequent re-insertion of the card the software status bits were wrong and pciehp reported Surprise removal. The bits were not adjusted properly (I am intentionally not saying they should have been reset). If I again unplugged the card, the PresDet correctly reported the card was just removed. In other words, every second (even) removal of a card resulted in correct slot status values reported. On every odd removal, the status reported by pciehp was wrong.
Quoted 2 lines
Martin, can you confirm my assumptions above or correct any mistakes I made?
Several questions I had were never answered.
Why is the IRQ40 not reported in pciehp (see the original message in thsi thread). Because it is really not used, compared to pciehp?
Is acpiphp capable reporting Surprise removal like pciehp does or is acpiphp silent just because it does not print a similar message?
sata_sil24 should not be allowed to bind to newly hotplugged controller too quickly because sometimes the card slips out immediately. A delay of 6 seconds would be helpful to prevent eventual Oops, as shown in kernel timings in this email thread (nicely colored at https://patchwork.kernel.org/patch/1957681/, btw).
Could the CorrErr+ device status bits prevent PresDet change from PresDet+ to PresDet-?
Once I saw the "Changed: MRL. PresDet. LinkState." line in lspci output under SltSta:
reports badly PresDet difference while SltSta: Status: line itself reports correct current value. How is this value determined?
In the original email in this thread I showed down below from
"And showing, that acpiphp works here while 'pcie_aspm=off':" lien that acpiphp also reports badly PresDet when the MediaCardReader is enabled in BIOS. I think acpiphp driver should complain if there are conflicting values. For example, is it meaningful to see in lspci output the following: "Changed: MRL- PresDet- LinkState+"?
Quoted 1 line
From "SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-" we know
the card is in the slot so why is the "Changed: line summary wrong?
Why does kernel claim the card has a virtual ROM after hotplug while coldplugged card reports "Expansion ROM at f6c00000 [disabled] [size=512K]"? Why the differences in cache line sizes, etc? Simply, why do not both scenarios lead to exactly same card configuration?
Quoted 9 lines
Bjorn
[1] http://marc.info/?l=linux-pci&m=133236584826563&w=1
[2] http://marc.info/?l=linux-pci&m=133457374507707&w=1
[3] http://marc.info/?l=linux-pci&m=133460527222076&w=1
[4] http://marc.info/?l=linux-kernel&m=133547823904339&w=1
[5] http://marc.info/?l=linux-kernel&m=135928915003566&w=4
[6] http://marc.info/?l=linux-kernel&m=135937601131152&w=4
I hope I made things a bit clearer.
Martin
Bjorn Helgaas
maintainer[+cc Sarah because problem only seems to happen with xhci. I'm assuming this is a pciehp issue for now]
Quoted 45 lines · Martin Mokrejs
Hi Bjorn,
thank you for your time on this issue.Quoted 19 lines · Bjorn Helgaas
Quoted 9 lines · Martin Mokrejs
Hi,
I am following up on a former thread
Re: 3.2.11: PCI Express card cannot be re-detected withing cca 60sec timeframe about the same issue. I think I found some new info while playing with 3.7.1 kernel. It happened to me that my hotplug of express cards stopped working so it made me to to dive in a figure out what driver did I do to my .config, and what combinations of drivers and kernel command-line parameters work and which not. This email will cover just one case.Quoted 3 lines
On this Dell Vostro 3550 express card slot works if kernel is without pciehp altogether and pci_hotplug+acpiphp are loaded as modules later on. The problem is that I must use pcie_aspm=off.
I confess I am completely bewildered here. Something is clearly badly broken, but I'm having a hard time figuring out exactly what it is. I think I'm overwhelmed by all the data :)
Quoted 6 lines
Quoted 1 line
From your previous "3.2.11: PCI Express card cannot be re-detected
withing cca 60sec timeframe" thread, I think:
1) With pciehp, insertion and removal of an NEC uPD720200 USB3.0 card doesn't work correctly [1]. The insertion/removal events don't seem to be detected immediately.
... except for the FireWire, serial/parallel and eSATA port-providing cards in 2) and 3). The one doing SATA is based on Silicon Image3132 chip, using
sata_sil24driver.Quoted 1 line
2) Insertion/removal of firewire card works correctly [2]
Yes.
Quoted 1 line
3) Insertion/removal of AXAGO ECA-SP serial/parallel card works correctly [3]
Yes.
Quoted 2 lines
4) When the xhci driver is not loaded, insertion/removal events of the NEC USB3.0 card *are* detected correctly [4]
Quoted 6 lines
And my suspicion was that when a USB device is attached to the ExpressCard USB3.0 controller the "usb-storage?" or whichever driver" realizes much earlier (in time) that the ExpressCard is gone and system thus behaves properly. I think something is delayed by xhci driver until a poll happens after every 60 seconds. With usb-storage the change gets visible "immediately".
I'm going to ignore this speculation about xhci and other drivers for now and see if we can just get pciehp to work. We need to reduce the number of variables here.
I opened https://bugzilla.kernel.org/show_bug.cgi?id=54921
"pciehp/xhci ExpressCard failure on Dell Vostro 3550" for this issue.
Quoted 2 lines
Or, the card is not PCI or PCIe hotplug capable [6]? Could that be the difference in behior?
All ExpressCards should support hotplug. The hotplug support is in the root port and other circuitry on the motherboard.
Quoted 5 lines
Quoted 1 line
But I think it's a bad idea to go down the road of using acpiphp.
Unfortunately I was forced to switch to acpiphp because commit
0d52f54e2ef64c189dedc332e680b2eb4a34590a(as diagnosed by Yinghai) becuase pciehp stopped working (although it worked badly anyways).
I understand you have to use acpiphp to make things work right now. But I think we can make pciehp work, and I think that's what distros will want to use in this situation, so that's what I want to debug.
As background for the whole collection of hotplug drivers we have, here's my understanding of the history:
1) Originally there was no standard for PCI hotplug hardware, and we have drivers like cpqphp, ibmphp, etc. to deal with various hardware designs.
2) ACPI defined an abstract hotplug model so an OS can have a single driver, e.g., acpiphp, for that model, and the BIOS can map the abstract model to various hardware designs.
3) PCIe defined a single hardware model, so an OS can have a single driver, e.g., pciehp. ACPI is not really involved here except that the OS has to ask the BIOS for permission to use this native hotplug driver.
Many machines, including yours, support both ACPI (acpiphp) and PCIe native (pciehp) hotplug so they can run both old OSes that don't have pciehp, and newer OSes that prefer to use pciehp.
Quoted 4 lines
Yighai in the 3.2.x thread postulated it is a BIOS or silicon bug incorrectly providing PresDet status. I was glad that I can show that with acpiphp the PresDet is *always* correct (3.7. kernel) *provided* I disabled MediaCard reader in BIOS.
Can you build a v3.9-rc1 kernel with this config:
CONFIG_HOTPLUG_PCI_PCIE=y
CONFIG_HOTPLUG_PCI_ACPI=n
CONFIG_USB_XHCI_HCD=n
I want to use pciehp, not acpiphp, and leave the xhci driver out of the picture for now. Boot it with pciehp.pciehp_debug=1 and the ExpressCard slot empty, and run this command:
# while true; do echo -n "$(date +%T) SlotStatus "; setpci -s1c.7 0x5a.w; sleep 1; done
That command reads the SlotStatus register from the bridge leading to your ExpressCard slot every second. While that command is running, do insertions and removals of all your ExpressCards. Bit 6 (0x0040) is the Presence Detect bit. It should change as you insert and remove cards.
I'd like to see the complete dmesg log, the output of "lspci -vv", and the output of the above command while you insert/remove cards (with notes about which card is being hotplugged when). Since there seems to be some interaction with the MediaCard reader BIOS settings, maybe you could do this whole experiment with the reader disabled, then with it enabled.
You can attach these logs to the bugzilla
(https://bugzilla.kernel.org/show_bug.cgi?id=54921) if you want, or point me to them and I'll do it.
Thanks,
Bjorn
[+cc Alan because it looks in general, USB is involved and compared to MS Win7
with the most restrictive BIOS settings no eSATA/FireWireOHCI cards work under linux ('nousb' on kernel commandline) while they both do under Win7. Hope we will fork that stuff under a new bug #]
Quoted 27 lines · Bjorn Helgaas
[+cc Sarah because problem only seems to happen with xhci. I'm assuming this is a pciehp issue for now]
Quoted 22 lines · Martin Mokrejs
Hi Bjorn,
thank you for your time on this issue.Quoted 18 lines · Bjorn Helgaas
Quoted 9 lines · Martin Mokrejs
Hi,
I am following up on a former thread
Re: 3.2.11: PCI Express card cannot be re-detected withing cca 60sec timeframe about the same issue. I think I found some new info while playing with 3.7.1 kernel. It happened to me that my hotplug of express cards stopped working so it made me to to dive in a figure out what driver did I do to my .config, and what combinations of drivers and kernel command-line parameters work and which not. This email will cover just one case.Quoted 3 lines
On this Dell Vostro 3550 express card slot works if kernel is without pciehp altogether and pci_hotplug+acpiphp are loaded as modules later on. The problem is that I must use pcie_aspm=off.
I confess I am completely bewildered here. Something is clearly badly broken, but I'm having a hard time figuring out exactly what it is. I think I'm overwhelmed by all the data :)
It won't be easier now but you asked for it. ;)
[cut]
Quoted 75 lines
I'm going to ignore this speculation about xhci and other drivers for now and see if we can just get pciehp to work. We need to reduce the number of variables here.
I opened https://bugzilla.kernel.org/show_bug.cgi?id=54921
"pciehp/xhci ExpressCard failure on Dell Vostro 3550" for this issue.Quoted 2 lines
Or, the card is not PCI or PCIe hotplug capable [6]? Could that be the difference in behior?
All ExpressCards should support hotplug. The hotplug support is in the root port and other circuitry on the motherboard.
Quoted 5 lines
Quoted 1 line
But I think it's a bad idea to go down the road of using acpiphp.
Unfortunately I was forced to switch to acpiphp because commit
0d52f54e2ef64c189dedc332e680b2eb4a34590a(as diagnosed by Yinghai) becuase pciehp stopped working (although it worked badly anyways).I understand you have to use acpiphp to make things work right now. But I think we can make pciehp work, and I think that's what distros will want to use in this situation, so that's what I want to debug.
As background for the whole collection of hotplug drivers we have, here's my understanding of the history:
1) Originally there was no standard for PCI hotplug hardware, and we have drivers like cpqphp, ibmphp, etc. to deal with various hardware designs.
2) ACPI defined an abstract hotplug model so an OS can have a single driver, e.g., acpiphp, for that model, and the BIOS can map the abstract model to various hardware designs.
3) PCIe defined a single hardware model, so an OS can have a single driver, e.g., pciehp. ACPI is not really involved here except that the OS has to ask the BIOS for permission to use this native hotplug driver.
Many machines, including yours, support both ACPI (acpiphp) and PCIe native (pciehp) hotplug so they can run both old OSes that don't have pciehp, and newer OSes that prefer to use pciehp.
Quoted 4 lines
Yighai in the 3.2.x thread postulated it is a BIOS or silicon bug incorrectly providing PresDet status. I was glad that I can show that with acpiphp the PresDet is *always* correct (3.7. kernel) *provided* I disabled MediaCard reader in BIOS.
Can you build a v3.9-rc1 kernel with this config:
CONFIG_HOTPLUG_PCI_PCIE=y
CONFIG_HOTPLUG_PCI_ACPI=n
CONFIG_USB_XHCI_HCD=nI want to use pciehp, not acpiphp, and leave the xhci driver out of the picture for now. Boot it with pciehp.pciehp_debug=1 and the ExpressCard slot empty, and run this command:
# while true; do echo -n "$(date +%T) SlotStatus "; setpci -s1c.7 0x5a.w; sleep 1; doneThat command reads the SlotStatus register from the bridge leading to your ExpressCard slot every second. While that command is running, do insertions and removals of all your ExpressCards. Bit 6 (
0x0040) is the Presence Detect bit. It should change as you insert and remove cards.I'd like to see the complete dmesg log, the output of "lspci -vv", and the output of the above command while you insert/remove cards (with notes about which card is being hotplugged when). Since there seems to be some interaction with the MediaCard reader BIOS settings, maybe you could do this whole experiment with the reader disabled, then with it enabled.
Quoted 3 lines
You can attach these logs to the bugzilla
(https://bugzilla.kernel.org/show_bug.cgi?id=54921) if you want, or point me to them and I'll do it.
I exposed the collected data (500kB) at http://195.113.57.32/~mmokrejs/tmp/ExpressCard_hotplug_tests_3.9-rc1.tar.bz2
1)
This is 3.9-rc1 kernel. From a quick glance I think it is different from 3.7.x and 3.2.x in a way that after ejecting eSATA card there is too many diffs in "lspci -vv" output when comparing cold boot state with the state after card is ejected. I don't remember that from the past and I don't see that with the USB3 card nor FireWireOHCI card. Could it be the drivers is in 3.9-rc1 still loaded? If yes, maybe the multiple attempts to insert/eject card are affected by this. Please verify what I say and summarize in bugzilla. I won't spoil it (this email is enough).
2)
I will go straight where the USB interference is. I noticed in the past (email threads were recapped in this thread already) that after insertion of some cards "USB" bus is reset or at least, a Media Card reader is detected, or re-detected, simply appears in dmesg. In dmesg outputs you will see ehci-pci driver being involved. I tried to disable more and more USB devices in BIOS and also used 'nousb' commandline option to disable BIOS. I do not understand why that completely ExpressCard slot functionality under Linux while not under Win7. In Linux pciehp still complains about Surprise Removal even when I insert the card for the very first time after a coldboot (so the ExpressCard slot is not completely dead but neither sata_sil24 nor fw_ohci pickus up the device). USB drivers had no chance to bind because of the 'nousb', of course. Alan/Sarah, please look possibly in this order into the following dirs (well, don't waste too much time on the first one, focus more on on difference between the second and third, and I placed a file Win7.txt in the third to recap the Win7 vs. Linux differences):
without_XHCI_with_EHCI_no_ACPIPHP_with_PCIEHP/ without_USB_no_ACPIPHP_with_PCIEHP/ without_USB_no_ACPIPHP_with_PCIEHP_without_microphone_USB_wakeup_USB_emulation_USB_powershare/
My question is. Has the laptop hardwired the ExpressCard slot somehow through USB to the SandyBridge chip? It seems not only the UVC Camera is, but also a CD/DVD optical drive is and the item "Microphone" in BIOS possibly involves whole Intel HD Audio soundcard. If that is true then I understand why under linux 'nousb' prevents functionality of all the three ExpressCard tested.
Puzzlingly, while 'nousb' was used my Alps touchpad still worked in VT console.
Further, what I might have had always screwed in my .config is that there are SDHC drivers and some other which might be competing with some USB drivers (under normal .config)
for the say MediaCardReader, etc.). Please compare without_XHCI_with_EHCI_no_ACPIPHP_with_PCIEHP/.config.gz with without_USB_no_ACPIPHP_with_PCIEHP_without_microphone_USB_wakeup_USB_emulation_USB_powershare/config.gz.
3)
The hotplug issue itself. I do not understand the PCI(e) hotplug, lspci output but why is there any difference between a cold booted status of an empty expresscard slot compared to the status when a card is unloaded?
# diff -u without_XHCI_with_EHCI_no_ACPIPHP_with_PCIEHP/disabled_MediaCard_reader/USB3/lspci_vv_initial.txt without_XHCI_with_EHCI_no_ACPIPHP_with_PCIEHP/disabled_MediaCard_reader/USB3/lspci_vv_unloading_USB3_card.txt
--- without_XHCI_with_EHCI_no_ACPIPHP_with_PCIEHP/disabled_MediaCard_reader/USB3/lspci_vv_initial.txt 2013-03-07 22:27:30.000000000 +0100
+++ without_XHCI_with_EHCI_no_ACPIPHP_with_PCIEHP/disabled_MediaCard_reader/USB3/lspci_vv_unloading_USB3_card.txt 2013-03-07 22:40:25.000000000 +0100
@@ -287,7 +287,7 @@
I/O behind bridge: 0000c000-0000dfff
Memory behind bridge: f6c00000-f7cfffff
Prefetchable memory behind bridge: 00000000f0000000-00000000f10fffff
- Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
+ Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: [40] Express (v2) Root Port (Slot+), MSI 00
#Shouldn't the MAbort been cleared sometimes? Doesn't this fool the PresDet interpretation in kernel?
The eSATA behaves differently maybe because 0100 does not change to 0140 like USB3 card but to 0103? Um, the shell while loop calling setpci does not report 0103 but the driver say this:
[ 211.879397] sata_sil24 0000:11:00.0: enabling device (0100 -> 0103)
I did not check myself whether the lspci differs between the odd and even unplug of USB3 card or not, but from the win7 behavior it should be the still same under Linux like it always was.
I do not remember with 3.2.x and 3.7x kernels seeing this with the USB3 card:
+[ 594.622211] pci 0000:11:00.0: calling quirk_usb_early_handoff+0x0/0x657 +[ 594.622223] pci 0000:11:00.0: device not available (can't reserve [mem 0x00000000-0x00001fff 64bit]) +[ 594.622225] pci 0000:11:00.0: Can't enable PCI device, BIOS handoff failed.
But somebody better check the previous tarball. ;-) Ah, well, with 3.7 I used acpiphp, hmm, and how was it with 3.2.x I for sure forgot. ;)
I do not know what else to disable in BIOS except eSATA port to make the SandyBridge chip doing better. Actually, I might have also tried to disable ExpressCard to see what it really does. ;)
4)
A new 3.9-rc1 bug is that sometimes, while switching a virtual console I get
(file without_XHCI_with_EHCI_no_ACPIPHP_with_PCIEHP/disabled_MediaCard_reader/USB3/dmesg_VT12_switching_while_reading_slot_status_caused_slowpath.txt):
[ 458.113692] WARNING: at drivers/tty/tty_buffer.c:428 flush_to_ldisc+0x55/0x189() [ 458.113693] Hardware name: Vostro 3550 [ 458.113693] tty is NULL
Will try to find a proper maintainer for that.
5)
I believe I reported already in the past that repeatedly inserting and removing the FireWireOHCI card can crash the kernel, completely. That is nothing new since 3.2 at least and although I captured a partial stacktrace I won't anymore. There are several OOPSes after each other in a row.
Well, thank you for your time. I would have already returned the whole laptop
but I don't know what other piece of HW to buy. I need eSATA, at least 2x USB3.0 ports (4 USB ports in total), expressCard for extra 2 eSATA ports or Firewire, anti-reflex display, HDMI or DVI at least. Keyboard with a back-light is a plus. ;-) 8GB RAM is a minimum, currently have 16GB in this thingie. ;)
Martin
Bjorn Helgaas
maintainerQuoted 18 lines · Martin Mokrejs
Quoted 17 lines · Bjorn Helgaas
Quoted 15 lines · Martin Mokrejs
Quoted 14 lines · Bjorn Helgaas
Quoted 11 lines · Martin Mokrejs
I am following up on a former thread
Re: 3.2.11: PCI Express card cannot be re-detected withing cca 60sec timeframe about the same issue. I think I found some new info while playing with 3.7.1 kernel. It happened to me that my hotplug of express cards stopped working so it made me to to dive in a figure out what driver did I do to my .config, and what combinations of drivers and kernel command-line parameters work and which not. This email will cover just one case.On this Dell Vostro 3550 express card slot works if kernel is without pciehp altogether and pci_hotplug+acpiphp are loaded as modules later on. The problem is that I must use pcie_aspm=off.
Quoted 5 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
I confess I am completely bewildered here. Something is clearly badly broken, but I'm having a hard time figuring out exactly what it is. I think I'm overwhelmed by all the data :)
It won't be easier now but you asked for it. ;)
Well, I didn't really ask for 500kB of logs with 150-character pathnames. I am unable to process that much stuff. I'm not interested in random lspci differences, virtual console bugs, or OHCI driver crashes at this point, so let's not muddy the waters with them.
I only want to figure out if pciehp can correctly detect and
enumerate a newly inserted card, and if it can correctly clean up when a card is removed. After we figure that out, we can worry about more complicated issues.
I *think* the bottom line of the Slot Status experiment is that the Presence Detect bit was reported correctly in every case, for every card, regardless of BIOS settings. Right?
There are three cards on the table now:
pci 0000:11:00.0: [1095:3132] SiI 3132 Serial ATA Raid II Controller pci 0000:11:00.0: [1106:3403] VT6315 Series Firewire Controller pci 0000:11:00.0: [1033:0194] NEC Corporation uPD720200 USB 3.0 Host
Controller
I assume all the cards work fine if there's no hotplug, i.e., if the card is present at boot and you never remove it. Right?
Quoted 3 lines
In Linux pciehp
still complains about Surprise Removal even when I insert the card for the very first time after a coldboot
Hmm. pciehp prints "Surprise Removal" whether you inserted or removed the card. Stupid driver.
Quoted 2 lines
(so the ExpressCard slot is not completely dead but neither
sata_sil24norfw_ohcipicks up the device).
I thought the only card with a problem was the USB3.0 card. But here you suggest that there *is* a problem with the SATA and Firewire cards. Can you describe that problem in one sentence?
Quoted 2 lines
My question is. Has the laptop hardwired the ExpressCard slot somehow through USB to the SandyBridge chip?
An ExpressCard slot (spec at [1]) supports both a PCIe interface and a USB interface, so the slot *should* be connected to a USB controller as well as to a PCIe root port. An ExpressCard can contain either a PCIe device or a USB device or both. Section 6.3 of the spec talks about ACPI requirements to describe the relationship between the PCIe and USB devices. I'm not sure that Linux pays any attention to this in the hotplug paths, so I'm a little worried about this. (Maybe it doesn't need to in the PCIe-aware case; I don't know.)
It would be interesting to know exactly what devices are on your cards. Assuming they all work when present at boot, you could find that by doing a single "lspci -vv" and "lsusb -v" after a boot with an empty slot, and doing it again after a boot with a card in the slot. The difference should be the ExpressCard devices. I'm sure this is buried in your tarball somewhere, but all I want is the info from a machine in default configuration -- MediaCard enabled, etc. Just the way a typical user would be using the machine.
[1] http://www.usb.org/developers/expresscard/EC_specifications/ExpressCard_2_0_FINAL.pdf
Quoted 3 lines
The hotplug issue itself. I do not understand the PCI(e) hotplug, lspci output but why is there any difference between a cold booted status of an empty expresscard slot compared to the status when a card is unloaded?
In principle there shouldn't be any difference, but Linux isn't that good yet.
Quoted 7 lines
disabled_MediaCard_reader/USB3/lspci_vv_unloading_USB3_card.txt 2013-03-07 renamed from disabled_MediaCard_reader/USB3/lspci_vv_initial.txt 2013-03-07 22:27:30.000000000 +0100 +1
-1- Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR- + Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ <SERR- <PERR-Shouldn't the MAbort been cleared sometimes? Doesn't this fool the PresDet interpretation in kernel?
I doubt it. Presence Detect is a very simple mechanism that basically just reports the current state of the CPPE# signal in the ExpressCard slot. There's no reason this should be related to MAbort.
Quoted 4 lines
The eSATA behaves differently maybe because 0100 does not change to 0140 like USB3 card but to 0103? Um, the shell while loop calling setpci does not report 0103 but the driver say this:
[ 211.879397] sata_sil24 0000:11:00.0: enabling device (0100 -> 0103)
This is completely unrelated. The shell "setpci" command is printing the Slot Status register; the "0100 -> 0103" above is a change in the PCI Command register.
Quoted 4 lines
I do not remember with 3.2.x and 3.7x kernels seeing this with the USB3 card:
+[ 594.622211] pci 0000:11:00.0: calling quirk_usb_early_handoff+0x0/0x657 +[ 594.622223] pci 0000:11:00.0: device not available (can't reserve [mem 0x00000000-0x00001fff 64bit]) +[ 594.622225] pci 0000:11:00.0: Can't enable PCI device, BIOS handoff failed.
This is a result of trying to run the quirk on a hot-inserted device where we haven't assigned resources to it yet. I don't think we should really be running quirks on a device that early. We can look at that later, if we think this is related to the immediate problem.
Bjorn
Quoted 35 lines · Bjorn Helgaas
Quoted 18 lines · Martin Mokrejs
Quoted 17 lines · Bjorn Helgaas
Quoted 15 lines · Martin Mokrejs
Quoted 14 lines · Bjorn Helgaas
Quoted 11 lines · Martin Mokrejs
I am following up on a former thread
Re: 3.2.11: PCI Express card cannot be re-detected withing cca 60sec timeframe about the same issue. I think I found some new info while playing with 3.7.1 kernel. It happened to me that my hotplug of express cards stopped working so it made me to to dive in a figure out what driver did I do to my .config, and what combinations of drivers and kernel command-line parameters work and which not. This email will cover just one case.On this Dell Vostro 3550 express card slot works if kernel is without pciehp altogether and pci_hotplug+acpiphp are loaded as modules later on. The problem is that I must use pcie_aspm=off.
Quoted 5 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
I confess I am completely bewildered here. Something is clearly badly broken, but I'm having a hard time figuring out exactly what it is. I think I'm overwhelmed by all the data :)
It won't be easier now but you asked for it. ;)
Well, I didn't really ask for 500kB of logs with 150-character pathnames. I am unable to process that much stuff. I'm not interested in random lspci differences, virtual console bugs, or OHCI driver crashes at this point, so let's not muddy the waters with them.
I only want to figure out if pciehp can correctly detect and
enumerate a newly inserted card, and if it can correctly clean up when a card is removed. After we figure that out, we can worry about more complicated issues.
Quoted 2 lines
I *think* the bottom line of the Slot Status experiment is that the Presence Detect bit was reported correctly in every case, for every
I suspect you mean the shell "setpci" experiment. Please try:
s=find . -name slot_status* | while read f; do echo -n "$f "; done; \
ls -latr $s | awk '{print $9}' | while read ss; do echo $ss; cat $ss; \
done | less
and tell me why sometimes I saw 0000 -> 0140 during insert but sometimes 0100 -> 0140. For eject I always saw 0140 -> 0100. That means when repeatedly inserting and removing a card, on the very first insert pciehp starts with 0000 and after first eject of the card it enters into the 0100 state and since that it should always flips between expected values: 0100->0140->0100->0140 and so on. I think am I still not correct here because I observed in the past that only on every second insert "kernel" recognized the card. I bet something else comes into play otherwise I should have realized that only very first card removal is unnoticed, not every odd removal.
However, the collected data confirms 0000->0140->0100->0140->0100->0140:
Please see ./without_USB_no_ACPIPHP_with_PCIEHP_without_microphone_USB_wakeup_USB_emulation_USB_powershare/USB3_eSATA_FireWire/slot_status_inserting_and_removing_USB3_eSATA_FireWire.txt
Although you don't seem to like the long dirnames/filenames the find command output should make you immediately aware what was collected under what BIOS and kernel settings.
Quoted 1 line
card, regardless of BIOS settings. Right?
No. The SltSta Status: and Changed: lines do not agree with each other whether MediaCardReader is enabled or disabled in BIOS.
# without_XHCI_with_EHCI_no_ACPIPHP_with_PCIEHP/disabled_MediaCard_reader/USB3
# diff -u -w lspci_vv_initial.txt lspci_vv_inserting_USB3_card_attempt_2.txt
--- lspci_vv_initial.txt 2013-03-07 22:27:30.000000000 +0100
+++ lspci_vv_inserting_USB3_card_attempt_2.txt 2013-03-07 22:38:05.000000000 +0100
@@ -301,12 +301,12 @@
ClockPM- Surprise- LLActRep+ BwNot-
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
- LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt+ ABWMgmt-
+ LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive+ BWMgmt+ ABWMgmt-
SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug+ Surprise+
Slot #7, PowerLimit 10.000W; Interlock- NoCompl+
SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet+ CmdCplt- HPIrq+ LinkChg-
Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
- SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet- Interlock-
+ SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
Changed: MRL- PresDet- LinkState+
RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
RootCap: CRSVisible-
[cut]# cd without_XHCI_with_EHCI_no_ACPIPHP_with_PCIEHP/enabled_MediaCard_reader/USB3
# diff -u -w lspci_vv_initial.txt lspci_vv_inserting_USB3_card.txt
--- lspci_vv_initial.txt 2013-03-07 23:19:21.000000000 +0100
+++ lspci_vv_inserting_USB3_card.txt 2013-03-07 23:21:01.000000000 +0100
@@ -297,17 +297,17 @@
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
MaxPayload 128 bytes, MaxReadReq 128 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
- LnkCap: Port #8, Speed 5GT/s, Width x1, ASPM L0s L1, Latency L0 <1us, L1 <16us
+ LnkCap: Port #8, Speed 5GT/s, Width x1, ASPM L0s L1, Latency L0 <512ns, L1 <16us
ClockPM- Surprise- LLActRep+ BwNot-
- LnkCtl: ASPM L1 Enabled; RCB 64 bytes Disabled- Retrain- CommClk-
+ LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
- LnkSta: Speed 2.5GT/s, Width x0, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
+ LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive+ BWMgmt+ ABWMgmt-
SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug+ Surprise+
Slot #7, PowerLimit 10.000W; Interlock- NoCompl+
SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet+ CmdCplt- HPIrq+ LinkChg-
Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
- SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet- Interlock-
- Changed: MRL- PresDet- LinkState-
+ SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
+ Changed: MRL- PresDet- LinkState+
RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
RootCap: CRSVisible-
RootSta: PME ReqID 0000, PMEStatus- PMEPending-
[cut]Quoted 10 lines
There are three cards on the table now:
pci 0000:11:00.0: [1095:3132] SiI 3132 Serial ATA Raid II Controller pci 0000:11:00.0: [1106:3403] VT6315 Series Firewire Controller pci 0000:11:00.0: [1033:0194] NEC Corporation uPD720200 USB 3.0 Host
ControllerI assume all the cards work fine if there's no hotplug, i.e., if the card is present at boot and you never remove it. Right?
Yes with 3.7 and earlier. I did not test any of them under the required
3.9-rc1 with attached devices. And, I thought you ;-) will tell me that
from reading ./without_XHCI_with_EHCI_no_ACPIPHP_with_PCIEHP/disabled_MediaCard_reader/USB3/dmesg_inserting_USB3_card_attempt_2.txt it looks that card won't work: ;-)
[ 755.519874] pciehp 0000:00:1c.7:pcie04: pcie_isr: intr_loc 8 [ 755.519877] pciehp 0000:00:1c.7:pcie04: Presence/Notify input change [ 755.519881] pciehp 0000:00:1c.7:pcie04: Card present on Slot(7) [ 755.519950] pciehp 0000:00:1c.7:pcie04: Surprise Removal [ 755.523258] pciehp 0000:00:1c.7:pcie04: check_link_active: lnk_status = 7011 [ 755.631276] pciehp 0000:00:1c.7:pcie04: pciehp_check_link_status: lnk_status = 7011 [ 755.631380] pci 0000:11:00.0: [1033:0194] type 00 class 0x0c0330 [ 755.631435] pci 0000:11:00.0: reg 10: [mem 0x00000000-0x00001fff 64bit] [ 755.631700] pci 0000:11:00.0: PME# supported from D0 D3hot [ 755.631711] pci 0000:11:00.0: PME# disabled [ 755.631763] pci 0000:11:00.0: calling quirk_usb_early_handoff+0x0/0x657 [ 755.631776] pci 0000:11:00.0: device not available (can't reserve [mem 0x00000000-0x00001fff 64bit]) [ 755.631777] pci 0000:11:00.0: Can't enable PCI device, BIOS handoff failed. [ 755.651366] pci 0000:11:00.0: BAR 0: assigned [mem 0xf6c00000-0xf6c01fff 64bit] [ 755.651391] pci 0000:11:00.0: BAR 0: set to [mem 0xf6c00000-0xf6c01fff 64bit] (PCI address [0xf6c00000-0xf6c01fff]) [ 755.651398] pcieport 0000:00:1c.7: PCI bridge to [bus 11-16] [ 755.651401] pcieport 0000:00:1c.7: bridge window [io 0xc000-0xdfff] [ 755.651408] pcieport 0000:00:1c.7: bridge window [mem 0xf6c00000-0xf7cfffff] [ 755.651413] pcieport 0000:00:1c.7: bridge window [mem 0xf0000000-0xf10fffff 64bit pref]
Quoted 7 lines
Quoted 4 lines
In Linux pciehp
still complains about Surprise Removal even when I insert the card for the very first time after a coldbootHmm. pciehp prints "Surprise Removal" whether you inserted or removed the card. Stupid driver.
Still you don't say if acpiphp driver is ever able to print something like that to tell the user that a surprise event happened. Until that I will never be sure if acpipho just did not report the issue or whether there is really no PresDet bug with acpiphp. From my experience the latter is true but still would like to hear that acpiphp is able to print a warning like pciehp. ;-)
The thing that pciehp always prints the message might mean that it always interprets some info from the chip in a wrong way? Unless the driver just prints is capabilities during its loading ...
Finally, both acpiphp and pciehp should be able to print same type of warnings to the user.
Quoted 7 lines
Quoted 3 lines
(so the ExpressCard slot is not completely dead but neither
sata_sil24norfw_ohcipicks up the device).I thought the only card with a problem was the USB3.0 card. But here you suggest that there *is* a problem with the SATA and Firewire cards. Can you describe that problem in one sentence?
One sentence? No. ;-)
None of the cards works when 'nousb' and while are disabled USB devices in BIOS (which can be altered at all, don't know whether that really disables all USB in BIOS or not, hence I used the 'nousb' to be sure).
The below appears to me like a USDB3 card not detected at PCI level. I would undersdtand ehci-pci could not pickup the device due to 'nousb' but
[ 155.846929] pciehp 0000:00:1c.7:pcie04: pcie_isr: intr_loc 8 [ 155.846933] pciehp 0000:00:1c.7:pcie04: Presence/Notify input change [ 155.846936] pciehp 0000:00:1c.7:pcie04: Card present on Slot(7) [ 155.847403] pciehp 0000:00:1c.7:pcie04: Surprise Removal [ 155.853089] pciehp 0000:00:1c.7:pcie04: check_link_active: lnk_status = 3011 [ 157.459550] pci 0000:11:00.0 id reading try 50 times with interval 20 ms to get ffffffff [ 157.459573] pciehp 0000:00:1c.7:pcie04: pciehp_check_link_status: lnk_status = 3011 [ 157.459580] pciehp 0000:00:1c.7:pcie04: Failed to check link status Same for eSATA card: [ 125.568940] pciehp 0000:00:1c.7:pcie04: pcie_isr: intr_loc 8 [ 125.568942] pciehp 0000:00:1c.7:pcie04: Presence/Notify input change [ 125.568946] pciehp 0000:00:1c.7:pcie04: Card present on Slot(7) [ 125.569213] pciehp 0000:00:1c.7:pcie04: Surprise Removal [ 125.573782] pciehp 0000:00:1c.7:pcie04: check_link_active: lnk_status = 3011 [ 127.181635] pci 0000:11:00.0 id reading try 50 times with interval 20 ms to get ffffffff [ 127.181659] pciehp 0000:00:1c.7:pcie04: pciehp_check_link_status: lnk_status = 3011 [ 127.181668] pciehp 0000:00:1c.7:pcie04: Failed to check link status Same for Firewire card: [ 211.755452] pciehp 0000:00:1c.7:pcie04: pcie_isr: intr_loc 8 [ 211.755455] pciehp 0000:00:1c.7:pcie04: Presence/Notify input change [ 211.755458] pciehp 0000:00:1c.7:pcie04: Card present on Slot(7) [ 211.755656] pciehp 0000:00:1c.7:pcie04: Surprise Removal [ 211.758989] pciehp 0000:00:1c.7:pcie04: check_link_active: lnk_status = 3011 [ 213.368708] pci 0000:11:00.0 id reading try 50 times with interval 20 ms to get ffffffff [ 213.368730] pciehp 0000:00:1c.7:pcie04: pciehp_check_link_status: lnk_status = 3011 [ 213.368737] pciehp 0000:00:1c.7:pcie04: Failed to check link status [ 218.439730] pciehp 0000:00:1c.7:pcie04: pcie_isr: intr_loc 8 [ 218.439733] pciehp 0000:00:1c.7:pcie04: Presence/Notify input change [ 218.439737] pciehp 0000:00:1c.7:pcie04: Card not present on Slot(7) [ 218.439877] pciehp 0000:00:1c.7:pcie04: Surprise Removal [ 218.442988] pciehp 0000:00:1c.7:pcie04: Disabling domain:bus:device=0000:11:00 [ 218.442990] pciehp 0000:00:1c.7:pcie04: pciehp_unconfigure_device: domain:bus:dev = 0000:11:00
There are more problem with the eSATA and FireWire Cards:
eSATA inserts and removals fool likely sata_sil24 driver so that the system cannot shutdown. It locks up during shutdown when it should have unmounted all non-root devices and is doing: "Remounting / readonly". USB HUB leds are turned off at that time, no local keyboard was working anymore either, no sysrq, short PWR button press gave on console: "Disabling IRQ #16. I had to hold PWR button for 5s to turn off completely. Provided there was no disk attached ever to the eSATA card I wonder what was so difficult for the driver. ;-)
Firewire card is completely undetected when MediacardReader is enabled in BIOS.
Quoted 4 lines
Quoted 3 lines
My question is. Has the laptop hardwired the ExpressCard slot somehow through USB to the SandyBridge chip?
Quoted 8 lines
An ExpressCard slot (spec at [1]) supports both a PCIe interface and a USB interface, so the slot *should* be connected to a USB controller as well as to a PCIe root port. An ExpressCard can contain either a PCIe device or a USB device or both. Section 6.3 of the spec talks about ACPI requirements to describe the relationship between the PCIe and USB devices. I'm not sure that Linux pays any attention to this in the hotplug paths, so I'm a little worried about this. (Maybe it doesn't need to in the PCIe-aware case; I don't know.)
They don't work if I disable as much USB stuff in BIOS as possible and disable USB in kernel via 'nousb'.
BTW I used to have for a few days an RS232/LPT express card
but that one was labeled on a package as using a USB bus and it really used some USB driver. I returned the crap as it just eate up my express card slot. There is plenty of USB dongles providing serial/parallel port and those can eat up just a port in an external USB hub and leave with an unoccupied express card slot. I find it silly to fill up an express card slot (the only one in a laptop) with such a card. Back to the devices "on the table", I don't see the sata_sil24 nor the firewire card use any usb driver ehci/xhci and the vendor did not tell me they will be slow due to the usb overhead. I therefore conclude the SATA and FW cards are just PCIe capable.
So could a bad/misconfigured USB remnant in the USB3eSATA/FW card cause an EHCI bus reset, trigerring that a MediaCardReader is detected by Linux? Note that I am not saying re-detected. MediaCardReader really did not appear in lsusb nor dmesg until the USB3 card was inserted into the slot.
Quoted 5 lines
It would be interesting to know exactly what devices are on your cards. Assuming they all work when present at boot, you could find that by doing a single "lspci -vv" and "lsusb -v" after a boot with an empty slot, and doing it again after a boot with a card in the slot.
I will try this later. Now, I can conclude that somehow, PCI Express Port
disappears from lspci when I disable in BIOS all USB stuff which can be
tweaked. Importantly, I left enabled expresscard slot support. To ensure
you I did not mess up, please do realize that in lspci you can see the
SATA Sil 3132 card being loaded in the express slot as 11:00.0. The slot
works! but why Expres Port 00:1c.4 is gone is you task now. ;-) The BIOS
tweaks disabled the internal TexasInstruments USB3 controller, so do not
worry about 0b:00.0 being gone. Please execute:
diff -u -w without_XHCI_with_EHCI_no_ACPIPHP_with_PCIEHP/enabled_MediaCard_reader/eSATA/lspci_vv_inserting_eSATA_card.txt without_XHCI_with_EHCI_no_ACPIPHP_with_PCIEHP/disabled_MediaCard_reader_and_OpticalDevice_and_Camera_and_external_USB_Ports_but_enabled_expresscard_microphone_eSATA/eSATA/lspci_vv_inserting_eSATA_card.txt
Further, regarding the USB3 card from NEC I wonder why there is:
# s=find . -name lspci* | while read f; do echo -n "$f "; done; ls -latr $s | awk '{print $9}' | while read ss; do echo $ss; cat $ss; done | grep 'IRQ 0'
Interrupt: pin A routed to IRQ 0
Interrupt: pin A routed to IRQ 0
Interrupt: pin A routed to IRQ 0
Interrupt: pin A routed to IRQ 0
Interrupt: pin A routed to IRQ 0
Interrupt: pin A routed to IRQ 0
Interrupt: pin A routed to IRQ 0
#
If you inspect the affected lspci filenames ;-) you can see that this was the second and third insert of the USB3 card into its slot (so not the very first during which it acquired IRQ 16) while MediaCardReader was disabled in BIOS. I do not see the IRQ 0 issue with eSATA nor Firewire cards under same BIOS setup.
But, when *MediaCardReader was enabled in BIOS* I got IRQ 0 upon the very first insert of the USB3 card as well. Still, does not happen with eSATA. The Firewire card appears dead under this BIOS/kernel setup.
Quoted 4 lines
The difference should be the ExpressCard devices. I'm sure this is buried in your tarball somewhere, but all I want is the info from a machine in default configuration -- MediaCard enabled, etc. Just the way a typical user would be using the machine.
Well, I spotted the issues in my reply above. Quite a lot more when the MediacardReader is enabled.
Quoted 12 lines
[1] http://www.usb.org/developers/expresscard/EC_specifications/ExpressCard_2_0_FINAL.pdf
Quoted 3 lines
The hotplug issue itself. I do not understand the PCI(e) hotplug, lspci output but why is there any difference between a cold booted status of an empty expresscard slot compared to the status when a card is unloaded?
In principle there shouldn't be any difference, but Linux isn't that good yet.
Quoted 2 lines
disabled_MediaCard_reader/USB3/lspci_vv_unloading_USB3_card.txt 2013-03-07 renamed from disabled_MediaCard_reader/USB3/lspci_vv_initial.txt 2013-03-07 22:27:30.000000000 +0100 +0
-0
Quoted 8 lines
Quoted 4 lines
disabled_MediaCard_reader/USB3/lspci_vv_unloading_USB3_card.txt 2013-03-07 renamed from disabled_MediaCard_reader/USB3/lspci_vv_initial.txt 2013-03-07 22:27:30.000000000 +0100 +1
-1- Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR- + Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ <SERR- <PERR-Shouldn't the MAbort been cleared sometimes? Doesn't this fool the PresDet interpretation in kernel?
I doubt it. Presence Detect is a very simple mechanism that basically just reports the current state of the CPPE# signal in the ExpressCard slot. There's no reason this should be related to MAbort.
Let me say it in another way. It is meaningless to have MAbort+ while the slot is physically empty. If the slot is empty the status bits should be cleared to some default state like on cold boot with no card inserted.
Second, while I don't understand kernel code nor PCI at all, I wouldn't be surprised on a general basis that some driver refuses to do some action if the slot report some error status. But I won't argue because I really do not know what MAbort really means and how other should interpret is status.
Until the kernel is not "there" then probably there is no explanation for these differences caused by BIOS change? Or are those just random differences? I always did a cold boot. I can't do more I think. :(
# diff -u -w without_XHCI_with_EHCI_no_ACPIPHP_with_PCIEHP/disabled_MediaCard_reader/eSATA/lspci_vv_inserting_eSATA_card.txt without_XHCI_with_EHCI_no_ACPIPHP_with_PCIEHP/enabled_MediaCard_reader/eSATA/lspci_vv_inserting_eSATA_card.txt
--- without_XHCI_with_EHCI_no_ACPIPHP_with_PCIEHP/disabled_MediaCard_reader/eSATA/lspci_vv_inserting_eSATA_card.txt 2013-03-07 22:53:14.000000000 +0100
+++ without_XHCI_with_EHCI_no_ACPIPHP_with_PCIEHP/enabled_MediaCard_reader/eSATA/lspci_vv_inserting_eSATA_card.txt 2013-03-07 23:31:42.000000000 +0100
@@ -287,7 +287,7 @@
I/O behind bridge: 0000c000-0000dfff
Memory behind bridge: f6c00000-f7cfffff
Prefetchable memory behind bridge: 00000000f0000000-00000000f10fffff
- Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
+ Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR+ <PERR-
BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: [40] Express (v2) Root Port (Slot+), MSI 00
@@ -296,7 +296,7 @@
DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
MaxPayload 128 bytes, MaxReadReq 128 bytes
- DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
+ DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
LnkCap: Port #8, Speed 5GT/s, Width x1, ASPM L0s L1, Latency L0 <512ns, L1 <16us
ClockPM- Surprise- LLActRep+ BwNot-
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
@@ -521,7 +521,7 @@
Subsystem: Silicon Image, Inc. SiI 3132 Serial ATA Raid II Controller
Physical Slot: 7
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR+ <PERR- INTx-
Latency: 0, Cache Line Size: 32 bytes
Interrupt: pin A routed to IRQ 19
Region 0: Memory at f6c04000 (64-bit, non-prefetchable) [size=128]
@@ -539,17 +539,17 @@
DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
MaxPayload 128 bytes, MaxReadReq 4096 bytes
- DevSta: CorrErr- UncorrErr+ FatalErr- UnsuppReq+ AuxPwr- TransPend-
+ DevSta: CorrErr+ UncorrErr+ FatalErr+ UnsuppReq+ AuxPwr- TransPend-
LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s, Latency L0 unlimited, L1 unlimited
ClockPM- Surprise- LLActRep- BwNot-
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
Capabilities: [100 v1] Advanced Error Reporting
- UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol-
+ UESta: DLP+ SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
+ CESta: RxErr- BadTLP- BadDLLP+ Rollover- Timeout- NonFatalErr-
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
AERCap: First Error Pointer: 14, GenCap+ CGenEn- ChkCap+ ChkEn-
Kernel driver in use: sata_sil24Quoted 9 lines
Quoted 5 lines
The eSATA behaves differently maybe because 0100 does not change to 0140 like USB3 card but to 0103? Um, the shell while loop calling setpci does not report 0103 but the driver say this:
[ 211.879397] sata_sil24 0000:11:00.0: enabling device (0100 -> 0103)This is completely unrelated. The shell "setpci" command is printing the Slot Status register; the "0100 -> 0103" above is a change in the PCI Command register.
OK, I mixed up two different things, sorry.
Quoted 10 lines
Quoted 5 lines
I do not remember with 3.2.x and 3.7x kernels seeing this with the USB3 card:
+[ 594.622211] pci 0000:11:00.0: calling quirk_usb_early_handoff+0x0/0x657 +[ 594.622223] pci 0000:11:00.0: device not available (can't reserve [mem 0x00000000-0x00001fff 64bit]) +[ 594.622225] pci 0000:11:00.0: Can't enable PCI device, BIOS handoff failed.This is a result of trying to run the quirk on a hot-inserted device where we haven't assigned resources to it yet. I don't think we should really be running quirks on a device that early. We can look at that later, if we think this is related to the immediate problem.
Unless you make me to boot again into 3.9-rc1 I will believe the USB3 card won't work under that kernel with pciehp at all, or at least while xhci is disabled in kernel as was your requirement. The message looks scary to me but I don't care now either. ;)
I think you really should poke through the files and check their diffs, at least briefly. They have timestamps and the meaning of filenames should be clear after you realize there are for each state of the computer three files:
dmesg, lspci, slot_status. From their contents you can always learn whether card was in the slot or not.
Martin
Quoted 9 lines · Martin Mokrejs
Quoted 3 lines
I thought the only card with a problem was the USB3.0 card. But here you suggest that there *is* a problem with the SATA and Firewire cards. Can you describe that problem in one sentence?
One sentence? No. ;-)
None of the cards works when 'nousb' and while are disabled USB devices in BIOS (which can be altered at all, don't know whether that really disables all USB in BIOS or not, hence I used the 'nousb' to be sure).
Martin:
I don't know about Bjorn, but I find it very difficult to work on more than one bug at a time. Since your low-level PCI hotplug problems seem to be more fundamental than the USB problems, I'll wait until the PCI part is under control before trying to contribute.
Alan Stern
Quoted 12 lines · Alan Stern
Quoted 9 lines · Martin Mokrejs
Quoted 3 lines
I thought the only card with a problem was the USB3.0 card. But here you suggest that there *is* a problem with the SATA and Firewire cards. Can you describe that problem in one sentence?
One sentence? No. ;-)
None of the cards works when 'nousb' and while are disabled USB devices in BIOS (which can be altered at all, don't know whether that really disables all USB in BIOS or not, hence I used the 'nousb' to be sure).
Quoted 6 lines
Martin:
I don't know about Bjorn, but I find it very difficult to work on more than one bug at a time. Since your low-level PCI hotplug problems seem to be more fundamental than the USB problems, I'll wait until the PCI part is under control before trying to contribute.
Alan, for me it is even more difficult because I really do not know what are the hardware details about nor what is an OS kernel. I really wanted you pickup anything you see broken in the collected data and we work on those bug separately. But I am not able to judge what if broken and what is not.
But I believe you could always say: Hey, if the eSATA or Firewire is USB-based (unlike PCIe based) it would have to use usb-storage and blah. I think you can try to come up with answer why USB-related changes disable PCI Express Root port or whether that was the 'nousb' outcome. I doubt PCI people will dive into that area. ;)
But thank you for your time on this.
Martin
Quoted 11 lines · Martin Mokrejs
Alan, for me it is even more difficult because I really do not know what are the hardware details about nor what is an OS kernel. I really wanted you pickup anything you see broken in the collected data and we work on those bug separately. But I am not able to judge what if broken and what is not.
But I believe you could always say: Hey, if the eSATA or Firewire is USB-based (unlike PCIe based) it would have to use usb-storage and blah. I think you can try to come up with answer why USB-related changes disable PCI Express Root port or whether that was the 'nousb' outcome. I doubt PCI people will dive into that area. ;)
As far as I can tell, neither the eSATA nor the Firewire is connected with USB -- except for the fact that the eSATA card also contains a USB controller. Maybe I have misunderstood.
Bjorn, can you explain the exact connection?
As for disabling USB in the BIOS configuration... What that does isn't clear. It might turn off the USB hardware entirely, so that the kernel can't detect it at all. Or it might leave the hardware turned on but tell the BIOS not to use it.
Do you have lspci output from an older kernel where this stuff all worked correctly?
Alan Stern
Quoted 23 lines · Bjorn Helgaas
On Thu, Mar 7, 2013 at 6:47 PM, Martin Mokrejs <[email protected]> wrote:
There are three cards on the table now:
pci 0000:11:00.0: [1095:3132] SiI 3132 Serial ATA Raid II Controller pci 0000:11:00.0: [1106:3403] VT6315 Series Firewire Controller pci 0000:11:00.0: [1033:0194] NEC Corporation uPD720200 USB 3.0 Host
ControllerQuoted 2 lines
My question is. Has the laptop hardwired the ExpressCard slot somehow through USB to the SandyBridge chip?
An ExpressCard slot (spec at [1]) supports both a PCIe interface and a USB interface, so the slot *should* be connected to a USB controller as well as to a PCIe root port. An ExpressCard can contain either a PCIe device or a USB device or both. Section 6.3 of the spec talks about ACPI requirements to describe the relationship between the PCIe and USB devices. I'm not sure that Linux pays any attention to this in the hotplug paths, so I'm a little worried about this. (Maybe it doesn't need to in the PCIe-aware case; I don't know.)
Quoted 4 lines
It would be interesting to know exactly what devices are on your cards. Assuming they all work when present at boot, you could find that by doing a single "lspci -vv" and "lsusb -v" after a boot with an empty slot, and doing it again after a boot with a card in the slot.
There is no USB device hidden in the eSATA card (SiI 3132) and lspci -vvvxxx does not reveal any other device either. I also failed to ind any hidden interface in the Firewire card (based on VIA VT6315) both using lspci -vvvxxx and lsusb -vv and comparing outputs gather upon coldplug with a card in a slot and an empty slot. I even compared to states with a hotplugged card.
The only difference I would mention is that acpiphp vs. pciehp give:
- Physical Slot: 1
+ Physical Slot: 7
for whatever card is placed in the card slot.
Sometimes there is a difference in
- Latency: 0, Cache Line Size: 64 bytes
+ Latency: 0, Cache Line Size: 32 bytes
- AERCap: First Error Pointer: 00, ...
+ AERCap: First Error Pointer: 14, ...
Quoted 6 lines
The difference should be the ExpressCard devices. I'm sure this is buried in your tarball somewhere, but all I want is the info from a machine in default configuration -- MediaCard enabled, etc. Just the way a typical user would be using the machine.
[1] http://www.usb.org/developers/expresscard/EC_specifications/ExpressCard_2_0_FINAL.pdf
I tried to find some differences when I have Enabled (default) or Disables Media Card Reader in BIOS. Unfortunately I had also disabled Intel WIFI and Bluetooth via Fn+F2 (there is no BIOS entry for it so I forgot about the hotkey). The wifi can be seen in lsusb and lspci. The bluetooth ... I don't know, looks btusb driver is using it.
The MediaCard Reader device appears only in lsusb. If I disable it in BIOS its port is left powered on. Below, one of the changes is the disabled wifi, the other is the card reader being gone.
lsusb_vv.txt 2013-03-11 22:49:51.000000000 +0100 renamed from lsusb_vv_after_HDD_attached.txt 2013-03-11 18:12:05.000000000 +0100 +2 -71
@@ -62,7 +62,7 @@ Port 1: 0000.0100 power Port 2: 0000.0100 power Port 3: 0000.0100 power - Port 4: 0000.0103 power enable connect + Port 4: 0000.0100 power Port 5: 0000.0503 highspeed power enable connect Port 6: 0000.0100 power Device Qualifier (for other device speed):
@@ -142,7 +142,7 @@ Port 3: 0000.0103 power enable connect Port 4: 0000.0100 power Port 5: 0000.0100 power - Port 6: 0000.0503 highspeed power enable connect + Port 6: 0000.0100 power Port 7: 0000.0100 power Port 8: 0000.0100 power Device Qualifier (for other device speed):
@@ -1568,76 +1318,7 @@ Device Status: 0x0000 (Bus Powered) -Bus 002 Device 005: ID 0bda:0138 Realtek Semiconductor Corp. RTS5138 Card Reader Controller -Device Descriptor: - bLength 18 - bDescriptorType 1 - bcdUSB 2.00 - bDeviceClass 0 (Defined at Interface level) - bDeviceSubClass 0 - bDeviceProtocol 0 - bMaxPacketSize0 64 - idVendor 0x0bda Realtek Semiconductor Corp. - idProduct 0x0138 RTS5138 Card Reader Controller - bcdDevice 38.82 - iManufacturer 1 Generic - iProduct 2 USB2.0-CRW - iSerial 3 20090516388200000 - bNumConfigurations 1 - Configuration Descriptor: - bLength 9 - bDescriptorType 2 - wTotalLength 32 - bNumInterfaces 1 - bConfigurationValue 1 - iConfiguration 4 CARD READER - bmAttributes 0xa0 - (Bus Powered) - Remote Wakeup - MaxPower 500mA - Interface Descriptor: - bLength 9 - bDescriptorType 4 - bInterfaceNumber 0 - bAlternateSetting 0 - bNumEndpoints 2 - bInterfaceClass 8 Mass Storage - bInterfaceSubClass 6 SCSI - bInterfaceProtocol 80 Bulk-Only - iInterface 5 Bulk-In, Bulk-Out, Interface - Endpoint Descriptor: - bLength 7 - bDescriptorType 5 - bEndpointAddress 0x01 EP 1 OUT - bmAttributes 2 - Transfer Type Bulk - Synch Type None - Usage Type Data - wMaxPacketSize 0x0200 1x 512 bytes - bInterval 0 - Endpoint Descriptor: - bLength 7 - bDescriptorType 5 - bEndpointAddress 0x82 EP 2 IN - bmAttributes 2 - Transfer Type Bulk - Synch Type None - Usage Type Data - wMaxPacketSize 0x0200 1x 512 bytes - bInterval 0 -Device Qualifier (for other device speed): - bLength 10 - bDescriptorType 6 - bcdUSB 2.00 - bDeviceClass 0 (Defined at Interface level) - bDeviceSubClass 0 - bDeviceProtocol 0 - bMaxPacketSize0 64 - bNumConfigurations 1 -Device Status: 0x0000 - (Bus Powered) -
[cut]
Looks that is all I could add to the whole story.
Martin