XHCI debug device is not detected after debug target enables it

Hi,

I have a setup where XHCI console is used to debug Xen on a laptop. There are two system involved:
1. SUT - an x86 laptop running Xen, and configured to expose console over XHCI debug capability
2. debugger - Raspberry Pi 4B to which the debug cable is connected and where the console can be accessed

The XHCI debug cable is a simple USB3 A-A cable, with D+, D- and Vbus pins cut. When SUT isn't configured to serve debug console (for example during boot while in firmware), debugger complains about connected device, like this:

usb usb2-port2: config error
usb usb2-port2: Cannot enable. Maybe the USB cable is bad? usb usb2-port2: Cannot enable. Maybe the USB cable is bad? usb usb2-port2: Cannot enable. Maybe the USB cable is bad? usb usb2-port2: Cannot enable. Maybe the USB cable is bad? usb usb2-port2: attempt power cycle
usb usb2-port2: Cannot enable. Maybe the USB cable is bad? usb usb2-port2: Cannot enable. Maybe the USB cable is bad? usb usb2-port2: unable to enumerate USB device usb usb2-port2: config error

This is expected, as two USB hosts are connected together here. But once Xen starts and configures XHCI debug console, it's supposed to be detected as ttyUSB0. When it works, it looks like this:

usb 2-2: new SuperSpeed USB device number 11 using xhci_hcd
usb 2-2: LPM exit latency is zeroed, disabling LPM.
usb 2-2: New USB device found, idVendor=1d6b, idProduct=0010, bcdDevice= 0.00 usb 2-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3 usb 2-2: Product: Debug console
usb 2-2: Manufacturer: Xen
usb 2-2: SerialNumber: 0
usb_debug 2-2:1.0: xhci_dbc converter detected
usb 2-2: xhci_dbc converter now attached to ttyUSB0

This worked fine with 5.15.92 (from Raspberry Pi fork) on debugger. It also works fine with RPi debugger replaced with another x86 box (regardless of the kernel version). But after updating RPi to 6.6.78 (also RPi fork) it stopped working. Updating further to 6.12.48 (RPi) or even 6.16.7 (vanilla) didn't fixed it either. I didn't test vanilla 5.15, but I find it unlikely it worked only due to some RPi patch before.

When it's broken, SUT waits for the XHCI to enter "configured" state, IIUC in this loop:
https://gitlab.com/xen-project/xen/-/blob/staging/xen/drivers/char/xhci-dbc.c?ref_type=heads#L864

There are two options to fix it in this state:
- unplugging the cable and plugging it back
- _reading_ /sys/bus/usb/devices/usb2/2-0:1.0/usb2-port2/disable

In the latter case, kernel prints this (note the first line):

xhci_hcd 0000:01:00.0: port 2-2 resume PLC timeout
usb 2-2: new SuperSpeed USB device number 11 using xhci_hcd
usb 2-2: LPM exit latency is zeroed, disabling LPM.
usb 2-2: New USB device found, idVendor=1d6b, idProduct=0010, bcdDevice= 0.00 usb 2-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3 usb 2-2: Product: Debug console
usb 2-2: Manufacturer: Xen
usb 2-2: SerialNumber: 0
usb_debug 2-2:1.0: xhci_dbc converter detected
usb 2-2: xhci_dbc converter now attached to ttyUSB0

I can 100% reliably reproduce the issue on warm reboot. On cold boot usually it works.

Any ideas?

Quoted 68 lines · Marek Marczykowski-Górecki

Hi,

I have a setup where XHCI console is used to debug Xen on a laptop. There are two system involved:
1. SUT - an x86 laptop running Xen, and configured to expose console over XHCI debug capability
2. debugger - Raspberry Pi 4B to which the debug cable is connected and where the console can be accessed

The XHCI debug cable is a simple USB3 A-A cable, with D+, D- and Vbus pins cut. When SUT isn't configured to serve debug console (for example during boot while in firmware), debugger complains about connected device, like this:

usb usb2-port2: config error
usb usb2-port2: Cannot enable. Maybe the USB cable is bad? usb usb2-port2: Cannot enable. Maybe the USB cable is bad? usb usb2-port2: Cannot enable. Maybe the USB cable is bad? usb usb2-port2: Cannot enable. Maybe the USB cable is bad? usb usb2-port2: attempt power cycle
usb usb2-port2: Cannot enable. Maybe the USB cable is bad? usb usb2-port2: Cannot enable. Maybe the USB cable is bad? usb usb2-port2: unable to enumerate USB device usb usb2-port2: config error

This is expected, as two USB hosts are connected together here. But once Xen starts and configures XHCI debug console, it's supposed to be detected as ttyUSB0. When it works, it looks like this:

usb 2-2: new SuperSpeed USB device number 11 using xhci_hcd
usb 2-2: LPM exit latency is zeroed, disabling LPM.
usb 2-2: New USB device found, idVendor=1d6b, idProduct=0010, bcdDevice= 0.00 usb 2-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3 usb 2-2: Product: Debug console
usb 2-2: Manufacturer: Xen
usb 2-2: SerialNumber: 0
usb_debug 2-2:1.0: xhci_dbc converter detected
usb 2-2: xhci_dbc converter now attached to ttyUSB0

This worked fine with 5.15.92 (from Raspberry Pi fork) on debugger. It also works fine with RPi debugger replaced with another x86 box (regardless of the kernel version). But after updating RPi to 6.6.78 (also RPi fork) it stopped working. Updating further to 6.12.48 (RPi) or even 6.16.7 (vanilla) didn't fixed it either. I didn't test vanilla 5.15, but I find it unlikely it worked only due to some RPi patch before.

When it's broken, SUT waits for the XHCI to enter "configured" state, IIUC in this loop:
https://gitlab.com/xen-project/xen/-/blob/staging/xen/drivers/char/xhci-dbc.c?ref_type=heads#L864

There are two options to fix it in this state:
- unplugging the cable and plugging it back
- _reading_ /sys/bus/usb/devices/usb2/2-0:1.0/usb2-port2/disable

In the latter case, kernel prints this (note the first line):

xhci_hcd 0000:01:00.0: port 2-2 resume PLC timeout
usb 2-2: new SuperSpeed USB device number 11 using xhci_hcd
usb 2-2: LPM exit latency is zeroed, disabling LPM.
usb 2-2: New USB device found, idVendor=1d6b, idProduct=0010, bcdDevice= 0.00 usb 2-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3 usb 2-2: Product: Debug console
usb 2-2: Manufacturer: Xen
usb 2-2: SerialNumber: 0
usb_debug 2-2:1.0: xhci_dbc converter detected
usb 2-2: xhci_dbc converter now attached to ttyUSB0

Quoted 5 lines

I can 100% reliably reproduce the issue on warm reboot. On cold boot usually it works.

Any ideas?

Best guess is that the Raspberry PI, debugger xHC is runtime suspended.

Reconnecting the cable, or reading the port "disable" sysfs entry causes the host to resume, run, and detect the connected device.

The xHCI host is probably allowed to suspend after several failed enumeration attempts, which is expected due to the connected device also being in host mode.

I think runtime pm support was enabled as default for most xHCI hosts around late 5.x, early 6.x kernels, so that would fit.

Does preventing the roothub, and thus the host from runtime suspend help:

On the Raspberry Pi debugger:
echo on > /sys/bus/usb/devices/usb2/power/control