New: qca6390 bluetooth fails after disabling/re-enabling bluetooth

https://bugzilla.kernel.org/show_bug.cgi?id=218726

            Bug ID: 218726
           Summary: qca6390 bluetooth fails after disabling/re-enabling
                    bluetooth
           Product: Drivers
           Version: 2.5
    Kernel Version: 6.9.0-rc4
          Hardware: IA-64
                OS: Linux
            Status: NEW
          Severity: normal
          Priority: P3
         Component: Bluetooth
          Assignee: [email protected]
          Reporter: [email protected]
        Regression: No

I have a Dell XPS 13 9310 laptop. This laptop has a QCA6390 bluetooth device. The device works on a cold boot. However, it fails whenever disabling and re-enabling bluetooth (e.g. by cycling bluetooth in the GNOME UI).

You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=218726

--- Comment #1 from Warren Turkal ([email protected]) --- Created attachment 306152
--> https://bugzilla.kernel.org/attachment.cgi?id=306152&action=edit
6.9.0-rc4 + logging patch - kernel logs during bt disable/enable

I have applied the patchset at
https://patchwork.kernel.org/project/bluetooth/list/?series=844357 on top of 6.9.0-rc4. Here's the kenrnel logs starting just before disabling bluetooth via the GNOME UI.

I pretty consistently got a very similar log to the above, so I only attached the one copy.

You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=218726

--- Comment #2 from Warren Turkal ([email protected]) --- How logs were captured

1) Apply below patch series
https://patchwork.kernel.org/project/bluetooth/list/?series=844357

2) Disable BT

3) Power off

4) Power on

5) enable more logs

after boot:
echo "module hci_uart +pft, module btqca +pft" > /sys/kernel/debug/dynamic_debug/control

as kernel args:
dyndbg="module hci_uart +pft; module btqca +pft"

6) enable BT

7) then check this issue again. several disable/enable or reboot cycles.

As for step 7, I will investigate other disable/enable reboot cycles if I find differences.

You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=218726

--- Comment #3 from Warren Turkal ([email protected]) --- Created attachment 306153
--> https://bugzilla.kernel.org/attachment.cgi?id=306153&action=edit
6.9.0-rc4+logging after warm boot since boot

The dyndbg didn't seem to add many messages in this one. Maybe I have the wrong kernel parm to enable the dynamic debugging messages?

After a warm boot, the bluetooth won't enable. This log includes everything past trying to enable bluetooth twice in GNOME. After the second attempt to enable bluetooth, I didn't see additional kernel logs.

You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=218726

--- Comment #4 from Warren Turkal ([email protected]) ---

This message appears to be wrong. I believe the new messages are in the log. I guess it just doesn't try to load the firmware and is much shorter.

You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=218726

Zijun Hu ([email protected]) changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |[email protected]

--- Comment #5 from Zijun Hu ([email protected]) --- it will not print below logs if the patches are applied successfully.

Apr 12 00:30:05 braindead.localdomain kernel: Bluetooth: hci0: setting up

ROME/QCA6390

You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=218726

--- Comment #6 from Zijun Hu ([email protected]) --- have reverted below commit and update a new patchset

gpiod_get_optional()")

You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=218726

--- Comment #7 from Zijun Hu ([email protected]) --- the new patchset is
https://patchwork.kernel.org/project/bluetooth/patch/[email protected]/

You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=218726

--- Comment #8 from Wren Turkal ([email protected]) ---
In the first log (after cold boot), I only captures shortly before toggling the bluetooth. In the second lot (after warm boot), I enabled them with the kernel args. I definitely see a message from your patch in the second log.

Do you want to capture additional logging?

I am applying and trying your new patchset. I am applying it on to of the logging patches. I will then use the kernel arg and try to capture as much as I can on a cold boot.

You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=218726

--- Comment #9 from Wren Turkal ([email protected]) --- (In reply to Zijun Hu from comment #7)

This patchset does not apply to an upstream commit on Linus' tree, and I see context that differs on the files that the patch changes. What is the parent of your commit? It also does not apply to the bluetooth-next repo master branch? What is the parent of this commit?

You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=218726

--- Comment #10 from Wren Turkal ([email protected]) ---
So, I tried just reverting the commit (56d074d26c58) you mentioned on the mailing list. That seems to have worked. I am building with that revert. However, you debug messages patch conflicts, so I do not have the extra logging now. It's just mainline + the revert of the commit.

Is this what you were asking me to try? If so, please confirm. If not, what did you actually want me to try?

You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=218726

--- Comment #11 from Wren Turkal ([email protected]) ---
Okay, after reverting 56d074d26c58, disable/enable bluetooth appears to work after a cold boot. I haven't tested a warm boot yet.

You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=218726

--- Comment #12 from Wren Turkal ([email protected]) --- Created attachment 306160
--> https://bugzilla.kernel.org/attachment.cgi?id=306160&action=edit
commit cef27048e5 + revert 56d074d26c5828773 after warm boot

Bluetooth does not work after a warm boot.

This log is after a warm boot. Unfortunately, the following kernel arg didn't seem to enable the logs for the hci_uart and btqca args like I thought it would:
dyndbg="module hci_uart +pft; module btqca +pft"

I am trying to figure out how to turn the debug logs on as soon as the modules are loaded.

You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=218726

--- Comment #13 from Zijun Hu ([email protected]) ---
you maybe apply below two changes over the tip of bluetooth-next tree, i should have not conflict, then test by following the steps shared

// it have no change compared with the orignial patch
https://patchwork.kernel.org/project/bluetooth/patch/[email protected]/

// it has contained the change reverting commit 56d074d26c58 compared with the origninal patch
https://patchwork.kernel.org/project/bluetooth/patch/[email protected]/

you maybe firstly apply above two changes, then perform a power off then power on.
then test disable/enable or warm reboot.

please share logs if any cases are not working fine.

thanks

You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=218726

--- Comment #14 from Zijun Hu ([email protected]) --- for issue "Bluetooth does not work after a warm boot.", we need to log of function
qca_serdev_shutdown(). it is called by the shutdown phase of reboot.

You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=218726

--- Comment #15 from Wren Turkal ([email protected]) --- (In reply to Zijun Hu from comment #13)

That worked. I am currently building a new kernel to test.

You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=218726

--- Comment #16 from Wren Turkal ([email protected]) --- (In reply to Zijun Hu from comment #14)

I can capture that.

Does stopping the bluetooth service cause that to happen as well? Would unloaded certain modules cause it to happen?

You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=218726

--- Comment #17 from Zijun Hu ([email protected]) --- it is great.thank you.
not sure if other cases will trigger this feature.

I think this regression issue is related to below two fixes. will sumbmit formal patches to fix it when have spare time.

// it will cause disable/enable failure.

gpiod_get_optional()")

// it maybe cause BT enable failure after reboot.

serdev")

You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=218726

--- Comment #18 from Wren Turkal ([email protected]) ---
Okay. so I applied your patches you linked above to the tip of the bluetooth-next repo master branch. That appears to not include all the changes from Linus' master branch. When booting into that kernel, bluetooth works when I login to GNOME. However, bluetooth disable/enable does not work.

I also have the previous kernel that I built that was essentially 6.9.0-rc4 with the revert of 56d074d26c5828773 (git revert ...). Bluetooth works after a cold boot after I login to GNOME. This kernel allows disable/re-enable to work multiple times. However, warm boots do not work.

So now I have some questions:
* Are there fixes in Linus' kernel that are not in the bluetooth-next/master kernel?
* Considering the only functional difference between my revert and your patch is the bottom hunk of the second patch you linked
(https://patchwork.kernel.org/project/bluetooth/patch/[email protected]/),
does that the difference between disable/re-enable working have something to do with the quirk added by that hunk?

You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=218726

--- Comment #19 from Wren Turkal ([email protected]) ---
I am trying the bare revert on bluetooth-next/master now to see if not including the quirk hunk allows disable/re-enable like it does on 6.9.0-rc4.

You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=218726

--- Comment #20 from Zijun Hu ([email protected]) ---
i think it is not related to linus kernel.
the disable/re-enable working is not related to the quirk added by the hunk.

You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=218726

--- Comment #21 from Wren Turkal ([email protected]) ---
On bluetooth-next/master, I simply reverted 56d074d26c5828773 (just like I did on 6.9.0-rc4). This allows disabling/re-enabling bluetooth over and over. I think this indicates there is some kind of problem with that quirk handling hunk.

You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=218726

--- Comment #22 from Wren Turkal ([email protected]) ---
BTW, replicate my only change with "git revert 56d074d26c5828773" on top of bluetooth-next or on top of linus' kernel. They both start allowing disabling/re-enabling after a cold boot.

Warm boots still do not work on either kernel.

You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=218726

--- Comment #23 from Wren Turkal ([email protected]) --- Apparently getting that log is not going to be as easy as I thought. I tried to capture it from systemd journal on the next boot, but for some reason the log appears to be incomplete.

You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=218726

--- Comment #24 from Wren Turkal ([email protected]) ---
Okay, I think what I am actually seeing is that the timestamps on early kernel logs in systemd's journal have the wrong timestamp. I am not sure why or how to fix.

Anyway, let me try applying your patches and getting a full log from boot to shutdown.

You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=218726

Wren Turkal ([email protected]) changed:

What |Removed |Added
----------------------------------------------------------------------------

 Attachment #306152|0                           |1
        is obsolete|                            |
 Attachment #306153|0                           |1
        is obsolete|                            |
 Attachment #306160|0                           |1
        is obsolete|                            |

--- Comment #25 from Wren Turkal ([email protected]) --- Created attachment 306171
--> https://bugzilla.kernel.org/attachment.cgi?id=306171&action=edit
00567f70051a41 + Zijun's 2 patches + cold_boot + toggle bt

This is a log from startup to shutdown of bluetooth-next/master + Zijin's patches after a cold boot. I toggled bluetooth, which failed. Then I shutdown.

You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=218726

--- Comment #26 from Wren Turkal ([email protected]) --- Created attachment 306172
--> https://bugzilla.kernel.org/attachment.cgi?id=306172&action=edit
00567f70051a41 + Zijun's 2 patches + cold_boot

This is a log from startup to shutdown of bluetooth-next/master + Zijin's patches after a cold boot. Then I restart (for a warm boot next).

You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=218726

--- Comment #27 from Wren Turkal ([email protected]) --- Created attachment 306173
--> https://bugzilla.kernel.org/attachment.cgi?id=306173&action=edit
00567f70051a41 + Zijun's 2 patches + warm boot after working cold boot

This is a log from startup to shutdown of bluetooth-next/master + Zijin's patches after a warm boot that was restarted from a working cold boot. Bluetooth does not work. Then I shutdown.

You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=218726

--- Comment #28 from Wren Turkal ([email protected]) ---
The last three logs I added add a cold boot with failing toggle, a cold boot, and a warm boot.

On the cold boots, bluetooth works if I before I manually toggle it within GNOME.

I also tried Plasma, but didn't collect logs. Plasma kills bluetooth almost immediately after logging in.

Note that the timestamps appear to have the wrong hour until systemd is signaled. I collected these with the help of journalctl.

You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=218726

--- Comment #29 from Zijun Hu ([email protected]) --- thank you, Wren, for help verification and capturing logs. will check logs today later.

You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=218726

--- Comment #30 from Wren Turkal ([email protected]) --- Just so you know, I added the following parameters to my kernel boot:
btqca.dyndbg="+pmft" hci_uart.dyndbg="+pmft"

Does this get you all the logging you were looking for?

You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=218726

--- Comment #31 from Wren Turkal ([email protected]) --- (In reply to Zijun Hu from comment #29)

Okay, cool. Thx.

You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=218726

--- Comment #32 from Zijun Hu ([email protected]) --- there are two issues as described below.

1) cold boot -> enable BT -> disable BT -> 2nd Enable BT -> 2nd Disable BT.... BT can't be enabled for 2nd and later enabled operations.

this issue is caused by below commit. i have reverted it and submit a formal below patch
https://patchwork.kernel.org/project/bluetooth/patch/[email protected]/

gpiod_get_optional()")

2) cold boot (power off then power on) -> enable BT -> warm reboot after disable BT or NOT -> enable is failed to be Enable after warm reboot.

let us explain the 2) issue we are checking as below:
your device does not have H/W reset way since BT reset pin is not configured. so
qca_serdev_shutdown() needs to send VSC EDL_RESET_REQ to make controller return to initial clean state. the only reason for 2) happens is that EDL_RESET_REQ is not sent successfully during warm reset.
let me check it.

we can get logs for issue 2) by this way.
cold boot -> enable BT -> disable BT -> unload hci_uart.ko -> dmesg kernel log to see if EDL_RESET_REQ was sent successfully.

You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=218726

--- Comment #33 from Zijun Hu ([email protected]) --- (In reply to Wren Turkal from comment #30)

yes. the debugging logs are turned on successfully by you.

You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=218726

--- Comment #34 from Zijun Hu ([email protected]) --- Hi Wren,
for logs within comment #26, #27, #28. i don't find abnormal logs.

1)
could you share your drivers/bluetooth/hci_qca.c ?

2) how do you toggle BT? by UI or command line?

3) could you try by following below steps?

apply changes -> cold boot (power off then power on) -> rfkill list(share results) -> hciconfig -> enble BT if it is not enabled -> disable BT -> warm reboot -> rfkill list -> hciconfig -> enble BT if it is not enabled-> check if BT is able to be enabled succussfully?

You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=218726

--- Comment #35 from Zijun Hu ([email protected]) ---
Hi Wren,
sorry, it is my mistake, my 2nd debugging patch don't completely revert the Commit
56d074d26c58. i have updated that debugging patches.

you ONLY need to apply below to changes over the tip of bluetooth-next tree.

https://patchwork.kernel.org/project/bluetooth/patch/[email protected]/

https://patchwork.kernel.org/project/bluetooth/patch/[email protected]/

if there are conflict, you maybe git reset beluetooth-next tree into below commit and apply the above two changes.
e00fc2700a3f Bluetooth: btusb: Fix triggering coredump implementation for QCA

You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=218726

--- Comment #36 from Zijun Hu ([email protected]) ---
Hi Wren,
sorry, it is my mistake, my 2nd debugging patch don't completely revert the Commit
56d074d26c58. i have updated that debugging patches.

you ONLY need to apply below two changes over the tip of bluetooth-next tree.

https://patchwork.kernel.org/project/bluetooth/patch/[email protected]/

https://patchwork.kernel.org/project/bluetooth/patch/[email protected]/

if there are conflict, you maybe git reset beluetooth-next tree into below commit and apply the above two changes.
e00fc2700a3f Bluetooth: btusb: Fix triggering coredump implementation for QCA

You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=218726

--- Comment #37 from Wren Turkal ([email protected]) ---
It would be hard for you to rock more. This seems to fix everything. Cold boot and warm booting with mutliple disable/re-enable cycles work. I am now not getting any errors in my kernel logs when performing these actions.

You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=218726

--- Comment #38 from Wren Turkal ([email protected]) --- Can I add a Signed-off-by footer to the patch somehow?

You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=218726

--- Comment #39 from Zijun Hu ([email protected]) ---
good news. thank you @Wren Turkal again for co-working to debug this issue. of course. you can add it.
i also add your Signed-off-by for later updated patche serials for this issue.

You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=218726

--- Comment #40 from Wren Turkal ([email protected]) ---
I know this is a total newb question. How do I add the footer properly for the purposes of the log? Do you handle it or is there something I need to do?

You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=218726

--- Comment #41 from Zijun Hu ([email protected]) --- (In reply to Wren Turkal from comment #40)

they are added by fixes submitter. yes. nothing need to do for you currently. i will submit new patchset and add them for you based on the changes which take effect actually.

thanks

You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=218726

--- Comment #42 from Wren Turkal ([email protected]) ---
Okay. Let me know if you end up needing anything from me. I really appreciate your work on getting this fixed...谢谢

You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=218726

Paul Menzel ([email protected]) changed:

What |Removed |Added
----------------------------------------------------------------------------

                 CC|                            |pmenzel+bugzilla.kernel.org
                   |                            |@molgen.mpg.de

--- Comment #43 from Paul Menzel ([email protected]) ---

Shouldn’t

be the correct tag?

You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=218726

--- Comment #44 from Wren Turkal ([email protected]) ---
That sounds correct. Thanks for the correction. I really appreciate it, Paul.

You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=218726

--- Comment #45 from Wren Turkal ([email protected]) ---
BTW, I ported the revert patch to Linus' latest. I can confirm that it works there as well.

You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=218726

Wren Turkal ([email protected]) changed:

What |Removed |Added
----------------------------------------------------------------------------
Hardware|IA-64 |All

You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=218726

--- Comment #46 from Zijun Hu ([email protected]) --- this issue was discucced at below link before it was reported to bugzilla.

https://lore.kernel.org/linux-bluetooth/[email protected]/T/#m73d6a71d2f454bb03588c66f3ef7912274d37c6f

as commented at Commented 37.

it is verified by applied apply fix debugging changes over below bluetoot-next tree commit:
e00fc2700a3f Bluetooth: btusb: Fix triggering coredump implementation for QCA

it is verified by the reported device a Dell XPS 13 9310.

You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.