PCI: comm "swapper/0" leaking memory
Subsystems (3)
Hi,
I noticed that starting from v6.5-rc1 my ath11k tests reported several memory leaks from swapper/0:
unreferenced object 0xffff88810a02b7a8 (size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):
80 b8 02 0a 81 88 ff ff b8 72 07 00 00 c9 ff ff .........r......
c8 b7 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00 ................ backtrace:
unreferenced object 0xffff88810a02b880 (size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):
58 b9 02 0a 81 88 ff ff a8 b7 02 0a 81 88 ff ff X...............
a0 b8 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00 ................ backtrace:
unreferenced object 0xffff88810a02b958 (size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):
30 ba 02 0a 81 88 ff ff 80 b8 02 0a 81 88 ff ff 0...............
78 b9 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00 x............... backtrace:
unreferenced object 0xffff88810a02ba30 (size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):
08 bb 02 0a 81 88 ff ff 58 b9 02 0a 81 88 ff ff ........X.......
50 ba 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00 P............... backtrace:
unreferenced object 0xffff88810a02bb08 (size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):
e0 bb 02 0a 81 88 ff ff 30 ba 02 0a 81 88 ff ff ........0.......
28 bb 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00 (...............backtrace:
I can easily reproduce this by doing a simple insmod and rmmod of ath11k and it's dependencies (mac80211, MHI etc). I can reliability reproduce the leaks but I only see them once after a boot, I need to reboot the host to see the leaks again. v6.4 has no leaks.
I did a bisect and found the commit below. I verified reverting the commit makes the leaks go away.
commit e54223275ba1bc6f704a6bab015fcd2ae4f72572
Author: Ross Lagerwall <[email protected]> AuthorDate: Thu May 25 16:32:48 2023 +0100 Commit: Bjorn Helgaas <[email protected]> CommitDate: Fri Jun 9 15:06:16 2023 -0500
PCI: Release resource invalidated by coalescing
Kalle
Hi, Thorsten here, the Linux kernel's regression tracker. Partially top-posting for once, to make this easily accessible to everyone.
Thx for the report. To quote one aspect:
Quoted 1 line
v6.4 has no leaks.
I pretty sure you mean stock 6.4 here. But FWIW, the culprit was backported to 6.4.y and 6.1.y, so they latest versions might be affected as well.
For the rest of this mail:
[TLDR: I'm adding this report to the list of tracked Linux kernel regressions; the text you find below is based on a few templates paragraphs you might have encountered already in similar form. See link in footer if these mails annoy you.]
Quoted 43 lines · Kalle Valo
I noticed that starting from v6.5-rc1 my ath11k tests reported several memory leaks from swapper/0:
unreferenced object
0xffff88810a02b7a8(size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):80 b8 02 0a 81 88 ff ff b8 72 07 00 00 c9 ff ff .........r...... c8 b7 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00 ................backtrace:
unreferenced object0xffff88810a02b880(size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):58 b9 02 0a 81 88 ff ff a8 b7 02 0a 81 88 ff ff X............... a0 b8 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00 ................backtrace:
unreferenced object0xffff88810a02b958(size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):30 ba 02 0a 81 88 ff ff 80 b8 02 0a 81 88 ff ff 0............... 78 b9 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00 x...............backtrace:
unreferenced object0xffff88810a02ba30(size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):08 bb 02 0a 81 88 ff ff 58 b9 02 0a 81 88 ff ff ........X....... 50 ba 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00 P...............backtrace:
unreferenced object0xffff88810a02bb08(size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):e0 bb 02 0a 81 88 ff ff 30 ba 02 0a 81 88 ff ff ........0....... 28 bb 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00 (...............backtrace:
I can easily reproduce this by doing a simple insmod and rmmod of ath11k and it's dependencies (mac80211, MHI etc). I can reliability reproduce the leaks but I only see them once after a boot, I need to reboot the host to see the leaks again. v6.4 has no leaks.
I did a bisect and found the commit below. I verified reverting the commit makes the leaks go away.
Quoted 7 lines
commit
e54223275ba1bc6f704a6bab015fcd2ae4f72572Author: Ross Lagerwall <[email protected]> AuthorDate: Thu May 25 16:32:48 2023 +0100 Commit: Bjorn Helgaas <[email protected]> CommitDate: Fri Jun 9 15:06:16 2023 -0500PCI: Release resource invalidated by coalescing
To be sure the issue doesn't fall through the cracks unnoticed, I'm adding it to regzbot, the Linux kernel regression tracking bot:
#regzbot ^introduced e54223275ba1bc6f704a6bab015fcd2ae4f72572 #regzbot title PCI: comm "swapper/0" leaking memory #regzbot ignore-activity
This isn't a regression? This issue or a fix for it are already discussed somewhere else? It was fixed already? You want to clarify when the regression started to happen? Or point out I got the title or something else totally wrong? Then just reply and tell me -- ideally while also telling regzbot about it, as explained by the page listed in the footer of this mail.
Developers: When fixing the issue, remember to add 'Link:' tags pointing to the report (the parent of this mail). See page linked in footer for details.
Ciao, Thorsten (wearing his 'the Linux kernel's regression tracker' hat) --
Everything you wanna know about Linux kernel regression tracking:
https://linux-regtracking.leemhuis.info/about/#tldr
That page also explains what to do if mails like this annoy you.
Signature · 3 lines
ath11k mailing list [email protected] http://lists.infradead.org/mailman/listinfo/ath11k
"Linux regression tracking (Thorsten Leemhuis)"
Quoted 10 lines · <[email protected]> writes:
Hi, Thorsten here, the Linux kernel's regression tracker. Partially top-posting for once, to make this easily accessible to everyone.
Thx for the report. To quote one aspect:
Quoted 1 line
v6.4 has no leaks.
I pretty sure you mean stock 6.4 here. But FWIW, the culprit was backported to 6.4.y and 6.1.y, so they latest versions might be affected as well.
Yes, I mean v6.4 release tagged by Linus:
6995e2de6891 Linux 6.4
Signature · 3 lines
https://patchwork.kernel.org/project/linux-wireless/list/ https://wireless.wiki.kernel.org/en/developers/documentation/submittingpatches
Quoted 48 lines · Kalle Valo
[CAUTION - EXTERNAL EMAIL] DO NOT reply, click links, or open attachments unless you have verified the sender and know the content is safe.
Hi,
I noticed that starting from v6.5-rc1 my ath11k tests reported several memory leaks from swapper/0:
unreferenced object
0xffff88810a02b7a8(size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):80 b8 02 0a 81 88 ff ff b8 72 07 00 00 c9 ff ff .........r...... c8 b7 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00 ................backtrace:
unreferenced object0xffff88810a02b880(size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):58 b9 02 0a 81 88 ff ff a8 b7 02 0a 81 88 ff ff X............... a0 b8 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00 ................backtrace:
unreferenced object0xffff88810a02b958(size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):30 ba 02 0a 81 88 ff ff 80 b8 02 0a 81 88 ff ff 0............... 78 b9 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00 x...............backtrace:
unreferenced object0xffff88810a02ba30(size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):08 bb 02 0a 81 88 ff ff 58 b9 02 0a 81 88 ff ff ........X....... 50 ba 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00 P...............backtrace:
unreferenced object0xffff88810a02bb08(size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):e0 bb 02 0a 81 88 ff ff 30 ba 02 0a 81 88 ff ff ........0....... 28 bb 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00 (...............backtrace:
I can easily reproduce this by doing a simple insmod and rmmod of ath11k and it's dependencies (mac80211, MHI etc). I can reliability reproduce the leaks but I only see them once after a boot, I need to reboot the host to see the leaks again. v6.4 has no leaks.
I did a bisect and found the commit below. I verified reverting the commit makes the leaks go away.
commit
e54223275ba1bc6f704a6bab015fcd2ae4f72572
Quoted 9 lines
Author: Ross Lagerwall <[email protected]> AuthorDate: Thu May 25 16:32:48 2023 +0100 Commit: Bjorn Helgaas <[email protected]> CommitDate: Fri Jun 9 15:06:16 2023 -0500PCI: Release resource invalidated by coalescing
Kalle
Hi Kalle,
I can't reproduce the leak by loading/unloading the ath11k module. I suspect that the leak is always there when PCI resources are coalesced but kmemleak doesn't notice until ath11k is loaded.
Can you please try the following to confirm it fixes it?
Ross
8<-----------------------
drivers/pci/probe.c +0 -0
@@ -998,6 +998,7 @@ static int pci_register_host_bridge(struct
pci_host_bridge *bridge)
res = window->res;
if (!res->flags && !res->start && !res->end) {
release_resource(res);
+ resource_list_destroy_entry(window);
continue;
}Kalle Valo
Tested-byQuoted 53 lines · Ross Lagerwall
Quoted 43 lines · Kalle Valo
I noticed that starting from v6.5-rc1 my ath11k tests reported several memory leaks from swapper/0:
unreferenced object
0xffff88810a02b7a8(size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):80 b8 02 0a 81 88 ff ff b8 72 07 00 00 c9 ff ff .........r...... c8 b7 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00 ................backtrace:
unreferenced object0xffff88810a02b880(size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):58 b9 02 0a 81 88 ff ff a8 b7 02 0a 81 88 ff ff X............... a0 b8 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00 ................backtrace:
unreferenced object0xffff88810a02b958(size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):30 ba 02 0a 81 88 ff ff 80 b8 02 0a 81 88 ff ff 0............... 78 b9 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00 x...............backtrace:
unreferenced object0xffff88810a02ba30(size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):08 bb 02 0a 81 88 ff ff 58 b9 02 0a 81 88 ff ff ........X....... 50 ba 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00 P...............backtrace:
unreferenced object0xffff88810a02bb08(size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):e0 bb 02 0a 81 88 ff ff 30 ba 02 0a 81 88 ff ff ........0....... 28 bb 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00 (...............backtrace:
I can easily reproduce this by doing a simple insmod and rmmod of ath11k and it's dependencies (mac80211, MHI etc). I can reliability reproduce the leaks but I only see them once after a boot, I need to reboot the host to see the leaks again. v6.4 has no leaks.
I did a bisect and found the commit below. I verified reverting the commit makes the leaks go away.
Quoted 7 lines
commit
e54223275ba1bc6f704a6bab015fcd2ae4f72572Author: Ross Lagerwall <[email protected]> AuthorDate: Thu May 25 16:32:48 2023 +0100 Commit: Bjorn Helgaas <[email protected]> CommitDate: Fri Jun 9 15:06:16 2023 -0500PCI: Release resource invalidated by coalescing
Quoted 7 lines
Hi Kalle,
I can't reproduce the leak by loading/unloading the ath11k module. I suspect that the leak is always there when PCI resources are coalesced but kmemleak doesn't notice until ath11k is loaded.
Can you please try the following to confirm it fixes it?
I run various tests with your patch and I don't see leaks anymore. I also veried that without your patch I see the leak immediately.
Thanks for fixing this so quickly, it would good to have this fix in v6.6 if possible.
Signature · 3 lines
https://patchwork.kernel.org/project/linux-wireless/list/ https://wireless.wiki.kernel.org/en/developers/documentation/submittingpatches
Quoted 64 lines · Kalle Valo
Quoted 53 lines · Ross Lagerwall
Quoted 43 lines · Kalle Valo
I noticed that starting from v6.5-rc1 my ath11k tests reported several memory leaks from swapper/0:
unreferenced object
0xffff88810a02b7a8(size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):80 b8 02 0a 81 88 ff ff b8 72 07 00 00 c9 ff ff .........r...... c8 b7 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00 ................backtrace:
unreferenced object0xffff88810a02b880(size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):58 b9 02 0a 81 88 ff ff a8 b7 02 0a 81 88 ff ff X............... a0 b8 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00 ................backtrace:
unreferenced object0xffff88810a02b958(size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):30 ba 02 0a 81 88 ff ff 80 b8 02 0a 81 88 ff ff 0............... 78 b9 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00 x...............backtrace:
unreferenced object0xffff88810a02ba30(size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):08 bb 02 0a 81 88 ff ff 58 b9 02 0a 81 88 ff ff ........X....... 50 ba 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00 P...............backtrace:
unreferenced object0xffff88810a02bb08(size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):e0 bb 02 0a 81 88 ff ff 30 ba 02 0a 81 88 ff ff ........0....... 28 bb 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00 (...............backtrace:
I can easily reproduce this by doing a simple insmod and rmmod of ath11k and it's dependencies (mac80211, MHI etc). I can reliability reproduce the leaks but I only see them once after a boot, I need to reboot the host to see the leaks again. v6.4 has no leaks.
I did a bisect and found the commit below. I verified reverting the commit makes the leaks go away.
Quoted 7 lines
commit
e54223275ba1bc6f704a6bab015fcd2ae4f72572Author: Ross Lagerwall <[email protected]> AuthorDate: Thu May 25 16:32:48 2023 +0100 Commit: Bjorn Helgaas <[email protected]> CommitDate: Fri Jun 9 15:06:16 2023 -0500PCI: Release resource invalidated by coalescing
Quoted 7 lines
Hi Kalle,
I can't reproduce the leak by loading/unloading the ath11k module. I suspect that the leak is always there when PCI resources are coalesced but kmemleak doesn't notice until ath11k is loaded.
Can you please try the following to confirm it fixes it?
Quoted 8 lines
I run various tests with your patch and I don't see leaks anymore. I also veried that without your patch I see the leak immediately.
Thanks for fixing this so quickly, it would good to have this fix in v6.6 if possible.
Tested-by Kalle Valo <[email protected]>
Thanks, I will send out a proper patch now.
Ross