Skip to contentSkip to conversation

PCI: comm "swapper/0" leaking memory

regression v6.5-rc1 Kalle Valo 3 participants 6 messages Aug 30, 2023 – Sep 6, 2023

Subsystems (3)
lore
Share & tools
lore

Kalle Valo

author
#0 lore raw

Hi,

I noticed that starting from v6.5-rc1 my ath11k tests reported several memory leaks from swapper/0:

unreferenced object 0xffff88810a02b7a8 (size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):

    80 b8 02 0a 81 88 ff ff b8 72 07 00 00 c9 ff ff  .........r......
    c8 b7 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00  ................

backtrace:
unreferenced object 0xffff88810a02b880 (size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):

    58 b9 02 0a 81 88 ff ff a8 b7 02 0a 81 88 ff ff  X...............
    a0 b8 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00  ................

backtrace:
unreferenced object 0xffff88810a02b958 (size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):

    30 ba 02 0a 81 88 ff ff 80 b8 02 0a 81 88 ff ff  0...............
    78 b9 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00  x...............

backtrace:
unreferenced object 0xffff88810a02ba30 (size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):

    08 bb 02 0a 81 88 ff ff 58 b9 02 0a 81 88 ff ff  ........X.......
    50 ba 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00  P...............

backtrace:
unreferenced object 0xffff88810a02bb08 (size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):

    e0 bb 02 0a 81 88 ff ff 30 ba 02 0a 81 88 ff ff  ........0.......
    28 bb 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00  (...............

backtrace:

I can easily reproduce this by doing a simple insmod and rmmod of ath11k and it's dependencies (mac80211, MHI etc). I can reliability reproduce the leaks but I only see them once after a boot, I need to reboot the host to see the leaks again. v6.4 has no leaks.

I did a bisect and found the commit below. I verified reverting the commit makes the leaks go away.

commit e54223275ba1bc6f704a6bab015fcd2ae4f72572

Author:     Ross Lagerwall <[email protected]>
AuthorDate: Thu May 25 16:32:48 2023 +0100
Commit:     Bjorn Helgaas <[email protected]>
CommitDate: Fri Jun 9 15:06:16 2023 -0500

PCI: Release resource invalidated by coalescing

Kalle

#1 lore raw

Hi, Thorsten here, the Linux kernel's regression tracker. Partially top-posting for once, to make this easily accessible to everyone.

Thx for the report. To quote one aspect:

Quoted 1 line

v6.4 has no leaks.

I pretty sure you mean stock 6.4 here. But FWIW, the culprit was backported to 6.4.y and 6.1.y, so they latest versions might be affected as well.

For the rest of this mail:

[TLDR: I'm adding this report to the list of tracked Linux kernel regressions; the text you find below is based on a few templates paragraphs you might have encountered already in similar form. See link in footer if these mails annoy you.]

Quoted 43 lines · Kalle Valo

I noticed that starting from v6.5-rc1 my ath11k tests reported several memory leaks from swapper/0:

unreferenced object 0xffff88810a02b7a8 (size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):

    80 b8 02 0a 81 88 ff ff b8 72 07 00 00 c9 ff ff  .........r......
    c8 b7 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00  ................

backtrace:
unreferenced object 0xffff88810a02b880 (size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):

    58 b9 02 0a 81 88 ff ff a8 b7 02 0a 81 88 ff ff  X...............
    a0 b8 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00  ................

backtrace:
unreferenced object 0xffff88810a02b958 (size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):

    30 ba 02 0a 81 88 ff ff 80 b8 02 0a 81 88 ff ff  0...............
    78 b9 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00  x...............

backtrace:
unreferenced object 0xffff88810a02ba30 (size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):

    08 bb 02 0a 81 88 ff ff 58 b9 02 0a 81 88 ff ff  ........X.......
    50 ba 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00  P...............

backtrace:
unreferenced object 0xffff88810a02bb08 (size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):

    e0 bb 02 0a 81 88 ff ff 30 ba 02 0a 81 88 ff ff  ........0.......
    28 bb 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00  (...............

backtrace:

I can easily reproduce this by doing a simple insmod and rmmod of ath11k and it's dependencies (mac80211, MHI etc). I can reliability reproduce the leaks but I only see them once after a boot, I need to reboot the host to see the leaks again. v6.4 has no leaks.

I did a bisect and found the commit below. I verified reverting the commit makes the leaks go away.

Quoted 7 lines

commit e54223275ba1bc6f704a6bab015fcd2ae4f72572

Author:     Ross Lagerwall <[email protected]>
AuthorDate: Thu May 25 16:32:48 2023 +0100
Commit:     Bjorn Helgaas <[email protected]>
CommitDate: Fri Jun 9 15:06:16 2023 -0500

PCI: Release resource invalidated by coalescing

To be sure the issue doesn't fall through the cracks unnoticed, I'm adding it to regzbot, the Linux kernel regression tracking bot:

#regzbot ^introduced e54223275ba1bc6f704a6bab015fcd2ae4f72572 #regzbot title PCI: comm "swapper/0" leaking memory #regzbot ignore-activity

This isn't a regression? This issue or a fix for it are already discussed somewhere else? It was fixed already? You want to clarify when the regression started to happen? Or point out I got the title or something else totally wrong? Then just reply and tell me -- ideally while also telling regzbot about it, as explained by the page listed in the footer of this mail.

Developers: When fixing the issue, remember to add 'Link:' tags pointing to the report (the parent of this mail). See page linked in footer for details.

Ciao, Thorsten (wearing his 'the Linux kernel's regression tracker' hat) --
Everything you wanna know about Linux kernel regression tracking:
https://linux-regtracking.leemhuis.info/about/#tldr
That page also explains what to do if mails like this annoy you.

Signature · 3 lines
ath11k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath11k
author
#2 lore raw

"Linux regression tracking (Thorsten Leemhuis)"

Quoted 10 lines · <[email protected]> writes:

Hi, Thorsten here, the Linux kernel's regression tracker. Partially top-posting for once, to make this easily accessible to everyone.

Thx for the report. To quote one aspect:

Quoted 1 line

v6.4 has no leaks.

I pretty sure you mean stock 6.4 here. But FWIW, the culprit was backported to 6.4.y and 6.1.y, so they latest versions might be affected as well.

Yes, I mean v6.4 release tagged by Linus:

6995e2de6891 Linux 6.4

Signature · 3 lines
https://patchwork.kernel.org/project/linux-wireless/list/

https://wireless.wiki.kernel.org/en/developers/documentation/submittingpatches
#3 lore raw
In reply to Kalle Valo
Quoted 48 lines · Kalle Valo

[CAUTION - EXTERNAL EMAIL] DO NOT reply, click links, or open attachments unless you have verified the sender and know the content is safe.

Hi,

I noticed that starting from v6.5-rc1 my ath11k tests reported several memory leaks from swapper/0:

unreferenced object 0xffff88810a02b7a8 (size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):

    80 b8 02 0a 81 88 ff ff b8 72 07 00 00 c9 ff ff  .........r......
    c8 b7 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00  ................

backtrace:
unreferenced object 0xffff88810a02b880 (size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):

    58 b9 02 0a 81 88 ff ff a8 b7 02 0a 81 88 ff ff  X...............
    a0 b8 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00  ................

backtrace:
unreferenced object 0xffff88810a02b958 (size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):

    30 ba 02 0a 81 88 ff ff 80 b8 02 0a 81 88 ff ff  0...............
    78 b9 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00  x...............

backtrace:
unreferenced object 0xffff88810a02ba30 (size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):

    08 bb 02 0a 81 88 ff ff 58 b9 02 0a 81 88 ff ff  ........X.......
    50 ba 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00  P...............

backtrace:
unreferenced object 0xffff88810a02bb08 (size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):

    e0 bb 02 0a 81 88 ff ff 30 ba 02 0a 81 88 ff ff  ........0.......
    28 bb 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00  (...............

backtrace:

I can easily reproduce this by doing a simple insmod and rmmod of ath11k and it's dependencies (mac80211, MHI etc). I can reliability reproduce the leaks but I only see them once after a boot, I need to reboot the host to see the leaks again. v6.4 has no leaks.

I did a bisect and found the commit below. I verified reverting the commit makes the leaks go away.

commit e54223275ba1bc6f704a6bab015fcd2ae4f72572

Quoted 9 lines
Author:     Ross Lagerwall <[email protected]>
AuthorDate: Thu May 25 16:32:48 2023 +0100
Commit:     Bjorn Helgaas <[email protected]>
CommitDate: Fri Jun 9 15:06:16 2023 -0500

PCI: Release resource invalidated by coalescing

Kalle

Hi Kalle,

I can't reproduce the leak by loading/unloading the ath11k module. I suspect that the leak is always there when PCI resources are coalesced but kmemleak doesn't notice until ath11k is loaded.

Can you please try the following to confirm it fixes it?

Ross

8<-----------------------

drivers/pci/probe.c +0 -0
index 8bac3ce02609..907c873473e2 100644
@@ -998,6 +998,7 @@ static int pci_register_host_bridge(struct

pci_host_bridge *bridge)

         res = window->res;
         if (!res->flags && !res->start && !res->end) {
             release_resource(res);
+            resource_list_destroy_entry(window);
             continue;
         }
author Tested-by
#4 lore raw
Quoted 53 lines · Ross Lagerwall
Quoted 43 lines · Kalle Valo

I noticed that starting from v6.5-rc1 my ath11k tests reported several memory leaks from swapper/0:

unreferenced object 0xffff88810a02b7a8 (size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):

    80 b8 02 0a 81 88 ff ff b8 72 07 00 00 c9 ff ff  .........r......
    c8 b7 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00  ................

backtrace:
unreferenced object 0xffff88810a02b880 (size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):

    58 b9 02 0a 81 88 ff ff a8 b7 02 0a 81 88 ff ff  X...............
    a0 b8 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00  ................

backtrace:
unreferenced object 0xffff88810a02b958 (size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):

    30 ba 02 0a 81 88 ff ff 80 b8 02 0a 81 88 ff ff  0...............
    78 b9 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00  x...............

backtrace:
unreferenced object 0xffff88810a02ba30 (size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):

    08 bb 02 0a 81 88 ff ff 58 b9 02 0a 81 88 ff ff  ........X.......
    50 ba 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00  P...............

backtrace:
unreferenced object 0xffff88810a02bb08 (size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):

    e0 bb 02 0a 81 88 ff ff 30 ba 02 0a 81 88 ff ff  ........0.......
    28 bb 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00  (...............

backtrace:

I can easily reproduce this by doing a simple insmod and rmmod of ath11k and it's dependencies (mac80211, MHI etc). I can reliability reproduce the leaks but I only see them once after a boot, I need to reboot the host to see the leaks again. v6.4 has no leaks.

I did a bisect and found the commit below. I verified reverting the commit makes the leaks go away.

Quoted 7 lines

commit e54223275ba1bc6f704a6bab015fcd2ae4f72572

Author:     Ross Lagerwall <[email protected]>
AuthorDate: Thu May 25 16:32:48 2023 +0100
Commit:     Bjorn Helgaas <[email protected]>
CommitDate: Fri Jun 9 15:06:16 2023 -0500

PCI: Release resource invalidated by coalescing

Quoted 7 lines

Hi Kalle,

I can't reproduce the leak by loading/unloading the ath11k module. I suspect that the leak is always there when PCI resources are coalesced but kmemleak doesn't notice until ath11k is loaded.

Can you please try the following to confirm it fixes it?

I run various tests with your patch and I don't see leaks anymore. I also veried that without your patch I see the leak immediately.

Thanks for fixing this so quickly, it would good to have this fix in v6.6 if possible.

Signature · 3 lines
https://patchwork.kernel.org/project/linux-wireless/list/

https://wireless.wiki.kernel.org/en/developers/documentation/submittingpatches
#5 lore raw
Quoted 64 lines · Kalle Valo
Quoted 53 lines · Ross Lagerwall
Quoted 43 lines · Kalle Valo

I noticed that starting from v6.5-rc1 my ath11k tests reported several memory leaks from swapper/0:

unreferenced object 0xffff88810a02b7a8 (size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):

    80 b8 02 0a 81 88 ff ff b8 72 07 00 00 c9 ff ff  .........r......
    c8 b7 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00  ................

backtrace:
unreferenced object 0xffff88810a02b880 (size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):

    58 b9 02 0a 81 88 ff ff a8 b7 02 0a 81 88 ff ff  X...............
    a0 b8 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00  ................

backtrace:
unreferenced object 0xffff88810a02b958 (size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):

    30 ba 02 0a 81 88 ff ff 80 b8 02 0a 81 88 ff ff  0...............
    78 b9 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00  x...............

backtrace:
unreferenced object 0xffff88810a02ba30 (size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):

    08 bb 02 0a 81 88 ff ff 58 b9 02 0a 81 88 ff ff  ........X.......
    50 ba 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00  P...............

backtrace:
unreferenced object 0xffff88810a02bb08 (size 96):
comm "swapper/0", pid 1, jiffies 4294671838 (age 98.120s) hex dump (first 32 bytes):

    e0 bb 02 0a 81 88 ff ff 30 ba 02 0a 81 88 ff ff  ........0.......
    28 bb 02 0a 81 88 ff ff 00 00 00 00 00 00 00 00  (...............

backtrace:

I can easily reproduce this by doing a simple insmod and rmmod of ath11k and it's dependencies (mac80211, MHI etc). I can reliability reproduce the leaks but I only see them once after a boot, I need to reboot the host to see the leaks again. v6.4 has no leaks.

I did a bisect and found the commit below. I verified reverting the commit makes the leaks go away.

Quoted 7 lines

commit e54223275ba1bc6f704a6bab015fcd2ae4f72572

Author:     Ross Lagerwall <[email protected]>
AuthorDate: Thu May 25 16:32:48 2023 +0100
Commit:     Bjorn Helgaas <[email protected]>
CommitDate: Fri Jun 9 15:06:16 2023 -0500

PCI: Release resource invalidated by coalescing

Quoted 7 lines

Hi Kalle,

I can't reproduce the leak by loading/unloading the ath11k module. I suspect that the leak is always there when PCI resources are coalesced but kmemleak doesn't notice until ath11k is loaded.

Can you please try the following to confirm it fixes it?

Quoted 8 lines

I run various tests with your patch and I don't see leaks anymore. I also veried that without your patch I see the leak immediately.

Thanks for fixing this so quickly, it would good to have this fix in v6.6 if possible.

Thanks, I will send out a proper patch now.

Ross