percpu: add basic double free check
Subsystems (2)
Dennis Zhou
maintainerPATCH v2 1/1 percpu: add basic double free check
This adds a basic double free check by validating the first bit of the allocation in alloc_map and bound_map are set. If the alloc_map bit is not set, then this means the area is currently unallocated. If the bound_map bit is not set, then we are not freeing from the beginning of the allocation.
This is a respin of [1] adding the requested changes from me and Christoph.
[1] https://lore.kernel.org/linux-mm/[email protected]/
---
v2:
- moved pcpu_stats_area_dealloc()
- added additional check for bit_off out of bounds
mm/percpu.c | 23 ++++++++++++++++++++--- 1 file changed, 20 insertions(+), 3 deletions(-)
mm/percpu.c +20 -3
@@ -79,6 +79,7 @@#include <linux/mutex.h> #include <linux/percpu.h> #include <linux/pfn.h> +#include <linux/ratelimit.h> #include <linux/slab.h> #include <linux/spinlock.h> #include <linux/vmalloc.h>
@@ -1276,18 +1277,24 @@ static int pcpu_alloc_area(struct pcpu_chunk *chunk, int alloc_bits,static int pcpu_free_area (struct pcpu_chunk *chunk,int off) {struct pcpu_block_md *chunk_md = &chunk->chunk_md; +int region_bits =pcpu_chunk_map_bits (chunk);int bit_off, bits, end, oslot, freed;lockdep_assert_held (&pcpu_lock); -pcpu_stats_area_dealloc (chunk); oslot =pcpu_chunk_slot (chunk); bit_off = off /PCPU_MIN_ALLOC_SIZE ; +if (unlikely (bit_off <0 || bit_off >= region_bits)) +return 0 ; + +/* check double free */ +if (!test_bit (bit_off, chunk->alloc_map) || + !test_bit (bit_off, chunk->bound_map)) +return 0 ;/* find end index */ - end =find_next_bit (chunk->bound_map,pcpu_chunk_map_bits (chunk), - bit_off +1 ); + end =find_next_bit (chunk->bound_map, region_bits, bit_off +1 ); bits = end - bit_off;bitmap_clear (chunk->alloc_map, bit_off, bits);
@@ -1303,6 +1310,8 @@ static int pcpu_free_area(struct pcpu_chunk *chunk, int off)pcpu_chunk_relocate (chunk, oslot); +pcpu_stats_area_dealloc (chunk); +return freed; }
@@ -2225,6 +2234,7 @@ static void pcpu_balance_workfn(struct work_struct *work)*/ void free_percpu (void __percpu *ptr) { + static DEFINE_RATELIMIT_STATE(_rs, 60 * HZ, DEFAULT_RATELIMIT_BURST);void *addr;struct pcpu_chunk *chunk;unsigned long flags;
@@ -2242,6 +2252,13 @@ void free_percpu(void __percpu *ptr)spin_lock_irqsave (&pcpu_lock, flags); size =pcpu_free_area (chunk, off); +if (size ==0 ) { +spin_unlock_irqrestore (&pcpu_lock, flags); + +if (__ratelimit (&_rs)) +WARN (1 ,"percpu double free or bad ptr\n" ); +return ; + }pcpu_alloc_tag_free_hook (chunk, off, size);
Signature · 1 line
2.43.0
Andrew Morton
maintainerQuoted 16 lines · On Thu, 15 Jan 2026 18:32:16 -0800 Dennis Zhou <[email protected]> wrote:
This adds a basic double free check by validating the first bit of the allocation in
alloc_mapandbound_mapare set. If thealloc_mapbit is not set, then this means the area is currently unallocated. If thebound_mapbit is not set, then we are not freeing from the beginning of the allocation.This is a respin of [1] adding the requested changes from me and Christoph.
...
mm/percpu.c +1
-0@@ -1276,18 +1277,24 @@ static int pcpu_alloc_area(struct pcpu_chunk *chunk, int alloc_bits,static int pcpu_free_area (struct pcpu_chunk *chunk,int off) {struct pcpu_block_md *chunk_md = &chunk->chunk_md; +int region_bits =pcpu_chunk_map_bits (chunk);
mm/percpu.c static int pcpu_alloc_area(struct pcpu_chunk *chunk, int alloc_bits, · line 1289
Quoted 10 lines
mm/percpu.c +2
-1int bit_off, bits, end, oslot, freed;lockdep_assert_held (&pcpu_lock); -pcpu_stats_area_dealloc (chunk); oslot =pcpu_chunk_slot (chunk); bit_off = off /PCPU_MIN_ALLOC_SIZE ; +if (unlikely (bit_off <0 || bit_off >= region_bits)) +return 0 ;
This (which looks sensible) wasn't changelogged?
mm/percpu.c void free_percpu(void __percpu *ptr) · line 2259
Quoted 9 lines
mm/percpu.c +5
-0@@ -2242,6 +2252,13 @@ void free_percpu(void __percpu *ptr)spin_lock_irqsave (&pcpu_lock, flags); size =pcpu_free_area (chunk, off); +if (size ==0 ) { +spin_unlock_irqrestore (&pcpu_lock, flags); + +if (__ratelimit (&_rs)) +WARN (1 ,"percpu double free or bad ptr\n" );
Is ratelimiting really needed? A WARN_ON_ONCE is enough to tell people that this kernel is wrecked?
mm/percpu.c void free_percpu(void __percpu *ptr) · line 2261
Quoted 2 lines
mm/percpu.c +2
-0+return ; + }
The patch does appear to do that which it set out to do. But do we want to do it? Is there a history of callers double-freeing percpu memory? Was there some bug which would have been more rapidly and easily solved had this change been in place?
Dennis Zhou
maintainerQuoted 23 lines · Andrew Morton
Quoted 21 lines · On Thu, 15 Jan 2026 18:32:16 -0800 Dennis Zhou <[email protected]> wrote:
This adds a basic double free check by validating the first bit of the allocation in
alloc_mapandbound_mapare set. If thealloc_mapbit is not set, then this means the area is currently unallocated. If thebound_mapbit is not set, then we are not freeing from the beginning of the allocation.This is a respin of [1] adding the requested changes from me and Christoph.
...
mm/percpu.c +1
-1@@ -1276,18 +1277,24 @@ static int pcpu_alloc_area(struct pcpu_chunk *chunk, int alloc_bits,static int pcpu_free_area (struct pcpu_chunk *chunk,int off) {struct pcpu_block_md *chunk_md = &chunk->chunk_md; +int region_bits =pcpu_chunk_map_bits (chunk);int bit_off, bits, end, oslot, freed;lockdep_assert_held (&pcpu_lock); -pcpu_stats_area_dealloc (chunk);
Quoted 8 lines
mm/percpu.c static int pcpu_alloc_area(struct pcpu_chunk *chunk, int alloc_bits, · line 1289
Quoted 5 lines
mm/percpu.c +2
-0oslot =pcpu_chunk_slot (chunk); bit_off = off /PCPU_MIN_ALLOC_SIZE ; +if (unlikely (bit_off <0 || bit_off >= region_bits)) +return 0 ;This (which looks sensible) wasn't changelogged?
Sorry that's my fault. I can respin and add it if you'd like.
Quoted 13 lines
mm/percpu.c void free_percpu(void __percpu *ptr) · line 2259
Quoted 9 lines
mm/percpu.c +5
-0@@ -2242,6 +2252,13 @@ void free_percpu(void __percpu *ptr)spin_lock_irqsave (&pcpu_lock, flags); size =pcpu_free_area (chunk, off); +if (size ==0 ) { +spin_unlock_irqrestore (&pcpu_lock, flags); + +if (__ratelimit (&_rs)) +WARN (1 ,"percpu double free or bad ptr\n" );Is ratelimiting really needed? A
WARN_ON_ONCEis enough to tell people that this kernel is wrecked?
I can see running multiple tests that might give me additional debug / signal to how badly I screwed up. In production a WARN_ON_ONCE is definitely more than enough, but might as well offer the chance to try and trigger it more than once.
Quoted 8 lines
mm/percpu.c void free_percpu(void __percpu *ptr) · line 2261
Quoted 2 lines
mm/percpu.c +2
-0+return ; + }The patch does appear to do that which it set out to do. But do we want to do it? Is there a history of callers double-freeing percpu memory? Was there some bug which would have been more rapidly and easily solved had this change been in place?
Originally, Sebastian posted he ran into the issue where he double freed in [1] (linked in patch). Maybe he can elaborate how that bug was introduced.
Wrt do we want to do it - I think it doesn't hurt and makes it more explicit that something very wrong occurred. Percpu memory really expects users to be good samaritans. If you do happen to accidentally double free without the warning, in a contrived case you could experience unexplained behavior for some time before crashing in a spot that would leave your head scratching. If anything I think there could be an argument to fail louder.
[1] https://lore.kernel.org/linux-mm/[email protected]/
Thanks,
Dennis
Andrew Morton
maintainerQuoted 26 lines · On Fri, 16 Jan 2026 21:15:33 -0800 Dennis Zhou <[email protected]> wrote:
Quoted 25 lines · Andrew Morton
Quoted 23 lines · On Thu, 15 Jan 2026 18:32:16 -0800 Dennis Zhou <[email protected]> wrote:
This adds a basic double free check by validating the first bit of the allocation in
alloc_mapandbound_mapare set. If thealloc_mapbit is not set, then this means the area is currently unallocated. If thebound_mapbit is not set, then we are not freeing from the beginning of the allocation.This is a respin of [1] adding the requested changes from me and Christoph.
...
mm/percpu.c +1
-1@@ -1276,18 +1277,24 @@ static int pcpu_alloc_area(struct pcpu_chunk *chunk, int alloc_bits,static int pcpu_free_area (struct pcpu_chunk *chunk,int off) {struct pcpu_block_md *chunk_md = &chunk->chunk_md; +int region_bits =pcpu_chunk_map_bits (chunk);int bit_off, bits, end, oslot, freed;lockdep_assert_held (&pcpu_lock); -pcpu_stats_area_dealloc (chunk); oslot =pcpu_chunk_slot (chunk);
Quoted 8 lines
Quoted 6 lines
mm/percpu.c static int pcpu_alloc_area(struct pcpu_chunk *chunk, int alloc_bits, · line 1289
Quoted 3 lines
mm/percpu.c +2
-0bit_off = off /PCPU_MIN_ALLOC_SIZE ; +if (unlikely (bit_off <0 || bit_off >= region_bits)) +return 0 ;This (which looks sensible) wasn't changelogged?
Sorry that's my fault. I can respin and add it if you'd like.
Yes please, I'm thinking a respin is needed anyway....
Quoted 14 lines
Quoted 13 lines
mm/percpu.c void free_percpu(void __percpu *ptr) · line 2259
Quoted 9 lines
mm/percpu.c +5
-0@@ -2242,6 +2252,13 @@ void free_percpu(void __percpu *ptr)spin_lock_irqsave (&pcpu_lock, flags); size =pcpu_free_area (chunk, off); +if (size ==0 ) { +spin_unlock_irqrestore (&pcpu_lock, flags); + +if (__ratelimit (&_rs)) +WARN (1 ,"percpu double free or bad ptr\n" );Is ratelimiting really needed? A
WARN_ON_ONCEis enough to tell people that this kernel is wrecked?
Quoted 4 lines
I can see running multiple tests that might give me additional debug / signal to how badly I screwed up. In production a
WARN_ON_ONCEis definitely more than enough, but might as well offer the chance to try and trigger it more than once.
If this is happening at development-time then developer can turn that into WARN_ON() or whatever.
I dunno, I do feel that WARN_ON_ONCE is sufficient but no strong feelings.
Quoted 21 lines
Quoted 8 lines
mm/percpu.c void free_percpu(void __percpu *ptr) · line 2261
Quoted 2 lines
mm/percpu.c +2
-0+return ; + }The patch does appear to do that which it set out to do. But do we want to do it? Is there a history of callers double-freeing percpu memory? Was there some bug which would have been more rapidly and easily solved had this change been in place?
Originally, Sebastian posted he ran into the issue where he double freed in [1] (linked in patch). Maybe he can elaborate how that bug was introduced.
Wrt do we want to do it - I think it doesn't hurt and makes it more explicit that something very wrong occurred. Percpu memory really expects users to be good samaritans. If you do happen to accidentally double free without the warning, in a contrived case you could experience unexplained behavior for some time before crashing in a spot that would leave your head scratching. If anything I think there could be an argument to fail louder.
Quoted 1 line
Could you please get this justification into the changelog? It's pretty important - explain to the world why we feel that Linux needs alteration and what benefit we're providing.
Quoted 10 lines · Dennis Zhou
Quoted 5 lines
The patch does appear to do that which it set out to do. But do we want to do it? Is there a history of callers double-freeing percpu memory? Was there some bug which would have been more rapidly and easily solved had this change been in place?
Originally, Sebastian posted he ran into the issue where he double freed in [1] (linked in patch). Maybe he can elaborate how that bug was introduced.
Quoted 7 lines
Wrt do we want to do it - I think it doesn't hurt and makes it more explicit that something very wrong occurred. Percpu memory really expects users to be good samaritans. If you do happen to accidentally double free without the warning, in a contrived case you could experience unexplained behavior for some time before crashing in a spot that would leave your head scratching. If anything I think there could be an argument to fail louder.
I did write some code and there was a free path I did not expect to happen. While it was rare to happen, it did not always crash right away. Once I had the pattern I was wondering why none of the memory debug switches did something.
Adding this does not look expensive, allocating per-CPU is hardly done in a hot path so I did not add anything but I don't mind hiding it behind a config switch similar to SLAB_DEBUG.
While we do have way less per-CPU allocations compared to SLAB, I think it is important to have some kind of sanity checks for it to ensure it is used properly.
I would go with a WARN_ON_ONCE but if there is a desire for rate limited multiple warnings, fine.
Quoted 4 lines
[1] https://lore.kernel.org/linux-mm/[email protected]/
Thanks,
Dennis
Sebastian