page_pool: Fix use-after-free in page_pool_recycle_in_ring
PATCH 1/1 page_pool: Fix use-after-free in page_pool_recycle_in_ring
From: Dong Chenchen <[email protected]>
[ Upstream commit 271683bb2cf32e5126c592b5d5e6a756fa374fd9 ]
syzbot reported a uaf in page_pool_recycle_in_ring:
BUG: KASAN: slab-use-after-free in lock_release+0x151/0xa30 kernel/locking/lockdep.c:5862 Read of size 8 at addr ffff8880286045a0 by task syz.0.284/6943
root cause is:
page_pool_recycle_in_ring
ptr_ring_produce
spin_lock(&r->producer_lock);
WRITE_ONCE(r->queue[r->producer++], ptr)
//recycle last page to pool
page_pool_release
page_pool_scrub
page_pool_empty_ring
ptr_ring_consume
page_pool_return_page //release all page
__page_pool_destroy
free_percpu(pool->recycle_stats);
free(pool) //free
spin_unlock(&r->producer_lock); //pool->ring uaf read recycle_stat_inc(pool, ring);
page_pool can be free while page pool recycle the last page in ring. Add producer-lock barrier to page_pool_release to prevent the page pool from being free before all pages have been recycled.
[v5.10: introduced page_pool_producer_lock/unlock helpers inline since
prerequisite commit 368d3cb406cd ("page_pool: fix inconsistency for page_pool_ring_[un]lock()") depends on page_pool_put_page_bulk which does not exist in 5.10; used in_serving_softirq() per 5.10 convention; kept struct page * API (no netmem_ref); dropped recycle_stat_inc change as page pool stats do not exist in this tree]
--- net/core/page_pool.c | 39 +++++++++++++++++++++++++++++++++------ 1 file changed, 33 insertions(+), 6 deletions(-)
net/core/page_pool.c +33 -6
@@ -318,16 +318,39 @@ static void page_pool_return_page(struct page_pool *pool, struct page *page)*/ } +static bool page_pool_producer_lock (struct page_pool *pool) + __acquires(&pool->ring.producer_lock) +{ +bool in_softirq =in_serving_softirq (); + +if (in_softirq) +spin_lock (&pool->ring.producer_lock); +else +spin_lock_bh (&pool->ring.producer_lock); + +return in_softirq; +} + +static void page_pool_producer_unlock (struct page_pool *pool, +bool in_softirq) + __releases(&pool->ring.producer_lock) +{ +if (in_softirq) +spin_unlock (&pool->ring.producer_lock); +else +spin_unlock_bh (&pool->ring.producer_lock); +} +static bool page_pool_recycle_in_ring (struct page_pool *pool,struct page *page) { -int ret; +bool in_softirq, ret; +/* BH protection not needed if current is serving softirq */ -if (in_serving_softirq ()) - ret =ptr_ring_produce (&pool->ring, page); -else - ret =ptr_ring_produce_bh (&pool->ring, page); + in_softirq =page_pool_producer_lock (pool); + ret = !__ptr_ring_produce (&pool->ring, page); +page_pool_producer_unlock (pool, in_softirq); -return (ret ==0 ) ? true : false; +return ret; }/* Only allow direct recycling in special circumstances, into the
@@ -464,10 +487,14 @@ static void page_pool_scrub(struct page_pool *pool)static int page_pool_release (struct page_pool *pool) { +bool in_softirq;int inflight;page_pool_scrub (pool); inflight =page_pool_inflight (pool); +/* Acquire producer lock to make sure producers have exited. */ + in_softirq =page_pool_producer_lock (pool); +page_pool_producer_unlock (pool, in_softirq);if (!inflight)page_pool_free (pool);
Signature · 12 lines
2.50.1 Amazon Web Services Development Center Germany GmbH Tamara-Danz-Str. 13 10243 Berlin Geschaeftsfuehrung: Christof Hellmis, Andreas Stieger Eingetragen am Amtsgericht Charlottenburg unter HRB 257764 B Sitz: Berlin Ust-ID: DE 365 538 597
Queued for 5.10, thanks.
Signature · 2 lines
Thanks, Sasha