pull request for net: batman-adv 2023-06-07
Subsystems (3)
Simon Wunderlich
maintainercover pull request for net: batman-adv 2023-06-07
Hi David, hi Jakub,
here is a bugfix for batman-adv which we would like to have integrated into net.
Please pull or let me know of any problem!
Thank you,
Simon
The following changes since commit 44c026a73be8038f03dbdeef028b642880cf1511:
Linux 6.4-rc3 (2023-05-21 14:05:48 -0700)
are available in the Git repository at:
git://git.open-mesh.org/linux-merge.git tags/batadv-net-pullrequest-20230607
for you to fetch changes up to abac3ac97fe8734b620e7322a116450d7f90aa43:
batman-adv: Broken sync while rescheduling delayed work (2023-05-26 23:14:49 +0200)
----------------------------------------------------------------
Here is a batman-adv bugfix:
- fix a broken sync while rescheduling delayed work, by Vladislav Efanov
----------------------------------------------------------------
Vladislav Efanov (1):
batman-adv: Broken sync while rescheduling delayed work
net/batman-adv/distributed-arp-table.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Simon Wunderlich
maintainerPATCH 1/1 batman-adv: Broken sync while rescheduling delayed work
From: Vladislav Efanov <[email protected]>
Syzkaller got a lot of crashes like:
KASAN: use-after-free Write in *_timers*
All of these crashes point to the same memory area:
The buggy address belongs to the object at ffff88801f870000 which belongs to the cache kmalloc-8k of size 8192
The buggy address is located 5320 bytes inside of
8192-byte region [ffff88801f870000, ffff88801f872000)
This area belongs to :
batadv_priv->batadv_priv_dat->delayed_work->timer_list
The reason for these issues is the lack of synchronization. Delayed work (batadv_dat_purge) schedules new timer/work while the device is being deleted. As the result new timer/delayed work is set after cancel_delayed_work_sync() was called. So after the device is freed the timer list contains pointer to already freed memory.
Found by Linux Verification Center (linuxtesting.org) with syzkaller.
--- net/batman-adv/distributed-arp-table.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
net/batman-adv/distributed-arp-table.c +1 -1
@@ -101,7 +101,6 @@ static void batadv_dat_purge(struct work_struct *work);*/ static void batadv_dat_start_timer (struct batadv_priv *bat_priv) { -INIT_DELAYED_WORK (&bat_priv->dat.work, batadv_dat_purge);queue_delayed_work (batadv_event_workqueue, &bat_priv->dat.work,msecs_to_jiffies (10000 )); }
@@ -819,6 +818,7 @@ int batadv_dat_init(struct batadv_priv *bat_priv)if (!bat_priv->dat.hash)return -ENOMEM ; +INIT_DELAYED_WORK (&bat_priv->dat.work, batadv_dat_purge);batadv_dat_start_timer (bat_priv); batadv_tvlv_handler_register(bat_priv, batadv_dat_tvlv_ogm_handler_v1,
Signature · 1 line
2.30.2
Re: [PATCH 1/1] batman-adv: Broken sync while rescheduling delayed work
This patch was applied to netdev/net.git (main) by Simon Wunderlich <[email protected]>:
Hello:
This patch was applied to netdev/net.git (main)
by Simon Wunderlich <[email protected]>:
Quoted 8 lines · On Wed, 7 Jun 2023 17:55:15 +0200 you wrote:
From: Vladislav Efanov <[email protected]>
Syzkaller got a lot of crashes like:
KASAN: use-after-free Write in *_timers*All of these crashes point to the same memory area:
[...]
Here is the summary with links:
- [1/1] batman-adv: Broken sync while rescheduling delayed work
https://git.kernel.org/netdev/net/c/abac3ac97fe8
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html
Jakub Kicinski
maintainerRe: [PATCH 1/1] batman-adv: Broken sync while rescheduling delayed work
In reply to Simon WunderlichQuoted 5 lines · On Wed, 7 Jun 2023 17:55:15 +0200 Simon Wunderlich wrote:
The reason for these issues is the lack of synchronization. Delayed work (
batadv_dat_purge) schedules new timer/work while the device is being deleted. As the result new timer/delayed work is set aftercancel_delayed_work_sync()was called. So after the device is freed the timer list contains pointer to already freed memory.
I guess this is better than status quo but is the fix really complete? We're still not preventing the timer / work from getting scheduled and staying alive after the netdev has been freed, right?
RE: [PATCH 1/1] batman-adv: Broken sync while rescheduling delayed work
Quoted 19 lines
-----Original Message-----
From: Jakub Kicinski <[email protected]>
Sent: Wednesday, June 7, 2023 10:01 PM
To: Simon Wunderlich <[email protected]>
Cc: [email protected]; [email protected]; [email protected] mesh.org; Vladislav Efanov <[email protected]>; [email protected]; Sven Eckelmann <[email protected]>
Subject: Re: [PATCH 1/1] batman-adv: Broken sync while rescheduling delayed workQuoted 5 lines · On Wed, 7 Jun 2023 17:55:15 +0200 Simon Wunderlich wrote:
The reason for these issues is the lack of synchronization. Delayed work (
batadv_dat_purge) schedules new timer/work while the device is being deleted. As the result new timer/delayed work is set aftercancel_delayed_work_sync()was called. So after the device is freed the timer list contains pointer to already freed memory.
Quoted 3 lines
I guess this is better than status quo but is the fix really complete? We're still not preventing the timer / work from getting scheduled and staying alive after the netdev has been freed, right?
Yea, I would expect some synchronization mechanism to ensure that after cancel_delayed_work_sync() you can't queue the work again.
I know for timers there is recently timer_shutdown_sync() which can be used to guarantee a timer can't re-arm at all, and its intended for some situations where there is a cyclic dependency...
Thanks,
Jake
Re: [PATCH 1/1] batman-adv: Broken sync while rescheduling delayed work
As far as I found the synchronization is provided by delayed work subsystem. It is based on the WORK_STRUCT_PENDING_BIT in work->data field.
The cancel_delayed_work_sync() atomically sets this bit and queue_delayed_work() checks it before scheduling new delayed work.
The problem is caused by the INIT_DELAYED_WORK() call inside batadv_dat_start_timer(). This call happens before the queue_delayed_work() call and clears this bit.
Best regards,
Vlad
Quoted 26 lines · Jacob E
Quoted 12 lines
-----Original Message-----
From: Jakub Kicinski <[email protected]>
Sent: Wednesday, June 7, 2023 10:01 PM
To: Simon Wunderlich <[email protected]>
Cc: [email protected]; [email protected]; [email protected] mesh.org; Vladislav Efanov <[email protected]>; [email protected]; Sven Eckelmann <[email protected]>
Subject: Re: [PATCH 1/1] batman-adv: Broken sync while rescheduling delayed workOn Wed, 7 Jun 2023 17:55:15 +0200 Simon Wunderlich wrote:
Quoted 8 lines
Quoted 5 lines
The reason for these issues is the lack of synchronization. Delayed work (
batadv_dat_purge) schedules new timer/work while the device is being deleted. As the result new timer/delayed work is set aftercancel_delayed_work_sync()was called. So after the device is freed the timer list contains pointer to already freed memory.I guess this is better than status quo but is the fix really complete? We're still not preventing the timer / work from getting scheduled and staying alive after the netdev has been freed, right?
Yea, I would expect some synchronization mechanism to ensure that after
cancel_delayed_work_sync()you can't queue the work again.I know for timers there is recently
timer_shutdown_sync()which can be used to guarantee a timer can't re-arm at all, and its intended for some situations where there is a cyclic dependency...Thanks,
Jake
Paolo Abeni
maintainerRe: [PATCH 1/1] batman-adv: Broken sync while rescheduling delayed work
In reply to Jakub KicinskiQuoted 10 lines · Jakub Kicinski
Quoted 5 lines · On Wed, 7 Jun 2023 17:55:15 +0200 Simon Wunderlich wrote:
The reason for these issues is the lack of synchronization. Delayed work (
batadv_dat_purge) schedules new timer/work while the device is being deleted. As the result new timer/delayed work is set aftercancel_delayed_work_sync()was called. So after the device is freed the timer list contains pointer to already freed memory.I guess this is better than status quo but is the fix really complete? We're still not preventing the timer / work from getting scheduled and staying alive after the netdev has been freed, right?
I *think* this specific use case does not expose such problem, as the delayed work is (AFAICS) scheduled only at device creation time and by the work itself, it should never be re-scheduled after cancel_delayed_work_sync()
Cheers,
Paolo
Sven Eckelmann
maintainerRe: [PATCH 1/1] batman-adv: Broken sync while rescheduling delayed work
On Thursday, 8 June 2023 11:27:31 CEST Paolo Abeni wrote:
[...]
Quoted 7 lines
Quoted 2 lines
We're still not preventing the timer / work from getting scheduled and staying alive after the netdev has been freed, right?
I *think* this specific use case does not expose such problem, as the delayed work is (AFAICS) scheduled only at device creation time and by the work itself, it should never be re-scheduled after
cancel_delayed_work_sync()
Correct.
* batadv_dat_start_timer is the only thing scheduling it
* batadv_dat_start_timer is called by:
- batadv_dat_purge (the worker rearming itself)
- batadv_dat_init (when the interface is created)
Kind regards,
Sven