Race condition in del_timer_sync (2.5)
PATCH 1/1 Race condition in del_timer_sync (2.5)
Hello,
This patch fixes a race between del_timer_sync and recursive timers.
Current implementation allows the value of timer->base that is used for timer_pending test to be fetched before finishing running_timer test, so it's possible for a recursive time to be pending after del_timer_sync. Adding smp_rmb before timer_pending removes the race.
The patch is against the latest 2.5 bk tree. Please point out if I
got something wrong. TIA.
Signature · 12 lines
tejun diff -Nru a/kernel/timer.c b/kernel/timer.c --- a/kernel/timer.c Tue Sep 2 11:56:58 2003 +++ b/kernel/timer.c Tue Sep 2 11:56:58 2003 @@ -338,6 +338,7 @@ break; } } + smp_rmb(); if (timer_pending(timer)) goto del_again;
Quoted 5 lines · Tejun Huh
This patch fixes a race between
del_timer_syncand recursive timers.
Current implementation allows the value oftimer->basethat is used fortimer_pendingtest to be fetched before finishingrunning_timertest, so it's possible for a recursive time to be pending afterdel_timer_sync. Addingsmp_rmbbeforetimer_pendingremoves the race.
good catch. Have you ever trigger this bug, or did you find it via code review?
just to explore the scope of this problem a bit more: at first glance all other timer_pending() uses seem to be safe. del_timer_sync()'s timer_pending() use is special, because it's next to the ->running_timer check without any barriers inbetween - so we could indeed in theory end up with having the two reads reordered and a freshly added timer (on another CPU) not being recognized properly. Also, this is the only timer API call that guarantees the complete stopping of a timer.
Ingo
Quoted 11 lines · Ingo Molnar
Quoted 5 lines · Tejun Huh
This patch fixes a race between
del_timer_syncand recursive timers.
Current implementation allows the value oftimer->basethat is used fortimer_pendingtest to be fetched before finishingrunning_timertest, so it's possible for a recursive time to be pending afterdel_timer_sync. Addingsmp_rmbbeforetimer_pendingremoves the race.good catch. Have you ever trigger this bug, or did you find it via code review?
I found it via code review. Actually, I haven't succeeded to boot
even once with 2.5 on my test box. It prints a lot of messages about scheduling while atomic and eventually panics. I'm trying to identify what's causing the problem. Wish me luck. :-)
Quoted 9 lines
just to explore the scope of this problem a bit more: at first glance all other
timer_pending()uses seem to be safe. del_timer_sync()'stimer_pending()use is special, because it's next to the ->running_timer check without any barriers inbetween - so we could indeed in theory end up with having the two reads reordered and a freshly added timer (on another CPU) not being recognized properly. Also, this is the only timer API call that guarantees the complete stopping of a timer.Ingo
Yeap, in other cases, when timer_pending() fails to see timer->base,
it can be thought as the test took place before the timer is added, and leaving pending timer behind is semantically ok. But because a recusive timer without some other entity trying to add it should actually be removed by timer_del_sync(), timer_pending() test should be barriered from running_timer test.
I'll submit the patch to Linus soon.
Signature · 1 line
tejun
Quoted 2 lines · Tejun Huh
I'll submit the patch to Linus soon.
I actually already committed it to my tree, since everybody seems to agree on it...
Linus
----
# This is a BitKeeper generated patch for the following project:
# Project Name: Linux kernel tree
# This patch format is intended for GNU patch command version 2.5 or higher. # This patch includes the following deltas:
# ChangeSet 1.1412 -> 1.1413
# kernel/timer.c 1.66 -> 1.67
#
# The following is the BitKeeper ChangeSet Log
# --------------------------------------------
# 03/09/02 [email protected] 1.1413
# Fix del_timer_sync() SMP memory ordering (from Tejun Huh <[email protected]>) #
# From Tejun's posting:
# >
# > This patch fixes a race between del_timer_sync and recursive timers. # > Current implementation allows the value of timer->base that is used # > for timer_pending test to be fetched before finishing running_timer # > test, so it's possible for a recursive time to be pending after # > del_timer_sync. Adding smp_rmb before timer_pending removes the race. # --------------------------------------------
#
kernel/timer.c +1 -0
@@ -338,6 +338,7 @@break ; } } +smp_rmb ();if (timer_pending (timer)) goto del_again;