Fix bug about invalid wake up problem
Subsystems (8)
cover Fix bug about invalid wake up problem
1)Problem Description:
The prototype of invalid wakeup problem is as follows:
========================================================================
----------------------------
Consumer Thread
----------------------------
...
if (list_empty(&list)){
//location a
set_current_state(TASK_INTERRUPTIBLE);
schedule();
}...
----------------------------
Producer Thread
----------------------------
...list_add_tail(&item, &list);wake_up_process(A);
...
========================================================================
Invalid wakeup problem occurs in preemptable kernel environment. The list is empty initially, if consumer is preempted after condition validated at location a, and at this time producer is woken up, adding a node to the list and calling wake_up_process to wake up consumer. After that when consumer being scheduled by scheduler, it calls set_current_state to set itself as state TASK_INTERRUPTIBLE, and calling schedule() to request scheduled. After consumer being scheduled out, it has no condition to be woken up, and causing invalid wakeup problem.
In most cases, the kernel codes use a form similar to the following:
--------------------------------------------
...
//location a
...
set_current_state(TASK_INTERRUPTIBLE);
//location b
while (list_empty(&product_list)){
schedule(); //location c
set_current_state(TASK_INTERRUPTIBLE);
//location d
}__set_current_state(TASK_RUNNING);
...
--------------------------------------------
At location a, consumer is preempted, and producer is scheduled,
adding item to product_list and waking up consumer. After consumer is re-scheduled, calling set_current_state to set itself as state TASK_INTERRUPTIBLE, if it is preempted again before __set_current_state(TASK_RUNNING), also trigger the invalid wakeup problem that the consumer will not be scheduled and the item be added by producer can't be consumed.
The following circumstance will also trigger this issue:
At location c, consumer is scheduled out, and be preempted after calling set_current_state(TASK_INTERRUPTIBLE) when it be re-schdeuled.
2)Test:
I have written a test module to trigger the problem by adding some synchronization condition. I will post it in the form of an attachment soon.
Test result as follows:
[103135.332683] wakeup_test: create two kernel threads - producer & consumer [103135.332686] wakeup_test: module loaded successfully [103135.332692] wakeup_test: kthread producer try to wake up the kthread consumer [103165.299865] wakeup_test: kthread consumer have waited for 30s, indicating
trigger an invalid wakeup problem!
3)Solution:
Using preempt_disable() to bound the operaion that setting the task state and the conditions(set by the wake thread) validation.
----------------------------------------------
...preempt_disable();set_current_state(TASK_INTERRUPTIBLE);
while (list_empty(&product_list)){
preempt_enable();
schedule();
preempt_disable();
set_current_state(TASK_INTERRUPTIBLE);
}__set_current_state(TASK_RUNNING);preempt_enable();
...
----------------------------------------------
And it also can be solved as follows:
----------------------------------------------
...preempt_disable();
while (list_empty(&product_list)){
set_current_state(TASK_INTERRUPTIBLE);
preempt_enable();
schedule();
preempt_disable();
}preempt_enable();
...
----------------------------------------------
Libin (14):
kthread: Fix invalid wakeup in kthreadd
audit: Fix invalid wakeup in kauditd_thread
audit: Fix invalid wakeup in wait_for_auditd
exit: Fix invalid wakeup in do_wait
hrtimer: Fix invalid wakeup in do_nanosleep
irq: Fix invalid wakeup in irq_wait_for_interrupt
module: Fix invalid wakeup in wait_for_zero_refcount
namespace: Fix invalid wakeup in zap_pid_ns_processes
rcutree: Fix invalid wakeup in rcu_wait
time: Fix invalid wakeup in alarmtimer_do_nsleep
trace: Fix invalid wakeup in wait_to_die
trace: Fix invalid wakeup in ring_buffer_consumer_thread
workqueue: Fix invalid wakeup in rescuer_thread
klist: Fix invalid wakeup in klist_remove
kernel/audit.c | 11 ++++- kernel/exit.c | 7 ++- kernel/hrtimer.c | 7 ++- kernel/irq/manage.c | 5 ++ kernel/kthread.c | 7 ++- kernel/module.c | 6 ++- kernel/pid_namespace.c | 4 ++ kernel/rcutree.h | 4 ++ kernel/time/alarmtimer.c | 7 ++- kernel/trace/ring_buffer_benchmark.c | 14 ++++++ kernel/workqueue.c | 92 +++++++++++++++++++----------------- lib/klist.c | 4 ++ 12 files changed, 118 insertions(+), 50 deletions(-)
Signature · 1 line
1.8.2.1
PATCH 1/14 kthread: Fix invalid wakeup in kthreadd
If kthreadd is preempted at(or before) location a, and the other thread, such as calling kthread_create_on_node(), adds a list item to the kthread_create_list followed with wake_up_process(kthread). After that when kthreadd is re-scheduled, calling set_current_state to set itself as state TASK_INTERRUPTIBLE, if it is preempted again after that and before __set_current_state(TASK_RUNNING), it triggers the invalid wakeup problem.
------------------------kthreadd()
------------------------
...
for (;;) {
//location a
set_current_state(TASK_INTERRUPTIBLE);
if (list_empty(&kthread_create_list)) {
//location b
schedule();
//location c
}
__set_current_state(TASK_RUNNING);
//location d...
------------------------kthread_create_on_node()
------------------------
...spin_lock(&kthread_create_lock); list_add_tail(&create.list, &kthread_create_list); spin_unlock(&kthread_create_lock);
...wake_up_process(kthreadd_task);
...
To solve this problem, using preempt_disable() to bound the operaion that setting the task state and the conditions(set by the wake thread) validation.
------------------------kthreadd()
------------------------
...
for (;;) {
preempt_disable();
set_current_state(TASK_INTERRUPTIBLE);
if (list_empty(&kthread_create_list)) {
preempt_enable();
schedule();
preempt_disable();
}...
--- kernel/kthread.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
kernel/kthread.c +6 -1
@@ -456,10 +456,15 @@ int kthreadd(void *unused) current->flags |=PF_NOFREEZE ;for (;;) { +preempt_disable ();set_current_state (TASK_INTERRUPTIBLE ); -if (list_empty (&kthread_create_list)) +if (list_empty (&kthread_create_list)) { +preempt_enable ();schedule (); +preempt_disable (); + }__set_current_state (TASK_RUNNING ); +preempt_enable ();spin_lock (&kthread_create_lock);while (!list_empty (&kthread_create_list)) {
Signature · 1 line
1.8.2.1
Re: [PATCH 01/14] kthread: Fix invalid wakeup in kthreadd
Quoted 58 lines · Libin
If kthreadd is preempted at(or before) location a, and the other thread, such as calling
kthread_create_on_node(), adds a list item to thekthread_create_listfollowed withwake_up_process(kthread). After that when kthreadd is re-scheduled, callingset_current_stateto set itself as stateTASK_INTERRUPTIBLE, if it is preempted again after that and before__set_current_state(TASK_RUNNING), it triggers the invalid wakeup problem.
------------------------kthreadd()
------------------------
...for (;;) { //location a set_current_state(TASK_INTERRUPTIBLE); if (list_empty(&kthread_create_list)) { //location b schedule(); //location c } __set_current_state(TASK_RUNNING); //location d...
------------------------kthread_create_on_node()
------------------------
...spin_lock(&kthread_create_lock);list_add_tail(&create.list, &kthread_create_list);spin_unlock(&kthread_create_lock);
...wake_up_process(kthreadd_task);
...To solve this problem, using
preempt_disable()to bound the operaion that setting the task state and the conditions(set by the wake thread) validation.
------------------------kthreadd()
------------------------
...for (;;) { preempt_disable(); set_current_state(TASK_INTERRUPTIBLE); if (list_empty(&kthread_create_list)) { preempt_enable(); schedule(); preempt_disable(); }...
Signed-off-by Libin <[email protected]>--- kernel/kthread.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)kernel/kthread.c +0
-0
kernel/kthread.c int kthreadd(void *unused) · line 460
Quoted 6 lines
kernel/kthread.c +1
-1@@ -456,10 +456,15 @@ int kthreadd(void *unused) current->flags |=PF_NOFREEZE ;for (;;) { +preempt_disable ();set_current_state (TASK_INTERRUPTIBLE ); -if (list_empty (&kthread_create_list))
And this one already has the list_empty() check after the call to set_current_state(), so no change should be needed here.
On the other hand, if your testing shows that you are losing wakeups with this exact code, please let us know!
Thanx, Paul
Quoted 14 lines
kernel/kthread.c +5
-0+if (list_empty (&kthread_create_list)) { +preempt_enable ();schedule (); +preempt_disable (); + }__set_current_state (TASK_RUNNING ); +preempt_enable ();spin_lock (&kthread_create_lock);while (!list_empty (&kthread_create_list)) {Signature · 1 line
1.8.2.1
PATCH 2/14 audit: Fix invalid wakeup in kauditd_thread
In reply to LibinIf this thread is preempted at(or before) location a, and the other thread set the condition followed with wake_up_process. After that
when this thread is re-scheduled, calling set_current_state to set itself as state TASK_INTERRUPTIBLE, if it is preempted again after that and before __set_current_state(TASK_RUNNING), it triggers the invalid wakeup problem.
----------------kauditd_thread()
----------------
while (!kthread_should_stop()) {
...
//location a
set_current_state(TASK_INTERRUPTIBLE);
...
if (!skb_queue_len(&audit_skb_queue)) {
//location b
try_to_freeze();
schedule();
//location c
}
__set_current_state(TASK_RUNNING);
//location d
...
}To solve this problem, using preempt_disable() to bound the operaion that setting the task state and the conditions(set by the wake thread) validation.
----------------kauditd_thread()
----------------
while (!kthread_should_stop()) {
...
preempt_disable();
set_current_state(TASK_INTERRUPTIBLE);
...
if (!skb_queue_len(&audit_skb_queue)) {
preempt_enable();
try_to_freeze();
schedule();
preempt_disable();
}
__set_current_state(TASK_RUNNING);
preempt_enable();
...
}--- kernel/audit.c | 4 ++++ 1 file changed, 4 insertions(+)
kernel/audit.c +4 -0
@@ -455,15 +455,19 @@ static int kauditd_thread(void *dummy)audit_printk_skb (skb);continue ; } +preempt_disable ();set_current_state (TASK_INTERRUPTIBLE );add_wait_queue (&kauditd_wait, &wait);if (!skb_queue_len (&audit_skb_queue)) { +preempt_enable ();try_to_freeze ();schedule (); +preempt_disable (); }__set_current_state (TASK_RUNNING ); +preempt_enable ();remove_wait_queue (&kauditd_wait, &wait); }return 0 ;
Signature · 1 line
1.8.2.1
PATCH 3/14 audit: Fix invalid wakeup in wait_for_auditd
In reply to LibinIf this thread is preempted at(or before) location a, and the other thread set the condition followed with wake_up_process. After that
when this thread is re-scheduled, calling set_current_state to set itself as state TASK_UNINTERRUPTIBLE, if it is preempted again after that and before __set_current_state(TASK_RUNNING), it triggers the invalid wakeup problem.
----------------wait_for_auditd()
----------------
...
//location aset_current_state(TASK_UNINTERRUPTIBLE);
...
if (audit_backlog_limit &&
skb_queue_len(&audit_skb_queue) > audit_backlog_limit) //location b schedule_timeout(sleep_time); //location c
__set_current_state(TASK_RUNNING);
//location d
...
To solve this problem, using preempt_disable() to bound the operaion that setting the task state and the conditions(set by the wake thread) validation.
----------------wait_for_auditd()
----------------
...preempt_disable();set_current_state(TASK_UNINTERRUPTIBLE);
...
if (audit_backlog_limit &&
skb_queue_len(&audit_skb_queue) > audit_backlog_limit) {
preempt_enable();
schedule_timeout(sleep_time);
preempt_disable();
}__set_current_state(TASK_RUNNING);preempt_enable();
...
--- kernel/audit.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
kernel/audit.c +6 -1
@@ -1060,14 +1060,19 @@ static inline void audit_get_stamp(struct audit_context *ctx,static void wait_for_auditd (unsigned long sleep_time) {DECLARE_WAITQUEUE (wait, current); +preempt_disable ();set_current_state (TASK_UNINTERRUPTIBLE );add_wait_queue (&audit_backlog_wait, &wait);if (audit_backlog_limit && -skb_queue_len (&audit_skb_queue) > audit_backlog_limit) +skb_queue_len (&audit_skb_queue) > audit_backlog_limit) { +preempt_enable ();schedule_timeout (sleep_time); +preempt_disable (); + }__set_current_state (TASK_RUNNING ); +preempt_enable ();remove_wait_queue (&audit_backlog_wait, &wait); }
Signature · 1 line
1.8.2.1
PATCH 4/14 exit: Fix invalid wakeup in do_wait
In reply to LibinIf thread is preempted before calling set_current_state(TASK_INTERRUPTIBLE), and the other thread set the condition followed with wake_up_process. After that when this thread is re-scheduled, calling set_current_state to set itself as state TASK_INTERRUPTIBLE, if it is preempted again after that and before __set_current_state(TASK_RUNNING), it triggers the invalid wakeup problem.
To solve this problem, using preempt_disable() to bound the operaion that setting the task state and the conditions(set by the wake thread) validation.
--- kernel/exit.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
kernel/exit.c +5 -2
@@ -1521,7 +1521,6 @@ repeat: (!wo->wo_pid || hlist_empty(&wo->wo_pid->tasks[wo->wo_type]))) goto notask; -set_current_state (TASK_INTERRUPTIBLE );read_lock (&tasklist_lock); tsk = current; do {
@@ -1539,16 +1538,20 @@ repeat:read_unlock (&tasklist_lock); notask: +preempt_disable (); +set_current_state (TASK_INTERRUPTIBLE ); retval = wo->notask_error;if (!retval && !(wo->wo_flags &WNOHANG )) { retval = -ERESTARTSYS ;if (!signal_pending (current)) { +preempt_enable ();schedule (); goto repeat; } } -end:__set_current_state (TASK_RUNNING ); +preempt_enable (); +end:remove_wait_queue (¤t->signal->wait_chldexit, &wo->child_wait);return retval; }
Signature · 1 line
1.8.2.1
PATCH 5/14 hrtimer: Fix invalid wakeup in do_nanosleep
In reply to LibinIf thread is preempted before calling set_current_state(TASK_INTERRUPTIBLE), and the other thread set the condition followed with wake_up_process. After that when this thread is re-scheduled, calling set_current_state to set itself as state TASK_INTERRUPTIBLE, if it is preempted again after that and before __set_current_state(TASK_RUNNING), it triggers the invalid wakeup problem.
To solve this problem, using preempt_disable() to bound the operaion that setting the task state and the conditions(set by the wake thread) validation.
--- kernel/hrtimer.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
kernel/hrtimer.c +6 -1
@@ -1539,14 +1539,18 @@ static int __sched do_nanosleep(struct hrtimer_sleeper *t, enum hrtimer_mode mod {hrtimer_init_sleeper (t, current); +preempt_disable (); do { set_current_state(TASK_INTERRUPTIBLE); hrtimer_start_expires(&t->timer, mode); if (!hrtimer_active(&t->timer)) t->task = NULL; - if (likely(t->task)) + if (likely(t->task)) { + preempt_enable(); freezable_schedule(); + preempt_disable(); + } hrtimer_cancel(&t->timer); mode = HRTIMER_MODE_ABS;
@@ -1554,6 +1558,7 @@ static int __sched do_nanosleep(struct hrtimer_sleeper *t, enum hrtimer_mode mod }while (t->task && !signal_pending (current));__set_current_state (TASK_RUNNING ); +preempt_enable ();return t->task ==NULL ; }
Signature · 1 line
1.8.2.1
Re: [PATCH 05/14] hrtimer: Fix invalid wakeup in do_nanosleep
Quoted 5 lines · Libin
If thread is preempted before calling
set_current_state(TASK_INTERRUPTIBLE), and the other thread set the condition followed withwake_up_process. After that when this thread is re-scheduled, callingset_current_stateto set itself as stateTASK_INTERRUPTIBLE, if it is preempted again after that and before__set_current_state(TASK_RUNNING), it triggers the invalid wakeup problem.
do_nanosleep() is only called from sys_nanosleep() and sys_clock_nanosleep() user space interfaces.
So the task is not going to be woken up by some magic other thread except by a signal. The latter is handled by the scheduler which will return with state running. So what kind of problem are you trying to solve?
Thanks,
tglx
PATCH 6/14 irq: Fix invalid wakeup in irq_wait_for_interrupt
In reply to LibinIf this thread is preempted at(or before) location a, and the other thread set the condition(kthread_stop). After that when this thread is re-scheduled, calling set_current_state to set itself as state TASK_INTERRUPTIBLE, if it is preempted again after that and before __set_current_state(TASK_RUNNING), it triggers the invalid wakeup problem.
-----------------------irq_wait_for_interrupt()
-----------------------
...
//location aset_current_state(TASK_INTERRUPTIBLE);
//location b
while (!kthread_should_stop()) {
if (test_and_clear_bit(IRQTF_RUNTHREAD,
&action->thread_flags)) {
__set_current_state(TASK_RUNNING);
return 0;
}
schedule();//location c
set_current_state(TASK_INTERRUPTIBLE);
//location d
}__set_current_state(TASK_RUNNING);
...
The following circumstance will also trigger this issue:
At location c, consumer is scheduled out, and be preempted after calling set_current_state(TASK_INTERRUPTIBLE) when it be re-schdeuled.
To solve this problem, using preempt_disable() to bound the operaion that setting the task state and the conditions(set by the wake thread) validation.
--- kernel/irq/manage.c | 5 +++++ 1 file changed, 5 insertions(+)
kernel/irq/manage.c +5 -0
@@ -655,6 +655,7 @@ static irqreturn_t irq_nested_primary_handler(int irq, void *dev_id)static int irq_wait_for_interrupt (struct irqaction *action) { +preempt_disable ();set_current_state (TASK_INTERRUPTIBLE );while (!kthread_should_stop ()) {
@@ -662,12 +663,16 @@ static int irq_wait_for_interrupt(struct irqaction *action)if (test_and_clear_bit (IRQTF_RUNTHREAD , &action->thread_flags)) {__set_current_state (TASK_RUNNING ); +preempt_enable ();return 0 ; } +preempt_enable ();schedule (); +preempt_disable ();set_current_state (TASK_INTERRUPTIBLE ); }__set_current_state (TASK_RUNNING ); +preempt_enable ();return -1 ; }
Signature · 1 line
1.8.2.1
Re: [PATCH 06/14] irq: Fix invalid wakeup in irq_wait_for_interrupt
Quoted 26 lines · Libin
If this thread is preempted at(or before) location a, and the other thread set the
condition(kthread_stop). After that when this thread is re-scheduled, callingset_current_stateto set itself as stateTASK_INTERRUPTIBLE, if it is preempted again after that and before__set_current_state(TASK_RUNNING), it triggers the invalid wakeup problem.
-----------------------irq_wait_for_interrupt()
-----------------------
...
//location aset_current_state(TASK_INTERRUPTIBLE);
//location b
while (!kthread_should_stop()) {if (test_and_clear_bit(IRQTF_RUNTHREAD, &action->thread_flags)) { __set_current_state(TASK_RUNNING); return 0; } schedule();//location c set_current_state(TASK_INTERRUPTIBLE); //location d }
__set_current_state(TASK_RUNNING);
...
Quoted 6 lines
The following circumstance will also trigger this issue:
At location c, consumer is scheduled out, and be preempted after callingset_current_state(TASK_INTERRUPTIBLE)when it be re-schdeuled.To solve this problem, using
preempt_disable()to bound the operaion that setting the task state and the conditions(set by the wake thread) validation.
This is a completely pointless exercise. The irq threads are only woken from the core code itself and not by some random other threads.
The core code already handles the legit wakeups correctly without having the need for preemption disable.
Thanks,
tglx
PATCH 7/14 module: Fix invalid wakeup in wait_for_zero_refcount
In reply to LibinIf thread is preempted before calling set_current_state(TASK_INTERRUPTIBLE), and the other thread set the condition followed with wake_up_process. After that when this thread is re-scheduled, calling set_current_state to set itself as state TASK_INTERRUPTIBLE, if it is preempted again after that and before __set_current_state(TASK_RUNNING), it triggers the invalid wakeup problem.
-----------------------wait_for_zero_refcount()
-----------------------
...
for (;;) {
pr_debug("Looking at refcount...\n");
set_current_state(TASK_UNINTERRUPTIBLE);
if (module_refcount(mod) == 0)
break;
schedule();
}__set_current_state(TASK_RUNNING);
...
To solve this problem, using preempt_disable() to bound the operaion that setting the task state and the conditions(set by the wake thread) validation.
-----------------------wait_for_zero_refcount()
-----------------------
...preempt_disable();
for (;;) {
pr_debug("Looking at refcount...\n");
set_current_state(TASK_UNINTERRUPTIBLE);
if (module_refcount(mod) == 0)
break;
preempt_enable();
schedule();
preempt_disable();
}__set_current_state(TASK_RUNNING);preempt_enable();
...
--- kernel/module.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
kernel/module.c +5 -1
@@ -816,14 +816,18 @@ static void wait_for_zero_refcount(struct module *mod) {/* Since we might sleep for some time, release the mutex first */ mutex_unlock (&module_mutex); +preempt_disable ();for (;;) {pr_debug ("Looking at refcount...\n" );set_current_state (TASK_UNINTERRUPTIBLE );if (module_refcount (mod) ==0 )break ; +preempt_enable ();schedule (); +preempt_disable (); } - current->state =TASK_RUNNING ; +__set_current_state (TASK_RUNNING ); +preempt_enable ();mutex_lock (&module_mutex); }
Signature · 1 line
1.8.2.1
PATCH 8/14 namespace: Fix invalid wakeup in zap_pid_ns_processes
In reply to LibinIf thread is preempted before calling set_current_state(TASK_UNINTERRUPTIBLE), and the other thread set the condition followed with wake_up_process. After that when this thread is re-scheduled, calling set_current_state to set itself as state TASK_UNINTERRUPTIBLE, if it is preempted again after that and before __set_current_state(TASK_RUNNING), it triggers the invalid wakeup problem.
----------------------zap_pid_ns_processes()
----------------------
...
for (;;) {
set_current_state(TASK_UNINTERRUPTIBLE);
if (pid_ns->nr_hashed == init_pids)
break;
schedule();
}__set_current_state(TASK_RUNNING);
...
To solve this problem, using preempt_disable() to bound the operaion that setting the task state and the conditions(set by the wake thread) validation.
--- kernel/pid_namespace.c | 4 ++++ 1 file changed, 4 insertions(+)
kernel/pid_namespace.c +4 -0
@@ -227,13 +227,17 @@ void zap_pid_ns_processes(struct pid_namespace *pid_ns)* sys_wait4() above can't reap the TASK_DEAD children. * Make sure they all go away, see free_pid(). */ +preempt_disable ();for (;;) {set_current_state (TASK_UNINTERRUPTIBLE );if (pid_ns->nr_hashed == init_pids)break ; +preempt_enable ();schedule (); +preempt_disable (); }__set_current_state (TASK_RUNNING ); +preempt_enable ();if (pid_ns->reboot) current->signal->group_exit_code = pid_ns->reboot;
Signature · 1 line
1.8.2.1
PATCH 9/14 rcutree: Fix invalid wakeup in rcu_wait
In reply to LibinIf thread is preempted before calling set_current_state(TASK_INTERRUPTIBLE), and the other thread set the condition followed with wake_up_process. After that when this thread is re-scheduled, calling set_current_state to set itself as state TASK_INTERRUPTIBLE, if it is preempted again after that and before __set_current_state(TASK_RUNNING), it triggers the invalid wakeup problem.
To solve this problem, using preempt_disable() to bound the operaion that setting the task state and the conditions(set by the wake thread) validation.
--- kernel/rcutree.h | 4 ++++ 1 file changed, 4 insertions(+)
kernel/rcutree.h +4 -0
@@ -357,13 +357,17 @@ struct rcu_data {#define rcu_wait (cond) \ do { \ + preempt_disable(); \ for (;;) { \ set_current_state(TASK_INTERRUPTIBLE); \ if (cond) \ break; \ + preempt_enable(); \ schedule(); \ + preempt_disable(); \ } \ __set_current_state(TASK_RUNNING); \ + preempt_enable(); \ } while (0)/*
Signature · 1 line
1.8.2.1
Re: [PATCH 09/14] rcutree: Fix invalid wakeup in rcu_wait
Quoted 19 lines · Libin
If thread is preempted before calling
set_current_state(TASK_INTERRUPTIBLE), and the other thread set the condition followed withwake_up_process. After that when this thread is re-scheduled, callingset_current_stateto set itself as stateTASK_INTERRUPTIBLE, if it is preempted again after that and before__set_current_state(TASK_RUNNING), it triggers the invalid wakeup problem.To solve this problem, using
preempt_disable()to bound the operaion that setting the task state and the conditions(set by the wake thread) validation.Signed-off-by Libin <[email protected]>--- kernel/rcutree.h | 4 ++++ 1 file changed, 4 insertions(+)kernel/rcutree.h +0
-0
kernel/rcutree.h struct rcu_data { · line 364
Quoted 8 lines
kernel/rcutree.h +1
-0@@ -357,13 +357,17 @@ struct rcu_data {#define rcu_wait (cond) \ do { \ + preempt_disable(); \ for (;;) { \ set_current_state(TASK_INTERRUPTIBLE); \ if (cond) \ break; \
Here also the condition check follows the call to set_current_state(), so this patch should not be needed.
Again, if you are seeing lost wakeups on this exact code path, please let me know.
Please re-check your other patches. If they really do follow the buggy pattern you called out in your patch 0, namely the test preceding the call to set_current_state(), please send patches that restore the correct ordering.
Your use of preemption is not fixing anything. It is at best hiding the problem by making it less likely to occur.
Thanx, Paul
Quoted 13 lines
kernel/rcutree.h +3
-0+preempt_enable (); \schedule (); \ +preempt_disable (); \ } \__set_current_state (TASK_RUNNING ); \ +preempt_enable (); \ }while (0 )/* Signature · 1 line
1.8.2.1
PATCH 10/14 time: Fix invalid wakeup in alarmtimer_do_nsleep
In reply to LibinIf thread is preempted before calling set_current_state(TASK_INTERRUPTIBLE), and the other thread set the condition followed with wake_up_process. After that when this thread is re-scheduled, calling set_current_state to set itself as state TASK_INTERRUPTIBLE, if it is preempted again after that and before __set_current_state(TASK_RUNNING), it triggers the invalid wakeup problem.
To solve this problem, using preempt_disable() to bound the operaion that setting the task state and the conditions(set by the wake thread) validation.
--- kernel/time/alarmtimer.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
kernel/time/alarmtimer.c +6 -1
@@ -629,16 +629,21 @@ static enum alarmtimer_restart alarmtimer_nsleep_wakeup(struct alarm *alarm,static int alarmtimer_do_nsleep (struct alarm *alarm,ktime_t absexp) { alarm->data = (void *)current; +preempt_disable ();do {set_current_state (TASK_INTERRUPTIBLE );alarm_start (alarm, absexp); -if (likely (alarm->data)) +if (likely (alarm->data)) { +preempt_enable ();schedule (); +preempt_disable (); + }alarm_cancel (alarm); }while (alarm->data && !signal_pending (current));__set_current_state (TASK_RUNNING ); +preempt_enable ();return (alarm->data ==NULL ); }
Signature · 1 line
1.8.2.1
Re: [PATCH 10/14] time: Fix invalid wakeup in alarmtimer_do_nsleep
Quoted 8 lines · Libin
If thread is preempted before calling
set_current_state(TASK_INTERRUPTIBLE), and the other thread set the condition followed withwake_up_process. After that when this thread is re-scheduled, callingset_current_stateto set itself as stateTASK_INTERRUPTIBLE, if it is preempted again after that and before__set_current_state(TASK_RUNNING), it triggers the invalid wakeup problem.To solve this problem, using
preempt_disable()to bound the operaion that setting the task state and the conditions(set by the wake thread) validation.
See the reply to the hrtimer patch. You're solving a non issue.
Thanks,
tglx
PATCH 11/14 trace: Fix invalid wakeup in wait_to_die
In reply to LibinIf thread is preempted before calling set_current_state(TASK_INTERRUPTIBLE), and the other thread set the condition(kthread_stop). After that when this thread is re-scheduled, calling set_current_state to set itself as state TASK_INTERRUPTIBLE, if it is preempted again after that and before __set_current_state(TASK_RUNNING), it triggers the invalid wakeup problem.
-------------wait_to_die()
-------------set_current_state(TASK_INTERRUPTIBLE);
while (!kthread_should_stop()) {
schedule();
set_current_state(TASK_INTERRUPTIBLE);
}__set_current_state(TASK_RUNNING);
To solve this problem, using preempt_disable() to bound the operaion that setting the task state and the conditions(set by the wake thread) validation.
-------------wait_to_die()
-------------preempt_disable();set_current_state(TASK_INTERRUPTIBLE);
while (!kthread_should_stop()) {
preempt_enable();
schedule();
preempt_disable();
set_current_state(TASK_INTERRUPTIBLE);
}__set_current_state(TASK_RUNNING);preempt_enable();
--- kernel/trace/ring_buffer_benchmark.c | 4 ++++ 1 file changed, 4 insertions(+)
kernel/trace/ring_buffer_benchmark.c +4 -0
@@ -359,12 +359,16 @@ static void ring_buffer_producer(void)static void wait_to_die (void ) { +preempt_disable ();set_current_state (TASK_INTERRUPTIBLE );while (!kthread_should_stop ()) { +preempt_enable ();schedule (); +preempt_disable ();set_current_state (TASK_INTERRUPTIBLE ); }__set_current_state (TASK_RUNNING ); +preempt_enable (); }static int ring_buffer_consumer_thread (void *arg)
Signature · 1 line
1.8.2.1
PATCH 12/14 trace: Fix invalid wakeup in ring_buffer_consumer_thread
In reply to LibinIf thread is preempted before calling set_current_state(TASK_INTERRUPTIBLE), and the other thread set the condition followed with wake_up_process. After that when this thread is re-scheduled, calling set_current_state to set itself as state TASK_INTERRUPTIBLE, if it is preempted again after that and before __set_current_state(TASK_RUNNING), it triggers the invalid wakeup problem.
To solve this problem, using preempt_disable() to bound the operaion that setting the task state and the conditions(set by the wake thread) validation.
--- kernel/trace/ring_buffer_benchmark.c | 10 ++++++++++ 1 file changed, 10 insertions(+)
kernel/trace/ring_buffer_benchmark.c +10 -0
@@ -172,6 +172,9 @@ static enum event_status read_page(int cpu)return EVENT_FOUND ; } +/* +* Called in preemptation disabled environment. +*/ static void ring_buffer_consumer (void ) {/* toggle between reading pages and events */
@@ -204,9 +207,12 @@ static void ring_buffer_consumer(void)if (reader_finish)break ; +preempt_enable ();schedule ();__set_current_state (TASK_RUNNING ); +preempt_disable (); } + reader_finish =0 ;complete (&read_done); }
@@ -373,6 +379,7 @@ static void wait_to_die(void)static int ring_buffer_consumer_thread (void *arg) { +preempt_disable ();while (!kthread_should_stop () && !kill_test) {complete (&read_start);
@@ -382,10 +389,13 @@ static int ring_buffer_consumer_thread(void *arg)if (kthread_should_stop () || kill_test)break ; +preempt_enable ();schedule ();__set_current_state (TASK_RUNNING ); +preempt_disable (); }__set_current_state (TASK_RUNNING ); +preempt_enable ();if (kill_test)wait_to_die ();
Signature · 1 line
1.8.2.1
PATCH 13/14 workqueue: Fix invalid wakeup in rescuer_thread
In reply to LibinIf thread is preempted before calling set_current_state(TASK_INTERRUPTIBLE), and the other thread set the condition followed with wake_up_process. After that when this thread is re-scheduled, calling set_current_state to set itself as state TASK_INTERRUPTIBLE, if it is preempted again after that and before __set_current_state(TASK_RUNNING), it triggers the invalid wakeup problem.
To solve this problem, using preempt_disable() to bound the operaion that setting the task state and the conditions(set by the wake thread) validation.
--- kernel/workqueue.c | 92 +++++++++++++++++++++++++++++------------------------- 1 file changed, 49 insertions(+), 43 deletions(-)
kernel/workqueue.c +49 -43
@@ -2358,63 +2358,69 @@ static int rescuer_thread(void *__rescuer)* doesn't participate in concurrency management. */ rescuer->task->flags |=PF_WQ_WORKER ; -repeat: -set_current_state (TASK_INTERRUPTIBLE ); +for (;;) { -if (kthread_should_stop ()) { +if (kthread_should_stop ()) { + rescuer->task->flags &= ~PF_WQ_WORKER ; +return 0 ; + } + +preempt_disable (); +set_current_state (TASK_INTERRUPTIBLE ); +if (list_empty (&wq->maydays)) { +preempt_enable (); +schedule (); +preempt_disable (); + }__set_current_state (TASK_RUNNING ); - rescuer->task->flags &= ~PF_WQ_WORKER ; -return 0 ; - } +preempt_enable (); -/* see whether any pwq is asking for help */ -spin_lock_irq (&wq_mayday_lock); +/* see whether any pwq is asking for help */ +spin_lock_irq (&wq_mayday_lock); -while (!list_empty (&wq->maydays)) { -struct pool_workqueue *pwq =list_first_entry (&wq->maydays, +while (!list_empty (&wq->maydays)) { +struct pool_workqueue *pwq =list_first_entry (&wq->maydays, struct pool_workqueue, mayday_node); -struct worker_pool *pool = pwq->pool; -struct work_struct *work, *n; +struct worker_pool *pool = pwq->pool; +struct work_struct *work, *n; -__set_current_state (TASK_RUNNING ); -list_del_init (&pwq->mayday_node); +list_del_init (&pwq->mayday_node); -spin_unlock_irq (&wq_mayday_lock); +spin_unlock_irq (&wq_mayday_lock); -/* migrate to the target cpu if possible */ -worker_maybe_bind_and_lock (pool); - rescuer->pool = pool; +/* migrate to the target cpu if possible */ +worker_maybe_bind_and_lock (pool); + rescuer->pool = pool; -/* -* Slurp in all works issued via this workqueue and -* process'em. -*/ -WARN_ON_ONCE (!list_empty (&rescuer->scheduled)); -list_for_each_entry_safe (work, n, &pool->worklist, entry) -if (get_work_pwq (work) == pwq) -move_linked_works (work, scheduled, &n); +/* +* Slurp in all works issued via this workqueue and +* process'em. +*/ +WARN_ON_ONCE (!list_empty (&rescuer->scheduled)); +list_for_each_entry_safe (work, n, &pool->worklist, entry) +if (get_work_pwq (work) == pwq) +move_linked_works (work, scheduled, &n); -process_scheduled_works (rescuer); +process_scheduled_works (rescuer); -/* -* Leave this pool. If keep_working() is %true, notify a -* regular worker; otherwise, we end up with 0 concurrency -* and stalling the execution. -*/ -if (keep_working (pool)) -wake_up_worker (pool); +/* +* Leave this pool. If keep_working() is %true, notify a +* regular worker; otherwise, we end up with 0 concurrency +* and stalling the execution. +*/ +if (keep_working (pool)) +wake_up_worker (pool); - rescuer->pool =NULL ; -spin_unlock (&pool->lock); -spin_lock (&wq_mayday_lock); - } + rescuer->pool =NULL ; +spin_unlock (&pool->lock); +spin_lock (&wq_mayday_lock); + } -spin_unlock_irq (&wq_mayday_lock); +spin_unlock_irq (&wq_mayday_lock); -/* rescuers should never participate in concurrency management */ -WARN_ON_ONCE (!(rescuer->flags &WORKER_NOT_RUNNING )); -schedule (); - goto repeat; +/* rescuers should never participate in concurrency management */ +WARN_ON_ONCE (!(rescuer->flags &WORKER_NOT_RUNNING )); + } }struct wq_barrier {
Signature · 1 line
1.8.2.1
PATCH 14/14 klist: Fix invalid wakeup in klist_remove
In reply to LibinIf thread is preempted before calling set_current_state(TASK_UNINTERRUPTIBLE), and the other thread set the condition followed with wake_up_process. After that when this thread is re-scheduled, calling set_current_state to set itself as state TASK_UNINTERRUPTIBLE, if it is preempted again after that and before __set_current_state(TASK_RUNNING), it triggers the invalid wakeup problem.
--------------klist_remove()
--------------
...
for (;;) {
set_current_state(TASK_UNINTERRUPTIBLE);
if (waiter.woken)
break;
schedule();
}__set_current_state(TASK_RUNNING);
...
To solve this problem, using preempt_disable() to bound the operaion that setting the task state and the conditions(set by the wake thread) validation.
--------------klist_remove()
--------------
...preempt_disable();
for (;;) {
set_current_state(TASK_UNINTERRUPTIBLE);
if (waiter.woken)
break;
preempt_enable();
schedule();
preempt_disable();
}__set_current_state(TASK_RUNNING);preempt_enable();
...
--- lib/klist.c | 4 ++++ 1 file changed, 4 insertions(+)
lib/klist.c +4 -0
@@ -249,13 +249,17 @@ void klist_remove(struct klist_node *n)klist_del (n); +preempt_disable ();for (;;) {set_current_state (TASK_UNINTERRUPTIBLE );if (waiter.woken)break ; +preempt_enable ();schedule (); +preempt_disable (); }__set_current_state (TASK_RUNNING ); +preempt_enable (); }EXPORT_SYMBOL_GPL (klist_remove);
Signature · 1 line
1.8.2.1
Hello, Libin.
I'm completely confused by this series....
Quoted 12 lines · Libin
1)Problem Description:
The prototype of invalid wakeup problem is as follows:
========================================================================
----------------------------
Consumer Thread
----------------------------
...if (list_empty(&list)){ //location a set_current_state(TASK_INTERRUPTIBLE); schedule(); }
Quoted 8 lines
...
----------------------------
Producer Thread
----------------------------
...list_add_tail(&item, &list);wake_up_process(A);
...
This is of course broken. set_current_state() should of course come *before* the conditions is checked. This is just plain broken code.
Quoted 12 lines
In most cases, the kernel codes use a form similar to the following:
--------------------------------------------
...
//location a
...
set_current_state(TASK_INTERRUPTIBLE);
//location bwhile (list_empty(&product_list)){ schedule(); //location c set_current_state(TASK_INTERRUPTIBLE); //location d }
Quoted 8 lines
__set_current_state(TASK_RUNNING);
...
--------------------------------------------
At location a, consumer is preempted, and producer is scheduled,
adding item toproduct_listand waking up consumer. After consumer is re-scheduled, callingset_current_stateto set itself as stateTASK_INTERRUPTIBLE, if it is preempted again before__set_current_state(TASK_RUNNING), also trigger the invalid wakeup problem that the consumer will not be scheduled
Why? Getting preempted != calling schedule(). A preempted task will be rescheduled *regardless* of ifs current->state; otherwise, the whole kernel is severely broken. Tasks never get deactivated while preempted.
Quoted 5 lines
and the item be added by producer can't be consumed.
The following circumstance will also trigger this issue:
At location c, consumer is scheduled out, and be preempted after callingset_current_state(TASK_INTERRUPTIBLE)when it be re-schdeuled.
What?
Quoted 10 lines
2)Test:
I have written a test module to trigger the problem by adding some synchronization condition. I will post it in the form of an attachment soon.Test result as follows:
[103135.332683] wakeup_test: create two kernel threads - producer & consumer [103135.332686] wakeup_test: module loaded successfully [103135.332692] wakeup_test: kthread producer try to wake up the kthread consumer [103165.299865] wakeup_test: kthread consumer have waited for 30s, indicatingtrigger an invalid wakeup problem!
The most interesting question here is "what were you testing?". Can you please post the test code?
For the whole series,
Thanks.
Signature · 1 line
tejun
On 2013/8/29 21:57, Libin wrote:
....
Quoted 11 lines
2)Test:
I have written a test module to trigger the problem by adding some synchronization condition. I will post it in the form of an attachment soon.Test result as follows:
[103135.332683] wakeup_test: create two kernel threads - producer & consumer [103135.332686] wakeup_test: module loaded successfully [103135.332692] wakeup_test: kthread producer try to wake up the kthread consumer [103165.299865] wakeup_test: kthread consumer have waited for 30s, indicatingtrigger an invalid wakeup problem!
....
/*
* wakeup_test.c -- Linux kernel invalid wake up problem simulation test module *
* Written By: Libin <[email protected]>
*
* History
* -------
*/
#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/kthread.h>
#include <linux/sched.h>
#include <linux/list.h>
#include <linux/slab.h>
#include <linux/completion.h>
#include <linux/timer.h>
#define NAME "wakeup_test"
#define WAIT_TIMEOUT HZ*30
static LIST_HEAD(product_list);
struct product_struct{
int data;
struct list_head list;
};static struct task_struct *producer, *consumer;
static struct completion done;
static int producer_thread(void *unused)
{
struct product_struct *product;
product = (struct product_struct *)kmalloc(sizeof (struct product_struct), GFP_KERNEL);
product->data = 1;
list_add_tail(&product->list, &product_list);
wake_up_process(consumer);
printk(KERN_INFO "%s: kthread producer try to wake up the kthread consumer\n", NAME);
complete(&done); /* NOTE: added for problem trigger simulation */
while (!kthread_should_stop()){
set_current_state(TASK_INTERRUPTIBLE);
schedule_timeout(HZ);
}
printk(KERN_INFO "%s: kthread producer exit\n", NAME);
return 0;
}
static void simulate_preempted(void)
{
schedule();
}static void wakeup_wait_timeout(unsigned long unused)
{
printk(KERN_ERR "%s: kthread consumer have waited for %ds, "
"indicating trigger an invalid wakeup problem!\n", NAME, WAIT_TIMEOUT/HZ);
}
static int consumer_thread(void *unused)
{
struct timer_list wakeup_wait_timer;
setup_timer(&wakeup_wait_timer, wakeup_wait_timeout, (unsigned long)NULL);
wait_for_completion(&done); /* NOTE: added for problem trigger simulation */
mod_timer(&wakeup_wait_timer, jiffies + WAIT_TIMEOUT);
set_current_state(TASK_INTERRUPTIBLE);
simulate_preempted(); /* NOTE: added for problem trigger simulation */
while (list_empty(&product_list)){
schedule();
set_current_state(TASK_INTERRUPTIBLE);
}
__set_current_state(TASK_RUNNING);
del_timer_sync(&wakeup_wait_timer);
if (kthread_should_stop()){
goto out;
}
if (!list_empty(&product_list)){
printk(KERN_INFO "%s: kthread consumer be woken up successfully, all right!\n", NAME);
}
while (!kthread_should_stop()){
set_current_state(TASK_INTERRUPTIBLE);
schedule_timeout(HZ);
}out:
printk(KERN_INFO "%s: kthread consumer exit\n", NAME); return 0; }
static int __init wakeup_test_init(void)
{
producer = kthread_create(producer_thread, NULL, "producer");
consumer = kthread_create(consumer_thread, NULL, "consumer");
init_completion(&done);
wake_up_process(producer);
wake_up_process(consumer);
printk(KERN_INFO "%s: create two kernel threads - producer & consumer\n", NAME);
printk(KERN_INFO "%s: module loaded successfully\n", NAME);
return 0;
}
static void __exit wakeup_test_exit(void)
{
kthread_stop(producer);
kthread_stop(consumer);
printk(KERN_INFO "%s: module unloaded successfully\n", NAME);
}module_init(wakeup_test_init);module_exit(wakeup_test_exit);MODULE_LICENSE("GPL");
Tejun Heo
maintainerHello, again.
On Thu, Aug 29, 2013 at 10:08:13PM +0800, Libin wrote:
...
Quoted 4 lines
static void
simulate_preempted(void){ schedule(); }
Gees... of course, your test code will stall. Preemption != schedule(). Please take a look at PREEMPT_ACTIVE handling in __schedule().
Thanks.
Signature · 1 line
tejun
Hi all,
I'm so sorry, please ignore this patch set!
I have realized that there is no this problem in our kernel. Preempt_schedule() has set PREEMPT_ACTIVE before calling __schedule() and __schedule will check it if current state is not TASK_RUNNING, avoiding this preemption.
Libin
Quoted 7 lines · Libin
1)Problem Description:
The prototype of invalid wakeup problem is as follows:
========================================================================
----------------------------
Consumer Thread
----------------------------
...
Quoted 10 lines · Libin
1)Problem Description:
The prototype of invalid wakeup problem is as follows:
========================================================================
----------------------------
Consumer Thread
----------------------------
...if (list_empty(&list)){ //location a set_current_state(TASK_INTERRUPTIBLE);
You need the list_empty() test here, not above. Or you at least need to retest here.
The reason for this is that set_current_state() has a memory barrier following the assignment, which matches the memory barrier in try_to_wake_up(). This means that if the test at this point sees the list as empty, then the checks in try_to_wake_up() must see the task in TASK_INTERRUPTIBLE state and must therefore wake it up.
Thanx, Paul
Quoted 124 lines
schedule(); }...
----------------------------
Producer Thread
----------------------------
...list_add_tail(&item, &list);wake_up_process(A);
...
========================================================================Invalid wakeup problem occurs in preemptable kernel environment. The list is empty initially, if consumer is preempted after condition validated at location a, and at this time producer is woken up, adding a node to the list and calling
wake_up_processto wake up consumer. After that when consumer being scheduled by scheduler, it callsset_current_stateto set itself as stateTASK_INTERRUPTIBLE, and callingschedule()to request scheduled. After consumer being scheduled out, it has no condition to be woken up, and causing invalid wakeup problem.In most cases, the kernel codes use a form similar to the following:
--------------------------------------------
...
//location a
...
set_current_state(TASK_INTERRUPTIBLE);
//location bwhile (list_empty(&product_list)){ schedule(); //location c set_current_state(TASK_INTERRUPTIBLE); //location d }
__set_current_state(TASK_RUNNING);
...
--------------------------------------------
At location a, consumer is preempted, and producer is scheduled,
adding item toproduct_listand waking up consumer. After consumer is re-scheduled, callingset_current_stateto set itself as stateTASK_INTERRUPTIBLE, if it is preempted again before__set_current_state(TASK_RUNNING), also trigger the invalid wakeup problem that the consumer will not be scheduled and the item be added by producer can't be consumed.The following circumstance will also trigger this issue:
At location c, consumer is scheduled out, and be preempted after callingset_current_state(TASK_INTERRUPTIBLE)when it be re-schdeuled.2)Test:
I have written a test module to trigger the problem by adding some synchronization condition. I will post it in the form of an attachment soon.Test result as follows:
[103135.332683] wakeup_test: create two kernel threads - producer & consumer [103135.332686] wakeup_test: module loaded successfully [103135.332692] wakeup_test: kthread producer try to wake up the kthread consumer [103165.299865] wakeup_test: kthread consumer have waited for 30s, indicatingtrigger an invalid wakeup problem!
3)Solution:
Usingpreempt_disable()to bound the operaion that setting the task state and the conditions(set by the wake thread) validation.----------------------------------------------
...preempt_disable();set_current_state(TASK_INTERRUPTIBLE);while (list_empty(&product_list)){ preempt_enable(); schedule(); preempt_disable(); set_current_state(TASK_INTERRUPTIBLE); }
__set_current_state(TASK_RUNNING);preempt_enable();
...
----------------------------------------------
And it also can be solved as follows:
----------------------------------------------
...preempt_disable();while (list_empty(&product_list)){ set_current_state(TASK_INTERRUPTIBLE); preempt_enable(); schedule(); preempt_disable(); }
preempt_enable();
...
----------------------------------------------Libin (14):
kthread: Fix invalid wakeup in kthreadd
audit: Fix invalid wakeup inkauditd_thread
audit: Fix invalid wakeup inwait_for_auditd
exit: Fix invalid wakeup indo_wait
hrtimer: Fix invalid wakeup indo_nanosleep
irq: Fix invalid wakeup inirq_wait_for_interrupt
module: Fix invalid wakeup inwait_for_zero_refcount
namespace: Fix invalid wakeup inzap_pid_ns_processes
rcutree: Fix invalid wakeup inrcu_wait
time: Fix invalid wakeup inalarmtimer_do_nsleep
trace: Fix invalid wakeup inwait_to_die
trace: Fix invalid wakeup inring_buffer_consumer_thread
workqueue: Fix invalid wakeup inrescuer_thread
klist: Fix invalid wakeup inklist_removekernel/audit.c | 11 ++++- kernel/exit.c | 7 ++- kernel/hrtimer.c | 7 ++- kernel/irq/manage.c | 5 ++ kernel/kthread.c | 7 ++- kernel/module.c | 6 ++- kernel/pid_namespace.c | 4 ++ kernel/rcutree.h | 4 ++ kernel/time/alarmtimer.c | 7 ++- kernel/trace/ring_buffer_benchmark.c | 14 ++++++ kernel/workqueue.c | 92 +++++++++++++++++++----------------- lib/klist.c | 4 ++ 12 files changed, 118 insertions(+), 50 deletions(-)Signature · 1 line
1.8.2.1