Convert all tasklets to workqueues V3
Subsystems (17)
cover Convert all tasklets to workqueues V3
Signature · 96 lines
This is Version 3 of tasklets to work queues conversion.
Changes from this and version 2:
- Removed DECLARE_TASKLET and DECLARE_TASKLET_DISABLED macros
Instead make all instances use tasklet_init.
(Recommended by Arnd Bergmann)
- Converted the net wireless nlevent_tasket into a work queue
(there was no good way to use tasklet_init, that I know of).
- Removed fastcall from tasklet_enable (why was that needed?)
- Removed tasklet_kill_immediate.
(suggested by Oleg Nesterov)
- Removed takeover_tasklet
(also suggested by Oleg Nesterov)
- Removed smp_mb from tasklet_disable since the
flush_workqueue should provide us a barrier.
- Converted most the functions to static inline.
(Recommended by Arjan van de Ven)
- Removed unused tasklet state flags.
- Put in stub for tasklet_unlock since nothing should call it
(the only user should be fixed in 2.6.22)
- I didn't add cleanups to make the moving of code better for
checkpatch.pl. But I did fix the spots it flagged for added
code.
Prologue from Version 1, you may skip if you already read this.
Added at Andrew Morton's request. Slightly modified from V1 to
accomodate changes.
There's a very nice paper by Matthew Willcox that describes Softirqs,
Tasklets, Bottom Halves, Task Queues, Work Queues and Timers[1].
In the paper it describes the history of these items. Softirqs and
tasklets were created to replace bottom halves after a company (Mindcraft)
showed that Microsoft on a 4x SMP box would out do Linux. It was discovered
that this was due to a bottle neck caused by the design of Bottom Halves.
So Alexey Kuznetsov and Dave Miller [1] (and I'm sure others) created
softirqs and tasklets to multithread the bottom halves.
This worked well, and for the time it shut-up Microsoft^WMindcraft from
saying Linux was slow at networking.
Time passed, and Linux developed other nifty tools, like kthreads and
work queues. These run in a process context and are not as menacing to
latencies as softirqs and tasklets are. Specifically, a tasklet,
acts as a task by only being able to run the function on one CPU
at a time. The same tasklet can not run on multiple CPUS. So in that
aspect it is like a task (a task can only exist on one CPU at a time).
But a tasklet is much harder on the rest of the system because it
runs in interrupt context. This means that if a higher priority process
wants to run, it must wait for the tasklet to finish before doing so.
The most part, tasklets today are not used for time critical functions.
Running tasklets in thread context is not harmful to performance of
the overall system. But running them in interrupt context is, since
they increase the overall latency for high priority tasks.
Even in Matthew's paper, he says that work queues have replaced tasklets.
But this is not truly the case. Tasklets are common and plentiful.
But to go and replace each driver that uses a tasklet with a work queue
would be very painful.
I've developed this way to replace all tasklets with work queues without
having to change all the drivers that use them. I created an API that
uses the tasklet API as a wrapper to a work queue. This API doesn't need
to be permanent. It shows 1) that work queues can replace tasklets, and
2) we can remove a duplicate functionality from the kernel. This API
only needs to be around until we removed all uses of tasklets from
all drivers.
I just want to state that tasklets served their time well. But it's time
to give them an honorable discharge. So lets get rid of tasklets and
given them a standing salute as they leave :-)
I've booted these patches on 6 machines, i386, x86_64, and my
PowerPC Powerbook.
I'd like to give thanks to Ingo Molnar and Oleg Nesterov for reviewing
my initial patch series and giving me some pointers.
Also thanks go to (random order) Arjan van de Ven, Arnd Bergmann,
Christoph Hellwig, Linus Torvalds, Andrew Morton, Thomas Gleixner,
Daniel Walker, and others that have read the patches and offered
advice and criticism.
[1] www.wil.cx/matthew/lca2003/paper.pdfRFC PATCH 1/8 Convert the RCU tasklet into a softirq
I believe this was originally done by Dipankar Sarma. I pulled these changes from the -rt kernel.
For better preformance, RCU should use a softirq instead of a tasklet.
From: Dipankar Sarma <[email protected]>
Signed-off-by Steven Rostedt <[email protected]>
include/linux/interrupt.h +3 -0
@@ -245,6 +245,9 @@ enum#ifdef CONFIG_HIGH_RES_TIMERS HRTIMER_SOFTIRQ , #endif +RCU_SOFTIRQ ,/* Preferable RCU should always be the last softirq */ +/* Entries after this are ignored in split softirq mode */ + MAX_SOFTIRQ, };/* softirq mask and active fields moved to irq_cpustat_t in
kernel/rcupdate.c +5 -7
@@ -67,7 +67,6 @@ DEFINE_PER_CPU(struct rcu_data, rcu_data DEFINE_PER_CPU(struct rcu_data, rcu_bh_data) = { 0L };/* Fake initialization required by compiler */ -static DEFINE_PER_CPU(struct tasklet_struct, rcu_tasklet) = {NULL};static int blimit =10 ;static int qhimark =10000 ;static int qlowmark =100 ;
@@ -253,7 +252,7 @@ static void rcu_do_batch(struct rcu_dataif (!rdp->donelist) rdp->donetail = &rdp->donelist;else -tasklet_schedule (&per_cpu (rcu_tasklet, rdp->cpu)); +raise_softirq (RCU_SOFTIRQ ); }/*
@@ -405,7 +404,6 @@ static void rcu_offline_cpu(int cpu) &per_cpu(rcu_bh_data, cpu));put_cpu_var (rcu_data); put_cpu_var(rcu_bh_data); - tasklet_kill_immediate(&per_cpu(rcu_tasklet, cpu), cpu); }#else
@@ -417,7 +415,7 @@ static void rcu_offline_cpu(int cpu)#endif /* -* This does the RCU processing work from tasklet context. +* This does the RCU processing work from softirq context. */ static void __rcu_process_callbacks (struct rcu_ctrlblk *rcp,struct rcu_data *rdp)
@@ -462,7 +460,7 @@ static void __rcu_process_callbacks(strurcu_do_batch (rdp); } -static void rcu_process_callbacks (unsigned long unused) +static void rcu_process_callbacks (struct softirq_action *unused) {__rcu_process_callbacks (&rcu_ctrlblk, &__get_cpu_var (rcu_data));__rcu_process_callbacks (&rcu_bh_ctrlblk, &__get_cpu_var (rcu_bh_data));
@@ -526,7 +524,7 @@ void rcu_check_callbacks(int cpu, int usrcu_bh_qsctr_inc (cpu); }else if (!in_softirq ())rcu_bh_qsctr_inc (cpu); -tasklet_schedule (&per_cpu (rcu_tasklet, cpu)); +raise_softirq (RCU_SOFTIRQ ); } static void rcu_init_percpu_data(int cpu, struct rcu_ctrlblk *rcp,
@@ -549,7 +547,7 @@ static void __devinit rcu_online_cpu(intrcu_init_percpu_data (cpu, &rcu_ctrlblk, rdp);rcu_init_percpu_data (cpu, &rcu_bh_ctrlblk, bh_rdp); -tasklet_init (&per_cpu (rcu_tasklet, cpu), rcu_process_callbacks,0UL ); +open_softirq (RCU_SOFTIRQ , rcu_process_callbacks,NULL ); } static int __cpuinit rcu_cpu_notify(struct notifier_block *self,
Re: [RFC PATCH 1/8] Convert the RCU tasklet into a softirq
Quoted 5 lines · Steven Rostedt
I believe this was originally done by Dipankar Sarma. I pulled these changes from the -rt kernel.
For better preformance, RCU should use a softirq instead of a tasklet.
Acked-by Paul E. McKenney <[email protected]>
Tested-by Paul E. McKenney <[email protected]>
Tested with both kernbench and rcutorture on i386, x86_64, and ppc64.
Thanx, Paul
Quoted 84 lines
From: Dipankar Sarma <[email protected]>
Signed-off-by Steven Rostedt <[email protected]>Index:
linux-2.6-test/include/linux/interrupt.h
===================================================================include/linux/interrupt.h +3
-0@@ -245,6 +245,9 @@ enum#ifdef CONFIG_HIGH_RES_TIMERS HRTIMER_SOFTIRQ , #endif +RCU_SOFTIRQ ,/* Preferable RCU should always be the last softirq */ +/* Entries after this are ignored in split softirq mode */ + MAX_SOFTIRQ, };/* softirq mask and active fields moved to irq_cpustat_t in Index:
linux-2.6-test/kernel/rcupdate.c
===================================================================kernel/rcupdate.c +5
-7@@ -67,7 +67,6 @@ DEFINE_PER_CPU(struct rcu_data, rcu_data DEFINE_PER_CPU(struct rcu_data, rcu_bh_data) = { 0L };/* Fake initialization required by compiler */ -static DEFINE_PER_CPU(struct tasklet_struct, rcu_tasklet) = {NULL};static int blimit =10 ;static int qhimark =10000 ;static int qlowmark =100 ;@@ -253,7 +252,7 @@ static void rcu_do_batch(struct rcu_dataif (!rdp->donelist) rdp->donetail = &rdp->donelist;else -tasklet_schedule (&per_cpu (rcu_tasklet, rdp->cpu)); +raise_softirq (RCU_SOFTIRQ ); }/* @@ -405,7 +404,6 @@ static void rcu_offline_cpu(int cpu) &per_cpu(rcu_bh_data, cpu));put_cpu_var (rcu_data); put_cpu_var(rcu_bh_data); - tasklet_kill_immediate(&per_cpu(rcu_tasklet, cpu), cpu); }#else @@ -417,7 +415,7 @@ static void rcu_offline_cpu(int cpu)#endif /* -* This does the RCU processing work from tasklet context. +* This does the RCU processing work from softirq context. */ static void __rcu_process_callbacks (struct rcu_ctrlblk *rcp,struct rcu_data *rdp)@@ -462,7 +460,7 @@ static void __rcu_process_callbacks(strurcu_do_batch (rdp); } -static void rcu_process_callbacks (unsigned long unused) +static void rcu_process_callbacks (struct softirq_action *unused) {__rcu_process_callbacks (&rcu_ctrlblk, &__get_cpu_var (rcu_data));__rcu_process_callbacks (&rcu_bh_ctrlblk, &__get_cpu_var (rcu_bh_data));@@ -526,7 +524,7 @@ void rcu_check_callbacks(int cpu, int usrcu_bh_qsctr_inc (cpu); }else if (!in_softirq ())rcu_bh_qsctr_inc (cpu); -tasklet_schedule (&per_cpu (rcu_tasklet, cpu)); +raise_softirq (RCU_SOFTIRQ ); } static void rcu_init_percpu_data(int cpu, struct rcu_ctrlblk *rcp,@@ -549,7 +547,7 @@ static void __devinit rcu_online_cpu(intrcu_init_percpu_data (cpu, &rcu_ctrlblk, rdp);rcu_init_percpu_data (cpu, &rcu_bh_ctrlblk, bh_rdp); -tasklet_init (&per_cpu (rcu_tasklet, cpu), rcu_process_callbacks,0UL ); +open_softirq (RCU_SOFTIRQ , rcu_process_callbacks,NULL ); } static int __cpuinit rcu_cpu_notify(struct notifier_block *self,Signature · 1 line
--
Re: [RFC PATCH 1/8] Convert the RCU tasklet into a softirq
Quoted 5 lines · Paul E. McKenney
Quoted 3 lines · Steven Rostedt
I believe this was originally done by Dipankar Sarma. I pulled these changes from the -rt kernel.
Quoted 7 lines
Quoted 2 lines
For better preformance, RCU should use a softirq instead of a tasklet.
Acked-by Paul E. McKenney <[email protected]>Tested-by Paul E. McKenney <[email protected]>Tested with both kernbench and rcutorture on i386,
x86_64, and ppc64.
Btw, are there any numbers on this? Is this whole thing actually noticeable?
Linus
Re: [RFC PATCH 1/8] Convert the RCU tasklet into a softirq
Quoted 3 lines · Linus Torvalds
Btw, are there any numbers on this? Is this whole thing actually noticeable?
Besides the obvious removal of code?
The old way actually made an effort to create per_cpu tasklets! So that the tasklets *can* run simultaneously, making it in essense a softirq. Since tasklets are implemented by the softirq, with a lot of code to make a tasklet function run only on one CPU at a time, and it can run on any CPU. By converting the rcu tasklet (which had work to make it act like a softirq) to just be a softirq, we removed all the extra overhead of a tasklet.
But to answer your question. No, I didn't take any actual measurements. The changes just seemed obvious to me (and others).
-- Steve
Re: [RFC PATCH 1/8] Convert the RCU tasklet into a softirq
Quoted 2 lines · Steven Rostedt
But to answer your question. No, I didn't take any actual measurements. The changes just seemed obvious to me (and others).
btw., does anyone know about some reliable way to stress and measure RCU completion performance, via some real userspace app? I think there used to be an open-tons-of-files thing that unearthed the RCU-completion-single-threadness bottleneck a year or two ago? Anyone got a link to it (or to some other performance tool for RCU)?
Ingo
Re: [RFC PATCH 1/8] Convert the RCU tasklet into a softirq
Quoted 11 lines · Ingo Molnar
Quoted 2 lines · Steven Rostedt
But to answer your question. No, I didn't take any actual measurements. The changes just seemed obvious to me (and others).
btw., does anyone know about some reliable way to stress and measure RCU completion performance, via some real userspace app? I think there used to be an open-tons-of-files thing that unearthed the RCU-completion-single-threadness bottleneck a year or two ago? Anyone got a link to it (or to some other performance tool for RCU)?
It was more of a stress test -- it simply opened a file and immediately closes it in a tight loop. 2.6.22 survives four of the following running on a 4-CPU machine.
Attached are the kernel modules I use to measure RCU performance (and to torture RCU), but they don't really measure completion performance. (All licensed under GPL, FWIW.)
Thanx, Paul
(The following is my reconstruction of the open/close stress test.)
#include <stdio.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
int main(int argc, char *argv[])
{
int fd;
for (;;) {
fd = open("/dev/null", O_RDONLY);
if (fd >= 0) {
close(fd);
}
}
}Re: [RFC PATCH 1/8] Convert the RCU tasklet into a softirq
In reply to Linus TorvaldsQuoted 9 lines · Linus Torvalds
Quoted 8 lines · Paul E. McKenney
Quoted 5 lines · Steven Rostedt
I believe this was originally done by Dipankar Sarma. I pulled these changes from the -rt kernel.
For better preformance, RCU should use a softirq instead of a tasklet.
Quoted 7 lines
Quoted 4 lines
Acked-by Paul E. McKenney <[email protected]>Tested-by Paul E. McKenney <[email protected]>Tested with both kernbench and rcutorture on i386,
x86_64, and ppc64.Btw, are there any numbers on this? Is this whole thing actually noticeable?
i doubt it's noticeable, because the RCU tasklet was not a "true" global tasklet, it was an array of tasklets put into a PER_CPU array. (which kind of defeats the purpose of tasklets)
So there should be no/little scalability disadvantage from this type of use of tasklets, and the tasklet use was at most a wart and a small, constant micro-cost that is likely masked by other costs. (and rcu never used any of the fancier tasklet ops like tasklet_disable() either)
Ingo
RFC PATCH 2/8 Split out tasklets from softirq.c
In reply to Steven RostedtTasklets are really a separate entity from softirqs, so they deserve their own file. Also this allows us to easily replace tasklets for something else ;-)
Signed-off-by Steven Rostedt <[email protected]>
include/linux/interrupt.h +1 -111
@@ -13,6 +13,7 @@#include <linux/irqflags.h> #include <linux/bottom_half.h> #include <linux/device.h> +#include <linux/tasklet.h> #include <asm/atomic.h> #include <asm/ptrace.h> #include <asm/system.h>
@@ -268,117 +269,6 @@ extern void FASTCALL(raise_softirq_irqofextern void FASTCALL (raise_softirq (unsigned int nr)); -/* Tasklets --- multithreaded analogue of BHs. - -Main feature differing them of generic softirqs: tasklet -is running only on one CPU simultaneously. - -Main feature differing them of BHs: different tasklets -may be run simultaneously on different CPUs. - -Properties: -* If tasklet_schedule() is called, then tasklet is guaranteed -to be executed on some cpu at least once after this. -* If the tasklet is already scheduled, but its excecution is still not -started, it will be executed only once. -* If this tasklet is already running on another CPU (or schedule is called -from tasklet itself), it is rescheduled for later. -* Tasklet is strictly serialized wrt itself, but not -wrt another tasklets. If client needs some intertask synchronization, -he makes it with spinlocks. -*/ - -struct tasklet_struct -{ -struct tasklet_struct *next; -unsigned long state; -atomic_t count; -void (*func)(unsigned long ); -unsigned long data; -}; - -#define DECLARE_TASKLET (name, func, data) \ -struct tasklet_struct name = { NULL, 0, ATOMIC_INIT(0), func, data } - -#define DECLARE_TASKLET_DISABLED (name, func, data) \ -struct tasklet_struct name = { NULL, 0, ATOMIC_INIT(1), func, data } - - -enum -{ -TASKLET_STATE_SCHED ,/* Tasklet is scheduled for execution */ -TASKLET_STATE_RUN /* Tasklet is running (SMP only) */ -}; - -#ifdef CONFIG_SMP -static inline int tasklet_trylock (struct tasklet_struct *t) -{ -return !test_and_set_bit (TASKLET_STATE_RUN , &(t)->state); -} - -static inline void tasklet_unlock (struct tasklet_struct *t) -{ -smp_mb__before_clear_bit (); -clear_bit (TASKLET_STATE_RUN , &(t)->state); -} - -static inline void tasklet_unlock_wait (struct tasklet_struct *t) -{ -while (test_bit (TASKLET_STATE_RUN , &(t)->state)) {barrier (); } -} -#else -#define tasklet_trylock (t) 1 -#define tasklet_unlock_wait (t) do { } while (0) -#define tasklet_unlock (t) do { } while (0) -#endif - -extern void FASTCALL (__tasklet_schedule (struct tasklet_struct *t)); - -static inline void tasklet_schedule (struct tasklet_struct *t) -{ -if (!test_and_set_bit (TASKLET_STATE_SCHED , &t->state)) -__tasklet_schedule (t); -} - -extern void FASTCALL (__tasklet_hi_schedule (struct tasklet_struct *t)); - -static inline void tasklet_hi_schedule (struct tasklet_struct *t) -{ -if (!test_and_set_bit (TASKLET_STATE_SCHED , &t->state)) -__tasklet_hi_schedule (t); -} - - -static inline void tasklet_disable_nosync (struct tasklet_struct *t) -{ -atomic_inc (&t->count); -smp_mb__after_atomic_inc (); -} - -static inline void tasklet_disable (struct tasklet_struct *t) -{ -tasklet_disable_nosync (t); -tasklet_unlock_wait (t); -smp_mb (); -} - -static inline void tasklet_enable (struct tasklet_struct *t) -{ -smp_mb__before_atomic_dec (); -atomic_dec (&t->count); -} - -static inline void tasklet_hi_enable (struct tasklet_struct *t) -{ -smp_mb__before_atomic_dec (); -atomic_dec (&t->count); -} - -extern void tasklet_kill (struct tasklet_struct *t); -extern void tasklet_kill_immediate (struct tasklet_struct *t,unsigned int cpu); -extern void tasklet_init (struct tasklet_struct *t, -void (*func)(unsigned long ),unsigned long data); -/* * Autoprobing for irqs: *
include/linux/tasklet.h new file +120 -0
120 lines not on this page · raw message
kernel/Makefile +1 -0
@@ -21,6 +21,7 @@ obj-$(CONFIG_FUTEX) += futex.o ifeq ($(CONFIG_COMPAT),y) obj-$(CONFIG_FUTEX) += futex_compat.o endif +obj-y += tasklet.o obj-$(CONFIG_RT_MUTEXES) += rtmutex.o obj-$(CONFIG_DEBUG_RT_MUTEXES) += rtmutex-debug.o obj-$(CONFIG_RT_MUTEX_TESTER) += rtmutex-tester.o
kernel/softirq.c +0 -190
202 lines not on this page · raw message
kernel/tasklet.c new file +203 -0
203 lines not on this page · raw message
RFC PATCH 3/8 Add a tasklet is-scheduled API
In reply to Steven RostedtThis patch adds a tasklet_is_scheduled API to allow a driver to know if its tasklet is already scheduled.
Signed-off-by Steven Rostedt <[email protected]>
include/linux/tasklet.h +5 -0
@@ -107,6 +107,11 @@ static inline void tasklet_hi_enable(stratomic_dec (&t->count); } +static inline int tasklet_is_scheduled (struct tasklet_struct *t) +{ +return test_bit (TASKLET_STATE_SCHED , &t->state); +} +extern void tasklet_kill (struct tasklet_struct *t);extern void tasklet_kill_immediate (struct tasklet_struct *t,unsigned int cpu); extern void tasklet_init(struct tasklet_struct *t,
RFC PATCH 4/8 Make DRM use the tasklet is-sched API
In reply to Steven RostedtUpdate the DRM driver to use the new tasklet API, which does not rely on the tasklet implementation details.
Signed-off-by Steven Rostedt <[email protected]>
drivers/char/drm/drm_irq.c +1 -1
@@ -461,7 +461,7 @@ void drm_locked_tasklet(drm_device_t *de static DECLARE_TASKLET(drm_tasklet, drm_locked_tasklet_func, 0);if (!drm_core_check_feature (dev,DRIVER_HAVE_IRQ ) || -test_bit (TASKLET_STATE_SCHED , &drm_tasklet.state)) +tasklet_is_scheduled (&drm_tasklet))return ;spin_lock_irqsave (&dev->tasklet_lock, irqflags);
RFC PATCH 5/8 Convert net wireless tasklet to workqueue
In reply to Steven RostedtSince we want to remove DECLARE_TASKLET API, there wasn't a simple way to do that elegantly with net/core/wireless.c, so this patch just replaces that tasklet with a work queue directly.
Signed-off-by Steven Rostedt <[email protected]>
net/core/wireless.c +3 -3
@@ -1916,7 +1916,7 @@ static int __init wireless_nlevent_init(subsys_initcall (wireless_nlevent_init); -static void wireless_nlevent_process (unsigned long data) +static void wireless_nlevent_process (struct work_struct *work) {struct sk_buff *skb;
@@ -1924,7 +1924,7 @@ static void wireless_nlevent_process(unsrtnl_notify (skb,0 ,RTNLGRP_LINK ,NULL ,GFP_ATOMIC ); } -static DECLARE_TASKLET(wireless_nlevent_tasklet, wireless_nlevent_process, 0); +static DECLARE_WORK(wireless_nlevent_wq, wireless_nlevent_process);/* ---------------------------------------------------------------- */ /*
@@ -1989,7 +1989,7 @@ static inline void rtmsg_iwinfo(struct n }NETLINK_CB (skb).dst_group =RTNLGRP_LINK ;skb_queue_tail (&wireless_nlevent_queue, skb); -tasklet_schedule (&wireless_nlevent_tasklet); +schedule_work (&wireless_nlevent_wq); }#endif /* WE_EVENT_RTNETLINK */
RFC PATCH 6/8 Remove DECLARE_TASKLET_DISABLE API
In reply to Steven RostedtThere's three uses of DECLARE_TASKLET_DISABLE and this patch removes those uses. Objective is to remove the declaration completely.
Signed-off-by Steven Rostedt <[email protected]>
drivers/input/serio/hil_mlc.c +3 -2
@@ -76,7 +76,7 @@ static struct timer_list hil_mlcs_kickerstatic int hil_mlcs_probe;static void hil_mlcs_process (unsigned long unused); -DECLARE_TASKLET_DISABLED (hil_mlcs_tasklet, hil_mlcs_process,0 ); +struct tasklet_struct hil_mlcs_tasklet;/* #define HIL_MLC_DEBUG */
@@ -933,12 +933,13 @@ int hil_mlc_unregister(hil_mlc *mlc) {static int __init hil_mlc_init (void ) { +tasklet_init (&hil_mlcs_tasklet, hil_mlcs_process,0 ); +init_timer (&hil_mlcs_kicker); hil_mlcs_kicker.expires = jiffies +HZ ; hil_mlcs_kicker.function = &hil_mlcs_timer;add_timer (&hil_mlcs_kicker); -tasklet_enable (&hil_mlcs_tasklet);return 0 ; }
drivers/char/keyboard.c +3 -2
@@ -1011,7 +1011,7 @@ static void kbd_bh(unsigned long dummy) ledstate = leds; } -DECLARE_TASKLET_DISABLED (keyboard_tasklet, kbd_bh,0 ); +struct tasklet_struct keyboard_tasklet; #if defined(CONFIG_X86) || defined(CONFIG_IA64) || defined(CONFIG_ALPHA) ||\ defined(CONFIG_MIPS) || defined(CONFIG_PPC) || defined(CONFIG_SPARC) ||\
@@ -1376,11 +1376,12 @@ int __init kbd_init(void) kbd_table[i].kbdmode =VC_XLATE ; } +tasklet_init (&keyboard_tasklet, kbd_bh,0 ); + error =input_register_handler (&kbd_handler);if (error)return error; -tasklet_enable (&keyboard_tasklet);tasklet_schedule (&keyboard_tasklet);return 0 ;
net/tipc/handler.c +2 -2
@@ -49,7 +49,7 @@ static int handler_enabled = 0;static void process_signal_queue (unsigned long dummy); -static DECLARE_TASKLET_DISABLED(tipc_tasklet, process_signal_queue, 0); +static struct tasklet_struct tipc_tasklet;unsigned int tipc_k_signal (Handler routine,unsigned long argument)
@@ -102,7 +102,7 @@ int tipc_handler_start(void)return -ENOMEM ;INIT_LIST_HEAD (&signal_queue_head); -tasklet_enable (&tipc_tasklet); +tasklet_init (&tipc_tasklet, process_signal_queue,0 ); handler_enabled =1 ;return 0 ; }
include/linux/tasklet.h +0 -3
@@ -33,9 +33,6 @@ struct tasklet_struct#define DECLARE_TASKLET (name, func, data) \ struct tasklet_struct name = { NULL, 0, ATOMIC_INIT(0), func, data } -#define DECLARE_TASKLET_DISABLED (name, func, data) \ -struct tasklet_struct name = { NULL, 0, ATOMIC_INIT(1), func, data } -enum {
RFC PATCH 7/8 remove DECLARE_TASKLET API
In reply to Steven RostedtThis patch replaces the DECLARE_TASKLET for tasklet_init.
Signed-off-by Steven Rostedt <[email protected]>
arch/cris/arch-v32/drivers/cryptocop.c +3 -1
@@ -1849,7 +1849,7 @@ static int cryptocop_job_queue_insert(cr }static void cryptocop_do_tasklet (unsigned long unused); -DECLARE_TASKLET (cryptocop_tasklet, cryptocop_do_tasklet,0 ); +struct tasklet_struct cryptocop_tasklet;static void cryptocop_do_tasklet (unsigned long unused) {
@@ -3467,6 +3467,8 @@ static int init_stream_coprocessor(void)printk ("ETRAX FS stream co-processor driver v0.01, (c) 2003 Axis Communications AB\n" ); +tasklet_init (&cryptocop_tasklet, cryptocop_do_tasklet,0 ); + err =register_chrdev (CRYPTOCOP_MAJOR , cryptocop_name, &cryptocop_fops);if (err <0 ) {printk (KERN_ERR "stream co-processor: could not get major number.\n" );
arch/ia64/hp/sim/simscsi.c +3 -1
@@ -47,7 +47,7 @@static struct Scsi_Host *host;static void simscsi_interrupt (unsigned long val); -static DECLARE_TASKLET(simscsi_tasklet, simscsi_interrupt, 0); +static struct tasklet_struct simscsi_tasklet;struct disk_req {unsigned long addr;
@@ -404,6 +404,8 @@ simscsi_init(void) {int error; +tasklet_init (&simscsi_tasklet, simscsi_interrupt,0 ); + host =scsi_host_alloc (&driver_template,0 );if (!host)return -ENOMEM ;
arch/mips/lasat/picvue_proc.c +3 -1
@@ -33,7 +33,7 @@ static void pvc_display(unsigned long dapvc_write_string (pvc_lines[i],0 , i); } -static DECLARE_TASKLET(pvc_display_tasklet, &pvc_display, 0); +static struct tasklet_struct pvc_display_tasklet; static int pvc_proc_read_line(char *page, char **start, off_t off, int count,
@@ -172,6 +172,8 @@ static int __init pvc_proc_init(void) proc_entry->write_proc = pvc_proc_write_scroll; proc_entry->read_proc = pvc_proc_read_scroll; +tasklet_init (&pvc_display_tasklet, &pvc_display,0 ); +init_timer (&timer); timer.function = pvc_proc_timerfunc;
drivers/cdrom/cm206.c +3 -2
@@ -740,8 +740,6 @@ static void cm206_tasklet_func(unsigned } } -static DECLARE_TASKLET(cm206_tasklet, cm206_tasklet_func, 0); -/* This command clears the dsb_possible_media_change flag, so we must * retain it. */
@@ -1413,6 +1411,9 @@ int __init cm206_init(void)struct gendisk *disk;printk (KERN_INFO "cm206 cdrom driver " REVISION ); + +tasklet_init (&cm206_tasklet, cm206_tasklet_func,0 ); + cm206_base =probe_base_port (auto_probe ?0 : cm206_base);if (!cm206_base) {printk (" can't find adapter!\n" );
drivers/char/drm/drm_irq.c +7 -1
@@ -458,7 +458,13 @@ static void drm_locked_tasklet_func(unsivoid drm_locked_tasklet (drm_device_t *dev,void (*func)(drm_device_t *)) {unsigned long irqflags; - static DECLARE_TASKLET(drm_tasklet, drm_locked_tasklet_func, 0); +static struct tasklet_struct drm_tasklet; +static int drm_tasklet_init; + +if (!drm_tasklet_init) { +tasklet_init (&drm_tasklet, drm_locked_tasklet_func,0 ); + drm_tasklet_init =1 ; + }if (!drm_core_check_feature (dev,DRIVER_HAVE_IRQ ) ||tasklet_is_scheduled (&drm_tasklet))
drivers/char/snsc_event.c +4 -1
@@ -25,7 +25,7 @@static struct subch_data_s *event_sd;void scdrv_event (unsigned long ); -DECLARE_TASKLET (sn_sysctl_event, scdrv_event,0 ); +struct tasklet_struct sn_sysctl_event;/* * scdrv_event_interrupt
@@ -285,6 +285,9 @@ scdrv_event_init(struct sysctl_data_s *s event_sd->sd_nasid = scd->scd_nasid;spin_lock_init (&event_sd->sd_rlock); + +tasklet_init (&sn_sysctl_event, scdrv_event,0 ); +/* ask the system controllers to send events to this node */ event_sd->sd_subch =ia64_sn_sysctl_event_init (scd->scd_nasid);
drivers/media/video/vino.c +5 -2
@@ -355,8 +355,8 @@ static const char *vino_v4l_device_name_static void vino_capture_tasklet (unsigned long channel); -DECLARE_TASKLET (vino_tasklet_a, vino_capture_tasklet,VINO_CHANNEL_A ); -DECLARE_TASKLET (vino_tasklet_b, vino_capture_tasklet,VINO_CHANNEL_B ); +struct tasklet_struct vino_tasklet_a; +struct tasklet_struct vino_tasklet_b;static const struct vino_input vino_inputs[] = { {
@@ -4605,6 +4605,9 @@ static int __init vino_module_init(void)/* initialize data structures */ +tasklet_init (&vino_tasklet_a, vino_capture_tasklet,VINO_CHANNEL_A ); +tasklet_init (&vino_tasklet_b, vino_capture_tasklet,VINO_CHANNEL_B ); +spin_lock_init (&vino_drvdata->vino_lock);spin_lock_init (&vino_drvdata->input_lock);
drivers/net/wan/farsync.c +4 -2
@@ -572,8 +572,8 @@ static void do_bottom_half_rx(struct fststatic void fst_process_tx_work_q (unsigned long work_q);static void fst_process_int_work_q (unsigned long work_q); -static DECLARE_TASKLET(fst_tx_task, fst_process_tx_work_q, 0); -static DECLARE_TASKLET(fst_int_task, fst_process_int_work_q, 0); +struct tasklet_struct fst_tx_task; +struct tasklet_struct fst_int_task;static struct fst_card_info *fst_card_array[FST_MAX_CARDS ];static spinlock_t fst_work_q_lock;
@@ -2697,6 +2697,8 @@ fst_init(void)for (i =0 ; i <FST_MAX_CARDS ; i++) fst_card_array[i] =NULL ;spin_lock_init (&fst_work_q_lock); +tasklet_init (&fst_tx_task, fst_process_tx_work_q,0 ); +tasklet_init (&fst_int_task, fst_process_int_work_q,0 );return pci_register_driver (&fst_driver); }
drivers/s390/cio/qdio.c +3 -1
@@ -98,7 +98,7 @@ static volatile struct qdio_q *tiq_list=static DEFINE_SPINLOCK (ttiq_list_lock);static int register_thinint_result;static void tiqdio_tl (unsigned long ); -static DECLARE_TASKLET(tiqdio_tasklet,tiqdio_tl,0); +static struct tasklet_struct tiqdio_tasklet;/* not a macro, as one of the arguments is atomic_read */ static inline int
@@ -3839,6 +3839,8 @@ init_QDIO(void)printk ("qdio: loading %s\n" ,version); +tasklet_init (&tiqdio_tasklet, tiqdio_tl,0 ); + res=qdio_get_qdio_memory ();if (res)return res;
drivers/s390/crypto/ap_bus.c +3 -1
@@ -80,7 +80,7 @@ static DECLARE_WORK(ap_config_work, ap_s* Tasklet & timer for AP request polling. */ static struct timer_list ap_poll_timer =TIMER_INITIALIZER (ap_poll_timeout,0 ,0 ); -static DECLARE_TASKLET(ap_tasklet, ap_poll_all, 0); +static struct tasklet_struct ap_tasklet;static atomic_t ap_poll_requests =ATOMIC_INIT (0 );static DECLARE_WAIT_QUEUE_HEAD (ap_poll_wait);static struct task_struct *ap_poll_kthread =NULL ;
@@ -1188,6 +1188,8 @@ int __init ap_module_init(void) }register_reset_call (&ap_reset_call); +tasklet_init (&ap_tasklet, ap_poll_all,0 ); +/* Create /sys/bus/ap. */ rc =bus_register (&ap_bus_type);if (rc)
drivers/usb/gadget/net2280.c +4 -1
@@ -511,7 +511,7 @@ static void do_free(unsigned long ignorespin_unlock_irq (&buflock); } -static DECLARE_TASKLET(deferred_free, do_free, 0); +static struct tasklet_struct deferred_free;static void net2280_free_buffer (
@@ -3051,6 +3051,9 @@ static int __init init (void) {if (!use_dma) use_dma_chaining =0 ; + +tasklet_init (&deferred_free, do_free,0 ); +return pci_register_driver (&net2280_pci_driver); }module_init (init);
drivers/usb/gadget/omap_udc.c +4 -1
@@ -365,7 +365,7 @@ static void do_free(unsigned long ignorespin_unlock_irq (&buflock); } -static DECLARE_TASKLET(deferred_free, do_free, 0); +static struct tasklet_struct deferred_free; static void omap_free_buffer( struct usb_ep *_ep,
@@ -3182,6 +3182,9 @@ static int __init udc_init(void)#endif "%s\n" , driver_desc, use_dma ? " (dma)" : ""); + +tasklet_init (&deferred_free, do_free,0 ); +return platform_driver_register (&udc_driver); }module_init (udc_init);
kernel/irq/resend.c +10 -1
@@ -44,7 +44,7 @@ static void resend_irqs(unsigned long ar }/* Tasklet to handle resend: */ -static DECLARE_TASKLET(resend_tasklet, resend_irqs, 0); +static struct tasklet_struct resend_tasklet;#endif
@@ -75,3 +75,12 @@ void check_irq_resend(struct irq_desc *d } } } + +#ifdef CONFIG_HARDIRQS_SW_RESEND +static int __init irq_resend_init (void ) +{ +tasklet_init (&resend_tasklet, resend_irqs,0 ); +} + +core_initcall (irq_resend_init); +#endif
net/atm/pppoatm.c +8 -1
@@ -281,7 +281,14 @@ static int pppoatm_assign_vcc(struct atm* Each PPPoATM instance has its own tasklet - this is just a * prototypical one used to initialize them */ - static const DECLARE_TASKLET(tasklet_proto, pppoatm_wakeup_sender, 0); +static struct tasklet_struct tasklet_proto; +static int tasklet_proto_init; + +if (!tasklet_proto_init) { +tasklet_init (&tasklet_proto, pppoatm_wakeup_sender,0 ); + tasklet_proto_init =1 ; + } +if (copy_from_user (&be, arg,sizeof be))return -EFAULT ; if (be.encaps != PPPOATM_ENCAPS_AUTODETECT &&
net/iucv/iucv.c +2 -1
@@ -103,7 +103,7 @@ static cpumask_t iucv_buffer_cpumask = Cstatic cpumask_t iucv_irq_cpumask =CPU_MASK_NONE ;static void iucv_tasklet_handler (unsigned long ); -static DECLARE_TASKLET(iucv_tasklet, iucv_tasklet_handler,0); +static struct tasklet_struct iucv_tasklet;enum iucv_command_codes {IUCV_QUERY =0 ,
@@ -1525,6 +1525,7 @@ static int iucv_init(void) rc = -EPROTONOSUPPORT ; goto out; } +tasklet_init (&iucv_tasklet, iucv_tasklet_handler,0 ); rc =iucv_query_maxconn ();if (rc) goto out;
include/linux/tasklet.h +0 -4
@@ -30,10 +30,6 @@ struct tasklet_structunsigned long data; }; -#define DECLARE_TASKLET (name, func, data) \ -struct tasklet_struct name = { NULL, 0, ATOMIC_INIT(0), func, data } - -enum {TASKLET_STATE_SCHED ,/* Tasklet is scheduled for execution */
RFC PATCH 8/8 Convert tasklets to work queues
In reply to Steven RostedtThis patch replaces the tasklet implementation with a work queue implementation while keeping the tasklet API. The API is still the same, and the drivers don't know that a work queue is being used.
This is (for now) a proof of concept approach to using work queues instead of tasklets. More can be done. For one thing, we could make individual work queues for each tasklet instead of using the global ktasklet_wq. But for now it's just easier to do this.
Signed-off-by Steven Rostedt <[email protected]>
kernel/Makefile +1 -1
@@ -21,7 +21,7 @@ obj-$(CONFIG_FUTEX) += futex.o ifeq ($(CONFIG_COMPAT),y) obj-$(CONFIG_FUTEX) += futex_compat.o endif -obj-y += tasklet.o +obj-y += tasklet_work.o obj-$(CONFIG_RT_MUTEXES) += rtmutex.o obj-$(CONFIG_DEBUG_RT_MUTEXES) += rtmutex-debug.o obj-$(CONFIG_RT_MUTEX_TESTER) += rtmutex-tester.o
init/main.c +2 -2
@@ -120,7 +120,7 @@ EXPORT_SYMBOL(system_state);extern void time_init (void );/* Default late time init is NULL. archs can override this later. */ void (*late_time_init)(void ); -extern void softirq_init (void ); +extern void init_tasklets (void );/* Untouched command line saved by arch-specific code. */ char __initdata boot_command_line[COMMAND_LINE_SIZE ];
@@ -563,7 +563,6 @@ asmlinkage void __init start_kernel(voidpidhash_init ();init_timers ();hrtimers_init (); -softirq_init ();timekeeping_init ();time_init ();profile_init ();
@@ -706,6 +705,7 @@ static void __init do_basic_setup(void) {/* drivers will send hotplug events */ init_workqueues (); +init_tasklets ();usermodehelper_init ();driver_init ();init_irq_proc ();
include/linux/tasklet.h +42 -76
156 lines not on this page · raw message
kernel/tasklet_work.c new file +59 -0
59 lines not on this page · raw message
kernel/tasklet.c deleted +0 -203
203 lines not on this page · raw message
kernel/softirq.c +0 -1
@@ -425,7 +425,6 @@ static int __cpuinit cpu_callback(struct p =per_cpu (ksoftirqd, hotcpu);per_cpu (ksoftirqd, hotcpu) =NULL ;kthread_stop (p); -takeover_tasklets (hotcpu);break ;#endif /* CONFIG_HOTPLUG_CPU */ }