Improve preempt-scheduling and x86 user access v3

PATCH In review Andi Kleen 5 participants 14 messages Aug 16, 2013 – Sep 11, 2013

Subsystems (62)

Networking Network drivers Memory management Arm (32-bit) +58 more subsystems

lore

Share & tools

lore

Andi Kleen

author

2013-08-16 22:48 UTC #0 lore raw

Various optimizations related to CONFIG_PREEMPT_VOLUNTARY and x86 uaccess

- Optimize copy_*_inatomic on x86-64 to handle 1-8 bytes without string instructions
- Inline might_sleep and other preempt code
to optimize various preemption paths
This costs about 10k text size, but generates far better code with less unnecessary function calls.

This patch kit is an attempt to get us back to sane code, mostly by doing proper inlining and doing sleep checks in the right place. Unfortunately I had to add one tree sweep to avoid an nasty include loop.

Unfortunately some of the inlining requires a tree sweep for moving might_sleep and friends to sched.h

v2: Now completely remove reschedule checks for uaccess functions.
v3: Drop unnecessary changes (thanks Michael).
Now it only optimized copy_*_inatomic and inlines might_sleep()

Andi Kleen

author

2013-08-16 22:47 UTC #1 lore raw

PATCH 1/6 x86: Add 1/2/4/8 byte optimization to 64bit __copy_{from,to}_user_inatomic

From: Andi Kleen <[email protected]>

The 64bit __copy_{from,to}_user_inatomic always called copy_from_user_generic, but skipped the special optimizations for 1/2/4/8 byte accesses.

This especially hurts the futex call, which accesses the 4 byte futex user value with a complicated fast string operation in a function call, instead of a single movl.

Use __copy_{from,to}_user for _inatomic instead to get the same optimizations. The only problem was the might_fault() in those functions. So move that to new wrapper and call __copy_{f,t}_user_nocheck() from *_inatomic directly.

32bit already did this correctly by duplicating the code.

Signed-off-by Andi Kleen <[email protected]>

---
 arch/x86/include/asm/uaccess_64.h | 24 ++++++++++++++++++------
 1 file changed, 18 insertions(+), 6 deletions(-)

arch/x86/include/asm/uaccess_64.h +18 -6

index 4f7923d..64476bb 100644
@@ -77,11 +77,10 @@ int copy_to_user(void __user *dst, const void *src, unsigned size)
 }
 
 static __always_inline __must_check
-int __copy_from_user(void *dst, const void __user *src, unsigned size)
+int __copy_from_user_nocheck(void *dst, const void __user *src, unsigned size)
 {
 	int ret = 0;
 
-	might_fault();
 	if (!__builtin_constant_p(size))
 		return copy_user_generic(dst, (__force void *)src, size);
 	switch (size) {

@@ -121,11 +120,17 @@ int __copy_from_user(void *dst, const void __user *src, unsigned size)
 }
 
 static __always_inline __must_check
-int __copy_to_user(void __user *dst, const void *src, unsigned size)
+int __copy_from_user(void *dst, const void __user *src, unsigned size)
+{
+	might_fault();
+	return __copy_from_user_nocheck(dst, src, size);
+}
+
+static __always_inline __must_check
+int __copy_to_user_nocheck(void __user *dst, const void *src, unsigned size)
 {
 	int ret = 0;
 
-	might_fault();
 	if (!__builtin_constant_p(size))
 		return copy_user_generic((__force void *)dst, src, size);
 	switch (size) {

@@ -165,6 +170,13 @@ int __copy_to_user(void __user *dst, const void *src, unsigned size)
 }
 
 static __always_inline __must_check
+int __copy_to_user(void __user *dst, const void *src, unsigned size)
+{
+	might_fault();
+	return __copy_to_user_nocheck(dst, src, size);
+}
+
+static __always_inline __must_check
 int __copy_in_user(void __user *dst, const void __user *src, unsigned size)
 {
 	int ret = 0;

@@ -220,13 +232,13 @@ int __copy_in_user(void __user *dst, const void __user *src, unsigned size)
 static __must_check __always_inline int
 __copy_from_user_inatomic(void *dst, const void __user *src, unsigned size)
 {
-	return copy_user_generic(dst, (__force const void *)src, size);
+	return __copy_from_user_nocheck(dst, (__force const void *)src, size);
 }
 
 static __must_check __always_inline int
 __copy_to_user_inatomic(void __user *dst, const void *src, unsigned size)
 {
-	return copy_user_generic((__force void *)dst, src, size);
+	return __copy_to_user_nocheck((__force void *)dst, src, size);
 }
 
 extern long __copy_user_nocache(void *dst, const void __user *src,

Signature · 1 line

1.8.3.1

tip-bot for Andi Kleen

merge bot

2013-09-10 23:30 UTC #2 lore raw

[tip:x86/uaccess] x86: Add 1/2/4/ 8 byte optimization to 64bit __copy_{from,to}_user_inatomic

x86: Add 1/2/4/8 byte optimization to 64bit __copy_{from,to}_user_inatomic

Commit-ID:  ff47ab4ff3cddfa7bc1b25b990e24abe2ae474ff
Gitweb:     http://git.kernel.org/tip/ff47ab4ff3cddfa7bc1b25b990e24abe2ae474ff
Author:     Andi Kleen <[email protected]>
AuthorDate: Fri, 16 Aug 2013 14:17:19 -0700
Committer:  H. Peter Anvin <[email protected]>
CommitDate: Tue, 10 Sep 2013 15:27:43 -0700

x86: Add 1/2/4/8 byte optimization to 64bit __copy_{from,to}_user_inatomic

The 64bit __copy_{from,to}_user_inatomic always called copy_from_user_generic, but skipped the special optimizations for 1/2/4/8 byte accesses.

This especially hurts the futex call, which accesses the 4 byte futex user value with a complicated fast string operation in a function call, instead of a single movl.

32bit already did this correctly by duplicating the code.

Signed-off-by Andi Kleen <[email protected]>

Link http://lkml.kernel.org/r/[email protected] open here

Signed-off-by H. Peter Anvin <[email protected]>

---
 arch/x86/include/asm/uaccess_64.h | 24 ++++++++++++++++++------
 1 file changed, 18 insertions(+), 6 deletions(-)

arch/x86/include/asm/uaccess_64.h +18 -6

index 4f7923d..64476bb 100644
@@ -77,11 +77,10 @@ int copy_to_user(void __user *dst, const void *src, unsigned size)
 }
 
 static __always_inline __must_check
-int __copy_from_user(void *dst, const void __user *src, unsigned size)
+int __copy_from_user_nocheck(void *dst, const void __user *src, unsigned size)
 {
 	int ret = 0;
 
-	might_fault();
 	if (!__builtin_constant_p(size))
 		return copy_user_generic(dst, (__force void *)src, size);
 	switch (size) {

@@ -121,11 +120,17 @@ int __copy_from_user(void *dst, const void __user *src, unsigned size)
 }
 
 static __always_inline __must_check
-int __copy_to_user(void __user *dst, const void *src, unsigned size)
+int __copy_from_user(void *dst, const void __user *src, unsigned size)
+{
+	might_fault();
+	return __copy_from_user_nocheck(dst, src, size);
+}
+
+static __always_inline __must_check
+int __copy_to_user_nocheck(void __user *dst, const void *src, unsigned size)
 {
 	int ret = 0;
 
-	might_fault();
 	if (!__builtin_constant_p(size))
 		return copy_user_generic((__force void *)dst, src, size);
 	switch (size) {

@@ -165,6 +170,13 @@ int __copy_to_user(void __user *dst, const void *src, unsigned size)
 }
 
 static __always_inline __must_check
+int __copy_to_user(void __user *dst, const void *src, unsigned size)
+{
+	might_fault();
+	return __copy_to_user_nocheck(dst, src, size);
+}
+
+static __always_inline __must_check
 int __copy_in_user(void __user *dst, const void __user *src, unsigned size)
 {
 	int ret = 0;

@@ -220,13 +232,13 @@ int __copy_in_user(void __user *dst, const void __user *src, unsigned size)
 static __must_check __always_inline int
 __copy_from_user_inatomic(void *dst, const void __user *src, unsigned size)
 {
-	return copy_user_generic(dst, (__force const void *)src, size);
+	return __copy_from_user_nocheck(dst, (__force const void *)src, size);
 }
 
 static __must_check __always_inline int
 __copy_to_user_inatomic(void __user *dst, const void *src, unsigned size)
 {
-	return copy_user_generic((__force void *)dst, src, size);
+	return __copy_to_user_nocheck((__force void *)dst, src, size);
 }
 
 extern long __copy_user_nocache(void *dst, const void __user *src,

Andi Kleen

author

2013-08-16 22:48 UTC #3 lore raw

PATCH 2/6 x86: Include linux/sched.h in asm/uaccess.h

In reply to Andi Kleen

From: Andi Kleen <[email protected]>

uaccess.h uses might_sleep, but there is currently no explicit include for this. Since a upcoming patch moves might_sleep into sched.h include sched.h here.

Signed-off-by Andi Kleen <[email protected]>

---
 arch/x86/include/asm/uaccess.h | 1 +
 1 file changed, 1 insertion(+)

arch/x86/include/asm/uaccess.h +1 -0

index 5ee2687..8fa3bd6 100644
@@ -3,6 +3,7 @@
 /*
  * User space memory access functions
  */
+#include <linux/sched.h>
 #include <linux/errno.h>
 #include <linux/compiler.h>
 #include <linux/thread_info.h>

Signature · 1 line

1.8.3.1

Andi Kleen

author

2013-08-16 22:46 UTC #4 lore raw

PATCH 3/6 tree-sweep: Include linux/sched.h for might_sleep users

In reply to Andi Kleen

From: Andi Kleen <[email protected]>

might_sleep is moving from linux/kernel.h to linux/sched.h, so any users need to include linux/sched.h

This was done with a mechanistic script and some uses may be redundant (already included in some other include file). However it's good practice to always include any needed symbols from the top level .c file.

Tested with x86-64 allyesconfig. I used to do a x86-32 allyesconfig on a old kernel, but since that is broken now I didn't retest.

Signed-off-by Andi Kleen <[email protected]>

---
 arch/arm/common/mcpm_entry.c                              | 1 +
 arch/arm/mach-omap2/omap_hwmod.c                          | 1 +
 arch/arm/mm/highmem.c                                     | 1 +
 arch/blackfin/kernel/bfin_gpio.c                          | 1 +
 arch/frv/mm/highmem.c                                     | 1 +
 arch/m32r/include/asm/uaccess.h                           | 1 +
 arch/microblaze/include/asm/highmem.h                     | 1 +
 arch/mn10300/include/asm/uaccess.h                        | 1 +
 arch/parisc/include/asm/cacheflush.h                      | 1 +
 arch/powerpc/include/asm/highmem.h                        | 1 +
 arch/powerpc/kernel/rtas.c                                | 1 +
 arch/powerpc/lib/checksum_wrappers_64.c                   | 1 +
 arch/powerpc/lib/usercopy_64.c                            | 1 +
 arch/tile/mm/highmem.c                                    | 1 +
 arch/x86/include/asm/checksum_32.h                        | 1 +
 arch/x86/lib/csum-wrappers_64.c                           | 1 +
 arch/x86/mm/highmem_32.c                                  | 1 +
 arch/x86/mm/mmio-mod.c                                    | 1 +
 block/blk-cgroup.c                                        | 1 +
 block/blk-core.c                                          | 1 +
 block/genhd.c                                             | 1 +
 drivers/base/dma-buf.c                                    | 1 +
 drivers/block/rsxx/dev.c                                  | 1 +
 drivers/dma/ipu/ipu_irq.c                                 | 1 +
 drivers/gpio/gpiolib.c                                    | 1 +
 drivers/ide/ide-io.c                                      | 1 +
 drivers/infiniband/hw/amso1100/c2_cq.c                    | 1 +
 drivers/infiniband/hw/cxgb3/iwch_cm.c                     | 1 +
 drivers/infiniband/hw/cxgb4/cm.c                          | 1 +
 drivers/md/dm.c                                           | 1 +
 drivers/md/raid5.c                                        | 1 +
 drivers/mmc/core/core.c                                   | 1 +
 drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c          | 1 +
 drivers/net/ethernet/broadcom/bnx2x/bnx2x_sp.c            | 1 +
 drivers/net/ethernet/broadcom/bnx2x/bnx2x_sriov.h         | 1 +
 drivers/net/ethernet/broadcom/bnx2x/bnx2x_stats.c         | 1 +
 drivers/net/ethernet/intel/e1000e/netdev.c                | 1 +
 drivers/net/ethernet/intel/igbvf/netdev.c                 | 1 +
 drivers/net/ethernet/sfc/falcon.c                         | 1 +
 drivers/net/ieee802154/at86rf230.c                        | 1 +
 drivers/net/ieee802154/fakelb.c                           | 1 +
 drivers/net/wireless/ath/carl9170/usb.c                   | 1 +
 drivers/net/wireless/ath/wil6210/wmi.c                    | 1 +
 drivers/net/wireless/b43/dma.c                            | 1 +
 drivers/net/wireless/b43/main.c                           | 1 +
 drivers/net/wireless/b43/phy_a.c                          | 1 +
 drivers/net/wireless/b43/phy_g.c                          | 1 +
 drivers/net/wireless/b43legacy/dma.c                      | 1 +
 drivers/net/wireless/b43legacy/radio.c                    | 1 +
 drivers/net/wireless/cw1200/cw1200_spi.c                  | 1 +
 drivers/net/wireless/iwlwifi/dvm/sta.c                    | 1 +
 drivers/net/wireless/iwlwifi/iwl-op-mode.h                | 1 +
 drivers/net/wireless/iwlwifi/iwl-trans.h                  | 1 +
 drivers/net/wireless/libertas_tf/cmd.c                    | 1 +
 drivers/pci/iov.c                                         | 1 +
 drivers/pci/pci.c                                         | 1 +
 drivers/platform/olpc/olpc-ec.c                           | 1 +
 drivers/ssb/driver_pcicore.c                              | 1 +
 drivers/staging/lustre/lustre/llite/remote_perm.c         | 1 +
 drivers/staging/lustre/lustre/obdclass/cl_lock.c          | 1 +
 drivers/staging/lustre/lustre/obdclass/cl_object.c        | 1 +
 drivers/staging/lustre/lustre/obdclass/cl_page.c          | 1 +
 drivers/staging/lustre/lustre/osc/osc_lock.c              | 1 +
 drivers/staging/lustre/lustre/osc/osc_page.c              | 1 +
 drivers/staging/lustre/lustre/ptlrpc/client.c             | 1 +
 drivers/staging/lustre/lustre/ptlrpc/gss/gss_cli_upcall.c | 1 +
 drivers/staging/lustre/lustre/ptlrpc/gss/gss_pipefs.c     | 1 +
 drivers/staging/lustre/lustre/ptlrpc/sec.c                | 1 +
 drivers/staging/lustre/lustre/ptlrpc/sec_config.c         | 1 +
 drivers/staging/lustre/lustre/ptlrpc/sec_gc.c             | 1 +
 drivers/usb/core/hcd.c                                    | 1 +
 drivers/usb/core/urb.c                                    | 1 +
 drivers/video/atmel_lcdfb.c                               | 1 +
 fs/block_dev.c                                            | 1 +
 fs/buffer.c                                               | 1 +
 fs/dcache.c                                               | 1 +
 fs/ext3/inode.c                                           | 1 +
 fs/ext4/ext4_jbd2.c                                       | 1 +
 fs/ext4/inode.c                                           | 1 +
 fs/ext4/mballoc.c                                         | 1 +
 fs/file_table.c                                           | 1 +
 fs/inode.c                                                | 1 +
 fs/jbd/revoke.c                                           | 1 +
 fs/jbd2/revoke.c                                          | 1 +
 fs/locks.c                                                | 1 +
 fs/nfs/nfs4filelayoutdev.c                                | 1 +
 fs/nfs/nfs4proc.c                                         | 1 +
 fs/nfs/nfs4state.c                                        | 1 +
 fs/nilfs2/the_nilfs.c                                     | 1 +
 fs/xfs/xfs_mount.c                                        | 1 +
 include/asm-generic/gpio.h                                | 1 +
 include/linux/buffer_head.h                               | 1 +
 include/linux/clk.h                                       | 1 +
 include/linux/gpio.h                                      | 1 +
 include/linux/highmem.h                                   | 1 +
 include/linux/pagemap.h                                   | 1 +
 kernel/fork.c                                             | 1 +
 kernel/freezer.c                                          | 1 +
 kernel/irq/chip.c                                         | 1 +
 kernel/nsproxy.c                                          | 1 +
 kernel/printk/printk.c                                    | 1 +
 kernel/sched/core.c                                       | 1 +
 kernel/smp.c                                              | 1 +
 mm/hugetlb.c                                              | 1 +
 mm/memory.c                                               | 1 +
 mm/mmap.c                                                 | 1 +
 mm/rmap.c                                                 | 1 +
 net/caif/cfcnfg.c                                         | 1 +
 net/mac80211/driver-ops.h                                 | 1 +
 net/mac80211/key.c                                        | 1 +
 net/mac80211/main.c                                       | 1 +
 net/mac80211/sta_info.c                                   | 1 +
 net/phonet/pep.c                                          | 1 +
 net/sunrpc/clnt.c                                         | 1 +
 net/wimax/op-msg.c                                        | 1 +
 net/wimax/op-reset.c                                      | 1 +
 net/wimax/op-rfkill.c                                     | 1 +
 net/wireless/wext-proc.c                                  | 1 +
 sound/core/info.c                                         | 1 +
 virt/kvm/async_pf.c                                       | 1 +
 120 files changed, 120 insertions(+)

arch/arm/common/mcpm_entry.c +1 -0

index 370236d..c083f90 100644
@@ -10,6 +10,7 @@
  */
 
 #include <linux/kernel.h>
+#include <linux/sched.h>
 #include <linux/init.h>
 #include <linux/irqflags.h>

arch/arm/mach-omap2/omap_hwmod.c +1 -0

index 7f4db12..04a2674 100644
@@ -127,6 +127,7 @@
  */
 #undef DEBUG
 
+#include <linux/sched.h>
 #include <linux/kernel.h>
 #include <linux/errno.h>
 #include <linux/io.h>

arch/arm/mm/highmem.c +1 -0

index 21b9e1b..a8be1f1 100644
@@ -10,6 +10,7 @@
  * published by the Free Software Foundation.
  */
 
+#include <linux/sched.h>
 #include <linux/module.h>
 #include <linux/highmem.h>
 #include <linux/interrupt.h>

arch/blackfin/kernel/bfin_gpio.c +1 -0

index ed978f1..0efe3d5 100644
@@ -6,6 +6,7 @@
  * Licensed under the GPL-2 or later
  */
 
+#include <linux/sched.h>
 #include <linux/delay.h>
 #include <linux/module.h>
 #include <linux/err.h>

arch/frv/mm/highmem.c +1 -0

index bed9a9b..766ff5b 100644
@@ -8,6 +8,7 @@
  * as published by the Free Software Foundation; either version
  * 2 of the License, or (at your option) any later version.
  */
+#include <linux/sched.h>
 #include <linux/highmem.h>
 #include <linux/module.h>

arch/m32r/include/asm/uaccess.h +1 -0

index 84fe7ba..a097157 100644
@@ -11,6 +11,7 @@
 /*
  * User space memory access functions
  */
+#include <linux/sched.h>
 #include <linux/errno.h>
 #include <linux/thread_info.h>
 #include <asm/page.h>

arch/microblaze/include/asm/highmem.h +1 -0

index d046389..40c5b59 100644
@@ -20,6 +20,7 @@
 #ifdef __KERNEL__
 
 #include <linux/init.h>
+#include <linux/sched.h>
 #include <linux/interrupt.h>
 #include <linux/uaccess.h>
 #include <asm/fixmap.h>

arch/mn10300/include/asm/uaccess.h +1 -0

index 5372787..274c9c2 100644
@@ -14,6 +14,7 @@
 /*
  * User space memory access functions
  */
+#include <linux/sched.h>
 #include <linux/thread_info.h>
 #include <linux/kernel.h>
 #include <asm/page.h>

arch/parisc/include/asm/cacheflush.h +1 -0

index f0e2784..ab4ed76 100644
@@ -2,6 +2,7 @@
 #define _PARISC_CACHEFLUSH_H
 
 #include <linux/mm.h>
+#include <linux/sched.h>
 #include <linux/uaccess.h>
 #include <asm/tlbflush.h>

arch/powerpc/include/asm/highmem.h +1 -0

index caaf6e0..721bc1b 100644
@@ -22,6 +22,7 @@
 
 #ifdef __KERNEL__
 
+#include <linux/sched.h>
 #include <linux/interrupt.h>
 #include <asm/kmap_types.h>
 #include <asm/tlbflush.h>

arch/powerpc/kernel/rtas.c +1 -0

index 80b5ef4..e479bcf 100644
@@ -11,6 +11,7 @@
  *      2 of the License, or (at your option) any later version.
  */
 
+#include <linux/sched.h>
 #include <stdarg.h>
 #include <linux/kernel.h>
 #include <linux/types.h>

arch/powerpc/lib/checksum_wrappers_64.c +1 -0

index 08e3a33..8d9598c 100644
@@ -17,6 +17,7 @@
  *
  * Author: Anton Blanchard <[email protected]>
  */
+#include <linux/sched.h>
 #include <linux/export.h>
 #include <linux/compiler.h>
 #include <linux/types.h>

arch/powerpc/lib/usercopy_64.c +1 -0

index 5eea6f3..8c8cfa6 100644
@@ -6,6 +6,7 @@
  * as published by the Free Software Foundation; either version
  * 2 of the License, or (at your option) any later version.
  */
+#include <linux/sched.h>
 #include <linux/module.h>
 #include <asm/uaccess.h>

arch/tile/mm/highmem.c +1 -0

index 347d123..f82b1e0 100644
@@ -12,6 +12,7 @@
  *   more details.
  */
 
+#include <linux/sched.h>
 #include <linux/highmem.h>
 #include <linux/module.h>
 #include <linux/pagemap.h>

arch/x86/include/asm/checksum_32.h +1 -0

index 46fc474..b9aa5d0 100644
@@ -2,6 +2,7 @@
 #define _ASM_X86_CHECKSUM_32_H
 
 #include <linux/in6.h>
+#include <linux/sched.h>
 
 #include <asm/uaccess.h>

arch/x86/lib/csum-wrappers_64.c +1 -0

index 25b7ae8..aaba241 100644
@@ -4,6 +4,7 @@
  *
  * Wrappers of assembly checksum functions for x86-64.
  */
+#include <linux/sched.h>
 #include <asm/checksum.h>
 #include <linux/module.h>

arch/x86/mm/highmem_32.c +1 -0

index 4500142..1212a56 100644
@@ -1,3 +1,4 @@
+#include <linux/sched.h>
 #include <linux/highmem.h>
 #include <linux/module.h>
 #include <linux/swap.h> /* for totalram_pages */

arch/x86/mm/mmio-mod.c +1 -0

index 0057a7a..07b0235 100644
@@ -24,6 +24,7 @@
 
 #define DEBUG 1
 
+#include <linux/sched.h>
 #include <linux/module.h>
 #include <linux/debugfs.h>
 #include <linux/slab.h>

block/blk-cgroup.c +1 -0

index 290792a..20664e2 100644
@@ -11,6 +11,7 @@
  * 	              Nauman Rafique <[email protected]>
  */
 #include <linux/ioprio.h>
+#include <linux/sched.h>
 #include <linux/kdev_t.h>
 #include <linux/module.h>
 #include <linux/err.h>

block/blk-core.c +1 -0

index 93a18d1..cf9bb93 100644
@@ -11,6 +11,7 @@
 /*
  * This handles all read/write requests to block devices
  */
+#include <linux/sched.h>
 #include <linux/kernel.h>
 #include <linux/module.h>
 #include <linux/backing-dev.h>

block/genhd.c +1 -0

index dadf42b..a5012aa9 100644
@@ -2,6 +2,7 @@
  *  gendisk handling
  */
 
+#include <linux/sched.h>
 #include <linux/module.h>
 #include <linux/fs.h>
 #include <linux/genhd.h>

drivers/base/dma-buf.c +1 -0

index 6687ba7..b116b28 100644
@@ -22,6 +22,7 @@
  * this program.  If not, see <http://www.gnu.org/licenses/>.
  */
 
+#include <linux/sched.h>
 #include <linux/fs.h>
 #include <linux/slab.h>
 #include <linux/dma-buf.h>

drivers/block/rsxx/dev.c +1 -0

index d7af441..a033bce 100644
@@ -23,6 +23,7 @@
 */
 
 #include <linux/kernel.h>
+#include <linux/sched.h>
 #include <linux/interrupt.h>
 #include <linux/module.h>
 #include <linux/pci.h>

drivers/dma/ipu/ipu_irq.c +1 -0

index 2e284a4..7f4b2e5 100644
@@ -7,6 +7,7 @@
  * published by the Free Software Foundation.
  */
 
+#include <linux/sched.h>
 #include <linux/init.h>
 #include <linux/err.h>
 #include <linux/spinlock.h>

drivers/gpio/gpiolib.c +1 -0

index ff0fd65..906a80d 100644
@@ -1,3 +1,4 @@
+#include <linux/sched.h>
 #include <linux/kernel.h>
 #include <linux/module.h>
 #include <linux/interrupt.h>

drivers/ide/ide-io.c +1 -0

index 177db6d..9aa758e 100644
@@ -24,6 +24,7 @@
  */
  
  
+#include <linux/sched.h>
 #include <linux/module.h>
 #include <linux/types.h>
 #include <linux/string.h>

drivers/infiniband/hw/amso1100/c2_cq.c +1 -0

index 49e0e85..07f9d3a 100644
@@ -35,6 +35,7 @@
  * SOFTWARE.
  *
  */
+#include <linux/sched.h>
 #include <linux/gfp.h>
 
 #include "c2.h"

drivers/infiniband/hw/cxgb3/iwch_cm.c +1 -0

index 3e094cd..a0fba9c 100644
@@ -29,6 +29,7 @@
  * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
  * SOFTWARE.
  */
+#include <linux/sched.h>
 #include <linux/module.h>
 #include <linux/list.h>
 #include <linux/slab.h>

drivers/infiniband/hw/cxgb4/cm.c +1 -0

index 65c30ea..f0e6a5b 100644
@@ -29,6 +29,7 @@
  * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
  * SOFTWARE.
  */
+#include <linux/sched.h>
 #include <linux/module.h>
 #include <linux/list.h>
 #include <linux/workqueue.h>

drivers/md/dm.c +1 -0

index 9e39d2b..43ee49a 100644
@@ -5,6 +5,7 @@
  * This file is released under the GPL.
  */
 
+#include <linux/sched.h>
 #include "dm.h"
 #include "dm-uevent.h"

drivers/md/raid5.c +1 -0

index 78ea443..fcf17e7 100644
@@ -43,6 +43,7 @@
  * miss any bits.
  */
 
+#include <linux/sched.h>
 #include <linux/blkdev.h>
 #include <linux/kthread.h>
 #include <linux/raid/pq.h>

drivers/mmc/core/core.c +1 -0

index 49a5bca..f1832b0 100644
@@ -10,6 +10,7 @@
  * it under the terms of the GNU General Public License version 2 as
  * published by the Free Software Foundation.
  */
+#include <linux/sched.h>
 #include <linux/module.h>
 #include <linux/init.h>
 #include <linux/interrupt.h>

drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c +1 -0

index e06186c..95eed3f 100644
@@ -17,6 +17,7 @@
 
 #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
 
+#include <linux/sched.h>
 #include <linux/module.h>
 #include <linux/moduleparam.h>
 #include <linux/kernel.h>

drivers/net/ethernet/broadcom/bnx2x/bnx2x_sp.c +1 -0

index 8f03c98..d908719 100644
@@ -19,6 +19,7 @@
 
 #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
 
+#include <linux/sched.h>
 #include <linux/module.h>
 #include <linux/crc32.h>
 #include <linux/netdevice.h>

drivers/net/ethernet/broadcom/bnx2x/bnx2x_sriov.h +1 -0

index d143a7c..01ebe72 100644
@@ -20,6 +20,7 @@
 #define BNX2X_SRIOV_H
 
 #include "bnx2x_vfpf.h"
+#include <linux/sched.h>
 #include "bnx2x.h"
 
 enum sample_bulletin_result {

drivers/net/ethernet/broadcom/bnx2x/bnx2x_stats.c +1 -0

index 98366ab..962e22a 100644
@@ -17,6 +17,7 @@
 
 #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
 
+#include <linux/sched.h>
 #include "bnx2x_stats.h"
 #include "bnx2x_cmn.h"
 #include "bnx2x_sriov.h"

drivers/net/ethernet/intel/e1000e/netdev.c +1 -0

index 77f81cb..36aca7e 100644
@@ -28,6 +28,7 @@
 
 #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
 
+#include <linux/sched.h>
 #include <linux/module.h>
 #include <linux/types.h>
 #include <linux/init.h>

drivers/net/ethernet/intel/igbvf/netdev.c +1 -0

index 93eb7ee..95d5430 100644
@@ -27,6 +27,7 @@
 
 #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
 
+#include <linux/sched.h>
 #include <linux/module.h>
 #include <linux/types.h>
 #include <linux/init.h>

drivers/net/ethernet/sfc/falcon.c +1 -0

index 71998e7..a7a390f 100644
@@ -8,6 +8,7 @@
  * by the Free Software Foundation, incorporated herein by reference.
  */
 
+#include <linux/sched.h>
 #include <linux/bitops.h>
 #include <linux/delay.h>
 #include <linux/pci.h>

drivers/net/ieee802154/at86rf230.c +1 -0

index 6f10b49..59350e3 100644
@@ -21,6 +21,7 @@
  * Alexander Smirnov <[email protected]>
  */
 #include <linux/kernel.h>
+#include <linux/sched.h>
 #include <linux/module.h>
 #include <linux/interrupt.h>
 #include <linux/gpio.h>

drivers/net/ieee802154/fakelb.c +1 -0

index b8d2217..124969c 100644
@@ -23,6 +23,7 @@
  */
 
 #include <linux/module.h>
+#include <linux/sched.h>
 #include <linux/timer.h>
 #include <linux/platform_device.h>
 #include <linux/netdevice.h>

drivers/net/wireless/ath/carl9170/usb.c +1 -0

index 307bc0d..a6bc868 100644
@@ -37,6 +37,7 @@
  *    OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
  */
 
+#include <linux/sched.h>
 #include <linux/module.h>
 #include <linux/slab.h>
 #include <linux/usb.h>

drivers/net/wireless/ath/wil6210/wmi.c +1 -0

index dc8059a..bc61ab3 100644
@@ -15,6 +15,7 @@
  */
 
 #include <linux/etherdevice.h>
+#include <linux/sched.h>
 #include <linux/if_arp.h>
 
 #include "wil6210.h"

drivers/net/wireless/b43/dma.c +1 -0

index f7c70b3..2bdbd63 100644
@@ -27,6 +27,7 @@
 
 */
 
+#include <linux/sched.h>
 #include "b43.h"
 #include "dma.h"
 #include "main.h"

drivers/net/wireless/b43/main.c +1 -0

index 0e933bb..6e49c09 100644
@@ -32,6 +32,7 @@
 
 */
 
+#include <linux/sched.h>
 #include <linux/delay.h>
 #include <linux/init.h>
 #include <linux/module.h>

drivers/net/wireless/b43/phy_a.c +1 -0

index a6c3810..0a7d6b6 100644
@@ -26,6 +26,7 @@
 
 */
 
+#include <linux/sched.h>
 #include <linux/slab.h>
 
 #include "b43.h"

drivers/net/wireless/b43/phy_g.c +1 -0

index 12f467b..01141cd 100644
@@ -26,6 +26,7 @@
 
 */
 
+#include <linux/sched.h>
 #include "b43.h"
 #include "phy_g.h"
 #include "phy_common.h"

drivers/net/wireless/b43legacy/dma.c +1 -0

index faeafe2..f709415 100644
@@ -27,6 +27,7 @@
 
 */
 
+#include <linux/sched.h>
 #include "b43legacy.h"
 #include "dma.h"
 #include "main.h"

drivers/net/wireless/b43legacy/radio.c +1 -0

index 8961776..ff59e7e 100644
@@ -29,6 +29,7 @@
 
 */
 
+#include <linux/sched.h>
 #include <linux/delay.h>
 
 #include "b43legacy.h"

drivers/net/wireless/cw1200/cw1200_spi.c +1 -0

index d063760..c2795d4 100644
@@ -14,6 +14,7 @@
  */
 
 #include <linux/module.h>
+#include <linux/sched.h>
 #include <linux/gpio.h>
 #include <linux/delay.h>
 #include <linux/spinlock.h>

drivers/net/wireless/iwlwifi/dvm/sta.c +1 -0

index c3c13ce..71df3ce 100644
@@ -27,6 +27,7 @@
  *
  *****************************************************************************/
 #include <linux/etherdevice.h>
+#include <linux/sched.h>
 #include <net/mac80211.h>
 #include "iwl-trans.h"
 #include "dev.h"

drivers/net/wireless/iwlwifi/iwl-op-mode.h +1 -0

index 98c7aa7..92695a6 100644
@@ -64,6 +64,7 @@
 #define __iwl_op_mode_h__
 
 #include <linux/debugfs.h>
+#include <linux/sched.h>
 
 struct iwl_op_mode;
 struct iwl_trans;

drivers/net/wireless/iwlwifi/iwl-trans.h +1 -0

index 8d91422c..231949f 100644
@@ -63,6 +63,7 @@
 #ifndef __iwl_trans_h__
 #define __iwl_trans_h__
 
+#include <linux/sched.h>
 #include <linux/ieee80211.h>
 #include <linux/mm.h> /* for page_address */
 #include <linux/lockdep.h>

drivers/net/wireless/libertas_tf/cmd.c +1 -0

index 909ac36..fa69e9f 100644
@@ -9,6 +9,7 @@
  */
 #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
 
+#include <linux/sched.h>
 #include <linux/hardirq.h>
 #include <linux/slab.h>
 #include <linux/export.h>

drivers/pci/iov.c +1 -0

index de8ffac..a581e34 100644
@@ -8,6 +8,7 @@
  *   Address Translation Service 1.0
  */
 
+#include <linux/sched.h>
 #include <linux/pci.h>
 #include <linux/slab.h>
 #include <linux/mutex.h>

drivers/pci/pci.c +1 -0

index e37fea6..6b44f63 100644
@@ -7,6 +7,7 @@
  *	Copyright 1997 -- 2000 Martin Mares <[email protected]>
  */
 
+#include <linux/sched.h>
 #include <linux/kernel.h>
 #include <linux/delay.h>
 #include <linux/init.h>

drivers/platform/olpc/olpc-ec.c +1 -0

index 0f9f859..dd3984a 100644
@@ -6,6 +6,7 @@
  * Licensed under the GPL v2 or later.
  */
 #include <linux/completion.h>
+#include <linux/sched.h>
 #include <linux/debugfs.h>
 #include <linux/spinlock.h>
 #include <linux/mutex.h>

drivers/ssb/driver_pcicore.c +1 -0

index d75b72b..c343ee7 100644
@@ -8,6 +8,7 @@
  * Licensed under the GNU/GPL. See COPYING for details.
  */
 
+#include <linux/sched.h>
 #include <linux/ssb/ssb.h>
 #include <linux/pci.h>
 #include <linux/export.h>

drivers/staging/lustre/lustre/llite/remote_perm.c +1 -0

index 68b2dc4..ceac936 100644
@@ -44,6 +44,7 @@
 #define DEBUG_SUBSYSTEM S_LLITE
 
 #include <linux/module.h>
+#include <linux/sched.h>
 #include <linux/types.h>
 #include <linux/version.h>

drivers/staging/lustre/lustre/obdclass/cl_lock.c +1 -0

index d34e044..3fb55ff 100644
@@ -41,6 +41,7 @@
 #define DEBUG_SUBSYSTEM S_CLASS
 
 #include <obd_class.h>
+#include <linux/sched.h>
 #include <obd_support.h>
 #include <lustre_fid.h>
 #include <linux/list.h>

drivers/staging/lustre/lustre/obdclass/cl_object.c +1 -0

index cdb5fba..72a1ac4 100644
@@ -52,6 +52,7 @@
 #define DEBUG_SUBSYSTEM S_CLASS
 
 #include <linux/libcfs/libcfs.h>
+#include <linux/sched.h>
 /* class_put_type() */
 #include <obd_class.h>
 #include <obd_support.h>

drivers/staging/lustre/lustre/obdclass/cl_page.c +1 -0

index bb93359..f5189f8 100644
@@ -41,6 +41,7 @@
 #define DEBUG_SUBSYSTEM S_CLASS
 
 #include <linux/libcfs/libcfs.h>
+#include <linux/sched.h>
 #include <obd_class.h>
 #include <obd_support.h>
 #include <linux/list.h>

drivers/staging/lustre/lustre/osc/osc_lock.c +1 -0

index 640bc3d..d38347b 100644
@@ -43,6 +43,7 @@
 # include <linux/libcfs/libcfs.h>
 /* fid_build_reg_res_name() */
 #include <lustre_fid.h>
+#include <linux/sched.h>
 
 #include "osc_cl_internal.h"

drivers/staging/lustre/lustre/osc/osc_page.c +1 -0

index baba959..ad2c6c7 100644
@@ -41,6 +41,7 @@
 #define DEBUG_SUBSYSTEM S_OSC
 
 #include "osc_cl_internal.h"
+#include <linux/sched.h>
 
 static void osc_lru_del(struct client_obd *cli, struct osc_page *opg, bool del);
 static void osc_lru_add(struct client_obd *cli, struct osc_page *opg);

drivers/staging/lustre/lustre/ptlrpc/client.c +1 -0

index 22f7e65..6791b65 100644
@@ -39,6 +39,7 @@
 #define DEBUG_SUBSYSTEM S_RPC
 
 #include <obd_support.h>
+#include <linux/sched.h>
 #include <obd_class.h>
 #include <lustre_lib.h>
 #include <lustre_ha.h>

drivers/staging/lustre/lustre/ptlrpc/gss/gss_cli_upcall.c +1 -0

index 142c789..b2b893c 100644
@@ -40,6 +40,7 @@
 
 #define DEBUG_SUBSYSTEM S_SEC
 #include <linux/init.h>
+#include <linux/sched.h>
 #include <linux/module.h>
 #include <linux/slab.h>
 #include <linux/dcache.h>

drivers/staging/lustre/lustre/ptlrpc/gss/gss_pipefs.c +1 -0

index 3df7257..9dddaa8 100644
@@ -48,6 +48,7 @@
 
 #define DEBUG_SUBSYSTEM S_SEC
 #include <linux/init.h>
+#include <linux/sched.h>
 #include <linux/module.h>
 #include <linux/slab.h>
 #include <linux/dcache.h>

drivers/staging/lustre/lustre/ptlrpc/sec.c +1 -0

index 36e8bed..28dc4e6 100644
@@ -41,6 +41,7 @@
 #define DEBUG_SUBSYSTEM S_SEC
 
 #include <linux/libcfs/libcfs.h>
+#include <linux/sched.h>
 #include <linux/crypto.h>
 #include <linux/key.h>

drivers/staging/lustre/lustre/ptlrpc/sec_config.c +1 -0

index a45a392..c4b206c 100644
@@ -37,6 +37,7 @@
 #define DEBUG_SUBSYSTEM S_SEC
 
 #include <linux/libcfs/libcfs.h>
+#include <linux/sched.h>
 #include <linux/crypto.h>
 #include <linux/key.h>

drivers/staging/lustre/lustre/ptlrpc/sec_gc.c +1 -0

index 4c96a14a..b512cbc 100644
@@ -41,6 +41,7 @@
 #define DEBUG_SUBSYSTEM S_SEC
 
 #include <linux/libcfs/libcfs.h>
+#include <linux/sched.h>
 
 #include <obd_support.h>
 #include <obd_class.h>

drivers/usb/core/hcd.c +1 -0

index 014dc99..ae84867 100644
@@ -23,6 +23,7 @@
  */
 
 #include <linux/bcd.h>
+#include <linux/sched.h>
 #include <linux/module.h>
 #include <linux/version.h>
 #include <linux/kernel.h>

drivers/usb/core/urb.c +1 -0

index 16927fa..f033bcd0 100644
@@ -1,3 +1,4 @@
+#include <linux/sched.h>
 #include <linux/module.h>
 #include <linux/string.h>
 #include <linux/bitops.h>

drivers/video/atmel_lcdfb.c +1 -0

index effdb37..915836b 100644
@@ -8,6 +8,7 @@
  * more details.
  */
 
+#include <linux/sched.h>
 #include <linux/kernel.h>
 #include <linux/platform_device.h>
 #include <linux/dma-mapping.h>

fs/block_dev.c +1 -0

index c7bda5c..00f49af 100644
@@ -5,6 +5,7 @@
  *  Copyright (C) 2001  Andrea Arcangeli <[email protected]> SuSE
  */
 
+#include <linux/sched.h>
 #include <linux/init.h>
 #include <linux/mm.h>
 #include <linux/fcntl.h>

fs/buffer.c +1 -0

index 4d74335..050c03b 100644
@@ -18,6 +18,7 @@
  * async buffer flushing, 1999 Andrea Arcangeli <[email protected]>
  */
 
+#include <linux/sched.h>
 #include <linux/kernel.h>
 #include <linux/syscalls.h>
 #include <linux/fs.h>

fs/dcache.c +1 -0

index 87bdb53..ab41273 100644
@@ -14,6 +14,7 @@
  * the dcache entry is deleted or garbage collected.
  */
 
+#include <linux/sched.h>
 #include <linux/syscalls.h>
 #include <linux/string.h>
 #include <linux/mm.h>

fs/ext3/inode.c +1 -0

index 2bd8548..8caf634 100644
@@ -22,6 +22,7 @@
  *  Assorted race fixes, rewrite of ext3_get_block() by Al Viro, 2000
  */
 
+#include <linux/sched.h>
 #include <linux/highuid.h>
 #include <linux/quotaops.h>
 #include <linux/writeback.h>

fs/ext4/ext4_jbd2.c +1 -0

index 72a3600..c94cb65 100644
@@ -2,6 +2,7 @@
  * Interface between ext4 and JBD
  */
 
+#include <linux/sched.h>
 #include "ext4_jbd2.h"
 
 #include <trace/events/ext4.h>

fs/ext4/inode.c +1 -0

index dd32a2e..d12f990 100644
@@ -18,6 +18,7 @@
  *  Assorted race fixes, rewrite of ext4_get_block() by Al Viro, 2000
  */
 
+#include <linux/sched.h>
 #include <linux/fs.h>
 #include <linux/time.h>
 #include <linux/jbd2.h>

fs/ext4/mballoc.c +1 -0

index 4bbbf13b..108e7f6 100644
@@ -22,6 +22,7 @@
  */
 
 #include "ext4_jbd2.h"
+#include <linux/sched.h>
 #include "mballoc.h"
 #include <linux/log2.h>
 #include <linux/module.h>

fs/file_table.c +1 -0

index b44e4c5..b35edc4 100644
@@ -5,6 +5,7 @@
  *  Copyright (C) 1997 David S. Miller ([email protected])
  */
 
+#include <linux/sched.h>
 #include <linux/string.h>
 #include <linux/slab.h>
 #include <linux/file.h>

fs/inode.c +1 -0

index d6dfb09..ef3a12d 100644
@@ -2,6 +2,7 @@
  * (C) 1997 Linus Torvalds
  * (C) 1999 Andrea Arcangeli <[email protected]> (dynamic inode allocation)
  */
+#include <linux/sched.h>
 #include <linux/export.h>
 #include <linux/fs.h>
 #include <linux/mm.h>

fs/jbd/revoke.c +1 -0

index 25c713e..4642e55 100644
@@ -81,6 +81,7 @@
  */
 
 #ifndef __KERNEL__
+#include <linux/sched.h>
 #include "jfs_user.h"
 #else
 #include <linux/time.h>

fs/jbd2/revoke.c +1 -0

index 198c9c1..b406894 100644
@@ -81,6 +81,7 @@
  */
 
 #ifndef __KERNEL__
+#include <linux/sched.h>
 #include "jfs_user.h"
 #else
 #include <linux/time.h>

fs/locks.c +1 -0

index b27a300..b8a6709 100644
@@ -114,6 +114,7 @@
  *  Stephen Rothwell <[email protected]>, June, 2000.
  */
 
+#include <linux/sched.h>
 #include <linux/capability.h>
 #include <linux/file.h>
 #include <linux/fdtable.h>

fs/nfs/nfs4filelayoutdev.c +1 -0

index 95604f6..c26be3b 100644
@@ -29,6 +29,7 @@
  */
 
 #include <linux/nfs_fs.h>
+#include <linux/sched.h>
 #include <linux/vmalloc.h>
 #include <linux/module.h>
 #include <linux/sunrpc/addr.h>

fs/nfs/nfs4proc.c +1 -0

index cf11799..90319a3 100644
@@ -35,6 +35,7 @@
  *  SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  */
 
+#include <linux/sched.h>
 #include <linux/mm.h>
 #include <linux/delay.h>
 #include <linux/errno.h>

fs/nfs/nfs4state.c +1 -0

index e22862f..c9acdbc 100644
@@ -39,6 +39,7 @@
  */
 
 #include <linux/kernel.h>
+#include <linux/sched.h>
 #include <linux/slab.h>
 #include <linux/fs.h>
 #include <linux/nfs_fs.h>

fs/nilfs2/the_nilfs.c +1 -0

index 94c451c..462be53 100644
@@ -21,6 +21,7 @@
  *
  */
 
+#include <linux/sched.h>
 #include <linux/buffer_head.h>
 #include <linux/slab.h>
 #include <linux/blkdev.h>

fs/xfs/xfs_mount.c +1 -0

index 2b0ba35..eb9ba15 100644
@@ -15,6 +15,7 @@
  * along with this program; if not, write the Free Software Foundation,
  * Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
  */
+#include <linux/sched.h>
 #include "xfs.h"
 #include "xfs_fs.h"
 #include "xfs_types.h"

include/asm-generic/gpio.h +1 -0

index bde6469..17114e0 100644
@@ -2,6 +2,7 @@
 #define _ASM_GENERIC_GPIO_H
 
 #include <linux/kernel.h>
+#include <linux/sched.h>
 #include <linux/types.h>
 #include <linux/errno.h>
 #include <linux/of.h>

include/linux/buffer_head.h +1 -0

index 91fa9a9..432e212 100644
@@ -8,6 +8,7 @@
 #define _LINUX_BUFFER_HEAD_H
 
 #include <linux/types.h>
+#include <linux/sched.h>
 #include <linux/fs.h>
 #include <linux/linkage.h>
 #include <linux/pagemap.h>

include/linux/clk.h +1 -0

index 9a6d045..8b440a2 100644
@@ -14,6 +14,7 @@
 
 #include <linux/err.h>
 #include <linux/kernel.h>
+#include <linux/sched.h>
 #include <linux/notifier.h>
 
 struct device;

include/linux/gpio.h +1 -0

index 552e3f4..be83d05 100644
@@ -2,6 +2,7 @@
 #define __LINUX_GPIO_H
 
 #include <linux/errno.h>
+#include <linux/sched.h>
 
 /* see Documentation/gpio.txt */

include/linux/highmem.h +1 -0

index 7fb31da..435de87 100644
@@ -2,6 +2,7 @@
 #define _LINUX_HIGHMEM_H
 
 #include <linux/fs.h>
+#include <linux/sched.h>
 #include <linux/kernel.h>
 #include <linux/bug.h>
 #include <linux/mm.h>

include/linux/pagemap.h +1 -0

index e3dea75..a409785 100644
@@ -5,6 +5,7 @@
  * Copyright 1995 Linus Torvalds
  */
 #include <linux/mm.h>
+#include <linux/sched.h>
 #include <linux/fs.h>
 #include <linux/list.h>
 #include <linux/highmem.h>

kernel/fork.c +1 -0

index 403d2bb..f12e417 100644
@@ -11,6 +11,7 @@
  * management can be a bitch. See 'mm/memory.c': 'copy_page_range()'
  */
 
+#include <linux/sched.h>
 #include <linux/slab.h>
 #include <linux/init.h>
 #include <linux/unistd.h>

kernel/freezer.c +1 -0

index b462fa1..1d975d1 100644
@@ -4,6 +4,7 @@
  * Originally from kernel/power/process.c
  */
 
+#include <linux/sched.h>
 #include <linux/interrupt.h>
 #include <linux/suspend.h>
 #include <linux/export.h>

kernel/irq/chip.c +1 -0

index a3bb14f..9948f28 100644
@@ -10,6 +10,7 @@
  * Detailed information is available in Documentation/DocBook/genericirq
  */
 
+#include <linux/sched.h>
 #include <linux/irq.h>
 #include <linux/msi.h>
 #include <linux/module.h>

kernel/nsproxy.c +1 -0

index 364ceab..d018644 100644
@@ -13,6 +13,7 @@
  *             Pavel Emelianov <[email protected]>
  */
 
+#include <linux/sched.h>
 #include <linux/slab.h>
 #include <linux/export.h>
 #include <linux/nsproxy.h>

kernel/printk/printk.c +1 -0

index 5b5a708..c97c144 100644
@@ -17,6 +17,7 @@
  */
 
 #include <linux/kernel.h>
+#include <linux/sched.h>
 #include <linux/mm.h>
 #include <linux/tty.h>
 #include <linux/tty_driver.h>

kernel/sched/core.c +1 -0

index b7c32cb..74d7c04 100644
@@ -26,6 +26,7 @@
  *              Thomas Gleixner, Mike Kravetz
  */
 
+#include <linux/sched.h>
 #include <linux/mm.h>
 #include <linux/module.h>
 #include <linux/nmi.h>

kernel/smp.c +1 -0

index fe9f773..5b58226 100644
@@ -9,6 +9,7 @@
 #include <linux/export.h>
 #include <linux/percpu.h>
 #include <linux/init.h>
+#include <linux/sched.h>
 #include <linux/gfp.h>
 #include <linux/smp.h>
 #include <linux/cpu.h>

mm/hugetlb.c +1 -0

index 83aff0a..e342674 100644
@@ -2,6 +2,7 @@
  * Generic hugetlb support.
  * (C) Nadia Yvette Chambers, April 2004
  */
+#include <linux/sched.h>
 #include <linux/list.h>
 #include <linux/init.h>
 #include <linux/module.h>

mm/memory.c +1 -0

index 1ce2e2a..61add89 100644
@@ -38,6 +38,7 @@
  * Aug/Sep 2004 Changed to four level page tables (Andi Kleen)
  */
 
+#include <linux/sched.h>
 #include <linux/kernel_stat.h>
 #include <linux/mm.h>
 #include <linux/hugetlb.h>

mm/mmap.c +1 -0

index 1edbaa3..1fda64c 100644
@@ -7,6 +7,7 @@
  */
 
 #include <linux/kernel.h>
+#include <linux/sched.h>
 #include <linux/slab.h>
 #include <linux/backing-dev.h>
 #include <linux/mm.h>

mm/rmap.c +1 -0

index cd356df..ca1c188 100644
@@ -42,6 +42,7 @@
  *     pte map lock
  */
 
+#include <linux/sched.h>
 #include <linux/mm.h>
 #include <linux/pagemap.h>
 #include <linux/swap.h>

net/caif/cfcnfg.c +1 -0

index fa39fc2..2809fd5 100644
@@ -6,6 +6,7 @@
 
 #define pr_fmt(fmt) KBUILD_MODNAME ":%s(): " fmt, __func__
 
+#include <linux/sched.h>
 #include <linux/kernel.h>
 #include <linux/stddef.h>
 #include <linux/slab.h>

net/mac80211/driver-ops.h +1 -0

index b931c96..f6de52c 100644
@@ -1,6 +1,7 @@
 #ifndef __MAC80211_DRIVER_OPS
 #define __MAC80211_DRIVER_OPS
 
+#include <linux/sched.h>
 #include <net/mac80211.h>
 #include "ieee80211_i.h"
 #include "trace.h"

net/mac80211/key.c +1 -0

index e39cc91..9ae77d9 100644
@@ -9,6 +9,7 @@
  * published by the Free Software Foundation.
  */
 
+#include <linux/sched.h>
 #include <linux/if_ether.h>
 #include <linux/etherdevice.h>
 #include <linux/list.h>

net/mac80211/main.c +1 -0

index 091088a..53db5ee 100644
@@ -8,6 +8,7 @@
  * published by the Free Software Foundation.
  */
 
+#include <linux/sched.h>
 #include <net/mac80211.h>
 #include <linux/module.h>
 #include <linux/init.h>

net/mac80211/sta_info.c +1 -0

index aeb967a..5a5ae34 100644
@@ -7,6 +7,7 @@
  * published by the Free Software Foundation.
  */
 
+#include <linux/sched.h>
 #include <linux/module.h>
 #include <linux/init.h>
 #include <linux/etherdevice.h>

net/phonet/pep.c +1 -0

index e774117..76bb2dc 100644
@@ -22,6 +22,7 @@
  * 02110-1301 USA
  */
 
+#include <linux/sched.h>
 #include <linux/kernel.h>
 #include <linux/slab.h>
 #include <linux/socket.h>

net/sunrpc/clnt.c +1 -0

index 74f6a70..6efea48 100644
@@ -19,6 +19,7 @@
 
 
 #include <linux/module.h>
+#include <linux/sched.h>
 #include <linux/types.h>
 #include <linux/kallsyms.h>
 #include <linux/mm.h>

net/wimax/op-msg.c +1 -0

index 0694d62..2d48d26 100644
@@ -71,6 +71,7 @@
  *   wimax_msg_alloc()
  *   wimax_msg_send()
  */
+#include <linux/sched.h>
 #include <linux/device.h>
 #include <linux/slab.h>
 #include <net/genetlink.h>

net/wimax/op-reset.c +1 -0

index 7ceffe3..30af0ee 100644
@@ -28,6 +28,7 @@
  * disconnect and reconnect the device).
  */
 
+#include <linux/sched.h>
 #include <net/wimax.h>
 #include <net/genetlink.h>
 #include <linux/wimax.h>

net/wimax/op-rfkill.c +1 -0

index 7ab60ba..6efe0bd 100644
@@ -60,6 +60,7 @@
  * wimax_rfkill_rm()            [called by wimax_dev_add/rm()]
  */
 
+#include <linux/sched.h>
 #include <net/wimax.h>
 #include <net/genetlink.h>
 #include <linux/wimax.h>

net/wireless/wext-proc.c +1 -0

index e98a01c..e771c39 100644
@@ -16,6 +16,7 @@
  * The content of the file is basically the content of "struct iw_statistics".
  */
 
+#include <linux/sched.h>
 #include <linux/module.h>
 #include <linux/proc_fs.h>
 #include <linux/seq_file.h>

sound/core/info.c +1 -0

index e79baa1..719a8ef 100644
@@ -19,6 +19,7 @@
  *
  */
 
+#include <linux/sched.h>
 #include <linux/init.h>
 #include <linux/time.h>
 #include <linux/mm.h>

virt/kvm/async_pf.c +1 -0

index ea475cd..b44cea0 100644
@@ -20,6 +20,7 @@
  * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301, USA.
  */
 
+#include <linux/sched.h>
 #include <linux/kvm_host.h>
 #include <linux/slab.h>
 #include <linux/module.h>

Signature · 1 line

1.8.3.1

Geert Uytterhoeven

2013-08-31 18:22 UTC #5 lore raw

Re: [PATCH 3/6] tree-sweep: Include linux/sched.h for might_sleep users

Quoted 2 lines · Andi Kleen

might_sleep is moving from linux/kernel.h to linux/sched.h, so any users need to include linux/sched.h

Really? <linux/sched.h> is the worst choice w.r.t. include hell.

Quoted 14 lines

This was done with a mechanistic script and some uses may be redundant (already included in some other include file). However it's good practice to always include any needed symbols from the top level .c file.
Tested with x86-64 allyesconfig. I used to do a x86-32 allyesconfig on a old kernel, but since that is broken now I didn't retest.
Signed-off-by Andi Kleen <[email protected]>
arch/mn10300/include/asm/uaccess.h +0 -0
index 5372787..274c9c2 100644

arch/mn10300/include/asm/uaccess.h line 20

Quoted 7 lines

arch/mn10300/include/asm/uaccess.h +1 -0

@@ -14,6 +14,7 @@
 /*
  * User space memory access functions
  */
+#include <linux/sched.h>
 #include <linux/thread_info.h>
 #include <linux/kernel.h>
 #include <asm/page.h>

This part breaks mn10300:

CC init/version.o
In file included from linux-next/include/linux/timex.h:56:0,

                 from linux-next/include/linux/sched.h:17,
                 from linux-next/arch/mn10300/include/asm/uaccess.h:17,
                 from linux-next/arch/mn10300/include/asm/processor.h:21,
                 from linux-next/include/linux/spinlock_up.h:8,
                 from linux-next/include/linux/spinlock.h:89,
                 from linux-next/include/linux/seqlock.h:29,
                 from linux-next/include/linux/time.h:5,
                 from linux-next/include/linux/stat.h:18,
                 from linux-next/include/linux/module.h:10,
                 from linux-next/init/version.c:10:

linux-next/include/uapi/linux/timex.h:76:17: error: field 'time' has incomplete type

http://kisskb.ellerman.id.au/kisskb/buildresult/9424440/
http://kisskb.ellerman.id.au/kisskb/buildresult/9424438/

Note that it doesn't use might_sleep(). It does use might_fault(), but only from a macro, not from a C inline function.

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

Andrew Morton

maintainer

2013-09-10 23:52 UTC #6 lore raw

Re: [PATCH 3/6] tree-sweep: Include linux/sched.h for might_sleep users

Quoted 5 lines · On Sat, 31 Aug 2013 20:22:23 +0200 Geert Uytterhoeven <[email protected]>…

Quoted 2 lines · Andi Kleen
might_sleep is moving from linux/kernel.h to linux/sched.h, so any users need to include linux/sched.h
Really? <linux/sched.h> is the worst choice w.r.t. include hell.

Yes. Mechanically adding sched.h inclusions into .h files will cause problems.

I've had decent success with this sort of thing by adding new, finer-grained headers. Maybe include/linux/might_sleep.h would be a good starting point to fix whatever it was that this fixed.

I think I'll make x86-include-linux-schedh-in-asm-uaccessh.patch, tree-sweep-include-linux-schedh-for-might_sleep-users.patch and sched-mark-should_resched-__always_inline.patch go away for now - I need a mergeable tree, fast.

Andi Kleen

author

2013-09-11 04:51 UTC #7 lore raw

Re: [PATCH 3/6] tree-sweep: Include linux/sched.h for might_sleep users

Quoted 6 lines · Andrew Morton

Quoted 4 lines · On Sat, 31 Aug 2013 20:22:23 +0200 Geert Uytterhoeven <[email protected]>…
Quoted 2 lines · Andi Kleen
might_sleep is moving from linux/kernel.h to linux/sched.h, so any users need to include linux/sched.h

Quoted 8 lines

Quoted 1 line
Really? <linux/sched.h> is the worst choice w.r.t. include hell.
Yes. Mechanically adding sched.h inclusions into .h files will cause problems.
I've had decent success with this sort of thing by adding new, finer-grained headers. Maybe include/linux/might_sleep.h would be a good starting point to fix whatever it was that this fixed.

Ok. I'll look into that and send new patches.

Some of it may be obsolete with Peter's recent preempt work.

Quoted 4 lines

I think I'll make x86-include-linux-schedh-in-asm-uaccessh.patch, tree-sweep-include-linux-schedh-for-might_sleep-users.patch and sched-mark-should_resched-__always_inline.patch go away for now - I need a mergeable tree, fast.

Fair enough.

-Andi

Signature · 1 line

[email protected] -- Speaking for myself only.

Ingo Molnar

2013-09-11 05:36 UTC #8 lore raw

Re: [PATCH 3/6] tree-sweep: Include linux/sched.h for might_sleep users

Quoted 12 lines · Andi Kleen

Quoted 11 lines · Andrew Morton
Quoted 5 lines · On Sat, 31 Aug 2013 20:22:23 +0200 Geert Uytterhoeven <[email protected]>…
Quoted 2 lines · Andi Kleen
might_sleep is moving from linux/kernel.h to linux/sched.h, so any users need to include linux/sched.h
Really? <linux/sched.h> is the worst choice w.r.t. include hell.
Yes. Mechanically adding sched.h inclusions into .h files will cause problems.

Quoted 7 lines

Quoted 3 lines
I've had decent success with this sort of thing by adding new, finer-grained headers. Maybe include/linux/might_sleep.h would be a good starting point to fix whatever it was that this fixed.
Ok. I'll look into that and send new patches.
Some of it may be obsolete with Peter's recent preempt work.

The best order from a maintenance and merge POV would be to rebase your bits on top of Peter's preempt_count optimization work, which is the more comprehensive approach. That allows you to drop the obsolete bits, and wait for Peter to pick up your bits.

The two interact so we want to handle them together in any case, in the scheduler tree.

Thanks,

Ingo

Andi Kleen

author

2013-08-16 22:42 UTC #9 lore raw

PATCH 4/6 Move might_sleep and friends from kernel.h to sched.h

In reply to Andi Kleen

From: Andi Kleen <[email protected]>

These are really related to scheduling, so they should be in sched.h Users usually will need to schedule anyways.

The advantage of having them there is that we can access some of the scheduler inlines to make their fast path more efficient. This will come in a followon patch.

Signed-off-by Andi Kleen <[email protected]>

---
 include/linux/kernel.h | 35 -----------------------------------
 include/linux/sched.h  | 37 +++++++++++++++++++++++++++++++++++++
 2 files changed, 37 insertions(+), 35 deletions(-)

include/linux/kernel.h +0 ~~-35~~

index 482ad2d..badcc13 100644
@@ -141,35 +141,6 @@ struct completion;
 struct pt_regs;
 struct user;
 
-#ifdef CONFIG_PREEMPT_VOLUNTARY
-extern int _cond_resched(void);
-# define might_resched() _cond_resched()
-#else
-# define might_resched() do { } while (0)
-#endif
-
-#ifdef CONFIG_DEBUG_ATOMIC_SLEEP
-  void __might_sleep(const char *file, int line, int preempt_offset);
-/**
- * might_sleep - annotation for functions that can sleep
- *
- * this macro will print a stack trace if it is executed in an atomic
- * context (spinlock, irq-handler, ...).
- *
- * This is a useful debugging help to be able to catch problems early and not
- * be bitten later when the calling function happens to sleep when it is not
- * supposed to.
- */
-# define might_sleep() \
-	do { __might_sleep(__FILE__, __LINE__, 0); might_resched(); } while (0)
-#else
-  static inline void __might_sleep(const char *file, int line,
-				   int preempt_offset) { }
-# define might_sleep() do { might_resched(); } while (0)
-#endif
-
-#define might_sleep_if(cond) do { if (cond) might_sleep(); } while (0)
-
 /*
  * abs() handles unsigned and signed longs, ints, shorts and chars.  For all
  * input types abs() returns a signed long.

@@ -193,12 +164,6 @@ extern int _cond_resched(void);
 		(__x < 0) ? -__x : __x;		\
 	})
 
-#if defined(CONFIG_PROVE_LOCKING) || defined(CONFIG_DEBUG_ATOMIC_SLEEP)
-void might_fault(void);
-#else
-static inline void might_fault(void) { }
-#endif
-
 extern struct atomic_notifier_head panic_notifier_list;
 extern long (*panic_blink)(int state);
 __printf(1, 2)

include/linux/sched.h +37 -0

index d722490..97f4b78 100644
@@ -2433,6 +2433,35 @@ extern int __cond_resched_softirq(void);
 	__cond_resched_softirq();					\
 })
 
+#ifdef CONFIG_PREEMPT_VOLUNTARY
+extern int _cond_resched(void);
+# define might_resched() _cond_resched()
+#else
+# define might_resched() do { } while (0)
+#endif
+
+#ifdef CONFIG_DEBUG_ATOMIC_SLEEP
+ void __might_sleep(const char *file, int line, int preempt_offset);
+/**
+ * might_sleep - annotation for functions that can sleep
+ *
+ * this macro will print a stack trace if it is executed in an atomic
+ * context (spinlock, irq-handler, ...).
+ *
+ * This is a useful debugging help to be able to catch problems early and not
+ * be bitten later when the calling function happens to sleep when it is not
+ * supposed to.
+ */
+# define might_sleep() \
+	do { __might_sleep(__FILE__, __LINE__, 0); might_resched(); } while (0)
+#else
+  static inline void __might_sleep(const char *file, int line,
+				   int preempt_offset) { }
+# define might_sleep() do { might_resched(); } while (0)
+#endif
+
+#define might_sleep_if(cond) do { if (cond) might_sleep(); } while (0)
+
 static inline void cond_resched_rcu(void)
 {
 #if defined(CONFIG_DEBUG_ATOMIC_SLEEP) || !defined(CONFIG_PREEMPT_RCU)

@@ -2442,6 +2471,14 @@ static inline void cond_resched_rcu(void)
 #endif
 }
 
+#ifdef CONFIG_PROVE_LOCKING
+void might_fault(void);
+#else
+static inline void might_fault(void)
+{
+}
+#endif
+
 /*
  * Does a critical section need to be broken due to another
  * task waiting?: (technically does not depend on CONFIG_PREEMPT,

Signature · 1 line

1.8.3.1

Andrew Morton

maintainer

2013-08-27 23:50 UTC #10 lore raw

Re: [PATCH 4/6] Move might_sleep and friends from kernel.h to sched.h

Quoted 6 lines · On Fri, 16 Aug 2013 14:17:22 -0700 Andi Kleen <[email protected]> wrote:

These are really related to scheduling, so they should be in sched.h Users usually will need to schedule anyways.
The advantage of having them there is that we can access some of the scheduler inlines to make their fast path more efficient. This will come in a followon patch.

alpha allmodconfig:

mm/memory.c:4236: error: redefinition of 'might_fault' include/linux/sched.h:2485: note: previous definition of 'might_fault' was here

Presumably because CONFIG_PROVE_LOCKING=n && CONFIG_DEBUG_ATOMIC_SLEEP=y.

That code is too tangly for me to fix it up with any confidence, sorry.

Andi Kleen

author

2013-08-16 22:48 UTC #11 lore raw

PATCH 5/6 sched: mark should_resched() __always_inline

In reply to Andi Kleen

From: Andi Kleen <[email protected]>

At least gcc 4.6 and some earlier ones does not inline this function. Since it's small and on relatively hot paths force inline it.

Signed-off-by Andi Kleen <[email protected]>

---
 kernel/sched/core.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

kernel/sched/core.c +1 -1

index 74d7c04..23df96a 100644
@@ -3767,7 +3767,7 @@ SYSCALL_DEFINE0(sched_yield)
 	return 0;
 }
 
-static inline int should_resched(void)
+static __always_inline int should_resched(void)
 {
 	return need_resched() && !(preempt_count() & PREEMPT_ACTIVE);
 }

Signature · 1 line

1.8.3.1

Andi Kleen

author

2013-08-16 22:48 UTC #12 lore raw

PATCH 6/6 sched: Inline the need_resched test into the caller for _cond_resched

In reply to Andi Kleen

From: Andi Kleen <[email protected]>

_cond_resched does at least two explicit calls just to decide to do
nothing: _cond_resched and should_resched(). Inline a need_resched() into the caller to avoid these calls in the common case of no reschedule being needed.

Signed-off-by Andi Kleen <[email protected]>

---
 include/linux/sched.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

include/linux/sched.h +1 -1

index 97f4b78..dd3fafc 100644
@@ -2435,7 +2435,7 @@ extern int __cond_resched_softirq(void);
 
 #ifdef CONFIG_PREEMPT_VOLUNTARY
 extern int _cond_resched(void);
-# define might_resched() _cond_resched()
+# define might_resched() (need_resched() ? _cond_resched() : 0)
 #else
 # define might_resched() do { } while (0)
 #endif

Signature · 1 line

1.8.3.1

Ingo Molnar

2013-08-28 09:29 UTC #13 lore raw

In reply to Andi Kleen

Quoted 2 lines · Andi Kleen

Various optimizations related to CONFIG_PREEMPT_VOLUNTARY and x86 uaccess

Looks mostly good.

Does this patchset change the number of cond_resched() preemption points on CONFIG_PREEMPT_VOLUNTARY=y, or is it a scheduling invariant?

Thanks,

Ingo