is TIF_NOTIFY_RESUME used?

PATCH In review Stephane Eranian 6 participants 19 messages May 22, 2007 – May 25, 2007

Subsystems (10)

Arm (32-bit) PowerPC MIPS s390 +6 more subsystems

lore

Share & tools

lore

Stephane Eranian

author

2007-05-22 12:47 UTC #0 lore raw

Hello,

For perfmon, we need a couple of TIF bits. It seems that with 2.6.22-rc2 there is now a TIF_RESTORE_SIGMASK which uses the last remaining bit in the first 7 bits of the thread flag. Many architectures, including IA-64, rely on the fact that some of the TIF flags (TIF_ALL_WORKMASK or TIF_ALL_WORK) tested on kernel exit reside in the low 8-bit or 7-bit because they use instructions (such as add r1=imm8,r2 on IA-64) which operate on 8 or 7 bit immediate.

On IA-64, adding that one perfmon flag (as bit 7) would cause some restructuring in the kernel exit path but also in all the lightweight syscall handlers.

I looked at all the low order TIF flags and found that TIF_NOTIFY_RESUME was never set nor used anywhere in any architecture. Is that really the case?

If so, we could get rid of it and free up one low-order TIF bit.

Thanks.

Signature · 1 line

-Stephane

Andrew Morton

2007-05-22 16:02 UTC #1 lore raw

Quoted 10 lines · On Tue, 22 May 2007 05:47:13 -0700 Stephane Eranian <[email protected]> wrote:

Hello,
For perfmon, we need a couple of TIF bits. It seems that with 2.6.22-rc2 there is now a TIF_RESTORE_SIGMASK which uses the last remaining bit in the first 7 bits of the thread flag. Many architectures, including IA-64, rely on the fact that some of the TIF flags (TIF_ALL_WORKMASK or TIF_ALL_WORK) tested on kernel exit reside in the low 8-bit or 7-bit because they use instructions (such as add r1=imm8,r2 on IA-64) which operate on 8 or 7 bit immediate.

Quoted 9 lines

On IA-64, adding that one perfmon flag (as bit 7) would cause some restructuring in the kernel exit path but also in all the lightweight syscall handlers.
I looked at all the low order TIF flags and found that TIF_NOTIFY_RESUME was never set nor used anywhere in any architecture. Is that really the case?
If so, we could get rid of it and free up one low-order TIF bit.

My grepping argees with yours. The only place where TIF_NOTIFY_RESUME gets altered is in ./arch/ia64/kernel/perfmon.c.

Stephane Eranian

author

2007-05-22 16:07 UTC #2 lore raw

Andrew,

Quoted 17 lines · Andrew Morton

Quoted 14 lines · On Tue, 22 May 2007 05:47:13 -0700 Stephane Eranian <[email protected]> wrote:
Hello,
For perfmon, we need a couple of TIF bits. It seems that with 2.6.22-rc2 there is now a TIF_RESTORE_SIGMASK which uses the last remaining bit in the first 7 bits of the thread flag. Many architectures, including IA-64, rely on the fact that some of the TIF flags (TIF_ALL_WORKMASK or TIF_ALL_WORK) tested on kernel exit reside in the low 8-bit or 7-bit because they use instructions (such as add r1=imm8,r2 on IA-64) which operate on 8 or 7 bit immediate.
On IA-64, adding that one perfmon flag (as bit 7) would cause some restructuring in the kernel exit path but also in all the lightweight syscall handlers.

Quoted 8 lines

Quoted 5 lines
I looked at all the low order TIF flags and found that TIF_NOTIFY_RESUME was never set nor used anywhere in any architecture. Is that really the case?
If so, we could get rid of it and free up one low-order TIF bit.
My grepping argees with yours. The only place where TIF_NOTIFY_RESUME gets altered is in ./arch/ia64/kernel/perfmon.c.

Yes, and that is with the old IA-64 code. In the new one I used a dedicated TIF flag.

Shall we just get rid of the flag, then?

Signature · 1 line

-Stephane

Andrew Morton

2007-05-22 16:15 UTC #3 lore raw

Quoted 26 lines · On Tue, 22 May 2007 09:07:37 -0700 Stephane Eranian <[email protected]> wrote:

Andrew,
Quoted 23 lines · Andrew Morton
Quoted 19 lines · On Tue, 22 May 2007 05:47:13 -0700 Stephane Eranian <[email protected]> wrote:
Hello,
For perfmon, we need a couple of TIF bits. It seems that with 2.6.22-rc2 there is now a TIF_RESTORE_SIGMASK which uses the last remaining bit in the first 7 bits of the thread flag. Many architectures, including IA-64, rely on the fact that some of the TIF flags (TIF_ALL_WORKMASK or TIF_ALL_WORK) tested on kernel exit reside in the low 8-bit or 7-bit because they use instructions (such as add r1=imm8,r2 on IA-64) which operate on 8 or 7 bit immediate.
On IA-64, adding that one perfmon flag (as bit 7) would cause some restructuring in the kernel exit path but also in all the lightweight syscall handlers.
I looked at all the low order TIF flags and found that TIF_NOTIFY_RESUME was never set nor used anywhere in any architecture. Is that really the case?
If so, we could get rid of it and free up one low-order TIF bit.

Quoted 8 lines

Quoted 2 lines
My grepping argees with yours. The only place where TIF_NOTIFY_RESUME gets altered is in ./arch/ia64/kernel/perfmon.c.
Yes, and that is with the old IA-64 code. In the new one I used a dedicated TIF flag.
Shall we just get rid of the flag, then?

I'd say so, yes.

Stephane Eranian

author

2007-05-22 22:51 UTC #4 lore raw

Andrew,

Quoted 32 lines · Andrew Morton

Quoted 29 lines · On Tue, 22 May 2007 09:07:37 -0700 Stephane Eranian <[email protected]> wrote:
Andrew,
Quoted 25 lines · Andrew Morton
Quoted 10 lines · On Tue, 22 May 2007 05:47:13 -0700 Stephane Eranian <[email protected]> wrote:
Hello,
For perfmon, we need a couple of TIF bits. It seems that with 2.6.22-rc2 there is now a TIF_RESTORE_SIGMASK which uses the last remaining bit in the first 7 bits of the thread flag. Many architectures, including IA-64, rely on the fact that some of the TIF flags (TIF_ALL_WORKMASK or TIF_ALL_WORK) tested on kernel exit reside in the low 8-bit or 7-bit because they use instructions (such as add r1=imm8,r2 on IA-64) which operate on 8 or 7 bit immediate.
Quoted 9 lines
On IA-64, adding that one perfmon flag (as bit 7) would cause some restructuring in the kernel exit path but also in all the lightweight syscall handlers.
I looked at all the low order TIF flags and found that TIF_NOTIFY_RESUME was never set nor used anywhere in any architecture. Is that really the case?
If so, we could get rid of it and free up one low-order TIF bit.
My grepping argees with yours. The only place where TIF_NOTIFY_RESUME gets altered is in ./arch/ia64/kernel/perfmon.c.

Quoted 7 lines

Quoted 5 lines
Yes, and that is with the old IA-64 code. In the new one I used a dedicated TIF flag.
Shall we just get rid of the flag, then?
I'd say so, yes.

Do you want a single patch or small patches broken-down by arch (and there are many...)?

Signature · 1 line

-Stephane

Andrew Morton

2007-05-22 23:02 UTC #5 lore raw

Quoted 38 lines · On Tue, 22 May 2007 15:51:35 -0700 Stephane Eranian <[email protected]> wrote:

Andrew,
Quoted 35 lines · Andrew Morton
Quoted 32 lines · On Tue, 22 May 2007 09:07:37 -0700 Stephane Eranian <[email protected]> wrote:
Andrew,
Quoted 17 lines · Andrew Morton
Quoted 14 lines · On Tue, 22 May 2007 05:47:13 -0700 Stephane Eranian <[email protected]> wrote:
Hello,
For perfmon, we need a couple of TIF bits. It seems that with 2.6.22-rc2 there is now a TIF_RESTORE_SIGMASK which uses the last remaining bit in the first 7 bits of the thread flag. Many architectures, including IA-64, rely on the fact that some of the TIF flags (TIF_ALL_WORKMASK or TIF_ALL_WORK) tested on kernel exit reside in the low 8-bit or 7-bit because they use instructions (such as add r1=imm8,r2 on IA-64) which operate on 8 or 7 bit immediate.
On IA-64, adding that one perfmon flag (as bit 7) would cause some restructuring in the kernel exit path but also in all the lightweight syscall handlers.
Quoted 8 lines
Quoted 5 lines
I looked at all the low order TIF flags and found that TIF_NOTIFY_RESUME was never set nor used anywhere in any architecture. Is that really the case?
If so, we could get rid of it and free up one low-order TIF bit.
My grepping argees with yours. The only place where TIF_NOTIFY_RESUME gets altered is in ./arch/ia64/kernel/perfmon.c.
Yes, and that is with the old IA-64 code. In the new one I used a dedicated TIF flag.

Quoted 6 lines

Quoted 4 lines
Quoted 2 lines
Shall we just get rid of the flag, then?
I'd say so, yes.
Do you want a single patch or small patches broken-down by arch (and there are many...)?

One patch is OK for this, thanks.

Stephane Eranian

author

2007-05-23 09:57 UTC #6 lore raw

PATCH 1/1 remove unused TIF_NOTIFY_RESUME flag

Hello,

As per the recent discussion, the folowing patch removes the TIF_NOTIFY_RESUME flag from all processor architectures. The flag was not used except on IA-64 for the perfmon subsystem. For IA-64, the patch replaces the flag with a new dedicated TIF flag. This provides an extra low-order bit for the TIF flags. I tested the kernel compiles on i386,x86_64,mips,ia64,powerpc.

The patch is relative to 2.6.22-rc2 but applies smoothly to 2.6.22-rc2-git3.

Changelog:
- remove unused TIF_NOTIFY_RESUME flag for all processor architectures. The flag was not used excecpt on IA-64 where the patch replaces it with TIF_PERFMON_WORK.

Signed-off-by stephane eranian <[email protected]>

arch/arm/kernel/entry-common.S +1 -1

diff -urNp linux-2.6.22.orig/arch/arm/kernel/entry-common.S linux-2.6.22.base/arch/arm/kernel/entry-common.S
@@ -46,7 +46,7 @@ fast_work_pending:
 work_pending:
 	tst	r1, #_TIF_NEED_RESCHED
 	bne	work_resched
-	tst	r1, #_TIF_NOTIFY_RESUME | _TIF_SIGPENDING
+	tst	r1, #_TIF_SIGPENDING
 	beq	no_work_pending
 	mov	r0, sp				@ 'regs'
 	mov	r2, why				@ 'syscall'

arch/arm26/kernel/entry.S +1 -1

diff -urNp linux-2.6.22.orig/arch/arm26/kernel/entry.S linux-2.6.22.base/arch/arm26/kernel/entry.S
@@ -194,7 +194,7 @@ fast_work_pending:
 work_pending:
 	tst	r1, #_TIF_NEED_RESCHED
 	bne	work_resched
-	tst	r1, #_TIF_NOTIFY_RESUME | _TIF_SIGPENDING
+	tst	r1, #_TIF_SIGPENDING
 	beq	no_work_pending
 	mov	r0, sp				@ 'regs'
 	mov	r2, why				@ 'syscall'

arch/ia64/kernel/perfmon.c +2 -2

diff -urNp linux-2.6.22.orig/arch/ia64/kernel/perfmon.c linux-2.6.22.base/arch/ia64/kernel/perfmon.c
@@ -591,13 +591,13 @@ pfm_set_task_notify(struct task_struct *
 	struct thread_info *info;
 
 	info = (struct thread_info *) ((char *) task + IA64_TASK_SIZE);
-	set_bit(TIF_NOTIFY_RESUME, &info->flags);
+	set_bit(TIF_PERFMON_WORK, &info->flags);
 }
 
 static inline void
 pfm_clear_task_notify(void)
 {
-	clear_thread_flag(TIF_NOTIFY_RESUME);
+	clear_thread_flag(TIF_PERFMON_WORK);
 }
 
 static inline void

arch/powerpc/kernel/entry_64.S +0 -1

diff -urNp linux-2.6.22.orig/arch/powerpc/kernel/entry_64.S linux-2.6.22.base/arch/powerpc/kernel/entry_64.S
@@ -546,7 +546,6 @@ END_FW_FTR_SECTION_IFSET(FW_FEATURE_ISER
 	rfid
 	b	.	/* prevent speculative execution */
 
-/* Note: this must change if we start using the TIF_NOTIFY_RESUME bit */
 do_work:
 #ifdef CONFIG_PREEMPT
 	andi.	r0,r3,MSR_PR	/* Returning to user mode? */

arch/x86_64/kernel/entry.S +3 -3

diff -urNp linux-2.6.22.orig/arch/x86_64/kernel/entry.S linux-2.6.22.base/arch/x86_64/kernel/entry.S
@@ -282,7 +282,7 @@ sysret_careful:
 sysret_signal:
 	TRACE_IRQS_ON
 	sti
-	testl $(_TIF_SIGPENDING|_TIF_NOTIFY_RESUME|_TIF_SINGLESTEP),%edx
+	testl $(_TIF_SIGPENDING|_TIF_SINGLESTEP),%edx
 	jz    1f
 
 	/* Really a signal */

@@ -375,7 +375,7 @@ int_very_careful:
 	jmp int_restore_rest
 	
 int_signal:
-	testl $(_TIF_NOTIFY_RESUME|_TIF_SIGPENDING|_TIF_SINGLESTEP),%edx
+	testl $(_TIF_SIGPENDING|_TIF_SINGLESTEP),%edx
 	jz 1f
 	movq %rsp,%rdi		# &ptregs -> arg1
 	xorl %esi,%esi		# oldset -> arg2

@@ -599,7 +599,7 @@ retint_careful:
 	jmp retint_check
 	
 retint_signal:
-	testl $(_TIF_SIGPENDING|_TIF_NOTIFY_RESUME|_TIF_SINGLESTEP),%edx
+	testl $(_TIF_SIGPENDING|_TIF_SINGLESTEP),%edx
 	jz    retint_swapgs
 	TRACE_IRQS_ON
 	sti

include/asm-alpha/thread_info.h +9 ~~-12~~

diff -urNp linux-2.6.22.orig/include/asm-alpha/thread_info.h linux-2.6.22.base/include/asm-alpha/thread_info.h
@@ -67,25 +67,22 @@ register struct thread_info *__current_t
  * TIF_SYSCALL_TRACE is known to be 0 via blbs.
  */
 #define TIF_SYSCALL_TRACE	0	/* syscall trace active */
-#define TIF_NOTIFY_RESUME	1	/* resumption notification requested */
-#define TIF_SIGPENDING		2	/* signal pending */
-#define TIF_NEED_RESCHED	3	/* rescheduling necessary */
-#define TIF_POLLING_NRFLAG	4	/* poll_idle is polling NEED_RESCHED */
-#define TIF_DIE_IF_KERNEL	5	/* dik recursion lock */
-#define TIF_UAC_NOPRINT		6	/* see sysinfo.h */
-#define TIF_UAC_NOFIX		7
-#define TIF_UAC_SIGBUS		8
-#define TIF_MEMDIE		9
+#define TIF_SIGPENDING		1	/* signal pending */
+#define TIF_NEED_RESCHED	2	/* rescheduling necessary */
+#define TIF_POLLING_NRFLAG	3	/* poll_idle is polling NEED_RESCHED */
+#define TIF_DIE_IF_KERNEL	4	/* dik recursion lock */
+#define TIF_UAC_NOPRINT		5	/* see sysinfo.h */
+#define TIF_UAC_NOFIX		6
+#define TIF_UAC_SIGBUS		7
+#define TIF_MEMDIE		8
 
 #define _TIF_SYSCALL_TRACE	(1<<TIF_SYSCALL_TRACE)
-#define _TIF_NOTIFY_RESUME	(1<<TIF_NOTIFY_RESUME)
 #define _TIF_SIGPENDING		(1<<TIF_SIGPENDING)
 #define _TIF_NEED_RESCHED	(1<<TIF_NEED_RESCHED)
 #define _TIF_POLLING_NRFLAG	(1<<TIF_POLLING_NRFLAG)
 
 /* Work to do on interrupt/exception return.  */
-#define _TIF_WORK_MASK		(_TIF_NOTIFY_RESUME	\
-				 | _TIF_SIGPENDING	\
+#define _TIF_WORK_MASK		(_TIF_SIGPENDING	\
 				 | _TIF_NEED_RESCHED)
 
 /* Work to do on any return to userspace.  */

include/asm-arm/thread_info.h +2 -5

diff -urNp linux-2.6.22.orig/include/asm-arm/thread_info.h linux-2.6.22.base/include/asm-arm/thread_info.h
@@ -135,22 +135,19 @@ extern void iwmmxt_task_switch(struct th
 /*
  * thread information flags:
  *  TIF_SYSCALL_TRACE	- syscall trace active
- *  TIF_NOTIFY_RESUME	- resumption notification requested
  *  TIF_SIGPENDING	- signal pending
  *  TIF_NEED_RESCHED	- rescheduling necessary
  *  TIF_USEDFPU		- FPU was used by this task this quantum (SMP)
  *  TIF_POLLING_NRFLAG	- true if poll_idle() is polling TIF_NEED_RESCHED
  */
-#define TIF_NOTIFY_RESUME	0
-#define TIF_SIGPENDING		1
-#define TIF_NEED_RESCHED	2
+#define TIF_SIGPENDING		0
+#define TIF_NEED_RESCHED	1
 #define TIF_SYSCALL_TRACE	8
 #define TIF_POLLING_NRFLAG	16
 #define TIF_USING_IWMMXT	17
 #define TIF_MEMDIE		18
 #define TIF_FREEZE		19
 
-#define _TIF_NOTIFY_RESUME	(1 << TIF_NOTIFY_RESUME)
 #define _TIF_SIGPENDING		(1 << TIF_SIGPENDING)
 #define _TIF_NEED_RESCHED	(1 << TIF_NEED_RESCHED)
 #define _TIF_SYSCALL_TRACE	(1 << TIF_SYSCALL_TRACE)

include/asm-arm26/thread_info.h +2 -5

diff -urNp linux-2.6.22.orig/include/asm-arm26/thread_info.h linux-2.6.22.base/include/asm-arm26/thread_info.h
@@ -110,21 +110,18 @@ extern void free_thread_info(struct thre
 /*
  * thread information flags:
  *  TIF_SYSCALL_TRACE	- syscall trace active
- *  TIF_NOTIFY_RESUME	- resumption notification requested
  *  TIF_SIGPENDING	- signal pending
  *  TIF_NEED_RESCHED	- rescheduling necessary
  *  TIF_USEDFPU		- FPU was used by this task this quantum (SMP)
  *  TIF_POLLING_NRFLAG	- true if poll_idle() is polling TIF_NEED_RESCHED
  */
-#define TIF_NOTIFY_RESUME	0
-#define TIF_SIGPENDING		1
-#define TIF_NEED_RESCHED	2
+#define TIF_SIGPENDING		0
+#define TIF_NEED_RESCHED	1
 #define TIF_SYSCALL_TRACE	8
 #define TIF_USED_FPU		16
 #define TIF_POLLING_NRFLAG	17
 #define TIF_MEMDIE		18
 
-#define _TIF_NOTIFY_RESUME	(1 << TIF_NOTIFY_RESUME)
 #define _TIF_SIGPENDING		(1 << TIF_SIGPENDING)
 #define _TIF_NEED_RESCHED	(1 << TIF_NEED_RESCHED)
 #define _TIF_SYSCALL_TRACE	(1 << TIF_SYSCALL_TRACE)

include/asm-avr32/thread_info.h +8 ~~-10~~

diff -urNp linux-2.6.22.orig/include/asm-avr32/thread_info.h linux-2.6.22.base/include/asm-avr32/thread_info.h
@@ -74,20 +74,18 @@ static inline struct thread_info *curren
  * - other flags in MSW
  */
 #define TIF_SYSCALL_TRACE       0       /* syscall trace active */
-#define TIF_NOTIFY_RESUME       1       /* resumption notification requested */
-#define TIF_SIGPENDING          2       /* signal pending */
-#define TIF_NEED_RESCHED        3       /* rescheduling necessary */
-#define TIF_POLLING_NRFLAG      4       /* true if poll_idle() is polling
+#define TIF_SIGPENDING          1       /* signal pending */
+#define TIF_NEED_RESCHED        2       /* rescheduling necessary */
+#define TIF_POLLING_NRFLAG      3       /* true if poll_idle() is polling
 					   TIF_NEED_RESCHED */
-#define TIF_BREAKPOINT		5	/* true if we should break after return */
-#define TIF_SINGLE_STEP		6	/* single step after next break */
-#define TIF_MEMDIE		7
-#define TIF_RESTORE_SIGMASK	8	/* restore signal mask in do_signal */
-#define TIF_CPU_GOING_TO_SLEEP	9	/* CPU is entering sleep 0 mode */
+#define TIF_BREAKPOINT		4	/* true if we should break after return */
+#define TIF_SINGLE_STEP		5	/* single step after next break */
+#define TIF_MEMDIE		6
+#define TIF_RESTORE_SIGMASK	7	/* restore signal mask in do_signal */
+#define TIF_CPU_GOING_TO_SLEEP	8	/* CPU is entering sleep 0 mode */
 #define TIF_USERSPACE		31      /* true if FS sets userspace */
 
 #define _TIF_SYSCALL_TRACE	(1 << TIF_SYSCALL_TRACE)
-#define _TIF_NOTIFY_RESUME	(1 << TIF_NOTIFY_RESUME)
 #define _TIF_SIGPENDING		(1 << TIF_SIGPENDING)
 #define _TIF_NEED_RESCHED	(1 << TIF_NEED_RESCHED)
 #define _TIF_POLLING_NRFLAG	(1 << TIF_POLLING_NRFLAG)

include/asm-blackfin/thread_info.h +6 -8

diff -urNp linux-2.6.22.orig/include/asm-blackfin/thread_info.h linux-2.6.22.base/include/asm-blackfin/thread_info.h
@@ -118,18 +118,16 @@ static inline struct thread_info *curren
  * thread information flag bit numbers
  */
 #define TIF_SYSCALL_TRACE	0	/* syscall trace active */
-#define TIF_NOTIFY_RESUME	1	/* resumption notification requested */
-#define TIF_SIGPENDING		2	/* signal pending */
-#define TIF_NEED_RESCHED	3	/* rescheduling necessary */
-#define TIF_POLLING_NRFLAG	4	/* true if poll_idle() is polling
+#define TIF_SIGPENDING		1	/* signal pending */
+#define TIF_NEED_RESCHED	2	/* rescheduling necessary */
+#define TIF_POLLING_NRFLAG	3	/* true if poll_idle() is polling
 					   TIF_NEED_RESCHED */
-#define TIF_MEMDIE              5
-#define TIF_RESTORE_SIGMASK	6	/* restore signal mask in do_signal() */
-#define TIF_FREEZE              7       /* is freezing for suspend */
+#define TIF_MEMDIE              4
+#define TIF_RESTORE_SIGMASK	5	/* restore signal mask in do_signal() */
+#define TIF_FREEZE              6       /* is freezing for suspend */
 
 /* as above, but as bit values */
 #define _TIF_SYSCALL_TRACE	(1<<TIF_SYSCALL_TRACE)
-#define _TIF_NOTIFY_RESUME	(1<<TIF_NOTIFY_RESUME)
 #define _TIF_SIGPENDING		(1<<TIF_SIGPENDING)
 #define _TIF_NEED_RESCHED	(1<<TIF_NEED_RESCHED)
 #define _TIF_POLLING_NRFLAG	(1<<TIF_POLLING_NRFLAG)

include/asm-cris/thread_info.h +2 -4

diff -urNp linux-2.6.22.orig/include/asm-cris/thread_info.h linux-2.6.22.base/include/asm-cris/thread_info.h
@@ -79,14 +79,12 @@ struct thread_info {
  * - other flags in MSW
  */
 #define TIF_SYSCALL_TRACE	0	/* syscall trace active */
-#define TIF_NOTIFY_RESUME	1	/* resumption notification requested */
-#define TIF_SIGPENDING		2	/* signal pending */
-#define TIF_NEED_RESCHED	3	/* rescheduling necessary */
+#define TIF_SIGPENDING		1	/* signal pending */
+#define TIF_NEED_RESCHED	2	/* rescheduling necessary */
 #define TIF_POLLING_NRFLAG	16	/* true if poll_idle() is polling TIF_NEED_RESCHED */
 #define TIF_MEMDIE		17
 
 #define _TIF_SYSCALL_TRACE	(1<<TIF_SYSCALL_TRACE)
-#define _TIF_NOTIFY_RESUME	(1<<TIF_NOTIFY_RESUME)
 #define _TIF_SIGPENDING		(1<<TIF_SIGPENDING)
 #define _TIF_NEED_RESCHED	(1<<TIF_NEED_RESCHED)
 #define _TIF_POLLING_NRFLAG	(1<<TIF_POLLING_NRFLAG)

include/asm-frv/thread_info.h +5 -7

diff -urNp linux-2.6.22.orig/include/asm-frv/thread_info.h linux-2.6.22.base/include/asm-frv/thread_info.h
@@ -108,18 +108,16 @@ register struct thread_info *__current_t
  * - other flags in MSW
  */
 #define TIF_SYSCALL_TRACE	0	/* syscall trace active */
-#define TIF_NOTIFY_RESUME	1	/* resumption notification requested */
-#define TIF_SIGPENDING		2	/* signal pending */
-#define TIF_NEED_RESCHED	3	/* rescheduling necessary */
-#define TIF_SINGLESTEP		4	/* restore singlestep on return to user mode */
-#define TIF_IRET		5	/* return with iret */
-#define TIF_RESTORE_SIGMASK	6	/* restore signal mask in do_signal() */
+#define TIF_SIGPENDING		1	/* signal pending */
+#define TIF_NEED_RESCHED	2	/* rescheduling necessary */
+#define TIF_SINGLESTEP		3	/* restore singlestep on return to user mode */
+#define TIF_IRET		4	/* return with iret */
+#define TIF_RESTORE_SIGMASK	5	/* restore signal mask in do_signal() */
 #define TIF_POLLING_NRFLAG	16	/* true if poll_idle() is polling TIF_NEED_RESCHED */
 #define TIF_MEMDIE		17	/* OOM killer killed process */
 #define TIF_FREEZE		18	/* freezing for suspend */
 
 #define _TIF_SYSCALL_TRACE	(1 << TIF_SYSCALL_TRACE)
-#define _TIF_NOTIFY_RESUME	(1 << TIF_NOTIFY_RESUME)
 #define _TIF_SIGPENDING		(1 << TIF_SIGPENDING)
 #define _TIF_NEED_RESCHED	(1 << TIF_NEED_RESCHED)
 #define _TIF_SINGLESTEP		(1 << TIF_SINGLESTEP)

include/asm-h8300/thread_info.h +4 -6

diff -urNp linux-2.6.22.orig/include/asm-h8300/thread_info.h linux-2.6.22.base/include/asm-h8300/thread_info.h
@@ -86,16 +86,14 @@ static inline struct thread_info *curren
  * thread information flag bit numbers
  */
 #define TIF_SYSCALL_TRACE	0	/* syscall trace active */
-#define TIF_NOTIFY_RESUME	1	/* resumption notification requested */
-#define TIF_SIGPENDING		2	/* signal pending */
-#define TIF_NEED_RESCHED	3	/* rescheduling necessary */
-#define TIF_POLLING_NRFLAG	4	/* true if poll_idle() is polling
+#define TIF_SIGPENDING		1	/* signal pending */
+#define TIF_NEED_RESCHED	2	/* rescheduling necessary */
+#define TIF_POLLING_NRFLAG	3	/* true if poll_idle() is polling
 					   TIF_NEED_RESCHED */
-#define TIF_MEMDIE		5
+#define TIF_MEMDIE		4
 
 /* as above, but as bit values */
 #define _TIF_SYSCALL_TRACE	(1<<TIF_SYSCALL_TRACE)
-#define _TIF_NOTIFY_RESUME	(1<<TIF_NOTIFY_RESUME)
 #define _TIF_SIGPENDING		(1<<TIF_SIGPENDING)
 #define _TIF_NEED_RESCHED	(1<<TIF_NEED_RESCHED)
 #define _TIF_POLLING_NRFLAG	(1<<TIF_POLLING_NRFLAG)

include/asm-i386/thread_info.h +8 ~~-10~~

diff -urNp linux-2.6.22.orig/include/asm-i386/thread_info.h linux-2.6.22.base/include/asm-i386/thread_info.h
@@ -124,22 +124,20 @@ static inline struct thread_info *curren
  * - other flags in MSW
  */
 #define TIF_SYSCALL_TRACE	0	/* syscall trace active */
-#define TIF_NOTIFY_RESUME	1	/* resumption notification requested */
-#define TIF_SIGPENDING		2	/* signal pending */
-#define TIF_NEED_RESCHED	3	/* rescheduling necessary */
-#define TIF_SINGLESTEP		4	/* restore singlestep on return to user mode */
-#define TIF_IRET		5	/* return with iret */
-#define TIF_SYSCALL_EMU		6	/* syscall emulation active */
-#define TIF_SYSCALL_AUDIT	7	/* syscall auditing active */
-#define TIF_SECCOMP		8	/* secure computing */
-#define TIF_RESTORE_SIGMASK	9	/* restore signal mask in do_signal() */
+#define TIF_SIGPENDING		1	/* signal pending */
+#define TIF_NEED_RESCHED	2	/* rescheduling necessary */
+#define TIF_SINGLESTEP		3	/* restore singlestep on return to user mode */
+#define TIF_IRET		4	/* return with iret */
+#define TIF_SYSCALL_EMU		5	/* syscall emulation active */
+#define TIF_SYSCALL_AUDIT	6	/* syscall auditing active */
+#define TIF_SECCOMP		7	/* secure computing */
+#define TIF_RESTORE_SIGMASK	8	/* restore signal mask in do_signal() */
 #define TIF_MEMDIE		16
 #define TIF_DEBUG		17	/* uses debug registers */
 #define TIF_IO_BITMAP		18	/* uses I/O bitmap */
 #define TIF_FREEZE		19	/* is freezing for suspend */
 
 #define _TIF_SYSCALL_TRACE	(1<<TIF_SYSCALL_TRACE)
-#define _TIF_NOTIFY_RESUME	(1<<TIF_NOTIFY_RESUME)
 #define _TIF_SIGPENDING		(1<<TIF_SIGPENDING)
 #define _TIF_NEED_RESCHED	(1<<TIF_NEED_RESCHED)
 #define _TIF_SINGLESTEP		(1<<TIF_SINGLESTEP)

include/asm-ia64/thread_info.h +11 -9

diff -urNp linux-2.6.22.orig/include/asm-ia64/thread_info.h linux-2.6.22.base/include/asm-ia64/thread_info.h
@@ -79,13 +79,13 @@ struct thread_info {
  * - pending work-to-be-done flags are in least-significant 16 bits, other flags
  *   in top 16 bits
  */
-#define TIF_NOTIFY_RESUME	0	/* resumption notification requested */
-#define TIF_SIGPENDING		1	/* signal pending */
-#define TIF_NEED_RESCHED	2	/* rescheduling necessary */
-#define TIF_SYSCALL_TRACE	3	/* syscall trace active */
-#define TIF_SYSCALL_AUDIT	4	/* syscall auditing active */
-#define TIF_SINGLESTEP		5	/* restore singlestep on return to user mode */
-#define TIF_RESTORE_SIGMASK	6	/* restore signal mask in do_signal() */
+#define TIF_SIGPENDING		0	/* signal pending */
+#define TIF_NEED_RESCHED	1	/* rescheduling necessary */
+#define TIF_SYSCALL_TRACE	2	/* syscall trace active */
+#define TIF_SYSCALL_AUDIT	3	/* syscall auditing active */
+#define TIF_SINGLESTEP		4	/* restore singlestep on return to user mode */
+#define TIF_RESTORE_SIGMASK	5	/* restore signal mask in do_signal() */
+#define TIF_PERFMON_WORK	6	/* work for pfm_handle_work() */
 #define TIF_POLLING_NRFLAG	16	/* true if poll_idle() is polling TIF_NEED_RESCHED */
 #define TIF_MEMDIE		17
 #define TIF_MCA_INIT		18	/* this task is processing MCA or INIT */

@@ -96,8 +96,8 @@ struct thread_info {
 #define _TIF_SYSCALL_AUDIT	(1 << TIF_SYSCALL_AUDIT)
 #define _TIF_SINGLESTEP		(1 << TIF_SINGLESTEP)
 #define _TIF_SYSCALL_TRACEAUDIT	(_TIF_SYSCALL_TRACE|_TIF_SYSCALL_AUDIT|_TIF_SINGLESTEP)
-#define _TIF_NOTIFY_RESUME	(1 << TIF_NOTIFY_RESUME)
 #define _TIF_RESTORE_SIGMASK	(1 << TIF_RESTORE_SIGMASK)
+#define _TIF_PERFMON_WORK	(1 << TIF_PERFMON_WORK)
 #define _TIF_SIGPENDING		(1 << TIF_SIGPENDING)
 #define _TIF_NEED_RESCHED	(1 << TIF_NEED_RESCHED)
 #define _TIF_POLLING_NRFLAG	(1 << TIF_POLLING_NRFLAG)

@@ -106,7 +106,9 @@ struct thread_info {
 #define _TIF_FREEZE		(1 << TIF_FREEZE)
 
 /* "work to do on user-return" bits */
-#define TIF_ALLWORK_MASK	(_TIF_NOTIFY_RESUME|_TIF_SIGPENDING|_TIF_NEED_RESCHED|_TIF_SYSCALL_TRACE|_TIF_SYSCALL_AUDIT|_TIF_RESTORE_SIGMASK)
+#define TIF_ALLWORK_MASK	(_TIF_SIGPENDING|_TIF_PERFMON_WORK|_TIF_SYSCALL_AUDIT|\
+				 _TIF_NEED_RESCHED| _TIF_SYSCALL_TRACE|\
+				 _TIF_RESTORE_SIGMASK)
 /* like TIF_ALLWORK_BITS but sans TIF_SYSCALL_TRACE or TIF_SYSCALL_AUDIT */
 #define TIF_WORK_MASK		(TIF_ALLWORK_MASK&~(_TIF_SYSCALL_TRACE|_TIF_SYSCALL_AUDIT))

include/asm-m32r/thread_info.h +4 -6

diff -urNp linux-2.6.22.orig/include/asm-m32r/thread_info.h linux-2.6.22.base/include/asm-m32r/thread_info.h
@@ -146,17 +146,15 @@ static inline unsigned int get_thread_fa
  * - other flags in MSW
  */
 #define TIF_SYSCALL_TRACE	0	/* syscall trace active */
-#define TIF_NOTIFY_RESUME	1	/* resumption notification requested */
-#define TIF_SIGPENDING		2	/* signal pending */
-#define TIF_NEED_RESCHED	3	/* rescheduling necessary */
-#define TIF_SINGLESTEP		4	/* restore singlestep on return to user mode */
-#define TIF_IRET		5	/* return with iret */
+#define TIF_SIGPENDING		1	/* signal pending */
+#define TIF_NEED_RESCHED	2	/* rescheduling necessary */
+#define TIF_SINGLESTEP		3	/* restore singlestep on return to user mode */
+#define TIF_IRET		4	/* return with iret */
 #define TIF_POLLING_NRFLAG	16	/* true if poll_idle() is polling TIF_NEED_RESCHED */
 					/* 31..28 fault code */
 #define TIF_MEMDIE		17
 
 #define _TIF_SYSCALL_TRACE	(1<<TIF_SYSCALL_TRACE)
-#define _TIF_NOTIFY_RESUME	(1<<TIF_NOTIFY_RESUME)
 #define _TIF_SIGPENDING		(1<<TIF_SIGPENDING)
 #define _TIF_NEED_RESCHED	(1<<TIF_NEED_RESCHED)
 #define _TIF_SINGLESTEP		(1<<TIF_SINGLESTEP)

include/asm-m68knommu/thread_info.h +4 -6

diff -urNp linux-2.6.22.orig/include/asm-m68knommu/thread_info.h linux-2.6.22.base/include/asm-m68knommu/thread_info.h
@@ -83,16 +83,14 @@ static inline struct thread_info *curren
  * thread information flag bit numbers
  */
 #define TIF_SYSCALL_TRACE	0	/* syscall trace active */
-#define TIF_NOTIFY_RESUME	1	/* resumption notification requested */
-#define TIF_SIGPENDING		2	/* signal pending */
-#define TIF_NEED_RESCHED	3	/* rescheduling necessary */
-#define TIF_POLLING_NRFLAG	4	/* true if poll_idle() is polling
+#define TIF_SIGPENDING		1	/* signal pending */
+#define TIF_NEED_RESCHED	2	/* rescheduling necessary */
+#define TIF_POLLING_NRFLAG	3	/* true if poll_idle() is polling
 					   TIF_NEED_RESCHED */
-#define TIF_MEMDIE		5
+#define TIF_MEMDIE		4
 
 /* as above, but as bit values */
 #define _TIF_SYSCALL_TRACE	(1<<TIF_SYSCALL_TRACE)
-#define _TIF_NOTIFY_RESUME	(1<<TIF_NOTIFY_RESUME)
 #define _TIF_SIGPENDING		(1<<TIF_SIGPENDING)
 #define _TIF_NEED_RESCHED	(1<<TIF_NEED_RESCHED)
 #define _TIF_POLLING_NRFLAG	(1<<TIF_POLLING_NRFLAG)

include/asm-mips/thread_info.h +4 -6

diff -urNp linux-2.6.22.orig/include/asm-mips/thread_info.h linux-2.6.22.base/include/asm-mips/thread_info.h
@@ -109,11 +109,10 @@ register struct thread_info *__current_t
  * - pending work-to-be-done flags are in LSW
  * - other flags in MSW
  */
-#define TIF_NOTIFY_RESUME	1	/* resumption notification requested */
-#define TIF_SIGPENDING		2	/* signal pending */
-#define TIF_NEED_RESCHED	3	/* rescheduling necessary */
-#define TIF_SYSCALL_AUDIT	4	/* syscall auditing active */
-#define TIF_SECCOMP		5	/* secure computing */
+#define TIF_SIGPENDING		1	/* signal pending */
+#define TIF_NEED_RESCHED	2	/* rescheduling necessary */
+#define TIF_SYSCALL_AUDIT	3	/* syscall auditing active */
+#define TIF_SECCOMP		4	/* secure computing */
 #define TIF_RESTORE_SIGMASK	9	/* restore signal mask in do_signal() */
 #define TIF_USEDFPU		16	/* FPU was used by this task this quantum (SMP) */
 #define TIF_POLLING_NRFLAG	17	/* true if poll_idle() is polling TIF_NEED_RESCHED */

@@ -122,7 +121,6 @@ register struct thread_info *__current_t
 #define TIF_SYSCALL_TRACE	31	/* syscall trace active */
 
 #define _TIF_SYSCALL_TRACE	(1<<TIF_SYSCALL_TRACE)
-#define _TIF_NOTIFY_RESUME	(1<<TIF_NOTIFY_RESUME)
 #define _TIF_SIGPENDING		(1<<TIF_SIGPENDING)
 #define _TIF_NEED_RESCHED	(1<<TIF_NEED_RESCHED)
 #define _TIF_SYSCALL_AUDIT	(1<<TIF_SYSCALL_AUDIT)

include/asm-parisc/thread_info.h +7 -9

diff -urNp linux-2.6.22.orig/include/asm-parisc/thread_info.h linux-2.6.22.base/include/asm-parisc/thread_info.h
@@ -56,23 +56,21 @@ struct thread_info {
  * thread information flags
  */
 #define TIF_SYSCALL_TRACE	0	/* syscall trace active */
-#define TIF_NOTIFY_RESUME	1	/* resumption notification requested */
-#define TIF_SIGPENDING		2	/* signal pending */
-#define TIF_NEED_RESCHED	3	/* rescheduling necessary */
-#define TIF_POLLING_NRFLAG	4	/* true if poll_idle() is polling TIF_NEED_RESCHED */
-#define TIF_32BIT               5       /* 32 bit binary */
-#define TIF_MEMDIE		6
-#define TIF_RESTORE_SIGMASK	7	/* restore saved signal mask */
+#define TIF_SIGPENDING		1	/* signal pending */
+#define TIF_NEED_RESCHED	2	/* rescheduling necessary */
+#define TIF_POLLING_NRFLAG	3	/* true if poll_idle() is polling TIF_NEED_RESCHED */
+#define TIF_32BIT               4       /* 32 bit binary */
+#define TIF_MEMDIE		5
+#define TIF_RESTORE_SIGMASK	6	/* restore saved signal mask */
 
 #define _TIF_SYSCALL_TRACE	(1 << TIF_SYSCALL_TRACE)
-#define _TIF_NOTIFY_RESUME	(1 << TIF_NOTIFY_RESUME)
 #define _TIF_SIGPENDING		(1 << TIF_SIGPENDING)
 #define _TIF_NEED_RESCHED	(1 << TIF_NEED_RESCHED)
 #define _TIF_POLLING_NRFLAG	(1 << TIF_POLLING_NRFLAG)
 #define _TIF_32BIT		(1 << TIF_32BIT)
 #define _TIF_RESTORE_SIGMASK	(1 << TIF_RESTORE_SIGMASK)
 
-#define _TIF_USER_WORK_MASK     (_TIF_NOTIFY_RESUME | _TIF_SIGPENDING | \
+#define _TIF_USER_WORK_MASK     (_TIF_SIGPENDING | \
                                  _TIF_NEED_RESCHED | _TIF_RESTORE_SIGMASK)
 
 #endif /* __KERNEL__ */

include/asm-powerpc/thread_info.h +12 ~~-14~~

diff -urNp linux-2.6.22.orig/include/asm-powerpc/thread_info.h linux-2.6.22.base/include/asm-powerpc/thread_info.h
@@ -107,26 +107,24 @@ static inline struct thread_info *curren
  * thread information flag bit numbers
  */
 #define TIF_SYSCALL_TRACE	0	/* syscall trace active */
-#define TIF_NOTIFY_RESUME	1	/* resumption notification requested */
-#define TIF_SIGPENDING		2	/* signal pending */
-#define TIF_NEED_RESCHED	3	/* rescheduling necessary */
-#define TIF_POLLING_NRFLAG	4	/* true if poll_idle() is polling
+#define TIF_SIGPENDING		1	/* signal pending */
+#define TIF_NEED_RESCHED	2	/* rescheduling necessary */
+#define TIF_POLLING_NRFLAG	3	/* true if poll_idle() is polling
 					   TIF_NEED_RESCHED */
-#define TIF_32BIT		5	/* 32 bit binary */
-#define TIF_RUNLATCH		6	/* Is the runlatch enabled? */
-#define TIF_ABI_PENDING		7	/* 32/64 bit switch needed */
-#define TIF_SYSCALL_AUDIT	8	/* syscall auditing active */
-#define TIF_SINGLESTEP		9	/* singlestepping active */
-#define TIF_MEMDIE		10
-#define TIF_SECCOMP		11	/* secure computing */
-#define TIF_RESTOREALL		12	/* Restore all regs (implies NOERROR) */
+#define TIF_32BIT		4	/* 32 bit binary */
+#define TIF_RUNLATCH		5	/* Is the runlatch enabled? */
+#define TIF_ABI_PENDING		6	/* 32/64 bit switch needed */
+#define TIF_SYSCALL_AUDIT	7	/* syscall auditing active */
+#define TIF_SINGLESTEP		8	/* singlestepping active */
+#define TIF_MEMDIE		9
+#define TIF_SECCOMP		10	/* secure computing */
+#define TIF_RESTOREALL		11	/* Restore all regs (implies NOERROR) */
 #define TIF_NOERROR		14	/* Force successful syscall return */
 #define TIF_RESTORE_SIGMASK	15	/* Restore signal mask in do_signal */
 #define TIF_FREEZE		16	/* Freezing for suspend */
 
 /* as above, but as bit values */
 #define _TIF_SYSCALL_TRACE	(1<<TIF_SYSCALL_TRACE)
-#define _TIF_NOTIFY_RESUME	(1<<TIF_NOTIFY_RESUME)
 #define _TIF_SIGPENDING		(1<<TIF_SIGPENDING)
 #define _TIF_NEED_RESCHED	(1<<TIF_NEED_RESCHED)
 #define _TIF_POLLING_NRFLAG	(1<<TIF_POLLING_NRFLAG)

@@ -142,7 +140,7 @@ static inline struct thread_info *curren
 #define _TIF_FREEZE		(1<<TIF_FREEZE)
 #define _TIF_SYSCALL_T_OR_A	(_TIF_SYSCALL_TRACE|_TIF_SYSCALL_AUDIT|_TIF_SECCOMP)
 
-#define _TIF_USER_WORK_MASK	(_TIF_NOTIFY_RESUME | _TIF_SIGPENDING | \
+#define _TIF_USER_WORK_MASK	( _TIF_SIGPENDING | \
 				 _TIF_NEED_RESCHED | _TIF_RESTORE_SIGMASK)
 #define _TIF_PERSYSCALL_MASK	(_TIF_RESTOREALL|_TIF_NOERROR)

include/asm-sh/thread_info.h +4 -6

diff -urNp linux-2.6.22.orig/include/asm-sh/thread_info.h linux-2.6.22.base/include/asm-sh/thread_info.h
@@ -107,18 +107,16 @@ static inline struct thread_info *curren
  * - other flags in MSW
  */
 #define TIF_SYSCALL_TRACE	0	/* syscall trace active */
-#define TIF_NOTIFY_RESUME	1	/* resumption notification requested */
-#define TIF_SIGPENDING		2	/* signal pending */
-#define TIF_NEED_RESCHED	3	/* rescheduling necessary */
-#define TIF_RESTORE_SIGMASK	4	/* restore signal mask in do_signal() */
-#define TIF_SINGLESTEP		5	/* singlestepping active */
+#define TIF_SIGPENDING		1	/* signal pending */
+#define TIF_NEED_RESCHED	2	/* rescheduling necessary */
+#define TIF_RESTORE_SIGMASK	3	/* restore signal mask in do_signal() */
+#define TIF_SINGLESTEP		4	/* singlestepping active */
 #define TIF_USEDFPU		16	/* FPU was used by this task this quantum (SMP) */
 #define TIF_POLLING_NRFLAG	17	/* true if poll_idle() is polling TIF_NEED_RESCHED */
 #define TIF_MEMDIE		18
 #define TIF_FREEZE		19
 
 #define _TIF_SYSCALL_TRACE	(1<<TIF_SYSCALL_TRACE)
-#define _TIF_NOTIFY_RESUME	(1<<TIF_NOTIFY_RESUME)
 #define _TIF_SIGPENDING		(1<<TIF_SIGPENDING)
 #define _TIF_NEED_RESCHED	(1<<TIF_NEED_RESCHED)
 #define _TIF_RESTORE_SIGMASK	(1<<TIF_RESTORE_SIGMASK)

include/asm-v850/thread_info.h +4 -6

diff -urNp linux-2.6.22.orig/include/asm-v850/thread_info.h linux-2.6.22.base/include/asm-v850/thread_info.h
@@ -77,16 +77,14 @@ struct thread_info {
  * thread information flag bit numbers
  */
 #define TIF_SYSCALL_TRACE	0	/* syscall trace active */
-#define TIF_NOTIFY_RESUME	1	/* resumption notification requested */
-#define TIF_SIGPENDING		2	/* signal pending */
-#define TIF_NEED_RESCHED	3	/* rescheduling necessary */
-#define TIF_POLLING_NRFLAG	4	/* true if poll_idle() is polling
+#define TIF_SIGPENDING		1	/* signal pending */
+#define TIF_NEED_RESCHED	2	/* rescheduling necessary */
+#define TIF_POLLING_NRFLAG	3	/* true if poll_idle() is polling
 					   TIF_NEED_RESCHED */
-#define TIF_MEMDIE		5
+#define TIF_MEMDIE		4
 
 /* as above, but as bit values */
 #define _TIF_SYSCALL_TRACE	(1<<TIF_SYSCALL_TRACE)
-#define _TIF_NOTIFY_RESUME	(1<<TIF_NOTIFY_RESUME)
 #define _TIF_SIGPENDING		(1<<TIF_SIGPENDING)
 #define _TIF_NEED_RESCHED	(1<<TIF_NEED_RESCHED)
 #define _TIF_POLLING_NRFLAG	(1<<TIF_POLLING_NRFLAG)

include/asm-x86_64/thread_info.h +7 -9

diff -urNp linux-2.6.22.orig/include/asm-x86_64/thread_info.h linux-2.6.22.base/include/asm-x86_64/thread_info.h
@@ -107,14 +107,13 @@ static inline struct thread_info *stack_
  * Warning: layout of LSW is hardcoded in entry.S
  */
 #define TIF_SYSCALL_TRACE	0	/* syscall trace active */
-#define TIF_NOTIFY_RESUME	1	/* resumption notification requested */
-#define TIF_SIGPENDING		2	/* signal pending */
-#define TIF_NEED_RESCHED	3	/* rescheduling necessary */
-#define TIF_SINGLESTEP		4	/* reenable singlestep on user return*/
-#define TIF_IRET		5	/* force IRET */
-#define TIF_SYSCALL_AUDIT	7	/* syscall auditing active */
-#define TIF_SECCOMP		8	/* secure computing */
-#define TIF_RESTORE_SIGMASK	9	/* restore signal mask in do_signal */
+#define TIF_SIGPENDING		1	/* signal pending */
+#define TIF_NEED_RESCHED	2	/* rescheduling necessary */
+#define TIF_SINGLESTEP		3	/* reenable singlestep on user return*/
+#define TIF_IRET		4	/* force IRET */
+#define TIF_SYSCALL_AUDIT	5	/* syscall auditing active */
+#define TIF_SECCOMP		6	/* secure computing */
+#define TIF_RESTORE_SIGMASK	7	/* restore signal mask in do_signal */
 /* 16 free */
 #define TIF_IA32		17	/* 32bit process */ 
 #define TIF_FORK		18	/* ret_from_fork */

@@ -125,7 +124,6 @@ static inline struct thread_info *stack_
 #define TIF_FREEZE		23	/* is freezing for suspend */
 
 #define _TIF_SYSCALL_TRACE	(1<<TIF_SYSCALL_TRACE)
-#define _TIF_NOTIFY_RESUME	(1<<TIF_NOTIFY_RESUME)
 #define _TIF_SIGPENDING		(1<<TIF_SIGPENDING)
 #define _TIF_SINGLESTEP		(1<<TIF_SINGLESTEP)
 #define _TIF_NEED_RESCHED	(1<<TIF_NEED_RESCHED)

include/asm-xtensa/thread_info.h +5 -7

diff -urNp linux-2.6.22.orig/include/asm-xtensa/thread_info.h linux-2.6.22.base/include/asm-xtensa/thread_info.h
@@ -110,16 +110,14 @@ static inline struct thread_info *curren
  * - other flags in MSW
  */
 #define TIF_SYSCALL_TRACE	0	/* syscall trace active */
-#define TIF_NOTIFY_RESUME	1	/* resumption notification requested */
-#define TIF_SIGPENDING		2	/* signal pending */
-#define TIF_NEED_RESCHED	3	/* rescheduling necessary */
-#define TIF_SINGLESTEP		4	/* restore singlestep on return to user mode */
-#define TIF_IRET		5	/* return with iret */
-#define TIF_MEMDIE		6
+#define TIF_SIGPENDING		1	/* signal pending */
+#define TIF_NEED_RESCHED	2	/* rescheduling necessary */
+#define TIF_SINGLESTEP		3	/* restore singlestep on return to user mode */
+#define TIF_IRET		4	/* return with iret */
+#define TIF_MEMDIE		5
 #define TIF_POLLING_NRFLAG	16	/* true if poll_idle() is polling TIF_NEED_RESCHED */
 
 #define _TIF_SYSCALL_TRACE	(1<<TIF_SYSCALL_TRACE)
-#define _TIF_NOTIFY_RESUME	(1<<TIF_NOTIFY_RESUME)
 #define _TIF_SIGPENDING		(1<<TIF_SIGPENDING)
 #define _TIF_NEED_RESCHED	(1<<TIF_NEED_RESCHED)
 #define _TIF_SINGLESTEP		(1<<TIF_SINGLESTEP)

Fenghua Yu

2007-05-22 18:23 UTC #7 lore raw

PATCH 1/2 Define new percpu interface for shared data -- version 3

In reply to Andrew Morton

Changes from previous version:
Fix a compilation error for ARM

per cpu data section contains two types of data. One set which is exclusively accessed by the local cpu and the other set which is per cpu, but also shared by remote cpus. In the current kernel, these two sets are not clearely separated out. This can potentially cause the same data cacheline shared between the two sets of data, which will result in unnecessary bouncing of the cacheline between cpus.

One way to fix the problem is to cacheline align the remotely accessed per cpu data, both at the beginning and at the end. Because of the padding at both ends, this will likely cause some memory wastage and also the interface to achieve this is not clean.

This patch:

Moves the remotely accessed per cpu data (which is currently marked as ____cacheline_aligned_in_smp) into a different section, where all the data elements are cacheline aligned. And as such, this differentiates the local only data and remotely accessed data cleanly.

igned-off-by Fenghua Yu <[email protected]>

Acked-by Suresh Siddha <[email protected]>

---

 arch/alpha/kernel/vmlinux.lds.S   |    5 +----
 arch/arm/kernel/vmlinux.lds.S     |    1 +
 arch/cris/arch-v32/vmlinux.lds.S  |    5 +----
 arch/frv/kernel/vmlinux.lds.S     |    5 +----
 arch/i386/kernel/vmlinux.lds.S    |    7 +------
 arch/ia64/kernel/vmlinux.lds.S    |    1 +
 arch/m32r/kernel/vmlinux.lds.S    |    5 +----
 arch/mips/kernel/vmlinux.lds.S    |    5 +----
 arch/parisc/kernel/vmlinux.lds.S  |    7 +++----
 arch/powerpc/kernel/vmlinux.lds.S |    7 +------
 arch/ppc/kernel/vmlinux.lds.S     |    5 +----
 arch/s390/kernel/vmlinux.lds.S    |    5 +----
 arch/sh/kernel/vmlinux.lds.S      |    5 +----
 arch/sh64/kernel/vmlinux.lds.S    |    5 +----
 arch/sparc/kernel/vmlinux.lds.S   |    5 +----
 arch/sparc64/kernel/vmlinux.lds.S |    5 +----
 arch/x86_64/kernel/vmlinux.lds.S  |    6 ++----
 arch/xtensa/kernel/vmlinux.lds.S  |    5 +----
 include/asm-generic/percpu.h      |    8 ++++++++
 include/asm-generic/vmlinux.lds.h |    8 ++++++++
 include/asm-i386/percpu.h         |    5 +++++
 include/asm-ia64/percpu.h         |   10 ++++++++++
 include/asm-powerpc/percpu.h      |    7 +++++++
 include/asm-s390/percpu.h         |    7 +++++++
 include/asm-sparc64/percpu.h      |    7 +++++++
 include/asm-x86_64/percpu.h       |    7 +++++++
 26 files changed, 80 insertions(+), 68 deletions(-)

arch/alpha/kernel/vmlinux.lds.S +1 -4

diff -Nurp linux-2.6.22-rc1-mm1.0/arch/alpha/kernel/vmlinux.lds.S linux-2.6.22-rc1-mm1.1/arch/alpha/kernel/vmlinux.lds.S
@@ -69,10 +69,7 @@ SECTIONS
   . = ALIGN(8);
   SECURITY_INIT
 
-  . = ALIGN(8192);
-  __per_cpu_start = .;
-  .data.percpu : { *(.data.percpu) }
-  __per_cpu_end = .;
+  PERCPU(8192)
 
   . = ALIGN(2*8192);
   __init_end = .;

arch/arm/kernel/vmlinux.lds.S +1 -0

diff -Nurp linux-2.6.22-rc1-mm1.0/arch/arm/kernel/vmlinux.lds.S linux-2.6.22-rc1-mm1.1/arch/arm/kernel/vmlinux.lds.S
@@ -66,6 +66,7 @@ SECTIONS
 		. = ALIGN(4096);
 		__per_cpu_start = .;
 			*(.data.percpu)
+			*(.data.percpu.shared_aligned)
 		__per_cpu_end = .;
 #ifndef CONFIG_XIP_KERNEL
 		__init_begin = _stext;

arch/cris/arch-v32/vmlinux.lds.S +1 -4

diff -Nurp linux-2.6.22-rc1-mm1.0/arch/cris/arch-v32/vmlinux.lds.S linux-2.6.22-rc1-mm1.1/arch/cris/arch-v32/vmlinux.lds.S
@@ -91,10 +91,7 @@ SECTIONS
 	}
 	SECURITY_INIT
 
-	. =  ALIGN (8192);
-	__per_cpu_start = .;
-	.data.percpu  : { *(.data.percpu) }
-	__per_cpu_end = .;
+	PERCPU(8192)
 
 #ifdef CONFIG_BLK_DEV_INITRD
 	.init.ramfs : {

arch/frv/kernel/vmlinux.lds.S +1 -4

diff -Nurp linux-2.6.22-rc1-mm1.0/arch/frv/kernel/vmlinux.lds.S linux-2.6.22-rc1-mm1.1/arch/frv/kernel/vmlinux.lds.S
@@ -57,10 +57,7 @@ SECTIONS
   __alt_instructions_end = .;
  .altinstr_replacement : { *(.altinstr_replacement) }
 
-  . = ALIGN(4096);
-  __per_cpu_start = .;
-  .data.percpu  : { *(.data.percpu) }
-  __per_cpu_end = .;
+  PERCPU(4096)
 
 #ifdef CONFIG_BLK_DEV_INITRD
   . = ALIGN(4096);

arch/i386/kernel/vmlinux.lds.S +1 -6

diff -Nurp linux-2.6.22-rc1-mm1.0/arch/i386/kernel/vmlinux.lds.S linux-2.6.22-rc1-mm1.1/arch/i386/kernel/vmlinux.lds.S
@@ -178,12 +178,7 @@ SECTIONS
 	__initramfs_end = .;
   }
 #endif
-  . = ALIGN(4096);
-  .data.percpu  : AT(ADDR(.data.percpu) - LOAD_OFFSET) {
-	__per_cpu_start = .;
-	*(.data.percpu)
-	__per_cpu_end = .;
-  }
+  PERCPU(4096)
   . = ALIGN(4096);
   /* freed after init ends here */

arch/ia64/kernel/vmlinux.lds.S +1 -0

diff -Nurp linux-2.6.22-rc1-mm1.0/arch/ia64/kernel/vmlinux.lds.S linux-2.6.22-rc1-mm1.1/arch/ia64/kernel/vmlinux.lds.S
@@ -206,6 +206,7 @@ SECTIONS
 	{
 		__per_cpu_start = .;
 		*(.data.percpu)
+		*(.data.percpu.shared_aligned)
 		__per_cpu_end = .;
 	}
   . = __phys_per_cpu_start + PERCPU_PAGE_SIZE;	/* ensure percpu data fits

arch/m32r/kernel/vmlinux.lds.S +1 -4

diff -Nurp linux-2.6.22-rc1-mm1.0/arch/m32r/kernel/vmlinux.lds.S linux-2.6.22-rc1-mm1.1/arch/m32r/kernel/vmlinux.lds.S
@@ -110,10 +110,7 @@ SECTIONS
   __initramfs_end = .;
 #endif
 
-  . = ALIGN(4096);
-  __per_cpu_start = .;
-  .data.percpu  : { *(.data.percpu) }
-  __per_cpu_end = .;
+  PERCPU(4096)
   . = ALIGN(4096);
   __init_end = .;
   /* freed after init ends here */

arch/mips/kernel/vmlinux.lds.S +1 -4

diff -Nurp linux-2.6.22-rc1-mm1.0/arch/mips/kernel/vmlinux.lds.S linux-2.6.22-rc1-mm1.1/arch/mips/kernel/vmlinux.lds.S
@@ -119,10 +119,7 @@ SECTIONS
   .init.ramfs : { *(.init.ramfs) }
   __initramfs_end = .;
 #endif
-  . = ALIGN(_PAGE_SIZE);
-  __per_cpu_start = .;
-  .data.percpu  : { *(.data.percpu) }
-  __per_cpu_end = .;
+  PERCPU(_PAGE_SIZE)
   . = ALIGN(_PAGE_SIZE);
   __init_end = .;
   /* freed after init ends here */

arch/parisc/kernel/vmlinux.lds.S +3 -4

diff -Nurp linux-2.6.22-rc1-mm1.0/arch/parisc/kernel/vmlinux.lds.S linux-2.6.22-rc1-mm1.1/arch/parisc/kernel/vmlinux.lds.S
@@ -181,10 +181,9 @@ SECTIONS
   .init.ramfs : { *(.init.ramfs) }
   __initramfs_end = .;
 #endif
-  . = ALIGN(ASM_PAGE_SIZE);
-  __per_cpu_start = .;
-  .data.percpu  : { *(.data.percpu) }
-  __per_cpu_end = .;
+
+  PERCPU(ASM_PAGE_SIZE)
+
   . = ALIGN(ASM_PAGE_SIZE);
   __init_end = .;
   /* freed after init ends here */

arch/powerpc/kernel/vmlinux.lds.S +1 -6

diff -Nurp linux-2.6.22-rc1-mm1.0/arch/powerpc/kernel/vmlinux.lds.S linux-2.6.22-rc1-mm1.1/arch/powerpc/kernel/vmlinux.lds.S
@@ -139,12 +139,7 @@ SECTIONS
 		__initramfs_end = .;
 	}
 #endif
-	. = ALIGN(PAGE_SIZE);
-	.data.percpu : {
-		__per_cpu_start = .;
-		*(.data.percpu)
-		__per_cpu_end = .;
-	}
+	PERCPU(PAGE_SIZE)
 
 	. = ALIGN(8);
 	.machine.desc : {

arch/ppc/kernel/vmlinux.lds.S +1 -4

diff -Nurp linux-2.6.22-rc1-mm1.0/arch/ppc/kernel/vmlinux.lds.S linux-2.6.22-rc1-mm1.1/arch/ppc/kernel/vmlinux.lds.S
@@ -130,10 +130,7 @@ SECTIONS
   __ftr_fixup : { *(__ftr_fixup) }
   __stop___ftr_fixup = .;
 
-  . = ALIGN(4096);
-  __per_cpu_start = .;
-  .data.percpu  : { *(.data.percpu) }
-  __per_cpu_end = .;
+  PERCPU(4096)
 
 #ifdef CONFIG_BLK_DEV_INITRD
   . = ALIGN(4096);

arch/s390/kernel/vmlinux.lds.S +1 -4

diff -Nurp linux-2.6.22-rc1-mm1.0/arch/s390/kernel/vmlinux.lds.S linux-2.6.22-rc1-mm1.1/arch/s390/kernel/vmlinux.lds.S
@@ -107,10 +107,7 @@ SECTIONS
   . = ALIGN(2);
   __initramfs_end = .;
 #endif
-  . = ALIGN(4096);
-  __per_cpu_start = .;
-  .data.percpu  : { *(.data.percpu) }
-  __per_cpu_end = .;
+  PERCPU(4096)
   . = ALIGN(4096);
   __init_end = .;
   /* freed after init ends here */

arch/sh/kernel/vmlinux.lds.S +1 -4

diff -Nurp linux-2.6.22-rc1-mm1.0/arch/sh/kernel/vmlinux.lds.S linux-2.6.22-rc1-mm1.1/arch/sh/kernel/vmlinux.lds.S
@@ -60,10 +60,7 @@ SECTIONS
   . = ALIGN(PAGE_SIZE);
   __nosave_end = .;
 
-  . = ALIGN(PAGE_SIZE);
-  __per_cpu_start = .;
-  .data.percpu : { *(.data.percpu) }
-  __per_cpu_end = .;
+  PERCPU(PAGE_SIZE)
   .data.cacheline_aligned : { *(.data.cacheline_aligned) }
 
   _edata = .;			/* End of data section */

arch/sh64/kernel/vmlinux.lds.S +1 -4

diff -Nurp linux-2.6.22-rc1-mm1.0/arch/sh64/kernel/vmlinux.lds.S linux-2.6.22-rc1-mm1.1/arch/sh64/kernel/vmlinux.lds.S
@@ -85,10 +85,7 @@ SECTIONS
   . = ALIGN(PAGE_SIZE);
   .data.page_aligned : C_PHYS(.data.page_aligned) { *(.data.page_aligned) }
 
-  . = ALIGN(PAGE_SIZE);
-  __per_cpu_start = .;
-  .data.percpu : C_PHYS(.data.percpu) { *(.data.percpu) }
-  __per_cpu_end = . ;
+  PERCPU(PAGE_SIZE)
   .data.cacheline_aligned : C_PHYS(.data.cacheline_aligned) { *(.data.cacheline_aligned) }
 
   _edata = .;			/* End of data section */

arch/sparc/kernel/vmlinux.lds.S +1 -4

diff -Nurp linux-2.6.22-rc1-mm1.0/arch/sparc/kernel/vmlinux.lds.S linux-2.6.22-rc1-mm1.1/arch/sparc/kernel/vmlinux.lds.S
@@ -65,10 +65,7 @@ SECTIONS
   __initramfs_end = .;
 #endif
 
-  . = ALIGN(4096);
-  __per_cpu_start = .;
-  .data.percpu  : { *(.data.percpu) }
-  __per_cpu_end = .;
+  PERCPU(4096)
   . = ALIGN(4096);
   __init_end = .;
   . = ALIGN(32);

arch/sparc64/kernel/vmlinux.lds.S +1 -4

diff -Nurp linux-2.6.22-rc1-mm1.0/arch/sparc64/kernel/vmlinux.lds.S linux-2.6.22-rc1-mm1.1/arch/sparc64/kernel/vmlinux.lds.S
@@ -89,10 +89,7 @@ SECTIONS
   __initramfs_end = .;
 #endif
 
-  . = ALIGN(8192);
-  __per_cpu_start = .;
-  .data.percpu  : { *(.data.percpu) }
-  __per_cpu_end = .;
+  PERCPU(8192)
   . = ALIGN(8192);
   __init_end = .;
   __bss_start = .;

arch/x86_64/kernel/vmlinux.lds.S +2 -4

diff -Nurp linux-2.6.22-rc1-mm1.0/arch/x86_64/kernel/vmlinux.lds.S linux-2.6.22-rc1-mm1.1/arch/x86_64/kernel/vmlinux.lds.S
@@ -203,10 +203,8 @@ SECTIONS
   __initramfs_end = .;
 #endif
 
-  . = ALIGN(4096);
-  __per_cpu_start = .;
-  .data.percpu  : AT(ADDR(.data.percpu) - LOAD_OFFSET) { *(.data.percpu) }
-  __per_cpu_end = .;
+  PERCPU(4096)
+
   . = ALIGN(4096);
   __init_end = .;

arch/xtensa/kernel/vmlinux.lds.S +1 -4

diff -Nurp linux-2.6.22-rc1-mm1.0/arch/xtensa/kernel/vmlinux.lds.S linux-2.6.22-rc1-mm1.1/arch/xtensa/kernel/vmlinux.lds.S
@@ -198,10 +198,7 @@ SECTIONS
   __ftr_fixup : { *(__ftr_fixup) }
   __stop___ftr_fixup = .;
 
-  . = ALIGN(4096);
-  __per_cpu_start = .;
-  .data.percpu  : { *(.data.percpu) }
-  __per_cpu_end = .;
+  PERCPU(4096)
 
 #ifdef CONFIG_BLK_DEV_INITRD
   . = ALIGN(4096);

include/asm-generic/percpu.h +8 -0

diff -Nurp linux-2.6.22-rc1-mm1.0/include/asm-generic/percpu.h linux-2.6.22-rc1-mm1.1/include/asm-generic/percpu.h
@@ -14,6 +14,11 @@ extern unsigned long __per_cpu_offset[NR
 #define DEFINE_PER_CPU(type, name) \
     __attribute__((__section__(".data.percpu"))) __typeof__(type) per_cpu__##name
 
+#define DEFINE_PER_CPU_SHARED_ALIGNED(type, name)		\
+    __attribute__((__section__(".data.percpu.shared_aligned"))) \
+    __typeof__(type) per_cpu__##name				\
+    ____cacheline_aligned_in_smp
+
 /* var is in discarded region: offset to particular copy we want */
 #define per_cpu(var, cpu) (*({				\
 	extern int simple_identifier_##var(void);	\

@@ -34,6 +39,9 @@ do {								\
 #define DEFINE_PER_CPU(type, name) \
     __typeof__(type) per_cpu__##name
 
+#define DEFINE_PER_CPU_SHARED_ALIGNED(type, name)	\
+    DEFINE_PER_CPU(type, name)
+
 #define per_cpu(var, cpu)			(*((void)(cpu), &per_cpu__##var))
 #define __get_cpu_var(var)			per_cpu__##var
 #define __raw_get_cpu_var(var)			per_cpu__##var

include/asm-generic/vmlinux.lds.h +8 -0

diff -Nurp linux-2.6.22-rc1-mm1.0/include/asm-generic/vmlinux.lds.h linux-2.6.22-rc1-mm1.1/include/asm-generic/vmlinux.lds.h
@@ -251,3 +251,11 @@
   	*(.initcall7.init)						\
   	*(.initcall7s.init)
 
+#define PERCPU(align)							\
+	. = ALIGN(align);						\
+	__per_cpu_start = .;						\
+	.data.percpu  : AT(ADDR(.data.percpu) - LOAD_OFFSET) {		\
+		*(.data.percpu)						\
+		*(.data.percpu.shared_aligned)				\
+	}								\
+	__per_cpu_end = .;

include/asm-i386/percpu.h +5 -0

diff -Nurp linux-2.6.22-rc1-mm1.0/include/asm-i386/percpu.h linux-2.6.22-rc1-mm1.1/include/asm-i386/percpu.h
@@ -54,6 +54,11 @@ extern unsigned long __per_cpu_offset[];
 #define DEFINE_PER_CPU(type, name) \
     __attribute__((__section__(".data.percpu"))) __typeof__(type) per_cpu__##name
 
+#define DEFINE_PER_CPU_SHARED_ALIGNED(type, name)		\
+    __attribute__((__section__(".data.percpu.shared_aligned"))) \
+    __typeof__(type) per_cpu__##name				\
+    ____cacheline_aligned_in_smp
+
 /* We can use this directly for local CPU (faster). */
 DECLARE_PER_CPU(unsigned long, this_cpu_off);

include/asm-ia64/percpu.h +10 -0

diff -Nurp linux-2.6.22-rc1-mm1.0/include/asm-ia64/percpu.h linux-2.6.22-rc1-mm1.1/include/asm-ia64/percpu.h
@@ -29,6 +29,16 @@
 	__attribute__((__section__(".data.percpu")))		\
 	__SMALL_ADDR_AREA __typeof__(type) per_cpu__##name
 
+#ifdef CONFIG_SMP
+#define DEFINE_PER_CPU_SHARED_ALIGNED(type, name)			\
+	__attribute__((__section__(".data.percpu.shared_aligned")))	\
+	__SMALL_ADDR_AREA __typeof__(type) per_cpu__##name		\
+	____cacheline_aligned_in_smp
+#else
+#define DEFINE_PER_CPU_SHARED_ALIGNED(type, name)	\
+	DEFINE_PER_CPU(type, name)
+#endif
+
 /*
  * Pretty much a literal copy of asm-generic/percpu.h, except that percpu_modcopy() is an
  * external routine, to avoid include-hell.

include/asm-powerpc/percpu.h +7 -0

diff -Nurp linux-2.6.22-rc1-mm1.0/include/asm-powerpc/percpu.h linux-2.6.22-rc1-mm1.1/include/asm-powerpc/percpu.h
@@ -20,6 +20,11 @@
 #define DEFINE_PER_CPU(type, name) \
     __attribute__((__section__(".data.percpu"))) __typeof__(type) per_cpu__##name
 
+#define DEFINE_PER_CPU_SHARED_ALIGNED(type, name)		\
+    __attribute__((__section__(".data.percpu.shared_aligned"))) \
+    __typeof__(type) per_cpu__##name				\
+    ____cacheline_aligned_in_smp
+
 /* var is in discarded region: offset to particular copy we want */
 #define per_cpu(var, cpu) (*RELOC_HIDE(&per_cpu__##var, __per_cpu_offset(cpu)))
 #define __get_cpu_var(var) (*RELOC_HIDE(&per_cpu__##var, __my_cpu_offset()))

@@ -40,6 +45,8 @@ extern void setup_per_cpu_areas(void);
 
 #define DEFINE_PER_CPU(type, name) \
     __typeof__(type) per_cpu__##name
+#define DEFINE_PER_CPU_SHARED_ALIGNED(type, name)	\
+    DEFINE_PER_CPU(type, name)
 
 #define per_cpu(var, cpu)			(*((void)(cpu), &per_cpu__##var))
 #define __get_cpu_var(var)			per_cpu__##var

include/asm-s390/percpu.h +7 -0

diff -Nurp linux-2.6.22-rc1-mm1.0/include/asm-s390/percpu.h linux-2.6.22-rc1-mm1.1/include/asm-s390/percpu.h
@@ -41,6 +41,11 @@ extern unsigned long __per_cpu_offset[NR
     __attribute__((__section__(".data.percpu"))) \
     __typeof__(type) per_cpu__##name
 
+#define DEFINE_PER_CPU_SHARED_ALIGNED(type, name)		\
+    __attribute__((__section__(".data.percpu.shared_aligned"))) \
+    __typeof__(type) per_cpu__##name				\
+    ____cacheline_aligned_in_smp
+
 #define __get_cpu_var(var) __reloc_hide(var,S390_lowcore.percpu_offset)
 #define __raw_get_cpu_var(var) __reloc_hide(var,S390_lowcore.percpu_offset)
 #define per_cpu(var,cpu) __reloc_hide(var,__per_cpu_offset[cpu])

@@ -59,6 +64,8 @@ do {								\
 
 #define DEFINE_PER_CPU(type, name) \
     __typeof__(type) per_cpu__##name
+#define DEFINE_PER_CPU_SHARED_ALIGNED(type, name)	\
+    DEFINE_PER_CPU(type, name)
 
 #define __get_cpu_var(var) __reloc_hide(var,0)
 #define __raw_get_cpu_var(var) __reloc_hide(var,0)

include/asm-sparc64/percpu.h +7 -0

diff -Nurp linux-2.6.22-rc1-mm1.0/include/asm-sparc64/percpu.h linux-2.6.22-rc1-mm1.1/include/asm-sparc64/percpu.h
@@ -17,6 +17,11 @@ extern unsigned long __per_cpu_shift;
 #define DEFINE_PER_CPU(type, name) \
     __attribute__((__section__(".data.percpu"))) __typeof__(type) per_cpu__##name
 
+#define DEFINE_PER_CPU_SHARED_ALIGNED(type, name)		\
+    __attribute__((__section__(".data.percpu.shared_aligned"))) \
+    __typeof__(type) per_cpu__##name				\
+    ____cacheline_aligned_in_smp
+
 register unsigned long __local_per_cpu_offset asm("g5");
 
 /* var is in discarded region: offset to particular copy we want */

@@ -36,6 +41,8 @@ do {								\
 
 #define DEFINE_PER_CPU(type, name) \
     __typeof__(type) per_cpu__##name
+#define DEFINE_PER_CPU_SHARED_ALIGNED(type, name)	\
+    DEFINE_PER_CPU(type, name)
 
 #define per_cpu(var, cpu)			(*((void)cpu, &per_cpu__##var))
 #define __get_cpu_var(var)			per_cpu__##var

include/asm-x86_64/percpu.h +7 -0

diff -Nurp linux-2.6.22-rc1-mm1.0/include/asm-x86_64/percpu.h linux-2.6.22-rc1-mm1.1/include/asm-x86_64/percpu.h
@@ -20,6 +20,11 @@
 #define DEFINE_PER_CPU(type, name) \
     __attribute__((__section__(".data.percpu"))) __typeof__(type) per_cpu__##name
 
+#define DEFINE_PER_CPU_SHARED_ALIGNED(type, name)		\
+    __attribute__((__section__(".data.percpu.shared_aligned"))) \
+    __typeof__(type) per_cpu__##name				\
+    ____cacheline_internodealigned_in_smp
+
 /* var is in discarded region: offset to particular copy we want */
 #define per_cpu(var, cpu) (*({				\
 	extern int simple_identifier_##var(void);	\

@@ -46,6 +51,8 @@ extern void setup_per_cpu_areas(void);
 
 #define DEFINE_PER_CPU(type, name) \
     __typeof__(type) per_cpu__##name
+#define DEFINE_PER_CPU_SHARED_ALIGNED(type, name)	\
+    DEFINE_PER_CPU(type, name)
 
 #define per_cpu(var, cpu)			(*((void)(cpu), &per_cpu__##var))
 #define __get_cpu_var(var)			per_cpu__##var

Andrew Morton

2007-05-23 17:50 UTC #8 lore raw

Re: [PATCH 1/2] Define new percpu interface for shared data -- version 3

Quoted 12 lines · On Tue, 22 May 2007 11:20:03 -0700 Fenghua Yu <[email protected]> wrote:

per cpu data section contains two types of data. One set which is exclusively accessed by the local cpu and the other set which is per cpu, but also shared by remote cpus. In the current kernel, these two sets are not clearely separated out. This can potentially cause the same data cacheline shared between the two sets of data, which will result in unnecessary bouncing of the cacheline between cpus.
One way to fix the problem is to cacheline align the remotely accessed per cpu data, both at the beginning and at the end. Because of the padding at both ends, this will likely cause some memory wastage and also the interface to achieve this is not clean.

Quoted 6 lines

This patch:
Moves the remotely accessed per cpu data (which is currently marked as ____cacheline_aligned_in_smp) into a different section, where all the data elements are cacheline aligned. And as such, this differentiates the local only data and remotely accessed data cleanly.

OK, but could we please have a concise description of the impact of these changes on kernel memory footprint? Increase or decrease? And by approximately how much?

Thanks.

Yu, Fenghua

2007-05-23 18:27 UTC #9 lore raw

RE: [PATCH 1/2] Define new percpu interface for shared data -- version 3

Quoted 1 line

elements are cacheline aligned. And as such, this differentiates the

local

Quoted 5 lines

only data and remotely accessed data cleanly.
OK, but could we please have a concise description of the impact of these changes on kernel memory footprint? Increase or decrease? And by approximately how much?

Depending on how linker places percpu data, the patches could increase or decrease percpu section size. Data from 2.6.21-rc7-mm2:

On x86 SMP, the section size is increased from 0x7768 to 0x790c. 1.3% increase.

On X86-64 SMP, the size is decreased from 0x72d0 to 0x6540. 11.8% decrease.

On X86-64 VSMP, the size is increased from 0x72d0 to 0x8340. 14.3% increase.

On IA64 SMP, the size is decreased from 0x8370 to 0x7fc0. 2.8% decrease.

Ravikiran G Thirumalai

2007-05-23 18:57 UTC #10 lore raw

Re: [PATCH 1/2] Define new percpu interface for shared data -- version 3

Quoted 14 lines · Fenghua

Quoted 1 line
elements are cacheline aligned. And as such, this differentiates the
local
Quoted 5 lines
only data and remotely accessed data cleanly.
OK, but could we please have a concise description of the impact of these changes on kernel memory footprint? Increase or decrease? And by approximately how much?
Depending on how linker places percpu data, the patches could increase or decrease percpu section size. Data from 2.6.21-rc7-mm2:
On x86 SMP, the section size is increased from 0x7768 to 0x790c. 1.3% increase.

Quoted 8 lines

On X86-64 SMP, the size is decreased from 0x72d0 to 0x6540. 11.8% decrease.
On X86-64 VSMP, the size is increased from 0x72d0 to 0x8340. 14.3% increase.
On IA64 SMP, the size is decreased from 0x8370 to 0x7fc0. 2.8% decrease.

Has there been any measurable benefit yet due to tail padding? It would also be interesting to check the wastage/savings on another large cache architecture like S390 (which has a 256 byte cache line)

Thanks,
Kiran

Yu, Fenghua

2007-05-23 19:10 UTC #11 lore raw

RE: [PATCH 1/2] Define new percpu interface for shared data -- version 3

Quoted 2 lines

Has there been any measurable benefit yet due to tail padding?

We don't have data that tail padding actually helps. It all depends on what data the linker lays out in the cachelines.

As of now we just want to create the infrastructure (so that more and more people who need it, can use it).

Quoted 1 line

It would also be interesting to check the wastage/savings on another

large

Quoted 1 line

cache architecture like S390 (which has a 256 byte cache line)

S390 folks or others might help get the data. I don't have other machines
on hand.

Thanks.

-Fenghua

Ravikiran G Thirumalai

2007-05-23 19:19 UTC #12 lore raw

Re: [PATCH 1/2] Define new percpu interface for shared data -- version 3

Quoted 8 lines · Fenghua

Quoted 2 lines
Has there been any measurable benefit yet due to tail padding?
We don't have data that tail padding actually helps. It all depends on what data the linker lays out in the cachelines.
As of now we just want to create the infrastructure (so that more and more people who need it, can use it).

So what we have now is space wastage on some architectures, space savings on some, but with no measurable performance benefit due to the infrastructure itself. Why not push the infrastructure when we really need it, as against pushing it now when we are not sure if it benefits?

Thanks,
Kiran

Andrew Morton

2007-05-23 19:32 UTC #13 lore raw

Re: [PATCH 1/2] Define new percpu interface for shared data -- version 3

Quoted 7 lines · On Wed, 23 May 2007 12:20:05 -0700 Ravikiran G Thirumalai <[email protected]>…

Quoted 6 lines · Fenghua
Quoted 2 lines
Has there been any measurable benefit yet due to tail padding?
We don't have data that tail padding actually helps. It all depends on what data the linker lays out in the cachelines.

Quoted 8 lines

Quoted 2 lines
As of now we just want to create the infrastructure (so that more and more people who need it, can use it).
So what we have now is space wastage on some architectures, space savings on some, but with no measurable performance benefit due to the infrastructure itself. Why not push the infrastructure when we really need it, as against pushing it now when we are not sure if it benefits?

It makes sense from a theoretical POV and is pretty much a no-op in terms of resource consumption.

The problem with the wait-until-it-hurts approach is that by the time someone hurts from this and we find out about it, they may well be using some year-old enterprise kernel and it's too late to fix it for them.

Yu, Fenghua

2007-05-23 21:13 UTC #14 lore raw

RE: [PATCH 1/2] Define new percpu interface for shared data -- version 3

Quoted 2 lines

Quoted 2 lines
So what we have now is space wastage on some architectures, space

savings on

Quoted 1 line

Quoted 1 line
some, but with no measurable performance benefit due to the

infrastructure

Quoted 1 line

Quoted 1 line
itself. Why not push the infrastructure when we really need it, as

against

Quoted 4 lines

Quoted 1 line
pushing it now when we are not sure if it benefits?
It makes sense from a theoretical POV and is pretty much a no-op in

terms

Quoted 4 lines

of resource consumption.
The problem with the wait-until-it-hurts approach is that by the time someone hurts from this and we find out about it, they may well be

using

Quoted 1 line

some year-old enterprise kernel and it's too late to fix it for them.

Yes, in theory, sharing shared percpu data with local percpu data in one cache line can cause cache line contention between remote and local access.
And this undesirable sharing does happen in current git.

On x86, without the patch, runqueue shares cacheline with cpu_domains. c0633e80 d per_cpu__runqueues
c06347e0 d per_cpu__cpu_domains

Other architectures also have this undesired sharing. IA64 is an example.

In future, this undesired sharing could happen with other data if we do not have a methodology to prevent it.

I just do not have data to show the performance hit for this sharing now.

Thanks.

-Fenghua

Russell King

2007-05-25 09:43 UTC #15 lore raw

Re: [PATCH 1/2] Define new percpu interface for shared data -- version 3

Quoted 3 lines · Fenghua

Yes, in theory, sharing shared percpu data with local percpu data in one cache line can cause cache line contention between remote and local access.

What's "shared percpu data" ? It sounds to me like a contradiction in terms. Isn't percpu data supposed to only be accessed by the CPU which owns it to prevent cache line bouncing? In which case, what's the point of sharing that data with other CPUs? Surely "shared percpu data" is just the same as normal data?

Signature · 3 lines

Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:

Yu, Fenghua

2007-05-25 15:24 UTC #16 lore raw

RE: [PATCH 1/2] Define new percpu interface for shared data -- version 3

Quoted 3 lines

What's "shared percpu data" ? It sounds to me like a contradiction in terms. Isn't percpu data supposed to only be accessed by the CPU which owns it to prevent cache line bouncing? In which case, what's the

point

Quoted 2 lines

of sharing that data with other CPUs? Surely "shared percpu data" is just the same as normal data?

Like said in the patch description:
"per cpu data section contains two types of data. One set which is exclusively accessed by the local cpu and the other set which is per cpu,
but also shared by remote cpus."

The patches separate shared and aligned per cpu data from others to eliminate cache pouncing between remote and local access.

The patches do not separate shared but non-aligned percpu data, e.g. cpu_idle_state, flush_state in x86-64. We can do that later if needed.

I think the shared percpu data are "mainly" accessed by local cpu. We need
to concentrate on their local cpu access i.e. define in percpu. If we treat them as "normal" data, we may lose percpu benefits e.g. numa allocation.

Thanks.

-Fenghua

Martin Schwidefsky

2007-05-24 09:03 UTC #17 lore raw

Re: [PATCH 1/2] Define new percpu interface for shared data -- version 3

In reply to Ravikiran G Thirumalai

Quoted 16 lines · Ravikiran G Thirumalai

Quoted 16 lines
Quoted 3 lines
OK, but could we please have a concise description of the impact of these changes on kernel memory footprint? Increase or decrease? And by approximately how much?
Depending on how linker places percpu data, the patches could increase or decrease percpu section size. Data from 2.6.21-rc7-mm2:
On x86 SMP, the section size is increased from 0x7768 to 0x790c. 1.3% increase.
On X86-64 SMP, the size is decreased from 0x72d0 to 0x6540. 11.8% decrease.
On X86-64 VSMP, the size is increased from 0x72d0 to 0x8340. 14.3% increase.

Quoted 7 lines

Quoted 2 lines
On IA64 SMP, the size is decreased from 0x8370 to 0x7fc0. 2.8% decrease.
Has there been any measurable benefit yet due to tail padding? It would also be interesting to check the wastage/savings on another large
cache architecture like S390 (which has a 256 byte cache line)

Current git with the patches applied and the default configuration for s390 decreases the section size fof .data.percpu from 0x3e50 to 0x3e00. 0.5% decrease.

Signature · 4 lines

blue skies,
  Martin.

"Reality continues to ruin my life." - Calvin.

Ravikiran G Thirumalai

Acked-by

2007-05-24 17:17 UTC #18 lore raw

Re: [PATCH 1/2] Define new percpu interface for shared data -- version 3

Quoted 6 lines · Martin Schwidefsky

On Wed, 2007-05-23 at 11:57 -0700, Ravikiran G Thirumalai wrote:
Current git with the patches applied and the default configuration for s390 decreases the section size fof .data.percpu from 0x3e50 to 0x3e00. 0.5% decrease.

Thanks!!
Fenghua, you could use my 'Acked by' if needed

Acked-by Ravikiran Thirumalai <[email protected]>