Skip to contentSkip to conversation

: Call do_notify_resume() with interrupts enabled

RFC PATCH In review Srikar Dronamraju 6 participants 13 messages Oct 25, 2011 – Dec 6, 2011 x86

lore
Share & tools
lore
#0 lore raw

RFC PATCH 1/2 : Call do_notify_resume() with interrupts enabled

do_notify_resume() gets called with interrupts disabled on x86_32. This is different from the x86_64 behavior, where interrupts are enabled at the time.

Queries on lkml on this issue hasn't yielded any clear answer. Lets make x86_32 behave the same as x86_64, unless there is a real reason to maintain status quo.

Please refer https://lkml.org/lkml/2011/9/27/130 for more details

A similar change was suggested in arm

	https://lkml.org/lkml/2011/8/25/231

My 32-bit machine works fine (tm) with the patch below

---
 arch/x86/kernel/entry_32.S |    4 ++++
 1 files changed, 4 insertions(+)
arch/x86/kernel/entry_32.S +4 -0
index f3f6f53..22d0e21 100644
@@ -625,6 +625,8 @@ work_notifysig:				# deal with pending signals and
 	movl %esp, %eax
 	jne work_notifysig_v86		# returning to kernel-space or
 					# vm86-space
+	TRACE_IRQS_ON
+	ENABLE_INTERRUPTS(CLBR_NONE)
 	xorl %edx, %edx
 	call do_notify_resume
 	jmp resume_userspace_sig
@@ -638,6 +640,8 @@ work_notifysig:				# deal with pending signals and
 #else
 	movl %esp, %eax
 #endif
+	TRACE_IRQS_ON
+	ENABLE_INTERRUPTS(CLBR_NONE)
 	xorl %edx, %edx
 	call do_notify_resume
 	jmp resume_userspace_sig
#1 lore raw

RFC PATCH 2/2 Cleanup do_int3

Since there is a possibility of !KPROBES int3 listeners and since DIE_TRAP is currently not being used by anybody, notify all listeners with DIE_INT3.

---
 arch/x86/kernel/traps.c |    7 +------
 1 files changed, 1 insertions(+), 6 deletions(-)
arch/x86/kernel/traps.c +1 -6
index 6913369..c9a4be5 100644
@@ -461,15 +461,10 @@ dotraplinkage void __kprobes do_int3(struct pt_regs *regs, long error_code)
 			== NOTIFY_STOP)
 		return;
 #endif /* CONFIG_KGDB_LOW_LEVEL_TRAP */
-#ifdef CONFIG_KPROBES
+
 	if (notify_die(DIE_INT3, "int3", regs, error_code, 3, SIGTRAP)
 			== NOTIFY_STOP)
 		return;
-#else
-	if (notify_die(DIE_TRAP, "int3", regs, error_code, 3, SIGTRAP)
-			== NOTIFY_STOP)
-		return;
-#endif
 
 	preempt_conditional_sti(regs);
 	do_trap(3, SIGTRAP, "int3", regs, error_code, NULL);
#2 lore raw

Re: [RFC] [PATCH x86 2/2] Cleanup do_int3

Quoted 4 lines · Srikar Dronamraju

Since there is a possibility of !KPROBES int3 listeners and since DIE_TRAP is currently not being used by anybody, notify all listeners with DIE_INT3.

Obviously I can't ack this change, but it looks like the bugfix to me.

The current code uses DIE_INT3 or DIE_TRAP depending on CONFIG_KPROBES, this can't be right.

Quoted 28 lines
---
 arch/x86/kernel/traps.c |    7 +------
 1 files changed, 1 insertions(+), 6 deletions(-)
arch/x86/kernel/traps.c +1 -6
index 6913369..c9a4be5 100644
@@ -461,15 +461,10 @@ dotraplinkage void __kprobes do_int3(struct pt_regs *regs, long error_code)
 			== NOTIFY_STOP)
 		return;
 #endif /* CONFIG_KGDB_LOW_LEVEL_TRAP */
-#ifdef CONFIG_KPROBES
+
 	if (notify_die(DIE_INT3, "int3", regs, error_code, 3, SIGTRAP)
 			== NOTIFY_STOP)
 		return;
-#else
-	if (notify_die(DIE_TRAP, "int3", regs, error_code, 3, SIGTRAP)
-			== NOTIFY_STOP)
-		return;
-#endif
 
 	preempt_conditional_sti(regs);
 	do_trap(3, SIGTRAP, "int3", regs, error_code, NULL);
Acked-by on v1 2/2
#3 lore raw

Re: [RFC] [PATCH x86 2/2] Cleanup do_int3

In reply to Srikar Dronamraju
Quoted 3 lines · Srikar Dronamraju

Since there is a possibility of !KPROBES int3 listeners and since DIE_TRAP is currently not being used by anybody, notify all listeners with DIE_INT3.

I'm not sure why this CONFIG_KPROBES dependency had been introduced, but it's the time to fix it. do_int3 should send DIE_INT3 always.

Thank you!

Quoted 34 lines
---
 arch/x86/kernel/traps.c |    7 +------
 1 files changed, 1 insertions(+), 6 deletions(-)
arch/x86/kernel/traps.c +1 -7
index 6913369..c9a4be5 100644
@@ -461,15 +461,10 @@ dotraplinkage void __kprobes do_int3(struct pt_regs *regs, long error_code)
 			== NOTIFY_STOP)
 		return;
 #endif /* CONFIG_KGDB_LOW_LEVEL_TRAP */
-#ifdef CONFIG_KPROBES
+
 	if (notify_die(DIE_INT3, "int3", regs, error_code, 3, SIGTRAP)
 			== NOTIFY_STOP)
 		return;
-#else
-	if (notify_die(DIE_TRAP, "int3", regs, error_code, 3, SIGTRAP)
-			== NOTIFY_STOP)
-		return;
-#endif
 
 	preempt_conditional_sti(regs);
 	do_trap(3, SIGTRAP, "int3", regs, error_code, NULL);
 
--

To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Signature · 4 lines
Masami HIRAMATSU
Software Platform Research Dept. Linux Technology Center
Hitachi, Ltd., Yokohama Research Laboratory
E-mail: [email protected]
#4 lore raw

Re: [RFC] [PATCH x86 2/2] Cleanup do_int3

Quoted 6 lines
Quoted 3 lines · Srikar Dronamraju

Since there is a possibility of !KPROBES int3 listeners and since DIE_TRAP is currently not being used by anybody, notify all listeners with DIE_INT3.

Quoted 7 lines

I'm not sure why this CONFIG_KPROBES dependency had been introduced, but it's the time to fix it. do_int3 should send DIE_INT3 always.

Thank you!

Hi Ingo,

Is there a reason for not taking this patch?
Oleg, Masami, Alexander(who was the last to modify this code) have all been in favour of this patch.

I didnt get any response from you for the other patch in this series too.

--
Thanks and Regards
Srikar

Quoted 42 lines
Quoted 34 lines
---
 arch/x86/kernel/traps.c |    7 +------
 1 files changed, 1 insertions(+), 6 deletions(-)
arch/x86/kernel/traps.c +1 -7
index 6913369..c9a4be5 100644
@@ -461,15 +461,10 @@ dotraplinkage void __kprobes do_int3(struct pt_regs *regs, long error_code)
 			== NOTIFY_STOP)
 		return;
 #endif /* CONFIG_KGDB_LOW_LEVEL_TRAP */
-#ifdef CONFIG_KPROBES
+
 	if (notify_die(DIE_INT3, "int3", regs, error_code, 3, SIGTRAP)
 			== NOTIFY_STOP)
 		return;
-#else
-	if (notify_die(DIE_TRAP, "int3", regs, error_code, 3, SIGTRAP)
-			== NOTIFY_STOP)
-		return;
-#endif
 
 	preempt_conditional_sti(regs);
 	do_trap(3, SIGTRAP, "int3", regs, error_code, NULL);
 
--

To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Signature · 4 lines
Masami HIRAMATSU
Software Platform Research Dept. Linux Technology Center
Hitachi, Ltd., Yokohama Research Laboratory
E-mail: [email protected]
#5 lore raw

[tip:x86/asm] x86: Clean up and extend do_int3()

In reply to Srikar Dronamraju
Commit-ID:  cc3a1bf52a9d2808c7cd6e8f413b02b650b6b84b
Gitweb:     http://git.kernel.org/tip/cc3a1bf52a9d2808c7cd6e8f413b02b650b6b84b
Author:     Srikar Dronamraju <[email protected]>
AuthorDate: Tue, 25 Oct 2011 19:51:59 +0530
Committer:  Ingo Molnar <[email protected]>
CommitDate: Tue, 6 Dec 2011 08:20:37 +0100

x86: Clean up and extend do_int3()

Since there is a possibility of !KPROBES int3 listeners (such as kgdb) and since DIE_TRAP is currently not being used by anybody, notify all listeners with DIE_INT3.

---
 arch/x86/kernel/traps.c |    7 +------
 1 files changed, 1 insertions(+), 6 deletions(-)
arch/x86/kernel/traps.c +1 -6
index a8e3eb8..fa1191f 100644
@@ -306,15 +306,10 @@ dotraplinkage void __kprobes do_int3(struct pt_regs *regs, long error_code)
 			== NOTIFY_STOP)
 		return;
 #endif /* CONFIG_KGDB_LOW_LEVEL_TRAP */
-#ifdef CONFIG_KPROBES
+
 	if (notify_die(DIE_INT3, "int3", regs, error_code, 3, SIGTRAP)
 			== NOTIFY_STOP)
 		return;
-#else
-	if (notify_die(DIE_TRAP, "int3", regs, error_code, 3, SIGTRAP)
-			== NOTIFY_STOP)
-		return;
-#endif
 
 	preempt_conditional_sti(regs);
 	do_trap(3, SIGTRAP, "int3", regs, error_code, NULL);
#6 lore raw
In reply to Srikar Dronamraju
Quoted 4 lines · Srikar Dronamraju

do_notify_resume() gets called with interrupts disabled on x86_32. This is different from the x86_64 behavior, where interrupts are enabled at the time.

And note that do_notify_resume() paths assume that irqs are enabled. key_replace_session_keyring/get_signal_to_deliver start with _lock_irq.

IOW, I vote for this change even if I can't ack it (although it looks "obviously correct" to me). In fact refrigerator() looks buggy without this change. Yes, it enables irqs but only "by accident", via unlock_irq(). And we are going to remove this recalc_sigpending() from freezer.

Quoted 39 lines

Queries on lkml on this issue hasn't yielded any clear answer. Lets make x86_32 behave the same as x86_64, unless there is a real reason to maintain status quo.

Please refer https://lkml.org/lkml/2011/9/27/130 for more details

A similar change was suggested in arm

	https://lkml.org/lkml/2011/8/25/231

My 32-bit machine works fine (tm) with the patch below

---
 arch/x86/kernel/entry_32.S |    4 ++++
 1 files changed, 4 insertions(+)
arch/x86/kernel/entry_32.S +4 -0
index f3f6f53..22d0e21 100644
@@ -625,6 +625,8 @@ work_notifysig:				# deal with pending signals and
 	movl %esp, %eax
 	jne work_notifysig_v86		# returning to kernel-space or
 					# vm86-space
+	TRACE_IRQS_ON
+	ENABLE_INTERRUPTS(CLBR_NONE)
 	xorl %edx, %edx
 	call do_notify_resume
 	jmp resume_userspace_sig
@@ -638,6 +640,8 @@ work_notifysig:				# deal with pending signals and
 #else
 	movl %esp, %eax
 #endif
+	TRACE_IRQS_ON
+	ENABLE_INTERRUPTS(CLBR_NONE)
 	xorl %edx, %edx
 	call do_notify_resume
 	jmp resume_userspace_sig
#7 lore raw

Ingo, Thomas, I think this is your call, but it seems valid,

Linus

Quoted 56 lines · Oleg Nesterov
Quoted 4 lines · Srikar Dronamraju

do_notify_resume() gets called with interrupts disabled on x86_32. This is different from the x86_64 behavior, where interrupts are enabled at the time.

And note that do_notify_resume() paths assume that irqs are enabled. key_replace_session_keyring/get_signal_to_deliver start with _lock_irq.

IOW, I vote for this change even if I can't ack it (although it looks "obviously correct" to me). In fact refrigerator() looks buggy without this change. Yes, it enables irqs but only "by accident", via unlock_irq(). And we are going to remove this recalc_sigpending() from freezer.

Quoted 39 lines

Queries on lkml on this issue hasn't yielded any clear answer. Lets make x86_32 behave the same as x86_64, unless there is a real reason to maintain status quo.

Please refer https://lkml.org/lkml/2011/9/27/130 for more details

A similar change was suggested in arm
      https://lkml.org/lkml/2011/8/25/231

My 32-bit machine works fine (tm) with the patch below

---
 arch/x86/kernel/entry_32.S |    4 ++++

 1 files changed, 4 insertions(+)
arch/x86/kernel/entry_32.S +0 -0
index f3f6f53..22d0e21 100644
@@ -625,6 +625,8 @@ work_notifysig:                           # deal with pending signals and

      movl %esp, %eax
      jne work_notifysig_v86          # returning to kernel-space or

                                      # vm86-space
+     TRACE_IRQS_ON
+     ENABLE_INTERRUPTS(CLBR_NONE)
      xorl %edx, %edx
      call do_notify_resume
      jmp resume_userspace_sig
arch/x86/kernel/entry_32.S +0 -0
@@ -638,6 +640,8 @@ work_notifysig:                           # deal with pending signals and
  #else

      movl %esp, %eax
 #endif
+     TRACE_IRQS_ON
+     ENABLE_INTERRUPTS(CLBR_NONE)
      xorl %edx, %edx
      call do_notify_resume
      jmp resume_userspace_sig

#8 lore raw
Quoted 1 line · Linus Torvalds

Ingo, Thomas, I think this is your call, but it seems valid,

Hi Linus,

I guess I should've talked to you about this during a moment during the kernel summit, but as I'm now back home it'll have to be email.

I've been toying with a similar patch for ARM, but I keep feeling uneasy about having interrupts enabled in this path (even though they get enabled in the depths of the signal handling code.)

I worry about are race condition like the following:

syscall enter
...
syscall returns -ERESTARTNOHAND
check for signal
signal pending, but no handler, setup for restart interrupt happens, sets need_resched
need_resched set
switch to another thread
...
something happens which queues SIGIO
switch back to this thread
check for signal
signal pending, has handler, but we've setup for a restart
return to userspace
run SIGIO handler
restart syscall

This feels like it violates the expectations of the syscall being restarted - which explicitly asks to be restarted only if there wasn't a handler run.

I've been working on the assumption that this is a problem and we should do something about it - but it's non-trivial to solve all the corner cases. We can do a lot better with the restarting if we delay setting up for a restart until either we setup the user stack for the sig handler or immediately before returning to userspace (with a TIF flag.)

If you're interested in seeing where I got to, the patch is available at:

	https://lkml.org/lkml/2011/8/25/231

However, that doesn't solve the (probably unsolvable) case where an ERESTARTSYS syscall is interrupted by a SA_RESTART-marked handler, and while that handler is running it is then interrupted by a non-SA_RESTART- marked handler. I think that is far too an obscure case to care about though.

Signature · 3 lines
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:
#9 lore raw
Quoted 18 lines

I worry about are race condition like the following:

syscall enter
...
syscall returns -ERESTARTNOHAND
check for signal
signal pending, but no handler, setup for restart interrupt happens, sets need_resched
need_resched set
switch to another thread
...
something happens which queues SIGIO
switch back to this thread
check for signal
signal pending, has handler, but we've setup for a restart
return to userspace
run SIGIO handler
restart syscall

I don't think this is a violation. That ERESTARTNOHAND referred to restarting for the signal that interrupted that first syscall. That decision has already been made. You can consider that restart already to have begun before the second signal arrives. In essence, it's like the first syscall had never been entered, because the user state is already rolled back to where the syscall instruction is about to execute (correct me if I'm wrong about that). Then the second signal comes along and prevents that instruction from executing. It's really no different from if you had gotten all the way back to user mode, but not yet executed the first user instruction (i.e. the syscall instruction), when the second signal interrupted the user task.

Quoted 5 lines

However, that doesn't solve the (probably unsolvable) case where an ERESTARTSYS syscall is interrupted by a SA_RESTART-marked handler, and while that handler is running it is then interrupted by a non-SA_RESTART- marked handler. I think that is far too an obscure case to care about though.

I'm personally for caring about all cases, no matter how obscure. For that particular one, I don't think I see the problem. I'm probably overlooking something. ERESTARTSYS says that the user state should be rolled back to the syscall instruction before the handler setup. So it's just as if the signal had arrived before user mode executed the syscall instruction in the first place. That just affects the sigcontext saved before the handler runs. Nothing about that interacts with further interruptions during the handler. What am I missing?

However, we already have the fundamental constraint that ERESTART_RESTARTBLOCK states are lost when a handler runs.

Thanks,
Roland

#10 lore raw
In reply to Russell King
Quoted 7 lines · Russell King

I've been toying with a similar patch for ARM, but I keep feeling uneasy about having interrupts enabled in this path (even though they get enabled in the depths of the signal handling code.)

I worry about are race condition like the following:

Quoted 11 lines

syscall enter
...
syscall returns -ERESTARTNOHAND
check for signal
signal pending, but no handler, setup for restart interrupt happens, sets need_resched
need_resched set
switch to another thread
...
something happens which queues SIGIO switch back to this thread

I don't understand how "interrupts disabled" can help... A signal can come without preempt_schedule().

Quoted 9 lines

check for signal
signal pending, has handler, but we've setup for a restart
return to userspace
run SIGIO handler
restart syscall

This feels like it violates the expectations of the syscall being restarted - which explicitly asks to be restarted only if there wasn't a handler run.

But this doesn't differ from the case when this signal comes after the sycall was already restarted?

Quoted 5 lines

However, that doesn't solve the (probably unsolvable) case where an ERESTARTSYS syscall is interrupted by a SA_RESTART-marked handler, and while that handler is running it is then interrupted by a non-SA_RESTART- marked handler. I think that is far too an obscure case to care about though.

If I understand correctly, this was already discussed:

	 Re: HR timers prevent an itimer from generating EINTR?
	 http://marc.info/?l=linux-kernel&m=125384722012869

Unfortunately, marc.info doesn't show the authoritative reply from Roland, but he agreed with "not a problem".

Or I misunderstood?

Oleg.

#11 lore raw
In reply to Linus Torvalds
Quoted 4 lines

Ingo, Thomas, I think this is your call, but it seems valid,

Linus

Hey Ingo, Thomas,

Can you please let me know if you have taken a look at this patch and if you have any reservations?

I would also be happy to know your thoughts on the other patch in the series which makes sure that notify_die is called irrespective of kprobes.

Signature · 2 lines
Thanks and Regards
Srikar
Quoted 27 lines · Oleg Nesterov
Quoted 26 lines · Srikar Dronamraju
Quoted 4 lines

do_notify_resume() gets called with interrupts disabled on x86_32. This is different from the x86_64 behavior, where interrupts are enabled at the time.

And note that do_notify_resume() paths assume that irqs are enabled. key_replace_session_keyring/get_signal_to_deliver start with _lock_irq.

IOW, I vote for this change even if I can't ack it (although it looks "obviously correct" to me). In fact refrigerator() looks buggy without this change. Yes, it enables irqs but only "by accident", via unlock_irq(). And we are going to remove this recalc_sigpending() from freezer.

Quoted 11 lines

Queries on lkml on this issue hasn't yielded any clear answer. Lets make x86_32 behave the same as x86_64, unless there is a real reason to maintain status quo.

Please refer https://lkml.org/lkml/2011/9/27/130 for more details

A similar change was suggested in arm
      https://lkml.org/lkml/2011/8/25/231

My 32-bit machine works fine (tm) with the patch below

#12 lore raw

[tip:x86/asm] x86: Call do_notify_resume() with interrupts enabled

In reply to Srikar Dronamraju
Commit-ID:  3596ff4e6b2aff8a28c69af389d5046090a53330
Gitweb:     http://git.kernel.org/tip/3596ff4e6b2aff8a28c69af389d5046090a53330
Author:     Srikar Dronamraju <[email protected]>
AuthorDate: Tue, 25 Oct 2011 19:48:12 +0530
Committer:  Ingo Molnar <[email protected]>
CommitDate: Tue, 6 Dec 2011 08:20:34 +0100

x86: Call do_notify_resume() with interrupts enabled

do_notify_resume() gets called with interrupts disabled on x86_32. This is different from the x86_64 behavior, where interrupts are enabled at the time.

Queries on lkml on this issue hasn't yielded any clear answer. Lets make x86_32 behave the same as x86_64, unless there is a real reason to maintain status quo.

Please refer https://lkml.org/lkml/2011/9/27/130 for more details.

A similar change was suggested in ARM:

	https://lkml.org/lkml/2011/8/25/231

My 32-bit machine works fine (tm) with this patch.

---
 arch/x86/kernel/entry_32.S |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)
arch/x86/kernel/entry_32.S +4 -0
index f3f6f53..22d0e21 100644
@@ -625,6 +625,8 @@ work_notifysig:				# deal with pending signals and
 	movl %esp, %eax
 	jne work_notifysig_v86		# returning to kernel-space or
 					# vm86-space
+	TRACE_IRQS_ON
+	ENABLE_INTERRUPTS(CLBR_NONE)
 	xorl %edx, %edx
 	call do_notify_resume
 	jmp resume_userspace_sig
@@ -638,6 +640,8 @@ work_notifysig_v86:
 #else
 	movl %esp, %eax
 #endif
+	TRACE_IRQS_ON
+	ENABLE_INTERRUPTS(CLBR_NONE)
 	xorl %edx, %edx
 	call do_notify_resume
 	jmp resume_userspace_sig