Skip to contentSkip to conversation

Clear reserved bits of DR6 in do_debug()

RFC PATCH Superseded K.Prasad 3 participants 20 messages Dec 26, 2009 – Jan 27, 2010 1 message not in the archive x86

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

K.Prasad

author
#1 lore raw

RFC PATCH 1/2 Clear reserved bits of DR6 in do_debug()

Clear the reserved bits from the stored copy of debug status register (DR6). This will help easy bitwise operations.

---
 arch/x86/include/asm/debugreg.h |    3 +++
 arch/x86/kernel/traps.c         |    3 +++
 2 files changed, 6 insertions(+)
arch/x86/include/asm/debugreg.h +3 -0
Index: linux-2.6-tip/arch/x86/include/asm/debugreg.h
===================================================================
@@ -14,6 +14,9 @@
    which debugging register was responsible for the trap.  The other bits
    are either reserved or not of interest to us. */
 
+/* Define reserved bits in DR6 which are always set to 1 */
+#define DR6_RESERVED	(0xFFFF0FF0)
+
 #define DR_TRAP0	(0x1)		/* db0 */
 #define DR_TRAP1	(0x2)		/* db1 */
 #define DR_TRAP2	(0x4)		/* db2 */
arch/x86/kernel/traps.c +3 -0
Index: linux-2.6-tip/arch/x86/kernel/traps.c
===================================================================
@@ -534,6 +534,9 @@ dotraplinkage void __kprobes do_debug(st
 
 	get_debugreg(dr6, 6);
 
+	/* Filter out all the reserved bits which are preset to 1 */
+	dr6 &= ~DR6_RESERVED;
+
 	/* Catch kmemcheck conditions first of all! */
 	if ((dr6 & DR_STEP) && kmemcheck_trap(regs))
 		return;
#2 lore raw
Quoted 14 lines · K.Prasad

Clear the reserved bits from the stored copy of debug status register (DR6). This will help easy bitwise operations.

---
 arch/x86/include/asm/debugreg.h |    3 +++
 arch/x86/kernel/traps.c         |    3 +++
 2 files changed, 6 insertions(+)

Index: linux-2.6-tip/arch/x86/include/asm/debugreg.h
===================================================================

arch/x86/include/asm/debugreg.h line 19

Quoted 6 lines
arch/x86/include/asm/debugreg.h +3 -0
@@ -14,6 +14,9 @@
    which debugging register was responsible for the trap.  The other bits
    are either reserved or not of interest to us. */
 
+/* Define reserved bits in DR6 which are always set to 1 */
+#define DR6_RESERVED	(0xFFFF0FF0)
+

The 12th bit seems to be also reserved.
Shouldn't it be 0xffff1ff0 ?

What kind of bitwise operations do you think it could help?

All of the operations I can find on dr6 are simple masks test/set/clear.

Quoted 18 lines
arch/x86/include/asm/debugreg.h +0 -0
 #define DR_TRAP0	(0x1)		/* db0 */
 #define DR_TRAP1	(0x2)		/* db1 */
 #define DR_TRAP2	(0x4)		/* db2 */

Index: linux-2.6-tip/arch/x86/kernel/traps.c
===================================================================

arch/x86/kernel/traps.c +3 -0
@@ -534,6 +534,9 @@ dotraplinkage void __kprobes do_debug(st
 
 	get_debugreg(dr6, 6);
 
+	/* Filter out all the reserved bits which are preset to 1 */
+	dr6 &= ~DR6_RESERVED;
+
 	/* Catch kmemcheck conditions first of all! */
 	if ((dr6 & DR_STEP) && kmemcheck_trap(regs))
 		return;
author
#3 lore raw
Quoted 18 lines · Frederic Weisbecker
Quoted 17 lines · K.Prasad

Clear the reserved bits from the stored copy of debug status register (DR6). This will help easy bitwise operations.

---
 arch/x86/include/asm/debugreg.h |    3 +++
 arch/x86/kernel/traps.c         |    3 +++
 2 files changed, 6 insertions(+)

Index: linux-2.6-tip/arch/x86/include/asm/debugreg.h
===================================================================

arch/x86/include/asm/debugreg.h +0 -0
@@ -14,6 +14,9 @@
    which debugging register was responsible for the trap.  The other bits
    are either reserved or not of interest to us. */
 
Quoted 8 lines

arch/x86/include/asm/debugreg.h line 19

Quoted 3 lines
arch/x86/include/asm/debugreg.h +3 -0
+/* Define reserved bits in DR6 which are always set to 1 */
+#define DR6_RESERVED	(0xFFFF0FF0)
+

The 12th bit seems to be also reserved.
Shouldn't it be 0xffff1ff0 ?

The 12th bit is reserved to be 0 always.

Quoted 5 lines

What kind of bitwise operations do you think it could help?

All of the operations I can find on dr6 are simple masks test/set/clear.

As you found out later, this bitmask helps us in
hw_breakpoint_handler().

Thanks,
K.Prasad

#4 lore raw
Quoted 28 lines · K.Prasad
Quoted 18 lines · Frederic Weisbecker
Quoted 17 lines · K.Prasad

Clear the reserved bits from the stored copy of debug status register (DR6). This will help easy bitwise operations.

---
 arch/x86/include/asm/debugreg.h |    3 +++
 arch/x86/kernel/traps.c         |    3 +++
 2 files changed, 6 insertions(+)

Index: linux-2.6-tip/arch/x86/include/asm/debugreg.h
===================================================================

arch/x86/include/asm/debugreg.h +0 -0
@@ -14,6 +14,9 @@
    which debugging register was responsible for the trap.  The other bits
    are either reserved or not of interest to us. */
 
Quoted 8 lines

arch/x86/include/asm/debugreg.h line 19

Quoted 3 lines
arch/x86/include/asm/debugreg.h +3 -0
+/* Define reserved bits in DR6 which are always set to 1 */
+#define DR6_RESERVED	(0xFFFF0FF0)
+

The 12th bit seems to be also reserved.
Shouldn't it be 0xffff1ff0 ?

Quoted 1 line

The 12th bit is reserved to be 0 always.

Ah, ok.

Quoted 8 lines
Quoted 5 lines

What kind of bitwise operations do you think it could help?

All of the operations I can find on dr6 are simple masks test/set/clear.

As you found out later, this bitmask helps us in
hw_breakpoint_handler().

Yeah, ok. Just waiting for Jan's answer to be sure it has not side effects :)

Thanks.

author
#5 lore raw

RFC PATCH 2/2 Fix return-code to notifier chain in hw_breakpoint_handler

In reply to a message not in the archive

The hw-breakpoint handler will return NOTIFY_DONE for user-space breakpoints to generate SIGTRAP signal (and not for kernel-space addresses).

---
 arch/x86/kernel/hw_breakpoint.c |    9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)
arch/x86/kernel/hw_breakpoint.c +7 -2
Index: linux-2.6-tip/arch/x86/kernel/hw_breakpoint.c
===================================================================
@@ -502,8 +502,6 @@ static int __kprobes hw_breakpoint_handl
 		rcu_read_lock();
 
 		bp = per_cpu(bp_per_reg[i], cpu);
-		if (bp)
-			rc = NOTIFY_DONE;
 		/*
 		 * Reset the 'i'th TRAP bit in dr6 to denote completion of
 		 * exception handling
@@ -517,6 +515,13 @@ static int __kprobes hw_breakpoint_handl
 			rcu_read_unlock();
 			break;
 		}
+		/*
+		 * Further processing in do_debug() is needed for a) user-space
+		 * breakpoints (to generate signals) and b) when the system has
+		 * taken exception due to multiple causes
+		 */
+		if (bp->attr.bp_addr < TASK_SIZE)
+			rc = NOTIFY_DONE;
 
 		perf_bp_event(bp, args->regs);
 
#6 lore raw

Re: [RFC Patch 2/2][Bugfix][x86][hw-breakpoint] Fix return-code to notifier chain in hw_breakpoint_handler

Quoted 26 lines · K.Prasad

The hw-breakpoint handler will return NOTIFY_DONE for user-space breakpoints to generate SIGTRAP signal (and not for kernel-space addresses).

---
 arch/x86/kernel/hw_breakpoint.c |    9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

Index: linux-2.6-tip/arch/x86/kernel/hw_breakpoint.c
===================================================================

arch/x86/kernel/hw_breakpoint.c +0 -2
@@ -502,8 +502,6 @@ static int __kprobes hw_breakpoint_handl
 		rcu_read_lock();
 
 		bp = per_cpu(bp_per_reg[i], cpu);
-		if (bp)
-			rc = NOTIFY_DONE;
 		/*
 		 * Reset the 'i'th TRAP bit in dr6 to denote completion of
 		 * exception handling
@@ -517,6 +515,13 @@ static int __kprobes hw_breakpoint_handl
 			rcu_read_unlock();
 			break;
 		}

arch/x86/kernel/hw_breakpoint.c static int __kprobes hw_breakpoint_handl · line 527

Quoted 10 lines
arch/x86/kernel/hw_breakpoint.c +7 -0
+		/*
+		 * Further processing in do_debug() is needed for a) user-space
+		 * breakpoints (to generate signals) and b) when the system has
+		 * taken exception due to multiple causes
+		 */
+		if (bp->attr.bp_addr < TASK_SIZE)
+			rc = NOTIFY_DONE;
 
 		perf_bp_event(bp, args->regs);
 

Looks good. This stops any further checks though. This is fine in case we have the DR_STEP bit as we will toggle to NOTIFY_DONE. But what about vm86 case? Is it possible we might still have something to handle in this side? (although I don't quite understand how we can have a breakpoint triggered in vm86 mode).

#7 lore raw

Re: [RFC Patch 2/2][Bugfix][x86][hw-breakpoint] Fix return-code to notifier chain in hw_breakpoint_handler

In reply to K.Prasad
Quoted 26 lines · K.Prasad

The hw-breakpoint handler will return NOTIFY_DONE for user-space breakpoints to generate SIGTRAP signal (and not for kernel-space addresses).

---
 arch/x86/kernel/hw_breakpoint.c |    9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

Index: linux-2.6-tip/arch/x86/kernel/hw_breakpoint.c
===================================================================

arch/x86/kernel/hw_breakpoint.c +0 -2
@@ -502,8 +502,6 @@ static int __kprobes hw_breakpoint_handl
 		rcu_read_lock();
 
 		bp = per_cpu(bp_per_reg[i], cpu);
-		if (bp)
-			rc = NOTIFY_DONE;
 		/*
 		 * Reset the 'i'th TRAP bit in dr6 to denote completion of
 		 * exception handling
@@ -517,6 +515,13 @@ static int __kprobes hw_breakpoint_handl
 			rcu_read_unlock();
 			break;
 		}

arch/x86/kernel/hw_breakpoint.c static int __kprobes hw_breakpoint_handl · line 527

Quoted 10 lines
arch/x86/kernel/hw_breakpoint.c +7 -0
+		/*
+		 * Further processing in do_debug() is needed for a) user-space
+		 * breakpoints (to generate signals) and b) when the system has
+		 * taken exception due to multiple causes
+		 */
+		if (bp->attr.bp_addr < TASK_SIZE)
+			rc = NOTIFY_DONE;
 
 		perf_bp_event(bp, args->regs);
 

Oh and now that I see this patch, the previous one indeed makes sense with this check:

if (dr6 & (~DR_TRAP_BITS))

	rc = NOTIFY_DONE;

That said, it means thread.debugreg6 won't get the reserved bits anymore. I see some use of them from kvm (it restores the reserved bits on guest<->host switch). Not sure if this inconsistency could affect kvm...

author
#8 lore raw

Re: [RFC Patch 2/2][Bugfix][x86][hw-breakpoint] Fix return-code to notifier chain in hw_breakpoint_handler

Quoted 42 lines · Frederic Weisbecker
Quoted 26 lines · K.Prasad

The hw-breakpoint handler will return NOTIFY_DONE for user-space breakpoints to generate SIGTRAP signal (and not for kernel-space addresses).

---
 arch/x86/kernel/hw_breakpoint.c |    9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

Index: linux-2.6-tip/arch/x86/kernel/hw_breakpoint.c
===================================================================

arch/x86/kernel/hw_breakpoint.c +0 -2
@@ -502,8 +502,6 @@ static int __kprobes hw_breakpoint_handl
 		rcu_read_lock();
 
 		bp = per_cpu(bp_per_reg[i], cpu);
-		if (bp)
-			rc = NOTIFY_DONE;
 		/*
 		 * Reset the 'i'th TRAP bit in dr6 to denote completion of
 		 * exception handling
@@ -517,6 +515,13 @@ static int __kprobes hw_breakpoint_handl
 			rcu_read_unlock();
 			break;
 		}

arch/x86/kernel/hw_breakpoint.c static int __kprobes hw_breakpoint_handl · line 527

Quoted 10 lines
arch/x86/kernel/hw_breakpoint.c +7 -0
+		/*
+		 * Further processing in do_debug() is needed for a) user-space
+		 * breakpoints (to generate signals) and b) when the system has
+		 * taken exception due to multiple causes
+		 */
+		if (bp->attr.bp_addr < TASK_SIZE)
+			rc = NOTIFY_DONE;
 
 		perf_bp_event(bp, args->regs);
 

Oh and now that I see this patch, the previous one indeed makes sense with this check:

Quoted 7 lines

if (dr6 & (~DR_TRAP_BITS))

	rc = NOTIFY_DONE;

That said, it means thread.debugreg6 won't get the reserved bits anymore. I see some use of them from kvm (it restores the reserved bits on guest<->host switch). Not sure if this inconsistency could affect kvm...

Can you point me to the relevant code?
Anyway will copy this to Jan Kiszka <[email protected]> to hear what this change means to KVM...on a similar note, will be happy to be re-assured by Roland/Oleg about the patch's harmlessness to the user-space (ptrace/utrace).

Hi Jan,

	Patch [email protected] introduces a change that

Patch 1/2: Clears the arch-reserved bits from debug status register. This helps easy bitwise operations - such as the check for non-trap bits in hw_breakpoint_handler. A check for the same using
"if (dr6 & (~DR_TRAP_BITS))" throws incorrect results due to the presence of preset reserved bits.

Let us know if you foresee any harm from the said change to the behaviour seen under KVM.

Thanks,
K.Prasad

#9 lore raw

Re: [RFC Patch 2/2][Bugfix][x86][hw-breakpoint] Fix return-code to notifier chain in hw_breakpoint_handler

Quoted 46 lines · K.Prasad
Quoted 45 lines · Frederic Weisbecker
Quoted 26 lines · K.Prasad

The hw-breakpoint handler will return NOTIFY_DONE for user-space breakpoints to generate SIGTRAP signal (and not for kernel-space addresses).

---
 arch/x86/kernel/hw_breakpoint.c |    9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

Index: linux-2.6-tip/arch/x86/kernel/hw_breakpoint.c
===================================================================

arch/x86/kernel/hw_breakpoint.c +0 -2
@@ -502,8 +502,6 @@ static int __kprobes hw_breakpoint_handl
 		rcu_read_lock();
 
 		bp = per_cpu(bp_per_reg[i], cpu);
-		if (bp)
-			rc = NOTIFY_DONE;
 		/*
 		 * Reset the 'i'th TRAP bit in dr6 to denote completion of
 		 * exception handling
@@ -517,6 +515,13 @@ static int __kprobes hw_breakpoint_handl
 			rcu_read_unlock();
 			break;
 		}

arch/x86/kernel/hw_breakpoint.c static int __kprobes hw_breakpoint_handl · line 527

Quoted 10 lines
arch/x86/kernel/hw_breakpoint.c +7 -0
+		/*
+		 * Further processing in do_debug() is needed for a) user-space
+		 * breakpoints (to generate signals) and b) when the system has
+		 * taken exception due to multiple causes
+		 */
+		if (bp->attr.bp_addr < TASK_SIZE)
+			rc = NOTIFY_DONE;
 
 		perf_bp_event(bp, args->regs);
 

Oh and now that I see this patch, the previous one indeed makes sense with this check:

if (dr6 & (~DR_TRAP_BITS))

	rc = NOTIFY_DONE;
Quoted 6 lines
Quoted 4 lines

That said, it means thread.debugreg6 won't get the reserved bits anymore. I see some use of them from kvm (it restores the reserved bits on guest<->host switch). Not sure if this inconsistency could affect kvm...

Can you point me to the relevant code?

I see various uses of DR6_VOLATILE and DR6_FIXED_1 in arch/x86/kvm/, DR6_FIXED_1 being the fixed unused bits in dr6. Not sure how this patch would affect what's set there.

I'll wait for Jan's answer.

Thanks.

#10 lore raw

Re: [RFC Patch 2/2][Bugfix][x86][hw-breakpoint] Fix return-code to notifier chain in hw_breakpoint_handler

Quoted 50 lines · Frederic Weisbecker
Quoted 49 lines · K.Prasad
Quoted 48 lines · Frederic Weisbecker
Quoted 26 lines · K.Prasad

The hw-breakpoint handler will return NOTIFY_DONE for user-space breakpoints to generate SIGTRAP signal (and not for kernel-space addresses).

---
 arch/x86/kernel/hw_breakpoint.c |    9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

Index: linux-2.6-tip/arch/x86/kernel/hw_breakpoint.c
===================================================================

arch/x86/kernel/hw_breakpoint.c +0 -2
@@ -502,8 +502,6 @@ static int __kprobes hw_breakpoint_handl
 		rcu_read_lock();
 
 		bp = per_cpu(bp_per_reg[i], cpu);
-		if (bp)
-			rc = NOTIFY_DONE;
 		/*
 		 * Reset the 'i'th TRAP bit in dr6 to denote completion of
 		 * exception handling
@@ -517,6 +515,13 @@ static int __kprobes hw_breakpoint_handl
 			rcu_read_unlock();
 			break;
 		}

arch/x86/kernel/hw_breakpoint.c static int __kprobes hw_breakpoint_handl · line 527

Quoted 10 lines
arch/x86/kernel/hw_breakpoint.c +7 -0
+		/*
+		 * Further processing in do_debug() is needed for a) user-space
+		 * breakpoints (to generate signals) and b) when the system has
+		 * taken exception due to multiple causes
+		 */
+		if (bp->attr.bp_addr < TASK_SIZE)
+			rc = NOTIFY_DONE;
 
 		perf_bp_event(bp, args->regs);
 

Oh and now that I see this patch, the previous one indeed makes sense with this check:

if (dr6 & (~DR_TRAP_BITS))

	rc = NOTIFY_DONE;

That said, it means thread.debugreg6 won't get the reserved bits anymore. I see some use of them from kvm (it restores the reserved bits on guest<->host switch). Not sure if this inconsistency could affect kvm...

Quoted 9 lines
Quoted 1 line

Can you point me to the relevant code?

I see various uses of DR6_VOLATILE and DR6_FIXED_1 in arch/x86/kvm/, DR6_FIXED_1 being the fixed unused bits in dr6. Not sure how this patch would affect what's set there.

I'll wait for Jan's answer.

You may need to synchronize me: What does the patch change, the shadow register KVM will restore into DR6 on return to the host? Or the register content KVM finds on guest entry?

The rules are simple: On entry, KVM assumes nothing about the register state, just overwrites it (on demand) with the guest state. On exit, it calls into hw_breakpoint_restore to ensure the host sees a proper state (if required). But there is at no time an architecturally invalid state loaded into the real register (that's basically what DR6_VOLATILE and DR6_FIXED_1 are used for while in guest mode).

Jan

Signature · 2 lines
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux
author
#11 lore raw

Re: [RFC Patch 2/2][Bugfix][x86][hw-breakpoint] Fix return-code to notifier chain in hw_breakpoint_handler

Quoted 58 lines · Jan Kiszka
Quoted 57 lines · Frederic Weisbecker
Quoted 42 lines · K.Prasad
Quoted 41 lines · Frederic Weisbecker
Quoted 26 lines · K.Prasad

The hw-breakpoint handler will return NOTIFY_DONE for user-space breakpoints to generate SIGTRAP signal (and not for kernel-space addresses).

---
 arch/x86/kernel/hw_breakpoint.c |    9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

Index: linux-2.6-tip/arch/x86/kernel/hw_breakpoint.c
===================================================================

arch/x86/kernel/hw_breakpoint.c +0 -2
@@ -502,8 +502,6 @@ static int __kprobes hw_breakpoint_handl
 		rcu_read_lock();
 
 		bp = per_cpu(bp_per_reg[i], cpu);
-		if (bp)
-			rc = NOTIFY_DONE;
 		/*
 		 * Reset the 'i'th TRAP bit in dr6 to denote completion of
 		 * exception handling
@@ -517,6 +515,13 @@ static int __kprobes hw_breakpoint_handl
 			rcu_read_unlock();
 			break;
 		}

arch/x86/kernel/hw_breakpoint.c static int __kprobes hw_breakpoint_handl · line 527

Quoted 10 lines
arch/x86/kernel/hw_breakpoint.c +7 -0
+		/*
+		 * Further processing in do_debug() is needed for a) user-space
+		 * breakpoints (to generate signals) and b) when the system has
+		 * taken exception due to multiple causes
+		 */
+		if (bp->attr.bp_addr < TASK_SIZE)
+			rc = NOTIFY_DONE;
 
 		perf_bp_event(bp, args->regs);
 

Oh and now that I see this patch, the previous one indeed makes sense with this check:

Quoted 8 lines
Quoted 7 lines

if (dr6 & (~DR_TRAP_BITS))

	rc = NOTIFY_DONE;

That said, it means thread.debugreg6 won't get the reserved bits anymore. I see some use of them from kvm (it restores the reserved bits on guest<->host switch). Not sure if this inconsistency could affect kvm...

Can you point me to the relevant code?

I see various uses of DR6_VOLATILE and DR6_FIXED_1 in arch/x86/kvm/, DR6_FIXED_1 being the fixed unused bits in dr6. Not sure how this patch would affect what's set there.

Quoted 7 lines
Quoted 2 lines

I'll wait for Jan's answer.

You may need to synchronize me: What does the patch change, the shadow register KVM will restore into DR6 on return to the host? Or the register content KVM finds on guest entry?

Sorry, this mail got buried deeply in my mailbox (hence the delay).

Basically, this patch tries to remove DR6 from its reserved bits to help easy checks for certain status bits (such as DR_STEP). For instance, in order to verify if DR_STEP (Bit 14) is set we must now do
if ((DR6 & ~DR6_RESERVED) & DR_STEP) {}
or
if (DR6 & (DR_STEP | DR6_RESERVED)) {}
which is redundant.

Instead this patch would expunge all reserved bits in DR6 before checks for various status bits (to detect the cause of exception) are made in do_debug().

At the outset, I don't think changes in the way the value of DR6 is used for comparison in do_debug() would affect exception handling for either KVM's guest or host OS (given that there are no hooks for the same in do_debug()).

Quoted 7 lines

The rules are simple: On entry, KVM assumes nothing about the register state, just overwrites it (on demand) with the guest state. On exit, it calls into hw_breakpoint_restore to ensure the host sees a proper state (if required). But there is at no time an architecturally invalid state loaded into the real register (that's basically what DR6_VOLATILE and DR6_FIXED_1 are used for while in guest mode).

Such a behaviour shouldn't be affected by the above change...your confirmation would help!

Thanks,
K.Prasad

author
#12 lore raw

Re: [RFC Patch 2/2][Bugfix][x86][hw-breakpoint] Fix return-code to notifier chain in hw_breakpoint_handler

Quoted 94 lines · K.Prasad
Quoted 58 lines · Jan Kiszka
Quoted 57 lines · Frederic Weisbecker
Quoted 42 lines · K.Prasad
Quoted 41 lines · Frederic Weisbecker
Quoted 26 lines · K.Prasad

The hw-breakpoint handler will return NOTIFY_DONE for user-space breakpoints to generate SIGTRAP signal (and not for kernel-space addresses).

---
 arch/x86/kernel/hw_breakpoint.c |    9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

Index: linux-2.6-tip/arch/x86/kernel/hw_breakpoint.c
===================================================================

arch/x86/kernel/hw_breakpoint.c +0 -2
@@ -502,8 +502,6 @@ static int __kprobes hw_breakpoint_handl
 		rcu_read_lock();
 
 		bp = per_cpu(bp_per_reg[i], cpu);
-		if (bp)
-			rc = NOTIFY_DONE;
 		/*
 		 * Reset the 'i'th TRAP bit in dr6 to denote completion of
 		 * exception handling
@@ -517,6 +515,13 @@ static int __kprobes hw_breakpoint_handl
 			rcu_read_unlock();
 			break;
 		}

arch/x86/kernel/hw_breakpoint.c static int __kprobes hw_breakpoint_handl · line 527

Quoted 10 lines
arch/x86/kernel/hw_breakpoint.c +7 -0
+		/*
+		 * Further processing in do_debug() is needed for a) user-space
+		 * breakpoints (to generate signals) and b) when the system has
+		 * taken exception due to multiple causes
+		 */
+		if (bp->attr.bp_addr < TASK_SIZE)
+			rc = NOTIFY_DONE;
 
 		perf_bp_event(bp, args->regs);
 

Oh and now that I see this patch, the previous one indeed makes sense with this check:

Quoted 8 lines
Quoted 7 lines

if (dr6 & (~DR_TRAP_BITS))

	rc = NOTIFY_DONE;

That said, it means thread.debugreg6 won't get the reserved bits anymore. I see some use of them from kvm (it restores the reserved bits on guest<->host switch). Not sure if this inconsistency could affect kvm...

Can you point me to the relevant code?

I see various uses of DR6_VOLATILE and DR6_FIXED_1 in arch/x86/kvm/, DR6_FIXED_1 being the fixed unused bits in dr6. Not sure how this patch would affect what's set there.

Quoted 7 lines
Quoted 2 lines

I'll wait for Jan's answer.

You may need to synchronize me: What does the patch change, the shadow register KVM will restore into DR6 on return to the host? Or the register content KVM finds on guest entry?

Sorry, this mail got buried deeply in my mailbox (hence the delay).

Basically, this patch tries to remove DR6 from its reserved bits to help easy checks for certain status bits (such as DR_STEP). For instance, in order to verify if DR_STEP (Bit 14) is set we must now do
if ((DR6 & ~DR6_RESERVED) & DR_STEP) {}
or
if (DR6 & (DR_STEP | DR6_RESERVED)) {}
which is redundant.

Instead this patch would expunge all reserved bits in DR6 before checks for various status bits (to detect the cause of exception) are made in do_debug().

At the outset, I don't think changes in the way the value of DR6 is used for comparison in do_debug() would affect exception handling for either KVM's guest or host OS (given that there are no hooks for the same in do_debug()).

Quoted 7 lines

The rules are simple: On entry, KVM assumes nothing about the register state, just overwrites it (on demand) with the guest state. On exit, it calls into hw_breakpoint_restore to ensure the host sees a proper state (if required). But there is at no time an architecturally invalid state loaded into the real register (that's basically what DR6_VOLATILE and DR6_FIXED_1 are used for while in guest mode).

Quoted 3 lines

Such a behaviour shouldn't be affected by the above change...your confirmation would help!

Hi Jan,
I presume that the above explanation makes the role of this
patch/bugfix clear.

Kindly let me know if you have any further queries.

Thanks,
K.Prasad

#13 lore raw

Re: [RFC Patch 2/2][Bugfix][x86][hw-breakpoint] Fix return-code to notifier chain in hw_breakpoint_handler

Quoted 94 lines · K.Prasad
Quoted 89 lines · K.Prasad
Quoted 56 lines · Jan Kiszka
Quoted 55 lines · Frederic Weisbecker
Quoted 41 lines · K.Prasad
Quoted 40 lines · Frederic Weisbecker
Quoted 26 lines · K.Prasad

The hw-breakpoint handler will return NOTIFY_DONE for user-space breakpoints to generate SIGTRAP signal (and not for kernel-space addresses).

---
 arch/x86/kernel/hw_breakpoint.c |    9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

Index: linux-2.6-tip/arch/x86/kernel/hw_breakpoint.c
===================================================================

arch/x86/kernel/hw_breakpoint.c +0 -2
@@ -502,8 +502,6 @@ static int __kprobes hw_breakpoint_handl
 		rcu_read_lock();
 
 		bp = per_cpu(bp_per_reg[i], cpu);
-		if (bp)
-			rc = NOTIFY_DONE;
 		/*
 		 * Reset the 'i'th TRAP bit in dr6 to denote completion of
 		 * exception handling
@@ -517,6 +515,13 @@ static int __kprobes hw_breakpoint_handl
 			rcu_read_unlock();
 			break;
 		}

arch/x86/kernel/hw_breakpoint.c static int __kprobes hw_breakpoint_handl · line 527

Quoted 10 lines
arch/x86/kernel/hw_breakpoint.c +7 -0
+		/*
+		 * Further processing in do_debug() is needed for a) user-space
+		 * breakpoints (to generate signals) and b) when the system has
+		 * taken exception due to multiple causes
+		 */
+		if (bp->attr.bp_addr < TASK_SIZE)
+			rc = NOTIFY_DONE;
 
 		perf_bp_event(bp, args->regs);
 

Oh and now that I see this patch, the previous one indeed makes sense with this check:

Quoted 8 lines
Quoted 7 lines

if (dr6 & (~DR_TRAP_BITS))

	rc = NOTIFY_DONE;

That said, it means thread.debugreg6 won't get the reserved bits anymore. I see some use of them from kvm (it restores the reserved bits on guest<->host switch). Not sure if this inconsistency could affect kvm...

Can you point me to the relevant code?

I see various uses of DR6_VOLATILE and DR6_FIXED_1 in arch/x86/kvm/, DR6_FIXED_1 being the fixed unused bits in dr6. Not sure how this patch would affect what's set there.

Quoted 6 lines
Quoted 2 lines

I'll wait for Jan's answer.

You may need to synchronize me: What does the patch change, the shadow register KVM will restore into DR6 on return to the host? Or the register content KVM finds on guest entry?

Sorry, this mail got buried deeply in my mailbox (hence the delay).

Basically, this patch tries to remove DR6 from its reserved bits to help easy checks for certain status bits (such as DR_STEP). For instance, in order to verify if DR_STEP (Bit 14) is set we must now do
if ((DR6 & ~DR6_RESERVED) & DR_STEP) {}
or
if (DR6 & (DR_STEP | DR6_RESERVED)) {}
which is redundant.

Instead this patch would expunge all reserved bits in DR6 before checks for various status bits (to detect the cause of exception) are made in do_debug().

At the outset, I don't think changes in the way the value of DR6 is used for comparison in do_debug() would affect exception handling for either KVM's guest or host OS (given that there are no hooks for the same in do_debug()).

Quoted 7 lines

The rules are simple: On entry, KVM assumes nothing about the register state, just overwrites it (on demand) with the guest state. On exit, it calls into hw_breakpoint_restore to ensure the host sees a proper state (if required). But there is at no time an architecturally invalid state loaded into the real register (that's basically what DR6_VOLATILE and DR6_FIXED_1 are used for while in guest mode).

Quoted 3 lines

Such a behaviour shouldn't be affected by the above change...your confirmation would help!

Quoted 6 lines

Hi Jan,
I presume that the above explanation makes the role of this
patch/bugfix clear.

Kindly let me know if you have any further queries.

Nope. There should be really no conflicts of your optimization with kvm.

Jan

Signature · 2 lines
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux
author
#14 lore raw

Re: [RFC Patch 2/2][Bugfix][x86][hw-breakpoint] Fix return-code to notifier chain in hw_breakpoint_handler

Quoted 10 lines · Jan Kiszka
Quoted 9 lines · K.Prasad
Quoted 8 lines · K.Prasad
Quoted 7 lines · Jan Kiszka
Quoted 6 lines · Frederic Weisbecker
Quoted 5 lines · K.Prasad
Quoted 4 lines · Frederic Weisbecker
Quoted 3 lines · K.Prasad

The hw-breakpoint handler will return NOTIFY_DONE for user-space breakpoints to generate SIGTRAP signal (and not for kernel-space addresses).

<snipped>

Quoted 8 lines
Quoted 8 lines
Quoted 3 lines

Such a behaviour shouldn't be affected by the above change...your confirmation would help!

Hi Jan,
I presume that the above explanation makes the role of this
patch/bugfix clear.

Quoted 8 lines
Quoted 2 lines

Kindly let me know if you have any further queries.

Nope. There should be really no conflicts of your optimization with kvm.

Jan

Hi Jan,
Thanks for the confirmation.

Hi Frederic,
Can you pull these fixes in? (LKML references:
[email protected] and [email protected]).

Thanks,
K.Prasad

#15 lore raw

Re: [RFC Patch 2/2][Bugfix][x86][hw-breakpoint] Fix return-code to notifier chain in hw_breakpoint_handler

Quoted 15 lines · K.Prasad
Quoted 15 lines
Quoted 10 lines
Quoted 3 lines

Such a behaviour shouldn't be affected by the above change...your confirmation would help!

Hi Jan,
I presume that the above explanation makes the role of this
patch/bugfix clear.

Kindly let me know if you have any further queries.

Nope. There should be really no conflicts of your optimization with kvm.

Jan

Quoted 8 lines
Quoted 1 line

Hi Jan,
Thanks for the confirmation.

Hi Frederic,
Can you pull these fixes in? (LKML references:
[email protected] and [email protected]).

I got a deeper look at these patches and answered with some comments.

If you address these, I can apply for .34 (because it's about optimizations and not fixes).

Thanks.

author
#16 lore raw

Re: [RFC Patch 2/2][Bugfix][x86][hw-breakpoint] Fix return-code to notifier chain in hw_breakpoint_handler

Quoted 26 lines · Frederic Weisbecker
Quoted 15 lines · K.Prasad
Quoted 15 lines
Quoted 10 lines
Quoted 3 lines

Such a behaviour shouldn't be affected by the above change...your confirmation would help!

Hi Jan,
I presume that the above explanation makes the role of this
patch/bugfix clear.

Kindly let me know if you have any further queries.

Nope. There should be really no conflicts of your optimization with kvm.

Jan

Quoted 8 lines
Quoted 1 line

Hi Jan,
Thanks for the confirmation.

Hi Frederic,
Can you pull these fixes in? (LKML references:
[email protected] and [email protected]).

Quoted 5 lines

I got a deeper look at these patches and answered with some comments.

If you address these, I can apply for .34 (because it's about optimizations and not fixes).

Sure, please apply them against a version that you deem appropriate.

Thanks,
K.Prasad

#17 lore raw

Re: [RFC Patch 2/2][Bugfix][x86][hw-breakpoint] Fix return-code to notifier chain in hw_breakpoint_handler

In reply to K.Prasad

arch/x86/kernel/hw_breakpoint.c static int __kprobes hw_breakpoint_handl · line 524

Quoted 7 lines · K.Prasad
arch/x86/kernel/hw_breakpoint.c +7 -0
+		/*
+		 * Further processing in do_debug() is needed for a) user-space
+		 * breakpoints (to generate signals) and b) when the system has
+		 * taken exception due to multiple causes
+		 */
+		if (bp->attr.bp_addr < TASK_SIZE)
+			rc = NOTIFY_DONE;

BTW, I'm not sure this is the right way to check if we want to send a signal or not. Although it's not yet supported, we'll probably bring the support for userspace breakpoints by perf.

May be we should put a "ptrace" flag in struct hw_perf_event instead.

#18 lore raw

Re: [RFC Patch 2/2][Bugfix][x86][hw-breakpoint] Fix return-code to notifier chain in hw_breakpoint_handler

In reply to K.Prasad
Quoted 2 lines · K.Prasad

The hw-breakpoint handler will return NOTIFY_DONE for user-space breakpoints to generate SIGTRAP signal (and not for kernel-space addresses).

Please tell a bit more in your changelogs. It took me some time to guess whether this is a fix or not.

And this is not a fix but an optimization because SIGTRAP is only sent if needed.

Here is what happens in do_debug() after handling the breakpoint:

	if (tsk->thread.debugreg6 & (DR_STEP | DR_TRAP_BITS))
		send_sigtrap(tsk, regs, error_code, si_code);

This can only happen if we took the ptrace handler path.

Also:

Quoted 19 lines
---
 arch/x86/kernel/hw_breakpoint.c |    9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

Index: linux-2.6-tip/arch/x86/kernel/hw_breakpoint.c
===================================================================

arch/x86/kernel/hw_breakpoint.c +0 -2
@@ -502,8 +502,6 @@ static int __kprobes hw_breakpoint_handl
 		rcu_read_lock();
 
 		bp = per_cpu(bp_per_reg[i], cpu);
-		if (bp)
-			rc = NOTIFY_DONE;
 		/*
 		 * Reset the 'i'th TRAP bit in dr6 to denote completion of
 		 * exception handling

arch/x86/kernel/hw_breakpoint.c static int __kprobes hw_breakpoint_handl · line 524

Quoted 10 lines
arch/x86/kernel/hw_breakpoint.c +7 -0
@@ -517,6 +515,13 @@ static int __kprobes hw_breakpoint_handl
 			rcu_read_unlock();
 			break;
 		}
+		/*
+		 * Further processing in do_debug() is needed for a) user-space
+		 * breakpoints (to generate signals) and b) when the system has
+		 * taken exception due to multiple causes
+		 */
+		if (bp->attr.bp_addr < TASK_SIZE)
+			rc = NOTIFY_DONE;

Is that < TASK_SIZE an accurate check? We want support for userspace breakpoints on perf tools later, and those don't want signals.

We do this cleanup in the beginning of the breakpoint handler:

	current->thread.debugreg6 &= ~DR_TRAP_BITS;

And from ptrace.c:ptrace_triggered():

	thread->debugreg6 |= (DR_TRAP0 << i);

This is called on perf_bp_event().
Instead of checking if this is a userspace thread, we should actually check if this is a ptrace breakpoint by looking at this
in the end of hw_breakpoint_handler().

	current->thread.debugreg6 & DR_TRAP_BITS

Only ptrace breakpoints require signals.

Thanks.

author
#19 lore raw

Re: [RFC Patch 2/2][Bugfix][x86][hw-breakpoint] Fix return-code to notifier chain in hw_breakpoint_handler

Quoted 8 lines · Frederic Weisbecker
Quoted 2 lines · K.Prasad

The hw-breakpoint handler will return NOTIFY_DONE for user-space breakpoints to generate SIGTRAP signal (and not for kernel-space addresses).

Please tell a bit more in your changelogs. It took me some time to guess whether this is a fix or not.

Sorry about that...will add a descriptive changelog.

Quoted 11 lines

And this is not a fix but an optimization because SIGTRAP is only sent if needed.

Here is what happens in do_debug() after handling the breakpoint:

	if (tsk->thread.debugreg6 & (DR_STEP | DR_TRAP_BITS))
		send_sigtrap(tsk, regs, error_code, si_code);

This can only happen if we took the ptrace handler path.

Agreed...signals are prevented as above...except that the notifier semantics aren't properly used (NOTIFY_DONE vs NOTIFY_STOP).

Quoted 28 lines

Also:

Quoted 25 lines
---
 arch/x86/kernel/hw_breakpoint.c |    9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

Index: linux-2.6-tip/arch/x86/kernel/hw_breakpoint.c
===================================================================

arch/x86/kernel/hw_breakpoint.c +3 -2
@@ -502,8 +502,6 @@ static int __kprobes hw_breakpoint_handl
 		rcu_read_lock();
 
 		bp = per_cpu(bp_per_reg[i], cpu);
-		if (bp)
-			rc = NOTIFY_DONE;
 		/*
 		 * Reset the 'i'th TRAP bit in dr6 to denote completion of
 		 * exception handling
@@ -517,6 +515,13 @@ static int __kprobes hw_breakpoint_handl
 			rcu_read_unlock();
 			break;
 		}
+		/*
+		 * Further processing in do_debug() is needed for a) user-space
+		 * breakpoints (to generate signals) and b) when the system has
Quoted 9 lines

arch/x86/kernel/hw_breakpoint.c static int __kprobes hw_breakpoint_handl · line 524

Quoted 4 lines
arch/x86/kernel/hw_breakpoint.c +4 -0
+		 * taken exception due to multiple causes
+		 */
+		if (bp->attr.bp_addr < TASK_SIZE)
+			rc = NOTIFY_DONE;

Is that < TASK_SIZE an accurate check? We want support for userspace breakpoints on perf tools later, and those don't want signals.

Well, signal generation for user-space breakpoints happened unconditionally for 'historical' reasons (guess that Alan Stern's original patch had it that way).

We could change that into a 'ptrace-only' signal generation now.

Quoted 8 lines

We do this cleanup in the beginning of the breakpoint handler:

	current->thread.debugreg6 &= ~DR_TRAP_BITS;

And from ptrace.c:ptrace_triggered():

	thread->debugreg6 |= (DR_TRAP0 << i);
Quoted 9 lines

This is called on perf_bp_event().
Instead of checking if this is a userspace thread, we should actually check if this is a ptrace breakpoint by looking at this
in the end of hw_breakpoint_handler().

	current->thread.debugreg6 & DR_TRAP_BITS

Only ptrace breakpoints require signals.

Yes, this does look like a clean way to limit signals to those requests that are interested (I was looking at round-about ways like doing a lookup based on callback functions).

I will send the next version of the patch with the above changes.

Thanks,
K.Prasad

#20 lore raw

Re: [RFC Patch 2/2][Bugfix][x86][hw-breakpoint] Fix return-code to notifier chain in hw_breakpoint_handler

Quoted 11 lines · K.Prasad
Quoted 4 lines · Frederic Weisbecker

Is that < TASK_SIZE an accurate check? We want support for userspace breakpoints on perf tools later, and those don't want signals.

Well, signal generation for user-space breakpoints happened unconditionally for 'historical' reasons (guess that Alan Stern's original patch had it that way).

We could change that into a 'ptrace-only' signal generation now.

Yeah, now that we can have multiple-purpose concurrent breakpoints, this is necessary.

Quoted 15 lines
Quoted 15 lines

We do this cleanup in the beginning of the breakpoint handler:

	current->thread.debugreg6 &= ~DR_TRAP_BITS;

And from ptrace.c:ptrace_triggered():

	thread->debugreg6 |= (DR_TRAP0 << i);

This is called on perf_bp_event().
Instead of checking if this is a userspace thread, we should actually check if this is a ptrace breakpoint by looking at this
in the end of hw_breakpoint_handler().

	current->thread.debugreg6 & DR_TRAP_BITS
Quoted 8 lines
Quoted 2 lines

Only ptrace breakpoints require signals.

Yes, this does look like a clean way to limit signals to those requests that are interested (I was looking at round-about ways like doing a lookup based on callback functions).

I will send the next version of the patch with the above changes.

Thanks.