Skip to contentSkip to conversation

uprobes/core: Clean up, refactor and improve the code

tip:perf/uprobes tip-bot for Ingo Molnar 2 participants 8 messages Feb 17, 2012 – Feb 20, 2012

Subsystems (4)
lore
Share & tools
lore
#0 lore raw
Commit-ID:  7b2d81d48a2d8e37efb6ce7b4d5ef58822b30d89
Gitweb:     http://git.kernel.org/tip/7b2d81d48a2d8e37efb6ce7b4d5ef58822b30d89
Author:     Ingo Molnar <[email protected]>
AuthorDate: Fri, 17 Feb 2012 09:27:41 +0100
Committer:  Ingo Molnar <[email protected]>
CommitDate: Fri, 17 Feb 2012 10:18:07 +0100

uprobes/core: Clean up, refactor and improve the code

Make the uprobes code readable to me:

- improve the Kconfig text so that a mere mortal gets some idea what CONFIG_UPROBES=y is really about

- do trivial renames to standardize around the uprobes_*() namespace

- clean up and simplify various code flow details

- separate basic blocks of functionality

- line break artifact and white space related removal

- use standard local varible definition blocks

- use vertical spacing to make things more readable

- remove unnecessary volatile

- restructure comment blocks to make them more uniform and more readable in general

Cc Srikar Dronamraju <[email protected]>
Cc Jim Keniston <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Oleg Nesterov <[email protected]>
Cc Masami Hiramatsu <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Anton Arapov <[email protected]>
Cc Ananth N Mavinakayanahalli <[email protected]>
---
 arch/Kconfig                   |   14 ++-
 arch/x86/include/asm/uprobes.h |   17 ++--
 arch/x86/kernel/uprobes.c      |  129 ++++++++++++-----------
 include/linux/uprobes.h        |   28 +++---
 kernel/uprobes.c               |  219 +++++++++++++++++++++++-----------------
 mm/mmap.c                      |   12 +-
 6 files changed, 233 insertions(+), 186 deletions(-)
arch/Kconfig +10 -4
index 284f589..cca5b54 100644
@@ -66,13 +66,19 @@ config OPTPROBES
 	depends on !PREEMPT
 
 config UPROBES
-	bool "User-space probes (EXPERIMENTAL)"
+	bool "Transparent user-space probes (EXPERIMENTAL)"
 	depends on ARCH_SUPPORTS_UPROBES
 	default n
 	help
-	  Uprobes enables kernel subsystems to establish probepoints
-	  in user applications and execute handler functions when
-	  the probepoints are hit.
+	  Uprobes is the user-space counterpart to kprobes: they
+	  enable instrumentation applications (such as 'perf probe')
+	  to establish unintrusive probes in user-space binaries and
+	  libraries, by executing handler functions when the probes
+	  are hit by user-space applications.
+
+	  ( These probes come in the form of single-byte breakpoints,
+	    managed by the kernel and kept transparent to the probed
+	    application. )
 
 	  If in doubt, say "N".
 
arch/x86/include/asm/uprobes.h +9 -8
index 8208234..072df39 100644
@@ -1,7 +1,7 @@
 #ifndef _ASM_UPROBES_H
 #define _ASM_UPROBES_H
 /*
- * Userspace Probes (UProbes) for x86
+ * User-space Probes (UProbes) for x86
  *
  * This program is free software; you can redistribute it and/or modify
  * it under the terms of the GNU General Public License as published by
@@ -24,19 +24,20 @@
  */
 
 typedef u8 uprobe_opcode_t;
-#define MAX_UINSN_BYTES 16
-#define UPROBES_XOL_SLOT_BYTES	128	/* to keep it cache aligned */
 
-#define UPROBES_BKPT_INSN 0xcc
-#define UPROBES_BKPT_INSN_SIZE 1
+#define MAX_UINSN_BYTES			  16
+#define UPROBES_XOL_SLOT_BYTES		 128	/* to keep it cache aligned */
+
+#define UPROBES_BKPT_INSN		0xcc
+#define UPROBES_BKPT_INSN_SIZE		   1
 
 struct uprobe_arch_info {
-	u16			fixups;
+	u16				fixups;
 #ifdef CONFIG_X86_64
-	unsigned long rip_rela_target_address;
+	unsigned long			rip_rela_target_address;
 #endif
 };
 
 struct uprobe;
-extern int analyze_insn(struct mm_struct *mm, struct uprobe *uprobe);
+extern int arch_uprobes_analyze_insn(struct mm_struct *mm, struct uprobe *uprobe);
 #endif	/* _ASM_UPROBES_H */
arch/x86/kernel/uprobes.c +68 -61

250 lines not on this page · raw message

include/linux/uprobes.h +13 -15
index f1d13fd..64e45f1 100644
@@ -1,7 +1,7 @@
 #ifndef _LINUX_UPROBES_H
 #define _LINUX_UPROBES_H
 /*
- * Userspace Probes (UProbes)
+ * User-space Probes (UProbes)
  *
  * This program is free software; you can redistribute it and/or modify
  * it under the terms of the GNU General Public License as published by
@@ -40,8 +40,10 @@ struct uprobe_arch_info {};
 #define uprobe_opcode_sz sizeof(uprobe_opcode_t)
 
 /* flags that denote/change uprobes behaviour */
+
 /* Have a copy of original instruction */
 #define UPROBES_COPY_INSN	0x1
+
 /* Dont run handlers when first register/ last unregister in progress*/
 #define UPROBES_RUN_HANDLER	0x2
 
@@ -70,27 +72,23 @@ struct uprobe {
 };
 
 #ifdef CONFIG_UPROBES
-extern int __weak set_bkpt(struct mm_struct *mm, struct uprobe *uprobe,
-							unsigned long vaddr);
-extern int __weak set_orig_insn(struct mm_struct *mm, struct uprobe *uprobe,
-					unsigned long vaddr, bool verify);
+extern int __weak set_bkpt(struct mm_struct *mm, struct uprobe *uprobe, unsigned long vaddr);
+extern int __weak set_orig_insn(struct mm_struct *mm, struct uprobe *uprobe, unsigned long vaddr, bool verify);
 extern bool __weak is_bkpt_insn(uprobe_opcode_t *insn);
-extern int register_uprobe(struct inode *inode, loff_t offset,
-				struct uprobe_consumer *consumer);
-extern void unregister_uprobe(struct inode *inode, loff_t offset,
-				struct uprobe_consumer *consumer);
-extern int mmap_uprobe(struct vm_area_struct *vma);
+extern int uprobe_register(struct inode *inode, loff_t offset, struct uprobe_consumer *consumer);
+extern void uprobe_unregister(struct inode *inode, loff_t offset, struct uprobe_consumer *consumer);
+extern int uprobe_mmap(struct vm_area_struct *vma);
 #else /* CONFIG_UPROBES is not defined */
-static inline int register_uprobe(struct inode *inode, loff_t offset,
-				struct uprobe_consumer *consumer)
+static inline int
+uprobe_register(struct inode *inode, loff_t offset, struct uprobe_consumer *consumer)
 {
 	return -ENOSYS;
 }
-static inline void unregister_uprobe(struct inode *inode, loff_t offset,
-				struct uprobe_consumer *consumer)
+static inline void
+uprobe_unregister(struct inode *inode, loff_t offset, struct uprobe_consumer *consumer)
 {
 }
-static inline int mmap_uprobe(struct vm_area_struct *vma)
+static inline int uprobe_mmap(struct vm_area_struct *vma)
 {
 	return 0;
 }
kernel/uprobes.c +127 -92

610 lines not on this page · raw message

mm/mmap.c +6 -6
index 1aed183..5a863d3 100644
@@ -618,10 +618,10 @@ again:			remove_next = 1 + (end > next->vm_end);
 		mutex_unlock(&mapping->i_mmap_mutex);
 
 	if (root) {
-		mmap_uprobe(vma);
+		uprobe_mmap(vma);
 
 		if (adjust_next)
-			mmap_uprobe(next);
+			uprobe_mmap(next);
 	}
 
 	if (remove_next) {
@@ -646,7 +646,7 @@ again:			remove_next = 1 + (end > next->vm_end);
 		}
 	}
 	if (insert && file)
-		mmap_uprobe(insert);
+		uprobe_mmap(insert);
 
 	validate_mm(mm);
 
@@ -1340,7 +1340,7 @@ out:
 	} else if ((flags & MAP_POPULATE) && !(flags & MAP_NONBLOCK))
 		make_pages_present(addr, addr + len);
 
-	if (file && mmap_uprobe(vma))
+	if (file && uprobe_mmap(vma))
 		/* matching probes but cannot insert */
 		goto unmap_and_free_vma;
 
@@ -2301,7 +2301,7 @@ int insert_vm_struct(struct mm_struct * mm, struct vm_area_struct * vma)
 	     security_vm_enough_memory_mm(mm, vma_pages(vma)))
 		return -ENOMEM;
 
-	if (vma->vm_file && mmap_uprobe(vma))
+	if (vma->vm_file && uprobe_mmap(vma))
 		return -EINVAL;
 
 	vma_link(mm, vma, prev, rb_link, rb_parent);
@@ -2374,7 +2374,7 @@ struct vm_area_struct *copy_vma(struct vm_area_struct **vmap,
 			if (new_vma->vm_file) {
 				get_file(new_vma->vm_file);
 
-				if (mmap_uprobe(new_vma))
+				if (uprobe_mmap(new_vma))
 					goto out_free_mempol;
 
 				if (vma->vm_flags & VM_EXECUTABLE)
#1 lore raw

Srikar,

Quoted 41 lines · tip-bot for Ingo Molnar
Commit-ID:  7b2d81d48a2d8e37efb6ce7b4d5ef58822b30d89
Gitweb:     http://git.kernel.org/tip/7b2d81d48a2d8e37efb6ce7b4d5ef58822b30d89
Author:     Ingo Molnar <[email protected]>
AuthorDate: Fri, 17 Feb 2012 09:27:41 +0100
Committer:  Ingo Molnar <[email protected]>
CommitDate: Fri, 17 Feb 2012 10:18:07 +0100

uprobes/core: Clean up, refactor and improve the code

Make the uprobes code readable to me:

- improve the Kconfig text so that a mere mortal gets some idea what CONFIG_UPROBES=y is really about

- do trivial renames to standardize around the uprobes_*() namespace

- clean up and simplify various code flow details

- separate basic blocks of functionality

- line break artifact and white space related removal

- use standard local varible definition blocks

- use vertical spacing to make things more readable

- remove unnecessary volatile

- restructure comment blocks to make them more uniform and more readable in general

Cc Srikar Dronamraju <[email protected]>
Cc Jim Keniston <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Oleg Nesterov <[email protected]>
Cc Masami Hiramatsu <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Anton Arapov <[email protected]>
Cc Ananth N Mavinakayanahalli <[email protected]>
Quoted 8 lines
---
 arch/Kconfig                   |   14 ++-
 arch/x86/include/asm/uprobes.h |   17 ++--
 arch/x86/kernel/uprobes.c      |  129 ++++++++++++-----------
 include/linux/uprobes.h        |   28 +++---
 kernel/uprobes.c               |  219 +++++++++++++++++++++++-----------------
 mm/mmap.c                      |   12 +-
 6 files changed, 233 insertions(+), 186 deletions(-)

FYI, I created a new -tip topic branch for uprobes commits [tip:perf/uprobes], with two patches applied so far:

- your v10 #1/9 patch
- the above clean-ups and restructurings above it

( Please have a good look at the code flow clean-ups I have
performed, I have only build-tested the result. )

As you can see it from the diffstat, there were many small details that hindered code cleanliness, and there's at least one bigger item that still needs to be addressed, which I have not done in the above patch:

The arch methods and structures need to be better separated from core uprobes code. Right now there's uprobes.h which exports 'struct uprobe' to the rest of the kernel, such as arch/x86/kernel/uprobes.c.

What we want instead is a clean separation of core code from architecture code:

- 'struct arch_uprobe' which includes all fields that 'struct uprobe_arch_info' contains today. In the first approximation this is a rename.

- please rename uprobe->arch_info to uprobe->arch - it's already clear that it's information.

- uprobe->insn[] needs to move from struct uprobe to uprobe->arch.insn

- The uprobes_arch_*() method(s) should be passed a 'struct arch_uprobe *', not a 'struct uprobe *'.

- Once this is done, 'struct uprobe' can move to the head of kernel/uprobes.c, without any ugly #ifdefs and wrappery - that code only compiles if uprobes are enabled and if the architecture supports it.

- asm/uprobes.h defines 'struct arch_uprobe' and the arch method(s) - nothing else.

- unless uprobe_opcode_t is non-byte on any architecture this typedef should probably be go away and be char * or void *.

- write_opcode() and any similar functions should be renamed to the arch_uprobes_write_opcode() pattern

These changes should be done on top of latest -tip, in a single patch or a small series, it's mostly pretty straightforward.

Please keep an eye on stylistic details for future patches - the above patch gives a laundry list of what details were wrong in the existing code.

Once this is done can we add #2, #3, etc. patches in a clean manner.

I'd suggest you don't send a full series but send small, edible steps on top of the minimal (and right now not yet functional) uprobes core code I've already applied, in series of 1-3 patches.

I'm sure there will be other small details we'll have to change in these patches as we go forward, so not porting the full series straight away would save you time and effort that the inevitable conflicts create.

As an additional benefit, if you do it in such a workflow then I can apply your new patches to tip:perf/uprobes pretty much the moment you send them out, reviewing and testing them quickly - without the time-consuming review of a long and interdependent patch series and the resulting churn on your side.

I believe we could iterate this through in a couple of days and arrive to a working 'perf probe' prototype.

Thanks,

Ingo

#4 lore raw
In reply to tip-bot for Ingo Molnar
Quoted 2 lines

- remove unnecessary volatile

volatiles were added because of warnings thrown by gcc-4.6. Please see below.

Quoted 4 lines

- restructure comment blocks to make them more uniform and more readable in general

...

Quoted 13 lines
arch/Kconfig +1 -1
index 284f589..cca5b54 100644
@@ -66,13 +66,19 @@ config OPTPROBES
 	depends on !PREEMPT
 
 config UPROBES
-	bool "User-space probes (EXPERIMENTAL)"
+	bool "Transparent user-space probes (EXPERIMENTAL)"
 	depends on ARCH_SUPPORTS_UPROBES
 	default n
 	help

arch/Kconfig config OPTPROBES · line 79

Quoted 10 lines
arch/Kconfig +7 -3
-	  Uprobes enables kernel subsystems to establish probepoints
-	  in user applications and execute handler functions when
-	  the probepoints are hit.
+	  Uprobes is the user-space counterpart to kprobes: they
+	  enable instrumentation applications (such as 'perf probe')
+	  to establish unintrusive probes in user-space binaries and
+	  libraries, by executing handler functions when the probes
+	  are hit by user-space applications.
+
+	  ( These probes come in the form of single-byte breakpoints,

One nit: In some architectures like powerpc, the breakpoints arent single-byte.

arch/Kconfig

Quoted 9 lines
arch/Kconfig +2 -0
+	    managed by the kernel and kept transparent to the probed
+	    application. )
 
 	  If in doubt, say "N".
 

#ifdef CONFIG_X86_64

arch/Kconfig +1 -1
-static volatile u32 good_insns_64[256 / 32] = {
+static u32 good_insns_64[256 / 32] = {

The volatiles were added to arch/x86/kernel/kprobes.c because of commit 7115e3fcf45 and 315eb8a2a1b. The volatiles are required because gcc 4.6 gave a warning about the asm operand for test_bit. So the same were added to arch/x86/kernel/uprobes.c.

Signature · 2 lines
Thanks and Regards
Srikar
#5 lore raw
Quoted 5 lines · Srikar Dronamraju

The volatiles were added to arch/x86/kernel/kprobes.c because of commit 7115e3fcf45 and 315eb8a2a1b. The volatiles are required because gcc 4.6 gave a warning about the asm operand for test_bit. So the same were added to arch/x86/kernel/uprobes.c.

Seems like a GCC bug - a bogus warning - or does it generate bad code as well?

In any case, kprobes.c did it correctly, it added the volatile *and a comment*, pointing out that it's a GCC bug. No such warning was added to uprobes.c, making the volatile look entirely spurious.

So feel free to re-add the volatile in a followup patch, just make sure the GCC workaround nature is documented.

Thanks,

Ingo

#2 lore raw
In reply to Ingo Molnar
Quoted 9 lines

FYI, I created a new -tip topic branch for uprobes commits [tip:perf/uprobes], with two patches applied so far:

- your v10 #1/9 patch
- the above clean-ups and restructurings above it

( Please have a good look at the code flow clean-ups I have
performed, I have only build-tested the result. )

Okay, Will do accordingly.

Quoted 14 lines

As you can see it from the diffstat, there were many small details that hindered code cleanliness, and there's at least one bigger item that still needs to be addressed, which I have not done in the above patch:

The arch methods and structures need to be better separated from core uprobes code. Right now there's uprobes.h which exports 'struct uprobe' to the rest of the kernel, such as arch/x86/kernel/uprobes.c.

What we want instead is a clean separation of core code from architecture code:

Quoted 7 lines

- 'struct arch_uprobe' which includes all fields that 'struct uprobe_arch_info' contains today. In the first approximation this is a rename.

- please rename uprobe->arch_info to uprobe->arch - it's already clear that it's information.

Okay.

Quoted 11 lines

- uprobe->insn[] needs to move from struct uprobe to uprobe->arch.insn

- The uprobes_arch_*() method(s) should be passed a 'struct arch_uprobe *', not a 'struct uprobe *'.

- Once this is done, 'struct uprobe' can move to the head of kernel/uprobes.c, without any ugly #ifdefs and wrappery - that code only compiles if uprobes are enabled and if the architecture supports it.

Quoted 5 lines

- asm/uprobes.h defines 'struct arch_uprobe' and the arch method(s) - nothing else.

- write_opcode() and any similar functions should be renamed to the arch_uprobes_write_opcode() pattern

Currently the kernel/uprobes.c code handles insn as arch agnostic in some cases and uses arch specific stuff for analysis, verification and to set up fixups. The analysis, verification, and fixups is only done at the probe insertion only.

The copy_insn code, write_opcode is mostly arch agnostic except for the maximum length of any supported instruction for that architecture. If we move the insn to arch_uprobe, then we would have to duplicate this code in arch specific files to do the copying of the instruction. (not only at registration/unregistration times and also at probe hit time to copy into the slot).
So I am still tempted to keep insn in struct uprobe rather than arch_uprobe.

Quoted 4 lines

- unless uprobe_opcode_t is non-byte on any architecture this typedef should probably be go away and be char * or void *.

For architectures like powerpc, uprobe_opcode_t would be non-byte. But I think we could still manage to remove uprobe_opcode_t.

Quoted 10 lines

These changes should be done on top of latest -tip, in a single patch or a small series, it's mostly pretty straightforward.

Please keep an eye on stylistic details for future patches - the above patch gives a laundry list of what details were wrong in the existing code.

Once this is done can we add #2, #3, etc. patches in a clean manner.

Okay.

Quoted 9 lines

I'd suggest you don't send a full series but send small, edible steps on top of the minimal (and right now not yet functional) uprobes core code I've already applied, in series of 1-3 patches.

I'm sure there will be other small details we'll have to change in these patches as we go forward, so not porting the full series straight away would save you time and effort that the inevitable conflicts create.

Okay.

Quoted 9 lines

As an additional benefit, if you do it in such a workflow then I can apply your new patches to tip:perf/uprobes pretty much the moment you send them out, reviewing and testing them quickly - without the time-consuming review of a long and interdependent patch series and the resulting churn on your side.

I believe we could iterate this through in a couple of days and arrive to a working 'perf probe' prototype.

Okay.

Signature · 2 lines
Thanks and Regards
Srikar
#6 lore raw
In reply to Ingo Molnar
Quoted 11 lines · Ingo Molnar
Quoted 5 lines · Srikar Dronamraju

The volatiles were added to arch/x86/kernel/kprobes.c because of commit 7115e3fcf45 and 315eb8a2a1b. The volatiles are required because gcc 4.6 gave a warning about the asm operand for test_bit. So the same were added to arch/x86/kernel/uprobes.c.

Seems like a GCC bug - a bogus warning - or does it generate bad code as well?

Yes it is a gcc bug and was fixed by Jakub.
As per Josh, only the first long is output if compiled on the buggy gcc.

Quoted 5 lines

In any case, kprobes.c did it correctly, it added the volatile *and a comment*, pointing out that it's a GCC bug. No such warning was added to uprobes.c, making the volatile look entirely spurious.

okay.

Quoted 8 lines

So feel free to re-add the volatile in a followup patch, just make sure the GCC workaround nature is documented.

Thanks,

Ingo

#3 lore raw
In reply to Srikar Dronamraju
Quoted 23 lines · Srikar Dronamraju
Quoted 11 lines

- uprobe->insn[] needs to move from struct uprobe to uprobe->arch.insn

- The uprobes_arch_*() method(s) should be passed a 'struct arch_uprobe *', not a 'struct uprobe *'.

- Once this is done, 'struct uprobe' can move to the head of kernel/uprobes.c, without any ugly #ifdefs and wrappery - that code only compiles if uprobes are enabled and if the architecture supports it.

Quoted 5 lines

- asm/uprobes.h defines 'struct arch_uprobe' and the arch method(s) - nothing else.

- write_opcode() and any similar functions should be renamed to the arch_uprobes_write_opcode() pattern

Currently the kernel/uprobes.c code handles insn as arch agnostic in some cases and uses arch specific stuff for analysis, verification and to set up fixups. The analysis, verification, and fixups is only done at the probe insertion only.

Quoted 7 lines

The copy_insn code, write_opcode is mostly arch agnostic except for the maximum length of any supported instruction for that architecture. If we move the insn to arch_uprobe, then we would have to duplicate this code in arch specific files to do the copying of the instruction. (not only at registration/unregistration times and also at probe hit time to copy into the slot).

Is there any reason why the core kernel uprobes.c code could not use uprobe->arch.insn directly?

It's in the architecture specific structure, mainly to encapsulate architecture-accessible fields and isolate low-level functionality from high-level one. This does not preclude the high-level code from using that field though.

Obviously every uprobes supporting architecture would have to define an 'insn' field in their 'struct arch_uprobe'.

Thanks,

Ingo

#7 lore raw
In reply to Srikar Dronamraju
Quoted 11 lines · Srikar Dronamraju
Quoted 9 lines · Ingo Molnar
Quoted 5 lines · Srikar Dronamraju

The volatiles were added to arch/x86/kernel/kprobes.c because of commit 7115e3fcf45 and 315eb8a2a1b. The volatiles are required because gcc 4.6 gave a warning about the asm operand for test_bit. So the same were added to arch/x86/kernel/uprobes.c.

Quoted 5 lines
Quoted 2 lines

Seems like a GCC bug - a bogus warning - or does it generate bad code as well?

Yes it is a gcc bug and was fixed by Jakub.
As per Josh, only the first long is output if compiled on the buggy gcc.

That's an important piece of information - the more reason to document the quirk.

Quoted 6 lines
Quoted 4 lines

In any case, kprobes.c did it correctly, it added the volatile *and a comment*, pointing out that it's a GCC bug. No such warning was added to uprobes.c, making the volatile look entirely spurious.

okay.

Thanks,

Ingo