kprobe fast unregistration

Hi Ananth,

Here is a series of patches which introduce two kinds of interfaces for speeding up unregistering process of kprobes.

Currently, the unregister_*probe() function takes a long time to unregister specified probe because it waits the rcu synchronization after each unregistration. So we need to introduce new method for unregistering a lot of probes at once.

I'd like to suggest introducing following two interfaces for this issue. The first interface is unregister_*probe_fast(). This removes breakpoint instruction from kernel code and adds specified probe to the unregistering list.
The second interface is commit_kprobes(). This waits the rcu synchronization and clean up each probe on the unregistering list. This patch also adds a list_head member to the kprobe data structure for linking to the unregistering list.

If using these interfaces, the probe handlers of unregistering probes might be called after unregister_*probe_fast() is called. So, you MUST call commit_kprobes() after calling unregisnter_*probe_fast() for all probes.

It is safe even if several threads unregister probes simultaneously, because the unregistration process is protected by kprobe_lock mutex. If one thread calls commit_kprobes(), it will cleanup not only its probes but also the other thread's probes. And it is transparently done. If there is no unregistering probes, commit_kprobes() do nothing.

Here is an example code.
--
struct kprobes *p;

for_each_probe(p) {
	unregister_kprobe_fast(p);
}

commit_kprobes();
--

Best regards,

---
This patch introduces unregister_kprobe_fast() and commit_kprobes().

 arch/i386/kernel/kprobes.c    |    2
 arch/ia64/kernel/kprobes.c    |    2
 arch/powerpc/kernel/kprobes.c |    2
 arch/s390/kernel/kprobes.c    |    2
 arch/x86_64/kernel/kprobes.c  |    2
 include/linux/kprobes.h       |   11 +++
 kernel/kprobes.c              |  138 +++++++++++++++++++++++++++++++-----------
 7 files changed, 115 insertions(+), 44 deletions(-)

Index: linux-2.6.21-rc4-mm1/include/linux/kprobes.h
===================================================================
@@ -68,6 +68,9 @@ struct kprobe {
 	/* list of kprobes for multi-handler support */
 	struct list_head list;
 
+	/* list of kprobes for fast unregistering support */
+	struct list_head commit_list;
+
 	/* Indicates that the corresponding module has been ref counted */
 	unsigned int mod_refcounted;
 

@@ -190,6 +193,8 @@ static inline struct kprobe_ctlblk *get_
 
 int register_kprobe(struct kprobe *p);
 void unregister_kprobe(struct kprobe *p);
+void unregister_kprobe_fast(struct kprobe *p);
+void commit_kprobes(void);
 int setjmp_pre_handler(struct kprobe *, struct pt_regs *);
 int longjmp_break_handler(struct kprobe *, struct pt_regs *);
 int register_jprobe(struct jprobe *p);

@@ -220,6 +225,9 @@ static inline int register_kprobe(struct
 static inline void unregister_kprobe(struct kprobe *p)
 {
 }
+static inline void unregister_kprobe_fast(struct kprobe *p)
+{
+}
 static inline int register_jprobe(struct jprobe *p)
 {
 	return -ENOSYS;

@@ -240,5 +248,8 @@ static inline void unregister_kretprobe(
 static inline void kprobe_flush_task(struct task_struct *tk)
 {
 }
+static inline void commit_kprobes(void)
+{
+}
 #endif				/* CONFIG_KPROBES */
 #endif				/* _LINUX_KPROBES_H */

Index: linux-2.6.21-rc4-mm1/arch/i386/kernel/kprobes.c
===================================================================
@@ -183,9 +183,7 @@ void __kprobes arch_disarm_kprobe(struct
 
 void __kprobes arch_remove_kprobe(struct kprobe *p)
 {
-	mutex_lock(&kprobe_mutex);
 	free_insn_slot(p->ainsn.insn, (p->ainsn.boostable == 1));
-	mutex_unlock(&kprobe_mutex);
 }
 
 static void __kprobes save_previous_kprobe(struct kprobe_ctlblk *kcb)

Index: linux-2.6.21-rc4-mm1/arch/ia64/kernel/kprobes.c
===================================================================
@@ -570,9 +570,7 @@ void __kprobes arch_disarm_kprobe(struct
 
 void __kprobes arch_remove_kprobe(struct kprobe *p)
 {
-	mutex_lock(&kprobe_mutex);
 	free_insn_slot(p->ainsn.insn, 0);
-	mutex_unlock(&kprobe_mutex);
 }
 /*
  * We are resuming execution after a single step fault, so the pt_regs

Index: linux-2.6.21-rc4-mm1/arch/powerpc/kernel/kprobes.c
===================================================================
@@ -84,9 +84,7 @@ void __kprobes arch_disarm_kprobe(struct
 
 void __kprobes arch_remove_kprobe(struct kprobe *p)
 {
-	mutex_lock(&kprobe_mutex);
 	free_insn_slot(p->ainsn.insn, 0);
-	mutex_unlock(&kprobe_mutex);
 }
 
 static void __kprobes prepare_singlestep(struct kprobe *p, struct pt_regs *regs)

Index: linux-2.6.21-rc4-mm1/arch/s390/kernel/kprobes.c
===================================================================
@@ -218,9 +218,7 @@ void __kprobes arch_disarm_kprobe(struct
 
 void __kprobes arch_remove_kprobe(struct kprobe *p)
 {
-	mutex_lock(&kprobe_mutex);
 	free_insn_slot(p->ainsn.insn, 0);
-	mutex_unlock(&kprobe_mutex);
 }
 
 static void __kprobes prepare_singlestep(struct kprobe *p, struct pt_regs *regs)

Index: linux-2.6.21-rc4-mm1/arch/x86_64/kernel/kprobes.c
===================================================================
@@ -223,9 +223,7 @@ void __kprobes arch_disarm_kprobe(struct
 
 void __kprobes arch_remove_kprobe(struct kprobe *p)
 {
-	mutex_lock(&kprobe_mutex);
 	free_insn_slot(p->ainsn.insn, 0);
-	mutex_unlock(&kprobe_mutex);
 }
 
 static void __kprobes save_previous_kprobe(struct kprobe_ctlblk *kcb)

This patch introduces unregister_kretprobe_fast() for kretprobe.

---

 include/linux/kprobes.h |   12 +++++++++++-
 kernel/kprobes.c        |   44 ++++++++++++++++++++++++++++----------------
 2 files changed, 39 insertions(+), 17 deletions(-)

Index: linux-2.6.21-rc4-mm1/include/linux/kprobes.h
===================================================================
@@ -202,7 +202,14 @@ void unregister_jprobe(struct jprobe *p)
 void jprobe_return(void);
 
 int register_kretprobe(struct kretprobe *rp);
-void unregister_kretprobe(struct kretprobe *rp);
+static inline void unregister_kretprobe(struct kretprobe *rp)
+{
+	unregister_kprobe(&rp->kp);
+}
+static inline void unregister_kretprobe_fast(struct kretprobe *rp)
+{
+	unregister_kprobe_fast(&rp->kp);
+}
 
 struct kretprobe_instance *get_free_rp_inst(struct kretprobe *rp);
 void add_rp_inst(struct kretprobe_instance *ri);

@@ -245,6 +252,9 @@ static inline int register_kretprobe(str
 static inline void unregister_kretprobe(struct kretprobe *rp)
 {
 }
+static inline void unregister_kretprobe_fast(struct kretprobe *rp)
+{
+}
 static inline void kprobe_flush_task(struct task_struct *tk)
 {
 }

This patch introduces unregister_jprobe_fast() for jprobe.

---
This patch introduces unregister_jprobe_fast() for jprobe.

 include/linux/kprobes.h |   12 +++++++++++-
 kernel/kprobes.c        |    6 ------
 2 files changed, 11 insertions(+), 7 deletions(-)

Index: linux-2.6.21-rc4-mm1/include/linux/kprobes.h
===================================================================
@@ -198,7 +198,14 @@ void commit_kprobes(void);
 int setjmp_pre_handler(struct kprobe *, struct pt_regs *);
 int longjmp_break_handler(struct kprobe *, struct pt_regs *);
 int register_jprobe(struct jprobe *p);
-void unregister_jprobe(struct jprobe *p);
+static inline void unregister_jprobe(struct jprobe *jp)
+{
+	unregister_kprobe(&jp->kp);
+}
+static inline void unregister_jprobe_fast(struct jprobe *jp)
+{
+	unregister_kprobe_fast(&jp->kp);
+}
 void jprobe_return(void);
 
 int register_kretprobe(struct kretprobe *rp);

@@ -242,6 +249,9 @@ static inline int register_jprobe(struct
 static inline void unregister_jprobe(struct jprobe *p)
 {
 }
+static inline void unregister_jprobe_fast(struct jprobe *p)
+{
+}
 static inline void jprobe_return(void)
 {
 }

Index: linux-2.6.21-rc4-mm1/kernel/kprobes.c
===================================================================
@@ -797,11 +797,6 @@ int __kprobes register_jprobe(struct jpr
 		(unsigned long)__builtin_return_address(0));
 }
 
-void __kprobes unregister_jprobe(struct jprobe *jp)
-{
-	unregister_kprobe(&jp->kp);
-}
-
 #ifdef ARCH_SUPPORTS_KRETPROBES
 
 /*

@@ -1013,6 +1008,5 @@ EXPORT_SYMBOL_GPL(unregister_kprobe);
 EXPORT_SYMBOL_GPL(unregister_kprobe_fast);
 EXPORT_SYMBOL_GPL(commit_kprobes);
 EXPORT_SYMBOL_GPL(register_jprobe);
-EXPORT_SYMBOL_GPL(unregister_jprobe);
 EXPORT_SYMBOL_GPL(jprobe_return);
 EXPORT_SYMBOL_GPL(register_kretprobe);

This patch adds the description of the fast unregisteration interfaces.

---

 Documentation/kprobes.txt |   30 ++++++++++++++++++++++++++++++
 1 file changed, 30 insertions(+)

Index: linux-2.6.21-rc4-mm1/Documentation/kprobes.txt
===================================================================
@@ -36,6 +36,15 @@ registration function such as register_k
 the probe is to be inserted and what handler is to be called when
 the probe is hit.
 
+There are two kinds of unregistration functions. unregister_*probe()
+functions are useful to unregister a few probes. However, it will take
+several seconds to remove all probes when you unregister hundreds of
+probes. In contrast, unregister_*probe_fast() and commit_kprobes() are
+fast to remove hundreds of probes. unregister_*probe_fast() function
+removes probes, but doesn't cleanup it. So, don't forget to call
+commit_kprobes() for cleaning up when you finish to remove all probes
+by using unregister_*probe_fast().
+
 The next three subsections explain how the different types of
 probes work.  They explain certain things that you'll need to
 know in order to make the best use of Kprobes -- e.g., the

@@ -295,6 +304,27 @@ void unregister_kretprobe(struct kretpro
 Removes the specified probe.  The unregister function can be called
 at any time after the probe has been registered.
 
+4.5 unregister_*probe_fast
+
+#include <linux/kprobes.h>
+void unregister_kprobe_fast(struct kprobe *kp);
+void unregister_jprobe_fast(struct jprobe *jp);
+void unregister_kretprobe_fast(struct kretprobe *rp);
+
+Prepares to remove the specified probe, and add it to cleanup list.
+The unregister function can be called at any time after the probe has
+been registered.
+WARNING: you MUST call commit_kprobes() before freeing or reusing
+those probes.
+
+4.6 commit_kprobes
+
+#include <linux/kprobes.h>
+void commit_kprobes(void);
+
+Commits to removing all prepared probes. This function synchronizes
+scheduler for RCU safety, and unregisters all probes on the cleanup list.
+
 5. Kprobes Features and Limitations
 
 Kprobes allows multiple probes at the same address.  Currently,

Speeding up the unregistration is a very good idea, but this interface is rather horrible. It's almost a receipe for users to get it wrong.

Instead just changes the register and unregister functions to take a NULL-terminated array of probes that get registered and unregisters at the same time (and renames them from *probe* to *probes*, please)

The ability to disable/reenable kprobes would be an interesting enhancement. However, unregister_disabled_kprobes() shouldn't have a global effect, because there might be a concurrent kprobes user that disabled a probe with the intention of reenabling it later.

Josh Stone

Hi -

Really? What possible problems can occur? The worst that occurs to me is that if someone forgets to call the commit function, the kprobes will still be disabled, but memory won't be recycled for a while. Is this really so bad, considering that a kprobe_unregister is to imply a commit? Maybe if kprobe_register can also implied a commit, we can bound the conceivable memory leak.

Would it be possible to allay even that concern with an automated deferred/periodic commit?

- FChE

Hi -

A little, but we were talking more about the interface than the implementation.

I believe there is already a kprobes patch in the queue for enumerating active probes in some /proc file. Should a module be locked into memory for such a reason, finding the culprit should not be difficult.

One problem with this idea is that if the unregister_fast()=disable() is to become reversible, then the renamed commit_kprobes() will no longer be indempotent. There can no longer be a single system-wide deferred-kprobe-cleanup list, since individual kprobes clients might want to reinstate their probes in the future.

But this would solve both problems (memory leaks and outstanding reference counts on modules). In this variant, unregister_kprobes_fast could replace unregister_kprobes outright, and the (builtin deferred) commit function would need not be exported.

- FChE

Hi Christoph and Anil,

Thank you for your comments.

OK, I agree. I had chosen a confusable name.

Thank you.

I'd like to change that to prepare_to_unregister_kprobe(p) instead of disable_kprobe(p).

I think Josh and other people want interfaces to disable/reenable all probes at once when the sysrq is pressed.
So, IMHO, these interfaces should use a global (and per-cpu?) flag which controls whether kprobes calls user-defined handler or not, instead of self-modifying.
For example,

if (p && p->pre_handler && kprobe_enable) {
                           ^^^^^^^^^^^^^

...
}

But, I think this would be another story.
I'd like to discuss this topic in other mails.

And also, I'd like to change it to unregister_prepared_kprobes(). What would you think about this idea?

Thanks,

Masami HIRAMATSU
Linux Technology Center
Hitachi, Ltd., Systems Development Laboratory
E-mail: [email protected]

Hi Anil,

Would you mean that we should integrate unregistration interface? Or, would you mean that you prefer "disable_kprobe()" since that name provides another meaning?

Thanks,

Masami HIRAMATSU
Linux Technology Center
Hitachi, Ltd., Systems Development Laboratory
E-mail: [email protected]

Exactly. It's a very unintuitive interface, exposing implementation details to the user. The array variant otoh is obvious to the user:
gice me an array of probes to register/unregister and do it.

That's even worse from the user perspective interface.

enabling/disabling probes sounds like a wonderful additional feature. I don't think we should expose it as a kernel interface, but rather through debugfs so I can temporarily disable and then reenable probes from the shell easily.