Skip to contentSkip to conversation

Text Section Edit Lock

PATCH In review Mathieu Desnoyers 3 participants 7 messages Jun 18, 2007

lore
Share & tools
lore

Mathieu Desnoyers

author
#0 lore raw

cover Text Section Edit Lock

Hi,

This code adds locking around text section modifications. It can be used by paravirt/alternatives, kprobes and eventually the immediate values to:

A - synchronize dynamic code patching
B - make sure the pages are writable

I only provided the i386 version, but it could be easily ported to other architectures.

It applies at the end of the 2.6.22-rc4-mm2 series in the following order:

text-edit-lock-i386.patch
text-edit-lock-alternative-i386.patch
text-edit-lock-kprobes-i386.patch

Comments are welcome, (testing is welcome too)

Mathieu

Signature · 3 lines
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68
author
#1 lore raw

PATCH 1/3 Text Edit Lock - i386

Interface to use for code patching : uses a mutex to insure mutual edit exclusion and makes sure the page is writable.

Signed-off-by Mathieu Desnoyers <[email protected]>
---
 arch/i386/mm/init.c           |   52 ++++++++++++++++++++++++++++++++++--------
 include/asm-i386/cacheflush.h |    4 +++
 2 files changed, 47 insertions(+), 9 deletions(-)
arch/i386/mm/init.c +43 -9
Index: linux-2.6-lttng/arch/i386/mm/init.c
===================================================================
@@ -31,6 +31,7 @@
 #include <linux/memory_hotplug.h>
 #include <linux/initrd.h>
 #include <linux/cpumask.h>
+#include <linux/mutex.h>
 
 #include <asm/processor.h>
 #include <asm/system.h>

@@ -52,6 +53,9 @@
 
 static int noinline do_test_wp_bit(void);
 
+/* Mutex protecting text section modification (dynamic code patching) */
+static DEFINE_MUTEX(text_mutex);
+
 /*
  * Creates a middle page table and puts a pointer to it in the
  * given global directory entry. This only returns the gd entry

@@ -800,15 +804,9 @@
 	unsigned long start = PFN_ALIGN(_text);
 	unsigned long size = PFN_ALIGN(_etext) - start;
 
-#ifdef CONFIG_HOTPLUG_CPU
-	/* It must still be possible to apply SMP alternatives. */
-	if (num_possible_cpus() <= 1)
-#endif
-	{
-		change_page_attr(virt_to_page(start),
-		                 size >> PAGE_SHIFT, PAGE_KERNEL_RX);
-		printk("Write protecting the kernel text: %luk\n", size >> 10);
-	}
+	change_page_attr(virt_to_page(start),
+			 size >> PAGE_SHIFT, PAGE_KERNEL_RX);
+	printk("Write protecting the kernel text: %luk\n", size >> 10);
 
 	start += size;
 	size = (unsigned long)__end_rodata - start;

@@ -827,6 +825,42 @@
 }
 #endif
 
+/*
+ * Lock the kernel text for mutual write exclusion.
+ * Make sure the pages are writable.
+ */
+void kernel_text_lock(unsigned long address, size_t len)
+{
+	mutex_lock(&text_mutex);
+#if defined(CONFIG_DEBUG_RODATA)
+	if (is_kernel_text(address) && is_kernel_text(address + len)) {
+		unsigned long nr_pages;
+		nr_pages = ((address + len) >> PAGE_SHIFT)
+				- (address >> PAGE_SHIFT) + 1;
+		change_page_attr(virt_to_page(address), nr_pages,
+				PAGE_KERNEL_EXEC);
+		global_flush_tlb();
+	}
+#endif
+}
+EXPORT_SYMBOL_GPL(kernel_text_lock);
+
+void kernel_text_unlock(unsigned long address, size_t len)
+{
+#if defined(CONFIG_DEBUG_RODATA)
+	if (is_kernel_text(address) && is_kernel_text(address + len)) {
+		unsigned long nr_pages;
+		nr_pages = ((address + len) >> PAGE_SHIFT)
+				- (address >> PAGE_SHIFT) + 1;
+		change_page_attr(virt_to_page(address), nr_pages,
+				PAGE_KERNEL_RX);
+		global_flush_tlb();
+	}
+#endif
+	mutex_unlock(&text_mutex);
+}
+EXPORT_SYMBOL_GPL(kernel_text_unlock);
+
 void free_init_pages(char *what, unsigned long begin, unsigned long end)
 {
 	unsigned long addr;
include/asm-i386/cacheflush.h +4 -0
Index: linux-2.6-lttng/include/asm-i386/cacheflush.h
===================================================================
@@ -36,4 +36,8 @@
 void mark_rodata_ro(void);
 #endif
 
+/* lock kernel text and mark pages writable */
+extern void kernel_text_lock(unsigned long address, size_t len);
+extern void kernel_text_unlock(unsigned long address, size_t len);
+
 #endif /* _I386_CACHEFLUSH_H */
Signature · 3 lines
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68
#2 lore raw

Re: [patch 1/3] Text Edit Lock - i386

Quoted 3 lines · Mathieu Desnoyers

Interface to use for code patching : uses a mutex to insure mutual edit exclusion and makes sure the page is writable.

...

arch/i386/mm/init.c line 58

Quoted 3 lines
arch/i386/mm/init.c +3 -0
+/* Mutex protecting text section modification (dynamic code patching) */
+static DEFINE_MUTEX(text_mutex);
+

Probably should be a spinlock.

And it just occurred to me, how does smp_alternatives deal with this? Is it broken now when the text section is read-only?

author
#3 lore raw

Re: [patch 1/3] Text Edit Lock - i386

Quoted 5 lines · * Chuck Ebbert ([email protected]) wrote:
Quoted 3 lines · Mathieu Desnoyers

Interface to use for code patching : uses a mutex to insure mutual edit exclusion and makes sure the page is writable.

...

Quoted 8 lines

arch/i386/mm/init.c line 58

Quoted 3 lines
arch/i386/mm/init.c +3 -0
+/* Mutex protecting text section modification (dynamic code patching) */
+static DEFINE_MUTEX(text_mutex);
+

Probably should be a spinlock.

And it just occurred to me, how does smp_alternatives deal with this? Is it broken now when the text section is read-only?

(note that the implementation I just posted is a proof of concept: I just noticed that I need to keep track of wether or not I am called before or after the mark_rodata is done, so the apply alternatives does not crash at early boot because of the global_flush_tlb().)

A spinlock it will be then :)

SMP alternatives deals with this by simply disabling the whole protection:

mark_rodata_ro():

#ifdef CONFIG_HOTPLUG_CPU
        /* It must still be possible to apply SMP alternatives. */
        if (num_possible_cpus() <= 1)
#endif
        {
                change_page_attr(virt_to_page(start),
                                 size >> PAGE_SHIFT, PAGE_KERNEL_RX);
                printk("Write protecting the kernel text: %luk\n", size >> 10);
        }

So it's ok if no CPU can be hotplugged, since the CPUs are brought up before the mark_rodata_ro is done, but not if HOTPLUG is selected.

Mathieu

Signature · 3 lines
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68
#4 lore raw

Re: [patch 1/3] Text Edit Lock - i386

In reply to Chuck Ebbert
Quoted 5 lines · On Tuesday 19 June 2007 00:52:25 Chuck Ebbert wrote:
Quoted 3 lines · Mathieu Desnoyers

Interface to use for code patching : uses a mutex to insure mutual edit exclusion and makes sure the page is writable.

...

Quoted 8 lines

arch/i386/mm/init.c line 58

Quoted 3 lines
arch/i386/mm/init.c +3 -0
+/* Mutex protecting text section modification (dynamic code patching) */
+static DEFINE_MUTEX(text_mutex);
+

Probably should be a spinlock.

And it just occurred to me, how does smp_alternatives deal with this? Is it broken now when the text section is read-only?

The text section is only changed to ro very late, alternative code runs earlier.

But when you unplug all CPUs to go back to UP I suspect it may break.

-Andi

author
#5 lore raw

PATCH 2/3 Text Edit Lock - Alternative i386

In reply to Mathieu Desnoyers

alternative.c can use the text edit lock to remove CPU HOTPLUG special cases.

Signed-off-by Mathieu Desnoyers <[email protected]>
---
 arch/i386/kernel/alternative.c |    5 +++++
 1 file changed, 5 insertions(+)
arch/i386/kernel/alternative.c +5 -0
Index: linux-2.6-lttng/arch/i386/kernel/alternative.c
===================================================================
@@ -4,6 +4,7 @@
 #include <linux/list.h>
 #include <asm/alternative.h>
 #include <asm/sections.h>
+#include <asm/cacheflush.h>
 
 static int noreplace_smp     = 0;
 static int smp_alt_once      = 0;

@@ -179,9 +180,11 @@
 				__FUNCTION__, a->instr, instr);
 		}
 #endif
+		kernel_text_lock((unsigned long)instr, a->instrlen);
 		memcpy(instr, a->replacement, a->replacementlen);
 		diff = a->instrlen - a->replacementlen;
 		nop_out(instr + a->replacementlen, diff);
+		kernel_text_unlock((unsigned long)instr, a->instrlen);
 	}
 }
 

@@ -346,6 +349,7 @@
 	for (p = start; p < end; p++) {
 		unsigned int used;
 
+		kernel_text_lock((unsigned long)p->instr, p->len);
 		used = paravirt_ops.patch(p->instrtype, p->clobbers, p->instr,
 					  p->len);
 

@@ -353,6 +357,7 @@
 
 		/* Pad the rest with nops */
 		nop_out(p->instr + used, p->len - used);
+		kernel_text_unlock((unsigned long)p->instr, p->len);
 	}
 
 	/* Sync to be conservative, in case we patched following
Signature · 3 lines
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68
author
#6 lore raw

PATCH 3/3 Text Edit Lock - kprobes i386

In reply to Mathieu Desnoyers

Kprobes can use the text edit lock to insure mutual exclusion when edition the code and make sure the pages are writable.

Signed-off-by Mathieu Desnoyers <[email protected]>
---
 arch/i386/kernel/kprobes.c |    4 ++++
 1 file changed, 4 insertions(+)
arch/i386/kernel/kprobes.c +4 -0
Index: linux-2.6-lttng/arch/i386/kernel/kprobes.c
===================================================================
@@ -169,14 +169,18 @@
 
 void __kprobes arch_arm_kprobe(struct kprobe *p)
 {
+	kernel_text_lock((unsigned long)p->addr, sizeof(kprobe_opcode_t));
 	*p->addr = BREAKPOINT_INSTRUCTION;
+	kernel_text_unlock((unsigned long)p->addr, sizeof(kprobe_opcode_t));
 	flush_icache_range((unsigned long) p->addr,
 			   (unsigned long) p->addr + sizeof(kprobe_opcode_t));
 }
 
 void __kprobes arch_disarm_kprobe(struct kprobe *p)
 {
+	kernel_text_lock((unsigned long)p->addr, sizeof(kprobe_opcode_t));
 	*p->addr = p->opcode;
+	kernel_text_unlock((unsigned long)p->addr, sizeof(kprobe_opcode_t));
 	flush_icache_range((unsigned long) p->addr,
 			   (unsigned long) p->addr + sizeof(kprobe_opcode_t));
 }
Signature · 3 lines
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68