Skip to contentSkip to conversation

Use per-CPU temporary mappings for patching

RFC PATCH In review Christopher M. Riedl 2 participants 12 messages Apr 29, 2020 – May 1, 2020

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

v2 cover Use per-CPU temporary mappings for patching

When compiled with CONFIG_STRICT_KERNEL_RWX, the kernel must create temporary mappings when patching itself. These mappings temporarily override the strict RWX text protections to permit a write. Currently, powerpc allocates a per-CPU VM area for patching. Patching occurs as follows:

1. Map page of text to be patched to per-CPU VM area w/ PAGE_KERNEL protection
2. Patch text
3. Remove the temporary mapping

While the VM area is per-CPU, the mapping is actually inserted into the kernel page tables. Presumably, this could allow another CPU to access the normally write-protected text - either malicously or accidentally - via this same mapping if the address of the VM area is known. Ideally, the mapping should be kept local to the CPU doing the patching (or any other sensitive operations requiring temporarily overriding memory protections) [0].

x86 introduced "temporary mm" structs which allow the creation of mappings local to a particular CPU [1]. This series intends to bring the notion of a temporary mm to powerpc and harden powerpc by using such a mapping for patching a kernel with strict RWX permissions.

The first patch introduces the temporary mm struct and API for powerpc along with a new function to retrieve a current hw breakpoint.

The second patch uses the poking_init init hook added by the x86 patches to initialize a temporary mm and patching address. The patching address is randomized between 0 and DEFAULT_MAP_WINDOW-PAGE_SIZE. The upper limit is necessary due to how the hash MMU operates - by default the space above DEFAULT_MAP_WINDOW is not available. For now, both hash and radix randomize inside this range. The number of possible random addresses is dependent on PAGE_SIZE and limited by DEFAULT_MAP_WINDOW.

Bits of entropy with 64K page size on BOOK3S_64:

	bits of entropy = log2(DEFAULT_MAP_WINDOW_USER64 / PAGE_SIZE)

	PAGE_SIZE=64K, DEFAULT_MAP_WINDOW_USER64=128TB
	bits of entropy = log2(128TB / 64K)
	bits of entropy = 31

Randomization occurs only once during initialization at boot.

The third patch replaces the VM area with the temporary mm in the patching code. The page for patching has to be mapped PAGE_SHARED with the hash MMU since hash prevents the kernel from accessing userspace pages with PAGE_PRIVILEGED bit set. On the radix MMU the page is mapped with PAGE_KERNEL which has the added benefit that we can skip KUAP.

The fourth and fifth patches implement an LKDTM test "proof-of-concept" which exploits the previous vulnerability (ie. the mapping during patching is exposed in kernel page tables and accessible by other CPUS). The LKDTM test is somewhat "rough" in that it uses a brute-force approach - I am open to any suggestions and/or ideas to improve this. Currently, the LKDTM test passes with this series on POWER8 (hash) and POWER9 (radix, hash) and fails without this series (ie. the temporary mapping for patching is exposed to CPUs other than the patching CPU).

The test can be applied to a tree without this new series by first adding this in /arch/powerpc/lib/code-patching.c:

@@ -41,6 +41,13 @@ int raw_patch_instruction(unsigned int *addr, unsigned int instr)
#ifdef CONFIG_STRICT_KERNEL_RWX
static DEFINE_PER_CPU(struct vm_struct *, text_poke_area);

+#ifdef CONFIG_LKDTM
+unsigned long read_cpu_patching_addr(unsigned int cpu) +{
+ return (unsigned long)(per_cpu(text_poke_area, cpu))->addr;
+}
+#endif
+
static int text_area_cpu_up(unsigned int cpu)

 {
        struct vm_struct *area;

And then applying the last patch of this series which adds the LKDTM test, (powerpc: Add LKDTM test to hijack a patch mapping).

Tested on QEMU (POWER8, POWER9), POWER8 VM, and a Blackbird (8-core POWER9).

v2: Many fixes and improvements mostly based on extensive feedback and testing by Christophe Leroy (thanks!).
* Make patching_mm and patching_addr static and mode '__ro_after_init' to after the variable name (more common in other parts of the kernel)
* Use 'asm/debug.h' header instead of 'asm/hw_breakpoint.h' to fix PPC64e compile
* Add comment explaining why we use BUG_ON() during the init call to setup for patching later
* Move ptep into patch_mapping to avoid walking page tables a second time when unmapping the temporary mapping
* Use KUAP under non-radix, also manually dirty the PTE for patch mapping on non-BOOK3S_64 platforms
* Properly return any error from __patch_instruction
* Do not use 'memcmp' where a simple comparison is appropriate
* Simplify expression for patch address by removing pointer maths
* Add LKDTM test

[0]: https://github.com/linuxppc/issues/issues/224
[1]: https://lore.kernel.org/kernel-hardening/[email protected]/

Christopher M. Riedl (5):
powerpc/mm: Introduce temporary mm
powerpc/lib: Initialize a temporary mm for code patching powerpc/lib: Use a temporary mm for code patching powerpc/lib: Add LKDTM accessor for patching addr
powerpc: Add LKDTM test to hijack a patch mapping

 arch/powerpc/include/asm/debug.h       |   1 +
 arch/powerpc/include/asm/mmu_context.h |  54 ++++++++
 arch/powerpc/kernel/process.c          |   5 +
 arch/powerpc/lib/code-patching.c       | 173 +++++++++++++------------
 drivers/misc/lkdtm/core.c              |   1 +
 drivers/misc/lkdtm/lkdtm.h             |   1 +
 drivers/misc/lkdtm/perms.c             |  99 ++++++++++++++
 7 files changed, 248 insertions(+), 86 deletions(-)
Signature · 1 line
2.26.1
#1 lore raw

RFC PATCH v2 1/5 powerpc/mm: Introduce temporary mm

x86 supports the notion of a temporary mm which restricts access to temporary PTEs to a single CPU. A temporary mm is useful for situations where a CPU needs to perform sensitive operations (such as patching a STRICT_KERNEL_RWX kernel) requiring temporary mappings without exposing said mappings to other CPUs. A side benefit is that other CPU TLBs do not need to be flushed when the temporary mm is torn down.

Mappings in the temporary mm can be set in the userspace portion of the address-space.

Interrupts must be disabled while the temporary mm is in use. HW breakpoints, which may have been set by userspace as watchpoints on addresses now within the temporary mm, are saved and disabled when loading the temporary mm. The HW breakpoints are restored when unloading the temporary mm. All HW breakpoints are indiscriminately disabled while the temporary mm is in use.

Based on x86 implementation:

commit cefa929c034e
("x86/mm: Introduce temporary mm structs")

---
 arch/powerpc/include/asm/debug.h       |  1 +
 arch/powerpc/include/asm/mmu_context.h | 54 ++++++++++++++++++++++++++
 arch/powerpc/kernel/process.c          |  5 +++
 3 files changed, 60 insertions(+)
arch/powerpc/include/asm/debug.h +1 -0
index 7756026b95ca..b945bc16c932 100644
@@ -45,6 +45,7 @@ static inline int debugger_break_match(struct pt_regs *regs) { return 0; }
 static inline int debugger_fault_handler(struct pt_regs *regs) { return 0; }
 #endif
 
+void __get_breakpoint(struct arch_hw_breakpoint *brk);
 void __set_breakpoint(struct arch_hw_breakpoint *brk);
 bool ppc_breakpoint_available(void);
 #ifdef CONFIG_PPC_ADV_DEBUG_REGS
arch/powerpc/include/asm/mmu_context.h +54 -0

65 lines not on this page · raw message

arch/powerpc/kernel/process.c +5 -0
index 9c21288f8645..ec4cf890d92c 100644
@@ -800,6 +800,11 @@ static inline int set_breakpoint_8xx(struct arch_hw_breakpoint *brk)
 	return 0;
 }
 
+void __get_breakpoint(struct arch_hw_breakpoint *brk)
+{
+	memcpy(brk, this_cpu_ptr(&current_brk), sizeof(*brk));
+}
+
 void __set_breakpoint(struct arch_hw_breakpoint *brk)
 {
 	memcpy(this_cpu_ptr(&current_brk), brk, sizeof(*brk));
Signature · 1 line
2.26.1
#2 lore raw

Re: [RFC PATCH v2 1/5] powerpc/mm: Introduce temporary mm

Le 29/04/2020 à 04:05, Christopher M. Riedl a écrit :

Quoted 87 lines

x86 supports the notion of a temporary mm which restricts access to temporary PTEs to a single CPU. A temporary mm is useful for situations where a CPU needs to perform sensitive operations (such as patching a STRICT_KERNEL_RWX kernel) requiring temporary mappings without exposing said mappings to other CPUs. A side benefit is that other CPU TLBs do not need to be flushed when the temporary mm is torn down.

Mappings in the temporary mm can be set in the userspace portion of the address-space.

Interrupts must be disabled while the temporary mm is in use. HW breakpoints, which may have been set by userspace as watchpoints on addresses now within the temporary mm, are saved and disabled when loading the temporary mm. The HW breakpoints are restored when unloading the temporary mm. All HW breakpoints are indiscriminately disabled while the temporary mm is in use.

Based on x86 implementation:

commit cefa929c034e
("x86/mm: Introduce temporary mm structs")

---
  arch/powerpc/include/asm/debug.h       |  1 +
  arch/powerpc/include/asm/mmu_context.h | 54 ++++++++++++++++++++++++++
  arch/powerpc/kernel/process.c          |  5 +++
  3 files changed, 60 insertions(+)
arch/powerpc/include/asm/debug.h +1 -0
index 7756026b95ca..b945bc16c932 100644
@@ -45,6 +45,7 @@ static inline int debugger_break_match(struct pt_regs *regs) { return 0; }
  static inline int debugger_fault_handler(struct pt_regs *regs) { return 0; }
  #endif
 
+void __get_breakpoint(struct arch_hw_breakpoint *brk);
  void __set_breakpoint(struct arch_hw_breakpoint *brk);
  bool ppc_breakpoint_available(void);
  #ifdef CONFIG_PPC_ADV_DEBUG_REGS
arch/powerpc/include/asm/mmu_context.h +31 -0
index 360367c579de..57a8695fe63f 100644
@@ -10,6 +10,7 @@
  #include <asm/mmu.h>	
  #include <asm/cputable.h>
  #include <asm/cputhreads.h>
+#include <asm/debug.h>
 
  /*
   * Most if the context management is out of line
@@ -270,5 +271,58 @@ static inline int arch_dup_mmap(struct mm_struct *oldmm,
  	return 0;
  }
 
+struct temp_mm {
+	struct mm_struct *temp;
+	struct mm_struct *prev;
+	bool is_kernel_thread;
+	struct arch_hw_breakpoint brk;
+};
+
+static inline void init_temp_mm(struct temp_mm *temp_mm, struct mm_struct *mm)
+{
+	temp_mm->temp = mm;
+	temp_mm->prev = NULL;
+	temp_mm->is_kernel_thread = false;
+	memset(&temp_mm->brk, 0, sizeof(temp_mm->brk));
+}
+
+static inline void use_temporary_mm(struct temp_mm *temp_mm)
+{
+	lockdep_assert_irqs_disabled();
+
+	temp_mm->is_kernel_thread = current->mm == NULL;
+	if (temp_mm->is_kernel_thread)
+		temp_mm->prev = current->active_mm;
+	else
+		temp_mm->prev = current->mm;
+
+	/*
+	 * Hash requires a non-NULL current->mm to allocate a userspace address
+	 * when handling a page fault. Does not appear to hurt in Radix either.
+	 */
+	current->mm = temp_mm->temp;

arch/powerpc/include/asm/mmu_context.h static inline int arch_dup_mmap(struct mm_struct *oldmm, · line 313

Quoted 10 lines
arch/powerpc/include/asm/mmu_context.h +10 -0
+	switch_mm_irqs_off(NULL, temp_mm->temp, current);
+
+	if (ppc_breakpoint_available()) {
+		__get_breakpoint(&temp_mm->brk);
+		if (temp_mm->brk.type != 0)
+			hw_breakpoint_disable();
+	}
+}
+
+static inline void unuse_temporary_mm(struct temp_mm *temp_mm)

Not sure "unuse" is a best naming, allthought I don't have a better suggestion a the moment. If not using temporary_mm anymore, what are we using now ?

Quoted 22 lines
arch/powerpc/include/asm/mmu_context.h +13 -0
+{
+	lockdep_assert_irqs_disabled();
+
+	if (temp_mm->is_kernel_thread)
+		current->mm = NULL;
+	else
+		current->mm = temp_mm->prev;
+	switch_mm_irqs_off(NULL, temp_mm->prev, current);
+
+	if (ppc_breakpoint_available() && temp_mm->brk.type != 0)
+		__set_breakpoint(&temp_mm->brk);
+}
+
  #endif /* __KERNEL__ */
  #endif /* __ASM_POWERPC_MMU_CONTEXT_H */
arch/powerpc/kernel/process.c +0 -0
index 9c21288f8645..ec4cf890d92c 100644
@@ -800,6 +800,11 @@ static inline int set_breakpoint_8xx(struct arch_hw_breakpoint *brk)
  	return 0;

arch/powerpc/kernel/process.c static inline int set_breakpoint_8xx(struct arch_hw_breakpoint *brk) · line 810

Quoted 10 lines
arch/powerpc/kernel/process.c +5 -0
  }
 
+void __get_breakpoint(struct arch_hw_breakpoint *brk)
+{
+	memcpy(brk, this_cpu_ptr(&current_brk), sizeof(*brk));
+}
+
  void __set_breakpoint(struct arch_hw_breakpoint *brk)
  {
  	memcpy(this_cpu_ptr(&current_brk), brk, sizeof(*brk));

Christophe

#3 lore raw

Re: [RFC PATCH v2 1/5] powerpc/mm: Introduce temporary mm

Quoted 99 lines · Christophe Leroy
Quoted 94 lines · Christopher M. Riedl

x86 supports the notion of a temporary mm which restricts access to temporary PTEs to a single CPU. A temporary mm is useful for situations where a CPU needs to perform sensitive operations (such as patching a STRICT_KERNEL_RWX kernel) requiring temporary mappings without exposing said mappings to other CPUs. A side benefit is that other CPU TLBs do not need to be flushed when the temporary mm is torn down.

Mappings in the temporary mm can be set in the userspace portion of the address-space.

Interrupts must be disabled while the temporary mm is in use. HW breakpoints, which may have been set by userspace as watchpoints on addresses now within the temporary mm, are saved and disabled when loading the temporary mm. The HW breakpoints are restored when unloading the temporary mm. All HW breakpoints are indiscriminately disabled while the temporary mm is in use.

Based on x86 implementation:

commit cefa929c034e
("x86/mm: Introduce temporary mm structs")

---
  arch/powerpc/include/asm/debug.h       |  1 +
  arch/powerpc/include/asm/mmu_context.h | 54 ++++++++++++++++++++++++++
  arch/powerpc/kernel/process.c          |  5 +++
  3 files changed, 60 insertions(+)
arch/powerpc/include/asm/debug.h +1 -0
index 7756026b95ca..b945bc16c932 100644
@@ -45,6 +45,7 @@ static inline int debugger_break_match(struct pt_regs *regs) { return 0; }
  static inline int debugger_fault_handler(struct pt_regs *regs) { return 0; }
  #endif
 
+void __get_breakpoint(struct arch_hw_breakpoint *brk);
  void __set_breakpoint(struct arch_hw_breakpoint *brk);
  bool ppc_breakpoint_available(void);
  #ifdef CONFIG_PPC_ADV_DEBUG_REGS
arch/powerpc/include/asm/mmu_context.h +38 -0
index 360367c579de..57a8695fe63f 100644
@@ -10,6 +10,7 @@
  #include <asm/mmu.h>	
  #include <asm/cputable.h>
  #include <asm/cputhreads.h>
+#include <asm/debug.h>
 
  /*
   * Most if the context management is out of line
@@ -270,5 +271,58 @@ static inline int arch_dup_mmap(struct mm_struct *oldmm,
  	return 0;
  }
 
+struct temp_mm {
+	struct mm_struct *temp;
+	struct mm_struct *prev;
+	bool is_kernel_thread;
+	struct arch_hw_breakpoint brk;
+};
+
+static inline void init_temp_mm(struct temp_mm *temp_mm, struct mm_struct *mm)
+{
+	temp_mm->temp = mm;
+	temp_mm->prev = NULL;
+	temp_mm->is_kernel_thread = false;
+	memset(&temp_mm->brk, 0, sizeof(temp_mm->brk));
+}
+
+static inline void use_temporary_mm(struct temp_mm *temp_mm)
+{
+	lockdep_assert_irqs_disabled();
+
+	temp_mm->is_kernel_thread = current->mm == NULL;
+	if (temp_mm->is_kernel_thread)
+		temp_mm->prev = current->active_mm;
+	else
+		temp_mm->prev = current->mm;
+
+	/*
+	 * Hash requires a non-NULL current->mm to allocate a userspace address
+	 * when handling a page fault. Does not appear to hurt in Radix either.
+	 */
+	current->mm = temp_mm->temp;
+	switch_mm_irqs_off(NULL, temp_mm->temp, current);
+
+	if (ppc_breakpoint_available()) {
+		__get_breakpoint(&temp_mm->brk);
+		if (temp_mm->brk.type != 0)
+			hw_breakpoint_disable();
+	}
Quoted 10 lines

arch/powerpc/include/asm/mmu_context.h static inline int arch_dup_mmap(struct mm_struct *oldmm, · line 313

Quoted 3 lines
arch/powerpc/include/asm/mmu_context.h +3 -0
+}
+
+static inline void unuse_temporary_mm(struct temp_mm *temp_mm)

Not sure "unuse" is a best naming, allthought I don't have a better suggestion a the moment. If not using temporary_mm anymore, what are we using now ?

I'm not too fond of 'unuse' either, but it's what x86 uses and I couldn't come up with anything better on the spot. Maybe 'undo' is better since we're switching back to whatever mm was in use before?

Quoted 39 lines
Quoted 22 lines
arch/powerpc/include/asm/mmu_context.h +13 -0
+{
+	lockdep_assert_irqs_disabled();
+
+	if (temp_mm->is_kernel_thread)
+		current->mm = NULL;
+	else
+		current->mm = temp_mm->prev;
+	switch_mm_irqs_off(NULL, temp_mm->prev, current);
+
+	if (ppc_breakpoint_available() && temp_mm->brk.type != 0)
+		__set_breakpoint(&temp_mm->brk);
+}
+
  #endif /* __KERNEL__ */
  #endif /* __ASM_POWERPC_MMU_CONTEXT_H */
arch/powerpc/kernel/process.c +0 -0
index 9c21288f8645..ec4cf890d92c 100644
@@ -800,6 +800,11 @@ static inline int set_breakpoint_8xx(struct arch_hw_breakpoint *brk)
  	return 0;

arch/powerpc/kernel/process.c static inline int set_breakpoint_8xx(struct arch_hw_breakpoint *brk) · line 810

Quoted 10 lines
arch/powerpc/kernel/process.c +5 -0
  }
 
+void __get_breakpoint(struct arch_hw_breakpoint *brk)
+{
+	memcpy(brk, this_cpu_ptr(&current_brk), sizeof(*brk));
+}
+
  void __set_breakpoint(struct arch_hw_breakpoint *brk)
  {
  	memcpy(this_cpu_ptr(&current_brk), brk, sizeof(*brk));

Christophe

#4 lore raw

Re: [RFC PATCH v2 1/5] powerpc/mm: Introduce temporary mm

In reply to Christopher M. Riedl

Le 29/04/2020 à 04:05, Christopher M. Riedl a écrit :

Quoted 10 lines

x86 supports the notion of a temporary mm which restricts access to temporary PTEs to a single CPU. A temporary mm is useful for situations where a CPU needs to perform sensitive operations (such as patching a STRICT_KERNEL_RWX kernel) requiring temporary mappings without exposing said mappings to other CPUs. A side benefit is that other CPU TLBs do not need to be flushed when the temporary mm is torn down.

Mappings in the temporary mm can be set in the userspace portion of the address-space.

Quoted 6 lines

Interrupts must be disabled while the temporary mm is in use. HW breakpoints, which may have been set by userspace as watchpoints on addresses now within the temporary mm, are saved and disabled when loading the temporary mm. The HW breakpoints are restored when unloading the temporary mm. All HW breakpoints are indiscriminately disabled while the temporary mm is in use.

Why do we need to use a temporary mm all the time ?

Doesn't each CPU have its own mm already ? Only the upper address space is shared between all mm's but each mm has its own lower address space, at least when it is running a user process. Why not just use that mm ? As we are mapping then unmapping with interrupts disabled, there is no risk at all that the user starts running while the patch page is mapped, so I'm not sure why switching to a temporary mm is needed.

Quoted 7 lines

Based on x86 implementation:

commit cefa929c034e
("x86/mm: Introduce temporary mm structs")

Christophe

#5 lore raw

Re: [RFC PATCH v2 1/5] powerpc/mm: Introduce temporary mm

Quoted 23 lines · Christophe Leroy
Quoted 10 lines · Christopher M. Riedl

x86 supports the notion of a temporary mm which restricts access to temporary PTEs to a single CPU. A temporary mm is useful for situations where a CPU needs to perform sensitive operations (such as patching a STRICT_KERNEL_RWX kernel) requiring temporary mappings without exposing said mappings to other CPUs. A side benefit is that other CPU TLBs do not need to be flushed when the temporary mm is torn down.

Mappings in the temporary mm can be set in the userspace portion of the address-space.

Quoted 6 lines

Interrupts must be disabled while the temporary mm is in use. HW breakpoints, which may have been set by userspace as watchpoints on addresses now within the temporary mm, are saved and disabled when loading the temporary mm. The HW breakpoints are restored when unloading the temporary mm. All HW breakpoints are indiscriminately disabled while the temporary mm is in use.

Quoted 2 lines

Why do we need to use a temporary mm all the time ?

Not sure I understand, the temporary mm is only in use for kernel patching in this series. We could have other uses in the future maybe where it's beneficial to keep mappings local.

Quoted 9 lines

Doesn't each CPU have its own mm already ? Only the upper address space is shared between all mm's but each mm has its own lower address space, at least when it is running a user process. Why not just use that mm ? As we are mapping then unmapping with interrupts disabled, there is no risk at all that the user starts running while the patch page is mapped, so I'm not sure why switching to a temporary mm is needed.

I suppose that's an option, but then we have to save and restore the mapping which we temporarily "steal" from userspace. I admit I didn't consider that as an option when I started this series based on the x86 patches. I think it's cleaner to switch mm, but that's a rather weak argument. Are you concerned about performance with the temporary mm?

Quoted 16 lines
Quoted 9 lines

Based on x86 implementation:

commit cefa929c034e
("x86/mm: Introduce temporary mm structs")

Christophe

#6 lore raw

RFC PATCH v2 2/5 powerpc/lib: Initialize a temporary mm for code patching

In reply to Christopher M. Riedl

When code patching a STRICT_KERNEL_RWX kernel the page containing the address to be patched is temporarily mapped with permissive memory protections. Currently, a per-cpu vmalloc patch area is used for this purpose. While the patch area is per-cpu, the temporary page mapping is inserted into the kernel page tables for the duration of the patching. The mapping is exposed to CPUs other than the patching CPU - this is undesirable from a hardening perspective.

Use the poking_init init hook to prepare a temporary mm and patching address. Initialize the temporary mm by copying the init mm. Choose a randomized patching address inside the temporary mm userspace address portion. The next patch uses the temporary mm and patching address for code patching.

Based on x86 implementation:

commit 4fc19708b165
("x86/alternatives: Initialize temporary mm for patching")

---
 arch/powerpc/lib/code-patching.c | 33 ++++++++++++++++++++++++++++++++
 1 file changed, 33 insertions(+)
arch/powerpc/lib/code-patching.c +33 -0
index 3345f039a876..259c19480a85 100644
@@ -11,6 +11,8 @@
 #include <linux/cpuhotplug.h>
 #include <linux/slab.h>
 #include <linux/uaccess.h>
+#include <linux/sched/task.h>
+#include <linux/random.h>
 
 #include <asm/pgtable.h>
 #include <asm/tlbflush.h>
@@ -39,6 +41,37 @@ int raw_patch_instruction(unsigned int *addr, unsigned int instr)
 }
 
 #ifdef CONFIG_STRICT_KERNEL_RWX
+
+static struct mm_struct *patching_mm __ro_after_init;
+static unsigned long patching_addr __ro_after_init;
+
+void __init poking_init(void)
+{
+	spinlock_t *ptl; /* for protecting pte table */
+	pte_t *ptep;
+
+	/*
+	 * Some parts of the kernel (static keys for example) depend on
+	 * successful code patching. Code patching under STRICT_KERNEL_RWX
+	 * requires this setup - otherwise we cannot patch at all. We use
+	 * BUG_ON() here and later since an early failure is preferred to
+	 * buggy behavior and/or strange crashes later.
+	 */
+	patching_mm = copy_init_mm();
+	BUG_ON(!patching_mm);
+
+	/*
+	 * In hash we cannot go above DEFAULT_MAP_WINDOW easily.
+	 * XXX: Do we want additional bits of entropy for radix?
+	 */
+	patching_addr = (get_random_long() & PAGE_MASK) %
+		(DEFAULT_MAP_WINDOW - PAGE_SIZE);
+
+	ptep = get_locked_pte(patching_mm, patching_addr, &ptl);
+	BUG_ON(!ptep);
+	pte_unmap_unlock(ptep, ptl);
+}
+
 static DEFINE_PER_CPU(struct vm_struct *, text_poke_area);
 
 static int text_area_cpu_up(unsigned int cpu)
Signature · 1 line
2.26.1
#7 lore raw

RFC PATCH v2 3/5 powerpc/lib: Use a temporary mm for code patching

In reply to Christopher M. Riedl

Currently, code patching a STRICT_KERNEL_RWX exposes the temporary mappings to other CPUs. These mappings should be kept local to the CPU doing the patching. Use the pre-initialized temporary mm and patching address for this purpose. Also add a check after patching to ensure the patch succeeded.

Use the KUAP functions on non-BOOKS3_64 platforms since the temporary mapping for patching uses a userspace address (to keep the mapping local). On BOOKS3_64 platforms hash does not implement KUAP and on radix the use of PAGE_KERNEL sets EAA[0] for the PTE which means the AMR (KUAP) protection is ignored (see PowerISA v3.0b, Fig, 35).

Based on x86 implementation:

commit b3fd8e83ada0
("x86/alternatives: Use temporary mm for text poking")

---
 arch/powerpc/lib/code-patching.c | 149 ++++++++++++-------------------
 1 file changed, 55 insertions(+), 94 deletions(-)
arch/powerpc/lib/code-patching.c +55 -94
index 259c19480a85..26f06cdb5d7e 100644
@@ -19,6 +19,7 @@
 #include <asm/page.h>
 #include <asm/code-patching.h>
 #include <asm/setup.h>
+#include <asm/mmu_context.h>
 
 static int __patch_instruction(unsigned int *exec_addr, unsigned int instr,
 			       unsigned int *patch_addr)
@@ -72,101 +73,58 @@ void __init poking_init(void)
 	pte_unmap_unlock(ptep, ptl);
 }
 
-static DEFINE_PER_CPU(struct vm_struct *, text_poke_area);
-
-static int text_area_cpu_up(unsigned int cpu)
-{
-	struct vm_struct *area;
-
-	area = get_vm_area(PAGE_SIZE, VM_ALLOC);
-	if (!area) {
-		WARN_ONCE(1, "Failed to create text area for cpu %d\n",
-			cpu);
-		return -1;
-	}
-	this_cpu_write(text_poke_area, area);
-
-	return 0;
-}
-
-static int text_area_cpu_down(unsigned int cpu)
-{
-	free_vm_area(this_cpu_read(text_poke_area));
-	return 0;
-}
-
-/*
- * Run as a late init call. This allows all the boot time patching to be done
- * simply by patching the code, and then we're called here prior to
- * mark_rodata_ro(), which happens after all init calls are run. Although
- * BUG_ON() is rude, in this case it should only happen if ENOMEM, and we judge
- * it as being preferable to a kernel that will crash later when someone tries
- * to use patch_instruction().
- */
-static int __init setup_text_poke_area(void)
-{
-	BUG_ON(!cpuhp_setup_state(CPUHP_AP_ONLINE_DYN,
-		"powerpc/text_poke:online", text_area_cpu_up,
-		text_area_cpu_down));
-
-	return 0;
-}
-late_initcall(setup_text_poke_area);
+struct patch_mapping {
+	spinlock_t *ptl; /* for protecting pte table */
+	pte_t *ptep;
+	struct temp_mm temp_mm;
+};
 
 /*
  * This can be called for kernel text or a module.
  */
-static int map_patch_area(void *addr, unsigned long text_poke_addr)
+static int map_patch(const void *addr, struct patch_mapping *patch_mapping)
 {
-	unsigned long pfn;
-	int err;
+	struct page *page;
+	pte_t pte;
+	pgprot_t pgprot;
 
 	if (is_vmalloc_addr(addr))
-		pfn = vmalloc_to_pfn(addr);
+		page = vmalloc_to_page(addr);
 	else
-		pfn = __pa_symbol(addr) >> PAGE_SHIFT;
+		page = virt_to_page(addr);
 
-	err = map_kernel_page(text_poke_addr, (pfn << PAGE_SHIFT), PAGE_KERNEL);
+	if (radix_enabled())
+		pgprot = PAGE_KERNEL;
+	else
+		pgprot = PAGE_SHARED;
 
-	pr_devel("Mapped addr %lx with pfn %lx:%d\n", text_poke_addr, pfn, err);
-	if (err)
+	patch_mapping->ptep = get_locked_pte(patching_mm, patching_addr,
+					     &patch_mapping->ptl);
+	if (unlikely(!patch_mapping->ptep)) {
+		pr_warn("map patch: failed to allocate pte for patching\n");
 		return -1;
+	}
+
+	pte = mk_pte(page, pgprot);
+	if (!IS_ENABLED(CONFIG_PPC_BOOK3S_64))
+		pte = pte_mkdirty(pte);
+	set_pte_at(patching_mm, patching_addr, patch_mapping->ptep, pte);
+
+	init_temp_mm(&patch_mapping->temp_mm, patching_mm);
+	use_temporary_mm(&patch_mapping->temp_mm);
 
 	return 0;
 }
 
-static inline int unmap_patch_area(unsigned long addr)
+static void unmap_patch(struct patch_mapping *patch_mapping)
 {
-	pte_t *ptep;
-	pmd_t *pmdp;
-	pud_t *pudp;
-	pgd_t *pgdp;
-
-	pgdp = pgd_offset_k(addr);
-	if (unlikely(!pgdp))
-		return -EINVAL;
-
-	pudp = pud_offset(pgdp, addr);
-	if (unlikely(!pudp))
-		return -EINVAL;
-
-	pmdp = pmd_offset(pudp, addr);
-	if (unlikely(!pmdp))
-		return -EINVAL;
-
-	ptep = pte_offset_kernel(pmdp, addr);
-	if (unlikely(!ptep))
-		return -EINVAL;
+	/* In hash, pte_clear flushes the tlb */
+	pte_clear(patching_mm, patching_addr, patch_mapping->ptep);
+	unuse_temporary_mm(&patch_mapping->temp_mm);
 
-	pr_devel("clearing mm %p, pte %p, addr %lx\n", &init_mm, ptep, addr);
-
-	/*
-	 * In hash, pte_clear flushes the tlb, in radix, we have to
-	 */
-	pte_clear(&init_mm, addr, ptep);
-	flush_tlb_kernel_range(addr, addr + PAGE_SIZE);
-
-	return 0;
+	/* In radix, we have to explicitly flush the tlb (no-op in hash) */
+	local_flush_tlb_mm(patching_mm);
+	pte_unmap_unlock(patch_mapping->ptep, patch_mapping->ptl);
 }
 
 static int do_patch_instruction(unsigned int *addr, unsigned int instr)
@@ -174,33 +132,36 @@ static int do_patch_instruction(unsigned int *addr, unsigned int instr)
 	int err;
 	unsigned int *patch_addr = NULL;
 	unsigned long flags;
-	unsigned long text_poke_addr;
-	unsigned long kaddr = (unsigned long)addr;
+	struct patch_mapping patch_mapping;
 
 	/*
-	 * During early early boot patch_instruction is called
-	 * when text_poke_area is not ready, but we still need
-	 * to allow patching. We just do the plain old patching
+	 * The patching_mm is initialized before calling mark_rodata_ro. Prior
+	 * to this, patch_instruction is called when we don't have (and don't
+	 * need) the patching_mm so just do plain old patching.
 	 */
-	if (!this_cpu_read(text_poke_area))
+	if (!patching_mm)
 		return raw_patch_instruction(addr, instr);
 
 	local_irq_save(flags);
 
-	text_poke_addr = (unsigned long)__this_cpu_read(text_poke_area)->addr;
-	if (map_patch_area(addr, text_poke_addr)) {
-		err = -1;
+	err = map_patch(addr, &patch_mapping);
+	if (err)
 		goto out;
-	}
 
-	patch_addr = (unsigned int *)(text_poke_addr) +
-			((kaddr & ~PAGE_MASK) / sizeof(unsigned int));
+	patch_addr = (unsigned int *)(patching_addr | offset_in_page(addr));
 
-	__patch_instruction(addr, instr, patch_addr);
+	if (!radix_enabled())
+		allow_write_to_user(patch_addr, sizeof(instr));
+	err = __patch_instruction(addr, instr, patch_addr);
+	if (!radix_enabled())
+		prevent_write_to_user(patch_addr, sizeof(instr));
 
-	err = unmap_patch_area(text_poke_addr);
-	if (err)
-		pr_warn("failed to unmap %lx\n", text_poke_addr);
+	unmap_patch(&patch_mapping);
+	/*
+	 * Something is wrong if what we just wrote doesn't match what we
+	 * think we just wrote.
+	 */
+	WARN_ON(*addr != instr);
 
 out:
 	local_irq_restore(flags);
Signature · 1 line
2.26.1
#8 lore raw

Re: [RFC PATCH v2 3/5] powerpc/lib: Use a temporary mm for code patching

Le 29/04/2020 à 04:05, Christopher M. Riedl a écrit :

Quoted 111 lines

Currently, code patching a STRICT_KERNEL_RWX exposes the temporary mappings to other CPUs. These mappings should be kept local to the CPU doing the patching. Use the pre-initialized temporary mm and patching address for this purpose. Also add a check after patching to ensure the patch succeeded.

Use the KUAP functions on non-BOOKS3_64 platforms since the temporary mapping for patching uses a userspace address (to keep the mapping local). On BOOKS3_64 platforms hash does not implement KUAP and on radix the use of PAGE_KERNEL sets EAA[0] for the PTE which means the AMR (KUAP) protection is ignored (see PowerISA v3.0b, Fig, 35).

Based on x86 implementation:

commit b3fd8e83ada0
("x86/alternatives: Use temporary mm for text poking")

---
  arch/powerpc/lib/code-patching.c | 149 ++++++++++++-------------------
  1 file changed, 55 insertions(+), 94 deletions(-)
arch/powerpc/lib/code-patching.c +16 -48
index 259c19480a85..26f06cdb5d7e 100644
@@ -19,6 +19,7 @@
  #include <asm/page.h>
  #include <asm/code-patching.h>
  #include <asm/setup.h>
+#include <asm/mmu_context.h>
 
  static int __patch_instruction(unsigned int *exec_addr, unsigned int instr,
  			       unsigned int *patch_addr)
@@ -72,101 +73,58 @@ void __init poking_init(void)
  	pte_unmap_unlock(ptep, ptl);
  }
 
-static DEFINE_PER_CPU(struct vm_struct *, text_poke_area);
-
-static int text_area_cpu_up(unsigned int cpu)
-{
-	struct vm_struct *area;
-
-	area = get_vm_area(PAGE_SIZE, VM_ALLOC);
-	if (!area) {
-		WARN_ONCE(1, "Failed to create text area for cpu %d\n",
-			cpu);
-		return -1;
-	}
-	this_cpu_write(text_poke_area, area);
-
-	return 0;
-}
-
-static int text_area_cpu_down(unsigned int cpu)
-{
-	free_vm_area(this_cpu_read(text_poke_area));
-	return 0;
-}
-
-/*
- * Run as a late init call. This allows all the boot time patching to be done
- * simply by patching the code, and then we're called here prior to
- * mark_rodata_ro(), which happens after all init calls are run. Although
- * BUG_ON() is rude, in this case it should only happen if ENOMEM, and we judge
- * it as being preferable to a kernel that will crash later when someone tries
- * to use patch_instruction().
- */
-static int __init setup_text_poke_area(void)
-{
-	BUG_ON(!cpuhp_setup_state(CPUHP_AP_ONLINE_DYN,
-		"powerpc/text_poke:online", text_area_cpu_up,
-		text_area_cpu_down));
-
-	return 0;
-}
-late_initcall(setup_text_poke_area);
+struct patch_mapping {
+	spinlock_t *ptl; /* for protecting pte table */
+	pte_t *ptep;
+	struct temp_mm temp_mm;
+};
 
  /*
   * This can be called for kernel text or a module.
   */
-static int map_patch_area(void *addr, unsigned long text_poke_addr)
+static int map_patch(const void *addr, struct patch_mapping *patch_mapping)
  {
-	unsigned long pfn;
-	int err;
+	struct page *page;
+	pte_t pte;
+	pgprot_t pgprot;
 
  	if (is_vmalloc_addr(addr))
-		pfn = vmalloc_to_pfn(addr);
+		page = vmalloc_to_page(addr);
  	else
-		pfn = __pa_symbol(addr) >> PAGE_SHIFT;
+		page = virt_to_page(addr);
 
-	err = map_kernel_page(text_poke_addr, (pfn << PAGE_SHIFT), PAGE_KERNEL);
+	if (radix_enabled())
+		pgprot = PAGE_KERNEL;
+	else
+		pgprot = PAGE_SHARED;
 
-	pr_devel("Mapped addr %lx with pfn %lx:%d\n", text_poke_addr, pfn, err);
-	if (err)

arch/powerpc/lib/code-patching.c void __init poking_init(void) · line 110

Quoted 10 lines
arch/powerpc/lib/code-patching.c +9 -0
+	patch_mapping->ptep = get_locked_pte(patching_mm, patching_addr,
+					     &patch_mapping->ptl);
+	if (unlikely(!patch_mapping->ptep)) {
+		pr_warn("map patch: failed to allocate pte for patching\n");
  		return -1;
+	}
+
+	pte = mk_pte(page, pgprot);
+	if (!IS_ENABLED(CONFIG_PPC_BOOK3S_64))
+		pte = pte_mkdirty(pte);

Why only when CONFIG_PPC_BOOK3S_64 is not set ?

PAGE_KERNEL should already be dirty, so making it dirty all the time shouldn't hurt.

Quoted 94 lines
arch/powerpc/lib/code-patching.c +24 -45
+	set_pte_at(patching_mm, patching_addr, patch_mapping->ptep, pte);
+
+	init_temp_mm(&patch_mapping->temp_mm, patching_mm);
+	use_temporary_mm(&patch_mapping->temp_mm);
 
  	return 0;
  }
 
-static inline int unmap_patch_area(unsigned long addr)
+static void unmap_patch(struct patch_mapping *patch_mapping)
  {
-	pte_t *ptep;
-	pmd_t *pmdp;
-	pud_t *pudp;
-	pgd_t *pgdp;
-
-	pgdp = pgd_offset_k(addr);
-	if (unlikely(!pgdp))
-		return -EINVAL;
-
-	pudp = pud_offset(pgdp, addr);
-	if (unlikely(!pudp))
-		return -EINVAL;
-
-	pmdp = pmd_offset(pudp, addr);
-	if (unlikely(!pmdp))
-		return -EINVAL;
-
-	ptep = pte_offset_kernel(pmdp, addr);
-	if (unlikely(!ptep))
-		return -EINVAL;
+	/* In hash, pte_clear flushes the tlb */
+	pte_clear(patching_mm, patching_addr, patch_mapping->ptep);
+	unuse_temporary_mm(&patch_mapping->temp_mm);
 
-	pr_devel("clearing mm %p, pte %p, addr %lx\n", &init_mm, ptep, addr);
-
-	/*
-	 * In hash, pte_clear flushes the tlb, in radix, we have to
-	 */
-	pte_clear(&init_mm, addr, ptep);
-	flush_tlb_kernel_range(addr, addr + PAGE_SIZE);
-
-	return 0;
+	/* In radix, we have to explicitly flush the tlb (no-op in hash) */
+	local_flush_tlb_mm(patching_mm);
+	pte_unmap_unlock(patch_mapping->ptep, patch_mapping->ptl);
  }
 
  static int do_patch_instruction(unsigned int *addr, unsigned int instr)
@@ -174,33 +132,36 @@ static int do_patch_instruction(unsigned int *addr, unsigned int instr)
  	int err;
  	unsigned int *patch_addr = NULL;
  	unsigned long flags;
-	unsigned long text_poke_addr;
-	unsigned long kaddr = (unsigned long)addr;
+	struct patch_mapping patch_mapping;
 
  	/*
-	 * During early early boot patch_instruction is called
-	 * when text_poke_area is not ready, but we still need
-	 * to allow patching. We just do the plain old patching
+	 * The patching_mm is initialized before calling mark_rodata_ro. Prior
+	 * to this, patch_instruction is called when we don't have (and don't
+	 * need) the patching_mm so just do plain old patching.
  	 */
-	if (!this_cpu_read(text_poke_area))
+	if (!patching_mm)
  		return raw_patch_instruction(addr, instr);
 
  	local_irq_save(flags);
 
-	text_poke_addr = (unsigned long)__this_cpu_read(text_poke_area)->addr;
-	if (map_patch_area(addr, text_poke_addr)) {
-		err = -1;
+	err = map_patch(addr, &patch_mapping);
+	if (err)
  		goto out;
-	}
 
-	patch_addr = (unsigned int *)(text_poke_addr) +
-			((kaddr & ~PAGE_MASK) / sizeof(unsigned int));
+	patch_addr = (unsigned int *)(patching_addr | offset_in_page(addr));
 
-	__patch_instruction(addr, instr, patch_addr);
+	if (!radix_enabled())
+		allow_write_to_user(patch_addr, sizeof(instr));
+	err = __patch_instruction(addr, instr, patch_addr);
+	if (!radix_enabled())
+		prevent_write_to_user(patch_addr, sizeof(instr));
 
-	err = unmap_patch_area(text_poke_addr);
-	if (err)

arch/powerpc/lib/code-patching.c static int do_patch_instruction(unsigned int *addr, unsigned int instr) · line 167

Quoted 10 lines
arch/powerpc/lib/code-patching.c +6 -1
-		pr_warn("failed to unmap %lx\n", text_poke_addr);
+	unmap_patch(&patch_mapping);
+	/*
+	 * Something is wrong if what we just wrote doesn't match what we
+	 * think we just wrote.
+	 */
+	WARN_ON(*addr != instr);
 
  out:
  	local_irq_restore(flags);

Christophe

#9 lore raw

Re: [RFC PATCH v2 3/5] powerpc/lib: Use a temporary mm for code patching

Quoted 131 lines · Christophe Leroy
Quoted 111 lines · Christopher M. Riedl

Currently, code patching a STRICT_KERNEL_RWX exposes the temporary mappings to other CPUs. These mappings should be kept local to the CPU doing the patching. Use the pre-initialized temporary mm and patching address for this purpose. Also add a check after patching to ensure the patch succeeded.

Use the KUAP functions on non-BOOKS3_64 platforms since the temporary mapping for patching uses a userspace address (to keep the mapping local). On BOOKS3_64 platforms hash does not implement KUAP and on radix the use of PAGE_KERNEL sets EAA[0] for the PTE which means the AMR (KUAP) protection is ignored (see PowerISA v3.0b, Fig, 35).

Based on x86 implementation:

commit b3fd8e83ada0
("x86/alternatives: Use temporary mm for text poking")

---
  arch/powerpc/lib/code-patching.c | 149 ++++++++++++-------------------
  1 file changed, 55 insertions(+), 94 deletions(-)
arch/powerpc/lib/code-patching.c +16 -48
index 259c19480a85..26f06cdb5d7e 100644
@@ -19,6 +19,7 @@
  #include <asm/page.h>
  #include <asm/code-patching.h>
  #include <asm/setup.h>
+#include <asm/mmu_context.h>
 
  static int __patch_instruction(unsigned int *exec_addr, unsigned int instr,
  			       unsigned int *patch_addr)
@@ -72,101 +73,58 @@ void __init poking_init(void)
  	pte_unmap_unlock(ptep, ptl);
  }
 
-static DEFINE_PER_CPU(struct vm_struct *, text_poke_area);
-
-static int text_area_cpu_up(unsigned int cpu)
-{
-	struct vm_struct *area;
-
-	area = get_vm_area(PAGE_SIZE, VM_ALLOC);
-	if (!area) {
-		WARN_ONCE(1, "Failed to create text area for cpu %d\n",
-			cpu);
-		return -1;
-	}
-	this_cpu_write(text_poke_area, area);
-
-	return 0;
-}
-
-static int text_area_cpu_down(unsigned int cpu)
-{
-	free_vm_area(this_cpu_read(text_poke_area));
-	return 0;
-}
-
-/*
- * Run as a late init call. This allows all the boot time patching to be done
- * simply by patching the code, and then we're called here prior to
- * mark_rodata_ro(), which happens after all init calls are run. Although
- * BUG_ON() is rude, in this case it should only happen if ENOMEM, and we judge
- * it as being preferable to a kernel that will crash later when someone tries
- * to use patch_instruction().
- */
-static int __init setup_text_poke_area(void)
-{
-	BUG_ON(!cpuhp_setup_state(CPUHP_AP_ONLINE_DYN,
-		"powerpc/text_poke:online", text_area_cpu_up,
-		text_area_cpu_down));
-
-	return 0;
-}
-late_initcall(setup_text_poke_area);
+struct patch_mapping {
+	spinlock_t *ptl; /* for protecting pte table */
+	pte_t *ptep;
+	struct temp_mm temp_mm;
+};
 
  /*
   * This can be called for kernel text or a module.
   */
-static int map_patch_area(void *addr, unsigned long text_poke_addr)
+static int map_patch(const void *addr, struct patch_mapping *patch_mapping)
  {
-	unsigned long pfn;
-	int err;
+	struct page *page;
+	pte_t pte;
+	pgprot_t pgprot;
 
  	if (is_vmalloc_addr(addr))
-		pfn = vmalloc_to_pfn(addr);
+		page = vmalloc_to_page(addr);
  	else
-		pfn = __pa_symbol(addr) >> PAGE_SHIFT;
+		page = virt_to_page(addr);
 
-	err = map_kernel_page(text_poke_addr, (pfn << PAGE_SHIFT), PAGE_KERNEL);
+	if (radix_enabled())
+		pgprot = PAGE_KERNEL;
+	else
+		pgprot = PAGE_SHARED;
 
-	pr_devel("Mapped addr %lx with pfn %lx:%d\n", text_poke_addr, pfn, err);
-	if (err)

arch/powerpc/lib/code-patching.c void __init poking_init(void) · line 110

Quoted 10 lines
arch/powerpc/lib/code-patching.c +9 -0
+	patch_mapping->ptep = get_locked_pte(patching_mm, patching_addr,
+					     &patch_mapping->ptl);
+	if (unlikely(!patch_mapping->ptep)) {
+		pr_warn("map patch: failed to allocate pte for patching\n");
  		return -1;
+	}
+
+	pte = mk_pte(page, pgprot);
+	if (!IS_ENABLED(CONFIG_PPC_BOOK3S_64))
+		pte = pte_mkdirty(pte);

Why only when CONFIG_PPC_BOOK3S_64 is not set ?

Quoted 4 lines

PAGE_KERNEL should already be dirty, so making it dirty all the time shouldn't hurt.

Ok, I'll remove this check to simplify.

Quoted 111 lines
Quoted 94 lines
arch/powerpc/lib/code-patching.c +24 -45
+	set_pte_at(patching_mm, patching_addr, patch_mapping->ptep, pte);
+
+	init_temp_mm(&patch_mapping->temp_mm, patching_mm);
+	use_temporary_mm(&patch_mapping->temp_mm);
 
  	return 0;
  }
 
-static inline int unmap_patch_area(unsigned long addr)
+static void unmap_patch(struct patch_mapping *patch_mapping)
  {
-	pte_t *ptep;
-	pmd_t *pmdp;
-	pud_t *pudp;
-	pgd_t *pgdp;
-
-	pgdp = pgd_offset_k(addr);
-	if (unlikely(!pgdp))
-		return -EINVAL;
-
-	pudp = pud_offset(pgdp, addr);
-	if (unlikely(!pudp))
-		return -EINVAL;
-
-	pmdp = pmd_offset(pudp, addr);
-	if (unlikely(!pmdp))
-		return -EINVAL;
-
-	ptep = pte_offset_kernel(pmdp, addr);
-	if (unlikely(!ptep))
-		return -EINVAL;
+	/* In hash, pte_clear flushes the tlb */
+	pte_clear(patching_mm, patching_addr, patch_mapping->ptep);
+	unuse_temporary_mm(&patch_mapping->temp_mm);
 
-	pr_devel("clearing mm %p, pte %p, addr %lx\n", &init_mm, ptep, addr);
-
-	/*
-	 * In hash, pte_clear flushes the tlb, in radix, we have to
-	 */
-	pte_clear(&init_mm, addr, ptep);
-	flush_tlb_kernel_range(addr, addr + PAGE_SIZE);
-
-	return 0;
+	/* In radix, we have to explicitly flush the tlb (no-op in hash) */
+	local_flush_tlb_mm(patching_mm);
+	pte_unmap_unlock(patch_mapping->ptep, patch_mapping->ptl);
  }
 
  static int do_patch_instruction(unsigned int *addr, unsigned int instr)
@@ -174,33 +132,36 @@ static int do_patch_instruction(unsigned int *addr, unsigned int instr)
  	int err;
  	unsigned int *patch_addr = NULL;
  	unsigned long flags;
-	unsigned long text_poke_addr;
-	unsigned long kaddr = (unsigned long)addr;
+	struct patch_mapping patch_mapping;
 
  	/*
-	 * During early early boot patch_instruction is called
-	 * when text_poke_area is not ready, but we still need
-	 * to allow patching. We just do the plain old patching
+	 * The patching_mm is initialized before calling mark_rodata_ro. Prior
+	 * to this, patch_instruction is called when we don't have (and don't
+	 * need) the patching_mm so just do plain old patching.
  	 */
-	if (!this_cpu_read(text_poke_area))
+	if (!patching_mm)
  		return raw_patch_instruction(addr, instr);
 
  	local_irq_save(flags);
 
-	text_poke_addr = (unsigned long)__this_cpu_read(text_poke_area)->addr;
-	if (map_patch_area(addr, text_poke_addr)) {
-		err = -1;
+	err = map_patch(addr, &patch_mapping);
+	if (err)
  		goto out;
-	}
 
-	patch_addr = (unsigned int *)(text_poke_addr) +
-			((kaddr & ~PAGE_MASK) / sizeof(unsigned int));
+	patch_addr = (unsigned int *)(patching_addr | offset_in_page(addr));
 
-	__patch_instruction(addr, instr, patch_addr);
+	if (!radix_enabled())
+		allow_write_to_user(patch_addr, sizeof(instr));
+	err = __patch_instruction(addr, instr, patch_addr);
+	if (!radix_enabled())
+		prevent_write_to_user(patch_addr, sizeof(instr));
 
-	err = unmap_patch_area(text_poke_addr);
-	if (err)

arch/powerpc/lib/code-patching.c static int do_patch_instruction(unsigned int *addr, unsigned int instr) · line 167

Quoted 10 lines
arch/powerpc/lib/code-patching.c +6 -1
-		pr_warn("failed to unmap %lx\n", text_poke_addr);
+	unmap_patch(&patch_mapping);
+	/*
+	 * Something is wrong if what we just wrote doesn't match what we
+	 * think we just wrote.
+	 */
+	WARN_ON(*addr != instr);
 
  out:
  	local_irq_restore(flags);

Christophe

#10 lore raw

RFC PATCH v2 4/5 powerpc/lib: Add LKDTM accessor for patching addr

In reply to Christopher M. Riedl

When live patching a STRICT_RWX kernel, a mapping is installed at a "patching address" with temporary write permissions. Provide a LKDTM-only accessor function for this address in preparation for a LKDTM test which attempts to "hijack" this mapping by writing to it from another CPU.

---
 arch/powerpc/lib/code-patching.c | 7 +++++++
 1 file changed, 7 insertions(+)
arch/powerpc/lib/code-patching.c +7 -0
index 26f06cdb5d7e..cfbdef90384e 100644
@@ -46,6 +46,13 @@ int raw_patch_instruction(unsigned int *addr, unsigned int instr)
 static struct mm_struct *patching_mm __ro_after_init;
 static unsigned long patching_addr __ro_after_init;
 
+#ifdef CONFIG_LKDTM
+unsigned long read_cpu_patching_addr(unsigned int cpu)
+{
+	return patching_addr;
+}
+#endif
+
 void __init poking_init(void)
 {
 	spinlock_t *ptl; /* for protecting pte table */
Signature · 1 line
2.26.1
#11 lore raw

RFC PATCH v2 5/5 powerpc: Add LKDTM test to hijack a patch mapping

In reply to Christopher M. Riedl

When live patching with STRICT_KERNEL_RWX, the CPU doing the patching must use a temporary mapping which allows for writing to kernel text. During the entire window of time when this temporary mapping is in use, another CPU could write to the same mapping and maliciously alter kernel text. Implement a LKDTM test to attempt to exploit such a openings when a CPU is patching under STRICT_KERNEL_RWX. The test is only implemented on powerpc for now.

The LKDTM "hijack" test works as follows:

1. A CPU executes an infinite loop to patch an instruction. This is the "patching" CPU.
2. Another CPU attempts to write to the address of the temporary mapping used by the "patching" CPU. This other CPU is the "hijacker" CPU. The hijack either fails with a segfault or succeeds, in which case some kernel text is now overwritten.

How to run the test:

mount -t debugfs none /sys/kernel/debug
(echo HIJACK_PATCH > /sys/kernel/debug/provoke-crash/DIRECT)

---
 drivers/misc/lkdtm/core.c  |  1 +
 drivers/misc/lkdtm/lkdtm.h |  1 +
 drivers/misc/lkdtm/perms.c | 99 ++++++++++++++++++++++++++++++++++++++
 3 files changed, 101 insertions(+)
drivers/misc/lkdtm/core.c +1 -0
index a5e344df9166..482e72f6a1e1 100644
@@ -145,6 +145,7 @@ static const struct crashtype crashtypes[] = {
 	CRASHTYPE(WRITE_RO),
 	CRASHTYPE(WRITE_RO_AFTER_INIT),
 	CRASHTYPE(WRITE_KERN),
+	CRASHTYPE(HIJACK_PATCH),
 	CRASHTYPE(REFCOUNT_INC_OVERFLOW),
 	CRASHTYPE(REFCOUNT_ADD_OVERFLOW),
 	CRASHTYPE(REFCOUNT_INC_NOT_ZERO_OVERFLOW),
drivers/misc/lkdtm/lkdtm.h +1 -0
index 601a2156a0d4..bfcf3542370d 100644
@@ -62,6 +62,7 @@ void lkdtm_EXEC_USERSPACE(void);
 void lkdtm_EXEC_NULL(void);
 void lkdtm_ACCESS_USERSPACE(void);
 void lkdtm_ACCESS_NULL(void);
+void lkdtm_HIJACK_PATCH(void);
 
 /* lkdtm_refcount.c */
 void lkdtm_REFCOUNT_INC_OVERFLOW(void);
drivers/misc/lkdtm/perms.c +99 -0

111 lines not on this page · raw message

Signature · 1 line
2.26.1