Skip to contentSkip to conversation

Support strict kernel memory permissions for security

PATCH In review Zong Li 6 participants 20 messages Apr 8, 2020 – Apr 21, 2020

Subsystems (2)
lore
Share & tools
lore

Zong Li

author
#0 lore raw

v5 cover Support strict kernel memory permissions for security

The main purpose of this patch series is changing the kernel mapping permission , make sure that code is not writeable, data is not executable, and read-only data is neither writable nor executable.

This patch series also supports the relevant implementations such as ARCH_HAS_SET_MEMORY, ARCH_HAS_SET_DIRECT_MAP, ARCH_SUPPORTS_DEBUG_PAGEALLOC and DEBUG_WX.

Changes in v5:
- Add lockdep_assert_held and more comments for text_mutex.

Changes in v4:
- Use NOKPROBE_SYMBOL instead of __kprobe annotation
- Use text_mutex instead of patch_lock
- Remove 'riscv_' prefix of function name

Changes in v3:
- Fix build error on nommu configuration. We already support nommu on RISC-V, so we should consider nommu case and test not only rv32/64, but also nommu.

Changes in v2:
- Use _data to specify the start of data section with write permission.
- Change ftrace patch text implementaion.
- Separate DEBUG_WX patch to another patchset.

Zong Li (9):
riscv: add macro to get instruction length
riscv: introduce interfaces to patch kernel code
riscv: patch code by fixmap mapping
riscv: add ARCH_HAS_SET_MEMORY support
riscv: add ARCH_HAS_SET_DIRECT_MAP support
riscv: add ARCH_SUPPORTS_DEBUG_PAGEALLOC support
riscv: move exception table immediately after RO_DATA
riscv: add alignment for text, rodata and data sections
riscv: add STRICT_KERNEL_RWX support

 arch/riscv/Kconfig                  |   6 +
 arch/riscv/include/asm/bug.h        |   8 ++
 arch/riscv/include/asm/fixmap.h     |   2 +
 arch/riscv/include/asm/patch.h      |  12 ++
 arch/riscv/include/asm/set_memory.h |  48 +++++++
 arch/riscv/kernel/Makefile          |   4 +-
 arch/riscv/kernel/ftrace.c          |  26 ++--
 arch/riscv/kernel/patch.c           | 128 +++++++++++++++++++
 arch/riscv/kernel/traps.c           |   3 +-
 arch/riscv/kernel/vmlinux.lds.S     |  11 +-
 arch/riscv/mm/Makefile              |   2 +-
 arch/riscv/mm/init.c                |  44 +++++++
 arch/riscv/mm/pageattr.c            | 187 ++++++++++++++++++++++++++++
 13 files changed, 466 insertions(+), 15 deletions(-)
 create mode 100644 arch/riscv/include/asm/patch.h
 create mode 100644 arch/riscv/include/asm/set_memory.h
 create mode 100644 arch/riscv/kernel/patch.c
 create mode 100644 arch/riscv/mm/pageattr.c
Signature · 1 line
2.26.0
author
#1 lore raw

PATCH v5 1/9 riscv: add macro to get instruction length

Extract the calculation of instruction length for common use.

---
 arch/riscv/include/asm/bug.h | 8 ++++++++
 arch/riscv/kernel/traps.c    | 3 ++-
 2 files changed, 10 insertions(+), 1 deletion(-)
arch/riscv/include/asm/bug.h +8 -0
index 75604fec1b1b..d6f1ec08d97b 100644
@@ -19,6 +19,14 @@
 #define __BUG_INSN_32	_UL(0x00100073) /* ebreak */
 #define __BUG_INSN_16	_UL(0x9002) /* c.ebreak */
 
+#define GET_INSN_LENGTH(insn)						\
+({									\
+	unsigned long __len;						\
+	__len = ((insn & __INSN_LENGTH_MASK) == __INSN_LENGTH_32) ?	\
+		4UL : 2UL;						\
+	__len;								\
+})
+
 typedef u32 bug_insn_t;
 
 #ifdef CONFIG_GENERIC_BUG_RELATIVE_POINTERS
arch/riscv/kernel/traps.c +2 -1
index ffb3d94bf0cc..a4d136355f78 100644
@@ -118,7 +118,8 @@ static inline unsigned long get_break_insn_length(unsigned long pc)
 
 	if (probe_kernel_address((bug_insn_t *)pc, insn))
 		return 0;
-	return (((insn & __INSN_LENGTH_MASK) == __INSN_LENGTH_32) ? 4UL : 2UL);
+
+	return GET_INSN_LENGTH(insn);
 }
 
 asmlinkage __visible void do_trap_break(struct pt_regs *regs)
Signature · 1 line
2.26.0
maintainer Reviewed-by on v5 1/9
#2 lore raw

Re: [PATCH v5 1/9] riscv: add macro to get instruction length

Quoted 32 lines · On Wed, 08 Apr 2020 00:56:56 PDT (-0700), [email protected] wrote:

Extract the calculation of instruction length for common use.

---
 arch/riscv/include/asm/bug.h | 8 ++++++++
 arch/riscv/kernel/traps.c    | 3 ++-
 2 files changed, 10 insertions(+), 1 deletion(-)
arch/riscv/include/asm/bug.h +8 -0
index 75604fec1b1b..d6f1ec08d97b 100644
@@ -19,6 +19,14 @@
 #define __BUG_INSN_32	_UL(0x00100073) /* ebreak */
 #define __BUG_INSN_16	_UL(0x9002) /* c.ebreak */
 
+#define GET_INSN_LENGTH(insn)						\
+({									\
+	unsigned long __len;						\
+	__len = ((insn & __INSN_LENGTH_MASK) == __INSN_LENGTH_32) ?	\
+		4UL : 2UL;						\
+	__len;								\
+})
+
 typedef u32 bug_insn_t;
 
 #ifdef CONFIG_GENERIC_BUG_RELATIVE_POINTERS
arch/riscv/kernel/traps.c +0 -0
index ffb3d94bf0cc..a4d136355f78 100644

arch/riscv/kernel/traps.c static inline unsigned long get_break_insn_length(unsigned long pc) · line 125

Quoted 9 lines
arch/riscv/kernel/traps.c +2 -1
@@ -118,7 +118,8 @@ static inline unsigned long get_break_insn_length(unsigned long pc)
 
 	if (probe_kernel_address((bug_insn_t *)pc, insn))
 		return 0;
-	return (((insn & __INSN_LENGTH_MASK) == __INSN_LENGTH_32) ? 4UL : 2UL);
+
+	return GET_INSN_LENGTH(insn);
 }
 
 asmlinkage __visible void do_trap_break(struct pt_regs *regs)
author
#3 lore raw

PATCH v5 2/9 riscv: introduce interfaces to patch kernel code

In reply to Zong Li

On strict kernel memory permission, we couldn't patch code without writable permission. Preserve two holes in fixmap area, so we can map the kernel code temporarily to fixmap area, then patch the instructions.

We need two pages here because we support the compressed instruction, so the instruction might be align to 2 bytes. When patching the 32-bit length instruction which is 2 bytes alignment, it will across two pages.

Introduce two interfaces to patch kernel code:
riscv_patch_text_nosync:
- patch code without synchronization, it's caller's responsibility to synchronize all CPUs if needed.
riscv_patch_text:
- patch code and always synchronize with stop_machine()

---
 arch/riscv/include/asm/fixmap.h |   2 +
 arch/riscv/include/asm/patch.h  |  12 +++
 arch/riscv/kernel/Makefile      |   4 +-
 arch/riscv/kernel/patch.c       | 128 ++++++++++++++++++++++++++++++++
 4 files changed, 145 insertions(+), 1 deletion(-)
 create mode 100644 arch/riscv/include/asm/patch.h
 create mode 100644 arch/riscv/kernel/patch.c
arch/riscv/include/asm/fixmap.h +2 -0
index 42d2c42f3cc9..2368d49eb4ef 100644
@@ -27,6 +27,8 @@ enum fixed_addresses {
 	FIX_FDT = FIX_FDT_END + FIX_FDT_SIZE / PAGE_SIZE - 1,
 	FIX_PTE,
 	FIX_PMD,
+	FIX_TEXT_POKE1,
+	FIX_TEXT_POKE0,
 	FIX_EARLYCON_MEM_BASE,
 	__end_of_fixed_addresses
 };
arch/riscv/include/asm/patch.h new file +12 -0
new file mode 100644
index 000000000000..9a7d7346001e
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2020 SiFive
+ */
+
+#ifndef _ASM_RISCV_PATCH_H
+#define _ASM_RISCV_PATCH_H
+
+int patch_text_nosync(void *addr, const void *insns, size_t len);
+int patch_text(void *addr, u32 insn);
+
+#endif /* _ASM_RISCV_PATCH_H */
arch/riscv/kernel/Makefile +3 -1
index f40205cb9a22..d189bd3d8501 100644
@@ -4,7 +4,8 @@
 #
 
 ifdef CONFIG_FTRACE
-CFLAGS_REMOVE_ftrace.o = -pg
+CFLAGS_REMOVE_ftrace.o	= -pg
+CFLAGS_REMOVE_patch.o	= -pg
 endif
 
 extra-y += head.o
@@ -26,6 +27,7 @@ obj-y	+= traps.o
 obj-y	+= riscv_ksyms.o
 obj-y	+= stacktrace.o
 obj-y	+= cacheinfo.o
+obj-y	+= patch.o
 obj-$(CONFIG_MMU) += vdso.o vdso/
 
 obj-$(CONFIG_RISCV_M_MODE)	+= clint.o
arch/riscv/kernel/patch.c new file +128 -0

128 lines not on this page · raw message

Signature · 1 line
2.26.0
maintainer Reviewed-by on v5 2/9
#4 lore raw

Re: [PATCH v5 2/9] riscv: introduce interfaces to patch kernel code

Quoted 8 lines · On Wed, 8 Apr 2020 15:56:57 +0800 Zong Li <[email protected]> wrote:

On strict kernel memory permission, we couldn't patch code without writable permission. Preserve two holes in fixmap area, so we can map the kernel code temporarily to fixmap area, then patch the instructions.

We need two pages here because we support the compressed instruction, so the instruction might be align to 2 bytes. When patching the 32-bit length instruction which is 2 bytes alignment, it will across two pages.

Quoted 7 lines

Introduce two interfaces to patch kernel code:
riscv_patch_text_nosync:
- patch code without synchronization, it's caller's responsibility to synchronize all CPUs if needed.
riscv_patch_text:
- patch code and always synchronize with stop_machine()

Looks good to me.

Thank you,

Quoted 201 lines
---
 arch/riscv/include/asm/fixmap.h |   2 +
 arch/riscv/include/asm/patch.h  |  12 +++
 arch/riscv/kernel/Makefile      |   4 +-
 arch/riscv/kernel/patch.c       | 128 ++++++++++++++++++++++++++++++++
 4 files changed, 145 insertions(+), 1 deletion(-)
 create mode 100644 arch/riscv/include/asm/patch.h
 create mode 100644 arch/riscv/kernel/patch.c
arch/riscv/include/asm/fixmap.h +2 -0
index 42d2c42f3cc9..2368d49eb4ef 100644
@@ -27,6 +27,8 @@ enum fixed_addresses {
 	FIX_FDT = FIX_FDT_END + FIX_FDT_SIZE / PAGE_SIZE - 1,
 	FIX_PTE,
 	FIX_PMD,
+	FIX_TEXT_POKE1,
+	FIX_TEXT_POKE0,
 	FIX_EARLYCON_MEM_BASE,
 	__end_of_fixed_addresses
 };
arch/riscv/include/asm/patch.h new file +12 -0
new file mode 100644
index 000000000000..9a7d7346001e
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2020 SiFive
+ */
+
+#ifndef _ASM_RISCV_PATCH_H
+#define _ASM_RISCV_PATCH_H
+
+int patch_text_nosync(void *addr, const void *insns, size_t len);
+int patch_text(void *addr, u32 insn);
+
+#endif /* _ASM_RISCV_PATCH_H */
arch/riscv/kernel/Makefile +3 -1
index f40205cb9a22..d189bd3d8501 100644
@@ -4,7 +4,8 @@
 #
 
 ifdef CONFIG_FTRACE
-CFLAGS_REMOVE_ftrace.o = -pg
+CFLAGS_REMOVE_ftrace.o	= -pg
+CFLAGS_REMOVE_patch.o	= -pg
 endif
 
 extra-y += head.o
@@ -26,6 +27,7 @@ obj-y	+= traps.o
 obj-y	+= riscv_ksyms.o
 obj-y	+= stacktrace.o
 obj-y	+= cacheinfo.o
+obj-y	+= patch.o
 obj-$(CONFIG_MMU) += vdso.o vdso/
 
 obj-$(CONFIG_RISCV_M_MODE)	+= clint.o
arch/riscv/kernel/patch.c new file +128 -0
new file mode 100644
index 000000000000..f16466123947
@@ -0,0 +1,128 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2020 SiFive
+ */
+
+#include <linux/mm.h>
+#include <linux/memory.h>
+#include <linux/uaccess.h>
+#include <linux/spinlock.h>
+#include <linux/stop_machine.h>
+#include <asm/fixmap.h>
+#include <asm/kprobes.h>
+#include <asm/cacheflush.h>
+
+struct patch_insn_patch {
+	void *addr;
+	u32 insn;
+	atomic_t cpu_count;
+};
+
+#ifdef CONFIG_MMU
+static void *patch_map(void *addr, int fixmap)
+{
+	uintptr_t uintaddr = (uintptr_t) addr;
+	struct page *page;
+
+	if (core_kernel_text(uintaddr))
+		page = phys_to_page(__pa_symbol(addr));
+	else if (IS_ENABLED(CONFIG_STRICT_MODULE_RWX))
+		page = vmalloc_to_page(addr);
+	else
+		return addr;
+
+	BUG_ON(!page);
+
+	return (void *)set_fixmap_offset(fixmap, page_to_phys(page) +
+					 (uintaddr & ~PAGE_MASK));
+}
+NOKPROBE_SYMBOL(patch_map);
+
+static void patch_unmap(int fixmap)
+{
+	clear_fixmap(fixmap);
+}
+NOKPROBE_SYMBOL(patch_unmap);
+
+static int patch_insn_write(void *addr, const void *insn, size_t len)
+{
+	void *waddr = addr;
+	bool across_pages = (((uintptr_t) addr & ~PAGE_MASK) + len) > PAGE_SIZE;
+	int ret;
+
+	/*
+	 * Before reaching here, it was expected to lock the text_mutex
+	 * already, so we don't need to give another lock here and could
+	 * ensure that it was safe between each cores.
+	 */
+	lockdep_assert_held(&text_mutex);
+
+	if (across_pages)
+		patch_map(addr + len, FIX_TEXT_POKE1);
+
+	waddr = patch_map(addr, FIX_TEXT_POKE0);
+
+	ret = probe_kernel_write(waddr, insn, len);
+
+	patch_unmap(FIX_TEXT_POKE0);
+
+	if (across_pages)
+		patch_unmap(FIX_TEXT_POKE1);
+
+	return ret;
+}
+NOKPROBE_SYMBOL(patch_insn_write);
+#else
+static int patch_insn_write(void *addr, const void *insn, size_t len)
+{
+	return probe_kernel_write(addr, insn, len);
+}
+NOKPROBE_SYMBOL(patch_insn_write);
+#endif /* CONFIG_MMU */
+
+int patch_text_nosync(void *addr, const void *insns, size_t len)
+{
+	u32 *tp = addr;
+	int ret;
+
+	ret = patch_insn_write(tp, insns, len);
+
+	if (!ret)
+		flush_icache_range((uintptr_t) tp, (uintptr_t) tp + len);
+
+	return ret;
+}
+NOKPROBE_SYMBOL(patch_text_nosync);
+
+static int patch_text_cb(void *data)
+{
+	struct patch_insn_patch *patch = data;
+	int ret = 0;
+
+	if (atomic_inc_return(&patch->cpu_count) == 1) {
+		ret =
+		    patch_text_nosync(patch->addr, &patch->insn,
+					    GET_INSN_LENGTH(patch->insn));
+		atomic_inc(&patch->cpu_count);
+	} else {
+		while (atomic_read(&patch->cpu_count) <= num_online_cpus())
+			cpu_relax();
+		smp_mb();
+	}
+
+	return ret;
+}
+NOKPROBE_SYMBOL(patch_text_cb);
+
+int patch_text(void *addr, u32 insn)
+{
+	struct patch_insn_patch patch = {
+		.addr = addr,
+		.insn = insn,
+		.cpu_count = ATOMIC_INIT(0),
+	};
+
+	return stop_machine_cpuslocked(patch_text_cb,
+				       &patch, cpu_online_mask);
+}
+NOKPROBE_SYMBOL(patch_text);
Signature · 1 line
2.26.0
Signature · 1 line
Masami Hiramatsu <[email protected]>
maintainer Reviewed-by on v5 2/9
#5 lore raw

Re: [PATCH v5 2/9] riscv: introduce interfaces to patch kernel code

In reply to Zong Li
Quoted 203 lines · On Wed, 08 Apr 2020 00:56:57 PDT (-0700), [email protected] wrote:

On strict kernel memory permission, we couldn't patch code without writable permission. Preserve two holes in fixmap area, so we can map the kernel code temporarily to fixmap area, then patch the instructions.

We need two pages here because we support the compressed instruction, so the instruction might be align to 2 bytes. When patching the 32-bit length instruction which is 2 bytes alignment, it will across two pages.

Introduce two interfaces to patch kernel code:
riscv_patch_text_nosync:
- patch code without synchronization, it's caller's responsibility to synchronize all CPUs if needed.
riscv_patch_text:
- patch code and always synchronize with stop_machine()

---
 arch/riscv/include/asm/fixmap.h |   2 +
 arch/riscv/include/asm/patch.h  |  12 +++
 arch/riscv/kernel/Makefile      |   4 +-
 arch/riscv/kernel/patch.c       | 128 ++++++++++++++++++++++++++++++++
 4 files changed, 145 insertions(+), 1 deletion(-)
 create mode 100644 arch/riscv/include/asm/patch.h
 create mode 100644 arch/riscv/kernel/patch.c
arch/riscv/include/asm/fixmap.h +2 -0
index 42d2c42f3cc9..2368d49eb4ef 100644
@@ -27,6 +27,8 @@ enum fixed_addresses {
 	FIX_FDT = FIX_FDT_END + FIX_FDT_SIZE / PAGE_SIZE - 1,
 	FIX_PTE,
 	FIX_PMD,
+	FIX_TEXT_POKE1,
+	FIX_TEXT_POKE0,
 	FIX_EARLYCON_MEM_BASE,
 	__end_of_fixed_addresses
 };
arch/riscv/include/asm/patch.h new file +12 -0
new file mode 100644
index 000000000000..9a7d7346001e
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2020 SiFive
+ */
+
+#ifndef _ASM_RISCV_PATCH_H
+#define _ASM_RISCV_PATCH_H
+
+int patch_text_nosync(void *addr, const void *insns, size_t len);
+int patch_text(void *addr, u32 insn);
+
+#endif /* _ASM_RISCV_PATCH_H */
arch/riscv/kernel/Makefile +3 -1
index f40205cb9a22..d189bd3d8501 100644
@@ -4,7 +4,8 @@
 #
 
 ifdef CONFIG_FTRACE
-CFLAGS_REMOVE_ftrace.o = -pg
+CFLAGS_REMOVE_ftrace.o	= -pg
+CFLAGS_REMOVE_patch.o	= -pg
 endif
 
 extra-y += head.o
@@ -26,6 +27,7 @@ obj-y	+= traps.o
 obj-y	+= riscv_ksyms.o
 obj-y	+= stacktrace.o
 obj-y	+= cacheinfo.o
+obj-y	+= patch.o
 obj-$(CONFIG_MMU) += vdso.o vdso/
 
 obj-$(CONFIG_RISCV_M_MODE)	+= clint.o
arch/riscv/kernel/patch.c new file +118 -0
new file mode 100644
index 000000000000..f16466123947
@@ -0,0 +1,128 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2020 SiFive
+ */
+
+#include <linux/mm.h>
+#include <linux/memory.h>
+#include <linux/uaccess.h>
+#include <linux/spinlock.h>
+#include <linux/stop_machine.h>
+#include <asm/fixmap.h>
+#include <asm/kprobes.h>
+#include <asm/cacheflush.h>
+
+struct patch_insn_patch {
+	void *addr;
+	u32 insn;
+	atomic_t cpu_count;
+};
+
+#ifdef CONFIG_MMU
+static void *patch_map(void *addr, int fixmap)
+{
+	uintptr_t uintaddr = (uintptr_t) addr;
+	struct page *page;
+
+	if (core_kernel_text(uintaddr))
+		page = phys_to_page(__pa_symbol(addr));
+	else if (IS_ENABLED(CONFIG_STRICT_MODULE_RWX))
+		page = vmalloc_to_page(addr);
+	else
+		return addr;
+
+	BUG_ON(!page);
+
+	return (void *)set_fixmap_offset(fixmap, page_to_phys(page) +
+					 (uintaddr & ~PAGE_MASK));
+}
+NOKPROBE_SYMBOL(patch_map);
+
+static void patch_unmap(int fixmap)
+{
+	clear_fixmap(fixmap);
+}
+NOKPROBE_SYMBOL(patch_unmap);
+
+static int patch_insn_write(void *addr, const void *insn, size_t len)
+{
+	void *waddr = addr;
+	bool across_pages = (((uintptr_t) addr & ~PAGE_MASK) + len) > PAGE_SIZE;
+	int ret;
+
+	/*
+	 * Before reaching here, it was expected to lock the text_mutex
+	 * already, so we don't need to give another lock here and could
+	 * ensure that it was safe between each cores.
+	 */
+	lockdep_assert_held(&text_mutex);
+
+	if (across_pages)
+		patch_map(addr + len, FIX_TEXT_POKE1);
+
+	waddr = patch_map(addr, FIX_TEXT_POKE0);
+
+	ret = probe_kernel_write(waddr, insn, len);
+
+	patch_unmap(FIX_TEXT_POKE0);
+
+	if (across_pages)
+		patch_unmap(FIX_TEXT_POKE1);
+
+	return ret;
+}
+NOKPROBE_SYMBOL(patch_insn_write);
+#else
+static int patch_insn_write(void *addr, const void *insn, size_t len)
+{
+	return probe_kernel_write(addr, insn, len);
+}
+NOKPROBE_SYMBOL(patch_insn_write);
+#endif /* CONFIG_MMU */
+
+int patch_text_nosync(void *addr, const void *insns, size_t len)
+{
+	u32 *tp = addr;
+	int ret;
+
+	ret = patch_insn_write(tp, insns, len);
+
+	if (!ret)
+		flush_icache_range((uintptr_t) tp, (uintptr_t) tp + len);
+
+	return ret;
+}
+NOKPROBE_SYMBOL(patch_text_nosync);
+
+static int patch_text_cb(void *data)
+{
+	struct patch_insn_patch *patch = data;
+	int ret = 0;
+
+	if (atomic_inc_return(&patch->cpu_count) == 1) {
+		ret =
+		    patch_text_nosync(patch->addr, &patch->insn,
+					    GET_INSN_LENGTH(patch->insn));
+		atomic_inc(&patch->cpu_count);
+	} else {
+		while (atomic_read(&patch->cpu_count) <= num_online_cpus())
+			cpu_relax();
+		smp_mb();
+	}
+
+	return ret;
+}
+NOKPROBE_SYMBOL(patch_text_cb);
+
+int patch_text(void *addr, u32 insn)
+{

arch/riscv/kernel/patch.c line 128

Quoted 10 lines
arch/riscv/kernel/patch.c new file +10 -0
+	struct patch_insn_patch patch = {
+		.addr = addr,
+		.insn = insn,
+		.cpu_count = ATOMIC_INIT(0),
+	};
+
+	return stop_machine_cpuslocked(patch_text_cb,
+				       &patch, cpu_online_mask);
+}
+NOKPROBE_SYMBOL(patch_text);
author
#6 lore raw

PATCH v5 3/9 riscv: patch code by fixmap mapping

In reply to Zong Li

On strict kernel memory permission, the ftrace have to change the permission of text for dynamic patching the intructions. Use riscv_patch_text_nosync() to patch code instead of probe_kernel_write.

---
 arch/riscv/kernel/ftrace.c | 26 +++++++++++++++++---------
 1 file changed, 17 insertions(+), 9 deletions(-)
arch/riscv/kernel/ftrace.c +17 -9
index c40fdcdeb950..08396614d6f4 100644
@@ -7,9 +7,23 @@
 
 #include <linux/ftrace.h>
 #include <linux/uaccess.h>
+#include <linux/memory.h>
 #include <asm/cacheflush.h>
+#include <asm/patch.h>
 
 #ifdef CONFIG_DYNAMIC_FTRACE
+int ftrace_arch_code_modify_prepare(void) __acquires(&text_mutex)
+{
+	mutex_lock(&text_mutex);
+	return 0;
+}
+
+int ftrace_arch_code_modify_post_process(void) __releases(&text_mutex)
+{
+	mutex_unlock(&text_mutex);
+	return 0;
+}
+
 static int ftrace_check_current_call(unsigned long hook_pos,
 				     unsigned int *expected)
 {
@@ -46,20 +60,14 @@ static int __ftrace_modify_call(unsigned long hook_pos, unsigned long target,
 {
 	unsigned int call[2];
 	unsigned int nops[2] = {NOP4, NOP4};
-	int ret = 0;
 
 	make_call(hook_pos, target, call);
 
-	/* replace the auipc-jalr pair at once */
-	ret = probe_kernel_write((void *)hook_pos, enable ? call : nops,
-				 MCOUNT_INSN_SIZE);
-	/* return must be -EPERM on write error */
-	if (ret)
+	/* Replace the auipc-jalr pair at once. Return -EPERM on write error. */
+	if (patch_text_nosync
+	    ((void *)hook_pos, enable ? call : nops, MCOUNT_INSN_SIZE))
 		return -EPERM;
 
-	smp_mb();
-	flush_icache_range((void *)hook_pos, (void *)hook_pos + MCOUNT_INSN_SIZE);
-
 	return 0;
 }
 
Signature · 1 line
2.26.0
maintainer Reviewed-by on v5 3/9
#7 lore raw

Re: [PATCH v5 3/9] riscv: patch code by fixmap mapping

Quoted 52 lines · On Wed, 08 Apr 2020 00:56:58 PDT (-0700), [email protected] wrote:

On strict kernel memory permission, the ftrace have to change the permission of text for dynamic patching the intructions. Use riscv_patch_text_nosync() to patch code instead of probe_kernel_write.

---
 arch/riscv/kernel/ftrace.c | 26 +++++++++++++++++---------
 1 file changed, 17 insertions(+), 9 deletions(-)
arch/riscv/kernel/ftrace.c +14 -6
index c40fdcdeb950..08396614d6f4 100644
@@ -7,9 +7,23 @@
 
 #include <linux/ftrace.h>
 #include <linux/uaccess.h>
+#include <linux/memory.h>
 #include <asm/cacheflush.h>
+#include <asm/patch.h>
 
 #ifdef CONFIG_DYNAMIC_FTRACE
+int ftrace_arch_code_modify_prepare(void) __acquires(&text_mutex)
+{
+	mutex_lock(&text_mutex);
+	return 0;
+}
+
+int ftrace_arch_code_modify_post_process(void) __releases(&text_mutex)
+{
+	mutex_unlock(&text_mutex);
+	return 0;
+}
+
 static int ftrace_check_current_call(unsigned long hook_pos,
 				     unsigned int *expected)
 {
@@ -46,20 +60,14 @@ static int __ftrace_modify_call(unsigned long hook_pos, unsigned long target,
 {
 	unsigned int call[2];
 	unsigned int nops[2] = {NOP4, NOP4};
-	int ret = 0;
 
 	make_call(hook_pos, target, call);
 
-	/* replace the auipc-jalr pair at once */
-	ret = probe_kernel_write((void *)hook_pos, enable ? call : nops,
-				 MCOUNT_INSN_SIZE);
-	/* return must be -EPERM on write error */
-	if (ret)

arch/riscv/kernel/ftrace.c static int __ftrace_modify_call(unsigned long hook_pos, unsigned long target, · line 72

Quoted 10 lines
arch/riscv/kernel/ftrace.c +3 -3
+	/* Replace the auipc-jalr pair at once. Return -EPERM on write error. */
+	if (patch_text_nosync
+	    ((void *)hook_pos, enable ? call : nops, MCOUNT_INSN_SIZE))
 		return -EPERM;
 
-	smp_mb();
-	flush_icache_range((void *)hook_pos, (void *)hook_pos + MCOUNT_INSN_SIZE);
-
 	return 0;
 }
author
#8 lore raw

PATCH v5 4/9 riscv: add ARCH_HAS_SET_MEMORY support

In reply to Zong Li

Add set_memory_ro/rw/x/nx architecture hooks to change the page attribution.

Use own set_memory.h rather than generic set_memory.h (i.e. include/asm-generic/set_memory.h), because we want to add other function prototypes here.

---
 arch/riscv/Kconfig                  |   1 +
 arch/riscv/include/asm/set_memory.h |  24 +++++
 arch/riscv/mm/Makefile              |   2 +-
 arch/riscv/mm/pageattr.c            | 150 ++++++++++++++++++++++++++++
 4 files changed, 176 insertions(+), 1 deletion(-)
 create mode 100644 arch/riscv/include/asm/set_memory.h
 create mode 100644 arch/riscv/mm/pageattr.c
arch/riscv/Kconfig +1 -0
index 39c93ddff0cb..9044e0dd95ea 100644
@@ -59,6 +59,7 @@ config RISCV
 	select HAVE_EBPF_JIT if 64BIT
 	select EDAC_SUPPORT
 	select ARCH_HAS_GIGANTIC_PAGE
+	select ARCH_HAS_SET_MEMORY
 	select ARCH_WANT_HUGE_PMD_SHARE if 64BIT
 	select SPARSEMEM_STATIC if 32BIT
 	select ARCH_WANT_DEFAULT_TOPDOWN_MMAP_LAYOUT if MMU
arch/riscv/include/asm/set_memory.h new file +24 -0
new file mode 100644
index 000000000000..79a810f0f38b
@@ -0,0 +1,24 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2019 SiFive
+ */
+
+#ifndef _ASM_RISCV_SET_MEMORY_H
+#define _ASM_RISCV_SET_MEMORY_H
+
+/*
+ * Functions to change memory attributes.
+ */
+#ifdef CONFIG_MMU
+int set_memory_ro(unsigned long addr, int numpages);
+int set_memory_rw(unsigned long addr, int numpages);
+int set_memory_x(unsigned long addr, int numpages);
+int set_memory_nx(unsigned long addr, int numpages);
+#else
+static inline int set_memory_ro(unsigned long addr, int numpages) { return 0; }
+static inline int set_memory_rw(unsigned long addr, int numpages) { return 0; }
+static inline int set_memory_x(unsigned long addr, int numpages) { return 0; }
+static inline int set_memory_nx(unsigned long addr, int numpages) { return 0; }
+#endif
+
+#endif /* _ASM_RISCV_SET_MEMORY_H */
arch/riscv/mm/Makefile +1 -1
index 814e16a8d68a..363ef01c30b1 100644
@@ -7,7 +7,7 @@ endif
 
 obj-y += init.o
 obj-y += extable.o
-obj-$(CONFIG_MMU) += fault.o
+obj-$(CONFIG_MMU) += fault.o pageattr.o
 obj-y += cacheflush.o
 obj-y += context.o
 
arch/riscv/mm/pageattr.c new file +150 -0

150 lines not on this page · raw message

Signature · 1 line
2.26.0
author
#9 lore raw

PATCH v5 5/9 riscv: add ARCH_HAS_SET_DIRECT_MAP support

In reply to Zong Li

Add set_direct_map_*() functions for setting the direct map alias for the page to its default permissions and to an invalid state that cannot be cached in a TLB. (See d253ca0c ("x86/mm/cpa: Add set_direct_map_*() functions")) Add a similar implementation for RISC-V.

---
 arch/riscv/Kconfig                  |  1 +
 arch/riscv/include/asm/set_memory.h |  3 +++
 arch/riscv/mm/pageattr.c            | 24 ++++++++++++++++++++++++
 3 files changed, 28 insertions(+)
arch/riscv/Kconfig +1 -0
index 9044e0dd95ea..a94d0f064d9c 100644
@@ -59,6 +59,7 @@ config RISCV
 	select HAVE_EBPF_JIT if 64BIT
 	select EDAC_SUPPORT
 	select ARCH_HAS_GIGANTIC_PAGE
+	select ARCH_HAS_SET_DIRECT_MAP
 	select ARCH_HAS_SET_MEMORY
 	select ARCH_WANT_HUGE_PMD_SHARE if 64BIT
 	select SPARSEMEM_STATIC if 32BIT
arch/riscv/include/asm/set_memory.h +3 -0
index 79a810f0f38b..620d81c372d9 100644
@@ -21,4 +21,7 @@ static inline int set_memory_x(unsigned long addr, int numpages) { return 0; }
 static inline int set_memory_nx(unsigned long addr, int numpages) { return 0; }
 #endif
 
+int set_direct_map_invalid_noflush(struct page *page);
+int set_direct_map_default_noflush(struct page *page);
+
 #endif /* _ASM_RISCV_SET_MEMORY_H */
arch/riscv/mm/pageattr.c +24 -0
index fcd59ef2835b..7be6cd67e2ef 100644
@@ -148,3 +148,27 @@ int set_memory_nx(unsigned long addr, int numpages)
 {
 	return __set_memory(addr, numpages, __pgprot(0), __pgprot(_PAGE_EXEC));
 }
+
+int set_direct_map_invalid_noflush(struct page *page)
+{
+	unsigned long start = (unsigned long)page_address(page);
+	unsigned long end = start + PAGE_SIZE;
+	struct pageattr_masks masks = {
+		.set_mask = __pgprot(0),
+		.clear_mask = __pgprot(_PAGE_PRESENT)
+	};
+
+	return walk_page_range(&init_mm, start, end, &pageattr_ops, &masks);
+}
+
+int set_direct_map_default_noflush(struct page *page)
+{
+	unsigned long start = (unsigned long)page_address(page);
+	unsigned long end = start + PAGE_SIZE;
+	struct pageattr_masks masks = {
+		.set_mask = PAGE_KERNEL,
+		.clear_mask = __pgprot(0)
+	};
+
+	return walk_page_range(&init_mm, start, end, &pageattr_ops, &masks);
+}
Signature · 1 line
2.26.0
author
#10 lore raw

PATCH v5 6/9 riscv: add ARCH_SUPPORTS_DEBUG_PAGEALLOC support

In reply to Zong Li

ARCH_SUPPORTS_DEBUG_PAGEALLOC provides a hook to map and unmap pages for debugging purposes. Implement the __kernel_map_pages functions to fill the poison pattern.

---
 arch/riscv/Kconfig       |  3 +++
 arch/riscv/mm/pageattr.c | 13 +++++++++++++
 2 files changed, 16 insertions(+)
arch/riscv/Kconfig +3 -0
index a94d0f064d9c..1e1efc998baf 100644
@@ -132,6 +132,9 @@ config ARCH_SELECT_MEMORY_MODEL
 config ARCH_WANT_GENERAL_HUGETLB
 	def_bool y
 
+config ARCH_SUPPORTS_DEBUG_PAGEALLOC
+	def_bool y
+
 config SYS_SUPPORTS_HUGETLBFS
 	def_bool y
 
arch/riscv/mm/pageattr.c +13 -0
index 7be6cd67e2ef..728759eb530a 100644
@@ -172,3 +172,16 @@ int set_direct_map_default_noflush(struct page *page)
 
 	return walk_page_range(&init_mm, start, end, &pageattr_ops, &masks);
 }
+
+void __kernel_map_pages(struct page *page, int numpages, int enable)
+{
+	if (!debug_pagealloc_enabled())
+		return;
+
+	if (enable)
+		__set_memory((unsigned long)page_address(page), numpages,
+			     __pgprot(_PAGE_PRESENT), __pgprot(0));
+	else
+		__set_memory((unsigned long)page_address(page), numpages,
+			     __pgprot(0), __pgprot(_PAGE_PRESENT));
+}
Signature · 1 line
2.26.0
author
#11 lore raw

PATCH v5 7/9 riscv: move exception table immediately after RO_DATA

In reply to Zong Li

Move EXCEPTION_TABLE immediately after RO_DATA. Make it easy to set the attribution of the sections which should be read-only at a time. Add _data to specify the start of data section with write permission. This patch is prepared for STRICT_KERNEL_RWX support.

---
 arch/riscv/kernel/vmlinux.lds.S | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
arch/riscv/kernel/vmlinux.lds.S +4 -2
index 1e0193ded420..02e948b620af 100644
@@ -58,6 +58,10 @@ SECTIONS
 		*(.srodata*)
 	}
 
+	EXCEPTION_TABLE(0x10)
+
+	_data = .;
+
 	RW_DATA(L1_CACHE_BYTES, PAGE_SIZE, THREAD_SIZE)
 	.sdata : {
 		__global_pointer$ = . + 0x800;
@@ -69,8 +73,6 @@ SECTIONS
 
 	BSS_SECTION(PAGE_SIZE, PAGE_SIZE, 0)
 
-	EXCEPTION_TABLE(0x10)
-
 	.rel.dyn : {
 		*(.rel.dyn*)
 	}
Signature · 1 line
2.26.0
author
#12 lore raw

PATCH v5 8/9 riscv: add alignment for text, rodata and data sections

In reply to Zong Li

The kernel mapping will tried to optimize its mapping by using bigger size. In rv64, it tries to use PMD_SIZE, and tryies to use PGDIR_SIZE in rv32. To ensure that the start address of these sections could fit the mapping entry size, make them align to the biggest alignment.

Define a macro SECTION_ALIGN because the HPAGE_SIZE or PMD_SIZE, etc., are invisible in linker script.

This patch is prepared for STRICT_KERNEL_RWX support.

---
 arch/riscv/include/asm/set_memory.h | 13 +++++++++++++
 arch/riscv/kernel/vmlinux.lds.S     |  5 ++++-
 2 files changed, 17 insertions(+), 1 deletion(-)
arch/riscv/include/asm/set_memory.h +13 -0
index 620d81c372d9..4c5bae7ca01c 100644
@@ -6,6 +6,7 @@
 #ifndef _ASM_RISCV_SET_MEMORY_H
 #define _ASM_RISCV_SET_MEMORY_H
 
+#ifndef __ASSEMBLY__
 /*
  * Functions to change memory attributes.
  */
@@ -24,4 +25,16 @@ static inline int set_memory_nx(unsigned long addr, int numpages) { return 0; }
 int set_direct_map_invalid_noflush(struct page *page);
 int set_direct_map_default_noflush(struct page *page);
 
+#endif /* __ASSEMBLY__ */
+
+#ifdef CONFIG_ARCH_HAS_STRICT_KERNEL_RWX
+#ifdef CONFIG_64BIT
+#define SECTION_ALIGN (1 << 21)
+#else
+#define SECTION_ALIGN (1 << 22)
+#endif
+#else /* !CONFIG_ARCH_HAS_STRICT_KERNEL_RWX */
+#define SECTION_ALIGN L1_CACHE_BYTES
+#endif /* CONFIG_ARCH_HAS_STRICT_KERNEL_RWX */
+
 #endif /* _ASM_RISCV_SET_MEMORY_H */
arch/riscv/kernel/vmlinux.lds.S +4 -1
index 02e948b620af..ef87deea0350 100644
@@ -9,6 +9,7 @@
 #include <asm/page.h>
 #include <asm/cache.h>
 #include <asm/thread_info.h>
+#include <asm/set_memory.h>
 
 OUTPUT_ARCH(riscv)
 ENTRY(_start)
@@ -36,6 +37,7 @@ SECTIONS
 	PERCPU_SECTION(L1_CACHE_BYTES)
 	__init_end = .;
 
+	. = ALIGN(SECTION_ALIGN);
 	.text : {
 		_text = .;
 		_stext = .;
@@ -53,13 +55,14 @@ SECTIONS
 
 	/* Start of data section */
 	_sdata = .;
-	RO_DATA(L1_CACHE_BYTES)
+	RO_DATA(SECTION_ALIGN)
 	.srodata : {
 		*(.srodata*)
 	}
 
 	EXCEPTION_TABLE(0x10)
 
+	. = ALIGN(SECTION_ALIGN);
 	_data = .;
 
 	RW_DATA(L1_CACHE_BYTES, PAGE_SIZE, THREAD_SIZE)
Signature · 1 line
2.26.0
author
#13 lore raw

PATCH v5 9/9 riscv: add STRICT_KERNEL_RWX support

In reply to Zong Li

The commit contains that make text section as non-writable, rodata section as read-only, and data section as non-executable.

The init section should be changed to non-executable.

---
 arch/riscv/Kconfig                  |  1 +
 arch/riscv/include/asm/set_memory.h |  8 ++++++
 arch/riscv/mm/init.c                | 44 +++++++++++++++++++++++++++++
 3 files changed, 53 insertions(+)
arch/riscv/Kconfig +1 -0
index 1e1efc998baf..58b556167d59 100644
@@ -61,6 +61,7 @@ config RISCV
 	select ARCH_HAS_GIGANTIC_PAGE
 	select ARCH_HAS_SET_DIRECT_MAP
 	select ARCH_HAS_SET_MEMORY
+	select ARCH_HAS_STRICT_KERNEL_RWX
 	select ARCH_WANT_HUGE_PMD_SHARE if 64BIT
 	select SPARSEMEM_STATIC if 32BIT
 	select ARCH_WANT_DEFAULT_TOPDOWN_MMAP_LAYOUT if MMU
arch/riscv/include/asm/set_memory.h +8 -0
index 4c5bae7ca01c..c38df4771c09 100644
@@ -22,6 +22,14 @@ static inline int set_memory_x(unsigned long addr, int numpages) { return 0; }
 static inline int set_memory_nx(unsigned long addr, int numpages) { return 0; }
 #endif
 
+#ifdef CONFIG_STRICT_KERNEL_RWX
+void set_kernel_text_ro(void);
+void set_kernel_text_rw(void);
+#else
+static inline void set_kernel_text_ro(void) { }
+static inline void set_kernel_text_rw(void) { }
+#endif
+
 int set_direct_map_invalid_noflush(struct page *page);
 int set_direct_map_default_noflush(struct page *page);
 
arch/riscv/mm/init.c +44 -0

62 lines not on this page · raw message

Signature · 1 line
2.26.0
maintainer Reviewed-by on v5 9/9
#14 lore raw

Re: [PATCH v5 9/9] riscv: add STRICT_KERNEL_RWX support

Quoted 102 lines · On Wed, 08 Apr 2020 00:57:04 PDT (-0700), [email protected] wrote:

The commit contains that make text section as non-writable, rodata section as read-only, and data section as non-executable.

The init section should be changed to non-executable.

---
 arch/riscv/Kconfig                  |  1 +
 arch/riscv/include/asm/set_memory.h |  8 ++++++
 arch/riscv/mm/init.c                | 44 +++++++++++++++++++++++++++++
 3 files changed, 53 insertions(+)
arch/riscv/Kconfig +1 -0
index 1e1efc998baf..58b556167d59 100644
@@ -61,6 +61,7 @@ config RISCV
 	select ARCH_HAS_GIGANTIC_PAGE
 	select ARCH_HAS_SET_DIRECT_MAP
 	select ARCH_HAS_SET_MEMORY
+	select ARCH_HAS_STRICT_KERNEL_RWX
 	select ARCH_WANT_HUGE_PMD_SHARE if 64BIT
 	select SPARSEMEM_STATIC if 32BIT
 	select ARCH_WANT_DEFAULT_TOPDOWN_MMAP_LAYOUT if MMU
arch/riscv/include/asm/set_memory.h +8 -0
index 4c5bae7ca01c..c38df4771c09 100644
@@ -22,6 +22,14 @@ static inline int set_memory_x(unsigned long addr, int numpages) { return 0; }
 static inline int set_memory_nx(unsigned long addr, int numpages) { return 0; }
 #endif
 
+#ifdef CONFIG_STRICT_KERNEL_RWX
+void set_kernel_text_ro(void);
+void set_kernel_text_rw(void);
+#else
+static inline void set_kernel_text_ro(void) { }
+static inline void set_kernel_text_rw(void) { }
+#endif
+
 int set_direct_map_invalid_noflush(struct page *page);
 int set_direct_map_default_noflush(struct page *page);
 
arch/riscv/mm/init.c +37 -0
index fab855963c73..b55be44ff9bd 100644
@@ -12,6 +12,7 @@
 #include <linux/sizes.h>
 #include <linux/of_fdt.h>
 #include <linux/libfdt.h>
+#include <linux/set_memory.h>
 
 #include <asm/fixmap.h>
 #include <asm/tlbflush.h>
@@ -477,6 +478,17 @@ static void __init setup_vm_final(void)
 	csr_write(CSR_SATP, PFN_DOWN(__pa_symbol(swapper_pg_dir)) | SATP_MODE);
 	local_flush_tlb_all();
 }
+
+void free_initmem(void)
+{
+	unsigned long init_begin = (unsigned long)__init_begin;
+	unsigned long init_end = (unsigned long)__init_end;
+
+	/* Make the region as non-execuatble. */
+	set_memory_nx(init_begin, (init_end - init_begin) >> PAGE_SHIFT);
+	free_initmem_default(POISON_FREE_INITMEM);
+}
+
 #else
 asmlinkage void __init setup_vm(uintptr_t dtb_pa)
 {
@@ -488,6 +500,38 @@ static inline void setup_vm_final(void)
 }
 #endif /* CONFIG_MMU */
 
+#ifdef CONFIG_STRICT_KERNEL_RWX
+void set_kernel_text_rw(void)
+{
+	unsigned long text_start = (unsigned long)_text;
+	unsigned long text_end = (unsigned long)_etext;
+
+	set_memory_rw(text_start, (text_end - text_start) >> PAGE_SHIFT);
+}
+
+void set_kernel_text_ro(void)
+{
+	unsigned long text_start = (unsigned long)_text;
+	unsigned long text_end = (unsigned long)_etext;
+
+	set_memory_ro(text_start, (text_end - text_start) >> PAGE_SHIFT);
+}
+
+void mark_rodata_ro(void)
+{
+	unsigned long text_start = (unsigned long)_text;
+	unsigned long text_end = (unsigned long)_etext;
+	unsigned long rodata_start = (unsigned long)__start_rodata;
+	unsigned long data_start = (unsigned long)_data;
+	unsigned long max_low = (unsigned long)(__va(PFN_PHYS(max_low_pfn)));
+

arch/riscv/mm/init.c static inline void setup_vm_final(void) · line 537

Quoted 10 lines
arch/riscv/mm/init.c +7 -0
+	set_memory_ro(text_start, (text_end - text_start) >> PAGE_SHIFT);
+	set_memory_ro(rodata_start, (data_start - rodata_start) >> PAGE_SHIFT);
+	set_memory_nx(rodata_start, (data_start - rodata_start) >> PAGE_SHIFT);
+	set_memory_nx(data_start, (max_low - data_start) >> PAGE_SHIFT);
+}
+#endif
+
 void __init paging_init(void)
 {
 	setup_vm_final();
#15 lore raw

Re: [PATCH v5 9/9] riscv: add STRICT_KERNEL_RWX support

Quoted 13 lines · Palmer Dabbelt
Quoted 12 lines · On Wed, 08 Apr 2020 00:57:04 PDT (-0700), [email protected] wrote:

The commit contains that make text section as non-writable, rodata section as read-only, and data section as non-executable.

The init section should be changed to non-executable.

---
 arch/riscv/Kconfig                  |  1 +
 arch/riscv/include/asm/set_memory.h |  8 ++++++
 arch/riscv/mm/init.c                | 44 +++++++++++++++++++++++++++++
 3 files changed, 53 insertions(+)
Quoted 9 lines

arch/riscv/Kconfig config RISCV · line 64

Quoted 9 lines
arch/riscv/Kconfig +1 -0
index 1e1efc998baf..58b556167d59 100644
@@ -61,6 +61,7 @@ config RISCV
 	select ARCH_HAS_GIGANTIC_PAGE
 	select ARCH_HAS_SET_DIRECT_MAP
 	select ARCH_HAS_SET_MEMORY
+	select ARCH_HAS_STRICT_KERNEL_RWX

This should be:

select ARCH_HAS_STRICT_KERNEL_RWX if !MMU

This option does not make sense for the no MMU case and more importantly, it ends up generating gigantic binary images which breaks things like the K210 support.

Palmer,

I sent a patch to fix that. You did not reply/comment on it.

Quoted 95 lines
Quoted 81 lines
arch/riscv/Kconfig +0 -0
 	select ARCH_WANT_HUGE_PMD_SHARE if 64BIT
 	select SPARSEMEM_STATIC if 32BIT
 	select ARCH_WANT_DEFAULT_TOPDOWN_MMAP_LAYOUT if MMU
arch/riscv/include/asm/set_memory.h +8 -0
index 4c5bae7ca01c..c38df4771c09 100644
@@ -22,6 +22,14 @@ static inline int set_memory_x(unsigned long addr, int numpages) { return 0; }
 static inline int set_memory_nx(unsigned long addr, int numpages) { return 0; }
 #endif
 
+#ifdef CONFIG_STRICT_KERNEL_RWX
+void set_kernel_text_ro(void);
+void set_kernel_text_rw(void);
+#else
+static inline void set_kernel_text_ro(void) { }
+static inline void set_kernel_text_rw(void) { }
+#endif
+
 int set_direct_map_invalid_noflush(struct page *page);
 int set_direct_map_default_noflush(struct page *page);
 
arch/riscv/mm/init.c +37 -0
index fab855963c73..b55be44ff9bd 100644
@@ -12,6 +12,7 @@
 #include <linux/sizes.h>
 #include <linux/of_fdt.h>
 #include <linux/libfdt.h>
+#include <linux/set_memory.h>
 
 #include <asm/fixmap.h>
 #include <asm/tlbflush.h>
@@ -477,6 +478,17 @@ static void __init setup_vm_final(void)
 	csr_write(CSR_SATP, PFN_DOWN(__pa_symbol(swapper_pg_dir)) | SATP_MODE);
 	local_flush_tlb_all();
 }
+
+void free_initmem(void)
+{
+	unsigned long init_begin = (unsigned long)__init_begin;
+	unsigned long init_end = (unsigned long)__init_end;
+
+	/* Make the region as non-execuatble. */
+	set_memory_nx(init_begin, (init_end - init_begin) >> PAGE_SHIFT);
+	free_initmem_default(POISON_FREE_INITMEM);
+}
+
 #else
 asmlinkage void __init setup_vm(uintptr_t dtb_pa)
 {
@@ -488,6 +500,38 @@ static inline void setup_vm_final(void)
 }
 #endif /* CONFIG_MMU */
 
+#ifdef CONFIG_STRICT_KERNEL_RWX
+void set_kernel_text_rw(void)
+{
+	unsigned long text_start = (unsigned long)_text;
+	unsigned long text_end = (unsigned long)_etext;
+
+	set_memory_rw(text_start, (text_end - text_start) >> PAGE_SHIFT);
+}
+
+void set_kernel_text_ro(void)
+{
+	unsigned long text_start = (unsigned long)_text;
+	unsigned long text_end = (unsigned long)_etext;
+
+	set_memory_ro(text_start, (text_end - text_start) >> PAGE_SHIFT);
+}
+
+void mark_rodata_ro(void)
+{
+	unsigned long text_start = (unsigned long)_text;
+	unsigned long text_end = (unsigned long)_etext;
+	unsigned long rodata_start = (unsigned long)__start_rodata;
+	unsigned long data_start = (unsigned long)_data;
+	unsigned long max_low = (unsigned long)(__va(PFN_PHYS(max_low_pfn)));
+

arch/riscv/mm/init.c static inline void setup_vm_final(void) · line 537

Quoted 10 lines
arch/riscv/mm/init.c +7 -0
+	set_memory_ro(text_start, (text_end - text_start) >> PAGE_SHIFT);
+	set_memory_ro(rodata_start, (data_start - rodata_start) >> PAGE_SHIFT);
+	set_memory_nx(rodata_start, (data_start - rodata_start) >> PAGE_SHIFT);
+	set_memory_nx(data_start, (max_low - data_start) >> PAGE_SHIFT);
+}
+#endif
+
 void __init paging_init(void)
 {
 	setup_vm_final();
Signature · 2 lines
Damien Le Moal
Western Digital Research
maintainer
#16 lore raw
In reply to Zong Li
Quoted 47 lines · On Wed, 08 Apr 2020 00:56:55 PDT (-0700), [email protected] wrote:

The main purpose of this patch series is changing the kernel mapping permission , make sure that code is not writeable, data is not executable, and read-only data is neither writable nor executable.

This patch series also supports the relevant implementations such as ARCH_HAS_SET_MEMORY, ARCH_HAS_SET_DIRECT_MAP, ARCH_SUPPORTS_DEBUG_PAGEALLOC and DEBUG_WX.

Changes in v5:
- Add lockdep_assert_held and more comments for text_mutex.

Changes in v4:
- Use NOKPROBE_SYMBOL instead of __kprobe annotation
- Use text_mutex instead of patch_lock
- Remove 'riscv_' prefix of function name

Changes in v3:
- Fix build error on nommu configuration. We already support nommu on RISC-V, so we should consider nommu case and test not only rv32/64, but also nommu.

Changes in v2:
- Use _data to specify the start of data section with write permission.
- Change ftrace patch text implementaion.
- Separate DEBUG_WX patch to another patchset.

Zong Li (9):
riscv: add macro to get instruction length
riscv: introduce interfaces to patch kernel code
riscv: patch code by fixmap mapping
riscv: add ARCH_HAS_SET_MEMORY support
riscv: add ARCH_HAS_SET_DIRECT_MAP support
riscv: add ARCH_SUPPORTS_DEBUG_PAGEALLOC support
riscv: move exception table immediately after RO_DATA
riscv: add alignment for text, rodata and data sections
riscv: add STRICT_KERNEL_RWX support

 arch/riscv/Kconfig                  |   6 +
 arch/riscv/include/asm/bug.h        |   8 ++
 arch/riscv/include/asm/fixmap.h     |   2 +
 arch/riscv/include/asm/patch.h      |  12 ++
 arch/riscv/include/asm/set_memory.h |  48 +++++++
 arch/riscv/kernel/Makefile          |   4 +-
 arch/riscv/kernel/ftrace.c          |  26 ++--
 arch/riscv/kernel/patch.c           | 128 +++++++++++++++++++
 arch/riscv/kernel/traps.c           |   3 +-
 arch/riscv/kernel/vmlinux.lds.S     |  11 +-
Quoted 8 lines
 arch/riscv/mm/Makefile              |   2 +-
 arch/riscv/mm/init.c                |  44 +++++++
 arch/riscv/mm/pageattr.c            | 187 ++++++++++++++++++++++++++++
 13 files changed, 466 insertions(+), 15 deletions(-)
 create mode 100644 arch/riscv/include/asm/patch.h
 create mode 100644 arch/riscv/include/asm/set_memory.h
 create mode 100644 arch/riscv/kernel/patch.c
 create mode 100644 arch/riscv/mm/pageattr.c

Looks like there are quite a few conflicts here. Do you mind re-spinning the patch set for me? It's a bit early to spin off for-next right now so there might be some more conflicts, but hopefully not too many.

Thanks!

#17 lore raw

Palmer Dabbelt <[email protected]> 於 2020年4月21日 週二 上午2:27寫道:

Quoted 58 lines · On Wed, 08 Apr 2020 00:56:55 PDT (-0700), [email protected] wrote:
Quoted 47 lines

The main purpose of this patch series is changing the kernel mapping permission , make sure that code is not writeable, data is not executable, and read-only data is neither writable nor executable.

This patch series also supports the relevant implementations such as ARCH_HAS_SET_MEMORY, ARCH_HAS_SET_DIRECT_MAP, ARCH_SUPPORTS_DEBUG_PAGEALLOC and DEBUG_WX.

Changes in v5:
- Add lockdep_assert_held and more comments for text_mutex.

Changes in v4:
- Use NOKPROBE_SYMBOL instead of __kprobe annotation
- Use text_mutex instead of patch_lock
- Remove 'riscv_' prefix of function name

Changes in v3:
- Fix build error on nommu configuration. We already support nommu on RISC-V, so we should consider nommu case and test not only rv32/64, but also nommu.

Changes in v2:
- Use _data to specify the start of data section with write permission.
- Change ftrace patch text implementaion.
- Separate DEBUG_WX patch to another patchset.

Zong Li (9):
riscv: add macro to get instruction length
riscv: introduce interfaces to patch kernel code
riscv: patch code by fixmap mapping
riscv: add ARCH_HAS_SET_MEMORY support
riscv: add ARCH_HAS_SET_DIRECT_MAP support
riscv: add ARCH_SUPPORTS_DEBUG_PAGEALLOC support
riscv: move exception table immediately after RO_DATA
riscv: add alignment for text, rodata and data sections
riscv: add STRICT_KERNEL_RWX support

 arch/riscv/Kconfig                  |   6 +
 arch/riscv/include/asm/bug.h        |   8 ++
 arch/riscv/include/asm/fixmap.h     |   2 +
 arch/riscv/include/asm/patch.h      |  12 ++
 arch/riscv/include/asm/set_memory.h |  48 +++++++
 arch/riscv/kernel/Makefile          |   4 +-
 arch/riscv/kernel/ftrace.c          |  26 ++--
 arch/riscv/kernel/patch.c           | 128 +++++++++++++++++++
 arch/riscv/kernel/traps.c           |   3 +-
 arch/riscv/kernel/vmlinux.lds.S     |  11 +-
Quoted 8 lines
 arch/riscv/mm/Makefile              |   2 +-
 arch/riscv/mm/init.c                |  44 +++++++
 arch/riscv/mm/pageattr.c            | 187 ++++++++++++++++++++++++++++
 13 files changed, 466 insertions(+), 15 deletions(-)
 create mode 100644 arch/riscv/include/asm/patch.h
 create mode 100644 arch/riscv/include/asm/set_memory.h
 create mode 100644 arch/riscv/kernel/patch.c
 create mode 100644 arch/riscv/mm/pageattr.c
Quoted 4 lines

Looks like there are quite a few conflicts here. Do you mind re-spinning the patch set for me? It's a bit early to spin off for-next right now so there might be some more conflicts, but hopefully not too many.

Sure, let me rebase it and send the next version. Thanks.

Quoted 2 lines

Thanks!

#18 lore raw
Quoted 60 lines · Zong Li

Palmer Dabbelt <[email protected]> 於 2020年4月21日 週二 上午2:27寫道:

Quoted 58 lines · On Wed, 08 Apr 2020 00:56:55 PDT (-0700), [email protected] wrote:
Quoted 55 lines

The main purpose of this patch series is changing the kernel mapping permission , make sure that code is not writeable, data is not executable, and read-only data is neither writable nor executable.

This patch series also supports the relevant implementations such as ARCH_HAS_SET_MEMORY, ARCH_HAS_SET_DIRECT_MAP, ARCH_SUPPORTS_DEBUG_PAGEALLOC and DEBUG_WX.

Changes in v5:
- Add lockdep_assert_held and more comments for text_mutex.

Changes in v4:
- Use NOKPROBE_SYMBOL instead of __kprobe annotation
- Use text_mutex instead of patch_lock
- Remove 'riscv_' prefix of function name

Changes in v3:
- Fix build error on nommu configuration. We already support nommu on RISC-V, so we should consider nommu case and test not only rv32/64, but also nommu.

Changes in v2:
- Use _data to specify the start of data section with write permission.
- Change ftrace patch text implementaion.
- Separate DEBUG_WX patch to another patchset.

Zong Li (9):
riscv: add macro to get instruction length
riscv: introduce interfaces to patch kernel code
riscv: patch code by fixmap mapping
riscv: add ARCH_HAS_SET_MEMORY support
riscv: add ARCH_HAS_SET_DIRECT_MAP support
riscv: add ARCH_SUPPORTS_DEBUG_PAGEALLOC support
riscv: move exception table immediately after RO_DATA
riscv: add alignment for text, rodata and data sections
riscv: add STRICT_KERNEL_RWX support

 arch/riscv/Kconfig                  |   6 +
 arch/riscv/include/asm/bug.h        |   8 ++
 arch/riscv/include/asm/fixmap.h     |   2 +
 arch/riscv/include/asm/patch.h      |  12 ++
 arch/riscv/include/asm/set_memory.h |  48 +++++++
 arch/riscv/kernel/Makefile          |   4 +-
 arch/riscv/kernel/ftrace.c          |  26 ++--
 arch/riscv/kernel/patch.c           | 128 +++++++++++++++++++
 arch/riscv/kernel/traps.c           |   3 +-
 arch/riscv/kernel/vmlinux.lds.S     |  11 +-
 arch/riscv/mm/Makefile              |   2 +-
 arch/riscv/mm/init.c                |  44 +++++++
 arch/riscv/mm/pageattr.c            | 187 ++++++++++++++++++++++++++++
 13 files changed, 466 insertions(+), 15 deletions(-)
 create mode 100644 arch/riscv/include/asm/patch.h
 create mode 100644 arch/riscv/include/asm/set_memory.h
 create mode 100644 arch/riscv/kernel/patch.c
 create mode 100644 arch/riscv/mm/pageattr.c
Quoted 6 lines
Quoted 4 lines

Looks like there are quite a few conflicts here. Do you mind re-spinning the patch set for me? It's a bit early to spin off for-next right now so there might be some more conflicts, but hopefully not too many.

Sure, let me rebase it and send the next version. Thanks.

This series is already merged in Linux kernel and it is available in Linux-5.7-rc2.

I don't see the point in rebasing this series or commenting on this series.

If you have fixes then send as separate patches based on latest Linux-5.7-rcX release.

Regards,
Anup

author
#19 lore raw
Quoted 69 lines · Anup Patel
Quoted 60 lines · Zong Li

Palmer Dabbelt <[email protected]> 於 2020年4月21日 週二 上午2:27寫道:

Quoted 58 lines · On Wed, 08 Apr 2020 00:56:55 PDT (-0700), [email protected] wrote:
Quoted 55 lines

The main purpose of this patch series is changing the kernel mapping permission , make sure that code is not writeable, data is not executable, and read-only data is neither writable nor executable.

This patch series also supports the relevant implementations such as ARCH_HAS_SET_MEMORY, ARCH_HAS_SET_DIRECT_MAP, ARCH_SUPPORTS_DEBUG_PAGEALLOC and DEBUG_WX.

Changes in v5:
- Add lockdep_assert_held and more comments for text_mutex.

Changes in v4:
- Use NOKPROBE_SYMBOL instead of __kprobe annotation
- Use text_mutex instead of patch_lock
- Remove 'riscv_' prefix of function name

Changes in v3:
- Fix build error on nommu configuration. We already support nommu on RISC-V, so we should consider nommu case and test not only rv32/64, but also nommu.

Changes in v2:
- Use _data to specify the start of data section with write permission.
- Change ftrace patch text implementaion.
- Separate DEBUG_WX patch to another patchset.

Zong Li (9):
riscv: add macro to get instruction length
riscv: introduce interfaces to patch kernel code
riscv: patch code by fixmap mapping
riscv: add ARCH_HAS_SET_MEMORY support
riscv: add ARCH_HAS_SET_DIRECT_MAP support
riscv: add ARCH_SUPPORTS_DEBUG_PAGEALLOC support
riscv: move exception table immediately after RO_DATA
riscv: add alignment for text, rodata and data sections
riscv: add STRICT_KERNEL_RWX support

 arch/riscv/Kconfig                  |   6 +
 arch/riscv/include/asm/bug.h        |   8 ++
 arch/riscv/include/asm/fixmap.h     |   2 +
 arch/riscv/include/asm/patch.h      |  12 ++
 arch/riscv/include/asm/set_memory.h |  48 +++++++
 arch/riscv/kernel/Makefile          |   4 +-
 arch/riscv/kernel/ftrace.c          |  26 ++--
 arch/riscv/kernel/patch.c           | 128 +++++++++++++++++++
 arch/riscv/kernel/traps.c           |   3 +-
 arch/riscv/kernel/vmlinux.lds.S     |  11 +-
 arch/riscv/mm/Makefile              |   2 +-
 arch/riscv/mm/init.c                |  44 +++++++
 arch/riscv/mm/pageattr.c            | 187 ++++++++++++++++++++++++++++
 13 files changed, 466 insertions(+), 15 deletions(-)
 create mode 100644 arch/riscv/include/asm/patch.h
 create mode 100644 arch/riscv/include/asm/set_memory.h
 create mode 100644 arch/riscv/kernel/patch.c
 create mode 100644 arch/riscv/mm/pageattr.c
Quoted 6 lines
Quoted 4 lines

Looks like there are quite a few conflicts here. Do you mind re-spinning the patch set for me? It's a bit early to spin off for-next right now so there might be some more conflicts, but hopefully not too many.

Sure, let me rebase it and send the next version. Thanks.

Quoted 9 lines

This series is already merged in Linux kernel and it is available in Linux-5.7-rc2.

I don't see the point in rebasing this series or commenting on this series.

If you have fixes then send as separate patches based on latest Linux-5.7-rcX release.

Yes, I would send the separate patches for the difference from the merged version. Thanks.

Quoted 2 lines

Regards,
Anup