riscv: patch: fix handling of bpf-jit execmem addresses
PATCH 1/1 riscv: patch: fix handling of bpf-jit execmem addresses
Commit 8718e5a3090b ("riscv: patch: skip fixmap mapping when kernel text is already writable") gated the core_kernel_text() branch of patch_map() on CONFIG_STRICT_KERNEL_RWX, and explicitly left the vmalloc branch unchanged. That branch has a separate problem.
That branch only creates a temporary writable fixmap alias when CONFIG_STRICT_MODULE_RWX is enabled, and otherwise assumes the target is directly writable and returns the address unchanged. That assumption no longer holds: bpf_prog_pack_alloc calls set_memory_rox() on every pack unconditionally in alloc_new_pack(), independently of any CONFIG_STRICT_*_RWX option, so BPF text in the vmalloc area is read-only regardless of what CONFIG_STRICT_MODULE_RWX says.
With CONFIG_STRICT_MODULE_RWX=n, patch_map() returns the read-only address directly, the subsequent copy_to_kernel_nofault() takes a page fault and returns -EFAULT. bpf_arch_text_copy() turns that into -EINVAL, which propagates through bpf_jit_binary_pack_finalize() to the WARN_ON() in bpf_int_jit_compile() and leaves the program un-JITed.
Note that BPF cannot be fixed the way kprobes was in
commit bdc46e507b59 ("riscv: mm: make EXECMEM_KPROBES writable without CONFIG_STRICT_MODULE_RWX"). EXECMEM_BPF is already PAGE_KERNEL, i.e. writable at allocation time; the read-only mapping is established afterwards by generic code in alloc_new_pack(), which arch code cannot influence. The only place this can be handled is patch_map().
This is the same problem that was fixed on arm64 by commit b1480ed230ac ("arm64: patching: fix handling of execmem addresses"), and the fix is the same: CONFIG_EXECMEM is what actually tracks whether the vmalloc area can hold executable memory that needs a temporary alias to be written. CONFIG_BPF_JIT, CONFIG_KPROBES and CONFIG_MODULES all select it.
Note that this is not limited to CONFIG_MODULES=n. Unlike arm64, riscv selects CONFIG_ARCH_OPTIONAL_KERNEL_RWX, so CONFIG_STRICT_MODULE_RWX can be disabled with CONFIG_MODULES=y as well, and the bug is reachable in that configuration too.
The !CONFIG_MMU build is unaffected: it has its own __patch_insn_set() and __patch_insn_write() implementations and never calls patch_map().
---
Based on riscv/fixes (b94cec5761d2).
Reproduced on qemu-system-riscv64 -M virt with CONFIG_BPF_JIT=y, CONFIG_BPF_JIT_ALWAYS_ON=y and CONFIG_STRICT_MODULE_RWX=n. ptp_classifier_init() builds a cBPF filter from sock_init(), so the failure happens during boot without any userspace involved:
WARNING: arch/riscv/net/bpf_jit_core.c:156 at bpf_int_jit_compile+0x3dc/0x43e Call Trace: bpf_int_jit_compile+0x3dc/0x43e __bpf_prog_select_runtime+0xec/0x186 bpf_prog_select_runtime+0x12/0x1a bpf_prepare_filter+0x368/0x46a bpf_prog_create+0x66/0x90 ptp_classifier_init+0x3e/0x60 sock_init+0xc4/0xe6 do_one_initcall+0x78/0x14a
kernel BUG at net/core/ptp_classifier.c:227!
Kernel panic - not syncing: Fatal exception in interrupt
The BUG_ON() is reached because CONFIG_BPF_JIT_ALWAYS_ON=y turns the silent interpreter fallback into -ENOTSUPP. With CONFIG_BPF_JIT_ALWAYS_ON=n the failure is silent: writing 1 to /proc/sys/net/core/bpf_jit_enable and loading any program reproduces the same warning, but the program simply falls back to the interpreter and the kernel keeps running.
Tested with CONFIG_BPF_JIT=y:
- CONFIG_MODULES=n, CONFIG_BPF_JIT_ALWAYS_ON=y:
panics before this patch, boots after
- CONFIG_MODULES=y, CONFIG_STRICT_MODULE_RWX=n, CONFIG_BPF_JIT_ALWAYS_ON=y: same panic before this patch, boots after
- CONFIG_BPF_JIT_ALWAYS_ON=n: the warning above is emitted when a program is loaded from userspace before this patch, and gone after
- riscv defconfig (CONFIG_STRICT_MODULE_RWX=y): unaffected, boots before and after
- kprobes (kretprobe on __riscv_sys_openat via kprobe_events):
works before and after this patch
arch/riscv/kernel/patch.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/riscv/kernel/patch.c +1 -1
@@ -48,7 +48,7 @@ static __always_inline void *patch_map(void *addr, const unsigned int fixmap)if (!IS_ENABLED (CONFIG_STRICT_KERNEL_RWX ))return addr; phys =__pa_symbol (addr); - }else if (IS_ENABLED (CONFIG_STRICT_MODULE_RWX )) { + }else if (IS_ENABLED (CONFIG_EXECMEM )) {struct page *page =vmalloc_to_page (addr);BUG_ON (!page);
Signature · 1 line
2.43.0
Quoted 92 lines · Wei-Jie Hung
Commit
8718e5a3090b("riscv: patch: skip fixmap mapping when kernel text is already writable") gated thecore_kernel_text()branch ofpatch_map()onCONFIG_STRICT_KERNEL_RWX, and explicitly left the vmalloc branch unchanged. That branch has a separate problem.That branch only creates a temporary writable fixmap alias when
CONFIG_STRICT_MODULE_RWXis enabled, and otherwise assumes the target is directly writable and returns the address unchanged. That assumption no longer holds:bpf_prog_pack_alloccallsset_memory_rox()on every pack unconditionally inalloc_new_pack(), independently of any CONFIG_STRICT_*_RWX option, so BPF text in the vmalloc area is read-only regardless of whatCONFIG_STRICT_MODULE_RWXsays.With
CONFIG_STRICT_MODULE_RWX=n,patch_map()returns the read-only address directly, the subsequentcopy_to_kernel_nofault()takes a page fault and returns-EFAULT.bpf_arch_text_copy()turns that into-EINVAL, which propagates throughbpf_jit_binary_pack_finalize()to theWARN_ON()inbpf_int_jit_compile()and leaves the program un-JITed.Note that BPF cannot be fixed the way kprobes was in
commitbdc46e507b59("riscv: mm: makeEXECMEM_KPROBESwritable withoutCONFIG_STRICT_MODULE_RWX").EXECMEM_BPFis alreadyPAGE_KERNEL, i.e. writable at allocation time; the read-only mapping is established afterwards by generic code inalloc_new_pack(), which arch code cannot influence. The only place this can be handled ispatch_map().This is the same problem that was fixed on arm64 by commit
b1480ed230ac("arm64: patching: fix handling of execmem addresses"), and the fix is the same:CONFIG_EXECMEMis what actually tracks whether the vmalloc area can hold executable memory that needs a temporary alias to be written.CONFIG_BPF_JIT,CONFIG_KPROBESandCONFIG_MODULESall select it.Note that this is not limited to
CONFIG_MODULES=n. Unlike arm64, riscv selectsCONFIG_ARCH_OPTIONAL_KERNEL_RWX, soCONFIG_STRICT_MODULE_RWXcan be disabled withCONFIG_MODULES=yas well, and the bug is reachable in that configuration too.The !CONFIG_MMU build is unaffected: it has its own
__patch_insn_set()and__patch_insn_write()implementations and never callspatch_map().Fixes 2c9e5d4a0082 ("bpf: remove CONFIG_BPF_JIT dependency on CONFIG_MODULES of")Signed-off-by Wei-Jie Hung <[email protected]>---
Based on riscv/fixes (b94cec5761d2).Reproduced on qemu-system-riscv64 -M virt with
CONFIG_BPF_JIT=y,CONFIG_BPF_JIT_ALWAYS_ON=yandCONFIG_STRICT_MODULE_RWX=n.ptp_classifier_init()builds a cBPF filter fromsock_init(), so the failure happens during boot without any userspace involved:WARNING: arch/riscv/net/bpf_jit_core.c:156 at bpf_int_jit_compile+0x3dc/0x43e Call Trace: bpf_int_jit_compile+0x3dc/0x43e __bpf_prog_select_runtime+0xec/0x186 bpf_prog_select_runtime+0x12/0x1a bpf_prepare_filter+0x368/0x46a bpf_prog_create+0x66/0x90 ptp_classifier_init+0x3e/0x60 sock_init+0xc4/0xe6 do_one_initcall+0x78/0x14akernel BUG at
net/core/ptp_classifier.c:227!Kernel panic - not syncing: Fatal exception in interruptThe
BUG_ON()is reached becauseCONFIG_BPF_JIT_ALWAYS_ON=yturns the silent interpreter fallback into-ENOTSUPP. WithCONFIG_BPF_JIT_ALWAYS_ON=nthe failure is silent: writing 1 to/proc/sys/net/core/bpf_jit_enableand loading any program reproduces the same warning, but the program simply falls back to the interpreter and the kernel keeps running.Tested with
CONFIG_BPF_JIT=y:
-CONFIG_MODULES=n,CONFIG_BPF_JIT_ALWAYS_ON=y:
panics before this patch, boots after
-CONFIG_MODULES=y,CONFIG_STRICT_MODULE_RWX=n,CONFIG_BPF_JIT_ALWAYS_ON=y: same panic before this patch, boots after
-CONFIG_BPF_JIT_ALWAYS_ON=n: the warning above is emitted when a program is loaded from userspace before this patch, and gone after
- riscv defconfig (CONFIG_STRICT_MODULE_RWX=y): unaffected, boots before and after
- kprobes (kretprobe on__riscv_sys_openatviakprobe_events):
works before and after this patcharch/riscv/kernel/patch.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)arch/riscv/kernel/patch.c +0
-0
arch/riscv/kernel/patch.c static __always_inline void *patch_map(void *addr, const unsigned int fixmap) · line 54
Quoted 8 lines
arch/riscv/kernel/patch.c +1
-1@@ -48,7 +48,7 @@ static __always_inline void *patch_map(void *addr, const unsigned int fixmap)if (!IS_ENABLED (CONFIG_STRICT_KERNEL_RWX ))return addr; phys =__pa_symbol (addr); - }else if (IS_ENABLED (CONFIG_STRICT_MODULE_RWX )) { + }else if (IS_ENABLED (CONFIG_EXECMEM )) {struct page *page =vmalloc_to_page (addr);BUG_ON (!page);
Quoted 41 lines · Pu Lehui
Quoted 38 lines · Wei-Jie Hung
Commit
8718e5a3090b("riscv: patch: skip fixmap mapping when kernel text is already writable") gated thecore_kernel_text()branch ofpatch_map()onCONFIG_STRICT_KERNEL_RWX, and explicitly left the vmalloc branch unchanged. That branch has a separate problem.That branch only creates a temporary writable fixmap alias when
CONFIG_STRICT_MODULE_RWXis enabled, and otherwise assumes the target is directly writable and returns the address unchanged. That assumption no longer holds:bpf_prog_pack_alloccallsset_memory_rox()on every pack unconditionally inalloc_new_pack(), independently of any CONFIG_STRICT_*_RWX option, so BPF text in the vmalloc area is read-only regardless of whatCONFIG_STRICT_MODULE_RWXsays.With
CONFIG_STRICT_MODULE_RWX=n,patch_map()returns the read-only address directly, the subsequentcopy_to_kernel_nofault()takes a page fault and returns-EFAULT.bpf_arch_text_copy()turns that into-EINVAL, which propagates throughbpf_jit_binary_pack_finalize()to theWARN_ON()inbpf_int_jit_compile()and leaves the program un-JITed.Note that BPF cannot be fixed the way kprobes was in
commitbdc46e507b59("riscv: mm: makeEXECMEM_KPROBESwritable withoutCONFIG_STRICT_MODULE_RWX").EXECMEM_BPFis alreadyPAGE_KERNEL, i.e. writable at allocation time; the read-only mapping is established afterwards by generic code inalloc_new_pack(), which arch code cannot influence. The only place this can be handled ispatch_map().This is the same problem that was fixed on arm64 by commit
b1480ed230ac("arm64: patching: fix handling of execmem addresses"), and the fix is the same:CONFIG_EXECMEMis what actually tracks whether the vmalloc area can hold executable memory that needs a temporary alias to be written.CONFIG_BPF_JIT,CONFIG_KPROBESandCONFIG_MODULESall select it.Note that this is not limited to
CONFIG_MODULES=n. Unlike arm64, riscv selectsCONFIG_ARCH_OPTIONAL_KERNEL_RWX, soCONFIG_STRICT_MODULE_RWXcan be disabled withCONFIG_MODULES=yas well, and the bug is reachable in that configuration too.
Quoted 7 lines
Quoted 7 lines
The !CONFIG_MMU build is unaffected: it has its own
__patch_insn_set()and__patch_insn_write()implementations and never callspatch_map().Fixes 2c9e5d4a0082 ("bpf: remove CONFIG_BPF_JIT dependency on CONFIG_MODULES of")Signed-off-by Wei-Jie Hung <[email protected]>---
Based on riscv/fixes (b94cec5761d2).
Not clear what tree this patch suppose to go to. If it's bpf-next then pls respin with correct subj,
pw-bot: cr
Quoted 3 lines
Reviewed-by Pu Lehui <[email protected]>Tested-by Pu Lehui <[email protected]>
and keep tags.
Thanks everyone.
Quoted 3 lines · Alexei Starovoitov
Not clear what tree this patch suppose to go to. If it's bpf-next then pls respin with correct subj,
It's not bpf-next -- this is a riscv patch for the riscv tree, based on riscv/fixes (b94cec5761d2). It only touches arch/riscv/kernel/patch.c, and the "riscv: patch:" prefix matches both the file name and the previous fix to the same function, commit 8718e5a3090b ("riscv: patch:
skip fixmap mapping when kernel text is already writable").
[email protected] is Cc'd because the breakage manifests in the BPF JIT -- bpf_arch_text_copy() fails and no program is ever JITed -- not because it should go through a BPF tree. Sorry for the noise in the BPF queue.
If a respin is wanted for any other reason, I'll carry Pu Lehui's tags.
Thanks,
Wei-Jie
Paul Walmsley
maintainerQuoted 33 lines · Wei-Jie Hung
Commit
8718e5a3090b("riscv: patch: skip fixmap mapping when kernel text is already writable") gated thecore_kernel_text()branch ofpatch_map()onCONFIG_STRICT_KERNEL_RWX, and explicitly left the vmalloc branch unchanged. That branch has a separate problem.That branch only creates a temporary writable fixmap alias when
CONFIG_STRICT_MODULE_RWXis enabled, and otherwise assumes the target is directly writable and returns the address unchanged. That assumption no longer holds:bpf_prog_pack_alloccallsset_memory_rox()on every pack unconditionally inalloc_new_pack(), independently of any CONFIG_STRICT_*_RWX option, so BPF text in the vmalloc area is read-only regardless of whatCONFIG_STRICT_MODULE_RWXsays.With
CONFIG_STRICT_MODULE_RWX=n,patch_map()returns the read-only address directly, the subsequentcopy_to_kernel_nofault()takes a page fault and returns-EFAULT.bpf_arch_text_copy()turns that into-EINVAL, which propagates throughbpf_jit_binary_pack_finalize()to theWARN_ON()inbpf_int_jit_compile()and leaves the program un-JITed.Note that BPF cannot be fixed the way kprobes was in
commitbdc46e507b59("riscv: mm: makeEXECMEM_KPROBESwritable withoutCONFIG_STRICT_MODULE_RWX").EXECMEM_BPFis alreadyPAGE_KERNEL, i.e. writable at allocation time; the read-only mapping is established afterwards by generic code inalloc_new_pack(), which arch code cannot influence. The only place this can be handled ispatch_map().This is the same problem that was fixed on arm64 by commit
b1480ed230ac("arm64: patching: fix handling of execmem addresses"), and the fix is the same:CONFIG_EXECMEMis what actually tracks whether the vmalloc area can hold executable memory that needs a temporary alias to be written.CONFIG_BPF_JIT,CONFIG_KPROBESandCONFIG_MODULESall select it.
Quoted 7 lines
Note that this is not limited to
CONFIG_MODULES=n. Unlike arm64, riscv selectsCONFIG_ARCH_OPTIONAL_KERNEL_RWX, soCONFIG_STRICT_MODULE_RWXcan be disabled withCONFIG_MODULES=yas well, and the bug is reachable in that configuration too.The !CONFIG_MMU build is unaffected: it has its own
__patch_insn_set()and__patch_insn_write()implementations and never callspatch_map().
Thanks, queued for v7.3-rc.
- Paul