bpf, riscv: use BPF prog pack allocator in BPF JIT
Subsystems (4)
v3 cover bpf, riscv: use BPF prog pack allocator in BPF JIT
Changes in v2 -> v3:
1. Fix maximum width of code in patches from 80 to 100. [All patches]
2. Add checks for ctx->ro_insns == NULL. [Patch 3]
3. Fix check for edge condition where amount of text to set > 2 * pagesize [Patch 1 and 2]
4. Add reviewed-by in patches.
5. Adding results of selftest here:
Using the command: ./test_progs on qemu
Without the series: Summary: 336/3162 PASSED, 56 SKIPPED, 90 FAILED With this series: Summary: 336/3162 PASSED, 56 SKIPPED, 90 FAILED
Changes in v1 -> v2:
1. Implement a new function patch_text_set_nosync() to be used in bpf_arch_text_invalidate(). The implementation in v1 called patch_text_nosync() in a loop and it was bad as it would call flush_icache_range() for every word making it really slow. This was found by running the test_tag selftest which would take forever to complete.
Here is some data to prove the V2 fixes the problem:
Without this series:
root@rv-selftester:~/src/kselftest/bpf# time ./test_tag test_tag: OK (40945 tests)
real 7m47.562s user 0m24.145s sys 6m37.064s
With this series applied:
root@rv-selftester:~/src/selftest/bpf# time ./test_tag test_tag: OK (40945 tests)
real 7m29.472s user 0m25.865s sys 6m18.401s
BPF programs currently consume a page each on RISCV. For systems with many BPF programs, this adds significant pressure to instruction TLB. High iTLB pressure usually causes slow down for the whole system.
Song Liu introduced the BPF prog pack allocator[1] to mitigate the above issue. It packs multiple BPF programs into a single huge page. It is currently only enabled for the x86_64 BPF JIT.
I enabled this allocator on the ARM64 BPF JIT[2]. It is being reviewed now.
This patch series enables the BPF prog pack allocator for the RISCV BPF JIT. This series needs a patch[3] from the ARM64 series to work.
======================================================
Performance Analysis of prog pack allocator on RISCV64
======================================================
Test setup:
===========
Host machine: Debian GNU/Linux 11 (bullseye)
Qemu Version: QEMU emulator version 8.0.3 (Debian 1:8.0.3+dfsg-1) u-boot-qemu Version: 2023.07+dfsg-1
opensbi Version: 1.3-1
To test the performance of the BPF prog pack allocator on RV, a stresser tool[4] linked below was built. This tool loads 8 BPF programs on the system and triggers 5 of them in an infinite loop by doing system calls.
The runner script starts 20 instances of the above which loads 8*20=160 BPF programs on the system, 5*20=100 of which are being constantly triggered. The script is passed a command which would be run in the above environment.
The script was run with following perf command:
./run.sh "perf stat -a \
-e iTLB-load-misses \
-e dTLB-load-misses \
-e dTLB-store-misses \
-e instructions \
--timeout 60000"The output of the above command is discussed below before and after enabling the BPF prog pack allocator.
The tests were run on qemu-system-riscv64 with 8 cpus, 16G memory. The rootfs was created using Bjorn's riscv-cross-builder[5] docker container linked below.
Results
=======
Before enabling prog pack allocator:
------------------------------------
Performance counter stats for 'system wide':
4939048 iTLB-load-misses
5468689 dTLB-load-misses
465234 dTLB-store-misses
1441082097998 instructions60.045791200 seconds time elapsed
After enabling prog pack allocator:
-----------------------------------
Performance counter stats for 'system wide':
3430035 iTLB-load-misses
5008745 dTLB-load-misses
409944 dTLB-store-misses
1441535637988 instructions60.046296600 seconds time elapsed
Improvements in metrics
=======================
It was expected that the iTLB-load-misses would decrease as now a single huge page is used to keep all the BPF programs compared to a single page for each program earlier.
--------------------------------------------
The improvement in iTLB-load-misses: -30.5 %
--------------------------------------------
I repeated this expriment more than 100 times in different setups and the improvement was always greater than 30%.
This patch series is boot tested on the Starfive VisionFive 2 board[6]. The performance analysis was not done on the board because it doesn't expose iTLB-load-misses, etc. The stresser program was run on the board to test the loading and unloading of BPF programs
[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/
[3] https://lore.kernel.org/all/[email protected]/
[4] https://github.com/puranjaymohan/BPF-Allocator-Bench
[5] https://github.com/bjoto/riscv-cross-builder
[6] https://www.starfivetech.com/en/site/boards
Puranjay Mohan (3):
riscv: extend patch_text_nosync() for multiple pages
riscv: implement a memset like function for text bpf, riscv: use prog pack allocator in the BPF JIT
arch/riscv/include/asm/patch.h | 1 + arch/riscv/kernel/patch.c | 114 ++++++++++++++++++++++++++++++-- arch/riscv/net/bpf_jit.h | 3 + arch/riscv/net/bpf_jit_comp64.c | 60 +++++++++++++---- arch/riscv/net/bpf_jit_core.c | 106 ++++++++++++++++++++++++----- 5 files changed, 251 insertions(+), 33 deletions(-)
Signature · 1 line
2.40.1
PATCH v3 1/3 riscv: extend patch_text_nosync() for multiple pages
The patch_insn_write() function currently doesn't work for multiple pages of instructions, therefore patch_text_nosync() will fail with a page fault if called with lengths spanning multiple pages.
This commit extends the patch_insn_write() function to support multiple pages by copying at max 2 pages at a time in a loop. This implementation is similar to text_poke_copy() function of x86.
--- arch/riscv/kernel/patch.c | 37 ++++++++++++++++++++++++++++++++----- 1 file changed, 32 insertions(+), 5 deletions(-)
arch/riscv/kernel/patch.c +32 -5
@@ -53,12 +53,18 @@ static void patch_unmap(int fixmap) }NOKPROBE_SYMBOL (patch_unmap); -static int patch_insn_write (void *addr,const void *insn,size_t len) +static int __patch_insn_write (void *addr,const void *insn,size_t len) {void *waddr = addr;bool across_pages = (((uintptr_t ) addr & ~PAGE_MASK ) + len) >PAGE_SIZE ;int ret; +/* +* Only two pages can be mapped at a time for writing. +*/ +if (len +offset_in_page (addr) >2 *PAGE_SIZE ) +return -EINVAL ; +/* * Before reaching here, it was expected to lock the text_mutex * already, so we don't need to give another lock here and could
@@ -74,7 +80,7 @@ static int patch_insn_write(void *addr, const void *insn, size_t len)lockdep_assert_held (&text_mutex);if (across_pages) -patch_map (addr + len,FIX_TEXT_POKE1 ); +patch_map (addr +PAGE_SIZE ,FIX_TEXT_POKE1 ); waddr =patch_map (addr,FIX_TEXT_POKE0 );
@@ -87,15 +93,36 @@ static int patch_insn_write(void *addr, const void *insn, size_t len)return ret; } -NOKPROBE_SYMBOL (patch_insn_write); +NOKPROBE_SYMBOL (__patch_insn_write);#else -static int patch_insn_write (void *addr,const void *insn,size_t len) +static int __patch_insn_write (void *addr,const void *insn,size_t len) {return copy_to_kernel_nofault (addr, insn, len); } -NOKPROBE_SYMBOL (patch_insn_write); +NOKPROBE_SYMBOL (__patch_insn_write);#endif /* CONFIG_MMU */ +static int patch_insn_write (void *addr,const void *insn,size_t len) +{ +size_t patched =0 ; +size_t size; +int ret =0 ; + +/* +* Copy the instructions to the destination address, two pages at a time +* because __patch_insn_write() can only handle len <= 2 * PAGE_SIZE. +*/ +while (patched < len && !ret) { + size =min_t (size_t,PAGE_SIZE *2 -offset_in_page (addr + patched), len - patched); + ret =__patch_insn_write (addr + patched, insn + patched, size); + + patched += size; + } + +return ret; +} +NOKPROBE_SYMBOL (patch_insn_write); +int patch_text_nosync (void *addr,const void *insns,size_t len) {u32 *tp = addr;
Signature · 1 line
2.40.1
Re: [PATCH bpf-next v3 1/3] riscv: extend patch_text_nosync() for multiple pages
Quoted 77 lines · Puranjay Mohan
The
patch_insn_write()function currently doesn't work for multiple pages of instructions, thereforepatch_text_nosync()will fail with a page fault if called with lengths spanning multiple pages.This commit extends the
patch_insn_write()function to support multiple pages by copying at max 2 pages at a time in a loop. This implementation is similar totext_poke_copy()function of x86.Signed-off-by Puranjay Mohan <[email protected]>Reviewed-by Björn Töpel <[email protected]>--- arch/riscv/kernel/patch.c | 37 ++++++++++++++++++++++++++++++++----- 1 file changed, 32 insertions(+), 5 deletions(-)arch/riscv/kernel/patch.c +25
-5@@ -53,12 +53,18 @@ static void patch_unmap(int fixmap) }NOKPROBE_SYMBOL (patch_unmap); -static int patch_insn_write (void *addr,const void *insn,size_t len) +static int __patch_insn_write (void *addr,const void *insn,size_t len) {void *waddr = addr;bool across_pages = (((uintptr_t ) addr & ~PAGE_MASK ) + len) >PAGE_SIZE ;int ret; +/* +* Only two pages can be mapped at a time for writing. +*/ +if (len +offset_in_page (addr) >2 *PAGE_SIZE ) +return -EINVAL ; +/* * Before reaching here, it was expected to lock the text_mutex * already, so we don't need to give another lock here and could @@ -74,7 +80,7 @@ static int patch_insn_write(void *addr, const void *insn, size_t len)lockdep_assert_held (&text_mutex);if (across_pages) -patch_map (addr + len,FIX_TEXT_POKE1 ); +patch_map (addr +PAGE_SIZE ,FIX_TEXT_POKE1 ); waddr =patch_map (addr,FIX_TEXT_POKE0 );@@ -87,15 +93,36 @@ static int patch_insn_write(void *addr, const void *insn, size_t len) return ret; } -NOKPROBE_SYMBOL (patch_insn_write); +NOKPROBE_SYMBOL (__patch_insn_write);#else -static int patch_insn_write (void *addr,const void *insn,size_t len) +static int __patch_insn_write (void *addr,const void *insn,size_t len) {return copy_to_kernel_nofault (addr, insn, len); } -NOKPROBE_SYMBOL (patch_insn_write); +NOKPROBE_SYMBOL (__patch_insn_write);#endif /* CONFIG_MMU */ +static int patch_insn_write (void *addr,const void *insn,size_t len) +{ +size_t patched =0 ; +size_t size; +int ret =0 ; + +/* +* Copy the instructions to the destination address, two pages at a time +* because __patch_insn_write() can only handle len <= 2 * PAGE_SIZE. +*/ +while (patched < len && !ret) { + size =min_t (size_t,PAGE_SIZE *2 -offset_in_page (addr + patched), len - patched); + ret =__patch_insn_write (addr + patched, insn + patched, size); +
arch/riscv/kernel/patch.c static int patch_insn_write(void *addr, const void *insn, size_t len) · line 128
Quoted 10 lines
arch/riscv/kernel/patch.c +7
-0+ patched += size; + } + +return ret; +} +NOKPROBE_SYMBOL (patch_insn_write); +int patch_text_nosync (void *addr,const void *insns,size_t len) {u32 *tp = addr;
Looks good to me,
PATCH v3 2/3 riscv: implement a memset like function for text
In reply to Puranjay MohanThe BPF JIT needs to write invalid instructions to RX regions of memory to invalidate removed BPF programs. This needs a function like memset() that can work with RX memory.
Implement patch_text_set_nosync() which is similar to text_poke_set() of x86.
--- arch/riscv/include/asm/patch.h | 1 + arch/riscv/kernel/patch.c | 77 ++++++++++++++++++++++++++++++++++ 2 files changed, 78 insertions(+)
arch/riscv/include/asm/patch.h +1 -0
@@ -7,6 +7,7 @@#define _ASM_RISCV_PATCH_Hint patch_text_nosync (void *addr,const void *insns,size_t len); +int patch_text_set_nosync (void *addr,u8 c,size_t len);int patch_text (void *addr,u32 *insns,int ninsns);extern int riscv_patch_in_stop_machine;
arch/riscv/kernel/patch.c +77 -0
101 lines not on this page · raw message
Signature · 1 line
2.40.1
Re: [PATCH bpf-next v3 2/3] riscv: implement a memset like function for text
Quoted 124 lines · Puranjay Mohan
The BPF JIT needs to write invalid instructions to RX regions of memory to invalidate removed BPF programs. This needs a function like
memset()that can work with RX memory.Implement
patch_text_set_nosync()which is similar totext_poke_set()of x86.Signed-off-by Puranjay Mohan <[email protected]>--- arch/riscv/include/asm/patch.h | 1 + arch/riscv/kernel/patch.c | 77 ++++++++++++++++++++++++++++++++++ 2 files changed, 78 insertions(+)arch/riscv/include/asm/patch.h +1
-0@@ -7,6 +7,7 @@#define _ASM_RISCV_PATCH_Hint patch_text_nosync (void *addr,const void *insns,size_t len); +int patch_text_set_nosync (void *addr,u8 c,size_t len);int patch_text (void *addr,u32 *insns,int ninsns);extern int riscv_patch_in_stop_machine;arch/riscv/kernel/patch.c +70
-0@@ -6,6 +6,7 @@#include <linux/spinlock.h> #include <linux/mm.h> #include <linux/memory.h> +#include <linux/string.h> #include <linux/uaccess.h> #include <linux/stop_machine.h> #include <asm/kprobes.h> @@ -53,6 +54,39 @@ static void patch_unmap(int fixmap) }NOKPROBE_SYMBOL (patch_unmap); +static int __patch_insn_set (void *addr,u8 c,size_t len) +{ +void *waddr = addr; +bool across_pages = (((uintptr_t )addr & ~PAGE_MASK ) + len) >PAGE_SIZE ; + +/* +* Only two pages can be mapped at a time for writing. +*/ +if (len +offset_in_page (addr) >2 *PAGE_SIZE ) +return -EINVAL ; +/* +* Before reaching here, it was expected to lock the text_mutex +* already, so we don't need to give another lock here and could +* ensure that it was safe between each cores. +*/ +lockdep_assert_held (&text_mutex); + +if (across_pages) +patch_map (addr +PAGE_SIZE ,FIX_TEXT_POKE1 ); + + waddr =patch_map (addr,FIX_TEXT_POKE0 ); + +memset (waddr, c, len); + +patch_unmap (FIX_TEXT_POKE0 ); + +if (across_pages) +patch_unmap (FIX_TEXT_POKE1 ); + +return 0 ; +} +NOKPROBE_SYMBOL (__patch_insn_set); +static int __patch_insn_write (void *addr,const void *insn,size_t len) {void *waddr = addr;@@ -95,6 +129,14 @@ static int __patch_insn_write(void *addr, const void *insn, size_t len) }NOKPROBE_SYMBOL (__patch_insn_write);#else +static int __patch_insn_set (void *addr,u8 c,size_t len) +{ +memset (addr, c, len); + +return 0 ; +} +NOKPROBE_SYMBOL (__patch_insn_set); +static int __patch_insn_write (void *addr,const void *insn,size_t len) {return copy_to_kernel_nofault (addr, insn, len);@@ -102,6 +144,41 @@ static int __patch_insn_write(void *addr, const void *insn, size_t len)NOKPROBE_SYMBOL (__patch_insn_write);#endif /* CONFIG_MMU */ +static int patch_insn_set (void *addr,u8 c,size_t len) +{ +size_t patched =0 ; +size_t size; +int ret =0 ; + +/* +* __patch_insn_set() can only work on 2 pages at a time so call it in a +* loop with len <= 2 * PAGE_SIZE. +*/ +while (patched < len && !ret) { + size =min_t (size_t,PAGE_SIZE *2 -offset_in_page (addr + patched), len - patched); + ret =__patch_insn_set (addr + patched, c, size); + + patched += size; + } + +return ret; +} +NOKPROBE_SYMBOL (patch_insn_set); + +int patch_text_set_nosync (void *addr,u8 c,size_t len) +{ +u32 *tp = addr; +int ret; + + ret =patch_insn_set (tp, c, len); +
arch/riscv/kernel/patch.c static int __patch_insn_write(void *addr, const void *insn, size_t len) · line 184
Quoted 10 lines
arch/riscv/kernel/patch.c +7
-0+if (!ret) +flush_icache_range ((uintptr_t )tp, (uintptr_t )tp + len); + +return ret; +} +NOKPROBE_SYMBOL (patch_text_set_nosync); +static int patch_insn_write (void *addr,const void *insn,size_t len) {size_t patched =0 ;
PATCH v3 3/3 bpf, riscv: use prog pack allocator in the BPF JIT
In reply to Puranjay MohanUse bpf_jit_binary_pack_alloc() for memory management of JIT binaries in RISCV BPF JIT. The bpf_jit_binary_pack_alloc creates a pair of RW and RX buffers. The JIT writes the program into the RW buffer. When the JIT is done, the program is copied to the final RX buffer with bpf_jit_binary_pack_finalize.
Implement bpf_arch_text_copy() and bpf_arch_text_invalidate() for RISCV JIT as these functions are required by bpf_jit_binary_pack allocator.
--- arch/riscv/net/bpf_jit.h | 3 + arch/riscv/net/bpf_jit_comp64.c | 60 ++++++++++++++---- arch/riscv/net/bpf_jit_core.c | 106 +++++++++++++++++++++++++++----- 3 files changed, 141 insertions(+), 28 deletions(-)
arch/riscv/net/bpf_jit.h +3 -0
@@ -68,6 +68,7 @@ static inline bool is_creg(u8 reg)struct rv_jit_context {struct bpf_prog *prog;u16 *insns;/* RV insns */ +u16 *ro_insns;int ninsns;int prologue_len;int epilogue_offset;
@@ -85,7 +86,9 @@ static inline int ninsns_rvoff(int ninsns)struct rv_jit_data {struct bpf_binary_header *header; +struct bpf_binary_header *ro_header;u8 *image; +u8 *ro_image;struct rv_jit_context ctx; };
arch/riscv/net/bpf_jit_comp64.c +48 -12
109 lines not on this page · raw message
arch/riscv/net/bpf_jit_core.c +90 -16
158 lines not on this page · raw message
Signature · 1 line
2.40.1
Re: [PATCH bpf-next v3 3/3] bpf, riscv: use prog pack allocator in the BPF JIT
Quoted 317 lines · Puranjay Mohan
Use
bpf_jit_binary_pack_alloc()for memory management of JIT binaries in RISCV BPF JIT. Thebpf_jit_binary_pack_alloccreates a pair of RW and RX buffers. The JIT writes the program into the RW buffer. When the JIT is done, the program is copied to the final RX buffer withbpf_jit_binary_pack_finalize.Implement
bpf_arch_text_copy()andbpf_arch_text_invalidate()for RISCV JIT as these functions are required bybpf_jit_binary_packallocator.Signed-off-by Puranjay Mohan <[email protected]>Reviewed-by Song Liu <[email protected]>--- arch/riscv/net/bpf_jit.h | 3 + arch/riscv/net/bpf_jit_comp64.c | 60 ++++++++++++++---- arch/riscv/net/bpf_jit_core.c | 106 +++++++++++++++++++++++++++----- 3 files changed, 141 insertions(+), 28 deletions(-)arch/riscv/net/bpf_jit.h +3
-0@@ -68,6 +68,7 @@ static inline bool is_creg(u8 reg)struct rv_jit_context {struct bpf_prog *prog;u16 *insns;/* RV insns */ +u16 *ro_insns;int ninsns;int prologue_len;int epilogue_offset;@@ -85,7 +86,9 @@ static inline int ninsns_rvoff(int ninsns)struct rv_jit_data {struct bpf_binary_header *header; +struct bpf_binary_header *ro_header;u8 *image; +u8 *ro_image;struct rv_jit_context ctx; };arch/riscv/net/bpf_jit_comp64.c +48
-12@@ -144,7 +144,11 @@ static bool in_auipc_jalr_range(s64 val)/* Emit fixed-length instructions for address */ static int emit_addr (u8 rd,u64 addr,bool extra_pass,struct rv_jit_context *ctx) { -u64 ip = (u64)(ctx->insns + ctx->ninsns); +/* +* Use the ro_insns(RX) to calculate the offset as the BPF program will +* finally run from this memory region. +*/ +u64 ip = (u64)(ctx->ro_insns + ctx->ninsns);s64 off = addr - ip;s64 upper = (off + (1 <<11 )) >>12 ;s64 lower = off &0xfff ;@@ -464,8 +468,12 @@ static int emit_call(u64 addr, bool fixed_addr, struct rv_jit_context *ctx)s64 off =0 ;u64 ip; -if (addr && ctx->insns) { - ip = (u64)(long)(ctx->insns + ctx->ninsns); +if (addr && ctx->insns && ctx->ro_insns) { +/* +* Use the ro_insns(RX) to calculate the offset as the BPF +* program will finally run from this memory region. +*/ + ip = (u64)(long)(ctx->ro_insns + ctx->ninsns); off = addr - ip; }@@ -578,9 +586,10 @@ static int add_exception_handler(const struct bpf_insn *insn, {struct exception_table_entry *ex;unsigned long pc; -off_t offset; +off_t ins_offset; +off_t fixup_offset; -if (!ctx->insns || !ctx->prog->aux->extable || +if (!ctx->insns || !ctx->ro_insns || !ctx->prog->aux->extable || (BPF_MODE (insn->code) !=BPF_PROBE_MEM &&BPF_MODE (insn->code) !=BPF_PROBE_MEMSX ))return 0 ;@@ -594,12 +603,17 @@ static int add_exception_handler(const struct bpf_insn *insn,return -EINVAL ; ex = &ctx->prog->aux->extable[ctx->nexentries]; - pc = (unsigned long )&ctx->insns[ctx->ninsns - insn_len]; + pc = (unsigned long )&ctx->ro_insns[ctx->ninsns - insn_len]; - offset = pc - (long )&ex->insn; -if (WARN_ON_ONCE (offset >=0 || offset <INT_MIN )) +/* +* This is the relative offset of the instruction that may fault from +* the exception table itself. This will be written to the exception +* table and if this instruction faults, the destination register will +* be set to '0' and the execution will jump to the next instruction. +*/ + ins_offset = pc - (long )&ex->insn; +if (WARN_ON_ONCE (ins_offset >=0 || ins_offset <INT_MIN ))return -ERANGE ; - ex->insn = offset;/* * Since the extable follows the program, the fixup offset is always @@ -608,12 +622,25 @@ static int add_exception_handler(const struct bpf_insn *insn,* bits. We don't need to worry about buildtime or runtime sort * modifying the upper bits because the table is already sorted, and * isn't part of the main exception table. +* +* The fixup_offset is set to the next instruction from the instruction +* that may fault. The execution will jump to this after handling the +* fault. */ - offset = (long )&ex->fixup - (pc + insn_len *sizeof (u16)); -if (!FIELD_FIT (BPF_FIXUP_OFFSET_MASK , offset)) + fixup_offset = (long )&ex->fixup - (pc + insn_len *sizeof (u16)); +if (!FIELD_FIT (BPF_FIXUP_OFFSET_MASK , fixup_offset))return -ERANGE ; - ex->fixup =FIELD_PREP (BPF_FIXUP_OFFSET_MASK , offset) | +/* +* The offsets above have been calculated using the RO buffer but we +* need to use the R/W buffer for writes. +* switch ex to rw buffer for writing. +*/ + ex = (void *)ctx->insns + ((void *)ex - (void *)ctx->ro_insns); + + ex->insn = ins_offset; + + ex->fixup =FIELD_PREP (BPF_FIXUP_OFFSET_MASK , fixup_offset) |FIELD_PREP (BPF_FIXUP_REG_MASK , dst_reg); ex->type =EX_TYPE_BPF ;@@ -1007,6 +1034,7 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, ctx.ninsns =0 ; ctx.insns =NULL ; + ctx.ro_insns =NULL ; ret =__arch_prepare_bpf_trampoline (im, m, tlinks, func_addr, flags, &ctx);if (ret <0 )return ret;@@ -1015,7 +1043,15 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image,return -EFBIG ; ctx.ninsns =0 ; +/* +* The bpf_int_jit_compile() uses a RW buffer (ctx.insns) to write the +* JITed instructions and later copies it to a RX region (ctx.ro_insns). +* It also uses ctx.ro_insns to calculate offsets for jumps etc. As the +* trampoline image uses the same memory area for writing and execution, +* both ctx.insns and ctx.ro_insns can be set to image. +*/ ctx.insns = image; + ctx.ro_insns = image; ret =__arch_prepare_bpf_trampoline (im, m, tlinks, func_addr, flags, &ctx);if (ret <0 )return ret;arch/riscv/net/bpf_jit_core.c +80
-16@@ -8,6 +8,8 @@#include <linux/bpf.h> #include <linux/filter.h> +#include <linux/memory.h> +#include <asm/patch.h> #include "bpf_jit.h" /* Number of iterations to try until offsets converge. */ @@ -117,16 +119,24 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)sizeof (struct exception_table_entry ); prog_size =sizeof (*ctx->insns) * ctx->ninsns; - jit_data->header = -bpf_jit_binary_alloc (prog_size + extable_size, - &jit_data->image, -sizeof (u32), - bpf_fill_ill_insns); -if (!jit_data->header) { + jit_data->ro_header = +bpf_jit_binary_pack_alloc (prog_size + extable_size, + &jit_data->ro_image,sizeof (u32), + &jit_data->header, &jit_data->image, + bpf_fill_ill_insns); +if (!jit_data->ro_header) { prog = orig_prog; goto out_offset; } +/* +* Use the image(RW) for writing the JITed instructions. But also save +* the ro_image(RX) for calculating the offsets in the image. The RW +* image will be later copied to the RX image from where the program +* will run. The bpf_jit_binary_pack_finalize() will do this copy in the +* final step. +*/ + ctx->ro_insns = (u16 *)jit_data->ro_image; ctx->insns = (u16 *)jit_data->image;/* * Now, when the image is allocated, the image can @@ -138,14 +148,12 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)if (i ==NR_JIT_ITERATIONS ) {pr_err ("bpf-jit: image did not converge in <%d passes!\n" , i); -if (jit_data->header) -bpf_jit_binary_free (jit_data->header); prog = orig_prog; - goto out_offset; + goto out_free_hdr; }if (extable_size) - prog->aux->extable = (void *)ctx->insns + prog_size; + prog->aux->extable = (void *)ctx->ro_insns + prog_size; skip_init_ctx: pass++;@@ -154,23 +162,33 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)bpf_jit_build_prologue (ctx);if (build_body (ctx, extra_pass,NULL )) { -bpf_jit_binary_free (jit_data->header); prog = orig_prog; - goto out_offset; + goto out_free_hdr; }bpf_jit_build_epilogue (ctx);if (bpf_jit_enable >1 )bpf_jit_dump (prog->len, prog_size, pass, ctx->insns); - prog->bpf_func = (void *)ctx->insns; + prog->bpf_func = (void *)ctx->ro_insns; prog->jited =1 ; prog->jited_len = prog_size; -bpf_flush_icache (jit_data->header, ctx->insns + ctx->ninsns); -if (!prog->is_func || extra_pass) { -bpf_jit_binary_lock_ro (jit_data->header); +if (WARN_ON (bpf_jit_binary_pack_finalize (prog, jit_data->ro_header, + jit_data->header))) { +/* ro_header has been freed */ + jit_data->ro_header =NULL ; + prog = orig_prog; + goto out_offset; + } +/* +* The instructions have now been copied to the ROX region from +* where they will execute. +* Write any modified data cache blocks out to memory and +* invalidate the corresponding blocks in the instruction cache. +*/ +bpf_flush_icache (jit_data->ro_header, ctx->ro_insns + ctx->ninsns);for (i =0 ; i < prog->len; i++) ctx->offset[i] =ninsns_rvoff (ctx->offset[i]);bpf_prog_fill_jited_linfo (prog, ctx->offset);@@ -185,6 +203,14 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)bpf_jit_prog_release_other (prog, prog == orig_prog ? tmp : orig_prog);return prog; + +out_free_hdr: +if (jit_data->header) { +bpf_arch_text_copy (&jit_data->ro_header->size, &jit_data->header->size, +sizeof (jit_data->header->size)); +bpf_jit_binary_pack_free (jit_data->ro_header, jit_data->header); + } + goto out_offset; }u64 bpf_jit_alloc_exec_limit (void )@@ -204,3 +230,51 @@ void bpf_jit_free_exec(void *addr) {return vfree (addr); } + +void *bpf_arch_text_copy (void *dst,void *src,size_t len) +{ +int ret; + +mutex_lock (&text_mutex); + ret =patch_text_nosync (dst, src, len); +mutex_unlock (&text_mutex); + +if (ret) +return ERR_PTR (-EINVAL ); + +return dst; +} + +int bpf_arch_text_invalidate (void *dst,size_t len) +{ +int ret; + +mutex_lock (&text_mutex); + ret =patch_text_set_nosync (dst,0 , len); +mutex_unlock (&text_mutex); + +return ret; +} + +void bpf_jit_free (struct bpf_prog *prog) +{ +if (prog->jited) { +struct rv_jit_data *jit_data = prog->aux->jit_data; +struct bpf_binary_header *hdr; + +/* +* If we fail the final pass of JIT (from jit_subprogs), +* the program may not be finalized yet. Call finalize here +* before freeing it. +*/ +if (jit_data) {
arch/riscv/net/bpf_jit_core.c void bpf_jit_free_exec(void *addr) · line 280
Quoted 10 lines
arch/riscv/net/bpf_jit_core.c +10
-0+bpf_jit_binary_pack_finalize (prog, jit_data->ro_header, jit_data->header); +kfree (jit_data); + } + hdr =bpf_jit_binary_pack_hdr (prog); +bpf_jit_binary_pack_free (hdr,NULL ); +WARN_ON_ONCE (!bpf_prog_kallsyms_verify_off (prog)); + } + +bpf_prog_unlock_free (prog); +}
Quoted 135 lines · Puranjay Mohan
Changes in v2 -> v3:
1. Fix maximum width of code in patches from 80 to 100. [All patches]
2. Add checks forctx->ro_insns== NULL. [Patch 3]
3. Fix check for edge condition where amount of text to set > 2 * pagesize [Patch 1 and 2]
4. Add reviewed-by in patches.
5. Adding results of selftest here:
Using the command: ./test_progs on qemu
Without the series: Summary: 336/3162 PASSED, 56 SKIPPED, 90 FAILED With this series: Summary: 336/3162 PASSED, 56 SKIPPED, 90 FAILEDChanges in v1 -> v2:
1. Implement a new functionpatch_text_set_nosync()to be used inbpf_arch_text_invalidate(). The implementation in v1 calledpatch_text_nosync()in a loop and it was bad as it would callflush_icache_range()for every word making it really slow. This was found by running thetest_tagselftest which would take forever to complete.Here is some data to prove the V2 fixes the problem:
Without this series:
root@rv-selftester:~/src/kselftest/bpf# time ./test_tag test_tag: OK (40945 tests)real 7m47.562s user 0m24.145s sys 6m37.064sWith this series applied:
root@rv-selftester:~/src/selftest/bpf# time ./test_tag test_tag: OK (40945 tests)real 7m29.472s user 0m25.865s sys 6m18.401sBPF programs currently consume a page each on RISCV. For systems with many BPF programs, this adds significant pressure to instruction TLB. High iTLB pressure usually causes slow down for the whole system.
Song Liu introduced the BPF prog pack allocator[1] to mitigate the above issue. It packs multiple BPF programs into a single huge page. It is currently only enabled for the
x86_64BPF JIT.I enabled this allocator on the ARM64 BPF JIT[2]. It is being reviewed now.
This patch series enables the BPF prog pack allocator for the RISCV BPF JIT. This series needs a patch[3] from the ARM64 series to work.
======================================================
Performance Analysis of prog pack allocator on RISCV64
======================================================Test setup:
===========Host machine: Debian GNU/Linux 11 (bullseye)
Qemu Version: QEMU emulator version 8.0.3 (Debian 1:8.0.3+dfsg-1) u-boot-qemu Version: 2023.07+dfsg-1
opensbi Version: 1.3-1To test the performance of the BPF prog pack allocator on RV, a stresser tool[4] linked below was built. This tool loads 8 BPF programs on the system and triggers 5 of them in an infinite loop by doing system calls.
The runner script starts 20 instances of the above which loads 8*20=160 BPF programs on the system, 5*20=100 of which are being constantly triggered. The script is passed a command which would be run in the above environment.
The script was run with following perf command:
./run.sh "perf stat -a \ -e iTLB-load-misses \ -e dTLB-load-misses \ -e dTLB-store-misses \ -e instructions \ --timeout 60000"The output of the above command is discussed below before and after enabling the BPF prog pack allocator.
The tests were run on qemu-system-riscv64 with 8 cpus, 16G memory. The rootfs was created using Bjorn's riscv-cross-builder[5] docker container linked below.
Results
=======Before enabling prog pack allocator:
------------------------------------Performance counter stats for 'system wide':
4939048 iTLB-load-misses 5468689 dTLB-load-misses 465234 dTLB-store-misses 1441082097998 instructions60.045791200 seconds time elapsedAfter enabling prog pack allocator:
-----------------------------------Performance counter stats for 'system wide':
3430035 iTLB-load-misses 5008745 dTLB-load-misses 409944 dTLB-store-misses 1441535637988 instructions60.046296600 seconds time elapsedImprovements in metrics
=======================It was expected that the iTLB-load-misses would decrease as now a single huge page is used to keep all the BPF programs compared to a single page for each program earlier.
--------------------------------------------
The improvement in iTLB-load-misses: -30.5 %
--------------------------------------------I repeated this expriment more than 100 times in different setups and the improvement was always greater than 30%.
This patch series is boot tested on the Starfive VisionFive 2 board[6]. The performance analysis was not done on the board because it doesn't expose iTLB-load-misses, etc. The stresser program was run on the board to test the loading and unloading of BPF programs
[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/
[3] https://lore.kernel.org/all/[email protected]/
[4] https://github.com/puranjaymohan/BPF-Allocator-Bench
[5] https://github.com/bjoto/riscv-cross-builder
[6] https://www.starfivetech.com/en/site/boards
Quoted 4 lines
Puranjay Mohan (3):
riscv: extendpatch_text_nosync()for multiple pages
riscv: implement a memset like function for text bpf, riscv: use prog pack allocator in the BPF JIT
Thank you! For the series:
@Alexei @Daniel This series depends on a core BPF patch from the Arm
series [3].
@Palmer LMK if you have any concerns taking the RISC-V text patching
stuff via the BPF tree.
Björn
Daniel Borkmann
maintainerQuoted 144 lines · Björn Töpel
Quoted 135 lines · Puranjay Mohan
Changes in v2 -> v3:
1. Fix maximum width of code in patches from 80 to 100. [All patches]
2. Add checks forctx->ro_insns== NULL. [Patch 3]
3. Fix check for edge condition where amount of text to set > 2 * pagesize[Patch 1 and 2]4. Add reviewed-by in patches.
5. Adding results of selftest here:
Using the command: ./test_progs on qemu
Without the series: Summary: 336/3162 PASSED, 56 SKIPPED, 90 FAILED With this series: Summary: 336/3162 PASSED, 56 SKIPPED, 90 FAILEDChanges in v1 -> v2:
1. Implement a new functionpatch_text_set_nosync()to be used inbpf_arch_text_invalidate(). The implementation in v1 calledpatch_text_nosync()in a loop and it was bad as it would callflush_icache_range()for every word making it really slow. This was found by running thetest_tagselftest which would take forever to complete.Here is some data to prove the V2 fixes the problem:
Without this series:
root@rv-selftester:~/src/kselftest/bpf# time ./test_tag test_tag: OK (40945 tests)real 7m47.562s user 0m24.145s sys 6m37.064sWith this series applied:
root@rv-selftester:~/src/selftest/bpf# time ./test_tag test_tag: OK (40945 tests)real 7m29.472s user 0m25.865s sys 6m18.401sBPF programs currently consume a page each on RISCV. For systems with many BPF programs, this adds significant pressure to instruction TLB. High iTLB pressure usually causes slow down for the whole system.
Song Liu introduced the BPF prog pack allocator[1] to mitigate the above issue. It packs multiple BPF programs into a single huge page. It is currently only enabled for the
x86_64BPF JIT.I enabled this allocator on the ARM64 BPF JIT[2]. It is being reviewed now.
This patch series enables the BPF prog pack allocator for the RISCV BPF JIT. This series needs a patch[3] from the ARM64 series to work.
======================================================
Performance Analysis of prog pack allocator on RISCV64
======================================================Test setup:
===========Host machine: Debian GNU/Linux 11 (bullseye)
Qemu Version: QEMU emulator version 8.0.3 (Debian 1:8.0.3+dfsg-1) u-boot-qemu Version: 2023.07+dfsg-1
opensbi Version: 1.3-1To test the performance of the BPF prog pack allocator on RV, a stresser tool[4] linked below was built. This tool loads 8 BPF programs on the system and triggers 5 of them in an infinite loop by doing system calls.
The runner script starts 20 instances of the above which loads 8*20=160 BPF programs on the system, 5*20=100 of which are being constantly triggered. The script is passed a command which would be run in the above environment.
The script was run with following perf command:
./run.sh "perf stat -a \ -e iTLB-load-misses \ -e dTLB-load-misses \ -e dTLB-store-misses \ -e instructions \ --timeout 60000"The output of the above command is discussed below before and after enabling the BPF prog pack allocator.
The tests were run on qemu-system-riscv64 with 8 cpus, 16G memory. The rootfs was created using Bjorn's riscv-cross-builder[5] docker container linked below.
Results
=======Before enabling prog pack allocator:
------------------------------------Performance counter stats for 'system wide':
4939048 iTLB-load-misses 5468689 dTLB-load-misses 465234 dTLB-store-misses 1441082097998 instructions60.045791200 seconds time elapsedAfter enabling prog pack allocator:
-----------------------------------Performance counter stats for 'system wide':
3430035 iTLB-load-misses 5008745 dTLB-load-misses 409944 dTLB-store-misses 1441535637988 instructions60.046296600 seconds time elapsedImprovements in metrics
=======================It was expected that the iTLB-load-misses would decrease as now a single huge page is used to keep all the BPF programs compared to a single page for each program earlier.
--------------------------------------------
The improvement in iTLB-load-misses: -30.5 %
--------------------------------------------I repeated this expriment more than 100 times in different setups and the improvement was always greater than 30%.
This patch series is boot tested on the Starfive VisionFive 2 board[6]. The performance analysis was not done on the board because it doesn't expose iTLB-load-misses, etc. The stresser program was run on the board to test the loading and unloading of BPF programs
[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/
[3] https://lore.kernel.org/all/[email protected]/
[4] https://github.com/puranjaymohan/BPF-Allocator-Bench
[5] https://github.com/bjoto/riscv-cross-builder
[6] https://www.starfivetech.com/en/site/boardsQuoted 4 lines
Puranjay Mohan (3):
riscv: extendpatch_text_nosync()for multiple pages
riscv: implement a memset like function for text bpf, riscv: use prog pack allocator in the BPF JITThank you! For the series:
Quoted 8 lines
Acked-by Björn Töpel <[email protected]>Tested-by Björn Töpel <[email protected]>@Alexei @Daniel This series depends on a core BPF patch from the Arm
series [3].@Palmer LMK if you have any concerns taking the RISC-V text patching
stuff via the BPF tree.
Palmer, did the riscv PR already go to Linus?
If not yet, perhaps you could ship this series along with your PR to Linus during this merge window given the big net PR (incl. bpf) was already merged yesterday. So from our side only fixes ship to Linus.
Otherwise we could take it into bpf-next for the next dev cycle if there are no objections, let us know.
Thanks,
Daniel
Björn Töpel
maintainerQuoted 154 lines · Daniel Borkmann
Quoted 144 lines · Björn Töpel
Quoted 135 lines · Puranjay Mohan
Changes in v2 -> v3:
1. Fix maximum width of code in patches from 80 to 100. [All patches]
2. Add checks forctx->ro_insns== NULL. [Patch 3]
3. Fix check for edge condition where amount of text to set > 2 * pagesize[Patch 1 and 2]4. Add reviewed-by in patches.
5. Adding results of selftest here:
Using the command: ./test_progs on qemu
Without the series: Summary: 336/3162 PASSED, 56 SKIPPED, 90 FAILED With this series: Summary: 336/3162 PASSED, 56 SKIPPED, 90 FAILEDChanges in v1 -> v2:
1. Implement a new functionpatch_text_set_nosync()to be used inbpf_arch_text_invalidate(). The implementation in v1 calledpatch_text_nosync()in a loop and it was bad as it would callflush_icache_range()for every word making it really slow. This was found by running thetest_tagselftest which would take forever to complete.Here is some data to prove the V2 fixes the problem:
Without this series:
root@rv-selftester:~/src/kselftest/bpf# time ./test_tag test_tag: OK (40945 tests)real 7m47.562s user 0m24.145s sys 6m37.064sWith this series applied:
root@rv-selftester:~/src/selftest/bpf# time ./test_tag test_tag: OK (40945 tests)real 7m29.472s user 0m25.865s sys 6m18.401sBPF programs currently consume a page each on RISCV. For systems with many BPF programs, this adds significant pressure to instruction TLB. High iTLB pressure usually causes slow down for the whole system.
Song Liu introduced the BPF prog pack allocator[1] to mitigate the above issue. It packs multiple BPF programs into a single huge page. It is currently only enabled for the
x86_64BPF JIT.I enabled this allocator on the ARM64 BPF JIT[2]. It is being reviewed now.
This patch series enables the BPF prog pack allocator for the RISCV BPF JIT. This series needs a patch[3] from the ARM64 series to work.
======================================================
Performance Analysis of prog pack allocator on RISCV64
======================================================Test setup:
===========Host machine: Debian GNU/Linux 11 (bullseye)
Qemu Version: QEMU emulator version 8.0.3 (Debian 1:8.0.3+dfsg-1) u-boot-qemu Version: 2023.07+dfsg-1
opensbi Version: 1.3-1To test the performance of the BPF prog pack allocator on RV, a stresser tool[4] linked below was built. This tool loads 8 BPF programs on the system and triggers 5 of them in an infinite loop by doing system calls.
The runner script starts 20 instances of the above which loads 8*20=160 BPF programs on the system, 5*20=100 of which are being constantly triggered. The script is passed a command which would be run in the above environment.
The script was run with following perf command:
./run.sh "perf stat -a \ -e iTLB-load-misses \ -e dTLB-load-misses \ -e dTLB-store-misses \ -e instructions \ --timeout 60000"The output of the above command is discussed below before and after enabling the BPF prog pack allocator.
The tests were run on qemu-system-riscv64 with 8 cpus, 16G memory. The rootfs was created using Bjorn's riscv-cross-builder[5] docker container linked below.
Results
=======Before enabling prog pack allocator:
------------------------------------Performance counter stats for 'system wide':
4939048 iTLB-load-misses 5468689 dTLB-load-misses 465234 dTLB-store-misses 1441082097998 instructions60.045791200 seconds time elapsedAfter enabling prog pack allocator:
-----------------------------------Performance counter stats for 'system wide':
3430035 iTLB-load-misses 5008745 dTLB-load-misses 409944 dTLB-store-misses 1441535637988 instructions60.046296600 seconds time elapsedImprovements in metrics
=======================It was expected that the iTLB-load-misses would decrease as now a single huge page is used to keep all the BPF programs compared to a single page for each program earlier.
--------------------------------------------
The improvement in iTLB-load-misses: -30.5 %
--------------------------------------------I repeated this expriment more than 100 times in different setups and the improvement was always greater than 30%.
This patch series is boot tested on the Starfive VisionFive 2 board[6]. The performance analysis was not done on the board because it doesn't expose iTLB-load-misses, etc. The stresser program was run on the board to test the loading and unloading of BPF programs
[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/
[3] https://lore.kernel.org/all/[email protected]/
[4] https://github.com/puranjaymohan/BPF-Allocator-Bench
[5] https://github.com/bjoto/riscv-cross-builder
[6] https://www.starfivetech.com/en/site/boardsQuoted 4 lines
Puranjay Mohan (3):
riscv: extendpatch_text_nosync()for multiple pages
riscv: implement a memset like function for text bpf, riscv: use prog pack allocator in the BPF JITThank you! For the series:
Quoted 8 lines
Acked-by Björn Töpel <[email protected]>Tested-by Björn Töpel <[email protected]>@Alexei @Daniel This series depends on a core BPF patch from the Arm
series [3].@Palmer LMK if you have any concerns taking the RISC-V text patching
stuff via the BPF tree.
Quoted 8 lines
Palmer, did the riscv PR already go to Linus?
If not yet, perhaps you could ship this series along with your PR to Linus during this merge window given the big net PR (incl. bpf) was already merged yesterday. So from our side only fixes ship to Linus.
Otherwise we could take it into bpf-next for the next dev cycle if there are no objections, let us know.
Daniel, I'll bring this up on the RISC-V patchwork sync meeting [1] today.
[1] https://lore.kernel.org/linux-riscv/mhng-775d4068-6c1e-48a4-a1dc-b4a76ff26bb3@palmer-ri-x1c9a/
Palmer Dabbelt
maintainerQuoted 148 lines · On Wed, 30 Aug 2023 01:18:46 PDT (-0700), [email protected] wrote:
Quoted 147 lines · Björn Töpel
Quoted 135 lines · Puranjay Mohan
Changes in v2 -> v3:
1. Fix maximum width of code in patches from 80 to 100. [All patches]
2. Add checks forctx->ro_insns== NULL. [Patch 3]
3. Fix check for edge condition where amount of text to set > 2 * pagesize[Patch 1 and 2]4. Add reviewed-by in patches.
5. Adding results of selftest here:
Using the command: ./test_progs on qemu
Without the series: Summary: 336/3162 PASSED, 56 SKIPPED, 90 FAILED With this series: Summary: 336/3162 PASSED, 56 SKIPPED, 90 FAILEDChanges in v1 -> v2:
1. Implement a new functionpatch_text_set_nosync()to be used inbpf_arch_text_invalidate(). The implementation in v1 calledpatch_text_nosync()in a loop and it was bad as it would callflush_icache_range()for every word making it really slow. This was found by running thetest_tagselftest which would take forever to complete.Here is some data to prove the V2 fixes the problem:
Without this series:
root@rv-selftester:~/src/kselftest/bpf# time ./test_tag test_tag: OK (40945 tests)real 7m47.562s user 0m24.145s sys 6m37.064sWith this series applied:
root@rv-selftester:~/src/selftest/bpf# time ./test_tag test_tag: OK (40945 tests)real 7m29.472s user 0m25.865s sys 6m18.401sBPF programs currently consume a page each on RISCV. For systems with many BPF programs, this adds significant pressure to instruction TLB. High iTLB pressure usually causes slow down for the whole system.
Song Liu introduced the BPF prog pack allocator[1] to mitigate the above issue. It packs multiple BPF programs into a single huge page. It is currently only enabled for the
x86_64BPF JIT.I enabled this allocator on the ARM64 BPF JIT[2]. It is being reviewed now.
This patch series enables the BPF prog pack allocator for the RISCV BPF JIT. This series needs a patch[3] from the ARM64 series to work.
======================================================
Performance Analysis of prog pack allocator on RISCV64
======================================================Test setup:
===========Host machine: Debian GNU/Linux 11 (bullseye)
Qemu Version: QEMU emulator version 8.0.3 (Debian 1:8.0.3+dfsg-1) u-boot-qemu Version: 2023.07+dfsg-1
opensbi Version: 1.3-1To test the performance of the BPF prog pack allocator on RV, a stresser tool[4] linked below was built. This tool loads 8 BPF programs on the system and triggers 5 of them in an infinite loop by doing system calls.
The runner script starts 20 instances of the above which loads 8*20=160 BPF programs on the system, 5*20=100 of which are being constantly triggered. The script is passed a command which would be run in the above environment.
The script was run with following perf command:
./run.sh "perf stat -a \ -e iTLB-load-misses \ -e dTLB-load-misses \ -e dTLB-store-misses \ -e instructions \ --timeout 60000"The output of the above command is discussed below before and after enabling the BPF prog pack allocator.
The tests were run on qemu-system-riscv64 with 8 cpus, 16G memory. The rootfs was created using Bjorn's riscv-cross-builder[5] docker container linked below.
Results
=======Before enabling prog pack allocator:
------------------------------------Performance counter stats for 'system wide':
4939048 iTLB-load-misses 5468689 dTLB-load-misses 465234 dTLB-store-misses 1441082097998 instructions60.045791200 seconds time elapsedAfter enabling prog pack allocator:
-----------------------------------Performance counter stats for 'system wide':
3430035 iTLB-load-misses 5008745 dTLB-load-misses 409944 dTLB-store-misses 1441535637988 instructions60.046296600 seconds time elapsedImprovements in metrics
=======================It was expected that the iTLB-load-misses would decrease as now a single huge page is used to keep all the BPF programs compared to a single page for each program earlier.
--------------------------------------------
The improvement in iTLB-load-misses: -30.5 %
--------------------------------------------I repeated this expriment more than 100 times in different setups and the improvement was always greater than 30%.
This patch series is boot tested on the Starfive VisionFive 2 board[6]. The performance analysis was not done on the board because it doesn't expose iTLB-load-misses, etc. The stresser program was run on the board to test the loading and unloading of BPF programs
[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/
[3] https://lore.kernel.org/all/[email protected]/
[4] https://github.com/puranjaymohan/BPF-Allocator-Bench
[5] https://github.com/bjoto/riscv-cross-builder
[6] https://www.starfivetech.com/en/site/boardsQuoted 4 lines
Puranjay Mohan (3):
riscv: extendpatch_text_nosync()for multiple pages
riscv: implement a memset like function for text bpf, riscv: use prog pack allocator in the BPF JITThank you! For the series:
Acked-by Björn Töpel <[email protected]>Tested-by Björn Töpel <[email protected]>
Quoted 7 lines
Quoted 5 lines
@Alexei @Daniel This series depends on a core BPF patch from the Arm
series [3].@Palmer LMK if you have any concerns taking the RISC-V text patching
stuff via the BPF tree.Palmer, did the riscv PR already go to Linus?
Not yet, I usually send on Friday mornings -- and I also generally send two, as there's some stragglers/fixes for the second week. I'm fine taking it (Bjorn just poked me), can someone provide a base commit? Bjorn says it depends on something in Linus' tree, so I'll just pick it up as a straggler for next week.
Also, do you mind sending an Ack?
Quoted 9 lines
If not yet, perhaps you could ship this series along with your PR to Linus during this merge window given the big net PR (incl. bpf) was already merged yesterday. So from our side only fixes ship to Linus.
Otherwise we could take it into bpf-next for the next dev cycle if there are no objections, let us know.
Thanks,
Daniel
Daniel Borkmann
maintainerQuoted 152 lines · Palmer Dabbelt
Quoted 151 lines · On Wed, 30 Aug 2023 01:18:46 PDT (-0700), [email protected] wrote:
Quoted 150 lines · Björn Töpel
Quoted 135 lines · Puranjay Mohan
Changes in v2 -> v3:
1. Fix maximum width of code in patches from 80 to 100. [All patches]
2. Add checks forctx->ro_insns== NULL. [Patch 3]
3. Fix check for edge condition where amount of text to set > 2 * pagesize [Patch 1 and 2]
4. Add reviewed-by in patches.
5. Adding results of selftest here:
Using the command: ./test_progs on qemu
Without the series: Summary: 336/3162 PASSED, 56 SKIPPED, 90 FAILED With this series: Summary: 336/3162 PASSED, 56 SKIPPED, 90 FAILEDChanges in v1 -> v2:
1. Implement a new functionpatch_text_set_nosync()to be used inbpf_arch_text_invalidate(). The implementation in v1 calledpatch_text_nosync()in a loop and it was bad as it would callflush_icache_range()for every word making it really slow. This was found by running thetest_tagselftest which would take forever to complete.Here is some data to prove the V2 fixes the problem:
Without this series:
root@rv-selftester:~/src/kselftest/bpf# time ./test_tag test_tag: OK (40945 tests)real 7m47.562s
user 0m24.145s
sys 6m37.064sWith this series applied:
root@rv-selftester:~/src/selftest/bpf# time ./test_tag test_tag: OK (40945 tests)real 7m29.472s
user 0m25.865s
sys 6m18.401sBPF programs currently consume a page each on RISCV. For systems with many BPF programs, this adds significant pressure to instruction TLB. High iTLB pressure usually causes slow down for the whole system.
Song Liu introduced the BPF prog pack allocator[1] to mitigate the above issue. It packs multiple BPF programs into a single huge page. It is currently only enabled for the
x86_64BPF JIT.I enabled this allocator on the ARM64 BPF JIT[2]. It is being reviewed now.
This patch series enables the BPF prog pack allocator for the RISCV BPF JIT. This series needs a patch[3] from the ARM64 series to work.
======================================================
Performance Analysis of prog pack allocator on RISCV64
======================================================Test setup:
===========Host machine: Debian GNU/Linux 11 (bullseye)
Qemu Version: QEMU emulator version 8.0.3 (Debian 1:8.0.3+dfsg-1) u-boot-qemu Version: 2023.07+dfsg-1
opensbi Version: 1.3-1To test the performance of the BPF prog pack allocator on RV, a stresser tool[4] linked below was built. This tool loads 8 BPF programs on the system and triggers 5 of them in an infinite loop by doing system calls.
The runner script starts 20 instances of the above which loads 8*20=160 BPF programs on the system, 5*20=100 of which are being constantly triggered. The script is passed a command which would be run in the above environment.
The script was run with following perf command:
./run.sh "perf stat -a \
-e iTLB-load-misses \
-e dTLB-load-misses \
-e dTLB-store-misses \
-e instructions \
--timeout 60000"The output of the above command is discussed below before and after enabling the BPF prog pack allocator.
The tests were run on qemu-system-riscv64 with 8 cpus, 16G memory. The rootfs was created using Bjorn's riscv-cross-builder[5] docker container linked below.
Results
=======Before enabling prog pack allocator:
------------------------------------Performance counter stats for 'system wide':
4939048 iTLB-load-misses
5468689 dTLB-load-misses
465234 dTLB-store-misses
1441082097998 instructions60.045791200 seconds time elapsed
After enabling prog pack allocator:
-----------------------------------Performance counter stats for 'system wide':
3430035 iTLB-load-misses
5008745 dTLB-load-misses
409944 dTLB-store-misses
1441535637988 instructions60.046296600 seconds time elapsed
Improvements in metrics
=======================It was expected that the iTLB-load-misses would decrease as now a single huge page is used to keep all the BPF programs compared to a single page for each program earlier.
--------------------------------------------
The improvement in iTLB-load-misses: -30.5 %
--------------------------------------------I repeated this expriment more than 100 times in different setups and the improvement was always greater than 30%.
This patch series is boot tested on the Starfive VisionFive 2 board[6]. The performance analysis was not done on the board because it doesn't expose iTLB-load-misses, etc. The stresser program was run on the board to test the loading and unloading of BPF programs
[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/
[3] https://lore.kernel.org/all/[email protected]/
[4] https://github.com/puranjaymohan/BPF-Allocator-Bench
[5] https://github.com/bjoto/riscv-cross-builder
[6] https://www.starfivetech.com/en/site/boardsQuoted 4 lines
Puranjay Mohan (3):
riscv: extendpatch_text_nosync()for multiple pages
riscv: implement a memset like function for text bpf, riscv: use prog pack allocator in the BPF JITThank you! For the series:
Acked-by Björn Töpel <[email protected]>Tested-by Björn Töpel <[email protected]>@Alexei @Daniel This series depends on a core BPF patch from the Arm series [3].
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
@Palmer LMK if you have any concerns taking the RISC-V text patching stuff via the BPF tree.
Palmer, did the riscv PR already go to Linus?
Not yet, I usually send on Friday mornings -- and I also generally send two, as there's some stragglers/fixes for the second week. I'm fine taking it (Bjorn just poked me), can someone provide a base commit? Bjorn says it depends on something in Linus' tree, so I'll just pick it up as a straggler for next week.
Okay, sgtm.
Quoted 1 line
Also, do you mind sending an Ack?
Björn / Puranjay, just to clarify since the arm64 series did not land, you are referring to this one as a dependency [0], right? Meaning, you'd route [0] + this series via riscv PR to Linus then during this merge win.
If yes, could one of you send the complete 4-patch series with the prior Acks from [0] + this series collected to both bpf+riscv list (with the small request to extend the commit desc in [0] a bit to better document implications of the change itself for other JITs)? After a final look and if BPF CI goes through we can then ack as well and unblock the routing.
Thanks,
Daniel
[0] https://lore.kernel.org/all/[email protected]/
_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
Quoted 174 lines · Daniel Borkmann
Quoted 152 lines · Palmer Dabbelt
Quoted 151 lines · On Wed, 30 Aug 2023 01:18:46 PDT (-0700), [email protected] wrote:
Quoted 150 lines · Björn Töpel
Quoted 135 lines · Puranjay Mohan
Changes in v2 -> v3:
1. Fix maximum width of code in patches from 80 to 100. [All patches]
2. Add checks forctx->ro_insns== NULL. [Patch 3]
3. Fix check for edge condition where amount of text to set > 2 * pagesize[Patch 1 and 2]4. Add reviewed-by in patches.
5. Adding results of selftest here:
Using the command: ./test_progs on qemu
Without the series: Summary: 336/3162 PASSED, 56 SKIPPED, 90 FAILED With this series: Summary: 336/3162 PASSED, 56 SKIPPED, 90 FAILEDChanges in v1 -> v2:
1. Implement a new functionpatch_text_set_nosync()to be used inbpf_arch_text_invalidate(). The implementation in v1 calledpatch_text_nosync()in a loop and it was bad as it would callflush_icache_range()for every word making it really slow. This was found by running thetest_tagselftest which would take forever to complete.Here is some data to prove the V2 fixes the problem:
Without this series:
root@rv-selftester:~/src/kselftest/bpf# time ./test_tag test_tag: OK (40945 tests)real 7m47.562s user 0m24.145s sys 6m37.064sWith this series applied:
root@rv-selftester:~/src/selftest/bpf# time ./test_tag test_tag: OK (40945 tests)real 7m29.472s user 0m25.865s sys 6m18.401sBPF programs currently consume a page each on RISCV. For systems with many BPF programs, this adds significant pressure to instruction TLB. High iTLB pressure usually causes slow down for the whole system.
Song Liu introduced the BPF prog pack allocator[1] to mitigate the above issue. It packs multiple BPF programs into a single huge page. It is currently only enabled for the
x86_64BPF JIT.I enabled this allocator on the ARM64 BPF JIT[2]. It is being reviewed now.
This patch series enables the BPF prog pack allocator for the RISCV BPF JIT. This series needs a patch[3] from the ARM64 series to work.
======================================================
Performance Analysis of prog pack allocator on RISCV64
======================================================Test setup:
===========Host machine: Debian GNU/Linux 11 (bullseye)
Qemu Version: QEMU emulator version 8.0.3 (Debian 1:8.0.3+dfsg-1) u-boot-qemu Version: 2023.07+dfsg-1
opensbi Version: 1.3-1To test the performance of the BPF prog pack allocator on RV, a stresser tool[4] linked below was built. This tool loads 8 BPF programs on the system and triggers 5 of them in an infinite loop by doing system calls.
The runner script starts 20 instances of the above which loads 8*20=160 BPF programs on the system, 5*20=100 of which are being constantly triggered. The script is passed a command which would be run in the above environment.
The script was run with following perf command:
./run.sh "perf stat -a \ -e iTLB-load-misses \ -e dTLB-load-misses \ -e dTLB-store-misses \ -e instructions \ --timeout 60000"The output of the above command is discussed below before and after enabling the BPF prog pack allocator.
The tests were run on qemu-system-riscv64 with 8 cpus, 16G memory. The rootfs was created using Bjorn's riscv-cross-builder[5] docker container linked below.
Results
=======Before enabling prog pack allocator:
------------------------------------Performance counter stats for 'system wide':
4939048 iTLB-load-misses 5468689 dTLB-load-misses 465234 dTLB-store-misses 1441082097998 instructions60.045791200 seconds time elapsedAfter enabling prog pack allocator:
-----------------------------------Performance counter stats for 'system wide':
3430035 iTLB-load-misses 5008745 dTLB-load-misses 409944 dTLB-store-misses 1441535637988 instructions60.046296600 seconds time elapsedImprovements in metrics
=======================It was expected that the iTLB-load-misses would decrease as now a single huge page is used to keep all the BPF programs compared to a single page for each program earlier.
--------------------------------------------
The improvement in iTLB-load-misses: -30.5 %
--------------------------------------------I repeated this expriment more than 100 times in different setups and the improvement was always greater than 30%.
This patch series is boot tested on the Starfive VisionFive 2 board[6]. The performance analysis was not done on the board because it doesn't expose iTLB-load-misses, etc. The stresser program was run on the board to test the loading and unloading of BPF programs
[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/
[3] https://lore.kernel.org/all/[email protected]/
[4] https://github.com/puranjaymohan/BPF-Allocator-Bench
[5] https://github.com/bjoto/riscv-cross-builder
[6] https://www.starfivetech.com/en/site/boardsQuoted 4 lines
Puranjay Mohan (3):
riscv: extendpatch_text_nosync()for multiple pages
riscv: implement a memset like function for text bpf, riscv: use prog pack allocator in the BPF JITThank you! For the series:
Acked-by Björn Töpel <[email protected]>Tested-by Björn Töpel <[email protected]>@Alexei @Daniel This series depends on a core BPF patch from the Arm
series [3].Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
@Palmer LMK if you have any concerns taking the RISC-V text patching
stuff via the BPF tree.Palmer, did the riscv PR already go to Linus?
Not yet, I usually send on Friday mornings -- and I also generally send two, as there's some stragglers/fixes for the second week. I'm fine taking it (Bjorn just poked me), can someone provide a base commit? Bjorn says it depends on something in Linus' tree, so I'll just pick it up as a straggler for next week.
Okay, sgtm.
Quoted 1 line
Also, do you mind sending an Ack?
Björn / Puranjay, just to clarify since the arm64 series did not land, you are referring to this one as a dependency [0], right? Meaning, you'd route [0] + this series via riscv PR to Linus then during this merge win.
If yes, could one of you send the complete 4-patch series with the prior Acks from [0] + this series collected to both bpf+riscv list (with the small request to extend the commit desc in [0] a bit to better document implications of the change itself for other JITs)? After a final look and if BPF CI goes through we can then ack as well and unblock the routing.
Quoted 4 lines
Thanks,
Daniel[0] https://lore.kernel.org/all/[email protected]/
Hi Daniel,
I have sent the v4[0] of this with the core patch included.
[0]https://lore.kernel.org/all/[email protected]/
Thanks.
Puranjay
Björn Töpel
maintainerQuoted 150 lines · Daniel Borkmann
Quoted 149 lines · Björn Töpel
Quoted 135 lines · Puranjay Mohan
Changes in v2 -> v3:
1. Fix maximum width of code in patches from 80 to 100. [All patches]
2. Add checks forctx->ro_insns== NULL. [Patch 3]
3. Fix check for edge condition where amount of text to set > 2 * pagesize[Patch 1 and 2]4. Add reviewed-by in patches.
5. Adding results of selftest here:
Using the command: ./test_progs on qemu
Without the series: Summary: 336/3162 PASSED, 56 SKIPPED, 90 FAILED With this series: Summary: 336/3162 PASSED, 56 SKIPPED, 90 FAILEDChanges in v1 -> v2:
1. Implement a new functionpatch_text_set_nosync()to be used inbpf_arch_text_invalidate(). The implementation in v1 calledpatch_text_nosync()in a loop and it was bad as it would callflush_icache_range()for every word making it really slow. This was found by running thetest_tagselftest which would take forever to complete.Here is some data to prove the V2 fixes the problem:
Without this series:
root@rv-selftester:~/src/kselftest/bpf# time ./test_tag test_tag: OK (40945 tests)real 7m47.562s user 0m24.145s sys 6m37.064sWith this series applied:
root@rv-selftester:~/src/selftest/bpf# time ./test_tag test_tag: OK (40945 tests)real 7m29.472s user 0m25.865s sys 6m18.401sBPF programs currently consume a page each on RISCV. For systems with many BPF programs, this adds significant pressure to instruction TLB. High iTLB pressure usually causes slow down for the whole system.
Song Liu introduced the BPF prog pack allocator[1] to mitigate the above issue. It packs multiple BPF programs into a single huge page. It is currently only enabled for the
x86_64BPF JIT.I enabled this allocator on the ARM64 BPF JIT[2]. It is being reviewed now.
This patch series enables the BPF prog pack allocator for the RISCV BPF JIT. This series needs a patch[3] from the ARM64 series to work.
======================================================
Performance Analysis of prog pack allocator on RISCV64
======================================================Test setup:
===========Host machine: Debian GNU/Linux 11 (bullseye)
Qemu Version: QEMU emulator version 8.0.3 (Debian 1:8.0.3+dfsg-1) u-boot-qemu Version: 2023.07+dfsg-1
opensbi Version: 1.3-1To test the performance of the BPF prog pack allocator on RV, a stresser tool[4] linked below was built. This tool loads 8 BPF programs on the system and triggers 5 of them in an infinite loop by doing system calls.
The runner script starts 20 instances of the above which loads 8*20=160 BPF programs on the system, 5*20=100 of which are being constantly triggered. The script is passed a command which would be run in the above environment.
The script was run with following perf command:
./run.sh "perf stat -a \ -e iTLB-load-misses \ -e dTLB-load-misses \ -e dTLB-store-misses \ -e instructions \ --timeout 60000"The output of the above command is discussed below before and after enabling the BPF prog pack allocator.
The tests were run on qemu-system-riscv64 with 8 cpus, 16G memory. The rootfs was created using Bjorn's riscv-cross-builder[5] docker container linked below.
Results
=======Before enabling prog pack allocator:
------------------------------------Performance counter stats for 'system wide':
4939048 iTLB-load-misses 5468689 dTLB-load-misses 465234 dTLB-store-misses 1441082097998 instructions60.045791200 seconds time elapsedAfter enabling prog pack allocator:
-----------------------------------Performance counter stats for 'system wide':
3430035 iTLB-load-misses 5008745 dTLB-load-misses 409944 dTLB-store-misses 1441535637988 instructions60.046296600 seconds time elapsedImprovements in metrics
=======================It was expected that the iTLB-load-misses would decrease as now a single huge page is used to keep all the BPF programs compared to a single page for each program earlier.
--------------------------------------------
The improvement in iTLB-load-misses: -30.5 %
--------------------------------------------I repeated this expriment more than 100 times in different setups and the improvement was always greater than 30%.
This patch series is boot tested on the Starfive VisionFive 2 board[6]. The performance analysis was not done on the board because it doesn't expose iTLB-load-misses, etc. The stresser program was run on the board to test the loading and unloading of BPF programs
[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/
[3] https://lore.kernel.org/all/[email protected]/
[4] https://github.com/puranjaymohan/BPF-Allocator-Bench
[5] https://github.com/bjoto/riscv-cross-builder
[6] https://www.starfivetech.com/en/site/boardsQuoted 4 lines
Puranjay Mohan (3):
riscv: extendpatch_text_nosync()for multiple pages
riscv: implement a memset like function for text bpf, riscv: use prog pack allocator in the BPF JITThank you! For the series:
Acked-by Björn Töpel <[email protected]>Tested-by Björn Töpel <[email protected]>@Alexei @Daniel This series depends on a core BPF patch from the Arm
series [3].
[snip]
Quoted 3 lines
If not yet, perhaps you could ship this series along with your PR to Linus during this merge window given the big net PR (incl. bpf) was already merged yesterday. So from our side only fixes ship to Linus.
Are you OK with this patch going thru the riscv tree as well?
Palmer Dabbelt
maintainerQuoted 153 lines · On Wed, 30 Aug 2023 06:59:13 PDT (-0700), [email protected] wrote:
Quoted 143 lines · Daniel Borkmann
Quoted 142 lines · Björn Töpel
Quoted 139 lines · Puranjay Mohan
Changes in v2 -> v3:
1. Fix maximum width of code in patches from 80 to 100. [All patches]
2. Add checks forctx->ro_insns== NULL. [Patch 3]
3. Fix check for edge condition where amount of text to set > 2 * pagesize[Patch 1 and 2]4. Add reviewed-by in patches.
5. Adding results of selftest here:
Using the command: ./test_progs on qemu
Without the series: Summary: 336/3162 PASSED, 56 SKIPPED, 90 FAILED With this series: Summary: 336/3162 PASSED, 56 SKIPPED, 90 FAILEDChanges in v1 -> v2:
1. Implement a new functionpatch_text_set_nosync()to be used inbpf_arch_text_invalidate(). The implementation in v1 calledpatch_text_nosync()in a loop and it was bad as it would callflush_icache_range()for every word making it really slow. This was found by running thetest_tagselftest which would take forever to complete.Here is some data to prove the V2 fixes the problem:
Without this series:
root@rv-selftester:~/src/kselftest/bpf# time ./test_tag test_tag: OK (40945 tests)real 7m47.562s user 0m24.145s sys 6m37.064sWith this series applied:
root@rv-selftester:~/src/selftest/bpf# time ./test_tag test_tag: OK (40945 tests)real 7m29.472s user 0m25.865s sys 6m18.401sBPF programs currently consume a page each on RISCV. For systems with many BPF programs, this adds significant pressure to instruction TLB. High iTLB pressure usually causes slow down for the whole system.
Song Liu introduced the BPF prog pack allocator[1] to mitigate the above issue. It packs multiple BPF programs into a single huge page. It is currently only enabled for the
x86_64BPF JIT.I enabled this allocator on the ARM64 BPF JIT[2]. It is being reviewed now.
This patch series enables the BPF prog pack allocator for the RISCV BPF JIT. This series needs a patch[3] from the ARM64 series to work.
======================================================
Performance Analysis of prog pack allocator on RISCV64
======================================================Test setup:
===========Host machine: Debian GNU/Linux 11 (bullseye)
Qemu Version: QEMU emulator version 8.0.3 (Debian 1:8.0.3+dfsg-1) u-boot-qemu Version: 2023.07+dfsg-1
opensbi Version: 1.3-1To test the performance of the BPF prog pack allocator on RV, a stresser tool[4] linked below was built. This tool loads 8 BPF programs on the system and triggers 5 of them in an infinite loop by doing system calls.
The runner script starts 20 instances of the above which loads 8*20=160 BPF programs on the system, 5*20=100 of which are being constantly triggered. The script is passed a command which would be run in the above environment.
The script was run with following perf command:
./run.sh "perf stat -a \ -e iTLB-load-misses \ -e dTLB-load-misses \ -e dTLB-store-misses \ -e instructions \ --timeout 60000"The output of the above command is discussed below before and after enabling the BPF prog pack allocator.
The tests were run on qemu-system-riscv64 with 8 cpus, 16G memory. The rootfs was created using Bjorn's riscv-cross-builder[5] docker container linked below.
Results
=======Before enabling prog pack allocator:
------------------------------------Performance counter stats for 'system wide':
4939048 iTLB-load-misses 5468689 dTLB-load-misses 465234 dTLB-store-misses 1441082097998 instructions60.045791200 seconds time elapsedAfter enabling prog pack allocator:
-----------------------------------Performance counter stats for 'system wide':
3430035 iTLB-load-misses 5008745 dTLB-load-misses 409944 dTLB-store-misses 1441535637988 instructions60.046296600 seconds time elapsedImprovements in metrics
=======================It was expected that the iTLB-load-misses would decrease as now a single huge page is used to keep all the BPF programs compared to a single page for each program earlier.
--------------------------------------------
The improvement in iTLB-load-misses: -30.5 %
--------------------------------------------I repeated this expriment more than 100 times in different setups and the improvement was always greater than 30%.
This patch series is boot tested on the Starfive VisionFive 2 board[6]. The performance analysis was not done on the board because it doesn't expose iTLB-load-misses, etc. The stresser program was run on the board to test the loading and unloading of BPF programs
[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/
[3] https://lore.kernel.org/all/[email protected]/
[4] https://github.com/puranjaymohan/BPF-Allocator-Bench
[5] https://github.com/bjoto/riscv-cross-builder
[6] https://www.starfivetech.com/en/site/boardsPuranjay Mohan (3):
riscv: extendpatch_text_nosync()for multiple pages
riscv: implement a memset like function for text bpf, riscv: use prog pack allocator in the BPF JITQuoted 7 lines
Quoted 7 lines
Thank you! For the series:
Acked-by Björn Töpel <[email protected]>Tested-by Björn Töpel <[email protected]>@Alexei @Daniel This series depends on a core BPF patch from the Arm
series [3].
Quoted 6 lines
[snip]
Quoted 3 lines
If not yet, perhaps you could ship this series along with your PR to Linus during this merge window given the big net PR (incl. bpf) was already merged yesterday. So from our side only fixes ship to Linus.
Are you OK with this patch going thru the riscv tree as well?
I'm generally fine taking almost anything, as long as whomever usually takes them acks it.
_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv