x86/jump_label: Bound IPIs sent when updating a static key
Subsystems (2)
v2 cover x86/jump_label: Bound IPIs sent when updating a static key
While tuning a system with CPUs isolated as much as possible, we've noticed that isolated CPUs were receiving bursts of 12 IPIs, periodically. Tracing the functions that emit IPIs, we saw chronyd - an unprivileged process - generating the IPIs when changing a static key, enabling network timestaping on sockets.
For instance, the trace pointed:
# trace-cmd record --func-stack -p function -l smp_call_function_single -e irq_vectors -f 'vector == 251'...
# trace-cmde report...
[...]
chronyd-858 [000] 433.286406: function: smp_call_function_single
chronyd-858 [000] 433.286408: kernel_stack: <stack trace>
=> smp_call_function_many (ffffffffbc10ab22)
=> smp_call_function (ffffffffbc10abaa)
=> on_each_cpu (ffffffffbc10ac0b)
=> text_poke_bp (ffffffffbc02436a)
=> arch_jump_label_transform (ffffffffbc020ec8)
=> __jump_label_update (ffffffffbc1b300f)
=> jump_label_update (ffffffffbc1b30ed)
=> static_key_slow_inc (ffffffffbc1b334d)
=> net_enable_timestamp (ffffffffbc625c44)
=> sock_enable_timestamp (ffffffffbc6127d5)
=> sock_setsockopt (ffffffffbc612d2f)
=> SyS_setsockopt (ffffffffbc60d5e6)
=> tracesys (ffffffffbc7681c5)
<idle>-0 [001] 433.286416: call_function_single_entry: vector=251
<idle>-0 [001] 433.286419: call_function_single_exit: vector=251[... The IPI takes place 12 times]
The static key in case was the netstamp_needed_key. A static key change from enabled->disabled/disabled->enabled causes the code to be changed, and this is done in three steps:
-- Pseudo-code #1 - Current implementation ---
For each key to be updated:
1) add an int3 trap to the address that will be patched sync cores (send IPI to all other CPUs)
2) update all but the first byte of the patched range sync cores (send IPI to all other CPUs)
3) replace the first byte (int3) by the first byte of replacing opcode sync cores (send IPI to all other CPUs)
-- Pseudo-code #1 ---
As the static key netstamp_needed_key has four entries (used in for places in the code) in our kernel, 3 IPIs were generated for each entry, resulting in 12 IPIs. The number of IPIs then is linear with regard to the number 'n' of entries of a key: O(n*3), which is O(n).
This algorithm works fine for the update of a single key. But we think it is possible to optimize the case in which a static key has more than one entry. For instance, the sched_schedstats jump label has 56 entries in my (updated) fedora kernel, resulting in 168 IPIs for each CPU in which the thread that is enabling is _not_ running.
In this patch, rather than doing each updated at once, it is possible to queue all updates first, and the, apply all updates at once, rewriting the pseudo-code #1 in this way:
-- Pseudo-code #2 - This patch ---
1) for each key in the queue:
add an int3 trap to the address that will be patched
sync cores (send IPI to all other CPUs)
2) for each key in the queue:
update all but the first byte of the patched range
sync cores (send IPI to all other CPUs)
3) for each key in the queue:
replace the first byte (int3) by the first byte of replacing opcode
sync cores (send IPI to all other CPUs)
-- Pseudo-code #2 - This patch ---
Doing the update in this way, the number of IPI becomes O(3) with regard to the number of keys, which is O(1).
Currently, the jump label of a static key is transformed via the arch specific function:
void arch_jump_label_transform(struct jump_entry *entry,
enum jump_label_type type)
The new approach (batch mode) uses two arch functions, the first has the same arguments of the arch_jump_label_transform(), and is the function:
void arch_jump_label_transform_queue(struct jump_entry *entry,
enum jump_label_type type)
Rather than transforming the code, it adds the jump_entry in a queue of entries to be updated.
After queuing all jump_entries, the function:
void arch_jump_label_transform_apply(void)
Applies the changes in the queue.
One easy way to see the benefits of this patch is switching the schedstats on and off. For instance:
-------------------------- %< ---------------------------- #!/bin/bash
while [ true ]; do
sysctl -w kernel.sched_schedstats=1
sleep 2
sysctl -w kernel.sched_schedstats=0
sleep 2
done
-------------------------- >% ----------------------------
while watching the IPI count:
-------------------------- %< ----------------------------
# watch -n1 "cat /proc/interrupts | grep Function" -------------------------- >% ----------------------------
With the current mode, it is possible to see +- 168 IPIs each 2 seconds, while with this patch the number of IPIs goes to 3 each 2 seconds.
Regarding the performance impact of this patch set, I made two measurements:
The time to update a key (the task that is causing the change) The time to run the int3 handler (the side effect on a thread that
hits the code being changed)
The schedstats static key was chosen as the key to being switched on and off. The reason being is that it is used in more than 56 places, in a hot path. The change in the schedstats static key will be done with the following command:
while [ true ]; do
sysctl -w kernel.sched_schedstats=1
usleep 500000
sysctl -w kernel.sched_schedstats=0
usleep 500000
done
In this way, they key will be updated twice per second. To force the hit of the int3 handler, the system will also run a kernel compilation with two jobs per CPU. The test machine is a two nodes/24 CPUs box with an Intel Xeon processor @2.27GHz.
Regarding the update part, on average, the regular kernel takes 57 ms to update the schedstats key, while the kernel with the batch updates takes just 1.4 ms on average. Although it seems to be too good to be true, it makes sense: the schedstats key is used in 56 places, so it was expected that it would take around 56 times to update the keys with the current implementation, as the IPIs are the most expensive part of the update.
Regarding the int3 handler, the non-batch handler takes 45 ns on average, while the batch version takes around 180 ns. At first glance, it seems to be a high value. But it is not, considering that it is doing 56 updates, rather than one! It is taking four times more, only. This gain is possible because the patch uses a binary search in the vector: log2(56)=5.8. So, it was expected to have an overhead within four times.
(voice of tv propaganda) But, that is not all! As the int3 handler keeps on for a shorter period (because the update part is on for a shorter time), the number of hits in the int3 handler decreased by 10%.
The question then is: Is it worth paying the price of "135 ns" more in the int3 handler?
Considering that, in this test case, we are saving the handling of 53 IPIs, that takes more than these 135 ns, it seems to be a meager price to be paid. Moreover, the test case was forcing the hit of the int3, in practice, it does not take that often. While the IPI takes place on all CPUs, hitting the int3 handler or not!
For instance, in an isolated CPU with a process running in user-space (nohz_full use-case), the chances of hitting the int3 handler is barely zero, while there is no way to avoid the IPIs. By bounding the IPIs, we are improving a lot this scenario.
Changes from v1:
- Split the patch in jump-label/x86-jump-label/alternative (Jason Baron)
- Use bserach in the int3 handler (Steven Rostedt)
- Use a pre-allocated page to store the vector (Jason/Steven)
- Do performance tests in the int3 handler (peterz)
Daniel Bristot de Oliveira (9):
jump_label: Add for_each_label_entry helper
jump_label: Add the jump_label_can_update_check() helper x86/jump_label: Move checking code away from __jump_label_transform() x86/jump_label: Add __jump_label_set_jump_code() helper x86/alternative: Split text_poke_bp() into tree steps
jump_label: Sort entries of the same key by the code x86/alternative: Batch of patch operations
x86/jump_label: Batch jump label updates
jump_label: Batch updates if arch supports it
arch/x86/include/asm/jump_label.h | 2 + arch/x86/include/asm/text-patching.h | 15 +++ arch/x86/kernel/alternative.c | 140 ++++++++++++++++++--- arch/x86/kernel/jump_label.c | 174 ++++++++++++++++++++++----- include/linux/jump_label.h | 6 + kernel/jump_label.c | 73 +++++++++-- 6 files changed, 359 insertions(+), 51 deletions(-)
Signature · 1 line
2.17.1
PATCH v2 1/9 jump_label: Add for_each_label_entry helper
This patch adds the helper:
for_each_label_entry(key, entry, stop)
For the "for each jump label entry" for defined as:
for (; (entry < stop) && (jump_entry_key(entry) == key); entry++)
Simplifying the reading and usage.
--- include/linux/jump_label.h | 3 +++ kernel/jump_label.c | 2 +- 2 files changed, 4 insertions(+), 1 deletion(-)
include/linux/jump_label.h +3 -0
@@ -231,6 +231,9 @@ extern void static_key_disable(struct static_key *key);extern void static_key_enable_cpuslocked (struct static_key *key);extern void static_key_disable_cpuslocked (struct static_key *key); +#define for_each_label_entry (key, entry, stop) \ + for (; (entry < stop) && (jump_entry_key(entry) == key); entry++) +/* * We should be using ATOMIC_INIT() for initializing .enabled, but * the inclusion of atomic.h is problematic for inclusion of jump_label.h
kernel/jump_label.c +1 -1
@@ -381,7 +381,7 @@ static void __jump_label_update(struct static_key *key, struct jump_entry *stop, bool init) { -for (; (entry < stop) && (jump_entry_key (entry) == key); entry++) { + for_each_label_entry(key, entry, stop) {/* * An entry->code of 0 indicates an entry which has been * disabled because it was in an init text area.
Signature · 1 line
2.17.1
PATCH v2 2/9 jump_label: Add the jump_label_can_update_check() helper
In reply to Daniel Bristot de OliveiraMove the check of if a jump_entry is valid to a function.
--- kernel/jump_label.c | 30 ++++++++++++++++++++---------- 1 file changed, 20 insertions(+), 10 deletions(-)
kernel/jump_label.c +20 -10
@@ -376,22 +376,32 @@ static enum jump_label_type jump_label_type(struct jump_entry *entry)return enabled ^ branch; } +bool jump_label_can_update_check (struct jump_entry *entry,bool init) +{ +/* +* An entry->code of 0 indicates an entry which has been +* disabled because it was in an init text area. +*/ +if (init || !jump_entry_is_init (entry)) { +if (!kernel_text_address (jump_entry_code (entry))) { +WARN_ONCE (1 ,"can't patch jump_label at %pS" , + (void *)jump_entry_code (entry)); +return 0 ; + } +return 1 ; + } +return 0 ; +} +static void __jump_label_update (struct static_key *key,struct jump_entry *entry,struct jump_entry *stop,bool init) {for_each_label_entry (key, entry, stop) { -/* -* An entry->code of 0 indicates an entry which has been -* disabled because it was in an init text area. -*/ -if (init || !jump_entry_is_init (entry)) { -if (kernel_text_address (jump_entry_code (entry))) -arch_jump_label_transform (entry,jump_label_type (entry)); -else -WARN_ONCE (1 ,"can't patch jump_label at %pS" , - (void *)jump_entry_code (entry)); +if (jump_label_can_update_check (entry, init)) { +arch_jump_label_transform (entry, +jump_label_type (entry)); } } }
Signature · 1 line
2.17.1
PATCH v2 3/9 x86/jump_label: Move checking code away from __jump_label_transform()
In reply to Daniel Bristot de OliveiraThis patch creates a new functions to check the current code before updating a jump_entry.
--- arch/x86/kernel/jump_label.c | 60 +++++++++++++++++++++++++----------- 1 file changed, 42 insertions(+), 18 deletions(-)
arch/x86/kernel/jump_label.c +42 -18
@@ -37,16 +37,53 @@ static void bug_at(unsigned char *ip, int line)BUG (); } +static inline void __jump_label_trans_check_enable (struct jump_entry *entry, +enum jump_label_type type, +const unsigned char *ideal_nop, +int init) +{ +const unsigned char default_nop[] = {STATIC_KEY_INIT_NOP }; +const void *expect; +int line; + +if (init) { + expect = default_nop; line = __LINE__; + }else { + expect = ideal_nop; line = __LINE__; + } + +if (memcmp ((void *)jump_entry_code (entry), expect,JUMP_LABEL_NOP_SIZE )) +bug_at ((void *)jump_entry_code (entry), line); +} + +static inline void __jump_label_trans_check_disable (struct jump_entry *entry, +enum jump_label_type type, +union jump_code_union *jmp, +int init) +{ +const unsigned char default_nop[] = {STATIC_KEY_INIT_NOP }; +const void *expect; +int line; + +if (init) { + expect = default_nop; line = __LINE__; + }else { + expect = jmp->code; line = __LINE__; + } + +if (memcmp ((void *)jump_entry_code (entry), expect,JUMP_LABEL_NOP_SIZE )) +bug_at ((void *)jump_entry_code (entry), line); +} + +static void __ref __jump_label_transform (struct jump_entry *entry,enum jump_label_type type,void *(*poker)(void *,const void *,size_t ),int init) {union jump_code_union jmp; -const unsigned char default_nop[] = {STATIC_KEY_INIT_NOP };const unsigned char *ideal_nop = ideal_nops[NOP_ATOMIC5 ]; -const void *expect, *code; -int line; +const void *code; jmp.jump =0xe9 ; jmp.offset = jump_entry_target(entry) -
@@ -56,26 +93,13 @@ static void __ref __jump_label_transform(struct jump_entry *entry, poker = text_poke_early;if (type ==JUMP_LABEL_JMP ) { -if (init) { - expect = default_nop; line = __LINE__; - }else { - expect = ideal_nop; line = __LINE__; - } - +__jump_label_trans_check_enable (entry, type, ideal_nop, init); code = &jmp.code; }else { -if (init) { - expect = default_nop; line = __LINE__; - }else { - expect = &jmp.code; line = __LINE__; - } - +__jump_label_trans_check_disable (entry, type, &jmp, init); code = ideal_nop; } -if (memcmp ((void *)jump_entry_code (entry), expect,JUMP_LABEL_NOP_SIZE )) -bug_at ((void *)jump_entry_code (entry), line); -/* * Make text_poke_bp() a default fallback poker. *
Signature · 1 line
2.17.1
PATCH v2 4/9 x86/jump_label: Add __jump_label_set_jump_code() helper
In reply to Daniel Bristot de OliveiraMove the definition of the code to be written from __jump_label_transform() to a specialized function.
--- arch/x86/kernel/jump_label.c | 38 +++++++++++++++++++++--------------- 1 file changed, 22 insertions(+), 16 deletions(-)
arch/x86/kernel/jump_label.c +22 -16
@@ -75,30 +75,36 @@ static inline void __jump_label_trans_check_disable(struct jump_entry *entry,bug_at ((void *)jump_entry_code (entry), line); } +static void __jump_label_set_jump_code (struct jump_entry *entry, +enum jump_label_type type, +union jump_code_union *code, +int init) +{ +const unsigned char *ideal_nop = ideal_nops[NOP_ATOMIC5 ]; + + code->jump =0xe9 ; + code->offset =jump_entry_target (entry) - + (jump_entry_code (entry) +JUMP_LABEL_NOP_SIZE ); + +if (type ==JUMP_LABEL_JMP ) { +__jump_label_trans_check_enable (entry, type, ideal_nop, init); + }else { +__jump_label_trans_check_disable (entry, type, code, init); +memcpy (code, ideal_nop,JUMP_LABEL_NOP_SIZE ); + } +}static void __ref __jump_label_transform (struct jump_entry *entry,enum jump_label_type type,void *(*poker)(void *,const void *,size_t ),int init) { -union jump_code_union jmp; -const unsigned char *ideal_nop = ideal_nops[NOP_ATOMIC5 ]; -const void *code; - - jmp.jump =0xe9 ; - jmp.offset =jump_entry_target (entry) - - (jump_entry_code (entry) +JUMP_LABEL_NOP_SIZE ); +union jump_code_union code;if (early_boot_irqs_disabled) poker = text_poke_early; -if (type ==JUMP_LABEL_JMP ) { -__jump_label_trans_check_enable (entry, type, ideal_nop, init); - code = &jmp.code; - }else { -__jump_label_trans_check_disable (entry, type, &jmp, init); - code = ideal_nop; - } +__jump_label_set_jump_code (entry, type, &code, init);/* * Make text_poke_bp() a default fallback poker.
@@ -109,12 +115,12 @@ static void __ref __jump_label_transform(struct jump_entry *entry,* */ if (poker) { - (*poker)((void *)jump_entry_code (entry), code, + (*poker)((void *)jump_entry_code (entry), &code,JUMP_LABEL_NOP_SIZE );return ; } -text_poke_bp ((void *)jump_entry_code (entry), code,JUMP_LABEL_NOP_SIZE , +text_poke_bp ((void *)jump_entry_code (entry), &code,JUMP_LABEL_NOP_SIZE , (void *)jump_entry_code (entry) +JUMP_LABEL_NOP_SIZE ); }
Signature · 1 line
2.17.1
PATCH v2 5/9 x86/alternative: Split text_poke_bp() into tree steps
In reply to Daniel Bristot de Oliveiratext_poke_bp() updates instructions on live kernel on SMP in three steps:
1) add a int3 trap to the address that will be patched
2) update all but the first byte of the patched range
3) replace the first byte (int3) by the first byte of
This patch creates one function for each of these steps.
--- arch/x86/kernel/alternative.c | 38 +++++++++++++++++++++++++---------- 1 file changed, 27 insertions(+), 11 deletions(-)
arch/x86/kernel/alternative.c +27 -11
@@ -767,6 +767,29 @@ int poke_int3_handler(struct pt_regs *regs) } +static void text_poke_bp_set_handler (void *addr,void *handler, +unsigned char int3) +{ + bp_int3_handler = handler; + bp_int3_addr = (u8 *)addr +sizeof (int3); +text_poke (addr, &int3,sizeof (int3)); +} + +static void patch_all_but_first_byte (void *addr,const void *opcode, +size_t len,unsigned char int3) +{ +/* patch all but the first byte */ +text_poke ((char *)addr +sizeof (int3), + (const char *) opcode +sizeof (int3), + len -sizeof (int3)); +} + +static void patch_first_byte (void *addr,const void *opcode,unsigned char int3) +{ +/* patch the first byte */ +text_poke (addr, opcode,sizeof (int3)); +} +/** * text_poke_bp() -- update instructions on live kernel on SMP * @addr: address to patch
@@ -791,27 +814,21 @@ void *text_poke_bp(void *addr, const void *opcode, size_t len, void *handler) {unsigned char int3 =0xcc ; - bp_int3_handler = handler; - bp_int3_addr = (u8 *)addr +sizeof (int3); - bp_patching_in_progress = true; -lockdep_assert_held (&text_mutex); + bp_patching_in_progress = true;/* * Corresponding read barrier in int3 notifier for making sure the * in_progress and handler are correctly ordered wrt. patching. */ smp_wmb (); -text_poke (addr, &int3,sizeof (int3)); +text_poke_bp_set_handler (addr, handler, int3);on_each_cpu (do_sync_core,NULL ,1 );if (len -sizeof (int3) >0 ) { -/* patch all but the first byte */ -text_poke ((char *)addr +sizeof (int3), - (const char *) opcode +sizeof (int3), - len -sizeof (int3)); +patch_all_but_first_byte (addr, opcode, len, int3);/* * According to Intel, this core syncing is very likely * not necessary and we'd be safe even without it. But
@@ -820,8 +837,7 @@ void *text_poke_bp(void *addr, const void *opcode, size_t len, void *handler)on_each_cpu (do_sync_core,NULL ,1 ); } -/* patch the first byte */ -text_poke (addr, opcode,sizeof (int3)); +patch_first_byte (addr, opcode, int3);on_each_cpu (do_sync_core,NULL ,1 );/*
Signature · 1 line
2.17.1
PATCH v2 6/9 jump_label: Sort entries of the same key by the code
In reply to Daniel Bristot de OliveiraIn the batching mode, entries with the same key should also be sorted by the code, enabling a bsearch() of a code/addr when updating a key.
--- kernel/jump_label.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)
kernel/jump_label.c +16 -0
@@ -38,12 +38,28 @@ static int jump_label_cmp(const void *a, const void *b)const struct jump_entry *jea = a;const struct jump_entry *jeb = b; +/* +* Entrires are sorted by key. +*/ if (jump_entry_key (jea) <jump_entry_key (jeb))return -1 ;if (jump_entry_key (jea) >jump_entry_key (jeb))return 1 ; +#ifdef HAVE_JUMP_LABEL_BATCH +/* +* In the batching mode, entries should also be sorted by the code +* inside the already sorted list of entries, enabling a bsearch in +* the vector. +*/ +if (jump_entry_code (jea) <jump_entry_code (jeb)) +return -1 ; + +if (jump_entry_code (jea) >jump_entry_code (jeb)) +return 1 ; +#endif +return 0 ; }
Signature · 1 line
2.17.1
PATCH v2 7/9 x86/alternative: Batch of patch operations
In reply to Daniel Bristot de OliveiraCurrently, the patch of an address is done in three steps:
-- Pseudo-code #1 - Current implementation ---
1) add an int3 trap to the address that will be patched sync cores (send IPI to all other CPUs)
2) update all but the first byte of the patched range sync cores (send IPI to all other CPUs)
3) replace the first byte (int3) by the first byte of replacing opcode sync cores (send IPI to all other CPUs)
-- Pseudo-code #1 ---
When a static key has more than one entry, these steps are called once for each entry. The number of IPIs then is linear with regard to the number 'n' of entries of a key: O(n*3), which is O(n).
This algorithm works fine for the update of a single key. But we think it is possible to optimize the case in which a static key has more than one entry. For instance, the sched_schedstats jump label has 56 entries in my (updated) fedora kernel, resulting in 168 IPIs for each CPU in which the thread that is enabling the key is _not_ running.
With this patch, rather than receiving a single patch to be processed, a vector of patches is passed, enabling the rewrite of the pseudo-code #1 in this way:
-- Pseudo-code #2 - This patch ---
1) for each patch in the vector:
add an int3 trap to the address that will be patched
sync cores (send IPI to all other CPUs)
2) for each patch in the vector:
update all but the first byte of the patched range
sync cores (send IPI to all other CPUs)
3) for each patch in the vector:
replace the first byte (int3) by the first byte of replacing opcode
sync cores (send IPI to all other CPUs)
-- Pseudo-code #2 - This patch ---
Doing the update in this way, the number of IPI becomes O(3) with regard to the number of keys, which is O(1).
The batch mode is done with the function text_poke_bp_batch(), that receives two arguments: a vector of "struct text_to_poke", and the number of entries in the vector.
The vector must be sorted by the addr field of the text_to_poke structure, enabling the binary search of a handler in the poke_int3_handler function (a fast path).
--- arch/x86/include/asm/text-patching.h | 15 ++++ arch/x86/kernel/alternative.c | 108 +++++++++++++++++++++++++-- 2 files changed, 117 insertions(+), 6 deletions(-)
arch/x86/include/asm/text-patching.h +15 -0
@@ -18,6 +18,20 @@ static inline void apply_paravirt(struct paravirt_patch_site *start,#define __parainstructions_end NULL#endif +/* +* Currently, the max observed size in the kernel code is +* JUMP_LABEL_NOP_SIZE/RELATIVEJUMP_SIZE, which are 5. +* Raise it if needed. +*/ +#define POKE_MAX_OPCODE_SIZE 5 + +struct text_to_poke { +void *handler; +void *addr; +size_t len; +const char opcode[POKE_MAX_OPCODE_SIZE ]; +}; +extern void *text_poke_early (void *addr,const void *opcode,size_t len);/*
@@ -37,6 +51,7 @@ extern void *text_poke_early(void *addr, const void *opcode, size_t len);extern void *text_poke (void *addr,const void *opcode,size_t len);extern int poke_int3_handler (struct pt_regs *regs);extern void *text_poke_bp (void *addr,const void *opcode,size_t len,void *handler); +extern void text_poke_bp_batch (struct text_to_poke *tp,unsigned int nr_entries);extern int after_bootmem;#endif /* _ASM_X86_TEXT_PATCHING_H */
arch/x86/kernel/alternative.c +102 -6
151 lines not on this page · raw message
Signature · 1 line
2.17.1
PATCH v2 8/9 x86/jump_label: Batch jump label updates
In reply to Daniel Bristot de OliveiraCurrently, the jump label of a static key is transformed via the arch specific function:
void arch_jump_label_transform(struct jump_entry *entry,
enum jump_label_type type)
The new approach (batch mode) uses two arch functions, the first has the same arguments of the arch_jump_label_transform(), and is the function:
void arch_jump_label_transform_queue(struct jump_entry *entry,
enum jump_label_type type)
Rather than transforming the code, it adds the jump_entry in a queue of entries to be updated. This functions returns 1 in the case of a successful enqueue of an entry. If it returns 0, the caller must to apply the queue and then try to queue again, for instance, because the queue is full.
This function expects the caller to sort the entries by the address before enqueueuing then. This is already done by the arch independent code, though.
After queuing all jump_entries, the function:
void arch_jump_label_transform_apply(void)
Applies the changes in the queue.
--- arch/x86/include/asm/jump_label.h | 2 + arch/x86/kernel/jump_label.c | 88 +++++++++++++++++++++++++++++++ 2 files changed, 90 insertions(+)
arch/x86/include/asm/jump_label.h +2 -0
@@ -2,6 +2,8 @@#ifndef _ASM_X86_JUMP_LABEL_H#define _ASM_X86_JUMP_LABEL_H +#define HAVE_JUMP_LABEL_BATCH +#define JUMP_LABEL_NOP_SIZE 5#ifdef CONFIG_X86_64
arch/x86/kernel/jump_label.c +88 -0
100 lines not on this page · raw message
Signature · 1 line
2.17.1
PATCH v2 9/9 jump_label: Batch up if arch supports it
In reply to Daniel Bristot de OliveiraIf the architecture supports the batching of jump label updates, use it!
An easy way to see the benefits of this patch is switching the schedstats on and off. For instance:
-------------------------- %< ---------------------------- #!/bin/bash
while [ true ]; do
sysctl -w kernel.sched_schedstats=1
sleep 2
sysctl -w kernel.sched_schedstats=0
sleep 2
done
-------------------------- >% ----------------------------
while watching the IPI count:
-------------------------- %< ----------------------------
# watch -n1 "cat /proc/interrupts | grep Function" -------------------------- >% ----------------------------
With the current mode, it is possible to see +- 168 IPIs each 2 seconds, while with this patch the number of IPIs goes to 3 each 2 seconds.
Regarding the performance impact of this patch set, I made two measurements:
The time to update a key (the task that is causing the change) The time to run the int3 handler (the side effect on a thread that
hits the code being changed)
The schedstats static key was chosen as the key to being switched on and off. The reason being is that it is used in more than 56 places, in a hot path. The change in the schedstats static key will be done with the following command:
while [ true ]; do
sysctl -w kernel.sched_schedstats=1
usleep 500000
sysctl -w kernel.sched_schedstats=0
usleep 500000
done
In this way, they key will be updated twice per second. To force the hit of the int3 handler, the system will also run a kernel compilation with two jobs per CPU. The test machine is a two nodes/24 CPUs box with an Intel Xeon processor @2.27GHz.
Regarding the update part, on average, the regular kernel takes 57 ms to update the schedstats key, while the kernel with the batch updates takes just 1.4 ms on average. Although it seems to be too good to be true, it makes sense: the schedstats key is used in 56 places, so it was expected that it would take around 56 times to update the keys with the current implementation, as the IPIs are the most expensive part of the update.
Regarding the int3 handler, the non-batch handler takes 45 ns on average, while the batch version takes around 180 ns. At first glance, it seems to be a high value. But it is not, considering that it is doing 56 updates, rather than one! It is taking four times more, only. This gain is possible because the patch uses a binary search in the vector: log2(56)=5.8. So, it was expected to have an overhead within four times.
(voice of tv propaganda) But, that is not all! As the int3 handler keeps on for a shorter period (because the update part is on for a shorter time), the number of hits in the int3 handler decreased by 10%.
The question then is: Is it worth paying the price of "135 ns" more in the int3 handler?
Considering that, in this test case, we are saving the handling of 53 IPIs, that takes more than these 135 ns, it seems to be a meager price to be paid. Moreover, the test case was forcing the hit of the int3, in practice, it does not take that often. While the IPI takes place on all CPUs, hitting the int3 handler or not!
For instance, in an isolated CPU with a process running in user-space (nohz_full use-case), the chances of hitting the int3 handler is barely zero, while there is no way to avoid the IPIs. By bounding the IPIs, we are improving a lot this scenario.
--- include/linux/jump_label.h | 3 +++ kernel/jump_label.c | 29 +++++++++++++++++++++++++++++ 2 files changed, 32 insertions(+)
include/linux/jump_label.h +3 -0
@@ -219,6 +219,9 @@ extern void arch_jump_label_transform(struct jump_entry *entry, enum jump_label_type type);extern void arch_jump_label_transform_static (struct jump_entry *entry,enum jump_label_type type); +extern int arch_jump_label_transform_queue (struct jump_entry *entry, +enum jump_label_type type); +extern void arch_jump_label_transform_apply (void );extern int jump_label_text_reserved (void *start,void *end);extern void static_key_slow_inc (struct static_key *key);extern void static_key_slow_dec (struct static_key *key);
kernel/jump_label.c +29 -0
@@ -409,6 +409,7 @@ bool jump_label_can_update_check(struct jump_entry *entry, bool init)return 0 ; } +#ifndef HAVE_JUMP_LABEL_BATCH static void __jump_label_update(struct static_key *key, struct jump_entry *entry, struct jump_entry *stop,
@@ -421,6 +422,34 @@ static void __jump_label_update(struct static_key *key, } } } +#else +static void __jump_label_update (struct static_key *key, +struct jump_entry *entry, +struct jump_entry *stop, +bool init) +{ +for_each_label_entry (key, entry, stop) { + +if (!jump_label_can_update_check (entry, init)) +continue ; + +if (arch_jump_label_transform_queue (entry, +jump_label_type (entry))) +continue ; + +/* +* Queue's overflow: Apply the current queue, and then +* queue again. If it stills not possible to queue, BUG! +*/ +arch_jump_label_transform_apply (); +if (!arch_jump_label_transform_queue (entry, +jump_label_type (entry))) { +BUG (); + } + } +arch_jump_label_transform_apply (); +} +#endif void __init jump_label_init (void ) {
Signature · 1 line
2.17.1
Steven Rostedt
reviewerRe: [PATCH V2 5/9] x86/alternative: Split text_poke_bp() into tree steps
In reply to Daniel Bristot de OliveiraSplitting text_poke_bp() into tree steps. Is this a subtle joke on static_branches?
;-)
-- Steve
Steven Rostedt
reviewerRe: [PATCH V2 7/9] x86/alternative: Batch of patch operations
In reply to Daniel Bristot de OliveiraQuoted 12 lines · On Tue, 18 Dec 2018 17:46:36 +0100 Daniel Bristot de Oliveira…
arch/x86/kernel/alternative.c +12
-0+void text_poke_bp_batch (struct text_to_poke *tp,unsigned int nr_entries) +{ +unsigned int i; +unsigned char int3 =0xcc ; +int patched_all_but_first =0 ; + + bp_int3_tpv = tp; + bp_int3_tpv_nr = nr_entries; + bp_patching_in_progress = true; +/* +* Corresponding read barrier in int3 notifier for making sure the +* in_progress and handler are correctly ordered wrt. patching.
arch/x86/kernel/alternative.c void *text_poke_bp(void *addr, const void *opcode, size_t len, void *handler) · line 920
Quoted 10 lines
arch/x86/kernel/alternative.c +10
-0+*/ +smp_wmb (); + +for (i =0 ; i < nr_entries; i++) +text_poke_bp_set_handler (tp[i].addr, tp[i].handler, int3); + +on_each_cpu (do_sync_core,NULL ,1 ); + +for (i =0 ; i < nr_entries; i++) { +if (tp->len -sizeof (int3) >0 ) {
Should this be:
if (tp[i].len - sizeof(int3) > 0) {?
-- Steve
Quoted 27 lines
arch/x86/kernel/alternative.c +27
-0+patch_all_but_first_byte (tp[i].addr, tp[i].opcode, + tp[i].len, int3); + patched_all_but_first++; + } + } + +if (patched_all_but_first) { +/* +* According to Intel, this core syncing is very likely +* not necessary and we'd be safe even without it. But +* better safe than sorry (plus there's not only Intel). +*/ +on_each_cpu (do_sync_core,NULL ,1 ); + } + +for (i =0 ; i < nr_entries; i++) +patch_first_byte (tp[i].addr, tp[i].opcode, int3); + +on_each_cpu (do_sync_core,NULL ,1 ); +/* +* sync_core() implies an smp_mb() and orders this store against +* the writing of the new instruction. +*/ + bp_int3_tpv_nr =0 ; + bp_int3_tpv =NULL ; + bp_patching_in_progress = false; +}
Steven Rostedt
reviewerRe: [PATCH V2 9/9] jump_label: Batch up if arch supports it
In reply to Daniel Bristot de OliveiraOn Tue, 18 Dec 2018 17:46:38 +0100
Daniel Bristot de Oliveira <[email protected]> wrote:
I'd say add this file first, before x86 supports it. That way it's easy for you to test if this file is correct for other archs that do not support it.
When x86 supports it, the "on switch" for that should be the added config, just like what other architectures will do.
Quoted 147 lines
If the architecture supports the batching of jump label updates, use it!
An easy way to see the benefits of this patch is switching the schedstats on and off. For instance:
-------------------------- %< ---------------------------- #!/bin/bash
while [ true ]; do
sysctl -w kernel.sched_schedstats=1
sleep 2
sysctl -w kernel.sched_schedstats=0
sleep 2
done
-------------------------- >% ----------------------------while watching the IPI count:
-------------------------- %< ----------------------------
# watch -n1 "cat /proc/interrupts | grep Function" -------------------------- >% ----------------------------With the current mode, it is possible to see +- 168 IPIs each 2 seconds, while with this patch the number of IPIs goes to 3 each 2 seconds.
Regarding the performance impact of this patch set, I made two measurements:
The time to update a key (the task that is causing the change) The time to run the int3 handler (the side effect on a thread that
hits the code being changed)The schedstats static key was chosen as the key to being switched on and off. The reason being is that it is used in more than 56 places, in a hot path. The change in the schedstats static key will be done with the following command:
while [ true ]; do
sysctl -w kernel.sched_schedstats=1
usleep 500000
sysctl -w kernel.sched_schedstats=0
usleep 500000
doneIn this way, they key will be updated twice per second. To force the hit of the int3 handler, the system will also run a kernel compilation with two jobs per CPU. The test machine is a two nodes/24 CPUs box with an Intel Xeon processor @2.27GHz.
Regarding the update part, on average, the regular kernel takes 57 ms to update the schedstats key, while the kernel with the batch updates takes just 1.4 ms on average. Although it seems to be too good to be true, it makes sense: the schedstats key is used in 56 places, so it was expected that it would take around 56 times to update the keys with the current implementation, as the IPIs are the most expensive part of the update.
Regarding the int3 handler, the non-batch handler takes 45 ns on average, while the batch version takes around 180 ns. At first glance, it seems to be a high value. But it is not, considering that it is doing 56 updates, rather than one! It is taking four times more, only. This gain is possible because the patch uses a binary search in the vector: log2(56)=5.8. So, it was expected to have an overhead within four times.
(voice of tv propaganda) But, that is not all! As the int3 handler keeps on for a shorter period (because the update part is on for a shorter time), the number of hits in the int3 handler decreased by 10%.
The question then is: Is it worth paying the price of "135 ns" more in the int3 handler?
Considering that, in this test case, we are saving the handling of 53 IPIs, that takes more than these 135 ns, it seems to be a meager price to be paid. Moreover, the test case was forcing the hit of the int3, in practice, it does not take that often. While the IPI takes place on all CPUs, hitting the int3 handler or not!
For instance, in an isolated CPU with a process running in user-space (
nohz_fulluse-case), the chances of hitting the int3 handler is barely zero, while there is no way to avoid the IPIs. By bounding the IPIs, we are improving a lot this scenario.Signed-off-by Daniel Bristot de Oliveira <[email protected]>Cc Thomas Gleixner <[email protected]>Cc Ingo Molnar <[email protected]>Cc Borislav Petkov <[email protected]>Cc "H. Peter Anvin" <[email protected]>Cc Greg Kroah-Hartman <[email protected]>Cc Pavel Tatashin <[email protected]>Cc Masami Hiramatsu <[email protected]>Cc "Steven Rostedt (VMware)" <[email protected]>Cc Zhou Chengming <[email protected]>Cc Jiri Kosina <[email protected]>Cc Josh Poimboeuf <[email protected]>Cc "Peter Zijlstra (Intel)" <[email protected]>Cc Chris von Recklinghausen <[email protected]>Cc Jason Baron <[email protected]>Cc Scott Wood <[email protected]>Cc Marcelo Tosatti <[email protected]>Cc Clark Williams <[email protected]>--- include/linux/jump_label.h | 3 +++ kernel/jump_label.c | 29 +++++++++++++++++++++++++++++ 2 files changed, 32 insertions(+)include/linux/jump_label.h +3
-0@@ -219,6 +219,9 @@ extern void arch_jump_label_transform(struct jump_entry *entry, enum jump_label_type type);extern void arch_jump_label_transform_static (struct jump_entry *entry,enum jump_label_type type); +extern int arch_jump_label_transform_queue (struct jump_entry *entry, +enum jump_label_type type); +extern void arch_jump_label_transform_apply (void );extern int jump_label_text_reserved (void *start,void *end);extern void static_key_slow_inc (struct static_key *key);extern void static_key_slow_dec (struct static_key *key);kernel/jump_label.c +14
-0@@ -409,6 +409,7 @@ bool jump_label_can_update_check(struct jump_entry *entry, bool init)return 0 ; } +#ifndef HAVE_JUMP_LABEL_BATCH static void __jump_label_update(struct static_key *key, struct jump_entry *entry, struct jump_entry *stop,@@ -421,6 +422,34 @@ static void __jump_label_update(struct static_key *key, } } } +#else +static void __jump_label_update (struct static_key *key, +struct jump_entry *entry, +struct jump_entry *stop, +bool init) +{ +for_each_label_entry (key, entry, stop) { + +if (!jump_label_can_update_check (entry, init)) +continue ; + +if (arch_jump_label_transform_queue (entry, +jump_label_type (entry)))
kernel/jump_label.c static void __jump_label_update(struct static_key *key, · line 447
Quoted 10 lines
kernel/jump_label.c +10
-0+continue ; + +/* +* Queue's overflow: Apply the current queue, and then +* queue again. If it stills not possible to queue, BUG! +*/ +arch_jump_label_transform_apply (); +if (!arch_jump_label_transform_queue (entry, +jump_label_type (entry))) { +BUG ();
Why BUG()? Do you really want to crash Linus's machine?
-- Steve
Quoted 8 lines
kernel/jump_label.c +5
-0+ } + } +arch_jump_label_transform_apply (); +} +#endif void __init jump_label_init (void ) {
Re: [PATCH V2 7/9] x86/alternative: Batch of patch operations
In reply to Steven RostedtQuoted 27 lines · Steven Rostedt
Quoted 12 lines · On Tue, 18 Dec 2018 17:46:36 +0100 Daniel Bristot de Oliveira…
arch/x86/kernel/alternative.c +12
-0+void text_poke_bp_batch (struct text_to_poke *tp,unsigned int nr_entries) +{ +unsigned int i; +unsigned char int3 =0xcc ; +int patched_all_but_first =0 ; + + bp_int3_tpv = tp; + bp_int3_tpv_nr = nr_entries; + bp_patching_in_progress = true; +/* +* Corresponding read barrier in int3 notifier for making sure the +* in_progress and handler are correctly ordered wrt. patching. arch/x86/kernel/alternative.c void *text_poke_bp(void *addr, const void *opcode, size_t len, void *handler) · line 920
Quoted 10 lines
arch/x86/kernel/alternative.c +10
-0+*/ +smp_wmb (); + +for (i =0 ; i < nr_entries; i++) +text_poke_bp_set_handler (tp[i].addr, tp[i].handler, int3); + +on_each_cpu (do_sync_core,NULL ,1 ); + +for (i =0 ; i < nr_entries; i++) { +if (tp->len -sizeof (int3) >0 ) {Should this be:
Quoted 3 lines
if (tp[i].len - sizeof(int3) > 0) {?
Ops! Missed that... you are right.
Fixing in the v3.
Thanks!
-- Daniel
Re: [PATCH V2 9/9] jump_label: Batch up if arch supports it
In reply to Steven RostedtQuoted 10 lines · Steven Rostedt
On Tue, 18 Dec 2018 17:46:38 +0100
Daniel Bristot de Oliveira <[email protected]> wrote:I'd say add this file first, before x86 supports it. That way it's easy for you to test if this file is correct for other archs that do not support it.
When x86 supports it, the "on switch" for that should be the added config, just like what other architectures will do.
right!
Quoted 151 lines
Quoted 151 lines
If the architecture supports the batching of jump label updates, use it!
An easy way to see the benefits of this patch is switching the schedstats on and off. For instance:
-------------------------- %< ---------------------------- #!/bin/bash
while [ true ]; do
sysctl -w kernel.sched_schedstats=1
sleep 2
sysctl -w kernel.sched_schedstats=0
sleep 2
done
-------------------------- >% ----------------------------while watching the IPI count:
-------------------------- %< ----------------------------
# watch -n1 "cat /proc/interrupts | grep Function" -------------------------- >% ----------------------------With the current mode, it is possible to see +- 168 IPIs each 2 seconds, while with this patch the number of IPIs goes to 3 each 2 seconds.
Regarding the performance impact of this patch set, I made two measurements:
The time to update a key (the task that is causing the change) The time to run the int3 handler (the side effect on a thread that
hits the code being changed)The schedstats static key was chosen as the key to being switched on and off. The reason being is that it is used in more than 56 places, in a hot path. The change in the schedstats static key will be done with the following command:
while [ true ]; do
sysctl -w kernel.sched_schedstats=1
usleep 500000
sysctl -w kernel.sched_schedstats=0
usleep 500000
doneIn this way, they key will be updated twice per second. To force the hit of the int3 handler, the system will also run a kernel compilation with two jobs per CPU. The test machine is a two nodes/24 CPUs box with an Intel Xeon processor @2.27GHz.
Regarding the update part, on average, the regular kernel takes 57 ms to update the schedstats key, while the kernel with the batch updates takes just 1.4 ms on average. Although it seems to be too good to be true, it makes sense: the schedstats key is used in 56 places, so it was expected that it would take around 56 times to update the keys with the current implementation, as the IPIs are the most expensive part of the update.
Regarding the int3 handler, the non-batch handler takes 45 ns on average, while the batch version takes around 180 ns. At first glance, it seems to be a high value. But it is not, considering that it is doing 56 updates, rather than one! It is taking four times more, only. This gain is possible because the patch uses a binary search in the vector: log2(56)=5.8. So, it was expected to have an overhead within four times.
(voice of tv propaganda) But, that is not all! As the int3 handler keeps on for a shorter period (because the update part is on for a shorter time), the number of hits in the int3 handler decreased by 10%.
The question then is: Is it worth paying the price of "135 ns" more in the int3 handler?
Considering that, in this test case, we are saving the handling of 53 IPIs, that takes more than these 135 ns, it seems to be a meager price to be paid. Moreover, the test case was forcing the hit of the int3, in practice, it does not take that often. While the IPI takes place on all CPUs, hitting the int3 handler or not!
For instance, in an isolated CPU with a process running in user-space (
nohz_fulluse-case), the chances of hitting the int3 handler is barely zero, while there is no way to avoid the IPIs. By bounding the IPIs, we are improving a lot this scenario.Signed-off-by Daniel Bristot de Oliveira <[email protected]>Cc Thomas Gleixner <[email protected]>Cc Ingo Molnar <[email protected]>Cc Borislav Petkov <[email protected]>Cc "H. Peter Anvin" <[email protected]>Cc Greg Kroah-Hartman <[email protected]>Cc Pavel Tatashin <[email protected]>Cc Masami Hiramatsu <[email protected]>Cc "Steven Rostedt (VMware)" <[email protected]>Cc Zhou Chengming <[email protected]>Cc Jiri Kosina <[email protected]>Cc Josh Poimboeuf <[email protected]>Cc "Peter Zijlstra (Intel)" <[email protected]>Cc Chris von Recklinghausen <[email protected]>Cc Jason Baron <[email protected]>Cc Scott Wood <[email protected]>Cc Marcelo Tosatti <[email protected]>Cc Clark Williams <[email protected]>--- include/linux/jump_label.h | 3 +++ kernel/jump_label.c | 29 +++++++++++++++++++++++++++++ 2 files changed, 32 insertions(+)include/linux/jump_label.h +3
-0@@ -219,6 +219,9 @@ extern void arch_jump_label_transform(struct jump_entry *entry, enum jump_label_type type);extern void arch_jump_label_transform_static (struct jump_entry *entry,enum jump_label_type type); +extern int arch_jump_label_transform_queue (struct jump_entry *entry, +enum jump_label_type type); +extern void arch_jump_label_transform_apply (void );extern int jump_label_text_reserved (void *start,void *end);extern void static_key_slow_inc (struct static_key *key);extern void static_key_slow_dec (struct static_key *key);kernel/jump_label.c +18
-0@@ -409,6 +409,7 @@ bool jump_label_can_update_check(struct jump_entry *entry, bool init)return 0 ; } +#ifndef HAVE_JUMP_LABEL_BATCH static void __jump_label_update(struct static_key *key, struct jump_entry *entry, struct jump_entry *stop,@@ -421,6 +422,34 @@ static void __jump_label_update(struct static_key *key, } } } +#else +static void __jump_label_update (struct static_key *key, +struct jump_entry *entry, +struct jump_entry *stop, +bool init) +{ +for_each_label_entry (key, entry, stop) { + +if (!jump_label_can_update_check (entry, init)) +continue ; + +if (arch_jump_label_transform_queue (entry, +jump_label_type (entry))) +continue ; + +/* +* Queue's overflow: Apply the current queue, and then
Quoted 8 lines
kernel/jump_label.c static void __jump_label_update(struct static_key *key, · line 447
Quoted 6 lines
kernel/jump_label.c +6
-0+* queue again. If it stills not possible to queue, BUG! +*/ +arch_jump_label_transform_apply (); +if (!arch_jump_label_transform_queue (entry, +jump_label_type (entry))) { +BUG ();Why
BUG()? Do you really want to crash Linus's machine?
I am using BUG() because that is what I see in other part of jump_label code:
If something goes wrong: BUG().
What I could do here is:
Add a "fallback" boll that is disabled by default. If I hit this case:
WARN() turn "fallback" on, returning to the old mode (without batch)
Sound better?
Thanks!
-- Daniel
Quoted 11 lines
-- Steve
Quoted 8 lines
kernel/jump_label.c +5
-0+ } + } +arch_jump_label_transform_apply (); +} +#endif void __init jump_label_init (void ) {
Steven Rostedt
reviewerRe: [PATCH V2 9/9] jump_label: Batch up if arch supports it
Quoted 26 lines · On Tue, 18 Dec 2018 20:33:48 +0100 Daniel Bristot de Oliveira…
Quoted 17 lines
Quoted 17 lines
kernel/jump_label.c +17
-0+#else +static void __jump_label_update (struct static_key *key, +struct jump_entry *entry, +struct jump_entry *stop, +bool init) +{ +for_each_label_entry (key, entry, stop) { + +if (!jump_label_can_update_check (entry, init)) +continue ; + +if (arch_jump_label_transform_queue (entry, +jump_label_type (entry))) +continue ; + +/* +* Queue's overflow: Apply the current queue, and then Quoted 8 lines
kernel/jump_label.c static void __jump_label_update(struct static_key *key, · line 447
Quoted 6 lines
kernel/jump_label.c +6
-0+* queue again. If it stills not possible to queue, BUG! +*/ +arch_jump_label_transform_apply (); +if (!arch_jump_label_transform_queue (entry, +jump_label_type (entry))) { +BUG ();Why
BUG()? Do you really want to crash Linus's machine?
Quoted 3 lines
I am using
BUG()because that is what I see in other part ofjump_labelcode:If something goes wrong: BUG().
Where? Mostly we have BUILD_BUG_ON() which isn't bad at all.
The only other BUG I see in the jump label code is in the arch specific code and that's from 2012. Lately, we are trying to get rid of BUG() and panic() entirely, with a few exceptions (where there's really no way to return. Like when the function graph stack is corrupted, and we don't know where to go).
If there's a way to continue in a critical state, it's best to WARN() and continue on. That way, the user can have a chance to see what happened.
Quoted 9 lines
What I could do here is:
Add a "fallback" boll that is disabled by default. If I hit this case:
WARN() turn "fallback" on, returning to the old mode (without batch)Sound better?
Yes, please do.
Thanks!
-- Steve
Re: [PATCH V2 9/9] jump_label: Batch up if arch supports it
Quoted 11 lines · Steven Rostedt
Where? Mostly we have
BUILD_BUG_ON()which isn't bad at all.The only other BUG I see in the jump label code is in the arch specific code and that's from 2012. Lately, we are trying to get rid of
BUG()andpanic()entirely, with a few exceptions (where there's really no way to return. Like when the function graph stack is corrupted, and we don't know where to go).If there's a way to continue in a critical state, it's best to
WARN()and continue on. That way, the user can have a chance to see what happened.
Got it!
Quoted 9 lines
Quoted 8 lines
What I could do here is:
Add a "fallback" boll that is disabled by default. If I hit this case:
WARN() turn "fallback" on, returning to the old mode (without batch)Sound better?
Yes, please do.
ack!
Thanks, Steve!
-- Daniel
Borislav Petkov
maintainerRe: [PATCH V2 1/9] jump_label: Add for_each_label_entry helper
In reply to Daniel Bristot de OliveiraQuoted 7 lines · Daniel Bristot de Oliveira
This patch adds the helper:
for_each_label_entry(key, entry, stop)For the "for each jump label entry" for defined as:
for (; (entry < stop) && (jump_entry_key(entry) == key); entry++)Simplifying the reading and usage.
Please avoid writing "This patch" in a commit message and also what it does - that should be obvious. Instead say something like:
"Add a helper macro to make jump entry iteration code more readable."
or so.
But then, IINM, this macro is being used only once. Isn't that a bit too much? I mean, you could just as well do:
# iterate over each jump entry
for (; (entry < stop) && (jump_entry_key(entry) == key); entry++) {and have it even more readable without introducing macro which is not going to be used elsewhere. Or is it going to...?
Thx.
Signature · 4 lines
Regards/Gruss,
Boris.
Good mailing practices for 400: avoid top-posting and trim the reply.Re: [PATCH V2 1/9] jump_label: Add for_each_label_entry helper
Quoted 7 lines · Borislav Petkov
Quoted 6 lines · Daniel Bristot de Oliveira
This patch adds the helper:
for_each_label_entry(key, entry, stop)For the "for each jump label entry" for defined as:
for (; (entry < stop) && (jump_entry_key(entry) == key); entry++)
Quoted 8 lines
Quoted 1 line
Simplifying the reading and usage.
Please avoid writing "This patch" in a commit message and also what it does - that should be obvious. Instead say something like:
"Add a helper macro to make jump entry iteration code more readable."
or so.
Right! I will change it!
Quoted 8 lines
But then, IINM, this macro is being used only once. Isn't that a bit too much? I mean, you could just as well do:
# iterate over each jump entry for (; (entry < stop) && (jump_entry_key(entry) == key); entry++) {and have it even more readable without introducing macro which is not going to be used elsewhere. Or is it going to...?
It is also used in the patch 9. But I can remove it, no problem.
Thoughts?
Thanks!
-- Daniel
Re: [PATCH V2 3/9] x86/jump_label: Move checking code away from __jump_label_transform()
In reply to Daniel Bristot de OliveiraQuoted 1 line · Daniel Bristot de Oliveira
This patch creates a new functions
I am not a native speaker, but this doesn't sound like proper english to me.
Quoted 1 line
to check the current code before updating a
jump_entry.
It should also be stated that this is equivalent code transformation, I think.
Thanks,
Signature · 2 lines
Jiri Kosina SUSE Labs
Re: [PATCH V2 5/9] x86/alternative: Split text_poke_bp() into tree steps
In reply to Daniel Bristot de OliveiraQuoted 7 lines · Daniel Bristot de Oliveira
text_poke_bp()updates instructions on live kernel on SMP in three steps:
1) add a int3 trap to the address that will be patched
2) update all but the first byte of the patched range
3) replace the first byte (int3) by the first byte ofThis patch creates one function for each of these steps.
Quoted 11 lines
Signed-off-by Daniel Bristot de Oliveira <[email protected]>Cc Thomas Gleixner <[email protected]>Cc Ingo Molnar <[email protected]>Cc Borislav Petkov <[email protected]>Cc "H. Peter Anvin" <[email protected]>Cc Greg Kroah-Hartman <[email protected]>Cc Pavel Tatashin <[email protected]>Cc Masami Hiramatsu <[email protected]>Cc "Steven Rostedt (VMware)" <[email protected]>Cc Zhou Chengming <[email protected]>Cc Jiri Kosina <[email protected]>
Signature · 2 lines
Jiri Kosina SUSE Labs
Re: [PATCH V2 3/9] x86/jump_label: Move checking code away from __jump_label_transform()
In reply to Jiri KosinaQuoted 6 lines · Jiri Kosina
Quoted 1 line · Daniel Bristot de Oliveira
This patch creates a new functions
I am not a native speaker, but this doesn't sound like proper english to me.
Aside of that 'This patch' is wrong to begin with. See Documentation/process/
Re: [PATCH V2 3/9] x86/jump_label: Move checking code away from __jump_label_transform()
Quoted 11 lines · Thomas Gleixner
Quoted 6 lines · Jiri Kosina
Quoted 1 line · Daniel Bristot de Oliveira
This patch creates a new functions
I am not a native speaker, but this doesn't sound like proper english to me.
Aside of that 'This patch' is wrong to begin with. See
Documentation/process/
I will address these errors in the next version.
Thanks.
-- Daniel
Borislav Petkov
maintainerRe: [PATCH V2 1/9] jump_label: Add for_each_label_entry helper
In reply to Daniel Bristot de OliveiraQuoted 1 line · Daniel Bristot de Oliveira
It is also used in the patch 9. But I can remove it, no problem.
Oh, that's a function with the same name but in the HAVE_JUMP_LABEL_BATCH #else branch. Oh well, whatever people prefer.
In my case, I always end up looking at the macro definition anyway. But this is just me.
Thx.
Signature · 4 lines
Regards/Gruss,
Boris.
Good mailing practices for 400: avoid top-posting and trim the reply.