Skip to content

RFC jump label v3

Jason Baron lkml 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 RFC Latest6 patches 24 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/6jump label v3 - kprobes/x86: Cleanup RELATIVEJUMP_INSTRUCTION to RELATIVEJUMP_OPCODEarch/x86/include/asm/kprobes.h arch/x86/kernel/kprobes.c+2 -20 replies
2/6jump label v3 - x86: Introduce generic jump patching without stop_machinearch/x86/include/asm/alternative.h arch/x86/include/asm/kprobes.h arch/x86/kernel/alternative.c +1 more+134 -119 replies
3/6jump label v3 - move opcode defsarch/x86/include/asm/alternative.h arch/x86/include/asm/kprobes.h+5 -40 replies
4/6jump label v3 - base patcharch/x86/include/asm/jump_label.h arch/x86/kernel/Makefile arch/x86/kernel/jump_label.c +2 more+154 -24 replies
5/6jump label v3 - add module supportarch/x86/kernel/jump_label.c include/linux/jump_label.h include/linux/module.h +1 more+42 -20 replies
6/6jump label v3 - tracepoint supportinclude/linux/tracepoint.h kernel/module.c kernel/tracepoint.c+38 -240 replies

Reviews trailers per patch

1 of 6 patches has a Reviewed-by or Acked-by.

PatchPaul E. McKenneyMasami Hiramatsu
1/6
2/6
3/6
4/6R-b R-b
5/6
6/6

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v1 thread

Cover letter [RFC PATCH 0/6] jump label v3

hi,

Refresh of the jump labeling patches. We introduce the following:

# ifdef CONFIG_X86_64
#  define JUMP_LABEL_NOP P6_NOP5
# else
#  define JUMP_LABEL_NOP ".byte 0xe9 \n\t .long 0\n\t"
# endif
# define JUMP_LABEL(tag, label, cond)                                       \
       do {                                                               \
               static const char __jlstrtab_##tag[]                       \
               __used __attribute__((section("__jump_strings")))  = #tag; \
               asm goto("1:"                                              \
                       JUMP_LABEL_NOP                                     \
                       ".pushsection __jump_table,  \"a\" \n\t"           \
                       _ASM_PTR "1b, %l[" #label "], %c0 \n\t"            \
                       ".popsection \n\t"                                 \
                       : :  "i" (__jlstrtab_##tag) :  : label);           \
       } while (0)

-------------

I'm using an atomic 5 byte no-op for x86_64 and a long jump for 32-bit x86. My understanding is that not all 32-bit processors have an atomic 5 byte no-op, and thus using a long jump or jump 0, for the off case is the safest.

which can then be used by the tracepoint code as:

#define DECLARE_TRACE(name, proto, args)                                \
       extern struct tracepoint __tracepoint_##name;                    \
       static inline void trace_##name(proto)                           \
       {                                                                \
               JUMP_LABEL(name, do_trace, __tracepoint_##name.state);   \
               return;                                                  \
do_trace:                                                               \
               __DO_TRACE(&__tracepoint_##name,                         \
                          TP_PROTO(proto), TP_ARGS(args));              \

--------------

Thus, in the disabled tracing case we have a no-op followed by a jump around the disabled code. When we enable the tracepoint, we simply patch the no-op with a jump to the 'do_trace' label. This relies on the 'asm goto' construct which is already merged into gcc 4.5. In subsequent gcc versions, we also hope to make use of 'cold' label for the 'do_trace' section. Thus, making the disabled or straight line codepath, simply a no-op.

As discussed in pervious mails I've seen an average improvement of 30 cycles per-tracepoint on x86_64 systems that I've tested.

The first 2 patches of the series are a repost of Masami's text_poke_fixup() function, which allows for efficient instruction patching. The final 4 patches, implement the the jump patching mechanism for x86 and x86_64.

The implementation is a 'low' level one, in the sense that it is geared specifically for tracepoints. However, I believe this mechanism will be more generally useful for other parts of the kernel. Thus, I will propose 'higher' level interfaces into the jump label code (layered on these 'low' level ones), as we go.

thanks,

-Jason

Masami Hiramatsu (2):
x86: Introduce generic jump patching without stop_machine kprobes/x86: Cleanup RELATIVEJUMP_INSTRUCTION to RELATIVEJUMP_OPCODE

Jason Baron(4):
move opcode defs from asm/kprobes.h to asm/alternative.h jump-label-basic
jump-module-support
jump-label-tracepoints

 arch/x86/include/asm/alternative.h |   17 +++++
 arch/x86/include/asm/jump_label.h  |   35 +++++++++++
 arch/x86/include/asm/kprobes.h     |    3 -
 arch/x86/kernel/Makefile           |    2 +-
 arch/x86/kernel/alternative.c      |  120 ++++++++++++++++++++++++++++++++++++
 arch/x86/kernel/jump_label.c       |   66 ++++++++++++++++++++
 arch/x86/kernel/kprobes.c          |    2 +-
 include/asm-generic/vmlinux.lds.h  |   11 +++-
 include/linux/jump_label.h         |   47 ++++++++++++++
 include/linux/module.h             |   12 +++-
 include/linux/tracepoint.h         |   35 ++++++-----
 kernel/kprobes.c                   |    2 +-
 kernel/module.c                    |   27 ++++++++-
 kernel/tracepoint.c                |   25 ++++++--
 14 files changed, 372 insertions(+), 32 deletions(-)
 create mode 100644 arch/x86/include/asm/jump_label.h
 create mode 100644 arch/x86/kernel/jump_label.c
 create mode 100644 include/linux/jump_label.h