perf clang: Builtin clang and perfhook support
v4 cover perf clang: Builtin clang and perfhook support
This is version 4 of perf builtin clang and perfhook patch series.
In this patch set:
1. Cleanup options passed to clang.
2. Check LLVM version. Fail if llvm version < 3.9.0, and warn when it > 3.9.0. More version will be allowed after releasing of LLVM 3.9.1 / 4.0.0.
3. Support dynamic linking LLVM and clang libraries.
4. Rebase to Arnaldo's newest perf/core.
Wang Nan (18):
perf build: Check LLVM version in feature check
perf build: Support dynamic linking clang libraries
perf clang: Cleanup clang options
perf clang: Pass full path to builtin clang
perf clang: Pass CFLAGS to builtin clang
perf clang jit: Wrap llvm::Module using PerfModule
perf clang jit: Insignt BPF and JIT functions in a Module perf clang jit: add PerfModule::doJIT to JIT perfhook functions perf clang jit: Export functions for jitted code
perf clang jit: Actually JIT and hook in bpf loader
perf clang jit: Collect the lowest address in maps section as map_base perf clang jit: Retrive fd of BPF map from its offset
perf clang jit: Allow jitted perf hook access BPF maps perf clang: Link BPF functions declaration into perf
perf clang: Declare BPF functions for BPF scripts automatically perf clang: Include helpers to BPF scripts
perf clang builtin: Define hook helpers by default
perf clang jit: Export getpid() to perf hook
tools/build/feature/Makefile | 8 +- tools/build/feature/test-llvm-version.cpp | 12 + tools/build/feature/test-llvm.cpp | 5 + tools/perf/Makefile.config | 8 +- tools/perf/Makefile.perf | 20 +- tools/perf/tests/Build | 2 +- tools/perf/tests/bpf-script-example.c | 30 ++- tools/perf/tests/bpf-script-test-kbuild.c | 2 + tools/perf/tests/bpf-script-test-prologue.c | 6 +- tools/perf/tests/bpf-script-test-relocation.c | 17 +- tools/perf/tests/clang.c | 4 + tools/perf/util/Build | 1 + tools/perf/util/bpf-loader.c | 85 ++++++- tools/perf/util/bpf-loader.h | 19 ++ tools/perf/util/c++/Build | 2 + tools/perf/util/c++/bpf-funcs-str.c | 49 ++++ tools/perf/util/c++/bpf-helper-str.c | 23 ++ tools/perf/util/c++/clang-bpf-includes.h | 13 + tools/perf/util/c++/clang-c.h | 24 +- tools/perf/util/c++/clang-test.cpp | 51 +++- tools/perf/util/c++/clang.cpp | 335 ++++++++++++++++++++++++-- tools/perf/util/c++/clang.h | 47 +++- tools/perf/util/jit-helpers.c | 57 +++++ tools/perf/util/jit-helpers.h | 28 +++ tools/perf/util/llvm-utils.h | 9 +- 25 files changed, 802 insertions(+), 55 deletions(-) create mode 100644 tools/build/feature/test-llvm-version.cpp create mode 100644 tools/perf/util/c++/bpf-funcs-str.c create mode 100644 tools/perf/util/c++/bpf-helper-str.c create mode 100644 tools/perf/util/c++/clang-bpf-includes.h create mode 100644 tools/perf/util/jit-helpers.c create mode 100644 tools/perf/util/jit-helpers.h
Signature · 1 line
2.10.1
PATCH v4 1/18 perf build: Check LLVM version in feature check
Cancel builtin llvm and clang support when LLVM version is less than 3.9.0: following commits uses newer API.
Since Clang/LLVM's API is not guaranteed to be stable, add a test-llvm-version.cpp feature checker, issue warning if LLVM found in compiling environment is not tested yet.
--- tools/build/feature/Makefile | 8 ++++++-- tools/build/feature/test-llvm-version.cpp | 12 ++++++++++++ tools/build/feature/test-llvm.cpp | 5 +++++ tools/perf/Makefile.config | 8 ++++++-- 4 files changed, 29 insertions(+), 4 deletions(-) create mode 100644 tools/build/feature/test-llvm-version.cpp
tools/build/feature/Makefile +6 -2
@@ -231,14 +231,18 @@ $(OUTPUT)test-jvmti.bin: $(BUILD) $(OUTPUT)test-llvm.bin: - $(BUILDXX) -std=gnu++11 \ + $(BUILDXX) -std=gnu++11 \ -I$(shell $(LLVM_CONFIG) --includedir) \ -L$(shell $(LLVM_CONFIG) --libdir) \ $(shell $(LLVM_CONFIG) --libs Core BPF) \ $(shell $(LLVM_CONFIG) --system-libs) +$(OUTPUT)test-llvm-version.bin: + $(BUILDXX) -std=gnu++11 \ + -I$(shell $(LLVM_CONFIG) --includedir) + $(OUTPUT)test-clang.bin: - $(BUILDXX) -std=gnu++11 \ + $(BUILDXX) -std=gnu++11 \ -I$(shell $(LLVM_CONFIG) --includedir) \ -L$(shell $(LLVM_CONFIG) --libdir) \ -Wl,--start-group -lclangBasic -lclangDriver \
tools/build/feature/test-llvm-version.cpp new file +12 -0
@@ -0,0 +1,12 @@ +#include <cstdio> +#include "llvm/Config/llvm-config.h" + +#define NUM_VERSION (((LLVM_VERSION_MAJOR) << 16) + (LLVM_VERSION_MINOR << 8) + LLVM_VERSION_PATCH) +#define pass int main() {printf("%x\n", NUM_VERSION); return 0;} + +#if NUM_VERSION >= 0x030900 +pass +#else +# error This LLVM is not tested yet. +#endif +
tools/build/feature/test-llvm.cpp +5 -0
@@ -1,5 +1,10 @@ #include "llvm/Support/ManagedStatic.h" #include "llvm/Support/raw_ostream.h" +#define NUM_VERSION (((LLVM_VERSION_MAJOR) << 16) + (LLVM_VERSION_MINOR << 8) + LLVM_VERSION_PATCH) + +#if NUM_VERSION < 0x030900 +# error "LLVM version too low" +#endif int main() { llvm::errs() << "Hello World!\n";
tools/perf/Makefile.config +6 -2
@@ -802,12 +802,13 @@ ifdef LIBCLANGLLVM msg := $(warning No g++ found, disable clang and llvm support. Please install g++) else $(call feature_check,llvm) + $(call feature_check,llvm-version) ifneq ($(feature-llvm), 1) - msg := $(warning No libLLVM found, disable clang and llvm support. Please install llvm-dev) + msg := $(warning No libLLVM found, disable clang and llvm support. Please install llvm-dev (>= 3.9.0)) else $(call feature_check,clang) ifneq ($(feature-clang), 1) - msg := $(warning No libclang found, disable clang and llvm support. Please install libclang-dev) + msg := $(warning No libclang found, disable clang and llvm support. Please install libclang-dev (>= 3.9.0)) else CFLAGS += -DHAVE_LIBCLANGLLVM_SUPPORT CXXFLAGS += -DHAVE_LIBCLANGLLVM_SUPPORT -I$(shell $(LLVM_CONFIG) --includedir)
@@ -816,6 +817,9 @@ ifdef LIBCLANGLLVM USE_CXX = 1 USE_LLVM = 1 USE_CLANG = 1 + ifneq ($(feature-llvm-version),1) + msg := $(warning This version of llvm is not tested. May cause building error) + endif endif endif endif
Signature · 1 line
2.10.1
Re: [PATCH v4 01/18] perf build: Check LLVM version in feature check
Quoted 22 lines · Wang Nan
Cancel builtin llvm and clang support when LLVM version is less than 3.9.0: following commits uses newer API.
Since Clang/LLVM's API is not guaranteed to be stable, add a test-llvm-version.cpp feature checker, issue warning if LLVM found in compiling environment is not tested yet.
Signed-off-by Wang Nan <[email protected]>Cc Alexei Starovoitov <[email protected]>Cc Arnaldo Carvalho de Melo <[email protected]>Cc He Kuang <[email protected]>Cc Jiri Olsa <[email protected]>Cc Joe Stringer <[email protected]>Cc Zefan Li <[email protected]>--- tools/build/feature/Makefile | 8 ++++++-- tools/build/feature/test-llvm-version.cpp | 12 ++++++++++++ tools/build/feature/test-llvm.cpp | 5 +++++ tools/perf/Makefile.config | 8 ++++++-- 4 files changed, 29 insertions(+), 4 deletions(-)create mode 100644
tools/build/feature/test-llvm-version.cpp
[SNIP]
Quoted 8 lines
tools/build/feature/test-llvm-version.cpp new file +2
-0@@ -0,0 +1,12 @@ +#include <cstdio> +#include "llvm/Config/llvm-config.h"
tools/build/feature/test-llvm-version.cpp line 12
Quoted 10 lines
tools/build/feature/test-llvm-version.cpp new file +10
-0+ +#define NUM_VERSION (((LLVM_VERSION_MAJOR) << 16) + (LLVM_VERSION_MINOR << 8) + LLVM_VERSION_PATCH) +#define pass int main() {printf("%x\n", NUM_VERSION); return 0;} + +#if NUM_VERSION >= 0x030900 +pass +#else +# error This LLVM is not tested yet. +#endif +
Sorry for this blank line. Will resend.
PATCH v4 1/18 perf build: Check LLVM version in feature check
In reply to Wang NanCancel builtin llvm and clang support when LLVM version is less than 3.9.0: following commits uses newer API.
Since Clang/LLVM's API is not guaranteed to be stable, add a test-llvm-version.cpp feature checker, issue warning if LLVM found in compiling environment is not tested yet.
--- tools/build/feature/Makefile | 8 ++++++-- tools/build/feature/test-llvm-version.cpp | 11 +++++++++++ tools/build/feature/test-llvm.cpp | 5 +++++ tools/perf/Makefile.config | 8 ++++++-- 4 files changed, 28 insertions(+), 4 deletions(-) create mode 100644 tools/build/feature/test-llvm-version.cpp
tools/build/feature/Makefile +6 -2
@@ -231,14 +231,18 @@ $(OUTPUT)test-jvmti.bin: $(BUILD) $(OUTPUT)test-llvm.bin: - $(BUILDXX) -std=gnu++11 \ + $(BUILDXX) -std=gnu++11 \ -I$(shell $(LLVM_CONFIG) --includedir) \ -L$(shell $(LLVM_CONFIG) --libdir) \ $(shell $(LLVM_CONFIG) --libs Core BPF) \ $(shell $(LLVM_CONFIG) --system-libs) +$(OUTPUT)test-llvm-version.bin: + $(BUILDXX) -std=gnu++11 \ + -I$(shell $(LLVM_CONFIG) --includedir) + $(OUTPUT)test-clang.bin: - $(BUILDXX) -std=gnu++11 \ + $(BUILDXX) -std=gnu++11 \ -I$(shell $(LLVM_CONFIG) --includedir) \ -L$(shell $(LLVM_CONFIG) --libdir) \ -Wl,--start-group -lclangBasic -lclangDriver \
tools/build/feature/test-llvm-version.cpp new file +11 -0
@@ -0,0 +1,11 @@ +#include <cstdio> +#include "llvm/Config/llvm-config.h" + +#define NUM_VERSION (((LLVM_VERSION_MAJOR) << 16) + (LLVM_VERSION_MINOR << 8) + LLVM_VERSION_PATCH) +#define pass int main() {printf("%x\n", NUM_VERSION); return 0;} + +#if NUM_VERSION >= 0x030900 +pass +#else +# error This LLVM is not tested yet. +#endif
tools/build/feature/test-llvm.cpp +5 -0
@@ -1,5 +1,10 @@ #include "llvm/Support/ManagedStatic.h" #include "llvm/Support/raw_ostream.h" +#define NUM_VERSION (((LLVM_VERSION_MAJOR) << 16) + (LLVM_VERSION_MINOR << 8) + LLVM_VERSION_PATCH) + +#if NUM_VERSION < 0x030900 +# error "LLVM version too low" +#endif int main() { llvm::errs() << "Hello World!\n";
tools/perf/Makefile.config +6 -2
@@ -802,12 +802,13 @@ ifdef LIBCLANGLLVM msg := $(warning No g++ found, disable clang and llvm support. Please install g++) else $(call feature_check,llvm) + $(call feature_check,llvm-version) ifneq ($(feature-llvm), 1) - msg := $(warning No libLLVM found, disable clang and llvm support. Please install llvm-dev) + msg := $(warning No libLLVM found, disable clang and llvm support. Please install llvm-dev (>= 3.9.0)) else $(call feature_check,clang) ifneq ($(feature-clang), 1) - msg := $(warning No libclang found, disable clang and llvm support. Please install libclang-dev) + msg := $(warning No libclang found, disable clang and llvm support. Please install libclang-dev (>= 3.9.0)) else CFLAGS += -DHAVE_LIBCLANGLLVM_SUPPORT CXXFLAGS += -DHAVE_LIBCLANGLLVM_SUPPORT -I$(shell $(LLVM_CONFIG) --includedir)
@@ -816,6 +817,9 @@ ifdef LIBCLANGLLVM USE_CXX = 1 USE_LLVM = 1 USE_CLANG = 1 + ifneq ($(feature-llvm-version),1) + msg := $(warning This version of llvm is not tested. May cause building error) + endif endif endif endif
Signature · 1 line
2.10.1
Arnaldo Carvalho de Melo
maintainerRe: [PATCH v4 01/18 -cleanup] perf build: Check LLVM version in feature check
Em Tue, Dec 06, 2016 at 07:22:30AM +0000, Wang Nan escreveu:
Quoted 6 lines
Cancel builtin llvm and clang support when LLVM version is less than 3.9.0: following commits uses newer API.
Since Clang/LLVM's API is not guaranteed to be stable, add a test-llvm-version.cpp feature checker, issue warning if LLVM found in compiling environment is not tested yet.
Ok, did a few clarifications in the warning message and tested it with a f25 kit, i.e. clang/LLVM 3.8, all seems ok now, please see:
For my case, I end up getting this message:
Makefile.config:807: No suitable libLLVM found, disabling builtin clang and LLVM support. Please install llvm-dev(el) (>= 3.9.0)
Please holler if it is in any way misleading or erroneous.
- Arnaldo
Quoted 105 lines
Signed-off-by Wang Nan <[email protected]>Cc Alexei Starovoitov <[email protected]>Cc Arnaldo Carvalho de Melo <[email protected]>Cc He Kuang <[email protected]>Cc Jiri Olsa <[email protected]>Cc Joe Stringer <[email protected]>Cc Zefan Li <[email protected]>--- tools/build/feature/Makefile | 8 ++++++-- tools/build/feature/test-llvm-version.cpp | 11 +++++++++++ tools/build/feature/test-llvm.cpp | 5 +++++ tools/perf/Makefile.config | 8 ++++++-- 4 files changed, 28 insertions(+), 4 deletions(-) create mode 100644 tools/build/feature/test-llvm-version.cpptools/build/feature/Makefile +6
-2@@ -231,14 +231,18 @@ $(OUTPUT)test-jvmti.bin: $(BUILD) $(OUTPUT)test-llvm.bin: - $(BUILDXX) -std=gnu++11 \ + $(BUILDXX) -std=gnu++11 \ -I$(shell $(LLVM_CONFIG) --includedir) \ -L$(shell $(LLVM_CONFIG) --libdir) \ $(shell $(LLVM_CONFIG) --libs Core BPF) \ $(shell $(LLVM_CONFIG) --system-libs) +$(OUTPUT)test-llvm-version.bin: + $(BUILDXX) -std=gnu++11 \ + -I$(shell $(LLVM_CONFIG) --includedir) + $(OUTPUT)test-clang.bin: - $(BUILDXX) -std=gnu++11 \ + $(BUILDXX) -std=gnu++11 \ -I$(shell $(LLVM_CONFIG) --includedir) \ -L$(shell $(LLVM_CONFIG) --libdir) \ -Wl,--start-group -lclangBasic -lclangDriver \tools/build/feature/test-llvm-version.cpp new file +11
-0@@ -0,0 +1,11 @@ +#include <cstdio> +#include "llvm/Config/llvm-config.h" + +#define NUM_VERSION (((LLVM_VERSION_MAJOR) << 16) + (LLVM_VERSION_MINOR << 8) + LLVM_VERSION_PATCH) +#define pass int main() {printf("%x\n", NUM_VERSION); return 0;} + +#if NUM_VERSION >= 0x030900 +pass +#else +# error This LLVM is not tested yet. +#endiftools/build/feature/test-llvm.cpp +5
-0@@ -1,5 +1,10 @@ #include "llvm/Support/ManagedStatic.h" #include "llvm/Support/raw_ostream.h" +#define NUM_VERSION (((LLVM_VERSION_MAJOR) << 16) + (LLVM_VERSION_MINOR << 8) + LLVM_VERSION_PATCH) + +#if NUM_VERSION < 0x030900 +# error "LLVM version too low" +#endif int main() { llvm::errs() << "Hello World!\n";tools/perf/Makefile.config +6
-2@@ -802,12 +802,13 @@ ifdef LIBCLANGLLVM msg := $(warning No g++ found, disable clang and llvm support. Please install g++) else $(call feature_check,llvm) + $(call feature_check,llvm-version) ifneq ($(feature-llvm), 1) - msg := $(warning No libLLVM found, disable clang and llvm support. Please install llvm-dev) + msg := $(warning No libLLVM found, disable clang and llvm support. Please install llvm-dev (>= 3.9.0)) else $(call feature_check,clang) ifneq ($(feature-clang), 1) - msg := $(warning No libclang found, disable clang and llvm support. Please install libclang-dev) + msg := $(warning No libclang found, disable clang and llvm support. Please install libclang-dev (>= 3.9.0)) else CFLAGS += -DHAVE_LIBCLANGLLVM_SUPPORT CXXFLAGS += -DHAVE_LIBCLANGLLVM_SUPPORT -I$(shell $(LLVM_CONFIG) --includedir)@@ -816,6 +817,9 @@ ifdef LIBCLANGLLVM USE_CXX = 1 USE_LLVM = 1 USE_CLANG = 1 + ifneq ($(feature-llvm-version),1) + msg := $(warning This version of llvm is not tested. May cause building error) + endif endif endif endifSignature · 1 line
2.10.1
tip-bot for Wang Nan
merge bot[tip:perf/core] perf build: Check LLVM version in feature check
In reply to Wang Nanperf build: Check LLVM version in feature check
Commit-ID: a940cad331e79cc03d9ae74f56a2c7cb810bdce9 Gitweb: http://git.kernel.org/tip/a940cad331e79cc03d9ae74f56a2c7cb810bdce9 Author: Wang Nan <[email protected]> AuthorDate: Tue, 6 Dec 2016 07:22:30 +0000 Committer: Arnaldo Carvalho de Melo <[email protected]> CommitDate: Tue, 6 Dec 2016 13:21:55 -0300
perf build: Check LLVM version in feature check
Cancel builtin llvm and clang support when LLVM version is less than 3.9.0: following commits uses newer API.
Since Clang/LLVM's API is not guaranteed to be stable, add a test-llvm-version.cpp feature checker, issue warning if LLVM found in compiling environment is not tested yet.
Committer Notes:
Testing it:
Environment:
$ cat /etc/fedora-release Fedora release 25 (Twenty Five) $ rpm -q llvm-devel clang-devel llvm-devel-3.8.0-1.fc25.x86_64 clang-devel-3.8.0-2.fc25.x86_64 $
Before:
$ make -k LIBCLANGLLVM=1 O=/tmp/build/perf -C tools/perf install-bin
make: Entering directory '/home/acme/git/linux/tools/perf'
BUILD: Doing 'make -j4' parallel build
Warning: tools/include/uapi/linux/bpf.h differs from kernel
Warning: tools/arch/arm/include/uapi/asm/kvm.h differs from kernel
INSTALL GTK UI
LINK /tmp/build/perf/perf
/tmp/build/perf/libperf.a(libperf-in.o): In function `perf::createCompilerInvocation(llvm::SmallVector<char const*, 16u>, llvm::StringRef&, clang::DiagnosticsEngine&)':
/home/acme/git/linux/tools/perf/util/c++/clang.cpp:56: undefined reference to `clang::tooling::newInvocation(clang::DiagnosticsEngine*, llvm::SmallVector<char const*, 16u> const&)'
/tmp/build/perf/libperf.a(libperf-in.o): In function `perf::getModuleFromSource(llvm::SmallVector<char const*, 16u>, llvm::StringRef, llvm::IntrusiveRefCntPtr<clang::vfs::FileSystem>)':
/home/acme/git/linux/tools/perf/util/c++/clang.cpp:68: undefined reference to `clang::CompilerInstance::CompilerInstance(std::shared_ptr<clang::PCHContainerOperations>, bool)'
/home/acme/git/linux/tools/perf/util/c++/clang.cpp:69: undefined reference to `clang::CompilerInstance::createDiagnostics(clang::DiagnosticConsumer*, bool)'
<SNIP>After:
Makefile.config:807: No suitable libLLVM found, disabling builtin clang and llvm support. Please install llvm-dev(el) (>= 3.9.0)
Updating the environment to a locally built LLVM 4.0 + clang 3.9 (forgot to git pull, duh) combo, all works as expected, it is properly detected and built into the resulting perf binary.
--- tools/build/feature/Makefile | 8 ++++++-- tools/build/feature/test-llvm-version.cpp | 11 +++++++++++ tools/build/feature/test-llvm.cpp | 5 +++++ tools/perf/Makefile.config | 8 ++++++-- 4 files changed, 28 insertions(+), 4 deletions(-)
tools/build/feature/Makefile +6 -2
@@ -231,14 +231,18 @@ $(OUTPUT)test-jvmti.bin: $(BUILD) $(OUTPUT)test-llvm.bin: - $(BUILDXX) -std=gnu++11 \ + $(BUILDXX) -std=gnu++11 \ -I$(shell $(LLVM_CONFIG) --includedir) \ -L$(shell $(LLVM_CONFIG) --libdir) \ $(shell $(LLVM_CONFIG) --libs Core BPF) \ $(shell $(LLVM_CONFIG) --system-libs) +$(OUTPUT)test-llvm-version.bin: + $(BUILDXX) -std=gnu++11 \ + -I$(shell $(LLVM_CONFIG) --includedir) + $(OUTPUT)test-clang.bin: - $(BUILDXX) -std=gnu++11 \ + $(BUILDXX) -std=gnu++11 \ -I$(shell $(LLVM_CONFIG) --includedir) \ -L$(shell $(LLVM_CONFIG) --libdir) \ -Wl,--start-group -lclangBasic -lclangDriver \
tools/build/feature/test-llvm-version.cpp new file +11 -0
@@ -0,0 +1,11 @@ +#include <cstdio> +#include "llvm/Config/llvm-config.h" + +#define NUM_VERSION (((LLVM_VERSION_MAJOR) << 16) + (LLVM_VERSION_MINOR << 8) + LLVM_VERSION_PATCH) +#define pass int main() {printf("%x\n", NUM_VERSION); return 0;} + +#if NUM_VERSION >= 0x030900 +pass +#else +# error This LLVM is not tested yet. +#endif
tools/build/feature/test-llvm.cpp +5 -0
@@ -1,5 +1,10 @@ #include "llvm/Support/ManagedStatic.h" #include "llvm/Support/raw_ostream.h" +#define NUM_VERSION (((LLVM_VERSION_MAJOR) << 16) + (LLVM_VERSION_MINOR << 8) + LLVM_VERSION_PATCH) + +#if NUM_VERSION < 0x030900 +# error "LLVM version too low" +#endif int main() { llvm::errs() << "Hello World!\n";
tools/perf/Makefile.config +6 -2
@@ -802,12 +802,13 @@ ifdef LIBCLANGLLVM msg := $(warning No g++ found, disable clang and llvm support. Please install g++) else $(call feature_check,llvm) + $(call feature_check,llvm-version) ifneq ($(feature-llvm), 1) - msg := $(warning No libLLVM found, disable clang and llvm support. Please install llvm-dev) + msg := $(warning No suitable libLLVM found, disabling builtin clang and LLVM support. Please install llvm-dev(el) (>= 3.9.0)) else $(call feature_check,clang) ifneq ($(feature-clang), 1) - msg := $(warning No libclang found, disable clang and llvm support. Please install libclang-dev) + msg := $(warning No suitable libclang found, disabling builtin clang and LLVM support. Please install libclang-dev(el) (>= 3.9.0)) else CFLAGS += -DHAVE_LIBCLANGLLVM_SUPPORT CXXFLAGS += -DHAVE_LIBCLANGLLVM_SUPPORT -I$(shell $(LLVM_CONFIG) --includedir)
@@ -816,6 +817,9 @@ ifdef LIBCLANGLLVM USE_CXX = 1 USE_LLVM = 1 USE_CLANG = 1 + ifneq ($(feature-llvm-version),1) + msg := $(warning This version of LLVM is not tested. May cause build errors) + endif endif endif endif
PATCH v4 2/18 perf build: Support dynamic linking clang libraries
In reply to Wang NanStatical linking result a very large perf executable. This patch makes perf link clang libraries dynamically by using '-lclangBasic' style linking option. If dynamic clang libraries are detected, gcc will use them by default.
Test result:
(Build clang/llvm dynamically by setting -DBUILD_SHARED_LIBS=ON
in its cmake configuration.)
$ size ~/perf text data bss dec hex filename 4223234 754544 23956048 28933826 1b97ec2 /home/wn/perf
$ strip ~/perf $ ls -sh ~/perf 4.8M /home/wn/perf
Compare with statical linking:
$ ls -sh ~/perf 969M /home/wn/perf $ strip ~/perf $ ls -sh ~/perf 52M /home/wn/perf
--- tools/perf/Makefile.perf | 20 ++++++++++++++++++-- 1 file changed, 18 insertions(+), 2 deletions(-)
tools/perf/Makefile.perf +18 -2
@@ -329,9 +329,25 @@ endif LIBS = -Wl,--whole-archive $(PERFLIBS) -Wl,--no-whole-archive -Wl,--start-group $(EXTLIBS) -Wl,--end-group ifeq ($(USE_CLANG), 1) + # Only partial of the required clang libraries are listed. + # + # In case of dynamical linking, if clang is built and installed + # correctly, ld can infer the full list automatically. However, + # I observed a potential bug in 3.9.0 that four libraries + # (ASTMatchers, Format Rewrite and ToolingCore) are lost. + # + # In case of statical linking, providing the full list causes + # linking time dramatically increases because of --{start,end}-group. + # + # If linking error, replace CLANGLIBS_LIST with the full list + # and try again. + # + # The full list should be: + # + # Basic CodeGen Frontend Tooling AST Lex Driver Edit Parse Sema + # Serialization ASTMatchers Format Rewrite ToolingCore Analysis CLANGLIBS_LIST = AST Basic CodeGen Driver Frontend Lex Tooling Edit Sema Analysis Parse Serialization - LIBCLANG = $(foreach l,$(CLANGLIBS_LIST),$(wildcard $(shell $(LLVM_CONFIG) --libdir)/libclang$(l).a)) - LIBS += -Wl,--start-group $(LIBCLANG) -Wl,--end-group + LIBS += -L$(shell $(LLVM_CONFIG) --libdir) -Wl,--start-group $(foreach l,$(CLANGLIBS_LIST),-lclang$(l)) -Wl,--end-group endif ifeq ($(USE_LLVM), 1)
Signature · 1 line
2.10.1
PATCH v4 3/18 perf clang: Cleanup clang options
In reply to Wang NanFollow Alexei's suggestion, remove "-ferror-limit=19", "-fmessage-length=127", "-vectorize-loops" and "-vectorize-slp" clang options: they are meaningless. Add comment for "-Wno-unused-value" and "-Wno-pointer-sign".
--- tools/perf/util/c++/clang.cpp | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-)
tools/perf/util/c++/clang.cpp +1 -4
@@ -41,13 +41,10 @@ createCompilerInvocation(llvm::opt::ArgStringList CFlags, StringRef& Path, "-cc1", "-triple", "bpf-pc-linux", "-fsyntax-only", - "-ferror-limit", "19", - "-fmessage-length", "127", "-O2", "-nostdsysteminc", "-nobuiltininc", - "-vectorize-loops", - "-vectorize-slp", + /* Suppress warnings when using kernel headers */ "-Wno-unused-value", "-Wno-pointer-sign", "-x", "c"};
Signature · 1 line
2.10.1
PATCH v4 4/18 perf clang: Pass full path to builtin clang
In reply to Wang NanIf clang changes its working directory, relative path passed to perf_clang__compile_bpf() becomes invalid. Before running clang, convert it to absolute path so file can be found even working directory is changed.
--- tools/perf/util/c++/clang.cpp | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
tools/perf/util/c++/clang.cpp +6 -2
@@ -13,6 +13,7 @@ #include "clang/Frontend/CompilerInstance.h" #include "clang/Frontend/TextDiagnosticPrinter.h" #include "clang/Tooling/Tooling.h" +#include "llvm/ADT/SmallString.h" #include "llvm/IR/LegacyPassManager.h" #include "llvm/IR/Module.h" #include "llvm/Option/Option.h"
@@ -161,7 +162,7 @@ void perf_clang__cleanup(void) llvm::llvm_shutdown(); } -int perf_clang__compile_bpf(const char *filename, +int perf_clang__compile_bpf(const char *_filename, void **p_obj_buf, size_t *p_obj_buf_sz) {
@@ -170,8 +171,11 @@ int perf_clang__compile_bpf(const char *filename, if (!p_obj_buf || !p_obj_buf_sz) return -EINVAL; + llvm::SmallString<PATH_MAX> FileName(_filename); + llvm::sys::fs::make_absolute(FileName); + llvm::opt::ArgStringList CFlags; - auto M = getModuleFromSource(std::move(CFlags), filename); + auto M = getModuleFromSource(std::move(CFlags), FileName.data()); if (!M) return -EINVAL; auto O = getBPFObjectFromModule(&*M);
Signature · 1 line
2.10.1
PATCH v4 5/18 perf clang: Pass CFLAGS to builtin clang
In reply to Wang NanPass -DLINUX_VERSION_CODE, -D__NR_CPUS__, llvm.clang-opt config options and CFLAGS detected by kbuild detector to builtin clang so BPF scripts can use kernel headers and user defined options like external clang compiler.
Test:
# perf record -v --dry-run -e tools/perf/tests/bpf-script-test-kbuild.c ls 2>&1 | grep built bpf: builtin compiling successful
Committer notes:
Before installing the required clang/llvm devel files to have it builtin:
# perf record -v --dry-run -e tools/perf/tests/bpf-script-test-kbuild.c ls 2>&1 | grep built bpf: builtin compiling failed: -95, try external compiler
I.e. it falls back to using the external compiler.
--- tools/perf/util/c++/clang.cpp | 105 ++++++++++++++++++++++++++++++++++++++++-- tools/perf/util/llvm-utils.h | 9 +++- 2 files changed, 109 insertions(+), 5 deletions(-)
tools/perf/util/c++/clang.cpp +101 -4
@@ -27,6 +27,8 @@ #include "clang.h" #include "clang-c.h" +#include "llvm-utils.h" +#include "util-cxx.h" namespace perf {
@@ -144,6 +146,101 @@ getBPFObjectFromModule(llvm::Module *Module) return std::move(Buffer); } +class ClangOptions { + llvm::SmallString<PATH_MAX> FileName; + llvm::SmallString<64> KVerDef; + llvm::SmallString<64> NRCpusDef; + char *kbuild_dir; + char *kbuild_include_opts; + char *clang_opt; +public: + ClangOptions(const char *filename) : FileName(filename), + KVerDef(""), + NRCpusDef(""), + kbuild_dir(NULL), + kbuild_include_opts(NULL), + clang_opt(NULL) + { + llvm::sys::fs::make_absolute(FileName); + + unsigned int kver; + if (!fetch_kernel_version(&kver, NULL, 0)) + KVerDef = "-DLINUX_VERSION_CODE=" + std::to_string(kver); + + int nr_cpus = llvm__get_nr_cpus(); + if (nr_cpus > 0) + NRCpusDef = "-D__NR_CPUS__=" + std::to_string(nr_cpus); + + if (llvm_param.clang_opt) + clang_opt = strdup(llvm_param.clang_opt); + + llvm__get_kbuild_opts(&kbuild_dir, &kbuild_include_opts); + if (!kbuild_dir || !kbuild_include_opts) { + free(kbuild_dir); + free(kbuild_include_opts); + kbuild_dir = kbuild_include_opts = NULL; + } + } + + ~ClangOptions() + { + free(kbuild_dir); + free(kbuild_include_opts); + free(clang_opt); + } + + static void fillCFlagsFromString(opt::ArgStringList &CFlags, char *s, bool check = false) + { + if (!s) + return; + + SmallVector<StringRef, 0> Terms; + StringRef Opts(s); + Opts.split(Terms, ' '); + + for (auto i = Terms.begin(); i != Terms.end(); i++) + s[i->end() - Opts.begin()] = '\0'; + + for (auto i = Terms.begin(); i != Terms.end(); i++) { + if (!check) { + CFlags.push_back(i->begin()); + continue; + } + + if (i->startswith("-I")) + CFlags.push_back(i->begin()); + else if (i->startswith("-D")) + CFlags.push_back(i->begin()); + else if (*i == "-include") { + CFlags.push_back((i++)->begin()); + /* Let clang report this error */ + if (i == Terms.end()) + break; + CFlags.push_back(i->begin()); + } + } + } + + void getCFlags(opt::ArgStringList &CFlags) + { + CFlags.push_back(KVerDef.c_str()); + CFlags.push_back(NRCpusDef.c_str()); + + fillCFlagsFromString(CFlags, clang_opt); + fillCFlagsFromString(CFlags, kbuild_include_opts, true); + + if (kbuild_dir) { + CFlags.push_back("-working-directory"); + CFlags.push_back(kbuild_dir); + } + } + + const char *getFileName(void) + { + return FileName.c_str(); + } +}; + } extern "C" {
@@ -171,11 +268,11 @@ int perf_clang__compile_bpf(const char *_filename, if (!p_obj_buf || !p_obj_buf_sz) return -EINVAL; - llvm::SmallString<PATH_MAX> FileName(_filename); - llvm::sys::fs::make_absolute(FileName); - + ClangOptions Opts(_filename); llvm::opt::ArgStringList CFlags; - auto M = getModuleFromSource(std::move(CFlags), FileName.data()); + + Opts.getCFlags(CFlags); + auto M = getModuleFromSource(std::move(CFlags), Opts.getFileName()); if (!M) return -EINVAL; auto O = getBPFObjectFromModule(&*M);
tools/perf/util/llvm-utils.h +8 -1
@@ -5,7 +5,11 @@ #ifndef __LLVM_UTILS_H #define __LLVM_UTILS_H -#include "debug.h" +#ifdef __cplusplus +extern "C" { +#endif + +#include <errno.h> struct llvm_param { /* Path of clang executable */
@@ -56,4 +60,7 @@ void llvm__get_kbuild_opts(char **kbuild_dir, char **kbuild_include_opts);int llvm__get_nr_cpus (void );void llvm__dump_obj (const char *path,void *obj_buf,size_t size); +#ifdef __cplusplus +} +#endif #endif
Signature · 1 line
2.10.1
PATCH v4 6/18 perf clang jit: Wrap llvm::Module using PerfModule
In reply to Wang NanUse PerfModule wrap llvm::Module and return perf::PerfModule in APIs to replace llvm::Module. Following commits are going to add new functions to PerfModule.
getBPFObjectFromModule is merged to a method of perf::PerfModule.
--- tools/perf/util/c++/clang-test.cpp | 10 +++++----- tools/perf/util/c++/clang.cpp | 20 +++++++++++++------- tools/perf/util/c++/clang.h | 22 ++++++++++++++++------ 3 files changed, 34 insertions(+), 18 deletions(-)
tools/perf/util/c++/clang-test.cpp +5 -5
@@ -13,18 +13,18 @@ public: ~perf_clang_scope() {perf_clang__cleanup();} }; -static std::unique_ptr<llvm::Module> +static std::unique_ptr<perf::PerfModule> __test__clang_to_IR(void) { unsigned int kernel_version; if (fetch_kernel_version(&kernel_version, NULL, 0)) - return std::unique_ptr<llvm::Module>(nullptr); + return std::unique_ptr<perf::PerfModule>(nullptr); std::string cflag_kver("-DLINUX_VERSION_CODE=" + std::to_string(kernel_version)); - std::unique_ptr<llvm::Module> M = + std::unique_ptr<perf::PerfModule> M = perf::getModuleFromSource({cflag_kver.c_str()}, "perf-test.c", test_llvm__bpf_base_prog);
@@ -39,7 +39,7 @@ int test__clang_to_IR(void) auto M = __test__clang_to_IR(); if (!M) return -1; - for (llvm::Function& F : *M) + for (llvm::Function& F : *(M->getModule())) if (F.getName() == "bpf_func__SyS_epoll_wait") return 0; return -1;
@@ -53,7 +53,7 @@ int test__clang_to_obj(void) if (!M) return -1; - auto Buffer = perf::getBPFObjectFromModule(&*M); + auto Buffer = M->toBPFObject(); if (!Buffer) return -1; return 0;
tools/perf/util/c++/clang.cpp +13 -7
@@ -61,7 +61,7 @@ createCompilerInvocation(llvm::opt::ArgStringList CFlags, StringRef& Path, return CI; } -static std::unique_ptr<llvm::Module> +static std::unique_ptr<PerfModule> getModuleFromSource(llvm::opt::ArgStringList CFlags, StringRef Path, IntrusiveRefCntPtr<vfs::FileSystem> VFS) {
@@ -77,12 +77,12 @@ getModuleFromSource(llvm::opt::ArgStringList CFlags, std::unique_ptr<CodeGenAction> Act(new EmitLLVMOnlyAction(&*LLVMCtx)); if (!Clang.ExecuteAction(*Act)) - return std::unique_ptr<llvm::Module>(nullptr); + return std::unique_ptr<PerfModule>(nullptr); - return Act->takeModule(); + return std::unique_ptr<PerfModule>(new PerfModule(std::move(Act->takeModule()))); } -std::unique_ptr<llvm::Module> +std::unique_ptr<PerfModule> getModuleFromSource(llvm::opt::ArgStringList CFlags, StringRef Name, StringRef Content) {
@@ -103,15 +103,21 @@ getModuleFromSource(llvm::opt::ArgStringList CFlags, return getModuleFromSource(std::move(CFlags), Name, OverlayFS); } -std::unique_ptr<llvm::Module> +std::unique_ptr<PerfModule> getModuleFromSource(llvm::opt::ArgStringList CFlags, StringRef Path) { IntrusiveRefCntPtr<vfs::FileSystem> VFS(vfs::getRealFileSystem()); return getModuleFromSource(std::move(CFlags), Path, VFS); } +PerfModule::PerfModule(std::unique_ptr<llvm::Module>&& M) : Module(std::move(M)) +{ + +} + + std::unique_ptr<llvm::SmallVectorImpl<char>> -getBPFObjectFromModule(llvm::Module *Module) +PerfModule::toBPFObject(void) { using namespace llvm;
@@ -275,7 +281,7 @@ int perf_clang__compile_bpf(const char *_filename, auto M = getModuleFromSource(std::move(CFlags), Opts.getFileName()); if (!M) return -EINVAL; - auto O = getBPFObjectFromModule(&*M); + auto O = M->toBPFObject(); if (!O) return -EINVAL;
tools/perf/util/c++/clang.h +16 -6
@@ -11,16 +11,26 @@ namespace perf { using namespace llvm; -std::unique_ptr<Module> +class PerfModule { +private: + std::unique_ptr<llvm::Module> Module; +public: + inline llvm::Module *getModule(void) + { +return Module.get (); + } + + PerfModule(std::unique_ptr<llvm::Module>&& M); + + std::unique_ptr<llvm::SmallVectorImpl<char>> toBPFObject(void); +}; + +std::unique_ptr<PerfModule> getModuleFromSource(opt::ArgStringList CFlags, StringRef Name, StringRef Content); -std::unique_ptr<Module> +std::unique_ptr<PerfModule> getModuleFromSource(opt::ArgStringList CFlags, StringRef Path); - -std::unique_ptr<llvm::SmallVectorImpl<char>> -getBPFObjectFromModule(llvm::Module *Module); - }#endif
Signature · 1 line
2.10.1
PATCH v4 7/18 perf clang jit: Insignt BPF and JIT functions in a Module
In reply to Wang NanIdentify BPF functions, JIT functions and maps during init. Functions in section starting with "perfhook:" are JIT functions. They will be JIT compiled and hooked at perfhooks.
During init of PerfModule, mark JIT functions as AvailableExternallyLinkage. LLVM skips functions with linkage like this so they won't be compiled into BPF objects.
--- tools/perf/util/c++/clang.cpp | 38 ++++++++++++++++++++++++++++++++++++++ tools/perf/util/c++/clang.h | 7 +++++++ 2 files changed, 45 insertions(+)
tools/perf/util/c++/clang.cpp +38 -0
@@ -112,15 +112,53 @@ getModuleFromSource(llvm::opt::ArgStringList CFlags, StringRef Path) PerfModule::PerfModule(std::unique_ptr<llvm::Module>&& M) : Module(std::move(M)) { + for (llvm::Function& F : *Module) { + if (F.getLinkage() != llvm::GlobalValue::ExternalLinkage) + continue; + + if (StringRef(F.getSection()).startswith("perfhook:")) + JITFunctions.insert(&F); + else + BPFFunctions.insert(&F); + } + for (auto V = Module->global_begin(); V != Module->global_end(); V++) { + llvm::GlobalVariable *GV = &*V; + if (StringRef(GV->getSection()) == llvm::StringRef("maps")) + Maps.insert(GV); + } } +void PerfModule::prepareBPF(void) +{ + /* + * setLinkage(AvailableExternallyLinkage) causes LLVM + * blindly skip the function. They still exist in Module + * so we can bring them back by resetting linkage to + * ExternalLinkage. + */ + for (llvm::Function *F : JITFunctions) + F->setLinkage(llvm::GlobalValue::AvailableExternallyLinkage); + for (llvm::Function *F : BPFFunctions) + F->setLinkage(llvm::GlobalValue::ExternalLinkage); + +} + +void PerfModule::prepareJIT(void) +{ + for (llvm::Function *F : BPFFunctions) + F->setLinkage(llvm::GlobalValue::AvailableExternallyLinkage); + for (llvm::Function *F : JITFunctions) + F->setLinkage(llvm::GlobalValue::ExternalLinkage); + +} std::unique_ptr<llvm::SmallVectorImpl<char>> PerfModule::toBPFObject(void) { using namespace llvm; + prepareBPF(); std::string TargetTriple("bpf-pc-linux"); std::string Error; const Target* Target = TargetRegistry::lookupTarget(TargetTriple, Error);
tools/perf/util/c++/clang.h +7 -0
@@ -6,6 +6,7 @@#include "llvm/IR/Module.h" #include "llvm/Option/Option.h" #include <memory> +#include <set> namespace perf {
@@ -14,6 +15,12 @@ using namespace llvm;class PerfModule { private: std::unique_ptr<llvm::Module> Module; + + std::set<llvm::GlobalVariable *> Maps; + std::set<llvm::Function *> BPFFunctions; + std::set<llvm::Function *> JITFunctions; +void prepareBPF (void ); +void prepareJIT (void ); public: inline llvm::Module *getModule(void) {
Signature · 1 line
2.10.1
PATCH v4 8/18 perf clang jit: add PerfModule::doJIT to JIT perfhook functions
In reply to Wang NanPerfModule::doJIT JIT compile perfhook functions and saves result into a map. Add a test case for it.
At this stage perfhook functions can do no useful things because they can't invoke external functions and can't return value. Following commits are going to make improvment.
Don't hook functions right after jitted because bpf_object is unavailable during jitting but it should be the context of jitted functions.
--- tools/perf/tests/bpf-script-example.c | 8 ++++ tools/perf/tests/clang.c | 4 ++ tools/perf/util/c++/clang-c.h | 2 + tools/perf/util/c++/clang-test.cpp | 32 +++++++++++++++- tools/perf/util/c++/clang.cpp | 71 +++++++++++++++++++++++++++++++++++ tools/perf/util/c++/clang.h | 13 +++++++ 6 files changed, 128 insertions(+), 2 deletions(-)
tools/perf/tests/bpf-script-example.c +8 -0
@@ -46,3 +46,11 @@ int bpf_func__SyS_epoll_wait(void *ctx) } char _license[] SEC("license") = "GPL"; int _version SEC("version") = LINUX_VERSION_CODE; + +#ifdef TEST_PERF_HOOK +SEC("perfhook:test") +void hook_test (void ) +{ +return ; +} +#endif
tools/perf/tests/clang.c +4 -0
@@ -16,6 +16,10 @@ static struct { .func = test__clang_to_obj, .desc = "builtin clang compile C source to ELF object", }, + { + .func = test__clang_jit, + .desc = "builtin clang compile mixed BPF and native code", + },#endif };
tools/perf/util/c++/clang-c.h +2 -0
@@ -14,6 +14,7 @@ extern void perf_clang__cleanup(void);extern int test__clang_to_IR (void );extern int test__clang_to_obj (void ); +extern int test__clang_jit (void ); extern int perf_clang__compile_bpf(const char *filename, void **p_obj_buf,
@@ -26,6 +27,7 @@ static inline void perf_clang__cleanup(void) { }static inline int test__clang_to_IR (void ) {return -1 ; }static inline int test__clang_to_obj (void ) {return -1 ;} +static inline int test__clang_jit (void ) {return -1 ;} static inline int perf_clang__compile_bpf(const char *filename __maybe_unused,
tools/perf/util/c++/clang-test.cpp +30 -2
@@ -5,6 +5,7 @@ #include <util-cxx.h> #include <tests/llvm.h> +#include <perf-hooks.h> #include <string> class perf_clang_scope {
@@ -14,7 +15,7 @@ public: }; static std::unique_ptr<perf::PerfModule> -__test__clang_to_IR(void) +__test__clang_to_IR(bool perfhook) { unsigned int kernel_version;
@@ -23,14 +24,22 @@ __test__clang_to_IR(void) std::string cflag_kver("-DLINUX_VERSION_CODE=" + std::to_string(kernel_version)); + std::string cflag_perfhook(perfhook ? "-DTEST_PERF_HOOK=1" : ""); std::unique_ptr<perf::PerfModule> M = - perf::getModuleFromSource({cflag_kver.c_str()}, + perf::getModuleFromSource({cflag_kver.c_str(), + cflag_perfhook.c_str()}, "perf-test.c", test_llvm__bpf_base_prog); return M; } +static std::unique_ptr<perf::PerfModule> +__test__clang_to_IR(void) +{ + return __test__clang_to_IR(false); +} + extern "C" { int test__clang_to_IR(void) {
@@ -59,4 +68,23 @@ int test__clang_to_obj(void) return 0; } +int test__clang_jit(void) +{ + perf_clang_scope _scope; + + auto M = __test__clang_to_IR(true); + if (!M) + return -1; + + if (M->doJIT()) + return -1; + + std::unique_ptr<perf::PerfModule::HookMap> hooks(M->copyJITResult()); + for (auto i : *hooks) + perf_hooks__set_hook(i.first.c_str(), i.second, NULL); + + perf_hooks__invoke_test(); + return 0; +} + }
tools/perf/util/c++/clang.cpp +71 -0
103 lines not on this page · raw message
tools/perf/util/c++/clang.h +13 -0
@@ -7,18 +7,26 @@#include "llvm/Option/Option.h" #include <memory> #include <set> +#include <map> + +#include "util/perf-hooks.h" namespace perf { using namespace llvm;class PerfModule { +public: + typedef std::map<std::string, perf_hook_func_t> HookMap; private: std::unique_ptr<llvm::Module> Module; std::set<llvm::GlobalVariable *> Maps; std::set<llvm::Function *> BPFFunctions; std::set<llvm::Function *> JITFunctions; + +HookMap JITResult; +void prepareBPF (void );void prepareJIT (void ); public:
@@ -26,10 +34,15 @@ class PerfModule { {return Module.get (); } +inline HookMap *copyJITResult (void ) + { + return new HookMap(JITResult); + } PerfModule(std::unique_ptr<llvm::Module>&& M); std::unique_ptr<llvm::SmallVectorImpl<char>> toBPFObject(void); +int doJIT (void ); }; std::unique_ptr<PerfModule>
Signature · 1 line
2.10.1
PATCH v4 9/18 perf clang jit: Export functions for jitted code
In reply to Wang NanAfter this patch functions attached on perf hooks is allowed to invoke external functions. Add a testcase for this feature.
--- tools/perf/tests/Build | 2 +- tools/perf/tests/bpf-script-example.c | 4 ++++ tools/perf/util/c++/clang-c.h | 2 ++ tools/perf/util/c++/clang-test.cpp | 9 +++++++++ tools/perf/util/c++/clang.cpp | 17 ++++++++++++++++- 5 files changed, 32 insertions(+), 2 deletions(-)
tools/perf/tests/Build +1 -1
@@ -49,7 +49,7 @@ $(OUTPUT)tests/llvm-src-base.c: tests/bpf-script-example.c tests/Build $(call rule_mkdir) $(Q)echo '#include <tests/llvm.h>' > $@ $(Q)echo 'const char test_llvm__bpf_base_prog[] =' >> $@ - $(Q)sed -e 's/"/\\"/g' -e 's/\(.*\)/"\1\\n"/g' $< >> $@ + $(Q)sed -e 's/\\/\\\\/g' -e 's/"/\\"/g' -e 's/\(.*\)/"\1\\n"/g' $< >> $@ $(Q)echo ';' >> $@ $(OUTPUT)tests/llvm-src-kbuild.c: tests/bpf-script-test-kbuild.c tests/Build
tools/perf/tests/bpf-script-example.c +4 -0
@@ -48,9 +48,13 @@ char _license[] SEC("license") = "GPL"; int _version SEC("version") = LINUX_VERSION_CODE;#ifdef TEST_PERF_HOOK +extern int printf (const char *fmt, ...); +extern void test__clang_callback (int x); SEC("perfhook:test")void hook_test (void ) { +printf ("Hello, hook_test\n" ); +test__clang_callback (1234 );return ; }#endif
tools/perf/util/c++/clang-c.h +2 -0
@@ -16,6 +16,8 @@ extern int test__clang_to_IR(void);extern int test__clang_to_obj (void );extern int test__clang_jit (void ); +extern void test__clang_callback (int x); +extern int perf_clang__compile_bpf (const char *filename,void **p_obj_buf,size_t *p_obj_buf_sz);
tools/perf/util/c++/clang-test.cpp +9 -0
@@ -68,6 +68,13 @@ int test__clang_to_obj(void) return 0; } +static int callback_flag; + +void test__clang_callback(int x) +{ + callback_flag = x; +} + int test__clang_jit(void) { perf_clang_scope _scope;
@@ -84,6 +91,8 @@ int test__clang_jit(void) perf_hooks__set_hook(i.first.c_str(), i.second, NULL); perf_hooks__invoke_test(); + if (callback_flag != 1234) + return -1; return 0; }
tools/perf/util/c++/clang.cpp +16 -1
@@ -30,6 +30,8 @@ #include "llvm/Target/TargetOptions.h" #include <memory> #include <vector> +#include <set> +#include <tuple> #include "clang.h" #include "clang-c.h"
@@ -197,6 +199,15 @@ PerfModule::toBPFObject(void) return std::move(Buffer); } +static std::map<const std::string, const void *> exported_funcs = +{ +#define EXPORT(f) {#f, (const void *)&f} + EXPORT(test__clang_callback), + EXPORT(printf), + EXPORT(puts), +#undef EXPORT +}; + /* * Use a global memory manager so allocated code and data won't be released * when object destroy.
@@ -223,7 +234,11 @@ int PerfModule::doJIT(void) auto Resolver = createLambdaResolver( [](const std::string &Name) { - return RuntimeDyld::SymbolInfo(nullptr); + auto i = exported_funcs.find(Name); + if (i == exported_funcs.end()) + return RuntimeDyld::SymbolInfo(nullptr); + return RuntimeDyld::SymbolInfo((uint64_t)(i->second), + JITSymbolFlags::Exported); }, [](const std::string &Name) { return RuntimeDyld::SymbolInfo(nullptr);
Signature · 1 line
2.10.1
PATCH v4 10/18 perf clang jit: Actually JIT and hook in bpf loader
In reply to Wang NanMakes perf_clang__compile_bpf() actually uses clang jit to compile perf hooks. Returns a map through perf_clang__compile_bpf(), and set hooks after bpf_object is created.
After this path jitting takes actions for bpf loader. For example:
$ cat ./test.c
/******************************************************/
#define SEC(name) __attribute__((section(name), used))
SEC("dofork=_do_fork")
int dofork(void *ctx)
{
return 0;
}
extern int printf(const char *fmt, ...);
SEC("perfhook:record_start")
void record_start(void)
{
printf("Welcom to perf record\n");
}
SEC("perfhook:record_end")
void record_end(void)
{
printf("Goodbye, perf record\n");
}
char _license[] SEC("license") = "GPL";
int _version SEC("version") = LINUX_VERSION_CODE;
/******************************************************/
$ perf record -e ./test.c sleep 1
Welcom to perf record
[ perf record: Woken up 1 times to write data ]
Goodbye, perf record
[ perf record: Captured and wrote 0.014 MB perf.data ]--- tools/perf/util/bpf-loader.c | 11 ++++++++++- tools/perf/util/c++/clang-c.h | 18 ++++++++++++++++-- tools/perf/util/c++/clang.cpp | 28 +++++++++++++++++++++++++++- 3 files changed, 53 insertions(+), 4 deletions(-)
tools/perf/util/bpf-loader.c +10 -1
@@ -85,9 +85,11 @@ struct bpf_object *bpf__prepare_load(const char *filename, bool source)int err;void *obj_buf;size_t obj_buf_sz; +jitted_funcs_map_t jitted_funcs_map;perf_clang__init (); - err =perf_clang__compile_bpf (filename, &obj_buf, &obj_buf_sz); + err =perf_clang__compile_bpf (filename, &obj_buf, + &obj_buf_sz, &jitted_funcs_map);perf_clang__cleanup ();if (err) {pr_warning ("bpf: builtin compilation failed: %d, try external compiler\n" , err);
@@ -101,6 +103,13 @@ struct bpf_object *bpf__prepare_load(const char *filename, bool source)if (!IS_ERR (obj) && llvm_param.dump_obj)llvm__dump_obj (filename, obj_buf, obj_buf_sz); +/* +* Call perf_clang__hook_jitted_func even IS_ERR(obj) to make sure +* the C++ map pointer is deleted. +*/ +if (jitted_funcs_map) +perf_clang__hook_jitted_func (jitted_funcs_map, obj,IS_ERR (obj)); +free (obj_buf); }else obj =bpf_object__open (filename);
tools/perf/util/c++/clang-c.h +16 -2
@@ -8,6 +8,7 @@extern "C" {#endif +typedef void *jitted_funcs_map_t ;#ifdef HAVE_LIBCLANGLLVM_SUPPORT extern void perf_clang__init (void );extern void perf_clang__cleanup (void );
@@ -20,7 +21,11 @@ extern void test__clang_callback(int x);extern int perf_clang__compile_bpf (const char *filename,void **p_obj_buf, -size_t *p_obj_buf_sz); +size_t *p_obj_buf_sz, +jitted_funcs_map_t *p_funcs_map); + +extern int +perf_clang__hook_jitted_func (jitted_funcs_map_t map,void *ctx,bool is_err);#else
@@ -34,7 +39,16 @@ static inline int test__clang_jit(void) { return -1;}static inline int perf_clang__compile_bpf (const char *filename__maybe_unused ,void **p_obj_buf__maybe_unused , -size_t *p_obj_buf_sz__maybe_unused ) +size_t *p_obj_buf_sz__maybe_unused , +jitted_funcs_map_t *p_funcs_map__maybe_unused ) +{ +return -ENOTSUP ; +} + +static inline int +perf_clang__hook_jitted_func (jitted_funcs_map_t map__maybe_unused , +void *ctx__maybe_unused , +bool is_err__maybe_unused ) {return -ENOTSUP ; }
tools/perf/util/c++/clang.cpp +27 -1
@@ -391,7 +391,8 @@ void perf_clang__cleanup(void) int perf_clang__compile_bpf(const char *_filename, void **p_obj_buf, - size_t *p_obj_buf_sz) + size_t *p_obj_buf_sz, + jitted_funcs_map_t *p_funcs_map) { using namespace perf;
@@ -418,6 +419,31 @@ int perf_clang__compile_bpf(const char *_filename, memcpy(buffer, O->data(), size); *p_obj_buf = buffer; *p_obj_buf_sz = size; + + if (M->doJIT()) + return -1; + + if (p_funcs_map) + *p_funcs_map = (jitted_funcs_map_t)(M->copyJITResult()); + return 0; +} + +int perf_clang__hook_jitted_func(jitted_funcs_map_t map, void *ctx, bool is_err) +{ + std::unique_ptr<perf::PerfModule::HookMap> + hook_map((perf::PerfModule::HookMap *)map); + + /* Do nothing but ensure map is deleted */ + if (is_err) + return -1; + + for (auto i : *hook_map) { + const char *hook_name = i.first.c_str(); + perf_hook_func_t hook_func = i.second; + + if (perf_hooks__set_hook(hook_name, hook_func, ctx)) + return -1; + } return 0; } }
Signature · 1 line
2.10.1
PATCH v4 11/18 perf clang jit: Collect the lowest address in maps section as map_base
In reply to Wang NanDuring jitting, find the lowest address in maps section and store its value to _map_base. Pass its value out through perf_clang__compile_bpf(). map_base is useful for jitted functions accessing BPF maps.
--- tools/perf/util/bpf-loader.c | 39 +++++++++++++++++++++++++++++++++++++-- tools/perf/util/c++/clang-c.h | 6 ++++-- tools/perf/util/c++/clang.cpp | 15 +++++++++++++-- tools/perf/util/c++/clang.h | 5 +++++ 4 files changed, 59 insertions(+), 6 deletions(-)
tools/perf/util/bpf-loader.c +37 -2
@@ -47,6 +47,10 @@ struct bpf_prog_priv {int *type_mapping; }; +struct bpf_obj_priv { +void *map_base; +}; +static bool libbpf_initialized; struct bpf_object *
@@ -70,9 +74,20 @@ bpf__prepare_load_buffer(void *obj_buf, size_t obj_buf_sz, const char *name)return obj; } +static void +clear_obj_priv (struct bpf_object *obj__maybe_unused , +void *_priv) +{ +struct bpf_obj_priv *priv = _priv; + +free (priv); +} +struct bpf_object *bpf__prepare_load (const char *filename,bool source) {struct bpf_object *obj; +void *map_base =NULL ; +int err;if (!libbpf_initialized) { libbpf_set_print(libbpf_warning,
@@ -82,14 +97,14 @@ struct bpf_object *bpf__prepare_load(const char *filename, bool source) }if (source) { -int err;void *obj_buf;size_t obj_buf_sz;jitted_funcs_map_t jitted_funcs_map;perf_clang__init (); err =perf_clang__compile_bpf (filename, &obj_buf, - &obj_buf_sz, &jitted_funcs_map); + &obj_buf_sz, &jitted_funcs_map, + &map_base);perf_clang__cleanup ();if (err) {pr_warning ("bpf: builtin compilation failed: %d, try external compiler\n" , err);
@@ -119,7 +134,27 @@ struct bpf_object *bpf__prepare_load(const char *filename, bool source)return obj; } +if (map_base) { +struct bpf_obj_priv *priv =calloc (sizeof (*priv),1 ); + +if (!priv) { +pr_debug ("bpf: failed to alloc priv for object\n" ); + err = -ENOMEM ; + goto errout; + } + priv->map_base = map_base; + + err =bpf_object__set_priv (obj, priv, clear_obj_priv); +if (err) { +pr_debug ("Failed to set priv for object '%s'\n" , filename); + goto errout; + } + } +return obj; +errout: +bpf_object__close (obj); +return ERR_PTR (err); }void bpf__clear (void )
tools/perf/util/c++/clang-c.h +4 -2
@@ -22,7 +22,8 @@ extern void test__clang_callback(int x);extern int perf_clang__compile_bpf (const char *filename,void **p_obj_buf,size_t *p_obj_buf_sz, -jitted_funcs_map_t *p_funcs_map); +jitted_funcs_map_t *p_funcs_map, +void **p_map_base);extern int perf_clang__hook_jitted_func (jitted_funcs_map_t map,void *ctx,bool is_err);
@@ -40,7 +41,8 @@ static inline int perf_clang__compile_bpf(const char *filename __maybe_unused, void **p_obj_buf __maybe_unused, size_t *p_obj_buf_sz __maybe_unused, - jitted_funcs_map_t *p_funcs_map __maybe_unused) + jitted_funcs_map_t *p_funcs_map __maybe_unused, + void **p_map_base __maybe_unused) { return -ENOTSUP; }
tools/perf/util/c++/clang.cpp +13 -2
@@ -119,7 +119,7 @@ getModuleFromSource(llvm::opt::ArgStringList CFlags, StringRef Path) return getModuleFromSource(std::move(CFlags), Path, VFS); } -PerfModule::PerfModule(std::unique_ptr<llvm::Module>&& M) : Module(std::move(M)) +PerfModule::PerfModule(std::unique_ptr<llvm::Module>&& M) : Module(std::move(M)), _map_base(NULL) { for (llvm::Function& F : *Module) { if (F.getLinkage() != llvm::GlobalValue::ExternalLinkage)
@@ -250,6 +250,13 @@ int PerfModule::doJIT(void) &JITMemoryManager, std::move(Resolver)); + void *map_base = NULL; + for (llvm::GlobalValue *map : Maps) { + JITSymbol sym = CompileLayer.findSymbol(map->getName().str(), true); + void *address = (void *)(intptr_t)sym.getAddress(); + if (!map_base || address < map_base) + map_base = address; + } for (Function *F : JITFunctions) { JITSymbol sym = CompileLayer.findSymbol(F->getName().str(), true);
@@ -269,6 +276,7 @@ int PerfModule::doJIT(void) << hook << ", only one is used\n"; JITResult[hook] = func; } + _map_base = map_base; return 0; }
@@ -392,7 +400,8 @@ void perf_clang__cleanup(void) int perf_clang__compile_bpf(const char *_filename, void **p_obj_buf, size_t *p_obj_buf_sz, - jitted_funcs_map_t *p_funcs_map) + jitted_funcs_map_t *p_funcs_map, + void **p_map_base) { using namespace perf;
@@ -425,6 +434,8 @@ int perf_clang__compile_bpf(const char *_filename, if (p_funcs_map) *p_funcs_map = (jitted_funcs_map_t)(M->copyJITResult()); + if (p_map_base) + *p_map_base = M->getMapBase(); return 0; }
tools/perf/util/c++/clang.h +5 -0
@@ -26,6 +26,7 @@ class PerfModule { std::set<llvm::Function *> JITFunctions;HookMap JITResult; +void *_map_base;void prepareBPF (void );void prepareJIT (void );
@@ -38,6 +39,10 @@ class PerfModule { { return new HookMap(JITResult); } +inline void *getMapBase (void ) + { +return _map_base; + } PerfModule(std::unique_ptr<llvm::Module>&& M);
Signature · 1 line
2.10.1
PATCH v4 12/18 perf clang jit: Retrive fd of BPF map from its offset
In reply to Wang Nanbpf__map_fd() is introduced to retrive fd of a BPF map through its offset in BPF object. This function is going be used in further commits which allow scripts jitted by builtin clang access BPF maps.
--- tools/perf/util/bpf-loader.c | 37 +++++++++++++++++++++++++++++++++++++ tools/perf/util/bpf-loader.h | 19 +++++++++++++++++++ 2 files changed, 56 insertions(+)
tools/perf/util/bpf-loader.c +37 -0
@@ -19,6 +19,7 @@#include "parse-events.h" #include "llvm-utils.h" #include "c++/clang-c.h" +#include "asm/bug.h" // for WARN_ONCE #define DEFINE_PRINT_FN (name, level) \ static int libbpf_##name(const char *fmt, ...) \
@@ -1644,6 +1645,28 @@ int bpf__setup_stdout(struct perf_evlist *evlist __maybe_unused)return 0 ; } +int bpf__map_fd (struct bpf_object *obj,void *jit_map) +{ +struct bpf_obj_priv *priv =bpf_object__priv (obj); +struct bpf_map *map; +size_t map_offset; +void *map_base; + +if (IS_ERR (priv)) +return PTR_ERR (priv); +if (!priv) +return -EINVAL ; + + map_base = priv->map_base; + map_offset = jit_map - map_base; + map =bpf_object__find_map_by_offset (obj, map_offset); +WARN_ONCE (IS_ERR (map),"can't find map offset %zu from '%s'\n" , + map_offset,bpf_object__name (obj)); +if (IS_ERR (map)) +return -ENOENT ; +return bpf_map__fd (map); +} +#define ERRNO_OFFSET (e) ((e) - __BPF_LOADER_ERRNO__START)#define ERRCODE_OFFSET (c) ERRNO_OFFSET(BPF_LOADER_ERRNO__##c)#define NR_ERRNO (__BPF_LOADER_ERRNO__END - __BPF_LOADER_ERRNO__START)
@@ -1825,3 +1848,17 @@ int bpf__strerror_setup_stdout(struct perf_evlist *evlist __maybe_unused,bpf__strerror_end (buf, size);return 0 ; } + +int bpf__strerror_map_fd (struct bpf_object *obj,void *jit_map, +int err,char *buf,size_t size) +{ +struct bpf_obj_priv *priv =bpf_object__priv (obj); +ptrdiff_t offset = priv ? jit_map - priv->map_base : jit_map -NULL ; + +bpf__strerror_head (err, buf, size); +bpf__strerror_entry (EINVAL ,"No map in BPF object %s" ,bpf_object__name (obj)); +bpf__strerror_entry (ENOENT ,"Can't find map offset %lx in BPF object %s" , + (unsigned long )offset,bpf_object__name (obj)); +bpf__strerror_end (buf, size); +return 0 ; +}
tools/perf/util/bpf-loader.h +19 -0
@@ -84,6 +84,9 @@ int bpf__setup_stdout(struct perf_evlist *evlist);int bpf__strerror_setup_stdout (struct perf_evlist *evlist,int err,char *buf,size_t size); +int bpf__map_fd (struct bpf_object *obj,void *jit_map); +int bpf__strerror_map_fd (struct bpf_object *obj,void *jit_map, +int err,char *buf,size_t size);#else static inline struct bpf_object * bpf__prepare_load(const char *filename __maybe_unused,
@@ -136,6 +139,13 @@ bpf__setup_stdout(struct perf_evlist *evlist __maybe_unused) }static inline int +bpf__map_fd (struct bpf_object *obj__maybe_unused , +void *map_ptr__maybe_unused ) +{ +return -ENOTSUP ; +} + +static inline int __bpf_strerror (char *buf,size_t size) {if (!size)
@@ -196,5 +206,14 @@ bpf__strerror_setup_stdout(struct perf_evlist *evlist __maybe_unused, {return __bpf_strerror (buf, size); } + +static inline int +bpf__strerror_map_fd (struct bpf_object *obj__maybe_unused , +void *jit_map__maybe_unused , +int err__maybe_unused , +char *buf,size_t size); +{ +return __bpf_strerror (buf, size); +}#endif #endif
Signature · 1 line
2.10.1
PATCH v4 13/18 perf clang jit: Allow jitted perf hook access BPF maps
In reply to Wang NanNewly introduced jit-helpers.[ch] defines a series of helpers which helps jitted perf hook functions accessing BPF maps defined in their BPF scripts. The helpers fetches fd of 'struct bpf_map' from 'struct bpf_object' and the address of 'struct bpf_map_def' in jitted file. 'struct bpf_object' is the context passed to hooks.
Jit helpers added in this commits are all leading with 'perf_'. We don't use 'bpf_' prefix because in following commits 'bpf_' prefix is going to be assigned to kernel side BPF map operations. Same operation has different protocol for kernel and user.
Example:
$ cat ./test.c
/*******************************************************/
#define SEC(name) __attribute__((section(name), used))
#define BPF_MAP_TYPE_ARRAY 2
#define BPF_MAP_TYPE_PERF_EVENT_ARRAY 4
#define BPF_FUNC_map_lookup_elem 1
static void *(*bpf_map_lookup_elem)(void *map, void *key) =
(void *) BPF_FUNC_map_lookup_elem;
struct bpf_map_def {
unsigned int type;
unsigned int key_size;
unsigned int value_size;
unsigned int max_entries;
};
struct bpf_map_def SEC("maps") counter = {
.type = BPF_MAP_TYPE_ARRAY,
.key_size = sizeof(int),
.value_size = sizeof(int),
.max_entries = 1,
};
extern int perf_map_update_elem(void *ctx, struct bpf_map_def *map,
void *key, void *value, unsigned long flags);
extern int perf_map_lookup_elem(void *ctx, struct bpf_map_def *map,
void *key, void *value);
SEC("sys_close=SyS_close")
int sys_close(void *ctx)
{
int key = 0;
int *value;
value = bpf_map_lookup_elem(&counter, &key);
if (!value)
return 0;
__sync_fetch_and_add(value, 1);
return 0;
}
extern int printf(const char *fmt, ...);
SEC("perfhook:record_start")
void record_start(void *ctx)
{
int key = 0;
int value = 100000000;
printf("Welcom to perf record\n");
perf_map_update_elem(ctx, &counter, &key, &value, 0);
} SEC("perfhook:record_end")
void record_end(void *ctx)
{
int key = 0;
int value;
perf_map_lookup_elem(ctx, &counter, &key, &value);
printf("Goodbye, perf record, value=%d\n", value);
}
char _license[] SEC("license") = "GPL";
int _version SEC("version") = LINUX_VERSION_CODE;
/*******************************************************/
$ sudo perf record -e ./test.c echo Hehe
Welcom to perf record
Hehe
[ perf record: Woken up 1 times to write data ]
Goodbye, perf record, value=100000644
[ perf record: Captured and wrote 0.014 MB perf.data ]--- tools/perf/util/Build | 1 + tools/perf/util/c++/clang.cpp | 9 ++++++- tools/perf/util/jit-helpers.c | 57 +++++++++++++++++++++++++++++++++++++++++++ tools/perf/util/jit-helpers.h | 28 +++++++++++++++++++++ 4 files changed, 94 insertions(+), 1 deletion(-) create mode 100644 tools/perf/util/jit-helpers.c create mode 100644 tools/perf/util/jit-helpers.h
tools/perf/util/Build +1 -0
@@ -125,6 +125,7 @@ libperf-$(CONFIG_DWARF) += genelf_debug.o endif libperf-y += perf-hooks.o +libperf-y += jit-helpers.o libperf-$(CONFIG_CXX) += c++/
tools/perf/util/c++/clang.cpp +8 -1
@@ -38,6 +38,7 @@ #include "llvm-utils.h" #include "util-cxx.h" #include "perf-hooks.h" +#include "jit-helpers.h" namespace perf {
@@ -199,12 +200,18 @@ PerfModule::toBPFObject(void) return std::move(Buffer); } +#define __stringify_1(x) #x +#define __stringify(x) __stringify_1(x) static std::map<const std::string, const void *> exported_funcs = { -#define EXPORT(f) {#f, (const void *)&f} +#define EXPORT(f) {__stringify(f), (const void *)&f} EXPORT(test__clang_callback), EXPORT(printf), EXPORT(puts), + EXPORT(JIT_HELPER_FUNC_NAME(map_update_elem)), + EXPORT(JIT_HELPER_FUNC_NAME(map_lookup_elem)), + EXPORT(JIT_HELPER_FUNC_NAME(map_get_next_key)), + EXPORT(JIT_HELPER_FUNC_NAME(map_pin)), #undef EXPORT };
tools/perf/util/jit-helpers.c new file +57 -0
57 lines not on this page · raw message
tools/perf/util/jit-helpers.h new file +28 -0
@@ -0,0 +1,28 @@ +#ifndef JIT_HELPERS_H +#define JIT_HELPERS_H + +#include <stdint.h> +#include <util/perf-hooks.h> + +#ifdef __cplusplus +extern "C" { +#endif + +#define JIT_HELPER_FUNC_NAME (name) perf_##name + +#define JIT_HELPER (type, name, ...) \ +type JIT_HELPER_FUNC_NAME(name)(__VA_ARGS__) + +#define JIT_BPF_MAP_HELPER (name, ...) \ + JIT_HELPER(int, map_##name, void *ctx, void *map, ##__VA_ARGS__) + +extern JIT_BPF_MAP_HELPER(update_elem, void *key, void *value, uint64_t flags); +extern JIT_BPF_MAP_HELPER(lookup_elem, void *key, void *value); +extern JIT_BPF_MAP_HELPER(get_next_key, void *key, void *next_key); +extern JIT_BPF_MAP_HELPER(pin, const char *pathname); + +#ifdef __cplusplus +} +#endif + +#endif
Signature · 1 line
2.10.1
PATCH v4 14/18 perf clang: Link BPF functions declaration into perf
In reply to Wang NanHardcode BPF functions declarations. Following commits will utilizes clang's virtual file system to automatically include this header to all BPF scripts.
The generated header is wrapped by a BUILTIN_CLANG_NO_DEFAULT_INCLUDE. This macro will be used by following commits to allow user disable this feature.
The C string looks ugly and heavily uses magic numbers because the header is designed to be included automatically at very first, makes any dependency or potential conflict harmful. Actually it is automatically generated using a script:
https://patchwork.kernel.org/patch/9350221
However, that script is fragile so not included by this commit.
--- tools/perf/util/c++/Build | 1 + tools/perf/util/c++/bpf-funcs-str.c | 49 ++++++++++++++++++++++++++++++++ tools/perf/util/c++/clang-bpf-includes.h | 12 ++++++++ 3 files changed, 62 insertions(+) create mode 100644 tools/perf/util/c++/bpf-funcs-str.c create mode 100644 tools/perf/util/c++/clang-bpf-includes.h
tools/perf/util/c++/Build +1 -0
@@ -1,2 +1,3 @@ libperf-$(CONFIG_CLANGLLVM) += clang.o libperf-$(CONFIG_CLANGLLVM) += clang-test.o +libperf-$(CONFIG_CLANGLLVM) += bpf-funcs-str.o
tools/perf/util/c++/bpf-funcs-str.c new file +49 -0
49 lines not on this page · raw message
tools/perf/util/c++/clang-bpf-includes.h new file +12 -0
@@ -0,0 +1,12 @@ +#ifndef CLANG_BPF_INCLUDS_H +#define CLANG_BPF_INCLUDS_H +#ifdef __cplusplus +extern "C" { +#endif + +extern const char clang_builtin_bpf_funcs_str[]; + +#ifdef __cplusplus +} +#endif +#endif
Signature · 1 line
2.10.1
PATCH v4 15/18 perf clang: Declare BPF functions for BPF scripts automatically
In reply to Wang NanUse Clang's OverlayFileSystem, add '-include' options to make builtin clang define BPF functions. After this patch BPF script writer needn't define BPF functions by their own.
Add -DBUILTIN_CLANG_DEFAULT_INCLUDE to builtin clang so when adopting builtin clang BPF functions can be automatically defined, and keep undefined when using external clang. Passing a -UBUILTIN_CLANG_DEFAULT_INCLUDE to cancel this defaudefinition.
Test cases are updated to avoid redefinition of these functions.
--- tools/perf/tests/bpf-script-example.c | 18 +++++++++------ tools/perf/tests/bpf-script-test-prologue.c | 2 ++ tools/perf/tests/bpf-script-test-relocation.c | 18 +++++++++------ tools/perf/util/c++/clang.cpp | 33 ++++++++++++++++++++++++++- 4 files changed, 56 insertions(+), 15 deletions(-)
tools/perf/tests/bpf-script-example.c +11 -7
@@ -8,13 +8,6 @@#endif #define BPF_ANY 0#define BPF_MAP_TYPE_ARRAY 2 -#define BPF_FUNC_map_lookup_elem 1 -#define BPF_FUNC_map_update_elem 2 - -static void *(*bpf_map_lookup_elem)(void *map,void *key) = - (void *) BPF_FUNC_map_lookup_elem; -static void *(*bpf_map_update_elem)(void *map,void *key,void *value,int flags) = - (void *) BPF_FUNC_map_update_elem;struct bpf_map_def {unsigned int type;
@@ -24,6 +17,17 @@ struct bpf_map_def { };#define SEC (NAME ) __attribute__((section(NAME), used)) + +#ifndef BUILTIN_CLANG_DEFAULT_INCLUDE +#define BPF_FUNC_map_lookup_elem 1 +#define BPF_FUNC_map_update_elem 2 + +static void *(*bpf_map_lookup_elem)(void *map,void *key) = + (void *) BPF_FUNC_map_lookup_elem; +static void *(*bpf_map_update_elem)(void *map,void *key,void *value,int flags) = + (void *) BPF_FUNC_map_update_elem; +#endif + struct bpf_map_def SEC("maps") flip_table = { .type = BPF_MAP_TYPE_ARRAY, .key_size = sizeof(int),
tools/perf/tests/bpf-script-test-prologue.c +2 -0
@@ -13,8 +13,10 @@#define FMODE_READ 0x1#define FMODE_WRITE 0x2 +#ifndef BUILTIN_CLANG_DEFAULT_INCLUDE static void (*bpf_trace_printk)(const char *fmt,int fmt_size, ...) = (void *)6 ; +#endif SEC("func=null_lseek file->f_mode offset orig") int bpf_func__null_lseek(void *ctx, int err, unsigned long f_mode,
tools/perf/tests/bpf-script-test-relocation.c +11 -7
@@ -8,13 +8,6 @@#endif #define BPF_ANY 0#define BPF_MAP_TYPE_ARRAY 2 -#define BPF_FUNC_map_lookup_elem 1 -#define BPF_FUNC_map_update_elem 2 - -static void *(*bpf_map_lookup_elem)(void *map,void *key) = - (void *) BPF_FUNC_map_lookup_elem; -static void *(*bpf_map_update_elem)(void *map,void *key,void *value,int flags) = - (void *) BPF_FUNC_map_update_elem;struct bpf_map_def {unsigned int type;
@@ -24,6 +17,17 @@ struct bpf_map_def { };#define SEC (NAME ) __attribute__((section(NAME), used)) + +#ifndef BUILTIN_CLANG_DEFAULT_INCLUDE +#define BPF_FUNC_map_lookup_elem 1 +#define BPF_FUNC_map_update_elem 2 + +static void *(*bpf_map_lookup_elem)(void *map,void *key) = + (void *) BPF_FUNC_map_lookup_elem; +static void *(*bpf_map_update_elem)(void *map,void *key,void *value,int flags) = + (void *) BPF_FUNC_map_update_elem; +#endif + struct bpf_map_def SEC("maps") my_table = { .type = BPF_MAP_TYPE_ARRAY, .key_size = sizeof(int),
tools/perf/util/c++/clang.cpp +32 -1
@@ -39,9 +39,17 @@ #include "util-cxx.h" #include "perf-hooks.h" #include "jit-helpers.h" +#include "clang-bpf-includes.h" namespace perf { +static struct BPFHeader { + llvm::StringRef Path; + llvm::StringRef Content; +} BPFHeaders[] = { + {"/virtual/bpf-funcs.h", clang_builtin_bpf_funcs_str}, +}; + static std::unique_ptr<llvm::LLVMContext> LLVMCtx; using namespace clang;
@@ -63,6 +71,11 @@ createCompilerInvocation(llvm::opt::ArgStringList CFlags, StringRef& Path, "-x", "c"}; CCArgs.append(CFlags.begin(), CFlags.end()); + for (BPFHeader &h : BPFHeaders) { + CCArgs.append(1, "-include"); + CCArgs.append(1, h.Path.begin()); + } + CompilerInvocation *CI = tooling::newInvocation(&Diags, CCArgs); FrontendOptions& Opts = CI->getFrontendOpts();
@@ -71,6 +84,22 @@ createCompilerInvocation(llvm::opt::ArgStringList CFlags, StringRef& Path, return CI; } +static IntrusiveRefCntPtr<vfs::FileSystem> +addBPFHeaders(IntrusiveRefCntPtr<vfs::FileSystem> VFS) +{ + using namespace vfs; + + llvm::IntrusiveRefCntPtr<OverlayFileSystem> OverlayFS( + new OverlayFileSystem(VFS)); + llvm::IntrusiveRefCntPtr<InMemoryFileSystem> MemFS( + new InMemoryFileSystem(true)); + OverlayFS->pushOverlay(MemFS); + + for (BPFHeader &h : BPFHeaders) + MemFS->addFile(h.Path, 0, llvm::MemoryBuffer::getMemBuffer(h.Content)); + return OverlayFS; +} + static std::unique_ptr<PerfModule> getModuleFromSource(llvm::opt::ArgStringList CFlags, StringRef Path, IntrusiveRefCntPtr<vfs::FileSystem> VFS)
@@ -78,7 +107,8 @@ getModuleFromSource(llvm::opt::ArgStringList CFlags, CompilerInstance Clang; Clang.createDiagnostics(); - Clang.setVirtualFileSystem(&*VFS); + IntrusiveRefCntPtr<vfs::FileSystem> OverlayVFS = addBPFHeaders(VFS); + Clang.setVirtualFileSystem(&*OverlayVFS); IntrusiveRefCntPtr<CompilerInvocation> CI = createCompilerInvocation(std::move(CFlags), Path,
@@ -366,6 +396,7 @@ public: { CFlags.push_back(KVerDef.c_str()); CFlags.push_back(NRCpusDef.c_str()); + CFlags.push_back("-DBUILTIN_CLANG_DEFAULT_INCLUDE"); fillCFlagsFromString(CFlags, clang_opt); fillCFlagsFromString(CFlags, kbuild_include_opts, true);
Signature · 1 line
2.10.1
PATCH v4 16/18 perf clang: Include helpers to BPF scripts
In reply to Wang NanAutomatically include some commonly used macros and struct definitions into BPF scripts. Script writers are no longer required to define 'SEC' and 'struct bpf_map_def' in each of their scripts.
--- tools/perf/tests/bpf-script-example.c | 2 +- tools/perf/tests/bpf-script-test-kbuild.c | 2 ++ tools/perf/tests/bpf-script-test-prologue.c | 4 +++- tools/perf/tests/bpf-script-test-relocation.c | 3 +-- tools/perf/util/c++/Build | 1 + tools/perf/util/c++/bpf-helper-str.c | 15 +++++++++++++++ tools/perf/util/c++/clang-bpf-includes.h | 1 + tools/perf/util/c++/clang.cpp | 1 + 8 files changed, 25 insertions(+), 4 deletions(-) create mode 100644 tools/perf/util/c++/bpf-helper-str.c
tools/perf/tests/bpf-script-example.c +1 -1
@@ -9,6 +9,7 @@#define BPF_ANY 0#define BPF_MAP_TYPE_ARRAY 2 +#ifndef BUILTIN_CLANG_DEFAULT_INCLUDE struct bpf_map_def { unsigned int type; unsigned int key_size;
@@ -18,7 +19,6 @@ struct bpf_map_def {#define SEC (NAME ) __attribute__((section(NAME), used)) -#ifndef BUILTIN_CLANG_DEFAULT_INCLUDE #define BPF_FUNC_map_lookup_elem 1#define BPF_FUNC_map_update_elem 2
tools/perf/tests/bpf-script-test-kbuild.c +2 -0
@@ -6,7 +6,9 @@# error Need LINUX_VERSION_CODE# error Example: for 4.2 kernel, put 'clang-opt="-DLINUX_VERSION_CODE=0x40200" into llvm section of ~/.perfconfig'#endif +#ifndef BUILTIN_CLANG_DEFAULT_INCLUDE #define SEC (NAME ) __attribute__((section(NAME), used)) +#endif #include <uapi/linux/fs.h> #include <uapi/asm/ptrace.h>
tools/perf/tests/bpf-script-test-prologue.c +3 -1
@@ -6,7 +6,6 @@# error Need LINUX_VERSION_CODE# error Example: for 4.2 kernel, put 'clang-opt="-DLINUX_VERSION_CODE=0x40200" into llvm section of ~/.perfconfig'#endif -#define SEC (NAME ) __attribute__((section(NAME), used))#include <uapi/linux/fs.h>
@@ -14,6 +13,9 @@#define FMODE_WRITE 0x2#ifndef BUILTIN_CLANG_DEFAULT_INCLUDE + +#define SEC (NAME ) __attribute__((section(NAME), used)) +static void (*bpf_trace_printk)(const char *fmt,int fmt_size, ...) = (void *)6 ;#endif
tools/perf/tests/bpf-script-test-relocation.c +1 -2
@@ -9,6 +9,7 @@#define BPF_ANY 0#define BPF_MAP_TYPE_ARRAY 2 +#ifndef BUILTIN_CLANG_DEFAULT_INCLUDE struct bpf_map_def { unsigned int type; unsigned int key_size;
@@ -17,8 +18,6 @@ struct bpf_map_def { };#define SEC (NAME ) __attribute__((section(NAME), used)) - -#ifndef BUILTIN_CLANG_DEFAULT_INCLUDE #define BPF_FUNC_map_lookup_elem 1#define BPF_FUNC_map_update_elem 2
tools/perf/util/c++/Build +1 -0
@@ -1,3 +1,4 @@ libperf-$(CONFIG_CLANGLLVM) += clang.o libperf-$(CONFIG_CLANGLLVM) += clang-test.o libperf-$(CONFIG_CLANGLLVM) += bpf-funcs-str.o +libperf-$(CONFIG_CLANGLLVM) += bpf-helper-str.o
tools/perf/util/c++/bpf-helper-str.c new file +15 -0
@@ -0,0 +1,15 @@ +#include "clang-bpf-includes.h" +const char clang_builtin_bpf_helper_str[] = +"#ifdef BUILTIN_CLANG_DEFAULT_INCLUDE\n" +"#ifndef BPF_HELPER_DEFINED\n" +"#define BPF_HELPER_DEFINED\n" +"struct bpf_map_def {\n" +" unsigned int type;\n" +" unsigned int key_size;\n" +" unsigned int value_size;\n" +" unsigned int max_entries;\n" +"};\n" +"#define SEC(NAME) __attribute__((section(NAME), used))\n" +"#endif\n" +"#endif" +;
tools/perf/util/c++/clang-bpf-includes.h +1 -0
@@ -5,6 +5,7 @@ extern "C" {#endif extern const char clang_builtin_bpf_funcs_str[]; +extern const char clang_builtin_bpf_helper_str[]; #ifdef __cplusplus }
tools/perf/util/c++/clang.cpp +1 -0
@@ -48,6 +48,7 @@ static struct BPFHeader { llvm::StringRef Content; } BPFHeaders[] = { {"/virtual/bpf-funcs.h", clang_builtin_bpf_funcs_str}, + {"/virtual/bpf-helper.h", clang_builtin_bpf_helper_str}, }; static std::unique_ptr<llvm::LLVMContext> LLVMCtx;
Signature · 1 line
2.10.1
PATCH v4 17/18 perf clang builtin: Define hook helpers by default
In reply to Wang NanAppend declarations of helpers to default include file. All functions appear in exported_funcs array should be declared here except test__clang_callback, because it is used for perf test only.
--- tools/perf/util/c++/bpf-helper-str.c | 7 +++++++ 1 file changed, 7 insertions(+)
tools/perf/util/c++/bpf-helper-str.c +7 -0
@@ -10,6 +10,13 @@ const char clang_builtin_bpf_helper_str[] =" unsigned int max_entries;\n" "};\n" "#define SEC(NAME) __attribute__((section(NAME), used))\n" +"extern int printf(const char *, ...);\n" +"extern int puts(const char *);\n" +"extern int perf_map_update_elem(void *, void *, void *, void *, unsigned long);\n" +"extern int perf_map_lookup_elem(void *, void *, void *, void *);\n" +"extern int perf_map_get_next_key(void *, void *, void *, void *);\n" +"extern int perf_map_pin(void *, void *, const char *);\n" +"extern int perf_map_get(const char *);\n" "#endif\n" "#endif" ;
Signature · 1 line
2.10.1
PATCH v4 18/18 perf clang jit: Export getpid() to perf hook
In reply to Wang NanAfter this patch perf hooks can retrive pid of perf itself by calling getpid. It is important for excluding event from perf.
This commit is also an example to show how to export more helpers to hooked script.
--- tools/perf/util/c++/bpf-helper-str.c | 1 + tools/perf/util/c++/clang.cpp | 1 + 2 files changed, 2 insertions(+)
tools/perf/util/c++/bpf-helper-str.c +1 -0
@@ -12,6 +12,7 @@ const char clang_builtin_bpf_helper_str[] ="#define SEC(NAME) __attribute__((section(NAME), used))\n" "extern int printf(const char *, ...);\n" "extern int puts(const char *);\n" +"extern int getpid(void);\n" "extern int perf_map_update_elem(void *, void *, void *, void *, unsigned long);\n" "extern int perf_map_lookup_elem(void *, void *, void *, void *);\n" "extern int perf_map_get_next_key(void *, void *, void *, void *);\n"
tools/perf/util/c++/clang.cpp +1 -0
@@ -239,6 +239,7 @@ static std::map<const std::string, const void *> exported_funcs = EXPORT(test__clang_callback), EXPORT(printf), EXPORT(puts), + EXPORT(getpid), EXPORT(JIT_HELPER_FUNC_NAME(map_update_elem)), EXPORT(JIT_HELPER_FUNC_NAME(map_lookup_elem)), EXPORT(JIT_HELPER_FUNC_NAME(map_get_next_key)),
Signature · 1 line
2.10.1