perf: enable compression of record mode trace to save storage space
Subsystems (2)
cover perf: enable compression of record mode trace to save storage space
The patch set implements runtime record trace compression accompanied by trace file decompression implemented in the tool report mode. Zstandard library API [1] is used for compression/decompression of data that come from perf_events kernel data buffers.
Realized -z,--compression_level=n option provides ~3-5x avg. trace file size reduction on the tested workloads what significantly saves user's storage space on larger server systems where trace file size can easily reach several tens or even hundreds of GiBs, especially when profiling with stacks for later dwarf unwinding, context-switches tracing and etc.
The option is effective jointly with asynchronous trace writing because compression requires auxiliary memory buffers to operate on and memory buffers for asynchronous trace writing serve that purpose.
Added --mmap-flush option can be used to avoid compressing every single byte of data from mmaped kernel buffers to the trace file and increase compression ratio at the same time lowering tool runtime overhead.
The feature can be disabled from the command line using NO_LIBZSTD define and Zstandard sources can be overridden using value of LIBZSTD_DIR define.
The patch set is for Arnaldo's perf/core repository.
Examples:
$ make -C tools/perf NO_LIBZSTD=1 clean all $ make -C tools/perf LIBZSTD_DIR=/path/to/zstd-1.3.7 clean all
$ tools/perf/perf record -F 42000 --aio -z 1 --mmap-flush 0x1000 -e cycles -- matrix.gcc Addr of buf1 = 0x7fc1bf183010 Offs of buf1 = 0x7fc1bf183180 Addr of buf2 = 0x7fc1bd182010 Offs of buf2 = 0x7fc1bd1821c0 Addr of buf3 = 0x7fc1bb181010 Offs of buf3 = 0x7fc1bb181100 Addr of buf4 = 0x7fc1b9180010 Offs of buf4 = 0x7fc1b9180140 Threads #: 8 Pthreads Matrix size: 2048 Using multiply kernel: multiply1 Execution time = 25.499 seconds [ perf record: Woken up 1157 times to write data ] [ perf record: Compressed 316.684 MB to 58.034 MB, ratio is 5.457 ] [ perf record: Captured and wrote 58.059 MB perf.data ]
$ tools/perf/perf report -D --header
# ========
# captured on : Mon Dec 24 13:19:52 2018
# header version : 1
# data offset : 296
# data size : 60878779
# feat offset : 60879075
# hostname : nntvtune39
# os release : 4.19.9-300.fc29.x86_64
# perf version : 4.13.rc5.gdbb7997
# arch : x86_64
# nrcpus online : 8
# nrcpus avail : 8
# cpudesc : Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz
# cpuid : GenuineIntel,6,94,3
# total memory : 16153380 kB
# cmdline : /root/abudanko/kernel/acme/tools/perf/perf record -F 42000 --aio -z 1 --mmap-flush 0x1000 -e cycles -- ../../matrix/linux/matrix.gcc
# event : name = cycles, , id = { 2315, 2316, 2317, 2318, 2319, 2320, 2321, 2322 }, size = 112, { sample_period, sample_freq } = 42000, sample_type = IP|TID|TIME|PERIOD, read_form>
# CPU_TOPOLOGY info available, use -I to display
# NUMA_TOPOLOGY info available, use -I to display
# pmu mappings: intel_pt = 8, software = 1, power = 11, uprobe = 7, uncore_imc = 12, cpu = 4, cstate_core = 18, uncore_cbox_2 = 15, breakpoint = 5, uncore_cbox_0 = 13, tracepoint >
# CACHE info available, use -I to display
# time of first sample : 0.000000
# time of last sample : 0.000000
# sample duration : 0.000 ms
# MEM_TOPOLOGY info available, use -I to display
# compressed : Zstd, level = 1, ratio = 5
# missing features: TRACING_DATA BUILD_ID BRANCH_STACK GROUP_DESC AUXTRACE STAT CLOCKID
# ========
# 0x128 [0x20]: event: 79
.
. ... raw event: size 32 bytes
. 0000: 4f 00 00 00 00 00 20 00 1f 00 00 00 00 00 00 00 O..... ......... . 0010: 11 a6 ef 1f 00 00 00 00 f8 fe 7c b5 f5 ff ff ff ..........|.....
0 0x128 [0x20]: PERF_RECORD_TIME_CONV: unhandled!
0x148 [0x50]: event: 1
.
. ... raw event: size 80 bytes
. 0000: 01 00 00 00 01 00 50 00 ff ff ff ff 00 00 00 00 ......P......... . 0010: 00 00 00 a8 ff ff ff ff 00 80 33 18 00 00 00 00 ..........3..... . 0020: 00 00 00 a8 ff ff ff ff 5b 6b 65 72 6e 65 6c 2e ........[kernel. . 0030: 6b 61 6c 6c 73 79 6d 73 5d 5f 74 65 78 74 00 00 kallsyms]_text.. . 0040: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0 0x148 [0x50]: PERF_RECORD_MMAP -1/0: [0xffffffffa8000000(0x18338000) @ 0xffffffffa8000000]: x [kernel.kallsyms]_text
...
0x62d8e [0x8]: event: 68
.
. ... raw event: size 8 bytes
. 0000: 44 00 00 00 00 00 08 00 D.......
0 0x62d8e [0x8]: PERF_RECORD_FINISHED_ROUND
0 [0x28]: event: 9
.
. ... raw event: size 40 bytes
. 0000: 09 00 00 00 01 00 28 00 76 78 06 a8 ff ff ff ff ......(.vx...... . 0010: 94 29 00 00 94 29 00 00 82 02 f4 af 33 3e 01 00 .)...)......3>.. . 0020: 01 00 00 00 00 00 00 00 ........
349866692969090 0 [0x28]: PERF_RECORD_SAMPLE(IP, 0x1): 10644/10644: 0xffffffffa8067876 period: 1 addr: 0
... thread: perf:10644
...... dso: vmlinux
0 [0x30]: event: 3
.
. ... raw event: size 48 bytes
. 0000: 03 00 00 00 00 20 30 00 94 29 00 00 94 29 00 00 ..... 0..)...).. . 0010: 6d 61 74 72 69 78 2e 67 63 63 00 00 00 00 00 00 matrix.gcc...... . 0020: 94 29 00 00 94 29 00 00 36 09 f4 af 33 3e 01 00 .)...)..6...3>..
...
349892217639276 0 [0x28]: PERF_RECORD_SAMPLE(IP, 0x1): 10644/10644: 0xffffffffa8288030 period: 45296 addr: 0
... thread: matrix.gcc:10644
...... dso: vmlinux
0 [0x30]: event: 4
.
. ... raw event: size 48 bytes
. 0000: 04 00 00 00 00 00 30 00 94 29 00 00 94 29 00 00 ......0..)...).. . 0010: 94 29 00 00 94 29 00 00 09 b6 57 a1 39 3e 01 00 .)...)....W.9>.. . 0020: 94 29 00 00 94 29 00 00 60 b5 57 a1 39 3e 01 00 .)...)..`.W.9>..
349892217648480 0 [0x30]: PERF_RECORD_EXIT(10644:10644):(10644:10644)
Aggregated stats:
TOTAL events: 8248327
MMAP events: 100
LOST events: 0
COMM events: 2
EXIT events: 9
THROTTLE events: 163377
UNTHROTTLE events: 163377
FORK events: 8
READ events: 0
SAMPLE events: 7909529
MMAP2 events: 9
AUX events: 0
ITRACE_START events: 0
LOST_SAMPLES events: 0
SWITCH events: 0
SWITCH_CPU_WIDE events: 0
NAMESPACES events: 0
ATTR events: 0
EVENT_TYPE events: 0
TRACING_DATA events: 0
BUILD_ID events: 0
FINISHED_ROUND events: 1562
ID_INDEX events: 0
AUXTRACE_INFO events: 0
AUXTRACE events: 0
AUXTRACE_ERROR events: 0
THREAD_MAP events: 1
CPU_MAP events: 1
STAT_CONFIG events: 0
STAT events: 0
STAT_ROUND events: 0
EVENT_UPDATE events: 0
TIME_CONV events: 1
FEATURE events: 0
COMPRESSED events: 10351
cycles stats:
TOTAL events: 7909529
MMAP events: 0
LOST events: 0
COMM events: 0
EXIT events: 0
THROTTLE events: 0
UNTHROTTLE events: 0
FORK events: 0
READ events: 0
SAMPLE events: 7909529
MMAP2 events: 0
AUX events: 0
ITRACE_START events: 0
LOST_SAMPLES events: 0
SWITCH events: 0
SWITCH_CPU_WIDE events: 0
NAMESPACES events: 0
ATTR events: 0
EVENT_TYPE events: 0
TRACING_DATA events: 0
BUILD_ID events: 0
FINISHED_ROUND events: 0
ID_INDEX events: 0
AUXTRACE_INFO events: 0
AUXTRACE events: 0
AUXTRACE_ERROR events: 0
THREAD_MAP events: 0
CPU_MAP events: 0
STAT_CONFIG events: 0
STAT events: 0
STAT_ROUND events: 0
EVENT_UPDATE events: 0
TIME_CONV events: 0
FEATURE events: 0
COMPRESSED events: 0Dump of trace without compression:
$ tools/perf/perf report -D --header # ======== # captured on : Fri Dec 21 16:44:00 2018 # header version : 1 .. # time of first sample : 102911.898905 # time of last sample : 102940.879058 # sample duration : 28980.153 ms # MEM_TOPOLOGY info available, use -I to display # missing features: TRACING_DATA BRANCH_STACK GROUP_DESC AUXTRACE STAT CLOCKID COMPRESSED # ======== #
---
Alexey Budankov (4):
feature: build libzstd feature check, LIBZSTD_DIR and NO_LIBZSTD defines perf record: introduce z, mmap-flush options and PERF_RECORD_COMPRESSED record perf record: enable runtime trace compression
perf report: support record trace file decompression
tools/build/Makefile.feature | 6 +- tools/build/feature/Makefile | 6 +- tools/build/feature/test-all.c | 5 + tools/build/feature/test-libzstd.c | 12 ++ tools/perf/Documentation/perf-record.txt | 11 ++ tools/perf/Makefile.config | 20 +++ tools/perf/Makefile.perf | 3 + tools/perf/builtin-record.c | 213 +++++++++++++++++++++-- tools/perf/builtin-report.c | 151 +++++++++++++++- tools/perf/perf.h | 2 + tools/perf/util/env.h | 10 ++ tools/perf/util/event.c | 1 + tools/perf/util/event.h | 7 + tools/perf/util/evlist.c | 6 +- tools/perf/util/evlist.h | 2 +- tools/perf/util/header.c | 47 ++++- tools/perf/util/header.h | 1 + tools/perf/util/machine.c | 4 + tools/perf/util/mmap.c | 17 +- tools/perf/util/mmap.h | 5 +- tools/perf/util/session.c | 59 ++++++- tools/perf/util/session.h | 16 ++ tools/perf/util/tool.h | 2 + 23 files changed, 572 insertions(+), 34 deletions(-) create mode 100644 tools/build/feature/test-libzstd.c
PATCH 1/4 feature: build libzstd feature check, LIBZSTD_DIR and NO_LIBZSTD defines
Build feature check, LIBZSTD_DIR and NO_LIBZSTD defines to have capability of overriding ZStandard library source and disabling of the feature and from the command line:
$ make -C tools/perf LIBZSTD_DIR=/root/abudanko/zstd-1.3.7 clean all $ make -C tools/perf NO_LIBZSTD=1 clean all
--- tools/build/Makefile.feature | 6 ++++-- tools/build/feature/Makefile | 6 +++++- tools/build/feature/test-all.c | 5 +++++ tools/build/feature/test-libzstd.c | 12 ++++++++++++ tools/perf/Makefile.config | 20 ++++++++++++++++++++ tools/perf/Makefile.perf | 3 +++ 6 files changed, 49 insertions(+), 3 deletions(-) create mode 100644 tools/build/feature/test-libzstd.c
tools/build/Makefile.feature +4 -2
@@ -71,7 +71,8 @@ FEATURE_TESTS_BASIC := \ sdt \ setns \ libopencsd \ - libaio + libaio \ + libzstd # FEATURE_TESTS_BASIC + FEATURE_TESTS_EXTRA is the complete list # of all feature tests
@@ -118,7 +119,8 @@ FEATURE_DISPLAY ?= \ lzma \ get_cpuid \ bpf \ - libaio + libaio \ + libzstd # Set FEATURE_CHECK_(C|LD)FLAGS-all for all FEATURE_TESTS features. # If in the future we need per-feature checks/flags for features not
tools/build/feature/Makefile +5 -1
@@ -62,7 +62,8 @@ FILES= \ test-clang.bin \ test-llvm.bin \ test-llvm-version.bin \ - test-libaio.bin + test-libaio.bin \ + test-libzstd.bin FILES := $(addprefix $(OUTPUT),$(FILES))
@@ -301,6 +302,9 @@ $(OUTPUT)test-clang.bin: $(OUTPUT)test-libaio.bin: $(BUILD) -lrt +$(OUTPUT)test-libzstd.bin: + $(BUILD) -lzstd + ############################### clean:
tools/build/feature/test-all.c +5 -0
@@ -178,6 +178,10 @@# include "test-libaio.c" #undef main +#define main main_test_zstd +# include "test-libzstd.c" +#undef main +int main (int argc,char *argv[]) {main_test_libpython ();
@@ -219,6 +223,7 @@ int main(int argc, char *argv[])main_test_setns ();main_test_libopencsd ();main_test_libaio (); +main_test_libzstd ();return 0 ; }
tools/build/feature/test-libzstd.c new file +12 -0
@@ -0,0 +1,12 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <zstd.h> + +int main (void ) +{ +ZSTD_CStream *cstream; + + cstream =ZSTD_createCStream (); +ZSTD_freeCStream (cstream); + +return 0 ; +}
tools/perf/Makefile.config +20 -0
@@ -145,6 +145,13 @@ endif FEATURE_CHECK_CFLAGS-libbabeltrace := $(LIBBABELTRACE_CFLAGS) FEATURE_CHECK_LDFLAGS-libbabeltrace := $(LIBBABELTRACE_LDFLAGS) -lbabeltrace-ctf +ifdef LIBZSTD_DIR + LIBZSTD_CFLAGS := -I$(LIBZSTD_DIR)/lib + LIBZSTD_LDFLAGS := -L$(LIBZSTD_DIR)/lib +endif +FEATURE_CHECK_CFLAGS-libzstd := $(LIBZSTD_CFLAGS) +FEATURE_CHECK_LDFLAGS-libzstd := $(LIBZSTD_LDFLAGS) + FEATURE_CHECK_CFLAGS-bpf = -I. -I$(srctree)/tools/include -I$(srctree)/tools/arch/$(SRCARCH)/include/uapi -I$(srctree)/tools/include/uapi # include ARCH specific config -include $(src-perf)/arch/$(SRCARCH)/Makefile
@@ -770,6 +777,19 @@ ifndef NO_LZMA endif endif +ifndef NO_LIBZSTD + ifeq ($(feature-libzstd), 1) + CFLAGS += -DHAVE_ZSTD_SUPPORT + CFLAGS += $(LIBZSTD_CFLAGS) + LDFLAGS += $(LIBZSTD_LDFLAGS) + EXTLIBS += -lzstd + $(call detected,CONFIG_ZSTD) + else + msg := $(warning No libzstd found, disables trace compression, please install libzstd-dev[el] and/or set LIBZSTD_DIR); + NO_LIBZSTD := 1 + endif +endif + ifndef NO_BACKTRACE ifeq ($(feature-backtrace), 1) CFLAGS += -DHAVE_BACKTRACE_SUPPORT
tools/perf/Makefile.perf +3 -0
@@ -108,6 +108,9 @@ include ../scripts/utilities.mak # streaming for record mode. Currently Posix AIO trace streaming is # supported only when linking with glibc. # +# Define NO_LIBZSTD if you do not want support of Zstandard based runtime +# trace compression in record mode. +# # As per kernel Makefile, avoid funny character set dependencies unexport LC_ALL
PATCH 2/4 perf record: introduce z, mmap-flush options and PERF_RECORD_COMPRESSED record
In reply to Alexey BudankovIntroduce --compression_level=n, --mmap-flush options and PERF_RECORD_COMPRESSED event record that contains compressed parts of mmap kernel buffer data.
--- tools/perf/Documentation/perf-record.txt | 11 +++ tools/perf/builtin-record.c | 97 ++++++++++++++++++++---- tools/perf/perf.h | 2 + tools/perf/util/env.h | 10 +++ tools/perf/util/event.c | 1 + tools/perf/util/event.h | 7 ++ tools/perf/util/evlist.c | 6 +- tools/perf/util/evlist.h | 2 +- tools/perf/util/header.c | 47 +++++++++++- tools/perf/util/header.h | 1 + tools/perf/util/mmap.c | 4 +- tools/perf/util/mmap.h | 3 +- 12 files changed, 169 insertions(+), 22 deletions(-)
tools/perf/Documentation/perf-record.txt +11 -0
@@ -440,6 +440,17 @@ Use <n> control blocks in asynchronous (Posix AIO) trace writing mode (default: Asynchronous mode is supported only when linking Perf tool with libc library providing implementation for Posix AIO API. +-z:: +--compression-level=n:: +Produce compressed trace file to save storage space using specified level n (default: 0, +best speed: 1, best compression: 22). Compression can be activated in asynchronous trace +writing mode (--aio) only. + +--mmap-flush=n:: +Minimal number of bytes accumulated in mmap buffer that is flushed to trace file (default: 1). +When compression mode (-z) is enabled it is recommended to set --mmap-flush to 4096 or more. +Maximal allowed value is a quater of mmap kernel buffer size. + --all-kernel:: Configure all used events to run in kernel space.
tools/perf/builtin-record.c +82 -15
213 lines not on this page · raw message
tools/perf/perf.h +2 -0
@@ -83,6 +83,8 @@ struct record_opts {clockid_t clockid;u64 clockid_res_ns;int nr_cblocks; +unsigned int comp_level; +int mmap_flush; };struct option ;
tools/perf/util/env.h +10 -0
@@ -64,6 +64,16 @@ struct perf_env {struct memory_node *memory_nodes;unsigned long long memory_bsize;u64 clockid_res_ns; +u32 comp_type; +u32 comp_level; +u32 comp_ratio; +u32 comp_mmap_len; +}; + +enum perf_compress_type { +PERF_COMP_NONE =0 , +PERF_COMP_ZSTD , +PERF_COMP_EOF };extern struct perf_env perf_env;
tools/perf/util/event.c +1 -0
@@ -62,6 +62,7 @@ static const char *perf_event__names[] = { [PERF_RECORD_EVENT_UPDATE] = "EVENT_UPDATE", [PERF_RECORD_TIME_CONV] = "TIME_CONV", [PERF_RECORD_HEADER_FEATURE] = "FEATURE", + [PERF_RECORD_COMPRESSED] = "COMPRESSED", };static const char *perf_ns__names[] = {
tools/perf/util/event.h +7 -0
@@ -249,6 +249,7 @@ enum perf_user_event_type { /* above any possible kernel type */PERF_RECORD_EVENT_UPDATE =78 ,PERF_RECORD_TIME_CONV =79 ,PERF_RECORD_HEADER_FEATURE =80 , +PERF_RECORD_COMPRESSED =81 , PERF_RECORD_HEADER_MAX };
@@ -620,6 +621,11 @@ struct feature_event {char data[]; }; +struct compressed_event { +struct perf_event_header header; +char data[]; +}; +union perf_event {struct perf_event_header header;struct mmap_event mmap;
@@ -651,6 +657,7 @@ union perf_event {struct stat_round_event stat_round;struct time_conv_event time_conv;struct feature_event feat; +struct compressed_event pack; };void perf_event__print_totals (void );
tools/perf/util/evlist.c +3 -3
@@ -1022,7 +1022,7 @@ int perf_evlist__parse_mmap_pages(const struct option *opt, const char *str,*/ int perf_evlist__mmap_ex (struct perf_evlist *evlist,unsigned int pages,unsigned int auxtrace_pages, -bool auxtrace_overwrite,int nr_cblocks) +bool auxtrace_overwrite,int nr_cblocks,int flush) {struct perf_evsel *evsel;const struct cpu_map *cpus = evlist->cpus;
@@ -1032,7 +1032,7 @@ int perf_evlist__mmap_ex(struct perf_evlist *evlist, unsigned int pages,* Its value is decided by evsel's write_backward. * So &mp should not be passed through const pointer. */ -struct mmap_params mp = { .nr_cblocks = nr_cblocks }; +struct mmap_params mp = { .nr_cblocks = nr_cblocks, .flush = flush };if (!evlist->mmap) evlist->mmap =perf_evlist__alloc_mmap (evlist, false);
@@ -1064,7 +1064,7 @@ int perf_evlist__mmap_ex(struct perf_evlist *evlist, unsigned int pages,int perf_evlist__mmap (struct perf_evlist *evlist,unsigned int pages) { -return perf_evlist__mmap_ex (evlist, pages,0 , false,0 ); +return perf_evlist__mmap_ex (evlist, pages,0 , false,0 ,1 ); }int perf_evlist__create_maps (struct perf_evlist *evlist,struct target *target)
tools/perf/util/evlist.h +1 -1
@@ -162,7 +162,7 @@ unsigned long perf_event_mlock_kb_in_pages(void);int perf_evlist__mmap_ex (struct perf_evlist *evlist,unsigned int pages,unsigned int auxtrace_pages, -bool auxtrace_overwrite,int nr_cblocks); +bool auxtrace_overwrite,int nr_cblocks,int flush);int perf_evlist__mmap (struct perf_evlist *evlist,unsigned int pages);void perf_evlist__munmap (struct perf_evlist *evlist);
tools/perf/util/header.c +46 -1
71 lines not on this page · raw message
tools/perf/util/header.h +1 -0
@@ -39,6 +39,7 @@ enum {HEADER_SAMPLE_TIME ,HEADER_MEM_TOPOLOGY ,HEADER_CLOCKID , +HEADER_COMPRESSED ,HEADER_LAST_FEATURE , HEADER_FEAT_BITS = 256, };
tools/perf/util/mmap.c +3 -1
@@ -347,6 +347,8 @@ int perf_mmap__mmap(struct perf_mmap *map, struct mmap_params *mp, int fd, int c &mp->auxtrace_mp, map->base, fd))return -1 ; + map->flush = mp->flush; +return perf_mmap__aio_mmap (map, mp); }
@@ -395,7 +397,7 @@ static int __perf_mmap__read_init(struct perf_mmap *md) md->start = md->overwrite ? head : old; md->end = md->overwrite ? old : head; -if (md->start == md->end) +if ((md->end - md->start) < md->flush)return -EAGAIN ; size = md->end - md->start;
tools/perf/util/mmap.h +2 -1
@@ -30,6 +30,7 @@ struct perf_mmap {bool overwrite;struct auxtrace_mmap auxtrace_mmap;char event_copy[PERF_SAMPLE_MAX_SIZE ]__aligned(8) ; +u64 flush; #ifdef HAVE_AIO_SUPPORT struct { void **data;
@@ -69,7 +70,7 @@ enum bkw_mmap_state { };struct mmap_params { -int prot, mask, nr_cblocks; +int prot, mask, nr_cblocks, flush;struct auxtrace_mmap_params auxtrace_mp; };
Re: [PATCH v1 2/4] perf record: introduce z, mmap-flush options and PERF_RECORD_COMPRESSED record
Quoted 5 lines · Alexey Budankov
Introduce --compression_level=n, --mmap-flush options and
PERF_RECORD_COMPRESSEDevent record that contains compressed parts of mmap kernel buffer data.Signed-off-by Alexey Budankov <[email protected]>
Quoted 14 lines
--- tools/perf/Documentation/perf-record.txt | 11 +++ tools/perf/builtin-record.c | 97 ++++++++++++++++++++---- tools/perf/perf.h | 2 + tools/perf/util/env.h | 10 +++ tools/perf/util/event.c | 1 + tools/perf/util/event.h | 7 ++ tools/perf/util/evlist.c | 6 +- tools/perf/util/evlist.h | 2 +- tools/perf/util/header.c | 47 +++++++++++- tools/perf/util/header.h | 1 + tools/perf/util/mmap.c | 4 +- tools/perf/util/mmap.h | 3 +- 12 files changed, 169 insertions(+), 22 deletions(-)
also I'm getting here similar error (like for the affinity patchset)
[jolsa@krava perf]$ git am /tmp/comp
Applying: feature: build libzstd feature check, LIBZSTD_DIR and NO_LIBZSTD defines
Applying: perf record: introduce z, mmap-flush options and PERF_RECORD_COMPRESSED record
error: corrupt patch at line 526
Patch failed at 0002 perf record: introduce z, mmap-flush options and PERF_RECORD_COMPRESSED record
again the raw patch applies correctly:
[jolsa@krava perf]$ patch -p3 < /tmp/comp2
patching file Documentation/perf-record.txt
patching file builtin-record.c
patching file perf.h
patching file util/env.h
patching file util/event.c
patching file util/event.h
patching file util/evlist.c
patching file util/evlist.h
patching file util/header.c
patching file util/header.h
patching file util/mmap.c
patching file util/mmap.h
jirka
Re: [PATCH v1 2/4] perf record: introduce z, mmap-flush options and PERF_RECORD_COMPRESSED record
Hi,
Quoted 21 lines · Jiri Olsa
Quoted 5 lines · Alexey Budankov
Introduce --compression_level=n, --mmap-flush options and
PERF_RECORD_COMPRESSEDevent record that contains compressed parts of mmap kernel buffer data.Signed-off-by Alexey Budankov <[email protected]>Quoted 14 lines
--- tools/perf/Documentation/perf-record.txt | 11 +++ tools/perf/builtin-record.c | 97 ++++++++++++++++++++---- tools/perf/perf.h | 2 + tools/perf/util/env.h | 10 +++ tools/perf/util/event.c | 1 + tools/perf/util/event.h | 7 ++ tools/perf/util/evlist.c | 6 +- tools/perf/util/evlist.h | 2 +- tools/perf/util/header.c | 47 +++++++++++- tools/perf/util/header.h | 1 + tools/perf/util/mmap.c | 4 +- tools/perf/util/mmap.h | 3 +- 12 files changed, 169 insertions(+), 22 deletions(-)
Quoted 1 line
also I'm getting here similar error (like for the affinity patchset)
Really weird. Hopefully it can be resolved in the next version of the patch set.
Thanks,
Alexey
Quoted 26 lines
[jolsa@krava perf]$ git am
/tmp/comp
Applying: feature: build libzstd feature check,LIBZSTD_DIRandNO_LIBZSTDdefines
Applying: perf record: introduce z, mmap-flush options andPERF_RECORD_COMPRESSEDrecord
error: corrupt patch at line 526
Patch failed at 0002 perf record: introduce z, mmap-flush options andPERF_RECORD_COMPRESSEDrecordagain the raw patch applies correctly:
[jolsa@krava perf]$ patch -p3 <
/tmp/comp2
patching fileDocumentation/perf-record.txt
patching filebuiltin-record.c
patching fileperf.h
patching fileutil/env.h
patching fileutil/event.c
patching fileutil/event.h
patching fileutil/evlist.c
patching fileutil/evlist.h
patching fileutil/header.c
patching fileutil/header.h
patching fileutil/mmap.c
patching fileutil/mmap.hjirka
PATCH 3/4 perf record: enable runtime trace compression
In reply to Alexey BudankovCompression is implemented using Zstandard API and employs AIO buffers as the memory to operate on so memcpy() is substituted by the API call. If the API call fails for some reason copying falls back to memcpy(). Data chunks are split and packed into PERF_RECORD_COMPRESSED records by 64KB at max. mmap-flush option value can be used to avoid compression of every single byte of data and increase compression ratio.
--- tools/perf/builtin-record.c | 122 ++++++++++++++++++++++++++++++++++-- tools/perf/util/mmap.c | 13 ++-- tools/perf/util/mmap.h | 2 + 3 files changed, 127 insertions(+), 10 deletions(-)
tools/perf/builtin-record.c +118 -4
@@ -53,6 +53,9 @@#include <sys/mman.h> #include <sys/wait.h> #include <linux/time64.h> +#ifdef HAVE_ZSTD_SUPPORT +#include <zstd.h> +#endif struct switch_output {bool enabled;
@@ -83,6 +86,9 @@ struct record {unsigned long long samples;u64 bytes_transferred;u64 bytes_compressed; +#ifdef HAVE_ZSTD_SUPPORT +ZSTD_CStream *zstd_cstream; +#endif };static volatile int auxtrace_record__snapshot_started;
@@ -358,6 +364,109 @@ static int record__mmap_flush_parse(const struct option *opt,return 0 ; } +#ifdef HAVE_ZSTD_SUPPORT +static int record__zstd_init (struct record *rec) +{ +size_t ret; + +if (rec->opts.comp_level ==0 ) +return 0 ; + + rec->zstd_cstream =ZSTD_createCStream (); +if (rec->zstd_cstream ==NULL ) { +pr_err ("Couldn't create compression stream, disables trace compression\n" ); +return -1 ; + } + + ret =ZSTD_initCStream (rec->zstd_cstream, rec->opts.comp_level); +if (ZSTD_isError (ret)) { +pr_err ("Failed to initialize compression stream: %s\n" ,ZSTD_getErrorName (ret)); +return -1 ; + } + +return 0 ; +} + +static int record__zstd_fini (struct record *rec) +{ +if (rec->zstd_cstream) { +ZSTD_freeCStream (rec->zstd_cstream); + rec->zstd_cstream =NULL ; + } + +return 0 ; +} + +static size_t record__zstd_compress (void *to,void *dst,size_t dst_size, +void *src,size_t src_size) +{ +void *dst_head = dst; +struct record *rec = to; +size_t ret, size, compressed =0 ; +struct compressed_event *event =NULL ; +/* maximum size of record data size (2^16 - 1 - header) */ +const size_t max_data_size = (1 <<8 *sizeof (event->header.size)) - +1 -sizeof (struct compressed_event ); +ZSTD_inBuffer input = { src, src_size,0 }; +ZSTD_outBuffer output; + +if (rec->opts.comp_level ==0 ) { +memcpy (dst_head, src, src_size); +return src_size; + } + +while (input.pos < input.size) { + event = dst; + + event->header.type =PERF_RECORD_COMPRESSED ; + event->header.size = size =sizeof (struct compressed_event ); + compressed += size; + dst += size; + dst_size -= size; + + output = (ZSTD_outBuffer ){ dst, (dst_size > max_data_size) ? + max_data_size : dst_size,0 }; + ret =ZSTD_compressStream (rec->zstd_cstream, &output, &input); +ZSTD_flushStream (rec->zstd_cstream, &output); +if (ZSTD_isError (ret)) { +pr_err ("failed to compress %ld bytes: %s\n" , + (long )src_size,ZSTD_getErrorName (ret)); +memcpy (dst_head, src, src_size); +return src_size; + } + size = output.pos; + + event->header.size += size; + compressed += size; + dst += size; + dst_size -= size; + } + + rec->bytes_transferred += src_size; + rec->bytes_compressed += compressed; + +return compressed; +} +#else /* !HAVE_ZSTD_SUPPORT */ +static int record__zstd_init (struct record *rec__maybe_unused ) +{ +return -1 ; +} + +static int record__zstd_fini (struct record *rec__maybe_unused ) +{ +return 0 ; +} + +static size_t record__zstd_compress (void *to__maybe_unused , +void *dst,size_t dst_size__maybe_unused , +void *src,size_t src_size) +{ +memcpy (dst, src, src_size); +return src_size; +} +#endif + static int process_synthesized_event(struct perf_tool *tool, union perf_event *event, struct perf_sample *sample __maybe_unused,
@@ -799,7 +908,8 @@ static int record__mmap_read_evlist(struct record *rec, struct perf_evlist *evli* becomes available after previous aio write request. */ idx =record__aio_sync (map, false); -if (perf_mmap__aio_push (map, rec, idx, record__aio_pushfn, &off) !=0 ) { +if (perf_mmap__aio_push (map, rec, idx, + record__zstd_compress, record__aio_pushfn, &off) !=0 ) {record__aio_set_pos (trace_fd, off);if (sync) map->flush = flush;
@@ -1175,8 +1285,12 @@ static int __cmd_record(struct record *rec, int argc, const char **argv) fd =perf_data__fd (data); rec->session = session; - session->header.env.comp_type =PERF_COMP_NONE ; - rec->opts.comp_level =0 ; +if (record__zstd_init (rec) ==0 ) { + session->header.env.comp_type =PERF_COMP_ZSTD ; + }else { + session->header.env.comp_type =PERF_COMP_NONE ; + rec->opts.comp_level =0 ; + } session->header.env.comp_level = rec->opts.comp_level;record__init_features (rec);
@@ -1447,7 +1561,7 @@ static int __cmd_record(struct record *rec, int argc, const char **argv) out_child:record__mmap_read_all (rec, true);record__aio_mmap_read_sync (rec); - +record__zstd_fini (rec);if (!quiet && rec->bytes_transferred && rec->bytes_compressed) {float ratio = (float )rec->bytes_transferred/(float )rec->bytes_compressed; session->header.env.comp_ratio = ratio +0.5 ;
tools/perf/util/mmap.c +7 -6
@@ -218,14 +218,16 @@ static void perf_mmap__aio_munmap(struct perf_mmap *map) }int perf_mmap__aio_push (struct perf_mmap *md,void *to,int idx, +size_t compress (void *to,void *dst,size_t dst_size,void *src,size_t src_size),int push (void *to,struct aiocb *cblock,void *buf,size_t size,off_t off),off_t *off) {u64 head =perf_mmap__read_head (md);unsigned char *data = md->base + page_size; -unsigned long size, size0 =0 ; +size_t size, size0 =0 , size1 =0 ;void *buf;int rc =0 ; +size_t mmap_len =perf_mmap__mmap_len (md); rc =perf_mmap__read_init (md);if (rc <0 )
@@ -254,14 +256,13 @@ int perf_mmap__aio_push(struct perf_mmap *md, void *to, int idx, buf = &data[md->start & md->mask]; size = md->mask +1 - (md->start & md->mask); md->start += size; -memcpy (md->aio.data[idx], buf, size); - size0 = size; + size0 =compress (to, md->aio.data[idx], mmap_len, buf, size); } buf = &data[md->start & md->mask]; size = md->end - md->start; md->start += size; -memcpy (md->aio.data[idx] + size0, buf, size); + size1 =compress (to, md->aio.data[idx] + size0, mmap_len - size0, buf, size);/* * Increment md->refcount to guard md->data[idx] buffer
@@ -277,9 +278,9 @@ int perf_mmap__aio_push(struct perf_mmap *md, void *to, int idx, md->prev = head;perf_mmap__consume (md); - rc =push (to, &md->aio.cblocks[idx], md->aio.data[idx], size0 + size, *off); + rc =push (to, &md->aio.cblocks[idx], md->aio.data[idx], size0 + size1, *off);if (!rc) { - *off += size0 + size; + *off += size0 + size1; }else {/* * Decrement md->refcount back if aio write
tools/perf/util/mmap.h +2 -0
@@ -100,10 +100,12 @@ int perf_mmap__push(struct perf_mmap *md, void *to, int push(struct perf_mmap *map, void *to, void *buf, size_t size));#ifdef HAVE_AIO_SUPPORT int perf_mmap__aio_push (struct perf_mmap *md,void *to,int idx, +size_t compress (void *to,void *dst,size_t dst_size,void *src,size_t src_size),int push (void *to,struct aiocb *cblock,void *buf,size_t size,off_t off),off_t *off);#else static inline int perf_mmap__aio_push (struct perf_mmap *md__maybe_unused ,void *to__maybe_unused ,int idx__maybe_unused , +size_t compress (void *to,void *dst,size_t dst_size,void *src,size_t src_size)__maybe_unused ,int push (void *to,struct aiocb *cblock,void *buf,size_t size,off_t off)__maybe_unused ,off_t *off__maybe_unused ) {
PATCH 4/4 perf report: support record trace file decompression
In reply to Alexey BudankovPERF_RECORD_COMPRESSED records are decompressed from trace file into a linked list of mmaped memory regions using Zstandard API. After that the region is loaded fetching uncompressed events. When dumping raw trace like perf report -D file offsets of events from compressed records are set to zero.
--- tools/perf/builtin-report.c | 151 +++++++++++++++++++++++++++++++++++- tools/perf/util/machine.c | 4 + tools/perf/util/session.c | 59 +++++++++++++- tools/perf/util/session.h | 16 ++++ tools/perf/util/tool.h | 2 + 5 files changed, 230 insertions(+), 2 deletions(-)
tools/perf/builtin-report.c +150 -1
@@ -52,7 +52,10 @@#include <sys/types.h> #include <sys/stat.h> #include <unistd.h> -#include <linux/mman.h> +#include <sys/mman.h> +#ifdef HAVE_ZSTD_SUPPORT +#include <zstd.h> +#endif struct report {struct perf_tool tool;
@@ -118,6 +121,94 @@ static int report__config(const char *var, const char *value, void *cb)return 0 ; } +#ifdef HAVE_ZSTD_SUPPORT +static int report__zstd_init (struct perf_session *session) +{ +size_t ret; + + session->zstd_dstream =ZSTD_createDStream (); +if (session->zstd_dstream ==NULL ) + { +pr_err ("Couldn't create decompression stream, disables trace compression\n" ); +return -1 ; + } + + ret =ZSTD_initDStream (session->zstd_dstream); +if (ZSTD_isError (ret)) + { +pr_err ("Failed to initialize decompression stream: %s\n" ,ZSTD_getErrorName (ret)); +return -1 ; + } + +return 0 ; +} + +static int report__zstd_fini (struct perf_session *session) +{ +struct decomp *next = session->decomp, *decomp; +size_t decomp_len = session->header.env.comp_mmap_len; + +if (session->zstd_dstream) { +ZSTD_freeDStream (session->zstd_dstream); + session->zstd_dstream =NULL ; + } + +do { + decomp = next; +if (decomp ==NULL ) +break ; + next = decomp->next; +munmap (decomp, decomp_len +sizeof (struct decomp )); + }while (1 ); + +return 0 ; +} + +static size_t report__zstd_decompress (struct perf_session *session, +void *src,size_t src_size, +void *dst,size_t dst_size) +{ +size_t ret; +ZSTD_inBuffer input = { src, src_size,0 }; +ZSTD_outBuffer output = { dst, dst_size,0 }; + +if (session->zstd_dstream ==NULL ) +return 0 ; + +while (input.pos < input.size) { + ret =ZSTD_decompressStream (session->zstd_dstream, &output, &input); +if (ZSTD_isError (ret)) + { +pr_err ("failed to decompress (B): %ld -> %ld : %s\n" , + src_size, output.size,ZSTD_getErrorName (ret)); +break ; + } + output.dst = dst + output.pos; + output.size = dst_size - output.pos; + } + +return output.pos; +} + +#else /* !HAVE_ZSTD_SUPPORT */ +static int report__zstd_init (struct perf_session *session__maybe_unused ) +{ +return -1 ; +} + +static int report__zstd_fini (struct perf_session *session__maybe_unused ) +{ +return 0 ; +} + +static size_t report__zstd_decompress (struct perf_session *session__maybe_unused , +void *src__maybe_unused ,size_t src_size__maybe_unused , +void *dst__maybe_unused ,size_t dst_size__maybe_unused ) +{ +return 0 ; +} +#endif +static int hist_iter__report_callback (struct hist_entry_iter *iter,struct addr_location *al,bool single,void *arg)
@@ -225,6 +316,57 @@ static int process_feature_event(struct perf_session *session,return 0 ; } +static int process_compressed_event (struct perf_session *session, +union perf_event *event,u64 file_offset) +{ +void *src; +size_t decomp_size, src_size; +u64 decomp_last_rem =0 ; +size_t decomp_len = session->header.env.comp_mmap_len; +struct decomp *decomp, *decomp_last = session->decomp_last; + + decomp =mmap (NULL ,sizeof (struct decomp ) + decomp_len,PROT_READ |PROT_WRITE , +MAP_ANONYMOUS |MAP_PRIVATE ,-1 ,0 ); +if (decomp ==MAP_FAILED ) { +pr_err ("Couldn't allocate memory for decompression\n" ); +return -1 ; + } + + decomp->file_pos = file_offset; + decomp->head =0 ; + +if (decomp_last) { + decomp_last_rem = decomp_last->size - decomp_last->head; +memcpy (decomp->data, &(decomp_last->data[decomp_last->head]), decomp_last_rem); + decomp->size = decomp_last_rem; + } + + src = (void *)event +sizeof (struct compressed_event ); + src_size = event->pack.header.size -sizeof (struct compressed_event ); + + decomp_size =report__zstd_decompress (session, src, src_size, + &(decomp->data[decomp_last_rem]), decomp_len - decomp_last_rem); +if (!decomp_size) { +munmap (decomp,sizeof (struct decomp ) + decomp_len); +pr_err ("Couldn't decompress data\n" ); +return -1 ; + } + + decomp->size += decomp_size; + +if (session->decomp ==NULL ) { + session->decomp = decomp; + session->decomp_last = decomp; + }else { + session->decomp_last->next = decomp; + session->decomp_last = decomp; + } + +pr_debug ("decomp (B): %ld to %ld\n" , src_size, decomp_size); + +return 0 ; +} + static int process_sample_event(struct perf_tool *tool, union perf_event *event, struct perf_sample *sample,
@@ -983,6 +1125,7 @@ int cmd_report(int argc, const char **argv) .auxtrace = perf_event__process_auxtrace, .event_update = perf_event__process_event_update, .feature = process_feature_event, + .compressed = process_compressed_event, .ordered_events = true, .ordering_requires_timestamps = true, },
@@ -1205,6 +1348,10 @@ int cmd_report(int argc, const char **argv) report.session = session; +if (session->header.env.comp_type ==PERF_COMP_ZSTD && + session->header.env.comp_level) +report__zstd_init (session); + has_br_stack =perf_header__has_feat (&session->header,HEADER_BRANCH_STACK );
@@ -1409,6 +1556,8 @@ int cmd_report(int argc, const char **argv) error:zfree (&report.ptime_range); +report__zstd_fini (session); +perf_session__delete (session);return ret; }
tools/perf/util/machine.c +4 -0
@@ -972,6 +972,10 @@ int machine__map_x86_64_entry_trampolines(struct machine *machine,continue ; dest_map =map_groups__find (kmaps, map->pgoff); +if (!dest_map) { +pr_debug ("dest_map for %lx is NULL\n" , map->pgoff); +continue ; + }if (dest_map != map) map->pgoff = dest_map->map_ip (dest_map, map->pgoff); found = true;
tools/perf/util/session.c +58 -1
107 lines not on this page · raw message
tools/perf/util/session.h +16 -0
@@ -11,6 +11,9 @@#include <linux/kernel.h> #include <linux/rbtree.h> #include <linux/perf_event.h> +#ifdef HAVE_ZSTD_SUPPORT +#include <zstd.h> +#endif struct ip_callchain ;struct symbol ;
@@ -35,6 +38,19 @@ struct perf_session {struct ordered_events ordered_events;struct perf_data *data;struct perf_tool *tool; +struct decomp *decomp; +struct decomp *decomp_last; +#ifdef HAVE_ZSTD_SUPPORT +ZSTD_DStream *zstd_dstream; +#endif +}; + +struct decomp { +struct decomp *next; +u64 file_pos; +u64 head; +size_t size; +char data[]; };struct perf_tool ;
tools/perf/util/tool.h +2 -0
@@ -28,6 +28,7 @@ typedef int (*event_attr_op)(struct perf_tool *tool,typedef int (*event_op2 )(struct perf_session *session,union perf_event *event);typedef s64 (*event_op3 )(struct perf_session *session,union perf_event *event); +typedef int (*event_op4 )(struct perf_session *session,union perf_event *event,u64 data);typedef int (*event_oe )(struct perf_tool *tool,union perf_event *event,struct ordered_events *oe);
@@ -69,6 +70,7 @@ struct perf_tool { stat, stat_round, feature; +event_op4 compressed;event_op3 auxtrace;bool ordered_events;bool ordering_requires_timestamps;
Quoted 12 lines · Alexey Budankov
The patch set implements runtime record trace compression accompanied by trace file decompression implemented in the tool report mode. Zstandard library API [1] is used for compression/decompression of data that come from
perf_eventskernel data buffers.Realized -z,--compression_level=n option provides ~3-5x avg. trace file size reduction on the tested workloads what significantly saves user's storage space on larger server systems where trace file size can easily reach several tens or even hundreds of GiBs, especially when profiling with stacks for later dwarf unwinding, context-switches tracing and etc.
Quoted 3 lines
The option is effective jointly with asynchronous trace writing because compression requires auxiliary memory buffers to operate on and memory buffers for asynchronous trace writing serve that purpose.
I dont like that it's onlt for aio only, I can't really see why it's a problem for normal data.. can't we just have one layer before and stream the data to the compress function instead of the file (or aio buffers).. and that compress functions would spit out 64K size COMPRESSED events, which would go to file (or aio buffers)
the report side would process them (decompress) on the session layer before the tool callbacks are called
jirka
Hi,
Quoted 13 lines · Jiri Olsa
Quoted 12 lines · Alexey Budankov
The patch set implements runtime record trace compression accompanied by trace file decompression implemented in the tool report mode. Zstandard library API [1] is used for compression/decompression of data that come from
perf_eventskernel data buffers.Realized -z,--compression_level=n option provides ~3-5x avg. trace file size reduction on the tested workloads what significantly saves user's storage space on larger server systems where trace file size can easily reach several tens or even hundreds of GiBs, especially when profiling with stacks for later dwarf unwinding, context-switches tracing and etc.
Quoted 5 lines
Quoted 3 lines
The option is effective jointly with asynchronous trace writing because compression requires auxiliary memory buffers to operate on and memory buffers for asynchronous trace writing serve that purpose.
I dont like that it's onlt for aio only, I can't really see why it's
For serial streaming, on CPU bound codes, under full system utilization it can induce more runtime overhead and increase data loss because amount of code on performance critical path grows, of course size of written data reduces but still. Feeding kernel buffer content by user space code to a syscall is extended with intermediate copying to user space memory with doing some math on it in the middle.
Quoted 4 lines
a problem for normal data.. can't we just have one layer before and stream the data to the compress function instead of the file (or aio buffers).. and that compress functions would spit out 64K size COMPRESSED events, which would go to file (or aio buffers)
It is already almost like that. Compression could be bridged using AIO buffers but then still streamed to file serially using record__pushfn() and that would make some sense for moderate profiling cases on systems without AIO support and trace streaming based on it.
Quoted 3 lines
the report side would process them (decompress) on the session layer before the tool callbacks are called
It is already pretty similar to that.
Thanks,
Alexey
Quoted 3 lines
jirka
Quoted 39 lines · Alexey Budankov
Hi,
Quoted 13 lines · Jiri Olsa
Quoted 12 lines · Alexey Budankov
The patch set implements runtime record trace compression accompanied by trace file decompression implemented in the tool report mode. Zstandard library API [1] is used for compression/decompression of data that come from
perf_eventskernel data buffers.Realized -z,--compression_level=n option provides ~3-5x avg. trace file size reduction on the tested workloads what significantly saves user's storage space on larger server systems where trace file size can easily reach several tens or even hundreds of GiBs, especially when profiling with stacks for later dwarf unwinding, context-switches tracing and etc.
Quoted 5 lines
Quoted 3 lines
The option is effective jointly with asynchronous trace writing because compression requires auxiliary memory buffers to operate on and memory buffers for asynchronous trace writing serve that purpose.
I dont like that it's onlt for aio only, I can't really see why it's
For serial streaming, on CPU bound codes, under full system utilization it can induce more runtime overhead and increase data loss because amount of code on performance critical path grows, of course size of written data reduces but still. Feeding kernel buffer content by user space code to a syscall is extended with intermediate copying to user space memory with doing some math on it in the middle.
Quoted 4 lines
a problem for normal data.. can't we just have one layer before and stream the data to the compress function instead of the file (or aio buffers).. and that compress functions would spit out 64K size COMPRESSED events, which would go to file (or aio buffers)
It is already almost like that. Compression could be bridged using AIO buffers but then still streamed to file serially using
record__pushfn()and that would make some sense for moderate profiling cases on systems without AIO support and trace streaming based on it.Quoted 1 line
Quoted 4 lines
Quoted 2 lines
the report side would process them (decompress) on the session layer before the tool callbacks are called
It is already pretty similar to that.
hum, AFAICS you do that in report code not in on the session layer
jirka
Quoted 6 lines · Jiri Olsa
Quoted 5 lines · Alexey Budankov
Hi,
Quoted 3 lines · Jiri Olsa
Quoted 2 lines · Alexey Budankov
buffers for asynchronous trace writing serve that purpose.
<SNIP>
Quoted 21 lines
Quoted 21 lines
Quoted 2 lines
I dont like that it's onlt for aio only, I can't really see why it's
For serial streaming, on CPU bound codes, under full system utilization it can induce more runtime overhead and increase data loss because amount of code on performance critical path grows, of course size of written data reduces but still. Feeding kernel buffer content by user space code to a syscall is extended with intermediate copying to user space memory with doing some math on it in the middle.
Quoted 4 lines
a problem for normal data.. can't we just have one layer before and stream the data to the compress function instead of the file (or aio buffers).. and that compress functions would spit out 64K size COMPRESSED events, which would go to file (or aio buffers)
It is already almost like that. Compression could be bridged using AIO buffers but then still streamed to file serially using
record__pushfn()and that would make some sense for moderate profiling cases on systems without AIO support and trace streaming based on it.Quoted 1 line
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
the report side would process them (decompress) on the session layer before the tool callbacks are called
It is already pretty similar to that.
hum, AFAICS you do that in report code not in on the session layer
Correct. Decompressor and handling of compressed data chunks could be moved to session related code.
Thanks,
Alexey
Quoted 3 lines
jirka