perf: enable compression of record mode trace to save storage space
Subsystems (2)
v2 cover perf: enable compression of record mode trace to save storage space
This is the rebase to the tip of Arnaldo's perf/core repository.
The patch set implements runtime trace compression for record mode and trace file decompression for report mode. Zstandard API [1] is used for compression/decompression of data that come from perf_events kernel data buffers.
Realized -z,--compression_level=n option provides ~3-5x avg. trace file size reduction on variety of tested workloads what saves user storage space on larger server systems where trace file size can easily reach several tens or even hundreds of GiBs, especially when profiling with dwarf-based stacks and tracing of context-switches.
--mmap-flush option can be used to avoid compressing every single byte of data and increase compression ratio at the same time lowering tool runtime overhead.
$ tools/perf/perf record -z 1 -e cycles -- matrix.gcc $ tools/perf/perf record -z 1 --mmap-flush 1024 -e cycles -- matrix.gcc $ tools/perf/perf record -z 1 --mmap-flush 1024 --aio -e cycles -- matrix.gcc
The compression functionality can be disabled from the command line using NO_LIBZSTD define and Zstandard sources can be overridden using value of LIBZSTD_DIR define:
$ make -C tools/perf NO_LIBZSTD=1 clean all $ make -C tools/perf LIBZSTD_DIR=/path/to/zstd-1.3.7 clean all
---
Alexey Budankov (4):
feature: realize libzstd check, LIBZSTD_DIR and NO_LIBZSTD defines perf record: implement -z=<level> and --mmap-flush=<thres> options perf record: enable runtime trace compression
perf report: support record trace file decompression
tools/build/Makefile.feature | 6 +- tools/build/feature/Makefile | 6 +- tools/build/feature/test-all.c | 5 + tools/build/feature/test-libzstd.c | 12 + tools/perf/Documentation/perf-record.txt | 9 + tools/perf/Makefile.config | 20 ++ tools/perf/Makefile.perf | 3 + tools/perf/builtin-record.c | 167 +++++++++++--- tools/perf/builtin-report.c | 5 +- tools/perf/perf.h | 2 + tools/perf/util/env.h | 10 + tools/perf/util/event.c | 1 + tools/perf/util/event.h | 7 + tools/perf/util/evlist.c | 6 +- tools/perf/util/evlist.h | 3 +- tools/perf/util/header.c | 45 +++- tools/perf/util/header.h | 1 + tools/perf/util/mmap.c | 265 +++++++++++++--------- tools/perf/util/mmap.h | 31 ++- tools/perf/util/session.c | 271 ++++++++++++++++++++++- tools/perf/util/session.h | 26 +++ tools/perf/util/tool.h | 2 + 22 files changed, 742 insertions(+), 161 deletions(-) create mode 100644 tools/build/feature/test-libzstd.c
---
Changes in v2:
- moved compression/decompression code to session layer
- enabled allocation aio data buffers for compression
- enabled trace compression for serial trace streaming
PATCH v2 1/4 feature: realize libzstd check, LIBZSTD_DIR and NO_LIBZSTD defines
Implement libzstd feature check, NO_LIBZSTD and LIBZSTD_DIR defines to override Zstd library sources or disable the feature from the command line:
$ make -C tools/perf LIBZSTD_DIR=/root/abudanko/zstd-1.3.7 clean all $ make -C tools/perf NO_LIBZSTD=1 clean all
--- tools/build/Makefile.feature | 6 ++++-- tools/build/feature/Makefile | 6 +++++- tools/build/feature/test-all.c | 5 +++++ tools/build/feature/test-libzstd.c | 12 ++++++++++++ tools/perf/Makefile.config | 20 ++++++++++++++++++++ tools/perf/Makefile.perf | 3 +++ 6 files changed, 49 insertions(+), 3 deletions(-) create mode 100644 tools/build/feature/test-libzstd.c
tools/build/Makefile.feature +4 -2
@@ -71,7 +71,8 @@ FEATURE_TESTS_BASIC := \ sdt \ setns \ libopencsd \ - libaio + libaio \ + libzstd # FEATURE_TESTS_BASIC + FEATURE_TESTS_EXTRA is the complete list # of all feature tests
@@ -118,7 +119,8 @@ FEATURE_DISPLAY ?= \ lzma \ get_cpuid \ bpf \ - libaio + libaio \ + libzstd # Set FEATURE_CHECK_(C|LD)FLAGS-all for all FEATURE_TESTS features. # If in the future we need per-feature checks/flags for features not
tools/build/feature/Makefile +5 -1
@@ -62,7 +62,8 @@ FILES= \ test-clang.bin \ test-llvm.bin \ test-llvm-version.bin \ - test-libaio.bin + test-libaio.bin \ + test-libzstd.bin FILES := $(addprefix $(OUTPUT),$(FILES))
@@ -301,6 +302,9 @@ $(OUTPUT)test-clang.bin: $(OUTPUT)test-libaio.bin: $(BUILD) -lrt +$(OUTPUT)test-libzstd.bin: + $(BUILD) -lzstd + ############################### clean:
tools/build/feature/test-all.c +5 -0
@@ -178,6 +178,10 @@# include "test-libaio.c" #undef main +#define main main_test_zstd +# include "test-libzstd.c" +#undef main +int main (int argc,char *argv[]) {main_test_libpython ();
@@ -219,6 +223,7 @@ int main(int argc, char *argv[])main_test_setns ();main_test_libopencsd ();main_test_libaio (); +main_test_libzstd ();return 0 ; }
tools/build/feature/test-libzstd.c new file +12 -0
@@ -0,0 +1,12 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <zstd.h> + +int main (void ) +{ +ZSTD_CStream *cstream; + + cstream =ZSTD_createCStream (); +ZSTD_freeCStream (cstream); + +return 0 ; +}
tools/perf/Makefile.config +20 -0
@@ -145,6 +145,13 @@ endif FEATURE_CHECK_CFLAGS-libbabeltrace := $(LIBBABELTRACE_CFLAGS) FEATURE_CHECK_LDFLAGS-libbabeltrace := $(LIBBABELTRACE_LDFLAGS) -lbabeltrace-ctf +ifdef LIBZSTD_DIR + LIBZSTD_CFLAGS := -I$(LIBZSTD_DIR)/lib + LIBZSTD_LDFLAGS := -L$(LIBZSTD_DIR)/lib +endif +FEATURE_CHECK_CFLAGS-libzstd := $(LIBZSTD_CFLAGS) +FEATURE_CHECK_LDFLAGS-libzstd := $(LIBZSTD_LDFLAGS) + FEATURE_CHECK_CFLAGS-bpf = -I. -I$(srctree)/tools/include -I$(srctree)/tools/arch/$(SRCARCH)/include/uapi -I$(srctree)/tools/include/uapi # include ARCH specific config -include $(src-perf)/arch/$(SRCARCH)/Makefile
@@ -770,6 +777,19 @@ ifndef NO_LZMA endif endif +ifndef NO_LIBZSTD + ifeq ($(feature-libzstd), 1) + CFLAGS += -DHAVE_ZSTD_SUPPORT + CFLAGS += $(LIBZSTD_CFLAGS) + LDFLAGS += $(LIBZSTD_LDFLAGS) + EXTLIBS += -lzstd + $(call detected,CONFIG_ZSTD) + else + msg := $(warning No libzstd found, disables trace compression, please install libzstd-dev[el] and/or set LIBZSTD_DIR); + NO_LIBZSTD := 1 + endif +endif + ifndef NO_BACKTRACE ifeq ($(feature-backtrace), 1) CFLAGS += -DHAVE_BACKTRACE_SUPPORT
tools/perf/Makefile.perf +3 -0
@@ -108,6 +108,9 @@ include ../scripts/utilities.mak # streaming for record mode. Currently Posix AIO trace streaming is # supported only when linking with glibc. # +# Define NO_LIBZSTD if you do not want support of Zstandard based runtime +# trace compression in record mode. +# # As per kernel Makefile, avoid funny character set dependencies unexport LC_ALL
PATCH v2 2/4 perf record: implement -z=<level> and --mmap-flush=<thres> options
In reply to Alexey BudankovImplement -z,--compression_level=<n> and --mmap-flush=<dump_least_size> options as well as a special PERF_RECORD_COMPRESSED record that contains compressed parts of kernel data buffer.
Because compression requires auxilary memory to implement encoding of kernel data record->opts.nr_cblocks == -1 signifies to preallocate single AIO data buffer aio.data[0] without accompnying AIO control blocks.
---
Changes in v2:
- enabled allocation aio data buffers for compression
--- tools/perf/Documentation/perf-record.txt | 9 ++ tools/perf/builtin-record.c | 110 +++++++++++++++++++---- tools/perf/perf.h | 2 + tools/perf/util/env.h | 10 +++ tools/perf/util/event.c | 1 + tools/perf/util/event.h | 7 ++ tools/perf/util/evlist.c | 6 +- tools/perf/util/evlist.h | 3 +- tools/perf/util/header.c | 45 +++++++++- tools/perf/util/header.h | 1 + tools/perf/util/mmap.c | 98 ++++++++++++-------- tools/perf/util/mmap.h | 7 +- tools/perf/util/session.h | 2 + 13 files changed, 240 insertions(+), 61 deletions(-)
tools/perf/Documentation/perf-record.txt +9 -0
@@ -459,6 +459,15 @@ Set affinity mask of trace reading thread according to the policy defined by 'mo node - thread affinity mask is set to NUMA node cpu mask of the processed mmap buffer cpu - thread affinity mask is set to cpu of the processed mmap buffer +-z:: +--compression-level=n:: +Produce compressed trace file using specified level n to save storage space (no compression: 0 - default, +fastest compression: 1, smallest trace file: 22) + +--mmap-flush=n:: +Minimal number of bytes accumulated in kernel buffer that is flushed to trace file (default: 1). +Maximal allowed value is a quater of kernel buffer size. + --all-kernel:: Configure all used events to run in kernel space.
tools/perf/builtin-record.c +93 -17
223 lines not on this page · raw message
tools/perf/perf.h +2 -0
@@ -85,6 +85,8 @@ struct record_opts {u64 clockid_res_ns;int nr_cblocks;int affinity; +unsigned int comp_level; +int mmap_flush; };enum perf_affinity {
tools/perf/util/env.h +10 -0
@@ -64,6 +64,16 @@ struct perf_env {struct memory_node *memory_nodes;unsigned long long memory_bsize;u64 clockid_res_ns; +u32 comp_type; +u32 comp_level; +u32 comp_ratio; +u32 comp_mmap_len; +}; + +enum perf_compress_type { +PERF_COMP_NONE =0 , +PERF_COMP_ZSTD , +PERF_COMP_MAX };extern struct perf_env perf_env;
tools/perf/util/event.c +1 -0
@@ -68,6 +68,7 @@ static const char *perf_event__names[] = { [PERF_RECORD_EVENT_UPDATE] = "EVENT_UPDATE", [PERF_RECORD_TIME_CONV] = "TIME_CONV", [PERF_RECORD_HEADER_FEATURE] = "FEATURE", + [PERF_RECORD_COMPRESSED] = "COMPRESSED", };static const char *perf_ns__names[] = {
tools/perf/util/event.h +7 -0
@@ -254,6 +254,7 @@ enum perf_user_event_type { /* above any possible kernel type */PERF_RECORD_EVENT_UPDATE =78 ,PERF_RECORD_TIME_CONV =79 ,PERF_RECORD_HEADER_FEATURE =80 , +PERF_RECORD_COMPRESSED =81 , PERF_RECORD_HEADER_MAX };
@@ -626,6 +627,11 @@ struct feature_event {char data[]; }; +struct compressed_event { +struct perf_event_header header; +char data[]; +}; +union perf_event {struct perf_event_header header;struct mmap_event mmap;
@@ -659,6 +665,7 @@ union perf_event {struct feature_event feat;struct ksymbol_event ksymbol_event;struct bpf_event bpf_event; +struct compressed_event pack; };void perf_event__print_totals (void );
tools/perf/util/evlist.c +3 -3
@@ -1022,7 +1022,7 @@ int perf_evlist__parse_mmap_pages(const struct option *opt, const char *str,*/ int perf_evlist__mmap_ex (struct perf_evlist *evlist,unsigned int pages,unsigned int auxtrace_pages, -bool auxtrace_overwrite,int nr_cblocks,int affinity) +bool auxtrace_overwrite,int nr_cblocks,int affinity,int flush) {struct perf_evsel *evsel;const struct cpu_map *cpus = evlist->cpus;
@@ -1032,7 +1032,7 @@ int perf_evlist__mmap_ex(struct perf_evlist *evlist, unsigned int pages,* Its value is decided by evsel's write_backward. * So &mp should not be passed through const pointer. */ -struct mmap_params mp = { .nr_cblocks = nr_cblocks, .affinity = affinity }; +struct mmap_params mp = { .nr_cblocks = nr_cblocks, .affinity = affinity, .flush = flush };if (!evlist->mmap) evlist->mmap =perf_evlist__alloc_mmap (evlist, false);
@@ -1064,7 +1064,7 @@ int perf_evlist__mmap_ex(struct perf_evlist *evlist, unsigned int pages,int perf_evlist__mmap (struct perf_evlist *evlist,unsigned int pages) { -return perf_evlist__mmap_ex (evlist, pages,0 , false,0 ,PERF_AFFINITY_SYS ); +return perf_evlist__mmap_ex (evlist, pages,0 , false,0 ,PERF_AFFINITY_SYS ,1 ); }int perf_evlist__create_maps (struct perf_evlist *evlist,struct target *target)
tools/perf/util/evlist.h +2 -1
@@ -165,7 +165,8 @@ unsigned long perf_event_mlock_kb_in_pages(void);int perf_evlist__mmap_ex (struct perf_evlist *evlist,unsigned int pages,unsigned int auxtrace_pages, -bool auxtrace_overwrite,int nr_cblocks,int affinity); +bool auxtrace_overwrite,int nr_cblocks, +int affinity,int flush);int perf_evlist__mmap (struct perf_evlist *evlist,unsigned int pages);void perf_evlist__munmap (struct perf_evlist *evlist);
tools/perf/util/header.c +44 -1
69 lines not on this page · raw message
tools/perf/util/header.h +1 -0
@@ -39,6 +39,7 @@ enum {HEADER_SAMPLE_TIME ,HEADER_MEM_TOPOLOGY ,HEADER_CLOCKID , +HEADER_COMPRESSED ,HEADER_LAST_FEATURE , HEADER_FEAT_BITS = 256, };
tools/perf/util/mmap.c +62 -36
159 lines not on this page · raw message
tools/perf/util/mmap.h +4 -3
@@ -30,14 +30,15 @@ struct perf_mmap {bool overwrite;struct auxtrace_mmap auxtrace_mmap;char event_copy[PERF_SAMPLE_MAX_SIZE ]__aligned(8) ; -#ifdef HAVE_AIO_SUPPORT +u64 flush;struct {void **data; +#ifdef HAVE_AIO_SUPPORT struct aiocb *cblocks;struct aiocb **aiocb; +#endif int nr_cblocks; } aio; -#endif cpu_set_t affinity_mask; };
@@ -70,7 +71,7 @@ enum bkw_mmap_state { };struct mmap_params { -int prot, mask, nr_cblocks, affinity; +int prot, mask, nr_cblocks, affinity, flush;struct auxtrace_mmap_params auxtrace_mp; };
tools/perf/util/session.h +2 -0
@@ -35,6 +35,8 @@ struct perf_session {struct ordered_events ordered_events;struct perf_data *data;struct perf_tool *tool; +u64 bytes_transferred; +u64 bytes_compressed; };struct perf_tool ;
PATCH v2 3/4 perf record: enable runtime trace compression
In reply to Alexey BudankovCompression is implemented using simple Zstd API and employs AIO data buffer as the memory to operate on. If the API call fails for some reason compression falls back to memcpy().
Data chunks are split and packed into PERF_RECORD_COMPRESSED records by 64KB at max. mmap-flush option value can be used to avoid compression of every single byte of data and increase compression ratio.
---
Changes in v2:
- enabled trace compression for serial trace streaming
- moved compression/decompression code to session layer
--- tools/perf/builtin-record.c | 67 +++++++++----- tools/perf/util/mmap.c | 173 +++++++++++++++++++++--------------- tools/perf/util/mmap.h | 24 +++-- tools/perf/util/session.c | 106 ++++++++++++++++++++++ tools/perf/util/session.h | 13 +++ 5 files changed, 280 insertions(+), 103 deletions(-)
tools/perf/builtin-record.c +43 -24
@@ -112,8 +112,7 @@ static bool switch_output_time(struct record *rec)trigger_is_ready (&switch_output_trigger); } -static int record__write (struct record *rec,struct perf_mmap *map__maybe_unused , -void *bf,size_t size) +static int record__write (struct record *rec,void *bf,size_t size) {struct perf_data_file *file = &rec->session->data->file;
@@ -237,7 +236,7 @@ static int record__aio_sync(struct perf_mmap *md, bool sync_all) }while (1 ); } -static int record__aio_pushfn (void *to,struct aiocb *cblock,void *bf,size_t size,off_t off) +static int record__aio_pushfn (void *to,void *bf,size_t size,off_t off,struct aiocb *cblock) {struct record *rec = to;int ret, trace_fd = rec->session->data->file.fd;
@@ -264,13 +263,15 @@ static void record__aio_set_pos(int trace_fd, off_t pos)lseek (trace_fd, pos,SEEK_SET ); } +static int record__aio_enabled (struct record *rec); +static void record__aio_mmap_read_sync (struct record *rec) {int i;struct perf_evlist *evlist = rec->evlist;struct perf_mmap *maps = evlist->mmap; -if (!rec->opts.nr_cblocks) +if (!record__aio_enabled (rec))return ;for (i =0 ; i < evlist->nr_mmaps; i++) {
@@ -311,8 +312,8 @@ static int record__aio_sync(struct perf_mmap *md __maybe_unused, bool sync_all _return -1 ; } -static int record__aio_pushfn (void *to__maybe_unused ,struct aiocb *cblock__maybe_unused , -void *bf__maybe_unused ,size_t size__maybe_unused ,off_t off__maybe_unused ) +static int record__aio_pushfn (void *to__maybe_unused ,void *bf__maybe_unused , +size_t size__maybe_unused ,off_t off__maybe_unused ,struct aiocb *cblock__maybe_unused ) {return -1 ; }
@@ -371,15 +372,15 @@ static int process_synthesized_event(struct perf_tool *tool, struct machine *machine __maybe_unused) { struct record *rec = container_of(tool, struct record, tool); -return record__write (rec,NULL , event, event->header.size); +return record__write (rec, event, event->header.size); } -static int record__pushfn (struct perf_mmap *map,void *to,void *bf,size_t size) +static int record__pushfn (void *to,void *bf,size_t size) {struct record *rec = to; rec->samples++; -return record__write (rec, map, bf, size); +return record__write (rec, bf, size); }static volatile int done;
@@ -414,7 +415,7 @@ static void record__sig_exit(void)#ifdef HAVE_AUXTRACE_SUPPORT static int record__process_auxtrace (struct perf_tool *tool, -struct perf_mmap *map, +struct perf_mmap *map__maybe_unused ,union perf_event *event,void *data1,size_t len1,void *data2,size_t len2) {
@@ -442,11 +443,11 @@ static int record__process_auxtrace(struct perf_tool *tool,if (padding) padding =8 - padding; -record__write (rec, map, event, event->header.size); -record__write (rec, map, data1, len1); +record__write (rec, event, event->header.size); +record__write (rec, data1, len1);if (len2) -record__write (rec, map, data2, len2); -record__write (rec, map, &pad, padding); +record__write (rec, data2, len2); +record__write (rec, &pad, padding);return 0 ; }
@@ -774,6 +775,8 @@ static int record__mmap_read_evlist(struct record *rec, struct perf_evlist *evlistruct perf_mmap *maps;int trace_fd = rec->data.file.fd;off_t off; +struct perf_session *session = rec->session; +perf_mmap__compress_fn_t compress_fn;if (!evlist)return 0 ;
@@ -785,6 +788,9 @@ static int record__mmap_read_evlist(struct record *rec, struct perf_evlist *evliif (overwrite && evlist->bkw_mmap_state !=BKW_MMAP_DATA_PENDING )return 0 ; + compress_fn = (record__comp_enabled (rec) ? + perf_session__zstd_compress : perf_session__zstd_copy); +if (record__aio_enabled (rec)) off =record__aio_get_pos (trace_fd);
@@ -799,11 +805,21 @@ static int record__mmap_read_evlist(struct record *rec, struct perf_evlist *evli map->flush =MMAP_FLUSH_DEFAULT ; }if (!record__aio_enabled (rec)) { -if (perf_mmap__push (map, rec, record__pushfn) !=0 ) { -if (sync) - map->flush = flush; - rc =-1 ; - goto out; +if (!record__comp_enabled (rec)) { +if (perf_mmap__push (map, rec, record__pushfn) !=0 ) { +if (sync) + map->flush = flush; + rc =-1 ; + goto out; + } + }else { +if (perf_mmap__pack (map, rec, record__pushfn, + compress_fn, session) !=0 ) { +if (sync) + map->flush = flush; + rc =-1 ; + goto out; + } } }else {int idx;
@@ -812,7 +828,8 @@ static int record__mmap_read_evlist(struct record *rec, struct perf_evlist *evli* becomes available after previous aio write request. */ idx =record__aio_sync (map, false); -if (perf_mmap__aio_push (map, rec, idx, record__aio_pushfn, &off) !=0 ) { +if (perf_mmap__aio_push (map, rec, record__aio_pushfn, + &off, idx, compress_fn, session) !=0 ) {record__aio_set_pos (trace_fd, off);if (sync) map->flush = flush;
@@ -839,7 +856,7 @@ static int record__mmap_read_evlist(struct record *rec, struct perf_evlist *evli* at least one event. */ if (bytes_written != rec->bytes_written) - rc =record__write (rec,NULL , &finished_round_event,sizeof (finished_round_event)); + rc =record__write (rec, &finished_round_event,sizeof (finished_round_event));if (overwrite)perf_evlist__toggle_bkw_mmap (evlist,BKW_MMAP_EMPTY );
@@ -1193,9 +1210,10 @@ static int __cmd_record(struct record *rec, int argc, const char **argv) fd =perf_data__fd (data); rec->session = session; - rec->opts.comp_level =0 ; - session->header.env.comp_level = rec->opts.comp_level; - session->header.env.comp_type =PERF_COMP_NONE ; +if (perf_session__zstd_init (session, rec->opts.comp_level) <0 ) { +pr_err ("Compression initialization failed.\n" ); +return -1 ; + }record__init_features (rec);
@@ -1526,6 +1544,7 @@ static int __cmd_record(struct record *rec, int argc, const char **argv) } out_delete_session: +perf_session__zstd_fini (session);perf_session__delete (session);return status; }
tools/perf/util/mmap.c +102 -71
205 lines not on this page · raw message
tools/perf/util/mmap.h +16 -8
@@ -97,16 +97,24 @@ union perf_event *perf_mmap__read_forward(struct perf_mmap *map);union perf_event *perf_mmap__read_event (struct perf_mmap *map); -int perf_mmap__push (struct perf_mmap *md,void *to, -int push (struct perf_mmap *map,void *to,void *buf,size_t size)); +typedef int (*perf_mmap__push_fn_t )(void *to,void *buf,size_t size); +int perf_mmap__push (struct perf_mmap *md,void *to,perf_mmap__push_fn_t push); + +typedef size_t (*perf_mmap__compress_fn_t )(void *where,void *dst,size_t dst_size, +void *src,size_t src_size); +int perf_mmap__pack (struct perf_mmap *md,void *to,perf_mmap__push_fn_t push, +perf_mmap__compress_fn_t compress,void *where); + +typedef int (*perf_mmap__aio_push_fn_t )(void *to,void *buf,size_t size, +off_t push_off,struct aiocb *cblock); +#ifdef HAVE_AIO_SUPPORT -int perf_mmap__aio_push (struct perf_mmap *md,void *to,int idx, -int push (void *to,struct aiocb *cblock,void *buf,size_t size,off_t off), -off_t *off); +int perf_mmap__aio_push (struct perf_mmap *md,void *to,perf_mmap__aio_push_fn_t push,off_t *push_off, +int idx,perf_mmap__compress_fn_t compress_fn,void *where);#else -static inline int perf_mmap__aio_push (struct perf_mmap *md__maybe_unused ,void *to__maybe_unused ,int idx__maybe_unused , -int push (void *to,struct aiocb *cblock,void *buf,size_t size,off_t off)__maybe_unused , -off_t *off__maybe_unused ) +static inline int perf_mmap__aio_push (struct perf_mmap *md__maybe_unused ,void *to__maybe_unused , +perf_mmap__aio_push_fn_t push__maybe_unused ,off_t *push_off__maybe_unused , +int idx__maybe_unused ,perf_mmap__compress_fn_t compress__maybe_unused ,void *where__maybe_unused ) {return 0 ; }
tools/perf/util/session.c +106 -0
112 lines not on this page · raw message
tools/perf/util/session.h +13 -0
@@ -11,6 +11,9 @@#include <linux/kernel.h> #include <linux/rbtree.h> #include <linux/perf_event.h> +#ifdef HAVE_ZSTD_SUPPORT +#include <zstd.h> +#endif struct ip_callchain ;struct symbol ;
@@ -37,6 +40,9 @@ struct perf_session {struct perf_tool *tool;u64 bytes_transferred;u64 bytes_compressed; +#ifdef HAVE_ZSTD_SUPPORT +ZSTD_CStream *zstd_cstream; +#endif };struct perf_tool ;
@@ -122,6 +128,13 @@ int perf_session__deliver_synth_event(struct perf_session *session,union perf_event *event, struct perf_sample *sample); +int perf_session__zstd_init (struct perf_session *session,int level); +int perf_session__zstd_fini (struct perf_session *session); +size_t perf_session__zstd_copy (void *to,void *dst,size_t dst_size, +void *src,size_t src_size); +size_t perf_session__zstd_compress (void *to,void *dst,size_t dst_size, +void *src,size_t src_size); +int perf_event__process_id_index (struct perf_session *session,union perf_event *event);
PATCH v2 4/4 perf report: support record trace file decompression
In reply to Alexey BudankovPERF_RECORD_COMPRESSED records are decompressed from trace file into a linked list of mmaped memory regions using streaming Zstandard API. After that the regions are loaded fetching uncompressed events. When dumping raw trace (e.g., perf report -D --header) file offsets of events from compressed records are set to zero.
---
Changes in v2:
- moved compression/decompression code to session layer
--- tools/perf/builtin-report.c | 5 +- tools/perf/util/session.c | 165 +++++++++++++++++++++++++++++++++++- tools/perf/util/session.h | 11 +++ tools/perf/util/tool.h | 2 + 4 files changed, 181 insertions(+), 2 deletions(-)
tools/perf/builtin-report.c +4 -1
@@ -1215,6 +1215,9 @@ int cmd_report(int argc, const char **argv)if (session ==NULL )return -1 ; +if (perf_session__zstd_init (session,0 ) <0 ) +pr_warning ("Decompression initialization failed. Reported data may be incomplete.\n" ); +if (report.queue_size) {ordered_events__set_alloc_size (&session->ordered_events, report.queue_size);
@@ -1427,7 +1430,7 @@ int cmd_report(int argc, const char **argv) error:zfree (&report.ptime_range); - +perf_session__zstd_fini (session);perf_session__delete (session);return ret; }
tools/perf/util/session.c +164 -1
231 lines not on this page · raw message
tools/perf/util/session.h +11 -0
@@ -42,7 +42,18 @@ struct perf_session {u64 bytes_compressed;#ifdef HAVE_ZSTD_SUPPORT ZSTD_CStream *zstd_cstream; +ZSTD_DStream *zstd_dstream;#endif +struct decomp *decomp; +struct decomp *decomp_last; +}; + +struct decomp { +struct decomp *next; +u64 file_pos; +u64 head; +size_t size; +char data[]; };struct perf_tool ;
tools/perf/util/tool.h +2 -0
@@ -28,6 +28,7 @@ typedef int (*event_attr_op)(struct perf_tool *tool,typedef int (*event_op2 )(struct perf_session *session,union perf_event *event);typedef s64 (*event_op3 )(struct perf_session *session,union perf_event *event); +typedef int (*event_op4 )(struct perf_session *session,union perf_event *event,u64 data);typedef int (*event_oe )(struct perf_tool *tool,union perf_event *event,struct ordered_events *oe);
@@ -72,6 +73,7 @@ struct perf_tool { stat, stat_round, feature; +event_op4 compressed;event_op3 auxtrace;bool ordered_events;bool ordering_requires_timestamps;
Arnaldo Carvalho de Melo
maintainerEm Mon, Feb 11, 2019 at 11:17:14PM +0300, Alexey Budankov escreveu:
Quoted 8 lines
This is the rebase to the tip of Arnaldo's perf/core repository.
The patch set implements runtime trace compression for record mode and trace file decompression for report mode. Zstandard API [1] is used for compression/decompression of data that come from
perf_eventskernel data buffers.Realized -z,--compression_level=n option provides ~3-5x avg. trace file
Realized? Perhaps "Using the -z/--compression_level=n options provides 3.5x average size reduction on..."
Quoted 6 lines
size reduction on variety of tested workloads what saves user storage space on larger server systems where trace file size can easily reach several tens or even hundreds of GiBs, especially when profiling with dwarf-based stacks and tracing of context-switches.
--mmap-flush option can be used to avoid compressing every single byte
What is the default for --mmap-flush when -z is used? Does it vary with the compression level? Can we expect something that is reasonable or is using --mmap-flush always needed? If so can we do away with that by using some sensible default according to the -z level used?
Quoted 6 lines
of data and increase compression ratio at the same time lowering tool runtime overhead.
$ tools/perf/perf record -z 1 -e cycles -- matrix.gcc $ tools/perf/perf record -z 1 --mmap-flush 1024 -e cycles -- matrix.gcc $ tools/perf/perf record -z 1 --mmap-flush 1024 --aio -e cycles -- matrix.gcc
Please avoid having these explanations just on the cover letter since it will not get recorded in the git history for the project, please add examples with output from real workloads, preferably providing real cases so that other people can readily replicate your results when having access to similar large machines.
Quoted 6 lines
The compression functionality can be disabled from the command line using
NO_LIBZSTDdefine and Zstandard sources can be overridden using value ofLIBZSTD_DIRdefine:$ make -C tools/perf NO_LIBZSTD=1 clean all $ make -C tools/perf LIBZSTD_DIR=/path/to/zstd-1.3.7 clean all
Showing the relevant output of the these commands before/after installing the libzstd library is useful as well. From what you use above is it fair to conclude that there are no readily made packages for this library and we need to get it from github and build it ourselves?
On my notebook, a Fedora 29 system:
[root@quaco ~]# dnf search zstd
========================================================== Name & Summary Matched: zstd ========================================================== zstd.x86_64 : Zstd compression library
zstd.x86_64 : Zstd compression library
libzstd.x86_64 : Zstd shared library
libzstd.i686 : Zstd shared library
libzstd.x86_64 : Zstd shared library
libzstd-devel.i686 : Header files for Zstd library
libzstd-devel.x86_64 : Header files for Zstd library
============================================================= Summary Matched: zstd ============================================================== fedora-repo-zdicts.noarch : Zstd dictionaries for Fedora repository metadata
[root@quaco ~]#
[root@quaco ~]# dnf install libzstd-devel
<SNIP>
libzstd-devel x86_64 1.3.8-1.fc29 updates 39 k
<SNIP>
So, do I need some specific 1.3.7 version? Is that why you used that LIBZSTD_DIR in your example?
I'm trying these steps, but please consider following these suggestions in future posts,
And thanks for working on this!
- Arnaldo
Quoted 40 lines
---
Alexey Budankov (4):
feature: realize libzstd check,LIBZSTD_DIRandNO_LIBZSTDdefines perf record: implement -z=<level> and --mmap-flush=<thres> options perf record: enable runtime trace compression
perf report: support record trace file decompressiontools/build/Makefile.feature | 6 +- tools/build/feature/Makefile | 6 +- tools/build/feature/test-all.c | 5 + tools/build/feature/test-libzstd.c | 12 + tools/perf/Documentation/perf-record.txt | 9 + tools/perf/Makefile.config | 20 ++ tools/perf/Makefile.perf | 3 + tools/perf/builtin-record.c | 167 +++++++++++--- tools/perf/builtin-report.c | 5 +- tools/perf/perf.h | 2 + tools/perf/util/env.h | 10 + tools/perf/util/event.c | 1 + tools/perf/util/event.h | 7 + tools/perf/util/evlist.c | 6 +- tools/perf/util/evlist.h | 3 +- tools/perf/util/header.c | 45 +++- tools/perf/util/header.h | 1 + tools/perf/util/mmap.c | 265 +++++++++++++--------- tools/perf/util/mmap.h | 31 ++- tools/perf/util/session.c | 271 ++++++++++++++++++++++- tools/perf/util/session.h | 26 +++ tools/perf/util/tool.h | 2 + 22 files changed, 742 insertions(+), 161 deletions(-) create mode 100644 tools/build/feature/test-libzstd.c---
Changes in v2:
- moved compression/decompression code to session layer
- enabled allocation aio data buffers for compression
- enabled trace compression for serial trace streaming
Signature · 1 line
- Arnaldo
Re: [PATCH v2 2/4] perf record: implement -z=<level> and --mmap-flush=<thres> options
In reply to Alexey BudankovOn Mon, Feb 11, 2019 at 11:22:38PM +0300, Alexey Budankov wrote:
SNIP
Quoted 45 lines
static int process_synthesized_event(struct
perf_tool*tool,tools/perf/builtin-record.c +10
-2union perf_event *event, struct perf_sample *sample __maybe_unused,@@ -543,7 +574,8 @@ static int record__mmap_evlist(struct record *rec,if (perf_evlist__mmap_ex (evlist, opts->mmap_pages, opts->auxtrace_mmap_pages, opts->auxtrace_snapshot_mode, - opts->nr_cblocks, opts->affinity) <0 ) { + opts->nr_cblocks, opts->affinity, + opts->mmap_flush) <0 ) {if (errno ==EPERM ) {pr_err ("Permission error mapping pages.\n" "Consider increasing "@@ -734,7 +766,7 @@ static void record__adjust_affinity(struct record *rec, struct perf_mmap *map) }static int record__mmap_read_evlist (struct record *rec,struct perf_evlist *evlist, -bool overwrite) +bool overwrite,bool sync) {u64 bytes_written = rec->bytes_written;int i;@@ -757,12 +789,19 @@ static int record__mmap_read_evlist(struct record *rec, struct perf_evlist *evli off =record__aio_get_pos (trace_fd);for (i =0 ; i < evlist->nr_mmaps; i++) { +u64 flush =MMAP_FLUSH_DEFAULT ;struct perf_mmap *map = &maps[i];if (map->base) {record__adjust_affinity (rec, map); +if (sync) { + flush = map->flush; + map->flush =MMAP_FLUSH_DEFAULT ; + }if (!record__aio_enabled (rec)) {if (perf_mmap__push (map, rec, record__pushfn) !=0 ) { +if (sync) + map->flush = flush; rc =-1 ; goto out; }@@ -775,10 +814,14 @@ static int record__mmap_read_evlist(struct record *rec, struct perf_evlist *evli idx =record__aio_sync (map, false);if (perf_mmap__aio_push (map, rec, idx, record__aio_pushfn, &off) !=0 ) {
tools/perf/builtin-record.c static int record__mmap_read_evlist(struct record *rec, struct perf_evlist *evli · line 825
Quoted 10 lines
tools/perf/builtin-record.c +4
-0record__aio_set_pos (trace_fd, off); +if (sync) + map->flush = flush; rc =-1 ; goto out; } } +if (sync) + map->flush = flush; }
what's 'flush' and 'sync' for? also the above handling seems confusing, why would you set it temporarily for default value if you let it set by user command line option?
SNIP
Quoted 15 lines
tools/perf/util/mmap.c +1
-1-static void perf_mmap__aio_munmap (struct perf_mmap *map__maybe_unused ) +static void perf_mmap__aio_munmap_blocks (struct perf_mmap *map__maybe_unused ) {}
#endiftools/perf/util/mmap.c +2
-0@@ -444,6 +468,8 @@ int perf_mmap__mmap(struct perf_mmap *map, struct mmap_params *mp, int fd, int c &mp->auxtrace_mp, map->base, fd))return -1 ; + map->flush = mp->flush; +return perf_mmap__aio_mmap (map, mp); }
tools/perf/util/mmap.c static int __perf_mmap__read_init(struct perf_mmap *md) · line 522
Quoted 6 lines
tools/perf/util/mmap.c +1
-1@@ -492,7 +518,7 @@ static int __perf_mmap__read_init(struct perf_mmap *md) md->start = md->overwrite ? head : old; md->end = md->overwrite ? old : head; -if (md->start == md->end) +if ((md->end - md->start) < md->flush)return -EAGAIN ;
we need document and explain this change in changelog in separate patch
thanks,
jirka
Re: [PATCH v2 2/4] perf record: implement -z=<level> and --mmap-flush=<thres> options
In reply to Alexey BudankovOn Mon, Feb 11, 2019 at 11:22:38PM +0300, Alexey Budankov wrote:
SNIP
tools/perf/util/session.h struct perf_session { · line 39
Quoted 10 lines
tools/perf/util/session.h +2
-0@@ -35,6 +35,8 @@ struct perf_session {struct ordered_events ordered_events;struct perf_data *data;struct perf_tool *tool; +u64 bytes_transferred; +u64 bytes_compressed;
please add those in separate patch, so we can see how they are incremented and displayed easily
thanks,
jirka
Re: [PATCH v2 2/4] perf record: implement -z=<level> and --mmap-flush=<thres> options
In reply to Alexey BudankovOn Mon, Feb 11, 2019 at 11:22:38PM +0300, Alexey Budankov wrote:
SNIP
Quoted 52 lines
tools/perf/util/mmap.c +8
-27+static int perf_mmap__aio_mmap_blocks (struct perf_mmap *map); +static int perf_mmap__aio_mmap (struct perf_mmap *map,struct mmap_params *mp) { -int delta_max, i, prio, ret; +int i, ret =0 , init_blocks =1 ; map->aio.nr_cblocks = mp->nr_cblocks; +if (map->aio.nr_cblocks ==-1 ) { + map->aio.nr_cblocks =1 ; + init_blocks =0 ; + } +if (map->aio.nr_cblocks) { - map->aio.aiocb =calloc (map->aio.nr_cblocks,sizeof (struct aiocb *)); -if (!map->aio.aiocb) { -pr_debug2 ("failed to allocate aiocb for data buffer, error %m\n" ); -return -1 ; - } - map->aio.cblocks =calloc (map->aio.nr_cblocks,sizeof (struct aiocb )); -if (!map->aio.cblocks) { -pr_debug2 ("failed to allocate cblocks for data buffer, error %m\n" ); -return -1 ; - } map->aio.data =calloc (map->aio.nr_cblocks,sizeof (void *));if (!map->aio.data) {pr_debug2 ("failed to allocate data buffer, error %m\n" );return -1 ; } - delta_max =sysconf (_SC_AIO_PRIO_DELTA_MAX);for (i =0 ; i < map->aio.nr_cblocks; ++i) { ret =perf_mmap__aio_alloc (map, i);if (ret ==-1 ) {@@ -251,29 +245,16 @@ static int perf_mmap__aio_mmap(struct perf_mmap *map, struct mmap_params *mp) ret =perf_mmap__aio_bind (map, i, map->cpu, mp->affinity);if (ret ==-1 )return -1 ; -/* -* Use cblock.aio_fildes value different from -1 -* to denote started aio write operation on the -* cblock so it requires explicit record__aio_sync() -* call prior the cblock may be reused again. -*/ - map->aio.cblocks[i].aio_fildes =-1 ; -/* -* Allocate cblocks with priority delta to have -* faster aio write system calls because queued requests -* are kept in separate per-prio queues and adding -* a new request will iterate thru shorter per-prio -* list. Blocks with numbers higher than -* _SC_AIO_PRIO_DELTA_MAX go with priority 0. -*/
tools/perf/util/mmap.c static int perf_mmap__aio_mmap(struct perf_mmap *map, struct mmap_params *mp) · line 254
Quoted 10 lines
tools/perf/util/mmap.c +3
-3- prio = delta_max - i; - map->aio.cblocks[i].aio_reqprio = prio >=0 ? prio :0 ; } +if (init_blocks) + ret =perf_mmap__aio_mmap_blocks (map); } -return 0 ; +return ret; }
SNIP
it seems like little refactoring happened in here (up and down) for aio code, which is not explained and I'm unable to follow it.. please separate this in simple change
SNIP
Quoted 39 lines
tools/perf/util/mmap.c +39
-0+#ifdef HAVE_AIO_SUPPORT +static int perf_mmap__aio_mmap_blocks (struct perf_mmap *map) +{ +int delta_max, i, prio; + + map->aio.aiocb =calloc (map->aio.nr_cblocks,sizeof (struct aiocb *)); +if (!map->aio.aiocb) { +pr_debug2 ("failed to allocate aiocb for data buffer, error %m\n" ); +return -1 ; + } + map->aio.cblocks =calloc (map->aio.nr_cblocks,sizeof (struct aiocb )); +if (!map->aio.cblocks) { +pr_debug2 ("failed to allocate cblocks for data buffer, error %m\n" ); +return -1 ; + } + delta_max =sysconf (_SC_AIO_PRIO_DELTA_MAX); +for (i =0 ; i < map->aio.nr_cblocks; ++i) { +/* +* Use cblock.aio_fildes value different from -1 +* to denote started aio write operation on the +* cblock so it requires explicit record__aio_sync() +* call prior the cblock may be reused again. +*/ + map->aio.cblocks[i].aio_fildes =-1 ; +/* +* Allocate cblocks with priority delta to have +* faster aio write system calls because queued requests +* are kept in separate per-prio queues and adding +* a new request will iterate thru shorter per-prio +* list. Blocks with numbers higher than +* _SC_AIO_PRIO_DELTA_MAX go with priority 0. +*/ + prio = delta_max - i; + map->aio.cblocks[i].aio_reqprio = prio >=0 ? prio :0 ; + } + +return 0 ; +} +
SNIP
Re: [PATCH v2 2/4] perf record: implement -z=<level> and --mmap-flush=<thres> options
In reply to Alexey BudankovOn Mon, Feb 11, 2019 at 11:22:38PM +0300, Alexey Budankov wrote:
SNIP
tools/perf/builtin-record.c int cmd_record(int argc, const char **argv) · line 2255
Quoted 12 lines
tools/perf/builtin-record.c +8
-4-if (rec->opts.nr_cblocks > nr_cblocks_max) - rec->opts.nr_cblocks = nr_cblocks_max; -if (verbose >0 ) -pr_info ("nr_cblocks: %d\n" , rec->opts.nr_cblocks); +if (rec->opts.comp_level >22 ) + rec->opts.comp_level =0 ; +if (record__comp_enabled (rec) && !rec->opts.nr_cblocks) { +/* +* Allocate aio.data[0] buffer for compression. +*/ + rec->opts.nr_cblocks =-1 ; + }
I assume you're using aio.data[0] as a buffer for non-aio case as compression buffer.. if that's the case, please dont do that and use separate buffer for that
jirka
Re: [PATCH v2 2/4] perf record: implement -z=<level> and --mmap-flush=<thres> options
In reply to Alexey BudankovOn Mon, Feb 11, 2019 at 11:22:38PM +0300, Alexey Budankov wrote:
SNIP
Quoted 12 lines
static void
record__init_features(struct record *rec)tools/perf/builtin-record.c +3
-0@@ -838,6 +881,9 @@ static void record__init_features(struct record *rec)if (!(rec->opts.use_clockid && rec->opts.clockid_res_ns))perf_header__clear_feat (&session->header,HEADER_CLOCKID ); +if (!record__comp_enabled (rec)) +perf_header__clear_feat (&session->header,HEADER_COMPRESSED ); +perf_header__clear_feat (&session->header,HEADER_STAT ); }
tools/perf/builtin-record.c static int __cmd_record(struct record *rec, int argc, const char **argv) · line 1198
Quoted 6 lines
tools/perf/builtin-record.c +3
-0@@ -1147,6 +1193,10 @@ static int __cmd_record(struct record *rec, int argc, const char **argv) fd =perf_data__fd (data); rec->session = session; + rec->opts.comp_level =0 ; + session->header.env.comp_level = rec->opts.comp_level; + session->header.env.comp_type =PERF_COMP_NONE ;
hum I think both record and session are zerod from start, so this seems superfluous
SNIP
Quoted 9 lines
tools/perf/util/header.c +1
-0@@ -1463,6 +1463,21 @@ static int write_mem_topology(struct feat_fd *ff __maybe_unused,return ret; } +static int write_compressed(struct feat_fd *ff __maybe_unused,
tools/perf/util/header.c static int write_mem_topology(struct feat_fd *ff __maybe_unused, · line 1476
Quoted 10 lines
tools/perf/util/header.c +10
-0+ struct perf_evlist *evlist __maybe_unused) +{ +int ret; +u64 compression_info = ((u64 )ff->ph->env.comp_type <<32 ) | ff->ph->env.comp_level; + + ret =do_write (ff, &compression_info,sizeof (compression_info)); +if (ret) +return ret; + + compression_info = ((u64 )ff->ph->env.comp_ratio <<32 ) | ff->ph->env.comp_mmap_len;
let's not complicate this, if you have two u32 values to store, store two u32 values ;-)
also please add version in case we will add new fields in the future, so we can stay backward compatible
jirka
Re: [PATCH v2 3/4] perf record: enable runtime trace compression
In reply to Alexey BudankovOn Mon, Feb 11, 2019 at 11:23:40PM +0300, Alexey Budankov wrote:
SNIP
tools/perf/util/session.c line 31
Quoted 8 lines
tools/perf/util/session.c +0
-0@@ -29,6 +29,112 @@#include "stat.h" #include "arch/common.h"
I think this should not be in session.c but separated in new object
Quoted 106 lines
tools/perf/util/session.c +106
-0+#ifdef HAVE_ZSTD_SUPPORT +int perf_session__zstd_init (struct perf_session *session,int level) +{ +size_t ret; + + session->header.env.comp_type =PERF_COMP_NONE ; + session->header.env.comp_level =0 ; + + session->zstd_cstream =ZSTD_createCStream (); +if (session->zstd_cstream ==NULL ) { +pr_err ("Couldn't create compression stream.\n" ); +return -1 ; + } + + ret =ZSTD_initCStream (session->zstd_cstream, level); +if (ZSTD_isError (ret)) { +pr_err ("Failed to initialize compression stream: %s\n" ,ZSTD_getErrorName (ret)); +return -1 ; + } + + session->header.env.comp_type =PERF_COMP_ZSTD ; + session->header.env.comp_level = level; + +return 0 ; +} + +int perf_session__zstd_fini (struct perf_session *session) +{ +if (session->zstd_cstream) { +ZSTD_freeCStream (session->zstd_cstream); + session->zstd_cstream =NULL ; + } + +return 0 ; +} + +size_t perf_session__zstd_compress (void *to,void *dst,size_t dst_size, +void *src,size_t src_size) +{ +struct perf_session *session = to; +size_t ret, size, compressed =0 ; +struct compressed_event *event =NULL ; +/* maximum size of record data size (2^16 - 1 - header) */ +const size_t max_data_size = (1 <<8 *sizeof (event->header.size)) - +1 -sizeof (struct compressed_event ); +ZSTD_inBuffer input = { src, src_size,0 }; +ZSTD_outBuffer output; + +while (input.pos < input.size) { + event = dst; + + event->header.type =PERF_RECORD_COMPRESSED ; + event->header.size = size =sizeof (struct compressed_event ); + compressed += size; + dst += size; + dst_size -= size; + + output = (ZSTD_outBuffer ){ dst, (dst_size > max_data_size) ? + max_data_size : dst_size,0 }; + ret =ZSTD_compressStream (session->zstd_cstream, &output, &input); +ZSTD_flushStream (session->zstd_cstream, &output); +if (ZSTD_isError (ret)) { +pr_err ("failed to compress %ld bytes: %s\n" , + (long )src_size,ZSTD_getErrorName (ret)); +return perf_session__zstd_copy (session, dst, dst_size, src, src_size); + } + size = output.pos; + + event->header.size += size; + compressed += size; + dst += size; + dst_size -= size; + } + + session->bytes_transferred += src_size; + session->bytes_compressed += compressed; + +return compressed; +} +#else /* !HAVE_ZSTD_SUPPORT */ +int perf_session__zstd_init (struct perf_session *session__maybe_unused ,int level__maybe_unused ) +{ +return 0 ; +} + +int perf_session__zstd_fini (struct perf_session *session__maybe_unused ) +{ +return 0 ; +} + +size_t perf_session__zstd_compress (void *to__maybe_unused , +void *dst__maybe_unused ,size_t dst_size__maybe_unused , +void *src__maybe_unused ,size_t src_size__maybe_unused ) +{ +return 0 ; +} +#endif + +size_t perf_session__zstd_copy (void *to__maybe_unused , +void *dst,size_t dst_size__maybe_unused , +void *src,size_t src_size) +{ +memcpy (dst, src, src_size); +return src_size; +} +
SNIP
Re: [PATCH v2 4/4] perf report: support record trace file decompression
In reply to Alexey BudankovOn Mon, Feb 11, 2019 at 11:25:00PM +0300, Alexey Budankov wrote:
SNIP
Quoted 21 lines
size_tperf_session__zstd_copy(void *to__maybe_unused,tools/perf/util/session.c +4
-1@@ -533,6 +646,8 @@ void perf_tool__fill_defaults(struct perf_tool *tool) tool->time_conv = process_event_op2_stub;if (tool->feature ==NULL ) tool->feature = process_event_op2_stub; +if (tool->compressed ==NULL ) + tool->compressed = perf_session__process_compressed_event; }static void swap_sample_id_all (union perf_event *event,void *data)@@ -1469,7 +1584,8 @@ static s64 perf_session__process_user_event(struct perf_session *session,int fd =perf_data__fd (session->data);int err; -dump_event (session->evlist, event, file_offset, &sample); +if (event->header.type !=PERF_RECORD_COMPRESSED ) +dump_event (session->evlist, event, file_offset, &sample);/* These events are processed right away */ switch (event->header.type) {
tools/perf/util/session.c static s64 perf_session__process_user_event(struct perf_session *session, · line 1644
Quoted 7 lines
tools/perf/util/session.c +4
-0@@ -1522,6 +1638,11 @@ static s64 perf_session__process_user_event(struct perf_session *session,return tool->time_conv (session, event);case PERF_RECORD_HEADER_FEATURE :return tool->feature (session, event); +case PERF_RECORD_COMPRESSED : + err = tool->compressed (session, event, file_offset); +if (err) +dump_event (session->evlist, event, file_offset, &sample);
I'm not sure about having compressed callback at all, but maybe it could be usefull for inject, to get the compressed data..?
I assume inject is working now and it will get you the uncompressed perf.data?
jirka
Re: [PATCH v2 3/4] perf record: enable runtime trace compression
In reply to Alexey BudankovOn Mon, Feb 11, 2019 at 11:23:40PM +0300, Alexey Budankov wrote:
SNIP
Quoted 10 lines
tools/perf/builtin-record.c +2
-0@@ -774,6 +775,8 @@ static int record__mmap_read_evlist(struct record *rec, struct perf_evlist *evlistruct perf_mmap *maps;int trace_fd = rec->data.file.fd;off_t off; +struct perf_session *session = rec->session; +perf_mmap__compress_fn_t compress_fn;if (!evlist)return 0 ;
tools/perf/builtin-record.c static int record__mmap_read_evlist(struct record *rec, struct perf_evlist *evli · line 793
Quoted 6 lines
tools/perf/builtin-record.c +3
-0@@ -785,6 +788,9 @@ static int record__mmap_read_evlist(struct record *rec, struct perf_evlist *evliif (overwrite && evlist->bkw_mmap_state !=BKW_MMAP_DATA_PENDING )return 0 ; + compress_fn = (record__comp_enabled (rec) ? + perf_session__zstd_compress : perf_session__zstd_copy); +
I don't follow what's the perf_session__zstd_copy function for..?
for !record__comp_enabled case we seem not to use it and calling the current perf_mmap__push interface
however I dont see point to have this function at all
jirka
Quoted 27 lines
tools/perf/builtin-record.c +15
-5if (record__aio_enabled (rec)) off =record__aio_get_pos (trace_fd);@@ -799,11 +805,21 @@ static int record__mmap_read_evlist(struct record *rec, struct perf_evlist *evli map->flush =MMAP_FLUSH_DEFAULT ; }if (!record__aio_enabled (rec)) { -if (perf_mmap__push (map, rec, record__pushfn) !=0 ) { -if (sync) - map->flush = flush; - rc =-1 ; - goto out; +if (!record__comp_enabled (rec)) { +if (perf_mmap__push (map, rec, record__pushfn) !=0 ) { +if (sync) + map->flush = flush; + rc =-1 ; + goto out; + } + }else { +if (perf_mmap__pack (map, rec, record__pushfn, + compress_fn, session) !=0 ) { +if (sync) + map->flush = flush; + rc =-1 ; + goto out; + }
SNIP
Re: [PATCH v2 3/4] perf record: enable runtime trace compression
In reply to Alexey BudankovOn Mon, Feb 11, 2019 at 11:23:40PM +0300, Alexey Budankov wrote:
SNIP
Quoted 20 lines
tools/perf/builtin-record.c +8
-5+ compress_fn = (record__comp_enabled (rec) ? + perf_session__zstd_compress : perf_session__zstd_copy); +if (record__aio_enabled (rec)) off =record__aio_get_pos (trace_fd);@@ -799,11 +805,21 @@ static int record__mmap_read_evlist(struct record *rec, struct perf_evlist *evli map->flush =MMAP_FLUSH_DEFAULT ; }if (!record__aio_enabled (rec)) { -if (perf_mmap__push (map, rec, record__pushfn) !=0 ) { -if (sync) - map->flush = flush; - rc =-1 ; - goto out; +if (!record__comp_enabled (rec)) { +if (perf_mmap__push (map, rec, record__pushfn) !=0 ) { +if (sync) + map->flush = flush; + rc =-1 ;
tools/perf/builtin-record.c static int record__mmap_read_evlist(struct record *rec, struct perf_evlist *evli · line 822
Quoted 10 lines
tools/perf/builtin-record.c +10
-0+ goto out; + } + }else { +if (perf_mmap__pack (map, rec, record__pushfn, + compress_fn, session) !=0 ) { +if (sync) + map->flush = flush; + rc =-1 ; + goto out; + }
I really dont like this.. the non-compression flow needs to stay untouched
why don't you overload the record__pushfn function for compression processing and stash the data in there?
thanks,
jirka
Re: [PATCH v2 3/4] perf record: enable runtime trace compression
In reply to Alexey BudankovOn Mon, Feb 11, 2019 at 11:23:40PM +0300, Alexey Budankov wrote:
SNIP
Quoted 7 lines
tools/perf/util/mmap.c +0
-0@@ -156,6 +156,86 @@ void __weak auxtrace_mmap_params__set_idx(struct auxtrace_mmap_params *mp __mayb { }
tools/perf/util/mmap.c void __weak auxtrace_mmap_params__set_idx(struct auxtrace_mmap_params *mp __mayb · line 167
Quoted 10 lines
tools/perf/util/mmap.c +9
-0+static ssize_t perf_mmap__capture (struct perf_mmap *md,int idx, +perf_mmap__compress_fn_t compress,void *where) +{ +u64 head =perf_mmap__read_head (md); +unsigned char *data = md->base + page_size; +unsigned long size, size0 =0 ; +void *buf; +int rc =0 ; +size_t mmap_len =perf_mmap__mmap_len (md);
again, this function seems like duplicate of perf_mmap__push, can't you just overload the push callback?
thanks,
jirka
Re: [PATCH v2 4/4] perf report: support record trace file decompression
In reply to Alexey BudankovQuoted 11 lines · Alexey Budankov
PERF_RECORD_COMPRESSEDrecords are decompressed from trace file into a linked list of mmaped memory regions using streaming Zstandard API. After that the regions are loaded fetching uncompressed events. When dumping raw trace (e.g., perf report -D --header) file offsets of events from compressed records are set to zero.Signed-off-by Alexey Budankov <[email protected]>---
Changes in v2:
- moved compression/decompression code to session layer
could you please add some automated test for this?
I think it could be part of the sample synthesize test or even new script test under tests/shell.. or anything else really.. just to see that we have some clean interface for this that we can test
thanks,
jirka
Re: [PATCH v2 4/4] perf report: support record trace file decompression
In reply to Alexey BudankovOn Mon, Feb 11, 2019 at 11:25:00PM +0300, Alexey Budankov wrote:
SNIP
Quoted 35 lines
tools/perf/util/session.c +35
-0+static int perf_session__process_compressed_event (struct perf_session *session, +union perf_event *event,u64 file_offset) +{ +void *src; +size_t decomp_size, src_size; +u64 decomp_last_rem =0 ; +size_t decomp_len = session->header.env.comp_mmap_len; +struct decomp *decomp, *decomp_last = session->decomp_last; + + decomp =mmap (NULL ,sizeof (struct decomp ) + decomp_len,PROT_READ |PROT_WRITE , +MAP_ANONYMOUS |MAP_PRIVATE ,-1 ,0 ); +if (decomp ==MAP_FAILED ) { +pr_err ("Couldn't allocate memory for decompression\n" ); +return -1 ; + } + + decomp->file_pos = file_offset; + decomp->head =0 ; + +if (decomp_last) { + decomp_last_rem = decomp_last->size - decomp_last->head; +memcpy (decomp->data, &(decomp_last->data[decomp_last->head]), decomp_last_rem); + decomp->size = decomp_last_rem; + } + + src = (void *)event +sizeof (struct compressed_event ); + src_size = event->pack.header.size -sizeof (struct compressed_event ); + + decomp_size =perf_session__zstd_decompress (session, src, src_size, + &(decomp->data[decomp_last_rem]), decomp_len - decomp_last_rem); +if (!decomp_size) { +munmap (decomp,sizeof (struct decomp ) + decomp_len); +pr_err ("Couldn't decompress data\n" ); +return -1 ; + }
tools/perf/util/session.c size_t perf_session__zstd_compress(void *to, void *dst, size_t dst_size, · line 209
Quoted 10 lines
tools/perf/util/session.c +10
-0+ + decomp->size += decomp_size; + +if (session->decomp ==NULL ) { + session->decomp = decomp; + session->decomp_last = decomp; + }else { + session->decomp_last->next = decomp; + session->decomp_last = decomp; + }
can this happen? can you process more compressed events before you deliver them in __perf_session__process_decomp_events?
I think I asked similar question in my other email..
thanks,
jirka
Re: [PATCH v2 3/4] perf record: enable runtime trace compression
In reply to Alexey BudankovOn Mon, Feb 11, 2019 at 11:23:40PM +0300, Alexey Budankov wrote:
SNIP
tools/perf/util/mmap.c void __weak auxtrace_mmap_params__set_idx(struct auxtrace_mmap_params *mp __mayb · line 160
Quoted 10 lines
tools/perf/util/mmap.c +2
-0@@ -156,6 +156,86 @@ void __weak auxtrace_mmap_params__set_idx(struct auxtrace_mmap_params *mp __mayb { } +static ssize_t perf_mmap__capture (struct perf_mmap *md,int idx, +perf_mmap__compress_fn_t compress,void *where)
what's the point of casting perf_session* to void* ? all the way down to the compression, where you cast it back?
SNIP
Quoted 4 lines
tools/perf/util/mmap.c +4
-0+int perf_session__zstd_fini (struct perf_session *session) +{ +if (session->zstd_cstream) { +ZSTD_freeCStream (session->zstd_cstream);
tools/perf/util/mmap.c void __weak auxtrace_mmap_params__set_idx(struct auxtrace_mmap_params *mp __mayb · line 174
Quoted 10 lines
tools/perf/util/mmap.c +10
-0+ session->zstd_cstream =NULL ; + } + +return 0 ; +} + +size_t perf_session__zstd_compress (void *to,void *dst,size_t dst_size, +void *src,size_t src_size) +{ +struct perf_session *session = to;
in here... it could be just struct perf_session* no?
jirka
Re: [PATCH v2 4/4] perf report: support record trace file decompression
In reply to Alexey BudankovOn Mon, Feb 11, 2019 at 11:25:00PM +0300, Alexey Budankov wrote:
SNIP
Quoted 5 lines
tools/perf/util/session.c +1
-0@@ -1932,6 +2059,38 @@ fetch_mmaped_event(struct perf_session *session,return event; } +static int __perf_session__process_decomp_events (struct perf_session *session)
tools/perf/util/session.c fetch_mmaped_event(struct perf_session *session, · line 2072
Quoted 10 lines
tools/perf/util/session.c +10
-0+{ +s64 skip; +u64 size, file_pos =0 ; +union perf_event *event; +struct decomp *decomp = session->decomp_last; + +if (!decomp) +return 0 ; + +while (decomp->head < decomp->size && !session_done ()) {
so how this actualy works? does one PERF_RECORD_COMPRESSED event carry complete data to unpack? or you wait to receive more PERF_RECORD_COMPRESSED to give you data you can unpack?
jirka
Re: [PATCH v2 2/4] perf record: implement -z=<level> and --mmap-flush=<thres> options
In reply to Alexey BudankovQuoted 14 lines · Alexey Budankov
Implement -z,--compression_level=<n> and --mmap-flush=<dump_least_size> options as well as a special
PERF_RECORD_COMPRESSEDrecord that contains compressed parts of kernel data buffer.Because compression requires auxilary memory to implement encoding of kernel data
record->opts.nr_cblocks== -1 signifies to preallocate single AIO data buffer aio.data[0] without accompnying AIO control blocks.Signed-off-by Alexey Budankov <[email protected]>---
Changes in v2:
- enabled allocation aio data buffers for compression
Quoted 15 lines
--- tools/perf/Documentation/perf-record.txt | 9 ++ tools/perf/builtin-record.c | 110 +++++++++++++++++++---- tools/perf/perf.h | 2 + tools/perf/util/env.h | 10 +++ tools/perf/util/event.c | 1 + tools/perf/util/event.h | 7 ++ tools/perf/util/evlist.c | 6 +- tools/perf/util/evlist.h | 3 +- tools/perf/util/header.c | 45 +++++++++- tools/perf/util/header.h | 1 + tools/perf/util/mmap.c | 98 ++++++++++++-------- tools/perf/util/mmap.h | 7 +- tools/perf/util/session.h | 2 + 13 files changed, 240 insertions(+), 61 deletions(-)
this patch is super complicated.. please think of reviewers when writing code ;-)
it needs to be split to small logical patches,
where each change is explained in changelog
jirka
Re: [PATCH v2 2/4] perf record: implement -z=<level> and --mmap-flush=<thres> options
In reply to Alexey BudankovOn Mon, Feb 11, 2019 at 11:22:38PM +0300, Alexey Budankov wrote:
SNIP
Quoted 24 lines
tools/perf/util/header.c +20
-0+static int process_compressed (struct feat_fd *ff, +void *data__maybe_unused ) +{ +u64 compression_info; + +if (do_read_u64 (ff, &compression_info)) +return -1 ; + + ff->ph->env.comp_type = (compression_info >>32 ) &0xffffffffULL ; + ff->ph->env.comp_level = compression_info &0xffffffffULL ; + +if (do_read_u64 (ff, &compression_info)) +return -1 ; + + ff->ph->env.comp_ratio = (compression_info >>32 ) &0xffffffffULL ; + ff->ph->env.comp_mmap_len = compression_info &0xffffffffULL ; + +return 0 ; +} +struct feature_ops {int (*write)(struct feat_fd *ff,struct perf_evlist *evlist);void (*print)(struct feat_fd *ff,FILE *fp);
tools/perf/util/header.c static const struct feature_ops feat_ops[HEADER_LAST_FEATURE] = { · line 2697
Quoted 6 lines
tools/perf/util/header.c +2
-1@@ -2651,7 +2693,8 @@ static const struct feature_ops feat_ops[HEADER_LAST_FEATURE] = {FEAT_OPN (CACHE , cache, true),FEAT_OPR (SAMPLE_TIME , sample_time, false),FEAT_OPR (MEM_TOPOLOGY , mem_topology, true), -FEAT_OPR (CLOCKID , clockid, false) +FEAT_OPR (CLOCKID , clockid, false), +FEAT_OPR (COMPRESSED , compressed, false)
please separate the COMPRESSED feature ddition into single patch
thanks,
jirka
Re: [PATCH v2 2/4] perf record: implement -z=<level> and --mmap-flush=<thres> options
In reply to Alexey BudankovOn Mon, Feb 11, 2019 at 11:22:38PM +0300, Alexey Budankov wrote:
SNIP
Quoted 12 lines
tools/perf/builtin-record.c +4
-0@@ -1147,6 +1193,10 @@ static int __cmd_record(struct record *rec, int argc, const char **argv) fd =perf_data__fd (data); rec->session = session; + rec->opts.comp_level =0 ; + session->header.env.comp_level = rec->opts.comp_level; + session->header.env.comp_type =PERF_COMP_NONE ; +record__init_features (rec);if (rec->opts.use_clockid && rec->opts.clockid_res_ns)
tools/perf/builtin-record.c static int __cmd_record(struct record *rec, int argc, const char **argv) · line 1229
Quoted 4 lines
tools/perf/builtin-record.c +1
-0@@ -1176,6 +1226,7 @@ static int __cmd_record(struct record *rec, int argc, const char **argv) err =-1 ; goto out_child; } + session->header.env.comp_mmap_len = session->evlist->mmap_len;
so the comp_mmap_len is the max length of the compressed packet?
any idea if this value might have some impact on the processing speed?
I see you mentioned the size reduction, could you also meassure the record overhead?
thanks,
jirka
Re: [PATCH v2 3/4] perf record: enable runtime trace compression
In reply to Alexey BudankovQuoted 28 lines · Alexey Budankov
Compression is implemented using simple Zstd API and employs AIO data buffer as the memory to operate on. If the API call fails for some reason compression falls back to
memcpy().Data chunks are split and packed into
PERF_RECORD_COMPRESSEDrecords by 64KB at max. mmap-flush option value can be used to avoid compression of every single byte of data and increase compression ratio.Signed-off-by Alexey Budankov <[email protected]>---
Changes in v2:
- enabled trace compression for serial trace streaming
- moved compression/decompression code to session layer--- tools/perf/builtin-record.c | 67 +++++++++----- tools/perf/util/mmap.c | 173 +++++++++++++++++++++--------------- tools/perf/util/mmap.h | 24 +++-- tools/perf/util/session.c | 106 ++++++++++++++++++++++ tools/perf/util/session.h | 13 +++ 5 files changed, 280 insertions(+), 103 deletions(-)tools/perf/builtin-record.c +0
-0
tools/perf/builtin-record.c static bool switch_output_time(struct record *rec) · line 115
Quoted 6 lines
tools/perf/builtin-record.c +1
-2@@ -112,8 +112,7 @@ static bool switch_output_time(struct record *rec)trigger_is_ready (&switch_output_trigger); } -static int record__write (struct record *rec,struct perf_mmap *map__maybe_unused , -void *bf,size_t size) +static int record__write (struct record *rec,void *bf,size_t size)
please keep the 'struct perf_mmap *map' in, I'm using it in the multiple file storage, where each map holds pointer to its data file
thanks,
jirka
Quoted 8 lines · Arnaldo Carvalho de Melo
Em Mon, Feb 11, 2019 at 11:17:14PM +0300, Alexey Budankov escreveu:
Quoted 6 lines
This is the rebase to the tip of Arnaldo's perf/core repository.
The patch set implements runtime trace compression for record mode and trace file decompression for report mode. Zstandard API [1] is used for compression/decompression of data that come from
perf_eventskernel data buffers.
Quoted 4 lines
Quoted 1 line
Realized -z,--compression_level=n option provides ~3-5x avg. trace file
Realized? Perhaps "Using the -z/--compression_level=n options provides 3.5x average size reduction on..."
Yep, wording could be adjusted. Thanks!
Quoted 6 lines
Quoted 5 lines
size reduction on variety of tested workloads what saves user storage space on larger server systems where trace file size can easily reach several tens or even hundreds of GiBs, especially when profiling with dwarf-based stacks and tracing of context-switches.
Quoted 6 lines
Quoted 1 line
--mmap-flush option can be used to avoid compressing every single byte
What is the default for --mmap-flush when -z is used? Does it vary with the compression level? Can we expect something that is reasonable or is using --mmap-flush always needed? If so can we do away with that by using some sensible default according to the -z level used?
Default value is 1 what is equal to the current perf record implementation. The option is independent from -z setting and doesn't vary with compression level. Default could be different if -z is specified.
Quoted 8 lines
Quoted 7 lines
of data and increase compression ratio at the same time lowering tool runtime overhead.
$ tools/perf/perf record -z 1 -e cycles -- matrix.gcc $ tools/perf/perf record -z 1 --mmap-flush 1024 -e cycles -- matrix.gcc $ tools/perf/perf record -z 1 --mmap-flush 1024 --aio -e cycles -- matrix.gcc
Quoted 5 lines
Please avoid having these explanations just on the cover letter since it will not get recorded in the git history for the project, please add examples with output from real workloads, preferably providing real cases so that other people can readily replicate your results when having access to similar large machines.
Will do. Thanks!
Quoted 5 lines
Quoted 5 lines
The compression functionality can be disabled from the command line using
NO_LIBZSTDdefine and Zstandard sources can be overridden using value ofLIBZSTD_DIRdefine:
Quoted 7 lines
Quoted 2 lines
$ make -C tools/perf NO_LIBZSTD=1 clean all $ make -C tools/perf LIBZSTD_DIR=/path/to/zstd-1.3.7 clean allShowing the relevant output of the these commands before/after installing the libzstd library is useful as well. From what you use above is it fair to conclude that there are no readily made packages for this library and we need to get it from github and build it ourselves?
If your system has some version of the package preinstalled then the build system finds and uses it during the build. Auto detection feature status is reported just before compilation starts.
If you still prefer to compile with some newer version that is not preinstalled, and you don't want upgrading your system with newer version, you have capability to download, build and then refer the compilation to the newer just built library version using LIBZSTD_DIR.
Quoted 20 lines
On my notebook, a Fedora 29 system:
[root@quaco ~]# dnf search zstd
========================================================== Name & Summary Matched: zstd ========================================================== zstd.x86_64 : Zstd compression library
zstd.x86_64 : Zstd compression library
libzstd.x86_64 : Zstd shared library
libzstd.i686 : Zstd shared library
libzstd.x86_64 : Zstd shared library
libzstd-devel.i686 : Header files for Zstd library
libzstd-devel.x86_64 : Header files for Zstd library
============================================================= Summary Matched: zstd ============================================================== fedora-repo-zdicts.noarch : Zstd dictionaries for Fedora repository metadata
[root@quaco ~]#
[root@quaco ~]# dnf install libzstd-devel
<SNIP>
libzstd-develx86_641.3.8-1.fc29 updates 39 k
<SNIP>
Quoted 2 lines
So, do I need some specific 1.3.7 version? Is that why you used that
LIBZSTD_DIRin your example?
v1.3.7 is just an example. v1.3.8 can be also provided.
Quoted 5 lines
I'm trying these steps, but please consider following these suggestions in future posts,
And thanks for working on this!
Thanks for meaningful comments. Will do my best to follow it.
Thanks,
Alexey
Quoted 3 lines
- Arnaldo
Re: [PATCH v2 2/4] perf record: implement -z=<level> and --mmap-flush=<thres> options
In reply to Jiri OlsaQuoted 7 lines · Jiri Olsa
On Mon, Feb 11, 2019 at 11:22:38PM +0300, Alexey Budankov wrote:
SNIP
Quoted 3 lines
static int process_synthesized_event(struct
perf_tool*tool,tools/perf/builtin-record.c +0
-0union perf_event *event, struct perf_sample *sample __maybe_unused,
<SNIP>
Quoted 26 lines
Quoted 26 lines
tools/perf/builtin-record.c +9
-0for (i =0 ; i < evlist->nr_mmaps; i++) { +u64 flush =MMAP_FLUSH_DEFAULT ;struct perf_mmap *map = &maps[i];if (map->base) {record__adjust_affinity (rec, map); +if (sync) { + flush = map->flush; + map->flush =MMAP_FLUSH_DEFAULT ; + }if (!record__aio_enabled (rec)) {if (perf_mmap__push (map, rec, record__pushfn) !=0 ) { +if (sync) + map->flush = flush; rc =-1 ; goto out; }@@ -775,10 +814,14 @@ static int record__mmap_read_evlist(struct record *rec, struct perf_evlist *evli idx =record__aio_sync (map, false);if (perf_mmap__aio_push (map, rec, idx, record__aio_pushfn, &off) !=0 ) {record__aio_set_pos (trace_fd, off); +if (sync) + map->flush = flush; rc =-1 ; goto out; }
Quoted 8 lines
tools/perf/builtin-record.c static int record__mmap_read_evlist(struct record *rec, struct perf_evlist *evli · line 825
Quoted 4 lines
tools/perf/builtin-record.c +2
-0} +if (sync) + map->flush = flush; }what's 'flush' and 'sync' for? also the above handling seems confusing, why would you set it temporarily for default value if you let it set by user command line option?
flush is the threshold postponing and triggering the move of data to a trace file from the memory buffers. sync is the mean to force that move independently from the threshold value.
Despite a user provides flush value from the command line, the tool has to drain memory buffers, at least in the end of the collection, so that it is technically implemented like this.
Quoted 5 lines
SNIP
Quoted 2 lines
-static void
perf_mmap__aio_munmap(struct perf_mmap *map __maybe_unused)
<SNIP>
Quoted 9 lines
tools/perf/util/mmap.c static int __perf_mmap__read_init(struct perf_mmap *md) · line 522
Quoted 7 lines
tools/perf/util/mmap.c +1
-1@@ -492,7 +518,7 @@ static int __perf_mmap__read_init(struct perf_mmap *md) md->start = md->overwrite ? head : old; md->end = md->overwrite ? old : head; -if (md->start == md->end) +if ((md->end - md->start) < md->flush)return -EAGAIN ;we need document and explain this change in changelog in separate patch
Moved --mmap-flush option implementation into a separate patch.
Thanks,
Alexey
Quoted 4 lines
thanks,
jirka
Re: [PATCH v2 2/4] perf record: implement -z=<level> and --mmap-flush=<thres> options
In reply to Jiri OlsaQuoted 4 lines · Jiri Olsa
On Mon, Feb 11, 2019 at 11:22:38PM +0300, Alexey Budankov wrote:
SNIP
Quoted 13 lines
tools/perf/util/session.h struct perf_session { · line 39
Quoted 10 lines
tools/perf/util/session.h +2
-0@@ -35,6 +35,8 @@ struct perf_session {struct ordered_events ordered_events;struct perf_data *data;struct perf_tool *tool; +u64 bytes_transferred; +u64 bytes_compressed;please add those in separate patch, so we can see how they are incremented and displayed easily
Moved declaration and printing of the values into a separate patch.
Thanks,
Alexey
Quoted 4 lines
thanks,
jirka
Re: [PATCH v2 2/4] perf record: implement -z=<level> and --mmap-flush=<thres> options
In reply to Jiri OlsaQuoted 19 lines · Jiri Olsa
On Mon, Feb 11, 2019 at 11:22:38PM +0300, Alexey Budankov wrote:
SNIP
Quoted 15 lines
static void
record__init_features(struct record *rec)tools/perf/builtin-record.c +3
-0@@ -838,6 +881,9 @@ static void record__init_features(struct record *rec)if (!(rec->opts.use_clockid && rec->opts.clockid_res_ns))perf_header__clear_feat (&session->header,HEADER_CLOCKID ); +if (!record__comp_enabled (rec)) +perf_header__clear_feat (&session->header,HEADER_COMPRESSED ); +perf_header__clear_feat (&session->header,HEADER_STAT ); }@@ -1147,6 +1193,10 @@ static int __cmd_record(struct record *rec, int argc, const char **argv) fd =perf_data__fd (data); rec->session = session;
Quoted 6 lines
tools/perf/builtin-record.c static int __cmd_record(struct record *rec, int argc, const char **argv) · line 1198
Quoted 3 lines
tools/perf/builtin-record.c +3
-0+ rec->opts.comp_level =0 ; + session->header.env.comp_level = rec->opts.comp_level; + session->header.env.comp_type =PERF_COMP_NONE ;hum I think both record and session are zerod from start, so this seems superfluous
Removed.
Quoted 11 lines
SNIP
Quoted 8 lines
tools/perf/util/header.c +0
-0@@ -1463,6 +1463,21 @@ static int write_mem_topology(struct feat_fd *ff __maybe_unused,return ret; }
Quoted 14 lines
tools/perf/util/header.c static int write_mem_topology(struct feat_fd *ff __maybe_unused, · line 1476
Quoted 11 lines
tools/perf/util/header.c +11
-0+static int write_compressed (struct feat_fd *ff__maybe_unused , +struct perf_evlist *evlist__maybe_unused ) +{ +int ret; +u64 compression_info = ((u64 )ff->ph->env.comp_type <<32 ) | ff->ph->env.comp_level; + + ret =do_write (ff, &compression_info,sizeof (compression_info)); +if (ret) +return ret; + + compression_info = ((u64 )ff->ph->env.comp_ratio <<32 ) | ff->ph->env.comp_mmap_len;let's not complicate this, if you have two u32 values to store, store two u32 values ;-)
Corrected.
Quoted 3 lines
also please add version in case we will add new fields in the future, so we can stay backward compatible
Implemented.
Thanks,
Alexey
Quoted 3 lines
jirka
Re: [PATCH v2 2/4] perf record: implement -z=<level> and --mmap-flush=<thres> options
In reply to Jiri OlsaQuoted 14 lines · Jiri Olsa
On Mon, Feb 11, 2019 at 11:22:38PM +0300, Alexey Budankov wrote:
SNIP
Quoted 10 lines
tools/perf/builtin-record.c +4
-0@@ -1147,6 +1193,10 @@ static int __cmd_record(struct record *rec, int argc, const char **argv) fd =perf_data__fd (data); rec->session = session; + rec->opts.comp_level =0 ; + session->header.env.comp_level = rec->opts.comp_level; + session->header.env.comp_type =PERF_COMP_NONE ; +record__init_features (rec);
Quoted 8 lines
tools/perf/builtin-record.c static int __cmd_record(struct record *rec, int argc, const char **argv) · line 1229
Quoted 6 lines
tools/perf/builtin-record.c +1
-0if (rec->opts.use_clockid && rec->opts.clockid_res_ns)@@ -1176,6 +1226,7 @@ static int __cmd_record(struct record *rec, int argc, const char **argv) err =-1 ; goto out_child; } + session->header.env.comp_mmap_len = session->evlist->mmap_len;so the
comp_mmap_lenis the max length of the compressed packet?
comp_mmap_len is the size of buffer to encompass one compressed chunk of data after its decompression.
Quoted 2 lines
any idea if this value might have some impact on the processing speed?
It increases memory consumption at the loading and processing stages.
Quoted 3 lines
I see you mentioned the size reduction, could you also meassure the record overhead?
Let's get back to this after the code review.
Thanks,
Alexey
Quoted 4 lines
thanks,
jirka
Re: [PATCH v2 2/4] perf record: implement -z=<level> and --mmap-flush=<thres> options
In reply to Jiri OlsaQuoted 4 lines · Jiri Olsa
On Mon, Feb 11, 2019 at 11:22:38PM +0300, Alexey Budankov wrote:
SNIP
Quoted 16 lines
tools/perf/builtin-record.c int cmd_record(int argc, const char **argv) · line 2255
Quoted 12 lines
tools/perf/builtin-record.c +8
-4-if (rec->opts.nr_cblocks > nr_cblocks_max) - rec->opts.nr_cblocks = nr_cblocks_max; -if (verbose >0 ) -pr_info ("nr_cblocks: %d\n" , rec->opts.nr_cblocks); +if (rec->opts.comp_level >22 ) + rec->opts.comp_level =0 ; +if (record__comp_enabled (rec) && !rec->opts.nr_cblocks) { +/* +* Allocate aio.data[0] buffer for compression. +*/ + rec->opts.nr_cblocks =-1 ; + }I assume you're using aio.data[0] as a buffer for non-aio case as compression buffer.. if that's the case, please dont do that and use separate buffer for that
Do you want it to be one more field in the mmap struct or aio related fields still could be reworked?
Thanks,
Alexey
Quoted 3 lines
jirka
Re: [PATCH v2 2/4] perf record: implement -z=<level> and --mmap-flush=<thres> options
In reply to Jiri OlsaQuoted 28 lines · Jiri Olsa
On Mon, Feb 11, 2019 at 11:22:38PM +0300, Alexey Budankov wrote:
SNIP
Quoted 24 lines
tools/perf/util/header.c +20
-0+static int process_compressed (struct feat_fd *ff, +void *data__maybe_unused ) +{ +u64 compression_info; + +if (do_read_u64 (ff, &compression_info)) +return -1 ; + + ff->ph->env.comp_type = (compression_info >>32 ) &0xffffffffULL ; + ff->ph->env.comp_level = compression_info &0xffffffffULL ; + +if (do_read_u64 (ff, &compression_info)) +return -1 ; + + ff->ph->env.comp_ratio = (compression_info >>32 ) &0xffffffffULL ; + ff->ph->env.comp_mmap_len = compression_info &0xffffffffULL ; + +return 0 ; +} +struct feature_ops {int (*write)(struct feat_fd *ff,struct perf_evlist *evlist);void (*print)(struct feat_fd *ff,FILE *fp);@@ -2651,7 +2693,8 @@ static const struct feature_ops feat_ops[HEADER_LAST_FEATURE] = {
Quoted 8 lines
tools/perf/util/header.c static const struct feature_ops feat_ops[HEADER_LAST_FEATURE] = { · line 2697
Quoted 6 lines
tools/perf/util/header.c +2
-1FEAT_OPN (CACHE , cache, true),FEAT_OPR (SAMPLE_TIME , sample_time, false),FEAT_OPR (MEM_TOPOLOGY , mem_topology, true), -FEAT_OPR (CLOCKID , clockid, false) +FEAT_OPR (CLOCKID , clockid, false), +FEAT_OPR (COMPRESSED , compressed, false)please separate the COMPRESSED feature ddition into single patch
Moved to a separate patch.
Thanks,
Alexey
Quoted 4 lines
thanks,
jirka
Re: [PATCH v2 4/4] perf report: support record trace file decompression
In reply to Jiri OlsaQuoted 8 lines · Jiri Olsa
On Mon, Feb 11, 2019 at 11:25:00PM +0300, Alexey Budankov wrote:
SNIP
Quoted 4 lines
tools/perf/util/session.c +0
-0@@ -1932,6 +2059,38 @@ fetch_mmaped_event(struct perf_session *session,return event; }
Quoted 15 lines
tools/perf/util/session.c fetch_mmaped_event(struct perf_session *session, · line 2072
Quoted 11 lines
tools/perf/util/session.c +11
-0+static int __perf_session__process_decomp_events (struct perf_session *session) +{ +s64 skip; +u64 size, file_pos =0 ; +union perf_event *event; +struct decomp *decomp = session->decomp_last; + +if (!decomp) +return 0 ; + +while (decomp->head < decomp->size && !session_done ()) {so how this actualy works? does one
PERF_RECORD_COMPRESSEDevent carry complete data to unpack? or you wait to receive morePERF_RECORD_COMPRESSEDto give you data you can unpack?
One record contains compressed data chunk that is ready for decompression. Record header is skipped and the rest is fetched to perf_session__zstd_decompress() also providing the buffer to decompress to, which is of the size encoded in the file header (comp_mmap_len). After decompression the buffer is iterated the same way as mmapped event records. Decompressed split event records are handled. Decompressed buffers are chained into the linked list so the records memory remains available for later references.
Thanks,
Alexey
Quoted 3 lines
jirka
Re: [PATCH v2 4/4] perf report: support record trace file decompression
In reply to Jiri OlsaQuoted 53 lines · Jiri Olsa
On Mon, Feb 11, 2019 at 11:25:00PM +0300, Alexey Budankov wrote:
SNIP
Quoted 35 lines
tools/perf/util/session.c +35
-0+static int perf_session__process_compressed_event (struct perf_session *session, +union perf_event *event,u64 file_offset) +{ +void *src; +size_t decomp_size, src_size; +u64 decomp_last_rem =0 ; +size_t decomp_len = session->header.env.comp_mmap_len; +struct decomp *decomp, *decomp_last = session->decomp_last; + + decomp =mmap (NULL ,sizeof (struct decomp ) + decomp_len,PROT_READ |PROT_WRITE , +MAP_ANONYMOUS |MAP_PRIVATE ,-1 ,0 ); +if (decomp ==MAP_FAILED ) { +pr_err ("Couldn't allocate memory for decompression\n" ); +return -1 ; + } + + decomp->file_pos = file_offset; + decomp->head =0 ; + +if (decomp_last) { + decomp_last_rem = decomp_last->size - decomp_last->head; +memcpy (decomp->data, &(decomp_last->data[decomp_last->head]), decomp_last_rem); + decomp->size = decomp_last_rem; + } + + src = (void *)event +sizeof (struct compressed_event ); + src_size = event->pack.header.size -sizeof (struct compressed_event ); + + decomp_size =perf_session__zstd_decompress (session, src, src_size, + &(decomp->data[decomp_last_rem]), decomp_len - decomp_last_rem); +if (!decomp_size) { +munmap (decomp,sizeof (struct decomp ) + decomp_len); +pr_err ("Couldn't decompress data\n" ); +return -1 ; + }tools/perf/util/session.c size_t perf_session__zstd_compress(void *to, void *dst, size_t dst_size, · line 209
Quoted 10 lines
tools/perf/util/session.c +10
-0+ + decomp->size += decomp_size; + +if (session->decomp ==NULL ) { + session->decomp = decomp; + session->decomp_last = decomp; + }else { + session->decomp_last->next = decomp; + session->decomp_last = decomp; + }can this happen? can you process more compressed events before you deliver them in
__perf_session__process_decomp_events?
Quoted 1 line
I think I asked similar question in my other email..
Answered to that mail.
Thanks,
Alexey
Quoted 4 lines
thanks,
jirka
Re: [PATCH v2 4/4] perf report: support record trace file decompression
In reply to Jiri OlsaQuoted 13 lines · Jiri Olsa
Quoted 11 lines · Alexey Budankov
PERF_RECORD_COMPRESSEDrecords are decompressed from trace file into a linked list of mmaped memory regions using streaming Zstandard API. After that the regions are loaded fetching uncompressed events. When dumping raw trace (e.g., perf report -D --header) file offsets of events from compressed records are set to zero.Signed-off-by Alexey Budankov <[email protected]>---
Changes in v2:
- moved compression/decompression code to session layer
Quoted 6 lines
could you please add some automated test for this?
I think it could be part of the sample synthesize test or even new script test under tests/shell.. or anything else really.. just to see that we have some clean interface for this that we can test
This is possible. Let's complete the code review first.
Thanks,
Alexey
Quoted 4 lines
thanks,
jirka
Re: [PATCH v2 3/4] perf record: enable runtime trace compression
In reply to Jiri OlsaQuoted 17 lines · Jiri Olsa
On Mon, Feb 11, 2019 at 11:23:40PM +0300, Alexey Budankov wrote:
SNIP
Quoted 13 lines
tools/perf/builtin-record.c +2
-0@@ -774,6 +775,8 @@ static int record__mmap_read_evlist(struct record *rec, struct perf_evlist *evlistruct perf_mmap *maps;int trace_fd = rec->data.file.fd;off_t off; +struct perf_session *session = rec->session; +perf_mmap__compress_fn_t compress_fn;if (!evlist)return 0 ;@@ -785,6 +788,9 @@ static int record__mmap_read_evlist(struct record *rec, struct perf_evlist *evliif (overwrite && evlist->bkw_mmap_state !=BKW_MMAP_DATA_PENDING )return 0 ;
Quoted 5 lines
tools/perf/builtin-record.c static int record__mmap_read_evlist(struct record *rec, struct perf_evlist *evli · line 793
Quoted 3 lines
tools/perf/builtin-record.c +3
-0+ compress_fn = (record__comp_enabled (rec) ? + perf_session__zstd_compress : perf_session__zstd_copy); +I don't follow what's the
perf_session__zstd_copyfunction for..?
It bridges AIO without compression case.
Thanks,
Alexey
Quoted 39 lines
for !record__comp_enabled case we seem not to use it and calling the current
perf_mmap__pushinterfacehowever I dont see point to have this function at all
jirka
Quoted 27 lines
tools/perf/builtin-record.c +15
-5if (record__aio_enabled (rec)) off =record__aio_get_pos (trace_fd);@@ -799,11 +805,21 @@ static int record__mmap_read_evlist(struct record *rec, struct perf_evlist *evli map->flush =MMAP_FLUSH_DEFAULT ; }if (!record__aio_enabled (rec)) { -if (perf_mmap__push (map, rec, record__pushfn) !=0 ) { -if (sync) - map->flush = flush; - rc =-1 ; - goto out; +if (!record__comp_enabled (rec)) { +if (perf_mmap__push (map, rec, record__pushfn) !=0 ) { +if (sync) + map->flush = flush; + rc =-1 ; + goto out; + } + }else { +if (perf_mmap__pack (map, rec, record__pushfn, + compress_fn, session) !=0 ) { +if (sync) + map->flush = flush; + rc =-1 ; + goto out; + }SNIP
Re: [PATCH v2 3/4] perf record: enable runtime trace compression
In reply to Jiri OlsaQuoted 20 lines · Jiri Olsa
On Mon, Feb 11, 2019 at 11:23:40PM +0300, Alexey Budankov wrote:
SNIP
tools/perf/util/mmap.c void __weak auxtrace_mmap_params__set_idx(struct auxtrace_mmap_params *mp __mayb · line 160
Quoted 10 lines
tools/perf/util/mmap.c +2
-0@@ -156,6 +156,86 @@ void __weak auxtrace_mmap_params__set_idx(struct auxtrace_mmap_params *mp __mayb { } +static ssize_t perf_mmap__capture (struct perf_mmap *md,int idx, +perf_mmap__compress_fn_t compress,void *where)what's the point of casting perf_session* to void* ? all the way down to the compression, where you cast it back?
SNIP
Quoted 16 lines
Quoted 4 lines
tools/perf/util/mmap.c +4
-0+int perf_session__zstd_fini (struct perf_session *session) +{ +if (session->zstd_cstream) { +ZSTD_freeCStream (session->zstd_cstream);tools/perf/util/mmap.c void __weak auxtrace_mmap_params__set_idx(struct auxtrace_mmap_params *mp __mayb · line 174
Quoted 10 lines
tools/perf/util/mmap.c +10
-0+ session->zstd_cstream =NULL ; + } + +return 0 ; +} + +size_t perf_session__zstd_compress (void *to,void *dst,size_t dst_size, +void *src,size_t src_size) +{ +struct perf_session *session = to;in here... it could be just struct perf_session* no?
As far as it is a pointer. Casting a pointer to void* and back is harmless. Well, let's clean that up.
Thanks,
Alexey
Quoted 3 lines
jirka
Re: [PATCH v2 3/4] perf record: enable runtime trace compression
In reply to Jiri OlsaQuoted 32 lines · Jiri Olsa
Quoted 31 lines · Alexey Budankov
Compression is implemented using simple Zstd API and employs AIO data buffer as the memory to operate on. If the API call fails for some reason compression falls back to
memcpy().Data chunks are split and packed into
PERF_RECORD_COMPRESSEDrecords by 64KB at max. mmap-flush option value can be used to avoid compression of every single byte of data and increase compression ratio.Signed-off-by Alexey Budankov <[email protected]>---
Changes in v2:
- enabled trace compression for serial trace streaming
- moved compression/decompression code to session layer--- tools/perf/builtin-record.c | 67 +++++++++----- tools/perf/util/mmap.c | 173 +++++++++++++++++++++--------------- tools/perf/util/mmap.h | 24 +++-- tools/perf/util/session.c | 106 ++++++++++++++++++++++ tools/perf/util/session.h | 13 +++ 5 files changed, 280 insertions(+), 103 deletions(-)tools/perf/builtin-record.c +0
-0@@ -112,8 +112,7 @@ static bool switch_output_time(struct record *rec)trigger_is_ready (&switch_output_trigger); }
Quoted 7 lines
tools/perf/builtin-record.c static bool switch_output_time(struct record *rec) · line 115
Quoted 3 lines
tools/perf/builtin-record.c +1
-2-static int record__write (struct record *rec,struct perf_mmap *map__maybe_unused , -void *bf,size_t size) +static int record__write (struct record *rec,void *bf,size_t size)please keep the 'struct
perf_mmap*map' in, I'm using it in the multiple file storage, where each map holds pointer to its data file
Yes, it will be reverted.
Thanks,
Alexey
Quoted 4 lines
thanks,
jirka
Re: [PATCH v2 3/4] perf record: enable runtime trace compression
In reply to Jiri OlsaQuoted 37 lines · Jiri Olsa
On Mon, Feb 11, 2019 at 11:23:40PM +0300, Alexey Budankov wrote:
SNIP
Quoted 20 lines
tools/perf/builtin-record.c +8
-5+ compress_fn = (record__comp_enabled (rec) ? + perf_session__zstd_compress : perf_session__zstd_copy); +if (record__aio_enabled (rec)) off =record__aio_get_pos (trace_fd);@@ -799,11 +805,21 @@ static int record__mmap_read_evlist(struct record *rec, struct perf_evlist *evli map->flush =MMAP_FLUSH_DEFAULT ; }if (!record__aio_enabled (rec)) { -if (perf_mmap__push (map, rec, record__pushfn) !=0 ) { -if (sync) - map->flush = flush; - rc =-1 ; - goto out; +if (!record__comp_enabled (rec)) { +if (perf_mmap__push (map, rec, record__pushfn) !=0 ) { +if (sync) + map->flush = flush; + rc =-1 ;tools/perf/builtin-record.c static int record__mmap_read_evlist(struct record *rec, struct perf_evlist *evli · line 822
Quoted 10 lines
tools/perf/builtin-record.c +10
-0+ goto out; + } + }else { +if (perf_mmap__pack (map, rec, record__pushfn, + compress_fn, session) !=0 ) { +if (sync) + map->flush = flush; + rc =-1 ; + goto out; + }I really dont like this.. the non-compression flow needs to stay untouched
Quoted 2 lines
why don't you overload the
record__pushfnfunction for compression processing and stash the data in there?
Yes, this becomes complicated and overloading could be an option to simplify it.
Thanks,
Alexey
Quoted 4 lines
thanks,
jirka
Re: [PATCH v2 3/4] perf record: enable runtime trace compression
In reply to Jiri OlsaQuoted 12 lines · Jiri Olsa
On Mon, Feb 11, 2019 at 11:23:40PM +0300, Alexey Budankov wrote:
SNIP
Quoted 8 lines
tools/perf/util/mmap.c +0
-0@@ -156,6 +156,86 @@ void __weak auxtrace_mmap_params__set_idx(struct auxtrace_mmap_params *mp __mayb { }
Quoted 12 lines
tools/perf/util/mmap.c void __weak auxtrace_mmap_params__set_idx(struct auxtrace_mmap_params *mp __mayb · line 167
Quoted 9 lines
tools/perf/util/mmap.c +9
-0+static ssize_t perf_mmap__capture (struct perf_mmap *md,int idx, +perf_mmap__compress_fn_t compress,void *where) +{ +u64 head =perf_mmap__read_head (md); +unsigned char *data = md->base + page_size; +unsigned long size, size0 =0 ; +void *buf; +int rc =0 ; +size_t mmap_len =perf_mmap__mmap_len (md);again, this function seems like duplicate of
perf_mmap__push, can't you just overload the push callback?
Answered related question in other mail.
Overloading could be an option.
Thanks,
Alexey
Quoted 4 lines
thanks,
jirka
Re: [PATCH v2 3/4] perf record: enable runtime trace compression
In reply to Jiri OlsaQuoted 4 lines · Jiri Olsa
On Mon, Feb 11, 2019 at 11:23:40PM +0300, Alexey Budankov wrote:
SNIP
Quoted 10 lines
tools/perf/util/session.c line 31
Quoted 8 lines
tools/perf/util/session.c +0
-0@@ -29,6 +29,112 @@#include "stat.h" #include "arch/common.h" I think this should not be in
session.cbut separated in new object
Do you mean the new file?
Thanks,
Alexey
Re: [PATCH v2 4/4] perf report: support record trace file decompression
In reply to Jiri OlsaQuoted 22 lines · Jiri Olsa
On Mon, Feb 11, 2019 at 11:25:00PM +0300, Alexey Budankov wrote:
SNIP
Quoted 18 lines
size_tperf_session__zstd_copy(void *to__maybe_unused,tools/perf/util/session.c +4
-1@@ -533,6 +646,8 @@ void perf_tool__fill_defaults(struct perf_tool *tool) tool->time_conv = process_event_op2_stub;if (tool->feature ==NULL ) tool->feature = process_event_op2_stub; +if (tool->compressed ==NULL ) + tool->compressed = perf_session__process_compressed_event; }static void swap_sample_id_all (union perf_event *event,void *data)@@ -1469,7 +1584,8 @@ static s64 perf_session__process_user_event(struct perf_session *session,int fd =perf_data__fd (session->data);int err; -dump_event (session->evlist, event, file_offset, &sample); +if (event->header.type !=PERF_RECORD_COMPRESSED ) +dump_event (session->evlist, event, file_offset, &sample);
Quoted 13 lines
tools/perf/util/session.c static s64 perf_session__process_user_event(struct perf_session *session, · line 1644
Quoted 10 lines
tools/perf/util/session.c +4
-0/* These events are processed right away */ switch (event->header.type) {@@ -1522,6 +1638,11 @@ static s64 perf_session__process_user_event(struct perf_session *session,return tool->time_conv (session, event);case PERF_RECORD_HEADER_FEATURE :return tool->feature (session, event); +case PERF_RECORD_COMPRESSED : + err = tool->compressed (session, event, file_offset); +if (err) +dump_event (session->evlist, event, file_offset, &sample);I'm not sure about having compressed callback at all, but maybe it could be usefull for inject, to get the compressed data..?
This code implements handling thru callbacks, for some reason, so the new part is designed the same way. Inject flow probably will benefit from this approach. It is required to be tested additionally.
Thanks,
Alexey
Quoted 6 lines
I assume inject is working now and it will get you the uncompressed perf.data?
jirka
Re: [PATCH v2 2/4] perf record: implement -z=<level> and --mmap-flush=<thres> options
In reply to Jiri OlsaQuoted 31 lines · Jiri Olsa
Quoted 14 lines · Alexey Budankov
Implement -z,--compression_level=<n> and --mmap-flush=<dump_least_size> options as well as a special
PERF_RECORD_COMPRESSEDrecord that contains compressed parts of kernel data buffer.Because compression requires auxilary memory to implement encoding of kernel data
record->opts.nr_cblocks== -1 signifies to preallocate single AIO data buffer aio.data[0] without accompnying AIO control blocks.Signed-off-by Alexey Budankov <[email protected]>---
Changes in v2:
- enabled allocation aio data buffers for compressionQuoted 15 lines
--- tools/perf/Documentation/perf-record.txt | 9 ++ tools/perf/builtin-record.c | 110 +++++++++++++++++++---- tools/perf/perf.h | 2 + tools/perf/util/env.h | 10 +++ tools/perf/util/event.c | 1 + tools/perf/util/event.h | 7 ++ tools/perf/util/evlist.c | 6 +- tools/perf/util/evlist.h | 3 +- tools/perf/util/header.c | 45 +++++++++- tools/perf/util/header.h | 1 + tools/perf/util/mmap.c | 98 ++++++++++++-------- tools/perf/util/mmap.h | 7 +- tools/perf/util/session.h | 2 + 13 files changed, 240 insertions(+), 61 deletions(-)
Quoted 5 lines
this patch is super complicated.. please think of reviewers when writing code ;-)
it needs to be split to small logical patches,
where each change is explained in changelog
Yes, Will do.
Thanks,
Alexey
Quoted 3 lines
jirka
Re: [PATCH v2 2/4] perf record: implement -z=<level> and --mmap-flush=<thres> options
In reply to Jiri OlsaQuoted 69 lines · Jiri Olsa
On Mon, Feb 11, 2019 at 11:22:38PM +0300, Alexey Budankov wrote:
SNIP
Quoted 62 lines
tools/perf/util/mmap.c +11
-30+static int perf_mmap__aio_mmap_blocks (struct perf_mmap *map); +static int perf_mmap__aio_mmap (struct perf_mmap *map,struct mmap_params *mp) { -int delta_max, i, prio, ret; +int i, ret =0 , init_blocks =1 ; map->aio.nr_cblocks = mp->nr_cblocks; +if (map->aio.nr_cblocks ==-1 ) { + map->aio.nr_cblocks =1 ; + init_blocks =0 ; + } +if (map->aio.nr_cblocks) { - map->aio.aiocb =calloc (map->aio.nr_cblocks,sizeof (struct aiocb *)); -if (!map->aio.aiocb) { -pr_debug2 ("failed to allocate aiocb for data buffer, error %m\n" ); -return -1 ; - } - map->aio.cblocks =calloc (map->aio.nr_cblocks,sizeof (struct aiocb )); -if (!map->aio.cblocks) { -pr_debug2 ("failed to allocate cblocks for data buffer, error %m\n" ); -return -1 ; - } map->aio.data =calloc (map->aio.nr_cblocks,sizeof (void *));if (!map->aio.data) {pr_debug2 ("failed to allocate data buffer, error %m\n" );return -1 ; } - delta_max =sysconf (_SC_AIO_PRIO_DELTA_MAX);for (i =0 ; i < map->aio.nr_cblocks; ++i) { ret =perf_mmap__aio_alloc (map, i);if (ret ==-1 ) {@@ -251,29 +245,16 @@ static int perf_mmap__aio_mmap(struct perf_mmap *map, struct mmap_params *mp) ret =perf_mmap__aio_bind (map, i, map->cpu, mp->affinity);if (ret ==-1 )return -1 ; -/* -* Use cblock.aio_fildes value different from -1 -* to denote started aio write operation on the -* cblock so it requires explicit record__aio_sync() -* call prior the cblock may be reused again. -*/ - map->aio.cblocks[i].aio_fildes =-1 ; -/* -* Allocate cblocks with priority delta to have -* faster aio write system calls because queued requests -* are kept in separate per-prio queues and adding -* a new request will iterate thru shorter per-prio -* list. Blocks with numbers higher than -* _SC_AIO_PRIO_DELTA_MAX go with priority 0. -*/ - prio = delta_max - i; - map->aio.cblocks[i].aio_reqprio = prio >=0 ? prio :0 ; } +if (init_blocks) + ret =perf_mmap__aio_mmap_blocks (map); } -return 0 ; +return ret; }SNIP
Quoted 3 lines
it seems like little refactoring happened in here (up and down) for aio code, which is not explained and I'm unable to follow it.. please separate this in simple change
AIO buffers management has been taken out of HAVE_AIO_SUPPORT define to be used for compression in case of serial streaming. It will be revisited after other issues are addressed.
Thanks,
Alexey
Quoted 45 lines
SNIP
Quoted 39 lines
tools/perf/util/mmap.c +39
-0+#ifdef HAVE_AIO_SUPPORT +static int perf_mmap__aio_mmap_blocks (struct perf_mmap *map) +{ +int delta_max, i, prio; + + map->aio.aiocb =calloc (map->aio.nr_cblocks,sizeof (struct aiocb *)); +if (!map->aio.aiocb) { +pr_debug2 ("failed to allocate aiocb for data buffer, error %m\n" ); +return -1 ; + } + map->aio.cblocks =calloc (map->aio.nr_cblocks,sizeof (struct aiocb )); +if (!map->aio.cblocks) { +pr_debug2 ("failed to allocate cblocks for data buffer, error %m\n" ); +return -1 ; + } + delta_max =sysconf (_SC_AIO_PRIO_DELTA_MAX); +for (i =0 ; i < map->aio.nr_cblocks; ++i) { +/* +* Use cblock.aio_fildes value different from -1 +* to denote started aio write operation on the +* cblock so it requires explicit record__aio_sync() +* call prior the cblock may be reused again. +*/ + map->aio.cblocks[i].aio_fildes =-1 ; +/* +* Allocate cblocks with priority delta to have +* faster aio write system calls because queued requests +* are kept in separate per-prio queues and adding +* a new request will iterate thru shorter per-prio +* list. Blocks with numbers higher than +* _SC_AIO_PRIO_DELTA_MAX go with priority 0. +*/ + prio = delta_max - i; + map->aio.cblocks[i].aio_reqprio = prio >=0 ? prio :0 ; + } + +return 0 ; +} +SNIP
Re: [PATCH v2 3/4] perf record: enable runtime trace compression
In reply to Alexey BudankovQuoted 17 lines · Alexey Budankov
Quoted 4 lines · Jiri Olsa
On Mon, Feb 11, 2019 at 11:23:40PM +0300, Alexey Budankov wrote:
SNIP
Quoted 10 lines
tools/perf/util/session.c line 31
Quoted 8 lines
tools/perf/util/session.c +0
-0@@ -29,6 +29,112 @@#include "stat.h" #include "arch/common.h" I think this should not be in
session.cbut separated in new object
Quoted 1 line
Do you mean the new file?
yes
jirka
Re: [PATCH v2 3/4] perf record: enable runtime trace compression
In reply to Alexey BudankovQuoted 19 lines · Alexey Budankov
Quoted 17 lines · Jiri Olsa
On Mon, Feb 11, 2019 at 11:23:40PM +0300, Alexey Budankov wrote:
SNIP
Quoted 13 lines
tools/perf/builtin-record.c +2
-0@@ -774,6 +775,8 @@ static int record__mmap_read_evlist(struct record *rec, struct perf_evlist *evlistruct perf_mmap *maps;int trace_fd = rec->data.file.fd;off_t off; +struct perf_session *session = rec->session; +perf_mmap__compress_fn_t compress_fn;if (!evlist)return 0 ;@@ -785,6 +788,9 @@ static int record__mmap_read_evlist(struct record *rec, struct perf_evlist *evliif (overwrite && evlist->bkw_mmap_state !=BKW_MMAP_DATA_PENDING )return 0 ;
Quoted 7 lines
Quoted 5 lines
tools/perf/builtin-record.c static int record__mmap_read_evlist(struct record *rec, struct perf_evlist *evli · line 793
Quoted 3 lines
tools/perf/builtin-record.c +3
-0+ compress_fn = (record__comp_enabled (rec) ? + perf_session__zstd_compress : perf_session__zstd_copy); +I don't follow what's the
perf_session__zstd_copyfunction for..?It bridges AIO without compression case.
so current state then.. why do we need new function for that?
jirka
Quoted 43 lines
Thanks,
AlexeyQuoted 39 lines
for !record__comp_enabled case we seem not to use it and calling the current
perf_mmap__pushinterfacehowever I dont see point to have this function at all
jirka
Quoted 27 lines
tools/perf/builtin-record.c +15
-5if (record__aio_enabled (rec)) off =record__aio_get_pos (trace_fd);@@ -799,11 +805,21 @@ static int record__mmap_read_evlist(struct record *rec, struct perf_evlist *evli map->flush =MMAP_FLUSH_DEFAULT ; }if (!record__aio_enabled (rec)) { -if (perf_mmap__push (map, rec, record__pushfn) !=0 ) { -if (sync) - map->flush = flush; - rc =-1 ; - goto out; +if (!record__comp_enabled (rec)) { +if (perf_mmap__push (map, rec, record__pushfn) !=0 ) { +if (sync) + map->flush = flush; + rc =-1 ; + goto out; + } + }else { +if (perf_mmap__pack (map, rec, record__pushfn, + compress_fn, session) !=0 ) { +if (sync) + map->flush = flush; + rc =-1 ; + goto out; + }SNIP
Re: [PATCH v2 2/4] perf record: implement -z=<level> and --mmap-flush=<thres> options
In reply to Alexey BudankovQuoted 23 lines · Alexey Budankov
Quoted 4 lines · Jiri Olsa
On Mon, Feb 11, 2019 at 11:22:38PM +0300, Alexey Budankov wrote:
SNIP
Quoted 16 lines
tools/perf/builtin-record.c int cmd_record(int argc, const char **argv) · line 2255
Quoted 12 lines
tools/perf/builtin-record.c +8
-4-if (rec->opts.nr_cblocks > nr_cblocks_max) - rec->opts.nr_cblocks = nr_cblocks_max; -if (verbose >0 ) -pr_info ("nr_cblocks: %d\n" , rec->opts.nr_cblocks); +if (rec->opts.comp_level >22 ) + rec->opts.comp_level =0 ; +if (record__comp_enabled (rec) && !rec->opts.nr_cblocks) { +/* +* Allocate aio.data[0] buffer for compression. +*/ + rec->opts.nr_cblocks =-1 ; + }I assume you're using aio.data[0] as a buffer for non-aio case as compression buffer.. if that's the case, please dont do that and use separate buffer for that
Quoted 2 lines
Do you want it to be one more field in the mmap struct or aio related fields still could be reworked?
I dont like it to use aio buffers if it's generic feature, so please add new pointer for the needed buffer
jirka
Re: [PATCH v2 2/4] perf record: implement -z=<level> and --mmap-flush=<thres> options
In reply to Alexey BudankovQuoted 71 lines · Alexey Budankov
Quoted 69 lines · Jiri Olsa
On Mon, Feb 11, 2019 at 11:22:38PM +0300, Alexey Budankov wrote:
SNIP
Quoted 62 lines
tools/perf/util/mmap.c +11
-30+static int perf_mmap__aio_mmap_blocks (struct perf_mmap *map); +static int perf_mmap__aio_mmap (struct perf_mmap *map,struct mmap_params *mp) { -int delta_max, i, prio, ret; +int i, ret =0 , init_blocks =1 ; map->aio.nr_cblocks = mp->nr_cblocks; +if (map->aio.nr_cblocks ==-1 ) { + map->aio.nr_cblocks =1 ; + init_blocks =0 ; + } +if (map->aio.nr_cblocks) { - map->aio.aiocb =calloc (map->aio.nr_cblocks,sizeof (struct aiocb *)); -if (!map->aio.aiocb) { -pr_debug2 ("failed to allocate aiocb for data buffer, error %m\n" ); -return -1 ; - } - map->aio.cblocks =calloc (map->aio.nr_cblocks,sizeof (struct aiocb )); -if (!map->aio.cblocks) { -pr_debug2 ("failed to allocate cblocks for data buffer, error %m\n" ); -return -1 ; - } map->aio.data =calloc (map->aio.nr_cblocks,sizeof (void *));if (!map->aio.data) {pr_debug2 ("failed to allocate data buffer, error %m\n" );return -1 ; } - delta_max =sysconf (_SC_AIO_PRIO_DELTA_MAX);for (i =0 ; i < map->aio.nr_cblocks; ++i) { ret =perf_mmap__aio_alloc (map, i);if (ret ==-1 ) {@@ -251,29 +245,16 @@ static int perf_mmap__aio_mmap(struct perf_mmap *map, struct mmap_params *mp) ret =perf_mmap__aio_bind (map, i, map->cpu, mp->affinity);if (ret ==-1 )return -1 ; -/* -* Use cblock.aio_fildes value different from -1 -* to denote started aio write operation on the -* cblock so it requires explicit record__aio_sync() -* call prior the cblock may be reused again. -*/ - map->aio.cblocks[i].aio_fildes =-1 ; -/* -* Allocate cblocks with priority delta to have -* faster aio write system calls because queued requests -* are kept in separate per-prio queues and adding -* a new request will iterate thru shorter per-prio -* list. Blocks with numbers higher than -* _SC_AIO_PRIO_DELTA_MAX go with priority 0. -*/ - prio = delta_max - i; - map->aio.cblocks[i].aio_reqprio = prio >=0 ? prio :0 ; } +if (init_blocks) + ret =perf_mmap__aio_mmap_blocks (map); } -return 0 ; +return ret; }SNIP
Quoted 7 lines
Quoted 3 lines
it seems like little refactoring happened in here (up and down) for aio code, which is not explained and I'm unable to follow it.. please separate this in simple change
AIO buffers management has been taken out of
HAVE_AIO_SUPPORTdefine to be used for compression in case of serial streaming. It will be revisited after other issues are addressed.
as I said earlier, please separate this from aio
jirka
Re: [PATCH v2 3/4] perf record: enable runtime trace compression
In reply to Jiri OlsaQuoted 26 lines · Jiri Olsa
Quoted 25 lines · Alexey Budankov
Quoted 22 lines · Jiri Olsa
On Mon, Feb 11, 2019 at 11:23:40PM +0300, Alexey Budankov wrote:
SNIP
Quoted 10 lines
tools/perf/builtin-record.c +2
-0@@ -774,6 +775,8 @@ static int record__mmap_read_evlist(struct record *rec, struct perf_evlist *evlistruct perf_mmap *maps;int trace_fd = rec->data.file.fd;off_t off; +struct perf_session *session = rec->session; +perf_mmap__compress_fn_t compress_fn;if (!evlist)return 0 ;tools/perf/builtin-record.c static int record__mmap_read_evlist(struct record *rec, struct perf_evlist *evli · line 793
Quoted 6 lines
tools/perf/builtin-record.c +3
-0@@ -785,6 +788,9 @@ static int record__mmap_read_evlist(struct record *rec, struct perf_evlist *evliif (overwrite && evlist->bkw_mmap_state !=BKW_MMAP_DATA_PENDING )return 0 ; + compress_fn = (record__comp_enabled (rec) ? + perf_session__zstd_compress : perf_session__zstd_copy); +I don't follow what's the
perf_session__zstd_copyfunction for..?
Quoted 3 lines
Quoted 1 line
It bridges AIO without compression case.
so current state then.. why do we need new function for that?
This copy() function has the same signature as compress() so it can be used as the replacement in the _pack() call instead of memcpy() with different signature.
Thanks,
Alexey
Quoted 2 lines
jirka
Re: [PATCH v2 2/4] perf record: implement -z=<level> and --mmap-flush=<thres> options
In reply to Jiri OlsaQuoted 24 lines · Jiri Olsa
Quoted 23 lines · Alexey Budankov
Quoted 20 lines · Jiri Olsa
On Mon, Feb 11, 2019 at 11:22:38PM +0300, Alexey Budankov wrote:
SNIP
tools/perf/builtin-record.c int cmd_record(int argc, const char **argv) · line 2255
Quoted 12 lines
tools/perf/builtin-record.c +8
-4-if (rec->opts.nr_cblocks > nr_cblocks_max) - rec->opts.nr_cblocks = nr_cblocks_max; -if (verbose >0 ) -pr_info ("nr_cblocks: %d\n" , rec->opts.nr_cblocks); +if (rec->opts.comp_level >22 ) + rec->opts.comp_level =0 ; +if (record__comp_enabled (rec) && !rec->opts.nr_cblocks) { +/* +* Allocate aio.data[0] buffer for compression. +*/ + rec->opts.nr_cblocks =-1 ; + }I assume you're using aio.data[0] as a buffer for non-aio case as compression buffer.. if that's the case, please dont do that and use separate buffer for that
Quoted 5 lines
Quoted 2 lines
Do you want it to be one more field in the mmap struct or aio related fields still could be reworked?
I dont like it to use aio buffers if it's generic feature, so please add new pointer for the needed buffer
Ok.
Thanks,
Alexey
Quoted 3 lines
jirka
Re: [PATCH v2 2/4] perf record: implement -z=<level> and --mmap-flush=<thres> options
In reply to Jiri OlsaQuoted 76 lines · Jiri Olsa
Quoted 75 lines · Alexey Budankov
Quoted 72 lines · Jiri Olsa
On Mon, Feb 11, 2019 at 11:22:38PM +0300, Alexey Budankov wrote:
SNIP
Quoted 52 lines
tools/perf/util/mmap.c +8
-27+static int perf_mmap__aio_mmap_blocks (struct perf_mmap *map); +static int perf_mmap__aio_mmap (struct perf_mmap *map,struct mmap_params *mp) { -int delta_max, i, prio, ret; +int i, ret =0 , init_blocks =1 ; map->aio.nr_cblocks = mp->nr_cblocks; +if (map->aio.nr_cblocks ==-1 ) { + map->aio.nr_cblocks =1 ; + init_blocks =0 ; + } +if (map->aio.nr_cblocks) { - map->aio.aiocb =calloc (map->aio.nr_cblocks,sizeof (struct aiocb *)); -if (!map->aio.aiocb) { -pr_debug2 ("failed to allocate aiocb for data buffer, error %m\n" ); -return -1 ; - } - map->aio.cblocks =calloc (map->aio.nr_cblocks,sizeof (struct aiocb )); -if (!map->aio.cblocks) { -pr_debug2 ("failed to allocate cblocks for data buffer, error %m\n" ); -return -1 ; - } map->aio.data =calloc (map->aio.nr_cblocks,sizeof (void *));if (!map->aio.data) {pr_debug2 ("failed to allocate data buffer, error %m\n" );return -1 ; } - delta_max =sysconf (_SC_AIO_PRIO_DELTA_MAX);for (i =0 ; i < map->aio.nr_cblocks; ++i) { ret =perf_mmap__aio_alloc (map, i);if (ret ==-1 ) {@@ -251,29 +245,16 @@ static int perf_mmap__aio_mmap(struct perf_mmap *map, struct mmap_params *mp) ret =perf_mmap__aio_bind (map, i, map->cpu, mp->affinity);if (ret ==-1 )return -1 ; -/* -* Use cblock.aio_fildes value different from -1 -* to denote started aio write operation on the -* cblock so it requires explicit record__aio_sync() -* call prior the cblock may be reused again. -*/ - map->aio.cblocks[i].aio_fildes =-1 ; -/* -* Allocate cblocks with priority delta to have -* faster aio write system calls because queued requests -* are kept in separate per-prio queues and adding -* a new request will iterate thru shorter per-prio -* list. Blocks with numbers higher than -* _SC_AIO_PRIO_DELTA_MAX go with priority 0. -*/ tools/perf/util/mmap.c static int perf_mmap__aio_mmap(struct perf_mmap *map, struct mmap_params *mp) · line 254
Quoted 10 lines
tools/perf/util/mmap.c +3
-3- prio = delta_max - i; - map->aio.cblocks[i].aio_reqprio = prio >=0 ? prio :0 ; } +if (init_blocks) + ret =perf_mmap__aio_mmap_blocks (map); } -return 0 ; +return ret; }SNIP
it seems like little refactoring happened in here (up and down) for aio code, which is not explained and I'm unable to follow it.. please separate this in simple change
Quoted 5 lines
Quoted 3 lines
AIO buffers management has been taken out of
HAVE_AIO_SUPPORTdefine to be used for compression in case of serial streaming. It will be revisited after other issues are addressed.as I said earlier, please separate this from aio
Ok.
Thanks,
Alexey
Quoted 3 lines
jirka
Re: [PATCH v2 3/4] perf record: enable runtime trace compression
In reply to Jiri OlsaQuoted 18 lines · Jiri Olsa
Quoted 17 lines · Alexey Budankov
Quoted 14 lines · Jiri Olsa
On Mon, Feb 11, 2019 at 11:23:40PM +0300, Alexey Budankov wrote:
SNIP
tools/perf/util/session.c line 31
Quoted 8 lines
tools/perf/util/session.c +0
-0@@ -29,6 +29,112 @@#include "stat.h" #include "arch/common.h" I think this should not be in
session.cbut separated in new object
Quoted 3 lines
Quoted 1 line
Do you mean the new file?
yes
tools/perf/util/compress.{c,h}
Thanks,
Alexey
Quoted 3 lines
jirka
Re: [PATCH v2 2/4] perf record: implement -z=<level> and --mmap-flush=<thres> options
In reply to Alexey BudankovQuoted 26 lines · Alexey Budankov
Quoted 24 lines · Jiri Olsa
Quoted 23 lines · Alexey Budankov
Quoted 20 lines · Jiri Olsa
On Mon, Feb 11, 2019 at 11:22:38PM +0300, Alexey Budankov wrote:
SNIP
tools/perf/builtin-record.c int cmd_record(int argc, const char **argv) · line 2255
Quoted 12 lines
tools/perf/builtin-record.c +8
-4-if (rec->opts.nr_cblocks > nr_cblocks_max) - rec->opts.nr_cblocks = nr_cblocks_max; -if (verbose >0 ) -pr_info ("nr_cblocks: %d\n" , rec->opts.nr_cblocks); +if (rec->opts.comp_level >22 ) + rec->opts.comp_level =0 ; +if (record__comp_enabled (rec) && !rec->opts.nr_cblocks) { +/* +* Allocate aio.data[0] buffer for compression. +*/ + rec->opts.nr_cblocks =-1 ; + }I assume you're using aio.data[0] as a buffer for non-aio case as compression buffer.. if that's the case, please dont do that and use separate buffer for that
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
Do you want it to be one more field in the mmap struct or aio related fields still could be reworked?
I dont like it to use aio buffers if it's generic feature, so please add new pointer for the needed buffer
Ok.
Addressed.
Quoted 8 lines
Thanks,
AlexeyQuoted 3 lines
jirka
Re: [PATCH v2 2/4] perf record: implement -z=<level> and --mmap-flush=<thres> options
In reply to Alexey BudankovQuoted 78 lines · Alexey Budankov
Quoted 76 lines · Jiri Olsa
Quoted 75 lines · Alexey Budankov
Quoted 72 lines · Jiri Olsa
On Mon, Feb 11, 2019 at 11:22:38PM +0300, Alexey Budankov wrote:
SNIP
Quoted 52 lines
tools/perf/util/mmap.c +8
-27+static int perf_mmap__aio_mmap_blocks (struct perf_mmap *map); +static int perf_mmap__aio_mmap (struct perf_mmap *map,struct mmap_params *mp) { -int delta_max, i, prio, ret; +int i, ret =0 , init_blocks =1 ; map->aio.nr_cblocks = mp->nr_cblocks; +if (map->aio.nr_cblocks ==-1 ) { + map->aio.nr_cblocks =1 ; + init_blocks =0 ; + } +if (map->aio.nr_cblocks) { - map->aio.aiocb =calloc (map->aio.nr_cblocks,sizeof (struct aiocb *)); -if (!map->aio.aiocb) { -pr_debug2 ("failed to allocate aiocb for data buffer, error %m\n" ); -return -1 ; - } - map->aio.cblocks =calloc (map->aio.nr_cblocks,sizeof (struct aiocb )); -if (!map->aio.cblocks) { -pr_debug2 ("failed to allocate cblocks for data buffer, error %m\n" ); -return -1 ; - } map->aio.data =calloc (map->aio.nr_cblocks,sizeof (void *));if (!map->aio.data) {pr_debug2 ("failed to allocate data buffer, error %m\n" );return -1 ; } - delta_max =sysconf (_SC_AIO_PRIO_DELTA_MAX);for (i =0 ; i < map->aio.nr_cblocks; ++i) { ret =perf_mmap__aio_alloc (map, i);if (ret ==-1 ) {@@ -251,29 +245,16 @@ static int perf_mmap__aio_mmap(struct perf_mmap *map, struct mmap_params *mp) ret =perf_mmap__aio_bind (map, i, map->cpu, mp->affinity);if (ret ==-1 )return -1 ; -/* -* Use cblock.aio_fildes value different from -1 -* to denote started aio write operation on the -* cblock so it requires explicit record__aio_sync() -* call prior the cblock may be reused again. -*/ - map->aio.cblocks[i].aio_fildes =-1 ; -/* -* Allocate cblocks with priority delta to have -* faster aio write system calls because queued requests -* are kept in separate per-prio queues and adding -* a new request will iterate thru shorter per-prio -* list. Blocks with numbers higher than -* _SC_AIO_PRIO_DELTA_MAX go with priority 0. -*/ tools/perf/util/mmap.c static int perf_mmap__aio_mmap(struct perf_mmap *map, struct mmap_params *mp) · line 254
Quoted 10 lines
tools/perf/util/mmap.c +3
-3- prio = delta_max - i; - map->aio.cblocks[i].aio_reqprio = prio >=0 ? prio :0 ; } +if (init_blocks) + ret =perf_mmap__aio_mmap_blocks (map); } -return 0 ; +return ret; }SNIP
it seems like little refactoring happened in here (up and down) for aio code, which is not explained and I'm unable to follow it.. please separate this in simple change
Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
AIO buffers management has been taken out of
HAVE_AIO_SUPPORTdefine to be used for compression in case of serial streaming. It will be revisited after other issues are addressed.as I said earlier, please separate this from aio
Ok.
Addressed.
Quoted 9 lines
Thanks,
AlexeyQuoted 3 lines
jirka
Re: [PATCH v2 3/4] perf record: enable runtime trace compression
In reply to Alexey BudankovQuoted 39 lines · Alexey Budankov
Quoted 37 lines · Jiri Olsa
On Mon, Feb 11, 2019 at 11:23:40PM +0300, Alexey Budankov wrote:
SNIP
Quoted 20 lines
tools/perf/builtin-record.c +8
-5+ compress_fn = (record__comp_enabled (rec) ? + perf_session__zstd_compress : perf_session__zstd_copy); +if (record__aio_enabled (rec)) off =record__aio_get_pos (trace_fd);@@ -799,11 +805,21 @@ static int record__mmap_read_evlist(struct record *rec, struct perf_evlist *evli map->flush =MMAP_FLUSH_DEFAULT ; }if (!record__aio_enabled (rec)) { -if (perf_mmap__push (map, rec, record__pushfn) !=0 ) { -if (sync) - map->flush = flush; - rc =-1 ; - goto out; +if (!record__comp_enabled (rec)) { +if (perf_mmap__push (map, rec, record__pushfn) !=0 ) { +if (sync) + map->flush = flush; + rc =-1 ;tools/perf/builtin-record.c static int record__mmap_read_evlist(struct record *rec, struct perf_evlist *evli · line 822
Quoted 10 lines
tools/perf/builtin-record.c +10
-0+ goto out; + } + }else { +if (perf_mmap__pack (map, rec, record__pushfn, + compress_fn, session) !=0 ) { +if (sync) + map->flush = flush; + rc =-1 ; + goto out; + }I really dont like this.. the non-compression flow needs to stay untouched
Quoted 4 lines
Quoted 2 lines
why don't you overload the
record__pushfnfunction for compression processing and stash the data in there?Yes, this becomes complicated and overloading could be an option to simplify it.
Redesigned compress() to be the param for __push() __aio_push() so this __comp_enabled() if could be avoided. Overloading of __pushfn() doesn't fit for aio case.
Quoted 9 lines
Thanks,
AlexeyQuoted 4 lines
thanks,
jirka
Re: [PATCH v2 4/4] perf report: support record trace file decompression
In reply to Alexey BudankovQuoted 38 lines · Alexey Budankov
Quoted 22 lines · Jiri Olsa
On Mon, Feb 11, 2019 at 11:25:00PM +0300, Alexey Budankov wrote:
SNIP
Quoted 18 lines
size_tperf_session__zstd_copy(void *to__maybe_unused,tools/perf/util/session.c +4
-1@@ -533,6 +646,8 @@ void perf_tool__fill_defaults(struct perf_tool *tool) tool->time_conv = process_event_op2_stub;if (tool->feature ==NULL ) tool->feature = process_event_op2_stub; +if (tool->compressed ==NULL ) + tool->compressed = perf_session__process_compressed_event; }static void swap_sample_id_all (union perf_event *event,void *data)@@ -1469,7 +1584,8 @@ static s64 perf_session__process_user_event(struct perf_session *session,int fd =perf_data__fd (session->data);int err; -dump_event (session->evlist, event, file_offset, &sample); +if (event->header.type !=PERF_RECORD_COMPRESSED ) +dump_event (session->evlist, event, file_offset, &sample);Quoted 13 lines
tools/perf/util/session.c static s64 perf_session__process_user_event(struct perf_session *session, · line 1644
Quoted 10 lines
tools/perf/util/session.c +4
-0/* These events are processed right away */ switch (event->header.type) {@@ -1522,6 +1638,11 @@ static s64 perf_session__process_user_event(struct perf_session *session,return tool->time_conv (session, event);case PERF_RECORD_HEADER_FEATURE :return tool->feature (session, event); +case PERF_RECORD_COMPRESSED : + err = tool->compressed (session, event, file_offset); +if (err) +dump_event (session->evlist, event, file_offset, &sample);I'm not sure about having compressed callback at all, but maybe it could be usefull for inject, to get the compressed data..?
Quoted 4 lines
This code implements handling thru callbacks, for some reason, so the new part is designed the same way. Inject flow probably will benefit from this approach. It is required to be tested additionally.
Inject flow is now enabled too.
Quoted 12 lines
Thanks,
AlexeyQuoted 6 lines
I assume inject is working now and it will get you the uncompressed perf.data?
jirka
Re: [PATCH v2 2/4] perf record: implement -z=<level> and --mmap-flush=<thres> options
In reply to Alexey BudankovQuoted 31 lines · Alexey Budankov
Quoted 14 lines · Jiri Olsa
On Mon, Feb 11, 2019 at 11:22:38PM +0300, Alexey Budankov wrote:
SNIP
Quoted 10 lines
tools/perf/builtin-record.c +4
-0@@ -1147,6 +1193,10 @@ static int __cmd_record(struct record *rec, int argc, const char **argv) fd =perf_data__fd (data); rec->session = session; + rec->opts.comp_level =0 ; + session->header.env.comp_level = rec->opts.comp_level; + session->header.env.comp_type =PERF_COMP_NONE ; +record__init_features (rec);Quoted 8 lines
tools/perf/builtin-record.c static int __cmd_record(struct record *rec, int argc, const char **argv) · line 1229
Quoted 6 lines
tools/perf/builtin-record.c +1
-0if (rec->opts.use_clockid && rec->opts.clockid_res_ns)@@ -1176,6 +1226,7 @@ static int __cmd_record(struct record *rec, int argc, const char **argv) err =-1 ; goto out_child; } + session->header.env.comp_mmap_len = session->evlist->mmap_len;so the
comp_mmap_lenis the max length of the compressed packet?
comp_mmap_lenis the size of buffer to encompass one compressed chunk of data after its decompression.Quoted 2 lines
any idea if this value might have some impact on the processing speed?
Quoted 7 lines
It increases memory consumption at the loading and processing stages.
Quoted 3 lines
I see you mentioned the size reduction, could you also meassure the record overhead?
Let's get back to this after the code review.
Overhead numbers are provided in v3.
Quoted 9 lines
Thanks,
AlexeyQuoted 4 lines
thanks,
jirka
Re: [PATCH v2 4/4] perf report: support record trace file decompression
In reply to a message not in the archiveHi,
Quoted 12 lines · Alexey Budankov
Quoted 10 lines · Alexey Budankov
Quoted 8 lines · Jiri Olsa
Quoted 7 lines · Alexey Budankov
PERF_RECORD_COMPRESSEDrecords are decompressed from trace file into a linked list of mmaped memory regions using streaming Zstandard API. After that the regions are loaded fetching uncompressed events. When dumping raw trace (e.g., perf report -D --header) file offsets of events from compressed records are set to zero.
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 4 lines
Signed-off-by Alexey Budankov <[email protected]>---
Changes in v2:
- moved compression/decompression code to session layercould you please add some automated test for this?
I think it could be part of the sample synthesize test
Do you mean this one?
tools/perf/tests/sample-parsing.c
Thanks,
Alexey
Quoted 14 lines
Quoted 14 lines
Quoted 3 lines
or even new script test under tests/shell.. or anything else really.. just to see that we have some clean interface for this that we can test
This is possible. Let's complete the code review first.
Thanks,
AlexeyQuoted 4 lines
thanks,
jirka
Re: [PATCH v2 4/4] perf report: support record trace file decompression
Quoted 20 lines · Alexey Budankov
Hi,
Quoted 17 lines · Alexey Budankov
Quoted 15 lines · Alexey Budankov
Quoted 13 lines · Jiri Olsa
Quoted 11 lines · Alexey Budankov
PERF_RECORD_COMPRESSEDrecords are decompressed from trace file into a linked list of mmaped memory regions using streaming Zstandard API. After that the regions are loaded fetching uncompressed events. When dumping raw trace (e.g., perf report -D --header) file offsets of events from compressed records are set to zero.Signed-off-by Alexey Budankov <[email protected]>---
Changes in v2:
- moved compression/decompression code to session layer
Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
could you please add some automated test for this?
I think it could be part of the sample synthesize test
Do you mean this one?
tools/perf/tests/sample-parsing.c
yea, but on second look it does not look suitable, more likely new shell test under tests/shell?
besically any test that would test that compression and decompression of events
thanks,
jirka