Skip to contentSkip to conversation

Add perf.data tracepoint events to trace.dat conversion

RFC PATCH New Tanushree Shah 2 participants 11 messages Aug 22 perf events

lore
Share & tools
lore
#0 lore raw

v4 cover Add perf.data tracepoint events to trace.dat conversion

This RFC patch series introduces support for converting perf.data files containing tracepoint events into trace.dat format, enabling seamless visualization and analysis using KernelShark.

======================
Background and Motivation
======================

Currently, perf and trace-cmd operate as separate tracing ecosystems with incompatible data formats. Users who collect tracepoint data with 'perf record' cannot easily visualize it in KernelShark's graphical timeline view or leverage trace-cmd's analysis capabilities.

This creates workflow friction when users need to:

- Visualize perf tracepoint data in KernelShark's interactive graphical timeline
- Share trace data between perf and trace-cmd workflows and toolchains
- Perform architecture-independent conversion and analysis of traces

This conversion bridge eliminates these barriers by enabling seamless data exchange between perf and trace-cmd ecosystems, allowing users to choose the best tool for each analysis phase.

======================
Implementation Overview
======================

The series implements the trace.dat file format specification (version 7) within perf's data conversion framework.

**Patch 1/5: Core trace.dat Export Infrastructure** Introduces util/trace-dat.c and util/trace-dat.h implementing:
- Per-CPU raw event buffer management (init, collect, free)
- Ftrace ring buffer page construction
- trace.dat section writers (strings, options, flyrecord sections)

**Patch 2/5: Metadata Integration**
Extends util/trace-event-read.c to write trace.dat metadata during perf.data
parsing:
- Initial format header (magic, version, endian, page size, compression)
- Section 16: HEADER INFO (header_page + header_event)
- Section 17: FTRACE EVENT FORMATS
- Section 18: EVENT FORMATS (per system/event format files)
- Section 19: KALLSYMS
- Section 21: CMDLINES
- Section 15: STRINGS (written last after all sections)

**Patch 3/5: Conversion Backend**
Implements util/data-convert-trace.c with trace_convert__perf2dat() function:
- Processes PERF_TYPE_TRACEPOINT samples via process_sample_event()
- Collects raw event data per-CPU using trace_dat__collect_cpu_event()
- Writes OPTIONS sections (CPUCOUNT, TRACECLOCK, metadata offsets)
- Writes FLYRECORD section with per-CPU ring buffer pages

**Patch 4/5: User Interface**
Extends tools/perf/builtin-data.c with --to-trace-dat option:
- Adds command-line option for trace.dat output
- Mutually exclusive with --to-ctf and --to-json
- Calls trace_convert__perf2dat() to perform conversion

**Patch 5/5: Shell Test**
Adds a shell test (tools/perf/tests/shell/) covering normal tracepoint recordings, pipe mode and mixed tracepoint/non-tracepoint recording conversions and --force flag behaviour.

======================
Current Implementation Details
======================

**trace.dat Format Version:**
The implementation currently targets trace.dat format version 7, which is the stable version supported by current trace-cmd releases (v3.x). This version is hardcoded to ensure compatibility with existing trace-cmd and KernelShark installations. Future enhancements could add version negotiation or support for newer format versions as they become standardized.

**Compression Strategy:**
Compression is explicitly disabled (set to NONE) in the generated trace.dat files.
This design choice:
- Simplifies the initial implementation and testing
- Ensures maximum compatibility across trace-cmd versions
- Avoids external compression library dependencies

Future work could add support for various compression algorithms (zlib, zstd, lz4) with runtime selection via command-line options, significantly reducing file sizes for large traces.

======================
Usage Example
======================

```bash
*Record tracepoint events with perf*
perf record -e sched:sched_switch -e sched:sched_wakeup -a sleep 10

*Convert to trace.dat format*
perf data convert --to-trace-dat=output.dat

*Verify trace.dat structure*
trace-cmd dump --summary output.dat

*Analyze with trace-cmd*
trace-cmd report output.dat

*Visualize in KernelShark*
kernelshark output.dat
```

**Conversion Output:**

```
[ perf data convert: Converted 'perf.data' into trace.dat format
'output.dat' ]
[ perf data convert: Converted 2684 events ]
```
**trace-cmd dump --summary Output:**
```
 Tracing meta data in file output.dat:
	[Initial format]
		7	[Version]
		0	[Little endian]
		8	[Bytes in a long]
		65536	[Page size, bytes]
		none	[Compression algorithm]
			[Compression version]
	[buffer "", "local" clock, 65536 page size, 16 cpus, 1048576 bytes
    flyrecord data]
	[10 options]
	[Saved command lines, 0 bytes]
	[Kallsyms, 0 bytes]
	[Ftrace format, 0 events]
	[Header page, 206 bytes]
	[Header event, 205 bytes]
	[Events format, 1 systems]
	[9 sections]
```	

======================
Testing and Verification
======================

The series has been extensively tested with:
- Various tracepoint events (sched, irq, syscalls, block I/O)
- Mixed recordings containing both tracepoint and non-tracepoint events (only tracepoints converted)
- Verification with trace-cmd report and KernelShark visualization
- Memory leak testing with Valgrind (0 bytes leaked)
- Cross-architecture testing: v1 tested x86_64 and ppc64le. v2 adds s390 (big-endian) perf.data converted on both ppc64le (little-endian) and x86_64 (little-endian) hosts, in addition to same-arch x86_64 (LE->LE) and ppc64le (BE->BE) conversion.
- Pipe mode support has been tested end-to-end. (in v2)
- Time filtering: verified --time option correctly limits converted events to the requested range, consistent with --to-json behaviour. (in v4)

All generated trace.dat files successfully open in:
- trace-cmd report (v3.1+)
- KernelShark (v2.0+)

======================
Next Steps
======================

We would highly appreciate reviews, comments, and feedback on:
- The overall architectural approach and integration points
- Compatibility considerations with trace-cmd ecosystem
- Performance characteristics for large-scale traces
- Additional use cases or workflow scenarios
- Future enhancement priorities

---
Changes in v4

Addressing Sashiko AI review findings on v3:

Timestamp correctness (trace-dat.c):
- Fix TIME_EXTEND delta_upper shift: >> 5 (TRACE_DAT_RECORD_TIME_SHIFT) should be >> 27, causing wildly inflated timestamps in trace-cmd.
- Advance base_ts after every event so time_delta is relative to the preceding record, not a stale page base.
- Introduce page_base_ts updated only at page boundaries, keeping the page header timestamp correct independently of per-event base_ts.

Memory safety (trace-dat.c):
- Fix leak on TIME_EXTEND calloc() failure: use goto out_free instead of return -ENOMEM so page_records/page_rec_sizes are freed.

I/O correctness (trace-event-read.c):
- Check fseek() return values when patching the event formats section size; set trace_dat_write_failed on any failure to prevent silent corruption of subsequent writes.

Resource management (data-convert-trace.c):
- Fix orphaned 0-byte output file when fdopen() fails in !opts->force
path: use goto out_close so unlink() is called on error.
- Honor --time filtering using perf_time__parse_for_ranges() and perf_time__ranges_skip_sample(), consistent with JSON/CTF converters.
- Add explicit <stdio.h> include for fdopen()/fopen()/fclose().

Header (trace-dat.h):
- Add explicit <stdint.h> for uint16_t/uint32_t/uint64_t on musl libc.

Shell test (patch 5):
- Replace 'cycles' with 'cpu-clock -a' for portability on s390, KVM guests and unprivileged containers without hardware PMU support.
- Add -a to all perf record/sleep invocations per Ian's suggestion.
- Fix EXIT trap to pass through exit code 2 (skip) unchanged.
- Move second mktemp after trap registration to avoid temp file leak.
- Drop rm -f "$result" before converter in test_trace_converter_command; --force handles overwriting and deleting first opens a symlink race.
- Move rm -f "$result" to before perf record in pipe and mixed-events tests to prevent stale output being validated on early return.

Changes in v3

- Rebase on latest perf-tools-next and resolve merge conflicts.
- Drop evsel parameter from process_sample_event() following upstream commit "perf tool: Remove evsel from tool APIs that pass the sample".

Changes in v2

Addressing the Sashiko AI review findings on v1:

Cross-arch correctness:
- Introduce to_file_u16/u32/u64 helpers (wrapping tep_read_number()) to write all multi-byte fields in the recorded machine's byte order; apply throughout metadata sections and flyrecord page/record headers (ts, commit, TIME_EXTEND, large-event data_len).
- Fix flyrecord record header bit layout for big-endian files. The record header word bit layout differs by file endianness, matching kbuffer-parse.c type_len4host()/ts4host():
LE: type_len in bits [4:0], time_delta in bits [31:5]
BE: type_len in bits [31:27], time_delta in bits [26:0]

Pipe mode:
- Add process_attr(), process_feature(), and process_tracing_data() callbacks required for pipe mode operation.
- Defer CPU buffer initialisation until the first tracepoint sample, after process_feature()/process_tracing_data() have populated the session header. This ensures the recorded machine's CPU count is used rather than the host's - critical for cross-platform analysis.

Format compliance:
- Implement TIME_EXTEND records for timestamp deltas >27 bits to prevent silent truncation and maintain chronological ordering.
- Fix large event encoding (>=29 words): use type_len=0 with a separate 32-bit length word, avoiding collision with reserved types (PADDING=29, TIME_EXTEND=30, TIME_STAMP=31).
- Add bounds check rejecting records larger than a page payload before batching, preventing heap overflow in trace_dat__write_page().
- Fix flyrecord section_size to exclude the 16-byte section header, matching trace.dat specification and trace-cmd behaviour.

CLI behavior:
- Fix --force flag: open with O_CREAT|O_EXCL when force is not set, failing with -EEXIST instead of silently overwriting existing files.

Memory safety:
- Fix realloc overwrite of cpu_events->events and page_records on
failure: use temporary pointers, only commit on success.
- Fix use-after-free/double-free in sequential page_records realloc
failure: replace with malloc+memcpy+free pattern.
- Fix section_size computed from before section header position.
- Add NULL checks for get_tracing_file(), calloc() padding, and trace_dat_options_offset assignment on write failure.
- Use goto out_free on record allocation failure to avoid leaking accumulated page_records entries.
- Replace direct read() with do_read() in read_proc_kallsyms() to handle short reads correctly.
- On fwrite failure, set trace_dat_write_failed and continue parsing so that perf.data processing completes normally.

Testing (new in v2):
- Add shell test covering conversion, trace-cmd dump validation, sched_switch event verification, and --force flag behaviour.

Documentation (new in v2):
- Add documentation for 'perf data convert --to-trace-dat', covering usage and supported options.

v1: https://lore.kernel.org/linux-perf-users/[email protected]/

Tanushree Shah (5):
perf/trace-dat: Add trace.dat export infrastructure perf/trace-event: Write trace.dat metadata sections during parsing perf data-convert: Add perf.data to trace.dat conversion backend perf data: Add --to-trace-dat option for converting perf.data
tracepoint events into trace.dat format
perf test: Add test validating trace.dat generated by 'perf data

    convert --to-trace-dat'
 tools/perf/Documentation/perf-data.txt        |   7 +
 tools/perf/builtin-data.c                     |  43 +
 ...rf_data_converter_tracepoints_trace_dat.sh | 173 ++++
 tools/perf/util/Build                         |   2 +
 tools/perf/util/data-convert-trace.c          | 264 ++++++
 tools/perf/util/data-convert.h                |   4 +
 tools/perf/util/trace-dat.c                   | 883 ++++++++++++++++++
 tools/perf/util/trace-dat.h                   | 114 +++
 tools/perf/util/trace-event-read.c            | 307 +++++-
 9 files changed, 1790 insertions(+), 7 deletions(-)
 create mode 100755 tools/perf/tests/shell/test_perf_data_converter_tracepoints_trace_dat.sh
 create mode 100644 tools/perf/util/data-convert-trace.c
 create mode 100644 tools/perf/util/trace-dat.c
 create mode 100644 tools/perf/util/trace-dat.h
Signature · 1 line
2.47.1
#1 lore raw

RFC PATCH v4 1/5 perf/trace-dat: Add trace.dat export infrastructure

Add new utility files util/trace-dat.c and util/trace-dat.h implementing the infrastructure for exporting perf.data tracepoints to trace.dat format compatible with trace-cmd and KernelShark.

trace-dat.c defines all globals and functions needed for:
- Per-cpu raw event buffer management (init_cpu_buffers, collect_cpu_event, free_cpu_buffers).
- ftrace ring buffer page construction (write_page, write_cpu_dat), including TIME_EXTEND records for large timestamp deltas.
- trace.dat section writers (write_strings_section, write_options_section1, write_options_section2, write_flyrecord_section).

trace-dat.h declares all globals and function prototypes to be used by data-convert-trace.c and trace-event-read.c.

---
 tools/perf/util/Build       |   1 +
 tools/perf/util/trace-dat.c | 825 ++++++++++++++++++++++++++++++++++++
 tools/perf/util/trace-dat.h |  83 ++++
 3 files changed, 909 insertions(+)
 create mode 100644 tools/perf/util/trace-dat.c
 create mode 100644 tools/perf/util/trace-dat.h
tools/perf/util/Build +1 -0
index b26a0b1ddfa3..12f5bbe32770 100644
@@ -100,6 +100,7 @@ perf-util-y += trace-event-scripting.o
 perf-util-$(CONFIG_LIBTRACEEVENT) += trace-event.o
 perf-util-$(CONFIG_LIBTRACEEVENT) += trace-event-parse.o
 perf-util-$(CONFIG_LIBTRACEEVENT) += trace-event-read.o
+perf-util-$(CONFIG_LIBTRACEEVENT) += trace-dat.o
 perf-util-y += sort.o
 perf-util-y += hist.o
 perf-util-y += util.o
tools/perf/util/trace-dat.c new file +825 -0

825 lines not on this page · raw message

tools/perf/util/trace-dat.h new file +83 -0

83 lines not on this page · raw message

Signature · 1 line
2.47.1
#2 lore raw

Re: [RFC PATCH v4 1/5] perf/trace-dat: Add trace.dat export infrastructure

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [Medium] Large trace events encode their payload length incorrectly, causing alignment errors and event…

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Medium] Large trace events encode their payload length incorrectly, causing alignment errors and event truncation.
- [Medium] Querying the local host system's tracefs for the trace_clock during file conversion leaks host environment state into the converted trace. --

--- Patch [1]: [RFC PATCH v4 1/5] perf/trace-dat: Add trace.dat export infrastructure --- commit 162f3240dcf1a2672aa00440674197558e8f2cec
Author: Tanushree Shah <[email protected]>

perf/trace-dat: Add trace.dat export infrastructure

Add new utility files util/trace-dat.c and util/trace-dat.h implementing the infrastructure for exporting perf.data tracepoints to trace.dat format compatible with trace-cmd and KernelShark.

Quoted 4 lines
tools/perf/util/trace-dat.c new file +0 -0
index 0000000000000..55a7bd982c6f8

[ ... ]

Quoted 2 lines
tools/perf/util/trace-dat.c new file +2 -0
+static int trace_dat__write_cpu_dat(FILE *fp, int cpu, unsigned long long *file_offset_out)
+{

[ ... ]

tools/perf/util/trace-dat.c line 315

Quoted 5 lines
tools/perf/util/trace-dat.c new file +5 -0
+		memcpy(data_rec, &hdr_word, TRACE_DAT_WORD_SIZE);
+
+		/* Large events: write actual byte length after header */
+		if (type_len == 0)
+			memcpy(data_rec + TRACE_DAT_WORD_SIZE, &data_len, TRACE_DAT_WORD_SIZE);

[Severity: Medium]
When writing large events (where type_len == 0), is data_len the correct value to write here? Ftrace ring buffer parsers typically expect this field to contain the size of the payload plus the 4-byte size field itself, properly padded to 4-byte alignment, rather than just the raw event size. Could this lead to event truncation or alignment errors when parsing?

[ ... ]

Quoted 2 lines
tools/perf/util/trace-dat.c new file +2 -0
+int trace_dat__write_options_section1(void)
+{

[ ... ]

tools/perf/util/trace-dat.c line 484

Quoted 10 lines
tools/perf/util/trace-dat.c new file +10 -0
+	/* TRACECLOCK option */
+	opt_id = TRACE_DAT_OPTION_TRACECLOCK;
+
+	path = get_tracing_file("trace_clock");
+	if (path) {
+		clock_file = fopen(path, "r");
+		put_tracing_file(path);
+	} else {
+		clock_file = NULL;
+	}

[Severity: Medium]
Does querying the host system's tracefs mount here break cross-platform analysis? Since this is an offline conversion of a perf.data file, the host machine's current trace clock might not match the clock used when the trace was originally recorded. Should this extract the clock from the perf_env instead?

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
#3 lore raw

RFC PATCH v4 2/5 perf/trace-event: Write trace.dat metadata sections during parsing

In reply to Tanushree Shah

Perf already captures the tracing metadata as a part of data section in perf.data

When trace_dat_fp is set, write trace.dat compatible metadata sections using the perf provided raw buffers.

Sections written:
- Initial format header (magic, version, endian, long_size, page_size, compression, options_offset placeholder)
- Section 16: HEADER INFO (header_page + header_event)
- Section 17: FTRACE EVENT FORMATS
- Section 18: EVENT FORMATS (per system/event format files)
- Section 19: KALLSYMS
- Section 21: CMDLINES
- Section 15: STRINGS (written last after all sections)

Introduce to_file_u16/u32/u64 helpers wrapping tep_read_number() to write all trace.dat metadata in the recorded machine's byte order, ensuring correct cross-platform compatibility.

---
 tools/perf/util/trace-dat.c        | 212 ++++++++++++--------
 tools/perf/util/trace-dat.h        |  40 +++-
 tools/perf/util/trace-event-read.c | 307 ++++++++++++++++++++++++++++-
 3 files changed, 465 insertions(+), 94 deletions(-)
tools/perf/util/trace-dat.c +130 -82
index 55a7bd982c6f..f71e03716e27 100644
@@ -43,6 +43,7 @@
 /* Buffer size for reading trace_clock string from debugfs/tracefs */
 #define CLOCK_BUFFER_SIZE 256
 
+bool trace_dat_write_failed;
 FILE *trace_dat_fp;
 int trace_dat_page_size;
 int trace_dat_nr_cpus;
@@ -143,10 +144,12 @@ int trace_dat__collect_cpu_event(int cpu, unsigned long long ts,
 }
 
 /* Write a single page of trace records */
-static int trace_dat__write_page(FILE *fp, unsigned long long base_ts,
+static int trace_dat__write_page(FILE *fp, struct tep_handle *pevent, unsigned long long base_ts,
 			char **records, int *rec_sizes, int nr_recs)
 {
 	unsigned long long commit = 0;
+	unsigned long long ts_out;
+	unsigned long long commit_out;
 	int offset = TRACE_DAT_RECORD_HEADER_SIZE;
 	int i;
 	char *page;
@@ -161,8 +164,12 @@ static int trace_dat__write_page(FILE *fp, unsigned long long base_ts,
 		commit += rec_sizes[i];
 	}
 
-	memcpy(page, &base_ts, sizeof(base_ts));
-	memcpy(page + sizeof(base_ts), &commit, sizeof(commit));
+	/* Byte-swap page header for cross-arch compatibility */
+	ts_out = to_file_u64(pevent, base_ts);
+	commit_out = to_file_u64(pevent, commit);
+
+	memcpy(page, &ts_out, sizeof(ts_out));
+	memcpy(page + sizeof(ts_out), &commit_out, sizeof(commit_out));
 
 	if (!fwrite(page, 1, trace_dat_page_size, fp)) {
 		free(page);
@@ -174,7 +181,8 @@ static int trace_dat__write_page(FILE *fp, unsigned long long base_ts,
 }
 
 /* Write all trace data for a single CPU as trace.dat flyrecord pages */
-static int trace_dat__write_cpu_dat(FILE *fp, int cpu, unsigned long long *file_offset_out)
+static int trace_dat__write_cpu_dat(FILE *fp, struct tep_handle *pevent,
+		int cpu, unsigned long long *file_offset_out)
 {
 	struct cpu_events *cpu_events = &trace_cpu_data[cpu];
 	unsigned long long base_ts;
@@ -229,11 +237,18 @@ static int trace_dat__write_cpu_dat(FILE *fp, int cpu, unsigned long long *file_
 			if (!extend)
 				return -ENOMEM;
 
-			extend_hdr =
-				((time_delta & TRACE_DAT_RECORD_TIME_MASK) <<
-				 TRACE_DAT_RECORD_TIME_SHIFT) |
-				TRACE_DAT_RECORD_TYPE_TIME_EXTEND;
-			delta_upper = time_delta >> TRACE_DAT_RECORD_TIME_SHIFT;
+			if (tep_is_file_bigendian(pevent)) {
+				extend_hdr = (time_delta & TRACE_DAT_RECORD_TIME_MASK) |
+				     (TRACE_DAT_RECORD_TYPE_TIME_EXTEND << 27);
+				delta_upper = time_delta >> 27;
+			} else {
+				extend_hdr = ((time_delta & TRACE_DAT_RECORD_TIME_MASK) <<
+					TRACE_DAT_RECORD_TIME_SHIFT) |
+					TRACE_DAT_RECORD_TYPE_TIME_EXTEND;
+				delta_upper = time_delta >> TRACE_DAT_RECORD_TIME_SHIFT;
+			}
+			extend_hdr  = to_file_u32(pevent, extend_hdr);   /* still needed */
+			delta_upper = to_file_u32(pevent, delta_upper);   /* still needed */
 
 			memcpy(extend, &extend_hdr, TRACE_DAT_WORD_SIZE);
 			memcpy(extend + TRACE_DAT_WORD_SIZE, &delta_upper,
@@ -273,7 +288,7 @@ static int trace_dat__write_cpu_dat(FILE *fp, int cpu, unsigned long long *file_
 		/* Check page fit BEFORE allocating data record */
 		if (page_size_used + needed_size >
 			trace_dat_page_size - TRACE_DAT_RECORD_HEADER_SIZE) {
-			ret = trace_dat__write_page(fp, base_ts,
+			ret = trace_dat__write_page(fp, pevent, base_ts,
 					page_records, page_rec_sizes,
 					nr_page_recs);
 
@@ -299,7 +314,13 @@ static int trace_dat__write_cpu_dat(FILE *fp, int cpu, unsigned long long *file_
 			time_delta = 0;
 		}
 
-		hdr_word = (time_delta << TRACE_DAT_RECORD_TIME_SHIFT) | type_len;
+		if (tep_is_file_bigendian(pevent))
+			hdr_word = ((unsigned int)time_delta & TRACE_DAT_RECORD_TIME_MASK) |
+				(type_len << 27);
+		else
+			hdr_word = ((unsigned int)time_delta << TRACE_DAT_RECORD_TIME_SHIFT) |
+				type_len;
+		hdr_word = to_file_u32(pevent, hdr_word);
 
 		data_rec = calloc(1, data_rec_size);
 		if (!data_rec) {
@@ -311,8 +332,11 @@ static int trace_dat__write_cpu_dat(FILE *fp, int cpu, unsigned long long *file_
 		memcpy(data_rec, &hdr_word, TRACE_DAT_WORD_SIZE);
 
 		/* Large events: write actual byte length after header */
-		if (type_len == 0)
-			memcpy(data_rec + TRACE_DAT_WORD_SIZE, &data_len, TRACE_DAT_WORD_SIZE);
+		if (type_len == 0) {
+			unsigned int data_len_out = to_file_u32(pevent, data_len);
+
+			memcpy(data_rec + TRACE_DAT_WORD_SIZE, &data_len_out, TRACE_DAT_WORD_SIZE);
+		}
 
 		memcpy(data_rec + payload_offset, event->raw, data_len);
 
@@ -366,7 +390,7 @@ static int trace_dat__write_cpu_dat(FILE *fp, int cpu, unsigned long long *file_
 	}
 
 	if (nr_page_recs > 0) {
-		ret = trace_dat__write_page(fp, base_ts,
+		ret = trace_dat__write_page(fp, pevent, base_ts,
 				page_records, page_rec_sizes, nr_page_recs);
 	}
 out_free:
@@ -378,11 +402,11 @@ static int trace_dat__write_cpu_dat(FILE *fp, int cpu, unsigned long long *file_
 }
 
 /* Write the strings section containing section name lookup table */
-int trace_dat__write_strings_section(void)
+int trace_dat__write_strings_section(struct tep_handle *pevent)
 {
-	unsigned short section_id = TRACE_DAT_SECTION_STRINGS;
-	unsigned short flags = 0;
 	unsigned long long section_size = 0;
+	unsigned short section_id = to_file_u16(pevent, TRACE_DAT_SECTION_STRINGS);
+	unsigned short flags = to_file_u16(pevent, 0);
 	static const char * const section_names[] = {
 		"headers",		/* offset 0 - strid for section 16  */
 		"ftrace event formats", /* offset 8 - strid for section 17  */
@@ -397,7 +421,7 @@ int trace_dat__write_strings_section(void)
 	};
 
 	/* string_id points to "strings" string itself */
-	unsigned int string_id = STRID_STRINGS;
+	unsigned int string_id = to_file_u32(pevent, STRID_STRINGS);
 	int i;
 
 	if (!trace_dat_fp)
@@ -405,47 +429,59 @@ int trace_dat__write_strings_section(void)
 
 	for (i = 0; section_names[i] != NULL; i++)
 		section_size += strlen(section_names[i]) + 1;
+	section_size = to_file_u64(pevent, section_size);
 
 	/* write section header */
 	if (!fwrite(&section_id, sizeof(unsigned short), 1, trace_dat_fp) ||
 		       !fwrite(&flags, sizeof(unsigned short), 1, trace_dat_fp) ||
 		       !fwrite(&string_id, sizeof(unsigned int), 1, trace_dat_fp) ||
-		       !fwrite(&section_size, sizeof(unsigned long long), 1, trace_dat_fp))
-		return -EIO;
+		       !fwrite(&section_size, sizeof(unsigned long long), 1, trace_dat_fp)) {
+		pr_warning("Failed to write strings section\n");
+		trace_dat_write_failed = true;
+	}
 
 	/* write strings */
 	for (i = 0; section_names[i] != NULL; i++)
-		if (!fwrite(section_names[i], 1, strlen(section_names[i]) + 1, trace_dat_fp))
-			return -EIO;
+		if (!fwrite(section_names[i], 1, strlen(section_names[i]) + 1, trace_dat_fp)) {
+			pr_warning("Failed to write strings section\n");
+			trace_dat_write_failed = true;
+		}
 	return 0;
 }
 
 /* Writes options section containing CPUCOUNT, TRACECLOCK, EVENT_FORMAT, HEADER_INFO,
  * FTRACE_EVENTS, KALLSYMS, CMDLINES options, ending with DONE option pointing to next section.
  */
-int trace_dat__write_options_section1(void)
+int trace_dat__write_options_section1(struct tep_handle *pevent)
 {
-	unsigned short section_id = TRACE_DAT_SECTION_OPTIONS;
-	unsigned short flags = 0;
-	unsigned int string_id = STRID_OPTIONS_1;
+	unsigned short section_id = to_file_u16(pevent, TRACE_DAT_SECTION_OPTIONS);
+	unsigned short flags = to_file_u16(pevent, 0);
+	unsigned int string_id = to_file_u32(pevent, STRID_OPTIONS_1);
 	unsigned long long section_size = 0;
 	long section_size_pos;
 	long payload_start;
 	unsigned long long section_start;
 	unsigned short opt_id;
 	unsigned int opt_size;
+	unsigned int raw_opt_size;
+	unsigned int nr_cpus;
 	char clock_buf[CLOCK_BUFFER_SIZE];
 	FILE *clock_file;
 	size_t bytes_read;
 	char *path;
 	unsigned long long next_offset;
+	unsigned long long events_off;
+	unsigned long long header_off;
+	unsigned long long ftrace_off;
+	unsigned long long kallsyms_off;
+	unsigned long long cmdline_off;
 	long end_pos;
 
 	if (!trace_dat_fp)
 		return -EBADF;
 
 	/* fill options_offset in initial format */
-	section_start = ftell(trace_dat_fp);
+	section_start = to_file_u64(pevent, ftell(trace_dat_fp));
 
 	if (fseek(trace_dat_fp, trace_dat_options_offset, SEEK_SET) < 0 ||
 	   !fwrite(&section_start, sizeof(unsigned long long), 1, trace_dat_fp) ||
@@ -464,16 +500,17 @@ int trace_dat__write_options_section1(void)
 	payload_start = ftell(trace_dat_fp);
 
 	/* CPUCOUNT option */
-	opt_id = TRACE_DAT_OPTION_CPUCOUNT;
-	opt_size = sizeof(unsigned int);
+	opt_id = to_file_u16(pevent, TRACE_DAT_OPTION_CPUCOUNT);
+	opt_size = to_file_u32(pevent, sizeof(unsigned int));
+	nr_cpus = to_file_u32(pevent, trace_dat_nr_cpus);
 
 	if (!fwrite(&opt_id, sizeof(unsigned short), 1, trace_dat_fp) ||
 	    !fwrite(&opt_size, sizeof(unsigned int), 1, trace_dat_fp) ||
-	    !fwrite(&trace_dat_nr_cpus, sizeof(unsigned int), 1, trace_dat_fp))
+	    !fwrite(&nr_cpus, sizeof(unsigned int), 1, trace_dat_fp))
 		return -EIO;
 
 	/* TRACECLOCK option */
-	opt_id = TRACE_DAT_OPTION_TRACECLOCK;
+	opt_id = to_file_u16(pevent, TRACE_DAT_OPTION_TRACECLOCK);
 
 	path = get_tracing_file("trace_clock");
 	if (path) {
@@ -490,66 +527,72 @@ int trace_dat__write_options_section1(void)
 		strcpy(clock_buf, "local\n");
 		bytes_read = strlen(clock_buf);
 	}
-	opt_size = bytes_read + 1;
+	raw_opt_size = bytes_read + 1;
+	opt_size = to_file_u32(pevent, bytes_read + 1);
 	if (!fwrite(&opt_id, sizeof(unsigned short), 1, trace_dat_fp) ||
 	    !fwrite(&opt_size, sizeof(unsigned int), 1, trace_dat_fp) ||
-	    !fwrite(clock_buf, 1, opt_size, trace_dat_fp))
+	    !fwrite(clock_buf, 1, raw_opt_size, trace_dat_fp))
 		return -EIO;
 
 	/* EVENT option */
-	opt_id = TRACE_DAT_OPTION_EVENT;
-	opt_size = sizeof(unsigned long long);
+	opt_id = to_file_u16(pevent, TRACE_DAT_OPTION_EVENT);
+	opt_size = to_file_u32(pevent, sizeof(unsigned long long));
+	events_off = to_file_u64(pevent, trace_dat_events_format_offset);
 
 	if (!fwrite(&opt_id, sizeof(unsigned short), 1, trace_dat_fp) ||
 	   !fwrite(&opt_size, sizeof(unsigned int), 1, trace_dat_fp) ||
-	   !fwrite(&trace_dat_events_format_offset, sizeof(unsigned long long),
+	   !fwrite(&events_off, sizeof(unsigned long long),
 		   1, trace_dat_fp))
 		return -EIO;
 
 	/* HEADER option */
-	opt_id = TRACE_DAT_OPTION_HEADER;
-	opt_size = sizeof(unsigned long long);
+	opt_id = to_file_u16(pevent, TRACE_DAT_OPTION_HEADER);
+	opt_size = to_file_u32(pevent, sizeof(unsigned long long));
+	header_off = to_file_u64(pevent, trace_dat_header_info_offset);
 
 	if (!fwrite(&opt_id, sizeof(unsigned short), 1, trace_dat_fp) ||
 	    !fwrite(&opt_size, sizeof(unsigned int), 1, trace_dat_fp) ||
-	    !fwrite(&trace_dat_header_info_offset, sizeof(unsigned long long),
+	    !fwrite(&header_off, sizeof(unsigned long long),
 		    1, trace_dat_fp))
 		return -EIO;
 
 	/* FTRACE option */
-	opt_id = TRACE_DAT_OPTION_FTRACE;
-	opt_size = sizeof(unsigned long long);
+	opt_id = to_file_u16(pevent, TRACE_DAT_OPTION_FTRACE);
+	opt_size = to_file_u32(pevent, sizeof(unsigned long long));
+	ftrace_off = to_file_u64(pevent, trace_dat_ftrace_format_offset);
 
 	if (!fwrite(&opt_id, sizeof(unsigned short), 1, trace_dat_fp) ||
 	    !fwrite(&opt_size, sizeof(unsigned int), 1, trace_dat_fp) ||
-	    !fwrite(&trace_dat_ftrace_format_offset, sizeof(unsigned long long),
+	    !fwrite(&ftrace_off, sizeof(unsigned long long),
 		   1, trace_dat_fp))
 		return -EIO;
 
 	/* KALLSYMS option */
-	opt_id = TRACE_DAT_OPTION_KALLSYMS;
-	opt_size = sizeof(unsigned long long);
+	opt_id = to_file_u16(pevent, TRACE_DAT_OPTION_KALLSYMS);
+	opt_size = to_file_u32(pevent, sizeof(unsigned long long));
+	kallsyms_off = to_file_u64(pevent, trace_dat_kallsyms_offset);
 
 	if (!fwrite(&opt_id, sizeof(unsigned short), 1, trace_dat_fp) ||
 	    !fwrite(&opt_size, sizeof(unsigned int), 1, trace_dat_fp) ||
-	    !fwrite(&trace_dat_kallsyms_offset, sizeof(unsigned long long),
+	    !fwrite(&kallsyms_off, sizeof(unsigned long long),
 		    1, trace_dat_fp))
 		return -EIO;
 
 	/* CMDLINE option */
-	opt_id = TRACE_DAT_OPTION_CMDLINE;
-	opt_size = sizeof(unsigned long long);
+	opt_id = to_file_u16(pevent, TRACE_DAT_OPTION_CMDLINE);
+	opt_size = to_file_u32(pevent, sizeof(unsigned long long));
+	cmdline_off = to_file_u64(pevent, trace_dat_cmdline_offset);
 
 	if (!fwrite(&opt_id, sizeof(unsigned short), 1, trace_dat_fp) ||
 	    !fwrite(&opt_size, sizeof(unsigned int), 1, trace_dat_fp) ||
-	    !fwrite(&trace_dat_cmdline_offset, sizeof(unsigned long long),
+	    !fwrite(&cmdline_off, sizeof(unsigned long long),
 		    1, trace_dat_fp))
 		return -EIO;
 
 	/* DONE option id - next_options_offset filled later */
-	opt_id = TRACE_DAT_OPTION_DONE;
-	opt_size = sizeof(unsigned long long);
-	next_offset = 0;  /* placeholder */
+	opt_id = to_file_u16(pevent, TRACE_DAT_OPTION_DONE);
+	opt_size = to_file_u32(pevent, sizeof(unsigned long long));
+	next_offset = to_file_u64(pevent, 0);
 
 	trace_dat_next_options_offset = ftell(trace_dat_fp);
 	if (!fwrite(&opt_id, sizeof(unsigned short), 1, trace_dat_fp) ||
@@ -560,7 +603,7 @@ int trace_dat__write_options_section1(void)
 	/* fill section size */
 	end_pos = ftell(trace_dat_fp);
 
-	section_size = end_pos - payload_start;
+	section_size = to_file_u64(pevent, end_pos - payload_start);
 	if (fseek(trace_dat_fp, section_size_pos, SEEK_SET) < 0 ||
 	    !fwrite(&section_size, sizeof(unsigned long long), 1, trace_dat_fp) ||
 	    fseek(trace_dat_fp, end_pos, SEEK_SET) < 0)
@@ -574,20 +617,21 @@ int trace_dat__write_options_section1(void)
  * (flyrecord section offset, clock type, page size, CPU count,
  * per-CPU offsets/sizes) and DONE option.
  */
-int trace_dat__write_options_section2(void)
+int trace_dat__write_options_section2(struct tep_handle *pevent)
 {
-	unsigned short section_id  = TRACE_DAT_SECTION_OPTIONS;
-	unsigned short flags = 0;
-	unsigned int string_id = STRID_OPTIONS_2;
+	unsigned short section_id = to_file_u16(pevent, TRACE_DAT_SECTION_OPTIONS);
+	unsigned short flags = to_file_u16(pevent, 0);
+	unsigned int string_id = to_file_u32(pevent, STRID_OPTIONS_2);
 	unsigned long long section_size = 0;
 	long section_size_pos;
 	long payload_start;
 	int cpu;
-	unsigned short opt_id = TRACE_DAT_OPTION_BUFFER;
-	unsigned int opt_size = 0;
+	unsigned short opt_id = to_file_u16(pevent, TRACE_DAT_OPTION_BUFFER);
+	unsigned int opt_size = 0;  /* filled after payload written */
 	long opt_size_pos;
 	unsigned long long data_offset = 0;
-	unsigned int page_size = (unsigned int)trace_dat_page_size;
+	unsigned int page_size = to_file_u32(pevent, (unsigned int)trace_dat_page_size);
+	unsigned int nr_cpus   = to_file_u32(pevent, trace_dat_nr_cpus);
 	const char *clock = "local";
 	unsigned long long next;
 	long end_pos;
@@ -600,7 +644,7 @@ int trace_dat__write_options_section2(void)
 		return -EINVAL;
 
 	/* fill done1 next offset - points to this section */
-	next = ftell(trace_dat_fp);
+	next = to_file_u64(pevent, ftell(trace_dat_fp));
 
 	if (fseek(trace_dat_fp, trace_dat_next_options_offset + 2 + 4, SEEK_SET) < 0 ||
 	    !fwrite(&next, sizeof(unsigned long long), 1, trace_dat_fp) ||
@@ -631,15 +675,16 @@ int trace_dat__write_options_section2(void)
 	    !fwrite("\0", 1, 1, trace_dat_fp) ||
 	    !fwrite(clock, 1, strlen(clock) + 1, trace_dat_fp) ||
 	    !fwrite(&page_size, sizeof(unsigned int), 1, trace_dat_fp) ||
-	    !fwrite(&trace_dat_nr_cpus, sizeof(unsigned int), 1, trace_dat_fp))
+	    !fwrite(&nr_cpus, sizeof(unsigned int), 1, trace_dat_fp))
 		return -EIO;
 
 	/* per cpu: cpu_id + offset placeholder + size */
 	for (cpu = 0; cpu < trace_dat_nr_cpus; cpu++) {
+		unsigned int cpu_out          = to_file_u32(pevent, cpu);
 		cpu_offset = 0;  /* filled in write_flyrecord */
-		cpu_size   = 0;  /* filled in write_flyrecord */
+		cpu_size = 0;  /* filled in write_flyrecord */
 
-		if (!fwrite(&cpu, sizeof(unsigned int), 1, trace_dat_fp))
+		if (!fwrite(&cpu_out, sizeof(unsigned int), 1, trace_dat_fp))
 			return -EIO;
 		buffer_opt_cpu_offsets_pos[cpu] = ftell(trace_dat_fp);
 		if (!fwrite(&cpu_offset, sizeof(unsigned long long), 1, trace_dat_fp) ||
@@ -650,17 +695,17 @@ int trace_dat__write_options_section2(void)
 	/* fill opt_size */
 	end_pos = ftell(trace_dat_fp);
 
-	opt_size = end_pos - opt_payload_start;
-	fseek(trace_dat_fp, opt_size_pos, SEEK_SET);
-	if (!fwrite(&opt_size, sizeof(unsigned int), 1, trace_dat_fp))
+	opt_size = to_file_u32(pevent, end_pos - opt_payload_start);
+	if (fseek(trace_dat_fp, opt_size_pos, SEEK_SET) < 0 ||
+	    !fwrite(&opt_size, sizeof(unsigned int), 1, trace_dat_fp) ||
+	    fseek(trace_dat_fp, end_pos, SEEK_SET) < 0)
 		return -EIO;
-	fseek(trace_dat_fp, end_pos, SEEK_SET);
 
 	/* DONE id=0 */
-	done_id = TRACE_DAT_OPTION_DONE;
-	done_size = sizeof(unsigned long long);
+	done_id = to_file_u16(pevent, TRACE_DAT_OPTION_DONE);
+	done_size = to_file_u32(pevent, sizeof(unsigned long long));
 	/* No additional options sections follow this one */
-	next = 0;
+	next = to_file_u64(pevent, 0);
 
 	if (!fwrite(&done_id, sizeof(unsigned short), 1, trace_dat_fp) ||
 	    !fwrite(&done_size, sizeof(unsigned int), 1, trace_dat_fp) ||
@@ -670,24 +715,25 @@ int trace_dat__write_options_section2(void)
 	/* fill section size */
 	end_pos = ftell(trace_dat_fp);
 
-	section_size = end_pos - payload_start;
-	fseek(trace_dat_fp, section_size_pos, SEEK_SET);
-	if (!fwrite(&section_size, sizeof(unsigned long long), 1, trace_dat_fp))
+	section_size = to_file_u64(pevent, end_pos - payload_start);
+	if (fseek(trace_dat_fp, section_size_pos, SEEK_SET) < 0 ||
+	    !fwrite(&section_size, sizeof(unsigned long long), 1, trace_dat_fp) ||
+	    fseek(trace_dat_fp, end_pos, SEEK_SET) < 0)
 		return -EIO;
-	fseek(trace_dat_fp, end_pos, SEEK_SET);
 
 	return 0;
 
 }
 
-int trace_dat__write_flyrecord_section(void)
+int trace_dat__write_flyrecord_section(struct tep_handle *pevent)
 {
-	unsigned short section_id = TRACE_DAT_SECTION_FLYRECORD;
-	unsigned short flags = 0;
-	unsigned int string_id = STRID_BUFFER_FLYRECORD;
+	unsigned short section_id = to_file_u16(pevent, TRACE_DAT_SECTION_FLYRECORD);
+	unsigned short flags = to_file_u16(pevent, 0);
+	unsigned int string_id = to_file_u32(pevent, STRID_BUFFER_FLYRECORD);
 	unsigned long long section_size = 0;
 	long section_size_pos;
 	long flyrecord_start;
+	unsigned long long flyrecord_out;
 	long after_header;
 	long padding_needed;
 	unsigned long long *cpu_offsets;
@@ -750,7 +796,7 @@ int trace_dat__write_flyrecord_section(void)
 	for (cpu = 0; cpu < trace_dat_nr_cpus; cpu++) {
 		start = ftell(trace_dat_fp);
 
-		ret = trace_dat__write_cpu_dat(trace_dat_fp, cpu, &cpu_offsets[cpu]);
+		ret = trace_dat__write_cpu_dat(trace_dat_fp, pevent, cpu, &cpu_offsets[cpu]);
 
 		if (ret < 0) {
 			pr_err("Failed to write CPU %d data\n", cpu);
@@ -762,7 +808,7 @@ int trace_dat__write_flyrecord_section(void)
 	/* fill section size */
 	end_pos = ftell(trace_dat_fp);
 
-	section_size = end_pos - after_header;
+	section_size = to_file_u64(pevent, end_pos - after_header);
 	if (fseek(trace_dat_fp, section_size_pos, SEEK_SET) < 0 ||
 	    !fwrite(&section_size, sizeof(unsigned long long), 1, trace_dat_fp)) {
 		ret = -EIO;
@@ -775,6 +821,8 @@ int trace_dat__write_flyrecord_section(void)
 
 	/* fill cpu offsets and sizes in BUFFER option */
 	for (cpu = 0; cpu < trace_dat_nr_cpus; cpu++) {
+		cpu_offsets[cpu] = to_file_u64(pevent, cpu_offsets[cpu]);
+		cpu_sizes[cpu]   = to_file_u64(pevent, cpu_sizes[cpu]);
 		if (fseek(trace_dat_fp, buffer_opt_cpu_offsets_pos[cpu], SEEK_SET) < 0 ||
 		    !fwrite(&cpu_offsets[cpu], sizeof(unsigned long long), 1, trace_dat_fp) ||
 		    !fwrite(&cpu_sizes[cpu], sizeof(unsigned long long), 1, trace_dat_fp)) {
@@ -784,8 +832,9 @@ int trace_dat__write_flyrecord_section(void)
 	}
 
 	/* fill data offset in buffer option */
+	flyrecord_out = to_file_u64(pevent, flyrecord_start);
 	if (fseek(trace_dat_fp, opt_payload_start, SEEK_SET) < 0 ||
-	    !fwrite(&flyrecord_start, sizeof(unsigned long long), 1, trace_dat_fp)) {
+	    !fwrite(&flyrecord_out, sizeof(unsigned long long), 1, trace_dat_fp)) {
 		ret = -EIO;
 		goto cleanup;
 	}
@@ -795,7 +844,6 @@ int trace_dat__write_flyrecord_section(void)
 		goto cleanup;
 	}
 
-
 cleanup:
 	free(cpu_offsets);
 	free(cpu_sizes);
tools/perf/util/trace-dat.h +35 -5
index 9aec37b708d4..5985083b275a 100644
@@ -8,9 +8,13 @@
 #define __PERF_TRACE_DAT_H
 
 #include <stdio.h>
+#include <stdbool.h>
+#include <event-parse.h>
+#include <byteswap.h>
+#include "util.h"
 
 /* trace.dat file format version */
-#define TRACE_DAT_VERSION '7'
+#define TRACE_DAT_VERSION "7"
 
 /*
  * Section IDs for trace.dat format
@@ -51,6 +55,32 @@
 #define STRID_OPTIONS_2        77
 #define STRID_BUFFER_FLYRECORD 85
 
+/*
+ * Set by trace.dat section writers (kallsyms, ftrace, events etc.) on
+ * fwrite() failure; checked by trace_convert__perf2dat() to clean up.
+ */
+extern bool trace_dat_write_failed;
+
+/*
+ * tep handle from the recording machine; set by trace_convert__perf2dat()
+ * before options/flyrecord sections are written.
+ */
+
+static inline uint16_t to_file_u16(struct tep_handle *pevent, uint16_t val)
+{
+	return tep_read_number(pevent, &val, 2);
+}
+
+static inline uint32_t to_file_u32(struct tep_handle *pevent, uint32_t val)
+{
+	return tep_read_number(pevent, &val, 4);
+}
+
+static inline uint64_t to_file_u64(struct tep_handle *pevent, uint64_t val)
+{
+	return tep_read_number(pevent, &val, 8);
+}
+
 struct perf_session;
 
 extern FILE *trace_dat_fp;
@@ -75,9 +105,9 @@ int trace_dat__collect_cpu_event(int cpu, unsigned long long ts,
 void trace_dat__free_cpu_buffers(void);
 
 /* write trace.dat file sections */
-int trace_dat__write_options_section1(void);
-int trace_dat__write_options_section2(void);
-int trace_dat__write_flyrecord_section(void);
-int trace_dat__write_strings_section(void);
+int trace_dat__write_options_section1(struct tep_handle *pevent);
+int trace_dat__write_options_section2(struct tep_handle *pevent);
+int trace_dat__write_flyrecord_section(struct tep_handle *pevent);
+int trace_dat__write_strings_section(struct tep_handle *pevent);
 
 #endif /* __PERF_TRACE_DAT_H */
tools/perf/util/trace-event-read.c +300 -7

448 lines not on this page · raw message

Signature · 1 line
2.47.1
#4 lore raw

Re: [RFC PATCH v4 2/5] perf/trace-event: Write trace.dat metadata sections during parsing

Thank you for your contribution! Sashiko AI review found 5 potential issue(s) to consider:

Thank you for your contribution! Sashiko AI review found 5 potential issue(s) to consider:

New issues:
- [Critical] Missing bounds check on raw_size leads to massive out-of-bounds read and heap buffer overflow.
- [High] Hardcoded 16-byte page header breaks parsing of trace data recorded on 32-bit architectures.
- [Medium] Host state leakage / Cross-platform analysis violation when embedding the TRACECLOCK option.
- [Medium] Signed integer overflow causing undefined behavior in TIME_EXTEND bitshift.

Pre-existing issues:
- [Critical] Integer overflow in read_saved_cmdline size calculation leads to a heap buffer overflow on 32-bit systems. --

--- Patch [2]: [RFC PATCH v4 2/5] perf/trace-event: Write trace.dat metadata sections during parsing --- commit 75d79e7a5c5c4a1484c389705c19fc30dbdd1deb
Author: Tanushree Shah <[email protected]>

perf/trace-event: Write trace.dat metadata sections during parsing

Perf already captures the tracing metadata as a part of data section in perf.data

When trace_dat_fp is set, write trace.dat compatible metadata sections using the perf provided raw buffers.

Quoted 3 lines
tools/perf/util/trace-dat.c +0 -0

[ ... ]

tools/perf/util/trace-dat.c static int trace_dat__write_page(FILE *fp, unsigned long long base_ts, · line 172

Quoted 12 lines
tools/perf/util/trace-dat.c +6 -2
@@ -161,8 +164,12 @@ static int trace_dat__write_page(FILE *fp, unsigned long long base_ts,
 		commit += rec_sizes[i];
 	}
 
-	memcpy(page, &base_ts, sizeof(base_ts));
-	memcpy(page + sizeof(base_ts), &commit, sizeof(commit));
+	/* Byte-swap page header for cross-arch compatibility */
+	ts_out = to_file_u64(pevent, base_ts);
+	commit_out = to_file_u64(pevent, commit);
+
+	memcpy(page, &ts_out, sizeof(ts_out));
+	memcpy(page + sizeof(ts_out), &commit_out, sizeof(commit_out));

[Severity: High]
Does this hardcode a 16-byte page header regardless of the original architecture's commit size?

If the original trace was recorded on a 32-bit machine, its header_page metadata usually specifies the commit field as a 4-byte local_t. Writing a hardcoded 8-byte commit field here might cause parsers to misalign by 4 bytes, potentially interpreting the upper half of the 8-byte commit as the first event record's header and breaking parsing.

[ ... ]

tools/perf/util/trace-dat.c static int trace_dat__write_cpu_dat(FILE *fp, int cpu, unsigned long long *file_ · line 243

Quoted 13 lines
tools/perf/util/trace-dat.c +4 -5
@@ -229,11 +237,18 @@ static int trace_dat__write_cpu_dat(FILE *fp, int cpu, unsigned long long *file_
 			if (!extend)
 				return -ENOMEM;
 
-			extend_hdr =
-				((time_delta & TRACE_DAT_RECORD_TIME_MASK) <<
-				 TRACE_DAT_RECORD_TIME_SHIFT) |
-				TRACE_DAT_RECORD_TYPE_TIME_EXTEND;
-			delta_upper = time_delta >> TRACE_DAT_RECORD_TIME_SHIFT;
+			if (tep_is_file_bigendian(pevent)) {
+				extend_hdr = (time_delta & TRACE_DAT_RECORD_TIME_MASK) |
+				     (TRACE_DAT_RECORD_TYPE_TIME_EXTEND << 27);
+				delta_upper = time_delta >> 27;

[Severity: Medium]
Could this left shift result in undefined behavior?

Assuming TRACE_DAT_RECORD_TYPE_TIME_EXTEND is defined as 30 (a 32-bit signed integer literal), shifting it left by 27 bits yields 4026531840. This exceeds the maximum representable value of a 32-bit signed integer, which can cause undefined behavior in C.

[ ... ]

tools/perf/util/trace-dat.c static int trace_dat__write_cpu_dat(FILE *fp, int cpu, unsigned long long *file_ · line 341

Quoted 13 lines
tools/perf/util/trace-dat.c +5 -2
@@ -311,8 +332,11 @@ static int trace_dat__write_cpu_dat(FILE *fp, int cpu, unsigned long long *file_
 		memcpy(data_rec, &hdr_word, TRACE_DAT_WORD_SIZE);
 
 		/* Large events: write actual byte length after header */
-		if (type_len == 0)
-			memcpy(data_rec + TRACE_DAT_WORD_SIZE, &data_len, TRACE_DAT_WORD_SIZE);
+		if (type_len == 0) {
+			unsigned int data_len_out = to_file_u32(pevent, data_len);
+
+			memcpy(data_rec + TRACE_DAT_WORD_SIZE, &data_len_out, TRACE_DAT_WORD_SIZE);
+		}
 
 		memcpy(data_rec + payload_offset, event->raw, data_len);

[Severity: Critical]
Is it possible for a crafted perf.data file to cause an out-of-bounds read and heap buffer overflow here?

If the original sample's raw_size is maliciously set near UINT_MAX (e.g., 0xFFFFFFFF), the data_rec_size calculation in trace_dat__write_cpu_dat() calculates data_rec_size = 8 + data_len, which wraps around to 7. This bypasses the page size check and allocates an 8-byte heap buffer. The memcpy() here then attempts to copy gigabytes of data into that small buffer.

Similarly, earlier in trace_dat__collect_cpu_event(), malloc(0xFFFFFFFF) could succeed on a 64-bit host, causing the subsequent memcpy() to read gigabytes out-of-bounds.

[ ... ]

Quoted 4 lines
tools/perf/util/trace-dat.c +0 -0
@@ -464,16 +500,17 @@ int trace_dat__write_options_section1(void)
 	payload_start = ftell(trace_dat_fp);
 
 	/* CPUCOUNT option */

[ ... ]

tools/perf/util/trace-dat.c int trace_dat__write_options_section1(void)

Quoted 7 lines
tools/perf/util/trace-dat.c +1 -1
 	/* TRACECLOCK option */
-	opt_id = TRACE_DAT_OPTION_TRACECLOCK;
+	opt_id = to_file_u16(pevent, TRACE_DAT_OPTION_TRACECLOCK);
 
 	path = get_tracing_file("trace_clock");
 	if (path) {
 		clock_file = fopen(path, "r");

[Severity: Medium]
Does this inadvertently leak the conversion host's state into the converted trace data?

Since this dynamically queries the local host's trace_clock via get_tracing_file(), it embeds the local machine's unparsed clock file contents (such as "[local] global...") instead of preserving the original recorded machine's clock metadata, potentially breaking cross-platform reproducibility.

[ ... ]

tools/perf/util/trace-event-read.c static int read_saved_cmdline(struct tep_handle *pevent) · line 621

Quoted 11 lines
tools/perf/util/trace-event-read.c +4 -0
@@ -383,6 +615,27 @@ static int read_saved_cmdline(struct tep_handle *pevent)
 		pr_debug("error reading saved cmdlines\n");
 		goto out;
 	}
+	/* Write cmdlines section with data */
+	if (trace_dat_fp && !trace_dat_write_failed) {
+		unsigned short section_id = to_file_u16(pevent, TRACE_DAT_SECTION_CMDLINE);
+		unsigned short flags = to_file_u16(pevent, 0);

[Severity: Critical]
This is a pre-existing issue, but does the size calculation for the saved command line buffer overflow on 32-bit systems?

Looking at read_saved_cmdline(), when size is read as a 64-bit value, if it is 0xFFFFFFFF, the allocation calculation of size + 1 overflows to 0x100000000.

On 32-bit architectures, this is truncated to 0, causing malloc(0) to return a small valid pointer. The subsequent do_read(buf, size) then passes 0xFFFFFFFF to the read() syscall, which can overwrite the heap and potentially cause a denial of service or arbitrary code execution.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
#5 lore raw

RFC PATCH v4 3/5 perf data-convert: Add perf.data to trace.dat conversion backend

In reply to Tanushree Shah

Add data-convert-trace.c implementing trace_convert__perf2dat() to convert perf.data tracepoint events to trace.dat format.

process_sample_event() is invoked for each PERF_TYPE_TRACEPOINT sample during perf_session__process_events(), storing raw event bytes per-cpu via trace_dat__collect_cpu_event().

Once all samples are collected:
- trace_dat__write_options_section1() writes the OPTIONS section with CPUCOUNT, TRACECLOCK, HEADER_INFO, FTRACE_EVENTS, EVENT_FORMATS, KALLSYMS, CMDLINES and DONE options.
- trace_dat__write_options_section2() writes the OPTIONS section with BUFFER option holding per-cpu data offset placeholders and the DONE option.
- trace_dat__write_flyrecord_section() builds ring buffer pages per-cpu and patches BUFFER option with final offsets and sizes,

Per-cpu buffers are sized to tep_get_page_size() from the session tep handle and released on all exit paths.

Register .attr, .feature, and .tracing_data callbacks on the tool, delegating to the standard perf_event__process_attr(), perf_event__process_feature(), and perf_event__process_tracing_data() helpers, so these records are processed as they arrive in the stream instead of being left as default no-op stubs.

Defer per-CPU buffer initialisation to the first call of the .sample callback instead of doing it eagerly after session open. By this point, pipe-mode ordering guarantees (record__synthesize()) that attr, feature, and tracing_data records have already been written and processed, so nr_cpus_online and the tep page size are guaranteed to be populated from the recording machine's own stream data. This is required because the converting host may differ in architecture, CPU count, or page size from the machine that recorded the data, so these values must never be taken from local sysconf() or similar.

The output path honours --force: without it, the destination file is opened with O_CREAT|O_EXCL so an existing file is detected atomically and conversion fails with -EEXIST rather than silently overwriting prior output.

---
 tools/perf/util/Build                |   1 +
 tools/perf/util/data-convert-trace.c | 243 +++++++++++++++++++++++++++
 tools/perf/util/data-convert.h       |   4 +
 3 files changed, 248 insertions(+)
 create mode 100644 tools/perf/util/data-convert-trace.c
tools/perf/util/Build +1 -0
index 12f5bbe32770..40db8e3c01c4 100644
@@ -237,6 +237,7 @@ ifeq ($(CONFIG_LIBTRACEEVENT),y)
 endif
 
 perf-util-y += data-convert-json.o
+perf-util-$(CONFIG_LIBTRACEEVENT) += data-convert-trace.o
 
 perf-util-y += scripting-engines/
 
tools/perf/util/data-convert-trace.c new file +243 -0

243 lines not on this page · raw message

tools/perf/util/data-convert.h +4 -0
index a96240f15671..f041c2325226 100644
@@ -19,4 +19,8 @@ int bt_convert__perf2ctf(const char *input_name, const char *to_ctf,
 int bt_convert__perf2json(const char *input_name, const char *to_ctf,
 			 struct perf_data_convert_opts *opts);
 
+#ifdef HAVE_LIBTRACEEVENT
+int trace_convert__perf2dat(const char *input, const char *to_trace,
+			   struct perf_data_convert_opts *opts);
+#endif /* HAVE_LIBTRACEEVENT */
 #endif /* __DATA_CONVERT_H */
Signature · 1 line
2.47.1
#6 lore raw

Re: [RFC PATCH v4 3/5] perf data-convert: Add perf.data to trace.dat conversion backend

Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider: - [High] Arbitrary file truncation and TOCTOU vulnerability via symlink in the `--force` path. - [Medium]…

Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:
- [High] Arbitrary file truncation and TOCTOU vulnerability via symlink in the --force path.
- [Medium] Directly accessing perf_env field without verifying initialization.
- [Low] Missing feature fallback stub for trace_convert__perf2dat in data-convert.h.
- [Low] Lost error code when fopen() fails in the --force path.
--

--- Patch [3]: [RFC PATCH v4 3/5] perf data-convert: Add perf.data to trace.dat conversion backend --- commit aaf748bbf23e74fad4be33c3834c0413e8def0f8
Author: Tanushree Shah <[email protected]>

perf data-convert: Add perf.data to trace.dat conversion backend

This commit adds a new conversion backend to perf to translate tracepoint events from perf.data into trace.dat format. It leverages a new per-CPU buffering mechanism and implements required callbacks for pipe mode.

Quoted 5 lines
tools/perf/util/data-convert-trace.c new file +0 -0
new file mode 100644
index 0000000000000..445479fae8884

[ ... ]

tools/perf/util/data-convert-trace.c line 60

Quoted 6 lines
tools/perf/util/data-convert-trace.c new file +6 -0
+	/*
+	 * In pipe mode, CPU count and page size arrive via feature/tracing_data
+	 * records before the first sample; initialize buffers lazily on first sample.
+	 */
+	if (!cpu_buffers_initialized) {
+		int nr_cpus = trace_dat_session->header.env.nr_cpus_online;

[Severity: Medium]
Is it safe to access nr_cpus_online directly here without first verifying the environment is initialized?

The perf subsystem guidelines suggest that accessing perf_env fields without first verifying those fields are populated can lead to bugs.

[ ... ]

Quoted 2 lines
tools/perf/util/data-convert-trace.c new file +2 -0
+	if (!opts->force) {
+		int fd = open(to_trace, O_WRONLY | O_CREAT | O_EXCL, 0644);

[ ... ]

tools/perf/util/data-convert-trace.c line 170

Quoted 2 lines
tools/perf/util/data-convert-trace.c new file +2 -0
+	} else {
+		trace_dat_fp = fopen(to_trace, "wb");

[Severity: High]
Could this fopen() in the --force path be susceptible to an arbitrary file truncation symlink attack?

If an attacker creates a symlink at the destination path, fopen() with "wb" will follow it and truncate the target file. If the conversion subsequently fails, the cleanup routine calls unlink(to_trace), which removes the symlink itself. This leaves the target file truncated while hiding the attacker's symlink from discovery.

tools/perf/util/data-convert-trace.c line 173

Quoted 3 lines
tools/perf/util/data-convert-trace.c new file +3 -0
+		if (!trace_dat_fp) {
+			pr_err("Failed to open output file: %s\n", to_trace);
+			return -EINVAL;

[Severity: Low]
Should this return the actual -errno from fopen() instead of a hardcoded -EINVAL so the exact filesystem error is not lost?

Quoted 2 lines
tools/perf/util/data-convert-trace.c new file +2 -0
+		}
+	}

[ ... ]

Quoted 4 lines
tools/perf/util/data-convert.h +0 -0
index a96240f156712..f041c23252263 100644

[ ... ]

tools/perf/util/data-convert.h int bt_convert__perf2ctf(const char *input_name, const char *to_ctf, · line 25

Quoted 4 lines
tools/perf/util/data-convert.h +4 -0
+#ifdef HAVE_LIBTRACEEVENT
+int trace_convert__perf2dat(const char *input, const char *to_trace,
+			   struct perf_data_convert_opts *opts);
+#endif /* HAVE_LIBTRACEEVENT */

[Severity: Low]
Does this declaration need an #else block with a compatible dummy inline stub?

Without a fallback stub when HAVE_LIBTRACEEVENT is absent, callers might be forced to clutter their code with preprocessor blocks to prevent build failures.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
#7 lore raw

RFC PATCH v4 4/5 perf data: Add --to-trace-dat option for converting perf.data tracepoint events into trace.dat format

In reply to Tanushree Shah

Add new command-line option to perf data convert for generating trace.dat output files.

The --to-trace-dat option:
- Accepts output filename for trace.dat format
- Mutually exclusive with --to-ctf and --to-json
- Calls trace_convert__perf2dat() to perform conversion

Usage:

  $ perf record -e sched:* -a sleep 1
  $ perf data convert --to-trace-dat=trace.dat
  $ trace-cmd report trace.dat

Document the new option in Documentation/perf-data.txt alongside the existing --to-ctf and --to-json entries.

Honor the --time filtering option in trace_convert__perf2dat() the same way the JSON and CTF converters do, using perf_time__parse_for_ranges() and perf_time__ranges_skip_sample() to skip samples outside the requested time range.

---
 tools/perf/Documentation/perf-data.txt |  7 +++++
 tools/perf/builtin-data.c              | 43 ++++++++++++++++++++++++++
 tools/perf/util/data-convert-trace.c   | 21 +++++++++++++
 tools/perf/util/trace-dat.c            | 20 +++++++++---
 tools/perf/util/trace-dat.h            |  1 +
 5 files changed, 87 insertions(+), 5 deletions(-)
tools/perf/Documentation/perf-data.txt +7 -0
index 20f178d61ed7..578ba6357183 100644
@@ -30,6 +30,13 @@ OPTIONS for 'convert'
 --to-json::
 	Triggers JSON conversion. Specify the JSON filename to output.
 
+--to-trace-dat::
+	Triggers trace.dat conversion. Converts perf.data tracepoint events
+	to trace.dat format v7, compatible with trace-cmd and KernelShark.
+	Only PERF_TYPE_TRACEPOINT events are converted. Specify the
+	trace.dat filename to output. Requires libtraceevent support.
+	Mutually exclusive with --to-ctf and --to-json.
+
 --tod::
 	Convert time to wall clock time.
 
tools/perf/builtin-data.c +43 -0

65 lines not on this page · raw message

tools/perf/util/data-convert-trace.c +21 -0
index 445479fae888..425b3aaa3f02 100644
@@ -22,6 +22,7 @@
 #include "evsel.h"
 #include "tool.h"
 #include "debug.h"
+#include "time-utils.h"
 #include "trace-dat.h"
 #include "trace-event.h"
 #include "event.h"
@@ -31,6 +32,10 @@
 struct trace_convert {
 	struct perf_tool tool;
 	u64 events_count;
+	struct perf_time_interval *ptime_range;
+	int range_size;
+	int range_num;
+	u64 skipped;
 };
 
 /* Session handle and init flag used for lazy CPU buffer init in pipe mode */
@@ -86,6 +91,11 @@ static int process_sample_event(const struct perf_tool *tool,
 		cpu_buffers_initialized = true;
 	}
 
+	if (perf_time__ranges_skip_sample(tc->ptime_range, tc->range_num, sample->time)) {
+		tc->skipped++;
+		return 0;
+	}
+
 	if (trace_dat__collect_cpu_event(sample->cpu, sample->time,
 				sample->raw_data, sample->raw_size) < 0) {
 		pr_err("Failed to collect CPU event\n");
@@ -185,6 +195,15 @@ int trace_convert__perf2dat(const char *input, const char *to_trace,
 	/* Stash session for lazy CPU buffer init on first sample (pipe and normal mode) */
 	trace_dat_session = session;
 
+	if (opts->time_str) {
+		ret = perf_time__parse_for_ranges(opts->time_str, session,
+				&tc.ptime_range,
+				&tc.range_size,
+				&tc.range_num);
+		if (ret < 0)
+			goto out_delete;
+	}
+
 	/* Process all events - collects raw data per-cpu */
 	ret = perf_session__process_events(session);
 	if (ret < 0) {
@@ -230,6 +249,8 @@ int trace_convert__perf2dat(const char *input, const char *to_trace,
 out_delete:
 	if (cpu_buffers_initialized)
 		trace_dat__free_cpu_buffers();
+	if (tc.ptime_range)
+		zfree(&tc.ptime_range);
 	perf_session__delete(session);
 	trace_dat_session = NULL;
 out_close:
tools/perf/util/trace-dat.c +15 -5
index f71e03716e27..1b236745bcac 100644
@@ -194,6 +194,7 @@ static int trace_dat__write_cpu_dat(FILE *fp, struct tep_handle *pevent,
 	int page_size_used = 0;
 	int ret = 0;
 	int i, j;
+	unsigned long long page_base_ts;
 
 	file_offset = ftell(fp);
 	*file_offset_out = file_offset;
@@ -212,6 +213,7 @@ static int trace_dat__write_cpu_dat(FILE *fp, struct tep_handle *pevent,
 	}
 
 	base_ts = cpu_events->events[0].ts;
+	page_base_ts = base_ts;
 
 	for (i = 0; i < cpu_events->count; i++) {
 		struct cpu_event *event = &cpu_events->events[i];
@@ -234,8 +236,10 @@ static int trace_dat__write_cpu_dat(FILE *fp, struct tep_handle *pevent,
 
 			extend_size = TRACE_DAT_RECORD_TIME_EXTEND_SIZE;
 			extend = calloc(1, extend_size);
-			if (!extend)
-				return -ENOMEM;
+			if (!extend) {
+				ret = -ENOMEM;
+				goto out_free;
+			}
 
 			if (tep_is_file_bigendian(pevent)) {
 				extend_hdr = (time_delta & TRACE_DAT_RECORD_TIME_MASK) |
@@ -245,7 +249,7 @@ static int trace_dat__write_cpu_dat(FILE *fp, struct tep_handle *pevent,
 				extend_hdr = ((time_delta & TRACE_DAT_RECORD_TIME_MASK) <<
 					TRACE_DAT_RECORD_TIME_SHIFT) |
 					TRACE_DAT_RECORD_TYPE_TIME_EXTEND;
-				delta_upper = time_delta >> TRACE_DAT_RECORD_TIME_SHIFT;
+				delta_upper = time_delta >> 27;
 			}
 			extend_hdr  = to_file_u32(pevent, extend_hdr);   /* still needed */
 			delta_upper = to_file_u32(pevent, delta_upper);   /* still needed */
@@ -288,7 +292,7 @@ static int trace_dat__write_cpu_dat(FILE *fp, struct tep_handle *pevent,
 		/* Check page fit BEFORE allocating data record */
 		if (page_size_used + needed_size >
 			trace_dat_page_size - TRACE_DAT_RECORD_HEADER_SIZE) {
-			ret = trace_dat__write_page(fp, pevent, base_ts,
+			ret = trace_dat__write_page(fp, pevent, page_base_ts,
 					page_records, page_rec_sizes,
 					nr_page_recs);
 
@@ -298,6 +302,7 @@ static int trace_dat__write_cpu_dat(FILE *fp, struct tep_handle *pevent,
 			nr_page_recs = 0;
 			page_size_used = 0;
 			base_ts = event->ts;
+			page_base_ts = event->ts;
 
 			if (ret < 0) {
 				free(extend);
@@ -312,6 +317,7 @@ static int trace_dat__write_cpu_dat(FILE *fp, struct tep_handle *pevent,
 			extend = NULL;
 			extend_size = 0;
 			time_delta = 0;
+			base_ts = event->ts;
 		}
 
 		if (tep_is_file_bigendian(pevent))
@@ -387,10 +393,11 @@ static int trace_dat__write_cpu_dat(FILE *fp, struct tep_handle *pevent,
 		page_rec_sizes[nr_page_recs] = data_rec_size;
 		nr_page_recs++;
 		page_size_used += data_rec_size;
+		base_ts = event->ts;
 	}
 
 	if (nr_page_recs > 0) {
-		ret = trace_dat__write_page(fp, pevent, base_ts,
+		ret = trace_dat__write_page(fp, pevent, page_base_ts,
 				page_records, page_rec_sizes, nr_page_recs);
 	}
 out_free:
@@ -861,6 +868,9 @@ void trace_dat__free_cpu_buffers(void)
 	for (cpu = 0; cpu < trace_dat_nr_cpus; cpu++) {
 		int i;
 
+		if (!trace_cpu_data[cpu].events)
+			continue;
+
 		for (i = 0; i < trace_cpu_data[cpu].count; i++)
 			free(trace_cpu_data[cpu].events[i].raw);
 		free(trace_cpu_data[cpu].events);
tools/perf/util/trace-dat.h +1 -0
index 5985083b275a..40041dac60d4 100644
@@ -11,6 +11,7 @@
 #include <stdbool.h>
 #include <event-parse.h>
 #include <byteswap.h>
+#include <stdint.h>
 #include "util.h"
 
 /* trace.dat file format version */
Signature · 1 line
2.47.1
#8 lore raw

Re: [RFC PATCH v4 4/5] perf data: Add --to-trace-dat option for converting perf.data tracepoint events into trace.dat format

Thank you for your contribution! Sashiko AI review found 6 potential issue(s) to consider: - [High] Trace parser desynchronization: Large event array lengths must be encoded as the fully padded size…

Thank you for your contribution! Sashiko AI review found 6 potential issue(s) to consider:
- [High] Trace parser desynchronization: Large event array lengths must be encoded as the fully padded size minus the length word.
- [High] Out of bounds rejection: CPU array limits must be scoped to nr_cpus_avail rather than nr_cpus_online.
- [High] Silent I/O truncation: Stream checks must explicitly assert identical element counts instead of binary negation.
- [High] Architecture violation: Page header offsets must dynamically query the ftrace event format to accommodate 32-bit local_t structures.
- [Low] Build architecture violation: The data-convert.h header must include a dummy inline fallback stub for trace_convert__perf2dat.
- [Low] Dead code elimination: The skipped sample counter is incremented but structurally isolated and unread.
--

--- Patch [4]: [RFC PATCH v4 4/5] perf data: Add --to-trace-dat option for converting perf.data tracepoint events into trace.dat format --- commit 9a258038767861211db47cd6d942b553034516f1
Author: Tanushree Shah <[email protected]>

perf data: Add --to-trace-dat option for converting perf.data tracepoint events into trace.dat format

Add new command-line option to perf data convert for generating trace.dat output files.

Quoted 4 lines
tools/perf/builtin-data.c +0 -0
index 1dd73ed6bdcb8..c9c863197d02c 100644

[ ... ]

Quoted 1 line
tools/perf/builtin-data.c +0 -0
@@ -63,10 +72,44 @@ static int cmd_data_convert(int argc, const char **argv)

[ ... ]

tools/perf/builtin-data.c static int cmd_data_convert(int argc, const char **argv) · line 112

Quoted 5 lines
tools/perf/builtin-data.c +5 -0
+#ifdef HAVE_LIBTRACEEVENT
+	if (trace_dat_output)
+		return trace_convert__perf2dat(input_name ? input_name : "perf.data",
+					       trace_dat_output, &opts);
+#endif

[Severity: Low]
Does the data-convert.h header need a dummy inline fallback stub for trace_convert__perf2dat()?

Without an #else block providing an inline dummy stub, could this cause a build failure when compiling without HAVE_LIBTRACEEVENT?

tools/perf/util/data-convert-trace.c line 39

Quoted 13 lines
tools/perf/util/data-convert-trace.c +4 -0
index 445479fae8884..425b3aaa3f026 100644
@@ -31,6 +32,10 @@
 struct trace_convert {
 	struct perf_tool tool;
 	u64 events_count;
+	struct perf_time_interval *ptime_range;
+	int range_size;
+	int range_num;
+	u64 skipped;
 };

[Severity: Low]
Is this skipped counter ever read?

It is incremented when events are outside the specified time range, but appears to be structurally isolated and never emitted in the final conversion summary.

tools/perf/util/data-convert-trace.c static int process_sample_event(const struct perf_tool *tool, · line 98

Quoted 9 lines
tools/perf/util/data-convert-trace.c +5 -0
@@ -86,6 +91,11 @@ static int process_sample_event(const struct perf_tool *tool,
 		cpu_buffers_initialized = true;
 	}
 
+	if (perf_time__ranges_skip_sample(tc->ptime_range, tc->range_num, sample->time)) {
+		tc->skipped++;
+		return 0;
+	}
+

[Severity: High]
Are valid samples from sparse CPU topologies rejected here?

In process_sample_event(), the CPU buffers are initialized using nr_cpus_online:

    int nr_cpus = trace_dat_session->header.env.nr_cpus_online;

But trace_dat__collect_cpu_event() rejects samples if the CPU index is >= trace_dat_nr_cpus:

    if (!trace_cpu_data || cpu < 0 || cpu >= trace_dat_nr_cpus)
        return -EINVAL;

If a system has hotplugged CPUs (e.g. CPUs 0 and 127 are online, making nr_cpus_online 2), a valid sample from CPU 127 will be evaluated against cpu >= 2. This causes trace_dat__collect_cpu_event() to return -EINVAL, aborting the conversion.

Should this use nr_cpus_avail instead of nr_cpus_online?

Quoted 4 lines
tools/perf/util/trace-dat.c +0 -0
index f71e03716e27e..1b236745bcacc 100644

[ ... ]

tools/perf/util/trace-dat.c static int trace_dat__write_cpu_dat(FILE *fp, struct tep_handle *pevent, · line 297

Quoted 8 lines
tools/perf/util/trace-dat.c +1 -1
@@ -288,7 +292,7 @@ static int trace_dat__write_cpu_dat(FILE *fp, struct tep_handle *pevent,
 		/* Check page fit BEFORE allocating data record */
 		if (page_size_used + needed_size >
 			trace_dat_page_size - TRACE_DAT_RECORD_HEADER_SIZE) {
-			ret = trace_dat__write_page(fp, pevent, base_ts,
+			ret = trace_dat__write_page(fp, pevent, page_base_ts,
 					page_records, page_rec_sizes,
 					nr_page_recs);

[Severity: High]
Could the length word for large trace events be miscalculated?

Later in trace_dat__write_cpu_dat(), the array length is encoded:

    /* Large events: write actual byte length after header */
    if (type_len == 0) {
        unsigned int data_len_out = to_file_u32(pevent, data_len);
        memcpy(data_rec + TRACE_DAT_WORD_SIZE, &data_len_out, TRACE_DAT_WORD_SIZE);
    }

For large events (>=29 words), the ftrace ring buffer format dictates that the first word of the payload contains the length of the data plus the length word itself (data_len + 4).

By writing just data_len, might this cause downstream tools like trace-cmd to compute the event size incorrectly and lose parser synchronization?

[Severity: High]
Can a short write silently corrupt the trace.dat file?

In trace_dat__write_page(), the file write checks for a boolean false:

    if (!fwrite(page, 1, trace_dat_page_size, fp)) {
        free(page);
        return -EIO;
    }

Since fwrite() returns the number of elements written, a short write (e.g. 512 out of 4096 bytes) evaluates to true.

Will this silently drop data without reporting an error to the user if the disk runs out of space? Should the stream check explicitly assert fwrite(...) == trace_dat_page_size?

[Severity: High]
Will this hardcoded page header size break parsing for trace data recorded on 32-bit architectures?

In trace_dat__write_page(), the page header offsets are hardcoded:

    /* Byte-swap page header for cross-arch compatibility */
    ts_out = to_file_u64(pevent, base_ts);
    commit_out = to_file_u64(pevent, commit);
    memcpy(page, &ts_out, sizeof(ts_out));
    memcpy(page + sizeof(ts_out), &commit_out, sizeof(commit_out));

The ftrace page header includes a local_t commit field, which is 8 bytes on 64-bit systems but 4 bytes on 32-bit systems (making the header 12 bytes). Because this unconditionally assumes a 16-byte header and writes 8 bytes for commit_out, trace-cmd reading 32-bit metadata will likely parse the remaining 4 bytes as the first trace event.

Should the page header offsets and sizing be dynamically queried from the ftrace event format?

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
#9 lore raw

RFC PATCH v4 5/5 perf test: Add test validating trace.dat generated by 'perf data convert --to-trace-dat'

In reply to Tanushree Shah

Add a shell test covering perf data convert --to-trace-dat, alongside the existing --to-json and --to-ctf tests.

The test:
- skips if perf isn't linked with libtraceevent
- converts a plain sched:sched_switch recording to trace.dat
- converts a pipe-mode recording (perf record -o - | perf data convert -i -)
- converts a recording with both tracepoint and non-tracepoint events (sched:sched_switch,cycles)
- validates the resulting file with trace-cmd report if trace-cmd is installed, otherwise falls back to a basic non-empty-file check

---
 ...rf_data_converter_tracepoints_trace_dat.sh | 173 ++++++++++++++++++
 1 file changed, 173 insertions(+)
 create mode 100755 tools/perf/tests/shell/test_perf_data_converter_tracepoints_trace_dat.sh
tools/perf/tests/shell/test_perf_data_converter_tracepoints_trace_dat.sh new file +173 -0
new file mode 100755
index 000000000000..1c0bce270c38
@@ -0,0 +1,173 @@
+#!/bin/bash
+# 'perf data convert --to-trace-dat' command test
+# SPDX-License-Identifier: GPL-2.0-or-later
+#
+# Copyright 2026, IBM Corporation
+# Author: Tanushree Shah <[email protected]>
+
+set -e
+
+err=0
+
+perfdata=$(mktemp /tmp/__perf_test.perf.data.XXXXX)
+
+cleanup()
+{
+	rm -f "${perfdata}"
+	rm -f "${result}"
+	trap - exit term int
+}
+
+trap_cleanup()
+{
+	local exit_code=$?
+	if [ $exit_code -ne 0 ] && [ $exit_code -ne 2 ]; then
+		echo "Unexpected signal in ${FUNCNAME[1]}"
+		cleanup
+		exit 1
+	fi
+	cleanup
+	exit $exit_code
+}
+trap trap_cleanup exit term int
+
+result=$(mktemp /tmp/__perf_test.output.trace.dat.XXXXX)
+
+# Check if libtraceevent support is available
+if ! perf check feature libtraceevent
+then
+	echo "perf not linked with libtraceevent, skipping test"
+	exit 2
+fi
+
+# Check if trace-cmd is available for validation
+have_trace_cmd=0
+if command -v trace-cmd >/dev/null 2>&1; then
+	have_trace_cmd=1
+fi
+
+check_sched_switch()
+{
+	if ! perf list | grep -q 'sched:sched_switch'
+	then
+		echo "sched:sched_switch tracepoint not available, skipping test"
+		exit 2
+	fi
+}
+
+test_trace_converter_command()
+{
+	echo "Testing Perf Data Conversion Command to trace.dat"
+
+	if ! perf record -e sched:sched_switch -a -o "$perfdata" -- sleep 0.1
+	then
+		echo "Failed to record perf data"
+		err=1
+		return
+	fi
+
+	if ! perf data convert --to-trace-dat "$result" --force -i "$perfdata"
+	then
+		echo "Perf Data Converter Command to trace.dat [FAILED]"
+		err=1
+		return
+	fi
+
+	if [ -f "$result" ] && [ -s "$result" ] ; then
+		echo "Perf Data Converter Command to trace.dat [SUCCESS]"
+	else
+		echo "Perf Data Converter Command to trace.dat [FAILED]"
+		err=1
+	fi
+}
+
+test_trace_converter_pipe()
+{
+	echo "Testing Perf Data Conversion Command to trace.dat (Pipe mode)"
+
+	rm -f "$result"
+
+	if ! perf record -e sched:sched_switch -a -o - -- sleep 0.1 | \
+	   perf data convert --to-trace-dat "$result" --force -i -
+	then
+		echo "Perf Data Converter Command to trace.dat (Pipe mode) [FAILED]"
+		err=1
+		return
+	fi
+
+	if [ -f "$result" ] && [ -s "$result" ]; then
+		echo "Perf Data Converter Command to trace.dat (Pipe mode) [SUCCESS]"
+	else
+		echo "Perf Data Converter Command to trace.dat (Pipe mode) [FAILED]"
+		err=1
+	fi
+}
+
+test_trace_converter_mixed_events()
+{
+	echo "Testing Perf Data Conversion with tracepoint and non-tracepoint events"
+
+	rm -f "$result"
+
+	# Record both tracepoint and non-tracepoint events
+	if ! perf record -e sched:sched_switch,cpu-clock -a -o "$perfdata" -- sleep 0.1
+	then
+		echo "Failed to record perf data (mixed events)"
+		err=1
+		return
+	fi
+
+	if ! perf data convert --to-trace-dat "$result" --force -i "$perfdata"
+	then
+		echo "Perf Data Converter with mixed events [FAILED]"
+		err=1
+		return
+	fi
+
+	if [ -f "$result" ] && [ -s "$result" ] ; then
+		echo "Perf Data Converter with mixed events [SUCCESS]"
+	else
+		echo "Perf Data Converter with mixed events [FAILED]"
+		err=1
+	fi
+}
+
+validate_trace_format()
+{
+	echo "Validating Perf Data Converted trace.dat file"
+
+	if [ ! -f "$result" ] ; then
+		echo "File not found [FAILED]"
+		err=1
+		return
+	fi
+
+	# If trace-cmd is available, use it to validate the file
+	if [ "$have_trace_cmd" -eq 1 ] ; then
+		if trace-cmd report -i "$result" >/dev/null 2>&1 ; then
+			echo "trace-cmd can read the file [SUCCESS]"
+		else
+			echo "trace-cmd cannot read the file [FAILED]"
+			err=1
+		fi
+	else
+		# Without trace-cmd, just check file exists and has content
+		if [ -s "$result" ] ; then
+			echo "Output file exists and has content [SUCCESS]"
+		else
+			echo "Output file is empty [FAILED]"
+			err=1
+		fi
+	fi
+}
+
+check_sched_switch
+test_trace_converter_command
+validate_trace_format
+test_trace_converter_pipe
+validate_trace_format
+test_trace_converter_mixed_events
+validate_trace_format
+
+cleanup
+exit ${err}
Signature · 1 line
2.47.1
#10 lore raw

Re: [RFC PATCH v4 5/5] perf test: Add test validating trace.dat generated by 'perf data convert --to-trace-dat'

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider: - [Medium] Insecure temporary file reuse creates a time-of-check to time-of-use (TOCTOU) race condition and…

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [Medium] Insecure temporary file reuse creates a time-of-check to time-of-use (TOCTOU) race condition and symlink attack vulnerability.
- [Medium] The commit message claims to test the 'cycles' event, but the script uses 'cpu-clock'.
- [Medium] The script leaks a temporary file (${perfdata}.old) in /tmp because perf record automatically backs up the existing non-empty output file. --

--- Patch [5]: [RFC PATCH v4 5/5] perf test: Add test validating trace.dat generated by 'perf data convert --to-trace-dat' --- commit 9c32c2021bfda9ef574a5486cbcea176af233faa
Author: Tanushree Shah <[email protected]>

perf test: Add test validating trace.dat generated by 'perf data convert --to-trace-dat'

Add a shell test covering perf data convert --to-trace-dat, alongside the existing --to-json and --to-ctf tests.

The test:
- skips if perf isn't linked with libtraceevent
- converts a plain sched:sched_switch recording to trace.dat
- converts a pipe-mode recording (perf record -o - | perf data convert -i -)

Quoted 2 lines

- converts a recording with both tracepoint and non-tracepoint events (sched:sched_switch,cycles)

[Severity: Medium]
Is this description accurate? The commit message mentions the "cycles" event, but the code later tests the "cpu-clock" event instead.

- validates the resulting file with trace-cmd report if trace-cmd is installed, otherwise falls back to a basic non-empty-file check

Quoted 12 lines
tools/perf/tests/shell/test_perf_data_converter_tracepoints_trace_dat.sh new file +6 -0
new file mode 100755
index 0000000000000..1c0bce270c389
@@ -0,0 +1,173 @@
+#!/bin/bash
+# 'perf data convert --to-trace-dat' command test
+# SPDX-License-Identifier: GPL-2.0-or-later
+#
+# Copyright 2026, IBM Corporation
+# Author: Tanushree Shah <[email protected]>

tools/perf/tests/shell/test_perf_data_converter_tracepoints_trace_dat.sh line 16

Quoted 10 lines
tools/perf/tests/shell/test_perf_data_converter_tracepoints_trace_dat.sh new file +10 -0
+
+set -e
+
+err=0
+
+perfdata=$(mktemp /tmp/__perf_test.perf.data.XXXXX)
+
+cleanup()
+{
+	rm -f "${perfdata}"

[Severity: Medium]
Does this code leak the backup file created by perf?

When perf record is called a second time in test_trace_converter_mixed_events() with -o "$perfdata", it detects that the file already exists and has content. perf record automatically renames the existing file to "${perfdata}.old" before creating a new one.

Since cleanup() only removes "${perfdata}", does the .old file get left behind in /tmp?

Quoted 4 lines
tools/perf/tests/shell/test_perf_data_converter_tracepoints_trace_dat.sh new file +4 -0
+	rm -f "${result}"
+	trap - exit term int
+}
+

[ ... ]

tools/perf/tests/shell/test_perf_data_converter_tracepoints_trace_dat.sh line 91

Quoted 8 lines
tools/perf/tests/shell/test_perf_data_converter_tracepoints_trace_dat.sh new file +8 -0
+test_trace_converter_pipe()
+{
+	echo "Testing Perf Data Conversion Command to trace.dat (Pipe mode)"
+
+	rm -f "$result"
+
+	if ! perf record -e sched:sched_switch -a -o - -- sleep 0.1 | \
+	   perf data convert --to-trace-dat "$result" --force -i -

[Severity: Medium]
Does this code create a time-of-check to time-of-use race condition?

The script securely creates a temporary file using mktemp, but explicitly deletes it here with rm -f "$result".

Since the predictable and unsecured path is then immediately reused for perf data convert, could a local attacker insert a symlink during this window, causing perf data convert to overwrite arbitrary files?

Quoted 6 lines
tools/perf/tests/shell/test_perf_data_converter_tracepoints_trace_dat.sh new file +6 -0
+	then
+		echo "Perf Data Converter Command to trace.dat (Pipe mode) [FAILED]"
+		err=1
+		return
+	fi
+

[ ... ]

Quoted 5 lines
tools/perf/tests/shell/test_perf_data_converter_tracepoints_trace_dat.sh new file +5 -0
+test_trace_converter_mixed_events()
+{
+	echo "Testing Perf Data Conversion with tracepoint and non-tracepoint events"
+
+	rm -f "$result"

tools/perf/tests/shell/test_perf_data_converter_tracepoints_trace_dat.sh line 120

Quoted 10 lines
tools/perf/tests/shell/test_perf_data_converter_tracepoints_trace_dat.sh new file +10 -0
+
+	# Record both tracepoint and non-tracepoint events
+	if ! perf record -e sched:sched_switch,cpu-clock -a -o "$perfdata" -- sleep 0.1
+	then
+		echo "Failed to record perf data (mixed events)"
+		err=1
+		return
+	fi
+
+	if ! perf data convert --to-trace-dat "$result" --force -i "$perfdata"

[Severity: Medium]
Does this code repeat the symlink race condition mentioned above?

The securely created $result file is again explicitly deleted and its path is reused here.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5