tools/perf: Add powerpc HTM auxtrace support
v5 cover tools/perf: Add powerpc HTM auxtrace support
Overview:
This series adds perf tool support for capturing and processing Hardware Trace Macro (HTM) trace data on POWER systems. HTM exposes raw bus-trace data via the H_HTM hypervisor call. The kernel-side HTM PMU driver (submitted separately) streams this data into perf AUX buffers and emits memory configuration as PERF_SAMPLE_RAW records.
This series wires up the record and report sides in the perf tool so that 'perf record' collects HTM data and 'perf report' writes the trace and memory configuration to files ready for post-processing with htmdecode.
Patch overview:
Patch 1 refactors the existing powerpc auxtrace dispatch. All VPA-DTL recording logic is moved from arch/powerpc/util/auxtrace.c into its own file, arch/powerpc/util/vpa-dtl.c, leaving auxtrace_record__init() as a thin per-PMU dispatcher. This is a prerequisite for adding the HTM recording path cleanly.
Patch 2 adds the HTM recording path (arch/powerpc/util/htm.c and util/powerpc-htm.h). htm_recording_init() sets up the auxtrace_record callbacks. htm_info_fill() stores the PMU type and a (cpu, attr.config) pair for each htm evsel into the PERF_RECORD_AUXTRACE_INFO priv[] area. The decode side reads these back to map each AUX buffer to its (node, chip, core) target using event->auxtrace.cpu. PERF_SAMPLE_RAW is also enabled here so that memory configuration records are captured alongside the AUX stream.
Patch 3 adds a generic weak arch_perf_record__need_read() hook in builtin-record.c. When the hook returns non-zero, perf record performs an extra mmap-read pass before disabling events. The drain loop runs until the hook returns 0 or no forward progress is made, then the events are disabled and closed normally.
Patch 4 implements arch_perf_record__need_read() for powerpc. It reads event->count for all open htm evsels via perf_evsel__read() and accumulates the values. The kernel driver sets this count to the number of 128-byte HTM trace records written on a successful dump, 1 when the AUX buffer is temporarily full but the hypervisor stream is intact, and 0 once the stream is exhausted or a hard error occurs. A non-zero total causes the recording loop to perform another read pass.
Patch 5 adds PERF_AUXTRACE_POWERPC_HTM to the auxtrace_type enum and wires the dispatch in perf_event__process_auxtrace_info(). It also amends htm_info_fill() to set auxtrace_info->type now that the constant is available.
Patch 6 adds the full HTM decode path in util/powerpc-htm.c. powerpc_htm_process_auxtrace_info() reads the (cpu, config) pairs from priv[] and builds a cpu_configs[] lookup table. As each PERF_RECORD_AUXTRACE event arrives, process_auxtrace_event() maps it to the correct target and writes the data to htm.bin.nX.pX.cX.tX immediately. PERF_RECORD_SAMPLE RAW records are written to translation.nX.pX.cX.tX by process_event(). Per-run first-write tracking (htm_target_seen) ensures O_TRUNC on the first write and O_APPEND on subsequent writes.
Patches not included in V2:
Three patches present in V1 are intentionally dropped from this series and will be submitted as a follow-on once the core infrastructure gets integrated:
- htmdecode integration: V1 patch 6 invoked the external htmdecode tool via fork/exec after writing htm.bin.* to decode the raw bus-trace data in-process. This is deferred because the interface between perf and an external decode tool needs more discussion.
- Physical-to-logical address mapping: V1 patch 7 read the current partition ID from /proc/powerpc/lparcfg, parsed the translation.* memory map entries, and mapped physical addresses in the decoded trace to logical addresses within the LPAR.
- Synthetic sample generation: V1 patches 8 and 9 created a PERF_TYPE_SYNTH event named "htm", injected synthetic perf samples with the translated logical addresses as sample IP, and wrote a separate .out.l file with logical addresses alongside the decoded trace. The appropriate abstraction for synthetic HTM samples in the perf tool requires further discussion.
Usage example:
Collect HTM trace data from a single chip target:
# perf record -C 9 -m,256 \
-e htm/nodalchipindex=2,nodeindex=0,htm_type=1/ sleep 3
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 256.277 MB perf.data ]# perf report
Output files written by perf report:
htm.bin.n0.p2.c0.t1 raw HTM bus-trace data
translation.n0.p2.c0.t1 memory configuration records
Multiple targets can be collected simultaneously:
# perf record -m,256 \
-e htm/nodalchipindex=2,nodeindex=0,htm_type=1,cpu=8/ \
-e htm/nodalchipindex=1,nodeindex=0,htm_type=1,cpu=9/ \
-a sleep 10Testing:
Tested on POWER11. Both htm.bin.* and translation.* files are produced correctly for single and dual target collection.
Note: Link to kernel patches:
https://lore.kernel.org/linuxppc-dev/[email protected]/T/#t
Athira Rajeev (6):
tools/perf: Move powerpc VPA-DTL auxtrace init into a separate file tools/perf: Add AUXTRACE recording support for powerpc HTM tools/perf: Add arch hook to drain remaining data before event close tools/perf: Add powerpc callback support for
arch_perf_record__need_read
tools/perf: Add powerpc HTM auxtrace event processing support tools/perf: Add perf tool support for processing powerpc HTM AUXTRACE
records
tools/perf/arch/powerpc/util/Build | 2 + tools/perf/arch/powerpc/util/auxtrace.c | 98 ++---- tools/perf/arch/powerpc/util/evsel.c | 77 ++++ tools/perf/arch/powerpc/util/htm.c | 185 ++++++++++ tools/perf/arch/powerpc/util/vpa-dtl.c | 96 +++++ tools/perf/builtin-record.c | 65 ++++ tools/perf/util/Build | 1 + tools/perf/util/auxtrace.c | 4 + tools/perf/util/auxtrace.h | 1 + tools/perf/util/powerpc-htm.c | 447 ++++++++++++++++++++++++ tools/perf/util/powerpc-htm.h | 43 +++ tools/perf/util/powerpc-vpadtl.h | 3 + tools/perf/util/record.h | 4 + 13 files changed, 954 insertions(+), 72 deletions(-) create mode 100644 tools/perf/arch/powerpc/util/htm.c create mode 100644 tools/perf/arch/powerpc/util/vpa-dtl.c create mode 100644 tools/perf/util/powerpc-htm.c create mode 100644 tools/perf/util/powerpc-htm.h
Signature · 1 line
2.53.0
PATCH v5 1/6 tools/perf: Move powerpc VPA-DTL auxtrace init into a separate file
The powerpc auxtrace dispatch lives entirely in arch/powerpc/util/auxtrace.c. As new PMUs such as HTM are added, this file would grow to contain the recording logic for all of them.
Factor out the VPA-DTL recording initializer into its own file, arch/powerpc/util/vpa-dtl.c, and reduce auxtrace.c to a thin dispatch layer. auxtrace_record__init() now detects the PMU by name and calls the appropriate per-PMU init function:
- vpa_dtl_recording_init() for VPA-DTL events (unchanged behaviour)
- Further PMU entries will follow in subsequent patches
This makes room in auxtrace_record__init() for the HTM recording path added in the next patch without growing a single monolithic file.
---
Changes in V5:
- Add forward declarations for struct evsel and struct auxtrace_record in powerpc-vpadtl.h before the vpa_dtl_recording_init() prototype. Without them, a translation unit that includes the header before the full definitions are visible may produce implicit-declaration warnings on strict compilers.
Changes in V4:
- No changes from V3.
Changes in V3:
Add #include <linux/zalloc.h> to vpa-dtl.c; without it the compiler treats zalloc() as implicitly returning int, truncating the upper 32 bits of the returned pointer on 64-bit PowerPC.
Changes in V2:
- Renamed the destination file from arch/powerpc/util/vpa-dtl.c (same name, unchanged) but the subject and commit message are reworded to clearly state that the goal is to make auxtrace_record__init() a thin per-PMU dispatcher, not merely to "allow multiple PMUs to use auxtrace".
- Handle failure from memory allocation
- Included stdlib and limits.h
- No functional change to the VPA-DTL path itself.
- Patch is now 1/6 instead of 1/9.
tools/perf/arch/powerpc/util/Build | 1 + tools/perf/arch/powerpc/util/auxtrace.c | 84 +++------------------- tools/perf/arch/powerpc/util/vpa-dtl.c | 96 +++++++++++++++++++++++++ tools/perf/util/powerpc-vpadtl.h | 3 + 4 files changed, 108 insertions(+), 76 deletions(-) create mode 100644 tools/perf/arch/powerpc/util/vpa-dtl.c
tools/perf/arch/powerpc/util/Build +1 -0
@@ -7,3 +7,4 @@ perf-util-y += evsel.o perf-util-$(CONFIG_LIBDW) += skip-callchain-idx.o perf-util-y += auxtrace.o +perf-util-y += vpa-dtl.o
tools/perf/arch/powerpc/util/auxtrace.c +8 -76
104 lines not on this page · raw message
tools/perf/arch/powerpc/util/vpa-dtl.c new file +96 -0
96 lines not on this page · raw message
tools/perf/util/powerpc-vpadtl.h +3 -0
@@ -20,4 +20,7 @@ struct perf_pmu;int powerpc_vpadtl_process_auxtrace_info (union perf_event *event,struct perf_session *session); +struct evsel ; +struct auxtrace_record ; +struct auxtrace_record *vpa_dtl_recording_init (struct evsel *pos,int *err);#endif
Signature · 1 line
2.53.0
PATCH v5 2/6 tools/perf: Add AUXTRACE recording support for powerpc HTM
In reply to Athira RajeevAdd powerpc perf tool support for the HTM PMU AUXTRACE recording path.
Introduce htm_recording_init() and the associated auxtrace callbacks so perf record can create PERF_RECORD_AUXTRACE records for HTM data.
htm_info_fill() stores the PMU type, number of HTM evsels, and a (cpu, attr.config) pair for each evsel into PERF_RECORD_AUXTRACE_INFO priv[]. The decode side reads these back to map each AUX buffer (identified by event->auxtrace.cpu) to the correct (node, chip, core) target when writing htm.bin.* output files.
Update auxtrace_record__init() to detect HTM events and dispatch to the HTM-specific recording initializer.
---
Changes in V5:
- htm_recording_options(): clamp aux_watermark with min_t(size_t, wm, UINT_MAX) before assigning to pos->core.attr.aux_watermark (u32). On a system with large auxtrace_mmap_pages the unclamped product can exceed UINT_MAX and silently truncate on 32-bit platforms.
- htm_info_fill(): iterate all CPUs in each evsel's CPU map and write one (cpu, config) entry per CPU instead of only the first CPU. An evsel opened with -C 0,1,2 now contributes three priv[] entries, one per CPU, each carrying the same attr.config. The decode side (powerpc_htm_process_auxtrace_event) uses event->auxtrace.cpu to look up the matching entry, so every CPU's AUX buffer is correctly mapped to its (node, chip, core, type) target and written to the right output file. An evsel with no CPU map records cpu=-1 as before.
Changes in V4:
- Add explicit mutual-exclusion check in auxtrace_record__init(): if both vpa_dtl_evsel and htm_evsel are set after scanning the evlist, print an error, set *err = -EINVAL, and return NULL. V3 silently fell through to prefer VPA-DTL when both PMU types were present, leaving HTM AUX buffers collected without a PERF_RECORD_AUXTRACE_INFO record and the trace undecodable.
- Drop the superfluous pos->name && null-guard from the HTM detection condition in auxtrace_record__init(): the condition becomes !strcmp(evsel__pmu_name(pos), "htm") without the leading check. evsel__pmu_name() handles a NULL name internally so the guard is redundant.
- Apply the same simplification in htm_recording_options() and htm_nr_events() in htm.c: remove the !pos->name || prefix from the evsel__pmu_name() comparisons, consistent with the dispatcher change above.
- Set pos->core.attr.sample_period = 1 for each HTM evsel inside htm_recording_options(). V3 set freq=0 and aux_watermark but did not set sample_period, leaving the kernel to use whatever default value was in the attr, which could cause the HTM event to fire at an unintended rate.
Changes in V3:
- Switch HTM event detection in auxtrace_record__init() from strstarts(pos->name, "htm") to !strcmp(evsel__pmu_name(pos), "htm"), matching the kernel-assigned PMU name and preventing false matches on user-named events.
- Switch htm_recording_options(), htm_nr_events(), and the loop in htm_info_fill() from strstarts(pos->name, "htm") to evsel__pmu_name(), consistent with the above and with patches 4 and 6.
- Add #include <linux/zalloc.h> to htm.c for zalloc().
- Set auxtrace_info->type = PERF_AUXTRACE_POWERPC_HTM directly in htm_info_fill(), removing the placeholder comment. (V2 deferred this to patch 5; it is cleaner to set it in the same patch that defines the enum.)
- Set aux_watermark to half the AUX ring-buffer size (auxtrace_mmap_pages * page_size / 2) so there is always headroom for hardware to write while userspace drains. V2 set it to the full buffer size, risking data loss under back-pressure.
- Add PERF_AUXTRACE_POWERPC_HTM to the enum auxtrace_type in util/auxtrace.h and add #include "powerpc-htm.h" plus a stub case PERF_AUXTRACE_POWERPC_HTM: (falls through to PERF_AUXTRACE_UNKNOWN) in util/auxtrace.c in this patch, so the enum is defined before it is used in htm_info_fill(). (V2 did this in patch 5.)
Changes in V2:
- htm_info_fill() now stores a (cpu, attr.config) pair for every HTM evsel into PERF_RECORD_AUXTRACE_INFO priv[]. V1 stored only the PMU type and a single config value; there was no per-CPU mapping.
- The priv[] layout is formalised in util/powerpc-htm.h with enum constants POWERPC_HTM_PMU_TYPE, POWERPC_HTM_NUM_EVENTS, and POWERPC_HTM_EVENT_DATA and the helper macros HTM_AUXTRACE_PRIV_FIXED and HTM_AUXTRACE_PRIV_SIZE(n). V1 used bare numeric offsets.
- PERF_SAMPLE_RAW is enabled in the recording options so that memory configuration records emitted by the kernel driver are captured alongside the AUX stream. V1 added this in a later patch.
- PERF_AUXTRACE_POWERPC_HTM type constant is set in htm_info_fill() (wired up in patch 5 once the enum is defined).
- Patch is now 2/6 instead of 2/9.
tools/perf/arch/powerpc/util/Build | 1 + tools/perf/arch/powerpc/util/auxtrace.c | 22 +++ tools/perf/arch/powerpc/util/htm.c | 185 ++++++++++++++++++++++++ tools/perf/util/auxtrace.c | 2 + tools/perf/util/auxtrace.h | 1 + tools/perf/util/powerpc-htm.h | 43 ++++++ 6 files changed, 254 insertions(+) create mode 100644 tools/perf/arch/powerpc/util/htm.c create mode 100644 tools/perf/util/powerpc-htm.h
tools/perf/arch/powerpc/util/Build +1 -0
@@ -8,3 +8,4 @@ perf-util-$(CONFIG_LIBDW) += skip-callchain-idx.o perf-util-y += auxtrace.o perf-util-y += vpa-dtl.o +perf-util-y += htm.o
tools/perf/arch/powerpc/util/auxtrace.c +22 -0
@@ -12,13 +12,16 @@#include "../../util/debug.h" #include "../../util/auxtrace.h" #include "../../util/powerpc-vpadtl.h" +#include "../../util/powerpc-htm.h" #include "../../util/record.h" +#include <string.h> struct auxtrace_record *auxtrace_record__init (struct evlist *evlist,int *err) {struct evsel *pos;struct evsel *vpa_dtl_evsel =NULL ; +struct evsel *htm_evsel =NULL ;/* * Set err value to zero here. Any fail later
@@ -32,11 +35,30 @@ struct auxtrace_record *auxtrace_record__init(struct evlist *evlist,/* Remember the first matching VPA DTL event */ if (!vpa_dtl_evsel) vpa_dtl_evsel = pos; + }else if (!strcmp (evsel__pmu_name (pos),"htm" )) { + pos->needs_auxtrace_mmap = true; +/* Remember the first matching HTM event */ +if (!htm_evsel) + htm_evsel = pos; } } +/* +* Only one auxtrace PMU can be initialised per session. Reject +* concurrent VPA DTL and HTM events: HTM AUX buffers would be +* collected without a PERF_RECORD_AUXTRACE_INFO record, making +* the trace undecodable. +*/ +if (vpa_dtl_evsel && htm_evsel) { +pr_err ("Cannot record VPA DTL and HTM auxtrace events simultaneously\n" ); + *err = -EINVAL ; +return NULL ; + } +if (vpa_dtl_evsel)return vpa_dtl_recording_init (vpa_dtl_evsel, err); +else if (htm_evsel) +return htm_recording_init (htm_evsel, err);return NULL ; }
tools/perf/arch/powerpc/util/htm.c new file +185 -0
185 lines not on this page · raw message
tools/perf/util/auxtrace.c +2 -0
@@ -56,6 +56,7 @@#include "s390-cpumsf.h" #include "util/mmap.h" #include "powerpc-vpadtl.h" +#include "powerpc-htm.h" #include <linux/ctype.h> #include "symbol/kallsyms.h"
@@ -1432,6 +1433,7 @@ int perf_event__process_auxtrace_info(const struct perf_tool *tool __maybe_unusecase PERF_AUXTRACE_VPA_DTL : err =powerpc_vpadtl_process_auxtrace_info (event, session);break ; +case PERF_AUXTRACE_POWERPC_HTM :case PERF_AUXTRACE_UNKNOWN :default :return -EINVAL ;
tools/perf/util/auxtrace.h +1 -0
@@ -46,6 +46,7 @@ enum auxtrace_type {PERF_AUXTRACE_S390_CPUMSF ,PERF_AUXTRACE_HISI_PTT ,PERF_AUXTRACE_VPA_DTL , + PERF_AUXTRACE_POWERPC_HTM, }; enum itrace_period_type {
tools/perf/util/powerpc-htm.h new file +43 -0
43 lines not on this page · raw message
Signature · 1 line
2.53.0
PATCH v5 3/6 tools/perf: Add arch hook to drain remaining data before event close
In reply to Athira RajeevWhile collecting samples using perf record, __cmd_record() disables the evlist once recording is complete. After that, event fds are no longer read and any remaining PMU-specific data cannot be drained.
Add a weak arch_perf_record__need_read() hook so architecture code can indicate that more data remains to be collected before events are disabled and closed. When the hook reports pending data, perf record performs another read pass via record__final_data().
A static volatile sig_atomic_t drain_interrupted flag is added. sig_handler() sets it when a second signal (SIGINT/SIGTERM) arrives while done is already set, allowing a second Ctrl+C during the drain loop to abort immediately. Without this, a user who presses Ctrl+C twice while HTM data is being drained would have no way to interrupt a stalled drain loop.
This allows architectures such as powerpc HTM to drain trace data and associated metadata before the event is closed.
---
Changes in V5:
- Rename record__final_data() to record__final_aux_data() to make the AUX-specific purpose explicit.
- Guard both call sites with if (rec->opts.full_auxtrace) so the function is only entered when AUX tracing is actually active.
- Remove the per-thread TLS redirect loop and 'int t' variable from record__final_aux_data(). The existing record__init_thread_masks() path already rejects --threads combined with full_auxtrace, so rec->nr_threads > 1 and rec->opts.full_auxtrace cannot both be true at runtime; the check was redundant dead code. A single record__mmap_read_all(rec, true) call on the main thread is sufficient and race-free.
Changes in V4:
- Fix drain_interrupted to not trigger on SIGCHLD: change the condition in sig_handler() from "if (done)" to "if (done && sig != SIGCHLD)". V3 set drain_interrupted on a SIGCHLD that arrived while done was already set, aborting the drain loop on normal child workload completion even though no second Ctrl+C was pressed.
- Drain all thread mmaps in record__final_data(): iterate over all rec->nr_threads slots and temporarily switch the TLS 'thread' pointer to each thread_data[t] before calling record__mmap_read_all(), then reset it to thread_data[0] afterwards. V3 called record__mmap_read_all() once with whatever 'thread' pointed to, leaving the mmaps of worker threads un-drained when --threads is in use. Adds local variable 'int t' for the loop counter.
- Fix the post-loop fallback condition from "if (!disabled)" to "if (target__none(&opts->target) || !disabled)" so that the final drain is always executed for child workloads. In the child-workload path (target__none == true) the in-loop disable block is never entered, so disabled stays false, but the old "!disabled" test happened to be true there only by accident; making the intent explicit also handles any future path where disabled could be set early. Remove the now-redundant evlist__disable() call that V3 placed inside the post-loop block, since the disable is handled by the existing code below.
Changes in V3:
- Add static volatile sig_atomic_t drain_interrupted. Set it in sig_handler() when a second signal arrives while done is already set. This lets a second Ctrl+C abort the drain loop immediately. V2 tested done > 1 to detect a second signal, which is unreachable because done is only ever set to 1.
- Replace rec->bytes_written with record__bytes_written(rec) (which includes rec->thread_bytes_written) so the no-progress check accounts for data written by the AIO/thread path.
- Add a retry counter (FINAL_DATA_MAX_RETRIES 20, 20 x 1 ms = 20 ms maximum) instead of aborting on the first stall. The no-progress sleep of 1 ms is enough for the AUX ring buffer consumer (perf's ~1 ms poll interval) to advance the tail pointer.
- Move record__final_data() into the in-loop disable block (before evlist__disable()) as well as into a post-loop if (!disabled) block that covers both the early-break and child-workload paths. V2 only called it inside the loop with a final_data_drained guard, missing the early-break and child-workload cases.
- Remove the now-unnecessary bool final_data_drained local variable.
Changes in V2:
- V1's callback was responsible for driving the read loop including evlist__enable cycling. Removed that logic
- Use bytes written to check if session needs to be continued.
- Patch is now 3/6 instead of 3/9.
tools/perf/builtin-record.c | 65 +++++++++++++++++++++++++++++++++++++ tools/perf/util/record.h | 4 +++ 2 files changed, 69 insertions(+)
tools/perf/builtin-record.c +65 -0
@@ -192,6 +192,7 @@ struct record { };static volatile int done; +static volatile sig_atomic_t drain_interrupted;static volatile int auxtrace_record__snapshot_started;static DEFINE_TRIGGER (auxtrace_snapshot_trigger);
@@ -681,6 +682,8 @@ static void sig_handler(int sig)else signr = sig; +if (done && sig !=SIGCHLD ) + drain_interrupted =1 ; done =1 ;#ifdef HAVE_EVENTFD_SUPPORT if (done_fd >=0 ) {
@@ -2437,6 +2440,57 @@ static unsigned long record__waking(struct record *rec)return waking; } +/* +* Weak arch hook called by record__final_data(). +* Returns 1 if the arch PMU driver still has records pending (the caller +* will call record__mmap_read_all() and retry), 0 when done. +* Implementations use perf_evsel__read() so this must be called while +* events are still ACTIVE (before evlist__disable()). +*/ +__weak int arch_perf_record__need_read (struct evlist *evlist__maybe_unused ) +{ +return 0 ; +} + +static void record__final_aux_data (struct record *rec) +{ +u64 last_bytes_written =0 ; +int retries =0 ; +#define FINAL_DATA_MAX_RETRIES 20/* 20 * 1 ms = 20 ms max wait */ + +/* +* Drain any remaining AUX data. Called only when full_auxtrace is +* set; --threads is mutually exclusive with full_auxtrace and is +* rejected at open time, so the main thread's thread_data[0] covers +* all CPUs here. +* arch_perf_record__need_read() calls perf_evsel__read() and +* therefore requires events to still be ACTIVE (before +* evlist__disable()). +* A second SIGINT/SIGTERM sets drain_interrupted to abort immediately. +*/ +while (arch_perf_record__need_read (rec->evlist)) { +if (drain_interrupted) +break ; + + last_bytes_written =record__bytes_written (rec); + +if (record__mmap_read_all (rec, true) <0 ) +return ; + +if (record__bytes_written (rec) == last_bytes_written) { +if (++retries >=FINAL_DATA_MAX_RETRIES ) { +pr_warning ("Final AUX data drain made no forward progress after %d retries.\n" , +FINAL_DATA_MAX_RETRIES ); +break ; + } +usleep (1000 );/* 1 ms: let AUX ring buffer consumer advance */ + }else { + retries =0 ; +usleep (100 ); + } + } +} +static int __cmd_record (struct record *rec,int argc,const char **argv) {int err;
@@ -2864,11 +2918,22 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)*/ if (done && !disabled && !target__none (&opts->target)) {trigger_off (&auxtrace_snapshot_trigger); +if (rec->opts.full_auxtrace) +record__final_aux_data (rec);evlist__disable (rec->evlist); disabled = true; } } +/* +* If the loop exited without entering the in-loop disable block +* (early break, or child workload where target__none is true and +* the block is never reached), drain any remaining AUX data now. +* Events are still live at this point. +*/ +if ((target__none (&opts->target) || !disabled) && rec->opts.full_auxtrace) +record__final_aux_data (rec); +trigger_off (&auxtrace_snapshot_trigger);trigger_off (&switch_output_trigger);
tools/perf/util/record.h +4 -0
@@ -95,4 +95,8 @@ static inline bool record_opts__no_switch_events(const struct record_opts *opts) return opts->record_switch_events_set && !opts->record_switch_events; } +struct evlist ; +struct record ; +int arch_perf_record__need_read (struct evlist *evlist); +#endif // _PERF_RECORD_H
Signature · 1 line
2.53.0
PATCH v5 4/6 tools/perf: Add powerpc callback support for arch_perf_record__need_read
In reply to Athira RajeevImplement the arch_perf_record__need_read() architecture-specific hook for powerpc in arch/powerpc/util/evsel.c.
The HTM kernel driver sets event->count to the number of records still staged in its internal buffers (total_size / record_size), and to 0 once the stream is exhausted. This hook reads that count for every open htm evsel via perf_evsel__read() and accumulates the values into total_pending_records. A non-zero total means at least one HTM target still has records pending; the recording loop added in the previous patch will perform another mmap-read pass.
The drain uses a two-layer safety check: event->count detects records staged by the driver, and record__bytes_written() in the drain loop confirms data was actually moved into perf.data. This combination handles the case where the driver count is briefly stale while hardware is still flushing.
The implementation scans the evlist using evsel__pmu_name() to identify HTM events by their kernel-assigned PMU name rather than the user-visible event name, preventing false matches. It iterates the fd/ sample-id xyarray, and skips any evsel whose fd and sample-id arrays are mismatched to avoid reading stale state. When the accumulated record count reaches zero the hook returns 0 and the recording loop proceeds to disable and close the events.
---
Changes in V5:
- When an HTM evsel is a group sibling (evsel->core.leader != &evsel->core), read through its group leader's struct perf_evsel instead of the sibling directly. perf_evsel__read_size() uses evsel->nr_members to compute the read buffer size; nr_members is 0 for siblings, so size=0 is passed to readn(), which returns <=0 and leaves count.val=0, causing the drain loop to terminate prematurely. Reading through the leader avoids the zero-size buffer and correctly accumulates the leader's pending count. HTM events are always standalone or per-target leaders in practice; the leader redirect handles any grouped configuration without losing counts.
Changes in V4:
- No changes from V3.
Changes in V3:
- Use evsel__pmu_name(evsel) instead of strstarts(evsel->name, "htm") to identify HTM events, matching by kernel-assigned PMU name rather than user-visible event name.
- Remove the redundant two-pass loop (first pass to set found_htm, second to accumulate counts); a single pass with evsel__pmu_name() is sufficient. if no HTM event exists total_pending_records stays 0 and the function returns 0.
- Remove the dead !strcmp(evsel->name, "dummy:u") check;
- evsel__pmu_name() will never return "htm" for a dummy:u software event.
- Rename total_pending_bytes -> total_pending_records to match what the driver actually reports (event->count = total_size / record_size, a record count, not a byte count).
- Add #include <string.h> for musl compatibility (strcmp() without it warns on some toolchains).
Changes in V2:
- Implements the renamed arch_perf_record__need_read() hook (V1 implemented arch_record__collect_final_data()).
- Skips evsels whose fd and sample-id xyarrays are mismatched, avoiding stale-state reads. V1 had no such guard.
- evlist__enable cycling is removed; that responsibility now belongs to the drain loop in builtin-record.c added in patch 3.
- File location changed to arch/powerpc/util/evsel.c (V1 used arch/powerpc/util/powerpc-htm.c).
- Patch is now 4/6 instead of 4/9.
tools/perf/arch/powerpc/util/evsel.c | 77 ++++++++++++++++++++++++++++ 1 file changed, 77 insertions(+)
tools/perf/arch/powerpc/util/evsel.c +77 -0
@@ -1,8 +1,85 @@// SPDX-License-Identifier: GPL-2.0 #include <stdio.h> +#include <string.h> +#include <unistd.h> +#include <linux/string.h> #include "util/evsel.h" +#include "util/record.h" +#include "util/evlist.h" +#include "util/debug.h" +#include <internal/xyarray.h> +#include <internal/lib.h> void arch_evsel__set_sample_weight (struct evsel *evsel) {evsel__set_sample_bit (evsel,WEIGHT_STRUCT ); } + +/* +* powerpc implementation of arch_perf_record__need_read(). +* +* Reads event->count for every open HTM evsel by issuing a direct +* read() on the event fd with a plain u64 buffer, bypassing the +* PERF_FORMAT_GROUP path in perf_evsel__read(). When an HTM evsel is +* a group sibling, evsel__config() sets PERF_FORMAT_GROUP on its attr; +* perf_evsel__read() would then call perf_evsel__read_group() which +* sizes the buffer by evsel->nr_members (0 for siblings), causing the +* kernel to return -ENOSPC. Reading the fd directly with sizeof(u64) +* retrieves the HTM driver's plain pending-record count regardless of +* group membership. +* +* Returns: 1 if more data exists, 0 if collection is complete +*/ +int arch_perf_record__need_read (struct evlist *evlist) +{ +struct evsel *evsel; +u64 total_pending_records =0 ; +int x, y; + +/* there was an error during record__open */ +if (!evlist) +return 0 ; + +/* Read HTM event counts to check if more data is available */ +evlist__for_each_entry (evlist, evsel) { +struct perf_evsel *rd_evsel; +struct xyarray *xy; + +if (strcmp (evsel__pmu_name (evsel),"htm" )) +continue ; + +/* +* For group siblings nr_members == 0, which makes +* perf_evsel__read_size() return 0 and readn() fail. +* Read through the leader instead; perf_evsel__read_group() +* extracts the leader's own count from the group buffer. +*/ +if (evsel->core.leader != &evsel->core) + rd_evsel = evsel->core.leader; +else + rd_evsel = &evsel->core; + + xy = rd_evsel->sample_id; + +if (xy ==NULL || rd_evsel->fd ==NULL ) +continue ; + +if (xyarray__max_x (rd_evsel->fd) !=xyarray__max_x (xy) || +xyarray__max_y (rd_evsel->fd) !=xyarray__max_y (xy)) { +pr_debug ("Unmatched FD vs sample ID array for HTM event\n" ); +continue ; + } + +for (x =0 ; x <xyarray__max_x (xy); x++) { +for (y =0 ; y <xyarray__max_y (xy); y++) { +struct perf_counts_values count = { .val =0 }; + +if (perf_evsel__read (rd_evsel, x, y, &count) ==0 ) + total_pending_records += count.val; + } + } + } + +/* Collection is complete only when ALL hardware queues have no pending records */ +return (total_pending_records >0 ) ?1 :0 ; +}
Signature · 1 line
2.53.0
PATCH v5 5/6 tools/perf: Add powerpc HTM auxtrace event processing support
In reply to Athira RajeevAdd the powerpc-htm.c decode stub and wire the dispatch in perf_event__process_auxtrace_info() to call powerpc_htm_process_auxtrace_info().
---
Changes in V4:
- Add #include <linux/zalloc.h> to the powerpc-htm.c stub. V3 omitted it from the stub; without it the zalloc() call added in patch 6 would require the include to be introduced there instead of alongside the struct definition it serves. Moving it here keeps the allocation header co-located with the powerpc_htm struct that patch 6 expands.
Changes in V3:
- Change powerpc_htm_dump_event() parameter from size_t to u64 and use %" PRIu64 ", matching the type of buffer->size and avoiding truncation on 32-bit platforms.
- Add #include <inttypes.h> for PRIu64.
- Remove the htm->auxtrace_type = auxtrace_info->type assignment that was added here in V2; this field is now set from priv[POWERPC_HTM_PMU_TYPE] in patch 6 where the full processing logic lives.
- Do not re-add PERF_AUXTRACE_POWERPC_HTM to auxtrace.h or the stub case in auxtrace.c; those are now in patch 2. This patch only adds the powerpc-htm.c decode stub and wires the dispatch to call powerpc_htm_process_auxtrace_info().
Changes in V2:
- Scope narrowed: this patch now only adds the PERF_AUXTRACE_POWERPC_HTM enum constant to auxtrace.h and wires the dispatch in perf_event__process_auxtrace_info() to call powerpc_htm_process_auxtrace_info(). It also amends htm_info_fill() (from patch 2) to set auxtrace_info->type now that the constant is defined.
- All file-writing and decoding logic is moved to patch 6.
- Patch is now 5/6 instead of 5/9.
tools/perf/util/Build | 1 + tools/perf/util/auxtrace.c | 2 + tools/perf/util/powerpc-htm.c | 118 ++++++++++++++++++++++++++++++++++ 3 files changed, 121 insertions(+) create mode 100644 tools/perf/util/powerpc-htm.c
tools/perf/util/Build +1 -0
@@ -141,6 +141,7 @@ perf-util-y += hisi-ptt.o perf-util-y += hisi-ptt-decoder/ perf-util-y += s390-cpumsf.o perf-util-y += powerpc-vpadtl.o +perf-util-y += powerpc-htm.o ifdef CONFIG_LIBOPENCSD perf-util-y += cs-etm.o
tools/perf/util/auxtrace.c +2 -0
@@ -1434,6 +1434,8 @@ int perf_event__process_auxtrace_info(const struct perf_tool *tool __maybe_unuse err =powerpc_vpadtl_process_auxtrace_info (event, session);break ;case PERF_AUXTRACE_POWERPC_HTM : + err =powerpc_htm_process_auxtrace_info (event, session); +break ;case PERF_AUXTRACE_UNKNOWN :default :return -EINVAL ;
tools/perf/util/powerpc-htm.c new file +118 -0
118 lines not on this page · raw message
Signature · 1 line
2.53.0
PATCH v5 6/6 tools/perf: Add perf tool support for processing powerpc HTM AUXTRACE records
In reply to Athira Rajeevpowerpc_htm_process_auxtrace_info() reads the (cpu, attr.config) pairs stored by htm_info_fill() at record time and builds a cpu_configs[] table. This lets process_auxtrace_event() map each AUX buffer, which carries event->auxtrace.cpu, to the correct (node, chip, core) target and write the raw trace data to htm.bin.nX.pX.cX.tX immediately.
PERF_RECORD_SAMPLE events carrying PERF_SAMPLE_RAW data are handled by process_event(), which extracts memory configuration records and writes them to translation.nX.pX.cX.tX
Per-run first-write tracking (htm_target_seen()) ensures each output file is opened with O_TRUNC on the first write and O_APPEND on subsequent writes, correctly handling multiple AUX chunks from the same target and stale files from prior runs.
For perf report -D the AUX buffer size is printed.
Example:
# perf record -C 9 -m,256 \
-e htm/nodalchipindex=2,nodeindex=0,htm_type=1/ sleep 3
# perf report
# ls htm.bin.* translation.*
htm.bin.n0.p2.c0.t1 translation.n0.p2.c0.t1---
Changes in V5:
- Include htm_type in the output filename and the first-write tracking key. Previously htm_pack_target() encoded only (node, chip, core), so a core HTM event (htm_type=2) and a nest HTM event (htm_type=1) on the same node/chip/core produced the same key and collided into the same htm.bin.nX.pX.cX file. The fix encodes type in bits [31:24] of a u64 key: key = (type<<24) | (node<<16) | (chip<<8) | core. Output filenames now include a .tX suffix:
htm.bin.nX.pX.cX.tX and translation.nX.pX.cX.tX
Both htm_bin_targets and translation_targets arrays are widened from u32 to u64, and htm_target_seen() is updated accordingly. The type field is extracted from attr.config bits [3:0] at both call sites (powerpc_htm_process_event and powerpc_htm_process_auxtrace_event) and passed through write_htm().
Changes in V4:
- Change htm_config_for_cpu() from returning u64 (using 0 as a "not found" sentinel) to returning bool with a u64 *config output parameter. V3's 0-return sentinel is ambiguous because 0 is a theoretically valid attr.config value; a CPU pinned to the default chip with default field values could legitimately produce config=0, causing htm_config_for_cpu() to silently report "not found" and skip writing the AUX buffer. The bool+output-param API removes the ambiguity entirely. Update the call site in powerpc_htm_process_auxtrace_event() to use the new signature:
replace "ev_config = htm_config_for_cpu(htm, cpu); if (!ev_config)" with "if (!htm_config_for_cpu(htm, cpu, &ev_config))".
- Fix the type of the written local variable in write_htm() from size_t to ssize_t, matching the return type of write(2). V3 stored the signed write() return value in an unsigned size_t, so a write error (-1) would silently wrap to SIZE_MAX and the mismatch check "written != size" would fire with a misleading byte count in the error message rather than surfacing the underlying errno. Update both pr_err format specifiers from %zu to %zd and both comparisons to cast written: (size_t)written != payload and (size_t)written != size.
- Drop the HTM_MAX_SAFE_TARGETS (1024) cap on num_events that V3 introduced. The num_events value is already bounded by the auxtrace_info header size validation added in V3, making the separate cap redundant; removing it simplifies the code without weakening the safety guarantee.
Changes in V3:
- Add #include <linux/unaligned.h> (placed after the perf "..."
headers so <linux/compiler.h> is already in scope) and use get_unaligned_be64(byte_ptr + 0x10) instead of be64_to_cpu(*(__be64 *)(byte_ptr + 0x10)) to avoid SIGBUS on strict-alignment architectures.
- Add raw_size underflow guard: skip silently if sample->raw_size <= sizeof(uint32_t) before subtracting the 4-byte alignment padding.
- Reject duplicate PERF_RECORD_AUXTRACE_INFO:
check session->auxtrace != NULL before allocation to prevent
overwriting the pointer and leaking the first allocation.
- Validate num_events against the actual auxtrace_info->header.size
before using it to index priv[] or allocate arrays, preventing out-of-bounds reads from a malformed record.
- Cap num_events at HTM_MAX_SAFE_TARGETS (1024) to prevent size_t truncation on 32-bit platforms when passed to calloc().
- Fix htm_target_seen(): return true (use O_APPEND) when capacity is exceeded, matching the warning message that already claimed append behaviour. V2 returned false (would have used O_TRUNC), discarding previously written data.
- Cast calloc() arguments to (size_t)num_events now that num_events
has been validated and capped.
Changes in V2:
- Consolidated: the file-writing, memory-configuration processing, and address-mapping logic that was spread across old patches 5 to 9 is rationalised into a single, focused implementation in tools/perf/util/powerpc-htm.c.
- Synthetic sample generation (old patch 8: PERF_TYPE_SYNTH "htm" event with logical addresses) and the separate logical-address mapping output file (old patch 9: .out.l file) are not included in V2. These features require further discussion on the appropriate abstraction and will be posted as a follow-on series.
- Physical-to-logical address mapping (old patch 7) is not included in V2 for the same reason.
- Memory configuration records are now written via process_event() handling PERF_RECORD_SAMPLE with PERF_SAMPLE_RAW, keyed on the "htm" PMU name from evsel__pmu_name(). V1 used PERF_SAMPLE_RAW boundary markers inside the AUX buffer to locate the configuration data.
- AUX buffer processing uses htm_config_for_cpu() to map event->auxtrace.cpu to the correct (node, chip, core) target using the (cpu, attr.config) table populated from AUXTRACE_INFO priv[] in powerpc_htm_process_auxtrace_info(). V1 iterated the evlist at report time.
- Per-run first-write tracking (htm_target_seen()) uses O_TRUNC on the first write and O_APPEND on subsequent writes for both htm.bin.* and translation.* files, correctly handling multiple AUX chunks per target and stale files from prior runs. V1 always opened with O_TRUNC.
- HTM_MAX_TARGETS compile-time constant is replaced by dynamic allocation sized to num_events read from AUXTRACE_INFO priv[POWERPC_HTM_NUM_EVENTS].
- Patch is now 6/6 instead of spanning patches 5 to 9.
tools/perf/util/powerpc-htm.c | 341 +++++++++++++++++++++++++++++++++- 1 file changed, 335 insertions(+), 6 deletions(-)
tools/perf/util/powerpc-htm.c +335 -6
@@ -17,6 +17,10 @@#include "debug.h" #include "sample.h" +#include <linux/unaligned.h> + +struct perf_session ; +struct powerpc_htm {struct auxtrace auxtrace;struct auxtrace_queues queues;
@@ -24,8 +28,56 @@ struct powerpc_htm {u32 auxtrace_type;struct perf_session *session;struct machine *machine; + +/* +* Capacity: number of distinct HTM targets (node/chip/core tuples) +* recorded, read from auxtrace_info->priv[POWERPC_HTM_NUM_EVENTS] at +* init time. All three arrays below are allocated to this size. +*/ +int nr_targets; + +/* +* Per-run first-write tracking for htm.bin.* and translation.* files. +* Each entry is a packed u64: type<<24 | node<<16 | chip<<8 | core. +* First write for a given key -> O_TRUNC; subsequent writes -> O_APPEND. +*/ +u64 *htm_bin_targets; +int nr_htm_bin_targets; +u64 *translation_targets; +int nr_translation_targets; + +/* +* CPU -> attr.config table, populated from auxtrace_info->priv[] at +* init time. htm_info_fill() (record side) stores the CPU and config +* for each htm evsel; we read them back here to map each +* event->auxtrace.cpu to the correct (node, chip, core) config. +*/ +struct { +int cpu; +u64 config; + } *cpu_configs; +int nr_cpu_configs; }; +/* +* Look up attr.config by the CPU number carried in event->auxtrace.cpu. +* Returns true and sets *config if found, false if not found. +* Using a bool+output-param avoids the ambiguity of returning 0 as both +* a sentinel (not found) and a theoretically valid config value. +*/ +static bool htm_config_for_cpu (struct powerpc_htm *htm,int cpu,u64 *config) +{ +int i; + +for (i =0 ; i < htm->nr_cpu_configs; i++) { +if (htm->cpu_configs[i].cpu == cpu) { + *config = htm->cpu_configs[i].config; +return true; + } + } +return false; +} +static void powerpc_htm_dump_event (u64 len) {const char *color =PERF_COLOR_BLUE ;
@@ -36,22 +88,238 @@ static void powerpc_htm_dump_event(u64 len) } } -static int powerpc_htm_process_event (struct perf_session *session__maybe_unused , -union perf_event *event__maybe_unused , -struct perf_sample *sample__maybe_unused , +#define HTM_MEM_ENTRY_SIZE 32 + +static inline u64 htm_pack_target (u32 node,u32 chip,u32 core,u32 type) +{ +return ((u64 )type <<24 ) | (node <<16 ) | (chip <<8 ) | core; +} + +static bool htm_target_seen (u64 *targets,int *nr,int capacity,u64 key) +{ +int i; + +for (i =0 ; i < *nr; i++) { +if (targets[i] == key) +return true; + } + +if (*nr < capacity) + targets[(*nr)++] = key; +else { +pr_warning ("htm: too many targets (max %d), appending to existing file\n" , + capacity); +return true;/* treat as seen: use O_APPEND not O_TRUNC */ + } + +return false; +} + +/* +* Write HTM data to a file. +* +* mem_maps == 0: AUX bus-trace path -> htm.bin.nX.pX.cX +* mem_maps != 0: memory config path -> translation.nX.pX.cX +* +* htm_target_seen() decides O_TRUNC (first write this run) vs O_APPEND +* (subsequent writes), keyed on what this process has already written -- +* not on whether the file exists on disk. +*/ +static int write_htm (struct powerpc_htm *htm,void *data,size_t size, +u32 node,u32 chip,u32 core,u32 type,int mem_maps) +{ +u64 target_key =htm_pack_target (node, chip, core, type); +char target_file[128 ]; +ssize_t written; +int flags; +int fd; + +if (!data || !size) +return -EINVAL ; + + flags =O_CREAT |O_WRONLY |O_NOFOLLOW |O_CLOEXEC ; + +if (mem_maps) { +uint8_t *byte_ptr = (uint8_t *)data; +size_t entries; +size_t payload; + +if (size <HTM_MEM_ENTRY_SIZE ) { +pr_err ("Malformed memory mapping entry trace segment\n" ); +return -EINVAL ; + } + +/* Entry count is at offset 0x10; add 1 for the 32-byte header */ + entries =get_unaligned_be64 (byte_ptr +0x10 ) +1 ; + payload = entries *HTM_MEM_ENTRY_SIZE ; + +if (payload != size) { +pr_err ("Bad memory mapping data, invalid number of entries\n" ); +return -EINVAL ; + } + +snprintf (target_file,sizeof (target_file), +"translation.n%d.p%d.c%d.t%d" , node, chip, core, type); + flags |=htm_target_seen (htm->translation_targets, + &htm->nr_translation_targets, + htm->nr_targets, + target_key) ?O_APPEND :O_TRUNC ; + fd =open (target_file, flags,0644 ); +if (fd ==-1 ) { +pr_err ("Failed to open %s: %s\n" , target_file,strerror (errno)); +return -errno; + } + + written =write (fd, data, payload); +close (fd); + +if ((size_t )written != payload) { +pr_err ("Failed to write memory config: expected %zu bytes, wrote %zd\n" , + payload, written); +return -EIO ; + } + +return 0 ; + } + +/* AUX bus-trace path */ +snprintf (target_file,sizeof (target_file), +"htm.bin.n%d.p%d.c%d.t%d" , node, chip, core, type); + flags |=htm_target_seen (htm->htm_bin_targets, + &htm->nr_htm_bin_targets, + htm->nr_targets, + target_key) ?O_APPEND :O_TRUNC ; + fd =open (target_file, flags,0644 ); +if (fd ==-1 ) { +pr_err ("Failed to open %s: %s\n" , target_file,strerror (errno)); +return -errno; + } + + written =write (fd, data, size); +close (fd); + +if ((size_t )written != size) { +pr_err ("Failed to write htm trace data: expected %zu bytes, wrote %zd\n" , + size, written); +return -EIO ; + } + +return 0 ; +} + +static int powerpc_htm_process_event (struct perf_session *session, +union perf_event *event, +struct perf_sample *sample,const struct perf_tool *tool__maybe_unused ) { +struct powerpc_htm *htm; +struct evsel *evsel; +u32 node, chip, core, type; +u64 ev_config; + +if (!session || !session->auxtrace || !event || !sample) +return 0 ; + +if (event->header.type !=PERF_RECORD_SAMPLE || !sample->raw_data) +return 0 ; + + htm = container_of(session->auxtrace, struct powerpc_htm, auxtrace); + evsel =evlist__event2evsel (session->evlist, event); + +if (!evsel ||strcmp (evsel__pmu_name (evsel),"htm" ) !=0 ) +return 0 ; + + ev_config = evsel->core.attr.config; + type = ev_config &0xf ; + node = (ev_config >>4 ) &0xff ; + chip = (ev_config >>12 ) &0xff ; + core = (ev_config >>20 ) &0xff ; + +/* +* raw_size includes 4 bytes of u64 alignment padding added by the +* kernel. Subtract sizeof(u32) to recover the true payload byte count. +* Guard against underflow: if raw_size is too small, skip silently. +*/ +if (sample->raw_size <=sizeof (uint32_t )) +return 0 ; +if (write_htm (htm, sample->raw_data, + sample->raw_size -sizeof (uint32_t ), + node, chip, core, type,1 ) <0 ) { +pr_err ("Failed to write memory translation block\n" ); +return -EIO ; + } +return 0 ; } -static int powerpc_htm_process_auxtrace_event (struct perf_session *session__maybe_unused , +static int powerpc_htm_process_auxtrace_event (struct perf_session *session,union perf_event *event,const struct perf_tool *tool__maybe_unused ) { +struct powerpc_htm *htm; +struct auxtrace_buffer *buffer; +off_t data_offset; +u32 node, chip, core, type; +u64 ev_config; +int fd; +int err; + +if (!session || !session->auxtrace) +return 0 ; + + htm = container_of(session->auxtrace, struct powerpc_htm, auxtrace); + fd =perf_data__fd (session->data); + +if (perf_data__is_pipe (session->data)) { + data_offset =0 ; + }else { + data_offset =lseek (fd,0 ,SEEK_CUR ); +if (data_offset ==-1 ) +return -errno; + } + +/* +* Queue the buffer and get back a pointer to it. We immediately load +* and write the data so htm.bin.* exists on disk before subsequent +* patches invoke htmdecode during the same session pass. +*/ + err =auxtrace_queues__add_event (&htm->queues, session, event, + data_offset, &buffer); +if (err) +return err; + +if (!buffer) +return 0 ; + +/* +* Map event->auxtrace.cpu -> attr.config using the table built from +* auxtrace_info->priv[] at init time. This is reliable because +* htm_info_fill() stored the exact (cpu, config) pair for each evsel +* at record time -- no CPU map or evlist iteration needed here. +*/ +if (!htm_config_for_cpu (htm, (int )event->auxtrace.cpu, &ev_config)) { +pr_err ("htm: no config found for auxtrace cpu %u\n" , + event->auxtrace.cpu); +return 0 ; + } + + type = ev_config &0xf ; + node = (ev_config >>4 ) &0xff ; + chip = (ev_config >>12 ) &0xff ; + core = (ev_config >>20 ) &0xff ; + +if (!auxtrace_buffer__get_data (buffer, fd)) { +pr_err ("Failed to read AUX buffer data\n" ); +return -ENOMEM ; + } +if (dump_trace) -powerpc_htm_dump_event (event->auxtrace.size); +powerpc_htm_dump_event (buffer->size); -return 0 ; + err =write_htm (htm, buffer->data, buffer->size, node, chip, core, type,0 ); +auxtrace_buffer__put_data (buffer); + +return err <0 ? err :0 ; } static int powerpc_htm_flush(struct perf_session *session __maybe_unused,
@@ -81,6 +349,9 @@ static void powerpc_htm_free(struct perf_session *session) htm = container_of(session->auxtrace, struct powerpc_htm, auxtrace);powerpc_htm_free_events (session); session->auxtrace =NULL ; +free (htm->cpu_configs); +free (htm->htm_bin_targets); +free (htm->translation_targets);free (htm); }
@@ -89,6 +360,8 @@ int powerpc_htm_process_auxtrace_info(union perf_event *event, { struct perf_record_auxtrace_info *auxtrace_info = &event->auxtrace_info; struct powerpc_htm *htm; + u64 num_events; + u64 i; int err; if (auxtrace_info->header.size < sizeof(struct perf_record_auxtrace_info) +
@@ -99,12 +372,68 @@ int powerpc_htm_process_auxtrace_info(union perf_event *event,if (!htm)return -ENOMEM ; +/* Reject duplicate AUXTRACE_INFO: would overwrite session->auxtrace and leak */ +if (session->auxtrace) { +pr_err ("htm: duplicate PERF_RECORD_AUXTRACE_INFO, ignoring\n" ); +free (htm); +return -EINVAL ; + } + + htm->auxtrace_type = auxtrace_info->priv[POWERPC_HTM_PMU_TYPE ]; + num_events = auxtrace_info->priv[POWERPC_HTM_NUM_EVENTS ]; + +/* +* Validate num_events against the actual header size before using it +* to index priv[] or allocate arrays. Each event contributes 2 u64 +* priv entries (cpu + config); reject if the header is too small. +*/ +if (num_events > (auxtrace_info->header.size - +sizeof (struct perf_record_auxtrace_info ) - +HTM_AUXTRACE_PRIV_FIXED ) / (2 *sizeof (u64))) { +pr_err ("htm: num_events %llu exceeds auxtrace_info payload\n" , + (unsigned long long )num_events); +free (htm); +return -EINVAL ; + } + err =auxtrace_queues__init (&htm->queues);if (err) {free (htm);return err; } +/* +* All three arrays are sized to num_events -- the exact count of HTM +* targets written by htm_info_fill() at record time. num_events has +* been validated above so the cast to int and size_t are safe. +*/ + htm->nr_targets = (int )num_events; + htm->cpu_configs =calloc ((size_t )num_events,sizeof (*htm->cpu_configs)); + htm->htm_bin_targets =calloc ((size_t )num_events,sizeof (*htm->htm_bin_targets)); + htm->translation_targets =calloc ((size_t )num_events,sizeof (*htm->translation_targets)); +if (!htm->cpu_configs || !htm->htm_bin_targets || !htm->translation_targets) { +free (htm->cpu_configs); +free (htm->htm_bin_targets); +free (htm->translation_targets); +auxtrace_queues__free (&htm->queues); +free (htm); +return -ENOMEM ; + } + +/* +* Read (cpu, config) pairs from priv[]. These were written by +* htm_info_fill() at record time -- one pair per htm evsel in evlist +* order. Keying by CPU lets process_auxtrace_event() look up the +* correct attr.config for each AUX buffer using event->auxtrace.cpu. +*/ +for (i =0 ; i < num_events; i++) { + htm->cpu_configs[i].cpu = + (int )auxtrace_info->priv[POWERPC_HTM_EVENT_DATA + i *2 ]; + htm->cpu_configs[i].config = + auxtrace_info->priv[POWERPC_HTM_EVENT_DATA + i *2 +1 ]; + htm->nr_cpu_configs++; + } + htm->session = session; htm->machine = &session->machines.host; htm->auxtrace.process_event = powerpc_htm_process_event;
Signature · 1 line
2.53.0
[email protected]
AI reviewRe: [PATCH V5 2/6] tools/perf: Add AUXTRACE recording support for powerpc HTM
In reply to Athira RajeevThank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [High] Heap buffer overflow in htm_info_fill() due to undersized PERF_RECORD_AUXTRACE_INFO allocation for…
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] Heap buffer overflow in htm_info_fill() due to undersized PERF_RECORD_AUXTRACE_INFO allocation for HTM evsels covering multiple CPUs.
- [Low] Missing explicit inclusion of <string.h> for strcmp(), violating musl libc compatibility guidelines.
--
--- Patch [2]: [PATCH V5 2/6] tools/perf: Add AUXTRACE recording support for powerpc HTM --- commit 36cc90227b9052ae32b72e6b7df67e4e6bc7ac1e
Author: Athira Rajeev <[email protected]>
tools/perf: Add AUXTRACE recording support for powerpc HTM
Add powerpc perf tool support for the HTM PMU AUXTRACE recording path. Introduce htm_recording_init() and the associated auxtrace callbacks so perf record can create PERF_RECORD_AUXTRACE records for HTM data.
Quoted 6 lines
tools/perf/arch/powerpc/util/htm.c new file +0
-0
tools/perf/arch/powerpc/util/htm.c line 9
Quoted 9 lines
tools/perf/arch/powerpc/util/htm.c new file +9
-0@@ -0,0 +1,185 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* HTM AUX tracing support +*/ + +#include <linux/kernel.h> +#include <linux/types.h> +#include <linux/string.h> +#include <linux/zalloc.h>
[Severity: Low]
This isn't a bug, but this file uses strcmp() which is declared in <string.h>. Does this code need to explicitly include <string.h> to avoid relying on implicit inclusion, which might cause build failures in strictly conformant musl libc environments?
[ ... ]
Quoted 18 lines
tools/perf/arch/powerpc/util/htm.c new file +18
-0+/* Count htm evsels in the evlist */ +static int htm_nr_events (struct evlist *evlist) +{ +struct evsel *pos; +int n =0 ; + +evlist__for_each_entry (evlist, pos) { +if (!strcmp (evsel__pmu_name (pos),"htm" )) + n++; + } +return n; +} + +static size_t htm_info_priv_size (struct auxtrace_record *itr__maybe_unused , +struct evlist *evlist) +{ +return HTM_AUXTRACE_PRIV_SIZE (htm_nr_events (evlist)); +}
[ ... ]
Quoted 47 lines
tools/perf/arch/powerpc/util/htm.c new file +47
-0+/* +* Fill the PERF_RECORD_AUXTRACE_INFO private data with: +* priv[POWERPC_HTM_PMU_TYPE] = pmu->type of the first htm evsel +* priv[POWERPC_HTM_NUM_EVENTS] = total number of HTM CPU entries +* priv[POWERPC_HTM_EVENT_DATA + n*2] = CPU number for nth entry +* priv[POWERPC_HTM_EVENT_DATA + n*2 + 1] = attr.config for nth entry +* +* One entry is written per CPU in each evsel's cpu map. An evsel opened +* with -C 0,1,2 contributes three entries (one per CPU), each carrying +* the same attr.config. The decode side uses event->auxtrace.cpu to look +* up the matching config and derive (node, chip, core) for the output +* file name. +*/ +static int +htm_info_fill (struct auxtrace_record *itr, +struct perf_session *session, +struct perf_record_auxtrace_info *auxtrace_info, +size_t priv_size) +{ + struct htm_recording *htm_r = container_of(itr, struct htm_recording, itr); +struct evlist *evlist = session->evlist; +struct evsel *pos; +int n =0 ; +int expected_n =htm_nr_events (evlist); + +if (priv_size !=HTM_AUXTRACE_PRIV_SIZE (expected_n)) +return -EINVAL ; + + auxtrace_info->type =PERF_AUXTRACE_POWERPC_HTM ; + auxtrace_info->priv[POWERPC_HTM_PMU_TYPE ] = htm_r->evsel->core.attr.type; + auxtrace_info->priv[POWERPC_HTM_NUM_EVENTS ] = expected_n; + +evlist__for_each_entry (evlist, pos) { +struct perf_cpu_map *cpus; +int i, nr; + +if (strcmp (evsel__pmu_name (pos),"htm" )) +continue ; + +/* +* Emit one (cpu, config) entry for every CPU in this evsel's +* map. perf record -C 0,1,2 creates one evsel with a +* three-entry cpu map; each CPU gets its own AUX buffer and +* must be individually mapped so the decoder can match +* event->auxtrace.cpu to the correct (node, chip, core). +*/ + cpus =evsel__cpus (pos);
tools/perf/arch/powerpc/util/htm.c line 135
Quoted 10 lines
tools/perf/arch/powerpc/util/htm.c new file +10
-0+ nr = cpus ?perf_cpu_map__nr (cpus) :0 ; + +if (nr >0 ) { +for (i =0 ; i < nr; i++) { +int cpu =perf_cpu_map__cpu (cpus, i).cpu; + + auxtrace_info->priv[POWERPC_HTM_EVENT_DATA + n *2 ] = cpu; + auxtrace_info->priv[POWERPC_HTM_EVENT_DATA + n *2 +1 ] = + pos->core.attr.config; + n++;
[Severity: High]
Does this code overflow the auxtrace_info->priv[] array when an HTM evsel covers multiple CPUs?
The auxtrace_info->priv[] buffer size is calculated by htm_nr_events() above, which only counts the total number of HTM evsels. If a user runs perf record with a CPU list (e.g., -C 0,1,2), there might be only 1 evsel, resulting in space allocated for a single (cpu, config) pair.
However, this loop iterates over the CPU map and writes a pair for every CPU in that evsel (e.g., 3 CPUs), incrementing n past the allocated size.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
[email protected]
AI reviewRe: [PATCH V5 3/6] tools/perf: Add arch hook to drain remaining data before event close
In reply to Athira RajeevThank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] The `record__final_aux_data()` drain loop can spin infinitely during system-wide or…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] The record__final_aux_data() drain loop can spin infinitely during system-wide or attached-process tracing because it is called before disabling events, allowing the active PMU to continuously generate new data. Furthermore, if the main loop exits via early break, events are never explicitly disabled. --
--- Patch [3]: [PATCH V5 3/6] tools/perf: Add arch hook to drain remaining data before event close --- commit a89566ddd972102f4f5d6b30845f8dc3daa34fb4
Author: Athira Rajeev <[email protected]>
tools/perf: Add arch hook to drain remaining data before event close
This commit introduces a mechanism to drain remaining AUX data before closing event file descriptors. It uses a weak architecture hook to check for pending data and retries reading until progress stalls.
Quoted 4 lines
tools/perf/builtin-record.c +0
-0
[ ... ]
Quoted 18 lines
tools/perf/builtin-record.c +14
-0@@ -2425,6 +2428,57 @@ static unsigned long record__waking(struct record *rec)return waking; } +/* +* Weak arch hook called by record__final_data(). +* Returns 1 if the arch PMU driver still has records pending (the caller +* will call record__mmap_read_all() and retry), 0 when done. +* Implementations use perf_evsel__read() so this must be called while +* events are still ACTIVE (before evlist__disable()). +*/ +__weak int arch_perf_record__need_read (struct evlist *evlist__maybe_unused ) +{ +return 0 ; +} + +static void record__final_aux_data (struct record *rec) +{
[ ... ]
Quoted 13 lines
tools/perf/builtin-record.c +13
-0+if (record__bytes_written (rec) == last_bytes_written) { +if (++retries >=FINAL_DATA_MAX_RETRIES ) { +pr_warning ("Final AUX data drain made no forward progress after %d retries.\n" , +FINAL_DATA_MAX_RETRIES ); +break ; + } +usleep (1000 );/* 1 ms: let AUX ring buffer consumer advance */ + }else { + retries =0 ; +usleep (100 ); + } + } +}
[ ... ]
Quoted 4 lines
tools/perf/builtin-record.c +0
-0@@ -2852,11 +2906,22 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)/* * When perf is starting the traced process, at the end events * die with the process and we wait for that. Thus no need to
tools/perf/builtin-record.c static int __cmd_record(struct record *rec, int argc, const char **argv) · line 2918
Quoted 10 lines
tools/perf/builtin-record.c +2
-0* disable events in this case. */ if (done && !disabled && !target__none (&opts->target)) {trigger_off (&auxtrace_snapshot_trigger); +if (rec->opts.full_auxtrace) +record__final_aux_data (rec);evlist__disable (rec->evlist); disabled = true; } }
[Severity: High]
Could this lead to an infinite loop during system-wide or attached-process tracing?
Because record__final_aux_data() is called before evlist__disable(), the events are still active and generating new data.
If the workload is continuously generating data, record__bytes_written() will constantly increase, resetting the retry counter to 0 in the record__final_aux_data() loop we see above.
Would it be possible to disable the events first, and then drain the remaining data?
tools/perf/builtin-record.c static int __cmd_record(struct record *rec, int argc, const char **argv) · line 2935
Quoted 8 lines
tools/perf/builtin-record.c +8
-0+/* +* If the loop exited without entering the in-loop disable block +* (early break, or child workload where target__none is true and +* the block is never reached), drain any remaining AUX data now. +* Events are still live at this point. +*/ +if ((target__none (&opts->target) || !disabled) && rec->opts.full_auxtrace) +record__final_aux_data (rec);
[Severity: High]
If the main loop exits via an early break and this fallback block is reached, does this leave the events permanently active?
Since evlist__disable() is never called in this path, we might spin infinitely in record__final_aux_data() for the same reason mentioned above, continuously reading newly generated data.
Quoted 2 lines
tools/perf/builtin-record.c +0
-0trigger_off (&auxtrace_snapshot_trigger);trigger_off (&switch_output_trigger);
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
[email protected]
AI reviewRe: [PATCH V5 4/6] tools/perf: Add powerpc callback support for arch_perf_record__need_read
In reply to Athira RajeevThank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] Redirecting group sibling reads to the group leader incorrectly accumulates the leader's event…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] Redirecting group sibling reads to the group leader incorrectly accumulates the leader's event count instead of the HTM sibling's pending record count, leading to silent data loss and spurious warnings. --
--- Patch [4]: [PATCH V5 4/6] tools/perf: Add powerpc callback support for arch_perf_record__need_read --- commit 27e1bb8fdcf56e1c7b4262795fafefb9a6b92d9e
Author: Athira Rajeev <[email protected]>
tools/perf: Add powerpc callback support for arch_perf_record__need_read
This commit implements the arch_perf_record__need_read() hook for powerpc. It reads the pending record count for HTM events to determine if more data needs to be drained during the recording loop.
Quoted 4 lines
tools/perf/arch/powerpc/util/evsel.c +0
-0
[ ... ]
Quoted 17 lines
tools/perf/arch/powerpc/util/evsel.c +17
-0+/* +* powerpc implementation of arch_perf_record__need_read(). +* +* Reads event->count for every open HTM evsel by issuing a direct +* read() on the event fd with a plain u64 buffer, bypassing the +* PERF_FORMAT_GROUP path in perf_evsel__read(). When an HTM evsel is +* a group sibling, evsel__config() sets PERF_FORMAT_GROUP on its attr; +* perf_evsel__read() would then call perf_evsel__read_group() which +* sizes the buffer by evsel->nr_members (0 for siblings), causing the +* kernel to return -ENOSPC. Reading the fd directly with sizeof(u64) +* retrieves the HTM driver's plain pending-record count regardless of +* group membership. +* +* Returns: 1 if more data exists, 0 if collection is complete +*/ +int arch_perf_record__need_read (struct evlist *evlist) +{
[ ... ]
Quoted 23 lines
tools/perf/arch/powerpc/util/evsel.c +23
-0+/* Read HTM event counts to check if more data is available */ +evlist__for_each_entry (evlist, evsel) { +struct perf_evsel *rd_evsel; +struct xyarray *xy; + +if (strcmp (evsel__pmu_name (evsel),"htm" )) +continue ; + +/* +* For group siblings nr_members == 0, which makes +* perf_evsel__read_size() return 0 and readn() fail. +* Read through the leader instead; perf_evsel__read_group() +* extracts the leader's own count from the group buffer. +*/ +if (evsel->core.leader != &evsel->core) + rd_evsel = evsel->core.leader; +else + rd_evsel = &evsel->core; + + xy = rd_evsel->sample_id; + +if (xy ==NULL || rd_evsel->fd ==NULL ) +continue ;
[ ... ]
tools/perf/arch/powerpc/util/evsel.c line 78
Quoted 6 lines
tools/perf/arch/powerpc/util/evsel.c +6
-0+for (x =0 ; x <xyarray__max_x (xy); x++) { +for (y =0 ; y <xyarray__max_y (xy); y++) { +struct perf_counts_values count = { .val =0 }; + +if (perf_evsel__read (rd_evsel, x, y, &count) ==0 ) + total_pending_records += count.val;
[Severity: Medium]
When an HTM event is a sibling, does this inadvertently read and accumulate the group leader's value rather than the HTM sibling's pending record count?
The block above redirects reading to the leader, and then perf_evsel__read() is called here. For group leaders, this delegates to perf_evsel__read_group(), which extracts only the leader's count and discards sibling counts. This would accumulate the wrong event's data. If the leader is a non-HTM event like cycles, its continuously increasing counter will be read instead. This might cause the drain loop to falsely believe data is still pending, exhaust the drain retry limit, and emit a spurious warning, while ignoring the actual HTM sibling's pending records.
Also, the function comment explicitly mentions bypassing the PERF_FORMAT_GROUP path in perf_evsel__read() by issuing a direct read(), but this code directly calls perf_evsel__read(). Could the comment and implementation be out of sync?
Quoted 3 lines
tools/perf/arch/powerpc/util/evsel.c +3
-0+ } + } + }
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
[email protected]
AI reviewRe: [PATCH V5 5/6] tools/perf: Add powerpc HTM auxtrace event processing support
In reply to Athira RajeevThank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [Medium] If multiple PERF_RECORD_AUXTRACE_INFO events are encountered, session->auxtrace is overwritten without freeing the previously allocated structures, leading to a memory leak. --
--- Patch [5]: [PATCH V5 5/6] tools/perf: Add powerpc HTM auxtrace event processing support --- commit bdf509e26b28cea056c3306c8b70fad62802e9ca
Author: Athira Rajeev <[email protected]>
tools/perf: Add powerpc HTM auxtrace event processing support
Add the powerpc-htm.c decode stub and wire the dispatch in perf_event__process_auxtrace_info() to call powerpc_htm_process_auxtrace_info().
Quoted 5 lines
tools/perf/util/powerpc-htm.c new file +0
-0
[ ... ]
Quoted 6 lines
tools/perf/util/powerpc-htm.c new file +6
-0+int powerpc_htm_process_auxtrace_info (union perf_event *event, +struct perf_session *session) +{ +struct perf_record_auxtrace_info *auxtrace_info = &event->auxtrace_info; +struct powerpc_htm *htm; +int err;
[ ... ]
Quoted 8 lines
tools/perf/util/powerpc-htm.c new file +8
-0+ htm =zalloc (sizeof (struct powerpc_htm )); +if (!htm) +return -ENOMEM ; + + err =auxtrace_queues__init (&htm->queues); +if (err) { +free (htm); +return err;
tools/perf/util/powerpc-htm.c line 115
Quoted 10 lines
tools/perf/util/powerpc-htm.c new file +10
-0+ } + + htm->session = session; + htm->machine = &session->machines.host; + htm->auxtrace.process_event = powerpc_htm_process_event; + htm->auxtrace.process_auxtrace_event = powerpc_htm_process_auxtrace_event; + htm->auxtrace.flush_events = powerpc_htm_flush; + htm->auxtrace.free_events = powerpc_htm_free_events; + htm->auxtrace.free = powerpc_htm_free; + session->auxtrace = &htm->auxtrace;
[Severity: Medium]
This isn't a bug introduced by this patch, but if multiple PERF_RECORD_AUXTRACE_INFO events are encountered in powerpc_htm_process_auxtrace_info(), does this overwrite session->auxtrace without freeing the previously allocated structures?
Could a maliciously crafted perf.data file containing multiple such events trigger a memory leak here?
Quoted 3 lines
tools/perf/util/powerpc-htm.c new file +3
-0+ +return 0 ; +}
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
Re: [PATCH V5 2/6] tools/perf: Add AUXTRACE recording support for powerpc HTM
In reply to [email protected]Quoted 126 lines · On 7 Aug 2026, at 8:29 PM, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] Heap buffer overflow inhtm_info_fill()due to undersizedPERF_RECORD_AUXTRACE_INFOallocation for HTM evsels covering multiple CPUs.
- [Low] Missing explicit inclusion of<string.h>forstrcmp(), violating musl libc compatibility guidelines.
----- Patch [2]: [PATCH V5 2/6]
tools/perf: Add AUXTRACE recording support for powerpc HTM --- commit36cc90227b9052ae32b72e6b7df67e4e6bc7ac1e
Author: Athira Rajeev <[email protected]>
tools/perf: Add AUXTRACE recording support for powerpc HTMAdd powerpc perf tool support for the HTM PMU AUXTRACE recording path. Introduce
htm_recording_init()and the associated auxtrace callbacks so perf record can createPERF_RECORD_AUXTRACErecords for HTM data.Quoted 6 lines
tools/perf/arch/powerpc/util/htm.c new file +0
-0tools/perf/arch/powerpc/util/htm.c line 9
Quoted 9 lines
tools/perf/arch/powerpc/util/htm.c new file +9
-0@@ -0,0 +1,185 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* HTM AUX tracing support +*/ + +#include <linux/kernel.h> +#include <linux/types.h> +#include <linux/string.h> +#include <linux/zalloc.h> [Severity: Low]
This isn't a bug, but this file usesstrcmp()which is declared in <string.h>. Does this code need to explicitly include <string.h> to avoid relying on implicit inclusion, which might cause build failures in strictly conformant musl libc environments?[ ... ]
Quoted 18 lines
tools/perf/arch/powerpc/util/htm.c new file +18
-0+/* Count htm evsels in the evlist */ +static int htm_nr_events (struct evlist *evlist) +{ +struct evsel *pos; +int n =0 ; + +evlist__for_each_entry (evlist, pos) { +if (!strcmp (evsel__pmu_name (pos),"htm" )) + n++; + } +return n; +} + +static size_t htm_info_priv_size (struct auxtrace_record *itr__maybe_unused , +struct evlist *evlist) +{ +return HTM_AUXTRACE_PRIV_SIZE (htm_nr_events (evlist)); +}[ ... ]
Quoted 47 lines
tools/perf/arch/powerpc/util/htm.c new file +47
-0+/* +* Fill the PERF_RECORD_AUXTRACE_INFO private data with: +* priv[POWERPC_HTM_PMU_TYPE] = pmu->type of the first htm evsel +* priv[POWERPC_HTM_NUM_EVENTS] = total number of HTM CPU entries +* priv[POWERPC_HTM_EVENT_DATA + n*2] = CPU number for nth entry +* priv[POWERPC_HTM_EVENT_DATA + n*2 + 1] = attr.config for nth entry +* +* One entry is written per CPU in each evsel's cpu map. An evsel opened +* with -C 0,1,2 contributes three entries (one per CPU), each carrying +* the same attr.config. The decode side uses event->auxtrace.cpu to look +* up the matching config and derive (node, chip, core) for the output +* file name. +*/ +static int +htm_info_fill (struct auxtrace_record *itr, +struct perf_session *session, +struct perf_record_auxtrace_info *auxtrace_info, +size_t priv_size) +{ + struct htm_recording *htm_r = container_of(itr, struct htm_recording, itr); +struct evlist *evlist = session->evlist; +struct evsel *pos; +int n =0 ; +int expected_n =htm_nr_events (evlist); + +if (priv_size !=HTM_AUXTRACE_PRIV_SIZE (expected_n)) +return -EINVAL ; + + auxtrace_info->type =PERF_AUXTRACE_POWERPC_HTM ; + auxtrace_info->priv[POWERPC_HTM_PMU_TYPE ] = htm_r->evsel->core.attr.type; + auxtrace_info->priv[POWERPC_HTM_NUM_EVENTS ] = expected_n; + +evlist__for_each_entry (evlist, pos) { +struct perf_cpu_map *cpus; +int i, nr; + +if (strcmp (evsel__pmu_name (pos),"htm" )) +continue ; + +/* +* Emit one (cpu, config) entry for every CPU in this evsel's +* map. perf record -C 0,1,2 creates one evsel with a +* three-entry cpu map; each CPU gets its own AUX buffer and +* must be individually mapped so the decoder can match +* event->auxtrace.cpu to the correct (node, chip, core). +*/ + cpus =evsel__cpus (pos);tools/perf/arch/powerpc/util/htm.c line 135
Quoted 10 lines
tools/perf/arch/powerpc/util/htm.c new file +10
-0+ nr = cpus ?perf_cpu_map__nr (cpus) :0 ; + +if (nr >0 ) { +for (i =0 ; i < nr; i++) { +int cpu =perf_cpu_map__cpu (cpus, i).cpu; + + auxtrace_info->priv[POWERPC_HTM_EVENT_DATA + n *2 ] = cpu; + auxtrace_info->priv[POWERPC_HTM_EVENT_DATA + n *2 +1 ] = + pos->core.attr.config; + n++;[Severity: High]
Does this code overflow the auxtrace_info->priv[] array when an HTM evsel covers multiple CPUs?
Quoted 7 lines
The auxtrace_info->priv[] buffer size is calculated by
htm_nr_events()above, which only counts the total number of HTM evsels. If a user runs perf record with a CPU list (e.g., -C 0,1,2), there might be only 1 evsel, resulting in space allocated for a single (cpu, config) pair.However, this loop iterates over the CPU map and writes a pair for every CPU in that evsel (e.g., 3 CPUs), incrementing n past the allocated size.
Thanks for the review
Will address this in next version
Thanks
Athira
Quoted 3 lines
Signature · 2 lines
-- Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
Re: [PATCH V5 3/6] tools/perf: Add arch hook to drain remaining data before event close
In reply to [email protected]Quoted 84 lines · On 7 Aug 2026, at 8:50 PM, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Therecord__final_aux_data()drain loop can spin infinitely during system-wide or attached-process tracing because it is called before disabling events, allowing the active PMU to continuously generate new data. Furthermore, if the main loop exits via early break, events are never explicitly disabled. ----- Patch [3]: [PATCH V5 3/6]
tools/perf: Add arch hook to drain remaining data before event close --- commita89566ddd972102f4f5d6b30845f8dc3daa34fb4
Author: Athira Rajeev <[email protected]>
tools/perf: Add arch hook to drain remaining data before event closeThis commit introduces a mechanism to drain remaining AUX data before closing event file descriptors. It uses a weak architecture hook to check for pending data and retries reading until progress stalls.
Quoted 4 lines
tools/perf/builtin-record.c +0
-0[ ... ]
Quoted 18 lines
tools/perf/builtin-record.c +14
-0@@ -2425,6 +2428,57 @@ static unsigned long record__waking(struct record *rec)return waking; } +/* +* Weak arch hook called by record__final_data(). +* Returns 1 if the arch PMU driver still has records pending (the caller +* will call record__mmap_read_all() and retry), 0 when done. +* Implementations use perf_evsel__read() so this must be called while +* events are still ACTIVE (before evlist__disable()). +*/ +__weak int arch_perf_record__need_read (struct evlist *evlist__maybe_unused ) +{ +return 0 ; +} + +static void record__final_aux_data (struct record *rec) +{[ ... ]
Quoted 13 lines
+ if (
record__bytes_written(rec)==last_bytes_written) {
+ if (++retries >=FINAL_DATA_MAX_RETRIES) {
+ pr_warning("Final AUX data drain made no forward progress after %d retries.\n",tools/perf/builtin-record.c +10
-0+ FINAL_DATA_MAX_RETRIES); +break ; + } +usleep (1000 );/* 1 ms: let AUX ring buffer consumer advance */ + }else { + retries =0 ; +usleep (100 ); + } + } +}[ ... ]
Quoted 4 lines
tools/perf/builtin-record.c +0
-0@@ -2852,11 +2906,22 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)/* * When perf is starting the traced process, at the end events * die with the process and we wait for that. Thus no need to tools/perf/builtin-record.c static int __cmd_record(struct record *rec, int argc, const char **argv) · line 2918
Quoted 10 lines
tools/perf/builtin-record.c +2
-0* disable events in this case. */ if (done && !disabled && !target__none (&opts->target)) {trigger_off (&auxtrace_snapshot_trigger); +if (rec->opts.full_auxtrace) +record__final_aux_data (rec);evlist__disable (rec->evlist); disabled = true; } }[Severity: High]
Could this lead to an infinite loop during system-wide or attached-process tracing?Because
record__final_aux_data()is called beforeevlist__disable(), the events are still active and generating new data.
Quoted 6 lines
If the workload is continuously generating data,
record__bytes_written()will constantly increase, resetting the retry counter to 0 in therecord__final_aux_data()loop we see above.Would it be possible to disable the events first, and then drain the remaining data?
Thanks for the review.
record__final_aux_data() is called before evlist__disable() by design, for two reasons:
- arch_perf_record__need_read() in powerpc callback is using perf_evsel__read() which requires an active event fd. Reading a disabled fd returns 0 immediately, which arch_perf_record__need_read() would interpret as “read complete", and not proceeding to further reads
- The driver needing this callback is expected to handle the event disable or disable of tracing before read.
- In case of powerpc callback which is using arch_perf_record__need_read() , issues an evsel read and the read callback in PMU freezes the hardware trace buffer before dumping it. So its made sure there won't be infinite loop
I will update the comments documenting these details for any future architecture wiring a PMU into this hook.
Quoted 10 lines
tools/perf/builtin-record.c static int __cmd_record(struct record *rec, int argc, const char **argv) · line 2935
Quoted 9 lines
tools/perf/builtin-record.c +8
-0+/* +* If the loop exited without entering the in-loop disable block +* (early break, or child workload where target__none is true and +* the block is never reached), drain any remaining AUX data now. +* Events are still live at this point. +*/ +if ((target__none (&opts->target) || !disabled) && rec->opts.full_auxtrace) +record__final_aux_data (rec);
Quoted 7 lines
[Severity: High]
If the main loop exits via an early break and this fallback block is reached, does this leave the events permanently active?Since
evlist__disable()is never called in this path, we might spin infinitely inrecord__final_aux_data()for the same reason mentioned above, continuously reading newly generated data.
Thanks again for the review
For an early break, example:
if (hits == thread->samples) {
if (done || draining)
break;We reaches to :
/*
* If the loop exited without entering the in-loop disable block
* (early break, or child workload where target__none is true and
* the block is never reached), drain any remaining AUX data now.
* Events are still live at this point.
*/
if ((target__none(&opts->target) || !disabled) && rec->opts.full_auxtrace)
record__final_aux_data(rec);
The reason for having record__final_aux_data without disable is conveyed in above response, But after this , we do call evlist__disable in the out_child and out_child_no_flush
out_child:
evlist__disable(rec->evlist);
record__stop_threads(rec);
record__mmap_read_all(rec, true);
goto out_free_threads;
out_child_no_flush:
/* mmap read already failed — retrying would just fail again */
evlist__disable(rec->evlist);
record__stop_threads(rec);So this won’t leave any event permanently active
Thanks
Athira
Quoted 6 lines
Quoted 3 lines
trigger_off(&auxtrace_snapshot_trigger);trigger_off(&switch_output_trigger);Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
Re: [PATCH V5 5/6] tools/perf: Add powerpc HTM auxtrace event processing support
In reply to [email protected]Quoted 58 lines · On 7 Aug 2026, at 9:03 PM, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [Medium] If multiplePERF_RECORD_AUXTRACE_INFOevents are encountered,session->auxtraceis overwritten without freeing the previously allocated structures, leading to a memory leak. ----- Patch [5]: [PATCH V5 5/6]
tools/perf: Add powerpc HTM auxtrace event processing support --- commitbdf509e26b28cea056c3306c8b70fad62802e9ca
Author: Athira Rajeev <[email protected]>
tools/perf: Add powerpc HTM auxtrace event processing supportAdd the
powerpc-htm.cdecode stub and wire the dispatch inperf_event__process_auxtrace_info()to callpowerpc_htm_process_auxtrace_info().Quoted 5 lines
tools/perf/util/powerpc-htm.c new file +0
-0[ ... ]
Quoted 6 lines
tools/perf/util/powerpc-htm.c new file +6
-0+int powerpc_htm_process_auxtrace_info (union perf_event *event, +struct perf_session *session) +{ +struct perf_record_auxtrace_info *auxtrace_info = &event->auxtrace_info; +struct powerpc_htm *htm; +int err;[ ... ]
Quoted 8 lines
+ htm =
zalloc(sizeof(struct powerpc_htm));
+ if (!htm)
+ return-ENOMEM;tools/perf/util/powerpc-htm.c new file +5
-0+ + err =auxtrace_queues__init (&htm->queues); +if (err) { +free (htm); +return err;tools/perf/util/powerpc-htm.c line 115
Quoted 10 lines
tools/perf/util/powerpc-htm.c new file +10
-0+ } + + htm->session = session; + htm->machine = &session->machines.host; + htm->auxtrace.process_event = powerpc_htm_process_event; + htm->auxtrace.process_auxtrace_event = powerpc_htm_process_auxtrace_event; + htm->auxtrace.flush_events = powerpc_htm_flush; + htm->auxtrace.free_events = powerpc_htm_free_events; + htm->auxtrace.free = powerpc_htm_free; + session->auxtrace = &htm->auxtrace;[Severity: Medium]
This isn't a bug introduced by this patch, but if multiplePERF_RECORD_AUXTRACE_INFOevents are encountered inpowerpc_htm_process_auxtrace_info(), does this overwritesession->auxtracewithout freeing the previously allocated structures?
Quoted 2 lines
Could a maliciously crafted perf.data file containing multiple such events trigger a memory leak here?
Thank you for the review. You are correct that this is a pre-existing pattern, not something introduced by this patch. I will check this and send a separate fix patch to handle this.
Thanks
Athira
Quoted 7 lines
Quoted 4 lines
tools/perf/util/powerpc-htm.c new file +3
-0+ +return 0 ; +}Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
Quoted 11 lines · Athira Rajeev
Overview:
This series adds perf tool support for capturing and processing Hardware Trace Macro (HTM) trace data on POWER systems. HTM exposes raw bus-trace data via theH_HTMhypervisor call. The kernel-side HTM PMU driver (submitted separately) streams this data into perf AUX buffers and emits memory configuration asPERF_SAMPLE_RAWrecords.
Quoted 4 lines
This series wires up the record and report sides in the perf tool so that 'perf record' collects HTM data and 'perf report' writes the trace and memory configuration to files ready for post-processing with htmdecode.
Hi All,
I am currently working on addressing Sashiko's review comments for V5 of the patch series.
Before I proceed with posting V6, requesting for any feedback or suggestions that I should incorporate in V6, apart from Sashiko's comments.
Thanks
Athira
Quoted 127 lines
Patch overview:
Patch 1 refactors the existing powerpc auxtrace dispatch. All VPA-DTL recording logic is moved fromarch/powerpc/util/auxtrace.cinto its own file,arch/powerpc/util/vpa-dtl.c, leavingauxtrace_record__init()as a thin per-PMU dispatcher. This is a prerequisite for adding the HTM recording path cleanly.Patch 2 adds the HTM recording path (
arch/powerpc/util/htm.candutil/powerpc-htm.h).htm_recording_init()sets up theauxtrace_recordcallbacks.htm_info_fill()stores the PMU type and a (cpu, attr.config) pair for each htm evsel into thePERF_RECORD_AUXTRACE_INFOpriv[] area. The decode side reads these back to map each AUX buffer to its (node, chip, core) target usingevent->auxtrace.cpu.PERF_SAMPLE_RAWis also enabled here so that memory configuration records are captured alongside the AUX stream.Patch 3 adds a generic weak
arch_perf_record__need_read()hook inbuiltin-record.c. When the hook returns non-zero, perf record performs an extra mmap-read pass before disabling events. The drain loop runs until the hook returns 0 or no forward progress is made, then the events are disabled and closed normally.Patch 4 implements
arch_perf_record__need_read()for powerpc. It readsevent->countfor all open htm evsels viaperf_evsel__read()and accumulates the values. The kernel driver sets this count to the number of 128-byte HTM trace records written on a successful dump, 1 when the AUX buffer is temporarily full but the hypervisor stream is intact, and 0 once the stream is exhausted or a hard error occurs. A non-zero total causes the recording loop to perform another read pass.Patch 5 adds
PERF_AUXTRACE_POWERPC_HTMto theauxtrace_typeenum and wires the dispatch inperf_event__process_auxtrace_info(). It also amendshtm_info_fill()to setauxtrace_info->typenow that the constant is available.Patch 6 adds the full HTM decode path in
util/powerpc-htm.c.powerpc_htm_process_auxtrace_info()reads the (cpu, config) pairs from priv[] and builds a cpu_configs[] lookup table. As eachPERF_RECORD_AUXTRACEevent arrives,process_auxtrace_event()maps it to the correct target and writes the data to htm.bin.nX.pX.cX.tX immediately.PERF_RECORD_SAMPLERAW records are written to translation.nX.pX.cX.tX byprocess_event(). Per-run first-write tracking (htm_target_seen) ensuresO_TRUNCon the first write andO_APPENDon subsequent writes.Patches not included in V2:
Three patches present in V1 are intentionally dropped from this series and will be submitted as a follow-on once the core infrastructure gets integrated:- htmdecode integration: V1 patch 6 invoked the external htmdecode tool via fork/exec after writing htm.bin.* to decode the raw bus-trace data in-process. This is deferred because the interface between perf and an external decode tool needs more discussion.
- Physical-to-logical address mapping: V1 patch 7 read the current partition ID from
/proc/powerpc/lparcfg, parsed the translation.* memory map entries, and mapped physical addresses in the decoded trace to logical addresses within the LPAR.- Synthetic sample generation: V1 patches 8 and 9 created a
PERF_TYPE_SYNTHevent named "htm", injected synthetic perf samples with the translated logical addresses as sample IP, and wrote a separate .out.l file with logical addresses alongside the decoded trace. The appropriate abstraction for synthetic HTM samples in the perf tool requires further discussion.Usage example:
Collect HTM trace data from a single chip target:# perf record -C 9 -m,256 \ -e htm/nodalchipindex=2,nodeindex=0,htm_type=1/ sleep 3 [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 256.277 MB perf.data ]# perf reportOutput files written by perf report:
htm.bin.n0.p2.c0.t1 raw HTM bus-trace data
translation.n0.p2.c0.t1 memory configuration recordsMultiple targets can be collected simultaneously:
# perf record -m,256 \ -e htm/nodalchipindex=2,nodeindex=0,htm_type=1,cpu=8/ \ -e htm/nodalchipindex=1,nodeindex=0,htm_type=1,cpu=9/ \ -a sleep 10Testing:
Tested on POWER11. Both htm.bin.* and translation.* files are produced correctly for single and dual target collection.Note: Link to kernel patches:
https://lore.kernel.org/linuxppc-dev/[email protected]/T/#tAthira Rajeev (6):
tools/perf: Move powerpc VPA-DTL auxtrace init into a separate filetools/perf: Add AUXTRACE recording support for powerpc HTMtools/perf: Add arch hook to drain remaining data before event closetools/perf: Add powerpc callback support for
arch_perf_record__need_read
tools/perf: Add powerpc HTM auxtrace event processing supporttools/perf: Add perf tool support for processing powerpc HTM AUXTRACE
recordstools/perf/arch/powerpc/util/Build | 2 + tools/perf/arch/powerpc/util/auxtrace.c | 98 ++---- tools/perf/arch/powerpc/util/evsel.c | 77 ++++ tools/perf/arch/powerpc/util/htm.c | 185 ++++++++++ tools/perf/arch/powerpc/util/vpa-dtl.c | 96 +++++ tools/perf/builtin-record.c | 65 ++++ tools/perf/util/Build | 1 + tools/perf/util/auxtrace.c | 4 + tools/perf/util/auxtrace.h | 1 + tools/perf/util/powerpc-htm.c | 447 ++++++++++++++++++++++++ tools/perf/util/powerpc-htm.h | 43 +++ tools/perf/util/powerpc-vpadtl.h | 3 + tools/perf/util/record.h | 4 + 13 files changed, 954 insertions(+), 72 deletions(-)create mode 100644
tools/perf/arch/powerpc/util/htm.ccreate mode 100644tools/perf/arch/powerpc/util/vpa-dtl.ccreate mode 100644tools/perf/util/powerpc-htm.ccreate mode 100644tools/perf/util/powerpc-htm.hSignature · 1 line
2.53.0
Re: [PATCH V5 1/6] tools/perf: Move powerpc VPA-DTL auxtrace init into a separate file
In reply to Athira RajeevQuoted 9 lines · Athira Rajeev
The powerpc auxtrace dispatch lives entirely in
arch/powerpc/util/auxtrace.c. As new PMUs such as HTM are added, this file would grow to contain the recording logic for all of them.Factor out the VPA-DTL recording initializer into its own file,
arch/powerpc/util/vpa-dtl.c, and reduceauxtrace.cto a thin dispatch layer.auxtrace_record__init()now detects the PMU by name and calls the appropriate per-PMU init function:
Quoted 7 lines
-
vpa_dtl_recording_init()for VPA-DTL events (unchanged behaviour)
- Further PMU entries will follow in subsequent patchesThis makes room in
auxtrace_record__init()for the HTM recording path added in the next patch without growing a single monolithic file.Signed-off-by Athira Rajeev <[email protected]>
A couple of cosmetic / very minor comments.
Nevertheless:
Quoted 98 lines
---
Changes in V5:
- Add forward declarations for struct evsel and structauxtrace_recordinpowerpc-vpadtl.hbefore thevpa_dtl_recording_init()prototype. Without them, a translation unit that includes the header before the full definitions are visible may produce implicit-declaration warnings on strict compilers.Changes in V4:
- No changes from V3.Changes in V3:
Add #include <linux/zalloc.h> tovpa-dtl.c; without it the compiler treatszalloc()as implicitly returning int, truncating the upper 32 bits of the returned pointer on 64-bit PowerPC.Changes in V2:
- Renamed the destination file fromarch/powerpc/util/vpa-dtl.c(same name, unchanged) but the subject and commit message are reworded to clearly state that the goal is to makeauxtrace_record__init()a thin per-PMU dispatcher, not merely to "allow multiple PMUs to use auxtrace".
- Handle failure from memory allocation
- Included stdlib andlimits.h
- No functional change to the VPA-DTL path itself.
- Patch is now 1/6 instead of 1/9.tools/perf/arch/powerpc/util/Build | 1 + tools/perf/arch/powerpc/util/auxtrace.c | 84 +++------------------- tools/perf/arch/powerpc/util/vpa-dtl.c | 96 +++++++++++++++++++++++++ tools/perf/util/powerpc-vpadtl.h | 3 + 4 files changed, 108 insertions(+), 76 deletions(-) create mode 100644 tools/perf/arch/powerpc/util/vpa-dtl.ctools/perf/arch/powerpc/util/Build +1
-0@@ -7,3 +7,4 @@ perf-util-y += evsel.o perf-util-$(CONFIG_LIBDW) += skip-callchain-idx.o perf-util-y += auxtrace.o +perf-util-y += vpa-dtl.otools/perf/arch/powerpc/util/auxtrace.c +0
-48@@ -13,63 +13,12 @@#include "../../util/auxtrace.h" #include "../../util/powerpc-vpadtl.h" #include "../../util/record.h" -#include <internal/lib.h> // page_size - -#define KiB (x) ((x) * 1024) - -static int -powerpc_vpadtl_recording_options (struct auxtrace_record *ar__maybe_unused , -struct evlist *evlist__maybe_unused , -struct record_opts *opts) -{ - opts->full_auxtrace = true; - -/* -* Set auxtrace_mmap_pages to minimum -* two pages -*/ -if (!opts->auxtrace_mmap_pages) { - opts->auxtrace_mmap_pages =KiB (128 ) / page_size; -if (opts->mmap_pages ==UINT_MAX ) - opts->mmap_pages =KiB (256 ) / page_size; - } - -return 0 ; -} - -static size_t powerpc_vpadtl_info_priv_size (struct auxtrace_record *itr__maybe_unused , -struct evlist *evlist__maybe_unused ) -{ -return VPADTL_AUXTRACE_PRIV_SIZE ; -} - -static int -powerpc_vpadtl_info_fill (struct auxtrace_record *itr__maybe_unused , -struct perf_session *session__maybe_unused , -struct perf_record_auxtrace_info *auxtrace_info, -size_t priv_size__maybe_unused ) -{ - auxtrace_info->type =PERF_AUXTRACE_VPA_DTL ; - -return 0 ; -} - -static void powerpc_vpadtl_free (struct auxtrace_record *itr) -{ -free (itr); -} - -static u64 powerpc_vpadtl_reference (struct auxtrace_record *itr__maybe_unused ) -{
tools/perf/arch/powerpc/util/auxtrace.c line 21
Quoted 10 lines
tools/perf/arch/powerpc/util/auxtrace.c +1
-4-return 0 ; -}struct auxtrace_record *auxtrace_record__init (struct evlist *evlist,int *err) { -struct auxtrace_record *aux;struct evsel *pos; -int found =0 ; +struct evsel *vpa_dtl_evsel =NULL ;
Ordering local definitions by descending line length is nicer e.g.
struct evsel *vpa_dtl_evsel = NULL;
struct evsel *pos;
Quoted 53 lines
tools/perf/arch/powerpc/util/auxtrace.c +7
-24/* * Set err value to zero here. Any fail later @@ -78,33 +27,16 @@ struct auxtrace_record *auxtrace_record__init(struct evlist *evlist, *err =0 ;evlist__for_each_entry (evlist, pos) { -if (strstarts (pos->name,"vpa_dtl" )) { - found =1 ; +if (pos->name &&strstarts (pos->name,"vpa_dtl" )) { pos->needs_auxtrace_mmap = true; -break ; +/* Remember the first matching VPA DTL event */ +if (!vpa_dtl_evsel) + vpa_dtl_evsel = pos; } } -if (!found) -return NULL ; - -/* -* To obtain the auxtrace buffer file descriptor, the auxtrace event -* must come first. -*/ -evlist__to_front (pos->evlist, pos); - - aux =zalloc (sizeof (*aux)); -if (aux ==NULL ) { -pr_debug ("aux record is NULL\n" ); - *err = -ENOMEM ; -return NULL ; - } +if (vpa_dtl_evsel) +return vpa_dtl_recording_init (vpa_dtl_evsel, err); - aux->recording_options = powerpc_vpadtl_recording_options; - aux->info_priv_size = powerpc_vpadtl_info_priv_size; - aux->info_fill = powerpc_vpadtl_info_fill; - aux->free = powerpc_vpadtl_free; - aux->reference = powerpc_vpadtl_reference; -return aux; +return NULL ; }tools/perf/arch/powerpc/util/vpa-dtl.c new file +3
-0@@ -0,0 +1,96 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* VPA DTL AUX tracing support
tools/perf/arch/powerpc/util/vpa-dtl.c line 13
Quoted 10 lines
tools/perf/arch/powerpc/util/vpa-dtl.c new file +10
-0+*/ + +#include <linux/kernel.h> +#include <linux/types.h> +#include <linux/string.h> +#include <linux/zalloc.h> +#include <errno.h> +#include <stdlib.h> +#include <limits.h> +#include "../../util/cpumap.h"
Is cpumap.h needed?
Quoted 95 lines
tools/perf/arch/powerpc/util/vpa-dtl.c new file +83
-0+#include "../../util/evsel.h" +#include "../../util/evlist.h" +#include "../../util/session.h" +#include "../../util/util.h" +#include "../../util/debug.h" +#include "../../util/auxtrace.h" +#include "../../util/powerpc-vpadtl.h" +#include "../../util/record.h" +#include <internal/lib.h> // page_size + +#define KiB (x) ((x) * 1024) + +static int +powerpc_vpadtl_recording_options (struct auxtrace_record *ar__maybe_unused , +struct evlist *evlist__maybe_unused , +struct record_opts *opts) +{ + opts->full_auxtrace = true; + +/* +* Set auxtrace_mmap_pages to minimum +* two pages +*/ +if (!opts->auxtrace_mmap_pages) { + opts->auxtrace_mmap_pages =KiB (128 ) / page_size; +if (opts->mmap_pages ==UINT_MAX ) + opts->mmap_pages =KiB (256 ) / page_size; + } + +return 0 ; +} + +static size_t powerpc_vpadtl_info_priv_size (struct auxtrace_record *itr__maybe_unused , +struct evlist *evlist__maybe_unused ) +{ +return VPADTL_AUXTRACE_PRIV_SIZE ; +} + +static int +powerpc_vpadtl_info_fill (struct auxtrace_record *itr__maybe_unused , +struct perf_session *session__maybe_unused , +struct perf_record_auxtrace_info *auxtrace_info, +size_t priv_size__maybe_unused ) +{ + auxtrace_info->type =PERF_AUXTRACE_VPA_DTL ; + +return 0 ; +} + +static void powerpc_vpadtl_free (struct auxtrace_record *itr) +{ +free (itr); +} + +static u64 powerpc_vpadtl_reference (struct auxtrace_record *itr__maybe_unused ) +{ +return 0 ; +} + +struct auxtrace_record *vpa_dtl_recording_init (struct evsel *pos,int *err) +{ +struct auxtrace_record *aux; + +/* +* To obtain the auxtrace buffer file descriptor, the auxtrace event +* must come first. +*/ +evlist__to_front (pos->evlist, pos); + + aux =zalloc (sizeof (*aux)); +if (aux ==NULL ) { +pr_debug ("aux record allocation failed (-ENOMEM)\n" ); + *err = -ENOMEM ; +return NULL ; + } + + aux->recording_options = powerpc_vpadtl_recording_options; + aux->info_priv_size = powerpc_vpadtl_info_priv_size; + aux->info_fill = powerpc_vpadtl_info_fill; + aux->free = powerpc_vpadtl_free; + aux->reference = powerpc_vpadtl_reference; +return aux; +}tools/perf/util/powerpc-vpadtl.h +3
-0@@ -20,4 +20,7 @@ struct perf_pmu;int powerpc_vpadtl_process_auxtrace_info (union perf_event *event,struct perf_session *session); +struct evsel ; +struct auxtrace_record ; +struct auxtrace_record *vpa_dtl_recording_init (struct evsel *pos,int *err);#endif
Re: [PATCH V5 2/6] tools/perf: Add AUXTRACE recording support for powerpc HTM
In reply to Athira RajeevQuoted 11 lines · Athira Rajeev
Add powerpc perf tool support for the HTM PMU AUXTRACE recording path.
Introduce
htm_recording_init()and the associated auxtrace callbacks so perf record can createPERF_RECORD_AUXTRACErecords for HTM data.
htm_info_fill()stores the PMU type, number of HTM evsels, and a (cpu, attr.config) pair for each evsel intoPERF_RECORD_AUXTRACE_INFOpriv[]. The decode side reads these back to map each AUX buffer (identified byevent->auxtrace.cpu) to the correct (node, chip, core) target when writing htm.bin.* output files.
Quoted 4 lines
Update
auxtrace_record__init()to detect HTM events and dispatch to the HTM-specific recording initializer.Signed-off-by Athira Rajeev <[email protected]>
A couple of very minor comments.
Nevertheless:
Quoted 101 lines
---
Changes in V5:
-htm_recording_options(): clampaux_watermarkwithmin_t(size_t, wm, UINT_MAX)before assigning topos->core.attr.aux_watermark(u32). On a system with largeauxtrace_mmap_pagesthe unclamped product can exceedUINT_MAXand silently truncate on 32-bit platforms.
-htm_info_fill(): iterate all CPUs in each evsel's CPU map and write one (cpu, config) entry per CPU instead of only the first CPU. An evsel opened with -C 0,1,2 now contributes three priv[] entries, one per CPU, each carrying the same attr.config. The decode side (powerpc_htm_process_auxtrace_event) usesevent->auxtrace.cputo look up the matching entry, so every CPU's AUX buffer is correctly mapped to its (node, chip, core, type) target and written to the right output file. An evsel with no CPU map records cpu=-1 as before.Changes in V4:
- Add explicit mutual-exclusion check inauxtrace_record__init(): if bothvpa_dtl_evselandhtm_evselare set after scanning the evlist, print an error, set *err =-EINVAL, and return NULL. V3 silently fell through to prefer VPA-DTL when both PMU types were present, leaving HTM AUX buffers collected without aPERF_RECORD_AUXTRACE_INFOrecord and the trace undecodable.
- Drop the superfluouspos->name&& null-guard from the HTM detection condition inauxtrace_record__init(): the condition becomes !strcmp(evsel__pmu_name(pos), "htm") without the leading check.evsel__pmu_name()handles a NULL name internally so the guard is redundant.
- Apply the same simplification inhtm_recording_options()andhtm_nr_events()inhtm.c: remove the !pos->name || prefix from theevsel__pmu_name()comparisons, consistent with the dispatcher change above.
- Setpos->core.attr.sample_period= 1 for each HTM evsel insidehtm_recording_options(). V3 set freq=0 andaux_watermarkbut did not setsample_period, leaving the kernel to use whatever default value was in the attr, which could cause the HTM event to fire at an unintended rate.Changes in V3:
- Switch HTM event detection inauxtrace_record__init()fromstrstarts(pos->name, "htm")to !strcmp(evsel__pmu_name(pos), "htm"), matching the kernel-assigned PMU name and preventing false matches on user-named events.
- Switchhtm_recording_options(),htm_nr_events(), and the loop inhtm_info_fill()fromstrstarts(pos->name, "htm")toevsel__pmu_name(), consistent with the above and with patches 4 and 6.
- Add #include <linux/zalloc.h> tohtm.cforzalloc().
- Setauxtrace_info->type=PERF_AUXTRACE_POWERPC_HTMdirectly inhtm_info_fill(), removing the placeholder comment. (V2 deferred this to patch 5; it is cleaner to set it in the same patch that defines the enum.)
- Setaux_watermarkto half the AUX ring-buffer size (auxtrace_mmap_pages*page_size/ 2) so there is always headroom for hardware to write while userspace drains. V2 set it to the full buffer size, risking data loss under back-pressure.
- AddPERF_AUXTRACE_POWERPC_HTMto the enumauxtrace_typeinutil/auxtrace.hand add #include "powerpc-htm.h" plus a stub casePERF_AUXTRACE_POWERPC_HTM: (falls through toPERF_AUXTRACE_UNKNOWN) inutil/auxtrace.cin this patch, so the enum is defined before it is used inhtm_info_fill(). (V2 did this in patch 5.)Changes in V2:
-htm_info_fill()now stores a (cpu, attr.config) pair for every HTM evsel intoPERF_RECORD_AUXTRACE_INFOpriv[]. V1 stored only the PMU type and a single config value; there was no per-CPU mapping.
- The priv[] layout is formalised inutil/powerpc-htm.hwith enum constantsPOWERPC_HTM_PMU_TYPE,POWERPC_HTM_NUM_EVENTS, andPOWERPC_HTM_EVENT_DATAand the helper macrosHTM_AUXTRACE_PRIV_FIXEDandHTM_AUXTRACE_PRIV_SIZE(n). V1 used bare numeric offsets.
-PERF_SAMPLE_RAWis enabled in the recording options so that memory configuration records emitted by the kernel driver are captured alongside the AUX stream. V1 added this in a later patch.
-PERF_AUXTRACE_POWERPC_HTMtype constant is set inhtm_info_fill()(wired up in patch 5 once the enum is defined).
- Patch is now 2/6 instead of 2/9.tools/perf/arch/powerpc/util/Build | 1 + tools/perf/arch/powerpc/util/auxtrace.c | 22 +++ tools/perf/arch/powerpc/util/htm.c | 185 ++++++++++++++++++++++++ tools/perf/util/auxtrace.c | 2 + tools/perf/util/auxtrace.h | 1 + tools/perf/util/powerpc-htm.h | 43 ++++++ 6 files changed, 254 insertions(+) create mode 100644 tools/perf/arch/powerpc/util/htm.c create mode 100644 tools/perf/util/powerpc-htm.htools/perf/arch/powerpc/util/Build +1
-0@@ -8,3 +8,4 @@ perf-util-$(CONFIG_LIBDW) += skip-callchain-idx.o perf-util-y += auxtrace.o perf-util-y += vpa-dtl.o +perf-util-y += htm.otools/perf/arch/powerpc/util/auxtrace.c +0
-0
tools/perf/arch/powerpc/util/auxtrace.c line 17
Quoted 6 lines
tools/perf/arch/powerpc/util/auxtrace.c +2
-0@@ -12,13 +12,16 @@#include "../../util/debug.h" #include "../../util/auxtrace.h" #include "../../util/powerpc-vpadtl.h" +#include "../../util/powerpc-htm.h" #include "../../util/record.h" +#include <string.h>
There is "#include <errno.h>" further up. Maybe nicer to put them together as standard library includes
Quoted 301 lines
tools/perf/arch/powerpc/util/auxtrace.c +20
-0struct auxtrace_record *auxtrace_record__init (struct evlist *evlist,int *err) {struct evsel *pos;struct evsel *vpa_dtl_evsel =NULL ; +struct evsel *htm_evsel =NULL ;/* * Set err value to zero here. Any fail later @@ -32,11 +35,30 @@ struct auxtrace_record *auxtrace_record__init(struct evlist *evlist,/* Remember the first matching VPA DTL event */ if (!vpa_dtl_evsel) vpa_dtl_evsel = pos; + }else if (!strcmp (evsel__pmu_name (pos),"htm" )) { + pos->needs_auxtrace_mmap = true; +/* Remember the first matching HTM event */ +if (!htm_evsel) + htm_evsel = pos; } } +/* +* Only one auxtrace PMU can be initialised per session. Reject +* concurrent VPA DTL and HTM events: HTM AUX buffers would be +* collected without a PERF_RECORD_AUXTRACE_INFO record, making +* the trace undecodable. +*/ +if (vpa_dtl_evsel && htm_evsel) { +pr_err ("Cannot record VPA DTL and HTM auxtrace events simultaneously\n" ); + *err = -EINVAL ; +return NULL ; + } +if (vpa_dtl_evsel)return vpa_dtl_recording_init (vpa_dtl_evsel, err); +else if (htm_evsel) +return htm_recording_init (htm_evsel, err);return NULL ; }tools/perf/arch/powerpc/util/htm.c new file +185
-0@@ -0,0 +1,185 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* HTM AUX tracing support +*/ + +#include <linux/kernel.h> +#include <linux/types.h> +#include <linux/string.h> +#include <linux/zalloc.h> +#include <stdlib.h> +#include <limits.h> +#include "../../util/evsel.h" +#include "../../util/evlist.h" +#include "../../util/session.h" +#include "../../util/debug.h" +#include "../../util/auxtrace.h" +#include "../../util/powerpc-htm.h" +#include "../../util/record.h" +#include <internal/lib.h> /* page_size */ +#include <errno.h> + +#define KiB (x) ((x) * 1024) + +struct htm_recording { +struct auxtrace_record itr; +struct evsel *evsel; +}; + +static int +htm_recording_options (struct auxtrace_record *itr__maybe_unused , +struct evlist *evlist, +struct record_opts *opts) +{ +struct evsel *pos; + + opts->full_auxtrace = true; + +if (!opts->auxtrace_mmap_pages) { + opts->auxtrace_mmap_pages =KiB (128 ) / page_size; +if (opts->mmap_pages ==UINT_MAX ) + opts->mmap_pages =KiB (256 ) / page_size; + } + +evlist__for_each_entry (evlist, pos) { +size_t wm; + +if (strcmp (evsel__pmu_name (pos),"htm" )) +continue ; + wm = opts->auxtrace_mmap_pages * (size_t )page_size /2 ; + pos->core.attr.aux_watermark =min_t (size_t, wm,UINT_MAX ); + pos->core.attr.sample_type |=PERF_SAMPLE_RAW ; + pos->core.attr.freq =0 ; + pos->core.attr.sample_period =1 ; + pos->needs_auxtrace_mmap = true; + } + +return 0 ; +} + +/* Count htm evsels in the evlist */ +static int htm_nr_events (struct evlist *evlist) +{ +struct evsel *pos; +int n =0 ; + +evlist__for_each_entry (evlist, pos) { +if (!strcmp (evsel__pmu_name (pos),"htm" )) + n++; + } +return n; +} + +static size_t htm_info_priv_size (struct auxtrace_record *itr__maybe_unused , +struct evlist *evlist) +{ +return HTM_AUXTRACE_PRIV_SIZE (htm_nr_events (evlist)); +} + +/* +* Fill the PERF_RECORD_AUXTRACE_INFO private data with: +* priv[POWERPC_HTM_PMU_TYPE] = pmu->type of the first htm evsel +* priv[POWERPC_HTM_NUM_EVENTS] = total number of HTM CPU entries +* priv[POWERPC_HTM_EVENT_DATA + n*2] = CPU number for nth entry +* priv[POWERPC_HTM_EVENT_DATA + n*2 + 1] = attr.config for nth entry +* +* One entry is written per CPU in each evsel's cpu map. An evsel opened +* with -C 0,1,2 contributes three entries (one per CPU), each carrying +* the same attr.config. The decode side uses event->auxtrace.cpu to look +* up the matching config and derive (node, chip, core) for the output +* file name. +*/ +static int +htm_info_fill (struct auxtrace_record *itr, +struct perf_session *session, +struct perf_record_auxtrace_info *auxtrace_info, +size_t priv_size) +{ + struct htm_recording *htm_r = container_of(itr, struct htm_recording, itr); +struct evlist *evlist = session->evlist; +struct evsel *pos; +int n =0 ; +int expected_n =htm_nr_events (evlist); + +if (priv_size !=HTM_AUXTRACE_PRIV_SIZE (expected_n)) +return -EINVAL ; + + auxtrace_info->type =PERF_AUXTRACE_POWERPC_HTM ; + auxtrace_info->priv[POWERPC_HTM_PMU_TYPE ] = htm_r->evsel->core.attr.type; + auxtrace_info->priv[POWERPC_HTM_NUM_EVENTS ] = expected_n; + +evlist__for_each_entry (evlist, pos) { +struct perf_cpu_map *cpus; +int i, nr; + +if (strcmp (evsel__pmu_name (pos),"htm" )) +continue ; + +/* +* Emit one (cpu, config) entry for every CPU in this evsel's +* map. perf record -C 0,1,2 creates one evsel with a +* three-entry cpu map; each CPU gets its own AUX buffer and +* must be individually mapped so the decoder can match +* event->auxtrace.cpu to the correct (node, chip, core). +*/ + cpus =evsel__cpus (pos); + nr = cpus ?perf_cpu_map__nr (cpus) :0 ; + +if (nr >0 ) { +for (i =0 ; i < nr; i++) { +int cpu =perf_cpu_map__cpu (cpus, i).cpu; + + auxtrace_info->priv[POWERPC_HTM_EVENT_DATA + n *2 ] = cpu; + auxtrace_info->priv[POWERPC_HTM_EVENT_DATA + n *2 +1 ] = + pos->core.attr.config; + n++; + } + }else { +/* cpu-agnostic evsel: record cpu = -1 */ + auxtrace_info->priv[POWERPC_HTM_EVENT_DATA + n *2 ] = (u64)-1 ; + auxtrace_info->priv[POWERPC_HTM_EVENT_DATA + n *2 +1 ] = + pos->core.attr.config; + n++; + } + } + +return 0 ; +} + +static u64 htm_reference (struct auxtrace_record *itr__maybe_unused ) +{ +return 0 ; +} + +static void htm_free (struct auxtrace_record *itr) +{ + struct htm_recording *htm_r = container_of(itr, struct htm_recording, itr); + +free (htm_r); +} + +struct auxtrace_record *htm_recording_init (struct evsel *pos,int *err) +{ +struct htm_recording *htm_r; + +/* +* To obtain the auxtrace buffer file descriptor, the auxtrace event +* must come first. +*/ +evlist__to_front (pos->evlist, pos); + + htm_r =zalloc (sizeof (*htm_r)); +if (!htm_r) { +pr_debug ("htm_recording allocation failed (-ENOMEM)\n" ); + *err = -ENOMEM ; +return NULL ; + } + + htm_r->evsel = pos; + htm_r->itr.recording_options = htm_recording_options; + htm_r->itr.info_priv_size = htm_info_priv_size; + htm_r->itr.info_fill = htm_info_fill; + htm_r->itr.free = htm_free; + htm_r->itr.reference = htm_reference; +return &htm_r->itr; +}tools/perf/util/auxtrace.c +2
-0@@ -56,6 +56,7 @@#include "s390-cpumsf.h" #include "util/mmap.h" #include "powerpc-vpadtl.h" +#include "powerpc-htm.h" #include <linux/ctype.h> #include "symbol/kallsyms.h" @@ -1432,6 +1433,7 @@ int perf_event__process_auxtrace_info(const struct perf_tool *tool __maybe_unusecase PERF_AUXTRACE_VPA_DTL : err =powerpc_vpadtl_process_auxtrace_info (event, session);break ; +case PERF_AUXTRACE_POWERPC_HTM :case PERF_AUXTRACE_UNKNOWN :default :return -EINVAL ;tools/perf/util/auxtrace.h +1
-0@@ -46,6 +46,7 @@ enum auxtrace_type {PERF_AUXTRACE_S390_CPUMSF ,PERF_AUXTRACE_HISI_PTT ,PERF_AUXTRACE_VPA_DTL , + PERF_AUXTRACE_POWERPC_HTM, }; enum itrace_period_type {tools/perf/util/powerpc-htm.h new file +31
-0@@ -0,0 +1,43 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __POWERPC_HTM_H +#define __POWERPC_HTM_H + +#include <linux/types.h> + +/* +* Layout of the private data in PERF_RECORD_AUXTRACE_INFO for HTM. +* +* priv[POWERPC_HTM_PMU_TYPE] = htm PMU type ID (pmu->type from kernel) +* priv[POWERPC_HTM_NUM_EVENTS] = number of htm evsels recorded (N) +* +* Followed by N pairs (2 u64 each): +* priv[POWERPC_HTM_EVENT_DATA + n*2 + 0] = CPU number for nth htm evsel +* priv[POWERPC_HTM_EVENT_DATA + n*2 + 1] = attr.config for nth htm evsel +* +* Total priv entries: POWERPC_HTM_EVENT_DATA + N * 2 +*/ +enum { +POWERPC_HTM_PMU_TYPE =0 , +POWERPC_HTM_NUM_EVENTS , +POWERPC_HTM_EVENT_DATA ,/* variable-length: 2 u64 per event */ +}; + +/* Fixed header size (everything before the per-event data) */ +#define HTM_AUXTRACE_PRIV_FIXED (POWERPC_HTM_EVENT_DATA * sizeof(u64)) + +/* Total priv size for N htm evsels */ +#define HTM_AUXTRACE_PRIV_SIZE (n) \ + (HTM_AUXTRACE_PRIV_FIXED + (n) * 2 * sizeof(u64)) +
tools/perf/util/powerpc-htm.h line 41
Quoted 10 lines
tools/perf/util/powerpc-htm.h new file +10
-0+struct evsel ; +struct evlist ; +union perf_event ; +struct perf_session ; +struct auxtrace_record ; + +struct auxtrace_record *htm_recording_init (struct evsel *pos,int *err); + +int powerpc_htm_process_auxtrace_info (union perf_event *event, +struct perf_session *session);
Ideally, should only add a declaration for powerpc_htm_process_auxtrace_info() when it actually exists
Quoted 2 lines
tools/perf/util/powerpc-htm.h new file +2
-0+ +#endif /* __POWERPC_HTM_H */
Re: [PATCH V5 2/6] tools/perf: Add AUXTRACE recording support for powerpc HTM
Quoted 19 lines · Adrian Hunter
Quoted 14 lines · Athira Rajeev
Add powerpc perf tool support for the HTM PMU AUXTRACE recording path.
Introduce
htm_recording_init()and the associated auxtrace callbacks so perf record can createPERF_RECORD_AUXTRACErecords for HTM data.
htm_info_fill()stores the PMU type, number of HTM evsels, and a (cpu, attr.config) pair for each evsel intoPERF_RECORD_AUXTRACE_INFOpriv[]. The decode side reads these back to map each AUX buffer (identified byevent->auxtrace.cpu) to the correct (node, chip, core) target when writing htm.bin.* output files.Update
auxtrace_record__init()to detect HTM events and dispatch to the HTM-specific recording initializer.
Quoted 7 lines
Quoted 1 line
Signed-off-by Athira Rajeev <[email protected]>A couple of very minor comments.
Nevertheless:
Reviewed-by Adrian Hunter <[email protected]>
Hi Adrian
Thanks for reviewing and sharing comments
I will include changes in the next version.
I will wait for feedback on the remaining patches and post a v6 incorporating all the changes together.
Thanks
Athira
Quoted 429 lines
Quoted 102 lines
---
Changes in V5:
-htm_recording_options(): clampaux_watermarkwithmin_t(size_t, wm, UINT_MAX)before assigning topos->core.attr.aux_watermark(u32). On a system with largeauxtrace_mmap_pagesthe unclamped product can exceedUINT_MAXand silently truncate on 32-bit platforms.
-htm_info_fill(): iterate all CPUs in each evsel's CPU map and write one (cpu, config) entry per CPU instead of only the first CPU. An evsel opened with -C 0,1,2 now contributes three priv[] entries, one per CPU, each carrying the same attr.config. The decode side (powerpc_htm_process_auxtrace_event) usesevent->auxtrace.cputo look up the matching entry, so every CPU's AUX buffer is correctly mapped to its (node, chip, core, type) target and written to the right output file. An evsel with no CPU map records cpu=-1 as before.Changes in V4:
- Add explicit mutual-exclusion check inauxtrace_record__init(): if bothvpa_dtl_evselandhtm_evselare set after scanning the evlist, print an error, set *err =-EINVAL, and return NULL. V3 silently fell through to prefer VPA-DTL when both PMU types were present, leaving HTM AUX buffers collected without aPERF_RECORD_AUXTRACE_INFOrecord and the trace undecodable.
- Drop the superfluouspos->name&& null-guard from the HTM detection condition inauxtrace_record__init(): the condition becomes !strcmp(evsel__pmu_name(pos), "htm") without the leading check.evsel__pmu_name()handles a NULL name internally so the guard is redundant.
- Apply the same simplification inhtm_recording_options()andhtm_nr_events()inhtm.c: remove the !pos->name || prefix from theevsel__pmu_name()comparisons, consistent with the dispatcher change above.
- Setpos->core.attr.sample_period= 1 for each HTM evsel insidehtm_recording_options(). V3 set freq=0 andaux_watermarkbut did not setsample_period, leaving the kernel to use whatever default value was in the attr, which could cause the HTM event to fire at an unintended rate.Changes in V3:
- Switch HTM event detection inauxtrace_record__init()fromstrstarts(pos->name, "htm")to !strcmp(evsel__pmu_name(pos), "htm"), matching the kernel-assigned PMU name and preventing false matches on user-named events.
- Switchhtm_recording_options(),htm_nr_events(), and the loop inhtm_info_fill()fromstrstarts(pos->name, "htm")toevsel__pmu_name(), consistent with the above and with patches 4 and 6.
- Add #include <linux/zalloc.h> tohtm.cforzalloc().
- Setauxtrace_info->type=PERF_AUXTRACE_POWERPC_HTMdirectly inhtm_info_fill(), removing the placeholder comment. (V2 deferred this to patch 5; it is cleaner to set it in the same patch that defines the enum.)
- Setaux_watermarkto half the AUX ring-buffer size (auxtrace_mmap_pages*page_size/ 2) so there is always headroom for hardware to write while userspace drains. V2 set it to the full buffer size, risking data loss under back-pressure.
- AddPERF_AUXTRACE_POWERPC_HTMto the enumauxtrace_typeinutil/auxtrace.hand add #include "powerpc-htm.h" plus a stub casePERF_AUXTRACE_POWERPC_HTM: (falls through toPERF_AUXTRACE_UNKNOWN) inutil/auxtrace.cin this patch, so the enum is defined before it is used inhtm_info_fill(). (V2 did this in patch 5.)Changes in V2:
-htm_info_fill()now stores a (cpu, attr.config) pair for every HTM evsel intoPERF_RECORD_AUXTRACE_INFOpriv[]. V1 stored only the PMU type and a single config value; there was no per-CPU mapping.
- The priv[] layout is formalised inutil/powerpc-htm.hwith enum constantsPOWERPC_HTM_PMU_TYPE,POWERPC_HTM_NUM_EVENTS, andPOWERPC_HTM_EVENT_DATAand the helper macrosHTM_AUXTRACE_PRIV_FIXEDandHTM_AUXTRACE_PRIV_SIZE(n). V1 used bare numeric offsets.
-PERF_SAMPLE_RAWis enabled in the recording options so that memory configuration records emitted by the kernel driver are captured alongside the AUX stream. V1 added this in a later patch.
-PERF_AUXTRACE_POWERPC_HTMtype constant is set inhtm_info_fill()(wired up in patch 5 once the enum is defined).
- Patch is now 2/6 instead of 2/9.tools/perf/arch/powerpc/util/Build | 1 + tools/perf/arch/powerpc/util/auxtrace.c | 22 +++ tools/perf/arch/powerpc/util/htm.c | 185 ++++++++++++++++++++++++ tools/perf/util/auxtrace.c | 2 + tools/perf/util/auxtrace.h | 1 + tools/perf/util/powerpc-htm.h | 43 ++++++ 6 files changed, 254 insertions(+)create mode 100644
tools/perf/arch/powerpc/util/htm.ccreate mode 100644tools/perf/util/powerpc-htm.htools/perf/arch/powerpc/util/Build +0
-0@@ -8,3 +8,4 @@ perf-util-$(CONFIG_LIBDW) += skip-callchain-idx.operf-util-y += auxtrace.o
perf-util-y += vpa-dtl.otools/perf/arch/powerpc/util/Build +1
-0+perf-util-y += htm.otools/perf/arch/powerpc/util/auxtrace.c +0
-0tools/perf/arch/powerpc/util/auxtrace.c line 17
Quoted 6 lines
tools/perf/arch/powerpc/util/auxtrace.c +2
-0@@ -12,13 +12,16 @@#include "../../util/debug.h" #include "../../util/auxtrace.h" #include "../../util/powerpc-vpadtl.h" +#include "../../util/powerpc-htm.h" #include "../../util/record.h" +#include <string.h> There is "#include <
errno.h>" further up. Maybe nicer to put them together as standard library includesQuoted 301 lines
tools/perf/arch/powerpc/util/auxtrace.c +20
-0struct auxtrace_record *auxtrace_record__init (struct evlist *evlist,int *err) {struct evsel *pos;struct evsel *vpa_dtl_evsel =NULL ; +struct evsel *htm_evsel =NULL ;/* * Set err value to zero here. Any fail later @@ -32,11 +35,30 @@ struct auxtrace_record *auxtrace_record__init(struct evlist *evlist,/* Remember the first matching VPA DTL event */ if (!vpa_dtl_evsel) vpa_dtl_evsel = pos; + }else if (!strcmp (evsel__pmu_name (pos),"htm" )) { + pos->needs_auxtrace_mmap = true; +/* Remember the first matching HTM event */ +if (!htm_evsel) + htm_evsel = pos; } } +/* +* Only one auxtrace PMU can be initialised per session. Reject +* concurrent VPA DTL and HTM events: HTM AUX buffers would be +* collected without a PERF_RECORD_AUXTRACE_INFO record, making +* the trace undecodable. +*/ +if (vpa_dtl_evsel && htm_evsel) { +pr_err ("Cannot record VPA DTL and HTM auxtrace events simultaneously\n" ); + *err = -EINVAL ; +return NULL ; + } +if (vpa_dtl_evsel)return vpa_dtl_recording_init (vpa_dtl_evsel, err); +else if (htm_evsel) +return htm_recording_init (htm_evsel, err);return NULL ; }tools/perf/arch/powerpc/util/htm.c new file +185
-0@@ -0,0 +1,185 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* HTM AUX tracing support +*/ + +#include <linux/kernel.h> +#include <linux/types.h> +#include <linux/string.h> +#include <linux/zalloc.h> +#include <stdlib.h> +#include <limits.h> +#include "../../util/evsel.h" +#include "../../util/evlist.h" +#include "../../util/session.h" +#include "../../util/debug.h" +#include "../../util/auxtrace.h" +#include "../../util/powerpc-htm.h" +#include "../../util/record.h" +#include <internal/lib.h> /* page_size */ +#include <errno.h> + +#define KiB (x) ((x) * 1024) + +struct htm_recording { +struct auxtrace_record itr; +struct evsel *evsel; +}; + +static int +htm_recording_options (struct auxtrace_record *itr__maybe_unused , +struct evlist *evlist, +struct record_opts *opts) +{ +struct evsel *pos; + + opts->full_auxtrace = true; + +if (!opts->auxtrace_mmap_pages) { + opts->auxtrace_mmap_pages =KiB (128 ) / page_size; +if (opts->mmap_pages ==UINT_MAX ) + opts->mmap_pages =KiB (256 ) / page_size; + } + +evlist__for_each_entry (evlist, pos) { +size_t wm; + +if (strcmp (evsel__pmu_name (pos),"htm" )) +continue ; + wm = opts->auxtrace_mmap_pages * (size_t )page_size /2 ; + pos->core.attr.aux_watermark =min_t (size_t, wm,UINT_MAX ); + pos->core.attr.sample_type |=PERF_SAMPLE_RAW ; + pos->core.attr.freq =0 ; + pos->core.attr.sample_period =1 ; + pos->needs_auxtrace_mmap = true; + } + +return 0 ; +} + +/* Count htm evsels in the evlist */ +static int htm_nr_events (struct evlist *evlist) +{ +struct evsel *pos; +int n =0 ; + +evlist__for_each_entry (evlist, pos) { +if (!strcmp (evsel__pmu_name (pos),"htm" )) + n++; + } +return n; +} + +static size_t htm_info_priv_size (struct auxtrace_record *itr__maybe_unused , +struct evlist *evlist) +{ +return HTM_AUXTRACE_PRIV_SIZE (htm_nr_events (evlist)); +} + +/* +* Fill the PERF_RECORD_AUXTRACE_INFO private data with: +* priv[POWERPC_HTM_PMU_TYPE] = pmu->type of the first htm evsel +* priv[POWERPC_HTM_NUM_EVENTS] = total number of HTM CPU entries +* priv[POWERPC_HTM_EVENT_DATA + n*2] = CPU number for nth entry +* priv[POWERPC_HTM_EVENT_DATA + n*2 + 1] = attr.config for nth entry +* +* One entry is written per CPU in each evsel's cpu map. An evsel opened +* with -C 0,1,2 contributes three entries (one per CPU), each carrying +* the same attr.config. The decode side uses event->auxtrace.cpu to look +* up the matching config and derive (node, chip, core) for the output +* file name. +*/ +static int +htm_info_fill (struct auxtrace_record *itr, +struct perf_session *session, +struct perf_record_auxtrace_info *auxtrace_info, +size_t priv_size) +{ + struct htm_recording *htm_r = container_of(itr, struct htm_recording, itr); +struct evlist *evlist = session->evlist; +struct evsel *pos; +int n =0 ; +int expected_n =htm_nr_events (evlist); + +if (priv_size !=HTM_AUXTRACE_PRIV_SIZE (expected_n)) +return -EINVAL ; + + auxtrace_info->type =PERF_AUXTRACE_POWERPC_HTM ; + auxtrace_info->priv[POWERPC_HTM_PMU_TYPE ] = htm_r->evsel->core.attr.type; + auxtrace_info->priv[POWERPC_HTM_NUM_EVENTS ] = expected_n; + +evlist__for_each_entry (evlist, pos) { +struct perf_cpu_map *cpus; +int i, nr; + +if (strcmp (evsel__pmu_name (pos),"htm" )) +continue ; + +/* +* Emit one (cpu, config) entry for every CPU in this evsel's +* map. perf record -C 0,1,2 creates one evsel with a +* three-entry cpu map; each CPU gets its own AUX buffer and +* must be individually mapped so the decoder can match +* event->auxtrace.cpu to the correct (node, chip, core). +*/ + cpus =evsel__cpus (pos); + nr = cpus ?perf_cpu_map__nr (cpus) :0 ; + +if (nr >0 ) { +for (i =0 ; i < nr; i++) { +int cpu =perf_cpu_map__cpu (cpus, i).cpu; + + auxtrace_info->priv[POWERPC_HTM_EVENT_DATA + n *2 ] = cpu; + auxtrace_info->priv[POWERPC_HTM_EVENT_DATA + n *2 +1 ] = + pos->core.attr.config; + n++; + } + }else { +/* cpu-agnostic evsel: record cpu = -1 */ + auxtrace_info->priv[POWERPC_HTM_EVENT_DATA + n *2 ] = (u64)-1 ; + auxtrace_info->priv[POWERPC_HTM_EVENT_DATA + n *2 +1 ] = + pos->core.attr.config; + n++; + } + } + +return 0 ; +} + +static u64 htm_reference (struct auxtrace_record *itr__maybe_unused ) +{ +return 0 ; +} + +static void htm_free (struct auxtrace_record *itr) +{ + struct htm_recording *htm_r = container_of(itr, struct htm_recording, itr); + +free (htm_r); +} + +struct auxtrace_record *htm_recording_init (struct evsel *pos,int *err) +{ +struct htm_recording *htm_r; + +/* +* To obtain the auxtrace buffer file descriptor, the auxtrace event +* must come first. +*/ +evlist__to_front (pos->evlist, pos); + + htm_r =zalloc (sizeof (*htm_r)); +if (!htm_r) { +pr_debug ("htm_recording allocation failed (-ENOMEM)\n" ); + *err = -ENOMEM ; +return NULL ; + } + + htm_r->evsel = pos; + htm_r->itr.recording_options = htm_recording_options; + htm_r->itr.info_priv_size = htm_info_priv_size; + htm_r->itr.info_fill = htm_info_fill; + htm_r->itr.free = htm_free; + htm_r->itr.reference = htm_reference; +return &htm_r->itr; +}tools/perf/util/auxtrace.c +1
-0@@ -56,6 +56,7 @@#include "s390-cpumsf.h" #include "util/mmap.h" #include "powerpc-vpadtl.h" +#include "powerpc-htm.h" #include <linux/ctype.h> #include "symbol/kallsyms.h" @@ -1432,6 +1433,7 @@ int perf_event__process_auxtrace_info(const struct perf_tool *tool __maybe_unusecase
PERF_AUXTRACE_VPA_DTL:
err =powerpc_vpadtl_process_auxtrace_info(event, session); break;
+ casePERF_AUXTRACE_POWERPC_HTM:
casePERF_AUXTRACE_UNKNOWN:
default:
return-EINVAL;tools/perf/util/auxtrace.h +1
-0@@ -46,6 +46,7 @@ enum auxtrace_type {PERF_AUXTRACE_S390_CPUMSF ,PERF_AUXTRACE_HISI_PTT ,PERF_AUXTRACE_VPA_DTL , + PERF_AUXTRACE_POWERPC_HTM, }; enum itrace_period_type {tools/perf/util/powerpc-htm.h new file +31
-0@@ -0,0 +1,43 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __POWERPC_HTM_H +#define __POWERPC_HTM_H + +#include <linux/types.h> + +/* +* Layout of the private data in PERF_RECORD_AUXTRACE_INFO for HTM. +* +* priv[POWERPC_HTM_PMU_TYPE] = htm PMU type ID (pmu->type from kernel) +* priv[POWERPC_HTM_NUM_EVENTS] = number of htm evsels recorded (N) +* +* Followed by N pairs (2 u64 each): +* priv[POWERPC_HTM_EVENT_DATA + n*2 + 0] = CPU number for nth htm evsel +* priv[POWERPC_HTM_EVENT_DATA + n*2 + 1] = attr.config for nth htm evsel +* +* Total priv entries: POWERPC_HTM_EVENT_DATA + N * 2 +*/ +enum { +POWERPC_HTM_PMU_TYPE =0 , +POWERPC_HTM_NUM_EVENTS , +POWERPC_HTM_EVENT_DATA ,/* variable-length: 2 u64 per event */ +}; + +/* Fixed header size (everything before the per-event data) */ +#define HTM_AUXTRACE_PRIV_FIXED (POWERPC_HTM_EVENT_DATA * sizeof(u64)) + +/* Total priv size for N htm evsels */ +#define HTM_AUXTRACE_PRIV_SIZE (n) \ + (HTM_AUXTRACE_PRIV_FIXED + (n) * 2 * sizeof(u64)) +tools/perf/util/powerpc-htm.h line 41
Quoted 10 lines
tools/perf/util/powerpc-htm.h new file +10
-0+struct evsel ; +struct evlist ; +union perf_event ; +struct perf_session ; +struct auxtrace_record ; + +struct auxtrace_record *htm_recording_init (struct evsel *pos,int *err); + +int powerpc_htm_process_auxtrace_info (union perf_event *event, +struct perf_session *session);Ideally, should only add a declaration for
powerpc_htm_process_auxtrace_info()when it actually existsQuoted 2 lines
tools/perf/util/powerpc-htm.h new file +2
-0+ +#endif /* __POWERPC_HTM_H */
Re: [PATCH V5 1/6] tools/perf: Move powerpc VPA-DTL auxtrace init into a separate file
In reply to Adrian HunterQuoted 139 lines · Adrian Hunter
Quoted 9 lines · Athira Rajeev
The powerpc auxtrace dispatch lives entirely in
arch/powerpc/util/auxtrace.c. As new PMUs such as HTM are added, this file would grow to contain the recording logic for all of them.Factor out the VPA-DTL recording initializer into its own file,
arch/powerpc/util/vpa-dtl.c, and reduceauxtrace.cto a thin dispatch layer.auxtrace_record__init()now detects the PMU by name and calls the appropriate per-PMU init function:Quoted 7 lines
-
vpa_dtl_recording_init()for VPA-DTL events (unchanged behaviour)
- Further PMU entries will follow in subsequent patchesThis makes room in
auxtrace_record__init()for the HTM recording path added in the next patch without growing a single monolithic file.Signed-off-by Athira Rajeev <[email protected]>A couple of cosmetic / very minor comments.
Nevertheless:
Reviewed-by Adrian Hunter <[email protected]>Quoted 98 lines
---
Changes in V5:
- Add forward declarations for struct evsel and structauxtrace_recordinpowerpc-vpadtl.hbefore thevpa_dtl_recording_init()prototype. Without them, a translation unit that includes the header before the full definitions are visible may produce implicit-declaration warnings on strict compilers.Changes in V4:
- No changes from V3.Changes in V3:
Add #include <linux/zalloc.h> tovpa-dtl.c; without it the compiler treatszalloc()as implicitly returning int, truncating the upper 32 bits of the returned pointer on 64-bit PowerPC.Changes in V2:
- Renamed the destination file fromarch/powerpc/util/vpa-dtl.c(same name, unchanged) but the subject and commit message are reworded to clearly state that the goal is to makeauxtrace_record__init()a thin per-PMU dispatcher, not merely to "allow multiple PMUs to use auxtrace".
- Handle failure from memory allocation
- Included stdlib andlimits.h
- No functional change to the VPA-DTL path itself.
- Patch is now 1/6 instead of 1/9.tools/perf/arch/powerpc/util/Build | 1 + tools/perf/arch/powerpc/util/auxtrace.c | 84 +++------------------- tools/perf/arch/powerpc/util/vpa-dtl.c | 96 +++++++++++++++++++++++++ tools/perf/util/powerpc-vpadtl.h | 3 + 4 files changed, 108 insertions(+), 76 deletions(-)create mode 100644
tools/perf/arch/powerpc/util/vpa-dtl.ctools/perf/arch/powerpc/util/Build +0
-0@@ -7,3 +7,4 @@ perf-util-y += evsel.operf-util-$(CONFIG_LIBDW) += skip-callchain-idx.o
perf-util-y += auxtrace.o
tools/perf/arch/powerpc/util/Build +1
-0+perf-util-y += vpa-dtl.otools/perf/arch/powerpc/util/auxtrace.c +0
-48@@ -13,63 +13,12 @@#include "../../util/auxtrace.h" #include "../../util/powerpc-vpadtl.h" #include "../../util/record.h" -#include <internal/lib.h> // page_size - -#define KiB (x) ((x) * 1024) - -static int -powerpc_vpadtl_recording_options (struct auxtrace_record *ar__maybe_unused , -struct evlist *evlist__maybe_unused , -struct record_opts *opts) -{ - opts->full_auxtrace = true; - -/* -* Set auxtrace_mmap_pages to minimum -* two pages -*/ -if (!opts->auxtrace_mmap_pages) { - opts->auxtrace_mmap_pages =KiB (128 ) / page_size; -if (opts->mmap_pages ==UINT_MAX ) - opts->mmap_pages =KiB (256 ) / page_size; - } - -return 0 ; -} - -static size_t powerpc_vpadtl_info_priv_size (struct auxtrace_record *itr__maybe_unused , -struct evlist *evlist__maybe_unused ) -{ -return VPADTL_AUXTRACE_PRIV_SIZE ; -} - -static int -powerpc_vpadtl_info_fill (struct auxtrace_record *itr__maybe_unused , -struct perf_session *session__maybe_unused , -struct perf_record_auxtrace_info *auxtrace_info, -size_t priv_size__maybe_unused ) -{ - auxtrace_info->type =PERF_AUXTRACE_VPA_DTL ; - -return 0 ; -} - -static void powerpc_vpadtl_free (struct auxtrace_record *itr) -{ -free (itr); -} - -static u64 powerpc_vpadtl_reference (struct auxtrace_record *itr__maybe_unused ) -{tools/perf/arch/powerpc/util/auxtrace.c line 21
Quoted 10 lines
tools/perf/arch/powerpc/util/auxtrace.c +1
-4-return 0 ; -}struct auxtrace_record *auxtrace_record__init (struct evlist *evlist,int *err) { -struct auxtrace_record *aux;struct evsel *pos; -int found =0 ; +struct evsel *vpa_dtl_evsel =NULL ;Ordering local definitions by descending line length is nicer e.g.
Quoted 2 lines
struct evsel
*vpa_dtl_evsel= NULL;
struct evsel *pos;
Sure, I will address this in next version
Thanks for review Adrian.
I will wait for feedback on the remaining patches and post a v6 incorporating all the changes together.
Athira
Quoted 58 lines
Quoted 58 lines
tools/perf/arch/powerpc/util/auxtrace.c +7
-24/* * Set err value to zero here. Any fail later @@ -78,33 +27,16 @@ struct auxtrace_record *auxtrace_record__init(struct evlist *evlist, *err =0 ; evlist__for_each_entry(evlist, pos) { -if (strstarts (pos->name,"vpa_dtl" )) { - found =1 ; +if (pos->name &&strstarts (pos->name,"vpa_dtl" )) { pos->needs_auxtrace_mmap = true; -break ; +/* Remember the first matching VPA DTL event */ +if (!vpa_dtl_evsel) + vpa_dtl_evsel = pos; } } -if (!found) -return NULL ; - -/* -* To obtain the auxtrace buffer file descriptor, the auxtrace event -* must come first. -*/ -evlist__to_front (pos->evlist, pos); - - aux =zalloc (sizeof (*aux)); -if (aux ==NULL ) { -pr_debug ("aux record is NULL\n" ); - *err = -ENOMEM ; -return NULL ; - } +if (vpa_dtl_evsel) +return vpa_dtl_recording_init (vpa_dtl_evsel, err); - aux->recording_options = powerpc_vpadtl_recording_options; - aux->info_priv_size = powerpc_vpadtl_info_priv_size; - aux->info_fill = powerpc_vpadtl_info_fill; - aux->free = powerpc_vpadtl_free; - aux->reference = powerpc_vpadtl_reference; -return aux; +return NULL ; }tools/perf/arch/powerpc/util/vpa-dtl.c new file +7
-0@@ -0,0 +1,96 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* VPA DTL AUX tracing support +*/ + +#include <linux/kernel.h> +#include <linux/types.h>
Quoted 8 lines
tools/perf/arch/powerpc/util/vpa-dtl.c line 13
Quoted 6 lines
tools/perf/arch/powerpc/util/vpa-dtl.c new file +6
-0+#include <linux/string.h> +#include <linux/zalloc.h> +#include <errno.h> +#include <stdlib.h> +#include <limits.h> +#include "../../util/cpumap.h" Is
cpumap.hneeded?
Ok, I will check this
Quoted 96 lines
Quoted 96 lines
tools/perf/arch/powerpc/util/vpa-dtl.c new file +83
-0+#include "../../util/evsel.h" +#include "../../util/evlist.h" +#include "../../util/session.h" +#include "../../util/util.h" +#include "../../util/debug.h" +#include "../../util/auxtrace.h" +#include "../../util/powerpc-vpadtl.h" +#include "../../util/record.h" +#include <internal/lib.h> // page_size + +#define KiB (x) ((x) * 1024) + +static int +powerpc_vpadtl_recording_options (struct auxtrace_record *ar__maybe_unused , +struct evlist *evlist__maybe_unused , +struct record_opts *opts) +{ + opts->full_auxtrace = true; + +/* +* Set auxtrace_mmap_pages to minimum +* two pages +*/ +if (!opts->auxtrace_mmap_pages) { + opts->auxtrace_mmap_pages =KiB (128 ) / page_size; +if (opts->mmap_pages ==UINT_MAX ) + opts->mmap_pages =KiB (256 ) / page_size; + } + +return 0 ; +} + +static size_t powerpc_vpadtl_info_priv_size (struct auxtrace_record *itr__maybe_unused , +struct evlist *evlist__maybe_unused ) +{ +return VPADTL_AUXTRACE_PRIV_SIZE ; +} + +static int +powerpc_vpadtl_info_fill (struct auxtrace_record *itr__maybe_unused , +struct perf_session *session__maybe_unused , +struct perf_record_auxtrace_info *auxtrace_info, +size_t priv_size__maybe_unused ) +{ + auxtrace_info->type =PERF_AUXTRACE_VPA_DTL ; + +return 0 ; +} + +static void powerpc_vpadtl_free (struct auxtrace_record *itr) +{ +free (itr); +} + +static u64 powerpc_vpadtl_reference (struct auxtrace_record *itr__maybe_unused ) +{ +return 0 ; +} + +struct auxtrace_record *vpa_dtl_recording_init (struct evsel *pos,int *err) +{ +struct auxtrace_record *aux; + +/* +* To obtain the auxtrace buffer file descriptor, the auxtrace event +* must come first. +*/ +evlist__to_front (pos->evlist, pos); + + aux =zalloc (sizeof (*aux)); +if (aux ==NULL ) { +pr_debug ("aux record allocation failed (-ENOMEM)\n" ); + *err = -ENOMEM ; +return NULL ; + } + + aux->recording_options = powerpc_vpadtl_recording_options; + aux->info_priv_size = powerpc_vpadtl_info_priv_size; + aux->info_fill = powerpc_vpadtl_info_fill; + aux->free = powerpc_vpadtl_free; + aux->reference = powerpc_vpadtl_reference; +return aux; +}tools/perf/util/powerpc-vpadtl.h +3
-0@@ -20,4 +20,7 @@ struct perf_pmu;int powerpc_vpadtl_process_auxtrace_info (union perf_event *event,struct perf_session *session); +struct evsel ; +struct auxtrace_record ; +struct auxtrace_record *vpa_dtl_recording_init (struct evsel *pos,int *err);#endif
Adrian Hunter
reviewerRe: [PATCH V5 4/6] tools/perf: Add powerpc callback support for arch_perf_record__need_read
In reply to Athira RajeevQuoted 17 lines · Athira Rajeev
Implement the
arch_perf_record__need_read()architecture-specific hook for powerpc inarch/powerpc/util/evsel.c.The HTM kernel driver sets
event->countto the number of records still staged in its internal buffers (total_size/record_size), and to 0 once the stream is exhausted. This hook reads that count for every open htm evsel viaperf_evsel__read()and accumulates the values intototal_pending_records. A non-zero total means at least one HTM target still has records pending; the recording loop added in the previous patch will perform another mmap-read pass.The drain uses a two-layer safety check:
event->countdetects records staged by the driver, andrecord__bytes_written()in the drain loop confirms data was actually moved into perf.data. This combination handles the case where the driver count is briefly stale while hardware is still flushing.
Quoted 7 lines
The implementation scans the evlist using
evsel__pmu_name()to identify HTM events by their kernel-assigned PMU name rather than the user-visible event name, preventing false matches. It iterates the fd/ sample-id xyarray, and skips any evsel whose fd and sample-id arrays are mismatched to avoid reading stale state. When the accumulated record count reaches zero the hook returns 0 and the recording loop proceeds to disable and close the events.
This looks like the proposed driver:
https://lore.kernel.org/all/[email protected]/
breaks the perf ABI.
I am not going to review any more tools patches for now.
Quoted 139 lines
Signed-off-by Athira Rajeev <[email protected]>---
Changes in V5:
- When an HTM evsel is a group sibling (evsel->core.leader!=&evsel->core), read through its group leader's structperf_evselinstead of the sibling directly.perf_evsel__read_size()usesevsel->nr_membersto compute the read buffer size;nr_membersis 0 for siblings, so size=0 is passed toreadn(), which returns <=0 and leaves count.val=0, causing the drain loop to terminate prematurely. Reading through the leader avoids the zero-size buffer and correctly accumulates the leader's pending count. HTM events are always standalone or per-target leaders in practice; the leader redirect handles any grouped configuration without losing counts.Changes in V4:
- No changes from V3.Changes in V3:
- Useevsel__pmu_name(evsel)instead ofstrstarts(evsel->name, "htm")to identify HTM events, matching by kernel-assigned PMU name rather than user-visible event name.
- Remove the redundant two-pass loop (first pass to setfound_htm, second to accumulate counts); a single pass withevsel__pmu_name()is sufficient. if no HTM event existstotal_pending_recordsstays 0 and the function returns 0.
- Remove the dead !strcmp(evsel->name, "dummy:u") check;
-evsel__pmu_name()will never return "htm" for a dummy:u software event.
- Renametotal_pending_bytes->total_pending_recordsto match what the driver actually reports (event->count=total_size/record_size, a record count, not a byte count).
- Add #include <string.h> for musl compatibility (strcmp()without it warns on some toolchains).Changes in V2:
- Implements the renamedarch_perf_record__need_read()hook (V1 implementedarch_record__collect_final_data()).
- Skips evsels whose fd and sample-id xyarrays are mismatched, avoiding stale-state reads. V1 had no such guard.
-evlist__enablecycling is removed; that responsibility now belongs to the drain loop inbuiltin-record.cadded in patch 3.
- File location changed toarch/powerpc/util/evsel.c(V1 usedarch/powerpc/util/powerpc-htm.c).
- Patch is now 4/6 instead of 4/9.tools/perf/arch/powerpc/util/evsel.c | 77 ++++++++++++++++++++++++++++ 1 file changed, 77 insertions(+)tools/perf/arch/powerpc/util/evsel.c +77
-0@@ -1,8 +1,85 @@// SPDX-License-Identifier: GPL-2.0 #include <stdio.h> +#include <string.h> +#include <unistd.h> +#include <linux/string.h> #include "util/evsel.h" +#include "util/record.h" +#include "util/evlist.h" +#include "util/debug.h" +#include <internal/xyarray.h> +#include <internal/lib.h> void arch_evsel__set_sample_weight (struct evsel *evsel) {evsel__set_sample_bit (evsel,WEIGHT_STRUCT ); } + +/* +* powerpc implementation of arch_perf_record__need_read(). +* +* Reads event->count for every open HTM evsel by issuing a direct +* read() on the event fd with a plain u64 buffer, bypassing the +* PERF_FORMAT_GROUP path in perf_evsel__read(). When an HTM evsel is +* a group sibling, evsel__config() sets PERF_FORMAT_GROUP on its attr; +* perf_evsel__read() would then call perf_evsel__read_group() which +* sizes the buffer by evsel->nr_members (0 for siblings), causing the +* kernel to return -ENOSPC. Reading the fd directly with sizeof(u64) +* retrieves the HTM driver's plain pending-record count regardless of +* group membership. +* +* Returns: 1 if more data exists, 0 if collection is complete +*/ +int arch_perf_record__need_read (struct evlist *evlist) +{ +struct evsel *evsel; +u64 total_pending_records =0 ; +int x, y; + +/* there was an error during record__open */ +if (!evlist) +return 0 ; + +/* Read HTM event counts to check if more data is available */ +evlist__for_each_entry (evlist, evsel) { +struct perf_evsel *rd_evsel; +struct xyarray *xy; + +if (strcmp (evsel__pmu_name (evsel),"htm" )) +continue ; + +/* +* For group siblings nr_members == 0, which makes +* perf_evsel__read_size() return 0 and readn() fail. +* Read through the leader instead; perf_evsel__read_group() +* extracts the leader's own count from the group buffer. +*/ +if (evsel->core.leader != &evsel->core) + rd_evsel = evsel->core.leader; +else + rd_evsel = &evsel->core; + + xy = rd_evsel->sample_id; + +if (xy ==NULL || rd_evsel->fd ==NULL ) +continue ; + +if (xyarray__max_x (rd_evsel->fd) !=xyarray__max_x (xy) || +xyarray__max_y (rd_evsel->fd) !=xyarray__max_y (xy)) { +pr_debug ("Unmatched FD vs sample ID array for HTM event\n" ); +continue ; + } + +for (x =0 ; x <xyarray__max_x (xy); x++) { +for (y =0 ; y <xyarray__max_y (xy); y++) { +struct perf_counts_values count = { .val =0 }; + +if (perf_evsel__read (rd_evsel, x, y, &count) ==0 ) + total_pending_records += count.val; + } + } + } + +/* Collection is complete only when ALL hardware queues have no pending records */ +return (total_pending_records >0 ) ?1 :0 ; +}
Re: [PATCH V5 4/6] tools/perf: Add powerpc callback support for arch_perf_record__need_read
Quoted 30 lines · Adrian Hunter
Quoted 17 lines · Athira Rajeev
Implement the
arch_perf_record__need_read()architecture-specific hook for powerpc inarch/powerpc/util/evsel.c.The HTM kernel driver sets
event->countto the number of records still staged in its internal buffers (total_size/record_size), and to 0 once the stream is exhausted. This hook reads that count for every open htm evsel viaperf_evsel__read()and accumulates the values intototal_pending_records. A non-zero total means at least one HTM target still has records pending; the recording loop added in the previous patch will perform another mmap-read pass.The drain uses a two-layer safety check:
event->countdetects records staged by the driver, andrecord__bytes_written()in the drain loop confirms data was actually moved into perf.data. This combination handles the case where the driver count is briefly stale while hardware is still flushing.Quoted 7 lines
The implementation scans the evlist using
evsel__pmu_name()to identify HTM events by their kernel-assigned PMU name rather than the user-visible event name, preventing false matches. It iterates the fd/ sample-id xyarray, and skips any evsel whose fd and sample-id arrays are mismatched to avoid reading stale state. When the accumulated record count reaches zero the hook returns 0 and the recording loop proceeds to disable and close the events.
Quoted 3 lines
This looks like the proposed driver:
Hi Adrian,
Thank you for the review.
Could you please help clarify which specific aspect of the driver breaks the perf ABI? Is it the sysfs format ABI where encoding raw hardware topology identifiers (nodeindex, nodalchipindex, coreindexonchip) are used as perf_event_attr.config bit fields? I am trying to confirm so that I get your feedback correctly for reworking on changes.
Thanks,
Athira
Quoted 145 lines
breaks the perf ABI.
I am not going to review any more tools patches for now.
Quoted 139 lines
Signed-off-by Athira Rajeev <[email protected]>---
Changes in V5:
- When an HTM evsel is a group sibling (evsel->core.leader!=&evsel->core), read through its group leader's structperf_evselinstead of the sibling directly.perf_evsel__read_size()usesevsel->nr_membersto compute the read buffer size;nr_membersis 0 for siblings, so size=0 is passed toreadn(), which returns <=0 and leaves count.val=0, causing the drain loop to terminate prematurely. Reading through the leader avoids the zero-size buffer and correctly accumulates the leader's pending count. HTM events are always standalone or per-target leaders in practice; the leader redirect handles any grouped configuration without losing counts.Changes in V4:
- No changes from V3.Changes in V3:
- Useevsel__pmu_name(evsel)instead ofstrstarts(evsel->name, "htm")to identify HTM events, matching by kernel-assigned PMU name rather than user-visible event name.
- Remove the redundant two-pass loop (first pass to setfound_htm, second to accumulate counts); a single pass withevsel__pmu_name()is sufficient. if no HTM event existstotal_pending_recordsstays 0 and the function returns 0.
- Remove the dead !strcmp(evsel->name, "dummy:u") check;
-evsel__pmu_name()will never return "htm" for a dummy:u software event.
- Renametotal_pending_bytes->total_pending_recordsto match what the driver actually reports (event->count=total_size/record_size, a record count, not a byte count).
- Add #include <string.h> for musl compatibility (strcmp()without it warns on some toolchains).Changes in V2:
- Implements the renamedarch_perf_record__need_read()hook (V1 implementedarch_record__collect_final_data()).
- Skips evsels whose fd and sample-id xyarrays are mismatched, avoiding stale-state reads. V1 had no such guard.
-evlist__enablecycling is removed; that responsibility now belongs to the drain loop inbuiltin-record.cadded in patch 3.
- File location changed toarch/powerpc/util/evsel.c(V1 usedarch/powerpc/util/powerpc-htm.c).
- Patch is now 4/6 instead of 4/9.
tools/perf/arch/powerpc/util/evsel.c| 77 ++++++++++++++++++++++++++++1 file changed, 77 insertions(+)tools/perf/arch/powerpc/util/evsel.c +77
-0@@ -1,8 +1,85 @@// SPDX-License-Identifier: GPL-2.0 #include <stdio.h> +#include <string.h> +#include <unistd.h> +#include <linux/string.h> #include "util/evsel.h" +#include "util/record.h" +#include "util/evlist.h" +#include "util/debug.h" +#include <internal/xyarray.h> +#include <internal/lib.h> void arch_evsel__set_sample_weight (struct evsel *evsel) {evsel__set_sample_bit (evsel,WEIGHT_STRUCT ); } + +/* +* powerpc implementation of arch_perf_record__need_read(). +* +* Reads event->count for every open HTM evsel by issuing a direct +* read() on the event fd with a plain u64 buffer, bypassing the +* PERF_FORMAT_GROUP path in perf_evsel__read(). When an HTM evsel is +* a group sibling, evsel__config() sets PERF_FORMAT_GROUP on its attr; +* perf_evsel__read() would then call perf_evsel__read_group() which +* sizes the buffer by evsel->nr_members (0 for siblings), causing the +* kernel to return -ENOSPC. Reading the fd directly with sizeof(u64) +* retrieves the HTM driver's plain pending-record count regardless of +* group membership. +* +* Returns: 1 if more data exists, 0 if collection is complete +*/ +int arch_perf_record__need_read (struct evlist *evlist) +{ +struct evsel *evsel; +u64 total_pending_records =0 ; +int x, y; + +/* there was an error during record__open */ +if (!evlist) +return 0 ; + +/* Read HTM event counts to check if more data is available */ +evlist__for_each_entry (evlist, evsel) { +struct perf_evsel *rd_evsel; +struct xyarray *xy; + +if (strcmp (evsel__pmu_name (evsel),"htm" )) +continue ; + +/* +* For group siblings nr_members == 0, which makes +* perf_evsel__read_size() return 0 and readn() fail. +* Read through the leader instead; perf_evsel__read_group() +* extracts the leader's own count from the group buffer. +*/ +if (evsel->core.leader != &evsel->core) + rd_evsel = evsel->core.leader; +else + rd_evsel = &evsel->core; + + xy = rd_evsel->sample_id; + +if (xy ==NULL || rd_evsel->fd ==NULL ) +continue ; + +if (xyarray__max_x (rd_evsel->fd) !=xyarray__max_x (xy) || +xyarray__max_y (rd_evsel->fd) !=xyarray__max_y (xy)) { +pr_debug ("Unmatched FD vs sample ID array for HTM event\n" ); +continue ; + } + +for (x =0 ; x <xyarray__max_x (xy); x++) { +for (y =0 ; y <xyarray__max_y (xy); y++) { +struct perf_counts_values count = { .val =0 }; + +if (perf_evsel__read (rd_evsel, x, y, &count) ==0 ) + total_pending_records += count.val; + } + } + } + +/* Collection is complete only when ALL hardware queues have no pending records */ +return (total_pending_records >0 ) ?1 :0 ; +}
Adrian Hunter
reviewerRe: [PATCH V5 4/6] tools/perf: Add powerpc callback support for arch_perf_record__need_read
Quoted 32 lines · Athira Rajeev
Quoted 32 lines · Adrian Hunter
Quoted 17 lines · Athira Rajeev
Implement the
arch_perf_record__need_read()architecture-specific hook for powerpc inarch/powerpc/util/evsel.c.The HTM kernel driver sets
event->countto the number of records still staged in its internal buffers (total_size/record_size), and to 0 once the stream is exhausted. This hook reads that count for every open htm evsel viaperf_evsel__read()and accumulates the values intototal_pending_records. A non-zero total means at least one HTM target still has records pending; the recording loop added in the previous patch will perform another mmap-read pass.The drain uses a two-layer safety check:
event->countdetects records staged by the driver, andrecord__bytes_written()in the drain loop confirms data was actually moved into perf.data. This combination handles the case where the driver count is briefly stale while hardware is still flushing.Quoted 7 lines
The implementation scans the evlist using
evsel__pmu_name()to identify HTM events by their kernel-assigned PMU name rather than the user-visible event name, preventing false matches. It iterates the fd/ sample-id xyarray, and skips any evsel whose fd and sample-id arrays are mismatched to avoid reading stale state. When the accumulated record count reaches zero the hook returns 0 and the recording loop proceeds to disable and close the events.This looks like the proposed driver:
Quoted 7 lines
Quoted 1 line
Hi Adrian,
Thank you for the review.Could you please help clarify which specific aspect of the driver breaks the perf ABI? Is it the sysfs format ABI where encoding raw hardware topology identifiers (nodeindex, nodalchipindex, coreindexonchip) are used as perf_event_attr.config bit fields? I am trying to confirm so that I get your feedback correctly for reworking on changes.
This "draining" and "read" usage looked unusual, so I asked AI:
I have just applied the patches for powerpc htm kernel driver - they are now the last 5 patches committed. Examine the design in comparison to how other PMU drivers are implemented. Does it violate the kernel API for PMUs?
It said yes.
Quoted 150 lines
Thanks,
AthiraQuoted 145 lines
breaks the perf ABI.
I am not going to review any more tools patches for now.
Quoted 139 lines
Signed-off-by Athira Rajeev <[email protected]>---
Changes in V5:
- When an HTM evsel is a group sibling (evsel->core.leader!=&evsel->core), read through its group leader's structperf_evselinstead of the sibling directly.perf_evsel__read_size()usesevsel->nr_membersto compute the read buffer size;nr_membersis 0 for siblings, so size=0 is passed toreadn(), which returns <=0 and leaves count.val=0, causing the drain loop to terminate prematurely. Reading through the leader avoids the zero-size buffer and correctly accumulates the leader's pending count. HTM events are always standalone or per-target leaders in practice; the leader redirect handles any grouped configuration without losing counts.Changes in V4:
- No changes from V3.Changes in V3:
- Useevsel__pmu_name(evsel)instead ofstrstarts(evsel->name, "htm")to identify HTM events, matching by kernel-assigned PMU name rather than user-visible event name.
- Remove the redundant two-pass loop (first pass to setfound_htm, second to accumulate counts); a single pass withevsel__pmu_name()is sufficient. if no HTM event existstotal_pending_recordsstays 0 and the function returns 0.
- Remove the dead !strcmp(evsel->name, "dummy:u") check;
-evsel__pmu_name()will never return "htm" for a dummy:u software event.
- Renametotal_pending_bytes->total_pending_recordsto match what the driver actually reports (event->count=total_size/record_size, a record count, not a byte count).
- Add #include <string.h> for musl compatibility (strcmp()without it warns on some toolchains).Changes in V2:
- Implements the renamedarch_perf_record__need_read()hook (V1 implementedarch_record__collect_final_data()).
- Skips evsels whose fd and sample-id xyarrays are mismatched, avoiding stale-state reads. V1 had no such guard.
-evlist__enablecycling is removed; that responsibility now belongs to the drain loop inbuiltin-record.cadded in patch 3.
- File location changed toarch/powerpc/util/evsel.c(V1 usedarch/powerpc/util/powerpc-htm.c).
- Patch is now 4/6 instead of 4/9.
tools/perf/arch/powerpc/util/evsel.c| 77 ++++++++++++++++++++++++++++1 file changed, 77 insertions(+)tools/perf/arch/powerpc/util/evsel.c +77
-0@@ -1,8 +1,85 @@// SPDX-License-Identifier: GPL-2.0 #include <stdio.h> +#include <string.h> +#include <unistd.h> +#include <linux/string.h> #include "util/evsel.h" +#include "util/record.h" +#include "util/evlist.h" +#include "util/debug.h" +#include <internal/xyarray.h> +#include <internal/lib.h> void arch_evsel__set_sample_weight (struct evsel *evsel) {evsel__set_sample_bit (evsel,WEIGHT_STRUCT ); } + +/* +* powerpc implementation of arch_perf_record__need_read(). +* +* Reads event->count for every open HTM evsel by issuing a direct +* read() on the event fd with a plain u64 buffer, bypassing the +* PERF_FORMAT_GROUP path in perf_evsel__read(). When an HTM evsel is +* a group sibling, evsel__config() sets PERF_FORMAT_GROUP on its attr; +* perf_evsel__read() would then call perf_evsel__read_group() which +* sizes the buffer by evsel->nr_members (0 for siblings), causing the +* kernel to return -ENOSPC. Reading the fd directly with sizeof(u64) +* retrieves the HTM driver's plain pending-record count regardless of +* group membership. +* +* Returns: 1 if more data exists, 0 if collection is complete +*/ +int arch_perf_record__need_read (struct evlist *evlist) +{ +struct evsel *evsel; +u64 total_pending_records =0 ; +int x, y; + +/* there was an error during record__open */ +if (!evlist) +return 0 ; + +/* Read HTM event counts to check if more data is available */ +evlist__for_each_entry (evlist, evsel) { +struct perf_evsel *rd_evsel; +struct xyarray *xy; + +if (strcmp (evsel__pmu_name (evsel),"htm" )) +continue ; + +/* +* For group siblings nr_members == 0, which makes +* perf_evsel__read_size() return 0 and readn() fail. +* Read through the leader instead; perf_evsel__read_group() +* extracts the leader's own count from the group buffer. +*/ +if (evsel->core.leader != &evsel->core) + rd_evsel = evsel->core.leader; +else + rd_evsel = &evsel->core; + + xy = rd_evsel->sample_id; + +if (xy ==NULL || rd_evsel->fd ==NULL ) +continue ; + +if (xyarray__max_x (rd_evsel->fd) !=xyarray__max_x (xy) || +xyarray__max_y (rd_evsel->fd) !=xyarray__max_y (xy)) { +pr_debug ("Unmatched FD vs sample ID array for HTM event\n" ); +continue ; + } + +for (x =0 ; x <xyarray__max_x (xy); x++) { +for (y =0 ; y <xyarray__max_y (xy); y++) { +struct perf_counts_values count = { .val =0 }; + +if (perf_evsel__read (rd_evsel, x, y, &count) ==0 ) + total_pending_records += count.val; + } + } + } + +/* Collection is complete only when ALL hardware queues have no pending records */ +return (total_pending_records >0 ) ?1 :0 ; +}
Re: [PATCH V5 4/6] tools/perf: Add powerpc callback support for arch_perf_record__need_read
Quoted 45 lines · Adrian Hunter
Quoted 32 lines · Athira Rajeev
Quoted 32 lines · Adrian Hunter
Quoted 17 lines · Athira Rajeev
Implement the
arch_perf_record__need_read()architecture-specific hook for powerpc inarch/powerpc/util/evsel.c.The HTM kernel driver sets
event->countto the number of records still staged in its internal buffers (total_size/record_size), and to 0 once the stream is exhausted. This hook reads that count for every open htm evsel viaperf_evsel__read()and accumulates the values intototal_pending_records. A non-zero total means at least one HTM target still has records pending; the recording loop added in the previous patch will perform another mmap-read pass.The drain uses a two-layer safety check:
event->countdetects records staged by the driver, andrecord__bytes_written()in the drain loop confirms data was actually moved into perf.data. This combination handles the case where the driver count is briefly stale while hardware is still flushing.Quoted 7 lines
The implementation scans the evlist using
evsel__pmu_name()to identify HTM events by their kernel-assigned PMU name rather than the user-visible event name, preventing false matches. It iterates the fd/ sample-id xyarray, and skips any evsel whose fd and sample-id arrays are mismatched to avoid reading stale state. When the accumulated record count reaches zero the hook returns 0 and the recording loop proceeds to disable and close the events.This looks like the proposed driver:
Quoted 7 lines
Quoted 1 line
Hi Adrian,
Thank you for the review.Could you please help clarify which specific aspect of the driver breaks the perf ABI? Is it the sysfs format ABI where encoding raw hardware topology identifiers (nodeindex, nodalchipindex, coreindexonchip) are used as perf_event_attr.config bit fields? I am trying to confirm so that I get your feedback correctly for reworking on changes.
Quoted 6 lines
This "draining" and "read" usage looked unusual, so I asked AI:
I have just applied the patches for powerpc htm kernel driver - they are now the last 5 patches committed. Examine the design in comparison to how other PMU drivers are implemented. Does it violate the kernel API for PMUs?
It said yes.
Hi Adrian,
Thanks for taking the time to check this.
I agree with the assessment. The core problems I understand are:
- pmu->read() issues hcalls that stop the hardware trace and dump the data, so any read() call ends up silently draining and stopping the trace — which isn't what read() is for.
- event->count is being used to signal "more data pending" which breaks the expected semantics.
I'm going to rework this rather than continue posting on top of it. Here is my rework plan for your review:
Kernel side: workqueue-based approach:
- pmu->read() becomes a no-op. No hcalls, no side effects.
- pmu->stop() issues hypervisor call to freeze the hardware buffer, then schedules a work_struct (htm_drain_work_fn()) that runs in process context and drains the frozen buffer chunk by chunk into the AUX ring via perf_aux_output_end(), backing off on -ENOSPC to let userspace consume between chunks. event->count is no longer used as a flow-control signal.
Tools side:
- arch_perf_record__need_read() hook and record__final_aux_data() are dropped entirely — no changes to builtin-record.c.
- “read_finish" callback is implemented in the HTM auxtrace_record:
During normal recording (evsel->disabled == false) it returns immediately.
At session teardown, evlist__disable() sets evsel->disabled = true before the
final record__mmap_read_all() call.
The read_finish callback detects this and polls aux_head (via perf_mmap__read_head())
in a loop, yielding delay between polls, until the AUX head stops advancing — signalling the workqueue drain is complete.
This gives the kernel workqueue time to finish copying chunks before evlist__close() frees the mmaps. The wait has a bounded maximum, so a stalled drain can't hang perf record indefinitely.
- snapshot_finish returns -EINVAL since HTM does not support snapshot mode.
HTM has no interrupt and trace data is consumed after pmu stop . Hence making this read_finish callback for auxtrace_record to consume data.
With this, data collection is moved of from .read() entirely and uses work queue based approach. Can you please review , Does this approach look acceptable before I send the updated series?
Thanks,
Athira
Quoted 149 lines
Quoted 149 lines
Thanks,
AthiraQuoted 144 lines
breaks the perf ABI.
I am not going to review any more tools patches for now.
Quoted 139 lines
Signed-off-by Athira Rajeev <[email protected]>---
Changes in V5:
- When an HTM evsel is a group sibling (evsel->core.leader!=&evsel->core), read through its group leader's structperf_evselinstead of the sibling directly.perf_evsel__read_size()usesevsel->nr_membersto compute the read buffer size;nr_membersis 0 for siblings, so size=0 is passed toreadn(), which returns <=0 and leaves count.val=0, causing the drain loop to terminate prematurely. Reading through the leader avoids the zero-size buffer and correctly accumulates the leader's pending count. HTM events are always standalone or per-target leaders in practice; the leader redirect handles any grouped configuration without losing counts.Changes in V4:
- No changes from V3.Changes in V3:
- Useevsel__pmu_name(evsel)instead ofstrstarts(evsel->name, "htm")to identify HTM events, matching by kernel-assigned PMU name rather than user-visible event name.
- Remove the redundant two-pass loop (first pass to setfound_htm, second to accumulate counts); a single pass withevsel__pmu_name()is sufficient. if no HTM event existstotal_pending_recordsstays 0 and the function returns 0.
- Remove the dead !strcmp(evsel->name, "dummy:u") check;
-evsel__pmu_name()will never return "htm" for a dummy:u software event.
- Renametotal_pending_bytes->total_pending_recordsto match what the driver actually reports (event->count=total_size/record_size, a record count, not a byte count).
- Add #include <string.h> for musl compatibility (strcmp()without it warns on some toolchains).Changes in V2:
- Implements the renamedarch_perf_record__need_read()hook (V1 implementedarch_record__collect_final_data()).
- Skips evsels whose fd and sample-id xyarrays are mismatched, avoiding stale-state reads. V1 had no such guard.
-evlist__enablecycling is removed; that responsibility now belongs to the drain loop inbuiltin-record.cadded in patch 3.
- File location changed toarch/powerpc/util/evsel.c(V1 usedarch/powerpc/util/powerpc-htm.c).
- Patch is now 4/6 instead of 4/9.
tools/perf/arch/powerpc/util/evsel.c| 77 ++++++++++++++++++++++++++++1 file changed, 77 insertions(+)tools/perf/arch/powerpc/util/evsel.c +77
-0@@ -1,8 +1,85 @@// SPDX-License-Identifier: GPL-2.0 #include <stdio.h> +#include <string.h> +#include <unistd.h> +#include <linux/string.h> #include "util/evsel.h" +#include "util/record.h" +#include "util/evlist.h" +#include "util/debug.h" +#include <internal/xyarray.h> +#include <internal/lib.h> void arch_evsel__set_sample_weight (struct evsel *evsel) {evsel__set_sample_bit (evsel,WEIGHT_STRUCT ); } + +/* +* powerpc implementation of arch_perf_record__need_read(). +* +* Reads event->count for every open HTM evsel by issuing a direct +* read() on the event fd with a plain u64 buffer, bypassing the +* PERF_FORMAT_GROUP path in perf_evsel__read(). When an HTM evsel is +* a group sibling, evsel__config() sets PERF_FORMAT_GROUP on its attr; +* perf_evsel__read() would then call perf_evsel__read_group() which +* sizes the buffer by evsel->nr_members (0 for siblings), causing the +* kernel to return -ENOSPC. Reading the fd directly with sizeof(u64) +* retrieves the HTM driver's plain pending-record count regardless of +* group membership. +* +* Returns: 1 if more data exists, 0 if collection is complete +*/ +int arch_perf_record__need_read (struct evlist *evlist) +{ +struct evsel *evsel; +u64 total_pending_records =0 ; +int x, y; + +/* there was an error during record__open */ +if (!evlist) +return 0 ; + +/* Read HTM event counts to check if more data is available */ +evlist__for_each_entry (evlist, evsel) { +struct perf_evsel *rd_evsel; +struct xyarray *xy; + +if (strcmp (evsel__pmu_name (evsel),"htm" )) +continue ; + +/* +* For group siblings nr_members == 0, which makes +* perf_evsel__read_size() return 0 and readn() fail. +* Read through the leader instead; perf_evsel__read_group() +* extracts the leader's own count from the group buffer. +*/ +if (evsel->core.leader != &evsel->core) + rd_evsel = evsel->core.leader; +else + rd_evsel = &evsel->core; + + xy = rd_evsel->sample_id; + +if (xy ==NULL || rd_evsel->fd ==NULL ) +continue ; + +if (xyarray__max_x (rd_evsel->fd) !=xyarray__max_x (xy) || +xyarray__max_y (rd_evsel->fd) !=xyarray__max_y (xy)) { +pr_debug ("Unmatched FD vs sample ID array for HTM event\n" ); +continue ; + } + +for (x =0 ; x <xyarray__max_x (xy); x++) { +for (y =0 ; y <xyarray__max_y (xy); y++) { +struct perf_counts_values count = { .val =0 }; + +if (perf_evsel__read (rd_evsel, x, y, &count) ==0 ) + total_pending_records += count.val; + } + } + } + +/* Collection is complete only when ALL hardware queues have no pending records */ +return (total_pending_records >0 ) ?1 :0 ; +}
Adrian Hunter
reviewerRe: [PATCH V5 4/6] tools/perf: Add powerpc callback support for arch_perf_record__need_read
Quoted 64 lines · Athira Rajeev
Quoted 45 lines · Adrian Hunter
Quoted 32 lines · Athira Rajeev
Quoted 32 lines · Adrian Hunter
Quoted 17 lines · Athira Rajeev
Implement the
arch_perf_record__need_read()architecture-specific hook for powerpc inarch/powerpc/util/evsel.c.The HTM kernel driver sets
event->countto the number of records still staged in its internal buffers (total_size/record_size), and to 0 once the stream is exhausted. This hook reads that count for every open htm evsel viaperf_evsel__read()and accumulates the values intototal_pending_records. A non-zero total means at least one HTM target still has records pending; the recording loop added in the previous patch will perform another mmap-read pass.The drain uses a two-layer safety check:
event->countdetects records staged by the driver, andrecord__bytes_written()in the drain loop confirms data was actually moved into perf.data. This combination handles the case where the driver count is briefly stale while hardware is still flushing.Quoted 7 lines
The implementation scans the evlist using
evsel__pmu_name()to identify HTM events by their kernel-assigned PMU name rather than the user-visible event name, preventing false matches. It iterates the fd/ sample-id xyarray, and skips any evsel whose fd and sample-id arrays are mismatched to avoid reading stale state. When the accumulated record count reaches zero the hook returns 0 and the recording loop proceeds to disable and close the events.This looks like the proposed driver:
Quoted 7 lines
Quoted 1 line
Hi Adrian,
Thank you for the review.Could you please help clarify which specific aspect of the driver breaks the perf ABI? Is it the sysfs format ABI where encoding raw hardware topology identifiers (nodeindex, nodalchipindex, coreindexonchip) are used as perf_event_attr.config bit fields? I am trying to confirm so that I get your feedback correctly for reworking on changes.
Quoted 6 lines
This "draining" and "read" usage looked unusual, so I asked AI:
I have just applied the patches for powerpc htm kernel driver - they are now the last 5 patches committed. Examine the design in comparison to how other PMU drivers are implemented. Does it violate the kernel API for PMUs?
It said yes.
Hi Adrian,
Thanks for taking the time to check this.I agree with the assessment. The core problems I understand are:
-pmu->read()issues hcalls that stop the hardware trace and dump the data, so anyread()call ends up silently draining and stopping the trace — which isn't whatread()is for.
-event->countis being used to signal "more data pending" which breaks the expected semantics.I'm going to rework this rather than continue posting on top of it. Here is my rework plan for your review:
Kernel side: workqueue-based approach:
Review
Quoted 5 lines
(quoted diff) +0
-2- pmu->read() becomes a no-op. No hcalls, no side effects. - pmu->stop() issues hypervisor call to freeze the hardware buffer, then schedules a work_struct (htm_drain_work_fn()) that runs in process context and drains the frozen buffer chunk by chunk into the AUX ring via perf_aux_output_end(), backing off on -ENOSPC to let userspace consume between chunks. event->count is no longer used as a flow-control signal.
That sounds a bit like pmu->stop() wouldn't actually stop. I can imagine there could be issues with that. I suspect only someone like Peter Zijlstra can advise you.
Quoted 175 lines
Tools side:
(quoted diff) +0
-3- arch_perf_record__need_read() hook and record__final_aux_data() are dropped entirely — no changes to builtin-record.c. - “read_finish" callback is implemented in the HTM auxtrace_record: During normal recording (evsel->disabled == false) it returns immediately. At session teardown, evlist__disable() sets evsel->disabled = true before the final record__mmap_read_all() call. The read_finish callback detects this and polls aux_head (via perf_mmap__read_head()) in a loop, yielding delay between polls, until the AUX head stops advancing — signalling the workqueue drain is complete. This gives the kernel workqueue time to finish copying chunks before evlist__close() frees the mmaps. The wait has a bounded maximum, so a stalled drain can't hang perf record indefinitely. - snapshot_finish returns -EINVAL since HTM does not support snapshot mode.HTM has no interrupt and trace data is consumed after pmu stop . Hence making this
read_finishcallback forauxtrace_recordto consume data.With this, data collection is moved of from .read() entirely and uses work queue based approach. Can you please review , Does this approach look acceptable before I send the updated series?
Thanks,
AthiraQuoted 149 lines
Quoted 149 lines
Thanks,
AthiraQuoted 144 lines
breaks the perf ABI.
I am not going to review any more tools patches for now.
Quoted 139 lines
Signed-off-by Athira Rajeev <[email protected]>---
Changes in V5:
- When an HTM evsel is a group sibling (evsel->core.leader!=&evsel->core), read through its group leader's structperf_evselinstead of the sibling directly.perf_evsel__read_size()usesevsel->nr_membersto compute the read buffer size;nr_membersis 0 for siblings, so size=0 is passed toreadn(), which returns <=0 and leaves count.val=0, causing the drain loop to terminate prematurely. Reading through the leader avoids the zero-size buffer and correctly accumulates the leader's pending count. HTM events are always standalone or per-target leaders in practice; the leader redirect handles any grouped configuration without losing counts.Changes in V4:
- No changes from V3.Changes in V3:
- Useevsel__pmu_name(evsel)instead ofstrstarts(evsel->name, "htm")to identify HTM events, matching by kernel-assigned PMU name rather than user-visible event name.
- Remove the redundant two-pass loop (first pass to setfound_htm, second to accumulate counts); a single pass withevsel__pmu_name()is sufficient. if no HTM event existstotal_pending_recordsstays 0 and the function returns 0.
- Remove the dead !strcmp(evsel->name, "dummy:u") check;
-evsel__pmu_name()will never return "htm" for a dummy:u software event.
- Renametotal_pending_bytes->total_pending_recordsto match what the driver actually reports (event->count=total_size/record_size, a record count, not a byte count).
- Add #include <string.h> for musl compatibility (strcmp()without it warns on some toolchains).Changes in V2:
- Implements the renamedarch_perf_record__need_read()hook (V1 implementedarch_record__collect_final_data()).
- Skips evsels whose fd and sample-id xyarrays are mismatched, avoiding stale-state reads. V1 had no such guard.
-evlist__enablecycling is removed; that responsibility now belongs to the drain loop inbuiltin-record.cadded in patch 3.
- File location changed toarch/powerpc/util/evsel.c(V1 usedarch/powerpc/util/powerpc-htm.c).
- Patch is now 4/6 instead of 4/9.
tools/perf/arch/powerpc/util/evsel.c| 77 ++++++++++++++++++++++++++++1 file changed, 77 insertions(+)tools/perf/arch/powerpc/util/evsel.c +77
-0@@ -1,8 +1,85 @@// SPDX-License-Identifier: GPL-2.0 #include <stdio.h> +#include <string.h> +#include <unistd.h> +#include <linux/string.h> #include "util/evsel.h" +#include "util/record.h" +#include "util/evlist.h" +#include "util/debug.h" +#include <internal/xyarray.h> +#include <internal/lib.h> void arch_evsel__set_sample_weight (struct evsel *evsel) {evsel__set_sample_bit (evsel,WEIGHT_STRUCT ); } + +/* +* powerpc implementation of arch_perf_record__need_read(). +* +* Reads event->count for every open HTM evsel by issuing a direct +* read() on the event fd with a plain u64 buffer, bypassing the +* PERF_FORMAT_GROUP path in perf_evsel__read(). When an HTM evsel is +* a group sibling, evsel__config() sets PERF_FORMAT_GROUP on its attr; +* perf_evsel__read() would then call perf_evsel__read_group() which +* sizes the buffer by evsel->nr_members (0 for siblings), causing the +* kernel to return -ENOSPC. Reading the fd directly with sizeof(u64) +* retrieves the HTM driver's plain pending-record count regardless of +* group membership. +* +* Returns: 1 if more data exists, 0 if collection is complete +*/ +int arch_perf_record__need_read (struct evlist *evlist) +{ +struct evsel *evsel; +u64 total_pending_records =0 ; +int x, y; + +/* there was an error during record__open */ +if (!evlist) +return 0 ; + +/* Read HTM event counts to check if more data is available */ +evlist__for_each_entry (evlist, evsel) { +struct perf_evsel *rd_evsel; +struct xyarray *xy; + +if (strcmp (evsel__pmu_name (evsel),"htm" )) +continue ; + +/* +* For group siblings nr_members == 0, which makes +* perf_evsel__read_size() return 0 and readn() fail. +* Read through the leader instead; perf_evsel__read_group() +* extracts the leader's own count from the group buffer. +*/ +if (evsel->core.leader != &evsel->core) + rd_evsel = evsel->core.leader; +else + rd_evsel = &evsel->core; + + xy = rd_evsel->sample_id; + +if (xy ==NULL || rd_evsel->fd ==NULL ) +continue ; + +if (xyarray__max_x (rd_evsel->fd) !=xyarray__max_x (xy) || +xyarray__max_y (rd_evsel->fd) !=xyarray__max_y (xy)) { +pr_debug ("Unmatched FD vs sample ID array for HTM event\n" ); +continue ; + } + +for (x =0 ; x <xyarray__max_x (xy); x++) { +for (y =0 ; y <xyarray__max_y (xy); y++) { +struct perf_counts_values count = { .val =0 }; + +if (perf_evsel__read (rd_evsel, x, y, &count) ==0 ) + total_pending_records += count.val; + } + } + } + +/* Collection is complete only when ALL hardware queues have no pending records */ +return (total_pending_records >0 ) ?1 :0 ; +}
Re: [PATCH V5 4/6] tools/perf: Add powerpc callback support for arch_perf_record__need_read
Quoted 75 lines · Adrian Hunter
Quoted 64 lines · Athira Rajeev
Quoted 45 lines · Adrian Hunter
Quoted 32 lines · Athira Rajeev
Quoted 32 lines · Adrian Hunter
Quoted 17 lines · Athira Rajeev
Implement the
arch_perf_record__need_read()architecture-specific hook for powerpc inarch/powerpc/util/evsel.c.The HTM kernel driver sets
event->countto the number of records still staged in its internal buffers (total_size/record_size), and to 0 once the stream is exhausted. This hook reads that count for every open htm evsel viaperf_evsel__read()and accumulates the values intototal_pending_records. A non-zero total means at least one HTM target still has records pending; the recording loop added in the previous patch will perform another mmap-read pass.The drain uses a two-layer safety check:
event->countdetects records staged by the driver, andrecord__bytes_written()in the drain loop confirms data was actually moved into perf.data. This combination handles the case where the driver count is briefly stale while hardware is still flushing.Quoted 7 lines
The implementation scans the evlist using
evsel__pmu_name()to identify HTM events by their kernel-assigned PMU name rather than the user-visible event name, preventing false matches. It iterates the fd/ sample-id xyarray, and skips any evsel whose fd and sample-id arrays are mismatched to avoid reading stale state. When the accumulated record count reaches zero the hook returns 0 and the recording loop proceeds to disable and close the events.This looks like the proposed driver:
Quoted 7 lines
Quoted 1 line
Hi Adrian,
Thank you for the review.Could you please help clarify which specific aspect of the driver breaks the perf ABI? Is it the sysfs format ABI where encoding raw hardware topology identifiers (nodeindex, nodalchipindex, coreindexonchip) are used as perf_event_attr.config bit fields? I am trying to confirm so that I get your feedback correctly for reworking on changes.
Quoted 6 lines
This "draining" and "read" usage looked unusual, so I asked AI:
I have just applied the patches for powerpc htm kernel driver - they are now the last 5 patches committed. Examine the design in comparison to how other PMU drivers are implemented. Does it violate the kernel API for PMUs?
It said yes.
Hi Adrian,
Thanks for taking the time to check this.I agree with the assessment. The core problems I understand are:
-pmu->read()issues hcalls that stop the hardware trace and dump the data, so anyread()call ends up silently draining and stopping the trace — which isn't whatread()is for.
-event->countis being used to signal "more data pending" which breaks the expected semantics.I'm going to rework this rather than continue posting on top of it. Here is my rework plan for your review:
Kernel side: workqueue-based approach:
Review
Quoted 5 lines
(quoted diff) +0
-2- pmu->read() becomes a no-op. No hcalls, no side effects. - pmu->stop() issues hypervisor call to freeze the hardware buffer, then schedules a work_struct (htm_drain_work_fn()) that runs in process context and drains the frozen buffer chunk by chunk into the AUX ring via perf_aux_output_end(), backing off on -ENOSPC to let userspace consume between chunks. event->count is no longer used as a flow-control signal.
Quoted 3 lines
That sounds a bit like
pmu->stop()wouldn't actually stop. I can imagine there could be issues with that. I suspect only someone like Peter Zijlstra can advise you.
Sure, thanks for directing Adrian,
Adding Peter Zijlstra to the thread..
Hi Peter,
I am working on a perf PMU driver for HTM (Hardware Trace Macro) on
IBM POWER systems. HTM captures hardware trace data via the H_HTM hypervisor call. The driver in the patch series exposes it as a perf AUX PMU so trace data can be correlated with other perf events in perf.data.
Patch series: https://lore.kernel.org/linux-perf-users/[email protected]/
https://lore.kernel.org/linux-perf-users/[email protected]/
Adrian Hunter reviewed the series and we have some queries. I am reaching out for your guidance on the right kernel mechanism for draining trace data into the perf AUX buffer. Please read details below and help share your thoughts. Apologies for the long text, I tried to explain context and concerns that came out of the review.
Hardware constraints
--------------------
HTM is a one-shot, firmware-managed trace buffer:
- Buffer size is fixed at boot time.
Cannot be changed without a reboot.
- No hardware interrupt (no PMI, no DMA completion signal).
- The entire buffer is read after issuing a stop hcall. Not while tracing is active.
- Buffer must be copied chunk by chunk via a hypervisor hcall into the perf AUX ring (physically contiguous pages).
What V5 of patch series did
-----------
In V5, pmu->read() did all the work:
- On the first call: issued stop hcall to freeze the hardware buffer, then called hcall to dump the trace data into the AUX ring via perf_aux_output_begin() / perf_aux_output_end().
- Subsequent calls: skipped the stop (already frozen), copied the
next chunk, advanced aux_buf->head.
- Set event->count to the record count written, or 1 if the AUX ring was full (-ENOSPC, meaning "retry"), or 0 at EOF.
- Userspace polled event->count via perf_evsel__read() in a new weak arch_perf_record__need_read() hook added to builtin-record.c; a non-zero count triggered another record__mmap_read_all() + retry.
Review from Adrian flagged concerns with this:
Concern 1: pmu->read() issues stop on its first call —
a hardware side effect inside pmu->read().
Concern 2: event->count is used as a flow-control signal
(1 = "retry", 0 = "done") rather than a real counter.
Addressing the two concerns for V6
-----------------------------------
Concern 2 is straightforward to fix: remove all event->count manipulation from pmu->read(). Loop termination is detected by AUX head not advancing. No event->count needed.
Concern 1 is harder. The query I have is:
Is it acceptable for pmu->read() to issue stop hcall exactly once (on the first drain call, not on all subsequent calls), then copy trace data chunk via perf_aux_output_begin() / dump trace data / perf_aux_output_end()?
The reason for considering this approach for HTM specifically:
- htm_event_init() rejects all non-sampling opens with -EOPNOTSUPP:
if (!is_sampling_event(event))
return -EOPNOTSUPP;
So perf stat, BPF helpers, and any other non-record caller get -EOPNOTSUPP at perf_event_open() time. An HTM fd can only exist inside a perf record session. The only caller of read(fd) on an HTM event is from read hook - HTM's own drain code — which calls it deliberately, knowing stop fires on the first call. There is no unsuspecting caller path for HTM.
- After the first call, pmu->read() is a pure data-mover with no further hardware side effects.
Alternative: workqueue after pmu->stop()
-----------------------------------------
If pmu->read() having any hardware side effect is not right path, the alternative I was proposing is a workqueue-based approach:
- pmu->read() becomes a true no-op.
- pmu->stop() issues hcall stop operation to freeze the hardware buffer, then schedules a work_struct (say htm_drain_work_fn()) that runs in process context and drains the frozen buffer chunk by chunk into the AUX ring via perf_aux_output_begin() / perf_aux_output_end(), backing off on -ENOSPC between chunks.
- event->count is not used as a flow-control signal.
- deconfigure the tracing ( releasing the resources ) in event->destroy()
Tools side:
- arch_perf_record__need_read() hook is dropped — no changes to builtin-record.c.
- A custom "read_finish" callback in the HTM "auxtrace_record" polls auxtrace_mmap__read_head() until the AUX head stops advancing, with a bounded maximum wait. This ensures the workqueue drain completes before evlist__close() frees the mmaps.
Adrian's concern with this approach: "That sounds like pmu->stop() wouldn't actually stop.”
Traced through the tools code, workqueue lifetime looks safe to me based on this teardown sequence:
main poll loop exits
|
v
out_child: (builtin-record.c)
record__mmap_read_all(rec, true) <- final AUX ring drain
|
| <- read_finish called here,
| fd still open at this point
|
perf_session__delete() |
v
return from __cmd_record()
|
v
cmd_record():
evlist__delete(rec->evlist)
+-- evlist__close()
+-- close(*fd) <- fd closed here
-> release the resourcesSo event->destroy fires after "read_finish" . cancel_work_sync() in event->destroy ensures the drain workqueue finishes before releasing the hypervisor buffer. The fd is still open when read_finish runs, so it can safely wait for the drain.
Peterz, I have tried to summarize these below and I am seeking for your guidance here. Please share your thoughts.
--------------------
1. Is it acceptable for pmu->read() to issue stop tracing once on the first drain call, given that HTM events can only be opened by perf record (non-sampling opens rejected in htm_event_init()), so there is no unsuspecting caller path?
2. If not, is the workqueue-after-pmu->stop() approach safe? Releasing the resources ( deconfigure ) will be placed in event->destroy (not pmu->del()), which fires only at close(fd) — after cancel_work_sync() ensures the workqueue is done. The AUX mmaps also remain valid until evlist__munmap() at that same point. So the workqueue can call perf_aux_output_begin() / perf_aux_output_end() safely because the hypervisor buffer and AUX mmaps are still live when it runs. Is this reasoning correct, or does the perf core free AUX resources (free_aux / rb teardown) earlier than event->destroy?
3. Is there a third approach we can go with ? please suggest if there is another way.
Thank you,
Athira Rajeev
Quoted 174 lines
Quoted 174 lines
Tools side:
(quoted diff) +0
-3- arch_perf_record__need_read() hook and record__final_aux_data() are dropped entirely — no changes to builtin-record.c. - “read_finish" callback is implemented in the HTM auxtrace_record: During normal recording (evsel->disabled == false) it returns immediately. At session teardown, evlist__disable() sets evsel->disabled = true before the final record__mmap_read_all() call. The read_finish callback detects this and polls aux_head (via perf_mmap__read_head()) in a loop, yielding delay between polls, until the AUX head stops advancing — signalling the workqueue drain is complete. This gives the kernel workqueue time to finish copying chunks before evlist__close() frees the mmaps. The wait has a bounded maximum, so a stalled drain can't hang perf record indefinitely. - snapshot_finish returns -EINVAL since HTM does not support snapshot mode.HTM has no interrupt and trace data is consumed after pmu stop . Hence making this
read_finishcallback forauxtrace_recordto consume data.With this, data collection is moved of from .read() entirely and uses work queue based approach. Can you please review , Does this approach look acceptable before I send the updated series?
Thanks,
AthiraQuoted 149 lines
Quoted 149 lines
Thanks,
AthiraQuoted 144 lines
breaks the perf ABI.
I am not going to review any more tools patches for now.
Quoted 139 lines
Signed-off-by Athira Rajeev <[email protected]>---
Changes in V5:
- When an HTM evsel is a group sibling (evsel->core.leader!=&evsel->core), read through its group leader's structperf_evselinstead of the sibling directly.perf_evsel__read_size()usesevsel->nr_membersto compute the read buffer size;nr_membersis 0 for siblings, so size=0 is passed toreadn(), which returns <=0 and leaves count.val=0, causing the drain loop to terminate prematurely. Reading through the leader avoids the zero-size buffer and correctly accumulates the leader's pending count. HTM events are always standalone or per-target leaders in practice; the leader redirect handles any grouped configuration without losing counts.Changes in V4:
- No changes from V3.Changes in V3:
- Useevsel__pmu_name(evsel)instead ofstrstarts(evsel->name, "htm")to identify HTM events, matching by kernel-assigned PMU name rather than user-visible event name.
- Remove the redundant two-pass loop (first pass to setfound_htm, second to accumulate counts); a single pass withevsel__pmu_name()is sufficient. if no HTM event existstotal_pending_recordsstays 0 and the function returns 0.
- Remove the dead !strcmp(evsel->name, "dummy:u") check;
-evsel__pmu_name()will never return "htm" for a dummy:u software event.
- Renametotal_pending_bytes->total_pending_recordsto match what the driver actually reports (event->count=total_size/record_size, a record count, not a byte count).
- Add #include <string.h> for musl compatibility (strcmp()without it warns on some toolchains).Changes in V2:
- Implements the renamedarch_perf_record__need_read()hook (V1 implementedarch_record__collect_final_data()).
- Skips evsels whose fd and sample-id xyarrays are mismatched, avoiding stale-state reads. V1 had no such guard.
-evlist__enablecycling is removed; that responsibility now belongs to the drain loop inbuiltin-record.cadded in patch 3.
- File location changed toarch/powerpc/util/evsel.c(V1 usedarch/powerpc/util/powerpc-htm.c).
- Patch is now 4/6 instead of 4/9.
tools/perf/arch/powerpc/util/evsel.c| 77 ++++++++++++++++++++++++++++1 file changed, 77 insertions(+)tools/perf/arch/powerpc/util/evsel.c +77
-0@@ -1,8 +1,85 @@// SPDX-License-Identifier: GPL-2.0 #include <stdio.h> +#include <string.h> +#include <unistd.h> +#include <linux/string.h> #include "util/evsel.h" +#include "util/record.h" +#include "util/evlist.h" +#include "util/debug.h" +#include <internal/xyarray.h> +#include <internal/lib.h> void arch_evsel__set_sample_weight (struct evsel *evsel) {evsel__set_sample_bit (evsel,WEIGHT_STRUCT ); } + +/* +* powerpc implementation of arch_perf_record__need_read(). +* +* Reads event->count for every open HTM evsel by issuing a direct +* read() on the event fd with a plain u64 buffer, bypassing the +* PERF_FORMAT_GROUP path in perf_evsel__read(). When an HTM evsel is +* a group sibling, evsel__config() sets PERF_FORMAT_GROUP on its attr; +* perf_evsel__read() would then call perf_evsel__read_group() which +* sizes the buffer by evsel->nr_members (0 for siblings), causing the +* kernel to return -ENOSPC. Reading the fd directly with sizeof(u64) +* retrieves the HTM driver's plain pending-record count regardless of +* group membership. +* +* Returns: 1 if more data exists, 0 if collection is complete +*/ +int arch_perf_record__need_read (struct evlist *evlist) +{ +struct evsel *evsel; +u64 total_pending_records =0 ; +int x, y; + +/* there was an error during record__open */ +if (!evlist) +return 0 ; + +/* Read HTM event counts to check if more data is available */ +evlist__for_each_entry (evlist, evsel) { +struct perf_evsel *rd_evsel; +struct xyarray *xy; + +if (strcmp (evsel__pmu_name (evsel),"htm" )) +continue ; + +/* +* For group siblings nr_members == 0, which makes +* perf_evsel__read_size() return 0 and readn() fail. +* Read through the leader instead; perf_evsel__read_group() +* extracts the leader's own count from the group buffer. +*/ +if (evsel->core.leader != &evsel->core) + rd_evsel = evsel->core.leader; +else + rd_evsel = &evsel->core; + + xy = rd_evsel->sample_id; + +if (xy ==NULL || rd_evsel->fd ==NULL ) +continue ; + +if (xyarray__max_x (rd_evsel->fd) !=xyarray__max_x (xy) || +xyarray__max_y (rd_evsel->fd) !=xyarray__max_y (xy)) { +pr_debug ("Unmatched FD vs sample ID array for HTM event\n" ); +continue ; + } + +for (x =0 ; x <xyarray__max_x (xy); x++) { +for (y =0 ; y <xyarray__max_y (xy); y++) { +struct perf_counts_values count = { .val =0 }; + +if (perf_evsel__read (rd_evsel, x, y, &count) ==0 ) + total_pending_records += count.val; + } + } + } + +/* Collection is complete only when ALL hardware queues have no pending records */ +return (total_pending_records >0 ) ?1 :0 ; +}
Ian Rogers
reviewerQuoted 17 lines · Athira Rajeev
Quoted 12 lines · Athira Rajeev
Overview:
This series adds perf tool support for capturing and processing Hardware Trace Macro (HTM) trace data on POWER systems. HTM exposes raw bus-trace data via theH_HTMhypervisor call. The kernel-side HTM PMU driver (submitted separately) streams this data into perf AUX buffers and emits memory configuration asPERF_SAMPLE_RAWrecords.Quoted 4 lines
This series wires up the record and report sides in the perf tool so that 'perf record' collects HTM data and 'perf report' writes the trace and memory configuration to files ready for post-processing with htmdecode.
Quoted 4 lines
Hi All,
I am currently working on addressing Sashiko's review comments for V5 of the patch series.
Before I proceed with posting V6, requesting for any feedback or suggestions that I should incorporate in V6, apart from Sashiko's comments.
Hi Athira,
Did you get what you needed for V6? I see Adrian's reviews.
Thanks,
Ian
Quoted 131 lines
Thanks
AthiraQuoted 127 lines
Patch overview:
Patch 1 refactors the existing powerpc auxtrace dispatch. All VPA-DTL recording logic is moved fromarch/powerpc/util/auxtrace.cinto its own file,arch/powerpc/util/vpa-dtl.c, leavingauxtrace_record__init()as a thin per-PMU dispatcher. This is a prerequisite for adding the HTM recording path cleanly.Patch 2 adds the HTM recording path (
arch/powerpc/util/htm.candutil/powerpc-htm.h).htm_recording_init()sets up theauxtrace_recordcallbacks.htm_info_fill()stores the PMU type and a (cpu, attr.config) pair for each htm evsel into thePERF_RECORD_AUXTRACE_INFOpriv[] area. The decode side reads these back to map each AUX buffer to its (node, chip, core) target usingevent->auxtrace.cpu.PERF_SAMPLE_RAWis also enabled here so that memory configuration records are captured alongside the AUX stream.Patch 3 adds a generic weak
arch_perf_record__need_read()hook inbuiltin-record.c. When the hook returns non-zero, perf record performs an extra mmap-read pass before disabling events. The drain loop runs until the hook returns 0 or no forward progress is made, then the events are disabled and closed normally.Patch 4 implements
arch_perf_record__need_read()for powerpc. It readsevent->countfor all open htm evsels viaperf_evsel__read()and accumulates the values. The kernel driver sets this count to the number of 128-byte HTM trace records written on a successful dump, 1 when the AUX buffer is temporarily full but the hypervisor stream is intact, and 0 once the stream is exhausted or a hard error occurs. A non-zero total causes the recording loop to perform another read pass.Patch 5 adds
PERF_AUXTRACE_POWERPC_HTMto theauxtrace_typeenum and wires the dispatch inperf_event__process_auxtrace_info(). It also amendshtm_info_fill()to setauxtrace_info->typenow that the constant is available.Patch 6 adds the full HTM decode path in
util/powerpc-htm.c.powerpc_htm_process_auxtrace_info()reads the (cpu, config) pairs from priv[] and builds a cpu_configs[] lookup table. As eachPERF_RECORD_AUXTRACEevent arrives,process_auxtrace_event()maps it to the correct target and writes the data to htm.bin.nX.pX.cX.tX immediately.PERF_RECORD_SAMPLERAW records are written to translation.nX.pX.cX.tX byprocess_event(). Per-run first-write tracking (htm_target_seen) ensuresO_TRUNCon the first write andO_APPENDon subsequent writes.Patches not included in V2:
Three patches present in V1 are intentionally dropped from this series and will be submitted as a follow-on once the core infrastructure gets integrated:- htmdecode integration: V1 patch 6 invoked the external htmdecode tool via fork/exec after writing htm.bin.* to decode the raw bus-trace data in-process. This is deferred because the interface between perf and an external decode tool needs more discussion.
- Physical-to-logical address mapping: V1 patch 7 read the current partition ID from
/proc/powerpc/lparcfg, parsed the translation.* memory map entries, and mapped physical addresses in the decoded trace to logical addresses within the LPAR.- Synthetic sample generation: V1 patches 8 and 9 created a
PERF_TYPE_SYNTHevent named "htm", injected synthetic perf samples with the translated logical addresses as sample IP, and wrote a separate .out.l file with logical addresses alongside the decoded trace. The appropriate abstraction for synthetic HTM samples in the perf tool requires further discussion.Usage example:
Collect HTM trace data from a single chip target:# perf record -C 9 -m,256 \ -e htm/nodalchipindex=2,nodeindex=0,htm_type=1/ sleep 3 [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 256.277 MB perf.data ]# perf reportOutput files written by perf report:
htm.bin.n0.p2.c0.t1 raw HTM bus-trace data
translation.n0.p2.c0.t1 memory configuration recordsMultiple targets can be collected simultaneously:
# perf record -m,256 \ -e htm/nodalchipindex=2,nodeindex=0,htm_type=1,cpu=8/ \ -e htm/nodalchipindex=1,nodeindex=0,htm_type=1,cpu=9/ \ -a sleep 10Testing:
Tested on POWER11. Both htm.bin.* and translation.* files are produced correctly for single and dual target collection.Note: Link to kernel patches:
https://lore.kernel.org/linuxppc-dev/[email protected]/T/#tAthira Rajeev (6):
tools/perf: Move powerpc VPA-DTL auxtrace init into a separate filetools/perf: Add AUXTRACE recording support for powerpc HTMtools/perf: Add arch hook to drain remaining data before event closetools/perf: Add powerpc callback support for
arch_perf_record__need_read
tools/perf: Add powerpc HTM auxtrace event processing supporttools/perf: Add perf tool support for processing powerpc HTM AUXTRACE
recordstools/perf/arch/powerpc/util/Build | 2 + tools/perf/arch/powerpc/util/auxtrace.c | 98 ++---- tools/perf/arch/powerpc/util/evsel.c | 77 ++++ tools/perf/arch/powerpc/util/htm.c | 185 ++++++++++ tools/perf/arch/powerpc/util/vpa-dtl.c | 96 +++++ tools/perf/builtin-record.c | 65 ++++ tools/perf/util/Build | 1 + tools/perf/util/auxtrace.c | 4 + tools/perf/util/auxtrace.h | 1 + tools/perf/util/powerpc-htm.c | 447 ++++++++++++++++++++++++ tools/perf/util/powerpc-htm.h | 43 +++ tools/perf/util/powerpc-vpadtl.h | 3 + tools/perf/util/record.h | 4 + 13 files changed, 954 insertions(+), 72 deletions(-)create mode 100644
tools/perf/arch/powerpc/util/htm.ccreate mode 100644tools/perf/arch/powerpc/util/vpa-dtl.ccreate mode 100644tools/perf/util/powerpc-htm.ccreate mode 100644tools/perf/util/powerpc-htm.h--
2.53.0
Re: [PATCH V5 4/6] tools/perf: Add powerpc callback support for arch_perf_record__need_read
In reply to Athira RajeevQuoted 92 lines · Athira Rajeev
Quoted 76 lines · Adrian Hunter
Quoted 64 lines · Athira Rajeev
Quoted 45 lines · Adrian Hunter
Quoted 32 lines · Athira Rajeev
Quoted 32 lines · Adrian Hunter
Quoted 17 lines · Athira Rajeev
Implement the
arch_perf_record__need_read()architecture-specific hook for powerpc inarch/powerpc/util/evsel.c.The HTM kernel driver sets
event->countto the number of records still staged in its internal buffers (total_size/record_size), and to 0 once the stream is exhausted. This hook reads that count for every open htm evsel viaperf_evsel__read()and accumulates the values intototal_pending_records. A non-zero total means at least one HTM target still has records pending; the recording loop added in the previous patch will perform another mmap-read pass.The drain uses a two-layer safety check:
event->countdetects records staged by the driver, andrecord__bytes_written()in the drain loop confirms data was actually moved into perf.data. This combination handles the case where the driver count is briefly stale while hardware is still flushing.Quoted 7 lines
The implementation scans the evlist using
evsel__pmu_name()to identify HTM events by their kernel-assigned PMU name rather than the user-visible event name, preventing false matches. It iterates the fd/ sample-id xyarray, and skips any evsel whose fd and sample-id arrays are mismatched to avoid reading stale state. When the accumulated record count reaches zero the hook returns 0 and the recording loop proceeds to disable and close the events.This looks like the proposed driver:
Quoted 7 lines
Quoted 1 line
Hi Adrian,
Thank you for the review.Could you please help clarify which specific aspect of the driver breaks the perf ABI? Is it the sysfs format ABI where encoding raw hardware topology identifiers (nodeindex, nodalchipindex, coreindexonchip) are used as perf_event_attr.config bit fields? I am trying to confirm so that I get your feedback correctly for reworking on changes.
Quoted 6 lines
This "draining" and "read" usage looked unusual, so I asked AI:
I have just applied the patches for powerpc htm kernel driver - they are now the last 5 patches committed. Examine the design in comparison to how other PMU drivers are implemented. Does it violate the kernel API for PMUs?
It said yes.
Hi Adrian,
Thanks for taking the time to check this.I agree with the assessment. The core problems I understand are:
-pmu->read()issues hcalls that stop the hardware trace and dump the data, so anyread()call ends up silently draining and stopping the trace — which isn't whatread()is for.
-event->countis being used to signal "more data pending" which breaks the expected semantics.I'm going to rework this rather than continue posting on top of it. Here is my rework plan for your review:
Kernel side: workqueue-based approach:
Review
Quoted 5 lines
(quoted diff) +0
-2- pmu->read() becomes a no-op. No hcalls, no side effects. - pmu->stop() issues hypervisor call to freeze the hardware buffer, then schedules a work_struct (htm_drain_work_fn()) that runs in process context and drains the frozen buffer chunk by chunk into the AUX ring via perf_aux_output_end(), backing off on -ENOSPC to let userspace consume between chunks. event->count is no longer used as a flow-control signal.Quoted 3 lines
That sounds a bit like
pmu->stop()wouldn't actually stop. I can imagine there could be issues with that. I suspect only someone like Peter Zijlstra can advise you.Sure, thanks for directing Adrian,
Adding Peter Zijlstra to the thread..Hi Peter,
I am working on a perf PMU driver for HTM (Hardware Trace Macro) on
IBM POWER systems. HTM captures hardware trace data via theH_HTMhypervisor call. The driver in the patch series exposes it as a perf AUX PMU so trace data can be correlated with other perf events in perf.data.
Quoted 9 lines
Patch series: https://lore.kernel.org/linux-perf-users/[email protected]/
https://lore.kernel.org/linux-perf-users/[email protected]/Adrian Hunter reviewed the series and we have some queries. I am reaching out for your guidance on the right kernel mechanism for draining trace data into the perf AUX buffer. Please read details below and help share your thoughts. Apologies for the long text, I tried to explain context and concerns that came out of the review.
Hi Peter,
Gentle ping on this. Sorry to bother you again. Just wanted to check if you could a chance to read through my query and if had any thoughts or if you need any additional context from my side.
Thanks,
Athira
Quoted 320 lines
Hardware constraints
--------------------
HTM is a one-shot, firmware-managed trace buffer:- Buffer size is fixed at boot time.
Cannot be changed without a reboot.
- No hardware interrupt (no PMI, no DMA completion signal).
- The entire buffer is read after issuing a stop hcall. Not while tracing is active.
- Buffer must be copied chunk by chunk via a hypervisor hcall into the perf AUX ring (physically contiguous pages).What V5 of patch series did
-----------
In V5,pmu->read()did all the work:- On the first call: issued stop hcall to freeze the hardware buffer, then called hcall to dump the trace data into the AUX ring via
perf_aux_output_begin()/perf_aux_output_end().
- Subsequent calls: skipped the stop (already frozen), copied the next chunk, advancedaux_buf->head.
- Setevent->countto the record count written, or 1 if the AUX ring was full (-ENOSPC, meaning "retry"), or 0 at EOF.
- Userspace polledevent->countviaperf_evsel__read()in a new weakarch_perf_record__need_read()hook added tobuiltin-record.c; a non-zero count triggered anotherrecord__mmap_read_all()+ retry.Review from Adrian flagged concerns with this:
Concern 1:
pmu->read()issues stop on its first call —a hardware side effect inside pmu->read().Concern 2:
event->countis used as a flow-control signal
(1 = "retry", 0 = "done") rather than a real counter.Addressing the two concerns for V6
-----------------------------------
Concern 2 is straightforward to fix: remove allevent->countmanipulation frompmu->read(). Loop termination is detected by AUX head not advancing. Noevent->countneeded.Concern 1 is harder. The query I have is:
Is it acceptable for
pmu->read()to issue stop hcall exactly once (on the first drain call, not on all subsequent calls), then copy trace data chunk viaperf_aux_output_begin()/ dump trace data /perf_aux_output_end()?The reason for considering this approach for HTM specifically:
-
htm_event_init()rejects all non-sampling opens with-EOPNOTSUPP:if (!is_sampling_event(event)) return -EOPNOTSUPP;So perf stat, BPF helpers, and any other non-record caller get
-EOPNOTSUPPatperf_event_open()time. An HTM fd can only exist inside a perf record session. The only caller of read(fd) on an HTM event is from read hook - HTM's own drain code — which calls it deliberately, knowing stop fires on the first call. There is no unsuspecting caller path for HTM.- After the first call,
pmu->read()is a pure data-mover with no further hardware side effects.Alternative: workqueue after
pmu->stop()
-----------------------------------------
Ifpmu->read()having any hardware side effect is not right path, the alternative I was proposing is a workqueue-based approach:-
pmu->read()becomes a true no-op.
-pmu->stop()issues hcall stop operation to freeze the hardware buffer, then schedules awork_struct(sayhtm_drain_work_fn()) that runs in process context and drains the frozen buffer chunk by chunk into the AUX ring viaperf_aux_output_begin()/perf_aux_output_end(), backing off on-ENOSPCbetween chunks.
-event->countis not used as a flow-control signal.
- deconfigure the tracing ( releasing the resources ) inevent->destroy()Tools side:
-arch_perf_record__need_read()hook is dropped — no changes tobuiltin-record.c.
- A custom "read_finish" callback in the HTM "auxtrace_record" pollsauxtrace_mmap__read_head()until the AUX head stops advancing, with a bounded maximum wait. This ensures the workqueue drain completes beforeevlist__close()frees the mmaps.Adrian's concern with this approach: "That sounds like
pmu->stop()wouldn't actually stop.”Traced through the tools code, workqueue lifetime looks safe to me based on this teardown sequence:
main poll loop exits
| v
out_child: (builtin-record.c)
record__mmap_read_all(rec, true)<- final AUX ring drain
|
| <-read_finishcalled here,
| fd still open at this point
|perf_session__delete()|
v
return from__cmd_record()| v cmd_record(): evlist__delete(rec->evlist) +-- evlist__close() +-- close(*fd) <- fd closed here -> release the resourcesSo
event->destroyfires after "read_finish" .cancel_work_sync()inevent->destroyensures the drain workqueue finishes before releasing the hypervisor buffer. The fd is still open whenread_finishruns, so it can safely wait for the drain.Peterz, I have tried to summarize these below and I am seeking for your guidance here. Please share your thoughts.
--------------------
1. Is it acceptable forpmu->read()to issue stop tracing once on the first drain call, given that HTM events can only be opened by perf record (non-sampling opens rejected inhtm_event_init()), so there is no unsuspecting caller path?2. If not, is the workqueue-after-pmu->stop() approach safe? Releasing the resources ( deconfigure ) will be placed in
event->destroy(notpmu->del()), which fires only at close(fd) — aftercancel_work_sync()ensures the workqueue is done. The AUX mmaps also remain valid untilevlist__munmap()at that same point. So the workqueue can callperf_aux_output_begin()/perf_aux_output_end()safely because the hypervisor buffer and AUX mmaps are still live when it runs. Is this reasoning correct, or does the perf core free AUX resources (free_aux/ rb teardown) earlier thanevent->destroy?3. Is there a third approach we can go with ? please suggest if there is another way.
Thank you,
Athira RajeevQuoted 174 lines
Quoted 174 lines
Tools side:
(quoted diff) +0
-3- arch_perf_record__need_read() hook and record__final_aux_data() are dropped entirely — no changes to builtin-record.c. - “read_finish" callback is implemented in the HTM auxtrace_record: During normal recording (evsel->disabled == false) it returns immediately. At session teardown, evlist__disable() sets evsel->disabled = true before the final record__mmap_read_all() call. The read_finish callback detects this and polls aux_head (via perf_mmap__read_head()) in a loop, yielding delay between polls, until the AUX head stops advancing — signalling the workqueue drain is complete. This gives the kernel workqueue time to finish copying chunks before evlist__close() frees the mmaps. The wait has a bounded maximum, so a stalled drain can't hang perf record indefinitely. - snapshot_finish returns -EINVAL since HTM does not support snapshot mode.HTM has no interrupt and trace data is consumed after pmu stop . Hence making this
read_finishcallback forauxtrace_recordto consume data.With this, data collection is moved of from .read() entirely and uses work queue based approach. Can you please review , Does this approach look acceptable before I send the updated series?
Thanks,
AthiraQuoted 149 lines
Quoted 149 lines
Thanks,
AthiraQuoted 144 lines
breaks the perf ABI.
I am not going to review any more tools patches for now.
Quoted 139 lines
Signed-off-by Athira Rajeev <[email protected]>---
Changes in V5:
- When an HTM evsel is a group sibling (evsel->core.leader!=&evsel->core), read through its group leader's structperf_evselinstead of the sibling directly.perf_evsel__read_size()usesevsel->nr_membersto compute the read buffer size;nr_membersis 0 for siblings, so size=0 is passed toreadn(), which returns <=0 and leaves count.val=0, causing the drain loop to terminate prematurely. Reading through the leader avoids the zero-size buffer and correctly accumulates the leader's pending count. HTM events are always standalone or per-target leaders in practice; the leader redirect handles any grouped configuration without losing counts.Changes in V4:
- No changes from V3.Changes in V3:
- Useevsel__pmu_name(evsel)instead ofstrstarts(evsel->name, "htm")to identify HTM events, matching by kernel-assigned PMU name rather than user-visible event name.
- Remove the redundant two-pass loop (first pass to setfound_htm, second to accumulate counts); a single pass withevsel__pmu_name()is sufficient. if no HTM event existstotal_pending_recordsstays 0 and the function returns 0.
- Remove the dead !strcmp(evsel->name, "dummy:u") check;
-evsel__pmu_name()will never return "htm" for a dummy:u software event.
- Renametotal_pending_bytes->total_pending_recordsto match what the driver actually reports (event->count=total_size/record_size, a record count, not a byte count).
- Add #include <string.h> for musl compatibility (strcmp()without it warns on some toolchains).Changes in V2:
- Implements the renamedarch_perf_record__need_read()hook (V1 implementedarch_record__collect_final_data()).
- Skips evsels whose fd and sample-id xyarrays are mismatched, avoiding stale-state reads. V1 had no such guard.
-evlist__enablecycling is removed; that responsibility now belongs to the drain loop inbuiltin-record.cadded in patch 3.
- File location changed toarch/powerpc/util/evsel.c(V1 usedarch/powerpc/util/powerpc-htm.c).
- Patch is now 4/6 instead of 4/9.
tools/perf/arch/powerpc/util/evsel.c| 77 ++++++++++++++++++++++++++++1 file changed, 77 insertions(+)tools/perf/arch/powerpc/util/evsel.c +77
-0@@ -1,8 +1,85 @@// SPDX-License-Identifier: GPL-2.0 #include <stdio.h> +#include <string.h> +#include <unistd.h> +#include <linux/string.h> #include "util/evsel.h" +#include "util/record.h" +#include "util/evlist.h" +#include "util/debug.h" +#include <internal/xyarray.h> +#include <internal/lib.h> void arch_evsel__set_sample_weight (struct evsel *evsel) {evsel__set_sample_bit (evsel,WEIGHT_STRUCT ); } + +/* +* powerpc implementation of arch_perf_record__need_read(). +* +* Reads event->count for every open HTM evsel by issuing a direct +* read() on the event fd with a plain u64 buffer, bypassing the +* PERF_FORMAT_GROUP path in perf_evsel__read(). When an HTM evsel is +* a group sibling, evsel__config() sets PERF_FORMAT_GROUP on its attr; +* perf_evsel__read() would then call perf_evsel__read_group() which +* sizes the buffer by evsel->nr_members (0 for siblings), causing the +* kernel to return -ENOSPC. Reading the fd directly with sizeof(u64) +* retrieves the HTM driver's plain pending-record count regardless of +* group membership. +* +* Returns: 1 if more data exists, 0 if collection is complete +*/ +int arch_perf_record__need_read (struct evlist *evlist) +{ +struct evsel *evsel; +u64 total_pending_records =0 ; +int x, y; + +/* there was an error during record__open */ +if (!evlist) +return 0 ; + +/* Read HTM event counts to check if more data is available */ +evlist__for_each_entry (evlist, evsel) { +struct perf_evsel *rd_evsel; +struct xyarray *xy; + +if (strcmp (evsel__pmu_name (evsel),"htm" )) +continue ; + +/* +* For group siblings nr_members == 0, which makes +* perf_evsel__read_size() return 0 and readn() fail. +* Read through the leader instead; perf_evsel__read_group() +* extracts the leader's own count from the group buffer. +*/ +if (evsel->core.leader != &evsel->core) + rd_evsel = evsel->core.leader; +else + rd_evsel = &evsel->core; + + xy = rd_evsel->sample_id; + +if (xy ==NULL || rd_evsel->fd ==NULL ) +continue ; + +if (xyarray__max_x (rd_evsel->fd) !=xyarray__max_x (xy) || +xyarray__max_y (rd_evsel->fd) !=xyarray__max_y (xy)) { +pr_debug ("Unmatched FD vs sample ID array for HTM event\n" ); +continue ; + } + +for (x =0 ; x <xyarray__max_x (xy); x++) { +for (y =0 ; y <xyarray__max_y (xy); y++) { +struct perf_counts_values count = { .val =0 }; + +if (perf_evsel__read (rd_evsel, x, y, &count) ==0 ) + total_pending_records += count.val; + } + } + } + +/* Collection is complete only when ALL hardware queues have no pending records */ +return (total_pending_records >0 ) ?1 :0 ; +}
Re: [PATCH V5 4/6] tools/perf: Add powerpc callback support for arch_perf_record__need_read
Quoted 105 lines · Athira Rajeev
Quoted 93 lines · Athira Rajeev
Quoted 76 lines · Adrian Hunter
Quoted 64 lines · Athira Rajeev
Quoted 45 lines · Adrian Hunter
Quoted 32 lines · Athira Rajeev
Quoted 32 lines · Adrian Hunter
Quoted 17 lines · Athira Rajeev
Implement the
arch_perf_record__need_read()architecture-specific hook for powerpc inarch/powerpc/util/evsel.c.The HTM kernel driver sets
event->countto the number of records still staged in its internal buffers (total_size/record_size), and to 0 once the stream is exhausted. This hook reads that count for every open htm evsel viaperf_evsel__read()and accumulates the values intototal_pending_records. A non-zero total means at least one HTM target still has records pending; the recording loop added in the previous patch will perform another mmap-read pass.The drain uses a two-layer safety check:
event->countdetects records staged by the driver, andrecord__bytes_written()in the drain loop confirms data was actually moved into perf.data. This combination handles the case where the driver count is briefly stale while hardware is still flushing.Quoted 7 lines
The implementation scans the evlist using
evsel__pmu_name()to identify HTM events by their kernel-assigned PMU name rather than the user-visible event name, preventing false matches. It iterates the fd/ sample-id xyarray, and skips any evsel whose fd and sample-id arrays are mismatched to avoid reading stale state. When the accumulated record count reaches zero the hook returns 0 and the recording loop proceeds to disable and close the events.This looks like the proposed driver:
Quoted 7 lines
Quoted 1 line
Hi Adrian,
Thank you for the review.Could you please help clarify which specific aspect of the driver breaks the perf ABI? Is it the sysfs format ABI where encoding raw hardware topology identifiers (nodeindex, nodalchipindex, coreindexonchip) are used as perf_event_attr.config bit fields? I am trying to confirm so that I get your feedback correctly for reworking on changes.
Quoted 6 lines
This "draining" and "read" usage looked unusual, so I asked AI:
I have just applied the patches for powerpc htm kernel driver - they are now the last 5 patches committed. Examine the design in comparison to how other PMU drivers are implemented. Does it violate the kernel API for PMUs?
It said yes.
Hi Adrian,
Thanks for taking the time to check this.I agree with the assessment. The core problems I understand are:
-pmu->read()issues hcalls that stop the hardware trace and dump the data, so anyread()call ends up silently draining and stopping the trace — which isn't whatread()is for.
-event->countis being used to signal "more data pending" which breaks the expected semantics.I'm going to rework this rather than continue posting on top of it. Here is my rework plan for your review:
Kernel side: workqueue-based approach:
Review
Quoted 5 lines
(quoted diff) +0
-2- pmu->read() becomes a no-op. No hcalls, no side effects. - pmu->stop() issues hypervisor call to freeze the hardware buffer, then schedules a work_struct (htm_drain_work_fn()) that runs in process context and drains the frozen buffer chunk by chunk into the AUX ring via perf_aux_output_end(), backing off on -ENOSPC to let userspace consume between chunks. event->count is no longer used as a flow-control signal.Quoted 3 lines
That sounds a bit like
pmu->stop()wouldn't actually stop. I can imagine there could be issues with that. I suspect only someone like Peter Zijlstra can advise you.Sure, thanks for directing Adrian,
Adding Peter Zijlstra to the thread..Hi Peter,
I am working on a perf PMU driver for HTM (Hardware Trace Macro) on
IBM POWER systems. HTM captures hardware trace data via theH_HTMhypervisor call. The driver in the patch series exposes it as a perf AUX PMU so trace data can be correlated with other perf events in perf.data.Quoted 9 lines
Patch series: https://lore.kernel.org/linux-perf-users/[email protected]/
https://lore.kernel.org/linux-perf-users/[email protected]/Adrian Hunter reviewed the series and we have some queries. I am reaching out for your guidance on the right kernel mechanism for draining trace data into the perf AUX buffer. Please read details below and help share your thoughts. Apologies for the long text, I tried to explain context and concerns that came out of the review.
Quoted 7 lines
Hi Peter,
Gentle ping on this. Sorry to bother you again. Just wanted to check if you could a chance to read through my query and if had any thoughts or if you need any additional context from my side.
Thanks,
Athira
Hi Ian, Peter, and community,
Hi Ian,
Thanks for checking in. On the tools side, I have the changes planned to address Adrian's comments (dropping the custom arch_perf_record__need_read() hook from builtin-record.c and removing event->count flow-control usage). However, on the kernel driver side, I am looking for feedback and guidance regarding the right mechanism for draining the HTM trace buffer into the AUX ring after pmu->stop().
Hi Peter and community,
Following up on the discussion regarding the kernel driver side in this thread [1]. Requesting for recommendations on the design (specifically regarding handling post-stop AUX buffer draining for interrupt-less, firmware-managed hardware buffers via a workqueue vs other approaches).
Brief recap of the problem:
- Hardware constraint: PowerPC HTM is a one-shot, firmware-managed buffer with no hardware interrupts. Trace data can only be read chunk-by-chunk via hypervisor hcalls *after* issuing a stop hcall.
What V5 of the patch series did:
--------------------------------
In V5, pmu->read() did all the work:
- On the first call: issued stop hcall to freeze the hardware buffer, then called hcall to dump the trace data into the AUX ring via perf_aux_output_begin() / perf_aux_output_end().
- Subsequent calls: skipped the stop (already frozen), copied the next chunk, and advanced aux_buf->head.
- Set event->count to the record count written, or 1 if the AUX ring was full (-ENOSPC, meaning "retry"), or 0 at EOF.
- Userspace polled event->count via perf_evsel__read() in a new weak arch_perf_record__need_read() hook added to builtin-record.c; a non-zero count triggered another record__mmap_read_all() + retry.
Review from Adrian flagged concerns with this:
- Concern 1: pmu->read() issues stop on its first call — a hardware side effect inside pmu->read().
- Concern 2: event->count is used as a flow-control signal (1 = "retry", 0 = "done") rather than a real counter.
Proposed design for V6:
-----------------------
1. pmu->read() will become a clean no-op.
2. pmu->stop() freezes the trace buffer, then schedules a process-context workqueue (e.g. htm_drain_work_fn) to pump chunks into the AUX ring via perf_aux_output_begin() / perf_aux_output_end().
3. Deconfiguration and cancel_work_sync() are placed in event->destroy() (at close(fd)), keeping AUX mmaps and buffers alive during the drain.
4. Tools side uses auxtrace_record->read_finish() to wait for aux_head to settle before closing event fds.
Queries:
1. Is scheduling drain work from pmu->stop() and synchronizing in event->destroy() an acceptable pattern here?
2. Releasing the resources (deconfigure) will be placed in event->destroy() (not pmu->del()), which fires only at close(fd) — after cancel_work_sync() ensures the workqueue is done. The AUX mmaps also remain valid until evlist__munmap() at that same point. So the workqueue can call perf_aux_output_begin() / perf_aux_output_end() safely because the hypervisor buffer and AUX mmaps are still live when it runs. Is this reasoning correct, or does perf core free AUX resources (free_aux / rb teardown) earlier than event->destroy()?
3. If not, is there another preferred pattern in perf core that can be used for this case?
It would be really helpful to get some pointers or guidance on this for posting V6. Please share thoughts or any suggestions on this.
Thanks,
Athira Rajeev
[1] https://lore.kernel.org/linux-perf-users/[email protected]/
Quoted 320 lines
Quoted 320 lines
Hardware constraints
--------------------
HTM is a one-shot, firmware-managed trace buffer:- Buffer size is fixed at boot time.
Cannot be changed without a reboot.
- No hardware interrupt (no PMI, no DMA completion signal).
- The entire buffer is read after issuing a stop hcall. Not while tracing is active.
- Buffer must be copied chunk by chunk via a hypervisor hcall into the perf AUX ring (physically contiguous pages).What V5 of patch series did
-----------
In V5,pmu->read()did all the work:- On the first call: issued stop hcall to freeze the hardware buffer, then called hcall to dump the trace data into the AUX ring via
perf_aux_output_begin()/perf_aux_output_end().
- Subsequent calls: skipped the stop (already frozen), copied the next chunk, advancedaux_buf->head.
- Setevent->countto the record count written, or 1 if the AUX ring was full (-ENOSPC, meaning "retry"), or 0 at EOF.
- Userspace polledevent->countviaperf_evsel__read()in a new weakarch_perf_record__need_read()hook added tobuiltin-record.c; a non-zero count triggered anotherrecord__mmap_read_all()+ retry.Review from Adrian flagged concerns with this:
Concern 1:
pmu->read()issues stop on its first call —a hardware side effect inside pmu->read().Concern 2:
event->countis used as a flow-control signal
(1 = "retry", 0 = "done") rather than a real counter.Addressing the two concerns for V6
-----------------------------------
Concern 2 is straightforward to fix: remove allevent->countmanipulation frompmu->read(). Loop termination is detected by AUX head not advancing. Noevent->countneeded.Concern 1 is harder. The query I have is:
Is it acceptable for
pmu->read()to issue stop hcall exactly once (on the first drain call, not on all subsequent calls), then copy trace data chunk viaperf_aux_output_begin()/ dump trace data /perf_aux_output_end()?The reason for considering this approach for HTM specifically:
-
htm_event_init()rejects all non-sampling opens with-EOPNOTSUPP:if (!is_sampling_event(event)) return -EOPNOTSUPP;So perf stat, BPF helpers, and any other non-record caller get
-EOPNOTSUPPatperf_event_open()time. An HTM fd can only exist inside a perf record session. The only caller of read(fd) on an HTM event is from read hook - HTM's own drain code — which calls it deliberately, knowing stop fires on the first call. There is no unsuspecting caller path for HTM.- After the first call,
pmu->read()is a pure data-mover with no further hardware side effects.Alternative: workqueue after
pmu->stop()
-----------------------------------------
Ifpmu->read()having any hardware side effect is not right path, the alternative I was proposing is a workqueue-based approach:-
pmu->read()becomes a true no-op.
-pmu->stop()issues hcall stop operation to freeze the hardware buffer, then schedules awork_struct(sayhtm_drain_work_fn()) that runs in process context and drains the frozen buffer chunk by chunk into the AUX ring viaperf_aux_output_begin()/perf_aux_output_end(), backing off on-ENOSPCbetween chunks.
-event->countis not used as a flow-control signal.
- deconfigure the tracing ( releasing the resources ) inevent->destroy()Tools side:
-arch_perf_record__need_read()hook is dropped — no changes tobuiltin-record.c.
- A custom "read_finish" callback in the HTM "auxtrace_record" pollsauxtrace_mmap__read_head()until the AUX head stops advancing,
with a bounded maximum wait. This ensures the workqueue drain completes beforeevlist__close()frees the mmaps.Adrian's concern with this approach: "That sounds like
pmu->stop()wouldn't actually stop.”Traced through the tools code, workqueue lifetime looks safe to me based on this teardown sequence:
main poll loop exits
|
vout_child: (builtin-record.c)
record__mmap_read_all(rec, true)<- final AUX ring drain| | <- read_finish called here, | fd still open at this point |
perf_session__delete()|
v
return from__cmd_record()
|
vcmd_record():evlist__delete(rec->evlist) +-- evlist__close() +-- close(*fd) <- fd closed here -> release the resourcesSo
event->destroyfires after "read_finish" .cancel_work_sync()inevent->destroyensures the drain workqueue finishes before releasing the hypervisor buffer. The fd is still open whenread_finishruns, so it can safely wait for the drain.Peterz, I have tried to summarize these below and I am seeking for your guidance here. Please share your thoughts.
--------------------
1. Is it acceptable forpmu->read()to issue stop tracing once on the first drain call, given that HTM events can only be opened by perf record (non-sampling opens rejected inhtm_event_init()), so there is no unsuspecting caller path?2. If not, is the workqueue-after-pmu->stop() approach safe? Releasing the resources ( deconfigure ) will be placed in
event->destroy(notpmu->del()), which fires only at close(fd) — aftercancel_work_sync()ensures the workqueue is done. The AUX mmaps also remain valid untilevlist__munmap()at that same point. So the workqueue can callperf_aux_output_begin()/perf_aux_output_end()safely because the hypervisor buffer and AUX mmaps are still live when it runs. Is this reasoning correct, or does the perf core free AUX resources (free_aux/ rb teardown) earlier thanevent->destroy?3. Is there a third approach we can go with ? please suggest if there is another way.
Thank you,
Athira RajeevQuoted 174 lines
Quoted 174 lines
Tools side:
(quoted diff) +0
-3- arch_perf_record__need_read() hook and record__final_aux_data() are dropped entirely — no changes to builtin-record.c. - “read_finish" callback is implemented in the HTM auxtrace_record: During normal recording (evsel->disabled == false) it returns immediately. At session teardown, evlist__disable() sets evsel->disabled = true before the final record__mmap_read_all() call. The read_finish callback detects this and polls aux_head (via perf_mmap__read_head()) in a loop, yielding delay between polls, until the AUX head stops advancing — signalling the workqueue drain is complete. This gives the kernel workqueue time to finish copying chunks before evlist__close() frees the mmaps. The wait has a bounded maximum, so a stalled drain can't hang perf record indefinitely. - snapshot_finish returns -EINVAL since HTM does not support snapshot mode.HTM has no interrupt and trace data is consumed after pmu stop . Hence making this
read_finishcallback forauxtrace_recordto consume data.With this, data collection is moved of from .read() entirely and uses work queue based approach. Can you please review , Does this approach look acceptable before I send the updated series?
Thanks,
AthiraQuoted 149 lines
Quoted 149 lines
Thanks,
AthiraQuoted 144 lines
breaks the perf ABI.
I am not going to review any more tools patches for now.
Quoted 139 lines
Signed-off-by Athira Rajeev <[email protected]>---
Changes in V5:
- When an HTM evsel is a group sibling (evsel->core.leader!=&evsel->core), read through its group leader's structperf_evselinstead of the sibling directly.perf_evsel__read_size()usesevsel->nr_membersto compute the read buffer size;nr_membersis 0 for siblings, so size=0 is passed toreadn(), which returns <=0 and leaves count.val=0, causing the drain loop to terminate prematurely. Reading through the leader avoids the zero-size buffer and correctly accumulates the leader's pending count. HTM events are always standalone or per-target leaders in practice; the leader redirect handles any grouped configuration without losing counts.Changes in V4:
- No changes from V3.Changes in V3:
- Useevsel__pmu_name(evsel)instead ofstrstarts(evsel->name, "htm")to identify HTM events, matching by kernel-assigned PMU name rather than user-visible event name.
- Remove the redundant two-pass loop (first pass to setfound_htm, second to accumulate counts); a single pass withevsel__pmu_name()is sufficient. if no HTM event existstotal_pending_recordsstays 0 and the function returns 0.
- Remove the dead !strcmp(evsel->name, "dummy:u") check;
-evsel__pmu_name()will never return "htm" for a dummy:u software event.
- Renametotal_pending_bytes->total_pending_recordsto match what the driver actually reports (event->count=total_size/record_size, a record count, not a byte count).
- Add #include <string.h> for musl compatibility (strcmp()without it warns on some toolchains).Changes in V2:
- Implements the renamedarch_perf_record__need_read()hook (V1 implementedarch_record__collect_final_data()).
- Skips evsels whose fd and sample-id xyarrays are mismatched, avoiding stale-state reads. V1 had no such guard.
-evlist__enablecycling is removed; that responsibility now belongs to the drain loop inbuiltin-record.cadded in patch 3.
- File location changed toarch/powerpc/util/evsel.c(V1 usedarch/powerpc/util/powerpc-htm.c).
- Patch is now 4/6 instead of 4/9.
tools/perf/arch/powerpc/util/evsel.c| 77 ++++++++++++++++++++++++++++1 file changed, 77 insertions(+)tools/perf/arch/powerpc/util/evsel.c +77
-0@@ -1,8 +1,85 @@// SPDX-License-Identifier: GPL-2.0 #include <stdio.h> +#include <string.h> +#include <unistd.h> +#include <linux/string.h> #include "util/evsel.h" +#include "util/record.h" +#include "util/evlist.h" +#include "util/debug.h" +#include <internal/xyarray.h> +#include <internal/lib.h> void arch_evsel__set_sample_weight (struct evsel *evsel) {evsel__set_sample_bit (evsel,WEIGHT_STRUCT ); } + +/* +* powerpc implementation of arch_perf_record__need_read(). +* +* Reads event->count for every open HTM evsel by issuing a direct +* read() on the event fd with a plain u64 buffer, bypassing the +* PERF_FORMAT_GROUP path in perf_evsel__read(). When an HTM evsel is +* a group sibling, evsel__config() sets PERF_FORMAT_GROUP on its attr; +* perf_evsel__read() would then call perf_evsel__read_group() which +* sizes the buffer by evsel->nr_members (0 for siblings), causing the +* kernel to return -ENOSPC. Reading the fd directly with sizeof(u64) +* retrieves the HTM driver's plain pending-record count regardless of +* group membership. +* +* Returns: 1 if more data exists, 0 if collection is complete +*/ +int arch_perf_record__need_read (struct evlist *evlist) +{ +struct evsel *evsel; +u64 total_pending_records =0 ; +int x, y; + +/* there was an error during record__open */ +if (!evlist) +return 0 ; + +/* Read HTM event counts to check if more data is available */ +evlist__for_each_entry (evlist, evsel) { +struct perf_evsel *rd_evsel; +struct xyarray *xy; + +if (strcmp (evsel__pmu_name (evsel),"htm" )) +continue ; + +/* +* For group siblings nr_members == 0, which makes +* perf_evsel__read_size() return 0 and readn() fail. +* Read through the leader instead; perf_evsel__read_group() +* extracts the leader's own count from the group buffer. +*/ +if (evsel->core.leader != &evsel->core) + rd_evsel = evsel->core.leader; +else + rd_evsel = &evsel->core; + + xy = rd_evsel->sample_id; + +if (xy ==NULL || rd_evsel->fd ==NULL ) +continue ; + +if (xyarray__max_x (rd_evsel->fd) !=xyarray__max_x (xy) || +xyarray__max_y (rd_evsel->fd) !=xyarray__max_y (xy)) { +pr_debug ("Unmatched FD vs sample ID array for HTM event\n" ); +continue ; + } + +for (x =0 ; x <xyarray__max_x (xy); x++) { +for (y =0 ; y <xyarray__max_y (xy); y++) { +struct perf_counts_values count = { .val =0 }; + +if (perf_evsel__read (rd_evsel, x, y, &count) ==0 ) + total_pending_records += count.val; + } + } + } + +/* Collection is complete only when ALL hardware queues have no pending records */ +return (total_pending_records >0 ) ?1 :0 ; +}