perf jevents: Add NVIDIA Tegra410 uncore DDR and PCIe metrics
cover perf jevents: Add NVIDIA Tegra410 uncore DDR and PCIe metrics
Add DDR bandwidth/latency and PCIe bandwidth metrics for the NVIDIA Tegra410 SoC to arm64_metrics.py. The metrics use the sysfs events of the Tegra410 UCF, CMEM latency and PCIE PMUs, and follow the formulas in Documentation/admin-guide/perf/nvidia-tegra410-pmu.rst.
Patch 1 adds the DDR metrics and makes pmu-events/Build also run arm64_metrics.py for the nvidia vendor models. Patch 2 adds the PCIe metrics, in total and per PCIe Root Complex (RC).
Tested on a 2-socket Tegra410 system:
- memcpy with multiload (from multichase) on socket 0 CPUs and memory:
lpm_ddr_bw is 740-744 GB/s at steady state (perf stat -I 2000), multiload reports 708063 MiB/s (742 GB/s).
- Read only (stream-sum) and write only (memset) loads on socket 1:
lpm_ddr_rd_bw is 608 GB/s with 2.4 GB/s of writes, lpm_ddr_wr_bw is 628 GB/s with 1.1 GB/s of reads.
- memcpy on socket 0 CPUs with memory on socket 1: socket 0's lpm_ddr_rem_rd_bw and socket 1's lpm_ddr_rd_bw count nearly the same bytes (258.03 vs 258.73 GB).
- lpm_ddr_lat idle is 150 ns in the default aggregation, and 132 ns and 194 ns for socket 0 and 1 with --per-socket. Under the memcpy load it is 555 ns by default and 563 ns for socket 0 with --per-socket, and stays at 551-558 ns per interval with -I 1000.
- 8 GiB O_DIRECT dd read from an NVMe drive behind RC 0 of socket 0:
lpm_pcie_wr_bw_0 counts the 8 GiB at 8.7 GB/s (dd: 8.7 GB/s). With the buffer on node 0 it shows up in lpm_pcie_loc_wr_bw_0, with the buffer on node 1 in lpm_pcie_rem_wr_bw_0. With --per-socket, socket 0 shows the same and no metric of either socket is nan.
- With the nvidia_t410 metrics forced on a machine without these PMUs (PERF_CPUID=0x000000004e0f0100 on an x86 JEVENTS_ARCH=all build), the PCIe metrics read 0 rather than failing to parse, also with --per-socket.
- "perf test 10" (PMU JSON event tests) passes on the Tegra410 system and with the x86 JEVENTS_ARCH=all build.
Chun-Tse Shao (2):
perf jevents: Add NVIDIA Tegra410 uncore DDR metrics perf jevents: Add NVIDIA Tegra410 uncore PCIe metrics
tools/perf/pmu-events/Build | 2 +- tools/perf/pmu-events/arm64_metrics.py | 126 ++++++++++++++++++++++++- tools/perf/pmu-events/metric.py | 2 +- 3 files changed, 126 insertions(+), 4 deletions(-)
base-commit: 0ae6fc78c5ce0dfd18d8712a50f0fd4602eff103 --
2.56.0.rc1.315.gc6ed9934b7-goog
PATCH 1/2 perf jevents: Add NVIDIA Tegra410 uncore DDR metrics
Add DDR bandwidth and latency metrics for the NVIDIA Tegra410 SoC. The formulas follow Documentation/admin-guide/perf/nvidia-tegra410-pmu.rst:
AVG_MEM_READ_BANDWIDTH_IN_GBPS = MEM_BYTES_RD / ELAPSED_TIME_IN_NS AVG_MEM_WRITE_BANDWIDTH_IN_GBPS = MEM_BYTES_WR / ELAPSED_TIME_IN_NS
FREQ_IN_GHZ = CYCLES / ELAPSED_TIME_IN_NS AVG_LATENCY_IN_CYCLES = RD_CUM_OUTS / RD_REQ AVERAGE_LATENCY_IN_NS = AVG_LATENCY_IN_CYCLES / FREQ_IN_GHZ
The bandwidth metrics use the mem_bytes_rd/mem_bytes_wr events of the Unified Coherence Fabric (UCF) PMU with source/destination filters:
lpm_ddr_bw, lpm_ddr_rd_bw, lpm_ddr_wr_bw: all sources to local CMEM. lpm_ddr_loc_rd_bw, lpm_ddr_loc_wr_bw: local CPU and non-CPU sources to
local CMEM.
lpm_ddr_rem_rd_bw, lpm_ddr_rem_wr_bw: local CPU and non-CPU sources to
remote memory.
The latency metrics use the CPU Memory (CMEM) latency PMU:
lpm_ddr_lat, lpm_ddr_rd_lat: average DDR read latency in ns.
lpm_ddr_wr_lat: the PMU only measures reads, so this is always 0.
There is one CMEM latency PMU per socket and perf sums the counts of the PMUs in an aggregation, e.g. of both sockets in the default aggregation mode. Use aggr_nr() to divide the summed cycles by the number of PMUs, otherwise the frequency is too high and the latency too low by that factor.
The events are sysfs events of the nvidia_ucf_pmu_<socket-id> and nvidia_cmem_latency_pmu_<socket-id> PMUs rather than json events, so add the "nvidia" PMU prefix to the prefixes that skip the named json event check in metric.py.
pmu-events/Build only runs arm64_metrics.py for the "arm" vendor models, so also run it for the "nvidia" vendor models to generate the Tegra410 metrics.
--- tools/perf/pmu-events/Build | 2 +- tools/perf/pmu-events/arm64_metrics.py | 66 +++++++++++++++++++++++++- tools/perf/pmu-events/metric.py | 2 +- 3 files changed, 66 insertions(+), 4 deletions(-)
tools/perf/pmu-events/Build +1 -1
@@ -78,7 +78,7 @@ endif ifeq ($(JEVENTS_ARCH),$(filter $(JEVENTS_ARCH),arm64 all)) # Generate ARM Json -ARMS := $(shell ls -d pmu-events/arch/arm64/arm/*|grep -v cmn) +ARMS := $(shell ls -d pmu-events/arch/arm64/arm/* pmu-events/arch/arm64/nvidia/*|grep -v cmn) ARM_METRICS = $(foreach x,$(ARMS),$(OUTPUT)$(x)/extra-metrics.json) ARM_METRICGROUPS = $(foreach x,$(ARMS),$(OUTPUT)$(x)/extra-metricgroups.json) GEN_JSON += $(ARM_METRICS) $(ARM_METRICGROUPS)
tools/perf/pmu-events/arm64_metrics.py +64 -2
84 lines not on this page · raw message
tools/perf/pmu-events/metric.py +1 -1
@@ -91,7 +91,7 @@ def CheckEveryEvent(*names: str) -> None: name = name[:name.find(':')] elif '/' in name: name = name[:name.find('/')] - if any([name.startswith(x) for x in ['amd', 'arm', 'cpu', 'msr', 'power', 'cha', 'uncore']]): + if any([name.startswith(x) for x in ['amd', 'arm', 'cpu', 'msr', 'power', 'cha', 'uncore', 'nvidia']]): continue if name not in all_events_all_models: raise ValueError(f"Is {name} a named json event?")
Signature · 1 line
2.56.0.rc1.315.gc6ed9934b7-goog
[email protected]
AI reviewRe: [PATCH 1/2] perf jevents: Add NVIDIA Tegra410 uncore DDR metrics
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 13 lines
Add DDR bandwidth and latency metrics for the NVIDIA Tegra410 SoC. The formulas follow
Documentation/admin-guide/perf/nvidia-tegra410-pmu.rst:
AVG_MEM_READ_BANDWIDTH_IN_GBPS=MEM_BYTES_RD/ELAPSED_TIME_IN_NSAVG_MEM_WRITE_BANDWIDTH_IN_GBPS=MEM_BYTES_WR/ELAPSED_TIME_IN_NS
FREQ_IN_GHZ= CYCLES /ELAPSED_TIME_IN_NSAVG_LATENCY_IN_CYCLES=RD_CUM_OUTS/RD_REQAVERAGE_LATENCY_IN_NS=AVG_LATENCY_IN_CYCLES/FREQ_IN_GHZThe bandwidth metrics use the mem_bytes_rd/mem_bytes_wr events of the Unified Coherence Fabric (UCF) PMU with source/destination filters:
Quoted 8 lines
lpm_ddr_bw,lpm_ddr_rd_bw,lpm_ddr_wr_bw: all sources to local CMEM.lpm_ddr_loc_rd_bw,lpm_ddr_loc_wr_bw: local CPU and non-CPU sources to
[ ... ]
pmu-events/Build only runsarm64_metrics.pyfor the "arm" vendor models, so also run it for the "nvidia" vendor models to generate the Tegra410 metrics.Signed-off-by Chun-Tse Shao <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
RE: [PATCH 1/2] perf jevents: Add NVIDIA Tegra410 uncore DDR metrics
In reply to Chun-Tse ShaoHi Chun-Tse,
I tested this patch on 2-socket Tegra410. Both DDR bandwidth and read latency metrics give correct values in the default, --per-socket, and -A modes.
Some small comments inline below.
Quoted 40 lines
-----Original Message-----
From: Chun-Tse Shao <[email protected]>
Sent: Monday, September 28, 2026 5:51 PM
To: [email protected]; [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Besar Wicaksono <[email protected]>; [email protected]; linux- [email protected]; Chun-Tse Shao <[email protected]>
Subject: [PATCH 1/2] perf jevents: Add NVIDIA Tegra410 uncore DDR metricsExternal email: Use caution opening links or attachments
Add DDR bandwidth and latency metrics for the NVIDIA Tegra410 SoC. The formulas follow
Documentation/admin-guide/perf/nvidia-tegra410-pmu.rst:
AVG_MEM_READ_BANDWIDTH_IN_GBPS=MEM_BYTES_RD/ELAPSED_TIME_IN_NS
AVG_MEM_WRITE_BANDWIDTH_IN_GBPS=MEM_BYTES_WR/ELAPSED_TIME_IN_NS
FREQ_IN_GHZ= CYCLES /ELAPSED_TIME_IN_NSAVG_LATENCY_IN_CYCLES=RD_CUM_OUTS/RD_REQAVERAGE_LATENCY_IN_NS=AVG_LATENCY_IN_CYCLES/FREQ_IN_GHZThe bandwidth metrics use the mem_bytes_rd/mem_bytes_wr events of the
Unified Coherence Fabric (UCF) PMU with source/destination filters:
lpm_ddr_bw,lpm_ddr_rd_bw,lpm_ddr_wr_bw: all sources to local CMEM.lpm_ddr_loc_rd_bw,lpm_ddr_loc_wr_bw: local CPU and non-CPU sources
to
local CMEM.
lpm_ddr_rem_rd_bw,lpm_ddr_rem_wr_bw: local CPU and non-CPU
sources to
remote memory.
Quoted 5 lines
The latency metrics use the CPU Memory (CMEM) latency PMU:
lpm_ddr_lat,lpm_ddr_rd_lat: average DDR read latency in ns.
lpm_ddr_wr_lat: the PMU only measures reads, so this is always 0.
A question on naming: what does the "lpm_" prefix stand for? Is it the convention for all metrics generated by the Python scripts (I see it on the Intel/AMD and common metrics too)?
Quoted 122 lines
There is one CMEM latency PMU per socket and perf sums the counts of the PMUs in an aggregation, e.g. of both sockets in the default aggregation mode. Use
aggr_nr()to divide the summed cycles by the number of PMUs, otherwise the frequency is too high and the latency too low by that factor.The events are sysfs events of the nvidia_ucf_pmu_<socket-id> and nvidia_cmem_latency_pmu_<socket-id> PMUs rather than json events, so add
the "nvidia" PMU prefix to the prefixes that skip the named json event check inmetric.py.pmu-events/Build only runs
arm64_metrics.pyfor the "arm" vendor models,
so also run it for the "nvidia" vendor models to generate the Tegra410 metrics.Signed-off-by Chun-Tse Shao <[email protected]>Assisted-by Gemini:gemini-3.1-pro-preview--- tools/perf/pmu-events/Build | 2 +- tools/perf/pmu-events/arm64_metrics.py | 66 +++++++++++++++++++++++++- tools/perf/pmu-events/metric.py | 2 +- 3 files changed, 66 insertions(+), 4 deletions(-)tools/perf/pmu-events/Build +1
-1@@ -78,7 +78,7 @@ endif ifeq ($(JEVENTS_ARCH),$(filter $(JEVENTS_ARCH),arm64 all)) # Generate ARM Json -ARMS := $(shell ls -d pmu-events/arch/arm64/arm/*|grep -v cmn) +ARMS := $(shell ls -d pmu-events/arch/arm64/arm/* pmu- events/arch/arm64/nvidia/*|grep -v cmn) ARM_METRICS = $(foreach x,$(ARMS),$(OUTPUT)$(x)/extra-metrics.json) ARM_METRICGROUPS = $(foreach x,$(ARMS),$(OUTPUT)$(x)/extra-metricgroups.json)
GEN_JSON+= $(ARM_METRICS) $(ARM_METRICGROUPS)tools/perf/pmu- renamed from tools/perf/pmu-events/arm64_metrics.py +0
-0
events/arm64_metrics.py
index4ecda96d11fa..b0d0651fec12100755tools/perf/pmu-events/arm64_metrics.py +7
-2@@ -2,14 +2,75 @@ # SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) import argparse import os -from metric import (JsonEncodeMetric, JsonEncodeMetricGroupDescriptions, LoadEvents, - MetricGroup) +from typing import Optional +from metric import (aggr_nr, d_ratio, Event, JsonEncodeMetric, + JsonEncodeMetricGroupDescriptions, LoadEvents, Metric, MetricGroup) from common_metrics import Cycles # Global command line arguments. _args = None +def NvidiaT410Uncore() -> Optional[MetricGroup]: + """Uncore metrics for the NVIDIA Tegra410 SoC. + + The formulas follow Documentation/admin-guide/perf/nvidia-tegra410-
pmu.rst.tools/perf/pmu-events/arm64_metrics.py +11
-0+ """ + assert _args is not None + if _args.vendor != "nvidia" or _args.model != "t410": + return None + + interval_sec = Event("duration_time") + mb_per_sec = f"{1/1e6}MB/s" + + def DdrBw() -> MetricGroup: + # Unified Coherence Fabric (UCF) PMU: + # AVG_MEM_{READ,WRITE}_BANDWIDTH = MEM_BYTES_{RD,WR} /
ELAPSED_TIMEtools/perf/pmu-events/arm64_metrics.py +4
-0+ loc_src = "src_loc_cpu=0x1,src_loc_noncpu=0x1" + rd = Event("nvidia_ucf_pmu/mem_bytes_rd,dst_loc_cmem=0x1/") + wr = Event("nvidia_ucf_pmu/mem_bytes_wr,dst_loc_cmem=0x1/") + loc_rd =
Event(f"nvidia_ucf_pmu/mem_bytes_rd,{loc_src},dst_loc_cmem=0x1/")
+loc_wr=Event(f"nvidia_ucf_pmu/mem_bytes_wr,{loc_src},dst_loc_cmem=0x1/")
+rem_rd=Event(f"nvidia_ucf_pmu/mem_bytes_rd,{loc_src},dst_rem=0x1/")
+rem_wr=Event(f"nvidia_ucf_pmu/mem_bytes_wr,{loc_src},dst_rem=0x1/")tools/perf/pmu-events/arm64_metrics.py +20
-0+ return MetricGroup("lpm_ddr_bw", [ + Metric("lpm_ddr_bw", "Total DDR bandwidth", + d_ratio(rd + wr, interval_sec), mb_per_sec), + Metric("lpm_ddr_rd_bw", "DDR read bandwidth", + d_ratio(rd, interval_sec), mb_per_sec), + Metric("lpm_ddr_wr_bw", "DDR write bandwidth", + d_ratio(wr, interval_sec), mb_per_sec), + Metric("lpm_ddr_loc_rd_bw", "Local DDR read bandwidth", + d_ratio(loc_rd, interval_sec), mb_per_sec), + Metric("lpm_ddr_loc_wr_bw", "Local DDR write bandwidth", + d_ratio(loc_wr, interval_sec), mb_per_sec), + Metric("lpm_ddr_rem_rd_bw", "Remote DDR read bandwidth", + d_ratio(rem_rd, interval_sec), mb_per_sec), + Metric("lpm_ddr_rem_wr_bw", "Remote DDR write bandwidth", + d_ratio(rem_wr, interval_sec), mb_per_sec), + ], description="NVIDIA Tegra410 DDR bandwidth") + + def DdrLat() -> MetricGroup: + # CPU Memory (CMEM) latency PMU, one per socket: + # AVERAGE_LATENCY_IN_NS = (RD_CUM_OUTS / RD_REQ) / (CYCLES /
ELAPSED_TIME_IN_NS)tools/perf/pmu-events/arm64_metrics.py +4
-0+ rd_cum_outs = Event("nvidia_cmem_latency_pmu/rd_cum_outs/") + rd_req = Event("nvidia_cmem_latency_pmu/rd_req/") + cycles = Event("nvidia_cmem_latency_pmu/cycles/") + # perf sums the counts of the PMUs in an aggregation, e.g. bothsockets
tools/perf/pmu-events/arm64_metrics.py +2
-0+ # by default, so divide cycles by their number to get the frequency. + rd_lat = d_ratio(1e9 * interval_sec * rd_cum_outs * aggr_nr(cycles),
rd_req* cycles)tools/perf/pmu-events/arm64_metrics.py +2
-0+ return MetricGroup("lpm_ddr_lat", [ + Metric("lpm_ddr_lat", "Average DDR read latency in ns", rd_lat,
Quoted 2 lines
"1ns"),
+Metric("lpm_ddr_rd_lat", "DDR read latency in ns", rd_lat, "1ns"),
Could we keep just lpm_ddr_rd_lat ?
tools/perf/pmu-events/arm64_metrics.py line 68
Quoted 5 lines
+ # The CMEM latency PMU only measures reads, write latency is
always 0.tools/perf/pmu-events/arm64_metrics.py +3
-0+ Metric("lpm_ddr_wr_lat", "DDR write latency in ns", + rd_cum_outs - rd_cum_outs, "1ns"), + ], description="NVIDIA Tegra410 DDR latency")
This always prints 0.0 ns, which reads like a real measurement of zero-latency writes. Since the hardware can't measure write latency, I'd drop this metric rather than report a placeholder.
Thanks,
Besar
Quoted 34 lines
tools/perf/pmu-events/arm64_metrics.py +6
-0+ + return MetricGroup("lpm_uncore", [DdrBw(), DdrLat()], + description="NVIDIA Tegra410 uncore metrics") + + def main() -> None: global _args@@ -37,6 +98,7 @@ def main() -> None: all_metrics = MetricGroup("", [ Cycles(), + NvidiaT410Uncore(), ]) if _args.metricgroups:tools/perf/pmu- renamed from tools/perf/pmu-events/metric.py +0
-0
events/metric.py
indexf5d81eccd5c3..6fb69024d1fd100644tools/perf/pmu-events/metric.py +0
-1@@ -91,7 +91,7 @@ def CheckEveryEvent(*names: str) -> None: name = name[:name.find(':')] elif '/' in name: name = name[:name.find('/')] - if any([name.startswith(x) for x in ['amd', 'arm', 'cpu', 'msr', 'power','cha', 'uncore']]):
+ if any([name.startswith(x) for x in ['amd', 'arm', 'cpu', 'msr', 'power',
'cha', 'uncore', 'nvidia']]):
continue
if name not inall_events_all_models:
raiseValueError(f"Is {name} a named json event?")
--
2.56.0.rc1.315.gc6ed9934b7-goog
Ian Rogers
reviewerRe: [PATCH 1/2] perf jevents: Add NVIDIA Tegra410 uncore DDR metrics
Quoted 51 lines · Besar Wicaksono
Hi Chun-Tse,
I tested this patch on 2-socket Tegra410. Both DDR bandwidth and read latency metrics give correct values in the default, --per-socket, and -A modes.
Some small comments inline below.
Quoted 42 lines
-----Original Message-----
From: Chun-Tse Shao <[email protected]>
Sent: Monday, September 28, 2026 5:51 PM
To: [email protected]; [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Besar Wicaksono <[email protected]>; [email protected]; linux- [email protected]; Chun-Tse Shao <[email protected]>
Subject: [PATCH 1/2] perf jevents: Add NVIDIA Tegra410 uncore DDR metricsExternal email: Use caution opening links or attachments
Add DDR bandwidth and latency metrics for the NVIDIA Tegra410 SoC. The formulas follow
Documentation/admin-guide/perf/nvidia-tegra410-pmu.rst:
AVG_MEM_READ_BANDWIDTH_IN_GBPS=MEM_BYTES_RD/ELAPSED_TIME_IN_NS
AVG_MEM_WRITE_BANDWIDTH_IN_GBPS=MEM_BYTES_WR/ELAPSED_TIME_IN_NS
FREQ_IN_GHZ= CYCLES /ELAPSED_TIME_IN_NSAVG_LATENCY_IN_CYCLES=RD_CUM_OUTS/RD_REQAVERAGE_LATENCY_IN_NS=AVG_LATENCY_IN_CYCLES/FREQ_IN_GHZThe bandwidth metrics use the mem_bytes_rd/mem_bytes_wr events of the
Unified Coherence Fabric (UCF) PMU with source/destination filters:
lpm_ddr_bw,lpm_ddr_rd_bw,lpm_ddr_wr_bw: all sources to local CMEM.lpm_ddr_loc_rd_bw,lpm_ddr_loc_wr_bw: local CPU and non-CPU sources
to
local CMEM.
lpm_ddr_rem_rd_bw,lpm_ddr_rem_wr_bw: local CPU and non-CPU
sources to
remote memory.The latency metrics use the CPU Memory (CMEM) latency PMU:
Quoted 7 lines
Quoted 3 lines
lpm_ddr_lat,lpm_ddr_rd_lat: average DDR read latency in ns.
lpm_ddr_wr_lat: the PMU only measures reads, so this is always 0.A question on naming: what does the "
lpm_" prefix stand for? Is it the convention for all metrics generated by the Python scripts (I see it on the Intel/AMD and common metrics too)?
Hi Besar, naming is one of the hardest problems in Computer Science :-) When we first added the metrics there was a request to differentiate between metrics coming from the python scripts and metrics coming from vendor code, like Intel top-down. We agreed on having a prefix, and lpm_ was suggested as meaning Linux perf metric. All of those words seem redundant given the metric is being measured on Linux using perf, but it was adopted as the best suggestion. If you have a better proposal, I think we can change it.
Thanks,
Ian
Quoted 177 lines
Quoted 122 lines
There is one CMEM latency PMU per socket and perf sums the counts of the PMUs in an aggregation, e.g. of both sockets in the default aggregation mode. Use
aggr_nr()to divide the summed cycles by the number of PMUs, otherwise the frequency is too high and the latency too low by that factor.The events are sysfs events of the nvidia_ucf_pmu_<socket-id> and nvidia_cmem_latency_pmu_<socket-id> PMUs rather than json events, so add
the "nvidia" PMU prefix to the prefixes that skip the named json event check inmetric.py.pmu-events/Build only runs
arm64_metrics.pyfor the "arm" vendor models,
so also run it for the "nvidia" vendor models to generate the Tegra410 metrics.Signed-off-by Chun-Tse Shao <[email protected]>Assisted-by Gemini:gemini-3.1-pro-preview--- tools/perf/pmu-events/Build | 2 +- tools/perf/pmu-events/arm64_metrics.py | 66 +++++++++++++++++++++++++- tools/perf/pmu-events/metric.py | 2 +- 3 files changed, 66 insertions(+), 4 deletions(-)tools/perf/pmu-events/Build +1
-1@@ -78,7 +78,7 @@ endif ifeq ($(JEVENTS_ARCH),$(filter $(JEVENTS_ARCH),arm64 all)) # Generate ARM Json -ARMS := $(shell ls -d pmu-events/arch/arm64/arm/*|grep -v cmn) +ARMS := $(shell ls -d pmu-events/arch/arm64/arm/* pmu- events/arch/arm64/nvidia/*|grep -v cmn) ARM_METRICS = $(foreach x,$(ARMS),$(OUTPUT)$(x)/extra-metrics.json) ARM_METRICGROUPS = $(foreach x,$(ARMS),$(OUTPUT)$(x)/extra-metricgroups.json)
GEN_JSON+= $(ARM_METRICS) $(ARM_METRICGROUPS)tools/perf/pmu- renamed from tools/perf/pmu-events/arm64_metrics.py +0
-0
events/arm64_metrics.py
index4ecda96d11fa..b0d0651fec12100755tools/perf/pmu-events/arm64_metrics.py +7
-2@@ -2,14 +2,75 @@ # SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) import argparse import os -from metric import (JsonEncodeMetric, JsonEncodeMetricGroupDescriptions, LoadEvents, - MetricGroup) +from typing import Optional +from metric import (aggr_nr, d_ratio, Event, JsonEncodeMetric, + JsonEncodeMetricGroupDescriptions, LoadEvents, Metric, MetricGroup) from common_metrics import Cycles # Global command line arguments. _args = None +def NvidiaT410Uncore() -> Optional[MetricGroup]: + """Uncore metrics for the NVIDIA Tegra410 SoC. + + The formulas follow Documentation/admin-guide/perf/nvidia-tegra410-
pmu.rst.tools/perf/pmu-events/arm64_metrics.py +11
-0+ """ + assert _args is not None + if _args.vendor != "nvidia" or _args.model != "t410": + return None + + interval_sec = Event("duration_time") + mb_per_sec = f"{1/1e6}MB/s" + + def DdrBw() -> MetricGroup: + # Unified Coherence Fabric (UCF) PMU: + # AVG_MEM_{READ,WRITE}_BANDWIDTH = MEM_BYTES_{RD,WR} /
ELAPSED_TIMEtools/perf/pmu-events/arm64_metrics.py +4
-0+ loc_src = "src_loc_cpu=0x1,src_loc_noncpu=0x1" + rd = Event("nvidia_ucf_pmu/mem_bytes_rd,dst_loc_cmem=0x1/") + wr = Event("nvidia_ucf_pmu/mem_bytes_wr,dst_loc_cmem=0x1/") + loc_rd =
Event(f"nvidia_ucf_pmu/mem_bytes_rd,{loc_src},dst_loc_cmem=0x1/")
+loc_wr=Event(f"nvidia_ucf_pmu/mem_bytes_wr,{loc_src},dst_loc_cmem=0x1/")
+rem_rd=Event(f"nvidia_ucf_pmu/mem_bytes_rd,{loc_src},dst_rem=0x1/")
+rem_wr=Event(f"nvidia_ucf_pmu/mem_bytes_wr,{loc_src},dst_rem=0x1/")tools/perf/pmu-events/arm64_metrics.py +20
-0+ return MetricGroup("lpm_ddr_bw", [ + Metric("lpm_ddr_bw", "Total DDR bandwidth", + d_ratio(rd + wr, interval_sec), mb_per_sec), + Metric("lpm_ddr_rd_bw", "DDR read bandwidth", + d_ratio(rd, interval_sec), mb_per_sec), + Metric("lpm_ddr_wr_bw", "DDR write bandwidth", + d_ratio(wr, interval_sec), mb_per_sec), + Metric("lpm_ddr_loc_rd_bw", "Local DDR read bandwidth", + d_ratio(loc_rd, interval_sec), mb_per_sec), + Metric("lpm_ddr_loc_wr_bw", "Local DDR write bandwidth", + d_ratio(loc_wr, interval_sec), mb_per_sec), + Metric("lpm_ddr_rem_rd_bw", "Remote DDR read bandwidth", + d_ratio(rem_rd, interval_sec), mb_per_sec), + Metric("lpm_ddr_rem_wr_bw", "Remote DDR write bandwidth", + d_ratio(rem_wr, interval_sec), mb_per_sec), + ], description="NVIDIA Tegra410 DDR bandwidth") + + def DdrLat() -> MetricGroup: + # CPU Memory (CMEM) latency PMU, one per socket: + # AVERAGE_LATENCY_IN_NS = (RD_CUM_OUTS / RD_REQ) / (CYCLES /
ELAPSED_TIME_IN_NS)tools/perf/pmu-events/arm64_metrics.py +4
-0+ rd_cum_outs = Event("nvidia_cmem_latency_pmu/rd_cum_outs/") + rd_req = Event("nvidia_cmem_latency_pmu/rd_req/") + cycles = Event("nvidia_cmem_latency_pmu/cycles/") + # perf sums the counts of the PMUs in an aggregation, e.g. bothsockets
tools/perf/pmu-events/arm64_metrics.py +2
-0+ # by default, so divide cycles by their number to get the frequency. + rd_lat = d_ratio(1e9 * interval_sec * rd_cum_outs * aggr_nr(cycles),
rd_req* cycles)tools/perf/pmu-events/arm64_metrics.py +2
-0+ return MetricGroup("lpm_ddr_lat", [ + Metric("lpm_ddr_lat", "Average DDR read latency in ns", rd_lat,Quoted 2 lines
"1ns"),
+Metric("lpm_ddr_rd_lat", "DDR read latency in ns", rd_lat, "1ns"),Could we keep just
lpm_ddr_rd_lat?tools/perf/pmu-events/arm64_metrics.py line 68
Quoted 5 lines
+ # The CMEM latency PMU only measures reads, write latency is
always 0.tools/perf/pmu-events/arm64_metrics.py +3
-0+ Metric("lpm_ddr_wr_lat", "DDR write latency in ns", + rd_cum_outs - rd_cum_outs, "1ns"), + ], description="NVIDIA Tegra410 DDR latency")This always prints 0.0 ns, which reads like a real measurement of zero-latency writes. Since the hardware can't measure write latency, I'd drop this metric rather than report a placeholder.
Tested-by Besar Wicaksono <[email protected]>Thanks,
BesarQuoted 34 lines
tools/perf/pmu-events/arm64_metrics.py +6
-0+ + return MetricGroup("lpm_uncore", [DdrBw(), DdrLat()], + description="NVIDIA Tegra410 uncore metrics") + + def main() -> None: global _args@@ -37,6 +98,7 @@ def main() -> None: all_metrics = MetricGroup("", [ Cycles(), + NvidiaT410Uncore(), ]) if _args.metricgroups:tools/perf/pmu- renamed from tools/perf/pmu-events/metric.py +0
-0
events/metric.py
indexf5d81eccd5c3..6fb69024d1fd100644tools/perf/pmu-events/metric.py +0
-1@@ -91,7 +91,7 @@ def CheckEveryEvent(*names: str) -> None: name = name[:name.find(':')] elif '/' in name: name = name[:name.find('/')] - if any([name.startswith(x) for x in ['amd', 'arm', 'cpu', 'msr', 'power','cha', 'uncore']]):
+ if any([name.startswith(x) for x in ['amd', 'arm', 'cpu', 'msr', 'power',
'cha', 'uncore', 'nvidia']]):
continue
if name not inall_events_all_models:
raiseValueError(f"Is {name} a named json event?")
--
2.56.0.rc1.315.gc6ed9934b7-goog
RE: [PATCH 1/2] perf jevents: Add NVIDIA Tegra410 uncore DDR metrics
Quoted 86 lines
-----Original Message-----
From: Ian Rogers <[email protected]>
Sent: Tuesday, October 13, 2026 9:30 PM
To: Besar Wicaksono <[email protected]>
Cc: Chun-Tse Shao <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; linux- [email protected]; [email protected]; Matt Ochs <[email protected]>; Sean Kelley <[email protected]>; Rich Wiley <[email protected]>; Vikram Sethi <[email protected]>; Raj Joshi <[email protected]>
Subject: Re: [PATCH 1/2] perf jevents: Add NVIDIA Tegra410 uncore DDR metricsExternal email: Use caution opening links or attachments
Quoted 14 lines · Besar Wicaksono
Hi Chun-Tse,
I tested this patch on 2-socket Tegra410. Both DDR bandwidth and read latency metrics give correct values in the default, --per-socket, and -A modes.
Some small comments inline below.
Quoted 5 lines
-----Original Message-----
From: Chun-Tse Shao <[email protected]>
Sent: Monday, September 28, 2026 5:51 PM
To: [email protected]; [email protected]; [email protected]
Cc: [email protected]; [email protected];Quoted 11 lines
Quoted 11 lines
[email protected]; [email protected]; [email protected]; [email protected]; Besar Wicaksono <[email protected]>; [email protected]; linux- [email protected]; Chun-Tse Shao <[email protected]>
Subject: [PATCH 1/2] perf jevents: Add NVIDIA Tegra410 uncore DDR metricsExternal email: Use caution opening links or attachments
Add DDR bandwidth and latency metrics for the NVIDIA Tegra410 SoC.
The
Quoted 8 lines
Quoted 8 lines
formulas follow
Documentation/admin-guide/perf/nvidia-tegra410-pmu.rst:
AVG_MEM_READ_BANDWIDTH_IN_GBPS=MEM_BYTES_RD/ELAPSED_TIME_IN_NS
AVG_MEM_WRITE_BANDWIDTH_IN_GBPS=MEM_BYTES_WR/ELAPSED_TIME_IN_NSQuoted 5 lines
Quoted 5 lines
FREQ_IN_GHZ= CYCLES /ELAPSED_TIME_IN_NSAVG_LATENCY_IN_CYCLES=RD_CUM_OUTS/RD_REQAVERAGE_LATENCY_IN_NS=AVG_LATENCY_IN_CYCLES/FREQ_IN_GHZThe bandwidth metrics use the mem_bytes_rd/mem_bytes_wr events
of
Quoted 4 lines
Quoted 4 lines
the
Unified Coherence Fabric (UCF) PMU with source/destination filters:
lpm_ddr_bw,lpm_ddr_rd_bw,lpm_ddr_wr_bw: all sources to localCMEM.
Quoted 1 line
Quoted 1 line
lpm_ddr_loc_rd_bw,lpm_ddr_loc_wr_bw: local CPU and non-CPUsources
Quoted 8 lines
Quoted 8 lines
to
local CMEM.
lpm_ddr_rem_rd_bw,lpm_ddr_rem_wr_bw: local CPU and non-CPU
sources to
remote memory.The latency metrics use the CPU Memory (CMEM) latency PMU:
Quoted 7 lines
Quoted 3 lines
lpm_ddr_lat,lpm_ddr_rd_lat: average DDR read latency in ns.
lpm_ddr_wr_lat: the PMU only measures reads, so this is always 0.A question on naming: what does the "
lpm_" prefix stand for? Is it the convention for all metrics generated by the Python scripts (I see it on the Intel/AMD and common metrics too)?
Quoted 9 lines
Hi Besar, naming is one of the hardest problems in Computer Science :-) When we first added the metrics there was a request to differentiate between metrics coming from the python scripts and metrics coming from vendor code, like Intel top-down. We agreed on having a prefix, and
lpm_was suggested as meaning Linux perf metric. All of those words seem redundant given the metric is being measured on Linux using perf, but it was adopted as the best suggestion. If you have a better proposal, I think we can change it.
Hi Ian,
Thanks for the explanation, this makes sense, and it keeps them distinguishable from the vendor JSON metrics.
Regards,
Besar
Quoted 189 lines
Thanks,
IanQuoted 1 line
Quoted 1 line
There is one CMEM latency PMU per socket and perf sums the counts of
the
Quoted 2 lines
Quoted 2 lines
PMUs in an aggregation, e.g. of both sockets in the default aggregation mode. Use
aggr_nr()to divide the summed cycles by the number ofPMUs,
Quoted 5 lines
Quoted 5 lines
otherwise the frequency is too high and the latency too low by that factor.
The events are sysfs events of the nvidia_ucf_pmu_<socket-id> and nvidia_cmem_latency_pmu_<socket-id> PMUs rather than json events,
so
Quoted 24 lines
Quoted 24 lines
add
the "nvidia" PMU prefix to the prefixes that skip the named json event check inmetric.py.pmu-events/Build only runs
arm64_metrics.pyfor the "arm" vendor models,
so also run it for the "nvidia" vendor models to generate the Tegra410 metrics.Signed-off-by Chun-Tse Shao <[email protected]>Assisted-by Gemini:gemini-3.1-pro-preview--- tools/perf/pmu-events/Build | 2 +- tools/perf/pmu-events/arm64_metrics.py | 66 +++++++++++++++++++++++++- tools/perf/pmu-events/metric.py | 2 +- 3 files changed, 66 insertions(+), 4 deletions(-)tools/perf/pmu-events/Build +0
-0@@ -78,7 +78,7 @@ endifQuoted 6 lines
tools/perf/pmu-events/Build endif
Quoted 6 lines
tools/perf/pmu-events/Build +1
-1ifeq ($(JEVENTS_ARCH),$(filter $(JEVENTS_ARCH),arm64 all)) # Generate ARM Json -ARMS := $(shell ls -d pmu-events/arch/arm64/arm/*|grep -v cmn) +ARMS := $(shell ls -d pmu-events/arch/arm64/arm/* pmu- events/arch/arm64/nvidia/*|grep -v cmn) ARM_METRICS = $(foreach x,$(ARMS),$(OUTPUT)$(x)/extra-metrics.json)
Quoted 21 lines
Quoted 21 lines
tools/perf/pmu-events/Build +0
-0ARM_METRICGROUPS = $(foreach x,$(ARMS),$(OUTPUT)$(x)/extra-metricgroups.json)
GEN_JSON+= $(ARM_METRICS) $(ARM_METRICGROUPS)tools/perf/pmu- renamed from tools/perf/pmu-events/arm64_metrics.py +0
-0
events/arm64_metrics.py
index4ecda96d11fa..b0d0651fec12100755tools/perf/pmu-events/arm64_metrics.py +3
-2@@ -2,14 +2,75 @@ # SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) import argparse import os -from metric import (JsonEncodeMetric, JsonEncodeMetricGroupDescriptions, LoadEvents, - MetricGroup) +from typing import Optional +from metric import (aggr_nr, d_ratio, Event, JsonEncodeMetric, + JsonEncodeMetricGroupDescriptions, LoadEvents, Metric, MetricGroup) from common_metrics import CyclesQuoted 8 lines
tools/perf/pmu-events/arm64_metrics.py
Quoted 8 lines
tools/perf/pmu-events/arm64_metrics.py +4
-0# Global command line arguments. _args = None +def NvidiaT410Uncore() -> Optional[MetricGroup]: + """Uncore metrics for the NVIDIA Tegra410 SoC. + + The formulas follow Documentation/admin-guide/perf/nvidia-tegra410-
Quoted 4 lines
Quoted 4 lines
pmu.rst.tools/perf/pmu-events/arm64_metrics.py +3
-0+ """ + assert _args is not None + if _args.vendor != "nvidia" or _args.model != "t410":Quoted 8 lines
tools/perf/pmu-events/arm64_metrics.py
Quoted 8 lines
tools/perf/pmu-events/arm64_metrics.py +8
-0+ return None + + interval_sec = Event("duration_time") + mb_per_sec = f"{1/1e6}MB/s" + + def DdrBw() -> MetricGroup: + # Unified Coherence Fabric (UCF) PMU: + # AVG_MEM_{READ,WRITE}_BANDWIDTH = MEM_BYTES_{RD,WR}/
Quoted 4 lines
tools/perf/pmu-events/arm64_metrics.py
Quoted 4 lines
ELAPSED_TIMEtools/perf/pmu-events/arm64_metrics.py +3
-0+ loc_src = "src_loc_cpu=0x1,src_loc_noncpu=0x1" + rd = Event("nvidia_ucf_pmu/mem_bytes_rd,dst_loc_cmem=0x1/") + wr =
Event("nvidia_ucf_pmu/mem_bytes_wr,dst_loc_cmem=0x1/")Quoted 4 lines
Quoted 4 lines
+
loc_rd=Event(f"nvidia_ucf_pmu/mem_bytes_rd,{loc_src},dst_loc_cmem=0x1/")
+loc_wr=
Event(f"nvidia_ucf_pmu/mem_bytes_wr,{loc_src},dst_loc_cmem=0x1/")Quoted 16 lines
Quoted 16 lines
+
rem_rd=Event(f"nvidia_ucf_pmu/mem_bytes_rd,{loc_src},dst_rem=0x1/")
+rem_wr=Event(f"nvidia_ucf_pmu/mem_bytes_wr,{loc_src},dst_rem=0x1/")tools/perf/pmu-events/arm64_metrics.py +12
-0+ return MetricGroup("lpm_ddr_bw", [ + Metric("lpm_ddr_bw", "Total DDR bandwidth", + d_ratio(rd + wr, interval_sec), mb_per_sec), + Metric("lpm_ddr_rd_bw", "DDR read bandwidth", + d_ratio(rd, interval_sec), mb_per_sec), + Metric("lpm_ddr_wr_bw", "DDR write bandwidth", + d_ratio(wr, interval_sec), mb_per_sec), + Metric("lpm_ddr_loc_rd_bw", "Local DDR read bandwidth", + d_ratio(loc_rd, interval_sec), mb_per_sec), + Metric("lpm_ddr_loc_wr_bw", "Local DDR write bandwidth", + d_ratio(loc_wr, interval_sec), mb_per_sec), + Metric("lpm_ddr_rem_rd_bw", "Remote DDR read bandwidth",Quoted 8 lines
tools/perf/pmu-events/arm64_metrics.py
Quoted 8 lines
tools/perf/pmu-events/arm64_metrics.py +8
-0+ d_ratio(rem_rd, interval_sec), mb_per_sec), + Metric("lpm_ddr_rem_wr_bw", "Remote DDR write bandwidth", + d_ratio(rem_wr, interval_sec), mb_per_sec), + ], description="NVIDIA Tegra410 DDR bandwidth") + + def DdrLat() -> MetricGroup: + # CPU Memory (CMEM) latency PMU, one per socket: + # AVERAGE_LATENCY_IN_NS = (RD_CUM_OUTS / RD_REQ) /(CYCLES /
Quoted 66 lines
Quoted 11 lines
ELAPSED_TIME_IN_NS)tools/perf/pmu-events/arm64_metrics.py +4
-0+ rd_cum_outs = Event("nvidia_cmem_latency_pmu/rd_cum_outs/") + rd_req = Event("nvidia_cmem_latency_pmu/rd_req/") + cycles = Event("nvidia_cmem_latency_pmu/cycles/") + # perf sums the counts of the PMUs in an aggregation, e.g. bothsockets
tools/perf/pmu-events/arm64_metrics.py +2
-0+ # by default, so divide cycles by their number to get the frequency. + rd_lat = d_ratio(1e9 * interval_sec * rd_cum_outs * aggr_nr(cycles),
rd_req* cycles)tools/perf/pmu-events/arm64_metrics.py +2
-0+ return MetricGroup("lpm_ddr_lat", [ + Metric("lpm_ddr_lat", "Average DDR read latency in ns", rd_lat,Quoted 2 lines
"1ns"),
+Metric("lpm_ddr_rd_lat", "DDR read latency in ns", rd_lat, "1ns"),Could we keep just
lpm_ddr_rd_lat?tools/perf/pmu-events/arm64_metrics.py line 68
Quoted 5 lines
+ # The CMEM latency PMU only measures reads, write latency is
always 0.tools/perf/pmu-events/arm64_metrics.py +3
-0+ Metric("lpm_ddr_wr_lat", "DDR write latency in ns", + rd_cum_outs - rd_cum_outs, "1ns"), + ], description="NVIDIA Tegra410 DDR latency")This always prints 0.0 ns, which reads like a real measurement of zero-latency writes. Since the hardware can't measure write latency, I'd drop this metric rather than report a placeholder.
Tested-by Besar Wicaksono <[email protected]>Thanks,
BesarQuoted 34 lines
tools/perf/pmu-events/arm64_metrics.py +6
-0+ + return MetricGroup("lpm_uncore", [DdrBw(), DdrLat()], + description="NVIDIA Tegra410 uncore metrics") + + def main() -> None: global _args@@ -37,6 +98,7 @@ def main() -> None: all_metrics = MetricGroup("", [ Cycles(), + NvidiaT410Uncore(), ]) if _args.metricgroups:tools/perf/pmu- renamed from tools/perf/pmu-events/metric.py +0
-0
events/metric.py
indexf5d81eccd5c3..6fb69024d1fd100644tools/perf/pmu-events/metric.py +0
-1@@ -91,7 +91,7 @@ def CheckEveryEvent(*names: str) -> None: name = name[:name.find(':')] elif '/' in name: name = name[:name.find('/')] - if any([name.startswith(x) for x in ['amd', 'arm', 'cpu', 'msr', 'power','cha', 'uncore']]):
+ if any([name.startswith(x) for x in ['amd', 'arm', 'cpu', 'msr', 'power',
'cha', 'uncore', 'nvidia']]):
continue
if name not inall_events_all_models:
raiseValueError(f"Is {name} a named json event?")
--
2.56.0.rc1.315.gc6ed9934b7-goog
PATCH 2/2 perf jevents: Add NVIDIA Tegra410 uncore PCIe metrics
In reply to Chun-Tse ShaoAdd PCIe bandwidth metrics for the NVIDIA Tegra410 SoC. The formulas follow Documentation/admin-guide/perf/nvidia-tegra410-pmu.rst:
AVG_RD_BANDWIDTH_IN_GBPS = RD_BYTES / ELAPSED_TIME_IN_NS AVG_WR_BANDWIDTH_IN_GBPS = WR_BYTES / ELAPSED_TIME_IN_NS
There is one PCIE PMU per socket and PCIe Root Complex (RC), named nvidia_pcie_pmu_<socket-id>_rc_<pcie-rc-id>, with RCs 0 to 5 in each socket. The metrics sum the rd_bytes/wr_bytes events of both sockets:
lpm_pcie_bw, lpm_pcie_rd_bw, lpm_pcie_wr_bw: all RCs. lpm_pcie_bw_<rc>, lpm_pcie_rd_bw_<rc>, lpm_pcie_wr_bw_<rc>: one RC. lpm_pcie_loc_rd_bw_<rc>, lpm_pcie_loc_wr_bw_<rc>: one RC to local
CMEM, GMEM, PCIe peer and CXL memory.
lpm_pcie_rem_rd_bw_<rc>, lpm_pcie_rem_wr_bw_<rc>: one RC to remote
memory.
The per RC metrics are in the lpm_pcie_rc<rc>_bw metric groups.
A socket's event counts as 0 if its PMU isn't present (has_event() is 0), e.g. on a single socket system, or if it isn't counted in the aggregation (source_count() is 0), e.g. the other socket's event with --per-socket. Otherwise the metric fails to parse or is nan.
--- tools/perf/pmu-events/arm64_metrics.py | 68 ++++++++++++++++++++++++-- 1 file changed, 64 insertions(+), 4 deletions(-)
tools/perf/pmu-events/arm64_metrics.py +64 -4
@@ -2,9 +2,10 @@ # SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) import argparse import os -from typing import Optional -from metric import (aggr_nr, d_ratio, Event, JsonEncodeMetric, - JsonEncodeMetricGroupDescriptions, LoadEvents, Metric, MetricGroup) +from typing import List, Optional, Union +from metric import (aggr_nr, d_ratio, has_event, source_count, Event, Expression, + JsonEncodeMetric, JsonEncodeMetricGroupDescriptions, LoadEvents, Metric, + MetricGroup, Select) from common_metrics import Cycles # Global command line arguments.
@@ -67,7 +68,66 @@ def NvidiaT410Uncore() -> Optional[MetricGroup]: rd_cum_outs - rd_cum_outs, "1ns"), ], description="NVIDIA Tegra410 DDR latency") - return MetricGroup("lpm_uncore", [DdrBw(), DdrLat()], + def PcieBw() -> MetricGroup: + # PCIE PMU, one per socket and PCIe Root Complex (RC): + # AVG_{RD,WR}_BANDWIDTH = {RD,WR}_BYTES / ELAPSED_TIME + loc_dst = "dst_loc_cmem=0x1,dst_loc_gmem=0x1,dst_loc_pcie_p2p=0x1,dst_loc_pcie_cxl=0x1" + rem_dst = "dst_rem=0x1" + + def SocketSum(rc: str, event: str) -> Expression: + # Sum the event over both sockets. A PMU counts as 0 if it isn't + # present (has_event() is 0) or isn't counted in the aggregation, + # e.g. the other socket with --per-socket (source_count() is 0). + def Count(e: Event) -> Expression: + return Select(Select(e, source_count(e), 0), has_event(e), 0) + + return (Count(Event(f"nvidia_pcie_pmu_0_{rc}/{event}/")) + + Count(Event(f"nvidia_pcie_pmu_1_{rc}/{event}/"))) + + # nvidia_pcie_pmu_<socket-id>_rc matches the PMUs of all RCs in the socket. + rd = SocketSum("rc", "rd_bytes") + wr = SocketSum("rc", "wr_bytes") + metrics: List[Union[Metric, MetricGroup]] = [ + Metric("lpm_pcie_bw", "Total PCIe bandwidth across all Root Complexes", + d_ratio(rd + wr, interval_sec), mb_per_sec), + Metric("lpm_pcie_rd_bw", "Total PCIe read bandwidth across all Root Complexes", + d_ratio(rd, interval_sec), mb_per_sec), + Metric("lpm_pcie_wr_bw", "Total PCIe write bandwidth across all Root Complexes", + d_ratio(wr, interval_sec), mb_per_sec), + ] + for i in range(6): + rc = f"rc_{i}" + rc_rd = SocketSum(rc, "rd_bytes") + rc_wr = SocketSum(rc, "wr_bytes") + loc_rd = SocketSum(rc, f"rd_bytes,{loc_dst}") + loc_wr = SocketSum(rc, f"wr_bytes,{loc_dst}") + rem_rd = SocketSum(rc, f"rd_bytes,{rem_dst}") + rem_wr = SocketSum(rc, f"wr_bytes,{rem_dst}") + metrics.append(MetricGroup(f"lpm_pcie_rc{i}_bw", [ + Metric(f"lpm_pcie_bw_{i}", f"PCIe total bandwidth on Root Complex {i}", + d_ratio(rc_rd + rc_wr, interval_sec), mb_per_sec), + Metric(f"lpm_pcie_rd_bw_{i}", f"PCIe read bandwidth on Root Complex {i}", + d_ratio(rc_rd, interval_sec), mb_per_sec), + Metric(f"lpm_pcie_wr_bw_{i}", f"PCIe write bandwidth on Root Complex {i}", + d_ratio(rc_wr, interval_sec), mb_per_sec), + Metric(f"lpm_pcie_loc_rd_bw_{i}", + f"PCIe local read bandwidth on Root Complex {i}", + d_ratio(loc_rd, interval_sec), mb_per_sec), + Metric(f"lpm_pcie_loc_wr_bw_{i}", + f"PCIe local write bandwidth on Root Complex {i}", + d_ratio(loc_wr, interval_sec), mb_per_sec), + Metric(f"lpm_pcie_rem_rd_bw_{i}", + f"PCIe remote read bandwidth on Root Complex {i}", + d_ratio(rem_rd, interval_sec), mb_per_sec), + Metric(f"lpm_pcie_rem_wr_bw_{i}", + f"PCIe remote write bandwidth on Root Complex {i}", + d_ratio(rem_wr, interval_sec), mb_per_sec), + ], description=f"PCIe bandwidth on Root Complex {i}")) + + return MetricGroup("lpm_pcie_bw", metrics, + description="NVIDIA Tegra410 PCIe Root Complex bandwidth") + + return MetricGroup("lpm_uncore", [DdrBw(), DdrLat(), PcieBw()], description="NVIDIA Tegra410 uncore metrics")
Signature · 1 line
2.56.0.rc1.315.gc6ed9934b7-goog
[email protected]
AI reviewRe: [PATCH 2/2] perf jevents: Add NVIDIA Tegra410 uncore PCIe metrics
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 17 lines
Add PCIe bandwidth metrics for the NVIDIA Tegra410 SoC. The formulas follow
Documentation/admin-guide/perf/nvidia-tegra410-pmu.rst:
AVG_RD_BANDWIDTH_IN_GBPS=RD_BYTES/ELAPSED_TIME_IN_NSAVG_WR_BANDWIDTH_IN_GBPS=WR_BYTES/ELAPSED_TIME_IN_NSThere is one PCIE PMU per socket and PCIe Root Complex (RC), named nvidia_pcie_pmu_<socket-id>_rc_<pcie-rc-id>, with RCs 0 to 5 in each socket. The metrics sum the rd_bytes/wr_bytes events of both sockets:
lpm_pcie_bw,lpm_pcie_rd_bw,lpm_pcie_wr_bw: all RCs. lpm_pcie_bw_<rc>, lpm_pcie_rd_bw_<rc>, lpm_pcie_wr_bw_<rc>: one RC. lpm_pcie_loc_rd_bw_<rc>, lpm_pcie_loc_wr_bw_<rc>: one RC to local
CMEM, GMEM, PCIe peer and CXL memory.
lpm_pcie_rem_rd_bw_<rc>, lpm_pcie_rem_wr_bw_<rc>: one RC to remote
memory.
Quoted 8 lines
The per RC metrics are in the lpm_pcie_rc<rc>_bw metric groups.
A socket's event counts as 0 if its PMU isn't present (
has_event()is 0), e.g. on a single socket system, or if it isn't counted in the aggregation (source_count()is 0), e.g. the other socket's event with --per-socket. Otherwise the metric fails to parse or is nan.Signed-off-by Chun-Tse Shao <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
RE: [PATCH 2/2] perf jevents: Add NVIDIA Tegra410 uncore PCIe metrics
In reply to Chun-Tse ShaoHi Chun-Tse,
I tested this on 2-sockets Tegra410 and found some issues.
1. With --per-socket or -A, every lpm_pcie_* metric is only reported for socket 0, so socket 1's PCIe traffic never shows up. Example output:
perf stat -a -A -M lpm_pcie_rc1_bw -- sleep 1
Performance counter stats for 'system wide':
CPU0 15,204 nvidia_pcie_pmu_0_rc_1/wr_bytes/ # 0.0 MB/s lpm_pcie_wr_bw_1
CPU0 # 33.8 MB/s lpm_pcie_bw_1
CPU0 33,910,952 nvidia_pcie_pmu_0_rc_1/rd_bytes/ # 33.8 MB/s lpm_pcie_rd_bw_1 CPU88 0 nvidia_pcie_pmu_1_rc_1/wr_bytes/ CPU88 0 nvidia_pcie_pmu_1_rc_1/rd_bytes/ CPU0 1,002,286,615 tool/duration_time/ CPU0 0 nvidia_pcie_pmu_0_rc_1/wr_bytes,dst_rem=0x1/ # 0.0 MB/s lpm_pcie_rem_wr_bw_1 CPU88 0 nvidia_pcie_pmu_1_rc_1/wr_bytes,dst_rem=0x1/ CPU0 1,002,275,059 tool/duration_time/ CPU0 0 nvidia_pcie_pmu_0_rc_1/rd_bytes,dst_rem=0x1/ # 0.0 MB/s lpm_pcie_rem_rd_bw_1 CPU88 0 nvidia_pcie_pmu_1_rc_1/rd_bytes,dst_rem=0x1/
<snip>
Note CPU88 (socket 1_ has no lpm_pcie_* values.
2. In the default mode, lpm_pcie_*_<i> adds RC i of socket 0 and RC i of socket 1. These are separate root complexes with different devices, so the "Root Complex i" value can combine traffic from two unrelated links.
Could the metrics follow the hardware hierarchy instead, starting at the socket and drilling down to the RC, with the same destination split at each level?
- per socket, all RCs (nvidia_pcie_pmu_<s>_rc):
lpm_pcie_s<s>_{bw,rd_bw,wr_bw}
lpm_pcie_s<s>_{loc,rem}_{rd,wr}_bw
- per socket and RC (nvidia_pcie_pmu_<s>_rc_<i>):
lpm_pcie_s<s>_{bw,rd_bw,wr_bw}_<i>
lpm_pcie_s<s>_{loc,rem}_{rd,wr}_bw_<i>
grouped as lpm_pcie_s<s>_rc<i>_bw.Thanks,
Besar
Quoted 148 lines
-----Original Message-----
From: Chun-Tse Shao <[email protected]>
Sent: Monday, September 28, 2026 5:51 PM
To: [email protected]; [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Besar Wicaksono <[email protected]>; [email protected]; linux- [email protected]; Chun-Tse Shao <[email protected]>
Subject: [PATCH 2/2] perf jevents: Add NVIDIA Tegra410 uncore PCIe metricsExternal email: Use caution opening links or attachments
Add PCIe bandwidth metrics for the NVIDIA Tegra410 SoC. The formulas follow
Documentation/admin-guide/perf/nvidia-tegra410-pmu.rst:
AVG_RD_BANDWIDTH_IN_GBPS=RD_BYTES/ELAPSED_TIME_IN_NSAVG_WR_BANDWIDTH_IN_GBPS=WR_BYTES/ELAPSED_TIME_IN_NSThere is one PCIE PMU per socket and PCIe Root Complex (RC), named nvidia_pcie_pmu_<socket-id>_rc_<pcie-rc-id>, with RCs 0 to 5 in each socket. The metrics sum the rd_bytes/wr_bytes events of both sockets:
lpm_pcie_bw,lpm_pcie_rd_bw,lpm_pcie_wr_bw: all RCs. lpm_pcie_bw_<rc>, lpm_pcie_rd_bw_<rc>, lpm_pcie_wr_bw_<rc>: one RC. lpm_pcie_loc_rd_bw_<rc>, lpm_pcie_loc_wr_bw_<rc>: one RC to local
CMEM, GMEM, PCIe peer and CXL memory.
lpm_pcie_rem_rd_bw_<rc>, lpm_pcie_rem_wr_bw_<rc>: one RC to remote
memory.The per RC metrics are in the lpm_pcie_rc<rc>_bw metric groups.
A socket's event counts as 0 if its PMU isn't present (
has_event()is 0), e.g. on a single socket system, or if it isn't counted in the aggregation (source_count()is 0), e.g. the other socket's event with --per-socket. Otherwise the metric fails to parse or is nan.Signed-off-by Chun-Tse Shao <[email protected]>Assisted-by Gemini:gemini-3.1-pro-preview--- tools/perf/pmu-events/arm64_metrics.py | 68 ++++++++++++++++++++++++-- 1 file changed, 64 insertions(+), 4 deletions(-)tools/perf/pmu- renamed from tools/perf/pmu-events/arm64_metrics.py +0
-0
events/arm64_metrics.py
indexb0d0651fec12..1ca3d8ddb4b7100755tools/perf/pmu-events/arm64_metrics.py +4
-3@@ -2,9 +2,10 @@ # SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) import argparse import os -from typing import Optional -from metric import (aggr_nr, d_ratio, Event, JsonEncodeMetric, - JsonEncodeMetricGroupDescriptions, LoadEvents, Metric, MetricGroup) +from typing import List, Optional, Union +from metric import (aggr_nr, d_ratio, has_event, source_count, Event, Expression, + JsonEncodeMetric, JsonEncodeMetricGroupDescriptions, LoadEvents, Metric, + MetricGroup, Select)from
common_metricsimport Cycles# Global command line arguments.
tools/perf/pmu-events/arm64_metrics.py +4
-1@@ -67,7 +68,66 @@ def NvidiaT410Uncore() -> Optional[MetricGroup]: rd_cum_outs - rd_cum_outs, "1ns"), ], description="NVIDIA Tegra410 DDR latency") - return MetricGroup("lpm_uncore", [DdrBw(), DdrLat()], + def PcieBw() -> MetricGroup: + # PCIE PMU, one per socket and PCIe Root Complex (RC): + # AVG_{RD,WR}_BANDWIDTH = {RD,WR}_BYTES / ELAPSED_TIME + loc_dst ="dst_loc_cmem=0x1,dst_loc_gmem=0x1,dst_loc_pcie_p2p=0x1,dst_loc_pci e_cxl=0x1"
tools/perf/pmu-events/arm64_metrics.py +13
-0+ rem_dst = "dst_rem=0x1" + + def SocketSum(rc: str, event: str) -> Expression: + # Sum the event over both sockets. A PMU counts as 0 if it isn't + # present (has_event() is 0) or isn't counted in the aggregation, + # e.g. the other socket with --per-socket (source_count() is 0). + def Count(e: Event) -> Expression: + return Select(Select(e, source_count(e), 0), has_event(e), 0) + + return (Count(Event(f"nvidia_pcie_pmu_0_{rc}/{event}/")) + + Count(Event(f"nvidia_pcie_pmu_1_{rc}/{event}/"))) + + # nvidia_pcie_pmu_<socket-id>_rc matches the PMUs of all RCs in thesocket.
tools/perf/pmu-events/arm64_metrics.py +4
-0+ rd = SocketSum("rc", "rd_bytes") + wr = SocketSum("rc", "wr_bytes") + metrics: List[Union[Metric, MetricGroup]] = [ + Metric("lpm_pcie_bw", "Total PCIe bandwidth across all RootComplexes",
tools/perf/pmu-events/arm64_metrics.py +2
-0+ d_ratio(rd + wr, interval_sec), mb_per_sec), + Metric("lpm_pcie_rd_bw", "Total PCIe read bandwidth across allRoot Complexes",
tools/perf/pmu-events/arm64_metrics.py +2
-0+ d_ratio(rd, interval_sec), mb_per_sec), + Metric("lpm_pcie_wr_bw", "Total PCIe write bandwidth across allRoot Complexes",
tools/perf/pmu-events/arm64_metrics.py +12
-0+ d_ratio(wr, interval_sec), mb_per_sec), + ] + for i in range(6): + rc = f"rc_{i}" + rc_rd = SocketSum(rc, "rd_bytes") + rc_wr = SocketSum(rc, "wr_bytes") + loc_rd = SocketSum(rc, f"rd_bytes,{loc_dst}") + loc_wr = SocketSum(rc, f"wr_bytes,{loc_dst}") + rem_rd = SocketSum(rc, f"rd_bytes,{rem_dst}") + rem_wr = SocketSum(rc, f"wr_bytes,{rem_dst}") + metrics.append(MetricGroup(f"lpm_pcie_rc{i}_bw", [ + Metric(f"lpm_pcie_bw_{i}", f"PCIe total bandwidth on RootComplex {i}",
tools/perf/pmu-events/arm64_metrics.py +2
-0+ d_ratio(rc_rd + rc_wr, interval_sec), mb_per_sec), + Metric(f"lpm_pcie_rd_bw_{i}", f"PCIe read bandwidth on RootComplex {i}",
tools/perf/pmu-events/arm64_metrics.py +2
-0+ d_ratio(rc_rd, interval_sec), mb_per_sec), + Metric(f"lpm_pcie_wr_bw_{i}", f"PCIe write bandwidth on RootComplex {i}",
tools/perf/pmu-events/arm64_metrics.py +17
-0+ d_ratio(rc_wr, interval_sec), mb_per_sec), + Metric(f"lpm_pcie_loc_rd_bw_{i}", + f"PCIe local read bandwidth on Root Complex {i}", + d_ratio(loc_rd, interval_sec), mb_per_sec), + Metric(f"lpm_pcie_loc_wr_bw_{i}", + f"PCIe local write bandwidth on Root Complex {i}", + d_ratio(loc_wr, interval_sec), mb_per_sec), + Metric(f"lpm_pcie_rem_rd_bw_{i}", + f"PCIe remote read bandwidth on Root Complex {i}", + d_ratio(rem_rd, interval_sec), mb_per_sec), + Metric(f"lpm_pcie_rem_wr_bw_{i}", + f"PCIe remote write bandwidth on Root Complex {i}", + d_ratio(rem_wr, interval_sec), mb_per_sec), + ], description=f"PCIe bandwidth on Root Complex {i}")) + + return MetricGroup("lpm_pcie_bw", metrics, + description="NVIDIA Tegra410 PCIe Root Complexbandwidth")
tools/perf/pmu-events/arm64_metrics.py +2
-1+ + return MetricGroup("lpm_uncore", [DdrBw(), DdrLat(), PcieBw()], description="NVIDIA Tegra410 uncore metrics") --2.56.0.rc1.315.gc6ed9934b7-goog
Re: [PATCH 2/2] perf jevents: Add NVIDIA Tegra410 uncore PCIe metrics
Quoted 33 lines · Besar Wicaksono
Hi Chun-Tse,
I tested this on 2-sockets Tegra410 and found some issues.
1. With --per-socket or -A, every lpm_pcie_* metric is only reported for socket 0, so socket 1's PCIe traffic never shows up. Example output:perf stat -a -A -M
lpm_pcie_rc1_bw-- sleep 1Performance counter stats for 'system wide':
CPU0 15,204 nvidia_pcie_pmu_0_rc_1/wr_bytes/ # 0.0 MB/s
lpm_pcie_wr_bw_1
CPU0 # 33.8 MB/slpm_pcie_bw_1CPU0 33,910,952 nvidia_pcie_pmu_0_rc_1/rd_bytes/ # 33.8 MB/s lpm_pcie_rd_bw_1 CPU88 0 nvidia_pcie_pmu_1_rc_1/wr_bytes/ CPU88 0 nvidia_pcie_pmu_1_rc_1/rd_bytes/ CPU0 1,002,286,615 tool/duration_time/ CPU0 0 nvidia_pcie_pmu_0_rc_1/wr_bytes,dst_rem=0x1/ # 0.0 MB/s lpm_pcie_rem_wr_bw_1 CPU88 0 nvidia_pcie_pmu_1_rc_1/wr_bytes,dst_rem=0x1/ CPU0 1,002,275,059 tool/duration_time/ CPU0 0 nvidia_pcie_pmu_0_rc_1/rd_bytes,dst_rem=0x1/ # 0.0 MB/s lpm_pcie_rem_rd_bw_1 CPU88 0 nvidia_pcie_pmu_1_rc_1/rd_bytes,dst_rem=0x1/<snip>
Note CPU88 (socket 1_ has no lpm_pcie_* values.
2. In the default mode, lpm_pcie_*_<i> adds RC i of socket 0 and RC i of socket 1. These are separate root complexes with different devices, so the "Root Complex i" value can combine traffic from two unrelated links.
Could the metrics follow the hardware hierarchy instead, starting at the socket and drilling down to the RC, with the same destination split at each level?
Quoted 7 lines
- per socket, all RCs (nvidia_pcie_pmu_<s>_rc):
lpm_pcie_s<s>_{bw,rd_bw,wr_bw}
lpm_pcie_s<s>_{loc,rem}_{rd,wr}_bw
- per socket and RC (nvidia_pcie_pmu_<s>_rc_<i>):
lpm_pcie_s<s>_{bw,rd_bw,wr_bw}_<i>
lpm_pcie_s<s>_{loc,rem}_{rd,wr}_bw_<i>
grouped as lpm_pcie_s<s>_rc<i>_bw.Thanks for testing and for the suggestion. I think both problems come from how perf expects a PMU to expose its cpumask. Ideally perf wants a single PMU for both sockets, with one CPU per socket in its cpumask, e.g.:
``` /sys/bus/event_source/devices/nvidia_pcie_rc_0/cpumask: 0,88 ```
We can work around the one-PMU-per-socket layout in the metrics, but would a single PMU per RC be a viable plan for you to implement?
Thanks,
CT
Quoted 153 lines
Thanks,
BesarQuoted 148 lines
-----Original Message-----
From: Chun-Tse Shao <[email protected]>
Sent: Monday, September 28, 2026 5:51 PM
To: [email protected]; [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Besar Wicaksono <[email protected]>; [email protected]; linux- [email protected]; Chun-Tse Shao <[email protected]>
Subject: [PATCH 2/2] perf jevents: Add NVIDIA Tegra410 uncore PCIe metricsExternal email: Use caution opening links or attachments
Add PCIe bandwidth metrics for the NVIDIA Tegra410 SoC. The formulas follow
Documentation/admin-guide/perf/nvidia-tegra410-pmu.rst:
AVG_RD_BANDWIDTH_IN_GBPS=RD_BYTES/ELAPSED_TIME_IN_NSAVG_WR_BANDWIDTH_IN_GBPS=WR_BYTES/ELAPSED_TIME_IN_NSThere is one PCIE PMU per socket and PCIe Root Complex (RC), named nvidia_pcie_pmu_<socket-id>_rc_<pcie-rc-id>, with RCs 0 to 5 in each socket. The metrics sum the rd_bytes/wr_bytes events of both sockets:
lpm_pcie_bw,lpm_pcie_rd_bw,lpm_pcie_wr_bw: all RCs. lpm_pcie_bw_<rc>, lpm_pcie_rd_bw_<rc>, lpm_pcie_wr_bw_<rc>: one RC. lpm_pcie_loc_rd_bw_<rc>, lpm_pcie_loc_wr_bw_<rc>: one RC to local
CMEM, GMEM, PCIe peer and CXL memory.
lpm_pcie_rem_rd_bw_<rc>, lpm_pcie_rem_wr_bw_<rc>: one RC to remote
memory.The per RC metrics are in the lpm_pcie_rc<rc>_bw metric groups.
A socket's event counts as 0 if its PMU isn't present (
has_event()is 0), e.g. on a single socket system, or if it isn't counted in the aggregation (source_count()is 0), e.g. the other socket's event with --per-socket. Otherwise the metric fails to parse or is nan.Signed-off-by Chun-Tse Shao <[email protected]>Assisted-by Gemini:gemini-3.1-pro-preview--- tools/perf/pmu-events/arm64_metrics.py | 68 ++++++++++++++++++++++++-- 1 file changed, 64 insertions(+), 4 deletions(-)tools/perf/pmu- renamed from tools/perf/pmu-events/arm64_metrics.py +0
-0
events/arm64_metrics.py
indexb0d0651fec12..1ca3d8ddb4b7100755tools/perf/pmu-events/arm64_metrics.py +4
-3@@ -2,9 +2,10 @@ # SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) import argparse import os -from typing import Optional -from metric import (aggr_nr, d_ratio, Event, JsonEncodeMetric, - JsonEncodeMetricGroupDescriptions, LoadEvents, Metric, MetricGroup) +from typing import List, Optional, Union +from metric import (aggr_nr, d_ratio, has_event, source_count, Event, Expression, + JsonEncodeMetric, JsonEncodeMetricGroupDescriptions, LoadEvents, Metric, + MetricGroup, Select)from
common_metricsimport Cycles# Global command line arguments.
tools/perf/pmu-events/arm64_metrics.py +4
-1@@ -67,7 +68,66 @@ def NvidiaT410Uncore() -> Optional[MetricGroup]: rd_cum_outs - rd_cum_outs, "1ns"), ], description="NVIDIA Tegra410 DDR latency") - return MetricGroup("lpm_uncore", [DdrBw(), DdrLat()], + def PcieBw() -> MetricGroup: + # PCIE PMU, one per socket and PCIe Root Complex (RC): + # AVG_{RD,WR}_BANDWIDTH = {RD,WR}_BYTES / ELAPSED_TIME + loc_dst ="dst_loc_cmem=0x1,dst_loc_gmem=0x1,dst_loc_pcie_p2p=0x1,dst_loc_pci e_cxl=0x1"
tools/perf/pmu-events/arm64_metrics.py +13
-0+ rem_dst = "dst_rem=0x1" + + def SocketSum(rc: str, event: str) -> Expression: + # Sum the event over both sockets. A PMU counts as 0 if it isn't + # present (has_event() is 0) or isn't counted in the aggregation, + # e.g. the other socket with --per-socket (source_count() is 0). + def Count(e: Event) -> Expression: + return Select(Select(e, source_count(e), 0), has_event(e), 0) + + return (Count(Event(f"nvidia_pcie_pmu_0_{rc}/{event}/")) + + Count(Event(f"nvidia_pcie_pmu_1_{rc}/{event}/"))) + + # nvidia_pcie_pmu_<socket-id>_rc matches the PMUs of all RCs in thesocket.
tools/perf/pmu-events/arm64_metrics.py +4
-0+ rd = SocketSum("rc", "rd_bytes") + wr = SocketSum("rc", "wr_bytes") + metrics: List[Union[Metric, MetricGroup]] = [ + Metric("lpm_pcie_bw", "Total PCIe bandwidth across all RootComplexes",
tools/perf/pmu-events/arm64_metrics.py +2
-0+ d_ratio(rd + wr, interval_sec), mb_per_sec), + Metric("lpm_pcie_rd_bw", "Total PCIe read bandwidth across allRoot Complexes",
tools/perf/pmu-events/arm64_metrics.py +2
-0+ d_ratio(rd, interval_sec), mb_per_sec), + Metric("lpm_pcie_wr_bw", "Total PCIe write bandwidth across allRoot Complexes",
tools/perf/pmu-events/arm64_metrics.py +12
-0+ d_ratio(wr, interval_sec), mb_per_sec), + ] + for i in range(6): + rc = f"rc_{i}" + rc_rd = SocketSum(rc, "rd_bytes") + rc_wr = SocketSum(rc, "wr_bytes") + loc_rd = SocketSum(rc, f"rd_bytes,{loc_dst}") + loc_wr = SocketSum(rc, f"wr_bytes,{loc_dst}") + rem_rd = SocketSum(rc, f"rd_bytes,{rem_dst}") + rem_wr = SocketSum(rc, f"wr_bytes,{rem_dst}") + metrics.append(MetricGroup(f"lpm_pcie_rc{i}_bw", [ + Metric(f"lpm_pcie_bw_{i}", f"PCIe total bandwidth on RootComplex {i}",
tools/perf/pmu-events/arm64_metrics.py +2
-0+ d_ratio(rc_rd + rc_wr, interval_sec), mb_per_sec), + Metric(f"lpm_pcie_rd_bw_{i}", f"PCIe read bandwidth on RootComplex {i}",
tools/perf/pmu-events/arm64_metrics.py +2
-0+ d_ratio(rc_rd, interval_sec), mb_per_sec), + Metric(f"lpm_pcie_wr_bw_{i}", f"PCIe write bandwidth on RootComplex {i}",
tools/perf/pmu-events/arm64_metrics.py +17
-0+ d_ratio(rc_wr, interval_sec), mb_per_sec), + Metric(f"lpm_pcie_loc_rd_bw_{i}", + f"PCIe local read bandwidth on Root Complex {i}", + d_ratio(loc_rd, interval_sec), mb_per_sec), + Metric(f"lpm_pcie_loc_wr_bw_{i}", + f"PCIe local write bandwidth on Root Complex {i}", + d_ratio(loc_wr, interval_sec), mb_per_sec), + Metric(f"lpm_pcie_rem_rd_bw_{i}", + f"PCIe remote read bandwidth on Root Complex {i}", + d_ratio(rem_rd, interval_sec), mb_per_sec), + Metric(f"lpm_pcie_rem_wr_bw_{i}", + f"PCIe remote write bandwidth on Root Complex {i}", + d_ratio(rem_wr, interval_sec), mb_per_sec), + ], description=f"PCIe bandwidth on Root Complex {i}")) + + return MetricGroup("lpm_pcie_bw", metrics, + description="NVIDIA Tegra410 PCIe Root Complexbandwidth")
tools/perf/pmu-events/arm64_metrics.py +2
-1+ + return MetricGroup("lpm_uncore", [DdrBw(), DdrLat(), PcieBw()], description="NVIDIA Tegra410 uncore metrics") --2.56.0.rc1.315.gc6ed9934b7-goog
RE: [PATCH 2/2] perf jevents: Add NVIDIA Tegra410 uncore PCIe metrics
Quoted 67 lines
-----Original Message-----
From: Chun-Tse Shao <[email protected]>
Sent: Friday, October 16, 2026 6:40 PM
To: Besar Wicaksono <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; linux-perf- [email protected]; [email protected]; Matt Ochs <[email protected]>; Sean Kelley <[email protected]>; Rich Wiley <[email protected]>; Vikram Sethi <[email protected]>; Raj Joshi <[email protected]>
Subject: Re: [PATCH 2/2] perf jevents: Add NVIDIA Tegra410 uncore PCIe metricsExternal email: Use caution opening links or attachments
Quoted 7 lines · Besar Wicaksono
Hi Chun-Tse,
I tested this on 2-sockets Tegra410 and found some issues.
1. With --per-socket or -A, every lpm_pcie_* metric is only reported for socket 0, so socket 1's PCIe traffic never shows up. Example output:Quoted 5 lines
perf stat -a -A -M
lpm_pcie_rc1_bw-- sleep 1Performance counter stats for 'system wide':
CPU0 15,204 nvidia_pcie_pmu_0_rc_1/wr_bytes/ # 0.0 MB/s
lpm_pcie_wr_bw_1Quoted 2 lines
CPU0 # 33.8 MB/s
lpm_pcie_bw_1
CPU0 33,910,952 nvidia_pcie_pmu_0_rc_1/rd_bytes/ # 33.8MB/s
lpm_pcie_rd_bw_1Quoted 4 lines
CPU88 0 nvidia_pcie_pmu_1_rc_1/wr_bytes/ CPU88 0 nvidia_pcie_pmu_1_rc_1/rd_bytes/ CPU0 1,002,286,615 tool/duration_time/ CPU0 0 nvidia_pcie_pmu_0_rc_1/wr_bytes,dst_rem=0x1/ #0.0 MB/s
lpm_pcie_rem_wr_bw_1Quoted 3 lines
CPU88 0 nvidia_pcie_pmu_1_rc_1/wr_bytes,dst_rem=0x1/ CPU0 1,002,275,059 tool/duration_time/ CPU0 0 nvidia_pcie_pmu_0_rc_1/rd_bytes,dst_rem=0x1/ #0.0 MB/s
lpm_pcie_rem_rd_bw_1Quoted 12 lines
CPU88 0 nvidia_pcie_pmu_1_rc_1/rd_bytes,dst_rem=0x1/
<snip>Note CPU88 (socket 1_ has no lpm_pcie_* values.
2. In the default mode, lpm_pcie_*_<i> adds RC i of socket 0 and RC i of socket 1. These are separate root complexes with different devices, so the "Root Complex i" value can combine traffic from two unrelated links.
Could the metrics follow the hardware hierarchy instead, starting at the socket and drilling down to the RC, with the same destination split at each level?Quoted 7 lines
- per socket, all RCs (nvidia_pcie_pmu_<s>_rc): lpm_pcie_s<s>_{bw,rd_bw,wr_bw} lpm_pcie_s<s>_{loc,rem}_{rd,wr}_bw - per socket and RC (nvidia_pcie_pmu_<s>_rc_<i>): lpm_pcie_s<s>_{bw,rd_bw,wr_bw}_<i> lpm_pcie_s<s>_{loc,rem}_{rd,wr}_bw_<i> grouped as lpm_pcie_s<s>_rc<i>_bw.
Quoted 9 lines
Thanks for testing and for the suggestion. I think both problems come from how perf expects a PMU to expose its cpumask. Ideally perf wants a single PMU for both sockets, with one CPU per socket in its cpumask, e.g.:
``` /sys/bus/event_source/devices/nvidia_pcie_rc_0/cpumask: 0,88 ```
I see this as a driver design choice: expose each physical PMU separately, or group several physical instances under one logical PMU. Either way, the PMU name and cpumask tell userspace which CPU(s) and root complex are associated with the PMU. With the first approach, each perf PMU has a cpumask of a single CPU. Tegra410 follows the first approach and exposes each physical PMU as a separate perf PMU.
Quoted 3 lines
We can work around the one-PMU-per-socket layout in the metrics, but would a single PMU per RC be a viable plan for you to implement?
Grouping instances across sockets is an interesting alternative, but it would require changes to the arm_cspmu driver design. We would also need to evaluate how such grouping could be supported generically in arm_cspmu, and whether the ACPI APMT would need changes.
For this patch, I think userspace gives us more flexibility in how we present and aggregate counts across sockets and root complexes, so I'd prefer to address it there and keep the current driver model.
FYI, I've posted the Tegra410 uncore JSON series:
https://lore.kernel.org/all/[email protected]/
It adds events and metrics for the same counters (e.g. ucf_avg_*, cmem_avg_rd_latency, pcie_avg_*), so it would be good to align the two and avoid duplicate metrics.
Thanks,
Besar
Quoted 163 lines
Thanks,
CTQuoted 9 lines
Thanks,
BesarQuoted 5 lines
-----Original Message-----
From: Chun-Tse Shao <[email protected]>
Sent: Monday, September 28, 2026 5:51 PM
To: [email protected]; [email protected]; [email protected]
Cc: [email protected]; [email protected];Quoted 16 lines
Quoted 16 lines
[email protected]; [email protected]; [email protected]; [email protected]; Besar Wicaksono <[email protected]>; [email protected]; linux- [email protected]; Chun-Tse Shao <[email protected]>
Subject: [PATCH 2/2] perf jevents: Add NVIDIA Tegra410 uncore PCIe metricsExternal email: Use caution opening links or attachments
Add PCIe bandwidth metrics for the NVIDIA Tegra410 SoC. The formulas follow
Documentation/admin-guide/perf/nvidia-tegra410-pmu.rst:
AVG_RD_BANDWIDTH_IN_GBPS=RD_BYTES/ELAPSED_TIME_IN_NSAVG_WR_BANDWIDTH_IN_GBPS=WR_BYTES/ELAPSED_TIME_IN_NSQuoted 6 lines
Quoted 6 lines
There is one PCIE PMU per socket and PCIe Root Complex (RC), named nvidia_pcie_pmu_<socket-id>_rc_<pcie-rc-id>, with RCs 0 to 5 in each socket. The metrics sum the rd_bytes/wr_bytes events of both sockets:
lpm_pcie_bw,lpm_pcie_rd_bw,lpm_pcie_wr_bw: all RCs. lpm_pcie_bw_<rc>, lpm_pcie_rd_bw_<rc>, lpm_pcie_wr_bw_<rc>: oneRC.
Quoted 3 lines
Quoted 3 lines
lpm_pcie_loc_rd_bw_<rc>, lpm_pcie_loc_wr_bw_<rc>: one RC to local
CMEM, GMEM, PCIe peer and CXL memory.
lpm_pcie_rem_rd_bw_<rc>, lpm_pcie_rem_wr_bw_<rc>: one RC toremote
Quoted 37 lines
Quoted 37 lines
memory.
The per RC metrics are in the lpm_pcie_rc<rc>_bw metric groups.
A socket's event counts as 0 if its PMU isn't present (
has_event()is 0), e.g. on a single socket system, or if it isn't counted in the aggregation (source_count()is 0), e.g. the other socket's event with --per-socket. Otherwise the metric fails to parse or is nan.Signed-off-by Chun-Tse Shao <[email protected]>Assisted-by Gemini:gemini-3.1-pro-preview--- tools/perf/pmu-events/arm64_metrics.py | 68 ++++++++++++++++++++++++-- 1 file changed, 64 insertions(+), 4 deletions(-)tools/perf/pmu- renamed from tools/perf/pmu-events/arm64_metrics.py +0
-0
events/arm64_metrics.py
indexb0d0651fec12..1ca3d8ddb4b7100755tools/perf/pmu-events/arm64_metrics.py +4
-3@@ -2,9 +2,10 @@ # SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) import argparse import os -from typing import Optional -from metric import (aggr_nr, d_ratio, Event, JsonEncodeMetric, - JsonEncodeMetricGroupDescriptions, LoadEvents, Metric, MetricGroup) +from typing import List, Optional, Union +from metric import (aggr_nr, d_ratio, has_event, source_count, Event, Expression, + JsonEncodeMetric, JsonEncodeMetricGroupDescriptions, LoadEvents, Metric, + MetricGroup, Select)from
common_metricsimport CyclesQuoted 2 lines
tools/perf/pmu-events/arm64_metrics.py def NvidiaT410Uncore() ->
Quoted 2 lines
# Global command line arguments.
tools/perf/pmu-events/arm64_metrics.py +0
-0@@ -67,7 +68,66 @@ def NvidiaT410Uncore() ->Optional[MetricGroup]:
Quoted 9 lines
tools/perf/pmu-events/arm64_metrics.py def NvidiaT410Uncore() -> Optional[MetricGroup]: · line 132
Quoted 9 lines
tools/perf/pmu-events/arm64_metrics.py +4
-1rd_cum_outs - rd_cum_outs, "1ns"), ], description="NVIDIA Tegra410 DDR latency") - return MetricGroup("lpm_uncore", [DdrBw(), DdrLat()], + def PcieBw() -> MetricGroup: + # PCIE PMU, one per socket and PCIe Root Complex (RC): + # AVG_{RD,WR}_BANDWIDTH = {RD,WR}_BYTES / ELAPSED_TIME + loc_dst ="dst_loc_cmem=0x1,dst_loc_gmem=0x1,dst_loc_pcie_p2p=0x1,dst_loc_pci
Quoted 6 lines
Quoted 6 lines
e_cxl=0x1"
tools/perf/pmu-events/arm64_metrics.py +5
-0+ rem_dst = "dst_rem=0x1" + + def SocketSum(rc: str, event: str) -> Expression: + # Sum the event over both sockets. A PMU counts as 0 if it isn't + # present (has_event() is 0) or isn't counted in the aggregation,Quoted 8 lines
tools/perf/pmu-events/arm64_metrics.py def NvidiaT410Uncore() -> Optional[MetricGroup]:
Quoted 8 lines
tools/perf/pmu-events/arm64_metrics.py +8
-0+ # e.g. the other socket with --per-socket (source_count() is 0). + def Count(e: Event) -> Expression: + return Select(Select(e, source_count(e), 0), has_event(e), 0) + + return (Count(Event(f"nvidia_pcie_pmu_0_{rc}/{event}/")) + + Count(Event(f"nvidia_pcie_pmu_1_{rc}/{event}/"))) + + # nvidia_pcie_pmu_<socket-id>_rc matches the PMUs of all RCs inthe
Quoted 22 lines
Quoted 22 lines
socket.
tools/perf/pmu-events/arm64_metrics.py +4
-0+ rd = SocketSum("rc", "rd_bytes") + wr = SocketSum("rc", "wr_bytes") + metrics: List[Union[Metric, MetricGroup]] = [ + Metric("lpm_pcie_bw", "Total PCIe bandwidth across all RootComplexes",
tools/perf/pmu-events/arm64_metrics.py +2
-0+ d_ratio(rd + wr, interval_sec), mb_per_sec), + Metric("lpm_pcie_rd_bw", "Total PCIe read bandwidth across allRoot Complexes",
tools/perf/pmu-events/arm64_metrics.py +2
-0+ d_ratio(rd, interval_sec), mb_per_sec), + Metric("lpm_pcie_wr_bw", "Total PCIe write bandwidth across allRoot Complexes",
tools/perf/pmu-events/arm64_metrics.py +10
-0+ d_ratio(wr, interval_sec), mb_per_sec), + ] + for i in range(6): + rc = f"rc_{i}" + rc_rd = SocketSum(rc, "rd_bytes") + rc_wr = SocketSum(rc, "wr_bytes") + loc_rd = SocketSum(rc, f"rd_bytes,{loc_dst}") + loc_wr = SocketSum(rc, f"wr_bytes,{loc_dst}") + rem_rd = SocketSum(rc, f"rd_bytes,{rem_dst}") + rem_wr = SocketSum(rc, f"wr_bytes,{rem_dst}")Quoted 8 lines
tools/perf/pmu-events/arm64_metrics.py
Quoted 8 lines
tools/perf/pmu-events/arm64_metrics.py +2
-0+ metrics.append(MetricGroup(f"lpm_pcie_rc{i}_bw", [ + Metric(f"lpm_pcie_bw_{i}", f"PCIe total bandwidth on RootComplex {i}",
tools/perf/pmu-events/arm64_metrics.py +2
-0+ d_ratio(rc_rd + rc_wr, interval_sec), mb_per_sec), + Metric(f"lpm_pcie_rd_bw_{i}", f"PCIe read bandwidth on RootComplex {i}",
tools/perf/pmu-events/arm64_metrics.py +2
-0+ d_ratio(rc_rd, interval_sec), mb_per_sec), + Metric(f"lpm_pcie_wr_bw_{i}", f"PCIe write bandwidth onRoot
Quoted 27 lines
Quoted 26 lines
Complex {i}",
tools/perf/pmu-events/arm64_metrics.py +17
-0+ d_ratio(rc_wr, interval_sec), mb_per_sec), + Metric(f"lpm_pcie_loc_rd_bw_{i}", + f"PCIe local read bandwidth on Root Complex {i}", + d_ratio(loc_rd, interval_sec), mb_per_sec), + Metric(f"lpm_pcie_loc_wr_bw_{i}", + f"PCIe local write bandwidth on Root Complex {i}", + d_ratio(loc_wr, interval_sec), mb_per_sec), + Metric(f"lpm_pcie_rem_rd_bw_{i}", + f"PCIe remote read bandwidth on Root Complex {i}", + d_ratio(rem_rd, interval_sec), mb_per_sec), + Metric(f"lpm_pcie_rem_wr_bw_{i}", + f"PCIe remote write bandwidth on Root Complex {i}", + d_ratio(rem_wr, interval_sec), mb_per_sec), + ], description=f"PCIe bandwidth on Root Complex {i}")) + + return MetricGroup("lpm_pcie_bw", metrics, + description="NVIDIA Tegra410 PCIe Root Complexbandwidth")
tools/perf/pmu-events/arm64_metrics.py +2
-1+ + return MetricGroup("lpm_uncore", [DdrBw(), DdrLat(), PcieBw()], description="NVIDIA Tegra410 uncore metrics") --2.56.0.rc1.315.gc6ed9934b7-goog
Re: [PATCH 2/2] perf jevents: Add NVIDIA Tegra410 uncore PCIe metrics
Hi Besar,
Thanks, that makes sense. I'll drop my series in favor of yours.
Two things from mine that your JSON doesn't have yet:
1. DDR bandwidth from all sources to local CMEM (mem_bytes_{rd,wr} with only dst_loc_cmem=1). Today it needs the sum of the local_cpu, local_noncpu and remote _to_local_cmem metrics.
2. Read + write total bandwidth, for UCF memory and PCIe.
Agreed on handling the PCIe socket grouping in userspace; the --metric-only issue I reported on your 3/6 is part of that.
Thanks,
CT
Quoted 270 lines · Besar Wicaksono
Quoted 68 lines
-----Original Message-----
From: Chun-Tse Shao <[email protected]>
Sent: Friday, October 16, 2026 6:40 PM
To: Besar Wicaksono <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; linux-perf- [email protected]; [email protected]; Matt Ochs <[email protected]>; Sean Kelley <[email protected]>; Rich Wiley <[email protected]>; Vikram Sethi <[email protected]>; Raj Joshi <[email protected]>
Subject: Re: [PATCH 2/2] perf jevents: Add NVIDIA Tegra410 uncore PCIe metricsExternal email: Use caution opening links or attachments
Quoted 7 lines · Besar Wicaksono
Hi Chun-Tse,
I tested this on 2-sockets Tegra410 and found some issues.
1. With --per-socket or -A, every lpm_pcie_* metric is only reported for socket 0, so socket 1's PCIe traffic never shows up. Example output:Quoted 5 lines
perf stat -a -A -M
lpm_pcie_rc1_bw-- sleep 1Performance counter stats for 'system wide':
CPU0 15,204 nvidia_pcie_pmu_0_rc_1/wr_bytes/ # 0.0 MB/s
lpm_pcie_wr_bw_1Quoted 2 lines
CPU0 # 33.8 MB/s
lpm_pcie_bw_1
CPU0 33,910,952 nvidia_pcie_pmu_0_rc_1/rd_bytes/ # 33.8MB/s
lpm_pcie_rd_bw_1Quoted 4 lines
CPU88 0 nvidia_pcie_pmu_1_rc_1/wr_bytes/ CPU88 0 nvidia_pcie_pmu_1_rc_1/rd_bytes/ CPU0 1,002,286,615 tool/duration_time/ CPU0 0 nvidia_pcie_pmu_0_rc_1/wr_bytes,dst_rem=0x1/ #0.0 MB/s
lpm_pcie_rem_wr_bw_1Quoted 3 lines
CPU88 0 nvidia_pcie_pmu_1_rc_1/wr_bytes,dst_rem=0x1/ CPU0 1,002,275,059 tool/duration_time/ CPU0 0 nvidia_pcie_pmu_0_rc_1/rd_bytes,dst_rem=0x1/ #0.0 MB/s
lpm_pcie_rem_rd_bw_1Quoted 12 lines
CPU88 0 nvidia_pcie_pmu_1_rc_1/rd_bytes,dst_rem=0x1/
<snip>Note CPU88 (socket 1_ has no lpm_pcie_* values.
2. In the default mode, lpm_pcie_*_<i> adds RC i of socket 0 and RC i of socket 1. These are separate root complexes with different devices, so the "Root Complex i" value can combine traffic from two unrelated links.
Could the metrics follow the hardware hierarchy instead, starting at the socket and drilling down to the RC, with the same destination split at each level?Quoted 7 lines
- per socket, all RCs (nvidia_pcie_pmu_<s>_rc): lpm_pcie_s<s>_{bw,rd_bw,wr_bw} lpm_pcie_s<s>_{loc,rem}_{rd,wr}_bw - per socket and RC (nvidia_pcie_pmu_<s>_rc_<i>): lpm_pcie_s<s>_{bw,rd_bw,wr_bw}_<i> lpm_pcie_s<s>_{loc,rem}_{rd,wr}_bw_<i> grouped as lpm_pcie_s<s>_rc<i>_bw.Quoted 9 lines
Thanks for testing and for the suggestion. I think both problems come from how perf expects a PMU to expose its cpumask. Ideally perf wants a single PMU for both sockets, with one CPU per socket in its cpumask, e.g.:
``` /sys/bus/event_source/devices/nvidia_pcie_rc_0/cpumask: 0,88 ```I see this as a driver design choice: expose each physical PMU separately, or group several physical instances under one logical PMU. Either way, the PMU name and cpumask tell userspace which CPU(s) and root complex are associated with the PMU. With the first approach, each perf PMU has a cpumask of a single CPU. Tegra410 follows the first approach and exposes each physical PMU as a separate perf PMU.
Quoted 3 lines
We can work around the one-PMU-per-socket layout in the metrics, but would a single PMU per RC be a viable plan for you to implement?
Grouping instances across sockets is an interesting alternative, but it would require changes to the
arm_cspmudriver design. We would also need to evaluate how such grouping could be supported generically inarm_cspmu, and whether the ACPI APMT would need changes.For this patch, I think userspace gives us more flexibility in how we present and aggregate counts across sockets and root complexes, so I'd prefer to address it there and keep the current driver model.
FYI, I've posted the Tegra410 uncore JSON series:
https://lore.kernel.org/all/[email protected]/
It adds events and metrics for the same counters (e.g. ucf_avg_*,cmem_avg_rd_latency, pcie_avg_*), so it would be good to align the two and avoid duplicate metrics.Thanks,
BesarQuoted 163 lines
Thanks,
CTQuoted 9 lines
Thanks,
BesarQuoted 5 lines
-----Original Message-----
From: Chun-Tse Shao <[email protected]>
Sent: Monday, September 28, 2026 5:51 PM
To: [email protected]; [email protected]; [email protected]
Cc: [email protected]; [email protected];Quoted 16 lines
Quoted 16 lines
[email protected]; [email protected]; [email protected]; [email protected]; Besar Wicaksono <[email protected]>; [email protected]; linux- [email protected]; Chun-Tse Shao <[email protected]>
Subject: [PATCH 2/2] perf jevents: Add NVIDIA Tegra410 uncore PCIe metricsExternal email: Use caution opening links or attachments
Add PCIe bandwidth metrics for the NVIDIA Tegra410 SoC. The formulas follow
Documentation/admin-guide/perf/nvidia-tegra410-pmu.rst:
AVG_RD_BANDWIDTH_IN_GBPS=RD_BYTES/ELAPSED_TIME_IN_NSAVG_WR_BANDWIDTH_IN_GBPS=WR_BYTES/ELAPSED_TIME_IN_NSQuoted 6 lines
Quoted 6 lines
There is one PCIE PMU per socket and PCIe Root Complex (RC), named nvidia_pcie_pmu_<socket-id>_rc_<pcie-rc-id>, with RCs 0 to 5 in each socket. The metrics sum the rd_bytes/wr_bytes events of both sockets:
lpm_pcie_bw,lpm_pcie_rd_bw,lpm_pcie_wr_bw: all RCs. lpm_pcie_bw_<rc>, lpm_pcie_rd_bw_<rc>, lpm_pcie_wr_bw_<rc>: oneRC.
Quoted 3 lines
Quoted 3 lines
lpm_pcie_loc_rd_bw_<rc>, lpm_pcie_loc_wr_bw_<rc>: one RC to local
CMEM, GMEM, PCIe peer and CXL memory.
lpm_pcie_rem_rd_bw_<rc>, lpm_pcie_rem_wr_bw_<rc>: one RC toremote
Quoted 37 lines
Quoted 37 lines
memory.
The per RC metrics are in the lpm_pcie_rc<rc>_bw metric groups.
A socket's event counts as 0 if its PMU isn't present (
has_event()is 0), e.g. on a single socket system, or if it isn't counted in the aggregation (source_count()is 0), e.g. the other socket's event with --per-socket. Otherwise the metric fails to parse or is nan.Signed-off-by Chun-Tse Shao <[email protected]>Assisted-by Gemini:gemini-3.1-pro-preview--- tools/perf/pmu-events/arm64_metrics.py | 68 ++++++++++++++++++++++++-- 1 file changed, 64 insertions(+), 4 deletions(-)tools/perf/pmu- renamed from tools/perf/pmu-events/arm64_metrics.py +0
-0
events/arm64_metrics.py
indexb0d0651fec12..1ca3d8ddb4b7100755tools/perf/pmu-events/arm64_metrics.py +4
-3@@ -2,9 +2,10 @@ # SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) import argparse import os -from typing import Optional -from metric import (aggr_nr, d_ratio, Event, JsonEncodeMetric, - JsonEncodeMetricGroupDescriptions, LoadEvents, Metric, MetricGroup) +from typing import List, Optional, Union +from metric import (aggr_nr, d_ratio, has_event, source_count, Event, Expression, + JsonEncodeMetric, JsonEncodeMetricGroupDescriptions, LoadEvents, Metric, + MetricGroup, Select)from
common_metricsimport CyclesQuoted 2 lines
tools/perf/pmu-events/arm64_metrics.py def NvidiaT410Uncore() ->
Quoted 2 lines
# Global command line arguments.
tools/perf/pmu-events/arm64_metrics.py +0
-0@@ -67,7 +68,66 @@ def NvidiaT410Uncore() ->Optional[MetricGroup]:
Quoted 9 lines
tools/perf/pmu-events/arm64_metrics.py def NvidiaT410Uncore() -> Optional[MetricGroup]: · line 132
Quoted 9 lines
tools/perf/pmu-events/arm64_metrics.py +4
-1rd_cum_outs - rd_cum_outs, "1ns"), ], description="NVIDIA Tegra410 DDR latency") - return MetricGroup("lpm_uncore", [DdrBw(), DdrLat()], + def PcieBw() -> MetricGroup: + # PCIE PMU, one per socket and PCIe Root Complex (RC): + # AVG_{RD,WR}_BANDWIDTH = {RD,WR}_BYTES / ELAPSED_TIME + loc_dst ="dst_loc_cmem=0x1,dst_loc_gmem=0x1,dst_loc_pcie_p2p=0x1,dst_loc_pci
Quoted 6 lines
Quoted 6 lines
e_cxl=0x1"
tools/perf/pmu-events/arm64_metrics.py +5
-0+ rem_dst = "dst_rem=0x1" + + def SocketSum(rc: str, event: str) -> Expression: + # Sum the event over both sockets. A PMU counts as 0 if it isn't + # present (has_event() is 0) or isn't counted in the aggregation,Quoted 8 lines
tools/perf/pmu-events/arm64_metrics.py def NvidiaT410Uncore() -> Optional[MetricGroup]:
Quoted 8 lines
tools/perf/pmu-events/arm64_metrics.py +8
-0+ # e.g. the other socket with --per-socket (source_count() is 0). + def Count(e: Event) -> Expression: + return Select(Select(e, source_count(e), 0), has_event(e), 0) + + return (Count(Event(f"nvidia_pcie_pmu_0_{rc}/{event}/")) + + Count(Event(f"nvidia_pcie_pmu_1_{rc}/{event}/"))) + + # nvidia_pcie_pmu_<socket-id>_rc matches the PMUs of all RCs inthe
Quoted 22 lines
Quoted 22 lines
socket.
tools/perf/pmu-events/arm64_metrics.py +4
-0+ rd = SocketSum("rc", "rd_bytes") + wr = SocketSum("rc", "wr_bytes") + metrics: List[Union[Metric, MetricGroup]] = [ + Metric("lpm_pcie_bw", "Total PCIe bandwidth across all RootComplexes",
tools/perf/pmu-events/arm64_metrics.py +2
-0+ d_ratio(rd + wr, interval_sec), mb_per_sec), + Metric("lpm_pcie_rd_bw", "Total PCIe read bandwidth across allRoot Complexes",
tools/perf/pmu-events/arm64_metrics.py +2
-0+ d_ratio(rd, interval_sec), mb_per_sec), + Metric("lpm_pcie_wr_bw", "Total PCIe write bandwidth across allRoot Complexes",
tools/perf/pmu-events/arm64_metrics.py +10
-0+ d_ratio(wr, interval_sec), mb_per_sec), + ] + for i in range(6): + rc = f"rc_{i}" + rc_rd = SocketSum(rc, "rd_bytes") + rc_wr = SocketSum(rc, "wr_bytes") + loc_rd = SocketSum(rc, f"rd_bytes,{loc_dst}") + loc_wr = SocketSum(rc, f"wr_bytes,{loc_dst}") + rem_rd = SocketSum(rc, f"rd_bytes,{rem_dst}") + rem_wr = SocketSum(rc, f"wr_bytes,{rem_dst}")Quoted 8 lines
tools/perf/pmu-events/arm64_metrics.py
Quoted 8 lines
tools/perf/pmu-events/arm64_metrics.py +2
-0+ metrics.append(MetricGroup(f"lpm_pcie_rc{i}_bw", [ + Metric(f"lpm_pcie_bw_{i}", f"PCIe total bandwidth on RootComplex {i}",
tools/perf/pmu-events/arm64_metrics.py +2
-0+ d_ratio(rc_rd + rc_wr, interval_sec), mb_per_sec), + Metric(f"lpm_pcie_rd_bw_{i}", f"PCIe read bandwidth on RootComplex {i}",
tools/perf/pmu-events/arm64_metrics.py +2
-0+ d_ratio(rc_rd, interval_sec), mb_per_sec), + Metric(f"lpm_pcie_wr_bw_{i}", f"PCIe write bandwidth onRoot
Quoted 27 lines
Quoted 26 lines
Complex {i}",
tools/perf/pmu-events/arm64_metrics.py +17
-0+ d_ratio(rc_wr, interval_sec), mb_per_sec), + Metric(f"lpm_pcie_loc_rd_bw_{i}", + f"PCIe local read bandwidth on Root Complex {i}", + d_ratio(loc_rd, interval_sec), mb_per_sec), + Metric(f"lpm_pcie_loc_wr_bw_{i}", + f"PCIe local write bandwidth on Root Complex {i}", + d_ratio(loc_wr, interval_sec), mb_per_sec), + Metric(f"lpm_pcie_rem_rd_bw_{i}", + f"PCIe remote read bandwidth on Root Complex {i}", + d_ratio(rem_rd, interval_sec), mb_per_sec), + Metric(f"lpm_pcie_rem_wr_bw_{i}", + f"PCIe remote write bandwidth on Root Complex {i}", + d_ratio(rem_wr, interval_sec), mb_per_sec), + ], description=f"PCIe bandwidth on Root Complex {i}")) + + return MetricGroup("lpm_pcie_bw", metrics, + description="NVIDIA Tegra410 PCIe Root Complexbandwidth")
tools/perf/pmu-events/arm64_metrics.py +2
-1+ + return MetricGroup("lpm_uncore", [DdrBw(), DdrLat(), PcieBw()], description="NVIDIA Tegra410 uncore metrics") --2.56.0.rc1.315.gc6ed9934b7-goog
Arnaldo Carvalho de Melo
maintainerQuoted 4 lines · Chun-Tse Shao
Add DDR bandwidth/latency and PCIe bandwidth metrics for the NVIDIA Tegra410 SoC to
arm64_metrics.py. The metrics use the sysfs events of the Tegra410 UCF, CMEM latency and PCIE PMUs, and follow the formulas inDocumentation/admin-guide/perf/nvidia-tegra410-pmu.rst.
Ian,
Can you take a look and please provide your Reviewed-by?
Thanks,
- Arnaldo
Quoted 44 lines
Patch 1 adds the DDR metrics and makes pmu-events/Build also run
arm64_metrics.pyfor the nvidia vendor models. Patch 2 adds the PCIe metrics, in total and per PCIe Root Complex (RC).Tested on a 2-socket Tegra410 system:
- memcpy with multiload (from multichase) on socket 0 CPUs and memory:
lpm_ddr_bw is 740-744 GB/s at steady state (perf stat -I 2000), multiload reports 708063 MiB/s (742 GB/s).- Read only (stream-sum) and write only (memset) loads on socket 1:
lpm_ddr_rd_bwis 608 GB/s with 2.4 GB/s of writes,lpm_ddr_wr_bwis 628 GB/s with 1.1 GB/s of reads.
- memcpy on socket 0 CPUs with memory on socket 1: socket 0'slpm_ddr_rem_rd_bwand socket 1'slpm_ddr_rd_bwcount nearly the same bytes (258.03 vs 258.73 GB).
-lpm_ddr_latidle is 150 ns in the default aggregation, and 132 ns and 194 ns for socket 0 and 1 with --per-socket. Under the memcpy load it is 555 ns by default and 563 ns for socket 0 with --per-socket, and stays at 551-558 ns per interval with -I 1000.
- 8 GiBO_DIRECTdd read from an NVMe drive behind RC 0 of socket 0:
lpm_pcie_wr_bw_0counts the 8 GiB at 8.7 GB/s (dd: 8.7 GB/s). With the buffer on node 0 it shows up inlpm_pcie_loc_wr_bw_0, with the buffer on node 1 inlpm_pcie_rem_wr_bw_0. With --per-socket, socket 0 shows the same and no metric of either socket is nan.
- With thenvidia_t410metrics forced on a machine without these PMUs (PERF_CPUID=0x000000004e0f0100 on an x86 JEVENTS_ARCH=all build), the PCIe metrics read 0 rather than failing to parse, also with --per-socket.
- "perf test 10" (PMU JSON event tests) passes on the Tegra410 system and with the x86 JEVENTS_ARCH=all build.Chun-Tse Shao (2):
perf jevents: Add NVIDIA Tegra410 uncore DDR metrics perf jevents: Add NVIDIA Tegra410 uncore PCIe metricstools/perf/pmu-events/Build | 2 +- tools/perf/pmu-events/arm64_metrics.py | 126 ++++++++++++++++++++++++- tools/perf/pmu-events/metric.py | 2 +- 3 files changed, 126 insertions(+), 4 deletions(-)base-commit:
0ae6fc78c5ce0dfd18d8712a50f0fd4602eff103--
2.56.0.rc1.315.gc6ed9934b7-goog
Hi Chun-Tse,
Thanks for the series.
FYI, I am working on the JSON patch series for Tegra410 uncore PMUs, which adds the events and metrics for the same counters in UCF, CMEM latency, PCIE, and others.
I will cc you when I post it.
Thanks,
Besar
Quoted 68 lines
-----Original Message-----
From: Chun-Tse Shao <[email protected]>
Sent: Monday, September 28, 2026 5:51 PM
To: [email protected]; [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Besar Wicaksono <[email protected]>; [email protected]; linux- [email protected]; Chun-Tse Shao <[email protected]>
Subject: [PATCH 0/2] perf jevents: Add NVIDIA Tegra410 uncore DDR and PCIe metricsExternal email: Use caution opening links or attachments
Add DDR bandwidth/latency and PCIe bandwidth metrics for the NVIDIA Tegra410 SoC to
arm64_metrics.py. The metrics use the sysfs events of the Tegra410 UCF, CMEM latency and PCIE PMUs, and follow the formulas inDocumentation/admin-guide/perf/nvidia-tegra410-pmu.rst.Patch 1 adds the DDR metrics and makes pmu-events/Build also run
arm64_metrics.pyfor the nvidia vendor models. Patch 2 adds the PCIe metrics, in total and per PCIe Root Complex (RC).Tested on a 2-socket Tegra410 system:
- memcpy with multiload (from multichase) on socket 0 CPUs and memory:
lpm_ddr_bw is 740-744 GB/s at steady state (perf stat -I 2000), multiload reports 708063 MiB/s (742 GB/s).- Read only (stream-sum) and write only (memset) loads on socket 1:
lpm_ddr_rd_bwis 608 GB/s with 2.4 GB/s of writes,lpm_ddr_wr_bwis 628 GB/s with 1.1 GB/s of reads.
- memcpy on socket 0 CPUs with memory on socket 1: socket 0'slpm_ddr_rem_rd_bwand socket 1'slpm_ddr_rd_bwcount nearly the
same
bytes (258.03 vs 258.73 GB).
-lpm_ddr_latidle is 150 ns in the default aggregation, and 132 ns and 194 ns for socket 0 and 1 with --per-socket. Under the memcpy load it is 555 ns by default and 563 ns for socket 0 with --per-socket, and stays at 551-558 ns per interval with -I 1000.
- 8 GiBO_DIRECTdd read from an NVMe drive behind RC 0 of socket 0:
lpm_pcie_wr_bw_0counts the 8 GiB at 8.7 GB/s (dd: 8.7 GB/s). With the buffer on node 0 it shows up inlpm_pcie_loc_wr_bw_0, with the buffer on node 1 inlpm_pcie_rem_wr_bw_0. With --per-socket, socket 0 shows the same and no metric of either socket is nan.
- With thenvidia_t410metrics forced on a machine without these PMUs (PERF_CPUID=0x000000004e0f0100 on an x86 JEVENTS_ARCH=all build),
the
PCIe metrics read 0 rather than failing to parse, also with --per-socket.
- "perf test 10" (PMU JSON event tests) passes on the Tegra410 system and with the x86 JEVENTS_ARCH=all build.Chun-Tse Shao (2):
perf jevents: Add NVIDIA Tegra410 uncore DDR metrics perf jevents: Add NVIDIA Tegra410 uncore PCIe metricstools/perf/pmu-events/Build | 2 +- tools/perf/pmu-events/arm64_metrics.py | 126 ++++++++++++++++++++++++- tools/perf/pmu-events/metric.py | 2 +- 3 files changed, 126 insertions(+), 4 deletions(-)base-commit:
0ae6fc78c5ce0dfd18d8712a50f0fd4602eff103--
2.56.0.rc1.315.gc6ed9934b7-goog