Arm CMN-600 PMU driver
Subsystems (4)
Robin Murphy
maintainercover Arm CMN-600 PMU driver
Hi all,
At long last, here's an initial cut of the CMN PMU driver that's been festering in on-and-off development for years. It should be functionally complete now, although there is still scope for improving the current implementation (e.g. watchpoint register allocation could be cleverer).
Of particular interest at this point is the user interface - is it sufficiently complete and useful? Is there any need for a third event targeting method in between "single node ID" and "all nodes"? Is it worth templating watchpoints by port and channel to mimic XP events? Do we want to expose watchpoint-based bandwidth events as synthetic per-node events? Not all of that would need to be implemented right now so as to further stall upstreaming, but I really want to make sure that the initial interface is solid and any further enhancements can cleanly extend it, rather than painting ourselves into a corner in terms of ABI support.
Robin.
Robin Murphy (2):
perf: Add Arm CMN-600 DT binding
perf: Add Arm CMN-600 PMU driver
Documentation/admin-guide/perf/arm-cmn.rst | 65 + Documentation/admin-guide/perf/index.rst | 1 + .../devicetree/bindings/perf/arm-cmn.yaml | 57 + drivers/perf/Kconfig | 7 + drivers/perf/Makefile | 1 + drivers/perf/arm-cmn.c | 1653 +++++++++++++++++ 6 files changed, 1784 insertions(+) create mode 100644 Documentation/admin-guide/perf/arm-cmn.rst create mode 100644 Documentation/devicetree/bindings/perf/arm-cmn.yaml create mode 100644 drivers/perf/arm-cmn.c
Signature · 7 lines
2.28.0.dirty _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Robin Murphy
maintainerPATCH 1/2 perf: Add Arm CMN-600 DT binding
Document the requirements for the CMN-600 DT binding. The internal topology is almost entirely discoverable by walking a tree of ID registers, but sadly both the starting point for that walk and the exact format of those registers are configuration-dependent and not discoverable from some sane fixed location. Oh well.
--- .../devicetree/bindings/perf/arm-cmn.yaml | 57 +++++++++++++++++++ 1 file changed, 57 insertions(+) create mode 100644 Documentation/devicetree/bindings/perf/arm-cmn.yaml
Documentation/devicetree/bindings/perf/arm-cmn.yaml new file +57 -0
@@ -0,0 +1,57 @@ +# SPDX-License-Identifier: GPL-2.0 +# Copyright 2020 Arm Ltd. +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/perf/arm-cmn.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: Arm CMN (Coherent Mesh Network) Performance Monitors + +maintainers: + - Robin Murphy <[email protected]> + +properties: + compatible: + const: arm,cmn-600 + + reg: + items: + - description: Physical address of the base (PERIPHBASE) and + size (up to 64MB) of the configuration address space. + + interrupts: + minItems: 1 + maxItems: 4 + items: + - description: Overflow interrupt for DTC0 + - description: Overflow interrupt for DTC1 + - description: Overflow interrupt for DTC2 + - description: Overflow interrupt for DTC3 + description: One interrupt for each DTC domain implemented must + be specified, in order. DTC0 is always present. + + arm,root-node : + $ref: /schemas/types.yaml#/definitions/uint32 + description: Offset from PERIPHBASE of the configuration + discovery node (see TRM definition of ROOTNODEBASE). + +required: + - compatible + - reg + - interrupts + - arm,root-node + +additionalProperties: false + +examples: + - | + #include <dt-bindings/interrupt-controller/arm-gic.h> + #include <dt-bindings/interrupt-controller/irq.h> + pmu@50000000 { + compatible = "arm,cmn-600"; + reg = <0x50000000 0x4000000>; + /* 4x2 mesh with one DTC, and CFG node at 0,1,1,0 */ + interrupts = <GIC_SPI 46 IRQ_TYPE_LEVEL_HIGH>; + arm,root-node = <0x104000>; + }; +...
Signature · 7 lines
2.28.0.dirty _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Rob Herring
maintainerRe: [PATCH 1/2] perf: Add Arm CMN-600 DT binding
Quoted 10 lines · Robin Murphy
Document the requirements for the CMN-600 DT binding. The internal topology is almost entirely discoverable by walking a tree of ID registers, but sadly both the starting point for that walk and the exact format of those registers are configuration-dependent and not discoverable from some sane fixed location. Oh well.
Signed-off-by Robin Murphy <[email protected]>--- .../devicetree/bindings/perf/arm-cmn.yaml | 57 +++++++++++++++++++
arm,cmn.yaml
Documentation/devicetree/bindings/perf/arm-cmn.yaml line 1
Quoted 10 lines
1 file changed, 57 insertions(+) create mode 100644 Documentation/devicetree/bindings/perf/arm-cmn.yamlDocumentation/devicetree/bindings/perf/arm-cmn.yaml new file +1
-0@@ -0,0 +1,57 @@ +# SPDX-License-Identifier: GPL-2.0
Dual license new bindings: (GPL-2.0-only OR BSD-2-Clause)
Quoted 22 lines
Documentation/devicetree/bindings/perf/arm-cmn.yaml new file +22
-0+# Copyright 2020 Arm Ltd. +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/perf/arm-cmn.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: Arm CMN (Coherent Mesh Network) Performance Monitors + +maintainers: + - Robin Murphy <[email protected]> + +properties: + compatible: + const: arm,cmn-600 + + reg: + items: + - description: Physical address of the base (PERIPHBASE) and + size (up to 64MB) of the configuration address space. + + interrupts: + minItems: 1
Documentation/devicetree/bindings/perf/arm-cmn.yaml line 33
Quoted 10 lines
Documentation/devicetree/bindings/perf/arm-cmn.yaml new file +10
-0+ maxItems: 4 + items: + - description: Overflow interrupt for DTC0 + - description: Overflow interrupt for DTC1 + - description: Overflow interrupt for DTC2 + - description: Overflow interrupt for DTC3 + description: One interrupt for each DTC domain implemented must + be specified, in order. DTC0 is always present. + + arm,root-node :
extra space ^
Quoted 32 lines
Documentation/devicetree/bindings/perf/arm-cmn.yaml new file +24
-1+ $ref: /schemas/types.yaml#/definitions/uint32 + description: Offset from PERIPHBASE of the configuration + discovery node (see TRM definition of ROOTNODEBASE). + +required: + - compatible + - reg + - interrupts + - arm,root-node + +additionalProperties: false + +examples: + - | + #include <dt-bindings/interrupt-controller/arm-gic.h> + #include <dt-bindings/interrupt-controller/irq.h> + pmu@50000000 { + compatible = "arm,cmn-600"; + reg = <0x50000000 0x4000000>; + /* 4x2 mesh with one DTC, and CFG node at 0,1,1,0 */ + interrupts = <GIC_SPI 46 IRQ_TYPE_LEVEL_HIGH>; + arm,root-node = <0x104000>; + }; +... --2.28.0.dirty
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Robin Murphy
maintainerPATCH 2/2 perf: Add Arm CMN-600 PMU driver
In reply to Robin MurphyInitial driver for PMU event counting on the Arm CMN-600 interconnect. CMN sports an obnoxiously complex distributed PMU system as part of its debug and trace features, which can do all manner of things like sampling, cross-triggering and generating CoreSight trace. This driver covers the PMU functionality, plus the relevant aspects of watchpoints for simply counting matching flits.
--- Documentation/admin-guide/perf/arm-cmn.rst | 65 + Documentation/admin-guide/perf/index.rst | 1 + drivers/perf/Kconfig | 7 + drivers/perf/Makefile | 1 + drivers/perf/arm-cmn.c | 1653 ++++++++++++++++++++ 5 files changed, 1727 insertions(+) create mode 100644 Documentation/admin-guide/perf/arm-cmn.rst create mode 100644 drivers/perf/arm-cmn.c
Documentation/admin-guide/perf/arm-cmn.rst new file +65 -0
@@ -0,0 +1,65 @@ +============================= +Arm Coherent Mesh Network PMU +============================= + +CMN-600 is a configurable mesh interconnect consisting of a rectangular +grid of crosspoints (XPs), with each crosspoint supporting up to two +device ports to which various AMBA CHI agents are attached. + +CMN implements a distributed PMU design as part of its debug and trace +functionality. This consists of a local monitor (DTM) at every XP, which +counts up to 4 event signals from the connected device nodes and/or the +XP itself. Overflow from these local counters is accumulated in up to 8 +global counters implemented by the main controller (DTC), which provides +overall PMU control and interrupts for global counter overflow. + +PMU events +---------- + +The PMU driver registers a single PMU device for the whole interconnect, +see /sys/bus/event_source/devices/arm_cmn. Multi-chip systems may link +more than one CMN together via external CCIX links - in this situation, +each mesh counts its own events entirely independently, and additional +PMU devices will be named arm_cmn_{1..n}. + +Most events are specified in a format based directly on the TRM +definitions - "type" selects the respective node type, and "eventid" the +event number. Some events require an additional occupancy ID, which is +specified by "occupid". + +* Since RN-D nodes do not have any distinct events from RN-I nodes, they + are treated as the same type (0xa), and the common event templates are + named "rnid_*". + +* The cycle counter is treated as a synthetic event belonging to the DTC + node ("type" == 0x3, "eventid" is ignored). + +* XP events also encode the port and channel in the "eventid" field, to + match the underlying pmu_event0_id encoding for the pmu_event_sel + register. The event templates are named with prefixes to cover all + permutations. + +By default each event provides an aggregate count over all nodes of the +given type. To target a specific node, "bynodeid" must be set to 1 and +"nodeid" to the appropriate value derived from the CMN configuration +(as defined in the "Node ID Mapping" section of the TRM). + +Watchpoints +----------- + +The PMU can also count watchpoint events to monitor specific flit +traffic. Watchpoints are treated as a synthetic event type, and like PMU +events can be global or targeted with a particular XP's "nodeid" value. +Since the watchpoint direction is otherwise implicit in the underlying +register selection, separate events are provided for flit uploads and +downloads. + +The flit match value and mask are passed in config1 and config2 ("val" +and "mask" respectively). "wp_dev_sel", "wp_chn_sel", "wp_grp" and +"wp_exclusive" are specified per the TRM definitions for dtm_wp_config0. +Where a watchpoint needs to match fields from both match groups on the +REQ or SNP channel, it can be specified as two events - one for each +group - with the same nonzero "combine" value. The count for such a +pair of combined events will be attributed to the primary match. +Watchpoint events with a "combine" value of 0 are considered independent +and will count individually.
Documentation/admin-guide/perf/index.rst +1 -0
@@ -12,6 +12,7 @@ Performance monitor support qcom_l2_pmu qcom_l3_pmu arm-ccn + arm-cmn xgene-pmu arm_dsu_pmu thunderx2-pmu
drivers/perf/Kconfig +7 -0
@@ -41,6 +41,13 @@ config ARM_CCN PMU (perf) driver supporting the ARM CCN (Cache Coherent Network) interconnect. +config ARM_CMN + tristate "Arm CMN-600 PMU support" + depends on ARM64 || (COMPILE_TEST && 64BIT) + help + Support for PMU events monitoring on the Arm CMN-600 Coherent Mesh + Network interconnect. + config ARM_PMU depends on ARM || ARM64 bool "ARM PMU framework"
drivers/perf/Makefile +1 -0
@@ -1,6 +1,7 @@ # SPDX-License-Identifier: GPL-2.0 obj-$(CONFIG_ARM_CCI_PMU) += arm-cci.o obj-$(CONFIG_ARM_CCN) += arm-ccn.o +obj-$(CONFIG_ARM_CMN) += arm-cmn.o obj-$(CONFIG_ARM_DSU_PMU) += arm_dsu_pmu.o obj-$(CONFIG_ARM_PMU) += arm_pmu.o arm_pmu_platform.o obj-$(CONFIG_ARM_PMU_ACPI) += arm_pmu_acpi.o
drivers/perf/arm-cmn.c new file +1653 -0
1,653 lines not on this page · raw message
Signature · 7 lines
2.28.0.dirty _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Will Deacon
maintainerRe: [PATCH 2/2] perf: Add Arm CMN-600 PMU driver
Quoted 8 lines · Robin Murphy
Initial driver for PMU event counting on the Arm CMN-600 interconnect. CMN sports an obnoxiously complex distributed PMU system as part of its debug and trace features, which can do all manner of things like sampling, cross-triggering and generating CoreSight trace. This driver covers the PMU functionality, plus the relevant aspects of watchpoints for simply counting matching flits.
Signed-off-by Robin Murphy <[email protected]>
Quoted 9 lines
--- Documentation/admin-guide/perf/arm-cmn.rst | 65 + Documentation/admin-guide/perf/index.rst | 1 + drivers/perf/Kconfig | 7 + drivers/perf/Makefile | 1 + drivers/perf/arm-cmn.c | 1653 ++++++++++++++++++++ 5 files changed, 1727 insertions(+) create mode 100644 Documentation/admin-guide/perf/arm-cmn.rst create mode 100644 drivers/perf/arm-cmn.c
I'm assuming some of the folks on cc care about this, so it would be great to get some reviews, tested-bys and comments on the questions in the cover letter.
Cheers,
Will
Re: [PATCH 2/2] perf: Add Arm CMN-600 PMU driver
In reply to Robin MurphyQuoted 7 lines · Robin Murphy
Initial driver for PMU event counting on the Arm CMN-600 interconnect. CMN sports an obnoxiously complex distributed PMU system as part of its debug and trace features, which can do all manner of things like sampling, cross-triggering and generating CoreSight trace. This driver covers the PMU functionality, plus the relevant aspects of watchpoints for simply counting matching flits.
Hi Robin,
I had a quick look at the TRM (nearly 1000 pages, wow) for this IP, and could not see anything obvious to know the implementation id (similar problem to SMMU PMCG today). Do you know if there is anything appropiate? Is por_hnf_ppu_iidr any use here?
Just a couple of more questions for now...
[snip]
Quoted 111 lines
drivers/perf/arm-cmn.c new file +111
-0+ +static struct attribute *arm_cmn_event_attrs[] = { +CMN_EVENT_DTC (cycles), + +/* +* DVM node events conflict with HN-I events in the equivalent PMU +* slot, but our lazy short-cut of using the DTM counter index for +* the PMU index as well happens to avoid that by construction. +*/ +CMN_EVENT_DVM (rxreq_dvmop,0x01 ), +CMN_EVENT_DVM (rxreq_dvmsync,0x02 ), +CMN_EVENT_DVM (rxreq_dvmop_vmid_filtered,0x03 ), +CMN_EVENT_DVM (rxreq_retried,0x04 ), +_CMN_EVENT_DVM (rxreq_trk_occupancy_all,0x05 ,0 ), +_CMN_EVENT_DVM (rxreq_trk_occupancy_dvmop,0x05 ,1 ), +_CMN_EVENT_DVM (rxreq_trk_occupancy_dvmsync,0x05 ,2 ), + +CMN_EVENT_HNF (cache_miss,0x01 ), +CMN_EVENT_HNF (slc_sf_cache_access,0x02 ), +CMN_EVENT_HNF (cache_fill,0x03 ), +CMN_EVENT_HNF (pocq_retry,0x04 ), +CMN_EVENT_HNF (pocq_reqs_recvd,0x05 ), +CMN_EVENT_HNF (sf_hit,0x06 ), +CMN_EVENT_HNF (sf_evictions,0x07 ), +CMN_EVENT_HNF (dir_snoops_sent,0x08 ), +CMN_EVENT_HNF (brd_snoops_sent,0x09 ), +CMN_EVENT_HNF (slc_eviction,0x0a ), +CMN_EVENT_HNF (slc_fill_invalid_way,0x0b ), +CMN_EVENT_HNF (mc_retries,0x0c ), +CMN_EVENT_HNF (mc_reqs,0x0d ), +CMN_EVENT_HNF (qos_hh_retry,0x0e ), +_CMN_EVENT_HNF (qos_pocq_occupancy_all,0x0f ,0 ), +_CMN_EVENT_HNF (qos_pocq_occupancy_read,0x0f ,1 ), +_CMN_EVENT_HNF (qos_pocq_occupancy_write,0x0f ,2 ), +_CMN_EVENT_HNF (qos_pocq_occupancy_atomic,0x0f ,3 ), +_CMN_EVENT_HNF (qos_pocq_occupancy_stash,0x0f ,4 ), +CMN_EVENT_HNF (pocq_addrhaz,0x10 ), +CMN_EVENT_HNF (pocq_atomic_addrhaz,0x11 ), +CMN_EVENT_HNF (ld_st_swp_adq_full,0x12 ), +CMN_EVENT_HNF (cmp_adq_full,0x13 ), +CMN_EVENT_HNF (txdat_stall,0x14 ), +CMN_EVENT_HNF (txrsp_stall,0x15 ), +CMN_EVENT_HNF (seq_full,0x16 ), +CMN_EVENT_HNF (seq_hit,0x17 ), +CMN_EVENT_HNF (snp_sent,0x18 ), +CMN_EVENT_HNF (sfbi_dir_snp_sent,0x19 ), +CMN_EVENT_HNF (sfbi_brd_snp_sent,0x1a ), +CMN_EVENT_HNF (snp_sent_untrk,0x1b ), +CMN_EVENT_HNF (intv_dirty,0x1c ), +CMN_EVENT_HNF (stash_snp_sent,0x1d ), +CMN_EVENT_HNF (stash_data_pull,0x1e ), +CMN_EVENT_HNF (snp_fwded,0x1f ), + +CMN_EVENT_HNI (rrt_rd_occ_cnt_ovfl,0x20 ), +CMN_EVENT_HNI (rrt_wr_occ_cnt_ovfl,0x21 ), +CMN_EVENT_HNI (rdt_rd_occ_cnt_ovfl,0x22 ), +CMN_EVENT_HNI (rdt_wr_occ_cnt_ovfl,0x23 ), +CMN_EVENT_HNI (wdb_occ_cnt_ovfl,0x24 ), +CMN_EVENT_HNI (rrt_rd_alloc,0x25 ), +CMN_EVENT_HNI (rrt_wr_alloc,0x26 ), +CMN_EVENT_HNI (rdt_rd_alloc,0x27 ), +CMN_EVENT_HNI (rdt_wr_alloc,0x28 ), +CMN_EVENT_HNI (wdb_alloc,0x29 ), +CMN_EVENT_HNI (txrsp_retryack,0x2a ), +CMN_EVENT_HNI (arvalid_no_arready,0x2b ), +CMN_EVENT_HNI (arready_no_arvalid,0x2c ), +CMN_EVENT_HNI (awvalid_no_awready,0x2d ), +CMN_EVENT_HNI (awready_no_awvalid,0x2e ), +CMN_EVENT_HNI (wvalid_no_wready,0x2f ), +CMN_EVENT_HNI (txdat_stall,0x30 ), +CMN_EVENT_HNI (nonpcie_serialization,0x31 ), +CMN_EVENT_HNI (pcie_serialization,0x32 ), + +CMN_EVENT_XP (txflit_valid,0x01 ), +CMN_EVENT_XP (txflit_stall,0x02 ), +CMN_EVENT_XP (partial_dat_flit,0x03 ), +/* We treat watchpoints as a special made-up class of XP events */ +CMN_EVENT_ATTR (watchpoint_up,CMN_TYPE_WP ,0 ,0 ), +CMN_EVENT_ATTR (watchpoint_down,CMN_TYPE_WP ,2 ,0 ), + +CMN_EVENT_SBSX (rd_req,0x01 ), +CMN_EVENT_SBSX (wr_req,0x02 ), +CMN_EVENT_SBSX (cmo_req,0x03 ), +CMN_EVENT_SBSX (txrsp_retryack,0x04 ), +CMN_EVENT_SBSX (txdat_flitv,0x05 ), +CMN_EVENT_SBSX (txrsp_flitv,0x06 ), +CMN_EVENT_SBSX (rd_req_trkr_occ_cnt_ovfl,0x11 ), +CMN_EVENT_SBSX (wr_req_trkr_occ_cnt_ovfl,0x12 ), +CMN_EVENT_SBSX (cmo_req_trkr_occ_cnt_ovfl,0x13 ), +CMN_EVENT_SBSX (wdb_occ_cnt_ovfl,0x14 ), +CMN_EVENT_SBSX (rd_axi_trkr_occ_cnt_ovfl,0x15 ), +CMN_EVENT_SBSX (cmo_axi_trkr_occ_cnt_ovfl,0x16 ), +CMN_EVENT_SBSX (arvalid_no_arready,0x21 ), +CMN_EVENT_SBSX (awvalid_no_awready,0x22 ), +CMN_EVENT_SBSX (wvalid_no_wready,0x23 ), +CMN_EVENT_SBSX (txdat_stall,0x24 ), +CMN_EVENT_SBSX (txrsp_stall,0x25 ), + +CMN_EVENT_RNID (s0_rdata_beats,0x01 ), +CMN_EVENT_RNID (s1_rdata_beats,0x02 ), +CMN_EVENT_RNID (s2_rdata_beats,0x03 ), +CMN_EVENT_RNID (rxdat_flits,0x04 ), +CMN_EVENT_RNID (txdat_flits,0x05 ), +CMN_EVENT_RNID (txreq_flits_total,0x06 ), +CMN_EVENT_RNID (txreq_flits_retried,0x07 ), +CMN_EVENT_RNID (rrt_occ_ovfl,0x08 ), +CMN_EVENT_RNID (wrt_occ_ovfl,0x09 ), +CMN_EVENT_RNID (txreq_flits_replayed,0x0a ), +CMN_EVENT_RNID (wrcancel_sent,0x0b ), +CMN_EVENT_RNID (s0_wdata_beats,0x0c ), +CMN_EVENT_RNID (s1_wdata_beats,0x0d ),
drivers/perf/arm-cmn.c line 548
Quoted 10 lines
drivers/perf/arm-cmn.c new file +10
-0+CMN_EVENT_RNID (s2_wdata_beats,0x0e ), +CMN_EVENT_RNID (rrt_alloc,0x0f ), +CMN_EVENT_RNID (wrt_alloc,0x10 ), +CMN_EVENT_RNID (rdb_unord,0x11 ), +CMN_EVENT_RNID (rdb_replay,0x12 ), +CMN_EVENT_RNID (rdb_hybrid,0x13 ), +CMN_EVENT_RNID (rdb_ord,0x14 ), + + NULL +};
Just wondering how does "perf list" look when you have multiple instances of the device in the system?
[snip]
Quoted 46 lines
drivers/perf/arm-cmn.c new file +46
-0+static int arm_cmn_probe (struct platform_device *pdev) +{ +struct arm_cmn *cmn; +const char *name; +static atomic_t id; +int err, rootnode; + + cmn =devm_kzalloc (&pdev->dev,sizeof (*cmn),GFP_KERNEL ); +if (!cmn) +return -ENOMEM ; + + cmn->dev = &pdev->dev; +platform_set_drvdata (pdev, cmn); + +if (has_acpi_companion (cmn->dev)) + rootnode =arm_cmn_acpi_probe (pdev, cmn); +else + rootnode =arm_cmn_of_probe (pdev, cmn); +if (rootnode <0 ) +return rootnode; + + err =arm_cmn_discover (cmn, rootnode); +if (err) +return err; + + err =arm_cmn_init_dtcs (cmn); +if (err) +return err; + + err =arm_cmn_init_irqs (cmn); +if (err) +return err; + + cmn->cpu =raw_smp_processor_id (); + cmn->pmu = (struct pmu ) { + .module =THIS_MODULE , + .attr_groups = arm_cmn_attr_groups, + .capabilities =PERF_PMU_CAP_NO_EXCLUDE , + .task_ctx_nr = perf_invalid_context, + .pmu_enable = arm_cmn_pmu_enable, + .pmu_disable = arm_cmn_pmu_disable, + .event_init = arm_cmn_event_init, + .add = arm_cmn_event_add, + .del = arm_cmn_event_del, + .start = arm_cmn_event_start, + .stop = arm_cmn_event_stop,
drivers/perf/arm-cmn.c line 1568
Quoted 10 lines
drivers/perf/arm-cmn.c new file +10
-0+ .read = arm_cmn_event_read, + .start_txn = arm_cmn_start_txn, + .commit_txn = arm_cmn_commit_txn, + .cancel_txn = arm_cmn_end_txn, + }; + +if (atomic_fetch_inc (&id) ==0 ) { + name ="arm_cmn" ; + }else { + name =devm_kasprintf (cmn->dev,GFP_KERNEL ,"arm_cmn_%d" ,atomic_read (&id));
How is userspace supposed to know which device is which when we have multiple instances? I mean, doesn't this depend on arbitary probe ordering?
Cheers,
John
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Robin Murphy
maintainerRe: [PATCH 2/2] perf: Add Arm CMN-600 PMU driver
Hi John,
Quoted 9 lines · John Garry
Quoted 7 lines · Robin Murphy
Initial driver for PMU event counting on the Arm CMN-600 interconnect. CMN sports an obnoxiously complex distributed PMU system as part of its debug and trace features, which can do all manner of things like sampling, cross-triggering and generating CoreSight trace. This driver covers the PMU functionality, plus the relevant aspects of watchpoints for simply counting matching flits.
Quoted 3 lines
Hi Robin,
I had a quick look at the TRM (nearly 1000 pages, wow) for this IP, and could not see anything obvious to know the implementation id (similar problem to SMMU PMCG today). Do you know if there is anything appropiate? Is
por_hnf_ppu_iidrany use here?
Note that PPU is a distinct thing with its own architecture (apparently https://developer.arm.com/documentation/den0051/latest/ if you're interested), so that particular IIDR is merely describing the implementation of the PPU interface, which AFAICS is not necessarily unique to CMN-600. In terms of the PMU and CMN overall there is no architecture to speak of, the implementation just is what it is :)
I'd expect that some future interconnect IPs are likely to be similar enough to share most of this driver, but those should get their own top-level compatibles since they will not be CMN-600. Thus I don't believe that identification is going to be too much of a concern.
Quoted 119 lines
Just a couple of more questions for now...
[snip]
Quoted 115 lines
drivers/perf/arm-cmn.c new file +115
-0+ +static struct attribute *arm_cmn_event_attrs[] = { + CMN_EVENT_DTC(cycles), + + /* + * DVM node events conflict with HN-I events in the equivalent PMU + * slot, but our lazy short-cut of using the DTM counter index for + * the PMU index as well happens to avoid that by construction. + */ + CMN_EVENT_DVM(rxreq_dvmop, 0x01), + CMN_EVENT_DVM(rxreq_dvmsync, 0x02), + CMN_EVENT_DVM(rxreq_dvmop_vmid_filtered, 0x03), + CMN_EVENT_DVM(rxreq_retried, 0x04), + _CMN_EVENT_DVM(rxreq_trk_occupancy_all, 0x05, 0), + _CMN_EVENT_DVM(rxreq_trk_occupancy_dvmop, 0x05, 1), + _CMN_EVENT_DVM(rxreq_trk_occupancy_dvmsync, 0x05, 2), + + CMN_EVENT_HNF(cache_miss, 0x01), + CMN_EVENT_HNF(slc_sf_cache_access, 0x02), + CMN_EVENT_HNF(cache_fill, 0x03), + CMN_EVENT_HNF(pocq_retry, 0x04), + CMN_EVENT_HNF(pocq_reqs_recvd, 0x05), + CMN_EVENT_HNF(sf_hit, 0x06), + CMN_EVENT_HNF(sf_evictions, 0x07), + CMN_EVENT_HNF(dir_snoops_sent, 0x08), + CMN_EVENT_HNF(brd_snoops_sent, 0x09), + CMN_EVENT_HNF(slc_eviction, 0x0a), + CMN_EVENT_HNF(slc_fill_invalid_way, 0x0b), + CMN_EVENT_HNF(mc_retries, 0x0c), + CMN_EVENT_HNF(mc_reqs, 0x0d), + CMN_EVENT_HNF(qos_hh_retry, 0x0e), + _CMN_EVENT_HNF(qos_pocq_occupancy_all, 0x0f, 0), + _CMN_EVENT_HNF(qos_pocq_occupancy_read, 0x0f, 1), + _CMN_EVENT_HNF(qos_pocq_occupancy_write, 0x0f, 2), + _CMN_EVENT_HNF(qos_pocq_occupancy_atomic, 0x0f, 3), + _CMN_EVENT_HNF(qos_pocq_occupancy_stash, 0x0f, 4), + CMN_EVENT_HNF(pocq_addrhaz, 0x10), + CMN_EVENT_HNF(pocq_atomic_addrhaz, 0x11), + CMN_EVENT_HNF(ld_st_swp_adq_full, 0x12), + CMN_EVENT_HNF(cmp_adq_full, 0x13), + CMN_EVENT_HNF(txdat_stall, 0x14), + CMN_EVENT_HNF(txrsp_stall, 0x15), + CMN_EVENT_HNF(seq_full, 0x16), + CMN_EVENT_HNF(seq_hit, 0x17), + CMN_EVENT_HNF(snp_sent, 0x18), + CMN_EVENT_HNF(sfbi_dir_snp_sent, 0x19), + CMN_EVENT_HNF(sfbi_brd_snp_sent, 0x1a), + CMN_EVENT_HNF(snp_sent_untrk, 0x1b), + CMN_EVENT_HNF(intv_dirty, 0x1c), + CMN_EVENT_HNF(stash_snp_sent, 0x1d), + CMN_EVENT_HNF(stash_data_pull, 0x1e), + CMN_EVENT_HNF(snp_fwded, 0x1f), + + CMN_EVENT_HNI(rrt_rd_occ_cnt_ovfl, 0x20), + CMN_EVENT_HNI(rrt_wr_occ_cnt_ovfl, 0x21), + CMN_EVENT_HNI(rdt_rd_occ_cnt_ovfl, 0x22), + CMN_EVENT_HNI(rdt_wr_occ_cnt_ovfl, 0x23), + CMN_EVENT_HNI(wdb_occ_cnt_ovfl, 0x24), + CMN_EVENT_HNI(rrt_rd_alloc, 0x25), + CMN_EVENT_HNI(rrt_wr_alloc, 0x26), + CMN_EVENT_HNI(rdt_rd_alloc, 0x27), + CMN_EVENT_HNI(rdt_wr_alloc, 0x28), + CMN_EVENT_HNI(wdb_alloc, 0x29), + CMN_EVENT_HNI(txrsp_retryack, 0x2a), + CMN_EVENT_HNI(arvalid_no_arready, 0x2b), + CMN_EVENT_HNI(arready_no_arvalid, 0x2c), + CMN_EVENT_HNI(awvalid_no_awready, 0x2d), + CMN_EVENT_HNI(awready_no_awvalid, 0x2e), + CMN_EVENT_HNI(wvalid_no_wready, 0x2f), + CMN_EVENT_HNI(txdat_stall, 0x30), + CMN_EVENT_HNI(nonpcie_serialization, 0x31), + CMN_EVENT_HNI(pcie_serialization, 0x32), + + CMN_EVENT_XP(txflit_valid, 0x01), + CMN_EVENT_XP(txflit_stall, 0x02), + CMN_EVENT_XP(partial_dat_flit, 0x03), + /* We treat watchpoints as a special made-up class of XP events */ + CMN_EVENT_ATTR(watchpoint_up, CMN_TYPE_WP, 0, 0), + CMN_EVENT_ATTR(watchpoint_down, CMN_TYPE_WP, 2, 0), + + CMN_EVENT_SBSX(rd_req, 0x01), + CMN_EVENT_SBSX(wr_req, 0x02), + CMN_EVENT_SBSX(cmo_req, 0x03), + CMN_EVENT_SBSX(txrsp_retryack, 0x04), + CMN_EVENT_SBSX(txdat_flitv, 0x05), + CMN_EVENT_SBSX(txrsp_flitv, 0x06), + CMN_EVENT_SBSX(rd_req_trkr_occ_cnt_ovfl, 0x11), + CMN_EVENT_SBSX(wr_req_trkr_occ_cnt_ovfl, 0x12), + CMN_EVENT_SBSX(cmo_req_trkr_occ_cnt_ovfl, 0x13), + CMN_EVENT_SBSX(wdb_occ_cnt_ovfl, 0x14), + CMN_EVENT_SBSX(rd_axi_trkr_occ_cnt_ovfl, 0x15), + CMN_EVENT_SBSX(cmo_axi_trkr_occ_cnt_ovfl, 0x16), + CMN_EVENT_SBSX(arvalid_no_arready, 0x21), + CMN_EVENT_SBSX(awvalid_no_awready, 0x22), + CMN_EVENT_SBSX(wvalid_no_wready, 0x23), + CMN_EVENT_SBSX(txdat_stall, 0x24), + CMN_EVENT_SBSX(txrsp_stall, 0x25), + + CMN_EVENT_RNID(s0_rdata_beats, 0x01), + CMN_EVENT_RNID(s1_rdata_beats, 0x02), + CMN_EVENT_RNID(s2_rdata_beats, 0x03), + CMN_EVENT_RNID(rxdat_flits, 0x04), + CMN_EVENT_RNID(txdat_flits, 0x05), + CMN_EVENT_RNID(txreq_flits_total, 0x06), + CMN_EVENT_RNID(txreq_flits_retried, 0x07), + CMN_EVENT_RNID(rrt_occ_ovfl, 0x08), + CMN_EVENT_RNID(wrt_occ_ovfl, 0x09), + CMN_EVENT_RNID(txreq_flits_replayed, 0x0a), + CMN_EVENT_RNID(wrcancel_sent, 0x0b), + CMN_EVENT_RNID(s0_wdata_beats, 0x0c), + CMN_EVENT_RNID(s1_wdata_beats, 0x0d), + CMN_EVENT_RNID(s2_wdata_beats, 0x0e), + CMN_EVENT_RNID(rrt_alloc, 0x0f), + CMN_EVENT_RNID(wrt_alloc, 0x10), + CMN_EVENT_RNID(rdb_unord, 0x11),
Quoted 8 lines
drivers/perf/arm-cmn.c line 548
Quoted 6 lines
+
CMN_EVENT_RNID(rdb_replay, 0x12),
+CMN_EVENT_RNID(rdb_hybrid, 0x13),
+CMN_EVENT_RNID(rdb_ord, 0x14),drivers/perf/arm-cmn.c new file +3
-0+ + NULL +};Just wondering how does "perf list" look when you have multiple instances of the device in the system?
Probably a giant mess, since I assume every (relevant) event should be listed for each instance, much like CPU events on big.LITTLE systems, and uncore events on many others (certainly most of the xgene_pmu events on my eMAG workstation are repeated up to 8 times).
Quoted 53 lines
[snip]
Quoted 51 lines
drivers/perf/arm-cmn.c new file +51
-0+static int arm_cmn_probe (struct platform_device *pdev) +{ + struct arm_cmn *cmn; + const char *name; + static atomic_t id; + int err, rootnode; + + cmn = devm_kzalloc(&pdev->dev, sizeof(*cmn), GFP_KERNEL); + if (!cmn) + return -ENOMEM; + + cmn->dev = &pdev->dev; + platform_set_drvdata(pdev, cmn); + + if (has_acpi_companion(cmn->dev)) + rootnode = arm_cmn_acpi_probe(pdev, cmn); + else + rootnode = arm_cmn_of_probe(pdev, cmn); + if (rootnode < 0) + return rootnode; + + err = arm_cmn_discover(cmn, rootnode); + if (err) + return err; + + err = arm_cmn_init_dtcs(cmn); + if (err) + return err; + + err = arm_cmn_init_irqs(cmn); + if (err) + return err; + + cmn->cpu = raw_smp_processor_id(); + cmn->pmu = (struct pmu) { + .module = THIS_MODULE, + .attr_groups = arm_cmn_attr_groups, + .capabilities = PERF_PMU_CAP_NO_EXCLUDE, + .task_ctx_nr = perf_invalid_context, + .pmu_enable = arm_cmn_pmu_enable, + .pmu_disable = arm_cmn_pmu_disable, + .event_init = arm_cmn_event_init, + .add = arm_cmn_event_add, + .del = arm_cmn_event_del, + .start = arm_cmn_event_start, + .stop = arm_cmn_event_stop, + .read = arm_cmn_event_read, + .start_txn = arm_cmn_start_txn, + .commit_txn = arm_cmn_commit_txn, + .cancel_txn = arm_cmn_end_txn, + };
Quoted 8 lines
Quoted 6 lines
+
+ if (atomic_fetch_inc(&id)== 0) {
+ name = "arm_cmn";
+ } else {
+ name =devm_kasprintf(cmn->dev, GFP_KERNEL, "arm_cmn_%d", atomic_read(&id));How is userspace supposed to know which device is which when we have multiple instances? I mean, doesn't this depend on arbitary probe ordering?
Right, I had it in my head that there should be enough information in sysfs to link the PMU device back to its parent platform device, but apparently that isn't the case :(
Furthermore, you've now got me thinking I might want to rejig the naming scheme anyway - if I did have two instances named "foo" and "foo_1", is perf tool going to interpret "-e foo/event/" as specifying a common prefix and create the event on both PMUs even if I only wanted it on the first one?
Robin.
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Re: [PATCH 2/2] perf: Add Arm CMN-600 PMU driver
Quoted 15 lines
Quoted 2 lines
I had a quick look at the TRM (nearly 1000 pages, wow) for this IP, and could not see anything obvious to know the implementation id (similar problem to SMMU PMCG today). Do you know if there is anything appropiate? Is
por_hnf_ppu_iidrany use here?Note that PPU is a distinct thing with its own architecture (apparently https://developer.arm.com/documentation/den0051/latest/ if you're interested), so that particular IIDR is merely describing the implementation of the PPU interface, which AFAICS is not necessarily unique to CMN-600. In terms of the PMU and CMN overall there is no architecture to speak of, the implementation just is what it is :)
I'd expect that some future interconnect IPs are likely to be similar enough to share most of this driver, but those should get their own top-level compatibles since they will not be CMN-600. Thus I don't believe that identification is going to be too much of a concern.
Quoted 8 lines
Just a couple of more questions for now...
[snip]
Quoted 4 lines
drivers/perf/arm-cmn.c new file +4
-0+ +static struct attribute *arm_cmn_event_attrs[] = { + CMN_EVENT_DTC(cycles), +
[...]
Quoted 7 lines
Quoted 5 lines
drivers/perf/arm-cmn.c line 548
Quoted 3 lines
drivers/perf/arm-cmn.c new file +3
-0+ + NULL +};Just wondering how does "perf list" look when you have multiple instances of the device in the system?
Probably a giant mess, since I assume every (relevant) event should be listed for each instance, much like CPU events on big.LITTLE systems, and uncore events on many others (certainly most of the
xgene_pmuevents on my eMAG workstation are repeated up to 8 times).
Yeah, so that's one thing we would solve with my "system PMU" perf support, such that we can have a single per-event alias listed for instances of that PMU.
So out of curiousity, for the big.LITTLE systems which you have seen with this issue, which cores do they have?
Quoted 11 lines
Quoted 11 lines
[snip]
Quoted 7 lines
+ };
+
+ if (atomic_fetch_inc(&id)== 0) {
+ name = "arm_cmn";
+ } else {
+ name =devm_kasprintf(cmn->dev, GFP_KERNEL, "arm_cmn_%d", atomic_read(&id));
Quoted 5 lines
Quoted 1 line
How is userspace supposed to know which device is which when we have multiple instances? I mean, doesn't this depend on arbitary probe ordering?
Right, I had it in my head that there should be enough information in sysfs to link the PMU device back to its parent platform device, but apparently that isn't the case :(
Furthermore, you've now got me thinking I might want to rejig the naming scheme anyway - if I did have two instances named "foo" and "
foo_1", is perf tool going to interpret "-e foo/event/" as specifying a common prefix and create the event on both PMUs even if I only wanted it on the first one?
It should just create the event on the PMU "foo", and not also "foo_1". I think.
As for naming, as you know, for the SMMUv3 PMU driver we suffixed the HW base register address for the name - not ideal, but at least which PMU is which can be determined from userspace. Or for this driver case, maybe ACPI _UID could be used instead.
Thanks,
john
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel