Skip to contentSkip to conversation

drm/msm: Add PERFCNTR_CONFIG ioctl

PATCH Superseded Rob Clark 3 participants 21 messages May 11 – May 13

Subsystems (5)

v5 of 10 Superseded Latest is v10 →

lore
Share & tools
lore

Rob Clark

author maintainer
#0 lore raw

v5 cover drm/msm: Add PERFCNTR_CONFIG ioctl

Add a new PERFCNTR_CONFIG ioctl, serving two functions:

1. Global counter collection (restricted to perfmon_capable()) using the MSM_PERFCNTR_STREAM flag. Global counter sampling is, global, across contexts. Only a single global counter stream is allowed at a time.
2. Reserve counters for local counter collection. Local counter collection is local to a cmdstream (GEM_SUBMIT), and as such is allowed in all processes without additional privileges.

The kernel enforces that counters assigned for global counter collection do not conflict with counters reserved for local counter collection, and visa versa. Since local counter collection is scoped to a single cmd- stream, multiple UMD processes can overlap in their reserved counters. But cannot conflict with global counter usage.

In the case of local counter collection, the UMD is still responsible for programming the corresponding SELect registers, and sampling the counter values, from it's cmdstream. But by performing the reservation step, the UMD protects itself from the kernel trying to use the same SEL/counter regs for global counter collection.

For global counter collection, the kernel programs SEL regs, and sets up a timer for counter sampling. Userspace reads out the sampled values from the returned perfcntr stream fd. Releasing the global perfcntr stream is simply a matter of close()ing the fd.

The final two patches wire up the needed support for global counter stream collection while IFPC is active, and drops disabling of IFPC.

The mesa side of this is at:
https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/41158

igt test at:
https://gitlab.freedesktop.org/robclark/igt-gpu-tools/-/commits/perfcntrs

wiki page about the design:
https://gitlab.freedesktop.org/drm/msm/-/wikis/adreno:-perfcounter-UABI

Changes in v5:
- Drop unnecessary runpm in ioctl path
- Link to v4: https://lore.kernel.org/all/[email protected]

Changes in v4:
- Fix null ptr deref on older gens without perfcntr support [Claude]
- Add upper limit to userspace controlled FIFO size [Claude]
- Fix nr_regs calculation [Claude]
- Link to v3: https://lore.kernel.org/all/[email protected]/

Changes in v3:
- Fix loop counter issue spotted by Claude review
- Add MSM_PERFCNTR_UPDATE flag to ask kernel to return the actual # of available counters in case of -E2BIG
- Proper barriers for modifying pwrup_Link
- Link to v2: https://lore.kernel.org/all/[email protected]

Changes in v2:
- Rework makefile magic based on Dmitry's suggestion, and add a2xx/a5xx
perfcntr tables (although only a6xx+ is supported at this point)
- Fix compile error for compilers that are picky about a struct that
only contains a flex array
- Drop a6xx_idle() under gpu->lock in a6xx_perfcntr_configure(), replace
with perfcntr_fence that sel_worker can check
- Add a7xx+ pwrup_reglist support for restoring SELect regs on exit from
IFPC. (a6xx doesn't support IFPC, and the pwrup_reglist works a bit differently)
- Stop disabling IFPC when global counter stream is active.
- Link to v1: https://lore.kernel.org/all/[email protected]/

Rob Clark (16):
drm/msm: Remove obsolete perf infrastructure drm/msm: Allow CAP_PERFMON for setting SYSPROF drm/msm/adreno: Sync registers from mesa drm/msm/registers: Sync gen_header.py from mesa drm/msm/registers: Add perfcntr json drm/msm: Add a6xx+ perfcntr tables drm/msm: Add sysprof accessors drm/msm/a6xx: Add yield & flush helper drm/msm: Add per-context perfcntr state drm/msm: Add basic perfcntr infrastructure drm/msm/a6xx+: Add support to configure perfcntrs drm/msm/a8xx: Add perfcntr flush sequence drm/msm: Add PERFCNTR_CONFIG ioctl drm/msm/a6xx: Increase pwrup_reglist size drm/msm/a6xx: Append SEL regs to dyn pwrup reglist drm/msm/a6xx: Allow IFPC with perfcntr stream

 drivers/gpu/drm/msm/Makefile                  |   27 +-
 drivers/gpu/drm/msm/adreno/a2xx_gpu.c         |    7 -
 drivers/gpu/drm/msm/adreno/a3xx_gpu.c         |   16 -
 drivers/gpu/drm/msm/adreno/a4xx_gpu.c         |    3 -
 drivers/gpu/drm/msm/adreno/a5xx_gpu.c         |   16 +-
 drivers/gpu/drm/msm/adreno/a6xx_gmu.c         |   10 +-
 drivers/gpu/drm/msm/adreno/a6xx_gpu.c         |  217 +-
 drivers/gpu/drm/msm/adreno/a6xx_gpu.h         |   15 +-
 drivers/gpu/drm/msm/adreno/a6xx_preempt.c     |    2 +-
 drivers/gpu/drm/msm/adreno/a8xx_gpu.c         |   33 +-
 drivers/gpu/drm/msm/adreno/a8xx_preempt.c     |    2 +-
 drivers/gpu/drm/msm/adreno/adreno_device.c    |    8 +-
 drivers/gpu/drm/msm/adreno/adreno_gpu.c       |    7 +-
 drivers/gpu/drm/msm/msm_debugfs.c             |    6 -
 drivers/gpu/drm/msm/msm_drv.c                 |    2 +-
 drivers/gpu/drm/msm/msm_drv.h                 |   13 +-
 drivers/gpu/drm/msm/msm_gpu.c                 |  119 +-
 drivers/gpu/drm/msm/msm_gpu.h                 |  104 +-
 drivers/gpu/drm/msm/msm_perf.c                |  235 --
 drivers/gpu/drm/msm/msm_perfcntr.c            |  648 +++++
 drivers/gpu/drm/msm/msm_perfcntr.h            |  155 ++
 drivers/gpu/drm/msm/msm_ringbuffer.h          |    2 +
 drivers/gpu/drm/msm/msm_submitqueue.c         |    3 +-
 .../msm/registers/adreno/a2xx_perfcntrs.json  |  109 +
 drivers/gpu/drm/msm/registers/adreno/a3xx.xml |    8 +-
 drivers/gpu/drm/msm/registers/adreno/a5xx.xml |  141 +-
 .../msm/registers/adreno/a5xx_perfcntrs.json  |  128 +
 drivers/gpu/drm/msm/registers/adreno/a6xx.xml | 1300 ++++++-----
 .../msm/registers/adreno/a6xx_descriptors.xml |   71 +-
 .../drm/msm/registers/adreno/a6xx_enums.xml   |    3 +
 .../msm/registers/adreno/a6xx_perfcntrs.json  |  105 +
 .../msm/registers/adreno/a7xx_perfcntrs.json  |  228 ++
 .../msm/registers/adreno/a8xx_descriptors.xml |   96 +-
 .../msm/registers/adreno/a8xx_perfcntrs.json  |  240 ++
 .../msm/registers/adreno/a8xx_perfcntrs.xml   | 1929 +++++++++++++++
 .../msm/registers/adreno/adreno_common.xml    |   42 +
 .../drm/msm/registers/adreno/adreno_pm4.xml   |   50 +-
 drivers/gpu/drm/msm/registers/gen_header.py   | 2079 +++++++++--------
 include/uapi/drm/msm_drm.h                    |   48 +
 39 files changed, 6015 insertions(+), 2212 deletions(-)
 delete mode 100644 drivers/gpu/drm/msm/msm_perf.c
 create mode 100644 drivers/gpu/drm/msm/msm_perfcntr.c
 create mode 100644 drivers/gpu/drm/msm/msm_perfcntr.h
 create mode 100644 drivers/gpu/drm/msm/registers/adreno/a2xx_perfcntrs.json
 create mode 100644 drivers/gpu/drm/msm/registers/adreno/a5xx_perfcntrs.json
 create mode 100644 drivers/gpu/drm/msm/registers/adreno/a6xx_perfcntrs.json
 create mode 100644 drivers/gpu/drm/msm/registers/adreno/a7xx_perfcntrs.json
 create mode 100644 drivers/gpu/drm/msm/registers/adreno/a8xx_perfcntrs.json
 create mode 100644 drivers/gpu/drm/msm/registers/adreno/a8xx_perfcntrs.xml
Signature · 1 line
2.54.0
author maintainer
#1 lore raw

PATCH v5 1/16 drm/msm: Remove obsolete perf infrastructure

Outside of a3xx, this was never really used. And it low-key gets in the way of the new perfcntr support (or at least it is confusing to have two things called "perf"). So lets remove it.

This drops the "perf" debugfs file. But these days, nvtop is a better option. (Plus perfetto for newer gens.)

---
 drivers/gpu/drm/msm/Makefile          |   1 -
 drivers/gpu/drm/msm/adreno/a2xx_gpu.c |   7 -
 drivers/gpu/drm/msm/adreno/a3xx_gpu.c |  16 --
 drivers/gpu/drm/msm/adreno/a4xx_gpu.c |   3 -
 drivers/gpu/drm/msm/msm_debugfs.c     |   6 -
 drivers/gpu/drm/msm/msm_drv.c         |   1 -
 drivers/gpu/drm/msm/msm_drv.h         |   5 -
 drivers/gpu/drm/msm/msm_gpu.c         | 107 ------------
 drivers/gpu/drm/msm/msm_gpu.h         |  31 ----
 drivers/gpu/drm/msm/msm_perf.c        | 235 --------------------------
 10 files changed, 412 deletions(-)
 delete mode 100644 drivers/gpu/drm/msm/msm_perf.c
drivers/gpu/drm/msm/Makefile +0 -1
index ba45e99be05b..ce00cfb0a875 100644
@@ -122,7 +122,6 @@ msm-y += \
 	msm_gpu_devfreq.o \
 	msm_io_utils.o \
 	msm_iommu.o \
-	msm_perf.o \
 	msm_rd.o \
 	msm_ringbuffer.o \
 	msm_submitqueue.o \
drivers/gpu/drm/msm/adreno/a2xx_gpu.c +0 -7
index d5a5fa9e2cf8..df4cded9143f 100644
@@ -489,10 +489,6 @@ static u32 a2xx_get_rptr(struct msm_gpu *gpu, struct msm_ringbuffer *ring)
 	return ring->memptrs->rptr;
 }
 
-static const struct msm_gpu_perfcntr perfcntrs[] = {
-/* TODO */
-};
-
 static struct msm_gpu *a2xx_gpu_init(struct drm_device *dev)
 {
 	struct a2xx_gpu *a2xx_gpu = NULL;
@@ -518,9 +514,6 @@ static struct msm_gpu *a2xx_gpu_init(struct drm_device *dev)
 	adreno_gpu = &a2xx_gpu->base;
 	gpu = &adreno_gpu->base;
 
-	gpu->perfcntrs = perfcntrs;
-	gpu->num_perfcntrs = ARRAY_SIZE(perfcntrs);
-
 	ret = adreno_gpu_init(dev, pdev, adreno_gpu, config->info->funcs, 1);
 	if (ret)
 		goto fail;
drivers/gpu/drm/msm/adreno/a3xx_gpu.c +0 -16
index 018183e0ac3f..c17e9777beae 100644
@@ -266,12 +266,6 @@ static int a3xx_hw_init(struct msm_gpu *gpu)
 	/* Turn on performance counters: */
 	gpu_write(gpu, REG_A3XX_RBBM_PERFCTR_CTL, 0x01);
 
-	/* Enable the perfcntrs that we use.. */
-	for (i = 0; i < gpu->num_perfcntrs; i++) {
-		const struct msm_gpu_perfcntr *perfcntr = &gpu->perfcntrs[i];
-		gpu_write(gpu, perfcntr->select_reg, perfcntr->select_val);
-	}
-
 	gpu_write(gpu, REG_A3XX_RBBM_INT_0_MASK, A3XX_INT0_MASK);
 
 	ret = adreno_hw_init(gpu);
@@ -508,13 +502,6 @@ static u32 a3xx_get_rptr(struct msm_gpu *gpu, struct msm_ringbuffer *ring)
 	return ring->memptrs->rptr;
 }
 
-static const struct msm_gpu_perfcntr perfcntrs[] = {
-	{ REG_A3XX_SP_PERFCOUNTER6_SELECT, REG_A3XX_RBBM_PERFCTR_SP_6_LO,
-			SP_ALU_ACTIVE_CYCLES, "ALUACTIVE" },
-	{ REG_A3XX_SP_PERFCOUNTER7_SELECT, REG_A3XX_RBBM_PERFCTR_SP_7_LO,
-			SP_FS_FULL_ALU_INSTRUCTIONS, "ALUFULL" },
-};
-
 static struct msm_gpu *a3xx_gpu_init(struct drm_device *dev)
 {
 	struct a3xx_gpu *a3xx_gpu = NULL;
@@ -542,9 +529,6 @@ static struct msm_gpu *a3xx_gpu_init(struct drm_device *dev)
 	adreno_gpu = &a3xx_gpu->base;
 	gpu = &adreno_gpu->base;
 
-	gpu->perfcntrs = perfcntrs;
-	gpu->num_perfcntrs = ARRAY_SIZE(perfcntrs);
-
 	adreno_gpu->registers = a3xx_registers;
 
 	ret = adreno_gpu_init(dev, pdev, adreno_gpu, config->info->funcs, 1);
drivers/gpu/drm/msm/adreno/a4xx_gpu.c +0 -3
index e6ab731f8e9a..6392126f48f2 100644
@@ -652,9 +652,6 @@ static struct msm_gpu *a4xx_gpu_init(struct drm_device *dev)
 	adreno_gpu = &a4xx_gpu->base;
 	gpu = &adreno_gpu->base;
 
-	gpu->perfcntrs = NULL;
-	gpu->num_perfcntrs = 0;
-
 	ret = adreno_gpu_init(dev, pdev, adreno_gpu, config->info->funcs, 1);
 	if (ret)
 		goto fail;
drivers/gpu/drm/msm/msm_debugfs.c +0 -6
index 1059a9b29d6a..f12701e286ec 100644
@@ -344,12 +344,6 @@ static int late_init_minor(struct drm_minor *minor)
 		return ret;
 	}
 
-	ret = msm_perf_debugfs_init(minor);
-	if (ret) {
-		DRM_DEV_ERROR(dev->dev, "could not install perf debugfs\n");
-		return ret;
-	}
-
 	return 0;
 }
 
drivers/gpu/drm/msm/msm_drv.c +0 -1
index edc3b4af14f4..3066547f319b 100644
@@ -87,7 +87,6 @@ static int msm_drm_uninit(struct device *dev, const struct component_ops *gpu_op
 
 	msm_gem_shrinker_cleanup(ddev);
 
-	msm_perf_debugfs_cleanup(priv);
 	msm_rd_debugfs_cleanup(priv);
 
 	if (priv->kms)
drivers/gpu/drm/msm/msm_drv.h +0 -5
index 6d847d593f1a..e53e4f220bed 100644
@@ -45,7 +45,6 @@ struct msm_gpu;
 struct msm_mmu;
 struct msm_mdss;
 struct msm_rd_state;
-struct msm_perf_state;
 struct msm_gem_submit;
 struct msm_fence_context;
 struct msm_disp_state;
@@ -89,7 +88,6 @@ struct msm_drm_private {
 
 	struct msm_rd_state *rd;       /* debugfs to dump all submits */
 	struct msm_rd_state *hangrd;   /* debugfs to dump hanging submits */
-	struct msm_perf_state *perf;
 
 	/**
 	 * total_mem: Total/global amount of memory backing GEM objects.
@@ -442,8 +440,6 @@ void msm_rd_debugfs_cleanup(struct msm_drm_private *priv);
 __printf(3, 4)
 void msm_rd_dump_submit(struct msm_rd_state *rd, struct msm_gem_submit *submit,
 		const char *fmt, ...);
-int msm_perf_debugfs_init(struct drm_minor *minor);
-void msm_perf_debugfs_cleanup(struct msm_drm_private *priv);
 #else
 static inline int msm_debugfs_late_init(struct drm_device *dev) { return 0; }
 __printf(3, 4)
@@ -451,7 +447,6 @@ static inline void msm_rd_dump_submit(struct msm_rd_state *rd,
 			struct msm_gem_submit *submit,
 			const char *fmt, ...) {}
 static inline void msm_rd_debugfs_cleanup(struct msm_drm_private *priv) {}
-static inline void msm_perf_debugfs_cleanup(struct msm_drm_private *priv) {}
 #endif
 
 struct clk *msm_clk_get(struct platform_device *pdev, const char *name);
drivers/gpu/drm/msm/msm_gpu.c +0 -107

137 lines not on this page · raw message

drivers/gpu/drm/msm/msm_gpu.h +0 -31
index 07abbe33d992..78e1478669be 100644
@@ -22,7 +22,6 @@
 
 struct msm_gem_submit;
 struct msm_gem_vm_log_entry;
-struct msm_gpu_perfcntr;
 struct msm_gpu_state;
 struct msm_context;
 
@@ -168,18 +167,6 @@ struct msm_gpu {
 
 	struct adreno_smmu_priv adreno_smmu;
 
-	/* performance counters (hw & sw): */
-	spinlock_t perf_lock;
-	bool perfcntr_active;
-	struct {
-		bool active;
-		ktime_t time;
-	} last_sample;
-	uint32_t totaltime, activetime;    /* sw counters */
-	uint32_t last_cntrs[5];            /* hw counters */
-	const struct msm_gpu_perfcntr *perfcntrs;
-	uint32_t num_perfcntrs;
-
 	struct msm_ringbuffer *rb[MSM_GPU_MAX_RINGS];
 	int nr_rings;
 
@@ -320,19 +307,6 @@ static inline bool msm_gpu_active(struct msm_gpu *gpu)
 	return false;
 }
 
-/* Perf-Counters:
- * The select_reg and select_val are just there for the benefit of the child
- * class that actually enables the perf counter..  but msm_gpu base class
- * will handle sampling/displaying the counters.
- */
-
-struct msm_gpu_perfcntr {
-	uint32_t select_reg;
-	uint32_t sample_reg;
-	uint32_t select_val;
-	const char *name;
-};
-
 /*
  * The number of priority levels provided by drm gpu scheduler.  The
  * DRM_SCHED_PRIORITY_KERNEL priority level is treated specially in some
@@ -689,11 +663,6 @@ void msm_devfreq_idle(struct msm_gpu *gpu);
 
 int msm_gpu_hw_init(struct msm_gpu *gpu);
 
-void msm_gpu_perfcntr_start(struct msm_gpu *gpu);
-void msm_gpu_perfcntr_stop(struct msm_gpu *gpu);
-int msm_gpu_perfcntr_sample(struct msm_gpu *gpu, uint32_t *activetime,
-		uint32_t *totaltime, uint32_t ncntrs, uint32_t *cntrs);
-
 void msm_gpu_retire(struct msm_gpu *gpu);
 void msm_gpu_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit);
 void msm_gpu_sysrq_kill(struct msm_gpu *gpu);
drivers/gpu/drm/msm/msm_perf.c deleted +0 -235

235 lines not on this page · raw message

Signature · 1 line
2.54.0
author maintainer
#2 lore raw

PATCH v5 2/16 drm/msm: Allow CAP_PERFMON for setting SYSPROF

In reply to Rob Clark

Use perfmon_capable() which checks both CAP_SYS_ADMIN and CAP_PERFMON. This matches what i915 and xe do, and seems more appropriate.

---
 drivers/gpu/drm/msm/adreno/adreno_gpu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/msm/adreno/adreno_gpu.c +1 -1
index 66f80f2d12f9..72b71e9e44f0 100644
@@ -494,7 +494,7 @@ int adreno_set_param(struct msm_gpu *gpu, struct msm_context *ctx,
 		return 0;
 	}
 	case MSM_PARAM_SYSPROF:
-		if (!capable(CAP_SYS_ADMIN))
+		if (!perfmon_capable())
 			return UERR(EPERM, drm, "invalid permissions");
 		return msm_context_set_sysprof(ctx, gpu, value);
 	case MSM_PARAM_EN_VM_BIND:
Signature · 1 line
2.54.0
author maintainer
#3 lore raw

PATCH v5 3/16 drm/msm/adreno: Sync registers from mesa

In reply to Rob Clark

Most of the churn is just reworking the usage attribute on the mesa side.

Sync from mesa commit 4d4a951ac622 ("fd: add a8xx perfcntr countables").

---
 drivers/gpu/drm/msm/adreno/a5xx_gpu.c         |   16 +-
 drivers/gpu/drm/msm/registers/adreno/a3xx.xml |    8 +-
 drivers/gpu/drm/msm/registers/adreno/a5xx.xml |  141 +-
 drivers/gpu/drm/msm/registers/adreno/a6xx.xml | 1300 +++++------
 .../msm/registers/adreno/a6xx_descriptors.xml |   71 +-
 .../drm/msm/registers/adreno/a6xx_enums.xml   |    3 +
 .../msm/registers/adreno/a8xx_descriptors.xml |   96 +-
 .../msm/registers/adreno/a8xx_perfcntrs.xml   | 1929 +++++++++++++++++
 .../msm/registers/adreno/adreno_common.xml    |   42 +
 .../drm/msm/registers/adreno/adreno_pm4.xml   |   50 +-
 10 files changed, 2846 insertions(+), 810 deletions(-)
 create mode 100644 drivers/gpu/drm/msm/registers/adreno/a8xx_perfcntrs.xml
drivers/gpu/drm/msm/adreno/a5xx_gpu.c +5 -11
index 79acae11154a..9a03684288d3 100644
@@ -752,17 +752,13 @@ static int a5xx_hw_init(struct msm_gpu *gpu)
 	gpu_write(gpu, REG_A5XX_UCHE_CACHE_WAYS, 0x02);
 
 	/* Disable L2 bypass in the UCHE */
-	gpu_write(gpu, REG_A5XX_UCHE_TRAP_BASE_LO, lower_32_bits(adreno_gpu->uche_trap_base));
-	gpu_write(gpu, REG_A5XX_UCHE_TRAP_BASE_HI, upper_32_bits(adreno_gpu->uche_trap_base));
-	gpu_write(gpu, REG_A5XX_UCHE_WRITE_THRU_BASE_LO, lower_32_bits(adreno_gpu->uche_trap_base));
-	gpu_write(gpu, REG_A5XX_UCHE_WRITE_THRU_BASE_HI, upper_32_bits(adreno_gpu->uche_trap_base));
+	gpu_write64(gpu, REG_A5XX_UCHE_TRAP_BASE, adreno_gpu->uche_trap_base);
+	gpu_write64(gpu, REG_A5XX_UCHE_WRITE_THRU_BASE, adreno_gpu->uche_trap_base);
 
 	/* Set the GMEM VA range (0 to gpu->gmem) */
-	gpu_write(gpu, REG_A5XX_UCHE_GMEM_RANGE_MIN_LO, 0x00100000);
-	gpu_write(gpu, REG_A5XX_UCHE_GMEM_RANGE_MIN_HI, 0x00000000);
-	gpu_write(gpu, REG_A5XX_UCHE_GMEM_RANGE_MAX_LO,
+	gpu_write64(gpu, REG_A5XX_UCHE_GMEM_RANGE_MIN, 0x00100000);
+	gpu_write64(gpu, REG_A5XX_UCHE_GMEM_RANGE_MAX,
 		0x00100000 + adreno_gpu->info->gmem - 1);
-	gpu_write(gpu, REG_A5XX_UCHE_GMEM_RANGE_MAX_HI, 0x00000000);
 
 	if (adreno_is_a505(adreno_gpu) || adreno_is_a506(adreno_gpu) ||
 	    adreno_is_a508(adreno_gpu) || adreno_is_a510(adreno_gpu)) {
@@ -1217,9 +1213,7 @@ static void a5xx_rbbm_err_irq(struct msm_gpu *gpu, u32 status)
 
 static void a5xx_uche_err_irq(struct msm_gpu *gpu)
 {
-	uint64_t addr = (uint64_t) gpu_read(gpu, REG_A5XX_UCHE_TRAP_LOG_HI);
-
-	addr |= gpu_read(gpu, REG_A5XX_UCHE_TRAP_LOG_LO);
+	uint64_t addr = gpu_read64(gpu, REG_A5XX_UCHE_TRAP_LOG);
 
 	dev_err_ratelimited(gpu->dev->dev, "UCHE | Out of bounds access | addr=0x%llX\n",
 		addr);
drivers/gpu/drm/msm/registers/adreno/a3xx.xml +2 -6
index 6717abc0a897..09c9606fc3e1 100644
@@ -1330,11 +1330,7 @@ xsi:schemaLocation="https://gitlab.freedesktop.org/freedreno/ rules-fd.xsd">
 	</reg32>
 	<reg32 offset="0x22c5" name="SP_VS_CTRL_REG1">
 		<bitfield name="CONSTLENGTH" low="0" high="9" type="uint"/>
-		<!--
-			not sure about full vs half const.. I can't get blob generate
-			something with a mediump/lowp uniform.
-		 -->
-		<bitfield name="CONSTFOOTPRINT" low="10" high="19" type="uint"/>
+		<bitfield name="CONSTMAXID" low="10" high="19" type="uint"/>
 		<bitfield name="INITIALOUTSTANDING" low="24" high="30" type="uint"/>
 	</reg32>
 	<reg32 offset="0x22c6" name="SP_VS_PARAM_REG">
@@ -1420,7 +1416,7 @@ xsi:schemaLocation="https://gitlab.freedesktop.org/freedreno/ rules-fd.xsd">
 	</reg32>
 	<reg32 offset="0x22e1" name="SP_FS_CTRL_REG1">
 		<bitfield name="CONSTLENGTH" low="0" high="9" type="uint"/>
-		<bitfield name="CONSTFOOTPRINT" low="10" high="19" type="uint"/>
+		<bitfield name="CONSTMAXID" low="10" high="19" type="uint"/>
 		<bitfield name="INITIALOUTSTANDING" low="20" high="23" type="uint"/>
 		<bitfield name="HALFPRECVAROFFSET" low="24" high="30" type="uint"/>
 	</reg32>
drivers/gpu/drm/msm/registers/adreno/a5xx.xml +48 -93

311 lines not on this page · raw message

drivers/gpu/drm/msm/registers/adreno/a6xx.xml +692 -608

2,620 lines not on this page · raw message

drivers/gpu/drm/msm/registers/adreno/a6xx_descriptors.xml +43 -28

18 lines not on this page · raw message

drivers/gpu/drm/msm/registers/adreno/a6xx_enums.xml +3 -0
index 81538831dc19..b44946f36fae 100644

Lines of this file are not shown. View the raw message.

drivers/gpu/drm/msm/registers/adreno/a8xx_descriptors.xml +56 -40

96 lines not on this page · raw message

drivers/gpu/drm/msm/registers/adreno/a8xx_perfcntrs.xml new file +1929 -0

1,929 lines not on this page · raw message

drivers/gpu/drm/msm/registers/adreno/adreno_common.xml +42 -0

42 lines not on this page · raw message

drivers/gpu/drm/msm/registers/adreno/adreno_pm4.xml +26 -24

50 lines not on this page · raw message

The rest of this diff is not shown. View the raw message.

Signature · 1 line
2.54.0

Part of the diff is not shown here. View the raw message.

author maintainer
#4 lore raw

PATCH v5 4/16 drm/msm/registers: Sync gen_header.py from mesa

In reply to Rob Clark

Update gen_header.py to bring in support for generating perfcntr tables.

Sync from mesa commit 1fd18a9734ad ("freedreno/registers: Add gen8 perfcntr support")

---
 drivers/gpu/drm/msm/registers/gen_header.py | 2079 ++++++++++---------
 1 file changed, 1146 insertions(+), 933 deletions(-)
drivers/gpu/drm/msm/registers/gen_header.py +1146 -933
index 2acad951f1e2..07e6f0cb4e66 100644
@@ -11,997 +11,1210 @@ import collections
 import argparse
 import time
 import datetime
+import json
+
 
 class Error(Exception):
-	def __init__(self, message):
-		self.message = message
+    def __init__(self, message):
+        self.message = message
+
 
 class Enum(object):
-	def __init__(self, name):
-		self.name = name
-		self.values = []
-
-	def has_name(self, name):
-		for (n, value) in self.values:
-			if n == name:
-				return True
-		return False
-
-	def names(self):
-		return [n for (n, value) in self.values]
-
-	def dump(self, is_deprecated):
-		use_hex = False
-		for (name, value) in self.values:
-			if value > 0x1000:
-				use_hex = True
-
-		print("enum %s {" % self.name)
-		for (name, value) in self.values:
-			if use_hex:
-				print("\t%s = 0x%08x," % (name, value))
-			else:
-				print("\t%s = %d," % (name, value))
-		print("};\n")
-
-	def dump_pack_struct(self, is_deprecated):
-		pass
+    def __init__(self, name):
+        self.name = name
+        self.values = []
+
+    def has_name(self, name):
+        for (n, value) in self.values:
+            if n == name:
+                return True
+        return False
+
+    def names(self):
+        return [n for (n, value) in self.values]
+
+    def value(self, name):
+        for (n, v) in self.values:
+            if n == name:
+                return v
+
+    def dump(self, has_variants):
+        use_hex = False
+        for (name, value) in self.values:
+            if value > 0x1000:
+                use_hex = True
+
+        print("enum %s {" % self.name)
+        for (name, value) in self.values:
+            if use_hex:
+                print("\t%s = 0x%08x," % (name, value))
+            else:
+                print("\t%s = %d," % (name, value))
+        print("};\n")
+
+    def dump_pack_struct(self, has_variants):
+        pass
+
 
 class Field(object):
-	def __init__(self, name, low, high, shr, type, parser):
-		self.name = name
-		self.low = low
-		self.high = high
-		self.shr = shr
-		self.type = type
-
-		builtin_types = [ None, "a3xx_regid", "boolean", "uint", "hex", "int", "fixed", "ufixed", "float", "address", "waddress" ]
-
-		maxpos = parser.current_bitsize - 1
-
-		if low < 0 or low > maxpos:
-			raise parser.error("low attribute out of range: %d" % low)
-		if high < 0 or high > maxpos:
-			raise parser.error("high attribute out of range: %d" % high)
-		if high < low:
-			raise parser.error("low is greater than high: low=%d, high=%d" % (low, high))
-		if self.type == "boolean" and not low == high:
-			raise parser.error("booleans should be 1 bit fields")
-		elif self.type == "float" and not (high - low == 31 or high - low == 15):
-			raise parser.error("floats should be 16 or 32 bit fields")
-		elif self.type not in builtin_types and self.type not in parser.enums:
-			raise parser.error("unknown type '%s'" % self.type)
-
-	def ctype(self, var_name):
-		if self.type is None:
-			type = "uint32_t"
-			val = var_name
-		elif self.type == "boolean":
-			type = "bool"
-			val = var_name
-		elif self.type == "uint" or self.type == "hex" or self.type == "a3xx_regid":
-			type = "uint32_t"
-			val = var_name
-		elif self.type == "int":
-			type = "int32_t"
-			val = var_name
-		elif self.type == "fixed":
-			type = "float"
-			val = "((int32_t)(%s * %d.0))" % (var_name, 1 << self.radix)
-		elif self.type == "ufixed":
-			type = "float"
-			val = "((uint32_t)(%s * %d.0))" % (var_name, 1 << self.radix)
-		elif self.type == "float" and self.high - self.low == 31:
-			type = "float"
-			val = "fui(%s)" % var_name
-		elif self.type == "float" and self.high - self.low == 15:
-			type = "float"
-			val = "_mesa_float_to_half(%s)" % var_name
-		elif self.type in [ "address", "waddress" ]:
-			type = "uint64_t"
-			val = var_name
-		else:
-			type = "enum %s" % self.type
-			val = var_name
-
-		if self.shr > 0:
-			val = "(%s >> %d)" % (val, self.shr)
-
-		return (type, val)
+    def __init__(self, name, low, high, shr, type, parser):
+        self.name = name
+        self.low = low
+        self.high = high
+        self.shr = shr
+        self.type = type
+
+        builtin_types = [None, "a3xx_regid", "boolean", "uint", "hex",
+                         "int", "fixed", "ufixed", "float", "address", "waddress"]
+
+        maxpos = parser.current_bitsize - 1
+
+        if low < 0 or low > maxpos:
+            raise parser.error("low attribute out of range: %d" % low)
+        if high < 0 or high > maxpos:
+            raise parser.error("high attribute out of range: %d" % high)
+        if high < low:
+            raise parser.error(
+                "low is greater than high: low=%d, high=%d" % (low, high))
+        if self.type == "boolean" and not low == high:
+            raise parser.error("booleans should be 1 bit fields")
+        elif self.type == "float" and not (high - low == 31 or high - low == 15):
+            raise parser.error("floats should be 16 or 32 bit fields")
+        elif self.type not in builtin_types and self.type not in parser.enums:
+            raise parser.error("unknown type '%s'" % self.type)
+
+    def ctype(self, var_name):
+        if self.type is None:
+            type = "uint32_t"
+            val = var_name
+        elif self.type == "boolean":
+            type = "bool"
+            val = var_name
+        elif self.type == "uint" or self.type == "hex" or self.type == "a3xx_regid":
+            type = "uint32_t"
+            val = var_name
+        elif self.type == "int":
+            type = "int32_t"
+            val = var_name
+        elif self.type == "fixed":
+            type = "float"
+            val = "(uint32_t)((int32_t)(%s * %d.0))" % (var_name, 1 << self.radix)
+        elif self.type == "ufixed":
+            type = "float"
+            val = "((uint32_t)(%s * %d.0))" % (var_name, 1 << self.radix)
+        elif self.type == "float" and self.high - self.low == 31:
+            type = "float"
+            val = "fui(%s)" % var_name
+        elif self.type == "float" and self.high - self.low == 15:
+            type = "float"
+            val = "_mesa_float_to_half(%s)" % var_name
+        elif self.type in ["address", "waddress"]:
+            type = "uint64_t"
+            val = var_name
+        else:
+            type = "enum %s" % self.type
+            val = var_name
+
+        if self.shr > 0:
+            val = "(%s >> %d)" % (val, self.shr)
+
+        return (type, val)
+
 
 def tab_to(name, value):
-	tab_count = (68 - (len(name) & ~7)) // 8
-	if tab_count <= 0:
-		tab_count = 1
-	print(name + ('\t' * tab_count) + value)
+    tab_count = (68 - (len(name) & ~7)) // 8
+    if tab_count <= 0:
+        tab_count = 1
+    print(name + ('\t' * tab_count) + value)
+
+def define_macro(name, value, has_variants):
+    if has_variants:
+        value = "__FD_DEPRECATED " + value
+    tab_to(name, value)
 
 def mask(low, high):
-	return ((0xffffffffffffffff >> (64 - (high + 1 - low))) << low)
+    return ((0xffffffffffffffff >> (64 - (high + 1 - low))) << low)
+
 
 def field_name(reg, f):
-	if f.name:
-		name = f.name.lower()
-	else:
-		# We hit this path when a reg is defined with no bitset fields, ie.
-		# 	<reg32 offset="0x88db" name="RB_RESOLVE_SYSTEM_BUFFER_ARRAY_PITCH" low="0" high="28" shr="6" type="uint"/>
-		name = reg.name.lower()
+    if f.name:
+        name = f.name.lower()
+    else:
+        # We hit this path when a reg is defined with no bitset fields, ie.
+        # 	<reg32 offset="0x88db" name="RB_RESOLVE_SYSTEM_BUFFER_ARRAY_PITCH" low="0" high="28" shr="6" type="uint"/>
+        name = reg.name.lower()
 
-	if (name in [ "double", "float", "int" ]) or not (name[0].isalpha()):
-			name = "_" + name
+    if (name in ["double", "float", "int"]) or not (name[0].isalpha()):
+        name = "_" + name
 
-	return name
+    return name
 
 # indices - array of (ctype, stride, __offsets_NAME)
+
+
 def indices_varlist(indices):
-	return ", ".join(["i%d" % i for i in range(len(indices))])
+    return ", ".join(["i%d" % i for i in range(len(indices))])
+
 
 def indices_prototype(indices):
-	return ", ".join(["%s i%d" % (ctype, idx)
-			for (idx, (ctype, stride, offset)) in  enumerate(indices)])
+    return ", ".join(["%s i%d" % (ctype, idx)
+                      for (idx, (ctype, stride, offset)) in enumerate(indices)])
+
 
 def indices_strides(indices):
-	return " + ".join(["0x%x*i%d" % (stride, idx)
-					if stride else
-					"%s(i%d)" % (offset, idx)
-			for (idx, (ctype, stride, offset)) in  enumerate(indices)])
+    return " + ".join(["0x%x*i%d" % (stride, idx)
+                       if stride else
+                       "%s(i%d)" % (offset, idx)
+                       for (idx, (ctype, stride, offset)) in enumerate(indices)])
+
 
 def is_number(str):
-	try:
-		int(str)
-		return True
-	except ValueError:
-		return False
+    try:
+        int(str)
+        return True
+    except ValueError:
+        return False
+
 
 def sanitize_variant(variant):
-	if variant and "-" in variant:
-		return variant[:variant.index("-")]
-	return variant
+    if variant and "-" in variant:
+        return variant[:variant.index("-")]
+    return variant
+
 
 class Bitset(object):
-	def __init__(self, name, template):
-		self.name = name
-		self.inline = False
-		self.reg = None
-		if template:
-			self.fields = template.fields[:]
-		else:
-			self.fields = []
-
-	# Get address field if there is one in the bitset, else return None:
-	def get_address_field(self):
-		for f in self.fields:
-			if f.type in [ "address", "waddress" ]:
-				return f
-		return None
-
-	def dump_regpair_builder(self, reg):
-		print("#ifndef NDEBUG")
-		known_mask = 0
-		for f in self.fields:
-			known_mask |= mask(f.low, f.high)
-			if f.type in [ "boolean", "address", "waddress" ]:
-				continue
-			type, val = f.ctype("fields.%s" % field_name(reg, f))
-			print("    assert((%-40s & 0x%08x) == 0);" % (val, 0xffffffff ^ mask(0 , f.high - f.low)))
-		print("    assert((%-40s & 0x%08x) == 0);" % ("fields.unknown", known_mask))
-		print("#endif\n")
-
-		print("    return (struct fd_reg_pair) {")
-		print("        .reg = (uint32_t)%s," % reg.reg_offset())
-		print("        .value =")
-		cast = "(uint64_t)" if reg.bit_size == 64 else ""
-		for f in self.fields:
-			if f.type in [ "address", "waddress" ]:
-				continue
-			else:
-				type, val = f.ctype("fields.%s" % field_name(reg, f))
-				print("            (%s%-40s << %2d) |" % (cast, val, f.low))
-		value_name = "dword"
-		if reg.bit_size == 64:
-			value_name = "qword"
-		print("            fields.unknown | fields.%s," % (value_name,))
-
-		address = self.get_address_field()
-		if address:
-			print("        .bo = fields.bo,")
-			print("        .is_address = true,")
-			if f.type == "waddress":
-				print("        .bo_write = true,")
-			print("        .bo_offset = fields.bo_offset,")
-			print("        .bo_shift = %d," % address.shr)
-			print("        .bo_low = %d," % address.low)
-
-		print("    };")
-
-	def dump_pack_struct(self, is_deprecated, reg=None):
-		if not reg:
-			return
-
-		prefix = reg.full_name
-
-		print("struct %s {" % prefix)
-		for f in self.fields:
-			if f.type in [ "address", "waddress" ]:
-				tab_to("    __bo_type", "bo;")
-				tab_to("    uint32_t", "bo_offset;")
-				continue
-			name = field_name(reg, f)
-
-			type, val = f.ctype("var")
-
-			tab_to("    %s" % type, "%s;" % name)
-		if reg.bit_size == 64:
-			tab_to("    uint64_t", "unknown;")
-			tab_to("    uint64_t", "qword;")
-		else:
-			tab_to("    uint32_t", "unknown;")
-			tab_to("    uint32_t", "dword;")
-		print("};\n")
-
-		depcrstr = ""
-		if is_deprecated:
-			depcrstr = " FD_DEPRECATED"
-		if reg.array:
-			print("static inline%s struct fd_reg_pair\npack_%s(uint32_t __i, struct %s fields)\n{" %
-				  (depcrstr, prefix, prefix))
-		else:
-			print("static inline%s struct fd_reg_pair\npack_%s(struct %s fields)\n{" %
-				  (depcrstr, prefix, prefix))
-
-		self.dump_regpair_builder(reg)
-
-		print("\n}\n")
-
-		if self.get_address_field():
-			skip = ", { .reg = 0 }"
-		else:
-			skip = ""
-
-		if reg.array:
-			print("#define %s(__i, ...) pack_%s(__i, __struct_cast(%s) { __VA_ARGS__ })%s\n" %
-				  (prefix, prefix, prefix, skip))
-		else:
-			print("#define %s(...) pack_%s(__struct_cast(%s) { __VA_ARGS__ })%s\n" %
-				  (prefix, prefix, prefix, skip))
-
-
-	def dump(self, is_deprecated, prefix=None, reg=None):
-		if prefix is None:
-			prefix = self.name
-		reg64 = reg and self.reg and self.reg.bit_size == 64
-		if reg64:
-			print("static inline uint32_t %s_LO(uint32_t val)\n{" % prefix)
-			print("\treturn val;\n}")
-			print("static inline uint32_t %s_HI(uint32_t val)\n{" % prefix)
-			print("\treturn val;\n}")
-		for f in self.fields:
-			if f.name:
-				name = prefix + "_" + f.name
-			else:
-				name = prefix
-
-			if not f.name and f.low == 0 and f.shr == 0 and f.type not in ["float", "fixed", "ufixed"]:
-				pass
-			elif f.type == "boolean" or (f.type is None and f.low == f.high):
-				tab_to("#define %s" % name, "0x%08x" % (1 << f.low))
-			else:
-				typespec = "ull" if reg64 else "u"
-				tab_to("#define %s__MASK" % name, "0x%08x%s" % (mask(f.low, f.high), typespec))
-				tab_to("#define %s__SHIFT" % name, "%d" % f.low)
-				type, val = f.ctype("val")
-				ret_type = "uint64_t" if reg64 else "uint32_t"
-				cast = "(uint64_t)" if reg64 else ""
-
-				print("static inline %s %s(%s val)\n{" % (ret_type, name, type))
-				if f.shr > 0:
-					print("\tassert(!(val & 0x%x));" % mask(0, f.shr - 1))
-				print("\treturn (%s(%s) << %s__SHIFT) & %s__MASK;\n}" % (cast, val, name, name))
-		print()
+    def __init__(self, name, template):
+        self.name = name
+        self.inline = False
+        self.reg = None
+        if template:
+            self.fields = template.fields[:]
+        else:
+            self.fields = []
+
+    # Get address field if there is one in the bitset, else return None:
+    def get_address_field(self):
+        for f in self.fields:
+            if f.type in ["address", "waddress"]:
+                return f
+        return None
+
+    def dump_regpair_builder(self, reg):
+        print("#ifndef NDEBUG")
+        known_mask = 0
+        for f in self.fields:
+            known_mask |= mask(f.low, f.high)
+            if f.type in ["boolean", "address", "waddress"]:
+                continue
+            type, val = f.ctype("fields.%s" % field_name(reg, f))
+            print("    assert((%-40s & 0x%08x) == 0);" %
+                  (val, 0xffffffff ^ mask(0, f.high - f.low)))
+        print("    assert((%-40s & 0x%08x) == 0);" %
+              ("fields.unknown", known_mask))
+        print("#endif\n")
+
+        print("    return (struct fd_reg_pair) {")
+        print("        .reg = (uint32_t)%s," % reg.reg_offset())
+        print("        .value =")
+        cast = "(uint64_t)" if reg.bit_size == 64 else ""
+        for f in self.fields:
+            if f.type in ["address", "waddress"]:
+                continue
+            else:
+                type, val = f.ctype("fields.%s" % field_name(reg, f))
+                print("            (%s%-40s << %2d) |" % (cast, val, f.low))
+        value_name = "dword"
+        if reg.bit_size == 64:
+            value_name = "qword"
+        print("            fields.unknown | fields.%s," % (value_name,))
+
+        address = self.get_address_field()
+        if address:
+            print("#ifndef TU_CS_H")
+            print("        .bo = fields.bo,")
+            print("        .is_address = true,")
+            print("        .bo_offset = fields.bo_offset,")
+            print("        .bo_shift = %d," % address.shr)
+            print("        .bo_low = %d," % address.low)
+            print("#else")
+            print("        .is_address = true,")
+            print("#endif")
+
+        print("    };")
+
+    def dump_pack_struct(self, has_variants, reg=None):
+        if not reg:
+            return
+
+        prefix = reg.full_name
+
+        constexpr_mark = " CONSTEXPR"
+
+        print("struct %s {" % prefix)
+        for f in self.fields:
+            if f.type in ["address", "waddress"]:
+                print("#ifndef TU_CS_H")
+                tab_to("    __bo_type", "bo;")
+                tab_to("    uint32_t", "bo_offset;")
+                print("#endif\n")
+                continue
+            name = field_name(reg, f)
+
+            type, val = f.ctype("var")
+
+            tab_to("    %s" % type, "%s;" % name)
+
+            if f.type == "float":
+                # Requires using `fui()` or `_mesa_float_to_half()`
+                constexpr_mark = ""
+        if reg.bit_size == 64:
+            tab_to("    uint64_t", "qword;")
+            tab_to("    uint64_t", "unknown;")
+        else:
+            tab_to("    uint32_t", "dword;")
+            tab_to("    uint32_t", "unknown;")
+        print("};\n")
+
+        if not has_variants:
+            print("static%s inline struct fd_reg_pair" % constexpr_mark)
+            if reg.array:
+                print("pack_%s(uint32_t __i, struct %s fields)\n{" % (prefix, prefix))
+            else:
+                print("pack_%s(struct %s fields)\n{" % (prefix, prefix))
+
+            self.dump_regpair_builder(reg)
+
+            print("\n}\n")
+
+        if self.get_address_field():
+            skip = ", { .reg = 0 }"
+        else:
+            skip = ""
+
+        if reg.array:
+            print("#define %s(__i, ...) pack_%s(__i, __struct_cast(%s) { __VA_ARGS__ })%s\n" %
+                  (prefix, prefix, prefix, skip))
+        else:
+            print("#define %s(...) pack_%s(__struct_cast(%s) { __VA_ARGS__ })%s\n" %
+                  (prefix, prefix, prefix, skip))
+
+    def dump(self, has_variants, prefix=None, reg=None):
+        if prefix is None:
+            prefix = self.name
+        suffix = ""
+        if self.reg and self.reg.bit_size == 64:
+            print(
+                "static CONSTEXPR inline uint32_t %s_LO(uint32_t val)\n{" % prefix)
+            print("\treturn val;\n}")
+            print(
+                "static CONSTEXPR inline uint32_t %s_HI(uint32_t val)\n{" % prefix)
+            print("\treturn val;\n}")
+            suffix = "ull"
+
+        for f in self.fields:
+            if f.name:
+                name = prefix + "_" + f.name
+            else:
+                name = prefix
+
+            if not f.name and f.low == 0 and f.shr == 0 and f.type not in ["float", "fixed", "ufixed"]:
+                pass
+            elif f.type == "boolean" or (f.type is None and f.low == f.high):
+                tab_to("#define %s" % name, "0x%08x%s" % ((1 << f.low), suffix))
+            else:
+                tab_to("#define %s__MASK" %
+                       name, "0x%08x%s" % (mask(f.low, f.high), suffix))
+                tab_to("#define %s__SHIFT" % name, "%d" % f.low)
+                type, val = f.ctype("val")
+                ret_type = "uint64_t" if reg and reg.bit_size == 64 else "uint32_t"
+                cast = "(uint64_t)" if reg and reg.bit_size == 64 else ""
+
+                constexpr_mark = "" if type == "float" else " CONSTEXPR"
+                print("static%s inline %s %s(%s val)\n{" % (
+                    constexpr_mark, ret_type, name, type))
+                if f.shr > 0:
+                    print("\tassert(!(val & 0x%x));" % mask(0, f.shr - 1))
+                print("\treturn (%s(%s) << %s__SHIFT) & %s__MASK;\n}" %
+                      (cast, val, name, name))
+        print()
+
 
 class Array(object):
-	def __init__(self, attrs, domain, variant, parent, index_type):
-		if "name" in attrs:
-			self.local_name = attrs["name"]
-		else:
-			self.local_name = ""
-		self.domain = domain
-		self.variant = variant
-		self.parent = parent
-		self.children = []
-		if self.parent:
-			self.name = self.parent.name + "_" + self.local_name
-		else:
-			self.name = self.local_name
-		if "offsets" in attrs:
-			self.offsets = map(lambda i: "0x%08x" % int(i, 0), attrs["offsets"].split(","))
-			self.fixed_offsets = True
-		elif "doffsets" in attrs:
-			self.offsets = map(lambda s: "(%s)" % s , attrs["doffsets"].split(","))
-			self.fixed_offsets = True
-		else:
-			self.offset = int(attrs["offset"], 0)
-			self.stride = int(attrs["stride"], 0)
-			self.fixed_offsets = False
-		if "index" in attrs:
-			self.index_type = index_type
-		else:
-			self.index_type = None
-		self.length = int(attrs["length"], 0)
-		if "usage" in attrs:
-			self.usages = attrs["usage"].split(',')
-		else:
-			self.usages = None
-
-	def index_ctype(self):
-		if not self.index_type:
-			return "uint32_t"
-		else:
-			return "enum %s" % self.index_type.name
-
-	# Generate array of (ctype, stride, __offsets_NAME)
-	def indices(self):
-		if self.parent:
-			indices = self.parent.indices()
-		else:
-			indices = []
-		if self.length != 1:
-			if self.fixed_offsets:
-				indices.append((self.index_ctype(), None, "__offset_%s" % self.local_name))
-			else:
-				indices.append((self.index_ctype(), self.stride, None))
-		return indices
-
-	def total_offset(self):
-		offset = 0
-		if not self.fixed_offsets:
-			offset += self.offset
-		if self.parent:
-			offset += self.parent.total_offset()
-		return offset
-
-	def dump(self, is_deprecated):
-		depcrstr = ""
-		if is_deprecated:
-			depcrstr = " FD_DEPRECATED"
-		proto = indices_varlist(self.indices())
-		strides = indices_strides(self.indices())
-		array_offset = self.total_offset()
-		if self.fixed_offsets:
-			print("static inline%s uint32_t __offset_%s(%s idx)" % (depcrstr, self.local_name, self.index_ctype()))
-			print("{\n\tswitch (idx) {")
-			if self.index_type:
-				for val, offset in zip(self.index_type.names(), self.offsets):
-					print("\t\tcase %s: return %s;" % (val, offset))
-			else:
-				for idx, offset in enumerate(self.offsets):
-					print("\t\tcase %d: return %s;" % (idx, offset))
-			print("\t\tdefault: return INVALID_IDX(idx);")
-			print("\t}\n}")
-		if proto == '':
-			tab_to("#define REG_%s_%s" % (self.domain, self.name), "0x%08x\n" % array_offset)
-		else:
-			tab_to("#define REG_%s_%s(%s)" % (self.domain, self.name, proto), "(0x%08x + %s )\n" % (array_offset, strides))
-
-	def dump_pack_struct(self, is_deprecated):
-		pass
-
-	def dump_regpair_builder(self):
-		pass
+    def __init__(self, attrs, domain, variant, parent, index_type):
+        if "name" in attrs:
+            self.local_name = attrs["name"]
+        else:
+            self.local_name = ""
+        self.domain = domain
+        self.variant = variant
+        self.parent = parent
+        self.children = []
+        if self.parent:
+            self.name = self.parent.name + "_" + self.local_name
+        else:
+            self.name = self.local_name
+        if "offsets" in attrs:
+            self.offsets = map(lambda i: "0x%08x" %
+                               int(i, 0), attrs["offsets"].split(","))
+            self.fixed_offsets = True
+        elif "doffsets" in attrs:
+            self.offsets = map(lambda s: "(%s)" %
+                               s, attrs["doffsets"].split(","))
+            self.fixed_offsets = True
+        else:
+            self.offset = int(attrs["offset"], 0)
+            self.stride = int(attrs["stride"], 0)
+            self.fixed_offsets = False
+        if "index" in attrs:
+            self.index_type = index_type
+        else:
+            self.index_type = None
+        self.length = int(attrs["length"], 0)
+        if "usage" in attrs:
+            self.usages = attrs["usage"].split(',')
+        else:
+            self.usages = None
+
+    def index_ctype(self):
+        if not self.index_type:
+            return "uint32_t"
+        else:
+            return "enum %s" % self.index_type.name
+
+    # Generate array of (ctype, stride, __offsets_NAME)
+    def indices(self):
+        if self.parent:
+            indices = self.parent.indices()
+        else:
+            indices = []
+        if self.length != 1:
+            if self.fixed_offsets:
+                indices.append((self.index_ctype(), None,
+                               "__offset_%s" % self.local_name))
+            else:
+                indices.append((self.index_ctype(), self.stride, None))
+        return indices
+
+    def total_offset(self):
+        offset = 0
+        if not self.fixed_offsets:
+            offset += self.offset
+        if self.parent:
+            offset += self.parent.total_offset()
+        return offset
+
+    def dump(self, has_variants):
+        proto = indices_varlist(self.indices())
+        strides = indices_strides(self.indices())
+        array_offset = self.total_offset()
+        if self.fixed_offsets and not has_variants:
+            print("static CONSTEXPR inline uint32_t __offset_%s(%s idx)" %
+                  (self.local_name, self.index_ctype()))
+            print("{\n\tswitch (idx) {")
+            if self.index_type:
+                for val, offset in zip(self.index_type.names(), self.offsets):
+                    print("\t\tcase %s: return %s;" % (val, offset))
+            else:
+                for idx, offset in enumerate(self.offsets):
+                    print("\t\tcase %d: return %s;" % (idx, offset))
+            print("\t\tdefault: return INVALID_IDX(idx);")
+            print("\t}\n}")
+        if proto == '':
+            define_macro("#define REG_%s_%s" %
+                         (self.domain, self.name), "0x%08x\n" % array_offset,
+                         has_variants)
+        else:
+            define_macro("#define REG_%s_%s(%s)" % (self.domain, self.name,
+                         proto), "(0x%08x + %s )\n" % (array_offset, strides),
+                         has_variants)
+
+    def dump_pack_struct(self, has_variants):
+        pass
+
+    def dump_regpair_builder(self):
+        pass
+
 
 class Reg(object):
-	def __init__(self, attrs, domain, array, bit_size):
-		self.name = attrs["name"]
-		self.domain = domain
-		self.array = array
-		self.offset = int(attrs["offset"], 0)
-		self.type = None
-		self.bit_size = bit_size
-		if array:
-			self.name = array.name + "_" + self.name
-			array.children.append(self)
-		self.full_name = self.domain + "_" + self.name
-		if "stride" in attrs:
-			self.stride = int(attrs["stride"], 0)
-			self.length = int(attrs["length"], 0)
-		else:
-			self.stride = None
-			self.length = None
-
-	# Generate array of (ctype, stride, __offsets_NAME)
-	def indices(self):
-		if self.array:
-			indices = self.array.indices()
-		else:
-			indices = []
-		if self.stride:
-			indices.append(("uint32_t", self.stride, None))
-		return indices
-
-	def total_offset(self):
-		if self.array:
-			return self.array.total_offset() + self.offset
-		else:
-			return self.offset
-
-	def reg_offset(self):
-		if self.array:
-			offset = self.array.offset + self.offset
-			return "(0x%08x + 0x%x*__i)" % (offset, self.array.stride)
-		return "0x%08x" % self.offset
-
-	def dump(self, is_deprecated):
-		depcrstr = ""
-		if is_deprecated:
-			depcrstr = " FD_DEPRECATED "
-		proto = indices_prototype(self.indices())
-		strides = indices_strides(self.indices())
-		offset = self.total_offset()
-		if proto == '':
-			tab_to("#define REG_%s" % self.full_name, "0x%08x" % offset)
-		else:
-			print("static inline%s uint32_t REG_%s(%s) { return 0x%08x + %s; }" % (depcrstr, self.full_name, proto, offset, strides))
-
-		if self.bitset.inline:
-			self.bitset.dump(is_deprecated, self.full_name, self)
-		print("")
-
-	def dump_pack_struct(self, is_deprecated):
-		if self.bitset.inline:
-			self.bitset.dump_pack_struct(is_deprecated, self)
-
-	def dump_regpair_builder(self):
-		self.bitset.dump_regpair_builder(self)
-
-	def dump_py(self):
-		print("\tREG_%s = 0x%08x" % (self.full_name, self.offset))
+    def __init__(self, attrs, domain, array, bit_size):
+        self.name = attrs["name"]
+        self.domain = domain
+        self.array = array
+        self.offset = int(attrs["offset"], 0)
+        self.type = None
+        self.bit_size = bit_size
+        if array:
+            self.name = array.name + "_" + self.name
+            array.children.append(self)
+        self.full_name = self.domain + "_" + self.name
+        if "stride" in attrs:
+            self.stride = int(attrs["stride"], 0)
+            self.length = int(attrs["length"], 0)
+        else:
+            self.stride = None
+            self.length = None
+
+    # Generate array of (ctype, stride, __offsets_NAME)
+    def indices(self):
+        if self.array:
+            indices = self.array.indices()
+        else:
+            indices = []
+        if self.stride:
+            indices.append(("uint32_t", self.stride, None))
+        return indices
+
+    def total_offset(self):
+        if self.array:
+            return self.array.total_offset() + self.offset
+        else:
+            return self.offset
+
+    def reg_offset(self):
+        if self.array:
+            offset = self.array.offset + self.offset
+            return "(0x%08x + 0x%x*__i)" % (offset, self.array.stride)
+        return "0x%08x" % self.offset
+
+    def dump(self, has_variants):
+        proto = indices_prototype(self.indices())
+        strides = indices_strides(self.indices())
+        offset = self.total_offset()
+        if proto == '':
+            define_macro("#define REG_%s" % self.full_name, "0x%08x" % offset, has_variants)
+        elif not has_variants:
+            depcrstr = ""
+            if has_variants:
+                depcrstr = " __FD_DEPRECATED "
+            print("static CONSTEXPR inline%s uint32_t REG_%s(%s) { return 0x%08x + %s; }" % (
+                  depcrstr, self.full_name, proto, offset, strides))
+
+        if self.bitset.inline:
+            self.bitset.dump(has_variants, self.full_name, self)
+        print("")
+
+    def dump_pack_struct(self, has_variants):
+        if self.bitset.inline:
+            self.bitset.dump_pack_struct(has_variants, self)
+
+    def dump_regpair_builder(self):
+        self.bitset.dump_regpair_builder(self)
+
+    def dump_py(self):
+        offset = self.offset
+        if self.array:
+            offset += self.array.offset
+        print("\tREG_%s = 0x%08x" % (self.full_name, offset))
 
 
 class Parser(object):
-	def __init__(self):
-		self.current_array = None
-		self.current_domain = None
-		self.current_prefix = None
-		self.current_prefix_type = None
-		self.current_stripe = None
-		self.current_bitset = None
-		self.current_bitsize = 32
-		# The varset attribute on the domain specifies the enum which
-		# specifies all possible hw variants:
-		self.current_varset = None
-		# Regs that have multiple variants.. we only generated the C++
-		# template based struct-packers for these
-		self.variant_regs = {}
-		# Information in which contexts regs are used, to be used in
-		# debug options
-		self.usage_regs = collections.defaultdict(list)
-		self.bitsets = {}
-		self.enums = {}
-		self.variants = set()
-		self.file = []
-		self.xml_files = []
-
-	def error(self, message):
-		parser, filename = self.stack[-1]
-		return Error("%s:%d:%d: %s" % (filename, parser.CurrentLineNumber, parser.CurrentColumnNumber, message))
-
-	def prefix(self, variant=None):
-		if self.current_prefix_type == "variant" and variant:
-			return sanitize_variant(variant)
-		elif self.current_stripe:
-			return self.current_stripe + "_" + self.current_domain
-		elif self.current_prefix:
-			return self.current_prefix + "_" + self.current_domain
-		else:
-			return self.current_domain
-
-	def parse_field(self, name, attrs):
-		try:
-			if "pos" in attrs:
-				high = low = int(attrs["pos"], 0)
-			elif "high" in attrs and "low" in attrs:
-				high = int(attrs["high"], 0)
-				low = int(attrs["low"], 0)
-			else:
-				low = 0
-				high = self.current_bitsize - 1
-
-			if "type" in attrs:
-				type = attrs["type"]
-			else:
-				type = None
-
-			if "shr" in attrs:
-				shr = int(attrs["shr"], 0)
-			else:
-				shr = 0
-
-			b = Field(name, low, high, shr, type, self)
-
-			if type == "fixed" or type == "ufixed":
-				b.radix = int(attrs["radix"], 0)
-
-			self.current_bitset.fields.append(b)
-		except ValueError as e:
-			raise self.error(e)
-
-	def parse_varset(self, attrs):
-		# Inherit the varset from the enclosing domain if not overriden:
-		varset = self.current_varset
-		if "varset" in attrs:
-			varset = self.enums[attrs["varset"]]
-		return varset
-
-	def parse_variants(self, attrs):
-		if "variants" not in attrs:
-				return None
-
-		variant = attrs["variants"].split(",")[0]
-		varset = self.parse_varset(attrs)
-
-		if "-" in variant:
-			# if we have a range, validate that both the start and end
-			# of the range are valid enums:
-			start = variant[:variant.index("-")]
-			end = variant[variant.index("-") + 1:]
-			assert varset.has_name(start)
-			if end != "":
-				assert varset.has_name(end)
-		else:
-			assert varset.has_name(variant)
-
-		return variant
-
-	def add_all_variants(self, reg, attrs, parent_variant):
-		# TODO this should really handle *all* variants, including dealing
-		# with open ended ranges (ie. "A2XX,A4XX-") (we have the varset
-		# enum now to make that possible)
-		variant = self.parse_variants(attrs)
-		if not variant:
-			variant = parent_variant
-
-		if reg.name not in self.variant_regs:
-			self.variant_regs[reg.name] = {}
-		else:
-			# All variants must be same size:
-			v = next(iter(self.variant_regs[reg.name]))
-			assert self.variant_regs[reg.name][v].bit_size == reg.bit_size
-
-		self.variant_regs[reg.name][variant] = reg
-
-	def add_all_usages(self, reg, usages):
-		if not usages:
-			return
-
-		for usage in usages:
-			self.usage_regs[usage].append(reg)
-
-		self.variants.add(reg.domain)
-
-	def do_validate(self, schemafile):
-		if not self.validate:
-			return
-
-		try:
-			from lxml import etree
-
-			parser, filename = self.stack[-1]
-			dirname = os.path.dirname(filename)
-
-			# we expect this to look like <namespace url> schema.xsd.. I think
-			# technically it is supposed to be just a URL, but that doesn't
-			# quite match up to what we do.. Just skip over everything up to
-			# and including the first whitespace character:
-			schemafile = schemafile[schemafile.rindex(" ")+1:]
-
-			# this is a bit cheezy, but the xml file to validate could be
-			# in a child director, ie. we don't really know where the schema
-			# file is, the way the rnn C code does.  So if it doesn't exist
-			# just look one level up
-			if not os.path.exists(dirname + "/" + schemafile):
-				schemafile = "../" + schemafile
-
-			if not os.path.exists(dirname + "/" + schemafile):
-				raise self.error("Cannot find schema for: " + filename)
-
-			xmlschema_doc = etree.parse(dirname + "/" + schemafile)
-			xmlschema = etree.XMLSchema(xmlschema_doc)
-
-			xml_doc = etree.parse(filename)
-			if not xmlschema.validate(xml_doc):
-				error_str = str(xmlschema.error_log.filter_from_errors()[0])
-				raise self.error("Schema validation failed for: " + filename + "\n" + error_str)
-		except ImportError as e:
-			print("lxml not found, skipping validation", file=sys.stderr)
-
-	def do_parse(self, filename):
-		filepath = os.path.abspath(filename)
-		if filepath in self.xml_files:
-			return
-		self.xml_files.append(filepath)
-		file = open(filename, "rb")
-		parser = xml.parsers.expat.ParserCreate()
-		self.stack.append((parser, filename))
-		parser.StartElementHandler = self.start_element
-		parser.EndElementHandler = self.end_element
-		parser.CharacterDataHandler = self.character_data
-		parser.buffer_text = True
-		parser.ParseFile(file)
-		self.stack.pop()
-		file.close()
-
-	def parse(self, rnn_path, filename, validate):
-		self.path = rnn_path
-		self.stack = []
-		self.validate = validate
-		self.do_parse(filename)
-
-	def parse_reg(self, attrs, bit_size):
-		self.current_bitsize = bit_size
-		if "type" in attrs and attrs["type"] in self.bitsets:
-			bitset = self.bitsets[attrs["type"]]
-			if bitset.inline:
-				self.current_bitset = Bitset(attrs["name"], bitset)
-				self.current_bitset.inline = True
-			else:
-				self.current_bitset = bitset
-		else:
-			self.current_bitset = Bitset(attrs["name"], None)
-			self.current_bitset.inline = True
-			if "type" in attrs:
-				self.parse_field(None, attrs)
-
-		variant = self.parse_variants(attrs)
-		if not variant and self.current_array:
-			variant = self.current_array.variant
-
-		self.current_reg = Reg(attrs, self.prefix(variant), self.current_array, bit_size)
-		self.current_reg.bitset = self.current_bitset
-		self.current_bitset.reg = self.current_reg
-
-		if len(self.stack) == 1:
-			self.file.append(self.current_reg)
-
-		if variant is not None:
-			self.add_all_variants(self.current_reg, attrs, variant)
-
-		usages = None
-		if "usage" in attrs:
-			usages = attrs["usage"].split(',')
-		elif self.current_array:
-			usages = self.current_array.usages
-
-		self.add_all_usages(self.current_reg, usages)
-
-	def start_element(self, name, attrs):
-		self.cdata = ""
-		if name == "import":
-			filename = attrs["file"]
-			self.do_parse(os.path.join(self.path, filename))
-		elif name == "domain":
-			self.current_domain = attrs["name"]
-			if "prefix" in attrs:
-				self.current_prefix = sanitize_variant(self.parse_variants(attrs))
-				self.current_prefix_type = attrs["prefix"]
-			else:
-				self.current_prefix = None
-				self.current_prefix_type = None
-			if "varset" in attrs:
-				self.current_varset = self.enums[attrs["varset"]]
-		elif name == "stripe":
-			self.current_stripe = sanitize_variant(self.parse_variants(attrs))
-		elif name == "enum":
-			self.current_enum_value = 0
-			self.current_enum = Enum(attrs["name"])
-			self.enums[attrs["name"]] = self.current_enum
-			if len(self.stack) == 1:
-				self.file.append(self.current_enum)
-		elif name == "value":
-			if "value" in attrs:
-				value = int(attrs["value"], 0)
-			else:
-				value = self.current_enum_value
-			self.current_enum.values.append((attrs["name"], value))
-		elif name == "reg32":
-			self.parse_reg(attrs, 32)
-		elif name == "reg64":
-			self.parse_reg(attrs, 64)
-		elif name == "array":
-			self.current_bitsize = 32
-			variant = self.parse_variants(attrs)
-			index_type = self.enums[attrs["index"]] if "index" in attrs else None
-			self.current_array = Array(attrs, self.prefix(variant), variant, self.current_array, index_type)
-			if len(self.stack) == 1:
-				self.file.append(self.current_array)
-		elif name == "bitset":
-			self.current_bitset = Bitset(attrs["name"], None)
-			if "inline" in attrs and attrs["inline"] == "yes":
-				self.current_bitset.inline = True
-			self.bitsets[self.current_bitset.name] = self.current_bitset
-			if len(self.stack) == 1 and not self.current_bitset.inline:
-				self.file.append(self.current_bitset)
-		elif name == "bitfield" and self.current_bitset:
-			self.parse_field(attrs["name"], attrs)
-		elif name == "database":
-			self.do_validate(attrs["xsi:schemaLocation"])
-
-	def end_element(self, name):
-		if name == "domain":
-			self.current_domain = None
-			self.current_prefix = None
-			self.current_prefix_type = None
-		elif name == "stripe":
-			self.current_stripe = None
-		elif name == "bitset":
-			self.current_bitset = None
-		elif name == "reg32":
-			self.current_reg = None
-		elif name == "array":
-			# if the array has no Reg children, push an implicit reg32:
-			if len(self.current_array.children) == 0:
-				attrs = {
-					"name": "REG",
-					"offset": "0",
-				}
-				self.parse_reg(attrs, 32)
-			self.current_array = self.current_array.parent
-		elif name == "enum":
-			self.current_enum = None
-
-	def character_data(self, data):
-		self.cdata += data
-
-	def dump_reg_usages(self):
-		d = collections.defaultdict(list)
-		for usage, regs in self.usage_regs.items():
-			for reg in regs:
-				variants = self.variant_regs.get(reg.name)
-				if variants:
-					for variant, vreg in variants.items():
-						if reg == vreg:
-							d[(usage, sanitize_variant(variant))].append(reg)
-				else:
-					for variant in self.variants:
-						d[(usage, sanitize_variant(variant))].append(reg)
-
-		print("#ifdef __cplusplus")
-
-		for usage, regs in self.usage_regs.items():
-			print("template<chip CHIP> constexpr inline uint16_t %s_REGS[] = {};" % (usage.upper()))
-
-		for (usage, variant), regs in d.items():
-			offsets = []
-
-			for reg in regs:
-				if reg.array:
-					for i in range(reg.array.length):
-						offsets.append(reg.array.offset + reg.offset + i * reg.array.stride)
-						if reg.bit_size == 64:
-							offsets.append(offsets[-1] + 1)
-				else:
-					offsets.append(reg.offset)
-					if reg.bit_size == 64:
-						offsets.append(offsets[-1] + 1)
-
-			offsets.sort()
-
-			print("template<> constexpr inline uint16_t %s_REGS<%s>[] = {" % (usage.upper(), variant))
-			for offset in offsets:
-				print("\t%s," % hex(offset))
-			print("};")
-
-		print("#endif")
-
-	def has_variants(self, reg):
-		return reg.name in self.variant_regs and not is_number(reg.name) and not is_number(reg.name[1:])
-
-	def dump(self):
-		enums = []
-		bitsets = []
-		regs = []
-		for e in self.file:
-			if isinstance(e, Enum):
-				enums.append(e)
-			elif isinstance(e, Bitset):
-				bitsets.append(e)
-			else:
-				regs.append(e)
-
-		for e in enums + bitsets + regs:
-			e.dump(self.has_variants(e))
-
-		self.dump_reg_usages()
-
-
-	def dump_regs_py(self):
-		regs = []
-		for e in self.file:
-			if isinstance(e, Reg):
-				regs.append(e)
-
-		for e in regs:
-			e.dump_py()
-
-
-	def dump_reg_variants(self, regname, variants):
-		if is_number(regname) or is_number(regname[1:]):
-			return
-		print("#ifdef __cplusplus")
-		print("struct __%s {" % regname)
-		# TODO be more clever.. we should probably figure out which
-		# fields have the same type in all variants (in which they
-		# appear) and stuff everything else in a variant specific
-		# sub-structure.
-		seen_fields = []
-		bit_size = 32
-		array = False
-		address = None
-		for variant in variants.keys():
-			print("    /* %s fields: */" % variant)
-			reg = variants[variant]
-			bit_size = reg.bit_size
-			array = reg.array
-			for f in reg.bitset.fields:
-				fld_name = field_name(reg, f)
-				if fld_name in seen_fields:
-					continue
-				seen_fields.append(fld_name)
-				name = fld_name.lower()
-				if f.type in [ "address", "waddress" ]:
-					if address:
-						continue
-					address = f
-					tab_to("    __bo_type", "bo;")
-					tab_to("    uint32_t", "bo_offset;")
-					continue
-				type, val = f.ctype("var")
-				tab_to("    %s" %type, "%s;" %name)
-		print("    /* fallback fields: */")
-		if bit_size == 64:
-			tab_to("    uint64_t", "unknown;")
-			tab_to("    uint64_t", "qword;")
-		else:
-			tab_to("    uint32_t", "unknown;")
-			tab_to("    uint32_t", "dword;")
-		print("};")
-		# TODO don't hardcode the varset enum name
-		varenum = "chip"
-		print("template <%s %s>" % (varenum, varenum.upper()))
-		print("static inline struct fd_reg_pair")
-		xtra = ""
-		xtravar = ""
-		if array:
-			xtra = "int __i, "
-			xtravar = "__i, "
-		print("__%s(%sstruct __%s fields) {" % (regname, xtra, regname))
-		for variant in variants.keys():
-			if "-" in variant:
-				start = variant[:variant.index("-")]
-				end = variant[variant.index("-") + 1:]
-				if end != "":
-					print("  if ((%s >= %s) && (%s <= %s)) {" % (varenum.upper(), start, varenum.upper(), end))
-				else:
-					print("  if (%s >= %s) {" % (varenum.upper(), start))
-			else:
-				print("  if (%s == %s) {" % (varenum.upper(), variant))
-			reg = variants[variant]
-			reg.dump_regpair_builder()
-			print("  } else")
-		print("    assert(!\"invalid variant\");")
-		print("  return (struct fd_reg_pair){};")
-		print("}")
-
-		if bit_size == 64:
-			skip = ", { .reg = 0 }"
-		else:
-			skip = ""
-
-		print("#define %s(VARIANT, %s...) __%s<VARIANT>(%s{__VA_ARGS__})%s" % (regname, xtravar, regname, xtravar, skip))
-		print("#endif /* __cplusplus */")
-
-	def dump_structs(self):
-		for e in self.file:
-			e.dump_pack_struct(self.has_variants(e))
-
-		for regname in self.variant_regs:
-			self.dump_reg_variants(regname, self.variant_regs[regname])
+    def __init__(self):
+        self.current_array = None
+        self.current_domain = None
+        self.current_prefix = None
+        self.current_prefix_type = None
+        self.current_stripe = None
+        self.current_bitset = None
+        self.current_bitsize = 32
+        # The varset attribute on the domain specifies the enum which
+        # specifies all possible hw variants:
+        self.current_varset = None
+        # Regs that have multiple variants.. we only generated the C++
+        # template based struct-packers for these
+        self.variant_regs = {}
+        # Information in which contexts regs are used, to be used in
+        # debug options
+        self.usage_regs = collections.defaultdict(list)
+        self.bitsets = {}
+        self.enums = {}
+        self.variants = set()
+        self.file = []
+        self.xml_files = []
+
+    def error(self, message):
+        parser, filename = self.stack[-1]
+        return Error("%s:%d:%d: %s" % (filename, parser.CurrentLineNumber, parser.CurrentColumnNumber, message))
+
+    def prefix(self, variant=None):
+        if self.current_prefix_type == "variant" and variant:
+            return sanitize_variant(variant)
+        elif self.current_stripe:
+            return self.current_stripe + "_" + self.current_domain
+        elif self.current_prefix:
+            return self.current_prefix + "_" + self.current_domain
+        else:
+            return self.current_domain
+
+    def parse_field(self, name, attrs):
+        try:
+            if "pos" in attrs:
+                high = low = int(attrs["pos"], 0)
+            elif "high" in attrs and "low" in attrs:
+                high = int(attrs["high"], 0)
+                low = int(attrs["low"], 0)
+            else:
+                low = 0
+                high = self.current_bitsize - 1
+
+            if "type" in attrs:
+                type = attrs["type"]
+            else:
+                type = None
+
+            if "shr" in attrs:
+                shr = int(attrs["shr"], 0)
+            else:
+                shr = 0
+
+            b = Field(name, low, high, shr, type, self)
+
+            if type == "fixed" or type == "ufixed":
+                b.radix = int(attrs["radix"], 0)
+
+            self.current_bitset.fields.append(b)
+        except ValueError as e:
+            raise self.error(e)
+
+    def parse_varset(self, attrs):
+        # Inherit the varset from the enclosing domain if not overriden:
+        varset = self.current_varset
+        if "varset" in attrs:
+            varset = self.enums[attrs["varset"]]
+        return varset
+
+    def parse_variants(self, attrs):
+        if "variants" not in attrs:
+            return None
+
+        variant = attrs["variants"].split(",")[0]
+        varset = self.parse_varset(attrs)
+
+        if "-" in variant:
+            # if we have a range, validate that both the start and end
+            # of the range are valid enums:
+            start = variant[:variant.index("-")]
+            end = variant[variant.index("-") + 1:]
+            assert varset.has_name(start)
+            if end != "":
+                assert varset.has_name(end)
+        else:
+            assert varset.has_name(variant)
+
+        return variant
+
+    def add_all_variants(self, reg, attrs, parent_variant):
+        # TODO this should really handle *all* variants, including dealing
+        # with open ended ranges (ie. "A2XX,A4XX-") (we have the varset
+        # enum now to make that possible)
+        variant = self.parse_variants(attrs)
+        if not variant:
+            variant = parent_variant
+
+        if reg.name not in self.variant_regs:
+            self.variant_regs[reg.name] = {}
+        else:
+            # All variants must be same size:
+            v = next(iter(self.variant_regs[reg.name]))
+            assert self.variant_regs[reg.name][v].bit_size == reg.bit_size
+
+        self.variant_regs[reg.name][variant] = reg
+
+    def add_all_usages(self, reg, usages):
+        if not usages:
+            return
+
+        for usage in usages:
+            self.usage_regs[usage].append(reg)
+
+        self.variants.add(reg.domain)
+
+    def do_validate(self, schemafile):
+        if not self.validate:
+            return
+
+        try:
+            from lxml import etree
+
+            parser, filename = self.stack[-1]
+            dirname = os.path.dirname(filename)
+
+            # we expect this to look like <namespace url> schema.xsd.. I think
+            # technically it is supposed to be just a URL, but that doesn't
+            # quite match up to what we do.. Just skip over everything up to
+            # and including the first whitespace character:
+            schemafile = schemafile[schemafile.rindex(" ")+1:]
+
+            # this is a bit cheezy, but the xml file to validate could be
+            # in a child director, ie. we don't really know where the schema
+            # file is, the way the rnn C code does.  So if it doesn't exist
+            # just look one level up
+            if not os.path.exists(dirname + "/" + schemafile):
+                schemafile = "../" + schemafile
+
+            if not os.path.exists(dirname + "/" + schemafile):
+                raise self.error("Cannot find schema for: " + filename)
+
+            xmlschema_doc = etree.parse(dirname + "/" + schemafile)
+            xmlschema = etree.XMLSchema(xmlschema_doc)
+
+            xml_doc = etree.parse(filename)
+            if not xmlschema.validate(xml_doc):
+                error_str = str(xmlschema.error_log.filter_from_errors()[0])
+                raise self.error(
+                    "Schema validation failed for: " + filename + "\n" + error_str)
+        except ImportError as e:
+            print("lxml not found, skipping validation", file=sys.stderr)
+
+    def do_parse(self, filename):
+        filepath = os.path.abspath(filename)
+        if filepath in self.xml_files:
+            return
+        self.xml_files.append(filepath)
+        file = open(filename, "rb")
+        parser = xml.parsers.expat.ParserCreate()
+        self.stack.append((parser, filename))
+        parser.StartElementHandler = self.start_element
+        parser.EndElementHandler = self.end_element
+        parser.CharacterDataHandler = self.character_data
+        parser.buffer_text = True
+        parser.ParseFile(file)
+        self.stack.pop()
+        file.close()
+
+    def parse(self, rnn_path, filename, validate):
+        self.path = rnn_path
+        self.stack = []
+        self.validate = validate
+        self.do_parse(filename)
+
+    def parse_reg(self, attrs, bit_size):
+        self.current_bitsize = bit_size
+        if "type" in attrs and attrs["type"] in self.bitsets:
+            bitset = self.bitsets[attrs["type"]]
+            if bitset.inline:
+                self.current_bitset = Bitset(attrs["name"], bitset)
+                self.current_bitset.inline = True
+            else:
+                self.current_bitset = bitset
+        else:
+            self.current_bitset = Bitset(attrs["name"], None)
+            self.current_bitset.inline = True
+            if "type" in attrs:
+                self.parse_field(None, attrs)
+
+        variant = self.parse_variants(attrs)
+        if not variant and self.current_array:
+            variant = self.current_array.variant
+
+        self.current_reg = Reg(attrs, self.prefix(
+            variant), self.current_array, bit_size)
+        self.current_reg.bitset = self.current_bitset
+        self.current_bitset.reg = self.current_reg
+
+        if len(self.stack) == 1:
+            self.file.append(self.current_reg)
+
+        if variant is not None:
+            self.add_all_variants(self.current_reg, attrs, variant)
+
+        usages = None
+        if "usage" in attrs:
+            usages = attrs["usage"].split(',')
+        elif self.current_array:
+            usages = self.current_array.usages
+
+        self.add_all_usages(self.current_reg, usages)
+
+    def start_element(self, name, attrs):
+        self.cdata = ""
+        if name == "import":
+            filename = attrs["file"]
+            self.do_parse(os.path.join(self.path, filename))
+        elif name == "domain":
+            self.current_domain = attrs["name"]
+            if "prefix" in attrs:
+                self.current_prefix = sanitize_variant(
+                    self.parse_variants(attrs))
+                self.current_prefix_type = attrs["prefix"]
+            else:
+                self.current_prefix = None
+                self.current_prefix_type = None
+            if "varset" in attrs:
+                self.current_varset = self.enums[attrs["varset"]]
+        elif name == "stripe":
+            self.current_stripe = sanitize_variant(self.parse_variants(attrs))
+        elif name == "enum":
+            self.current_enum_value = 0
+            self.current_enum = Enum(attrs["name"])
+            self.enums[attrs["name"]] = self.current_enum
+            if len(self.stack) == 1:
+                self.file.append(self.current_enum)
+        elif name == "value":
+            if "value" in attrs:
+                value = int(attrs["value"], 0)
+            else:
+                value = self.current_enum_value
+            self.current_enum.values.append((attrs["name"], value))
+        elif name == "reg32":
+            self.parse_reg(attrs, 32)
+        elif name == "reg64":
+            self.parse_reg(attrs, 64)
+        elif name == "array":
+            self.current_bitsize = 32
+            variant = self.parse_variants(attrs)
+            index_type = self.enums[attrs["index"]
+                                    ] if "index" in attrs else None
+            self.current_array = Array(attrs, self.prefix(
+                variant), variant, self.current_array, index_type)
+            if len(self.stack) == 1:
+                self.file.append(self.current_array)
+        elif name == "bitset":
+            self.current_bitset = Bitset(attrs["name"], None)
+            if "inline" in attrs and attrs["inline"] == "yes":
+                self.current_bitset.inline = True
+            self.bitsets[self.current_bitset.name] = self.current_bitset
+            if len(self.stack) == 1 and not self.current_bitset.inline:
+                self.file.append(self.current_bitset)
+        elif name == "bitfield" and self.current_bitset:
+            self.parse_field(attrs["name"], attrs)
+        elif name == "database":
+            self.do_validate(attrs["xsi:schemaLocation"])
+
+    def end_element(self, name):
+        if name == "domain":
+            self.current_domain = None
+            self.current_prefix = None
+            self.current_prefix_type = None
+        elif name == "stripe":
+            self.current_stripe = None
+        elif name == "bitset":
+            self.current_bitset = None
+        elif name == "reg32":
+            self.current_reg = None
+        elif name == "array":
+            # if the array has no Reg children, push an implicit reg32:
+            if len(self.current_array.children) == 0:
+                attrs = {
+                    "name": "REG",
+                    "offset": "0",
+                }
+                self.parse_reg(attrs, 32)
+            self.current_array = self.current_array.parent
+        elif name == "enum":
+            self.current_enum = None
+
+    def character_data(self, data):
+        self.cdata += data
+
+    def dump_reg_usages(self):
+        d = collections.defaultdict(list)
+        for usage, regs in self.usage_regs.items():
+            for reg in regs:
+                variants = self.variant_regs.get(reg.name)
+                if variants:
+                    for variant, vreg in variants.items():
+                        if reg == vreg:
+                            d[(usage, sanitize_variant(variant))].append(reg)
+                else:
+                    for variant in self.variants:
+                        d[(usage, sanitize_variant(variant))].append(reg)
+
+        print("#ifdef __cplusplus")
+
+        for usage, regs in self.usage_regs.items():
+            print("template<chip CHIP> constexpr inline uint16_t %s_REGS[] = {};" % (
+                usage.upper()))
+
+        for (usage, variant), regs in d.items():
+            offsets = []
+
+            for reg in regs:
+                if reg.array:
+                    for i in range(reg.array.length):
+                        offsets.append(reg.array.offset +
+                                       reg.offset + i * reg.array.stride)
+                        if reg.bit_size == 64:
+                            offsets.append(offsets[-1] + 1)
+                else:
+                    offsets.append(reg.offset)
+                    if reg.bit_size == 64:
+                        offsets.append(offsets[-1] + 1)
+
+            offsets.sort()
+
+            print("template<> constexpr inline uint16_t %s_REGS<%s>[] = {" % (
+                usage.upper(), variant))
+            for offset in offsets:
+                print("\t%s," % hex(offset))
+            print("};")
+
+        print("#endif")
+
+    def has_variants(self, reg):
+        return reg.name in self.variant_regs and not is_number(reg.name) and not is_number(reg.name[1:])
+
+    def dump(self):
+        enums = []
+        bitsets = []
+        regs = []
+        for e in self.file:
+            if isinstance(e, Enum):
+                enums.append(e)
+            elif isinstance(e, Bitset):
+                bitsets.append(e)
+            else:
+                regs.append(e)
+
+        for e in enums + bitsets + regs:
+            e.dump(self.has_variants(e))
+
+        self.dump_reg_usages()
+
+    def dump_regs_py(self):
+        regs = []
+        for e in self.file:
+            if isinstance(e, Reg):
+                regs.append(e)
+
+        for e in regs:
+            e.dump_py()
+
+    def dump_reg_variants(self, regname, variants):
+        if is_number(regname) or is_number(regname[1:]):
+            return
+        print("#ifdef __cplusplus")
+        print("struct __%s {" % regname)
+        # TODO be more clever.. we should probably figure out which
+        # fields have the same type in all variants (in which they
+        # appear) and stuff everything else in a variant specific
+        # sub-structure.
+        seen_fields = []
+        bit_size = 32
+        array = False
+        address = None
+        constexpr_mark = " CONSTEXPR"
+        for variant in variants.keys():
+            print("    /* %s fields: */" % variant)
+            reg = variants[variant]
+            bit_size = reg.bit_size
+            array = reg.array
+            for f in reg.bitset.fields:
+                fld_name = field_name(reg, f)
+                if fld_name in seen_fields:
+                    continue
+                seen_fields.append(fld_name)
+                name = fld_name.lower()
+                if f.type in ["address", "waddress"]:
+                    if address:
+                        continue
+                    address = f
+                    print("#ifndef TU_CS_H")
+                    tab_to("    __bo_type", "bo;")
+                    tab_to("    uint32_t", "bo_offset;")
+                    print("#endif")
+                    continue
+                type, val = f.ctype("var")
+                tab_to("    %s" % type, "%s;" % name)
+                if f.type == "float":
+                    constexpr_mark = ""
+        print("    /* fallback fields: */")
+        if bit_size == 64:
+            tab_to("    uint64_t", "unknown;")
+            tab_to("    uint64_t", "qword;")
+        else:
+            tab_to("    uint32_t", "unknown;")
+            tab_to("    uint32_t", "dword;")
+        print("};")
+        # TODO don't hardcode the varset enum name
+        varenum = "chip"
+        print("template <%s %s>" % (varenum, varenum.upper()))
+        print("static%s inline struct fd_reg_pair" % (constexpr_mark))
+        xtra = ""
+        xtravar = ""
+        if array:
+            xtra = "int __i, "
+            xtravar = "__i, "
+        print("__%s(%sstruct __%s fields) {" % (regname, xtra, regname))
+        for variant in variants.keys():
+            if "-" in variant:
+                start = variant[:variant.index("-")]
+                end = variant[variant.index("-") + 1:]
+                if end != "":
+                    print("  if ((%s >= %s) && (%s <= %s)) {" % (
+                        varenum.upper(), start, varenum.upper(), end))
+                else:
+                    print("  if (%s >= %s) {" % (varenum.upper(), start))
+            else:
+                print("  if (%s == %s) {" % (varenum.upper(), variant))
+            reg = variants[variant]
+            reg.dump_regpair_builder()
+            print("  } else")
+        print("    assert(!\"invalid variant\");")
+        print("  return (struct fd_reg_pair){};")
+        print("}")
+
+        if bit_size == 64:
+            skip = ", { .reg = 0 }"
+        else:
+            skip = ""
+
+        print("#define %s(VARIANT, %s...) __%s<VARIANT>(%s{__VA_ARGS__})%s" % (
+            regname, xtravar, regname, xtravar, skip))
+        print("#endif /* __cplusplus */")
+
+    def dump_structs(self):
+        for e in self.file:
+            e.dump_pack_struct(self.has_variants(e))
+
+        for regname in self.variant_regs:
+            self.dump_reg_variants(regname, self.variant_regs[regname])
 
 
 def dump_c(args, guard, func):
-	p = Parser()
-
-	try:
-		p.parse(args.rnn, args.xml, args.validate)
-	except Error as e:
-		print(e, file=sys.stderr)
-		exit(1)
-
-	print("#ifndef %s\n#define %s\n" % (guard, guard))
-
-	print("/* Autogenerated file, DO NOT EDIT manually! */")
-
-	print()
-	print("#ifdef __KERNEL__")
-	print("#include <linux/bug.h>")
-	print("#define assert(x) BUG_ON(!(x))")
-	print("#else")
-	print("#include <assert.h>")
-	print("#endif")
-	print()
-
-	print("#ifdef __cplusplus")
-	print("#define __struct_cast(X)")
-	print("#else")
-	print("#define __struct_cast(X) (struct X)")
-	print("#endif")
-	print()
-
-	print("#ifndef FD_NO_DEPRECATED_PACK")
-	print("#define FD_DEPRECATED __attribute__((deprecated))")
-	print("#else")
-	print("#define FD_DEPRECATED")
-	print("#endif")
-	print()
-
-	func(p)
-
-	print()
-	print("#undef FD_DEPRECATED")
-	print()
-
-	print("#endif /* %s */" % guard)
+    p = Parser()
+
+    try:
+        p.parse(args.rnn, args.xml, args.validate)
+    except Error as e:
+        print(e, file=sys.stderr)
+        exit(1)
+
+    print("#ifndef %s\n#define %s\n" % (guard, guard))
+
+    print("/* Autogenerated file, DO NOT EDIT manually! */")
+
+    print()
+    print("#ifdef __KERNEL__")
+    print("#include <linux/bug.h>")
+    print("#define assert(x) BUG_ON(!(x))")
+    print("#else")
+    print("#include <assert.h>")
+    print("#endif")
+    print()
+
+    print("#ifdef __cplusplus")
+    print("#define __struct_cast(X)")
+    print("#define CONSTEXPR constexpr")
+    print("#else")
+    print("#define __struct_cast(X) (struct X)")
+    print("#define CONSTEXPR")
+    print("#endif")
+    print()
+
+    # TODO figure out what to do about fd_reg_stomp_allowed()
+    # vs gcc.. for now only enable the warnings with clang:
+    print("#if defined(__clang__) && !defined(FD_NO_DEPRECATED_PACK)")
+    print("#define __FD_DEPRECATED _Pragma (\"GCC warning \\\"Deprecated reg builder\\\"\")")
+    print("#else")
+    print("#define __FD_DEPRECATED")
+    print("#endif")
+    print()
+
+    func(p)
+
+    print("#endif /* %s */" % guard)
 
 
 def dump_c_defines(args):
-	guard = str.replace(os.path.basename(args.xml), '.', '_').upper()
-	dump_c(args, guard, lambda p: p.dump())
+    guard = str.replace(os.path.basename(args.xml), '.', '_').upper()
+    dump_c(args, guard, lambda p: p.dump())
 
 
 def dump_c_pack_structs(args):
-	guard = str.replace(os.path.basename(args.xml), '.', '_').upper() + '_STRUCTS'
-	dump_c(args, guard, lambda p: p.dump_structs())
-
+    guard = str.replace(os.path.basename(args.xml),
+                        '.', '_').upper() + '_STRUCTS'
+    dump_c(args, guard, lambda p: p.dump_structs())
+
+
+def dump_perfcntrs(args):
+    p = Parser()
+
+    try:
+        p.parse(args.rnn, args.xml, args.validate)
+    except Error as e:
+        print(e, file=sys.stderr)
+        exit(1)
+
+    perfcntrs = json.load(open(args.json, "r", encoding="utf-8"))
+
+    chip_type = p.enums['chip']
+    chip = perfcntrs['chip']
+    if not chip_type.has_name(chip):
+        raise Error("Invalid chip: " + chip)
+
+    groups = perfcntrs['groups']
+
+    guard = "__" + chip + "_PERFCNTRS_"
+    print("#ifndef %s\n#define %s\n" % (guard, guard))
+    print("/* Autogenerated file, DO NOT EDIT manually! */")
+    print()
+    print("#ifdef __KERNEL__")
+    print("#include \"msm_perfcntr.h\"")
+    print("#endif")
+    print()
+
+    def has_variant(variant):
+        if variant is None:
+            return True
+        if "-" in variant:
+            start = chip_type.value(variant[:variant.index("-")])
+            end = chip_type.value(variant[variant.index("-") + 1:])
+            chipn = chip_type.value(chip)
+
+            return (start is None or chipn >= start) and (end is None or chipn <= end)
+        return chip == variant
+
+    # Split out arrays and regs for later access:
+    arrays = {}
+    regs = {}
+    for e in p.file:
+        if isinstance(e, Array) and has_variant(e.variant):
+            arrays[e.local_name] = e
+        if isinstance(e, Reg):
+            regs[e.name] = e
+
+    # For variant regs, overwrite 'regs' entries with correct variant:
+    for regname in p.variant_regs:
+        for (variant, reg) in p.variant_regs[regname].items():
+            if has_variant(variant):
+                regs[regname] = reg
+                break
+
+    for group in groups:
+        name = group['name']
+        name_low = name.lower()
+        num = group['num']
+        countable_type_name = group['countable_type']
+
+        if not countable_type_name in p.enums:
+            raise Error("Invalid type: " + countable_type_name)
+
+        countable_type = p.enums[countable_type_name]
+
+        print("#ifndef __KERNEL__")
+        print("static const struct fd_perfcntr_countable " + name_low + "_countables[] = {")
+        for (name, value) in countable_type.values:
+            # if the countable is prefixed with the chip, strip that:
+            # (note: avoid py3.9 dependency for kernel)
+            if name.startswith(chip + "_"):
+                name = name[len(chip)+1:]
+            print("   { \"" + name + "\", " + str(value) + " },")
+        print("};")
+        print("#endif")
+
+        print("static const struct fd_perfcntr_counter " + name_low + "_counters[] = {")
+        for i in range(0, num):
+            if "reserved" in group and i in group["reserved"]:
+                continue
+            def get_reg(name):
+                # if reg has {} pattern, expand that first:
+                name = name.format(i)
+
+                if name in arrays:
+                    arr = arrays[name]
+                    return arr.offset + (i * arr.stride)
+
+                if not name in regs:
+                    raise Error("Invalid reg: " + name)
+
+                reg = regs[name]
+                return reg.offset
+
+            def get_counter():
+                # if the counter is <reg64> just a single "counter" value
+                # should be specified in the json, but for legacy separate
+                # hi/lo <reg32> pairs "counter_lo" and "counter_hi" should
+                # be specified
+                if "counter" in group:
+                    counter = get_reg(group["counter"])
+                    return [counter, counter+1]
+                counter_lo = get_reg(group["counter_lo"])
+                counter_hi = get_reg(group["counter_hi"])
+                return [counter_lo, counter_hi]
+
+            (counter_lo, counter_hi) = get_counter()
+            select = get_reg(group['select'])
+
+            select_offset = 0
+            if "select_offset" in group:
+                select_offset = int(group["select_offset"])
+                select = select + select_offset
+
+            slice_select_str = ""
+            if "slice_select" in group:
+                slice_select = group["slice_select"]
+                for reg in slice_select:
+                    val = get_reg(reg) + select_offset
+                    slice_select_str += "0x%04x, " % val
+
+            # TODO add support for things that need enable/clear regs
+
+            print("   { 0x%04x, {%s}, 0x%04x, 0x%04x }," % (select, slice_select_str, counter_lo, counter_hi))
+        print("};")
+
+        print()
+
+    print("const struct fd_perfcntr_group " + chip.lower() + "_perfcntr_groups[] = {")
+    for group in groups:
+        name = group['name']
+        name_low = name.lower()
+        pipe = 'NONE'
+        if 'pipe' in group:
+            pipe = group['pipe']
+
+        print("   GROUP(\"%s\", PIPE_%s, %s_counters, %s_countables)," % (name, pipe, name_low, name_low))
+
+    print("};")
+    print("const unsigned " + chip.lower() + "_num_perfcntr_groups = ARRAY_SIZE(" + chip.lower() + "_perfcntr_groups);")
+
+    print()
+    print("#endif /* %s */" % guard)
 
 def dump_py_defines(args):
-	p = Parser()
+    p = Parser()
 
-	try:
-		p.parse(args.rnn, args.xml, args.validate)
-	except Error as e:
-		print(e, file=sys.stderr)
-		exit(1)
+    try:
+        p.parse(args.rnn, args.xml, args.validate)
+    except Error as e:
+        print(e, file=sys.stderr)
+        exit(1)
 
-	file_name = os.path.splitext(os.path.basename(args.xml))[0]
+    file_name = os.path.splitext(os.path.basename(args.xml))[0]
 
-	print("from enum import IntEnum")
-	print("class %sRegs(IntEnum):" % file_name.upper())
+    print("from enum import IntEnum")
+    print("class %sRegs(IntEnum):" % file_name.upper())
 
-	os.path.basename(args.xml)
+    os.path.basename(args.xml)
 
-	p.dump_regs_py()
+    p.dump_regs_py()
 
 
 def main():
-	parser = argparse.ArgumentParser()
-	parser.add_argument('--rnn', type=str, required=True)
-	parser.add_argument('--xml', type=str, required=True)
-	parser.add_argument('--validate', default=False, action='store_true')
-	parser.add_argument('--no-validate', dest='validate', action='store_false')
+    parser = argparse.ArgumentParser()
+    parser.add_argument('--rnn', type=str, required=True)
+    parser.add_argument('--xml', type=str, required=True)
+    parser.add_argument('--validate', default=False, action='store_true')
+    parser.add_argument('--no-validate', dest='validate', action='store_false')
+
+    subparsers = parser.add_subparsers()
+    subparsers.required = True
 
-	subparsers = parser.add_subparsers()
-	subparsers.required = True
+    parser_c_defines = subparsers.add_parser('c-defines')
+    parser_c_defines.set_defaults(func=dump_c_defines)
 
-	parser_c_defines = subparsers.add_parser('c-defines')
-	parser_c_defines.set_defaults(func=dump_c_defines)
+    parser_c_pack_structs = subparsers.add_parser('c-pack-structs')
+    parser_c_pack_structs.set_defaults(func=dump_c_pack_structs)
 
-	parser_c_pack_structs = subparsers.add_parser('c-pack-structs')
-	parser_c_pack_structs.set_defaults(func=dump_c_pack_structs)
+    parser_perfcntrs = subparsers.add_parser('perfcntrs')
+    parser_perfcntrs.add_argument('--json', type=str, required=True)
+    parser_perfcntrs.set_defaults(func=dump_perfcntrs)
 
-	parser_py_defines = subparsers.add_parser('py-defines')
-	parser_py_defines.set_defaults(func=dump_py_defines)
+    parser_py_defines = subparsers.add_parser('py-defines')
+    parser_py_defines.set_defaults(func=dump_py_defines)
 
-	args = parser.parse_args()
-	args.func(args)
+    args = parser.parse_args()
+    args.func(args)
 
 
 if __name__ == '__main__':
-	main()
+    main()
Signature · 1 line
2.54.0
author maintainer
#5 lore raw

PATCH v5 5/16 drm/msm/registers: Add perfcntr json

In reply to Rob Clark

Pull in perfcntr json and wire up generation of perfcntr tables.

Sync from mesa commit d2c4653ee953 ("freedreno/registers: Add gen8 perfcntrs")

---
 drivers/gpu/drm/msm/Makefile                  |  25 +-
 drivers/gpu/drm/msm/msm_perfcntr.h            |  48 ++++
 .../msm/registers/adreno/a2xx_perfcntrs.json  | 109 ++++++++
 .../msm/registers/adreno/a5xx_perfcntrs.json  | 128 ++++++++++
 .../msm/registers/adreno/a6xx_perfcntrs.json  | 105 ++++++++
 .../msm/registers/adreno/a7xx_perfcntrs.json  | 228 +++++++++++++++++
 .../msm/registers/adreno/a8xx_perfcntrs.json  | 240 ++++++++++++++++++
 7 files changed, 882 insertions(+), 1 deletion(-)
 create mode 100644 drivers/gpu/drm/msm/msm_perfcntr.h
 create mode 100644 drivers/gpu/drm/msm/registers/adreno/a2xx_perfcntrs.json
 create mode 100644 drivers/gpu/drm/msm/registers/adreno/a5xx_perfcntrs.json
 create mode 100644 drivers/gpu/drm/msm/registers/adreno/a6xx_perfcntrs.json
 create mode 100644 drivers/gpu/drm/msm/registers/adreno/a7xx_perfcntrs.json
 create mode 100644 drivers/gpu/drm/msm/registers/adreno/a8xx_perfcntrs.json
drivers/gpu/drm/msm/Makefile +24 -1
index ce00cfb0a875..337634e7e247 100644
@@ -176,6 +176,11 @@ quiet_cmd_headergen = GENHDR  $@
       cmd_headergen = mkdir -p $(obj)/generated && $(PYTHON3) $(src)/registers/gen_header.py \
 		      $(headergen-opts) --rnn $(src)/registers --xml $< c-defines > $@
 
+# TODO how to do this for a2xx/a5xx which have different .xml arg?
+quiet_cmd_headergen_json = GENHDRJSN  $@
+      cmd_headergen_json = mkdir -p $(obj)/generated && $(PYTHON3) $(src)/registers/gen_header.py \
+		      $(headergen-opts) --rnn $(src)/registers --xml $(filter %.xml,$^) perfcntrs --json $< > $@
+
 $(obj)/generated/%.xml.h: $(src)/registers/adreno/%.xml \
 		$(src)/registers/adreno/adreno_common.xml \
 		$(src)/registers/adreno/adreno_pm4.xml \
@@ -192,6 +197,24 @@ $(obj)/generated/%.xml.h: $(src)/registers/display/%.xml \
 		FORCE
 	$(call if_changed,headergen)
 
+ADRENO_PERFCNTRS =
+
+define adreno_perfcntrs
+ADRENO_PERFCNTRS += generated/$(1)_perfcntrs.json.c
+$$(obj)/generated/$(1)_perfcntrs.json.c: $$(src)/registers/adreno/$(1)_perfcntrs.json \
+		$$(src)/registers/adreno/$(2).xml \
+		FORCE
+	$$(call if_changed,headergen_json)
+endef
+
+$(eval $(call adreno_perfcntrs,a2xx,a2xx))
+$(eval $(call adreno_perfcntrs,a5xx,a5xx))
+$(eval $(call adreno_perfcntrs,a6xx,a6xx))
+$(eval $(call adreno_perfcntrs,a7xx,a6xx))
+$(eval $(call adreno_perfcntrs,a8xx,a6xx))
+
+adreno-y += $(ADRENO_PERFCNTRS:.c=.o)
+
 ADRENO_HEADERS = \
 	generated/a2xx.xml.h \
 	generated/a3xx.xml.h \
@@ -226,4 +249,4 @@ DISPLAY_HEADERS = \
 $(addprefix $(obj)/,$(adreno-y)): $(addprefix $(obj)/,$(ADRENO_HEADERS))
 $(addprefix $(obj)/,$(msm-display-y)): $(addprefix $(obj)/,$(DISPLAY_HEADERS))
 
-targets += $(ADRENO_HEADERS) $(DISPLAY_HEADERS)
+targets += $(ADRENO_HEADERS) $(DISPLAY_HEADERS) $(ADRENO_PERFCNTRS)
drivers/gpu/drm/msm/msm_perfcntr.h new file +48 -0

48 lines not on this page · raw message

drivers/gpu/drm/msm/registers/adreno/a2xx_perfcntrs.json new file +109 -0

109 lines not on this page · raw message

drivers/gpu/drm/msm/registers/adreno/a5xx_perfcntrs.json new file +128 -0

128 lines not on this page · raw message

drivers/gpu/drm/msm/registers/adreno/a6xx_perfcntrs.json new file +105 -0

105 lines not on this page · raw message

drivers/gpu/drm/msm/registers/adreno/a7xx_perfcntrs.json new file +228 -0

228 lines not on this page · raw message

drivers/gpu/drm/msm/registers/adreno/a8xx_perfcntrs.json new file +240 -0

240 lines not on this page · raw message

Signature · 1 line
2.54.0
author maintainer
#6 lore raw

PATCH v5 6/16 drm/msm: Add a6xx+ perfcntr tables

In reply to Rob Clark

Wire up the generated perfcntr tables for a6xx+. The PERFCNTR_CONFIG ioctl will use this information to assign counters.

---
 drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 15 +++++++++++++++
 drivers/gpu/drm/msm/msm_gpu.h         |  4 ++++
 drivers/gpu/drm/msm/msm_perfcntr.h    |  9 +++++++++
 3 files changed, 28 insertions(+)
drivers/gpu/drm/msm/adreno/a6xx_gpu.c +15 -0
index e578417a4949..727281fbef36 100644
@@ -5,6 +5,7 @@
 #include "msm_gem.h"
 #include "msm_mmu.h"
 #include "msm_gpu_trace.h"
+#include "msm_perfcntr.h"
 #include "a6xx_gpu.h"
 #include "a6xx_gmu.xml.h"
 
@@ -2637,6 +2638,20 @@ static struct msm_gpu *a6xx_gpu_init(struct drm_device *dev)
 	adreno_gpu = &a6xx_gpu->base;
 	gpu = &adreno_gpu->base;
 
+	if ((ADRENO_6XX_GEN1 <= config->info->family) &&
+	    (config->info->family <= ADRENO_6XX_GEN4)) {
+		gpu->perfcntr_groups = a6xx_perfcntr_groups;
+		gpu->num_perfcntr_groups = a6xx_num_perfcntr_groups;
+	} else if ((ADRENO_7XX_GEN1 <= config->info->family) &&
+		   (config->info->family <= ADRENO_7XX_GEN3)) {
+		gpu->perfcntr_groups = a7xx_perfcntr_groups;
+		gpu->num_perfcntr_groups = a7xx_num_perfcntr_groups;
+	} else if ((ADRENO_8XX_GEN1 <= config->info->family) &&
+		   (config->info->family <= ADRENO_8XX_GEN2)) {
+		gpu->perfcntr_groups = a8xx_perfcntr_groups;
+		gpu->num_perfcntr_groups = a8xx_num_perfcntr_groups;
+	}
+
 	mutex_init(&a6xx_gpu->gmu.lock);
 	spin_lock_init(&a6xx_gpu->aperture_lock);
 
drivers/gpu/drm/msm/msm_gpu.h +4 -0
index 78e1478669be..8c08dc065372 100644
@@ -24,6 +24,7 @@ struct msm_gem_submit;
 struct msm_gem_vm_log_entry;
 struct msm_gpu_state;
 struct msm_context;
+struct msm_perfcntr_group;
 
 struct msm_gpu_config {
 	const char *ioname;
@@ -262,6 +263,9 @@ struct msm_gpu {
 	bool allow_relocs;
 
 	struct thermal_cooling_device *cooling;
+
+	const struct msm_perfcntr_group *perfcntr_groups;
+	unsigned num_perfcntr_groups;
 };
 
 static inline struct msm_gpu *dev_to_gpu(struct device *dev)
drivers/gpu/drm/msm/msm_perfcntr.h +9 -0
index 305dcde15c5e..64a5d29feba1 100644
@@ -35,6 +35,15 @@ struct msm_perfcntr_group {
    const struct msm_perfcntr_counter *counters;
 };
 
+extern const struct msm_perfcntr_group a6xx_perfcntr_groups[];
+extern const unsigned a6xx_num_perfcntr_groups;
+
+extern const struct msm_perfcntr_group a7xx_perfcntr_groups[];
+extern const unsigned a7xx_num_perfcntr_groups;
+
+extern const struct msm_perfcntr_group a8xx_perfcntr_groups[];
+extern const unsigned a8xx_num_perfcntr_groups;
+
 #define GROUP(_name, _pipe, _counters, _countables) {                          \
       .name = _name,                                                           \
       .pipe = _pipe,                                                           \
Signature · 1 line
2.54.0
Reviewed-by on v5 6/16
#7 lore raw

Re: [PATCH v5 06/16] drm/msm: Add a6xx+ perfcntr tables

Quoted 10 lines · Rob Clark

Wire up the generated perfcntr tables for a6xx+. The PERFCNTR_CONFIG ioctl will use this information to assign counters.

---
 drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 15 +++++++++++++++
 drivers/gpu/drm/msm/msm_gpu.h         |  4 ++++
 drivers/gpu/drm/msm/msm_perfcntr.h    |  9 +++++++++
 3 files changed, 28 insertions(+)
Signature · 2 lines
With best wishes
Dmitry
author maintainer
#8 lore raw

PATCH v5 7/16 drm/msm: Add sysprof accessors

In reply to Rob Clark

Currently the sysprof param serves two functions, (a) disabling perfcntr clearing on context switch/preemption, and (b) disabling IFPC. In the future, with kernel side global perfcntr collection/stream, the decision about disabling IFPC will change.

To prepare for this, split out two helpers/accessors for the two different cases. For now, they are the same thing, but this will change.

---
 drivers/gpu/drm/msm/adreno/a6xx_gmu.c     |  8 +++-----
 drivers/gpu/drm/msm/adreno/a6xx_gpu.c     |  5 +++--
 drivers/gpu/drm/msm/adreno/a6xx_preempt.c |  2 +-
 drivers/gpu/drm/msm/adreno/a8xx_gpu.c     |  2 +-
 drivers/gpu/drm/msm/adreno/a8xx_preempt.c |  2 +-
 drivers/gpu/drm/msm/msm_gpu.h             | 18 ++++++++++++++++++
 6 files changed, 27 insertions(+), 10 deletions(-)
drivers/gpu/drm/msm/adreno/a6xx_gmu.c +3 -5
index 1b44b9e21ad8..aba08fb76249 100644
@@ -2036,10 +2036,10 @@ static int a6xx_gmu_get_irq(struct a6xx_gmu *gmu, struct platform_device *pdev,
 
 void a6xx_gmu_sysprof_setup(struct msm_gpu *gpu)
 {
+	bool sysprof = msm_gpu_sysprof_no_ifpc(gpu);
 	struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu);
 	struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu);
 	struct a6xx_gmu *gmu = &a6xx_gpu->gmu;
-	unsigned int sysprof_active;
 
 	/* Nothing to do if GPU is suspended. We will handle this during GMU resume */
 	if (!pm_runtime_get_if_active(&gpu->pdev->dev))
@@ -2047,15 +2047,13 @@ void a6xx_gmu_sysprof_setup(struct msm_gpu *gpu)
 
 	mutex_lock(&gmu->lock);
 
-	sysprof_active = refcount_read(&gpu->sysprof_active);
-
 	/*
 	 * 'Perfcounter select' register values are lost during IFPC collapse. To avoid that,
 	 * use the currently unused perfcounter oob vote to block IFPC when sysprof is active
 	 */
-	if ((sysprof_active > 1) && !test_and_set_bit(GMU_STATUS_OOB_PERF_SET, &gmu->status))
+	if (sysprof && !test_and_set_bit(GMU_STATUS_OOB_PERF_SET, &gmu->status))
 		a6xx_gmu_set_oob(gmu, GMU_OOB_PERFCOUNTER_SET);
-	else if ((sysprof_active == 1) && test_and_clear_bit(GMU_STATUS_OOB_PERF_SET, &gmu->status))
+	else if (!sysprof && test_and_clear_bit(GMU_STATUS_OOB_PERF_SET, &gmu->status))
 		a6xx_gmu_clear_oob(gmu, GMU_OOB_PERFCOUNTER_SET);
 
 	mutex_unlock(&gmu->lock);
drivers/gpu/drm/msm/adreno/a6xx_gpu.c +3 -2
index 727281fbef36..71f54ab5425d 100644
@@ -203,7 +203,7 @@ static void get_stats_counter(struct msm_ringbuffer *ring, u32 counter,
 static void a6xx_set_pagetable(struct a6xx_gpu *a6xx_gpu,
 		struct msm_ringbuffer *ring, struct msm_gem_submit *submit)
 {
-	bool sysprof = refcount_read(&a6xx_gpu->base.base.sysprof_active) > 1;
+	bool sysprof = msm_gpu_sysprof_no_perfcntr_zap(&a6xx_gpu->base.base);
 	struct msm_context *ctx = submit->queue->ctx;
 	struct drm_gpuvm *vm = msm_context_vm(submit->dev, ctx);
 	struct adreno_gpu *adreno_gpu = &a6xx_gpu->base;
@@ -1608,7 +1608,7 @@ static int hw_init(struct msm_gpu *gpu)
 		a6xx_gmu_clear_oob(&a6xx_gpu->gmu, GMU_OOB_BOOT_SLUMBER);
 	}
 
-	if (!ret && (refcount_read(&gpu->sysprof_active) > 1)) {
+	if (!ret && msm_gpu_sysprof_no_ifpc(gpu)) {
 		ret = a6xx_gmu_set_oob(gmu, GMU_OOB_PERFCOUNTER_SET);
 		if (!ret)
 			set_bit(GMU_STATUS_OOB_PERF_SET, &gmu->status);
@@ -2854,6 +2854,7 @@ const struct adreno_gpu_funcs a8xx_gpu_funcs = {
 		.create_private_vm = a6xx_create_private_vm,
 		.get_rptr = a6xx_get_rptr,
 		.progress = a8xx_progress,
+		.sysprof_setup = a6xx_gmu_sysprof_setup,
 	},
 	.init = a6xx_gpu_init,
 	.get_timestamp = a8xx_gmu_get_timestamp,
drivers/gpu/drm/msm/adreno/a6xx_preempt.c +1 -1
index df4cbf42e9a4..1e599d4ddea1 100644
@@ -261,7 +261,7 @@ void a6xx_preempt_trigger(struct msm_gpu *gpu)
 	mod_timer(&a6xx_gpu->preempt_timer, jiffies + msecs_to_jiffies(10000));
 
 	/* Enable or disable postamble as needed */
-	sysprof = refcount_read(&a6xx_gpu->base.base.sysprof_active) > 1;
+	sysprof = msm_gpu_sysprof_no_perfcntr_zap(gpu);
 
 	if (!sysprof && !a6xx_gpu->postamble_enabled)
 		preempt_prepare_postamble(a6xx_gpu);
drivers/gpu/drm/msm/adreno/a8xx_gpu.c +1 -1
index ccfccc45133f..e022c9a162a4 100644
@@ -849,7 +849,7 @@ static int hw_init(struct msm_gpu *gpu)
 	 */
 	a6xx_gmu_clear_oob(&a6xx_gpu->gmu, GMU_OOB_GPU_SET);
 
-	if (!ret && (refcount_read(&gpu->sysprof_active) > 1)) {
+	if (!ret && msm_gpu_sysprof_no_perfcntr_zap(gpu)) {
 		ret = a6xx_gmu_set_oob(gmu, GMU_OOB_PERFCOUNTER_SET);
 		if (!ret)
 			set_bit(GMU_STATUS_OOB_PERF_SET, &gmu->status);
drivers/gpu/drm/msm/adreno/a8xx_preempt.c +1 -1
index 3d8c33ba722e..6cb53a071801 100644
@@ -242,7 +242,7 @@ void a8xx_preempt_trigger(struct msm_gpu *gpu)
 	mod_timer(&a6xx_gpu->preempt_timer, jiffies + msecs_to_jiffies(10000));
 
 	/* Enable or disable postamble as needed */
-	sysprof = refcount_read(&a6xx_gpu->base.base.sysprof_active) > 1;
+	sysprof = msm_gpu_sysprof_no_perfcntr_zap(gpu);
 
 	if (!sysprof && !a6xx_gpu->postamble_enabled)
 		preempt_prepare_postamble(a6xx_gpu);
drivers/gpu/drm/msm/msm_gpu.h +18 -0
index 8c08dc065372..9e5c753437c2 100644
@@ -311,6 +311,24 @@ static inline bool msm_gpu_active(struct msm_gpu *gpu)
 	return false;
 }
 
+static inline bool
+msm_gpu_sysprof_no_perfcntr_zap(struct msm_gpu *gpu)
+{
+	return refcount_read(&gpu->sysprof_active) > 1;
+}
+
+static inline bool
+msm_gpu_sysprof_no_ifpc(struct msm_gpu *gpu)
+{
+	/*
+	 * For now, this is the same condition as disabling perfcntr clears
+	 * on context switch.  But once kernel perfcntr IFPC support is in
+	 * place, we will only need to disable IFPC for legacy userspace
+	 * setting SYSPROF param.
+	 */
+	return msm_gpu_sysprof_no_perfcntr_zap(gpu);
+}
+
 /*
  * The number of priority levels provided by drm gpu scheduler.  The
  * DRM_SCHED_PRIORITY_KERNEL priority level is treated specially in some
Signature · 1 line
2.54.0
author maintainer
#9 lore raw

PATCH v5 8/16 drm/msm/a6xx: Add yield & flush helper

In reply to Rob Clark

It's a common pattern, needing to insert a yield packet before flushing the rb. And we'll need this once again for configuring perfcntr SEL regs. So add a helper.

---
 drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 55 +++++++++++++--------------
 drivers/gpu/drm/msm/adreno/a6xx_gpu.h |  1 +
 drivers/gpu/drm/msm/adreno/a8xx_gpu.c | 10 +----
 3 files changed, 28 insertions(+), 38 deletions(-)
drivers/gpu/drm/msm/adreno/a6xx_gpu.c +26 -29
index 71f54ab5425d..415902f6e5d7 100644
@@ -189,6 +189,30 @@ void a6xx_flush(struct msm_gpu *gpu, struct msm_ringbuffer *ring)
 	spin_unlock_irqrestore(&ring->preempt_lock, flags);
 }
 
+void
+a6xx_flush_yield(struct msm_gpu *gpu, struct msm_ringbuffer *ring)
+{
+	/* If preemption is enabled */
+	if (gpu->nr_rings > 1) {
+		/* Yield the floor on command completion */
+		OUT_PKT7(ring, CP_CONTEXT_SWITCH_YIELD, 4);
+
+		/*
+		 * If dword[2:1] are non zero, they specify an address for
+		 * the CP to write the value of dword[3] to on preemption
+		 * complete. Write 0 to skip the write
+		 */
+		OUT_RING(ring, 0x00);
+		OUT_RING(ring, 0x00);
+		/* Data value - not used if the address above is 0 */
+		OUT_RING(ring, 0x01);
+		/* generate interrupt on preemption completion */
+		OUT_RING(ring, 0x00);
+	}
+
+	a6xx_flush(gpu, ring);
+}
+
 static void get_stats_counter(struct msm_ringbuffer *ring, u32 counter,
 		u64 iova)
 {
@@ -597,28 +621,9 @@ static void a7xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit)
 	OUT_PKT7(ring, CP_SET_MARKER, 1);
 	OUT_RING(ring, 0x100); /* IFPC enable */
 
-	/* If preemption is enabled */
-	if (gpu->nr_rings > 1) {
-		/* Yield the floor on command completion */
-		OUT_PKT7(ring, CP_CONTEXT_SWITCH_YIELD, 4);
-
-		/*
-		 * If dword[2:1] are non zero, they specify an address for
-		 * the CP to write the value of dword[3] to on preemption
-		 * complete. Write 0 to skip the write
-		 */
-		OUT_RING(ring, 0x00);
-		OUT_RING(ring, 0x00);
-		/* Data value - not used if the address above is 0 */
-		OUT_RING(ring, 0x01);
-		/* generate interrupt on preemption completion */
-		OUT_RING(ring, 0x00);
-	}
-
-
 	trace_msm_gpu_submit_flush(submit, adreno_gpu->funcs->get_timestamp(gpu));
 
-	a6xx_flush(gpu, ring);
+	a6xx_flush_yield(gpu, ring);
 
 	/* Check to see if we need to start preemption */
 	if (adreno_is_a8xx(adreno_gpu))
@@ -958,15 +963,7 @@ static int a7xx_preempt_start(struct msm_gpu *gpu)
 
 	a6xx_emit_set_pseudo_reg(ring, a6xx_gpu, NULL);
 
-	/* Yield the floor on command completion */
-	OUT_PKT7(ring, CP_CONTEXT_SWITCH_YIELD, 4);
-	OUT_RING(ring, 0x00);
-	OUT_RING(ring, 0x00);
-	OUT_RING(ring, 0x00);
-	/* Generate interrupt on preemption completion */
-	OUT_RING(ring, 0x00);
-
-	a6xx_flush(gpu, ring);
+	a6xx_flush_yield(gpu, ring);
 
 	return a6xx_idle(gpu, ring) ? 0 : -EINVAL;
 }
drivers/gpu/drm/msm/adreno/a6xx_gpu.h +1 -0
index eb431e5e00b1..99c3e55f5ca8 100644
@@ -317,6 +317,7 @@ void a6xx_bus_clear_pending_transactions(struct adreno_gpu *adreno_gpu, bool gx_
 void a6xx_gpu_sw_reset(struct msm_gpu *gpu, bool assert);
 int a6xx_fenced_write(struct a6xx_gpu *gpu, u32 offset, u64 value, u32 mask, bool is_64b);
 void a6xx_flush(struct msm_gpu *gpu, struct msm_ringbuffer *ring);
+void a6xx_flush_yield(struct msm_gpu *gpu, struct msm_ringbuffer *ring);
 int a6xx_zap_shader_init(struct msm_gpu *gpu);
 
 void a8xx_bus_clear_pending_transactions(struct adreno_gpu *adreno_gpu, bool gx_off);
drivers/gpu/drm/msm/adreno/a8xx_gpu.c +1 -9
index e022c9a162a4..124d315b2469 100644
@@ -488,15 +488,7 @@ static int a8xx_preempt_start(struct msm_gpu *gpu)
 
 	a6xx_emit_set_pseudo_reg(ring, a6xx_gpu, NULL);
 
-	/* Yield the floor on command completion */
-	OUT_PKT7(ring, CP_CONTEXT_SWITCH_YIELD, 4);
-	OUT_RING(ring, 0x00);
-	OUT_RING(ring, 0x00);
-	OUT_RING(ring, 0x00);
-	/* Generate interrupt on preemption completion */
-	OUT_RING(ring, 0x00);
-
-	a6xx_flush(gpu, ring);
+	a6xx_flush_yield(gpu, ring);
 
 	return a8xx_idle(gpu, ring) ? 0 : -EINVAL;
 }
Signature · 1 line
2.54.0
Reviewed-by on v5 8/16
#10 lore raw

Re: [PATCH v5 08/16] drm/msm/a6xx: Add yield & flush helper

Quoted 11 lines · Rob Clark

It's a common pattern, needing to insert a yield packet before flushing the rb. And we'll need this once again for configuring perfcntr SEL regs. So add a helper.

---
 drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 55 +++++++++++++--------------
 drivers/gpu/drm/msm/adreno/a6xx_gpu.h |  1 +
 drivers/gpu/drm/msm/adreno/a8xx_gpu.c | 10 +----
 3 files changed, 28 insertions(+), 38 deletions(-)
Signature · 2 lines
With best wishes
Dmitry
author maintainer
#11 lore raw

PATCH v5 9/16 drm/msm: Add per-context perfcntr state

In reply to Rob Clark

The upcoming PERFCNTR_CONFIG ioctl will allow for both global counter collection, and per-context counter reservation for local (ie. within a single GEM_SUBMIT ioctl) counter collection.

Any number of contexts can reserve the same counters, but we will need to ensure that counters reserved for local counter collection do not conflict with counters used for global counter collection.

So add tracking for per-context local counter reservations.

---
 drivers/gpu/drm/msm/msm_gpu.h         |  5 +++++
 drivers/gpu/drm/msm/msm_perfcntr.h    | 21 +++++++++++++++++++++
 drivers/gpu/drm/msm/msm_submitqueue.c |  1 +
 3 files changed, 27 insertions(+)
drivers/gpu/drm/msm/msm_gpu.h +5 -0
index 9e5c753437c2..19484774f369 100644
@@ -434,6 +434,11 @@ struct msm_context {
 	 * this context.
 	 */
 	atomic64_t ctx_mem;
+
+	/**
+	 * @perfcntrs: Per-context reserved perfcntrs state
+	 */
+	struct msm_perfcntr_context_state *perfctx;
 };
 
 struct drm_gpuvm *msm_context_vm(struct drm_device *dev, struct msm_context *ctx);
drivers/gpu/drm/msm/msm_perfcntr.h +21 -0
index 64a5d29feba1..7f0654182496 100644
@@ -35,6 +35,27 @@ struct msm_perfcntr_group {
    const struct msm_perfcntr_counter *counters;
 };
 
+/**
+ * struct msm_perfcntr_context_state - per-msm_context counter state
+ *
+ * A given counter can either be unused, reserved for global counter
+ * collection exclusively, or reserved for local per-context counter
+ * collection inclusively.  Multiple contexts can reserve the same
+ * counter, since SEL reg programming and counter begin/end sampling
+ * happen locally (within a single GEM_SUBMIT ioctl).
+ */
+struct msm_perfcntr_context_state {
+	/** @dummy: Some compilers dislike structs with only a flex array */
+	unsigned dummy;
+
+	/**
+	 * @reserved_counters:
+	 *
+	 * The number of reserved counters indexed by perfcntr group.
+	 */
+	unsigned reserved_counters[];
+};
+
 extern const struct msm_perfcntr_group a6xx_perfcntr_groups[];
 extern const unsigned a6xx_num_perfcntr_groups;
 
drivers/gpu/drm/msm/msm_submitqueue.c +1 -0
index 2598d674a99d..a58fe41602c6 100644
@@ -66,6 +66,7 @@ void __msm_context_destroy(struct kref *kref)
 	drm_gpuvm_put(ctx->vm);
 	kfree(ctx->comm);
 	kfree(ctx->cmdline);
+	kfree(ctx->perfctx);
 	kfree(ctx);
 }
 
Signature · 1 line
2.54.0
Reviewed-by on v5 9/16
#12 lore raw

Re: [PATCH v5 09/16] drm/msm: Add per-context perfcntr state

Quoted 8 lines · Rob Clark

The upcoming PERFCNTR_CONFIG ioctl will allow for both global counter collection, and per-context counter reservation for local (ie. within a single GEM_SUBMIT ioctl) counter collection.

Any number of contexts can reserve the same counters, but we will need to ensure that counters reserved for local counter collection do not conflict with counters used for global counter collection.

Quoted 9 lines

So add tracking for per-context local counter reservations.

---
 drivers/gpu/drm/msm/msm_gpu.h         |  5 +++++
 drivers/gpu/drm/msm/msm_perfcntr.h    | 21 +++++++++++++++++++++
 drivers/gpu/drm/msm/msm_submitqueue.c |  1 +
 3 files changed, 27 insertions(+)
Signature · 2 lines
With best wishes
Dmitry
author maintainer
#13 lore raw

PATCH v5 10/16 drm/msm: Add basic perfcntr infrastructure

In reply to Rob Clark

Add the basic infrastructure for tracking assigned perfcntrs.

---
 drivers/gpu/drm/msm/Makefile               |   1 +
 drivers/gpu/drm/msm/adreno/adreno_device.c |   8 +-
 drivers/gpu/drm/msm/adreno/adreno_gpu.c    |   5 +-
 drivers/gpu/drm/msm/msm_drv.h              |   6 +
 drivers/gpu/drm/msm/msm_gpu.c              |  12 ++
 drivers/gpu/drm/msm/msm_gpu.h              |  57 +++++++++-
 drivers/gpu/drm/msm/msm_perfcntr.c         | 124 +++++++++++++++++++++
 drivers/gpu/drm/msm/msm_perfcntr.h         |  23 ++++
 8 files changed, 230 insertions(+), 6 deletions(-)
 create mode 100644 drivers/gpu/drm/msm/msm_perfcntr.c
drivers/gpu/drm/msm/Makefile +1 -0
index 337634e7e247..2466cb32dac5 100644
@@ -122,6 +122,7 @@ msm-y += \
 	msm_gpu_devfreq.o \
 	msm_io_utils.o \
 	msm_iommu.o \
+	msm_perfcntr.o \
 	msm_rd.o \
 	msm_ringbuffer.o \
 	msm_submitqueue.o \
drivers/gpu/drm/msm/adreno/adreno_device.c +6 -2
index fc38331ce640..7f20320ef66a 100644
@@ -307,8 +307,10 @@ MODULE_DEVICE_TABLE(of, dt_match);
 static int adreno_runtime_resume(struct device *dev)
 {
 	struct msm_gpu *gpu = dev_to_gpu(dev);
-
-	return gpu->funcs->pm_resume(gpu);
+	int ret = gpu->funcs->pm_resume(gpu);
+	if (!ret)
+		ret = msm_perfcntr_resume(gpu);
+	return ret;
 }
 
 static int adreno_runtime_suspend(struct device *dev)
@@ -322,6 +324,8 @@ static int adreno_runtime_suspend(struct device *dev)
 	 */
 	WARN_ON_ONCE(gpu->active_submits);
 
+	msm_perfcntr_suspend(gpu);
+
 	return gpu->funcs->pm_suspend(gpu);
 }
 
drivers/gpu/drm/msm/adreno/adreno_gpu.c +2 -3
index 72b71e9e44f0..ee0bcf985934 100644
@@ -702,11 +702,10 @@ void adreno_recover(struct msm_gpu *gpu)
 	struct drm_device *dev = gpu->dev;
 	int ret;
 
-	// XXX pm-runtime??  we *need* the device to be off after this
-	// so maybe continuing to call ->pm_suspend/resume() is better?
-
+	msm_perfcntr_suspend(gpu);
 	gpu->funcs->pm_suspend(gpu);
 	gpu->funcs->pm_resume(gpu);
+	msm_perfcntr_resume(gpu);
 
 	ret = msm_gpu_hw_init(gpu);
 	if (ret) {
drivers/gpu/drm/msm/msm_drv.h +6 -0
index e53e4f220bed..f00b2e7aeb91 100644
@@ -235,6 +235,12 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
 int msm_ioctl_vm_bind(struct drm_device *dev, void *data,
 		      struct drm_file *file);
 
+int msm_perfcntr_resume(struct msm_gpu *gpu);
+void msm_perfcntr_suspend(struct msm_gpu *gpu);
+
+struct msm_perfcntr_state * msm_perfcntr_init(struct msm_gpu *gpu);
+void msm_perfcntr_cleanup(struct msm_gpu *gpu);
+
 #ifdef CONFIG_DEBUG_FS
 unsigned long msm_gem_shrinker_shrink(struct drm_device *dev, unsigned long nr_to_scan);
 #endif
drivers/gpu/drm/msm/msm_gpu.c +12 -0
index 1bac70473f80..bf6845e5719e 100644
@@ -1028,6 +1028,17 @@ int msm_gpu_init(struct drm_device *drm, struct platform_device *pdev,
 
 	refcount_set(&gpu->sysprof_active, 1);
 
+	mutex_init(&gpu->perfcntr_lock);
+
+	if (gpu->num_perfcntr_groups > 0) {
+		gpu->perfcntrs = msm_perfcntr_init(gpu);
+		if (IS_ERR(gpu->perfcntrs)) {
+			ret = PTR_ERR(gpu->perfcntrs);
+			gpu->perfcntrs = NULL;
+			goto fail;
+		}
+	}
+
 	return 0;
 
 fail:
@@ -1066,6 +1077,7 @@ void msm_gpu_cleanup(struct msm_gpu *gpu)
 	}
 
 	msm_devfreq_cleanup(gpu);
+	msm_perfcntr_cleanup(gpu);
 
 	platform_set_drvdata(gpu->pdev, NULL);
 }
drivers/gpu/drm/msm/msm_gpu.h +56 -1

84 lines not on this page · raw message

drivers/gpu/drm/msm/msm_perfcntr.c new file +124 -0

124 lines not on this page · raw message

drivers/gpu/drm/msm/msm_perfcntr.h +23 -0
index 7f0654182496..bfda19e01535 100644
@@ -35,6 +35,29 @@ struct msm_perfcntr_group {
    const struct msm_perfcntr_counter *counters;
 };
 
+/**
+ * struct msm_perfcntr_stream - state for a single open stream fd
+ */
+struct msm_perfcntr_stream {
+	/** @gpu: Back-link to the GPU */
+	struct msm_gpu *gpu;
+
+	/** @nr_groups: # of counter groups with enabled counters */
+	uint32_t nr_groups;
+
+	/**
+	 * @group_idx: array of nr_groups
+	 *
+	 * Maps the order of groups in PERFCNTR_CONFIG ioctl to group idx,
+	 * so that results in the results stream can be ordered to match
+	 * the ioctl call that setup the stream
+	 */
+	uint32_t *group_idx;
+};
+
+uint32_t msm_perfcntr_group_idx(const struct msm_perfcntr_stream *stream, uint32_t n);
+uint32_t msm_perfcntr_counter_base(const struct msm_perfcntr_stream *stream, uint32_t group_idx);
+
 /**
  * struct msm_perfcntr_context_state - per-msm_context counter state
  *
Signature · 1 line
2.54.0
Reviewed-by on v5 10/16
#14 lore raw

Re: [PATCH v5 10/16] drm/msm: Add basic perfcntr infrastructure

Quoted 275 lines · Rob Clark

Add the basic infrastructure for tracking assigned perfcntrs.

---
 drivers/gpu/drm/msm/Makefile               |   1 +
 drivers/gpu/drm/msm/adreno/adreno_device.c |   8 +-
 drivers/gpu/drm/msm/adreno/adreno_gpu.c    |   5 +-
 drivers/gpu/drm/msm/msm_drv.h              |   6 +
 drivers/gpu/drm/msm/msm_gpu.c              |  12 ++
 drivers/gpu/drm/msm/msm_gpu.h              |  57 +++++++++-
 drivers/gpu/drm/msm/msm_perfcntr.c         | 124 +++++++++++++++++++++
 drivers/gpu/drm/msm/msm_perfcntr.h         |  23 ++++
 8 files changed, 230 insertions(+), 6 deletions(-)
 create mode 100644 drivers/gpu/drm/msm/msm_perfcntr.c
drivers/gpu/drm/msm/Makefile +1 -0
index 337634e7e247..2466cb32dac5 100644
@@ -122,6 +122,7 @@ msm-y += \
 	msm_gpu_devfreq.o \
 	msm_io_utils.o \
 	msm_iommu.o \
+	msm_perfcntr.o \
 	msm_rd.o \
 	msm_ringbuffer.o \
 	msm_submitqueue.o \
drivers/gpu/drm/msm/adreno/adreno_device.c +6 -2
index fc38331ce640..7f20320ef66a 100644
@@ -307,8 +307,10 @@ MODULE_DEVICE_TABLE(of, dt_match);
 static int adreno_runtime_resume(struct device *dev)
 {
 	struct msm_gpu *gpu = dev_to_gpu(dev);
-
-	return gpu->funcs->pm_resume(gpu);
+	int ret = gpu->funcs->pm_resume(gpu);
+	if (!ret)
+		ret = msm_perfcntr_resume(gpu);
+	return ret;
 }
 
 static int adreno_runtime_suspend(struct device *dev)
@@ -322,6 +324,8 @@ static int adreno_runtime_suspend(struct device *dev)
 	 */
 	WARN_ON_ONCE(gpu->active_submits);
 
+	msm_perfcntr_suspend(gpu);
+
 	return gpu->funcs->pm_suspend(gpu);
 }
 
drivers/gpu/drm/msm/adreno/adreno_gpu.c +2 -3
index 72b71e9e44f0..ee0bcf985934 100644
@@ -702,11 +702,10 @@ void adreno_recover(struct msm_gpu *gpu)
 	struct drm_device *dev = gpu->dev;
 	int ret;
 
-	// XXX pm-runtime??  we *need* the device to be off after this
-	// so maybe continuing to call ->pm_suspend/resume() is better?
-
+	msm_perfcntr_suspend(gpu);
 	gpu->funcs->pm_suspend(gpu);
 	gpu->funcs->pm_resume(gpu);
+	msm_perfcntr_resume(gpu);
 
 	ret = msm_gpu_hw_init(gpu);
 	if (ret) {
drivers/gpu/drm/msm/msm_drv.h +6 -0
index e53e4f220bed..f00b2e7aeb91 100644
@@ -235,6 +235,12 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
 int msm_ioctl_vm_bind(struct drm_device *dev, void *data,
 		      struct drm_file *file);
 
+int msm_perfcntr_resume(struct msm_gpu *gpu);
+void msm_perfcntr_suspend(struct msm_gpu *gpu);
+
+struct msm_perfcntr_state * msm_perfcntr_init(struct msm_gpu *gpu);
+void msm_perfcntr_cleanup(struct msm_gpu *gpu);
+
 #ifdef CONFIG_DEBUG_FS
 unsigned long msm_gem_shrinker_shrink(struct drm_device *dev, unsigned long nr_to_scan);
 #endif
drivers/gpu/drm/msm/msm_gpu.c +12 -0
index 1bac70473f80..bf6845e5719e 100644
@@ -1028,6 +1028,17 @@ int msm_gpu_init(struct drm_device *drm, struct platform_device *pdev,
 
 	refcount_set(&gpu->sysprof_active, 1);
 
+	mutex_init(&gpu->perfcntr_lock);
+
+	if (gpu->num_perfcntr_groups > 0) {
+		gpu->perfcntrs = msm_perfcntr_init(gpu);
+		if (IS_ERR(gpu->perfcntrs)) {
+			ret = PTR_ERR(gpu->perfcntrs);
+			gpu->perfcntrs = NULL;
+			goto fail;
+		}
+	}
+
 	return 0;
 
 fail:
@@ -1066,6 +1077,7 @@ void msm_gpu_cleanup(struct msm_gpu *gpu)
 	}
 
 	msm_devfreq_cleanup(gpu);
+	msm_perfcntr_cleanup(gpu);
 
 	platform_set_drvdata(gpu->pdev, NULL);
 }
drivers/gpu/drm/msm/msm_gpu.h +56 -1
index 19484774f369..be922641a14f 100644
@@ -25,6 +25,7 @@ struct msm_gem_vm_log_entry;
 struct msm_gpu_state;
 struct msm_context;
 struct msm_perfcntr_group;
+struct msm_perfcntr_stream;
 
 struct msm_gpu_config {
 	const char *ioname;
@@ -93,6 +94,13 @@ struct msm_gpu_funcs {
 	 */
 	bool (*progress)(struct msm_gpu *gpu, struct msm_ringbuffer *ring);
 	void (*sysprof_setup)(struct msm_gpu *gpu);
+
+	/* Configure perfcntr SELect regs: */
+	void (*perfcntr_configure)(struct msm_gpu *gpu, struct msm_ringbuffer *ring,
+				   const struct msm_perfcntr_stream *stream);
+
+	/* Flush perfcntrs before reading (optional): */
+	void (*perfcntr_flush)(struct msm_gpu *gpu);
 };
 
 /* Additional state for iommu faults: */
@@ -266,6 +274,11 @@ struct msm_gpu {
 
 	const struct msm_perfcntr_group *perfcntr_groups;
 	unsigned num_perfcntr_groups;
+
+	struct msm_perfcntr_state *perfcntrs;
+
+	/** @perfcntr_lock: protects perfcntr related state */
+	struct mutex perfcntr_lock;
 };
 
 static inline struct msm_gpu *dev_to_gpu(struct device *dev)
@@ -311,10 +324,52 @@ static inline bool msm_gpu_active(struct msm_gpu *gpu)
 	return false;
 }
 
+/**
+ * struct msm_perfcntr_group_state - Tracking for the currently allocated counter state
+ */
+struct msm_perfcntr_group_state {
+	/**
+	 * @allocated_counters:
+	 *
+	 * allocated counters for global counter collection.  The
+	 * corresponding counters are allocated from highest to
+	 * lowest, to minimize chance of conflict with old userspace
+	 * allocating from lowest to highest.
+	 */
+	unsigned allocated_counters;
+
+	/**
+	 * @countables:
+	 *
+	 * The correspnding SELect reg values for the allocated counters
+	 */
+	uint32_t countables[];
+};
+
+/**
+ * struct msm_perfcntr_state - overall global perfcntr state
+ */
+struct msm_perfcntr_state {
+	/** @stream: current global counter stream if active */
+	struct msm_perfcntr_stream *stream;
+
+	/**
+	 * @groups: Global perfcntr stream group state.
+	 *
+	 * Conceptually this is part of msm_perfcntr_stream state, but is
+	 * statically pre-allocated when the gpu is initialized to simplify
+	 * error path cleanup in PERFCNTR_CONFIG ioctl.  (__free(kfree)
+	 * doesn't really help with variable length arrays of allocated
+	 * pointers.)
+	 */
+	struct msm_perfcntr_group_state *groups[];
+};
+
 static inline bool
 msm_gpu_sysprof_no_perfcntr_zap(struct msm_gpu *gpu)
 {
-	return refcount_read(&gpu->sysprof_active) > 1;
+	return (refcount_read(&gpu->sysprof_active) > 1) ||
+		(gpu->perfcntrs && READ_ONCE(gpu->perfcntrs->stream));
 }
 
 static inline bool
drivers/gpu/drm/msm/msm_perfcntr.c new file +59 -0
new file mode 100644
index 000000000000..dad98c96863c
@@ -0,0 +1,124 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) Qualcomm Technologies, Inc. and/or its subsidiaries.
+ */
+
+#include "msm_drv.h"
+#include "msm_gpu.h"
+#include "msm_perfcntr.h"
+
+static int
+msm_perfcntr_resume_locked(struct msm_perfcntr_stream *stream)
+{
+	return 0;
+}
+
+int
+msm_perfcntr_resume(struct msm_gpu *gpu)
+{
+	if (!gpu->perfcntrs)
+		return 0;
+	guard(mutex)(&gpu->perfcntr_lock);
+	return msm_perfcntr_resume_locked(gpu->perfcntrs->stream);
+}
+
+static void
+msm_perfcntr_suspend_locked(struct msm_perfcntr_stream *stream)
+{
+}
+
+void
+msm_perfcntr_suspend(struct msm_gpu *gpu)
+{
+	if (!gpu->perfcntr_groups)
+		return;
+	guard(mutex)(&gpu->perfcntr_lock);
+	msm_perfcntr_suspend_locked(gpu->perfcntrs->stream);
+}
+
+/**
+ * msm_perfcntr_group_idx - map idx of perfcntr group to group_idx
+ * @stream: The global perfcntr stream
+ * @n: The requested group_idx
+ *
+ * The PERFCNTR_CONFIG ioctl requested N counters/countables per perfcntr
+ * group, but the order of groups is not required to match the order they
+ * are defined in the perfcntr tables (which is not stable/UABI, only the
+ * group names are UABI).
+ *
+ * But the order samples are returned in the stream should match the
+ * order they are requested in the PERFCNTR_CONFIG ioctl.  This helper
+ * handles the order remapping.
+ *
+ * Returns an index into gpu->perfcntr_groups[] and perfcntrs->groups[].
+ */
+uint32_t
+msm_perfcntr_group_idx(const struct msm_perfcntr_stream *stream, uint32_t n)
+{
+	WARN_ON_ONCE(n > stream->nr_groups);
+	return stream->group_idx[n];

drivers/gpu/drm/msm/msm_perfcntr.c line 69

Quoted 10 lines
drivers/gpu/drm/msm/msm_perfcntr.c new file +10 -0
+}
+
+/**
+ * msm_perfcntr_counter_base - get idx of the first counter in group
+ * @stream: The global perfcntr stream
+ * @group_idx: the index of the counter group
+ *
+ * For global counter collection, counters are allocated from the end
+ * (last counter) to minimize the chance of conflict with an old UMD
+ * which predates PERFCNTR_CONFIG ioctl (since UMD assigned from 0..N-1).

As is this comment seems to imply that userspace wont't be allocating from the start anymore which isn't the case. May I suggest:
For global counter collection, counters are allocated from the end while UMD
allocates them from the start. Since UMD always allocated them from the start
it also minimizes the chance of conflict when using old UMD which predates
PERFCNTR_CONFIG ioctl.

Aside from this nit series is:

Quoted 88 lines
drivers/gpu/drm/msm/msm_perfcntr.c new file +54 -0
+ * Returns the index of first counter to use.  An index into
+ * msm_perfcntr_group::counters[].
+ */
+uint32_t
+msm_perfcntr_counter_base(const struct msm_perfcntr_stream *stream, uint32_t group_idx)
+{
+	struct msm_gpu *gpu = stream->gpu;
+	struct msm_perfcntr_state *perfcntrs = gpu->perfcntrs;
+	unsigned num_counters = gpu->perfcntr_groups[group_idx].num_counters;
+	unsigned allocated_counters = perfcntrs->groups[group_idx]->allocated_counters;
+
+	return num_counters - allocated_counters;
+}
+
+struct msm_perfcntr_state *
+msm_perfcntr_init(struct msm_gpu *gpu)
+{
+	struct msm_perfcntr_state *perfcntrs;
+	struct device *dev = &gpu->pdev->dev;
+	size_t sz;
+
+	sz = struct_size(perfcntrs, groups, gpu->num_perfcntr_groups);
+	perfcntrs = devm_kzalloc(dev, sz, GFP_KERNEL);
+	if (!perfcntrs)
+		return ERR_PTR(-ENOMEM);
+
+	for (unsigned i = 0; i < gpu->num_perfcntr_groups; i++) {
+		const struct msm_perfcntr_group *group =
+			&gpu->perfcntr_groups[i];
+
+		sz = struct_size(perfcntrs->groups[i], countables, group->num_counters);
+		perfcntrs->groups[i] = devm_kzalloc(dev, sz, GFP_KERNEL);
+		if (!perfcntrs->groups[i]) {
+			msm_perfcntr_cleanup(gpu);
+			return ERR_PTR(-ENOMEM);
+		}
+	}
+
+	return perfcntrs;
+}
+
+void
+msm_perfcntr_cleanup(struct msm_gpu *gpu)
+{
+	struct msm_perfcntr_state *perfcntrs = gpu->perfcntrs;
+	struct device *dev = &gpu->pdev->dev;
+
+	gpu->perfcntrs = NULL;
+
+	for (unsigned i = 0; i < gpu->num_perfcntr_groups; i++)
+		devm_kfree(dev, perfcntrs->groups[i]);
+
+	devm_kfree(dev, perfcntrs);
+}
drivers/gpu/drm/msm/msm_perfcntr.h +23 -0
index 7f0654182496..bfda19e01535 100644
@@ -35,6 +35,29 @@ struct msm_perfcntr_group {
    const struct msm_perfcntr_counter *counters;
 };
 
+/**
+ * struct msm_perfcntr_stream - state for a single open stream fd
+ */
+struct msm_perfcntr_stream {
+	/** @gpu: Back-link to the GPU */
+	struct msm_gpu *gpu;
+
+	/** @nr_groups: # of counter groups with enabled counters */
+	uint32_t nr_groups;
+
+	/**
+	 * @group_idx: array of nr_groups
+	 *
+	 * Maps the order of groups in PERFCNTR_CONFIG ioctl to group idx,
+	 * so that results in the results stream can be ordered to match
+	 * the ioctl call that setup the stream
+	 */
+	uint32_t *group_idx;
+};
+
+uint32_t msm_perfcntr_group_idx(const struct msm_perfcntr_stream *stream, uint32_t n);
+uint32_t msm_perfcntr_counter_base(const struct msm_perfcntr_stream *stream, uint32_t group_idx);
+
 /**
  * struct msm_perfcntr_context_state - per-msm_context counter state
  *
author maintainer
#15 lore raw

PATCH v5 11/16 drm/msm/a6xx+: Add support to configure perfcntrs

In reply to Rob Clark

Add support to configure counter SELect regs. In some cases the reg writes need to happen while the GPU is idle. And for a7xx+, in some cases SEL regs need to be configured from BV or BR aperture. The easiest way to deal with this is to configure from the RB.

---
 drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 69 +++++++++++++++++++++++++++
 drivers/gpu/drm/msm/msm_perfcntr.h    |  3 ++
 drivers/gpu/drm/msm/msm_ringbuffer.h  |  2 +
 3 files changed, 74 insertions(+)
drivers/gpu/drm/msm/adreno/a6xx_gpu.c +69 -0
index 415902f6e5d7..30df9bfa9ef8 100644
@@ -2535,6 +2535,71 @@ static bool a6xx_progress(struct msm_gpu *gpu, struct msm_ringbuffer *ring)
 	return progress;
 }
 
+static void
+a6xx_perfcntr_configure(struct msm_gpu *gpu, struct msm_ringbuffer *ring,
+			const struct msm_perfcntr_stream *stream)
+{
+	enum adreno_pipe pipe = PIPE_NONE;
+
+	for (unsigned i = 0; i < stream->nr_groups; i++) {
+		unsigned group_idx = msm_perfcntr_group_idx(stream, i);
+		unsigned base = msm_perfcntr_counter_base(stream, group_idx);
+
+		const struct msm_perfcntr_group *group =
+			&gpu->perfcntr_groups[group_idx];
+
+		struct msm_perfcntr_group_state *group_state =
+			gpu->perfcntrs->groups[group_idx];
+
+		if (group->pipe != pipe) {
+			pipe = group->pipe;
+
+			OUT_PKT7(ring, CP_THREAD_CONTROL, 1);
+
+			if (pipe == PIPE_BR) {
+				OUT_RING(ring, CP_SET_THREAD_BR);
+			} else if (pipe == PIPE_BV) {
+				OUT_RING(ring, CP_SET_THREAD_BV);
+			} else {
+				OUT_RING(ring, CP_SET_THREAD_BOTH);
+			}
+		}
+
+		const struct msm_perfcntr_counter *counter = &group->counters[base];
+		unsigned nr = group_state->allocated_counters;
+		OUT_PKT4(ring, counter->select_reg, nr);
+		for (unsigned c = 0; c < nr; c++)
+			OUT_RING(ring, group_state->countables[c]);
+
+		for (unsigned s = 0; s < ARRAY_SIZE(counter->slice_select_regs); s++) {
+			if (!counter->slice_select_regs[s])
+				break;
+
+			OUT_PKT4(ring, counter->slice_select_regs[s], nr);
+			for (unsigned c = 0; c < nr; c++)
+				OUT_RING(ring, group_state->countables[c]);
+		}
+	}
+
+	if (pipe != PIPE_NONE) {
+		OUT_PKT7(ring, CP_THREAD_CONTROL, 1);
+		OUT_RING(ring, CP_SET_THREAD_BOTH);
+	}
+
+	OUT_PKT7(ring, CP_MEM_WRITE, 3);
+	OUT_RING(ring, lower_32_bits(rbmemptr(ring, perfcntr_fence)));
+	OUT_RING(ring, upper_32_bits(rbmemptr(ring, perfcntr_fence)));
+	OUT_RING(ring, stream->sel_fence);
+
+	a6xx_flush_yield(gpu, ring);
+
+	/* Check to see if we need to start preemption */
+	if (adreno_is_a8xx(to_adreno_gpu(gpu)))
+		a8xx_preempt_trigger(gpu);
+	else
+		a6xx_preempt_trigger(gpu);
+}
+
 static u32 fuse_to_supp_hw(const struct adreno_info *info, u32 fuse)
 {
 	if (!info->speedbins)
@@ -2753,6 +2818,7 @@ const struct adreno_gpu_funcs a6xx_gpu_funcs = {
 		.get_rptr = a6xx_get_rptr,
 		.progress = a6xx_progress,
 		.sysprof_setup = a6xx_gmu_sysprof_setup,
+		.perfcntr_configure = a6xx_perfcntr_configure,
 	},
 	.init = a6xx_gpu_init,
 	.get_timestamp = a6xx_gmu_get_timestamp,
@@ -2786,6 +2852,7 @@ const struct adreno_gpu_funcs a6xx_gmuwrapper_funcs = {
 		.create_private_vm = a6xx_create_private_vm,
 		.get_rptr = a6xx_get_rptr,
 		.progress = a6xx_progress,
+		.perfcntr_configure = a6xx_perfcntr_configure,
 	},
 	.init = a6xx_gpu_init,
 	.get_timestamp = a6xx_get_timestamp,
@@ -2822,6 +2889,7 @@ const struct adreno_gpu_funcs a7xx_gpu_funcs = {
 		.get_rptr = a6xx_get_rptr,
 		.progress = a6xx_progress,
 		.sysprof_setup = a6xx_gmu_sysprof_setup,
+		.perfcntr_configure = a6xx_perfcntr_configure,
 	},
 	.init = a6xx_gpu_init,
 	.get_timestamp = a6xx_gmu_get_timestamp,
@@ -2852,6 +2920,7 @@ const struct adreno_gpu_funcs a8xx_gpu_funcs = {
 		.get_rptr = a6xx_get_rptr,
 		.progress = a8xx_progress,
 		.sysprof_setup = a6xx_gmu_sysprof_setup,
+		.perfcntr_configure = a6xx_perfcntr_configure,
 	},
 	.init = a6xx_gpu_init,
 	.get_timestamp = a8xx_gmu_get_timestamp,
drivers/gpu/drm/msm/msm_perfcntr.h +3 -0
index bfda19e01535..14506bc37d05 100644
@@ -45,6 +45,9 @@ struct msm_perfcntr_stream {
 	/** @nr_groups: # of counter groups with enabled counters */
 	uint32_t nr_groups;
 
+	/** @sel_fence: Fence for SEL reg programming  */
+	uint32_t sel_fence;
+
 	/**
 	 * @group_idx: array of nr_groups
 	 *
drivers/gpu/drm/msm/msm_ringbuffer.h +2 -0
index d1e49f701c81..28ca8c9f7463 100644
@@ -37,6 +37,8 @@ struct msm_rbmemptrs {
 	volatile struct msm_gpu_submit_stats stats[MSM_GPU_SUBMIT_STATS_COUNT];
 	volatile u64 ttbr0;
 	volatile u32 context_idr;
+
+	volatile u32 perfcntr_fence;
 };
 
 struct msm_cp_state {
Signature · 1 line
2.54.0
author maintainer
#16 lore raw

PATCH v5 12/16 drm/msm/a8xx: Add perfcntr flush sequence

In reply to Rob Clark

With the slice architecture, we need to flush the slice and unslice counters to perf RAM before reading counters.

---
 drivers/gpu/drm/msm/adreno/a6xx_gpu.c |  1 +
 drivers/gpu/drm/msm/adreno/a6xx_gpu.h |  1 +
 drivers/gpu/drm/msm/adreno/a8xx_gpu.c | 20 ++++++++++++++++++++
 3 files changed, 22 insertions(+)
drivers/gpu/drm/msm/adreno/a6xx_gpu.c +1 -0
index 30df9bfa9ef8..a329d20033d7 100644
@@ -2921,6 +2921,7 @@ const struct adreno_gpu_funcs a8xx_gpu_funcs = {
 		.progress = a8xx_progress,
 		.sysprof_setup = a6xx_gmu_sysprof_setup,
 		.perfcntr_configure = a6xx_perfcntr_configure,
+		.perfcntr_flush = a8xx_perfcntr_flush,
 	},
 	.init = a6xx_gpu_init,
 	.get_timestamp = a8xx_gmu_get_timestamp,
drivers/gpu/drm/msm/adreno/a6xx_gpu.h +1 -0
index 99c3e55f5ca8..3491a24a9320 100644
@@ -334,5 +334,6 @@ void a8xx_preempt_hw_init(struct msm_gpu *gpu);
 void a8xx_preempt_trigger(struct msm_gpu *gpu);
 void a8xx_preempt_irq(struct msm_gpu *gpu);
 bool a8xx_progress(struct msm_gpu *gpu, struct msm_ringbuffer *ring);
+void a8xx_perfcntr_flush(struct msm_gpu *gpu);
 void a8xx_recover(struct msm_gpu *gpu);
 #endif /* __A6XX_GPU_H__ */
drivers/gpu/drm/msm/adreno/a8xx_gpu.c +20 -0
index 124d315b2469..6c040f718176 100644
@@ -1345,3 +1345,23 @@ bool a8xx_progress(struct msm_gpu *gpu, struct msm_ringbuffer *ring)
 {
 	return true;
 }
+
+void a8xx_perfcntr_flush(struct msm_gpu *gpu)
+{
+	u32 val;
+
+	/*
+	 * Flush delta counters (both perf counters and pipe stats) present in
+	 * RBBM_S and RBBM_US to perf RAM logic to get the latest data.
+	 */
+	gpu_write(gpu, REG_A8XX_RBBM_PERFCTR_FLUSH_HOST_CMD, BIT(0));
+	gpu_write(gpu, REG_A8XX_RBBM_SLICE_PERFCTR_FLUSH_HOST_CMD, BIT(0));
+
+	/* Ensure all writes are posted before polling status register */
+	wmb();
+
+	if (gpu_poll_timeout(gpu, REG_A8XX_RBBM_PERFCTR_FLUSH_HOST_STATUS, val,
+			     val & BIT(0), 100, 100 * 1000)) {
+		dev_err(&gpu->pdev->dev, "Perfcounter flush timed out: status=0x%08x\n", val);
+	}
+}
Signature · 1 line
2.54.0
author maintainer
#17 lore raw

PATCH v5 13/16 drm/msm: Add PERFCNTR_CONFIG ioctl

In reply to Rob Clark

Add new UABI and implementation of PERFCNTR_CONFIG ioctl.

A bit more work is required to configure the pwrup_reglist for the GMU to restore SELect regs on exist of IFPC, before we can stop disabling IFPC while global counter collection. This will follow in a later commit, but will be transparent to userspace.

---
 drivers/gpu/drm/msm/msm_drv.c      |   1 +
 drivers/gpu/drm/msm/msm_drv.h      |   2 +
 drivers/gpu/drm/msm/msm_gpu.h      |   3 +
 drivers/gpu/drm/msm/msm_perfcntr.c | 516 +++++++++++++++++++++++++++++
 drivers/gpu/drm/msm/msm_perfcntr.h |  51 +++
 include/uapi/drm/msm_drm.h         |  48 +++
 6 files changed, 621 insertions(+)
drivers/gpu/drm/msm/msm_drv.c +1 -0
index 3066547f319b..0a7fc06113e0 100644
@@ -801,6 +801,7 @@ static const struct drm_ioctl_desc msm_ioctls[] = {
 	DRM_IOCTL_DEF_DRV(MSM_SUBMITQUEUE_CLOSE, msm_ioctl_submitqueue_close, DRM_RENDER_ALLOW),
 	DRM_IOCTL_DEF_DRV(MSM_SUBMITQUEUE_QUERY, msm_ioctl_submitqueue_query, DRM_RENDER_ALLOW),
 	DRM_IOCTL_DEF_DRV(MSM_VM_BIND,      msm_ioctl_vm_bind,      DRM_RENDER_ALLOW),
+	DRM_IOCTL_DEF_DRV(MSM_PERFCNTR_CONFIG,   msm_ioctl_perfcntr_config,    DRM_RENDER_ALLOW),
 };
 
 static void msm_show_fdinfo(struct drm_printer *p, struct drm_file *file)
drivers/gpu/drm/msm/msm_drv.h +2 -0
index f00b2e7aeb91..204e140ac8e9 100644
@@ -237,6 +237,8 @@ int msm_ioctl_vm_bind(struct drm_device *dev, void *data,
 
 int msm_perfcntr_resume(struct msm_gpu *gpu);
 void msm_perfcntr_suspend(struct msm_gpu *gpu);
+int msm_ioctl_perfcntr_config(struct drm_device *dev, void *data,
+			     struct drm_file *file);
 
 struct msm_perfcntr_state * msm_perfcntr_init(struct msm_gpu *gpu);
 void msm_perfcntr_cleanup(struct msm_gpu *gpu);
drivers/gpu/drm/msm/msm_gpu.h +3 -0
index be922641a14f..cb74b7606987 100644
@@ -353,6 +353,9 @@ struct msm_perfcntr_state {
 	/** @stream: current global counter stream if active */
 	struct msm_perfcntr_stream *stream;
 
+	/** @sel_seqno: counter for sel_fence */
+	uint32_t sel_seqno;
+
 	/**
 	 * @groups: Global perfcntr stream group state.
 	 *
drivers/gpu/drm/msm/msm_perfcntr.c +516 -0

541 lines not on this page · raw message

drivers/gpu/drm/msm/msm_perfcntr.h +51 -0

75 lines not on this page · raw message

include/uapi/drm/msm_drm.h +48 -0

66 lines not on this page · raw message

Signature · 1 line
2.54.0
author maintainer
#18 lore raw

PATCH v5 14/16 drm/msm/a6xx: Increase pwrup_reglist size

In reply to Rob Clark

To make room for appending SEL reg programming. Without increasing the size, we would overflow the pwrup_reglist at ~190 counters on gen8. Or possibly fewer, considering that some gen8 counter groups also have separate slice vs unslice SELectors.

---
 drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/msm/adreno/a6xx_gpu.c +1 -1
index a329d20033d7..61c6b0e781ce 100644
@@ -1183,7 +1183,7 @@ static int a6xx_ucode_load(struct msm_gpu *gpu)
 		msm_gem_object_set_name(a6xx_gpu->shadow_bo, "shadow");
 	}
 
-	a6xx_gpu->pwrup_reglist_ptr = msm_gem_kernel_new(gpu->dev, PAGE_SIZE,
+	a6xx_gpu->pwrup_reglist_ptr = msm_gem_kernel_new(gpu->dev, 2 * PAGE_SIZE,
 							 MSM_BO_WC  | MSM_BO_MAP_PRIV,
 							 gpu->vm, &a6xx_gpu->pwrup_reglist_bo,
 							 &a6xx_gpu->pwrup_reglist_iova);
Signature · 1 line
2.54.0
author maintainer
#19 lore raw

PATCH v5 15/16 drm/msm/a6xx: Append SEL regs to dyn pwrup reglist

In reply to Rob Clark

This is needed so that SEL reg values are restored on exit from IFPC.

---
 drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 82 +++++++++++++++++++++++++--
 drivers/gpu/drm/msm/adreno/a6xx_gpu.h | 11 +++-
 drivers/gpu/drm/msm/adreno/a8xx_gpu.c |  1 +
 3 files changed, 87 insertions(+), 7 deletions(-)
drivers/gpu/drm/msm/adreno/a6xx_gpu.c +76 -6
index 61c6b0e781ce..47627e76adcf 100644
@@ -946,6 +946,7 @@ static void a7xx_patch_pwrup_reglist(struct msm_gpu *gpu)
 			  A7XX_CP_APERTURE_CNTL_HOST_PIPE(PIPE_NONE));
 	}
 	lock->dynamic_list_len = dyn_pwrup_reglist_count;
+	a6xx_gpu->dynamic_sel_reglist_offset = dyn_pwrup_reglist_count;
 }
 
 static int a7xx_preempt_start(struct msm_gpu *gpu)
@@ -2535,11 +2536,60 @@ static bool a6xx_progress(struct msm_gpu *gpu, struct msm_ringbuffer *ring)
 	return progress;
 }
 
+static void
+perfcntr_select(struct msm_ringbuffer *ring, enum adreno_pipe pipe,
+		uint32_t regidx, uint32_t *countables, uint32_t nr,
+		uint32_t **reglist)
+{
+	OUT_PKT4(ring, regidx, nr);
+	for (unsigned i = 0; i < nr; i++)
+		OUT_RING(ring, countables[i]);
+
+	if (!*reglist)
+		return;
+
+	for (unsigned i = 0; i < nr; i++) {
+		/*
+		 * Bitfield is in same position on a7xx, but only 2 bits..
+		 * which is sufficient for NONE/BR/BV:
+		 */
+		*(*reglist)++ = A8XX_CP_APERTURE_CNTL_HOST_PIPEID(pipe);
+		*(*reglist)++ = regidx + i;
+		*(*reglist)++ = countables[i];
+	}
+}
+
 static void
 a6xx_perfcntr_configure(struct msm_gpu *gpu, struct msm_ringbuffer *ring,
 			const struct msm_perfcntr_stream *stream)
 {
+	struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu);
+	struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu);
 	enum adreno_pipe pipe = PIPE_NONE;
+	uint32_t *reglist = NULL;
+	uint32_t *reglist_sel_start;
+
+	if (to_adreno_gpu(gpu)->info->family >= ADRENO_7XX_GEN1) {
+		WARN_ON(!a6xx_gpu->pwrup_reglist_emitted);
+
+		struct cpu_gpu_lock *lock = a6xx_gpu->pwrup_reglist_ptr;
+		int off = (2 * lock->ifpc_list_len) +
+			  (2 * lock->preemption_list_len) +
+			  (3 * a6xx_gpu->dynamic_sel_reglist_offset);
+
+		reglist = (uint32_t *)&lock->regs[0];
+		reglist += off;
+		reglist_sel_start = reglist;
+
+		/* Clear any previously configured SEL reg entries: */
+		lock->dynamic_list_len = a6xx_gpu->dynamic_sel_reglist_offset;
+
+		/*
+		 * Ensure CP sees the dynamic_list_len update before we
+		 * start modifying the SEL entries:
+		 */
+		wmb();
+	}
 
 	for (unsigned i = 0; i < stream->nr_groups; i++) {
 		unsigned group_idx = msm_perfcntr_group_idx(stream, i);
@@ -2567,17 +2617,15 @@ a6xx_perfcntr_configure(struct msm_gpu *gpu, struct msm_ringbuffer *ring,
 
 		const struct msm_perfcntr_counter *counter = &group->counters[base];
 		unsigned nr = group_state->allocated_counters;
-		OUT_PKT4(ring, counter->select_reg, nr);
-		for (unsigned c = 0; c < nr; c++)
-			OUT_RING(ring, group_state->countables[c]);
+		perfcntr_select(ring, pipe, counter->select_reg,
+				group_state->countables, nr, &reglist);
 
 		for (unsigned s = 0; s < ARRAY_SIZE(counter->slice_select_regs); s++) {
 			if (!counter->slice_select_regs[s])
 				break;
 
-			OUT_PKT4(ring, counter->slice_select_regs[s], nr);
-			for (unsigned c = 0; c < nr; c++)
-				OUT_RING(ring, group_state->countables[c]);
+			perfcntr_select(ring, pipe, counter->slice_select_regs[s],
+					group_state->countables, nr, &reglist);
 		}
 	}
 
@@ -2591,6 +2639,28 @@ a6xx_perfcntr_configure(struct msm_gpu *gpu, struct msm_ringbuffer *ring,
 	OUT_RING(ring, upper_32_bits(rbmemptr(ring, perfcntr_fence)));
 	OUT_RING(ring, stream->sel_fence);
 
+	/*
+	 * Update the pwrup reglist size before flushing.  Kgsl does a shared-
+	 * memory spinlock dance with SQE to avoid racing with IFPC exit.  But
+	 * we can skip that since the ringbuffer programming will be executed
+	 * by SQE after dynamic reglist size is updated.  So even if we lose
+	 * the race, the register programming in the rb will overwrite/correct
+	 * the SEL regs restored by SQE on IFPC exit, before sampling begins.
+	 */
+	if (reglist) {
+		struct cpu_gpu_lock *lock = a6xx_gpu->pwrup_reglist_ptr;
+		unsigned nr_regs = (reglist - reglist_sel_start) / 3;
+
+		/*
+		 * Ensure CP sees updates to the pwrup_reglist before it
+		 * sees the new (increased) length:
+		 */
+		wmb();
+
+		/* Update dynamic reglist len to include new SEL reg programming: */
+		lock->dynamic_list_len = a6xx_gpu->dynamic_sel_reglist_offset + nr_regs;
+	}
+
 	a6xx_flush_yield(gpu, ring);
 
 	/* Check to see if we need to start preemption */
drivers/gpu/drm/msm/adreno/a6xx_gpu.h +10 -1
index 3491a24a9320..f3cc9478b079 100644
@@ -21,17 +21,19 @@ struct cpu_gpu_lock {
 	uint32_t cpu_req;
 	uint32_t turn;
 	union {
+		/* a6xx: */
 		struct {
 			uint16_t list_length;
 			uint16_t list_offset;
 		};
+		/* a7xx+: */
 		struct {
 			uint8_t ifpc_list_len;
 			uint8_t preemption_list_len;
 			uint16_t dynamic_list_len;
 		};
 	};
-	uint64_t regs[62];
+	uint64_t regs[];
 };
 
 /**
@@ -100,6 +102,13 @@ struct a6xx_gpu {
 	uint64_t pwrup_reglist_iova;
 	bool pwrup_reglist_emitted;
 
+	/*
+	 * Offset of start of SEL regs appended to pwrup_reglist.  This
+	 * is equal to lock->dynamic_list_len if no SEL regs are appended
+	 * to the end of the dynamic reglist.
+	 */
+	uint16_t dynamic_sel_reglist_offset;
+
 	bool has_whereami;
 
 	void __iomem *llc_mmio;
drivers/gpu/drm/msm/adreno/a8xx_gpu.c +1 -0
index 6c040f718176..2ce7c6ac4521 100644
@@ -468,6 +468,7 @@ static void a8xx_patch_pwrup_reglist(struct msm_gpu *gpu)
 	}
 
 	lock->dynamic_list_len = dyn_pwrup_reglist_count;
+	a6xx_gpu->dynamic_sel_reglist_offset = dyn_pwrup_reglist_count;
 
 done:
 	a8xx_aperture_clear(gpu);
Signature · 1 line
2.54.0
author maintainer
#20 lore raw

PATCH v5 16/16 drm/msm/a6xx: Allow IFPC with perfcntr stream

In reply to Rob Clark

Now that the dynamic pwrup reglist has SEL reg values to restore appended, so that SEL regs are restored on IFPC exit, we can stop completely disabling IFPC while global counter sampling is active.

To accomplish this, we re-use sysprof_setup() with a force_on param to inhibit IFPC specifically while the counter regs are being read, while leaving IFPC enabled the rest of the time.

---
 drivers/gpu/drm/msm/adreno/a6xx_gmu.c |  4 ++--
 drivers/gpu/drm/msm/adreno/a6xx_gpu.h |  2 +-
 drivers/gpu/drm/msm/msm_gpu.h         | 10 ++--------
 drivers/gpu/drm/msm/msm_perfcntr.c    |  8 ++++++++
 drivers/gpu/drm/msm/msm_submitqueue.c |  2 +-
 5 files changed, 14 insertions(+), 12 deletions(-)
drivers/gpu/drm/msm/adreno/a6xx_gmu.c +2 -2
index aba08fb76249..0a7d49a2c877 100644
@@ -2034,9 +2034,9 @@ static int a6xx_gmu_get_irq(struct a6xx_gmu *gmu, struct platform_device *pdev,
 	return irq;
 }
 
-void a6xx_gmu_sysprof_setup(struct msm_gpu *gpu)
+void a6xx_gmu_sysprof_setup(struct msm_gpu *gpu, bool force_on)
 {
-	bool sysprof = msm_gpu_sysprof_no_ifpc(gpu);
+	bool sysprof = msm_gpu_sysprof_no_ifpc(gpu) || force_on;
 	struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu);
 	struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu);
 	struct a6xx_gmu *gmu = &a6xx_gpu->gmu;
drivers/gpu/drm/msm/adreno/a6xx_gpu.h +1 -1
index f3cc9478b079..eecc71843bed 100644
@@ -280,7 +280,7 @@ void a6xx_gmu_clear_oob(struct a6xx_gmu *gmu, enum a6xx_gmu_oob_state state);
 int a6xx_gmu_init(struct a6xx_gpu *a6xx_gpu, struct device_node *node);
 int a6xx_gmu_wrapper_init(struct a6xx_gpu *a6xx_gpu, struct device_node *node);
 void a6xx_gmu_remove(struct a6xx_gpu *a6xx_gpu);
-void a6xx_gmu_sysprof_setup(struct msm_gpu *gpu);
+void a6xx_gmu_sysprof_setup(struct msm_gpu *gpu, bool force_on);
 
 void a6xx_preempt_init(struct msm_gpu *gpu);
 void a6xx_preempt_hw_init(struct msm_gpu *gpu);
drivers/gpu/drm/msm/msm_gpu.h +2 -8
index cb74b7606987..e3b5fab6f68f 100644
@@ -93,7 +93,7 @@ struct msm_gpu_funcs {
 	 * for cmdstream that is buffered in this FIFO upstream of the CP fw.
 	 */
 	bool (*progress)(struct msm_gpu *gpu, struct msm_ringbuffer *ring);
-	void (*sysprof_setup)(struct msm_gpu *gpu);
+	void (*sysprof_setup)(struct msm_gpu *gpu, bool force_on);
 
 	/* Configure perfcntr SELect regs: */
 	void (*perfcntr_configure)(struct msm_gpu *gpu, struct msm_ringbuffer *ring,
@@ -378,13 +378,7 @@ msm_gpu_sysprof_no_perfcntr_zap(struct msm_gpu *gpu)
 static inline bool
 msm_gpu_sysprof_no_ifpc(struct msm_gpu *gpu)
 {
-	/*
-	 * For now, this is the same condition as disabling perfcntr clears
-	 * on context switch.  But once kernel perfcntr IFPC support is in
-	 * place, we will only need to disable IFPC for legacy userspace
-	 * setting SYSPROF param.
-	 */
-	return msm_gpu_sysprof_no_perfcntr_zap(gpu);
+	return refcount_read(&gpu->sysprof_active) > 1;
 }
 
 /*
drivers/gpu/drm/msm/msm_perfcntr.c +8 -0
index 64f788e4c95e..f06a51d1d25a 100644
@@ -261,6 +261,10 @@ sample_worker(struct kthread_work *work)
 		return;
 	}
 
+	/* Inhibit IFPC while accessing registers: */
+	if (gpu->funcs->sysprof_setup)
+		gpu->funcs->sysprof_setup(gpu, true);
+
 	if (gpu->funcs->perfcntr_flush)
 		gpu->funcs->perfcntr_flush(gpu);
 
@@ -295,6 +299,10 @@ sample_worker(struct kthread_work *work)
 		}
 	}
 
+	/* Re-enable IFPC: */
+	if (gpu->funcs->sysprof_setup)
+		gpu->funcs->sysprof_setup(gpu, false);
+
 	smp_store_release(&stream->fifo.head, head);
 	wake_up_all(&stream->poll_wq);
 }
drivers/gpu/drm/msm/msm_submitqueue.c +1 -1
index a58fe41602c6..1a5a77b28016 100644
@@ -42,7 +42,7 @@ int msm_context_set_sysprof(struct msm_context *ctx, struct msm_gpu *gpu, int sy
 
 	/* Some gpu families require additional setup for sysprof */
 	if (gpu->funcs->sysprof_setup)
-		gpu->funcs->sysprof_setup(gpu);
+		gpu->funcs->sysprof_setup(gpu, false);
 
 	ctx->sysprof = sysprof;
 
Signature · 1 line
2.54.0