drm/msm: Add PERFCNTR_CONFIG ioctl
Subsystems (5)
Rob Clark
maintainerv3 cover drm/msm: Add PERFCNTR_CONFIG ioctl
Add a new PERFCNTR_CONFIG ioctl, serving two functions:
1. Global counter collection (restricted to perfmon_capable()) using the MSM_PERFCNTR_STREAM flag. Global counter sampling is, global, across contexts. Only a single global counter stream is allowed at a time.
2. Reserve counters for local counter collection. Local counter collection is local to a cmdstream (GEM_SUBMIT), and as such is allowed in all processes without additional privileges.
The kernel enforces that counters assigned for global counter collection do not conflict with counters reserved for local counter collection, and visa versa. Since local counter collection is scoped to a single cmd- stream, multiple UMD processes can overlap in their reserved counters. But cannot conflict with global counter usage.
In the case of local counter collection, the UMD is still responsible for programming the corresponding SELect registers, and sampling the counter values, from it's cmdstream. But by performing the reservation step, the UMD protects itself from the kernel trying to use the same SEL/counter regs for global counter collection.
For global counter collection, the kernel programs SEL regs, and sets up a timer for counter sampling. Userspace reads out the sampled values from the returned perfcntr stream fd. Releasing the global perfcntr stream is simply a matter of close()ing the fd.
The final two patches wire up the needed support for global counter stream collection while IFPC is active, and drops disabling of IFPC.
The mesa side of this is at:
https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/41158
igt test at:
https://gitlab.freedesktop.org/robclark/igt-gpu-tools/-/commits/perfcntrs
Changes in v3:
- Fix loop counter issue spotted by Claude review
- Add MSM_PERFCNTR_UPDATE flag to ask kernel to return the actual # of available counters in case of -E2BIG
- Proper barriers for modifying pwrup_reglist
- Link to v2: https://lore.kernel.org/all/[email protected]
Changes in v2:
- Rework makefile magic based on Dmitry's suggestion, and add a2xx/a5xx
perfcntr tables (although only a6xx+ is supported at this point)
- Fix compile error for compilers that are picky about a struct that
only contains a flex array
- Drop a6xx_idle() under gpu->lock in a6xx_perfcntr_configure(), replace
with perfcntr_fence that sel_worker can check
- Add a7xx+ pwrup_reglist support for restoring SELect regs on exit from
IFPC. (a6xx doesn't support IFPC, and the pwrup_reglist works a bit differently)
- Stop disabling IFPC when global counter stream is active.
- Link to v1: https://lore.kernel.org/all/[email protected]/
Rob Clark (16):
drm/msm: Remove obsolete perf infrastructure drm/msm: Allow CAP_PERFMON for setting SYSPROF drm/msm/adreno: Sync registers from mesa drm/msm/registers: Sync gen_header.py from mesa drm/msm/registers: Add perfcntr json drm/msm: Add a6xx+ perfcntr tables drm/msm: Add sysprof accessors drm/msm/a6xx: Add yield & flush helper drm/msm: Add per-context perfcntr state drm/msm: Add basic perfcntr infrastructure drm/msm/a6xx+: Add support to configure perfcntrs drm/msm/a8xx: Add perfcntr flush sequence drm/msm: Add PERFCNTR_CONFIG ioctl drm/msm/a6xx: Increase pwrup_reglist size drm/msm/a6xx: Append SEL regs to dyn pwrup reglist drm/msm/a6xx: Allow IFPC with perfcntr stream
drivers/gpu/drm/msm/Makefile | 27 +- drivers/gpu/drm/msm/adreno/a2xx_gpu.c | 7 - drivers/gpu/drm/msm/adreno/a3xx_gpu.c | 16 - drivers/gpu/drm/msm/adreno/a4xx_gpu.c | 3 - drivers/gpu/drm/msm/adreno/a5xx_gpu.c | 16 +- drivers/gpu/drm/msm/adreno/a6xx_gmu.c | 10 +- drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 217 +- drivers/gpu/drm/msm/adreno/a6xx_gpu.h | 15 +- drivers/gpu/drm/msm/adreno/a6xx_preempt.c | 2 +- drivers/gpu/drm/msm/adreno/a8xx_gpu.c | 33 +- drivers/gpu/drm/msm/adreno/a8xx_preempt.c | 2 +- drivers/gpu/drm/msm/adreno/adreno_device.c | 8 +- drivers/gpu/drm/msm/adreno/adreno_gpu.c | 7 +- drivers/gpu/drm/msm/msm_debugfs.c | 6 - drivers/gpu/drm/msm/msm_drv.c | 2 +- drivers/gpu/drm/msm/msm_drv.h | 13 +- drivers/gpu/drm/msm/msm_gpu.c | 119 +- drivers/gpu/drm/msm/msm_gpu.h | 104 +- drivers/gpu/drm/msm/msm_perf.c | 235 -- drivers/gpu/drm/msm/msm_perfcntr.c | 638 +++++ drivers/gpu/drm/msm/msm_perfcntr.h | 155 ++ drivers/gpu/drm/msm/msm_ringbuffer.h | 2 + drivers/gpu/drm/msm/msm_submitqueue.c | 3 +- .../msm/registers/adreno/a2xx_perfcntrs.json | 109 + drivers/gpu/drm/msm/registers/adreno/a3xx.xml | 8 +- drivers/gpu/drm/msm/registers/adreno/a5xx.xml | 141 +- .../msm/registers/adreno/a5xx_perfcntrs.json | 128 + drivers/gpu/drm/msm/registers/adreno/a6xx.xml | 1300 ++++++----- .../msm/registers/adreno/a6xx_descriptors.xml | 71 +- .../drm/msm/registers/adreno/a6xx_enums.xml | 3 + .../msm/registers/adreno/a6xx_perfcntrs.json | 105 + .../msm/registers/adreno/a7xx_perfcntrs.json | 228 ++ .../msm/registers/adreno/a8xx_descriptors.xml | 96 +- .../msm/registers/adreno/a8xx_perfcntrs.json | 240 ++ .../msm/registers/adreno/a8xx_perfcntrs.xml | 1929 +++++++++++++++ .../msm/registers/adreno/adreno_common.xml | 42 + .../drm/msm/registers/adreno/adreno_pm4.xml | 50 +- drivers/gpu/drm/msm/registers/gen_header.py | 2079 +++++++++-------- include/uapi/drm/msm_drm.h | 48 + 39 files changed, 6005 insertions(+), 2212 deletions(-) delete mode 100644 drivers/gpu/drm/msm/msm_perf.c create mode 100644 drivers/gpu/drm/msm/msm_perfcntr.c create mode 100644 drivers/gpu/drm/msm/msm_perfcntr.h create mode 100644 drivers/gpu/drm/msm/registers/adreno/a2xx_perfcntrs.json create mode 100644 drivers/gpu/drm/msm/registers/adreno/a5xx_perfcntrs.json create mode 100644 drivers/gpu/drm/msm/registers/adreno/a6xx_perfcntrs.json create mode 100644 drivers/gpu/drm/msm/registers/adreno/a7xx_perfcntrs.json create mode 100644 drivers/gpu/drm/msm/registers/adreno/a8xx_perfcntrs.json create mode 100644 drivers/gpu/drm/msm/registers/adreno/a8xx_perfcntrs.xml
Signature · 1 line
2.54.0
Rob Clark
maintainerPATCH v3 1/16 drm/msm: Remove obsolete perf infrastructure
Outside of a3xx, this was never really used. And it low-key gets in the way of the new perfcntr support (or at least it is confusing to have two things called "perf"). So lets remove it.
This drops the "perf" debugfs file. But these days, nvtop is a better option. (Plus perfetto for newer gens.)
--- drivers/gpu/drm/msm/Makefile | 1 - drivers/gpu/drm/msm/adreno/a2xx_gpu.c | 7 - drivers/gpu/drm/msm/adreno/a3xx_gpu.c | 16 -- drivers/gpu/drm/msm/adreno/a4xx_gpu.c | 3 - drivers/gpu/drm/msm/msm_debugfs.c | 6 - drivers/gpu/drm/msm/msm_drv.c | 1 - drivers/gpu/drm/msm/msm_drv.h | 5 - drivers/gpu/drm/msm/msm_gpu.c | 107 ------------ drivers/gpu/drm/msm/msm_gpu.h | 31 ---- drivers/gpu/drm/msm/msm_perf.c | 235 -------------------------- 10 files changed, 412 deletions(-) delete mode 100644 drivers/gpu/drm/msm/msm_perf.c
drivers/gpu/drm/msm/Makefile +0 -1
@@ -122,7 +122,6 @@ msm-y += \ msm_gpu_devfreq.o \ msm_io_utils.o \ msm_iommu.o \ - msm_perf.o \ msm_rd.o \ msm_ringbuffer.o \ msm_submitqueue.o \
drivers/gpu/drm/msm/adreno/a2xx_gpu.c +0 -7
@@ -489,10 +489,6 @@ static u32 a2xx_get_rptr(struct msm_gpu *gpu, struct msm_ringbuffer *ring)return ring->memptrs->rptr; } -static const struct msm_gpu_perfcntr perfcntrs[] = { -/* TODO */ -}; -static struct msm_gpu *a2xx_gpu_init (struct drm_device *dev) {struct a2xx_gpu *a2xx_gpu =NULL ;
@@ -518,9 +514,6 @@ static struct msm_gpu *a2xx_gpu_init(struct drm_device *dev) adreno_gpu = &a2xx_gpu->base; gpu = &adreno_gpu->base; - gpu->perfcntrs = perfcntrs; - gpu->num_perfcntrs =ARRAY_SIZE (perfcntrs); - ret =adreno_gpu_init (dev, pdev, adreno_gpu, config->info->funcs,1 );if (ret) goto fail;
drivers/gpu/drm/msm/adreno/a3xx_gpu.c +0 -16
@@ -266,12 +266,6 @@ static int a3xx_hw_init(struct msm_gpu *gpu)/* Turn on performance counters: */ gpu_write (gpu,REG_A3XX_RBBM_PERFCTR_CTL ,0x01 ); -/* Enable the perfcntrs that we use.. */ -for (i =0 ; i < gpu->num_perfcntrs; i++) { -const struct msm_gpu_perfcntr *perfcntr = &gpu->perfcntrs[i]; -gpu_write (gpu, perfcntr->select_reg, perfcntr->select_val); - } -gpu_write (gpu,REG_A3XX_RBBM_INT_0_MASK ,A3XX_INT0_MASK ); ret =adreno_hw_init (gpu);
@@ -508,13 +502,6 @@ static u32 a3xx_get_rptr(struct msm_gpu *gpu, struct msm_ringbuffer *ring)return ring->memptrs->rptr; } -static const struct msm_gpu_perfcntr perfcntrs[] = { - {REG_A3XX_SP_PERFCOUNTER6_SELECT ,REG_A3XX_RBBM_PERFCTR_SP_6_LO , -SP_ALU_ACTIVE_CYCLES ,"ALUACTIVE" }, - {REG_A3XX_SP_PERFCOUNTER7_SELECT ,REG_A3XX_RBBM_PERFCTR_SP_7_LO , -SP_FS_FULL_ALU_INSTRUCTIONS ,"ALUFULL" }, -}; -static struct msm_gpu *a3xx_gpu_init (struct drm_device *dev) {struct a3xx_gpu *a3xx_gpu =NULL ;
@@ -542,9 +529,6 @@ static struct msm_gpu *a3xx_gpu_init(struct drm_device *dev) adreno_gpu = &a3xx_gpu->base; gpu = &adreno_gpu->base; - gpu->perfcntrs = perfcntrs; - gpu->num_perfcntrs =ARRAY_SIZE (perfcntrs); - adreno_gpu->registers = a3xx_registers; ret =adreno_gpu_init (dev, pdev, adreno_gpu, config->info->funcs,1 );
drivers/gpu/drm/msm/adreno/a4xx_gpu.c +0 -3
@@ -652,9 +652,6 @@ static struct msm_gpu *a4xx_gpu_init(struct drm_device *dev) adreno_gpu = &a4xx_gpu->base; gpu = &adreno_gpu->base; - gpu->perfcntrs =NULL ; - gpu->num_perfcntrs =0 ; - ret =adreno_gpu_init (dev, pdev, adreno_gpu, config->info->funcs,1 );if (ret) goto fail;
drivers/gpu/drm/msm/msm_debugfs.c +0 -6
@@ -344,12 +344,6 @@ static int late_init_minor(struct drm_minor *minor)return ret; } - ret =msm_perf_debugfs_init (minor); -if (ret) { -DRM_DEV_ERROR (dev->dev,"could not install perf debugfs\n" ); -return ret; - } -return 0 ; }
drivers/gpu/drm/msm/msm_drv.c +0 -1
@@ -87,7 +87,6 @@ static int msm_drm_uninit(struct device *dev, const struct component_ops *gpu_opmsm_gem_shrinker_cleanup (ddev); -msm_perf_debugfs_cleanup (priv);msm_rd_debugfs_cleanup (priv);if (priv->kms)
drivers/gpu/drm/msm/msm_drv.h +0 -5
@@ -45,7 +45,6 @@ struct msm_gpu;struct msm_mmu ;struct msm_mdss ;struct msm_rd_state ; -struct msm_perf_state ;struct msm_gem_submit ;struct msm_fence_context ;struct msm_disp_state ;
@@ -89,7 +88,6 @@ struct msm_drm_private {struct msm_rd_state *rd;/* debugfs to dump all submits */ struct msm_rd_state *hangrd;/* debugfs to dump hanging submits */ -struct msm_perf_state *perf;/** * total_mem: Total/global amount of memory backing GEM objects.
@@ -442,8 +440,6 @@ void msm_rd_debugfs_cleanup(struct msm_drm_private *priv); __printf(3, 4)void msm_rd_dump_submit (struct msm_rd_state *rd,struct msm_gem_submit *submit,const char *fmt, ...); -int msm_perf_debugfs_init (struct drm_minor *minor); -void msm_perf_debugfs_cleanup (struct msm_drm_private *priv);#else static inline int msm_debugfs_late_init (struct drm_device *dev) {return 0 ; }__printf (3 ,4 )
@@ -451,7 +447,6 @@ static inline void msm_rd_dump_submit(struct msm_rd_state *rd,struct msm_gem_submit *submit, const char *fmt, ...) {} static inline void msm_rd_debugfs_cleanup(struct msm_drm_private *priv) {} -static inline void msm_perf_debugfs_cleanup(struct msm_drm_private *priv) {} #endif struct clk *msm_clk_get(struct platform_device *pdev, const char *name);
drivers/gpu/drm/msm/msm_gpu.c +0 -107
137 lines not on this page · raw message
drivers/gpu/drm/msm/msm_gpu.h +0 -31
@@ -22,7 +22,6 @@struct msm_gem_submit ;struct msm_gem_vm_log_entry ; -struct msm_gpu_perfcntr ;struct msm_gpu_state ;struct msm_context ;
@@ -168,18 +167,6 @@ struct msm_gpu {struct adreno_smmu_priv adreno_smmu; -/* performance counters (hw & sw): */ -spinlock_t perf_lock; -bool perfcntr_active; -struct { -bool active; -ktime_t time; - } last_sample; -uint32_t totaltime, activetime;/* sw counters */ -uint32_t last_cntrs[5 ];/* hw counters */ -const struct msm_gpu_perfcntr *perfcntrs; -uint32_t num_perfcntrs; -struct msm_ringbuffer *rb[MSM_GPU_MAX_RINGS ];int nr_rings;
@@ -320,19 +307,6 @@ static inline bool msm_gpu_active(struct msm_gpu *gpu)return false; } -/* Perf-Counters: -* The select_reg and select_val are just there for the benefit of the child -* class that actually enables the perf counter.. but msm_gpu base class -* will handle sampling/displaying the counters. -*/ - -struct msm_gpu_perfcntr { -uint32_t select_reg; -uint32_t sample_reg; -uint32_t select_val; -const char *name; -}; -/* * The number of priority levels provided by drm gpu scheduler. The * DRM_SCHED_PRIORITY_KERNEL priority level is treated specially in some
@@ -689,11 +663,6 @@ void msm_devfreq_idle(struct msm_gpu *gpu);int msm_gpu_hw_init (struct msm_gpu *gpu); -void msm_gpu_perfcntr_start (struct msm_gpu *gpu); -void msm_gpu_perfcntr_stop (struct msm_gpu *gpu); -int msm_gpu_perfcntr_sample (struct msm_gpu *gpu,uint32_t *activetime, -uint32_t *totaltime,uint32_t ncntrs,uint32_t *cntrs); -void msm_gpu_retire (struct msm_gpu *gpu);void msm_gpu_submit (struct msm_gpu *gpu,struct msm_gem_submit *submit);void msm_gpu_sysrq_kill (struct msm_gpu *gpu);
drivers/gpu/drm/msm/msm_perf.c deleted +0 -235
235 lines not on this page · raw message
Signature · 1 line
2.54.0
Rob Clark
maintainerPATCH v3 2/16 drm/msm: Allow CAP_PERFMON for setting SYSPROF
In reply to Rob ClarkUse perfmon_capable() which checks both CAP_SYS_ADMIN and CAP_PERFMON. This matches what i915 and xe do, and seems more appropriate.
--- drivers/gpu/drm/msm/adreno/adreno_gpu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/msm/adreno/adreno_gpu.c +1 -1
@@ -494,7 +494,7 @@ int adreno_set_param(struct msm_gpu *gpu, struct msm_context *ctx,return 0 ; }case MSM_PARAM_SYSPROF : -if (!capable (CAP_SYS_ADMIN )) +if (!perfmon_capable ())return UERR (EPERM , drm,"invalid permissions" );return msm_context_set_sysprof (ctx, gpu, value);case MSM_PARAM_EN_VM_BIND :
Signature · 1 line
2.54.0
Rob Clark
maintainerPATCH v3 3/16 drm/msm/adreno: Sync registers from mesa
In reply to Rob ClarkMost of the churn is just reworking the usage attribute on the mesa side.
Sync from mesa commit 4d4a951ac622 ("fd: add a8xx perfcntr countables").
--- drivers/gpu/drm/msm/adreno/a5xx_gpu.c | 16 +- drivers/gpu/drm/msm/registers/adreno/a3xx.xml | 8 +- drivers/gpu/drm/msm/registers/adreno/a5xx.xml | 141 +- drivers/gpu/drm/msm/registers/adreno/a6xx.xml | 1300 +++++------ .../msm/registers/adreno/a6xx_descriptors.xml | 71 +- .../drm/msm/registers/adreno/a6xx_enums.xml | 3 + .../msm/registers/adreno/a8xx_descriptors.xml | 96 +- .../msm/registers/adreno/a8xx_perfcntrs.xml | 1929 +++++++++++++++++ .../msm/registers/adreno/adreno_common.xml | 42 + .../drm/msm/registers/adreno/adreno_pm4.xml | 50 +- 10 files changed, 2846 insertions(+), 810 deletions(-) create mode 100644 drivers/gpu/drm/msm/registers/adreno/a8xx_perfcntrs.xml
drivers/gpu/drm/msm/adreno/a5xx_gpu.c +5 -11
@@ -752,17 +752,13 @@ static int a5xx_hw_init(struct msm_gpu *gpu)gpu_write (gpu,REG_A5XX_UCHE_CACHE_WAYS ,0x02 );/* Disable L2 bypass in the UCHE */ -gpu_write (gpu,REG_A5XX_UCHE_TRAP_BASE_LO ,lower_32_bits (adreno_gpu->uche_trap_base)); -gpu_write (gpu,REG_A5XX_UCHE_TRAP_BASE_HI ,upper_32_bits (adreno_gpu->uche_trap_base)); -gpu_write (gpu,REG_A5XX_UCHE_WRITE_THRU_BASE_LO ,lower_32_bits (adreno_gpu->uche_trap_base)); -gpu_write (gpu,REG_A5XX_UCHE_WRITE_THRU_BASE_HI ,upper_32_bits (adreno_gpu->uche_trap_base)); +gpu_write64 (gpu,REG_A5XX_UCHE_TRAP_BASE , adreno_gpu->uche_trap_base); +gpu_write64 (gpu,REG_A5XX_UCHE_WRITE_THRU_BASE , adreno_gpu->uche_trap_base);/* Set the GMEM VA range (0 to gpu->gmem) */ -gpu_write (gpu,REG_A5XX_UCHE_GMEM_RANGE_MIN_LO ,0x00100000 ); -gpu_write (gpu,REG_A5XX_UCHE_GMEM_RANGE_MIN_HI ,0x00000000 ); -gpu_write (gpu,REG_A5XX_UCHE_GMEM_RANGE_MAX_LO , +gpu_write64 (gpu,REG_A5XX_UCHE_GMEM_RANGE_MIN ,0x00100000 ); +gpu_write64 (gpu,REG_A5XX_UCHE_GMEM_RANGE_MAX ,0x00100000 + adreno_gpu->info->gmem -1 ); -gpu_write (gpu,REG_A5XX_UCHE_GMEM_RANGE_MAX_HI ,0x00000000 );if (adreno_is_a505 (adreno_gpu) ||adreno_is_a506 (adreno_gpu) ||adreno_is_a508 (adreno_gpu) ||adreno_is_a510 (adreno_gpu)) {
@@ -1217,9 +1213,7 @@ static void a5xx_rbbm_err_irq(struct msm_gpu *gpu, u32 status)static void a5xx_uche_err_irq (struct msm_gpu *gpu) { -uint64_t addr = (uint64_t )gpu_read (gpu,REG_A5XX_UCHE_TRAP_LOG_HI ); - - addr |=gpu_read (gpu,REG_A5XX_UCHE_TRAP_LOG_LO ); +uint64_t addr =gpu_read64 (gpu,REG_A5XX_UCHE_TRAP_LOG );dev_err_ratelimited (gpu->dev->dev,"UCHE | Out of bounds access | addr=0x%llX\n" , addr);
drivers/gpu/drm/msm/registers/adreno/a3xx.xml +2 -6
@@ -1330,11 +1330,7 @@ xsi:schemaLocation="https://gitlab.freedesktop.org/freedreno/ rules-fd.xsd"> </reg32> <reg32 offset="0x22c5" name="SP_VS_CTRL_REG1"> <bitfield name="CONSTLENGTH" low="0" high="9" type="uint"/> - <!-- - not sure about full vs half const.. I can't get blob generate - something with a mediump/lowp uniform. - --> - <bitfield name="CONSTFOOTPRINT" low="10" high="19" type="uint"/> + <bitfield name="CONSTMAXID" low="10" high="19" type="uint"/> <bitfield name="INITIALOUTSTANDING" low="24" high="30" type="uint"/> </reg32> <reg32 offset="0x22c6" name="SP_VS_PARAM_REG">
@@ -1420,7 +1416,7 @@ xsi:schemaLocation="https://gitlab.freedesktop.org/freedreno/ rules-fd.xsd"> </reg32> <reg32 offset="0x22e1" name="SP_FS_CTRL_REG1"> <bitfield name="CONSTLENGTH" low="0" high="9" type="uint"/> - <bitfield name="CONSTFOOTPRINT" low="10" high="19" type="uint"/> + <bitfield name="CONSTMAXID" low="10" high="19" type="uint"/> <bitfield name="INITIALOUTSTANDING" low="20" high="23" type="uint"/> <bitfield name="HALFPRECVAROFFSET" low="24" high="30" type="uint"/> </reg32>
drivers/gpu/drm/msm/registers/adreno/a5xx.xml +48 -93
311 lines not on this page · raw message
drivers/gpu/drm/msm/registers/adreno/a6xx.xml +692 -608
2,620 lines not on this page · raw message
drivers/gpu/drm/msm/registers/adreno/a6xx_descriptors.xml +43 -28
18 lines not on this page · raw message
drivers/gpu/drm/msm/registers/adreno/a6xx_enums.xml +3 -0
Lines of this file are not shown. View the raw message.
drivers/gpu/drm/msm/registers/adreno/a8xx_descriptors.xml +56 -40
96 lines not on this page · raw message
drivers/gpu/drm/msm/registers/adreno/a8xx_perfcntrs.xml new file +1929 -0
1,929 lines not on this page · raw message
drivers/gpu/drm/msm/registers/adreno/adreno_common.xml +42 -0
42 lines not on this page · raw message
drivers/gpu/drm/msm/registers/adreno/adreno_pm4.xml +26 -24
50 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Signature · 1 line
2.54.0
Part of the diff is not shown here. View the raw message.
Rob Clark
maintainerPATCH v3 4/16 drm/msm/registers: Sync gen_header.py from mesa
In reply to Rob ClarkUpdate gen_header.py to bring in support for generating perfcntr tables.
Sync from https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/40522
--- drivers/gpu/drm/msm/registers/gen_header.py | 2079 ++++++++++--------- 1 file changed, 1146 insertions(+), 933 deletions(-)
drivers/gpu/drm/msm/registers/gen_header.py +1146 -933
@@ -11,997 +11,1210 @@ import collections import argparse import time import datetime +import json + class Error(Exception): - def __init__(self, message): - self.message = message + def __init__(self, message): + self.message = message + class Enum(object): - def __init__(self, name): - self.name = name - self.values = [] - - def has_name(self, name): - for (n, value) in self.values: - if n == name: - return True - return False - - def names(self): - return [n for (n, value) in self.values] - - def dump(self, is_deprecated): - use_hex = False - for (name, value) in self.values: - if value > 0x1000: - use_hex = True - - print("enum %s {" % self.name) - for (name, value) in self.values: - if use_hex: - print("\t%s = 0x%08x," % (name, value)) - else: - print("\t%s = %d," % (name, value)) - print("};\n") - - def dump_pack_struct(self, is_deprecated): - pass + def __init__(self, name): + self.name = name + self.values = [] + + def has_name(self, name): + for (n, value) in self.values: + if n == name: + return True + return False + + def names(self): + return [n for (n, value) in self.values] + + def value(self, name): + for (n, v) in self.values: + if n == name: + return v + + def dump(self, has_variants): + use_hex = False + for (name, value) in self.values: + if value > 0x1000: + use_hex = True + + print("enum %s {" % self.name) + for (name, value) in self.values: + if use_hex: + print("\t%s = 0x%08x," % (name, value)) + else: + print("\t%s = %d," % (name, value)) + print("};\n") + + def dump_pack_struct(self, has_variants): + pass + class Field(object): - def __init__(self, name, low, high, shr, type, parser): - self.name = name - self.low = low - self.high = high - self.shr = shr - self.type = type - - builtin_types = [ None, "a3xx_regid", "boolean", "uint", "hex", "int", "fixed", "ufixed", "float", "address", "waddress" ] - - maxpos = parser.current_bitsize - 1 - - if low < 0 or low > maxpos: - raise parser.error("low attribute out of range: %d" % low) - if high < 0 or high > maxpos: - raise parser.error("high attribute out of range: %d" % high) - if high < low: - raise parser.error("low is greater than high: low=%d, high=%d" % (low, high)) - if self.type == "boolean" and not low == high: - raise parser.error("booleans should be 1 bit fields") - elif self.type == "float" and not (high - low == 31 or high - low == 15): - raise parser.error("floats should be 16 or 32 bit fields") - elif self.type not in builtin_types and self.type not in parser.enums: - raise parser.error("unknown type '%s'" % self.type) - - def ctype(self, var_name): - if self.type is None: - type = "uint32_t" - val = var_name - elif self.type == "boolean": - type = "bool" - val = var_name - elif self.type == "uint" or self.type == "hex" or self.type == "a3xx_regid": - type = "uint32_t" - val = var_name - elif self.type == "int": - type = "int32_t" - val = var_name - elif self.type == "fixed": - type = "float" - val = "((int32_t)(%s * %d.0))" % (var_name, 1 << self.radix) - elif self.type == "ufixed": - type = "float" - val = "((uint32_t)(%s * %d.0))" % (var_name, 1 << self.radix) - elif self.type == "float" and self.high - self.low == 31: - type = "float" - val = "fui(%s)" % var_name - elif self.type == "float" and self.high - self.low == 15: - type = "float" - val = "_mesa_float_to_half(%s)" % var_name - elif self.type in [ "address", "waddress" ]: - type = "uint64_t" - val = var_name - else: - type = "enum %s" % self.type - val = var_name - - if self.shr > 0: - val = "(%s >> %d)" % (val, self.shr) - - return (type, val) + def __init__(self, name, low, high, shr, type, parser): + self.name = name + self.low = low + self.high = high + self.shr = shr + self.type = type + + builtin_types = [None, "a3xx_regid", "boolean", "uint", "hex", + "int", "fixed", "ufixed", "float", "address", "waddress"] + + maxpos = parser.current_bitsize - 1 + + if low < 0 or low > maxpos: + raise parser.error("low attribute out of range: %d" % low) + if high < 0 or high > maxpos: + raise parser.error("high attribute out of range: %d" % high) + if high < low: + raise parser.error( + "low is greater than high: low=%d, high=%d" % (low, high)) + if self.type == "boolean" and not low == high: + raise parser.error("booleans should be 1 bit fields") + elif self.type == "float" and not (high - low == 31 or high - low == 15): + raise parser.error("floats should be 16 or 32 bit fields") + elif self.type not in builtin_types and self.type not in parser.enums: + raise parser.error("unknown type '%s'" % self.type) + + def ctype(self, var_name): + if self.type is None: + type = "uint32_t" + val = var_name + elif self.type == "boolean": + type = "bool" + val = var_name + elif self.type == "uint" or self.type == "hex" or self.type == "a3xx_regid": + type = "uint32_t" + val = var_name + elif self.type == "int": + type = "int32_t" + val = var_name + elif self.type == "fixed": + type = "float" + val = "(uint32_t)((int32_t)(%s * %d.0))" % (var_name, 1 << self.radix) + elif self.type == "ufixed": + type = "float" + val = "((uint32_t)(%s * %d.0))" % (var_name, 1 << self.radix) + elif self.type == "float" and self.high - self.low == 31: + type = "float" + val = "fui(%s)" % var_name + elif self.type == "float" and self.high - self.low == 15: + type = "float" + val = "_mesa_float_to_half(%s)" % var_name + elif self.type in ["address", "waddress"]: + type = "uint64_t" + val = var_name + else: + type = "enum %s" % self.type + val = var_name + + if self.shr > 0: + val = "(%s >> %d)" % (val, self.shr) + + return (type, val) + def tab_to(name, value): - tab_count = (68 - (len(name) & ~7)) // 8 - if tab_count <= 0: - tab_count = 1 - print(name + ('\t' * tab_count) + value) + tab_count = (68 - (len(name) & ~7)) // 8 + if tab_count <= 0: + tab_count = 1 + print(name + ('\t' * tab_count) + value) + +def define_macro(name, value, has_variants): + if has_variants: + value = "__FD_DEPRECATED " + value + tab_to(name, value) def mask(low, high): - return ((0xffffffffffffffff >> (64 - (high + 1 - low))) << low) + return ((0xffffffffffffffff >> (64 - (high + 1 - low))) << low) + def field_name(reg, f): - if f.name: - name = f.name.lower() - else: - # We hit this path when a reg is defined with no bitset fields, ie. - # <reg32 offset="0x88db" name="RB_RESOLVE_SYSTEM_BUFFER_ARRAY_PITCH" low="0" high="28" shr="6" type="uint"/> - name = reg.name.lower() + if f.name: + name = f.name.lower() + else: + # We hit this path when a reg is defined with no bitset fields, ie. + # <reg32 offset="0x88db" name="RB_RESOLVE_SYSTEM_BUFFER_ARRAY_PITCH" low="0" high="28" shr="6" type="uint"/> + name = reg.name.lower() - if (name in [ "double", "float", "int" ]) or not (name[0].isalpha()): - name = "_" + name + if (name in ["double", "float", "int"]) or not (name[0].isalpha()): + name = "_" + name - return name + return name # indices - array of (ctype, stride, __offsets_NAME) + + def indices_varlist(indices): - return ", ".join(["i%d" % i for i in range(len(indices))]) + return ", ".join(["i%d" % i for i in range(len(indices))]) + def indices_prototype(indices): - return ", ".join(["%s i%d" % (ctype, idx) - for (idx, (ctype, stride, offset)) in enumerate(indices)]) + return ", ".join(["%s i%d" % (ctype, idx) + for (idx, (ctype, stride, offset)) in enumerate(indices)]) + def indices_strides(indices): - return " + ".join(["0x%x*i%d" % (stride, idx) - if stride else - "%s(i%d)" % (offset, idx) - for (idx, (ctype, stride, offset)) in enumerate(indices)]) + return " + ".join(["0x%x*i%d" % (stride, idx) + if stride else + "%s(i%d)" % (offset, idx) + for (idx, (ctype, stride, offset)) in enumerate(indices)]) + def is_number(str): - try: - int(str) - return True - except ValueError: - return False + try: + int(str) + return True + except ValueError: + return False + def sanitize_variant(variant): - if variant and "-" in variant: - return variant[:variant.index("-")] - return variant + if variant and "-" in variant: + return variant[:variant.index("-")] + return variant + class Bitset(object): - def __init__(self, name, template): - self.name = name - self.inline = False - self.reg = None - if template: - self.fields = template.fields[:] - else: - self.fields = [] - - # Get address field if there is one in the bitset, else return None: - def get_address_field(self): - for f in self.fields: - if f.type in [ "address", "waddress" ]: - return f - return None - - def dump_regpair_builder(self, reg): - print("#ifndef NDEBUG") - known_mask = 0 - for f in self.fields: - known_mask |= mask(f.low, f.high) - if f.type in [ "boolean", "address", "waddress" ]: - continue - type, val = f.ctype("fields.%s" % field_name(reg, f)) - print(" assert((%-40s & 0x%08x) == 0);" % (val, 0xffffffff ^ mask(0 , f.high - f.low))) - print(" assert((%-40s & 0x%08x) == 0);" % ("fields.unknown", known_mask)) - print("#endif\n") - - print(" return (struct fd_reg_pair) {") - print(" .reg = (uint32_t)%s," % reg.reg_offset()) - print(" .value =") - cast = "(uint64_t)" if reg.bit_size == 64 else "" - for f in self.fields: - if f.type in [ "address", "waddress" ]: - continue - else: - type, val = f.ctype("fields.%s" % field_name(reg, f)) - print(" (%s%-40s << %2d) |" % (cast, val, f.low)) - value_name = "dword" - if reg.bit_size == 64: - value_name = "qword" - print(" fields.unknown | fields.%s," % (value_name,)) - - address = self.get_address_field() - if address: - print(" .bo = fields.bo,") - print(" .is_address = true,") - if f.type == "waddress": - print(" .bo_write = true,") - print(" .bo_offset = fields.bo_offset,") - print(" .bo_shift = %d," % address.shr) - print(" .bo_low = %d," % address.low) - - print(" };") - - def dump_pack_struct(self, is_deprecated, reg=None): - if not reg: - return - - prefix = reg.full_name - - print("struct %s {" % prefix) - for f in self.fields: - if f.type in [ "address", "waddress" ]: - tab_to(" __bo_type", "bo;") - tab_to(" uint32_t", "bo_offset;") - continue - name = field_name(reg, f) - - type, val = f.ctype("var") - - tab_to(" %s" % type, "%s;" % name) - if reg.bit_size == 64: - tab_to(" uint64_t", "unknown;") - tab_to(" uint64_t", "qword;") - else: - tab_to(" uint32_t", "unknown;") - tab_to(" uint32_t", "dword;") - print("};\n") - - depcrstr = "" - if is_deprecated: - depcrstr = " FD_DEPRECATED" - if reg.array: - print("static inline%s struct fd_reg_pair\npack_%s(uint32_t __i, struct %s fields)\n{" % - (depcrstr, prefix, prefix)) - else: - print("static inline%s struct fd_reg_pair\npack_%s(struct %s fields)\n{" % - (depcrstr, prefix, prefix)) - - self.dump_regpair_builder(reg) - - print("\n}\n") - - if self.get_address_field(): - skip = ", { .reg = 0 }" - else: - skip = "" - - if reg.array: - print("#define %s(__i, ...) pack_%s(__i, __struct_cast(%s) { __VA_ARGS__ })%s\n" % - (prefix, prefix, prefix, skip)) - else: - print("#define %s(...) pack_%s(__struct_cast(%s) { __VA_ARGS__ })%s\n" % - (prefix, prefix, prefix, skip)) - - - def dump(self, is_deprecated, prefix=None, reg=None): - if prefix is None: - prefix = self.name - reg64 = reg and self.reg and self.reg.bit_size == 64 - if reg64: - print("static inline uint32_t %s_LO(uint32_t val)\n{" % prefix) - print("\treturn val;\n}") - print("static inline uint32_t %s_HI(uint32_t val)\n{" % prefix) - print("\treturn val;\n}") - for f in self.fields: - if f.name: - name = prefix + "_" + f.name - else: - name = prefix - - if not f.name and f.low == 0 and f.shr == 0 and f.type not in ["float", "fixed", "ufixed"]: - pass - elif f.type == "boolean" or (f.type is None and f.low == f.high): - tab_to("#define %s" % name, "0x%08x" % (1 << f.low)) - else: - typespec = "ull" if reg64 else "u" - tab_to("#define %s__MASK" % name, "0x%08x%s" % (mask(f.low, f.high), typespec)) - tab_to("#define %s__SHIFT" % name, "%d" % f.low) - type, val = f.ctype("val") - ret_type = "uint64_t" if reg64 else "uint32_t" - cast = "(uint64_t)" if reg64 else "" - - print("static inline %s %s(%s val)\n{" % (ret_type, name, type)) - if f.shr > 0: - print("\tassert(!(val & 0x%x));" % mask(0, f.shr - 1)) - print("\treturn (%s(%s) << %s__SHIFT) & %s__MASK;\n}" % (cast, val, name, name)) - print() + def __init__(self, name, template): + self.name = name + self.inline = False + self.reg = None + if template: + self.fields = template.fields[:] + else: + self.fields = [] + + # Get address field if there is one in the bitset, else return None: + def get_address_field(self): + for f in self.fields: + if f.type in ["address", "waddress"]: + return f + return None + + def dump_regpair_builder(self, reg): + print("#ifndef NDEBUG") + known_mask = 0 + for f in self.fields: + known_mask |= mask(f.low, f.high) + if f.type in ["boolean", "address", "waddress"]: + continue + type, val = f.ctype("fields.%s" % field_name(reg, f)) + print(" assert((%-40s & 0x%08x) == 0);" % + (val, 0xffffffff ^ mask(0, f.high - f.low))) + print(" assert((%-40s & 0x%08x) == 0);" % + ("fields.unknown", known_mask)) + print("#endif\n") + + print(" return (struct fd_reg_pair) {") + print(" .reg = (uint32_t)%s," % reg.reg_offset()) + print(" .value =") + cast = "(uint64_t)" if reg.bit_size == 64 else "" + for f in self.fields: + if f.type in ["address", "waddress"]: + continue + else: + type, val = f.ctype("fields.%s" % field_name(reg, f)) + print(" (%s%-40s << %2d) |" % (cast, val, f.low)) + value_name = "dword" + if reg.bit_size == 64: + value_name = "qword" + print(" fields.unknown | fields.%s," % (value_name,)) + + address = self.get_address_field() + if address: + print("#ifndef TU_CS_H") + print(" .bo = fields.bo,") + print(" .is_address = true,") + print(" .bo_offset = fields.bo_offset,") + print(" .bo_shift = %d," % address.shr) + print(" .bo_low = %d," % address.low) + print("#else") + print(" .is_address = true,") + print("#endif") + + print(" };") + + def dump_pack_struct(self, has_variants, reg=None): + if not reg: + return + + prefix = reg.full_name + + constexpr_mark = " CONSTEXPR" + + print("struct %s {" % prefix) + for f in self.fields: + if f.type in ["address", "waddress"]: + print("#ifndef TU_CS_H") + tab_to(" __bo_type", "bo;") + tab_to(" uint32_t", "bo_offset;") + print("#endif\n") + continue + name = field_name(reg, f) + + type, val = f.ctype("var") + + tab_to(" %s" % type, "%s;" % name) + + if f.type == "float": + # Requires using `fui()` or `_mesa_float_to_half()` + constexpr_mark = "" + if reg.bit_size == 64: + tab_to(" uint64_t", "qword;") + tab_to(" uint64_t", "unknown;") + else: + tab_to(" uint32_t", "dword;") + tab_to(" uint32_t", "unknown;") + print("};\n") + + if not has_variants: + print("static%s inline struct fd_reg_pair" % constexpr_mark) + if reg.array: + print("pack_%s(uint32_t __i, struct %s fields)\n{" % (prefix, prefix)) + else: + print("pack_%s(struct %s fields)\n{" % (prefix, prefix)) + + self.dump_regpair_builder(reg) + + print("\n}\n") + + if self.get_address_field(): + skip = ", { .reg = 0 }" + else: + skip = "" + + if reg.array: + print("#define %s(__i, ...) pack_%s(__i, __struct_cast(%s) { __VA_ARGS__ })%s\n" % + (prefix, prefix, prefix, skip)) + else: + print("#define %s(...) pack_%s(__struct_cast(%s) { __VA_ARGS__ })%s\n" % + (prefix, prefix, prefix, skip)) + + def dump(self, has_variants, prefix=None, reg=None): + if prefix is None: + prefix = self.name + suffix = "" + if self.reg and self.reg.bit_size == 64: + print( + "static CONSTEXPR inline uint32_t %s_LO(uint32_t val)\n{" % prefix) + print("\treturn val;\n}") + print( + "static CONSTEXPR inline uint32_t %s_HI(uint32_t val)\n{" % prefix) + print("\treturn val;\n}") + suffix = "ull" + + for f in self.fields: + if f.name: + name = prefix + "_" + f.name + else: + name = prefix + + if not f.name and f.low == 0 and f.shr == 0 and f.type not in ["float", "fixed", "ufixed"]: + pass + elif f.type == "boolean" or (f.type is None and f.low == f.high): + tab_to("#define %s" % name, "0x%08x%s" % ((1 << f.low), suffix)) + else: + tab_to("#define %s__MASK" % + name, "0x%08x%s" % (mask(f.low, f.high), suffix)) + tab_to("#define %s__SHIFT" % name, "%d" % f.low) + type, val = f.ctype("val") + ret_type = "uint64_t" if reg and reg.bit_size == 64 else "uint32_t" + cast = "(uint64_t)" if reg and reg.bit_size == 64 else "" + + constexpr_mark = "" if type == "float" else " CONSTEXPR" + print("static%s inline %s %s(%s val)\n{" % ( + constexpr_mark, ret_type, name, type)) + if f.shr > 0: + print("\tassert(!(val & 0x%x));" % mask(0, f.shr - 1)) + print("\treturn (%s(%s) << %s__SHIFT) & %s__MASK;\n}" % + (cast, val, name, name)) + print() + class Array(object): - def __init__(self, attrs, domain, variant, parent, index_type): - if "name" in attrs: - self.local_name = attrs["name"] - else: - self.local_name = "" - self.domain = domain - self.variant = variant - self.parent = parent - self.children = [] - if self.parent: - self.name = self.parent.name + "_" + self.local_name - else: - self.name = self.local_name - if "offsets" in attrs: - self.offsets = map(lambda i: "0x%08x" % int(i, 0), attrs["offsets"].split(",")) - self.fixed_offsets = True - elif "doffsets" in attrs: - self.offsets = map(lambda s: "(%s)" % s , attrs["doffsets"].split(",")) - self.fixed_offsets = True - else: - self.offset = int(attrs["offset"], 0) - self.stride = int(attrs["stride"], 0) - self.fixed_offsets = False - if "index" in attrs: - self.index_type = index_type - else: - self.index_type = None - self.length = int(attrs["length"], 0) - if "usage" in attrs: - self.usages = attrs["usage"].split(',') - else: - self.usages = None - - def index_ctype(self): - if not self.index_type: - return "uint32_t" - else: - return "enum %s" % self.index_type.name - - # Generate array of (ctype, stride, __offsets_NAME) - def indices(self): - if self.parent: - indices = self.parent.indices() - else: - indices = [] - if self.length != 1: - if self.fixed_offsets: - indices.append((self.index_ctype(), None, "__offset_%s" % self.local_name)) - else: - indices.append((self.index_ctype(), self.stride, None)) - return indices - - def total_offset(self): - offset = 0 - if not self.fixed_offsets: - offset += self.offset - if self.parent: - offset += self.parent.total_offset() - return offset - - def dump(self, is_deprecated): - depcrstr = "" - if is_deprecated: - depcrstr = " FD_DEPRECATED" - proto = indices_varlist(self.indices()) - strides = indices_strides(self.indices()) - array_offset = self.total_offset() - if self.fixed_offsets: - print("static inline%s uint32_t __offset_%s(%s idx)" % (depcrstr, self.local_name, self.index_ctype())) - print("{\n\tswitch (idx) {") - if self.index_type: - for val, offset in zip(self.index_type.names(), self.offsets): - print("\t\tcase %s: return %s;" % (val, offset)) - else: - for idx, offset in enumerate(self.offsets): - print("\t\tcase %d: return %s;" % (idx, offset)) - print("\t\tdefault: return INVALID_IDX(idx);") - print("\t}\n}") - if proto == '': - tab_to("#define REG_%s_%s" % (self.domain, self.name), "0x%08x\n" % array_offset) - else: - tab_to("#define REG_%s_%s(%s)" % (self.domain, self.name, proto), "(0x%08x + %s )\n" % (array_offset, strides)) - - def dump_pack_struct(self, is_deprecated): - pass - - def dump_regpair_builder(self): - pass + def __init__(self, attrs, domain, variant, parent, index_type): + if "name" in attrs: + self.local_name = attrs["name"] + else: + self.local_name = "" + self.domain = domain + self.variant = variant + self.parent = parent + self.children = [] + if self.parent: + self.name = self.parent.name + "_" + self.local_name + else: + self.name = self.local_name + if "offsets" in attrs: + self.offsets = map(lambda i: "0x%08x" % + int(i, 0), attrs["offsets"].split(",")) + self.fixed_offsets = True + elif "doffsets" in attrs: + self.offsets = map(lambda s: "(%s)" % + s, attrs["doffsets"].split(",")) + self.fixed_offsets = True + else: + self.offset = int(attrs["offset"], 0) + self.stride = int(attrs["stride"], 0) + self.fixed_offsets = False + if "index" in attrs: + self.index_type = index_type + else: + self.index_type = None + self.length = int(attrs["length"], 0) + if "usage" in attrs: + self.usages = attrs["usage"].split(',') + else: + self.usages = None + + def index_ctype(self): + if not self.index_type: + return "uint32_t" + else: + return "enum %s" % self.index_type.name + + # Generate array of (ctype, stride, __offsets_NAME) + def indices(self): + if self.parent: + indices = self.parent.indices() + else: + indices = [] + if self.length != 1: + if self.fixed_offsets: + indices.append((self.index_ctype(), None, + "__offset_%s" % self.local_name)) + else: + indices.append((self.index_ctype(), self.stride, None)) + return indices + + def total_offset(self): + offset = 0 + if not self.fixed_offsets: + offset += self.offset + if self.parent: + offset += self.parent.total_offset() + return offset + + def dump(self, has_variants): + proto = indices_varlist(self.indices()) + strides = indices_strides(self.indices()) + array_offset = self.total_offset() + if self.fixed_offsets and not has_variants: + print("static CONSTEXPR inline uint32_t __offset_%s(%s idx)" % + (self.local_name, self.index_ctype())) + print("{\n\tswitch (idx) {") + if self.index_type: + for val, offset in zip(self.index_type.names(), self.offsets): + print("\t\tcase %s: return %s;" % (val, offset)) + else: + for idx, offset in enumerate(self.offsets): + print("\t\tcase %d: return %s;" % (idx, offset)) + print("\t\tdefault: return INVALID_IDX(idx);") + print("\t}\n}") + if proto == '': + define_macro("#define REG_%s_%s" % + (self.domain, self.name), "0x%08x\n" % array_offset, + has_variants) + else: + define_macro("#define REG_%s_%s(%s)" % (self.domain, self.name, + proto), "(0x%08x + %s )\n" % (array_offset, strides), + has_variants) + + def dump_pack_struct(self, has_variants): + pass + + def dump_regpair_builder(self): + pass + class Reg(object): - def __init__(self, attrs, domain, array, bit_size): - self.name = attrs["name"] - self.domain = domain - self.array = array - self.offset = int(attrs["offset"], 0) - self.type = None - self.bit_size = bit_size - if array: - self.name = array.name + "_" + self.name - array.children.append(self) - self.full_name = self.domain + "_" + self.name - if "stride" in attrs: - self.stride = int(attrs["stride"], 0) - self.length = int(attrs["length"], 0) - else: - self.stride = None - self.length = None - - # Generate array of (ctype, stride, __offsets_NAME) - def indices(self): - if self.array: - indices = self.array.indices() - else: - indices = [] - if self.stride: - indices.append(("uint32_t", self.stride, None)) - return indices - - def total_offset(self): - if self.array: - return self.array.total_offset() + self.offset - else: - return self.offset - - def reg_offset(self): - if self.array: - offset = self.array.offset + self.offset - return "(0x%08x + 0x%x*__i)" % (offset, self.array.stride) - return "0x%08x" % self.offset - - def dump(self, is_deprecated): - depcrstr = "" - if is_deprecated: - depcrstr = " FD_DEPRECATED " - proto = indices_prototype(self.indices()) - strides = indices_strides(self.indices()) - offset = self.total_offset() - if proto == '': - tab_to("#define REG_%s" % self.full_name, "0x%08x" % offset) - else: - print("static inline%s uint32_t REG_%s(%s) { return 0x%08x + %s; }" % (depcrstr, self.full_name, proto, offset, strides)) - - if self.bitset.inline: - self.bitset.dump(is_deprecated, self.full_name, self) - print("") - - def dump_pack_struct(self, is_deprecated): - if self.bitset.inline: - self.bitset.dump_pack_struct(is_deprecated, self) - - def dump_regpair_builder(self): - self.bitset.dump_regpair_builder(self) - - def dump_py(self): - print("\tREG_%s = 0x%08x" % (self.full_name, self.offset)) + def __init__(self, attrs, domain, array, bit_size): + self.name = attrs["name"] + self.domain = domain + self.array = array + self.offset = int(attrs["offset"], 0) + self.type = None + self.bit_size = bit_size + if array: + self.name = array.name + "_" + self.name + array.children.append(self) + self.full_name = self.domain + "_" + self.name + if "stride" in attrs: + self.stride = int(attrs["stride"], 0) + self.length = int(attrs["length"], 0) + else: + self.stride = None + self.length = None + + # Generate array of (ctype, stride, __offsets_NAME) + def indices(self): + if self.array: + indices = self.array.indices() + else: + indices = [] + if self.stride: + indices.append(("uint32_t", self.stride, None)) + return indices + + def total_offset(self): + if self.array: + return self.array.total_offset() + self.offset + else: + return self.offset + + def reg_offset(self): + if self.array: + offset = self.array.offset + self.offset + return "(0x%08x + 0x%x*__i)" % (offset, self.array.stride) + return "0x%08x" % self.offset + + def dump(self, has_variants): + proto = indices_prototype(self.indices()) + strides = indices_strides(self.indices()) + offset = self.total_offset() + if proto == '': + define_macro("#define REG_%s" % self.full_name, "0x%08x" % offset, has_variants) + elif not has_variants: + depcrstr = "" + if has_variants: + depcrstr = " __FD_DEPRECATED " + print("static CONSTEXPR inline%s uint32_t REG_%s(%s) { return 0x%08x + %s; }" % ( + depcrstr, self.full_name, proto, offset, strides)) + + if self.bitset.inline: + self.bitset.dump(has_variants, self.full_name, self) + print("") + + def dump_pack_struct(self, has_variants): + if self.bitset.inline: + self.bitset.dump_pack_struct(has_variants, self) + + def dump_regpair_builder(self): + self.bitset.dump_regpair_builder(self) + + def dump_py(self): + offset = self.offset + if self.array: + offset += self.array.offset + print("\tREG_%s = 0x%08x" % (self.full_name, offset)) class Parser(object): - def __init__(self): - self.current_array = None - self.current_domain = None - self.current_prefix = None - self.current_prefix_type = None - self.current_stripe = None - self.current_bitset = None - self.current_bitsize = 32 - # The varset attribute on the domain specifies the enum which - # specifies all possible hw variants: - self.current_varset = None - # Regs that have multiple variants.. we only generated the C++ - # template based struct-packers for these - self.variant_regs = {} - # Information in which contexts regs are used, to be used in - # debug options - self.usage_regs = collections.defaultdict(list) - self.bitsets = {} - self.enums = {} - self.variants = set() - self.file = [] - self.xml_files = [] - - def error(self, message): - parser, filename = self.stack[-1] - return Error("%s:%d:%d: %s" % (filename, parser.CurrentLineNumber, parser.CurrentColumnNumber, message)) - - def prefix(self, variant=None): - if self.current_prefix_type == "variant" and variant: - return sanitize_variant(variant) - elif self.current_stripe: - return self.current_stripe + "_" + self.current_domain - elif self.current_prefix: - return self.current_prefix + "_" + self.current_domain - else: - return self.current_domain - - def parse_field(self, name, attrs): - try: - if "pos" in attrs: - high = low = int(attrs["pos"], 0) - elif "high" in attrs and "low" in attrs: - high = int(attrs["high"], 0) - low = int(attrs["low"], 0) - else: - low = 0 - high = self.current_bitsize - 1 - - if "type" in attrs: - type = attrs["type"] - else: - type = None - - if "shr" in attrs: - shr = int(attrs["shr"], 0) - else: - shr = 0 - - b = Field(name, low, high, shr, type, self) - - if type == "fixed" or type == "ufixed": - b.radix = int(attrs["radix"], 0) - - self.current_bitset.fields.append(b) - except ValueError as e: - raise self.error(e) - - def parse_varset(self, attrs): - # Inherit the varset from the enclosing domain if not overriden: - varset = self.current_varset - if "varset" in attrs: - varset = self.enums[attrs["varset"]] - return varset - - def parse_variants(self, attrs): - if "variants" not in attrs: - return None - - variant = attrs["variants"].split(",")[0] - varset = self.parse_varset(attrs) - - if "-" in variant: - # if we have a range, validate that both the start and end - # of the range are valid enums: - start = variant[:variant.index("-")] - end = variant[variant.index("-") + 1:] - assert varset.has_name(start) - if end != "": - assert varset.has_name(end) - else: - assert varset.has_name(variant) - - return variant - - def add_all_variants(self, reg, attrs, parent_variant): - # TODO this should really handle *all* variants, including dealing - # with open ended ranges (ie. "A2XX,A4XX-") (we have the varset - # enum now to make that possible) - variant = self.parse_variants(attrs) - if not variant: - variant = parent_variant - - if reg.name not in self.variant_regs: - self.variant_regs[reg.name] = {} - else: - # All variants must be same size: - v = next(iter(self.variant_regs[reg.name])) - assert self.variant_regs[reg.name][v].bit_size == reg.bit_size - - self.variant_regs[reg.name][variant] = reg - - def add_all_usages(self, reg, usages): - if not usages: - return - - for usage in usages: - self.usage_regs[usage].append(reg) - - self.variants.add(reg.domain) - - def do_validate(self, schemafile): - if not self.validate: - return - - try: - from lxml import etree - - parser, filename = self.stack[-1] - dirname = os.path.dirname(filename) - - # we expect this to look like <namespace url> schema.xsd.. I think - # technically it is supposed to be just a URL, but that doesn't - # quite match up to what we do.. Just skip over everything up to - # and including the first whitespace character: - schemafile = schemafile[schemafile.rindex(" ")+1:] - - # this is a bit cheezy, but the xml file to validate could be - # in a child director, ie. we don't really know where the schema - # file is, the way the rnn C code does. So if it doesn't exist - # just look one level up - if not os.path.exists(dirname + "/" + schemafile): - schemafile = "../" + schemafile - - if not os.path.exists(dirname + "/" + schemafile): - raise self.error("Cannot find schema for: " + filename) - - xmlschema_doc = etree.parse(dirname + "/" + schemafile) - xmlschema = etree.XMLSchema(xmlschema_doc) - - xml_doc = etree.parse(filename) - if not xmlschema.validate(xml_doc): - error_str = str(xmlschema.error_log.filter_from_errors()[0]) - raise self.error("Schema validation failed for: " + filename + "\n" + error_str) - except ImportError as e: - print("lxml not found, skipping validation", file=sys.stderr) - - def do_parse(self, filename): - filepath = os.path.abspath(filename) - if filepath in self.xml_files: - return - self.xml_files.append(filepath) - file = open(filename, "rb") - parser = xml.parsers.expat.ParserCreate() - self.stack.append((parser, filename)) - parser.StartElementHandler = self.start_element - parser.EndElementHandler = self.end_element - parser.CharacterDataHandler = self.character_data - parser.buffer_text = True - parser.ParseFile(file) - self.stack.pop() - file.close() - - def parse(self, rnn_path, filename, validate): - self.path = rnn_path - self.stack = [] - self.validate = validate - self.do_parse(filename) - - def parse_reg(self, attrs, bit_size): - self.current_bitsize = bit_size - if "type" in attrs and attrs["type"] in self.bitsets: - bitset = self.bitsets[attrs["type"]] - if bitset.inline: - self.current_bitset = Bitset(attrs["name"], bitset) - self.current_bitset.inline = True - else: - self.current_bitset = bitset - else: - self.current_bitset = Bitset(attrs["name"], None) - self.current_bitset.inline = True - if "type" in attrs: - self.parse_field(None, attrs) - - variant = self.parse_variants(attrs) - if not variant and self.current_array: - variant = self.current_array.variant - - self.current_reg = Reg(attrs, self.prefix(variant), self.current_array, bit_size) - self.current_reg.bitset = self.current_bitset - self.current_bitset.reg = self.current_reg - - if len(self.stack) == 1: - self.file.append(self.current_reg) - - if variant is not None: - self.add_all_variants(self.current_reg, attrs, variant) - - usages = None - if "usage" in attrs: - usages = attrs["usage"].split(',') - elif self.current_array: - usages = self.current_array.usages - - self.add_all_usages(self.current_reg, usages) - - def start_element(self, name, attrs): - self.cdata = "" - if name == "import": - filename = attrs["file"] - self.do_parse(os.path.join(self.path, filename)) - elif name == "domain": - self.current_domain = attrs["name"] - if "prefix" in attrs: - self.current_prefix = sanitize_variant(self.parse_variants(attrs)) - self.current_prefix_type = attrs["prefix"] - else: - self.current_prefix = None - self.current_prefix_type = None - if "varset" in attrs: - self.current_varset = self.enums[attrs["varset"]] - elif name == "stripe": - self.current_stripe = sanitize_variant(self.parse_variants(attrs)) - elif name == "enum": - self.current_enum_value = 0 - self.current_enum = Enum(attrs["name"]) - self.enums[attrs["name"]] = self.current_enum - if len(self.stack) == 1: - self.file.append(self.current_enum) - elif name == "value": - if "value" in attrs: - value = int(attrs["value"], 0) - else: - value = self.current_enum_value - self.current_enum.values.append((attrs["name"], value)) - elif name == "reg32": - self.parse_reg(attrs, 32) - elif name == "reg64": - self.parse_reg(attrs, 64) - elif name == "array": - self.current_bitsize = 32 - variant = self.parse_variants(attrs) - index_type = self.enums[attrs["index"]] if "index" in attrs else None - self.current_array = Array(attrs, self.prefix(variant), variant, self.current_array, index_type) - if len(self.stack) == 1: - self.file.append(self.current_array) - elif name == "bitset": - self.current_bitset = Bitset(attrs["name"], None) - if "inline" in attrs and attrs["inline"] == "yes": - self.current_bitset.inline = True - self.bitsets[self.current_bitset.name] = self.current_bitset - if len(self.stack) == 1 and not self.current_bitset.inline: - self.file.append(self.current_bitset) - elif name == "bitfield" and self.current_bitset: - self.parse_field(attrs["name"], attrs) - elif name == "database": - self.do_validate(attrs["xsi:schemaLocation"]) - - def end_element(self, name): - if name == "domain": - self.current_domain = None - self.current_prefix = None - self.current_prefix_type = None - elif name == "stripe": - self.current_stripe = None - elif name == "bitset": - self.current_bitset = None - elif name == "reg32": - self.current_reg = None - elif name == "array": - # if the array has no Reg children, push an implicit reg32: - if len(self.current_array.children) == 0: - attrs = { - "name": "REG", - "offset": "0", - } - self.parse_reg(attrs, 32) - self.current_array = self.current_array.parent - elif name == "enum": - self.current_enum = None - - def character_data(self, data): - self.cdata += data - - def dump_reg_usages(self): - d = collections.defaultdict(list) - for usage, regs in self.usage_regs.items(): - for reg in regs: - variants = self.variant_regs.get(reg.name) - if variants: - for variant, vreg in variants.items(): - if reg == vreg: - d[(usage, sanitize_variant(variant))].append(reg) - else: - for variant in self.variants: - d[(usage, sanitize_variant(variant))].append(reg) - - print("#ifdef __cplusplus") - - for usage, regs in self.usage_regs.items(): - print("template<chip CHIP> constexpr inline uint16_t %s_REGS[] = {};" % (usage.upper())) - - for (usage, variant), regs in d.items(): - offsets = [] - - for reg in regs: - if reg.array: - for i in range(reg.array.length): - offsets.append(reg.array.offset + reg.offset + i * reg.array.stride) - if reg.bit_size == 64: - offsets.append(offsets[-1] + 1) - else: - offsets.append(reg.offset) - if reg.bit_size == 64: - offsets.append(offsets[-1] + 1) - - offsets.sort() - - print("template<> constexpr inline uint16_t %s_REGS<%s>[] = {" % (usage.upper(), variant)) - for offset in offsets: - print("\t%s," % hex(offset)) - print("};") - - print("#endif") - - def has_variants(self, reg): - return reg.name in self.variant_regs and not is_number(reg.name) and not is_number(reg.name[1:]) - - def dump(self): - enums = [] - bitsets = [] - regs = [] - for e in self.file: - if isinstance(e, Enum): - enums.append(e) - elif isinstance(e, Bitset): - bitsets.append(e) - else: - regs.append(e) - - for e in enums + bitsets + regs: - e.dump(self.has_variants(e)) - - self.dump_reg_usages() - - - def dump_regs_py(self): - regs = [] - for e in self.file: - if isinstance(e, Reg): - regs.append(e) - - for e in regs: - e.dump_py() - - - def dump_reg_variants(self, regname, variants): - if is_number(regname) or is_number(regname[1:]): - return - print("#ifdef __cplusplus") - print("struct __%s {" % regname) - # TODO be more clever.. we should probably figure out which - # fields have the same type in all variants (in which they - # appear) and stuff everything else in a variant specific - # sub-structure. - seen_fields = [] - bit_size = 32 - array = False - address = None - for variant in variants.keys(): - print(" /* %s fields: */" % variant) - reg = variants[variant] - bit_size = reg.bit_size - array = reg.array - for f in reg.bitset.fields: - fld_name = field_name(reg, f) - if fld_name in seen_fields: - continue - seen_fields.append(fld_name) - name = fld_name.lower() - if f.type in [ "address", "waddress" ]: - if address: - continue - address = f - tab_to(" __bo_type", "bo;") - tab_to(" uint32_t", "bo_offset;") - continue - type, val = f.ctype("var") - tab_to(" %s" %type, "%s;" %name) - print(" /* fallback fields: */") - if bit_size == 64: - tab_to(" uint64_t", "unknown;") - tab_to(" uint64_t", "qword;") - else: - tab_to(" uint32_t", "unknown;") - tab_to(" uint32_t", "dword;") - print("};") - # TODO don't hardcode the varset enum name - varenum = "chip" - print("template <%s %s>" % (varenum, varenum.upper())) - print("static inline struct fd_reg_pair") - xtra = "" - xtravar = "" - if array: - xtra = "int __i, " - xtravar = "__i, " - print("__%s(%sstruct __%s fields) {" % (regname, xtra, regname)) - for variant in variants.keys(): - if "-" in variant: - start = variant[:variant.index("-")] - end = variant[variant.index("-") + 1:] - if end != "": - print(" if ((%s >= %s) && (%s <= %s)) {" % (varenum.upper(), start, varenum.upper(), end)) - else: - print(" if (%s >= %s) {" % (varenum.upper(), start)) - else: - print(" if (%s == %s) {" % (varenum.upper(), variant)) - reg = variants[variant] - reg.dump_regpair_builder() - print(" } else") - print(" assert(!\"invalid variant\");") - print(" return (struct fd_reg_pair){};") - print("}") - - if bit_size == 64: - skip = ", { .reg = 0 }" - else: - skip = "" - - print("#define %s(VARIANT, %s...) __%s<VARIANT>(%s{__VA_ARGS__})%s" % (regname, xtravar, regname, xtravar, skip)) - print("#endif /* __cplusplus */") - - def dump_structs(self): - for e in self.file: - e.dump_pack_struct(self.has_variants(e)) - - for regname in self.variant_regs: - self.dump_reg_variants(regname, self.variant_regs[regname]) + def __init__(self): + self.current_array = None + self.current_domain = None + self.current_prefix = None + self.current_prefix_type = None + self.current_stripe = None + self.current_bitset = None + self.current_bitsize = 32 + # The varset attribute on the domain specifies the enum which + # specifies all possible hw variants: + self.current_varset = None + # Regs that have multiple variants.. we only generated the C++ + # template based struct-packers for these + self.variant_regs = {} + # Information in which contexts regs are used, to be used in + # debug options + self.usage_regs = collections.defaultdict(list) + self.bitsets = {} + self.enums = {} + self.variants = set() + self.file = [] + self.xml_files = [] + + def error(self, message): + parser, filename = self.stack[-1] + return Error("%s:%d:%d: %s" % (filename, parser.CurrentLineNumber, parser.CurrentColumnNumber, message)) + + def prefix(self, variant=None): + if self.current_prefix_type == "variant" and variant: + return sanitize_variant(variant) + elif self.current_stripe: + return self.current_stripe + "_" + self.current_domain + elif self.current_prefix: + return self.current_prefix + "_" + self.current_domain + else: + return self.current_domain + + def parse_field(self, name, attrs): + try: + if "pos" in attrs: + high = low = int(attrs["pos"], 0) + elif "high" in attrs and "low" in attrs: + high = int(attrs["high"], 0) + low = int(attrs["low"], 0) + else: + low = 0 + high = self.current_bitsize - 1 + + if "type" in attrs: + type = attrs["type"] + else: + type = None + + if "shr" in attrs: + shr = int(attrs["shr"], 0) + else: + shr = 0 + + b = Field(name, low, high, shr, type, self) + + if type == "fixed" or type == "ufixed": + b.radix = int(attrs["radix"], 0) + + self.current_bitset.fields.append(b) + except ValueError as e: + raise self.error(e) + + def parse_varset(self, attrs): + # Inherit the varset from the enclosing domain if not overriden: + varset = self.current_varset + if "varset" in attrs: + varset = self.enums[attrs["varset"]] + return varset + + def parse_variants(self, attrs): + if "variants" not in attrs: + return None + + variant = attrs["variants"].split(",")[0] + varset = self.parse_varset(attrs) + + if "-" in variant: + # if we have a range, validate that both the start and end + # of the range are valid enums: + start = variant[:variant.index("-")] + end = variant[variant.index("-") + 1:] + assert varset.has_name(start) + if end != "": + assert varset.has_name(end) + else: + assert varset.has_name(variant) + + return variant + + def add_all_variants(self, reg, attrs, parent_variant): + # TODO this should really handle *all* variants, including dealing + # with open ended ranges (ie. "A2XX,A4XX-") (we have the varset + # enum now to make that possible) + variant = self.parse_variants(attrs) + if not variant: + variant = parent_variant + + if reg.name not in self.variant_regs: + self.variant_regs[reg.name] = {} + else: + # All variants must be same size: + v = next(iter(self.variant_regs[reg.name])) + assert self.variant_regs[reg.name][v].bit_size == reg.bit_size + + self.variant_regs[reg.name][variant] = reg + + def add_all_usages(self, reg, usages): + if not usages: + return + + for usage in usages: + self.usage_regs[usage].append(reg) + + self.variants.add(reg.domain) + + def do_validate(self, schemafile): + if not self.validate: + return + + try: + from lxml import etree + + parser, filename = self.stack[-1] + dirname = os.path.dirname(filename) + + # we expect this to look like <namespace url> schema.xsd.. I think + # technically it is supposed to be just a URL, but that doesn't + # quite match up to what we do.. Just skip over everything up to + # and including the first whitespace character: + schemafile = schemafile[schemafile.rindex(" ")+1:] + + # this is a bit cheezy, but the xml file to validate could be + # in a child director, ie. we don't really know where the schema + # file is, the way the rnn C code does. So if it doesn't exist + # just look one level up + if not os.path.exists(dirname + "/" + schemafile): + schemafile = "../" + schemafile + + if not os.path.exists(dirname + "/" + schemafile): + raise self.error("Cannot find schema for: " + filename) + + xmlschema_doc = etree.parse(dirname + "/" + schemafile) + xmlschema = etree.XMLSchema(xmlschema_doc) + + xml_doc = etree.parse(filename) + if not xmlschema.validate(xml_doc): + error_str = str(xmlschema.error_log.filter_from_errors()[0]) + raise self.error( + "Schema validation failed for: " + filename + "\n" + error_str) + except ImportError as e: + print("lxml not found, skipping validation", file=sys.stderr) + + def do_parse(self, filename): + filepath = os.path.abspath(filename) + if filepath in self.xml_files: + return + self.xml_files.append(filepath) + file = open(filename, "rb") + parser = xml.parsers.expat.ParserCreate() + self.stack.append((parser, filename)) + parser.StartElementHandler = self.start_element + parser.EndElementHandler = self.end_element + parser.CharacterDataHandler = self.character_data + parser.buffer_text = True + parser.ParseFile(file) + self.stack.pop() + file.close() + + def parse(self, rnn_path, filename, validate): + self.path = rnn_path + self.stack = [] + self.validate = validate + self.do_parse(filename) + + def parse_reg(self, attrs, bit_size): + self.current_bitsize = bit_size + if "type" in attrs and attrs["type"] in self.bitsets: + bitset = self.bitsets[attrs["type"]] + if bitset.inline: + self.current_bitset = Bitset(attrs["name"], bitset) + self.current_bitset.inline = True + else: + self.current_bitset = bitset + else: + self.current_bitset = Bitset(attrs["name"], None) + self.current_bitset.inline = True + if "type" in attrs: + self.parse_field(None, attrs) + + variant = self.parse_variants(attrs) + if not variant and self.current_array: + variant = self.current_array.variant + + self.current_reg = Reg(attrs, self.prefix( + variant), self.current_array, bit_size) + self.current_reg.bitset = self.current_bitset + self.current_bitset.reg = self.current_reg + + if len(self.stack) == 1: + self.file.append(self.current_reg) + + if variant is not None: + self.add_all_variants(self.current_reg, attrs, variant) + + usages = None + if "usage" in attrs: + usages = attrs["usage"].split(',') + elif self.current_array: + usages = self.current_array.usages + + self.add_all_usages(self.current_reg, usages) + + def start_element(self, name, attrs): + self.cdata = "" + if name == "import": + filename = attrs["file"] + self.do_parse(os.path.join(self.path, filename)) + elif name == "domain": + self.current_domain = attrs["name"] + if "prefix" in attrs: + self.current_prefix = sanitize_variant( + self.parse_variants(attrs)) + self.current_prefix_type = attrs["prefix"] + else: + self.current_prefix = None + self.current_prefix_type = None + if "varset" in attrs: + self.current_varset = self.enums[attrs["varset"]] + elif name == "stripe": + self.current_stripe = sanitize_variant(self.parse_variants(attrs)) + elif name == "enum": + self.current_enum_value = 0 + self.current_enum = Enum(attrs["name"]) + self.enums[attrs["name"]] = self.current_enum + if len(self.stack) == 1: + self.file.append(self.current_enum) + elif name == "value": + if "value" in attrs: + value = int(attrs["value"], 0) + else: + value = self.current_enum_value + self.current_enum.values.append((attrs["name"], value)) + elif name == "reg32": + self.parse_reg(attrs, 32) + elif name == "reg64": + self.parse_reg(attrs, 64) + elif name == "array": + self.current_bitsize = 32 + variant = self.parse_variants(attrs) + index_type = self.enums[attrs["index"] + ] if "index" in attrs else None + self.current_array = Array(attrs, self.prefix( + variant), variant, self.current_array, index_type) + if len(self.stack) == 1: + self.file.append(self.current_array) + elif name == "bitset": + self.current_bitset = Bitset(attrs["name"], None) + if "inline" in attrs and attrs["inline"] == "yes": + self.current_bitset.inline = True + self.bitsets[self.current_bitset.name] = self.current_bitset + if len(self.stack) == 1 and not self.current_bitset.inline: + self.file.append(self.current_bitset) + elif name == "bitfield" and self.current_bitset: + self.parse_field(attrs["name"], attrs) + elif name == "database": + self.do_validate(attrs["xsi:schemaLocation"]) + + def end_element(self, name): + if name == "domain": + self.current_domain = None + self.current_prefix = None + self.current_prefix_type = None + elif name == "stripe": + self.current_stripe = None + elif name == "bitset": + self.current_bitset = None + elif name == "reg32": + self.current_reg = None + elif name == "array": + # if the array has no Reg children, push an implicit reg32: + if len(self.current_array.children) == 0: + attrs = { + "name": "REG", + "offset": "0", + } + self.parse_reg(attrs, 32) + self.current_array = self.current_array.parent + elif name == "enum": + self.current_enum = None + + def character_data(self, data): + self.cdata += data + + def dump_reg_usages(self): + d = collections.defaultdict(list) + for usage, regs in self.usage_regs.items(): + for reg in regs: + variants = self.variant_regs.get(reg.name) + if variants: + for variant, vreg in variants.items(): + if reg == vreg: + d[(usage, sanitize_variant(variant))].append(reg) + else: + for variant in self.variants: + d[(usage, sanitize_variant(variant))].append(reg) + + print("#ifdef __cplusplus") + + for usage, regs in self.usage_regs.items(): + print("template<chip CHIP> constexpr inline uint16_t %s_REGS[] = {};" % ( + usage.upper())) + + for (usage, variant), regs in d.items(): + offsets = [] + + for reg in regs: + if reg.array: + for i in range(reg.array.length): + offsets.append(reg.array.offset + + reg.offset + i * reg.array.stride) + if reg.bit_size == 64: + offsets.append(offsets[-1] + 1) + else: + offsets.append(reg.offset) + if reg.bit_size == 64: + offsets.append(offsets[-1] + 1) + + offsets.sort() + + print("template<> constexpr inline uint16_t %s_REGS<%s>[] = {" % ( + usage.upper(), variant)) + for offset in offsets: + print("\t%s," % hex(offset)) + print("};") + + print("#endif") + + def has_variants(self, reg): + return reg.name in self.variant_regs and not is_number(reg.name) and not is_number(reg.name[1:]) + + def dump(self): + enums = [] + bitsets = [] + regs = [] + for e in self.file: + if isinstance(e, Enum): + enums.append(e) + elif isinstance(e, Bitset): + bitsets.append(e) + else: + regs.append(e) + + for e in enums + bitsets + regs: + e.dump(self.has_variants(e)) + + self.dump_reg_usages() + + def dump_regs_py(self): + regs = [] + for e in self.file: + if isinstance(e, Reg): + regs.append(e) + + for e in regs: + e.dump_py() + + def dump_reg_variants(self, regname, variants): + if is_number(regname) or is_number(regname[1:]): + return + print("#ifdef __cplusplus") + print("struct __%s {" % regname) + # TODO be more clever.. we should probably figure out which + # fields have the same type in all variants (in which they + # appear) and stuff everything else in a variant specific + # sub-structure. + seen_fields = [] + bit_size = 32 + array = False + address = None + constexpr_mark = " CONSTEXPR" + for variant in variants.keys(): + print(" /* %s fields: */" % variant) + reg = variants[variant] + bit_size = reg.bit_size + array = reg.array + for f in reg.bitset.fields: + fld_name = field_name(reg, f) + if fld_name in seen_fields: + continue + seen_fields.append(fld_name) + name = fld_name.lower() + if f.type in ["address", "waddress"]: + if address: + continue + address = f + print("#ifndef TU_CS_H") + tab_to(" __bo_type", "bo;") + tab_to(" uint32_t", "bo_offset;") + print("#endif") + continue + type, val = f.ctype("var") + tab_to(" %s" % type, "%s;" % name) + if f.type == "float": + constexpr_mark = "" + print(" /* fallback fields: */") + if bit_size == 64: + tab_to(" uint64_t", "unknown;") + tab_to(" uint64_t", "qword;") + else: + tab_to(" uint32_t", "unknown;") + tab_to(" uint32_t", "dword;") + print("};") + # TODO don't hardcode the varset enum name + varenum = "chip" + print("template <%s %s>" % (varenum, varenum.upper())) + print("static%s inline struct fd_reg_pair" % (constexpr_mark)) + xtra = "" + xtravar = "" + if array: + xtra = "int __i, " + xtravar = "__i, " + print("__%s(%sstruct __%s fields) {" % (regname, xtra, regname)) + for variant in variants.keys(): + if "-" in variant: + start = variant[:variant.index("-")] + end = variant[variant.index("-") + 1:] + if end != "": + print(" if ((%s >= %s) && (%s <= %s)) {" % ( + varenum.upper(), start, varenum.upper(), end)) + else: + print(" if (%s >= %s) {" % (varenum.upper(), start)) + else: + print(" if (%s == %s) {" % (varenum.upper(), variant)) + reg = variants[variant] + reg.dump_regpair_builder() + print(" } else") + print(" assert(!\"invalid variant\");") + print(" return (struct fd_reg_pair){};") + print("}") + + if bit_size == 64: + skip = ", { .reg = 0 }" + else: + skip = "" + + print("#define %s(VARIANT, %s...) __%s<VARIANT>(%s{__VA_ARGS__})%s" % ( + regname, xtravar, regname, xtravar, skip)) + print("#endif /* __cplusplus */") + + def dump_structs(self): + for e in self.file: + e.dump_pack_struct(self.has_variants(e)) + + for regname in self.variant_regs: + self.dump_reg_variants(regname, self.variant_regs[regname]) def dump_c(args, guard, func): - p = Parser() - - try: - p.parse(args.rnn, args.xml, args.validate) - except Error as e: - print(e, file=sys.stderr) - exit(1) - - print("#ifndef %s\n#define %s\n" % (guard, guard)) - - print("/* Autogenerated file, DO NOT EDIT manually! */") - - print() - print("#ifdef __KERNEL__") - print("#include <linux/bug.h>") - print("#define assert(x) BUG_ON(!(x))") - print("#else") - print("#include <assert.h>") - print("#endif") - print() - - print("#ifdef __cplusplus") - print("#define __struct_cast(X)") - print("#else") - print("#define __struct_cast(X) (struct X)") - print("#endif") - print() - - print("#ifndef FD_NO_DEPRECATED_PACK") - print("#define FD_DEPRECATED __attribute__((deprecated))") - print("#else") - print("#define FD_DEPRECATED") - print("#endif") - print() - - func(p) - - print() - print("#undef FD_DEPRECATED") - print() - - print("#endif /* %s */" % guard) + p = Parser() + + try: + p.parse(args.rnn, args.xml, args.validate) + except Error as e: + print(e, file=sys.stderr) + exit(1) + + print("#ifndef %s\n#define %s\n" % (guard, guard)) + + print("/* Autogenerated file, DO NOT EDIT manually! */") + + print() + print("#ifdef __KERNEL__") + print("#include <linux/bug.h>") + print("#define assert(x) BUG_ON(!(x))") + print("#else") + print("#include <assert.h>") + print("#endif") + print() + + print("#ifdef __cplusplus") + print("#define __struct_cast(X)") + print("#define CONSTEXPR constexpr") + print("#else") + print("#define __struct_cast(X) (struct X)") + print("#define CONSTEXPR") + print("#endif") + print() + + # TODO figure out what to do about fd_reg_stomp_allowed() + # vs gcc.. for now only enable the warnings with clang: + print("#if defined(__clang__) && !defined(FD_NO_DEPRECATED_PACK)") + print("#define __FD_DEPRECATED _Pragma (\"GCC warning \\\"Deprecated reg builder\\\"\")") + print("#else") + print("#define __FD_DEPRECATED") + print("#endif") + print() + + func(p) + + print("#endif /* %s */" % guard) def dump_c_defines(args): - guard = str.replace(os.path.basename(args.xml), '.', '_').upper() - dump_c(args, guard, lambda p: p.dump()) + guard = str.replace(os.path.basename(args.xml), '.', '_').upper() + dump_c(args, guard, lambda p: p.dump()) def dump_c_pack_structs(args): - guard = str.replace(os.path.basename(args.xml), '.', '_').upper() + '_STRUCTS' - dump_c(args, guard, lambda p: p.dump_structs()) - + guard = str.replace(os.path.basename(args.xml), + '.', '_').upper() + '_STRUCTS' + dump_c(args, guard, lambda p: p.dump_structs()) + + +def dump_perfcntrs(args): + p = Parser() + + try: + p.parse(args.rnn, args.xml, args.validate) + except Error as e: + print(e, file=sys.stderr) + exit(1) + + perfcntrs = json.load(open(args.json, "r", encoding="utf-8")) + + chip_type = p.enums['chip'] + chip = perfcntrs['chip'] + if not chip_type.has_name(chip): + raise Error("Invalid chip: " + chip) + + groups = perfcntrs['groups'] + + guard = "__" + chip + "_PERFCNTRS_" + print("#ifndef %s\n#define %s\n" % (guard, guard)) + print("/* Autogenerated file, DO NOT EDIT manually! */") + print() + print("#ifdef __KERNEL__") + print("#include \"msm_perfcntr.h\"") + print("#endif") + print() + + def has_variant(variant): + if variant is None: + return True + if "-" in variant: + start = chip_type.value(variant[:variant.index("-")]) + end = chip_type.value(variant[variant.index("-") + 1:]) + chipn = chip_type.value(chip) + + return (start is None or chipn >= start) and (end is None or chipn <= end) + return chip == variant + + # Split out arrays and regs for later access: + arrays = {} + regs = {} + for e in p.file: + if isinstance(e, Array) and has_variant(e.variant): + arrays[e.local_name] = e + if isinstance(e, Reg): + regs[e.name] = e + + # For variant regs, overwrite 'regs' entries with correct variant: + for regname in p.variant_regs: + for (variant, reg) in p.variant_regs[regname].items(): + if has_variant(variant): + regs[regname] = reg + break + + for group in groups: + name = group['name'] + name_low = name.lower() + num = group['num'] + countable_type_name = group['countable_type'] + + if not countable_type_name in p.enums: + raise Error("Invalid type: " + countable_type_name) + + countable_type = p.enums[countable_type_name] + + print("#ifndef __KERNEL__") + print("static const struct fd_perfcntr_countable " + name_low + "_countables[] = {") + for (name, value) in countable_type.values: + # if the countable is prefixed with the chip, strip that: + # (note: avoid py3.9 dependency for kernel) + if name.startswith(chip + "_"): + name = name[len(chip)+1:] + print(" { \"" + name + "\", " + str(value) + " },") + print("};") + print("#endif") + + print("static const struct fd_perfcntr_counter " + name_low + "_counters[] = {") + for i in range(0, num): + if "reserved" in group and i in group["reserved"]: + continue + def get_reg(name): + # if reg has {} pattern, expand that first: + name = name.format(i) + + if name in arrays: + arr = arrays[name] + return arr.offset + (i * arr.stride) + + if not name in regs: + raise Error("Invalid reg: " + name) + + reg = regs[name] + return reg.offset + + def get_counter(): + # if the counter is <reg64> just a single "counter" value + # should be specified in the json, but for legacy separate + # hi/lo <reg32> pairs "counter_lo" and "counter_hi" should + # be specified + if "counter" in group: + counter = get_reg(group["counter"]) + return [counter, counter+1] + counter_lo = get_reg(group["counter_lo"]) + counter_hi = get_reg(group["counter_hi"]) + return [counter_lo, counter_hi] + + (counter_lo, counter_hi) = get_counter() + select = get_reg(group['select']) + + select_offset = 0 + if "select_offset" in group: + select_offset = int(group["select_offset"]) + select = select + select_offset + + slice_select_str = "" + if "slice_select" in group: + slice_select = group["slice_select"] + for reg in slice_select: + val = get_reg(reg) + select_offset + slice_select_str += "0x%04x, " % val + + # TODO add support for things that need enable/clear regs + + print(" { 0x%04x, {%s}, 0x%04x, 0x%04x }," % (select, slice_select_str, counter_lo, counter_hi)) + print("};") + + print() + + print("const struct fd_perfcntr_group " + chip.lower() + "_perfcntr_groups[] = {") + for group in groups: + name = group['name'] + name_low = name.lower() + pipe = 'NONE' + if 'pipe' in group: + pipe = group['pipe'] + + print(" GROUP(\"%s\", PIPE_%s, %s_counters, %s_countables)," % (name, pipe, name_low, name_low)) + + print("};") + print("const unsigned " + chip.lower() + "_num_perfcntr_groups = ARRAY_SIZE(" + chip.lower() + "_perfcntr_groups);") + + print() + print("#endif /* %s */" % guard) def dump_py_defines(args): - p = Parser() + p = Parser() - try: - p.parse(args.rnn, args.xml, args.validate) - except Error as e: - print(e, file=sys.stderr) - exit(1) + try: + p.parse(args.rnn, args.xml, args.validate) + except Error as e: + print(e, file=sys.stderr) + exit(1) - file_name = os.path.splitext(os.path.basename(args.xml))[0] + file_name = os.path.splitext(os.path.basename(args.xml))[0] - print("from enum import IntEnum") - print("class %sRegs(IntEnum):" % file_name.upper()) + print("from enum import IntEnum") + print("class %sRegs(IntEnum):" % file_name.upper()) - os.path.basename(args.xml) + os.path.basename(args.xml) - p.dump_regs_py() + p.dump_regs_py() def main(): - parser = argparse.ArgumentParser() - parser.add_argument('--rnn', type=str, required=True) - parser.add_argument('--xml', type=str, required=True) - parser.add_argument('--validate', default=False, action='store_true') - parser.add_argument('--no-validate', dest='validate', action='store_false') + parser = argparse.ArgumentParser() + parser.add_argument('--rnn', type=str, required=True) + parser.add_argument('--xml', type=str, required=True) + parser.add_argument('--validate', default=False, action='store_true') + parser.add_argument('--no-validate', dest='validate', action='store_false') + + subparsers = parser.add_subparsers() + subparsers.required = True - subparsers = parser.add_subparsers() - subparsers.required = True + parser_c_defines = subparsers.add_parser('c-defines') + parser_c_defines.set_defaults(func=dump_c_defines) - parser_c_defines = subparsers.add_parser('c-defines') - parser_c_defines.set_defaults(func=dump_c_defines) + parser_c_pack_structs = subparsers.add_parser('c-pack-structs') + parser_c_pack_structs.set_defaults(func=dump_c_pack_structs) - parser_c_pack_structs = subparsers.add_parser('c-pack-structs') - parser_c_pack_structs.set_defaults(func=dump_c_pack_structs) + parser_perfcntrs = subparsers.add_parser('perfcntrs') + parser_perfcntrs.add_argument('--json', type=str, required=True) + parser_perfcntrs.set_defaults(func=dump_perfcntrs) - parser_py_defines = subparsers.add_parser('py-defines') - parser_py_defines.set_defaults(func=dump_py_defines) + parser_py_defines = subparsers.add_parser('py-defines') + parser_py_defines.set_defaults(func=dump_py_defines) - args = parser.parse_args() - args.func(args) + args = parser.parse_args() + args.func(args) if __name__ == '__main__': - main() + main()
Signature · 1 line
2.54.0
Rob Clark
maintainerPATCH v3 5/16 drm/msm/registers: Add perfcntr json
In reply to Rob ClarkPull in perfcntr json and wire up generation of perfcntr tables.
Sync from https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/40522
--- drivers/gpu/drm/msm/Makefile | 25 +- drivers/gpu/drm/msm/msm_perfcntr.h | 48 ++++ .../msm/registers/adreno/a2xx_perfcntrs.json | 109 ++++++++ .../msm/registers/adreno/a5xx_perfcntrs.json | 128 ++++++++++ .../msm/registers/adreno/a6xx_perfcntrs.json | 105 ++++++++ .../msm/registers/adreno/a7xx_perfcntrs.json | 228 +++++++++++++++++ .../msm/registers/adreno/a8xx_perfcntrs.json | 240 ++++++++++++++++++ 7 files changed, 882 insertions(+), 1 deletion(-) create mode 100644 drivers/gpu/drm/msm/msm_perfcntr.h create mode 100644 drivers/gpu/drm/msm/registers/adreno/a2xx_perfcntrs.json create mode 100644 drivers/gpu/drm/msm/registers/adreno/a5xx_perfcntrs.json create mode 100644 drivers/gpu/drm/msm/registers/adreno/a6xx_perfcntrs.json create mode 100644 drivers/gpu/drm/msm/registers/adreno/a7xx_perfcntrs.json create mode 100644 drivers/gpu/drm/msm/registers/adreno/a8xx_perfcntrs.json
drivers/gpu/drm/msm/Makefile +24 -1
@@ -176,6 +176,11 @@ quiet_cmd_headergen = GENHDR $@ cmd_headergen = mkdir -p $(obj)/generated && $(PYTHON3) $(src)/registers/gen_header.py \ $(headergen-opts) --rnn $(src)/registers --xml $< c-defines > $@ +# TODO how to do this for a2xx/a5xx which have different .xml arg? +quiet_cmd_headergen_json = GENHDRJSN $@ + cmd_headergen_json = mkdir -p $(obj)/generated && $(PYTHON3) $(src)/registers/gen_header.py \ + $(headergen-opts) --rnn $(src)/registers --xml $(filter %.xml,$^) perfcntrs --json $< > $@ + $(obj)/generated/%.xml.h: $(src)/registers/adreno/%.xml \ $(src)/registers/adreno/adreno_common.xml \ $(src)/registers/adreno/adreno_pm4.xml \
@@ -192,6 +197,24 @@ $(obj)/generated/%.xml.h: $(src)/registers/display/%.xml \ FORCE $(call if_changed,headergen) +ADRENO_PERFCNTRS = + +define adreno_perfcntrs +ADRENO_PERFCNTRS += generated/$(1)_perfcntrs.json.c +$$(obj)/generated/$(1)_perfcntrs.json.c: $$(src)/registers/adreno/$(1)_perfcntrs.json \ + $$(src)/registers/adreno/$(2).xml \ + FORCE + $$(call if_changed,headergen_json) +endef + +$(eval $(call adreno_perfcntrs,a2xx,a2xx)) +$(eval $(call adreno_perfcntrs,a5xx,a5xx)) +$(eval $(call adreno_perfcntrs,a6xx,a6xx)) +$(eval $(call adreno_perfcntrs,a7xx,a6xx)) +$(eval $(call adreno_perfcntrs,a8xx,a6xx)) + +adreno-y += $(ADRENO_PERFCNTRS:.c=.o) + ADRENO_HEADERS = \ generated/a2xx.xml.h \ generated/a3xx.xml.h \
@@ -226,4 +249,4 @@ DISPLAY_HEADERS = \ $(addprefix $(obj)/,$(adreno-y)): $(addprefix $(obj)/,$(ADRENO_HEADERS)) $(addprefix $(obj)/,$(msm-display-y)): $(addprefix $(obj)/,$(DISPLAY_HEADERS)) -targets += $(ADRENO_HEADERS) $(DISPLAY_HEADERS) +targets += $(ADRENO_HEADERS) $(DISPLAY_HEADERS) $(ADRENO_PERFCNTRS)
drivers/gpu/drm/msm/msm_perfcntr.h new file +48 -0
48 lines not on this page · raw message
drivers/gpu/drm/msm/registers/adreno/a2xx_perfcntrs.json new file +109 -0
109 lines not on this page · raw message
drivers/gpu/drm/msm/registers/adreno/a5xx_perfcntrs.json new file +128 -0
128 lines not on this page · raw message
drivers/gpu/drm/msm/registers/adreno/a6xx_perfcntrs.json new file +105 -0
105 lines not on this page · raw message
drivers/gpu/drm/msm/registers/adreno/a7xx_perfcntrs.json new file +228 -0
228 lines not on this page · raw message
drivers/gpu/drm/msm/registers/adreno/a8xx_perfcntrs.json new file +240 -0
240 lines not on this page · raw message
Signature · 1 line
2.54.0
Rob Clark
maintainerPATCH v3 6/16 drm/msm: Add a6xx+ perfcntr tables
In reply to Rob ClarkWire up the generated perfcntr tables for a6xx+. The PERFCNTR_CONFIG ioctl will use this information to assign counters.
--- drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 15 +++++++++++++++ drivers/gpu/drm/msm/msm_gpu.h | 4 ++++ drivers/gpu/drm/msm/msm_perfcntr.h | 9 +++++++++ 3 files changed, 28 insertions(+)
drivers/gpu/drm/msm/adreno/a6xx_gpu.c +15 -0
@@ -5,6 +5,7 @@#include "msm_gem.h" #include "msm_mmu.h" #include "msm_gpu_trace.h" +#include "msm_perfcntr.h" #include "a6xx_gpu.h" #include "a6xx_gmu.xml.h"
@@ -2637,6 +2638,20 @@ static struct msm_gpu *a6xx_gpu_init(struct drm_device *dev) adreno_gpu = &a6xx_gpu->base; gpu = &adreno_gpu->base; +if ((ADRENO_6XX_GEN1 <= config->info->family) && + (config->info->family <=ADRENO_6XX_GEN4 )) { + gpu->perfcntr_groups = a6xx_perfcntr_groups; + gpu->num_perfcntr_groups = a6xx_num_perfcntr_groups; + }else if ((ADRENO_7XX_GEN1 <= config->info->family) && + (config->info->family <=ADRENO_7XX_GEN3 )) { + gpu->perfcntr_groups = a7xx_perfcntr_groups; + gpu->num_perfcntr_groups = a7xx_num_perfcntr_groups; + }else if ((ADRENO_8XX_GEN1 <= config->info->family) && + (config->info->family <=ADRENO_8XX_GEN2 )) { + gpu->perfcntr_groups = a8xx_perfcntr_groups; + gpu->num_perfcntr_groups = a8xx_num_perfcntr_groups; + } +mutex_init (&a6xx_gpu->gmu.lock);spin_lock_init (&a6xx_gpu->aperture_lock);
drivers/gpu/drm/msm/msm_gpu.h +4 -0
@@ -24,6 +24,7 @@ struct msm_gem_submit;struct msm_gem_vm_log_entry ;struct msm_gpu_state ;struct msm_context ; +struct msm_perfcntr_group ;struct msm_gpu_config {const char *ioname;
@@ -262,6 +263,9 @@ struct msm_gpu {bool allow_relocs;struct thermal_cooling_device *cooling; + +const struct msm_perfcntr_group *perfcntr_groups; +unsigned num_perfcntr_groups; };static inline struct msm_gpu *dev_to_gpu (struct device *dev)
drivers/gpu/drm/msm/msm_perfcntr.h +9 -0
@@ -35,6 +35,15 @@ struct msm_perfcntr_group {const struct msm_perfcntr_counter *counters; }; +extern const struct msm_perfcntr_group a6xx_perfcntr_groups[]; +extern const unsigned a6xx_num_perfcntr_groups; + +extern const struct msm_perfcntr_group a7xx_perfcntr_groups[]; +extern const unsigned a7xx_num_perfcntr_groups; + +extern const struct msm_perfcntr_group a8xx_perfcntr_groups[]; +extern const unsigned a8xx_num_perfcntr_groups; +#define GROUP (_name, _pipe, _counters, _countables) { \ .name = _name, \ .pipe = _pipe, \
Signature · 1 line
2.54.0
Rob Clark
maintainerPATCH v3 7/16 drm/msm: Add sysprof accessors
In reply to Rob ClarkCurrently the sysprof param serves two functions, (a) disabling perfcntr clearing on context switch/preemption, and (b) disabling IFPC. In the future, with kernel side global perfcntr collection/stream, the decision about disabling IFPC will change.
To prepare for this, split out two helpers/accessors for the two different cases. For now, they are the same thing, but this will change.
--- drivers/gpu/drm/msm/adreno/a6xx_gmu.c | 8 +++----- drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 5 +++-- drivers/gpu/drm/msm/adreno/a6xx_preempt.c | 2 +- drivers/gpu/drm/msm/adreno/a8xx_gpu.c | 2 +- drivers/gpu/drm/msm/adreno/a8xx_preempt.c | 2 +- drivers/gpu/drm/msm/msm_gpu.h | 18 ++++++++++++++++++ 6 files changed, 27 insertions(+), 10 deletions(-)
drivers/gpu/drm/msm/adreno/a6xx_gmu.c +3 -5
@@ -2036,10 +2036,10 @@ static int a6xx_gmu_get_irq(struct a6xx_gmu *gmu, struct platform_device *pdev,void a6xx_gmu_sysprof_setup (struct msm_gpu *gpu) { +bool sysprof =msm_gpu_sysprof_no_ifpc (gpu);struct adreno_gpu *adreno_gpu =to_adreno_gpu (gpu);struct a6xx_gpu *a6xx_gpu =to_a6xx_gpu (adreno_gpu);struct a6xx_gmu *gmu = &a6xx_gpu->gmu; -unsigned int sysprof_active;/* Nothing to do if GPU is suspended. We will handle this during GMU resume */ if (!pm_runtime_get_if_active (&gpu->pdev->dev))
@@ -2047,15 +2047,13 @@ void a6xx_gmu_sysprof_setup(struct msm_gpu *gpu)mutex_lock (&gmu->lock); - sysprof_active =refcount_read (&gpu->sysprof_active); -/* * 'Perfcounter select' register values are lost during IFPC collapse. To avoid that, * use the currently unused perfcounter oob vote to block IFPC when sysprof is active */ -if ((sysprof_active >1 ) && !test_and_set_bit (GMU_STATUS_OOB_PERF_SET , &gmu->status)) +if (sysprof && !test_and_set_bit (GMU_STATUS_OOB_PERF_SET , &gmu->status))a6xx_gmu_set_oob (gmu,GMU_OOB_PERFCOUNTER_SET ); -else if ((sysprof_active ==1 ) &&test_and_clear_bit (GMU_STATUS_OOB_PERF_SET , &gmu->status)) +else if (!sysprof &&test_and_clear_bit (GMU_STATUS_OOB_PERF_SET , &gmu->status))a6xx_gmu_clear_oob (gmu,GMU_OOB_PERFCOUNTER_SET );mutex_unlock (&gmu->lock);
drivers/gpu/drm/msm/adreno/a6xx_gpu.c +3 -2
@@ -203,7 +203,7 @@ static void get_stats_counter(struct msm_ringbuffer *ring, u32 counter,static void a6xx_set_pagetable (struct a6xx_gpu *a6xx_gpu,struct msm_ringbuffer *ring,struct msm_gem_submit *submit) { -bool sysprof =refcount_read (&a6xx_gpu->base.base.sysprof_active) >1 ; +bool sysprof =msm_gpu_sysprof_no_perfcntr_zap (&a6xx_gpu->base.base);struct msm_context *ctx = submit->queue->ctx;struct drm_gpuvm *vm =msm_context_vm (submit->dev, ctx);struct adreno_gpu *adreno_gpu = &a6xx_gpu->base;
@@ -1608,7 +1608,7 @@ static int hw_init(struct msm_gpu *gpu)a6xx_gmu_clear_oob (&a6xx_gpu->gmu,GMU_OOB_BOOT_SLUMBER ); } -if (!ret && (refcount_read (&gpu->sysprof_active) >1 )) { +if (!ret &&msm_gpu_sysprof_no_ifpc (gpu)) { ret =a6xx_gmu_set_oob (gmu,GMU_OOB_PERFCOUNTER_SET );if (!ret)set_bit (GMU_STATUS_OOB_PERF_SET , &gmu->status);
@@ -2854,6 +2854,7 @@ const struct adreno_gpu_funcs a8xx_gpu_funcs = { .create_private_vm = a6xx_create_private_vm, .get_rptr = a6xx_get_rptr, .progress = a8xx_progress, + .sysprof_setup = a6xx_gmu_sysprof_setup, }, .init = a6xx_gpu_init, .get_timestamp = a8xx_gmu_get_timestamp,
drivers/gpu/drm/msm/adreno/a6xx_preempt.c +1 -1
@@ -261,7 +261,7 @@ void a6xx_preempt_trigger(struct msm_gpu *gpu)mod_timer (&a6xx_gpu->preempt_timer, jiffies +msecs_to_jiffies (10000 ));/* Enable or disable postamble as needed */ - sysprof =refcount_read (&a6xx_gpu->base.base.sysprof_active) >1 ; + sysprof =msm_gpu_sysprof_no_perfcntr_zap (gpu);if (!sysprof && !a6xx_gpu->postamble_enabled)preempt_prepare_postamble (a6xx_gpu);
drivers/gpu/drm/msm/adreno/a8xx_gpu.c +1 -1
@@ -849,7 +849,7 @@ static int hw_init(struct msm_gpu *gpu)*/ a6xx_gmu_clear_oob (&a6xx_gpu->gmu,GMU_OOB_GPU_SET ); -if (!ret && (refcount_read (&gpu->sysprof_active) >1 )) { +if (!ret &&msm_gpu_sysprof_no_perfcntr_zap (gpu)) { ret =a6xx_gmu_set_oob (gmu,GMU_OOB_PERFCOUNTER_SET );if (!ret)set_bit (GMU_STATUS_OOB_PERF_SET , &gmu->status);
drivers/gpu/drm/msm/adreno/a8xx_preempt.c +1 -1
@@ -242,7 +242,7 @@ void a8xx_preempt_trigger(struct msm_gpu *gpu)mod_timer (&a6xx_gpu->preempt_timer, jiffies +msecs_to_jiffies (10000 ));/* Enable or disable postamble as needed */ - sysprof =refcount_read (&a6xx_gpu->base.base.sysprof_active) >1 ; + sysprof =msm_gpu_sysprof_no_perfcntr_zap (gpu);if (!sysprof && !a6xx_gpu->postamble_enabled)preempt_prepare_postamble (a6xx_gpu);
drivers/gpu/drm/msm/msm_gpu.h +18 -0
@@ -311,6 +311,24 @@ static inline bool msm_gpu_active(struct msm_gpu *gpu)return false; } +static inline bool +msm_gpu_sysprof_no_perfcntr_zap (struct msm_gpu *gpu) +{ +return refcount_read (&gpu->sysprof_active) >1 ; +} + +static inline bool +msm_gpu_sysprof_no_ifpc (struct msm_gpu *gpu) +{ +/* +* For now, this is the same condition as disabling perfcntr clears +* on context switch. But once kernel perfcntr IFPC support is in +* place, we will only need to disable IFPC for legacy userspace +* setting SYSPROF param. +*/ +return msm_gpu_sysprof_no_perfcntr_zap (gpu); +} +/* * The number of priority levels provided by drm gpu scheduler. The * DRM_SCHED_PRIORITY_KERNEL priority level is treated specially in some
Signature · 1 line
2.54.0
Rob Clark
maintainerPATCH v3 8/16 drm/msm/a6xx: Add yield & flush helper
In reply to Rob ClarkIt's a common pattern, needing to insert a yield packet before flushing the rb. And we'll need this once again for configuring perfcntr SEL regs. So add a helper.
--- drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 55 +++++++++++++-------------- drivers/gpu/drm/msm/adreno/a6xx_gpu.h | 1 + drivers/gpu/drm/msm/adreno/a8xx_gpu.c | 10 +---- 3 files changed, 28 insertions(+), 38 deletions(-)
drivers/gpu/drm/msm/adreno/a6xx_gpu.c +26 -29
@@ -189,6 +189,30 @@ void a6xx_flush(struct msm_gpu *gpu, struct msm_ringbuffer *ring)spin_unlock_irqrestore (&ring->preempt_lock, flags); } +void +a6xx_flush_yield (struct msm_gpu *gpu,struct msm_ringbuffer *ring) +{ +/* If preemption is enabled */ +if (gpu->nr_rings >1 ) { +/* Yield the floor on command completion */ +OUT_PKT7 (ring,CP_CONTEXT_SWITCH_YIELD ,4 ); + +/* +* If dword[2:1] are non zero, they specify an address for +* the CP to write the value of dword[3] to on preemption +* complete. Write 0 to skip the write +*/ +OUT_RING (ring,0x00 ); +OUT_RING (ring,0x00 ); +/* Data value - not used if the address above is 0 */ +OUT_RING (ring,0x01 ); +/* generate interrupt on preemption completion */ +OUT_RING (ring,0x00 ); + } + +a6xx_flush (gpu, ring); +} +static void get_stats_counter (struct msm_ringbuffer *ring,u32 counter,u64 iova) {
@@ -597,28 +621,9 @@ static void a7xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit)OUT_PKT7 (ring,CP_SET_MARKER ,1 );OUT_RING (ring,0x100 );/* IFPC enable */ -/* If preemption is enabled */ -if (gpu->nr_rings >1 ) { -/* Yield the floor on command completion */ -OUT_PKT7 (ring,CP_CONTEXT_SWITCH_YIELD ,4 ); - -/* -* If dword[2:1] are non zero, they specify an address for -* the CP to write the value of dword[3] to on preemption -* complete. Write 0 to skip the write -*/ -OUT_RING (ring,0x00 ); -OUT_RING (ring,0x00 ); -/* Data value - not used if the address above is 0 */ -OUT_RING (ring,0x01 ); -/* generate interrupt on preemption completion */ -OUT_RING (ring,0x00 ); - } - -trace_msm_gpu_submit_flush (submit, adreno_gpu->funcs->get_timestamp (gpu)); -a6xx_flush (gpu, ring); +a6xx_flush_yield (gpu, ring);/* Check to see if we need to start preemption */ if (adreno_is_a8xx (adreno_gpu))
@@ -958,15 +963,7 @@ static int a7xx_preempt_start(struct msm_gpu *gpu)a6xx_emit_set_pseudo_reg (ring, a6xx_gpu,NULL ); -/* Yield the floor on command completion */ -OUT_PKT7 (ring,CP_CONTEXT_SWITCH_YIELD ,4 ); -OUT_RING (ring,0x00 ); -OUT_RING (ring,0x00 ); -OUT_RING (ring,0x00 ); -/* Generate interrupt on preemption completion */ -OUT_RING (ring,0x00 ); - -a6xx_flush (gpu, ring); +a6xx_flush_yield (gpu, ring);return a6xx_idle (gpu, ring) ?0 : -EINVAL ; }
drivers/gpu/drm/msm/adreno/a6xx_gpu.h +1 -0
@@ -317,6 +317,7 @@ void a6xx_bus_clear_pending_transactions(struct adreno_gpu *adreno_gpu, bool gx_void a6xx_gpu_sw_reset (struct msm_gpu *gpu,bool assert);int a6xx_fenced_write (struct a6xx_gpu *gpu,u32 offset,u64 value,u32 mask,bool is_64b);void a6xx_flush (struct msm_gpu *gpu,struct msm_ringbuffer *ring); +void a6xx_flush_yield (struct msm_gpu *gpu,struct msm_ringbuffer *ring);int a6xx_zap_shader_init (struct msm_gpu *gpu);void a8xx_bus_clear_pending_transactions (struct adreno_gpu *adreno_gpu,bool gx_off);
drivers/gpu/drm/msm/adreno/a8xx_gpu.c +1 -9
@@ -488,15 +488,7 @@ static int a8xx_preempt_start(struct msm_gpu *gpu)a6xx_emit_set_pseudo_reg (ring, a6xx_gpu,NULL ); -/* Yield the floor on command completion */ -OUT_PKT7 (ring,CP_CONTEXT_SWITCH_YIELD ,4 ); -OUT_RING (ring,0x00 ); -OUT_RING (ring,0x00 ); -OUT_RING (ring,0x00 ); -/* Generate interrupt on preemption completion */ -OUT_RING (ring,0x00 ); - -a6xx_flush (gpu, ring); +a6xx_flush_yield (gpu, ring);return a8xx_idle (gpu, ring) ?0 : -EINVAL ; }
Signature · 1 line
2.54.0
Rob Clark
maintainerPATCH v3 9/16 drm/msm: Add per-context perfcntr state
In reply to Rob ClarkThe upcoming PERFCNTR_CONFIG ioctl will allow for both global counter collection, and per-context counter reservation for local (ie. within a single GEM_SUBMIT ioctl) counter collection.
Any number of contexts can reserve the same counters, but we will need to ensure that counters reserved for local counter collection do not conflict with counters used for global counter collection.
So add tracking for per-context local counter reservations.
--- drivers/gpu/drm/msm/msm_gpu.h | 5 +++++ drivers/gpu/drm/msm/msm_perfcntr.h | 21 +++++++++++++++++++++ drivers/gpu/drm/msm/msm_submitqueue.c | 1 + 3 files changed, 27 insertions(+)
drivers/gpu/drm/msm/msm_gpu.h +5 -0
@@ -434,6 +434,11 @@ struct msm_context {* this context. */ atomic64_t ctx_mem; + +/** +* @perfcntrs: Per-context reserved perfcntrs state +*/ +struct msm_perfcntr_context_state *perfctx; };struct drm_gpuvm *msm_context_vm (struct drm_device *dev,struct msm_context *ctx);
drivers/gpu/drm/msm/msm_perfcntr.h +21 -0
@@ -35,6 +35,27 @@ struct msm_perfcntr_group {const struct msm_perfcntr_counter *counters; }; +/** +* struct msm_perfcntr_context_state - per-msm_context counter state +* +* A given counter can either be unused, reserved for global counter +* collection exclusively, or reserved for local per-context counter +* collection inclusively. Multiple contexts can reserve the same +* counter, since SEL reg programming and counter begin/end sampling +* happen locally (within a single GEM_SUBMIT ioctl). +*/ +struct msm_perfcntr_context_state { +/** @dummy: Some compilers dislike structs with only a flex array */ +unsigned dummy; + +/** +* @reserved_counters: +* +* The number of reserved counters indexed by perfcntr group. +*/ +unsigned reserved_counters[]; +}; +extern const struct msm_perfcntr_group a6xx_perfcntr_groups[];extern const unsigned a6xx_num_perfcntr_groups;
drivers/gpu/drm/msm/msm_submitqueue.c +1 -0
@@ -66,6 +66,7 @@ void __msm_context_destroy(struct kref *kref)drm_gpuvm_put (ctx->vm);kfree (ctx->comm);kfree (ctx->cmdline); +kfree (ctx->perfctx);kfree (ctx); }
Signature · 1 line
2.54.0
Rob Clark
maintainerPATCH v3 10/16 drm/msm: Add basic perfcntr infrastructure
In reply to Rob ClarkAdd the basic infrastructure for tracking assigned perfcntrs.
--- drivers/gpu/drm/msm/Makefile | 1 + drivers/gpu/drm/msm/adreno/adreno_device.c | 8 +- drivers/gpu/drm/msm/adreno/adreno_gpu.c | 5 +- drivers/gpu/drm/msm/msm_drv.h | 6 ++ drivers/gpu/drm/msm/msm_gpu.c | 12 +++ drivers/gpu/drm/msm/msm_gpu.h | 57 +++++++++- drivers/gpu/drm/msm/msm_perfcntr.c | 120 +++++++++++++++++++++ drivers/gpu/drm/msm/msm_perfcntr.h | 23 ++++ 8 files changed, 226 insertions(+), 6 deletions(-) create mode 100644 drivers/gpu/drm/msm/msm_perfcntr.c
drivers/gpu/drm/msm/Makefile +1 -0
@@ -122,6 +122,7 @@ msm-y += \ msm_gpu_devfreq.o \ msm_io_utils.o \ msm_iommu.o \ + msm_perfcntr.o \ msm_rd.o \ msm_ringbuffer.o \ msm_submitqueue.o \
drivers/gpu/drm/msm/adreno/adreno_device.c +6 -2
@@ -307,8 +307,10 @@ MODULE_DEVICE_TABLE(of, dt_match);static int adreno_runtime_resume (struct device *dev) {struct msm_gpu *gpu =dev_to_gpu (dev); - -return gpu->funcs->pm_resume (gpu); +int ret = gpu->funcs->pm_resume (gpu); +if (!ret) + ret =msm_perfcntr_resume (gpu); +return ret; }static int adreno_runtime_suspend (struct device *dev)
@@ -322,6 +324,8 @@ static int adreno_runtime_suspend(struct device *dev)*/ WARN_ON_ONCE (gpu->active_submits); +msm_perfcntr_suspend (gpu); +return gpu->funcs->pm_suspend (gpu); }
drivers/gpu/drm/msm/adreno/adreno_gpu.c +2 -3
@@ -702,11 +702,10 @@ void adreno_recover(struct msm_gpu *gpu)struct drm_device *dev = gpu->dev;int ret; -// XXX pm-runtime?? we *need* the device to be off after this -// so maybe continuing to call ->pm_suspend/resume() is better? - +msm_perfcntr_suspend (gpu); gpu->funcs->pm_suspend (gpu); gpu->funcs->pm_resume (gpu); +msm_perfcntr_resume (gpu); ret =msm_gpu_hw_init (gpu);if (ret) {
drivers/gpu/drm/msm/msm_drv.h +6 -0
@@ -235,6 +235,12 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,int msm_ioctl_vm_bind (struct drm_device *dev,void *data,struct drm_file *file); +int msm_perfcntr_resume (struct msm_gpu *gpu); +void msm_perfcntr_suspend (struct msm_gpu *gpu); + +struct msm_perfcntr_state *msm_perfcntr_init (struct msm_gpu *gpu); +void msm_perfcntr_cleanup (struct msm_gpu *gpu); +#ifdef CONFIG_DEBUG_FS unsigned long msm_gem_shrinker_shrink (struct drm_device *dev,unsigned long nr_to_scan);#endif
drivers/gpu/drm/msm/msm_gpu.c +12 -0
@@ -1028,6 +1028,17 @@ int msm_gpu_init(struct drm_device *drm, struct platform_device *pdev,refcount_set (&gpu->sysprof_active,1 ); +mutex_init (&gpu->perfcntr_lock); + +if (gpu->num_perfcntr_groups >0 ) { + gpu->perfcntrs =msm_perfcntr_init (gpu); +if (IS_ERR (gpu->perfcntrs)) { + ret =PTR_ERR (gpu->perfcntrs); + gpu->perfcntrs =NULL ; + goto fail; + } + } +return 0 ; fail:
@@ -1066,6 +1077,7 @@ void msm_gpu_cleanup(struct msm_gpu *gpu) }msm_devfreq_cleanup (gpu); +msm_perfcntr_cleanup (gpu);platform_set_drvdata (gpu->pdev,NULL ); }
drivers/gpu/drm/msm/msm_gpu.h +56 -1
84 lines not on this page · raw message
drivers/gpu/drm/msm/msm_perfcntr.c new file +120 -0
120 lines not on this page · raw message
drivers/gpu/drm/msm/msm_perfcntr.h +23 -0
@@ -35,6 +35,29 @@ struct msm_perfcntr_group {const struct msm_perfcntr_counter *counters; }; +/** +* struct msm_perfcntr_stream - state for a single open stream fd +*/ +struct msm_perfcntr_stream { +/** @gpu: Back-link to the GPU */ +struct msm_gpu *gpu; + +/** @nr_groups: # of counter groups with enabled counters */ +uint32_t nr_groups; + +/** +* @group_idx: array of nr_groups +* +* Maps the order of groups in PERFCNTR_CONFIG ioctl to group idx, +* so that results in the results stream can be ordered to match +* the ioctl call that setup the stream +*/ +uint32_t *group_idx; +}; + +uint32_t msm_perfcntr_group_idx (const struct msm_perfcntr_stream *stream,uint32_t n); +uint32_t msm_perfcntr_counter_base (const struct msm_perfcntr_stream *stream,uint32_t group_idx); +/** * struct msm_perfcntr_context_state - per-msm_context counter state *
Signature · 1 line
2.54.0
Rob Clark
maintainerPATCH v3 11/16 drm/msm/a6xx+: Add support to configure perfcntrs
In reply to Rob ClarkAdd support to configure counter SELect regs. In some cases the reg writes need to happen while the GPU is idle. And for a7xx+, in some cases SEL regs need to be configured from BV or BR aperture. The easiest way to deal with this is to configure from the RB.
--- drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 69 +++++++++++++++++++++++++++ drivers/gpu/drm/msm/msm_perfcntr.h | 3 ++ drivers/gpu/drm/msm/msm_ringbuffer.h | 2 + 3 files changed, 74 insertions(+)
drivers/gpu/drm/msm/adreno/a6xx_gpu.c +69 -0
@@ -2535,6 +2535,71 @@ static bool a6xx_progress(struct msm_gpu *gpu, struct msm_ringbuffer *ring)return progress; } +static void +a6xx_perfcntr_configure (struct msm_gpu *gpu,struct msm_ringbuffer *ring, +const struct msm_perfcntr_stream *stream) +{ +enum adreno_pipe pipe =PIPE_NONE ; + +for (unsigned i =0 ; i < stream->nr_groups; i++) { +unsigned group_idx =msm_perfcntr_group_idx (stream, i); +unsigned base =msm_perfcntr_counter_base (stream, group_idx); + +const struct msm_perfcntr_group *group = + &gpu->perfcntr_groups[group_idx]; + +struct msm_perfcntr_group_state *group_state = + gpu->perfcntrs->groups[group_idx]; + +if (group->pipe != pipe) { + pipe = group->pipe; + +OUT_PKT7 (ring,CP_THREAD_CONTROL ,1 ); + +if (pipe ==PIPE_BR ) { +OUT_RING (ring,CP_SET_THREAD_BR ); + }else if (pipe ==PIPE_BV ) { +OUT_RING (ring,CP_SET_THREAD_BV ); + }else { +OUT_RING (ring,CP_SET_THREAD_BOTH ); + } + } + +const struct msm_perfcntr_counter *counter = &group->counters[base]; +unsigned nr = group_state->allocated_counters; +OUT_PKT4 (ring, counter->select_reg, nr); +for (unsigned c =0 ; c < nr; c++) +OUT_RING (ring, group_state->countables[c]); + +for (unsigned s =0 ; s <ARRAY_SIZE (counter->slice_select_regs); s++) { +if (!counter->slice_select_regs[s]) +break ; + +OUT_PKT4 (ring, counter->slice_select_regs[s], nr); +for (unsigned c =0 ; c < nr; c++) +OUT_RING (ring, group_state->countables[c]); + } + } + +if (pipe !=PIPE_NONE ) { +OUT_PKT7 (ring,CP_THREAD_CONTROL ,1 ); +OUT_RING (ring,CP_SET_THREAD_BOTH ); + } + +OUT_PKT7 (ring,CP_MEM_WRITE ,3 ); +OUT_RING (ring,lower_32_bits (rbmemptr (ring, perfcntr_fence))); +OUT_RING (ring,upper_32_bits (rbmemptr (ring, perfcntr_fence))); +OUT_RING (ring, stream->sel_fence); + +a6xx_flush_yield (gpu, ring); + +/* Check to see if we need to start preemption */ +if (adreno_is_a8xx (to_adreno_gpu (gpu))) +a8xx_preempt_trigger (gpu); +else +a6xx_preempt_trigger (gpu); +} +static u32 fuse_to_supp_hw (const struct adreno_info *info,u32 fuse) {if (!info->speedbins)
@@ -2753,6 +2818,7 @@ const struct adreno_gpu_funcs a6xx_gpu_funcs = { .get_rptr = a6xx_get_rptr, .progress = a6xx_progress, .sysprof_setup = a6xx_gmu_sysprof_setup, + .perfcntr_configure = a6xx_perfcntr_configure, }, .init = a6xx_gpu_init, .get_timestamp = a6xx_gmu_get_timestamp,
@@ -2786,6 +2852,7 @@ const struct adreno_gpu_funcs a6xx_gmuwrapper_funcs = { .create_private_vm = a6xx_create_private_vm, .get_rptr = a6xx_get_rptr, .progress = a6xx_progress, + .perfcntr_configure = a6xx_perfcntr_configure, }, .init = a6xx_gpu_init, .get_timestamp = a6xx_get_timestamp,
@@ -2822,6 +2889,7 @@ const struct adreno_gpu_funcs a7xx_gpu_funcs = { .get_rptr = a6xx_get_rptr, .progress = a6xx_progress, .sysprof_setup = a6xx_gmu_sysprof_setup, + .perfcntr_configure = a6xx_perfcntr_configure, }, .init = a6xx_gpu_init, .get_timestamp = a6xx_gmu_get_timestamp,
@@ -2852,6 +2920,7 @@ const struct adreno_gpu_funcs a8xx_gpu_funcs = { .get_rptr = a6xx_get_rptr, .progress = a8xx_progress, .sysprof_setup = a6xx_gmu_sysprof_setup, + .perfcntr_configure = a6xx_perfcntr_configure, }, .init = a6xx_gpu_init, .get_timestamp = a8xx_gmu_get_timestamp,
drivers/gpu/drm/msm/msm_perfcntr.h +3 -0
@@ -45,6 +45,9 @@ struct msm_perfcntr_stream {/** @nr_groups: # of counter groups with enabled counters */ uint32_t nr_groups; +/** @sel_fence: Fence for SEL reg programming */ +uint32_t sel_fence; +/** * @group_idx: array of nr_groups *
drivers/gpu/drm/msm/msm_ringbuffer.h +2 -0
@@ -37,6 +37,8 @@ struct msm_rbmemptrs {volatile struct msm_gpu_submit_stats stats[MSM_GPU_SUBMIT_STATS_COUNT ];volatile u64 ttbr0;volatile u32 context_idr; + +volatile u32 perfcntr_fence; };struct msm_cp_state {
Signature · 1 line
2.54.0
Rob Clark
maintainerPATCH v3 12/16 drm/msm/a8xx: Add perfcntr flush sequence
In reply to Rob ClarkWith the slice architecture, we need to flush the slice and unslice counters to perf RAM before reading counters.
--- drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 1 + drivers/gpu/drm/msm/adreno/a6xx_gpu.h | 1 + drivers/gpu/drm/msm/adreno/a8xx_gpu.c | 20 ++++++++++++++++++++ 3 files changed, 22 insertions(+)
drivers/gpu/drm/msm/adreno/a6xx_gpu.c +1 -0
@@ -2921,6 +2921,7 @@ const struct adreno_gpu_funcs a8xx_gpu_funcs = { .progress = a8xx_progress, .sysprof_setup = a6xx_gmu_sysprof_setup, .perfcntr_configure = a6xx_perfcntr_configure, + .perfcntr_flush = a8xx_perfcntr_flush, }, .init = a6xx_gpu_init, .get_timestamp = a8xx_gmu_get_timestamp,
drivers/gpu/drm/msm/adreno/a6xx_gpu.h +1 -0
@@ -334,5 +334,6 @@ void a8xx_preempt_hw_init(struct msm_gpu *gpu);void a8xx_preempt_trigger (struct msm_gpu *gpu);void a8xx_preempt_irq (struct msm_gpu *gpu);bool a8xx_progress (struct msm_gpu *gpu,struct msm_ringbuffer *ring); +void a8xx_perfcntr_flush (struct msm_gpu *gpu);void a8xx_recover (struct msm_gpu *gpu);#endif /* __A6XX_GPU_H__ */
drivers/gpu/drm/msm/adreno/a8xx_gpu.c +20 -0
@@ -1345,3 +1345,23 @@ bool a8xx_progress(struct msm_gpu *gpu, struct msm_ringbuffer *ring) {return true; } + +void a8xx_perfcntr_flush (struct msm_gpu *gpu) +{ +u32 val; + +/* +* Flush delta counters (both perf counters and pipe stats) present in +* RBBM_S and RBBM_US to perf RAM logic to get the latest data. +*/ +gpu_write (gpu,REG_A8XX_RBBM_PERFCTR_FLUSH_HOST_CMD ,BIT (0 )); +gpu_write (gpu,REG_A8XX_RBBM_SLICE_PERFCTR_FLUSH_HOST_CMD ,BIT (0 )); + +/* Ensure all writes are posted before polling status register */ +wmb (); + +if (gpu_poll_timeout (gpu,REG_A8XX_RBBM_PERFCTR_FLUSH_HOST_STATUS , val, + val &BIT (0 ),100 ,100 *1000 )) { +dev_err (&gpu->pdev->dev,"Perfcounter flush timed out: status=0x%08x\n" , val); + } +}
Signature · 1 line
2.54.0
Rob Clark
maintainerPATCH v3 13/16 drm/msm: Add PERFCNTR_CONFIG ioctl
In reply to Rob ClarkAdd new UABI and implementation of PERFCNTR_CONFIG ioctl.
A bit more work is required to configure the pwrup_reglist for the GMU to restore SELect regs on exist of IFPC, before we can stop disabling IFPC while global counter collection. This will follow in a later commit, but will be transparent to userspace.
--- drivers/gpu/drm/msm/msm_drv.c | 1 + drivers/gpu/drm/msm/msm_drv.h | 2 + drivers/gpu/drm/msm/msm_gpu.h | 3 + drivers/gpu/drm/msm/msm_perfcntr.c | 510 +++++++++++++++++++++++++++++ drivers/gpu/drm/msm/msm_perfcntr.h | 51 +++ include/uapi/drm/msm_drm.h | 48 +++ 6 files changed, 615 insertions(+)
drivers/gpu/drm/msm/msm_drv.c +1 -0
@@ -801,6 +801,7 @@ static const struct drm_ioctl_desc msm_ioctls[] = {DRM_IOCTL_DEF_DRV (MSM_SUBMITQUEUE_CLOSE , msm_ioctl_submitqueue_close,DRM_RENDER_ALLOW ),DRM_IOCTL_DEF_DRV (MSM_SUBMITQUEUE_QUERY , msm_ioctl_submitqueue_query,DRM_RENDER_ALLOW ),DRM_IOCTL_DEF_DRV (MSM_VM_BIND , msm_ioctl_vm_bind,DRM_RENDER_ALLOW ), + DRM_IOCTL_DEF_DRV(MSM_PERFCNTR_CONFIG, msm_ioctl_perfcntr_config, DRM_RENDER_ALLOW), };static void msm_show_fdinfo (struct drm_printer *p,struct drm_file *file)
drivers/gpu/drm/msm/msm_drv.h +2 -0
@@ -237,6 +237,8 @@ int msm_ioctl_vm_bind(struct drm_device *dev, void *data,int msm_perfcntr_resume (struct msm_gpu *gpu);void msm_perfcntr_suspend (struct msm_gpu *gpu); +int msm_ioctl_perfcntr_config (struct drm_device *dev,void *data, +struct drm_file *file);struct msm_perfcntr_state *msm_perfcntr_init (struct msm_gpu *gpu);void msm_perfcntr_cleanup (struct msm_gpu *gpu);
drivers/gpu/drm/msm/msm_gpu.h +3 -0
@@ -353,6 +353,9 @@ struct msm_perfcntr_state {/** @stream: current global counter stream if active */ struct msm_perfcntr_stream *stream; +/** @sel_seqno: counter for sel_fence */ +uint32_t sel_seqno; +/** * @groups: Global perfcntr stream group state. *
drivers/gpu/drm/msm/msm_perfcntr.c +510 -0
535 lines not on this page · raw message
drivers/gpu/drm/msm/msm_perfcntr.h +51 -0
75 lines not on this page · raw message
include/uapi/drm/msm_drm.h +48 -0
66 lines not on this page · raw message
Signature · 1 line
2.54.0
Rob Clark
maintainerPATCH v3 14/16 drm/msm/a6xx: Increase pwrup_reglist size
In reply to Rob ClarkTo make room for appending SEL reg programming. Without increasing the size, we would overflow the pwrup_reglist at ~190 counters on gen8. Or possibly fewer, considering that some gen8 counter groups also have separate slice vs unslice SELectors.
--- drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/msm/adreno/a6xx_gpu.c +1 -1
@@ -1183,7 +1183,7 @@ static int a6xx_ucode_load(struct msm_gpu *gpu)msm_gem_object_set_name (a6xx_gpu->shadow_bo,"shadow" ); } - a6xx_gpu->pwrup_reglist_ptr =msm_gem_kernel_new (gpu->dev,PAGE_SIZE , + a6xx_gpu->pwrup_reglist_ptr =msm_gem_kernel_new (gpu->dev,2 *PAGE_SIZE ,MSM_BO_WC |MSM_BO_MAP_PRIV , gpu->vm, &a6xx_gpu->pwrup_reglist_bo, &a6xx_gpu->pwrup_reglist_iova);
Signature · 1 line
2.54.0
Rob Clark
maintainerPATCH v3 15/16 drm/msm/a6xx: Append SEL regs to dyn pwrup reglist
In reply to Rob ClarkThis is needed so that SEL reg values are restored on exit from IFPC.
--- drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 82 +++++++++++++++++++++++++-- drivers/gpu/drm/msm/adreno/a6xx_gpu.h | 11 +++- drivers/gpu/drm/msm/adreno/a8xx_gpu.c | 1 + 3 files changed, 87 insertions(+), 7 deletions(-)
drivers/gpu/drm/msm/adreno/a6xx_gpu.c +76 -6
@@ -946,6 +946,7 @@ static void a7xx_patch_pwrup_reglist(struct msm_gpu *gpu) A7XX_CP_APERTURE_CNTL_HOST_PIPE(PIPE_NONE)); } lock->dynamic_list_len = dyn_pwrup_reglist_count; + a6xx_gpu->dynamic_sel_reglist_offset = dyn_pwrup_reglist_count; }static int a7xx_preempt_start (struct msm_gpu *gpu)
@@ -2535,11 +2536,60 @@ static bool a6xx_progress(struct msm_gpu *gpu, struct msm_ringbuffer *ring)return progress; } +static void +perfcntr_select (struct msm_ringbuffer *ring,enum adreno_pipe pipe, +uint32_t regidx,uint32_t *countables,uint32_t nr, +uint32_t **reglist) +{ +OUT_PKT4 (ring, regidx, nr); +for (unsigned i =0 ; i < nr; i++) +OUT_RING (ring, countables[i]); + +if (!*reglist) +return ; + +for (unsigned i =0 ; i < nr; i++) { +/* +* Bitfield is in same position on a7xx, but only 2 bits.. +* which is sufficient for NONE/BR/BV: +*/ + *(*reglist)++ =A8XX_CP_APERTURE_CNTL_HOST_PIPEID (pipe); + *(*reglist)++ = regidx + i; + *(*reglist)++ = countables[i]; + } +} +static void a6xx_perfcntr_configure (struct msm_gpu *gpu,struct msm_ringbuffer *ring,const struct msm_perfcntr_stream *stream) { +struct adreno_gpu *adreno_gpu =to_adreno_gpu (gpu); +struct a6xx_gpu *a6xx_gpu =to_a6xx_gpu (adreno_gpu);enum adreno_pipe pipe =PIPE_NONE ; +uint32_t *reglist =NULL ; +uint32_t *reglist_sel_start; + +if (to_adreno_gpu (gpu)->info->family >=ADRENO_7XX_GEN1 ) { +WARN_ON (!a6xx_gpu->pwrup_reglist_emitted); + +struct cpu_gpu_lock *lock = a6xx_gpu->pwrup_reglist_ptr; +int off = (2 * lock->ifpc_list_len) + + (2 * lock->preemption_list_len) + + (3 * a6xx_gpu->dynamic_sel_reglist_offset); + + reglist = (uint32_t *)&lock->regs[0 ]; + reglist += off; + reglist_sel_start = reglist; + +/* Clear any previously configured SEL reg entries: */ + lock->dynamic_list_len = a6xx_gpu->dynamic_sel_reglist_offset; + +/* +* Ensure CP sees the dynamic_list_len update before we +* start modifying the SEL entries: +*/ +wmb (); + }for (unsigned i =0 ; i < stream->nr_groups; i++) {unsigned group_idx =msm_perfcntr_group_idx (stream, i);
@@ -2567,17 +2617,15 @@ a6xx_perfcntr_configure(struct msm_gpu *gpu, struct msm_ringbuffer *ring,const struct msm_perfcntr_counter *counter = &group->counters[base];unsigned nr = group_state->allocated_counters; -OUT_PKT4 (ring, counter->select_reg, nr); -for (unsigned c =0 ; c < nr; c++) -OUT_RING (ring, group_state->countables[c]); +perfcntr_select (ring, pipe, counter->select_reg, + group_state->countables, nr, ®list);for (unsigned s =0 ; s <ARRAY_SIZE (counter->slice_select_regs); s++) {if (!counter->slice_select_regs[s])break ; -OUT_PKT4 (ring, counter->slice_select_regs[s], nr); -for (unsigned c =0 ; c < nr; c++) -OUT_RING (ring, group_state->countables[c]); +perfcntr_select (ring, pipe, counter->slice_select_regs[s], + group_state->countables, nr, ®list); } }
@@ -2591,6 +2639,28 @@ a6xx_perfcntr_configure(struct msm_gpu *gpu, struct msm_ringbuffer *ring,OUT_RING (ring,upper_32_bits (rbmemptr (ring, perfcntr_fence)));OUT_RING (ring, stream->sel_fence); +/* +* Update the pwrup reglist size before flushing. Kgsl does a shared- +* memory spinlock dance with SQE to avoid racing with IFPC exit. But +* we can skip that since the ringbuffer programming will be executed +* by SQE after dynamic reglist size is updated. So even if we lose +* the race, the register programming in the rb will overwrite/correct +* the SEL regs restored by SQE on IFPC exit, before sampling begins. +*/ +if (reglist) { +struct cpu_gpu_lock *lock = a6xx_gpu->pwrup_reglist_ptr; +unsigned nr_regs = (reglist_sel_start - reglist) /3 ; + +/* +* Ensure CP sees updates to the pwrup_reglist before it +* sees the new (increased) length: +*/ +wmb (); + +/* Update dynamic reglist len to include new SEL reg programming: */ + lock->dynamic_list_len = a6xx_gpu->dynamic_sel_reglist_offset + nr_regs; + } +a6xx_flush_yield (gpu, ring);/* Check to see if we need to start preemption */
drivers/gpu/drm/msm/adreno/a6xx_gpu.h +10 -1
@@ -21,17 +21,19 @@ struct cpu_gpu_lock {uint32_t cpu_req;uint32_t turn;union { +/* a6xx: */ struct {uint16_t list_length;uint16_t list_offset; }; +/* a7xx+: */ struct {uint8_t ifpc_list_len;uint8_t preemption_list_len;uint16_t dynamic_list_len; }; }; -uint64_t regs[62 ]; +uint64_t regs[]; };/**
@@ -100,6 +102,13 @@ struct a6xx_gpu {uint64_t pwrup_reglist_iova;bool pwrup_reglist_emitted; +/* +* Offset of start of SEL regs appended to pwrup_reglist. This +* is equal to lock->dynamic_list_len if no SEL regs are appended +* to the end of the dynamic reglist. +*/ +uint16_t dynamic_sel_reglist_offset; +bool has_whereami;void __iomem *llc_mmio;
drivers/gpu/drm/msm/adreno/a8xx_gpu.c +1 -0
@@ -468,6 +468,7 @@ static void a8xx_patch_pwrup_reglist(struct msm_gpu *gpu) } lock->dynamic_list_len = dyn_pwrup_reglist_count; + a6xx_gpu->dynamic_sel_reglist_offset = dyn_pwrup_reglist_count; done:a8xx_aperture_clear (gpu);
Signature · 1 line
2.54.0
Rob Clark
maintainerPATCH v3 16/16 drm/msm/a6xx: Allow IFPC with perfcntr stream
In reply to Rob ClarkNow that the dynamic pwrup reglist has SEL reg values to restore appended, so that SEL regs are restored on IFPC exit, we can stop completely disabling IFPC while global counter sampling is active.
To accomplish this, we re-use sysprof_setup() with a force_on param to inhibit IFPC specifically while the counter regs are being read, while leaving IFPC enabled the rest of the time.
--- drivers/gpu/drm/msm/adreno/a6xx_gmu.c | 4 ++-- drivers/gpu/drm/msm/adreno/a6xx_gpu.h | 2 +- drivers/gpu/drm/msm/msm_gpu.h | 10 ++-------- drivers/gpu/drm/msm/msm_perfcntr.c | 8 ++++++++ drivers/gpu/drm/msm/msm_submitqueue.c | 2 +- 5 files changed, 14 insertions(+), 12 deletions(-)
drivers/gpu/drm/msm/adreno/a6xx_gmu.c +2 -2
@@ -2034,9 +2034,9 @@ static int a6xx_gmu_get_irq(struct a6xx_gmu *gmu, struct platform_device *pdev,return irq; } -void a6xx_gmu_sysprof_setup (struct msm_gpu *gpu) +void a6xx_gmu_sysprof_setup (struct msm_gpu *gpu,bool force_on) { -bool sysprof =msm_gpu_sysprof_no_ifpc (gpu); +bool sysprof =msm_gpu_sysprof_no_ifpc (gpu) | force_on;struct adreno_gpu *adreno_gpu =to_adreno_gpu (gpu);struct a6xx_gpu *a6xx_gpu =to_a6xx_gpu (adreno_gpu);struct a6xx_gmu *gmu = &a6xx_gpu->gmu;
drivers/gpu/drm/msm/adreno/a6xx_gpu.h +1 -1
@@ -280,7 +280,7 @@ void a6xx_gmu_clear_oob(struct a6xx_gmu *gmu, enum a6xx_gmu_oob_state state);int a6xx_gmu_init (struct a6xx_gpu *a6xx_gpu,struct device_node *node);int a6xx_gmu_wrapper_init (struct a6xx_gpu *a6xx_gpu,struct device_node *node);void a6xx_gmu_remove (struct a6xx_gpu *a6xx_gpu); -void a6xx_gmu_sysprof_setup (struct msm_gpu *gpu); +void a6xx_gmu_sysprof_setup (struct msm_gpu *gpu,bool force_on);void a6xx_preempt_init (struct msm_gpu *gpu);void a6xx_preempt_hw_init (struct msm_gpu *gpu);
drivers/gpu/drm/msm/msm_gpu.h +2 -8
@@ -93,7 +93,7 @@ struct msm_gpu_funcs {* for cmdstream that is buffered in this FIFO upstream of the CP fw. */ bool (*progress)(struct msm_gpu *gpu,struct msm_ringbuffer *ring); -void (*sysprof_setup)(struct msm_gpu *gpu); +void (*sysprof_setup)(struct msm_gpu *gpu,bool force_on);/* Configure perfcntr SELect regs: */ void (*perfcntr_configure)(struct msm_gpu *gpu, struct msm_ringbuffer *ring,
@@ -378,13 +378,7 @@ msm_gpu_sysprof_no_perfcntr_zap(struct msm_gpu *gpu)static inline bool msm_gpu_sysprof_no_ifpc (struct msm_gpu *gpu) { -/* -* For now, this is the same condition as disabling perfcntr clears -* on context switch. But once kernel perfcntr IFPC support is in -* place, we will only need to disable IFPC for legacy userspace -* setting SYSPROF param. -*/ -return msm_gpu_sysprof_no_perfcntr_zap (gpu); +return refcount_read (&gpu->sysprof_active) >1 ; }/*
drivers/gpu/drm/msm/msm_perfcntr.c +8 -0
@@ -256,6 +256,10 @@ sample_worker(struct kthread_work *work)return ; } +/* Inhibit IFPC while accessing registers: */ +if (gpu->funcs->sysprof_setup) + gpu->funcs->sysprof_setup (gpu, true); +if (gpu->funcs->perfcntr_flush) gpu->funcs->perfcntr_flush (gpu);
@@ -290,6 +294,10 @@ sample_worker(struct kthread_work *work) } } +/* Re-enable IFPC: */ +if (gpu->funcs->sysprof_setup) + gpu->funcs->sysprof_setup (gpu, false); +smp_store_release (&stream->fifo.head, head);wake_up_all (&stream->poll_wq); }
drivers/gpu/drm/msm/msm_submitqueue.c +1 -1
@@ -42,7 +42,7 @@ int msm_context_set_sysprof(struct msm_context *ctx, struct msm_gpu *gpu, int sy/* Some gpu families require additional setup for sysprof */ if (gpu->funcs->sysprof_setup) - gpu->funcs->sysprof_setup (gpu); + gpu->funcs->sysprof_setup (gpu, false); ctx->sysprof = sysprof;
Signature · 1 line
2.54.0