Enable OA unit for Gen 8 and 9 in i915 perf
Subsystems (3)
v4 cover Enable OA unit for Gen 8 and 9 in i915 perf
Updates based on latest review feedback from Matthew and Lionel and includes an update to the TestOa register config for SKL GT2 compared the last series (based on the latest XML files from VPG)
Although the _[SUB]SLICE_MASK GETPARM patches were reviewed, it's worth mentioning there was a TODO comment in the last patches about rebasing the parameter ID before upstreaming which is removed now, and there's a minimal comment for what the new parameters are, consistent with other more recently added parameters. Conveniently the actual IDs didn't need rebasing so there's no last moment uapi change for gputop, mesa and igt.
The series is longer just because I've included the gen7 prep patches (already reviewed) that I haven't landed yet but the gen8+ bits depend on.
Regards,
- Robert
Robert Bragg (15):
drm/i915/perf: fix gen7_append_oa_reports comment drm/i915/perf: avoid poll, read, EAGAIN busy loops drm/i915/perf: avoid read back of head register drm/i915/perf: no head/tail ref in gen7_oa_read drm/i915/perf: improve tail race workaround drm/i915/perf: improve invalid OA format debug message drm/i915/perf: better pipeline aged/aging tail updates drm/i915/perf: rate limit spurious oa report notice drm/i915: expose _SLICE_MASK GETPARM
drm/i915: expose _SUBSLICE_MASK GETPARM drm/i915/perf: Add 'render basic' Gen8+ OA unit configs drm/i915/perf: Add OA unit support for Gen 8+ drm/i915/perf: Add more OA configs for BDW, CHV, SKL + BXT drm/i915/perf: per-gen timebase for checking sample freq drm/i915/perf: remove perf.hook_lock
drivers/gpu/drm/i915/Makefile | 8 +- drivers/gpu/drm/i915/i915_drv.c | 10 + drivers/gpu/drm/i915/i915_drv.h | 121 +- drivers/gpu/drm/i915/i915_gem_context.h | 1 + drivers/gpu/drm/i915/i915_oa_bdw.c | 5154 +++++++++++++++++++++++++++++++ drivers/gpu/drm/i915/i915_oa_bdw.h | 38 + drivers/gpu/drm/i915/i915_oa_bxt.c | 2541 +++++++++++++++ drivers/gpu/drm/i915/i915_oa_bxt.h | 38 + drivers/gpu/drm/i915/i915_oa_chv.c | 2730 ++++++++++++++++ drivers/gpu/drm/i915/i915_oa_chv.h | 38 + drivers/gpu/drm/i915/i915_oa_hsw.c | 58 +- drivers/gpu/drm/i915/i915_oa_sklgt2.c | 3302 ++++++++++++++++++++ drivers/gpu/drm/i915/i915_oa_sklgt2.h | 38 + drivers/gpu/drm/i915/i915_oa_sklgt3.c | 2856 +++++++++++++++++ drivers/gpu/drm/i915/i915_oa_sklgt3.h | 38 + drivers/gpu/drm/i915/i915_oa_sklgt4.c | 2910 +++++++++++++++++ drivers/gpu/drm/i915/i915_oa_sklgt4.h | 38 + drivers/gpu/drm/i915/i915_perf.c | 1341 ++++++-- drivers/gpu/drm/i915/i915_reg.h | 22 + drivers/gpu/drm/i915/intel_lrc.c | 5 + include/uapi/drm/i915_drm.h | 27 +- 21 files changed, 21076 insertions(+), 238 deletions(-) create mode 100644 drivers/gpu/drm/i915/i915_oa_bdw.c create mode 100644 drivers/gpu/drm/i915/i915_oa_bdw.h create mode 100644 drivers/gpu/drm/i915/i915_oa_bxt.c create mode 100644 drivers/gpu/drm/i915/i915_oa_bxt.h create mode 100644 drivers/gpu/drm/i915/i915_oa_chv.c create mode 100644 drivers/gpu/drm/i915/i915_oa_chv.h create mode 100644 drivers/gpu/drm/i915/i915_oa_sklgt2.c create mode 100644 drivers/gpu/drm/i915/i915_oa_sklgt2.h create mode 100644 drivers/gpu/drm/i915/i915_oa_sklgt3.c create mode 100644 drivers/gpu/drm/i915/i915_oa_sklgt3.h create mode 100644 drivers/gpu/drm/i915/i915_oa_sklgt4.c create mode 100644 drivers/gpu/drm/i915/i915_oa_sklgt4.h
Signature · 6 lines
2.12.0 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v4 1/15 drm/i915/perf: fix gen7_append_oa_reports comment
If I'm going to complain about a back-to-front convention then the least I can do is not muddle the comment up too.
--- drivers/gpu/drm/i915/i915_perf.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/i915/i915_perf.c +1 -1
@@ -431,7 +431,7 @@ static int append_oa_sample(struct i915_perf_stream *stream,* userspace. * * Note: reports are consumed from the head, and appended to the -* tail, so the head chases the tail?... If you think that's mad +* tail, so the tail chases the head?... If you think that's mad * and back-to-front you're not alone, but this follows the * Gen PRM naming convention. *
Signature · 6 lines
2.12.0 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v4 2/15 drm/i915/perf: avoid poll, read, EAGAIN busy loops
In reply to Robert BraggIf the function for checking whether there is OA buffer data available (during a poll or blocking read) has false positives then we want to avoid a situation where the subsequent read() returns EAGAIN (after a more accurate check) followed by a poll() immediately reporting the same false positive POLLIN event and effectively maintaining a busy loop until there really is data.
This makes sure that we clear the .pollin event status whenever we return EAGAIN to userspace which will throttle subsequent POLLIN events and repeated attempts to read to the 5ms intervals of the hrtimer callback we have.
--- drivers/gpu/drm/i915/i915_perf.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/i915_perf.c +9 -1
@@ -1352,7 +1352,15 @@ static ssize_t i915_perf_read(struct file *file,mutex_unlock (&dev_priv->perf.lock); } -if (ret >=0 ) { +/* We allow the poll checking to sometimes report false positive POLLIN +* events where we might actually report EAGAIN on read() if there's +* not really any data available. In this situation though we don't +* want to enter a busy loop between poll() reporting a POLLIN event +* and read() returning -EAGAIN. Clearing the oa.pollin state here +* effectively ensures we back off until the next hrtimer callback +* before reporting another POLLIN event. +*/ +if (ret >=0 || ret == -EAGAIN ) {/* Maybe make ->pollin per-stream state if we support multiple * concurrent streams in the future. */
Signature · 6 lines
2.12.0 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v4 3/15 drm/i915/perf: avoid read back of head register
In reply to Robert BraggThere's no need for the driver to keep reading back the head pointer from hardware since the hardware doesn't update it automatically. This way we can treat any invalid head pointer value as a software/driver bug instead of spurious hardware behaviour.
This change is also a small stepping stone towards re-working how the head and tail state is managed as part of an improved workaround for the tail register race condition.
--- drivers/gpu/drm/i915/i915_drv.h | 11 ++++++++++ drivers/gpu/drm/i915/i915_perf.c | 46 ++++++++++++++++++---------------------- 2 files changed, 32 insertions(+), 25 deletions(-)
drivers/gpu/drm/i915/i915_drv.h +11 -0
@@ -2466,6 +2466,17 @@ struct drm_i915_private {u8 *vaddr;int format;int format_size; + +/** +* Although we can always read back the head +* pointer register, we prefer to avoid +* trusting the HW state, just to avoid any +* risk that some hardware condition could +* somehow bump the head pointer unpredictably +* and cause us to forward the wrong OA buffer +* data to userspace. +*/ +u32 head; } oa_buffer;u32 gen7_latched_oastatus1;
drivers/gpu/drm/i915/i915_perf.c +21 -25
104 lines not on this page · raw message
Signature · 6 lines
2.12.0 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v4 4/15 drm/i915/perf: no head/tail ref in gen7_oa_read
In reply to Robert BraggThis avoids redundantly passing an (inout) head and tail pointer to gen7_append_oa_reports() from gen7_oa_read which doesn't need to reference either itself.
Moving the head/tail reads and writes into gen7_append_oa_reports should have no functional effect except to avoid some redundant head pointer writes in cases where nothing was copied to userspace.
This is a stepping stone towards updating how the head and tail pointer state is managed to improve the workaround for the OA unit's tail pointer race. It reduces the number of places we need to read/write the head and tail pointers.
--- drivers/gpu/drm/i915/i915_perf.c | 51 +++++++++++++++------------------------- 1 file changed, 19 insertions(+), 32 deletions(-)
drivers/gpu/drm/i915/i915_perf.c +19 -32
@@ -420,8 +420,6 @@ static int append_oa_sample(struct i915_perf_stream *stream,* @buf: destination buffer given by userspace * @count: the number of bytes userspace wants to read * @offset: (inout): the current position for writing into @buf -* @head_ptr: (inout): the current oa buffer cpu read position -* @tail: the current oa buffer gpu write position * * Notably any error condition resulting in a short read (-%ENOSPC or * -%EFAULT) will be returned even though one or more records may
@@ -439,9 +437,7 @@ static int append_oa_sample(struct i915_perf_stream *stream,static int gen7_append_oa_reports (struct i915_perf_stream *stream,char __user *buf,size_t count, -size_t *offset, -u32 *head_ptr, -u32 tail) +size_t *offset) {struct drm_i915_private *dev_priv = stream->dev_priv;int report_size = dev_priv->perf.oa.oa_buffer.format_size;
@@ -449,14 +445,15 @@ static int gen7_append_oa_reports(struct i915_perf_stream *stream,int tail_margin = dev_priv->perf.oa.tail_margin;u32 gtt_offset =i915_ggtt_offset (dev_priv->perf.oa.oa_buffer.vma);u32 mask = (OA_BUFFER_SIZE -1 ); -u32 head; +size_t start_offset = *offset; +u32 head, oastatus1, tail;u32 taken;int ret =0 ;if (WARN_ON (!stream->enabled))return -EIO ; - head = *head_ptr - gtt_offset; + head = dev_priv->perf.oa.oa_buffer.head - gtt_offset;/* An out of bounds or misaligned head pointer implies a driver bug * since we are in full control of head pointer which should only
@@ -467,7 +464,8 @@ static int gen7_append_oa_reports(struct i915_perf_stream *stream, "Inconsistent OA buffer head pointer = %u\n", head))return -EIO ; - tail -= gtt_offset; + oastatus1 =I915_READ (GEN7_OASTATUS1 ); + tail = (oastatus1 &GEN7_OASTATUS1_TAIL_MASK ) - gtt_offset;/* The OA unit is expected to wrap the tail pointer according to the OA * buffer size
@@ -477,8 +475,6 @@ static int gen7_append_oa_reports(struct i915_perf_stream *stream, tail); dev_priv->perf.oa.ops.oa_disable (dev_priv); dev_priv->perf.oa.ops.oa_enable (dev_priv); - *head_ptr =I915_READ (GEN7_OASTATUS2 ) & -GEN7_OASTATUS2_HEAD_MASK ;return -EIO ; }
@@ -542,7 +538,18 @@ static int gen7_append_oa_reports(struct i915_perf_stream *stream, report32[0 ] =0 ; } - *head_ptr = gtt_offset + head; + +if (start_offset != *offset) { +/* We removed the gtt_offset for the copy loop above, indexing +* relative to oa_buf_base so put back here... +*/ + head += gtt_offset; + +I915_WRITE (GEN7_OASTATUS2 , + ((head &GEN7_OASTATUS2_HEAD_MASK ) | +OA_MEM_SELECT_GGTT )); + dev_priv->perf.oa.oa_buffer.head = head; + }return ret; }
@@ -570,8 +577,6 @@ static int gen7_oa_read(struct i915_perf_stream *stream, {struct drm_i915_private *dev_priv = stream->dev_priv;u32 oastatus1; -u32 head; -u32 tail;int ret;if (WARN_ON (!dev_priv->perf.oa.oa_buffer.vaddr))
@@ -579,9 +584,6 @@ static int gen7_oa_read(struct i915_perf_stream *stream, oastatus1 =I915_READ (GEN7_OASTATUS1 ); - head = dev_priv->perf.oa.oa_buffer.head; - tail = oastatus1 &GEN7_OASTATUS1_TAIL_MASK ; -/* XXX: On Haswell we don't have a safe way to clear oastatus1 * bits while the OA unit is enabled (while the tail pointer * may be updated asynchronously) so we ignore status bits
@@ -621,9 +623,6 @@ static int gen7_oa_read(struct i915_perf_stream *stream, dev_priv->perf.oa.ops.oa_enable (dev_priv); oastatus1 =I915_READ (GEN7_OASTATUS1 ); - - head = dev_priv->perf.oa.oa_buffer.head; - tail = oastatus1 &GEN7_OASTATUS1_TAIL_MASK ; }if (unlikely (oastatus1 &GEN7_OASTATUS1_REPORT_LOST )) {
@@ -635,19 +634,7 @@ static int gen7_oa_read(struct i915_perf_stream *stream,GEN7_OASTATUS1_REPORT_LOST ; } - ret =gen7_append_oa_reports (stream, buf, count, offset, - &head, tail); - -/* Note: we update the head pointer here even if an error -* was returned since the error may represent a short read -* where some some reports were successfully copied. -*/ -I915_WRITE (GEN7_OASTATUS2 , - ((head &GEN7_OASTATUS2_HEAD_MASK ) | -OA_MEM_SELECT_GGTT )); - dev_priv->perf.oa.oa_buffer.head = head; - -return ret; +return gen7_append_oa_reports (stream, buf, count, offset); }/**
Signature · 6 lines
2.12.0 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v4 5/15 drm/i915/perf: improve tail race workaround
In reply to Robert BraggThere's a HW race condition between OA unit tail pointer register updates and writes to memory whereby the tail pointer can sometimes get ahead of what's been written out to the OA buffer so far (in terms of what's visible to the CPU).
Although this can be observed explicitly while copying reports to userspace by checking for a zeroed report-id field in tail reports, we want to account for this earlier, as part of the _oa_buffer_check to avoid lots of redundant read() attempts.
Previously the driver used to define an effective tail pointer that lagged the real pointer by a 'tail margin' measured in bytes derived from OA_TAIL_MARGIN_NSEC and the configured sampling frequency. Unfortunately this was flawed considering that the OA unit may also automatically generate non-periodic reports (such as on context switch) or the OA unit may be enabled without any periodic sampling.
This improves how we define a tail pointer for reading that lags the real tail pointer by at least %OA_TAIL_MARGIN_NSEC nanoseconds, which gives enough time for the corresponding reports to become visible to the CPU.
The driver now maintains two tail pointers:
1) An 'aging' tail with an associated timestamp that is tracked until we can trust the corresponding data is visible to the CPU; at which point it is considered 'aged'.
2) An 'aged' tail that can be used for read()ing.
The two separate pointers let us decouple read()s from tail pointer aging.
The tail pointers are checked and updated at a limited rate within a hrtimer callback (the same callback that is used for delivering POLLIN events) and since we're now measuring the wall clock time elapsed since a given tail pointer was read the mechanism no longer cares about the OA unit's periodic sampling frequency.
The natural place to handle the tail pointer updates was in gen7_oa_buffer_is_empty() which is called as part of blocking reads and the hrtimer callback used for polling, and so this was renamed to oa_buffer_check() considering the added side effect while checking whether the buffer contains data.
--- drivers/gpu/drm/i915/i915_drv.h | 60 ++++++++- drivers/gpu/drm/i915/i915_perf.c | 277 ++++++++++++++++++++++++++------------- 2 files changed, 241 insertions(+), 96 deletions(-)
drivers/gpu/drm/i915/i915_drv.h +55 -5
@@ -2094,7 +2094,7 @@ struct i915_oa_ops { size_t *offset);/** -* @oa_buffer_is_empty: Check if OA buffer empty (false positives OK) +* @oa_buffer_check: Check for OA buffer data + update tail * * This is either called via fops or the poll check hrtimer (atomic * ctx) without any locks taken.
@@ -2107,7 +2107,7 @@ struct i915_oa_ops {* here, which will be handled gracefully - likely resulting in an * %EAGAIN error for userspace. */ -bool (*oa_buffer_is_empty)(struct drm_i915_private *dev_priv); +bool (*oa_buffer_check)(struct drm_i915_private *dev_priv); };struct intel_cdclk_state {
@@ -2450,9 +2450,6 @@ struct drm_i915_private {bool periodic;int period_exponent; -int timestamp_frequency; - -int tail_margin;int metrics_set;
@@ -2468,6 +2465,59 @@ struct drm_i915_private {int format_size;/** +* Locks reads and writes to all head/tail state +* +* Consider: the head and tail pointer state +* needs to be read consistently from a hrtimer +* callback (atomic context) and read() fop +* (user context) with tail pointer updates +* happening in atomic context and head updates +* in user context and the (unlikely) +* possibility of read() errors needing to +* reset all head/tail state. +* +* Note: Contention or performance aren't +* currently a significant concern here +* considering the relatively low frequency of +* hrtimer callbacks (5ms period) and that +* reads typically only happen in response to a +* hrtimer event and likely complete before the +* next callback. +* +* Note: This lock is not held *while* reading +* and copying data to userspace so the value +* of head observed in htrimer callbacks won't +* represent any partial consumption of data. +*/ +spinlock_t ptr_lock; + +/** +* One 'aging' tail pointer and one 'aged' +* tail pointer ready to used for reading. +* +* Initial values of 0xffffffff are invalid +* and imply that an update is required +* (and should be ignored by an attempted +* read) +*/ +struct { +u32 offset; + } tails[2 ]; + +/** +* Index for the aged tail ready to read() +* data up to. +*/ +unsigned int aged_tail_idx; + +/** +* A monotonic timestamp for when the current +* aging tail pointer was read; used to +* determine when it is old enough to trust. +*/ +u64 aging_timestamp; + +/** * Although we can always read back the head * pointer register, we prefer to avoid * trusting the HW state, just to avoid any
drivers/gpu/drm/i915/i915_perf.c +186 -91
388 lines not on this page · raw message
Signature · 6 lines
2.12.0 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v4 6/15 drm/i915/perf: improve invalid OA format debug message
In reply to Robert BraggA minor improvement to debugging output
--- drivers/gpu/drm/i915/i915_perf.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
drivers/gpu/drm/i915/i915_perf.c +4 -2
@@ -1904,11 +1904,13 @@ static int read_properties_unlocked(struct drm_i915_private *dev_priv,break ;case DRM_I915_PERF_PROP_OA_FORMAT :if (value ==0 || value >=I915_OA_FORMAT_MAX ) { -DRM_DEBUG ("Invalid OA report format\n" ); +DRM_DEBUG ("Out-of-range OA report format %llu\n" , + value);return -EINVAL ; }if (!dev_priv->perf.oa.oa_formats[value].size) { -DRM_DEBUG ("Invalid OA report format\n" ); +DRM_DEBUG ("Unsupported OA report format %llu\n" , + value);return -EINVAL ; } props->oa_format = value;
Signature · 6 lines
2.12.0 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v4 7/15 drm/i915/perf: better pipeline aged/aging tail updates
In reply to Robert BraggThis updates the tail pointer race workaround handling to updating the 'aged' pointer before looking to start aging a new one. There's the possibility that there is already new data available and so we can immediately start aging a new pointer without having to first wait for a later hrtimer callback (and then another to age).
--- drivers/gpu/drm/i915/i915_perf.c | 41 ++++++++++++++++++++++------------------ 1 file changed, 23 insertions(+), 18 deletions(-)
drivers/gpu/drm/i915/i915_perf.c +23 -18
@@ -391,6 +391,29 @@ static bool gen7_oa_buffer_check_unlocked(struct drm_i915_private *dev_priv) now =ktime_get_mono_fast_ns (); +/* Update the aged tail +* +* Flip the tail pointer available for read()s once the aging tail is +* old enough to trust that the corresponding data will be visible to +* the CPU... +* +* Do this before updating the aging pointer in case we may be able to +* immediately start aging a new pointer too (if new data has become +* available) without needing to wait for a later hrtimer callback. +*/ +if (aging_tail !=INVALID_TAIL_PTR && + ((now - dev_priv->perf.oa.oa_buffer.aging_timestamp) > +OA_TAIL_MARGIN_NSEC )) { + aged_idx ^=1 ; + dev_priv->perf.oa.oa_buffer.aged_tail_idx = aged_idx; + + aged_tail = aging_tail; + +/* Mark that we need a new pointer to start aging... */ + dev_priv->perf.oa.oa_buffer.tails[!aged_idx].offset =INVALID_TAIL_PTR ; + aging_tail =INVALID_TAIL_PTR ; + } +/* Update the aging tail * * We throttle aging tail updates until we have a new tail that
@@ -420,24 +443,6 @@ static bool gen7_oa_buffer_check_unlocked(struct drm_i915_private *dev_priv) } } -/* Update the aged tail -* -* Flip the tail pointer available for read()s once the aging tail is -* old enough to trust that the corresponding data will be visible to -* the CPU... -*/ -if (aging_tail !=INVALID_TAIL_PTR && - ((now - dev_priv->perf.oa.oa_buffer.aging_timestamp) > -OA_TAIL_MARGIN_NSEC )) { - aged_idx ^=1 ; - dev_priv->perf.oa.oa_buffer.aged_tail_idx = aged_idx; - - aged_tail = aging_tail; - -/* Mark that we need a new pointer to start aging... */ - dev_priv->perf.oa.oa_buffer.tails[!aged_idx].offset =INVALID_TAIL_PTR ; - } -spin_unlock_irqrestore (&dev_priv->perf.oa.oa_buffer.ptr_lock, flags); return aged_tail == INVALID_TAIL_PTR ?
Signature · 6 lines
2.12.0 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v4 8/15 drm/i915/perf: rate limit spurious oa report notice
In reply to Robert BraggThis change is pre-emptively aiming to avoid a potential cause of kernel logging noise in case some condition were to result in us seeing invalid OA reports.
The workaround for the OA unit's tail pointer race condition is what avoids the primary known cause of invalid reports being seen and with that in place we aren't expecting to see this notice but it can't be entirely ruled out.
Just in case some condition does lead to the notice then it's likely that it will be triggered repeatedly while attempting to append a sequence of reports and depending on the configured OA sampling frequency that might be a large number of repeat notices.
v2: (Chris) avoid inconsistent warning on throttle with
printk_ratelimit()
v3: (Matt) init and summarise with stream init/close not driver init/fini
--- drivers/gpu/drm/i915/i915_drv.h | 6 ++++++ drivers/gpu/drm/i915/i915_perf.c | 28 +++++++++++++++++++++++++++- 2 files changed, 33 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/i915_drv.h +6 -0
@@ -2448,6 +2448,12 @@ struct drm_i915_private {wait_queue_head_t poll_wq;bool pollin; +/** +* For rate limiting any notifications of spurious +* invalid OA reports +*/ +struct ratelimit_state spurious_report_rs; +bool periodic;int period_exponent;
drivers/gpu/drm/i915/i915_perf.c +27 -1
@@ -632,7 +632,8 @@ static int gen7_append_oa_reports(struct i915_perf_stream *stream,* copying it to userspace... */ if (report32[0 ] ==0 ) { -DRM_NOTE ("Skipping spurious, invalid OA report\n" ); +if (__ratelimit (&dev_priv->perf.oa.spurious_report_rs)) +DRM_NOTE ("Skipping spurious, invalid OA report\n" );continue ; }
@@ -913,6 +914,11 @@ static void i915_oa_stream_destroy(struct i915_perf_stream *stream)oa_put_render_ctx_id (stream); dev_priv->perf.oa.exclusive_stream =NULL ; + +if (dev_priv->perf.oa.spurious_report_rs.missed) { +DRM_NOTE ("%d spurious OA report notices suppressed due to ratelimiting\n" , + dev_priv->perf.oa.spurious_report_rs.missed); + } }static void gen7_init_oa_buffer (struct drm_i915_private *dev_priv)
@@ -1268,6 +1274,26 @@ static int i915_oa_stream_init(struct i915_perf_stream *stream,return -EINVAL ; } +/* We set up some ratelimit state to potentially throttle any _NOTES +* about spurious, invalid OA reports which we don't forward to +* userspace. +* +* The initialization is associated with opening the stream (not driver +* init) considering we print a _NOTE about any throttling when closing +* the stream instead of waiting until driver _fini which no one would +* ever see. +* +* Using the same limiting factors as printk_ratelimit() +*/ +ratelimit_state_init (&dev_priv->perf.oa.spurious_report_rs, +5 *HZ ,10 ); +/* Since we use a DRM_NOTE for spurious reports it would be +* inconsistent to let __ratelimit() automatically print a warning for +* throttling. +*/ +ratelimit_set_flags (&dev_priv->perf.oa.spurious_report_rs, +RATELIMIT_MSG_ON_RELEASE ); + stream->sample_size =sizeof (struct drm_i915_perf_record_header ); format_size = dev_priv->perf.oa.oa_formats[props->oa_format].size;
Signature · 6 lines
2.12.0 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v4 9/15 drm/i915: expose _SLICE_MASK GETPARM
In reply to Robert BraggEnables userspace to determine the number of slices enabled and also know what specific slices are enabled. This information is required, for example, to be able to analyse some OA counter reports where the counter configuration depends on the HW slice configuration.
--- drivers/gpu/drm/i915/i915_drv.c | 5 +++++ include/uapi/drm/i915_drm.h | 3 +++ 2 files changed, 8 insertions(+)
drivers/gpu/drm/i915/i915_drv.c +5 -0
@@ -357,6 +357,11 @@ static int i915_getparam(struct drm_device *dev, void *data,*/ value =1 ;break ; +case I915_PARAM_SLICE_MASK : + value =INTEL_INFO (dev_priv)->sseu.slice_mask; +if (!value) +return -ENODEV ; +break ;default :DRM_DEBUG ("Unknown parameter %d\n" , param->param);return -EINVAL ;
include/uapi/drm/i915_drm.h +3 -0
@@ -412,6 +412,9 @@ typedef struct drm_i915_irq_wait {*/ #define I915_PARAM_HAS_EXEC_FENCE 44 +/* Query the mask of slices available for this system */ +#define I915_PARAM_SLICE_MASK 45 + typedef struct drm_i915_getparam { __s32 param; /*
Signature · 6 lines
2.12.0 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v4 10/15 drm/i915: expose _SUBSLICE_MASK GETPARM
In reply to Robert BraggAssuming a uniform mask across all slices, this enables userspace to determine the specific sub slices enabled. This information is required, for example, to be able to analyse some OA counter reports where the counter configuration depends on the HW sub slice configuration.
--- drivers/gpu/drm/i915/i915_drv.c | 5 +++++ include/uapi/drm/i915_drm.h | 5 +++++ 2 files changed, 10 insertions(+)
drivers/gpu/drm/i915/i915_drv.c +5 -0
@@ -362,6 +362,11 @@ static int i915_getparam(struct drm_device *dev, void *data,if (!value)return -ENODEV ;break ; +case I915_PARAM_SUBSLICE_MASK : + value =INTEL_INFO (dev_priv)->sseu.subslice_mask; +if (!value) +return -ENODEV ; +break ;default :DRM_DEBUG ("Unknown parameter %d\n" , param->param);return -EINVAL ;
include/uapi/drm/i915_drm.h +5 -0
@@ -415,6 +415,11 @@ typedef struct drm_i915_irq_wait {/* Query the mask of slices available for this system */ #define I915_PARAM_SLICE_MASK 45 +/* Assuming it's uniform for each slice, this queries the mask of subslices +* per-slice for this system. +*/ +#define I915_PARAM_SUBSLICE_MASK 46 + typedef struct drm_i915_getparam { __s32 param; /*
Signature · 6 lines
2.12.0 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v4 11/15 drm/i915/perf: Add 'render basic' Gen8+ OA unit configs
In reply to Robert BraggAdds a static OA unit, MUX, B Counter + Flex EU configurations for basic render metrics on Broadwell, Cherryview, Skylake and Broxton. These are auto generated from an XML description of metric sets, currently maintained in gputop, ref:
Quoted 2 lines
gputop-data/oa-*.xmlscripts/i915-perf-kernelgen.py
$ make -C gputop-data -f Makefile.xml WHITELIST=RenderBasic
v2: add newlines to debug messages + fix comment (Matthew Auld)
--- drivers/gpu/drm/i915/Makefile | 8 +- drivers/gpu/drm/i915/i915_drv.h | 2 + drivers/gpu/drm/i915/i915_oa_bdw.c | 380 ++++++++++++++++++++++++++++++++++ drivers/gpu/drm/i915/i915_oa_bdw.h | 38 ++++ drivers/gpu/drm/i915/i915_oa_bxt.c | 238 +++++++++++++++++++++ drivers/gpu/drm/i915/i915_oa_bxt.h | 38 ++++ drivers/gpu/drm/i915/i915_oa_chv.c | 225 ++++++++++++++++++++ drivers/gpu/drm/i915/i915_oa_chv.h | 38 ++++ drivers/gpu/drm/i915/i915_oa_sklgt2.c | 228 ++++++++++++++++++++ drivers/gpu/drm/i915/i915_oa_sklgt2.h | 38 ++++ drivers/gpu/drm/i915/i915_oa_sklgt3.c | 236 +++++++++++++++++++++ drivers/gpu/drm/i915/i915_oa_sklgt3.h | 38 ++++ drivers/gpu/drm/i915/i915_oa_sklgt4.c | 247 ++++++++++++++++++++++ drivers/gpu/drm/i915/i915_oa_sklgt4.h | 38 ++++ 14 files changed, 1791 insertions(+), 1 deletion(-) create mode 100644 drivers/gpu/drm/i915/i915_oa_bdw.c create mode 100644 drivers/gpu/drm/i915/i915_oa_bdw.h create mode 100644 drivers/gpu/drm/i915/i915_oa_bxt.c create mode 100644 drivers/gpu/drm/i915/i915_oa_bxt.h create mode 100644 drivers/gpu/drm/i915/i915_oa_chv.c create mode 100644 drivers/gpu/drm/i915/i915_oa_chv.h create mode 100644 drivers/gpu/drm/i915/i915_oa_sklgt2.c create mode 100644 drivers/gpu/drm/i915/i915_oa_sklgt2.h create mode 100644 drivers/gpu/drm/i915/i915_oa_sklgt3.c create mode 100644 drivers/gpu/drm/i915/i915_oa_sklgt3.h create mode 100644 drivers/gpu/drm/i915/i915_oa_sklgt4.c create mode 100644 drivers/gpu/drm/i915/i915_oa_sklgt4.h
drivers/gpu/drm/i915/Makefile +7 -1
@@ -127,7 +127,13 @@ i915-y += i915_vgpu.o # perf code i915-y += i915_perf.o \ - i915_oa_hsw.o + i915_oa_hsw.o \ + i915_oa_bdw.o \ + i915_oa_chv.o \ + i915_oa_sklgt2.o \ + i915_oa_sklgt3.o \ + i915_oa_sklgt4.o \ + i915_oa_bxt.o ifeq ($(CONFIG_DRM_I915_GVT),y) i915-y += intel_gvt.o
drivers/gpu/drm/i915/i915_drv.h +2 -0
@@ -2463,6 +2463,8 @@ struct drm_i915_private {int mux_regs_len;const struct i915_oa_reg *b_counter_regs;int b_counter_regs_len; +const struct i915_oa_reg *flex_regs; +int flex_regs_len;struct {struct i915_vma *vma;
drivers/gpu/drm/i915/i915_oa_bdw.c new file +380 -0
380 lines not on this page · raw message
drivers/gpu/drm/i915/i915_oa_bdw.h new file +38 -0
@@ -0,0 +1,38 @@ +/* +* Autogenerated file, DO NOT EDIT manually! +* +* Copyright (c) 2015 Intel Corporation +* +* Permission is hereby granted, free of charge, to any person obtaining a +* copy of this software and associated documentation files (the "Software"), +* to deal in the Software without restriction, including without limitation +* the rights to use, copy, modify, merge, publish, distribute, sublicense, +* and/or sell copies of the Software, and to permit persons to whom the +* Software is furnished to do so, subject to the following conditions: +* +* The above copyright notice and this permission notice (including the next +* paragraph) shall be included in all copies or substantial portions of the +* Software. +* +* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL +* THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING +* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS +* IN THE SOFTWARE. +* +*/ + +#ifndef __I915_OA_BDW_H__ +#define __I915_OA_BDW_H__ + +extern int i915_oa_n_builtin_metric_sets_bdw; + +extern int i915_oa_select_metric_set_bdw (struct drm_i915_private *dev_priv); + +extern int i915_perf_register_sysfs_bdw (struct drm_i915_private *dev_priv); + +extern void i915_perf_unregister_sysfs_bdw (struct drm_i915_private *dev_priv); + +#endif
drivers/gpu/drm/i915/i915_oa_bxt.c new file +238 -0
238 lines not on this page · raw message
drivers/gpu/drm/i915/i915_oa_bxt.h new file +38 -0
@@ -0,0 +1,38 @@ +/* +* Autogenerated file, DO NOT EDIT manually! +* +* Copyright (c) 2015 Intel Corporation +* +* Permission is hereby granted, free of charge, to any person obtaining a +* copy of this software and associated documentation files (the "Software"), +* to deal in the Software without restriction, including without limitation +* the rights to use, copy, modify, merge, publish, distribute, sublicense, +* and/or sell copies of the Software, and to permit persons to whom the +* Software is furnished to do so, subject to the following conditions: +* +* The above copyright notice and this permission notice (including the next +* paragraph) shall be included in all copies or substantial portions of the +* Software. +* +* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL +* THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING +* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS +* IN THE SOFTWARE. +* +*/ + +#ifndef __I915_OA_BXT_H__ +#define __I915_OA_BXT_H__ + +extern int i915_oa_n_builtin_metric_sets_bxt; + +extern int i915_oa_select_metric_set_bxt (struct drm_i915_private *dev_priv); + +extern int i915_perf_register_sysfs_bxt (struct drm_i915_private *dev_priv); + +extern void i915_perf_unregister_sysfs_bxt (struct drm_i915_private *dev_priv); + +#endif
drivers/gpu/drm/i915/i915_oa_chv.c new file +225 -0
225 lines not on this page · raw message
drivers/gpu/drm/i915/i915_oa_chv.h new file +38 -0
@@ -0,0 +1,38 @@ +/* +* Autogenerated file, DO NOT EDIT manually! +* +* Copyright (c) 2015 Intel Corporation +* +* Permission is hereby granted, free of charge, to any person obtaining a +* copy of this software and associated documentation files (the "Software"), +* to deal in the Software without restriction, including without limitation +* the rights to use, copy, modify, merge, publish, distribute, sublicense, +* and/or sell copies of the Software, and to permit persons to whom the +* Software is furnished to do so, subject to the following conditions: +* +* The above copyright notice and this permission notice (including the next +* paragraph) shall be included in all copies or substantial portions of the +* Software. +* +* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL +* THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING +* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS +* IN THE SOFTWARE. +* +*/ + +#ifndef __I915_OA_CHV_H__ +#define __I915_OA_CHV_H__ + +extern int i915_oa_n_builtin_metric_sets_chv; + +extern int i915_oa_select_metric_set_chv (struct drm_i915_private *dev_priv); + +extern int i915_perf_register_sysfs_chv (struct drm_i915_private *dev_priv); + +extern void i915_perf_unregister_sysfs_chv (struct drm_i915_private *dev_priv); + +#endif
drivers/gpu/drm/i915/i915_oa_sklgt2.c new file +228 -0
228 lines not on this page · raw message
drivers/gpu/drm/i915/i915_oa_sklgt2.h new file +38 -0
@@ -0,0 +1,38 @@ +/* +* Autogenerated file, DO NOT EDIT manually! +* +* Copyright (c) 2015 Intel Corporation +* +* Permission is hereby granted, free of charge, to any person obtaining a +* copy of this software and associated documentation files (the "Software"), +* to deal in the Software without restriction, including without limitation +* the rights to use, copy, modify, merge, publish, distribute, sublicense, +* and/or sell copies of the Software, and to permit persons to whom the +* Software is furnished to do so, subject to the following conditions: +* +* The above copyright notice and this permission notice (including the next +* paragraph) shall be included in all copies or substantial portions of the +* Software. +* +* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL +* THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING +* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS +* IN THE SOFTWARE. +* +*/ + +#ifndef __I915_OA_SKLGT2_H__ +#define __I915_OA_SKLGT2_H__ + +extern int i915_oa_n_builtin_metric_sets_sklgt2; + +extern int i915_oa_select_metric_set_sklgt2 (struct drm_i915_private *dev_priv); + +extern int i915_perf_register_sysfs_sklgt2 (struct drm_i915_private *dev_priv); + +extern void i915_perf_unregister_sysfs_sklgt2 (struct drm_i915_private *dev_priv); + +#endif
drivers/gpu/drm/i915/i915_oa_sklgt3.c new file +236 -0
236 lines not on this page · raw message
drivers/gpu/drm/i915/i915_oa_sklgt3.h new file +38 -0
@@ -0,0 +1,38 @@ +/* +* Autogenerated file, DO NOT EDIT manually! +* +* Copyright (c) 2015 Intel Corporation +* +* Permission is hereby granted, free of charge, to any person obtaining a +* copy of this software and associated documentation files (the "Software"), +* to deal in the Software without restriction, including without limitation +* the rights to use, copy, modify, merge, publish, distribute, sublicense, +* and/or sell copies of the Software, and to permit persons to whom the +* Software is furnished to do so, subject to the following conditions: +* +* The above copyright notice and this permission notice (including the next +* paragraph) shall be included in all copies or substantial portions of the +* Software. +* +* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL +* THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING +* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS +* IN THE SOFTWARE. +* +*/ + +#ifndef __I915_OA_SKLGT3_H__ +#define __I915_OA_SKLGT3_H__ + +extern int i915_oa_n_builtin_metric_sets_sklgt3; + +extern int i915_oa_select_metric_set_sklgt3 (struct drm_i915_private *dev_priv); + +extern int i915_perf_register_sysfs_sklgt3 (struct drm_i915_private *dev_priv); + +extern void i915_perf_unregister_sysfs_sklgt3 (struct drm_i915_private *dev_priv); + +#endif
drivers/gpu/drm/i915/i915_oa_sklgt4.c new file +247 -0
247 lines not on this page · raw message
drivers/gpu/drm/i915/i915_oa_sklgt4.h new file +38 -0
@@ -0,0 +1,38 @@ +/* +* Autogenerated file, DO NOT EDIT manually! +* +* Copyright (c) 2015 Intel Corporation +* +* Permission is hereby granted, free of charge, to any person obtaining a +* copy of this software and associated documentation files (the "Software"), +* to deal in the Software without restriction, including without limitation +* the rights to use, copy, modify, merge, publish, distribute, sublicense, +* and/or sell copies of the Software, and to permit persons to whom the +* Software is furnished to do so, subject to the following conditions: +* +* The above copyright notice and this permission notice (including the next +* paragraph) shall be included in all copies or substantial portions of the +* Software. +* +* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL +* THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING +* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS +* IN THE SOFTWARE. +* +*/ + +#ifndef __I915_OA_SKLGT4_H__ +#define __I915_OA_SKLGT4_H__ + +extern int i915_oa_n_builtin_metric_sets_sklgt4; + +extern int i915_oa_select_metric_set_sklgt4 (struct drm_i915_private *dev_priv); + +extern int i915_perf_register_sysfs_sklgt4 (struct drm_i915_private *dev_priv); + +extern void i915_perf_unregister_sysfs_sklgt4 (struct drm_i915_private *dev_priv); + +#endif
Signature · 6 lines
2.12.0 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v4 12/15 drm/i915/perf: Add OA unit support for Gen 8+
In reply to Robert BraggEnables access to OA unit metrics for BDW, CHV, SKL and BXT which all share (more-or-less) the same OA unit design.
Of particular note in comparison to Haswell: some OA unit HW config state has become per-context state and as a consequence it is somewhat more complicated to manage synchronous state changes from the cpu while there's no guarantee of what context (if any) is currently actively running on the gpu.
The periodic sampling frequency which can be particularly useful for system-wide analysis (as opposed to command stream synchronised MI_REPORT_PERF_COUNT commands) is perhaps the most surprising state to have become per-context save and restored (while the OABUFFER destination is still a shared, system-wide resource).
This support for gen8+ takes care to consider a number of timing challenges involved in synchronously updating per-context state primarily by programming all config state from the cpu and updating all current and saved contexts synchronously while the OA unit is still disabled.
The driver intentionally avoids depending on command streamer programming to update OA state considering the lack of synchronization between the automatic loading of OACTXCONTROL state (that includes the periodic sampling state and enable state) on context restore and the parsing of any general purpose BB the driver can control. I.e. this implementation is careful to avoid the possibility of a context restore temporarily enabling any out-of-date periodic sampling state. In addition to the risk of transiently-out-of-date state being loaded automatically; there are also internal HW latencies involved in the loading of MUX configurations which would be difficult to account for from the command streamer (and we only want to enable the unit when once the MUX configuration is complete).
Since the Gen8+ OA unit design no longer supports clock gating the unit off for a single given context (which effectively stopped any progress of counters while any other context was running) and instead supports tagging OA reports with a context ID for filtering on the CPU, it means we can no longer hide the system-wide progress of counters from a non-privileged application only interested in metrics for its own context. Although we could theoretically try and subtract the progress of other contexts before forwarding reports via read() we aren't in a position to filter reports captured via MI_REPORT_PERF_COUNT commands. As a result, for Gen8+, we always require the dev.i915.perf_stream_paranoid to be unset for any access to OA metrics if not root.
--- drivers/gpu/drm/i915/i915_drv.h | 45 +- drivers/gpu/drm/i915/i915_gem_context.h | 1 + drivers/gpu/drm/i915/i915_perf.c | 949 +++++++++++++++++++++++++++++--- drivers/gpu/drm/i915/i915_reg.h | 22 + drivers/gpu/drm/i915/intel_lrc.c | 5 + include/uapi/drm/i915_drm.h | 19 +- 6 files changed, 948 insertions(+), 93 deletions(-)
drivers/gpu/drm/i915/i915_drv.h +30 -15
@@ -2061,9 +2061,17 @@ struct i915_oa_ops {void (*init_oa_buffer)(struct drm_i915_private *dev_priv);/** -* @enable_metric_set: Applies any MUX configuration to set up the -* Boolean and Custom (B/C) counters that are part of the counter -* reports being sampled. May apply system constraints such as +* @select_metric_set: The auto generated code that checks whether a +* requested OA config is applicable to the system and if so sets up +* the mux, oa and flex eu register config pointers according to the +* current dev_priv->perf.oa.metrics_set. +*/ +int (*select_metric_set)(struct drm_i915_private *dev_priv); + +/** +* @enable_metric_set: Selects and applies any MUX configuration to set +* up the Boolean and Custom (B/C) counters that are part of the +* counter reports being sampled. May apply system constraints such as * disabling EU clock gating as required. */ int (*enable_metric_set)(struct drm_i915_private *dev_priv);
@@ -2094,20 +2102,13 @@ struct i915_oa_ops { size_t *offset);/** -* @oa_buffer_check: Check for OA buffer data + update tail -* -* This is either called via fops or the poll check hrtimer (atomic -* ctx) without any locks taken. +* @oa_hw_tail_read: read the OA tail pointer register * -* It's safe to read OA config state here unlocked, assuming that this -* is only called while the stream is enabled, while the global OA -* configuration can't be modified. -* -* Efficiency is more important than avoiding some false positives -* here, which will be handled gracefully - likely resulting in an -* %EAGAIN error for userspace. +* In particular this enables us to share all the fiddly code for +* handling the OA unit tail pointer race that affects multiple +* generations. */ -bool (*oa_buffer_check)(struct drm_i915_private *dev_priv); +u32 (*oa_hw_tail_read)(struct drm_i915_private *dev_priv); };struct intel_cdclk_state {
@@ -2469,6 +2470,7 @@ struct drm_i915_private {struct {struct i915_vma *vma;u8 *vaddr; +u32 last_ctx_id;int format;int format_size;
@@ -2538,6 +2540,14 @@ struct drm_i915_private { } oa_buffer;u32 gen7_latched_oastatus1; +u32 ctx_oactxctrl_offset; +u32 ctx_flexeu0_offset; + +/* The RPT_ID/reason field for Gen8+ includes a bit +* to determine if the CTX ID in the report is valid +* but the specific bit differs between Gen 8 and 9 +*/ +u32 gen8_valid_ctx_bit;struct i915_oa_ops ops;const struct i915_oa_format *oa_formats;
@@ -2848,6 +2858,8 @@ intel_info(const struct drm_i915_private *dev_priv)#define IS_KBL_ULX (dev_priv) (INTEL_DEVID(dev_priv) == 0x590E || \ INTEL_DEVID(dev_priv) == 0x5915 || \ INTEL_DEVID(dev_priv) == 0x591E) +#define IS_SKL_GT2 (dev_priv) (IS_SKYLAKE(dev_priv) && \ + (INTEL_DEVID(dev_priv) & 0x00F0) == 0x0010)#define IS_SKL_GT3 (dev_priv) (IS_SKYLAKE(dev_priv) && \ (INTEL_DEVID(dev_priv) & 0x00F0) == 0x0020)#define IS_SKL_GT4 (dev_priv) (IS_SKYLAKE(dev_priv) && \
@@ -3612,6 +3624,9 @@ i915_gem_context_lookup_timeline(struct i915_gem_context *ctx,int i915_perf_open_ioctl (struct drm_device *dev,void *data,struct drm_file *file); +void i915_oa_update_reg_state (struct intel_engine_cs *engine, +struct i915_gem_context *ctx, +uint32_t *reg_state);/* i915_gem_evict.c */ int __must_check i915_gem_evict_something(struct i915_address_space *vm,
drivers/gpu/drm/i915/i915_gem_context.h +1 -0
@@ -151,6 +151,7 @@ struct i915_gem_context {u64 lrc_desc;int pin_count;bool initialised; +atomic_t oa_state_dirty; } engine[I915_NUM_ENGINES ];/** ring_size: size for allocating the per-engine ring buffer */
drivers/gpu/drm/i915/i915_perf.c +878 -71
1,138 lines not on this page · raw message
drivers/gpu/drm/i915/i915_reg.h +22 -0
@@ -653,6 +653,12 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)#define GEN8_OACTXID _MMIO(0x2364) +#define GEN8_OA_DEBUG _MMIO(0x2B04) +#define GEN9_OA_DEBUG_DISABLE_CLK_RATIO_REPORTS (1<<5) +#define GEN9_OA_DEBUG_INCLUDE_CLK_RATIO (1<<6) +#define GEN9_OA_DEBUG_DISABLE_GO_1_0_REPORTS (1<<2) +#define GEN9_OA_DEBUG_DISABLE_CTX_SWITCH_REPORTS (1<<1) +#define GEN8_OACONTROL _MMIO(0x2B00)#define GEN8_OA_REPORT_FORMAT_A12 (0<<2)#define GEN8_OA_REPORT_FORMAT_A12_B8_C8 (2<<2)
@@ -674,6 +680,7 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)#define GEN7_OABUFFER_STOP_RESUME_ENABLE (1<<1)#define GEN7_OABUFFER_RESUME (1<<0) +#define GEN8_OABUFFER_UDW _MMIO(0x23b4)#define GEN8_OABUFFER _MMIO(0x2b14)#define GEN7_OASTATUS1 _MMIO(0x2364)
@@ -692,7 +699,9 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)#define GEN8_OASTATUS_REPORT_LOST (1<<0)#define GEN8_OAHEADPTR _MMIO(0x2B0C) +#define GEN8_OAHEADPTR_MASK 0xffffffc0#define GEN8_OATAILPTR _MMIO(0x2B10) +#define GEN8_OATAILPTR_MASK 0xffffffc0#define OABUFFER_SIZE_128K (0<<3)#define OABUFFER_SIZE_256K (1<<3)
@@ -705,7 +714,17 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)#define OA_MEM_SELECT_GGTT (1<<0) +/* +* Flexible, Aggregate EU Counter Registers. +* Note: these aren't contiguous +*/ #define EU_PERF_CNTL0 _MMIO(0xe458) +#define EU_PERF_CNTL1 _MMIO(0xe558) +#define EU_PERF_CNTL2 _MMIO(0xe658) +#define EU_PERF_CNTL3 _MMIO(0xe758) +#define EU_PERF_CNTL4 _MMIO(0xe45c) +#define EU_PERF_CNTL5 _MMIO(0xe55c) +#define EU_PERF_CNTL6 _MMIO(0xe65c)#define GDT_CHICKEN_BITS _MMIO(0x9840)#define GT_NOA_ENABLE 0x00000080
@@ -2325,6 +2344,9 @@ enum skl_disp_power_wells {#define GEN8_RC_SEMA_IDLE_MSG_DISABLE (1 << 12)#define GEN8_FF_DOP_CLOCK_GATE_DISABLE (1<<10) +#define GEN6_RCS_PWR_FSM _MMIO(0x22ac) +#define GEN9_RCS_FE_FSM2 _MMIO(0x22a4) +/* Fuse readout registers for GT */ #define CHV_FUSE_GT _MMIO(VLV_DISPLAY_BASE + 0x2168)#define CHV_FGT_DISABLE_SS0 (1 << 10)
drivers/gpu/drm/i915/intel_lrc.c +5 -0
@@ -337,6 +337,8 @@ static u64 execlists_update_context(struct drm_i915_gem_request *rq)if (ppgtt && !i915_vm_is_48bit (&ppgtt->base))execlists_update_context_pdps (ppgtt, reg_state); +i915_oa_update_reg_state (rq->engine, rq->ctx, reg_state); +return ce->lrc_desc; }
@@ -1879,6 +1881,9 @@ static void execlists_init_reg_state(u32 *regs, regs[CTX_LRI_HEADER_2 ] =MI_LOAD_REGISTER_IMM (1 );CTX_REG (regs,CTX_R_PWR_CLK_STATE ,GEN8_R_PWR_CLK_STATE ,make_rpcs (dev_priv)); + +atomic_set (&ctx->engine[RCS ].oa_state_dirty,1 ); +i915_oa_update_reg_state (engine, ctx, regs); } }
include/uapi/drm/i915_drm.h +12 -7
@@ -1303,13 +1303,18 @@ struct drm_i915_gem_context_param { };enum drm_i915_oa_format { -I915_OA_FORMAT_A13 =1 , -I915_OA_FORMAT_A29 , -I915_OA_FORMAT_A13_B8_C8 , -I915_OA_FORMAT_B4_C8 , -I915_OA_FORMAT_A45_B8_C8 , -I915_OA_FORMAT_B4_C8_A16 , -I915_OA_FORMAT_C4_B8 , +I915_OA_FORMAT_A13 =1 ,/* HSW only */ +I915_OA_FORMAT_A29 ,/* HSW only */ +I915_OA_FORMAT_A13_B8_C8 ,/* HSW only */ +I915_OA_FORMAT_B4_C8 ,/* HSW only */ +I915_OA_FORMAT_A45_B8_C8 ,/* HSW only */ +I915_OA_FORMAT_B4_C8_A16 ,/* HSW only */ +I915_OA_FORMAT_C4_B8 ,/* HSW+ */ + +/* Gen8+ */ +I915_OA_FORMAT_A12 , +I915_OA_FORMAT_A12_B8_C8 , + I915_OA_FORMAT_A32u40_A4u32_B8_C8,I915_OA_FORMAT_MAX /* non-ABI */ };
Signature · 6 lines
2.12.0 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [PATCH v4 12/15] drm/i915/perf: Add OA unit support for Gen 8+
Quoted 47 lines · Robert Bragg
Enables access to OA unit metrics for BDW, CHV, SKL and BXT which all share (more-or-less) the same OA unit design.
Of particular note in comparison to Haswell: some OA unit HW config state has become per-context state and as a consequence it is somewhat more complicated to manage synchronous state changes from the cpu while there's no guarantee of what context (if any) is currently actively running on the gpu.
The periodic sampling frequency which can be particularly useful for system-wide analysis (as opposed to command stream synchronised
MI_REPORT_PERF_COUNTcommands) is perhaps the most surprising state to have become per-context save and restored (while the OABUFFER destination is still a shared, system-wide resource).This support for gen8+ takes care to consider a number of timing challenges involved in synchronously updating per-context state primarily by programming all config state from the cpu and updating all current and saved contexts synchronously while the OA unit is still disabled.
The driver intentionally avoids depending on command streamer programming to update OA state considering the lack of synchronization between the automatic loading of OACTXCONTROL state (that includes the periodic sampling state and enable state) on context restore and the parsing of any general purpose BB the driver can control. I.e. this implementation is careful to avoid the possibility of a context restore temporarily enabling any out-of-date periodic sampling state. In addition to the risk of transiently-out-of-date state being loaded automatically; there are also internal HW latencies involved in the loading of MUX configurations which would be difficult to account for from the command streamer (and we only want to enable the unit when once the MUX configuration is complete).
Since the Gen8+ OA unit design no longer supports clock gating the unit off for a single given context (which effectively stopped any progress of counters while any other context was running) and instead supports tagging OA reports with a context ID for filtering on the CPU, it means we can no longer hide the system-wide progress of counters from a non-privileged application only interested in metrics for its own context. Although we could theoretically try and subtract the progress of other contexts before forwarding reports via
read()we aren't in a position to filter reports captured viaMI_REPORT_PERF_COUNTcommands. As a result, for Gen8+, we always require the dev.i915.perf_stream_paranoid to be unset for any access to OA metrics if not root.
Quoted 2 lines
Signed-off-by Robert Bragg <[email protected]>
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v4 13/15 drm/i915/perf: Add more OA configs for BDW, CHV, SKL + BXT
In reply to Robert BraggThese are auto generated from an XML description of metric sets, currently maintained in gputop, ref:
Quoted 2 lines
gputop-data/oa-*.xmlscripts/i915-perf-kernelgen.py
$ make -C gputop-data -f Makefile.xml
--- drivers/gpu/drm/i915/i915_oa_bdw.c | 4856 ++++++++++++++++++++++++++++++++- drivers/gpu/drm/i915/i915_oa_bxt.c | 2323 +++++++++++++++- drivers/gpu/drm/i915/i915_oa_chv.c | 2525 ++++++++++++++++- drivers/gpu/drm/i915/i915_oa_hsw.c | 58 +- drivers/gpu/drm/i915/i915_oa_sklgt2.c | 3156 ++++++++++++++++++++- drivers/gpu/drm/i915/i915_oa_sklgt3.c | 2702 +++++++++++++++++- drivers/gpu/drm/i915/i915_oa_sklgt4.c | 2745 ++++++++++++++++++- 7 files changed, 18176 insertions(+), 189 deletions(-)
drivers/gpu/drm/i915/i915_oa_bdw.c +4815 -41
@@ -31,9 +31,30 @@enum metric_set_id {METRIC_SET_ID_RENDER_BASIC =1 , +METRIC_SET_ID_COMPUTE_BASIC , +METRIC_SET_ID_RENDER_PIPE_PROFILE , +METRIC_SET_ID_MEMORY_READS , +METRIC_SET_ID_MEMORY_WRITES , +METRIC_SET_ID_COMPUTE_EXTENDED , +METRIC_SET_ID_COMPUTE_L3_CACHE , +METRIC_SET_ID_DATA_PORT_READS_COALESCING , +METRIC_SET_ID_DATA_PORT_WRITES_COALESCING , +METRIC_SET_ID_HDC_AND_SF , +METRIC_SET_ID_L3_1 , +METRIC_SET_ID_L3_2 , +METRIC_SET_ID_L3_3 , +METRIC_SET_ID_L3_4 , +METRIC_SET_ID_RASTERIZER_AND_PIXEL_BACKEND , +METRIC_SET_ID_SAMPLER_1 , +METRIC_SET_ID_SAMPLER_2 , +METRIC_SET_ID_TDL_1 , +METRIC_SET_ID_TDL_2 , +METRIC_SET_ID_COMPUTE_EXTRA , +METRIC_SET_ID_VME_PIPE , +METRIC_SET_ID_TEST_OA , }; -int i915_oa_n_builtin_metric_sets_bdw =1 ; +int i915_oa_n_builtin_metric_sets_bdw =22 ;static const struct i915_oa_reg b_counter_config_render_basic[] = { {_MMIO (0x2710 ),0x00000000 },
@@ -290,66 +311,4609 @@ get_render_basic_mux_config(struct drm_i915_private *dev_priv,return NULL ; } +static const struct i915_oa_reg b_counter_config_compute_basic[] = { + {_MMIO (0x2710 ),0x00000000 }, + {_MMIO (0x2714 ),0x00800000 }, + {_MMIO (0x2720 ),0x00000000 }, + {_MMIO (0x2724 ),0x00800000 }, + {_MMIO (0x2740 ),0x00000000 }, +}; + +static const struct i915_oa_reg flex_eu_config_compute_basic[] = { + {_MMIO (0xe458 ),0x00005004 }, + {_MMIO (0xe558 ),0x00000003 }, + {_MMIO (0xe658 ),0x00002001 }, + {_MMIO (0xe758 ),0x00778008 }, + {_MMIO (0xe45c ),0x00088078 }, + {_MMIO (0xe55c ),0x00808708 }, + {_MMIO (0xe65c ),0x00a08908 }, +}; + +static const struct i915_oa_reg mux_config_compute_basic_0_slices_0x01[] = { + {_MMIO (0x9888 ),0x105c00e0 }, + {_MMIO (0x9888 ),0x105800e0 }, + {_MMIO (0x9888 ),0x103800e0 }, + {_MMIO (0x9888 ),0x3580001a }, + {_MMIO (0x9888 ),0x3b800060 }, + {_MMIO (0x9888 ),0x3d800005 }, + {_MMIO (0x9888 ),0x065c2100 }, + {_MMIO (0x9888 ),0x0a5c0041 }, + {_MMIO (0x9888 ),0x0c5c6600 }, + {_MMIO (0x9888 ),0x005c6580 }, + {_MMIO (0x9888 ),0x085c8000 }, + {_MMIO (0x9888 ),0x0e5c8000 }, + {_MMIO (0x9888 ),0x00580042 }, + {_MMIO (0x9888 ),0x08582080 }, + {_MMIO (0x9888 ),0x0c58004c }, + {_MMIO (0x9888 ),0x0e582580 }, + {_MMIO (0x9888 ),0x005b4000 }, + {_MMIO (0x9888 ),0x185b1000 }, + {_MMIO (0x9888 ),0x1a5b0104 }, + {_MMIO (0x9888 ),0x0c1fa800 }, + {_MMIO (0x9888 ),0x0e1faa00 }, + {_MMIO (0x9888 ),0x101f02aa }, + {_MMIO (0x9888 ),0x08380042 }, + {_MMIO (0x9888 ),0x0a382080 }, + {_MMIO (0x9888 ),0x0e38404c }, + {_MMIO (0x9888 ),0x0238404b }, + {_MMIO (0x9888 ),0x00384000 }, + {_MMIO (0x9888 ),0x16380000 }, + {_MMIO (0x9888 ),0x18381145 }, + {_MMIO (0x9888 ),0x04380000 }, + {_MMIO (0x9888 ),0x0039a000 }, + {_MMIO (0x9888 ),0x06398000 }, + {_MMIO (0x9888 ),0x0839a000 }, + {_MMIO (0x9888 ),0x0a39a000 }, + {_MMIO (0x9888 ),0x0c39a000 }, + {_MMIO (0x9888 ),0x0e39a000 }, + {_MMIO (0x9888 ),0x02392000 }, + {_MMIO (0x9888 ),0x018a8000 }, + {_MMIO (0x9888 ),0x0f8a8000 }, + {_MMIO (0x9888 ),0x198a8000 }, + {_MMIO (0x9888 ),0x1b8aaaa0 }, + {_MMIO (0x9888 ),0x1d8a0002 }, + {_MMIO (0x9888 ),0x038a8000 }, + {_MMIO (0x9888 ),0x058a8000 }, + {_MMIO (0x9888 ),0x238b02a0 }, + {_MMIO (0x9888 ),0x258b5550 }, + {_MMIO (0x9888 ),0x278b0015 }, + {_MMIO (0x9888 ),0x1f850a80 }, + {_MMIO (0x9888 ),0x2185aaa0 }, + {_MMIO (0x9888 ),0x2385002a }, + {_MMIO (0x9888 ),0x01834000 }, + {_MMIO (0x9888 ),0x0f834000 }, + {_MMIO (0x9888 ),0x19835400 }, + {_MMIO (0x9888 ),0x1b830155 }, + {_MMIO (0x9888 ),0x03834000 }, + {_MMIO (0x9888 ),0x05834000 }, + {_MMIO (0x9888 ),0x0184c000 }, + {_MMIO (0x9888 ),0x07848000 }, + {_MMIO (0x9888 ),0x0984c000 }, + {_MMIO (0x9888 ),0x0b84c000 }, + {_MMIO (0x9888 ),0x0d84c000 }, + {_MMIO (0x9888 ),0x0f84c000 }, + {_MMIO (0x9888 ),0x03844000 }, + {_MMIO (0x9888 ),0x17808137 }, + {_MMIO (0x9888 ),0x1980c147 }, + {_MMIO (0x9888 ),0x1b80c0e5 }, + {_MMIO (0x9888 ),0x1d80c0e3 }, + {_MMIO (0x9888 ),0x21800000 }, + {_MMIO (0x9888 ),0x1180c000 }, + {_MMIO (0x9888 ),0x1f80c000 }, + {_MMIO (0x9888 ),0x13804000 }, + {_MMIO (0x9888 ),0x15800000 }, + {_MMIO (0xd24 ),0x00000000 }, + {_MMIO (0x9888 ),0x4d801000 }, + {_MMIO (0x9888 ),0x4f800111 }, + {_MMIO (0x9888 ),0x43800062 }, + {_MMIO (0x9888 ),0x51800000 }, + {_MMIO (0x9888 ),0x45800062 }, + {_MMIO (0x9888 ),0x53800000 }, + {_MMIO (0x9888 ),0x47800062 }, + {_MMIO (0x9888 ),0x31800000 }, + {_MMIO (0x9888 ),0x3f801062 }, + {_MMIO (0x9888 ),0x41801084 }, +}; + +static const struct i915_oa_reg mux_config_compute_basic_2_slices_0x02[] = { + {_MMIO (0x9888 ),0x10dc00e0 }, + {_MMIO (0x9888 ),0x10d800e0 }, + {_MMIO (0x9888 ),0x10b800e0 }, + {_MMIO (0x9888 ),0x3580001a }, + {_MMIO (0x9888 ),0x3b800060 }, + {_MMIO (0x9888 ),0x3d800005 }, + {_MMIO (0x9888 ),0x06dc2100 }, + {_MMIO (0x9888 ),0x0adc0041 }, + {_MMIO (0x9888 ),0x0cdc6600 }, + {_MMIO (0x9888 ),0x00dc6580 }, + {_MMIO (0x9888 ),0x08dc8000 }, + {_MMIO (0x9888 ),0x0edc8000 }, + {_MMIO (0x9888 ),0x00d80042 }, + {_MMIO (0x9888 ),0x08d82080 }, + {_MMIO (0x9888 ),0x0cd8004c }, + {_MMIO (0x9888 ),0x0ed82580 }, + {_MMIO (0x9888 ),0x00db4000 }, + {_MMIO (0x9888 ),0x18db1000 }, + {_MMIO (0x9888 ),0x1adb0104 }, + {_MMIO (0x9888 ),0x0c9fa800 }, + {_MMIO (0x9888 ),0x0e9faa00 }, + {_MMIO (0x9888 ),0x109f02aa }, + {_MMIO (0x9888 ),0x08b80042 }, + {_MMIO (0x9888 ),0x0ab82080 }, + {_MMIO (0x9888 ),0x0eb8404c }, + {_MMIO (0x9888 ),0x02b8404b }, + {_MMIO (0x9888 ),0x00b84000 }, + {_MMIO (0x9888 ),0x16b80000 }, + {_MMIO (0x9888 ),0x18b81145 }, + {_MMIO (0x9888 ),0x04b80000 }, + {_MMIO (0x9888 ),0x00b9a000 }, + {_MMIO (0x9888 ),0x06b98000 }, + {_MMIO (0x9888 ),0x08b9a000 }, + {_MMIO (0x9888 ),0x0ab9a000 }, + {_MMIO (0x9888 ),0x0cb9a000 }, + {_MMIO (0x9888 ),0x0eb9a000 }, + {_MMIO (0x9888 ),0x02b92000 }, + {_MMIO (0x9888 ),0x01888000 }, + {_MMIO (0x9888 ),0x0d88f800 }, + {_MMIO (0x9888 ),0x0f88000f }, + {_MMIO (0x9888 ),0x03888000 }, + {_MMIO (0x9888 ),0x05888000 }, + {_MMIO (0x9888 ),0x238b0540 }, + {_MMIO (0x9888 ),0x258baaa0 }, + {_MMIO (0x9888 ),0x278b002a }, + {_MMIO (0x9888 ),0x018c4000 }, + {_MMIO (0x9888 ),0x0f8c4000 }, + {_MMIO (0x9888 ),0x178c2000 }, + {_MMIO (0x9888 ),0x198c5500 }, + {_MMIO (0x9888 ),0x1b8c0015 }, + {_MMIO (0x9888 ),0x038c4000 }, + {_MMIO (0x9888 ),0x058c4000 }, + {_MMIO (0x9888 ),0x018da000 }, + {_MMIO (0x9888 ),0x078d8000 }, + {_MMIO (0x9888 ),0x098da000 }, + {_MMIO (0x9888 ),0x0b8da000 }, + {_MMIO (0x9888 ),0x0d8da000 }, + {_MMIO (0x9888 ),0x0f8da000 }, + {_MMIO (0x9888 ),0x038d2000 }, + {_MMIO (0x9888 ),0x1f850a80 }, + {_MMIO (0x9888 ),0x2185aaa0 }, + {_MMIO (0x9888 ),0x2385002a }, + {_MMIO (0x9888 ),0x01834000 }, + {_MMIO (0x9888 ),0x0f834000 }, + {_MMIO (0x9888 ),0x19835400 }, + {_MMIO (0x9888 ),0x1b830155 }, + {_MMIO (0x9888 ),0x03834000 }, + {_MMIO (0x9888 ),0x05834000 }, + {_MMIO (0x9888 ),0x0184c000 }, + {_MMIO (0x9888 ),0x07848000 }, + {_MMIO (0x9888 ),0x0984c000 }, + {_MMIO (0x9888 ),0x0b84c000 }, + {_MMIO (0x9888 ),0x0d84c000 }, + {_MMIO (0x9888 ),0x0f84c000 }, + {_MMIO (0x9888 ),0x03844000 }, + {_MMIO (0x9888 ),0x17808137 }, + {_MMIO (0x9888 ),0x1980c147 }, + {_MMIO (0x9888 ),0x1b80c0e5 }, + {_MMIO (0x9888 ),0x1d80c0e3 }, + {_MMIO (0x9888 ),0x21800000 }, + {_MMIO (0x9888 ),0x1180c000 }, + {_MMIO (0x9888 ),0x1f80c000 }, + {_MMIO (0x9888 ),0x13804000 }, + {_MMIO (0x9888 ),0x15800000 }, + {_MMIO (0xd24 ),0x00000000 }, + {_MMIO (0x9888 ),0x4d805000 }, + {_MMIO (0x9888 ),0x4f800555 }, + {_MMIO (0x9888 ),0x43800062 }, + {_MMIO (0x9888 ),0x51800000 }, + {_MMIO (0x9888 ),0x45800062 }, + {_MMIO (0x9888 ),0x53800000 }, + {_MMIO (0x9888 ),0x47800062 }, + {_MMIO (0x9888 ),0x31800000 }, + {_MMIO (0x9888 ),0x3f800062 }, + {_MMIO (0x9888 ),0x41800000 }, +}; + +static const struct i915_oa_reg * +get_compute_basic_mux_config (struct drm_i915_private *dev_priv, +int *len) +{ +if (INTEL_INFO (dev_priv)->sseu.slice_mask &0x01 ) { + *len =ARRAY_SIZE (mux_config_compute_basic_0_slices_0x01); +return mux_config_compute_basic_0_slices_0x01; + }else if (INTEL_INFO (dev_priv)->sseu.slice_mask &0x02 ) { + *len =ARRAY_SIZE (mux_config_compute_basic_2_slices_0x02); +return mux_config_compute_basic_2_slices_0x02; + } + + *len =0 ; +return NULL ; +} + +static const struct i915_oa_reg b_counter_config_render_pipe_profile[] = { + {_MMIO (0x2724 ),0xf0800000 }, + {_MMIO (0x2720 ),0x00000000 }, + {_MMIO (0x2714 ),0xf0800000 }, + {_MMIO (0x2710 ),0x00000000 }, + {_MMIO (0x2740 ),0x00000000 }, + {_MMIO (0x2770 ),0x0007ffea }, + {_MMIO (0x2774 ),0x00007ffc }, + {_MMIO (0x2778 ),0x0007affa }, + {_MMIO (0x277c ),0x0000f5fd }, + {_MMIO (0x2780 ),0x00079ffa }, + {_MMIO (0x2784 ),0x0000f3fb }, + {_MMIO (0x2788 ),0x0007bf7a }, + {_MMIO (0x278c ),0x0000f7e7 }, + {_MMIO (0x2790 ),0x0007fefa }, + {_MMIO (0x2794 ),0x0000f7cf }, + {_MMIO (0x2798 ),0x00077ffa }, + {_MMIO (0x279c ),0x0000efdf }, + {_MMIO (0x27a0 ),0x0006fffa }, + {_MMIO (0x27a4 ),0x0000cfbf }, + {_MMIO (0x27a8 ),0x0003fffa }, + {_MMIO (0x27ac ),0x00005f7f }, +}; + +static const struct i915_oa_reg flex_eu_config_render_pipe_profile[] = { + {_MMIO (0xe458 ),0x00005004 }, + {_MMIO (0xe558 ),0x00015014 }, + {_MMIO (0xe658 ),0x00025024 }, + {_MMIO (0xe758 ),0x00035034 }, + {_MMIO (0xe45c ),0x00045044 }, + {_MMIO (0xe55c ),0x00055054 }, + {_MMIO (0xe65c ),0x00065064 }, +}; + +static const struct i915_oa_reg mux_config_render_pipe_profile[] = { + {_MMIO (0x9888 ),0x0a1e0000 }, + {_MMIO (0x9888 ),0x0c1f000f }, + {_MMIO (0x9888 ),0x10176800 }, + {_MMIO (0x9888 ),0x1191001f }, + {_MMIO (0x9888 ),0x0b880320 }, + {_MMIO (0x9888 ),0x01890c40 }, + {_MMIO (0x9888 ),0x118a1c00 }, + {_MMIO (0x9888 ),0x118d7c00 }, + {_MMIO (0x9888 ),0x118e0020 }, + {_MMIO (0x9888 ),0x118f4c00 }, + {_MMIO (0x9888 ),0x11900000 }, + {_MMIO (0x9888 ),0x13900001 }, + {_MMIO (0x9888 ),0x065c4000 }, + {_MMIO (0x9888 ),0x0c3d8000 }, + {_MMIO (0x9888 ),0x06584000 }, + {_MMIO (0x9888 ),0x0c5b4000 }, + {_MMIO (0x9888 ),0x081e0040 }, + {_MMIO (0x9888 ),0x0e1e0000 }, + {_MMIO (0x9888 ),0x021f5400 }, + {_MMIO (0x9888 ),0x001f0000 }, + {_MMIO (0x9888 ),0x101f0010 }, + {_MMIO (0x9888 ),0x0e1f0080 }, + {_MMIO (0x9888 ),0x0c384000 }, + {_MMIO (0x9888 ),0x06392000 }, + {_MMIO (0x9888 ),0x0c13c000 }, + {_MMIO (0x9888 ),0x06164000 }, + {_MMIO (0x9888 ),0x06170012 }, + {_MMIO (0x9888 ),0x00170000 }, + {_MMIO (0x9888 ),0x01910005 }, + {_MMIO (0x9888 ),0x07880002 }, + {_MMIO (0x9888 ),0x01880c00 }, + {_MMIO (0x9888 ),0x0f880000 }, + {_MMIO (0x9888 ),0x0d880000 }, + {_MMIO (0x9888 ),0x05880000 }, + {_MMIO (0x9888 ),0x09890032 }, + {_MMIO (0x9888 ),0x078a0800 }, + {_MMIO (0x9888 ),0x0f8a0a00 }, + {_MMIO (0x9888 ),0x198a4000 }, + {_MMIO (0x9888 ),0x1b8a2000 }, + {_MMIO (0x9888 ),0x1d8a0000 }, + {_MMIO (0x9888 ),0x038a4000 }, + {_MMIO (0x9888 ),0x0b8a8000 }, + {_MMIO (0x9888 ),0x0d8a8000 }, + {_MMIO (0x9888 ),0x238b54c0 }, + {_MMIO (0x9888 ),0x258baa55 }, + {_MMIO (0x9888 ),0x278b0019 }, + {_MMIO (0x9888 ),0x198c0100 }, + {_MMIO (0x9888 ),0x058c4000 }, + {_MMIO (0x9888 ),0x0f8d0015 }, + {_MMIO (0x9888 ),0x018d1000 }, + {_MMIO (0x9888 ),0x098d8000 }, + {_MMIO (0x9888 ),0x0b8df000 }, + {_MMIO (0x9888 ),0x0d8d3000 }, + {_MMIO (0x9888 ),0x038de000 }, + {_MMIO (0x9888 ),0x058d3000 }, + {_MMIO (0x9888 ),0x0d8e0004 }, + {_MMIO (0x9888 ),0x058e000c }, + {_MMIO (0x9888 ),0x098e0000 }, + {_MMIO (0x9888 ),0x078e0000 }, + {_MMIO (0x9888 ),0x038e0000 }, + {_MMIO (0x9888 ),0x0b8f0020 }, + {_MMIO (0x9888 ),0x198f0c00 }, + {_MMIO (0x9888 ),0x078f8000 }, + {_MMIO (0x9888 ),0x098f4000 }, + {_MMIO (0x9888 ),0x0b900980 }, + {_MMIO (0x9888 ),0x03900d80 }, + {_MMIO (0x9888 ),0x01900000 }, + {_MMIO (0x9888 ),0x1f85aa80 }, + {_MMIO (0x9888 ),0x2185aaaa }, + {_MMIO (0x9888 ),0x2385002a }, + {_MMIO (0x9888 ),0x01834000 }, + {_MMIO (0x9888 ),0x0f834000 }, + {_MMIO (0x9888 ),0x19835400 }, + {_MMIO (0x9888 ),0x1b830155 }, + {_MMIO (0x9888 ),0x03834000 }, + {_MMIO (0x9888 ),0x05834000 }, + {_MMIO (0x9888 ),0x07834000 }, + {_MMIO (0x9888 ),0x09834000 }, + {_MMIO (0x9888 ),0x0b834000 }, + {_MMIO (0x9888 ),0x0d834000 }, + {_MMIO (0x9888 ),0x0184c000 }, + {_MMIO (0x9888 ),0x0784c000 }, + {_MMIO (0x9888 ),0x0984c000 }, + {_MMIO (0x9888 ),0x0b84c000 }, + {_MMIO (0x9888 ),0x0d84c000 }, + {_MMIO (0x9888 ),0x0f84c000 }, + {_MMIO (0x9888 ),0x0384c000 }, + {_MMIO (0x9888 ),0x0584c000 }, + {_MMIO (0x9888 ),0x1180c000 }, + {_MMIO (0x9888 ),0x1780c000 }, + {_MMIO (0x9888 ),0x1980c000 }, + {_MMIO (0x9888 ),0x1b80c000 }, + {_MMIO (0x9888 ),0x1d80c000 }, + {_MMIO (0x9888 ),0x1f80c000 }, + {_MMIO (0x9888 ),0x1380c000 }, + {_MMIO (0x9888 ),0x1580c000 }, + {_MMIO (0xd24 ),0x00000000 }, + {_MMIO (0x9888 ),0x4d801111 }, + {_MMIO (0x9888 ),0x3d800800 }, + {_MMIO (0x9888 ),0x4f801011 }, + {_MMIO (0x9888 ),0x43800443 }, + {_MMIO (0x9888 ),0x51801111 }, + {_MMIO (0x9888 ),0x45800422 }, + {_MMIO (0x9888 ),0x53801111 }, + {_MMIO (0x9888 ),0x47800c60 }, + {_MMIO (0x9888 ),0x21800000 }, + {_MMIO (0x9888 ),0x31800000 }, + {_MMIO (0x9888 ),0x3f800422 }, + {_MMIO (0x9888 ),0x41800021 }, +}; + +static const struct i915_oa_reg * +get_render_pipe_profile_mux_config (struct drm_i915_private *dev_priv, +int *len) +{ + *len =ARRAY_SIZE (mux_config_render_pipe_profile); +return mux_config_render_pipe_profile; +} + +static const struct i915_oa_reg b_counter_config_memory_reads[] = { + {_MMIO (0x2724 ),0xf0800000 }, + {_MMIO (0x2720 ),0x00000000 }, + {_MMIO (0x2714 ),0xf0800000 }, + {_MMIO (0x2710 ),0x00000000 }, + {_MMIO (0x274c ),0x86543210 }, + {_MMIO (0x2748 ),0x86543210 }, + {_MMIO (0x2744 ),0x00006667 }, + {_MMIO (0x2740 ),0x00000000 }, + {_MMIO (0x275c ),0x86543210 }, + {_MMIO (0x2758 ),0x86543210 }, + {_MMIO (0x2754 ),0x00006465 }, + {_MMIO (0x2750 ),0x00000000 }, + {_MMIO (0x2770 ),0x0007f81a }, + {_MMIO (0x2774 ),0x0000fe00 }, + {_MMIO (0x2778 ),0x0007f82a }, + {_MMIO (0x277c ),0x0000fe00 }, + {_MMIO (0x2780 ),0x0007f872 }, + {_MMIO (0x2784 ),0x0000fe00 }, + {_MMIO (0x2788 ),0x0007f8ba }, + {_MMIO (0x278c ),0x0000fe00 }, + {_MMIO (0x2790 ),0x0007f87a }, + {_MMIO (0x2794 ),0x0000fe00 }, + {_MMIO (0x2798 ),0x0007f8ea }, + {_MMIO (0x279c ),0x0000fe00 }, + {_MMIO (0x27a0 ),0x0007f8e2 }, + {_MMIO (0x27a4 ),0x0000fe00 }, + {_MMIO (0x27a8 ),0x0007f8f2 }, + {_MMIO (0x27ac ),0x0000fe00 }, +}; + +static const struct i915_oa_reg flex_eu_config_memory_reads[] = { + {_MMIO (0xe458 ),0x00005004 }, + {_MMIO (0xe558 ),0x00015014 }, + {_MMIO (0xe658 ),0x00025024 }, + {_MMIO (0xe758 ),0x00035034 }, + {_MMIO (0xe45c ),0x00045044 }, + {_MMIO (0xe55c ),0x00055054 }, + {_MMIO (0xe65c ),0x00065064 }, +}; + +static const struct i915_oa_reg mux_config_memory_reads[] = { + {_MMIO (0x9888 ),0x198b0343 }, + {_MMIO (0x9888 ),0x13845800 }, + {_MMIO (0x9888 ),0x15840018 }, + {_MMIO (0x9888 ),0x3580001a }, + {_MMIO (0x9888 ),0x038b6300 }, + {_MMIO (0x9888 ),0x058b6b62 }, + {_MMIO (0x9888 ),0x078b006a }, + {_MMIO (0x9888 ),0x118b0000 }, + {_MMIO (0x9888 ),0x238b0000 }, + {_MMIO (0x9888 ),0x258b0000 }, + {_MMIO (0x9888 ),0x1f85a080 }, + {_MMIO (0x9888 ),0x2185aaaa }, + {_MMIO (0x9888 ),0x2385000a }, + {_MMIO (0x9888 ),0x07834000 }, + {_MMIO (0x9888 ),0x09834000 }, + {_MMIO (0x9888 ),0x0b834000 }, + {_MMIO (0x9888 ),0x0d834000 }, + {_MMIO (0x9888 ),0x01840018 }, + {_MMIO (0x9888 ),0x07844c80 }, + {_MMIO (0x9888 ),0x09840d9a }, + {_MMIO (0x9888 ),0x0b840e9c }, + {_MMIO (0x9888 ),0x0d840f9e }, + {_MMIO (0x9888 ),0x0f840010 }, + {_MMIO (0x9888 ),0x11840000 }, + {_MMIO (0x9888 ),0x03848000 }, + {_MMIO (0x9888 ),0x0584c000 }, + {_MMIO (0x9888 ),0x2f8000e5 }, + {_MMIO (0x9888 ),0x138080e3 }, + {_MMIO (0x9888 ),0x1580c0e1 }, + {_MMIO (0x9888 ),0x21800000 }, + {_MMIO (0x9888 ),0x11804000 }, + {_MMIO (0x9888 ),0x1780c000 }, + {_MMIO (0x9888 ),0x1980c000 }, + {_MMIO (0x9888 ),0x1b80c000 }, + {_MMIO (0x9888 ),0x1d80c000 }, + {_MMIO (0x9888 ),0x1f804000 }, + {_MMIO (0xd24 ),0x00000000 }, + {_MMIO (0x9888 ),0x4d800000 }, + {_MMIO (0x9888 ),0x3d800800 }, + {_MMIO (0x9888 ),0x4f800000 }, + {_MMIO (0x9888 ),0x43800842 }, + {_MMIO (0x9888 ),0x51800000 }, + {_MMIO (0x9888 ),0x45800842 }, + {_MMIO (0x9888 ),0x53800000 }, + {_MMIO (0x9888 ),0x47801042 }, + {_MMIO (0x9888 ),0x31800000 }, + {_MMIO (0x9888 ),0x3f800084 }, + {_MMIO (0x9888 ),0x41800000 }, +}; + +static const struct i915_oa_reg * +get_memory_reads_mux_config (struct drm_i915_private *dev_priv, +int *len) +{ + *len =ARRAY_SIZE (mux_config_memory_reads); +return mux_config_memory_reads; +} + +static const struct i915_oa_reg b_counter_config_memory_writes[] = { + {_MMIO (0x2724 ),0xf0800000 }, + {_MMIO (0x2720 ),0x00000000 }, + {_MMIO (0x2714 ),0xf0800000 }, + {_MMIO (0x2710 ),0x00000000 }, + {_MMIO (0x274c ),0x86543210 }, + {_MMIO (0x2748 ),0x86543210 }, + {_MMIO (0x2744 ),0x00006667 }, + {_MMIO (0x2740 ),0x00000000 }, + {_MMIO (0x275c ),0x86543210 }, + {_MMIO (0x2758 ),0x86543210 }, + {_MMIO (0x2754 ),0x00006465 }, + {_MMIO (0x2750 ),0x00000000 }, + {_MMIO (0x2770 ),0x0007f81a }, + {_MMIO (0x2774 ),0x0000fe00 }, + {_MMIO (0x2778 ),0x0007f82a }, + {_MMIO (0x277c ),0x0000fe00 }, + {_MMIO (0x2780 ),0x0007f822 }, + {_MMIO (0x2784 ),0x0000fe00 }, + {_MMIO (0x2788 ),0x0007f8ba }, + {_MMIO (0x278c ),0x0000fe00 }, + {_MMIO (0x2790 ),0x0007f87a }, + {_MMIO (0x2794 ),0x0000fe00 }, + {_MMIO (0x2798 ),0x0007f8ea }, + {_MMIO (0x279c ),0x0000fe00 }, + {_MMIO (0x27a0 ),0x0007f8e2 }, + {_MMIO (0x27a4 ),0x0000fe00 }, + {_MMIO (0x27a8 ),0x0007f8f2 }, + {_MMIO (0x27ac ),0x0000fe00 }, +}; + +static const struct i915_oa_reg flex_eu_config_memory_writes[] = { + {_MMIO (0xe458 ),0x00005004 }, + {_MMIO (0xe558 ),0x00015014 }, + {_MMIO (0xe658 ),0x00025024 }, + {_MMIO (0xe758 ),0x00035034 }, + {_MMIO (0xe45c ),0x00045044 }, + {_MMIO (0xe55c ),0x00055054 }, + {_MMIO (0xe65c ),0x00065064 }, +}; + +static const struct i915_oa_reg mux_config_memory_writes[] = { + {_MMIO (0x9888 ),0x198b0343 }, + {_MMIO (0x9888 ),0x13845400 }, + {_MMIO (0x9888 ),0x3580001a }, + {_MMIO (0x9888 ),0x3d800805 }, + {_MMIO (0x9888 ),0x038b6300 }, + {_MMIO (0x9888 ),0x058b6b62 }, + {_MMIO (0x9888 ),0x078b006a }, + {_MMIO (0x9888 ),0x118b0000 }, + {_MMIO (0x9888 ),0x238b0000 }, + {_MMIO (0x9888 ),0x258b0000 }, + {_MMIO (0x9888 ),0x1f85a080 }, + {_MMIO (0x9888 ),0x2185aaaa }, + {_MMIO (0x9888 ),0x23850002 }, + {_MMIO (0x9888 ),0x07834000 }, + {_MMIO (0x9888 ),0x09834000 }, + {_MMIO (0x9888 ),0x0b834000 }, + {_MMIO (0x9888 ),0x0d834000 }, + {_MMIO (0x9888 ),0x01840010 }, + {_MMIO (0x9888 ),0x07844880 }, + {_MMIO (0x9888 ),0x09840992 }, + {_MMIO (0x9888 ),0x0b840a94 }, + {_MMIO (0x9888 ),0x0d840b96 }, + {_MMIO (0x9888 ),0x11840000 }, + {_MMIO (0x9888 ),0x03848000 }, + {_MMIO (0x9888 ),0x0584c000 }, + {_MMIO (0x9888 ),0x2d800147 }, + {_MMIO (0x9888 ),0x2f8000e5 }, + {_MMIO (0x9888 ),0x138080e3 }, + {_MMIO (0x9888 ),0x1580c0e1 }, + {_MMIO (0x9888 ),0x21800000 }, + {_MMIO (0x9888 ),0x11804000 }, + {_MMIO (0x9888 ),0x1780c000 }, + {_MMIO (0x9888 ),0x1980c000 }, + {_MMIO (0x9888 ),0x1b80c000 }, + {_MMIO (0x9888 ),0x1d80c000 }, + {_MMIO (0x9888 ),0x1f800000 }, + {_MMIO (0xd24 ),0x00000000 }, + {_MMIO (0x9888 ),0x4d800000 }, + {_MMIO (0x9888 ),0x4f800000 }, + {_MMIO (0x9888 ),0x43800842 }, + {_MMIO (0x9888 ),0x51800000 }, + {_MMIO (0x9888 ),0x45800842 }, + {_MMIO (0x9888 ),0x53800000 }, + {_MMIO (0x9888 ),0x47801082 }, + {_MMIO (0x9888 ),0x31800000 }, + {_MMIO (0x9888 ),0x3f800084 }, + {_MMIO (0x9888 ),0x41800000 }, +}; + +static const struct i915_oa_reg * +get_memory_writes_mux_config (struct drm_i915_private *dev_priv, +int *len) +{ + *len =ARRAY_SIZE (mux_config_memory_writes); +return mux_config_memory_writes; +} + +static const struct i915_oa_reg b_counter_config_compute_extended[] = { + {_MMIO (0x2724 ),0xf0800000 }, + {_MMIO (0x2720 ),0x00000000 }, + {_MMIO (0x2714 ),0xf0800000 }, + {_MMIO (0x2710 ),0x00000000 }, + {_MMIO (0x2740 ),0x00000000 }, + {_MMIO (0x2770 ),0x0007fc2a }, + {_MMIO (0x2774 ),0x0000bf00 }, + {_MMIO (0x2778 ),0x0007fc6a }, + {_MMIO (0x277c ),0x0000bf00 }, + {_MMIO (0x2780 ),0x0007fc92 }, + {_MMIO (0x2784 ),0x0000bf00 }, + {_MMIO (0x2788 ),0x0007fca2 }, + {_MMIO (0x278c ),0x0000bf00 }, + {_MMIO (0x2790 ),0x0007fc32 }, + {_MMIO (0x2794 ),0x0000bf00 }, + {_MMIO (0x2798 ),0x0007fc9a }, + {_MMIO (0x279c ),0x0000bf00 }, + {_MMIO (0x27a0 ),0x0007fe6a }, + {_MMIO (0x27a4 ),0x0000bf00 }, + {_MMIO (0x27a8 ),0x0007fe7a }, + {_MMIO (0x27ac ),0x0000bf00 }, +}; + +static const struct i915_oa_reg flex_eu_config_compute_extended[] = { + {_MMIO (0xe458 ),0x00005004 }, + {_MMIO (0xe558 ),0x00000003 }, + {_MMIO (0xe658 ),0x00002001 }, + {_MMIO (0xe758 ),0x00778008 }, + {_MMIO (0xe45c ),0x00088078 }, + {_MMIO (0xe55c ),0x00808708 }, + {_MMIO (0xe65c ),0x00a08908 }, +}; + +static const struct i915_oa_reg mux_config_compute_extended_0_subslices_0x01[] = { + {_MMIO (0x9888 ),0x143d0160 }, + {_MMIO (0x9888 ),0x163d2800 }, + {_MMIO (0x9888 ),0x183d0120 }, + {_MMIO (0x9888 ),0x105800e0 }, + {_MMIO (0x9888 ),0x005cc000 }, + {_MMIO (0x9888 ),0x065c8000 }, + {_MMIO (0x9888 ),0x085cc000 }, + {_MMIO (0x9888 ),0x0a5cc000 }, + {_MMIO (0x9888 ),0x0c5cc000 }, + {_MMIO (0x9888 ),0x0e5cc000 }, + {_MMIO (0x9888 ),0x025cc000 }, + {_MMIO (0x9888 ),0x045cc000 }, + {_MMIO (0x9888 ),0x003d0011 }, + {_MMIO (0x9888 ),0x063d0900 }, + {_MMIO (0x9888 ),0x083d0a13 }, + {_MMIO (0x9888 ),0x0a3d0b15 }, + {_MMIO (0x9888 ),0x0c3d2317 }, + {_MMIO (0x9888 ),0x043d21b7 }, + {_MMIO (0x9888 ),0x103d0000 }, + {_MMIO (0x9888 ),0x0e3d0000 }, + {_MMIO (0x9888 ),0x1a3d0000 }, + {_MMIO (0x9888 ),0x0e5825c1 }, + {_MMIO (0x9888 ),0x00586100 }, + {_MMIO (0x9888 ),0x0258204c }, + {_MMIO (0x9888 ),0x06588000 }, + {_MMIO (0x9888 ),0x0858c000 }, + {_MMIO (0x9888 ),0x0a58c000 }, + {_MMIO (0x9888 ),0x0c58c000 }, + {_MMIO (0x9888 ),0x0458c000 }, + {_MMIO (0x9888 ),0x005b4000 }, + {_MMIO (0x9888 ),0x0e5b4000 }, + {_MMIO (0x9888 ),0x185b5400 }, + {_MMIO (0x9888 ),0x1a5b0155 }, + {_MMIO (0x9888 ),0x025b4000 }, + {_MMIO (0x9888 ),0x045b4000 }, + {_MMIO (0x9888 ),0x065b4000 }, + {_MMIO (0x9888 ),0x085b4000 }, + {_MMIO (0x9888 ),0x0a5b4000 }, + {_MMIO (0x9888 ),0x0c1fa800 }, + {_MMIO (0x9888 ),0x0e1faa2a }, + {_MMIO (0x9888 ),0x101f02aa }, + {_MMIO (0x9888 ),0x00384000 }, + {_MMIO (0x9888 ),0x0e384000 }, + {_MMIO (0x9888 ),0x16384000 }, + {_MMIO (0x9888 ),0x18381555 }, + {_MMIO (0x9888 ),0x02384000 }, + {_MMIO (0x9888 ),0x04384000 }, + {_MMIO (0x9888 ),0x06384000 }, + {_MMIO (0x9888 ),0x08384000 }, + {_MMIO (0x9888 ),0x0a384000 }, + {_MMIO (0x9888 ),0x0039a000 }, + {_MMIO (0x9888 ),0x06398000 }, + {_MMIO (0x9888 ),0x0839a000 }, + {_MMIO (0x9888 ),0x0a39a000 }, + {_MMIO (0x9888 ),0x0c39a000 }, + {_MMIO (0x9888 ),0x0e39a000 }, + {_MMIO (0x9888 ),0x0239a000 }, + {_MMIO (0x9888 ),0x0439a000 }, + {_MMIO (0x9888 ),0x018a8000 }, + {_MMIO (0x9888 ),0x0f8a8000 }, + {_MMIO (0x9888 ),0x198a8000 }, + {_MMIO (0x9888 ),0x1b8aaaa0 }, + {_MMIO (0x9888 ),0x1d8a0002 }, + {_MMIO (0x9888 ),0x038a8000 }, + {_MMIO (0x9888 ),0x058a8000 }, + {_MMIO (0x9888 ),0x078a8000 }, + {_MMIO (0x9888 ),0x098a8000 }, + {_MMIO (0x9888 ),0x0b8a8000 }, + {_MMIO (0x9888 ),0x238b2aa0 }, + {_MMIO (0x9888 ),0x258b5551 }, + {_MMIO (0x9888 ),0x278b0015 }, + {_MMIO (0x9888 ),0x1f85aa80 }, + {_MMIO (0x9888 ),0x2185aaa2 }, + {_MMIO (0x9888 ),0x2385002a }, + {_MMIO (0x9888 ),0x01834000 }, + {_MMIO (0x9888 ),0x0f834000 }, + {_MMIO (0x9888 ),0x19835400 }, + {_MMIO (0x9888 ),0x1b830155 }, + {_MMIO (0x9888 ),0x03834000 }, + {_MMIO (0x9888 ),0x05834000 }, + {_MMIO (0x9888 ),0x07834000 }, + {_MMIO (0x9888 ),0x09834000 }, + {_MMIO (0x9888 ),0x0b834000 }, + {_MMIO (0x9888 ),0x0184c000 }, + {_MMIO (0x9888 ),0x07848000 }, + {_MMIO (0x9888 ),0x0984c000 }, + {_MMIO (0x9888 ),0x0b84c000 }, + {_MMIO (0x9888 ),0x0d84c000 }, + {_MMIO (0x9888 ),0x0f84c000 }, + {_MMIO (0x9888 ),0x0384c000 }, + {_MMIO (0x9888 ),0x0584c000 }, + {_MMIO (0x9888 ),0x1180c000 }, + {_MMIO (0x9888 ),0x17808000 }, + {_MMIO (0x9888 ),0x1980c000 }, + {_MMIO (0x9888 ),0x1b80c000 }, + {_MMIO (0x9888 ),0x1d80c000 }, + {_MMIO (0x9888 ),0x1f80c000 }, + {_MMIO (0x9888 ),0x1380c000 }, + {_MMIO (0x9888 ),0x1580c000 }, + {_MMIO (0xd24 ),0x00000000 }, + {_MMIO (0x9888 ),0x4d800000 }, + {_MMIO (0x9888 ),0x3d800000 }, + {_MMIO (0x9888 ),0x4f800000 }, + {_MMIO (0x9888 ),0x43800000 }, + {_MMIO (0x9888 ),0x51800000 }, + {_MMIO (0x9888 ),0x45800000 }, + {_MMIO (0x9888 ),0x53800000 }, + {_MMIO (0x9888 ),0x47800420 }, + {_MMIO (0x9888 ),0x21800000 }, + {_MMIO (0x9888 ),0x31800000 }, + {_MMIO (0x9888 ),0x3f800421 }, + {_MMIO (0x9888 ),0x41800000 }, +}; + +static const struct i915_oa_reg mux_config_compute_extended_2_subslices_0x02[] = { + {_MMIO (0x9888 ),0x105c00e0 }, + {_MMIO (0x9888 ),0x145b0160 }, + {_MMIO (0x9888 ),0x165b2800 }, + {_MMIO (0x9888 ),0x185b0120 }, + {_MMIO (0x9888 ),0x0e5c25c1 }, + {_MMIO (0x9888 ),0x005c6100 }, + {_MMIO (0x9888 ),0x025c204c }, + {_MMIO (0x9888 ),0x065c8000 }, + {_MMIO (0x9888 ),0x085cc000 }, + {_MMIO (0x9888 ),0x0a5cc000 }, + {_MMIO (0x9888 ),0x0c5cc000 }, + {_MMIO (0x9888 ),0x045cc000 }, + {_MMIO (0x9888 ),0x005b0011 }, + {_MMIO (0x9888 ),0x065b0900 }, + {_MMIO (0x9888 ),0x085b0a13 }, + {_MMIO (0x9888 ),0x0a5b0b15 }, + {_MMIO (0x9888 ),0x0c5b2317 }, + {_MMIO (0x9888 ),0x045b21b7 }, + {_MMIO (0x9888 ),0x105b0000 }, + {_MMIO (0x9888 ),0x0e5b0000 }, + {_MMIO (0x9888 ),0x1a5b0000 }, + {_MMIO (0x9888 ),0x0c1fa800 }, + {_MMIO (0x9888 ),0x0e1faa2a }, + {_MMIO (0x9888 ),0x101f02aa }, + {_MMIO (0x9888 ),0x00384000 }, + {_MMIO (0x9888 ),0x0e384000 }, + {_MMIO (0x9888 ),0x16384000 }, + {_MMIO (0x9888 ),0x18381555 }, + {_MMIO (0x9888 ),0x02384000 }, + {_MMIO (0x9888 ),0x04384000 }, + {_MMIO (0x9888 ),0x06384000 }, + {_MMIO (0x9888 ),0x08384000 }, + {_MMIO (0x9888 ),0x0a384000 }, + {_MMIO (0x9888 ),0x0039a000 }, + {_MMIO (0x9888 ),0x06398000 }, + {_MMIO (0x9888 ),0x0839a000 }, + {_MMIO (0x9888 ),0x0a39a000 }, + {_MMIO (0x9888 ),0x0c39a000 }, + {_MMIO (0x9888 ),0x0e39a000 }, + {_MMIO (0x9888 ),0x0239a000 }, + {_MMIO (0x9888 ),0x0439a000 }, + {_MMIO (0x9888 ),0x018a8000 }, + {_MMIO (0x9888 ),0x0f8a8000 }, + {_MMIO (0x9888 ),0x198a8000 }, + {_MMIO (0x9888 ),0x1b8aaaa0 }, + {_MMIO (0x9888 ),0x1d8a0002 }, + {_MMIO (0x9888 ),0x038a8000 }, + {_MMIO (0x9888 ),0x058a8000 }, + {_MMIO (0x9888 ),0x078a8000 }, + {_MMIO (0x9888 ),0x098a8000 }, + {_MMIO (0x9888 ),0x0b8a8000 }, + {_MMIO (0x9888 ),0x238b2aa0 }, + {_MMIO (0x9888 ),0x258b5551 }, + {_MMIO (0x9888 ),0x278b0015 }, + {_MMIO (0x9888 ),0x1f85aa80 }, + {_MMIO (0x9888 ),0x2185aaa2 }, + {_MMIO (0x9888 ),0x2385002a }, + {_MMIO (0x9888 ),0x01834000 }, + {_MMIO (0x9888 ),0x0f834000 }, + {_MMIO (0x9888 ),0x19835400 }, + {_MMIO (0x9888 ),0x1b830155 }, + {_MMIO (0x9888 ),0x03834000 }, + {_MMIO (0x9888 ),0x05834000 }, + {_MMIO (0x9888 ),0x07834000 }, + {_MMIO (0x9888 ),0x09834000 }, + {_MMIO (0x9888 ),0x0b834000 }, + {_MMIO (0x9888 ),0x0184c000 }, + {_MMIO (0x9888 ),0x07848000 }, + {_MMIO (0x9888 ),0x0984c000 }, + {_MMIO (0x9888 ),0x0b84c000 }, + {_MMIO (0x9888 ),0x0d84c000 }, + {_MMIO (0x9888 ),0x0f84c000 }, + {_MMIO (0x9888 ),0x0384c000 }, + {_MMIO (0x9888 ),0x0584c000 }, + {_MMIO (0x9888 ),0x1180c000 }, + {_MMIO (0x9888 ),0x17808000 }, + {_MMIO (0x9888 ),0x1980c000 }, + {_MMIO (0x9888 ),0x1b80c000 }, + {_MMIO (0x9888 ),0x1d80c000 }, + {_MMIO (0x9888 ),0x1f80c000 }, + {_MMIO (0x9888 ),0x1380c000 }, + {_MMIO (0x9888 ),0x1580c000 }, + {_MMIO (0xd24 ),0x00000000 }, + {_MMIO (0x9888 ),0x4d800000 }, + {_MMIO (0x9888 ),0x3d800000 }, + {_MMIO (0x9888 ),0x4f800000 }, + {_MMIO (0x9888 ),0x43800000 }, + {_MMIO (0x9888 ),0x51800000 }, + {_MMIO (0x9888 ),0x45800000 }, + {_MMIO (0x9888 ),0x53800000 }, + {_MMIO (0x9888 ),0x47800420 }, + {_MMIO (0x9888 ),0x21800000 }, + {_MMIO (0x9888 ),0x31800000 }, + {_MMIO (0x9888 ),0x3f800421 }, + {_MMIO (0x9888 ),0x41800000 }, +}; + +static const struct i915_oa_reg mux_config_compute_extended_4_subslices_0x04[] = { + {_MMIO (0x9888 ),0x103800e0 }, + {_MMIO (0x9888 ),0x143a0160 }, + {_MMIO (0x9888 ),0x163a2800 }, + {_MMIO (0x9888 ),0x183a0120 }, + {_MMIO (0x9888 ),0x0c1fa800 }, + {_MMIO (0x9888 ),0x0e1faa2a }, + {_MMIO (0x9888 ),0x101f02aa }, + {_MMIO (0x9888 ),0x0e38a5c1 }, + {_MMIO (0x9888 ),0x0038a100 }, + {_MMIO (0x9888 ),0x0238204c }, + {_MMIO (0x9888 ),0x16388000 }, + {_MMIO (0x9888 ),0x183802aa }, + {_MMIO (0x9888 ),0x04380000 }, + {_MMIO (0x9888 ),0x06380000 }, + {_MMIO (0x9888 ),0x08388000 }, + {_MMIO (0x9888 ),0x0a388000 }, + {_MMIO (0x9888 ),0x0039a000 }, + {_MMIO (0x9888 ),0x06398000 }, + {_MMIO (0x9888 ),0x0839a000 }, + {_MMIO (0x9888 ),0x0a39a000 }, + {_MMIO (0x9888 ),0x0c39a000 }, + {_MMIO (0x9888 ),0x0e39a000 }, + {_MMIO (0x9888 ),0x0239a000 }, + {_MMIO (0x9888 ),0x0439a000 }, + {_MMIO (0x9888 ),0x003a0011 }, + {_MMIO (0x9888 ),0x063a0900 }, + {_MMIO (0x9888 ),0x083a0a13 }, + {_MMIO (0x9888 ),0x0a3a0b15 }, + {_MMIO (0x9888 ),0x0c3a2317 }, + {_MMIO (0x9888 ),0x043a21b7 }, + {_MMIO (0x9888 ),0x103a0000 }, + {_MMIO (0x9888 ),0x0e3a0000 }, + {_MMIO (0x9888 ),0x1a3a0000 }, + {_MMIO (0x9888 ),0x018a8000 }, + {_MMIO (0x9888 ),0x0f8a8000 }, + {_MMIO (0x9888 ),0x198a8000 }, + {_MMIO (0x9888 ),0x1b8aaaa0 }, + {_MMIO (0x9888 ),0x1d8a0002 }, + {_MMIO (0x9888 ),0x038a8000 }, + {_MMIO (0x9888 ),0x058a8000 }, + {_MMIO (0x9888 ),0x078a8000 }, + {_MMIO (0x9888 ),0x098a8000 }, + {_MMIO (0x9888 ),0x0b8a8000 }, + {_MMIO (0x9888 ),0x238b2aa0 }, + {_MMIO (0x9888 ),0x258b5551 }, + {_MMIO (0x9888 ),0x278b0015 }, + {_MMIO (0x9888 ),0x1f85aa80 }, + {_MMIO (0x9888 ),0x2185aaa2 }, + {_MMIO (0x9888 ),0x2385002a }, + {_MMIO (0x9888 ),0x01834000 }, + {_MMIO (0x9888 ),0x0f834000 }, + {_MMIO (0x9888 ),0x19835400 }, + {_MMIO (0x9888 ),0x1b830155 }, + {_MMIO (0x9888 ),0x03834000 }, + {_MMIO (0x9888 ),0x05834000 }, + {_MMIO (0x9888 ),0x07834000 }, + {_MMIO (0x9888 ),0x09834000 }, + {_MMIO (0x9888 ),0x0b834000 }, + {_MMIO (0x9888 ),0x0184c000 }, + {_MMIO (0x9888 ),0x07848000 }, + {_MMIO (0x9888 ),0x0984c000 }, + {_MMIO (0x9888 ),0x0b84c000 }, + {_MMIO (0x9888 ),0x0d84c000 }, + {_MMIO (0x9888 ),0x0f84c000 }, + {_MMIO (0x9888 ),0x0384c000 }, + {_MMIO (0x9888 ),0x0584c000 }, + {_MMIO (0x9888 ),0x1180c000 }, + {_MMIO (0x9888 ),0x17808000 }, + {_MMIO (0x9888 ),0x1980c000 }, + {_MMIO (0x9888 ),0x1b80c000 }, + {_MMIO (0x9888 ),0x1d80c000 }, + {_MMIO (0x9888 ),0x1f80c000 }, + {_MMIO (0x9888 ),0x1380c000 }, + {_MMIO (0x9888 ),0x1580c000 }, + {_MMIO (0xd24 ),0x00000000 }, + {_MMIO (0x9888 ),0x4d800000 }, + {_MMIO (0x9888 ),0x3d800000 }, + {_MMIO (0x9888 ),0x4f800000 }, + {_MMIO (0x9888 ),0x43800000 }, + {_MMIO (0x9888 ),0x51800000 }, + {_MMIO (0x9888 ),0x45800000 }, + {_MMIO (0x9888 ),0x53800000 }, + {_MMIO (0x9888 ),0x47800420 }, + {_MMIO (0x9888 ),0x21800000 }, + {_MMIO (0x9888 ),0x31800000 }, + {_MMIO (0x9888 ),0x3f800421 }, + {_MMIO (0x9888 ),0x41800000 }, +}; + +static const struct i915_oa_reg mux_config_compute_extended_1_subslices_0x08[] = { + {_MMIO (0x9888 ),0x14bd0160 }, + {_MMIO (0x9888 ),0x16bd2800 }, + {_MMIO (0x9888 ),0x18bd0120 }, + {_MMIO (0x9888 ),0x10d800e0 }, + {_MMIO (0x9888 ),0x00dcc000 }, + {_MMIO (0x9888 ),0x06dc8000 }, + {_MMIO (0x9888 ),0x08dcc000 }, + {_MMIO (0x9888 ),0x0adcc000 }, + {_MMIO (0x9888 ),0x0cdcc000 }, + {_MMIO (0x9888 ),0x0edcc000 }, + {_MMIO (0x9888 ),0x02dcc000 }, + {_MMIO (0x9888 ),0x04dcc000 }, + {_MMIO (0x9888 ),0x00bd0011 }, + {_MMIO (0x9888 ),0x06bd0900 }, + {_MMIO (0x9888 ),0x08bd0a13 }, + {_MMIO (0x9888 ),0x0abd0b15 }, + {_MMIO (0x9888 ),0x0cbd2317 }, + {_MMIO (0x9888 ),0x04bd21b7 }, + {_MMIO (0x9888 ),0x10bd0000 }, + {_MMIO (0x9888 ),0x0ebd0000 }, + {_MMIO (0x9888 ),0x1abd0000 }, + {_MMIO (0x9888 ),0x0ed825c1 }, + {_MMIO (0x9888 ),0x00d86100 }, + {_MMIO (0x9888 ),0x02d8204c }, + {_MMIO (0x9888 ),0x06d88000 }, + {_MMIO (0x9888 ),0x08d8c000 }, + {_MMIO (0x9888 ),0x0ad8c000 }, + {_MMIO (0x9888 ),0x0cd8c000 }, + {_MMIO (0x9888 ),0x04d8c000 }, + {_MMIO (0x9888 ),0x00db4000 }, + {_MMIO (0x9888 ),0x0edb4000 }, + {_MMIO (0x9888 ),0x18db5400 }, + {_MMIO (0x9888 ),0x1adb0155 }, + {_MMIO (0x9888 ),0x02db4000 }, + {_MMIO (0x9888 ),0x04db4000 }, + {_MMIO (0x9888 ),0x06db4000 }, + {_MMIO (0x9888 ),0x08db4000 }, + {_MMIO (0x9888 ),0x0adb4000 }, + {_MMIO (0x9888 ),0x0c9fa800 }, + {_MMIO (0x9888 ),0x0e9faa2a }, + {_MMIO (0x9888 ),0x109f02aa }, + {_MMIO (0x9888 ),0x00b84000 }, + {_MMIO (0x9888 ),0x0eb84000 }, + {_MMIO (0x9888 ),0x16b84000 }, + {_MMIO (0x9888 ),0x18b81555 }, + {_MMIO (0x9888 ),0x02b84000 }, + {_MMIO (0x9888 ),0x04b84000 }, + {_MMIO (0x9888 ),0x06b84000 }, + {_MMIO (0x9888 ),0x08b84000 }, + {_MMIO (0x9888 ),0x0ab84000 }, + {_MMIO (0x9888 ),0x00b9a000 }, + {_MMIO (0x9888 ),0x06b98000 }, + {_MMIO (0x9888 ),0x08b9a000 }, + {_MMIO (0x9888 ),0x0ab9a000 }, + {_MMIO (0x9888 ),0x0cb9a000 }, + {_MMIO (0x9888 ),0x0eb9a000 }, + {_MMIO (0x9888 ),0x02b9a000 }, + {_MMIO (0x9888 ),0x04b9a000 }, + {_MMIO (0x9888 ),0x01888000 }, + {_MMIO (0x9888 ),0x0d88f800 }, + {_MMIO (0x9888 ),0x0f88000f }, + {_MMIO (0x9888 ),0x03888000 }, + {_MMIO (0x9888 ),0x05888000 }, + {_MMIO (0x9888 ),0x07888000 }, + {_MMIO (0x9888 ),0x09888000 }, + {_MMIO (0x9888 ),0x0b888000 }, + {_MMIO (0x9888 ),0x238b5540 }, + {_MMIO (0x9888 ),0x258baaa2 }, + {_MMIO (0x9888 ),0x278b002a }, + {_MMIO (0x9888 ),0x018c4000 }, + {_MMIO (0x9888 ),0x0f8c4000 }, + {_MMIO (0x9888 ),0x178c2000 }, + {_MMIO (0x9888 ),0x198c5500 }, + {_MMIO (0x9888 ),0x1b8c0015 }, + {_MMIO (0x9888 ),0x038c4000 }, + {_MMIO (0x9888 ),0x058c4000 }, + {_MMIO (0x9888 ),0x078c4000 }, + {_MMIO (0x9888 ),0x098c4000 }, + {_MMIO (0x9888 ),0x0b8c4000 }, + {_MMIO (0x9888 ),0x018da000 }, + {_MMIO (0x9888 ),0x078d8000 }, + {_MMIO (0x9888 ),0x098da000 }, + {_MMIO (0x9888 ),0x0b8da000 }, + {_MMIO (0x9888 ),0x0d8da000 }, + {_MMIO (0x9888 ),0x0f8da000 }, + {_MMIO (0x9888 ),0x038da000 }, + {_MMIO (0x9888 ),0x058da000 }, + {_MMIO (0x9888 ),0x1f85aa80 }, + {_MMIO (0x9888 ),0x2185aaa2 }, + {_MMIO (0x9888 ),0x2385002a }, + {_MMIO (0x9888 ),0x01834000 }, + {_MMIO (0x9888 ),0x0f834000 }, + {_MMIO (0x9888 ),0x19835400 }, + {_MMIO (0x9888 ),0x1b830155 }, + {_MMIO (0x9888 ),0x03834000 }, + {_MMIO (0x9888 ),0x05834000 }, + {_MMIO (0x9888 ),0x07834000 }, + {_MMIO (0x9888 ),0x09834000 }, + {_MMIO (0x9888 ),0x0b834000 }, + {_MMIO (0x9888 ),0x0184c000 }, + {_MMIO (0x9888 ),0x07848000 }, + {_MMIO (0x9888 ),0x0984c000 }, + {_MMIO (0x9888 ),0x0b84c000 }, + {_MMIO (0x9888 ),0x0d84c000 }, + {_MMIO (0x9888 ),0x0f84c000 }, + {_MMIO (0x9888 ),0x0384c000 }, + {_MMIO (0x9888 ),0x0584c000 }, + {_MMIO (0x9888 ),0x1180c000 }, + {_MMIO (0x9888 ),0x17808000 }, + {_MMIO (0x9888 ),0x1980c000 }, + {_MMIO (0x9888 ),0x1b80c000 }, + {_MMIO (0x9888 ),0x1d80c000 }, + {_MMIO (0x9888 ),0x1f80c000 }, + {_MMIO (0x9888 ),0x1380c000 }, + {_MMIO (0x9888 ),0x1580c000 }, + {_MMIO (0xd24 ),0x00000000 }, + {_MMIO (0x9888 ),0x4d800000 }, + {_MMIO (0x9888 ),0x3d800000 }, + {_MMIO (0x9888 ),0x4f800000 }, + {_MMIO (0x9888 ),0x43800000 }, + {_MMIO (0x9888 ),0x51800000 }, + {_MMIO (0x9888 ),0x45800000 }, + {_MMIO (0x9888 ),0x53800000 }, + {_MMIO (0x9888 ),0x47800420 }, + {_MMIO (0x9888 ),0x21800000 }, + {_MMIO (0x9888 ),0x31800000 }, + {_MMIO (0x9888 ),0x3f800421 }, + {_MMIO (0x9888 ),0x41800000 }, +}; + +static const struct i915_oa_reg mux_config_compute_extended_3_subslices_0x10[] = { + {_MMIO (0x9888 ),0x10dc00e0 }, + {_MMIO (0x9888 ),0x14db0160 }, + {_MMIO (0x9888 ),0x16db2800 }, + {_MMIO (0x9888 ),0x18db0120 }, + {_MMIO (0x9888 ),0x0edc25c1 }, + {_MMIO (0x9888 ),0x00dc6100 }, + {_MMIO (0x9888 ),0x02dc204c }, + {_MMIO (0x9888 ),0x06dc8000 }, + {_MMIO (0x9888 ),0x08dcc000 }, + {_MMIO (0x9888 ),0x0adcc000 }, + {_MMIO (0x9888 ),0x0cdcc000 }, + {_MMIO (0x9888 ),0x04dcc000 }, + {_MMIO (0x9888 ),0x00db0011 }, + {_MMIO (0x9888 ),0x06db0900 }, + {_MMIO (0x9888 ),0x08db0a13 }, + {_MMIO (0x9888 ),0x0adb0b15 }, + {_MMIO (0x9888 ),0x0cdb2317 }, + {_MMIO (0x9888 ),0x04db21b7 }, + {_MMIO (0x9888 ),0x10db0000 }, + {_MMIO (0x9888 ),0x0edb0000 }, + {_MMIO (0x9888 ),0x1adb0000 }, + {_MMIO (0x9888 ),0x0c9fa800 }, + {_MMIO (0x9888 ),0x0e9faa2a }, + {_MMIO (0x9888 ),0x109f02aa }, + {_MMIO (0x9888 ),0x00b84000 }, + {_MMIO (0x9888 ),0x0eb84000 }, + {_MMIO (0x9888 ),0x16b84000 }, + {_MMIO (0x9888 ),0x18b81555 }, + {_MMIO (0x9888 ),0x02b84000 }, + {_MMIO (0x9888 ),0x04b84000 }, + {_MMIO (0x9888 ),0x06b84000 }, + {_MMIO (0x9888 ),0x08b84000 }, + {_MMIO (0x9888 ),0x0ab84000 }, + {_MMIO (0x9888 ),0x00b9a000 }, + {_MMIO (0x9888 ),0x06b98000 }, + {_MMIO (0x9888 ),0x08b9a000 }, + {_MMIO (0x9888 ),0x0ab9a000 }, + {_MMIO (0x9888 ),0x0cb9a000 }, + {_MMIO (0x9888 ),0x0eb9a000 }, + {_MMIO (0x9888 ),0x02b9a000 }, + {_MMIO (0x9888 ),0x04b9a000 }, + {_MMIO (0x9888 ),0x01888000 }, + {_MMIO (0x9888 ),0x0d88f800 }, + {_MMIO (0x9888 ),0x0f88000f }, + {_MMIO (0x9888 ),0x03888000 }, + {_MMIO (0x9888 ),0x05888000 }, + {_MMIO (0x9888 ),0x07888000 }, + {_MMIO (0x9888 ),0x09888000 }, + {_MMIO (0x9888 ),0x0b888000 }, + {_MMIO (0x9888 ),0x238b5540 }, + {_MMIO (0x9888 ),0x258baaa2 }, + {_MMIO (0x9888 ),0x278b002a }, + {_MMIO (0x9888 ),0x018c4000 }, + {_MMIO (0x9888 ),0x0f8c4000 }, + {_MMIO (0x9888 ),0x178c2000 }, + {_MMIO (0x9888 ),0x198c5500 }, + {_MMIO (0x9888 ),0x1b8c0015 }, + {_MMIO (0x9888 ),0x038c4000 }, + {_MMIO (0x9888 ),0x058c4000 }, + {_MMIO (0x9888 ),0x078c4000 }, + {_MMIO (0x9888 ),0x098c4000 }, + {_MMIO (0x9888 ),0x0b8c4000 }, + {_MMIO (0x9888 ),0x018da000 }, + {_MMIO (0x9888 ),0x078d8000 }, + {_MMIO (0x9888 ),0x098da000 }, + {_MMIO (0x9888 ),0x0b8da000 }, + {_MMIO (0x9888 ),0x0d8da000 }, + {_MMIO (0x9888 ),0x0f8da000 }, + {_MMIO (0x9888 ),0x038da000 }, + {_MMIO (0x9888 ),0x058da000 }, + {_MMIO (0x9888 ),0x1f85aa80 }, + {_MMIO (0x9888 ),0x2185aaa2 }, + {_MMIO (0x9888 ),0x2385002a }, + {_MMIO (0x9888 ),0x01834000 }, + {_MMIO (0x9888 ),0x0f834000 }, + {_MMIO (0x9888 ),0x19835400 }, + {_MMIO (0x9888 ),0x1b830155 }, + {_MMIO (0x9888 ),0x03834000 }, + {_MMIO (0x9888 ),0x05834000 }, + {_MMIO (0x9888 ),0x07834000 }, + {_MMIO (0x9888 ),0x09834000 }, + {_MMIO (0x9888 ),0x0b834000 }, + {_MMIO (0x9888 ),0x0184c000 }, + {_MMIO (0x9888 ),0x07848000 }, + {_MMIO (0x9888 ),0x0984c000 }, + {_MMIO (0x9888 ),0x0b84c000 }, + {_MMIO (0x9888 ),0x0d84c000 }, + {_MMIO (0x9888 ),0x0f84c000 }, + {_MMIO (0x9888 ),0x0384c000 }, + {_MMIO (0x9888 ),0x0584c000 }, + {_MMIO (0x9888 ),0x1180c000 }, + {_MMIO (0x9888 ),0x17808000 }, + {_MMIO (0x9888 ),0x1980c000 }, + {_MMIO (0x9888 ),0x1b80c000 }, + {_MMIO (0x9888 ),0x1d80c000 }, + {_MMIO (0x9888 ),0x1f80c000 }, + {_MMIO (0x9888 ),0x1380c000 }, + {_MMIO (0x9888 ),0x1580c000 }, + {_MMIO (0xd24 ),0x00000000 }, + {_MMIO (0x9888 ),0x4d800000 }, + {_MMIO (0x9888 ),0x3d800000 }, + {_MMIO (0x9888 ),0x4f800000 }, + {_MMIO (0x9888 ),0x43800000 }, + {_MMIO (0x9888 ),0x51800000 }, + {_MMIO (0x9888 ),0x45800000 }, + {_MMIO (0x9888 ),0x53800000 }, + {_MMIO (0x9888 ),0x47800420 }, + {_MMIO (0x9888 ),0x21800000 }, + {_MMIO (0x9888 ),0x31800000 }, + {_MMIO (0x9888 ),0x3f800421 }, + {_MMIO (0x9888 ),0x41800000 }, +}; + +static const struct i915_oa_reg mux_config_compute_extended_5_subslices_0x20[] = { + {_MMIO (0x9888 ),0x10b800e0 }, + {_MMIO (0x9888 ),0x14ba0160 }, + {_MMIO (0x9888 ),0x16ba2800 }, + {_MMIO (0x9888 ),0x18ba0120 }, + {_MMIO (0x9888 ),0x0c9fa800 }, + {_MMIO (0x9888 ),0x0e9faa2a }, + {_MMIO (0x9888 ),0x109f02aa }, + {_MMIO (0x9888 ),0x0eb8a5c1 }, + {_MMIO (0x9888 ),0x00b8a100 }, + {_MMIO (0x9888 ),0x02b8204c }, + {_MMIO (0x9888 ),0x16b88000 }, + {_MMIO (0x9888 ),0x18b802aa }, + {_MMIO (0x9888 ),0x04b80000 }, + {_MMIO (0x9888 ),0x06b80000 }, + {_MMIO (0x9888 ),0x08b88000 }, + {_MMIO (0x9888 ),0x0ab88000 }, + {_MMIO (0x9888 ),0x00b9a000 }, + {_MMIO (0x9888 ),0x06b98000 }, + {_MMIO (0x9888 ),0x08b9a000 }, + {_MMIO (0x9888 ),0x0ab9a000 }, + {_MMIO (0x9888 ),0x0cb9a000 }, + {_MMIO (0x9888 ),0x0eb9a000 }, + {_MMIO (0x9888 ),0x02b9a000 }, + {_MMIO (0x9888 ),0x04b9a000 }, + {_MMIO (0x9888 ),0x00ba0011 }, + {_MMIO (0x9888 ),0x06ba0900 }, + {_MMIO (0x9888 ),0x08ba0a13 }, + {_MMIO (0x9888 ),0x0aba0b15 }, + {_MMIO (0x9888 ),0x0cba2317 }, + {_MMIO (0x9888 ),0x04ba21b7 }, + {_MMIO (0x9888 ),0x10ba0000 }, + {_MMIO (0x9888 ),0x0eba0000 }, + {_MMIO (0x9888 ),0x1aba0000 }, + {_MMIO (0x9888 ),0x01888000 }, + {_MMIO (0x9888 ),0x0d88f800 }, + {_MMIO (0x9888 ),0x0f88000f }, + {_MMIO (0x9888 ),0x03888000 }, + {_MMIO (0x9888 ),0x05888000 }, + {_MMIO (0x9888 ),0x07888000 }, + {_MMIO (0x9888 ),0x09888000 }, + {_MMIO (0x9888 ),0x0b888000 }, + {_MMIO (0x9888 ),0x238b5540 }, + {_MMIO (0x9888 ),0x258baaa2 }, + {_MMIO (0x9888 ),0x278b002a }, + {_MMIO (0x9888 ),0x018c4000 }, + {_MMIO (0x9888 ),0x0f8c4000 }, + {_MMIO (0x9888 ),0x178c2000 }, + {_MMIO (0x9888 ),0x198c5500 }, + {_MMIO (0x9888 ),0x1b8c0015 }, + {_MMIO (0x9888 ),0x038c4000 }, + {_MMIO (0x9888 ),0x058c4000 }, + {_MMIO (0x9888 ),0x078c4000 }, + {_MMIO (0x9888 ),0x098c4000 }, + {_MMIO (0x9888 ),0x0b8c4000 }, + {_MMIO (0x9888 ),0x018da000 }, + {_MMIO (0x9888 ),0x078d8000 }, + {_MMIO (0x9888 ),0x098da000 }, + {_MMIO (0x9888 ),0x0b8da000 }, + {_MMIO (0x9888 ),0x0d8da000 }, + {_MMIO (0x9888 ),0x0f8da000 }, + {_MMIO (0x9888 ),0x038da000 }, + {_MMIO (0x9888 ),0x058da000 }, + {_MMIO (0x9888 ),0x1f85aa80 }, + {_MMIO (0x9888 ),0x2185aaa2 }, + {_MMIO (0x9888 ),0x2385002a }, + {_MMIO (0x9888 ),0x01834000 }, + {_MMIO (0x9888 ),0x0f834000 }, + {_MMIO (0x9888 ),0x19835400 }, + {_MMIO (0x9888 ),0x1b830155 }, + {_MMIO (0x9888 ),0x03834000 }, + {_MMIO (0x9888 ),0x05834000 }, + {_MMIO (0x9888 ),0x07834000 }, + {_MMIO (0x9888 ),0x09834000 }, + {_MMIO (0x9888 ),0x0b834000 }, + {_MMIO (0x9888 ),0x0184c000 }, + {_MMIO (0x9888 ),0x07848000 }, + {_MMIO (0x9888 ),0x0984c000 }, + {_MMIO (0x9888 ),0x0b84c000 }, + {_MMIO (0x9888 ),0x0d84c000 }, + {_MMIO (0x9888 ),0x0f84c000 }, + {_MMIO (0x9888 ),0x0384c000 }, + {_MMIO (0x9888 ),0x0584c000 }, + {_MMIO (0x9888 ),0x1180c000 }, + {_MMIO (0x9888 ),0x17808000 }, + {_MMIO (0x9888 ),0x1980c000 }, + {_MMIO (0x9888 ),0x1b80c000 }, + {_MMIO (0x9888 ),0x1d80c000 }, + {_MMIO (0x9888 ),0x1f80c000 }, + {_MMIO (0x9888 ),0x1380c000 }, + {_MMIO (0x9888 ),0x1580c000 }, + {_MMIO (0xd24 ),0x00000000 }, + {_MMIO (0x9888 ),0x4d800000 }, + {_MMIO (0x9888 ),0x3d800000 }, + {_MMIO (0x9888 ),0x4f800000 }, + {_MMIO (0x9888 ),0x43800000 }, + {_MMIO (0x9888 ),0x51800000 }, + {_MMIO (0x9888 ),0x45800000 }, + {_MMIO (0x9888 ),0x53800000 }, + {_MMIO (0x9888 ),0x47800420 }, + {_MMIO (0x9888 ),0x21800000 }, + {_MMIO (0x9888 ),0x31800000 }, + {_MMIO (0x9888 ),0x3f800421 }, + {_MMIO (0x9888 ),0x41800000 }, +}; + +static const struct i915_oa_reg * +get_compute_extended_mux_config (struct drm_i915_private *dev_priv, +int *len) +{ +if (INTEL_INFO (dev_priv)->sseu.subslice_mask &0x01 ) { + *len =ARRAY_SIZE (mux_config_compute_extended_0_subslices_0x01); +return mux_config_compute_extended_0_subslices_0x01; + }else if (INTEL_INFO (dev_priv)->sseu.subslice_mask &0x02 ) { + *len =ARRAY_SIZE (mux_config_compute_extended_2_subslices_0x02); +return mux_config_compute_extended_2_subslices_0x02; + }else if (INTEL_INFO (dev_priv)->sseu.subslice_mask &0x04 ) { + *len =ARRAY_SIZE (mux_config_compute_extended_4_subslices_0x04); +return mux_config_compute_extended_4_subslices_0x04; + }else if (INTEL_INFO (dev_priv)->sseu.subslice_mask &0x08 ) { + *len =ARRAY_SIZE (mux_config_compute_extended_1_subslices_0x08); +return mux_config_compute_extended_1_subslices_0x08; + }else if (INTEL_INFO (dev_priv)->sseu.subslice_mask &0x10 ) { + *len =ARRAY_SIZE (mux_config_compute_extended_3_subslices_0x10); +return mux_config_compute_extended_3_subslices_0x10; + }else if (INTEL_INFO (dev_priv)->sseu.subslice_mask &0x20 ) { + *len =ARRAY_SIZE (mux_config_compute_extended_5_subslices_0x20); +return mux_config_compute_extended_5_subslices_0x20; + } + + *len =0 ; +return NULL ; +} + +static const struct i915_oa_reg b_counter_config_compute_l3_cache[] = { + {_MMIO (0x2710 ),0x00000000 }, + {_MMIO (0x2714 ),0x30800000 }, + {_MMIO (0x2720 ),0x00000000 }, + {_MMIO (0x2724 ),0x30800000 }, + {_MMIO (0x2740 ),0x00000000 }, + {_MMIO (0x2770 ),0x0007fffa }, + {_MMIO (0x2774 ),0x0000fefe }, + {_MMIO (0x2778 ),0x0007fffa }, + {_MMIO (0x277c ),0x0000fefd }, + {_MMIO (0x2790 ),0x0007fffa }, + {_MMIO (0x2794 ),0x0000fbef }, + {_MMIO (0x2798 ),0x0007fffa }, + {_MMIO (0x279c ),0x0000fbdf }, +}; + +static const struct i915_oa_reg flex_eu_config_compute_l3_cache[] = { + {_MMIO (0xe458 ),0x00005004 }, + {_MMIO (0xe558 ),0x00000003 }, + {_MMIO (0xe658 ),0x00002001 }, + {_MMIO (0xe758 ),0x00101100 }, + {_MMIO (0xe45c ),0x00201200 }, + {_MMIO (0xe55c ),0x00301300 }, + {_MMIO (0xe65c ),0x00401400 }, +}; + +static const struct i915_oa_reg mux_config_compute_l3_cache[] = { + {_MMIO (0x9888 ),0x143f00b3 }, + {_MMIO (0x9888 ),0x14bf00b3 }, + {_MMIO (0x9888 ),0x138303c0 }, + {_MMIO (0x9888 ),0x3b800060 }, + {_MMIO (0x9888 ),0x3d800805 }, + {_MMIO (0x9888 ),0x003f0029 }, + {_MMIO (0x9888 ),0x063f1400 }, + {_MMIO (0x9888 ),0x083f1225 }, + {_MMIO (0x9888 ),0x0e3f1327 }, + {_MMIO (0x9888 ),0x103f0000 }, + {_MMIO (0x9888 ),0x005a4000 }, + {_MMIO (0x9888 ),0x065a8000 }, + {_MMIO (0x9888 ),0x085ac000 }, + {_MMIO (0x9888 ),0x0e5ac000 }, + {_MMIO (0x9888 ),0x001d4000 }, + {_MMIO (0x9888 ),0x061d8000 }, + {_MMIO (0x9888 ),0x081dc000 }, + {_MMIO (0x9888 ),0x0e1dc000 }, + {_MMIO (0x9888 ),0x0c1f0800 }, + {_MMIO (0x9888 ),0x0e1f2a00 }, + {_MMIO (0x9888 ),0x101f0280 }, + {_MMIO (0x9888 ),0x00391000 }, + {_MMIO (0x9888 ),0x06394000 }, + {_MMIO (0x9888 ),0x08395000 }, + {_MMIO (0x9888 ),0x0e395000 }, + {_MMIO (0x9888 ),0x0abf1429 }, + {_MMIO (0x9888 ),0x0cbf1225 }, + {_MMIO (0x9888 ),0x00bf1380 }, + {_MMIO (0x9888 ),0x02bf0026 }, + {_MMIO (0x9888 ),0x10bf0000 }, + {_MMIO (0x9888 ),0x0adac000 }, + {_MMIO (0x9888 ),0x0cdac000 }, + {_MMIO (0x9888 ),0x00da8000 }, + {_MMIO (0x9888 ),0x02da4000 }, + {_MMIO (0x9888 ),0x0a9dc000 }, + {_MMIO (0x9888 ),0x0c9dc000 }, + {_MMIO (0x9888 ),0x009d8000 }, + {_MMIO (0x9888 ),0x029d4000 }, + {_MMIO (0x9888 ),0x0e9f8000 }, + {_MMIO (0x9888 ),0x109f002a }, + {_MMIO (0x9888 ),0x0c9fa000 }, + {_MMIO (0x9888 ),0x0ab95000 }, + {_MMIO (0x9888 ),0x0cb95000 }, + {_MMIO (0x9888 ),0x00b94000 }, + {_MMIO (0x9888 ),0x02b91000 }, + {_MMIO (0x9888 ),0x0d88c000 }, + {_MMIO (0x9888 ),0x0f880003 }, + {_MMIO (0x9888 ),0x03888000 }, + {_MMIO (0x9888 ),0x05888000 }, + {_MMIO (0x9888 ),0x018a8000 }, + {_MMIO (0x9888 ),0x0f8a8000 }, + {_MMIO (0x9888 ),0x198a8000 }, + {_MMIO (0x9888 ),0x1b8a8020 }, + {_MMIO (0x9888 ),0x1d8a0002 }, + {_MMIO (0x9888 ),0x238b0520 }, + {_MMIO (0x9888 ),0x258ba950 }, + {_MMIO (0x9888 ),0x278b0016 }, + {_MMIO (0x9888 ),0x198c5400 }, + {_MMIO (0x9888 ),0x1b8c0001 }, + {_MMIO (0x9888 ),0x038c4000 }, + {_MMIO (0x9888 ),0x058c4000 }, + {_MMIO (0x9888 ),0x0b8da000 }, + {_MMIO (0x9888 ),0x0d8da000 }, + {_MMIO (0x9888 ),0x018d8000 }, + {_MMIO (0x9888 ),0x038d2000 }, + {_MMIO (0x9888 ),0x1f85aa80 }, + {_MMIO (0x9888 ),0x2185aaa0 }, + {_MMIO (0x9888 ),0x2385002a }, + {_MMIO (0x9888 ),0x03835180 }, + {_MMIO (0x9888 ),0x05834022 }, + {_MMIO (0x9888 ),0x11830000 }, + {_MMIO (0x9888 ),0x01834000 }, + {_MMIO (0x9888 ),0x0f834000 }, + {_MMIO (0x9888 ),0x19835400 }, + {_MMIO (0x9888 ),0x1b830155 }, + {_MMIO (0x9888 ),0x07830000 }, + {_MMIO (0x9888 ),0x09830000 }, + {_MMIO (0x9888 ),0x0184c000 }, + {_MMIO (0x9888 ),0x07848000 }, + {_MMIO (0x9888 ),0x0984c000 }, + {_MMIO (0x9888 ),0x0b84c000 }, + {_MMIO (0x9888 ),0x0d84c000 }, + {_MMIO (0x9888 ),0x0f84c000 }, + {_MMIO (0x9888 ),0x0384c000 }, + {_MMIO (0x9888 ),0x05844000 }, + {_MMIO (0x9888 ),0x1b80c137 }, + {_MMIO (0x9888 ),0x1d80c147 }, + {_MMIO (0x9888 ),0x21800000 }, + {_MMIO (0x9888 ),0x1180c000 }, + {_MMIO (0x9888 ),0x17808000 }, + {_MMIO (0x9888 ),0x1980c000 }, + {_MMIO (0x9888 ),0x1f80c000 }, + {_MMIO (0x9888 ),0x1380c000 }, + {_MMIO (0x9888 ),0x15804000 }, + {_MMIO (0xd24 ),0x00000000 }, + {_MMIO (0x9888 ),0x4d801000 }, + {_MMIO (0x9888 ),0x4f800111 }, + {_MMIO (0x9888 ),0x43800842 }, + {_MMIO (0x9888 ),0x51800000 }, + {_MMIO (0x9888 ),0x45800000 }, + {_MMIO (0x9888 ),0x53800000 }, + {_MMIO (0x9888 ),0x47800840 }, + {_MMIO (0x9888 ),0x31800000 }, + {_MMIO (0x9888 ),0x3f800800 }, + {_MMIO (0x9888 ),0x418014a2 }, +}; + +static const struct i915_oa_reg * +get_compute_l3_cache_mux_config (struct drm_i915_private *dev_priv, +int *len) +{ + *len =ARRAY_SIZE (mux_config_compute_l3_cache); +return mux_config_compute_l3_cache; +} + +static const struct i915_oa_reg b_counter_config_data_port_reads_coalescing[] = { + {_MMIO (0x2724 ),0xf0800000 }, + {_MMIO (0x2720 ),0x00000000 }, + {_MMIO (0x2714 ),0xf0800000 }, + {_MMIO (0x2710 ),0x00000000 }, + {_MMIO (0x274c ),0xba98ba98 }, + {_MMIO (0x2748 ),0xba98ba98 }, + {_MMIO (0x2744 ),0x00003377 }, + {_MMIO (0x2740 ),0x00000000 }, + {_MMIO (0x2770 ),0x0007fff2 }, + {_MMIO (0x2774 ),0x00007ff0 }, + {_MMIO (0x2778 ),0x0007ffe2 }, + {_MMIO (0x277c ),0x00007ff0 }, + {_MMIO (0x2780 ),0x0007ffc2 }, + {_MMIO (0x2784 ),0x00007ff0 }, + {_MMIO (0x2788 ),0x0007ff82 }, + {_MMIO (0x278c ),0x00007ff0 }, + {_MMIO (0x2790 ),0x0007fffa }, + {_MMIO (0x2794 ),0x0000bfef }, + {_MMIO (0x2798 ),0x0007fffa }, + {_MMIO (0x279c ),0x0000bfdf }, + {_MMIO (0x27a0 ),0x0007fffa }, + {_MMIO (0x27a4 ),0x0000bfbf }, + {_MMIO (0x27a8 ),0x0007fffa }, + {_MMIO (0x27ac ),0x0000bf7f }, +}; + +static const struct i915_oa_reg flex_eu_config_data_port_reads_coalescing[] = { + {_MMIO (0xe458 ),0x00005004 }, + {_MMIO (0xe558 ),0x00000003 }, + {_MMIO (0xe658 ),0x00002001 }, + {_MMIO (0xe758 ),0x00778008 }, + {_MMIO (0xe45c ),0x00088078 }, + {_MMIO (0xe55c ),0x00808708 }, + {_MMIO (0xe65c ),0x00a08908 }, +}; + +static const struct i915_oa_reg mux_config_data_port_reads_coalescing_0_subslices_0x01[] = { + {_MMIO (0x9888 ),0x103d0005 }, + {_MMIO (0x9888 ),0x163d240b }, + {_MMIO (0x9888 ),0x1058022f }, + {_MMIO (0x9888 ),0x185b5520 }, + {_MMIO (0x9888 ),0x198b0003 }, + {_MMIO (0x9888 ),0x005cc000 }, + {_MMIO (0x9888 ),0x065cc000 }, + {_MMIO (0x9888 ),0x085cc000 }, + {_MMIO (0x9888 ),0x0a5cc000 }, + {_MMIO (0x9888 ),0x0c5cc000 }, + {_MMIO (0x9888 ),0x0e5cc000 }, + {_MMIO (0x9888 ),0x025c4000 }, + {_MMIO (0x9888 ),0x045c8000 }, + {_MMIO (0x9888 ),0x003d0000 }, + {_MMIO (0x9888 ),0x063d00b0 }, + {_MMIO (0x9888 ),0x083d0182 }, + {_MMIO (0x9888 ),0x0a3d10a0 }, + {_MMIO (0x9888 ),0x0c3d11a2 }, + {_MMIO (0x9888 ),0x0e3d0000 }, + {_MMIO (0x9888 ),0x183d0000 }, + {_MMIO (0x9888 ),0x1a3d0000 }, + {_MMIO (0x9888 ),0x0e582242 }, + {_MMIO (0x9888 ),0x00586700 }, + {_MMIO (0x9888 ),0x0258004f }, + {_MMIO (0x9888 ),0x0658c000 }, + {_MMIO (0x9888 ),0x0858c000 }, + {_MMIO (0x9888 ),0x0a58c000 }, + {_MMIO (0x9888 ),0x0c58c000 }, + {_MMIO (0x9888 ),0x045b6300 }, + {_MMIO (0x9888 ),0x105b0000 }, + {_MMIO (0x9888 ),0x005b4000 }, + {_MMIO (0x9888 ),0x0e5b4000 }, + {_MMIO (0x9888 ),0x1a5b0155 }, + {_MMIO (0x9888 ),0x025b4000 }, + {_MMIO (0x9888 ),0x0a5b0000 }, + {_MMIO (0x9888 ),0x0c5b4000 }, + {_MMIO (0x9888 ),0x0c1fa800 }, + {_MMIO (0x9888 ),0x0e1faaa0 }, + {_MMIO (0x9888 ),0x101f02aa }, + {_MMIO (0x9888 ),0x00384000 }, + {_MMIO (0x9888 ),0x0e384000 }, + {_MMIO (0x9888 ),0x16384000 }, + {_MMIO (0x9888 ),0x18381555 }, + {_MMIO (0x9888 ),0x02384000 }, + {_MMIO (0x9888 ),0x04384000 }, + {_MMIO (0x9888 ),0x0a384000 }, + {_MMIO (0x9888 ),0x0c384000 }, + {_MMIO (0x9888 ),0x0039a000 }, + {_MMIO (0x9888 ),0x0639a000 }, + {_MMIO (0x9888 ),0x0839a000 }, + {_MMIO (0x9888 ),0x0a39a000 }, + {_MMIO (0x9888 ),0x0c39a000 }, + {_MMIO (0x9888 ),0x0e39a000 }, + {_MMIO (0x9888 ),0x02392000 }, + {_MMIO (0x9888 ),0x04398000 }, + {_MMIO (0x9888 ),0x018a8000 }, + {_MMIO (0x9888 ),0x0f8a8000 }, + {_MMIO (0x9888 ),0x198a8000 }, + {_MMIO (0x9888 ),0x1b8aaaa0 }, + {_MMIO (0x9888 ),0x1d8a0002 }, + {_MMIO (0x9888 ),0x038a8000 }, + {_MMIO (0x9888 ),0x058a8000 }, + {_MMIO (0x9888 ),0x0b8a8000 }, + {_MMIO (0x9888 ),0x0d8a8000 }, + {_MMIO (0x9888 ),0x038b6300 }, + {_MMIO (0x9888 ),0x058b0062 }, + {_MMIO (0x9888 ),0x118b0000 }, + {_MMIO (0x9888 ),0x238b02a0 }, + {_MMIO (0x9888 ),0x258b5555 }, + {_MMIO (0x9888 ),0x278b0015 }, + {_MMIO (0x9888 ),0x1f85aa80 }, + {_MMIO (0x9888 ),0x2185aaaa }, + {_MMIO (0x9888 ),0x2385002a }, + {_MMIO (0x9888 ),0x01834000 }, + {_MMIO (0x9888 ),0x0f834000 }, + {_MMIO (0x9888 ),0x19835400 }, + {_MMIO (0x9888 ),0x1b830155 }, + {_MMIO (0x9888 ),0x03834000 }, + {_MMIO (0x9888 ),0x05834000 }, + {_MMIO (0x9888 ),0x07834000 }, + {_MMIO (0x9888 ),0x09834000 }, + {_MMIO (0x9888 ),0x0b834000 }, + {_MMIO (0x9888 ),0x0d834000 }, + {_MMIO (0x9888 ),0x0184c000 }, + {_MMIO (0x9888 ),0x0784c000 }, + {_MMIO (0x9888 ),0x0984c000 }, + {_MMIO (0x9888 ),0x0b84c000 }, + {_MMIO (0x9888 ),0x0d84c000 }, + {_MMIO (0x9888 ),0x0f84c000 }, + {_MMIO (0x9888 ),0x0384c000 }, + {_MMIO (0x9888 ),0x0584c000 }, + {_MMIO (0x9888 ),0x1180c000 }, + {_MMIO (0x9888 ),0x1780c000 }, + {_MMIO (0x9888 ),0x1980c000 }, + {_MMIO (0x9888 ),0x1b80c000 }, + {_MMIO (0x9888 ),0x1d80c000 }, + {_MMIO (0x9888 ),0x1f80c000 }, + {_MMIO (0x9888 ),0x1380c000 }, + {_MMIO (0x9888 ),0x1580c000 }, + {_MMIO (0xd24 ),0x00000000 }, + {_MMIO (0x9888 ),0x4d801000 }, + {_MMIO (0x9888 ),0x3d800000 }, + {_MMIO (0x9888 ),0x4f800001 }, + {_MMIO (0x9888 ),0x43800000 }, + {_MMIO (0x9888 ),0x51800000 }, + {_MMIO (0x9888 ),0x45800000 }, + {_MMIO (0x9888 ),0x53800000 }, + {_MMIO (0x9888 ),0x47800420 }, + {_MMIO (0x9888 ),0x21800000 }, + {_MMIO (0x9888 ),0x31800000 }, + {_MMIO (0x9888 ),0x3f800421 }, + {_MMIO (0x9888 ),0x41800041 }, +}; + +static const struct i915_oa_reg * +get_data_port_reads_coalescing_mux_config (struct drm_i915_private *dev_priv, +int *len) +{ +if (INTEL_INFO (dev_priv)->sseu.subslice_mask &0x01 ) { + *len =ARRAY_SIZE (mux_config_data_port_reads_coalescing_0_subslices_0x01); +return mux_config_data_port_reads_coalescing_0_subslices_0x01; + } + + *len =0 ; +return NULL ; +} + +static const struct i915_oa_reg b_counter_config_data_port_writes_coalescing[] = { + {_MMIO (0x2724 ),0xf0800000 }, + {_MMIO (0x2720 ),0x00000000 }, + {_MMIO (0x2714 ),0xf0800000 }, + {_MMIO (0x2710 ),0x00000000 }, + {_MMIO (0x274c ),0xba98ba98 }, + {_MMIO (0x2748 ),0xba98ba98 }, + {_MMIO (0x2744 ),0x00003377 }, + {_MMIO (0x2740 ),0x00000000 }, + {_MMIO (0x2770 ),0x0007ff72 }, + {_MMIO (0x2774 ),0x0000bfd0 }, + {_MMIO (0x2778 ),0x0007ff62 }, + {_MMIO (0x277c ),0x0000bfd0 }, + {_MMIO (0x2780 ),0x0007ff42 }, + {_MMIO (0x2784 ),0x0000bfd0 }, + {_MMIO (0x2788 ),0x0007ff02 }, + {_MMIO (0x278c ),0x0000bfd0 }, + {_MMIO (0x2790 ),0x0005fff2 }, + {_MMIO (0x2794 ),0x0000bfd0 }, + {_MMIO (0x2798 ),0x0005ffe2 }, + {_MMIO (0x279c ),0x0000bfd0 }, + {_MMIO (0x27a0 ),0x0005ffc2 }, + {_MMIO (0x27a4 ),0x0000bfd0 }, + {_MMIO (0x27a8 ),0x0005ff82 }, + {_MMIO (0x27ac ),0x0000bfd0 }, +}; + +static const struct i915_oa_reg flex_eu_config_data_port_writes_coalescing[] = { + {_MMIO (0xe458 ),0x00005004 }, + {_MMIO (0xe558 ),0x00000003 }, + {_MMIO (0xe658 ),0x00002001 }, + {_MMIO (0xe758 ),0x00778008 }, + {_MMIO (0xe45c ),0x00088078 }, + {_MMIO (0xe55c ),0x00808708 }, + {_MMIO (0xe65c ),0x00a08908 }, +}; + +static const struct i915_oa_reg mux_config_data_port_writes_coalescing_0_subslices_0x01[] = { + {_MMIO (0x9888 ),0x103d0005 }, + {_MMIO (0x9888 ),0x143d0120 }, + {_MMIO (0x9888 ),0x163d2400 }, + {_MMIO (0x9888 ),0x1058022f }, + {_MMIO (0x9888 ),0x105b0000 }, + {_MMIO (0x9888 ),0x198b0003 }, + {_MMIO (0x9888 ),0x005cc000 }, + {_MMIO (0x9888 ),0x065cc000 }, + {_MMIO (0x9888 ),0x085cc000 }, + {_MMIO (0x9888 ),0x0a5cc000 }, + {_MMIO (0x9888 ),0x0e5cc000 }, + {_MMIO (0x9888 ),0x025c4000 }, + {_MMIO (0x9888 ),0x045c8000 }, + {_MMIO (0x9888 ),0x003d0000 }, + {_MMIO (0x9888 ),0x063d0094 }, + {_MMIO (0x9888 ),0x083d0182 }, + {_MMIO (0x9888 ),0x0a3d1814 }, + {_MMIO (0x9888 ),0x0e3d0000 }, + {_MMIO (0x9888 ),0x183d0000 }, + {_MMIO (0x9888 ),0x1a3d0000 }, + {_MMIO (0x9888 ),0x0c3d0000 }, + {_MMIO (0x9888 ),0x0e582242 }, + {_MMIO (0x9888 ),0x00586700 }, + {_MMIO (0x9888 ),0x0258004f }, + {_MMIO (0x9888 ),0x0658c000 }, + {_MMIO (0x9888 ),0x0858c000 }, + {_MMIO (0x9888 ),0x0a58c000 }, + {_MMIO (0x9888 ),0x045b6a80 }, + {_MMIO (0x9888 ),0x005b4000 }, + {_MMIO (0x9888 ),0x0e5b4000 }, + {_MMIO (0x9888 ),0x185b5400 }, + {_MMIO (0x9888 ),0x1a5b0141 }, + {_MMIO (0x9888 ),0x025b4000 }, + {_MMIO (0x9888 ),0x0a5b0000 }, + {_MMIO (0x9888 ),0x0c5b4000 }, + {_MMIO (0x9888 ),0x0c1fa800 }, + {_MMIO (0x9888 ),0x0e1faaa0 }, + {_MMIO (0x9888 ),0x101f0282 }, + {_MMIO (0x9888 ),0x00384000 }, + {_MMIO (0x9888 ),0x0e384000 }, + {_MMIO (0x9888 ),0x16384000 }, + {_MMIO (0x9888 ),0x18381415 }, + {_MMIO (0x9888 ),0x02384000 }, + {_MMIO (0x9888 ),0x04384000 }, + {_MMIO (0x9888 ),0x0a384000 }, + {_MMIO (0x9888 ),0x0c384000 }, + {_MMIO (0x9888 ),0x0039a000 }, + {_MMIO (0x9888 ),0x0639a000 }, + {_MMIO (0x9888 ),0x0839a000 }, + {_MMIO (0x9888 ),0x0a39a000 }, + {_MMIO (0x9888 ),0x0e39a000 }, + {_MMIO (0x9888 ),0x02392000 }, + {_MMIO (0x9888 ),0x04398000 }, + {_MMIO (0x9888 ),0x018a8000 }, + {_MMIO (0x9888 ),0x0f8a8000 }, + {_MMIO (0x9888 ),0x198a8000 }, + {_MMIO (0x9888 ),0x1b8a82a0 }, + {_MMIO (0x9888 ),0x1d8a0002 }, + {_MMIO (0x9888 ),0x038a8000 }, + {_MMIO (0x9888 ),0x058a8000 }, + {_MMIO (0x9888 ),0x0b8a8000 }, + {_MMIO (0x9888 ),0x0d8a8000 }, + {_MMIO (0x9888 ),0x038b6300 }, + {_MMIO (0x9888 ),0x058b0062 }, + {_MMIO (0x9888 ),0x118b0000 }, + {_MMIO (0x9888 ),0x238b02a0 }, + {_MMIO (0x9888 ),0x258b1555 }, + {_MMIO (0x9888 ),0x278b0014 }, + {_MMIO (0x9888 ),0x1f85aa80 }, + {_MMIO (0x9888 ),0x21852aaa }, + {_MMIO (0x9888 ),0x23850028 }, + {_MMIO (0x9888 ),0x01834000 }, + {_MMIO (0x9888 ),0x0f834000 }, + {_MMIO (0x9888 ),0x19835400 }, + {_MMIO (0x9888 ),0x1b830141 }, + {_MMIO (0x9888 ),0x03834000 }, + {_MMIO (0x9888 ),0x05834000 }, + {_MMIO (0x9888 ),0x07834000 }, + {_MMIO (0x9888 ),0x09834000 }, + {_MMIO (0x9888 ),0x0b834000 }, + {_MMIO (0x9888 ),0x0d834000 }, + {_MMIO (0x9888 ),0x0184c000 }, + {_MMIO (0x9888 ),0x0784c000 }, + {_MMIO (0x9888 ),0x0984c000 }, + {_MMIO (0x9888 ),0x0b84c000 }, + {_MMIO (0x9888 ),0x0f84c000 }, + {_MMIO (0x9888 ),0x0384c000 }, + {_MMIO (0x9888 ),0x0584c000 }, + {_MMIO (0x9888 ),0x1180c000 }, + {_MMIO (0x9888 ),0x1780c000 }, + {_MMIO (0x9888 ),0x1980c000 }, + {_MMIO (0x9888 ),0x1b80c000 }, + {_MMIO (0x9888 ),0x1f80c000 }, + {_MMIO (0x9888 ),0x1380c000 }, + {_MMIO (0x9888 ),0x1580c000 }, + {_MMIO (0xd24 ),0x00000000 }, + {_MMIO (0x9888 ),0x4d801000 }, + {_MMIO (0x9888 ),0x3d800000 }, + {_MMIO (0x9888 ),0x4f800001 }, + {_MMIO (0x9888 ),0x43800000 }, + {_MMIO (0x9888 ),0x51800000 }, + {_MMIO (0x9888 ),0x45800000 }, + {_MMIO (0x9888 ),0x21800000 }, + {_MMIO (0x9888 ),0x31800000 }, + {_MMIO (0x9888 ),0x53800000 }, + {_MMIO (0x9888 ),0x47800420 }, + {_MMIO (0x9888 ),0x3f800421 }, + {_MMIO (0x9888 ),0x41800041 }, +}; + +static const struct i915_oa_reg * +get_data_port_writes_coalescing_mux_config (struct drm_i915_private *dev_priv, +int *len) +{ +if (INTEL_INFO (dev_priv)->sseu.subslice_mask &0x01 ) { + *len =ARRAY_SIZE (mux_config_data_port_writes_coalescing_0_subslices_0x01); +return mux_config_data_port_writes_coalescing_0_subslices_0x01; + } + + *len =0 ; +return NULL ; +} + +static const struct i915_oa_reg b_counter_config_hdc_and_sf[] = { + {_MMIO (0x2740 ),0x00000000 }, + {_MMIO (0x2744 ),0x00800000 }, + {_MMIO (0x2710 ),0x00000000 }, + {_MMIO (0x2714 ),0x10800000 }, + {_MMIO (0x2720 ),0x00000000 }, + {_MMIO (0x2724 ),0x00800000 }, + {_MMIO (0x2770 ),0x00000002 }, + {_MMIO (0x2774 ),0x0000fff7 }, +}; + +static const struct i915_oa_reg flex_eu_config_hdc_and_sf[] = { + {_MMIO (0xe458 ),0x00005004 }, + {_MMIO (0xe558 ),0x00010003 }, + {_MMIO (0xe658 ),0x00012011 }, + {_MMIO (0xe758 ),0x00015014 }, + {_MMIO (0xe45c ),0x00051050 }, + {_MMIO (0xe55c ),0x00053052 }, + {_MMIO (0xe65c ),0x00055054 }, +}; + +static const struct i915_oa_reg mux_config_hdc_and_sf[] = { + {_MMIO (0x9888 ),0x105c0232 }, + {_MMIO (0x9888 ),0x10580232 }, + {_MMIO (0x9888 ),0x10380232 }, + {_MMIO (0x9888 ),0x10dc0232 }, + {_MMIO (0x9888 ),0x10d80232 }, + {_MMIO (0x9888 ),0x10b80232 }, + {_MMIO (0x9888 ),0x118e4400 }, + {_MMIO (0x9888 ),0x025c6080 }, + {_MMIO (0x9888 ),0x045c004b }, + {_MMIO (0x9888 ),0x005c8000 }, + {_MMIO (0x9888 ),0x00582080 }, + {_MMIO (0x9888 ),0x0258004b }, + {_MMIO (0x9888 ),0x025b4000 }, + {_MMIO (0x9888 ),0x045b4000 }, + {_MMIO (0x9888 ),0x0c1fa000 }, + {_MMIO (0x9888 ),0x0e1f00aa }, + {_MMIO (0x9888 ),0x04386080 }, + {_MMIO (0x9888 ),0x0638404b }, + {_MMIO (0x9888 ),0x02384000 }, + {_MMIO (0x9888 ),0x08384000 }, + {_MMIO (0x9888 ),0x0a380000 }, + {_MMIO (0x9888 ),0x0c380000 }, + {_MMIO (0x9888 ),0x00398000 }, + {_MMIO (0x9888 ),0x0239a000 }, + {_MMIO (0x9888 ),0x0439a000 }, + {_MMIO (0x9888 ),0x06392000 }, + {_MMIO (0x9888 ),0x0cdc25c1 }, + {_MMIO (0x9888 ),0x0adcc000 }, + {_MMIO (0x9888 ),0x0ad825c1 }, + {_MMIO (0x9888 ),0x18db4000 }, + {_MMIO (0x9888 ),0x1adb0001 }, + {_MMIO (0x9888 ),0x0e9f8000 }, + {_MMIO (0x9888 ),0x109f02aa }, + {_MMIO (0x9888 ),0x0eb825c1 }, + {_MMIO (0x9888 ),0x18b80154 }, + {_MMIO (0x9888 ),0x0ab9a000 }, + {_MMIO (0x9888 ),0x0cb9a000 }, + {_MMIO (0x9888 ),0x0eb9a000 }, + {_MMIO (0x9888 ),0x0d88c000 }, + {_MMIO (0x9888 ),0x0f88000f }, + {_MMIO (0x9888 ),0x038a8000 }, + {_MMIO (0x9888 ),0x058a8000 }, + {_MMIO (0x9888 ),0x078a8000 }, + {_MMIO (0x9888 ),0x098a8000 }, + {_MMIO (0x9888 ),0x0b8a8000 }, + {_MMIO (0x9888 ),0x0d8a8000 }, + {_MMIO (0x9888 ),0x258baa05 }, + {_MMIO (0x9888 ),0x278b002a }, + {_MMIO (0x9888 ),0x238b2a80 }, + {_MMIO (0x9888 ),0x198c5400 }, + {_MMIO (0x9888 ),0x1b8c0015 }, + {_MMIO (0x9888 ),0x098dc000 }, + {_MMIO (0x9888 ),0x0b8da000 }, + {_MMIO (0x9888 ),0x0d8da000 }, + {_MMIO (0x9888 ),0x0f8da000 }, + {_MMIO (0x9888 ),0x098e05c0 }, + {_MMIO (0x9888 ),0x058e0000 }, + {_MMIO (0x9888 ),0x198f0020 }, + {_MMIO (0x9888 ),0x2185aa0a }, + {_MMIO (0x9888 ),0x2385002a }, + {_MMIO (0x9888 ),0x1f85aa00 }, + {_MMIO (0x9888 ),0x19835000 }, + {_MMIO (0x9888 ),0x1b830155 }, + {_MMIO (0x9888 ),0x03834000 }, + {_MMIO (0x9888 ),0x05834000 }, + {_MMIO (0x9888 ),0x07834000 }, + {_MMIO (0x9888 ),0x09834000 }, + {_MMIO (0x9888 ),0x0b834000 }, + {_MMIO (0x9888 ),0x0d834000 }, + {_MMIO (0x9888 ),0x09848000 }, + {_MMIO (0x9888 ),0x0b84c000 }, + {_MMIO (0x9888 ),0x0d84c000 }, + {_MMIO (0x9888 ),0x0f84c000 }, + {_MMIO (0x9888 ),0x01848000 }, + {_MMIO (0x9888 ),0x0384c000 }, + {_MMIO (0x9888 ),0x0584c000 }, + {_MMIO (0x9888 ),0x07844000 }, + {_MMIO (0x9888 ),0x19808000 }, + {_MMIO (0x9888 ),0x1b80c000 }, + {_MMIO (0x9888 ),0x1d80c000 }, + {_MMIO (0x9888 ),0x1f80c000 }, + {_MMIO (0x9888 ),0x11808000 }, + {_MMIO (0x9888 ),0x1380c000 }, + {_MMIO (0x9888 ),0x1580c000 }, + {_MMIO (0x9888 ),0x17804000 }, + {_MMIO (0x9888 ),0x51800040 }, + {_MMIO (0x9888 ),0x43800400 }, + {_MMIO (0x9888 ),0x45800800 }, + {_MMIO (0x9888 ),0x53800000 }, + {_MMIO (0x9888 ),0x47800c62 }, + {_MMIO (0x9888 ),0x21800000 }, + {_MMIO (0x9888 ),0x31800000 }, + {_MMIO (0x9888 ),0x4d800000 }, + {_MMIO (0x9888 ),0x3f801042 }, + {_MMIO (0x9888 ),0x4f800000 }, + {_MMIO (0x9888 ),0x418014a4 }, +}; + +static const struct i915_oa_reg * +get_hdc_and_sf_mux_config (struct drm_i915_private *dev_priv, +int *len) +{ + *len =ARRAY_SIZE (mux_config_hdc_and_sf); +return mux_config_hdc_and_sf; +} + +static const struct i915_oa_reg b_counter_config_l3_1[] = { + {_MMIO (0x2740 ),0x00000000 }, + {_MMIO (0x2744 ),0x00800000 }, + {_MMIO (0x2710 ),0x00000000 }, + {_MMIO (0x2714 ),0xf0800000 }, + {_MMIO (0x2720 ),0x00000000 }, + {_MMIO (0x2724 ),0xf0800000 }, + {_MMIO (0x2770 ),0x00100070 }, + {_MMIO (0x2774 ),0x0000fff1 }, + {_MMIO (0x2778 ),0x00014002 }, + {_MMIO (0x277c ),0x0000c3ff }, + {_MMIO (0x2780 ),0x00010002 }, + {_MMIO (0x2784 ),0x0000c7ff }, + {_MMIO (0x2788 ),0x00004002 }, + {_MMIO (0x278c ),0x0000d3ff }, + {_MMIO (0x2790 ),0x00100700 }, + {_MMIO (0x2794 ),0x0000ff1f }, + {_MMIO (0x2798 ),0x00001402 }, + {_MMIO (0x279c ),0x0000fc3f }, + {_MMIO (0x27a0 ),0x00001002 }, + {_MMIO (0x27a4 ),0x0000fc7f }, + {_MMIO (0x27a8 ),0x00000402 }, + {_MMIO (0x27ac ),0x0000fd3f }, +}; + +static const struct i915_oa_reg flex_eu_config_l3_1[] = { + {_MMIO (0xe458 ),0x00005004 }, + {_MMIO (0xe558 ),0x00010003 }, + {_MMIO (0xe658 ),0x00012011 }, + {_MMIO (0xe758 ),0x00015014 }, + {_MMIO (0xe45c ),0x00051050 }, + {_MMIO (0xe55c ),0x00053052 }, + {_MMIO (0xe65c ),0x00055054 }, +}; + +static const struct i915_oa_reg mux_config_l3_1[] = { + {_MMIO (0x9888 ),0x10bf03da }, + {_MMIO (0x9888 ),0x14bf0001 }, + {_MMIO (0x9888 ),0x12980340 }, + {_MMIO (0x9888 ),0x12990340 }, + {_MMIO (0x9888 ),0x0cbf1187 }, + {_MMIO (0x9888 ),0x0ebf1205 }, + {_MMIO (0x9888 ),0x00bf0500 }, + {_MMIO (0x9888 ),0x02bf042b }, + {_MMIO (0x9888 ),0x04bf002c }, + {_MMIO (0x9888 ),0x0cdac000 }, + {_MMIO (0x9888 ),0x0edac000 }, + {_MMIO (0x9888 ),0x00da8000 }, + {_MMIO (0x9888 ),0x02dac000 }, + {_MMIO (0x9888 ),0x04da4000 }, + {_MMIO (0x9888 ),0x04983400 }, + {_MMIO (0x9888 ),0x10980000 }, + {_MMIO (0x9888 ),0x06990034 }, + {_MMIO (0x9888 ),0x10990000 }, + {_MMIO (0x9888 ),0x0c9dc000 }, + {_MMIO (0x9888 ),0x0e9dc000 }, + {_MMIO (0x9888 ),0x009d8000 }, + {_MMIO (0x9888 ),0x029dc000 }, + {_MMIO (0x9888 ),0x049d4000 }, + {_MMIO (0x9888 ),0x109f02a8 }, + {_MMIO (0x9888 ),0x0c9fa000 }, + {_MMIO (0x9888 ),0x0e9f00ba }, + {_MMIO (0x9888 ),0x0cb88000 }, + {_MMIO (0x9888 ),0x0cb95000 }, + {_MMIO (0x9888 ),0x0eb95000 }, + {_MMIO (0x9888 ),0x00b94000 }, + {_MMIO (0x9888 ),0x02b95000 }, + {_MMIO (0x9888 ),0x04b91000 }, + {_MMIO (0x9888 ),0x06b92000 }, + {_MMIO (0x9888 ),0x0cba4000 }, + {_MMIO (0x9888 ),0x0f88000f }, + {_MMIO (0x9888 ),0x03888000 }, + {_MMIO (0x9888 ),0x05888000 }, + {_MMIO (0x9888 ),0x07888000 }, + {_MMIO (0x9888 ),0x09888000 }, + {_MMIO (0x9888 ),0x0b888000 }, + {_MMIO (0x9888 ),0x0d880400 }, + {_MMIO (0x9888 ),0x258b800a }, + {_MMIO (0x9888 ),0x278b002a }, + {_MMIO (0x9888 ),0x238b5500 }, + {_MMIO (0x9888 ),0x198c4000 }, + {_MMIO (0x9888 ),0x1b8c0015 }, + {_MMIO (0x9888 ),0x038c4000 }, + {_MMIO (0x9888 ),0x058c4000 }, + {_MMIO (0x9888 ),0x078c4000 }, + {_MMIO (0x9888 ),0x098c4000 }, + {_MMIO (0x9888 ),0x0b8c4000 }, + {_MMIO (0x9888 ),0x0d8c4000 }, + {_MMIO (0x9888 ),0x0d8da000 }, + {_MMIO (0x9888 ),0x0f8da000 }, + {_MMIO (0x9888 ),0x018d8000 }, + {_MMIO (0x9888 ),0x038da000 }, + {_MMIO (0x9888 ),0x058da000 }, + {_MMIO (0x9888 ),0x078d2000 }, + {_MMIO (0x9888 ),0x2185800a }, + {_MMIO (0x9888 ),0x2385002a }, + {_MMIO (0x9888 ),0x1f85aa00 }, + {_MMIO (0x9888 ),0x1b830154 }, + {_MMIO (0x9888 ),0x03834000 }, + {_MMIO (0x9888 ),0x05834000 }, + {_MMIO (0x9888 ),0x07834000 }, + {_MMIO (0x9888 ),0x09834000 }, + {_MMIO (0x9888 ),0x0b834000 }, + {_MMIO (0x9888 ),0x0d834000 }, + {_MMIO (0x9888 ),0x0d84c000 }, + {_MMIO (0x9888 ),0x0f84c000 }, + {_MMIO (0x9888 ),0x01848000 }, + {_MMIO (0x9888 ),0x0384c000 }, + {_MMIO (0x9888 ),0x0584c000 }, + {_MMIO (0x9888 ),0x07844000 }, + {_MMIO (0x9888 ),0x1d80c000 }, + {_MMIO (0x9888 ),0x1f80c000 }, + {_MMIO (0x9888 ),0x11808000 }, + {_MMIO (0x9888 ),0x1380c000 }, + {_MMIO (0x9888 ),0x1580c000 }, + {_MMIO (0x9888 ),0x17804000 }, + {_MMIO (0x9888 ),0x53800000 }, + {_MMIO (0x9888 ),0x45800000 }, + {_MMIO (0x9888 ),0x47800000 }, + {_MMIO (0x9888 ),0x21800000 }, + {_MMIO (0x9888 ),0x31800000 }, + {_MMIO (0x9888 ),0x4d800000 }, + {_MMIO (0x9888 ),0x3f800000 }, + {_MMIO (0x9888 ),0x4f800000 }, + {_MMIO (0x9888 ),0x41800060 }, +}; + +static const struct i915_oa_reg * +get_l3_1_mux_config (struct drm_i915_private *dev_priv, +int *len) +{ + *len =ARRAY_SIZE (mux_config_l3_1); +return mux_config_l3_1; +} + +static const struct i915_oa_reg b_counter_config_l3_2[] = { + {_MMIO (0x2740 ),0x00000000 }, + {_MMIO (0x2744 ),0x00800000 }, + {_MMIO (0x2710 ),0x00000000 }, + {_MMIO (0x2714 ),0xf0800000 }, + {_MMIO (0x2720 ),0x00000000 }, + {_MMIO (0x2724 ),0xf0800000 }, + {_MMIO (0x2770 ),0x00100070 }, + {_MMIO (0x2774 ),0x0000fff1 }, + {_MMIO (0x2778 ),0x00014002 }, + {_MMIO (0x277c ),0x0000c3ff }, + {_MMIO (0x2780 ),0x00010002 }, + {_MMIO (0x2784 ),0x0000c7ff }, + {_MMIO (0x2788 ),0x00004002 }, + {_MMIO (0x278c ),0x0000d3ff }, + {_MMIO (0x2790 ),0x00100700 }, + {_MMIO (0x2794 ),0x0000ff1f }, + {_MMIO (0x2798 ),0x00001402 }, + {_MMIO (0x279c ),0x0000fc3f }, + {_MMIO (0x27a0 ),0x00001002 }, + {_MMIO (0x27a4 ),0x0000fc7f }, + {_MMIO (0x27a8 ),0x00000402 }, + {_MMIO (0x27ac ),0x0000fd3f }, +}; + +static const struct i915_oa_reg flex_eu_config_l3_2[] = { + {_MMIO (0xe458 ),0x00005004 }, + {_MMIO (0xe558 ),0x00010003 }, + {_MMIO (0xe658 ),0x00012011 }, + {_MMIO (0xe758 ),0x00015014 }, + {_MMIO (0xe45c ),0x00051050 }, + {_MMIO (0xe55c ),0x00053052 }, + {_MMIO (0xe65c ),0x00055054 }, +}; + +static const struct i915_oa_reg mux_config_l3_2[] = { + {_MMIO (0x9888 ),0x103f03da }, + {_MMIO (0x9888 ),0x143f0001 }, + {_MMIO (0x9888 ),0x12180340 }, + {_MMIO (0x9888 ),0x12190340 }, + {_MMIO (0x9888 ),0x0c3f1187 }, + {_MMIO (0x9888 ),0x0e3f1205 }, + {_MMIO (0x9888 ),0x003f0500 }, + {_MMIO (0x9888 ),0x023f042b }, + {_MMIO (0x9888 ),0x043f002c }, + {_MMIO (0x9888 ),0x0c5ac000 }, + {_MMIO (0x9888 ),0x0e5ac000 }, + {_MMIO (0x9888 ),0x005a8000 }, + {_MMIO (0x9888 ),0x025ac000 }, + {_MMIO (0x9888 ),0x045a4000 }, + {_MMIO (0x9888 ),0x04183400 }, + {_MMIO (0x9888 ),0x10180000 }, + {_MMIO (0x9888 ),0x06190034 }, + {_MMIO (0x9888 ),0x10190000 }, + {_MMIO (0x9888 ),0x0c1dc000 }, + {_MMIO (0x9888 ),0x0e1dc000 }, + {_MMIO (0x9888 ),0x001d8000 }, + {_MMIO (0x9888 ),0x021dc000 }, + {_MMIO (0x9888 ),0x041d4000 }, + {_MMIO (0x9888 ),0x101f02a8 }, + {_MMIO (0x9888 ),0x0c1fa000 }, + {_MMIO (0x9888 ),0x0e1f00ba }, + {_MMIO (0x9888 ),0x0c388000 }, + {_MMIO (0x9888 ),0x0c395000 }, + {_MMIO (0x9888 ),0x0e395000 }, + {_MMIO (0x9888 ),0x00394000 }, + {_MMIO (0x9888 ),0x02395000 }, + {_MMIO (0x9888 ),0x04391000 }, + {_MMIO (0x9888 ),0x06392000 }, + {_MMIO (0x9888 ),0x0c3a4000 }, + {_MMIO (0x9888 ),0x1b8aa800 }, + {_MMIO (0x9888 ),0x1d8a0002 }, + {_MMIO (0x9888 ),0x038a8000 }, + {_MMIO (0x9888 ),0x058a8000 }, + {_MMIO (0x9888 ),0x078a8000 }, + {_MMIO (0x9888 ),0x098a8000 }, + {_MMIO (0x9888 ),0x0b8a8000 }, + {_MMIO (0x9888 ),0x0d8a8000 }, + {_MMIO (0x9888 ),0x258b4005 }, + {_MMIO (0x9888 ),0x278b0015 }, + {_MMIO (0x9888 ),0x238b2a80 }, + {_MMIO (0x9888 ),0x2185800a }, + {_MMIO (0x9888 ),0x2385002a }, + {_MMIO (0x9888 ),0x1f85aa00 }, + {_MMIO (0x9888 ),0x1b830154 }, + {_MMIO (0x9888 ),0x03834000 }, + {_MMIO (0x9888 ),0x05834000 }, + {_MMIO (0x9888 ),0x07834000 }, + {_MMIO (0x9888 ),0x09834000 }, + {_MMIO (0x9888 ),0x0b834000 }, + {_MMIO (0x9888 ),0x0d834000 }, + {_MMIO (0x9888 ),0x0d84c000 }, + {_MMIO (0x9888 ),0x0f84c000 }, + {_MMIO (0x9888 ),0x01848000 }, + {_MMIO (0x9888 ),0x0384c000 }, + {_MMIO (0x9888 ),0x0584c000 }, + {_MMIO (0x9888 ),0x07844000 }, + {_MMIO (0x9888 ),0x1d80c000 }, + {_MMIO (0x9888 ),0x1f80c000 }, + {_MMIO (0x9888 ),0x11808000 }, + {_MMIO (0x9888 ),0x1380c000 }, + {_MMIO (0x9888 ),0x1580c000 }, + {_MMIO (0x9888 ),0x17804000 }, + {_MMIO (0x9888 ),0x53800000 }, + {_MMIO (0x9888 ),0x45800000 }, + {_MMIO (0x9888 ),0x47800000 }, + {_MMIO (0x9888 ),0x21800000 }, + {_MMIO (0x9888 ),0x31800000 }, + {_MMIO (0x9888 ),0x4d800000 }, + {_MMIO (0x9888 ),0x3f800000 }, + {_MMIO (0x9888 ),0x4f800000 }, + {_MMIO (0x9888 ),0x41800060 }, +}; + +static const struct i915_oa_reg * +get_l3_2_mux_config (struct drm_i915_private *dev_priv, +int *len) +{ + *len =ARRAY_SIZE (mux_config_l3_2); +return mux_config_l3_2; +} + +static const struct i915_oa_reg b_counter_config_l3_3[] = { + {_MMIO (0x2740 ),0x00000000 }, + {_MMIO (0x2744 ),0x00800000 }, + {_MMIO (0x2710 ),0x00000000 }, + {_MMIO (0x2714 ),0xf0800000 }, + {_MMIO (0x2720 ),0x00000000 }, + {_MMIO (0x2724 ),0xf0800000 }, + {_MMIO (0x2770 ),0x00100070 }, + {_MMIO (0x2774 ),0x0000fff1 }, + {_MMIO (0x2778 ),0x00014002 }, + {_MMIO (0x277c ),0x0000c3ff }, + {_MMIO (0x2780 ),0x00010002 }, + {_MMIO (0x2784 ),0x0000c7ff }, + {_MMIO (0x2788 ),0x00004002 }, + {_MMIO (0x278c ),0x0000d3ff }, + {_MMIO (0x2790 ),0x00100700 }, + {_MMIO (0x2794 ),0x0000ff1f }, + {_MMIO (0x2798 ),0x00001402 }, + {_MMIO (0x279c ),0x0000fc3f }, + {_MMIO (0x27a0 ),0x00001002 }, + {_MMIO (0x27a4 ),0x0000fc7f }, + {_MMIO (0x27a8 ),0x00000402 }, + {_MMIO (0x27ac ),0x0000fd3f }, +}; + +static const struct i915_oa_reg flex_eu_config_l3_3[] = { + {_MMIO (0xe458 ),0x00005004 }, + {_MMIO (0xe558 ),0x00010003 }, + {_MMIO (0xe658 ),0x00012011 }, + {_MMIO (0xe758 ),0x00015014 }, + {_MMIO (0xe45c ),0x00051050 }, + {_MMIO (0xe55c ),0x00053052 }, + {_MMIO (0xe65c ),0x00055054 }, +}; + +static const struct i915_oa_reg mux_config_l3_3[] = { + {_MMIO (0x9888 ),0x121b0340 }, + {_MMIO (0x9888 ),0x103f0274 }, + {_MMIO (0x9888 ),0x123f0000 }, + {_MMIO (0x9888 ),0x129b0340 }, + {_MMIO (0x9888 ),0x10bf0274 }, + {_MMIO (0x9888 ),0x12bf0000 }, + {_MMIO (0x9888 ),0x041b3400 }, + {_MMIO (0x9888 ),0x101b0000 }, + {_MMIO (0x9888 ),0x045c8000 }, + {_MMIO (0x9888 ),0x0a3d4000 }, + {_MMIO (0x9888 ),0x003f0080 }, + {_MMIO (0x9888 ),0x023f0793 }, + {_MMIO (0x9888 ),0x043f0014 }, + {_MMIO (0x9888 ),0x04588000 }, + {_MMIO (0x9888 ),0x005a8000 }, + {_MMIO (0x9888 ),0x025ac000 }, + {_MMIO (0x9888 ),0x045a4000 }, + {_MMIO (0x9888 ),0x0a5b4000 }, + {_MMIO (0x9888 ),0x001d8000 }, + {_MMIO (0x9888 ),0x021dc000 }, + {_MMIO (0x9888 ),0x041d4000 }, + {_MMIO (0x9888 ),0x0c1fa000 }, + {_MMIO (0x9888 ),0x0e1f002a }, + {_MMIO (0x9888 ),0x0a384000 }, + {_MMIO (0x9888 ),0x00394000 }, + {_MMIO (0x9888 ),0x02395000 }, + {_MMIO (0x9888 ),0x04399000 }, + {_MMIO (0x9888 ),0x069b0034 }, + {_MMIO (0x9888 ),0x109b0000 }, + {_MMIO (0x9888 ),0x06dc4000 }, + {_MMIO (0x9888 ),0x0cbd4000 }, + {_MMIO (0x9888 ),0x0cbf0981 }, + {_MMIO (0x9888 ),0x0ebf0a0f }, + {_MMIO (0x9888 ),0x06d84000 }, + {_MMIO (0x9888 ),0x0cdac000 }, + {_MMIO (0x9888 ),0x0edac000 }, + {_MMIO (0x9888 ),0x0cdb4000 }, + {_MMIO (0x9888 ),0x0c9dc000 }, + {_MMIO (0x9888 ),0x0e9dc000 }, + {_MMIO (0x9888 ),0x109f02a8 }, + {_MMIO (0x9888 ),0x0e9f0080 }, + {_MMIO (0x9888 ),0x0cb84000 }, + {_MMIO (0x9888 ),0x0cb95000 }, + {_MMIO (0x9888 ),0x0eb95000 }, + {_MMIO (0x9888 ),0x06b92000 }, + {_MMIO (0x9888 ),0x0f88000f }, + {_MMIO (0x9888 ),0x0d880400 }, + {_MMIO (0x9888 ),0x038a8000 }, + {_MMIO (0x9888 ),0x058a8000 }, + {_MMIO (0x9888 ),0x078a8000 }, + {_MMIO (0x9888 ),0x098a8000 }, + {_MMIO (0x9888 ),0x0b8a8000 }, + {_MMIO (0x9888 ),0x258b8009 }, + {_MMIO (0x9888 ),0x278b002a }, + {_MMIO (0x9888 ),0x238b2a80 }, + {_MMIO (0x9888 ),0x198c4000 }, + {_MMIO (0x9888 ),0x1b8c0015 }, + {_MMIO (0x9888 ),0x0d8c4000 }, + {_MMIO (0x9888 ),0x0d8da000 }, + {_MMIO (0x9888 ),0x0f8da000 }, + {_MMIO (0x9888 ),0x078d2000 }, + {_MMIO (0x9888 ),0x2185800a }, + {_MMIO (0x9888 ),0x2385002a }, + {_MMIO (0x9888 ),0x1f85aa00 }, + {_MMIO (0x9888 ),0x1b830154 }, + {_MMIO (0x9888 ),0x03834000 }, + {_MMIO (0x9888 ),0x05834000 }, + {_MMIO (0x9888 ),0x07834000 }, + {_MMIO (0x9888 ),0x09834000 }, + {_MMIO (0x9888 ),0x0b834000 }, + {_MMIO (0x9888 ),0x0d834000 }, + {_MMIO (0x9888 ),0x0d84c000 }, + {_MMIO (0x9888 ),0x0f84c000 }, + {_MMIO (0x9888 ),0x01848000 }, + {_MMIO (0x9888 ),0x0384c000 }, + {_MMIO (0x9888 ),0x0584c000 }, + {_MMIO (0x9888 ),0x07844000 }, + {_MMIO (0x9888 ),0x1d80c000 }, + {_MMIO (0x9888 ),0x1f80c000 }, + {_MMIO (0x9888 ),0x11808000 }, + {_MMIO (0x9888 ),0x1380c000 }, + {_MMIO (0x9888 ),0x1580c000 }, + {_MMIO (0x9888 ),0x17804000 }, + {_MMIO (0x9888 ),0x53800000 }, + {_MMIO (0x9888 ),0x45800c00 }, + {_MMIO (0x9888 ),0x47800c63 }, + {_MMIO (0x9888 ),0x21800000 }, + {_MMIO (0x9888 ),0x31800000 }, + {_MMIO (0x9888 ),0x4d800000 }, + {_MMIO (0x9888 ),0x3f8014a5 }, + {_MMIO (0x9888 ),0x4f800000 }, + {_MMIO (0x9888 ),0x41800045 }, +}; + +static const struct i915_oa_reg * +get_l3_3_mux_config (struct drm_i915_private *dev_priv, +int *len) +{ + *len =ARRAY_SIZE (mux_config_l3_3); +return mux_config_l3_3; +} + +static const struct i915_oa_reg b_counter_config_l3_4[] = { + {_MMIO (0x2740 ),0x00000000 }, + {_MMIO (0x2744 ),0x00800000 }, + {_MMIO (0x2710 ),0x00000000 }, + {_MMIO (0x2714 ),0xf0800000 }, + {_MMIO (0x2720 ),0x00000000 }, + {_MMIO (0x2724 ),0xf0800000 }, + {_MMIO (0x2770 ),0x00100070 }, + {_MMIO (0x2774 ),0x0000fff1 }, + {_MMIO (0x2778 ),0x00014002 }, + {_MMIO (0x277c ),0x0000c3ff }, + {_MMIO (0x2780 ),0x00010002 }, + {_MMIO (0x2784 ),0x0000c7ff }, + {_MMIO (0x2788 ),0x00004002 }, + {_MMIO (0x278c ),0x0000d3ff }, + {_MMIO (0x2790 ),0x00100700 }, + {_MMIO (0x2794 ),0x0000ff1f }, + {_MMIO (0x2798 ),0x00001402 }, + {_MMIO (0x279c ),0x0000fc3f }, + {_MMIO (0x27a0 ),0x00001002 }, + {_MMIO (0x27a4 ),0x0000fc7f }, + {_MMIO (0x27a8 ),0x00000402 }, + {_MMIO (0x27ac ),0x0000fd3f }, +}; + +static const struct i915_oa_reg flex_eu_config_l3_4[] = { + {_MMIO (0xe458 ),0x00005004 }, + {_MMIO (0xe558 ),0x00010003 }, + {_MMIO (0xe658 ),0x00012011 }, + {_MMIO (0xe758 ),0x00015014 }, + {_MMIO (0xe45c ),0x00051050 }, + {_MMIO (0xe55c ),0x00053052 }, + {_MMIO (0xe65c ),0x00055054 }, +}; + +static const struct i915_oa_reg mux_config_l3_4[] = { + {_MMIO (0x9888 ),0x121a0340 }, + {_MMIO (0x9888 ),0x103f0017 }, + {_MMIO (0x9888 ),0x123f0020 }, + {_MMIO (0x9888 ),0x129a0340 }, + {_MMIO (0x9888 ),0x10bf0017 }, + {_MMIO (0x9888 ),0x12bf0020 }, + {_MMIO (0x9888 ),0x041a3400 }, + {_MMIO (0x9888 ),0x101a0000 }, + {_MMIO (0x9888 ),0x043b8000 }, + {_MMIO (0x9888 ),0x0a3e0010 }, + {_MMIO (0x9888 ),0x003f0200 }, + {_MMIO (0x9888 ),0x023f0113 }, + {_MMIO (0x9888 ),0x043f0014 }, + {_MMIO (0x9888 ),0x02592000 }, + {_MMIO (0x9888 ),0x005a8000 }, + {_MMIO (0x9888 ),0x025ac000 }, + {_MMIO (0x9888 ),0x045a4000 }, + {_MMIO (0x9888 ),0x0a1c8000 }, + {_MMIO (0x9888 ),0x001d8000 }, + {_MMIO (0x9888 ),0x021dc000 }, + {_MMIO (0x9888 ),0x041d4000 }, + {_MMIO (0x9888 ),0x0a1e8000 }, + {_MMIO (0x9888 ),0x0c1fa000 }, + {_MMIO (0x9888 ),0x0e1f001a }, + {_MMIO (0x9888 ),0x00394000 }, + {_MMIO (0x9888 ),0x02395000 }, + {_MMIO (0x9888 ),0x04391000 }, + {_MMIO (0x9888 ),0x069a0034 }, + {_MMIO (0x9888 ),0x109a0000 }, + {_MMIO (0x9888 ),0x06bb4000 }, + {_MMIO (0x9888 ),0x0abe0040 }, + {_MMIO (0x9888 ),0x0cbf0984 }, + {_MMIO (0x9888 ),0x0ebf0a02 }, + {_MMIO (0x9888 ),0x02d94000 }, + {_MMIO (0x9888 ),0x0cdac000 }, + {_MMIO (0x9888 ),0x0edac000 }, + {_MMIO (0x9888 ),0x0c9c0400 }, + {_MMIO (0x9888 ),0x0c9dc000 }, + {_MMIO (0x9888 ),0x0e9dc000 }, + {_MMIO (0x9888 ),0x0c9e0400 }, + {_MMIO (0x9888 ),0x109f02a8 }, + {_MMIO (0x9888 ),0x0e9f0040 }, + {_MMIO (0x9888 ),0x0cb95000 }, + {_MMIO (0x9888 ),0x0eb95000 }, + {_MMIO (0x9888 ),0x0f88000f }, + {_MMIO (0x9888 ),0x0d880400 }, + {_MMIO (0x9888 ),0x038a8000 }, + {_MMIO (0x9888 ),0x058a8000 }, + {_MMIO (0x9888 ),0x078a8000 }, + {_MMIO (0x9888 ),0x098a8000 }, + {_MMIO (0x9888 ),0x0b8a8000 }, + {_MMIO (0x9888 ),0x258b8009 }, + {_MMIO (0x9888 ),0x278b002a }, + {_MMIO (0x9888 ),0x238b2a80 }, + {_MMIO (0x9888 ),0x198c4000 }, + {_MMIO (0x9888 ),0x1b8c0015 }, + {_MMIO (0x9888 ),0x0d8c4000 }, + {_MMIO (0x9888 ),0x0d8da000 }, + {_MMIO (0x9888 ),0x0f8da000 }, + {_MMIO (0x9888 ),0x078d2000 }, + {_MMIO (0x9888 ),0x2185800a }, + {_MMIO (0x9888 ),0x2385002a }, + {_MMIO (0x9888 ),0x1f85aa00 }, + {_MMIO (0x9888 ),0x1b830154 }, + {_MMIO (0x9888 ),0x03834000 }, + {_MMIO (0x9888 ),0x05834000 }, + {_MMIO (0x9888 ),0x07834000 }, + {_MMIO (0x9888 ),0x09834000 }, + {_MMIO (0x9888 ),0x0b834000 }, + {_MMIO (0x9888 ),0x0d834000 }, + {_MMIO (0x9888 ),0x0d84c000 }, + {_MMIO (0x9888 ),0x0f84c000 }, + {_MMIO (0x9888 ),0x01848000 }, + {_MMIO (0x9888 ),0x0384c000 }, + {_MMIO (0x9888 ),0x0584c000 }, + {_MMIO (0x9888 ),0x07844000 }, + {_MMIO (0x9888 ),0x1d80c000 }, + {_MMIO (0x9888 ),0x1f80c000 }, + {_MMIO (0x9888 ),0x11808000 }, + {_MMIO (0x9888 ),0x1380c000 }, + {_MMIO (0x9888 ),0x1580c000 }, + {_MMIO (0x9888 ),0x17804000 }, + {_MMIO (0x9888 ),0x53800000 }, + {_MMIO (0x9888 ),0x45800800 }, + {_MMIO (0x9888 ),0x47800842 }, + {_MMIO (0x9888 ),0x21800000 }, + {_MMIO (0x9888 ),0x31800000 }, + {_MMIO (0x9888 ),0x4d800000 }, + {_MMIO (0x9888 ),0x3f801084 }, + {_MMIO (0x9888 ),0x4f800000 }, + {_MMIO (0x9888 ),0x41800044 }, +}; + +static const struct i915_oa_reg * +get_l3_4_mux_config (struct drm_i915_private *dev_priv, +int *len) +{ + *len =ARRAY_SIZE (mux_config_l3_4); +return mux_config_l3_4; +} + +static const struct i915_oa_reg b_counter_config_rasterizer_and_pixel_backend[] = { + {_MMIO (0x2740 ),0x00000000 }, + {_MMIO (0x2744 ),0x00800000 }, + {_MMIO (0x2710 ),0x00000000 }, + {_MMIO (0x2714 ),0xf0800000 }, + {_MMIO (0x2720 ),0x00000000 }, + {_MMIO (0x2724 ),0x30800000 }, + {_MMIO (0x2770 ),0x00006000 }, + {_MMIO (0x2774 ),0x0000f3ff }, + {_MMIO (0x2778 ),0x00001800 }, + {_MMIO (0x277c ),0x0000fcff }, + {_MMIO (0x2780 ),0x00000600 }, + {_MMIO (0x2784 ),0x0000ff3f }, + {_MMIO (0x2788 ),0x00000180 }, + {_MMIO (0x278c ),0x0000ffcf }, + {_MMIO (0x2790 ),0x00000060 }, + {_MMIO (0x2794 ),0x0000fff3 }, + {_MMIO (0x2798 ),0x00000018 }, + {_MMIO (0x279c ),0x0000fffc }, +}; + +static const struct i915_oa_reg flex_eu_config_rasterizer_and_pixel_backend[] = { + {_MMIO (0xe458 ),0x00005004 }, + {_MMIO (0xe558 ),0x00010003 }, + {_MMIO (0xe658 ),0x00012011 }, + {_MMIO (0xe758 ),0x00015014 }, + {_MMIO (0xe45c ),0x00051050 }, + {_MMIO (0xe55c ),0x00053052 }, + {_MMIO (0xe65c ),0x00055054 }, +}; + +static const struct i915_oa_reg mux_config_rasterizer_and_pixel_backend[] = { + {_MMIO (0x9888 ),0x143b000e }, + {_MMIO (0x9888 ),0x043c55c0 }, + {_MMIO (0x9888 ),0x0a1e0280 }, + {_MMIO (0x9888 ),0x0c1e0408 }, + {_MMIO (0x9888 ),0x10390000 }, + {_MMIO (0x9888 ),0x12397a1f }, + {_MMIO (0x9888 ),0x14bb000e }, + {_MMIO (0x9888 ),0x04bc5000 }, + {_MMIO (0x9888 ),0x0a9e0296 }, + {_MMIO (0x9888 ),0x0c9e0008 }, + {_MMIO (0x9888 ),0x10b90000 }, + {_MMIO (0x9888 ),0x12b97a1f }, + {_MMIO (0x9888 ),0x063b0042 }, + {_MMIO (0x9888 ),0x103b0000 }, + {_MMIO (0x9888 ),0x083c0000 }, + {_MMIO (0x9888 ),0x0a3e0040 }, + {_MMIO (0x9888 ),0x043f8000 }, + {_MMIO (0x9888 ),0x02594000 }, + {_MMIO (0x9888 ),0x045a8000 }, + {_MMIO (0x9888 ),0x0c1c0400 }, + {_MMIO (0x9888 ),0x041d8000 }, + {_MMIO (0x9888 ),0x081e02c0 }, + {_MMIO (0x9888 ),0x0e1e0000 }, + {_MMIO (0x9888 ),0x0c1fa800 }, + {_MMIO (0x9888 ),0x0e1f0260 }, + {_MMIO (0x9888 ),0x101f0014 }, + {_MMIO (0x9888 ),0x003905e0 }, + {_MMIO (0x9888 ),0x06390bc0 }, + {_MMIO (0x9888 ),0x02390018 }, + {_MMIO (0x9888 ),0x04394000 }, + {_MMIO (0x9888 ),0x04bb0042 }, + {_MMIO (0x9888 ),0x10bb0000 }, + {_MMIO (0x9888 ),0x02bc05c0 }, + {_MMIO (0x9888 ),0x08bc0000 }, + {_MMIO (0x9888 ),0x0abe0004 }, + {_MMIO (0x9888 ),0x02bf8000 }, + {_MMIO (0x9888 ),0x02d91000 }, + {_MMIO (0x9888 ),0x02da8000 }, + {_MMIO (0x9888 ),0x089c8000 }, + {_MMIO (0x9888 ),0x029d8000 }, + {_MMIO (0x9888 ),0x089e8000 }, + {_MMIO (0x9888 ),0x0e9e0000 }, + {_MMIO (0x9888 ),0x0e9fa806 }, + {_MMIO (0x9888 ),0x109f0142 }, + {_MMIO (0x9888 ),0x08b90617 }, + {_MMIO (0x9888 ),0x0ab90be0 }, + {_MMIO (0x9888 ),0x02b94000 }, + {_MMIO (0x9888 ),0x0d88f000 }, + {_MMIO (0x9888 ),0x0f88000c }, + {_MMIO (0x9888 ),0x07888000 }, + {_MMIO (0x9888 ),0x09888000 }, + {_MMIO (0x9888 ),0x018a8000 }, + {_MMIO (0x9888 ),0x0f8a8000 }, + {_MMIO (0x9888 ),0x1b8a2800 }, + {_MMIO (0x9888 ),0x038a8000 }, + {_MMIO (0x9888 ),0x058a8000 }, + {_MMIO (0x9888 ),0x0b8a8000 }, + {_MMIO (0x9888 ),0x0d8a8000 }, + {_MMIO (0x9888 ),0x238b52a0 }, + {_MMIO (0x9888 ),0x258b6a95 }, + {_MMIO (0x9888 ),0x278b0029 }, + {_MMIO (0x9888 ),0x178c2000 }, + {_MMIO (0x9888 ),0x198c1500 }, + {_MMIO (0x9888 ),0x1b8c0014 }, + {_MMIO (0x9888 ),0x078c4000 }, + {_MMIO (0x9888 ),0x098c4000 }, + {_MMIO (0x9888 ),0x098da000 }, + {_MMIO (0x9888 ),0x0b8da000 }, + {_MMIO (0x9888 ),0x0f8da000 }, + {_MMIO (0x9888 ),0x038d8000 }, + {_MMIO (0x9888 ),0x058d2000 }, + {_MMIO (0x9888 ),0x1f85aa80 }, + {_MMIO (0x9888 ),0x2185aaaa }, + {_MMIO (0x9888 ),0x2385002a }, + {_MMIO (0x9888 ),0x01834000 }, + {_MMIO (0x9888 ),0x0f834000 }, + {_MMIO (0x9888 ),0x19835400 }, + {_MMIO (0x9888 ),0x1b830155 }, + {_MMIO (0x9888 ),0x03834000 }, + {_MMIO (0x9888 ),0x05834000 }, + {_MMIO (0x9888 ),0x07834000 }, + {_MMIO (0x9888 ),0x09834000 }, + {_MMIO (0x9888 ),0x0b834000 }, + {_MMIO (0x9888 ),0x0d834000 }, + {_MMIO (0x9888 ),0x0184c000 }, + {_MMIO (0x9888 ),0x0784c000 }, + {_MMIO (0x9888 ),0x0984c000 }, + {_MMIO (0x9888 ),0x0b84c000 }, + {_MMIO (0x9888 ),0x0d84c000 }, + {_MMIO (0x9888 ),0x0f84c000 }, + {_MMIO (0x9888 ),0x0384c000 }, + {_MMIO (0x9888 ),0x0584c000 }, + {_MMIO (0x9888 ),0x1180c000 }, + {_MMIO (0x9888 ),0x1780c000 }, + {_MMIO (0x9888 ),0x1980c000 }, + {_MMIO (0x9888 ),0x1b80c000 }, + {_MMIO (0x9888 ),0x1d80c000 }, + {_MMIO (0x9888 ),0x1f80c000 }, + {_MMIO (0x9888 ),0x1380c000 }, + {_MMIO (0x9888 ),0x1580c000 }, + {_MMIO (0x9888 ),0x4d800444 }, + {_MMIO (0x9888 ),0x3d800000 }, + {_MMIO (0x9888 ),0x4f804000 }, + {_MMIO (0x9888 ),0x43801080 }, + {_MMIO (0x9888 ),0x51800000 }, + {_MMIO (0x9888 ),0x45800084 }, + {_MMIO (0x9888 ),0x53800044 }, + {_MMIO (0x9888 ),0x47801080 }, + {_MMIO (0x9888 ),0x21800000 }, + {_MMIO (0x9888 ),0x31800000 }, + {_MMIO (0x9888 ),0x3f800000 }, + {_MMIO (0x9888 ),0x41800840 }, +}; + +static const struct i915_oa_reg * +get_rasterizer_and_pixel_backend_mux_config (struct drm_i915_private *dev_priv, +int *len) +{ + *len =ARRAY_SIZE (mux_config_rasterizer_and_pixel_backend); +return mux_config_rasterizer_and_pixel_backend; +} + +static const struct i915_oa_reg b_counter_config_sampler_1[] = { + {_MMIO (0x2740 ),0x00000000 }, + {_MMIO (0x2744 ),0x00800000 }, + {_MMIO (0x2710 ),0x00000000 }, + {_MMIO (0x2714 ),0x70800000 }, + {_MMIO (0x2720 ),0x00000000 }, + {_MMIO (0x2724 ),0x00800000 }, + {_MMIO (0x2770 ),0x0000c000 }, + {_MMIO (0x2774 ),0x0000e7ff }, + {_MMIO (0x2778 ),0x00003000 }, + {_MMIO (0x277c ),0x0000f9ff }, + {_MMIO (0x2780 ),0x00000c00 }, + {_MMIO (0x2784 ),0x0000fe7f }, +}; + +static const struct i915_oa_reg flex_eu_config_sampler_1[] = { + {_MMIO (0xe458 ),0x00005004 }, + {_MMIO (0xe558 ),0x00010003 }, + {_MMIO (0xe658 ),0x00012011 }, + {_MMIO (0xe758 ),0x00015014 }, + {_MMIO (0xe45c ),0x00051050 }, + {_MMIO (0xe55c ),0x00053052 }, + {_MMIO (0xe65c ),0x00055054 }, +}; + +static const struct i915_oa_reg mux_config_sampler_1[] = { + {_MMIO (0x9888 ),0x18921400 }, + {_MMIO (0x9888 ),0x149500ab }, + {_MMIO (0x9888 ),0x18b21400 }, + {_MMIO (0x9888 ),0x14b500ab }, + {_MMIO (0x9888 ),0x18d21400 }, + {_MMIO (0x9888 ),0x14d500ab }, + {_MMIO (0x9888 ),0x0cdc8000 }, + {_MMIO (0x9888 ),0x0edc4000 }, + {_MMIO (0x9888 ),0x02dcc000 }, + {_MMIO (0x9888 ),0x04dcc000 }, + {_MMIO (0x9888 ),0x1abd00a0 }, + {_MMIO (0x9888 ),0x0abd8000 }, + {_MMIO (0x9888 ),0x0cd88000 }, + {_MMIO (0x9888 ),0x0ed84000 }, + {_MMIO (0x9888 ),0x04d88000 }, + {_MMIO (0x9888 ),0x1adb0050 }, + {_MMIO (0x9888 ),0x04db8000 }, + {_MMIO (0x9888 ),0x06db8000 }, + {_MMIO (0x9888 ),0x08db8000 }, + {_MMIO (0x9888 ),0x0adb4000 }, + {_MMIO (0x9888 ),0x109f02a0 }, + {_MMIO (0x9888 ),0x0c9fa000 }, + {_MMIO (0x9888 ),0x0e9f00aa }, + {_MMIO (0x9888 ),0x18b82500 }, + {_MMIO (0x9888 ),0x02b88000 }, + {_MMIO (0x9888 ),0x04b84000 }, + {_MMIO (0x9888 ),0x06b84000 }, + {_MMIO (0x9888 ),0x08b84000 }, + {_MMIO (0x9888 ),0x0ab84000 }, + {_MMIO (0x9888 ),0x0cb88000 }, + {_MMIO (0x9888 ),0x0cb98000 }, + {_MMIO (0x9888 ),0x0eb9a000 }, + {_MMIO (0x9888 ),0x00b98000 }, + {_MMIO (0x9888 ),0x02b9a000 }, + {_MMIO (0x9888 ),0x04b9a000 }, + {_MMIO (0x9888 ),0x06b92000 }, + {_MMIO (0x9888 ),0x1aba0200 }, + {_MMIO (0x9888 ),0x02ba8000 }, + {_MMIO (0x9888 ),0x0cba8000 }, + {_MMIO (0x9888 ),0x04908000 }, + {_MMIO (0x9888 ),0x04918000 }, + {_MMIO (0x9888 ),0x04927300 }, + {_MMIO (0x9888 ),0x10920000 }, + {_MMIO (0x9888 ),0x1893000a }, + {_MMIO (0x9888 ),0x0a934000 }, + {_MMIO (0x9888 ),0x0a946000 }, + {_MMIO (0x9888 ),0x0c959000 }, + {_MMIO (0x9888 ),0x0e950098 }, + {_MMIO (0x9888 ),0x10950000 }, + {_MMIO (0x9888 ),0x04b04000 }, + {_MMIO (0x9888 ),0x04b14000 }, + {_MMIO (0x9888 ),0x04b20073 }, + {_MMIO (0x9888 ),0x10b20000 }, + {_MMIO (0x9888 ),0x04b38000 }, + {_MMIO (0x9888 ),0x06b38000 }, + {_MMIO (0x9888 ),0x08b34000 }, + {_MMIO (0x9888 ),0x04b4c000 }, + {_MMIO (0x9888 ),0x02b59890 }, + {_MMIO (0x9888 ),0x10b50000 }, + {_MMIO (0x9888 ),0x06d04000 }, + {_MMIO (0x9888 ),0x06d14000 }, + {_MMIO (0x9888 ),0x06d20073 }, + {_MMIO (0x9888 ),0x10d20000 }, + {_MMIO (0x9888 ),0x18d30020 }, + {_MMIO (0x9888 ),0x02d38000 }, + {_MMIO (0x9888 ),0x0cd34000 }, + {_MMIO (0x9888 ),0x0ad48000 }, + {_MMIO (0x9888 ),0x04d42000 }, + {_MMIO (0x9888 ),0x0ed59000 }, + {_MMIO (0x9888 ),0x00d59800 }, + {_MMIO (0x9888 ),0x10d50000 }, + {_MMIO (0x9888 ),0x0f88000e }, + {_MMIO (0x9888 ),0x03888000 }, + {_MMIO (0x9888 ),0x05888000 }, + {_MMIO (0x9888 ),0x07888000 }, + {_MMIO (0x9888 ),0x09888000 }, + {_MMIO (0x9888 ),0x0b888000 }, + {_MMIO (0x9888 ),0x0d880400 }, + {_MMIO (0x9888 ),0x278b002a }, + {_MMIO (0x9888 ),0x238b5500 }, + {_MMIO (0x9888 ),0x258b000a }, + {_MMIO (0x9888 ),0x1b8c0015 }, + {_MMIO (0x9888 ),0x038c4000 }, + {_MMIO (0x9888 ),0x058c4000 }, + {_MMIO (0x9888 ),0x078c4000 }, + {_MMIO (0x9888 ),0x098c4000 }, + {_MMIO (0x9888 ),0x0b8c4000 }, + {_MMIO (0x9888 ),0x0d8c4000 }, + {_MMIO (0x9888 ),0x0d8d8000 }, + {_MMIO (0x9888 ),0x0f8da000 }, + {_MMIO (0x9888 ),0x018d8000 }, + {_MMIO (0x9888 ),0x038da000 }, + {_MMIO (0x9888 ),0x058da000 }, + {_MMIO (0x9888 ),0x078d2000 }, + {_MMIO (0x9888 ),0x2385002a }, + {_MMIO (0x9888 ),0x1f85aa00 }, + {_MMIO (0x9888 ),0x2185000a }, + {_MMIO (0x9888 ),0x1b830150 }, + {_MMIO (0x9888 ),0x03834000 }, + {_MMIO (0x9888 ),0x05834000 }, + {_MMIO (0x9888 ),0x07834000 }, + {_MMIO (0x9888 ),0x09834000 }, + {_MMIO (0x9888 ),0x0b834000 }, + {_MMIO (0x9888 ),0x0d834000 }, + {_MMIO (0x9888 ),0x0d848000 }, + {_MMIO (0x9888 ),0x0f84c000 }, + {_MMIO (0x9888 ),0x01848000 }, + {_MMIO (0x9888 ),0x0384c000 }, + {_MMIO (0x9888 ),0x0584c000 }, + {_MMIO (0x9888 ),0x07844000 }, + {_MMIO (0x9888 ),0x1d808000 }, + {_MMIO (0x9888 ),0x1f80c000 }, + {_MMIO (0x9888 ),0x11808000 }, + {_MMIO (0x9888 ),0x1380c000 }, + {_MMIO (0x9888 ),0x1580c000 }, + {_MMIO (0x9888 ),0x17804000 }, + {_MMIO (0x9888 ),0x53800000 }, + {_MMIO (0x9888 ),0x47801021 }, + {_MMIO (0x9888 ),0x21800000 }, + {_MMIO (0x9888 ),0x31800000 }, + {_MMIO (0x9888 ),0x4d800000 }, + {_MMIO (0x9888 ),0x3f800c64 }, + {_MMIO (0x9888 ),0x4f800000 }, + {_MMIO (0x9888 ),0x41800c02 }, +}; + +static const struct i915_oa_reg * +get_sampler_1_mux_config (struct drm_i915_private *dev_priv, +int *len) +{ + *len =ARRAY_SIZE (mux_config_sampler_1); +return mux_config_sampler_1; +} + +static const struct i915_oa_reg b_counter_config_sampler_2[] = { + {_MMIO (0x2740 ),0x00000000 }, + {_MMIO (0x2744 ),0x00800000 }, + {_MMIO (0x2710 ),0x00000000 }, + {_MMIO (0x2714 ),0x70800000 }, + {_MMIO (0x2720 ),0x00000000 }, + {_MMIO (0x2724 ),0x00800000 }, + {_MMIO (0x2770 ),0x0000c000 }, + {_MMIO (0x2774 ),0x0000e7ff }, + {_MMIO (0x2778 ),0x00003000 }, + {_MMIO (0x277c ),0x0000f9ff }, + {_MMIO (0x2780 ),0x00000c00 }, + {_MMIO (0x2784 ),0x0000fe7f }, +}; + +static const struct i915_oa_reg flex_eu_config_sampler_2[] = { + {_MMIO (0xe458 ),0x00005004 }, + {_MMIO (0xe558 ),0x00010003 }, + {_MMIO (0xe658 ),0x00012011 }, + {_MMIO (0xe758 ),0x00015014 }, + {_MMIO (0xe45c ),0x00051050 }, + {_MMIO (0xe55c ),0x00053052 }, + {_MMIO (0xe65c ),0x00055054 }, +}; + +static const struct i915_oa_reg mux_config_sampler_2[] = { + {_MMIO (0x9888 ),0x18121400 }, + {_MMIO (0x9888 ),0x141500ab }, + {_MMIO (0x9888 ),0x18321400 }, + {_MMIO (0x9888 ),0x143500ab }, + {_MMIO (0x9888 ),0x18521400 }, + {_MMIO (0x9888 ),0x145500ab }, + {_MMIO (0x9888 ),0x0c5c8000 }, + {_MMIO (0x9888 ),0x0e5c4000 }, + {_MMIO (0x9888 ),0x025cc000 }, + {_MMIO (0x9888 ),0x045cc000 }, + {_MMIO (0x9888 ),0x1a3d00a0 }, + {_MMIO (0x9888 ),0x0a3d8000 }, + {_MMIO (0x9888 ),0x0c588000 }, + {_MMIO (0x9888 ),0x0e584000 }, + {_MMIO (0x9888 ),0x04588000 }, + {_MMIO (0x9888 ),0x1a5b0050 }, + {_MMIO (0x9888 ),0x045b8000 }, + {_MMIO (0x9888 ),0x065b8000 }, + {_MMIO (0x9888 ),0x085b8000 }, + {_MMIO (0x9888 ),0x0a5b4000 }, + {_MMIO (0x9888 ),0x101f02a0 }, + {_MMIO (0x9888 ),0x0c1fa000 }, + {_MMIO (0x9888 ),0x0e1f00aa }, + {_MMIO (0x9888 ),0x18382500 }, + {_MMIO (0x9888 ),0x02388000 }, + {_MMIO (0x9888 ),0x04384000 }, + {_MMIO (0x9888 ),0x06384000 }, + {_MMIO (0x9888 ),0x08384000 }, + {_MMIO (0x9888 ),0x0a384000 }, + {_MMIO (0x9888 ),0x0c388000 }, + {_MMIO (0x9888 ),0x0c398000 }, + {_MMIO (0x9888 ),0x0e39a000 }, + {_MMIO (0x9888 ),0x00398000 }, + {_MMIO (0x9888 ),0x0239a000 }, + {_MMIO (0x9888 ),0x0439a000 }, + {_MMIO (0x9888 ),0x06392000 }, + {_MMIO (0x9888 ),0x1a3a0200 }, + {_MMIO (0x9888 ),0x023a8000 }, + {_MMIO (0x9888 ),0x0c3a8000 }, + {_MMIO (0x9888 ),0x04108000 }, + {_MMIO (0x9888 ),0x04118000 }, + {_MMIO (0x9888 ),0x04127300 }, + {_MMIO (0x9888 ),0x10120000 }, + {_MMIO (0x9888 ),0x1813000a }, + {_MMIO (0x9888 ),0x0a134000 }, + {_MMIO (0x9888 ),0x0a146000 }, + {_MMIO (0x9888 ),0x0c159000 }, + {_MMIO (0x9888 ),0x0e150098 }, + {_MMIO (0x9888 ),0x10150000 }, + {_MMIO (0x9888 ),0x04304000 }, + {_MMIO (0x9888 ),0x04314000 }, + {_MMIO (0x9888 ),0x04320073 }, + {_MMIO (0x9888 ),0x10320000 }, + {_MMIO (0x9888 ),0x04338000 }, + {_MMIO (0x9888 ),0x06338000 }, + {_MMIO (0x9888 ),0x08334000 }, + {_MMIO (0x9888 ),0x0434c000 }, + {_MMIO (0x9888 ),0x02359890 }, + {_MMIO (0x9888 ),0x10350000 }, + {_MMIO (0x9888 ),0x06504000 }, + {_MMIO (0x9888 ),0x06514000 }, + {_MMIO (0x9888 ),0x06520073 }, + {_MMIO (0x9888 ),0x10520000 }, + {_MMIO (0x9888 ),0x18530020 }, + {_MMIO (0x9888 ),0x02538000 }, + {_MMIO (0x9888 ),0x0c534000 }, + {_MMIO (0x9888 ),0x0a548000 }, + {_MMIO (0x9888 ),0x04542000 }, + {_MMIO (0x9888 ),0x0e559000 }, + {_MMIO (0x9888 ),0x00559800 }, + {_MMIO (0x9888 ),0x10550000 }, + {_MMIO (0x9888 ),0x1b8aa000 }, + {_MMIO (0x9888 ),0x1d8a0002 }, + {_MMIO (0x9888 ),0x038a8000 }, + {_MMIO (0x9888 ),0x058a8000 }, + {_MMIO (0x9888 ),0x078a8000 }, + {_MMIO (0x9888 ),0x098a8000 }, + {_MMIO (0x9888 ),0x0b8a8000 }, + {_MMIO (0x9888 ),0x0d8a8000 }, + {_MMIO (0x9888 ),0x278b0015 }, + {_MMIO (0x9888 ),0x238b2a80 }, + {_MMIO (0x9888 ),0x258b0005 }, + {_MMIO (0x9888 ),0x2385002a }, + {_MMIO (0x9888 ),0x1f85aa00 }, + {_MMIO (0x9888 ),0x2185000a }, + {_MMIO (0x9888 ),0x1b830150 }, + {_MMIO (0x9888 ),0x03834000 }, + {_MMIO (0x9888 ),0x05834000 }, + {_MMIO (0x9888 ),0x07834000 }, + {_MMIO (0x9888 ),0x09834000 }, + {_MMIO (0x9888 ),0x0b834000 }, + {_MMIO (0x9888 ),0x0d834000 }, + {_MMIO (0x9888 ),0x0d848000 }, + {_MMIO (0x9888 ),0x0f84c000 }, + {_MMIO (0x9888 ),0x01848000 }, + {_MMIO (0x9888 ),0x0384c000 }, + {_MMIO (0x9888 ),0x0584c000 }, + {_MMIO (0x9888 ),0x07844000 }, + {_MMIO (0x9888 ),0x1d808000 }, + {_MMIO (0x9888 ),0x1f80c000 }, + {_MMIO (0x9888 ),0x11808000 }, + {_MMIO (0x9888 ),0x1380c000 }, + {_MMIO (0x9888 ),0x1580c000 }, + {_MMIO (0x9888 ),0x17804000 }, + {_MMIO (0x9888 ),0x53800000 }, + {_MMIO (0x9888 ),0x47801021 }, + {_MMIO (0x9888 ),0x21800000 }, + {_MMIO (0x9888 ),0x31800000 }, + {_MMIO (0x9888 ),0x4d800000 }, + {_MMIO (0x9888 ),0x3f800c64 }, + {_MMIO (0x9888 ),0x4f800000 }, + {_MMIO (0x9888 ),0x41800c02 }, +}; + +static const struct i915_oa_reg * +get_sampler_2_mux_config (struct drm_i915_private *dev_priv, +int *len) +{ + *len =ARRAY_SIZE (mux_config_sampler_2); +return mux_config_sampler_2; +} + +static const struct i915_oa_reg b_counter_config_tdl_1[] = { + {_MMIO (0x2740 ),0x00000000 }, + {_MMIO (0x2744 ),0x00800000 }, + {_MMIO (0x2710 ),0x00000000 }, + {_MMIO (0x2714 ),0xf0800000 }, + {_MMIO (0x2720 ),0x00000000 }, + {_MMIO (0x2724 ),0x30800000 }, + {_MMIO (0x2770 ),0x00000002 }, + {_MMIO (0x2774 ),0x0000fdff }, + {_MMIO (0x2778 ),0x00000000 }, + {_MMIO (0x277c ),0x0000fe7f }, + {_MMIO (0x2780 ),0x00000002 }, + {_MMIO (0x2784 ),0x0000ffbf }, + {_MMIO (0x2788 ),0x00000000 }, + {_MMIO (0x278c ),0x0000ffcf }, + {_MMIO (0x2790 ),0x00000002 }, + {_MMIO (0x2794 ),0x0000fff7 }, + {_MMIO (0x2798 ),0x00000000 }, + {_MMIO (0x279c ),0x0000fff9 }, +}; + +static const struct i915_oa_reg flex_eu_config_tdl_1[] = { + {_MMIO (0xe458 ),0x00005004 }, + {_MMIO (0xe558 ),0x00010003 }, + {_MMIO (0xe658 ),0x00012011 }, + {_MMIO (0xe758 ),0x00015014 }, + {_MMIO (0xe45c ),0x00051050 }, + {_MMIO (0xe55c ),0x00053052 }, + {_MMIO (0xe65c ),0x00055054 }, +}; + +static const struct i915_oa_reg mux_config_tdl_1[] = { + {_MMIO (0x9888 ),0x16154d60 }, + {_MMIO (0x9888 ),0x16352e60 }, + {_MMIO (0x9888 ),0x16554d60 }, + {_MMIO (0x9888 ),0x16950000 }, + {_MMIO (0x9888 ),0x16b50000 }, + {_MMIO (0x9888 ),0x16d50000 }, + {_MMIO (0x9888 ),0x005c8000 }, + {_MMIO (0x9888 ),0x045cc000 }, + {_MMIO (0x9888 ),0x065c4000 }, + {_MMIO (0x9888 ),0x083d8000 }, + {_MMIO (0x9888 ),0x0a3d8000 }, + {_MMIO (0x9888 ),0x0458c000 }, + {_MMIO (0x9888 ),0x025b8000 }, + {_MMIO (0x9888 ),0x085b4000 }, + {_MMIO (0x9888 ),0x0a5b4000 }, + {_MMIO (0x9888 ),0x0c5b8000 }, + {_MMIO (0x9888 ),0x0c1fa000 }, + {_MMIO (0x9888 ),0x0e1f00aa }, + {_MMIO (0x9888 ),0x02384000 }, + {_MMIO (0x9888 ),0x04388000 }, + {_MMIO (0x9888 ),0x06388000 }, + {_MMIO (0x9888 ),0x08384000 }, + {_MMIO (0x9888 ),0x0a384000 }, + {_MMIO (0x9888 ),0x0c384000 }, + {_MMIO (0x9888 ),0x00398000 }, + {_MMIO (0x9888 ),0x0239a000 }, + {_MMIO (0x9888 ),0x0439a000 }, + {_MMIO (0x9888 ),0x06392000 }, + {_MMIO (0x9888 ),0x043a8000 }, + {_MMIO (0x9888 ),0x063a8000 }, + {_MMIO (0x9888 ),0x08138000 }, + {_MMIO (0x9888 ),0x0a138000 }, + {_MMIO (0x9888 ),0x06143000 }, + {_MMIO (0x9888 ),0x0415cfc7 }, + {_MMIO (0x9888 ),0x10150000 }, + {_MMIO (0x9888 ),0x02338000 }, + {_MMIO (0x9888 ),0x0c338000 }, + {_MMIO (0x9888 ),0x04342000 }, + {_MMIO (0x9888 ),0x06344000 }, + {_MMIO (0x9888 ),0x0035c700 }, + {_MMIO (0x9888 ),0x063500cf }, + {_MMIO (0x9888 ),0x10350000 }, + {_MMIO (0x9888 ),0x04538000 }, + {_MMIO (0x9888 ),0x06538000 }, + {_MMIO (0x9888 ),0x0454c000 }, + {_MMIO (0x9888 ),0x0255cfc7 }, + {_MMIO (0x9888 ),0x10550000 }, + {_MMIO (0x9888 ),0x06dc8000 }, + {_MMIO (0x9888 ),0x08dc4000 }, + {_MMIO (0x9888 ),0x0cdcc000 }, + {_MMIO (0x9888 ),0x0edcc000 }, + {_MMIO (0x9888 ),0x1abd00a8 }, + {_MMIO (0x9888 ),0x0cd8c000 }, + {_MMIO (0x9888 ),0x0ed84000 }, + {_MMIO (0x9888 ),0x0edb8000 }, + {_MMIO (0x9888 ),0x18db0800 }, + {_MMIO (0x9888 ),0x1adb0254 },
Lines of this file are not shown. View the raw message.
drivers/gpu/drm/i915/i915_oa_bxt.c +2313 -10
2,323 lines not on this page · raw message
drivers/gpu/drm/i915/i915_oa_chv.c +2515 -10
2,525 lines not on this page · raw message
drivers/gpu/drm/i915/i915_oa_hsw.c +53 -5
58 lines not on this page · raw message
drivers/gpu/drm/i915/i915_oa_sklgt2.c +3115 -41
3,156 lines not on this page · raw message
drivers/gpu/drm/i915/i915_oa_sklgt3.c +2661 -41
2,702 lines not on this page · raw message
drivers/gpu/drm/i915/i915_oa_sklgt4.c +2704 -41
2,745 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Signature · 6 lines
2.12.0 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Part of the diff is not shown here. View the raw message.
PATCH v4 14/15 drm/i915/perf: per-gen timebase for checking sample freq
In reply to Robert BraggAn oa_exponent_to_ns() utility and per-gen timebase constants where recently removed when updating the tail pointer race condition WA, and this restores those so we can update the _PROP_OA_EXPONENT validation done in read_properties_unlocked() to not assume we have a 12.5MHz timebase as we did for Haswell.
Accordingly the oa_sample_rate_hard_limit value that's referenced by proc_dointvec_minmax defining the absolute limit for the OA sampling frequency is now initialized to (timestamp_frequency / 2) instead of the 6.25MHz constant for Haswell.
v2:
Specify frequency of 19.2MHz for BXT (Ville) Initialize oa_sample_rate_hard_limit per-gen too (Lionel)
--- drivers/gpu/drm/i915/i915_drv.h | 1 + drivers/gpu/drm/i915/i915_perf.c | 37 ++++++++++++++++++++++++++----------- 2 files changed, 27 insertions(+), 11 deletions(-)
drivers/gpu/drm/i915/i915_drv.h +1 -0
@@ -2457,6 +2457,7 @@ struct drm_i915_private {bool periodic;int period_exponent; +int timestamp_frequency;int metrics_set;
drivers/gpu/drm/i915/i915_perf.c +26 -11
@@ -288,10 +288,12 @@ static u32 i915_perf_stream_paranoid = true;/* For sysctl proc_dointvec_minmax of i915_oa_max_sample_rate * -* 160ns is the smallest sampling period we can theoretically program the OA -* unit with on Haswell, corresponding to 6.25MHz. +* The highest sampling frequency we can theoretically program the OA unit +* with is always half the timestamp frequency: E.g. 6.25Mhz for Haswell. +* +* Initialized just before we register the sysctl parameter. */ -static int oa_sample_rate_hard_limit =6250000 ; +static int oa_sample_rate_hard_limit;/* Theoretically we can program the OA unit to sample every 160ns but don't * allow that by default unless root...
@@ -2560,6 +2562,12 @@ i915_perf_open_ioctl_locked(struct drm_i915_private *dev_priv,return ret; } +static u64 oa_exponent_to_ns (struct drm_i915_private *dev_priv,int exponent) +{ +return div_u64 (1000000000ULL * (2ULL << exponent), + dev_priv->perf.oa.timestamp_frequency); +} +/** * read_properties_unlocked - validate + copy userspace stream open properties * @dev_priv: i915 device instance
@@ -2656,16 +2664,13 @@ static int read_properties_unlocked(struct drm_i915_private *dev_priv, }/* Theoretically we can program the OA unit to sample -* every 160ns but don't allow that by default unless -* root. -* -* On Haswell the period is derived from the exponent -* as: -* -* period = 80ns * 2^(exponent + 1) +* e.g. every 160ns for HSW, 167ns for BDW/SKL or 104ns +* for BXT. We don't allow such high sampling +* frequencies by default unless root. */ +BUILD_BUG_ON (sizeof (oa_period) !=8 ); - oa_period =80ull * (2ull << value); + oa_period =oa_exponent_to_ns (dev_priv, value);/* This check is primarily to ensure that oa_period <= * UINT32_MAX (before passing to do_div which only
@@ -2921,6 +2926,8 @@ void i915_perf_init(struct drm_i915_private *dev_priv) dev_priv->perf.oa.ops.oa_hw_tail_read = gen7_oa_hw_tail_read; + dev_priv->perf.oa.timestamp_frequency =12500000 ; + dev_priv->perf.oa.oa_formats = hsw_oa_formats; dev_priv->perf.oa.n_builtin_sets =
@@ -2934,6 +2941,8 @@ void i915_perf_init(struct drm_i915_private *dev_priv)*/ if (IS_GEN8 (dev_priv)) { + dev_priv->perf.oa.timestamp_frequency =12500000 ; + dev_priv->perf.oa.ctx_oactxctrl_offset =0x120 ; dev_priv->perf.oa.ctx_flexeu0_offset =0x2ce ; dev_priv->perf.oa.gen8_valid_ctx_bit = (1 <<25 );
@@ -2950,6 +2959,8 @@ void i915_perf_init(struct drm_i915_private *dev_priv) i915_oa_select_metric_set_chv; } }else if (IS_GEN9 (dev_priv)) { + dev_priv->perf.oa.timestamp_frequency =12000000 ; + dev_priv->perf.oa.ctx_oactxctrl_offset =0x128 ; dev_priv->perf.oa.ctx_flexeu0_offset =0x3de ; dev_priv->perf.oa.gen8_valid_ctx_bit = (1 <<16 );
@@ -2970,6 +2981,8 @@ void i915_perf_init(struct drm_i915_private *dev_priv) dev_priv->perf.oa.ops.select_metric_set = i915_oa_select_metric_set_sklgt4; }else if (IS_BROXTON (dev_priv)) { + dev_priv->perf.oa.timestamp_frequency =19200000 ; + dev_priv->perf.oa.n_builtin_sets = i915_oa_n_builtin_metric_sets_bxt; dev_priv->perf.oa.ops.select_metric_set =
@@ -3004,6 +3017,8 @@ void i915_perf_init(struct drm_i915_private *dev_priv)spin_lock_init (&dev_priv->perf.hook_lock);spin_lock_init (&dev_priv->perf.oa.oa_buffer.ptr_lock); + oa_sample_rate_hard_limit = + dev_priv->perf.oa.timestamp_frequency /2 ; dev_priv->perf.sysctl_header =register_sysctl_table (dev_root); dev_priv->perf.initialized = true;
Signature · 6 lines
2.12.0 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v4 15/15 drm/i915/perf: remove perf.hook_lock
In reply to Robert BraggIn earlier iterations of the i915-perf driver we had a number of callbacks/hooks from other parts of the i915 driver to e.g. notify us when a legacy context was pinned and these could run asynchronously with respect to the stream file operations and might also run in atomic context.
dev_priv->perf.hook_lock had been for serialising access to state needed within these callbacks, but as the code has evolved some of the hooks have gone away or are implemented to avoid needing to lock any state.
The remaining use of this lock was actually redundant considering how the gen7 oacontrol state used to be updated as part of a context pin hook.
--- drivers/gpu/drm/i915/i915_drv.h | 2 -- drivers/gpu/drm/i915/i915_perf.c | 32 ++++++++++---------------------- 2 files changed, 10 insertions(+), 24 deletions(-)
drivers/gpu/drm/i915/i915_drv.h +0 -2
@@ -2438,8 +2438,6 @@ struct drm_i915_private {struct mutex lock;struct list_head streams; -spinlock_t hook_lock; -struct {struct i915_perf_stream *exclusive_stream;
drivers/gpu/drm/i915/i915_perf.c +10 -22
@@ -1690,9 +1690,17 @@ static void gen8_disable_metric_set(struct drm_i915_private *dev_priv)/* NOP */ } -static void gen7_update_oacontrol_locked (struct drm_i915_private *dev_priv) +static void gen7_oa_enable (struct drm_i915_private *dev_priv) { -lockdep_assert_held (&dev_priv->perf.hook_lock); +/* Reset buf pointers so we don't forward reports from before now. +* +* Think carefully if considering trying to avoid this, since it +* also ensures status flags and the buffer itself are cleared +* in error paths, and we have checks for invalid reports based +* on the assumption that certain fields are written to zeroed +* memory which this helps maintains. +*/ +gen7_init_oa_buffer (dev_priv);if (dev_priv->perf.oa.exclusive_stream->enabled) { struct i915_gem_context *ctx =
@@ -1715,25 +1723,6 @@ static void gen7_update_oacontrol_locked(struct drm_i915_private *dev_priv)I915_WRITE (GEN7_OACONTROL ,0 ); } -static void gen7_oa_enable (struct drm_i915_private *dev_priv) -{ -unsigned long flags; - -/* Reset buf pointers so we don't forward reports from before now. -* -* Think carefully if considering trying to avoid this, since it -* also ensures status flags and the buffer itself are cleared -* in error paths, and we have checks for invalid reports based -* on the assumption that certain fields are written to zeroed -* memory which this helps maintains. -*/ -gen7_init_oa_buffer (dev_priv); - -spin_lock_irqsave (&dev_priv->perf.hook_lock, flags); -gen7_update_oacontrol_locked (dev_priv); -spin_unlock_irqrestore (&dev_priv->perf.hook_lock, flags); -} -static void gen8_oa_enable (struct drm_i915_private *dev_priv) {u32 report_format = dev_priv->perf.oa.oa_buffer.format;
@@ -3014,7 +3003,6 @@ void i915_perf_init(struct drm_i915_private *dev_priv)INIT_LIST_HEAD (&dev_priv->perf.streams);mutex_init (&dev_priv->perf.lock); -spin_lock_init (&dev_priv->perf.hook_lock);spin_lock_init (&dev_priv->perf.oa.oa_buffer.ptr_lock); oa_sample_rate_hard_limit =
Signature · 6 lines
2.12.0 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Patchwork
CI✓ Fi.CI.BAT: success for Enable OA unit for Gen 8 and 9 in i915 perf (rev6)
In reply to Robert Bragg== Series Details ==
== Series Details ==
Series: Enable OA unit for Gen 8 and 9 in i915 perf (rev6)
URL : https://patchwork.freedesktop.org/series/20084/
State : success
== Summary ==
Series 20084v6 Enable OA unit for Gen 8 and 9 in i915 perf
https://patchwork.freedesktop.org/api/1.0/series/20084/revisions/6/mbox/
Test gem_exec_flush:
Subgroup basic-batch-kernel-default-uc:
fail -> PASS (fi-snb-2600) fdo#100007
Test gem_exec_suspend:
Subgroup basic-s4-devices:
dmesg-warn -> PASS (fi-kbl-7560u) fdo#100125fdo#100007 https://bugs.freedesktop.org/show_bug.cgi?id=100007
fdo#100125 https://bugs.freedesktop.org/show_bug.cgi?id=100125
fi-bdw-5557u total:278 pass:267 dwarn:0 dfail:0 fail:0 skip:11 time:428s fi-bdw-gvtdvm total:278 pass:256 dwarn:8 dfail:0 fail:0 skip:14 time:428s fi-bsw-n3050 total:278 pass:242 dwarn:0 dfail:0 fail:0 skip:36 time:590s fi-bxt-t5700 total:278 pass:258 dwarn:0 dfail:0 fail:0 skip:20 time:549s fi-byt-n2820 total:278 pass:250 dwarn:0 dfail:0 fail:0 skip:28 time:479s fi-hsw-4770 total:278 pass:262 dwarn:0 dfail:0 fail:0 skip:16 time:411s fi-hsw-4770r total:278 pass:262 dwarn:0 dfail:0 fail:0 skip:16 time:403s fi-ilk-650 total:278 pass:228 dwarn:0 dfail:0 fail:0 skip:50 time:422s fi-ivb-3520m total:278 pass:260 dwarn:0 dfail:0 fail:0 skip:18 time:492s fi-ivb-3770 total:278 pass:260 dwarn:0 dfail:0 fail:0 skip:18 time:466s fi-kbl-7500u total:278 pass:260 dwarn:0 dfail:0 fail:0 skip:18 time:454s fi-kbl-7560u total:278 pass:268 dwarn:0 dfail:0 fail:0 skip:10 time:564s fi-skl-6260u total:278 pass:268 dwarn:0 dfail:0 fail:0 skip:10 time:452s fi-skl-6700hq total:278 pass:261 dwarn:0 dfail:0 fail:0 skip:17 time:575s fi-skl-6700k total:278 pass:256 dwarn:4 dfail:0 fail:0 skip:18 time:456s fi-skl-6770hq total:278 pass:268 dwarn:0 dfail:0 fail:0 skip:10 time:494s fi-skl-gvtdvm total:278 pass:265 dwarn:0 dfail:0 fail:0 skip:13 time:429s fi-snb-2520m total:278 pass:250 dwarn:0 dfail:0 fail:0 skip:28 time:533s fi-snb-2600 total:278 pass:249 dwarn:0 dfail:0 fail:0 skip:29 time:403s
fi-bxt-j4205 failed to collect. IGT log at Patchwork_4497/fi-bxt-j4205/igt.log fi-byt-j1900 failed to collect. IGT log at Patchwork_4497/fi-byt-j1900/igt.log
57188aaf3b96d08e4095755fda0e577b1e0d8c60 drm-tip: 2017y-04m-12d-16h-12m-10s UTC integration manifest 19c9282 drm/i915/perf: remove perf.hook_lock
2f78071 drm/i915/perf: per-gen timebase for checking sample freq
953347e drm/i915/perf: Add more OA configs for BDW, CHV, SKL + BXT
96f8110 drm/i915/perf: Add OA unit support for Gen 8+
c5ccb72 drm/i915/perf: Add 'render basic' Gen8+ OA unit configs
8241dda drm/i915: expose _SUBSLICE_MASK GETPARM
0bae603 drm/i915: expose _SLICE_MASK GETPARM
29590b2 drm/i915/perf: rate limit spurious oa report notice
5ab4f33 drm/i915/perf: better pipeline aged/aging tail updates
818cc70 drm/i915/perf: improve invalid OA format debug message
9fe2f7e drm/i915/perf: improve tail race workaround
69ac693 drm/i915/perf: no head/tail ref in gen7_oa_read
00e051e drm/i915/perf: avoid read back of head register
372021d drm/i915/perf: avoid poll, read, EAGAIN busy loops
227c908 drm/i915/perf: fix gen7_append_oa_reports comment
== Logs ==
For more details see: https://intel-gfx-ci.01.org/CI/Patchwork_4497/
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
cover Enable OA unit for Gen 8 and 9 in i915 perf
In reply to Robert BraggHi,
Taking over from Rob for this v5. This series only has the following changes from v4 :
- patch 9 & 10 : updated number for GETPARAM after rebase
- patch 12 : drain the GPU before reconfiguring the OA unit to work around a race condition where the CPU & GPU update the context image at the same time (see gen8_configure_all_contexts)
The second item had been causing instabilities for a while in the IGT tests. With that out of the way, tests are a lot more stable now.
Cheers,
Robert Bragg (15):
drm/i915/perf: fix gen7_append_oa_reports comment drm/i915/perf: avoid poll, read, EAGAIN busy loops drm/i915/perf: avoid read back of head register drm/i915/perf: no head/tail ref in gen7_oa_read drm/i915/perf: improve tail race workaround drm/i915/perf: improve invalid OA format debug message drm/i915/perf: better pipeline aged/aging tail updates drm/i915/perf: rate limit spurious oa report notice drm/i915: expose _SLICE_MASK GETPARM
drm/i915: expose _SUBSLICE_MASK GETPARM drm/i915/perf: Add 'render basic' Gen8+ OA unit configs drm/i915/perf: Add OA unit support for Gen 8+ drm/i915/perf: Add more OA configs for BDW, CHV, SKL + BXT drm/i915/perf: per-gen timebase for checking sample freq drm/i915/perf: remove perf.hook_lock
drivers/gpu/drm/i915/Makefile | 8 +- drivers/gpu/drm/i915/i915_drv.c | 10 + drivers/gpu/drm/i915/i915_drv.h | 121 +- drivers/gpu/drm/i915/i915_gem_context.h | 1 + drivers/gpu/drm/i915/i915_oa_bdw.c | 5154 +++++++++++++++++++++++++++++++ drivers/gpu/drm/i915/i915_oa_bdw.h | 38 + drivers/gpu/drm/i915/i915_oa_bxt.c | 2541 +++++++++++++++ drivers/gpu/drm/i915/i915_oa_bxt.h | 38 + drivers/gpu/drm/i915/i915_oa_chv.c | 2730 ++++++++++++++++ drivers/gpu/drm/i915/i915_oa_chv.h | 38 + drivers/gpu/drm/i915/i915_oa_hsw.c | 58 +- drivers/gpu/drm/i915/i915_oa_sklgt2.c | 3302 ++++++++++++++++++++ drivers/gpu/drm/i915/i915_oa_sklgt2.h | 38 + drivers/gpu/drm/i915/i915_oa_sklgt3.c | 2856 +++++++++++++++++ drivers/gpu/drm/i915/i915_oa_sklgt3.h | 38 + drivers/gpu/drm/i915/i915_oa_sklgt4.c | 2910 +++++++++++++++++ drivers/gpu/drm/i915/i915_oa_sklgt4.h | 38 + drivers/gpu/drm/i915/i915_perf.c | 1361 ++++++-- drivers/gpu/drm/i915/i915_reg.h | 22 + drivers/gpu/drm/i915/intel_lrc.c | 5 + include/uapi/drm/i915_drm.h | 27 +- 21 files changed, 21096 insertions(+), 238 deletions(-) create mode 100644 drivers/gpu/drm/i915/i915_oa_bdw.c create mode 100644 drivers/gpu/drm/i915/i915_oa_bdw.h create mode 100644 drivers/gpu/drm/i915/i915_oa_bxt.c create mode 100644 drivers/gpu/drm/i915/i915_oa_bxt.h create mode 100644 drivers/gpu/drm/i915/i915_oa_chv.c create mode 100644 drivers/gpu/drm/i915/i915_oa_chv.h create mode 100644 drivers/gpu/drm/i915/i915_oa_sklgt2.c create mode 100644 drivers/gpu/drm/i915/i915_oa_sklgt2.h create mode 100644 drivers/gpu/drm/i915/i915_oa_sklgt3.c create mode 100644 drivers/gpu/drm/i915/i915_oa_sklgt3.h create mode 100644 drivers/gpu/drm/i915/i915_oa_sklgt4.c create mode 100644 drivers/gpu/drm/i915/i915_oa_sklgt4.h
--
2.11.0
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v5 1/15 drm/i915/perf: fix gen7_append_oa_reports comment
From: Robert Bragg <[email protected]>
If I'm going to complain about a back-to-front convention then the least I can do is not muddle the comment up too.
--- drivers/gpu/drm/i915/i915_perf.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/i915/i915_perf.c +1 -1
@@ -431,7 +431,7 @@ static int append_oa_sample(struct i915_perf_stream *stream,* userspace. * * Note: reports are consumed from the head, and appended to the -* tail, so the head chases the tail?... If you think that's mad +* tail, so the tail chases the head?... If you think that's mad * and back-to-front you're not alone, but this follows the * Gen PRM naming convention. *
--
2.11.0
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v5 2/15 drm/i915/perf: avoid poll, read, EAGAIN busy loops
In reply to Lionel LandwerlinFrom: Robert Bragg <[email protected]>
If the function for checking whether there is OA buffer data available (during a poll or blocking read) has false positives then we want to avoid a situation where the subsequent read() returns EAGAIN (after a more accurate check) followed by a poll() immediately reporting the same false positive POLLIN event and effectively maintaining a busy loop until there really is data.
This makes sure that we clear the .pollin event status whenever we return EAGAIN to userspace which will throttle subsequent POLLIN events and repeated attempts to read to the 5ms intervals of the hrtimer callback we have.
--- drivers/gpu/drm/i915/i915_perf.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/i915_perf.c +9 -1
@@ -1352,7 +1352,15 @@ static ssize_t i915_perf_read(struct file *file,mutex_unlock (&dev_priv->perf.lock); } -if (ret >=0 ) { +/* We allow the poll checking to sometimes report false positive POLLIN +* events where we might actually report EAGAIN on read() if there's +* not really any data available. In this situation though we don't +* want to enter a busy loop between poll() reporting a POLLIN event +* and read() returning -EAGAIN. Clearing the oa.pollin state here +* effectively ensures we back off until the next hrtimer callback +* before reporting another POLLIN event. +*/ +if (ret >=0 || ret == -EAGAIN ) {/* Maybe make ->pollin per-stream state if we support multiple * concurrent streams in the future. */
--
2.11.0
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v5 3/15 drm/i915/perf: avoid read back of head register
In reply to Lionel LandwerlinFrom: Robert Bragg <[email protected]>
There's no need for the driver to keep reading back the head pointer from hardware since the hardware doesn't update it automatically. This way we can treat any invalid head pointer value as a software/driver bug instead of spurious hardware behaviour.
This change is also a small stepping stone towards re-working how the head and tail state is managed as part of an improved workaround for the tail register race condition.
--- drivers/gpu/drm/i915/i915_drv.h | 11 ++++++++++ drivers/gpu/drm/i915/i915_perf.c | 46 ++++++++++++++++++---------------------- 2 files changed, 32 insertions(+), 25 deletions(-)
drivers/gpu/drm/i915/i915_drv.h +11 -0
@@ -2469,6 +2469,17 @@ struct drm_i915_private {u8 *vaddr;int format;int format_size; + +/** +* Although we can always read back the head +* pointer register, we prefer to avoid +* trusting the HW state, just to avoid any +* risk that some hardware condition could +* somehow bump the head pointer unpredictably +* and cause us to forward the wrong OA buffer +* data to userspace. +*/ +u32 head; } oa_buffer;u32 gen7_latched_oastatus1;
drivers/gpu/drm/i915/i915_perf.c +21 -25
104 lines not on this page · raw message
--
2.11.0
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v5 4/15 drm/i915/perf: no head/tail ref in gen7_oa_read
In reply to Lionel LandwerlinFrom: Robert Bragg <[email protected]>
This avoids redundantly passing an (inout) head and tail pointer to gen7_append_oa_reports() from gen7_oa_read which doesn't need to reference either itself.
Moving the head/tail reads and writes into gen7_append_oa_reports should have no functional effect except to avoid some redundant head pointer writes in cases where nothing was copied to userspace.
This is a stepping stone towards updating how the head and tail pointer state is managed to improve the workaround for the OA unit's tail pointer race. It reduces the number of places we need to read/write the head and tail pointers.
--- drivers/gpu/drm/i915/i915_perf.c | 51 +++++++++++++++------------------------- 1 file changed, 19 insertions(+), 32 deletions(-)
drivers/gpu/drm/i915/i915_perf.c +19 -32
@@ -420,8 +420,6 @@ static int append_oa_sample(struct i915_perf_stream *stream,* @buf: destination buffer given by userspace * @count: the number of bytes userspace wants to read * @offset: (inout): the current position for writing into @buf -* @head_ptr: (inout): the current oa buffer cpu read position -* @tail: the current oa buffer gpu write position * * Notably any error condition resulting in a short read (-%ENOSPC or * -%EFAULT) will be returned even though one or more records may
@@ -439,9 +437,7 @@ static int append_oa_sample(struct i915_perf_stream *stream,static int gen7_append_oa_reports (struct i915_perf_stream *stream,char __user *buf,size_t count, -size_t *offset, -u32 *head_ptr, -u32 tail) +size_t *offset) {struct drm_i915_private *dev_priv = stream->dev_priv;int report_size = dev_priv->perf.oa.oa_buffer.format_size;
@@ -449,14 +445,15 @@ static int gen7_append_oa_reports(struct i915_perf_stream *stream,int tail_margin = dev_priv->perf.oa.tail_margin;u32 gtt_offset =i915_ggtt_offset (dev_priv->perf.oa.oa_buffer.vma);u32 mask = (OA_BUFFER_SIZE -1 ); -u32 head; +size_t start_offset = *offset; +u32 head, oastatus1, tail;u32 taken;int ret =0 ;if (WARN_ON (!stream->enabled))return -EIO ; - head = *head_ptr - gtt_offset; + head = dev_priv->perf.oa.oa_buffer.head - gtt_offset;/* An out of bounds or misaligned head pointer implies a driver bug * since we are in full control of head pointer which should only
@@ -467,7 +464,8 @@ static int gen7_append_oa_reports(struct i915_perf_stream *stream, "Inconsistent OA buffer head pointer = %u\n", head))return -EIO ; - tail -= gtt_offset; + oastatus1 =I915_READ (GEN7_OASTATUS1 ); + tail = (oastatus1 &GEN7_OASTATUS1_TAIL_MASK ) - gtt_offset;/* The OA unit is expected to wrap the tail pointer according to the OA * buffer size
@@ -477,8 +475,6 @@ static int gen7_append_oa_reports(struct i915_perf_stream *stream, tail); dev_priv->perf.oa.ops.oa_disable (dev_priv); dev_priv->perf.oa.ops.oa_enable (dev_priv); - *head_ptr =I915_READ (GEN7_OASTATUS2 ) & -GEN7_OASTATUS2_HEAD_MASK ;return -EIO ; }
@@ -542,7 +538,18 @@ static int gen7_append_oa_reports(struct i915_perf_stream *stream, report32[0 ] =0 ; } - *head_ptr = gtt_offset + head; + +if (start_offset != *offset) { +/* We removed the gtt_offset for the copy loop above, indexing +* relative to oa_buf_base so put back here... +*/ + head += gtt_offset; + +I915_WRITE (GEN7_OASTATUS2 , + ((head &GEN7_OASTATUS2_HEAD_MASK ) | +OA_MEM_SELECT_GGTT )); + dev_priv->perf.oa.oa_buffer.head = head; + }return ret; }
@@ -570,8 +577,6 @@ static int gen7_oa_read(struct i915_perf_stream *stream, {struct drm_i915_private *dev_priv = stream->dev_priv;u32 oastatus1; -u32 head; -u32 tail;int ret;if (WARN_ON (!dev_priv->perf.oa.oa_buffer.vaddr))
@@ -579,9 +584,6 @@ static int gen7_oa_read(struct i915_perf_stream *stream, oastatus1 =I915_READ (GEN7_OASTATUS1 ); - head = dev_priv->perf.oa.oa_buffer.head; - tail = oastatus1 &GEN7_OASTATUS1_TAIL_MASK ; -/* XXX: On Haswell we don't have a safe way to clear oastatus1 * bits while the OA unit is enabled (while the tail pointer * may be updated asynchronously) so we ignore status bits
@@ -621,9 +623,6 @@ static int gen7_oa_read(struct i915_perf_stream *stream, dev_priv->perf.oa.ops.oa_enable (dev_priv); oastatus1 =I915_READ (GEN7_OASTATUS1 ); - - head = dev_priv->perf.oa.oa_buffer.head; - tail = oastatus1 &GEN7_OASTATUS1_TAIL_MASK ; }if (unlikely (oastatus1 &GEN7_OASTATUS1_REPORT_LOST )) {
@@ -635,19 +634,7 @@ static int gen7_oa_read(struct i915_perf_stream *stream,GEN7_OASTATUS1_REPORT_LOST ; } - ret =gen7_append_oa_reports (stream, buf, count, offset, - &head, tail); - -/* Note: we update the head pointer here even if an error -* was returned since the error may represent a short read -* where some some reports were successfully copied. -*/ -I915_WRITE (GEN7_OASTATUS2 , - ((head &GEN7_OASTATUS2_HEAD_MASK ) | -OA_MEM_SELECT_GGTT )); - dev_priv->perf.oa.oa_buffer.head = head; - -return ret; +return gen7_append_oa_reports (stream, buf, count, offset); }/**
--
2.11.0
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v5 5/15 drm/i915/perf: improve tail race workaround
In reply to Lionel LandwerlinFrom: Robert Bragg <[email protected]>
There's a HW race condition between OA unit tail pointer register updates and writes to memory whereby the tail pointer can sometimes get ahead of what's been written out to the OA buffer so far (in terms of what's visible to the CPU).
Although this can be observed explicitly while copying reports to userspace by checking for a zeroed report-id field in tail reports, we want to account for this earlier, as part of the _oa_buffer_check to avoid lots of redundant read() attempts.
Previously the driver used to define an effective tail pointer that lagged the real pointer by a 'tail margin' measured in bytes derived from OA_TAIL_MARGIN_NSEC and the configured sampling frequency. Unfortunately this was flawed considering that the OA unit may also automatically generate non-periodic reports (such as on context switch) or the OA unit may be enabled without any periodic sampling.
This improves how we define a tail pointer for reading that lags the real tail pointer by at least %OA_TAIL_MARGIN_NSEC nanoseconds, which gives enough time for the corresponding reports to become visible to the CPU.
The driver now maintains two tail pointers:
1) An 'aging' tail with an associated timestamp that is tracked until we can trust the corresponding data is visible to the CPU; at which point it is considered 'aged'.
2) An 'aged' tail that can be used for read()ing.
The two separate pointers let us decouple read()s from tail pointer aging.
The tail pointers are checked and updated at a limited rate within a hrtimer callback (the same callback that is used for delivering POLLIN events) and since we're now measuring the wall clock time elapsed since a given tail pointer was read the mechanism no longer cares about the OA unit's periodic sampling frequency.
The natural place to handle the tail pointer updates was in gen7_oa_buffer_is_empty() which is called as part of blocking reads and the hrtimer callback used for polling, and so this was renamed to oa_buffer_check() considering the added side effect while checking whether the buffer contains data.
--- drivers/gpu/drm/i915/i915_drv.h | 60 ++++++++- drivers/gpu/drm/i915/i915_perf.c | 277 ++++++++++++++++++++++++++------------- 2 files changed, 241 insertions(+), 96 deletions(-)
drivers/gpu/drm/i915/i915_drv.h +55 -5
@@ -2097,7 +2097,7 @@ struct i915_oa_ops { size_t *offset);/** -* @oa_buffer_is_empty: Check if OA buffer empty (false positives OK) +* @oa_buffer_check: Check for OA buffer data + update tail * * This is either called via fops or the poll check hrtimer (atomic * ctx) without any locks taken.
@@ -2110,7 +2110,7 @@ struct i915_oa_ops {* here, which will be handled gracefully - likely resulting in an * %EAGAIN error for userspace. */ -bool (*oa_buffer_is_empty)(struct drm_i915_private *dev_priv); +bool (*oa_buffer_check)(struct drm_i915_private *dev_priv); };struct intel_cdclk_state {
@@ -2453,9 +2453,6 @@ struct drm_i915_private {bool periodic;int period_exponent; -int timestamp_frequency; - -int tail_margin;int metrics_set;
@@ -2471,6 +2468,59 @@ struct drm_i915_private {int format_size;/** +* Locks reads and writes to all head/tail state +* +* Consider: the head and tail pointer state +* needs to be read consistently from a hrtimer +* callback (atomic context) and read() fop +* (user context) with tail pointer updates +* happening in atomic context and head updates +* in user context and the (unlikely) +* possibility of read() errors needing to +* reset all head/tail state. +* +* Note: Contention or performance aren't +* currently a significant concern here +* considering the relatively low frequency of +* hrtimer callbacks (5ms period) and that +* reads typically only happen in response to a +* hrtimer event and likely complete before the +* next callback. +* +* Note: This lock is not held *while* reading +* and copying data to userspace so the value +* of head observed in htrimer callbacks won't +* represent any partial consumption of data. +*/ +spinlock_t ptr_lock; + +/** +* One 'aging' tail pointer and one 'aged' +* tail pointer ready to used for reading. +* +* Initial values of 0xffffffff are invalid +* and imply that an update is required +* (and should be ignored by an attempted +* read) +*/ +struct { +u32 offset; + } tails[2 ]; + +/** +* Index for the aged tail ready to read() +* data up to. +*/ +unsigned int aged_tail_idx; + +/** +* A monotonic timestamp for when the current +* aging tail pointer was read; used to +* determine when it is old enough to trust. +*/ +u64 aging_timestamp; + +/** * Although we can always read back the head * pointer register, we prefer to avoid * trusting the HW state, just to avoid any
drivers/gpu/drm/i915/i915_perf.c +186 -91
388 lines not on this page · raw message
--
2.11.0
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v5 6/15 drm/i915/perf: improve invalid OA format debug message
In reply to Lionel LandwerlinFrom: Robert Bragg <[email protected]>
A minor improvement to debugging output
--- drivers/gpu/drm/i915/i915_perf.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
drivers/gpu/drm/i915/i915_perf.c +4 -2
@@ -1904,11 +1904,13 @@ static int read_properties_unlocked(struct drm_i915_private *dev_priv,break ;case DRM_I915_PERF_PROP_OA_FORMAT :if (value ==0 || value >=I915_OA_FORMAT_MAX ) { -DRM_DEBUG ("Invalid OA report format\n" ); +DRM_DEBUG ("Out-of-range OA report format %llu\n" , + value);return -EINVAL ; }if (!dev_priv->perf.oa.oa_formats[value].size) { -DRM_DEBUG ("Invalid OA report format\n" ); +DRM_DEBUG ("Unsupported OA report format %llu\n" , + value);return -EINVAL ; } props->oa_format = value;
--
2.11.0
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v5 7/15 drm/i915/perf: better pipeline aged/aging tail updates
In reply to Lionel LandwerlinFrom: Robert Bragg <[email protected]>
This updates the tail pointer race workaround handling to updating the 'aged' pointer before looking to start aging a new one. There's the possibility that there is already new data available and so we can immediately start aging a new pointer without having to first wait for a later hrtimer callback (and then another to age).
--- drivers/gpu/drm/i915/i915_perf.c | 41 ++++++++++++++++++++++------------------ 1 file changed, 23 insertions(+), 18 deletions(-)
drivers/gpu/drm/i915/i915_perf.c +23 -18
@@ -391,6 +391,29 @@ static bool gen7_oa_buffer_check_unlocked(struct drm_i915_private *dev_priv) now =ktime_get_mono_fast_ns (); +/* Update the aged tail +* +* Flip the tail pointer available for read()s once the aging tail is +* old enough to trust that the corresponding data will be visible to +* the CPU... +* +* Do this before updating the aging pointer in case we may be able to +* immediately start aging a new pointer too (if new data has become +* available) without needing to wait for a later hrtimer callback. +*/ +if (aging_tail !=INVALID_TAIL_PTR && + ((now - dev_priv->perf.oa.oa_buffer.aging_timestamp) > +OA_TAIL_MARGIN_NSEC )) { + aged_idx ^=1 ; + dev_priv->perf.oa.oa_buffer.aged_tail_idx = aged_idx; + + aged_tail = aging_tail; + +/* Mark that we need a new pointer to start aging... */ + dev_priv->perf.oa.oa_buffer.tails[!aged_idx].offset =INVALID_TAIL_PTR ; + aging_tail =INVALID_TAIL_PTR ; + } +/* Update the aging tail * * We throttle aging tail updates until we have a new tail that
@@ -420,24 +443,6 @@ static bool gen7_oa_buffer_check_unlocked(struct drm_i915_private *dev_priv) } } -/* Update the aged tail -* -* Flip the tail pointer available for read()s once the aging tail is -* old enough to trust that the corresponding data will be visible to -* the CPU... -*/ -if (aging_tail !=INVALID_TAIL_PTR && - ((now - dev_priv->perf.oa.oa_buffer.aging_timestamp) > -OA_TAIL_MARGIN_NSEC )) { - aged_idx ^=1 ; - dev_priv->perf.oa.oa_buffer.aged_tail_idx = aged_idx; - - aged_tail = aging_tail; - -/* Mark that we need a new pointer to start aging... */ - dev_priv->perf.oa.oa_buffer.tails[!aged_idx].offset =INVALID_TAIL_PTR ; - } -spin_unlock_irqrestore (&dev_priv->perf.oa.oa_buffer.ptr_lock, flags); return aged_tail == INVALID_TAIL_PTR ?
--
2.11.0
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v5 8/15 drm/i915/perf: rate limit spurious oa report notice
In reply to Lionel LandwerlinFrom: Robert Bragg <[email protected]>
This change is pre-emptively aiming to avoid a potential cause of kernel logging noise in case some condition were to result in us seeing invalid OA reports.
The workaround for the OA unit's tail pointer race condition is what avoids the primary known cause of invalid reports being seen and with that in place we aren't expecting to see this notice but it can't be entirely ruled out.
Just in case some condition does lead to the notice then it's likely that it will be triggered repeatedly while attempting to append a sequence of reports and depending on the configured OA sampling frequency that might be a large number of repeat notices.
v2: (Chris) avoid inconsistent warning on throttle with
printk_ratelimit()
v3: (Matt) init and summarise with stream init/close not driver init/fini
--- drivers/gpu/drm/i915/i915_drv.h | 6 ++++++ drivers/gpu/drm/i915/i915_perf.c | 28 +++++++++++++++++++++++++++- 2 files changed, 33 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/i915_drv.h +6 -0
@@ -2451,6 +2451,12 @@ struct drm_i915_private {wait_queue_head_t poll_wq;bool pollin; +/** +* For rate limiting any notifications of spurious +* invalid OA reports +*/ +struct ratelimit_state spurious_report_rs; +bool periodic;int period_exponent;
drivers/gpu/drm/i915/i915_perf.c +27 -1
@@ -632,7 +632,8 @@ static int gen7_append_oa_reports(struct i915_perf_stream *stream,* copying it to userspace... */ if (report32[0 ] ==0 ) { -DRM_NOTE ("Skipping spurious, invalid OA report\n" ); +if (__ratelimit (&dev_priv->perf.oa.spurious_report_rs)) +DRM_NOTE ("Skipping spurious, invalid OA report\n" );continue ; }
@@ -913,6 +914,11 @@ static void i915_oa_stream_destroy(struct i915_perf_stream *stream)oa_put_render_ctx_id (stream); dev_priv->perf.oa.exclusive_stream =NULL ; + +if (dev_priv->perf.oa.spurious_report_rs.missed) { +DRM_NOTE ("%d spurious OA report notices suppressed due to ratelimiting\n" , + dev_priv->perf.oa.spurious_report_rs.missed); + } }static void gen7_init_oa_buffer (struct drm_i915_private *dev_priv)
@@ -1268,6 +1274,26 @@ static int i915_oa_stream_init(struct i915_perf_stream *stream,return -EINVAL ; } +/* We set up some ratelimit state to potentially throttle any _NOTES +* about spurious, invalid OA reports which we don't forward to +* userspace. +* +* The initialization is associated with opening the stream (not driver +* init) considering we print a _NOTE about any throttling when closing +* the stream instead of waiting until driver _fini which no one would +* ever see. +* +* Using the same limiting factors as printk_ratelimit() +*/ +ratelimit_state_init (&dev_priv->perf.oa.spurious_report_rs, +5 *HZ ,10 ); +/* Since we use a DRM_NOTE for spurious reports it would be +* inconsistent to let __ratelimit() automatically print a warning for +* throttling. +*/ +ratelimit_set_flags (&dev_priv->perf.oa.spurious_report_rs, +RATELIMIT_MSG_ON_RELEASE ); + stream->sample_size =sizeof (struct drm_i915_perf_record_header ); format_size = dev_priv->perf.oa.oa_formats[props->oa_format].size;
--
2.11.0
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v5 9/15 drm/i915: expose _SLICE_MASK GETPARM
In reply to Lionel LandwerlinFrom: Robert Bragg <[email protected]>
Enables userspace to determine the number of slices enabled and also know what specific slices are enabled. This information is required, for example, to be able to analyse some OA counter reports where the counter configuration depends on the HW slice configuration.
--- drivers/gpu/drm/i915/i915_drv.c | 5 +++++ include/uapi/drm/i915_drm.h | 3 +++ 2 files changed, 8 insertions(+)
drivers/gpu/drm/i915/i915_drv.c +5 -0
@@ -358,6 +358,11 @@ static int i915_getparam(struct drm_device *dev, void *data,*/ value =1 ;break ; +case I915_PARAM_SLICE_MASK : + value =INTEL_INFO (dev_priv)->sseu.slice_mask; +if (!value) +return -ENODEV ; +break ;default :DRM_DEBUG ("Unknown parameter %d\n" , param->param);return -EINVAL ;
include/uapi/drm/i915_drm.h +3 -0
@@ -418,6 +418,9 @@ typedef struct drm_i915_irq_wait {*/ #define I915_PARAM_HAS_EXEC_CAPTURE 45 +/* Query the mask of slices available for this system */ +#define I915_PARAM_SLICE_MASK 46 + typedef struct drm_i915_getparam { __s32 param; /*
--
2.11.0
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v5 10/15 drm/i915: expose _SUBSLICE_MASK GETPARM
In reply to Lionel LandwerlinFrom: Robert Bragg <[email protected]>
Assuming a uniform mask across all slices, this enables userspace to determine the specific sub slices enabled. This information is required, for example, to be able to analyse some OA counter reports where the counter configuration depends on the HW sub slice configuration.
--- drivers/gpu/drm/i915/i915_drv.c | 5 +++++ include/uapi/drm/i915_drm.h | 5 +++++ 2 files changed, 10 insertions(+)
drivers/gpu/drm/i915/i915_drv.c +5 -0
@@ -363,6 +363,11 @@ static int i915_getparam(struct drm_device *dev, void *data,if (!value)return -ENODEV ;break ; +case I915_PARAM_SUBSLICE_MASK : + value =INTEL_INFO (dev_priv)->sseu.subslice_mask; +if (!value) +return -ENODEV ; +break ;default :DRM_DEBUG ("Unknown parameter %d\n" , param->param);return -EINVAL ;
include/uapi/drm/i915_drm.h +5 -0
@@ -421,6 +421,11 @@ typedef struct drm_i915_irq_wait {/* Query the mask of slices available for this system */ #define I915_PARAM_SLICE_MASK 46 +/* Assuming it's uniform for each slice, this queries the mask of subslices +* per-slice for this system. +*/ +#define I915_PARAM_SUBSLICE_MASK 47 + typedef struct drm_i915_getparam { __s32 param; /*
--
2.11.0
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v5 11/15 drm/i915/perf: Add 'render basic' Gen8+ OA unit configs
In reply to Lionel LandwerlinFrom: Robert Bragg <[email protected]>
Adds a static OA unit, MUX, B Counter + Flex EU configurations for basic render metrics on Broadwell, Cherryview, Skylake and Broxton. These are auto generated from an XML description of metric sets, currently maintained in gputop, ref:
Quoted 2 lines
gputop-data/oa-*.xmlscripts/i915-perf-kernelgen.py
$ make -C gputop-data -f Makefile.xml WHITELIST=RenderBasic
v2: add newlines to debug messages + fix comment (Matthew Auld)
--- drivers/gpu/drm/i915/Makefile | 8 +- drivers/gpu/drm/i915/i915_drv.h | 2 + drivers/gpu/drm/i915/i915_oa_bdw.c | 380 ++++++++++++++++++++++++++++++++++ drivers/gpu/drm/i915/i915_oa_bdw.h | 38 ++++ drivers/gpu/drm/i915/i915_oa_bxt.c | 238 +++++++++++++++++++++ drivers/gpu/drm/i915/i915_oa_bxt.h | 38 ++++ drivers/gpu/drm/i915/i915_oa_chv.c | 225 ++++++++++++++++++++ drivers/gpu/drm/i915/i915_oa_chv.h | 38 ++++ drivers/gpu/drm/i915/i915_oa_sklgt2.c | 228 ++++++++++++++++++++ drivers/gpu/drm/i915/i915_oa_sklgt2.h | 38 ++++ drivers/gpu/drm/i915/i915_oa_sklgt3.c | 236 +++++++++++++++++++++ drivers/gpu/drm/i915/i915_oa_sklgt3.h | 38 ++++ drivers/gpu/drm/i915/i915_oa_sklgt4.c | 247 ++++++++++++++++++++++ drivers/gpu/drm/i915/i915_oa_sklgt4.h | 38 ++++ 14 files changed, 1791 insertions(+), 1 deletion(-) create mode 100644 drivers/gpu/drm/i915/i915_oa_bdw.c create mode 100644 drivers/gpu/drm/i915/i915_oa_bdw.h create mode 100644 drivers/gpu/drm/i915/i915_oa_bxt.c create mode 100644 drivers/gpu/drm/i915/i915_oa_bxt.h create mode 100644 drivers/gpu/drm/i915/i915_oa_chv.c create mode 100644 drivers/gpu/drm/i915/i915_oa_chv.h create mode 100644 drivers/gpu/drm/i915/i915_oa_sklgt2.c create mode 100644 drivers/gpu/drm/i915/i915_oa_sklgt2.h create mode 100644 drivers/gpu/drm/i915/i915_oa_sklgt3.c create mode 100644 drivers/gpu/drm/i915/i915_oa_sklgt3.h create mode 100644 drivers/gpu/drm/i915/i915_oa_sklgt4.c create mode 100644 drivers/gpu/drm/i915/i915_oa_sklgt4.h
drivers/gpu/drm/i915/Makefile +7 -1
@@ -127,7 +127,13 @@ i915-y += i915_vgpu.o # perf code i915-y += i915_perf.o \ - i915_oa_hsw.o + i915_oa_hsw.o \ + i915_oa_bdw.o \ + i915_oa_chv.o \ + i915_oa_sklgt2.o \ + i915_oa_sklgt3.o \ + i915_oa_sklgt4.o \ + i915_oa_bxt.o ifeq ($(CONFIG_DRM_I915_GVT),y) i915-y += intel_gvt.o
drivers/gpu/drm/i915/i915_drv.h +2 -0
@@ -2466,6 +2466,8 @@ struct drm_i915_private {int mux_regs_len;const struct i915_oa_reg *b_counter_regs;int b_counter_regs_len; +const struct i915_oa_reg *flex_regs; +int flex_regs_len;struct {struct i915_vma *vma;
drivers/gpu/drm/i915/i915_oa_bdw.c new file +380 -0
380 lines not on this page · raw message
drivers/gpu/drm/i915/i915_oa_bdw.h new file +38 -0
@@ -0,0 +1,38 @@ +/* +* Autogenerated file, DO NOT EDIT manually! +* +* Copyright (c) 2015 Intel Corporation +* +* Permission is hereby granted, free of charge, to any person obtaining a +* copy of this software and associated documentation files (the "Software"), +* to deal in the Software without restriction, including without limitation +* the rights to use, copy, modify, merge, publish, distribute, sublicense, +* and/or sell copies of the Software, and to permit persons to whom the +* Software is furnished to do so, subject to the following conditions: +* +* The above copyright notice and this permission notice (including the next +* paragraph) shall be included in all copies or substantial portions of the +* Software. +* +* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL +* THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING +* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS +* IN THE SOFTWARE. +* +*/ + +#ifndef __I915_OA_BDW_H__ +#define __I915_OA_BDW_H__ + +extern int i915_oa_n_builtin_metric_sets_bdw; + +extern int i915_oa_select_metric_set_bdw (struct drm_i915_private *dev_priv); + +extern int i915_perf_register_sysfs_bdw (struct drm_i915_private *dev_priv); + +extern void i915_perf_unregister_sysfs_bdw (struct drm_i915_private *dev_priv); + +#endif
drivers/gpu/drm/i915/i915_oa_bxt.c new file +238 -0
238 lines not on this page · raw message
drivers/gpu/drm/i915/i915_oa_bxt.h new file +38 -0
@@ -0,0 +1,38 @@ +/* +* Autogenerated file, DO NOT EDIT manually! +* +* Copyright (c) 2015 Intel Corporation +* +* Permission is hereby granted, free of charge, to any person obtaining a +* copy of this software and associated documentation files (the "Software"), +* to deal in the Software without restriction, including without limitation +* the rights to use, copy, modify, merge, publish, distribute, sublicense, +* and/or sell copies of the Software, and to permit persons to whom the +* Software is furnished to do so, subject to the following conditions: +* +* The above copyright notice and this permission notice (including the next +* paragraph) shall be included in all copies or substantial portions of the +* Software. +* +* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL +* THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING +* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS +* IN THE SOFTWARE. +* +*/ + +#ifndef __I915_OA_BXT_H__ +#define __I915_OA_BXT_H__ + +extern int i915_oa_n_builtin_metric_sets_bxt; + +extern int i915_oa_select_metric_set_bxt (struct drm_i915_private *dev_priv); + +extern int i915_perf_register_sysfs_bxt (struct drm_i915_private *dev_priv); + +extern void i915_perf_unregister_sysfs_bxt (struct drm_i915_private *dev_priv); + +#endif
drivers/gpu/drm/i915/i915_oa_chv.c new file +225 -0
225 lines not on this page · raw message
drivers/gpu/drm/i915/i915_oa_chv.h new file +38 -0
@@ -0,0 +1,38 @@ +/* +* Autogenerated file, DO NOT EDIT manually! +* +* Copyright (c) 2015 Intel Corporation +* +* Permission is hereby granted, free of charge, to any person obtaining a +* copy of this software and associated documentation files (the "Software"), +* to deal in the Software without restriction, including without limitation +* the rights to use, copy, modify, merge, publish, distribute, sublicense, +* and/or sell copies of the Software, and to permit persons to whom the +* Software is furnished to do so, subject to the following conditions: +* +* The above copyright notice and this permission notice (including the next +* paragraph) shall be included in all copies or substantial portions of the +* Software. +* +* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL +* THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING +* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS +* IN THE SOFTWARE. +* +*/ + +#ifndef __I915_OA_CHV_H__ +#define __I915_OA_CHV_H__ + +extern int i915_oa_n_builtin_metric_sets_chv; + +extern int i915_oa_select_metric_set_chv (struct drm_i915_private *dev_priv); + +extern int i915_perf_register_sysfs_chv (struct drm_i915_private *dev_priv); + +extern void i915_perf_unregister_sysfs_chv (struct drm_i915_private *dev_priv); + +#endif
drivers/gpu/drm/i915/i915_oa_sklgt2.c new file +228 -0
228 lines not on this page · raw message
drivers/gpu/drm/i915/i915_oa_sklgt2.h new file +38 -0
@@ -0,0 +1,38 @@ +/* +* Autogenerated file, DO NOT EDIT manually! +* +* Copyright (c) 2015 Intel Corporation +* +* Permission is hereby granted, free of charge, to any person obtaining a +* copy of this software and associated documentation files (the "Software"), +* to deal in the Software without restriction, including without limitation +* the rights to use, copy, modify, merge, publish, distribute, sublicense, +* and/or sell copies of the Software, and to permit persons to whom the +* Software is furnished to do so, subject to the following conditions: +* +* The above copyright notice and this permission notice (including the next +* paragraph) shall be included in all copies or substantial portions of the +* Software. +* +* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL +* THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING +* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS +* IN THE SOFTWARE. +* +*/ + +#ifndef __I915_OA_SKLGT2_H__ +#define __I915_OA_SKLGT2_H__ + +extern int i915_oa_n_builtin_metric_sets_sklgt2; + +extern int i915_oa_select_metric_set_sklgt2 (struct drm_i915_private *dev_priv); + +extern int i915_perf_register_sysfs_sklgt2 (struct drm_i915_private *dev_priv); + +extern void i915_perf_unregister_sysfs_sklgt2 (struct drm_i915_private *dev_priv); + +#endif
drivers/gpu/drm/i915/i915_oa_sklgt3.c new file +236 -0
236 lines not on this page · raw message
drivers/gpu/drm/i915/i915_oa_sklgt3.h new file +38 -0
@@ -0,0 +1,38 @@ +/* +* Autogenerated file, DO NOT EDIT manually! +* +* Copyright (c) 2015 Intel Corporation +* +* Permission is hereby granted, free of charge, to any person obtaining a +* copy of this software and associated documentation files (the "Software"), +* to deal in the Software without restriction, including without limitation +* the rights to use, copy, modify, merge, publish, distribute, sublicense, +* and/or sell copies of the Software, and to permit persons to whom the +* Software is furnished to do so, subject to the following conditions: +* +* The above copyright notice and this permission notice (including the next +* paragraph) shall be included in all copies or substantial portions of the +* Software. +* +* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL +* THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING +* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS +* IN THE SOFTWARE. +* +*/ + +#ifndef __I915_OA_SKLGT3_H__ +#define __I915_OA_SKLGT3_H__ + +extern int i915_oa_n_builtin_metric_sets_sklgt3; + +extern int i915_oa_select_metric_set_sklgt3 (struct drm_i915_private *dev_priv); + +extern int i915_perf_register_sysfs_sklgt3 (struct drm_i915_private *dev_priv); + +extern void i915_perf_unregister_sysfs_sklgt3 (struct drm_i915_private *dev_priv); + +#endif
drivers/gpu/drm/i915/i915_oa_sklgt4.c new file +247 -0
247 lines not on this page · raw message
drivers/gpu/drm/i915/i915_oa_sklgt4.h new file +38 -0
@@ -0,0 +1,38 @@ +/* +* Autogenerated file, DO NOT EDIT manually! +* +* Copyright (c) 2015 Intel Corporation +* +* Permission is hereby granted, free of charge, to any person obtaining a +* copy of this software and associated documentation files (the "Software"), +* to deal in the Software without restriction, including without limitation +* the rights to use, copy, modify, merge, publish, distribute, sublicense, +* and/or sell copies of the Software, and to permit persons to whom the +* Software is furnished to do so, subject to the following conditions: +* +* The above copyright notice and this permission notice (including the next +* paragraph) shall be included in all copies or substantial portions of the +* Software. +* +* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL +* THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING +* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS +* IN THE SOFTWARE. +* +*/ + +#ifndef __I915_OA_SKLGT4_H__ +#define __I915_OA_SKLGT4_H__ + +extern int i915_oa_n_builtin_metric_sets_sklgt4; + +extern int i915_oa_select_metric_set_sklgt4 (struct drm_i915_private *dev_priv); + +extern int i915_perf_register_sysfs_sklgt4 (struct drm_i915_private *dev_priv); + +extern void i915_perf_unregister_sysfs_sklgt4 (struct drm_i915_private *dev_priv); + +#endif
--
2.11.0
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v5 12/15 drm/i915/perf: Add OA unit support for Gen 8+
In reply to Lionel LandwerlinFrom: Robert Bragg <[email protected]>
Enables access to OA unit metrics for BDW, CHV, SKL and BXT which all share (more-or-less) the same OA unit design.
Of particular note in comparison to Haswell: some OA unit HW config state has become per-context state and as a consequence it is somewhat more complicated to manage synchronous state changes from the cpu while there's no guarantee of what context (if any) is currently actively running on the gpu.
The periodic sampling frequency which can be particularly useful for system-wide analysis (as opposed to command stream synchronised MI_REPORT_PERF_COUNT commands) is perhaps the most surprising state to have become per-context save and restored (while the OABUFFER destination is still a shared, system-wide resource).
This support for gen8+ takes care to consider a number of timing challenges involved in synchronously updating per-context state primarily by programming all config state from the cpu and updating all current and saved contexts synchronously while the OA unit is still disabled.
The driver intentionally avoids depending on command streamer programming to update OA state considering the lack of synchronization between the automatic loading of OACTXCONTROL state (that includes the periodic sampling state and enable state) on context restore and the parsing of any general purpose BB the driver can control. I.e. this implementation is careful to avoid the possibility of a context restore temporarily enabling any out-of-date periodic sampling state. In addition to the risk of transiently-out-of-date state being loaded automatically; there are also internal HW latencies involved in the loading of MUX configurations which would be difficult to account for from the command streamer (and we only want to enable the unit when once the MUX configuration is complete).
Since the Gen8+ OA unit design no longer supports clock gating the unit off for a single given context (which effectively stopped any progress of counters while any other context was running) and instead supports tagging OA reports with a context ID for filtering on the CPU, it means we can no longer hide the system-wide progress of counters from a non-privileged application only interested in metrics for its own context. Although we could theoretically try and subtract the progress of other contexts before forwarding reports via read() we aren't in a position to filter reports captured via MI_REPORT_PERF_COUNT commands. As a result, for Gen8+, we always require the dev.i915.perf_stream_paranoid to be unset for any access to OA metrics if not root.
v5: Drain submitted requests when enabling metric set to ensure no
lite-restore erases the context image we just updated (Lionel)
--- drivers/gpu/drm/i915/i915_drv.h | 45 +- drivers/gpu/drm/i915/i915_gem_context.h | 1 + drivers/gpu/drm/i915/i915_perf.c | 969 +++++++++++++++++++++++++++++--- drivers/gpu/drm/i915/i915_reg.h | 22 + drivers/gpu/drm/i915/intel_lrc.c | 5 + include/uapi/drm/i915_drm.h | 19 +- 6 files changed, 968 insertions(+), 93 deletions(-)
drivers/gpu/drm/i915/i915_drv.h +30 -15
@@ -2064,9 +2064,17 @@ struct i915_oa_ops {void (*init_oa_buffer)(struct drm_i915_private *dev_priv);/** -* @enable_metric_set: Applies any MUX configuration to set up the -* Boolean and Custom (B/C) counters that are part of the counter -* reports being sampled. May apply system constraints such as +* @select_metric_set: The auto generated code that checks whether a +* requested OA config is applicable to the system and if so sets up +* the mux, oa and flex eu register config pointers according to the +* current dev_priv->perf.oa.metrics_set. +*/ +int (*select_metric_set)(struct drm_i915_private *dev_priv); + +/** +* @enable_metric_set: Selects and applies any MUX configuration to set +* up the Boolean and Custom (B/C) counters that are part of the +* counter reports being sampled. May apply system constraints such as * disabling EU clock gating as required. */ int (*enable_metric_set)(struct drm_i915_private *dev_priv);
@@ -2097,20 +2105,13 @@ struct i915_oa_ops { size_t *offset);/** -* @oa_buffer_check: Check for OA buffer data + update tail -* -* This is either called via fops or the poll check hrtimer (atomic -* ctx) without any locks taken. +* @oa_hw_tail_read: read the OA tail pointer register * -* It's safe to read OA config state here unlocked, assuming that this -* is only called while the stream is enabled, while the global OA -* configuration can't be modified. -* -* Efficiency is more important than avoiding some false positives -* here, which will be handled gracefully - likely resulting in an -* %EAGAIN error for userspace. +* In particular this enables us to share all the fiddly code for +* handling the OA unit tail pointer race that affects multiple +* generations. */ -bool (*oa_buffer_check)(struct drm_i915_private *dev_priv); +u32 (*oa_hw_tail_read)(struct drm_i915_private *dev_priv); };struct intel_cdclk_state {
@@ -2472,6 +2473,7 @@ struct drm_i915_private {struct {struct i915_vma *vma;u8 *vaddr; +u32 last_ctx_id;int format;int format_size;
@@ -2541,6 +2543,14 @@ struct drm_i915_private { } oa_buffer;u32 gen7_latched_oastatus1; +u32 ctx_oactxctrl_offset; +u32 ctx_flexeu0_offset; + +/* The RPT_ID/reason field for Gen8+ includes a bit +* to determine if the CTX ID in the report is valid +* but the specific bit differs between Gen 8 and 9 +*/ +u32 gen8_valid_ctx_bit;struct i915_oa_ops ops;const struct i915_oa_format *oa_formats;
@@ -2851,6 +2861,8 @@ intel_info(const struct drm_i915_private *dev_priv)#define IS_KBL_ULX (dev_priv) (INTEL_DEVID(dev_priv) == 0x590E || \ INTEL_DEVID(dev_priv) == 0x5915 || \ INTEL_DEVID(dev_priv) == 0x591E) +#define IS_SKL_GT2 (dev_priv) (IS_SKYLAKE(dev_priv) && \ + (INTEL_DEVID(dev_priv) & 0x00F0) == 0x0010)#define IS_SKL_GT3 (dev_priv) (IS_SKYLAKE(dev_priv) && \ (INTEL_DEVID(dev_priv) & 0x00F0) == 0x0020)#define IS_SKL_GT4 (dev_priv) (IS_SKYLAKE(dev_priv) && \
@@ -3615,6 +3627,9 @@ i915_gem_context_lookup_timeline(struct i915_gem_context *ctx,int i915_perf_open_ioctl (struct drm_device *dev,void *data,struct drm_file *file); +void i915_oa_update_reg_state (struct intel_engine_cs *engine, +struct i915_gem_context *ctx, +uint32_t *reg_state);/* i915_gem_evict.c */ int __must_check i915_gem_evict_something(struct i915_address_space *vm,
drivers/gpu/drm/i915/i915_gem_context.h +1 -0
@@ -151,6 +151,7 @@ struct i915_gem_context {u64 lrc_desc;int pin_count;bool initialised; +atomic_t oa_state_dirty; } engine[I915_NUM_ENGINES ];/** ring_size: size for allocating the per-engine ring buffer */
drivers/gpu/drm/i915/i915_perf.c +898 -71
1,158 lines not on this page · raw message
drivers/gpu/drm/i915/i915_reg.h +22 -0
@@ -653,6 +653,12 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)#define GEN8_OACTXID _MMIO(0x2364) +#define GEN8_OA_DEBUG _MMIO(0x2B04) +#define GEN9_OA_DEBUG_DISABLE_CLK_RATIO_REPORTS (1<<5) +#define GEN9_OA_DEBUG_INCLUDE_CLK_RATIO (1<<6) +#define GEN9_OA_DEBUG_DISABLE_GO_1_0_REPORTS (1<<2) +#define GEN9_OA_DEBUG_DISABLE_CTX_SWITCH_REPORTS (1<<1) +#define GEN8_OACONTROL _MMIO(0x2B00)#define GEN8_OA_REPORT_FORMAT_A12 (0<<2)#define GEN8_OA_REPORT_FORMAT_A12_B8_C8 (2<<2)
@@ -674,6 +680,7 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)#define GEN7_OABUFFER_STOP_RESUME_ENABLE (1<<1)#define GEN7_OABUFFER_RESUME (1<<0) +#define GEN8_OABUFFER_UDW _MMIO(0x23b4)#define GEN8_OABUFFER _MMIO(0x2b14)#define GEN7_OASTATUS1 _MMIO(0x2364)
@@ -692,7 +699,9 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)#define GEN8_OASTATUS_REPORT_LOST (1<<0)#define GEN8_OAHEADPTR _MMIO(0x2B0C) +#define GEN8_OAHEADPTR_MASK 0xffffffc0#define GEN8_OATAILPTR _MMIO(0x2B10) +#define GEN8_OATAILPTR_MASK 0xffffffc0#define OABUFFER_SIZE_128K (0<<3)#define OABUFFER_SIZE_256K (1<<3)
@@ -705,7 +714,17 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)#define OA_MEM_SELECT_GGTT (1<<0) +/* +* Flexible, Aggregate EU Counter Registers. +* Note: these aren't contiguous +*/ #define EU_PERF_CNTL0 _MMIO(0xe458) +#define EU_PERF_CNTL1 _MMIO(0xe558) +#define EU_PERF_CNTL2 _MMIO(0xe658) +#define EU_PERF_CNTL3 _MMIO(0xe758) +#define EU_PERF_CNTL4 _MMIO(0xe45c) +#define EU_PERF_CNTL5 _MMIO(0xe55c) +#define EU_PERF_CNTL6 _MMIO(0xe65c)#define GDT_CHICKEN_BITS _MMIO(0x9840)#define GT_NOA_ENABLE 0x00000080
@@ -2325,6 +2344,9 @@ enum skl_disp_power_wells {#define GEN8_RC_SEMA_IDLE_MSG_DISABLE (1 << 12)#define GEN8_FF_DOP_CLOCK_GATE_DISABLE (1<<10) +#define GEN6_RCS_PWR_FSM _MMIO(0x22ac) +#define GEN9_RCS_FE_FSM2 _MMIO(0x22a4) +/* Fuse readout registers for GT */ #define CHV_FUSE_GT _MMIO(VLV_DISPLAY_BASE + 0x2168)#define CHV_FGT_DISABLE_SS0 (1 << 10)
drivers/gpu/drm/i915/intel_lrc.c +5 -0
@@ -337,6 +337,8 @@ static u64 execlists_update_context(struct drm_i915_gem_request *rq)if (ppgtt && !i915_vm_is_48bit (&ppgtt->base))execlists_update_context_pdps (ppgtt, reg_state); +i915_oa_update_reg_state (rq->engine, rq->ctx, reg_state); +return ce->lrc_desc; }
@@ -1879,6 +1881,9 @@ static void execlists_init_reg_state(u32 *regs, regs[CTX_LRI_HEADER_2 ] =MI_LOAD_REGISTER_IMM (1 );CTX_REG (regs,CTX_R_PWR_CLK_STATE ,GEN8_R_PWR_CLK_STATE ,make_rpcs (dev_priv)); + +atomic_set (&ctx->engine[RCS ].oa_state_dirty,1 ); +i915_oa_update_reg_state (engine, ctx, regs); } }
include/uapi/drm/i915_drm.h +12 -7
@@ -1316,13 +1316,18 @@ struct drm_i915_gem_context_param { };enum drm_i915_oa_format { -I915_OA_FORMAT_A13 =1 , -I915_OA_FORMAT_A29 , -I915_OA_FORMAT_A13_B8_C8 , -I915_OA_FORMAT_B4_C8 , -I915_OA_FORMAT_A45_B8_C8 , -I915_OA_FORMAT_B4_C8_A16 , -I915_OA_FORMAT_C4_B8 , +I915_OA_FORMAT_A13 =1 ,/* HSW only */ +I915_OA_FORMAT_A29 ,/* HSW only */ +I915_OA_FORMAT_A13_B8_C8 ,/* HSW only */ +I915_OA_FORMAT_B4_C8 ,/* HSW only */ +I915_OA_FORMAT_A45_B8_C8 ,/* HSW only */ +I915_OA_FORMAT_B4_C8_A16 ,/* HSW only */ +I915_OA_FORMAT_C4_B8 ,/* HSW+ */ + +/* Gen8+ */ +I915_OA_FORMAT_A12 , +I915_OA_FORMAT_A12_B8_C8 , + I915_OA_FORMAT_A32u40_A4u32_B8_C8,I915_OA_FORMAT_MAX /* non-ABI */ };
--
2.11.0
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [PATCH v5 12/15] drm/i915/perf: Add OA unit support for Gen 8+
drivers/gpu/drm/i915/i915_perf.c static void hsw_disable_metric_set(struct drm_i915_private *dev_priv) · line 1584
Quoted 8 lines · Lionel Landwerlin
drivers/gpu/drm/i915/i915_perf.c +8
-0+static int gen8_configure_all_contexts (struct drm_i915_private *dev_priv) +{ +struct i915_gem_context *ctx; +int ret; + + ret =i915_mutex_lock_interruptible (&dev_priv->drm); +if (ret) +return ret;
Missed a switch to kernel context, to ensure that the last context is also saved to memory.
Quoted 24 lines
drivers/gpu/drm/i915/i915_perf.c +24
-0+ +/* The OA register config is setup through the context image. This image +* might be written to by the GPU on context switch (in particular on +* lite-restore). This means we can't safely update a context's image, +* if this context is scheduled/submitted to run on the GPU. +* +* We could emit the OA register config through the batch buffer but +* this might leave small interval of time where the OA unit is +* configured at an invalid sampling period. +* +* So far the best way to work around this issue seems to be draining +* the GPU from any submitted work. +*/ + ret =i915_gem_wait_for_idle (dev_priv, +I915_WAIT_INTERRUPTIBLE | +I915_WAIT_LOCKED ); +if (ret) { +mutex_unlock (&dev_priv->drm.struct_mutex); +return ret; + } + +/* Since execlist submission may be happening asynchronously here then +* we first mark existing contexts dirty before we update the current +* context so if any switches happen in the middle we can expect
drivers/gpu/drm/i915/i915_perf.c static void hsw_disable_metric_set(struct drm_i915_private *dev_priv) · line 1618
Quoted 10 lines
drivers/gpu/drm/i915/i915_perf.c +10
-0+* that the act of scheduling will have itself ensured a consistent +* OA state update. +*/ +list_for_each_entry (ctx, &dev_priv->context_list, link) { +/* The actual update of the register state context will happen +* the next time this logical ring is submitted. (See +* i915_oa_update_reg_state() which hooks into +* execlists_update_context()) +*/ +atomic_set (&ctx->engine[RCS ].oa_state_dirty,1 );
Considering that everything is now available to be modified, you can change the contexts right here and now and remove the checks from the much more common path.
-Chris
Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v6 12/15 drm/i915/perf: Add OA unit support for Gen 8+
From: Robert Bragg <[email protected]>
Enables access to OA unit metrics for BDW, CHV, SKL and BXT which all share (more-or-less) the same OA unit design.
Of particular note in comparison to Haswell: some OA unit HW config state has become per-context state and as a consequence it is somewhat more complicated to manage synchronous state changes from the cpu while there's no guarantee of what context (if any) is currently actively running on the gpu.
The periodic sampling frequency which can be particularly useful for system-wide analysis (as opposed to command stream synchronised MI_REPORT_PERF_COUNT commands) is perhaps the most surprising state to have become per-context save and restored (while the OABUFFER destination is still a shared, system-wide resource).
This support for gen8+ takes care to consider a number of timing challenges involved in synchronously updating per-context state primarily by programming all config state from the cpu and updating all current and saved contexts synchronously while the OA unit is still disabled.
The driver intentionally avoids depending on command streamer programming to update OA state considering the lack of synchronization between the automatic loading of OACTXCONTROL state (that includes the periodic sampling state and enable state) on context restore and the parsing of any general purpose BB the driver can control. I.e. this implementation is careful to avoid the possibility of a context restore temporarily enabling any out-of-date periodic sampling state. In addition to the risk of transiently-out-of-date state being loaded automatically; there are also internal HW latencies involved in the loading of MUX configurations which would be difficult to account for from the command streamer (and we only want to enable the unit when once the MUX configuration is complete).
Since the Gen8+ OA unit design no longer supports clock gating the unit off for a single given context (which effectively stopped any progress of counters while any other context was running) and instead supports tagging OA reports with a context ID for filtering on the CPU, it means we can no longer hide the system-wide progress of counters from a non-privileged application only interested in metrics for its own context. Although we could theoretically try and subtract the progress of other contexts before forwarding reports via read() we aren't in a position to filter reports captured via MI_REPORT_PERF_COUNT commands. As a result, for Gen8+, we always require the dev.i915.perf_stream_paranoid to be unset for any access to OA metrics if not root.
v5: Drain submitted requests when enabling metric set to ensure no
lite-restore erases the context image we just updated (Lionel)
v6: In addition to drain, switch to kernel context & update all
context in place (Chris)
--- drivers/gpu/drm/i915/i915_drv.h | 45 +- drivers/gpu/drm/i915/i915_perf.c | 957 ++++++++++++++++++++++++++++++++++++--- drivers/gpu/drm/i915/i915_reg.h | 22 + drivers/gpu/drm/i915/intel_lrc.c | 2 + include/uapi/drm/i915_drm.h | 19 +- 5 files changed, 952 insertions(+), 93 deletions(-)
drivers/gpu/drm/i915/i915_drv.h +30 -15
@@ -2064,9 +2064,17 @@ struct i915_oa_ops {void (*init_oa_buffer)(struct drm_i915_private *dev_priv);/** -* @enable_metric_set: Applies any MUX configuration to set up the -* Boolean and Custom (B/C) counters that are part of the counter -* reports being sampled. May apply system constraints such as +* @select_metric_set: The auto generated code that checks whether a +* requested OA config is applicable to the system and if so sets up +* the mux, oa and flex eu register config pointers according to the +* current dev_priv->perf.oa.metrics_set. +*/ +int (*select_metric_set)(struct drm_i915_private *dev_priv); + +/** +* @enable_metric_set: Selects and applies any MUX configuration to set +* up the Boolean and Custom (B/C) counters that are part of the +* counter reports being sampled. May apply system constraints such as * disabling EU clock gating as required. */ int (*enable_metric_set)(struct drm_i915_private *dev_priv);
@@ -2097,20 +2105,13 @@ struct i915_oa_ops { size_t *offset);/** -* @oa_buffer_check: Check for OA buffer data + update tail -* -* This is either called via fops or the poll check hrtimer (atomic -* ctx) without any locks taken. +* @oa_hw_tail_read: read the OA tail pointer register * -* It's safe to read OA config state here unlocked, assuming that this -* is only called while the stream is enabled, while the global OA -* configuration can't be modified. -* -* Efficiency is more important than avoiding some false positives -* here, which will be handled gracefully - likely resulting in an -* %EAGAIN error for userspace. +* In particular this enables us to share all the fiddly code for +* handling the OA unit tail pointer race that affects multiple +* generations. */ -bool (*oa_buffer_check)(struct drm_i915_private *dev_priv); +u32 (*oa_hw_tail_read)(struct drm_i915_private *dev_priv); };struct intel_cdclk_state {
@@ -2472,6 +2473,7 @@ struct drm_i915_private {struct {struct i915_vma *vma;u8 *vaddr; +u32 last_ctx_id;int format;int format_size;
@@ -2541,6 +2543,14 @@ struct drm_i915_private { } oa_buffer;u32 gen7_latched_oastatus1; +u32 ctx_oactxctrl_offset; +u32 ctx_flexeu0_offset; + +/* The RPT_ID/reason field for Gen8+ includes a bit +* to determine if the CTX ID in the report is valid +* but the specific bit differs between Gen 8 and 9 +*/ +u32 gen8_valid_ctx_bit;struct i915_oa_ops ops;const struct i915_oa_format *oa_formats;
@@ -2851,6 +2861,8 @@ intel_info(const struct drm_i915_private *dev_priv)#define IS_KBL_ULX (dev_priv) (INTEL_DEVID(dev_priv) == 0x590E || \ INTEL_DEVID(dev_priv) == 0x5915 || \ INTEL_DEVID(dev_priv) == 0x591E) +#define IS_SKL_GT2 (dev_priv) (IS_SKYLAKE(dev_priv) && \ + (INTEL_DEVID(dev_priv) & 0x00F0) == 0x0010)#define IS_SKL_GT3 (dev_priv) (IS_SKYLAKE(dev_priv) && \ (INTEL_DEVID(dev_priv) & 0x00F0) == 0x0020)#define IS_SKL_GT4 (dev_priv) (IS_SKYLAKE(dev_priv) && \
@@ -3615,6 +3627,9 @@ i915_gem_context_lookup_timeline(struct i915_gem_context *ctx,int i915_perf_open_ioctl (struct drm_device *dev,void *data,struct drm_file *file); +void i915_oa_init_reg_state (struct intel_engine_cs *engine, +struct i915_gem_context *ctx, +uint32_t *reg_state);/* i915_gem_evict.c */ int __must_check i915_gem_evict_something(struct i915_address_space *vm,
drivers/gpu/drm/i915/i915_perf.c +886 -71
1,146 lines not on this page · raw message
drivers/gpu/drm/i915/i915_reg.h +22 -0
@@ -653,6 +653,12 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)#define GEN8_OACTXID _MMIO(0x2364) +#define GEN8_OA_DEBUG _MMIO(0x2B04) +#define GEN9_OA_DEBUG_DISABLE_CLK_RATIO_REPORTS (1<<5) +#define GEN9_OA_DEBUG_INCLUDE_CLK_RATIO (1<<6) +#define GEN9_OA_DEBUG_DISABLE_GO_1_0_REPORTS (1<<2) +#define GEN9_OA_DEBUG_DISABLE_CTX_SWITCH_REPORTS (1<<1) +#define GEN8_OACONTROL _MMIO(0x2B00)#define GEN8_OA_REPORT_FORMAT_A12 (0<<2)#define GEN8_OA_REPORT_FORMAT_A12_B8_C8 (2<<2)
@@ -674,6 +680,7 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)#define GEN7_OABUFFER_STOP_RESUME_ENABLE (1<<1)#define GEN7_OABUFFER_RESUME (1<<0) +#define GEN8_OABUFFER_UDW _MMIO(0x23b4)#define GEN8_OABUFFER _MMIO(0x2b14)#define GEN7_OASTATUS1 _MMIO(0x2364)
@@ -692,7 +699,9 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)#define GEN8_OASTATUS_REPORT_LOST (1<<0)#define GEN8_OAHEADPTR _MMIO(0x2B0C) +#define GEN8_OAHEADPTR_MASK 0xffffffc0#define GEN8_OATAILPTR _MMIO(0x2B10) +#define GEN8_OATAILPTR_MASK 0xffffffc0#define OABUFFER_SIZE_128K (0<<3)#define OABUFFER_SIZE_256K (1<<3)
@@ -705,7 +714,17 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)#define OA_MEM_SELECT_GGTT (1<<0) +/* +* Flexible, Aggregate EU Counter Registers. +* Note: these aren't contiguous +*/ #define EU_PERF_CNTL0 _MMIO(0xe458) +#define EU_PERF_CNTL1 _MMIO(0xe558) +#define EU_PERF_CNTL2 _MMIO(0xe658) +#define EU_PERF_CNTL3 _MMIO(0xe758) +#define EU_PERF_CNTL4 _MMIO(0xe45c) +#define EU_PERF_CNTL5 _MMIO(0xe55c) +#define EU_PERF_CNTL6 _MMIO(0xe65c)#define GDT_CHICKEN_BITS _MMIO(0x9840)#define GT_NOA_ENABLE 0x00000080
@@ -2325,6 +2344,9 @@ enum skl_disp_power_wells {#define GEN8_RC_SEMA_IDLE_MSG_DISABLE (1 << 12)#define GEN8_FF_DOP_CLOCK_GATE_DISABLE (1<<10) +#define GEN6_RCS_PWR_FSM _MMIO(0x22ac) +#define GEN9_RCS_FE_FSM2 _MMIO(0x22a4) +/* Fuse readout registers for GT */ #define CHV_FUSE_GT _MMIO(VLV_DISPLAY_BASE + 0x2168)#define CHV_FGT_DISABLE_SS0 (1 << 10)
drivers/gpu/drm/i915/intel_lrc.c +2 -0
@@ -1879,6 +1879,8 @@ static void execlists_init_reg_state(u32 *regs, regs[CTX_LRI_HEADER_2 ] =MI_LOAD_REGISTER_IMM (1 );CTX_REG (regs,CTX_R_PWR_CLK_STATE ,GEN8_R_PWR_CLK_STATE ,make_rpcs (dev_priv)); + +i915_oa_init_reg_state (engine, ctx, regs); } }
include/uapi/drm/i915_drm.h +12 -7
@@ -1316,13 +1316,18 @@ struct drm_i915_gem_context_param { };enum drm_i915_oa_format { -I915_OA_FORMAT_A13 =1 , -I915_OA_FORMAT_A29 , -I915_OA_FORMAT_A13_B8_C8 , -I915_OA_FORMAT_B4_C8 , -I915_OA_FORMAT_A45_B8_C8 , -I915_OA_FORMAT_B4_C8_A16 , -I915_OA_FORMAT_C4_B8 , +I915_OA_FORMAT_A13 =1 ,/* HSW only */ +I915_OA_FORMAT_A29 ,/* HSW only */ +I915_OA_FORMAT_A13_B8_C8 ,/* HSW only */ +I915_OA_FORMAT_B4_C8 ,/* HSW only */ +I915_OA_FORMAT_A45_B8_C8 ,/* HSW only */ +I915_OA_FORMAT_B4_C8_A16 ,/* HSW only */ +I915_OA_FORMAT_C4_B8 ,/* HSW+ */ + +/* Gen8+ */ +I915_OA_FORMAT_A12 , +I915_OA_FORMAT_A12_B8_C8 , + I915_OA_FORMAT_A32u40_A4u32_B8_C8,I915_OA_FORMAT_MAX /* non-ABI */ };
--
2.11.0
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [PATCH v6 12/15] drm/i915/perf: Add OA unit support for Gen 8+
Hey Matt,
This commit had your reviewed-by on v4, are you still okay with it?
Thanks!
-
Lionel
Quoted 1,443 lines · Lionel Landwerlin
From: Robert Bragg <[email protected]>
Enables access to OA unit metrics for BDW, CHV, SKL and BXT which all share (more-or-less) the same OA unit design.
Of particular note in comparison to Haswell: some OA unit HW config state has become per-context state and as a consequence it is somewhat more complicated to manage synchronous state changes from the cpu while there's no guarantee of what context (if any) is currently actively running on the gpu.
The periodic sampling frequency which can be particularly useful for system-wide analysis (as opposed to command stream synchronised
MI_REPORT_PERF_COUNTcommands) is perhaps the most surprising state to have become per-context save and restored (while the OABUFFER destination is still a shared, system-wide resource).This support for gen8+ takes care to consider a number of timing challenges involved in synchronously updating per-context state primarily by programming all config state from the cpu and updating all current and saved contexts synchronously while the OA unit is still disabled.
The driver intentionally avoids depending on command streamer programming to update OA state considering the lack of synchronization between the automatic loading of OACTXCONTROL state (that includes the periodic sampling state and enable state) on context restore and the parsing of any general purpose BB the driver can control. I.e. this implementation is careful to avoid the possibility of a context restore temporarily enabling any out-of-date periodic sampling state. In addition to the risk of transiently-out-of-date state being loaded automatically; there are also internal HW latencies involved in the loading of MUX configurations which would be difficult to account for from the command streamer (and we only want to enable the unit when once the MUX configuration is complete).
Since the Gen8+ OA unit design no longer supports clock gating the unit off for a single given context (which effectively stopped any progress of counters while any other context was running) and instead supports tagging OA reports with a context ID for filtering on the CPU, it means we can no longer hide the system-wide progress of counters from a non-privileged application only interested in metrics for its own context. Although we could theoretically try and subtract the progress of other contexts before forwarding reports via
read()we aren't in a position to filter reports captured viaMI_REPORT_PERF_COUNTcommands. As a result, for Gen8+, we always require the dev.i915.perf_stream_paranoid to be unset for any access to OA metrics if not root.v5: Drain submitted requests when enabling metric set to ensure no
lite-restore erases the context image we just updated (Lionel)v6: In addition to drain, switch to kernel context & update all
context in place (Chris)Signed-off-by Robert Bragg <[email protected]>Signed-off-by Lionel Landwerlin <[email protected]>Reviewed-by Matthew Auld <[email protected]> \o/--- drivers/gpu/drm/i915/i915_drv.h | 45 +- drivers/gpu/drm/i915/i915_perf.c | 957 ++++++++++++++++++++++++++++++++++++--- drivers/gpu/drm/i915/i915_reg.h | 22 + drivers/gpu/drm/i915/intel_lrc.c | 2 + include/uapi/drm/i915_drm.h | 19 +- 5 files changed, 952 insertions(+), 93 deletions(-)drivers/gpu/drm/i915/i915_drv.h +30
-15@@ -2064,9 +2064,17 @@ struct i915_oa_ops {void (*init_oa_buffer)(struct drm_i915_private *dev_priv);/** -* @enable_metric_set: Applies any MUX configuration to set up the -* Boolean and Custom (B/C) counters that are part of the counter -* reports being sampled. May apply system constraints such as +* @select_metric_set: The auto generated code that checks whether a +* requested OA config is applicable to the system and if so sets up +* the mux, oa and flex eu register config pointers according to the +* current dev_priv->perf.oa.metrics_set. +*/ +int (*select_metric_set)(struct drm_i915_private *dev_priv); + +/** +* @enable_metric_set: Selects and applies any MUX configuration to set +* up the Boolean and Custom (B/C) counters that are part of the +* counter reports being sampled. May apply system constraints such as * disabling EU clock gating as required. */ int (*enable_metric_set)(struct drm_i915_private *dev_priv);@@ -2097,20 +2105,13 @@ struct i915_oa_ops { size_t *offset);/** -* @oa_buffer_check: Check for OA buffer data + update tail -* -* This is either called via fops or the poll check hrtimer (atomic -* ctx) without any locks taken. +* @oa_hw_tail_read: read the OA tail pointer register * -* It's safe to read OA config state here unlocked, assuming that this -* is only called while the stream is enabled, while the global OA -* configuration can't be modified. -* -* Efficiency is more important than avoiding some false positives -* here, which will be handled gracefully - likely resulting in an -* %EAGAIN error for userspace. +* In particular this enables us to share all the fiddly code for +* handling the OA unit tail pointer race that affects multiple +* generations. */ -bool (*oa_buffer_check)(struct drm_i915_private *dev_priv); +u32 (*oa_hw_tail_read)(struct drm_i915_private *dev_priv); };struct intel_cdclk_state {@@ -2472,6 +2473,7 @@ struct drm_i915_private {struct {struct i915_vma *vma;u8 *vaddr; +u32 last_ctx_id;int format;int format_size;@@ -2541,6 +2543,14 @@ struct drm_i915_private { } oa_buffer;u32 gen7_latched_oastatus1; +u32 ctx_oactxctrl_offset; +u32 ctx_flexeu0_offset; + +/* The RPT_ID/reason field for Gen8+ includes a bit +* to determine if the CTX ID in the report is valid +* but the specific bit differs between Gen 8 and 9 +*/ +u32 gen8_valid_ctx_bit;struct i915_oa_ops ops;const struct i915_oa_format *oa_formats;@@ -2851,6 +2861,8 @@ intel_info(const struct drm_i915_private *dev_priv)#define IS_KBL_ULX (dev_priv) (INTEL_DEVID(dev_priv) == 0x590E || \ INTEL_DEVID(dev_priv) == 0x5915 || \ INTEL_DEVID(dev_priv) == 0x591E) +#define IS_SKL_GT2 (dev_priv) (IS_SKYLAKE(dev_priv) && \ + (INTEL_DEVID(dev_priv) & 0x00F0) == 0x0010)#define IS_SKL_GT3 (dev_priv) (IS_SKYLAKE(dev_priv) && \ (INTEL_DEVID(dev_priv) & 0x00F0) == 0x0020)#define IS_SKL_GT4 (dev_priv) (IS_SKYLAKE(dev_priv) && \@@ -3615,6 +3627,9 @@ i915_gem_context_lookup_timeline(struct i915_gem_context *ctx,int i915_perf_open_ioctl (struct drm_device *dev,void *data,struct drm_file *file); +void i915_oa_init_reg_state (struct intel_engine_cs *engine, +struct i915_gem_context *ctx, +uint32_t *reg_state);/* i915_gem_evict.c */ int __must_check i915_gem_evict_something(struct i915_address_space *vm,drivers/gpu/drm/i915/i915_perf.c +886
-71@@ -196,6 +196,12 @@#include "i915_drv.h" #include "i915_oa_hsw.h" +#include "i915_oa_bdw.h" +#include "i915_oa_chv.h" +#include "i915_oa_sklgt2.h" +#include "i915_oa_sklgt3.h" +#include "i915_oa_sklgt4.h" +#include "i915_oa_bxt.h" /* HW requires this to be a power of two, between 128k and 16M, though driver * is currently generally designed assuming the largest 16M size is used such @@ -215,7 +221,7 @@* * Although this can be observed explicitly while copying reports to userspace * by checking for a zeroed report-id field in tail reports, we want to account -* for this earlier, as part of the _oa_buffer_check to avoid lots of redundant +* for this earlier, as part of the oa_buffer_check to avoid lots of redundant * read() attempts. * * In effect we define a tail pointer for reading that lags the real tail @@ -237,7 +243,7 @@* indicates that an updated tail pointer is needed. * * Most of the implementation details for this workaround are in -* gen7_oa_buffer_check_unlocked() and gen7_appand_oa_reports() +* oa_buffer_check_unlocked() and _append_oa_reports() * * Note for posterity: previously the driver used to define an effective tail * pointer that lagged the real pointer by a 'tail margin' measured in bytes @@ -272,6 +278,13 @@ static u32 i915_perf_stream_paranoid = true;#define INVALID_CTX_ID 0xffffffff +/* On Gen8+ automatically triggered OA reports include a 'reason' field... */ +#define OAREPORT_REASON_MASK 0x3f +#define OAREPORT_REASON_SHIFT 19 +#define OAREPORT_REASON_TIMER (1<<0) +#define OAREPORT_REASON_CTX_SWITCH (1<<3) +#define OAREPORT_REASON_CLK_RATIO (1<<5) +/* For sysctl proc_dointvec_minmax of i915_oa_max_sample_rate * @@ -303,6 +316,13 @@ static struct i915_oa_format hsw_oa_formats[I915_OA_FORMAT_MAX] = { [I915_OA_FORMAT_C4_B8] = { 7, 64 }, }; +static struct i915_oa_format gen8_plus_oa_formats[I915_OA_FORMAT_MAX ] = { + [I915_OA_FORMAT_A12 ] = {0 ,64 }, + [I915_OA_FORMAT_A12_B8_C8 ] = {2 ,128 }, + [I915_OA_FORMAT_A32u40_A4u32_B8_C8] = {5 ,256 }, + [I915_OA_FORMAT_C4_B8 ] = {7 ,64 }, +}; +#define SAMPLE_OA_REPORT (1<<0)/** @@ -332,8 +352,20 @@ struct perf_open_properties {int oa_period_exponent; }; +static u32 gen8_oa_hw_tail_read (struct drm_i915_private *dev_priv) +{ +return I915_READ (GEN8_OATAILPTR ) &GEN8_OATAILPTR_MASK ; +} + +static u32 gen7_oa_hw_tail_read (struct drm_i915_private *dev_priv) +{ +u32 oastatus1 =I915_READ (GEN7_OASTATUS1 ); + +return oastatus1 &GEN7_OASTATUS1_TAIL_MASK ; +} +/** -* gen7_oa_buffer_check_unlocked - check for data and update tail ptr state +* oa_buffer_check_unlocked - check for data and update tail ptr state * @dev_priv: i915 device instance * * This is either called via fops (for blocking reads in user ctx) or the poll @@ -356,12 +388,11 @@ struct perf_open_properties {* * Returns: %true if the OA buffer contains data, else %false */ -static bool gen7_oa_buffer_check_unlocked (struct drm_i915_private *dev_priv) +static bool oa_buffer_check_unlocked (struct drm_i915_private *dev_priv) {int report_size = dev_priv->perf.oa.oa_buffer.format_size;unsigned long flags;unsigned int aged_idx; -u32 oastatus1;u32 head, hw_tail, aged_tail, aging_tail;u64 now;@@ -381,8 +412,7 @@ static bool gen7_oa_buffer_check_unlocked(struct drm_i915_private *dev_priv) aged_tail = dev_priv->perf.oa.oa_buffer.tails[aged_idx].offset; aging_tail = dev_priv->perf.oa.oa_buffer.tails[!aged_idx].offset; - oastatus1 =I915_READ (GEN7_OASTATUS1 ); - hw_tail = oastatus1 &GEN7_OASTATUS1_TAIL_MASK ; + hw_tail = dev_priv->perf.oa.ops.oa_hw_tail_read (dev_priv);/* The tail pointer increases in 64 byte increments, * not in report_size steps... @@ -404,6 +434,7 @@ static bool gen7_oa_buffer_check_unlocked(struct drm_i915_private *dev_priv)if (aging_tail !=INVALID_TAIL_PTR && ((now - dev_priv->perf.oa.oa_buffer.aging_timestamp) >OA_TAIL_MARGIN_NSEC )) { + aged_idx ^=1 ; dev_priv->perf.oa.oa_buffer.aged_tail_idx = aged_idx;@@ -553,6 +584,287 @@ static int append_oa_sample(struct i915_perf_stream *stream,* * Returns: 0 on success, negative error code on failure. */ +static int gen8_append_oa_reports (struct i915_perf_stream *stream, +char __user *buf, +size_t count, +size_t *offset) +{ +struct drm_i915_private *dev_priv = stream->dev_priv; +int report_size = dev_priv->perf.oa.oa_buffer.format_size; +u8 *oa_buf_base = dev_priv->perf.oa.oa_buffer.vaddr; +u32 gtt_offset =i915_ggtt_offset (dev_priv->perf.oa.oa_buffer.vma); +u32 mask = (OA_BUFFER_SIZE -1 ); +size_t start_offset = *offset; +unsigned long flags; +unsigned int aged_tail_idx; +u32 head, tail; +u32 taken; +int ret =0 ; + +if (WARN_ON (!stream->enabled)) +return -EIO ; + +spin_lock_irqsave (&dev_priv->perf.oa.oa_buffer.ptr_lock, flags); + + head = dev_priv->perf.oa.oa_buffer.head; + aged_tail_idx = dev_priv->perf.oa.oa_buffer.aged_tail_idx; + tail = dev_priv->perf.oa.oa_buffer.tails[aged_tail_idx].offset; + +spin_unlock_irqrestore (&dev_priv->perf.oa.oa_buffer.ptr_lock, flags); + +/* An invalid tail pointer here means we're still waiting for the poll +* hrtimer callback to give us a pointer +*/ +if (tail ==INVALID_TAIL_PTR ) +return -EAGAIN ; + +/* NB: oa_buffer.head/tail include the gtt_offset which we don't want +* while indexing relative to oa_buf_base. +*/ + head -= gtt_offset; + tail -= gtt_offset; + +/* An out of bounds or misaligned head or tail pointer implies a driver +* bug since we validate + align the tail pointers we read from the +* hardware and we are in full control of the head pointer which should +* only be incremented by multiples of the report size (notably also +* all a power of two). +*/ +if (WARN_ONCE (head >OA_BUFFER_SIZE || head % report_size || + tail >OA_BUFFER_SIZE || tail % report_size, +"Inconsistent OA buffer pointers: head = %u, tail = %u\n" , + head, tail)) +return -EIO ; + + +for (/* none */ ; + (taken =OA_TAKEN (tail, head)); + head = (head + report_size) & mask) { +u8 *report = oa_buf_base + head; +u32 *report32 = (void *)report; +u32 ctx_id; +u32 reason; + +/* All the report sizes factor neatly into the buffer +* size so we never expect to see a report split +* between the beginning and end of the buffer. +* +* Given the initial alignment check a misalignment +* here would imply a driver bug that would result +* in an overrun. +*/ +if (WARN_ON ((OA_BUFFER_SIZE - head) < report_size)) { +DRM_ERROR ("Spurious OA head ptr: non-integral report offset\n" ); +break ; + } + +/* The reason field includes flags identifying what +* triggered this specific report (mostly timer +* triggered or e.g. due to a context switch). +* +* This field is never expected to be zero so we can +* check that the report isn't invalid before copying +* it to userspace... +*/ + reason = ((report32[0 ] >>OAREPORT_REASON_SHIFT ) & +OAREPORT_REASON_MASK ); +if (reason ==0 ) { +if (__ratelimit (&dev_priv->perf.oa.spurious_report_rs)) +DRM_NOTE ("Skipping spurious, invalid OA report\n" ); +continue ; + } + +/* XXX: Just keep the lower 21 bits for now since I'm not +* entirely sure if the HW touches any of the higher bits in +* this field +*/ + ctx_id = report32[2 ] &0x1fffff ; + +/* Squash whatever is in the CTX_ID field if it's marked as +* invalid to be sure we avoid false-positive, single-context +* filtering below... +* +* Note: that we don't clear the valid_ctx_bit so userspace can +* understand that the ID has been squashed by the kernel. +*/ +if (!(report32[0 ] & dev_priv->perf.oa.gen8_valid_ctx_bit)) + ctx_id = report32[2 ] =INVALID_CTX_ID ; + +/* NB: For Gen 8 the OA unit no longer supports clock gating +* off for a specific context and the kernel can't securely +* stop the counters from updating as system-wide / global +* values. +* +* Automatic reports now include a context ID so reports can be +* filtered on the cpu but it's not worth trying to +* automatically subtract/hide counter progress for other +* contexts while filtering since we can't stop userspace +* issuing MI_REPORT_PERF_COUNT commands which would still +* provide a side-band view of the real values. +* +* To allow userspace (such as Mesa/GL_INTEL_performance_query) +* to normalize counters for a single filtered context then it +* needs be forwarded bookend context-switch reports so that it +* can track switches in between MI_REPORT_PERF_COUNT commands +* and can itself subtract/ignore the progress of counters +* associated with other contexts. Note that the hardware +* automatically triggers reports when switching to a new +* context which are tagged with the ID of the newly active +* context. To avoid the complexity (and likely fragility) of +* reading ahead while parsing reports to try and minimize +* forwarding redundant context switch reports (i.e. between +* other, unrelated contexts) we simply elect to forward them +* all. +* +* We don't rely solely on the reason field to identify context +* switches since it's not-uncommon for periodic samples to +* identify a switch before any 'context switch' report. +*/ +if (!dev_priv->perf.oa.exclusive_stream->ctx || + dev_priv->perf.oa.specific_ctx_id == ctx_id || + (dev_priv->perf.oa.oa_buffer.last_ctx_id == + dev_priv->perf.oa.specific_ctx_id) || + reason &OAREPORT_REASON_CTX_SWITCH ) { + +/* While filtering for a single context we avoid +* leaking the IDs of other contexts. +*/ +if (dev_priv->perf.oa.exclusive_stream->ctx && + dev_priv->perf.oa.specific_ctx_id != ctx_id) { + report32[2 ] =INVALID_CTX_ID ; + } + + ret =append_oa_sample (stream, buf, count, offset, + report); +if (ret) +break ; + + dev_priv->perf.oa.oa_buffer.last_ctx_id = ctx_id; + } + +/* The above reason field sanity check is based on +* the assumption that the OA buffer is initially +* zeroed and we reset the field after copying so the +* check is still meaningful once old reports start +* being overwritten. +*/ + report32[0 ] =0 ; + } + +if (start_offset != *offset) { +spin_lock_irqsave (&dev_priv->perf.oa.oa_buffer.ptr_lock, flags); + +/* We removed the gtt_offset for the copy loop above, indexing +* relative to oa_buf_base so put back here... +*/ + head += gtt_offset; + +I915_WRITE (GEN8_OAHEADPTR , head &GEN8_OAHEADPTR_MASK ); + dev_priv->perf.oa.oa_buffer.head = head; + +spin_unlock_irqrestore (&dev_priv->perf.oa.oa_buffer.ptr_lock, flags); + } + +return ret; +} + +/** +* gen8_oa_read - copy status records then buffered OA reports +* @stream: An i915-perf stream opened for OA metrics +* @buf: destination buffer given by userspace +* @count: the number of bytes userspace wants to read +* @offset: (inout): the current position for writing into @buf +* +* Checks OA unit status registers and if necessary appends corresponding +* status records for userspace (such as for a buffer full condition) and then +* initiate appending any buffered OA reports. +* +* Updates @offset according to the number of bytes successfully copied into +* the userspace buffer. +* +* NB: some data may be successfully copied to the userspace buffer +* even if an error is returned, and this is reflected in the +* updated @offset. +* +* Returns: zero on success or a negative error code +*/ +static int gen8_oa_read (struct i915_perf_stream *stream, +char __user *buf, +size_t count, +size_t *offset) +{ +struct drm_i915_private *dev_priv = stream->dev_priv; +u32 oastatus; +int ret; + +if (WARN_ON (!dev_priv->perf.oa.oa_buffer.vaddr)) +return -EIO ; + + oastatus =I915_READ (GEN8_OASTATUS ); + +/* We treat OABUFFER_OVERFLOW as a significant error: +* +* Although theoretically we could handle this more gracefully +* sometimes, some Gens don't correctly suppress certain +* automatically triggered reports in this condition and so we +* have to assume that old reports are now being trampled +* over. +* +* Considering how we don't currently give userspace control +* over the OA buffer size and always configure a large 16MB +* buffer, then a buffer overflow does anyway likely indicate +* that something has gone quite badly wrong. +*/ +if (oastatus &GEN8_OASTATUS_OABUFFER_OVERFLOW ) { + ret =append_oa_status (stream, buf, count, offset, +DRM_I915_PERF_RECORD_OA_BUFFER_LOST ); +if (ret) +return ret; + +DRM_DEBUG ("OA buffer overflow (exponent = %d): force restart\n" , + dev_priv->perf.oa.period_exponent); + + dev_priv->perf.oa.ops.oa_disable (dev_priv); + dev_priv->perf.oa.ops.oa_enable (dev_priv); + +/* Note: .oa_enable() is expected to re-init the oabuffer +* and reset GEN8_OASTATUS for us +*/ + oastatus =I915_READ (GEN8_OASTATUS ); + } + +if (oastatus &GEN8_OASTATUS_REPORT_LOST ) { + ret =append_oa_status (stream, buf, count, offset, +DRM_I915_PERF_RECORD_OA_REPORT_LOST ); +if (ret) +return ret; +I915_WRITE (GEN8_OASTATUS , + oastatus & ~GEN8_OASTATUS_REPORT_LOST ); + } + +return gen8_append_oa_reports (stream, buf, count, offset); +} + +/** +* Copies all buffered OA reports into userspace read() buffer. +* @stream: An i915-perf stream opened for OA metrics +* @buf: destination buffer given by userspace +* @count: the number of bytes userspace wants to read +* @offset: (inout): the current position for writing into @buf +* +* Notably any error condition resulting in a short read (-%ENOSPC or +* -%EFAULT) will be returned even though one or more records may +* have been successfully copied. In this case it's up to the caller +* to decide if the error should be squashed before returning to +* userspace. +* +* Note: reports are consumed from the head, and appended to the +* tail, so the tail chases the head?... If you think that's mad +* and back-to-front you're not alone, but this follows the +* Gen PRM naming convention. +* +* Returns: 0 on success, negative error code on failure. +*/ static int gen7_append_oa_reports(struct i915_perf_stream *stream, char __user *buf, size_t count,@@ -733,7 +1045,8 @@ static int gen7_oa_read(struct i915_perf_stream *stream,if (ret)return ret; -DRM_DEBUG ("OA buffer overflow: force restart\n" ); +DRM_DEBUG ("OA buffer overflow (exponent = %d): force restart\n" , + dev_priv->perf.oa.period_exponent); dev_priv->perf.oa.ops.oa_disable (dev_priv); dev_priv->perf.oa.ops.oa_enable (dev_priv);@@ -776,7 +1089,7 @@ static int i915_oa_wait_unlocked(struct i915_perf_stream *stream)return -EIO ;return wait_event_interruptible (dev_priv->perf.oa.poll_wq, - dev_priv->perf.oa.ops.oa_buffer_check (dev_priv)); +oa_buffer_check_unlocked (dev_priv)); }/** @@ -833,33 +1146,37 @@ static int i915_oa_read(struct i915_perf_stream *stream,static int oa_get_render_ctx_id (struct i915_perf_stream *stream) {struct drm_i915_private *dev_priv = stream->dev_priv; -struct intel_engine_cs *engine = dev_priv->engine[RCS ]; -int ret; - ret =i915_mutex_lock_interruptible (&dev_priv->drm); -if (ret) -return ret; +if (i915.enable_execlists) + dev_priv->perf.oa.specific_ctx_id = stream->ctx->hw_id; +else { +struct intel_engine_cs *engine = dev_priv->engine[RCS ]; +int ret; -/* As the ID is the gtt offset of the context's vma we pin -* the vma to ensure the ID remains fixed. -* -* NB: implied RCS engine... -*/ - ret = engine->context_pin (engine, stream->ctx); -if (ret) - goto unlock; + ret =i915_mutex_lock_interruptible (&dev_priv->drm); +if (ret) +return ret; -/* Explicitly track the ID (instead of calling i915_ggtt_offset() -* on the fly) considering the difference with gen8+ and -* execlists -*/ - dev_priv->perf.oa.specific_ctx_id = -i915_ggtt_offset (stream->ctx->engine[engine->id].state); +/* As the ID is the gtt offset of the context's vma we pin +* the vma to ensure the ID remains fixed. +*/ + ret = engine->context_pin (engine, stream->ctx); +if (ret) { +mutex_unlock (&dev_priv->drm.struct_mutex); +return ret; + } -unlock: -mutex_unlock (&dev_priv->drm.struct_mutex); +/* Explicitly track the ID (instead of calling +* i915_ggtt_offset() on the fly) considering the difference +* with gen8+ and execlists +*/ + dev_priv->perf.oa.specific_ctx_id = +i915_ggtt_offset (stream->ctx->engine[engine->id].state); -return ret; +mutex_unlock (&dev_priv->drm.struct_mutex); + } + +return 0 ; }/** @@ -874,12 +1191,16 @@ static void oa_put_render_ctx_id(struct i915_perf_stream *stream)struct drm_i915_private *dev_priv = stream->dev_priv;struct intel_engine_cs *engine = dev_priv->engine[RCS ]; -mutex_lock (&dev_priv->drm.struct_mutex); +if (i915.enable_execlists) { + dev_priv->perf.oa.specific_ctx_id =INVALID_CTX_ID ; + }else { +mutex_lock (&dev_priv->drm.struct_mutex); - dev_priv->perf.oa.specific_ctx_id =INVALID_CTX_ID ; - engine->context_unpin (engine, stream->ctx); + dev_priv->perf.oa.specific_ctx_id =INVALID_CTX_ID ; + engine->context_unpin (engine, stream->ctx); -mutex_unlock (&dev_priv->drm.struct_mutex); +mutex_unlock (&dev_priv->drm.struct_mutex); + } } static void@@ -969,6 +1290,61 @@ static void gen7_init_oa_buffer(struct drm_i915_private *dev_priv) dev_priv->perf.oa.pollin = false; } +static void gen8_init_oa_buffer (struct drm_i915_private *dev_priv) +{ +u32 gtt_offset =i915_ggtt_offset (dev_priv->perf.oa.oa_buffer.vma); +unsigned long flags; + +spin_lock_irqsave (&dev_priv->perf.oa.oa_buffer.ptr_lock, flags); + +I915_WRITE (GEN8_OASTATUS ,0 ); +I915_WRITE (GEN8_OAHEADPTR , gtt_offset); + dev_priv->perf.oa.oa_buffer.head = gtt_offset; + +I915_WRITE (GEN8_OABUFFER_UDW ,0 ); + +/* PRM says: +* +* "This MMIO must be set before the OATAILPTR +* register and after the OAHEADPTR register. This is +* to enable proper functionality of the overflow +* bit." +*/ +I915_WRITE (GEN8_OABUFFER , gtt_offset | +OABUFFER_SIZE_16M |OA_MEM_SELECT_GGTT ); +I915_WRITE (GEN8_OATAILPTR , gtt_offset &GEN8_OATAILPTR_MASK ); + +/* Mark that we need updated tail pointers to read from... */ + dev_priv->perf.oa.oa_buffer.tails[0 ].offset =INVALID_TAIL_PTR ; + dev_priv->perf.oa.oa_buffer.tails[1 ].offset =INVALID_TAIL_PTR ; + +/* Reset state used to recognise context switches, affecting which +* reports we will forward to userspace while filtering for a single +* context. +*/ + dev_priv->perf.oa.oa_buffer.last_ctx_id =INVALID_CTX_ID ; + +spin_unlock_irqrestore (&dev_priv->perf.oa.oa_buffer.ptr_lock, flags); + +/* NB: although the OA buffer will initially be allocated +* zeroed via shmfs (and so this memset is redundant when +* first allocating), we may re-init the OA buffer, either +* when re-enabling a stream or in error/reset paths. +* +* The reason we clear the buffer for each re-init is for the +* sanity check in gen8_append_oa_reports() that looks at the +* reason field to make sure it's non-zero which relies on +* the assumption that new reports are being written to zeroed +* memory... +*/ +memset (dev_priv->perf.oa.oa_buffer.vaddr,0 ,OA_BUFFER_SIZE ); + +/* Maybe make ->pollin per-stream state if we support multiple +* concurrent streams in the future. +*/ + dev_priv->perf.oa.pollin = false; +} +static int alloc_oa_buffer (struct drm_i915_private *dev_priv) {struct drm_i915_gem_object *bo;@@ -1113,6 +1489,280 @@ static void hsw_disable_metric_set(struct drm_i915_private *dev_priv) ~GT_NOA_ENABLE)); } +/* +* From Broadwell PRM, 3D-Media-GPGPU -> Register State Context +* +* MMIO reads or writes to any of the registers listed in the +* “Register State Context image” subsections through HOST/IA +* MMIO interface for debug purposes must follow the steps below: +* +* - SW should set the Force Wakeup bit to prevent GT from entering C6. +* - Write 0x2050[31:0] = 0x00010001 (disable sequence). +* - Disable IDLE messaging in CS (Write 0x2050[31:0] = 0x00010001). +* - BDW: Poll/Wait for register bits of 0x22AC[6:0] turn to 0x30 value. +* - SKL+: Poll/Wait for register bits of 0x22A4[6:0] turn to 0x30 value. +* - Read/Write to desired MMIO registers. +* - Enable IDLE messaging in CS (Write 0x2050[31:0] = 0x00010000). +* - Force Wakeup bit should be reset to enable C6 entry. +* +* XXX: don't nest or overlap calls to this API, it has no ref +* counting to track how many entities require the RCS to be +* blocked from being idle. +*/ +static int gen8_begin_ctx_mmio (struct drm_i915_private *dev_priv) +{ +i915_reg_t fsm_reg =INTEL_GEN (dev_priv) >8 ? +GEN9_RCS_FE_FSM2 :GEN6_RCS_PWR_FSM ; +int ret =0 ; + +/* There's only no active context while idle in execlist mode +* (though we shouldn't be using this in any other case) +*/ +if (WARN_ON (!i915.enable_execlists)) +return -EINVAL ; + +intel_uncore_forcewake_get (dev_priv,FORCEWAKE_RENDER ); + +I915_WRITE (GEN6_RC_SLEEP_PSMI_CONTROL , +_MASKED_BIT_ENABLE (GEN6_PSMI_SLEEP_MSG_DISABLE )); + +/* Note: we don't currently have a good handle on the maximum +* latency for this wake up so while we only need to hold rcs +* busy from process context we at least keep the waiting +* interruptible... +*/ + ret =intel_wait_for_register (dev_priv, fsm_reg,0x3f ,0x30 ,50 ); +if (ret == -ETIMEDOUT ) +DRM_ERROR ("Timeout waiting to be able to begin per-ctx mmio\n" ); + +if (ret) +intel_uncore_forcewake_put (dev_priv,FORCEWAKE_RENDER ); + +return ret; +} + +static void gen8_end_ctx_mmio (struct drm_i915_private *dev_priv) +{ +if (WARN_ON (!i915.enable_execlists)) +return ; + +I915_WRITE (GEN6_RC_SLEEP_PSMI_CONTROL , +_MASKED_BIT_DISABLE (GEN6_PSMI_SLEEP_MSG_DISABLE )); +intel_uncore_forcewake_put (dev_priv,FORCEWAKE_RENDER ); +} + +/* NB: It must always remain pointer safe to run this even if the OA unit +* has been disabled. +* +* It's fine to put out-of-date values into these per-context registers +* in the case that the OA unit has been disabled. +*/ +static void gen8_update_reg_state_unlocked (struct i915_gem_context *ctx, +u32 *reg_state) +{ +struct drm_i915_private *dev_priv = ctx->i915; +const struct i915_oa_reg *flex_regs = dev_priv->perf.oa.flex_regs; +int n_flex_regs = dev_priv->perf.oa.flex_regs_len; +u32 ctx_oactxctrl = dev_priv->perf.oa.ctx_oactxctrl_offset; +u32 ctx_flexeu0 = dev_priv->perf.oa.ctx_flexeu0_offset; +/* The MMIO offsets for Flex EU registers aren't contiguous */ +u32 flex_mmio[] = { +i915_mmio_reg_offset (EU_PERF_CNTL0 ), +i915_mmio_reg_offset (EU_PERF_CNTL1 ), +i915_mmio_reg_offset (EU_PERF_CNTL2 ), +i915_mmio_reg_offset (EU_PERF_CNTL3 ), +i915_mmio_reg_offset (EU_PERF_CNTL4 ), +i915_mmio_reg_offset (EU_PERF_CNTL5 ), +i915_mmio_reg_offset (EU_PERF_CNTL6 ), + }; +int i; + + reg_state[ctx_oactxctrl] =i915_mmio_reg_offset (GEN8_OACTXCONTROL ); + reg_state[ctx_oactxctrl+1 ] = (dev_priv->perf.oa.period_exponent << +GEN8_OA_TIMER_PERIOD_SHIFT ) | + (dev_priv->perf.oa.periodic ? +GEN8_OA_TIMER_ENABLE :0 ) | +GEN8_OA_COUNTER_RESUME ; + +for (i =0 ; i <ARRAY_SIZE (flex_mmio); i++) { +u32 state_offset = ctx_flexeu0 + i *2 ; +u32 mmio = flex_mmio[i]; + +/* This arbitrary default will select the 'EU FPU0 Pipeline +* Active' event. In the future it's anticipated that there +* will be an explicit 'No Event' we can select, but not yet... +*/ +u32 value =0 ; +int j; + +for (j =0 ; j < n_flex_regs; j++) { +if (i915_mmio_reg_offset (flex_regs[j].addr) == mmio) { + value = flex_regs[j].value; +break ; + } + } + + reg_state[state_offset] = mmio; + reg_state[state_offset+1 ] = value; + } +} + +/* Manages updating the per-context aspects of the OA stream +* configuration across all contexts. +* +* The awkward consideration here is that OACTXCONTROL controls the +* exponent for periodic sampling which is primarily used for system +* wide profiling where we'd like a consistent sampling period even in +* the face of context switches. +* +* Our approach of updating the register state context (as opposed to +* say using a workaround batch buffer) ensures that the hardware +* won't automatically reload an out-of-date timer exponent even +* transiently before a WA BB could be parsed. +* +* This function needs to: +* - Ensure the currently running context's per-context OA state is +* updated +* - Ensure that all existing contexts will have the correct per-context +* OA state if they are scheduled for use. +* - Ensure any new contexts will be initialized with the correct +* per-context OA state. +* +* Note: it's only the RCS/Render context that has any OA state. +*/ +static int gen8_configure_all_contexts (struct drm_i915_private *dev_priv) +{ +struct i915_gem_context *ctx; +int ret; + + ret =i915_mutex_lock_interruptible (&dev_priv->drm); +if (ret) +return ret; + +/* Switch away from any user context. */ + ret =i915_gem_switch_to_kernel_context (dev_priv); +if (ret) +return ret; + +/* The OA register config is setup through the context image. This image +* might be written to by the GPU on context switch (in particular on +* lite-restore). This means we can't safely update a context's image, +* if this context is scheduled/submitted to run on the GPU. +* +* We could emit the OA register config through the batch buffer but +* this might leave small interval of time where the OA unit is +* configured at an invalid sampling period. +* +* So far the best way to work around this issue seems to be draining +* the GPU from any submitted work. +*/ + ret =i915_gem_wait_for_idle (dev_priv, +I915_WAIT_INTERRUPTIBLE | +I915_WAIT_LOCKED ); +if (ret) { +mutex_unlock (&dev_priv->drm.struct_mutex); +return ret; + } + +/* Update all contexts now that we've stalled the submission. */ +list_for_each_entry (ctx, &dev_priv->context_list, link) { +if (!ctx->engine[RCS ].initialised) +continue ; + +gen8_update_reg_state_unlocked (ctx, + ctx->engine[RCS ].lrc_reg_state); + } + +mutex_unlock (&dev_priv->drm.struct_mutex); + +/* Now update the current context. +* +* Note: Using MMIO to update per-context registers requires +* some extra care... +*/ + ret =gen8_begin_ctx_mmio (dev_priv); +if (ret) { +DRM_ERROR ("Failed to bring RCS out of idle to update current ctx OA state\n" ); +return ret; + } + +I915_WRITE (GEN8_OACTXCONTROL , ((dev_priv->perf.oa.period_exponent << +GEN8_OA_TIMER_PERIOD_SHIFT ) | + (dev_priv->perf.oa.periodic ? +GEN8_OA_TIMER_ENABLE :0 ) | +GEN8_OA_COUNTER_RESUME )); + +config_oa_regs (dev_priv, dev_priv->perf.oa.flex_regs, + dev_priv->perf.oa.flex_regs_len); + +gen8_end_ctx_mmio (dev_priv); + +return 0 ; +} + +static int gen8_enable_metric_set (struct drm_i915_private *dev_priv) +{ +int ret = dev_priv->perf.oa.ops.select_metric_set (dev_priv); + +if (ret) +return ret; + +/* We disable slice/unslice clock ratio change reports on SKL since +* they are too noisy. The HW generates a lot of redundant reports +* where the ratio hasn't really changed causing a lot of redundant +* work to processes and increasing the chances we'll hit buffer +* overruns. +* +* Although we don't currently use the 'disable overrun' OABUFFER +* feature it's worth noting that clock ratio reports have to be +* disabled before considering to use that feature since the HW doesn't +* correctly block these reports. +* +* Currently none of the high-level metrics we have depend on knowing +* this ratio to normalize. +* +* Note: This register is not power context saved and restored, but +* that's OK considering that we disable RC6 while the OA unit is +* enabled. +* +* The _INCLUDE_CLK_RATIO bit allows the slice/unslice frequency to +* be read back from automatically triggered reports, as part of the +* RPT_ID field. +*/ +if (IS_SKYLAKE (dev_priv) ||IS_BROXTON (dev_priv)) { +I915_WRITE (GEN8_OA_DEBUG , +_MASKED_BIT_ENABLE (GEN9_OA_DEBUG_DISABLE_CLK_RATIO_REPORTS | +GEN9_OA_DEBUG_INCLUDE_CLK_RATIO )); + } + +I915_WRITE (GDT_CHICKEN_BITS ,0xA0 ); +config_oa_regs (dev_priv, dev_priv->perf.oa.mux_regs, + dev_priv->perf.oa.mux_regs_len); +I915_WRITE (GDT_CHICKEN_BITS ,0x80 ); + +/* It takes a fairly long time for a new MUX configuration to +* be be applied after these register writes. This delay +* duration is take from Haswell (derived empirically based on +* the render_basic config) but hopefully it covers the +* maximum configuration latency for Gen8+ too... +*/ +usleep_range (15000 ,20000 ); + +config_oa_regs (dev_priv, dev_priv->perf.oa.b_counter_regs, + dev_priv->perf.oa.b_counter_regs_len); + + ret =gen8_configure_all_contexts (dev_priv); +if (ret) +return ret; + +return 0 ; +} + +static void gen8_disable_metric_set (struct drm_i915_private *dev_priv) +{ +/* NOP */ +} +static void gen7_update_oacontrol_locked (struct drm_i915_private *dev_priv) {lockdep_assert_held (&dev_priv->perf.hook_lock);@@ -1157,6 +1807,29 @@ static void gen7_oa_enable(struct drm_i915_private *dev_priv)spin_unlock_irqrestore (&dev_priv->perf.hook_lock, flags); } +static void gen8_oa_enable (struct drm_i915_private *dev_priv) +{ +u32 report_format = dev_priv->perf.oa.oa_buffer.format; + +/* Reset buf pointers so we don't forward reports from before now. +* +* Think carefully if considering trying to avoid this, since it +* also ensures status flags and the buffer itself are cleared +* in error paths, and we have checks for invalid reports based +* on the assumption that certain fields are written to zeroed +* memory which this helps maintains. +*/ +gen8_init_oa_buffer (dev_priv); + +/* Note: we don't rely on the hardware to perform single context +* filtering and instead filter on the cpu based on the context-id +* field of reports +*/ +I915_WRITE (GEN8_OACONTROL , (report_format << +GEN8_OA_REPORT_FORMAT_SHIFT ) | +GEN8_OA_COUNTER_ENABLE ); +} +/** * i915_oa_stream_enable - handle `I915_PERF_IOCTL_ENABLE` for OA stream * @stream: An i915 perf stream opened for OA metrics @@ -1183,6 +1856,11 @@ static void gen7_oa_disable(struct drm_i915_private *dev_priv)I915_WRITE (GEN7_OACONTROL ,0 ); } +static void gen8_oa_disable (struct drm_i915_private *dev_priv) +{ +I915_WRITE (GEN8_OACONTROL ,0 ); +} +/** * i915_oa_stream_disable - handle `I915_PERF_IOCTL_DISABLE` for OA stream * @stream: An i915 perf stream opened for OA metrics @@ -1361,6 +2039,21 @@ static int i915_oa_stream_init(struct i915_perf_stream *stream,return ret; } +void i915_oa_init_reg_state (struct intel_engine_cs *engine, +struct i915_gem_context *ctx, +u32 *reg_state) +{ +struct drm_i915_private *dev_priv = engine->i915; + +if (engine->id !=RCS ) +return ; + +if (!dev_priv->perf.initialized) +return ; + +gen8_update_reg_state_unlocked (ctx, reg_state); +} +/** * i915_perf_read_locked - &i915_perf_stream_ops->read with error normalisation * @stream: An i915 perf stream @@ -1486,7 +2179,7 @@ static enum hrtimer_restart oa_poll_check_timer_cb(struct hrtimer *hrtimer)container_of (hrtimer,typeof (*dev_priv), perf.oa.poll_check_timer); -if (dev_priv->perf.oa.ops.oa_buffer_check (dev_priv)) { +if (oa_buffer_check_unlocked (dev_priv)) { dev_priv->perf.oa.pollin = true;wake_up (&dev_priv->perf.oa.poll_wq); }@@ -1775,6 +2468,7 @@ i915_perf_open_ioctl_locked(struct drm_i915_private *dev_priv,struct i915_gem_context *specific_ctx =NULL ;struct i915_perf_stream *stream =NULL ;unsigned long f_flags =0 ; +bool privileged_op = true;int stream_fd;int ret;@@ -1792,12 +2486,28 @@ i915_perf_open_ioctl_locked(struct drm_i915_private *dev_priv, } } +/* On Haswell the OA unit supports clock gating off for a specific +* context and in this mode there's no visibility of metrics for the +* rest of the system, which we consider acceptable for a +* non-privileged client. +* +* For Gen8+ the OA unit no longer supports clock gating off for a +* specific context and the kernel can't securely stop the counters +* from updating as system-wide / global values. Even though we can +* filter reports based on the included context ID we can't block +* clients from seeing the raw / global counter values via +* MI_REPORT_PERF_COUNT commands and so consider it a privileged op to +* enable the OA unit by default. +*/ +if (IS_HASWELL (dev_priv) && specific_ctx) + privileged_op = false; +/* Similar to perf's kernel.perf_paranoid_cpu sysctl option * we check a dev.i915.perf_stream_paranoid sysctl option * to determine if it's ok to access system wide OA counters * without CAP_SYS_ADMIN privileges. */ -if (!specific_ctx && +if (privileged_op && i915_perf_stream_paranoid && !capable (CAP_SYS_ADMIN )) {DRM_DEBUG ("Insufficient privileges to open system-wide i915 perf stream\n" ); ret = -EACCES ;@@ -2069,9 +2779,6 @@ int i915_perf_open_ioctl(struct drm_device *dev, void *data,*/ void i915_perf_register (struct drm_i915_private *dev_priv) { -if (!IS_HASWELL (dev_priv)) -return ; -if (!dev_priv->perf.initialized)return ;@@ -2087,11 +2794,38 @@ void i915_perf_register(struct drm_i915_private *dev_priv)if (!dev_priv->perf.metrics_kobj) goto exit; -if (i915_perf_register_sysfs_hsw (dev_priv)) { -kobject_put (dev_priv->perf.metrics_kobj); - dev_priv->perf.metrics_kobj =NULL ; +if (IS_HASWELL (dev_priv)) { +if (i915_perf_register_sysfs_hsw (dev_priv)) + goto sysfs_error; + }else if (IS_BROADWELL (dev_priv)) { +if (i915_perf_register_sysfs_bdw (dev_priv)) + goto sysfs_error; + }else if (IS_CHERRYVIEW (dev_priv)) { +if (i915_perf_register_sysfs_chv (dev_priv)) + goto sysfs_error; + }else if (IS_SKYLAKE (dev_priv)) { +if (IS_SKL_GT2 (dev_priv)) { +if (i915_perf_register_sysfs_sklgt2 (dev_priv)) + goto sysfs_error; + }else if (IS_SKL_GT3 (dev_priv)) { +if (i915_perf_register_sysfs_sklgt3 (dev_priv)) + goto sysfs_error; + }else if (IS_SKL_GT4 (dev_priv)) { +if (i915_perf_register_sysfs_sklgt4 (dev_priv)) + goto sysfs_error; + }else + goto sysfs_error; + }else if (IS_BROXTON (dev_priv)) { +if (i915_perf_register_sysfs_bxt (dev_priv)) + goto sysfs_error; } + goto exit; + +sysfs_error: +kobject_put (dev_priv->perf.metrics_kobj); + dev_priv->perf.metrics_kobj =NULL ; + exit:mutex_unlock (&dev_priv->perf.lock); }@@ -2107,13 +2841,24 @@ void i915_perf_register(struct drm_i915_private *dev_priv)*/ void i915_perf_unregister (struct drm_i915_private *dev_priv) { -if (!IS_HASWELL (dev_priv)) -return ; -if (!dev_priv->perf.metrics_kobj)return ; -i915_perf_unregister_sysfs_hsw (dev_priv); +if (IS_HASWELL (dev_priv)) +i915_perf_unregister_sysfs_hsw (dev_priv); +else if (IS_BROADWELL (dev_priv)) +i915_perf_unregister_sysfs_bdw (dev_priv); +else if (IS_CHERRYVIEW (dev_priv)) +i915_perf_unregister_sysfs_chv (dev_priv); +else if (IS_SKYLAKE (dev_priv)) { +if (IS_SKL_GT2 (dev_priv)) +i915_perf_unregister_sysfs_sklgt2 (dev_priv); +else if (IS_SKL_GT3 (dev_priv)) +i915_perf_unregister_sysfs_sklgt3 (dev_priv); +else if (IS_SKL_GT4 (dev_priv)) +i915_perf_unregister_sysfs_sklgt4 (dev_priv); + }else if (IS_BROXTON (dev_priv)) +i915_perf_unregister_sysfs_bxt (dev_priv);kobject_put (dev_priv->perf.metrics_kobj); dev_priv->perf.metrics_kobj =NULL ;@@ -2172,36 +2917,105 @@ static struct ctl_table dev_root[] = {*/ void i915_perf_init (struct drm_i915_private *dev_priv) { -if (!IS_HASWELL (dev_priv)) -return ; - -hrtimer_init (&dev_priv->perf.oa.poll_check_timer, -CLOCK_MONOTONIC ,HRTIMER_MODE_REL ); - dev_priv->perf.oa.poll_check_timer.function = oa_poll_check_timer_cb; -init_waitqueue_head (&dev_priv->perf.oa.poll_wq); + dev_priv->perf.oa.n_builtin_sets =0 ; + +if (IS_HASWELL (dev_priv)) { + dev_priv->perf.oa.ops.init_oa_buffer = gen7_init_oa_buffer; + dev_priv->perf.oa.ops.enable_metric_set = hsw_enable_metric_set; + dev_priv->perf.oa.ops.disable_metric_set = hsw_disable_metric_set; + dev_priv->perf.oa.ops.oa_enable = gen7_oa_enable; + dev_priv->perf.oa.ops.oa_disable = gen7_oa_disable; + dev_priv->perf.oa.ops.read = gen7_oa_read; + dev_priv->perf.oa.ops.oa_hw_tail_read = + gen7_oa_hw_tail_read; + + dev_priv->perf.oa.oa_formats = hsw_oa_formats; + + dev_priv->perf.oa.n_builtin_sets = + i915_oa_n_builtin_metric_sets_hsw; + }else if (i915.enable_execlists) { +/* Note: that although we could theoretically also support the +* legacy ringbuffer mode on BDW (and earlier iterations of +* this driver, before upstreaming did this) it didn't seem +* worth the complexity to maintain now that BDW+ enable +* execlist mode by default. +*/ -INIT_LIST_HEAD (&dev_priv->perf.streams); -mutex_init (&dev_priv->perf.lock); -spin_lock_init (&dev_priv->perf.hook_lock); -spin_lock_init (&dev_priv->perf.oa.oa_buffer.ptr_lock); +if (IS_GEN8 (dev_priv)) { + dev_priv->perf.oa.ctx_oactxctrl_offset =0x120 ; + dev_priv->perf.oa.ctx_flexeu0_offset =0x2ce ; + dev_priv->perf.oa.gen8_valid_ctx_bit = (1 <<25 ); + +if (IS_BROADWELL (dev_priv)) { + dev_priv->perf.oa.n_builtin_sets = + i915_oa_n_builtin_metric_sets_bdw; + dev_priv->perf.oa.ops.select_metric_set = + i915_oa_select_metric_set_bdw; + }else if (IS_CHERRYVIEW (dev_priv)) { + dev_priv->perf.oa.n_builtin_sets = + i915_oa_n_builtin_metric_sets_chv; + dev_priv->perf.oa.ops.select_metric_set = + i915_oa_select_metric_set_chv; + } + }else if (IS_GEN9 (dev_priv)) { + dev_priv->perf.oa.ctx_oactxctrl_offset =0x128 ; + dev_priv->perf.oa.ctx_flexeu0_offset =0x3de ; + dev_priv->perf.oa.gen8_valid_ctx_bit = (1 <<16 ); + +if (IS_SKL_GT2 (dev_priv)) { + dev_priv->perf.oa.n_builtin_sets = + i915_oa_n_builtin_metric_sets_sklgt2; + dev_priv->perf.oa.ops.select_metric_set = + i915_oa_select_metric_set_sklgt2; + }else if (IS_SKL_GT3 (dev_priv)) { + dev_priv->perf.oa.n_builtin_sets = + i915_oa_n_builtin_metric_sets_sklgt3; + dev_priv->perf.oa.ops.select_metric_set = + i915_oa_select_metric_set_sklgt3; + }else if (IS_SKL_GT4 (dev_priv)) { + dev_priv->perf.oa.n_builtin_sets = + i915_oa_n_builtin_metric_sets_sklgt4; + dev_priv->perf.oa.ops.select_metric_set = + i915_oa_select_metric_set_sklgt4; + }else if (IS_BROXTON (dev_priv)) { + dev_priv->perf.oa.n_builtin_sets = + i915_oa_n_builtin_metric_sets_bxt; + dev_priv->perf.oa.ops.select_metric_set = + i915_oa_select_metric_set_bxt; + } + } - dev_priv->perf.oa.ops.init_oa_buffer = gen7_init_oa_buffer; - dev_priv->perf.oa.ops.enable_metric_set = hsw_enable_metric_set; - dev_priv->perf.oa.ops.disable_metric_set = hsw_disable_metric_set; - dev_priv->perf.oa.ops.oa_enable = gen7_oa_enable; - dev_priv->perf.oa.ops.oa_disable = gen7_oa_disable; - dev_priv->perf.oa.ops.read = gen7_oa_read; - dev_priv->perf.oa.ops.oa_buffer_check = - gen7_oa_buffer_check_unlocked; +if (dev_priv->perf.oa.n_builtin_sets) { + dev_priv->perf.oa.ops.init_oa_buffer = gen8_init_oa_buffer; + dev_priv->perf.oa.ops.enable_metric_set = + gen8_enable_metric_set; + dev_priv->perf.oa.ops.disable_metric_set = + gen8_disable_metric_set; + dev_priv->perf.oa.ops.oa_enable = gen8_oa_enable; + dev_priv->perf.oa.ops.oa_disable = gen8_oa_disable; + dev_priv->perf.oa.ops.read = gen8_oa_read; + dev_priv->perf.oa.ops.oa_hw_tail_read = + gen8_oa_hw_tail_read; + + dev_priv->perf.oa.oa_formats = gen8_plus_oa_formats; + } + } - dev_priv->perf.oa.oa_formats = hsw_oa_formats; +if (dev_priv->perf.oa.n_builtin_sets) { +hrtimer_init (&dev_priv->perf.oa.poll_check_timer, +CLOCK_MONOTONIC ,HRTIMER_MODE_REL ); + dev_priv->perf.oa.poll_check_timer.function = oa_poll_check_timer_cb; +init_waitqueue_head (&dev_priv->perf.oa.poll_wq); - dev_priv->perf.oa.n_builtin_sets = - i915_oa_n_builtin_metric_sets_hsw; +INIT_LIST_HEAD (&dev_priv->perf.streams); +mutex_init (&dev_priv->perf.lock); +spin_lock_init (&dev_priv->perf.hook_lock); +spin_lock_init (&dev_priv->perf.oa.oa_buffer.ptr_lock); - dev_priv->perf.sysctl_header =register_sysctl_table (dev_root); + dev_priv->perf.sysctl_header =register_sysctl_table (dev_root); - dev_priv->perf.initialized = true; + dev_priv->perf.initialized = true; + } }/** @@ -2216,5 +3030,6 @@ void i915_perf_fini(struct drm_i915_private *dev_priv)unregister_sysctl_table (dev_priv->perf.sysctl_header);memset (&dev_priv->perf.oa.ops,0 ,sizeof (dev_priv->perf.oa.ops)); + dev_priv->perf.initialized = false; }drivers/gpu/drm/i915/i915_reg.h +22
-0@@ -653,6 +653,12 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)#define GEN8_OACTXID _MMIO(0x2364) +#define GEN8_OA_DEBUG _MMIO(0x2B04) +#define GEN9_OA_DEBUG_DISABLE_CLK_RATIO_REPORTS (1<<5) +#define GEN9_OA_DEBUG_INCLUDE_CLK_RATIO (1<<6) +#define GEN9_OA_DEBUG_DISABLE_GO_1_0_REPORTS (1<<2) +#define GEN9_OA_DEBUG_DISABLE_CTX_SWITCH_REPORTS (1<<1) +#define GEN8_OACONTROL _MMIO(0x2B00)#define GEN8_OA_REPORT_FORMAT_A12 (0<<2)#define GEN8_OA_REPORT_FORMAT_A12_B8_C8 (2<<2)@@ -674,6 +680,7 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)#define GEN7_OABUFFER_STOP_RESUME_ENABLE (1<<1)#define GEN7_OABUFFER_RESUME (1<<0) +#define GEN8_OABUFFER_UDW _MMIO(0x23b4)#define GEN8_OABUFFER _MMIO(0x2b14)#define GEN7_OASTATUS1 _MMIO(0x2364)@@ -692,7 +699,9 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)#define GEN8_OASTATUS_REPORT_LOST (1<<0)#define GEN8_OAHEADPTR _MMIO(0x2B0C) +#define GEN8_OAHEADPTR_MASK 0xffffffc0#define GEN8_OATAILPTR _MMIO(0x2B10) +#define GEN8_OATAILPTR_MASK 0xffffffc0#define OABUFFER_SIZE_128K (0<<3)#define OABUFFER_SIZE_256K (1<<3)@@ -705,7 +714,17 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)#define OA_MEM_SELECT_GGTT (1<<0) +/* +* Flexible, Aggregate EU Counter Registers. +* Note: these aren't contiguous +*/ #define EU_PERF_CNTL0 _MMIO(0xe458) +#define EU_PERF_CNTL1 _MMIO(0xe558) +#define EU_PERF_CNTL2 _MMIO(0xe658) +#define EU_PERF_CNTL3 _MMIO(0xe758) +#define EU_PERF_CNTL4 _MMIO(0xe45c) +#define EU_PERF_CNTL5 _MMIO(0xe55c) +#define EU_PERF_CNTL6 _MMIO(0xe65c)#define GDT_CHICKEN_BITS _MMIO(0x9840)#define GT_NOA_ENABLE 0x00000080@@ -2325,6 +2344,9 @@ enum skl_disp_power_wells {#define GEN8_RC_SEMA_IDLE_MSG_DISABLE (1 << 12)#define GEN8_FF_DOP_CLOCK_GATE_DISABLE (1<<10) +#define GEN6_RCS_PWR_FSM _MMIO(0x22ac) +#define GEN9_RCS_FE_FSM2 _MMIO(0x22a4) +/* Fuse readout registers for GT */ #define CHV_FUSE_GT _MMIO(VLV_DISPLAY_BASE + 0x2168)#define CHV_FGT_DISABLE_SS0 (1 << 10)drivers/gpu/drm/i915/intel_lrc.c +2
-0@@ -1879,6 +1879,8 @@ static void execlists_init_reg_state(u32 *regs, regs[CTX_LRI_HEADER_2 ] =MI_LOAD_REGISTER_IMM (1 );CTX_REG (regs,CTX_R_PWR_CLK_STATE ,GEN8_R_PWR_CLK_STATE ,make_rpcs (dev_priv)); + +i915_oa_init_reg_state (engine, ctx, regs); } }include/uapi/drm/i915_drm.h +12
-8@@ -1316,13 +1316,18 @@ struct drm_i915_gem_context_param { };enum drm_i915_oa_format { -I915_OA_FORMAT_A13 =1 , -I915_OA_FORMAT_A29 , -I915_OA_FORMAT_A13_B8_C8 , -I915_OA_FORMAT_B4_C8 , -I915_OA_FORMAT_A45_B8_C8 , -I915_OA_FORMAT_B4_C8_A16 , -I915_OA_FORMAT_C4_B8 , +I915_OA_FORMAT_A13 =1 ,/* HSW only */ +I915_OA_FORMAT_A29 ,/* HSW only */ +I915_OA_FORMAT_A13_B8_C8 ,/* HSW only */ +I915_OA_FORMAT_B4_C8 ,/* HSW only */ +I915_OA_FORMAT_A45_B8_C8 ,/* HSW only */ +I915_OA_FORMAT_B4_C8_A16 ,/* HSW only */ +I915_OA_FORMAT_C4_B8 ,/* HSW+ */ + +/* Gen8+ */ +I915_OA_FORMAT_A12 , +I915_OA_FORMAT_A12_B8_C8 , + I915_OA_FORMAT_A32u40_A4u32_B8_C8,I915_OA_FORMAT_MAX /* non-ABI */ }; --
Quoted 2 lines
2.11.0
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [PATCH v6 12/15] drm/i915/perf: Add OA unit support for Gen 8+
In reply to Lionel LandwerlinQuoted 873 lines · Lionel Landwerlin
From: Robert Bragg <[email protected]>
Enables access to OA unit metrics for BDW, CHV, SKL and BXT which all share (more-or-less) the same OA unit design.
Of particular note in comparison to Haswell: some OA unit HW config state has become per-context state and as a consequence it is somewhat more complicated to manage synchronous state changes from the cpu while there's no guarantee of what context (if any) is currently actively running on the gpu.
The periodic sampling frequency which can be particularly useful for system-wide analysis (as opposed to command stream synchronised
MI_REPORT_PERF_COUNTcommands) is perhaps the most surprising state to have become per-context save and restored (while the OABUFFER destination is still a shared, system-wide resource).This support for gen8+ takes care to consider a number of timing challenges involved in synchronously updating per-context state primarily by programming all config state from the cpu and updating all current and saved contexts synchronously while the OA unit is still disabled.
The driver intentionally avoids depending on command streamer programming to update OA state considering the lack of synchronization between the automatic loading of OACTXCONTROL state (that includes the periodic sampling state and enable state) on context restore and the parsing of any general purpose BB the driver can control. I.e. this implementation is careful to avoid the possibility of a context restore temporarily enabling any out-of-date periodic sampling state. In addition to the risk of transiently-out-of-date state being loaded automatically; there are also internal HW latencies involved in the loading of MUX configurations which would be difficult to account for from the command streamer (and we only want to enable the unit when once the MUX configuration is complete).
Since the Gen8+ OA unit design no longer supports clock gating the unit off for a single given context (which effectively stopped any progress of counters while any other context was running) and instead supports tagging OA reports with a context ID for filtering on the CPU, it means we can no longer hide the system-wide progress of counters from a non-privileged application only interested in metrics for its own context. Although we could theoretically try and subtract the progress of other contexts before forwarding reports via
read()we aren't in a position to filter reports captured viaMI_REPORT_PERF_COUNTcommands. As a result, for Gen8+, we always require the dev.i915.perf_stream_paranoid to be unset for any access to OA metrics if not root.v5: Drain submitted requests when enabling metric set to ensure no
lite-restore erases the context image we just updated (Lionel)v6: In addition to drain, switch to kernel context & update all
context in place (Chris)Signed-off-by Robert Bragg <[email protected]>Signed-off-by Lionel Landwerlin <[email protected]>Reviewed-by Matthew Auld <[email protected]> \o/--- drivers/gpu/drm/i915/i915_drv.h | 45 +- drivers/gpu/drm/i915/i915_perf.c | 957 ++++++++++++++++++++++++++++++++++++--- drivers/gpu/drm/i915/i915_reg.h | 22 + drivers/gpu/drm/i915/intel_lrc.c | 2 + include/uapi/drm/i915_drm.h | 19 +- 5 files changed, 952 insertions(+), 93 deletions(-)drivers/gpu/drm/i915/i915_drv.h +30
-15@@ -2064,9 +2064,17 @@ struct i915_oa_ops {void (*init_oa_buffer)(struct drm_i915_private *dev_priv);/** -* @enable_metric_set: Applies any MUX configuration to set up the -* Boolean and Custom (B/C) counters that are part of the counter -* reports being sampled. May apply system constraints such as +* @select_metric_set: The auto generated code that checks whether a +* requested OA config is applicable to the system and if so sets up +* the mux, oa and flex eu register config pointers according to the +* current dev_priv->perf.oa.metrics_set. +*/ +int (*select_metric_set)(struct drm_i915_private *dev_priv); + +/** +* @enable_metric_set: Selects and applies any MUX configuration to set +* up the Boolean and Custom (B/C) counters that are part of the +* counter reports being sampled. May apply system constraints such as * disabling EU clock gating as required. */ int (*enable_metric_set)(struct drm_i915_private *dev_priv);@@ -2097,20 +2105,13 @@ struct i915_oa_ops { size_t *offset);/** -* @oa_buffer_check: Check for OA buffer data + update tail -* -* This is either called via fops or the poll check hrtimer (atomic -* ctx) without any locks taken. +* @oa_hw_tail_read: read the OA tail pointer register * -* It's safe to read OA config state here unlocked, assuming that this -* is only called while the stream is enabled, while the global OA -* configuration can't be modified. -* -* Efficiency is more important than avoiding some false positives -* here, which will be handled gracefully - likely resulting in an -* %EAGAIN error for userspace. +* In particular this enables us to share all the fiddly code for +* handling the OA unit tail pointer race that affects multiple +* generations. */ -bool (*oa_buffer_check)(struct drm_i915_private *dev_priv); +u32 (*oa_hw_tail_read)(struct drm_i915_private *dev_priv); };struct intel_cdclk_state {@@ -2472,6 +2473,7 @@ struct drm_i915_private {struct {struct i915_vma *vma;u8 *vaddr; +u32 last_ctx_id;int format;int format_size;@@ -2541,6 +2543,14 @@ struct drm_i915_private { } oa_buffer;u32 gen7_latched_oastatus1; +u32 ctx_oactxctrl_offset; +u32 ctx_flexeu0_offset; + +/* The RPT_ID/reason field for Gen8+ includes a bit +* to determine if the CTX ID in the report is valid +* but the specific bit differs between Gen 8 and 9 +*/ +u32 gen8_valid_ctx_bit;struct i915_oa_ops ops;const struct i915_oa_format *oa_formats;@@ -2851,6 +2861,8 @@ intel_info(const struct drm_i915_private *dev_priv)#define IS_KBL_ULX (dev_priv) (INTEL_DEVID(dev_priv) == 0x590E || \ INTEL_DEVID(dev_priv) == 0x5915 || \ INTEL_DEVID(dev_priv) == 0x591E) +#define IS_SKL_GT2 (dev_priv) (IS_SKYLAKE(dev_priv) && \ + (INTEL_DEVID(dev_priv) & 0x00F0) == 0x0010)#define IS_SKL_GT3 (dev_priv) (IS_SKYLAKE(dev_priv) && \ (INTEL_DEVID(dev_priv) & 0x00F0) == 0x0020)#define IS_SKL_GT4 (dev_priv) (IS_SKYLAKE(dev_priv) && \@@ -3615,6 +3627,9 @@ i915_gem_context_lookup_timeline(struct i915_gem_context *ctx,int i915_perf_open_ioctl (struct drm_device *dev,void *data,struct drm_file *file); +void i915_oa_init_reg_state (struct intel_engine_cs *engine, +struct i915_gem_context *ctx, +uint32_t *reg_state);/* i915_gem_evict.c */ int __must_check i915_gem_evict_something(struct i915_address_space *vm,drivers/gpu/drm/i915/i915_perf.c +554
-35@@ -196,6 +196,12 @@#include "i915_drv.h" #include "i915_oa_hsw.h" +#include "i915_oa_bdw.h" +#include "i915_oa_chv.h" +#include "i915_oa_sklgt2.h" +#include "i915_oa_sklgt3.h" +#include "i915_oa_sklgt4.h" +#include "i915_oa_bxt.h" /* HW requires this to be a power of two, between 128k and 16M, though driver * is currently generally designed assuming the largest 16M size is used such @@ -215,7 +221,7 @@* * Although this can be observed explicitly while copying reports to userspace * by checking for a zeroed report-id field in tail reports, we want to account -* for this earlier, as part of the _oa_buffer_check to avoid lots of redundant +* for this earlier, as part of the oa_buffer_check to avoid lots of redundant * read() attempts. * * In effect we define a tail pointer for reading that lags the real tail @@ -237,7 +243,7 @@* indicates that an updated tail pointer is needed. * * Most of the implementation details for this workaround are in -* gen7_oa_buffer_check_unlocked() and gen7_appand_oa_reports() +* oa_buffer_check_unlocked() and _append_oa_reports() * * Note for posterity: previously the driver used to define an effective tail * pointer that lagged the real pointer by a 'tail margin' measured in bytes @@ -272,6 +278,13 @@ static u32 i915_perf_stream_paranoid = true;#define INVALID_CTX_ID 0xffffffff +/* On Gen8+ automatically triggered OA reports include a 'reason' field... */ +#define OAREPORT_REASON_MASK 0x3f +#define OAREPORT_REASON_SHIFT 19 +#define OAREPORT_REASON_TIMER (1<<0) +#define OAREPORT_REASON_CTX_SWITCH (1<<3) +#define OAREPORT_REASON_CLK_RATIO (1<<5) +/* For sysctl proc_dointvec_minmax of i915_oa_max_sample_rate * @@ -303,6 +316,13 @@ static struct i915_oa_format hsw_oa_formats[I915_OA_FORMAT_MAX] = { [I915_OA_FORMAT_C4_B8] = { 7, 64 }, }; +static struct i915_oa_format gen8_plus_oa_formats[I915_OA_FORMAT_MAX ] = { + [I915_OA_FORMAT_A12 ] = {0 ,64 }, + [I915_OA_FORMAT_A12_B8_C8 ] = {2 ,128 }, + [I915_OA_FORMAT_A32u40_A4u32_B8_C8] = {5 ,256 }, + [I915_OA_FORMAT_C4_B8 ] = {7 ,64 }, +}; +#define SAMPLE_OA_REPORT (1<<0)/** @@ -332,8 +352,20 @@ struct perf_open_properties {int oa_period_exponent; }; +static u32 gen8_oa_hw_tail_read (struct drm_i915_private *dev_priv) +{ +return I915_READ (GEN8_OATAILPTR ) &GEN8_OATAILPTR_MASK ; +} + +static u32 gen7_oa_hw_tail_read (struct drm_i915_private *dev_priv) +{ +u32 oastatus1 =I915_READ (GEN7_OASTATUS1 ); + +return oastatus1 &GEN7_OASTATUS1_TAIL_MASK ; +} +/** -* gen7_oa_buffer_check_unlocked - check for data and update tail ptr state +* oa_buffer_check_unlocked - check for data and update tail ptr state * @dev_priv: i915 device instance * * This is either called via fops (for blocking reads in user ctx) or the poll @@ -356,12 +388,11 @@ struct perf_open_properties {* * Returns: %true if the OA buffer contains data, else %false */ -static bool gen7_oa_buffer_check_unlocked (struct drm_i915_private *dev_priv) +static bool oa_buffer_check_unlocked (struct drm_i915_private *dev_priv) {int report_size = dev_priv->perf.oa.oa_buffer.format_size;unsigned long flags;unsigned int aged_idx; -u32 oastatus1;u32 head, hw_tail, aged_tail, aging_tail;u64 now;@@ -381,8 +412,7 @@ static bool gen7_oa_buffer_check_unlocked(struct drm_i915_private *dev_priv) aged_tail = dev_priv->perf.oa.oa_buffer.tails[aged_idx].offset; aging_tail = dev_priv->perf.oa.oa_buffer.tails[!aged_idx].offset; - oastatus1 =I915_READ (GEN7_OASTATUS1 ); - hw_tail = oastatus1 &GEN7_OASTATUS1_TAIL_MASK ; + hw_tail = dev_priv->perf.oa.ops.oa_hw_tail_read (dev_priv);/* The tail pointer increases in 64 byte increments, * not in report_size steps... @@ -404,6 +434,7 @@ static bool gen7_oa_buffer_check_unlocked(struct drm_i915_private *dev_priv)if (aging_tail !=INVALID_TAIL_PTR && ((now - dev_priv->perf.oa.oa_buffer.aging_timestamp) >OA_TAIL_MARGIN_NSEC )) { + aged_idx ^=1 ; dev_priv->perf.oa.oa_buffer.aged_tail_idx = aged_idx;@@ -553,6 +584,287 @@ static int append_oa_sample(struct i915_perf_stream *stream,* * Returns: 0 on success, negative error code on failure. */ +static int gen8_append_oa_reports (struct i915_perf_stream *stream, +char __user *buf, +size_t count, +size_t *offset) +{ +struct drm_i915_private *dev_priv = stream->dev_priv; +int report_size = dev_priv->perf.oa.oa_buffer.format_size; +u8 *oa_buf_base = dev_priv->perf.oa.oa_buffer.vaddr; +u32 gtt_offset =i915_ggtt_offset (dev_priv->perf.oa.oa_buffer.vma); +u32 mask = (OA_BUFFER_SIZE -1 ); +size_t start_offset = *offset; +unsigned long flags; +unsigned int aged_tail_idx; +u32 head, tail; +u32 taken; +int ret =0 ; + +if (WARN_ON (!stream->enabled)) +return -EIO ; + +spin_lock_irqsave (&dev_priv->perf.oa.oa_buffer.ptr_lock, flags); + + head = dev_priv->perf.oa.oa_buffer.head; + aged_tail_idx = dev_priv->perf.oa.oa_buffer.aged_tail_idx; + tail = dev_priv->perf.oa.oa_buffer.tails[aged_tail_idx].offset; + +spin_unlock_irqrestore (&dev_priv->perf.oa.oa_buffer.ptr_lock, flags); + +/* An invalid tail pointer here means we're still waiting for the poll +* hrtimer callback to give us a pointer +*/ +if (tail ==INVALID_TAIL_PTR ) +return -EAGAIN ; + +/* NB: oa_buffer.head/tail include the gtt_offset which we don't want +* while indexing relative to oa_buf_base. +*/ + head -= gtt_offset; + tail -= gtt_offset; + +/* An out of bounds or misaligned head or tail pointer implies a driver +* bug since we validate + align the tail pointers we read from the +* hardware and we are in full control of the head pointer which should +* only be incremented by multiples of the report size (notably also +* all a power of two). +*/ +if (WARN_ONCE (head >OA_BUFFER_SIZE || head % report_size || + tail >OA_BUFFER_SIZE || tail % report_size, +"Inconsistent OA buffer pointers: head = %u, tail = %u\n" , + head, tail)) +return -EIO ; + + +for (/* none */ ; + (taken =OA_TAKEN (tail, head)); + head = (head + report_size) & mask) { +u8 *report = oa_buf_base + head; +u32 *report32 = (void *)report; +u32 ctx_id; +u32 reason; + +/* All the report sizes factor neatly into the buffer +* size so we never expect to see a report split +* between the beginning and end of the buffer. +* +* Given the initial alignment check a misalignment +* here would imply a driver bug that would result +* in an overrun. +*/ +if (WARN_ON ((OA_BUFFER_SIZE - head) < report_size)) { +DRM_ERROR ("Spurious OA head ptr: non-integral report offset\n" ); +break ; + } + +/* The reason field includes flags identifying what +* triggered this specific report (mostly timer +* triggered or e.g. due to a context switch). +* +* This field is never expected to be zero so we can +* check that the report isn't invalid before copying +* it to userspace... +*/ + reason = ((report32[0 ] >>OAREPORT_REASON_SHIFT ) & +OAREPORT_REASON_MASK ); +if (reason ==0 ) { +if (__ratelimit (&dev_priv->perf.oa.spurious_report_rs)) +DRM_NOTE ("Skipping spurious, invalid OA report\n" ); +continue ; + } + +/* XXX: Just keep the lower 21 bits for now since I'm not +* entirely sure if the HW touches any of the higher bits in +* this field +*/ + ctx_id = report32[2 ] &0x1fffff ; + +/* Squash whatever is in the CTX_ID field if it's marked as +* invalid to be sure we avoid false-positive, single-context +* filtering below... +* +* Note: that we don't clear the valid_ctx_bit so userspace can +* understand that the ID has been squashed by the kernel. +*/ +if (!(report32[0 ] & dev_priv->perf.oa.gen8_valid_ctx_bit)) + ctx_id = report32[2 ] =INVALID_CTX_ID ; + +/* NB: For Gen 8 the OA unit no longer supports clock gating +* off for a specific context and the kernel can't securely +* stop the counters from updating as system-wide / global +* values. +* +* Automatic reports now include a context ID so reports can be +* filtered on the cpu but it's not worth trying to +* automatically subtract/hide counter progress for other +* contexts while filtering since we can't stop userspace +* issuing MI_REPORT_PERF_COUNT commands which would still +* provide a side-band view of the real values. +* +* To allow userspace (such as Mesa/GL_INTEL_performance_query) +* to normalize counters for a single filtered context then it +* needs be forwarded bookend context-switch reports so that it +* can track switches in between MI_REPORT_PERF_COUNT commands +* and can itself subtract/ignore the progress of counters +* associated with other contexts. Note that the hardware +* automatically triggers reports when switching to a new +* context which are tagged with the ID of the newly active +* context. To avoid the complexity (and likely fragility) of +* reading ahead while parsing reports to try and minimize +* forwarding redundant context switch reports (i.e. between +* other, unrelated contexts) we simply elect to forward them +* all. +* +* We don't rely solely on the reason field to identify context +* switches since it's not-uncommon for periodic samples to +* identify a switch before any 'context switch' report. +*/ +if (!dev_priv->perf.oa.exclusive_stream->ctx || + dev_priv->perf.oa.specific_ctx_id == ctx_id || + (dev_priv->perf.oa.oa_buffer.last_ctx_id == + dev_priv->perf.oa.specific_ctx_id) || + reason &OAREPORT_REASON_CTX_SWITCH ) { + +/* While filtering for a single context we avoid +* leaking the IDs of other contexts. +*/ +if (dev_priv->perf.oa.exclusive_stream->ctx && + dev_priv->perf.oa.specific_ctx_id != ctx_id) { + report32[2 ] =INVALID_CTX_ID ; + } + + ret =append_oa_sample (stream, buf, count, offset, + report); +if (ret) +break ; + + dev_priv->perf.oa.oa_buffer.last_ctx_id = ctx_id; + } + +/* The above reason field sanity check is based on +* the assumption that the OA buffer is initially +* zeroed and we reset the field after copying so the +* check is still meaningful once old reports start +* being overwritten. +*/ + report32[0 ] =0 ; + } + +if (start_offset != *offset) { +spin_lock_irqsave (&dev_priv->perf.oa.oa_buffer.ptr_lock, flags); + +/* We removed the gtt_offset for the copy loop above, indexing +* relative to oa_buf_base so put back here... +*/ + head += gtt_offset; + +I915_WRITE (GEN8_OAHEADPTR , head &GEN8_OAHEADPTR_MASK ); + dev_priv->perf.oa.oa_buffer.head = head; + +spin_unlock_irqrestore (&dev_priv->perf.oa.oa_buffer.ptr_lock, flags); + } + +return ret; +} + +/** +* gen8_oa_read - copy status records then buffered OA reports +* @stream: An i915-perf stream opened for OA metrics +* @buf: destination buffer given by userspace +* @count: the number of bytes userspace wants to read +* @offset: (inout): the current position for writing into @buf +* +* Checks OA unit status registers and if necessary appends corresponding +* status records for userspace (such as for a buffer full condition) and then +* initiate appending any buffered OA reports. +* +* Updates @offset according to the number of bytes successfully copied into +* the userspace buffer. +* +* NB: some data may be successfully copied to the userspace buffer +* even if an error is returned, and this is reflected in the +* updated @offset. +* +* Returns: zero on success or a negative error code +*/ +static int gen8_oa_read (struct i915_perf_stream *stream, +char __user *buf, +size_t count, +size_t *offset) +{ +struct drm_i915_private *dev_priv = stream->dev_priv; +u32 oastatus; +int ret; + +if (WARN_ON (!dev_priv->perf.oa.oa_buffer.vaddr)) +return -EIO ; + + oastatus =I915_READ (GEN8_OASTATUS ); + +/* We treat OABUFFER_OVERFLOW as a significant error: +* +* Although theoretically we could handle this more gracefully +* sometimes, some Gens don't correctly suppress certain +* automatically triggered reports in this condition and so we +* have to assume that old reports are now being trampled +* over. +* +* Considering how we don't currently give userspace control +* over the OA buffer size and always configure a large 16MB +* buffer, then a buffer overflow does anyway likely indicate +* that something has gone quite badly wrong. +*/ +if (oastatus &GEN8_OASTATUS_OABUFFER_OVERFLOW ) { + ret =append_oa_status (stream, buf, count, offset, +DRM_I915_PERF_RECORD_OA_BUFFER_LOST ); +if (ret) +return ret; + +DRM_DEBUG ("OA buffer overflow (exponent = %d): force restart\n" , + dev_priv->perf.oa.period_exponent); + + dev_priv->perf.oa.ops.oa_disable (dev_priv); + dev_priv->perf.oa.ops.oa_enable (dev_priv); + +/* Note: .oa_enable() is expected to re-init the oabuffer +* and reset GEN8_OASTATUS for us +*/ + oastatus =I915_READ (GEN8_OASTATUS ); + } + +if (oastatus &GEN8_OASTATUS_REPORT_LOST ) { + ret =append_oa_status (stream, buf, count, offset, +DRM_I915_PERF_RECORD_OA_REPORT_LOST ); +if (ret) +return ret; +I915_WRITE (GEN8_OASTATUS , + oastatus & ~GEN8_OASTATUS_REPORT_LOST ); + } + +return gen8_append_oa_reports (stream, buf, count, offset); +} + +/** +* Copies all buffered OA reports into userspace read() buffer. +* @stream: An i915-perf stream opened for OA metrics +* @buf: destination buffer given by userspace +* @count: the number of bytes userspace wants to read +* @offset: (inout): the current position for writing into @buf +* +* Notably any error condition resulting in a short read (-%ENOSPC or +* -%EFAULT) will be returned even though one or more records may +* have been successfully copied. In this case it's up to the caller +* to decide if the error should be squashed before returning to +* userspace. +* +* Note: reports are consumed from the head, and appended to the +* tail, so the tail chases the head?... If you think that's mad +* and back-to-front you're not alone, but this follows the +* Gen PRM naming convention. +* +* Returns: 0 on success, negative error code on failure. +*/ static int gen7_append_oa_reports(struct i915_perf_stream *stream, char __user *buf, size_t count,@@ -733,7 +1045,8 @@ static int gen7_oa_read(struct i915_perf_stream *stream,if (ret)return ret; -DRM_DEBUG ("OA buffer overflow: force restart\n" ); +DRM_DEBUG ("OA buffer overflow (exponent = %d): force restart\n" , + dev_priv->perf.oa.period_exponent); dev_priv->perf.oa.ops.oa_disable (dev_priv); dev_priv->perf.oa.ops.oa_enable (dev_priv);@@ -776,7 +1089,7 @@ static int i915_oa_wait_unlocked(struct i915_perf_stream *stream)return -EIO ;return wait_event_interruptible (dev_priv->perf.oa.poll_wq, - dev_priv->perf.oa.ops.oa_buffer_check (dev_priv)); +oa_buffer_check_unlocked (dev_priv)); }/** @@ -833,33 +1146,37 @@ static int i915_oa_read(struct i915_perf_stream *stream,static int oa_get_render_ctx_id (struct i915_perf_stream *stream) {struct drm_i915_private *dev_priv = stream->dev_priv; -struct intel_engine_cs *engine = dev_priv->engine[RCS ]; -int ret; - ret =i915_mutex_lock_interruptible (&dev_priv->drm); -if (ret) -return ret; +if (i915.enable_execlists) + dev_priv->perf.oa.specific_ctx_id = stream->ctx->hw_id; +else { +struct intel_engine_cs *engine = dev_priv->engine[RCS ]; +int ret; -/* As the ID is the gtt offset of the context's vma we pin -* the vma to ensure the ID remains fixed. -* -* NB: implied RCS engine... -*/ - ret = engine->context_pin (engine, stream->ctx); -if (ret) - goto unlock; + ret =i915_mutex_lock_interruptible (&dev_priv->drm); +if (ret) +return ret; -/* Explicitly track the ID (instead of calling i915_ggtt_offset() -* on the fly) considering the difference with gen8+ and -* execlists -*/ - dev_priv->perf.oa.specific_ctx_id = -i915_ggtt_offset (stream->ctx->engine[engine->id].state); +/* As the ID is the gtt offset of the context's vma we pin +* the vma to ensure the ID remains fixed. +*/ + ret = engine->context_pin (engine, stream->ctx); +if (ret) { +mutex_unlock (&dev_priv->drm.struct_mutex); +return ret; + } -unlock: -mutex_unlock (&dev_priv->drm.struct_mutex); +/* Explicitly track the ID (instead of calling +* i915_ggtt_offset() on the fly) considering the difference +* with gen8+ and execlists +*/ + dev_priv->perf.oa.specific_ctx_id = +i915_ggtt_offset (stream->ctx->engine[engine->id].state); -return ret; +mutex_unlock (&dev_priv->drm.struct_mutex); + } + +return 0 ; }/** @@ -874,12 +1191,16 @@ static void oa_put_render_ctx_id(struct i915_perf_stream *stream)struct drm_i915_private *dev_priv = stream->dev_priv;struct intel_engine_cs *engine = dev_priv->engine[RCS ]; -mutex_lock (&dev_priv->drm.struct_mutex); +if (i915.enable_execlists) { + dev_priv->perf.oa.specific_ctx_id =INVALID_CTX_ID ; + }else { +mutex_lock (&dev_priv->drm.struct_mutex); - dev_priv->perf.oa.specific_ctx_id =INVALID_CTX_ID ; - engine->context_unpin (engine, stream->ctx); + dev_priv->perf.oa.specific_ctx_id =INVALID_CTX_ID ; + engine->context_unpin (engine, stream->ctx); -mutex_unlock (&dev_priv->drm.struct_mutex); +mutex_unlock (&dev_priv->drm.struct_mutex); + } } static void@@ -969,6 +1290,61 @@ static void gen7_init_oa_buffer(struct drm_i915_private *dev_priv) dev_priv->perf.oa.pollin = false; } +static void gen8_init_oa_buffer (struct drm_i915_private *dev_priv) +{ +u32 gtt_offset =i915_ggtt_offset (dev_priv->perf.oa.oa_buffer.vma); +unsigned long flags; + +spin_lock_irqsave (&dev_priv->perf.oa.oa_buffer.ptr_lock, flags); + +I915_WRITE (GEN8_OASTATUS ,0 ); +I915_WRITE (GEN8_OAHEADPTR , gtt_offset); + dev_priv->perf.oa.oa_buffer.head = gtt_offset; + +I915_WRITE (GEN8_OABUFFER_UDW ,0 ); + +/* PRM says: +* +* "This MMIO must be set before the OATAILPTR +* register and after the OAHEADPTR register. This is +* to enable proper functionality of the overflow +* bit." +*/ +I915_WRITE (GEN8_OABUFFER , gtt_offset | +OABUFFER_SIZE_16M |OA_MEM_SELECT_GGTT ); +I915_WRITE (GEN8_OATAILPTR , gtt_offset &GEN8_OATAILPTR_MASK ); + +/* Mark that we need updated tail pointers to read from... */ + dev_priv->perf.oa.oa_buffer.tails[0 ].offset =INVALID_TAIL_PTR ; + dev_priv->perf.oa.oa_buffer.tails[1 ].offset =INVALID_TAIL_PTR ; + +/* Reset state used to recognise context switches, affecting which +* reports we will forward to userspace while filtering for a single +* context. +*/ + dev_priv->perf.oa.oa_buffer.last_ctx_id =INVALID_CTX_ID ; + +spin_unlock_irqrestore (&dev_priv->perf.oa.oa_buffer.ptr_lock, flags); + +/* NB: although the OA buffer will initially be allocated +* zeroed via shmfs (and so this memset is redundant when +* first allocating), we may re-init the OA buffer, either +* when re-enabling a stream or in error/reset paths. +* +* The reason we clear the buffer for each re-init is for the +* sanity check in gen8_append_oa_reports() that looks at the +* reason field to make sure it's non-zero which relies on +* the assumption that new reports are being written to zeroed +* memory... +*/ +memset (dev_priv->perf.oa.oa_buffer.vaddr,0 ,OA_BUFFER_SIZE ); + +/* Maybe make ->pollin per-stream state if we support multiple +* concurrent streams in the future. +*/ + dev_priv->perf.oa.pollin = false; +} +static int alloc_oa_buffer (struct drm_i915_private *dev_priv) {struct drm_i915_gem_object *bo;@@ -1113,6 +1489,280 @@ static void hsw_disable_metric_set(struct drm_i915_private *dev_priv) ~GT_NOA_ENABLE)); } +/* +* From Broadwell PRM, 3D-Media-GPGPU -> Register State Context +* +* MMIO reads or writes to any of the registers listed in the +* “Register State Context image” subsections through HOST/IA +* MMIO interface for debug purposes must follow the steps below: +* +* - SW should set the Force Wakeup bit to prevent GT from entering C6. +* - Write 0x2050[31:0] = 0x00010001 (disable sequence). +* - Disable IDLE messaging in CS (Write 0x2050[31:0] = 0x00010001). +* - BDW: Poll/Wait for register bits of 0x22AC[6:0] turn to 0x30 value. +* - SKL+: Poll/Wait for register bits of 0x22A4[6:0] turn to 0x30 value. +* - Read/Write to desired MMIO registers. +* - Enable IDLE messaging in CS (Write 0x2050[31:0] = 0x00010000). +* - Force Wakeup bit should be reset to enable C6 entry. +* +* XXX: don't nest or overlap calls to this API, it has no ref +* counting to track how many entities require the RCS to be +* blocked from being idle. +*/ +static int gen8_begin_ctx_mmio (struct drm_i915_private *dev_priv) +{ +i915_reg_t fsm_reg =INTEL_GEN (dev_priv) >8 ? +GEN9_RCS_FE_FSM2 :GEN6_RCS_PWR_FSM ; +int ret =0 ; + +/* There's only no active context while idle in execlist mode +* (though we shouldn't be using this in any other case) +*/ +if (WARN_ON (!i915.enable_execlists)) +return -EINVAL ; + +intel_uncore_forcewake_get (dev_priv,FORCEWAKE_RENDER ); + +I915_WRITE (GEN6_RC_SLEEP_PSMI_CONTROL , +_MASKED_BIT_ENABLE (GEN6_PSMI_SLEEP_MSG_DISABLE )); + +/* Note: we don't currently have a good handle on the maximum +* latency for this wake up so while we only need to hold rcs +* busy from process context we at least keep the waiting +* interruptible... +*/ + ret =intel_wait_for_register (dev_priv, fsm_reg,0x3f ,0x30 ,50 ); +if (ret == -ETIMEDOUT ) +DRM_ERROR ("Timeout waiting to be able to begin per-ctx mmio\n" ); + +if (ret) +intel_uncore_forcewake_put (dev_priv,FORCEWAKE_RENDER ); + +return ret; +} + +static void gen8_end_ctx_mmio (struct drm_i915_private *dev_priv) +{ +if (WARN_ON (!i915.enable_execlists)) +return ; + +I915_WRITE (GEN6_RC_SLEEP_PSMI_CONTROL , +_MASKED_BIT_DISABLE (GEN6_PSMI_SLEEP_MSG_DISABLE )); +intel_uncore_forcewake_put (dev_priv,FORCEWAKE_RENDER ); +} + +/* NB: It must always remain pointer safe to run this even if the OA unit +* has been disabled. +* +* It's fine to put out-of-date values into these per-context registers +* in the case that the OA unit has been disabled. +*/ +static void gen8_update_reg_state_unlocked (struct i915_gem_context *ctx, +u32 *reg_state) +{ +struct drm_i915_private *dev_priv = ctx->i915; +const struct i915_oa_reg *flex_regs = dev_priv->perf.oa.flex_regs; +int n_flex_regs = dev_priv->perf.oa.flex_regs_len; +u32 ctx_oactxctrl = dev_priv->perf.oa.ctx_oactxctrl_offset; +u32 ctx_flexeu0 = dev_priv->perf.oa.ctx_flexeu0_offset; +/* The MMIO offsets for Flex EU registers aren't contiguous */ +u32 flex_mmio[] = { +i915_mmio_reg_offset (EU_PERF_CNTL0 ), +i915_mmio_reg_offset (EU_PERF_CNTL1 ), +i915_mmio_reg_offset (EU_PERF_CNTL2 ), +i915_mmio_reg_offset (EU_PERF_CNTL3 ), +i915_mmio_reg_offset (EU_PERF_CNTL4 ), +i915_mmio_reg_offset (EU_PERF_CNTL5 ), +i915_mmio_reg_offset (EU_PERF_CNTL6 ), + }; +int i; + + reg_state[ctx_oactxctrl] =i915_mmio_reg_offset (GEN8_OACTXCONTROL ); + reg_state[ctx_oactxctrl+1 ] = (dev_priv->perf.oa.period_exponent << +GEN8_OA_TIMER_PERIOD_SHIFT ) | + (dev_priv->perf.oa.periodic ? +GEN8_OA_TIMER_ENABLE :0 ) | +GEN8_OA_COUNTER_RESUME ; + +for (i =0 ; i <ARRAY_SIZE (flex_mmio); i++) { +u32 state_offset = ctx_flexeu0 + i *2 ; +u32 mmio = flex_mmio[i]; + +/* This arbitrary default will select the 'EU FPU0 Pipeline +* Active' event. In the future it's anticipated that there +* will be an explicit 'No Event' we can select, but not yet... +*/ +u32 value =0 ; +int j; + +for (j =0 ; j < n_flex_regs; j++) { +if (i915_mmio_reg_offset (flex_regs[j].addr) == mmio) { + value = flex_regs[j].value; +break ; + } + } + + reg_state[state_offset] = mmio; + reg_state[state_offset+1 ] = value; + } +} + +/* Manages updating the per-context aspects of the OA stream +* configuration across all contexts. +* +* The awkward consideration here is that OACTXCONTROL controls the +* exponent for periodic sampling which is primarily used for system +* wide profiling where we'd like a consistent sampling period even in +* the face of context switches. +* +* Our approach of updating the register state context (as opposed to +* say using a workaround batch buffer) ensures that the hardware +* won't automatically reload an out-of-date timer exponent even +* transiently before a WA BB could be parsed. +* +* This function needs to: +* - Ensure the currently running context's per-context OA state is +* updated +* - Ensure that all existing contexts will have the correct per-context +* OA state if they are scheduled for use. +* - Ensure any new contexts will be initialized with the correct +* per-context OA state. +* +* Note: it's only the RCS/Render context that has any OA state. +*/ +static int gen8_configure_all_contexts (struct drm_i915_private *dev_priv) +{
drivers/gpu/drm/i915/i915_perf.c static void hsw_disable_metric_set(struct drm_i915_private *dev_priv) · line 1644
Quoted 10 lines
drivers/gpu/drm/i915/i915_perf.c +10
-0+struct i915_gem_context *ctx; +int ret; + + ret =i915_mutex_lock_interruptible (&dev_priv->drm); +if (ret) +return ret; + +/* Switch away from any user context. */ + ret =i915_gem_switch_to_kernel_context (dev_priv); +if (ret)
mutex_unlock(&dev_priv->drm.struct_mutex);
Quoted 33 lines
drivers/gpu/drm/i915/i915_perf.c +33
-0+return ret; + +/* The OA register config is setup through the context image. This image +* might be written to by the GPU on context switch (in particular on +* lite-restore). This means we can't safely update a context's image, +* if this context is scheduled/submitted to run on the GPU. +* +* We could emit the OA register config through the batch buffer but +* this might leave small interval of time where the OA unit is +* configured at an invalid sampling period. +* +* So far the best way to work around this issue seems to be draining +* the GPU from any submitted work. +*/ + ret =i915_gem_wait_for_idle (dev_priv, +I915_WAIT_INTERRUPTIBLE | +I915_WAIT_LOCKED ); +if (ret) { +mutex_unlock (&dev_priv->drm.struct_mutex); +return ret; + } + +/* Update all contexts now that we've stalled the submission. */ +list_for_each_entry (ctx, &dev_priv->context_list, link) { +if (!ctx->engine[RCS ].initialised) +continue ; + +gen8_update_reg_state_unlocked (ctx, + ctx->engine[RCS ].lrc_reg_state); + } + +mutex_unlock (&dev_priv->drm.struct_mutex); +
drivers/gpu/drm/i915/i915_perf.c static void hsw_disable_metric_set(struct drm_i915_private *dev_priv) · line 1687
Quoted 10 lines
drivers/gpu/drm/i915/i915_perf.c +10
-0+/* Now update the current context. +* +* Note: Using MMIO to update per-context registers requires +* some extra care... +*/ + ret =gen8_begin_ctx_mmio (dev_priv); +if (ret) { +DRM_ERROR ("Failed to bring RCS out of idle to update current ctx OA state\n" ); +return ret; + }
So do we still need the mmio for the current context, given that we now idle everything and then update the reg state?
Quoted 517 lines
drivers/gpu/drm/i915/i915_perf.c +279
-36+ +I915_WRITE (GEN8_OACTXCONTROL , ((dev_priv->perf.oa.period_exponent << +GEN8_OA_TIMER_PERIOD_SHIFT ) | + (dev_priv->perf.oa.periodic ? +GEN8_OA_TIMER_ENABLE :0 ) | +GEN8_OA_COUNTER_RESUME )); + +config_oa_regs (dev_priv, dev_priv->perf.oa.flex_regs, + dev_priv->perf.oa.flex_regs_len); + +gen8_end_ctx_mmio (dev_priv); + +return 0 ; +} + +static int gen8_enable_metric_set (struct drm_i915_private *dev_priv) +{ +int ret = dev_priv->perf.oa.ops.select_metric_set (dev_priv); + +if (ret) +return ret; + +/* We disable slice/unslice clock ratio change reports on SKL since +* they are too noisy. The HW generates a lot of redundant reports +* where the ratio hasn't really changed causing a lot of redundant +* work to processes and increasing the chances we'll hit buffer +* overruns. +* +* Although we don't currently use the 'disable overrun' OABUFFER +* feature it's worth noting that clock ratio reports have to be +* disabled before considering to use that feature since the HW doesn't +* correctly block these reports. +* +* Currently none of the high-level metrics we have depend on knowing +* this ratio to normalize. +* +* Note: This register is not power context saved and restored, but +* that's OK considering that we disable RC6 while the OA unit is +* enabled. +* +* The _INCLUDE_CLK_RATIO bit allows the slice/unslice frequency to +* be read back from automatically triggered reports, as part of the +* RPT_ID field. +*/ +if (IS_SKYLAKE (dev_priv) ||IS_BROXTON (dev_priv)) { +I915_WRITE (GEN8_OA_DEBUG , +_MASKED_BIT_ENABLE (GEN9_OA_DEBUG_DISABLE_CLK_RATIO_REPORTS | +GEN9_OA_DEBUG_INCLUDE_CLK_RATIO )); + } + +I915_WRITE (GDT_CHICKEN_BITS ,0xA0 ); +config_oa_regs (dev_priv, dev_priv->perf.oa.mux_regs, + dev_priv->perf.oa.mux_regs_len); +I915_WRITE (GDT_CHICKEN_BITS ,0x80 ); + +/* It takes a fairly long time for a new MUX configuration to +* be be applied after these register writes. This delay +* duration is take from Haswell (derived empirically based on +* the render_basic config) but hopefully it covers the +* maximum configuration latency for Gen8+ too... +*/ +usleep_range (15000 ,20000 ); + +config_oa_regs (dev_priv, dev_priv->perf.oa.b_counter_regs, + dev_priv->perf.oa.b_counter_regs_len); + + ret =gen8_configure_all_contexts (dev_priv); +if (ret) +return ret; + +return 0 ; +} + +static void gen8_disable_metric_set (struct drm_i915_private *dev_priv) +{ +/* NOP */ +} +static void gen7_update_oacontrol_locked (struct drm_i915_private *dev_priv) {lockdep_assert_held (&dev_priv->perf.hook_lock);@@ -1157,6 +1807,29 @@ static void gen7_oa_enable(struct drm_i915_private *dev_priv)spin_unlock_irqrestore (&dev_priv->perf.hook_lock, flags); } +static void gen8_oa_enable (struct drm_i915_private *dev_priv) +{ +u32 report_format = dev_priv->perf.oa.oa_buffer.format; + +/* Reset buf pointers so we don't forward reports from before now. +* +* Think carefully if considering trying to avoid this, since it +* also ensures status flags and the buffer itself are cleared +* in error paths, and we have checks for invalid reports based +* on the assumption that certain fields are written to zeroed +* memory which this helps maintains. +*/ +gen8_init_oa_buffer (dev_priv); + +/* Note: we don't rely on the hardware to perform single context +* filtering and instead filter on the cpu based on the context-id +* field of reports +*/ +I915_WRITE (GEN8_OACONTROL , (report_format << +GEN8_OA_REPORT_FORMAT_SHIFT ) | +GEN8_OA_COUNTER_ENABLE ); +} +/** * i915_oa_stream_enable - handle `I915_PERF_IOCTL_ENABLE` for OA stream * @stream: An i915 perf stream opened for OA metrics @@ -1183,6 +1856,11 @@ static void gen7_oa_disable(struct drm_i915_private *dev_priv)I915_WRITE (GEN7_OACONTROL ,0 ); } +static void gen8_oa_disable (struct drm_i915_private *dev_priv) +{ +I915_WRITE (GEN8_OACONTROL ,0 ); +} +/** * i915_oa_stream_disable - handle `I915_PERF_IOCTL_DISABLE` for OA stream * @stream: An i915 perf stream opened for OA metrics @@ -1361,6 +2039,21 @@ static int i915_oa_stream_init(struct i915_perf_stream *stream,return ret; } +void i915_oa_init_reg_state (struct intel_engine_cs *engine, +struct i915_gem_context *ctx, +u32 *reg_state) +{ +struct drm_i915_private *dev_priv = engine->i915; + +if (engine->id !=RCS ) +return ; + +if (!dev_priv->perf.initialized) +return ; + +gen8_update_reg_state_unlocked (ctx, reg_state); +} +/** * i915_perf_read_locked - &i915_perf_stream_ops->read with error normalisation * @stream: An i915 perf stream @@ -1486,7 +2179,7 @@ static enum hrtimer_restart oa_poll_check_timer_cb(struct hrtimer *hrtimer)container_of (hrtimer,typeof (*dev_priv), perf.oa.poll_check_timer); -if (dev_priv->perf.oa.ops.oa_buffer_check (dev_priv)) { +if (oa_buffer_check_unlocked (dev_priv)) { dev_priv->perf.oa.pollin = true;wake_up (&dev_priv->perf.oa.poll_wq); }@@ -1775,6 +2468,7 @@ i915_perf_open_ioctl_locked(struct drm_i915_private *dev_priv,struct i915_gem_context *specific_ctx =NULL ;struct i915_perf_stream *stream =NULL ;unsigned long f_flags =0 ; +bool privileged_op = true;int stream_fd;int ret;@@ -1792,12 +2486,28 @@ i915_perf_open_ioctl_locked(struct drm_i915_private *dev_priv, } } +/* On Haswell the OA unit supports clock gating off for a specific +* context and in this mode there's no visibility of metrics for the +* rest of the system, which we consider acceptable for a +* non-privileged client. +* +* For Gen8+ the OA unit no longer supports clock gating off for a +* specific context and the kernel can't securely stop the counters +* from updating as system-wide / global values. Even though we can +* filter reports based on the included context ID we can't block +* clients from seeing the raw / global counter values via +* MI_REPORT_PERF_COUNT commands and so consider it a privileged op to +* enable the OA unit by default. +*/ +if (IS_HASWELL (dev_priv) && specific_ctx) + privileged_op = false; +/* Similar to perf's kernel.perf_paranoid_cpu sysctl option * we check a dev.i915.perf_stream_paranoid sysctl option * to determine if it's ok to access system wide OA counters * without CAP_SYS_ADMIN privileges. */ -if (!specific_ctx && +if (privileged_op && i915_perf_stream_paranoid && !capable (CAP_SYS_ADMIN )) {DRM_DEBUG ("Insufficient privileges to open system-wide i915 perf stream\n" ); ret = -EACCES ;@@ -2069,9 +2779,6 @@ int i915_perf_open_ioctl(struct drm_device *dev, void *data,*/ void i915_perf_register (struct drm_i915_private *dev_priv) { -if (!IS_HASWELL (dev_priv)) -return ; -if (!dev_priv->perf.initialized)return ;@@ -2087,11 +2794,38 @@ void i915_perf_register(struct drm_i915_private *dev_priv)if (!dev_priv->perf.metrics_kobj) goto exit; -if (i915_perf_register_sysfs_hsw (dev_priv)) { -kobject_put (dev_priv->perf.metrics_kobj); - dev_priv->perf.metrics_kobj =NULL ; +if (IS_HASWELL (dev_priv)) { +if (i915_perf_register_sysfs_hsw (dev_priv)) + goto sysfs_error; + }else if (IS_BROADWELL (dev_priv)) { +if (i915_perf_register_sysfs_bdw (dev_priv)) + goto sysfs_error; + }else if (IS_CHERRYVIEW (dev_priv)) { +if (i915_perf_register_sysfs_chv (dev_priv)) + goto sysfs_error; + }else if (IS_SKYLAKE (dev_priv)) { +if (IS_SKL_GT2 (dev_priv)) { +if (i915_perf_register_sysfs_sklgt2 (dev_priv)) + goto sysfs_error; + }else if (IS_SKL_GT3 (dev_priv)) { +if (i915_perf_register_sysfs_sklgt3 (dev_priv)) + goto sysfs_error; + }else if (IS_SKL_GT4 (dev_priv)) { +if (i915_perf_register_sysfs_sklgt4 (dev_priv)) + goto sysfs_error; + }else + goto sysfs_error; + }else if (IS_BROXTON (dev_priv)) { +if (i915_perf_register_sysfs_bxt (dev_priv)) + goto sysfs_error; } + goto exit; + +sysfs_error: +kobject_put (dev_priv->perf.metrics_kobj); + dev_priv->perf.metrics_kobj =NULL ; + exit:mutex_unlock (&dev_priv->perf.lock); }@@ -2107,13 +2841,24 @@ void i915_perf_register(struct drm_i915_private *dev_priv)*/ void i915_perf_unregister (struct drm_i915_private *dev_priv) { -if (!IS_HASWELL (dev_priv)) -return ; -if (!dev_priv->perf.metrics_kobj)return ; -i915_perf_unregister_sysfs_hsw (dev_priv); +if (IS_HASWELL (dev_priv)) +i915_perf_unregister_sysfs_hsw (dev_priv); +else if (IS_BROADWELL (dev_priv)) +i915_perf_unregister_sysfs_bdw (dev_priv); +else if (IS_CHERRYVIEW (dev_priv)) +i915_perf_unregister_sysfs_chv (dev_priv); +else if (IS_SKYLAKE (dev_priv)) { +if (IS_SKL_GT2 (dev_priv)) +i915_perf_unregister_sysfs_sklgt2 (dev_priv); +else if (IS_SKL_GT3 (dev_priv)) +i915_perf_unregister_sysfs_sklgt3 (dev_priv); +else if (IS_SKL_GT4 (dev_priv)) +i915_perf_unregister_sysfs_sklgt4 (dev_priv); + }else if (IS_BROXTON (dev_priv)) +i915_perf_unregister_sysfs_bxt (dev_priv);kobject_put (dev_priv->perf.metrics_kobj); dev_priv->perf.metrics_kobj =NULL ;@@ -2172,36 +2917,105 @@ static struct ctl_table dev_root[] = {*/ void i915_perf_init (struct drm_i915_private *dev_priv) { -if (!IS_HASWELL (dev_priv)) -return ; - -hrtimer_init (&dev_priv->perf.oa.poll_check_timer, -CLOCK_MONOTONIC ,HRTIMER_MODE_REL ); - dev_priv->perf.oa.poll_check_timer.function = oa_poll_check_timer_cb; -init_waitqueue_head (&dev_priv->perf.oa.poll_wq); + dev_priv->perf.oa.n_builtin_sets =0 ; + +if (IS_HASWELL (dev_priv)) { + dev_priv->perf.oa.ops.init_oa_buffer = gen7_init_oa_buffer; + dev_priv->perf.oa.ops.enable_metric_set = hsw_enable_metric_set; + dev_priv->perf.oa.ops.disable_metric_set = hsw_disable_metric_set; + dev_priv->perf.oa.ops.oa_enable = gen7_oa_enable; + dev_priv->perf.oa.ops.oa_disable = gen7_oa_disable; + dev_priv->perf.oa.ops.read = gen7_oa_read; + dev_priv->perf.oa.ops.oa_hw_tail_read = + gen7_oa_hw_tail_read; + + dev_priv->perf.oa.oa_formats = hsw_oa_formats; + + dev_priv->perf.oa.n_builtin_sets = + i915_oa_n_builtin_metric_sets_hsw; + }else if (i915.enable_execlists) { +/* Note: that although we could theoretically also support the +* legacy ringbuffer mode on BDW (and earlier iterations of +* this driver, before upstreaming did this) it didn't seem +* worth the complexity to maintain now that BDW+ enable +* execlist mode by default. +*/ -INIT_LIST_HEAD (&dev_priv->perf.streams); -mutex_init (&dev_priv->perf.lock); -spin_lock_init (&dev_priv->perf.hook_lock); -spin_lock_init (&dev_priv->perf.oa.oa_buffer.ptr_lock); +if (IS_GEN8 (dev_priv)) { + dev_priv->perf.oa.ctx_oactxctrl_offset =0x120 ; + dev_priv->perf.oa.ctx_flexeu0_offset =0x2ce ; + dev_priv->perf.oa.gen8_valid_ctx_bit = (1 <<25 ); + +if (IS_BROADWELL (dev_priv)) { + dev_priv->perf.oa.n_builtin_sets = + i915_oa_n_builtin_metric_sets_bdw; + dev_priv->perf.oa.ops.select_metric_set = + i915_oa_select_metric_set_bdw; + }else if (IS_CHERRYVIEW (dev_priv)) { + dev_priv->perf.oa.n_builtin_sets = + i915_oa_n_builtin_metric_sets_chv; + dev_priv->perf.oa.ops.select_metric_set = + i915_oa_select_metric_set_chv; + } + }else if (IS_GEN9 (dev_priv)) { + dev_priv->perf.oa.ctx_oactxctrl_offset =0x128 ; + dev_priv->perf.oa.ctx_flexeu0_offset =0x3de ; + dev_priv->perf.oa.gen8_valid_ctx_bit = (1 <<16 ); + +if (IS_SKL_GT2 (dev_priv)) { + dev_priv->perf.oa.n_builtin_sets = + i915_oa_n_builtin_metric_sets_sklgt2; + dev_priv->perf.oa.ops.select_metric_set = + i915_oa_select_metric_set_sklgt2; + }else if (IS_SKL_GT3 (dev_priv)) { + dev_priv->perf.oa.n_builtin_sets = + i915_oa_n_builtin_metric_sets_sklgt3; + dev_priv->perf.oa.ops.select_metric_set = + i915_oa_select_metric_set_sklgt3; + }else if (IS_SKL_GT4 (dev_priv)) { + dev_priv->perf.oa.n_builtin_sets = + i915_oa_n_builtin_metric_sets_sklgt4; + dev_priv->perf.oa.ops.select_metric_set = + i915_oa_select_metric_set_sklgt4; + }else if (IS_BROXTON (dev_priv)) { + dev_priv->perf.oa.n_builtin_sets = + i915_oa_n_builtin_metric_sets_bxt; + dev_priv->perf.oa.ops.select_metric_set = + i915_oa_select_metric_set_bxt; + } + } - dev_priv->perf.oa.ops.init_oa_buffer = gen7_init_oa_buffer; - dev_priv->perf.oa.ops.enable_metric_set = hsw_enable_metric_set; - dev_priv->perf.oa.ops.disable_metric_set = hsw_disable_metric_set; - dev_priv->perf.oa.ops.oa_enable = gen7_oa_enable; - dev_priv->perf.oa.ops.oa_disable = gen7_oa_disable; - dev_priv->perf.oa.ops.read = gen7_oa_read; - dev_priv->perf.oa.ops.oa_buffer_check = - gen7_oa_buffer_check_unlocked; +if (dev_priv->perf.oa.n_builtin_sets) { + dev_priv->perf.oa.ops.init_oa_buffer = gen8_init_oa_buffer; + dev_priv->perf.oa.ops.enable_metric_set = + gen8_enable_metric_set; + dev_priv->perf.oa.ops.disable_metric_set = + gen8_disable_metric_set; + dev_priv->perf.oa.ops.oa_enable = gen8_oa_enable; + dev_priv->perf.oa.ops.oa_disable = gen8_oa_disable; + dev_priv->perf.oa.ops.read = gen8_oa_read; + dev_priv->perf.oa.ops.oa_hw_tail_read = + gen8_oa_hw_tail_read; + + dev_priv->perf.oa.oa_formats = gen8_plus_oa_formats; + } + } - dev_priv->perf.oa.oa_formats = hsw_oa_formats; +if (dev_priv->perf.oa.n_builtin_sets) { +hrtimer_init (&dev_priv->perf.oa.poll_check_timer, +CLOCK_MONOTONIC ,HRTIMER_MODE_REL ); + dev_priv->perf.oa.poll_check_timer.function = oa_poll_check_timer_cb; +init_waitqueue_head (&dev_priv->perf.oa.poll_wq); - dev_priv->perf.oa.n_builtin_sets = - i915_oa_n_builtin_metric_sets_hsw; +INIT_LIST_HEAD (&dev_priv->perf.streams); +mutex_init (&dev_priv->perf.lock); +spin_lock_init (&dev_priv->perf.hook_lock); +spin_lock_init (&dev_priv->perf.oa.oa_buffer.ptr_lock); - dev_priv->perf.sysctl_header =register_sysctl_table (dev_root); + dev_priv->perf.sysctl_header =register_sysctl_table (dev_root); - dev_priv->perf.initialized = true; + dev_priv->perf.initialized = true; + } }/** @@ -2216,5 +3030,6 @@ void i915_perf_fini(struct drm_i915_private *dev_priv)unregister_sysctl_table (dev_priv->perf.sysctl_header);memset (&dev_priv->perf.oa.ops,0 ,sizeof (dev_priv->perf.oa.ops)); + dev_priv->perf.initialized = false; }drivers/gpu/drm/i915/i915_reg.h +22
-0@@ -653,6 +653,12 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)#define GEN8_OACTXID _MMIO(0x2364) +#define GEN8_OA_DEBUG _MMIO(0x2B04) +#define GEN9_OA_DEBUG_DISABLE_CLK_RATIO_REPORTS (1<<5) +#define GEN9_OA_DEBUG_INCLUDE_CLK_RATIO (1<<6) +#define GEN9_OA_DEBUG_DISABLE_GO_1_0_REPORTS (1<<2) +#define GEN9_OA_DEBUG_DISABLE_CTX_SWITCH_REPORTS (1<<1) +#define GEN8_OACONTROL _MMIO(0x2B00)#define GEN8_OA_REPORT_FORMAT_A12 (0<<2)#define GEN8_OA_REPORT_FORMAT_A12_B8_C8 (2<<2)@@ -674,6 +680,7 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)#define GEN7_OABUFFER_STOP_RESUME_ENABLE (1<<1)#define GEN7_OABUFFER_RESUME (1<<0) +#define GEN8_OABUFFER_UDW _MMIO(0x23b4)#define GEN8_OABUFFER _MMIO(0x2b14)#define GEN7_OASTATUS1 _MMIO(0x2364)@@ -692,7 +699,9 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)#define GEN8_OASTATUS_REPORT_LOST (1<<0)#define GEN8_OAHEADPTR _MMIO(0x2B0C) +#define GEN8_OAHEADPTR_MASK 0xffffffc0#define GEN8_OATAILPTR _MMIO(0x2B10) +#define GEN8_OATAILPTR_MASK 0xffffffc0#define OABUFFER_SIZE_128K (0<<3)#define OABUFFER_SIZE_256K (1<<3)@@ -705,7 +714,17 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)#define OA_MEM_SELECT_GGTT (1<<0) +/* +* Flexible, Aggregate EU Counter Registers. +* Note: these aren't contiguous +*/ #define EU_PERF_CNTL0 _MMIO(0xe458) +#define EU_PERF_CNTL1 _MMIO(0xe558) +#define EU_PERF_CNTL2 _MMIO(0xe658) +#define EU_PERF_CNTL3 _MMIO(0xe758) +#define EU_PERF_CNTL4 _MMIO(0xe45c) +#define EU_PERF_CNTL5 _MMIO(0xe55c) +#define EU_PERF_CNTL6 _MMIO(0xe65c)#define GDT_CHICKEN_BITS _MMIO(0x9840)#define GT_NOA_ENABLE 0x00000080@@ -2325,6 +2344,9 @@ enum skl_disp_power_wells {#define GEN8_RC_SEMA_IDLE_MSG_DISABLE (1 << 12)#define GEN8_FF_DOP_CLOCK_GATE_DISABLE (1<<10) +#define GEN6_RCS_PWR_FSM _MMIO(0x22ac) +#define GEN9_RCS_FE_FSM2 _MMIO(0x22a4) +/* Fuse readout registers for GT */ #define CHV_FUSE_GT _MMIO(VLV_DISPLAY_BASE + 0x2168)#define CHV_FGT_DISABLE_SS0 (1 << 10)drivers/gpu/drm/i915/intel_lrc.c +2
-0@@ -1879,6 +1879,8 @@ static void execlists_init_reg_state(u32 *regs, regs[CTX_LRI_HEADER_2 ] =MI_LOAD_REGISTER_IMM (1 );CTX_REG (regs,CTX_R_PWR_CLK_STATE ,GEN8_R_PWR_CLK_STATE ,make_rpcs (dev_priv)); + +i915_oa_init_reg_state (engine, ctx, regs); } }include/uapi/drm/i915_drm.h +12
-8@@ -1316,13 +1316,18 @@ struct drm_i915_gem_context_param { };enum drm_i915_oa_format { -I915_OA_FORMAT_A13 =1 , -I915_OA_FORMAT_A29 , -I915_OA_FORMAT_A13_B8_C8 , -I915_OA_FORMAT_B4_C8 , -I915_OA_FORMAT_A45_B8_C8 , -I915_OA_FORMAT_B4_C8_A16 , -I915_OA_FORMAT_C4_B8 , +I915_OA_FORMAT_A13 =1 ,/* HSW only */ +I915_OA_FORMAT_A29 ,/* HSW only */ +I915_OA_FORMAT_A13_B8_C8 ,/* HSW only */ +I915_OA_FORMAT_B4_C8 ,/* HSW only */ +I915_OA_FORMAT_A45_B8_C8 ,/* HSW only */ +I915_OA_FORMAT_B4_C8_A16 ,/* HSW only */ +I915_OA_FORMAT_C4_B8 ,/* HSW+ */ + +/* Gen8+ */ +I915_OA_FORMAT_A12 , +I915_OA_FORMAT_A12_B8_C8 , + I915_OA_FORMAT_A32u40_A4u32_B8_C8,I915_OA_FORMAT_MAX /* non-ABI */ }; --
Quoted 5 lines
2.11.0
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [PATCH v6 12/15] drm/i915/perf: Add OA unit support for Gen 8+
Quoted 878 lines · Matthew Auld
Quoted 876 lines · Lionel Landwerlin
From: Robert Bragg <[email protected]>
Enables access to OA unit metrics for BDW, CHV, SKL and BXT which all share (more-or-less) the same OA unit design.
Of particular note in comparison to Haswell: some OA unit HW config state has become per-context state and as a consequence it is somewhat more complicated to manage synchronous state changes from the cpu while there's no guarantee of what context (if any) is currently actively running on the gpu.
The periodic sampling frequency which can be particularly useful for system-wide analysis (as opposed to command stream synchronised
MI_REPORT_PERF_COUNTcommands) is perhaps the most surprising state to have become per-context save and restored (while the OABUFFER destination is still a shared, system-wide resource).This support for gen8+ takes care to consider a number of timing challenges involved in synchronously updating per-context state primarily by programming all config state from the cpu and updating all current and saved contexts synchronously while the OA unit is still disabled.
The driver intentionally avoids depending on command streamer programming to update OA state considering the lack of synchronization between the automatic loading of OACTXCONTROL state (that includes the periodic sampling state and enable state) on context restore and the parsing of any general purpose BB the driver can control. I.e. this implementation is careful to avoid the possibility of a context restore temporarily enabling any out-of-date periodic sampling state. In addition to the risk of transiently-out-of-date state being loaded automatically; there are also internal HW latencies involved in the loading of MUX configurations which would be difficult to account for from the command streamer (and we only want to enable the unit when once the MUX configuration is complete).
Since the Gen8+ OA unit design no longer supports clock gating the unit off for a single given context (which effectively stopped any progress of counters while any other context was running) and instead supports tagging OA reports with a context ID for filtering on the CPU, it means we can no longer hide the system-wide progress of counters from a non-privileged application only interested in metrics for its own context. Although we could theoretically try and subtract the progress of other contexts before forwarding reports via
read()we aren't in a position to filter reports captured viaMI_REPORT_PERF_COUNTcommands. As a result, for Gen8+, we always require the dev.i915.perf_stream_paranoid to be unset for any access to OA metrics if not root.v5: Drain submitted requests when enabling metric set to ensure no
lite-restore erases the context image we just updated (Lionel)v6: In addition to drain, switch to kernel context & update all
context in place (Chris)Signed-off-by Robert Bragg <[email protected]>Signed-off-by Lionel Landwerlin <[email protected]>Reviewed-by Matthew Auld <[email protected]> \o/--- drivers/gpu/drm/i915/i915_drv.h | 45 +- drivers/gpu/drm/i915/i915_perf.c | 957 ++++++++++++++++++++++++++++++++++++--- drivers/gpu/drm/i915/i915_reg.h | 22 + drivers/gpu/drm/i915/intel_lrc.c | 2 + include/uapi/drm/i915_drm.h | 19 +- 5 files changed, 952 insertions(+), 93 deletions(-)drivers/gpu/drm/i915/i915_drv.h +30
-15@@ -2064,9 +2064,17 @@ struct i915_oa_ops {void (*init_oa_buffer)(struct drm_i915_private *dev_priv);/** -* @enable_metric_set: Applies any MUX configuration to set up the -* Boolean and Custom (B/C) counters that are part of the counter -* reports being sampled. May apply system constraints such as +* @select_metric_set: The auto generated code that checks whether a +* requested OA config is applicable to the system and if so sets up +* the mux, oa and flex eu register config pointers according to the +* current dev_priv->perf.oa.metrics_set. +*/ +int (*select_metric_set)(struct drm_i915_private *dev_priv); + +/** +* @enable_metric_set: Selects and applies any MUX configuration to set +* up the Boolean and Custom (B/C) counters that are part of the +* counter reports being sampled. May apply system constraints such as * disabling EU clock gating as required. */ int (*enable_metric_set)(struct drm_i915_private *dev_priv);@@ -2097,20 +2105,13 @@ struct i915_oa_ops { size_t *offset);/** -* @oa_buffer_check: Check for OA buffer data + update tail -* -* This is either called via fops or the poll check hrtimer (atomic -* ctx) without any locks taken. +* @oa_hw_tail_read: read the OA tail pointer register * -* It's safe to read OA config state here unlocked, assuming that this -* is only called while the stream is enabled, while the global OA -* configuration can't be modified. -* -* Efficiency is more important than avoiding some false positives -* here, which will be handled gracefully - likely resulting in an -* %EAGAIN error for userspace. +* In particular this enables us to share all the fiddly code for +* handling the OA unit tail pointer race that affects multiple +* generations. */ -bool (*oa_buffer_check)(struct drm_i915_private *dev_priv); +u32 (*oa_hw_tail_read)(struct drm_i915_private *dev_priv); };struct intel_cdclk_state {@@ -2472,6 +2473,7 @@ struct drm_i915_private {struct {struct i915_vma *vma;u8 *vaddr; +u32 last_ctx_id;int format;int format_size;@@ -2541,6 +2543,14 @@ struct drm_i915_private { } oa_buffer;u32 gen7_latched_oastatus1; +u32 ctx_oactxctrl_offset; +u32 ctx_flexeu0_offset; + +/* The RPT_ID/reason field for Gen8+ includes a bit +* to determine if the CTX ID in the report is valid +* but the specific bit differs between Gen 8 and 9 +*/ +u32 gen8_valid_ctx_bit;struct i915_oa_ops ops;const struct i915_oa_format *oa_formats;@@ -2851,6 +2861,8 @@ intel_info(const struct drm_i915_private *dev_priv)#define IS_KBL_ULX (dev_priv) (INTEL_DEVID(dev_priv) == 0x590E || \ INTEL_DEVID(dev_priv) == 0x5915 || \ INTEL_DEVID(dev_priv) == 0x591E) +#define IS_SKL_GT2 (dev_priv) (IS_SKYLAKE(dev_priv) && \ + (INTEL_DEVID(dev_priv) & 0x00F0) == 0x0010)#define IS_SKL_GT3 (dev_priv) (IS_SKYLAKE(dev_priv) && \ (INTEL_DEVID(dev_priv) & 0x00F0) == 0x0020)#define IS_SKL_GT4 (dev_priv) (IS_SKYLAKE(dev_priv) && \@@ -3615,6 +3627,9 @@ i915_gem_context_lookup_timeline(struct i915_gem_context *ctx,int i915_perf_open_ioctl (struct drm_device *dev,void *data,struct drm_file *file); +void i915_oa_init_reg_state (struct intel_engine_cs *engine, +struct i915_gem_context *ctx, +uint32_t *reg_state);/* i915_gem_evict.c */ int __must_check i915_gem_evict_something(struct i915_address_space *vm,drivers/gpu/drm/i915/i915_perf.c +557
-35@@ -196,6 +196,12 @@#include "i915_drv.h" #include "i915_oa_hsw.h" +#include "i915_oa_bdw.h" +#include "i915_oa_chv.h" +#include "i915_oa_sklgt2.h" +#include "i915_oa_sklgt3.h" +#include "i915_oa_sklgt4.h" +#include "i915_oa_bxt.h" /* HW requires this to be a power of two, between 128k and 16M, though driver * is currently generally designed assuming the largest 16M size is used such @@ -215,7 +221,7 @@* * Although this can be observed explicitly while copying reports to userspace * by checking for a zeroed report-id field in tail reports, we want to account -* for this earlier, as part of the _oa_buffer_check to avoid lots of redundant +* for this earlier, as part of the oa_buffer_check to avoid lots of redundant * read() attempts. * * In effect we define a tail pointer for reading that lags the real tail @@ -237,7 +243,7 @@* indicates that an updated tail pointer is needed. * * Most of the implementation details for this workaround are in -* gen7_oa_buffer_check_unlocked() and gen7_appand_oa_reports() +* oa_buffer_check_unlocked() and _append_oa_reports() * * Note for posterity: previously the driver used to define an effective tail * pointer that lagged the real pointer by a 'tail margin' measured in bytes @@ -272,6 +278,13 @@ static u32 i915_perf_stream_paranoid = true;#define INVALID_CTX_ID 0xffffffff +/* On Gen8+ automatically triggered OA reports include a 'reason' field... */ +#define OAREPORT_REASON_MASK 0x3f +#define OAREPORT_REASON_SHIFT 19 +#define OAREPORT_REASON_TIMER (1<<0) +#define OAREPORT_REASON_CTX_SWITCH (1<<3) +#define OAREPORT_REASON_CLK_RATIO (1<<5) +/* For sysctl proc_dointvec_minmax of i915_oa_max_sample_rate * @@ -303,6 +316,13 @@ static struct i915_oa_format hsw_oa_formats[I915_OA_FORMAT_MAX] = { [I915_OA_FORMAT_C4_B8] = { 7, 64 }, }; +static struct i915_oa_format gen8_plus_oa_formats[I915_OA_FORMAT_MAX ] = { + [I915_OA_FORMAT_A12 ] = {0 ,64 }, + [I915_OA_FORMAT_A12_B8_C8 ] = {2 ,128 }, + [I915_OA_FORMAT_A32u40_A4u32_B8_C8] = {5 ,256 }, + [I915_OA_FORMAT_C4_B8 ] = {7 ,64 }, +}; +#define SAMPLE_OA_REPORT (1<<0)/** @@ -332,8 +352,20 @@ struct perf_open_properties {int oa_period_exponent; }; +static u32 gen8_oa_hw_tail_read (struct drm_i915_private *dev_priv) +{ +return I915_READ (GEN8_OATAILPTR ) &GEN8_OATAILPTR_MASK ; +} + +static u32 gen7_oa_hw_tail_read (struct drm_i915_private *dev_priv) +{ +u32 oastatus1 =I915_READ (GEN7_OASTATUS1 ); + +return oastatus1 &GEN7_OASTATUS1_TAIL_MASK ; +} +/** -* gen7_oa_buffer_check_unlocked - check for data and update tail ptr state +* oa_buffer_check_unlocked - check for data and update tail ptr state * @dev_priv: i915 device instance * * This is either called via fops (for blocking reads in user ctx) or the poll @@ -356,12 +388,11 @@ struct perf_open_properties {* * Returns: %true if the OA buffer contains data, else %false */ -static bool gen7_oa_buffer_check_unlocked (struct drm_i915_private *dev_priv) +static bool oa_buffer_check_unlocked (struct drm_i915_private *dev_priv) {int report_size = dev_priv->perf.oa.oa_buffer.format_size;unsigned long flags;unsigned int aged_idx; -u32 oastatus1;u32 head, hw_tail, aged_tail, aging_tail;u64 now;@@ -381,8 +412,7 @@ static bool gen7_oa_buffer_check_unlocked(struct drm_i915_private *dev_priv) aged_tail = dev_priv->perf.oa.oa_buffer.tails[aged_idx].offset; aging_tail = dev_priv->perf.oa.oa_buffer.tails[!aged_idx].offset; - oastatus1 =I915_READ (GEN7_OASTATUS1 ); - hw_tail = oastatus1 &GEN7_OASTATUS1_TAIL_MASK ; + hw_tail = dev_priv->perf.oa.ops.oa_hw_tail_read (dev_priv);/* The tail pointer increases in 64 byte increments, * not in report_size steps... @@ -404,6 +434,7 @@ static bool gen7_oa_buffer_check_unlocked(struct drm_i915_private *dev_priv)if (aging_tail !=INVALID_TAIL_PTR && ((now - dev_priv->perf.oa.oa_buffer.aging_timestamp) >OA_TAIL_MARGIN_NSEC )) { + aged_idx ^=1 ; dev_priv->perf.oa.oa_buffer.aged_tail_idx = aged_idx;@@ -553,6 +584,287 @@ static int append_oa_sample(struct i915_perf_stream *stream,* * Returns: 0 on success, negative error code on failure. */ +static int gen8_append_oa_reports (struct i915_perf_stream *stream, +char __user *buf, +size_t count, +size_t *offset) +{ +struct drm_i915_private *dev_priv = stream->dev_priv; +int report_size = dev_priv->perf.oa.oa_buffer.format_size; +u8 *oa_buf_base = dev_priv->perf.oa.oa_buffer.vaddr; +u32 gtt_offset =i915_ggtt_offset (dev_priv->perf.oa.oa_buffer.vma); +u32 mask = (OA_BUFFER_SIZE -1 ); +size_t start_offset = *offset; +unsigned long flags; +unsigned int aged_tail_idx; +u32 head, tail; +u32 taken; +int ret =0 ; + +if (WARN_ON (!stream->enabled)) +return -EIO ; + +spin_lock_irqsave (&dev_priv->perf.oa.oa_buffer.ptr_lock, flags); + + head = dev_priv->perf.oa.oa_buffer.head; + aged_tail_idx = dev_priv->perf.oa.oa_buffer.aged_tail_idx; + tail = dev_priv->perf.oa.oa_buffer.tails[aged_tail_idx].offset; + +spin_unlock_irqrestore (&dev_priv->perf.oa.oa_buffer.ptr_lock, flags); + +/* An invalid tail pointer here means we're still waiting for the poll +* hrtimer callback to give us a pointer +*/ +if (tail ==INVALID_TAIL_PTR ) +return -EAGAIN ; + +/* NB: oa_buffer.head/tail include the gtt_offset which we don't want +* while indexing relative to oa_buf_base. +*/ + head -= gtt_offset; + tail -= gtt_offset; + +/* An out of bounds or misaligned head or tail pointer implies a driver +* bug since we validate + align the tail pointers we read from the +* hardware and we are in full control of the head pointer which should +* only be incremented by multiples of the report size (notably also +* all a power of two). +*/ +if (WARN_ONCE (head >OA_BUFFER_SIZE || head % report_size || + tail >OA_BUFFER_SIZE || tail % report_size, +"Inconsistent OA buffer pointers: head = %u, tail = %u\n" , + head, tail)) +return -EIO ; + + +for (/* none */ ; + (taken =OA_TAKEN (tail, head)); + head = (head + report_size) & mask) { +u8 *report = oa_buf_base + head; +u32 *report32 = (void *)report; +u32 ctx_id; +u32 reason; + +/* All the report sizes factor neatly into the buffer +* size so we never expect to see a report split +* between the beginning and end of the buffer. +* +* Given the initial alignment check a misalignment +* here would imply a driver bug that would result +* in an overrun. +*/ +if (WARN_ON ((OA_BUFFER_SIZE - head) < report_size)) { +DRM_ERROR ("Spurious OA head ptr: non-integral report offset\n" ); +break ; + } + +/* The reason field includes flags identifying what +* triggered this specific report (mostly timer +* triggered or e.g. due to a context switch). +* +* This field is never expected to be zero so we can +* check that the report isn't invalid before copying +* it to userspace... +*/ + reason = ((report32[0 ] >>OAREPORT_REASON_SHIFT ) & +OAREPORT_REASON_MASK ); +if (reason ==0 ) { +if (__ratelimit (&dev_priv->perf.oa.spurious_report_rs)) +DRM_NOTE ("Skipping spurious, invalid OA report\n" ); +continue ; + } + +/* XXX: Just keep the lower 21 bits for now since I'm not +* entirely sure if the HW touches any of the higher bits in +* this field +*/ + ctx_id = report32[2 ] &0x1fffff ; + +/* Squash whatever is in the CTX_ID field if it's marked as +* invalid to be sure we avoid false-positive, single-context +* filtering below... +* +* Note: that we don't clear the valid_ctx_bit so userspace can +* understand that the ID has been squashed by the kernel. +*/ +if (!(report32[0 ] & dev_priv->perf.oa.gen8_valid_ctx_bit)) + ctx_id = report32[2 ] =INVALID_CTX_ID ; + +/* NB: For Gen 8 the OA unit no longer supports clock gating +* off for a specific context and the kernel can't securely +* stop the counters from updating as system-wide / global +* values. +* +* Automatic reports now include a context ID so reports can be +* filtered on the cpu but it's not worth trying to +* automatically subtract/hide counter progress for other +* contexts while filtering since we can't stop userspace +* issuing MI_REPORT_PERF_COUNT commands which would still +* provide a side-band view of the real values. +* +* To allow userspace (such as Mesa/GL_INTEL_performance_query) +* to normalize counters for a single filtered context then it +* needs be forwarded bookend context-switch reports so that it +* can track switches in between MI_REPORT_PERF_COUNT commands +* and can itself subtract/ignore the progress of counters +* associated with other contexts. Note that the hardware +* automatically triggers reports when switching to a new +* context which are tagged with the ID of the newly active +* context. To avoid the complexity (and likely fragility) of +* reading ahead while parsing reports to try and minimize +* forwarding redundant context switch reports (i.e. between +* other, unrelated contexts) we simply elect to forward them +* all. +* +* We don't rely solely on the reason field to identify context +* switches since it's not-uncommon for periodic samples to +* identify a switch before any 'context switch' report. +*/ +if (!dev_priv->perf.oa.exclusive_stream->ctx || + dev_priv->perf.oa.specific_ctx_id == ctx_id || + (dev_priv->perf.oa.oa_buffer.last_ctx_id == + dev_priv->perf.oa.specific_ctx_id) || + reason &OAREPORT_REASON_CTX_SWITCH ) { + +/* While filtering for a single context we avoid +* leaking the IDs of other contexts. +*/ +if (dev_priv->perf.oa.exclusive_stream->ctx && + dev_priv->perf.oa.specific_ctx_id != ctx_id) { + report32[2 ] =INVALID_CTX_ID ; + } + + ret =append_oa_sample (stream, buf, count, offset, + report); +if (ret) +break ; + + dev_priv->perf.oa.oa_buffer.last_ctx_id = ctx_id; + } + +/* The above reason field sanity check is based on +* the assumption that the OA buffer is initially +* zeroed and we reset the field after copying so the +* check is still meaningful once old reports start +* being overwritten. +*/ + report32[0 ] =0 ; + } + +if (start_offset != *offset) { +spin_lock_irqsave (&dev_priv->perf.oa.oa_buffer.ptr_lock, flags); + +/* We removed the gtt_offset for the copy loop above, indexing +* relative to oa_buf_base so put back here... +*/ + head += gtt_offset; + +I915_WRITE (GEN8_OAHEADPTR , head &GEN8_OAHEADPTR_MASK ); + dev_priv->perf.oa.oa_buffer.head = head; + +spin_unlock_irqrestore (&dev_priv->perf.oa.oa_buffer.ptr_lock, flags); + } + +return ret; +} + +/** +* gen8_oa_read - copy status records then buffered OA reports +* @stream: An i915-perf stream opened for OA metrics +* @buf: destination buffer given by userspace +* @count: the number of bytes userspace wants to read +* @offset: (inout): the current position for writing into @buf +* +* Checks OA unit status registers and if necessary appends corresponding +* status records for userspace (such as for a buffer full condition) and then +* initiate appending any buffered OA reports. +* +* Updates @offset according to the number of bytes successfully copied into +* the userspace buffer. +* +* NB: some data may be successfully copied to the userspace buffer +* even if an error is returned, and this is reflected in the +* updated @offset. +* +* Returns: zero on success or a negative error code +*/ +static int gen8_oa_read (struct i915_perf_stream *stream, +char __user *buf, +size_t count, +size_t *offset) +{ +struct drm_i915_private *dev_priv = stream->dev_priv; +u32 oastatus; +int ret; + +if (WARN_ON (!dev_priv->perf.oa.oa_buffer.vaddr)) +return -EIO ; + + oastatus =I915_READ (GEN8_OASTATUS ); + +/* We treat OABUFFER_OVERFLOW as a significant error: +* +* Although theoretically we could handle this more gracefully +* sometimes, some Gens don't correctly suppress certain +* automatically triggered reports in this condition and so we +* have to assume that old reports are now being trampled +* over. +* +* Considering how we don't currently give userspace control +* over the OA buffer size and always configure a large 16MB +* buffer, then a buffer overflow does anyway likely indicate +* that something has gone quite badly wrong. +*/ +if (oastatus &GEN8_OASTATUS_OABUFFER_OVERFLOW ) { + ret =append_oa_status (stream, buf, count, offset, +DRM_I915_PERF_RECORD_OA_BUFFER_LOST ); +if (ret) +return ret; + +DRM_DEBUG ("OA buffer overflow (exponent = %d): force restart\n" , + dev_priv->perf.oa.period_exponent); + + dev_priv->perf.oa.ops.oa_disable (dev_priv); + dev_priv->perf.oa.ops.oa_enable (dev_priv); + +/* Note: .oa_enable() is expected to re-init the oabuffer +* and reset GEN8_OASTATUS for us +*/ + oastatus =I915_READ (GEN8_OASTATUS ); + } + +if (oastatus &GEN8_OASTATUS_REPORT_LOST ) { + ret =append_oa_status (stream, buf, count, offset, +DRM_I915_PERF_RECORD_OA_REPORT_LOST ); +if (ret) +return ret; +I915_WRITE (GEN8_OASTATUS , + oastatus & ~GEN8_OASTATUS_REPORT_LOST ); + } + +return gen8_append_oa_reports (stream, buf, count, offset); +} + +/** +* Copies all buffered OA reports into userspace read() buffer. +* @stream: An i915-perf stream opened for OA metrics +* @buf: destination buffer given by userspace +* @count: the number of bytes userspace wants to read +* @offset: (inout): the current position for writing into @buf +* +* Notably any error condition resulting in a short read (-%ENOSPC or +* -%EFAULT) will be returned even though one or more records may +* have been successfully copied. In this case it's up to the caller +* to decide if the error should be squashed before returning to +* userspace. +* +* Note: reports are consumed from the head, and appended to the +* tail, so the tail chases the head?... If you think that's mad +* and back-to-front you're not alone, but this follows the +* Gen PRM naming convention. +* +* Returns: 0 on success, negative error code on failure. +*/ static int gen7_append_oa_reports(struct i915_perf_stream *stream, char __user *buf, size_t count,@@ -733,7 +1045,8 @@ static int gen7_oa_read(struct i915_perf_stream *stream,if (ret)return ret; -DRM_DEBUG ("OA buffer overflow: force restart\n" ); +DRM_DEBUG ("OA buffer overflow (exponent = %d): force restart\n" , + dev_priv->perf.oa.period_exponent); dev_priv->perf.oa.ops.oa_disable (dev_priv); dev_priv->perf.oa.ops.oa_enable (dev_priv);@@ -776,7 +1089,7 @@ static int i915_oa_wait_unlocked(struct i915_perf_stream *stream)return -EIO ;return wait_event_interruptible (dev_priv->perf.oa.poll_wq, - dev_priv->perf.oa.ops.oa_buffer_check (dev_priv)); +oa_buffer_check_unlocked (dev_priv)); }/** @@ -833,33 +1146,37 @@ static int i915_oa_read(struct i915_perf_stream *stream,static int oa_get_render_ctx_id (struct i915_perf_stream *stream) {struct drm_i915_private *dev_priv = stream->dev_priv; -struct intel_engine_cs *engine = dev_priv->engine[RCS ]; -int ret; - ret =i915_mutex_lock_interruptible (&dev_priv->drm); -if (ret) -return ret; +if (i915.enable_execlists) + dev_priv->perf.oa.specific_ctx_id = stream->ctx->hw_id; +else { +struct intel_engine_cs *engine = dev_priv->engine[RCS ]; +int ret; -/* As the ID is the gtt offset of the context's vma we pin -* the vma to ensure the ID remains fixed. -* -* NB: implied RCS engine... -*/ - ret = engine->context_pin (engine, stream->ctx); -if (ret) - goto unlock; + ret =i915_mutex_lock_interruptible (&dev_priv->drm); +if (ret) +return ret; -/* Explicitly track the ID (instead of calling i915_ggtt_offset() -* on the fly) considering the difference with gen8+ and -* execlists -*/ - dev_priv->perf.oa.specific_ctx_id = -i915_ggtt_offset (stream->ctx->engine[engine->id].state); +/* As the ID is the gtt offset of the context's vma we pin +* the vma to ensure the ID remains fixed. +*/ + ret = engine->context_pin (engine, stream->ctx); +if (ret) { +mutex_unlock (&dev_priv->drm.struct_mutex); +return ret; + } -unlock: -mutex_unlock (&dev_priv->drm.struct_mutex); +/* Explicitly track the ID (instead of calling +* i915_ggtt_offset() on the fly) considering the difference +* with gen8+ and execlists +*/ + dev_priv->perf.oa.specific_ctx_id = +i915_ggtt_offset (stream->ctx->engine[engine->id].state); -return ret; +mutex_unlock (&dev_priv->drm.struct_mutex); + } + +return 0 ; }/** @@ -874,12 +1191,16 @@ static void oa_put_render_ctx_id(struct i915_perf_stream *stream)struct drm_i915_private *dev_priv = stream->dev_priv;struct intel_engine_cs *engine = dev_priv->engine[RCS ]; -mutex_lock (&dev_priv->drm.struct_mutex); +if (i915.enable_execlists) { + dev_priv->perf.oa.specific_ctx_id =INVALID_CTX_ID ; + }else { +mutex_lock (&dev_priv->drm.struct_mutex); - dev_priv->perf.oa.specific_ctx_id =INVALID_CTX_ID ; - engine->context_unpin (engine, stream->ctx); + dev_priv->perf.oa.specific_ctx_id =INVALID_CTX_ID ; + engine->context_unpin (engine, stream->ctx); -mutex_unlock (&dev_priv->drm.struct_mutex); +mutex_unlock (&dev_priv->drm.struct_mutex); + } } static void@@ -969,6 +1290,61 @@ static void gen7_init_oa_buffer(struct drm_i915_private *dev_priv) dev_priv->perf.oa.pollin = false; } +static void gen8_init_oa_buffer (struct drm_i915_private *dev_priv) +{ +u32 gtt_offset =i915_ggtt_offset (dev_priv->perf.oa.oa_buffer.vma); +unsigned long flags; + +spin_lock_irqsave (&dev_priv->perf.oa.oa_buffer.ptr_lock, flags); + +I915_WRITE (GEN8_OASTATUS ,0 ); +I915_WRITE (GEN8_OAHEADPTR , gtt_offset); + dev_priv->perf.oa.oa_buffer.head = gtt_offset; + +I915_WRITE (GEN8_OABUFFER_UDW ,0 ); + +/* PRM says: +* +* "This MMIO must be set before the OATAILPTR +* register and after the OAHEADPTR register. This is +* to enable proper functionality of the overflow +* bit." +*/ +I915_WRITE (GEN8_OABUFFER , gtt_offset | +OABUFFER_SIZE_16M |OA_MEM_SELECT_GGTT ); +I915_WRITE (GEN8_OATAILPTR , gtt_offset &GEN8_OATAILPTR_MASK ); + +/* Mark that we need updated tail pointers to read from... */ + dev_priv->perf.oa.oa_buffer.tails[0 ].offset =INVALID_TAIL_PTR ; + dev_priv->perf.oa.oa_buffer.tails[1 ].offset =INVALID_TAIL_PTR ; + +/* Reset state used to recognise context switches, affecting which +* reports we will forward to userspace while filtering for a single +* context. +*/ + dev_priv->perf.oa.oa_buffer.last_ctx_id =INVALID_CTX_ID ; + +spin_unlock_irqrestore (&dev_priv->perf.oa.oa_buffer.ptr_lock, flags); + +/* NB: although the OA buffer will initially be allocated +* zeroed via shmfs (and so this memset is redundant when +* first allocating), we may re-init the OA buffer, either +* when re-enabling a stream or in error/reset paths. +* +* The reason we clear the buffer for each re-init is for the +* sanity check in gen8_append_oa_reports() that looks at the +* reason field to make sure it's non-zero which relies on +* the assumption that new reports are being written to zeroed +* memory... +*/ +memset (dev_priv->perf.oa.oa_buffer.vaddr,0 ,OA_BUFFER_SIZE ); + +/* Maybe make ->pollin per-stream state if we support multiple +* concurrent streams in the future. +*/ + dev_priv->perf.oa.pollin = false; +} +static int alloc_oa_buffer (struct drm_i915_private *dev_priv) {struct drm_i915_gem_object *bo;@@ -1113,6 +1489,280 @@ static void hsw_disable_metric_set(struct drm_i915_private *dev_priv) ~GT_NOA_ENABLE)); } +/* +* From Broadwell PRM, 3D-Media-GPGPU -> Register State Context +* +* MMIO reads or writes to any of the registers listed in the +* “Register State Context image” subsections through HOST/IA +* MMIO interface for debug purposes must follow the steps below: +* +* - SW should set the Force Wakeup bit to prevent GT from entering C6. +* - Write 0x2050[31:0] = 0x00010001 (disable sequence). +* - Disable IDLE messaging in CS (Write 0x2050[31:0] = 0x00010001). +* - BDW: Poll/Wait for register bits of 0x22AC[6:0] turn to 0x30 value. +* - SKL+: Poll/Wait for register bits of 0x22A4[6:0] turn to 0x30 value. +* - Read/Write to desired MMIO registers. +* - Enable IDLE messaging in CS (Write 0x2050[31:0] = 0x00010000). +* - Force Wakeup bit should be reset to enable C6 entry. +* +* XXX: don't nest or overlap calls to this API, it has no ref +* counting to track how many entities require the RCS to be +* blocked from being idle. +*/ +static int gen8_begin_ctx_mmio (struct drm_i915_private *dev_priv) +{ +i915_reg_t fsm_reg =INTEL_GEN (dev_priv) >8 ? +GEN9_RCS_FE_FSM2 :GEN6_RCS_PWR_FSM ; +int ret =0 ; + +/* There's only no active context while idle in execlist mode +* (though we shouldn't be using this in any other case) +*/ +if (WARN_ON (!i915.enable_execlists)) +return -EINVAL ; + +intel_uncore_forcewake_get (dev_priv,FORCEWAKE_RENDER ); + +I915_WRITE (GEN6_RC_SLEEP_PSMI_CONTROL , +_MASKED_BIT_ENABLE (GEN6_PSMI_SLEEP_MSG_DISABLE )); + +/* Note: we don't currently have a good handle on the maximum +* latency for this wake up so while we only need to hold rcs +* busy from process context we at least keep the waiting +* interruptible... +*/ + ret =intel_wait_for_register (dev_priv, fsm_reg,0x3f ,0x30 ,50 ); +if (ret == -ETIMEDOUT ) +DRM_ERROR ("Timeout waiting to be able to begin per-ctx mmio\n" ); + +if (ret) +intel_uncore_forcewake_put (dev_priv,FORCEWAKE_RENDER ); + +return ret; +} + +static void gen8_end_ctx_mmio (struct drm_i915_private *dev_priv) +{ +if (WARN_ON (!i915.enable_execlists)) +return ; + +I915_WRITE (GEN6_RC_SLEEP_PSMI_CONTROL , +_MASKED_BIT_DISABLE (GEN6_PSMI_SLEEP_MSG_DISABLE )); +intel_uncore_forcewake_put (dev_priv,FORCEWAKE_RENDER ); +} + +/* NB: It must always remain pointer safe to run this even if the OA unit +* has been disabled. +* +* It's fine to put out-of-date values into these per-context registers +* in the case that the OA unit has been disabled. +*/ +static void gen8_update_reg_state_unlocked (struct i915_gem_context *ctx, +u32 *reg_state) +{ +struct drm_i915_private *dev_priv = ctx->i915; +const struct i915_oa_reg *flex_regs = dev_priv->perf.oa.flex_regs; +int n_flex_regs = dev_priv->perf.oa.flex_regs_len; +u32 ctx_oactxctrl = dev_priv->perf.oa.ctx_oactxctrl_offset; +u32 ctx_flexeu0 = dev_priv->perf.oa.ctx_flexeu0_offset; +/* The MMIO offsets for Flex EU registers aren't contiguous */ +u32 flex_mmio[] = { +i915_mmio_reg_offset (EU_PERF_CNTL0 ), +i915_mmio_reg_offset (EU_PERF_CNTL1 ), +i915_mmio_reg_offset (EU_PERF_CNTL2 ), +i915_mmio_reg_offset (EU_PERF_CNTL3 ), +i915_mmio_reg_offset (EU_PERF_CNTL4 ), +i915_mmio_reg_offset (EU_PERF_CNTL5 ), +i915_mmio_reg_offset (EU_PERF_CNTL6 ), + }; +int i; + + reg_state[ctx_oactxctrl] =i915_mmio_reg_offset (GEN8_OACTXCONTROL ); + reg_state[ctx_oactxctrl+1 ] = (dev_priv->perf.oa.period_exponent << +GEN8_OA_TIMER_PERIOD_SHIFT ) | + (dev_priv->perf.oa.periodic ? +GEN8_OA_TIMER_ENABLE :0 ) | +GEN8_OA_COUNTER_RESUME ; + +for (i =0 ; i <ARRAY_SIZE (flex_mmio); i++) { +u32 state_offset = ctx_flexeu0 + i *2 ; +u32 mmio = flex_mmio[i]; + +/* This arbitrary default will select the 'EU FPU0 Pipeline +* Active' event. In the future it's anticipated that there +* will be an explicit 'No Event' we can select, but not yet... +*/ +u32 value =0 ; +int j; + +for (j =0 ; j < n_flex_regs; j++) { +if (i915_mmio_reg_offset (flex_regs[j].addr) == mmio) { + value = flex_regs[j].value; +break ; + } + } + + reg_state[state_offset] = mmio; + reg_state[state_offset+1 ] = value; + } +} + +/* Manages updating the per-context aspects of the OA stream +* configuration across all contexts. +* +* The awkward consideration here is that OACTXCONTROL controls the +* exponent for periodic sampling which is primarily used for system +* wide profiling where we'd like a consistent sampling period even in +* the face of context switches. +* +* Our approach of updating the register state context (as opposed to +* say using a workaround batch buffer) ensures that the hardware +* won't automatically reload an out-of-date timer exponent even +* transiently before a WA BB could be parsed. +* +* This function needs to: +* - Ensure the currently running context's per-context OA state is +* updated +* - Ensure that all existing contexts will have the correct per-context +* OA state if they are scheduled for use. +* - Ensure any new contexts will be initialized with the correct +* per-context OA state. +* +* Note: it's only the RCS/Render context that has any OA state. +*/ +static int gen8_configure_all_contexts (struct drm_i915_private *dev_priv) +{ +struct i915_gem_context *ctx; +int ret; +
Quoted 8 lines
drivers/gpu/drm/i915/i915_perf.c static void hsw_disable_metric_set(struct drm_i915_private *dev_priv) · line 1644
Quoted 7 lines
drivers/gpu/drm/i915/i915_perf.c +7
-0+ ret =i915_mutex_lock_interruptible (&dev_priv->drm); +if (ret) +return ret; + +/* Switch away from any user context. */ + ret =i915_gem_switch_to_kernel_context (dev_priv); +if (ret)
mutex_unlock(&dev_priv->drm.struct_mutex);
Duh! Thanks!
Quoted 38 lines
Quoted 38 lines
drivers/gpu/drm/i915/i915_perf.c +37
-0+return ret; + +/* The OA register config is setup through the context image. This image +* might be written to by the GPU on context switch (in particular on +* lite-restore). This means we can't safely update a context's image, +* if this context is scheduled/submitted to run on the GPU. +* +* We could emit the OA register config through the batch buffer but +* this might leave small interval of time where the OA unit is +* configured at an invalid sampling period. +* +* So far the best way to work around this issue seems to be draining +* the GPU from any submitted work. +*/ + ret =i915_gem_wait_for_idle (dev_priv, +I915_WAIT_INTERRUPTIBLE | +I915_WAIT_LOCKED ); +if (ret) { +mutex_unlock (&dev_priv->drm.struct_mutex); +return ret; + } + +/* Update all contexts now that we've stalled the submission. */ +list_for_each_entry (ctx, &dev_priv->context_list, link) { +if (!ctx->engine[RCS ].initialised) +continue ; + +gen8_update_reg_state_unlocked (ctx, + ctx->engine[RCS ].lrc_reg_state); + } + +mutex_unlock (&dev_priv->drm.struct_mutex); + +/* Now update the current context. +* +* Note: Using MMIO to update per-context registers requires +* some extra care...
Quoted 8 lines
drivers/gpu/drm/i915/i915_perf.c static void hsw_disable_metric_set(struct drm_i915_private *dev_priv) · line 1687
Quoted 6 lines
drivers/gpu/drm/i915/i915_perf.c +6
-0+*/ + ret =gen8_begin_ctx_mmio (dev_priv); +if (ret) { +DRM_ERROR ("Failed to bring RCS out of idle to update current ctx OA state\n" ); +return ret; + }So do we still need the mmio for the current context, given that we now idle everything and then update the reg state?
I would say yes, because we don't have any guarantee that another context is going to be scheduled after that. Yet we want the config to be applied before we return.
Quoted 514 lines
Quoted 514 lines
drivers/gpu/drm/i915/i915_perf.c +279
-36+ +I915_WRITE (GEN8_OACTXCONTROL , ((dev_priv->perf.oa.period_exponent << +GEN8_OA_TIMER_PERIOD_SHIFT ) | + (dev_priv->perf.oa.periodic ? +GEN8_OA_TIMER_ENABLE :0 ) | +GEN8_OA_COUNTER_RESUME )); + +config_oa_regs (dev_priv, dev_priv->perf.oa.flex_regs, + dev_priv->perf.oa.flex_regs_len); + +gen8_end_ctx_mmio (dev_priv); + +return 0 ; +} + +static int gen8_enable_metric_set (struct drm_i915_private *dev_priv) +{ +int ret = dev_priv->perf.oa.ops.select_metric_set (dev_priv); + +if (ret) +return ret; + +/* We disable slice/unslice clock ratio change reports on SKL since +* they are too noisy. The HW generates a lot of redundant reports +* where the ratio hasn't really changed causing a lot of redundant +* work to processes and increasing the chances we'll hit buffer +* overruns. +* +* Although we don't currently use the 'disable overrun' OABUFFER +* feature it's worth noting that clock ratio reports have to be +* disabled before considering to use that feature since the HW doesn't +* correctly block these reports. +* +* Currently none of the high-level metrics we have depend on knowing +* this ratio to normalize. +* +* Note: This register is not power context saved and restored, but +* that's OK considering that we disable RC6 while the OA unit is +* enabled. +* +* The _INCLUDE_CLK_RATIO bit allows the slice/unslice frequency to +* be read back from automatically triggered reports, as part of the +* RPT_ID field. +*/ +if (IS_SKYLAKE (dev_priv) ||IS_BROXTON (dev_priv)) { +I915_WRITE (GEN8_OA_DEBUG , +_MASKED_BIT_ENABLE (GEN9_OA_DEBUG_DISABLE_CLK_RATIO_REPORTS | +GEN9_OA_DEBUG_INCLUDE_CLK_RATIO )); + } + +I915_WRITE (GDT_CHICKEN_BITS ,0xA0 ); +config_oa_regs (dev_priv, dev_priv->perf.oa.mux_regs, + dev_priv->perf.oa.mux_regs_len); +I915_WRITE (GDT_CHICKEN_BITS ,0x80 ); + +/* It takes a fairly long time for a new MUX configuration to +* be be applied after these register writes. This delay +* duration is take from Haswell (derived empirically based on +* the render_basic config) but hopefully it covers the +* maximum configuration latency for Gen8+ too... +*/ +usleep_range (15000 ,20000 ); + +config_oa_regs (dev_priv, dev_priv->perf.oa.b_counter_regs, + dev_priv->perf.oa.b_counter_regs_len); + + ret =gen8_configure_all_contexts (dev_priv); +if (ret) +return ret; + +return 0 ; +} + +static void gen8_disable_metric_set (struct drm_i915_private *dev_priv) +{ +/* NOP */ +} +static void gen7_update_oacontrol_locked (struct drm_i915_private *dev_priv) {lockdep_assert_held (&dev_priv->perf.hook_lock);@@ -1157,6 +1807,29 @@ static void gen7_oa_enable(struct drm_i915_private *dev_priv)spin_unlock_irqrestore (&dev_priv->perf.hook_lock, flags); } +static void gen8_oa_enable (struct drm_i915_private *dev_priv) +{ +u32 report_format = dev_priv->perf.oa.oa_buffer.format; + +/* Reset buf pointers so we don't forward reports from before now. +* +* Think carefully if considering trying to avoid this, since it +* also ensures status flags and the buffer itself are cleared +* in error paths, and we have checks for invalid reports based +* on the assumption that certain fields are written to zeroed +* memory which this helps maintains. +*/ +gen8_init_oa_buffer (dev_priv); + +/* Note: we don't rely on the hardware to perform single context +* filtering and instead filter on the cpu based on the context-id +* field of reports +*/ +I915_WRITE (GEN8_OACONTROL , (report_format << +GEN8_OA_REPORT_FORMAT_SHIFT ) | +GEN8_OA_COUNTER_ENABLE ); +} +/** * i915_oa_stream_enable - handle `I915_PERF_IOCTL_ENABLE` for OA stream * @stream: An i915 perf stream opened for OA metrics @@ -1183,6 +1856,11 @@ static void gen7_oa_disable(struct drm_i915_private *dev_priv)I915_WRITE (GEN7_OACONTROL ,0 ); } +static void gen8_oa_disable (struct drm_i915_private *dev_priv) +{ +I915_WRITE (GEN8_OACONTROL ,0 ); +} +/** * i915_oa_stream_disable - handle `I915_PERF_IOCTL_DISABLE` for OA stream * @stream: An i915 perf stream opened for OA metrics @@ -1361,6 +2039,21 @@ static int i915_oa_stream_init(struct i915_perf_stream *stream,return ret; } +void i915_oa_init_reg_state (struct intel_engine_cs *engine, +struct i915_gem_context *ctx, +u32 *reg_state) +{ +struct drm_i915_private *dev_priv = engine->i915; + +if (engine->id !=RCS ) +return ; + +if (!dev_priv->perf.initialized) +return ; + +gen8_update_reg_state_unlocked (ctx, reg_state); +} +/** * i915_perf_read_locked - &i915_perf_stream_ops->read with error normalisation * @stream: An i915 perf stream @@ -1486,7 +2179,7 @@ static enum hrtimer_restart oa_poll_check_timer_cb(struct hrtimer *hrtimer)container_of (hrtimer,typeof (*dev_priv), perf.oa.poll_check_timer); -if (dev_priv->perf.oa.ops.oa_buffer_check (dev_priv)) { +if (oa_buffer_check_unlocked (dev_priv)) { dev_priv->perf.oa.pollin = true;wake_up (&dev_priv->perf.oa.poll_wq); }@@ -1775,6 +2468,7 @@ i915_perf_open_ioctl_locked(struct drm_i915_private *dev_priv,struct i915_gem_context *specific_ctx =NULL ;struct i915_perf_stream *stream =NULL ;unsigned long f_flags =0 ; +bool privileged_op = true;int stream_fd;int ret;@@ -1792,12 +2486,28 @@ i915_perf_open_ioctl_locked(struct drm_i915_private *dev_priv, } } +/* On Haswell the OA unit supports clock gating off for a specific +* context and in this mode there's no visibility of metrics for the +* rest of the system, which we consider acceptable for a +* non-privileged client. +* +* For Gen8+ the OA unit no longer supports clock gating off for a +* specific context and the kernel can't securely stop the counters +* from updating as system-wide / global values. Even though we can +* filter reports based on the included context ID we can't block +* clients from seeing the raw / global counter values via +* MI_REPORT_PERF_COUNT commands and so consider it a privileged op to +* enable the OA unit by default. +*/ +if (IS_HASWELL (dev_priv) && specific_ctx) + privileged_op = false; +/* Similar to perf's kernel.perf_paranoid_cpu sysctl option * we check a dev.i915.perf_stream_paranoid sysctl option * to determine if it's ok to access system wide OA counters * without CAP_SYS_ADMIN privileges. */ -if (!specific_ctx && +if (privileged_op && i915_perf_stream_paranoid && !capable (CAP_SYS_ADMIN )) {DRM_DEBUG ("Insufficient privileges to open system-wide i915 perf stream\n" ); ret = -EACCES ;@@ -2069,9 +2779,6 @@ int i915_perf_open_ioctl(struct drm_device *dev, void *data,*/ void i915_perf_register (struct drm_i915_private *dev_priv) { -if (!IS_HASWELL (dev_priv)) -return ; -if (!dev_priv->perf.initialized)return ;@@ -2087,11 +2794,38 @@ void i915_perf_register(struct drm_i915_private *dev_priv)if (!dev_priv->perf.metrics_kobj) goto exit; -if (i915_perf_register_sysfs_hsw (dev_priv)) { -kobject_put (dev_priv->perf.metrics_kobj); - dev_priv->perf.metrics_kobj =NULL ; +if (IS_HASWELL (dev_priv)) { +if (i915_perf_register_sysfs_hsw (dev_priv)) + goto sysfs_error; + }else if (IS_BROADWELL (dev_priv)) { +if (i915_perf_register_sysfs_bdw (dev_priv)) + goto sysfs_error; + }else if (IS_CHERRYVIEW (dev_priv)) { +if (i915_perf_register_sysfs_chv (dev_priv)) + goto sysfs_error; + }else if (IS_SKYLAKE (dev_priv)) { +if (IS_SKL_GT2 (dev_priv)) { +if (i915_perf_register_sysfs_sklgt2 (dev_priv)) + goto sysfs_error; + }else if (IS_SKL_GT3 (dev_priv)) { +if (i915_perf_register_sysfs_sklgt3 (dev_priv)) + goto sysfs_error; + }else if (IS_SKL_GT4 (dev_priv)) { +if (i915_perf_register_sysfs_sklgt4 (dev_priv)) + goto sysfs_error; + }else + goto sysfs_error; + }else if (IS_BROXTON (dev_priv)) { +if (i915_perf_register_sysfs_bxt (dev_priv)) + goto sysfs_error; } + goto exit; + +sysfs_error: +kobject_put (dev_priv->perf.metrics_kobj); + dev_priv->perf.metrics_kobj =NULL ; + exit:mutex_unlock (&dev_priv->perf.lock); }@@ -2107,13 +2841,24 @@ void i915_perf_register(struct drm_i915_private *dev_priv)*/ void i915_perf_unregister (struct drm_i915_private *dev_priv) { -if (!IS_HASWELL (dev_priv)) -return ; -if (!dev_priv->perf.metrics_kobj)return ; -i915_perf_unregister_sysfs_hsw (dev_priv); +if (IS_HASWELL (dev_priv)) +i915_perf_unregister_sysfs_hsw (dev_priv); +else if (IS_BROADWELL (dev_priv)) +i915_perf_unregister_sysfs_bdw (dev_priv); +else if (IS_CHERRYVIEW (dev_priv)) +i915_perf_unregister_sysfs_chv (dev_priv); +else if (IS_SKYLAKE (dev_priv)) { +if (IS_SKL_GT2 (dev_priv)) +i915_perf_unregister_sysfs_sklgt2 (dev_priv); +else if (IS_SKL_GT3 (dev_priv)) +i915_perf_unregister_sysfs_sklgt3 (dev_priv); +else if (IS_SKL_GT4 (dev_priv)) +i915_perf_unregister_sysfs_sklgt4 (dev_priv); + }else if (IS_BROXTON (dev_priv)) +i915_perf_unregister_sysfs_bxt (dev_priv);kobject_put (dev_priv->perf.metrics_kobj); dev_priv->perf.metrics_kobj =NULL ;@@ -2172,36 +2917,105 @@ static struct ctl_table dev_root[] = {*/ void i915_perf_init (struct drm_i915_private *dev_priv) { -if (!IS_HASWELL (dev_priv)) -return ; - -hrtimer_init (&dev_priv->perf.oa.poll_check_timer, -CLOCK_MONOTONIC ,HRTIMER_MODE_REL ); - dev_priv->perf.oa.poll_check_timer.function = oa_poll_check_timer_cb; -init_waitqueue_head (&dev_priv->perf.oa.poll_wq); + dev_priv->perf.oa.n_builtin_sets =0 ; + +if (IS_HASWELL (dev_priv)) { + dev_priv->perf.oa.ops.init_oa_buffer = gen7_init_oa_buffer; + dev_priv->perf.oa.ops.enable_metric_set = hsw_enable_metric_set; + dev_priv->perf.oa.ops.disable_metric_set = hsw_disable_metric_set; + dev_priv->perf.oa.ops.oa_enable = gen7_oa_enable; + dev_priv->perf.oa.ops.oa_disable = gen7_oa_disable; + dev_priv->perf.oa.ops.read = gen7_oa_read; + dev_priv->perf.oa.ops.oa_hw_tail_read = + gen7_oa_hw_tail_read; + + dev_priv->perf.oa.oa_formats = hsw_oa_formats; + + dev_priv->perf.oa.n_builtin_sets = + i915_oa_n_builtin_metric_sets_hsw; + }else if (i915.enable_execlists) { +/* Note: that although we could theoretically also support the +* legacy ringbuffer mode on BDW (and earlier iterations of +* this driver, before upstreaming did this) it didn't seem +* worth the complexity to maintain now that BDW+ enable +* execlist mode by default. +*/ -INIT_LIST_HEAD (&dev_priv->perf.streams); -mutex_init (&dev_priv->perf.lock); -spin_lock_init (&dev_priv->perf.hook_lock); -spin_lock_init (&dev_priv->perf.oa.oa_buffer.ptr_lock); +if (IS_GEN8 (dev_priv)) { + dev_priv->perf.oa.ctx_oactxctrl_offset =0x120 ; + dev_priv->perf.oa.ctx_flexeu0_offset =0x2ce ; + dev_priv->perf.oa.gen8_valid_ctx_bit = (1 <<25 ); + +if (IS_BROADWELL (dev_priv)) { + dev_priv->perf.oa.n_builtin_sets = + i915_oa_n_builtin_metric_sets_bdw; + dev_priv->perf.oa.ops.select_metric_set = + i915_oa_select_metric_set_bdw; + }else if (IS_CHERRYVIEW (dev_priv)) { + dev_priv->perf.oa.n_builtin_sets = + i915_oa_n_builtin_metric_sets_chv; + dev_priv->perf.oa.ops.select_metric_set = + i915_oa_select_metric_set_chv; + } + }else if (IS_GEN9 (dev_priv)) { + dev_priv->perf.oa.ctx_oactxctrl_offset =0x128 ; + dev_priv->perf.oa.ctx_flexeu0_offset =0x3de ; + dev_priv->perf.oa.gen8_valid_ctx_bit = (1 <<16 ); + +if (IS_SKL_GT2 (dev_priv)) { + dev_priv->perf.oa.n_builtin_sets = + i915_oa_n_builtin_metric_sets_sklgt2; + dev_priv->perf.oa.ops.select_metric_set = + i915_oa_select_metric_set_sklgt2; + }else if (IS_SKL_GT3 (dev_priv)) { + dev_priv->perf.oa.n_builtin_sets = + i915_oa_n_builtin_metric_sets_sklgt3; + dev_priv->perf.oa.ops.select_metric_set = + i915_oa_select_metric_set_sklgt3; + }else if (IS_SKL_GT4 (dev_priv)) { + dev_priv->perf.oa.n_builtin_sets = + i915_oa_n_builtin_metric_sets_sklgt4; + dev_priv->perf.oa.ops.select_metric_set = + i915_oa_select_metric_set_sklgt4; + }else if (IS_BROXTON (dev_priv)) { + dev_priv->perf.oa.n_builtin_sets = + i915_oa_n_builtin_metric_sets_bxt; + dev_priv->perf.oa.ops.select_metric_set = + i915_oa_select_metric_set_bxt; + } + } - dev_priv->perf.oa.ops.init_oa_buffer = gen7_init_oa_buffer; - dev_priv->perf.oa.ops.enable_metric_set = hsw_enable_metric_set; - dev_priv->perf.oa.ops.disable_metric_set = hsw_disable_metric_set; - dev_priv->perf.oa.ops.oa_enable = gen7_oa_enable; - dev_priv->perf.oa.ops.oa_disable = gen7_oa_disable; - dev_priv->perf.oa.ops.read = gen7_oa_read; - dev_priv->perf.oa.ops.oa_buffer_check = - gen7_oa_buffer_check_unlocked; +if (dev_priv->perf.oa.n_builtin_sets) { + dev_priv->perf.oa.ops.init_oa_buffer = gen8_init_oa_buffer; + dev_priv->perf.oa.ops.enable_metric_set = + gen8_enable_metric_set; + dev_priv->perf.oa.ops.disable_metric_set = + gen8_disable_metric_set; + dev_priv->perf.oa.ops.oa_enable = gen8_oa_enable; + dev_priv->perf.oa.ops.oa_disable = gen8_oa_disable; + dev_priv->perf.oa.ops.read = gen8_oa_read; + dev_priv->perf.oa.ops.oa_hw_tail_read = + gen8_oa_hw_tail_read; + + dev_priv->perf.oa.oa_formats = gen8_plus_oa_formats; + } + } - dev_priv->perf.oa.oa_formats = hsw_oa_formats; +if (dev_priv->perf.oa.n_builtin_sets) { +hrtimer_init (&dev_priv->perf.oa.poll_check_timer, +CLOCK_MONOTONIC ,HRTIMER_MODE_REL ); + dev_priv->perf.oa.poll_check_timer.function = oa_poll_check_timer_cb; +init_waitqueue_head (&dev_priv->perf.oa.poll_wq); - dev_priv->perf.oa.n_builtin_sets = - i915_oa_n_builtin_metric_sets_hsw; +INIT_LIST_HEAD (&dev_priv->perf.streams); +mutex_init (&dev_priv->perf.lock); +spin_lock_init (&dev_priv->perf.hook_lock); +spin_lock_init (&dev_priv->perf.oa.oa_buffer.ptr_lock); - dev_priv->perf.sysctl_header =register_sysctl_table (dev_root); + dev_priv->perf.sysctl_header =register_sysctl_table (dev_root); - dev_priv->perf.initialized = true; + dev_priv->perf.initialized = true; + } }/** @@ -2216,5 +3030,6 @@ void i915_perf_fini(struct drm_i915_private *dev_priv)unregister_sysctl_table (dev_priv->perf.sysctl_header);memset (&dev_priv->perf.oa.ops,0 ,sizeof (dev_priv->perf.oa.ops)); + dev_priv->perf.initialized = false; }drivers/gpu/drm/i915/i915_reg.h +22
-0@@ -653,6 +653,12 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)#define GEN8_OACTXID _MMIO(0x2364) +#define GEN8_OA_DEBUG _MMIO(0x2B04) +#define GEN9_OA_DEBUG_DISABLE_CLK_RATIO_REPORTS (1<<5) +#define GEN9_OA_DEBUG_INCLUDE_CLK_RATIO (1<<6) +#define GEN9_OA_DEBUG_DISABLE_GO_1_0_REPORTS (1<<2) +#define GEN9_OA_DEBUG_DISABLE_CTX_SWITCH_REPORTS (1<<1) +#define GEN8_OACONTROL _MMIO(0x2B00)#define GEN8_OA_REPORT_FORMAT_A12 (0<<2)#define GEN8_OA_REPORT_FORMAT_A12_B8_C8 (2<<2)@@ -674,6 +680,7 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)#define GEN7_OABUFFER_STOP_RESUME_ENABLE (1<<1)#define GEN7_OABUFFER_RESUME (1<<0) +#define GEN8_OABUFFER_UDW _MMIO(0x23b4)#define GEN8_OABUFFER _MMIO(0x2b14)#define GEN7_OASTATUS1 _MMIO(0x2364)@@ -692,7 +699,9 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)#define GEN8_OASTATUS_REPORT_LOST (1<<0)#define GEN8_OAHEADPTR _MMIO(0x2B0C) +#define GEN8_OAHEADPTR_MASK 0xffffffc0#define GEN8_OATAILPTR _MMIO(0x2B10) +#define GEN8_OATAILPTR_MASK 0xffffffc0#define OABUFFER_SIZE_128K (0<<3)#define OABUFFER_SIZE_256K (1<<3)@@ -705,7 +714,17 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)#define OA_MEM_SELECT_GGTT (1<<0) +/* +* Flexible, Aggregate EU Counter Registers. +* Note: these aren't contiguous +*/ #define EU_PERF_CNTL0 _MMIO(0xe458) +#define EU_PERF_CNTL1 _MMIO(0xe558) +#define EU_PERF_CNTL2 _MMIO(0xe658) +#define EU_PERF_CNTL3 _MMIO(0xe758) +#define EU_PERF_CNTL4 _MMIO(0xe45c) +#define EU_PERF_CNTL5 _MMIO(0xe55c) +#define EU_PERF_CNTL6 _MMIO(0xe65c)#define GDT_CHICKEN_BITS _MMIO(0x9840)#define GT_NOA_ENABLE 0x00000080@@ -2325,6 +2344,9 @@ enum skl_disp_power_wells {#define GEN8_RC_SEMA_IDLE_MSG_DISABLE (1 << 12)#define GEN8_FF_DOP_CLOCK_GATE_DISABLE (1<<10) +#define GEN6_RCS_PWR_FSM _MMIO(0x22ac) +#define GEN9_RCS_FE_FSM2 _MMIO(0x22a4) +/* Fuse readout registers for GT */ #define CHV_FUSE_GT _MMIO(VLV_DISPLAY_BASE + 0x2168)#define CHV_FGT_DISABLE_SS0 (1 << 10)drivers/gpu/drm/i915/intel_lrc.c +2
-0@@ -1879,6 +1879,8 @@ static void execlists_init_reg_state(u32 *regs, regs[CTX_LRI_HEADER_2 ] =MI_LOAD_REGISTER_IMM (1 );CTX_REG (regs,CTX_R_PWR_CLK_STATE ,GEN8_R_PWR_CLK_STATE ,make_rpcs (dev_priv)); + +i915_oa_init_reg_state (engine, ctx, regs); } }include/uapi/drm/i915_drm.h +12
-7@@ -1316,13 +1316,18 @@ struct drm_i915_gem_context_param { };enum drm_i915_oa_format { -I915_OA_FORMAT_A13 =1 , -I915_OA_FORMAT_A29 , -I915_OA_FORMAT_A13_B8_C8 , -I915_OA_FORMAT_B4_C8 , -I915_OA_FORMAT_A45_B8_C8 , -I915_OA_FORMAT_B4_C8_A16 , -I915_OA_FORMAT_C4_B8 , +I915_OA_FORMAT_A13 =1 ,/* HSW only */ +I915_OA_FORMAT_A29 ,/* HSW only */ +I915_OA_FORMAT_A13_B8_C8 ,/* HSW only */ +I915_OA_FORMAT_B4_C8 ,/* HSW only */ +I915_OA_FORMAT_A45_B8_C8 ,/* HSW only */ +I915_OA_FORMAT_B4_C8_A16 ,/* HSW only */ +I915_OA_FORMAT_C4_B8 ,/* HSW+ */ + +/* Gen8+ */ +I915_OA_FORMAT_A12 , +I915_OA_FORMAT_A12_B8_C8 , + I915_OA_FORMAT_A32u40_A4u32_B8_C8,
Quoted 8 lines
Quoted 8 lines
include/uapi/drm/i915_drm.h +0
-1I915_OA_FORMAT_MAX /* non-ABI */ }; --2.11.0
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [PATCH v6 12/15] drm/i915/perf: Add OA unit support for Gen 8+
Quoted 1,064 lines · Lionel Landwerlin
Quoted 995 lines · Matthew Auld
Quoted 992 lines · Lionel Landwerlin
From: Robert Bragg <[email protected]>
Enables access to OA unit metrics for BDW, CHV, SKL and BXT which all share (more-or-less) the same OA unit design.
Of particular note in comparison to Haswell: some OA unit HW config state has become per-context state and as a consequence it is somewhat more complicated to manage synchronous state changes from the cpu while there's no guarantee of what context (if any) is currently actively running on the gpu.
The periodic sampling frequency which can be particularly useful for system-wide analysis (as opposed to command stream synchronised
MI_REPORT_PERF_COUNTcommands) is perhaps the most surprising state to have become per-context save and restored (while the OABUFFER destination is still a shared, system-wide resource).This support for gen8+ takes care to consider a number of timing challenges involved in synchronously updating per-context state primarily by programming all config state from the cpu and updating all current and saved contexts synchronously while the OA unit is still disabled.
The driver intentionally avoids depending on command streamer programming to update OA state considering the lack of synchronization between the automatic loading of OACTXCONTROL state (that includes the periodic sampling state and enable state) on context restore and the parsing of any general purpose BB the driver can control. I.e. this implementation is careful to avoid the possibility of a context restore temporarily enabling any out-of-date periodic sampling state. In addition to the risk of transiently-out-of-date state being loaded automatically; there are also internal HW latencies involved in the loading of MUX configurations which would be difficult to account for from the command streamer (and we only want to enable the unit when once the MUX configuration is complete).
Since the Gen8+ OA unit design no longer supports clock gating the unit off for a single given context (which effectively stopped any progress of counters while any other context was running) and instead supports tagging OA reports with a context ID for filtering on the CPU, it means we can no longer hide the system-wide progress of counters from a non-privileged application only interested in metrics for its own context. Although we could theoretically try and subtract the progress of other contexts before forwarding reports via
read()we aren't in a position to filter reports captured viaMI_REPORT_PERF_COUNTcommands. As a result, for Gen8+, we always require the dev.i915.perf_stream_paranoid to be unset for any access to OA metrics if not root.v5: Drain submitted requests when enabling metric set to ensure no
lite-restore erases the context image we just updated (Lionel)v6: In addition to drain, switch to kernel context & update all
context in place (Chris)Signed-off-by Robert Bragg <[email protected]>Signed-off-by Lionel Landwerlin <[email protected]>Reviewed-by Matthew Auld <[email protected]> \o/--- drivers/gpu/drm/i915/i915_drv.h | 45 +- drivers/gpu/drm/i915/i915_perf.c | 957 ++++++++++++++++++++++++++++++++++++--- drivers/gpu/drm/i915/i915_reg.h | 22 + drivers/gpu/drm/i915/intel_lrc.c | 2 + include/uapi/drm/i915_drm.h | 19 +- 5 files changed, 952 insertions(+), 93 deletions(-)drivers/gpu/drm/i915/i915_drv.h +0
-1@@ -2064,9 +2064,17 @@ struct i915_oa_ops {void (*init_oa_buffer)(struct drm_i915_private *dev_priv);/** -* @enable_metric_set: Applies any MUX configuration to set up the
drivers/gpu/drm/i915/i915_drv.h +1
-2-* Boolean and Custom (B/C) counters that are part of the counter -* reports being sampled. May apply system constraints such as +* @select_metric_set: The auto generated code that checks whether a
+ * requested OA config is applicable to the system and if so sets
up
+ * the mux, oa and flex eu register config pointers according to
thedrivers/gpu/drm/i915/i915_drv.h +6
-0+* current dev_priv->perf.oa.metrics_set. +*/ +int (*select_metric_set)(struct drm_i915_private *dev_priv); + +/** +* @enable_metric_set: Selects and applies any MUX configuration to set
drivers/gpu/drm/i915/i915_drv.h +2
-0+* up the Boolean and Custom (B/C) counters that are part of the +* counter reports being sampled. May apply system constraints such as
* disabling EU clock gating as required. */ int (*enable_metric_set)(struct drm_i915_private *dev_priv);drivers/gpu/drm/i915/i915_drv.h +0
-3@@ -2097,20 +2105,13 @@ struct i915_oa_ops { size_t *offset);/** -* @oa_buffer_check: Check for OA buffer data + update tail -* -* This is either called via fops or the poll check hrtimer (atomic
drivers/gpu/drm/i915/i915_drv.h +1
-2-* ctx) without any locks taken. +* @oa_hw_tail_read: read the OA tail pointer register * -* It's safe to read OA config state here unlocked, assuming that this
- * is only called while the stream is enabled, while the global
OAdrivers/gpu/drm/i915/i915_drv.h +0
-3-* configuration can't be modified. -* -* Efficiency is more important than avoiding some false positives
- * here, which will be handled gracefully - likely resulting in
andrivers/gpu/drm/i915/i915_drv.h +9
-2-* %EAGAIN error for userspace. +* In particular this enables us to share all the fiddly code for +* handling the OA unit tail pointer race that affects multiple +* generations. */ -bool (*oa_buffer_check)(struct drm_i915_private *dev_priv); +u32 (*oa_hw_tail_read)(struct drm_i915_private *dev_priv); };struct intel_cdclk_state {@@ -2472,6 +2473,7 @@ struct drm_i915_private {struct {struct i915_vma *vma;u8 *vaddr; +u32 last_ctx_id;int format;int format_size;@@ -2541,6 +2543,14 @@ struct drm_i915_private { } oa_buffer;u32 gen7_latched_oastatus1; +u32 ctx_oactxctrl_offset; +u32 ctx_flexeu0_offset; + +/* The RPT_ID/reason field for Gen8+ includes a bit
+ * to determine if the CTX ID in the report is
valid
+ * but the specific bit differs between Gen 8 and
9drivers/gpu/drm/i915/i915_drv.h +4
-0+*/ +u32 gen8_valid_ctx_bit;struct i915_oa_ops ops;const struct i915_oa_format *oa_formats;@@ -2851,6 +2861,8 @@ intel_info(const struct drm_i915_private *dev_priv)#define IS_KBL_ULX (dev_priv) (INTEL_DEVID(dev_priv) == 0x590E || \ INTEL_DEVID(dev_priv) == 0x5915 || \ INTEL_DEVID(dev_priv) == 0x591E) +#define IS_SKL_GT2 (dev_priv) (IS_SKYLAKE(dev_priv) && \ + (INTEL_DEVID(dev_priv) & 0x00F0) == 0x0010)#define IS_SKL_GT3 (dev_priv) (IS_SKYLAKE(dev_priv) && \ (INTEL_DEVID(dev_priv) & 0x00F0) ==
0x0020)
#defineIS_SKL_GT4(dev_priv)(IS_SKYLAKE(dev_priv)&& \drivers/gpu/drm/i915/i915_drv.h +3
-0@@ -3615,6 +3627,9 @@ i915_gem_context_lookup_timeline(structi915_gem_context *ctx, int i915_perf_open_ioctl(struct drm_device *dev, void *data,struct drm_file *file); +void i915_oa_init_reg_state (struct intel_engine_cs *engine, +struct i915_gem_context *ctx, +uint32_t *reg_state);/* i915_gem_evict.c */ int
__must_checki915_gem_evict_something(structi915_address_space
*vm,drivers/gpu/drm/i915/i915_perf.c +6
-0@@ -196,6 +196,12 @@#include "i915_drv.h" #include "i915_oa_hsw.h" +#include "i915_oa_bdw.h" +#include "i915_oa_chv.h" +#include "i915_oa_sklgt2.h" +#include "i915_oa_sklgt3.h" +#include "i915_oa_sklgt4.h" +#include "i915_oa_bxt.h" /* HW requires this to be a power of two, between 128k and 16M, though driver
* is currently generally designed assuming the largest 16M size is
used suchdrivers/gpu/drm/i915/i915_perf.c +0
-0@@ -215,7 +221,7 @@* * Although this can be observed explicitly while copying reports to userspace
* by checking for a zeroed report-id field in tail reports, we want to
account
- * for this earlier, as part of the_oa_buffer_checkto avoid lots of redundant
+ * for this earlier, as part of theoa_buffer_checkto avoid lots of redundant
*read()attempts.
*
* In effect we define a tail pointer for reading that lags the real
taildrivers/gpu/drm/i915/i915_perf.c +1
-1@@ -237,7 +243,7 @@* indicates that an updated tail pointer is needed. * * Most of the implementation details for this workaround are in -* gen7_oa_buffer_check_unlocked() and gen7_appand_oa_reports() +* oa_buffer_check_unlocked() and _append_oa_reports() * * Note for posterity: previously the driver used to define an effective tail
* pointer that lagged the real pointer by a 'tail margin' measured in
bytesdrivers/gpu/drm/i915/i915_perf.c +1
-0@@ -272,6 +278,13 @@ static u32 i915_perf_stream_paranoid = true;#define INVALID_CTX_ID 0xffffffff +/* On Gen8+ automatically triggered OA reports include a 'reason' field... */
drivers/gpu/drm/i915/i915_perf.c +13
-0+#define OAREPORT_REASON_MASK 0x3f +#define OAREPORT_REASON_SHIFT 19 +#define OAREPORT_REASON_TIMER (1<<0) +#define OAREPORT_REASON_CTX_SWITCH (1<<3) +#define OAREPORT_REASON_CLK_RATIO (1<<5) +/* For sysctl proc_dointvec_minmax of i915_oa_max_sample_rate * @@ -303,6 +316,13 @@ static struct i915_oa_format hsw_oa_formats[I915_OA_FORMAT_MAX] = { [I915_OA_FORMAT_C4_B8] = { 7, 64 }, }; +static struct i915_oa_format gen8_plus_oa_formats[I915_OA_FORMAT_MAX ] = { + [I915_OA_FORMAT_A12 ] = {0 ,64 }, + [I915_OA_FORMAT_A12_B8_C8 ] = {2 ,128 }, + [I915_OA_FORMAT_A32u40_A4u32_B8_C8] = {5 ,256 }, + [I915_OA_FORMAT_C4_B8 ] = {7 ,64 }, +}; +#define SAMPLE_OA_REPORT (1<<0)/**
drivers/gpu/drm/i915/i915_perf.c +12
-1@@ -332,8 +352,20 @@ struct perf_open_properties {int oa_period_exponent; }; +static u32 gen8_oa_hw_tail_read (struct drm_i915_private *dev_priv) +{ +return I915_READ (GEN8_OATAILPTR ) &GEN8_OATAILPTR_MASK ; +} + +static u32 gen7_oa_hw_tail_read (struct drm_i915_private *dev_priv) +{ +u32 oastatus1 =I915_READ (GEN7_OASTATUS1 ); + +return oastatus1 &GEN7_OASTATUS1_TAIL_MASK ; +} +/** -* gen7_oa_buffer_check_unlocked - check for data and update tail ptr state
+ *oa_buffer_check_unlocked- check for data and update tail ptr state
* @dev_priv: i915 device instance
*
* This is either called via fops (for blocking reads in user ctx) or
the polldrivers/gpu/drm/i915/i915_perf.c +2
-4@@ -356,12 +388,11 @@ struct perf_open_properties {* * Returns: %true if the OA buffer contains data, else %false */ -static bool gen7_oa_buffer_check_unlocked (struct drm_i915_private *dev_priv) +static bool oa_buffer_check_unlocked (struct drm_i915_private *dev_priv) {int report_size = dev_priv->perf.oa.oa_buffer.format_size;unsigned long flags;unsigned int aged_idx; -u32 oastatus1;u32 head, hw_tail, aged_tail, aging_tail;u64 now;@@ -381,8 +412,7 @@ static bool gen7_oa_buffer_check_unlocked(struct drm_i915_private *dev_priv) aged_tail = dev_priv->perf.oa.oa_buffer.tails[aged_idx].offset; aging_tail = dev_priv->perf.oa.oa_buffer.tails[!aged_idx].offset; - oastatus1 =I915_READ (GEN7_OASTATUS1 ); - hw_tail = oastatus1 &GEN7_OASTATUS1_TAIL_MASK ; + hw_tail = dev_priv->perf.oa.ops.oa_hw_tail_read (dev_priv);/* The tail pointer increases in 64 byte increments,
* not inreport_sizesteps...drivers/gpu/drm/i915/i915_perf.c +30
-0@@ -404,6 +434,7 @@ static bool gen7_oa_buffer_check_unlocked(struct drm_i915_private *dev_priv)if (aging_tail !=INVALID_TAIL_PTR && ((now - dev_priv->perf.oa.oa_buffer.aging_timestamp) >OA_TAIL_MARGIN_NSEC )) { + aged_idx ^=1 ; dev_priv->perf.oa.oa_buffer.aged_tail_idx = aged_idx;@@ -553,6 +584,287 @@ static int append_oa_sample(struct i915_perf_stream*stream, * * Returns: 0 on success, negative error code on failure. */ +static int gen8_append_oa_reports (struct i915_perf_stream *stream, +char __user *buf, +size_t count, +size_t *offset) +{ +struct drm_i915_private *dev_priv = stream->dev_priv; +int report_size = dev_priv->perf.oa.oa_buffer.format_size; +u8 *oa_buf_base = dev_priv->perf.oa.oa_buffer.vaddr; +u32 gtt_offset =i915_ggtt_offset (dev_priv->perf.oa.oa_buffer.vma); +u32 mask = (OA_BUFFER_SIZE -1 ); +size_t start_offset = *offset; +unsigned long flags; +unsigned int aged_tail_idx; +u32 head, tail; +u32 taken; +int ret =0 ; + +if (WARN_ON (!stream->enabled)) +return -EIO ; + +spin_lock_irqsave (&dev_priv->perf.oa.oa_buffer.ptr_lock, flags); + + head = dev_priv->perf.oa.oa_buffer.head; + aged_tail_idx = dev_priv->perf.oa.oa_buffer.aged_tail_idx; + tail = dev_priv->perf.oa.oa_buffer.tails[aged_tail_idx].offset; + +spin_unlock_irqrestore (&dev_priv->perf.oa.oa_buffer.ptr_lock, flags); + +/* An invalid tail pointer here means we're still waiting for the poll
drivers/gpu/drm/i915/i915_perf.c +6
-0+* hrtimer callback to give us a pointer +*/ +if (tail ==INVALID_TAIL_PTR ) +return -EAGAIN ; + +/* NB: oa_buffer.head/tail include the gtt_offset which we don't want
drivers/gpu/drm/i915/i915_perf.c +6
-0+* while indexing relative to oa_buf_base. +*/ + head -= gtt_offset; + tail -= gtt_offset; + +/* An out of bounds or misaligned head or tail pointer implies a driver
+ * bug since we validate + align the tail pointers we read from
the
+ * hardware and we are in full control of the head pointer which
should
+ * only be incremented by multiples of the report size (notably
alsodrivers/gpu/drm/i915/i915_perf.c +5
-0+* all a power of two). +*/ + if (WARN_ONCE(head > OA_BUFFER_SIZE || head % report_size || + tail > OA_BUFFER_SIZE || tail % report_size, + "Inconsistent OA buffer pointers: head = %u, tail =%u\n",
drivers/gpu/drm/i915/i915_perf.c +22
-0+ head, tail)) +return -EIO ; + + +for (/* none */ ; + (taken =OA_TAKEN (tail, head)); + head = (head + report_size) & mask) { +u8 *report = oa_buf_base + head; +u32 *report32 = (void *)report; +u32 ctx_id; +u32 reason; + +/* All the report sizes factor neatly into the buffer +* size so we never expect to see a report split +* between the beginning and end of the buffer. +* +* Given the initial alignment check a misalignment +* here would imply a driver bug that would result +* in an overrun. +*/ +if (WARN_ON ((OA_BUFFER_SIZE - head) < report_size)) { + DRM_ERROR("Spurious OA head ptr: non-integralreport offset\n");
drivers/gpu/drm/i915/i915_perf.c +15
-0+break ; + } + +/* The reason field includes flags identifying what +* triggered this specific report (mostly timer +* triggered or e.g. due to a context switch). +* +* This field is never expected to be zero so we can +* check that the report isn't invalid before copying +* it to userspace... +*/ + reason = ((report32[0 ] >>OAREPORT_REASON_SHIFT ) & +OAREPORT_REASON_MASK ); +if (reason ==0 ) { + if(
__ratelimit(&dev_priv->perf.oa.spurious_report_rs))
+ DRM_NOTE("Skipping spurious, invalid OA
report\n");drivers/gpu/drm/i915/i915_perf.c +5
-0+continue ; + } + +/* XXX: Just keep the lower 21 bits for now since I'm not +* entirely sure if the HW touches any of the higher bits in
drivers/gpu/drm/i915/i915_perf.c +5
-0+* this field +*/ + ctx_id = report32[2 ] &0x1fffff ; + +/* Squash whatever is in the CTX_ID field if it's marked as
+ * invalid to be sure we avoid false-positive,
single-contextdrivers/gpu/drm/i915/i915_perf.c +3
-0+* filtering below... +* +* Note: that we don't clear the valid_ctx_bit so userspace can
+ * understand that the ID has been squashed by the
kernel.drivers/gpu/drm/i915/i915_perf.c +2
-0+*/ + if (!(report32[0] &
dev_priv->perf.oa.gen8_valid_ctx_bit))drivers/gpu/drm/i915/i915_perf.c +3
-0+ ctx_id = report32[2 ] =INVALID_CTX_ID ; + +/* NB: For Gen 8 the OA unit no longer supports clock gating
+ * off for a specific context and the kernel can't
securely
+ * stop the counters from updating as system-wide /
globaldrivers/gpu/drm/i915/i915_perf.c +3
-0+* values. +* +* Automatic reports now include a context ID so reports can be
drivers/gpu/drm/i915/i915_perf.c +4
-0+* filtered on the cpu but it's not worth trying to +* automatically subtract/hide counter progress for other +* contexts while filtering since we can't stop userspace +* issuing MI_REPORT_PERF_COUNT commands which would still
drivers/gpu/drm/i915/i915_perf.c +3
-0+* provide a side-band view of the real values. +* +* To allow userspace (such as Mesa/GL_INTEL_performance_query)
+ * to normalize counters for a single filtered context
then it
+ * needs be forwarded bookend context-switch reports so
that it
+ * can track switches in betweenMI_REPORT_PERF_COUNT
commands
+ * and can itself subtract/ignore the progress of
countersdrivers/gpu/drm/i915/i915_perf.c +3
-0+* associated with other contexts. Note that the hardware +* automatically triggers reports when switching to a new +* context which are tagged with the ID of the newly active
+ * context. To avoid the complexity (and likely
fragility) of
+ * reading ahead while parsing reports to try and
minimize
+ * forwarding redundant context switch reports (i.e.
between
+ * other, unrelated contexts) we simply elect to forward
themdrivers/gpu/drm/i915/i915_perf.c +3
-0+* all. +* +* We don't rely solely on the reason field to identify context
+ * switches since it's not-uncommon for periodic samples
todrivers/gpu/drm/i915/i915_perf.c +13
-0+* identify a switch before any 'context switch' report. +*/ +if (!dev_priv->perf.oa.exclusive_stream->ctx || + dev_priv->perf.oa.specific_ctx_id == ctx_id || + (dev_priv->perf.oa.oa_buffer.last_ctx_id == + dev_priv->perf.oa.specific_ctx_id) || + reason &OAREPORT_REASON_CTX_SWITCH ) { + +/* While filtering for a single context we avoid +* leaking the IDs of other contexts. +*/ +if (dev_priv->perf.oa.exclusive_stream->ctx && + dev_priv->perf.oa.specific_ctx_id != ctx_id){
drivers/gpu/drm/i915/i915_perf.c +4
-0+ report32[2 ] =INVALID_CTX_ID ; + } + + ret = append_oa_sample(stream, buf, count,offset,
drivers/gpu/drm/i915/i915_perf.c +18
-0+ report); +if (ret) +break ; + + dev_priv->perf.oa.oa_buffer.last_ctx_id = ctx_id; + } + +/* The above reason field sanity check is based on +* the assumption that the OA buffer is initially +* zeroed and we reset the field after copying so the +* check is still meaningful once old reports start +* being overwritten. +*/ + report32[0 ] =0 ; + } + +if (start_offset != *offset) { + spin_lock_irqsave(&dev_priv->perf.oa.oa_buffer.ptr_lock,flags);
drivers/gpu/drm/i915/i915_perf.c +2
-0+ +/* We removed the gtt_offset for the copy loop above, indexing
drivers/gpu/drm/i915/i915_perf.c +8
-0+* relative to oa_buf_base so put back here... +*/ + head += gtt_offset; + +I915_WRITE (GEN8_OAHEADPTR , head &GEN8_OAHEADPTR_MASK ); + dev_priv->perf.oa.oa_buffer.head = head; + +
spin_unlock_irqrestore(&dev_priv->perf.oa.oa_buffer.ptr_lock, flags);drivers/gpu/drm/i915/i915_perf.c +13
-0+ } + +return ret; +} + +/** +* gen8_oa_read - copy status records then buffered OA reports +* @stream: An i915-perf stream opened for OA metrics +* @buf: destination buffer given by userspace +* @count: the number of bytes userspace wants to read +* @offset: (inout): the current position for writing into @buf +* +* Checks OA unit status registers and if necessary appends corresponding
+ * status records for userspace (such as for a buffer full condition) and then
+ * initiate appending any buffered OA reports.
+ *
+ * Updates @offset according to the number of bytes successfully copied into
+ * the userspace buffer.
+ *
+ * NB: some data may be successfully copied to the userspace buffer
+ * even if an error is returned, and this is reflected in the
+ * updated @offset.
+ *
+ * Returns: zero on success or a negative error code
+ */drivers/gpu/drm/i915/i915_perf.c +30
-0+static int gen8_oa_read (struct i915_perf_stream *stream, +char __user *buf, +size_t count, +size_t *offset) +{ +struct drm_i915_private *dev_priv = stream->dev_priv; +u32 oastatus; +int ret; + +if (WARN_ON (!dev_priv->perf.oa.oa_buffer.vaddr)) +return -EIO ; + + oastatus =I915_READ (GEN8_OASTATUS ); + +/* We treat OABUFFER_OVERFLOW as a significant error: +* +* Although theoretically we could handle this more gracefully +* sometimes, some Gens don't correctly suppress certain +* automatically triggered reports in this condition and so we +* have to assume that old reports are now being trampled +* over. +* +* Considering how we don't currently give userspace control +* over the OA buffer size and always configure a large 16MB +* buffer, then a buffer overflow does anyway likely indicate +* that something has gone quite badly wrong. +*/ +if (oastatus &GEN8_OASTATUS_OABUFFER_OVERFLOW ) { + ret = append_oa_status(stream, buf, count, offset, +
DRM_I915_PERF_RECORD_OA_BUFFER_LOST);drivers/gpu/drm/i915/i915_perf.c +4
-0+if (ret) +return ret; + + DRM_DEBUG("OA buffer overflow (exponent = %d): forcerestart\n",
drivers/gpu/drm/i915/i915_perf.c +14
-0+ dev_priv->perf.oa.period_exponent); + + dev_priv->perf.oa.ops.oa_disable (dev_priv); + dev_priv->perf.oa.ops.oa_enable (dev_priv); + +/* Note: .oa_enable() is expected to re-init the oabuffer +* and reset GEN8_OASTATUS for us +*/ + oastatus =I915_READ (GEN8_OASTATUS ); + } + +if (oastatus &GEN8_OASTATUS_REPORT_LOST ) { + ret = append_oa_status(stream, buf, count, offset, +
DRM_I915_PERF_RECORD_OA_REPORT_LOST);drivers/gpu/drm/i915/i915_perf.c +31
-1+if (ret) +return ret; +I915_WRITE (GEN8_OASTATUS , + oastatus & ~GEN8_OASTATUS_REPORT_LOST ); + } + +return gen8_append_oa_reports (stream, buf, count, offset); +} + +/** +* Copies all buffered OA reports into userspace read() buffer. +* @stream: An i915-perf stream opened for OA metrics +* @buf: destination buffer given by userspace +* @count: the number of bytes userspace wants to read +* @offset: (inout): the current position for writing into @buf +* +* Notably any error condition resulting in a short read (-%ENOSPC or +* -%EFAULT) will be returned even though one or more records may +* have been successfully copied. In this case it's up to the caller +* to decide if the error should be squashed before returning to +* userspace. +* +* Note: reports are consumed from the head, and appended to the +* tail, so the tail chases the head?... If you think that's mad +* and back-to-front you're not alone, but this follows the +* Gen PRM naming convention. +* +* Returns: 0 on success, negative error code on failure. +*/ static int gen7_append_oa_reports(struct i915_perf_stream *stream, char __user *buf, size_t count,@@ -733,7 +1045,8 @@ static int gen7_oa_read(struct i915_perf_stream *stream, if (ret)return ret; -DRM_DEBUG ("OA buffer overflow: force restart\n" ); +DRM_DEBUG ("OA buffer overflow (exponent = %d): force restart\n", + dev_priv->perf.oa.period_exponent);dev_priv->perf.oa.ops.oa_disable(dev_priv); dev_priv->perf.oa.ops.oa_enable(dev_priv);drivers/gpu/drm/i915/i915_perf.c +1
-1@@ -776,7 +1089,7 @@ static int i915_oa_wait_unlocked(struct i915_perf_stream *stream)return -EIO ;return wait_event_interruptible (dev_priv->perf.oa.poll_wq, - dev_priv->perf.oa.ops.oa_buffer_check (dev_priv)); + oa_buffer_check_unlocked(dev_priv));}/**
drivers/gpu/drm/i915/i915_perf.c +9
-19@@ -833,33 +1146,37 @@ static int i915_oa_read(struct i915_perf_stream *stream, static int oa_get_render_ctx_id(struct i915_perf_stream *stream) {struct drm_i915_private *dev_priv = stream->dev_priv; -struct intel_engine_cs *engine = dev_priv->engine[RCS ]; -int ret; - ret =i915_mutex_lock_interruptible (&dev_priv->drm); -if (ret) -return ret; +if (i915.enable_execlists) + dev_priv->perf.oa.specific_ctx_id = stream->ctx->hw_id; +else { +struct intel_engine_cs *engine = dev_priv->engine[RCS ]; +int ret; -/* As the ID is the gtt offset of the context's vma we pin -* the vma to ensure the ID remains fixed. -* -* NB: implied RCS engine... -*/ - ret = engine->context_pin (engine, stream->ctx); -if (ret) - goto unlock; + ret =i915_mutex_lock_interruptible (&dev_priv->drm); +if (ret) +return ret; -/* Explicitly track the ID (instead of calling i915_ggtt_offset() -* on the fly) considering the difference with gen8+ and -* execlists -*/ - dev_priv->perf.oa.specific_ctx_id = -i915_ggtt_offset (stream->ctx->engine[engine->id].state); +/* As the ID is the gtt offset of the context's vma we pin
drivers/gpu/drm/i915/i915_perf.c +9
-2+* the vma to ensure the ID remains fixed. +*/ + ret = engine->context_pin (engine, stream->ctx); +if (ret) { +mutex_unlock (&dev_priv->drm.struct_mutex); +return ret; + } -unlock: -mutex_unlock (&dev_priv->drm.struct_mutex); +/* Explicitly track the ID (instead of calling +* i915_ggtt_offset() on the fly) considering the difference
drivers/gpu/drm/i915/i915_perf.c +4
-0+* with gen8+ and execlists +*/ + dev_priv->perf.oa.specific_ctx_id = +
i915_ggtt_offset(stream->ctx->engine[engine->id].state);drivers/gpu/drm/i915/i915_perf.c +12
-5-return ret; +mutex_unlock (&dev_priv->drm.struct_mutex); + } + +return 0 ; }/** @@ -874,12 +1191,16 @@ static void oa_put_render_ctx_id(struct i915_perf_stream *stream)struct drm_i915_private *dev_priv = stream->dev_priv;struct intel_engine_cs *engine = dev_priv->engine[RCS ]; -mutex_lock (&dev_priv->drm.struct_mutex); +if (i915.enable_execlists) { + dev_priv->perf.oa.specific_ctx_id =INVALID_CTX_ID ; + }else { +mutex_lock (&dev_priv->drm.struct_mutex); - dev_priv->perf.oa.specific_ctx_id =INVALID_CTX_ID ; - engine->context_unpin (engine, stream->ctx); + dev_priv->perf.oa.specific_ctx_id =INVALID_CTX_ID ; + engine->context_unpin (engine, stream->ctx); -mutex_unlock (&dev_priv->drm.struct_mutex); +mutex_unlock (&dev_priv->drm.struct_mutex); + } }static void
drivers/gpu/drm/i915/i915_perf.c +29
-0@@ -969,6 +1290,61 @@ static void gen7_init_oa_buffer(struct drm_i915_private *dev_priv) dev_priv->perf.oa.pollin = false; } +static void gen8_init_oa_buffer (struct drm_i915_private *dev_priv) +{ +u32 gtt_offset =i915_ggtt_offset (dev_priv->perf.oa.oa_buffer.vma); +unsigned long flags; + +spin_lock_irqsave (&dev_priv->perf.oa.oa_buffer.ptr_lock, flags); + +I915_WRITE (GEN8_OASTATUS ,0 ); +I915_WRITE (GEN8_OAHEADPTR , gtt_offset); + dev_priv->perf.oa.oa_buffer.head = gtt_offset; + +I915_WRITE (GEN8_OABUFFER_UDW ,0 ); + +/* PRM says: +* +* "This MMIO must be set before the OATAILPTR +* register and after the OAHEADPTR register. This is +* to enable proper functionality of the overflow +* bit." +*/ +I915_WRITE (GEN8_OABUFFER , gtt_offset | +OABUFFER_SIZE_16M |OA_MEM_SELECT_GGTT ); +I915_WRITE (GEN8_OATAILPTR , gtt_offset &GEN8_OATAILPTR_MASK ); + +/* Mark that we need updated tail pointers to read from... */ + dev_priv->perf.oa.oa_buffer.tails[0 ].offset =INVALID_TAIL_PTR ; + dev_priv->perf.oa.oa_buffer.tails[1 ].offset =INVALID_TAIL_PTR ; + +/* Reset state used to recognise context switches, affecting which
+ * reports we will forward to userspace while filtering for a
singledrivers/gpu/drm/i915/i915_perf.c +5
-0+* context. +*/ + dev_priv->perf.oa.oa_buffer.last_ctx_id =INVALID_CTX_ID ; + + spin_unlock_irqrestore(&dev_priv->perf.oa.oa_buffer.ptr_lock,flags);
drivers/gpu/drm/i915/i915_perf.c +31
-0+ +/* NB: although the OA buffer will initially be allocated +* zeroed via shmfs (and so this memset is redundant when +* first allocating), we may re-init the OA buffer, either +* when re-enabling a stream or in error/reset paths. +* +* The reason we clear the buffer for each re-init is for the +* sanity check in gen8_append_oa_reports() that looks at the +* reason field to make sure it's non-zero which relies on +* the assumption that new reports are being written to zeroed +* memory... +*/ +memset (dev_priv->perf.oa.oa_buffer.vaddr,0 ,OA_BUFFER_SIZE ); + +/* Maybe make ->pollin per-stream state if we support multiple +* concurrent streams in the future. +*/ + dev_priv->perf.oa.pollin = false; +} +static int alloc_oa_buffer (struct drm_i915_private *dev_priv) {struct drm_i915_gem_object *bo;@@ -1113,6 +1489,280 @@ static void hsw_disable_metric_set(struct drm_i915_private *dev_priv) ~GT_NOA_ENABLE)); } +/* +* From Broadwell PRM, 3D-Media-GPGPU -> Register State Context +* +* MMIO reads or writes to any of the registers listed in the +* “Register State Context image” subsections through HOST/IA +* MMIO interface for debug purposes must follow the steps below: +* +* - SW should set the Force Wakeup bit to prevent GT from entering C6. +* - Write 0x2050[31:0] = 0x00010001 (disable sequence). +* - Disable IDLE messaging in CS (Write 0x2050[31:0] = 0x00010001). +* - BDW: Poll/Wait for register bits of 0x22AC[6:0] turn to 0x30 value.
+ * - SKL+: Poll/Wait for register bits of 0x22A4[6:0] turn to0x30value.
+ * - Read/Write to desired MMIO registers.
+ * - Enable IDLE messaging in CS (Write 0x2050[31:0] =0x00010000).
+ * - Force Wakeup bit should be reset to enable C6 entry.
+ *
+ * XXX: don't nest or overlap calls to this API, it has no ref
+ * counting to track how many entities require the RCS to be
+ * blocked from being idle.
+ */drivers/gpu/drm/i915/i915_perf.c +25
-0+static int gen8_begin_ctx_mmio (struct drm_i915_private *dev_priv) +{ +i915_reg_t fsm_reg =INTEL_GEN (dev_priv) >8 ? +GEN9_RCS_FE_FSM2 :GEN6_RCS_PWR_FSM ; +int ret =0 ; + +/* There's only no active context while idle in execlist mode +* (though we shouldn't be using this in any other case) +*/ +if (WARN_ON (!i915.enable_execlists)) +return -EINVAL ; + +intel_uncore_forcewake_get (dev_priv,FORCEWAKE_RENDER ); + +I915_WRITE (GEN6_RC_SLEEP_PSMI_CONTROL , +_MASKED_BIT_ENABLE (GEN6_PSMI_SLEEP_MSG_DISABLE )); + +/* Note: we don't currently have a good handle on the maximum +* latency for this wake up so while we only need to hold rcs +* busy from process context we at least keep the waiting +* interruptible... +*/ + ret =intel_wait_for_register (dev_priv, fsm_reg,0x3f ,0x30 ,50 ); +if (ret == -ETIMEDOUT ) + DRM_ERROR("Timeout waiting to be able to begin per-ctxmmio\n");
drivers/gpu/drm/i915/i915_perf.c +18
-0+ +if (ret) +intel_uncore_forcewake_put (dev_priv,FORCEWAKE_RENDER ); + +return ret; +} + +static void gen8_end_ctx_mmio (struct drm_i915_private *dev_priv) +{ +if (WARN_ON (!i915.enable_execlists)) +return ; + +I915_WRITE (GEN6_RC_SLEEP_PSMI_CONTROL , +_MASKED_BIT_DISABLE (GEN6_PSMI_SLEEP_MSG_DISABLE )); +intel_uncore_forcewake_put (dev_priv,FORCEWAKE_RENDER ); +} + +/* NB: It must always remain pointer safe to run this even if the OA unit
+ * has been disabled.
+ *
+ * It's fine to put out-of-date values into these per-context registers
+ * in the case that the OA unit has been disabled.
+ */drivers/gpu/drm/i915/i915_perf.c +5
-0+static void gen8_update_reg_state_unlocked (struct i915_gem_context *ctx, +u32 *reg_state) +{ +struct drm_i915_private *dev_priv = ctx->i915; + const struct i915_oa_reg *flex_regs =
dev_priv->perf.oa.flex_regs;drivers/gpu/drm/i915/i915_perf.c +16
-0+int n_flex_regs = dev_priv->perf.oa.flex_regs_len; +u32 ctx_oactxctrl = dev_priv->perf.oa.ctx_oactxctrl_offset; +u32 ctx_flexeu0 = dev_priv->perf.oa.ctx_flexeu0_offset; +/* The MMIO offsets for Flex EU registers aren't contiguous */ +u32 flex_mmio[] = { +i915_mmio_reg_offset (EU_PERF_CNTL0 ), +i915_mmio_reg_offset (EU_PERF_CNTL1 ), +i915_mmio_reg_offset (EU_PERF_CNTL2 ), +i915_mmio_reg_offset (EU_PERF_CNTL3 ), +i915_mmio_reg_offset (EU_PERF_CNTL4 ), +i915_mmio_reg_offset (EU_PERF_CNTL5 ), +i915_mmio_reg_offset (EU_PERF_CNTL6 ), + }; +int i; + + reg_state[ctx_oactxctrl] =
i915_mmio_reg_offset(GEN8_OACTXCONTROL);
+ reg_state[ctx_oactxctrl+1] = (dev_priv->perf.oa.period_exponent
<<drivers/gpu/drm/i915/i915_perf.c +10
-0+ GEN8_OA_TIMER_PERIOD_SHIFT) | + (dev_priv->perf.oa.periodic ? +GEN8_OA_TIMER_ENABLE :0 ) | +GEN8_OA_COUNTER_RESUME ; + +for (i =0 ; i <ARRAY_SIZE (flex_mmio); i++) { +u32 state_offset = ctx_flexeu0 + i *2 ; +u32 mmio = flex_mmio[i]; + +/* This arbitrary default will select the 'EU FPU0 Pipeline
+ * Active' event. In the future it's anticipated that
there
+ * will be an explicit 'No Event' we can select, but not
yet...drivers/gpu/drm/i915/i915_perf.c +6
-0+*/ +u32 value =0 ; +int j; + +for (j =0 ; j < n_flex_regs; j++) { + if (i915_mmio_reg_offset(flex_regs[j].addr) ==mmio) {
drivers/gpu/drm/i915/i915_perf.c +34
-0+ value = flex_regs[j].value; +break ; + } + } + + reg_state[state_offset] = mmio; + reg_state[state_offset+1 ] = value; + } +} + +/* Manages updating the per-context aspects of the OA stream +* configuration across all contexts. +* +* The awkward consideration here is that OACTXCONTROL controls the +* exponent for periodic sampling which is primarily used for system +* wide profiling where we'd like a consistent sampling period even in +* the face of context switches. +* +* Our approach of updating the register state context (as opposed to +* say using a workaround batch buffer) ensures that the hardware +* won't automatically reload an out-of-date timer exponent even +* transiently before a WA BB could be parsed. +* +* This function needs to: +* - Ensure the currently running context's per-context OA state is +* updated +* - Ensure that all existing contexts will have the correct per-context +* OA state if they are scheduled for use. +* - Ensure any new contexts will be initialized with the correct +* per-context OA state. +* +* Note: it's only the RCS/Render context that has any OA state. +*/ +static int gen8_configure_all_contexts(struct drm_i915_private
*dev_priv)drivers/gpu/drm/i915/i915_perf.c +5
-0+{ +struct i915_gem_context *ctx; +int ret; + + ret =i915_mutex_lock_interruptible (&dev_priv->drm);Quoted 8 lines
drivers/gpu/drm/i915/i915_perf.c static void hsw_disable_metric_set(struct drm_i915_private *dev_priv) · line 1644
Quoted 6 lines
drivers/gpu/drm/i915/i915_perf.c +6
-0+if (ret) +return ret; + +/* Switch away from any user context. */ + ret =i915_gem_switch_to_kernel_context (dev_priv); +if (ret)mutex_unlock(&dev_priv->drm.struct_mutex);Duh! Thanks!
Quoted 46 lines
Quoted 46 lines
drivers/gpu/drm/i915/i915_perf.c +3
-0+return ret; + +/* The OA register config is setup through the context image. This image
+ * might be written to by the GPU on context switch (in
particular on
+ * lite-restore). This means we can't safely update a context's
image,drivers/gpu/drm/i915/i915_perf.c +3
-0+* if this context is scheduled/submitted to run on the GPU. +* +* We could emit the OA register config through the batch buffer but
drivers/gpu/drm/i915/i915_perf.c +4
-0+* this might leave small interval of time where the OA unit is +* configured at an invalid sampling period. +* +* So far the best way to work around this issue seems to be draining
drivers/gpu/drm/i915/i915_perf.c +17
-0+* the GPU from any submitted work. +*/ + ret =i915_gem_wait_for_idle (dev_priv, +I915_WAIT_INTERRUPTIBLE | +I915_WAIT_LOCKED ); +if (ret) { +mutex_unlock (&dev_priv->drm.struct_mutex); +return ret; + } + +/* Update all contexts now that we've stalled the submission. */ +list_for_each_entry (ctx, &dev_priv->context_list, link) { +if (!ctx->engine[RCS ].initialised) +continue ; + + gen8_update_reg_state_unlocked(ctx, +ctx->engine[RCS].lrc_reg_state);
drivers/gpu/drm/i915/i915_perf.c +10
-0+ } + +mutex_unlock (&dev_priv->drm.struct_mutex); + +/* Now update the current context. +* +* Note: Using MMIO to update per-context registers requires +* some extra care... +*/ + ret =gen8_begin_ctx_mmio (dev_priv);Quoted 8 lines
drivers/gpu/drm/i915/i915_perf.c
Quoted 5 lines
drivers/gpu/drm/i915/i915_perf.c +2
-0+if (ret) { + DRM_ERROR("Failed to bring RCS out of idle to updatecurrent ctx OA state\n");
drivers/gpu/drm/i915/i915_perf.c +2
-0+return ret; + }So do we still need the mmio for the current context, given that we now idle everything and then update the reg state?
Quoted 3 lines
I would say yes, because we don't have any guarantee that another context is going to be scheduled after that.
Yet we want the config to be applied before we return.
Okay, feel free to keep my r-b then.
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v7 12/15 drm/i915/perf: Add OA unit support for Gen 8+
In reply to Matthew AuldFrom: Robert Bragg <[email protected]>
Enables access to OA unit metrics for BDW, CHV, SKL and BXT which all share (more-or-less) the same OA unit design.
Of particular note in comparison to Haswell: some OA unit HW config state has become per-context state and as a consequence it is somewhat more complicated to manage synchronous state changes from the cpu while there's no guarantee of what context (if any) is currently actively running on the gpu.
The periodic sampling frequency which can be particularly useful for system-wide analysis (as opposed to command stream synchronised MI_REPORT_PERF_COUNT commands) is perhaps the most surprising state to have become per-context save and restored (while the OABUFFER destination is still a shared, system-wide resource).
This support for gen8+ takes care to consider a number of timing challenges involved in synchronously updating per-context state primarily by programming all config state from the cpu and updating all current and saved contexts synchronously while the OA unit is still disabled.
The driver intentionally avoids depending on command streamer programming to update OA state considering the lack of synchronization between the automatic loading of OACTXCONTROL state (that includes the periodic sampling state and enable state) on context restore and the parsing of any general purpose BB the driver can control. I.e. this implementation is careful to avoid the possibility of a context restore temporarily enabling any out-of-date periodic sampling state. In addition to the risk of transiently-out-of-date state being loaded automatically; there are also internal HW latencies involved in the loading of MUX configurations which would be difficult to account for from the command streamer (and we only want to enable the unit when once the MUX configuration is complete).
Since the Gen8+ OA unit design no longer supports clock gating the unit off for a single given context (which effectively stopped any progress of counters while any other context was running) and instead supports tagging OA reports with a context ID for filtering on the CPU, it means we can no longer hide the system-wide progress of counters from a non-privileged application only interested in metrics for its own context. Although we could theoretically try and subtract the progress of other contexts before forwarding reports via read() we aren't in a position to filter reports captured via MI_REPORT_PERF_COUNT commands. As a result, for Gen8+, we always require the dev.i915.perf_stream_paranoid to be unset for any access to OA metrics if not root.
v5: Drain submitted requests when enabling metric set to ensure no
lite-restore erases the context image we just updated (Lionel)
v6: In addition to drain, switch to kernel context & update all
context in place (Chris)
v7: Add missing mutex_unlock() if switching to kernel context fails
(Matthew)
--- drivers/gpu/drm/i915/i915_drv.h | 45 +- drivers/gpu/drm/i915/i915_perf.c | 959 ++++++++++++++++++++++++++++++++++++--- drivers/gpu/drm/i915/i915_reg.h | 22 + drivers/gpu/drm/i915/intel_lrc.c | 2 + include/uapi/drm/i915_drm.h | 19 +- 5 files changed, 954 insertions(+), 93 deletions(-)
drivers/gpu/drm/i915/i915_drv.h +30 -15
@@ -2064,9 +2064,17 @@ struct i915_oa_ops {void (*init_oa_buffer)(struct drm_i915_private *dev_priv);/** -* @enable_metric_set: Applies any MUX configuration to set up the -* Boolean and Custom (B/C) counters that are part of the counter -* reports being sampled. May apply system constraints such as +* @select_metric_set: The auto generated code that checks whether a +* requested OA config is applicable to the system and if so sets up +* the mux, oa and flex eu register config pointers according to the +* current dev_priv->perf.oa.metrics_set. +*/ +int (*select_metric_set)(struct drm_i915_private *dev_priv); + +/** +* @enable_metric_set: Selects and applies any MUX configuration to set +* up the Boolean and Custom (B/C) counters that are part of the +* counter reports being sampled. May apply system constraints such as * disabling EU clock gating as required. */ int (*enable_metric_set)(struct drm_i915_private *dev_priv);
@@ -2097,20 +2105,13 @@ struct i915_oa_ops { size_t *offset);/** -* @oa_buffer_check: Check for OA buffer data + update tail -* -* This is either called via fops or the poll check hrtimer (atomic -* ctx) without any locks taken. +* @oa_hw_tail_read: read the OA tail pointer register * -* It's safe to read OA config state here unlocked, assuming that this -* is only called while the stream is enabled, while the global OA -* configuration can't be modified. -* -* Efficiency is more important than avoiding some false positives -* here, which will be handled gracefully - likely resulting in an -* %EAGAIN error for userspace. +* In particular this enables us to share all the fiddly code for +* handling the OA unit tail pointer race that affects multiple +* generations. */ -bool (*oa_buffer_check)(struct drm_i915_private *dev_priv); +u32 (*oa_hw_tail_read)(struct drm_i915_private *dev_priv); };struct intel_cdclk_state {
@@ -2472,6 +2473,7 @@ struct drm_i915_private {struct {struct i915_vma *vma;u8 *vaddr; +u32 last_ctx_id;int format;int format_size;
@@ -2541,6 +2543,14 @@ struct drm_i915_private { } oa_buffer;u32 gen7_latched_oastatus1; +u32 ctx_oactxctrl_offset; +u32 ctx_flexeu0_offset; + +/* The RPT_ID/reason field for Gen8+ includes a bit +* to determine if the CTX ID in the report is valid +* but the specific bit differs between Gen 8 and 9 +*/ +u32 gen8_valid_ctx_bit;struct i915_oa_ops ops;const struct i915_oa_format *oa_formats;
@@ -2851,6 +2861,8 @@ intel_info(const struct drm_i915_private *dev_priv)#define IS_KBL_ULX (dev_priv) (INTEL_DEVID(dev_priv) == 0x590E || \ INTEL_DEVID(dev_priv) == 0x5915 || \ INTEL_DEVID(dev_priv) == 0x591E) +#define IS_SKL_GT2 (dev_priv) (IS_SKYLAKE(dev_priv) && \ + (INTEL_DEVID(dev_priv) & 0x00F0) == 0x0010)#define IS_SKL_GT3 (dev_priv) (IS_SKYLAKE(dev_priv) && \ (INTEL_DEVID(dev_priv) & 0x00F0) == 0x0020)#define IS_SKL_GT4 (dev_priv) (IS_SKYLAKE(dev_priv) && \
@@ -3615,6 +3627,9 @@ i915_gem_context_lookup_timeline(struct i915_gem_context *ctx,int i915_perf_open_ioctl (struct drm_device *dev,void *data,struct drm_file *file); +void i915_oa_init_reg_state (struct intel_engine_cs *engine, +struct i915_gem_context *ctx, +uint32_t *reg_state);/* i915_gem_evict.c */ int __must_check i915_gem_evict_something(struct i915_address_space *vm,
drivers/gpu/drm/i915/i915_perf.c +888 -71
1,148 lines not on this page · raw message
drivers/gpu/drm/i915/i915_reg.h +22 -0
@@ -653,6 +653,12 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)#define GEN8_OACTXID _MMIO(0x2364) +#define GEN8_OA_DEBUG _MMIO(0x2B04) +#define GEN9_OA_DEBUG_DISABLE_CLK_RATIO_REPORTS (1<<5) +#define GEN9_OA_DEBUG_INCLUDE_CLK_RATIO (1<<6) +#define GEN9_OA_DEBUG_DISABLE_GO_1_0_REPORTS (1<<2) +#define GEN9_OA_DEBUG_DISABLE_CTX_SWITCH_REPORTS (1<<1) +#define GEN8_OACONTROL _MMIO(0x2B00)#define GEN8_OA_REPORT_FORMAT_A12 (0<<2)#define GEN8_OA_REPORT_FORMAT_A12_B8_C8 (2<<2)
@@ -674,6 +680,7 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)#define GEN7_OABUFFER_STOP_RESUME_ENABLE (1<<1)#define GEN7_OABUFFER_RESUME (1<<0) +#define GEN8_OABUFFER_UDW _MMIO(0x23b4)#define GEN8_OABUFFER _MMIO(0x2b14)#define GEN7_OASTATUS1 _MMIO(0x2364)
@@ -692,7 +699,9 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)#define GEN8_OASTATUS_REPORT_LOST (1<<0)#define GEN8_OAHEADPTR _MMIO(0x2B0C) +#define GEN8_OAHEADPTR_MASK 0xffffffc0#define GEN8_OATAILPTR _MMIO(0x2B10) +#define GEN8_OATAILPTR_MASK 0xffffffc0#define OABUFFER_SIZE_128K (0<<3)#define OABUFFER_SIZE_256K (1<<3)
@@ -705,7 +714,17 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)#define OA_MEM_SELECT_GGTT (1<<0) +/* +* Flexible, Aggregate EU Counter Registers. +* Note: these aren't contiguous +*/ #define EU_PERF_CNTL0 _MMIO(0xe458) +#define EU_PERF_CNTL1 _MMIO(0xe558) +#define EU_PERF_CNTL2 _MMIO(0xe658) +#define EU_PERF_CNTL3 _MMIO(0xe758) +#define EU_PERF_CNTL4 _MMIO(0xe45c) +#define EU_PERF_CNTL5 _MMIO(0xe55c) +#define EU_PERF_CNTL6 _MMIO(0xe65c)#define GDT_CHICKEN_BITS _MMIO(0x9840)#define GT_NOA_ENABLE 0x00000080
@@ -2325,6 +2344,9 @@ enum skl_disp_power_wells {#define GEN8_RC_SEMA_IDLE_MSG_DISABLE (1 << 12)#define GEN8_FF_DOP_CLOCK_GATE_DISABLE (1<<10) +#define GEN6_RCS_PWR_FSM _MMIO(0x22ac) +#define GEN9_RCS_FE_FSM2 _MMIO(0x22a4) +/* Fuse readout registers for GT */ #define CHV_FUSE_GT _MMIO(VLV_DISPLAY_BASE + 0x2168)#define CHV_FGT_DISABLE_SS0 (1 << 10)
drivers/gpu/drm/i915/intel_lrc.c +2 -0
@@ -1879,6 +1879,8 @@ static void execlists_init_reg_state(u32 *regs, regs[CTX_LRI_HEADER_2 ] =MI_LOAD_REGISTER_IMM (1 );CTX_REG (regs,CTX_R_PWR_CLK_STATE ,GEN8_R_PWR_CLK_STATE ,make_rpcs (dev_priv)); + +i915_oa_init_reg_state (engine, ctx, regs); } }
include/uapi/drm/i915_drm.h +12 -7
@@ -1316,13 +1316,18 @@ struct drm_i915_gem_context_param { };enum drm_i915_oa_format { -I915_OA_FORMAT_A13 =1 , -I915_OA_FORMAT_A29 , -I915_OA_FORMAT_A13_B8_C8 , -I915_OA_FORMAT_B4_C8 , -I915_OA_FORMAT_A45_B8_C8 , -I915_OA_FORMAT_B4_C8_A16 , -I915_OA_FORMAT_C4_B8 , +I915_OA_FORMAT_A13 =1 ,/* HSW only */ +I915_OA_FORMAT_A29 ,/* HSW only */ +I915_OA_FORMAT_A13_B8_C8 ,/* HSW only */ +I915_OA_FORMAT_B4_C8 ,/* HSW only */ +I915_OA_FORMAT_A45_B8_C8 ,/* HSW only */ +I915_OA_FORMAT_B4_C8_A16 ,/* HSW only */ +I915_OA_FORMAT_C4_B8 ,/* HSW+ */ + +/* Gen8+ */ +I915_OA_FORMAT_A12 , +I915_OA_FORMAT_A12_B8_C8 , + I915_OA_FORMAT_A32u40_A4u32_B8_C8,I915_OA_FORMAT_MAX /* non-ABI */ };
--
2.11.0
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [PATCH v7 12/15] drm/i915/perf: Add OA unit support for Gen 8+
Quoted 31 lines · Lionel Landwerlin
drivers/gpu/drm/i915/i915_perf.c +31
-0+static int gen8_configure_all_contexts (struct drm_i915_private *dev_priv) +{ +struct i915_gem_context *ctx; +int ret; + + ret =i915_mutex_lock_interruptible (&dev_priv->drm); +if (ret) +return ret; + +/* Switch away from any user context. */ + ret =i915_gem_switch_to_kernel_context (dev_priv); +if (ret) { +mutex_unlock (&dev_priv->drm.struct_mutex); +return ret; + } + +/* The OA register config is setup through the context image. This image +* might be written to by the GPU on context switch (in particular on +* lite-restore). This means we can't safely update a context's image, +* if this context is scheduled/submitted to run on the GPU. +* +* We could emit the OA register config through the batch buffer but +* this might leave small interval of time where the OA unit is +* configured at an invalid sampling period. +* +* So far the best way to work around this issue seems to be draining +* the GPU from any submitted work. +*/ + ret =i915_gem_wait_for_idle (dev_priv, +I915_WAIT_INTERRUPTIBLE | +I915_WAIT_LOCKED );
drivers/gpu/drm/i915/i915_perf.c static void hsw_disable_metric_set(struct drm_i915_private *dev_priv) · line 1673
Quoted 10 lines
drivers/gpu/drm/i915/i915_perf.c +10
-0+if (ret) { +mutex_unlock (&dev_priv->drm.struct_mutex); +return ret; + } + +/* Update all contexts now that we've stalled the submission. */ +list_for_each_entry (ctx, &dev_priv->context_list, link) { +if (!ctx->engine[RCS ].initialised) +continue ; +
You need to pin the context here, otherwise there is not guarrantee that the lrc_reg_state exists, or map it directly.
drivers/gpu/drm/i915/i915_perf.c static void hsw_disable_metric_set(struct drm_i915_private *dev_priv) · line 1680
Quoted 7 lines
drivers/gpu/drm/i915/i915_perf.c +7
-0+gen8_update_reg_state_unlocked (ctx, + ctx->engine[RCS ].lrc_reg_state); + } + +mutex_unlock (&dev_priv->drm.struct_mutex); + +/* Now update the current context.
You don't need to. The current context is the kernel context, it is scratch and never used by userspace so no oa-reports.
Quoted 10 lines
drivers/gpu/drm/i915/i915_perf.c +10
-0+* +* Note: Using MMIO to update per-context registers requires +* some extra care... +*/ + ret =gen8_begin_ctx_mmio (dev_priv); +if (ret) { +DRM_ERROR ("Failed to bring RCS out of idle to update current ctx OA state\n" ); +return ret; + } +
drivers/gpu/drm/i915/i915_perf.c static void hsw_disable_metric_set(struct drm_i915_private *dev_priv) · line 1700
Quoted 10 lines
drivers/gpu/drm/i915/i915_perf.c +10
-0+I915_WRITE (GEN8_OACTXCONTROL , ((dev_priv->perf.oa.period_exponent << +GEN8_OA_TIMER_PERIOD_SHIFT ) | + (dev_priv->perf.oa.periodic ? +GEN8_OA_TIMER_ENABLE :0 ) | +GEN8_OA_COUNTER_RESUME )); + +config_oa_regs (dev_priv, dev_priv->perf.oa.flex_regs, + dev_priv->perf.oa.flex_regs_len); + +gen8_end_ctx_mmio (dev_priv);
This entire chunk can go and I don't need to critique gen8_begin_ctx_mmio() -- it needs a bit of tlc.
This patch is not ready.
-Chris
Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v9 12/15 drm/i915/perf: Add OA unit support for Gen 8+
From: Robert Bragg <[email protected]>
Enables access to OA unit metrics for BDW, CHV, SKL and BXT which all share (more-or-less) the same OA unit design.
Of particular note in comparison to Haswell: some OA unit HW config state has become per-context state and as a consequence it is somewhat more complicated to manage synchronous state changes from the cpu while there's no guarantee of what context (if any) is currently actively running on the gpu.
The periodic sampling frequency which can be particularly useful for system-wide analysis (as opposed to command stream synchronised MI_REPORT_PERF_COUNT commands) is perhaps the most surprising state to have become per-context save and restored (while the OABUFFER destination is still a shared, system-wide resource).
This support for gen8+ takes care to consider a number of timing challenges involved in synchronously updating per-context state primarily by programming all config state from the cpu and updating all current and saved contexts synchronously while the OA unit is still disabled.
The driver intentionally avoids depending on command streamer programming to update OA state considering the lack of synchronization between the automatic loading of OACTXCONTROL state (that includes the periodic sampling state and enable state) on context restore and the parsing of any general purpose BB the driver can control. I.e. this implementation is careful to avoid the possibility of a context restore temporarily enabling any out-of-date periodic sampling state. In addition to the risk of transiently-out-of-date state being loaded automatically; there are also internal HW latencies involved in the loading of MUX configurations which would be difficult to account for from the command streamer (and we only want to enable the unit when once the MUX configuration is complete).
Since the Gen8+ OA unit design no longer supports clock gating the unit off for a single given context (which effectively stopped any progress of counters while any other context was running) and instead supports tagging OA reports with a context ID for filtering on the CPU, it means we can no longer hide the system-wide progress of counters from a non-privileged application only interested in metrics for its own context. Although we could theoretically try and subtract the progress of other contexts before forwarding reports via read() we aren't in a position to filter reports captured via MI_REPORT_PERF_COUNT commands. As a result, for Gen8+, we always require the dev.i915.perf_stream_paranoid to be unset for any access to OA metrics if not root.
v5: Drain submitted requests when enabling metric set to ensure no
lite-restore erases the context image we just updated (Lionel)
v6: In addition to drain, switch to kernel context & update all
context in place (Chris)
v7: Add missing mutex_unlock() if switching to kernel context fails
(Matthew)
v8: Simplify OA period/flex-eu-counters programming by using the
batchbuffer instead of modifying ctx-image (Lionel)
v9: Back to updating the context image (due to erroneous testing,
batchbuffer programming the OA unit doesn't actually work) (Lionel)
Pin context before updating context image (Chris)
Drop MMIO programming now that we switch to a kernel context with right values in initial context image (Chris)
--- drivers/gpu/drm/i915/i915_drv.h | 45 +- drivers/gpu/drm/i915/i915_perf.c | 879 +++++++++++++++++++++++++++++++++++---- drivers/gpu/drm/i915/i915_reg.h | 22 + drivers/gpu/drm/i915/intel_lrc.c | 2 + include/uapi/drm/i915_drm.h | 19 +- 5 files changed, 874 insertions(+), 93 deletions(-)
drivers/gpu/drm/i915/i915_drv.h +30 -15
@@ -2063,9 +2063,17 @@ struct i915_oa_ops {void (*init_oa_buffer)(struct drm_i915_private *dev_priv);/** -* @enable_metric_set: Applies any MUX configuration to set up the -* Boolean and Custom (B/C) counters that are part of the counter -* reports being sampled. May apply system constraints such as +* @select_metric_set: The auto generated code that checks whether a +* requested OA config is applicable to the system and if so sets up +* the mux, oa and flex eu register config pointers according to the +* current dev_priv->perf.oa.metrics_set. +*/ +int (*select_metric_set)(struct drm_i915_private *dev_priv); + +/** +* @enable_metric_set: Selects and applies any MUX configuration to set +* up the Boolean and Custom (B/C) counters that are part of the +* counter reports being sampled. May apply system constraints such as * disabling EU clock gating as required. */ int (*enable_metric_set)(struct drm_i915_private *dev_priv);
@@ -2096,20 +2104,13 @@ struct i915_oa_ops { size_t *offset);/** -* @oa_buffer_check: Check for OA buffer data + update tail -* -* This is either called via fops or the poll check hrtimer (atomic -* ctx) without any locks taken. +* @oa_hw_tail_read: read the OA tail pointer register * -* It's safe to read OA config state here unlocked, assuming that this -* is only called while the stream is enabled, while the global OA -* configuration can't be modified. -* -* Efficiency is more important than avoiding some false positives -* here, which will be handled gracefully - likely resulting in an -* %EAGAIN error for userspace. +* In particular this enables us to share all the fiddly code for +* handling the OA unit tail pointer race that affects multiple +* generations. */ -bool (*oa_buffer_check)(struct drm_i915_private *dev_priv); +u32 (*oa_hw_tail_read)(struct drm_i915_private *dev_priv); };struct intel_cdclk_state {
@@ -2470,6 +2471,7 @@ struct drm_i915_private {struct {struct i915_vma *vma;u8 *vaddr; +u32 last_ctx_id;int format;int format_size;
@@ -2539,6 +2541,14 @@ struct drm_i915_private { } oa_buffer;u32 gen7_latched_oastatus1; +u32 ctx_oactxctrl_offset; +u32 ctx_flexeu0_offset; + +/* The RPT_ID/reason field for Gen8+ includes a bit +* to determine if the CTX ID in the report is valid +* but the specific bit differs between Gen 8 and 9 +*/ +u32 gen8_valid_ctx_bit;struct i915_oa_ops ops;const struct i915_oa_format *oa_formats;
@@ -2849,6 +2859,8 @@ intel_info(const struct drm_i915_private *dev_priv)#define IS_KBL_ULX (dev_priv) (INTEL_DEVID(dev_priv) == 0x590E || \ INTEL_DEVID(dev_priv) == 0x5915 || \ INTEL_DEVID(dev_priv) == 0x591E) +#define IS_SKL_GT2 (dev_priv) (IS_SKYLAKE(dev_priv) && \ + (INTEL_DEVID(dev_priv) & 0x00F0) == 0x0010)#define IS_SKL_GT3 (dev_priv) (IS_SKYLAKE(dev_priv) && \ (INTEL_DEVID(dev_priv) & 0x00F0) == 0x0020)#define IS_SKL_GT4 (dev_priv) (IS_SKYLAKE(dev_priv) && \
@@ -3612,6 +3624,9 @@ i915_gem_context_lookup_timeline(struct i915_gem_context *ctx,int i915_perf_open_ioctl (struct drm_device *dev,void *data,struct drm_file *file); +void i915_oa_init_reg_state (struct intel_engine_cs *engine, +struct i915_gem_context *ctx, +uint32_t *reg_state);/* i915_gem_evict.c */ int __must_check i915_gem_evict_something(struct i915_address_space *vm,
drivers/gpu/drm/i915/i915_perf.c +808 -71
1,068 lines not on this page · raw message
drivers/gpu/drm/i915/i915_reg.h +22 -0
@@ -653,6 +653,12 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)#define GEN8_OACTXID _MMIO(0x2364) +#define GEN8_OA_DEBUG _MMIO(0x2B04) +#define GEN9_OA_DEBUG_DISABLE_CLK_RATIO_REPORTS (1<<5) +#define GEN9_OA_DEBUG_INCLUDE_CLK_RATIO (1<<6) +#define GEN9_OA_DEBUG_DISABLE_GO_1_0_REPORTS (1<<2) +#define GEN9_OA_DEBUG_DISABLE_CTX_SWITCH_REPORTS (1<<1) +#define GEN8_OACONTROL _MMIO(0x2B00)#define GEN8_OA_REPORT_FORMAT_A12 (0<<2)#define GEN8_OA_REPORT_FORMAT_A12_B8_C8 (2<<2)
@@ -674,6 +680,7 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)#define GEN7_OABUFFER_STOP_RESUME_ENABLE (1<<1)#define GEN7_OABUFFER_RESUME (1<<0) +#define GEN8_OABUFFER_UDW _MMIO(0x23b4)#define GEN8_OABUFFER _MMIO(0x2b14)#define GEN7_OASTATUS1 _MMIO(0x2364)
@@ -692,7 +699,9 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)#define GEN8_OASTATUS_REPORT_LOST (1<<0)#define GEN8_OAHEADPTR _MMIO(0x2B0C) +#define GEN8_OAHEADPTR_MASK 0xffffffc0#define GEN8_OATAILPTR _MMIO(0x2B10) +#define GEN8_OATAILPTR_MASK 0xffffffc0#define OABUFFER_SIZE_128K (0<<3)#define OABUFFER_SIZE_256K (1<<3)
@@ -705,7 +714,17 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)#define OA_MEM_SELECT_GGTT (1<<0) +/* +* Flexible, Aggregate EU Counter Registers. +* Note: these aren't contiguous +*/ #define EU_PERF_CNTL0 _MMIO(0xe458) +#define EU_PERF_CNTL1 _MMIO(0xe558) +#define EU_PERF_CNTL2 _MMIO(0xe658) +#define EU_PERF_CNTL3 _MMIO(0xe758) +#define EU_PERF_CNTL4 _MMIO(0xe45c) +#define EU_PERF_CNTL5 _MMIO(0xe55c) +#define EU_PERF_CNTL6 _MMIO(0xe65c)#define GDT_CHICKEN_BITS _MMIO(0x9840)#define GT_NOA_ENABLE 0x00000080
@@ -2325,6 +2344,9 @@ enum skl_disp_power_wells {#define GEN8_RC_SEMA_IDLE_MSG_DISABLE (1 << 12)#define GEN8_FF_DOP_CLOCK_GATE_DISABLE (1<<10) +#define GEN6_RCS_PWR_FSM _MMIO(0x22ac) +#define GEN9_RCS_FE_FSM2 _MMIO(0x22a4) +/* Fuse readout registers for GT */ #define CHV_FUSE_GT _MMIO(VLV_DISPLAY_BASE + 0x2168)#define CHV_FGT_DISABLE_SS0 (1 << 10)
drivers/gpu/drm/i915/intel_lrc.c +2 -0
@@ -1877,6 +1877,8 @@ static void execlists_init_reg_state(u32 *regs, regs[CTX_LRI_HEADER_2 ] =MI_LOAD_REGISTER_IMM (1 );CTX_REG (regs,CTX_R_PWR_CLK_STATE ,GEN8_R_PWR_CLK_STATE ,make_rpcs (dev_priv)); + +i915_oa_init_reg_state (engine, ctx, regs); } }
include/uapi/drm/i915_drm.h +12 -7
@@ -1316,13 +1316,18 @@ struct drm_i915_gem_context_param { };enum drm_i915_oa_format { -I915_OA_FORMAT_A13 =1 , -I915_OA_FORMAT_A29 , -I915_OA_FORMAT_A13_B8_C8 , -I915_OA_FORMAT_B4_C8 , -I915_OA_FORMAT_A45_B8_C8 , -I915_OA_FORMAT_B4_C8_A16 , -I915_OA_FORMAT_C4_B8 , +I915_OA_FORMAT_A13 =1 ,/* HSW only */ +I915_OA_FORMAT_A29 ,/* HSW only */ +I915_OA_FORMAT_A13_B8_C8 ,/* HSW only */ +I915_OA_FORMAT_B4_C8 ,/* HSW only */ +I915_OA_FORMAT_A45_B8_C8 ,/* HSW only */ +I915_OA_FORMAT_B4_C8_A16 ,/* HSW only */ +I915_OA_FORMAT_C4_B8 ,/* HSW+ */ + +/* Gen8+ */ +I915_OA_FORMAT_A12 , +I915_OA_FORMAT_A12_B8_C8 , + I915_OA_FORMAT_A32u40_A4u32_B8_C8,I915_OA_FORMAT_MAX /* non-ABI */ };
--
2.11.0
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [PATCH v9 12/15] drm/i915/perf: Add OA unit support for Gen 8+
On Mon, May 01, 2017 at 06:17:09PM -0700, Lionel Landwerlin wrote:
Focusing on the bit I know best and leaving the hw mumbo jumbo to one side...
Quoted 37 lines
drivers/gpu/drm/i915/i915_perf.c +37
-0+static int gen8_configure_all_contexts (struct drm_i915_private *dev_priv) +{ +struct intel_engine_cs *engine = dev_priv->engine[RCS ]; +struct i915_gem_context *ctx; +int ret; + + ret =i915_mutex_lock_interruptible (&dev_priv->drm); +if (ret) +return ret; + +/* Switch away from any user context. */ + ret =i915_gem_switch_to_kernel_context (dev_priv); +if (ret) { +mutex_unlock (&dev_priv->drm.struct_mutex); +return ret; + } + +/* The OA register config is setup through the context image. This image +* might be written to by the GPU on context switch (in particular on +* lite-restore). This means we can't safely update a context's image, +* if this context is scheduled/submitted to run on the GPU. +* +* We could emit the OA register config through the batch buffer but +* this might leave small interval of time where the OA unit is +* configured at an invalid sampling period. +* +* So far the best way to work around this issue seems to be draining +* the GPU from any submitted work. +*/ + ret =i915_gem_wait_for_idle (dev_priv, +I915_WAIT_INTERRUPTIBLE | +I915_WAIT_LOCKED ); +if (ret) { +mutex_unlock (&dev_priv->drm.struct_mutex); +return ret; + } +
drivers/gpu/drm/i915/i915_perf.c static void hsw_disable_metric_set(struct drm_i915_private *dev_priv) · line 1617
Quoted 10 lines
drivers/gpu/drm/i915/i915_perf.c +10
-0+/* Update all contexts now that we've stalled the submission. */ +list_for_each_entry (ctx, &dev_priv->context_list, link) { + ret = engine->context_pin (engine, ctx); +if (ret) { +mutex_unlock (&dev_priv->drm.struct_mutex); +return ret; + } + +gen8_update_reg_state_unlocked (ctx, + ctx->engine[RCS ].lrc_reg_state);
engine->context_unpin is missing.
Since you do populate the OA settings for contexts initialised later, you can skip the context_pin here (keeping the deferred initialisation for currently unused contexts) and use
list_for_each_entry(ctx, &dev_priv->context_list, link) {
struct intel_context *ce = &ctx->engine[RCS];
u32 *regs;
if (!ce->state) /* OA settings will be set upon first use */
continue;
/* Hmm, export this from intel_lrc.c? */
regs = i915_gem_object_pin_map(ce->state->obj, I915_MAP_WB);
if (IS_ERR(regs)) {
/* teardown? */
mutex_unlock(&ctx->i915->drm.struct_mutex);
return PTR_ERR(regs);
}
ce->state->obj->mm.dirty = true;
regs += LRC_STATE_PN * PAGE_SIZE / sizeof(*regs);
/* Probably best since we don't really want to expose
* the LRC_STATE_PN trick.
*/
gen8_update_reg_state_unlocked(ctx, regs);
i915_gem_object_unpin_map(ce->state->obj);
}Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v10 12/15 drm/i915/perf: Add OA unit support for Gen 8+
From: Robert Bragg <[email protected]>
Enables access to OA unit metrics for BDW, CHV, SKL and BXT which all share (more-or-less) the same OA unit design.
Of particular note in comparison to Haswell: some OA unit HW config state has become per-context state and as a consequence it is somewhat more complicated to manage synchronous state changes from the cpu while there's no guarantee of what context (if any) is currently actively running on the gpu.
The periodic sampling frequency which can be particularly useful for system-wide analysis (as opposed to command stream synchronised MI_REPORT_PERF_COUNT commands) is perhaps the most surprising state to have become per-context save and restored (while the OABUFFER destination is still a shared, system-wide resource).
This support for gen8+ takes care to consider a number of timing challenges involved in synchronously updating per-context state primarily by programming all config state from the cpu and updating all current and saved contexts synchronously while the OA unit is still disabled.
The driver intentionally avoids depending on command streamer programming to update OA state considering the lack of synchronization between the automatic loading of OACTXCONTROL state (that includes the periodic sampling state and enable state) on context restore and the parsing of any general purpose BB the driver can control. I.e. this implementation is careful to avoid the possibility of a context restore temporarily enabling any out-of-date periodic sampling state. In addition to the risk of transiently-out-of-date state being loaded automatically; there are also internal HW latencies involved in the loading of MUX configurations which would be difficult to account for from the command streamer (and we only want to enable the unit when once the MUX configuration is complete).
Since the Gen8+ OA unit design no longer supports clock gating the unit off for a single given context (which effectively stopped any progress of counters while any other context was running) and instead supports tagging OA reports with a context ID for filtering on the CPU, it means we can no longer hide the system-wide progress of counters from a non-privileged application only interested in metrics for its own context. Although we could theoretically try and subtract the progress of other contexts before forwarding reports via read() we aren't in a position to filter reports captured via MI_REPORT_PERF_COUNT commands. As a result, for Gen8+, we always require the dev.i915.perf_stream_paranoid to be unset for any access to OA metrics if not root.
v5: Drain submitted requests when enabling metric set to ensure no
lite-restore erases the context image we just updated (Lionel)
v6: In addition to drain, switch to kernel context & update all
context in place (Chris)
v7: Add missing mutex_unlock() if switching to kernel context fails
(Matthew)
v8: Simplify OA period/flex-eu-counters programming by using the
batchbuffer instead of modifying ctx-image (Lionel)
v9: Back to updating the context image (due to erroneous testing,
batchbuffer programming the OA unit doesn't actually work) (Lionel)
Pin context before updating context image (Chris)
Drop MMIO programming now that we switch to a kernel context with right values in initial context image (Chris)
v10: Just pin_map the contexts we want to modify or let the
configuration happen on first use (Chris)
--- drivers/gpu/drm/i915/i915_drv.h | 45 +- drivers/gpu/drm/i915/i915_perf.c | 889 +++++++++++++++++++++++++++++++++++---- drivers/gpu/drm/i915/i915_reg.h | 22 + drivers/gpu/drm/i915/intel_lrc.c | 2 + include/uapi/drm/i915_drm.h | 19 +- 5 files changed, 884 insertions(+), 93 deletions(-)
drivers/gpu/drm/i915/i915_drv.h +30 -15
@@ -2063,9 +2063,17 @@ struct i915_oa_ops {void (*init_oa_buffer)(struct drm_i915_private *dev_priv);/** -* @enable_metric_set: Applies any MUX configuration to set up the -* Boolean and Custom (B/C) counters that are part of the counter -* reports being sampled. May apply system constraints such as +* @select_metric_set: The auto generated code that checks whether a +* requested OA config is applicable to the system and if so sets up +* the mux, oa and flex eu register config pointers according to the +* current dev_priv->perf.oa.metrics_set. +*/ +int (*select_metric_set)(struct drm_i915_private *dev_priv); + +/** +* @enable_metric_set: Selects and applies any MUX configuration to set +* up the Boolean and Custom (B/C) counters that are part of the +* counter reports being sampled. May apply system constraints such as * disabling EU clock gating as required. */ int (*enable_metric_set)(struct drm_i915_private *dev_priv);
@@ -2096,20 +2104,13 @@ struct i915_oa_ops { size_t *offset);/** -* @oa_buffer_check: Check for OA buffer data + update tail -* -* This is either called via fops or the poll check hrtimer (atomic -* ctx) without any locks taken. +* @oa_hw_tail_read: read the OA tail pointer register * -* It's safe to read OA config state here unlocked, assuming that this -* is only called while the stream is enabled, while the global OA -* configuration can't be modified. -* -* Efficiency is more important than avoiding some false positives -* here, which will be handled gracefully - likely resulting in an -* %EAGAIN error for userspace. +* In particular this enables us to share all the fiddly code for +* handling the OA unit tail pointer race that affects multiple +* generations. */ -bool (*oa_buffer_check)(struct drm_i915_private *dev_priv); +u32 (*oa_hw_tail_read)(struct drm_i915_private *dev_priv); };struct intel_cdclk_state {
@@ -2470,6 +2471,7 @@ struct drm_i915_private {struct {struct i915_vma *vma;u8 *vaddr; +u32 last_ctx_id;int format;int format_size;
@@ -2539,6 +2541,14 @@ struct drm_i915_private { } oa_buffer;u32 gen7_latched_oastatus1; +u32 ctx_oactxctrl_offset; +u32 ctx_flexeu0_offset; + +/* The RPT_ID/reason field for Gen8+ includes a bit +* to determine if the CTX ID in the report is valid +* but the specific bit differs between Gen 8 and 9 +*/ +u32 gen8_valid_ctx_bit;struct i915_oa_ops ops;const struct i915_oa_format *oa_formats;
@@ -2849,6 +2859,8 @@ intel_info(const struct drm_i915_private *dev_priv)#define IS_KBL_ULX (dev_priv) (INTEL_DEVID(dev_priv) == 0x590E || \ INTEL_DEVID(dev_priv) == 0x5915 || \ INTEL_DEVID(dev_priv) == 0x591E) +#define IS_SKL_GT2 (dev_priv) (IS_SKYLAKE(dev_priv) && \ + (INTEL_DEVID(dev_priv) & 0x00F0) == 0x0010)#define IS_SKL_GT3 (dev_priv) (IS_SKYLAKE(dev_priv) && \ (INTEL_DEVID(dev_priv) & 0x00F0) == 0x0020)#define IS_SKL_GT4 (dev_priv) (IS_SKYLAKE(dev_priv) && \
@@ -3612,6 +3624,9 @@ i915_gem_context_lookup_timeline(struct i915_gem_context *ctx,int i915_perf_open_ioctl (struct drm_device *dev,void *data,struct drm_file *file); +void i915_oa_init_reg_state (struct intel_engine_cs *engine, +struct i915_gem_context *ctx, +uint32_t *reg_state);/* i915_gem_evict.c */ int __must_check i915_gem_evict_something(struct i915_address_space *vm,
drivers/gpu/drm/i915/i915_perf.c +818 -71
1,078 lines not on this page · raw message
drivers/gpu/drm/i915/i915_reg.h +22 -0
@@ -653,6 +653,12 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)#define GEN8_OACTXID _MMIO(0x2364) +#define GEN8_OA_DEBUG _MMIO(0x2B04) +#define GEN9_OA_DEBUG_DISABLE_CLK_RATIO_REPORTS (1<<5) +#define GEN9_OA_DEBUG_INCLUDE_CLK_RATIO (1<<6) +#define GEN9_OA_DEBUG_DISABLE_GO_1_0_REPORTS (1<<2) +#define GEN9_OA_DEBUG_DISABLE_CTX_SWITCH_REPORTS (1<<1) +#define GEN8_OACONTROL _MMIO(0x2B00)#define GEN8_OA_REPORT_FORMAT_A12 (0<<2)#define GEN8_OA_REPORT_FORMAT_A12_B8_C8 (2<<2)
@@ -674,6 +680,7 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)#define GEN7_OABUFFER_STOP_RESUME_ENABLE (1<<1)#define GEN7_OABUFFER_RESUME (1<<0) +#define GEN8_OABUFFER_UDW _MMIO(0x23b4)#define GEN8_OABUFFER _MMIO(0x2b14)#define GEN7_OASTATUS1 _MMIO(0x2364)
@@ -692,7 +699,9 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)#define GEN8_OASTATUS_REPORT_LOST (1<<0)#define GEN8_OAHEADPTR _MMIO(0x2B0C) +#define GEN8_OAHEADPTR_MASK 0xffffffc0#define GEN8_OATAILPTR _MMIO(0x2B10) +#define GEN8_OATAILPTR_MASK 0xffffffc0#define OABUFFER_SIZE_128K (0<<3)#define OABUFFER_SIZE_256K (1<<3)
@@ -705,7 +714,17 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)#define OA_MEM_SELECT_GGTT (1<<0) +/* +* Flexible, Aggregate EU Counter Registers. +* Note: these aren't contiguous +*/ #define EU_PERF_CNTL0 _MMIO(0xe458) +#define EU_PERF_CNTL1 _MMIO(0xe558) +#define EU_PERF_CNTL2 _MMIO(0xe658) +#define EU_PERF_CNTL3 _MMIO(0xe758) +#define EU_PERF_CNTL4 _MMIO(0xe45c) +#define EU_PERF_CNTL5 _MMIO(0xe55c) +#define EU_PERF_CNTL6 _MMIO(0xe65c)#define GDT_CHICKEN_BITS _MMIO(0x9840)#define GT_NOA_ENABLE 0x00000080
@@ -2325,6 +2344,9 @@ enum skl_disp_power_wells {#define GEN8_RC_SEMA_IDLE_MSG_DISABLE (1 << 12)#define GEN8_FF_DOP_CLOCK_GATE_DISABLE (1<<10) +#define GEN6_RCS_PWR_FSM _MMIO(0x22ac) +#define GEN9_RCS_FE_FSM2 _MMIO(0x22a4) +/* Fuse readout registers for GT */ #define CHV_FUSE_GT _MMIO(VLV_DISPLAY_BASE + 0x2168)#define CHV_FGT_DISABLE_SS0 (1 << 10)
drivers/gpu/drm/i915/intel_lrc.c +2 -0
@@ -1877,6 +1877,8 @@ static void execlists_init_reg_state(u32 *regs, regs[CTX_LRI_HEADER_2 ] =MI_LOAD_REGISTER_IMM (1 );CTX_REG (regs,CTX_R_PWR_CLK_STATE ,GEN8_R_PWR_CLK_STATE ,make_rpcs (dev_priv)); + +i915_oa_init_reg_state (engine, ctx, regs); } }
include/uapi/drm/i915_drm.h +12 -7
@@ -1316,13 +1316,18 @@ struct drm_i915_gem_context_param { };enum drm_i915_oa_format { -I915_OA_FORMAT_A13 =1 , -I915_OA_FORMAT_A29 , -I915_OA_FORMAT_A13_B8_C8 , -I915_OA_FORMAT_B4_C8 , -I915_OA_FORMAT_A45_B8_C8 , -I915_OA_FORMAT_B4_C8_A16 , -I915_OA_FORMAT_C4_B8 , +I915_OA_FORMAT_A13 =1 ,/* HSW only */ +I915_OA_FORMAT_A29 ,/* HSW only */ +I915_OA_FORMAT_A13_B8_C8 ,/* HSW only */ +I915_OA_FORMAT_B4_C8 ,/* HSW only */ +I915_OA_FORMAT_A45_B8_C8 ,/* HSW only */ +I915_OA_FORMAT_B4_C8_A16 ,/* HSW only */ +I915_OA_FORMAT_C4_B8 ,/* HSW+ */ + +/* Gen8+ */ +I915_OA_FORMAT_A12 , +I915_OA_FORMAT_A12_B8_C8 , + I915_OA_FORMAT_A32u40_A4u32_B8_C8,I915_OA_FORMAT_MAX /* non-ABI */ };
--
2.11.0
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [PATCH v10 12/15] drm/i915/perf: Add OA unit support for Gen 8+
Quoted 867 lines · Lionel Landwerlin
From: Robert Bragg <[email protected]>
Enables access to OA unit metrics for BDW, CHV, SKL and BXT which all share (more-or-less) the same OA unit design.
Of particular note in comparison to Haswell: some OA unit HW config state has become per-context state and as a consequence it is somewhat more complicated to manage synchronous state changes from the cpu while there's no guarantee of what context (if any) is currently actively running on the gpu.
The periodic sampling frequency which can be particularly useful for system-wide analysis (as opposed to command stream synchronised
MI_REPORT_PERF_COUNTcommands) is perhaps the most surprising state to have become per-context save and restored (while the OABUFFER destination is still a shared, system-wide resource).This support for gen8+ takes care to consider a number of timing challenges involved in synchronously updating per-context state primarily by programming all config state from the cpu and updating all current and saved contexts synchronously while the OA unit is still disabled.
The driver intentionally avoids depending on command streamer programming to update OA state considering the lack of synchronization between the automatic loading of OACTXCONTROL state (that includes the periodic sampling state and enable state) on context restore and the parsing of any general purpose BB the driver can control. I.e. this implementation is careful to avoid the possibility of a context restore temporarily enabling any out-of-date periodic sampling state. In addition to the risk of transiently-out-of-date state being loaded automatically; there are also internal HW latencies involved in the loading of MUX configurations which would be difficult to account for from the command streamer (and we only want to enable the unit when once the MUX configuration is complete).
Since the Gen8+ OA unit design no longer supports clock gating the unit off for a single given context (which effectively stopped any progress of counters while any other context was running) and instead supports tagging OA reports with a context ID for filtering on the CPU, it means we can no longer hide the system-wide progress of counters from a non-privileged application only interested in metrics for its own context. Although we could theoretically try and subtract the progress of other contexts before forwarding reports via
read()we aren't in a position to filter reports captured viaMI_REPORT_PERF_COUNTcommands. As a result, for Gen8+, we always require the dev.i915.perf_stream_paranoid to be unset for any access to OA metrics if not root.v5: Drain submitted requests when enabling metric set to ensure no
lite-restore erases the context image we just updated (Lionel)v6: In addition to drain, switch to kernel context & update all
context in place (Chris)v7: Add missing
mutex_unlock()if switching to kernel context fails
(Matthew)v8: Simplify OA period/flex-eu-counters programming by using the
batchbuffer instead of modifying ctx-image (Lionel)v9: Back to updating the context image (due to erroneous testing,
batchbuffer programming the OA unit doesn't actually work) (Lionel)
Pin context before updating context image (Chris)
Drop MMIO programming now that we switch to a kernel context with right values in initial context image (Chris)v10: Just
pin_mapthe contexts we want to modify or let the
configuration happen on first use (Chris)Signed-off-by Robert Bragg <[email protected]>Signed-off-by Lionel Landwerlin <[email protected]>Reviewed-by Matthew Auld <[email protected]> \o/--- drivers/gpu/drm/i915/i915_drv.h | 45 +- drivers/gpu/drm/i915/i915_perf.c | 889 +++++++++++++++++++++++++++++++++++---- drivers/gpu/drm/i915/i915_reg.h | 22 + drivers/gpu/drm/i915/intel_lrc.c | 2 + include/uapi/drm/i915_drm.h | 19 +- 5 files changed, 884 insertions(+), 93 deletions(-)drivers/gpu/drm/i915/i915_drv.h +30
-15@@ -2063,9 +2063,17 @@ struct i915_oa_ops {void (*init_oa_buffer)(struct drm_i915_private *dev_priv);/** -* @enable_metric_set: Applies any MUX configuration to set up the -* Boolean and Custom (B/C) counters that are part of the counter -* reports being sampled. May apply system constraints such as +* @select_metric_set: The auto generated code that checks whether a +* requested OA config is applicable to the system and if so sets up +* the mux, oa and flex eu register config pointers according to the +* current dev_priv->perf.oa.metrics_set. +*/ +int (*select_metric_set)(struct drm_i915_private *dev_priv); + +/** +* @enable_metric_set: Selects and applies any MUX configuration to set +* up the Boolean and Custom (B/C) counters that are part of the +* counter reports being sampled. May apply system constraints such as * disabling EU clock gating as required. */ int (*enable_metric_set)(struct drm_i915_private *dev_priv);@@ -2096,20 +2104,13 @@ struct i915_oa_ops { size_t *offset);/** -* @oa_buffer_check: Check for OA buffer data + update tail -* -* This is either called via fops or the poll check hrtimer (atomic -* ctx) without any locks taken. +* @oa_hw_tail_read: read the OA tail pointer register * -* It's safe to read OA config state here unlocked, assuming that this -* is only called while the stream is enabled, while the global OA -* configuration can't be modified. -* -* Efficiency is more important than avoiding some false positives -* here, which will be handled gracefully - likely resulting in an -* %EAGAIN error for userspace. +* In particular this enables us to share all the fiddly code for +* handling the OA unit tail pointer race that affects multiple +* generations. */ -bool (*oa_buffer_check)(struct drm_i915_private *dev_priv); +u32 (*oa_hw_tail_read)(struct drm_i915_private *dev_priv); };struct intel_cdclk_state {@@ -2470,6 +2471,7 @@ struct drm_i915_private {struct {struct i915_vma *vma;u8 *vaddr; +u32 last_ctx_id;int format;int format_size;@@ -2539,6 +2541,14 @@ struct drm_i915_private { } oa_buffer;u32 gen7_latched_oastatus1; +u32 ctx_oactxctrl_offset; +u32 ctx_flexeu0_offset; + +/* The RPT_ID/reason field for Gen8+ includes a bit +* to determine if the CTX ID in the report is valid +* but the specific bit differs between Gen 8 and 9 +*/ +u32 gen8_valid_ctx_bit;struct i915_oa_ops ops;const struct i915_oa_format *oa_formats;@@ -2849,6 +2859,8 @@ intel_info(const struct drm_i915_private *dev_priv)#define IS_KBL_ULX (dev_priv) (INTEL_DEVID(dev_priv) == 0x590E || \ INTEL_DEVID(dev_priv) == 0x5915 || \ INTEL_DEVID(dev_priv) == 0x591E) +#define IS_SKL_GT2 (dev_priv) (IS_SKYLAKE(dev_priv) && \ + (INTEL_DEVID(dev_priv) & 0x00F0) == 0x0010)#define IS_SKL_GT3 (dev_priv) (IS_SKYLAKE(dev_priv) && \ (INTEL_DEVID(dev_priv) & 0x00F0) == 0x0020)#define IS_SKL_GT4 (dev_priv) (IS_SKYLAKE(dev_priv) && \@@ -3612,6 +3624,9 @@ i915_gem_context_lookup_timeline(struct i915_gem_context *ctx,int i915_perf_open_ioctl (struct drm_device *dev,void *data,struct drm_file *file); +void i915_oa_init_reg_state (struct intel_engine_cs *engine, +struct i915_gem_context *ctx, +uint32_t *reg_state);/* i915_gem_evict.c */ int __must_check i915_gem_evict_something(struct i915_address_space *vm,drivers/gpu/drm/i915/i915_perf.c +532
-35@@ -196,6 +196,12 @@#include "i915_drv.h" #include "i915_oa_hsw.h" +#include "i915_oa_bdw.h" +#include "i915_oa_chv.h" +#include "i915_oa_sklgt2.h" +#include "i915_oa_sklgt3.h" +#include "i915_oa_sklgt4.h" +#include "i915_oa_bxt.h" /* HW requires this to be a power of two, between 128k and 16M, though driver * is currently generally designed assuming the largest 16M size is used such @@ -215,7 +221,7 @@* * Although this can be observed explicitly while copying reports to userspace * by checking for a zeroed report-id field in tail reports, we want to account -* for this earlier, as part of the _oa_buffer_check to avoid lots of redundant +* for this earlier, as part of the oa_buffer_check to avoid lots of redundant * read() attempts. * * In effect we define a tail pointer for reading that lags the real tail @@ -237,7 +243,7 @@* indicates that an updated tail pointer is needed. * * Most of the implementation details for this workaround are in -* gen7_oa_buffer_check_unlocked() and gen7_appand_oa_reports() +* oa_buffer_check_unlocked() and _append_oa_reports() * * Note for posterity: previously the driver used to define an effective tail * pointer that lagged the real pointer by a 'tail margin' measured in bytes @@ -272,6 +278,13 @@ static u32 i915_perf_stream_paranoid = true;#define INVALID_CTX_ID 0xffffffff +/* On Gen8+ automatically triggered OA reports include a 'reason' field... */ +#define OAREPORT_REASON_MASK 0x3f +#define OAREPORT_REASON_SHIFT 19 +#define OAREPORT_REASON_TIMER (1<<0) +#define OAREPORT_REASON_CTX_SWITCH (1<<3) +#define OAREPORT_REASON_CLK_RATIO (1<<5) +/* For sysctl proc_dointvec_minmax of i915_oa_max_sample_rate * @@ -303,6 +316,13 @@ static struct i915_oa_format hsw_oa_formats[I915_OA_FORMAT_MAX] = { [I915_OA_FORMAT_C4_B8] = { 7, 64 }, }; +static struct i915_oa_format gen8_plus_oa_formats[I915_OA_FORMAT_MAX ] = { + [I915_OA_FORMAT_A12 ] = {0 ,64 }, + [I915_OA_FORMAT_A12_B8_C8 ] = {2 ,128 }, + [I915_OA_FORMAT_A32u40_A4u32_B8_C8] = {5 ,256 }, + [I915_OA_FORMAT_C4_B8 ] = {7 ,64 }, +}; +#define SAMPLE_OA_REPORT (1<<0)/** @@ -332,8 +352,20 @@ struct perf_open_properties {int oa_period_exponent; }; +static u32 gen8_oa_hw_tail_read (struct drm_i915_private *dev_priv) +{ +return I915_READ (GEN8_OATAILPTR ) &GEN8_OATAILPTR_MASK ; +} + +static u32 gen7_oa_hw_tail_read (struct drm_i915_private *dev_priv) +{ +u32 oastatus1 =I915_READ (GEN7_OASTATUS1 ); + +return oastatus1 &GEN7_OASTATUS1_TAIL_MASK ; +} +/** -* gen7_oa_buffer_check_unlocked - check for data and update tail ptr state +* oa_buffer_check_unlocked - check for data and update tail ptr state * @dev_priv: i915 device instance * * This is either called via fops (for blocking reads in user ctx) or the poll @@ -356,12 +388,11 @@ struct perf_open_properties {* * Returns: %true if the OA buffer contains data, else %false */ -static bool gen7_oa_buffer_check_unlocked (struct drm_i915_private *dev_priv) +static bool oa_buffer_check_unlocked (struct drm_i915_private *dev_priv) {int report_size = dev_priv->perf.oa.oa_buffer.format_size;unsigned long flags;unsigned int aged_idx; -u32 oastatus1;u32 head, hw_tail, aged_tail, aging_tail;u64 now;@@ -381,8 +412,7 @@ static bool gen7_oa_buffer_check_unlocked(struct drm_i915_private *dev_priv) aged_tail = dev_priv->perf.oa.oa_buffer.tails[aged_idx].offset; aging_tail = dev_priv->perf.oa.oa_buffer.tails[!aged_idx].offset; - oastatus1 =I915_READ (GEN7_OASTATUS1 ); - hw_tail = oastatus1 &GEN7_OASTATUS1_TAIL_MASK ; + hw_tail = dev_priv->perf.oa.ops.oa_hw_tail_read (dev_priv);/* The tail pointer increases in 64 byte increments, * not in report_size steps... @@ -404,6 +434,7 @@ static bool gen7_oa_buffer_check_unlocked(struct drm_i915_private *dev_priv)if (aging_tail !=INVALID_TAIL_PTR && ((now - dev_priv->perf.oa.oa_buffer.aging_timestamp) >OA_TAIL_MARGIN_NSEC )) { + aged_idx ^=1 ; dev_priv->perf.oa.oa_buffer.aged_tail_idx = aged_idx;@@ -553,6 +584,287 @@ static int append_oa_sample(struct i915_perf_stream *stream,* * Returns: 0 on success, negative error code on failure. */ +static int gen8_append_oa_reports (struct i915_perf_stream *stream, +char __user *buf, +size_t count, +size_t *offset) +{ +struct drm_i915_private *dev_priv = stream->dev_priv; +int report_size = dev_priv->perf.oa.oa_buffer.format_size; +u8 *oa_buf_base = dev_priv->perf.oa.oa_buffer.vaddr; +u32 gtt_offset =i915_ggtt_offset (dev_priv->perf.oa.oa_buffer.vma); +u32 mask = (OA_BUFFER_SIZE -1 ); +size_t start_offset = *offset; +unsigned long flags; +unsigned int aged_tail_idx; +u32 head, tail; +u32 taken; +int ret =0 ; + +if (WARN_ON (!stream->enabled)) +return -EIO ; + +spin_lock_irqsave (&dev_priv->perf.oa.oa_buffer.ptr_lock, flags); + + head = dev_priv->perf.oa.oa_buffer.head; + aged_tail_idx = dev_priv->perf.oa.oa_buffer.aged_tail_idx; + tail = dev_priv->perf.oa.oa_buffer.tails[aged_tail_idx].offset; + +spin_unlock_irqrestore (&dev_priv->perf.oa.oa_buffer.ptr_lock, flags); + +/* An invalid tail pointer here means we're still waiting for the poll +* hrtimer callback to give us a pointer +*/ +if (tail ==INVALID_TAIL_PTR ) +return -EAGAIN ; + +/* NB: oa_buffer.head/tail include the gtt_offset which we don't want +* while indexing relative to oa_buf_base. +*/ + head -= gtt_offset; + tail -= gtt_offset; + +/* An out of bounds or misaligned head or tail pointer implies a driver +* bug since we validate + align the tail pointers we read from the +* hardware and we are in full control of the head pointer which should +* only be incremented by multiples of the report size (notably also +* all a power of two). +*/ +if (WARN_ONCE (head >OA_BUFFER_SIZE || head % report_size || + tail >OA_BUFFER_SIZE || tail % report_size, +"Inconsistent OA buffer pointers: head = %u, tail = %u\n" , + head, tail)) +return -EIO ; + + +for (/* none */ ; + (taken =OA_TAKEN (tail, head)); + head = (head + report_size) & mask) { +u8 *report = oa_buf_base + head; +u32 *report32 = (void *)report; +u32 ctx_id; +u32 reason; + +/* All the report sizes factor neatly into the buffer +* size so we never expect to see a report split +* between the beginning and end of the buffer. +* +* Given the initial alignment check a misalignment +* here would imply a driver bug that would result +* in an overrun. +*/ +if (WARN_ON ((OA_BUFFER_SIZE - head) < report_size)) { +DRM_ERROR ("Spurious OA head ptr: non-integral report offset\n" ); +break ; + } + +/* The reason field includes flags identifying what +* triggered this specific report (mostly timer +* triggered or e.g. due to a context switch). +* +* This field is never expected to be zero so we can +* check that the report isn't invalid before copying +* it to userspace... +*/ + reason = ((report32[0 ] >>OAREPORT_REASON_SHIFT ) & +OAREPORT_REASON_MASK ); +if (reason ==0 ) { +if (__ratelimit (&dev_priv->perf.oa.spurious_report_rs)) +DRM_NOTE ("Skipping spurious, invalid OA report\n" ); +continue ; + } + +/* XXX: Just keep the lower 21 bits for now since I'm not +* entirely sure if the HW touches any of the higher bits in +* this field +*/ + ctx_id = report32[2 ] &0x1fffff ; + +/* Squash whatever is in the CTX_ID field if it's marked as +* invalid to be sure we avoid false-positive, single-context +* filtering below... +* +* Note: that we don't clear the valid_ctx_bit so userspace can +* understand that the ID has been squashed by the kernel. +*/ +if (!(report32[0 ] & dev_priv->perf.oa.gen8_valid_ctx_bit)) + ctx_id = report32[2 ] =INVALID_CTX_ID ; + +/* NB: For Gen 8 the OA unit no longer supports clock gating +* off for a specific context and the kernel can't securely +* stop the counters from updating as system-wide / global +* values. +* +* Automatic reports now include a context ID so reports can be +* filtered on the cpu but it's not worth trying to +* automatically subtract/hide counter progress for other +* contexts while filtering since we can't stop userspace +* issuing MI_REPORT_PERF_COUNT commands which would still +* provide a side-band view of the real values. +* +* To allow userspace (such as Mesa/GL_INTEL_performance_query) +* to normalize counters for a single filtered context then it +* needs be forwarded bookend context-switch reports so that it +* can track switches in between MI_REPORT_PERF_COUNT commands +* and can itself subtract/ignore the progress of counters +* associated with other contexts. Note that the hardware +* automatically triggers reports when switching to a new +* context which are tagged with the ID of the newly active +* context. To avoid the complexity (and likely fragility) of +* reading ahead while parsing reports to try and minimize +* forwarding redundant context switch reports (i.e. between +* other, unrelated contexts) we simply elect to forward them +* all. +* +* We don't rely solely on the reason field to identify context +* switches since it's not-uncommon for periodic samples to +* identify a switch before any 'context switch' report. +*/ +if (!dev_priv->perf.oa.exclusive_stream->ctx || + dev_priv->perf.oa.specific_ctx_id == ctx_id || + (dev_priv->perf.oa.oa_buffer.last_ctx_id == + dev_priv->perf.oa.specific_ctx_id) || + reason &OAREPORT_REASON_CTX_SWITCH ) { + +/* While filtering for a single context we avoid +* leaking the IDs of other contexts. +*/ +if (dev_priv->perf.oa.exclusive_stream->ctx && + dev_priv->perf.oa.specific_ctx_id != ctx_id) { + report32[2 ] =INVALID_CTX_ID ; + } + + ret =append_oa_sample (stream, buf, count, offset, + report); +if (ret) +break ; + + dev_priv->perf.oa.oa_buffer.last_ctx_id = ctx_id; + } + +/* The above reason field sanity check is based on +* the assumption that the OA buffer is initially +* zeroed and we reset the field after copying so the +* check is still meaningful once old reports start +* being overwritten. +*/ + report32[0 ] =0 ; + } + +if (start_offset != *offset) { +spin_lock_irqsave (&dev_priv->perf.oa.oa_buffer.ptr_lock, flags); + +/* We removed the gtt_offset for the copy loop above, indexing +* relative to oa_buf_base so put back here... +*/ + head += gtt_offset; + +I915_WRITE (GEN8_OAHEADPTR , head &GEN8_OAHEADPTR_MASK ); + dev_priv->perf.oa.oa_buffer.head = head; + +spin_unlock_irqrestore (&dev_priv->perf.oa.oa_buffer.ptr_lock, flags); + } + +return ret; +} + +/** +* gen8_oa_read - copy status records then buffered OA reports +* @stream: An i915-perf stream opened for OA metrics +* @buf: destination buffer given by userspace +* @count: the number of bytes userspace wants to read +* @offset: (inout): the current position for writing into @buf +* +* Checks OA unit status registers and if necessary appends corresponding +* status records for userspace (such as for a buffer full condition) and then +* initiate appending any buffered OA reports. +* +* Updates @offset according to the number of bytes successfully copied into +* the userspace buffer. +* +* NB: some data may be successfully copied to the userspace buffer +* even if an error is returned, and this is reflected in the +* updated @offset. +* +* Returns: zero on success or a negative error code +*/ +static int gen8_oa_read (struct i915_perf_stream *stream, +char __user *buf, +size_t count, +size_t *offset) +{ +struct drm_i915_private *dev_priv = stream->dev_priv; +u32 oastatus; +int ret; + +if (WARN_ON (!dev_priv->perf.oa.oa_buffer.vaddr)) +return -EIO ; + + oastatus =I915_READ (GEN8_OASTATUS ); + +/* We treat OABUFFER_OVERFLOW as a significant error: +* +* Although theoretically we could handle this more gracefully +* sometimes, some Gens don't correctly suppress certain +* automatically triggered reports in this condition and so we +* have to assume that old reports are now being trampled +* over. +* +* Considering how we don't currently give userspace control +* over the OA buffer size and always configure a large 16MB +* buffer, then a buffer overflow does anyway likely indicate +* that something has gone quite badly wrong. +*/ +if (oastatus &GEN8_OASTATUS_OABUFFER_OVERFLOW ) { + ret =append_oa_status (stream, buf, count, offset, +DRM_I915_PERF_RECORD_OA_BUFFER_LOST ); +if (ret) +return ret; + +DRM_DEBUG ("OA buffer overflow (exponent = %d): force restart\n" , + dev_priv->perf.oa.period_exponent); + + dev_priv->perf.oa.ops.oa_disable (dev_priv); + dev_priv->perf.oa.ops.oa_enable (dev_priv); + +/* Note: .oa_enable() is expected to re-init the oabuffer +* and reset GEN8_OASTATUS for us +*/ + oastatus =I915_READ (GEN8_OASTATUS ); + } + +if (oastatus &GEN8_OASTATUS_REPORT_LOST ) { + ret =append_oa_status (stream, buf, count, offset, +DRM_I915_PERF_RECORD_OA_REPORT_LOST ); +if (ret) +return ret; +I915_WRITE (GEN8_OASTATUS , + oastatus & ~GEN8_OASTATUS_REPORT_LOST ); + } + +return gen8_append_oa_reports (stream, buf, count, offset); +} + +/** +* Copies all buffered OA reports into userspace read() buffer. +* @stream: An i915-perf stream opened for OA metrics +* @buf: destination buffer given by userspace +* @count: the number of bytes userspace wants to read +* @offset: (inout): the current position for writing into @buf +* +* Notably any error condition resulting in a short read (-%ENOSPC or +* -%EFAULT) will be returned even though one or more records may +* have been successfully copied. In this case it's up to the caller +* to decide if the error should be squashed before returning to +* userspace. +* +* Note: reports are consumed from the head, and appended to the +* tail, so the tail chases the head?... If you think that's mad +* and back-to-front you're not alone, but this follows the +* Gen PRM naming convention. +* +* Returns: 0 on success, negative error code on failure. +*/ static int gen7_append_oa_reports(struct i915_perf_stream *stream, char __user *buf, size_t count,@@ -733,7 +1045,8 @@ static int gen7_oa_read(struct i915_perf_stream *stream,if (ret)return ret; -DRM_DEBUG ("OA buffer overflow: force restart\n" ); +DRM_DEBUG ("OA buffer overflow (exponent = %d): force restart\n" , + dev_priv->perf.oa.period_exponent); dev_priv->perf.oa.ops.oa_disable (dev_priv); dev_priv->perf.oa.ops.oa_enable (dev_priv);@@ -776,7 +1089,7 @@ static int i915_oa_wait_unlocked(struct i915_perf_stream *stream)return -EIO ;return wait_event_interruptible (dev_priv->perf.oa.poll_wq, - dev_priv->perf.oa.ops.oa_buffer_check (dev_priv)); +oa_buffer_check_unlocked (dev_priv)); }/** @@ -833,33 +1146,37 @@ static int i915_oa_read(struct i915_perf_stream *stream,static int oa_get_render_ctx_id (struct i915_perf_stream *stream) {struct drm_i915_private *dev_priv = stream->dev_priv; -struct intel_engine_cs *engine = dev_priv->engine[RCS ]; -int ret; - ret =i915_mutex_lock_interruptible (&dev_priv->drm); -if (ret) -return ret; +if (i915.enable_execlists) + dev_priv->perf.oa.specific_ctx_id = stream->ctx->hw_id; +else { +struct intel_engine_cs *engine = dev_priv->engine[RCS ]; +int ret; -/* As the ID is the gtt offset of the context's vma we pin -* the vma to ensure the ID remains fixed. -* -* NB: implied RCS engine... -*/ - ret = engine->context_pin (engine, stream->ctx); -if (ret) - goto unlock; + ret =i915_mutex_lock_interruptible (&dev_priv->drm); +if (ret) +return ret; -/* Explicitly track the ID (instead of calling i915_ggtt_offset() -* on the fly) considering the difference with gen8+ and -* execlists -*/ - dev_priv->perf.oa.specific_ctx_id = -i915_ggtt_offset (stream->ctx->engine[engine->id].state); +/* As the ID is the gtt offset of the context's vma we pin +* the vma to ensure the ID remains fixed. +*/ + ret = engine->context_pin (engine, stream->ctx); +if (ret) { +mutex_unlock (&dev_priv->drm.struct_mutex); +return ret; + } -unlock: -mutex_unlock (&dev_priv->drm.struct_mutex); +/* Explicitly track the ID (instead of calling +* i915_ggtt_offset() on the fly) considering the difference +* with gen8+ and execlists +*/ + dev_priv->perf.oa.specific_ctx_id = +i915_ggtt_offset (stream->ctx->engine[engine->id].state); -return ret; +mutex_unlock (&dev_priv->drm.struct_mutex); + } + +return 0 ; }/** @@ -874,12 +1191,16 @@ static void oa_put_render_ctx_id(struct i915_perf_stream *stream)struct drm_i915_private *dev_priv = stream->dev_priv;struct intel_engine_cs *engine = dev_priv->engine[RCS ]; -mutex_lock (&dev_priv->drm.struct_mutex); +if (i915.enable_execlists) { + dev_priv->perf.oa.specific_ctx_id =INVALID_CTX_ID ; + }else { +mutex_lock (&dev_priv->drm.struct_mutex); - dev_priv->perf.oa.specific_ctx_id =INVALID_CTX_ID ; - engine->context_unpin (engine, stream->ctx); + dev_priv->perf.oa.specific_ctx_id =INVALID_CTX_ID ; + engine->context_unpin (engine, stream->ctx); -mutex_unlock (&dev_priv->drm.struct_mutex); +mutex_unlock (&dev_priv->drm.struct_mutex); + } } static void@@ -969,6 +1290,61 @@ static void gen7_init_oa_buffer(struct drm_i915_private *dev_priv) dev_priv->perf.oa.pollin = false; } +static void gen8_init_oa_buffer (struct drm_i915_private *dev_priv) +{ +u32 gtt_offset =i915_ggtt_offset (dev_priv->perf.oa.oa_buffer.vma); +unsigned long flags; + +spin_lock_irqsave (&dev_priv->perf.oa.oa_buffer.ptr_lock, flags); + +I915_WRITE (GEN8_OASTATUS ,0 ); +I915_WRITE (GEN8_OAHEADPTR , gtt_offset); + dev_priv->perf.oa.oa_buffer.head = gtt_offset; + +I915_WRITE (GEN8_OABUFFER_UDW ,0 ); + +/* PRM says: +* +* "This MMIO must be set before the OATAILPTR +* register and after the OAHEADPTR register. This is +* to enable proper functionality of the overflow +* bit." +*/ +I915_WRITE (GEN8_OABUFFER , gtt_offset | +OABUFFER_SIZE_16M |OA_MEM_SELECT_GGTT ); +I915_WRITE (GEN8_OATAILPTR , gtt_offset &GEN8_OATAILPTR_MASK ); + +/* Mark that we need updated tail pointers to read from... */ + dev_priv->perf.oa.oa_buffer.tails[0 ].offset =INVALID_TAIL_PTR ; + dev_priv->perf.oa.oa_buffer.tails[1 ].offset =INVALID_TAIL_PTR ; + +/* Reset state used to recognise context switches, affecting which +* reports we will forward to userspace while filtering for a single +* context. +*/ + dev_priv->perf.oa.oa_buffer.last_ctx_id =INVALID_CTX_ID ; + +spin_unlock_irqrestore (&dev_priv->perf.oa.oa_buffer.ptr_lock, flags); + +/* NB: although the OA buffer will initially be allocated +* zeroed via shmfs (and so this memset is redundant when +* first allocating), we may re-init the OA buffer, either +* when re-enabling a stream or in error/reset paths. +* +* The reason we clear the buffer for each re-init is for the +* sanity check in gen8_append_oa_reports() that looks at the +* reason field to make sure it's non-zero which relies on +* the assumption that new reports are being written to zeroed +* memory... +*/ +memset (dev_priv->perf.oa.oa_buffer.vaddr,0 ,OA_BUFFER_SIZE ); + +/* Maybe make ->pollin per-stream state if we support multiple +* concurrent streams in the future. +*/ + dev_priv->perf.oa.pollin = false; +} +static int alloc_oa_buffer (struct drm_i915_private *dev_priv) {struct drm_i915_gem_object *bo;@@ -1113,6 +1489,212 @@ static void hsw_disable_metric_set(struct drm_i915_private *dev_priv) ~GT_NOA_ENABLE)); } +/* NB: It must always remain pointer safe to run this even if the OA unit +* has been disabled. +* +* It's fine to put out-of-date values into these per-context registers +* in the case that the OA unit has been disabled. +*/ +static void gen8_update_reg_state_unlocked (struct i915_gem_context *ctx, +u32 *reg_state) +{ +struct drm_i915_private *dev_priv = ctx->i915; +const struct i915_oa_reg *flex_regs = dev_priv->perf.oa.flex_regs; +int n_flex_regs = dev_priv->perf.oa.flex_regs_len; +u32 ctx_oactxctrl = dev_priv->perf.oa.ctx_oactxctrl_offset; +u32 ctx_flexeu0 = dev_priv->perf.oa.ctx_flexeu0_offset; +/* The MMIO offsets for Flex EU registers aren't contiguous */ +u32 flex_mmio[] = { +i915_mmio_reg_offset (EU_PERF_CNTL0 ), +i915_mmio_reg_offset (EU_PERF_CNTL1 ), +i915_mmio_reg_offset (EU_PERF_CNTL2 ), +i915_mmio_reg_offset (EU_PERF_CNTL3 ), +i915_mmio_reg_offset (EU_PERF_CNTL4 ), +i915_mmio_reg_offset (EU_PERF_CNTL5 ), +i915_mmio_reg_offset (EU_PERF_CNTL6 ), + }; +int i; + + reg_state[ctx_oactxctrl] =i915_mmio_reg_offset (GEN8_OACTXCONTROL ); + reg_state[ctx_oactxctrl+1 ] = (dev_priv->perf.oa.period_exponent << +GEN8_OA_TIMER_PERIOD_SHIFT ) | + (dev_priv->perf.oa.periodic ? +GEN8_OA_TIMER_ENABLE :0 ) | +GEN8_OA_COUNTER_RESUME ; + +for (i =0 ; i <ARRAY_SIZE (flex_mmio); i++) { +u32 state_offset = ctx_flexeu0 + i *2 ; +u32 mmio = flex_mmio[i]; + +/* This arbitrary default will select the 'EU FPU0 Pipeline +* Active' event. In the future it's anticipated that there +* will be an explicit 'No Event' we can select, but not yet... +*/ +u32 value =0 ; +int j; + +for (j =0 ; j < n_flex_regs; j++) { +if (i915_mmio_reg_offset (flex_regs[j].addr) == mmio) { + value = flex_regs[j].value; +break ; + } + } + + reg_state[state_offset] = mmio; + reg_state[state_offset+1 ] = value; + } +} + +/* Manages updating the per-context aspects of the OA stream +* configuration across all contexts. +* +* The awkward consideration here is that OACTXCONTROL controls the +* exponent for periodic sampling which is primarily used for system +* wide profiling where we'd like a consistent sampling period even in +* the face of context switches. +* +* Our approach of updating the register state context (as opposed to +* say using a workaround batch buffer) ensures that the hardware +* won't automatically reload an out-of-date timer exponent even +* transiently before a WA BB could be parsed. +* +* This function needs to: +* - Ensure the currently running context's per-context OA state is +* updated +* - Ensure that all existing contexts will have the correct per-context +* OA state if they are scheduled for use. +* - Ensure any new contexts will be initialized with the correct +* per-context OA state. +* +* Note: it's only the RCS/Render context that has any OA state. +*/ +static int gen8_configure_all_contexts (struct drm_i915_private *dev_priv) +{ +struct i915_gem_context *ctx; +int ret; + + ret =i915_mutex_lock_interruptible (&dev_priv->drm); +if (ret) +return ret; + +/* Switch away from any user context. */ + ret =i915_gem_switch_to_kernel_context (dev_priv); +if (ret) + goto out; + +/* The OA register config is setup through the context image. This image +* might be written to by the GPU on context switch (in particular on +* lite-restore). This means we can't safely update a context's image, +* if this context is scheduled/submitted to run on the GPU. +* +* We could emit the OA register config through the batch buffer but +* this might leave small interval of time where the OA unit is +* configured at an invalid sampling period. +* +* So far the best way to work around this issue seems to be draining +* the GPU from any submitted work. +*/ + ret =i915_gem_wait_for_idle (dev_priv, +I915_WAIT_INTERRUPTIBLE | +I915_WAIT_LOCKED ); +if (ret) + goto out; + +/* Update all contexts now that we've stalled the submission. */ +list_for_each_entry (ctx, &dev_priv->context_list, link) { +struct intel_context *ce = &ctx->engine[RCS ]; +u32 *regs; + +/* OA settings will be set upon first use */ +if (!ce->state) +continue ; + + regs =i915_gem_object_pin_map (ce->state->obj,I915_MAP_WB );
drivers/gpu/drm/i915/i915_perf.c static void hsw_disable_metric_set(struct drm_i915_private *dev_priv) · line 1622
Quoted 10 lines
drivers/gpu/drm/i915/i915_perf.c +10
-0+if (IS_ERR (regs)) { + ret =PTR_ERR (regs); + goto out; + } + + ce->state->obj->mm.dirty = true; + regs +=LRC_STATE_PN *PAGE_SIZE /sizeof (*regs); +/* Probably best since we don't really want to expose +* the LRC_STATE_PN trick. +*/
Sorry, was writing comments to myself :) This is not about the code, just about the interface to export mapping the context from intel_lrc.c
The only worry I have here is a failure whilst pinning the contexts, leaving some updated, some not, and upon error we will assume the change did not occur.
Other than that, this looks correct, and fwiw I do much prefer the idea of the OA adjustment being synchronous wrt to request submission (that all previous requests complete using the current OA settings and all new submissions will use the next OA config). The downside is that it may stall the driver for an indefinite period of time - but for a global config that seems acceptable in order to have it well defined. Per-context OA configs presumably are faster to apply?
For the context image adjustment,
and Matthew can take responsibility for having checked the rest ;) -Chris
Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx