Explicity steer l3bank multicast reads when necessary
Subsystems (2)
v2 cover Explicity steer l3bank multicast reads when necessary
We've recently learned that when steering reads of multicast registers that use 'subslice' replication, it's not only important to steer to a subslice that isn't fused off, but also to steer to the lowest-numbered subslice. This is because when Render Power Gating is enabled, grabbing forcewake will only cause the hardware to power up a single subslice (referred to as the "minconfig") until/unless a real workload is being run on the EUs. If we try to read back a value from a register instance other than the minconfig subslice, the read operation will either return 0 or random garbage.
Unfortunately this extra requirement to steer to the minconfig means that the steering target we use for subslice-replicated registers may not select a valid instance for l3bank-replicated registers. In cases where the two types of multicast registers do not have compatible steering targets, we'll initialize the steering control register to the proper subslice target at driver load, and then explicitly re-steer individual reads of l3bank registers as they occur at runtime.
This series sets up an infrastructure to handle explicit resteering of multiple multicast register types, and then applies it to l3bank registers. Our next upcoming platform (which we'll probably start upstreaming soon) will bring several more types of multicast registers, each with their own steering criteria, so the infrastructure here is partially in preparation for those extra multicast types that will be arriving soon.
v2:
- Use {} as table terminator and check for end==0 instead of 0xFFFFFF on loop iteration. (Rodrigo)
- Use gt->uncore instead of gt->i915->uncore. (Tvrtko)
- Now that wa_list_verify() uses _fw accessors we need to explicitly grab forcewake.
Daniele Ceraolo Spurio (1):
drm/i915: extract steered reg access to common function
Matt Roper (2):
drm/i915: Add GT support for multiple types of multicast steering drm/i915: Add support for explicit L3BANK steering
drivers/gpu/drm/i915/gt/intel_engine_cs.c | 41 +----- drivers/gpu/drm/i915/gt/intel_gt.c | 102 +++++++++++++++ drivers/gpu/drm/i915/gt/intel_gt.h | 8 ++ drivers/gpu/drm/i915/gt/intel_gt_types.h | 26 ++++ drivers/gpu/drm/i915/gt/intel_workarounds.c | 123 ++++++++---------- .../gpu/drm/i915/gt/selftest_workarounds.c | 2 +- drivers/gpu/drm/i915/intel_uncore.c | 55 ++++++++ drivers/gpu/drm/i915/intel_uncore.h | 6 + 8 files changed, 251 insertions(+), 112 deletions(-)
Signature · 6 lines
2.25.4 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v2 1/3 drm/i915: extract steered reg access to common function
From: Daniele Ceraolo Spurio <[email protected]>
New steering cases will be added in the follow-up patches, so prepare a common helper to avoid code duplication.
--- drivers/gpu/drm/i915/gt/intel_engine_cs.c | 41 +---------------- drivers/gpu/drm/i915/intel_uncore.c | 55 +++++++++++++++++++++++ drivers/gpu/drm/i915/intel_uncore.h | 6 +++ 3 files changed, 63 insertions(+), 39 deletions(-)
drivers/gpu/drm/i915/gt/intel_engine_cs.c +2 -39
@@ -1105,45 +1105,8 @@ static u32 read_subslice_reg(const struct intel_engine_cs *engine, int slice, int subslice, i915_reg_t reg) { - struct drm_i915_private *i915 = engine->i915; -struct intel_uncore *uncore = engine->uncore; -u32 mcr_mask, mcr_ss, mcr, old_mcr, val; -enum forcewake_domains fw_domains; - -if (GRAPHICS_VER (i915) >=11 ) { - mcr_mask =GEN11_MCR_SLICE_MASK |GEN11_MCR_SUBSLICE_MASK ; - mcr_ss =GEN11_MCR_SLICE (slice) |GEN11_MCR_SUBSLICE (subslice); - }else { - mcr_mask =GEN8_MCR_SLICE_MASK |GEN8_MCR_SUBSLICE_MASK ; - mcr_ss =GEN8_MCR_SLICE (slice) |GEN8_MCR_SUBSLICE (subslice); - } - - fw_domains =intel_uncore_forcewake_for_reg (uncore, reg, -FW_REG_READ ); - fw_domains |=intel_uncore_forcewake_for_reg (uncore, -GEN8_MCR_SELECTOR , -FW_REG_READ |FW_REG_WRITE ); - -spin_lock_irq (&uncore->lock); -intel_uncore_forcewake_get__locked (uncore, fw_domains); - - old_mcr = mcr =intel_uncore_read_fw (uncore,GEN8_MCR_SELECTOR ); - - mcr &= ~mcr_mask; - mcr |= mcr_ss; -intel_uncore_write_fw (uncore,GEN8_MCR_SELECTOR , mcr); - - val =intel_uncore_read_fw (uncore, reg); - - mcr &= ~mcr_mask; - mcr |= old_mcr & mcr_mask; - -intel_uncore_write_fw (uncore,GEN8_MCR_SELECTOR , mcr); - -intel_uncore_forcewake_put__locked (uncore, fw_domains); -spin_unlock_irq (&uncore->lock); - -return val; + return intel_uncore_read_with_mcr_steering(engine->uncore, reg, + slice, subslice); }/* NB: please notice the memset */
drivers/gpu/drm/i915/intel_uncore.c +55 -0
61 lines not on this page · raw message
drivers/gpu/drm/i915/intel_uncore.h +6 -0
@@ -182,6 +182,12 @@ intel_uncore_has_fifo(const struct intel_uncore *uncore)return uncore->flags &UNCORE_HAS_FIFO ; } +u32 intel_uncore_read_with_mcr_steering_fw (struct intel_uncore *uncore, +i915_reg_t reg, +int slice,int subslice); +u32 intel_uncore_read_with_mcr_steering (struct intel_uncore *uncore, +i915_reg_t reg,int slice,int subslice); +void intel_uncore_mmio_debug_init_early (struct intel_uncore_mmio_debug *mmio_debug); void intel_uncore_init_early(struct intel_uncore *uncore,
Signature · 6 lines
2.25.4 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v2 2/3 drm/i915: Add GT support for multiple types of multicast steering
In reply to Matt RoperAlthough most of our multicast registers are replicated per-subslice, we also have a small number of multicast registers that are replicated per-l3 bank instead. For both types of multicast registers we need to make sure we steer reads of these registers to a valid instance. Ideally we'd like to find a specific instance ID that would steer reads of either type of multicast register to a valid instance (i.e., not fused off and not powered down), but sometimes the combination of part-specific fusing and the additional restrictions imposed by Render Power Gating make it impossible to find any overlap between the set of valid subslices and valid l3 banks. This problem will become even more noticeable on our upcoming platforms since they will be adding additional types of multicast registers with new types of replication and rules for finding valid instances for reads.
To handle this we'll continue to pick a suitable subslice instance at driver startup and program this as the default (sliceid,subsliceid) setting in the steering control register (0xFDC). In cases where we need to read another type of multicast GT register, but the default subslice steering would not correspond to a valid instance, we'll explicitly re-steer the single read to a valid value, perform the read, and then reset the steering to it's "subslice" default.
This patch adds the general functionality to prepare for this explicit steering of other multicast register types. We'll plug L3 bank steering into this in the next patch, and then add additional types of multicast registers when the support for our next upcoming platform arrives.
v2:
- Use entry->end==0 as table terminator. (Rodrigo)
- Grab forcewake in wa_list_verify() now that we're using accessors that assume forcewake is already held.
--- drivers/gpu/drm/i915/gt/intel_gt.c | 84 +++++++++++++++++++ drivers/gpu/drm/i915/gt/intel_gt.h | 8 ++ drivers/gpu/drm/i915/gt/intel_gt_types.h | 22 +++++ drivers/gpu/drm/i915/gt/intel_workarounds.c | 39 ++++++--- .../gpu/drm/i915/gt/selftest_workarounds.c | 2 +- 5 files changed, 142 insertions(+), 13 deletions(-)
drivers/gpu/drm/i915/gt/intel_gt.c +84 -0
@@ -697,6 +697,90 @@ void intel_gt_driver_late_release(struct intel_gt *gt)intel_engines_free (gt); } +/** +* intel_gt_reg_needs_read_steering - determine whether a register read +* requires explicit steering +* @gt: GT structure +* @reg: the register to check steering requirements for +* @type: type of multicast steering to check +* +* Determines whether @reg needs explicit steering of a specific type for +* reads. +* +* Returns false if @reg does not belong to a register range of the given +* steering type, or if the default (subslice-based) steering IDs are suitable +* for @type steering too. +*/ +static bool intel_gt_reg_needs_read_steering (struct intel_gt *gt, +i915_reg_t reg, +enum intel_steering_type type) +{ +const u32 offset =i915_mmio_reg_offset (reg); +const struct intel_mmio_range *entry; + +if (likely (!intel_gt_needs_read_steering (gt, type))) +return false; + +for (entry = gt->steering_table[type]; !entry->end; entry++) { +if (offset >= entry->start && offset <= entry->end) +return true; + } + +return false; +} + +/** +* intel_gt_get_valid_steering - determines valid IDs for a class of MCR steering +* @gt: GT structure +* @type: multicast register type +* @sliceid: Slice ID returned +* @subsliceid: Subslice ID returned +* +* Determines sliceid and subsliceid values that will steer reads +* of a specific multicast register class to a valid value. +*/ +static void intel_gt_get_valid_steering (struct intel_gt *gt, +enum intel_steering_type type, +u8 *sliceid,u8 *subsliceid) +{ +switch (type) { +default : +MISSING_CASE (type); + *sliceid =0 ; + *subsliceid =0 ; + } +} + +/** +* intel_gt_read_register_fw - reads a GT register with support for multicast +* @gt: GT structure +* @reg: register to read +* +* This function will read a GT register. If the register is a multicast +* register, the read will be steered to a valid instance (i.e., one that +* isn't fused off or powered down by power gating). +* +* Returns the value from a valid instance of @reg. +*/ +u32 intel_gt_read_register_fw (struct intel_gt *gt,i915_reg_t reg) +{ +int type; +u8 sliceid, subsliceid; + +for (type =0 ; type <NUM_STEERING_TYPES ; type++) { +if (intel_gt_reg_needs_read_steering (gt, reg, type)) { +intel_gt_get_valid_steering (gt, type, &sliceid, + &subsliceid); +return intel_uncore_read_with_mcr_steering_fw (gt->uncore, + reg, + sliceid, + subsliceid); + } + } + +return intel_uncore_read_fw (gt->uncore, reg); +} +void intel_gt_info_print (const struct intel_gt_info *info,struct drm_printer *p) {
drivers/gpu/drm/i915/gt/intel_gt.h +8 -0
@@ -75,6 +75,14 @@ static inline bool intel_gt_is_wedged(const struct intel_gt *gt)return unlikely (test_bit (I915_WEDGED , >->reset.flags)); } +static inline bool intel_gt_needs_read_steering (struct intel_gt *gt, +enum intel_steering_type type) +{ +return gt->steering_table[type]; +} + +u32 intel_gt_read_register_fw (struct intel_gt *gt,i915_reg_t reg); +void intel_gt_info_print (const struct intel_gt_info *info,struct drm_printer *p);
drivers/gpu/drm/i915/gt/intel_gt_types.h +22 -0
@@ -31,6 +31,26 @@ struct i915_ggtt;struct intel_engine_cs ;struct intel_uncore ; +struct intel_mmio_range { +u32 start; +u32 end; +}; + +/* +* The hardware has multiple kinds of multicast register ranges that need +* special register steering (and future platforms are expected to add +* additional types). +* +* During driver startup, we initialize the steering control register to +* direct reads to a slice/subslice that are valid for the 'subslice' class +* of multicast registers. If another type of steering does not have any +* overlap in valid steering targets with 'subslice' style registers, we will +* need to explicitly re-steer reads of registers of the other type. +*/ +enum intel_steering_type { +NUM_STEERING_TYPES +}; +enum intel_submission_method {INTEL_SUBMISSION_RING ,INTEL_SUBMISSION_ELSP ,
@@ -145,6 +165,8 @@ struct intel_gt {struct i915_vma *scratch; +const struct intel_mmio_range *steering_table[NUM_STEERING_TYPES ]; +struct intel_gt_info {intel_engine_mask_t engine_mask;u8 num_engines;
drivers/gpu/drm/i915/gt/intel_workarounds.c +27 -12
@@ -1247,8 +1247,9 @@ wa_verify(const struct i915_wa *wa, u32 cur, const char *name, const char *from) }static void -wa_list_apply (struct intel_uncore *uncore,const struct i915_wa_list *wal) +wa_list_apply (struct intel_gt *gt,const struct i915_wa_list *wal) { +struct intel_uncore *uncore = gt->uncore;enum forcewake_domains fw;unsigned long flags;struct i915_wa *wa;
@@ -1263,13 +1264,16 @@ wa_list_apply(struct intel_uncore *uncore, const struct i915_wa_list *wal)intel_uncore_forcewake_get__locked (uncore, fw);for (i =0 , wa = wal->list; i < wal->count; i++, wa++) { -if (wa->clr) -intel_uncore_rmw_fw (uncore, wa->reg, wa->clr, wa->set); -else -intel_uncore_write_fw (uncore, wa->reg, wa->set); +u32 val, old =0 ; + +/* open-coded rmw due to steering */ + old = wa->clr ?intel_gt_read_register_fw (gt, wa->reg) :0 ; + val = (old & ~wa->clr) | wa->set; +if (val != old || !wa->clr) +intel_uncore_write_fw (uncore, wa->reg, val); +if (IS_ENABLED (CONFIG_DRM_I915_DEBUG_GEM )) -wa_verify (wa, -intel_uncore_read_fw (uncore, wa->reg), +wa_verify (wa,intel_gt_read_register_fw (gt, wa->reg), wal->name,"application" ); }
@@ -1279,28 +1283,39 @@ wa_list_apply(struct intel_uncore *uncore, const struct i915_wa_list *wal)void intel_gt_apply_workarounds (struct intel_gt *gt) { -wa_list_apply (gt->uncore, >->i915->gt_wa_list); +wa_list_apply (gt, >->i915->gt_wa_list); } -static bool wa_list_verify (struct intel_uncore *uncore, +static bool wa_list_verify (struct intel_gt *gt,const struct i915_wa_list *wal,const char *from) { +struct intel_uncore *uncore = gt->uncore;struct i915_wa *wa; +enum forcewake_domains fw; +unsigned long flags;unsigned int i;bool ok = true; + fw =wal_get_fw_for_rmw (uncore, wal); + +spin_lock_irqsave (&uncore->lock, flags); +intel_uncore_forcewake_get__locked (uncore, fw); +for (i =0 , wa = wal->list; i < wal->count; i++, wa++) ok &=wa_verify (wa, -intel_uncore_read (uncore, wa->reg), +intel_gt_read_register_fw (gt, wa->reg), wal->name, from); +intel_uncore_forcewake_put__locked (uncore, fw); +spin_unlock_irqrestore (&uncore->lock, flags); +return ok; }bool intel_gt_verify_workarounds (struct intel_gt *gt,const char *from) { -return wa_list_verify (gt->uncore, >->i915->gt_wa_list, from); +return wa_list_verify (gt, >->i915->gt_wa_list, from); }__maybe_unused
@@ -2081,7 +2096,7 @@ void intel_engine_init_workarounds(struct intel_engine_cs *engine)void intel_engine_apply_workarounds (struct intel_engine_cs *engine) { -wa_list_apply (engine->uncore, &engine->wa_list); +wa_list_apply (engine->gt, &engine->wa_list); }struct mcr_range {
drivers/gpu/drm/i915/gt/selftest_workarounds.c +1 -1
@@ -1147,7 +1147,7 @@ verify_wa_lists(struct intel_gt *gt, struct wa_lists *lists,enum intel_engine_id id;bool ok = true; - ok &=wa_list_verify (gt->uncore, &lists->gt_wa_list, str); + ok &=wa_list_verify (gt, &lists->gt_wa_list, str);for_each_engine (engine, gt, id) {struct intel_context *ce;
Signature · 6 lines
2.25.4 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v2 3/3 drm/i915: Add support for explicit L3BANK steering
In reply to Matt RoperBecause Render Power Gating restricts us to just a single subslice as a valid steering target for reads of multicast registers in a SUBSLICE range, the default steering we setup at init may not lead to a suitable target for L3BANK multicast register. In cases where it does not, use explicit runtime steering whenever an L3BANK multicast register is read.
While we're at it, let's simplify the function a little bit and drop its support for gen10/CNL since no such platforms ever materialized for real use. Multicast register steering is already an area that causes enough confusion; no need to complicate it with what's effectively dead code.
v2:
- Use gt->uncore instead of gt->i915->uncore. (Tvrtko)
- Use {} as table terminator. (Rodrigo)
--- drivers/gpu/drm/i915/gt/intel_gt.c | 18 +++++ drivers/gpu/drm/i915/gt/intel_gt_types.h | 4 + drivers/gpu/drm/i915/gt/intel_workarounds.c | 84 ++++++--------------- 3 files changed, 46 insertions(+), 60 deletions(-)
drivers/gpu/drm/i915/gt/intel_gt.c +18 -0
@@ -83,6 +83,11 @@ void intel_gt_init_hw_early(struct intel_gt *gt, struct i915_ggtt *ggtt) gt->ggtt = ggtt; } +static const struct intel_mmio_range icl_l3bank_steering_table[] = { + {0x00B100 ,0x00B3FF }, + {}, +}; +int intel_gt_init_mmio (struct intel_gt *gt) {intel_gt_init_clock_frequency (gt);
@@ -90,6 +95,13 @@ int intel_gt_init_mmio(struct intel_gt *gt)intel_uc_init_mmio (>->uc);intel_sseu_info_init (gt); +if (GRAPHICS_VER (gt->i915) >=11 ) { + gt->steering_table[L3BANK ] = icl_l3bank_steering_table; + gt->info.l3bank_mask = +intel_uncore_read (gt->uncore,GEN10_MIRROR_FUSE3 ) & +GEN10_L3BANK_MASK ; + } +return intel_engines_init_mmio (gt); }
@@ -744,6 +756,12 @@ static void intel_gt_get_valid_steering(struct intel_gt *gt, u8 *sliceid, u8 *subsliceid) {switch (type) { +case L3BANK : +GEM_DEBUG_WARN_ON (!gt->info.l3bank_mask);/* should be impossible! */ + + *sliceid =__ffs (gt->info.l3bank_mask); + *subsliceid =0 ;/* unused */ +break ;default :MISSING_CASE (type); *sliceid =0 ;
drivers/gpu/drm/i915/gt/intel_gt_types.h +4 -0
@@ -48,6 +48,8 @@ struct intel_mmio_range {* need to explicitly re-steer reads of registers of the other type. */ enum intel_steering_type { +L3BANK , +NUM_STEERING_TYPES };
@@ -174,6 +176,8 @@ struct intel_gt {/* Media engine access to SFC per instance */ u8 vdbox_sfc_access; +u32 l3bank_mask; +/* Slice/subslice/EU info */ struct sseu_dev_info sseu; } info;
drivers/gpu/drm/i915/gt/intel_workarounds.c +24 -60
117 lines not on this page · raw message
Signature · 6 lines
2.25.4 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Patchwork
CI[Intel-gfx] ✗ Fi.CI.BUILD: failure for Explicity steer l3bank multicast reads when necessary (rev2)
In reply to Matt Roper== Series Details ==
== Series Details ==
Series: Explicity steer l3bank multicast reads when necessary (rev2)
URL : https://patchwork.freedesktop.org/series/91485/
State : failure
== Summary ==
Applying: drm/i915: extract steered reg access to common function
Applying: drm/i915: Add GT support for multiple types of multicast steering
Applying: drm/i915: Add support for explicit L3BANK steering
error: sha1 information is lacking or useless (drivers/gpu/drm/i915/gt/intel_workarounds.c).
error: could not build fake ancestor
hint: Use 'git am --show-current-patch=diff' to see the failed patch
Patch failed at 0003 drm/i915: Add support for explicit L3BANK steering
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx