drm/xe/guc: Add GuC based register capture for error capture
Subsystems (2)
v19 cover drm/xe/guc: Add GuC based register capture for error capture
Port GuC based register capture for error capture from i915 to Xe.
There are 3 parts inside:
. Prepare for capture registers
There is a bo create at guc ads init time, that is very early and engine map is not ready, make it hard to calculate the capture buffer size, new function created for worst case size caluation. Other than that, this part basically follows the i915 design.
. Process capture notification message
Basically follows i915 design
. Sysfs command process.
Xe switched to devcoredump, adopted command line process with captured node list.
Changes from prior revs:
v19:- Avoid remove of locked node
v18:- Bug fix of steering needed bit not set for steering register
Move SFC_DONE register list insertion to patch 1.
Buf fix of missing engine class to guc class conversion Split plumbing GuC capture patch into 2 patches
Add matched_node pointer to remember the node, reduce search for node rate
v17:- Update steering register condition check to check if current gt has
rcs/ccs engine.
Add additional null check
Rollback patch #3, take back RB
v16:- Switch to single list of capture register define, remove MMIO registers
from snapshot structure.
Seperate register capture list for legacy GPUs
Rewrite 64bit register support method, add field to indicate hi/low dword of 64 bit or a single 32 bit register
Update the wrost size calculation method
v15:- Optimized guc log size code, remove the unnecessary init structure.
Fix a rebase line number alignment error
v14:- Fixed ring buffer wrap around offset issue
v13:- Move guc_mmio_reg structure define to guc_capture_abi.h
Remove duplicated crash/debug/capture unit check
Remove unnecessary guc_capture_data_extracted
Update u32 align check in guc_capture_log_remove_bytes
v12:- Rewrite guc log size init from runtime to compile time implementation.
Change log buffer flush to file from structure bitfield to genmask. Change the capture log data copy from u32 copy to size copy There are 3 types of engine class refrenced in this series, hw engine class, GuC class and GuC capture class, update function parameter type to enum for easy to read.
Update macro names to follow GuC interface specification.
v11:- Fixed a bug of missing captured check on register snapshot pre-capture
Fixed kernel-doc warnings
v10:- Resync with updated job timed out follow
Add pre-capture by read from hw engine if GuC capture data is not ready, the pre-captured data will be refereshed if GuC capture is ready at later time.
Add xe_guc_capture_is_ready_for to check if GuC capture is ready for a job.
Re-orgnize some header files to xe abi folder
Reduce some meesage level from warn/info to debug
Remove duplicated enum of GuC log type.
v9:- Merged snapshot register list into capture register lists
Optimized devcoredump timing to take snapshot after guc reset Add global and engine class registers into capture list Fixed bug of incorrect matching guc class id with guc capture class id
v8:- Reorgnize the order of patches
Change the capture size check from worst min size to worst size Replace the kernel alloc with drm managed alloc
Replace the memcpy with xe_map_memcpy_from
Free GuC capture outlist as part of xe_devcoredump_free
v7:- Kconfig CONFIG_DRM_XE_CAPTURE_ERROR removed
v6:- Change hardcoded register snapshot fill to follow mapping tables
When capture is empty, take snapshot from engine
v5:- Split dss helper code out as an standalone patch
Remove old platform registers definition.
Split register map table to 32 and 64bit each
v4:- Move register map table to xe_hw_engine.c v3:- Remove condition compilation in code v2:- Split into multiple chunks
Zhanjun Dong (6):
drm/xe/guc: Prepare GuC register list and update ADS size for error
capture
drm/xe/guc: Add XE_LP steered register lists
drm/xe/guc: Add capture size check in GuC log buffer drm/xe/guc: Extract GuC error capture lists
drm/xe/guc: Plumb GuC-capture into dev coredump drm/xe/guc: Save manual engine capture into capture list
drivers/gpu/drm/xe/Makefile | 1 + drivers/gpu/drm/xe/abi/guc_actions_abi.h | 8 + drivers/gpu/drm/xe/abi/guc_capture_abi.h | 186 ++ drivers/gpu/drm/xe/abi/guc_log_abi.h | 75 + drivers/gpu/drm/xe/regs/xe_gt_regs.h | 2 + drivers/gpu/drm/xe/xe_devcoredump.c | 20 +- drivers/gpu/drm/xe/xe_devcoredump_types.h | 8 + drivers/gpu/drm/xe/xe_gt_mcr.c | 13 + drivers/gpu/drm/xe/xe_gt_mcr.h | 1 + drivers/gpu/drm/xe/xe_guc.c | 5 + drivers/gpu/drm/xe/xe_guc.h | 5 + drivers/gpu/drm/xe/xe_guc_ads.c | 157 +- drivers/gpu/drm/xe/xe_guc_ads_types.h | 2 + drivers/gpu/drm/xe/xe_guc_capture.c | 1938 +++++++++++++++++++++ drivers/gpu/drm/xe/xe_guc_capture.h | 61 + drivers/gpu/drm/xe/xe_guc_capture_types.h | 68 + drivers/gpu/drm/xe/xe_guc_ct.c | 2 + drivers/gpu/drm/xe/xe_guc_fwif.h | 26 +- drivers/gpu/drm/xe/xe_guc_log.c | 101 ++ drivers/gpu/drm/xe/xe_guc_log.h | 10 +- drivers/gpu/drm/xe/xe_guc_log_types.h | 7 + drivers/gpu/drm/xe/xe_guc_submit.c | 83 +- drivers/gpu/drm/xe/xe_guc_submit.h | 2 + drivers/gpu/drm/xe/xe_guc_types.h | 2 + drivers/gpu/drm/xe/xe_hw_engine.c | 250 +-- drivers/gpu/drm/xe/xe_hw_engine.h | 6 +- drivers/gpu/drm/xe/xe_hw_engine_types.h | 66 +- drivers/gpu/drm/xe/xe_lrc.c | 18 - drivers/gpu/drm/xe/xe_lrc.h | 19 +- 29 files changed, 2763 insertions(+), 379 deletions(-) create mode 100644 drivers/gpu/drm/xe/abi/guc_capture_abi.h create mode 100644 drivers/gpu/drm/xe/abi/guc_log_abi.h create mode 100644 drivers/gpu/drm/xe/xe_guc_capture.c create mode 100644 drivers/gpu/drm/xe/xe_guc_capture.h create mode 100644 drivers/gpu/drm/xe/xe_guc_capture_types.h
Signature · 1 line
2.34.1
PATCH v19 1/6 drm/xe/guc: Prepare GuC register list and update ADS size for error capture
Add referenced registers defines and list of registers. Update GuC ADS size allocation to include space for the lists of error state capture register descriptors.
Then, populate GuC ADS with the lists of registers we want GuC to report back to host on engine reset events. This list should include global, engine-class and engine-instance registers for every engine-class type on the current hardware.
Ensure we allocate a persistent storage for the register lists that are populated into ADS so that we don't need to allocate memory during GT resets when GuC is reloaded and ADS population happens again.
--- drivers/gpu/drm/xe/Makefile | 1 + drivers/gpu/drm/xe/abi/guc_capture_abi.h | 186 ++++++++ drivers/gpu/drm/xe/regs/xe_gt_regs.h | 2 + drivers/gpu/drm/xe/xe_guc.c | 5 + drivers/gpu/drm/xe/xe_guc.h | 5 + drivers/gpu/drm/xe/xe_guc_ads.c | 151 ++++++- drivers/gpu/drm/xe/xe_guc_ads_types.h | 2 + drivers/gpu/drm/xe/xe_guc_capture.c | 520 ++++++++++++++++++++++ drivers/gpu/drm/xe/xe_guc_capture.h | 48 ++ drivers/gpu/drm/xe/xe_guc_capture_types.h | 68 +++ drivers/gpu/drm/xe/xe_guc_fwif.h | 26 +- drivers/gpu/drm/xe/xe_guc_types.h | 2 + 12 files changed, 979 insertions(+), 37 deletions(-) create mode 100644 drivers/gpu/drm/xe/abi/guc_capture_abi.h create mode 100644 drivers/gpu/drm/xe/xe_guc_capture.c create mode 100644 drivers/gpu/drm/xe/xe_guc_capture.h create mode 100644 drivers/gpu/drm/xe/xe_guc_capture_types.h
drivers/gpu/drm/xe/Makefile +1 -0
@@ -56,6 +56,7 @@ xe-y += xe_bb.o \ xe_gt_topology.o \ xe_guc.o \ xe_guc_ads.o \ + xe_guc_capture.o \ xe_guc_ct.o \ xe_guc_db_mgr.o \ xe_guc_hwconfig.o \
drivers/gpu/drm/xe/abi/guc_capture_abi.h new file +186 -0
186 lines not on this page · raw message
drivers/gpu/drm/xe/regs/xe_gt_regs.h +2 -0
@@ -613,4 +613,6 @@#define GT_CS_MASTER_ERROR_INTERRUPT REG_BIT(3)#define GT_RENDER_USER_INTERRUPT REG_BIT(0) +#define SFC_DONE (n) XE_REG(0x1cc000 + (n) * 0x1000) +#endif
drivers/gpu/drm/xe/xe_guc.c +5 -0
@@ -22,6 +22,7 @@#include "xe_gt_sriov_vf.h" #include "xe_gt_throttle.h" #include "xe_guc_ads.h" +#include "xe_guc_capture.h" #include "xe_guc_ct.h" #include "xe_guc_db_mgr.h" #include "xe_guc_hwconfig.h"
@@ -338,6 +339,10 @@ int xe_guc_init(struct xe_guc *guc)if (ret) goto out; + ret =xe_guc_capture_init (guc); +if (ret) + goto out; + ret =xe_guc_ads_init (&guc->ads);if (ret) goto out;
drivers/gpu/drm/xe/xe_guc.h +5 -0
@@ -80,4 +80,9 @@ static inline struct xe_device *guc_to_xe(struct xe_guc *guc) return gt_to_xe(guc_to_gt(guc)); } +static inline struct drm_device *guc_to_drm (struct xe_guc *guc) +{ +return &guc_to_xe (guc)->drm; +} +#endif
drivers/gpu/drm/xe/xe_guc_ads.c +138 -13
200 lines not on this page · raw message
drivers/gpu/drm/xe/xe_guc_ads_types.h +2 -0
@@ -22,6 +22,8 @@ struct xe_guc_ads {u32 regset_size;/** @ads_waklv_size: total waklv size supported by platform */ u32 ads_waklv_size; +/** @capture_size: size of register set passed to GuC for capture */ + u32 capture_size; };#endif
drivers/gpu/drm/xe/xe_guc_capture.c new file +520 -0
520 lines not on this page · raw message
drivers/gpu/drm/xe/xe_guc_capture.h new file +48 -0
48 lines not on this page · raw message
drivers/gpu/drm/xe/xe_guc_capture_types.h new file +68 -0
68 lines not on this page · raw message
drivers/gpu/drm/xe/xe_guc_fwif.h +2 -24
@@ -8,7 +8,9 @@#include <linux/bits.h> +#include "abi/guc_capture_abi.h" #include "abi/guc_klvs_abi.h" +#include "xe_hw_engine_types.h" #define G2H_LEN_DW_SCHED_CONTEXT_MODE_SET 4#define G2H_LEN_DW_DEREGISTER_CONTEXT 3
@@ -157,24 +159,6 @@ struct guc_policies {u32 reserved[4 ]; } __packed; -/* GuC MMIO reg state struct */ -struct guc_mmio_reg { -u32 offset; -u32 value; -u32 flags; -u32 mask; -#define GUC_REGSET_MASKED BIT(0) -#define GUC_REGSET_MASKED_WITH_VALUE BIT(2) -#define GUC_REGSET_RESTORE_ONLY BIT(3) -} __packed; - -/* GuC register sets */ -struct guc_mmio_reg_set { -u32 address; -u16 count; -u16 reserved; -} __packed; -/* Generic GT SysInfo data types */ #define GUC_GENERIC_GT_SYSINFO_SLICE_ENABLED 0#define GUC_GENERIC_GT_SYSINFO_VDBOX_SFC_SUPPORT_MASK 1
@@ -188,12 +172,6 @@ struct guc_gt_system_info {u32 generic_gt_sysinfo[GUC_GENERIC_GT_SYSINFO_MAX ]; } __packed; -enum { -GUC_CAPTURE_LIST_INDEX_PF =0 , -GUC_CAPTURE_LIST_INDEX_VF =1 , -GUC_CAPTURE_LIST_INDEX_MAX =2 , -}; -/* GuC Additional Data Struct */ struct guc_ads {struct guc_mmio_reg_set reg_state_list[GUC_MAX_ENGINE_CLASSES ][GUC_MAX_INSTANCES_PER_CLASS ];
drivers/gpu/drm/xe/xe_guc_types.h +2 -0
@@ -58,6 +58,8 @@ struct xe_guc {struct xe_guc_ads ads;/** @ct: GuC ct */ struct xe_guc_ct ct; +/** @capture: the error-state-capture module's data and objects */ +struct xe_guc_state_capture *capture;/** @pc: GuC Power Conservation */ struct xe_guc_pc pc;/** @dbm: GuC Doorbell Manager */
Signature · 1 line
2.34.1
PATCH v19 2/6 drm/xe/guc: Add XE_LP steered register lists
In reply to Zhanjun DongAdd the ability for runtime allocation and freeing of steered register list extentions that depend on the detected HW config fuses.
--- drivers/gpu/drm/xe/xe_guc_ads.c | 6 + drivers/gpu/drm/xe/xe_guc_capture.c | 212 ++++++++++++++++++++++++++-- drivers/gpu/drm/xe/xe_guc_capture.h | 1 + 3 files changed, 206 insertions(+), 13 deletions(-)
drivers/gpu/drm/xe/xe_guc_ads.c +6 -0
@@ -596,6 +596,12 @@ static int guc_capture_prep_lists(struct xe_guc_ads *ads)void *ptr;int i, j; +/* +* GuC Capture's steered reg-list needs to be allocated and initialized +* after the GuC-hwconfig is available which guaranteed from here. +*/ +xe_guc_capture_steered_list_init (ads_to_guc (ads)); + capture_offset =guc_ads_capture_offset (ads); ads_ggtt =xe_bo_ggtt_addr (ads->bo); info_map = IOSYS_MAP_INIT_OFFSET(ads_to_map(ads),
drivers/gpu/drm/xe/xe_guc_capture.c +199 -13
260 lines not on this page · raw message
drivers/gpu/drm/xe/xe_guc_capture.h +1 -0
@@ -43,6 +43,7 @@ int xe_guc_capture_getlistsize(struct xe_guc *guc, u32 owner, u32 type, enum guc_capture_list_class_type capture_class, size_t *size);int xe_guc_capture_getnullheader (struct xe_guc *guc,void **outptr,size_t *size);size_t xe_guc_capture_ads_input_worst_size (struct xe_guc *guc); +void xe_guc_capture_steered_list_init (struct xe_guc *guc);int xe_guc_capture_init (struct xe_guc *guc);#endif
Signature · 1 line
2.34.1
PATCH v19 3/6 drm/xe/guc: Add capture size check in GuC log buffer
In reply to Zhanjun DongCapture-nodes generated by GuC are placed in the GuC capture ring buffer which is a sub-region of the larger Guc-Log-buffer. Add capture output size check before allocating the shared buffer.
--- drivers/gpu/drm/xe/abi/guc_log_abi.h | 20 +++++++ drivers/gpu/drm/xe/xe_guc_capture.c | 83 +++++++++++++++++++++++++++- drivers/gpu/drm/xe/xe_guc_log.c | 65 ++++++++++++++++++++++ drivers/gpu/drm/xe/xe_guc_log.h | 7 ++- 4 files changed, 173 insertions(+), 2 deletions(-) create mode 100644 drivers/gpu/drm/xe/abi/guc_log_abi.h
drivers/gpu/drm/xe/abi/guc_log_abi.h new file +20 -0
@@ -0,0 +1,20 @@ +/* SPDX-License-Identifier: MIT */ +/* +* Copyright © 2024 Intel Corporation +*/ + +#ifndef _ABI_GUC_LOG_ABI_H +#define _ABI_GUC_LOG_ABI_H + +#include <linux/types.h> + +/* GuC logging buffer types */ +enum guc_log_buffer_type { +GUC_LOG_BUFFER_CRASH_DUMP , +GUC_LOG_BUFFER_DEBUG , +GUC_LOG_BUFFER_CAPTURE , +}; + +#define GUC_LOG_BUFFER_TYPE_MAX 3 + +#endif
drivers/gpu/drm/xe/xe_guc_capture.c +82 -1
103 lines not on this page · raw message
drivers/gpu/drm/xe/xe_guc_log.c +65 -0
68 lines not on this page · raw message
drivers/gpu/drm/xe/xe_guc_log.h +6 -1
@@ -7,6 +7,7 @@#define _XE_GUC_LOG_H_#include "xe_guc_log_types.h" +#include "abi/guc_log_abi.h" struct drm_printer ;
@@ -17,7 +18,7 @@ struct drm_printer;#else #define CRASH_BUFFER_SIZE SZ_8K#define DEBUG_BUFFER_SIZE SZ_64K -#define CAPTURE_BUFFER_SIZE SZ_16K +#define CAPTURE_BUFFER_SIZE SZ_1M#endif /* * While we're using plain log level in i915, GuC controls are much more...
@@ -45,4 +46,8 @@ xe_guc_log_get_level(struct xe_guc_log *log) return log->level; } +u32 xe_guc_log_section_size_capture (struct xe_guc_log *log); +u32 xe_guc_get_log_buffer_size (struct xe_guc_log *log,enum guc_log_buffer_type type); +u32 xe_guc_get_log_buffer_offset (struct xe_guc_log *log,enum guc_log_buffer_type type); +#endif
Signature · 1 line
2.34.1
PATCH v19 4/6 drm/xe/guc: Extract GuC error capture lists
In reply to Zhanjun DongUpon the G2H Notify-Err-Capture event, parse through the GuC Log Buffer (error-capture-subregion) and generate one or more capture-nodes. A single node represents a single "engine- instance-capture-dump" and contains at least 3 register lists:
global, engine-class and engine-instance. An internal link list is maintained to store one or more nodes.
Because the link-list node generation happen before the call to devcoredump, duplicate global and engine-class register lists for each engine-instance register dump if we find dependent-engine resets in a engine-capture-group.
To avoid dynamically allocate the output nodes during gt reset, pre-allocate a fixed number of empty nodes up front (at the time of ADS registration) that we can consume from or return to an internal cached list of nodes.
--- drivers/gpu/drm/xe/abi/guc_actions_abi.h | 8 + drivers/gpu/drm/xe/abi/guc_log_abi.h | 55 ++ drivers/gpu/drm/xe/xe_guc_capture.c | 702 ++++++++++++++++++++++- drivers/gpu/drm/xe/xe_guc_capture.h | 1 + drivers/gpu/drm/xe/xe_guc_ct.c | 2 + drivers/gpu/drm/xe/xe_guc_log.c | 36 ++ drivers/gpu/drm/xe/xe_guc_log.h | 3 + drivers/gpu/drm/xe/xe_guc_log_types.h | 7 + drivers/gpu/drm/xe/xe_guc_submit.c | 65 ++- drivers/gpu/drm/xe/xe_guc_submit.h | 2 + 10 files changed, 863 insertions(+), 18 deletions(-)
drivers/gpu/drm/xe/abi/guc_actions_abi.h +8 -0
@@ -176,6 +176,14 @@ enum xe_guc_sleep_state_status {#define GUC_LOG_CONTROL_VERBOSITY_MASK (0xF << GUC_LOG_CONTROL_VERBOSITY_SHIFT)#define GUC_LOG_CONTROL_DEFAULT_LOGGING (1 << 8) +enum xe_guc_state_capture_event_status { +XE_GUC_STATE_CAPTURE_EVENT_STATUS_SUCCESS =0x0 , +XE_GUC_STATE_CAPTURE_EVENT_STATUS_NOSPACE =0x1 , +}; + +#define XE_GUC_STATE_CAPTURE_EVENT_STATUS_MASK 0x000000FF +#define XE_GUC_ACTION_STATE_CAPTURE_NOTIFICATION_DATA_LEN 1 +#define XE_GUC_TLB_INVAL_TYPE_SHIFT 0#define XE_GUC_TLB_INVAL_MODE_SHIFT 8/* Flush PPC or SMRO caches along with TLB invalidation request */
drivers/gpu/drm/xe/abi/guc_log_abi.h +55 -0
59 lines not on this page · raw message
drivers/gpu/drm/xe/xe_guc_capture.c +701 -1
741 lines not on this page · raw message
drivers/gpu/drm/xe/xe_guc_capture.h +1 -0
@@ -37,6 +37,7 @@ xe_engine_class_to_guc_capture_class(enum xe_engine_class class)return xe_guc_class_to_capture_class (xe_engine_class_to_guc_class (class)); } +void xe_guc_capture_process (struct xe_guc *guc);int xe_guc_capture_getlist (struct xe_guc *guc,u32 owner,u32 type,enum guc_capture_list_class_type capture_class,void **outptr); int xe_guc_capture_getlistsize(struct xe_guc *guc, u32 owner, u32 type,
drivers/gpu/drm/xe/xe_guc_ct.c +2 -0
@@ -1123,6 +1123,8 @@ static int process_g2h_msg(struct xe_guc_ct *ct, u32 *msg, u32 len)/* Selftest only at the moment */ break ;case XE_GUC_ACTION_STATE_CAPTURE_NOTIFICATION : + ret =xe_guc_error_capture_handler (guc, payload, adj_len); +break ;case XE_GUC_ACTION_NOTIFY_FLUSH_LOG_BUFFER_TO_FILE :/* FIXME: Handle this */ break ;
drivers/gpu/drm/xe/xe_guc_log.c +36 -0
@@ -9,6 +9,7 @@#include "xe_bo.h" #include "xe_gt.h" +#include "xe_gt_printk.h" #include "xe_map.h" #include "xe_module.h"
@@ -161,3 +162,38 @@ u32 xe_guc_get_log_buffer_offset(struct xe_guc_log *log, enum guc_log_buffer_typreturn offset; } + +/** +* xe_guc_check_log_buf_overflow - Check if log buffer overflowed +* @log: The log object. +* @type: The log buffer type +* @full_cnt: The count of buffer full +* +* This function will check count of buffer full against previous, mismatch +* indicate overflowed. +* Update the sampled_overflow counter, if the 4 bit counter overflowed, add +* up 16 to correct the value. +* +* Return: True if overflowed. +*/ +bool xe_guc_check_log_buf_overflow (struct xe_guc_log *log,enum guc_log_buffer_type type, +unsigned int full_cnt) +{ +unsigned int prev_full_cnt = log->stats[type].sampled_overflow; +bool overflow = false; + +if (full_cnt != prev_full_cnt) { + overflow = true; + + log->stats[type].overflow = full_cnt; + log->stats[type].sampled_overflow += full_cnt - prev_full_cnt; + +if (full_cnt < prev_full_cnt) { +/* buffer_full_cnt is a 4 bit counter */ + log->stats[type].sampled_overflow +=16 ; + } +xe_gt_notice (log_to_gt (log),"log buffer overflow\n" ); + } + +return overflow; +}
drivers/gpu/drm/xe/xe_guc_log.h +3 -0
@@ -49,5 +49,8 @@ xe_guc_log_get_level(struct xe_guc_log *log)u32 xe_guc_log_section_size_capture (struct xe_guc_log *log);u32 xe_guc_get_log_buffer_size (struct xe_guc_log *log,enum guc_log_buffer_type type);u32 xe_guc_get_log_buffer_offset (struct xe_guc_log *log,enum guc_log_buffer_type type); +bool xe_guc_check_log_buf_overflow (struct xe_guc_log *log, +enum guc_log_buffer_type type, +unsigned int full_cnt);#endif
drivers/gpu/drm/xe/xe_guc_log_types.h +7 -0
@@ -7,6 +7,7 @@#define _XE_GUC_LOG_TYPES_H_#include <linux/types.h> +#include "abi/guc_log_abi.h" struct xe_bo ;
@@ -18,6 +19,12 @@ struct xe_guc_log {u32 level;/** @bo: XE BO for GuC log */ struct xe_bo *bo; +/** @stats: logging related stats */ +struct { +u32 sampled_overflow; +u32 overflow; +u32 flush; + } stats[GUC_LOG_BUFFER_TYPE_MAX]; };#endif
drivers/gpu/drm/xe/xe_guc_submit.c +48 -17
156 lines not on this page · raw message
drivers/gpu/drm/xe/xe_guc_submit.h +2 -0
@@ -20,12 +20,14 @@ void xe_guc_submit_stop(struct xe_guc *guc);int xe_guc_submit_start (struct xe_guc *guc);void xe_guc_submit_wedge (struct xe_guc *guc); +int xe_guc_read_stopped (struct xe_guc *guc);int xe_guc_sched_done_handler (struct xe_guc *guc,u32 *msg,u32 len);int xe_guc_deregister_done_handler (struct xe_guc *guc,u32 *msg,u32 len);int xe_guc_exec_queue_reset_handler (struct xe_guc *guc,u32 *msg,u32 len);int xe_guc_exec_queue_memory_cat_error_handler (struct xe_guc *guc,u32 *msg,u32 len);int xe_guc_exec_queue_reset_failure_handler (struct xe_guc *guc,u32 *msg,u32 len); +int xe_guc_error_capture_handler (struct xe_guc *guc,u32 *msg,u32 len);struct xe_guc_submit_exec_queue_snapshot *xe_guc_exec_queue_snapshot_capture (struct xe_exec_queue *q);
Signature · 1 line
2.34.1
PATCH v19 5/6 drm/xe/guc: Plumb GuC-capture into dev coredump
In reply to Zhanjun DongWhen we decide to kill a job, (from guc_exec_queue_timedout_job), we could end up with 4 possible scenarios at this starting point of this decision:
1. the guc-captured register-dump is already there.
2. the driver is wedged.mode > 1, so GuC-engine-reset / GuC-err-capture will not happen.
3. the user has started the driver in execlist-submission mode.
4. the guc-captured register-dump is not ready yet so we force GuC to kill that context now, but:
A. we don't know yet if GuC will be successful on the engine-reset
and get the guc-err-capture, else kmd will do a manual reset later
OR B. guc will be successful and we will get a guc-err-capture
shortly.
So to accomdate the scenarios of 2 and 4A, we will need to do a manual KMD capture first(which is not be reliable in guc-submission mode) and decide later if we need to use that for the cases of 2 or 4A. So this flow is part of the implementation for this patch.
Provide xe_guc_capture_get_reg_desc_list to get the register dscriptor list.
Add manual capture by read from hw engine if GuC capture is not ready. If it becomes ready at later time, GuC sourced data will be used.
Although there may only be a small delay between (1) the check for whether guc-err-capture is available at the start of guc_exec_queue_timedout_job and (2) the decision on using a valid guc-err-capture or manual-capture, lets not take any chances and lock the matching node down so it doesn't get re-claimed if GuC-Err-Capture subsystem is running out of pre-cached nodes.
--- drivers/gpu/drm/xe/xe_devcoredump.c | 17 +- drivers/gpu/drm/xe/xe_devcoredump_types.h | 8 + drivers/gpu/drm/xe/xe_gt_mcr.c | 13 + drivers/gpu/drm/xe/xe_gt_mcr.h | 1 + drivers/gpu/drm/xe/xe_guc_capture.c | 322 +++++++++++++++++++++- drivers/gpu/drm/xe/xe_guc_capture.h | 10 + drivers/gpu/drm/xe/xe_guc_submit.c | 20 +- drivers/gpu/drm/xe/xe_hw_engine.c | 165 ++++++----- drivers/gpu/drm/xe/xe_hw_engine.h | 4 +- drivers/gpu/drm/xe/xe_hw_engine_types.h | 7 + drivers/gpu/drm/xe/xe_lrc.c | 18 -- drivers/gpu/drm/xe/xe_lrc.h | 19 +- 12 files changed, 501 insertions(+), 103 deletions(-)
drivers/gpu/drm/xe/xe_devcoredump.c +7 -10
@@ -16,6 +16,7 @@#include "xe_force_wake.h" #include "xe_gt.h" #include "xe_gt_printk.h" +#include "xe_guc_capture.h" #include "xe_guc_ct.h" #include "xe_guc_submit.h" #include "xe_hw_engine.h"
@@ -121,6 +122,9 @@ static void xe_devcoredump_snapshot_free(struct xe_devcoredump_snapshot *ss)xe_guc_ct_snapshot_free (ss->ct); ss->ct =NULL ; +xe_guc_capture_free (&ss->gt->uc.guc); + ss->matched_node =NULL ; +xe_guc_exec_queue_snapshot_free (ss->ge); ss->ge =NULL ;
@@ -204,6 +208,7 @@ static void xe_devcoredump_free(void *data)/* To prevent stale data on next snapshot, clear everything */ memset (&coredump->snapshot,0 ,sizeof (coredump->snapshot)); coredump->captured = false; + coredump->job =NULL ;drm_info (&coredump_to_xe (coredump)->drm,"Xe device coredump has been deleted.\n" ); }
@@ -214,8 +219,6 @@ static void devcoredump_snapshot(struct xe_devcoredump *coredump,struct xe_devcoredump_snapshot *ss = &coredump->snapshot;struct xe_exec_queue *q = job->q;struct xe_guc *guc =exec_queue_to_guc (q); -struct xe_hw_engine *hwe; -enum xe_hw_engine_id id;u32 adj_logical_mask = q->logical_mask;u32 width_mask = (0x1 << q->width) -1 ;const char *process_name ="no process" ;
@@ -231,6 +234,7 @@ static void devcoredump_snapshot(struct xe_devcoredump *coredump,strscpy (ss->process_name, process_name); ss->gt = q->gt; + coredump->job = job;INIT_WORK (&ss->work, xe_devcoredump_deferred_snap_work); cookie =dma_fence_begin_signalling ();
@@ -252,14 +256,7 @@ static void devcoredump_snapshot(struct xe_devcoredump *coredump, coredump->snapshot.job =xe_sched_job_snapshot_capture (job); coredump->snapshot.vm =xe_vm_snapshot_capture (q->vm); -for_each_hw_engine (hwe, q->gt, id) { -if (hwe->class != q->hwe->class || - !(BIT (hwe->logical_instance) & adj_logical_mask)) { - coredump->snapshot.hwe[id] =NULL ; -continue ; - } - coredump->snapshot.hwe[id] =xe_hw_engine_snapshot_capture (hwe); - } +xe_engine_snapshot_capture_for_job (job);queue_work (system_unbound_wq, &ss->work);
drivers/gpu/drm/xe/xe_devcoredump_types.h +8 -0
@@ -44,6 +44,12 @@ struct xe_devcoredump_snapshot {struct xe_hw_engine_snapshot *hwe[XE_NUM_HW_ENGINES ];/** @job: Snapshot of job state */ struct xe_sched_job_snapshot *job; +/** +* @matched_node: The matched capture node for timedout job +* this single-node tracker works because devcoredump will always only +* produce one hw-engine capture per devcoredump event +*/ +struct __guc_capture_parsed_output *matched_node;/** @vm: Snapshot of VM state */ struct xe_vm_snapshot *vm;
@@ -69,6 +75,8 @@ struct xe_devcoredump {bool captured;/** @snapshot: Snapshot is captured at time of the first crash */ struct xe_devcoredump_snapshot snapshot; +/** @job: Point to the faulting job */ + struct xe_sched_job *job; };#endif
drivers/gpu/drm/xe/xe_gt_mcr.c +13 -0
@@ -352,6 +352,19 @@ void xe_gt_mcr_get_dss_steering(struct xe_gt *gt, unsigned int dss, u16 *group, *instance = dss % gt->steering_dss_per_grp; } +/** +* xe_gt_mcr_steering_info_to_dss_id - Get DSS ID from group/instance steering +* @gt: GT structure +* @group: steering group ID +* @instance: steering instance ID +* +* Return: the coverted DSS id. +*/ +u32 xe_gt_mcr_steering_info_to_dss_id (struct xe_gt *gt,u16 group,u16 instance) +{ +return group *dss_per_group (gt) + instance; +} +static void init_steering_dss (struct xe_gt *gt) { gt->steering_dss_per_grp =dss_per_group (gt);
drivers/gpu/drm/xe/xe_gt_mcr.h +1 -0
@@ -28,6 +28,7 @@ void xe_gt_mcr_multicast_write(struct xe_gt *gt, struct xe_reg_mcr mcr_reg,void xe_gt_mcr_steering_dump (struct xe_gt *gt,struct drm_printer *p);void xe_gt_mcr_get_dss_steering (struct xe_gt *gt,unsigned int dss,u16 *group,u16 *instance); +u32 xe_gt_mcr_steering_info_to_dss_id (struct xe_gt *gt,u16 group,u16 instance);/* * Loop over each DSS and determine the group and instance IDs that
drivers/gpu/drm/xe/xe_guc_capture.c +318 -4
387 lines not on this page · raw message
drivers/gpu/drm/xe/xe_guc_capture.h +10 -0
@@ -12,6 +12,9 @@#include "xe_guc_fwif.h" struct xe_guc ; +struct xe_hw_engine ; +struct xe_hw_engine_snapshot ; +struct xe_sched_job ;static inline enum guc_capture_list_class_type xe_guc_class_to_capture_class (u16 class) {
@@ -44,7 +47,14 @@ int xe_guc_capture_getlistsize(struct xe_guc *guc, u32 owner, u32 type, enum guc_capture_list_class_type capture_class, size_t *size);int xe_guc_capture_getnullheader (struct xe_guc *guc,void **outptr,size_t *size);size_t xe_guc_capture_ads_input_worst_size (struct xe_guc *guc); +const struct __guc_mmio_reg_descr_group * +xe_guc_capture_get_reg_desc_list (struct xe_gt *gt,u32 owner,u32 type, +enum guc_capture_list_class_type capture_class,bool is_ext); +struct __guc_capture_parsed_output *xe_guc_capture_get_matching_and_lock (struct xe_sched_job *job); +void xe_engine_snapshot_capture_for_job (struct xe_sched_job *job); +void xe_engine_guc_capture_print (struct xe_hw_engine_snapshot *snapshot,struct drm_printer *p);void xe_guc_capture_steered_list_init (struct xe_guc *guc); +void xe_guc_capture_free (struct xe_guc *guc);int xe_guc_capture_init (struct xe_guc *guc);#endif
drivers/gpu/drm/xe/xe_guc_submit.c +17 -3
@@ -1073,6 +1073,7 @@ guc_exec_queue_timedout_job(struct drm_sched_job *drm_job)struct xe_gpu_scheduler *sched = &q->guc->sched;struct xe_guc *guc =exec_queue_to_guc (q);const char *process_name ="no process" ; +struct xe_device *xe =guc_to_xe (guc);int err = -ETIME ;pid_t pid =-1 ;int i =0 ;
@@ -1100,6 +1101,21 @@ guc_exec_queue_timedout_job(struct drm_sched_job *drm_job)if (!skip_timeout_check && !xe_sched_job_started (job)) goto rearm; +/* +* If devcoredump not captured and GuC capture for the job is not ready +* do manual capture first and decide later if we need to use it +*/ +if (!exec_queue_killed (q) && !xe->devcoredump.captured && + !xe_guc_capture_get_matching_and_lock (job)) { +/* take force wake before engine register manual capture */ +if (xe_force_wake_get (gt_to_fw (q->gt),XE_FORCEWAKE_ALL )) +xe_gt_info (q->gt,"failed to get forcewake for coredump capture\n" ); + +xe_engine_snapshot_capture_for_job (job); + +xe_force_wake_put (gt_to_fw (q->gt),XE_FORCEWAKE_ALL ); + } +/* * XXX: Sampling timeout doesn't work in wedged mode as we have to * modify scheduling state to read timestamp. We could read the
@@ -1979,8 +1995,6 @@ int xe_guc_exec_queue_reset_handler(struct xe_guc *guc, u32 *msg, u32 len)xe_gt_info (gt,"Engine reset: engine_class=%s, logical_mask: 0x%x, guc_id=%d" ,xe_hw_engine_class_to_str (q->class), q->logical_mask, guc_id); -/* FIXME: Do error capture, most likely async */ -trace_xe_exec_queue_reset (q);/*
@@ -2006,7 +2020,7 @@ int xe_guc_exec_queue_reset_handler(struct xe_guc *guc, u32 *msg, u32 len)* XE_GUC_ACTION_STATE_CAPTURE_NOTIFICATION to host, this function will be * called 1st to check status before process the data comes with the message. * -* Returns: None +* Returns: error code. 0 if success */ int xe_guc_error_capture_handler (struct xe_guc *guc,u32 *msg,u32 len) {
drivers/gpu/drm/xe/xe_hw_engine.c +100 -65
216 lines not on this page · raw message
drivers/gpu/drm/xe/xe_hw_engine.h +2 -2
@@ -11,6 +11,7 @@struct drm_printer ;struct drm_xe_engine_class_instance ;struct xe_device ; +struct xe_sched_job ;#ifdef CONFIG_DRM_XE_JOB_TIMEOUT_MIN #define XE_HW_ENGINE_JOB_TIMEOUT_MIN CONFIG_DRM_XE_JOB_TIMEOUT_MIN
@@ -54,9 +55,8 @@ void xe_hw_engine_handle_irq(struct xe_hw_engine *hwe, u16 intr_vec);void xe_hw_engine_enable_ring (struct xe_hw_engine *hwe);u32 xe_hw_engine_mask_per_class (struct xe_gt *gt,enum xe_engine_class engine_class); -struct xe_hw_engine_snapshot * -xe_hw_engine_snapshot_capture (struct xe_hw_engine *hwe); +xe_hw_engine_snapshot_capture (struct xe_hw_engine *hwe,struct xe_sched_job *job);void xe_hw_engine_snapshot_free (struct xe_hw_engine_snapshot *snapshot);void xe_hw_engine_snapshot_print (struct xe_hw_engine_snapshot *snapshot,struct drm_printer *p);
drivers/gpu/drm/xe/xe_hw_engine_types.h +7 -0
@@ -152,6 +152,11 @@ struct xe_hw_engine {struct xe_hw_engine_group *hw_engine_group; }; +enum xe_hw_engine_snapshot_source_id { +XE_ENGINE_CAPTURE_SOURCE_MANUAL , +XE_ENGINE_CAPTURE_SOURCE_GUC +}; +/** * struct xe_hw_engine_snapshot - Hardware engine snapshot *
@@ -160,6 +165,8 @@ struct xe_hw_engine {struct xe_hw_engine_snapshot {/** @name: name of the hw engine */ char *name; +/** @source: Data source, either manual or GuC */ +enum xe_hw_engine_snapshot_source_id source;/** @hwe: hw engine */ struct xe_hw_engine *hwe;/** @logical_instance: logical instance of this hw engine */
drivers/gpu/drm/xe/xe_lrc.c +0 -18
@@ -38,24 +38,6 @@#define LRC_INDIRECT_RING_STATE_SIZE SZ_4K -struct xe_lrc_snapshot { -struct xe_bo *lrc_bo; -void *lrc_snapshot; -unsigned long lrc_size, lrc_offset; - -u32 context_desc; -u32 indirect_context_desc; -u32 head; -struct { -u32 internal; -u32 memory; - } tail; -u32 start_seqno; -u32 seqno; -u32 ctx_timestamp; -u32 ctx_job_timestamp; -}; -static struct xe_device *lrc_to_xe (struct xe_lrc *lrc) {
drivers/gpu/drm/xe/xe_lrc.h +18 -1
@@ -17,9 +17,26 @@ enum xe_engine_class;struct xe_gt ;struct xe_hw_engine ;struct xe_lrc ; -struct xe_lrc_snapshot ;struct xe_vm ; +struct xe_lrc_snapshot { +struct xe_bo *lrc_bo; +void *lrc_snapshot; +unsigned long lrc_size, lrc_offset; + +u32 context_desc; +u32 indirect_context_desc; +u32 head; +struct { +u32 internal; +u32 memory; + } tail; +u32 start_seqno; +u32 seqno; +u32 ctx_timestamp; +u32 ctx_job_timestamp; +}; +#define LRC_PPHWSP_SCRATCH_ADDR (0x34 * 4) struct xe_lrc *xe_lrc_create(struct xe_hw_engine *hwe, struct xe_vm *vm,
Signature · 1 line
2.34.1
PATCH v19 6/6 drm/xe/guc: Save manual engine capture into capture list
In reply to Zhanjun DongSave manual engine capture into capture list.
This removes duplicate register definitions across manual-capture vs guc-err-capture.
--- drivers/gpu/drm/xe/xe_devcoredump.c | 3 +- drivers/gpu/drm/xe/xe_guc_capture.c | 145 +++++++++++++- drivers/gpu/drm/xe/xe_guc_capture.h | 3 +- drivers/gpu/drm/xe/xe_hw_engine.c | 247 +----------------------- drivers/gpu/drm/xe/xe_hw_engine.h | 2 - drivers/gpu/drm/xe/xe_hw_engine_types.h | 59 ------ 6 files changed, 147 insertions(+), 312 deletions(-)
drivers/gpu/drm/xe/xe_devcoredump.c +1 -2
@@ -107,8 +107,7 @@ static ssize_t __xe_devcoredump_read(char *buffer, size_t count,drm_printf (&p,"\n**** HW Engines ****\n" );for (i =0 ; i <XE_NUM_HW_ENGINES ; i++)if (coredump->snapshot.hwe[i]) -xe_hw_engine_snapshot_print (coredump->snapshot.hwe[i], - &p); +xe_engine_snapshot_print (coredump->snapshot.hwe[i], &p);drm_printf (&p,"\n**** VM state ****\n" );xe_vm_snapshot_print (coredump->snapshot.vm, &p);
drivers/gpu/drm/xe/xe_guc_capture.c +141 -4
187 lines not on this page · raw message
drivers/gpu/drm/xe/xe_guc_capture.h +2 -1
@@ -51,8 +51,9 @@ const struct __guc_mmio_reg_descr_group * xe_guc_capture_get_reg_desc_list(struct xe_gt *gt, u32 owner, u32 type, enum guc_capture_list_class_type capture_class, bool is_ext);struct __guc_capture_parsed_output *xe_guc_capture_get_matching_and_lock (struct xe_sched_job *job); +void xe_engine_manual_capture (struct xe_hw_engine *hwe,struct xe_hw_engine_snapshot *snapshot); +void xe_engine_snapshot_print (struct xe_hw_engine_snapshot *snapshot,struct drm_printer *p);void xe_engine_snapshot_capture_for_job (struct xe_sched_job *job); -void xe_engine_guc_capture_print (struct xe_hw_engine_snapshot *snapshot,struct drm_printer *p);void xe_guc_capture_steered_list_init (struct xe_guc *guc);void xe_guc_capture_free (struct xe_guc *guc);int xe_guc_capture_init (struct xe_guc *guc);
drivers/gpu/drm/xe/xe_hw_engine.c +3 -244
292 lines not on this page · raw message
drivers/gpu/drm/xe/xe_hw_engine.h +0 -2
@@ -58,8 +58,6 @@ u32 xe_hw_engine_mask_per_class(struct xe_gt *gt,struct xe_hw_engine_snapshot *xe_hw_engine_snapshot_capture (struct xe_hw_engine *hwe,struct xe_sched_job *job);void xe_hw_engine_snapshot_free (struct xe_hw_engine_snapshot *snapshot); -void xe_hw_engine_snapshot_print (struct xe_hw_engine_snapshot *snapshot, -struct drm_printer *p);void xe_hw_engine_print (struct xe_hw_engine *hwe,struct drm_printer *p);void xe_hw_engine_setup_default_lrc_state (struct xe_hw_engine *hwe);
drivers/gpu/drm/xe/xe_hw_engine_types.h +0 -59
65 lines not on this page · raw message
Signature · 1 line
2.34.1
Patchwork
CI✗ CI.Patch_applied: failure for drm/xe/guc: Add GuC based register capture for error capture (rev19)
In reply to Zhanjun Dong== Series Details ==
== Series Details ==
Series: drm/xe/guc: Add GuC based register capture for error capture (rev19)
URL : https://patchwork.freedesktop.org/series/128077/
State : failure
== Summary ==
=== Applying kernel patches on branch 'drm-tip' with base: ===
Base commit: 5c43f7fc693a drm-tip: 2024y-09m-12d-10h-31m-18s UTC integration manifest === git am output follows ===
error: patch failed: drivers/gpu/drm/xe/xe_hw_engine.c:909
error: drivers/gpu/drm/xe/xe_hw_engine.c: patch does not apply
hint: Use 'git am --show-current-patch=diff' to see the failed patch
hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config advice.mergeConflict false"
Applying: drm/xe/guc: Prepare GuC register list and update ADS size for error capture
Applying: drm/xe/guc: Add XE_LP steered register lists
Applying: drm/xe/guc: Add capture size check in GuC log buffer
Applying: drm/xe/guc: Extract GuC error capture lists
Applying: drm/xe/guc: Plumb GuC-capture into dev coredump
Patch failed at 0005 drm/xe/guc: Plumb GuC-capture into dev coredump