GPU debug support (eudebug)
Subsystems (4)
cover GPU debug support (eudebug)
Hi,
We (Intel eudebug kernel team) would like to submit this patchset to enable debug support for Intel GPU devices.
The aim is to allow Level-Zero + GDB (or some other tool) to attach to xe driver in order to receive information about relevant driver resources, hardware events and to allow debug related hardware control. End goal is full debug capability of supported Intel hardware, see [4].
Debugger first opens a connection to a device through drm ioctl with debug target process as a pid. This will return a dedicated file descriptor used for debugging for further events and control.
Xe internal resources that are considered essential to debugger functionality are relayed as events to the debugger. On debugger connection, all existing resources are relayed to debugger (discovery) and from that point onwards, as they are created/destroyed.
uapi is extended to allow an application/lib to provide debug metadata information. These are relayed as events to the debugger so it can decode the program state.
Along with the resource and metadata events, an event for hardware exceptions, called EU attention, is provided. The debugger, with the assistance of an exception handling program called System Routine (short: SIP) provided with the pipeline setup, can determine which specific EU/thread and instruction encountered the breakpoint or other exceptions.
EU controlling ioctl interface is also introduced where debugger can manipulate individual threads of the currently active workload. This interface enables the debugger to interrupt all threads on demand, check their current state and resume them individually.
The intent is to provide a similar but not API compatible functionality as in out-of-tree i915 debugger support:
https://dgpu-docs.intel.com/driver/gpu-debugging.html
For xe the aim is to have all support merged in upstream, starting with this series. With Lunarlake being first targetted hardware.
I have split the events into xe_drm_eudebug.h instead pushing everything into xe_drm.h, in order to help distinguish what is controlled by which descriptor. If it's through the original xe fd, it is in xe_drm.h and if it's through the opened debugger connection fd, it is in xe_drm_eudebug.h.
Latest code can be found in:
[1] https://gitlab.freedesktop.org/miku/kernel/-/tree/eudebug-dev
With the associated IGT tests:
[2] https://gitlab.freedesktop.org/cmanszew/igt-gpu-tools/-/tree/eudebug-dev
The user for this uapi:
[3] https://github.com/intel/compute-runtime
Event loop and thread control interaction can be found at:
https://github.com/intel/compute-runtime/tree/master/level_zero/tools/source/debug/linux/xe
And the wrappers in:
https://github.com/intel/compute-runtime/tree/master/shared/source/os_interface/linux/xe
https://github.com/intel/compute-runtime/blob/master/shared/source/os_interface/linux/xe/ioctl_helper_xe_debugger.cpp
Note that the XE support is disabled by default and you will need NEO_ENABLE_XE_EU_DEBUG_SUPPORT enabled in order to test.
GDB support:
[4]: https://sourceware.org/pipermail/gdb-patches/2024-July/210264.html
Thank you in advance for any comments and insight.
Andrzej Hajda (1):
drm/xe/eudebug: implement userptr_vma access
Christoph Manszewski (3):
drm/xe/eudebug: Add vm bind and vm bind ops drm/xe/eudebug: Dynamically toggle debugger functionality drm/xe/eudebug_test: Introduce xe_eudebug wa kunit test
Dominik Grzegorzek (10):
drm/xe: Export xe_hw_engine's mmio accessors drm/xe: Move and export xe_hw_engine lookup. drm/xe/eudebug: Introduce exec_queue events drm/xe/eudebug: hw enablement for eudebug drm/xe: Add EUDEBUG_ENABLE exec queue property drm/xe/eudebug: Introduce per device attention scan worker drm/xe/eudebug: Introduce EU control interface drm/xe: Debug metadata create/destroy ioctls drm/xe: Attach debug metadata to vma drm/xe/eudebug: Add debug metadata support for xe_eudebug
Jonathan Cavitt (1):
drm/xe/eudebug: Use ptrace_may_access for xe_eudebug_attach
Mika Kuoppala (6):
drm/xe/eudebug: Introduce eudebug support
kernel: export ptrace_may_access drm/xe/eudebug: Introduce discovery for resources drm/xe/eudebug: Add UFENCE events with acks drm/xe/eudebug: vm open/pread/pwrite drm/xe/eudebug: Implement vm_bind_op discovery
drivers/gpu/drm/xe/Makefile | 5 +- drivers/gpu/drm/xe/regs/xe_engine_regs.h | 8 + drivers/gpu/drm/xe/regs/xe_gt_regs.h | 43 + drivers/gpu/drm/xe/tests/xe_eudebug.c | 170 + drivers/gpu/drm/xe/tests/xe_live_test_mod.c | 2 + drivers/gpu/drm/xe/xe_debug_metadata.c | 125 + drivers/gpu/drm/xe/xe_debug_metadata.h | 25 + drivers/gpu/drm/xe/xe_debug_metadata_types.h | 28 + drivers/gpu/drm/xe/xe_device.c | 47 +- drivers/gpu/drm/xe/xe_device_types.h | 45 + drivers/gpu/drm/xe/xe_eudebug.c | 3841 ++++++++++++++++++ drivers/gpu/drm/xe/xe_eudebug.h | 51 + drivers/gpu/drm/xe/xe_eudebug_types.h | 326 ++ drivers/gpu/drm/xe/xe_exec.c | 2 +- drivers/gpu/drm/xe/xe_exec_queue.c | 80 +- drivers/gpu/drm/xe/xe_exec_queue_types.h | 7 + drivers/gpu/drm/xe/xe_gt_debug.c | 152 + drivers/gpu/drm/xe/xe_gt_debug.h | 27 + drivers/gpu/drm/xe/xe_hw_engine.c | 39 +- drivers/gpu/drm/xe/xe_hw_engine.h | 11 + drivers/gpu/drm/xe/xe_lrc.c | 16 +- drivers/gpu/drm/xe/xe_lrc.h | 4 +- drivers/gpu/drm/xe/xe_reg_sr.c | 21 +- drivers/gpu/drm/xe/xe_reg_sr.h | 4 +- drivers/gpu/drm/xe/xe_rtp.c | 2 +- drivers/gpu/drm/xe/xe_rtp_types.h | 1 + drivers/gpu/drm/xe/xe_sync.c | 49 +- drivers/gpu/drm/xe/xe_sync.h | 8 +- drivers/gpu/drm/xe/xe_sync_types.h | 26 +- drivers/gpu/drm/xe/xe_vm.c | 227 +- drivers/gpu/drm/xe/xe_vm_types.h | 26 + include/uapi/drm/xe_drm.h | 96 +- include/uapi/drm/xe_drm_eudebug.h | 226 ++ kernel/ptrace.c | 1 + 34 files changed, 5655 insertions(+), 86 deletions(-) create mode 100644 drivers/gpu/drm/xe/tests/xe_eudebug.c create mode 100644 drivers/gpu/drm/xe/xe_debug_metadata.c create mode 100644 drivers/gpu/drm/xe/xe_debug_metadata.h create mode 100644 drivers/gpu/drm/xe/xe_debug_metadata_types.h create mode 100644 drivers/gpu/drm/xe/xe_eudebug.c create mode 100644 drivers/gpu/drm/xe/xe_eudebug.h create mode 100644 drivers/gpu/drm/xe/xe_eudebug_types.h create mode 100644 drivers/gpu/drm/xe/xe_gt_debug.c create mode 100644 drivers/gpu/drm/xe/xe_gt_debug.h create mode 100644 include/uapi/drm/xe_drm_eudebug.h
Signature · 1 line
2.34.1
PATCH 1/21 drm/xe: Export xe_hw_engine's mmio accessors
From: Dominik Grzegorzek <[email protected]>
Declare hw engine's mmio accessors in header file. This is in preparation to use these from eudebug code.
--- drivers/gpu/drm/xe/xe_hw_engine.c | 5 ++--- drivers/gpu/drm/xe/xe_hw_engine.h | 5 +++++ 2 files changed, 7 insertions(+), 3 deletions(-)
drivers/gpu/drm/xe/xe_hw_engine.c +2 -3
@@ -274,8 +274,7 @@ static void hw_engine_fini(struct drm_device *drm, void *arg) hwe->gt =NULL ; } -static void hw_engine_mmio_write32 (struct xe_hw_engine *hwe,struct xe_reg reg, -u32 val) +void hw_engine_mmio_write32 (struct xe_hw_engine *hwe,struct xe_reg reg,u32 val) {xe_gt_assert (hwe->gt, !(reg.addr & hwe->mmio_base));xe_force_wake_assert_held (gt_to_fw (hwe->gt), hwe->domain);
@@ -285,7 +284,7 @@ static void hw_engine_mmio_write32(struct xe_hw_engine *hwe, struct xe_reg reg,xe_mmio_write32 (hwe->gt, reg, val); } -static u32 hw_engine_mmio_read32 (struct xe_hw_engine *hwe,struct xe_reg reg) +u32 hw_engine_mmio_read32 (struct xe_hw_engine *hwe,struct xe_reg reg) {xe_gt_assert (hwe->gt, !(reg.addr & hwe->mmio_base));xe_force_wake_assert_held (gt_to_fw (hwe->gt), hwe->domain);
drivers/gpu/drm/xe/xe_hw_engine.h +5 -0
@@ -62,6 +62,11 @@ void xe_hw_engine_print(struct xe_hw_engine *hwe, struct drm_printer *p);void xe_hw_engine_setup_default_lrc_state (struct xe_hw_engine *hwe);bool xe_hw_engine_is_reserved (struct xe_hw_engine *hwe); + +void hw_engine_mmio_write32 (struct xe_hw_engine *hwe,struct xe_reg reg,u32 val); + +u32 hw_engine_mmio_read32 (struct xe_hw_engine *hwe,struct xe_reg reg); +static inline bool xe_hw_engine_is_valid (struct xe_hw_engine *hwe) {return hwe->name;
Signature · 1 line
2.34.1
Matthew Brost
maintainerRe: [PATCH 01/21] drm/xe: Export xe_hw_engine's mmio accessors
Quoted 17 lines · Mika Kuoppala
From: Dominik Grzegorzek <[email protected]>
Declare hw engine's mmio accessors in header file. This is in preparation to use these from eudebug code.
Signed-off-by Dominik Grzegorzek <[email protected]>Signed-off-by Mika Kuoppala <[email protected]>--- drivers/gpu/drm/xe/xe_hw_engine.c | 5 ++--- drivers/gpu/drm/xe/xe_hw_engine.h | 5 +++++ 2 files changed, 7 insertions(+), 3 deletions(-)drivers/gpu/drm/xe/xe_hw_engine.c +0
-0
drivers/gpu/drm/xe/xe_hw_engine.c static void hw_engine_fini(struct drm_device *drm, void *arg) · line 277
Quoted 6 lines
drivers/gpu/drm/xe/xe_hw_engine.c +1
-2@@ -274,8 +274,7 @@ static void hw_engine_fini(struct drm_device *drm, void *arg) hwe->gt =NULL ; } -static void hw_engine_mmio_write32 (struct xe_hw_engine *hwe,struct xe_reg reg, -u32 val) +void hw_engine_mmio_write32 (struct xe_hw_engine *hwe,struct xe_reg reg,u32 val)
Rename this xe_hw_engine_mmio_write32.
drivers/gpu/drm/xe/xe_hw_engine.c static void hw_engine_mmio_write32(struct xe_hw_engine *hwe, struct xe_reg reg, · line 287
Quoted 9 lines
drivers/gpu/drm/xe/xe_hw_engine.c +1
-1{xe_gt_assert (hwe->gt, !(reg.addr & hwe->mmio_base));xe_force_wake_assert_held (gt_to_fw (hwe->gt), hwe->domain);@@ -285,7 +284,7 @@ static void hw_engine_mmio_write32(struct xe_hw_engine *hwe, struct xe_reg reg,xe_mmio_write32 (hwe->gt, reg, val); } -static u32 hw_engine_mmio_read32 (struct xe_hw_engine *hwe,struct xe_reg reg) +u32 hw_engine_mmio_read32 (struct xe_hw_engine *hwe,struct xe_reg reg)
Rename this xe_hw_engine_mmio_read32.
Matt
Quoted 22 lines
drivers/gpu/drm/xe/xe_hw_engine.c +0
-0{xe_gt_assert (hwe->gt, !(reg.addr & hwe->mmio_base));xe_force_wake_assert_held (gt_to_fw (hwe->gt), hwe->domain);drivers/gpu/drm/xe/xe_hw_engine.h +5
-0@@ -62,6 +62,11 @@ void xe_hw_engine_print(struct xe_hw_engine *hwe, struct drm_printer *p);void xe_hw_engine_setup_default_lrc_state (struct xe_hw_engine *hwe);bool xe_hw_engine_is_reserved (struct xe_hw_engine *hwe); + +void hw_engine_mmio_write32 (struct xe_hw_engine *hwe,struct xe_reg reg,u32 val); + +u32 hw_engine_mmio_read32 (struct xe_hw_engine *hwe,struct xe_reg reg); +static inline bool xe_hw_engine_is_valid (struct xe_hw_engine *hwe) {return hwe->name;Signature · 1 line
2.34.1
Re: [PATCH 01/21] drm/xe: Export xe_hw_engine's mmio accessors
In reply to Mika KuoppalaHi Mika,
Quoted 7 lines · Mika Kuoppala
From: Dominik Grzegorzek <[email protected]>
Declare hw engine's mmio accessors in header file. This is in preparation to use these from eudebug code.
Signed-off-by Dominik Grzegorzek <[email protected]>Signed-off-by Mika Kuoppala <[email protected]>
with the renames suggested by Matt:
Thanks,
Andi
PATCH 2/21 drm/xe: Move and export xe_hw_engine lookup.
In reply to Mika KuoppalaFrom: Dominik Grzegorzek <[email protected]>
Move and export xe_hw_engine lookup. This is in preparation to use this in eudebug code where we want to find active engine.
v2: s/tile/gt due to uapi changes (Mika)
--- drivers/gpu/drm/xe/xe_exec_queue.c | 37 ++++-------------------------- drivers/gpu/drm/xe/xe_hw_engine.c | 31 +++++++++++++++++++++++++ drivers/gpu/drm/xe/xe_hw_engine.h | 6 +++++ 3 files changed, 41 insertions(+), 33 deletions(-)
drivers/gpu/drm/xe/xe_exec_queue.c +4 -33
@@ -413,34 +413,6 @@ static int exec_queue_user_extensions(struct xe_device *xe, struct xe_exec_queuereturn 0 ; } -static const enum xe_engine_class user_to_xe_engine_class[] = { - [DRM_XE_ENGINE_CLASS_RENDER ] =XE_ENGINE_CLASS_RENDER , - [DRM_XE_ENGINE_CLASS_COPY ] =XE_ENGINE_CLASS_COPY , - [DRM_XE_ENGINE_CLASS_VIDEO_DECODE ] =XE_ENGINE_CLASS_VIDEO_DECODE , - [DRM_XE_ENGINE_CLASS_VIDEO_ENHANCE ] =XE_ENGINE_CLASS_VIDEO_ENHANCE , - [DRM_XE_ENGINE_CLASS_COMPUTE ] =XE_ENGINE_CLASS_COMPUTE , -}; - -static struct xe_hw_engine * -find_hw_engine (struct xe_device *xe, -struct drm_xe_engine_class_instance eci) -{ -u32 idx; - -if (eci.engine_class >=ARRAY_SIZE (user_to_xe_engine_class)) -return NULL ; - -if (eci.gt_id >= xe->info.gt_count) -return NULL ; - - idx =array_index_nospec (eci.engine_class, -ARRAY_SIZE (user_to_xe_engine_class)); - -return xe_gt_hw_engine (xe_device_get_gt (xe, eci.gt_id), - user_to_xe_engine_class[idx], - eci.engine_instance, true); -} -static u32 bind_exec_queue_logical_mask (struct xe_device *xe,struct xe_gt *gt,struct drm_xe_engine_class_instance *eci,u16 width,u16 num_placements)
@@ -462,8 +434,7 @@ static u32 bind_exec_queue_logical_mask(struct xe_device *xe, struct xe_gt *gt,if (xe_hw_engine_is_reserved (hwe))continue ; -if (hwe->class == - user_to_xe_engine_class[DRM_XE_ENGINE_CLASS_COPY ]) +if (hwe->class ==XE_ENGINE_CLASS_COPY ) logical_mask |=BIT (hwe->logical_instance); }
@@ -492,7 +463,7 @@ static u32 calc_validate_logical_mask(struct xe_device *xe, struct xe_gt *gt, n = j * width + i; - hwe =find_hw_engine (xe, eci[n]); + hwe =xe_hw_engine_lookup (xe, eci[n]);if (XE_IOCTL_DBG (xe, !hwe))return 0 ;
@@ -571,7 +542,7 @@ int xe_exec_queue_create_ioctl(struct drm_device *dev, void *data,if (XE_IOCTL_DBG (xe, !logical_mask))return -EINVAL ; - hwe =find_hw_engine (xe, eci[0 ]); + hwe =xe_hw_engine_lookup (xe, eci[0 ]);if (XE_IOCTL_DBG (xe, !hwe))return -EINVAL ;
@@ -608,7 +579,7 @@ int xe_exec_queue_create_ioctl(struct drm_device *dev, void *data,if (XE_IOCTL_DBG (xe, !logical_mask))return -EINVAL ; - hwe =find_hw_engine (xe, eci[0 ]); + hwe =xe_hw_engine_lookup (xe, eci[0 ]);if (XE_IOCTL_DBG (xe, !hwe))return -EINVAL ;
drivers/gpu/drm/xe/xe_hw_engine.c +31 -0
@@ -5,7 +5,10 @@#include "xe_hw_engine.h" +#include <linux/nospec.h> +#include <drm/drm_managed.h> +#include <drm/xe_drm.h> #include "regs/xe_engine_regs.h" #include "regs/xe_gt_regs.h"
@@ -1134,3 +1137,31 @@ enum xe_force_wake_domains xe_hw_engine_to_fw_domain(struct xe_hw_engine *hwe) {return engine_infos[hwe->engine_id].domain; } + +static const enum xe_engine_class user_to_xe_engine_class[] = { + [DRM_XE_ENGINE_CLASS_RENDER ] =XE_ENGINE_CLASS_RENDER , + [DRM_XE_ENGINE_CLASS_COPY ] =XE_ENGINE_CLASS_COPY , + [DRM_XE_ENGINE_CLASS_VIDEO_DECODE ] =XE_ENGINE_CLASS_VIDEO_DECODE , + [DRM_XE_ENGINE_CLASS_VIDEO_ENHANCE ] =XE_ENGINE_CLASS_VIDEO_ENHANCE , + [DRM_XE_ENGINE_CLASS_COMPUTE ] =XE_ENGINE_CLASS_COMPUTE , +}; + +struct xe_hw_engine * +xe_hw_engine_lookup (struct xe_device *xe, +struct drm_xe_engine_class_instance eci) +{ +unsigned int idx; + +if (eci.engine_class >ARRAY_SIZE (user_to_xe_engine_class)) +return NULL ; + +if (eci.gt_id >= xe->info.gt_count) +return NULL ; + + idx =array_index_nospec (eci.engine_class, +ARRAY_SIZE (user_to_xe_engine_class)); + +return xe_gt_hw_engine (xe_device_get_gt (xe, eci.gt_id), + user_to_xe_engine_class[idx], + eci.engine_instance, true); +}
drivers/gpu/drm/xe/xe_hw_engine.h +6 -0
@@ -9,6 +9,8 @@#include "xe_hw_engine_types.h" struct drm_printer ; +struct drm_xe_engine_class_instance ; +struct xe_device ;#ifdef CONFIG_DRM_XE_JOB_TIMEOUT_MIN #define XE_HW_ENGINE_JOB_TIMEOUT_MIN CONFIG_DRM_XE_JOB_TIMEOUT_MIN
@@ -67,6 +69,10 @@ void hw_engine_mmio_write32(struct xe_hw_engine *hwe, struct xe_reg reg, u32 valu32 hw_engine_mmio_read32 (struct xe_hw_engine *hwe,struct xe_reg reg); +struct xe_hw_engine * +xe_hw_engine_lookup (struct xe_device *xe, +struct drm_xe_engine_class_instance eci); +static inline bool xe_hw_engine_is_valid (struct xe_hw_engine *hwe) {return hwe->name;
Signature · 1 line
2.34.1
Re: [PATCH 02/21] drm/xe: Move and export xe_hw_engine lookup.
Quoted 10 lines · Mika Kuoppala
From: Dominik Grzegorzek <[email protected]>
Move and export
xe_hw_enginelookup. This is in preparation to use this in eudebug code where we want to find active engine.v2: s/tile/gt due to uapi changes (Mika)
Signed-off-by Dominik Grzegorzek <[email protected]>Signed-off-by Mika Kuoppala <[email protected]>
Feel free to send this independently, immediately with:
Quoted 156 lines
--- drivers/gpu/drm/xe/xe_exec_queue.c | 37 ++++-------------------------- drivers/gpu/drm/xe/xe_hw_engine.c | 31 +++++++++++++++++++++++++ drivers/gpu/drm/xe/xe_hw_engine.h | 6 +++++ 3 files changed, 41 insertions(+), 33 deletions(-)drivers/gpu/drm/xe/xe_exec_queue.c +4
-33@@ -413,34 +413,6 @@ static int exec_queue_user_extensions(struct xe_device *xe, struct xe_exec_queuereturn 0 ; } -static const enum xe_engine_class user_to_xe_engine_class[] = { - [DRM_XE_ENGINE_CLASS_RENDER ] =XE_ENGINE_CLASS_RENDER , - [DRM_XE_ENGINE_CLASS_COPY ] =XE_ENGINE_CLASS_COPY , - [DRM_XE_ENGINE_CLASS_VIDEO_DECODE ] =XE_ENGINE_CLASS_VIDEO_DECODE , - [DRM_XE_ENGINE_CLASS_VIDEO_ENHANCE ] =XE_ENGINE_CLASS_VIDEO_ENHANCE , - [DRM_XE_ENGINE_CLASS_COMPUTE ] =XE_ENGINE_CLASS_COMPUTE , -}; - -static struct xe_hw_engine * -find_hw_engine (struct xe_device *xe, -struct drm_xe_engine_class_instance eci) -{ -u32 idx; - -if (eci.engine_class >=ARRAY_SIZE (user_to_xe_engine_class)) -return NULL ; - -if (eci.gt_id >= xe->info.gt_count) -return NULL ; - - idx =array_index_nospec (eci.engine_class, -ARRAY_SIZE (user_to_xe_engine_class)); - -return xe_gt_hw_engine (xe_device_get_gt (xe, eci.gt_id), - user_to_xe_engine_class[idx], - eci.engine_instance, true); -} -static u32 bind_exec_queue_logical_mask (struct xe_device *xe,struct xe_gt *gt,struct drm_xe_engine_class_instance *eci,u16 width,u16 num_placements)@@ -462,8 +434,7 @@ static u32 bind_exec_queue_logical_mask(struct xe_device *xe, struct xe_gt *gt,if (xe_hw_engine_is_reserved (hwe))continue ; -if (hwe->class == - user_to_xe_engine_class[DRM_XE_ENGINE_CLASS_COPY ]) +if (hwe->class ==XE_ENGINE_CLASS_COPY ) logical_mask |=BIT (hwe->logical_instance); }@@ -492,7 +463,7 @@ static u32 calc_validate_logical_mask(struct xe_device *xe, struct xe_gt *gt, n = j * width + i; - hwe =find_hw_engine (xe, eci[n]); + hwe =xe_hw_engine_lookup (xe, eci[n]);if (XE_IOCTL_DBG (xe, !hwe))return 0 ;@@ -571,7 +542,7 @@ int xe_exec_queue_create_ioctl(struct drm_device *dev, void *data,if (XE_IOCTL_DBG (xe, !logical_mask))return -EINVAL ; - hwe =find_hw_engine (xe, eci[0 ]); + hwe =xe_hw_engine_lookup (xe, eci[0 ]);if (XE_IOCTL_DBG (xe, !hwe))return -EINVAL ;@@ -608,7 +579,7 @@ int xe_exec_queue_create_ioctl(struct drm_device *dev, void *data,if (XE_IOCTL_DBG (xe, !logical_mask))return -EINVAL ; - hwe =find_hw_engine (xe, eci[0 ]); + hwe =xe_hw_engine_lookup (xe, eci[0 ]);if (XE_IOCTL_DBG (xe, !hwe))return -EINVAL ;drivers/gpu/drm/xe/xe_hw_engine.c +31
-0@@ -5,7 +5,10 @@#include "xe_hw_engine.h" +#include <linux/nospec.h> +#include <drm/drm_managed.h> +#include <drm/xe_drm.h> #include "regs/xe_engine_regs.h" #include "regs/xe_gt_regs.h" @@ -1134,3 +1137,31 @@ enum xe_force_wake_domains xe_hw_engine_to_fw_domain(struct xe_hw_engine *hwe) {return engine_infos[hwe->engine_id].domain; } + +static const enum xe_engine_class user_to_xe_engine_class[] = { + [DRM_XE_ENGINE_CLASS_RENDER ] =XE_ENGINE_CLASS_RENDER , + [DRM_XE_ENGINE_CLASS_COPY ] =XE_ENGINE_CLASS_COPY , + [DRM_XE_ENGINE_CLASS_VIDEO_DECODE ] =XE_ENGINE_CLASS_VIDEO_DECODE , + [DRM_XE_ENGINE_CLASS_VIDEO_ENHANCE ] =XE_ENGINE_CLASS_VIDEO_ENHANCE , + [DRM_XE_ENGINE_CLASS_COMPUTE ] =XE_ENGINE_CLASS_COMPUTE , +}; + +struct xe_hw_engine * +xe_hw_engine_lookup (struct xe_device *xe, +struct drm_xe_engine_class_instance eci) +{ +unsigned int idx; + +if (eci.engine_class >ARRAY_SIZE (user_to_xe_engine_class)) +return NULL ; + +if (eci.gt_id >= xe->info.gt_count) +return NULL ; + + idx =array_index_nospec (eci.engine_class, +ARRAY_SIZE (user_to_xe_engine_class)); + +return xe_gt_hw_engine (xe_device_get_gt (xe, eci.gt_id), + user_to_xe_engine_class[idx], + eci.engine_instance, true); +}drivers/gpu/drm/xe/xe_hw_engine.h +6
-0@@ -9,6 +9,8 @@#include "xe_hw_engine_types.h" struct drm_printer ; +struct drm_xe_engine_class_instance ; +struct xe_device ;#ifdef CONFIG_DRM_XE_JOB_TIMEOUT_MIN #define XE_HW_ENGINE_JOB_TIMEOUT_MIN CONFIG_DRM_XE_JOB_TIMEOUT_MIN@@ -67,6 +69,10 @@ void hw_engine_mmio_write32(struct xe_hw_engine *hwe, struct xe_reg reg, u32 valu32 hw_engine_mmio_read32 (struct xe_hw_engine *hwe,struct xe_reg reg); +struct xe_hw_engine * +xe_hw_engine_lookup (struct xe_device *xe, +struct drm_xe_engine_class_instance eci); +static inline bool xe_hw_engine_is_valid (struct xe_hw_engine *hwe) {return hwe->name;Signature · 1 line
2.34.1
Re: [PATCH 02/21] drm/xe: Move and export xe_hw_engine lookup.
In reply to Mika KuoppalaQuoted 10 lines · Mika Kuoppala
From: Dominik Grzegorzek <[email protected]>
Move and export
xe_hw_enginelookup. This is in preparation to use this in eudebug code where we want to find active engine.v2: s/tile/gt due to uapi changes (Mika)
Signed-off-by Dominik Grzegorzek <[email protected]>Signed-off-by Mika Kuoppala <[email protected]>
I see this is already appled, so that I'm too late to add my rb here.
Thanks,
Andi
PATCH 3/21 drm/xe/eudebug: Introduce eudebug support
In reply to Mika KuoppalaWith eudebug event interface, user space debugger process (like gdb) is able to keep track of resources created by another process (debuggee using drm/xe) and act upon these resources.
For example, debugger can find a client vm which contains isa/elf for a particular shader/eu-kernel and then inspect and modify it (for example installing a breakpoint).
Debugger first opens a connection to xe with a drm ioctl specifying target pid to connect. This returns an anon fd handle that can then be used to listen for events with dedicated ioctl.
This patch introduces eudebug connection and event queuing, adding client create/destroy and vm create/destroy events as a baseline. More events for full debugger operation are needed and those will be introduced in follow up patches.
The resource tracking parts are inspired by the work of Maciej Patelczyk on resource handling for i915. Chris Wilson suggested improvement of two ways mapping which makes it easy to use resource map as a definitive bookkeep of what resources are played to debugger in the discovery phase (on follow up patch).
v2: - event printer removed (Maarten)
- trim down kfifo accessors (Maarten)
- xa_alloc spurious locking removed (Maarten)
v3: - use list for clients (Matt)
v4: - avoid reporting clients too early (Maciej, VLK-56889)
v5: - return from wait if disconnect (Mika)
v6: - read waitqueue (Mika)
v7: - ENOTCONN early in read_event (Mika)
v8: - consolidate resource error handling (Mika, Maciej)
v9: - ptrace access check (Jonathan)
v10: - detach error handling (Dominik)
--- drivers/gpu/drm/xe/Makefile | 3 +- drivers/gpu/drm/xe/xe_device.c | 24 + drivers/gpu/drm/xe/xe_device_types.h | 26 + drivers/gpu/drm/xe/xe_eudebug.c | 1093 +++++++++++++++++++++++++ drivers/gpu/drm/xe/xe_eudebug.h | 27 + drivers/gpu/drm/xe/xe_eudebug_types.h | 169 ++++ drivers/gpu/drm/xe/xe_vm.c | 7 +- include/uapi/drm/xe_drm.h | 21 + include/uapi/drm/xe_drm_eudebug.h | 57 ++ 9 files changed, 1425 insertions(+), 2 deletions(-) create mode 100644 drivers/gpu/drm/xe/xe_eudebug.c create mode 100644 drivers/gpu/drm/xe/xe_eudebug.h create mode 100644 drivers/gpu/drm/xe/xe_eudebug_types.h create mode 100644 include/uapi/drm/xe_drm_eudebug.h
drivers/gpu/drm/xe/Makefile +2 -1
@@ -114,7 +114,8 @@ xe-y += xe_bb.o \ xe_vram_freq.o \ xe_wait_user_fence.o \ xe_wa.o \ - xe_wopcm.o + xe_wopcm.o \ + xe_eudebug.o xe-$(CONFIG_HMM_MIRROR) += xe_hmm.o
drivers/gpu/drm/xe/xe_device.c +24 -0
@@ -54,6 +54,7 @@#include "xe_vm.h" #include "xe_vram.h" #include "xe_wait_user_fence.h" +#include "xe_eudebug.h" #include "xe_wa.h" #include <generated/xe_wa_oob.h>
@@ -100,6 +101,14 @@ static int xe_file_open(struct drm_device *dev, struct drm_file *file)put_task_struct (task); } +INIT_LIST_HEAD (&xef->link); + +mutex_lock (&xe->files.lock); +list_add_tail (&xef->link, &xe->files.list); +mutex_unlock (&xe->files.lock); + +xe_eudebug_file_open (xef); +return 0 ; }
@@ -158,6 +167,12 @@ static void xe_file_close(struct drm_device *dev, struct drm_file *file)xe_pm_runtime_get (xe); +xe_eudebug_file_close (xef); + +mutex_lock (&xef->xe->files.lock); +list_del_init (&xef->link); +mutex_unlock (&xef->xe->files.lock); +/* * No need for exec_queue.lock here as there is no contention for it * when FD is closing as IOCTLs presumably can't be modifying the
@@ -196,6 +211,7 @@ static const struct drm_ioctl_desc xe_ioctls[] = {DRM_IOCTL_DEF_DRV (XE_WAIT_USER_FENCE , xe_wait_user_fence_ioctl,DRM_RENDER_ALLOW ),DRM_IOCTL_DEF_DRV (XE_OBSERVATION , xe_observation_ioctl,DRM_RENDER_ALLOW ), + DRM_IOCTL_DEF_DRV(XE_EUDEBUG_CONNECT, xe_eudebug_connect_ioctl, DRM_RENDER_ALLOW), };static long xe_drm_ioctl (struct file *file,unsigned int cmd,unsigned long arg)
@@ -285,6 +301,8 @@ static void xe_device_destroy(struct drm_device *dev, void *dummy) {struct xe_device *xe =to_xe_device (dev); +xe_eudebug_fini (xe); +if (xe->preempt_fence_wq)destroy_workqueue (xe->preempt_fence_wq);
@@ -330,6 +348,9 @@ struct xe_device *xe_device_create(struct pci_dev *pdev,spin_lock_init (&xe->irq.lock);spin_lock_init (&xe->clients.lock); +drmm_mutex_init (&xe->drm, &xe->files.lock); +INIT_LIST_HEAD (&xe->files.list); +init_waitqueue_head (&xe->ufence_wq); err =drmm_mutex_init (&xe->drm, &xe->usm.lock);
@@ -356,7 +377,10 @@ struct xe_device *xe_device_create(struct pci_dev *pdev,INIT_LIST_HEAD (&xe->pinned.external_vram);INIT_LIST_HEAD (&xe->pinned.evicted); +xe_eudebug_init (xe); + xe->preempt_fence_wq =alloc_ordered_workqueue ("xe-preempt-fence-wq" ,0 ); + xe->ordered_wq =alloc_ordered_workqueue ("xe-ordered-wq" ,0 ); xe->unordered_wq =alloc_workqueue ("xe-unordered-wq" ,0 ,0 ); if (!xe->ordered_wq || !xe->unordered_wq ||
drivers/gpu/drm/xe/xe_device_types.h +26 -0
@@ -355,6 +355,15 @@ struct xe_device {u64 count; } clients; +/** @files: xe_file list (opens) */ +struct { +/** @lock: Protects xe_file list */ +struct mutex lock; + +/** @list: xe file list */ +struct list_head list; + } files; +/** @usm: unified memory state */ struct {/** @usm.asid: convert a ASID to VM */
@@ -492,6 +501,20 @@ struct xe_device {u8 vm_inject_error_position;#endif +/** @debugger connection list and globals for device */ +struct { +/** @lock: protects the list of connections */ +spinlock_t lock; +/** @list: list of connections, aka debuggers */ +struct list_head list; + +/** @session_count: session counter to track connections */ +u64 session_count; + +/** @available: is the debugging functionality available */ +bool available; + } eudebug; +/* private: */ #if IS_ENABLED (CONFIG_DRM_XE_DISPLAY )
@@ -596,6 +619,9 @@ struct xe_file {/** @refcount: ref count of this xe file */ struct kref refcount; + +/** @link: link into xe_device.client.list */ + struct list_head link; };#endif
drivers/gpu/drm/xe/xe_eudebug.c new file +1093 -0
1,093 lines not on this page · raw message
drivers/gpu/drm/xe/xe_eudebug.h new file +27 -0
@@ -0,0 +1,27 @@ +/* SPDX-License-Identifier: MIT */ +/* +* Copyright © 2023 Intel Corporation +*/ + +#ifndef _XE_EUDEBUG_H_ + +struct drm_device ; +struct drm_file ; +struct xe_device ; +struct xe_file ; +struct xe_vm ; + +int xe_eudebug_connect_ioctl (struct drm_device *dev, +void *data, +struct drm_file *file); + +void xe_eudebug_init (struct xe_device *xe); +void xe_eudebug_fini (struct xe_device *xe); + +void xe_eudebug_file_open (struct xe_file *xef); +void xe_eudebug_file_close (struct xe_file *xef); + +void xe_eudebug_vm_create (struct xe_file *xef,struct xe_vm *vm); +void xe_eudebug_vm_destroy (struct xe_file *xef,struct xe_vm *vm); + +#endif
drivers/gpu/drm/xe/xe_eudebug_types.h new file +169 -0
169 lines not on this page · raw message
drivers/gpu/drm/xe/xe_vm.c +6 -1
@@ -39,6 +39,7 @@#include "xe_trace_bo.h" #include "xe_wa.h" #include "xe_hmm.h" +#include "xe_eudebug.h" static struct drm_gem_object *xe_vm_obj (struct xe_vm *vm) {
@@ -1818,6 +1819,8 @@ int xe_vm_create_ioctl(struct drm_device *dev, void *data, args->reserved[0 ] =xe_bo_main_addr (vm->pt_root[0 ]->bo,XE_PAGE_SIZE );#endif +xe_eudebug_vm_create (xef, vm); +return 0 ; err_free_id:
@@ -1853,8 +1856,10 @@ int xe_vm_destroy_ioctl(struct drm_device *dev, void *data,xa_erase (&xef->vm.xa, args->vm_id);mutex_unlock (&xef->vm.lock); -if (!err) +if (!err) { +xe_eudebug_vm_destroy (xef, vm);xe_vm_close_and_put (vm); + }return err; }
include/uapi/drm/xe_drm.h +21 -0
@@ -102,6 +102,7 @@ extern "C" {#define DRM_XE_EXEC 0x09#define DRM_XE_WAIT_USER_FENCE 0x0a#define DRM_XE_OBSERVATION 0x0b +#define DRM_XE_EUDEBUG_CONNECT 0x0c/* Must be kept compact -- no holes */
@@ -117,6 +118,7 @@ extern "C" {#define DRM_IOCTL_XE_EXEC DRM_IOW(DRM_COMMAND_BASE + DRM_XE_EXEC, struct drm_xe_exec)#define DRM_IOCTL_XE_WAIT_USER_FENCE DRM_IOWR(DRM_COMMAND_BASE + DRM_XE_WAIT_USER_FENCE, struct drm_xe_wait_user_fence)#define DRM_IOCTL_XE_OBSERVATION DRM_IOW(DRM_COMMAND_BASE + DRM_XE_OBSERVATION, struct drm_xe_observation_param) +#define DRM_IOCTL_XE_EUDEBUG_CONNECT DRM_IOWR(DRM_COMMAND_BASE + DRM_XE_EUDEBUG_CONNECT, struct drm_xe_eudebug_connect)/** * DOC: Xe IOCTL Extensions
@@ -1694,6 +1696,25 @@ struct drm_xe_oa_stream_info {__u64 reserved[3 ]; }; +/* +* Debugger ABI (ioctl and events) Version History: +* 0 - No debugger available +* 1 - Initial version +*/ +#define DRM_XE_EUDEBUG_VERSION 1 + +struct drm_xe_eudebug_connect { +/** @extensions: Pointer to the first extension struct, if any */ +__u64 extensions; + +__u64 pid;/* input: Target process ID */ +__u32 flags;/* MBZ */ + +__u32 version;/* output: current ABI (ioctl / events) version */ +}; + +#include "xe_drm_eudebug.h" +#if defined(__cplusplus) }#endif
include/uapi/drm/xe_drm_eudebug.h new file +57 -0
57 lines not on this page · raw message
Signature · 1 line
2.34.1
Matthew Brost
maintainerRe: [PATCH 03/21] drm/xe/eudebug: Introduce eudebug support
Quoted 17 lines · Mika Kuoppala
With eudebug event interface, user space debugger process (like gdb) is able to keep track of resources created by another process (debuggee using drm/xe) and act upon these resources.
For example, debugger can find a client vm which contains isa/elf for a particular shader/eu-kernel and then inspect and modify it (for example installing a breakpoint).
Debugger first opens a connection to xe with a drm ioctl specifying target pid to connect. This returns an anon fd handle that can then be used to listen for events with dedicated ioctl.
This patch introduces eudebug connection and event queuing, adding client create/destroy and vm create/destroy events as a baseline. More events for full debugger operation are needed and those will be introduced in follow up patches.
Quoted 6 lines
The resource tracking parts are inspired by the work of Maciej Patelczyk on resource handling for i915. Chris Wilson suggested improvement of two ways mapping which makes it easy to use resource map as a definitive bookkeep of what resources are played to debugger in the discovery phase (on follow up patch).
Not a full review just an idea. Do you think it would be worthwhile to add a Kconfig option to enable EU debug support so if we don't care about EU debug it compiles out?
The layering would have to be fixed up to make this sane (e.g. all common Xe code calls into the EU layer so we don't have 'if IS_ENABLED(CONFIG_EU_DEBUG)' sprinkled all over the place. Fixing up layering probably isn't bad idea anyways and as common Xe code really shouldn't open code EU debug specific things anyways. This would enforce good layering too.
Matt
Quoted 1,631 lines
v2: - event printer removed (Maarten)
- trim down kfifo accessors (Maarten)
-xa_allocspurious locking removed (Maarten)
v3: - use list for clients (Matt)
v4: - avoid reporting clients too early (Maciej, VLK-56889)
v5: - return from wait if disconnect (Mika)
v6: - read waitqueue (Mika)
v7: - ENOTCONN early inread_event(Mika)
v8: - consolidate resource error handling (Mika, Maciej)
v9: - ptrace access check (Jonathan)
v10: - detach error handling (Dominik)Cc Maarten Lankhorst <[email protected]>Cc Lucas De Marchi <[email protected]>Cc Dominik Grzegorzek <[email protected]>Cc Andi Shyti <[email protected]>Cc Matt Roper <[email protected]>Signed-off-by Mika Kuoppala <[email protected]>Signed-off-by Maciej Patelczyk <[email protected]>--- drivers/gpu/drm/xe/Makefile | 3 +- drivers/gpu/drm/xe/xe_device.c | 24 + drivers/gpu/drm/xe/xe_device_types.h | 26 + drivers/gpu/drm/xe/xe_eudebug.c | 1093 +++++++++++++++++++++++++ drivers/gpu/drm/xe/xe_eudebug.h | 27 + drivers/gpu/drm/xe/xe_eudebug_types.h | 169 ++++ drivers/gpu/drm/xe/xe_vm.c | 7 +- include/uapi/drm/xe_drm.h | 21 + include/uapi/drm/xe_drm_eudebug.h | 57 ++ 9 files changed, 1425 insertions(+), 2 deletions(-) create mode 100644 drivers/gpu/drm/xe/xe_eudebug.c create mode 100644 drivers/gpu/drm/xe/xe_eudebug.h create mode 100644 drivers/gpu/drm/xe/xe_eudebug_types.h create mode 100644 include/uapi/drm/xe_drm_eudebug.hdrivers/gpu/drm/xe/Makefile +2
-1@@ -114,7 +114,8 @@ xe-y += xe_bb.o \ xe_vram_freq.o \ xe_wait_user_fence.o \ xe_wa.o \ - xe_wopcm.o + xe_wopcm.o \ + xe_eudebug.o xe-$(CONFIG_HMM_MIRROR) += xe_hmm.odrivers/gpu/drm/xe/xe_device.c +24
-0@@ -54,6 +54,7 @@#include "xe_vm.h" #include "xe_vram.h" #include "xe_wait_user_fence.h" +#include "xe_eudebug.h" #include "xe_wa.h" #include <generated/xe_wa_oob.h> @@ -100,6 +101,14 @@ static int xe_file_open(struct drm_device *dev, struct drm_file *file)put_task_struct (task); } +INIT_LIST_HEAD (&xef->link); + +mutex_lock (&xe->files.lock); +list_add_tail (&xef->link, &xe->files.list); +mutex_unlock (&xe->files.lock); + +xe_eudebug_file_open (xef); +return 0 ; }@@ -158,6 +167,12 @@ static void xe_file_close(struct drm_device *dev, struct drm_file *file)xe_pm_runtime_get (xe); +xe_eudebug_file_close (xef); + +mutex_lock (&xef->xe->files.lock); +list_del_init (&xef->link); +mutex_unlock (&xef->xe->files.lock); +/* * No need for exec_queue.lock here as there is no contention for it * when FD is closing as IOCTLs presumably can't be modifying the @@ -196,6 +211,7 @@ static const struct drm_ioctl_desc xe_ioctls[] = {DRM_IOCTL_DEF_DRV (XE_WAIT_USER_FENCE , xe_wait_user_fence_ioctl,DRM_RENDER_ALLOW ),DRM_IOCTL_DEF_DRV (XE_OBSERVATION , xe_observation_ioctl,DRM_RENDER_ALLOW ), + DRM_IOCTL_DEF_DRV(XE_EUDEBUG_CONNECT, xe_eudebug_connect_ioctl, DRM_RENDER_ALLOW), };static long xe_drm_ioctl (struct file *file,unsigned int cmd,unsigned long arg)@@ -285,6 +301,8 @@ static void xe_device_destroy(struct drm_device *dev, void *dummy) {struct xe_device *xe =to_xe_device (dev); +xe_eudebug_fini (xe); +if (xe->preempt_fence_wq)destroy_workqueue (xe->preempt_fence_wq);@@ -330,6 +348,9 @@ struct xe_device *xe_device_create(struct pci_dev *pdev,spin_lock_init (&xe->irq.lock);spin_lock_init (&xe->clients.lock); +drmm_mutex_init (&xe->drm, &xe->files.lock); +INIT_LIST_HEAD (&xe->files.list); +init_waitqueue_head (&xe->ufence_wq); err =drmm_mutex_init (&xe->drm, &xe->usm.lock);@@ -356,7 +377,10 @@ struct xe_device *xe_device_create(struct pci_dev *pdev,INIT_LIST_HEAD (&xe->pinned.external_vram);INIT_LIST_HEAD (&xe->pinned.evicted); +xe_eudebug_init (xe); + xe->preempt_fence_wq =alloc_ordered_workqueue ("xe-preempt-fence-wq" ,0 ); + xe->ordered_wq =alloc_ordered_workqueue ("xe-ordered-wq" ,0 ); xe->unordered_wq =alloc_workqueue ("xe-unordered-wq" ,0 ,0 ); if (!xe->ordered_wq || !xe->unordered_wq ||drivers/gpu/drm/xe/xe_device_types.h +26
-0@@ -355,6 +355,15 @@ struct xe_device {u64 count; } clients; +/** @files: xe_file list (opens) */ +struct { +/** @lock: Protects xe_file list */ +struct mutex lock; + +/** @list: xe file list */ +struct list_head list; + } files; +/** @usm: unified memory state */ struct {/** @usm.asid: convert a ASID to VM */ @@ -492,6 +501,20 @@ struct xe_device {u8 vm_inject_error_position;#endif +/** @debugger connection list and globals for device */ +struct { +/** @lock: protects the list of connections */ +spinlock_t lock; +/** @list: list of connections, aka debuggers */ +struct list_head list; + +/** @session_count: session counter to track connections */ +u64 session_count; + +/** @available: is the debugging functionality available */ +bool available; + } eudebug; +/* private: */ #if IS_ENABLED (CONFIG_DRM_XE_DISPLAY )@@ -596,6 +619,9 @@ struct xe_file {/** @refcount: ref count of this xe file */ struct kref refcount; + +/** @link: link into xe_device.client.list */ + struct list_head link; };#endif drivers/gpu/drm/xe/xe_eudebug.c new file +1093
-0@@ -0,0 +1,1093 @@ +// SPDX-License-Identifier: MIT +/* +* Copyright © 2023 Intel Corporation +*/ + +#include <linux/uaccess.h> + +#include <linux/anon_inodes.h> +#include <linux/poll.h> +#include <linux/delay.h> + +#include <drm/drm_managed.h> + +#include "xe_device.h" +#include "xe_assert.h" +#include "xe_macros.h" + +#include "xe_eudebug_types.h" +#include "xe_eudebug.h" + +/* +* If there is no detected event read by userspace, during this period, assume +* userspace problem and disconnect debugger to allow forward progress. +*/ +#define XE_EUDEBUG_NO_READ_DETECTED_TIMEOUT_MS (25 * 1000) + +#define for_each_debugger_rcu (debugger, head) \ + list_for_each_entry_rcu((debugger), (head), connection_link) +#define for_each_debugger (debugger, head) \ + list_for_each_entry((debugger), (head), connection_link) + +#define cast_event (T , event) container_of((event), typeof(*(T)), base) + +#define XE_EUDEBUG_DBG_STR "eudbg: %lld:%lu:%s (%d/%d) -> (%d/%d): " +#define XE_EUDEBUG_DBG_ARGS (d) (d)->session, \ + atomic_long_read(&(d)->events.seqno), \ + READ_ONCE(d->connection.status) <= 0 ? "disconnected" : "", \ + current->pid, \ + task_tgid_nr(current), \ + (d)->target_task->pid, \ + task_tgid_nr((d)->target_task) + +#define eu_err (d, fmt, ...) drm_err(&(d)->xe->drm, XE_EUDEBUG_DBG_STR # fmt, \ + XE_EUDEBUG_DBG_ARGS(d), ##__VA_ARGS__) +#define eu_warn (d, fmt, ...) drm_warn(&(d)->xe->drm, XE_EUDEBUG_DBG_STR # fmt, \ + XE_EUDEBUG_DBG_ARGS(d), ##__VA_ARGS__) +#define eu_dbg (d, fmt, ...) drm_dbg(&(d)->xe->drm, XE_EUDEBUG_DBG_STR # fmt, \ + XE_EUDEBUG_DBG_ARGS(d), ##__VA_ARGS__) + +#define xe_eudebug_assert (d, ...) xe_assert((d)->xe, ##__VA_ARGS__) + +#define struct_member (T , member) (((T *)0)->member) + +/* Keep 1:1 parity with uapi events */ +#define write_member (T_out, ptr, member, value) { \ + BUILD_BUG_ON(sizeof(*ptr) != sizeof(T_out)); \ + BUILD_BUG_ON(offsetof(typeof(*ptr), member) != \ + offsetof(typeof(T_out), member)); \ + BUILD_BUG_ON(sizeof(ptr->member) != sizeof(value)); \ + BUILD_BUG_ON(sizeof(struct_member(T_out, member)) != sizeof(value)); \ + BUILD_BUG_ON(!typecheck(typeof((ptr)->member), value)); \ + (ptr)->member = (value); \ + } + +static struct xe_eudebug_event * +event_fifo_pending (struct xe_eudebug *d) +{ +struct xe_eudebug_event *event; + +if (kfifo_peek (&d->events.fifo, &event)) +return event; + +return NULL ; +} + +/* +* This is racy as we dont take the lock for read but all the +* callsites can handle the race so we can live without lock. +*/ +__no_kcsan +static unsigned int +event_fifo_num_events_peek (const struct xe_eudebug *const d) +{ +return kfifo_len (&d->events.fifo); +} + +static bool +xe_eudebug_detached (struct xe_eudebug *d) +{ +int status; + +spin_lock (&d->connection.lock); + status = d->connection.status; +spin_unlock (&d->connection.lock); + +return status <=0 ; +} + +static int +xe_eudebug_error (const struct xe_eudebug *const d) +{ +const int status =READ_ONCE (d->connection.status); + +return status <=0 ? status :0 ; +} + +static unsigned int +event_fifo_has_events (struct xe_eudebug *d) +{ +if (xe_eudebug_detached (d)) +return 1 ; + +return event_fifo_num_events_peek (d); +} + +static const struct rhashtable_params rhash_res = { + .head_offset = offsetof(struct xe_eudebug_handle , rh_head), + .key_len = sizeof_field(struct xe_eudebug_handle, key), + .key_offset = offsetof(struct xe_eudebug_handle , key), + .automatic_shrinking = true, +}; + +static struct xe_eudebug_resource * +resource_from_type (struct xe_eudebug_resources *const res,const int t) +{ +return &res->rt[t]; +} + +static struct xe_eudebug_resources * +xe_eudebug_resources_alloc (void ) +{ +struct xe_eudebug_resources *res; +int err; +int i; + + res =kzalloc (sizeof (*res),GFP_ATOMIC ); +if (!res) +return ERR_PTR (-ENOMEM ); + +mutex_init (&res->lock); + +for (i =0 ; i <XE_EUDEBUG_RES_TYPE_COUNT ; i++) { +xa_init_flags (&res->rt[i].xa,XA_FLAGS_ALLOC1 ); + err =rhashtable_init (&res->rt[i].rh, &rhash_res); + +if (err) +break ; + } + +if (err) { +while (i--) { +xa_destroy (&res->rt[i].xa); +rhashtable_destroy (&res->rt[i].rh); + } + +kfree (res); +return ERR_PTR (err); + } + +return res; +} + +static void res_free_fn (void *ptr,void *arg) +{ +XE_WARN_ON (ptr); +kfree (ptr); +} + +static void +xe_eudebug_destroy_resources (struct xe_eudebug *d) +{ +struct xe_eudebug_resources *res = d->res; +struct xe_eudebug_handle *h; +unsigned long j; +int i; +int err; + +mutex_lock (&res->lock); +for (i =0 ; i <XE_EUDEBUG_RES_TYPE_COUNT ; i++) { +struct xe_eudebug_resource *r = &res->rt[i]; + +xa_for_each (&r->xa, j, h) { +struct xe_eudebug_handle *t; + + err =rhashtable_remove_fast (&r->rh, + &h->rh_head, + rhash_res); +xe_eudebug_assert (d, !err); + t =xa_erase (&r->xa, h->id); +xe_eudebug_assert (d, t == h); +kfree (t); + } + } +mutex_unlock (&res->lock); + +for (i =0 ; i <XE_EUDEBUG_RES_TYPE_COUNT ; i++) { +struct xe_eudebug_resource *r = &res->rt[i]; + +rhashtable_free_and_destroy (&r->rh, res_free_fn,NULL ); +xe_eudebug_assert (d,xa_empty (&r->xa)); +xa_destroy (&r->xa); + } + +mutex_destroy (&res->lock); + +kfree (res); +} + +static void xe_eudebug_free (struct kref *ref) +{ +struct xe_eudebug *d =container_of (ref,typeof (*d), ref); +struct xe_eudebug_event *event; + +while (kfifo_get (&d->events.fifo, &event)) +kfree (event); + +xe_eudebug_destroy_resources (d); +put_task_struct (d->target_task); + +xe_eudebug_assert (d, !kfifo_len (&d->events.fifo)); + +kfree_rcu (d, rcu); +} + +static void xe_eudebug_put (struct xe_eudebug *d) +{ +kref_put (&d->ref, xe_eudebug_free); +} + +static struct task_struct *find_get_target (const pid_t nr) +{ +struct task_struct *task; + +rcu_read_lock (); + task =pid_task (find_pid_ns (nr,task_active_pid_ns (current)),PIDTYPE_PID ); +if (task) +get_task_struct (task); +rcu_read_unlock (); + +return task; +} + +static int +xe_eudebug_attach (struct xe_device *xe,struct xe_eudebug *d, +const pid_t pid_nr) +{ +struct task_struct *target; +struct xe_eudebug *iter; +kuid_t uid =current_uid (); +int ret =0 ; + + target =find_get_target (pid_nr); +if (!target) +return -ENOENT ; + +if (!uid_eq (uid,task_uid (target)) && !capable (CAP_SYS_ADMIN )) { +put_task_struct (target); +return -EACCES ; + } + +XE_WARN_ON (d->connection.status !=0 ); + +spin_lock (&xe->eudebug.lock); +for_each_debugger (iter, &xe->eudebug.list) { +if (!same_thread_group (iter->target_task, target)) +continue ; + + ret = -EBUSY ; + } + +if (!ret && xe->eudebug.session_count +1 ==0 ) + ret = -ENOSPC ; + +if (!ret) { + d->connection.status =XE_EUDEBUG_STATUS_CONNECTED ; + d->xe = xe; + d->target_task = target; + d->session = ++xe->eudebug.session_count; +kref_get (&d->ref); +list_add_tail_rcu (&d->connection_link, &xe->eudebug.list); + } +spin_unlock (&xe->eudebug.lock); + +if (ret) +put_task_struct (target); + +return ret; +} + +static bool xe_eudebug_detach (struct xe_device *xe, +struct xe_eudebug *d, +const int err) +{ +bool detached = false; + +XE_WARN_ON (err >0 ); + +spin_lock (&d->connection.lock); +if (d->connection.status ==XE_EUDEBUG_STATUS_CONNECTED ) { + d->connection.status = err; + detached = true; + } +spin_unlock (&d->connection.lock); + +if (!detached) +return false; + +spin_lock (&xe->eudebug.lock); +list_del_rcu (&d->connection_link); +spin_unlock (&xe->eudebug.lock); + +eu_dbg (d,"session %lld detached with %d" , d->session, err); + +/* Our ref with the connection_link */ +xe_eudebug_put (d); + +return true; +} + +static int _xe_eudebug_disconnect (struct xe_eudebug *d, +const int err) +{ +wake_up_all (&d->events.write_done); +wake_up_all (&d->events.read_done); + +return xe_eudebug_detach (d->xe, d, err); +} + +#define xe_eudebug_disconnect (_d, _err) ({ \ + if (_xe_eudebug_disconnect((_d), (_err))) { \ + if ((_err) == 0 || (_err) == -ETIMEDOUT) \ + eu_dbg(d, "Session closed (%d)", (_err)); \ + else \ + eu_err(d, "Session disconnected, err = %d (%s:%d)", \ + (_err), __func__, __LINE__); \ + } \ +}) + +static int xe_eudebug_release (struct inode *inode,struct file *file) +{ +struct xe_eudebug *d = file->private_data; + +xe_eudebug_disconnect (d,0 ); +xe_eudebug_put (d); + +return 0 ; +} + +static __poll_t xe_eudebug_poll (struct file *file,poll_table *wait) +{ +struct xe_eudebug *const d = file->private_data; +__poll_t ret =0 ; + +poll_wait (file, &d->events.write_done, wait); + +if (xe_eudebug_detached (d)) { + ret |=EPOLLHUP ; +if (xe_eudebug_error (d)) + ret |=EPOLLERR ; + } + +if (event_fifo_num_events_peek (d)) + ret |=EPOLLIN ; + +return ret; +} + +static ssize_t xe_eudebug_read (struct file *file, +char __user *buf, +size_t count, +loff_t *ppos) +{ +return -EINVAL ; +} + +static struct xe_eudebug * +xe_eudebug_for_task_get (struct xe_device *xe, +struct task_struct *task) +{ +struct xe_eudebug *d, *iter; + + d =NULL ; + +rcu_read_lock (); +for_each_debugger_rcu (iter, &xe->eudebug.list) { +if (!same_thread_group (iter->target_task, task)) +continue ; + +if (kref_get_unless_zero (&iter->ref)) + d = iter; + +break ; + } +rcu_read_unlock (); + +return d; +} + +static struct task_struct *find_task_get (struct xe_file *xef) +{ +struct task_struct *task; +struct pid *pid; + +rcu_read_lock (); + pid =rcu_dereference (xef->drm->pid); + task =pid_task (pid,PIDTYPE_PID ); +if (task) +get_task_struct (task); +rcu_read_unlock (); + +return task; +} + +static struct xe_eudebug * +xe_eudebug_get (struct xe_file *xef) +{ +struct task_struct *task; +struct xe_eudebug *d; + + d =NULL ; + task =find_task_get (xef); +if (task) { + d =xe_eudebug_for_task_get (to_xe_device (xef->drm->minor->dev), + task); +put_task_struct (task); + } + +if (!d) +return NULL ; + +if (xe_eudebug_detached (d)) { +xe_eudebug_put (d); +return NULL ; + } + +return d; +} + +static int xe_eudebug_queue_event (struct xe_eudebug *d, +struct xe_eudebug_event *event) +{ +const u64 wait_jiffies =msecs_to_jiffies (1000 ); +u64 last_read_detected_ts, last_head_seqno, start_ts; + +xe_eudebug_assert (d, event->len >sizeof (struct xe_eudebug_event )); +xe_eudebug_assert (d, event->type); +xe_eudebug_assert (d, event->type !=DRM_XE_EUDEBUG_EVENT_READ ); + + start_ts =ktime_get (); + last_read_detected_ts = start_ts; + last_head_seqno =0 ; + +do { +struct xe_eudebug_event *head; +u64 head_seqno; +bool was_queued; + +if (xe_eudebug_detached (d)) +break ; + +spin_lock (&d->events.lock); + head =event_fifo_pending (d); +if (head) + head_seqno = event->seqno; +else + head_seqno =0 ; + + was_queued =kfifo_in (&d->events.fifo, &event,1 ); +spin_unlock (&d->events.lock); + +wake_up_all (&d->events.write_done); + +if (was_queued) { + event =NULL ; +break ; + } + +XE_WARN_ON (!head_seqno); + +/* If we detect progress, restart timeout */ +if (last_head_seqno != head_seqno) + last_read_detected_ts =ktime_get (); + + last_head_seqno = head_seqno; + +wait_event_interruptible_timeout (d->events.read_done, + !kfifo_is_full (&d->events.fifo), + wait_jiffies); + + }while (ktime_ms_delta (ktime_get (), last_read_detected_ts) < +XE_EUDEBUG_NO_READ_DETECTED_TIMEOUT_MS ); + +if (event) { +eu_dbg (d, +"event %llu queue failed (blocked %lld ms, avail %d)" , + event ? event->seqno :0 , +ktime_ms_delta (ktime_get (), start_ts), +kfifo_avail (&d->events.fifo)); + +kfree (event); + +return -ETIMEDOUT ; + } + +return 0 ; +} + +static struct xe_eudebug_handle * +alloc_handle (const int type,const u64 key) +{ +struct xe_eudebug_handle *h; + + h =kzalloc (sizeof (*h),GFP_ATOMIC ); +if (!h) +return NULL ; + + h->key = key; + +return h; +} + +static struct xe_eudebug_handle * +__find_handle (struct xe_eudebug_resource *r, +const u64 key) +{ +struct xe_eudebug_handle *h; + + h =rhashtable_lookup_fast (&r->rh, + &key, + rhash_res); +return h; +} + +static int find_handle (struct xe_eudebug_resources *res, +const int type, +const void *p) +{ +const u64 key = (u64 )p; +struct xe_eudebug_resource *r; +struct xe_eudebug_handle *h; +int id; + +if (XE_WARN_ON (!key)) +return -EINVAL ; + + r =resource_from_type (res, type); + +mutex_lock (&res->lock); + h =__find_handle (r, key); + id = h ? h->id : -ENOENT ; +mutex_unlock (&res->lock); + +return id; +} + +static int _xe_eudebug_add_handle (struct xe_eudebug *d, +int type, +void *p, +u64 *seqno, +int *handle) +{ +const u64 key = (u64 )p; +struct xe_eudebug_resource *r; +struct xe_eudebug_handle *h, *o; +int err; + +if (XE_WARN_ON (p ==NULL )) +return -EINVAL ; + +if (xe_eudebug_detached (d)) +return -ENOTCONN ; + + h =alloc_handle (type, key); +if (!h) +return -ENOMEM ; + + r =resource_from_type (d->res, type); + +mutex_lock (&d->res->lock); + o =__find_handle (r, key); +if (!o) { + err =xa_alloc (&r->xa, &h->id, h, xa_limit_31b,GFP_KERNEL ); + +if (h->id >=INT_MAX ) { +xa_erase (&r->xa, h->id); + err = -ENOSPC ; + } + +if (!err) + err =rhashtable_insert_fast (&r->rh, + &h->rh_head, + rhash_res); + +if (err) { +xa_erase (&r->xa, h->id); + }else { +if (seqno) + *seqno =atomic_long_inc_return (&d->events.seqno); + } + }else { +xe_eudebug_assert (d, o->id); + err = -EEXIST ; + } +mutex_unlock (&d->res->lock); + +if (handle) + *handle = o ? o->id : h->id; + +if (err) { +kfree (h); +XE_WARN_ON (err >0 ); +return err; + } + +xe_eudebug_assert (d, h->id); + +return h->id; +} + +static int xe_eudebug_add_handle (struct xe_eudebug *d, +int type, +void *p, +u64 *seqno) +{ +int ret; + + ret =_xe_eudebug_add_handle (d, type, p, seqno,NULL ); +if (ret == -EEXIST || ret == -ENOTCONN ) { +eu_dbg (d,"%d on adding %d" , ret, type); +return 0 ; + } + +if (ret <0 ) +xe_eudebug_disconnect (d, ret); + +return ret; +} + +static int _xe_eudebug_remove_handle (struct xe_eudebug *d,int type,void *p, +u64 *seqno) +{ +const u64 key = (u64 )p; +struct xe_eudebug_resource *r; +struct xe_eudebug_handle *h, *xa_h; +int ret; + +if (XE_WARN_ON (!key)) +return -EINVAL ; + +if (xe_eudebug_detached (d)) +return -ENOTCONN ; + + r =resource_from_type (d->res, type); + +mutex_lock (&d->res->lock); + h =__find_handle (r, key); +if (h) { + ret =rhashtable_remove_fast (&r->rh, + &h->rh_head, + rhash_res); +xe_eudebug_assert (d, !ret); + xa_h =xa_erase (&r->xa, h->id); +xe_eudebug_assert (d, xa_h == h); +if (!ret) { + ret = h->id; +if (seqno) + *seqno =atomic_long_inc_return (&d->events.seqno); + } + }else { + ret = -ENOENT ; + } +mutex_unlock (&d->res->lock); + +kfree (h); + +xe_eudebug_assert (d, ret); + +return ret; +} + +static int xe_eudebug_remove_handle (struct xe_eudebug *d,int type,void *p, +u64 *seqno) +{ +int ret; + + ret =_xe_eudebug_remove_handle (d, type, p, seqno); +if (ret == -ENOENT || ret == -ENOTCONN ) { +eu_dbg (d,"%d on removing %d" , ret, type); +return 0 ; + } + +if (ret <0 ) +xe_eudebug_disconnect (d, ret); + +return ret; +} + +static struct xe_eudebug_event * +__xe_eudebug_create_event (struct xe_eudebug *d, +u64 seqno,u16 type,u16 flags,u32 len,gfp_t gfp) +{ +struct xe_eudebug_event *event; + +xe_eudebug_assert (d, len >sizeof (*event)); + + event =kzalloc (len, gfp); +if (!event) +return NULL ; + + event->type = type; + event->flags = flags; + event->len = len; + event->seqno = seqno; + +return event; +} + +static struct xe_eudebug_event * +xe_eudebug_create_event (struct xe_eudebug *d,u16 type,u64 seqno,u16 flags, +u32 len,gfp_t gfp) +{ +return __xe_eudebug_create_event (d, seqno, type, flags, len, gfp); +} + +static long xe_eudebug_read_event (struct xe_eudebug *d, +const u64 arg, +const bool wait) +{ +struct xe_device *xe = d->xe; +struct drm_xe_eudebug_event __user *const user_orig = +u64_to_user_ptr (arg); +struct drm_xe_eudebug_event user_event; +struct xe_eudebug_event *event; +long ret =0 ; + +if (XE_IOCTL_DBG (xe,copy_from_user (&user_event, user_orig,sizeof (user_event)))) +return -EFAULT ; + +if (XE_IOCTL_DBG (xe, !user_event.type)) +return -EINVAL ; + +if (XE_IOCTL_DBG (xe, user_event.type >DRM_XE_EUDEBUG_EVENT_MAX_EVENT )) +return -EINVAL ; + +if (XE_IOCTL_DBG (xe, user_event.type !=DRM_XE_EUDEBUG_EVENT_READ )) +return -EINVAL ; + +if (XE_IOCTL_DBG (xe, user_event.len <sizeof (*user_orig))) +return -EINVAL ; + +if (XE_IOCTL_DBG (xe, user_event.flags)) +return -EINVAL ; + +if (XE_IOCTL_DBG (xe, user_event.reserved)) +return -EINVAL ; + +/* XXX: define wait time in connect arguments ? */ +if (wait) { + ret =wait_event_interruptible_timeout (d->events.write_done, +event_fifo_has_events (d), +msecs_to_jiffies (5 *1000 )); + +if (XE_IOCTL_DBG (xe, ret <0 )) +return ret; + } + + ret =0 ; +spin_lock (&d->events.lock); + event =event_fifo_pending (d); +if (event) { +if (user_event.len < event->len) { + ret = -EMSGSIZE ; + }else if (!kfifo_out (&d->events.fifo, &event,1 )) { +eu_warn (d,"internal fifo corruption" ); + ret = -ENOTCONN ; + } + } +spin_unlock (&d->events.lock); + +wake_up_all (&d->events.read_done); + +if (XE_IOCTL_DBG (xe, ret)) +return ret; + +if (!event) { +if (xe_eudebug_detached (d)) +return -ENOTCONN ; +if (!wait) +return -EAGAIN ; + +return -ENOENT ; + } + +if (copy_to_user (user_orig, event, event->len)) + ret = -EFAULT ; +else +eu_dbg (d,"event read: type=%u, flags=0x%x, seqno=%llu" , event->type, + event->flags, event->seqno); + +kfree (event); + +return ret; +} + +static long xe_eudebug_ioctl (struct file *file, +unsigned int cmd, +unsigned long arg) +{ +struct xe_eudebug *const d = file->private_data; +long ret; + +switch (cmd) { +case DRM_XE_EUDEBUG_IOCTL_READ_EVENT : + ret =xe_eudebug_read_event (d, arg, + !(file->f_flags &O_NONBLOCK )); +break ; + +default : + ret = -EINVAL ; + } + +return ret; +} + +static const struct file_operations fops = { + .owner =THIS_MODULE , + .llseek = no_llseek, + .release = xe_eudebug_release, + .poll = xe_eudebug_poll, + .read = xe_eudebug_read, + .unlocked_ioctl = xe_eudebug_ioctl, +}; + +static int +xe_eudebug_connect (struct xe_device *xe, +struct drm_xe_eudebug_connect *param) +{ +const u64 known_open_flags =0 ; +unsigned long f_flags =0 ; +struct xe_eudebug *d; +int fd, err; + +if (param->extensions) +return -EINVAL ; + +if (!param->pid) +return -EINVAL ; + +if (param->flags & ~known_open_flags) +return -EINVAL ; + +if (param->version && param->version !=DRM_XE_EUDEBUG_VERSION ) +return -EINVAL ; + + param->version =DRM_XE_EUDEBUG_VERSION ; + +if (!xe->eudebug.available) +return -EOPNOTSUPP ; + + d =kzalloc (sizeof (*d),GFP_KERNEL ); +if (!d) +return -ENOMEM ; + +kref_init (&d->ref); +spin_lock_init (&d->connection.lock); +init_waitqueue_head (&d->events.write_done); +init_waitqueue_head (&d->events.read_done); + +spin_lock_init (&d->events.lock); +INIT_KFIFO (d->events.fifo); + + d->res =xe_eudebug_resources_alloc (); +if (IS_ERR (d->res)) { + err =PTR_ERR (d->res); + goto err_free; + } + + err =xe_eudebug_attach (xe, d, param->pid); +if (err) + goto err_free_res; + + fd =anon_inode_getfd ("[xe_eudebug]" , &fops, d, f_flags); +if (fd <0 ) { + err = fd; + goto err_detach; + } + +eu_dbg (d,"connected session %lld" , d->session); + +return fd; + +err_detach: +xe_eudebug_detach (xe, d, err); +err_free_res: +xe_eudebug_destroy_resources (d); +err_free: +kfree (d); + +return err; +} + +int xe_eudebug_connect_ioctl (struct drm_device *dev, +void *data, +struct drm_file *file) +{ +struct xe_device *xe =to_xe_device (dev); +struct drm_xe_eudebug_connect *const param = data; +int ret =0 ; + + ret =xe_eudebug_connect (xe, param); + +return ret; +} + +void xe_eudebug_init (struct xe_device *xe) +{ +spin_lock_init (&xe->eudebug.lock); +INIT_LIST_HEAD (&xe->eudebug.list); + + xe->eudebug.available = true; +} + +void xe_eudebug_fini (struct xe_device *xe) +{ +xe_assert (xe,list_empty_careful (&xe->eudebug.list)); +} + +static int send_open_event (struct xe_eudebug *d,u32 flags,const u64 handle, +const u64 seqno) +{ +struct xe_eudebug_event *event; +struct xe_eudebug_event_open *eo; + +if (!handle) +return -EINVAL ; + +if (XE_WARN_ON ((long )handle >=INT_MAX )) +return -EINVAL ; + + event =xe_eudebug_create_event (d,DRM_XE_EUDEBUG_EVENT_OPEN , seqno, + flags,sizeof (*eo),GFP_KERNEL ); +if (!event) +return -ENOMEM ; + + eo =cast_event (eo, event); + + write_member(struct drm_xe_eudebug_event_client, eo, + client_handle, handle); + +return xe_eudebug_queue_event (d, event); +} + +static int client_create_event (struct xe_eudebug *d,struct xe_file *xef) +{ +u64 seqno; +int ret; + + ret =xe_eudebug_add_handle (d,XE_EUDEBUG_RES_TYPE_CLIENT , xef, &seqno); +if (ret >0 ) + ret =send_open_event (d,DRM_XE_EUDEBUG_EVENT_CREATE , + ret, seqno); + +return ret; +} + +static int client_destroy_event (struct xe_eudebug *d,struct xe_file *xef) +{ +u64 seqno; +int ret; + + ret =xe_eudebug_remove_handle (d,XE_EUDEBUG_RES_TYPE_CLIENT , + xef, &seqno); +if (ret >0 ) + ret =send_open_event (d,DRM_XE_EUDEBUG_EVENT_DESTROY , + ret, seqno); + +return ret; +} + +#define xe_eudebug_event_put (_d, _err) ({ \ + if ((_err)) \ + xe_eudebug_disconnect((_d), (_err)); \ + xe_eudebug_put((_d)); \ + }) + +void xe_eudebug_file_open (struct xe_file *xef) +{ +struct xe_eudebug *d; + + d =xe_eudebug_get (xef); +if (!d) +return ; + +xe_eudebug_event_put (d,client_create_event (d, xef)); +} + +void xe_eudebug_file_close (struct xe_file *xef) +{ +struct xe_eudebug *d; + + d =xe_eudebug_get (xef); +if (!d) +return ; + +xe_eudebug_event_put (d,client_destroy_event (d, xef)); +} + +static int send_vm_event (struct xe_eudebug *d,u32 flags, +const u64 client_handle, +const u64 vm_handle, +const u64 seqno) +{ +struct xe_eudebug_event *event; +struct xe_eudebug_event_vm *e; + + event =xe_eudebug_create_event (d,DRM_XE_EUDEBUG_EVENT_VM , + seqno, flags,sizeof (*e),GFP_KERNEL ); +if (!event) +return -ENOMEM ; + + e =cast_event (e, event); + + write_member(struct drm_xe_eudebug_event_vm, e, client_handle, client_handle); + write_member(struct drm_xe_eudebug_event_vm, e, vm_handle, vm_handle); + +return xe_eudebug_queue_event (d, event); +} + +static int vm_create_event (struct xe_eudebug *d, +struct xe_file *xef,struct xe_vm *vm) +{ +int h_c, h_vm; +u64 seqno; +int ret; + + h_c =find_handle (d->res,XE_EUDEBUG_RES_TYPE_CLIENT , xef); +if (h_c <0 ) +return h_c; + +xe_eudebug_assert (d, h_c); + + h_vm =xe_eudebug_add_handle (d,XE_EUDEBUG_RES_TYPE_VM , vm, &seqno); +if (h_vm <=0 ) +return h_vm; + + ret =send_vm_event (d,DRM_XE_EUDEBUG_EVENT_CREATE , h_c, h_vm, seqno); + +return ret; +} + +static int vm_destroy_event (struct xe_eudebug *d, +struct xe_file *xef,struct xe_vm *vm) +{ +int h_c, h_vm; +u64 seqno; + + h_c =find_handle (d->res,XE_EUDEBUG_RES_TYPE_CLIENT , xef); +if (h_c <0 ) { +XE_WARN_ON ("no client found for vm" ); +eu_warn (d,"no client found for vm" ); +return h_c; + } + +xe_eudebug_assert (d, h_c); + + h_vm =xe_eudebug_remove_handle (d,XE_EUDEBUG_RES_TYPE_VM , vm, &seqno); +if (h_vm <=0 ) +return h_vm; + +return send_vm_event (d,DRM_XE_EUDEBUG_EVENT_DESTROY , h_c, h_vm, seqno); +} + +void xe_eudebug_vm_create (struct xe_file *xef,struct xe_vm *vm) +{ +struct xe_eudebug *d; + + d =xe_eudebug_get (xef); +if (!d) +return ; + +xe_eudebug_event_put (d,vm_create_event (d, xef, vm)); +} + +void xe_eudebug_vm_destroy (struct xe_file *xef,struct xe_vm *vm) +{ +struct xe_eudebug *d; + + d =xe_eudebug_get (xef); +if (!d) +return ; + +xe_eudebug_event_put (d,vm_destroy_event (d, xef, vm)); +}drivers/gpu/drm/xe/xe_eudebug.h new file +27
-0@@ -0,0 +1,27 @@ +/* SPDX-License-Identifier: MIT */ +/* +* Copyright © 2023 Intel Corporation +*/ + +#ifndef _XE_EUDEBUG_H_ + +struct drm_device ; +struct drm_file ; +struct xe_device ; +struct xe_file ; +struct xe_vm ; + +int xe_eudebug_connect_ioctl (struct drm_device *dev, +void *data, +struct drm_file *file); + +void xe_eudebug_init (struct xe_device *xe); +void xe_eudebug_fini (struct xe_device *xe); + +void xe_eudebug_file_open (struct xe_file *xef); +void xe_eudebug_file_close (struct xe_file *xef); + +void xe_eudebug_vm_create (struct xe_file *xef,struct xe_vm *vm); +void xe_eudebug_vm_destroy (struct xe_file *xef,struct xe_vm *vm); + +#endif drivers/gpu/drm/xe/xe_eudebug_types.h new file +169
-0@@ -0,0 +1,169 @@ +/* SPDX-License-Identifier: MIT */ +/* +* Copyright © 2023 Intel Corporation +*/ + +#ifndef __XE_EUDEBUG_TYPES_H_ + +#include <linux/mutex.h> +#include <linux/kref.h> +#include <linux/kfifo.h> +#include <linux/completion.h> +#include <linux/wait.h> +#include <linux/xarray.h> +#include <linux/rbtree.h> +#include <linux/rhashtable.h> + +#include <uapi/drm/xe_drm.h> + +struct xe_device ; +struct task_struct ; +struct xe_eudebug_event ; + +#define CONFIG_DRM_XE_DEBUGGER_EVENT_QUEUE_SIZE 64 + +/** +* struct xe_eudebug_handle - eudebug resource handle +*/ +struct xe_eudebug_handle { +/** @key: key value in rhashtable <key:id> */ +u64 key; + +/** @id: opaque handle id for xarray <id:key> */ +int id; + +/** @rh_head: rhashtable head */ +struct rhash_head rh_head; +}; + +/** +* struct xe_eudebug_resource - Resource map for one resource +*/ +struct xe_eudebug_resource { +/** @xa: xarrays for <id->key> */ +struct xarray xa; + +/** @rh rhashtable for <key->id> */ +struct rhashtable rh; +}; + +#define XE_EUDEBUG_RES_TYPE_CLIENT 0 +#define XE_EUDEBUG_RES_TYPE_VM 1 +#define XE_EUDEBUG_RES_TYPE_COUNT (XE_EUDEBUG_RES_TYPE_VM + 1) + +/** +* struct xe_eudebug_resources - eudebug resources for all types +*/ +struct xe_eudebug_resources { +/** @lock: guards access into rt */ +struct mutex lock; + +/** @rt: resource maps for all types */ +struct xe_eudebug_resource rt[XE_EUDEBUG_RES_TYPE_COUNT ]; +}; + +/** +* struct xe_eudebug - Top level struct for eudebug: the connection +*/ +struct xe_eudebug { +/** @ref: kref counter for this struct */ +struct kref ref; + +/** @rcu: rcu_head for rcu destruction */ +struct rcu_head rcu; + +/** @connection_link: our link into the xe_device:eudebug.list */ +struct list_head connection_link; + +struct { +/** @status: connected = 1, disconnected = error */ +#define XE_EUDEBUG_STATUS_CONNECTED 1 +int status; + +/** @lock: guards access to status */ +spinlock_t lock; + } connection; + +/** @xe: the parent device we are serving */ +struct xe_device *xe; + +/** @target_task: the task that we are debugging */ +struct task_struct *target_task; + +/** @res: the resource maps we track for target_task */ +struct xe_eudebug_resources *res; + +/** @session: session number for this connection (for logs) */ +u64 session; + +/** @events: kfifo queue of to-be-delivered events */ +struct { +/** @lock: guards access to fifo */ +spinlock_t lock; + +/** @fifo: queue of events pending */ + DECLARE_KFIFO(fifo, + struct xe_eudebug_event *, + CONFIG_DRM_XE_DEBUGGER_EVENT_QUEUE_SIZE); + +/** @write_done: waitqueue for signalling write to fifo */ +wait_queue_head_t write_done; + +/** @read_done: waitqueue for signalling read from fifo */ +wait_queue_head_t read_done; + +/** @event_seqno: seqno counter to stamp events for fifo */ +atomic_long_t seqno; + } events; + +}; + +/** +* struct xe_eudebug_event - Internal base event struct for eudebug +*/ +struct xe_eudebug_event { +/** @len: length of this event, including payload */ +u32 len; + +/** @type: message type */ +u16 type; + +/** @flags: message flags */ +u16 flags; + +/** @seqno: sequence number for ordering */ +u64 seqno; + +/** @reserved: reserved field MBZ */ +u64 reserved; + +/** @data: payload bytes */ +u8 data[]; +}; + +/** +* struct xe_eudebug_event_open - Internal event for client open/close +*/ +struct xe_eudebug_event_open { +/** @base: base event */ +struct xe_eudebug_event base; + +/** @client_handle: opaque handle for client */ +u64 client_handle; +}; + +/** +* struct xe_eudebug_event_vm - Internal event for vm open/close +*/ +struct xe_eudebug_event_vm { +/** @base: base event */ +struct xe_eudebug_event base; + +/** @client_handle: client containing the vm open/close */ +u64 client_handle; + +/** @vm_handle: vm handle it's open/close */ +u64 vm_handle; +}; + +#endif drivers/gpu/drm/xe/xe_vm.c +6
-1@@ -39,6 +39,7 @@#include "xe_trace_bo.h" #include "xe_wa.h" #include "xe_hmm.h" +#include "xe_eudebug.h" static struct drm_gem_object *xe_vm_obj (struct xe_vm *vm) {@@ -1818,6 +1819,8 @@ int xe_vm_create_ioctl(struct drm_device *dev, void *data, args->reserved[0 ] =xe_bo_main_addr (vm->pt_root[0 ]->bo,XE_PAGE_SIZE );#endif +xe_eudebug_vm_create (xef, vm); +return 0 ; err_free_id:@@ -1853,8 +1856,10 @@ int xe_vm_destroy_ioctl(struct drm_device *dev, void *data,xa_erase (&xef->vm.xa, args->vm_id);mutex_unlock (&xef->vm.lock); -if (!err) +if (!err) { +xe_eudebug_vm_destroy (xef, vm);xe_vm_close_and_put (vm); + }return err; }include/uapi/drm/xe_drm.h +21
-0@@ -102,6 +102,7 @@ extern "C" {#define DRM_XE_EXEC 0x09#define DRM_XE_WAIT_USER_FENCE 0x0a#define DRM_XE_OBSERVATION 0x0b +#define DRM_XE_EUDEBUG_CONNECT 0x0c/* Must be kept compact -- no holes */ @@ -117,6 +118,7 @@ extern "C" {#define DRM_IOCTL_XE_EXEC DRM_IOW(DRM_COMMAND_BASE + DRM_XE_EXEC, struct drm_xe_exec)#define DRM_IOCTL_XE_WAIT_USER_FENCE DRM_IOWR(DRM_COMMAND_BASE + DRM_XE_WAIT_USER_FENCE, struct drm_xe_wait_user_fence)#define DRM_IOCTL_XE_OBSERVATION DRM_IOW(DRM_COMMAND_BASE + DRM_XE_OBSERVATION, struct drm_xe_observation_param) +#define DRM_IOCTL_XE_EUDEBUG_CONNECT DRM_IOWR(DRM_COMMAND_BASE + DRM_XE_EUDEBUG_CONNECT, struct drm_xe_eudebug_connect)/** * DOC: Xe IOCTL Extensions @@ -1694,6 +1696,25 @@ struct drm_xe_oa_stream_info {__u64 reserved[3 ]; }; +/* +* Debugger ABI (ioctl and events) Version History: +* 0 - No debugger available +* 1 - Initial version +*/ +#define DRM_XE_EUDEBUG_VERSION 1 + +struct drm_xe_eudebug_connect { +/** @extensions: Pointer to the first extension struct, if any */ +__u64 extensions; + +__u64 pid;/* input: Target process ID */ +__u32 flags;/* MBZ */ + +__u32 version;/* output: current ABI (ioctl / events) version */ +}; + +#include "xe_drm_eudebug.h" +#if defined(__cplusplus) }#endif include/uapi/drm/xe_drm_eudebug.h new file +57
-0@@ -0,0 +1,57 @@ +/* SPDX-License-Identifier: MIT */ +/* +* Copyright © 2023 Intel Corporation +*/ + +#ifndef _UAPI_XE_DRM_EUDEBUG_H_ +#define _UAPI_XE_DRM_EUDEBUG_H_ + +#if defined(__cplusplus) +extern "C" { +#endif + +/** +* Do a eudebug event read for a debugger connection. +* +* This ioctl is available in debug version 1. +*/ +#define DRM_XE_EUDEBUG_IOCTL_READ_EVENT _IO('j', 0x0) + +/* XXX: Document events to match their internal counterparts when moved to xe_drm.h */ +struct drm_xe_eudebug_event { +__u32 len; + +__u16 type; +#define DRM_XE_EUDEBUG_EVENT_NONE 0 +#define DRM_XE_EUDEBUG_EVENT_READ 1 +#define DRM_XE_EUDEBUG_EVENT_OPEN 2 +#define DRM_XE_EUDEBUG_EVENT_VM 3 +#define DRM_XE_EUDEBUG_EVENT_MAX_EVENT DRM_XE_EUDEBUG_EVENT_VM + +__u16 flags; +#define DRM_XE_EUDEBUG_EVENT_CREATE (1 << 0) +#define DRM_XE_EUDEBUG_EVENT_DESTROY (1 << 1) +#define DRM_XE_EUDEBUG_EVENT_STATE_CHANGE (1 << 2) +#define DRM_XE_EUDEBUG_EVENT_NEED_ACK (1 << 3) +__u64 seqno; +__u64 reserved; +}; + +struct drm_xe_eudebug_event_client { +struct drm_xe_eudebug_event base; + +__u64 client_handle;/* This is unique per debug connection */ +}; + +struct drm_xe_eudebug_event_vm { +struct drm_xe_eudebug_event base; + +__u64 client_handle; +__u64 vm_handle; +}; + +#if defined(__cplusplus) +} +#endif + +#endif Signature · 1 line
2.34.1
Re: [PATCH 03/21] drm/xe/eudebug: Introduce eudebug support
Quoted 25 lines · Matthew Brost
Quoted 17 lines · Mika Kuoppala
With eudebug event interface, user space debugger process (like gdb) is able to keep track of resources created by another process (debuggee using drm/xe) and act upon these resources.
For example, debugger can find a client vm which contains isa/elf for a particular shader/eu-kernel and then inspect and modify it (for example installing a breakpoint).
Debugger first opens a connection to xe with a drm ioctl specifying target pid to connect. This returns an anon fd handle that can then be used to listen for events with dedicated ioctl.
This patch introduces eudebug connection and event queuing, adding client create/destroy and vm create/destroy events as a baseline. More events for full debugger operation are needed and those will be introduced in follow up patches.
Quoted 6 lines
The resource tracking parts are inspired by the work of Maciej Patelczyk on resource handling for i915. Chris Wilson suggested improvement of two ways mapping which makes it easy to use resource map as a definitive bookkeep of what resources are played to debugger in the discovery phase (on follow up patch).
Quoted 4 lines
Not a full review just an idea. Do you think it would be worthwhile to add a
Kconfigoption to enable EU debug support so if we don't care about EU debug it compiles out?
It is possible and it was done like this in i915. Test triage space will grow. On the other hand, we can isolate on hunting issues.
Quoted 6 lines
The layering would have to be fixed up to make this sane (e.g. all common Xe code calls into the EU layer so we don't have 'if
IS_ENABLED(CONFIG_EU_DEBUG)' sprinkled all over the place. Fixing up layering probably isn't bad idea anyways and as common Xe code really shouldn't open code EU debug specific things anyways. This would enforce good layering too.
Agreed that it would enforce better layering. Naturally we cant reach 100% here as there are some ioctl argument and null checks sprinkled. But perfect is enemy is good. We will push more into the xe_eudebug_* files and lets see how it look. Kconfig is trivial and icing on the cake after this.
-Mika
Quoted 1,633 lines
Matt
Quoted 1,631 lines
v2: - event printer removed (Maarten)
- trim down kfifo accessors (Maarten)
-xa_allocspurious locking removed (Maarten)
v3: - use list for clients (Matt)
v4: - avoid reporting clients too early (Maciej, VLK-56889)
v5: - return from wait if disconnect (Mika)
v6: - read waitqueue (Mika)
v7: - ENOTCONN early inread_event(Mika)
v8: - consolidate resource error handling (Mika, Maciej)
v9: - ptrace access check (Jonathan)
v10: - detach error handling (Dominik)Cc Maarten Lankhorst <[email protected]>Cc Lucas De Marchi <[email protected]>Cc Dominik Grzegorzek <[email protected]>Cc Andi Shyti <[email protected]>Cc Matt Roper <[email protected]>Signed-off-by Mika Kuoppala <[email protected]>Signed-off-by Maciej Patelczyk <[email protected]>--- drivers/gpu/drm/xe/Makefile | 3 +- drivers/gpu/drm/xe/xe_device.c | 24 + drivers/gpu/drm/xe/xe_device_types.h | 26 + drivers/gpu/drm/xe/xe_eudebug.c | 1093 +++++++++++++++++++++++++ drivers/gpu/drm/xe/xe_eudebug.h | 27 + drivers/gpu/drm/xe/xe_eudebug_types.h | 169 ++++ drivers/gpu/drm/xe/xe_vm.c | 7 +- include/uapi/drm/xe_drm.h | 21 + include/uapi/drm/xe_drm_eudebug.h | 57 ++ 9 files changed, 1425 insertions(+), 2 deletions(-) create mode 100644 drivers/gpu/drm/xe/xe_eudebug.c create mode 100644 drivers/gpu/drm/xe/xe_eudebug.h create mode 100644 drivers/gpu/drm/xe/xe_eudebug_types.h create mode 100644 include/uapi/drm/xe_drm_eudebug.hdrivers/gpu/drm/xe/Makefile +2
-1@@ -114,7 +114,8 @@ xe-y += xe_bb.o \ xe_vram_freq.o \ xe_wait_user_fence.o \ xe_wa.o \ - xe_wopcm.o + xe_wopcm.o \ + xe_eudebug.o xe-$(CONFIG_HMM_MIRROR) += xe_hmm.odrivers/gpu/drm/xe/xe_device.c +24
-0@@ -54,6 +54,7 @@#include "xe_vm.h" #include "xe_vram.h" #include "xe_wait_user_fence.h" +#include "xe_eudebug.h" #include "xe_wa.h" #include <generated/xe_wa_oob.h> @@ -100,6 +101,14 @@ static int xe_file_open(struct drm_device *dev, struct drm_file *file)put_task_struct (task); } +INIT_LIST_HEAD (&xef->link); + +mutex_lock (&xe->files.lock); +list_add_tail (&xef->link, &xe->files.list); +mutex_unlock (&xe->files.lock); + +xe_eudebug_file_open (xef); +return 0 ; }@@ -158,6 +167,12 @@ static void xe_file_close(struct drm_device *dev, struct drm_file *file)xe_pm_runtime_get (xe); +xe_eudebug_file_close (xef); + +mutex_lock (&xef->xe->files.lock); +list_del_init (&xef->link); +mutex_unlock (&xef->xe->files.lock); +/* * No need for exec_queue.lock here as there is no contention for it * when FD is closing as IOCTLs presumably can't be modifying the @@ -196,6 +211,7 @@ static const struct drm_ioctl_desc xe_ioctls[] = {DRM_IOCTL_DEF_DRV (XE_WAIT_USER_FENCE , xe_wait_user_fence_ioctl,DRM_RENDER_ALLOW ),DRM_IOCTL_DEF_DRV (XE_OBSERVATION , xe_observation_ioctl,DRM_RENDER_ALLOW ), + DRM_IOCTL_DEF_DRV(XE_EUDEBUG_CONNECT, xe_eudebug_connect_ioctl, DRM_RENDER_ALLOW), };static long xe_drm_ioctl (struct file *file,unsigned int cmd,unsigned long arg)@@ -285,6 +301,8 @@ static void xe_device_destroy(struct drm_device *dev, void *dummy) {struct xe_device *xe =to_xe_device (dev); +xe_eudebug_fini (xe); +if (xe->preempt_fence_wq)destroy_workqueue (xe->preempt_fence_wq);@@ -330,6 +348,9 @@ struct xe_device *xe_device_create(struct pci_dev *pdev,spin_lock_init (&xe->irq.lock);spin_lock_init (&xe->clients.lock); +drmm_mutex_init (&xe->drm, &xe->files.lock); +INIT_LIST_HEAD (&xe->files.list); +init_waitqueue_head (&xe->ufence_wq); err =drmm_mutex_init (&xe->drm, &xe->usm.lock);@@ -356,7 +377,10 @@ struct xe_device *xe_device_create(struct pci_dev *pdev,INIT_LIST_HEAD (&xe->pinned.external_vram);INIT_LIST_HEAD (&xe->pinned.evicted); +xe_eudebug_init (xe); + xe->preempt_fence_wq =alloc_ordered_workqueue ("xe-preempt-fence-wq" ,0 ); + xe->ordered_wq =alloc_ordered_workqueue ("xe-ordered-wq" ,0 ); xe->unordered_wq =alloc_workqueue ("xe-unordered-wq" ,0 ,0 ); if (!xe->ordered_wq || !xe->unordered_wq ||drivers/gpu/drm/xe/xe_device_types.h +26
-0@@ -355,6 +355,15 @@ struct xe_device {u64 count; } clients; +/** @files: xe_file list (opens) */ +struct { +/** @lock: Protects xe_file list */ +struct mutex lock; + +/** @list: xe file list */ +struct list_head list; + } files; +/** @usm: unified memory state */ struct {/** @usm.asid: convert a ASID to VM */ @@ -492,6 +501,20 @@ struct xe_device {u8 vm_inject_error_position;#endif +/** @debugger connection list and globals for device */ +struct { +/** @lock: protects the list of connections */ +spinlock_t lock; +/** @list: list of connections, aka debuggers */ +struct list_head list; + +/** @session_count: session counter to track connections */ +u64 session_count; + +/** @available: is the debugging functionality available */ +bool available; + } eudebug; +/* private: */ #if IS_ENABLED (CONFIG_DRM_XE_DISPLAY )@@ -596,6 +619,9 @@ struct xe_file {/** @refcount: ref count of this xe file */ struct kref refcount; + +/** @link: link into xe_device.client.list */ + struct list_head link; };#endif drivers/gpu/drm/xe/xe_eudebug.c new file +1093
-0@@ -0,0 +1,1093 @@ +// SPDX-License-Identifier: MIT +/* +* Copyright © 2023 Intel Corporation +*/ + +#include <linux/uaccess.h> + +#include <linux/anon_inodes.h> +#include <linux/poll.h> +#include <linux/delay.h> + +#include <drm/drm_managed.h> + +#include "xe_device.h" +#include "xe_assert.h" +#include "xe_macros.h" + +#include "xe_eudebug_types.h" +#include "xe_eudebug.h" + +/* +* If there is no detected event read by userspace, during this period, assume +* userspace problem and disconnect debugger to allow forward progress. +*/ +#define XE_EUDEBUG_NO_READ_DETECTED_TIMEOUT_MS (25 * 1000) + +#define for_each_debugger_rcu (debugger, head) \ + list_for_each_entry_rcu((debugger), (head), connection_link) +#define for_each_debugger (debugger, head) \ + list_for_each_entry((debugger), (head), connection_link) + +#define cast_event (T , event) container_of((event), typeof(*(T)), base) + +#define XE_EUDEBUG_DBG_STR "eudbg: %lld:%lu:%s (%d/%d) -> (%d/%d): " +#define XE_EUDEBUG_DBG_ARGS (d) (d)->session, \ + atomic_long_read(&(d)->events.seqno), \ + READ_ONCE(d->connection.status) <= 0 ? "disconnected" : "", \ + current->pid, \ + task_tgid_nr(current), \ + (d)->target_task->pid, \ + task_tgid_nr((d)->target_task) + +#define eu_err (d, fmt, ...) drm_err(&(d)->xe->drm, XE_EUDEBUG_DBG_STR # fmt, \ + XE_EUDEBUG_DBG_ARGS(d), ##__VA_ARGS__) +#define eu_warn (d, fmt, ...) drm_warn(&(d)->xe->drm, XE_EUDEBUG_DBG_STR # fmt, \ + XE_EUDEBUG_DBG_ARGS(d), ##__VA_ARGS__) +#define eu_dbg (d, fmt, ...) drm_dbg(&(d)->xe->drm, XE_EUDEBUG_DBG_STR # fmt, \ + XE_EUDEBUG_DBG_ARGS(d), ##__VA_ARGS__) + +#define xe_eudebug_assert (d, ...) xe_assert((d)->xe, ##__VA_ARGS__) + +#define struct_member (T , member) (((T *)0)->member) + +/* Keep 1:1 parity with uapi events */ +#define write_member (T_out, ptr, member, value) { \ + BUILD_BUG_ON(sizeof(*ptr) != sizeof(T_out)); \ + BUILD_BUG_ON(offsetof(typeof(*ptr), member) != \ + offsetof(typeof(T_out), member)); \ + BUILD_BUG_ON(sizeof(ptr->member) != sizeof(value)); \ + BUILD_BUG_ON(sizeof(struct_member(T_out, member)) != sizeof(value)); \ + BUILD_BUG_ON(!typecheck(typeof((ptr)->member), value)); \ + (ptr)->member = (value); \ + } + +static struct xe_eudebug_event * +event_fifo_pending (struct xe_eudebug *d) +{ +struct xe_eudebug_event *event; + +if (kfifo_peek (&d->events.fifo, &event)) +return event; + +return NULL ; +} + +/* +* This is racy as we dont take the lock for read but all the +* callsites can handle the race so we can live without lock. +*/ +__no_kcsan +static unsigned int +event_fifo_num_events_peek (const struct xe_eudebug *const d) +{ +return kfifo_len (&d->events.fifo); +} + +static bool +xe_eudebug_detached (struct xe_eudebug *d) +{ +int status; + +spin_lock (&d->connection.lock); + status = d->connection.status; +spin_unlock (&d->connection.lock); + +return status <=0 ; +} + +static int +xe_eudebug_error (const struct xe_eudebug *const d) +{ +const int status =READ_ONCE (d->connection.status); + +return status <=0 ? status :0 ; +} + +static unsigned int +event_fifo_has_events (struct xe_eudebug *d) +{ +if (xe_eudebug_detached (d)) +return 1 ; + +return event_fifo_num_events_peek (d); +} + +static const struct rhashtable_params rhash_res = { + .head_offset = offsetof(struct xe_eudebug_handle , rh_head), + .key_len = sizeof_field(struct xe_eudebug_handle, key), + .key_offset = offsetof(struct xe_eudebug_handle , key), + .automatic_shrinking = true, +}; + +static struct xe_eudebug_resource * +resource_from_type (struct xe_eudebug_resources *const res,const int t) +{ +return &res->rt[t]; +} + +static struct xe_eudebug_resources * +xe_eudebug_resources_alloc (void ) +{ +struct xe_eudebug_resources *res; +int err; +int i; + + res =kzalloc (sizeof (*res),GFP_ATOMIC ); +if (!res) +return ERR_PTR (-ENOMEM ); + +mutex_init (&res->lock); + +for (i =0 ; i <XE_EUDEBUG_RES_TYPE_COUNT ; i++) { +xa_init_flags (&res->rt[i].xa,XA_FLAGS_ALLOC1 ); + err =rhashtable_init (&res->rt[i].rh, &rhash_res); + +if (err) +break ; + } + +if (err) { +while (i--) { +xa_destroy (&res->rt[i].xa); +rhashtable_destroy (&res->rt[i].rh); + } + +kfree (res); +return ERR_PTR (err); + } + +return res; +} + +static void res_free_fn (void *ptr,void *arg) +{ +XE_WARN_ON (ptr); +kfree (ptr); +} + +static void +xe_eudebug_destroy_resources (struct xe_eudebug *d) +{ +struct xe_eudebug_resources *res = d->res; +struct xe_eudebug_handle *h; +unsigned long j; +int i; +int err; + +mutex_lock (&res->lock); +for (i =0 ; i <XE_EUDEBUG_RES_TYPE_COUNT ; i++) { +struct xe_eudebug_resource *r = &res->rt[i]; + +xa_for_each (&r->xa, j, h) { +struct xe_eudebug_handle *t; + + err =rhashtable_remove_fast (&r->rh, + &h->rh_head, + rhash_res); +xe_eudebug_assert (d, !err); + t =xa_erase (&r->xa, h->id); +xe_eudebug_assert (d, t == h); +kfree (t); + } + } +mutex_unlock (&res->lock); + +for (i =0 ; i <XE_EUDEBUG_RES_TYPE_COUNT ; i++) { +struct xe_eudebug_resource *r = &res->rt[i]; + +rhashtable_free_and_destroy (&r->rh, res_free_fn,NULL ); +xe_eudebug_assert (d,xa_empty (&r->xa)); +xa_destroy (&r->xa); + } + +mutex_destroy (&res->lock); + +kfree (res); +} + +static void xe_eudebug_free (struct kref *ref) +{ +struct xe_eudebug *d =container_of (ref,typeof (*d), ref); +struct xe_eudebug_event *event; + +while (kfifo_get (&d->events.fifo, &event)) +kfree (event); + +xe_eudebug_destroy_resources (d); +put_task_struct (d->target_task); + +xe_eudebug_assert (d, !kfifo_len (&d->events.fifo)); + +kfree_rcu (d, rcu); +} + +static void xe_eudebug_put (struct xe_eudebug *d) +{ +kref_put (&d->ref, xe_eudebug_free); +} + +static struct task_struct *find_get_target (const pid_t nr) +{ +struct task_struct *task; + +rcu_read_lock (); + task =pid_task (find_pid_ns (nr,task_active_pid_ns (current)),PIDTYPE_PID ); +if (task) +get_task_struct (task); +rcu_read_unlock (); + +return task; +} + +static int +xe_eudebug_attach (struct xe_device *xe,struct xe_eudebug *d, +const pid_t pid_nr) +{ +struct task_struct *target; +struct xe_eudebug *iter; +kuid_t uid =current_uid (); +int ret =0 ; + + target =find_get_target (pid_nr); +if (!target) +return -ENOENT ; + +if (!uid_eq (uid,task_uid (target)) && !capable (CAP_SYS_ADMIN )) { +put_task_struct (target); +return -EACCES ; + } + +XE_WARN_ON (d->connection.status !=0 ); + +spin_lock (&xe->eudebug.lock); +for_each_debugger (iter, &xe->eudebug.list) { +if (!same_thread_group (iter->target_task, target)) +continue ; + + ret = -EBUSY ; + } + +if (!ret && xe->eudebug.session_count +1 ==0 ) + ret = -ENOSPC ; + +if (!ret) { + d->connection.status =XE_EUDEBUG_STATUS_CONNECTED ; + d->xe = xe; + d->target_task = target; + d->session = ++xe->eudebug.session_count; +kref_get (&d->ref); +list_add_tail_rcu (&d->connection_link, &xe->eudebug.list); + } +spin_unlock (&xe->eudebug.lock); + +if (ret) +put_task_struct (target); + +return ret; +} + +static bool xe_eudebug_detach (struct xe_device *xe, +struct xe_eudebug *d, +const int err) +{ +bool detached = false; + +XE_WARN_ON (err >0 ); + +spin_lock (&d->connection.lock); +if (d->connection.status ==XE_EUDEBUG_STATUS_CONNECTED ) { + d->connection.status = err; + detached = true; + } +spin_unlock (&d->connection.lock); + +if (!detached) +return false; + +spin_lock (&xe->eudebug.lock); +list_del_rcu (&d->connection_link); +spin_unlock (&xe->eudebug.lock); + +eu_dbg (d,"session %lld detached with %d" , d->session, err); + +/* Our ref with the connection_link */ +xe_eudebug_put (d); + +return true; +} + +static int _xe_eudebug_disconnect (struct xe_eudebug *d, +const int err) +{ +wake_up_all (&d->events.write_done); +wake_up_all (&d->events.read_done); + +return xe_eudebug_detach (d->xe, d, err); +} + +#define xe_eudebug_disconnect (_d, _err) ({ \ + if (_xe_eudebug_disconnect((_d), (_err))) { \ + if ((_err) == 0 || (_err) == -ETIMEDOUT) \ + eu_dbg(d, "Session closed (%d)", (_err)); \ + else \ + eu_err(d, "Session disconnected, err = %d (%s:%d)", \ + (_err), __func__, __LINE__); \ + } \ +}) + +static int xe_eudebug_release (struct inode *inode,struct file *file) +{ +struct xe_eudebug *d = file->private_data; + +xe_eudebug_disconnect (d,0 ); +xe_eudebug_put (d); + +return 0 ; +} + +static __poll_t xe_eudebug_poll (struct file *file,poll_table *wait) +{ +struct xe_eudebug *const d = file->private_data; +__poll_t ret =0 ; + +poll_wait (file, &d->events.write_done, wait); + +if (xe_eudebug_detached (d)) { + ret |=EPOLLHUP ; +if (xe_eudebug_error (d)) + ret |=EPOLLERR ; + } + +if (event_fifo_num_events_peek (d)) + ret |=EPOLLIN ; + +return ret; +} + +static ssize_t xe_eudebug_read (struct file *file, +char __user *buf, +size_t count, +loff_t *ppos) +{ +return -EINVAL ; +} + +static struct xe_eudebug * +xe_eudebug_for_task_get (struct xe_device *xe, +struct task_struct *task) +{ +struct xe_eudebug *d, *iter; + + d =NULL ; + +rcu_read_lock (); +for_each_debugger_rcu (iter, &xe->eudebug.list) { +if (!same_thread_group (iter->target_task, task)) +continue ; + +if (kref_get_unless_zero (&iter->ref)) + d = iter; + +break ; + } +rcu_read_unlock (); + +return d; +} + +static struct task_struct *find_task_get (struct xe_file *xef) +{ +struct task_struct *task; +struct pid *pid; + +rcu_read_lock (); + pid =rcu_dereference (xef->drm->pid); + task =pid_task (pid,PIDTYPE_PID ); +if (task) +get_task_struct (task); +rcu_read_unlock (); + +return task; +} + +static struct xe_eudebug * +xe_eudebug_get (struct xe_file *xef) +{ +struct task_struct *task; +struct xe_eudebug *d; + + d =NULL ; + task =find_task_get (xef); +if (task) { + d =xe_eudebug_for_task_get (to_xe_device (xef->drm->minor->dev), + task); +put_task_struct (task); + } + +if (!d) +return NULL ; + +if (xe_eudebug_detached (d)) { +xe_eudebug_put (d); +return NULL ; + } + +return d; +} + +static int xe_eudebug_queue_event (struct xe_eudebug *d, +struct xe_eudebug_event *event) +{ +const u64 wait_jiffies =msecs_to_jiffies (1000 ); +u64 last_read_detected_ts, last_head_seqno, start_ts; + +xe_eudebug_assert (d, event->len >sizeof (struct xe_eudebug_event )); +xe_eudebug_assert (d, event->type); +xe_eudebug_assert (d, event->type !=DRM_XE_EUDEBUG_EVENT_READ ); + + start_ts =ktime_get (); + last_read_detected_ts = start_ts; + last_head_seqno =0 ; + +do { +struct xe_eudebug_event *head; +u64 head_seqno; +bool was_queued; + +if (xe_eudebug_detached (d)) +break ; + +spin_lock (&d->events.lock); + head =event_fifo_pending (d); +if (head) + head_seqno = event->seqno; +else + head_seqno =0 ; + + was_queued =kfifo_in (&d->events.fifo, &event,1 ); +spin_unlock (&d->events.lock); + +wake_up_all (&d->events.write_done); + +if (was_queued) { + event =NULL ; +break ; + } + +XE_WARN_ON (!head_seqno); + +/* If we detect progress, restart timeout */ +if (last_head_seqno != head_seqno) + last_read_detected_ts =ktime_get (); + + last_head_seqno = head_seqno; + +wait_event_interruptible_timeout (d->events.read_done, + !kfifo_is_full (&d->events.fifo), + wait_jiffies); + + }while (ktime_ms_delta (ktime_get (), last_read_detected_ts) < +XE_EUDEBUG_NO_READ_DETECTED_TIMEOUT_MS ); + +if (event) { +eu_dbg (d, +"event %llu queue failed (blocked %lld ms, avail %d)" , + event ? event->seqno :0 , +ktime_ms_delta (ktime_get (), start_ts), +kfifo_avail (&d->events.fifo)); + +kfree (event); + +return -ETIMEDOUT ; + } + +return 0 ; +} + +static struct xe_eudebug_handle * +alloc_handle (const int type,const u64 key) +{ +struct xe_eudebug_handle *h; + + h =kzalloc (sizeof (*h),GFP_ATOMIC ); +if (!h) +return NULL ; + + h->key = key; + +return h; +} + +static struct xe_eudebug_handle * +__find_handle (struct xe_eudebug_resource *r, +const u64 key) +{ +struct xe_eudebug_handle *h; + + h =rhashtable_lookup_fast (&r->rh, + &key, + rhash_res); +return h; +} + +static int find_handle (struct xe_eudebug_resources *res, +const int type, +const void *p) +{ +const u64 key = (u64 )p; +struct xe_eudebug_resource *r; +struct xe_eudebug_handle *h; +int id; + +if (XE_WARN_ON (!key)) +return -EINVAL ; + + r =resource_from_type (res, type); + +mutex_lock (&res->lock); + h =__find_handle (r, key); + id = h ? h->id : -ENOENT ; +mutex_unlock (&res->lock); + +return id; +} + +static int _xe_eudebug_add_handle (struct xe_eudebug *d, +int type, +void *p, +u64 *seqno, +int *handle) +{ +const u64 key = (u64 )p; +struct xe_eudebug_resource *r; +struct xe_eudebug_handle *h, *o; +int err; + +if (XE_WARN_ON (p ==NULL )) +return -EINVAL ; + +if (xe_eudebug_detached (d)) +return -ENOTCONN ; + + h =alloc_handle (type, key); +if (!h) +return -ENOMEM ; + + r =resource_from_type (d->res, type); + +mutex_lock (&d->res->lock); + o =__find_handle (r, key); +if (!o) { + err =xa_alloc (&r->xa, &h->id, h, xa_limit_31b,GFP_KERNEL ); + +if (h->id >=INT_MAX ) { +xa_erase (&r->xa, h->id); + err = -ENOSPC ; + } + +if (!err) + err =rhashtable_insert_fast (&r->rh, + &h->rh_head, + rhash_res); + +if (err) { +xa_erase (&r->xa, h->id); + }else { +if (seqno) + *seqno =atomic_long_inc_return (&d->events.seqno); + } + }else { +xe_eudebug_assert (d, o->id); + err = -EEXIST ; + } +mutex_unlock (&d->res->lock); + +if (handle) + *handle = o ? o->id : h->id; + +if (err) { +kfree (h); +XE_WARN_ON (err >0 ); +return err; + } + +xe_eudebug_assert (d, h->id); + +return h->id; +} + +static int xe_eudebug_add_handle (struct xe_eudebug *d, +int type, +void *p, +u64 *seqno) +{ +int ret; + + ret =_xe_eudebug_add_handle (d, type, p, seqno,NULL ); +if (ret == -EEXIST || ret == -ENOTCONN ) { +eu_dbg (d,"%d on adding %d" , ret, type); +return 0 ; + } + +if (ret <0 ) +xe_eudebug_disconnect (d, ret); + +return ret; +} + +static int _xe_eudebug_remove_handle (struct xe_eudebug *d,int type,void *p, +u64 *seqno) +{ +const u64 key = (u64 )p; +struct xe_eudebug_resource *r; +struct xe_eudebug_handle *h, *xa_h; +int ret; + +if (XE_WARN_ON (!key)) +return -EINVAL ; + +if (xe_eudebug_detached (d)) +return -ENOTCONN ; + + r =resource_from_type (d->res, type); + +mutex_lock (&d->res->lock); + h =__find_handle (r, key); +if (h) { + ret =rhashtable_remove_fast (&r->rh, + &h->rh_head, + rhash_res); +xe_eudebug_assert (d, !ret); + xa_h =xa_erase (&r->xa, h->id); +xe_eudebug_assert (d, xa_h == h); +if (!ret) { + ret = h->id; +if (seqno) + *seqno =atomic_long_inc_return (&d->events.seqno); + } + }else { + ret = -ENOENT ; + } +mutex_unlock (&d->res->lock); + +kfree (h); + +xe_eudebug_assert (d, ret); + +return ret; +} + +static int xe_eudebug_remove_handle (struct xe_eudebug *d,int type,void *p, +u64 *seqno) +{ +int ret; + + ret =_xe_eudebug_remove_handle (d, type, p, seqno); +if (ret == -ENOENT || ret == -ENOTCONN ) { +eu_dbg (d,"%d on removing %d" , ret, type); +return 0 ; + } + +if (ret <0 ) +xe_eudebug_disconnect (d, ret); + +return ret; +} + +static struct xe_eudebug_event * +__xe_eudebug_create_event (struct xe_eudebug *d, +u64 seqno,u16 type,u16 flags,u32 len,gfp_t gfp) +{ +struct xe_eudebug_event *event; + +xe_eudebug_assert (d, len >sizeof (*event)); + + event =kzalloc (len, gfp); +if (!event) +return NULL ; + + event->type = type; + event->flags = flags; + event->len = len; + event->seqno = seqno; + +return event; +} + +static struct xe_eudebug_event * +xe_eudebug_create_event (struct xe_eudebug *d,u16 type,u64 seqno,u16 flags, +u32 len,gfp_t gfp) +{ +return __xe_eudebug_create_event (d, seqno, type, flags, len, gfp); +} + +static long xe_eudebug_read_event (struct xe_eudebug *d, +const u64 arg, +const bool wait) +{ +struct xe_device *xe = d->xe; +struct drm_xe_eudebug_event __user *const user_orig = +u64_to_user_ptr (arg); +struct drm_xe_eudebug_event user_event; +struct xe_eudebug_event *event; +long ret =0 ; + +if (XE_IOCTL_DBG (xe,copy_from_user (&user_event, user_orig,sizeof (user_event)))) +return -EFAULT ; + +if (XE_IOCTL_DBG (xe, !user_event.type)) +return -EINVAL ; + +if (XE_IOCTL_DBG (xe, user_event.type >DRM_XE_EUDEBUG_EVENT_MAX_EVENT )) +return -EINVAL ; + +if (XE_IOCTL_DBG (xe, user_event.type !=DRM_XE_EUDEBUG_EVENT_READ )) +return -EINVAL ; + +if (XE_IOCTL_DBG (xe, user_event.len <sizeof (*user_orig))) +return -EINVAL ; + +if (XE_IOCTL_DBG (xe, user_event.flags)) +return -EINVAL ; + +if (XE_IOCTL_DBG (xe, user_event.reserved)) +return -EINVAL ; + +/* XXX: define wait time in connect arguments ? */ +if (wait) { + ret =wait_event_interruptible_timeout (d->events.write_done, +event_fifo_has_events (d), +msecs_to_jiffies (5 *1000 )); + +if (XE_IOCTL_DBG (xe, ret <0 )) +return ret; + } + + ret =0 ; +spin_lock (&d->events.lock); + event =event_fifo_pending (d); +if (event) { +if (user_event.len < event->len) { + ret = -EMSGSIZE ; + }else if (!kfifo_out (&d->events.fifo, &event,1 )) { +eu_warn (d,"internal fifo corruption" ); + ret = -ENOTCONN ; + } + } +spin_unlock (&d->events.lock); + +wake_up_all (&d->events.read_done); + +if (XE_IOCTL_DBG (xe, ret)) +return ret; + +if (!event) { +if (xe_eudebug_detached (d)) +return -ENOTCONN ; +if (!wait) +return -EAGAIN ; + +return -ENOENT ; + } + +if (copy_to_user (user_orig, event, event->len)) + ret = -EFAULT ; +else +eu_dbg (d,"event read: type=%u, flags=0x%x, seqno=%llu" , event->type, + event->flags, event->seqno); + +kfree (event); + +return ret; +} + +static long xe_eudebug_ioctl (struct file *file, +unsigned int cmd, +unsigned long arg) +{ +struct xe_eudebug *const d = file->private_data; +long ret; + +switch (cmd) { +case DRM_XE_EUDEBUG_IOCTL_READ_EVENT : + ret =xe_eudebug_read_event (d, arg, + !(file->f_flags &O_NONBLOCK )); +break ; + +default : + ret = -EINVAL ; + } + +return ret; +} + +static const struct file_operations fops = { + .owner =THIS_MODULE , + .llseek = no_llseek, + .release = xe_eudebug_release, + .poll = xe_eudebug_poll, + .read = xe_eudebug_read, + .unlocked_ioctl = xe_eudebug_ioctl, +}; + +static int +xe_eudebug_connect (struct xe_device *xe, +struct drm_xe_eudebug_connect *param) +{ +const u64 known_open_flags =0 ; +unsigned long f_flags =0 ; +struct xe_eudebug *d; +int fd, err; + +if (param->extensions) +return -EINVAL ; + +if (!param->pid) +return -EINVAL ; + +if (param->flags & ~known_open_flags) +return -EINVAL ; + +if (param->version && param->version !=DRM_XE_EUDEBUG_VERSION ) +return -EINVAL ; + + param->version =DRM_XE_EUDEBUG_VERSION ; + +if (!xe->eudebug.available) +return -EOPNOTSUPP ; + + d =kzalloc (sizeof (*d),GFP_KERNEL ); +if (!d) +return -ENOMEM ; + +kref_init (&d->ref); +spin_lock_init (&d->connection.lock); +init_waitqueue_head (&d->events.write_done); +init_waitqueue_head (&d->events.read_done); + +spin_lock_init (&d->events.lock); +INIT_KFIFO (d->events.fifo); + + d->res =xe_eudebug_resources_alloc (); +if (IS_ERR (d->res)) { + err =PTR_ERR (d->res); + goto err_free; + } + + err =xe_eudebug_attach (xe, d, param->pid); +if (err) + goto err_free_res; + + fd =anon_inode_getfd ("[xe_eudebug]" , &fops, d, f_flags); +if (fd <0 ) { + err = fd; + goto err_detach; + } + +eu_dbg (d,"connected session %lld" , d->session); + +return fd; + +err_detach: +xe_eudebug_detach (xe, d, err); +err_free_res: +xe_eudebug_destroy_resources (d); +err_free: +kfree (d); + +return err; +} + +int xe_eudebug_connect_ioctl (struct drm_device *dev, +void *data, +struct drm_file *file) +{ +struct xe_device *xe =to_xe_device (dev); +struct drm_xe_eudebug_connect *const param = data; +int ret =0 ; + + ret =xe_eudebug_connect (xe, param); + +return ret; +} + +void xe_eudebug_init (struct xe_device *xe) +{ +spin_lock_init (&xe->eudebug.lock); +INIT_LIST_HEAD (&xe->eudebug.list); + + xe->eudebug.available = true; +} + +void xe_eudebug_fini (struct xe_device *xe) +{ +xe_assert (xe,list_empty_careful (&xe->eudebug.list)); +} + +static int send_open_event (struct xe_eudebug *d,u32 flags,const u64 handle, +const u64 seqno) +{ +struct xe_eudebug_event *event; +struct xe_eudebug_event_open *eo; + +if (!handle) +return -EINVAL ; + +if (XE_WARN_ON ((long )handle >=INT_MAX )) +return -EINVAL ; + + event =xe_eudebug_create_event (d,DRM_XE_EUDEBUG_EVENT_OPEN , seqno, + flags,sizeof (*eo),GFP_KERNEL ); +if (!event) +return -ENOMEM ; + + eo =cast_event (eo, event); + + write_member(struct drm_xe_eudebug_event_client, eo, + client_handle, handle); + +return xe_eudebug_queue_event (d, event); +} + +static int client_create_event (struct xe_eudebug *d,struct xe_file *xef) +{ +u64 seqno; +int ret; + + ret =xe_eudebug_add_handle (d,XE_EUDEBUG_RES_TYPE_CLIENT , xef, &seqno); +if (ret >0 ) + ret =send_open_event (d,DRM_XE_EUDEBUG_EVENT_CREATE , + ret, seqno); + +return ret; +} + +static int client_destroy_event (struct xe_eudebug *d,struct xe_file *xef) +{ +u64 seqno; +int ret; + + ret =xe_eudebug_remove_handle (d,XE_EUDEBUG_RES_TYPE_CLIENT , + xef, &seqno); +if (ret >0 ) + ret =send_open_event (d,DRM_XE_EUDEBUG_EVENT_DESTROY , + ret, seqno); + +return ret; +} + +#define xe_eudebug_event_put (_d, _err) ({ \ + if ((_err)) \ + xe_eudebug_disconnect((_d), (_err)); \ + xe_eudebug_put((_d)); \ + }) + +void xe_eudebug_file_open (struct xe_file *xef) +{ +struct xe_eudebug *d; + + d =xe_eudebug_get (xef); +if (!d) +return ; + +xe_eudebug_event_put (d,client_create_event (d, xef)); +} + +void xe_eudebug_file_close (struct xe_file *xef) +{ +struct xe_eudebug *d; + + d =xe_eudebug_get (xef); +if (!d) +return ; + +xe_eudebug_event_put (d,client_destroy_event (d, xef)); +} + +static int send_vm_event (struct xe_eudebug *d,u32 flags, +const u64 client_handle, +const u64 vm_handle, +const u64 seqno) +{ +struct xe_eudebug_event *event; +struct xe_eudebug_event_vm *e; + + event =xe_eudebug_create_event (d,DRM_XE_EUDEBUG_EVENT_VM , + seqno, flags,sizeof (*e),GFP_KERNEL ); +if (!event) +return -ENOMEM ; + + e =cast_event (e, event); + + write_member(struct drm_xe_eudebug_event_vm, e, client_handle, client_handle); + write_member(struct drm_xe_eudebug_event_vm, e, vm_handle, vm_handle); + +return xe_eudebug_queue_event (d, event); +} + +static int vm_create_event (struct xe_eudebug *d, +struct xe_file *xef,struct xe_vm *vm) +{ +int h_c, h_vm; +u64 seqno; +int ret; + + h_c =find_handle (d->res,XE_EUDEBUG_RES_TYPE_CLIENT , xef); +if (h_c <0 ) +return h_c; + +xe_eudebug_assert (d, h_c); + + h_vm =xe_eudebug_add_handle (d,XE_EUDEBUG_RES_TYPE_VM , vm, &seqno); +if (h_vm <=0 ) +return h_vm; + + ret =send_vm_event (d,DRM_XE_EUDEBUG_EVENT_CREATE , h_c, h_vm, seqno); + +return ret; +} + +static int vm_destroy_event (struct xe_eudebug *d, +struct xe_file *xef,struct xe_vm *vm) +{ +int h_c, h_vm; +u64 seqno; + + h_c =find_handle (d->res,XE_EUDEBUG_RES_TYPE_CLIENT , xef); +if (h_c <0 ) { +XE_WARN_ON ("no client found for vm" ); +eu_warn (d,"no client found for vm" ); +return h_c; + } + +xe_eudebug_assert (d, h_c); + + h_vm =xe_eudebug_remove_handle (d,XE_EUDEBUG_RES_TYPE_VM , vm, &seqno); +if (h_vm <=0 ) +return h_vm; + +return send_vm_event (d,DRM_XE_EUDEBUG_EVENT_DESTROY , h_c, h_vm, seqno); +} + +void xe_eudebug_vm_create (struct xe_file *xef,struct xe_vm *vm) +{ +struct xe_eudebug *d; + + d =xe_eudebug_get (xef); +if (!d) +return ; + +xe_eudebug_event_put (d,vm_create_event (d, xef, vm)); +} + +void xe_eudebug_vm_destroy (struct xe_file *xef,struct xe_vm *vm) +{ +struct xe_eudebug *d; + + d =xe_eudebug_get (xef); +if (!d) +return ; + +xe_eudebug_event_put (d,vm_destroy_event (d, xef, vm)); +}drivers/gpu/drm/xe/xe_eudebug.h new file +27
-0@@ -0,0 +1,27 @@ +/* SPDX-License-Identifier: MIT */ +/* +* Copyright © 2023 Intel Corporation +*/ + +#ifndef _XE_EUDEBUG_H_ + +struct drm_device ; +struct drm_file ; +struct xe_device ; +struct xe_file ; +struct xe_vm ; + +int xe_eudebug_connect_ioctl (struct drm_device *dev, +void *data, +struct drm_file *file); + +void xe_eudebug_init (struct xe_device *xe); +void xe_eudebug_fini (struct xe_device *xe); + +void xe_eudebug_file_open (struct xe_file *xef); +void xe_eudebug_file_close (struct xe_file *xef); + +void xe_eudebug_vm_create (struct xe_file *xef,struct xe_vm *vm); +void xe_eudebug_vm_destroy (struct xe_file *xef,struct xe_vm *vm); + +#endif drivers/gpu/drm/xe/xe_eudebug_types.h new file +169
-0@@ -0,0 +1,169 @@ +/* SPDX-License-Identifier: MIT */ +/* +* Copyright © 2023 Intel Corporation +*/ + +#ifndef __XE_EUDEBUG_TYPES_H_ + +#include <linux/mutex.h> +#include <linux/kref.h> +#include <linux/kfifo.h> +#include <linux/completion.h> +#include <linux/wait.h> +#include <linux/xarray.h> +#include <linux/rbtree.h> +#include <linux/rhashtable.h> + +#include <uapi/drm/xe_drm.h> + +struct xe_device ; +struct task_struct ; +struct xe_eudebug_event ; + +#define CONFIG_DRM_XE_DEBUGGER_EVENT_QUEUE_SIZE 64 + +/** +* struct xe_eudebug_handle - eudebug resource handle +*/ +struct xe_eudebug_handle { +/** @key: key value in rhashtable <key:id> */ +u64 key; + +/** @id: opaque handle id for xarray <id:key> */ +int id; + +/** @rh_head: rhashtable head */ +struct rhash_head rh_head; +}; + +/** +* struct xe_eudebug_resource - Resource map for one resource +*/ +struct xe_eudebug_resource { +/** @xa: xarrays for <id->key> */ +struct xarray xa; + +/** @rh rhashtable for <key->id> */ +struct rhashtable rh; +}; + +#define XE_EUDEBUG_RES_TYPE_CLIENT 0 +#define XE_EUDEBUG_RES_TYPE_VM 1 +#define XE_EUDEBUG_RES_TYPE_COUNT (XE_EUDEBUG_RES_TYPE_VM + 1) + +/** +* struct xe_eudebug_resources - eudebug resources for all types +*/ +struct xe_eudebug_resources { +/** @lock: guards access into rt */ +struct mutex lock; + +/** @rt: resource maps for all types */ +struct xe_eudebug_resource rt[XE_EUDEBUG_RES_TYPE_COUNT ]; +}; + +/** +* struct xe_eudebug - Top level struct for eudebug: the connection +*/ +struct xe_eudebug { +/** @ref: kref counter for this struct */ +struct kref ref; + +/** @rcu: rcu_head for rcu destruction */ +struct rcu_head rcu; + +/** @connection_link: our link into the xe_device:eudebug.list */ +struct list_head connection_link; + +struct { +/** @status: connected = 1, disconnected = error */ +#define XE_EUDEBUG_STATUS_CONNECTED 1 +int status; + +/** @lock: guards access to status */ +spinlock_t lock; + } connection; + +/** @xe: the parent device we are serving */ +struct xe_device *xe; + +/** @target_task: the task that we are debugging */ +struct task_struct *target_task; + +/** @res: the resource maps we track for target_task */ +struct xe_eudebug_resources *res; + +/** @session: session number for this connection (for logs) */ +u64 session; + +/** @events: kfifo queue of to-be-delivered events */ +struct { +/** @lock: guards access to fifo */ +spinlock_t lock; + +/** @fifo: queue of events pending */ + DECLARE_KFIFO(fifo, + struct xe_eudebug_event *, + CONFIG_DRM_XE_DEBUGGER_EVENT_QUEUE_SIZE); + +/** @write_done: waitqueue for signalling write to fifo */ +wait_queue_head_t write_done; + +/** @read_done: waitqueue for signalling read from fifo */ +wait_queue_head_t read_done; + +/** @event_seqno: seqno counter to stamp events for fifo */ +atomic_long_t seqno; + } events; + +}; + +/** +* struct xe_eudebug_event - Internal base event struct for eudebug +*/ +struct xe_eudebug_event { +/** @len: length of this event, including payload */ +u32 len; + +/** @type: message type */ +u16 type; + +/** @flags: message flags */ +u16 flags; + +/** @seqno: sequence number for ordering */ +u64 seqno; + +/** @reserved: reserved field MBZ */ +u64 reserved; + +/** @data: payload bytes */ +u8 data[]; +}; + +/** +* struct xe_eudebug_event_open - Internal event for client open/close +*/ +struct xe_eudebug_event_open { +/** @base: base event */ +struct xe_eudebug_event base; + +/** @client_handle: opaque handle for client */ +u64 client_handle; +}; + +/** +* struct xe_eudebug_event_vm - Internal event for vm open/close +*/ +struct xe_eudebug_event_vm { +/** @base: base event */ +struct xe_eudebug_event base; + +/** @client_handle: client containing the vm open/close */ +u64 client_handle; + +/** @vm_handle: vm handle it's open/close */ +u64 vm_handle; +}; + +#endif drivers/gpu/drm/xe/xe_vm.c +6
-1@@ -39,6 +39,7 @@#include "xe_trace_bo.h" #include "xe_wa.h" #include "xe_hmm.h" +#include "xe_eudebug.h" static struct drm_gem_object *xe_vm_obj (struct xe_vm *vm) {@@ -1818,6 +1819,8 @@ int xe_vm_create_ioctl(struct drm_device *dev, void *data, args->reserved[0 ] =xe_bo_main_addr (vm->pt_root[0 ]->bo,XE_PAGE_SIZE );#endif +xe_eudebug_vm_create (xef, vm); +return 0 ; err_free_id:@@ -1853,8 +1856,10 @@ int xe_vm_destroy_ioctl(struct drm_device *dev, void *data,xa_erase (&xef->vm.xa, args->vm_id);mutex_unlock (&xef->vm.lock); -if (!err) +if (!err) { +xe_eudebug_vm_destroy (xef, vm);xe_vm_close_and_put (vm); + }return err; }include/uapi/drm/xe_drm.h +21
-0@@ -102,6 +102,7 @@ extern "C" {#define DRM_XE_EXEC 0x09#define DRM_XE_WAIT_USER_FENCE 0x0a#define DRM_XE_OBSERVATION 0x0b +#define DRM_XE_EUDEBUG_CONNECT 0x0c/* Must be kept compact -- no holes */ @@ -117,6 +118,7 @@ extern "C" {#define DRM_IOCTL_XE_EXEC DRM_IOW(DRM_COMMAND_BASE + DRM_XE_EXEC, struct drm_xe_exec)#define DRM_IOCTL_XE_WAIT_USER_FENCE DRM_IOWR(DRM_COMMAND_BASE + DRM_XE_WAIT_USER_FENCE, struct drm_xe_wait_user_fence)#define DRM_IOCTL_XE_OBSERVATION DRM_IOW(DRM_COMMAND_BASE + DRM_XE_OBSERVATION, struct drm_xe_observation_param) +#define DRM_IOCTL_XE_EUDEBUG_CONNECT DRM_IOWR(DRM_COMMAND_BASE + DRM_XE_EUDEBUG_CONNECT, struct drm_xe_eudebug_connect)/** * DOC: Xe IOCTL Extensions @@ -1694,6 +1696,25 @@ struct drm_xe_oa_stream_info {__u64 reserved[3 ]; }; +/* +* Debugger ABI (ioctl and events) Version History: +* 0 - No debugger available +* 1 - Initial version +*/ +#define DRM_XE_EUDEBUG_VERSION 1 + +struct drm_xe_eudebug_connect { +/** @extensions: Pointer to the first extension struct, if any */ +__u64 extensions; + +__u64 pid;/* input: Target process ID */ +__u32 flags;/* MBZ */ + +__u32 version;/* output: current ABI (ioctl / events) version */ +}; + +#include "xe_drm_eudebug.h" +#if defined(__cplusplus) }#endif include/uapi/drm/xe_drm_eudebug.h new file +57
-0@@ -0,0 +1,57 @@ +/* SPDX-License-Identifier: MIT */ +/* +* Copyright © 2023 Intel Corporation +*/ + +#ifndef _UAPI_XE_DRM_EUDEBUG_H_ +#define _UAPI_XE_DRM_EUDEBUG_H_ + +#if defined(__cplusplus) +extern "C" { +#endif + +/** +* Do a eudebug event read for a debugger connection. +* +* This ioctl is available in debug version 1. +*/ +#define DRM_XE_EUDEBUG_IOCTL_READ_EVENT _IO('j', 0x0) + +/* XXX: Document events to match their internal counterparts when moved to xe_drm.h */ +struct drm_xe_eudebug_event { +__u32 len; + +__u16 type; +#define DRM_XE_EUDEBUG_EVENT_NONE 0 +#define DRM_XE_EUDEBUG_EVENT_READ 1 +#define DRM_XE_EUDEBUG_EVENT_OPEN 2 +#define DRM_XE_EUDEBUG_EVENT_VM 3 +#define DRM_XE_EUDEBUG_EVENT_MAX_EVENT DRM_XE_EUDEBUG_EVENT_VM + +__u16 flags; +#define DRM_XE_EUDEBUG_EVENT_CREATE (1 << 0) +#define DRM_XE_EUDEBUG_EVENT_DESTROY (1 << 1) +#define DRM_XE_EUDEBUG_EVENT_STATE_CHANGE (1 << 2) +#define DRM_XE_EUDEBUG_EVENT_NEED_ACK (1 << 3) +__u64 seqno; +__u64 reserved; +}; + +struct drm_xe_eudebug_event_client { +struct drm_xe_eudebug_event base; + +__u64 client_handle;/* This is unique per debug connection */ +}; + +struct drm_xe_eudebug_event_vm { +struct drm_xe_eudebug_event base; + +__u64 client_handle; +__u64 vm_handle; +}; + +#if defined(__cplusplus) +} +#endif + +#endif Signature · 1 line
2.34.1
Matthew Brost
maintainerRe: [PATCH 03/21] drm/xe/eudebug: Introduce eudebug support
Quoted 27 lines · Mika Kuoppala
Quoted 25 lines · Matthew Brost
Quoted 23 lines · Mika Kuoppala
With eudebug event interface, user space debugger process (like gdb) is able to keep track of resources created by another process (debuggee using drm/xe) and act upon these resources.
For example, debugger can find a client vm which contains isa/elf for a particular shader/eu-kernel and then inspect and modify it (for example installing a breakpoint).
Debugger first opens a connection to xe with a drm ioctl specifying target pid to connect. This returns an anon fd handle that can then be used to listen for events with dedicated ioctl.
This patch introduces eudebug connection and event queuing, adding client create/destroy and vm create/destroy events as a baseline. More events for full debugger operation are needed and those will be introduced in follow up patches.
The resource tracking parts are inspired by the work of Maciej Patelczyk on resource handling for i915. Chris Wilson suggested improvement of two ways mapping which makes it easy to use resource map as a definitive bookkeep of what resources are played to debugger in the discovery phase (on follow up patch).
Quoted 8 lines
Quoted 4 lines
Not a full review just an idea. Do you think it would be worthwhile to add a
Kconfigoption to enable EU debug support so if we don't care about EU debug it compiles out?It is possible and it was done like this in i915. Test triage space will grow. On the other hand, we can isolate on hunting issues.
Yes, too many Kconfig options is always a concern wrt to testing.
Quoted 7 lines
Quoted 6 lines
The layering would have to be fixed up to make this sane (e.g. all common Xe code calls into the EU layer so we don't have 'if
IS_ENABLED(CONFIG_EU_DEBUG)' sprinkled all over the place. Fixing up layering probably isn't bad idea anyways and as common Xe code really shouldn't open code EU debug specific things anyways. This would enforce good layering too.
Quoted 6 lines
Agreed that it would enforce better layering. Naturally we cant reach 100% here as there are some ioctl argument and null checks sprinkled. But perfect is enemy is good. We will push more into the xe_eudebug_* files and lets see how it look.
Kconfigis trivial and icing on the cake after this.
Sounds good.
Matt
Quoted 1,635 lines
-Mika
Quoted 1,633 lines
Matt
Quoted 1,631 lines
v2: - event printer removed (Maarten)
- trim down kfifo accessors (Maarten)
-xa_allocspurious locking removed (Maarten)
v3: - use list for clients (Matt)
v4: - avoid reporting clients too early (Maciej, VLK-56889)
v5: - return from wait if disconnect (Mika)
v6: - read waitqueue (Mika)
v7: - ENOTCONN early inread_event(Mika)
v8: - consolidate resource error handling (Mika, Maciej)
v9: - ptrace access check (Jonathan)
v10: - detach error handling (Dominik)Cc Maarten Lankhorst <[email protected]>Cc Lucas De Marchi <[email protected]>Cc Dominik Grzegorzek <[email protected]>Cc Andi Shyti <[email protected]>Cc Matt Roper <[email protected]>Signed-off-by Mika Kuoppala <[email protected]>Signed-off-by Maciej Patelczyk <[email protected]>--- drivers/gpu/drm/xe/Makefile | 3 +- drivers/gpu/drm/xe/xe_device.c | 24 + drivers/gpu/drm/xe/xe_device_types.h | 26 + drivers/gpu/drm/xe/xe_eudebug.c | 1093 +++++++++++++++++++++++++ drivers/gpu/drm/xe/xe_eudebug.h | 27 + drivers/gpu/drm/xe/xe_eudebug_types.h | 169 ++++ drivers/gpu/drm/xe/xe_vm.c | 7 +- include/uapi/drm/xe_drm.h | 21 + include/uapi/drm/xe_drm_eudebug.h | 57 ++ 9 files changed, 1425 insertions(+), 2 deletions(-) create mode 100644 drivers/gpu/drm/xe/xe_eudebug.c create mode 100644 drivers/gpu/drm/xe/xe_eudebug.h create mode 100644 drivers/gpu/drm/xe/xe_eudebug_types.h create mode 100644 include/uapi/drm/xe_drm_eudebug.hdrivers/gpu/drm/xe/Makefile +2
-1@@ -114,7 +114,8 @@ xe-y += xe_bb.o \ xe_vram_freq.o \ xe_wait_user_fence.o \ xe_wa.o \ - xe_wopcm.o + xe_wopcm.o \ + xe_eudebug.o xe-$(CONFIG_HMM_MIRROR) += xe_hmm.odrivers/gpu/drm/xe/xe_device.c +24
-0@@ -54,6 +54,7 @@#include "xe_vm.h" #include "xe_vram.h" #include "xe_wait_user_fence.h" +#include "xe_eudebug.h" #include "xe_wa.h" #include <generated/xe_wa_oob.h> @@ -100,6 +101,14 @@ static int xe_file_open(struct drm_device *dev, struct drm_file *file)put_task_struct (task); } +INIT_LIST_HEAD (&xef->link); + +mutex_lock (&xe->files.lock); +list_add_tail (&xef->link, &xe->files.list); +mutex_unlock (&xe->files.lock); + +xe_eudebug_file_open (xef); +return 0 ; }@@ -158,6 +167,12 @@ static void xe_file_close(struct drm_device *dev, struct drm_file *file)xe_pm_runtime_get (xe); +xe_eudebug_file_close (xef); + +mutex_lock (&xef->xe->files.lock); +list_del_init (&xef->link); +mutex_unlock (&xef->xe->files.lock); +/* * No need for exec_queue.lock here as there is no contention for it * when FD is closing as IOCTLs presumably can't be modifying the @@ -196,6 +211,7 @@ static const struct drm_ioctl_desc xe_ioctls[] = {DRM_IOCTL_DEF_DRV (XE_WAIT_USER_FENCE , xe_wait_user_fence_ioctl,DRM_RENDER_ALLOW ),DRM_IOCTL_DEF_DRV (XE_OBSERVATION , xe_observation_ioctl,DRM_RENDER_ALLOW ), + DRM_IOCTL_DEF_DRV(XE_EUDEBUG_CONNECT, xe_eudebug_connect_ioctl, DRM_RENDER_ALLOW), };static long xe_drm_ioctl (struct file *file,unsigned int cmd,unsigned long arg)@@ -285,6 +301,8 @@ static void xe_device_destroy(struct drm_device *dev, void *dummy) {struct xe_device *xe =to_xe_device (dev); +xe_eudebug_fini (xe); +if (xe->preempt_fence_wq)destroy_workqueue (xe->preempt_fence_wq);@@ -330,6 +348,9 @@ struct xe_device *xe_device_create(struct pci_dev *pdev,spin_lock_init (&xe->irq.lock);spin_lock_init (&xe->clients.lock); +drmm_mutex_init (&xe->drm, &xe->files.lock); +INIT_LIST_HEAD (&xe->files.list); +init_waitqueue_head (&xe->ufence_wq); err =drmm_mutex_init (&xe->drm, &xe->usm.lock);@@ -356,7 +377,10 @@ struct xe_device *xe_device_create(struct pci_dev *pdev,INIT_LIST_HEAD (&xe->pinned.external_vram);INIT_LIST_HEAD (&xe->pinned.evicted); +xe_eudebug_init (xe); + xe->preempt_fence_wq =alloc_ordered_workqueue ("xe-preempt-fence-wq" ,0 ); + xe->ordered_wq =alloc_ordered_workqueue ("xe-ordered-wq" ,0 ); xe->unordered_wq =alloc_workqueue ("xe-unordered-wq" ,0 ,0 ); if (!xe->ordered_wq || !xe->unordered_wq ||drivers/gpu/drm/xe/xe_device_types.h +26
-0@@ -355,6 +355,15 @@ struct xe_device {u64 count; } clients; +/** @files: xe_file list (opens) */ +struct { +/** @lock: Protects xe_file list */ +struct mutex lock; + +/** @list: xe file list */ +struct list_head list; + } files; +/** @usm: unified memory state */ struct {/** @usm.asid: convert a ASID to VM */ @@ -492,6 +501,20 @@ struct xe_device {u8 vm_inject_error_position;#endif +/** @debugger connection list and globals for device */ +struct { +/** @lock: protects the list of connections */ +spinlock_t lock; +/** @list: list of connections, aka debuggers */ +struct list_head list; + +/** @session_count: session counter to track connections */ +u64 session_count; + +/** @available: is the debugging functionality available */ +bool available; + } eudebug; +/* private: */ #if IS_ENABLED (CONFIG_DRM_XE_DISPLAY )@@ -596,6 +619,9 @@ struct xe_file {/** @refcount: ref count of this xe file */ struct kref refcount; + +/** @link: link into xe_device.client.list */ + struct list_head link; };#endif drivers/gpu/drm/xe/xe_eudebug.c new file +1093
-0@@ -0,0 +1,1093 @@ +// SPDX-License-Identifier: MIT +/* +* Copyright © 2023 Intel Corporation +*/ + +#include <linux/uaccess.h> + +#include <linux/anon_inodes.h> +#include <linux/poll.h> +#include <linux/delay.h> + +#include <drm/drm_managed.h> + +#include "xe_device.h" +#include "xe_assert.h" +#include "xe_macros.h" + +#include "xe_eudebug_types.h" +#include "xe_eudebug.h" + +/* +* If there is no detected event read by userspace, during this period, assume +* userspace problem and disconnect debugger to allow forward progress. +*/ +#define XE_EUDEBUG_NO_READ_DETECTED_TIMEOUT_MS (25 * 1000) + +#define for_each_debugger_rcu (debugger, head) \ + list_for_each_entry_rcu((debugger), (head), connection_link) +#define for_each_debugger (debugger, head) \ + list_for_each_entry((debugger), (head), connection_link) + +#define cast_event (T , event) container_of((event), typeof(*(T)), base) + +#define XE_EUDEBUG_DBG_STR "eudbg: %lld:%lu:%s (%d/%d) -> (%d/%d): " +#define XE_EUDEBUG_DBG_ARGS (d) (d)->session, \ + atomic_long_read(&(d)->events.seqno), \ + READ_ONCE(d->connection.status) <= 0 ? "disconnected" : "", \ + current->pid, \ + task_tgid_nr(current), \ + (d)->target_task->pid, \ + task_tgid_nr((d)->target_task) + +#define eu_err (d, fmt, ...) drm_err(&(d)->xe->drm, XE_EUDEBUG_DBG_STR # fmt, \ + XE_EUDEBUG_DBG_ARGS(d), ##__VA_ARGS__) +#define eu_warn (d, fmt, ...) drm_warn(&(d)->xe->drm, XE_EUDEBUG_DBG_STR # fmt, \ + XE_EUDEBUG_DBG_ARGS(d), ##__VA_ARGS__) +#define eu_dbg (d, fmt, ...) drm_dbg(&(d)->xe->drm, XE_EUDEBUG_DBG_STR # fmt, \ + XE_EUDEBUG_DBG_ARGS(d), ##__VA_ARGS__) + +#define xe_eudebug_assert (d, ...) xe_assert((d)->xe, ##__VA_ARGS__) + +#define struct_member (T , member) (((T *)0)->member) + +/* Keep 1:1 parity with uapi events */ +#define write_member (T_out, ptr, member, value) { \ + BUILD_BUG_ON(sizeof(*ptr) != sizeof(T_out)); \ + BUILD_BUG_ON(offsetof(typeof(*ptr), member) != \ + offsetof(typeof(T_out), member)); \ + BUILD_BUG_ON(sizeof(ptr->member) != sizeof(value)); \ + BUILD_BUG_ON(sizeof(struct_member(T_out, member)) != sizeof(value)); \ + BUILD_BUG_ON(!typecheck(typeof((ptr)->member), value)); \ + (ptr)->member = (value); \ + } + +static struct xe_eudebug_event * +event_fifo_pending (struct xe_eudebug *d) +{ +struct xe_eudebug_event *event; + +if (kfifo_peek (&d->events.fifo, &event)) +return event; + +return NULL ; +} + +/* +* This is racy as we dont take the lock for read but all the +* callsites can handle the race so we can live without lock. +*/ +__no_kcsan +static unsigned int +event_fifo_num_events_peek (const struct xe_eudebug *const d) +{ +return kfifo_len (&d->events.fifo); +} + +static bool +xe_eudebug_detached (struct xe_eudebug *d) +{ +int status; + +spin_lock (&d->connection.lock); + status = d->connection.status; +spin_unlock (&d->connection.lock); + +return status <=0 ; +} + +static int +xe_eudebug_error (const struct xe_eudebug *const d) +{ +const int status =READ_ONCE (d->connection.status); + +return status <=0 ? status :0 ; +} + +static unsigned int +event_fifo_has_events (struct xe_eudebug *d) +{ +if (xe_eudebug_detached (d)) +return 1 ; + +return event_fifo_num_events_peek (d); +} + +static const struct rhashtable_params rhash_res = { + .head_offset = offsetof(struct xe_eudebug_handle , rh_head), + .key_len = sizeof_field(struct xe_eudebug_handle, key), + .key_offset = offsetof(struct xe_eudebug_handle , key), + .automatic_shrinking = true, +}; + +static struct xe_eudebug_resource * +resource_from_type (struct xe_eudebug_resources *const res,const int t) +{ +return &res->rt[t]; +} + +static struct xe_eudebug_resources * +xe_eudebug_resources_alloc (void ) +{ +struct xe_eudebug_resources *res; +int err; +int i; + + res =kzalloc (sizeof (*res),GFP_ATOMIC ); +if (!res) +return ERR_PTR (-ENOMEM ); + +mutex_init (&res->lock); + +for (i =0 ; i <XE_EUDEBUG_RES_TYPE_COUNT ; i++) { +xa_init_flags (&res->rt[i].xa,XA_FLAGS_ALLOC1 ); + err =rhashtable_init (&res->rt[i].rh, &rhash_res); + +if (err) +break ; + } + +if (err) { +while (i--) { +xa_destroy (&res->rt[i].xa); +rhashtable_destroy (&res->rt[i].rh); + } + +kfree (res); +return ERR_PTR (err); + } + +return res; +} + +static void res_free_fn (void *ptr,void *arg) +{ +XE_WARN_ON (ptr); +kfree (ptr); +} + +static void +xe_eudebug_destroy_resources (struct xe_eudebug *d) +{ +struct xe_eudebug_resources *res = d->res; +struct xe_eudebug_handle *h; +unsigned long j; +int i; +int err; + +mutex_lock (&res->lock); +for (i =0 ; i <XE_EUDEBUG_RES_TYPE_COUNT ; i++) { +struct xe_eudebug_resource *r = &res->rt[i]; + +xa_for_each (&r->xa, j, h) { +struct xe_eudebug_handle *t; + + err =rhashtable_remove_fast (&r->rh, + &h->rh_head, + rhash_res); +xe_eudebug_assert (d, !err); + t =xa_erase (&r->xa, h->id); +xe_eudebug_assert (d, t == h); +kfree (t); + } + } +mutex_unlock (&res->lock); + +for (i =0 ; i <XE_EUDEBUG_RES_TYPE_COUNT ; i++) { +struct xe_eudebug_resource *r = &res->rt[i]; + +rhashtable_free_and_destroy (&r->rh, res_free_fn,NULL ); +xe_eudebug_assert (d,xa_empty (&r->xa)); +xa_destroy (&r->xa); + } + +mutex_destroy (&res->lock); + +kfree (res); +} + +static void xe_eudebug_free (struct kref *ref) +{ +struct xe_eudebug *d =container_of (ref,typeof (*d), ref); +struct xe_eudebug_event *event; + +while (kfifo_get (&d->events.fifo, &event)) +kfree (event); + +xe_eudebug_destroy_resources (d); +put_task_struct (d->target_task); + +xe_eudebug_assert (d, !kfifo_len (&d->events.fifo)); + +kfree_rcu (d, rcu); +} + +static void xe_eudebug_put (struct xe_eudebug *d) +{ +kref_put (&d->ref, xe_eudebug_free); +} + +static struct task_struct *find_get_target (const pid_t nr) +{ +struct task_struct *task; + +rcu_read_lock (); + task =pid_task (find_pid_ns (nr,task_active_pid_ns (current)),PIDTYPE_PID ); +if (task) +get_task_struct (task); +rcu_read_unlock (); + +return task; +} + +static int +xe_eudebug_attach (struct xe_device *xe,struct xe_eudebug *d, +const pid_t pid_nr) +{ +struct task_struct *target; +struct xe_eudebug *iter; +kuid_t uid =current_uid (); +int ret =0 ; + + target =find_get_target (pid_nr); +if (!target) +return -ENOENT ; + +if (!uid_eq (uid,task_uid (target)) && !capable (CAP_SYS_ADMIN )) { +put_task_struct (target); +return -EACCES ; + } + +XE_WARN_ON (d->connection.status !=0 ); + +spin_lock (&xe->eudebug.lock); +for_each_debugger (iter, &xe->eudebug.list) { +if (!same_thread_group (iter->target_task, target)) +continue ; + + ret = -EBUSY ; + } + +if (!ret && xe->eudebug.session_count +1 ==0 ) + ret = -ENOSPC ; + +if (!ret) { + d->connection.status =XE_EUDEBUG_STATUS_CONNECTED ; + d->xe = xe; + d->target_task = target; + d->session = ++xe->eudebug.session_count; +kref_get (&d->ref); +list_add_tail_rcu (&d->connection_link, &xe->eudebug.list); + } +spin_unlock (&xe->eudebug.lock); + +if (ret) +put_task_struct (target); + +return ret; +} + +static bool xe_eudebug_detach (struct xe_device *xe, +struct xe_eudebug *d, +const int err) +{ +bool detached = false; + +XE_WARN_ON (err >0 ); + +spin_lock (&d->connection.lock); +if (d->connection.status ==XE_EUDEBUG_STATUS_CONNECTED ) { + d->connection.status = err; + detached = true; + } +spin_unlock (&d->connection.lock); + +if (!detached) +return false; + +spin_lock (&xe->eudebug.lock); +list_del_rcu (&d->connection_link); +spin_unlock (&xe->eudebug.lock); + +eu_dbg (d,"session %lld detached with %d" , d->session, err); + +/* Our ref with the connection_link */ +xe_eudebug_put (d); + +return true; +} + +static int _xe_eudebug_disconnect (struct xe_eudebug *d, +const int err) +{ +wake_up_all (&d->events.write_done); +wake_up_all (&d->events.read_done); + +return xe_eudebug_detach (d->xe, d, err); +} + +#define xe_eudebug_disconnect (_d, _err) ({ \ + if (_xe_eudebug_disconnect((_d), (_err))) { \ + if ((_err) == 0 || (_err) == -ETIMEDOUT) \ + eu_dbg(d, "Session closed (%d)", (_err)); \ + else \ + eu_err(d, "Session disconnected, err = %d (%s:%d)", \ + (_err), __func__, __LINE__); \ + } \ +}) + +static int xe_eudebug_release (struct inode *inode,struct file *file) +{ +struct xe_eudebug *d = file->private_data; + +xe_eudebug_disconnect (d,0 ); +xe_eudebug_put (d); + +return 0 ; +} + +static __poll_t xe_eudebug_poll (struct file *file,poll_table *wait) +{ +struct xe_eudebug *const d = file->private_data; +__poll_t ret =0 ; + +poll_wait (file, &d->events.write_done, wait); + +if (xe_eudebug_detached (d)) { + ret |=EPOLLHUP ; +if (xe_eudebug_error (d)) + ret |=EPOLLERR ; + } + +if (event_fifo_num_events_peek (d)) + ret |=EPOLLIN ; + +return ret; +} + +static ssize_t xe_eudebug_read (struct file *file, +char __user *buf, +size_t count, +loff_t *ppos) +{ +return -EINVAL ; +} + +static struct xe_eudebug * +xe_eudebug_for_task_get (struct xe_device *xe, +struct task_struct *task) +{ +struct xe_eudebug *d, *iter; + + d =NULL ; + +rcu_read_lock (); +for_each_debugger_rcu (iter, &xe->eudebug.list) { +if (!same_thread_group (iter->target_task, task)) +continue ; + +if (kref_get_unless_zero (&iter->ref)) + d = iter; + +break ; + } +rcu_read_unlock (); + +return d; +} + +static struct task_struct *find_task_get (struct xe_file *xef) +{ +struct task_struct *task; +struct pid *pid; + +rcu_read_lock (); + pid =rcu_dereference (xef->drm->pid); + task =pid_task (pid,PIDTYPE_PID ); +if (task) +get_task_struct (task); +rcu_read_unlock (); + +return task; +} + +static struct xe_eudebug * +xe_eudebug_get (struct xe_file *xef) +{ +struct task_struct *task; +struct xe_eudebug *d; + + d =NULL ; + task =find_task_get (xef); +if (task) { + d =xe_eudebug_for_task_get (to_xe_device (xef->drm->minor->dev), + task); +put_task_struct (task); + } + +if (!d) +return NULL ; + +if (xe_eudebug_detached (d)) { +xe_eudebug_put (d); +return NULL ; + } + +return d; +} + +static int xe_eudebug_queue_event (struct xe_eudebug *d, +struct xe_eudebug_event *event) +{ +const u64 wait_jiffies =msecs_to_jiffies (1000 ); +u64 last_read_detected_ts, last_head_seqno, start_ts; + +xe_eudebug_assert (d, event->len >sizeof (struct xe_eudebug_event )); +xe_eudebug_assert (d, event->type); +xe_eudebug_assert (d, event->type !=DRM_XE_EUDEBUG_EVENT_READ ); + + start_ts =ktime_get (); + last_read_detected_ts = start_ts; + last_head_seqno =0 ; + +do { +struct xe_eudebug_event *head; +u64 head_seqno; +bool was_queued; + +if (xe_eudebug_detached (d)) +break ; + +spin_lock (&d->events.lock); + head =event_fifo_pending (d); +if (head) + head_seqno = event->seqno; +else + head_seqno =0 ; + + was_queued =kfifo_in (&d->events.fifo, &event,1 ); +spin_unlock (&d->events.lock); + +wake_up_all (&d->events.write_done); + +if (was_queued) { + event =NULL ; +break ; + } + +XE_WARN_ON (!head_seqno); + +/* If we detect progress, restart timeout */ +if (last_head_seqno != head_seqno) + last_read_detected_ts =ktime_get (); + + last_head_seqno = head_seqno; + +wait_event_interruptible_timeout (d->events.read_done, + !kfifo_is_full (&d->events.fifo), + wait_jiffies); + + }while (ktime_ms_delta (ktime_get (), last_read_detected_ts) < +XE_EUDEBUG_NO_READ_DETECTED_TIMEOUT_MS ); + +if (event) { +eu_dbg (d, +"event %llu queue failed (blocked %lld ms, avail %d)" , + event ? event->seqno :0 , +ktime_ms_delta (ktime_get (), start_ts), +kfifo_avail (&d->events.fifo)); + +kfree (event); + +return -ETIMEDOUT ; + } + +return 0 ; +} + +static struct xe_eudebug_handle * +alloc_handle (const int type,const u64 key) +{ +struct xe_eudebug_handle *h; + + h =kzalloc (sizeof (*h),GFP_ATOMIC ); +if (!h) +return NULL ; + + h->key = key; + +return h; +} + +static struct xe_eudebug_handle * +__find_handle (struct xe_eudebug_resource *r, +const u64 key) +{ +struct xe_eudebug_handle *h; + + h =rhashtable_lookup_fast (&r->rh, + &key, + rhash_res); +return h; +} + +static int find_handle (struct xe_eudebug_resources *res, +const int type, +const void *p) +{ +const u64 key = (u64 )p; +struct xe_eudebug_resource *r; +struct xe_eudebug_handle *h; +int id; + +if (XE_WARN_ON (!key)) +return -EINVAL ; + + r =resource_from_type (res, type); + +mutex_lock (&res->lock); + h =__find_handle (r, key); + id = h ? h->id : -ENOENT ; +mutex_unlock (&res->lock); + +return id; +} + +static int _xe_eudebug_add_handle (struct xe_eudebug *d, +int type, +void *p, +u64 *seqno, +int *handle) +{ +const u64 key = (u64 )p; +struct xe_eudebug_resource *r; +struct xe_eudebug_handle *h, *o; +int err; + +if (XE_WARN_ON (p ==NULL )) +return -EINVAL ; + +if (xe_eudebug_detached (d)) +return -ENOTCONN ; + + h =alloc_handle (type, key); +if (!h) +return -ENOMEM ; + + r =resource_from_type (d->res, type); + +mutex_lock (&d->res->lock); + o =__find_handle (r, key); +if (!o) { + err =xa_alloc (&r->xa, &h->id, h, xa_limit_31b,GFP_KERNEL ); + +if (h->id >=INT_MAX ) { +xa_erase (&r->xa, h->id); + err = -ENOSPC ; + } + +if (!err) + err =rhashtable_insert_fast (&r->rh, + &h->rh_head, + rhash_res); + +if (err) { +xa_erase (&r->xa, h->id); + }else { +if (seqno) + *seqno =atomic_long_inc_return (&d->events.seqno); + } + }else { +xe_eudebug_assert (d, o->id); + err = -EEXIST ; + } +mutex_unlock (&d->res->lock); + +if (handle) + *handle = o ? o->id : h->id; + +if (err) { +kfree (h); +XE_WARN_ON (err >0 ); +return err; + } + +xe_eudebug_assert (d, h->id); + +return h->id; +} + +static int xe_eudebug_add_handle (struct xe_eudebug *d, +int type, +void *p, +u64 *seqno) +{ +int ret; + + ret =_xe_eudebug_add_handle (d, type, p, seqno,NULL ); +if (ret == -EEXIST || ret == -ENOTCONN ) { +eu_dbg (d,"%d on adding %d" , ret, type); +return 0 ; + } + +if (ret <0 ) +xe_eudebug_disconnect (d, ret); + +return ret; +} + +static int _xe_eudebug_remove_handle (struct xe_eudebug *d,int type,void *p, +u64 *seqno) +{ +const u64 key = (u64 )p; +struct xe_eudebug_resource *r; +struct xe_eudebug_handle *h, *xa_h; +int ret; + +if (XE_WARN_ON (!key)) +return -EINVAL ; + +if (xe_eudebug_detached (d)) +return -ENOTCONN ; + + r =resource_from_type (d->res, type); + +mutex_lock (&d->res->lock); + h =__find_handle (r, key); +if (h) { + ret =rhashtable_remove_fast (&r->rh, + &h->rh_head, + rhash_res); +xe_eudebug_assert (d, !ret); + xa_h =xa_erase (&r->xa, h->id); +xe_eudebug_assert (d, xa_h == h); +if (!ret) { + ret = h->id; +if (seqno) + *seqno =atomic_long_inc_return (&d->events.seqno); + } + }else { + ret = -ENOENT ; + } +mutex_unlock (&d->res->lock); + +kfree (h); + +xe_eudebug_assert (d, ret); + +return ret; +} + +static int xe_eudebug_remove_handle (struct xe_eudebug *d,int type,void *p, +u64 *seqno) +{ +int ret; + + ret =_xe_eudebug_remove_handle (d, type, p, seqno); +if (ret == -ENOENT || ret == -ENOTCONN ) { +eu_dbg (d,"%d on removing %d" , ret, type); +return 0 ; + } + +if (ret <0 ) +xe_eudebug_disconnect (d, ret); + +return ret; +} + +static struct xe_eudebug_event * +__xe_eudebug_create_event (struct xe_eudebug *d, +u64 seqno,u16 type,u16 flags,u32 len,gfp_t gfp) +{ +struct xe_eudebug_event *event; + +xe_eudebug_assert (d, len >sizeof (*event)); + + event =kzalloc (len, gfp); +if (!event) +return NULL ; + + event->type = type; + event->flags = flags; + event->len = len; + event->seqno = seqno; + +return event; +} + +static struct xe_eudebug_event * +xe_eudebug_create_event (struct xe_eudebug *d,u16 type,u64 seqno,u16 flags, +u32 len,gfp_t gfp) +{ +return __xe_eudebug_create_event (d, seqno, type, flags, len, gfp); +} + +static long xe_eudebug_read_event (struct xe_eudebug *d, +const u64 arg, +const bool wait) +{ +struct xe_device *xe = d->xe; +struct drm_xe_eudebug_event __user *const user_orig = +u64_to_user_ptr (arg); +struct drm_xe_eudebug_event user_event; +struct xe_eudebug_event *event; +long ret =0 ; + +if (XE_IOCTL_DBG (xe,copy_from_user (&user_event, user_orig,sizeof (user_event)))) +return -EFAULT ; + +if (XE_IOCTL_DBG (xe, !user_event.type)) +return -EINVAL ; + +if (XE_IOCTL_DBG (xe, user_event.type >DRM_XE_EUDEBUG_EVENT_MAX_EVENT )) +return -EINVAL ; + +if (XE_IOCTL_DBG (xe, user_event.type !=DRM_XE_EUDEBUG_EVENT_READ )) +return -EINVAL ; + +if (XE_IOCTL_DBG (xe, user_event.len <sizeof (*user_orig))) +return -EINVAL ; + +if (XE_IOCTL_DBG (xe, user_event.flags)) +return -EINVAL ; + +if (XE_IOCTL_DBG (xe, user_event.reserved)) +return -EINVAL ; + +/* XXX: define wait time in connect arguments ? */ +if (wait) { + ret =wait_event_interruptible_timeout (d->events.write_done, +event_fifo_has_events (d), +msecs_to_jiffies (5 *1000 )); + +if (XE_IOCTL_DBG (xe, ret <0 )) +return ret; + } + + ret =0 ; +spin_lock (&d->events.lock); + event =event_fifo_pending (d); +if (event) { +if (user_event.len < event->len) { + ret = -EMSGSIZE ; + }else if (!kfifo_out (&d->events.fifo, &event,1 )) { +eu_warn (d,"internal fifo corruption" ); + ret = -ENOTCONN ; + } + } +spin_unlock (&d->events.lock); + +wake_up_all (&d->events.read_done); + +if (XE_IOCTL_DBG (xe, ret)) +return ret; + +if (!event) { +if (xe_eudebug_detached (d)) +return -ENOTCONN ; +if (!wait) +return -EAGAIN ; + +return -ENOENT ; + } + +if (copy_to_user (user_orig, event, event->len)) + ret = -EFAULT ; +else +eu_dbg (d,"event read: type=%u, flags=0x%x, seqno=%llu" , event->type, + event->flags, event->seqno); + +kfree (event); + +return ret; +} + +static long xe_eudebug_ioctl (struct file *file, +unsigned int cmd, +unsigned long arg) +{ +struct xe_eudebug *const d = file->private_data; +long ret; + +switch (cmd) { +case DRM_XE_EUDEBUG_IOCTL_READ_EVENT : + ret =xe_eudebug_read_event (d, arg, + !(file->f_flags &O_NONBLOCK )); +break ; + +default : + ret = -EINVAL ; + } + +return ret; +} + +static const struct file_operations fops = { + .owner =THIS_MODULE , + .llseek = no_llseek, + .release = xe_eudebug_release, + .poll = xe_eudebug_poll, + .read = xe_eudebug_read, + .unlocked_ioctl = xe_eudebug_ioctl, +}; + +static int +xe_eudebug_connect (struct xe_device *xe, +struct drm_xe_eudebug_connect *param) +{ +const u64 known_open_flags =0 ; +unsigned long f_flags =0 ; +struct xe_eudebug *d; +int fd, err; + +if (param->extensions) +return -EINVAL ; + +if (!param->pid) +return -EINVAL ; + +if (param->flags & ~known_open_flags) +return -EINVAL ; + +if (param->version && param->version !=DRM_XE_EUDEBUG_VERSION ) +return -EINVAL ; + + param->version =DRM_XE_EUDEBUG_VERSION ; + +if (!xe->eudebug.available) +return -EOPNOTSUPP ; + + d =kzalloc (sizeof (*d),GFP_KERNEL ); +if (!d) +return -ENOMEM ; + +kref_init (&d->ref); +spin_lock_init (&d->connection.lock); +init_waitqueue_head (&d->events.write_done); +init_waitqueue_head (&d->events.read_done); + +spin_lock_init (&d->events.lock); +INIT_KFIFO (d->events.fifo); + + d->res =xe_eudebug_resources_alloc (); +if (IS_ERR (d->res)) { + err =PTR_ERR (d->res); + goto err_free; + } + + err =xe_eudebug_attach (xe, d, param->pid); +if (err) + goto err_free_res; + + fd =anon_inode_getfd ("[xe_eudebug]" , &fops, d, f_flags); +if (fd <0 ) { + err = fd; + goto err_detach; + } + +eu_dbg (d,"connected session %lld" , d->session); + +return fd; + +err_detach: +xe_eudebug_detach (xe, d, err); +err_free_res: +xe_eudebug_destroy_resources (d); +err_free: +kfree (d); + +return err; +} + +int xe_eudebug_connect_ioctl (struct drm_device *dev, +void *data, +struct drm_file *file) +{ +struct xe_device *xe =to_xe_device (dev); +struct drm_xe_eudebug_connect *const param = data; +int ret =0 ; + + ret =xe_eudebug_connect (xe, param); + +return ret; +} + +void xe_eudebug_init (struct xe_device *xe) +{ +spin_lock_init (&xe->eudebug.lock); +INIT_LIST_HEAD (&xe->eudebug.list); + + xe->eudebug.available = true; +} + +void xe_eudebug_fini (struct xe_device *xe) +{ +xe_assert (xe,list_empty_careful (&xe->eudebug.list)); +} + +static int send_open_event (struct xe_eudebug *d,u32 flags,const u64 handle, +const u64 seqno) +{ +struct xe_eudebug_event *event; +struct xe_eudebug_event_open *eo; + +if (!handle) +return -EINVAL ; + +if (XE_WARN_ON ((long )handle >=INT_MAX )) +return -EINVAL ; + + event =xe_eudebug_create_event (d,DRM_XE_EUDEBUG_EVENT_OPEN , seqno, + flags,sizeof (*eo),GFP_KERNEL ); +if (!event) +return -ENOMEM ; + + eo =cast_event (eo, event); + + write_member(struct drm_xe_eudebug_event_client, eo, + client_handle, handle); + +return xe_eudebug_queue_event (d, event); +} + +static int client_create_event (struct xe_eudebug *d,struct xe_file *xef) +{ +u64 seqno; +int ret; + + ret =xe_eudebug_add_handle (d,XE_EUDEBUG_RES_TYPE_CLIENT , xef, &seqno); +if (ret >0 ) + ret =send_open_event (d,DRM_XE_EUDEBUG_EVENT_CREATE , + ret, seqno); + +return ret; +} + +static int client_destroy_event (struct xe_eudebug *d,struct xe_file *xef) +{ +u64 seqno; +int ret; + + ret =xe_eudebug_remove_handle (d,XE_EUDEBUG_RES_TYPE_CLIENT , + xef, &seqno); +if (ret >0 ) + ret =send_open_event (d,DRM_XE_EUDEBUG_EVENT_DESTROY , + ret, seqno); + +return ret; +} + +#define xe_eudebug_event_put (_d, _err) ({ \ + if ((_err)) \ + xe_eudebug_disconnect((_d), (_err)); \ + xe_eudebug_put((_d)); \ + }) + +void xe_eudebug_file_open (struct xe_file *xef) +{ +struct xe_eudebug *d; + + d =xe_eudebug_get (xef); +if (!d) +return ; + +xe_eudebug_event_put (d,client_create_event (d, xef)); +} + +void xe_eudebug_file_close (struct xe_file *xef) +{ +struct xe_eudebug *d; + + d =xe_eudebug_get (xef); +if (!d) +return ; + +xe_eudebug_event_put (d,client_destroy_event (d, xef)); +} + +static int send_vm_event (struct xe_eudebug *d,u32 flags, +const u64 client_handle, +const u64 vm_handle, +const u64 seqno) +{ +struct xe_eudebug_event *event; +struct xe_eudebug_event_vm *e; + + event =xe_eudebug_create_event (d,DRM_XE_EUDEBUG_EVENT_VM , + seqno, flags,sizeof (*e),GFP_KERNEL ); +if (!event) +return -ENOMEM ; + + e =cast_event (e, event); + + write_member(struct drm_xe_eudebug_event_vm, e, client_handle, client_handle); + write_member(struct drm_xe_eudebug_event_vm, e, vm_handle, vm_handle); + +return xe_eudebug_queue_event (d, event); +} + +static int vm_create_event (struct xe_eudebug *d, +struct xe_file *xef,struct xe_vm *vm) +{ +int h_c, h_vm; +u64 seqno; +int ret; + + h_c =find_handle (d->res,XE_EUDEBUG_RES_TYPE_CLIENT , xef); +if (h_c <0 ) +return h_c; + +xe_eudebug_assert (d, h_c); + + h_vm =xe_eudebug_add_handle (d,XE_EUDEBUG_RES_TYPE_VM , vm, &seqno); +if (h_vm <=0 ) +return h_vm; + + ret =send_vm_event (d,DRM_XE_EUDEBUG_EVENT_CREATE , h_c, h_vm, seqno); + +return ret; +} + +static int vm_destroy_event (struct xe_eudebug *d, +struct xe_file *xef,struct xe_vm *vm) +{ +int h_c, h_vm; +u64 seqno; + + h_c =find_handle (d->res,XE_EUDEBUG_RES_TYPE_CLIENT , xef); +if (h_c <0 ) { +XE_WARN_ON ("no client found for vm" ); +eu_warn (d,"no client found for vm" ); +return h_c; + } + +xe_eudebug_assert (d, h_c); + + h_vm =xe_eudebug_remove_handle (d,XE_EUDEBUG_RES_TYPE_VM , vm, &seqno); +if (h_vm <=0 ) +return h_vm; + +return send_vm_event (d,DRM_XE_EUDEBUG_EVENT_DESTROY , h_c, h_vm, seqno); +} + +void xe_eudebug_vm_create (struct xe_file *xef,struct xe_vm *vm) +{ +struct xe_eudebug *d; + + d =xe_eudebug_get (xef); +if (!d) +return ; + +xe_eudebug_event_put (d,vm_create_event (d, xef, vm)); +} + +void xe_eudebug_vm_destroy (struct xe_file *xef,struct xe_vm *vm) +{ +struct xe_eudebug *d; + + d =xe_eudebug_get (xef); +if (!d) +return ; + +xe_eudebug_event_put (d,vm_destroy_event (d, xef, vm)); +}drivers/gpu/drm/xe/xe_eudebug.h new file +27
-0@@ -0,0 +1,27 @@ +/* SPDX-License-Identifier: MIT */ +/* +* Copyright © 2023 Intel Corporation +*/ + +#ifndef _XE_EUDEBUG_H_ + +struct drm_device ; +struct drm_file ; +struct xe_device ; +struct xe_file ; +struct xe_vm ; + +int xe_eudebug_connect_ioctl (struct drm_device *dev, +void *data, +struct drm_file *file); + +void xe_eudebug_init (struct xe_device *xe); +void xe_eudebug_fini (struct xe_device *xe); + +void xe_eudebug_file_open (struct xe_file *xef); +void xe_eudebug_file_close (struct xe_file *xef); + +void xe_eudebug_vm_create (struct xe_file *xef,struct xe_vm *vm); +void xe_eudebug_vm_destroy (struct xe_file *xef,struct xe_vm *vm); + +#endif drivers/gpu/drm/xe/xe_eudebug_types.h new file +169
-0@@ -0,0 +1,169 @@ +/* SPDX-License-Identifier: MIT */ +/* +* Copyright © 2023 Intel Corporation +*/ + +#ifndef __XE_EUDEBUG_TYPES_H_ + +#include <linux/mutex.h> +#include <linux/kref.h> +#include <linux/kfifo.h> +#include <linux/completion.h> +#include <linux/wait.h> +#include <linux/xarray.h> +#include <linux/rbtree.h> +#include <linux/rhashtable.h> + +#include <uapi/drm/xe_drm.h> + +struct xe_device ; +struct task_struct ; +struct xe_eudebug_event ; + +#define CONFIG_DRM_XE_DEBUGGER_EVENT_QUEUE_SIZE 64 + +/** +* struct xe_eudebug_handle - eudebug resource handle +*/ +struct xe_eudebug_handle { +/** @key: key value in rhashtable <key:id> */ +u64 key; + +/** @id: opaque handle id for xarray <id:key> */ +int id; + +/** @rh_head: rhashtable head */ +struct rhash_head rh_head; +}; + +/** +* struct xe_eudebug_resource - Resource map for one resource +*/ +struct xe_eudebug_resource { +/** @xa: xarrays for <id->key> */ +struct xarray xa; + +/** @rh rhashtable for <key->id> */ +struct rhashtable rh; +}; + +#define XE_EUDEBUG_RES_TYPE_CLIENT 0 +#define XE_EUDEBUG_RES_TYPE_VM 1 +#define XE_EUDEBUG_RES_TYPE_COUNT (XE_EUDEBUG_RES_TYPE_VM + 1) + +/** +* struct xe_eudebug_resources - eudebug resources for all types +*/ +struct xe_eudebug_resources { +/** @lock: guards access into rt */ +struct mutex lock; + +/** @rt: resource maps for all types */ +struct xe_eudebug_resource rt[XE_EUDEBUG_RES_TYPE_COUNT ]; +}; + +/** +* struct xe_eudebug - Top level struct for eudebug: the connection +*/ +struct xe_eudebug { +/** @ref: kref counter for this struct */ +struct kref ref; + +/** @rcu: rcu_head for rcu destruction */ +struct rcu_head rcu; + +/** @connection_link: our link into the xe_device:eudebug.list */ +struct list_head connection_link; + +struct { +/** @status: connected = 1, disconnected = error */ +#define XE_EUDEBUG_STATUS_CONNECTED 1 +int status; + +/** @lock: guards access to status */ +spinlock_t lock; + } connection; + +/** @xe: the parent device we are serving */ +struct xe_device *xe; + +/** @target_task: the task that we are debugging */ +struct task_struct *target_task; + +/** @res: the resource maps we track for target_task */ +struct xe_eudebug_resources *res; + +/** @session: session number for this connection (for logs) */ +u64 session; + +/** @events: kfifo queue of to-be-delivered events */ +struct { +/** @lock: guards access to fifo */ +spinlock_t lock; + +/** @fifo: queue of events pending */ + DECLARE_KFIFO(fifo, + struct xe_eudebug_event *, + CONFIG_DRM_XE_DEBUGGER_EVENT_QUEUE_SIZE); + +/** @write_done: waitqueue for signalling write to fifo */ +wait_queue_head_t write_done; + +/** @read_done: waitqueue for signalling read from fifo */ +wait_queue_head_t read_done; + +/** @event_seqno: seqno counter to stamp events for fifo */ +atomic_long_t seqno; + } events; + +}; + +/** +* struct xe_eudebug_event - Internal base event struct for eudebug +*/ +struct xe_eudebug_event { +/** @len: length of this event, including payload */ +u32 len; + +/** @type: message type */ +u16 type; + +/** @flags: message flags */ +u16 flags; + +/** @seqno: sequence number for ordering */ +u64 seqno; + +/** @reserved: reserved field MBZ */ +u64 reserved; + +/** @data: payload bytes */ +u8 data[]; +}; + +/** +* struct xe_eudebug_event_open - Internal event for client open/close +*/ +struct xe_eudebug_event_open { +/** @base: base event */ +struct xe_eudebug_event base; + +/** @client_handle: opaque handle for client */ +u64 client_handle; +}; + +/** +* struct xe_eudebug_event_vm - Internal event for vm open/close +*/ +struct xe_eudebug_event_vm { +/** @base: base event */ +struct xe_eudebug_event base; + +/** @client_handle: client containing the vm open/close */ +u64 client_handle; + +/** @vm_handle: vm handle it's open/close */ +u64 vm_handle; +}; + +#endif drivers/gpu/drm/xe/xe_vm.c +6
-1@@ -39,6 +39,7 @@#include "xe_trace_bo.h" #include "xe_wa.h" #include "xe_hmm.h" +#include "xe_eudebug.h" static struct drm_gem_object *xe_vm_obj (struct xe_vm *vm) {@@ -1818,6 +1819,8 @@ int xe_vm_create_ioctl(struct drm_device *dev, void *data, args->reserved[0 ] =xe_bo_main_addr (vm->pt_root[0 ]->bo,XE_PAGE_SIZE );#endif +xe_eudebug_vm_create (xef, vm); +return 0 ; err_free_id:@@ -1853,8 +1856,10 @@ int xe_vm_destroy_ioctl(struct drm_device *dev, void *data,xa_erase (&xef->vm.xa, args->vm_id);mutex_unlock (&xef->vm.lock); -if (!err) +if (!err) { +xe_eudebug_vm_destroy (xef, vm);xe_vm_close_and_put (vm); + }return err; }include/uapi/drm/xe_drm.h +21
-0@@ -102,6 +102,7 @@ extern "C" {#define DRM_XE_EXEC 0x09#define DRM_XE_WAIT_USER_FENCE 0x0a#define DRM_XE_OBSERVATION 0x0b +#define DRM_XE_EUDEBUG_CONNECT 0x0c/* Must be kept compact -- no holes */ @@ -117,6 +118,7 @@ extern "C" {#define DRM_IOCTL_XE_EXEC DRM_IOW(DRM_COMMAND_BASE + DRM_XE_EXEC, struct drm_xe_exec)#define DRM_IOCTL_XE_WAIT_USER_FENCE DRM_IOWR(DRM_COMMAND_BASE + DRM_XE_WAIT_USER_FENCE, struct drm_xe_wait_user_fence)#define DRM_IOCTL_XE_OBSERVATION DRM_IOW(DRM_COMMAND_BASE + DRM_XE_OBSERVATION, struct drm_xe_observation_param) +#define DRM_IOCTL_XE_EUDEBUG_CONNECT DRM_IOWR(DRM_COMMAND_BASE + DRM_XE_EUDEBUG_CONNECT, struct drm_xe_eudebug_connect)/** * DOC: Xe IOCTL Extensions @@ -1694,6 +1696,25 @@ struct drm_xe_oa_stream_info {__u64 reserved[3 ]; }; +/* +* Debugger ABI (ioctl and events) Version History: +* 0 - No debugger available +* 1 - Initial version +*/ +#define DRM_XE_EUDEBUG_VERSION 1 + +struct drm_xe_eudebug_connect { +/** @extensions: Pointer to the first extension struct, if any */ +__u64 extensions; + +__u64 pid;/* input: Target process ID */ +__u32 flags;/* MBZ */ + +__u32 version;/* output: current ABI (ioctl / events) version */ +}; + +#include "xe_drm_eudebug.h" +#if defined(__cplusplus) }#endif include/uapi/drm/xe_drm_eudebug.h new file +57
-0@@ -0,0 +1,57 @@ +/* SPDX-License-Identifier: MIT */ +/* +* Copyright © 2023 Intel Corporation +*/ + +#ifndef _UAPI_XE_DRM_EUDEBUG_H_ +#define _UAPI_XE_DRM_EUDEBUG_H_ + +#if defined(__cplusplus) +extern "C" { +#endif + +/** +* Do a eudebug event read for a debugger connection. +* +* This ioctl is available in debug version 1. +*/ +#define DRM_XE_EUDEBUG_IOCTL_READ_EVENT _IO('j', 0x0) + +/* XXX: Document events to match their internal counterparts when moved to xe_drm.h */ +struct drm_xe_eudebug_event { +__u32 len; + +__u16 type; +#define DRM_XE_EUDEBUG_EVENT_NONE 0 +#define DRM_XE_EUDEBUG_EVENT_READ 1 +#define DRM_XE_EUDEBUG_EVENT_OPEN 2 +#define DRM_XE_EUDEBUG_EVENT_VM 3 +#define DRM_XE_EUDEBUG_EVENT_MAX_EVENT DRM_XE_EUDEBUG_EVENT_VM + +__u16 flags; +#define DRM_XE_EUDEBUG_EVENT_CREATE (1 << 0) +#define DRM_XE_EUDEBUG_EVENT_DESTROY (1 << 1) +#define DRM_XE_EUDEBUG_EVENT_STATE_CHANGE (1 << 2) +#define DRM_XE_EUDEBUG_EVENT_NEED_ACK (1 << 3) +__u64 seqno; +__u64 reserved; +}; + +struct drm_xe_eudebug_event_client { +struct drm_xe_eudebug_event base; + +__u64 client_handle;/* This is unique per debug connection */ +}; + +struct drm_xe_eudebug_event_vm { +struct drm_xe_eudebug_event base; + +__u64 client_handle; +__u64 vm_handle; +}; + +#if defined(__cplusplus) +} +#endif + +#endif Signature · 1 line
2.34.1
Re: [PATCH 03/21] drm/xe/eudebug: Introduce eudebug support
In reply to Mika KuoppalaOn Fri, Jul 26, 2024 at 05:08:00PM +0300, Mika Kuoppala wrote:
<cut>
drivers/gpu/drm/xe/xe_eudebug.c line 254
Quoted 13 lines
drivers/gpu/drm/xe/xe_eudebug.c new file +13
-0+ +static int +xe_eudebug_attach (struct xe_device *xe,struct xe_eudebug *d, +const pid_t pid_nr) +{ +struct task_struct *target; +struct xe_eudebug *iter; +kuid_t uid =current_uid (); +int ret =0 ; + + target =find_get_target (pid_nr); +if (!target) +return -ENOENT ;
This goes outside and is returned via ioctl() what reminds me some LKML thread this code is reserved for file operations only.
--
Zbigniew
Re: [PATCH 03/21] drm/xe/eudebug: Introduce eudebug support
Quoted 4 lines · Zbigniew Kempczyński
On Fri, Jul 26, 2024 at 05:08:00PM +0300, Mika Kuoppala wrote:
<cut>
Quoted 16 lines
drivers/gpu/drm/xe/xe_eudebug.c line 254
Quoted 13 lines
drivers/gpu/drm/xe/xe_eudebug.c new file +13
-0+ +static int +xe_eudebug_attach (struct xe_device *xe,struct xe_eudebug *d, +const pid_t pid_nr) +{ +struct task_struct *target; +struct xe_eudebug *iter; +kuid_t uid =current_uid (); +int ret =0 ; + + target =find_get_target (pid_nr); +if (!target) +return -ENOENT ;This goes outside and is returned via
ioctl()what reminds me some LKML thread this code is reserved for file operations only.
Yes, my excuse here is that I try to mimic what other xe code does. And I can't think of other than EINVAL here. -Mika
Quoted 3 lines
--
Zbigniew
Re: [PATCH 03/21] drm/xe/eudebug: Introduce eudebug support
In reply to Mika KuoppalaOn Fri, Jul 26, 2024 at 05:08:00PM +0300, Mika Kuoppala wrote:
<cut>
Quoted 38 lines
drivers/gpu/drm/xe/xe_eudebug.c new file +38
-0+static long xe_eudebug_read_event (struct xe_eudebug *d, +const u64 arg, +const bool wait) +{ +struct xe_device *xe = d->xe; +struct drm_xe_eudebug_event __user *const user_orig = +u64_to_user_ptr (arg); +struct drm_xe_eudebug_event user_event; +struct xe_eudebug_event *event; +long ret =0 ; + +if (XE_IOCTL_DBG (xe,copy_from_user (&user_event, user_orig,sizeof (user_event)))) +return -EFAULT ; + +if (XE_IOCTL_DBG (xe, !user_event.type)) +return -EINVAL ; + +if (XE_IOCTL_DBG (xe, user_event.type >DRM_XE_EUDEBUG_EVENT_MAX_EVENT )) +return -EINVAL ; + +if (XE_IOCTL_DBG (xe, user_event.type !=DRM_XE_EUDEBUG_EVENT_READ )) +return -EINVAL ; + +if (XE_IOCTL_DBG (xe, user_event.len <sizeof (*user_orig))) +return -EINVAL ; + +if (XE_IOCTL_DBG (xe, user_event.flags)) +return -EINVAL ; + +if (XE_IOCTL_DBG (xe, user_event.reserved)) +return -EINVAL ; + +/* XXX: define wait time in connect arguments ? */ +if (wait) { + ret =wait_event_interruptible_timeout (d->events.write_done, +event_fifo_has_events (d), +msecs_to_jiffies (5 *1000 )); +
drivers/gpu/drm/xe/xe_eudebug.c line 772
Quoted 10 lines
drivers/gpu/drm/xe/xe_eudebug.c new file +10
-0+if (XE_IOCTL_DBG (xe, ret <0 )) +return ret; + } + + ret =0 ; +spin_lock (&d->events.lock); + event =event_fifo_pending (d); +if (event) { +if (user_event.len < event->len) { + ret = -EMSGSIZE ;
Maybe it is worth to copy event->len to user_orig? I wonder how userspace guesses the size of the buffer to allocate for the event.
--
Zbigniew
Re: [PATCH 03/21] drm/xe/eudebug: Introduce eudebug support
Quoted 47 lines · Zbigniew Kempczyński
On Fri, Jul 26, 2024 at 05:08:00PM +0300, Mika Kuoppala wrote:
<cut>
Quoted 43 lines
drivers/gpu/drm/xe/xe_eudebug.c new file +43
-0+static long xe_eudebug_read_event (struct xe_eudebug *d, +const u64 arg, +const bool wait) +{ +struct xe_device *xe = d->xe; +struct drm_xe_eudebug_event __user *const user_orig = +u64_to_user_ptr (arg); +struct drm_xe_eudebug_event user_event; +struct xe_eudebug_event *event; +long ret =0 ; + +if (XE_IOCTL_DBG (xe,copy_from_user (&user_event, user_orig,sizeof (user_event)))) +return -EFAULT ; + +if (XE_IOCTL_DBG (xe, !user_event.type)) +return -EINVAL ; + +if (XE_IOCTL_DBG (xe, user_event.type >DRM_XE_EUDEBUG_EVENT_MAX_EVENT )) +return -EINVAL ; + +if (XE_IOCTL_DBG (xe, user_event.type !=DRM_XE_EUDEBUG_EVENT_READ )) +return -EINVAL ; + +if (XE_IOCTL_DBG (xe, user_event.len <sizeof (*user_orig))) +return -EINVAL ; + +if (XE_IOCTL_DBG (xe, user_event.flags)) +return -EINVAL ; + +if (XE_IOCTL_DBG (xe, user_event.reserved)) +return -EINVAL ; + +/* XXX: define wait time in connect arguments ? */ +if (wait) { + ret =wait_event_interruptible_timeout (d->events.write_done, +event_fifo_has_events (d), +msecs_to_jiffies (5 *1000 )); + +if (XE_IOCTL_DBG (xe, ret <0 )) +return ret; + } + + ret =0 ;
Quoted 8 lines
drivers/gpu/drm/xe/xe_eudebug.c line 772
Quoted 5 lines
drivers/gpu/drm/xe/xe_eudebug.c new file +5
-0+spin_lock (&d->events.lock); + event =event_fifo_pending (d); +if (event) { +if (user_event.len < event->len) { + ret = -EMSGSIZE ;Maybe it is worth to copy
event->lentouser_orig? I wonder how userspace guesses the size of the buffer to allocate for the event.
Makes sense. I will deliver event->len in v2. Thanks. -Mika
Quoted 3 lines
--
Zbigniew
PATCH 4/21 kernel: export ptrace_may_access
In reply to Mika KuoppalaWe need to export ptrace_may_access to be able to allow debugger connection to check for debuggee access and at the same time allow xe.ko module build.
--- kernel/ptrace.c | 1 + 1 file changed, 1 insertion(+)
kernel/ptrace.c +1 -0
@@ -354,6 +354,7 @@ bool ptrace_may_access(struct task_struct *task, unsigned int mode)task_unlock (task);return !err; } +EXPORT_SYMBOL_GPL (ptrace_may_access);static int check_ptrace_options (unsigned long data) {
Signature · 1 line
2.34.1
Re: [PATCH 04/21] kernel: export ptrace_may_access
Quoted 3 lines · Mika Kuoppala
We need to export
ptrace_may_accessto be able to allow debugger connection to check for debuggee access and at the same time allow xe.ko module build.
I don't follow this explanation, particularly this last line. Based on other patches you want to export the function because it would cause the calling process to manipulate another process. Like described in Documentation/process/adding-syscalls.rst.
The reason to export this function is that this comes as an ioctl to the xe driver, which may be built as a module.
Quoted 6 lines
CC Lucas De Marchi <[email protected]>Signed-off-by Mika Kuoppala <[email protected]>CC Andi Shyti <[email protected]>CC Maciej Patelczyk <[email protected]>Signed-off-by Jonathan Cavitt <[email protected]>
$ ./scripts/get_maintainer.pl kernel/ptrace.c Oleg Nesterov <[email protected]> (maintainer:PTRACE SUPPORT) [email protected] (open list)
and I don't see these Cc'ed. Also the commit title should likely start with "ptrace:" rather than "kernel:"
Lucas De Marchi
Quoted 19 lines
---
kernel/ptrace.c| 1 +1 file changed, 1 insertion(+)kernel/ptrace.c +1
-0@@ -354,6 +354,7 @@ bool ptrace_may_access(struct task_struct *task, unsigned int mode)task_unlock (task);return !err; } +EXPORT_SYMBOL_GPL (ptrace_may_access);static int check_ptrace_options (unsigned long data) {Signature · 1 line
2.34.1
Re: [PATCH 04/21] kernel: export ptrace_may_access
In reply to Mika KuoppalaHi Mika,
Quoted 9 lines · Mika Kuoppala
We need to export
ptrace_may_accessto be able to allow debugger connection to check for debuggee access and at the same time allow xe.ko module build.CC Lucas De Marchi <[email protected]>Signed-off-by Mika Kuoppala <[email protected]>CC Andi Shyti <[email protected]>CC Maciej Patelczyk <[email protected]>Signed-off-by Jonathan Cavitt <[email protected]>
Just one note here: the tag order matters a little, the signature of from the sender should take the last place.
kernel/ptrace.c bool ptrace_may_access(struct task_struct *task, unsigned int mode) · line 357
Quoted 13 lines
--- kernel/ptrace.c | 1 + 1 file changed, 1 insertion(+)kernel/ptrace.c +1
-0@@ -354,6 +354,7 @@ bool ptrace_may_access(struct task_struct *task, unsigned int mode)task_unlock (task);return !err; } +EXPORT_SYMBOL_GPL (ptrace_may_access);
I also believe, as suggested, that we should seek an ack from the maintainers here. Maybe we could have CC'ed just for this single patch. (You can add people in CC even after the '---' section not to pollute the commit log. This way you let git-send-email handle it).
Nevertheless, same as I agreed earlier:
Thanks,
Andi
Quoted 4 lines
kernel/ptrace.c +0
-0static int check_ptrace_options (unsigned long data) {Signature · 1 line
2.34.1
Re: [PATCH 04/21] kernel: export ptrace_may_access
Quoted 7 lines · Andi Shyti
Hi Mika,
Quoted 4 lines · Mika Kuoppala
We need to export
ptrace_may_accessto be able to allow debugger connection to check for debuggee access and at the same time allow xe.ko module build.
Quoted 8 lines
Quoted 5 lines
CC Lucas De Marchi <[email protected]>Signed-off-by Mika Kuoppala <[email protected]>CC Andi Shyti <[email protected]>CC Maciej Patelczyk <[email protected]>Signed-off-by Jonathan Cavitt <[email protected]>Just one note here: the tag order matters a little, the signature of from the sender should take the last place.
I will ping Jonathan to remove the sob as this might be residue of splitting patches. He is credited on the actual application of ptrace_may_access and the original EXPORT parts are mine.
Quoted 5 lines
Quoted 5 lines
--- kernel/ptrace.c | 1 + 1 file changed, 1 insertion(+)
Quoted 15 lines
kernel/ptrace.c bool ptrace_may_access(struct task_struct *task, unsigned int mode) · line 357
Quoted 9 lines
kernel/ptrace.c +1
-0@@ -354,6 +354,7 @@ bool ptrace_may_access(struct task_struct *task, unsigned int mode)task_unlock (task);return !err; } +EXPORT_SYMBOL_GPL (ptrace_may_access);I also believe, as suggested, that we should seek an ack from the maintainers here. Maybe we could have CC'ed just for this single patch. (You can add people in CC even after the '---' section not to pollute the commit log. This way you let git-send-email handle it).
I will move this patch to be first in series with ptrace author and lkml included in cc.
Quoted 4 lines
Nevertheless, same as I agreed earlier:
Reviewed-by Andi Shyti <[email protected]>
Thanks,
-Mika
Quoted 8 lines
Thanks,
AndiQuoted 4 lines
kernel/ptrace.c +0
-0static int check_ptrace_options (unsigned long data) {Signature · 1 line
2.34.1
PATCH 5/21 drm/xe/eudebug: Use ptrace_may_access for xe_eudebug_attach
In reply to Mika KuoppalaFrom: Jonathan Cavitt <[email protected]>
xe_eudebug_attach wants to use ptrace_may_access for determining when the user has the right to attach a debugger to a given process. With ptrace_may_access now exported, this has become possible.
--- drivers/gpu/drm/xe/xe_eudebug.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
drivers/gpu/drm/xe/xe_eudebug.c +1 -2
@@ -246,14 +246,13 @@ xe_eudebug_attach(struct xe_device *xe, struct xe_eudebug *d, {struct task_struct *target;struct xe_eudebug *iter; -kuid_t uid =current_uid ();int ret =0 ; target =find_get_target (pid_nr);if (!target)return -ENOENT ; -if (!uid_eq (uid,task_uid (target)) && !capable (CAP_SYS_ADMIN )) { +if (!ptrace_may_access (target,PTRACE_MODE_READ_REALCREDS )) {put_task_struct (target);return -EACCES ; }
Signature · 1 line
2.34.1
Re: [PATCH 05/21] drm/xe/eudebug: Use ptrace_may_access for xe_eudebug_attach
Quoted 8 lines · Mika Kuoppala
From: Jonathan Cavitt <[email protected]>
xe_eudebug_attachwants to useptrace_may_accessfor determining when the user has the right to attach a debugger to a given process. Withptrace_may_accessnow exported, this has become possible.Signed-off-by Jonathan Cavitt <[email protected]>Signed-off-by Mika Kuoppala <[email protected]>
this patch shouldn't exist. The patch exporting ptrace_may_access() should be moved before xe_eudebug_attach() is added, so it's already added with the right perm check.
If exporting that is not acceptable (which needs to be discussed with proper maintainers / mainling-list) then this could remain as a CAP_SYS_ADMIN.
Lucas De Marchi
Quoted 27 lines
---
drivers/gpu/drm/xe/xe_eudebug.c| 3 +--1 file changed, 1 insertion(+), 2 deletions(-)drivers/gpu/drm/xe/xe_eudebug.c +1
-2@@ -246,14 +246,13 @@ xe_eudebug_attach(struct xe_device *xe, struct xe_eudebug *d, {struct task_struct *target;struct xe_eudebug *iter; -kuid_t uid =current_uid ();int ret =0 ; target =find_get_target (pid_nr);if (!target)return -ENOENT ; -if (!uid_eq (uid,task_uid (target)) && !capable (CAP_SYS_ADMIN )) { +if (!ptrace_may_access (target,PTRACE_MODE_READ_REALCREDS )) {put_task_struct (target);return -EACCES ; }Signature · 1 line
2.34.1
PATCH 6/21 drm/xe/eudebug: Introduce discovery for resources
In reply to Mika KuoppalaDebugger connection can happen way after the client has created and destroyed arbitrary number of resources.
We need to playback all currently existing resources for the debugger. The client is held until this so called discovery process, executed by workqueue, is complete.
This patch is based on discovery work by Maciej Patelczyk for i915 driver.
--- drivers/gpu/drm/xe/xe_device.c | 6 +- drivers/gpu/drm/xe/xe_device_types.h | 3 + drivers/gpu/drm/xe/xe_eudebug.c | 91 +++++++++++++++++++++++++++ drivers/gpu/drm/xe/xe_eudebug_types.h | 7 +++ 4 files changed, 106 insertions(+), 1 deletion(-)
drivers/gpu/drm/xe/xe_device.c +5 -1
@@ -312,6 +312,9 @@ static void xe_device_destroy(struct drm_device *dev, void *dummy)if (xe->unordered_wq)destroy_workqueue (xe->unordered_wq); +if (xe->eudebug.ordered_wq) +destroy_workqueue (xe->eudebug.ordered_wq); +ttm_device_fini (&xe->ttm); }
@@ -382,8 +385,9 @@ struct xe_device *xe_device_create(struct pci_dev *pdev, xe->preempt_fence_wq =alloc_ordered_workqueue ("xe-preempt-fence-wq" ,0 ); xe->ordered_wq =alloc_ordered_workqueue ("xe-ordered-wq" ,0 ); + xe->eudebug.ordered_wq =alloc_ordered_workqueue ("xe-eudebug-ordered-wq" ,0 ); xe->unordered_wq =alloc_workqueue ("xe-unordered-wq" ,0 ,0 ); -if (!xe->ordered_wq || !xe->unordered_wq || +if (!xe->ordered_wq || !xe->unordered_wq || !xe->eudebug.ordered_wq || !xe->preempt_fence_wq) {/* * Cleanup done in xe_device_destroy via
drivers/gpu/drm/xe/xe_device_types.h +3 -0
@@ -513,6 +513,9 @@ struct xe_device {/** @available: is the debugging functionality available */ bool available; + +/** @ordered_wq: used to discovery */ +struct workqueue_struct *ordered_wq; } eudebug;/* private: */
drivers/gpu/drm/xe/xe_eudebug.c +91 -0
121 lines not on this page · raw message
drivers/gpu/drm/xe/xe_eudebug_types.h +7 -0
@@ -19,6 +19,7 @@struct xe_device ;struct task_struct ;struct xe_eudebug_event ; +struct workqueue_struct ;#define CONFIG_DRM_XE_DEBUGGER_EVENT_QUEUE_SIZE 64
@@ -96,6 +97,12 @@ struct xe_eudebug {/** @session: session number for this connection (for logs) */ u64 session; +/** @discovery: completion to wait for discovery */ +struct completion discovery; + +/** @discovery_work: worker to discover resources for target_task */ +struct work_struct discovery_work; +/** @events: kfifo queue of to-be-delivered events */ struct {/** @lock: guards access to fifo */
Signature · 1 line
2.34.1
PATCH 7/21 drm/xe/eudebug: Introduce exec_queue events
In reply to Mika KuoppalaFrom: Dominik Grzegorzek <[email protected]>
Inform debugger about creation and destruction of exec_queues.
1) Use user engine class types instead of internal xe_engine_class enum in exec_queue event.
2) During discovery do not advertise every execqueue created, only ones with class render or compute.
v2: struct alignment (Mika)
v3: checkpatch (Maciej)
--- drivers/gpu/drm/xe/xe_eudebug.c | 184 +++++++++++++++++++++++++- drivers/gpu/drm/xe/xe_eudebug.h | 4 + drivers/gpu/drm/xe/xe_eudebug_types.h | 31 ++++- drivers/gpu/drm/xe/xe_exec_queue.c | 5 + include/uapi/drm/xe_drm_eudebug.h | 14 +- 5 files changed, 235 insertions(+), 3 deletions(-)
drivers/gpu/drm/xe/xe_eudebug.c +183 -1
@@ -14,9 +14,10 @@#include "xe_device.h" #include "xe_assert.h" #include "xe_macros.h" -#include "xe_eudebug_types.h" #include "xe_eudebug.h" +#include "xe_exec_queue_types.h" +#include "xe_vm.h" /* * If there is no detected event read by userspace, during this period, assume
@@ -1104,8 +1105,178 @@ void xe_eudebug_vm_destroy(struct xe_file *xef, struct xe_vm *vm)xe_eudebug_event_put (d,vm_destroy_event (d, xef, vm)); } +static bool exec_queue_class_is_tracked (enum xe_engine_class class) +{ +return class ==XE_ENGINE_CLASS_COMPUTE || + class ==XE_ENGINE_CLASS_RENDER ; +} + +static const u16 xe_to_user_engine_class[] = { + [XE_ENGINE_CLASS_RENDER ] =DRM_XE_ENGINE_CLASS_RENDER , + [XE_ENGINE_CLASS_COPY ] =DRM_XE_ENGINE_CLASS_COPY , + [XE_ENGINE_CLASS_VIDEO_DECODE ] =DRM_XE_ENGINE_CLASS_VIDEO_DECODE , + [XE_ENGINE_CLASS_VIDEO_ENHANCE ] =DRM_XE_ENGINE_CLASS_VIDEO_ENHANCE , + [XE_ENGINE_CLASS_COMPUTE ] =DRM_XE_ENGINE_CLASS_COMPUTE , +}; + +static int send_exec_queue_event (struct xe_eudebug *d,u32 flags, +u64 client_handle,u64 vm_handle, +u64 exec_queue_handle,enum xe_engine_class class, +u32 width,u64 *lrc_handles,u64 seqno) +{ +struct xe_eudebug_event *event; +struct xe_eudebug_event_exec_queue *e; +const u32 sz =struct_size (e, lrc_handle, width); +const u32 xe_engine_class = xe_to_user_engine_class[class]; + +if (!exec_queue_class_is_tracked (class)) +return -EINVAL ; + + event =xe_eudebug_create_event (d,DRM_XE_EUDEBUG_EVENT_EXEC_QUEUE , + seqno, flags, sz,GFP_KERNEL ); +if (!event) +return -ENOMEM ; + + e =cast_event (e, event); + + write_member(struct drm_xe_eudebug_event_exec_queue, e, client_handle, client_handle); + write_member(struct drm_xe_eudebug_event_exec_queue, e, vm_handle, vm_handle); + write_member(struct drm_xe_eudebug_event_exec_queue, e, exec_queue_handle, + exec_queue_handle); + write_member(struct drm_xe_eudebug_event_exec_queue, e, engine_class, xe_engine_class); + write_member(struct drm_xe_eudebug_event_exec_queue, e, width, width); + +memcpy (e->lrc_handle, lrc_handles, width); + +return xe_eudebug_queue_event (d, event); +} + +static int exec_queue_create_event (struct xe_eudebug *d, +struct xe_file *xef,struct xe_exec_queue *q) +{ +int h_c, h_vm, h_queue; +u64 h_lrc[XE_HW_ENGINE_MAX_INSTANCE ], seqno; +int i; + + h_c =find_handle (d->res,XE_EUDEBUG_RES_TYPE_CLIENT , xef); +if (h_c <0 ) +return h_c; + + h_vm =find_handle (d->res,XE_EUDEBUG_RES_TYPE_VM , q->vm); +if (h_vm <0 ) +return h_vm; + +if (XE_WARN_ON (q->width >=XE_HW_ENGINE_MAX_INSTANCE )) +return -EINVAL ; + +for (i =0 ; i < q->width; i++) { +int h, ret; + + ret =_xe_eudebug_add_handle (d, +XE_EUDEBUG_RES_TYPE_LRC , + q->lrc[i], +NULL , + &h); + +if (ret <0 && ret != -EEXIST ) +return ret; + +XE_WARN_ON (!h); + + h_lrc[i] = h; + } + + h_queue =xe_eudebug_add_handle (d,XE_EUDEBUG_RES_TYPE_EXEC_QUEUE , q, &seqno); +if (h_queue <=0 ) +return h_queue; + +/* No need to cleanup for added handles on error as if we fail +* we disconnect +*/ + +return send_exec_queue_event (d,DRM_XE_EUDEBUG_EVENT_CREATE , + h_c, h_vm, h_queue, q->class, + q->width, h_lrc, seqno); +} + +static int exec_queue_destroy_event (struct xe_eudebug *d, +struct xe_file *xef, +struct xe_exec_queue *q) +{ +int h_c, h_vm, h_queue; +u64 h_lrc[XE_HW_ENGINE_MAX_INSTANCE ], seqno; +int i; + + h_c =find_handle (d->res,XE_EUDEBUG_RES_TYPE_CLIENT , xef); +if (h_c <0 ) +return h_c; + + h_vm =find_handle (d->res,XE_EUDEBUG_RES_TYPE_VM , q->vm); +if (h_vm <0 ) +return h_vm; + + +if (XE_WARN_ON (q->width >=XE_HW_ENGINE_MAX_INSTANCE )) +return -EINVAL ; + + h_queue =xe_eudebug_remove_handle (d, +XE_EUDEBUG_RES_TYPE_EXEC_QUEUE , + q, + &seqno); +if (h_queue <=0 ) +return h_queue; + +for (i =0 ; i < q->width; i++) { +int ret; + + ret =_xe_eudebug_remove_handle (d, +XE_EUDEBUG_RES_TYPE_LRC , + q->lrc[i], +NULL ); +if (ret <0 && ret != -ENOENT ) +return ret; + +XE_WARN_ON (!ret); + + h_lrc[i] = ret; + } + +return send_exec_queue_event (d,DRM_XE_EUDEBUG_EVENT_DESTROY , + h_c, h_vm, h_queue, q->class, + q->width, h_lrc, seqno); +} + +void xe_eudebug_exec_queue_create (struct xe_file *xef,struct xe_exec_queue *q) +{ +struct xe_eudebug *d; + +if (!exec_queue_class_is_tracked (q->class)) +return ; + + d =xe_eudebug_get (xef); +if (!d) +return ; + +xe_eudebug_event_put (d,exec_queue_create_event (d, xef, q)); +} + +void xe_eudebug_exec_queue_destroy (struct xe_file *xef,struct xe_exec_queue *q) +{ +struct xe_eudebug *d; + +if (!exec_queue_class_is_tracked (q->class)) +return ; + + d =xe_eudebug_get (xef); +if (!d) +return ; + +xe_eudebug_event_put (d,exec_queue_destroy_event (d, xef, q)); +} +static int discover_client (struct xe_eudebug *d,struct xe_file *xef) { +struct xe_exec_queue *q;struct xe_vm *vm;unsigned long i;int err;
@@ -1122,6 +1293,17 @@ static int discover_client(struct xe_eudebug *d, struct xe_file *xef) }mutex_unlock (&xef->vm.lock); +mutex_lock (&xef->exec_queue.lock); +xa_for_each (&xef->exec_queue.xa, i, q) { +if (!exec_queue_class_is_tracked (q->class)) +continue ; + + err =exec_queue_create_event (d, xef, q); +if (err) +break ; + } +mutex_unlock (&xef->exec_queue.lock); +return err; }
drivers/gpu/drm/xe/xe_eudebug.h +4 -0
@@ -10,6 +10,7 @@ struct drm_file;struct xe_device ;struct xe_file ;struct xe_vm ; +struct xe_exec_queue ; int xe_eudebug_connect_ioctl(struct drm_device *dev, void *data,
@@ -24,4 +25,7 @@ void xe_eudebug_file_close(struct xe_file *xef);void xe_eudebug_vm_create (struct xe_file *xef,struct xe_vm *vm);void xe_eudebug_vm_destroy (struct xe_file *xef,struct xe_vm *vm); +void xe_eudebug_exec_queue_create (struct xe_file *xef,struct xe_exec_queue *q); +void xe_eudebug_exec_queue_destroy (struct xe_file *xef,struct xe_exec_queue *q); +#endif
drivers/gpu/drm/xe/xe_eudebug_types.h +30 -1
@@ -50,7 +50,9 @@ struct xe_eudebug_resource {#define XE_EUDEBUG_RES_TYPE_CLIENT 0#define XE_EUDEBUG_RES_TYPE_VM 1 -#define XE_EUDEBUG_RES_TYPE_COUNT (XE_EUDEBUG_RES_TYPE_VM + 1) +#define XE_EUDEBUG_RES_TYPE_EXEC_QUEUE 2 +#define XE_EUDEBUG_RES_TYPE_LRC 3 +#define XE_EUDEBUG_RES_TYPE_COUNT (XE_EUDEBUG_RES_TYPE_LRC + 1)/** * struct xe_eudebug_resources - eudebug resources for all types
@@ -173,4 +175,31 @@ struct xe_eudebug_event_vm { u64 vm_handle; }; +/** +* struct xe_eudebug_event_exec_queue - Internal event for +* exec_queue create/destroy +*/ +struct xe_eudebug_event_exec_queue { +/** @base: base event */ +struct xe_eudebug_event base; + +/** @client_handle: client for the engine create/destroy */ +u64 client_handle; + +/** @vm_handle: vm handle for the engine create/destroy */ +u64 vm_handle; + +/** @exec_queue_handle: engine handle */ +u64 exec_queue_handle; + +/** @engine_handle: engine class */ +u32 engine_class; + +/** @width: submission width (number BB per exec) for this exec queue */ +u32 width; + +/** @lrc_handles: handles for each logical ring context created with this exec queue */ +u64 lrc_handle[]; +}; +#endif
drivers/gpu/drm/xe/xe_exec_queue.c +5 -0
@@ -22,6 +22,7 @@#include "xe_ring_ops_types.h" #include "xe_trace.h" #include "xe_vm.h" +#include "xe_eudebug.h" enum xe_exec_queue_sched_prop {XE_EXEC_QUEUE_JOB_TIMEOUT =0 ,
@@ -626,6 +627,8 @@ int xe_exec_queue_create_ioctl(struct drm_device *dev, void *data, args->exec_queue_id = id; q->xef =xe_file_get (xef); +xe_eudebug_exec_queue_create (xef, q); +return 0 ; kill_exec_queue:
@@ -797,6 +800,8 @@ int xe_exec_queue_destroy_ioctl(struct drm_device *dev, void *data,if (XE_IOCTL_DBG (xe, !q))return -ENOENT ; +xe_eudebug_exec_queue_destroy (xef, q); +xe_exec_queue_kill (q);trace_xe_exec_queue_close (q);
include/uapi/drm/xe_drm_eudebug.h +13 -1
@@ -26,7 +26,8 @@ struct drm_xe_eudebug_event {#define DRM_XE_EUDEBUG_EVENT_READ 1#define DRM_XE_EUDEBUG_EVENT_OPEN 2#define DRM_XE_EUDEBUG_EVENT_VM 3 -#define DRM_XE_EUDEBUG_EVENT_MAX_EVENT DRM_XE_EUDEBUG_EVENT_VM +#define DRM_XE_EUDEBUG_EVENT_EXEC_QUEUE 4 +#define DRM_XE_EUDEBUG_EVENT_MAX_EVENT DRM_XE_EUDEBUG_EVENT_EXEC_QUEUE__u16 flags;#define DRM_XE_EUDEBUG_EVENT_CREATE (1 << 0)
@@ -50,6 +51,17 @@ struct drm_xe_eudebug_event_vm {__u64 vm_handle; }; +struct drm_xe_eudebug_event_exec_queue { +struct drm_xe_eudebug_event base; + +__u64 client_handle; +__u64 vm_handle; +__u64 exec_queue_handle; +__u32 engine_class; +__u32 width; +__u64 lrc_handle[]; +}; +#if defined(__cplusplus) }#endif
Signature · 1 line
2.34.1
PATCH 8/21 drm/xe/eudebug: hw enablement for eudebug
In reply to Mika KuoppalaFrom: Dominik Grzegorzek <[email protected]>
In order to turn on debug capabilities, (i.e. breakpoints), TD_CTL and some other registers needs to be programmed. Implement eudebug mode enabling including eudebug related workarounds.
1) Enable exceptions only on platforms that does not have per context debugging functionality
2) Fixup 22015693276:
Use lineage HSD instead of per platform specific, be precise defining graphics range.
--- drivers/gpu/drm/xe/regs/xe_engine_regs.h | 4 ++ drivers/gpu/drm/xe/regs/xe_gt_regs.h | 10 +++++ drivers/gpu/drm/xe/xe_eudebug.c | 50 ++++++++++++++++++++++++ drivers/gpu/drm/xe/xe_eudebug.h | 2 + drivers/gpu/drm/xe/xe_hw_engine.c | 2 + 5 files changed, 68 insertions(+)
drivers/gpu/drm/xe/regs/xe_engine_regs.h +4 -0
@@ -114,6 +114,10 @@#define INDIRECT_RING_STATE (base) XE_REG((base) + 0x108) +#define CS_DEBUG_MODE2 (base) XE_REG((base) + 0xd8, XE_REG_OPTION_MASKED) +#define INST_STATE_CACHE_INVALIDATE REG_BIT(6) +#define GLOBAL_DEBUG_ENABLE REG_BIT(5) +#define RING_BBADDR (base) XE_REG((base) + 0x140)#define RING_BBADDR_UDW (base) XE_REG((base) + 0x168)
drivers/gpu/drm/xe/regs/xe_gt_regs.h +10 -0
@@ -425,6 +425,14 @@#define DG2_DISABLE_ROUND_ENABLE_ALLOW_FOR_SSLA REG_BIT(15)#define CLEAR_OPTIMIZATION_DISABLE REG_BIT(6) +#define TD_CTL XE_REG_MCR(0xe400) +#define TD_CTL_FEH_AND_FEE_ENABLE REG_BIT(7)/* forced halt and exception */ +#define TD_CTL_FORCE_EXTERNAL_HALT REG_BIT(6) +#define TD_CTL_FORCE_THREAD_BREAKPOINT_ENABLE REG_BIT(4) +#define TD_CTL_FORCE_EXCEPTION REG_BIT(3) +#define TD_CTL_BREAKPOINT_ENABLE REG_BIT(2) +#define TD_CTL_GLOBAL_DEBUG_ENABLE REG_BIT(0)/* XeHP */ +#define CACHE_MODE_SS XE_REG_MCR(0xe420, XE_REG_OPTION_MASKED)#define DISABLE_ECC REG_BIT(5)#define ENABLE_PREFETCH_INTO_IC REG_BIT(3)
@@ -450,11 +458,13 @@#define MDQ_ARBITRATION_MODE REG_BIT(12)#define STALL_DOP_GATING_DISABLE REG_BIT(5)#define EARLY_EOT_DIS REG_BIT(1) +#define STALL_DOP_GATING_DISABLE REG_BIT(5)#define ROW_CHICKEN2 XE_REG_MCR(0xe4f4, XE_REG_OPTION_MASKED)#define DISABLE_READ_SUPPRESSION REG_BIT(15)#define DISABLE_EARLY_READ REG_BIT(14)#define ENABLE_LARGE_GRF_MODE REG_BIT(12) +#define XEHPC_DISABLE_BTB REG_BIT(11)#define PUSH_CONST_DEREF_HOLD_DIS REG_BIT(8)#define DISABLE_TDL_SVHS_GATING REG_BIT(1)#define DISABLE_DOP_GATING REG_BIT(0)
drivers/gpu/drm/xe/xe_eudebug.c +50 -0
69 lines not on this page · raw message
drivers/gpu/drm/xe/xe_eudebug.h +2 -0
@@ -11,6 +11,7 @@ struct xe_device;struct xe_file ;struct xe_vm ;struct xe_exec_queue ; +struct xe_hw_engine ; int xe_eudebug_connect_ioctl(struct drm_device *dev, void *data,
@@ -18,6 +19,7 @@ int xe_eudebug_connect_ioctl(struct drm_device *dev,void xe_eudebug_init (struct xe_device *xe);void xe_eudebug_fini (struct xe_device *xe); +void xe_eudebug_init_hw_engine (struct xe_hw_engine *hwe);void xe_eudebug_file_open (struct xe_file *xef);void xe_eudebug_file_close (struct xe_file *xef);
drivers/gpu/drm/xe/xe_hw_engine.c +2 -0
@@ -15,6 +15,7 @@#include "xe_assert.h" #include "xe_bo.h" #include "xe_device.h" +#include "xe_eudebug.h" #include "xe_execlist.h" #include "xe_force_wake.h" #include "xe_gsc.h"
@@ -503,6 +504,7 @@ static void hw_engine_init_early(struct xe_gt *gt, struct xe_hw_engine *hwe,xe_tuning_process_engine (hwe);xe_wa_process_engine (hwe);hw_engine_setup_default_state (hwe); +xe_eudebug_init_hw_engine (hwe);xe_reg_sr_init (&hwe->reg_whitelist, hwe->name,gt_to_xe (gt));xe_reg_whitelist_process_engine (hwe);
Signature · 1 line
2.34.1
Re: [PATCH 08/21] drm/xe/eudebug: hw enablement for eudebug
Quoted 5 lines · Mika Kuoppala
From: Dominik Grzegorzek <[email protected]>
In order to turn on debug capabilities, (i.e. breakpoints),
TD_CTLand some other registers needs to be programmed. Implement eudebug mode enabling including eudebug related workarounds.
nack for the workadound part. See below.
Quoted 119 lines
1) Enable exceptions only on platforms that does not have per context debugging functionality
2) Fixup 22015693276:
Use lineage HSD instead of per platform specific, be precise defining graphics range.Signed-off-by Dominik Grzegorzek <[email protected]>Signed-off-by Mika Kuoppala <[email protected]>---
drivers/gpu/drm/xe/regs/xe_engine_regs.h | 4 ++ drivers/gpu/drm/xe/regs/xe_gt_regs.h | 10 +++++ drivers/gpu/drm/xe/xe_eudebug.c | 50 ++++++++++++++++++++++++ drivers/gpu/drm/xe/xe_eudebug.h | 2 + drivers/gpu/drm/xe/xe_hw_engine.c | 2 + 5 files changed, 68 insertions(+)drivers/gpu/drm/xe/regs/xe_engine_regs.h +4
-0@@ -114,6 +114,10 @@#define INDIRECT_RING_STATE (base) XE_REG((base) + 0x108) +#define CS_DEBUG_MODE2 (base) XE_REG((base) + 0xd8, XE_REG_OPTION_MASKED) +#define INST_STATE_CACHE_INVALIDATE REG_BIT(6) +#define GLOBAL_DEBUG_ENABLE REG_BIT(5) +#define RING_BBADDR (base) XE_REG((base) + 0x140)#define RING_BBADDR_UDW (base) XE_REG((base) + 0x168)drivers/gpu/drm/xe/regs/xe_gt_regs.h +10
-0@@ -425,6 +425,14 @@#define DG2_DISABLE_ROUND_ENABLE_ALLOW_FOR_SSLA REG_BIT(15)#define CLEAR_OPTIMIZATION_DISABLE REG_BIT(6) +#define TD_CTL XE_REG_MCR(0xe400) +#define TD_CTL_FEH_AND_FEE_ENABLE REG_BIT(7)/* forced halt and exception */ +#define TD_CTL_FORCE_EXTERNAL_HALT REG_BIT(6) +#define TD_CTL_FORCE_THREAD_BREAKPOINT_ENABLE REG_BIT(4) +#define TD_CTL_FORCE_EXCEPTION REG_BIT(3) +#define TD_CTL_BREAKPOINT_ENABLE REG_BIT(2) +#define TD_CTL_GLOBAL_DEBUG_ENABLE REG_BIT(0)/* XeHP */ +#define CACHE_MODE_SS XE_REG_MCR(0xe420, XE_REG_OPTION_MASKED)#define DISABLE_ECC REG_BIT(5)#define ENABLE_PREFETCH_INTO_IC REG_BIT(3)@@ -450,11 +458,13 @@#define MDQ_ARBITRATION_MODE REG_BIT(12)#define STALL_DOP_GATING_DISABLE REG_BIT(5)#define EARLY_EOT_DIS REG_BIT(1) +#define STALL_DOP_GATING_DISABLE REG_BIT(5)#define ROW_CHICKEN2 XE_REG_MCR(0xe4f4, XE_REG_OPTION_MASKED)#define DISABLE_READ_SUPPRESSION REG_BIT(15)#define DISABLE_EARLY_READ REG_BIT(14)#define ENABLE_LARGE_GRF_MODE REG_BIT(12) +#define XEHPC_DISABLE_BTB REG_BIT(11)#define PUSH_CONST_DEREF_HOLD_DIS REG_BIT(8)#define DISABLE_TDL_SVHS_GATING REG_BIT(1)#define DISABLE_DOP_GATING REG_BIT(0)drivers/gpu/drm/xe/xe_eudebug.c +31
-0@@ -11,13 +11,19 @@#include <drm/drm_managed.h> +#include "regs/xe_gt_regs.h" +#include "regs/xe_engine_regs.h" #include "xe_device.h" #include "xe_assert.h" #include "xe_macros.h" +#include "xe_gt.h" #include "xe_eudebug_types.h" #include "xe_eudebug.h" #include "xe_exec_queue_types.h" +#include "xe_module.h" +#include "xe_rtp.h" #include "xe_vm.h" +#include "xe_wa.h" /* * If there is no detected event read by userspace, during this period, assume @@ -925,6 +931,50 @@ int xe_eudebug_connect_ioctl(struct drm_device *dev,return ret; } +#undef XE_REG_MCR +#define XE_REG_MCR (...) XE_REG(__VA_ARGS__, .mcr = 1) + +void xe_eudebug_init_hw_engine (struct xe_hw_engine *hwe) +{ +const struct xe_rtp_entry_sr eudebug_was[] = { + {XE_RTP_NAME ("GlobalDebugEnable" ), +XE_RTP_RULES (GRAPHICS_VERSION_RANGE (1200 ,1210 ), +ENGINE_CLASS (RENDER )), +XE_RTP_ACTIONS (SET (CS_DEBUG_MODE2 (RENDER_RING_BASE ), +GLOBAL_DEBUG_ENABLE )) + }, + {XE_RTP_NAME ("TdCtlDebugEnable" ), +XE_RTP_RULES (GRAPHICS_VERSION_RANGE (1200 ,3499 ), +FUNC (xe_rtp_match_first_render_or_compute)), +XE_RTP_ACTIONS (SET (TD_CTL , +TD_CTL_BREAKPOINT_ENABLE | +TD_CTL_FORCE_THREAD_BREAKPOINT_ENABLE | +TD_CTL_FEH_AND_FEE_ENABLE )) + }, + {XE_RTP_NAME ("TdCtlGlobalDebugEnable" ), +XE_RTP_RULES (GRAPHICS_VERSION_RANGE (1250 ,XE_RTP_END_VERSION_UNDEFINED ), +FUNC (xe_rtp_match_first_render_or_compute)), +XE_RTP_ACTIONS (SET (TD_CTL ,TD_CTL_GLOBAL_DEBUG_ENABLE )) + },
drivers/gpu/drm/xe/xe_eudebug.c int xe_eudebug_connect_ioctl(struct drm_device *dev, · line 968
Quoted 10 lines
drivers/gpu/drm/xe/xe_eudebug.c +10
-0+ {XE_RTP_NAME ("18022722726" ), +XE_RTP_RULES (GRAPHICS_VERSION_RANGE (1250 ,1274 ), +FUNC (xe_rtp_match_first_render_or_compute)), +XE_RTP_ACTIONS (SET (ROW_CHICKEN ,STALL_DOP_GATING_DISABLE )) + }, + { XE_RTP_NAME("14015527279"), +XE_RTP_RULES (PLATFORM (PVC ), +FUNC (xe_rtp_match_first_render_or_compute)), +XE_RTP_ACTIONS (SET (ROW_CHICKEN2 ,XEHPC_DISABLE_BTB )) + },
workarounds don't belong here. All workarounds should be moved to xe_wa.c, which also has tracking for active workarounds and make them show up in debugfs.
tuning and other non-wa stuff may be added separately, like we have in xe_tuning.c, xe_hw_engine.c etc.
Lucas De Marchi
Quoted 55 lines
drivers/gpu/drm/xe/xe_eudebug.c +9
-0+ {} + }; +struct xe_rtp_process_ctx ctx =XE_RTP_PROCESS_CTX_INITIALIZER (hwe); +struct xe_device *xe =gt_to_xe (hwe->gt); + +if (xe->eudebug.available) +xe_rtp_process_to_sr (&ctx, eudebug_was, &hwe->reg_sr); +} +void xe_eudebug_init (struct xe_device *xe) {spin_lock_init (&xe->eudebug.lock);drivers/gpu/drm/xe/xe_eudebug.h +2
-0@@ -11,6 +11,7 @@ struct xe_device;struct xe_file ;struct xe_vm ;struct xe_exec_queue ; +struct xe_hw_engine ; int xe_eudebug_connect_ioctl(struct drm_device *dev, void *data,@@ -18,6 +19,7 @@ int xe_eudebug_connect_ioctl(struct drm_device *dev,void xe_eudebug_init (struct xe_device *xe);void xe_eudebug_fini (struct xe_device *xe); +void xe_eudebug_init_hw_engine (struct xe_hw_engine *hwe);void xe_eudebug_file_open (struct xe_file *xef);void xe_eudebug_file_close (struct xe_file *xef);drivers/gpu/drm/xe/xe_hw_engine.c +2
-0@@ -15,6 +15,7 @@#include "xe_assert.h" #include "xe_bo.h" #include "xe_device.h" +#include "xe_eudebug.h" #include "xe_execlist.h" #include "xe_force_wake.h" #include "xe_gsc.h" @@ -503,6 +504,7 @@ static void hw_engine_init_early(struct xe_gt *gt, struct xe_hw_engine *hwe,xe_tuning_process_engine (hwe);xe_wa_process_engine (hwe);hw_engine_setup_default_state (hwe); +xe_eudebug_init_hw_engine (hwe);xe_reg_sr_init (&hwe->reg_whitelist, hwe->name,gt_to_xe (gt));xe_reg_whitelist_process_engine (hwe);Signature · 1 line
2.34.1
Re: [PATCH 08/21] drm/xe/eudebug: hw enablement for eudebug
Quoted 16 lines · Lucas De Marchi
Quoted 5 lines · Mika Kuoppala
From: Dominik Grzegorzek <[email protected]>
In order to turn on debug capabilities, (i.e. breakpoints),
TD_CTLand some other registers needs to be programmed. Implement eudebug mode enabling including eudebug related workarounds.nack for the workadound part. See below.
Quoted 7 lines
1) Enable exceptions only on platforms that does not have per context debugging functionality
2) Fixup 22015693276:
Use lineage HSD instead of per platform specific, be precise defining graphics range.
also no idea what "1)" and "2)" are here.
Lucas De Marchi
Re: [PATCH 08/21] drm/xe/eudebug: hw enablement for eudebug
In reply to Lucas De MarchiQuoted 143 lines · Lucas De Marchi
Quoted 5 lines · Mika Kuoppala
From: Dominik Grzegorzek <[email protected]>
In order to turn on debug capabilities, (i.e. breakpoints),
TD_CTLand some other registers needs to be programmed. Implement eudebug mode enabling including eudebug related workarounds.nack for the workadound part. See below.
Quoted 119 lines
1) Enable exceptions only on platforms that does not have per context debugging functionality
2) Fixup 22015693276:
Use lineage HSD instead of per platform specific, be precise defining graphics range.Signed-off-by Dominik Grzegorzek <[email protected]>Signed-off-by Mika Kuoppala <[email protected]>---
drivers/gpu/drm/xe/regs/xe_engine_regs.h | 4 ++ drivers/gpu/drm/xe/regs/xe_gt_regs.h | 10 +++++ drivers/gpu/drm/xe/xe_eudebug.c | 50 ++++++++++++++++++++++++ drivers/gpu/drm/xe/xe_eudebug.h | 2 + drivers/gpu/drm/xe/xe_hw_engine.c | 2 + 5 files changed, 68 insertions(+)drivers/gpu/drm/xe/regs/xe_engine_regs.h +4
-0@@ -114,6 +114,10 @@#define INDIRECT_RING_STATE (base) XE_REG((base) + 0x108) +#define CS_DEBUG_MODE2 (base) XE_REG((base) + 0xd8, XE_REG_OPTION_MASKED) +#define INST_STATE_CACHE_INVALIDATE REG_BIT(6) +#define GLOBAL_DEBUG_ENABLE REG_BIT(5) +#define RING_BBADDR (base) XE_REG((base) + 0x140)#define RING_BBADDR_UDW (base) XE_REG((base) + 0x168)drivers/gpu/drm/xe/regs/xe_gt_regs.h +10
-0@@ -425,6 +425,14 @@#define DG2_DISABLE_ROUND_ENABLE_ALLOW_FOR_SSLA REG_BIT(15)#define CLEAR_OPTIMIZATION_DISABLE REG_BIT(6) +#define TD_CTL XE_REG_MCR(0xe400) +#define TD_CTL_FEH_AND_FEE_ENABLE REG_BIT(7)/* forced halt and exception */ +#define TD_CTL_FORCE_EXTERNAL_HALT REG_BIT(6) +#define TD_CTL_FORCE_THREAD_BREAKPOINT_ENABLE REG_BIT(4) +#define TD_CTL_FORCE_EXCEPTION REG_BIT(3) +#define TD_CTL_BREAKPOINT_ENABLE REG_BIT(2) +#define TD_CTL_GLOBAL_DEBUG_ENABLE REG_BIT(0)/* XeHP */ +#define CACHE_MODE_SS XE_REG_MCR(0xe420, XE_REG_OPTION_MASKED)#define DISABLE_ECC REG_BIT(5)#define ENABLE_PREFETCH_INTO_IC REG_BIT(3)@@ -450,11 +458,13 @@#define MDQ_ARBITRATION_MODE REG_BIT(12)#define STALL_DOP_GATING_DISABLE REG_BIT(5)#define EARLY_EOT_DIS REG_BIT(1) +#define STALL_DOP_GATING_DISABLE REG_BIT(5)#define ROW_CHICKEN2 XE_REG_MCR(0xe4f4, XE_REG_OPTION_MASKED)#define DISABLE_READ_SUPPRESSION REG_BIT(15)#define DISABLE_EARLY_READ REG_BIT(14)#define ENABLE_LARGE_GRF_MODE REG_BIT(12) +#define XEHPC_DISABLE_BTB REG_BIT(11)#define PUSH_CONST_DEREF_HOLD_DIS REG_BIT(8)#define DISABLE_TDL_SVHS_GATING REG_BIT(1)#define DISABLE_DOP_GATING REG_BIT(0)drivers/gpu/drm/xe/xe_eudebug.c +31
-0@@ -11,13 +11,19 @@#include <drm/drm_managed.h> +#include "regs/xe_gt_regs.h" +#include "regs/xe_engine_regs.h" #include "xe_device.h" #include "xe_assert.h" #include "xe_macros.h" +#include "xe_gt.h" #include "xe_eudebug_types.h" #include "xe_eudebug.h" #include "xe_exec_queue_types.h" +#include "xe_module.h" +#include "xe_rtp.h" #include "xe_vm.h" +#include "xe_wa.h" /* * If there is no detected event read by userspace, during this period, assume @@ -925,6 +931,50 @@ int xe_eudebug_connect_ioctl(struct drm_device *dev,return ret; } +#undef XE_REG_MCR +#define XE_REG_MCR (...) XE_REG(__VA_ARGS__, .mcr = 1) + +void xe_eudebug_init_hw_engine (struct xe_hw_engine *hwe) +{ +const struct xe_rtp_entry_sr eudebug_was[] = { + {XE_RTP_NAME ("GlobalDebugEnable" ), +XE_RTP_RULES (GRAPHICS_VERSION_RANGE (1200 ,1210 ), +ENGINE_CLASS (RENDER )), +XE_RTP_ACTIONS (SET (CS_DEBUG_MODE2 (RENDER_RING_BASE ), +GLOBAL_DEBUG_ENABLE )) + }, + {XE_RTP_NAME ("TdCtlDebugEnable" ), +XE_RTP_RULES (GRAPHICS_VERSION_RANGE (1200 ,3499 ), +FUNC (xe_rtp_match_first_render_or_compute)), +XE_RTP_ACTIONS (SET (TD_CTL , +TD_CTL_BREAKPOINT_ENABLE | +TD_CTL_FORCE_THREAD_BREAKPOINT_ENABLE | +TD_CTL_FEH_AND_FEE_ENABLE )) + }, + {XE_RTP_NAME ("TdCtlGlobalDebugEnable" ), +XE_RTP_RULES (GRAPHICS_VERSION_RANGE (1250 ,XE_RTP_END_VERSION_UNDEFINED ), +FUNC (xe_rtp_match_first_render_or_compute)), +XE_RTP_ACTIONS (SET (TD_CTL ,TD_CTL_GLOBAL_DEBUG_ENABLE )) + },drivers/gpu/drm/xe/xe_eudebug.c int xe_eudebug_connect_ioctl(struct drm_device *dev, · line 968
Quoted 10 lines
drivers/gpu/drm/xe/xe_eudebug.c +10
-0+ {XE_RTP_NAME ("18022722726" ), +XE_RTP_RULES (GRAPHICS_VERSION_RANGE (1250 ,1274 ), +FUNC (xe_rtp_match_first_render_or_compute)), +XE_RTP_ACTIONS (SET (ROW_CHICKEN ,STALL_DOP_GATING_DISABLE )) + }, + { XE_RTP_NAME("14015527279"), +XE_RTP_RULES (PLATFORM (PVC ), +FUNC (xe_rtp_match_first_render_or_compute)), +XE_RTP_ACTIONS (SET (ROW_CHICKEN2 ,XEHPC_DISABLE_BTB )) + },workarounds don't belong here. All workarounds should be moved to
xe_wa.c, which also has tracking for active workarounds and make them show up in debugfs.
Quoted 4 lines
tuning and other non-wa stuff may be added separately, like we have in
xe_tuning.c,xe_hw_engine.cetc.Lucas De Marchi
The thing is that these workarounds are purely specific to EU debug. For instance, the description in 18022722726 states, "When enabling the Debug feature, [...]" I assumed that there is no need to enable them when the debugger is not active to limit any negative interference, and we address this later in the series. Please take a look on "[PATCH 20/21] drm/xe/eudebug: Dynamically toggle debugger functionality." Could you also share your thoughts on that?
xe_rtp isn't really designed for dynamic changes and we may be bending it a bit there. If so, we would appreciate any ideas on how to dynamically feed GuC restore list in a proper way.
Regards,
Dominik
Quoted 56 lines
Quoted 56 lines
drivers/gpu/drm/xe/xe_eudebug.c +9
-0+ {} + }; +struct xe_rtp_process_ctx ctx =XE_RTP_PROCESS_CTX_INITIALIZER (hwe); +struct xe_device *xe =gt_to_xe (hwe->gt); + +if (xe->eudebug.available) +xe_rtp_process_to_sr (&ctx, eudebug_was, &hwe->reg_sr); +} +void xe_eudebug_init (struct xe_device *xe) {spin_lock_init (&xe->eudebug.lock);drivers/gpu/drm/xe/xe_eudebug.h +2
-0@@ -11,6 +11,7 @@ struct xe_device;struct xe_file ;struct xe_vm ;struct xe_exec_queue ; +struct xe_hw_engine ; int xe_eudebug_connect_ioctl(struct drm_device *dev, void *data,@@ -18,6 +19,7 @@ int xe_eudebug_connect_ioctl(struct drm_device *dev,void xe_eudebug_init (struct xe_device *xe);void xe_eudebug_fini (struct xe_device *xe); +void xe_eudebug_init_hw_engine (struct xe_hw_engine *hwe);void xe_eudebug_file_open (struct xe_file *xef);void xe_eudebug_file_close (struct xe_file *xef);drivers/gpu/drm/xe/xe_hw_engine.c +2
-0@@ -15,6 +15,7 @@#include "xe_assert.h" #include "xe_bo.h" #include "xe_device.h" +#include "xe_eudebug.h" #include "xe_execlist.h" #include "xe_force_wake.h" #include "xe_gsc.h" @@ -503,6 +504,7 @@ static void hw_engine_init_early(struct xe_gt *gt, struct xe_hw_engine *hwe,xe_tuning_process_engine (hwe);xe_wa_process_engine (hwe);hw_engine_setup_default_state (hwe); +xe_eudebug_init_hw_engine (hwe);xe_reg_sr_init (&hwe->reg_whitelist, hwe->name,gt_to_xe (gt));xe_reg_whitelist_process_engine (hwe);Signature · 1 line
2.34.1
Re: [PATCH 08/21] drm/xe/eudebug: hw enablement for eudebug
Quoted 144 lines · Dominik
Quoted 143 lines · Lucas De Marchi
Quoted 5 lines · Mika Kuoppala
From: Dominik Grzegorzek <[email protected]>
In order to turn on debug capabilities, (i.e. breakpoints),
TD_CTLand some other registers needs to be programmed. Implement eudebug mode enabling including eudebug related workarounds.nack for the workadound part. See below.
Quoted 119 lines
1) Enable exceptions only on platforms that does not have per context debugging functionality
2) Fixup 22015693276:
Use lineage HSD instead of per platform specific, be precise defining graphics range.Signed-off-by Dominik Grzegorzek <[email protected]>Signed-off-by Mika Kuoppala <[email protected]>---
drivers/gpu/drm/xe/regs/xe_engine_regs.h | 4 ++ drivers/gpu/drm/xe/regs/xe_gt_regs.h | 10 +++++ drivers/gpu/drm/xe/xe_eudebug.c | 50 ++++++++++++++++++++++++ drivers/gpu/drm/xe/xe_eudebug.h | 2 + drivers/gpu/drm/xe/xe_hw_engine.c | 2 + 5 files changed, 68 insertions(+)drivers/gpu/drm/xe/regs/xe_engine_regs.h +4
-0@@ -114,6 +114,10 @@#define INDIRECT_RING_STATE (base) XE_REG((base) + 0x108) +#define CS_DEBUG_MODE2 (base) XE_REG((base) + 0xd8, XE_REG_OPTION_MASKED) +#define INST_STATE_CACHE_INVALIDATE REG_BIT(6) +#define GLOBAL_DEBUG_ENABLE REG_BIT(5) +#define RING_BBADDR (base) XE_REG((base) + 0x140)#define RING_BBADDR_UDW (base) XE_REG((base) + 0x168)drivers/gpu/drm/xe/regs/xe_gt_regs.h +10
-0@@ -425,6 +425,14 @@#define DG2_DISABLE_ROUND_ENABLE_ALLOW_FOR_SSLA REG_BIT(15)#define CLEAR_OPTIMIZATION_DISABLE REG_BIT(6) +#define TD_CTL XE_REG_MCR(0xe400) +#define TD_CTL_FEH_AND_FEE_ENABLE REG_BIT(7)/* forced halt and exception */ +#define TD_CTL_FORCE_EXTERNAL_HALT REG_BIT(6) +#define TD_CTL_FORCE_THREAD_BREAKPOINT_ENABLE REG_BIT(4) +#define TD_CTL_FORCE_EXCEPTION REG_BIT(3) +#define TD_CTL_BREAKPOINT_ENABLE REG_BIT(2) +#define TD_CTL_GLOBAL_DEBUG_ENABLE REG_BIT(0)/* XeHP */ +#define CACHE_MODE_SS XE_REG_MCR(0xe420, XE_REG_OPTION_MASKED)#define DISABLE_ECC REG_BIT(5)#define ENABLE_PREFETCH_INTO_IC REG_BIT(3)@@ -450,11 +458,13 @@#define MDQ_ARBITRATION_MODE REG_BIT(12)#define STALL_DOP_GATING_DISABLE REG_BIT(5)#define EARLY_EOT_DIS REG_BIT(1) +#define STALL_DOP_GATING_DISABLE REG_BIT(5)#define ROW_CHICKEN2 XE_REG_MCR(0xe4f4, XE_REG_OPTION_MASKED)#define DISABLE_READ_SUPPRESSION REG_BIT(15)#define DISABLE_EARLY_READ REG_BIT(14)#define ENABLE_LARGE_GRF_MODE REG_BIT(12) +#define XEHPC_DISABLE_BTB REG_BIT(11)#define PUSH_CONST_DEREF_HOLD_DIS REG_BIT(8)#define DISABLE_TDL_SVHS_GATING REG_BIT(1)#define DISABLE_DOP_GATING REG_BIT(0)drivers/gpu/drm/xe/xe_eudebug.c +31
-0@@ -11,13 +11,19 @@#include <drm/drm_managed.h> +#include "regs/xe_gt_regs.h" +#include "regs/xe_engine_regs.h" #include "xe_device.h" #include "xe_assert.h" #include "xe_macros.h" +#include "xe_gt.h" #include "xe_eudebug_types.h" #include "xe_eudebug.h" #include "xe_exec_queue_types.h" +#include "xe_module.h" +#include "xe_rtp.h" #include "xe_vm.h" +#include "xe_wa.h" /* * If there is no detected event read by userspace, during this period, assume @@ -925,6 +931,50 @@ int xe_eudebug_connect_ioctl(struct drm_device *dev,return ret; } +#undef XE_REG_MCR +#define XE_REG_MCR (...) XE_REG(__VA_ARGS__, .mcr = 1) + +void xe_eudebug_init_hw_engine (struct xe_hw_engine *hwe) +{ +const struct xe_rtp_entry_sr eudebug_was[] = { + {XE_RTP_NAME ("GlobalDebugEnable" ), +XE_RTP_RULES (GRAPHICS_VERSION_RANGE (1200 ,1210 ), +ENGINE_CLASS (RENDER )), +XE_RTP_ACTIONS (SET (CS_DEBUG_MODE2 (RENDER_RING_BASE ), +GLOBAL_DEBUG_ENABLE )) + }, + {XE_RTP_NAME ("TdCtlDebugEnable" ), +XE_RTP_RULES (GRAPHICS_VERSION_RANGE (1200 ,3499 ), +FUNC (xe_rtp_match_first_render_or_compute)), +XE_RTP_ACTIONS (SET (TD_CTL , +TD_CTL_BREAKPOINT_ENABLE | +TD_CTL_FORCE_THREAD_BREAKPOINT_ENABLE | +TD_CTL_FEH_AND_FEE_ENABLE )) + }, + {XE_RTP_NAME ("TdCtlGlobalDebugEnable" ), +XE_RTP_RULES (GRAPHICS_VERSION_RANGE (1250 ,XE_RTP_END_VERSION_UNDEFINED ), +FUNC (xe_rtp_match_first_render_or_compute)), +XE_RTP_ACTIONS (SET (TD_CTL ,TD_CTL_GLOBAL_DEBUG_ENABLE )) + },drivers/gpu/drm/xe/xe_eudebug.c int xe_eudebug_connect_ioctl(struct drm_device *dev, · line 968
Quoted 10 lines
drivers/gpu/drm/xe/xe_eudebug.c +10
-0+ {XE_RTP_NAME ("18022722726" ), +XE_RTP_RULES (GRAPHICS_VERSION_RANGE (1250 ,1274 ), +FUNC (xe_rtp_match_first_render_or_compute)), +XE_RTP_ACTIONS (SET (ROW_CHICKEN ,STALL_DOP_GATING_DISABLE )) + }, + { XE_RTP_NAME("14015527279"), +XE_RTP_RULES (PLATFORM (PVC ), +FUNC (xe_rtp_match_first_render_or_compute)), +XE_RTP_ACTIONS (SET (ROW_CHICKEN2 ,XEHPC_DISABLE_BTB )) + },workarounds don't belong here. All workarounds should be moved to
xe_wa.c, which also has tracking for active workarounds and make them show up in debugfs.
Quoted 8 lines
Quoted 4 lines
tuning and other non-wa stuff may be added separately, like we have in
xe_tuning.c,xe_hw_engine.cetc.Lucas De Marchi
The thing is that these workarounds are purely specific to EU debug. For instance, the description in 18022722726 states, "When enabling the Debug feature, [...]" I assumed that there is no need to enable them when
ok, we may have to do something else for that then:
1) if there's no side-effect of setting it, enable it unconditionally
2) set it as an OOB WA - this will enable the WA tracking and allow us to check the WAs implemented for each platform
(1) is the simplest one
For (2), then if you are using an RTP table for enabling/disabling you basically check if that OOB WA is enabled as the rule. Currently you will need to split some rules as the OOB WAs are gt-centric (so the engine check part would still be in outside OOB). +Matt Roper if he has other ideas.
Quoted 6 lines
the debugger is not active to limit any negative interference, and we address this later in the series. Please take a look on "[PATCH 20/21] drm/xe/eudebug: Dynamically toggle debugger functionality." Could you also share your thoughts on that?
xe_rtpisn't really designed for dynamic changes and we may be bending it
that part is beyond rtp. RTP is mainly about getting a bunch of entries on a table. Applying the outcome of that table to the hw is left to xe_sr and guc-ads
Quoted 2 lines
a bit there. If so, we would appreciate any ideas on how to dynamically feed GuC restore list in a proper way.
will try to take a look later today, but I believe you will need tweak the ADS and reset GuC.
Lucas De Marchi
Quoted 60 lines
Regards,
DominikQuoted 56 lines
Quoted 56 lines
drivers/gpu/drm/xe/xe_eudebug.c +9
-0+ {} + }; +struct xe_rtp_process_ctx ctx =XE_RTP_PROCESS_CTX_INITIALIZER (hwe); +struct xe_device *xe =gt_to_xe (hwe->gt); + +if (xe->eudebug.available) +xe_rtp_process_to_sr (&ctx, eudebug_was, &hwe->reg_sr); +} +void xe_eudebug_init (struct xe_device *xe) {spin_lock_init (&xe->eudebug.lock);drivers/gpu/drm/xe/xe_eudebug.h +2
-0@@ -11,6 +11,7 @@ struct xe_device;struct xe_file ;struct xe_vm ;struct xe_exec_queue ; +struct xe_hw_engine ; int xe_eudebug_connect_ioctl(struct drm_device *dev, void *data,@@ -18,6 +19,7 @@ int xe_eudebug_connect_ioctl(struct drm_device *dev,void xe_eudebug_init (struct xe_device *xe);void xe_eudebug_fini (struct xe_device *xe); +void xe_eudebug_init_hw_engine (struct xe_hw_engine *hwe);void xe_eudebug_file_open (struct xe_file *xef);void xe_eudebug_file_close (struct xe_file *xef);drivers/gpu/drm/xe/xe_hw_engine.c +2
-1@@ -15,6 +15,7 @@#include "xe_assert.h" #include "xe_bo.h" #include "xe_device.h" +#include "xe_eudebug.h" #include "xe_execlist.h" #include "xe_force_wake.h" #include "xe_gsc.h" @@ -503,6 +504,7 @@ static void hw_engine_init_early(struct xe_gt *gt, struct xe_hw_engine *hwe,xe_tuning_process_engine (hwe);xe_wa_process_engine (hwe);hw_engine_setup_default_state (hwe); +xe_eudebug_init_hw_engine (hwe);xe_reg_sr_init (&hwe->reg_whitelist, hwe->name,gt_to_xe (gt));xe_reg_whitelist_process_engine (hwe); --2.34.1
PATCH 9/21 drm/xe: Add EUDEBUG_ENABLE exec queue property
In reply to Mika KuoppalaFrom: Dominik Grzegorzek <[email protected]>
Introduce exec queue immutable property of eudebug with a flags as value to enable eudebug specific feature(s).
For now engine lrc will use this flag to set up runalone hw feature. Runalone is used to ensure that only one hw engine of group [rcs0, ccs0-3] is active on a tile.
Note: unlike the i915, xe allows user to set runalone also on devices with single render/compute engine. It should not make much difference, but leave control to the user.
v2: use exec queue flags (Mika)
v3: eudebug enable as flags (Mika)
v4: adapt to lrc_create (Mika)
v5: EUDEBUG property space squash (Mika)
v6: runalone as rmw (Dominik)
--- drivers/gpu/drm/xe/regs/xe_engine_regs.h | 1 + drivers/gpu/drm/xe/xe_exec_queue.c | 35 ++++++++++++++++++++++-- drivers/gpu/drm/xe/xe_exec_queue_types.h | 7 +++++ drivers/gpu/drm/xe/xe_hw_engine.c | 2 +- drivers/gpu/drm/xe/xe_lrc.c | 16 +++++++++-- drivers/gpu/drm/xe/xe_lrc.h | 4 ++- include/uapi/drm/xe_drm.h | 3 +- 7 files changed, 60 insertions(+), 8 deletions(-)
drivers/gpu/drm/xe/regs/xe_engine_regs.h +1 -0
@@ -136,6 +136,7 @@#define CTX_CTRL_OAC_CONTEXT_ENABLE REG_BIT(8)#define CTX_CTRL_RUN_ALONE REG_BIT(7)#define CTX_CTRL_INDIRECT_RING_STATE_ENABLE REG_BIT(4) +#define CTX_CTRL_RUN_ALONE REG_BIT(7)#define CTX_CTRL_INHIBIT_SYN_CTX_SWITCH REG_BIT(3)#define CTX_CTRL_ENGINE_CTX_RESTORE_INHIBIT REG_BIT(0)
drivers/gpu/drm/xe/xe_exec_queue.c +33 -2
@@ -106,10 +106,14 @@ static struct xe_exec_queue *__xe_exec_queue_alloc(struct xe_device *xe,static int __xe_exec_queue_init (struct xe_exec_queue *q) { +u32 flags =0 ;int i, err; +if (q->eudebug_flags &EXEC_QUEUE_EUDEBUG_FLAG_ENABLE ) + flags |=LRC_CREATE_RUNALONE ; +for (i =0 ; i < q->width; ++i) { - q->lrc[i] =xe_lrc_create (q->hwe, q->vm,SZ_16K ); + q->lrc[i] =xe_lrc_create (q->hwe, q->vm,SZ_16K , flags);if (IS_ERR (q->lrc[i])) { err =PTR_ERR (q->lrc[i]); goto err_lrc;
@@ -336,6 +340,31 @@ static int exec_queue_set_timeslice(struct xe_device *xe, struct xe_exec_queue *return 0 ; } +static int exec_queue_set_eudebug (struct xe_device *xe,struct xe_exec_queue *q, +u64 value) +{ +const u64 known_flags =DRM_XE_EXEC_QUEUE_EUDEBUG_FLAG_ENABLE ; + +if (XE_IOCTL_DBG (xe, (q->class !=XE_ENGINE_CLASS_RENDER && + q->class !=XE_ENGINE_CLASS_COMPUTE ))) +return -EINVAL ; + +if (XE_IOCTL_DBG (xe, (value & ~known_flags))) +return -EINVAL ; + +/* +* We want to explicitly set the global feature if +* property is set. +*/ +if (XE_IOCTL_DBG (xe, + !(value &DRM_XE_EXEC_QUEUE_EUDEBUG_FLAG_ENABLE ))) +return -EINVAL ; + + q->eudebug_flags =EXEC_QUEUE_EUDEBUG_FLAG_ENABLE ; + +return 0 ; +} +typedef int (*xe_exec_queue_set_property_fn )(struct xe_device *xe,struct xe_exec_queue *q,u64 value);
@@ -343,6 +372,7 @@ typedef int (*xe_exec_queue_set_property_fn)(struct xe_device *xe,static const xe_exec_queue_set_property_fn exec_queue_set_property_funcs[] = { [DRM_XE_EXEC_QUEUE_SET_PROPERTY_PRIORITY ] = exec_queue_set_priority, [DRM_XE_EXEC_QUEUE_SET_PROPERTY_TIMESLICE ] = exec_queue_set_timeslice, + [DRM_XE_EXEC_QUEUE_SET_PROPERTY_EUDEBUG ] = exec_queue_set_eudebug, }; static int exec_queue_user_ext_set_property(struct xe_device *xe,
@@ -362,7 +392,8 @@ static int exec_queue_user_ext_set_property(struct xe_device *xe, ARRAY_SIZE(exec_queue_set_property_funcs)) ||XE_IOCTL_DBG (xe, ext.pad) ||XE_IOCTL_DBG (xe, ext.property !=DRM_XE_EXEC_QUEUE_SET_PROPERTY_PRIORITY && - ext.property != DRM_XE_EXEC_QUEUE_SET_PROPERTY_TIMESLICE)) + ext.property !=DRM_XE_EXEC_QUEUE_SET_PROPERTY_TIMESLICE && + ext.property != DRM_XE_EXEC_QUEUE_SET_PROPERTY_EUDEBUG)) return -EINVAL; idx =array_index_nospec (ext.property,ARRAY_SIZE (exec_queue_set_property_funcs));
drivers/gpu/drm/xe/xe_exec_queue_types.h +7 -0
@@ -90,6 +90,13 @@ struct xe_exec_queue {*/ unsigned long flags; +/** +* @eudebug_flags: immutable eudebug flags for this exec queue. +* Set up with DRM_XE_EXEC_QUEUE_SET_PROPERTY_EUDEBUG. +*/ +#define EXEC_QUEUE_EUDEBUG_FLAG_ENABLE BIT(0) +unsigned long eudebug_flags; +union {/** @multi_gt_list: list head for VM bind engines if multi-GT */ struct list_head multi_gt_list;
drivers/gpu/drm/xe/xe_hw_engine.c +1 -1
@@ -532,7 +532,7 @@ static int hw_engine_init(struct xe_gt *gt, struct xe_hw_engine *hwe, goto err_name; } - hwe->kernel_lrc =xe_lrc_create (hwe,NULL ,SZ_16K ); + hwe->kernel_lrc =xe_lrc_create (hwe,NULL ,SZ_16K ,0 );if (IS_ERR (hwe->kernel_lrc)) { err =PTR_ERR (hwe->kernel_lrc); goto err_hwsp;
drivers/gpu/drm/xe/xe_lrc.c +13 -3
@@ -890,7 +890,7 @@ static void xe_lrc_finish(struct xe_lrc *lrc)#define PVC_CTX_ACC_CTR_THOLD (0x2a + 1)static int xe_lrc_init (struct xe_lrc *lrc,struct xe_hw_engine *hwe, -struct xe_vm *vm,u32 ring_size) +struct xe_vm *vm,u32 ring_size,u32 flags) {struct xe_gt *gt = hwe->gt;struct xe_tile *tile =gt_to_tile (gt);
@@ -1007,6 +1007,16 @@ static int xe_lrc_init(struct xe_lrc *lrc, struct xe_hw_engine *hwe, map =__xe_lrc_start_seqno_map (lrc);xe_map_write32 (lrc_to_xe (lrc), &map, lrc->fence_ctx.next_seqno -1 ); +if (flags &LRC_CREATE_RUNALONE ) { +u32 ctx_control =xe_lrc_read_ctx_reg (lrc,CTX_CONTEXT_CONTROL ); + +drm_dbg (&xe->drm,"read CTX_CONTEXT_CONTROL: 0x%x\n" , ctx_control); + ctx_control |=_MASKED_BIT_ENABLE (CTX_CTRL_RUN_ALONE ); +drm_dbg (&xe->drm,"written CTX_CONTEXT_CONTROL: 0x%x\n" , ctx_control); + +xe_lrc_write_ctx_reg (lrc,CTX_CONTEXT_CONTROL , ctx_control); + } +return 0 ; err_lrc_finish:
@@ -1026,7 +1036,7 @@ static int xe_lrc_init(struct xe_lrc *lrc, struct xe_hw_engine *hwe,* upon failure. */ struct xe_lrc *xe_lrc_create (struct xe_hw_engine *hwe,struct xe_vm *vm, -u32 ring_size) +u32 ring_size,u32 flags) {struct xe_lrc *lrc;int err;
@@ -1035,7 +1045,7 @@ struct xe_lrc *xe_lrc_create(struct xe_hw_engine *hwe, struct xe_vm *vm,if (!lrc)return ERR_PTR (-ENOMEM ); - err =xe_lrc_init (lrc, hwe, vm, ring_size); + err =xe_lrc_init (lrc, hwe, vm, ring_size, flags);if (err) {kfree (lrc);return ERR_PTR (err);
drivers/gpu/drm/xe/xe_lrc.h +3 -1
@@ -22,8 +22,10 @@ struct xe_vm;#define LRC_PPHWSP_SCRATCH_ADDR (0x34 * 4) +#define LRC_CREATE_RUNALONE BIT(0) +struct xe_lrc *xe_lrc_create (struct xe_hw_engine *hwe,struct xe_vm *vm, -u32 ring_size); +u32 ring_size,u32 flags);void xe_lrc_destroy (struct kref *ref);/**
include/uapi/drm/xe_drm.h +2 -1
@@ -1110,7 +1110,8 @@ struct drm_xe_exec_queue_create {#define DRM_XE_EXEC_QUEUE_EXTENSION_SET_PROPERTY 0#define DRM_XE_EXEC_QUEUE_SET_PROPERTY_PRIORITY 0#define DRM_XE_EXEC_QUEUE_SET_PROPERTY_TIMESLICE 1 - +#define DRM_XE_EXEC_QUEUE_SET_PROPERTY_EUDEBUG 2 +#define DRM_XE_EXEC_QUEUE_EUDEBUG_FLAG_ENABLE (1 << 0)/** @extensions: Pointer to the first extension struct, if any */ __u64 extensions;
Signature · 1 line
2.34.1
Matthew Brost
maintainerRe: [PATCH 09/21] drm/xe: Add EUDEBUG_ENABLE exec queue property
Quoted 66 lines · Mika Kuoppala
From: Dominik Grzegorzek <[email protected]>
Introduce exec queue immutable property of eudebug with a flags as value to enable eudebug specific feature(s).
For now engine lrc will use this flag to set up runalone hw feature. Runalone is used to ensure that only one hw engine of group [rcs0, ccs0-3] is active on a tile.
Note: unlike the i915, xe allows user to set runalone also on devices with single render/compute engine. It should not make much difference, but leave control to the user.
v2: use exec queue flags (Mika)
v3: eudebug enable as flags (Mika)
v4: adapt tolrc_create(Mika)
v5: EUDEBUG property space squash (Mika)
v6: runalone as rmw (Dominik)Signed-off-by Dominik Grzegorzek <[email protected]>Signed-off-by Mika Kuoppala <[email protected]>--- drivers/gpu/drm/xe/regs/xe_engine_regs.h | 1 + drivers/gpu/drm/xe/xe_exec_queue.c | 35 ++++++++++++++++++++++-- drivers/gpu/drm/xe/xe_exec_queue_types.h | 7 +++++ drivers/gpu/drm/xe/xe_hw_engine.c | 2 +- drivers/gpu/drm/xe/xe_lrc.c | 16 +++++++++-- drivers/gpu/drm/xe/xe_lrc.h | 4 ++- include/uapi/drm/xe_drm.h | 3 +- 7 files changed, 60 insertions(+), 8 deletions(-)drivers/gpu/drm/xe/regs/xe_engine_regs.h +1
-0@@ -136,6 +136,7 @@#define CTX_CTRL_OAC_CONTEXT_ENABLE REG_BIT(8)#define CTX_CTRL_RUN_ALONE REG_BIT(7)#define CTX_CTRL_INDIRECT_RING_STATE_ENABLE REG_BIT(4) +#define CTX_CTRL_RUN_ALONE REG_BIT(7)#define CTX_CTRL_INHIBIT_SYN_CTX_SWITCH REG_BIT(3)#define CTX_CTRL_ENGINE_CTX_RESTORE_INHIBIT REG_BIT(0)drivers/gpu/drm/xe/xe_exec_queue.c +5
-1@@ -106,10 +106,14 @@ static struct xe_exec_queue *__xe_exec_queue_alloc(struct xe_device *xe,static int __xe_exec_queue_init (struct xe_exec_queue *q) { +u32 flags =0 ;int i, err; +if (q->eudebug_flags &EXEC_QUEUE_EUDEBUG_FLAG_ENABLE ) + flags |=LRC_CREATE_RUNALONE ; +for (i =0 ; i < q->width; ++i) { - q->lrc[i] =xe_lrc_create (q->hwe, q->vm,SZ_16K ); + q->lrc[i] =xe_lrc_create (q->hwe, q->vm,SZ_16K , flags);if (IS_ERR (q->lrc[i])) { err =PTR_ERR (q->lrc[i]); goto err_lrc;@@ -336,6 +340,31 @@ static int exec_queue_set_timeslice(struct xe_device *xe, struct xe_exec_queue *return 0 ; }
drivers/gpu/drm/xe/xe_exec_queue.c static int exec_queue_set_timeslice(struct xe_device *xe, struct xe_exec_queue * · line 351
Quoted 10 lines
drivers/gpu/drm/xe/xe_exec_queue.c +9
-0+static int exec_queue_set_eudebug (struct xe_device *xe,struct xe_exec_queue *q, +u64 value) +{ +const u64 known_flags =DRM_XE_EXEC_QUEUE_EUDEBUG_FLAG_ENABLE ; + +if (XE_IOCTL_DBG (xe, (q->class !=XE_ENGINE_CLASS_RENDER && + q->class !=XE_ENGINE_CLASS_COMPUTE ))) +return -EINVAL ; +
The EU debugger only works on LR VMs, right? i.e. We can't use dma-fences with EU debugger enabled. If so, then I'd check the q->vm to ensure it is in a support mode.
Matt
Quoted 149 lines
drivers/gpu/drm/xe/xe_exec_queue.c +19
-1+if (XE_IOCTL_DBG (xe, (value & ~known_flags))) +return -EINVAL ; + +/* +* We want to explicitly set the global feature if +* property is set. +*/ +if (XE_IOCTL_DBG (xe, + !(value &DRM_XE_EXEC_QUEUE_EUDEBUG_FLAG_ENABLE ))) +return -EINVAL ; + + q->eudebug_flags =EXEC_QUEUE_EUDEBUG_FLAG_ENABLE ; + +return 0 ; +} +typedef int (*xe_exec_queue_set_property_fn )(struct xe_device *xe,struct xe_exec_queue *q,u64 value);@@ -343,6 +372,7 @@ typedef int (*xe_exec_queue_set_property_fn)(struct xe_device *xe,static const xe_exec_queue_set_property_fn exec_queue_set_property_funcs[] = { [DRM_XE_EXEC_QUEUE_SET_PROPERTY_PRIORITY ] = exec_queue_set_priority, [DRM_XE_EXEC_QUEUE_SET_PROPERTY_TIMESLICE ] = exec_queue_set_timeslice, + [DRM_XE_EXEC_QUEUE_SET_PROPERTY_EUDEBUG ] = exec_queue_set_eudebug, }; static int exec_queue_user_ext_set_property(struct xe_device *xe,@@ -362,7 +392,8 @@ static int exec_queue_user_ext_set_property(struct xe_device *xe, ARRAY_SIZE(exec_queue_set_property_funcs)) ||XE_IOCTL_DBG (xe, ext.pad) ||XE_IOCTL_DBG (xe, ext.property !=DRM_XE_EXEC_QUEUE_SET_PROPERTY_PRIORITY && - ext.property != DRM_XE_EXEC_QUEUE_SET_PROPERTY_TIMESLICE)) + ext.property !=DRM_XE_EXEC_QUEUE_SET_PROPERTY_TIMESLICE && + ext.property != DRM_XE_EXEC_QUEUE_SET_PROPERTY_EUDEBUG)) return -EINVAL; idx =array_index_nospec (ext.property,ARRAY_SIZE (exec_queue_set_property_funcs));drivers/gpu/drm/xe/xe_exec_queue_types.h +7
-0@@ -90,6 +90,13 @@ struct xe_exec_queue {*/ unsigned long flags; +/** +* @eudebug_flags: immutable eudebug flags for this exec queue. +* Set up with DRM_XE_EXEC_QUEUE_SET_PROPERTY_EUDEBUG. +*/ +#define EXEC_QUEUE_EUDEBUG_FLAG_ENABLE BIT(0) +unsigned long eudebug_flags; +union {/** @multi_gt_list: list head for VM bind engines if multi-GT */ struct list_head multi_gt_list;drivers/gpu/drm/xe/xe_hw_engine.c +1
-1@@ -532,7 +532,7 @@ static int hw_engine_init(struct xe_gt *gt, struct xe_hw_engine *hwe, goto err_name; } - hwe->kernel_lrc =xe_lrc_create (hwe,NULL ,SZ_16K ); + hwe->kernel_lrc =xe_lrc_create (hwe,NULL ,SZ_16K ,0 );if (IS_ERR (hwe->kernel_lrc)) { err =PTR_ERR (hwe->kernel_lrc); goto err_hwsp;drivers/gpu/drm/xe/xe_lrc.c +13
-3@@ -890,7 +890,7 @@ static void xe_lrc_finish(struct xe_lrc *lrc)#define PVC_CTX_ACC_CTR_THOLD (0x2a + 1)static int xe_lrc_init (struct xe_lrc *lrc,struct xe_hw_engine *hwe, -struct xe_vm *vm,u32 ring_size) +struct xe_vm *vm,u32 ring_size,u32 flags) {struct xe_gt *gt = hwe->gt;struct xe_tile *tile =gt_to_tile (gt);@@ -1007,6 +1007,16 @@ static int xe_lrc_init(struct xe_lrc *lrc, struct xe_hw_engine *hwe, map =__xe_lrc_start_seqno_map (lrc);xe_map_write32 (lrc_to_xe (lrc), &map, lrc->fence_ctx.next_seqno -1 ); +if (flags &LRC_CREATE_RUNALONE ) { +u32 ctx_control =xe_lrc_read_ctx_reg (lrc,CTX_CONTEXT_CONTROL ); + +drm_dbg (&xe->drm,"read CTX_CONTEXT_CONTROL: 0x%x\n" , ctx_control); + ctx_control |=_MASKED_BIT_ENABLE (CTX_CTRL_RUN_ALONE ); +drm_dbg (&xe->drm,"written CTX_CONTEXT_CONTROL: 0x%x\n" , ctx_control); + +xe_lrc_write_ctx_reg (lrc,CTX_CONTEXT_CONTROL , ctx_control); + } +return 0 ; err_lrc_finish:@@ -1026,7 +1036,7 @@ static int xe_lrc_init(struct xe_lrc *lrc, struct xe_hw_engine *hwe,* upon failure. */ struct xe_lrc *xe_lrc_create (struct xe_hw_engine *hwe,struct xe_vm *vm, -u32 ring_size) +u32 ring_size,u32 flags) {struct xe_lrc *lrc;int err;@@ -1035,7 +1045,7 @@ struct xe_lrc *xe_lrc_create(struct xe_hw_engine *hwe, struct xe_vm *vm,if (!lrc)return ERR_PTR (-ENOMEM ); - err =xe_lrc_init (lrc, hwe, vm, ring_size); + err =xe_lrc_init (lrc, hwe, vm, ring_size, flags);if (err) {kfree (lrc);return ERR_PTR (err);drivers/gpu/drm/xe/xe_lrc.h +3
-1@@ -22,8 +22,10 @@ struct xe_vm;#define LRC_PPHWSP_SCRATCH_ADDR (0x34 * 4) +#define LRC_CREATE_RUNALONE BIT(0) +struct xe_lrc *xe_lrc_create (struct xe_hw_engine *hwe,struct xe_vm *vm, -u32 ring_size); +u32 ring_size,u32 flags);void xe_lrc_destroy (struct kref *ref);/** include/uapi/drm/xe_drm.h +2
-1@@ -1110,7 +1110,8 @@ struct drm_xe_exec_queue_create {#define DRM_XE_EXEC_QUEUE_EXTENSION_SET_PROPERTY 0#define DRM_XE_EXEC_QUEUE_SET_PROPERTY_PRIORITY 0#define DRM_XE_EXEC_QUEUE_SET_PROPERTY_TIMESLICE 1 - +#define DRM_XE_EXEC_QUEUE_SET_PROPERTY_EUDEBUG 2 +#define DRM_XE_EXEC_QUEUE_EUDEBUG_FLAG_ENABLE (1 << 0)/** @extensions: Pointer to the first extension struct, if any */ __u64 extensions;Signature · 1 line
2.34.1
Matthew Brost
maintainerRe: [PATCH 09/21] drm/xe: Add EUDEBUG_ENABLE exec queue property
In reply to Mika KuoppalaQuoted 69 lines · Mika Kuoppala
From: Dominik Grzegorzek <[email protected]>
Introduce exec queue immutable property of eudebug with a flags as value to enable eudebug specific feature(s).
For now engine lrc will use this flag to set up runalone hw feature. Runalone is used to ensure that only one hw engine of group [rcs0, ccs0-3] is active on a tile.
Note: unlike the i915, xe allows user to set runalone also on devices with single render/compute engine. It should not make much difference, but leave control to the user.
v2: use exec queue flags (Mika)
v3: eudebug enable as flags (Mika)
v4: adapt tolrc_create(Mika)
v5: EUDEBUG property space squash (Mika)
v6: runalone as rmw (Dominik)Signed-off-by Dominik Grzegorzek <[email protected]>Signed-off-by Mika Kuoppala <[email protected]>--- drivers/gpu/drm/xe/regs/xe_engine_regs.h | 1 + drivers/gpu/drm/xe/xe_exec_queue.c | 35 ++++++++++++++++++++++-- drivers/gpu/drm/xe/xe_exec_queue_types.h | 7 +++++ drivers/gpu/drm/xe/xe_hw_engine.c | 2 +- drivers/gpu/drm/xe/xe_lrc.c | 16 +++++++++-- drivers/gpu/drm/xe/xe_lrc.h | 4 ++- include/uapi/drm/xe_drm.h | 3 +- 7 files changed, 60 insertions(+), 8 deletions(-)drivers/gpu/drm/xe/regs/xe_engine_regs.h +1
-0@@ -136,6 +136,7 @@#define CTX_CTRL_OAC_CONTEXT_ENABLE REG_BIT(8)#define CTX_CTRL_RUN_ALONE REG_BIT(7)#define CTX_CTRL_INDIRECT_RING_STATE_ENABLE REG_BIT(4) +#define CTX_CTRL_RUN_ALONE REG_BIT(7)#define CTX_CTRL_INHIBIT_SYN_CTX_SWITCH REG_BIT(3)#define CTX_CTRL_ENGINE_CTX_RESTORE_INHIBIT REG_BIT(0)drivers/gpu/drm/xe/xe_exec_queue.c +7
-1@@ -106,10 +106,14 @@ static struct xe_exec_queue *__xe_exec_queue_alloc(struct xe_device *xe,static int __xe_exec_queue_init (struct xe_exec_queue *q) { +u32 flags =0 ;int i, err; +if (q->eudebug_flags &EXEC_QUEUE_EUDEBUG_FLAG_ENABLE ) + flags |=LRC_CREATE_RUNALONE ; +for (i =0 ; i < q->width; ++i) { - q->lrc[i] =xe_lrc_create (q->hwe, q->vm,SZ_16K ); + q->lrc[i] =xe_lrc_create (q->hwe, q->vm,SZ_16K , flags);if (IS_ERR (q->lrc[i])) { err =PTR_ERR (q->lrc[i]); goto err_lrc;@@ -336,6 +340,31 @@ static int exec_queue_set_timeslice(struct xe_device *xe, struct xe_exec_queue *return 0 ; } +static int exec_queue_set_eudebug (struct xe_device *xe,struct xe_exec_queue *q, +u64 value)
drivers/gpu/drm/xe/xe_exec_queue.c static int exec_queue_set_timeslice(struct xe_device *xe, struct xe_exec_queue * · line 354
Quoted 10 lines
drivers/gpu/drm/xe/xe_exec_queue.c +10
-0+{ +const u64 known_flags =DRM_XE_EXEC_QUEUE_EUDEBUG_FLAG_ENABLE ; + +if (XE_IOCTL_DBG (xe, (q->class !=XE_ENGINE_CLASS_RENDER && + q->class !=XE_ENGINE_CLASS_COMPUTE ))) +return -EINVAL ; + +if (XE_IOCTL_DBG (xe, (value & ~known_flags))) +return -EINVAL ; +
Since I've talked about layering a bit [1] [2] it worth mentioning I think this patch fine wrt layering.
If we add a Kconfig as I suggest [1] here you'd wanna add something like:
if (!IS_ENABLED(CONFIG_EU_DEBUG))
return -EOPNOTSUPP;
Just make sure it is unique error code return which the UMD can interrupt as 'kernel not compiled with EU debug support'.
Matt
[1] https://patchwork.freedesktop.org/patch/606049/?series=136572&rev=1#comment_1101330
[2] https://patchwork.freedesktop.org/patch/606056/?series=136572&rev=1#comment_1101337
Quoted 146 lines
drivers/gpu/drm/xe/xe_exec_queue.c +16
-1+/* +* We want to explicitly set the global feature if +* property is set. +*/ +if (XE_IOCTL_DBG (xe, + !(value &DRM_XE_EXEC_QUEUE_EUDEBUG_FLAG_ENABLE ))) +return -EINVAL ; + + q->eudebug_flags =EXEC_QUEUE_EUDEBUG_FLAG_ENABLE ; + +return 0 ; +} +typedef int (*xe_exec_queue_set_property_fn )(struct xe_device *xe,struct xe_exec_queue *q,u64 value);@@ -343,6 +372,7 @@ typedef int (*xe_exec_queue_set_property_fn)(struct xe_device *xe,static const xe_exec_queue_set_property_fn exec_queue_set_property_funcs[] = { [DRM_XE_EXEC_QUEUE_SET_PROPERTY_PRIORITY ] = exec_queue_set_priority, [DRM_XE_EXEC_QUEUE_SET_PROPERTY_TIMESLICE ] = exec_queue_set_timeslice, + [DRM_XE_EXEC_QUEUE_SET_PROPERTY_EUDEBUG ] = exec_queue_set_eudebug, }; static int exec_queue_user_ext_set_property(struct xe_device *xe,@@ -362,7 +392,8 @@ static int exec_queue_user_ext_set_property(struct xe_device *xe, ARRAY_SIZE(exec_queue_set_property_funcs)) ||XE_IOCTL_DBG (xe, ext.pad) ||XE_IOCTL_DBG (xe, ext.property !=DRM_XE_EXEC_QUEUE_SET_PROPERTY_PRIORITY && - ext.property != DRM_XE_EXEC_QUEUE_SET_PROPERTY_TIMESLICE)) + ext.property !=DRM_XE_EXEC_QUEUE_SET_PROPERTY_TIMESLICE && + ext.property != DRM_XE_EXEC_QUEUE_SET_PROPERTY_EUDEBUG)) return -EINVAL; idx =array_index_nospec (ext.property,ARRAY_SIZE (exec_queue_set_property_funcs));drivers/gpu/drm/xe/xe_exec_queue_types.h +7
-0@@ -90,6 +90,13 @@ struct xe_exec_queue {*/ unsigned long flags; +/** +* @eudebug_flags: immutable eudebug flags for this exec queue. +* Set up with DRM_XE_EXEC_QUEUE_SET_PROPERTY_EUDEBUG. +*/ +#define EXEC_QUEUE_EUDEBUG_FLAG_ENABLE BIT(0) +unsigned long eudebug_flags; +union {/** @multi_gt_list: list head for VM bind engines if multi-GT */ struct list_head multi_gt_list;drivers/gpu/drm/xe/xe_hw_engine.c +1
-1@@ -532,7 +532,7 @@ static int hw_engine_init(struct xe_gt *gt, struct xe_hw_engine *hwe, goto err_name; } - hwe->kernel_lrc =xe_lrc_create (hwe,NULL ,SZ_16K ); + hwe->kernel_lrc =xe_lrc_create (hwe,NULL ,SZ_16K ,0 );if (IS_ERR (hwe->kernel_lrc)) { err =PTR_ERR (hwe->kernel_lrc); goto err_hwsp;drivers/gpu/drm/xe/xe_lrc.c +13
-3@@ -890,7 +890,7 @@ static void xe_lrc_finish(struct xe_lrc *lrc)#define PVC_CTX_ACC_CTR_THOLD (0x2a + 1)static int xe_lrc_init (struct xe_lrc *lrc,struct xe_hw_engine *hwe, -struct xe_vm *vm,u32 ring_size) +struct xe_vm *vm,u32 ring_size,u32 flags) {struct xe_gt *gt = hwe->gt;struct xe_tile *tile =gt_to_tile (gt);@@ -1007,6 +1007,16 @@ static int xe_lrc_init(struct xe_lrc *lrc, struct xe_hw_engine *hwe, map =__xe_lrc_start_seqno_map (lrc);xe_map_write32 (lrc_to_xe (lrc), &map, lrc->fence_ctx.next_seqno -1 ); +if (flags &LRC_CREATE_RUNALONE ) { +u32 ctx_control =xe_lrc_read_ctx_reg (lrc,CTX_CONTEXT_CONTROL ); + +drm_dbg (&xe->drm,"read CTX_CONTEXT_CONTROL: 0x%x\n" , ctx_control); + ctx_control |=_MASKED_BIT_ENABLE (CTX_CTRL_RUN_ALONE ); +drm_dbg (&xe->drm,"written CTX_CONTEXT_CONTROL: 0x%x\n" , ctx_control); + +xe_lrc_write_ctx_reg (lrc,CTX_CONTEXT_CONTROL , ctx_control); + } +return 0 ; err_lrc_finish:@@ -1026,7 +1036,7 @@ static int xe_lrc_init(struct xe_lrc *lrc, struct xe_hw_engine *hwe,* upon failure. */ struct xe_lrc *xe_lrc_create (struct xe_hw_engine *hwe,struct xe_vm *vm, -u32 ring_size) +u32 ring_size,u32 flags) {struct xe_lrc *lrc;int err;@@ -1035,7 +1045,7 @@ struct xe_lrc *xe_lrc_create(struct xe_hw_engine *hwe, struct xe_vm *vm,if (!lrc)return ERR_PTR (-ENOMEM ); - err =xe_lrc_init (lrc, hwe, vm, ring_size); + err =xe_lrc_init (lrc, hwe, vm, ring_size, flags);if (err) {kfree (lrc);return ERR_PTR (err);drivers/gpu/drm/xe/xe_lrc.h +3
-1@@ -22,8 +22,10 @@ struct xe_vm;#define LRC_PPHWSP_SCRATCH_ADDR (0x34 * 4) +#define LRC_CREATE_RUNALONE BIT(0) +struct xe_lrc *xe_lrc_create (struct xe_hw_engine *hwe,struct xe_vm *vm, -u32 ring_size); +u32 ring_size,u32 flags);void xe_lrc_destroy (struct kref *ref);/** include/uapi/drm/xe_drm.h +2
-1@@ -1110,7 +1110,8 @@ struct drm_xe_exec_queue_create {#define DRM_XE_EXEC_QUEUE_EXTENSION_SET_PROPERTY 0#define DRM_XE_EXEC_QUEUE_SET_PROPERTY_PRIORITY 0#define DRM_XE_EXEC_QUEUE_SET_PROPERTY_TIMESLICE 1 - +#define DRM_XE_EXEC_QUEUE_SET_PROPERTY_EUDEBUG 2 +#define DRM_XE_EXEC_QUEUE_EUDEBUG_FLAG_ENABLE (1 << 0)/** @extensions: Pointer to the first extension struct, if any */ __u64 extensions;Signature · 1 line
2.34.1
PATCH 10/21 drm/xe/eudebug: Introduce per device attention scan worker
In reply to Mika KuoppalaFrom: Dominik Grzegorzek <[email protected]>
Scan for EU debugging attention bits periodically to detect if some EU thread has entered the system routine (SIP) due to EU thread exception.
Make the scanning interval 10 times slower when there is no debugger connection open. Send attention event whenever we see attention with debugger presence. If there is no debugger connection active - reset.
Based on work by authors and other folks who were part of attentions in i915.
- v2 Do not validate potentially active hwe against engine->hwe. Whenever the engine has width > 1, this field contains only the first hwe of the class.
- squash dss walking and semaphore to mutex
- v3 error path fix in xe_send_gt_attention (Christoph)
- v4 runalone active fix (Mika)
- v5 q->lrc changes (Mika)
- v6 Use C99 flexible arrays (Maciej, checkpatch)
function with 'for_each' in name (Maciej, checkpatch)
- v7 long running active fix (Dominik)
- v8 resource handling errors rebase (Mika)
- v9 find out lrc handles first before sending event (Mika)
- v10 adjust runalone shift according to hw
--- drivers/gpu/drm/xe/Makefile | 1 + drivers/gpu/drm/xe/regs/xe_engine_regs.h | 3 + drivers/gpu/drm/xe/regs/xe_gt_regs.h | 7 + drivers/gpu/drm/xe/xe_device.c | 2 + drivers/gpu/drm/xe/xe_device_types.h | 3 + drivers/gpu/drm/xe/xe_eudebug.c | 389 ++++++++++++++++++++++- drivers/gpu/drm/xe/xe_eudebug.h | 1 + drivers/gpu/drm/xe/xe_eudebug_types.h | 32 ++ drivers/gpu/drm/xe/xe_gt_debug.c | 152 +++++++++ drivers/gpu/drm/xe/xe_gt_debug.h | 21 ++ include/uapi/drm/xe_drm_eudebug.h | 15 +- 11 files changed, 624 insertions(+), 2 deletions(-) create mode 100644 drivers/gpu/drm/xe/xe_gt_debug.c create mode 100644 drivers/gpu/drm/xe/xe_gt_debug.h
drivers/gpu/drm/xe/Makefile +1 -0
@@ -49,6 +49,7 @@ xe-y += xe_bb.o \ xe_gt_debugfs.o \ xe_gt_freq.o \ xe_gt_idle.o \ + xe_gt_debug.o \ xe_gt_mcr.o \ xe_gt_pagefault.o \ xe_gt_sysfs.o \
drivers/gpu/drm/xe/regs/xe_engine_regs.h +3 -0
@@ -132,6 +132,9 @@#define RING_EXECLIST_STATUS_LO (base) XE_REG((base) + 0x234)#define RING_EXECLIST_STATUS_HI (base) XE_REG((base) + 0x234 + 4) +#define RING_CURRENT_LRCA (base) XE_REG((base) + 0x240) +#define CURRENT_LRCA_VALID REG_BIT(0) +#define RING_CONTEXT_CONTROL (base) XE_REG((base) + 0x244, XE_REG_OPTION_MASKED)#define CTX_CTRL_OAC_CONTEXT_ENABLE REG_BIT(8)#define CTX_CTRL_RUN_ALONE REG_BIT(7)
drivers/gpu/drm/xe/regs/xe_gt_regs.h +7 -0
@@ -437,6 +437,8 @@#define DISABLE_ECC REG_BIT(5)#define ENABLE_PREFETCH_INTO_IC REG_BIT(3) +#define TD_ATT (x) XE_REG_MCR(0xe470 + (x) * 4) +#define ROW_CHICKEN4 XE_REG_MCR(0xe48c, XE_REG_OPTION_MASKED)#define DISABLE_GRF_CLEAR REG_BIT(13)#define XEHP_DIS_BBL_SYSPIPE REG_BIT(11)
@@ -516,6 +518,11 @@#define CCS_MODE_CSLICE (cslice, ccs) \ ((ccs) << ((cslice) * CCS_MODE_CSLICE_WIDTH)) +#define RCU_DEBUG_1 XE_REG(0x14a00) +#define RCU_DEBUG_1_ENGINE_STATUS REG_GENMASK(2, 0) +#define RCU_DEBUG_1_RUNALONE_ACTIVE REG_BIT(2) +#define RCU_DEBUG_1_CONTEXT_ACTIVE REG_BIT(0) +#define FORCEWAKE_ACK_GT XE_REG(0x130044)/* Applicable for all FORCEWAKE_DOMAIN and FORCEWAKE_ACK_DOMAIN regs */
drivers/gpu/drm/xe/xe_device.c +2 -0
@@ -768,6 +768,8 @@ int xe_device_probe(struct xe_device *xe)xe_debugfs_register (xe); +xe_eudebug_init_late (xe); +xe_hwmon_register (xe);for_each_gt (gt, xe, id)
drivers/gpu/drm/xe/xe_device_types.h +3 -0
@@ -516,6 +516,9 @@ struct xe_device {/** @ordered_wq: used to discovery */ struct workqueue_struct *ordered_wq; + +/** @attention_scan: attention scan worker */ +struct delayed_work attention_scan; } eudebug;/* private: */
drivers/gpu/drm/xe/xe_eudebug.c +388 -1
437 lines not on this page · raw message
drivers/gpu/drm/xe/xe_eudebug.h +1 -0
@@ -18,6 +18,7 @@ int xe_eudebug_connect_ioctl(struct drm_device *dev, struct drm_file *file);void xe_eudebug_init (struct xe_device *xe); +void xe_eudebug_init_late (struct xe_device *xe);void xe_eudebug_fini (struct xe_device *xe);void xe_eudebug_init_hw_engine (struct xe_hw_engine *hwe);
drivers/gpu/drm/xe/xe_eudebug_types.h +32 -0
@@ -105,6 +105,9 @@ struct xe_eudebug {/** @discovery_work: worker to discover resources for target_task */ struct work_struct discovery_work; +/** eu_lock: guards operations on eus (eu thread control and attention) */ +struct mutex eu_lock; +/** @events: kfifo queue of to-be-delivered events */ struct {/** @lock: guards access to fifo */
@@ -202,4 +205,33 @@ struct xe_eudebug_event_exec_queue { u64 lrc_handle[]; }; +/** +* struct xe_eudebug_event_eu_attention - Internal event for EU attention +*/ +struct xe_eudebug_event_eu_attention { +/** @base: base event */ +struct xe_eudebug_event base; + +/** @client_handle: client for the attention */ +u64 client_handle; + +/** @exec_queue_handle: handle of exec_queue which raised attention */ +u64 exec_queue_handle; + +/** @lrc_handle: lrc handle of the workload which raised attention */ +u64 lrc_handle; + +/** @flags: eu attention event flags, currently MBZ */ +u32 flags; + +/** @bitmask_size: size of the bitmask, specific to device */ +u32 bitmask_size; + +/** +* @bitmask: reflects threads currently signalling attention, +* starting from natural hardware order of DSS=0, eu=0 +*/ +u8 bitmask[]; +}; +#endif
drivers/gpu/drm/xe/xe_gt_debug.c new file +152 -0
152 lines not on this page · raw message
drivers/gpu/drm/xe/xe_gt_debug.h new file +21 -0
@@ -0,0 +1,21 @@ +/* SPDX-License-Identifier: MIT */ +/* +* Copyright © 2023 Intel Corporation +*/ + +#ifndef __XE_GT_DEBUG_ +#define __XE_GT_DEBUG_ + +#define TD_EU_ATTENTION_MAX_ROWS 2u + +#include "xe_gt_types.h" + +#define XE_GT_ATTENTION_TIMEOUT_MS 100 + +int xe_gt_eu_threads_needing_attention (struct xe_gt *gt); + +int xe_gt_eu_attention_bitmap_size (struct xe_gt *gt); +int xe_gt_eu_attention_bitmap (struct xe_gt *gt,u8 *bits, +unsigned int bitmap_size); + +#endif
include/uapi/drm/xe_drm_eudebug.h +14 -1
@@ -27,13 +27,15 @@ struct drm_xe_eudebug_event {#define DRM_XE_EUDEBUG_EVENT_OPEN 2#define DRM_XE_EUDEBUG_EVENT_VM 3#define DRM_XE_EUDEBUG_EVENT_EXEC_QUEUE 4 -#define DRM_XE_EUDEBUG_EVENT_MAX_EVENT DRM_XE_EUDEBUG_EVENT_EXEC_QUEUE +#define DRM_XE_EUDEBUG_EVENT_EU_ATTENTION 5 +#define DRM_XE_EUDEBUG_EVENT_MAX_EVENT DRM_XE_EUDEBUG_EVENT_EU_ATTENTION__u16 flags;#define DRM_XE_EUDEBUG_EVENT_CREATE (1 << 0)#define DRM_XE_EUDEBUG_EVENT_DESTROY (1 << 1)#define DRM_XE_EUDEBUG_EVENT_STATE_CHANGE (1 << 2)#define DRM_XE_EUDEBUG_EVENT_NEED_ACK (1 << 3) +__u64 seqno;__u64 reserved; };
@@ -62,6 +64,17 @@ struct drm_xe_eudebug_event_exec_queue {__u64 lrc_handle[]; }; +struct drm_xe_eudebug_event_eu_attention { +struct drm_xe_eudebug_event base; + +__u64 client_handle; +__u64 exec_queue_handle; +__u64 lrc_handle; +__u32 flags; +__u32 bitmask_size; +__u8 bitmask[]; +}; +#if defined(__cplusplus) }#endif
Signature · 1 line
2.34.1
Matthew Brost
maintainerRe: [PATCH 10/21] drm/xe/eudebug: Introduce per device attention scan worker
Quoted 160 lines · Mika Kuoppala
From: Dominik Grzegorzek <[email protected]>
Scan for EU debugging attention bits periodically to detect if some EU thread has entered the system routine (SIP) due to EU thread exception.
Make the scanning interval 10 times slower when there is no debugger connection open. Send attention event whenever we see attention with debugger presence. If there is no debugger connection active - reset.
Based on work by authors and other folks who were part of attentions in i915.
- v2 Do not validate potentially active hwe against
engine->hwe. Whenever the engine has width > 1, this field contains only the first hwe of the class.
- squash dss walking and semaphore to mutex
- v3 error path fix inxe_send_gt_attention(Christoph)
- v4 runalone active fix (Mika)
- v5q->lrcchanges (Mika)
- v6 Use C99 flexible arrays (Maciej, checkpatch)
function with 'for_each' in name (Maciej, checkpatch)
- v7 long running active fix (Dominik)
- v8 resource handling errors rebase (Mika)
- v9 find out lrc handles first before sending event (Mika)
- v10 adjust runalone shift according to hwSigned-off-by Dominik Grzegorzek <[email protected]>Signed-off-by Christoph Manszewski <[email protected]>Signed-off-by Maciej Patelczyk <[email protected]>Signed-off-by Mika Kuoppala <[email protected]>--- drivers/gpu/drm/xe/Makefile | 1 + drivers/gpu/drm/xe/regs/xe_engine_regs.h | 3 + drivers/gpu/drm/xe/regs/xe_gt_regs.h | 7 + drivers/gpu/drm/xe/xe_device.c | 2 + drivers/gpu/drm/xe/xe_device_types.h | 3 + drivers/gpu/drm/xe/xe_eudebug.c | 389 ++++++++++++++++++++++- drivers/gpu/drm/xe/xe_eudebug.h | 1 + drivers/gpu/drm/xe/xe_eudebug_types.h | 32 ++ drivers/gpu/drm/xe/xe_gt_debug.c | 152 +++++++++ drivers/gpu/drm/xe/xe_gt_debug.h | 21 ++ include/uapi/drm/xe_drm_eudebug.h | 15 +- 11 files changed, 624 insertions(+), 2 deletions(-) create mode 100644 drivers/gpu/drm/xe/xe_gt_debug.c create mode 100644 drivers/gpu/drm/xe/xe_gt_debug.hdrivers/gpu/drm/xe/Makefile +1
-0@@ -49,6 +49,7 @@ xe-y += xe_bb.o \ xe_gt_debugfs.o \ xe_gt_freq.o \ xe_gt_idle.o \ + xe_gt_debug.o \ xe_gt_mcr.o \ xe_gt_pagefault.o \ xe_gt_sysfs.o \drivers/gpu/drm/xe/regs/xe_engine_regs.h +3
-0@@ -132,6 +132,9 @@#define RING_EXECLIST_STATUS_LO (base) XE_REG((base) + 0x234)#define RING_EXECLIST_STATUS_HI (base) XE_REG((base) + 0x234 + 4) +#define RING_CURRENT_LRCA (base) XE_REG((base) + 0x240) +#define CURRENT_LRCA_VALID REG_BIT(0) +#define RING_CONTEXT_CONTROL (base) XE_REG((base) + 0x244, XE_REG_OPTION_MASKED)#define CTX_CTRL_OAC_CONTEXT_ENABLE REG_BIT(8)#define CTX_CTRL_RUN_ALONE REG_BIT(7)drivers/gpu/drm/xe/regs/xe_gt_regs.h +7
-0@@ -437,6 +437,8 @@#define DISABLE_ECC REG_BIT(5)#define ENABLE_PREFETCH_INTO_IC REG_BIT(3) +#define TD_ATT (x) XE_REG_MCR(0xe470 + (x) * 4) +#define ROW_CHICKEN4 XE_REG_MCR(0xe48c, XE_REG_OPTION_MASKED)#define DISABLE_GRF_CLEAR REG_BIT(13)#define XEHP_DIS_BBL_SYSPIPE REG_BIT(11)@@ -516,6 +518,11 @@#define CCS_MODE_CSLICE (cslice, ccs) \ ((ccs) << ((cslice) * CCS_MODE_CSLICE_WIDTH)) +#define RCU_DEBUG_1 XE_REG(0x14a00) +#define RCU_DEBUG_1_ENGINE_STATUS REG_GENMASK(2, 0) +#define RCU_DEBUG_1_RUNALONE_ACTIVE REG_BIT(2) +#define RCU_DEBUG_1_CONTEXT_ACTIVE REG_BIT(0) +#define FORCEWAKE_ACK_GT XE_REG(0x130044)/* Applicable for all FORCEWAKE_DOMAIN and FORCEWAKE_ACK_DOMAIN regs */ drivers/gpu/drm/xe/xe_device.c +2
-0@@ -768,6 +768,8 @@ int xe_device_probe(struct xe_device *xe)xe_debugfs_register (xe); +xe_eudebug_init_late (xe); +xe_hwmon_register (xe);for_each_gt (gt, xe, id)drivers/gpu/drm/xe/xe_device_types.h +3
-0@@ -516,6 +516,9 @@ struct xe_device {/** @ordered_wq: used to discovery */ struct workqueue_struct *ordered_wq; + +/** @attention_scan: attention scan worker */ +struct delayed_work attention_scan; } eudebug;/* private: */ drivers/gpu/drm/xe/xe_eudebug.c +11
-1@@ -11,19 +11,29 @@#include <drm/drm_managed.h> -#include "regs/xe_gt_regs.h" #include "regs/xe_engine_regs.h" +#include "regs/xe_gt_regs.h" #include "xe_device.h" #include "xe_assert.h" #include "xe_macros.h" #include "xe_gt.h" +#include "xe_gt_debug.h" +#include "xe_lrc.h" +#include "xe_hw_engine.h" +#include "xe_exec_queue.h" #include "xe_eudebug_types.h" #include "xe_eudebug.h" #include "xe_exec_queue_types.h" +#include "xe_guc_exec_queue_types.h" +#include "xe_execlist_types.h" +#include "xe_mmio.h" #include "xe_module.h" +#include "xe_pm.h" #include "xe_rtp.h" +#include "xe_sched_job.h" #include "xe_vm.h" #include "xe_wa.h" +#include "xe_force_wake.h" /* * If there is no detected event read by userspace, during this period, assume
drivers/gpu/drm/xe/xe_eudebug.c static const struct file_operations fops = { · line 856
Quoted 4 lines
drivers/gpu/drm/xe/xe_eudebug.c +1
-0@@ -843,6 +853,371 @@ static const struct file_operations fops = { .unlocked_ioctl = xe_eudebug_ioctl, }; +static bool queue_has_active_job (struct xe_exec_queue *q)
This should probably be a generic function in xe_exec_queue.c. We seemly already have one - xe_exec_queue_is_idle.
drivers/gpu/drm/xe/xe_eudebug.c static const struct file_operations fops = { · line 862
Quoted 6 lines
drivers/gpu/drm/xe/xe_eudebug.c +6
-0+{ + +struct drm_gpu_scheduler *sched; +struct drm_sched_job *drm_job; + +if (xe_device_uc_enabled (gt_to_xe (q->gt)))
General comment, we likely don't want to the EU debugger enabled unless GuC submission is enabled as the execlist backend really doesn't work.
drivers/gpu/drm/xe/xe_eudebug.c static const struct file_operations fops = { · line 867
Quoted 5 lines
drivers/gpu/drm/xe/xe_eudebug.c +5
-0+ sched = &q->guc->sched.base; + else + sched = &q->execlist->sched; + + drm_job = list_first_entry_or_null(&sched->pending_list, struct drm_sched_job, list);
Random musing that don't apply as we have xe_exec_queue_is_idle but...
You need a scheduler lock here which is missing. If you wanted to see the scheduler pending list was not empty, we'd call into the drm scheduler layer.
That being said, I think the EU debugger only support LR VMs? If so, then this always going to be empty.
Quoted 159 lines
drivers/gpu/drm/xe/xe_eudebug.c +159
-0+ +if (drm_job) { +struct xe_sched_job *job =to_xe_sched_job (drm_job); + +return xe_sched_job_started (job) && !xe_sched_job_completed (job); + }else if (xe_exec_queue_is_lr (q) && + (xe_lrc_ring_head (q->lrc[0 ]) !=xe_lrc_ring_tail (q->lrc[0 ]))) { +return true; + } + +return false; +} + +static int current_lrc (struct xe_hw_engine *hwe,u32 *lrc_hw) +{ +u32 lrc_reg; +int err; + + err =xe_force_wake_get (gt_to_fw (hwe->gt), hwe->domain); +if (err) +return err; + + lrc_reg =hw_engine_mmio_read32 (hwe,RING_CURRENT_LRCA (0 )); + +xe_force_wake_put (gt_to_fw (hwe->gt), hwe->domain); + +if (!(lrc_reg &CURRENT_LRCA_VALID )) +return -ENOENT ; + + *lrc_hw = lrc_reg &GENMASK (31 ,12 ); + +return 0 ; +} + +static int match_engine_lrc (struct xe_exec_queue *q,u32 lrc_hw) +{ +int i; +u32 lrc_ggtt; + +for (i =0 ; i < q->width; i++) { + lrc_ggtt =lower_32_bits (xe_lrc_descriptor (q->lrc[i])); + lrc_ggtt &=GENMASK (31 ,12 ); +if (lrc_ggtt == lrc_hw) +return i; + } + +return -1 ; +} + +static u32 engine_status (const struct xe_hw_engine *const hwe, +u32 rcu_debug1) +{ +const bool xe1 =GRAPHICS_VER (gt_to_xe (hwe->gt)) <20 ; +unsigned int shift; + +if (hwe->class ==XE_ENGINE_CLASS_RENDER ) { + shift =7 ; +XE_WARN_ON (hwe->instance !=0 ); + }else if (hwe->class ==XE_ENGINE_CLASS_COMPUTE ) { +XE_WARN_ON (hwe->instance >3 ); + +if (xe1) + shift =10 + (hwe->instance *3 ); +else + shift =11 + (hwe->instance *4 ); + }else { +XE_WARN_ON (hwe->class); +return 0 ; + } + +return (rcu_debug1 >> shift) &RCU_DEBUG_1_ENGINE_STATUS ; +} + +static bool engine_runalone_set (const struct xe_hw_engine *const hwe, +u32 rcu_debug1) +{ +return engine_status (hwe, rcu_debug1) &RCU_DEBUG_1_RUNALONE_ACTIVE ; +} + +static bool engine_context_set (const struct xe_hw_engine *const hwe, +u32 rcu_debug1) +{ +return engine_status (hwe, rcu_debug1) &RCU_DEBUG_1_CONTEXT_ACTIVE ; +} + +static bool engine_has_runalone (const struct xe_hw_engine *const hwe) +{ +return hwe->class ==XE_ENGINE_CLASS_RENDER || + hwe->class ==XE_ENGINE_CLASS_COMPUTE ; +} + +static struct xe_hw_engine *get_runalone_active_hw_engine (struct xe_gt *gt) +{ +struct xe_hw_engine *hwe, *first =NULL ; +unsigned int num_active, id; +u32 val; + +if (xe_force_wake_get (gt_to_fw (gt),XE_FW_GT )) { +drm_dbg (>_to_xe (gt)->drm,"eudbg: runalone failed to get force wake\n" ); +return NULL ; + } + + val =xe_mmio_read32 (gt,RCU_DEBUG_1 ); +xe_force_wake_put (gt_to_fw (gt),XE_FW_GT ); + +drm_dbg (>_to_xe (gt)->drm,"eudbg: runalone RCU_DEBUG_1 = 0x%08x\n" , val); + + num_active =0 ; +for_each_hw_engine (hwe, gt, id) { +bool runalone, ctx; + +if (!engine_has_runalone (hwe)) +continue ; + + runalone =engine_runalone_set (hwe, val); + ctx =engine_context_set (hwe, val); + +drm_dbg (>_to_xe (gt)->drm,"eudbg: engine %s: runalone=%s, context=%s" , + hwe->name, runalone ?"active" :"inactive" , + ctx ?"active" :"inactive" ); + +/* +* On earlier gen12 the context status seems to be idle when +* it has raised attention. We have to omit the active bit. +*/ +if (IS_DGFX (gt_to_xe (gt))) + ctx = true; + +if (runalone && ctx) { + num_active++; + +drm_dbg (>_to_xe (gt)->drm,"eudbg: runalone engine %s %s" , + hwe->name, first ?"selected" :"found" ); +if (!first) + first = hwe; + } + } + +if (num_active >1 ) +drm_err (>_to_xe (gt)->drm,"eudbg: %d runalone engines active!" , + num_active); + +return first; +} + +static struct xe_exec_queue *runalone_active_queue_get (struct xe_gt *gt,int *lrc_idx) +{ +struct xe_device *xe =gt_to_xe (gt); +struct xe_exec_queue *q, *found =NULL ; +struct xe_hw_engine *active; +struct xe_file *xef, *tmp; +unsigned long i; +int idx, err; +u32 lrc_hw; + + active =get_runalone_active_hw_engine (gt); +if (!active) { +drm_dbg (>_to_xe (gt)->drm,"Runalone engine not found!" ); +return ERR_PTR (-ENOENT );
drivers/gpu/drm/xe/xe_eudebug.c static const struct file_operations fops = { · line 1036
Quoted 10 lines
drivers/gpu/drm/xe/xe_eudebug.c +10
-0+ } + + err =current_lrc (active, &lrc_hw); +if (err) +return ERR_PTR (err); + +mutex_lock (&xe->files.lock); +list_for_each_entry_safe (xef, tmp, &xe->files.list, link) { +mutex_lock (&xef->exec_queue.lock); +xa_for_each (&xef->exec_queue.xa, i, q) {
Same comment here as [1]. Don't hold xe->files.lock or xef->exec_queue.lock for anything but the lookup. Holding locks longer than needed creates unwanted lock dep chains which creates all sorts of problems. The only time chains should exist is when we have well defined locking chain like we have for a VM (vm->lock -> dma-resv -> notifier lock).
So this should look something like this:
mutex_lock(&xe->files.lock);
xa_for_each(.., xef) {
xe_file_get(xef);
mutex_unlock(&xe->files.lock);
mutex_lock(&xef->exec_queue.lock);
xa_for_each(&xef->exec_queue.xa, i, q) {
xe_exec_queue_get(q);
mutex_unlock(&xef->exec_queue.lock);
/* Do something */
xe_exec_queue_put(q);
mutex_lock(&xef->exec_queue.lock);
}
mutex_unlock(&xef->exec_queue.lock);
xe_file_put(xef);
mutex_lock(&xe->files.lock);
}mutex_unlock(&xe->files.lock);
Note you will need to change xe->files.list to an xarray to make this safe too.
Matt
[1] https://patchwork.freedesktop.org/patch/606052/?series=136572&rev=1
Quoted 513 lines
drivers/gpu/drm/xe/xe_eudebug.c +196
-0+if (q->gt != gt) +continue ; + +if (q->class != active->class) +continue ; + +if (!queue_has_active_job (q)) +continue ; + + idx =match_engine_lrc (q, lrc_hw); +if (idx <0 ) +continue ; + +xe_exec_queue_get (q); + found = q; + +if (lrc_idx) + *lrc_idx = idx; + +break ; + } +mutex_unlock (&xef->exec_queue.lock); + +if (found) +break ; + } +mutex_unlock (&xe->files.lock); + +if (!found) +return ERR_PTR (-ENOENT ); + +if (XE_WARN_ON (current_lrc (active, &lrc_hw)) && +XE_WARN_ON (match_engine_lrc (found, lrc_hw) <0 )) { +xe_exec_queue_put (found); +return ERR_PTR (-ENOENT ); + } + +return found; +} + +static int send_attention_event (struct xe_eudebug *d,struct xe_exec_queue *q,int lrc_idx) +{ +struct xe_eudebug_event_eu_attention *ea; +struct xe_eudebug_event *event; +int h_c, h_queue, h_lrc; +u32 size =xe_gt_eu_attention_bitmap_size (q->gt); +u32 sz =struct_size (ea, bitmask, size); +int ret; + +XE_WARN_ON (lrc_idx <0 || lrc_idx >= q->width); + + h_c =find_handle (d->res,XE_EUDEBUG_RES_TYPE_CLIENT , q->vm->xef); +if (h_c <0 ) +return h_c; + + h_queue =find_handle (d->res,XE_EUDEBUG_RES_TYPE_EXEC_QUEUE , q); +if (h_queue <0 ) +return h_queue; + + h_lrc =find_handle (d->res,XE_EUDEBUG_RES_TYPE_LRC , q->lrc[lrc_idx]); +if (h_lrc <0 ) +return h_lrc; + + event =__xe_eudebug_create_event (d,0 ,DRM_XE_EUDEBUG_EVENT_EU_ATTENTION , +DRM_XE_EUDEBUG_EVENT_STATE_CHANGE , sz,GFP_KERNEL ); + +if (!event) +return -ENOSPC ; + + ea =cast_event (ea, event); + write_member(struct drm_xe_eudebug_event_eu_attention, ea, client_handle, (u64)h_c); + write_member(struct drm_xe_eudebug_event_eu_attention, ea, exec_queue_handle, (u64)h_queue); + write_member(struct drm_xe_eudebug_event_eu_attention, ea, lrc_handle, (u64)h_lrc); + write_member(struct drm_xe_eudebug_event_eu_attention, ea, bitmask_size, size); + +mutex_lock (&d->eu_lock); + event->seqno =atomic_long_inc_return (&d->events.seqno); + ret =xe_gt_eu_attention_bitmap (q->gt, &ea->bitmask[0 ], ea->bitmask_size); +mutex_unlock (&d->eu_lock); + +if (ret) +return ret; + +return xe_eudebug_queue_event (d, event); +} + + +static int xe_send_gt_attention (struct xe_gt *gt) +{ +struct xe_eudebug *d; +struct xe_exec_queue *q; +int ret, lrc_idx; + +if (list_empty_careful (>_to_xe (gt)->eudebug.list)) +return -ENOTCONN ; + + q =runalone_active_queue_get (gt, &lrc_idx); +if (IS_ERR (q)) +return PTR_ERR (q); + + d =xe_eudebug_get (q->vm->xef); +if (!d) { + ret = -ENOTCONN ; + goto err_exec_queue_put; + } + +if (!completion_done (&d->discovery)) { +eu_dbg (d,"discovery not yet done\n" ); + ret = -EBUSY ; + goto err_eudebug_put; + } + + ret =send_attention_event (d, q, lrc_idx); +if (ret) +xe_eudebug_disconnect (d, ret); + +err_eudebug_put: +xe_eudebug_put (d); +err_exec_queue_put: +xe_exec_queue_put (q); + +return ret; +} + +static int xe_eudebug_handle_gt_attention (struct xe_gt *gt) +{ +int ret; + + ret =xe_gt_eu_threads_needing_attention (gt); +if (ret <=0 ) +return ret; + + ret =xe_send_gt_attention (gt); + +/* Discovery in progress, fake it */ +if (ret == -EBUSY ) +return 0 ; + +return ret; +} + +#define XE_EUDEBUG_ATTENTION_INTERVAL 100 +static void attention_scan_fn (struct work_struct *work) +{ +struct xe_device *xe =container_of (work,typeof (*xe), eudebug.attention_scan.work); +long delay =msecs_to_jiffies (XE_EUDEBUG_ATTENTION_INTERVAL ); +struct xe_gt *gt; +u8 gt_id; + +if (list_empty_careful (&xe->eudebug.list)) + delay *=10 ; + +if (delay >=HZ ) + delay =round_jiffies_up_relative (delay); + +if (pm_runtime_active (xe->drm.dev)) { +for_each_gt (gt, xe, gt_id) { +int ret; + + ret =xe_eudebug_handle_gt_attention (gt); +if (ret) { +// TODO: error capture +drm_info (>_to_xe (gt)->drm, +"gt:%d unable to handle eu attention ret=%d\n" , + gt_id, ret); + +xe_gt_reset_async (gt); + } + } + } + +schedule_delayed_work (&xe->eudebug.attention_scan, delay); +} + +static void attention_scan_cancel (struct xe_device *xe) +{ +cancel_delayed_work_sync (&xe->eudebug.attention_scan); +} + +static void attention_scan_flush (struct xe_device *xe) +{ +mod_delayed_work (system_wq, &xe->eudebug.attention_scan,0 ); +} +static void discovery_work_fn (struct work_struct *work); static int@@ -877,6 +1252,7 @@ xe_eudebug_connect(struct xe_device *xe,kref_init (&d->ref);spin_lock_init (&d->connection.lock); +mutex_init (&d->eu_lock);init_waitqueue_head (&d->events.write_done);init_waitqueue_head (&d->events.read_done);init_completion (&d->discovery);@@ -903,6 +1279,7 @@ xe_eudebug_connect(struct xe_device *xe,kref_get (&d->ref);queue_work (xe->eudebug.ordered_wq, &d->discovery_work); +attention_scan_flush (xe);eu_dbg (d,"connected session %lld" , d->session);@@ -979,12 +1356,22 @@ void xe_eudebug_init(struct xe_device *xe) {spin_lock_init (&xe->eudebug.lock);INIT_LIST_HEAD (&xe->eudebug.list); +INIT_DELAYED_WORK (&xe->eudebug.attention_scan, attention_scan_fn); xe->eudebug.available = true; } +void xe_eudebug_init_late (struct xe_device *xe) +{ +if (!xe->eudebug.available) +return ; + +attention_scan_flush (xe); +} +void xe_eudebug_fini (struct xe_device *xe) { +attention_scan_cancel (xe);xe_assert (xe,list_empty_careful (&xe->eudebug.list)); }drivers/gpu/drm/xe/xe_eudebug.h +1
-0@@ -18,6 +18,7 @@ int xe_eudebug_connect_ioctl(struct drm_device *dev, struct drm_file *file);void xe_eudebug_init (struct xe_device *xe); +void xe_eudebug_init_late (struct xe_device *xe);void xe_eudebug_fini (struct xe_device *xe);void xe_eudebug_init_hw_engine (struct xe_hw_engine *hwe);drivers/gpu/drm/xe/xe_eudebug_types.h +32
-0@@ -105,6 +105,9 @@ struct xe_eudebug {/** @discovery_work: worker to discover resources for target_task */ struct work_struct discovery_work; +/** eu_lock: guards operations on eus (eu thread control and attention) */ +struct mutex eu_lock; +/** @events: kfifo queue of to-be-delivered events */ struct {/** @lock: guards access to fifo */ @@ -202,4 +205,33 @@ struct xe_eudebug_event_exec_queue { u64 lrc_handle[]; }; +/** +* struct xe_eudebug_event_eu_attention - Internal event for EU attention +*/ +struct xe_eudebug_event_eu_attention { +/** @base: base event */ +struct xe_eudebug_event base; + +/** @client_handle: client for the attention */ +u64 client_handle; + +/** @exec_queue_handle: handle of exec_queue which raised attention */ +u64 exec_queue_handle; + +/** @lrc_handle: lrc handle of the workload which raised attention */ +u64 lrc_handle; + +/** @flags: eu attention event flags, currently MBZ */ +u32 flags; + +/** @bitmask_size: size of the bitmask, specific to device */ +u32 bitmask_size; + +/** +* @bitmask: reflects threads currently signalling attention, +* starting from natural hardware order of DSS=0, eu=0 +*/ +u8 bitmask[]; +}; +#endif drivers/gpu/drm/xe/xe_gt_debug.c new file +152
-0@@ -0,0 +1,152 @@ +// SPDX-License-Identifier: MIT +/* +* Copyright © 2023 Intel Corporation +*/ + +#include "regs/xe_gt_regs.h" +#include "xe_device.h" +#include "xe_force_wake.h" +#include "xe_gt.h" +#include "xe_gt_topology.h" +#include "xe_gt_debug.h" +#include "xe_gt_mcr.h" +#include "xe_pm.h" +#include "xe_macros.h" + +static int xe_gt_foreach_dss_group_instance (struct xe_gt *gt, +int (*fn)(struct xe_gt *gt, +void *data, +u16 group, +u16 instance), +void *data) +{ +const enum xe_force_wake_domains fw_domains =XE_FW_GT |XE_FW_RENDER ; +unsigned int dss; +u16 group, instance; +int ret; + +xe_pm_runtime_get (gt_to_xe (gt)); + ret =xe_force_wake_get (gt_to_fw (gt), fw_domains); +if (ret) + goto pm_runtime_put; + +for_each_dss_steering (dss, gt, group, instance) { + ret =fn (gt, data, group, instance); +if (ret) +break ; + } + +xe_force_wake_put (gt_to_fw (gt), fw_domains); +pm_runtime_put: +xe_pm_runtime_put (gt_to_xe (gt)); + +return ret; +} + +static int read_first_attention_mcr (struct xe_gt *gt,void *data, +u16 group,u16 instance) +{ +unsigned int row; + +for (row =0 ; row <2 ; row++) { +u32 val; + + val =xe_gt_mcr_unicast_read (gt,TD_ATT (row), group, instance); + +if (val) +return 1 ; + } + +return 0 ; +} + +#define MAX_EUS_PER_ROW 4u +#define MAX_THREADS 8u + +/** +* xe_gt_eu_attention_bitmap_size - query size of the attention bitmask +* +* @gt: pointer to struct xe_gt +* +* Return: size in bytes. +*/ +int xe_gt_eu_attention_bitmap_size (struct xe_gt *gt) +{ +xe_dss_mask_t dss_mask; + +bitmap_or (dss_mask, gt->fuse_topo.c_dss_mask, + gt->fuse_topo.g_dss_mask,XE_MAX_DSS_FUSE_BITS ); + +return bitmap_weight (dss_mask,XE_MAX_DSS_FUSE_BITS ) * +TD_EU_ATTENTION_MAX_ROWS *MAX_THREADS * +MAX_EUS_PER_ROW /8 ; +} + +struct attn_read_iter { +struct xe_gt *gt; +unsigned int i; +unsigned int size; +u8 *bits; +}; + +static int read_eu_attentions_mcr (struct xe_gt *gt,void *data, +u16 group,u16 instance) +{ +struct attn_read_iter *const iter = data; +unsigned int row; + +for (row =0 ; row <TD_EU_ATTENTION_MAX_ROWS ; row++) { +u32 val; + +if (iter->i >= iter->size) +return 0 ; + +XE_WARN_ON (iter->i +sizeof (val) >xe_gt_eu_attention_bitmap_size (gt)); + + val =xe_gt_mcr_unicast_read (gt,TD_ATT (row), group, instance); + + +memcpy (&iter->bits[iter->i], &val,sizeof (val)); + iter->i +=sizeof (val); + } + +return 0 ; +} + +/** +* xe_gt_eu_attention_bitmap - query host attention +* +* @gt: pointer to struct xe_gt +* +* Return: 0 on success, negative otherwise. +*/ +int xe_gt_eu_attention_bitmap (struct xe_gt *gt,u8 *bits, +unsigned int bitmap_size) +{ +struct attn_read_iter iter = { + .gt = gt, + .i =0 , + .size = bitmap_size, + .bits = bits + }; + +return xe_gt_foreach_dss_group_instance (gt, read_eu_attentions_mcr, &iter); +} + +/** +* xe_gt_eu_threads_needing_attention - Query host attention +* +* @gt: pointer to struct xe_gt +* +* Return: 1 if threads waiting host attention, 0 otherwise. +*/ +int xe_gt_eu_threads_needing_attention (struct xe_gt *gt) +{ +int err; + + err =xe_gt_foreach_dss_group_instance (gt, read_first_attention_mcr,NULL ); + +XE_WARN_ON (err <0 ); + +return err <0 ?0 : err; +}drivers/gpu/drm/xe/xe_gt_debug.h new file +21
-0@@ -0,0 +1,21 @@ +/* SPDX-License-Identifier: MIT */ +/* +* Copyright © 2023 Intel Corporation +*/ + +#ifndef __XE_GT_DEBUG_ +#define __XE_GT_DEBUG_ + +#define TD_EU_ATTENTION_MAX_ROWS 2u + +#include "xe_gt_types.h" + +#define XE_GT_ATTENTION_TIMEOUT_MS 100 + +int xe_gt_eu_threads_needing_attention (struct xe_gt *gt); + +int xe_gt_eu_attention_bitmap_size (struct xe_gt *gt); +int xe_gt_eu_attention_bitmap (struct xe_gt *gt,u8 *bits, +unsigned int bitmap_size); + +#endif include/uapi/drm/xe_drm_eudebug.h +14
-1@@ -27,13 +27,15 @@ struct drm_xe_eudebug_event {#define DRM_XE_EUDEBUG_EVENT_OPEN 2#define DRM_XE_EUDEBUG_EVENT_VM 3#define DRM_XE_EUDEBUG_EVENT_EXEC_QUEUE 4 -#define DRM_XE_EUDEBUG_EVENT_MAX_EVENT DRM_XE_EUDEBUG_EVENT_EXEC_QUEUE +#define DRM_XE_EUDEBUG_EVENT_EU_ATTENTION 5 +#define DRM_XE_EUDEBUG_EVENT_MAX_EVENT DRM_XE_EUDEBUG_EVENT_EU_ATTENTION__u16 flags;#define DRM_XE_EUDEBUG_EVENT_CREATE (1 << 0)#define DRM_XE_EUDEBUG_EVENT_DESTROY (1 << 1)#define DRM_XE_EUDEBUG_EVENT_STATE_CHANGE (1 << 2)#define DRM_XE_EUDEBUG_EVENT_NEED_ACK (1 << 3) +__u64 seqno;__u64 reserved; };@@ -62,6 +64,17 @@ struct drm_xe_eudebug_event_exec_queue {__u64 lrc_handle[]; }; +struct drm_xe_eudebug_event_eu_attention { +struct drm_xe_eudebug_event base; + +__u64 client_handle; +__u64 exec_queue_handle; +__u64 lrc_handle; +__u32 flags; +__u32 bitmask_size; +__u8 bitmask[]; +}; +#if defined(__cplusplus) }#endif Signature · 1 line
2.34.1
Re: [PATCH 10/21] drm/xe/eudebug: Introduce per device attention scan worker
Quoted 187 lines · Matthew Brost
Quoted 160 lines · Mika Kuoppala
From: Dominik Grzegorzek <[email protected]>
Scan for EU debugging attention bits periodically to detect if some EU thread has entered the system routine (SIP) due to EU thread exception.
Make the scanning interval 10 times slower when there is no debugger connection open. Send attention event whenever we see attention with debugger presence. If there is no debugger connection active - reset.
Based on work by authors and other folks who were part of attentions in i915.
- v2 Do not validate potentially active hwe against
engine->hwe. Whenever the engine has width > 1, this field contains only the first hwe of the class.
- squash dss walking and semaphore to mutex
- v3 error path fix inxe_send_gt_attention(Christoph)
- v4 runalone active fix (Mika)
- v5q->lrcchanges (Mika)
- v6 Use C99 flexible arrays (Maciej, checkpatch)
function with 'for_each' in name (Maciej, checkpatch)
- v7 long running active fix (Dominik)
- v8 resource handling errors rebase (Mika)
- v9 find out lrc handles first before sending event (Mika)
- v10 adjust runalone shift according to hwSigned-off-by Dominik Grzegorzek <[email protected]>Signed-off-by Christoph Manszewski <[email protected]>Signed-off-by Maciej Patelczyk <[email protected]>Signed-off-by Mika Kuoppala <[email protected]>--- drivers/gpu/drm/xe/Makefile | 1 + drivers/gpu/drm/xe/regs/xe_engine_regs.h | 3 + drivers/gpu/drm/xe/regs/xe_gt_regs.h | 7 + drivers/gpu/drm/xe/xe_device.c | 2 + drivers/gpu/drm/xe/xe_device_types.h | 3 + drivers/gpu/drm/xe/xe_eudebug.c | 389 ++++++++++++++++++++++- drivers/gpu/drm/xe/xe_eudebug.h | 1 + drivers/gpu/drm/xe/xe_eudebug_types.h | 32 ++ drivers/gpu/drm/xe/xe_gt_debug.c | 152 +++++++++ drivers/gpu/drm/xe/xe_gt_debug.h | 21 ++ include/uapi/drm/xe_drm_eudebug.h | 15 +- 11 files changed, 624 insertions(+), 2 deletions(-) create mode 100644 drivers/gpu/drm/xe/xe_gt_debug.c create mode 100644 drivers/gpu/drm/xe/xe_gt_debug.hdrivers/gpu/drm/xe/Makefile +1
-0@@ -49,6 +49,7 @@ xe-y += xe_bb.o \ xe_gt_debugfs.o \ xe_gt_freq.o \ xe_gt_idle.o \ + xe_gt_debug.o \ xe_gt_mcr.o \ xe_gt_pagefault.o \ xe_gt_sysfs.o \drivers/gpu/drm/xe/regs/xe_engine_regs.h +3
-0@@ -132,6 +132,9 @@#define RING_EXECLIST_STATUS_LO (base) XE_REG((base) + 0x234)#define RING_EXECLIST_STATUS_HI (base) XE_REG((base) + 0x234 + 4) +#define RING_CURRENT_LRCA (base) XE_REG((base) + 0x240) +#define CURRENT_LRCA_VALID REG_BIT(0) +#define RING_CONTEXT_CONTROL (base) XE_REG((base) + 0x244, XE_REG_OPTION_MASKED)#define CTX_CTRL_OAC_CONTEXT_ENABLE REG_BIT(8)#define CTX_CTRL_RUN_ALONE REG_BIT(7)drivers/gpu/drm/xe/regs/xe_gt_regs.h +7
-0@@ -437,6 +437,8 @@#define DISABLE_ECC REG_BIT(5)#define ENABLE_PREFETCH_INTO_IC REG_BIT(3) +#define TD_ATT (x) XE_REG_MCR(0xe470 + (x) * 4) +#define ROW_CHICKEN4 XE_REG_MCR(0xe48c, XE_REG_OPTION_MASKED)#define DISABLE_GRF_CLEAR REG_BIT(13)#define XEHP_DIS_BBL_SYSPIPE REG_BIT(11)@@ -516,6 +518,11 @@#define CCS_MODE_CSLICE (cslice, ccs) \ ((ccs) << ((cslice) * CCS_MODE_CSLICE_WIDTH)) +#define RCU_DEBUG_1 XE_REG(0x14a00) +#define RCU_DEBUG_1_ENGINE_STATUS REG_GENMASK(2, 0) +#define RCU_DEBUG_1_RUNALONE_ACTIVE REG_BIT(2) +#define RCU_DEBUG_1_CONTEXT_ACTIVE REG_BIT(0) +#define FORCEWAKE_ACK_GT XE_REG(0x130044)/* Applicable for all FORCEWAKE_DOMAIN and FORCEWAKE_ACK_DOMAIN regs */ drivers/gpu/drm/xe/xe_device.c +2
-0@@ -768,6 +768,8 @@ int xe_device_probe(struct xe_device *xe)xe_debugfs_register (xe); +xe_eudebug_init_late (xe); +xe_hwmon_register (xe);for_each_gt (gt, xe, id)drivers/gpu/drm/xe/xe_device_types.h +3
-0@@ -516,6 +516,9 @@ struct xe_device {/** @ordered_wq: used to discovery */ struct workqueue_struct *ordered_wq; + +/** @attention_scan: attention scan worker */ +struct delayed_work attention_scan; } eudebug;/* private: */ drivers/gpu/drm/xe/xe_eudebug.c +11
-1@@ -11,19 +11,29 @@#include <drm/drm_managed.h> -#include "regs/xe_gt_regs.h" #include "regs/xe_engine_regs.h" +#include "regs/xe_gt_regs.h" #include "xe_device.h" #include "xe_assert.h" #include "xe_macros.h" #include "xe_gt.h" +#include "xe_gt_debug.h" +#include "xe_lrc.h" +#include "xe_hw_engine.h" +#include "xe_exec_queue.h" #include "xe_eudebug_types.h" #include "xe_eudebug.h" #include "xe_exec_queue_types.h" +#include "xe_guc_exec_queue_types.h" +#include "xe_execlist_types.h" +#include "xe_mmio.h" #include "xe_module.h" +#include "xe_pm.h" #include "xe_rtp.h" +#include "xe_sched_job.h" #include "xe_vm.h" #include "xe_wa.h" +#include "xe_force_wake.h" /* * If there is no detected event read by userspace, during this period, assume drivers/gpu/drm/xe/xe_eudebug.c static const struct file_operations fops = { · line 856
Quoted 4 lines
drivers/gpu/drm/xe/xe_eudebug.c +1
-0@@ -843,6 +853,371 @@ static const struct file_operations fops = { .unlocked_ioctl = xe_eudebug_ioctl, }; +static bool queue_has_active_job (struct xe_exec_queue *q)This should probably be a generic function in
xe_exec_queue.c. We seemly already have one -xe_exec_queue_is_idle.drivers/gpu/drm/xe/xe_eudebug.c static const struct file_operations fops = { · line 862
Quoted 6 lines
drivers/gpu/drm/xe/xe_eudebug.c +6
-0+{ + +struct drm_gpu_scheduler *sched; +struct drm_sched_job *drm_job; + +if (xe_device_uc_enabled (gt_to_xe (q->gt)))General comment, we likely don't want to the EU debugger enabled unless GuC submission is enabled as the execlist backend really doesn't work.
drivers/gpu/drm/xe/xe_eudebug.c static const struct file_operations fops = { · line 867
Quoted 5 lines
drivers/gpu/drm/xe/xe_eudebug.c +5
-0+ sched = &q->guc->sched.base; + else + sched = &q->execlist->sched; + + drm_job = list_first_entry_or_null(&sched->pending_list, struct drm_sched_job, list);Random musing that don't apply as we have
xe_exec_queue_is_idlebut...
Quoted 6 lines
You need a scheduler lock here which is missing. If you wanted to see the scheduler pending list was not empty, we'd call into the drm scheduler layer.
That being said, I think the EU debugger only support LR VMs? If so, then this always going to be empty.
Up until now, we were not requiring a VM to be in lr mode, thus I wanted to support both paths. However, we do depend on LR mode, as job timeouts would make debugging effectively ineffective. Therefore, we should add this constraint.
Thanks for your comments! I will reuse xe_exec_queue_is_idle here as proposed, and I will try to follow your hints with respect to locking and forcewake.
Regards,
Dominik
Quoted 722 lines
Quoted 160 lines
drivers/gpu/drm/xe/xe_eudebug.c +159
-0+ +if (drm_job) { +struct xe_sched_job *job =to_xe_sched_job (drm_job); + +return xe_sched_job_started (job) && !xe_sched_job_completed (job); + }else if (xe_exec_queue_is_lr (q) && + (xe_lrc_ring_head (q->lrc[0 ]) !=xe_lrc_ring_tail (q->lrc[0 ]))) { +return true; + } + +return false; +} + +static int current_lrc (struct xe_hw_engine *hwe,u32 *lrc_hw) +{ +u32 lrc_reg; +int err; + + err =xe_force_wake_get (gt_to_fw (hwe->gt), hwe->domain); +if (err) +return err; + + lrc_reg =hw_engine_mmio_read32 (hwe,RING_CURRENT_LRCA (0 )); + +xe_force_wake_put (gt_to_fw (hwe->gt), hwe->domain); + +if (!(lrc_reg &CURRENT_LRCA_VALID )) +return -ENOENT ; + + *lrc_hw = lrc_reg &GENMASK (31 ,12 ); + +return 0 ; +} + +static int match_engine_lrc (struct xe_exec_queue *q,u32 lrc_hw) +{ +int i; +u32 lrc_ggtt; + +for (i =0 ; i < q->width; i++) { + lrc_ggtt =lower_32_bits (xe_lrc_descriptor (q->lrc[i])); + lrc_ggtt &=GENMASK (31 ,12 ); +if (lrc_ggtt == lrc_hw) +return i; + } + +return -1 ; +} + +static u32 engine_status (const struct xe_hw_engine *const hwe, +u32 rcu_debug1) +{ +const bool xe1 =GRAPHICS_VER (gt_to_xe (hwe->gt)) <20 ; +unsigned int shift; + +if (hwe->class ==XE_ENGINE_CLASS_RENDER ) { + shift =7 ; +XE_WARN_ON (hwe->instance !=0 ); + }else if (hwe->class ==XE_ENGINE_CLASS_COMPUTE ) { +XE_WARN_ON (hwe->instance >3 ); + +if (xe1) + shift =10 + (hwe->instance *3 ); +else + shift =11 + (hwe->instance *4 ); + }else { +XE_WARN_ON (hwe->class); +return 0 ; + } + +return (rcu_debug1 >> shift) &RCU_DEBUG_1_ENGINE_STATUS ; +} + +static bool engine_runalone_set (const struct xe_hw_engine *const hwe, +u32 rcu_debug1) +{ +return engine_status (hwe, rcu_debug1) &RCU_DEBUG_1_RUNALONE_ACTIVE ; +} + +static bool engine_context_set (const struct xe_hw_engine *const hwe, +u32 rcu_debug1) +{ +return engine_status (hwe, rcu_debug1) &RCU_DEBUG_1_CONTEXT_ACTIVE ; +} + +static bool engine_has_runalone (const struct xe_hw_engine *const hwe) +{ +return hwe->class ==XE_ENGINE_CLASS_RENDER || + hwe->class ==XE_ENGINE_CLASS_COMPUTE ; +} + +static struct xe_hw_engine *get_runalone_active_hw_engine (struct xe_gt *gt) +{ +struct xe_hw_engine *hwe, *first =NULL ; +unsigned int num_active, id; +u32 val; + +if (xe_force_wake_get (gt_to_fw (gt),XE_FW_GT )) { +drm_dbg (>_to_xe (gt)->drm,"eudbg: runalone failed to get force wake\n" ); +return NULL ; + } + + val =xe_mmio_read32 (gt,RCU_DEBUG_1 ); +xe_force_wake_put (gt_to_fw (gt),XE_FW_GT ); + +drm_dbg (>_to_xe (gt)->drm,"eudbg: runalone RCU_DEBUG_1 = 0x%08x\n" , val); + + num_active =0 ; +for_each_hw_engine (hwe, gt, id) { +bool runalone, ctx; + +if (!engine_has_runalone (hwe)) +continue ; + + runalone =engine_runalone_set (hwe, val); + ctx =engine_context_set (hwe, val); + +drm_dbg (>_to_xe (gt)->drm,"eudbg: engine %s: runalone=%s, context=%s" , + hwe->name, runalone ?"active" :"inactive" , + ctx ?"active" :"inactive" ); + +/* +* On earlier gen12 the context status seems to be idle when +* it has raised attention. We have to omit the active bit. +*/ +if (IS_DGFX (gt_to_xe (gt))) + ctx = true; + +if (runalone && ctx) { + num_active++; + +drm_dbg (>_to_xe (gt)->drm,"eudbg: runalone engine %s %s" , + hwe->name, first ?"selected" :"found" ); +if (!first) + first = hwe; + } + } + +if (num_active >1 ) +drm_err (>_to_xe (gt)->drm,"eudbg: %d runalone engines active!" , + num_active); + +return first; +} + +static struct xe_exec_queue *runalone_active_queue_get (struct xe_gt *gt,int *lrc_idx) +{ +struct xe_device *xe =gt_to_xe (gt); +struct xe_exec_queue *q, *found =NULL ; +struct xe_hw_engine *active; +struct xe_file *xef, *tmp; +unsigned long i; +int idx, err; +u32 lrc_hw; + + active =get_runalone_active_hw_engine (gt); +if (!active) { +drm_dbg (>_to_xe (gt)->drm,"Runalone engine not found!" ); +return ERR_PTR (-ENOENT );drivers/gpu/drm/xe/xe_eudebug.c static const struct file_operations fops = { · line 1036
Quoted 10 lines
drivers/gpu/drm/xe/xe_eudebug.c +10
-0+ } + + err =current_lrc (active, &lrc_hw); +if (err) +return ERR_PTR (err); + +mutex_lock (&xe->files.lock); +list_for_each_entry_safe (xef, tmp, &xe->files.list, link) { +mutex_lock (&xef->exec_queue.lock); +xa_for_each (&xef->exec_queue.xa, i, q) {Same comment here as [1]. Don't hold
xe->files.lockorxef->exec_queue.lockfor anything but the lookup. Holding locks longer than needed creates unwanted lock dep chains which creates all sorts of problems. The only time chains should exist is when we have well defined locking chain like we have for a VM (vm->lock-> dma-resv -> notifier lock).So this should look something like this:
mutex_lock(&xe->files.lock);xa_for_each(.., xef) { xe_file_get(xef); mutex_unlock(&xe->files.lock); mutex_lock(&xef->exec_queue.lock); xa_for_each(&xef->exec_queue.xa, i, q) { xe_exec_queue_get(q); mutex_unlock(&xef->exec_queue.lock); /* Do something */ xe_exec_queue_put(q); mutex_lock(&xef->exec_queue.lock); } mutex_unlock(&xef->exec_queue.lock); xe_file_put(xef); mutex_lock(&xe->files.lock); }
mutex_unlock(&xe->files.lock);Note you will need to change
xe->files.listto an xarray to make this safe too.Matt
[1] https://patchwork.freedesktop.org/patch/606052/?series=136572&rev=1
Quoted 513 lines
drivers/gpu/drm/xe/xe_eudebug.c +196
-0+if (q->gt != gt) +continue ; + +if (q->class != active->class) +continue ; + +if (!queue_has_active_job (q)) +continue ; + + idx =match_engine_lrc (q, lrc_hw); +if (idx <0 ) +continue ; + +xe_exec_queue_get (q); + found = q; + +if (lrc_idx) + *lrc_idx = idx; + +break ; + } +mutex_unlock (&xef->exec_queue.lock); + +if (found) +break ; + } +mutex_unlock (&xe->files.lock); + +if (!found) +return ERR_PTR (-ENOENT ); + +if (XE_WARN_ON (current_lrc (active, &lrc_hw)) && +XE_WARN_ON (match_engine_lrc (found, lrc_hw) <0 )) { +xe_exec_queue_put (found); +return ERR_PTR (-ENOENT ); + } + +return found; +} + +static int send_attention_event (struct xe_eudebug *d,struct xe_exec_queue *q,int lrc_idx) +{ +struct xe_eudebug_event_eu_attention *ea; +struct xe_eudebug_event *event; +int h_c, h_queue, h_lrc; +u32 size =xe_gt_eu_attention_bitmap_size (q->gt); +u32 sz =struct_size (ea, bitmask, size); +int ret; + +XE_WARN_ON (lrc_idx <0 || lrc_idx >= q->width); + + h_c =find_handle (d->res,XE_EUDEBUG_RES_TYPE_CLIENT , q->vm->xef); +if (h_c <0 ) +return h_c; + + h_queue =find_handle (d->res,XE_EUDEBUG_RES_TYPE_EXEC_QUEUE , q); +if (h_queue <0 ) +return h_queue; + + h_lrc =find_handle (d->res,XE_EUDEBUG_RES_TYPE_LRC , q->lrc[lrc_idx]); +if (h_lrc <0 ) +return h_lrc; + + event =__xe_eudebug_create_event (d,0 ,DRM_XE_EUDEBUG_EVENT_EU_ATTENTION , +DRM_XE_EUDEBUG_EVENT_STATE_CHANGE , sz,GFP_KERNEL ); + +if (!event) +return -ENOSPC ; + + ea =cast_event (ea, event); + write_member(struct drm_xe_eudebug_event_eu_attention, ea, client_handle, (u64)h_c); + write_member(struct drm_xe_eudebug_event_eu_attention, ea, exec_queue_handle, (u64)h_queue); + write_member(struct drm_xe_eudebug_event_eu_attention, ea, lrc_handle, (u64)h_lrc); + write_member(struct drm_xe_eudebug_event_eu_attention, ea, bitmask_size, size); + +mutex_lock (&d->eu_lock); + event->seqno =atomic_long_inc_return (&d->events.seqno); + ret =xe_gt_eu_attention_bitmap (q->gt, &ea->bitmask[0 ], ea->bitmask_size); +mutex_unlock (&d->eu_lock); + +if (ret) +return ret; + +return xe_eudebug_queue_event (d, event); +} + + +static int xe_send_gt_attention (struct xe_gt *gt) +{ +struct xe_eudebug *d; +struct xe_exec_queue *q; +int ret, lrc_idx; + +if (list_empty_careful (>_to_xe (gt)->eudebug.list)) +return -ENOTCONN ; + + q =runalone_active_queue_get (gt, &lrc_idx); +if (IS_ERR (q)) +return PTR_ERR (q); + + d =xe_eudebug_get (q->vm->xef); +if (!d) { + ret = -ENOTCONN ; + goto err_exec_queue_put; + } + +if (!completion_done (&d->discovery)) { +eu_dbg (d,"discovery not yet done\n" ); + ret = -EBUSY ; + goto err_eudebug_put; + } + + ret =send_attention_event (d, q, lrc_idx); +if (ret) +xe_eudebug_disconnect (d, ret); + +err_eudebug_put: +xe_eudebug_put (d); +err_exec_queue_put: +xe_exec_queue_put (q); + +return ret; +} + +static int xe_eudebug_handle_gt_attention (struct xe_gt *gt) +{ +int ret; + + ret =xe_gt_eu_threads_needing_attention (gt); +if (ret <=0 ) +return ret; + + ret =xe_send_gt_attention (gt); + +/* Discovery in progress, fake it */ +if (ret == -EBUSY ) +return 0 ; + +return ret; +} + +#define XE_EUDEBUG_ATTENTION_INTERVAL 100 +static void attention_scan_fn (struct work_struct *work) +{ +struct xe_device *xe =container_of (work,typeof (*xe), eudebug.attention_scan.work); +long delay =msecs_to_jiffies (XE_EUDEBUG_ATTENTION_INTERVAL ); +struct xe_gt *gt; +u8 gt_id; + +if (list_empty_careful (&xe->eudebug.list)) + delay *=10 ; + +if (delay >=HZ ) + delay =round_jiffies_up_relative (delay); + +if (pm_runtime_active (xe->drm.dev)) { +for_each_gt (gt, xe, gt_id) { +int ret; + + ret =xe_eudebug_handle_gt_attention (gt); +if (ret) { +// TODO: error capture +drm_info (>_to_xe (gt)->drm, +"gt:%d unable to handle eu attention ret=%d\n" , + gt_id, ret); + +xe_gt_reset_async (gt); + } + } + } + +schedule_delayed_work (&xe->eudebug.attention_scan, delay); +} + +static void attention_scan_cancel (struct xe_device *xe) +{ +cancel_delayed_work_sync (&xe->eudebug.attention_scan); +} + +static void attention_scan_flush (struct xe_device *xe) +{ +mod_delayed_work (system_wq, &xe->eudebug.attention_scan,0 ); +} +static void discovery_work_fn (struct work_struct *work); static int@@ -877,6 +1252,7 @@ xe_eudebug_connect(struct xe_device *xe,kref_init (&d->ref);spin_lock_init (&d->connection.lock); +mutex_init (&d->eu_lock);init_waitqueue_head (&d->events.write_done);init_waitqueue_head (&d->events.read_done);init_completion (&d->discovery);@@ -903,6 +1279,7 @@ xe_eudebug_connect(struct xe_device *xe,kref_get (&d->ref);queue_work (xe->eudebug.ordered_wq, &d->discovery_work); +attention_scan_flush (xe);eu_dbg (d,"connected session %lld" , d->session);@@ -979,12 +1356,22 @@ void xe_eudebug_init(struct xe_device *xe) {spin_lock_init (&xe->eudebug.lock);INIT_LIST_HEAD (&xe->eudebug.list); +INIT_DELAYED_WORK (&xe->eudebug.attention_scan, attention_scan_fn); xe->eudebug.available = true; } +void xe_eudebug_init_late (struct xe_device *xe) +{ +if (!xe->eudebug.available) +return ; + +attention_scan_flush (xe); +} +void xe_eudebug_fini (struct xe_device *xe) { +attention_scan_cancel (xe);xe_assert (xe,list_empty_careful (&xe->eudebug.list)); }drivers/gpu/drm/xe/xe_eudebug.h +1
-0@@ -18,6 +18,7 @@ int xe_eudebug_connect_ioctl(struct drm_device *dev, struct drm_file *file);void xe_eudebug_init (struct xe_device *xe); +void xe_eudebug_init_late (struct xe_device *xe);void xe_eudebug_fini (struct xe_device *xe);void xe_eudebug_init_hw_engine (struct xe_hw_engine *hwe);drivers/gpu/drm/xe/xe_eudebug_types.h +32
-0@@ -105,6 +105,9 @@ struct xe_eudebug {/** @discovery_work: worker to discover resources for target_task */ struct work_struct discovery_work; +/** eu_lock: guards operations on eus (eu thread control and attention) */ +struct mutex eu_lock; +/** @events: kfifo queue of to-be-delivered events */ struct {/** @lock: guards access to fifo */ @@ -202,4 +205,33 @@ struct xe_eudebug_event_exec_queue { u64 lrc_handle[]; }; +/** +* struct xe_eudebug_event_eu_attention - Internal event for EU attention +*/ +struct xe_eudebug_event_eu_attention { +/** @base: base event */ +struct xe_eudebug_event base; + +/** @client_handle: client for the attention */ +u64 client_handle; + +/** @exec_queue_handle: handle of exec_queue which raised attention */ +u64 exec_queue_handle; + +/** @lrc_handle: lrc handle of the workload which raised attention */ +u64 lrc_handle; + +/** @flags: eu attention event flags, currently MBZ */ +u32 flags; + +/** @bitmask_size: size of the bitmask, specific to device */ +u32 bitmask_size; + +/** +* @bitmask: reflects threads currently signalling attention, +* starting from natural hardware order of DSS=0, eu=0 +*/ +u8 bitmask[]; +}; +#endif drivers/gpu/drm/xe/xe_gt_debug.c new file +152
-0@@ -0,0 +1,152 @@ +// SPDX-License-Identifier: MIT +/* +* Copyright © 2023 Intel Corporation +*/ + +#include "regs/xe_gt_regs.h" +#include "xe_device.h" +#include "xe_force_wake.h" +#include "xe_gt.h" +#include "xe_gt_topology.h" +#include "xe_gt_debug.h" +#include "xe_gt_mcr.h" +#include "xe_pm.h" +#include "xe_macros.h" + +static int xe_gt_foreach_dss_group_instance (struct xe_gt *gt, +int (*fn)(struct xe_gt *gt, +void *data, +u16 group, +u16 instance), +void *data) +{ +const enum xe_force_wake_domains fw_domains =XE_FW_GT |XE_FW_RENDER ; +unsigned int dss; +u16 group, instance; +int ret; + +xe_pm_runtime_get (gt_to_xe (gt)); + ret =xe_force_wake_get (gt_to_fw (gt), fw_domains); +if (ret) + goto pm_runtime_put; + +for_each_dss_steering (dss, gt, group, instance) { + ret =fn (gt, data, group, instance); +if (ret) +break ; + } + +xe_force_wake_put (gt_to_fw (gt), fw_domains); +pm_runtime_put: +xe_pm_runtime_put (gt_to_xe (gt)); + +return ret; +} + +static int read_first_attention_mcr (struct xe_gt *gt,void *data, +u16 group,u16 instance) +{ +unsigned int row; + +for (row =0 ; row <2 ; row++) { +u32 val; + + val =xe_gt_mcr_unicast_read (gt,TD_ATT (row), group, instance); + +if (val) +return 1 ; + } + +return 0 ; +} + +#define MAX_EUS_PER_ROW 4u +#define MAX_THREADS 8u + +/** +* xe_gt_eu_attention_bitmap_size - query size of the attention bitmask +* +* @gt: pointer to struct xe_gt +* +* Return: size in bytes. +*/ +int xe_gt_eu_attention_bitmap_size (struct xe_gt *gt) +{ +xe_dss_mask_t dss_mask; + +bitmap_or (dss_mask, gt->fuse_topo.c_dss_mask, + gt->fuse_topo.g_dss_mask,XE_MAX_DSS_FUSE_BITS ); + +return bitmap_weight (dss_mask,XE_MAX_DSS_FUSE_BITS ) * +TD_EU_ATTENTION_MAX_ROWS *MAX_THREADS * +MAX_EUS_PER_ROW /8 ; +} + +struct attn_read_iter { +struct xe_gt *gt; +unsigned int i; +unsigned int size; +u8 *bits; +}; + +static int read_eu_attentions_mcr (struct xe_gt *gt,void *data, +u16 group,u16 instance) +{ +struct attn_read_iter *const iter = data; +unsigned int row; + +for (row =0 ; row <TD_EU_ATTENTION_MAX_ROWS ; row++) { +u32 val; + +if (iter->i >= iter->size) +return 0 ; + +XE_WARN_ON (iter->i +sizeof (val) >xe_gt_eu_attention_bitmap_size (gt)); + + val =xe_gt_mcr_unicast_read (gt,TD_ATT (row), group, instance); + + +memcpy (&iter->bits[iter->i], &val,sizeof (val)); + iter->i +=sizeof (val); + } + +return 0 ; +} + +/** +* xe_gt_eu_attention_bitmap - query host attention +* +* @gt: pointer to struct xe_gt +* +* Return: 0 on success, negative otherwise. +*/ +int xe_gt_eu_attention_bitmap (struct xe_gt *gt,u8 *bits, +unsigned int bitmap_size) +{ +struct attn_read_iter iter = { + .gt = gt, + .i =0 , + .size = bitmap_size, + .bits = bits + }; + +return xe_gt_foreach_dss_group_instance (gt, read_eu_attentions_mcr, &iter); +} + +/** +* xe_gt_eu_threads_needing_attention - Query host attention +* +* @gt: pointer to struct xe_gt +* +* Return: 1 if threads waiting host attention, 0 otherwise. +*/ +int xe_gt_eu_threads_needing_attention (struct xe_gt *gt) +{ +int err; + + err =xe_gt_foreach_dss_group_instance (gt, read_first_attention_mcr,NULL ); + +XE_WARN_ON (err <0 ); + +return err <0 ?0 : err; +}drivers/gpu/drm/xe/xe_gt_debug.h new file +21
-0@@ -0,0 +1,21 @@ +/* SPDX-License-Identifier: MIT */ +/* +* Copyright © 2023 Intel Corporation +*/ + +#ifndef __XE_GT_DEBUG_ +#define __XE_GT_DEBUG_ + +#define TD_EU_ATTENTION_MAX_ROWS 2u + +#include "xe_gt_types.h" + +#define XE_GT_ATTENTION_TIMEOUT_MS 100 + +int xe_gt_eu_threads_needing_attention (struct xe_gt *gt); + +int xe_gt_eu_attention_bitmap_size (struct xe_gt *gt); +int xe_gt_eu_attention_bitmap (struct xe_gt *gt,u8 *bits, +unsigned int bitmap_size); + +#endif include/uapi/drm/xe_drm_eudebug.h +14
-1@@ -27,13 +27,15 @@ struct drm_xe_eudebug_event {#define DRM_XE_EUDEBUG_EVENT_OPEN 2#define DRM_XE_EUDEBUG_EVENT_VM 3#define DRM_XE_EUDEBUG_EVENT_EXEC_QUEUE 4 -#define DRM_XE_EUDEBUG_EVENT_MAX_EVENT DRM_XE_EUDEBUG_EVENT_EXEC_QUEUE +#define DRM_XE_EUDEBUG_EVENT_EU_ATTENTION 5 +#define DRM_XE_EUDEBUG_EVENT_MAX_EVENT DRM_XE_EUDEBUG_EVENT_EU_ATTENTION__u16 flags;#define DRM_XE_EUDEBUG_EVENT_CREATE (1 << 0)#define DRM_XE_EUDEBUG_EVENT_DESTROY (1 << 1)#define DRM_XE_EUDEBUG_EVENT_STATE_CHANGE (1 << 2)#define DRM_XE_EUDEBUG_EVENT_NEED_ACK (1 << 3) +__u64 seqno;__u64 reserved; };@@ -62,6 +64,17 @@ struct drm_xe_eudebug_event_exec_queue {__u64 lrc_handle[]; }; +struct drm_xe_eudebug_event_eu_attention { +struct drm_xe_eudebug_event base; + +__u64 client_handle; +__u64 exec_queue_handle; +__u64 lrc_handle; +__u32 flags; +__u32 bitmask_size; +__u8 bitmask[]; +}; +#if defined(__cplusplus) }#endif Signature · 1 line
2.34.1
Matthew Brost
maintainerRe: [PATCH 10/21] drm/xe/eudebug: Introduce per device attention scan worker
Quoted 192 lines · Dominik
Quoted 191 lines · Matthew Brost
Quoted 160 lines · Mika Kuoppala
From: Dominik Grzegorzek <[email protected]>
Scan for EU debugging attention bits periodically to detect if some EU thread has entered the system routine (SIP) due to EU thread exception.
Make the scanning interval 10 times slower when there is no debugger connection open. Send attention event whenever we see attention with debugger presence. If there is no debugger connection active - reset.
Based on work by authors and other folks who were part of attentions in i915.
- v2 Do not validate potentially active hwe against
engine->hwe. Whenever the engine has width > 1, this field contains only the first hwe of the class.
- squash dss walking and semaphore to mutex
- v3 error path fix inxe_send_gt_attention(Christoph)
- v4 runalone active fix (Mika)
- v5q->lrcchanges (Mika)
- v6 Use C99 flexible arrays (Maciej, checkpatch)
function with 'for_each' in name (Maciej, checkpatch)
- v7 long running active fix (Dominik)
- v8 resource handling errors rebase (Mika)
- v9 find out lrc handles first before sending event (Mika)
- v10 adjust runalone shift according to hwSigned-off-by Dominik Grzegorzek <[email protected]>Signed-off-by Christoph Manszewski <[email protected]>Signed-off-by Maciej Patelczyk <[email protected]>Signed-off-by Mika Kuoppala <[email protected]>--- drivers/gpu/drm/xe/Makefile | 1 + drivers/gpu/drm/xe/regs/xe_engine_regs.h | 3 + drivers/gpu/drm/xe/regs/xe_gt_regs.h | 7 + drivers/gpu/drm/xe/xe_device.c | 2 + drivers/gpu/drm/xe/xe_device_types.h | 3 + drivers/gpu/drm/xe/xe_eudebug.c | 389 ++++++++++++++++++++++- drivers/gpu/drm/xe/xe_eudebug.h | 1 + drivers/gpu/drm/xe/xe_eudebug_types.h | 32 ++ drivers/gpu/drm/xe/xe_gt_debug.c | 152 +++++++++ drivers/gpu/drm/xe/xe_gt_debug.h | 21 ++ include/uapi/drm/xe_drm_eudebug.h | 15 +- 11 files changed, 624 insertions(+), 2 deletions(-) create mode 100644 drivers/gpu/drm/xe/xe_gt_debug.c create mode 100644 drivers/gpu/drm/xe/xe_gt_debug.hdrivers/gpu/drm/xe/Makefile +1
-0@@ -49,6 +49,7 @@ xe-y += xe_bb.o \ xe_gt_debugfs.o \ xe_gt_freq.o \ xe_gt_idle.o \ + xe_gt_debug.o \ xe_gt_mcr.o \ xe_gt_pagefault.o \ xe_gt_sysfs.o \drivers/gpu/drm/xe/regs/xe_engine_regs.h +3
-0@@ -132,6 +132,9 @@#define RING_EXECLIST_STATUS_LO (base) XE_REG((base) + 0x234)#define RING_EXECLIST_STATUS_HI (base) XE_REG((base) + 0x234 + 4) +#define RING_CURRENT_LRCA (base) XE_REG((base) + 0x240) +#define CURRENT_LRCA_VALID REG_BIT(0) +#define RING_CONTEXT_CONTROL (base) XE_REG((base) + 0x244, XE_REG_OPTION_MASKED)#define CTX_CTRL_OAC_CONTEXT_ENABLE REG_BIT(8)#define CTX_CTRL_RUN_ALONE REG_BIT(7)drivers/gpu/drm/xe/regs/xe_gt_regs.h +7
-0@@ -437,6 +437,8 @@#define DISABLE_ECC REG_BIT(5)#define ENABLE_PREFETCH_INTO_IC REG_BIT(3) +#define TD_ATT (x) XE_REG_MCR(0xe470 + (x) * 4) +#define ROW_CHICKEN4 XE_REG_MCR(0xe48c, XE_REG_OPTION_MASKED)#define DISABLE_GRF_CLEAR REG_BIT(13)#define XEHP_DIS_BBL_SYSPIPE REG_BIT(11)@@ -516,6 +518,11 @@#define CCS_MODE_CSLICE (cslice, ccs) \ ((ccs) << ((cslice) * CCS_MODE_CSLICE_WIDTH)) +#define RCU_DEBUG_1 XE_REG(0x14a00) +#define RCU_DEBUG_1_ENGINE_STATUS REG_GENMASK(2, 0) +#define RCU_DEBUG_1_RUNALONE_ACTIVE REG_BIT(2) +#define RCU_DEBUG_1_CONTEXT_ACTIVE REG_BIT(0) +#define FORCEWAKE_ACK_GT XE_REG(0x130044)/* Applicable for all FORCEWAKE_DOMAIN and FORCEWAKE_ACK_DOMAIN regs */ drivers/gpu/drm/xe/xe_device.c +2
-0@@ -768,6 +768,8 @@ int xe_device_probe(struct xe_device *xe)xe_debugfs_register (xe); +xe_eudebug_init_late (xe); +xe_hwmon_register (xe);for_each_gt (gt, xe, id)drivers/gpu/drm/xe/xe_device_types.h +3
-0@@ -516,6 +516,9 @@ struct xe_device {/** @ordered_wq: used to discovery */ struct workqueue_struct *ordered_wq; + +/** @attention_scan: attention scan worker */ +struct delayed_work attention_scan; } eudebug;/* private: */ drivers/gpu/drm/xe/xe_eudebug.c +11
-1@@ -11,19 +11,29 @@#include <drm/drm_managed.h> -#include "regs/xe_gt_regs.h" #include "regs/xe_engine_regs.h" +#include "regs/xe_gt_regs.h" #include "xe_device.h" #include "xe_assert.h" #include "xe_macros.h" #include "xe_gt.h" +#include "xe_gt_debug.h" +#include "xe_lrc.h" +#include "xe_hw_engine.h" +#include "xe_exec_queue.h" #include "xe_eudebug_types.h" #include "xe_eudebug.h" #include "xe_exec_queue_types.h" +#include "xe_guc_exec_queue_types.h" +#include "xe_execlist_types.h" +#include "xe_mmio.h" #include "xe_module.h" +#include "xe_pm.h" #include "xe_rtp.h" +#include "xe_sched_job.h" #include "xe_vm.h" #include "xe_wa.h" +#include "xe_force_wake.h" /* * If there is no detected event read by userspace, during this period, assume drivers/gpu/drm/xe/xe_eudebug.c static const struct file_operations fops = { · line 856
Quoted 4 lines
drivers/gpu/drm/xe/xe_eudebug.c +1
-0@@ -843,6 +853,371 @@ static const struct file_operations fops = { .unlocked_ioctl = xe_eudebug_ioctl, }; +static bool queue_has_active_job (struct xe_exec_queue *q)This should probably be a generic function in
xe_exec_queue.c. We seemly already have one -xe_exec_queue_is_idle.drivers/gpu/drm/xe/xe_eudebug.c static const struct file_operations fops = { · line 862
Quoted 6 lines
drivers/gpu/drm/xe/xe_eudebug.c +6
-0+{ + +struct drm_gpu_scheduler *sched; +struct drm_sched_job *drm_job; + +if (xe_device_uc_enabled (gt_to_xe (q->gt)))General comment, we likely don't want to the EU debugger enabled unless GuC submission is enabled as the execlist backend really doesn't work.
drivers/gpu/drm/xe/xe_eudebug.c static const struct file_operations fops = { · line 867
Quoted 5 lines
drivers/gpu/drm/xe/xe_eudebug.c +5
-0+ sched = &q->guc->sched.base; + else + sched = &q->execlist->sched; + + drm_job = list_first_entry_or_null(&sched->pending_list, struct drm_sched_job, list);Random musing that don't apply as we have
xe_exec_queue_is_idlebut...You need a scheduler lock here which is missing. If you wanted to see the scheduler pending list was not empty, we'd call into the drm scheduler layer.
Quoted 6 lines
Quoted 2 lines
That being said, I think the EU debugger only support LR VMs? If so, then this always going to be empty.
Up until now, we were not requiring a VM to be in lr mode, thus I wanted to support both paths. However, we do depend on LR mode, as job timeouts would make debugging effectively ineffective. Therefore, we should add this constraint.
It would great if we could enable EU debug for non-LR VM's but we'd have to throw dma-fencing rules out the window then (e.g. disable job timeouts) and I don't think that is likely to get accepted. I think we'd also have to intercept some dma-fences from signaling too which this series doesn't do.
Quoted 1 line
Thanks for your comments! I will reuse
xe_exec_queue_is_idlehere as proposed, and I will try to
+1
Quoted 2 lines
follow your hints with respect to locking and forcewake.
s/forcewake/pm, taking a forcewake ref where ever is safe while taking a pm ref is not.
Matt
Quoted 725 lines
Regards,
DominikQuoted 722 lines
Quoted 160 lines
drivers/gpu/drm/xe/xe_eudebug.c +159
-0+ +if (drm_job) { +struct xe_sched_job *job =to_xe_sched_job (drm_job); + +return xe_sched_job_started (job) && !xe_sched_job_completed (job); + }else if (xe_exec_queue_is_lr (q) && + (xe_lrc_ring_head (q->lrc[0 ]) !=xe_lrc_ring_tail (q->lrc[0 ]))) { +return true; + } + +return false; +} + +static int current_lrc (struct xe_hw_engine *hwe,u32 *lrc_hw) +{ +u32 lrc_reg; +int err; + + err =xe_force_wake_get (gt_to_fw (hwe->gt), hwe->domain); +if (err) +return err; + + lrc_reg =hw_engine_mmio_read32 (hwe,RING_CURRENT_LRCA (0 )); + +xe_force_wake_put (gt_to_fw (hwe->gt), hwe->domain); + +if (!(lrc_reg &CURRENT_LRCA_VALID )) +return -ENOENT ; + + *lrc_hw = lrc_reg &GENMASK (31 ,12 ); + +return 0 ; +} + +static int match_engine_lrc (struct xe_exec_queue *q,u32 lrc_hw) +{ +int i; +u32 lrc_ggtt; + +for (i =0 ; i < q->width; i++) { + lrc_ggtt =lower_32_bits (xe_lrc_descriptor (q->lrc[i])); + lrc_ggtt &=GENMASK (31 ,12 ); +if (lrc_ggtt == lrc_hw) +return i; + } + +return -1 ; +} + +static u32 engine_status (const struct xe_hw_engine *const hwe, +u32 rcu_debug1) +{ +const bool xe1 =GRAPHICS_VER (gt_to_xe (hwe->gt)) <20 ; +unsigned int shift; + +if (hwe->class ==XE_ENGINE_CLASS_RENDER ) { + shift =7 ; +XE_WARN_ON (hwe->instance !=0 ); + }else if (hwe->class ==XE_ENGINE_CLASS_COMPUTE ) { +XE_WARN_ON (hwe->instance >3 ); + +if (xe1) + shift =10 + (hwe->instance *3 ); +else + shift =11 + (hwe->instance *4 ); + }else { +XE_WARN_ON (hwe->class); +return 0 ; + } + +return (rcu_debug1 >> shift) &RCU_DEBUG_1_ENGINE_STATUS ; +} + +static bool engine_runalone_set (const struct xe_hw_engine *const hwe, +u32 rcu_debug1) +{ +return engine_status (hwe, rcu_debug1) &RCU_DEBUG_1_RUNALONE_ACTIVE ; +} + +static bool engine_context_set (const struct xe_hw_engine *const hwe, +u32 rcu_debug1) +{ +return engine_status (hwe, rcu_debug1) &RCU_DEBUG_1_CONTEXT_ACTIVE ; +} + +static bool engine_has_runalone (const struct xe_hw_engine *const hwe) +{ +return hwe->class ==XE_ENGINE_CLASS_RENDER || + hwe->class ==XE_ENGINE_CLASS_COMPUTE ; +} + +static struct xe_hw_engine *get_runalone_active_hw_engine (struct xe_gt *gt) +{ +struct xe_hw_engine *hwe, *first =NULL ; +unsigned int num_active, id; +u32 val; + +if (xe_force_wake_get (gt_to_fw (gt),XE_FW_GT )) { +drm_dbg (>_to_xe (gt)->drm,"eudbg: runalone failed to get force wake\n" ); +return NULL ; + } + + val =xe_mmio_read32 (gt,RCU_DEBUG_1 ); +xe_force_wake_put (gt_to_fw (gt),XE_FW_GT ); + +drm_dbg (>_to_xe (gt)->drm,"eudbg: runalone RCU_DEBUG_1 = 0x%08x\n" , val); + + num_active =0 ; +for_each_hw_engine (hwe, gt, id) { +bool runalone, ctx; + +if (!engine_has_runalone (hwe)) +continue ; + + runalone =engine_runalone_set (hwe, val); + ctx =engine_context_set (hwe, val); + +drm_dbg (>_to_xe (gt)->drm,"eudbg: engine %s: runalone=%s, context=%s" , + hwe->name, runalone ?"active" :"inactive" , + ctx ?"active" :"inactive" ); + +/* +* On earlier gen12 the context status seems to be idle when +* it has raised attention. We have to omit the active bit. +*/ +if (IS_DGFX (gt_to_xe (gt))) + ctx = true; + +if (runalone && ctx) { + num_active++; + +drm_dbg (>_to_xe (gt)->drm,"eudbg: runalone engine %s %s" , + hwe->name, first ?"selected" :"found" ); +if (!first) + first = hwe; + } + } + +if (num_active >1 ) +drm_err (>_to_xe (gt)->drm,"eudbg: %d runalone engines active!" , + num_active); + +return first; +} + +static struct xe_exec_queue *runalone_active_queue_get (struct xe_gt *gt,int *lrc_idx) +{ +struct xe_device *xe =gt_to_xe (gt); +struct xe_exec_queue *q, *found =NULL ; +struct xe_hw_engine *active; +struct xe_file *xef, *tmp; +unsigned long i; +int idx, err; +u32 lrc_hw; + + active =get_runalone_active_hw_engine (gt); +if (!active) { +drm_dbg (>_to_xe (gt)->drm,"Runalone engine not found!" ); +return ERR_PTR (-ENOENT );drivers/gpu/drm/xe/xe_eudebug.c static const struct file_operations fops = { · line 1036
Quoted 10 lines
drivers/gpu/drm/xe/xe_eudebug.c +10
-0+ } + + err =current_lrc (active, &lrc_hw); +if (err) +return ERR_PTR (err); + +mutex_lock (&xe->files.lock); +list_for_each_entry_safe (xef, tmp, &xe->files.list, link) { +mutex_lock (&xef->exec_queue.lock); +xa_for_each (&xef->exec_queue.xa, i, q) {Same comment here as [1]. Don't hold
xe->files.lockorxef->exec_queue.lockfor anything but the lookup. Holding locks longer than needed creates unwanted lock dep chains which creates all sorts of problems. The only time chains should exist is when we have well defined locking chain like we have for a VM (vm->lock-> dma-resv -> notifier lock).So this should look something like this:
mutex_lock(&xe->files.lock);xa_for_each(.., xef) { xe_file_get(xef); mutex_unlock(&xe->files.lock); mutex_lock(&xef->exec_queue.lock); xa_for_each(&xef->exec_queue.xa, i, q) { xe_exec_queue_get(q); mutex_unlock(&xef->exec_queue.lock); /* Do something */ xe_exec_queue_put(q); mutex_lock(&xef->exec_queue.lock); } mutex_unlock(&xef->exec_queue.lock); xe_file_put(xef); mutex_lock(&xe->files.lock); }
mutex_unlock(&xe->files.lock);Note you will need to change
xe->files.listto an xarray to make this safe too.Matt
[1] https://patchwork.freedesktop.org/patch/606052/?series=136572&rev=1
Quoted 513 lines
drivers/gpu/drm/xe/xe_eudebug.c +196
-0+if (q->gt != gt) +continue ; + +if (q->class != active->class) +continue ; + +if (!queue_has_active_job (q)) +continue ; + + idx =match_engine_lrc (q, lrc_hw); +if (idx <0 ) +continue ; + +xe_exec_queue_get (q); + found = q; + +if (lrc_idx) + *lrc_idx = idx; + +break ; + } +mutex_unlock (&xef->exec_queue.lock); + +if (found) +break ; + } +mutex_unlock (&xe->files.lock); + +if (!found) +return ERR_PTR (-ENOENT ); + +if (XE_WARN_ON (current_lrc (active, &lrc_hw)) && +XE_WARN_ON (match_engine_lrc (found, lrc_hw) <0 )) { +xe_exec_queue_put (found); +return ERR_PTR (-ENOENT ); + } + +return found; +} + +static int send_attention_event (struct xe_eudebug *d,struct xe_exec_queue *q,int lrc_idx) +{ +struct xe_eudebug_event_eu_attention *ea; +struct xe_eudebug_event *event; +int h_c, h_queue, h_lrc; +u32 size =xe_gt_eu_attention_bitmap_size (q->gt); +u32 sz =struct_size (ea, bitmask, size); +int ret; + +XE_WARN_ON (lrc_idx <0 || lrc_idx >= q->width); + + h_c =find_handle (d->res,XE_EUDEBUG_RES_TYPE_CLIENT , q->vm->xef); +if (h_c <0 ) +return h_c; + + h_queue =find_handle (d->res,XE_EUDEBUG_RES_TYPE_EXEC_QUEUE , q); +if (h_queue <0 ) +return h_queue; + + h_lrc =find_handle (d->res,XE_EUDEBUG_RES_TYPE_LRC , q->lrc[lrc_idx]); +if (h_lrc <0 ) +return h_lrc; + + event =__xe_eudebug_create_event (d,0 ,DRM_XE_EUDEBUG_EVENT_EU_ATTENTION , +DRM_XE_EUDEBUG_EVENT_STATE_CHANGE , sz,GFP_KERNEL ); + +if (!event) +return -ENOSPC ; + + ea =cast_event (ea, event); + write_member(struct drm_xe_eudebug_event_eu_attention, ea, client_handle, (u64)h_c); + write_member(struct drm_xe_eudebug_event_eu_attention, ea, exec_queue_handle, (u64)h_queue); + write_member(struct drm_xe_eudebug_event_eu_attention, ea, lrc_handle, (u64)h_lrc); + write_member(struct drm_xe_eudebug_event_eu_attention, ea, bitmask_size, size); + +mutex_lock (&d->eu_lock); + event->seqno =atomic_long_inc_return (&d->events.seqno); + ret =xe_gt_eu_attention_bitmap (q->gt, &ea->bitmask[0 ], ea->bitmask_size); +mutex_unlock (&d->eu_lock); + +if (ret) +return ret; + +return xe_eudebug_queue_event (d, event); +} + + +static int xe_send_gt_attention (struct xe_gt *gt) +{ +struct xe_eudebug *d; +struct xe_exec_queue *q; +int ret, lrc_idx; + +if (list_empty_careful (>_to_xe (gt)->eudebug.list)) +return -ENOTCONN ; + + q =runalone_active_queue_get (gt, &lrc_idx); +if (IS_ERR (q)) +return PTR_ERR (q); + + d =xe_eudebug_get (q->vm->xef); +if (!d) { + ret = -ENOTCONN ; + goto err_exec_queue_put; + } + +if (!completion_done (&d->discovery)) { +eu_dbg (d,"discovery not yet done\n" ); + ret = -EBUSY ; + goto err_eudebug_put; + } + + ret =send_attention_event (d, q, lrc_idx); +if (ret) +xe_eudebug_disconnect (d, ret); + +err_eudebug_put: +xe_eudebug_put (d); +err_exec_queue_put: +xe_exec_queue_put (q); + +return ret; +} + +static int xe_eudebug_handle_gt_attention (struct xe_gt *gt) +{ +int ret; + + ret =xe_gt_eu_threads_needing_attention (gt); +if (ret <=0 ) +return ret; + + ret =xe_send_gt_attention (gt); + +/* Discovery in progress, fake it */ +if (ret == -EBUSY ) +return 0 ; + +return ret; +} + +#define XE_EUDEBUG_ATTENTION_INTERVAL 100 +static void attention_scan_fn (struct work_struct *work) +{ +struct xe_device *xe =container_of (work,typeof (*xe), eudebug.attention_scan.work); +long delay =msecs_to_jiffies (XE_EUDEBUG_ATTENTION_INTERVAL ); +struct xe_gt *gt; +u8 gt_id; + +if (list_empty_careful (&xe->eudebug.list)) + delay *=10 ; + +if (delay >=HZ ) + delay =round_jiffies_up_relative (delay); + +if (pm_runtime_active (xe->drm.dev)) { +for_each_gt (gt, xe, gt_id) { +int ret; + + ret =xe_eudebug_handle_gt_attention (gt); +if (ret) { +// TODO: error capture +drm_info (>_to_xe (gt)->drm, +"gt:%d unable to handle eu attention ret=%d\n" , + gt_id, ret); + +xe_gt_reset_async (gt); + } + } + } + +schedule_delayed_work (&xe->eudebug.attention_scan, delay); +} + +static void attention_scan_cancel (struct xe_device *xe) +{ +cancel_delayed_work_sync (&xe->eudebug.attention_scan); +} + +static void attention_scan_flush (struct xe_device *xe) +{ +mod_delayed_work (system_wq, &xe->eudebug.attention_scan,0 ); +} +static void discovery_work_fn (struct work_struct *work); static int@@ -877,6 +1252,7 @@ xe_eudebug_connect(struct xe_device *xe,kref_init (&d->ref);spin_lock_init (&d->connection.lock); +mutex_init (&d->eu_lock);init_waitqueue_head (&d->events.write_done);init_waitqueue_head (&d->events.read_done);init_completion (&d->discovery);@@ -903,6 +1279,7 @@ xe_eudebug_connect(struct xe_device *xe,kref_get (&d->ref);queue_work (xe->eudebug.ordered_wq, &d->discovery_work); +attention_scan_flush (xe);eu_dbg (d,"connected session %lld" , d->session);@@ -979,12 +1356,22 @@ void xe_eudebug_init(struct xe_device *xe) {spin_lock_init (&xe->eudebug.lock);INIT_LIST_HEAD (&xe->eudebug.list); +INIT_DELAYED_WORK (&xe->eudebug.attention_scan, attention_scan_fn); xe->eudebug.available = true; } +void xe_eudebug_init_late (struct xe_device *xe) +{ +if (!xe->eudebug.available) +return ; + +attention_scan_flush (xe); +} +void xe_eudebug_fini (struct xe_device *xe) { +attention_scan_cancel (xe);xe_assert (xe,list_empty_careful (&xe->eudebug.list)); }drivers/gpu/drm/xe/xe_eudebug.h +1
-0@@ -18,6 +18,7 @@ int xe_eudebug_connect_ioctl(struct drm_device *dev, struct drm_file *file);void xe_eudebug_init (struct xe_device *xe); +void xe_eudebug_init_late (struct xe_device *xe);void xe_eudebug_fini (struct xe_device *xe);void xe_eudebug_init_hw_engine (struct xe_hw_engine *hwe);drivers/gpu/drm/xe/xe_eudebug_types.h +32
-0@@ -105,6 +105,9 @@ struct xe_eudebug {/** @discovery_work: worker to discover resources for target_task */ struct work_struct discovery_work; +/** eu_lock: guards operations on eus (eu thread control and attention) */ +struct mutex eu_lock; +/** @events: kfifo queue of to-be-delivered events */ struct {/** @lock: guards access to fifo */ @@ -202,4 +205,33 @@ struct xe_eudebug_event_exec_queue { u64 lrc_handle[]; }; +/** +* struct xe_eudebug_event_eu_attention - Internal event for EU attention +*/ +struct xe_eudebug_event_eu_attention { +/** @base: base event */ +struct xe_eudebug_event base; + +/** @client_handle: client for the attention */ +u64 client_handle; + +/** @exec_queue_handle: handle of exec_queue which raised attention */ +u64 exec_queue_handle; + +/** @lrc_handle: lrc handle of the workload which raised attention */ +u64 lrc_handle; + +/** @flags: eu attention event flags, currently MBZ */ +u32 flags; + +/** @bitmask_size: size of the bitmask, specific to device */ +u32 bitmask_size; + +/** +* @bitmask: reflects threads currently signalling attention, +* starting from natural hardware order of DSS=0, eu=0 +*/ +u8 bitmask[]; +}; +#endif drivers/gpu/drm/xe/xe_gt_debug.c new file +152
-0@@ -0,0 +1,152 @@ +// SPDX-License-Identifier: MIT +/* +* Copyright © 2023 Intel Corporation +*/ + +#include "regs/xe_gt_regs.h" +#include "xe_device.h" +#include "xe_force_wake.h" +#include "xe_gt.h" +#include "xe_gt_topology.h" +#include "xe_gt_debug.h" +#include "xe_gt_mcr.h" +#include "xe_pm.h" +#include "xe_macros.h" + +static int xe_gt_foreach_dss_group_instance (struct xe_gt *gt, +int (*fn)(struct xe_gt *gt, +void *data, +u16 group, +u16 instance), +void *data) +{ +const enum xe_force_wake_domains fw_domains =XE_FW_GT |XE_FW_RENDER ; +unsigned int dss; +u16 group, instance; +int ret; + +xe_pm_runtime_get (gt_to_xe (gt)); + ret =xe_force_wake_get (gt_to_fw (gt), fw_domains); +if (ret) + goto pm_runtime_put; + +for_each_dss_steering (dss, gt, group, instance) { + ret =fn (gt, data, group, instance); +if (ret) +break ; + } + +xe_force_wake_put (gt_to_fw (gt), fw_domains); +pm_runtime_put: +xe_pm_runtime_put (gt_to_xe (gt)); + +return ret; +} + +static int read_first_attention_mcr (struct xe_gt *gt,void *data, +u16 group,u16 instance) +{ +unsigned int row; + +for (row =0 ; row <2 ; row++) { +u32 val; + + val =xe_gt_mcr_unicast_read (gt,TD_ATT (row), group, instance); + +if (val) +return 1 ; + } + +return 0 ; +} + +#define MAX_EUS_PER_ROW 4u +#define MAX_THREADS 8u + +/** +* xe_gt_eu_attention_bitmap_size - query size of the attention bitmask +* +* @gt: pointer to struct xe_gt +* +* Return: size in bytes. +*/ +int xe_gt_eu_attention_bitmap_size (struct xe_gt *gt) +{ +xe_dss_mask_t dss_mask; + +bitmap_or (dss_mask, gt->fuse_topo.c_dss_mask, + gt->fuse_topo.g_dss_mask,XE_MAX_DSS_FUSE_BITS ); + +return bitmap_weight (dss_mask,XE_MAX_DSS_FUSE_BITS ) * +TD_EU_ATTENTION_MAX_ROWS *MAX_THREADS * +MAX_EUS_PER_ROW /8 ; +} + +struct attn_read_iter { +struct xe_gt *gt; +unsigned int i; +unsigned int size; +u8 *bits; +}; + +static int read_eu_attentions_mcr (struct xe_gt *gt,void *data, +u16 group,u16 instance) +{ +struct attn_read_iter *const iter = data; +unsigned int row; + +for (row =0 ; row <TD_EU_ATTENTION_MAX_ROWS ; row++) { +u32 val; + +if (iter->i >= iter->size) +return 0 ; + +XE_WARN_ON (iter->i +sizeof (val) >xe_gt_eu_attention_bitmap_size (gt)); + + val =xe_gt_mcr_unicast_read (gt,TD_ATT (row), group, instance); + + +memcpy (&iter->bits[iter->i], &val,sizeof (val)); + iter->i +=sizeof (val); + } + +return 0 ; +} + +/** +* xe_gt_eu_attention_bitmap - query host attention +* +* @gt: pointer to struct xe_gt +* +* Return: 0 on success, negative otherwise. +*/ +int xe_gt_eu_attention_bitmap (struct xe_gt *gt,u8 *bits, +unsigned int bitmap_size) +{ +struct attn_read_iter iter = { + .gt = gt, + .i =0 , + .size = bitmap_size, + .bits = bits + }; + +return xe_gt_foreach_dss_group_instance (gt, read_eu_attentions_mcr, &iter); +} + +/** +* xe_gt_eu_threads_needing_attention - Query host attention +* +* @gt: pointer to struct xe_gt +* +* Return: 1 if threads waiting host attention, 0 otherwise. +*/ +int xe_gt_eu_threads_needing_attention (struct xe_gt *gt) +{ +int err; + + err =xe_gt_foreach_dss_group_instance (gt, read_first_attention_mcr,NULL ); + +XE_WARN_ON (err <0 ); + +return err <0 ?0 : err; +}drivers/gpu/drm/xe/xe_gt_debug.h new file +21
-0@@ -0,0 +1,21 @@ +/* SPDX-License-Identifier: MIT */ +/* +* Copyright © 2023 Intel Corporation +*/ + +#ifndef __XE_GT_DEBUG_ +#define __XE_GT_DEBUG_ + +#define TD_EU_ATTENTION_MAX_ROWS 2u + +#include "xe_gt_types.h" + +#define XE_GT_ATTENTION_TIMEOUT_MS 100 + +int xe_gt_eu_threads_needing_attention (struct xe_gt *gt); + +int xe_gt_eu_attention_bitmap_size (struct xe_gt *gt); +int xe_gt_eu_attention_bitmap (struct xe_gt *gt,u8 *bits, +unsigned int bitmap_size); + +#endif include/uapi/drm/xe_drm_eudebug.h +14
-1@@ -27,13 +27,15 @@ struct drm_xe_eudebug_event {#define DRM_XE_EUDEBUG_EVENT_OPEN 2#define DRM_XE_EUDEBUG_EVENT_VM 3#define DRM_XE_EUDEBUG_EVENT_EXEC_QUEUE 4 -#define DRM_XE_EUDEBUG_EVENT_MAX_EVENT DRM_XE_EUDEBUG_EVENT_EXEC_QUEUE +#define DRM_XE_EUDEBUG_EVENT_EU_ATTENTION 5 +#define DRM_XE_EUDEBUG_EVENT_MAX_EVENT DRM_XE_EUDEBUG_EVENT_EU_ATTENTION__u16 flags;#define DRM_XE_EUDEBUG_EVENT_CREATE (1 << 0)#define DRM_XE_EUDEBUG_EVENT_DESTROY (1 << 1)#define DRM_XE_EUDEBUG_EVENT_STATE_CHANGE (1 << 2)#define DRM_XE_EUDEBUG_EVENT_NEED_ACK (1 << 3) +__u64 seqno;__u64 reserved; };@@ -62,6 +64,17 @@ struct drm_xe_eudebug_event_exec_queue {__u64 lrc_handle[]; }; +struct drm_xe_eudebug_event_eu_attention { +struct drm_xe_eudebug_event base; + +__u64 client_handle; +__u64 exec_queue_handle; +__u64 lrc_handle; +__u32 flags; +__u32 bitmask_size; +__u8 bitmask[]; +}; +#if defined(__cplusplus) }#endif Signature · 1 line
2.34.1
Matthew Brost
maintainerRe: [PATCH 10/21] drm/xe/eudebug: Introduce per device attention scan worker
In reply to Mika KuoppalaQuoted 654 lines · Mika Kuoppala
From: Dominik Grzegorzek <[email protected]>
Scan for EU debugging attention bits periodically to detect if some EU thread has entered the system routine (SIP) due to EU thread exception.
Make the scanning interval 10 times slower when there is no debugger connection open. Send attention event whenever we see attention with debugger presence. If there is no debugger connection active - reset.
Based on work by authors and other folks who were part of attentions in i915.
- v2 Do not validate potentially active hwe against
engine->hwe. Whenever the engine has width > 1, this field contains only the first hwe of the class.
- squash dss walking and semaphore to mutex
- v3 error path fix inxe_send_gt_attention(Christoph)
- v4 runalone active fix (Mika)
- v5q->lrcchanges (Mika)
- v6 Use C99 flexible arrays (Maciej, checkpatch)
function with 'for_each' in name (Maciej, checkpatch)
- v7 long running active fix (Dominik)
- v8 resource handling errors rebase (Mika)
- v9 find out lrc handles first before sending event (Mika)
- v10 adjust runalone shift according to hwSigned-off-by Dominik Grzegorzek <[email protected]>Signed-off-by Christoph Manszewski <[email protected]>Signed-off-by Maciej Patelczyk <[email protected]>Signed-off-by Mika Kuoppala <[email protected]>--- drivers/gpu/drm/xe/Makefile | 1 + drivers/gpu/drm/xe/regs/xe_engine_regs.h | 3 + drivers/gpu/drm/xe/regs/xe_gt_regs.h | 7 + drivers/gpu/drm/xe/xe_device.c | 2 + drivers/gpu/drm/xe/xe_device_types.h | 3 + drivers/gpu/drm/xe/xe_eudebug.c | 389 ++++++++++++++++++++++- drivers/gpu/drm/xe/xe_eudebug.h | 1 + drivers/gpu/drm/xe/xe_eudebug_types.h | 32 ++ drivers/gpu/drm/xe/xe_gt_debug.c | 152 +++++++++ drivers/gpu/drm/xe/xe_gt_debug.h | 21 ++ include/uapi/drm/xe_drm_eudebug.h | 15 +- 11 files changed, 624 insertions(+), 2 deletions(-) create mode 100644 drivers/gpu/drm/xe/xe_gt_debug.c create mode 100644 drivers/gpu/drm/xe/xe_gt_debug.hdrivers/gpu/drm/xe/Makefile +1
-0@@ -49,6 +49,7 @@ xe-y += xe_bb.o \ xe_gt_debugfs.o \ xe_gt_freq.o \ xe_gt_idle.o \ + xe_gt_debug.o \ xe_gt_mcr.o \ xe_gt_pagefault.o \ xe_gt_sysfs.o \drivers/gpu/drm/xe/regs/xe_engine_regs.h +3
-0@@ -132,6 +132,9 @@#define RING_EXECLIST_STATUS_LO (base) XE_REG((base) + 0x234)#define RING_EXECLIST_STATUS_HI (base) XE_REG((base) + 0x234 + 4) +#define RING_CURRENT_LRCA (base) XE_REG((base) + 0x240) +#define CURRENT_LRCA_VALID REG_BIT(0) +#define RING_CONTEXT_CONTROL (base) XE_REG((base) + 0x244, XE_REG_OPTION_MASKED)#define CTX_CTRL_OAC_CONTEXT_ENABLE REG_BIT(8)#define CTX_CTRL_RUN_ALONE REG_BIT(7)drivers/gpu/drm/xe/regs/xe_gt_regs.h +7
-0@@ -437,6 +437,8 @@#define DISABLE_ECC REG_BIT(5)#define ENABLE_PREFETCH_INTO_IC REG_BIT(3) +#define TD_ATT (x) XE_REG_MCR(0xe470 + (x) * 4) +#define ROW_CHICKEN4 XE_REG_MCR(0xe48c, XE_REG_OPTION_MASKED)#define DISABLE_GRF_CLEAR REG_BIT(13)#define XEHP_DIS_BBL_SYSPIPE REG_BIT(11)@@ -516,6 +518,11 @@#define CCS_MODE_CSLICE (cslice, ccs) \ ((ccs) << ((cslice) * CCS_MODE_CSLICE_WIDTH)) +#define RCU_DEBUG_1 XE_REG(0x14a00) +#define RCU_DEBUG_1_ENGINE_STATUS REG_GENMASK(2, 0) +#define RCU_DEBUG_1_RUNALONE_ACTIVE REG_BIT(2) +#define RCU_DEBUG_1_CONTEXT_ACTIVE REG_BIT(0) +#define FORCEWAKE_ACK_GT XE_REG(0x130044)/* Applicable for all FORCEWAKE_DOMAIN and FORCEWAKE_ACK_DOMAIN regs */ drivers/gpu/drm/xe/xe_device.c +2
-0@@ -768,6 +768,8 @@ int xe_device_probe(struct xe_device *xe)xe_debugfs_register (xe); +xe_eudebug_init_late (xe); +xe_hwmon_register (xe);for_each_gt (gt, xe, id)drivers/gpu/drm/xe/xe_device_types.h +3
-0@@ -516,6 +516,9 @@ struct xe_device {/** @ordered_wq: used to discovery */ struct workqueue_struct *ordered_wq; + +/** @attention_scan: attention scan worker */ +struct delayed_work attention_scan; } eudebug;/* private: */ drivers/gpu/drm/xe/xe_eudebug.c +388
-1@@ -11,19 +11,29 @@#include <drm/drm_managed.h> -#include "regs/xe_gt_regs.h" #include "regs/xe_engine_regs.h" +#include "regs/xe_gt_regs.h" #include "xe_device.h" #include "xe_assert.h" #include "xe_macros.h" #include "xe_gt.h" +#include "xe_gt_debug.h" +#include "xe_lrc.h" +#include "xe_hw_engine.h" +#include "xe_exec_queue.h" #include "xe_eudebug_types.h" #include "xe_eudebug.h" #include "xe_exec_queue_types.h" +#include "xe_guc_exec_queue_types.h" +#include "xe_execlist_types.h" +#include "xe_mmio.h" #include "xe_module.h" +#include "xe_pm.h" #include "xe_rtp.h" +#include "xe_sched_job.h" #include "xe_vm.h" #include "xe_wa.h" +#include "xe_force_wake.h" /* * If there is no detected event read by userspace, during this period, assume @@ -843,6 +853,371 @@ static const struct file_operations fops = { .unlocked_ioctl = xe_eudebug_ioctl, }; +static bool queue_has_active_job (struct xe_exec_queue *q) +{ + +struct drm_gpu_scheduler *sched; +struct drm_sched_job *drm_job; + +if (xe_device_uc_enabled (gt_to_xe (q->gt))) + sched = &q->guc->sched.base; +else + sched = &q->execlist->sched; + + drm_job = list_first_entry_or_null(&sched->pending_list, struct drm_sched_job, list); + +if (drm_job) { +struct xe_sched_job *job =to_xe_sched_job (drm_job); + +return xe_sched_job_started (job) && !xe_sched_job_completed (job); + }else if (xe_exec_queue_is_lr (q) && + (xe_lrc_ring_head (q->lrc[0 ]) !=xe_lrc_ring_tail (q->lrc[0 ]))) { +return true; + } + +return false; +} + +static int current_lrc (struct xe_hw_engine *hwe,u32 *lrc_hw) +{ +u32 lrc_reg; +int err; + + err =xe_force_wake_get (gt_to_fw (hwe->gt), hwe->domain); +if (err) +return err; + + lrc_reg =hw_engine_mmio_read32 (hwe,RING_CURRENT_LRCA (0 )); + +xe_force_wake_put (gt_to_fw (hwe->gt), hwe->domain); + +if (!(lrc_reg &CURRENT_LRCA_VALID )) +return -ENOENT ; + + *lrc_hw = lrc_reg &GENMASK (31 ,12 ); + +return 0 ; +} + +static int match_engine_lrc (struct xe_exec_queue *q,u32 lrc_hw) +{ +int i; +u32 lrc_ggtt; + +for (i =0 ; i < q->width; i++) { + lrc_ggtt =lower_32_bits (xe_lrc_descriptor (q->lrc[i])); + lrc_ggtt &=GENMASK (31 ,12 ); +if (lrc_ggtt == lrc_hw) +return i; + } + +return -1 ; +} + +static u32 engine_status (const struct xe_hw_engine *const hwe, +u32 rcu_debug1) +{ +const bool xe1 =GRAPHICS_VER (gt_to_xe (hwe->gt)) <20 ; +unsigned int shift; + +if (hwe->class ==XE_ENGINE_CLASS_RENDER ) { + shift =7 ; +XE_WARN_ON (hwe->instance !=0 ); + }else if (hwe->class ==XE_ENGINE_CLASS_COMPUTE ) { +XE_WARN_ON (hwe->instance >3 ); + +if (xe1) + shift =10 + (hwe->instance *3 ); +else + shift =11 + (hwe->instance *4 ); + }else { +XE_WARN_ON (hwe->class); +return 0 ; + } + +return (rcu_debug1 >> shift) &RCU_DEBUG_1_ENGINE_STATUS ; +} + +static bool engine_runalone_set (const struct xe_hw_engine *const hwe, +u32 rcu_debug1) +{ +return engine_status (hwe, rcu_debug1) &RCU_DEBUG_1_RUNALONE_ACTIVE ; +} + +static bool engine_context_set (const struct xe_hw_engine *const hwe, +u32 rcu_debug1) +{ +return engine_status (hwe, rcu_debug1) &RCU_DEBUG_1_CONTEXT_ACTIVE ; +} + +static bool engine_has_runalone (const struct xe_hw_engine *const hwe) +{ +return hwe->class ==XE_ENGINE_CLASS_RENDER || + hwe->class ==XE_ENGINE_CLASS_COMPUTE ; +} + +static struct xe_hw_engine *get_runalone_active_hw_engine (struct xe_gt *gt) +{ +struct xe_hw_engine *hwe, *first =NULL ; +unsigned int num_active, id; +u32 val; + +if (xe_force_wake_get (gt_to_fw (gt),XE_FW_GT )) { +drm_dbg (>_to_xe (gt)->drm,"eudbg: runalone failed to get force wake\n" ); +return NULL ; + } + + val =xe_mmio_read32 (gt,RCU_DEBUG_1 ); +xe_force_wake_put (gt_to_fw (gt),XE_FW_GT ); + +drm_dbg (>_to_xe (gt)->drm,"eudbg: runalone RCU_DEBUG_1 = 0x%08x\n" , val); + + num_active =0 ; +for_each_hw_engine (hwe, gt, id) { +bool runalone, ctx; + +if (!engine_has_runalone (hwe)) +continue ; + + runalone =engine_runalone_set (hwe, val); + ctx =engine_context_set (hwe, val); + +drm_dbg (>_to_xe (gt)->drm,"eudbg: engine %s: runalone=%s, context=%s" , + hwe->name, runalone ?"active" :"inactive" , + ctx ?"active" :"inactive" ); + +/* +* On earlier gen12 the context status seems to be idle when +* it has raised attention. We have to omit the active bit. +*/ +if (IS_DGFX (gt_to_xe (gt))) + ctx = true; + +if (runalone && ctx) { + num_active++; + +drm_dbg (>_to_xe (gt)->drm,"eudbg: runalone engine %s %s" , + hwe->name, first ?"selected" :"found" ); +if (!first) + first = hwe; + } + } + +if (num_active >1 ) +drm_err (>_to_xe (gt)->drm,"eudbg: %d runalone engines active!" , + num_active); + +return first; +} + +static struct xe_exec_queue *runalone_active_queue_get (struct xe_gt *gt,int *lrc_idx) +{ +struct xe_device *xe =gt_to_xe (gt); +struct xe_exec_queue *q, *found =NULL ; +struct xe_hw_engine *active; +struct xe_file *xef, *tmp; +unsigned long i; +int idx, err; +u32 lrc_hw; + + active =get_runalone_active_hw_engine (gt); +if (!active) { +drm_dbg (>_to_xe (gt)->drm,"Runalone engine not found!" ); +return ERR_PTR (-ENOENT ); + } + + err =current_lrc (active, &lrc_hw); +if (err) +return ERR_PTR (err); + +mutex_lock (&xe->files.lock); +list_for_each_entry_safe (xef, tmp, &xe->files.list, link) { +mutex_lock (&xef->exec_queue.lock); +xa_for_each (&xef->exec_queue.xa, i, q) { +if (q->gt != gt) +continue ; + +if (q->class != active->class) +continue ; + +if (!queue_has_active_job (q)) +continue ; + + idx =match_engine_lrc (q, lrc_hw); +if (idx <0 ) +continue ; + +xe_exec_queue_get (q); + found = q; + +if (lrc_idx) + *lrc_idx = idx; + +break ; + } +mutex_unlock (&xef->exec_queue.lock); + +if (found) +break ; + } +mutex_unlock (&xe->files.lock); + +if (!found) +return ERR_PTR (-ENOENT ); + +if (XE_WARN_ON (current_lrc (active, &lrc_hw)) && +XE_WARN_ON (match_engine_lrc (found, lrc_hw) <0 )) { +xe_exec_queue_put (found); +return ERR_PTR (-ENOENT ); + } + +return found; +} + +static int send_attention_event (struct xe_eudebug *d,struct xe_exec_queue *q,int lrc_idx) +{ +struct xe_eudebug_event_eu_attention *ea; +struct xe_eudebug_event *event; +int h_c, h_queue, h_lrc; +u32 size =xe_gt_eu_attention_bitmap_size (q->gt); +u32 sz =struct_size (ea, bitmask, size); +int ret; + +XE_WARN_ON (lrc_idx <0 || lrc_idx >= q->width); + + h_c =find_handle (d->res,XE_EUDEBUG_RES_TYPE_CLIENT , q->vm->xef); +if (h_c <0 ) +return h_c; + + h_queue =find_handle (d->res,XE_EUDEBUG_RES_TYPE_EXEC_QUEUE , q); +if (h_queue <0 ) +return h_queue; + + h_lrc =find_handle (d->res,XE_EUDEBUG_RES_TYPE_LRC , q->lrc[lrc_idx]); +if (h_lrc <0 ) +return h_lrc; + + event =__xe_eudebug_create_event (d,0 ,DRM_XE_EUDEBUG_EVENT_EU_ATTENTION , +DRM_XE_EUDEBUG_EVENT_STATE_CHANGE , sz,GFP_KERNEL ); + +if (!event) +return -ENOSPC ; + + ea =cast_event (ea, event); + write_member(struct drm_xe_eudebug_event_eu_attention, ea, client_handle, (u64)h_c); + write_member(struct drm_xe_eudebug_event_eu_attention, ea, exec_queue_handle, (u64)h_queue); + write_member(struct drm_xe_eudebug_event_eu_attention, ea, lrc_handle, (u64)h_lrc); + write_member(struct drm_xe_eudebug_event_eu_attention, ea, bitmask_size, size); + +mutex_lock (&d->eu_lock); + event->seqno =atomic_long_inc_return (&d->events.seqno); + ret =xe_gt_eu_attention_bitmap (q->gt, &ea->bitmask[0 ], ea->bitmask_size); +mutex_unlock (&d->eu_lock); + +if (ret) +return ret; + +return xe_eudebug_queue_event (d, event); +} + + +static int xe_send_gt_attention (struct xe_gt *gt) +{ +struct xe_eudebug *d; +struct xe_exec_queue *q; +int ret, lrc_idx; + +if (list_empty_careful (>_to_xe (gt)->eudebug.list)) +return -ENOTCONN ; + + q =runalone_active_queue_get (gt, &lrc_idx); +if (IS_ERR (q)) +return PTR_ERR (q); + + d =xe_eudebug_get (q->vm->xef); +if (!d) { + ret = -ENOTCONN ; + goto err_exec_queue_put; + } + +if (!completion_done (&d->discovery)) { +eu_dbg (d,"discovery not yet done\n" ); + ret = -EBUSY ; + goto err_eudebug_put; + } + + ret =send_attention_event (d, q, lrc_idx); +if (ret) +xe_eudebug_disconnect (d, ret); + +err_eudebug_put: +xe_eudebug_put (d); +err_exec_queue_put: +xe_exec_queue_put (q); + +return ret; +} + +static int xe_eudebug_handle_gt_attention (struct xe_gt *gt) +{ +int ret; + + ret =xe_gt_eu_threads_needing_attention (gt); +if (ret <=0 ) +return ret; + + ret =xe_send_gt_attention (gt); + +/* Discovery in progress, fake it */ +if (ret == -EBUSY ) +return 0 ; + +return ret; +} + +#define XE_EUDEBUG_ATTENTION_INTERVAL 100 +static void attention_scan_fn (struct work_struct *work) +{ +struct xe_device *xe =container_of (work,typeof (*xe), eudebug.attention_scan.work); +long delay =msecs_to_jiffies (XE_EUDEBUG_ATTENTION_INTERVAL ); +struct xe_gt *gt; +u8 gt_id; + +if (list_empty_careful (&xe->eudebug.list)) + delay *=10 ; + +if (delay >=HZ ) + delay =round_jiffies_up_relative (delay); + +if (pm_runtime_active (xe->drm.dev)) { +for_each_gt (gt, xe, gt_id) { +int ret; + + ret =xe_eudebug_handle_gt_attention (gt); +if (ret) { +// TODO: error capture +drm_info (>_to_xe (gt)->drm, +"gt:%d unable to handle eu attention ret=%d\n" , + gt_id, ret); + +xe_gt_reset_async (gt); + } + } + } + +schedule_delayed_work (&xe->eudebug.attention_scan, delay); +} + +static void attention_scan_cancel (struct xe_device *xe) +{ +cancel_delayed_work_sync (&xe->eudebug.attention_scan); +} + +static void attention_scan_flush (struct xe_device *xe) +{ +mod_delayed_work (system_wq, &xe->eudebug.attention_scan,0 ); +} +static void discovery_work_fn (struct work_struct *work); static int@@ -877,6 +1252,7 @@ xe_eudebug_connect(struct xe_device *xe,kref_init (&d->ref);spin_lock_init (&d->connection.lock); +mutex_init (&d->eu_lock);init_waitqueue_head (&d->events.write_done);init_waitqueue_head (&d->events.read_done);init_completion (&d->discovery);@@ -903,6 +1279,7 @@ xe_eudebug_connect(struct xe_device *xe,kref_get (&d->ref);queue_work (xe->eudebug.ordered_wq, &d->discovery_work); +attention_scan_flush (xe);eu_dbg (d,"connected session %lld" , d->session);@@ -979,12 +1356,22 @@ void xe_eudebug_init(struct xe_device *xe) {spin_lock_init (&xe->eudebug.lock);INIT_LIST_HEAD (&xe->eudebug.list); +INIT_DELAYED_WORK (&xe->eudebug.attention_scan, attention_scan_fn); xe->eudebug.available = true; } +void xe_eudebug_init_late (struct xe_device *xe) +{ +if (!xe->eudebug.available) +return ; + +attention_scan_flush (xe); +} +void xe_eudebug_fini (struct xe_device *xe) { +attention_scan_cancel (xe);xe_assert (xe,list_empty_careful (&xe->eudebug.list)); }drivers/gpu/drm/xe/xe_eudebug.h +1
-0@@ -18,6 +18,7 @@ int xe_eudebug_connect_ioctl(struct drm_device *dev, struct drm_file *file);void xe_eudebug_init (struct xe_device *xe); +void xe_eudebug_init_late (struct xe_device *xe);void xe_eudebug_fini (struct xe_device *xe);void xe_eudebug_init_hw_engine (struct xe_hw_engine *hwe);drivers/gpu/drm/xe/xe_eudebug_types.h +32
-0@@ -105,6 +105,9 @@ struct xe_eudebug {/** @discovery_work: worker to discover resources for target_task */ struct work_struct discovery_work; +/** eu_lock: guards operations on eus (eu thread control and attention) */ +struct mutex eu_lock; +/** @events: kfifo queue of to-be-delivered events */ struct {/** @lock: guards access to fifo */ @@ -202,4 +205,33 @@ struct xe_eudebug_event_exec_queue { u64 lrc_handle[]; }; +/** +* struct xe_eudebug_event_eu_attention - Internal event for EU attention +*/ +struct xe_eudebug_event_eu_attention { +/** @base: base event */ +struct xe_eudebug_event base; + +/** @client_handle: client for the attention */ +u64 client_handle; + +/** @exec_queue_handle: handle of exec_queue which raised attention */ +u64 exec_queue_handle; + +/** @lrc_handle: lrc handle of the workload which raised attention */ +u64 lrc_handle; + +/** @flags: eu attention event flags, currently MBZ */ +u32 flags; + +/** @bitmask_size: size of the bitmask, specific to device */ +u32 bitmask_size; + +/** +* @bitmask: reflects threads currently signalling attention, +* starting from natural hardware order of DSS=0, eu=0 +*/ +u8 bitmask[]; +}; +#endif drivers/gpu/drm/xe/xe_gt_debug.c new file +18
-0@@ -0,0 +1,152 @@ +// SPDX-License-Identifier: MIT +/* +* Copyright © 2023 Intel Corporation +*/ + +#include "regs/xe_gt_regs.h" +#include "xe_device.h" +#include "xe_force_wake.h" +#include "xe_gt.h" +#include "xe_gt_topology.h" +#include "xe_gt_debug.h" +#include "xe_gt_mcr.h" +#include "xe_pm.h" +#include "xe_macros.h" + +static int xe_gt_foreach_dss_group_instance(struct xe_gt *gt, + int (*fn)(struct xe_gt *gt, + void *data,
drivers/gpu/drm/xe/xe_gt_debug.c line 28
Quoted 10 lines
drivers/gpu/drm/xe/xe_gt_debug.c new file +10
-0+u16 group, + u16 instance), + void *data) +{ + const enum xe_force_wake_domains fw_domains = XE_FW_GT | XE_FW_RENDER; +unsigned int dss; +u16 group, instance; +int ret; + +xe_pm_runtime_get (gt_to_xe (gt));
Missed this in my previous reply...
This looks pretty dangerous. The 'xe_pm_runtime_get' is designed to called on the outer most layers. This is an inner layer.
i.e. Calling xe_pm_runtime_get with any locks held is not allowed as it can wake the device and waking a device takes numerous locks which then deadlock. What makes this really scary is most times xe_pm_runtime_get doesn't wake the device, it just increments a counter so we have silent bug until xe_pm_runtime_get actually wakes the device.
A quick audit shows this is called with d->eu_lock, probably actually safe in case this but still not a good idea.
I suggest just protecting all of the EU debug code with xe_pm_runtime_get / xe_pm_runtime_put at the outer most layers (e.g. workers, IOCTLs already have this built in see xe_drm_ioctl).
Matt
Quoted 193 lines
drivers/gpu/drm/xe/xe_gt_debug.c new file +124
-0+ ret =xe_force_wake_get (gt_to_fw (gt), fw_domains); +if (ret) + goto pm_runtime_put; + +for_each_dss_steering (dss, gt, group, instance) { + ret =fn (gt, data, group, instance); +if (ret) +break ; + } + +xe_force_wake_put (gt_to_fw (gt), fw_domains); +pm_runtime_put: +xe_pm_runtime_put (gt_to_xe (gt)); + +return ret; +} + +static int read_first_attention_mcr (struct xe_gt *gt,void *data, +u16 group,u16 instance) +{ +unsigned int row; + +for (row =0 ; row <2 ; row++) { +u32 val; + + val =xe_gt_mcr_unicast_read (gt,TD_ATT (row), group, instance); + +if (val) +return 1 ; + } + +return 0 ; +} + +#define MAX_EUS_PER_ROW 4u +#define MAX_THREADS 8u + +/** +* xe_gt_eu_attention_bitmap_size - query size of the attention bitmask +* +* @gt: pointer to struct xe_gt +* +* Return: size in bytes. +*/ +int xe_gt_eu_attention_bitmap_size (struct xe_gt *gt) +{ +xe_dss_mask_t dss_mask; + +bitmap_or (dss_mask, gt->fuse_topo.c_dss_mask, + gt->fuse_topo.g_dss_mask,XE_MAX_DSS_FUSE_BITS ); + +return bitmap_weight (dss_mask,XE_MAX_DSS_FUSE_BITS ) * +TD_EU_ATTENTION_MAX_ROWS *MAX_THREADS * +MAX_EUS_PER_ROW /8 ; +} + +struct attn_read_iter { +struct xe_gt *gt; +unsigned int i; +unsigned int size; +u8 *bits; +}; + +static int read_eu_attentions_mcr (struct xe_gt *gt,void *data, +u16 group,u16 instance) +{ +struct attn_read_iter *const iter = data; +unsigned int row; + +for (row =0 ; row <TD_EU_ATTENTION_MAX_ROWS ; row++) { +u32 val; + +if (iter->i >= iter->size) +return 0 ; + +XE_WARN_ON (iter->i +sizeof (val) >xe_gt_eu_attention_bitmap_size (gt)); + + val =xe_gt_mcr_unicast_read (gt,TD_ATT (row), group, instance); + + +memcpy (&iter->bits[iter->i], &val,sizeof (val)); + iter->i +=sizeof (val); + } + +return 0 ; +} + +/** +* xe_gt_eu_attention_bitmap - query host attention +* +* @gt: pointer to struct xe_gt +* +* Return: 0 on success, negative otherwise. +*/ +int xe_gt_eu_attention_bitmap (struct xe_gt *gt,u8 *bits, +unsigned int bitmap_size) +{ +struct attn_read_iter iter = { + .gt = gt, + .i =0 , + .size = bitmap_size, + .bits = bits + }; + +return xe_gt_foreach_dss_group_instance (gt, read_eu_attentions_mcr, &iter); +} + +/** +* xe_gt_eu_threads_needing_attention - Query host attention +* +* @gt: pointer to struct xe_gt +* +* Return: 1 if threads waiting host attention, 0 otherwise. +*/ +int xe_gt_eu_threads_needing_attention (struct xe_gt *gt) +{ +int err; + + err =xe_gt_foreach_dss_group_instance (gt, read_first_attention_mcr,NULL ); + +XE_WARN_ON (err <0 ); + +return err <0 ?0 : err; +}drivers/gpu/drm/xe/xe_gt_debug.h new file +21
-0@@ -0,0 +1,21 @@ +/* SPDX-License-Identifier: MIT */ +/* +* Copyright © 2023 Intel Corporation +*/ + +#ifndef __XE_GT_DEBUG_ +#define __XE_GT_DEBUG_ + +#define TD_EU_ATTENTION_MAX_ROWS 2u + +#include "xe_gt_types.h" + +#define XE_GT_ATTENTION_TIMEOUT_MS 100 + +int xe_gt_eu_threads_needing_attention (struct xe_gt *gt); + +int xe_gt_eu_attention_bitmap_size (struct xe_gt *gt); +int xe_gt_eu_attention_bitmap (struct xe_gt *gt,u8 *bits, +unsigned int bitmap_size); + +#endif include/uapi/drm/xe_drm_eudebug.h +14
-1@@ -27,13 +27,15 @@ struct drm_xe_eudebug_event {#define DRM_XE_EUDEBUG_EVENT_OPEN 2#define DRM_XE_EUDEBUG_EVENT_VM 3#define DRM_XE_EUDEBUG_EVENT_EXEC_QUEUE 4 -#define DRM_XE_EUDEBUG_EVENT_MAX_EVENT DRM_XE_EUDEBUG_EVENT_EXEC_QUEUE +#define DRM_XE_EUDEBUG_EVENT_EU_ATTENTION 5 +#define DRM_XE_EUDEBUG_EVENT_MAX_EVENT DRM_XE_EUDEBUG_EVENT_EU_ATTENTION__u16 flags;#define DRM_XE_EUDEBUG_EVENT_CREATE (1 << 0)#define DRM_XE_EUDEBUG_EVENT_DESTROY (1 << 1)#define DRM_XE_EUDEBUG_EVENT_STATE_CHANGE (1 << 2)#define DRM_XE_EUDEBUG_EVENT_NEED_ACK (1 << 3) +__u64 seqno;__u64 reserved; };@@ -62,6 +64,17 @@ struct drm_xe_eudebug_event_exec_queue {__u64 lrc_handle[]; }; +struct drm_xe_eudebug_event_eu_attention { +struct drm_xe_eudebug_event base; + +__u64 client_handle; +__u64 exec_queue_handle; +__u64 lrc_handle; +__u32 flags; +__u32 bitmask_size; +__u8 bitmask[]; +}; +#if defined(__cplusplus) }#endif Signature · 1 line
2.34.1
PATCH 11/21 drm/xe/eudebug: Introduce EU control interface
In reply to Mika KuoppalaFrom: Dominik Grzegorzek <[email protected]>
Introduce EU control functionality, which allows EU debugger to interrupt, resume, and inform about the current state of EU threads during execution. Provide an abstraction layer, so in the future guc will only need to provide appropriate callbacks.
Based on implementation created by authors and other folks within i915 driver.
v2: checkpatch (Maciej)
v3: lrc index off by one fix (Mika)
--- drivers/gpu/drm/xe/regs/xe_gt_regs.h | 2 + drivers/gpu/drm/xe/xe_eudebug.c | 559 ++++++++++++++++++++++++-- drivers/gpu/drm/xe/xe_eudebug_types.h | 23 ++ drivers/gpu/drm/xe/xe_gt_debug.c | 12 +- drivers/gpu/drm/xe/xe_gt_debug.h | 6 + include/uapi/drm/xe_drm_eudebug.h | 21 +- 6 files changed, 586 insertions(+), 37 deletions(-)
drivers/gpu/drm/xe/regs/xe_gt_regs.h +2 -0
@@ -448,6 +448,8 @@#define THREAD_EX_ARB_MODE REG_GENMASK(3, 2)#define THREAD_EX_ARB_MODE_RR_AFTER_DEP REG_FIELD_PREP(THREAD_EX_ARB_MODE, 0x2) +#define TD_CLR (i) XE_REG_MCR(0xe490 + (i) * 4) +#define ROW_CHICKEN3 XE_REG_MCR(0xe49c, XE_REG_OPTION_MASKED)#define XE2_EUPEND_CHK_FLUSH_DIS REG_BIT(14)#define DIS_FIX_EOT1_FLUSH REG_BIT(9)
drivers/gpu/drm/xe/xe_eudebug.c +529 -30
658 lines not on this page · raw message
drivers/gpu/drm/xe/xe_eudebug_types.h +23 -0
@@ -18,8 +18,12 @@struct xe_device ;struct task_struct ; +struct xe_eudebug ;struct xe_eudebug_event ; +struct xe_hw_engine ;struct workqueue_struct ; +struct xe_exec_queue ; +struct xe_lrc ;#define CONFIG_DRM_XE_DEBUGGER_EVENT_QUEUE_SIZE 64
@@ -65,6 +69,23 @@ struct xe_eudebug_resources {struct xe_eudebug_resource rt[XE_EUDEBUG_RES_TYPE_COUNT ]; }; +/** +* struct xe_eudebug_eu_control_ops - interface for eu thread +* state control backend +*/ +struct xe_eudebug_eu_control_ops { +/** @interrupt_all: interrupts workload active on given hwe */ +int (*interrupt_all)(struct xe_eudebug *e,struct xe_exec_queue *q, +struct xe_lrc *lrc); + +/** @resume: resumes threads reflected by bitmask active on given hwe */ +int (*resume)(struct xe_eudebug *e,struct xe_exec_queue *q, +struct xe_lrc *lrc,u8 *bitmap,unsigned int bitmap_size); + +/** @stopped: returns bitmap reflecting threads which signal attention */ +int (*stopped)(struct xe_eudebug *e,struct xe_exec_queue *q, +struct xe_lrc *lrc,u8 *bitmap,unsigned int bitmap_size); +};/** * struct xe_eudebug - Top level struct for eudebug: the connection */
@@ -128,6 +149,8 @@ struct xe_eudebug {atomic_long_t seqno; } events; +/** @ops operations for eu_control */ +struct xe_eudebug_eu_control_ops *ops; };/**
drivers/gpu/drm/xe/xe_gt_debug.c +6 -6
@@ -13,12 +13,12 @@#include "xe_pm.h" #include "xe_macros.h" -static int xe_gt_foreach_dss_group_instance (struct xe_gt *gt, -int (*fn)(struct xe_gt *gt, -void *data, -u16 group, -u16 instance), -void *data) +int xe_gt_foreach_dss_group_instance (struct xe_gt *gt, +int (*fn)(struct xe_gt *gt, +void *data, +u16 group, +u16 instance), +void *data) {const enum xe_force_wake_domains fw_domains =XE_FW_GT |XE_FW_RENDER ;unsigned int dss;
drivers/gpu/drm/xe/xe_gt_debug.h +6 -0
@@ -13,6 +13,12 @@#define XE_GT_ATTENTION_TIMEOUT_MS 100int xe_gt_eu_threads_needing_attention (struct xe_gt *gt); +int xe_gt_foreach_dss_group_instance (struct xe_gt *gt, +int (*fn)(struct xe_gt *gt, +void *data, +u16 group, +u16 instance), +void *data);int xe_gt_eu_attention_bitmap_size (struct xe_gt *gt); int xe_gt_eu_attention_bitmap(struct xe_gt *gt, u8 *bits,
include/uapi/drm/xe_drm_eudebug.h +20 -1
@@ -15,7 +15,8 @@ extern "C" {* * This ioctl is available in debug version 1. */ -#define DRM_XE_EUDEBUG_IOCTL_READ_EVENT _IO('j', 0x0) +#define DRM_XE_EUDEBUG_IOCTL_READ_EVENT _IO('j', 0x0) +#define DRM_XE_EUDEBUG_IOCTL_EU_CONTROL _IOWR('j', 0x2, struct drm_xe_eudebug_eu_control)/* XXX: Document events to match their internal counterparts when moved to xe_drm.h */ struct drm_xe_eudebug_event {
@@ -75,6 +76,24 @@ struct drm_xe_eudebug_event_eu_attention {__u8 bitmask[]; }; +struct drm_xe_eudebug_eu_control { +__u64 client_handle; + +#define DRM_XE_EUDEBUG_EU_CONTROL_CMD_INTERRUPT_ALL 0 +#define DRM_XE_EUDEBUG_EU_CONTROL_CMD_STOPPED 1 +#define DRM_XE_EUDEBUG_EU_CONTROL_CMD_RESUME 2 +__u32 cmd; +__u32 flags; + +__u64 seqno; + +__u64 exec_queue_handle; +__u64 lrc_handle; +__u32 reserved; +__u32 bitmask_size; +__u64 bitmask_ptr; +}; +#if defined(__cplusplus) }#endif
Signature · 1 line
2.34.1
PATCH 12/21 drm/xe/eudebug: Add vm bind and vm bind ops
In reply to Mika KuoppalaFrom: Christoph Manszewski <[email protected]>
Add events dedicated to track vma bind and vma unbind operations. The events are generated for operations performed on xe_vma MAP and UNMAP for boss and userptrs.
As one bind can result in multiple operations and fail in the middle, we want to store the events until full successful chain of operations can be relayed to debugger.
v2: !bind_ref warning in wrong spot (Christoph, Maciej)
v3: vm bind flags
v4: _packed fixed in event_vm_bind (jcavitt)
v6: vmbind ops without map/unmaps handled (Mika)
v7: guard ref with spinlock (Mika)
v8: teardown events on vm destroy to avoid leak (Mika)
--- drivers/gpu/drm/xe/xe_eudebug.c | 297 ++++++++++++++++++++++++++ drivers/gpu/drm/xe/xe_eudebug.h | 7 + drivers/gpu/drm/xe/xe_eudebug_types.h | 29 +++ drivers/gpu/drm/xe/xe_vm.c | 18 +- drivers/gpu/drm/xe/xe_vm_types.h | 11 + include/uapi/drm/xe_drm_eudebug.h | 66 +++++- 6 files changed, 425 insertions(+), 3 deletions(-)
drivers/gpu/drm/xe/xe_eudebug.c +297 -0
@@ -2210,6 +2210,303 @@ void xe_eudebug_exec_queue_destroy(struct xe_file *xef, struct xe_exec_queue *q)xe_eudebug_event_put (d,exec_queue_destroy_event (d, xef, q)); } +static int xe_eudebug_queue_bind_event (struct xe_eudebug *d, +struct xe_vm *vm, +struct xe_eudebug_event *event) +{ +struct xe_eudebug_event_envelope *env; + +lockdep_assert_held_write (&vm->lock); + + env =kmalloc (sizeof (*env),GFP_KERNEL ); +if (!env) +return -ENOMEM ; + +INIT_LIST_HEAD (&env->link); + env->event = event; + +spin_lock (&vm->eudebug_bind.lock); +list_add_tail (&env->link, &vm->eudebug_bind.events); + +if (event->type ==DRM_XE_EUDEBUG_EVENT_VM_BIND_OP ) + ++vm->eudebug_bind.ops; +spin_unlock (&vm->eudebug_bind.lock); + +return 0 ; +} + +static int queue_vm_bind_event (struct xe_eudebug *d, +struct xe_vm *vm, +u64 client_handle, +u64 vm_handle, +u32 bind_flags, +u32 num_ops,u64 *seqno) +{ +struct xe_eudebug_event_vm_bind *e; +struct xe_eudebug_event *event; +const u32 sz =sizeof (*e); +const u32 base_flags =DRM_XE_EUDEBUG_EVENT_STATE_CHANGE ; + + *seqno =atomic_long_inc_return (&d->events.seqno); + + event =xe_eudebug_create_event (d,DRM_XE_EUDEBUG_EVENT_VM_BIND , + *seqno, base_flags, sz,GFP_KERNEL ); +if (!event) +return -ENOMEM ; + + e =cast_event (e, event); + write_member(struct drm_xe_eudebug_event_vm_bind, e, client_handle, client_handle); + write_member(struct drm_xe_eudebug_event_vm_bind, e, vm_handle, vm_handle); + write_member(struct drm_xe_eudebug_event_vm_bind, e, flags, bind_flags); + write_member(struct drm_xe_eudebug_event_vm_bind, e, num_binds, num_ops); + +/* If in discovery, no need to collect ops */ +if (!completion_done (&d->discovery)) { +XE_WARN_ON (!num_ops); +return xe_eudebug_queue_event (d, event); + } + +return xe_eudebug_queue_bind_event (d, vm, event); +} + +static int vm_bind_event (struct xe_eudebug *d, +struct xe_vm *vm, +u32 num_ops, +u64 *seqno) +{ +int h_c, h_vm; + + h_c =find_handle (d->res,XE_EUDEBUG_RES_TYPE_CLIENT , vm->xef); +if (h_c <0 ) +return h_c; + + h_vm =find_handle (d->res,XE_EUDEBUG_RES_TYPE_VM , vm); +if (h_vm <0 ) +return h_vm; + +return queue_vm_bind_event (d, vm, h_c, h_vm,0 , + num_ops, seqno); +} + +static int vm_bind_op_event (struct xe_eudebug *d, +struct xe_vm *vm, +const u32 flags, +const u64 bind_ref_seqno, +const u64 num_extensions, +u64 addr,u64 range, +u64 *op_seqno) +{ +struct xe_eudebug_event_vm_bind_op *e; +struct xe_eudebug_event *event; +const u32 sz =sizeof (*e); + + *op_seqno =atomic_long_inc_return (&d->events.seqno); + + event =xe_eudebug_create_event (d,DRM_XE_EUDEBUG_EVENT_VM_BIND_OP , + *op_seqno, flags, sz,GFP_KERNEL ); +if (!event) +return -ENOMEM ; + + e =cast_event (e, event); + + write_member(struct drm_xe_eudebug_event_vm_bind_op, e, vm_bind_ref_seqno, bind_ref_seqno); + write_member(struct drm_xe_eudebug_event_vm_bind_op, e, num_extensions, num_extensions); + write_member(struct drm_xe_eudebug_event_vm_bind_op, e, addr, addr); + write_member(struct drm_xe_eudebug_event_vm_bind_op, e, range, range); + +/* If in discovery, no need to collect ops */ +if (!completion_done (&d->discovery)) +return xe_eudebug_queue_event (d, event); + +return xe_eudebug_queue_bind_event (d, vm, event); +} + +static int vm_bind_op (struct xe_eudebug *d,struct xe_vm *vm, +const u32 flags,const u64 bind_ref_seqno, +u64 addr,u64 range) +{ +u64 op_seqno =0 ; +u64 num_extensions =0 ; +int ret; + + ret =vm_bind_op_event (d, vm, flags, bind_ref_seqno, num_extensions, + addr, range, &op_seqno); +if (ret) +return ret; + +return 0 ; +} + +void xe_eudebug_vm_bind_start (struct xe_vm *vm) +{ +struct xe_eudebug *d; +u64 seqno =0 ; +int err; + + d =xe_eudebug_get (vm->xef); +if (!d) +return ; + +lockdep_assert_held_write (&vm->lock); + +if (XE_WARN_ON (!list_empty (&vm->eudebug_bind.events)) || +XE_WARN_ON (vm->eudebug_bind.ops) || +XE_WARN_ON (vm->eudebug_bind.ref)) { +eu_err (d,"bind busy on %s" , __func__); +xe_eudebug_disconnect (d, -EINVAL ); + } + + err =vm_bind_event (d, vm,0 , &seqno); +if (err) { +eu_err (d,"error %d on %s" , err, __func__); +xe_eudebug_disconnect (d, err); + } + +spin_lock (&vm->eudebug_bind.lock); +XE_WARN_ON (vm->eudebug_bind.ref); + vm->eudebug_bind.ref = seqno; + vm->eudebug_bind.ops =0 ; +spin_unlock (&vm->eudebug_bind.lock); + +xe_eudebug_put (d); +} + +void xe_eudebug_vm_bind_op_add (struct xe_vm *vm,u32 op,u64 addr,u64 range) +{ +struct xe_eudebug *d; +u32 flags; + +switch (op) { +case DRM_XE_VM_BIND_OP_MAP : +case DRM_XE_VM_BIND_OP_MAP_USERPTR : + { + flags =DRM_XE_EUDEBUG_EVENT_CREATE ; +break ; + } +case DRM_XE_VM_BIND_OP_UNMAP : +case DRM_XE_VM_BIND_OP_UNMAP_ALL : + flags =DRM_XE_EUDEBUG_EVENT_DESTROY ; +break ; +default : + flags =0 ; +break ; + } + +if (!flags) +return ; + + d =xe_eudebug_get (vm->xef); +if (!d) +return ; + +xe_eudebug_event_put (d,vm_bind_op (d, vm, flags,0 , addr, range)); +} + +static struct xe_eudebug_event *fetch_bind_event (struct xe_vm *const vm) +{ +struct xe_eudebug_event_envelope *env; +struct xe_eudebug_event *e =NULL ; + +spin_lock (&vm->eudebug_bind.lock); + env =list_first_entry_or_null (&vm->eudebug_bind.events, + struct xe_eudebug_event_envelope, link); +if (env) { + e = env->event; +list_del (&env->link); + } +spin_unlock (&vm->eudebug_bind.lock); + +kfree (env); + +return e; +} + +static void fill_vm_bind_fields (struct xe_vm *vm, +struct xe_eudebug_event *e, +bool ufence, +u32 bind_ops) +{ +struct xe_eudebug_event_vm_bind *eb =cast_event (eb, e); + + eb->flags = ufence ? +DRM_XE_EUDEBUG_EVENT_VM_BIND_FLAG_UFENCE :0 ; + eb->num_binds = bind_ops; +} + +static void fill_vm_bind_op_fields (struct xe_vm *vm, +struct xe_eudebug_event *e, +u64 ref_seqno) +{ +struct xe_eudebug_event_vm_bind_op *op; + +if (e->type !=DRM_XE_EUDEBUG_EVENT_VM_BIND_OP ) +return ; + + op =cast_event (op, e); + op->vm_bind_ref_seqno = ref_seqno; +} + +void xe_eudebug_vm_bind_end (struct xe_vm *vm,bool has_ufence,int bind_err) +{ +struct xe_eudebug_event *e; +struct xe_eudebug *d; +u32 bind_ops; +u64 ref; + +spin_lock (&vm->eudebug_bind.lock); + ref = vm->eudebug_bind.ref; + vm->eudebug_bind.ref =0 ; + bind_ops = vm->eudebug_bind.ops; + vm->eudebug_bind.ops =0 ; +spin_unlock (&vm->eudebug_bind.lock); + + e =fetch_bind_event (vm); +if (!e) +return ; + + d =NULL ; +if (!bind_err && ref) { + d =xe_eudebug_get (vm->xef); +if (d) { +if (bind_ops) { +fill_vm_bind_fields (vm, e, has_ufence, bind_ops); + }else { +/* +* If there was no ops we are interested in, +* we can omit the whole sequence +*/ +xe_eudebug_put (d); + d =NULL ; + } + } + } + +while (e) { +int err =0 ; + +if (d) { + err =xe_eudebug_queue_event (d, e); +if (!err) + e =NULL ; + } + +if (err) { +xe_eudebug_disconnect (d, err); +xe_eudebug_put (d); + d =NULL ; + } + +kfree (e); + + e =fetch_bind_event (vm); +if (e && ref) +fill_vm_bind_op_fields (vm, e, ref); + } + +if (d) +xe_eudebug_put (d); +} +static int discover_client (struct xe_eudebug *d,struct xe_file *xef) {struct xe_exec_queue *q;
drivers/gpu/drm/xe/xe_eudebug.h +7 -0
@@ -5,11 +5,14 @@#ifndef _XE_EUDEBUG_H_ +#include <linux/types.h> +struct drm_device ;struct drm_file ;struct xe_device ;struct xe_file ;struct xe_vm ; +struct xe_vma ;struct xe_exec_queue ;struct xe_hw_engine ;
@@ -31,4 +34,8 @@ void xe_eudebug_vm_destroy(struct xe_file *xef, struct xe_vm *vm);void xe_eudebug_exec_queue_create (struct xe_file *xef,struct xe_exec_queue *q);void xe_eudebug_exec_queue_destroy (struct xe_file *xef,struct xe_exec_queue *q); +void xe_eudebug_vm_bind_start (struct xe_vm *vm); +void xe_eudebug_vm_bind_op_add (struct xe_vm *vm,u32 op,u64 addr,u64 range); +void xe_eudebug_vm_bind_end (struct xe_vm *vm,bool has_ufence,int err); +#endif
drivers/gpu/drm/xe/xe_eudebug_types.h +29 -0
@@ -176,6 +176,11 @@ struct xe_eudebug_event {u8 data[]; }; +struct xe_eudebug_event_envelope { +struct list_head link; +struct xe_eudebug_event *event; +}; +/** * struct xe_eudebug_event_open - Internal event for client open/close */
@@ -257,4 +262,28 @@ struct xe_eudebug_event_eu_attention { u8 bitmask[]; }; +/** +* struct xe_eudebug_event_vm_bind - Internal event for vm bind/unbind operation +*/ +struct xe_eudebug_event_vm_bind { +/** @base: base event */ +struct xe_eudebug_event base; + +u64 client_handle; +u64 vm_handle; + +u32 flags; +u32 num_binds; +}; + +struct xe_eudebug_event_vm_bind_op { +/** @base: base event */ +struct xe_eudebug_event base; +u64 vm_bind_ref_seqno; +u64 num_extensions; + +u64 addr;/* Zero for unmap all ? */ +u64 range;/* Zero for unmap all ? */ +}; +#endif
drivers/gpu/drm/xe/xe_vm.c +16 -2
@@ -1411,6 +1411,9 @@ struct xe_vm *xe_vm_create(struct xe_device *xe, u32 flags)for_each_tile (tile, xe, id)xe_range_fence_tree_init (&vm->rftree[id]); +INIT_LIST_HEAD (&vm->eudebug_bind.events); +spin_lock_init (&vm->eudebug_bind.lock); + vm->pt_ops = &xelp_pt_ops;/*
@@ -1656,6 +1659,8 @@ static void vm_destroy_work_func(struct work_struct *w)struct xe_tile *tile;u8 id; +xe_eudebug_vm_bind_end (vm,0 ,0 ); +/* xe_vm_close_and_put was not called? */ xe_assert (xe, !vm->size);
@@ -2675,7 +2680,7 @@ static void vm_bind_ioctl_ops_fini(struct xe_vm *vm, struct xe_vma_ops *vops, struct dma_fence *fence) {struct xe_exec_queue *wait_exec_queue =to_wait_exec_queue (vm, vops->q); -struct xe_user_fence *ufence; +struct xe_user_fence *ufence =NULL ;struct xe_vma_op *op;int i;
@@ -2690,6 +2695,9 @@ static void vm_bind_ioctl_ops_fini(struct xe_vm *vm, struct xe_vma_ops *vops,xe_vma_destroy (gpuva_to_vma (op->base.remap.unmap->va), fence); } + +xe_eudebug_vm_bind_end (vm, ufence,0 ); +if (ufence)xe_sync_ufence_put (ufence);for (i =0 ; i < vops->num_syncs; i++)
@@ -3067,6 +3075,7 @@ int xe_vm_bind_ioctl(struct drm_device *dev, void *data, struct drm_file *file) }xe_vma_ops_init (&vops, vm, q, syncs, num_syncs); +for (i =0 ; i < args->num_binds; ++i) {u64 range = bind_ops[i].range;u64 addr = bind_ops[i].addr;
@@ -3089,6 +3098,8 @@ int xe_vm_bind_ioctl(struct drm_device *dev, void *data, struct drm_file *file)if (err) goto unwind_ops; +xe_eudebug_vm_bind_op_add (vm, op, addr, range); +#ifdef TEST_VM_OPS_ERROR if (flags &FORCE_OP_ERROR ) { vops.inject_error = true;
@@ -3112,8 +3123,11 @@ int xe_vm_bind_ioctl(struct drm_device *dev, void *data, struct drm_file *file) err =vm_bind_ioctl_ops_execute (vm, &vops); unwind_ops: -if (err && err != -ENODATA ) +if (err && err != -ENODATA ) { +xe_eudebug_vm_bind_end (vm, num_ufence >0 , err);vm_bind_ioctl_ops_unwind (vm, ops, args->num_binds); + } +xe_vma_ops_fini (&vops);for (i = args->num_binds -1 ; i >=0 ; --i)if (ops[i])
drivers/gpu/drm/xe/xe_vm_types.h +11 -0
@@ -282,6 +282,17 @@ struct xe_vm {bool batch_invalidate_tlb;/** @xef: XE file handle for tracking this VM's drm client */ struct xe_file *xef; + +struct { +/** @lock: Lock for eudebug_bind members */ +spinlock_t lock; +/** @events: List of vm bind ops gathered */ +struct list_head events; +/** @ops: How many operations we have stored */ +u32 ops; +/** @ref: Reference to the VM_BIND that the ops relate */ +u64 ref; + } eudebug_bind; };/** struct xe_vma_op_map - VMA map operation */
include/uapi/drm/xe_drm_eudebug.h +65 -1
78 lines not on this page · raw message
Signature · 1 line
2.34.1
PATCH 13/21 drm/xe/eudebug: Add UFENCE events with acks
In reply to Mika KuoppalaWhen vma is in place, debugger needs to intercept before userspace proceeds with the workload. For example to install a breakpoint in a eu shader.
Attach debugger in xe_user_fence, send UFENCE event and stall normal user fence signal path to yield if there is debugger attached to ufence.
When ack (ioctl) is received for the corresponding seqno, signal ufence.
v2: ufence worker in own workqueue
--- drivers/gpu/drm/xe/xe_eudebug.c | 239 +++++++++++++++++++++++++- drivers/gpu/drm/xe/xe_eudebug.h | 6 + drivers/gpu/drm/xe/xe_eudebug_types.h | 12 ++ drivers/gpu/drm/xe/xe_exec.c | 2 +- drivers/gpu/drm/xe/xe_sync.c | 49 ++++-- drivers/gpu/drm/xe/xe_sync.h | 8 +- drivers/gpu/drm/xe/xe_sync_types.h | 26 ++- drivers/gpu/drm/xe/xe_vm.c | 4 +- include/uapi/drm/xe_drm_eudebug.h | 15 +- 9 files changed, 333 insertions(+), 28 deletions(-)
drivers/gpu/drm/xe/xe_eudebug.c +236 -3
@@ -35,6 +35,7 @@#include "xe_vm.h" #include "xe_wa.h" #include "xe_force_wake.h" +#include "xe_sync.h" /* * If there is no detected event read by userspace, during this period, assume
@@ -240,11 +241,115 @@ static void xe_eudebug_free(struct kref *ref)kfree_rcu (d, rcu); } -static void xe_eudebug_put (struct xe_eudebug *d) +void xe_eudebug_put (struct xe_eudebug *d) {kref_put (&d->ref, xe_eudebug_free); } +struct xe_eudebug_ack { +struct rb_node rb_node; +u64 seqno; +u64 ts_insert; +struct xe_user_fence *ufence; +}; + +#define fetch_ack (x) rb_entry(x, struct xe_eudebug_ack, rb_node) + +static int compare_ack (const u64 a,const u64 b) +{ +if (a < b) +return -1 ; +else if (a > b) +return 1 ; + +return 0 ; +} + +static int ack_insert_cmp (struct rb_node *const node, +const struct rb_node *const p) +{ +return compare_ack (fetch_ack (node)->seqno, +fetch_ack (p)->seqno); +} + +static int ack_lookup_cmp (const void *const key, +const struct rb_node *const node) +{ +return compare_ack (*(const u64 *)key, +fetch_ack (node)->seqno); +} + +static struct xe_eudebug_ack *remove_ack (struct xe_eudebug *d,u64 seqno) +{ +struct rb_root *const root = &d->acks.tree; +struct rb_node *node; + +spin_lock (&d->acks.lock); + node =rb_find (&seqno, root, ack_lookup_cmp); +if (node) +rb_erase (node, root); +spin_unlock (&d->acks.lock); + +if (!node) +return NULL ; + + return rb_entry_safe(node, struct xe_eudebug_ack, rb_node); +} + +static void ufence_signal_worker (struct work_struct *w) +{ +struct xe_user_fence *const ufence = + container_of(w, struct xe_user_fence, eudebug.worker); + +if (READ_ONCE (ufence->signalled)) +xe_sync_ufence_signal (ufence); + +xe_sync_ufence_put (ufence); +} + +static void kick_ufence_worker (struct xe_user_fence *f) +{ +INIT_WORK (&f->eudebug.worker, ufence_signal_worker); +queue_work (f->xe->eudebug.ordered_wq, &f->eudebug.worker); +} + +static void handle_ack (struct xe_eudebug *d,struct xe_eudebug_ack *ack, +bool on_disconnect) +{ +struct xe_user_fence *f = ack->ufence; +u64 signaller_ack, signalled_by; + + signaller_ack =cmpxchg64 (&f->eudebug.signalled_seqno,0 , ack->seqno); + signalled_by = f->eudebug.signalled_seqno; + +if (!signaller_ack) +kick_ufence_worker (f); +else +xe_sync_ufence_put (f); + +eu_dbg (d,"ACK: seqno=%llu: %ssignalled by %s (%llu) (held %lluus)" , + ack->seqno, signaller_ack ?"already " :"" , + on_disconnect ?"disconnect" :"debugger" , + signalled_by, +ktime_us_delta (ktime_get (), ack->ts_insert)); + +kfree (ack); +} + +static void release_acks (struct xe_eudebug *d) +{ +struct xe_eudebug_ack *ack, *n; +struct rb_root root; + +spin_lock (&d->acks.lock); + root = d->acks.tree; + d->acks.tree =RB_ROOT ; +spin_unlock (&d->acks.lock); + +rbtree_postorder_for_each_entry_safe (ack, n, &root, rb_node) +handle_ack (d, ack, true); +} +static struct task_struct *find_get_target (const pid_t nr) {struct task_struct *task;
@@ -328,6 +433,8 @@ static bool xe_eudebug_detach(struct xe_device *xe,eu_dbg (d,"session %lld detached with %d" , d->session, err); +release_acks (d); +/* Our ref with the connection_link */ xe_eudebug_put (d);
@@ -428,7 +535,7 @@ static struct task_struct *find_task_get(struct xe_file *xef)return task; } -static struct xe_eudebug * +struct xe_eudebug *xe_eudebug_get (struct xe_file *xef) {struct task_struct *task;
@@ -889,6 +996,44 @@ static long xe_eudebug_read_event(struct xe_eudebug *d,return ret; } +static long +xe_eudebug_ack_event_ioctl (struct xe_eudebug *d, +const unsigned int cmd, +const u64 arg) +{ +struct drm_xe_eudebug_ack_event __user *const user_ptr = +u64_to_user_ptr (arg); +struct drm_xe_eudebug_ack_event user_arg; +struct xe_eudebug_ack *ack; +struct xe_device *xe = d->xe; + +if (XE_IOCTL_DBG (xe,_IOC_SIZE (cmd) <sizeof (user_arg))) +return -EINVAL ; + +/* Userland write */ +if (XE_IOCTL_DBG (xe, !(_IOC_DIR (cmd) & _IOC_WRITE))) +return -EINVAL ; + +if (XE_IOCTL_DBG (xe,copy_from_user (&user_arg, + user_ptr, +sizeof (user_arg)))) +return -EFAULT ; + +if (XE_IOCTL_DBG (xe, user_arg.flags)) +return -EINVAL ; + +if (XE_IOCTL_DBG (xe,xe_eudebug_detached (d))) +return -ENOTCONN ; + + ack =remove_ack (d, user_arg.seqno); +if (XE_IOCTL_DBG (xe, !ack)) +return -EINVAL ; + +handle_ack (d, ack, false); + +return 0 ; +} +static int do_eu_control (struct xe_eudebug *d,const struct drm_xe_eudebug_eu_control *const arg,struct drm_xe_eudebug_eu_control __user *const user_ptr)
@@ -1070,7 +1215,10 @@ static long xe_eudebug_ioctl(struct file *file, ret =xe_eudebug_eu_control (d, arg);eu_dbg (d,"ioctl cmd=EU_CONTROL ret=%ld\n" , ret);break ; - +case DRM_XE_EUDEBUG_IOCTL_ACK_EVENT : + ret =xe_eudebug_ack_event_ioctl (d, cmd, arg); +eu_dbg (d,"ioctl cmd=EVENT_ACK ret=%ld\n" , ret); +break ;default : ret = -EINVAL ; }
@@ -1759,6 +1907,9 @@ xe_eudebug_connect(struct xe_device *xe,INIT_KFIFO (d->events.fifo);INIT_WORK (&d->discovery_work, discovery_work_fn); +spin_lock_init (&d->acks.lock); + d->acks.tree =RB_ROOT ; + d->res =xe_eudebug_resources_alloc ();if (IS_ERR (d->res)) { err =PTR_ERR (d->res);
@@ -2337,6 +2488,70 @@ static int vm_bind_op(struct xe_eudebug *d, struct xe_vm *vm,return 0 ; } +static int xe_eudebug_track_ufence (struct xe_eudebug *d, +struct xe_user_fence *f, +u64 seqno) +{ +struct xe_eudebug_ack *ack; +struct rb_node *old; + + ack =kzalloc (sizeof (*ack),GFP_KERNEL ); +if (!ack) +return -ENOMEM ; + + ack->seqno = seqno; + ack->ts_insert =ktime_get (); + +spin_lock (&d->acks.lock); + old =rb_find_add (&ack->rb_node, + &d->acks.tree, ack_insert_cmp); +if (!old) { +kref_get (&f->refcount); + ack->ufence = f; + } +spin_unlock (&d->acks.lock); + +if (old) { +eu_dbg (d,"ACK: seqno=%llu: already exists" , seqno); +kfree (ack); +return -EEXIST ; + } + +eu_dbg (d,"ACK: seqno=%llu: tracking started" , seqno); + +return 0 ; +} + +static int vm_bind_ufence_event (struct xe_eudebug *d, +struct xe_user_fence *ufence) +{ +struct xe_eudebug_event *event; +struct xe_eudebug_event_vm_bind_ufence *e; +const u32 sz =sizeof (*e); +const u32 flags =DRM_XE_EUDEBUG_EVENT_CREATE | +DRM_XE_EUDEBUG_EVENT_NEED_ACK ; +u64 seqno; +int ret; + + seqno =atomic_long_inc_return (&d->events.seqno); + + event =xe_eudebug_create_event (d,DRM_XE_EUDEBUG_EVENT_VM_BIND_UFENCE , + seqno, flags, sz,GFP_KERNEL ); +if (!event) +return -ENOMEM ; + + e =cast_event (e, event); + + write_member(struct drm_xe_eudebug_event_vm_bind_ufence, + e, vm_bind_ref_seqno, ufence->eudebug.bind_ref_seqno); + + ret =xe_eudebug_track_ufence (d, ufence, seqno); +if (!ret) + ret =xe_eudebug_queue_event (d, event); + +return ret; +} +void xe_eudebug_vm_bind_start (struct xe_vm *vm) {struct xe_eudebug *d;
@@ -2507,6 +2722,24 @@ void xe_eudebug_vm_bind_end(struct xe_vm *vm, bool has_ufence, int bind_err)xe_eudebug_put (d); } +int xe_eudebug_vm_bind_ufence (struct xe_user_fence *ufence) +{ +struct xe_eudebug *d; +int err; + + d = ufence->eudebug.debugger; +if (!d ||xe_eudebug_detached (d)) +return -ENOTCONN ; + + err =vm_bind_ufence_event (d, ufence); +if (err) { +eu_err (d,"error %d on %s" , err, __func__); +xe_eudebug_disconnect (d, err); + } + +return 0 ; +} +static int discover_client (struct xe_eudebug *d,struct xe_file *xef) {struct xe_exec_queue *q;
drivers/gpu/drm/xe/xe_eudebug.h +6 -0
@@ -15,6 +15,7 @@ struct xe_vm;struct xe_vma ;struct xe_exec_queue ;struct xe_hw_engine ; +struct xe_user_fence ; int xe_eudebug_connect_ioctl(struct drm_device *dev, void *data,
@@ -38,4 +39,9 @@ void xe_eudebug_vm_bind_start(struct xe_vm *vm);void xe_eudebug_vm_bind_op_add (struct xe_vm *vm,u32 op,u64 addr,u64 range);void xe_eudebug_vm_bind_end (struct xe_vm *vm,bool has_ufence,int err); +int xe_eudebug_vm_bind_ufence (struct xe_user_fence *ufence); + +struct xe_eudebug *xe_eudebug_get (struct xe_file *xef); +void xe_eudebug_put (struct xe_eudebug *d); +#endif
drivers/gpu/drm/xe/xe_eudebug_types.h +12 -0
@@ -86,6 +86,7 @@ struct xe_eudebug_eu_control_ops {int (*stopped)(struct xe_eudebug *e,struct xe_exec_queue *q,struct xe_lrc *lrc,u8 *bitmap,unsigned int bitmap_size); }; +/** * struct xe_eudebug - Top level struct for eudebug: the connection */
@@ -149,6 +150,12 @@ struct xe_eudebug {atomic_long_t seqno; } events; +/* user fences tracked by this debugger */ +struct { +spinlock_t lock; +struct rb_root tree; + } acks; +/** @ops operations for eu_control */ struct xe_eudebug_eu_control_ops *ops; };
@@ -286,4 +293,9 @@ struct xe_eudebug_event_vm_bind_op { u64 range; /* Zero for unmap all ? */ }; +struct xe_eudebug_event_vm_bind_ufence { +struct xe_eudebug_event base; +u64 vm_bind_ref_seqno; +}; +#endif
drivers/gpu/drm/xe/xe_exec.c +1 -1
@@ -157,7 +157,7 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file) vm = q->vm;for (num_syncs =0 ; num_syncs < args->num_syncs; num_syncs++) { - err =xe_sync_entry_parse (xe, xef, &syncs[num_syncs], + err =xe_sync_entry_parse (xe, xef, vm, &syncs[num_syncs], &syncs_user[num_syncs],SYNC_PARSE_FLAG_EXEC | (xe_vm_in_lr_mode (vm) ?SYNC_PARSE_FLAG_LR_MODE :0 ));
drivers/gpu/drm/xe/xe_sync.c +33 -16
102 lines not on this page · raw message
drivers/gpu/drm/xe/xe_sync.h +7 -1
@@ -9,8 +9,12 @@#include "xe_sync_types.h" struct xe_device ; -struct xe_exec_queue ;struct xe_file ; +struct xe_exec_queue ; +struct drm_syncobj ; +struct dma_fence ; +struct dma_fence_chain ; +struct drm_xe_sync ;struct xe_sched_job ;struct xe_vm ;
@@ -19,6 +23,7 @@ struct xe_vm;#define SYNC_PARSE_FLAG_DISALLOW_USER_FENCE BIT(2)int xe_sync_entry_parse (struct xe_device *xe,struct xe_file *xef, +struct xe_vm *vm,struct xe_sync_entry *sync,struct drm_xe_sync __user *sync_user,unsigned int flags);
@@ -40,5 +45,6 @@ struct xe_user_fence *__xe_sync_ufence_get(struct xe_user_fence *ufence);struct xe_user_fence *xe_sync_ufence_get (struct xe_sync_entry *sync);void xe_sync_ufence_put (struct xe_user_fence *ufence);int xe_sync_ufence_get_status (struct xe_user_fence *ufence); +void xe_sync_ufence_signal (struct xe_user_fence *ufence);#endif
drivers/gpu/drm/xe/xe_sync_types.h +21 -5
@@ -7,12 +7,28 @@#define _XE_SYNC_TYPES_H_#include <linux/types.h> +#include <linux/spinlock.h> +#include <linux/kref.h> +#include <linux/dma-fence-array.h> -struct drm_syncobj ; -struct dma_fence ; -struct dma_fence_chain ; -struct drm_xe_sync ; -struct user_fence ; +struct xe_eudebug ; + +struct xe_user_fence { +struct xe_device *xe; +struct kref refcount; +struct dma_fence_cb cb; +struct work_struct worker; +struct mm_struct *mm; +u64 __user *addr; +u64 value; +int signalled; +struct { +struct xe_eudebug *debugger; +u64 bind_ref_seqno; +u64 signalled_seqno; +struct work_struct worker; + } eudebug; +};struct xe_sync_entry {struct drm_syncobj *syncobj;
drivers/gpu/drm/xe/xe_vm.c +3 -1
@@ -3049,9 +3049,11 @@ int xe_vm_bind_ioctl(struct drm_device *dev, void *data, struct drm_file *file) } } +xe_eudebug_vm_bind_start (vm); + syncs_user =u64_to_user_ptr (args->syncs);for (num_syncs =0 ; num_syncs < args->num_syncs; num_syncs++) { - err = xe_sync_entry_parse(xe, xef, &syncs[num_syncs], + err = xe_sync_entry_parse(xe, xef, vm, &syncs[num_syncs], &syncs_user[num_syncs], (xe_vm_in_lr_mode(vm) ? SYNC_PARSE_FLAG_LR_MODE : 0) |
include/uapi/drm/xe_drm_eudebug.h +14 -1
@@ -17,6 +17,7 @@ extern "C" {*/ #define DRM_XE_EUDEBUG_IOCTL_READ_EVENT _IO('j', 0x0)#define DRM_XE_EUDEBUG_IOCTL_EU_CONTROL _IOWR('j', 0x2, struct drm_xe_eudebug_eu_control) +#define DRM_XE_EUDEBUG_IOCTL_ACK_EVENT _IOW('j', 0x4, struct drm_xe_eudebug_ack_event)/* XXX: Document events to match their internal counterparts when moved to xe_drm.h */ struct drm_xe_eudebug_event {
@@ -31,7 +32,8 @@ struct drm_xe_eudebug_event {#define DRM_XE_EUDEBUG_EVENT_EU_ATTENTION 5#define DRM_XE_EUDEBUG_EVENT_VM_BIND 6#define DRM_XE_EUDEBUG_EVENT_VM_BIND_OP 7 -#define DRM_XE_EUDEBUG_EVENT_MAX_EVENT DRM_XE_EUDEBUG_EVENT_VM_BIND_OP +#define DRM_XE_EUDEBUG_EVENT_VM_BIND_UFENCE 8 +#define DRM_XE_EUDEBUG_EVENT_MAX_EVENT DRM_XE_EUDEBUG_EVENT_VM_BIND_UFENCE__u16 flags;#define DRM_XE_EUDEBUG_EVENT_CREATE (1 << 0)
@@ -158,6 +160,17 @@ struct drm_xe_eudebug_event_vm_bind_op {__u64 range;/* XXX: Zero for unmap all? */ }; +struct drm_xe_eudebug_event_vm_bind_ufence { +struct drm_xe_eudebug_event base; +__u64 vm_bind_ref_seqno;/* *_event_vm_bind.base.seqno */ +}; + +struct drm_xe_eudebug_ack_event { +__u32 type; +__u32 flags;/* MBZ */ +__u64 seqno; +}; +#if defined(__cplusplus) }#endif
Signature · 1 line
2.34.1
Matthew Brost
maintainerRe: [PATCH 13/21] drm/xe/eudebug: Add UFENCE events with acks
Quoted 4 lines · Mika Kuoppala
When vma is in place, debugger needs to intercept before userspace proceeds with the workload. For example to install a breakpoint in a eu shader.
Quoted 9 lines
Attach debugger in
xe_user_fence, send UFENCE event and stall normal user fence signal path to yield if there is debugger attached to ufence.When ack (ioctl) is received for the corresponding seqno, signal ufence.
v2: ufence worker in own workqueue
Not a complete review again, just a couple of quick comments.
Quoted 28 lines
Signed-off-by Mika Kuoppala <[email protected]>--- drivers/gpu/drm/xe/xe_eudebug.c | 239 +++++++++++++++++++++++++- drivers/gpu/drm/xe/xe_eudebug.h | 6 + drivers/gpu/drm/xe/xe_eudebug_types.h | 12 ++ drivers/gpu/drm/xe/xe_exec.c | 2 +- drivers/gpu/drm/xe/xe_sync.c | 49 ++++-- drivers/gpu/drm/xe/xe_sync.h | 8 +- drivers/gpu/drm/xe/xe_sync_types.h | 26 ++- drivers/gpu/drm/xe/xe_vm.c | 4 +- include/uapi/drm/xe_drm_eudebug.h | 15 +- 9 files changed, 333 insertions(+), 28 deletions(-)drivers/gpu/drm/xe/xe_eudebug.c +1
-0@@ -35,6 +35,7 @@#include "xe_vm.h" #include "xe_wa.h" #include "xe_force_wake.h" +#include "xe_sync.h" /* * If there is no detected event read by userspace, during this period, assume @@ -240,11 +241,115 @@ static void xe_eudebug_free(struct kref *ref)kfree_rcu (d, rcu); }
drivers/gpu/drm/xe/xe_eudebug.c static void xe_eudebug_free(struct kref *ref) · line 251
Quoted 10 lines
drivers/gpu/drm/xe/xe_eudebug.c +4
-1-static void xe_eudebug_put (struct xe_eudebug *d) +void xe_eudebug_put (struct xe_eudebug *d) {kref_put (&d->ref, xe_eudebug_free); } +struct xe_eudebug_ack { +struct rb_node rb_node; +u64 seqno;
Here we have a seqno but then an RB tree for search which is a bit incongruent. Let me explain, seqno typically mean something which signals sequentially in order. But then you have RB tree for searching, if everything signaled in order a list would be sufficient.
So without looking too much I think we want one of two things.
1. seqno + list
2. key + rb tree
Make sense?
Kinda a nit but naming matters in this case as a name implies in-order vs out-of-order signaling.
Matt
Quoted 600 lines
drivers/gpu/drm/xe/xe_eudebug.c +231
-2+u64 ts_insert; +struct xe_user_fence *ufence; +}; + +#define fetch_ack (x) rb_entry(x, struct xe_eudebug_ack, rb_node) + +static int compare_ack (const u64 a,const u64 b) +{ +if (a < b) +return -1 ; +else if (a > b) +return 1 ; + +return 0 ; +} + +static int ack_insert_cmp (struct rb_node *const node, +const struct rb_node *const p) +{ +return compare_ack (fetch_ack (node)->seqno, +fetch_ack (p)->seqno); +} + +static int ack_lookup_cmp (const void *const key, +const struct rb_node *const node) +{ +return compare_ack (*(const u64 *)key, +fetch_ack (node)->seqno); +} + +static struct xe_eudebug_ack *remove_ack (struct xe_eudebug *d,u64 seqno) +{ +struct rb_root *const root = &d->acks.tree; +struct rb_node *node; + +spin_lock (&d->acks.lock); + node =rb_find (&seqno, root, ack_lookup_cmp); +if (node) +rb_erase (node, root); +spin_unlock (&d->acks.lock); + +if (!node) +return NULL ; + + return rb_entry_safe(node, struct xe_eudebug_ack, rb_node); +} + +static void ufence_signal_worker (struct work_struct *w) +{ +struct xe_user_fence *const ufence = + container_of(w, struct xe_user_fence, eudebug.worker); + +if (READ_ONCE (ufence->signalled)) +xe_sync_ufence_signal (ufence); + +xe_sync_ufence_put (ufence); +} + +static void kick_ufence_worker (struct xe_user_fence *f) +{ +INIT_WORK (&f->eudebug.worker, ufence_signal_worker); +queue_work (f->xe->eudebug.ordered_wq, &f->eudebug.worker); +} + +static void handle_ack (struct xe_eudebug *d,struct xe_eudebug_ack *ack, +bool on_disconnect) +{ +struct xe_user_fence *f = ack->ufence; +u64 signaller_ack, signalled_by; + + signaller_ack =cmpxchg64 (&f->eudebug.signalled_seqno,0 , ack->seqno); + signalled_by = f->eudebug.signalled_seqno; + +if (!signaller_ack) +kick_ufence_worker (f); +else +xe_sync_ufence_put (f); + +eu_dbg (d,"ACK: seqno=%llu: %ssignalled by %s (%llu) (held %lluus)" , + ack->seqno, signaller_ack ?"already " :"" , + on_disconnect ?"disconnect" :"debugger" , + signalled_by, +ktime_us_delta (ktime_get (), ack->ts_insert)); + +kfree (ack); +} + +static void release_acks (struct xe_eudebug *d) +{ +struct xe_eudebug_ack *ack, *n; +struct rb_root root; + +spin_lock (&d->acks.lock); + root = d->acks.tree; + d->acks.tree =RB_ROOT ; +spin_unlock (&d->acks.lock); + +rbtree_postorder_for_each_entry_safe (ack, n, &root, rb_node) +handle_ack (d, ack, true); +} +static struct task_struct *find_get_target (const pid_t nr) {struct task_struct *task;@@ -328,6 +433,8 @@ static bool xe_eudebug_detach(struct xe_device *xe,eu_dbg (d,"session %lld detached with %d" , d->session, err); +release_acks (d); +/* Our ref with the connection_link */ xe_eudebug_put (d);@@ -428,7 +535,7 @@ static struct task_struct *find_task_get(struct xe_file *xef)return task; } -static struct xe_eudebug * +struct xe_eudebug *xe_eudebug_get (struct xe_file *xef) {struct task_struct *task;@@ -889,6 +996,44 @@ static long xe_eudebug_read_event(struct xe_eudebug *d,return ret; } +static long +xe_eudebug_ack_event_ioctl (struct xe_eudebug *d, +const unsigned int cmd, +const u64 arg) +{ +struct drm_xe_eudebug_ack_event __user *const user_ptr = +u64_to_user_ptr (arg); +struct drm_xe_eudebug_ack_event user_arg; +struct xe_eudebug_ack *ack; +struct xe_device *xe = d->xe; + +if (XE_IOCTL_DBG (xe,_IOC_SIZE (cmd) <sizeof (user_arg))) +return -EINVAL ; + +/* Userland write */ +if (XE_IOCTL_DBG (xe, !(_IOC_DIR (cmd) & _IOC_WRITE))) +return -EINVAL ; + +if (XE_IOCTL_DBG (xe,copy_from_user (&user_arg, + user_ptr, +sizeof (user_arg)))) +return -EFAULT ; + +if (XE_IOCTL_DBG (xe, user_arg.flags)) +return -EINVAL ; + +if (XE_IOCTL_DBG (xe,xe_eudebug_detached (d))) +return -ENOTCONN ; + + ack =remove_ack (d, user_arg.seqno); +if (XE_IOCTL_DBG (xe, !ack)) +return -EINVAL ; + +handle_ack (d, ack, false); + +return 0 ; +} +static int do_eu_control (struct xe_eudebug *d,const struct drm_xe_eudebug_eu_control *const arg,struct drm_xe_eudebug_eu_control __user *const user_ptr)@@ -1070,7 +1215,10 @@ static long xe_eudebug_ioctl(struct file *file, ret =xe_eudebug_eu_control (d, arg);eu_dbg (d,"ioctl cmd=EU_CONTROL ret=%ld\n" , ret);break ; - +case DRM_XE_EUDEBUG_IOCTL_ACK_EVENT : + ret =xe_eudebug_ack_event_ioctl (d, cmd, arg); +eu_dbg (d,"ioctl cmd=EVENT_ACK ret=%ld\n" , ret); +break ;default : ret = -EINVAL ; }@@ -1759,6 +1907,9 @@ xe_eudebug_connect(struct xe_device *xe,INIT_KFIFO (d->events.fifo);INIT_WORK (&d->discovery_work, discovery_work_fn); +spin_lock_init (&d->acks.lock); + d->acks.tree =RB_ROOT ; + d->res =xe_eudebug_resources_alloc ();if (IS_ERR (d->res)) { err =PTR_ERR (d->res);@@ -2337,6 +2488,70 @@ static int vm_bind_op(struct xe_eudebug *d, struct xe_vm *vm,return 0 ; } +static int xe_eudebug_track_ufence (struct xe_eudebug *d, +struct xe_user_fence *f, +u64 seqno) +{ +struct xe_eudebug_ack *ack; +struct rb_node *old; + + ack =kzalloc (sizeof (*ack),GFP_KERNEL ); +if (!ack) +return -ENOMEM ; + + ack->seqno = seqno; + ack->ts_insert =ktime_get (); + +spin_lock (&d->acks.lock); + old =rb_find_add (&ack->rb_node, + &d->acks.tree, ack_insert_cmp); +if (!old) { +kref_get (&f->refcount); + ack->ufence = f; + } +spin_unlock (&d->acks.lock); + +if (old) { +eu_dbg (d,"ACK: seqno=%llu: already exists" , seqno); +kfree (ack); +return -EEXIST ; + } + +eu_dbg (d,"ACK: seqno=%llu: tracking started" , seqno); + +return 0 ; +} + +static int vm_bind_ufence_event (struct xe_eudebug *d, +struct xe_user_fence *ufence) +{ +struct xe_eudebug_event *event; +struct xe_eudebug_event_vm_bind_ufence *e; +const u32 sz =sizeof (*e); +const u32 flags =DRM_XE_EUDEBUG_EVENT_CREATE | +DRM_XE_EUDEBUG_EVENT_NEED_ACK ; +u64 seqno; +int ret; + + seqno =atomic_long_inc_return (&d->events.seqno); + + event =xe_eudebug_create_event (d,DRM_XE_EUDEBUG_EVENT_VM_BIND_UFENCE , + seqno, flags, sz,GFP_KERNEL ); +if (!event) +return -ENOMEM ; + + e =cast_event (e, event); + + write_member(struct drm_xe_eudebug_event_vm_bind_ufence, + e, vm_bind_ref_seqno, ufence->eudebug.bind_ref_seqno); + + ret =xe_eudebug_track_ufence (d, ufence, seqno); +if (!ret) + ret =xe_eudebug_queue_event (d, event); + +return ret; +} +void xe_eudebug_vm_bind_start (struct xe_vm *vm) {struct xe_eudebug *d;@@ -2507,6 +2722,24 @@ void xe_eudebug_vm_bind_end(struct xe_vm *vm, bool has_ufence, int bind_err)xe_eudebug_put (d); } +int xe_eudebug_vm_bind_ufence (struct xe_user_fence *ufence) +{ +struct xe_eudebug *d; +int err; + + d = ufence->eudebug.debugger; +if (!d ||xe_eudebug_detached (d)) +return -ENOTCONN ; + + err =vm_bind_ufence_event (d, ufence); +if (err) { +eu_err (d,"error %d on %s" , err, __func__); +xe_eudebug_disconnect (d, err); + } + +return 0 ; +} +static int discover_client (struct xe_eudebug *d,struct xe_file *xef) {struct xe_exec_queue *q;drivers/gpu/drm/xe/xe_eudebug.h +6
-0@@ -15,6 +15,7 @@ struct xe_vm;struct xe_vma ;struct xe_exec_queue ;struct xe_hw_engine ; +struct xe_user_fence ; int xe_eudebug_connect_ioctl(struct drm_device *dev, void *data,@@ -38,4 +39,9 @@ void xe_eudebug_vm_bind_start(struct xe_vm *vm);void xe_eudebug_vm_bind_op_add (struct xe_vm *vm,u32 op,u64 addr,u64 range);void xe_eudebug_vm_bind_end (struct xe_vm *vm,bool has_ufence,int err); +int xe_eudebug_vm_bind_ufence (struct xe_user_fence *ufence); + +struct xe_eudebug *xe_eudebug_get (struct xe_file *xef); +void xe_eudebug_put (struct xe_eudebug *d); +#endif drivers/gpu/drm/xe/xe_eudebug_types.h +12
-0@@ -86,6 +86,7 @@ struct xe_eudebug_eu_control_ops {int (*stopped)(struct xe_eudebug *e,struct xe_exec_queue *q,struct xe_lrc *lrc,u8 *bitmap,unsigned int bitmap_size); }; +/** * struct xe_eudebug - Top level struct for eudebug: the connection */ @@ -149,6 +150,12 @@ struct xe_eudebug {atomic_long_t seqno; } events; +/* user fences tracked by this debugger */ +struct { +spinlock_t lock; +struct rb_root tree; + } acks; +/** @ops operations for eu_control */ struct xe_eudebug_eu_control_ops *ops; };@@ -286,4 +293,9 @@ struct xe_eudebug_event_vm_bind_op { u64 range; /* Zero for unmap all ? */ }; +struct xe_eudebug_event_vm_bind_ufence { +struct xe_eudebug_event base; +u64 vm_bind_ref_seqno; +}; +#endif drivers/gpu/drm/xe/xe_exec.c +1
-1@@ -157,7 +157,7 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file) vm = q->vm;for (num_syncs =0 ; num_syncs < args->num_syncs; num_syncs++) { - err =xe_sync_entry_parse (xe, xef, &syncs[num_syncs], + err =xe_sync_entry_parse (xe, xef, vm, &syncs[num_syncs], &syncs_user[num_syncs],SYNC_PARSE_FLAG_EXEC | (xe_vm_in_lr_mode (vm) ?SYNC_PARSE_FLAG_LR_MODE :0 ));drivers/gpu/drm/xe/xe_sync.c +33
-16@@ -18,17 +18,7 @@#include "xe_exec_queue.h" #include "xe_macros.h" #include "xe_sched_job_types.h" - -struct xe_user_fence { -struct xe_device *xe; -struct kref refcount; -struct dma_fence_cb cb; -struct work_struct worker; -struct mm_struct *mm; -u64 __user *addr; -u64 value; -int signalled; -}; +#include "xe_eudebug.h" static void user_fence_destroy (struct kref *kref) {@@ -36,6 +26,10 @@ static void user_fence_destroy(struct kref *kref) refcount);mmdrop (ufence->mm); + +if (ufence->eudebug.debugger) +xe_eudebug_put (ufence->eudebug.debugger); +kfree (ufence); }@@ -49,7 +43,10 @@ static void user_fence_put(struct xe_user_fence *ufence)kref_put (&ufence->refcount, user_fence_destroy); } -static struct xe_user_fence *user_fence_create (struct xe_device *xe,u64 addr, +static struct xe_user_fence *user_fence_create (struct xe_device *xe, +struct xe_file *xef, +struct xe_vm *vm, +u64 addr,u64 value) {struct xe_user_fence *ufence;@@ -58,7 +55,7 @@ static struct xe_user_fence *user_fence_create(struct xe_device *xe, u64 addr,if (!access_ok (ptr,sizeof (ptr)))return ERR_PTR (-EFAULT ); - ufence =kmalloc (sizeof (*ufence),GFP_KERNEL ); + ufence =kzalloc (sizeof (*ufence),GFP_KERNEL );if (!ufence)return ERR_PTR (-ENOMEM );@@ -69,12 +66,17 @@ static struct xe_user_fence *user_fence_create(struct xe_device *xe, u64 addr, ufence->mm = current->mm;mmgrab (ufence->mm); +if (vm->eudebug_bind.ref) { + ufence->eudebug.debugger =xe_eudebug_get (xef); + ufence->eudebug.bind_ref_seqno = vm->eudebug_bind.ref; + } +return ufence; } -static void user_fence_worker (struct work_struct *w) +void xe_sync_ufence_signal (struct xe_user_fence *ufence) { - struct xe_user_fence *ufence = container_of(w, struct xe_user_fence, worker); +XE_WARN_ON (!ufence->signalled);if (mmget_not_zero (ufence->mm)) {kthread_use_mm (ufence->mm);@@ -85,7 +87,20 @@ static void user_fence_worker(struct work_struct *w) }wake_up_all (&ufence->xe->ufence_wq); +} + +static void user_fence_worker (struct work_struct *w) +{ + struct xe_user_fence *ufence = container_of(w, struct xe_user_fence, worker); +int ret; +WRITE_ONCE (ufence->signalled,1 ); + +/* Lets see if debugger wants to track this */ + ret =xe_eudebug_vm_bind_ufence (ufence); +if (ret) +xe_sync_ufence_signal (ufence); +user_fence_put (ufence); }@@ -104,6 +119,7 @@ static void user_fence_cb(struct dma_fence *fence, struct dma_fence_cb *cb) }int xe_sync_entry_parse (struct xe_device *xe,struct xe_file *xef, +struct xe_vm *vm,struct xe_sync_entry *sync,struct drm_xe_sync __user *sync_user,unsigned int flags)@@ -185,7 +201,8 @@ int xe_sync_entry_parse(struct xe_device *xe, struct xe_file *xef,if (exec) { sync->addr = sync_in.addr; }else { - sync->ufence =user_fence_create (xe, sync_in.addr, + sync->ufence =user_fence_create (xe, xef, vm, + sync_in.addr, sync_in.timeline_value);if (XE_IOCTL_DBG (xe,IS_ERR (sync->ufence)))return PTR_ERR (sync->ufence);drivers/gpu/drm/xe/xe_sync.h +7
-1@@ -9,8 +9,12 @@#include "xe_sync_types.h" struct xe_device ; -struct xe_exec_queue ;struct xe_file ; +struct xe_exec_queue ; +struct drm_syncobj ; +struct dma_fence ; +struct dma_fence_chain ; +struct drm_xe_sync ;struct xe_sched_job ;struct xe_vm ;@@ -19,6 +23,7 @@ struct xe_vm;#define SYNC_PARSE_FLAG_DISALLOW_USER_FENCE BIT(2)int xe_sync_entry_parse (struct xe_device *xe,struct xe_file *xef, +struct xe_vm *vm,struct xe_sync_entry *sync,struct drm_xe_sync __user *sync_user,unsigned int flags);@@ -40,5 +45,6 @@ struct xe_user_fence *__xe_sync_ufence_get(struct xe_user_fence *ufence);struct xe_user_fence *xe_sync_ufence_get (struct xe_sync_entry *sync);void xe_sync_ufence_put (struct xe_user_fence *ufence);int xe_sync_ufence_get_status (struct xe_user_fence *ufence); +void xe_sync_ufence_signal (struct xe_user_fence *ufence);#endif drivers/gpu/drm/xe/xe_sync_types.h +21
-5@@ -7,12 +7,28 @@#define _XE_SYNC_TYPES_H_#include <linux/types.h> +#include <linux/spinlock.h> +#include <linux/kref.h> +#include <linux/dma-fence-array.h> -struct drm_syncobj ; -struct dma_fence ; -struct dma_fence_chain ; -struct drm_xe_sync ; -struct user_fence ; +struct xe_eudebug ; + +struct xe_user_fence { +struct xe_device *xe; +struct kref refcount; +struct dma_fence_cb cb; +struct work_struct worker; +struct mm_struct *mm; +u64 __user *addr; +u64 value; +int signalled; +struct { +struct xe_eudebug *debugger; +u64 bind_ref_seqno; +u64 signalled_seqno; +struct work_struct worker; + } eudebug; +};struct xe_sync_entry {struct drm_syncobj *syncobj;drivers/gpu/drm/xe/xe_vm.c +3
-1@@ -3049,9 +3049,11 @@ int xe_vm_bind_ioctl(struct drm_device *dev, void *data, struct drm_file *file) } } +xe_eudebug_vm_bind_start (vm); + syncs_user =u64_to_user_ptr (args->syncs);for (num_syncs =0 ; num_syncs < args->num_syncs; num_syncs++) { - err = xe_sync_entry_parse(xe, xef, &syncs[num_syncs], + err = xe_sync_entry_parse(xe, xef, vm, &syncs[num_syncs], &syncs_user[num_syncs], (xe_vm_in_lr_mode(vm) ? SYNC_PARSE_FLAG_LR_MODE : 0) |include/uapi/drm/xe_drm_eudebug.h +14
-1@@ -17,6 +17,7 @@ extern "C" {*/ #define DRM_XE_EUDEBUG_IOCTL_READ_EVENT _IO('j', 0x0)#define DRM_XE_EUDEBUG_IOCTL_EU_CONTROL _IOWR('j', 0x2, struct drm_xe_eudebug_eu_control) +#define DRM_XE_EUDEBUG_IOCTL_ACK_EVENT _IOW('j', 0x4, struct drm_xe_eudebug_ack_event)/* XXX: Document events to match their internal counterparts when moved to xe_drm.h */ struct drm_xe_eudebug_event {@@ -31,7 +32,8 @@ struct drm_xe_eudebug_event {#define DRM_XE_EUDEBUG_EVENT_EU_ATTENTION 5#define DRM_XE_EUDEBUG_EVENT_VM_BIND 6#define DRM_XE_EUDEBUG_EVENT_VM_BIND_OP 7 -#define DRM_XE_EUDEBUG_EVENT_MAX_EVENT DRM_XE_EUDEBUG_EVENT_VM_BIND_OP +#define DRM_XE_EUDEBUG_EVENT_VM_BIND_UFENCE 8 +#define DRM_XE_EUDEBUG_EVENT_MAX_EVENT DRM_XE_EUDEBUG_EVENT_VM_BIND_UFENCE__u16 flags;#define DRM_XE_EUDEBUG_EVENT_CREATE (1 << 0)@@ -158,6 +160,17 @@ struct drm_xe_eudebug_event_vm_bind_op {__u64 range;/* XXX: Zero for unmap all? */ }; +struct drm_xe_eudebug_event_vm_bind_ufence { +struct drm_xe_eudebug_event base; +__u64 vm_bind_ref_seqno;/* *_event_vm_bind.base.seqno */ +}; + +struct drm_xe_eudebug_ack_event { +__u32 type; +__u32 flags;/* MBZ */ +__u64 seqno; +}; +#if defined(__cplusplus) }#endif Signature · 1 line
2.34.1
Re: [PATCH 13/21] drm/xe/eudebug: Add UFENCE events with acks
Quoted 63 lines · Matthew Brost
Quoted 4 lines · Mika Kuoppala
When vma is in place, debugger needs to intercept before userspace proceeds with the workload. For example to install a breakpoint in a eu shader.
Quoted 9 lines
Attach debugger in
xe_user_fence, send UFENCE event and stall normal user fence signal path to yield if there is debugger attached to ufence.When ack (ioctl) is received for the corresponding seqno, signal ufence.
v2: ufence worker in own workqueue
Not a complete review again, just a couple of quick comments.
Quoted 28 lines
Signed-off-by Mika Kuoppala <[email protected]>--- drivers/gpu/drm/xe/xe_eudebug.c | 239 +++++++++++++++++++++++++- drivers/gpu/drm/xe/xe_eudebug.h | 6 + drivers/gpu/drm/xe/xe_eudebug_types.h | 12 ++ drivers/gpu/drm/xe/xe_exec.c | 2 +- drivers/gpu/drm/xe/xe_sync.c | 49 ++++-- drivers/gpu/drm/xe/xe_sync.h | 8 +- drivers/gpu/drm/xe/xe_sync_types.h | 26 ++- drivers/gpu/drm/xe/xe_vm.c | 4 +- include/uapi/drm/xe_drm_eudebug.h | 15 +- 9 files changed, 333 insertions(+), 28 deletions(-)drivers/gpu/drm/xe/xe_eudebug.c +1
-0@@ -35,6 +35,7 @@#include "xe_vm.h" #include "xe_wa.h" #include "xe_force_wake.h" +#include "xe_sync.h" /* * If there is no detected event read by userspace, during this period, assume @@ -240,11 +241,115 @@ static void xe_eudebug_free(struct kref *ref)kfree_rcu (d, rcu); }drivers/gpu/drm/xe/xe_eudebug.c static void xe_eudebug_free(struct kref *ref) · line 251
Quoted 10 lines
drivers/gpu/drm/xe/xe_eudebug.c +4
-1-static void xe_eudebug_put (struct xe_eudebug *d) +void xe_eudebug_put (struct xe_eudebug *d) {kref_put (&d->ref, xe_eudebug_free); } +struct xe_eudebug_ack { +struct rb_node rb_node; +u64 seqno;Here we have a seqno but then an RB tree for search which is a bit incongruent. Let me explain, seqno typically mean something which signals sequentially in order. But then you have RB tree for searching, if everything signaled in order a list would be sufficient.
So without looking too much I think we want one of two things.
Quoted 7 lines
1. seqno + list
2. key + rb treeMake sense?
Kinda a nit but naming matters in this case as a name implies in-order vs out-of-order signaling.
This is the seqno of event that was associated with ufence. If userspace does multiple binds with ufences, it can then pinpoint, with ack ioctl, the particular bind it wants to be released, with this seqno.
And userspace can ack out of sequence. As it can potentially receive events out of sequence.
Would it help if it would be event_seqno instead of just seqno?
-Mika
Quoted 603 lines
Matt
Quoted 600 lines
drivers/gpu/drm/xe/xe_eudebug.c +231
-2+u64 ts_insert; +struct xe_user_fence *ufence; +}; + +#define fetch_ack (x) rb_entry(x, struct xe_eudebug_ack, rb_node) + +static int compare_ack (const u64 a,const u64 b) +{ +if (a < b) +return -1 ; +else if (a > b) +return 1 ; + +return 0 ; +} + +static int ack_insert_cmp (struct rb_node *const node, +const struct rb_node *const p) +{ +return compare_ack (fetch_ack (node)->seqno, +fetch_ack (p)->seqno); +} + +static int ack_lookup_cmp (const void *const key, +const struct rb_node *const node) +{ +return compare_ack (*(const u64 *)key, +fetch_ack (node)->seqno); +} + +static struct xe_eudebug_ack *remove_ack (struct xe_eudebug *d,u64 seqno) +{ +struct rb_root *const root = &d->acks.tree; +struct rb_node *node; + +spin_lock (&d->acks.lock); + node =rb_find (&seqno, root, ack_lookup_cmp); +if (node) +rb_erase (node, root); +spin_unlock (&d->acks.lock); + +if (!node) +return NULL ; + + return rb_entry_safe(node, struct xe_eudebug_ack, rb_node); +} + +static void ufence_signal_worker (struct work_struct *w) +{ +struct xe_user_fence *const ufence = + container_of(w, struct xe_user_fence, eudebug.worker); + +if (READ_ONCE (ufence->signalled)) +xe_sync_ufence_signal (ufence); + +xe_sync_ufence_put (ufence); +} + +static void kick_ufence_worker (struct xe_user_fence *f) +{ +INIT_WORK (&f->eudebug.worker, ufence_signal_worker); +queue_work (f->xe->eudebug.ordered_wq, &f->eudebug.worker); +} + +static void handle_ack (struct xe_eudebug *d,struct xe_eudebug_ack *ack, +bool on_disconnect) +{ +struct xe_user_fence *f = ack->ufence; +u64 signaller_ack, signalled_by; + + signaller_ack =cmpxchg64 (&f->eudebug.signalled_seqno,0 , ack->seqno); + signalled_by = f->eudebug.signalled_seqno; + +if (!signaller_ack) +kick_ufence_worker (f); +else +xe_sync_ufence_put (f); + +eu_dbg (d,"ACK: seqno=%llu: %ssignalled by %s (%llu) (held %lluus)" , + ack->seqno, signaller_ack ?"already " :"" , + on_disconnect ?"disconnect" :"debugger" , + signalled_by, +ktime_us_delta (ktime_get (), ack->ts_insert)); + +kfree (ack); +} + +static void release_acks (struct xe_eudebug *d) +{ +struct xe_eudebug_ack *ack, *n; +struct rb_root root; + +spin_lock (&d->acks.lock); + root = d->acks.tree; + d->acks.tree =RB_ROOT ; +spin_unlock (&d->acks.lock); + +rbtree_postorder_for_each_entry_safe (ack, n, &root, rb_node) +handle_ack (d, ack, true); +} +static struct task_struct *find_get_target (const pid_t nr) {struct task_struct *task;@@ -328,6 +433,8 @@ static bool xe_eudebug_detach(struct xe_device *xe,eu_dbg (d,"session %lld detached with %d" , d->session, err); +release_acks (d); +/* Our ref with the connection_link */ xe_eudebug_put (d);@@ -428,7 +535,7 @@ static struct task_struct *find_task_get(struct xe_file *xef)return task; } -static struct xe_eudebug * +struct xe_eudebug *xe_eudebug_get (struct xe_file *xef) {struct task_struct *task;@@ -889,6 +996,44 @@ static long xe_eudebug_read_event(struct xe_eudebug *d,return ret; } +static long +xe_eudebug_ack_event_ioctl (struct xe_eudebug *d, +const unsigned int cmd, +const u64 arg) +{ +struct drm_xe_eudebug_ack_event __user *const user_ptr = +u64_to_user_ptr (arg); +struct drm_xe_eudebug_ack_event user_arg; +struct xe_eudebug_ack *ack; +struct xe_device *xe = d->xe; + +if (XE_IOCTL_DBG (xe,_IOC_SIZE (cmd) <sizeof (user_arg))) +return -EINVAL ; + +/* Userland write */ +if (XE_IOCTL_DBG (xe, !(_IOC_DIR (cmd) & _IOC_WRITE))) +return -EINVAL ; + +if (XE_IOCTL_DBG (xe,copy_from_user (&user_arg, + user_ptr, +sizeof (user_arg)))) +return -EFAULT ; + +if (XE_IOCTL_DBG (xe, user_arg.flags)) +return -EINVAL ; + +if (XE_IOCTL_DBG (xe,xe_eudebug_detached (d))) +return -ENOTCONN ; + + ack =remove_ack (d, user_arg.seqno); +if (XE_IOCTL_DBG (xe, !ack)) +return -EINVAL ; + +handle_ack (d, ack, false); + +return 0 ; +} +static int do_eu_control (struct xe_eudebug *d,const struct drm_xe_eudebug_eu_control *const arg,struct drm_xe_eudebug_eu_control __user *const user_ptr)@@ -1070,7 +1215,10 @@ static long xe_eudebug_ioctl(struct file *file, ret =xe_eudebug_eu_control (d, arg);eu_dbg (d,"ioctl cmd=EU_CONTROL ret=%ld\n" , ret);break ; - +case DRM_XE_EUDEBUG_IOCTL_ACK_EVENT : + ret =xe_eudebug_ack_event_ioctl (d, cmd, arg); +eu_dbg (d,"ioctl cmd=EVENT_ACK ret=%ld\n" , ret); +break ;default : ret = -EINVAL ; }@@ -1759,6 +1907,9 @@ xe_eudebug_connect(struct xe_device *xe,INIT_KFIFO (d->events.fifo);INIT_WORK (&d->discovery_work, discovery_work_fn); +spin_lock_init (&d->acks.lock); + d->acks.tree =RB_ROOT ; + d->res =xe_eudebug_resources_alloc ();if (IS_ERR (d->res)) { err =PTR_ERR (d->res);@@ -2337,6 +2488,70 @@ static int vm_bind_op(struct xe_eudebug *d, struct xe_vm *vm,return 0 ; } +static int xe_eudebug_track_ufence (struct xe_eudebug *d, +struct xe_user_fence *f, +u64 seqno) +{ +struct xe_eudebug_ack *ack; +struct rb_node *old; + + ack =kzalloc (sizeof (*ack),GFP_KERNEL ); +if (!ack) +return -ENOMEM ; + + ack->seqno = seqno; + ack->ts_insert =ktime_get (); + +spin_lock (&d->acks.lock); + old =rb_find_add (&ack->rb_node, + &d->acks.tree, ack_insert_cmp); +if (!old) { +kref_get (&f->refcount); + ack->ufence = f; + } +spin_unlock (&d->acks.lock); + +if (old) { +eu_dbg (d,"ACK: seqno=%llu: already exists" , seqno); +kfree (ack); +return -EEXIST ; + } + +eu_dbg (d,"ACK: seqno=%llu: tracking started" , seqno); + +return 0 ; +} + +static int vm_bind_ufence_event (struct xe_eudebug *d, +struct xe_user_fence *ufence) +{ +struct xe_eudebug_event *event; +struct xe_eudebug_event_vm_bind_ufence *e; +const u32 sz =sizeof (*e); +const u32 flags =DRM_XE_EUDEBUG_EVENT_CREATE | +DRM_XE_EUDEBUG_EVENT_NEED_ACK ; +u64 seqno; +int ret; + + seqno =atomic_long_inc_return (&d->events.seqno); + + event =xe_eudebug_create_event (d,DRM_XE_EUDEBUG_EVENT_VM_BIND_UFENCE , + seqno, flags, sz,GFP_KERNEL ); +if (!event) +return -ENOMEM ; + + e =cast_event (e, event); + + write_member(struct drm_xe_eudebug_event_vm_bind_ufence, + e, vm_bind_ref_seqno, ufence->eudebug.bind_ref_seqno); + + ret =xe_eudebug_track_ufence (d, ufence, seqno); +if (!ret) + ret =xe_eudebug_queue_event (d, event); + +return ret; +} +void xe_eudebug_vm_bind_start (struct xe_vm *vm) {struct xe_eudebug *d;@@ -2507,6 +2722,24 @@ void xe_eudebug_vm_bind_end(struct xe_vm *vm, bool has_ufence, int bind_err)xe_eudebug_put (d); } +int xe_eudebug_vm_bind_ufence (struct xe_user_fence *ufence) +{ +struct xe_eudebug *d; +int err; + + d = ufence->eudebug.debugger; +if (!d ||xe_eudebug_detached (d)) +return -ENOTCONN ; + + err =vm_bind_ufence_event (d, ufence); +if (err) { +eu_err (d,"error %d on %s" , err, __func__); +xe_eudebug_disconnect (d, err); + } + +return 0 ; +} +static int discover_client (struct xe_eudebug *d,struct xe_file *xef) {struct xe_exec_queue *q;drivers/gpu/drm/xe/xe_eudebug.h +6
-0@@ -15,6 +15,7 @@ struct xe_vm;struct xe_vma ;struct xe_exec_queue ;struct xe_hw_engine ; +struct xe_user_fence ; int xe_eudebug_connect_ioctl(struct drm_device *dev, void *data,@@ -38,4 +39,9 @@ void xe_eudebug_vm_bind_start(struct xe_vm *vm);void xe_eudebug_vm_bind_op_add (struct xe_vm *vm,u32 op,u64 addr,u64 range);void xe_eudebug_vm_bind_end (struct xe_vm *vm,bool has_ufence,int err); +int xe_eudebug_vm_bind_ufence (struct xe_user_fence *ufence); + +struct xe_eudebug *xe_eudebug_get (struct xe_file *xef); +void xe_eudebug_put (struct xe_eudebug *d); +#endif drivers/gpu/drm/xe/xe_eudebug_types.h +12
-0@@ -86,6 +86,7 @@ struct xe_eudebug_eu_control_ops {int (*stopped)(struct xe_eudebug *e,struct xe_exec_queue *q,struct xe_lrc *lrc,u8 *bitmap,unsigned int bitmap_size); }; +/** * struct xe_eudebug - Top level struct for eudebug: the connection */ @@ -149,6 +150,12 @@ struct xe_eudebug {atomic_long_t seqno; } events; +/* user fences tracked by this debugger */ +struct { +spinlock_t lock; +struct rb_root tree; + } acks; +/** @ops operations for eu_control */ struct xe_eudebug_eu_control_ops *ops; };@@ -286,4 +293,9 @@ struct xe_eudebug_event_vm_bind_op { u64 range; /* Zero for unmap all ? */ }; +struct xe_eudebug_event_vm_bind_ufence { +struct xe_eudebug_event base; +u64 vm_bind_ref_seqno; +}; +#endif drivers/gpu/drm/xe/xe_exec.c +1
-1@@ -157,7 +157,7 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file) vm = q->vm;for (num_syncs =0 ; num_syncs < args->num_syncs; num_syncs++) { - err =xe_sync_entry_parse (xe, xef, &syncs[num_syncs], + err =xe_sync_entry_parse (xe, xef, vm, &syncs[num_syncs], &syncs_user[num_syncs],SYNC_PARSE_FLAG_EXEC | (xe_vm_in_lr_mode (vm) ?SYNC_PARSE_FLAG_LR_MODE :0 ));drivers/gpu/drm/xe/xe_sync.c +33
-16@@ -18,17 +18,7 @@#include "xe_exec_queue.h" #include "xe_macros.h" #include "xe_sched_job_types.h" - -struct xe_user_fence { -struct xe_device *xe; -struct kref refcount; -struct dma_fence_cb cb; -struct work_struct worker; -struct mm_struct *mm; -u64 __user *addr; -u64 value; -int signalled; -}; +#include "xe_eudebug.h" static void user_fence_destroy (struct kref *kref) {@@ -36,6 +26,10 @@ static void user_fence_destroy(struct kref *kref) refcount);mmdrop (ufence->mm); + +if (ufence->eudebug.debugger) +xe_eudebug_put (ufence->eudebug.debugger); +kfree (ufence); }@@ -49,7 +43,10 @@ static void user_fence_put(struct xe_user_fence *ufence)kref_put (&ufence->refcount, user_fence_destroy); } -static struct xe_user_fence *user_fence_create (struct xe_device *xe,u64 addr, +static struct xe_user_fence *user_fence_create (struct xe_device *xe, +struct xe_file *xef, +struct xe_vm *vm, +u64 addr,u64 value) {struct xe_user_fence *ufence;@@ -58,7 +55,7 @@ static struct xe_user_fence *user_fence_create(struct xe_device *xe, u64 addr,if (!access_ok (ptr,sizeof (ptr)))return ERR_PTR (-EFAULT ); - ufence =kmalloc (sizeof (*ufence),GFP_KERNEL ); + ufence =kzalloc (sizeof (*ufence),GFP_KERNEL );if (!ufence)return ERR_PTR (-ENOMEM );@@ -69,12 +66,17 @@ static struct xe_user_fence *user_fence_create(struct xe_device *xe, u64 addr, ufence->mm = current->mm;mmgrab (ufence->mm); +if (vm->eudebug_bind.ref) { + ufence->eudebug.debugger =xe_eudebug_get (xef); + ufence->eudebug.bind_ref_seqno = vm->eudebug_bind.ref; + } +return ufence; } -static void user_fence_worker (struct work_struct *w) +void xe_sync_ufence_signal (struct xe_user_fence *ufence) { - struct xe_user_fence *ufence = container_of(w, struct xe_user_fence, worker); +XE_WARN_ON (!ufence->signalled);if (mmget_not_zero (ufence->mm)) {kthread_use_mm (ufence->mm);@@ -85,7 +87,20 @@ static void user_fence_worker(struct work_struct *w) }wake_up_all (&ufence->xe->ufence_wq); +} + +static void user_fence_worker (struct work_struct *w) +{ + struct xe_user_fence *ufence = container_of(w, struct xe_user_fence, worker); +int ret; +WRITE_ONCE (ufence->signalled,1 ); + +/* Lets see if debugger wants to track this */ + ret =xe_eudebug_vm_bind_ufence (ufence); +if (ret) +xe_sync_ufence_signal (ufence); +user_fence_put (ufence); }@@ -104,6 +119,7 @@ static void user_fence_cb(struct dma_fence *fence, struct dma_fence_cb *cb) }int xe_sync_entry_parse (struct xe_device *xe,struct xe_file *xef, +struct xe_vm *vm,struct xe_sync_entry *sync,struct drm_xe_sync __user *sync_user,unsigned int flags)@@ -185,7 +201,8 @@ int xe_sync_entry_parse(struct xe_device *xe, struct xe_file *xef,if (exec) { sync->addr = sync_in.addr; }else { - sync->ufence =user_fence_create (xe, sync_in.addr, + sync->ufence =user_fence_create (xe, xef, vm, + sync_in.addr, sync_in.timeline_value);if (XE_IOCTL_DBG (xe,IS_ERR (sync->ufence)))return PTR_ERR (sync->ufence);drivers/gpu/drm/xe/xe_sync.h +7
-1@@ -9,8 +9,12 @@#include "xe_sync_types.h" struct xe_device ; -struct xe_exec_queue ;struct xe_file ; +struct xe_exec_queue ; +struct drm_syncobj ; +struct dma_fence ; +struct dma_fence_chain ; +struct drm_xe_sync ;struct xe_sched_job ;struct xe_vm ;@@ -19,6 +23,7 @@ struct xe_vm;#define SYNC_PARSE_FLAG_DISALLOW_USER_FENCE BIT(2)int xe_sync_entry_parse (struct xe_device *xe,struct xe_file *xef, +struct xe_vm *vm,struct xe_sync_entry *sync,struct drm_xe_sync __user *sync_user,unsigned int flags);@@ -40,5 +45,6 @@ struct xe_user_fence *__xe_sync_ufence_get(struct xe_user_fence *ufence);struct xe_user_fence *xe_sync_ufence_get (struct xe_sync_entry *sync);void xe_sync_ufence_put (struct xe_user_fence *ufence);int xe_sync_ufence_get_status (struct xe_user_fence *ufence); +void xe_sync_ufence_signal (struct xe_user_fence *ufence);#endif drivers/gpu/drm/xe/xe_sync_types.h +21
-5@@ -7,12 +7,28 @@#define _XE_SYNC_TYPES_H_#include <linux/types.h> +#include <linux/spinlock.h> +#include <linux/kref.h> +#include <linux/dma-fence-array.h> -struct drm_syncobj ; -struct dma_fence ; -struct dma_fence_chain ; -struct drm_xe_sync ; -struct user_fence ; +struct xe_eudebug ; + +struct xe_user_fence { +struct xe_device *xe; +struct kref refcount; +struct dma_fence_cb cb; +struct work_struct worker; +struct mm_struct *mm; +u64 __user *addr; +u64 value; +int signalled; +struct { +struct xe_eudebug *debugger; +u64 bind_ref_seqno; +u64 signalled_seqno; +struct work_struct worker; + } eudebug; +};struct xe_sync_entry {struct drm_syncobj *syncobj;drivers/gpu/drm/xe/xe_vm.c +3
-1@@ -3049,9 +3049,11 @@ int xe_vm_bind_ioctl(struct drm_device *dev, void *data, struct drm_file *file) } } +xe_eudebug_vm_bind_start (vm); + syncs_user =u64_to_user_ptr (args->syncs);for (num_syncs =0 ; num_syncs < args->num_syncs; num_syncs++) { - err = xe_sync_entry_parse(xe, xef, &syncs[num_syncs], + err = xe_sync_entry_parse(xe, xef, vm, &syncs[num_syncs], &syncs_user[num_syncs], (xe_vm_in_lr_mode(vm) ? SYNC_PARSE_FLAG_LR_MODE : 0) |include/uapi/drm/xe_drm_eudebug.h +14
-1@@ -17,6 +17,7 @@ extern "C" {*/ #define DRM_XE_EUDEBUG_IOCTL_READ_EVENT _IO('j', 0x0)#define DRM_XE_EUDEBUG_IOCTL_EU_CONTROL _IOWR('j', 0x2, struct drm_xe_eudebug_eu_control) +#define DRM_XE_EUDEBUG_IOCTL_ACK_EVENT _IOW('j', 0x4, struct drm_xe_eudebug_ack_event)/* XXX: Document events to match their internal counterparts when moved to xe_drm.h */ struct drm_xe_eudebug_event {@@ -31,7 +32,8 @@ struct drm_xe_eudebug_event {#define DRM_XE_EUDEBUG_EVENT_EU_ATTENTION 5#define DRM_XE_EUDEBUG_EVENT_VM_BIND 6#define DRM_XE_EUDEBUG_EVENT_VM_BIND_OP 7 -#define DRM_XE_EUDEBUG_EVENT_MAX_EVENT DRM_XE_EUDEBUG_EVENT_VM_BIND_OP +#define DRM_XE_EUDEBUG_EVENT_VM_BIND_UFENCE 8 +#define DRM_XE_EUDEBUG_EVENT_MAX_EVENT DRM_XE_EUDEBUG_EVENT_VM_BIND_UFENCE__u16 flags;#define DRM_XE_EUDEBUG_EVENT_CREATE (1 << 0)@@ -158,6 +160,17 @@ struct drm_xe_eudebug_event_vm_bind_op {__u64 range;/* XXX: Zero for unmap all? */ }; +struct drm_xe_eudebug_event_vm_bind_ufence { +struct drm_xe_eudebug_event base; +__u64 vm_bind_ref_seqno;/* *_event_vm_bind.base.seqno */ +}; + +struct drm_xe_eudebug_ack_event { +__u32 type; +__u32 flags;/* MBZ */ +__u64 seqno; +}; +#if defined(__cplusplus) }#endif Signature · 1 line
2.34.1
Matthew Brost
maintainerRe: [PATCH 13/21] drm/xe/eudebug: Add UFENCE events with acks
Quoted 73 lines · Mika Kuoppala
Quoted 63 lines · Matthew Brost
Quoted 4 lines · Mika Kuoppala
When vma is in place, debugger needs to intercept before userspace proceeds with the workload. For example to install a breakpoint in a eu shader.
Quoted 9 lines
Attach debugger in
xe_user_fence, send UFENCE event and stall normal user fence signal path to yield if there is debugger attached to ufence.When ack (ioctl) is received for the corresponding seqno, signal ufence.
v2: ufence worker in own workqueue
Not a complete review again, just a couple of quick comments.
Quoted 28 lines
Signed-off-by Mika Kuoppala <[email protected]>--- drivers/gpu/drm/xe/xe_eudebug.c | 239 +++++++++++++++++++++++++- drivers/gpu/drm/xe/xe_eudebug.h | 6 + drivers/gpu/drm/xe/xe_eudebug_types.h | 12 ++ drivers/gpu/drm/xe/xe_exec.c | 2 +- drivers/gpu/drm/xe/xe_sync.c | 49 ++++-- drivers/gpu/drm/xe/xe_sync.h | 8 +- drivers/gpu/drm/xe/xe_sync_types.h | 26 ++- drivers/gpu/drm/xe/xe_vm.c | 4 +- include/uapi/drm/xe_drm_eudebug.h | 15 +- 9 files changed, 333 insertions(+), 28 deletions(-)drivers/gpu/drm/xe/xe_eudebug.c +1
-0@@ -35,6 +35,7 @@#include "xe_vm.h" #include "xe_wa.h" #include "xe_force_wake.h" +#include "xe_sync.h" /* * If there is no detected event read by userspace, during this period, assume @@ -240,11 +241,115 @@ static void xe_eudebug_free(struct kref *ref)kfree_rcu (d, rcu); }drivers/gpu/drm/xe/xe_eudebug.c static void xe_eudebug_free(struct kref *ref) · line 251
Quoted 10 lines
drivers/gpu/drm/xe/xe_eudebug.c +4
-1-static void xe_eudebug_put (struct xe_eudebug *d) +void xe_eudebug_put (struct xe_eudebug *d) {kref_put (&d->ref, xe_eudebug_free); } +struct xe_eudebug_ack { +struct rb_node rb_node; +u64 seqno;Here we have a seqno but then an RB tree for search which is a bit incongruent. Let me explain, seqno typically mean something which signals sequentially in order. But then you have RB tree for searching, if everything signaled in order a list would be sufficient.
So without looking too much I think we want one of two things.
Quoted 7 lines
1. seqno + list
2. key + rb treeMake sense?
Kinda a nit but naming matters in this case as a name implies in-order vs out-of-order signaling.
Quoted 8 lines
This is the seqno of event that was associated with ufence. If userspace does multiple binds with ufences, it can then pinpoint, with ack ioctl, the particular bind it wants to be released, with this seqno.
And userspace can ack out of sequence. As it can potentially receive events out of sequence.
Would it help if it would be
event_seqnoinstead of just seqno?
If userspace can ack out of sequence, I'd rename it 'event_id'.
So s/events.seqno/events.id too.
Does that work?
Matt
Quoted 606 lines
-Mika
Quoted 603 lines
Matt
Quoted 600 lines
drivers/gpu/drm/xe/xe_eudebug.c +231
-2+u64 ts_insert; +struct xe_user_fence *ufence; +}; + +#define fetch_ack (x) rb_entry(x, struct xe_eudebug_ack, rb_node) + +static int compare_ack (const u64 a,const u64 b) +{ +if (a < b) +return -1 ; +else if (a > b) +return 1 ; + +return 0 ; +} + +static int ack_insert_cmp (struct rb_node *const node, +const struct rb_node *const p) +{ +return compare_ack (fetch_ack (node)->seqno, +fetch_ack (p)->seqno); +} + +static int ack_lookup_cmp (const void *const key, +const struct rb_node *const node) +{ +return compare_ack (*(const u64 *)key, +fetch_ack (node)->seqno); +} + +static struct xe_eudebug_ack *remove_ack (struct xe_eudebug *d,u64 seqno) +{ +struct rb_root *const root = &d->acks.tree; +struct rb_node *node; + +spin_lock (&d->acks.lock); + node =rb_find (&seqno, root, ack_lookup_cmp); +if (node) +rb_erase (node, root); +spin_unlock (&d->acks.lock); + +if (!node) +return NULL ; + + return rb_entry_safe(node, struct xe_eudebug_ack, rb_node); +} + +static void ufence_signal_worker (struct work_struct *w) +{ +struct xe_user_fence *const ufence = + container_of(w, struct xe_user_fence, eudebug.worker); + +if (READ_ONCE (ufence->signalled)) +xe_sync_ufence_signal (ufence); + +xe_sync_ufence_put (ufence); +} + +static void kick_ufence_worker (struct xe_user_fence *f) +{ +INIT_WORK (&f->eudebug.worker, ufence_signal_worker); +queue_work (f->xe->eudebug.ordered_wq, &f->eudebug.worker); +} + +static void handle_ack (struct xe_eudebug *d,struct xe_eudebug_ack *ack, +bool on_disconnect) +{ +struct xe_user_fence *f = ack->ufence; +u64 signaller_ack, signalled_by; + + signaller_ack =cmpxchg64 (&f->eudebug.signalled_seqno,0 , ack->seqno); + signalled_by = f->eudebug.signalled_seqno; + +if (!signaller_ack) +kick_ufence_worker (f); +else +xe_sync_ufence_put (f); + +eu_dbg (d,"ACK: seqno=%llu: %ssignalled by %s (%llu) (held %lluus)" , + ack->seqno, signaller_ack ?"already " :"" , + on_disconnect ?"disconnect" :"debugger" , + signalled_by, +ktime_us_delta (ktime_get (), ack->ts_insert)); + +kfree (ack); +} + +static void release_acks (struct xe_eudebug *d) +{ +struct xe_eudebug_ack *ack, *n; +struct rb_root root; + +spin_lock (&d->acks.lock); + root = d->acks.tree; + d->acks.tree =RB_ROOT ; +spin_unlock (&d->acks.lock); + +rbtree_postorder_for_each_entry_safe (ack, n, &root, rb_node) +handle_ack (d, ack, true); +} +static struct task_struct *find_get_target (const pid_t nr) {struct task_struct *task;@@ -328,6 +433,8 @@ static bool xe_eudebug_detach(struct xe_device *xe,eu_dbg (d,"session %lld detached with %d" , d->session, err); +release_acks (d); +/* Our ref with the connection_link */ xe_eudebug_put (d);@@ -428,7 +535,7 @@ static struct task_struct *find_task_get(struct xe_file *xef)return task; } -static struct xe_eudebug * +struct xe_eudebug *xe_eudebug_get (struct xe_file *xef) {struct task_struct *task;@@ -889,6 +996,44 @@ static long xe_eudebug_read_event(struct xe_eudebug *d,return ret; } +static long +xe_eudebug_ack_event_ioctl (struct xe_eudebug *d, +const unsigned int cmd, +const u64 arg) +{ +struct drm_xe_eudebug_ack_event __user *const user_ptr = +u64_to_user_ptr (arg); +struct drm_xe_eudebug_ack_event user_arg; +struct xe_eudebug_ack *ack; +struct xe_device *xe = d->xe; + +if (XE_IOCTL_DBG (xe,_IOC_SIZE (cmd) <sizeof (user_arg))) +return -EINVAL ; + +/* Userland write */ +if (XE_IOCTL_DBG (xe, !(_IOC_DIR (cmd) & _IOC_WRITE))) +return -EINVAL ; + +if (XE_IOCTL_DBG (xe,copy_from_user (&user_arg, + user_ptr, +sizeof (user_arg)))) +return -EFAULT ; + +if (XE_IOCTL_DBG (xe, user_arg.flags)) +return -EINVAL ; + +if (XE_IOCTL_DBG (xe,xe_eudebug_detached (d))) +return -ENOTCONN ; + + ack =remove_ack (d, user_arg.seqno); +if (XE_IOCTL_DBG (xe, !ack)) +return -EINVAL ; + +handle_ack (d, ack, false); + +return 0 ; +} +static int do_eu_control (struct xe_eudebug *d,const struct drm_xe_eudebug_eu_control *const arg,struct drm_xe_eudebug_eu_control __user *const user_ptr)@@ -1070,7 +1215,10 @@ static long xe_eudebug_ioctl(struct file *file, ret =xe_eudebug_eu_control (d, arg);eu_dbg (d,"ioctl cmd=EU_CONTROL ret=%ld\n" , ret);break ; - +case DRM_XE_EUDEBUG_IOCTL_ACK_EVENT : + ret =xe_eudebug_ack_event_ioctl (d, cmd, arg); +eu_dbg (d,"ioctl cmd=EVENT_ACK ret=%ld\n" , ret); +break ;default : ret = -EINVAL ; }@@ -1759,6 +1907,9 @@ xe_eudebug_connect(struct xe_device *xe,INIT_KFIFO (d->events.fifo);INIT_WORK (&d->discovery_work, discovery_work_fn); +spin_lock_init (&d->acks.lock); + d->acks.tree =RB_ROOT ; + d->res =xe_eudebug_resources_alloc ();if (IS_ERR (d->res)) { err =PTR_ERR (d->res);@@ -2337,6 +2488,70 @@ static int vm_bind_op(struct xe_eudebug *d, struct xe_vm *vm,return 0 ; } +static int xe_eudebug_track_ufence (struct xe_eudebug *d, +struct xe_user_fence *f, +u64 seqno) +{ +struct xe_eudebug_ack *ack; +struct rb_node *old; + + ack =kzalloc (sizeof (*ack),GFP_KERNEL ); +if (!ack) +return -ENOMEM ; + + ack->seqno = seqno; + ack->ts_insert =ktime_get (); + +spin_lock (&d->acks.lock); + old =rb_find_add (&ack->rb_node, + &d->acks.tree, ack_insert_cmp); +if (!old) { +kref_get (&f->refcount); + ack->ufence = f; + } +spin_unlock (&d->acks.lock); + +if (old) { +eu_dbg (d,"ACK: seqno=%llu: already exists" , seqno); +kfree (ack); +return -EEXIST ; + } + +eu_dbg (d,"ACK: seqno=%llu: tracking started" , seqno); + +return 0 ; +} + +static int vm_bind_ufence_event (struct xe_eudebug *d, +struct xe_user_fence *ufence) +{ +struct xe_eudebug_event *event; +struct xe_eudebug_event_vm_bind_ufence *e; +const u32 sz =sizeof (*e); +const u32 flags =DRM_XE_EUDEBUG_EVENT_CREATE | +DRM_XE_EUDEBUG_EVENT_NEED_ACK ; +u64 seqno; +int ret; + + seqno =atomic_long_inc_return (&d->events.seqno); + + event =xe_eudebug_create_event (d,DRM_XE_EUDEBUG_EVENT_VM_BIND_UFENCE , + seqno, flags, sz,GFP_KERNEL ); +if (!event) +return -ENOMEM ; + + e =cast_event (e, event); + + write_member(struct drm_xe_eudebug_event_vm_bind_ufence, + e, vm_bind_ref_seqno, ufence->eudebug.bind_ref_seqno); + + ret =xe_eudebug_track_ufence (d, ufence, seqno); +if (!ret) + ret =xe_eudebug_queue_event (d, event); + +return ret; +} +void xe_eudebug_vm_bind_start (struct xe_vm *vm) {struct xe_eudebug *d;@@ -2507,6 +2722,24 @@ void xe_eudebug_vm_bind_end(struct xe_vm *vm, bool has_ufence, int bind_err)xe_eudebug_put (d); } +int xe_eudebug_vm_bind_ufence (struct xe_user_fence *ufence) +{ +struct xe_eudebug *d; +int err; + + d = ufence->eudebug.debugger; +if (!d ||xe_eudebug_detached (d)) +return -ENOTCONN ; + + err =vm_bind_ufence_event (d, ufence); +if (err) { +eu_err (d,"error %d on %s" , err, __func__); +xe_eudebug_disconnect (d, err); + } + +return 0 ; +} +static int discover_client (struct xe_eudebug *d,struct xe_file *xef) {struct xe_exec_queue *q;drivers/gpu/drm/xe/xe_eudebug.h +6
-0@@ -15,6 +15,7 @@ struct xe_vm;struct xe_vma ;struct xe_exec_queue ;struct xe_hw_engine ; +struct xe_user_fence ; int xe_eudebug_connect_ioctl(struct drm_device *dev, void *data,@@ -38,4 +39,9 @@ void xe_eudebug_vm_bind_start(struct xe_vm *vm);void xe_eudebug_vm_bind_op_add (struct xe_vm *vm,u32 op,u64 addr,u64 range);void xe_eudebug_vm_bind_end (struct xe_vm *vm,bool has_ufence,int err); +int xe_eudebug_vm_bind_ufence (struct xe_user_fence *ufence); + +struct xe_eudebug *xe_eudebug_get (struct xe_file *xef); +void xe_eudebug_put (struct xe_eudebug *d); +#endif drivers/gpu/drm/xe/xe_eudebug_types.h +12
-0@@ -86,6 +86,7 @@ struct xe_eudebug_eu_control_ops {int (*stopped)(struct xe_eudebug *e,struct xe_exec_queue *q,struct xe_lrc *lrc,u8 *bitmap,unsigned int bitmap_size); }; +/** * struct xe_eudebug - Top level struct for eudebug: the connection */ @@ -149,6 +150,12 @@ struct xe_eudebug {atomic_long_t seqno; } events; +/* user fences tracked by this debugger */ +struct { +spinlock_t lock; +struct rb_root tree; + } acks; +/** @ops operations for eu_control */ struct xe_eudebug_eu_control_ops *ops; };@@ -286,4 +293,9 @@ struct xe_eudebug_event_vm_bind_op { u64 range; /* Zero for unmap all ? */ }; +struct xe_eudebug_event_vm_bind_ufence { +struct xe_eudebug_event base; +u64 vm_bind_ref_seqno; +}; +#endif drivers/gpu/drm/xe/xe_exec.c +1
-1@@ -157,7 +157,7 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file) vm = q->vm;for (num_syncs =0 ; num_syncs < args->num_syncs; num_syncs++) { - err =xe_sync_entry_parse (xe, xef, &syncs[num_syncs], + err =xe_sync_entry_parse (xe, xef, vm, &syncs[num_syncs], &syncs_user[num_syncs],SYNC_PARSE_FLAG_EXEC | (xe_vm_in_lr_mode (vm) ?SYNC_PARSE_FLAG_LR_MODE :0 ));drivers/gpu/drm/xe/xe_sync.c +33
-16@@ -18,17 +18,7 @@#include "xe_exec_queue.h" #include "xe_macros.h" #include "xe_sched_job_types.h" - -struct xe_user_fence { -struct xe_device *xe; -struct kref refcount; -struct dma_fence_cb cb; -struct work_struct worker; -struct mm_struct *mm; -u64 __user *addr; -u64 value; -int signalled; -}; +#include "xe_eudebug.h" static void user_fence_destroy (struct kref *kref) {@@ -36,6 +26,10 @@ static void user_fence_destroy(struct kref *kref) refcount);mmdrop (ufence->mm); + +if (ufence->eudebug.debugger) +xe_eudebug_put (ufence->eudebug.debugger); +kfree (ufence); }@@ -49,7 +43,10 @@ static void user_fence_put(struct xe_user_fence *ufence)kref_put (&ufence->refcount, user_fence_destroy); } -static struct xe_user_fence *user_fence_create (struct xe_device *xe,u64 addr, +static struct xe_user_fence *user_fence_create (struct xe_device *xe, +struct xe_file *xef, +struct xe_vm *vm, +u64 addr,u64 value) {struct xe_user_fence *ufence;@@ -58,7 +55,7 @@ static struct xe_user_fence *user_fence_create(struct xe_device *xe, u64 addr,if (!access_ok (ptr,sizeof (ptr)))return ERR_PTR (-EFAULT ); - ufence =kmalloc (sizeof (*ufence),GFP_KERNEL ); + ufence =kzalloc (sizeof (*ufence),GFP_KERNEL );if (!ufence)return ERR_PTR (-ENOMEM );@@ -69,12 +66,17 @@ static struct xe_user_fence *user_fence_create(struct xe_device *xe, u64 addr, ufence->mm = current->mm;mmgrab (ufence->mm); +if (vm->eudebug_bind.ref) { + ufence->eudebug.debugger =xe_eudebug_get (xef); + ufence->eudebug.bind_ref_seqno = vm->eudebug_bind.ref; + } +return ufence; } -static void user_fence_worker (struct work_struct *w) +void xe_sync_ufence_signal (struct xe_user_fence *ufence) { - struct xe_user_fence *ufence = container_of(w, struct xe_user_fence, worker); +XE_WARN_ON (!ufence->signalled);if (mmget_not_zero (ufence->mm)) {kthread_use_mm (ufence->mm);@@ -85,7 +87,20 @@ static void user_fence_worker(struct work_struct *w) }wake_up_all (&ufence->xe->ufence_wq); +} + +static void user_fence_worker (struct work_struct *w) +{ + struct xe_user_fence *ufence = container_of(w, struct xe_user_fence, worker); +int ret; +WRITE_ONCE (ufence->signalled,1 ); + +/* Lets see if debugger wants to track this */ + ret =xe_eudebug_vm_bind_ufence (ufence); +if (ret) +xe_sync_ufence_signal (ufence); +user_fence_put (ufence); }@@ -104,6 +119,7 @@ static void user_fence_cb(struct dma_fence *fence, struct dma_fence_cb *cb) }int xe_sync_entry_parse (struct xe_device *xe,struct xe_file *xef, +struct xe_vm *vm,struct xe_sync_entry *sync,struct drm_xe_sync __user *sync_user,unsigned int flags)@@ -185,7 +201,8 @@ int xe_sync_entry_parse(struct xe_device *xe, struct xe_file *xef,if (exec) { sync->addr = sync_in.addr; }else { - sync->ufence =user_fence_create (xe, sync_in.addr, + sync->ufence =user_fence_create (xe, xef, vm, + sync_in.addr, sync_in.timeline_value);if (XE_IOCTL_DBG (xe,IS_ERR (sync->ufence)))return PTR_ERR (sync->ufence);drivers/gpu/drm/xe/xe_sync.h +7
-1@@ -9,8 +9,12 @@#include "xe_sync_types.h" struct xe_device ; -struct xe_exec_queue ;struct xe_file ; +struct xe_exec_queue ; +struct drm_syncobj ; +struct dma_fence ; +struct dma_fence_chain ; +struct drm_xe_sync ;struct xe_sched_job ;struct xe_vm ;@@ -19,6 +23,7 @@ struct xe_vm;#define SYNC_PARSE_FLAG_DISALLOW_USER_FENCE BIT(2)int xe_sync_entry_parse (struct xe_device *xe,struct xe_file *xef, +struct xe_vm *vm,struct xe_sync_entry *sync,struct drm_xe_sync __user *sync_user,unsigned int flags);@@ -40,5 +45,6 @@ struct xe_user_fence *__xe_sync_ufence_get(struct xe_user_fence *ufence);struct xe_user_fence *xe_sync_ufence_get (struct xe_sync_entry *sync);void xe_sync_ufence_put (struct xe_user_fence *ufence);int xe_sync_ufence_get_status (struct xe_user_fence *ufence); +void xe_sync_ufence_signal (struct xe_user_fence *ufence);#endif drivers/gpu/drm/xe/xe_sync_types.h +21
-5@@ -7,12 +7,28 @@#define _XE_SYNC_TYPES_H_#include <linux/types.h> +#include <linux/spinlock.h> +#include <linux/kref.h> +#include <linux/dma-fence-array.h> -struct drm_syncobj ; -struct dma_fence ; -struct dma_fence_chain ; -struct drm_xe_sync ; -struct user_fence ; +struct xe_eudebug ; + +struct xe_user_fence { +struct xe_device *xe; +struct kref refcount; +struct dma_fence_cb cb; +struct work_struct worker; +struct mm_struct *mm; +u64 __user *addr; +u64 value; +int signalled; +struct { +struct xe_eudebug *debugger; +u64 bind_ref_seqno; +u64 signalled_seqno; +struct work_struct worker; + } eudebug; +};struct xe_sync_entry {struct drm_syncobj *syncobj;drivers/gpu/drm/xe/xe_vm.c +3
-1@@ -3049,9 +3049,11 @@ int xe_vm_bind_ioctl(struct drm_device *dev, void *data, struct drm_file *file) } } +xe_eudebug_vm_bind_start (vm); + syncs_user =u64_to_user_ptr (args->syncs);for (num_syncs =0 ; num_syncs < args->num_syncs; num_syncs++) { - err = xe_sync_entry_parse(xe, xef, &syncs[num_syncs], + err = xe_sync_entry_parse(xe, xef, vm, &syncs[num_syncs], &syncs_user[num_syncs], (xe_vm_in_lr_mode(vm) ? SYNC_PARSE_FLAG_LR_MODE : 0) |include/uapi/drm/xe_drm_eudebug.h +14
-1@@ -17,6 +17,7 @@ extern "C" {*/ #define DRM_XE_EUDEBUG_IOCTL_READ_EVENT _IO('j', 0x0)#define DRM_XE_EUDEBUG_IOCTL_EU_CONTROL _IOWR('j', 0x2, struct drm_xe_eudebug_eu_control) +#define DRM_XE_EUDEBUG_IOCTL_ACK_EVENT _IOW('j', 0x4, struct drm_xe_eudebug_ack_event)/* XXX: Document events to match their internal counterparts when moved to xe_drm.h */ struct drm_xe_eudebug_event {@@ -31,7 +32,8 @@ struct drm_xe_eudebug_event {#define DRM_XE_EUDEBUG_EVENT_EU_ATTENTION 5#define DRM_XE_EUDEBUG_EVENT_VM_BIND 6#define DRM_XE_EUDEBUG_EVENT_VM_BIND_OP 7 -#define DRM_XE_EUDEBUG_EVENT_MAX_EVENT DRM_XE_EUDEBUG_EVENT_VM_BIND_OP +#define DRM_XE_EUDEBUG_EVENT_VM_BIND_UFENCE 8 +#define DRM_XE_EUDEBUG_EVENT_MAX_EVENT DRM_XE_EUDEBUG_EVENT_VM_BIND_UFENCE__u16 flags;#define DRM_XE_EUDEBUG_EVENT_CREATE (1 << 0)@@ -158,6 +160,17 @@ struct drm_xe_eudebug_event_vm_bind_op {__u64 range;/* XXX: Zero for unmap all? */ }; +struct drm_xe_eudebug_event_vm_bind_ufence { +struct drm_xe_eudebug_event base; +__u64 vm_bind_ref_seqno;/* *_event_vm_bind.base.seqno */ +}; + +struct drm_xe_eudebug_ack_event { +__u32 type; +__u32 flags;/* MBZ */ +__u64 seqno; +}; +#if defined(__cplusplus) }#endif Signature · 1 line
2.34.1
PATCH 14/21 drm/xe/eudebug: vm open/pread/pwrite
In reply to Mika KuoppalaDebugger needs access to the client's vm to read and write. For example inspecting ISA/ELF and setting up breakpoints.
Add ioctl to open target vm with debugger client and vm_handle and hook up pread/pwrite possibility.
Open will take timeout argument so that standard fsync can be used for explicit flushing between cpu/gpu for the target vm.
Implement this for bo backed storage. userptr will be done in following patch.
v2: checkpatch (Maciej)
--- drivers/gpu/drm/xe/regs/xe_gt_regs.h | 24 ++ drivers/gpu/drm/xe/xe_eudebug.c | 470 +++++++++++++++++++++++++++ include/uapi/drm/xe_drm_eudebug.h | 18 + 3 files changed, 512 insertions(+)
drivers/gpu/drm/xe/regs/xe_gt_regs.h +24 -0
@@ -520,6 +520,30 @@#define CCS_MODE_CSLICE (cslice, ccs) \ ((ccs) << ((cslice) * CCS_MODE_CSLICE_WIDTH)) +#define RCU_ASYNC_FLUSH XE_REG(0x149fc) +#define RCU_ASYNC_FLUSH_IN_PROGRESS REG_BIT(31) +#define RCU_ASYNC_FLUSH_ENGINE_ID_SHIFT 28 +#define RCU_ASYNC_FLUSH_ENGINE_ID_DECODE1 REG_BIT(26) +#define RCU_ASYNC_FLUSH_AMFS REG_BIT(8) +#define RCU_ASYNC_FLUSH_PREFETCH REG_BIT(7) +#define RCU_ASYNC_FLUSH_DATA_PORT REG_BIT(6) +#define RCU_ASYNC_FLUSH_DATA_CACHE REG_BIT(5) +#define RCU_ASYNC_FLUSH_HDC_PIPELINE REG_BIT(4) +#define RCU_ASYNC_INVALIDATE_HDC_PIPELINE REG_BIT(3) +#define RCU_ASYNC_INVALIDATE_CONSTANT_CACHE REG_BIT(2) +#define RCU_ASYNC_INVALIDATE_TEXTURE_CACHE REG_BIT(1) +#define RCU_ASYNC_INVALIDATE_INSTRUCTION_CACHE REG_BIT(0) +#define RCU_ASYNC_FLUSH_AND_INVALIDATE_ALL ( \ + RCU_ASYNC_FLUSH_AMFS | \ + RCU_ASYNC_FLUSH_PREFETCH | \ + RCU_ASYNC_FLUSH_DATA_PORT | \ + RCU_ASYNC_FLUSH_DATA_CACHE | \ + RCU_ASYNC_FLUSH_HDC_PIPELINE | \ + RCU_ASYNC_INVALIDATE_HDC_PIPELINE | \ + RCU_ASYNC_INVALIDATE_CONSTANT_CACHE | \ + RCU_ASYNC_INVALIDATE_TEXTURE_CACHE | \ + RCU_ASYNC_INVALIDATE_INSTRUCTION_CACHE) +#define RCU_DEBUG_1 XE_REG(0x14a00)#define RCU_DEBUG_1_ENGINE_STATUS REG_GENMASK(2, 0)#define RCU_DEBUG_1_RUNALONE_ACTIVE REG_BIT(2)
drivers/gpu/drm/xe/xe_eudebug.c +470 -0
504 lines not on this page · raw message
include/uapi/drm/xe_drm_eudebug.h +18 -0
@@ -18,6 +18,7 @@ extern "C" {#define DRM_XE_EUDEBUG_IOCTL_READ_EVENT _IO('j', 0x0)#define DRM_XE_EUDEBUG_IOCTL_EU_CONTROL _IOWR('j', 0x2, struct drm_xe_eudebug_eu_control)#define DRM_XE_EUDEBUG_IOCTL_ACK_EVENT _IOW('j', 0x4, struct drm_xe_eudebug_ack_event) +#define DRM_XE_EUDEBUG_IOCTL_VM_OPEN _IOW('j', 0x1, struct drm_xe_eudebug_vm_open)/* XXX: Document events to match their internal counterparts when moved to xe_drm.h */ struct drm_xe_eudebug_event {
@@ -171,6 +172,23 @@ struct drm_xe_eudebug_ack_event {__u64 seqno; }; +struct drm_xe_eudebug_vm_open { +/** @extensions: Pointer to the first extension struct, if any */ +__u64 extensions; + +/** @client_handle: id of client */ +__u64 client_handle; + +/** @vm_handle: id of vm */ +__u64 vm_handle; + +/** @flags: flags */ +__u64 flags; + +/** @timeout_ns: Timeout value in nanoseconds operations (fsync) */ +__u64 timeout_ns; +}; +#if defined(__cplusplus) }#endif
Signature · 1 line
2.34.1