Skip to contentSkip to conversation

drm/xe/client: Check return value of xe_force_wake_get

PATCH In review Nirmoy Das 3 participants 5 messages Jun 3, 2024

Subsystems (2)
lore
Share & tools
lore

Nirmoy Das

author
#0 lore raw

PATCH v2 1/1 drm/xe/client: Check return value of xe_force_wake_get

xe_force_wake_get() can return error so check it's return value before reading gpu_timestamp value.

v2: set HWE to NULL instead of setting timestamp to 0(Lucas)
Add a warn on for xe_force_wake_put(Himal)

Fixes 188ced1e0ff8 ("drm/xe/client: Print runtime to fdinfo")
Cc Himal Prasad Ghimiray <[email protected]>
Cc Lucas De Marchi <[email protected]>
Cc Rodrigo Vivi <[email protected]>
---
 drivers/gpu/drm/xe/xe_drm_client.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
drivers/gpu/drm/xe/xe_drm_client.c +6 -2
index 4a19b771e3a0..4bdb909eabfc 100644
@@ -264,9 +264,13 @@ static void show_run_ticks(struct drm_printer *p, struct drm_file *file)
 		if (!hwe)
 			continue;
 
-		xe_force_wake_get(gt_to_fw(gt), XE_FW_GT);
+		if (xe_force_wake_get(gt_to_fw(gt), XE_FW_GT)) {
+			hwe = NULL;
+			break;
+		}
+
 		gpu_timestamp = xe_hw_engine_read_timestamp(hwe);
-		xe_force_wake_put(gt_to_fw(gt), XE_FW_GT);
+		XE_WARN_ON(xe_force_wake_put(gt_to_fw(gt), XE_FW_GT));
 		break;
 	}
 
Signature · 1 line
2.42.0
#1 lore raw
Quoted 20 lines · Nirmoy Das

xe_force_wake_get() can return error so check it's return value before reading gpu_timestamp value.

v2: set HWE to NULL instead of setting timestamp to 0(Lucas)
Add a warn on for xe_force_wake_put(Himal)

Fixes 188ced1e0ff8 ("drm/xe/client: Print runtime to fdinfo")
Cc Himal Prasad Ghimiray <[email protected]>
Cc Lucas De Marchi <[email protected]>
Cc Rodrigo Vivi <[email protected]>
---
  drivers/gpu/drm/xe/xe_drm_client.c | 8 ++++++--
  1 file changed, 6 insertions(+), 2 deletions(-)
drivers/gpu/drm/xe/xe_drm_client.c +0 -0
index 4a19b771e3a0..4bdb909eabfc 100644

drivers/gpu/drm/xe/xe_drm_client.c static void show_run_ticks(struct drm_printer *p, struct drm_file *file) · line 267

Quoted 5 lines
drivers/gpu/drm/xe/xe_drm_client.c +1 -1
@@ -264,9 +264,13 @@ static void show_run_ticks(struct drm_printer *p, struct drm_file *file)
  		if (!hwe)
  			continue;
 
-		xe_force_wake_get(gt_to_fw(gt), XE_FW_GT);
+		if (xe_force_wake_get(gt_to_fw(gt), XE_FW_GT)) {

Since xe_force_wake_get will increase the ref count irrespective of acknowledgment from forcewake ack register fails or succeeds.

It is required to decrement the ref count taken by xe_force_wake_get by calling xe_force_wake_put before break.

something like :

if (xe_force_wake_get(gt_to_fw(gt), XE_FW_GT))

hwe = NULL;

else

gpu_timestamp = xe_hw_engine_read_timestamp(hwe);

XE_WARN_ON(xe_force_wake_put(gt_to_fw(gt), XE_FW_GT));

Quoted 10 lines
drivers/gpu/drm/xe/xe_drm_client.c +5 -1
+			hwe = NULL;
+			break;
+		}
+
  		gpu_timestamp = xe_hw_engine_read_timestamp(hwe);
-		xe_force_wake_put(gt_to_fw(gt), XE_FW_GT);
+		XE_WARN_ON(xe_force_wake_put(gt_to_fw(gt), XE_FW_GT));
  		break;
  	}
 
#2 lore raw
Quoted 17 lines · Himal Prasad
Quoted 15 lines · Nirmoy Das

xe_force_wake_get() can return error so check it's return value before reading gpu_timestamp value.

v2: set HWE to NULL instead of setting timestamp to 0(Lucas) Add a warn on for xe_force_wake_put(Himal)

Fixes 188ced1e0ff8 ("drm/xe/client: Print runtime to fdinfo")
Cc Himal Prasad Ghimiray <[email protected]>
Cc Lucas De Marchi <[email protected]>
Cc Rodrigo Vivi <[email protected]>

---
  drivers/gpu/drm/xe/xe_drm_client.c | 8 ++++++--

  1 file changed, 6 insertions(+), 2 deletions(-)
Quoted 12 lines

drivers/gpu/drm/xe/xe_drm_client.c static void show_run_ticks(struct drm_printer · line 268

Quoted 10 lines
drivers/gpu/drm/xe/xe_drm_client.c +1 -0
index 4a19b771e3a0..4bdb909eabfc 100644
@@ -264,9 +264,13 @@ static void show_run_ticks(struct drm_printer 
 *p, struct drm_file *file)
           if (!hwe)
               continue;
   -        xe_force_wake_get(gt_to_fw(gt), XE_FW_GT);
+        if (xe_force_wake_get(gt_to_fw(gt), XE_FW_GT)) {

Since xe_force_wake_get will increase the ref count irrespective of acknowledgment from forcewake ack register fails or succeeds.

General pattern is to skip xe_force_wake_put() though at some places we do call xe_force_wake_put() when the caller is unabortable.

I don't know the exact reason but I  guess failure to wake a domain is serious enough so we want to  keep already woken domains  in wake state for system stability.

Regards,

Nirmoy

Quoted 25 lines

It is required to decrement the ref count taken by xe_force_wake_get by calling xe_force_wake_put before break.

something like :

if (xe_force_wake_get(gt_to_fw(gt), XE_FW_GT))

hwe = NULL;

else

gpu_timestamp = xe_hw_engine_read_timestamp(hwe);

XE_WARN_ON(xe_force_wake_put(gt_to_fw(gt), XE_FW_GT));

Quoted 9 lines
drivers/gpu/drm/xe/xe_drm_client.c +5 -1
+            hwe = NULL;
+            break;
+        }
+
           gpu_timestamp = xe_hw_engine_read_timestamp(hwe);
-        xe_force_wake_put(gt_to_fw(gt), XE_FW_GT);
+        XE_WARN_ON(xe_force_wake_put(gt_to_fw(gt), XE_FW_GT));
           break;
       }
#3 lore raw
Quoted 32 lines · Nirmoy Das
Quoted 17 lines · Himal Prasad
Quoted 15 lines · Nirmoy Das

xe_force_wake_get() can return error so check it's return value before reading gpu_timestamp value.

v2: set HWE to NULL instead of setting timestamp to 0(Lucas) Add a warn on for xe_force_wake_put(Himal)

Fixes 188ced1e0ff8 ("drm/xe/client: Print runtime to fdinfo")
Cc Himal Prasad Ghimiray <[email protected]>
Cc Lucas De Marchi <[email protected]>
Cc Rodrigo Vivi <[email protected]>

---
  drivers/gpu/drm/xe/xe_drm_client.c | 8 ++++++--

  1 file changed, 6 insertions(+), 2 deletions(-)
Quoted 12 lines

drivers/gpu/drm/xe/xe_drm_client.c static void show_run_ticks(struct drm_printer · line 268

Quoted 10 lines
drivers/gpu/drm/xe/xe_drm_client.c +1 -0
index 4a19b771e3a0..4bdb909eabfc 100644
@@ -264,9 +264,13 @@ static void show_run_ticks(struct drm_printer 
 *p, struct drm_file *file)
           if (!hwe)
               continue;
   -        xe_force_wake_get(gt_to_fw(gt), XE_FW_GT);
+        if (xe_force_wake_get(gt_to_fw(gt), XE_FW_GT)) {

Since xe_force_wake_get will increase the ref count irrespective of acknowledgment from forcewake ack register fails or succeeds.

Quoted 3 lines

General pattern is to skip xe_force_wake_put() though at some places we do call xe_force_wake_put() when the caller is unabortable.

I don't know the exact reason but I  guess failure to wake a domain is serious enough so we want to  keep already woken domains  in wake state for system stability.

If you break without xe_force_Wake_put, subsequent force_wake_get will assume domain is awake(which might not be true, because there was no acknowledgment)

Please consider below scenario:

function a:
xe_force_wake_get ---> increased domain ref from 0 to 1 -----> wake domain ---> timedout the acknowledgment -> failed and returned without decreasing the ref count

read/write operations -> not called
xe_force_wake_put -> not called

function b : called after function a
xe_force_wake_get -----> domain ref increased to 2 ----> exit success read/write operations -> executed but domain was not awake xe_force_wake_put ----> domain ref decreased to 1 again.

BR
Himal

Quoted 32 lines

Regards,

Nirmoy

Quoted 25 lines

It is required to decrement the ref count taken by xe_force_wake_get by calling xe_force_wake_put before break.

something like :

if (xe_force_wake_get(gt_to_fw(gt), XE_FW_GT))

hwe = NULL;

else

gpu_timestamp = xe_hw_engine_read_timestamp(hwe);

XE_WARN_ON(xe_force_wake_put(gt_to_fw(gt), XE_FW_GT));

Quoted 9 lines
drivers/gpu/drm/xe/xe_drm_client.c +5 -1
+            hwe = NULL;
+            break;
+        }
+
           gpu_timestamp = xe_hw_engine_read_timestamp(hwe);
-        xe_force_wake_put(gt_to_fw(gt), XE_FW_GT);
+        XE_WARN_ON(xe_force_wake_put(gt_to_fw(gt), XE_FW_GT));
           break;
       }
#4 lore raw

Hi Himal,

Quoted 46 lines · Himal Prasad
Quoted 32 lines · Nirmoy Das
Quoted 17 lines · Himal Prasad
Quoted 15 lines · Nirmoy Das

xe_force_wake_get() can return error so check it's return value before reading gpu_timestamp value.

v2: set HWE to NULL instead of setting timestamp to 0(Lucas) Add a warn on for xe_force_wake_put(Himal)

Fixes 188ced1e0ff8 ("drm/xe/client: Print runtime to fdinfo")
Cc Himal Prasad Ghimiray <[email protected]>
Cc Lucas De Marchi <[email protected]>
Cc Rodrigo Vivi <[email protected]>

---
  drivers/gpu/drm/xe/xe_drm_client.c | 8 ++++++--

  1 file changed, 6 insertions(+), 2 deletions(-)
Quoted 12 lines

drivers/gpu/drm/xe/xe_drm_client.c static void show_run_ticks(struct drm_printer · line 268

Quoted 10 lines
drivers/gpu/drm/xe/xe_drm_client.c +1 -0
index 4a19b771e3a0..4bdb909eabfc 100644
@@ -264,9 +264,13 @@ static void show_run_ticks(struct drm_printer 
 *p, struct drm_file *file)
           if (!hwe)
               continue;
   -        xe_force_wake_get(gt_to_fw(gt), XE_FW_GT);
+        if (xe_force_wake_get(gt_to_fw(gt), XE_FW_GT)) {

Since xe_force_wake_get will increase the ref count irrespective of acknowledgment from forcewake ack register fails or succeeds.

Quoted 3 lines

General pattern is to skip xe_force_wake_put() though at some places we do call xe_force_wake_put() when the caller is unabortable.

I don't know the exact reason but I  guess failure to wake a domain is serious enough so we want to  keep already woken domains  in wake state for system stability.

If you break without xe_force_Wake_put, subsequent force_wake_get will assume domain is awake(which might not be true, because there was no acknowledgment)

Please consider below scenario:

function a:
xe_force_wake_get ---> increased domain ref from 0 to 1 -----> wake domain ---> timedout the acknowledgment -> failed and returned without decreasing the ref count

Quoted 7 lines

read/write operations  -> not called
xe_force_wake_put -> not called

function b : called after function a
xe_force_wake_get -----> domain ref increased to 2 ----> exit success read/write operations -> executed but domain was not awake xe_force_wake_put ----> domain ref decreased to 1 again.

I assumed this was known to everyone as a known side effect, as we mostly skip calling xe_force_wake_put(), but I guess it is not.

Had a chat with Rodrigo about this, and he acknowledged the current inconsistency and the need for cleanup. Going to create a series

to add the missing xe_force_wake_put() calls where we haven't been doing it.

Thanks,

Nirmoy

Quoted 37 lines

BR
Himal

Quoted 32 lines

Regards,

Nirmoy

Quoted 25 lines

It is required to decrement the ref count taken by xe_force_wake_get by calling xe_force_wake_put before break.

something like :

if (xe_force_wake_get(gt_to_fw(gt), XE_FW_GT))

hwe = NULL;

else

gpu_timestamp = xe_hw_engine_read_timestamp(hwe);

XE_WARN_ON(xe_force_wake_put(gt_to_fw(gt), XE_FW_GT));

Quoted 9 lines
drivers/gpu/drm/xe/xe_drm_client.c +5 -1
+            hwe = NULL;
+            break;
+        }
+
           gpu_timestamp = xe_hw_engine_read_timestamp(hwe);
-        xe_force_wake_put(gt_to_fw(gt), XE_FW_GT);
+        XE_WARN_ON(xe_force_wake_put(gt_to_fw(gt), XE_FW_GT));
           break;
       }