Skip to contentSkip to conversation

drm/amdgpu: Convert to common fdinfo format

RFC In review Tvrtko Ursulin 2 participants 5 messages May 10, 2022

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

RFC 1/1 drm/amdgpu: Convert to common fdinfo format

From: Tvrtko Ursulin <[email protected]>

Convert fdinfo format to one documented in drm-usage-stats.rst.

Opens/TODO:
* Does someone from AMD want to take over this patch?
(I have no access to amdgpu hardware so won't be able to test
any hypothetical gputop work.)
* What are the semantics of AMD engine utilisation reported in percents?
* Can it align with what i915 does (same what msm will do) or need to document the alternative in the specification document? Both option are workable with instantaneous percent only needing support to be added to vendor agnostic gputop.
* Can amdgpu expose drm-client-id? Without it gputop will not work.
* drm-engine-capacity - does the concept translate etc.

---
 Documentation/gpu/amdgpu/usage-stats.rst   | 28 ++++++++++++++++++++++
 Documentation/gpu/drm-usage-stats.rst      |  7 +++++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_fdinfo.c | 18 ++++++++++----
 3 files changed, 47 insertions(+), 6 deletions(-)
 create mode 100644 Documentation/gpu/amdgpu/usage-stats.rst
Documentation/gpu/amdgpu/usage-stats.rst new file +28 -0
new file mode 100644
index 000000000000..8be5009bd1a9
@@ -0,0 +1,28 @@
+.. _amdgpu-usage-stats:
+
+============================================
+AMDGPU DRM client usage stats implementation
+============================================
+
+The amdgpu driver implements the DRM client usage stats specification as
+documented in :ref:`drm-client-usage-stats`.
+
+Example of the output showing the implemented key value pairs and entirety of
+the currenly possible format options:
+
+::
+
+      pos:    0
+      flags:  0100002
+      mnt_id: 21
+      drm-driver: amdgpu
+      drm-pdev:   0000:03:00.0
+      drm-memory-vram: 0 KiB
+      drm-memory-gtt: 0 KiB
+      drm-memory-cpu: 0 KiB
+      drm-engine-...: 0 %
+                 ...
+
+Possible `drm-memory-` key names are: `vram`, `gtt`, `cpu`.
+
+Possible `drm-engine-` key names are: ``.
Documentation/gpu/drm-usage-stats.rst +6 -1
index 6c9f166a8d6f..2d0ff6f2cc74 100644
@@ -69,7 +69,7 @@ scope of each device, in which case `drm-pdev` shall be present as well.
 Userspace should make sure to not double account any usage statistics by using
 the above described criteria in order to associate data to individual clients.
 
-- drm-engine-<str>: <uint> ns
+- drm-engine-<str>: <uint> [ns|%]
 
 GPUs usually contain multiple execution engines. Each shall be given a stable
 and unique name (str), with possible values documented in the driver specific
@@ -84,6 +84,9 @@ larger value within a reasonable period. Upon observing a value lower than what
 was previously read, userspace is expected to stay with that larger previous
 value until a monotonic update is seen.
 
+Where time unit is given as a percentage...[AMD folks to fill the semantics
+and interpretation of that]...
+
 - drm-engine-capacity-<str>: <uint>
 
 Engine identifier string must be the same as the one specified in the
@@ -110,3 +113,5 @@ Driver specific implementations
 ===============================
 
 :ref:`i915-usage-stats`
+
+:ref:`amdgpu-usage-stats`
drivers/gpu/drm/amd/amdgpu/amdgpu_fdinfo.c +13 -5
index 5a6857c44bb6..8cbae61f1b3b 100644
@@ -32,6 +32,7 @@
 
 #include <drm/amdgpu_drm.h>
 #include <drm/drm_debugfs.h>
+#include <drm/drm_drv.h>
 
 #include "amdgpu.h"
 #include "amdgpu_vm.h"
@@ -83,11 +84,18 @@ void amdgpu_show_fdinfo(struct seq_file *m, struct file *f)
 	amdgpu_bo_unreserve(root);
 	amdgpu_bo_unref(&root);
 
-	seq_printf(m, "pdev:\t%04x:%02x:%02x.%d\npasid:\t%u\n", domain, bus,
+	/*
+	 * ******************************************************************
+	 * For text output format description please see drm-usage-stats.rst!
+	 * ******************************************************************
+	 */
+
+	seq_printf(m, "drm-driver:\t%s\n", file->minor->dev->driver->name);
+	seq_printf(m, "drm-pdev:\t%04x:%02x:%02x.%d\npasid:\t%u\n", domain, bus,
 			dev, fn, fpriv->vm.pasid);
-	seq_printf(m, "vram mem:\t%llu kB\n", vram_mem/1024UL);
-	seq_printf(m, "gtt mem:\t%llu kB\n", gtt_mem/1024UL);
-	seq_printf(m, "cpu mem:\t%llu kB\n", cpu_mem/1024UL);
+	seq_printf(m, "drm-memory-vram:\t%llu KiB\n", vram_mem/1024UL);
+	seq_printf(m, "drm-memory-gtt:\t%llu KiB\n", gtt_mem/1024UL);
+	seq_printf(m, "drm-memory-cpu:\t%llu KiB\n", cpu_mem/1024UL);
 	for (i = 0; i < AMDGPU_HW_IP_NUM; i++) {
 		uint32_t count = amdgpu_ctx_num_entities[i];
 		int idx = 0;
@@ -103,7 +111,7 @@ void amdgpu_show_fdinfo(struct seq_file *m, struct file *f)
 			perc = div64_u64(10000 * total, min);
 			frac = perc % 100;
 
-			seq_printf(m, "%s%d:\t%d.%d%%\n",
+			seq_printf(m, "drm-engine-%s%d:\t%d.%d %%\n",
 					amdgpu_ip_name[i],
 					idx, perc/100, frac);
 		}
Signature · 1 line
2.32.0
#1 lore raw

Hi Tvrtko,

Am 10.05.22 um 10:23 schrieb Tvrtko Ursulin:

Quoted 8 lines

From: Tvrtko Ursulin <[email protected]>

Convert fdinfo format to one documented in drm-usage-stats.rst.

Opens/TODO:
* Does someone from AMD want to take over this patch? (I have no access to amdgpu hardware so won't be able to test any hypothetical gputop work.)

I can give that a try as soon as it is completed.

Quoted 1 line

* What are the semantics of AMD engine utilisation reported in percents?

To be honest I haven't understood why we are using percents here either, that is not something the kernel should mess with.

Quoted 4 lines

* Can it align with what i915 does (same what msm will do) or need to document the alternative in the specification document? Both option are workable with instantaneous percent only needing support to be added to vendor agnostic gputop.

I would prefer to just change to the ns format i915 and msm will be using, that makes much more sense from my experience.

As far as I know we haven't released any publicly available userspace using the existing AMD specific format. So that should still be possible.

Quoted 1 line

* Can amdgpu expose drm-client-id? Without it gputop will not work.

How is that determined on i915 ? Does struct drm_file has that somewhere?

Quoted 1 line

* drm-engine-capacity - does the concept translate etc.

I don't think we are going to need that.

Regards,
Christian.

Quoted 120 lines
---
  Documentation/gpu/amdgpu/usage-stats.rst   | 28 ++++++++++++++++++++++
  Documentation/gpu/drm-usage-stats.rst      |  7 +++++-
  drivers/gpu/drm/amd/amdgpu/amdgpu_fdinfo.c | 18 ++++++++++----
  3 files changed, 47 insertions(+), 6 deletions(-)

create mode 100644 Documentation/gpu/amdgpu/usage-stats.rst

Documentation/gpu/amdgpu/usage-stats.rst new file +28 -0
new file mode 100644
index 000000000000..8be5009bd1a9
@@ -0,0 +1,28 @@
+.. _amdgpu-usage-stats:
+
+============================================
+AMDGPU DRM client usage stats implementation
+============================================
+
+The amdgpu driver implements the DRM client usage stats specification as
+documented in :ref:`drm-client-usage-stats`.
+
+Example of the output showing the implemented key value pairs and entirety of
+the currenly possible format options:
+
+::
+
+      pos:    0
+      flags:  0100002
+      mnt_id: 21
+      drm-driver: amdgpu
+      drm-pdev:   0000:03:00.0
+      drm-memory-vram: 0 KiB
+      drm-memory-gtt: 0 KiB
+      drm-memory-cpu: 0 KiB
+      drm-engine-...: 0 %
+                 ...
+
+Possible `drm-memory-` key names are: `vram`, `gtt`, `cpu`.
+
+Possible `drm-engine-` key names are: ``.
Documentation/gpu/drm-usage-stats.rst +6 -1
index 6c9f166a8d6f..2d0ff6f2cc74 100644
@@ -69,7 +69,7 @@ scope of each device, in which case `drm-pdev` shall be present as well.
  Userspace should make sure to not double account any usage statistics by using
  the above described criteria in order to associate data to individual clients.
 
-- drm-engine-<str>: <uint> ns
+- drm-engine-<str>: <uint> [ns|%]
 
  GPUs usually contain multiple execution engines. Each shall be given a stable
  and unique name (str), with possible values documented in the driver specific
@@ -84,6 +84,9 @@ larger value within a reasonable period. Upon observing a value lower than what
  was previously read, userspace is expected to stay with that larger previous
  value until a monotonic update is seen.
 
+Where time unit is given as a percentage...[AMD folks to fill the semantics
+and interpretation of that]...
+
  - drm-engine-capacity-<str>: <uint>
 
  Engine identifier string must be the same as the one specified in the
@@ -110,3 +113,5 @@ Driver specific implementations
  ===============================
 
  :ref:`i915-usage-stats`
+
+:ref:`amdgpu-usage-stats`
drivers/gpu/drm/amd/amdgpu/amdgpu_fdinfo.c +13 -5
index 5a6857c44bb6..8cbae61f1b3b 100644
@@ -32,6 +32,7 @@
 
  #include <drm/amdgpu_drm.h>
  #include <drm/drm_debugfs.h>
+#include <drm/drm_drv.h>
 
  #include "amdgpu.h"
  #include "amdgpu_vm.h"
@@ -83,11 +84,18 @@ void amdgpu_show_fdinfo(struct seq_file *m, struct file *f)
  	amdgpu_bo_unreserve(root);
  	amdgpu_bo_unref(&root);
 
-	seq_printf(m, "pdev:\t%04x:%02x:%02x.%d\npasid:\t%u\n", domain, bus,
+	/*
+	 * ******************************************************************
+	 * For text output format description please see drm-usage-stats.rst!
+	 * ******************************************************************
+	 */
+
+	seq_printf(m, "drm-driver:\t%s\n", file->minor->dev->driver->name);
+	seq_printf(m, "drm-pdev:\t%04x:%02x:%02x.%d\npasid:\t%u\n", domain, bus,
  			dev, fn, fpriv->vm.pasid);
-	seq_printf(m, "vram mem:\t%llu kB\n", vram_mem/1024UL);
-	seq_printf(m, "gtt mem:\t%llu kB\n", gtt_mem/1024UL);
-	seq_printf(m, "cpu mem:\t%llu kB\n", cpu_mem/1024UL);
+	seq_printf(m, "drm-memory-vram:\t%llu KiB\n", vram_mem/1024UL);
+	seq_printf(m, "drm-memory-gtt:\t%llu KiB\n", gtt_mem/1024UL);
+	seq_printf(m, "drm-memory-cpu:\t%llu KiB\n", cpu_mem/1024UL);
  	for (i = 0; i < AMDGPU_HW_IP_NUM; i++) {
  		uint32_t count = amdgpu_ctx_num_entities[i];
  		int idx = 0;
@@ -103,7 +111,7 @@ void amdgpu_show_fdinfo(struct seq_file *m, struct file *f)
  			perc = div64_u64(10000 * total, min);
  			frac = perc % 100;
 
-			seq_printf(m, "%s%d:\t%d.%d%%\n",
+			seq_printf(m, "drm-engine-%s%d:\t%d.%d %%\n",
  					amdgpu_ip_name[i],
  					idx, perc/100, frac);
  		}
#2 lore raw

Hi,

Quoted 5 lines · Christian König

Hi Tvrtko,

Am 10.05.22 um 10:23 schrieb Tvrtko Ursulin:

Quoted 2 lines

From: Tvrtko Ursulin <[email protected]>

Quoted 8 lines
Quoted 6 lines

Convert fdinfo format to one documented in drm-usage-stats.rst.

Opens/TODO:
  * Does someone from AMD want to take over this patch? (I have no access to amdgpu hardware so won't be able to test any hypothetical gputop work.)

I can give that a try as soon as it is completed.

And how to motivate someone on your side to pick up the amdgpu work? :)

Quoted 4 lines
Quoted 1 line

  * What are the semantics of AMD engine utilisation reported in percents?

To be honest I haven't understood why we are using percents here either, that is not something the kernel should mess with.

Quoted 8 lines
Quoted 4 lines

     * Can it align with what i915 does (same what msm will do) or need to document the alternative in the specification document? Both option are workable with instantaneous percent only needing support to be added to vendor agnostic gputop.

I would prefer to just change to the ns format i915 and msm will be using, that makes much more sense from my experience.

As far as I know we haven't released any publicly available userspace using the existing AMD specific format. So that should still be possible.

If amdgpu could export accumulated time context spent on engines that would indeed be perfect. It would make the gputop I sketched out most probably just work, as it did for Rob on msm.

In which case, apart from the admgpu work, it would just be a matter of me tidying that tool a bit and re-sending out for review.

Quoted 3 lines
Quoted 1 line

  * Can amdgpu expose drm-client-id? Without it gputop will not work.

How is that determined on i915 ? Does struct drm_file has that somewhere?

It should correspond 1:1 with drm_file, since the purpose is to enable gputop distinguish between unique open file descriptors (aka clients).

In theory it could be just a hash value of a struct drm_file pointer but that could confuse userspace if the struct gets reused within a single userspace sampling period.

Because of that I track it in i915 separately since I wanted to have an incrementing cyclic property to it - so that when a fd is closed and new opened there is no practical chance they would have the same drm-client-id.

Quoted 3 lines
Quoted 1 line

* drm-engine-capacity - does the concept translate etc.

I don't think we are going to need that.

Okay, that one is optional for cases when there is more than one engine of a type/class shown under a single entry in fdinfo. So when gputop translates accumulated time into percentages it can do the right thing. Code can already handle it not being present and assume one.

Regards,

Tvrtko

Quoted 126 lines

Regards,
Christian.

Quoted 121 lines

---
  Documentation/gpu/amdgpu/usage-stats.rst   | 28 ++++++++++++++++++++++ Documentation/gpu/drm-usage-stats.rst      |  7 +++++- drivers/gpu/drm/amd/amdgpu/amdgpu_fdinfo.c | 18 ++++++++++----

  3 files changed, 47 insertions(+), 6 deletions(-)

  create mode 100644 Documentation/gpu/amdgpu/usage-stats.rst

Documentation/gpu/amdgpu/usage-stats.rst new file +10 -0
new file mode 100644
index 000000000000..8be5009bd1a9
@@ -0,0 +1,28 @@
+.. _amdgpu-usage-stats:
+
+============================================
+AMDGPU DRM client usage stats implementation
+============================================
+
+The amdgpu driver implements the DRM client usage stats specification as
+documented in :ref:`drm-client-usage-stats`.
+
+Example of the output showing the implemented key value pairs and 

entirety of

Documentation/gpu/amdgpu/usage-stats.rst new file +18 -0
+the currenly possible format options:
+
+::
+
+      pos:    0
+      flags:  0100002
+      mnt_id: 21
+      drm-driver: amdgpu
+      drm-pdev:   0000:03:00.0
+      drm-memory-vram: 0 KiB
+      drm-memory-gtt: 0 KiB
+      drm-memory-cpu: 0 KiB
+      drm-engine-...: 0 %
+                 ...
+
+Possible `drm-memory-` key names are: `vram`, `gtt`, `cpu`.
+
+Possible `drm-engine-` key names are: ``.
Documentation/gpu/drm-usage-stats.rst +0 -0
index 6c9f166a8d6f..2d0ff6f2cc74 100644
@@ -69,7 +69,7 @@ scope of each device, in which case `drm-pdev` shall 

be present as well.
  Userspace should make sure to not double account any usage statistics by using the above described criteria in order to associate data to individual clients. -- drm-engine-<str>: <uint> ns
+- drm-engine-<str>: <uint> [ns|%]
  GPUs usually contain multiple execution engines. Each shall be given a stable and unique name (str), with possible values documented in the driver specific

Documentation/gpu/drm-usage-stats.rst +0 -0
@@ -84,6 +84,9 @@ larger value within a reasonable period. Upon 

observing a value lower than what
  was previously read, userspace is expected to stay with that larger previous value until a monotonic update is seen.
+Where time unit is given as a percentage...[AMD folks to fill the semantics

Documentation/gpu/drm-usage-stats.rst +2 -0
+and interpretation of that]...
+

  - drm-engine-capacity-<str>: <uint>
  Engine identifier string must be the same as the one specified in the

Documentation/gpu/drm-usage-stats.rst +0 -0
@@ -110,3 +113,5 @@ Driver specific implementations

  ===============================
  :ref:i915-usage-stats

Documentation/gpu/drm-usage-stats.rst +2 -0
+
+:ref:`amdgpu-usage-stats`
drivers/gpu/drm/amd/amdgpu/amdgpu_fdinfo.c +4 -1
index 5a6857c44bb6..8cbae61f1b3b 100644
@@ -32,6 +32,7 @@
   #include <drm/amdgpu_drm.h>
   #include <drm/drm_debugfs.h>
+#include <drm/drm_drv.h>
   #include "amdgpu.h"
   #include "amdgpu_vm.h"
@@ -83,11 +84,18 @@ void amdgpu_show_fdinfo(struct seq_file *m, struct 
 file *f)
       amdgpu_bo_unreserve(root);
       amdgpu_bo_unref(&root);
-    seq_printf(m, "pdev:\t%04x:%02x:%02x.%d\npasid:\t%u\n", domain, bus,
+    /*
+     * 
 ******************************************************************
+     * For text output format description please see 

drm-usage-stats.rst!
+     *
******************************************************************
+     */
+
+    seq_printf(m, "drm-driver:\t%s\n", file->minor->dev->driver->name); +    seq_printf(m, "drm-pdev:\t%04x:%02x:%02x.%d\npasid:\t%u\n", domain, bus,
              dev, fn, fpriv->vm.pasid);
-    seq_printf(m, "vram mem:\t%llu kB\n", vram_mem/1024UL); -    seq_printf(m, "gtt mem:\t%llu kB\n", gtt_mem/1024UL);
-    seq_printf(m, "cpu mem:\t%llu kB\n", cpu_mem/1024UL);
+    seq_printf(m, "drm-memory-vram:\t%llu KiB\n", vram_mem/1024UL); +    seq_printf(m, "drm-memory-gtt:\t%llu KiB\n", gtt_mem/1024UL); +    seq_printf(m, "drm-memory-cpu:\t%llu KiB\n", cpu_mem/1024UL); for (i = 0; i < AMDGPU_HW_IP_NUM; i++) {
          uint32_t count = amdgpu_ctx_num_entities[i];
          int idx = 0;

drivers/gpu/drm/amd/amdgpu/amdgpu_fdinfo.c +1 -1
@@ -103,7 +111,7 @@ void amdgpu_show_fdinfo(struct seq_file *m, struct 
 file *f)
               perc = div64_u64(10000 * total, min);
               frac = perc % 100;
-            seq_printf(m, "%s%d:\t%d.%d%%\n",
+            seq_printf(m, "drm-engine-%s%d:\t%d.%d %%\n",
                       amdgpu_ip_name[i],
                       idx, perc/100, frac);
           }
#3 lore raw

Am 10.05.22 um 12:50 schrieb Tvrtko Ursulin:

Quoted 11 lines

Hi,

Quoted 7 lines · Christian König

Hi Tvrtko,

Am 10.05.22 um 10:23 schrieb Tvrtko Ursulin:

Quoted 4 lines

From: Tvrtko Ursulin <[email protected]>

Convert fdinfo format to one documented in drm-usage-stats.rst.

Quoted 8 lines
Quoted 6 lines
Quoted 4 lines

Opens/TODO:
  * Does someone from AMD want to take over this patch? (I have no access to amdgpu hardware so won't be able to test any hypothetical gputop work.)

I can give that a try as soon as it is completed.

And how to motivate someone on your side to pick up the amdgpu work? :)

Well if we could get more of my TTM/DRM patches reviewed I could have same free time to do this :)

Quoted 9 lines
Quoted 9 lines
Quoted 1 line

  * What are the semantics of AMD engine utilisation reported in percents?

To be honest I haven't understood why we are using percents here either, that is not something the kernel should mess with.

Quoted 4 lines

     * Can it align with what i915 does (same what msm will do) or need to document the alternative in the specification document? Both option are workable with instantaneous percent only needing support to be added to vendor agnostic gputop.

Quoted 7 lines
Quoted 3 lines

I would prefer to just change to the ns format i915 and msm will be using, that makes much more sense from my experience.

As far as I know we haven't released any publicly available userspace using the existing AMD specific format. So that should still be possible.

If amdgpu could export accumulated time context spent on engines that would indeed be perfect. It would make the gputop I sketched out most probably just work, as it did for Rob on msm.

In which case, apart from the admgpu work, it would just be a matter of me tidying that tool a bit and re-sending out for review.

Could you push this to some repository on fdo and send me a link? Going to pick up this patch here and give it a try, shouldn't be more than a day of work.

Quoted 6 lines
Quoted 4 lines
Quoted 2 lines

  * Can amdgpu expose drm-client-id? Without it gputop will not work.

How is that determined on i915 ? Does struct drm_file has that somewhere?

It should correspond 1:1 with drm_file, since the purpose is to enable gputop distinguish between unique open file descriptors (aka clients).

Ah! We do have a 64bit counter for that already because of technical needs.

Quoted 10 lines

In theory it could be just a hash value of a struct drm_file pointer but that could confuse userspace if the struct gets reused within a single userspace sampling period.

Because of that I track it in i915 separately since I wanted to have an incrementing cyclic property to it - so that when a fd is closed and new opened there is no practical chance they would have the same drm-client-id.

Quoted 3 lines
Quoted 1 line

* drm-engine-capacity - does the concept translate etc.

I don't think we are going to need that.

Okay, that one is optional for cases when there is more than one engine of a type/class shown under a single entry in fdinfo. So when gputop translates accumulated time into percentages it can do the right thing. Code can already handle it not being present and assume one.

Yeah, we have that case for a couple of things. The GFX, SDMA and multimedia engines all have different queues which needs to be accounted together as far as I can see.

E.g. we have video decode and video encode as two separate rings, but essentially they use the same engine.

Need to think about how to represent that.

Regards,
Christian.

Quoted 135 lines

Regards,

Tvrtko

Quoted 130 lines

Regards,
Christian.

Quoted 125 lines

---
  Documentation/gpu/amdgpu/usage-stats.rst   | 28
++++++++++++++++++++++
  Documentation/gpu/drm-usage-stats.rst      |  7 +++++- drivers/gpu/drm/amd/amdgpu/amdgpu_fdinfo.c | 18 ++++++++++----

  3 files changed, 47 insertions(+), 6 deletions(-)

  create mode 100644 Documentation/gpu/amdgpu/usage-stats.rst

Documentation/gpu/amdgpu/usage-stats.rst new file +7 -0
new file mode 100644
index 000000000000..8be5009bd1a9
@@ -0,0 +1,28 @@
+.. _amdgpu-usage-stats:
+
+============================================
+AMDGPU DRM client usage stats implementation
+============================================
+
+The amdgpu driver implements the DRM client usage stats 

specification as

Documentation/gpu/amdgpu/usage-stats.rst new file +3 -0
+documented in :ref:`drm-client-usage-stats`.
+
+Example of the output showing the implemented key value pairs and 

entirety of

Documentation/gpu/amdgpu/usage-stats.rst new file +18 -0
+the currenly possible format options:
+
+::
+
+      pos:    0
+      flags:  0100002
+      mnt_id: 21
+      drm-driver: amdgpu
+      drm-pdev:   0000:03:00.0
+      drm-memory-vram: 0 KiB
+      drm-memory-gtt: 0 KiB
+      drm-memory-cpu: 0 KiB
+      drm-engine-...: 0 %
+                 ...
+
+Possible `drm-memory-` key names are: `vram`, `gtt`, `cpu`.
+
+Possible `drm-engine-` key names are: ``.
Documentation/gpu/drm-usage-stats.rst +0 -0
index 6c9f166a8d6f..2d0ff6f2cc74 100644
@@ -69,7 +69,7 @@ scope of each device, in which case `drm-pdev` 

shall be present as well.
  Userspace should make sure to not double account any usage statistics by using the above described criteria in order to associate data to individual clients. -- drm-engine-<str>: <uint> ns
+- drm-engine-<str>: <uint> [ns|%]
  GPUs usually contain multiple execution engines. Each shall be given a stable and unique name (str), with possible values documented in the driver specific

Documentation/gpu/drm-usage-stats.rst +0 -0
@@ -84,6 +84,9 @@ larger value within a reasonable period. Upon 

observing a value lower than what
  was previously read, userspace is expected to stay with that larger previous value until a monotonic update is seen.
+Where time unit is given as a percentage...[AMD folks to fill the semantics

Documentation/gpu/drm-usage-stats.rst +2 -0
+and interpretation of that]...
+

  - drm-engine-capacity-<str>: <uint>
  Engine identifier string must be the same as the one specified in the

Documentation/gpu/drm-usage-stats.rst +0 -0
@@ -110,3 +113,5 @@ Driver specific implementations

  ===============================
  :ref:i915-usage-stats

Documentation/gpu/drm-usage-stats.rst +2 -0
+
+:ref:`amdgpu-usage-stats`
drivers/gpu/drm/amd/amdgpu/amdgpu_fdinfo.c +4 -1
index 5a6857c44bb6..8cbae61f1b3b 100644
@@ -32,6 +32,7 @@
   #include <drm/amdgpu_drm.h>
   #include <drm/drm_debugfs.h>
+#include <drm/drm_drv.h>
   #include "amdgpu.h"
   #include "amdgpu_vm.h"
@@ -83,11 +84,18 @@ void amdgpu_show_fdinfo(struct seq_file *m, 
 struct file *f)
       amdgpu_bo_unreserve(root);
       amdgpu_bo_unref(&root);
-    seq_printf(m, "pdev:\t%04x:%02x:%02x.%d\npasid:\t%u\n", domain, 
 bus,
+    /*
+     * 
 ******************************************************************
+     * For text output format description please see 

drm-usage-stats.rst!
+     *
******************************************************************
+     */
+
+    seq_printf(m, "drm-driver:\t%s\n", file->minor->dev->driver->name);
+    seq_printf(m, "drm-pdev:\t%04x:%02x:%02x.%d\npasid:\t%u\n", domain, bus,
              dev, fn, fpriv->vm.pasid);
-    seq_printf(m, "vram mem:\t%llu kB\n", vram_mem/1024UL); -    seq_printf(m, "gtt mem:\t%llu kB\n", gtt_mem/1024UL); -    seq_printf(m, "cpu mem:\t%llu kB\n", cpu_mem/1024UL); +    seq_printf(m, "drm-memory-vram:\t%llu KiB\n", vram_mem/1024UL); +    seq_printf(m, "drm-memory-gtt:\t%llu KiB\n", gtt_mem/1024UL); +    seq_printf(m, "drm-memory-cpu:\t%llu KiB\n", cpu_mem/1024UL); for (i = 0; i < AMDGPU_HW_IP_NUM; i++) {
          uint32_t count = amdgpu_ctx_num_entities[i]; int idx = 0;

drivers/gpu/drm/amd/amdgpu/amdgpu_fdinfo.c +1 -1
@@ -103,7 +111,7 @@ void amdgpu_show_fdinfo(struct seq_file *m, 
 struct file *f)
               perc = div64_u64(10000 * total, min);
               frac = perc % 100;
-            seq_printf(m, "%s%d:\t%d.%d%%\n",
+            seq_printf(m, "drm-engine-%s%d:\t%d.%d %%\n",
                       amdgpu_ip_name[i],
                       idx, perc/100, frac);
           }
#4 lore raw
Quoted 17 lines · Christian König

Am 10.05.22 um 12:50 schrieb Tvrtko Ursulin:

Quoted 16 lines

Hi,

Quoted 12 lines · Christian König

Hi Tvrtko,

Am 10.05.22 um 10:23 schrieb Tvrtko Ursulin:

Quoted 8 lines

From: Tvrtko Ursulin <[email protected]>

Convert fdinfo format to one documented in drm-usage-stats.rst.

Opens/TODO:
  * Does someone from AMD want to take over this patch? (I have no access to amdgpu hardware so won't be able to test any hypothetical gputop work.)

Quoted 5 lines
Quoted 3 lines
Quoted 1 line

I can give that a try as soon as it is completed.

And how to motivate someone on your side to pick up the amdgpu work? :)

Well if we could get more of my TTM/DRM patches reviewed I could have same free time to do this :)

Yeah, we have a bunch of folks dedicatedly working on TTM and scheduling refactoring so I am hoping they will notice and get involved.

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 1 line

  * What are the semantics of AMD engine utilisation reported in percents?

To be honest I haven't understood why we are using percents here either, that is not something the kernel should mess with.

Quoted 4 lines

     * Can it align with what i915 does (same what msm will do) or need to document the alternative in the specification document? Both option are workable with instantaneous percent only needing support to be added to vendor agnostic gputop.

I would prefer to just change to the ns format i915 and msm will be using, that makes much more sense from my experience.

Quoted 7 lines
Quoted 5 lines
Quoted 1 line

As far as I know we haven't released any publicly available userspace using the existing AMD specific format. So that should still be possible.

If amdgpu could export accumulated time context spent on engines that would indeed be perfect. It would make the gputop I sketched out most probably just work, as it did for Rob on msm.

In which case, apart from the admgpu work, it would just be a matter of me tidying that tool a bit and re-sending out for review.

Could you push this to some repository on fdo and send me a link? Going to pick up this patch here and give it a try, shouldn't be more than a day of work.

Done, https://cgit.freedesktop.org/~tursulin/intel-gpu-tools/log/?h=gputop.

For extra reference the msm patch was this: https://lore.kernel.org/lkml/[email protected]/

If you can expose the same fields gputtop should work.

Quoted 17 lines
Quoted 5 lines
Quoted 3 lines
Quoted 1 line

  * Can amdgpu expose drm-client-id? Without it gputop will not work.

How is that determined on i915 ? Does struct drm_file has that somewhere?

It should correspond 1:1 with drm_file, since the purpose is to enable gputop distinguish between unique open file descriptors (aka clients).

Ah! We do have a 64bit counter for that already because of technical needs.

Quoted 9 lines

In theory it could be just a hash value of a struct drm_file pointer but that could confuse userspace if the struct gets reused within a single userspace sampling period.

Because of that I track it in i915 separately since I wanted to have an incrementing cyclic property to it - so that when a fd is closed and new opened there is no practical chance they would have the same drm-client-id.

Quoted 3 lines
Quoted 1 line

* drm-engine-capacity - does the concept translate etc.

I don't think we are going to need that.

Quoted 7 lines
Quoted 1 line

Okay, that one is optional for cases when there is more than one engine of a type/class shown under a single entry in fdinfo. So when gputop translates accumulated time into percentages it can do the right thing. Code can already handle it not being present and assume one.

Yeah, we have that case for a couple of things. The GFX, SDMA and multimedia engines all have different queues which needs to be accounted together as far as I can see.

E.g. we have video decode and video encode as two separate rings, but essentially they use the same engine.

Need to think about how to represent that.

I think you have some freedom there as to what to export - whether the entity userspace submits to (is this ring in amdgpu?), or the entity hardware actually executes on (your engine?).

We have somewhat similar setup in i915 and I decided to expose the former. This makes it both (almost) match what our total metrics show (engine performance counters exported via perf/pmu).

So in case of your video decode and encode rings which lead to the same hw engine, that would mean exposing them as two entities, decode and encode.

But as said, my spec does prescribe that so it is up to implementations. As long as it is useful for users as first port of enquiry for performance problems I think it is fine.

Analogue would be hyper-threading from the CPU scheduling world and how top(1) cannot distinguish why one core at 0% does not mean there is half of the performance still one the table.

And then for a deeper down into performance more specialized GPU profiling tools are required.

Regards,

Tvrtko