Skip to contentSkip to conversation

Per client engine busyness (all aboard the sysfs train!)

RFC In review Tvrtko Ursulin 3 participants 12 messages Oct 25, 2019

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover Per client engine busyness (all aboard the sysfs train!)

From: Tvrtko Ursulin <[email protected]>

It was quite some time since I last posted this RFC, but recently there has been some new interest, this time from OpenCL and related customers, so I decided to give it a quick respin and test the waters.

This time round it has been really hastily rebased since the upstream changed quite a lot and I have very little confidence it is technically correct. But it is enough to illustrate a point of what this feature could provide:

In short it enables a "top-like" display for GPU tasks. Or with a screenshot:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
intel-gpu-top - 948/ 999 MHz; 0% RC6; 3.65 Watts; 2165 irqs/s

IMC reads: 5015 MiB/s
IMC writes: 143 MiB/s

ENGINE BUSY MI_SEMA MI_WAIT
Render/3D/0 56.60% |███████████████████████████▋ | 0% 0%
Blitter/0 95.65% |██████████████████████████████████████████████▊ | 0% 0%
Video/0 40.92% |████████████████████ | 0% 0%
VideoEnhance/0 0.00% | | 0% 0%

PID NAME RCS BCS VCS VECS

 5347        gem_wsim |███████▍        ||███████████████▏||██████▌         ||                |
 4929            Xorg |▎               ||                ||                ||                |
 5305        glxgears |                ||                ||                ||                |
 5303        glxgears |                ||                ||                ||                |
 5024           xfwm4 |                ||                ||                ||                |
 4929            Xorg |                ||                ||                ||                |

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Implementation wise we would get a a bunch of per-drm-client-per-engine-class files in sysfs like:

	# cd /sys/class/drm/card0/clients/
	# tree
	.
	├── 7
	│   ├── busy
	│   │   ├── 0
	│   │   ├── 1
	│   │   ├── 2
	│   │   └── 3
	│   ├── name
	│   └── pid
	├── 8
	│   ├── busy
	│   │   ├── 0
	│   │   ├── 1
	│   │   ├── 2
	│   │   └── 3
	│   ├── name
	│   └── pid
	├── 9
	│   ├── busy
	│   │   ├── 0
	│   │   ├── 1
	│   │   ├── 2
	│   │   └── 3
	│   ├── name
	│   └── pid
	└── enable_stats

I will post the corresponding patch to intel_gpu_top for reference as well.

Tvrtko Ursulin (5):
drm/i915: Track per-context engine busyness drm/i915: Expose list of clients in sysfs drm/i915: Update client name on context create drm/i915: Expose per-engine client busyness drm/i915: Add sysfs toggle to enable per-client engine stats

 drivers/gpu/drm/i915/gem/i915_gem_context.c   |  17 +-
 drivers/gpu/drm/i915/gt/intel_context.c       |  20 ++
 drivers/gpu/drm/i915/gt/intel_context.h       |   9 +
 drivers/gpu/drm/i915/gt/intel_context_types.h |   9 +
 drivers/gpu/drm/i915/gt/intel_engine_cs.c     |  16 +-
 drivers/gpu/drm/i915/gt/intel_lrc.c           |  66 +++++-
 drivers/gpu/drm/i915/i915_drv.h               |  38 +++
 drivers/gpu/drm/i915/i915_gem.c               | 218 +++++++++++++++++-
 drivers/gpu/drm/i915/i915_sysfs.c             |  81 +++++++
 9 files changed, 451 insertions(+), 23 deletions(-)
Signature · 6 lines
2.20.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#1 lore raw

RFC 1/5 drm/i915: Track per-context engine busyness

From: Tvrtko Ursulin <[email protected]>

Some customers want to know how much of the GPU time are their clients using in order to make dynamic load balancing decisions.

With the hooks already in place which track the overall engine busyness, we can extend that slightly to split that time between contexts.

v2: Fix accounting for tail updates.
v3: Rebase.
v4: Mark currently running contexts as active on stats enable.
v5: Include some headers to fix the build.
v6: Added fine grained lock.
v7: Convert to seqlock. (Chris Wilson)
v8: Rebase and tidy with helpers.
v9: Rebase.

---
 drivers/gpu/drm/i915/gt/intel_context.c       | 20 ++++++
 drivers/gpu/drm/i915/gt/intel_context.h       |  9 +++
 drivers/gpu/drm/i915/gt/intel_context_types.h |  9 +++
 drivers/gpu/drm/i915/gt/intel_engine_cs.c     | 16 ++++-
 drivers/gpu/drm/i915/gt/intel_lrc.c           | 66 ++++++++++++++++---
 5 files changed, 108 insertions(+), 12 deletions(-)
drivers/gpu/drm/i915/gt/intel_context.c +20 -0
index ee9d2bcd2c13..3d68720df512 100644
@@ -248,6 +248,7 @@ intel_context_init(struct intel_context *ce,
 	INIT_LIST_HEAD(&ce->signals);
 
 	mutex_init(&ce->pin_mutex);
+	seqlock_init(&ce->stats.lock);
 
 	i915_active_init(&ce->active,
 			 __intel_context_active, __intel_context_retire);
@@ -348,6 +349,25 @@ struct i915_request *intel_context_create_request(struct intel_context *ce)
 	return rq;
 }
 
+ktime_t intel_context_get_busy_time(struct intel_context *ce)
+{
+	unsigned int seq;
+	ktime_t total;
+
+	do {
+		seq = read_seqbegin(&ce->stats.lock);
+
+		total = ce->stats.total;
+
+		if (ce->stats.active)
+			total = ktime_add(total,
+					  ktime_sub(ktime_get(),
+						    ce->stats.start));
+	} while (read_seqretry(&ce->stats.lock, seq));
+
+	return total;
+}
+
 #if IS_ENABLED(CONFIG_DRM_I915_SELFTEST)
 #include "selftest_context.c"
 #endif
drivers/gpu/drm/i915/gt/intel_context.h +9 -0
index 68b3d317d959..c7ab8efa3573 100644
@@ -153,4 +153,13 @@ static inline struct intel_ring *__intel_context_ring_size(u64 sz)
 	return u64_to_ptr(struct intel_ring, sz);
 }
 
+static inline void
+__intel_context_stats_start(struct intel_context_stats *stats, ktime_t now)
+{
+	stats->start = now;
+	stats->active = true;
+}
+
+ktime_t intel_context_get_busy_time(struct intel_context *ce);
+
 #endif /* __INTEL_CONTEXT_H__ */
drivers/gpu/drm/i915/gt/intel_context_types.h +9 -0
index 6959b05ae5f8..b3384e01d033 100644
@@ -11,6 +11,7 @@
 #include <linux/list.h>
 #include <linux/mutex.h>
 #include <linux/types.h>
+#include <linux/seqlock.h>
 
 #include "i915_active_types.h"
 #include "i915_utils.h"
@@ -75,6 +76,14 @@ struct intel_context {
 
 	/** sseu: Control eu/slice partitioning */
 	struct intel_sseu sseu;
+
+	/** stats: Context GPU engine busyness tracking. */
+	struct intel_context_stats {
+		seqlock_t lock;
+		bool active;
+		ktime_t start;
+		ktime_t total;
+	} stats;
 };
 
 #endif /* __INTEL_CONTEXT_TYPES__ */
drivers/gpu/drm/i915/gt/intel_engine_cs.c +14 -2
index 9cc1ea6519ec..6792ec01f3f2 100644
@@ -1568,8 +1568,20 @@ int intel_enable_engine_stats(struct intel_engine_cs *engine)
 
 		engine->stats.enabled_at = ktime_get();
 
-		/* XXX submission method oblivious? */
-		for (port = execlists->active; (rq = *port); port++)
+		/*
+		 * Mark currently running context as active.
+		 * XXX submission method oblivious?
+		 */
+
+		rq = NULL;
+		port = execlists->active;
+		if (port)
+			rq = *port;
+		if (rq)
+			__intel_context_stats_start(&rq->hw_context->stats,
+						    engine->stats.enabled_at);
+
+		for (; (rq = *port); port++)
 			engine->stats.active++;
 
 		for (port = execlists->pending; (rq = *port); port++) {
drivers/gpu/drm/i915/gt/intel_lrc.c +56 -10

131 lines not on this page · raw message

Signature · 6 lines
2.20.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#2 lore raw

RFC 2/5 drm/i915: Expose list of clients in sysfs

In reply to Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

Expose a list of clients with open file handles in sysfs.

This will be a basis for a top-like utility showing per-client and per- engine GPU load.

Currently we only expose each client's pid and name under opaque numbered directories in /sys/class/drm/card0/clients/.

For instance:

/sys/class/drm/card0/clients/3/name: Xorg
/sys/class/drm/card0/clients/3/pid: 5664

v2:
Chris Wilson:
* Enclose new members into dedicated structs.
* Protect against failed sysfs registration.

v3:
* sysfs_attr_init.

v4:
* Fix for internal clients.

---
 drivers/gpu/drm/i915/i915_drv.h   |  19 +++++
 drivers/gpu/drm/i915/i915_gem.c   | 124 ++++++++++++++++++++++++++++--
 drivers/gpu/drm/i915/i915_sysfs.c |   8 ++
 3 files changed, 143 insertions(+), 8 deletions(-)
drivers/gpu/drm/i915/i915_drv.h +19 -0
index 16e58a74fa6f..4dc8cadf56eb 100644
@@ -222,6 +222,20 @@ struct drm_i915_file_private {
 	/** ban_score: Accumulated score of all ctx bans and fast hangs. */
 	atomic_t ban_score;
 	unsigned long hang_timestamp;
+
+	struct i915_drm_client {
+		unsigned int id;
+
+		pid_t pid;
+		char *name;
+
+		struct kobject *root;
+
+		struct {
+			struct device_attribute pid;
+			struct device_attribute name;
+		} attr;
+	} client;
 };
 
 /* Interface history:
@@ -1372,6 +1386,11 @@ struct drm_i915_private {
 
 	struct i915_pmu pmu;
 
+	struct i915_drm_clients {
+		struct kobject *root;
+		atomic_t serial;
+	} clients;
+
 	struct i915_hdcp_comp_master *hdcp_master;
 	bool hdcp_comp_added;
 
drivers/gpu/drm/i915/i915_gem.c +116 -8

155 lines not on this page · raw message

drivers/gpu/drm/i915/i915_sysfs.c +8 -0
index bf039b8ba593..a9f27f4fc245 100644
@@ -574,6 +574,11 @@ void i915_setup_sysfs(struct drm_i915_private *dev_priv)
 	struct device *kdev = dev_priv->drm.primary->kdev;
 	int ret;
 
+	dev_priv->clients.root =
+		kobject_create_and_add("clients", &kdev->kobj);
+	if (!dev_priv->clients.root)
+		DRM_ERROR("Per-client sysfs setup failed\n");
+
 #ifdef CONFIG_PM
 	if (HAS_RC6(dev_priv)) {
 		ret = sysfs_merge_group(&kdev->kobj,
@@ -634,4 +639,7 @@ void i915_teardown_sysfs(struct drm_i915_private *dev_priv)
 	sysfs_unmerge_group(&kdev->kobj, &rc6_attr_group);
 	sysfs_unmerge_group(&kdev->kobj, &rc6p_attr_group);
 #endif
+
+	if (dev_priv->clients.root)
+		kobject_put(dev_priv->clients.root);
 }
Signature · 6 lines
2.20.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#3 lore raw

Re: [RFC 2/5] drm/i915: Expose list of clients in sysfs

Quoting Tvrtko Ursulin (2019-10-25 15:21:28)

Quoted 4 lines

int i915_gem_open(struct drm_i915_private *i915, struct drm_file *file) {

drivers/gpu/drm/i915/i915_gem.c +1 -0
+       int ret = -ENOMEM;
        struct drm_i915_file_private *file_priv;

drivers/gpu/drm/i915/i915_gem.c void i915_gem_release(struct drm_device *dev, struct drm_file *file) · line 1616

Quoted 10 lines
drivers/gpu/drm/i915/i915_gem.c +3 -2
-       int ret;
 
        DRM_DEBUG("\n");
 
        file_priv = kzalloc(sizeof(*file_priv), GFP_KERNEL);
        if (!file_priv)
-               return -ENOMEM;
+               goto err_alloc;
+
+       file_priv->client.id = atomic_inc_return(&i915->clients.serial);

We should make this a cyclic ida to avoid reuse on wraparound. 32b wraps will happen, and they will still have client 0 alive! :)

That will mean we need a lock.

(Of course you could use -EEXIST from add_client and keep incrementing serial until you find a hole :)
-Chris
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#4 lore raw

RFC 3/5 drm/i915: Update client name on context create

In reply to Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

Some clients have the DRM fd passed to them over a socket by the X server.

Grab the real client and pid when they create their first context and update the exposed data for more useful enumeration.

---
 drivers/gpu/drm/i915/gem/i915_gem_context.c | 17 ++++++++++++++---
 drivers/gpu/drm/i915/i915_drv.h             |  7 +++++++
 drivers/gpu/drm/i915/i915_gem.c             |  4 ++--
 3 files changed, 23 insertions(+), 5 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_context.c +14 -3
index 55f1f93c0925..c7f6684eb366 100644
@@ -2084,6 +2084,8 @@ int i915_gem_context_create_ioctl(struct drm_device *dev, void *data,
 {
 	struct drm_i915_private *i915 = to_i915(dev);
 	struct drm_i915_gem_context_create_ext *args = data;
+	pid_t pid = pid_nr(get_task_pid(current, PIDTYPE_PID));
+	struct drm_i915_file_private *file_priv = file->driver_priv;
 	struct create_ext ext_data;
 	int ret;
 
@@ -2097,14 +2099,23 @@ int i915_gem_context_create_ioctl(struct drm_device *dev, void *data,
 	if (ret)
 		return ret;
 
-	ext_data.fpriv = file->driver_priv;
+	ext_data.fpriv = file_priv;
 	if (client_is_banned(ext_data.fpriv)) {
 		DRM_DEBUG("client %s[%d] banned from creating ctx\n",
-			  current->comm,
-			  pid_nr(get_task_pid(current, PIDTYPE_PID)));
+			  current->comm, pid);
 		return -EIO;
 	}
 
+	mutex_lock(&dev->struct_mutex);
+	if (file_priv->client.pid != pid) {
+		i915_gem_remove_client(file_priv);
+		ret = i915_gem_add_client(i915, file_priv, current,
+					  file_priv->client.id);
+	}
+	mutex_unlock(&dev->struct_mutex);
+	if (ret)
+		return ret;
+
 	ext_data.ctx = i915_gem_create_context(i915, args->flags);
 	if (IS_ERR(ext_data.ctx))
 		return PTR_ERR(ext_data.ctx);
drivers/gpu/drm/i915/i915_drv.h +7 -0
index 4dc8cadf56eb..b8f7b0637224 100644
@@ -1983,6 +1983,13 @@ void i915_gem_suspend_late(struct drm_i915_private *dev_priv);
 void i915_gem_resume(struct drm_i915_private *dev_priv);
 vm_fault_t i915_gem_fault(struct vm_fault *vmf);
 
+int
+i915_gem_add_client(struct drm_i915_private *i915,
+		struct drm_i915_file_private *file_priv,
+		struct task_struct *task,
+		unsigned int serial);
+void i915_gem_remove_client(struct drm_i915_file_private *file_priv);
+
 int i915_gem_open(struct drm_i915_private *i915, struct drm_file *file);
 void i915_gem_release(struct drm_device *dev, struct drm_file *file);
 
drivers/gpu/drm/i915/i915_gem.c +2 -2
index d8d352efb9ef..54a00c954066 100644
@@ -1512,7 +1512,7 @@ show_client_pid(struct device *kdev, struct device_attribute *attr, char *buf)
 	return snprintf(buf, PAGE_SIZE, "%u", file_priv->client.pid);
 }
 
-static int
+int
 i915_gem_add_client(struct drm_i915_private *i915,
 		struct drm_i915_file_private *file_priv,
 		struct task_struct *task,
@@ -1572,7 +1572,7 @@ i915_gem_add_client(struct drm_i915_private *i915,
 	return ret;
 }
 
-static void i915_gem_remove_client(struct drm_i915_file_private *file_priv)
+void i915_gem_remove_client(struct drm_i915_file_private *file_priv)
 {
 	if (!file_priv->client.name)
 		return; /* intel_fbdev_init registers a client before sysfs */
Signature · 6 lines
2.20.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#5 lore raw

Re: [RFC 3/5] drm/i915: Update client name on context create

Quoting Tvrtko Ursulin (2019-10-25 15:21:29)

Quoted 24 lines
drivers/gpu/drm/i915/gem/i915_gem_context.c +4 -3
index 55f1f93c0925..c7f6684eb366 100644
@@ -2084,6 +2084,8 @@ int i915_gem_context_create_ioctl(struct drm_device *dev, void *data,
 {
        struct drm_i915_private *i915 = to_i915(dev);
        struct drm_i915_gem_context_create_ext *args = data;
+       pid_t pid = pid_nr(get_task_pid(current, PIDTYPE_PID));
+       struct drm_i915_file_private *file_priv = file->driver_priv;
        struct create_ext ext_data;
        int ret;
 
@@ -2097,14 +2099,23 @@ int i915_gem_context_create_ioctl(struct drm_device *dev, void *data,
        if (ret)
                return ret;
 
-       ext_data.fpriv = file->driver_priv;
+       ext_data.fpriv = file_priv;
        if (client_is_banned(ext_data.fpriv)) {
                DRM_DEBUG("client %s[%d] banned from creating ctx\n",
-                         current->comm,
-                         pid_nr(get_task_pid(current, PIDTYPE_PID)));
+                         current->comm, pid);

drivers/gpu/drm/i915/gem/i915_gem_context.c int i915_gem_context_create_ioctl(struct drm_device *dev, void *data, · line 2115

Quoted 10 lines
drivers/gpu/drm/i915/gem/i915_gem_context.c +7 -0
                return -EIO;
        }
 
+       mutex_lock(&dev->struct_mutex);
+       if (file_priv->client.pid != pid) {
+               i915_gem_remove_client(file_priv);
+               ret = i915_gem_add_client(i915, file_priv, current,
+                                         file_priv->client.id);
+       }
+       mutex_unlock(&dev->struct_mutex);

You are serialising against multiple context_create_ioctl from the same file, right? Could abuse fpriv->context_idr_lock. Or add a new one.

drivers/gpu/drm/i915/gem/i915_gem_context.c int i915_gem_context_create_ioctl(struct drm_device *dev, void *data, · line 2118

Quoted 3 lines
drivers/gpu/drm/i915/gem/i915_gem_context.c +3 -0
+       if (ret)
+               return ret;
+

Hmm, is get_task_pid() the one that returns a reference to the pid_t? Aye, it is, we need a put_pid().
-Chris
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#6 lore raw

RFC 4/5 drm/i915: Expose per-engine client busyness

In reply to Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

Expose per-client and per-engine busyness under the previously added sysfs client root.

The new files are one per-engine instance and located under the 'busy' directory. Each contains a monotonically increasing nano-second resolution times each client's jobs were executing on the GPU.

This enables userspace to create a top-like tool for GPU utilization:

==========================================================================
intel-gpu-top - 935/ 935 MHz; 0% RC6; 14.73 Watts; 1097 irqs/s

IMC reads: 1401 MiB/s
IMC writes: 4 MiB/s

ENGINE BUSY MI_SEMA MI_WAIT
Render/3D/0 63.73% |███████████████████ | 3% 0%
Blitter/0 9.53% |██▊ | 6% 0%
Video/0 39.32% |███████████▊ | 16% 0%
Video/1 15.62% |████▋ | 0% 0%
VideoEnhance/0 0.00% | | 0% 0%

PID NAME RCS BCS VCS VECS

 4084        gem_wsim |█████▌     ||█          ||           ||           |
 4086        gem_wsim |█▌         ||           ||███        ||           |

==========================================================================

v2: Use intel_context_engine_get_busy_time.
v3: New directory structure.
v4: Rebase.
v5: sysfs_attr_init.
v6: Small tidy in i915_gem_add_client.
v7: Rebase to be engine class based.

---
 drivers/gpu/drm/i915/i915_drv.h |   8 +++
 drivers/gpu/drm/i915/i915_gem.c | 102 ++++++++++++++++++++++++++++++--
 2 files changed, 106 insertions(+), 4 deletions(-)
drivers/gpu/drm/i915/i915_drv.h +8 -0
index b8f7b0637224..45f0e2455322 100644
@@ -186,6 +186,12 @@ struct drm_i915_private;
 struct i915_mm_struct;
 struct i915_mmu_object;
 
+struct i915_engine_busy_attribute {
+	struct device_attribute attr;
+	struct drm_i915_file_private *file_priv;
+	unsigned int engine_class;
+};
+
 struct drm_i915_file_private {
 	struct drm_i915_private *dev_priv;
 
@@ -230,10 +236,12 @@ struct drm_i915_file_private {
 		char *name;
 
 		struct kobject *root;
+		struct kobject *busy_root;
 
 		struct {
 			struct device_attribute pid;
 			struct device_attribute name;
+			struct i915_engine_busy_attribute busy[MAX_ENGINE_CLASS];
 		} attr;
 	} client;
 };
drivers/gpu/drm/i915/i915_gem.c +98 -4

140 lines not on this page · raw message

Signature · 6 lines
2.20.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#7 lore raw

Re: [RFC 4/5] drm/i915: Expose per-engine client busyness

Quoting Tvrtko Ursulin (2019-10-25 15:21:30)

Quoted 24 lines
drivers/gpu/drm/i915/i915_gem.c +24 -0
+static int busy_add(int id, void *p, void *data)
+{
+       struct busy_ctx *bc = data;
+       struct i915_gem_context *ctx = p;
+       unsigned int engine_class = bc->engine_class;
+       struct i915_gem_engines_iter it;
+       struct intel_context *ce;
+       uint64_t total = bc->total;
+
+       for_each_gem_engine(ce, i915_gem_context_lock_engines(ctx), it) {
+               if (ce->engine->uabi_class == engine_class)
+                       total += ktime_to_ns(intel_context_get_busy_time(ce));
+       }
+       i915_gem_context_unlock_engines(ctx);
+
+       bc->total = total;
+
+       return 0;
+}
+
+static ssize_t
+show_client_busy(struct device *kdev, struct device_attribute *attr, char *buf)
+{
+       struct i915_engine_busy_attribute *i915_attr =

drivers/gpu/drm/i915/i915_gem.c show_client_pid(struct device *kdev, struct device_attribute *attr, char *buf) · line 1553

Quoted 10 lines
drivers/gpu/drm/i915/i915_gem.c +10 -0
+               container_of(attr, typeof(*i915_attr), attr);
+       struct drm_i915_file_private *file_priv = i915_attr->file_priv;
+       struct busy_ctx bc = { .engine_class = i915_attr->engine_class };
+       int ret;
+
+       ret = mutex_lock_interruptible(&file_priv->context_idr_lock);
+       if (ret)
+               return ret;
+
+       idr_for_each(&file_priv->context_idr, busy_add, &bc);

If you don a hard hat, this can all be done under rcu_read_lock(). -Chris
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#8 lore raw

RFC 5/5 drm/i915: Add sysfs toggle to enable per-client engine stats

In reply to Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

By default we are not collecting any per-engine and per-context statistcs.

Add a new sysfs toggle to enable this facility:

$ echo 1 >/sys/class/drm/card0/clients/enable_stats

v2: Rebase.
v3: sysfs_attr_init.
v4: Scheduler caps.

---
 drivers/gpu/drm/i915/i915_drv.h   |  4 ++
 drivers/gpu/drm/i915/i915_sysfs.c | 73 +++++++++++++++++++++++++++++++
 2 files changed, 77 insertions(+)
drivers/gpu/drm/i915/i915_drv.h +4 -0
index 45f0e2455322..3d2459e9fff4 100644
@@ -1397,6 +1397,10 @@ struct drm_i915_private {
 	struct i915_drm_clients {
 		struct kobject *root;
 		atomic_t serial;
+		struct {
+			bool enabled;
+			struct device_attribute attr;
+		} stats;
 	} clients;
 
 	struct i915_hdcp_comp_master *hdcp_master;
drivers/gpu/drm/i915/i915_sysfs.c +73 -0

94 lines not on this page · raw message

Signature · 6 lines
2.20.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#9 lore raw

Re: [RFC 5/5] drm/i915: Add sysfs toggle to enable per-client engine stats

Quoting Tvrtko Ursulin (2019-10-25 15:21:31)

drivers/gpu/drm/i915/i915_sysfs.c static void i915_setup_error_capture(struct device *kdev) {} · line 609

Quoted 8 lines
drivers/gpu/drm/i915/i915_sysfs.c +8 -0
+       ret = i915_mutex_lock_interruptible(&i915->drm);
+       if (ret)
+               return ret;
+
+       if (val && !i915->clients.stats.enabled)
+               enable = true;
+       else if (!val && i915->clients.stats.enabled)
+               disable = true;

The struct_mutex is just for atomically enabling/disabling stats, right? Only one user may toggle status at a time.

I'd wrap it a i915->spinlock just so the locking is clear from the outset.

drivers/gpu/drm/i915/i915_sysfs.c static void i915_setup_error_capture(struct device *kdev) {} · line 614

Quoted 4 lines
drivers/gpu/drm/i915/i915_sysfs.c +4 -0
+       if (!enable && !disable)
+               goto out;
+
+       for_each_engine(engine, i915, id) {

A quick s/for_each_engine/for_each_uabi_engine/

drivers/gpu/drm/i915/i915_sysfs.c static void i915_setup_error_capture(struct device *kdev) {} · line 621

Quoted 7 lines
drivers/gpu/drm/i915/i915_sysfs.c +7 -0
+               if (enable)
+                       WARN_ON_ONCE(intel_enable_engine_stats(engine));
+               else if (disable)
+                       intel_disable_engine_stats(engine);
+       }
+
+       i915->clients.stats.enabled = val;

Now as for whether we want a toggle approach, or only while the file is open (and refcount)?
-Chris
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#10 lore raw

✗ Fi.CI.CHECKPATCH: warning for Per client engine busyness (all aboard the sysfs train!)

In reply to Tvrtko Ursulin
== Series Details ==

== Series Details ==

Series: Per client engine busyness (all aboard the sysfs train!)
URL : https://patchwork.freedesktop.org/series/68570/
State : warning

== Summary ==

$ dim checkpatch origin/drm-tip
c5f5abc9d897 drm/i915: Track per-context engine busyness
2c3edc69e7b6 drm/i915: Expose list of clients in sysfs
-:99: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#99: FILE: drivers/gpu/drm/i915/i915_gem.c:1517:
+i915_gem_add_client(struct drm_i915_private *i915,
+		struct drm_i915_file_private *file_priv,

total: 0 errors, 0 warnings, 1 checks, 204 lines checked a142a38e3f25 drm/i915: Update client name on context create -:63: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis #63: FILE: drivers/gpu/drm/i915/i915_drv.h:1990:
+i915_gem_add_client(struct drm_i915_private *i915,
+ struct drm_i915_file_private *file_priv,

total: 0 errors, 0 warnings, 1 checks, 63 lines checked
84b4d495586c drm/i915: Expose per-engine client busyness
-:25: WARNING:COMMIT_LOG_LONG_LINE: Possible unwrapped commit description (prefer a maximum 75 chars per line) #25:
Render/3D/0 63.73% |███████████████████ | 3% 0%

-:95: CHECK:PREFER_KERNEL_TYPES: Prefer kernel type 'u64' over 'uint64_t' #95: FILE: drivers/gpu/drm/i915/i915_gem.c:1527:
+ uint64_t total = bc->total;

-:128: WARNING:STATIC_CONST_CHAR_ARRAY: static const char * array should probably be static const char * const #128: FILE: drivers/gpu/drm/i915/i915_gem.c:1560:
+static const char *uabi_class_names[] = {

-:185: ERROR:CODE_INDENT: code indent should use tabs where possible #185: FILE: drivers/gpu/drm/i915/i915_gem.c:1633:
+^I^I^I^I (struct attribute *)attr);$

-:185: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis #185: FILE: drivers/gpu/drm/i915/i915_gem.c:1633:
+ ret = sysfs_create_file(file_priv->client.busy_root,
+ (struct attribute *)attr);

total: 1 errors, 2 warnings, 2 checks, 164 lines checked 9021ed40837a drm/i915: Add sysfs toggle to enable per-client engine stats

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#11 lore raw

✗ Fi.CI.BAT: failure for Per client engine busyness (all aboard the sysfs train!)

In reply to Tvrtko Ursulin
== Series Details ==

== Series Details ==

Series: Per client engine busyness (all aboard the sysfs train!)
URL : https://patchwork.freedesktop.org/series/68570/
State : failure

== Summary ==

CI Bug Log - changes from CI_DRM_7187 -> Patchwork_14987
====================================================

Summary
-------

**FAILURE**

Serious unknown changes coming with Patchwork_14987 absolutely need to be verified manually.

If you think the reported changes have nothing to do with the changes introduced in Patchwork_14987, please notify your bug team to allow them to document this new failure mode, which will reduce false positives in CI.

External URL: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14987/index.html

Possible new issues
-------------------

Here are the unknown changes that may have been introduced in Patchwork_14987:

### IGT changes ###

#### Possible regressions ####

* igt@i915_selftest@live_gem_contexts:
- fi-bsw-n3050: [PASS][1] -> [INCOMPLETE][2]
[1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7187/fi-bsw-n3050/igt@i915_selftest@live_gem_contexts.html
[2]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14987/fi-bsw-n3050/igt@i915_selftest@live_gem_contexts.html

Known issues
------------

Here are the changes found in Patchwork_14987 that come from known issues:

### IGT changes ###

#### Issues hit ####

* igt@prime_busy@basic-before-default:
- fi-icl-u3: [PASS][3] -> [DMESG-WARN][4] ([fdo#107724]) +1 similar issue
[3]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7187/fi-icl-u3/igt@[email protected]
[4]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14987/fi-icl-u3/igt@[email protected]

#### Possible fixes ####

* igt@gem_exec_suspend@basic-s4-devices:
- fi-icl-u3: [DMESG-WARN][5] ([fdo#107724]) -> [PASS][6] +2 similar issues
[5]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7187/fi-icl-u3/igt@[email protected]
[6]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14987/fi-icl-u3/igt@[email protected]

* {igt@i915_selftest@live_gt_heartbeat}:
- {fi-icl-dsi}: [DMESG-FAIL][7] ([fdo#112096]) -> [PASS][8]
[7]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7187/fi-icl-dsi/igt@i915_selftest@live_gt_heartbeat.html
[8]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14987/fi-icl-dsi/igt@i915_selftest@live_gt_heartbeat.html
- fi-cml-u2: [DMESG-FAIL][9] ([fdo#112096]) -> [PASS][10]
[9]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7187/fi-cml-u2/igt@i915_selftest@live_gt_heartbeat.html
[10]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14987/fi-cml-u2/igt@i915_selftest@live_gt_heartbeat.html

* igt@i915_selftest@live_hangcheck:
- {fi-tgl-u}: [INCOMPLETE][11] ([fdo#111747]) -> [PASS][12]
[11]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7187/fi-tgl-u/igt@i915_selftest@live_hangcheck.html
[12]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14987/fi-tgl-u/igt@i915_selftest@live_hangcheck.html

* igt@kms_chamelium@hdmi-hpd-fast:
- fi-kbl-7500u: [FAIL][13] ([fdo#111407]) -> [PASS][14]
[13]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7187/fi-kbl-7500u/igt@[email protected]
[14]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14987/fi-kbl-7500u/igt@[email protected]

{name}: This element is suppressed. This means it is ignored when computing
the status of the difference (SUCCESS, WARNING, or FAILURE).

[fdo#107713]: https://bugs.freedesktop.org/show_bug.cgi?id=107713
[fdo#107724]: https://bugs.freedesktop.org/show_bug.cgi?id=107724
[fdo#111381]: https://bugs.freedesktop.org/show_bug.cgi?id=111381
[fdo#111407]: https://bugs.freedesktop.org/show_bug.cgi?id=111407
[fdo#111747]: https://bugs.freedesktop.org/show_bug.cgi?id=111747
[fdo#112096]: https://bugs.freedesktop.org/show_bug.cgi?id=112096

Participating hosts (49 -> 42)
------------------------------

Additional (1): fi-pnv-d510
Missing (8): fi-ilk-m540 fi-cml-s fi-hsw-4200u fi-byt-squawks fi-bsw-cyan fi-icl-y fi-byt-clapper fi-bdw-samus

Build changes
-------------

* CI: CI-20190529 -> None
* Linux: CI_DRM_7187 -> Patchwork_14987

CI-20190529: 20190529
CI_DRM_7187: 9df5aeba240a65ea80008020d3027484bc6055b3 @ git://anongit.freedesktop.org/gfx-ci/linux
IGT_5241: 17b87c378fa155390b13a43f141371fd899d567b @ git://anongit.freedesktop.org/xorg/app/intel-gpu-tools
Patchwork_14987: 9021ed40837a0b8ca2bb29e7c53bc15a18ee3a02 @ git://anongit.freedesktop.org/gfx-ci/linux

== Linux commits ==

9021ed40837a drm/i915: Add sysfs toggle to enable per-client engine stats 84b4d495586c drm/i915: Expose per-engine client busyness a142a38e3f25 drm/i915: Update client name on context create 2c3edc69e7b6 drm/i915: Expose list of clients in sysfs c5f5abc9d897 drm/i915: Track per-context engine busyness

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_14987/index.html
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx