Skip to contentSkip to conversation

drm/i915/guc: CTB improvements

PATCH In review [email protected] 4 participants 18 messages Nov 20, 2019 – Nov 25, 2019

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover drm/i915/guc: CTB improvements

From: John Harrison <[email protected]>

These patches improve the CTB infrastructure - Command Transport Buffer, the communication mechanism between i915 and GuC.

They are part of the (large) series for updating the i915 GuC implementation to support the new GuC API. That series is still in progress (but getting close). However, it was suggested that these patches could be pushed early to help reduce the patch burden. They are not directly related to the new GuC API and so are compatible with the old GuC implementation.

The new GuC API makes much heavier use of the CTB. Indeed, it becomes part of the command submission path. Hence, the need for optimisation, larger buffers and support for sending without a mutex lock.

Matthew Brost (3):
drm/i915/guc: Add non blocking CTB send function drm/i915/guc: Optimized CTB writes and reads drm/i915/guc: Increase size of CTB buffers

 drivers/gpu/drm/i915/gt/uc/intel_guc.h    |   2 +
 drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c | 214 +++++++++++++++-------
 drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h |  18 +-
 3 files changed, 162 insertions(+), 72 deletions(-)
Signature · 6 lines
2.21.0.5.gaeb582a983

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#1 lore raw

PATCH 1/3 drm/i915/guc: Add non blocking CTB send function

From: Matthew Brost <[email protected]>

Add non blocking CTB send fuction, intel_guc_send_nb. In order to support a non blocking CTB send fuction a spin lock is needed to protect the CTB descriptors fields. Also the non blocking call must not update the fence value as this value is owned by the blocking call (intel_guc_send).

The blocking CTB now must have a flow control mechanism to ensure the buffer isn't overrun. A lazy spin wait is used as we believe the flow control condition should be rare with properly sized buffer. A retry counter is also implemented which fails H2G CTBs once a limit is reached to prevent deadlock.

The function, intel_guc_send_nb, is exported in this patch but unused. Several patches later in the series make use of this function.

---
 drivers/gpu/drm/i915/gt/uc/intel_guc.h    |  2 +
 drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c | 97 +++++++++++++++++++----
 drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h | 10 ++-
 3 files changed, 91 insertions(+), 18 deletions(-)
drivers/gpu/drm/i915/gt/uc/intel_guc.h +2 -0
index e6400204a2bd..77c5af919ace 100644
@@ -94,6 +94,8 @@ intel_guc_send_and_receive(struct intel_guc *guc, const u32 *action, u32 len,
 	return guc->send(guc, action, len, response_buf, response_buf_size);
 }
 
+int intel_guc_send_nb(struct intel_guc_ct *ct, const u32 *action, u32 len);
+
 static inline void intel_guc_notify(struct intel_guc *guc)
 {
 	guc->notify(guc);
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +81 -16

193 lines not on this page · raw message

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h +8 -2
index 7c24d83f5c24..bc670a796bd8 100644
@@ -62,8 +62,11 @@ struct intel_guc_ct {
 	struct intel_guc_ct_channel host_channel;
 	/* other channels are tbd */
 
-	/** @lock: protects pending requests list */
-	spinlock_t lock;
+	/** @request_lock: protects pending requests list */
+	spinlock_t request_lock;
+
+	/** @send_lock: protects h2g channel */
+	spinlock_t send_lock;
 
 	/** @pending_requests: list of requests waiting for response */
 	struct list_head pending_requests;
@@ -73,6 +76,9 @@ struct intel_guc_ct {
 
 	/** @worker: worker for handling incoming requests */
 	struct work_struct worker;
+
+	/** @retry: the number of times a H2G CTB has been retried */
+	u32 retry;
 };
 
 void intel_guc_ct_init_early(struct intel_guc_ct *ct);
Signature · 6 lines
2.21.0.5.gaeb582a983

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#2 lore raw

Re: [PATCH 1/3] drm/i915/guc: Add non blocking CTB send function

Quoted 4 lines · On Thu, 21 Nov 2019 00:56:02 +0100, <[email protected]> wrote:

From: Matthew Brost <[email protected]>

Add non blocking CTB send fuction, intel_guc_send_nb. In order to support a non blocking CTB send fuction a spin lock is needed to

2x typos

Quoted 3 lines

protect the CTB descriptors fields. Also the non blocking call must not update the fence value as this value is owned by the blocking call (intel_guc_send).

you probably mean "intel_guc_send_ct", as intel_guc_send is just a wrapper around guc->send

Quoted 9 lines

The blocking CTB now must have a flow control mechanism to ensure the buffer isn't overrun. A lazy spin wait is used as we believe the flow control condition should be rare with properly sized buffer. A retry counter is also implemented which fails H2G CTBs once a limit is reached to prevent deadlock.

The function, intel_guc_send_nb, is exported in this patch but unused. Several patches later in the series make use of this function.

It's likely in yet another series

Quoted 14 lines
---
 drivers/gpu/drm/i915/gt/uc/intel_guc.h    |  2 +
 drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c | 97 +++++++++++++++++++----
 drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h | 10 ++-
 3 files changed, 91 insertions(+), 18 deletions(-)
drivers/gpu/drm/i915/gt/uc/intel_guc.h +0 -0
index e6400204a2bd..77c5af919ace 100644

drivers/gpu/drm/i915/gt/uc/intel_guc.h intel_guc_send_and_receive(struct intel_guc *guc, · line 99

Quoted 6 lines
drivers/gpu/drm/i915/gt/uc/intel_guc.h +2 -0
@@ -94,6 +94,8 @@ intel_guc_send_and_receive(struct intel_guc *guc,  
 const u32 *action, u32 len,
 	return guc->send(guc, action, len, response_buf, response_buf_size);
 }
+int intel_guc_send_nb(struct intel_guc_ct *ct, const u32 *action, u32  
 len);
+

Hmm, this mismatch of guc/ct parameter breaks the our layering. But we can keep this layering intact by introducing some flags to the existing guc_send() function. These flags could be passed as high bits in action[0], like this:

#define GUC_ACTION_FLAG_DONT_WAIT 0x80000000

int intel_guc_auth_huc(struct intel_guc *guc, u32 rsa_offset)

{
	u32 action[] = {
		INTEL_GUC_ACTION_AUTHENTICATE_HUC | GUC_ACTION_FLAG_DONT_WAIT,
		rsa_offset
	};

	return intel_guc_send(guc, action, ARRAY_SIZE(action));
}

then actual back-end of guc->send can take proper steps based on this flag:

@@ int intel_guc_send_mmio(struct intel_guc *guc, const u32 *action, u32 len,

        GEM_BUG_ON(!len);
        GEM_BUG_ON(len > guc->send_regs.count);

+ if (*action & GUC_ACTION_FLAG_DONT_WAIT)
+ return -EINVAL;
+ *action &= ~GUC_ACTION_FLAG_DONT_WAIT;
+

        /* We expect only action code */
        GEM_BUG_ON(*action & ~INTEL_GUC_MSG_CODE_MASK);

@@ @@ int intel_guc_send_ct(struct intel_guc *guc, const u32 *action, u32 len,

        u32 status = ~0; /* undefined */
        int ret;

+ if (*action & GUC_ACTION_FLAG_DONT_WAIT) {

+               GEM_BUG_ON(response_buf);
+               GEM_BUG_ON(response_buf_size);
+               return ctch_send_nb(ct, ctch, action, len);

+ }
+
mutex_lock(&guc->send_mutex);

ret = ctch_send(ct, ctch, action, len, response_buf,
response_buf_size,

Quoted 32 lines
drivers/gpu/drm/i915/gt/uc/intel_guc.h +0 -0
 static inline void intel_guc_notify(struct intel_guc *guc)
 {
drivers/gpu/drm/i915/gt/uc/intel_guc.h +0 -0
 	guc->notify(guc);
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +6 -1
index b49115517510..e50d968b15d5 100644
@@ -3,6 +3,8 @@
  * Copyright © 2016-2019 Intel Corporation
  */
+#include <linux/circ_buf.h>
+
 #include "i915_drv.h"
 #include "intel_guc_ct.h"
@@ -12,6 +14,8 @@
 #define CT_DEBUG_DRIVER(...)	do { } while (0)
 #endif
+#define MAX_RETRY		0x1000000
+
 struct ct_request {
 	struct list_head link;
 	u32 fence;
@@ -40,7 +44,8 @@ void intel_guc_ct_init_early(struct intel_guc_ct *ct)
 	/* we're using static channel owners */
 	ct->host_channel.owner = CTB_OWNER_HOST;
-	spin_lock_init(&ct->lock);
+	spin_lock_init(&ct->request_lock);
+	spin_lock_init(&ct->send_lock);
 	INIT_LIST_HEAD(&ct->pending_requests);
 	INIT_LIST_HEAD(&ct->incoming_requests);
 	INIT_WORK(&ct->worker, ct_incoming_request_worker_func);

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c static u32 ctch_get_next_fence(struct · line 301

Quoted 7 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +2 -1
@@ -291,7 +296,8 @@ static u32 ctch_get_next_fence(struct  
 intel_guc_ct_channel *ctch)
 static int ctb_write(struct intel_guc_ct_buffer *ctb,
 		     const u32 *action,
 		     u32 len /* in dwords */,
-		     u32 fence,
+		     u32 fence_value,
+		     bool enable_fence,

maybe we can just guarantee that fence=0 will never be used as a valid fence id, then this flag could be replaced with (fence != 0) check.

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c static int ctb_write(struct intel_guc_ct_buffer · line 338

Quoted 10 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +0 -0
 		     bool want_response)
 {
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +1 -1
 	struct guc_ct_buffer_desc *desc = ctb->desc;
@@ -328,18 +334,18 @@ static int ctb_write(struct intel_guc_ct_buffer  
 *ctb,
 	 * DW2+: action data
 	 */
 	header = (len << GUC_CT_MSG_LEN_SHIFT) |
-		 (GUC_CT_MSG_WRITE_FENCE_TO_DESC) |
+		 (enable_fence ? GUC_CT_MSG_WRITE_FENCE_TO_DESC : 0) |

Hmm, even if we ask fw to do not write back fence to the descriptor, IIRC current firmware will unconditionally write back return status of this non-blocking call, possibly overwriting status of the blocked call.

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c static int ctb_write(struct intel_guc_ct_buffer *ctb, · line 338

Quoted 1 line
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +0 -0
 		 (want_response ? GUC_CT_MSG_SEND_STATUS : 0) |

btw, if we switch all requests to expect reply send back over CTB, then we can possibly drop the send_mutex in CTB paths, and block only when there is no DONT_WAIT flag and we have to wait for response.

Quoted 29 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +15 -2
 		 (action[0] << GUC_CT_MSG_ACTION_SHIFT);
 	CT_DEBUG_DRIVER("CT: writing %*ph %*ph %*ph\n",
-			4, &header, 4, &fence,
+			4, &header, 4, &fence_value,
 			4 * (len - 1), &action[1]);
 	cmds[tail] = header;
 	tail = (tail + 1) % size;
-	cmds[tail] = fence;
+	cmds[tail] = fence_value;
 	tail = (tail + 1) % size;
 	for (i = 1; i < len; i++) {
@@ -440,6 +446,47 @@ static int wait_for_ct_request_update(struct  
 ct_request *req, u32 *status)
 	return err;
 }
+static inline bool ctb_has_room(struct guc_ct_buffer_desc *desc, u32  
 len)
+{
+	u32 head = READ_ONCE(desc->head);
+	u32 space;
+
+	space = CIRC_SPACE(desc->tail, head, desc->size);
+
+	return space >= len;
+}
+
+int intel_guc_send_nb(struct intel_guc_ct *ct,
+		      const u32 *action,
+		      u32 len)

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c static int wait_for_ct_request_update(struct · line 472

Quoted 10 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +10 -0
+{
+	struct intel_guc_ct_channel *ctch = &ct->host_channel;
+	struct intel_guc_ct_buffer *ctb = &ctch->ctbs[CTB_SEND];
+	struct guc_ct_buffer_desc *desc = ctb->desc;
+	int err;
+
+	GEM_BUG_ON(!ctch->enabled);
+	GEM_BUG_ON(!len);
+	GEM_BUG_ON(len & ~GUC_CT_MSG_LEN_MASK);
+	lockdep_assert_held(&ct->send_lock);

hmm, does it mean that now it's caller responsibility to spinlock on CT private lock ? That is not how other guc_send() functions work.

Quoted 26 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +18 -0
+
+	if (unlikely(!ctb_has_room(desc, (len + 1) * 4))) {
+		ct->retry++;
+		if (ct->retry >= MAX_RETRY)
+			return -EDEADLK;
+		else
+			return -EBUSY;
+	}
+
+	ct->retry = 0;
+	err = ctb_write(ctb, action, len, 0, false, false);
+	if (unlikely(err))
+		return err;
+
+	intel_guc_notify(ct_to_guc(ct));
+	return 0;
+}
+
 static int ctch_send(struct intel_guc_ct *ct,
 		     struct intel_guc_ct_channel *ctch,
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +2 -0
 		     const u32 *action,
@@ -460,17 +507,35 @@ static int ctch_send(struct intel_guc_ct *ct,
 	GEM_BUG_ON(len & ~GUC_CT_MSG_LEN_MASK);
 	GEM_BUG_ON(!response_buf && response_buf_size);
+	/*
+	 * We use a lazy spin wait loop here as we believe that if the CT

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c static int ctch_send(struct intel_guc_ct *ct, · line 520

Quoted 10 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +10 -0
+	 * buffers are sized correctly the flow control condition should be
+	 * rare.
+	 */
+retry:
+	spin_lock_irqsave(&ct->send_lock, flags);
+	if (unlikely(!ctb_has_room(desc, (len + 1) * 4))) {
+		spin_unlock_irqrestore(&ct->send_lock, flags);
+		ct->retry++;
+		if (ct->retry >= MAX_RETRY)
+			return -EDEADLK;

I'm not sure what's better: have secret deadlock hard to reproduce, or deadlocks easier to catch that helps improve to be deadlock-clean

Quoted 90 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +9 -3
+		cpu_relax();
+		goto retry;
+	}
+
+	ct->retry = 0;
 	fence = ctch_get_next_fence(ctch);
 	request.fence = fence;
 	request.status = 0;
 	request.response_len = response_buf_size;
 	request.response_buf = response_buf;
-	spin_lock_irqsave(&ct->lock, flags);
+	spin_lock(&ct->request_lock);
 	list_add_tail(&request.link, &ct->pending_requests);
-	spin_unlock_irqrestore(&ct->lock, flags);
+	spin_unlock(&ct->request_lock);
-	err = ctb_write(ctb, action, len, fence, !!response_buf);
+	err = ctb_write(ctb, action, len, fence, true, !!response_buf);
+	spin_unlock_irqrestore(&ct->send_lock, flags);
 	if (unlikely(err))
 		goto unlink;
@@ -501,9 +566,9 @@ static int ctch_send(struct intel_guc_ct *ct,
 	}

unlink:

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +2 -2
-	spin_lock_irqsave(&ct->lock, flags);
+	spin_lock_irqsave(&ct->request_lock, flags);
 	list_del(&request.link);
-	spin_unlock_irqrestore(&ct->lock, flags);
+	spin_unlock_irqrestore(&ct->request_lock, flags);
	return err;

}

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +6 -6
@@ -653,7 +718,7 @@ static int ct_handle_response(struct intel_guc_ct  
 *ct, const u32 *msg)
 	CT_DEBUG_DRIVER("CT: response fence %u status %#x\n", fence, status);
-	spin_lock(&ct->lock);
+	spin_lock(&ct->request_lock);
 	list_for_each_entry(req, &ct->pending_requests, link) {
 		if (unlikely(fence != req->fence)) {
 			CT_DEBUG_DRIVER("CT: request %u awaits response\n",
@@ -672,7 +737,7 @@ static int ct_handle_response(struct intel_guc_ct  
 *ct, const u32 *msg)
 		found = true;
 		break;
 	}
-	spin_unlock(&ct->lock);
+	spin_unlock(&ct->request_lock);
 	if (!found)
 		DRM_ERROR("CT: unsolicited response %*ph\n", 4 * msglen, msg);
@@ -710,13 +775,13 @@ static bool ct_process_incoming_requests(struct  
 intel_guc_ct *ct)
 	u32 *payload;
 	bool done;
-	spin_lock_irqsave(&ct->lock, flags);
+	spin_lock_irqsave(&ct->request_lock, flags);
 	request = list_first_entry_or_null(&ct->incoming_requests,
 					   struct ct_incoming_request, link);
 	if (request)
 		list_del(&request->link);
 	done = !!list_empty(&ct->incoming_requests);
-	spin_unlock_irqrestore(&ct->lock, flags);
+	spin_unlock_irqrestore(&ct->request_lock, flags);
 	if (!request)
 		return true;
@@ -777,9 +842,9 @@ static int ct_handle_request(struct intel_guc_ct  
 *ct, const u32 *msg)
 	}
 	memcpy(request->msg, msg, 4 * msglen);
-	spin_lock_irqsave(&ct->lock, flags);
+	spin_lock_irqsave(&ct->request_lock, flags);
 	list_add_tail(&request->link, &ct->incoming_requests);
-	spin_unlock_irqrestore(&ct->lock, flags);
+	spin_unlock_irqrestore(&ct->request_lock, flags);
 	queue_work(system_unbound_wq, &ct->worker);
 	return 0;
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h +5 -2
index 7c24d83f5c24..bc670a796bd8 100644
@@ -62,8 +62,11 @@ struct intel_guc_ct {
 	struct intel_guc_ct_channel host_channel;
 	/* other channels are tbd */
-	/** @lock: protects pending requests list */
-	spinlock_t lock;
+	/** @request_lock: protects pending requests list */
+	spinlock_t request_lock;
+
+	/** @send_lock: protects h2g channel */
+	spinlock_t send_lock;
 	/** @pending_requests: list of requests waiting for response */
 	struct list_head pending_requests;

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h struct intel_guc_ct { · line 82

Quoted 7 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h +3 -0
@@ -73,6 +76,9 @@ struct intel_guc_ct {
 	/** @worker: worker for handling incoming requests */
 	struct work_struct worker;
+
+	/** @retry: the number of times a H2G CTB has been retried */
+	u32 retry;
 };
 void intel_guc_ct_init_early(struct intel_guc_ct *ct);

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#3 lore raw

Re: [PATCH 1/3] drm/i915/guc: Add non blocking CTB send function

Quoted 7 lines · Michal Wajdeczko
Quoted 4 lines · On Thu, 21 Nov 2019 00:56:02 +0100, <[email protected]> wrote:

From: Matthew Brost <[email protected]>

Add non blocking CTB send fuction, intel_guc_send_nb. In order to support a non blocking CTB send fuction a spin lock is needed to

Quoted 9 lines

2x typos

Quoted 3 lines

protect the CTB descriptors fields. Also the non blocking call must not update the fence value as this value is owned by the blocking call (intel_guc_send).

you probably mean "intel_guc_send_ct", as intel_guc_send is just a wrapper around guc->send

Ah, yes.

Quoted 7 lines
Quoted 7 lines

The blocking CTB now must have a flow control mechanism to ensure the buffer isn't overrun. A lazy spin wait is used as we believe the flow control condition should be rare with properly sized buffer. A retry counter is also implemented which fails H2G CTBs once a limit is reached to prevent deadlock.

Quoted 5 lines
Quoted 2 lines

The function, intel_guc_send_nb, is exported in this patch but unused. Several patches later in the series make use of this function.

It's likely in yet another series

Yes, it is.

Quoted 9 lines
Quoted 9 lines

---

drivers/gpu/drm/i915/gt/uc/intel_guc.h    |  2 +
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c | 97 +++++++++++++++++++----
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h | 10 ++-
3 files changed, 91 insertions(+), 18 deletions(-)
Quoted 16 lines

drivers/gpu/drm/i915/gt/uc/intel_guc.h intel_guc_send_and_receive(struct intel_guc *guc, · line 99

Quoted 11 lines
drivers/gpu/drm/i915/gt/uc/intel_guc.h +2 -0
index e6400204a2bd..77c5af919ace 100644
@@ -94,6 +94,8 @@ intel_guc_send_and_receive(struct intel_guc *guc, 
 const u32 *action, u32 len,
 	return guc->send(guc, action, len, response_buf, response_buf_size);
 }
+int intel_guc_send_nb(struct intel_guc_ct *ct, const u32 *action, 
 u32 len);
+

Hmm, this mismatch of guc/ct parameter breaks the our layering. But we can keep this layering intact by introducing some flags to the existing guc_send() function. These flags could be passed as high bits in action[0], like this:

This seems reasonable.

Quoted 78 lines

#define GUC_ACTION_FLAG_DONT_WAIT 0x80000000

int intel_guc_auth_huc(struct intel_guc *guc, u32 rsa_offset)

{
	u32 action[] = {
		INTEL_GUC_ACTION_AUTHENTICATE_HUC | GUC_ACTION_FLAG_DONT_WAIT,
		rsa_offset
	};

	return intel_guc_send(guc, action, ARRAY_SIZE(action));
}

then actual back-end of guc->send can take proper steps based on this flag:

@@ int intel_guc_send_mmio(struct intel_guc *guc, const u32 *action, u32 len,

       GEM_BUG_ON(!len);
       GEM_BUG_ON(len > guc->send_regs.count);
(quoted diff) +4 -0
+       if (*action & GUC_ACTION_FLAG_DONT_WAIT)
+               return -EINVAL;
+       *action &= ~GUC_ACTION_FLAG_DONT_WAIT;
+
       /* We expect only action code */
       GEM_BUG_ON(*action & ~INTEL_GUC_MSG_CODE_MASK);

@@ @@ int intel_guc_send_ct(struct intel_guc *guc, const u32 *action, u32 len,

       u32 status = ~0; /* undefined */
       int ret;
(quoted diff) +6 -0
+       if (*action & GUC_ACTION_FLAG_DONT_WAIT) {
+               GEM_BUG_ON(response_buf);
+               GEM_BUG_ON(response_buf_size);
+               return ctch_send_nb(ct, ctch, action, len);
+       }
+
       mutex_lock(&guc->send_mutex);
 
       ret = ctch_send(ct, ctch, action, len, response_buf, 

response_buf_size,

Quoted 34 lines
drivers/gpu/drm/i915/gt/uc/intel_guc.h +0 -0
 static inline void intel_guc_notify(struct intel_guc *guc)
 {
	guc->notify(guc);
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +6 -1
index b49115517510..e50d968b15d5 100644
@@ -3,6 +3,8 @@
 * Copyright © 2016-2019 Intel Corporation
 */
+#include <linux/circ_buf.h>
+
 #include "i915_drv.h"
 #include "intel_guc_ct.h"
@@ -12,6 +14,8 @@
 #define CT_DEBUG_DRIVER(...)	do { } while (0)
 #endif
+#define MAX_RETRY		0x1000000
+
 struct ct_request {
 	struct list_head link;
 	u32 fence;
@@ -40,7 +44,8 @@ void intel_guc_ct_init_early(struct intel_guc_ct *ct)
 	/* we're using static channel owners */
 	ct->host_channel.owner = CTB_OWNER_HOST;
-	spin_lock_init(&ct->lock);
+	spin_lock_init(&ct->request_lock);
+	spin_lock_init(&ct->send_lock);
 	INIT_LIST_HEAD(&ct->pending_requests);
 	INIT_LIST_HEAD(&ct->incoming_requests);
 	INIT_WORK(&ct->worker, ct_incoming_request_worker_func);
@@ -291,7 +296,8 @@ static u32 ctch_get_next_fence(struct 
 intel_guc_ct_channel *ctch)
 static int ctb_write(struct intel_guc_ct_buffer *ctb,
Quoted 9 lines

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c static u32 ctch_get_next_fence(struct intel_guc_ct_channel *ctch) · line 300

Quoted 5 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +2 -1
 		     const u32 *action,
 		     u32 len /* in dwords */,
-		     u32 fence,
+		     u32 fence_value,
+		     bool enable_fence,

maybe we can just guarantee that fence=0 will never be used as a valid fence id, then this flag could be replaced with (fence != 0) check.

Yes, again seems reasonable. Initialize next_fence = 1, then increment by 2 each time and this works.

Quoted 7 lines
Quoted 7 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +0 -0
 		     bool want_response)
 {
	struct guc_ct_buffer_desc *desc = ctb->desc;
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +0 -0
@@ -328,18 +334,18 @@ static int ctb_write(struct 
 intel_guc_ct_buffer *ctb,
 	 * DW2+: action data
 	 */
Quoted 9 lines

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c static int ctb_write(struct intel_guc_ct_buffer *ctb, · line 337

Quoted 3 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +1 -1
 	header = (len << GUC_CT_MSG_LEN_SHIFT) |
-		 (GUC_CT_MSG_WRITE_FENCE_TO_DESC) |
+		 (enable_fence ? GUC_CT_MSG_WRITE_FENCE_TO_DESC : 0) |

Hmm, even if we ask fw to do not write back fence to the descriptor, IIRC current firmware will unconditionally write back return status of this non-blocking call, possibly overwriting status of the blocked call.

Yes, known problem with the interface that needs to be fixed.

Quoted 6 lines

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c static int ctb_write(struct intel_guc_ct_buffer *ctb, · line 338

Quoted 1 line
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +0 -0
 		 (want_response ? GUC_CT_MSG_SEND_STATUS : 0) |

btw, if we switch all requests to expect reply send back over CTB, then we can possibly drop the send_mutex in CTB paths, and block only when there is no DONT_WAIT flag and we have to wait for response.

Rather just wait for the GuC to fix this.

Quoted 34 lines
Quoted 34 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +20 -2
 		 (action[0] << GUC_CT_MSG_ACTION_SHIFT);
 	CT_DEBUG_DRIVER("CT: writing %*ph %*ph %*ph\n",
-			4, &header, 4, &fence,
+			4, &header, 4, &fence_value,
 			4 * (len - 1), &action[1]);
 	cmds[tail] = header;
 	tail = (tail + 1) % size;
-	cmds[tail] = fence;
+	cmds[tail] = fence_value;
 	tail = (tail + 1) % size;
 	for (i = 1; i < len; i++) {
@@ -440,6 +446,47 @@ static int wait_for_ct_request_update(struct 
 ct_request *req, u32 *status)
 	return err;
 }
+static inline bool ctb_has_room(struct guc_ct_buffer_desc *desc, 
 u32 len)
+{
+	u32 head = READ_ONCE(desc->head);
+	u32 space;
+
+	space = CIRC_SPACE(desc->tail, head, desc->size);
+
+	return space >= len;
+}
+
+int intel_guc_send_nb(struct intel_guc_ct *ct,
+		      const u32 *action,
+		      u32 len)
+{
+	struct intel_guc_ct_channel *ctch = &ct->host_channel;
+	struct intel_guc_ct_buffer *ctb = &ctch->ctbs[CTB_SEND];
+	struct guc_ct_buffer_desc *desc = ctb->desc;
+	int err;
Quoted 9 lines

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c static int wait_for_ct_request_update(struct ct_request *req, u32 *status) · line 471

Quoted 5 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +5 -0
+
+	GEM_BUG_ON(!ctch->enabled);
+	GEM_BUG_ON(!len);
+	GEM_BUG_ON(len & ~GUC_CT_MSG_LEN_MASK);
+	lockdep_assert_held(&ct->send_lock);

hmm, does it mean that now it's caller responsibility to spinlock on CT private lock ? That is not how other guc_send() functions work.

Yes, that how I would like this work as I feel like it gives more flexability to the caller on the -EBUSY case. The caller can call intel_guc_send_nb again while still holding the lock or it release lock the and use a different form of flow control. Perhaps locking / unlocking should be exposed via static inlines rather than the caller directly manipulating the lock?

Quoted 31 lines
Quoted 31 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +18 -0
+
+	if (unlikely(!ctb_has_room(desc, (len + 1) * 4))) {
+		ct->retry++;
+		if (ct->retry >= MAX_RETRY)
+			return -EDEADLK;
+		else
+			return -EBUSY;
+	}
+
+	ct->retry = 0;
+	err = ctb_write(ctb, action, len, 0, false, false);
+	if (unlikely(err))
+		return err;
+
+	intel_guc_notify(ct_to_guc(ct));
+	return 0;
+}
+
 static int ctch_send(struct intel_guc_ct *ct,
 		     struct intel_guc_ct_channel *ctch,

const u32 *action,

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +7 -0
@@ -460,17 +507,35 @@ static int ctch_send(struct intel_guc_ct *ct,
 	GEM_BUG_ON(len & ~GUC_CT_MSG_LEN_MASK);
 	GEM_BUG_ON(!response_buf && response_buf_size);
+	/*
+	 * We use a lazy spin wait loop here as we believe that if the CT
+	 * buffers are sized correctly the flow control condition should be
+	 * rare.
+	 */
+retry:
+	spin_lock_irqsave(&ct->send_lock, flags);
Quoted 9 lines

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c static int ctch_send(struct intel_guc_ct *ct, · line 521

Quoted 5 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +5 -0
+	if (unlikely(!ctb_has_room(desc, (len + 1) * 4))) {
+		spin_unlock_irqrestore(&ct->send_lock, flags);
+		ct->retry++;
+		if (ct->retry >= MAX_RETRY)
+			return -EDEADLK;

I'm not sure what's better: have secret deadlock hard to reproduce, or deadlocks easier to catch that helps improve to be deadlock-clean

This is covering the case where the has died and to avoid deadlock. Eventually we will have some GuC health check code that will trigger a full GPU reset if the GuC has died. We need a way for code spinning on the CTBs to exit.

I've already tweaked this code locally a bit to use an atomic too with the idea being the GuC health check code can set this value to have all code spinning on CTBs immediately return --EDEADLK when the GuC has died.

Quoted 93 lines
Quoted 93 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +9 -3
+		cpu_relax();
+		goto retry;
+	}
+
+	ct->retry = 0;
 	fence = ctch_get_next_fence(ctch);
 	request.fence = fence;
 	request.status = 0;
 	request.response_len = response_buf_size;
 	request.response_buf = response_buf;
-	spin_lock_irqsave(&ct->lock, flags);
+	spin_lock(&ct->request_lock);
 	list_add_tail(&request.link, &ct->pending_requests);
-	spin_unlock_irqrestore(&ct->lock, flags);
+	spin_unlock(&ct->request_lock);
-	err = ctb_write(ctb, action, len, fence, !!response_buf);
+	err = ctb_write(ctb, action, len, fence, true, !!response_buf);
+	spin_unlock_irqrestore(&ct->send_lock, flags);
 	if (unlikely(err))
 		goto unlink;
@@ -501,9 +566,9 @@ static int ctch_send(struct intel_guc_ct *ct,
 	}

unlink:

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +1 -1
-	spin_lock_irqsave(&ct->lock, flags);
+	spin_lock_irqsave(&ct->request_lock, flags);
	list_del(&request.link);
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +1 -1
-	spin_unlock_irqrestore(&ct->lock, flags);
+	spin_unlock_irqrestore(&ct->request_lock, flags);
	return err;
}
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +6 -6
@@ -653,7 +718,7 @@ static int ct_handle_response(struct 
 intel_guc_ct *ct, const u32 *msg)
 	CT_DEBUG_DRIVER("CT: response fence %u status %#x\n", fence, status);
-	spin_lock(&ct->lock);
+	spin_lock(&ct->request_lock);
 	list_for_each_entry(req, &ct->pending_requests, link) {
 		if (unlikely(fence != req->fence)) {
 			CT_DEBUG_DRIVER("CT: request %u awaits response\n",
@@ -672,7 +737,7 @@ static int ct_handle_response(struct 
 intel_guc_ct *ct, const u32 *msg)
 		found = true;
 		break;
 	}
-	spin_unlock(&ct->lock);
+	spin_unlock(&ct->request_lock);
 	if (!found)
 		DRM_ERROR("CT: unsolicited response %*ph\n", 4 * msglen, msg);
@@ -710,13 +775,13 @@ static bool 
 ct_process_incoming_requests(struct intel_guc_ct *ct)
 	u32 *payload;
 	bool done;
-	spin_lock_irqsave(&ct->lock, flags);
+	spin_lock_irqsave(&ct->request_lock, flags);
 	request = list_first_entry_or_null(&ct->incoming_requests,
 					   struct ct_incoming_request, link);
 	if (request)
 		list_del(&request->link);
 	done = !!list_empty(&ct->incoming_requests);
-	spin_unlock_irqrestore(&ct->lock, flags);
+	spin_unlock_irqrestore(&ct->request_lock, flags);
 	if (!request)
 		return true;
@@ -777,9 +842,9 @@ static int ct_handle_request(struct intel_guc_ct 
 *ct, const u32 *msg)
 	}
 	memcpy(request->msg, msg, 4 * msglen);
-	spin_lock_irqsave(&ct->lock, flags);
+	spin_lock_irqsave(&ct->request_lock, flags);
 	list_add_tail(&request->link, &ct->incoming_requests);
-	spin_unlock_irqrestore(&ct->lock, flags);
+	spin_unlock_irqrestore(&ct->request_lock, flags);
 	queue_work(system_unbound_wq, &ct->worker);
 	return 0;
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h +6 -2
index 7c24d83f5c24..bc670a796bd8 100644
@@ -62,8 +62,11 @@ struct intel_guc_ct {
 	struct intel_guc_ct_channel host_channel;
 	/* other channels are tbd */
-	/** @lock: protects pending requests list */
-	spinlock_t lock;
+	/** @request_lock: protects pending requests list */
+	spinlock_t request_lock;
+
+	/** @send_lock: protects h2g channel */
+	spinlock_t send_lock;
 	/** @pending_requests: list of requests waiting for response */
 	struct list_head pending_requests;
@@ -73,6 +76,9 @@ struct intel_guc_ct {
 	/** @worker: worker for handling incoming requests */
 	struct work_struct worker;
+
Quoted 8 lines

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h struct intel_guc_ct { · line 84

Quoted 4 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h +2 -0
+	/** @retry: the number of times a H2G CTB has been retried */
+	u32 retry;
 };
 void intel_guc_ct_init_early(struct intel_guc_ct *ct);

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#4 lore raw

Re: [PATCH 1/3] drm/i915/guc: Add non blocking CTB send function

Quoted 61 lines · Matthew Brost
Quoted 7 lines · Michal Wajdeczko
Quoted 4 lines · On Thu, 21 Nov 2019 00:56:02 +0100, <[email protected]> wrote:

From: Matthew Brost <[email protected]>

Add non blocking CTB send fuction, intel_guc_send_nb. In order to support a non blocking CTB send fuction a spin lock is needed to

Quoted 9 lines

2x typos

Quoted 3 lines

protect the CTB descriptors fields. Also the non blocking call must not update the fence value as this value is owned by the blocking call (intel_guc_send).

you probably mean "intel_guc_send_ct", as intel_guc_send is just a wrapper around guc->send

Ah, yes.

Quoted 7 lines
Quoted 7 lines

The blocking CTB now must have a flow control mechanism to ensure the buffer isn't overrun. A lazy spin wait is used as we believe the flow control condition should be rare with properly sized buffer. A retry counter is also implemented which fails H2G CTBs once a limit is reached to prevent deadlock.

Quoted 5 lines
Quoted 2 lines

The function, intel_guc_send_nb, is exported in this patch but unused. Several patches later in the series make use of this function.

It's likely in yet another series

Yes, it is.

Quoted 9 lines
Quoted 9 lines

---

drivers/gpu/drm/i915/gt/uc/intel_guc.h    |  2 +
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c | 97 +++++++++++++++++++----
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h | 10 ++-
3 files changed, 91 insertions(+), 18 deletions(-)
Quoted 16 lines

drivers/gpu/drm/i915/gt/uc/intel_guc.h intel_guc_send_and_receive(struct intel_guc · line 99

Quoted 11 lines
drivers/gpu/drm/i915/gt/uc/intel_guc.h +2 -0
index e6400204a2bd..77c5af919ace 100644
@@ -94,6 +94,8 @@ intel_guc_send_and_receive(struct intel_guc 
 *guc, const u32 *action, u32 len,
 	return guc->send(guc, action, len, response_buf, response_buf_size);
 }
+int intel_guc_send_nb(struct intel_guc_ct *ct, const u32 *action, 
 u32 len);
+

Hmm, this mismatch of guc/ct parameter breaks the our layering. But we can keep this layering intact by introducing some flags to the existing guc_send() function. These flags could be passed as high bits in action[0], like this:

Quoted 2 lines

This seems reasonable.

Prototyped this and I don't like it all. First 'action' is a const so what you are suggesting doesn't work unless that is changed. Also what if all bits in DW eventually mean something, to me overloading a field isn't a good idea if anything we should add another argument to guc->send(). But I'd honestly prefer we just leave it as is. Non-blocking only applies to CTs (not MMIO) and we have GEM_BUG_ON to protect us if this function is called incorrectly. Doing what you suggest just makes everything more complicated IMO.

Matt

Quoted 319 lines
Quoted 78 lines

#define GUC_ACTION_FLAG_DONT_WAIT 0x80000000

int intel_guc_auth_huc(struct intel_guc *guc, u32 rsa_offset)

{
	u32 action[] = {
		INTEL_GUC_ACTION_AUTHENTICATE_HUC | GUC_ACTION_FLAG_DONT_WAIT,
		rsa_offset
	};

	return intel_guc_send(guc, action, ARRAY_SIZE(action));
}

then actual back-end of guc->send can take proper steps based on this flag:

@@ int intel_guc_send_mmio(struct intel_guc *guc, const u32 *action, u32 len,

      GEM_BUG_ON(!len);
      GEM_BUG_ON(len > guc->send_regs.count);
(quoted diff) +4 -0
+       if (*action & GUC_ACTION_FLAG_DONT_WAIT)
+               return -EINVAL;
+       *action &= ~GUC_ACTION_FLAG_DONT_WAIT;
+
      /* We expect only action code */
      GEM_BUG_ON(*action & ~INTEL_GUC_MSG_CODE_MASK);

@@ @@ int intel_guc_send_ct(struct intel_guc *guc, const u32 *action, u32 len,

      u32 status = ~0; /* undefined */
      int ret;
(quoted diff) +6 -0
+       if (*action & GUC_ACTION_FLAG_DONT_WAIT) {
+               GEM_BUG_ON(response_buf);
+               GEM_BUG_ON(response_buf_size);
+               return ctch_send_nb(ct, ctch, action, len);
+       }
+
      mutex_lock(&guc->send_mutex);
 
      ret = ctch_send(ct, ctch, action, len, response_buf, 

response_buf_size,

Quoted 34 lines
drivers/gpu/drm/i915/gt/uc/intel_guc.h +0 -0
 static inline void intel_guc_notify(struct intel_guc *guc)
 {
	guc->notify(guc);
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +6 -1
index b49115517510..e50d968b15d5 100644
@@ -3,6 +3,8 @@
 * Copyright © 2016-2019 Intel Corporation
 */
+#include <linux/circ_buf.h>
+
 #include "i915_drv.h"
 #include "intel_guc_ct.h"
@@ -12,6 +14,8 @@
 #define CT_DEBUG_DRIVER(...)	do { } while (0)
 #endif
+#define MAX_RETRY		0x1000000
+
 struct ct_request {
 	struct list_head link;
 	u32 fence;
@@ -40,7 +44,8 @@ void intel_guc_ct_init_early(struct intel_guc_ct *ct)
 	/* we're using static channel owners */
 	ct->host_channel.owner = CTB_OWNER_HOST;
-	spin_lock_init(&ct->lock);
+	spin_lock_init(&ct->request_lock);
+	spin_lock_init(&ct->send_lock);
 	INIT_LIST_HEAD(&ct->pending_requests);
 	INIT_LIST_HEAD(&ct->incoming_requests);
 	INIT_WORK(&ct->worker, ct_incoming_request_worker_func);
@@ -291,7 +296,8 @@ static u32 ctch_get_next_fence(struct 
 intel_guc_ct_channel *ctch)
 static int ctb_write(struct intel_guc_ct_buffer *ctb,
Quoted 9 lines

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c static u32 ctch_get_next_fence(struct intel_guc_ct_channel *ctch) · line 300

Quoted 5 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +2 -1
 		     const u32 *action,
 		     u32 len /* in dwords */,
-		     u32 fence,
+		     u32 fence_value,
+		     bool enable_fence,

maybe we can just guarantee that fence=0 will never be used as a valid fence id, then this flag could be replaced with (fence != 0) check.

Yes, again seems reasonable. Initialize next_fence = 1, then increment by 2 each time and this works.

Quoted 7 lines
Quoted 7 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +0 -0
 		     bool want_response)
 {
	struct guc_ct_buffer_desc *desc = ctb->desc;
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +0 -0
@@ -328,18 +334,18 @@ static int ctb_write(struct 
 intel_guc_ct_buffer *ctb,
 	 * DW2+: action data
 	 */
Quoted 9 lines

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c static int ctb_write(struct intel_guc_ct_buffer *ctb, · line 337

Quoted 3 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +1 -1
 	header = (len << GUC_CT_MSG_LEN_SHIFT) |
-		 (GUC_CT_MSG_WRITE_FENCE_TO_DESC) |
+		 (enable_fence ? GUC_CT_MSG_WRITE_FENCE_TO_DESC : 0) |

Hmm, even if we ask fw to do not write back fence to the descriptor, IIRC current firmware will unconditionally write back return status of this non-blocking call, possibly overwriting status of the blocked call.

Yes, known problem with the interface that needs to be fixed.

Quoted 6 lines

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c static int ctb_write(struct intel_guc_ct_buffer *ctb, · line 338

Quoted 1 line
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +0 -0
 		 (want_response ? GUC_CT_MSG_SEND_STATUS : 0) |

btw, if we switch all requests to expect reply send back over CTB, then we can possibly drop the send_mutex in CTB paths, and block only when there is no DONT_WAIT flag and we have to wait for response.

Rather just wait for the GuC to fix this.

Quoted 34 lines
Quoted 34 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +20 -2
 		 (action[0] << GUC_CT_MSG_ACTION_SHIFT);
 	CT_DEBUG_DRIVER("CT: writing %*ph %*ph %*ph\n",
-			4, &header, 4, &fence,
+			4, &header, 4, &fence_value,
 			4 * (len - 1), &action[1]);
 	cmds[tail] = header;
 	tail = (tail + 1) % size;
-	cmds[tail] = fence;
+	cmds[tail] = fence_value;
 	tail = (tail + 1) % size;
 	for (i = 1; i < len; i++) {
@@ -440,6 +446,47 @@ static int wait_for_ct_request_update(struct 
 ct_request *req, u32 *status)
 	return err;
 }
+static inline bool ctb_has_room(struct guc_ct_buffer_desc *desc, 
 u32 len)
+{
+	u32 head = READ_ONCE(desc->head);
+	u32 space;
+
+	space = CIRC_SPACE(desc->tail, head, desc->size);
+
+	return space >= len;
+}
+
+int intel_guc_send_nb(struct intel_guc_ct *ct,
+		      const u32 *action,
+		      u32 len)
+{
+	struct intel_guc_ct_channel *ctch = &ct->host_channel;
+	struct intel_guc_ct_buffer *ctb = &ctch->ctbs[CTB_SEND];
+	struct guc_ct_buffer_desc *desc = ctb->desc;
+	int err;
Quoted 9 lines

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c static int wait_for_ct_request_update(struct ct_request *req, u32 *status) · line 471

Quoted 5 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +5 -0
+
+	GEM_BUG_ON(!ctch->enabled);
+	GEM_BUG_ON(!len);
+	GEM_BUG_ON(len & ~GUC_CT_MSG_LEN_MASK);
+	lockdep_assert_held(&ct->send_lock);

hmm, does it mean that now it's caller responsibility to spinlock on CT private lock ? That is not how other guc_send() functions work.

Yes, that how I would like this work as I feel like it gives more flexability to the caller on the -EBUSY case. The caller can call intel_guc_send_nb again while still holding the lock or it release lock the and use a different form of flow control. Perhaps locking / unlocking should be exposed via static inlines rather than the caller directly manipulating the lock?

Quoted 31 lines
Quoted 31 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +18 -0
+
+	if (unlikely(!ctb_has_room(desc, (len + 1) * 4))) {
+		ct->retry++;
+		if (ct->retry >= MAX_RETRY)
+			return -EDEADLK;
+		else
+			return -EBUSY;
+	}
+
+	ct->retry = 0;
+	err = ctb_write(ctb, action, len, 0, false, false);
+	if (unlikely(err))
+		return err;
+
+	intel_guc_notify(ct_to_guc(ct));
+	return 0;
+}
+
 static int ctch_send(struct intel_guc_ct *ct,
 		     struct intel_guc_ct_channel *ctch,

const u32 *action,

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +7 -0
@@ -460,17 +507,35 @@ static int ctch_send(struct intel_guc_ct *ct,
 	GEM_BUG_ON(len & ~GUC_CT_MSG_LEN_MASK);
 	GEM_BUG_ON(!response_buf && response_buf_size);
+	/*
+	 * We use a lazy spin wait loop here as we believe that if the CT
+	 * buffers are sized correctly the flow control condition should be
+	 * rare.
+	 */
+retry:
+	spin_lock_irqsave(&ct->send_lock, flags);
Quoted 9 lines

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c static int ctch_send(struct intel_guc_ct *ct, · line 521

Quoted 5 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +5 -0
+	if (unlikely(!ctb_has_room(desc, (len + 1) * 4))) {
+		spin_unlock_irqrestore(&ct->send_lock, flags);
+		ct->retry++;
+		if (ct->retry >= MAX_RETRY)
+			return -EDEADLK;

I'm not sure what's better: have secret deadlock hard to reproduce, or deadlocks easier to catch that helps improve to be deadlock-clean

This is covering the case where the has died and to avoid deadlock. Eventually we will have some GuC health check code that will trigger a full GPU reset if the GuC has died. We need a way for code spinning on the CTBs to exit.

I've already tweaked this code locally a bit to use an atomic too with the idea being the GuC health check code can set this value to have all code spinning on CTBs immediately return --EDEADLK when the GuC has died.

Quoted 93 lines
Quoted 93 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +9 -3
+		cpu_relax();
+		goto retry;
+	}
+
+	ct->retry = 0;
 	fence = ctch_get_next_fence(ctch);
 	request.fence = fence;
 	request.status = 0;
 	request.response_len = response_buf_size;
 	request.response_buf = response_buf;
-	spin_lock_irqsave(&ct->lock, flags);
+	spin_lock(&ct->request_lock);
 	list_add_tail(&request.link, &ct->pending_requests);
-	spin_unlock_irqrestore(&ct->lock, flags);
+	spin_unlock(&ct->request_lock);
-	err = ctb_write(ctb, action, len, fence, !!response_buf);
+	err = ctb_write(ctb, action, len, fence, true, !!response_buf);
+	spin_unlock_irqrestore(&ct->send_lock, flags);
 	if (unlikely(err))
 		goto unlink;
@@ -501,9 +566,9 @@ static int ctch_send(struct intel_guc_ct *ct,
 	}

unlink:

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +1 -1
-	spin_lock_irqsave(&ct->lock, flags);
+	spin_lock_irqsave(&ct->request_lock, flags);
	list_del(&request.link);
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +1 -1
-	spin_unlock_irqrestore(&ct->lock, flags);
+	spin_unlock_irqrestore(&ct->request_lock, flags);
	return err;
}
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +6 -6
@@ -653,7 +718,7 @@ static int ct_handle_response(struct 
 intel_guc_ct *ct, const u32 *msg)
 	CT_DEBUG_DRIVER("CT: response fence %u status %#x\n", fence, status);
-	spin_lock(&ct->lock);
+	spin_lock(&ct->request_lock);
 	list_for_each_entry(req, &ct->pending_requests, link) {
 		if (unlikely(fence != req->fence)) {
 			CT_DEBUG_DRIVER("CT: request %u awaits response\n",
@@ -672,7 +737,7 @@ static int ct_handle_response(struct 
 intel_guc_ct *ct, const u32 *msg)
 		found = true;
 		break;
 	}
-	spin_unlock(&ct->lock);
+	spin_unlock(&ct->request_lock);
 	if (!found)
 		DRM_ERROR("CT: unsolicited response %*ph\n", 4 * msglen, msg);
@@ -710,13 +775,13 @@ static bool 
 ct_process_incoming_requests(struct intel_guc_ct *ct)
 	u32 *payload;
 	bool done;
-	spin_lock_irqsave(&ct->lock, flags);
+	spin_lock_irqsave(&ct->request_lock, flags);
 	request = list_first_entry_or_null(&ct->incoming_requests,
 					   struct ct_incoming_request, link);
 	if (request)
 		list_del(&request->link);
 	done = !!list_empty(&ct->incoming_requests);
-	spin_unlock_irqrestore(&ct->lock, flags);
+	spin_unlock_irqrestore(&ct->request_lock, flags);
 	if (!request)
 		return true;
@@ -777,9 +842,9 @@ static int ct_handle_request(struct 
 intel_guc_ct *ct, const u32 *msg)
 	}
 	memcpy(request->msg, msg, 4 * msglen);
-	spin_lock_irqsave(&ct->lock, flags);
+	spin_lock_irqsave(&ct->request_lock, flags);
 	list_add_tail(&request->link, &ct->incoming_requests);
-	spin_unlock_irqrestore(&ct->lock, flags);
+	spin_unlock_irqrestore(&ct->request_lock, flags);
 	queue_work(system_unbound_wq, &ct->worker);
 	return 0;
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h +6 -2
index 7c24d83f5c24..bc670a796bd8 100644
@@ -62,8 +62,11 @@ struct intel_guc_ct {
 	struct intel_guc_ct_channel host_channel;
 	/* other channels are tbd */
-	/** @lock: protects pending requests list */
-	spinlock_t lock;
+	/** @request_lock: protects pending requests list */
+	spinlock_t request_lock;
+
+	/** @send_lock: protects h2g channel */
+	spinlock_t send_lock;
 	/** @pending_requests: list of requests waiting for response */
 	struct list_head pending_requests;
@@ -73,6 +76,9 @@ struct intel_guc_ct {
 	/** @worker: worker for handling incoming requests */
 	struct work_struct worker;
+
Quoted 8 lines

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h struct intel_guc_ct { · line 84

Quoted 4 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h +2 -0
+	/** @retry: the number of times a H2G CTB has been retried */
+	u32 retry;
 };
 void intel_guc_ct_init_early(struct intel_guc_ct *ct);

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Quoted 4 lines

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#5 lore raw

Re: [PATCH 1/3] drm/i915/guc: Add non blocking CTB send function

Quoted 62 lines · Matthew Brost
Quoted 61 lines · Matthew Brost
Quoted 7 lines · Michal Wajdeczko
Quoted 4 lines · On Thu, 21 Nov 2019 00:56:02 +0100, <[email protected]> wrote:

From: Matthew Brost <[email protected]>

Add non blocking CTB send fuction, intel_guc_send_nb. In order to support a non blocking CTB send fuction a spin lock is needed to

Quoted 9 lines

2x typos

Quoted 3 lines

protect the CTB descriptors fields. Also the non blocking call must not update the fence value as this value is owned by the blocking call (intel_guc_send).

you probably mean "intel_guc_send_ct", as intel_guc_send is just a wrapper around guc->send

Ah, yes.

Quoted 7 lines
Quoted 7 lines

The blocking CTB now must have a flow control mechanism to ensure the buffer isn't overrun. A lazy spin wait is used as we believe the flow control condition should be rare with properly sized buffer. A retry counter is also implemented which fails H2G CTBs once a limit is reached to prevent deadlock.

Quoted 5 lines
Quoted 2 lines

The function, intel_guc_send_nb, is exported in this patch but unused. Several patches later in the series make use of this function.

It's likely in yet another series

Yes, it is.

Quoted 25 lines
Quoted 14 lines

---

drivers/gpu/drm/i915/gt/uc/intel_guc.h    |  2 +
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c | 97 +++++++++++++++++++----
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h | 10 ++-
3 files changed, 91 insertions(+), 18 deletions(-)
drivers/gpu/drm/i915/gt/uc/intel_guc.h +0 -0
index e6400204a2bd..77c5af919ace 100644

drivers/gpu/drm/i915/gt/uc/intel_guc.h intel_guc_send_and_receive(struct intel_guc *guc, · line 99

Quoted 6 lines
drivers/gpu/drm/i915/gt/uc/intel_guc.h +2 -0
@@ -94,6 +94,8 @@ intel_guc_send_and_receive(struct intel_guc *guc,  
 const u32 *action, u32 len,
 	return guc->send(guc, action, len, response_buf, response_buf_size);
 }
+int intel_guc_send_nb(struct intel_guc_ct *ct, const u32 *action,  
 u32 len);
+

Hmm, this mismatch of guc/ct parameter breaks the our layering. But we can keep this layering intact by introducing some flags to the existing guc_send() function. These flags could be passed as high bits in action[0], like this:

Quoted 5 lines
Quoted 2 lines

This seems reasonable.

Prototyped this and I don't like it all. First 'action' is a const so what you are suggesting doesn't work unless that is changed.

I guess only explicit clearing of that flag from const action was a mistake, rest should be ok.

Quoted 2 lines

Also what if all bits in DW
eventually mean something,

In all guc_send functions we only use lower 16 bits for action code. This action code is then passed in MMIO messages in bits 15:0 and in CTB messages in bits 31:16.

Quoted 2 lines

to me overloading a field isn't a good idea if anything we should add another argument to guc->send().

guc->send already takes 5 params:

	int (*send)(struct intel_guc *guc, const u32 *data, u32 len,
		    u32 *response_buf, u32 response_buf_size);

We can add explicit flags, but I see nothing wrong in using free top bits from data[0] to pass these flags there.

Quoted 3 lines

But I'd honestly prefer
we just leave it as is. Non-blocking only applies to CTs (not MMIO) and we have GEM_BUG_ON to protect us if this function is called incorrectly.

Well, I guess many would still prefer to be hit by WARN from guc-send-nop rather than BUG_ON from guc-send-nb

Quoted 2 lines

Doing what you
suggest just makes everything more complicated IMO.

I'm not sold that adding GUC_ACTION_FLAG_DONT_WAIT to action[0] (as shown below) is that complicated

Michal

Quoted 327 lines

Matt

Quoted 324 lines
Quoted 79 lines

#define GUC_ACTION_FLAG_DONT_WAIT 0x80000000

int intel_guc_auth_huc(struct intel_guc *guc, u32 rsa_offset)

{
	u32 action[] = {
		INTEL_GUC_ACTION_AUTHENTICATE_HUC | GUC_ACTION_FLAG_DONT_WAIT,
		rsa_offset
	};

	return intel_guc_send(guc, action, ARRAY_SIZE(action));
}

then actual back-end of guc->send can take proper steps based on this flag:

@@ int intel_guc_send_mmio(struct intel_guc *guc, const u32 *action, u32 len,

      GEM_BUG_ON(!len);
      GEM_BUG_ON(len > guc->send_regs.count);
(quoted diff) +4 -0
+       if (*action & GUC_ACTION_FLAG_DONT_WAIT)
+               return -EINVAL;
+       *action &= ~GUC_ACTION_FLAG_DONT_WAIT;
+
      /* We expect only action code */
      GEM_BUG_ON(*action & ~INTEL_GUC_MSG_CODE_MASK);

@@ @@ int intel_guc_send_ct(struct intel_guc *guc, const u32 *action, u32 len,

      u32 status = ~0; /* undefined */
      int ret;
(quoted diff) +6 -0
+       if (*action & GUC_ACTION_FLAG_DONT_WAIT) {
+               GEM_BUG_ON(response_buf);
+               GEM_BUG_ON(response_buf_size);
+               return ctch_send_nb(ct, ctch, action, len);
+       }
+
      mutex_lock(&guc->send_mutex);
 
      ret = ctch_send(ct, ctch, action, len, response_buf,  

response_buf_size,

Quoted 35 lines
drivers/gpu/drm/i915/gt/uc/intel_guc.h +0 -0
 static inline void intel_guc_notify(struct intel_guc *guc)
 {
	guc->notify(guc);
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +6 -1
index b49115517510..e50d968b15d5 100644
@@ -3,6 +3,8 @@
 * Copyright © 2016-2019 Intel Corporation
 */
+#include <linux/circ_buf.h>
+
 #include "i915_drv.h"
 #include "intel_guc_ct.h"
@@ -12,6 +14,8 @@
 #define CT_DEBUG_DRIVER(...)	do { } while (0)
 #endif
+#define MAX_RETRY		0x1000000
+
 struct ct_request {
 	struct list_head link;
 	u32 fence;
@@ -40,7 +44,8 @@ void intel_guc_ct_init_early(struct intel_guc_ct  
 *ct)
 	/* we're using static channel owners */
 	ct->host_channel.owner = CTB_OWNER_HOST;
-	spin_lock_init(&ct->lock);
+	spin_lock_init(&ct->request_lock);
+	spin_lock_init(&ct->send_lock);
 	INIT_LIST_HEAD(&ct->pending_requests);
 	INIT_LIST_HEAD(&ct->incoming_requests);
 	INIT_WORK(&ct->worker, ct_incoming_request_worker_func);
@@ -291,7 +296,8 @@ static u32 ctch_get_next_fence(struct  
 intel_guc_ct_channel *ctch)
 static int ctb_write(struct intel_guc_ct_buffer *ctb,
Quoted 9 lines

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c static u32 ctch_get_next_fence(struct intel_guc_ct_channel *ctch) · line 300

Quoted 5 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +2 -1
 		     const u32 *action,
 		     u32 len /* in dwords */,
-		     u32 fence,
+		     u32 fence_value,
+		     bool enable_fence,

maybe we can just guarantee that fence=0 will never be used as a valid fence id, then this flag could be replaced with (fence != 0) check.

Yes, again seems reasonable. Initialize next_fence = 1, then increment by 2 each time and this works.

Quoted 7 lines
Quoted 7 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +0 -0
 		     bool want_response)
 {
	struct guc_ct_buffer_desc *desc = ctb->desc;
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +0 -0
@@ -328,18 +334,18 @@ static int ctb_write(struct intel_guc_ct_buffer  
 *ctb,
 	 * DW2+: action data
 	 */
Quoted 9 lines

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c static int ctb_write(struct intel_guc_ct_buffer *ctb, · line 337

Quoted 3 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +1 -1
 	header = (len << GUC_CT_MSG_LEN_SHIFT) |
-		 (GUC_CT_MSG_WRITE_FENCE_TO_DESC) |
+		 (enable_fence ? GUC_CT_MSG_WRITE_FENCE_TO_DESC : 0) |

Hmm, even if we ask fw to do not write back fence to the descriptor, IIRC current firmware will unconditionally write back return status of this non-blocking call, possibly overwriting status of the blocked call.

Yes, known problem with the interface that needs to be fixed.

Quoted 6 lines

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c static int ctb_write(struct intel_guc_ct_buffer *ctb, · line 338

Quoted 1 line
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +0 -0
 		 (want_response ? GUC_CT_MSG_SEND_STATUS : 0) |

btw, if we switch all requests to expect reply send back over CTB, then we can possibly drop the send_mutex in CTB paths, and block only when there is no DONT_WAIT flag and we have to wait for response.

Rather just wait for the GuC to fix this.

Quoted 34 lines
Quoted 34 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +20 -2
 		 (action[0] << GUC_CT_MSG_ACTION_SHIFT);
 	CT_DEBUG_DRIVER("CT: writing %*ph %*ph %*ph\n",
-			4, &header, 4, &fence,
+			4, &header, 4, &fence_value,
 			4 * (len - 1), &action[1]);
 	cmds[tail] = header;
 	tail = (tail + 1) % size;
-	cmds[tail] = fence;
+	cmds[tail] = fence_value;
 	tail = (tail + 1) % size;
 	for (i = 1; i < len; i++) {
@@ -440,6 +446,47 @@ static int wait_for_ct_request_update(struct  
 ct_request *req, u32 *status)
 	return err;
 }
+static inline bool ctb_has_room(struct guc_ct_buffer_desc *desc, u32  
 len)
+{
+	u32 head = READ_ONCE(desc->head);
+	u32 space;
+
+	space = CIRC_SPACE(desc->tail, head, desc->size);
+
+	return space >= len;
+}
+
+int intel_guc_send_nb(struct intel_guc_ct *ct,
+		      const u32 *action,
+		      u32 len)
+{
+	struct intel_guc_ct_channel *ctch = &ct->host_channel;
+	struct intel_guc_ct_buffer *ctb = &ctch->ctbs[CTB_SEND];
+	struct guc_ct_buffer_desc *desc = ctb->desc;
+	int err;
Quoted 9 lines

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c static int wait_for_ct_request_update(struct ct_request *req, u32 *status) · line 471

Quoted 5 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +5 -0
+
+	GEM_BUG_ON(!ctch->enabled);
+	GEM_BUG_ON(!len);
+	GEM_BUG_ON(len & ~GUC_CT_MSG_LEN_MASK);
+	lockdep_assert_held(&ct->send_lock);

hmm, does it mean that now it's caller responsibility to spinlock on CT private lock ? That is not how other guc_send() functions work.

Yes, that how I would like this work as I feel like it gives more flexability to the caller on the -EBUSY case. The caller can call intel_guc_send_nb again while still holding the lock or it release lock the and use a different form of flow control. Perhaps locking / unlocking should be exposed via static inlines rather than the caller directly manipulating the lock?

Quoted 31 lines
Quoted 31 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +18 -0
+
+	if (unlikely(!ctb_has_room(desc, (len + 1) * 4))) {
+		ct->retry++;
+		if (ct->retry >= MAX_RETRY)
+			return -EDEADLK;
+		else
+			return -EBUSY;
+	}
+
+	ct->retry = 0;
+	err = ctb_write(ctb, action, len, 0, false, false);
+	if (unlikely(err))
+		return err;
+
+	intel_guc_notify(ct_to_guc(ct));
+	return 0;
+}
+
 static int ctch_send(struct intel_guc_ct *ct,
 		     struct intel_guc_ct_channel *ctch,

const u32 *action,

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +7 -0
@@ -460,17 +507,35 @@ static int ctch_send(struct intel_guc_ct *ct,
 	GEM_BUG_ON(len & ~GUC_CT_MSG_LEN_MASK);
 	GEM_BUG_ON(!response_buf && response_buf_size);
+	/*
+	 * We use a lazy spin wait loop here as we believe that if the CT
+	 * buffers are sized correctly the flow control condition should be
+	 * rare.
+	 */
+retry:
+	spin_lock_irqsave(&ct->send_lock, flags);
Quoted 9 lines

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c static int ctch_send(struct intel_guc_ct *ct, · line 521

Quoted 5 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +5 -0
+	if (unlikely(!ctb_has_room(desc, (len + 1) * 4))) {
+		spin_unlock_irqrestore(&ct->send_lock, flags);
+		ct->retry++;
+		if (ct->retry >= MAX_RETRY)
+			return -EDEADLK;

I'm not sure what's better: have secret deadlock hard to reproduce, or deadlocks easier to catch that helps improve to be deadlock-clean

This is covering the case where the has died and to avoid deadlock. Eventually we will have some GuC health check code that will trigger a full GPU reset if the GuC has died. We need a way for code spinning on the CTBs to exit.

I've already tweaked this code locally a bit to use an atomic too with the idea being the GuC health check code can set this value to have all code spinning on CTBs immediately return --EDEADLK when the GuC has died.

Quoted 93 lines
Quoted 93 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +9 -3
+		cpu_relax();
+		goto retry;
+	}
+
+	ct->retry = 0;
 	fence = ctch_get_next_fence(ctch);
 	request.fence = fence;
 	request.status = 0;
 	request.response_len = response_buf_size;
 	request.response_buf = response_buf;
-	spin_lock_irqsave(&ct->lock, flags);
+	spin_lock(&ct->request_lock);
 	list_add_tail(&request.link, &ct->pending_requests);
-	spin_unlock_irqrestore(&ct->lock, flags);
+	spin_unlock(&ct->request_lock);
-	err = ctb_write(ctb, action, len, fence, !!response_buf);
+	err = ctb_write(ctb, action, len, fence, true, !!response_buf);
+	spin_unlock_irqrestore(&ct->send_lock, flags);
 	if (unlikely(err))
 		goto unlink;
@@ -501,9 +566,9 @@ static int ctch_send(struct intel_guc_ct *ct,
 	}

unlink:

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +1 -1
-	spin_lock_irqsave(&ct->lock, flags);
+	spin_lock_irqsave(&ct->request_lock, flags);
	list_del(&request.link);
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +1 -1
-	spin_unlock_irqrestore(&ct->lock, flags);
+	spin_unlock_irqrestore(&ct->request_lock, flags);
	return err;
}
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +6 -6
@@ -653,7 +718,7 @@ static int ct_handle_response(struct intel_guc_ct  
 *ct, const u32 *msg)
 	CT_DEBUG_DRIVER("CT: response fence %u status %#x\n", fence, status);
-	spin_lock(&ct->lock);
+	spin_lock(&ct->request_lock);
 	list_for_each_entry(req, &ct->pending_requests, link) {
 		if (unlikely(fence != req->fence)) {
 			CT_DEBUG_DRIVER("CT: request %u awaits response\n",
@@ -672,7 +737,7 @@ static int ct_handle_response(struct intel_guc_ct  
 *ct, const u32 *msg)
 		found = true;
 		break;
 	}
-	spin_unlock(&ct->lock);
+	spin_unlock(&ct->request_lock);
 	if (!found)
 		DRM_ERROR("CT: unsolicited response %*ph\n", 4 * msglen, msg);
@@ -710,13 +775,13 @@ static bool ct_process_incoming_requests(struct  
 intel_guc_ct *ct)
 	u32 *payload;
 	bool done;
-	spin_lock_irqsave(&ct->lock, flags);
+	spin_lock_irqsave(&ct->request_lock, flags);
 	request = list_first_entry_or_null(&ct->incoming_requests,
 					   struct ct_incoming_request, link);
 	if (request)
 		list_del(&request->link);
 	done = !!list_empty(&ct->incoming_requests);
-	spin_unlock_irqrestore(&ct->lock, flags);
+	spin_unlock_irqrestore(&ct->request_lock, flags);
 	if (!request)
 		return true;
@@ -777,9 +842,9 @@ static int ct_handle_request(struct intel_guc_ct  
 *ct, const u32 *msg)
 	}
 	memcpy(request->msg, msg, 4 * msglen);
-	spin_lock_irqsave(&ct->lock, flags);
+	spin_lock_irqsave(&ct->request_lock, flags);
 	list_add_tail(&request->link, &ct->incoming_requests);
-	spin_unlock_irqrestore(&ct->lock, flags);
+	spin_unlock_irqrestore(&ct->request_lock, flags);
 	queue_work(system_unbound_wq, &ct->worker);
 	return 0;
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h +6 -2
index 7c24d83f5c24..bc670a796bd8 100644
@@ -62,8 +62,11 @@ struct intel_guc_ct {
 	struct intel_guc_ct_channel host_channel;
 	/* other channels are tbd */
-	/** @lock: protects pending requests list */
-	spinlock_t lock;
+	/** @request_lock: protects pending requests list */
+	spinlock_t request_lock;
+
+	/** @send_lock: protects h2g channel */
+	spinlock_t send_lock;
 	/** @pending_requests: list of requests waiting for response */
 	struct list_head pending_requests;
@@ -73,6 +76,9 @@ struct intel_guc_ct {
 	/** @worker: worker for handling incoming requests */
 	struct work_struct worker;
+
Quoted 8 lines

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h struct intel_guc_ct { · line 84

Quoted 4 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h +2 -0
+	/** @retry: the number of times a H2G CTB has been retried */
+	u32 retry;
 };
 void intel_guc_ct_init_early(struct intel_guc_ct *ct);

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#6 lore raw

Re: [PATCH 1/3] drm/i915/guc: Add non blocking CTB send function

In reply to Matthew Brost
Quoted 151 lines · Matthew Brost
Quoted 7 lines · Michal Wajdeczko
Quoted 4 lines · On Thu, 21 Nov 2019 00:56:02 +0100, <[email protected]> wrote:

From: Matthew Brost <[email protected]>

Add non blocking CTB send fuction, intel_guc_send_nb. In order to support a non blocking CTB send fuction a spin lock is needed to

Quoted 9 lines

2x typos

Quoted 3 lines

protect the CTB descriptors fields. Also the non blocking call must not update the fence value as this value is owned by the blocking call (intel_guc_send).

you probably mean "intel_guc_send_ct", as intel_guc_send is just a wrapper around guc->send

Ah, yes.

Quoted 7 lines
Quoted 7 lines

The blocking CTB now must have a flow control mechanism to ensure the buffer isn't overrun. A lazy spin wait is used as we believe the flow control condition should be rare with properly sized buffer. A retry counter is also implemented which fails H2G CTBs once a limit is reached to prevent deadlock.

Quoted 5 lines
Quoted 2 lines

The function, intel_guc_send_nb, is exported in this patch but unused. Several patches later in the series make use of this function.

It's likely in yet another series

Yes, it is.

Quoted 9 lines
Quoted 9 lines

---

drivers/gpu/drm/i915/gt/uc/intel_guc.h    |  2 +
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c | 97 +++++++++++++++++++----
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h | 10 ++-
3 files changed, 91 insertions(+), 18 deletions(-)
Quoted 16 lines

drivers/gpu/drm/i915/gt/uc/intel_guc.h intel_guc_send_and_receive(struct intel_guc *guc, · line 99

Quoted 11 lines
drivers/gpu/drm/i915/gt/uc/intel_guc.h +2 -0
index e6400204a2bd..77c5af919ace 100644
@@ -94,6 +94,8 @@ intel_guc_send_and_receive(struct intel_guc *guc,  
 const u32 *action, u32 len,
 	return guc->send(guc, action, len, response_buf, response_buf_size);
 }
+int intel_guc_send_nb(struct intel_guc_ct *ct, const u32 *action, u32  
 len);
+

Hmm, this mismatch of guc/ct parameter breaks the our layering. But we can keep this layering intact by introducing some flags to the existing guc_send() function. These flags could be passed as high bits in action[0], like this:

This seems reasonable.

Quoted 78 lines

#define GUC_ACTION_FLAG_DONT_WAIT 0x80000000

int intel_guc_auth_huc(struct intel_guc *guc, u32 rsa_offset)

{
	u32 action[] = {
		INTEL_GUC_ACTION_AUTHENTICATE_HUC | GUC_ACTION_FLAG_DONT_WAIT,
		rsa_offset
	};

	return intel_guc_send(guc, action, ARRAY_SIZE(action));
}

then actual back-end of guc->send can take proper steps based on this flag:

@@ int intel_guc_send_mmio(struct intel_guc *guc, const u32 *action, u32 len,

       GEM_BUG_ON(!len);
       GEM_BUG_ON(len > guc->send_regs.count);
(quoted diff) +4 -0
+       if (*action & GUC_ACTION_FLAG_DONT_WAIT)
+               return -EINVAL;
+       *action &= ~GUC_ACTION_FLAG_DONT_WAIT;
+
       /* We expect only action code */
       GEM_BUG_ON(*action & ~INTEL_GUC_MSG_CODE_MASK);

@@ @@ int intel_guc_send_ct(struct intel_guc *guc, const u32 *action, u32 len,

       u32 status = ~0; /* undefined */
       int ret;
(quoted diff) +6 -0
+       if (*action & GUC_ACTION_FLAG_DONT_WAIT) {
+               GEM_BUG_ON(response_buf);
+               GEM_BUG_ON(response_buf_size);
+               return ctch_send_nb(ct, ctch, action, len);
+       }
+
       mutex_lock(&guc->send_mutex);
 
       ret = ctch_send(ct, ctch, action, len, response_buf,  

response_buf_size,

Quoted 34 lines
drivers/gpu/drm/i915/gt/uc/intel_guc.h +0 -0
 static inline void intel_guc_notify(struct intel_guc *guc)
 {
	guc->notify(guc);
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +6 -1
index b49115517510..e50d968b15d5 100644
@@ -3,6 +3,8 @@
 * Copyright © 2016-2019 Intel Corporation
 */
+#include <linux/circ_buf.h>
+
 #include "i915_drv.h"
 #include "intel_guc_ct.h"
@@ -12,6 +14,8 @@
 #define CT_DEBUG_DRIVER(...)	do { } while (0)
 #endif
+#define MAX_RETRY		0x1000000
+
 struct ct_request {
 	struct list_head link;
 	u32 fence;
@@ -40,7 +44,8 @@ void intel_guc_ct_init_early(struct intel_guc_ct *ct)
 	/* we're using static channel owners */
 	ct->host_channel.owner = CTB_OWNER_HOST;
-	spin_lock_init(&ct->lock);
+	spin_lock_init(&ct->request_lock);
+	spin_lock_init(&ct->send_lock);
 	INIT_LIST_HEAD(&ct->pending_requests);
 	INIT_LIST_HEAD(&ct->incoming_requests);
 	INIT_WORK(&ct->worker, ct_incoming_request_worker_func);
@@ -291,7 +296,8 @@ static u32 ctch_get_next_fence(struct  
 intel_guc_ct_channel *ctch)
 static int ctb_write(struct intel_guc_ct_buffer *ctb,
Quoted 9 lines

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c static u32 ctch_get_next_fence(struct intel_guc_ct_channel *ctch) · line 300

Quoted 5 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +2 -1
 		     const u32 *action,
 		     u32 len /* in dwords */,
-		     u32 fence,
+		     u32 fence_value,
+		     bool enable_fence,

maybe we can just guarantee that fence=0 will never be used as a valid fence id, then this flag could be replaced with (fence != 0) check.

Quoted 2 lines

Yes, again seems reasonable. Initialize next_fence = 1, then increment by 2 each time and this works.

As we need fence only for low volume of blocking messages that requires response I'm not sure that we need to super optimize get_fence to use +2 all the time - maybe just check for zero wrap ?

Quoted 20 lines
Quoted 17 lines

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c static int ctb_write(struct intel_guc_ct_buffer · line 338

Quoted 11 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +0 -0
 		     bool want_response)
 {
	struct guc_ct_buffer_desc *desc = ctb->desc;
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +1 -1
@@ -328,18 +334,18 @@ static int ctb_write(struct intel_guc_ct_buffer  
 *ctb,
 	 * DW2+: action data
 	 */
 	header = (len << GUC_CT_MSG_LEN_SHIFT) |
-		 (GUC_CT_MSG_WRITE_FENCE_TO_DESC) |
+		 (enable_fence ? GUC_CT_MSG_WRITE_FENCE_TO_DESC : 0) |

Hmm, even if we ask fw to do not write back fence to the descriptor, IIRC current firmware will unconditionally write back return status of this non-blocking call, possibly overwriting status of the blocked call.

Yes, known problem with the interface that needs to be fixed.

Quoted 8 lines
Quoted 6 lines

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c static int ctb_write(struct intel_guc_ct_buffer *ctb, · line 338

Quoted 1 line
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +0 -0
 		 (want_response ? GUC_CT_MSG_SEND_STATUS : 0) |

btw, if we switch all requests to expect reply send back over CTB, then we can possibly drop the send_mutex in CTB paths, and block only when there is no DONT_WAIT flag and we have to wait for response.

Rather just wait for the GuC to fix this.

But fixing will probably require some extra flags, while switching to replies over CTB should work today on existing fw.

Quoted 45 lines
Quoted 35 lines
Quoted 35 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +20 -2
 		 (action[0] << GUC_CT_MSG_ACTION_SHIFT);
 	CT_DEBUG_DRIVER("CT: writing %*ph %*ph %*ph\n",
-			4, &header, 4, &fence,
+			4, &header, 4, &fence_value,
 			4 * (len - 1), &action[1]);
 	cmds[tail] = header;
 	tail = (tail + 1) % size;
-	cmds[tail] = fence;
+	cmds[tail] = fence_value;
 	tail = (tail + 1) % size;
 	for (i = 1; i < len; i++) {
@@ -440,6 +446,47 @@ static int wait_for_ct_request_update(struct  
 ct_request *req, u32 *status)
 	return err;
 }
+static inline bool ctb_has_room(struct guc_ct_buffer_desc *desc, u32  
 len)
+{
+	u32 head = READ_ONCE(desc->head);
+	u32 space;
+
+	space = CIRC_SPACE(desc->tail, head, desc->size);
+
+	return space >= len;
+}
+
+int intel_guc_send_nb(struct intel_guc_ct *ct,
+		      const u32 *action,
+		      u32 len)
+{
+	struct intel_guc_ct_channel *ctch = &ct->host_channel;
+	struct intel_guc_ct_buffer *ctb = &ctch->ctbs[CTB_SEND];
+	struct guc_ct_buffer_desc *desc = ctb->desc;
+	int err;
Quoted 9 lines

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c static int wait_for_ct_request_update(struct ct_request *req, u32 *status) · line 471

Quoted 5 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +5 -0
+
+	GEM_BUG_ON(!ctch->enabled);
+	GEM_BUG_ON(!len);
+	GEM_BUG_ON(len & ~GUC_CT_MSG_LEN_MASK);
+	lockdep_assert_held(&ct->send_lock);

hmm, does it mean that now it's caller responsibility to spinlock on CT private lock ? That is not how other guc_send() functions work.

Quoted 5 lines

Yes, that how I would like this work as I feel like it gives more flexability to the caller on the -EBUSY case. The caller can call intel_guc_send_nb again while still holding the lock or it release lock the and use a different form of flow control. Perhaps locking / unlocking should be exposed via static inlines rather than the caller directly manipulating the lock?

Hmm, I'm not sure that I like such flexibility that one set of callers is allowed to grab and hold CTB internal send_lock, while other caller don't even know that such lock exists, is a good direction.

Why do you need to sync your callers on CTB internal lock ? Are you afraid that other clients might steal next free slot in CTB ? Who are these other clients ? maybe you can lock on something else ?

Quoted 42 lines
Quoted 32 lines
Quoted 32 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +18 -0
+
+	if (unlikely(!ctb_has_room(desc, (len + 1) * 4))) {
+		ct->retry++;
+		if (ct->retry >= MAX_RETRY)
+			return -EDEADLK;
+		else
+			return -EBUSY;
+	}
+
+	ct->retry = 0;
+	err = ctb_write(ctb, action, len, 0, false, false);
+	if (unlikely(err))
+		return err;
+
+	intel_guc_notify(ct_to_guc(ct));
+	return 0;
+}
+
 static int ctch_send(struct intel_guc_ct *ct,
 		     struct intel_guc_ct_channel *ctch,

const u32 *action,

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +7 -0
@@ -460,17 +507,35 @@ static int ctch_send(struct intel_guc_ct *ct,
 	GEM_BUG_ON(len & ~GUC_CT_MSG_LEN_MASK);
 	GEM_BUG_ON(!response_buf && response_buf_size);
+	/*
+	 * We use a lazy spin wait loop here as we believe that if the CT
+	 * buffers are sized correctly the flow control condition should be
+	 * rare.
+	 */
+retry:
+	spin_lock_irqsave(&ct->send_lock, flags);
Quoted 9 lines

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c static int ctch_send(struct intel_guc_ct *ct, · line 521

Quoted 5 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +5 -0
+	if (unlikely(!ctb_has_room(desc, (len + 1) * 4))) {
+		spin_unlock_irqrestore(&ct->send_lock, flags);
+		ct->retry++;
+		if (ct->retry >= MAX_RETRY)
+			return -EDEADLK;

I'm not sure what's better: have secret deadlock hard to reproduce, or deadlocks easier to catch that helps improve to be deadlock-clean

Quoted 7 lines

This is covering the case where the has died and to avoid deadlock. Eventually we will have some GuC health check code that will trigger a full GPU reset if the GuC has died. We need a way for code spinning on the CTBs to exit.

I've already tweaked this code locally a bit to use an atomic too with the idea being the GuC health check code can set this value to have all code spinning on CTBs immediately return --EDEADLK when the GuC has died.

Will wait then to see next revision.

Quoted 94 lines
Quoted 94 lines
Quoted 94 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +9 -3
+		cpu_relax();
+		goto retry;
+	}
+
+	ct->retry = 0;
 	fence = ctch_get_next_fence(ctch);
 	request.fence = fence;
 	request.status = 0;
 	request.response_len = response_buf_size;
 	request.response_buf = response_buf;
-	spin_lock_irqsave(&ct->lock, flags);
+	spin_lock(&ct->request_lock);
 	list_add_tail(&request.link, &ct->pending_requests);
-	spin_unlock_irqrestore(&ct->lock, flags);
+	spin_unlock(&ct->request_lock);
-	err = ctb_write(ctb, action, len, fence, !!response_buf);
+	err = ctb_write(ctb, action, len, fence, true, !!response_buf);
+	spin_unlock_irqrestore(&ct->send_lock, flags);
 	if (unlikely(err))
 		goto unlink;
@@ -501,9 +566,9 @@ static int ctch_send(struct intel_guc_ct *ct,
 	}

unlink:

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +1 -1
-	spin_lock_irqsave(&ct->lock, flags);
+	spin_lock_irqsave(&ct->request_lock, flags);
	list_del(&request.link);
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +1 -1
-	spin_unlock_irqrestore(&ct->lock, flags);
+	spin_unlock_irqrestore(&ct->request_lock, flags);
	return err;
}
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +6 -6
@@ -653,7 +718,7 @@ static int ct_handle_response(struct intel_guc_ct  
 *ct, const u32 *msg)
 	CT_DEBUG_DRIVER("CT: response fence %u status %#x\n", fence, status);
-	spin_lock(&ct->lock);
+	spin_lock(&ct->request_lock);
 	list_for_each_entry(req, &ct->pending_requests, link) {
 		if (unlikely(fence != req->fence)) {
 			CT_DEBUG_DRIVER("CT: request %u awaits response\n",
@@ -672,7 +737,7 @@ static int ct_handle_response(struct intel_guc_ct  
 *ct, const u32 *msg)
 		found = true;
 		break;
 	}
-	spin_unlock(&ct->lock);
+	spin_unlock(&ct->request_lock);
 	if (!found)
 		DRM_ERROR("CT: unsolicited response %*ph\n", 4 * msglen, msg);
@@ -710,13 +775,13 @@ static bool ct_process_incoming_requests(struct  
 intel_guc_ct *ct)
 	u32 *payload;
 	bool done;
-	spin_lock_irqsave(&ct->lock, flags);
+	spin_lock_irqsave(&ct->request_lock, flags);
 	request = list_first_entry_or_null(&ct->incoming_requests,
 					   struct ct_incoming_request, link);
 	if (request)
 		list_del(&request->link);
 	done = !!list_empty(&ct->incoming_requests);
-	spin_unlock_irqrestore(&ct->lock, flags);
+	spin_unlock_irqrestore(&ct->request_lock, flags);
 	if (!request)
 		return true;
@@ -777,9 +842,9 @@ static int ct_handle_request(struct intel_guc_ct  
 *ct, const u32 *msg)
 	}
 	memcpy(request->msg, msg, 4 * msglen);
-	spin_lock_irqsave(&ct->lock, flags);
+	spin_lock_irqsave(&ct->request_lock, flags);
 	list_add_tail(&request->link, &ct->incoming_requests);
-	spin_unlock_irqrestore(&ct->lock, flags);
+	spin_unlock_irqrestore(&ct->request_lock, flags);
 	queue_work(system_unbound_wq, &ct->worker);
 	return 0;
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h +6 -2
index 7c24d83f5c24..bc670a796bd8 100644
@@ -62,8 +62,11 @@ struct intel_guc_ct {
 	struct intel_guc_ct_channel host_channel;
 	/* other channels are tbd */
-	/** @lock: protects pending requests list */
-	spinlock_t lock;
+	/** @request_lock: protects pending requests list */
+	spinlock_t request_lock;
+
+	/** @send_lock: protects h2g channel */
+	spinlock_t send_lock;
 	/** @pending_requests: list of requests waiting for response */
 	struct list_head pending_requests;
@@ -73,6 +76,9 @@ struct intel_guc_ct {
 	/** @worker: worker for handling incoming requests */
 	struct work_struct worker;
+
Quoted 8 lines
Quoted 8 lines

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h struct intel_guc_ct { · line 84

Quoted 4 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h +2 -0
+	/** @retry: the number of times a H2G CTB has been retried */
+	u32 retry;
 };
 void intel_guc_ct_init_early(struct intel_guc_ct *ct);

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#7 lore raw

PATCH 2/3 drm/i915/guc: Optimized CTB writes and reads

In reply to [email protected]

From: Matthew Brost <[email protected]>

CTB writes are now in the path of command submission and should be optimized for performance. Rather than reading CTB descriptor values (e.g. head, tail, size) which could result in accesses across the PCIe bus, store shadow local copies and only read/write the descriptor values when absolutely necessary.

---
 drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c | 79 +++++++++++------------
 drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h |  8 +++
 2 files changed, 45 insertions(+), 42 deletions(-)
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +37 -42
index e50d968b15d5..4d8a4c6afd71 100644
@@ -68,23 +68,29 @@ static inline const char *guc_ct_buffer_type_to_str(u32 type)
 	}
 }
 
-static void guc_ct_buffer_desc_init(struct guc_ct_buffer_desc *desc,
+static void guc_ct_buffer_desc_init(struct intel_guc_ct_buffer *ctb,
 				    u32 cmds_addr, u32 size, u32 owner)
 {
+	struct guc_ct_buffer_desc *desc = ctb->desc;
 	CT_DEBUG_DRIVER("CT: desc %p init addr=%#x size=%u owner=%u\n",
 			desc, cmds_addr, size, owner);
 	memset(desc, 0, sizeof(*desc));
 	desc->addr = cmds_addr;
-	desc->size = size;
+	ctb->size = desc->size = size;
 	desc->owner = owner;
+	ctb->tail = 0;
+	ctb->head = 0;
+	ctb->space = CIRC_SPACE(ctb->tail, ctb->head, ctb->size);
 }
 
-static void guc_ct_buffer_desc_reset(struct guc_ct_buffer_desc *desc)
+static void guc_ct_buffer_desc_reset(struct intel_guc_ct_buffer *ctb)
 {
+	struct guc_ct_buffer_desc *desc = ctb->desc;
 	CT_DEBUG_DRIVER("CT: desc %p reset head=%u tail=%u\n",
 			desc, desc->head, desc->tail);
-	desc->head = 0;
-	desc->tail = 0;
+	ctb->head = desc->head = 0;
+	ctb->tail = desc->tail = 0;
+	ctb->space = CIRC_SPACE(ctb->tail, ctb->head, ctb->size);
 	desc->is_in_error = 0;
 }
 
@@ -220,7 +226,7 @@ static int ctch_enable(struct intel_guc *guc,
 	 */
 	for (i = 0; i < ARRAY_SIZE(ctch->ctbs); i++) {
 		GEM_BUG_ON((i != CTB_SEND) && (i != CTB_RECV));
-		guc_ct_buffer_desc_init(ctch->ctbs[i].desc,
+		guc_ct_buffer_desc_init(&ctch->ctbs[i],
 					base + PAGE_SIZE/4 * i + PAGE_SIZE/2,
 					PAGE_SIZE/4,
 					ctch->owner);
@@ -301,32 +307,16 @@ static int ctb_write(struct intel_guc_ct_buffer *ctb,
 		     bool want_response)
 {
 	struct guc_ct_buffer_desc *desc = ctb->desc;
-	u32 head = desc->head / 4;	/* in dwords */
-	u32 tail = desc->tail / 4;	/* in dwords */
-	u32 size = desc->size / 4;	/* in dwords */
-	u32 used;			/* in dwords */
+	u32 tail = ctb->tail / 4;	/* in dwords */
+	u32 size = ctb->size / 4;	/* in dwords */
 	u32 header;
 	u32 *cmds = ctb->cmds;
 	unsigned int i;
 
-	GEM_BUG_ON(desc->size % 4);
-	GEM_BUG_ON(desc->head % 4);
-	GEM_BUG_ON(desc->tail % 4);
+	GEM_BUG_ON(ctb->size % 4);
+	GEM_BUG_ON(ctb->tail % 4);
 	GEM_BUG_ON(tail >= size);
 
-	/*
-	 * tail == head condition indicates empty. GuC FW does not support
-	 * using up the entire buffer to get tail == head meaning full.
-	 */
-	if (tail < head)
-		used = (size - head) + tail;
-	else
-		used = tail - head;
-
-	/* make sure there is a space including extra dw for the fence */
-	if (unlikely(used + len + 1 >= size))
-		return -ENOSPC;
-
 	/*
 	 * Write the message. The format is the following:
 	 * DW0: header (including action code)
@@ -354,15 +344,16 @@ static int ctb_write(struct intel_guc_ct_buffer *ctb,
 	}
 
 	/* now update desc tail (back in bytes) */
-	desc->tail = tail * 4;
-	GEM_BUG_ON(desc->tail > desc->size);
+	ctb->tail = desc->tail = tail * 4;
+	ctb->space -= (len + 1) * 4;
+	GEM_BUG_ON(ctb->tail > ctb->size);
 
 	return 0;
 }
 
 /**
  * wait_for_ctb_desc_update - Wait for the CT buffer descriptor update.
- * @desc:	buffer descriptor
+ * @ctb:	ctb buffer
  * @fence:	response fence
  * @status:	placeholder for status
  *
@@ -376,11 +367,12 @@ static int ctb_write(struct intel_guc_ct_buffer *ctb,
  * *	-ETIMEDOUT no response within hardcoded timeout
  * *	-EPROTO no response, CT buffer is in error
  */
-static int wait_for_ctb_desc_update(struct guc_ct_buffer_desc *desc,
+static int wait_for_ctb_desc_update(struct intel_guc_ct_buffer *ctb,
 				    u32 fence,
 				    u32 *status)
 {
 	int err;
+	struct guc_ct_buffer_desc *desc = ctb->desc;
 
 	/*
 	 * Fast commands should complete in less than 10us, so sample quickly
@@ -401,7 +393,7 @@ static int wait_for_ctb_desc_update(struct guc_ct_buffer_desc *desc,
 			/* Something went wrong with the messaging, try to reset
 			 * the buffer and hope for the best
 			 */
-			guc_ct_buffer_desc_reset(desc);
+			guc_ct_buffer_desc_reset(ctb);
 			err = -EPROTO;
 		}
 	}
@@ -446,12 +438,17 @@ static int wait_for_ct_request_update(struct ct_request *req, u32 *status)
 	return err;
 }
 
-static inline bool ctb_has_room(struct guc_ct_buffer_desc *desc, u32 len)
+static inline bool ctb_has_room(struct intel_guc_ct_buffer *ctb, u32 len)
 {
-	u32 head = READ_ONCE(desc->head);
+	u32 head;
 	u32 space;
 
-	space = CIRC_SPACE(desc->tail, head, desc->size);
+	if (ctb->space >= len)
+		return true;
+
+	head = READ_ONCE(ctb->desc->head);
+	space = CIRC_SPACE(ctb->tail, head, ctb->size);
+	ctb->space = space;
 
 	return space >= len;
 }
@@ -462,7 +459,6 @@ int intel_guc_send_nb(struct intel_guc_ct *ct,
 {
 	struct intel_guc_ct_channel *ctch = &ct->host_channel;
 	struct intel_guc_ct_buffer *ctb = &ctch->ctbs[CTB_SEND];
-	struct guc_ct_buffer_desc *desc = ctb->desc;
 	int err;
 
 	GEM_BUG_ON(!ctch->enabled);
@@ -470,7 +466,7 @@ int intel_guc_send_nb(struct intel_guc_ct *ct,
 	GEM_BUG_ON(len & ~GUC_CT_MSG_LEN_MASK);
 	lockdep_assert_held(&ct->send_lock);
 
-	if (unlikely(!ctb_has_room(desc, (len + 1) * 4))) {
+	if (unlikely(!ctb_has_room(ctb, (len + 1) * 4))) {
 		ct->retry++;
 		if (ct->retry >= MAX_RETRY)
 			return -EDEADLK;
@@ -496,7 +492,6 @@ static int ctch_send(struct intel_guc_ct *ct,
 		     u32 *status)
 {
 	struct intel_guc_ct_buffer *ctb = &ctch->ctbs[CTB_SEND];
-	struct guc_ct_buffer_desc *desc = ctb->desc;
 	struct ct_request request;
 	unsigned long flags;
 	u32 fence;
@@ -514,7 +509,7 @@ static int ctch_send(struct intel_guc_ct *ct,
 	 */
 retry:
 	spin_lock_irqsave(&ct->send_lock, flags);
-	if (unlikely(!ctb_has_room(desc, (len + 1) * 4))) {
+	if (unlikely(!ctb_has_room(ctb, (len + 1) * 4))) {
 		spin_unlock_irqrestore(&ct->send_lock, flags);
 		ct->retry++;
 		if (ct->retry >= MAX_RETRY)
@@ -544,7 +539,7 @@ static int ctch_send(struct intel_guc_ct *ct,
 	if (response_buf)
 		err = wait_for_ct_request_update(&request, status);
 	else
-		err = wait_for_ctb_desc_update(desc, fence, status);
+		err = wait_for_ctb_desc_update(ctb, fence, status);
 	if (unlikely(err))
 		goto unlink;
 
@@ -618,9 +613,9 @@ static inline bool ct_header_is_response(u32 header)
 static int ctb_read(struct intel_guc_ct_buffer *ctb, u32 *data)
 {
 	struct guc_ct_buffer_desc *desc = ctb->desc;
-	u32 head = desc->head / 4;	/* in dwords */
+	u32 head = ctb->head / 4;	/* in dwords */
 	u32 tail = desc->tail / 4;	/* in dwords */
-	u32 size = desc->size / 4;	/* in dwords */
+	u32 size = ctb->size / 4;	/* in dwords */
 	u32 *cmds = ctb->cmds;
 	s32 available;			/* in dwords */
 	unsigned int len;
@@ -664,7 +659,7 @@ static int ctb_read(struct intel_guc_ct_buffer *ctb, u32 *data)
 	}
 	CT_DEBUG_DRIVER("CT: received %*ph\n", 4 * len, data);
 
-	desc->head = head * 4;
+	ctb->head = desc->head = head * 4;
 	return 0;
 }
 
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h +8 -0
index bc670a796bd8..1bff4f0b91f7 100644
@@ -29,10 +29,18 @@ struct intel_guc;
  *
  * @desc: pointer to the buffer descriptor
  * @cmds: pointer to the commands buffer
+ * @size: local shadow copy of size
+ * @head: local shadow copy of head
+ * @tail: local shadow copy of tail
+ * @space: local shadow copy of space
  */
 struct intel_guc_ct_buffer {
 	struct guc_ct_buffer_desc *desc;
 	u32 *cmds;
+	u32 size;
+	u32 tail;
+	u32 head;
+	u32 space;
 };
 
 /** Represents pair of command transport buffers.
Signature · 6 lines
2.21.0.5.gaeb582a983

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#8 lore raw

Re: [PATCH 2/3] drm/i915/guc: Optimized CTB writes and reads

Quoted 20 lines · On Thu, 21 Nov 2019 00:56:03 +0100, <[email protected]> wrote:

From: Matthew Brost <[email protected]>

CTB writes are now in the path of command submission and should be optimized for performance. Rather than reading CTB descriptor values (e.g. head, tail, size) which could result in accesses across the PCIe bus, store shadow local copies and only read/write the descriptor values when absolutely necessary.

---
 drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c | 79 +++++++++++------------
 drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h |  8 +++
 2 files changed, 45 insertions(+), 42 deletions(-)
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +0 -0
index e50d968b15d5..4d8a4c6afd71 100644

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c static inline const char · line 72

Quoted 6 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +1 -1
@@ -68,23 +68,29 @@ static inline const char  
 *guc_ct_buffer_type_to_str(u32 type)
 	}
 }
-static void guc_ct_buffer_desc_init(struct guc_ct_buffer_desc *desc,
+static void guc_ct_buffer_desc_init(struct intel_guc_ct_buffer *ctb,
 				    u32 cmds_addr, u32 size, u32 owner)

as now this function takes ctb instead of desc, it should be renamed or make it separate from guc_ct_buffer_desc_init

Quoted 5 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +1 -0
 {
+	struct guc_ct_buffer_desc *desc = ctb->desc;
 	CT_DEBUG_DRIVER("CT: desc %p init addr=%#x size=%u owner=%u\n",
 			desc, cmds_addr, size, owner);
 	memset(desc, 0, sizeof(*desc));

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c static inline const char *guc_ct_buffer_type_to_str(u32 type) · line 86

Quoted 10 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +5 -2
 	desc->addr = cmds_addr;
-	desc->size = size;
+	ctb->size = desc->size = size;
 	desc->owner = owner;
+	ctb->tail = 0;
+	ctb->head = 0;
+	ctb->space = CIRC_SPACE(ctb->tail, ctb->head, ctb->size);
 }
-static void guc_ct_buffer_desc_reset(struct guc_ct_buffer_desc *desc)
+static void guc_ct_buffer_desc_reset(struct intel_guc_ct_buffer *ctb)

same here

Quoted 184 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +16 -28
 {
+	struct guc_ct_buffer_desc *desc = ctb->desc;
 	CT_DEBUG_DRIVER("CT: desc %p reset head=%u tail=%u\n",
 			desc, desc->head, desc->tail);
-	desc->head = 0;
-	desc->tail = 0;
+	ctb->head = desc->head = 0;
+	ctb->tail = desc->tail = 0;
+	ctb->space = CIRC_SPACE(ctb->tail, ctb->head, ctb->size);
 	desc->is_in_error = 0;
 }
@@ -220,7 +226,7 @@ static int ctch_enable(struct intel_guc *guc,
 	 */
 	for (i = 0; i < ARRAY_SIZE(ctch->ctbs); i++) {
 		GEM_BUG_ON((i != CTB_SEND) && (i != CTB_RECV));
-		guc_ct_buffer_desc_init(ctch->ctbs[i].desc,
+		guc_ct_buffer_desc_init(&ctch->ctbs[i],
 					base + PAGE_SIZE/4 * i + PAGE_SIZE/2,
 					PAGE_SIZE/4,
 					ctch->owner);
@@ -301,32 +307,16 @@ static int ctb_write(struct intel_guc_ct_buffer  
 *ctb,
 		     bool want_response)
 {
 	struct guc_ct_buffer_desc *desc = ctb->desc;
-	u32 head = desc->head / 4;	/* in dwords */
-	u32 tail = desc->tail / 4;	/* in dwords */
-	u32 size = desc->size / 4;	/* in dwords */
-	u32 used;			/* in dwords */
+	u32 tail = ctb->tail / 4;	/* in dwords */
+	u32 size = ctb->size / 4;	/* in dwords */
 	u32 header;
 	u32 *cmds = ctb->cmds;
 	unsigned int i;
-	GEM_BUG_ON(desc->size % 4);
-	GEM_BUG_ON(desc->head % 4);
-	GEM_BUG_ON(desc->tail % 4);
+	GEM_BUG_ON(ctb->size % 4);
+	GEM_BUG_ON(ctb->tail % 4);
 	GEM_BUG_ON(tail >= size);
-	/*
-	 * tail == head condition indicates empty. GuC FW does not support
-	 * using up the entire buffer to get tail == head meaning full.
-	 */
-	if (tail < head)
-		used = (size - head) + tail;
-	else
-		used = tail - head;
-
-	/* make sure there is a space including extra dw for the fence */
-	if (unlikely(used + len + 1 >= size))
-		return -ENOSPC;
-
 	/*
 	 * Write the message. The format is the following:
 	 * DW0: header (including action code)
@@ -354,15 +344,16 @@ static int ctb_write(struct intel_guc_ct_buffer  
 *ctb,
 	}
 	/* now update desc tail (back in bytes) */
-	desc->tail = tail * 4;
-	GEM_BUG_ON(desc->tail > desc->size);
+	ctb->tail = desc->tail = tail * 4;
+	ctb->space -= (len + 1) * 4;
+	GEM_BUG_ON(ctb->tail > ctb->size);
 	return 0;
 }
 /**
  * wait_for_ctb_desc_update - Wait for the CT buffer descriptor update.
- * @desc:	buffer descriptor
+ * @ctb:	ctb buffer
  * @fence:	response fence
  * @status:	placeholder for status
  *
@@ -376,11 +367,12 @@ static int ctb_write(struct intel_guc_ct_buffer  
 *ctb,
  * *	-ETIMEDOUT no response within hardcoded timeout
  * *	-EPROTO no response, CT buffer is in error
  */
-static int wait_for_ctb_desc_update(struct guc_ct_buffer_desc *desc,
+static int wait_for_ctb_desc_update(struct intel_guc_ct_buffer *ctb,
 				    u32 fence,
 				    u32 *status)
 {
 	int err;
+	struct guc_ct_buffer_desc *desc = ctb->desc;
 	/*
 	 * Fast commands should complete in less than 10us, so sample quickly
@@ -401,7 +393,7 @@ static int wait_for_ctb_desc_update(struct  
 guc_ct_buffer_desc *desc,
 			/* Something went wrong with the messaging, try to reset
 			 * the buffer and hope for the best
 			 */
-			guc_ct_buffer_desc_reset(desc);
+			guc_ct_buffer_desc_reset(ctb);
 			err = -EPROTO;
 		}
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +14 -11
 	}
@@ -446,12 +438,17 @@ static int wait_for_ct_request_update(struct  
 ct_request *req, u32 *status)
 	return err;
 }
-static inline bool ctb_has_room(struct guc_ct_buffer_desc *desc, u32  
 len)
+static inline bool ctb_has_room(struct intel_guc_ct_buffer *ctb, u32  
 len)
 {
-	u32 head = READ_ONCE(desc->head);
+	u32 head;
 	u32 space;
-	space = CIRC_SPACE(desc->tail, head, desc->size);
+	if (ctb->space >= len)
+		return true;
+
+	head = READ_ONCE(ctb->desc->head);
+	space = CIRC_SPACE(ctb->tail, head, ctb->size);
+	ctb->space = space;
 	return space >= len;
 }
@@ -462,7 +459,6 @@ int intel_guc_send_nb(struct intel_guc_ct *ct,
 {
 	struct intel_guc_ct_channel *ctch = &ct->host_channel;
 	struct intel_guc_ct_buffer *ctb = &ctch->ctbs[CTB_SEND];
-	struct guc_ct_buffer_desc *desc = ctb->desc;
 	int err;
 	GEM_BUG_ON(!ctch->enabled);
@@ -470,7 +466,7 @@ int intel_guc_send_nb(struct intel_guc_ct *ct,
 	GEM_BUG_ON(len & ~GUC_CT_MSG_LEN_MASK);
 	lockdep_assert_held(&ct->send_lock);
-	if (unlikely(!ctb_has_room(desc, (len + 1) * 4))) {
+	if (unlikely(!ctb_has_room(ctb, (len + 1) * 4))) {
 		ct->retry++;
 		if (ct->retry >= MAX_RETRY)
 			return -EDEADLK;
@@ -496,7 +492,6 @@ static int ctch_send(struct intel_guc_ct *ct,
 		     u32 *status)
 {
 	struct intel_guc_ct_buffer *ctb = &ctch->ctbs[CTB_SEND];
-	struct guc_ct_buffer_desc *desc = ctb->desc;
 	struct ct_request request;
 	unsigned long flags;
 	u32 fence;
@@ -514,7 +509,7 @@ static int ctch_send(struct intel_guc_ct *ct,
 	 */
 retry:
 	spin_lock_irqsave(&ct->send_lock, flags);
-	if (unlikely(!ctb_has_room(desc, (len + 1) * 4))) {
+	if (unlikely(!ctb_has_room(ctb, (len + 1) * 4))) {
 		spin_unlock_irqrestore(&ct->send_lock, flags);
 		ct->retry++;
 		if (ct->retry >= MAX_RETRY)
@@ -544,7 +539,7 @@ static int ctch_send(struct intel_guc_ct *ct,
 	if (response_buf)
 		err = wait_for_ct_request_update(&request, status);
 	else
-		err = wait_for_ctb_desc_update(desc, fence, status);
+		err = wait_for_ctb_desc_update(ctb, fence, status);
 	if (unlikely(err))
 		goto unlink;
@@ -618,9 +613,9 @@ static inline bool ct_header_is_response(u32 header)
 static int ctb_read(struct intel_guc_ct_buffer *ctb, u32 *data)
 {
 	struct guc_ct_buffer_desc *desc = ctb->desc;
-	u32 head = desc->head / 4;	/* in dwords */
+	u32 head = ctb->head / 4;	/* in dwords */
 	u32 tail = desc->tail / 4;	/* in dwords */
-	u32 size = desc->size / 4;	/* in dwords */
+	u32 size = ctb->size / 4;	/* in dwords */
 	u32 *cmds = ctb->cmds;
 	s32 available;			/* in dwords */
 	unsigned int len;
@@ -664,7 +659,7 @@ static int ctb_read(struct intel_guc_ct_buffer *ctb,  
 u32 *data)
 	}
 	CT_DEBUG_DRIVER("CT: received %*ph\n", 4 * len, data);
-	desc->head = head * 4;
+	ctb->head = desc->head = head * 4;
 	return 0;
 }
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h +0 -0
index bc670a796bd8..1bff4f0b91f7 100644

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h struct intel_guc; · line 32

Quoted 4 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h +1 -0
@@ -29,10 +29,18 @@ struct intel_guc;
  *
  * @desc: pointer to the buffer descriptor
  * @cmds: pointer to the commands buffer
+ * @size: local shadow copy of size

I would rather expect this as real fixed size,
note that size is not expected to change

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h struct intel_guc; · line 46

Quoted 13 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h +7 -0
+ * @head: local shadow copy of head
+ * @tail: local shadow copy of tail
+ * @space: local shadow copy of space
  */
 struct intel_guc_ct_buffer {
 	struct guc_ct_buffer_desc *desc;
 	u32 *cmds;
+	u32 size;
+	u32 tail;
+	u32 head;
+	u32 space;
 };
 /** Represents pair of command transport buffers.

Can we reorder this patch to be first in the series ?

Michal
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#9 lore raw

Re: [PATCH 2/3] drm/i915/guc: Optimized CTB writes and reads

Quoted 17 lines · Michal Wajdeczko
Quoted 15 lines · On Thu, 21 Nov 2019 00:56:03 +0100, <[email protected]> wrote:

From: Matthew Brost <[email protected]>

CTB writes are now in the path of command submission and should be optimized for performance. Rather than reading CTB descriptor values (e.g. head, tail, size) which could result in accesses across the PCIe bus, store shadow local copies and only read/write the descriptor values when absolutely necessary.

---
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c | 79 +++++++++++------------ drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h | 8 +++

2 files changed, 45 insertions(+), 42 deletions(-)
Quoted 15 lines

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c static inline const char · line 72

Quoted 11 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +1 -1
index e50d968b15d5..4d8a4c6afd71 100644
@@ -68,23 +68,29 @@ static inline const char 
 *guc_ct_buffer_type_to_str(u32 type)
 	}
 }
-static void guc_ct_buffer_desc_init(struct guc_ct_buffer_desc *desc,
+static void guc_ct_buffer_desc_init(struct intel_guc_ct_buffer *ctb,
 				    u32 cmds_addr, u32 size, u32 owner)

as now this function takes ctb instead of desc, it should be renamed or make it separate from guc_ct_buffer_desc_init

Yes, makes sense.

Quoted 9 lines
Quoted 9 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +2 -1
 {
+	struct guc_ct_buffer_desc *desc = ctb->desc;
 	CT_DEBUG_DRIVER("CT: desc %p init addr=%#x size=%u owner=%u\n",
 			desc, cmds_addr, size, owner);
 	memset(desc, 0, sizeof(*desc));
 	desc->addr = cmds_addr;
-	desc->size = size;
+	ctb->size = desc->size = size;
 	desc->owner = owner;
Quoted 9 lines

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c static inline const char *guc_ct_buffer_type_to_str(u32 type) · line 86

Quoted 6 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +4 -1
+	ctb->tail = 0;
+	ctb->head = 0;
+	ctb->space = CIRC_SPACE(ctb->tail, ctb->head, ctb->size);
 }
-static void guc_ct_buffer_desc_reset(struct guc_ct_buffer_desc *desc)
+static void guc_ct_buffer_desc_reset(struct intel_guc_ct_buffer *ctb)

same here

Same.

Quoted 183 lines
Quoted 183 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +16 -28
 {
+	struct guc_ct_buffer_desc *desc = ctb->desc;
 	CT_DEBUG_DRIVER("CT: desc %p reset head=%u tail=%u\n",
 			desc, desc->head, desc->tail);
-	desc->head = 0;
-	desc->tail = 0;
+	ctb->head = desc->head = 0;
+	ctb->tail = desc->tail = 0;
+	ctb->space = CIRC_SPACE(ctb->tail, ctb->head, ctb->size);
 	desc->is_in_error = 0;
 }
@@ -220,7 +226,7 @@ static int ctch_enable(struct intel_guc *guc,
 	 */
 	for (i = 0; i < ARRAY_SIZE(ctch->ctbs); i++) {
 		GEM_BUG_ON((i != CTB_SEND) && (i != CTB_RECV));
-		guc_ct_buffer_desc_init(ctch->ctbs[i].desc,
+		guc_ct_buffer_desc_init(&ctch->ctbs[i],
 					base + PAGE_SIZE/4 * i + PAGE_SIZE/2,
 					PAGE_SIZE/4,
 					ctch->owner);
@@ -301,32 +307,16 @@ static int ctb_write(struct 
 intel_guc_ct_buffer *ctb,
 		     bool want_response)
 {
 	struct guc_ct_buffer_desc *desc = ctb->desc;
-	u32 head = desc->head / 4;	/* in dwords */
-	u32 tail = desc->tail / 4;	/* in dwords */
-	u32 size = desc->size / 4;	/* in dwords */
-	u32 used;			/* in dwords */
+	u32 tail = ctb->tail / 4;	/* in dwords */
+	u32 size = ctb->size / 4;	/* in dwords */
 	u32 header;
 	u32 *cmds = ctb->cmds;
 	unsigned int i;
-	GEM_BUG_ON(desc->size % 4);
-	GEM_BUG_ON(desc->head % 4);
-	GEM_BUG_ON(desc->tail % 4);
+	GEM_BUG_ON(ctb->size % 4);
+	GEM_BUG_ON(ctb->tail % 4);
 	GEM_BUG_ON(tail >= size);
-	/*
-	 * tail == head condition indicates empty. GuC FW does not support
-	 * using up the entire buffer to get tail == head meaning full.
-	 */
-	if (tail < head)
-		used = (size - head) + tail;
-	else
-		used = tail - head;
-
-	/* make sure there is a space including extra dw for the fence */
-	if (unlikely(used + len + 1 >= size))
-		return -ENOSPC;
-
 	/*
 	 * Write the message. The format is the following:
 	 * DW0: header (including action code)
@@ -354,15 +344,16 @@ static int ctb_write(struct 
 intel_guc_ct_buffer *ctb,
 	}
 	/* now update desc tail (back in bytes) */
-	desc->tail = tail * 4;
-	GEM_BUG_ON(desc->tail > desc->size);
+	ctb->tail = desc->tail = tail * 4;
+	ctb->space -= (len + 1) * 4;
+	GEM_BUG_ON(ctb->tail > ctb->size);
 	return 0;
 }
 /**
 * wait_for_ctb_desc_update - Wait for the CT buffer descriptor update.
- * @desc:	buffer descriptor
+ * @ctb:	ctb buffer
 * @fence:	response fence
 * @status:	placeholder for status
 *
@@ -376,11 +367,12 @@ static int ctb_write(struct 
 intel_guc_ct_buffer *ctb,
 * *	-ETIMEDOUT no response within hardcoded timeout
 * *	-EPROTO no response, CT buffer is in error
 */
-static int wait_for_ctb_desc_update(struct guc_ct_buffer_desc *desc,
+static int wait_for_ctb_desc_update(struct intel_guc_ct_buffer *ctb,
 				    u32 fence,
 				    u32 *status)
 {
 	int err;
+	struct guc_ct_buffer_desc *desc = ctb->desc;
 	/*
 	 * Fast commands should complete in less than 10us, so sample quickly
@@ -401,7 +393,7 @@ static int wait_for_ctb_desc_update(struct 
 guc_ct_buffer_desc *desc,
 			/* Something went wrong with the messaging, try to reset
 			 * the buffer and hope for the best
 			 */
-			guc_ct_buffer_desc_reset(desc);
+			guc_ct_buffer_desc_reset(ctb);
 			err = -EPROTO;
 		}
	}
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +9 -6
@@ -446,12 +438,17 @@ static int wait_for_ct_request_update(struct 
 ct_request *req, u32 *status)
 	return err;
 }
-static inline bool ctb_has_room(struct guc_ct_buffer_desc *desc, 
 u32 len)
+static inline bool ctb_has_room(struct intel_guc_ct_buffer *ctb, 
 u32 len)
 {
-	u32 head = READ_ONCE(desc->head);
+	u32 head;
 	u32 space;
-	space = CIRC_SPACE(desc->tail, head, desc->size);
+	if (ctb->space >= len)
+		return true;
+
+	head = READ_ONCE(ctb->desc->head);
+	space = CIRC_SPACE(ctb->tail, head, ctb->size);
+	ctb->space = space;
 	return space >= len;
 }
@@ -462,7 +459,6 @@ int intel_guc_send_nb(struct intel_guc_ct *ct,
 {
 	struct intel_guc_ct_channel *ctch = &ct->host_channel;
 	struct intel_guc_ct_buffer *ctb = &ctch->ctbs[CTB_SEND];
-	struct guc_ct_buffer_desc *desc = ctb->desc;
 	int err;
 	GEM_BUG_ON(!ctch->enabled);
@@ -470,7 +466,7 @@ int intel_guc_send_nb(struct intel_guc_ct *ct,
 	GEM_BUG_ON(len & ~GUC_CT_MSG_LEN_MASK);
 	lockdep_assert_held(&ct->send_lock);
-	if (unlikely(!ctb_has_room(desc, (len + 1) * 4))) {
+	if (unlikely(!ctb_has_room(ctb, (len + 1) * 4))) {
 		ct->retry++;
 		if (ct->retry >= MAX_RETRY)
 			return -EDEADLK;
@@ -496,7 +492,6 @@ static int ctch_send(struct intel_guc_ct *ct,
 		     u32 *status)
 {
 	struct intel_guc_ct_buffer *ctb = &ctch->ctbs[CTB_SEND];
-	struct guc_ct_buffer_desc *desc = ctb->desc;
 	struct ct_request request;
 	unsigned long flags;
 	u32 fence;
@@ -514,7 +509,7 @@ static int ctch_send(struct intel_guc_ct *ct,
 	 */

retry:

	spin_lock_irqsave(&ct->send_lock, flags);
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +1 -1
-	if (unlikely(!ctb_has_room(desc, (len + 1) * 4))) {
+	if (unlikely(!ctb_has_room(ctb, (len + 1) * 4))) {
		spin_unlock_irqrestore(&ct->send_lock, flags);
		ct->retry++;
		if (ct->retry >= MAX_RETRY)
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +4 -4
@@ -544,7 +539,7 @@ static int ctch_send(struct intel_guc_ct *ct,
 	if (response_buf)
 		err = wait_for_ct_request_update(&request, status);
 	else
-		err = wait_for_ctb_desc_update(desc, fence, status);
+		err = wait_for_ctb_desc_update(ctb, fence, status);
 	if (unlikely(err))
 		goto unlink;
@@ -618,9 +613,9 @@ static inline bool ct_header_is_response(u32 header)
 static int ctb_read(struct intel_guc_ct_buffer *ctb, u32 *data)
 {
 	struct guc_ct_buffer_desc *desc = ctb->desc;
-	u32 head = desc->head / 4;	/* in dwords */
+	u32 head = ctb->head / 4;	/* in dwords */
 	u32 tail = desc->tail / 4;	/* in dwords */
-	u32 size = desc->size / 4;	/* in dwords */
+	u32 size = ctb->size / 4;	/* in dwords */
 	u32 *cmds = ctb->cmds;
 	s32 available;			/* in dwords */
 	unsigned int len;
@@ -664,7 +659,7 @@ static int ctb_read(struct intel_guc_ct_buffer 
 *ctb, u32 *data)
 	}
 	CT_DEBUG_DRIVER("CT: received %*ph\n", 4 * len, data);
-	desc->head = head * 4;
+	ctb->head = desc->head = head * 4;
 	return 0;
 }
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h +0 -0
index bc670a796bd8..1bff4f0b91f7 100644
Quoted 9 lines

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h struct intel_guc; · line 32

Quoted 5 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h +1 -0
@@ -29,10 +29,18 @@ struct intel_guc;
 *
 * @desc: pointer to the buffer descriptor
 * @cmds: pointer to the commands buffer
+ * @size: local shadow copy of size

I would rather expect this as real fixed size,
note that size is not expected to change

Yes, it is fixed over the life of the CTB but not all CTBs need to be the same size. e.g. The H2G & G2H may and likely will be different sizes with the new Guc interface.

Quoted 16 lines

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h struct intel_guc; · line 46

Quoted 13 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h +7 -0
+ * @head: local shadow copy of head
+ * @tail: local shadow copy of tail
+ * @space: local shadow copy of space
 */
 struct intel_guc_ct_buffer {
 	struct guc_ct_buffer_desc *desc;
 	u32 *cmds;
+	u32 size;
+	u32 tail;
+	u32 head;
+	u32 space;
 };
 /** Represents pair of command transport buffers.

Can we reorder this patch to be first in the series ?

Quoted 2 lines

Michal
_______________________________________________

Yes.

Quoted 3 lines

Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#10 lore raw

Re: [PATCH 2/3] drm/i915/guc: Optimized CTB writes and reads

Quoted 250 lines · Matthew Brost
Quoted 17 lines · Michal Wajdeczko
Quoted 15 lines · On Thu, 21 Nov 2019 00:56:03 +0100, <[email protected]> wrote:

From: Matthew Brost <[email protected]>

CTB writes are now in the path of command submission and should be optimized for performance. Rather than reading CTB descriptor values (e.g. head, tail, size) which could result in accesses across the PCIe bus, store shadow local copies and only read/write the descriptor values when absolutely necessary.

---
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c | 79 +++++++++++------------ drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h | 8 +++

2 files changed, 45 insertions(+), 42 deletions(-)
Quoted 15 lines

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c static inline const char · line 72

Quoted 11 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +1 -1
index e50d968b15d5..4d8a4c6afd71 100644
@@ -68,23 +68,29 @@ static inline const char 
 *guc_ct_buffer_type_to_str(u32 type)
 	}
 }
-static void guc_ct_buffer_desc_init(struct guc_ct_buffer_desc *desc,
+static void guc_ct_buffer_desc_init(struct intel_guc_ct_buffer *ctb,
 				    u32 cmds_addr, u32 size, u32 owner)

as now this function takes ctb instead of desc, it should be renamed or make it separate from guc_ct_buffer_desc_init

Yes, makes sense.

Quoted 9 lines
Quoted 9 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +2 -1
 {
+	struct guc_ct_buffer_desc *desc = ctb->desc;
 	CT_DEBUG_DRIVER("CT: desc %p init addr=%#x size=%u owner=%u\n",
 			desc, cmds_addr, size, owner);
 	memset(desc, 0, sizeof(*desc));
 	desc->addr = cmds_addr;
-	desc->size = size;
+	ctb->size = desc->size = size;
 	desc->owner = owner;
Quoted 9 lines

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c static inline const char *guc_ct_buffer_type_to_str(u32 type) · line 86

Quoted 6 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +4 -1
+	ctb->tail = 0;
+	ctb->head = 0;
+	ctb->space = CIRC_SPACE(ctb->tail, ctb->head, ctb->size);
 }
-static void guc_ct_buffer_desc_reset(struct guc_ct_buffer_desc *desc)
+static void guc_ct_buffer_desc_reset(struct intel_guc_ct_buffer *ctb)

same here

Same.

Quoted 183 lines
Quoted 183 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +16 -28
 {
+	struct guc_ct_buffer_desc *desc = ctb->desc;
 	CT_DEBUG_DRIVER("CT: desc %p reset head=%u tail=%u\n",
 			desc, desc->head, desc->tail);
-	desc->head = 0;
-	desc->tail = 0;
+	ctb->head = desc->head = 0;
+	ctb->tail = desc->tail = 0;
+	ctb->space = CIRC_SPACE(ctb->tail, ctb->head, ctb->size);
 	desc->is_in_error = 0;
 }
@@ -220,7 +226,7 @@ static int ctch_enable(struct intel_guc *guc,
 	 */
 	for (i = 0; i < ARRAY_SIZE(ctch->ctbs); i++) {
 		GEM_BUG_ON((i != CTB_SEND) && (i != CTB_RECV));
-		guc_ct_buffer_desc_init(ctch->ctbs[i].desc,
+		guc_ct_buffer_desc_init(&ctch->ctbs[i],
 					base + PAGE_SIZE/4 * i + PAGE_SIZE/2,
 					PAGE_SIZE/4,
 					ctch->owner);
@@ -301,32 +307,16 @@ static int ctb_write(struct 
 intel_guc_ct_buffer *ctb,
 		     bool want_response)
 {
 	struct guc_ct_buffer_desc *desc = ctb->desc;
-	u32 head = desc->head / 4;	/* in dwords */
-	u32 tail = desc->tail / 4;	/* in dwords */
-	u32 size = desc->size / 4;	/* in dwords */
-	u32 used;			/* in dwords */
+	u32 tail = ctb->tail / 4;	/* in dwords */
+	u32 size = ctb->size / 4;	/* in dwords */
 	u32 header;
 	u32 *cmds = ctb->cmds;
 	unsigned int i;
-	GEM_BUG_ON(desc->size % 4);
-	GEM_BUG_ON(desc->head % 4);
-	GEM_BUG_ON(desc->tail % 4);
+	GEM_BUG_ON(ctb->size % 4);
+	GEM_BUG_ON(ctb->tail % 4);
 	GEM_BUG_ON(tail >= size);
-	/*
-	 * tail == head condition indicates empty. GuC FW does not support
-	 * using up the entire buffer to get tail == head meaning full.
-	 */
-	if (tail < head)
-		used = (size - head) + tail;
-	else
-		used = tail - head;
-
-	/* make sure there is a space including extra dw for the fence */
-	if (unlikely(used + len + 1 >= size))
-		return -ENOSPC;
-
 	/*
 	 * Write the message. The format is the following:
 	 * DW0: header (including action code)
@@ -354,15 +344,16 @@ static int ctb_write(struct 
 intel_guc_ct_buffer *ctb,
 	}
 	/* now update desc tail (back in bytes) */
-	desc->tail = tail * 4;
-	GEM_BUG_ON(desc->tail > desc->size);
+	ctb->tail = desc->tail = tail * 4;
+	ctb->space -= (len + 1) * 4;
+	GEM_BUG_ON(ctb->tail > ctb->size);
 	return 0;
 }
 /**
 * wait_for_ctb_desc_update - Wait for the CT buffer descriptor update.
- * @desc:	buffer descriptor
+ * @ctb:	ctb buffer
 * @fence:	response fence
 * @status:	placeholder for status
 *
@@ -376,11 +367,12 @@ static int ctb_write(struct 
 intel_guc_ct_buffer *ctb,
 * *	-ETIMEDOUT no response within hardcoded timeout
 * *	-EPROTO no response, CT buffer is in error
 */
-static int wait_for_ctb_desc_update(struct guc_ct_buffer_desc *desc,
+static int wait_for_ctb_desc_update(struct intel_guc_ct_buffer *ctb,
 				    u32 fence,
 				    u32 *status)
 {
 	int err;
+	struct guc_ct_buffer_desc *desc = ctb->desc;
 	/*
 	 * Fast commands should complete in less than 10us, so sample quickly
@@ -401,7 +393,7 @@ static int wait_for_ctb_desc_update(struct 
 guc_ct_buffer_desc *desc,
 			/* Something went wrong with the messaging, try to reset
 			 * the buffer and hope for the best
 			 */
-			guc_ct_buffer_desc_reset(desc);
+			guc_ct_buffer_desc_reset(ctb);
 			err = -EPROTO;
 		}
	}
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +9 -6
@@ -446,12 +438,17 @@ static int wait_for_ct_request_update(struct 
 ct_request *req, u32 *status)
 	return err;
 }
-static inline bool ctb_has_room(struct guc_ct_buffer_desc *desc, 
 u32 len)
+static inline bool ctb_has_room(struct intel_guc_ct_buffer *ctb, 
 u32 len)
 {
-	u32 head = READ_ONCE(desc->head);
+	u32 head;
 	u32 space;
-	space = CIRC_SPACE(desc->tail, head, desc->size);
+	if (ctb->space >= len)
+		return true;
+
+	head = READ_ONCE(ctb->desc->head);
+	space = CIRC_SPACE(ctb->tail, head, ctb->size);
+	ctb->space = space;
 	return space >= len;
 }
@@ -462,7 +459,6 @@ int intel_guc_send_nb(struct intel_guc_ct *ct,
 {
 	struct intel_guc_ct_channel *ctch = &ct->host_channel;
 	struct intel_guc_ct_buffer *ctb = &ctch->ctbs[CTB_SEND];
-	struct guc_ct_buffer_desc *desc = ctb->desc;
 	int err;
 	GEM_BUG_ON(!ctch->enabled);
@@ -470,7 +466,7 @@ int intel_guc_send_nb(struct intel_guc_ct *ct,
 	GEM_BUG_ON(len & ~GUC_CT_MSG_LEN_MASK);
 	lockdep_assert_held(&ct->send_lock);
-	if (unlikely(!ctb_has_room(desc, (len + 1) * 4))) {
+	if (unlikely(!ctb_has_room(ctb, (len + 1) * 4))) {
 		ct->retry++;
 		if (ct->retry >= MAX_RETRY)
 			return -EDEADLK;
@@ -496,7 +492,6 @@ static int ctch_send(struct intel_guc_ct *ct,
 		     u32 *status)
 {
 	struct intel_guc_ct_buffer *ctb = &ctch->ctbs[CTB_SEND];
-	struct guc_ct_buffer_desc *desc = ctb->desc;
 	struct ct_request request;
 	unsigned long flags;
 	u32 fence;
@@ -514,7 +509,7 @@ static int ctch_send(struct intel_guc_ct *ct,
 	 */

retry:

	spin_lock_irqsave(&ct->send_lock, flags);
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +1 -1
-	if (unlikely(!ctb_has_room(desc, (len + 1) * 4))) {
+	if (unlikely(!ctb_has_room(ctb, (len + 1) * 4))) {
		spin_unlock_irqrestore(&ct->send_lock, flags);
		ct->retry++;
		if (ct->retry >= MAX_RETRY)
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +4 -4
@@ -544,7 +539,7 @@ static int ctch_send(struct intel_guc_ct *ct,
 	if (response_buf)
 		err = wait_for_ct_request_update(&request, status);
 	else
-		err = wait_for_ctb_desc_update(desc, fence, status);
+		err = wait_for_ctb_desc_update(ctb, fence, status);
 	if (unlikely(err))
 		goto unlink;
@@ -618,9 +613,9 @@ static inline bool ct_header_is_response(u32 header)
 static int ctb_read(struct intel_guc_ct_buffer *ctb, u32 *data)
 {
 	struct guc_ct_buffer_desc *desc = ctb->desc;
-	u32 head = desc->head / 4;	/* in dwords */
+	u32 head = ctb->head / 4;	/* in dwords */
 	u32 tail = desc->tail / 4;	/* in dwords */
-	u32 size = desc->size / 4;	/* in dwords */
+	u32 size = ctb->size / 4;	/* in dwords */
 	u32 *cmds = ctb->cmds;
 	s32 available;			/* in dwords */
 	unsigned int len;
@@ -664,7 +659,7 @@ static int ctb_read(struct intel_guc_ct_buffer 
 *ctb, u32 *data)
 	}
 	CT_DEBUG_DRIVER("CT: received %*ph\n", 4 * len, data);
-	desc->head = head * 4;
+	ctb->head = desc->head = head * 4;
 	return 0;
 }
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h +0 -0
index bc670a796bd8..1bff4f0b91f7 100644
Quoted 9 lines

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h struct intel_guc; · line 32

Quoted 5 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h +1 -0
@@ -29,10 +29,18 @@ struct intel_guc;
 *
 * @desc: pointer to the buffer descriptor
 * @cmds: pointer to the commands buffer
+ * @size: local shadow copy of size

I would rather expect this as real fixed size,
note that size is not expected to change

Quoted 21 lines

Yes, it is fixed over the life of the CTB but not all CTBs need to be the same size. e.g. The H2G & G2H may and likely will be different sizes with the new Guc interface.

Quoted 17 lines

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h struct intel_guc; · line 46

Quoted 13 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h +7 -0
+ * @head: local shadow copy of head
+ * @tail: local shadow copy of tail
+ * @space: local shadow copy of space
 */
 struct intel_guc_ct_buffer {
 	struct guc_ct_buffer_desc *desc;
 	u32 *cmds;
+	u32 size;
+	u32 tail;
+	u32 head;
+	u32 space;
 };
 /** Represents pair of command transport buffers.

Can we reorder this patch to be first in the series ?

Michal

Tried this and I think it makes more sense the way it is. The 'space' value doesn't have a meaning before the non blocking call is introduced. Also it ends up changing a bunch of code that is then deleted in the non blocking call patch. Better to leave it as is.

Matt

Quoted 11 lines
Quoted 1 line

_______________________________________________

Yes.

Quoted 3 lines

Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#11 lore raw

Re: [PATCH 2/3] drm/i915/guc: Optimized CTB writes and reads

Quoted 272 lines · Matthew Brost
Quoted 271 lines · Matthew Brost
Quoted 17 lines · Michal Wajdeczko
Quoted 15 lines · On Thu, 21 Nov 2019 00:56:03 +0100, <[email protected]> wrote:

From: Matthew Brost <[email protected]>

CTB writes are now in the path of command submission and should be optimized for performance. Rather than reading CTB descriptor values (e.g. head, tail, size) which could result in accesses across the PCIe bus, store shadow local copies and only read/write the descriptor values when absolutely necessary.

---
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c | 79 +++++++++++------------ drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h | 8 +++

2 files changed, 45 insertions(+), 42 deletions(-)
Quoted 15 lines

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c static inline const char · line 72

Quoted 11 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +1 -1
index e50d968b15d5..4d8a4c6afd71 100644
@@ -68,23 +68,29 @@ static inline const char  
 *guc_ct_buffer_type_to_str(u32 type)
 	}
 }
-static void guc_ct_buffer_desc_init(struct guc_ct_buffer_desc *desc,
+static void guc_ct_buffer_desc_init(struct intel_guc_ct_buffer *ctb,
 				    u32 cmds_addr, u32 size, u32 owner)

as now this function takes ctb instead of desc, it should be renamed or make it separate from guc_ct_buffer_desc_init

Yes, makes sense.

Quoted 9 lines
Quoted 9 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +2 -1
 {
+	struct guc_ct_buffer_desc *desc = ctb->desc;
 	CT_DEBUG_DRIVER("CT: desc %p init addr=%#x size=%u owner=%u\n",
 			desc, cmds_addr, size, owner);
 	memset(desc, 0, sizeof(*desc));
 	desc->addr = cmds_addr;
-	desc->size = size;
+	ctb->size = desc->size = size;
 	desc->owner = owner;
Quoted 9 lines

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c static inline const char *guc_ct_buffer_type_to_str(u32 type) · line 86

Quoted 6 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +4 -1
+	ctb->tail = 0;
+	ctb->head = 0;
+	ctb->space = CIRC_SPACE(ctb->tail, ctb->head, ctb->size);
 }
-static void guc_ct_buffer_desc_reset(struct guc_ct_buffer_desc *desc)
+static void guc_ct_buffer_desc_reset(struct intel_guc_ct_buffer *ctb)

same here

Same.

Quoted 184 lines
Quoted 184 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +16 -28
 {
+	struct guc_ct_buffer_desc *desc = ctb->desc;
 	CT_DEBUG_DRIVER("CT: desc %p reset head=%u tail=%u\n",
 			desc, desc->head, desc->tail);
-	desc->head = 0;
-	desc->tail = 0;
+	ctb->head = desc->head = 0;
+	ctb->tail = desc->tail = 0;
+	ctb->space = CIRC_SPACE(ctb->tail, ctb->head, ctb->size);
 	desc->is_in_error = 0;
 }
@@ -220,7 +226,7 @@ static int ctch_enable(struct intel_guc *guc,
 	 */
 	for (i = 0; i < ARRAY_SIZE(ctch->ctbs); i++) {
 		GEM_BUG_ON((i != CTB_SEND) && (i != CTB_RECV));
-		guc_ct_buffer_desc_init(ctch->ctbs[i].desc,
+		guc_ct_buffer_desc_init(&ctch->ctbs[i],
 					base + PAGE_SIZE/4 * i + PAGE_SIZE/2,
 					PAGE_SIZE/4,
 					ctch->owner);
@@ -301,32 +307,16 @@ static int ctb_write(struct intel_guc_ct_buffer  
 *ctb,
 		     bool want_response)
 {
 	struct guc_ct_buffer_desc *desc = ctb->desc;
-	u32 head = desc->head / 4;	/* in dwords */
-	u32 tail = desc->tail / 4;	/* in dwords */
-	u32 size = desc->size / 4;	/* in dwords */
-	u32 used;			/* in dwords */
+	u32 tail = ctb->tail / 4;	/* in dwords */
+	u32 size = ctb->size / 4;	/* in dwords */
 	u32 header;
 	u32 *cmds = ctb->cmds;
 	unsigned int i;
-	GEM_BUG_ON(desc->size % 4);
-	GEM_BUG_ON(desc->head % 4);
-	GEM_BUG_ON(desc->tail % 4);
+	GEM_BUG_ON(ctb->size % 4);
+	GEM_BUG_ON(ctb->tail % 4);
 	GEM_BUG_ON(tail >= size);
-	/*
-	 * tail == head condition indicates empty. GuC FW does not support
-	 * using up the entire buffer to get tail == head meaning full.
-	 */
-	if (tail < head)
-		used = (size - head) + tail;
-	else
-		used = tail - head;
-
-	/* make sure there is a space including extra dw for the fence */
-	if (unlikely(used + len + 1 >= size))
-		return -ENOSPC;
-
 	/*
 	 * Write the message. The format is the following:
 	 * DW0: header (including action code)
@@ -354,15 +344,16 @@ static int ctb_write(struct intel_guc_ct_buffer  
 *ctb,
 	}
 	/* now update desc tail (back in bytes) */
-	desc->tail = tail * 4;
-	GEM_BUG_ON(desc->tail > desc->size);
+	ctb->tail = desc->tail = tail * 4;
+	ctb->space -= (len + 1) * 4;
+	GEM_BUG_ON(ctb->tail > ctb->size);
 	return 0;
 }
 /**
 * wait_for_ctb_desc_update - Wait for the CT buffer descriptor update.
- * @desc:	buffer descriptor
+ * @ctb:	ctb buffer
 * @fence:	response fence
 * @status:	placeholder for status
 *
@@ -376,11 +367,12 @@ static int ctb_write(struct intel_guc_ct_buffer  
 *ctb,
 * *	-ETIMEDOUT no response within hardcoded timeout
 * *	-EPROTO no response, CT buffer is in error
 */
-static int wait_for_ctb_desc_update(struct guc_ct_buffer_desc *desc,
+static int wait_for_ctb_desc_update(struct intel_guc_ct_buffer *ctb,
 				    u32 fence,
 				    u32 *status)
 {
 	int err;
+	struct guc_ct_buffer_desc *desc = ctb->desc;
 	/*
 	 * Fast commands should complete in less than 10us, so sample quickly
@@ -401,7 +393,7 @@ static int wait_for_ctb_desc_update(struct  
 guc_ct_buffer_desc *desc,
 			/* Something went wrong with the messaging, try to reset
 			 * the buffer and hope for the best
 			 */
-			guc_ct_buffer_desc_reset(desc);
+			guc_ct_buffer_desc_reset(ctb);
 			err = -EPROTO;
 		}
	}
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +9 -6
@@ -446,12 +438,17 @@ static int wait_for_ct_request_update(struct  
 ct_request *req, u32 *status)
 	return err;
 }
-static inline bool ctb_has_room(struct guc_ct_buffer_desc *desc, u32  
 len)
+static inline bool ctb_has_room(struct intel_guc_ct_buffer *ctb, u32  
 len)
 {
-	u32 head = READ_ONCE(desc->head);
+	u32 head;
 	u32 space;
-	space = CIRC_SPACE(desc->tail, head, desc->size);
+	if (ctb->space >= len)
+		return true;
+
+	head = READ_ONCE(ctb->desc->head);
+	space = CIRC_SPACE(ctb->tail, head, ctb->size);
+	ctb->space = space;
 	return space >= len;
 }
@@ -462,7 +459,6 @@ int intel_guc_send_nb(struct intel_guc_ct *ct,
 {
 	struct intel_guc_ct_channel *ctch = &ct->host_channel;
 	struct intel_guc_ct_buffer *ctb = &ctch->ctbs[CTB_SEND];
-	struct guc_ct_buffer_desc *desc = ctb->desc;
 	int err;
 	GEM_BUG_ON(!ctch->enabled);
@@ -470,7 +466,7 @@ int intel_guc_send_nb(struct intel_guc_ct *ct,
 	GEM_BUG_ON(len & ~GUC_CT_MSG_LEN_MASK);
 	lockdep_assert_held(&ct->send_lock);
-	if (unlikely(!ctb_has_room(desc, (len + 1) * 4))) {
+	if (unlikely(!ctb_has_room(ctb, (len + 1) * 4))) {
 		ct->retry++;
 		if (ct->retry >= MAX_RETRY)
 			return -EDEADLK;
@@ -496,7 +492,6 @@ static int ctch_send(struct intel_guc_ct *ct,
 		     u32 *status)
 {
 	struct intel_guc_ct_buffer *ctb = &ctch->ctbs[CTB_SEND];
-	struct guc_ct_buffer_desc *desc = ctb->desc;
 	struct ct_request request;
 	unsigned long flags;
 	u32 fence;
@@ -514,7 +509,7 @@ static int ctch_send(struct intel_guc_ct *ct,
 	 */

retry:

	spin_lock_irqsave(&ct->send_lock, flags);
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +1 -1
-	if (unlikely(!ctb_has_room(desc, (len + 1) * 4))) {
+	if (unlikely(!ctb_has_room(ctb, (len + 1) * 4))) {
		spin_unlock_irqrestore(&ct->send_lock, flags);
		ct->retry++;
		if (ct->retry >= MAX_RETRY)
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +3 -3
@@ -544,7 +539,7 @@ static int ctch_send(struct intel_guc_ct *ct,
 	if (response_buf)
 		err = wait_for_ct_request_update(&request, status);
 	else
-		err = wait_for_ctb_desc_update(desc, fence, status);
+		err = wait_for_ctb_desc_update(ctb, fence, status);
 	if (unlikely(err))
 		goto unlink;
@@ -618,9 +613,9 @@ static inline bool ct_header_is_response(u32  
 header)
 static int ctb_read(struct intel_guc_ct_buffer *ctb, u32 *data)
 {
 	struct guc_ct_buffer_desc *desc = ctb->desc;
-	u32 head = desc->head / 4;	/* in dwords */
+	u32 head = ctb->head / 4;	/* in dwords */
 	u32 tail = desc->tail / 4;	/* in dwords */
-	u32 size = desc->size / 4;	/* in dwords */
+	u32 size = ctb->size / 4;	/* in dwords */
 	u32 *cmds = ctb->cmds;
 	s32 available;			/* in dwords */
	unsigned int len;
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +1 -1
@@ -664,7 +659,7 @@ static int ctb_read(struct intel_guc_ct_buffer  
 *ctb, u32 *data)
 	}
 	CT_DEBUG_DRIVER("CT: received %*ph\n", 4 * len, data);
-	desc->head = head * 4;
+	ctb->head = desc->head = head * 4;
 	return 0;
 }
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h +0 -0
index bc670a796bd8..1bff4f0b91f7 100644
Quoted 9 lines

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h struct intel_guc; · line 32

Quoted 5 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h +1 -0
@@ -29,10 +29,18 @@ struct intel_guc;
 *
 * @desc: pointer to the buffer descriptor
 * @cmds: pointer to the commands buffer
+ * @size: local shadow copy of size

I would rather expect this as real fixed size,
note that size is not expected to change

Yes, it is fixed over the life of the CTB but not all CTBs need to be the same size. e.g. The H2G & G2H may and likely will be different sizes with the new Guc interface.

Quoted 16 lines

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h struct intel_guc; · line 46

Quoted 13 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h +7 -0
+ * @head: local shadow copy of head
+ * @tail: local shadow copy of tail
+ * @space: local shadow copy of space
 */
 struct intel_guc_ct_buffer {
 	struct guc_ct_buffer_desc *desc;
 	u32 *cmds;
+	u32 size;
+	u32 tail;
+	u32 head;
+	u32 space;
 };
 /** Represents pair of command transport buffers.

Can we reorder this patch to be first in the series ?

Quoted 4 lines
Quoted 1 line
Quoted 1 line

Michal

Tried this and I think it makes more sense the way it is. The 'space' value doesn't have a meaning before the non blocking call is introduced.

hmm, blocking ctb_write also needs to check free space, and it is already doing that on its own, so there is need/usage for "space"

Quoted 3 lines

Also it ends
up changing a bunch of code that is then deleted in the non blocking call patch. Better to leave it as is.

this patch as 2nd also modifies some lines already touched by 1st patch but after reordering there will be more chances to merge this patch earlier

Quoted 14 lines

Matt

Quoted 11 lines
Quoted 1 line

_______________________________________________

Yes.

Quoted 3 lines

Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#12 lore raw

PATCH 3/3 drm/i915/guc: Increase size of CTB buffers

In reply to [email protected]

From: Matthew Brost <[email protected]>

With the introduction of non-blocking CTBs more than one CTB can be in flight at a time. Increasing the size of the CTBs should reduce how often software hits the case where no space is available in the CTB buffer.

---
 drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c | 50 +++++++++++++++--------
 1 file changed, 32 insertions(+), 18 deletions(-)
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +32 -18
index 4d8a4c6afd71..31c512e7ecc2 100644
@@ -14,6 +14,10 @@
 #define CT_DEBUG_DRIVER(...)	do { } while (0)
 #endif
 
+#define CTB_DESC_SIZE		(PAGE_SIZE / 2)
+#define CTB_H2G_BUFFER_SIZE	(PAGE_SIZE)
+#define CTB_G2H_BUFFER_SIZE	(CTB_H2G_BUFFER_SIZE * 2)
+
 #define MAX_RETRY		0x1000000
 
 struct ct_request {
@@ -143,30 +147,35 @@ static int ctch_init(struct intel_guc *guc,
 
 	GEM_BUG_ON(ctch->vma);
 
-	/* We allocate 1 page to hold both descriptors and both buffers.
+	/* We allocate 3 pages to hold both descriptors and both buffers.
 	 *       ___________.....................
 	 *      |desc (SEND)|                   :
-	 *      |___________|                   PAGE/4
+	 *      |___________|                   PAGE/2
 	 *      :___________....................:
 	 *      |desc (RECV)|                   :
-	 *      |___________|                   PAGE/4
+	 *      |___________|                   PAGE/2
 	 *      :_______________________________:
 	 *      |cmds (SEND)                    |
-	 *      |                               PAGE/4
+	 *      |                               PAGE
 	 *      |_______________________________|
 	 *      |cmds (RECV)                    |
-	 *      |                               PAGE/4
+	 *      |                               PAGE * 2
 	 *      |_______________________________|
 	 *
 	 * Each message can use a maximum of 32 dwords and we don't expect to
-	 * have more than 1 in flight at any time, so we have enough space.
-	 * Some logic further ahead will rely on the fact that there is only 1
-	 * page and that it is always mapped, so if the size is changed the
-	 * other code will need updating as well.
+	 * have more than 1 in flight at any time, unless we are using the GuC
+	 * submission. In that case each request requires a minimum 8 bytes
+	 * which gives us a maximum 512 queue'd requests. Hopefully this enough
+	 * space to avoid backpressure on the driver. We also double the size of
+	 * the receive buffer (relative to the send) to ensure a g2h response
+	 * CTB has a landing spot.
 	 */
 
 	/* allocate vma */
-	vma = intel_guc_allocate_vma(guc, PAGE_SIZE);
+	vma = intel_guc_allocate_vma(guc, CTB_DESC_SIZE *
+				     ARRAY_SIZE(ctch->ctbs) +
+				     CTB_H2G_BUFFER_SIZE +
+				     CTB_G2H_BUFFER_SIZE);
 	if (IS_ERR(vma)) {
 		err = PTR_ERR(vma);
 		goto err_out;
@@ -185,8 +194,9 @@ static int ctch_init(struct intel_guc *guc,
 	/* store pointers to desc and cmds */
 	for (i = 0; i < ARRAY_SIZE(ctch->ctbs); i++) {
 		GEM_BUG_ON((i != CTB_SEND) && (i != CTB_RECV));
-		ctch->ctbs[i].desc = blob + PAGE_SIZE/4 * i;
-		ctch->ctbs[i].cmds = blob + PAGE_SIZE/4 * i + PAGE_SIZE/2;
+		ctch->ctbs[i].desc = blob + CTB_DESC_SIZE * i;
+		ctch->ctbs[i].cmds = blob + CTB_H2G_BUFFER_SIZE * i +
+			CTB_DESC_SIZE * ARRAY_SIZE(ctch->ctbs);
 	}
 
 	return 0;
@@ -210,7 +220,7 @@ static void ctch_fini(struct intel_guc *guc,
 static int ctch_enable(struct intel_guc *guc,
 		       struct intel_guc_ct_channel *ctch)
 {
-	u32 base;
+	u32 base, size;
 	int err;
 	int i;
 
@@ -226,9 +236,12 @@ static int ctch_enable(struct intel_guc *guc,
 	 */
 	for (i = 0; i < ARRAY_SIZE(ctch->ctbs); i++) {
 		GEM_BUG_ON((i != CTB_SEND) && (i != CTB_RECV));
+		size = (i == CTB_SEND) ? CTB_H2G_BUFFER_SIZE :
+			CTB_G2H_BUFFER_SIZE;
 		guc_ct_buffer_desc_init(&ctch->ctbs[i],
-					base + PAGE_SIZE/4 * i + PAGE_SIZE/2,
-					PAGE_SIZE/4,
+					base + CTB_H2G_BUFFER_SIZE * i +
+					CTB_DESC_SIZE * ARRAY_SIZE(ctch->ctbs),
+					size,
 					ctch->owner);
 	}
 
@@ -236,13 +249,13 @@ static int ctch_enable(struct intel_guc *guc,
 	 * descriptors are in first half of the blob
 	 */
 	err = guc_action_register_ct_buffer(guc,
-					    base + PAGE_SIZE/4 * CTB_RECV,
+					    base + CTB_DESC_SIZE * CTB_RECV,
 					    INTEL_GUC_CT_BUFFER_TYPE_RECV);
 	if (unlikely(err))
 		goto err_out;
 
 	err = guc_action_register_ct_buffer(guc,
-					    base + PAGE_SIZE/4 * CTB_SEND,
+					    base + CTB_DESC_SIZE * CTB_SEND,
 					    INTEL_GUC_CT_BUFFER_TYPE_SEND);
 	if (unlikely(err))
 		goto err_deregister;
@@ -635,7 +648,8 @@ static int ctb_read(struct intel_guc_ct_buffer *ctb, u32 *data)
 	/* beware of buffer wrap case */
 	if (unlikely(available < 0))
 		available += size;
-	CT_DEBUG_DRIVER("CT: available %d (%u:%u)\n", available, head, tail);
+	CT_DEBUG_DRIVER("CT: available %d (%u:%u:%d)\n", available, head, tail,
+			size);
 	GEM_BUG_ON(available < 0);
 
 	data[0] = cmds[head];
Signature · 6 lines
2.21.0.5.gaeb582a983

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#13 lore raw

Re: [PATCH 3/3] drm/i915/guc: Increase size of CTB buffers

Quoted 18 lines · On Thu, 21 Nov 2019 00:56:04 +0100, <[email protected]> wrote:

From: Matthew Brost <[email protected]>

With the introduction of non-blocking CTBs more than one CTB can be in flight at a time. Increasing the size of the CTBs should reduce how often software hits the case where no space is available in the CTB buffer.

---
 drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c | 50 +++++++++++++++--------
 1 file changed, 32 insertions(+), 18 deletions(-)
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +0 -0
index 4d8a4c6afd71..31c512e7ecc2 100644

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c line 16

Quoted 3 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +1 -0
@@ -14,6 +14,10 @@
 #define CT_DEBUG_DRIVER(...)	do { } while (0)
 #endif
+#define CTB_DESC_SIZE		(PAGE_SIZE / 2)

CTB descriptors is now 64B each, not sure why we want to waste whole page for them. Maybe to better use space (and be ready for upcoming changes) we can place the buffer right after descriptor:

* <------ DESCRIPTOR ------> <--- BUFFER ------------->
*

 *      +--------------------------+--------------------------+
 *      | addr | head | tail | ... |                          |
 *      +--------------------------+--------------------------+

* \ ^
* \____________________/
*
* <------------ 64B -------> <---- n * PAGE - 64B ---->

Quoted 25 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +8 -5
+#define CTB_H2G_BUFFER_SIZE	(PAGE_SIZE)
+#define CTB_G2H_BUFFER_SIZE	(CTB_H2G_BUFFER_SIZE * 2)
+
 #define MAX_RETRY		0x1000000
 struct ct_request {
@@ -143,30 +147,35 @@ static int ctch_init(struct intel_guc *guc,
 	GEM_BUG_ON(ctch->vma);
-	/* We allocate 1 page to hold both descriptors and both buffers.
+	/* We allocate 3 pages to hold both descriptors and both buffers.
 	 *       ___________.....................
 	 *      |desc (SEND)|                   :
-	 *      |___________|                   PAGE/4
+	 *      |___________|                   PAGE/2
 	 *      :___________....................:
 	 *      |desc (RECV)|                   :
-	 *      |___________|                   PAGE/4
+	 *      |___________|                   PAGE/2
 	 *      :_______________________________:
 	 *      |cmds (SEND)                    |
-	 *      |                               PAGE/4
+	 *      |                               PAGE
 	 *      |_______________________________|
 	 *      |cmds (RECV)                    |
-	 *      |                               PAGE/4
+	 *      |                               PAGE * 2

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c static int ctch_init(struct intel_guc *guc, · line 166

Quoted 10 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +3 -4
 	 *      |_______________________________|
 	 *
 	 * Each message can use a maximum of 32 dwords and we don't expect to
-	 * have more than 1 in flight at any time, so we have enough space.
-	 * Some logic further ahead will rely on the fact that there is only 1
-	 * page and that it is always mapped, so if the size is changed the
-	 * other code will need updating as well.
+	 * have more than 1 in flight at any time, unless we are using the GuC
+	 * submission. In that case each request requires a minimum 8 bytes
+	 * which gives us a maximum 512 queue'd requests. Hopefully this enough

hmm, do we really expect to have 512 messages in flight ?

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c static int ctch_init(struct intel_guc *guc, · line 171

Quoted 4 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +1 -0
+	 * space to avoid backpressure on the driver. We also double the size  

of

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +2 -0
+	 * the receive buffer (relative to the send) to ensure a g2h response
+	 * CTB has a landing spot.

We do plan to send nob-blocking messages that might generate higher traffic, but do we expect matching increase in incoming traffic ? what kind of data will be there ? and do we expect that driver will be unable consume them in timely manner?

Quoted 59 lines

*/

	/* allocate vma */
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +15 -8
-	vma = intel_guc_allocate_vma(guc, PAGE_SIZE);
+	vma = intel_guc_allocate_vma(guc, CTB_DESC_SIZE *
+				     ARRAY_SIZE(ctch->ctbs) +
+				     CTB_H2G_BUFFER_SIZE +
+				     CTB_G2H_BUFFER_SIZE);
 	if (IS_ERR(vma)) {
 		err = PTR_ERR(vma);
 		goto err_out;
@@ -185,8 +194,9 @@ static int ctch_init(struct intel_guc *guc,
 	/* store pointers to desc and cmds */
 	for (i = 0; i < ARRAY_SIZE(ctch->ctbs); i++) {
 		GEM_BUG_ON((i != CTB_SEND) && (i != CTB_RECV));
-		ctch->ctbs[i].desc = blob + PAGE_SIZE/4 * i;
-		ctch->ctbs[i].cmds = blob + PAGE_SIZE/4 * i + PAGE_SIZE/2;
+		ctch->ctbs[i].desc = blob + CTB_DESC_SIZE * i;
+		ctch->ctbs[i].cmds = blob + CTB_H2G_BUFFER_SIZE * i +
+			CTB_DESC_SIZE * ARRAY_SIZE(ctch->ctbs);
 	}
 	return 0;
@@ -210,7 +220,7 @@ static void ctch_fini(struct intel_guc *guc,
 static int ctch_enable(struct intel_guc *guc,
 		       struct intel_guc_ct_channel *ctch)
 {
-	u32 base;
+	u32 base, size;
 	int err;
 	int i;
@@ -226,9 +236,12 @@ static int ctch_enable(struct intel_guc *guc,
 	 */
 	for (i = 0; i < ARRAY_SIZE(ctch->ctbs); i++) {
 		GEM_BUG_ON((i != CTB_SEND) && (i != CTB_RECV));
+		size = (i == CTB_SEND) ? CTB_H2G_BUFFER_SIZE :
+			CTB_G2H_BUFFER_SIZE;
 		guc_ct_buffer_desc_init(&ctch->ctbs[i],
-					base + PAGE_SIZE/4 * i + PAGE_SIZE/2,
-					PAGE_SIZE/4,
+					base + CTB_H2G_BUFFER_SIZE * i +
+					CTB_DESC_SIZE * ARRAY_SIZE(ctch->ctbs),
+					size,
 					ctch->owner);
 	}
@@ -236,13 +249,13 @@ static int ctch_enable(struct intel_guc *guc,
 	 * descriptors are in first half of the blob
 	 */
 	err = guc_action_register_ct_buffer(guc,
-					    base + PAGE_SIZE/4 * CTB_RECV,
+					    base + CTB_DESC_SIZE * CTB_RECV,
 					    INTEL_GUC_CT_BUFFER_TYPE_RECV);
 	if (unlikely(err))
 		goto err_out;
 	err = guc_action_register_ct_buffer(guc,
-					    base + PAGE_SIZE/4 * CTB_SEND,
+					    base + CTB_DESC_SIZE * CTB_SEND,
 					    INTEL_GUC_CT_BUFFER_TYPE_SEND);
 	if (unlikely(err))
 		goto err_deregister;

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c static int ctb_read(struct intel_guc_ct_buffer *ctb, · line 653

Quoted 7 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +2 -1
@@ -635,7 +648,8 @@ static int ctb_read(struct intel_guc_ct_buffer *ctb,  
 u32 *data)
 	/* beware of buffer wrap case */
 	if (unlikely(available < 0))
 		available += size;
-	CT_DEBUG_DRIVER("CT: available %d (%u:%u)\n", available, head, tail);
+	CT_DEBUG_DRIVER("CT: available %d (%u:%u:%d)\n", available, head, tail,
+			size);

size will not change, not sure if it is worth to repeat that in every log

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c static int ctb_read(struct intel_guc_ct_buffer *ctb, u32 *data) · line 655

Quoted 2 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +0 -0
 	GEM_BUG_ON(available < 0);
 	data[0] = cmds[head];

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#14 lore raw

Re: [PATCH 3/3] drm/i915/guc: Increase size of CTB buffers

Quoted 30 lines · Michal Wajdeczko
Quoted 18 lines · On Thu, 21 Nov 2019 00:56:04 +0100, <[email protected]> wrote:

From: Matthew Brost <[email protected]>

With the introduction of non-blocking CTBs more than one CTB can be in flight at a time. Increasing the size of the CTBs should reduce how often software hits the case where no space is available in the CTB buffer.

---
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c | 50 +++++++++++++++--------

1 file changed, 32 insertions(+), 18 deletions(-)
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +0 -0
index 4d8a4c6afd71..31c512e7ecc2 100644

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c line 16

Quoted 3 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +1 -0
@@ -14,6 +14,10 @@
 #define CT_DEBUG_DRIVER(...)	do { } while (0)
 #endif
+#define CTB_DESC_SIZE		(PAGE_SIZE / 2)

CTB descriptors is now 64B each, not sure why we want to waste whole page for them. Maybe to better use space (and be ready for upcoming changes) we can place the buffer right after descriptor:

* <------ DESCRIPTOR ------> <--- BUFFER ------------->
*

Quoted 8 lines
*      +--------------------------+--------------------------+
*      | addr | head | tail | ... |                          |
*      +--------------------------+--------------------------+

* \ ^
* \____________________/
*
* <------------ 64B -------> <---- n * PAGE - 64B ---->

I agree that is wasteful but if the linux/circ_buf.h wants to be used to determine the space in the buffer, the buffer size has to be a power of 2. I suspect the GuC determines space in a similar way and requires the size to be a power of 2. I can reach out and find out if this the case.

Quoted 29 lines
Quoted 29 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +8 -6
+#define CTB_H2G_BUFFER_SIZE	(PAGE_SIZE)
+#define CTB_G2H_BUFFER_SIZE	(CTB_H2G_BUFFER_SIZE * 2)
+
 #define MAX_RETRY		0x1000000
 struct ct_request {
@@ -143,30 +147,35 @@ static int ctch_init(struct intel_guc *guc,
 	GEM_BUG_ON(ctch->vma);
-	/* We allocate 1 page to hold both descriptors and both buffers.
+	/* We allocate 3 pages to hold both descriptors and both buffers.
 	 *       ___________.....................
 	 *      |desc (SEND)|                   :
-	 *      |___________|                   PAGE/4
+	 *      |___________|                   PAGE/2
 	 *      :___________....................:
 	 *      |desc (RECV)|                   :
-	 *      |___________|                   PAGE/4
+	 *      |___________|                   PAGE/2
 	 *      :_______________________________:
 	 *      |cmds (SEND)                    |
-	 *      |                               PAGE/4
+	 *      |                               PAGE
 	 *      |_______________________________|
 	 *      |cmds (RECV)                    |
-	 *      |                               PAGE/4
+	 *      |                               PAGE * 2
 	 *      |_______________________________|
 	 *
 	 * Each message can use a maximum of 32 dwords and we don't expect to
-	 * have more than 1 in flight at any time, so we have enough space.
Quoted 9 lines

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c static int ctch_init(struct intel_guc *guc, · line 168

Quoted 6 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +3 -3
-	 * Some logic further ahead will rely on the fact that there is only 1
-	 * page and that it is always mapped, so if the size is changed the
-	 * other code will need updating as well.
+	 * have more than 1 in flight at any time, unless we are using the GuC
+	 * submission. In that case each request requires a minimum 8 bytes
+	 * which gives us a maximum 512 queue'd requests. Hopefully this enough

hmm, do we really expect to have 512 messages in flight ?

Potentially, yes. In fact we may expect more than that. Part of the reason for this patch is to add the defines CTB_H2G_BUFFER_SIZE, CTB_G2H_BUFFER_SIZE so that the CTB sizes can easily be changed when profiling the code. We are going to want to size these buffers in way that flow control (no space in the buffer) is rarely hit.

BTW - This comment is wrong. The header CTB header is 8 bytes and I think the minimum payload is 8 bytes actually this should be 256 in flight.

Quoted 9 lines

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c static int ctch_init(struct intel_guc *guc, · line 171

Quoted 4 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +1 -0
+	 * space to avoid backpressure on the driver. We also double the 

size of

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +2 -0
+	 * the receive buffer (relative to the send) to ensure a g2h response
+	 * CTB has a landing spot.

We do plan to send nob-blocking messages that might generate higher traffic, but do we expect matching increase in incoming traffic ? what kind of data will be there ? and do we expect that driver will be unable consume them in timely manner?

Yes, as part of the new interface there are asynchronous G2H received as response to a H2G. In addition there several spontaneous G2H generated by the GuC.

Quoted 60 lines
Quoted 60 lines
	 */
	/* allocate vma */
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +4 -1
-	vma = intel_guc_allocate_vma(guc, PAGE_SIZE);
+	vma = intel_guc_allocate_vma(guc, CTB_DESC_SIZE *
+				     ARRAY_SIZE(ctch->ctbs) +
+				     CTB_H2G_BUFFER_SIZE +
+				     CTB_G2H_BUFFER_SIZE);
	if (IS_ERR(vma)) {
		err = PTR_ERR(vma);
		goto err_out;
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +11 -7
@@ -185,8 +194,9 @@ static int ctch_init(struct intel_guc *guc,
 	/* store pointers to desc and cmds */
 	for (i = 0; i < ARRAY_SIZE(ctch->ctbs); i++) {
 		GEM_BUG_ON((i != CTB_SEND) && (i != CTB_RECV));
-		ctch->ctbs[i].desc = blob + PAGE_SIZE/4 * i;
-		ctch->ctbs[i].cmds = blob + PAGE_SIZE/4 * i + PAGE_SIZE/2;
+		ctch->ctbs[i].desc = blob + CTB_DESC_SIZE * i;
+		ctch->ctbs[i].cmds = blob + CTB_H2G_BUFFER_SIZE * i +
+			CTB_DESC_SIZE * ARRAY_SIZE(ctch->ctbs);
 	}
 	return 0;
@@ -210,7 +220,7 @@ static void ctch_fini(struct intel_guc *guc,
 static int ctch_enable(struct intel_guc *guc,
 		       struct intel_guc_ct_channel *ctch)
 {
-	u32 base;
+	u32 base, size;
 	int err;
 	int i;
@@ -226,9 +236,12 @@ static int ctch_enable(struct intel_guc *guc,
 	 */
 	for (i = 0; i < ARRAY_SIZE(ctch->ctbs); i++) {
 		GEM_BUG_ON((i != CTB_SEND) && (i != CTB_RECV));
+		size = (i == CTB_SEND) ? CTB_H2G_BUFFER_SIZE :
+			CTB_G2H_BUFFER_SIZE;
 		guc_ct_buffer_desc_init(&ctch->ctbs[i],
-					base + PAGE_SIZE/4 * i + PAGE_SIZE/2,
-					PAGE_SIZE/4,
+					base + CTB_H2G_BUFFER_SIZE * i +
+					CTB_DESC_SIZE * ARRAY_SIZE(ctch->ctbs),
+					size,
 					ctch->owner);
 	}
@@ -236,13 +249,13 @@ static int ctch_enable(struct intel_guc *guc,
 	 * descriptors are in first half of the blob
 	 */
 	err = guc_action_register_ct_buffer(guc,
-					    base + PAGE_SIZE/4 * CTB_RECV,
+					    base + CTB_DESC_SIZE * CTB_RECV,
 					    INTEL_GUC_CT_BUFFER_TYPE_RECV);
 	if (unlikely(err))
 		goto err_out;
 	err = guc_action_register_ct_buffer(guc,
-					    base + PAGE_SIZE/4 * CTB_SEND,
+					    base + CTB_DESC_SIZE * CTB_SEND,
 					    INTEL_GUC_CT_BUFFER_TYPE_SEND);
 	if (unlikely(err))
 		goto err_deregister;
@@ -635,7 +648,8 @@ static int ctb_read(struct intel_guc_ct_buffer 
 *ctb, u32 *data)
Quoted 9 lines

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c static int ctb_read(struct intel_guc_ct_buffer *ctb, u32 *data) · line 652

Quoted 6 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +2 -1
 	/* beware of buffer wrap case */
 	if (unlikely(available < 0))
 		available += size;
-	CT_DEBUG_DRIVER("CT: available %d (%u:%u)\n", available, head, tail);
+	CT_DEBUG_DRIVER("CT: available %d (%u:%u:%d)\n", available, head, tail,
+			size);

size will not change, not sure if it is worth to repeat that in every log

This is a debug mechanism not turned on in normal operation. I find more information helpful when debugging problems.

Quoted 6 lines

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c static int ctb_read(struct intel_guc_ct_buffer *ctb, u32 *data) · line 655

Quoted 2 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +0 -0
 	GEM_BUG_ON(available < 0);
 	data[0] = cmds[head];

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#15 lore raw

✗ Fi.CI.CHECKPATCH: warning for drm/i915/guc: CTB improvements

In reply to [email protected]
== Series Details ==

== Series Details ==

Series: drm/i915/guc: CTB improvements
URL : https://patchwork.freedesktop.org/series/69788/
State : warning

== Summary ==

$ dim checkpatch origin/drm-tip
3db17158f33d drm/i915/guc: Add non blocking CTB send function
-:6: WARNING:TYPO_SPELLING: 'fuction' may be misspelled - perhaps 'function'?
#6: 
Add non blocking CTB send fuction, intel_guc_send_nb. In order to

-:7: WARNING:TYPO_SPELLING: 'fuction' may be misspelled - perhaps 'function'? #7:
support a non blocking CTB send fuction a spin lock is needed to

total: 0 errors, 2 warnings, 0 checks, 223 lines checked 6022fc4c10a0 drm/i915/guc: Optimized CTB writes and reads -:33: CHECK:MULTIPLE_ASSIGNMENTS: multiple assignments should be avoided #33: FILE: drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c:79:
+ ctb->size = desc->size = size;

-:48: CHECK:MULTIPLE_ASSIGNMENTS: multiple assignments should be avoided #48: FILE: drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c:91:
+ ctb->head = desc->head = 0;

-:49: CHECK:MULTIPLE_ASSIGNMENTS: multiple assignments should be avoided #49: FILE: drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c:92:
+ ctb->tail = desc->tail = 0;

-:106: CHECK:MULTIPLE_ASSIGNMENTS: multiple assignments should be avoided #106: FILE: drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c:347:
+ ctb->tail = desc->tail = tail * 4;

-:224: CHECK:MULTIPLE_ASSIGNMENTS: multiple assignments should be avoided #224: FILE: drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c:662:
+ ctb->head = desc->head = head * 4;

total: 0 errors, 0 warnings, 5 checks, 213 lines checked 1c5b53a0ae0e drm/i915/guc: Increase size of CTB buffers

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#16 lore raw

✓ Fi.CI.BAT: success for drm/i915/guc: CTB improvements

In reply to [email protected]
== Series Details ==

== Series Details ==

Series: drm/i915/guc: CTB improvements
URL : https://patchwork.freedesktop.org/series/69788/
State : success

== Summary ==

CI Bug Log - changes from CI_DRM_7394 -> Patchwork_15363
====================================================

Summary
-------

**SUCCESS**

No regressions found.

External URL: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/index.html

Known issues
------------

Here are the changes found in Patchwork_15363 that come from known issues:

### IGT changes ###

#### Issues hit ####

* igt@gem_mmap_gtt@basic-read-write:
- fi-glk-dsi: [PASS][1] -> [INCOMPLETE][2] ([fdo#103359] / [k.org#198133])
[1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/fi-glk-dsi/igt@[email protected]
[2]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/fi-glk-dsi/igt@[email protected]

* igt@kms_frontbuffer_tracking@basic:
- fi-icl-guc: [PASS][3] -> [FAIL][4] ([fdo#103167])
[3]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/fi-icl-guc/igt@[email protected]
[4]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/fi-icl-guc/igt@[email protected]

#### Possible fixes ####

* igt@gem_exec_gttfill@basic:
- {fi-tgl-u}: [INCOMPLETE][5] ([fdo#111593]) -> [PASS][6]
[5]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/fi-tgl-u/igt@[email protected]
[6]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/fi-tgl-u/igt@[email protected]

* igt@i915_selftest@live_execlists:
- fi-whl-u: [INCOMPLETE][7] ([fdo#112066]) -> [PASS][8]
[7]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/fi-whl-u/igt@i915_selftest@live_execlists.html
[8]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/fi-whl-u/igt@i915_selftest@live_execlists.html

* igt@i915_selftest@live_uncore:
- fi-bxt-dsi: [INCOMPLETE][9] ([fdo#103927]) -> [PASS][10]
[9]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/fi-bxt-dsi/igt@i915_selftest@live_uncore.html
[10]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/fi-bxt-dsi/igt@i915_selftest@live_uncore.html

* igt@kms_frontbuffer_tracking@basic:
- fi-hsw-peppy: [DMESG-WARN][11] ([fdo#102614]) -> [PASS][12]
[11]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/fi-hsw-peppy/igt@[email protected]
[12]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/fi-hsw-peppy/igt@[email protected]

{name}: This element is suppressed. This means it is ignored when computing
the status of the difference (SUCCESS, WARNING, or FAILURE).

[fdo#102614]: https://bugs.freedesktop.org/show_bug.cgi?id=102614
[fdo#103167]: https://bugs.freedesktop.org/show_bug.cgi?id=103167
[fdo#103359]: https://bugs.freedesktop.org/show_bug.cgi?id=103359
[fdo#103927]: https://bugs.freedesktop.org/show_bug.cgi?id=103927
[fdo#111593]: https://bugs.freedesktop.org/show_bug.cgi?id=111593
[fdo#112066]: https://bugs.freedesktop.org/show_bug.cgi?id=112066
[k.org#198133]: https://bugzilla.kernel.org/show_bug.cgi?id=198133

Participating hosts (51 -> 45)
------------------------------

Missing (6): fi-hsw-4200u fi-byt-squawks fi-bsw-cyan fi-ctg-p8600 fi-byt-clapper fi-bdw-samus

Build changes
-------------

* CI: CI-20190529 -> None
* Linux: CI_DRM_7394 -> Patchwork_15363

CI-20190529: 20190529
CI_DRM_7394: fdf3c3d9ba80a629caf1f76952ce619dc3dc8500 @ git://anongit.freedesktop.org/gfx-ci/linux
IGT_5299: 65fed6a79adea14f7bef6d55530da47d7731d370 @ git://anongit.freedesktop.org/xorg/app/intel-gpu-tools
Patchwork_15363: 1c5b53a0ae0e400024133a668c905bc10175c78f @ git://anongit.freedesktop.org/gfx-ci/linux

== Linux commits ==

1c5b53a0ae0e drm/i915/guc: Increase size of CTB buffers 6022fc4c10a0 drm/i915/guc: Optimized CTB writes and reads 3db17158f33d drm/i915/guc: Add non blocking CTB send function

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/index.html
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#17 lore raw

✓ Fi.CI.IGT: success for drm/i915/guc: CTB improvements

In reply to [email protected]
== Series Details ==

== Series Details ==

Series: drm/i915/guc: CTB improvements
URL : https://patchwork.freedesktop.org/series/69788/
State : success

== Summary ==

CI Bug Log - changes from CI_DRM_7394_full -> Patchwork_15363_full
====================================================

Summary
-------

**SUCCESS**

No regressions found.

Known issues
------------

Here are the changes found in Patchwork_15363_full that come from known issues:

### IGT changes ###

#### Issues hit ####

* igt@gem_busy@extended-parallel-vcs1:
- shard-iclb: [PASS][1] -> [SKIP][2] ([fdo#112080]) +5 similar issues
[1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/shard-iclb4/igt@[email protected]
[2]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/shard-iclb8/igt@[email protected]

* igt@gem_ctx_isolation@vcs1-none:
- shard-iclb: [PASS][3] -> [SKIP][4] ([fdo#109276] / [fdo#112080]) +1 similar issue
[3]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/shard-iclb1/igt@[email protected]
[4]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/shard-iclb3/igt@[email protected]

* igt@gem_exec_schedule@preempt-queue-bsd:
- shard-iclb: [PASS][5] -> [SKIP][6] ([fdo#112146]) +3 similar issues
[5]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/shard-iclb6/igt@[email protected]
[6]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/shard-iclb1/igt@[email protected]

* igt@gem_exec_schedule@preempt-queue-bsd2:
- shard-tglb: [PASS][7] -> [INCOMPLETE][8] ([fdo#111606] / [fdo#111677])
[7]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/shard-tglb7/igt@[email protected]
[8]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/shard-tglb6/igt@[email protected]

* igt@gem_userptr_blits@dmabuf-unsync:
- shard-hsw: [PASS][9] -> [DMESG-WARN][10] ([fdo#111870])
[9]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/shard-hsw7/igt@[email protected]
[10]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/shard-hsw7/igt@[email protected]

* igt@gem_userptr_blits@sync-unmap:
- shard-snb: [PASS][11] -> [DMESG-WARN][12] ([fdo#111870])
[11]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/shard-snb2/igt@[email protected]
[12]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/shard-snb2/igt@[email protected]

* igt@kms_cursor_crc@pipe-a-cursor-suspend:
- shard-tglb: [PASS][13] -> [INCOMPLETE][14] ([fdo#111832] / [fdo#111850]) +1 similar issue
[13]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/shard-tglb2/igt@[email protected]
[14]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/shard-tglb1/igt@[email protected]

* igt@kms_draw_crc@draw-method-xrgb2101010-pwrite-untiled:
- shard-skl: [PASS][15] -> [FAIL][16] ([fdo#103184] / [fdo#103232] / [fdo#108472])
[15]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/shard-skl2/igt@[email protected]
[16]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/shard-skl7/igt@[email protected]

* igt@kms_draw_crc@draw-method-xrgb8888-mmap-gtt-ytiled:
- shard-skl: [PASS][17] -> [FAIL][18] ([fdo#103184] / [fdo#103232] / [fdo#108145])
[17]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/shard-skl2/igt@[email protected]
[18]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/shard-skl7/igt@[email protected]

* igt@kms_flip@flip-vs-expired-vblank:
- shard-skl: [PASS][19] -> [FAIL][20] ([fdo#105363]) +1 similar issue
[19]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/shard-skl9/igt@[email protected]
[20]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/shard-skl6/igt@[email protected]

* igt@kms_flip@flip-vs-suspend:
- shard-skl: [PASS][21] -> [INCOMPLETE][22] ([fdo#109507])
[21]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/shard-skl3/igt@[email protected]
[22]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/shard-skl2/igt@[email protected]

* igt@kms_flip@flip-vs-suspend-interruptible:
- shard-kbl: [PASS][23] -> [DMESG-WARN][24] ([fdo#108566]) +5 similar issues
[23]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/shard-kbl3/igt@[email protected]
[24]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/shard-kbl2/igt@[email protected]

* igt@kms_frontbuffer_tracking@fbc-1p-offscren-pri-shrfb-draw-pwrite:
- shard-iclb: [PASS][25] -> [FAIL][26] ([fdo#103167]) +2 similar issues
[25]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/shard-iclb1/igt@kms_frontbuffer_tracking@fbc-1p-offscren-pri-shrfb-draw-pwrite.html
[26]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/shard-iclb1/igt@kms_frontbuffer_tracking@fbc-1p-offscren-pri-shrfb-draw-pwrite.html

* igt@kms_frontbuffer_tracking@fbcpsr-suspend:
- shard-tglb: [PASS][27] -> [INCOMPLETE][28] ([fdo#111832] / [fdo#111850] / [fdo#111884])
[27]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/shard-tglb9/igt@[email protected]
[28]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/shard-tglb8/igt@[email protected]

* igt@kms_plane_alpha_blend@pipe-b-coverage-7efc:
- shard-skl: [PASS][29] -> [FAIL][30] ([fdo#108145] / [fdo#110403])
[29]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/shard-skl5/igt@[email protected]
[30]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/shard-skl3/igt@[email protected]

* igt@kms_plane_alpha_blend@pipe-c-constant-alpha-min:
- shard-skl: [PASS][31] -> [FAIL][32] ([fdo#108145])
[31]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/shard-skl1/igt@[email protected]
[32]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/shard-skl2/igt@[email protected]

* igt@kms_plane_lowres@pipe-a-tiling-x:
- shard-iclb: [PASS][33] -> [FAIL][34] ([fdo#103166])
[33]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/shard-iclb5/igt@[email protected]
[34]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/shard-iclb8/igt@[email protected]

* igt@kms_psr@psr2_primary_mmap_cpu:
- shard-iclb: [PASS][35] -> [SKIP][36] ([fdo#109441]) +1 similar issue
[35]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/shard-iclb2/igt@kms_psr@psr2_primary_mmap_cpu.html
[36]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/shard-iclb4/igt@kms_psr@psr2_primary_mmap_cpu.html

* igt@kms_vblank@pipe-b-ts-continuation-dpms-suspend:
- shard-kbl: [PASS][37] -> [INCOMPLETE][38] ([fdo#103665])
[37]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/shard-kbl1/igt@[email protected]
[38]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/shard-kbl3/igt@[email protected]
- shard-tglb: [PASS][39] -> [INCOMPLETE][40] ([fdo#111850])
[39]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/shard-tglb7/igt@[email protected]
[40]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/shard-tglb8/igt@[email protected]

* igt@perf@gen8-unprivileged-single-ctx-counters:
- shard-skl: [PASS][41] -> [INCOMPLETE][42] ([fdo#111747])
[41]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/shard-skl6/igt@[email protected]
[42]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/shard-skl1/igt@[email protected]

* igt@prime_vgem@fence-wait-bsd2:
- shard-iclb: [PASS][43] -> [SKIP][44] ([fdo#109276]) +12 similar issues
[43]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/shard-iclb1/igt@[email protected]
[44]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/shard-iclb6/igt@[email protected]

#### Possible fixes ####

* igt@gem_ctx_isolation@vcs1-s3:
- shard-tglb: [INCOMPLETE][45] ([fdo#111832]) -> [PASS][46]
[45]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/shard-tglb5/igt@[email protected]
[46]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/shard-tglb6/igt@[email protected]

* igt@gem_ctx_persistence@vcs1-cleanup:
- shard-iclb: [SKIP][47] ([fdo#109276] / [fdo#112080]) -> [PASS][48] +1 similar issue
[47]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/shard-iclb3/igt@[email protected]
[48]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/shard-iclb4/igt@[email protected]

* igt@gem_ctx_switch@queue-heavy:
- shard-tglb: [INCOMPLETE][49] ([fdo#111747]) -> [PASS][50]
[49]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/shard-tglb6/igt@[email protected]
[50]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/shard-tglb1/igt@[email protected]

* igt@gem_exec_balancer@smoke:
- shard-iclb: [SKIP][51] ([fdo#110854]) -> [PASS][52]
[51]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/shard-iclb6/igt@[email protected]
[52]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/shard-iclb1/igt@[email protected]

* igt@gem_exec_create@forked:
- shard-tglb: [INCOMPLETE][53] ([fdo#108838] / [fdo#111747]) -> [PASS][54]
[53]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/shard-tglb6/igt@[email protected]
[54]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/shard-tglb5/igt@[email protected]

* igt@gem_exec_parallel@vcs1-fds:
- shard-iclb: [SKIP][55] ([fdo#112080]) -> [PASS][56] +15 similar issues
[55]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/shard-iclb6/igt@[email protected]
[56]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/shard-iclb1/igt@[email protected]

* igt@gem_exec_schedule@wide-bsd:
- shard-iclb: [SKIP][57] ([fdo#112146]) -> [PASS][58] +2 similar issues
[57]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/shard-iclb2/igt@[email protected]
[58]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/shard-iclb3/igt@[email protected]

* igt@gem_exec_suspend@basic-s3:
- shard-tglb: [INCOMPLETE][59] ([fdo#111736] / [fdo#111850]) -> [PASS][60]
[59]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/shard-tglb8/igt@[email protected]
[60]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/shard-tglb5/igt@[email protected]

* igt@gem_softpin@noreloc-s3:
- shard-skl: [INCOMPLETE][61] ([fdo#104108]) -> [PASS][62]
[61]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/shard-skl10/igt@[email protected]
[62]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/shard-skl7/igt@[email protected]

* igt@gem_userptr_blits@sync-unmap-cycles:
- shard-snb: [DMESG-WARN][63] ([fdo#111870]) -> [PASS][64]
[63]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/shard-snb2/igt@[email protected]
[64]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/shard-snb1/igt@[email protected]

* igt@i915_selftest@live_execlists:
- shard-kbl: [INCOMPLETE][65] ([fdo#103665] / [fdo#112259]) -> [PASS][66]
[65]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/shard-kbl6/igt@i915_selftest@live_execlists.html
[66]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/shard-kbl3/igt@i915_selftest@live_execlists.html

* igt@i915_selftest@live_gem_contexts:
- shard-tglb: [INCOMPLETE][67] ([fdo#111831]) -> [PASS][68]
[67]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/shard-tglb6/igt@i915_selftest@live_gem_contexts.html
[68]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/shard-tglb9/igt@i915_selftest@live_gem_contexts.html

* igt@kms_cursor_crc@pipe-c-cursor-suspend:
- shard-kbl: [DMESG-WARN][69] ([fdo#108566]) -> [PASS][70] +4 similar issues
[69]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/shard-kbl4/igt@[email protected]
[70]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/shard-kbl4/igt@[email protected]

* igt@kms_fbcon_fbt@fbc-suspend:
- shard-tglb: [INCOMPLETE][71] ([fdo#111747] / [fdo#111832] / [fdo#111850]) -> [PASS][72]
[71]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/shard-tglb4/igt@[email protected]
[72]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/shard-tglb9/igt@[email protected]

* igt@kms_flip@2x-flip-vs-suspend:
- shard-hsw: [INCOMPLETE][73] ([fdo#103540]) -> [PASS][74] +2 similar issues
[73]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/shard-hsw5/igt@[email protected]
[74]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/shard-hsw6/igt@[email protected]

* igt@kms_flip@flip-vs-suspend-interruptible:
- shard-apl: [DMESG-WARN][75] ([fdo#108566]) -> [PASS][76] +3 similar issues
[75]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/shard-apl4/igt@[email protected]
[76]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/shard-apl6/igt@[email protected]

* igt@kms_frontbuffer_tracking@fbc-1p-primscrn-cur-indfb-draw-render:
- shard-iclb: [FAIL][77] ([fdo#103167]) -> [PASS][78] +6 similar issues
[77]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/shard-iclb4/igt@kms_frontbuffer_tracking@fbc-1p-primscrn-cur-indfb-draw-render.html
[78]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/shard-iclb2/igt@kms_frontbuffer_tracking@fbc-1p-primscrn-cur-indfb-draw-render.html

* igt@kms_frontbuffer_tracking@fbcpsr-1p-primscrn-pri-indfb-draw-render:
- shard-tglb: [FAIL][79] ([fdo#103167]) -> [PASS][80] +2 similar issues
[79]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/shard-tglb9/igt@kms_frontbuffer_tracking@fbcpsr-1p-primscrn-pri-indfb-draw-render.html
[80]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/shard-tglb9/igt@kms_frontbuffer_tracking@fbcpsr-1p-primscrn-pri-indfb-draw-render.html

* igt@kms_frontbuffer_tracking@psr-slowdraw:
- shard-skl: [FAIL][81] ([fdo#103167]) -> [PASS][82]
[81]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/shard-skl8/igt@[email protected]
[82]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/shard-skl4/igt@[email protected]

* igt@kms_plane_alpha_blend@pipe-a-constant-alpha-min:
- shard-skl: [FAIL][83] ([fdo#108145]) -> [PASS][84]
[83]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/shard-skl8/igt@[email protected]
[84]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/shard-skl4/igt@[email protected]

* igt@kms_psr@psr2_primary_blt:
- shard-iclb: [SKIP][85] ([fdo#109441]) -> [PASS][86]
[85]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/shard-iclb1/igt@kms_psr@psr2_primary_blt.html
[86]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/shard-iclb2/igt@kms_psr@psr2_primary_blt.html

* igt@prime_busy@hang-bsd2:
- shard-iclb: [SKIP][87] ([fdo#109276]) -> [PASS][88] +11 similar issues
[87]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/shard-iclb3/igt@[email protected]
[88]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/shard-iclb4/igt@[email protected]

#### Warnings ####

* igt@gem_ctx_isolation@vcs1-nonpriv-switch:
- shard-iclb: [SKIP][89] ([fdo#109276] / [fdo#112080]) -> [FAIL][90] ([fdo#111329])
[89]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/shard-iclb6/igt@[email protected]
[90]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/shard-iclb1/igt@[email protected]

* igt@gem_eio@kms:
- shard-snb: [DMESG-WARN][91] ([fdo#111781] / [fdo#112001]) -> [INCOMPLETE][92] ([fdo#105411])
[91]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/shard-snb1/igt@[email protected]
[92]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/shard-snb5/igt@[email protected]

* igt@gem_exec_schedule@deep-bsd1:
- shard-tglb: [FAIL][93] ([fdo#111646]) -> [INCOMPLETE][94] ([fdo#111671]) +1 similar issue
[93]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7394/shard-tglb5/igt@[email protected]
[94]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/shard-tglb4/igt@[email protected]

{name}: This element is suppressed. This means it is ignored when computing
the status of the difference (SUCCESS, WARNING, or FAILURE).

[fdo#103166]: https://bugs.freedesktop.org/show_bug.cgi?id=103166
[fdo#103167]: https://bugs.freedesktop.org/show_bug.cgi?id=103167
[fdo#103184]: https://bugs.freedesktop.org/show_bug.cgi?id=103184
[fdo#103232]: https://bugs.freedesktop.org/show_bug.cgi?id=103232
[fdo#103540]: https://bugs.freedesktop.org/show_bug.cgi?id=103540
[fdo#103665]: https://bugs.freedesktop.org/show_bug.cgi?id=103665
[fdo#104108]: https://bugs.freedesktop.org/show_bug.cgi?id=104108
[fdo#105363]: https://bugs.freedesktop.org/show_bug.cgi?id=105363
[fdo#105411]: https://bugs.freedesktop.org/show_bug.cgi?id=105411
[fdo#108145]: https://bugs.freedesktop.org/show_bug.cgi?id=108145
[fdo#108472]: https://bugs.freedesktop.org/show_bug.cgi?id=108472
[fdo#108566]: https://bugs.freedesktop.org/show_bug.cgi?id=108566
[fdo#108838]: https://bugs.freedesktop.org/show_bug.cgi?id=108838
[fdo#109276]: https://bugs.freedesktop.org/show_bug.cgi?id=109276
[fdo#109441]: https://bugs.freedesktop.org/show_bug.cgi?id=109441
[fdo#109507]: https://bugs.freedesktop.org/show_bug.cgi?id=109507
[fdo#110403]: https://bugs.freedesktop.org/show_bug.cgi?id=110403
[fdo#110854]: https://bugs.freedesktop.org/show_bug.cgi?id=110854
[fdo#111329]: https://bugs.freedesktop.org/show_bug.cgi?id=111329
[fdo#111606]: https://bugs.freedesktop.org/show_bug.cgi?id=111606
[fdo#111646]: https://bugs.freedesktop.org/show_bug.cgi?id=111646
[fdo#111671]: https://bugs.freedesktop.org/show_bug.cgi?id=111671
[fdo#111677]: https://bugs.freedesktop.org/show_bug.cgi?id=111677
[fdo#111736]: https://bugs.freedesktop.org/show_bug.cgi?id=111736
[fdo#111747]: https://bugs.freedesktop.org/show_bug.cgi?id=111747
[fdo#111781]: https://bugs.freedesktop.org/show_bug.cgi?id=111781
[fdo#111831]: https://bugs.freedesktop.org/show_bug.cgi?id=111831
[fdo#111832]: https://bugs.freedesktop.org/show_bug.cgi?id=111832
[fdo#111850]: https://bugs.freedesktop.org/show_bug.cgi?id=111850
[fdo#111870]: https://bugs.freedesktop.org/show_bug.cgi?id=111870
[fdo#111884]: https://bugs.freedesktop.org/show_bug.cgi?id=111884
[fdo#112001]: https://bugs.freedesktop.org/show_bug.cgi?id=112001
[fdo#112080]: https://bugs.freedesktop.org/show_bug.cgi?id=112080
[fdo#1

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_15363/index.html
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx