Skip to contentSkip to conversation

PSP TEE driver update and bug fixes

PATCH Superseded Rijo Thomas 2 participants 8 messages Mar 9, 2021 – Mar 10, 2021

Subsystems (2)

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

cover PSP TEE driver update and bug fixes

The first patch helps to improve the response time by reducing the polling time of the tee command status variable.

Second patch is a bug fix to handle multi-threaded use-case. During testing, race condition was seen due to missing synchronisation in writes to the TEE ring buffer. This patch helps to resolve that.

Third patch is to update the copyright year for the tee driver files.

Rijo Thomas (3):
crypto: ccp - reduce tee command status polling interval from 5ms to

    1ms

crypto: ccp - fix command queuing to TEE ring buffer
crypto: ccp - update copyright year for tee

 drivers/crypto/ccp/tee-dev.c | 57 ++++++++++++++++++++++++------------
 drivers/crypto/ccp/tee-dev.h | 20 +++++++++++--
 2 files changed, 57 insertions(+), 20 deletions(-)
Signature · 1 line
2.17.1
#1 lore raw

PATCH 1/3 crypto: ccp - reduce tee command status polling interval from 5ms to 1ms

The PSP TEE device driver polls the command status variable every 5ms to check for command completion. Reduce this time to 1ms so that there is an improvement in driver response time to clients which submit TEE commands.

---
 drivers/crypto/ccp/tee-dev.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
drivers/crypto/ccp/tee-dev.c +3 -3
index 5e697a90ea7f..fe18a92d51ad 100644
@@ -309,14 +309,14 @@ static int tee_wait_cmd_completion(struct psp_tee_device *tee,
 				   struct tee_ring_cmd *resp,
 				   unsigned int timeout)
 {
-	/* ~5ms sleep per loop => nloop = timeout * 200 */
-	int nloop = timeout * 200;
+	/* ~1ms sleep per loop => nloop = timeout * 1000 */
+	int nloop = timeout * 1000;
 
 	while (--nloop) {
 		if (resp->cmd_state == TEE_CMD_STATE_COMPLETED)
 			return 0;
 
-		usleep_range(5000, 5100);
+		usleep_range(1000, 1100);
 	}
 
 	dev_err(tee->dev, "tee: command 0x%x timed out, disabling PSP\n",
Signature · 1 line
2.17.1
#2 lore raw

PATCH 2/3 crypto: ccp - fix command queuing to TEE ring buffer

In reply to Rijo Thomas

Multiple threads or clients can submit a command to the TEE ring buffer. This patch helps to synchronize command submission to the ring.

One thread shall write a command to a TEE ring buffer entry only if:

- Trusted OS has notified that the TEE command for the given entry has been processed and driver has copied the TEE response into client buffer.

- The command entry is empty and can be written into.

After a command has been written to the TEE ring buffer, the global wptr (mutex protected) shall be incremented for use by next client.

If PSP became unresponsive while processing TEE request from a client, then further command submission to queue will be disabled.

Fixes 33960acccfbd (crypto: ccp - add TEE support for Raven Ridge)
---
 drivers/crypto/ccp/tee-dev.c | 49 +++++++++++++++++++++++++-----------
 drivers/crypto/ccp/tee-dev.h | 18 ++++++++++++-
 2 files changed, 52 insertions(+), 15 deletions(-)
drivers/crypto/ccp/tee-dev.c +35 -14
index fe18a92d51ad..1aa264815028 100644
@@ -36,6 +36,7 @@ static int tee_alloc_ring(struct psp_tee_device *tee, int ring_size)
 	if (!start_addr)
 		return -ENOMEM;
 
+	memset(start_addr, 0x0, ring_size);
 	rb_mgr->ring_start = start_addr;
 	rb_mgr->ring_size = ring_size;
 	rb_mgr->ring_pa = __psp_pa(start_addr);
@@ -244,41 +245,54 @@ static int tee_submit_cmd(struct psp_tee_device *tee, enum tee_cmd_id cmd_id,
 			  void *buf, size_t len, struct tee_ring_cmd **resp)
 {
 	struct tee_ring_cmd *cmd;
-	u32 rptr, wptr;
 	int nloop = 1000, ret = 0;
+	u32 rptr;
 
 	*resp = NULL;
 
 	mutex_lock(&tee->rb_mgr.mutex);
 
-	wptr = tee->rb_mgr.wptr;
-
-	/* Check if ring buffer is full */
+	/* Loop until empty entry found in ring buffer */
 	do {
+		/* Get pointer to ring buffer command entry */
+		cmd = (struct tee_ring_cmd *)
+			(tee->rb_mgr.ring_start + tee->rb_mgr.wptr);
+
 		rptr = ioread32(tee->io_regs + tee->vdata->ring_rptr_reg);
 
-		if (!(wptr + sizeof(struct tee_ring_cmd) == rptr))
+		/* Check if ring buffer is full or command entry is waiting
+		 * for response from TEE
+		 */
+		if (!(tee->rb_mgr.wptr + sizeof(struct tee_ring_cmd) == rptr ||
+		      cmd->flag == CMD_WAITING_FOR_RESPONSE))
 			break;
 
-		dev_info(tee->dev, "tee: ring buffer full. rptr = %u wptr = %u\n",
-			 rptr, wptr);
+		dev_dbg(tee->dev, "tee: ring buffer full. rptr = %u wptr = %u\n",
+			rptr, tee->rb_mgr.wptr);
 
-		/* Wait if ring buffer is full */
+		/* Wait if ring buffer is full or TEE is processing data */
 		mutex_unlock(&tee->rb_mgr.mutex);
 		schedule_timeout_interruptible(msecs_to_jiffies(10));
 		mutex_lock(&tee->rb_mgr.mutex);
 
 	} while (--nloop);
 
-	if (!nloop && (wptr + sizeof(struct tee_ring_cmd) == rptr)) {
-		dev_err(tee->dev, "tee: ring buffer full. rptr = %u wptr = %u\n",
-			rptr, wptr);
+	if (!nloop &&
+	    (tee->rb_mgr.wptr + sizeof(struct tee_ring_cmd) == rptr ||
+	     cmd->flag == CMD_WAITING_FOR_RESPONSE)) {
+		dev_err(tee->dev, "tee: ring buffer full. rptr = %u wptr = %u response flag %u\n",
+			rptr, tee->rb_mgr.wptr, cmd->flag);
 		ret = -EBUSY;
 		goto unlock;
 	}
 
-	/* Pointer to empty data entry in ring buffer */
-	cmd = (struct tee_ring_cmd *)(tee->rb_mgr.ring_start + wptr);
+	/* Do not submit command if PSP got disabled while processing any
+	 * command in another thread
+	 */
+	if (psp_dead == true) {
+		ret = -EBUSY;
+		goto unlock;
+	}
 
 	/* Write command data into ring buffer */
 	cmd->cmd_id = cmd_id;
@@ -286,6 +300,9 @@ static int tee_submit_cmd(struct psp_tee_device *tee, enum tee_cmd_id cmd_id,
 	memset(&cmd->buf[0], 0, sizeof(cmd->buf));
 	memcpy(&cmd->buf[0], buf, len);
 
+	/* Indicate driver is waiting for response */
+	cmd->flag = CMD_WAITING_FOR_RESPONSE;
+
 	/* Update local copy of write pointer */
 	tee->rb_mgr.wptr += sizeof(struct tee_ring_cmd);
 	if (tee->rb_mgr.wptr >= tee->rb_mgr.ring_size)
@@ -353,12 +370,16 @@ int psp_tee_process_cmd(enum tee_cmd_id cmd_id, void *buf, size_t len,
 		return ret;
 
 	ret = tee_wait_cmd_completion(tee, resp, TEE_DEFAULT_TIMEOUT);
-	if (ret)
+	if (ret) {
+		resp->flag = CMD_RESPONSE_TIMEDOUT;
 		return ret;
+	}
 
 	memcpy(buf, &resp->buf[0], len);
 	*status = resp->status;
 
+	resp->flag = CMD_RESPONSE_COPIED;
+
 	return 0;
 }
 EXPORT_SYMBOL(psp_tee_process_cmd);
drivers/crypto/ccp/tee-dev.h +17 -1
index f09960112115..dbeb7d289acb 100644
@@ -18,7 +18,7 @@
 #include <linux/mutex.h>
 
 #define TEE_DEFAULT_TIMEOUT		10
-#define MAX_BUFFER_SIZE			992
+#define MAX_BUFFER_SIZE			988
 
 /**
  * enum tee_ring_cmd_id - TEE interface commands for ring buffer configuration
@@ -81,6 +81,20 @@ enum tee_cmd_state {
 	TEE_CMD_STATE_COMPLETED,
 };
 
+/**
+ * enum cmd_resp_state - TEE command's response status maintained by driver
+ * @CMD_RESPONSE_INVALID:      initial state when no command is written to ring
+ * @CMD_WAITING_FOR_RESPONSE:  driver waiting for response from TEE
+ * @CMD_RESPONSE_TIMEDOUT:     failed to get response from TEE
+ * @CMD_RESPONSE_COPIED:       driver has copied response from TEE
+ */
+enum cmd_resp_state {
+	CMD_RESPONSE_INVALID,
+	CMD_WAITING_FOR_RESPONSE,
+	CMD_RESPONSE_TIMEDOUT,
+	CMD_RESPONSE_COPIED,
+};
+
 /**
  * struct tee_ring_cmd - Structure of the command buffer in TEE ring
  * @cmd_id:      refers to &enum tee_cmd_id. Command id for the ring buffer
@@ -91,6 +105,7 @@ enum tee_cmd_state {
  * @pdata:       private data (currently unused)
  * @res1:        reserved region
  * @buf:         TEE command specific buffer
+ * @flag:	 refers to &enum cmd_resp_state
  */
 struct tee_ring_cmd {
 	u32 cmd_id;
@@ -100,6 +115,7 @@ struct tee_ring_cmd {
 	u64 pdata;
 	u32 res1[2];
 	u8 buf[MAX_BUFFER_SIZE];
+	u32 flag;
 
 	/* Total size: 1024 bytes */
 } __packed;
Signature · 1 line
2.17.1
#3 lore raw

PATCH 3/3 crypto: ccp - update copyright year for tee

In reply to Rijo Thomas

Update the copyright year for PSP TEE driver files.

---
 drivers/crypto/ccp/tee-dev.c | 2 +-
 drivers/crypto/ccp/tee-dev.h | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
drivers/crypto/ccp/tee-dev.c +1 -1
index 1aa264815028..8cade4775115 100644
@@ -5,7 +5,7 @@
  * Author: Rijo Thomas <[email protected]>
  * Author: Devaraj Rangasamy <[email protected]>
  *
- * Copyright 2019 Advanced Micro Devices, Inc.
+ * Copyright (C) 2019,2021 Advanced Micro Devices, Inc.
  */
 
 #include <linux/types.h>
drivers/crypto/ccp/tee-dev.h +1 -1
index dbeb7d289acb..49d26158b71e 100644
@@ -1,6 +1,6 @@
 /* SPDX-License-Identifier: MIT */
 /*
- * Copyright 2019 Advanced Micro Devices, Inc.
+ * Copyright (C) 2019,2021 Advanced Micro Devices, Inc.
  *
  * Author: Rijo Thomas <[email protected]>
  * Author: Devaraj Rangasamy <[email protected]>
Signature · 1 line
2.17.1
maintainer
#4 lore raw

Re: [PATCH 3/3] crypto: ccp - update copyright year for tee

Quoted 3 lines · Rijo Thomas

Update the copyright year for PSP TEE driver files.

The copyright updates really should occur as part of the changes in the other patches vs a separate patch.

Thanks,
Tom

Quoted 31 lines
---
 drivers/crypto/ccp/tee-dev.c | 2 +-
 drivers/crypto/ccp/tee-dev.h | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
drivers/crypto/ccp/tee-dev.c +1 -1
index 1aa264815028..8cade4775115 100644
@@ -5,7 +5,7 @@
  * Author: Rijo Thomas <[email protected]>
  * Author: Devaraj Rangasamy <[email protected]>
  *
- * Copyright 2019 Advanced Micro Devices, Inc.
+ * Copyright (C) 2019,2021 Advanced Micro Devices, Inc.
  */
 
 #include <linux/types.h>
drivers/crypto/ccp/tee-dev.h +1 -1
index dbeb7d289acb..49d26158b71e 100644
@@ -1,6 +1,6 @@
 /* SPDX-License-Identifier: MIT */
 /*
- * Copyright 2019 Advanced Micro Devices, Inc.
+ * Copyright (C) 2019,2021 Advanced Micro Devices, Inc.
  *
  * Author: Rijo Thomas <[email protected]>
  * Author: Devaraj Rangasamy <[email protected]>
maintainer
#6 lore raw
In reply to Rijo Thomas
Quoted 9 lines · Rijo Thomas

The first patch helps to improve the response time by reducing the polling time of the tee command status variable.

Second patch is a bug fix to handle multi-threaded use-case. During testing, race condition was seen due to missing synchronisation in writes to the TEE ring buffer. This patch helps to resolve that.

Third patch is to update the copyright year for the tee driver files.

Just something to think about and not as part of this patch series, but think about submitting a patch that adds you as maintainer of the TEE portion of the driver (see how the SEV portion is handled).

Thanks,
Tom

Quoted 10 lines

Rijo Thomas (3):
crypto: ccp - reduce tee command status polling interval from 5ms to

    1ms

crypto: ccp - fix command queuing to TEE ring buffer
crypto: ccp - update copyright year for tee

 drivers/crypto/ccp/tee-dev.c | 57 ++++++++++++++++++++++++------------
 drivers/crypto/ccp/tee-dev.h | 20 +++++++++++--
 2 files changed, 57 insertions(+), 20 deletions(-)
#5 lore raw

Re: [PATCH 3/3] crypto: ccp - update copyright year for tee

In reply to Tom Lendacky
Quoted 7 lines · Tom Lendacky
Quoted 3 lines · Rijo Thomas

Update the copyright year for PSP TEE driver files.

The copyright updates really should occur as part of the changes in the other patches vs a separate patch.

Ack. Shall update in next patch version.

Thanks,
Rijo

Quoted 35 lines

Thanks,
Tom

Quoted 31 lines
---
 drivers/crypto/ccp/tee-dev.c | 2 +-
 drivers/crypto/ccp/tee-dev.h | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
drivers/crypto/ccp/tee-dev.c +1 -1
index 1aa264815028..8cade4775115 100644
@@ -5,7 +5,7 @@
  * Author: Rijo Thomas <[email protected]>
  * Author: Devaraj Rangasamy <[email protected]>
  *
- * Copyright 2019 Advanced Micro Devices, Inc.
+ * Copyright (C) 2019,2021 Advanced Micro Devices, Inc.
  */
 
 #include <linux/types.h>
drivers/crypto/ccp/tee-dev.h +1 -1
index dbeb7d289acb..49d26158b71e 100644
@@ -1,6 +1,6 @@
 /* SPDX-License-Identifier: MIT */
 /*
- * Copyright 2019 Advanced Micro Devices, Inc.
+ * Copyright (C) 2019,2021 Advanced Micro Devices, Inc.
  *
  * Author: Rijo Thomas <[email protected]>
  * Author: Devaraj Rangasamy <[email protected]>
#7 lore raw
In reply to Tom Lendacky
Quoted 8 lines · Tom Lendacky
Quoted 7 lines · Rijo Thomas

The first patch helps to improve the response time by reducing the polling time of the tee command status variable.

Second patch is a bug fix to handle multi-threaded use-case. During testing, race condition was seen due to missing synchronisation in writes to the TEE ring buffer. This patch helps to resolve that.

Quoted 7 lines
Quoted 2 lines

Third patch is to update the copyright year for the tee driver files.

Just something to think about and not as part of this patch series, but think about submitting a patch that adds you as maintainer of the TEE portion of the driver (see how the SEV portion is handled).

Sure Tom. I will add myself as maintainer for TEE portion of driver. I will post that as a separate patch.

Thanks,
Rijo

Quoted 13 lines

Thanks,
Tom

Quoted 10 lines

Rijo Thomas (3):
crypto: ccp - reduce tee command status polling interval from 5ms to

    1ms

crypto: ccp - fix command queuing to TEE ring buffer
crypto: ccp - update copyright year for tee

 drivers/crypto/ccp/tee-dev.c | 57 ++++++++++++++++++++++++------------
 drivers/crypto/ccp/tee-dev.h | 20 +++++++++++--
 2 files changed, 57 insertions(+), 20 deletions(-)