Skip to contentSkip to conversation

powerpc/powernv: Cache OPAL check_token() results

PATCH In review Shivang Upadhyay 4 participants 8 messages Jun 26 – Sep 23 PowerPC

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 powerpc/powernv: Cache OPAL check_token() results

opal_check_token() function is used to determine if a specific OPAL firmware call is supported on the current platform. This check is performed frequently during boot and runtime, resulting in unnecessary firmware calls for the same token values.

Add a caching layer for the opal_check_token() OPAL call to avoid repeated firmware calls for token availability checks. This reduces firmware call overhead during boot.

Testing with buildroot images shows OPAL calls reduced from 35578 to 28983, before console bring-up.

---
Changelog:
* v1: https://lore.kernel.org/all/[email protected]/
- implemented tri-state check as suggestd by Sourabh.
- made the implementation more clear.

---
 arch/powerpc/include/asm/opal.h            |  1 +
 arch/powerpc/platforms/powernv/opal-call.c |  2 +-
 arch/powerpc/platforms/powernv/opal.c      | 30 ++++++++++++++++++++++
 3 files changed, 32 insertions(+), 1 deletion(-)
arch/powerpc/include/asm/opal.h +1 -0
index 0a398265ba04..e7e11479122b 100644
@@ -156,6 +156,7 @@ int64_t opal_pci_next_error(uint64_t phb_id, __be64 *first_frozen_pe,
 int64_t opal_pci_poll(uint64_t id);
 int64_t opal_return_cpu(void);
 int64_t opal_check_token(uint64_t token);
+int64_t opal_check_token_call(uint64_t token);
 int64_t opal_reinit_cpus(uint64_t flags);
 
 int64_t opal_xscom_read(uint32_t gcid, uint64_t pcb_addr, __be64 *val);
arch/powerpc/platforms/powernv/opal-call.c +1 -1
index 021b0ec29e24..00325c189e69 100644
@@ -207,7 +207,7 @@ OPAL_CALL(opal_validate_flash,			OPAL_FLASH_VALIDATE);
 OPAL_CALL(opal_manage_flash,			OPAL_FLASH_MANAGE);
 OPAL_CALL(opal_update_flash,			OPAL_FLASH_UPDATE);
 OPAL_CALL(opal_resync_timebase,			OPAL_RESYNC_TIMEBASE);
-OPAL_CALL(opal_check_token,			OPAL_CHECK_TOKEN);
+OPAL_CALL(opal_check_token_call,		OPAL_CHECK_TOKEN);
 OPAL_CALL(opal_dump_init,			OPAL_DUMP_INIT);
 OPAL_CALL(opal_dump_info,			OPAL_DUMP_INFO);
 OPAL_CALL(opal_dump_info2,			OPAL_DUMP_INFO2);
arch/powerpc/platforms/powernv/opal.c +30 -0
index 1946dbdc9fa1..1e9cb5271ee7 100644
@@ -1125,6 +1125,36 @@ EXPORT_SYMBOL_GPL(opal_flash_read);
 EXPORT_SYMBOL_GPL(opal_flash_write);
 EXPORT_SYMBOL_GPL(opal_flash_erase);
 EXPORT_SYMBOL_GPL(opal_prd_msg);
+
+/**
+ * opal_check_token - Check if an OPAL call token is supported
+ * @token: OPAL token number to check
+ *
+ * Returns 1 if supported, 0 if not.
+ */
+int64_t opal_check_token(uint64_t token)
+{
+	static u8 token_cache[OPAL_LAST];
+	enum {
+		SUPP_UNKNOWN = 0,
+		PRESENT,
+		ABSENT
+	};
+
+	if (token > OPAL_LAST)
+		return 0;
+
+	if (token_cache[token] == SUPP_UNKNOWN) {
+		/* Do the actual opal_call here */
+		if (opal_check_token_call(token)) {
+			token_cache[token] = PRESENT;
+		} else {
+			token_cache[token] = ABSENT;
+		}
+	}
+
+	return (token_cache[token] == PRESENT);
+}
 EXPORT_SYMBOL_GPL(opal_check_token);
 
 /* Convert a region of vmalloc memory to an opal sg list */
Signature · 1 line
2.54.0
Reviewed-by on v2 1/1
#1 lore raw
Quoted 11 lines · Shivang Upadhyay

opal_check_token() function is used to determine if a specific OPAL firmware call is supported on the current platform. This check is performed frequently during boot and runtime, resulting in unnecessary firmware calls for the same token values.

Add a caching layer for the opal_check_token() OPAL call to avoid repeated firmware calls for token availability checks. This reduces firmware call overhead during boot.

Testing with buildroot images shows OPAL calls reduced from 35578 to 28983, before console bring-up.

My testing shows similar statistics.

As per my testing, the upstream kernel (without this patch) at commit 840ef6c78e6a2f694b578, compiled with powernv_defconfig, shows ~7K CHECK_TOKEN OPAL calls to boot the kernel and reach the console.

With this patch included, there are only 8 CHECK_TOKEN opal calls.

There is a significant reduction in CHECK_TOKEN OPAL calls with this patch. Thanks for working on this improvement.

The changes look good to me. Feel free to add:

Quoted 80 lines

---
Changelog:
* v1: https://lore.kernel.org/all/[email protected]/
- implemented tri-state check as suggestd by Sourabh.
- made the implementation more clear.

---
  arch/powerpc/include/asm/opal.h            |  1 +
  arch/powerpc/platforms/powernv/opal-call.c |  2 +-
  arch/powerpc/platforms/powernv/opal.c      | 30 ++++++++++++++++++++++
  3 files changed, 32 insertions(+), 1 deletion(-)
arch/powerpc/include/asm/opal.h +1 -0
index 0a398265ba04..e7e11479122b 100644
@@ -156,6 +156,7 @@ int64_t opal_pci_next_error(uint64_t phb_id, __be64 *first_frozen_pe,
  int64_t opal_pci_poll(uint64_t id);
  int64_t opal_return_cpu(void);
  int64_t opal_check_token(uint64_t token);
+int64_t opal_check_token_call(uint64_t token);
  int64_t opal_reinit_cpus(uint64_t flags);
 
  int64_t opal_xscom_read(uint32_t gcid, uint64_t pcb_addr, __be64 *val);
arch/powerpc/platforms/powernv/opal-call.c +1 -1
index 021b0ec29e24..00325c189e69 100644
@@ -207,7 +207,7 @@ OPAL_CALL(opal_validate_flash,			OPAL_FLASH_VALIDATE);
  OPAL_CALL(opal_manage_flash,			OPAL_FLASH_MANAGE);
  OPAL_CALL(opal_update_flash,			OPAL_FLASH_UPDATE);
  OPAL_CALL(opal_resync_timebase,			OPAL_RESYNC_TIMEBASE);
-OPAL_CALL(opal_check_token,			OPAL_CHECK_TOKEN);
+OPAL_CALL(opal_check_token_call,		OPAL_CHECK_TOKEN);
  OPAL_CALL(opal_dump_init,			OPAL_DUMP_INIT);
  OPAL_CALL(opal_dump_info,			OPAL_DUMP_INFO);
  OPAL_CALL(opal_dump_info2,			OPAL_DUMP_INFO2);
arch/powerpc/platforms/powernv/opal.c +30 -0
index 1946dbdc9fa1..1e9cb5271ee7 100644
@@ -1125,6 +1125,36 @@ EXPORT_SYMBOL_GPL(opal_flash_read);
  EXPORT_SYMBOL_GPL(opal_flash_write);
  EXPORT_SYMBOL_GPL(opal_flash_erase);
  EXPORT_SYMBOL_GPL(opal_prd_msg);
+
+/**
+ * opal_check_token - Check if an OPAL call token is supported
+ * @token: OPAL token number to check
+ *
+ * Returns 1 if supported, 0 if not.
+ */
+int64_t opal_check_token(uint64_t token)
+{
+	static u8 token_cache[OPAL_LAST];
+	enum {
+		SUPP_UNKNOWN = 0,
+		PRESENT,
+		ABSENT
+	};
+
+	if (token > OPAL_LAST)
+		return 0;
+
+	if (token_cache[token] == SUPP_UNKNOWN) {
+		/* Do the actual opal_call here */
+		if (opal_check_token_call(token)) {
+			token_cache[token] = PRESENT;
+		} else {
+			token_cache[token] = ABSENT;
+		}
+	}
+
+	return (token_cache[token] == PRESENT);
+}
  EXPORT_SYMBOL_GPL(opal_check_token);
 
  /* Convert a region of vmalloc memory to an opal sg list */
#2 lore raw
In reply to Shivang Upadhyay
Quoted 18 lines · Shivang Upadhyay
arch/powerpc/platforms/powernv/opal.c +10 -0
index 1946dbdc9fa1..1e9cb5271ee7 100644
@@ -1125,6 +1125,36 @@ EXPORT_SYMBOL_GPL(opal_flash_read);
  EXPORT_SYMBOL_GPL(opal_flash_write);
  EXPORT_SYMBOL_GPL(opal_flash_erase);
  EXPORT_SYMBOL_GPL(opal_prd_msg);
+
+/**
+ * opal_check_token - Check if an OPAL call token is supported
+ * @token: OPAL token number to check
+ *
+ * Returns 1 if supported, 0 if not.
+ */
+int64_t opal_check_token(uint64_t token)
+{
+	static u8 token_cache[OPAL_LAST];

arch/powerpc/platforms/powernv/opal.c EXPORT_SYMBOL_GPL(opal_flash_read); · line 1147

Quoted 10 lines
arch/powerpc/platforms/powernv/opal.c +10 -0
+	enum {
+		SUPP_UNKNOWN = 0,
+		PRESENT,
+		ABSENT
+	};
+
+	if (token > OPAL_LAST)
+		return 0;
+
+	if (token_cache[token] == SUPP_UNKNOWN) {

Hi Shivang,
Thanks for the v2 patch. This is a great optimization and definitely helps reduce unnecessary overhead. I have one small suggestion.

You can wrap the if (token_cache[token] == SUPP_UNKNOWN) check, as well as the if (token > OPAL_LAST) check above it, with unlikely(), since this function is in the hot path. Out-of-bounds tokens are very rare, and after the cache is populated, the SUPP_UNKNOWN condition is expected to be false in almost all cases. Wrapping these checks with unlikely() will help the compiler optimize branch prediction.

Thanks,
~ Nikhil

Quoted 13 lines
arch/powerpc/platforms/powernv/opal.c +10 -0
+		/* Do the actual opal_call here */
+		if (opal_check_token_call(token)) {
+			token_cache[token] = PRESENT;
+		} else {
+			token_cache[token] = ABSENT;
+		}
+	}
+
+	return (token_cache[token] == PRESENT);
+}
  EXPORT_SYMBOL_GPL(opal_check_token);
 
  /* Convert a region of vmalloc memory to an opal sg list */
#3 lore raw
Quoted 6 lines · Nikhil Kumar Singh

You can wrap the if (token_cache[token] == SUPP_UNKNOWN) check, as well as the if (token > OPAL_LAST) check above it, with unlikely(), since this function is in the hot path. Out-of-bounds tokens are very rare, and after the cache is populated, the SUPP_UNKNOWN condition is expected to be false in almost all cases. Wrapping these checks with unlikely() will help the compiler optimize branch

Thanks for your suggestion, but Which hotpath code is calling this? we can maybe change that hotpath to not do rtas calls.

~Shivang.

#4 lore raw

I discussed this with Sourabh, and we confirmed this function is actually not on a hot path. However, he is open to the idea. Please take a look and see if adding unlikely() provides a meaningful benefit here, or if it falls into premature optimization.

Thanks
~ Nikhil

Quoted 7 lines · Shivang Upadhyay
Quoted 1 line · Nikhil Kumar Singh

You can wrap the if (token_cache[token] == SUPP_UNKNOWN) check, as well as the if (token > OPAL_LAST) check above it, with unlikely(), since this function is in the hot path. Out-of-bounds tokens are very rare, and after the cache is populated, the SUPP_UNKNOWN condition is expected to be false in almost all cases. Wrapping these checks with unlikely() will help the compiler optimize branch

Thanks for your suggestion, but Which hotpath code is calling this? we can maybe change that hotpath to not do rtas calls.

~Shivang.

Reviewed-by on v2 1/1
#5 lore raw
In reply to Shivang Upadhyay
Quoted 11 lines · Shivang Upadhyay

opal_check_token() function is used to determine if a specific OPAL firmware call is supported on the current platform. This check is performed frequently during boot and runtime, resulting in unnecessary firmware calls for the same token values.

Add a caching layer for the opal_check_token() OPAL call to avoid repeated firmware calls for token availability checks. This reduces firmware call overhead during boot.

Testing with buildroot images shows OPAL calls reduced from 35578 to 28983, before console bring-up.

nice optimisation !

Quoted 24 lines

<...snip...>
int64_t opal_xscom_read(uint32_t gcid, uint64_t pcb_addr, __be64 *val);

arch/powerpc/platforms/powernv/opal-call.c +1 -1
index 021b0ec29e24..00325c189e69 100644
@@ -207,7 +207,7 @@ OPAL_CALL(opal_validate_flash,			OPAL_FLASH_VALIDATE);
 OPAL_CALL(opal_manage_flash,			OPAL_FLASH_MANAGE);
 OPAL_CALL(opal_update_flash,			OPAL_FLASH_UPDATE);
 OPAL_CALL(opal_resync_timebase,			OPAL_RESYNC_TIMEBASE);
-OPAL_CALL(opal_check_token,			OPAL_CHECK_TOKEN);
+OPAL_CALL(opal_check_token_call,		OPAL_CHECK_TOKEN);
 OPAL_CALL(opal_dump_init,			OPAL_DUMP_INIT);
 OPAL_CALL(opal_dump_info,			OPAL_DUMP_INFO);
 OPAL_CALL(opal_dump_info2,			OPAL_DUMP_INFO2);
arch/powerpc/platforms/powernv/opal.c +0 -0
index 1946dbdc9fa1..1e9cb5271ee7 100644
@@ -1125,6 +1125,36 @@ EXPORT_SYMBOL_GPL(opal_flash_read);
 EXPORT_SYMBOL_GPL(opal_flash_write);
 EXPORT_SYMBOL_GPL(opal_flash_erase);
 EXPORT_SYMBOL_GPL(opal_prd_msg);

arch/powerpc/platforms/powernv/opal.c EXPORT_SYMBOL_GPL(opal_flash_read); · line 1137

Quoted 10 lines
arch/powerpc/platforms/powernv/opal.c +10 -0
+
+/**
+ * opal_check_token - Check if an OPAL call token is supported
+ * @token: OPAL token number to check
+ *
+ * Returns 1 if supported, 0 if not.
+ */
+int64_t opal_check_token(uint64_t token)
+{
+	static u8 token_cache[OPAL_LAST];

not a suggestion to change anything, just an observation:

based on this in skiboot:

```
void __opal_register(uint64_t token, void *func, unsigned int nargs)
{
	assert(token <= OPAL_LAST);
...

static int64_t opal_check_token(uint64_t token)
{
	if (token > OPAL_LAST)
		return OPAL_TOKEN_ABSENT;
```

along with other usages of OPAL_LAST in skiboot, OPAL_LAST is being treated as a valid token.

there are no handlers for OPAL_LAST and kernel doesn't do opal_check_token for this

I feel the logic should better be fixed in opal to not consider OPAL_LAST

at the same time it maybe safer to have length as OPAL_LAST+1 here, though it is not necessary, so the patch looks good to me

Thanks,
- Aditya G

#6 lore raw

Hi Aditya,

Thanks for the review.

Quoted 15 lines · Aditya Gupta

not a suggestion to change anything, just an observation:

based on this in skiboot:

```
void __opal_register(uint64_t token, void *func, unsigned int nargs)
{
	assert(token <= OPAL_LAST);
...

static int64_t opal_check_token(uint64_t token)
{
	if (token > OPAL_LAST)
		return OPAL_TOKEN_ABSENT;
```
Quoted 9 lines

along with other usages of OPAL_LAST in skiboot, OPAL_LAST is being treated as a valid token.

there are no handlers for OPAL_LAST and kernel doesn't do opal_check_token for this

I feel the logic should better be fixed in opal to not consider OPAL_LAST

As per my current understanding, I assume OPAL_LAST is meant to be the length of all valid opal tokens, Not a valid call itself.

Quoted 5 lines

at the same time it maybe safer to have length as OPAL_LAST+1 here, though it is not necessary, so the patch looks good to me

Thanks.
~Shivang.

#7 lore raw
Quoted 23 lines · Shivang Upadhyay

Hi Aditya,

Thanks for the review.

Quoted 18 lines · Aditya Gupta

not a suggestion to change anything, just an observation:

based on this in skiboot:

```
void __opal_register(uint64_t token, void *func, unsigned int nargs)
{
	assert(token <= OPAL_LAST);
...

static int64_t opal_check_token(uint64_t token)
{
	if (token > OPAL_LAST)
		return OPAL_TOKEN_ABSENT;
```

along with other usages of OPAL_LAST in skiboot, OPAL_LAST is being treated as a valid token.

Quoted 8 lines
Quoted 6 lines

there are no handlers for OPAL_LAST and kernel doesn't do opal_check_token for this

I feel the logic should better be fixed in opal to not consider OPAL_LAST

As per my current understanding, I assume OPAL_LAST is meant to be the length of all valid opal tokens, Not a valid call itself.

I agree, hence the current implementation looks good to me.

Thanks,
- Aditya G