arm64: runtime-const: save one instruction when ARM64_VA_BITS <= 48

PATCH In review Jisheng Zhang 2 participants 3 messages Feb 25 – Mar 2 arm64

Share & tools

Jisheng Zhang

author

2026-02-25 15:04 UTC #0 lore raw

PATCH 1/1 arm64: runtime-const: save one instruction when ARM64_VA_BITS <= 48

Currently, the runtime_const_ptr() uses 4 instructions to move a long imm to GP, but when ARM64_VA_BITS <= 48(which is true for android and armbian), the top 8bits of runtime cont ptr is all '1', so we can make use of the movn instruction to construct the imm's top 8bits and lower 16bits at the same time, thus save one instruction.

Signed-off-by Jisheng Zhang <[email protected]>

---
 arch/arm64/include/asm/runtime-const.h | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)

arch/arm64/include/asm/runtime-const.h +26 -0

index be5915669d23..6797dd37d690 100644
@@ -7,6 +7,8 @@
 /* Sigh. You can still run arm64 in BE mode */
 #include <asm/byteorder.h>
 
+#if CONFIG_ARM64_VA_BITS > 48
+
 #define runtime_const_ptr(sym) ({				\
 	typeof(sym) __ret;					\
 	asm_inline("1:\t"					\

@@ -20,6 +22,22 @@
 		:"=r" (__ret));					\
 	__ret; })
 
+#else
+
+#define runtime_const_ptr(sym) ({				\
+	typeof(sym) __ret;					\
+	asm_inline("1:\t"					\
+		"movn %0, #0x3210\n\t"				\
+		"movk %0, #0x89ab, lsl #16\n\t"			\
+		"movk %0, #0x4567, lsl #32\n\t"			\
+		".pushsection runtime_ptr_" #sym ",\"a\"\n\t"	\
+		".long 1b - .\n\t"				\
+		".popsection"					\
+		: "=r" (__ret));				\
+	__ret; })
+
+#endif
+
 #define runtime_const_shift_right_32(val, sym) ({		\
 	unsigned long __ret;					\
 	asm_inline("1:\t"					\

@@ -58,11 +76,19 @@ static inline void __runtime_fixup_caches(void *where, unsigned int insns)
 static inline void __runtime_fixup_ptr(void *where, unsigned long val)
 {
 	__le32 *p = lm_alias(where);
+#if CONFIG_ARM64_VA_BITS > 48
 	__runtime_fixup_16(p, val);
+#else
+	__runtime_fixup_16(p, ~val);
+#endif
 	__runtime_fixup_16(p+1, val >> 16);
 	__runtime_fixup_16(p+2, val >> 32);
+#if CONFIG_ARM64_VA_BITS > 48
 	__runtime_fixup_16(p+3, val >> 48);
 	__runtime_fixup_caches(where, 4);
+#else
+	__runtime_fixup_caches(where, 3);
+#endif
 }
 
 /* Immediate value is 6 bits starting at bit #16 */

Signature · 1 line

2.51.0

Catalin Marinas

maintainer

2026-02-27 16:34 UTC #1 lore raw

Quoted 3 lines · Jisheng Zhang

Currently, the runtime_const_ptr() uses 4 instructions to move a long imm to GP, but when ARM64_VA_BITS <= 48(which is true for android and armbian), the top 8bits of runtime cont ptr is all '1', so we can make

                    ^^^^^
                    8 or 16?

Quoted 2 lines

use of the movn instruction to construct the imm's top 8bits and lower 16bits at the same time, thus save one instruction.

This works as long as KASAN_{SW,HW}_TAGS is disabled, otherwise the top byte of a pointer is not guaranteed to be 0xff. I think both filename_init() and dcache_init() can pass tagged pointers.

arch/arm64/include/asm/runtime-const.h line 10

Quoted 9 lines

arch/arm64/include/asm/runtime-const.h +1 -0

index be5915669d23..6797dd37d690 100644
@@ -7,6 +7,8 @@
 /* Sigh. You can still run arm64 in BE mode */
 #include <asm/byteorder.h>
 
+#if CONFIG_ARM64_VA_BITS > 48

You could use VA_BITS, it's shorter, though if you add the KASAN checks it's a pretty long #if to copy all over the place. We could untag the pointer but it kind of defeats the purpose of enabling KASAN in the first place.

Given that Android enables KASAN_HW_TAGS by default, not sure we should bother with this change. Do you have any perf data to show that it's worth it?

Signature · 1 line

Catalin

Jisheng Zhang

author

2026-03-02 16:26 UTC #2 lore raw

Quoted 6 lines · Catalin Marinas

Quoted 3 lines · Jisheng Zhang
Currently, the runtime_const_ptr() uses 4 instructions to move a long imm to GP, but when ARM64_VA_BITS <= 48(which is true for android and armbian), the top 8bits of runtime cont ptr is all '1', so we can make
                    ^^^^^
                    8 or 16?

16 ;)

Quoted 7 lines

Quoted 3 lines
use of the movn instruction to construct the imm's top 8bits and lower 16bits at the same time, thus save one instruction.
This works as long as KASAN_{SW,HW}_TAGS is disabled, otherwise the top byte of a pointer is not guaranteed to be 0xff. I think both filename_init() and dcache_init() can pass tagged pointers.

oops, you are right! I missed both: KASAN_SW_TAGS is disabled due to overhead while KASAN_HW_TAGS doesn't work since I don't have the platform. Will take care these two options in the future.

Quoted 9 lines

arch/arm64/include/asm/runtime-const.h +0 -0

index be5915669d23..6797dd37d690 100644
@@ -7,6 +7,8 @@
 /* Sigh. You can still run arm64 in BE mode */
 #include <asm/byteorder.h>

Quoted 6 lines

arch/arm64/include/asm/runtime-const.h line 10
Quoted 1 line
arch/arm64/include/asm/runtime-const.h +1 -0
+#if CONFIG_ARM64_VA_BITS > 48
You could use VA_BITS, it's shorter, though if you add the KASAN checks it's a pretty long #if to copy all over the place. We could untag the pointer but it kind of defeats the purpose of enabling KASAN in the first place.

Usually, the runtime const ptr is set once during boot then read onlly so IMHO we don't need KASAN to catch the ptr related memory bugs.

Quoted 4 lines

Given that Android enables KASAN_HW_TAGS by default, not sure we should bother with this change. Do you have any perf data to show that it's worth it?

Good question. I guess a micro benchmark just measure the 4 instructions vs 3 instructions thus 25% saving can't persuade you to merge it. Let me find or write a userspace program to iterate a deep directory to show the improvement. Any hint is appreciated.

Thanks