ARM: imx: Fix suspend/resume crash with Clang CFI
PATCH v3 1/1 ARM: imx: Fix suspend/resume crash with Clang CFI
Relocated suspend code in OCRAM lacks compiler-generated CFI type signatures. When CONFIG_CFI=y is active, the indirect call to imx6_suspend_in_ocram_fn triggers a strict CFI violation panic.
To resolve this safely without reducing CFI protection scope:
1. Create a minimal wrapper function imx6_suspend_in_ocram annotated with __nocfi to handle the unverified indirect call.
2. Remove the __nocfi annotation from the main imx6q_suspend_finish function to preserve full CFI coverage for other indirect calls in that scope (such as cpu_do_idle() and flush_cache_all()).
3. Mark global variables ccm_base, suspend_ocram_base, and the imx6_suspend_in_ocram_fn pointer as __ro_after_init to prevent them from being used as target vectors for CFI bypass exploits.
---
Tested on a Kobo Clara HD (i.MX6SLL SoC) running postmarketOS edge. Before this patch, suspending the device caused an immediate silent hang requiring a hard-reboot. With this patch applied, suspend and resume work successfully.
Differences from v2:
- Restrained __nocfi scope by adding a dedicated, minimal 1-line wrapper function (imx6_suspend_in_ocram) for the OCRAM call, avoiding disabling CFI checks for cpu_do_idle() and flush_cache_all().
- Marked global pointers ccm_base and suspend_ocram_base as __ro_after_init to fully neutralize Write-What-Where exploit bypasses.
arch/arm/mach-imx/pm-imx6.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-)
arch/arm/mach-imx/pm-imx6.c +9 -4
@@ -61,9 +61,9 @@#define MX6Q_SUSPEND_OCRAM_SIZE 0x1000#define MX6_MAX_MMDC_IO_NUM 33 -static void __iomem *ccm_base; -static void __iomem *suspend_ocram_base; -static void (*imx6_suspend_in_ocram_fn)(void __iomem *ocram_vbase); +static void __iomem *ccm_base__ro_after_init ; +static void __iomem *suspend_ocram_base__ro_after_init ; +static void (*imx6_suspend_in_ocram_fn)(void __iomem *ocram_vbase)__ro_after_init ;/* * suspend ocram space layout:
@@ -360,6 +360,11 @@ int imx6_set_lpm(enum mxc_cpu_pwr_mode mode)return 0 ; } +static void __nocfi imx6_suspend_in_ocram(void __iomem *ocram_vbase) +{ +imx6_suspend_in_ocram_fn (ocram_vbase); +} +static int imx6q_suspend_finish (unsigned long val) {if (!imx6_suspend_in_ocram_fn) {
@@ -374,7 +379,7 @@ static int imx6q_suspend_finish(unsigned long val)if (!((struct imx6_cpu_pm_info *) suspend_ocram_base)->l2_base.vbase)flush_cache_all (); -imx6_suspend_in_ocram_fn (suspend_ocram_base); +imx6_suspend_in_ocram (suspend_ocram_base); }return 0 ;
Signature · 1 line
2.55.0
[email protected]
AI reviewThank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [Critical] W^X Violation / Exploit Mitigation Bypass via Writable OCRAM Alias --
--- Patch [1]: [PATCH v3] ARM: imx: Fix suspend/resume crash with Clang CFI --- commit b6dc32274a7f40075998a9c67f552a3982169e81
Author: Yo'av Moshe <[email protected]>
ARM: imx: Fix suspend/resume crash with Clang CFI
This commit fixes a suspend/resume crash caused by CFI violations on indirect calls to relocated OCRAM code. It introduces a minimal __nocfi wrapper for the OCRAM call and marks relevant pointers as __ro_after_init to mitigate potential exploit risks.
Quoted 14 lines
arch/arm/mach-imx/pm-imx6.c +3
-3@@ -61,9 +61,9 @@#define MX6Q_SUSPEND_OCRAM_SIZE 0x1000#define MX6_MAX_MMDC_IO_NUM 33 -static void __iomem *ccm_base; -static void __iomem *suspend_ocram_base; -static void (*imx6_suspend_in_ocram_fn)(void __iomem *ocram_vbase); +static void __iomem *ccm_base__ro_after_init ; +static void __iomem *suspend_ocram_base__ro_after_init ; +static void (*imx6_suspend_in_ocram_fn)(void __iomem *ocram_vbase)__ro_after_init ;
[ ... ]
arch/arm/mach-imx/pm-imx6.c int imx6_set_lpm(enum mxc_cpu_pwr_mode mode) · line 366
Quoted 8 lines
arch/arm/mach-imx/pm-imx6.c +4
-0@@ -360,6 +360,11 @@ int imx6_set_lpm(enum mxc_cpu_pwr_mode mode)return 0 ; } +static void __nocfi imx6_suspend_in_ocram(void __iomem *ocram_vbase) +{ +imx6_suspend_in_ocram_fn (ocram_vbase); +}
[Severity: Critical]
This is a pre-existing issue, but does adding __ro_after_init fully protect this memory from write-what-where attacks?
Looking at imx6q_suspend_init(), physical OCRAM memory is mapped with both a persistent writable alias (ocram_base) and an executable alias (suspend_ocram_base):
arch/arm/mach-imx/pm-imx6.c:imx6q_suspend_init() {
ocram_base = gen_pool_alloc(ocram_pool, MX6Q_SUSPEND_OCRAM_SIZE);
...
ocram_pbase = gen_pool_virt_to_phys(ocram_pool, ocram_base);
suspend_ocram_base = __arm_ioremap_exec(ocram_pbase,
MX6Q_SUSPEND_OCRAM_SIZE, false);
...
__arm_iomem_set_ro(suspend_ocram_base, MX6Q_SUSPEND_OCRAM_SIZE);
}Even though suspend_ocram_base is marked read-only and its pointer is now __ro_after_init, the initial writable alias (ocram_base) appears to remain fully active without being unmapped.
Since this patch introduces a __nocfi wrapper to execute this memory, could an attacker still bypass CFI by writing shellcode directly into the active writable ocram_base alias?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Quoted 38 lines · Yo'av Moshe
Relocated suspend code in OCRAM lacks compiler-generated CFI type signatures. When
CONFIG_CFI=yis active, the indirect call toimx6_suspend_in_ocram_fntriggers a strict CFI violation panic.To resolve this safely without reducing CFI protection scope:
1. Create a minimal wrapper functionimx6_suspend_in_ocramannotated with__nocfito handle the unverified indirect call.
2. Remove the__nocfiannotation from the mainimx6q_suspend_finishfunction to preserve full CFI coverage for other indirect calls in that scope (such ascpu_do_idle()andflush_cache_all()).
3. Mark global variablesccm_base,suspend_ocram_base, and theimx6_suspend_in_ocram_fnpointer as__ro_after_initto prevent them from being used as target vectors for CFI bypass exploits.Signed-off-by Yo'av Moshe <[email protected]>---
Tested on a Kobo Clara HD (i.MX6SLL SoC) running postmarketOS edge. Before this patch, suspending the device caused an immediate silent hang requiring a hard-reboot. With this patch applied, suspend and resume work successfully.Differences from v2:
- Restrained__nocfiscope by adding a dedicated, minimal 1-line wrapper function (imx6_suspend_in_ocram) for the OCRAM call, avoiding disabling CFI checks forcpu_do_idle()andflush_cache_all().
- Marked global pointersccm_baseandsuspend_ocram_baseas__ro_after_initto fully neutralize Write-What-Where exploit bypasses.arch/arm/mach-imx/pm-imx6.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-)arch/arm/mach-imx/pm-imx6.c +0
-0
arch/arm/mach-imx/pm-imx6.c line 66
Quoted 9 lines
arch/arm/mach-imx/pm-imx6.c +3
-3@@ -61,9 +61,9 @@#define MX6Q_SUSPEND_OCRAM_SIZE 0x1000#define MX6_MAX_MMDC_IO_NUM 33 -static void __iomem *ccm_base; -static void __iomem *suspend_ocram_base; -static void (*imx6_suspend_in_ocram_fn)(void __iomem *ocram_vbase); +static void __iomem *ccm_base__ro_after_init ; +static void __iomem *suspend_ocram_base__ro_after_init ; +static void (*imx6_suspend_in_ocram_fn)(void __iomem *ocram_vbase)__ro_after_init ;
Are we able to just put __nocfi on the declaration of imx6_suspend_in_ocram_fn, rather than bother with a wrapper (imx6_suspend_in_ocram)? I don't know if that works, but surely you can test that quickly?
Quoted 27 lines
arch/arm/mach-imx/pm-imx6.c +6
-2/* * suspend ocram space layout: @@ -360,6 +360,11 @@ int imx6_set_lpm(enum mxc_cpu_pwr_mode mode)return 0 ; } +static void __nocfi imx6_suspend_in_ocram(void __iomem *ocram_vbase) +{ +imx6_suspend_in_ocram_fn (ocram_vbase); +} +static int imx6q_suspend_finish (unsigned long val) {if (!imx6_suspend_in_ocram_fn) {@@ -374,7 +379,7 @@ static int imx6q_suspend_finish(unsigned long val)if (!((struct imx6_cpu_pm_info *) suspend_ocram_base)->l2_base.vbase)flush_cache_all (); -imx6_suspend_in_ocram_fn (suspend_ocram_base); +imx6_suspend_in_ocram (suspend_ocram_base); }return 0 ; --2.55.0
Signature · 2 lines
Thanks, ~Nick Desaulniers
Quoted 5 lines · Nick Desaulniers
Are we able to just put
__nocfion the declaration ofimx6_suspend_in_ocram_fn, rather than bother with a wrapper (imx6_suspend_in_ocram)? I don't know if that works, but surely you can test that quickly?
Thanks for the suggestion! I tested placing __nocfi directly on the imx6_suspend_in_ocram_fn variable declaration:
static void (* __nocfi imx6_suspend_in_ocram_fn)(void __iomem *ocram_vbase);
Unfortunately, Clang ignores no_sanitize("cfi") on variable declarations and emits a compiler warning:
warning: 'no_sanitize' attribute argument 'cfi' not supported on a global variable [-Wignored-attributes]
Because Clang ignores it, it still injects the CFI check at the call site. I tested this on physical hardware (Kobo Clara HD), and it crashes on suspend.
Best regards,
Yo'av
Quoted 4 lines · Yo'av Moshe
Relocated suspend code in OCRAM lacks compiler-generated CFI type signatures. When
CONFIG_CFI=yis active, the indirect call toimx6_suspend_in_ocram_fntriggers a strict CFI violation panic.
Would it be possible to just copy the 4-byte CFI hash prefix to OCRAM when relocating the function? If not, the __nocfi approach seems reasonable to me.
Sami
Quoted 5 lines · Sami Tolvanen
Would it be possible to just copy the 4-byte CFI hash prefix to OCRAM when relocating the function? If not, the
__nocfiapproach seems reasonable to me.Sami
I gave this a try - I tried copying 4 bytes from before imx6_suspend into OCRAM, but when I tested it on physical hardware (Kobo Clara HD), it still crashed on suspend.
I suspect it's because imx6_suspend is written in assembly (suspend-imx6.S) rather than C, so Clang doesn't emit a CFI hash prefix before it in the first place.
Best regards,
Yo'av
Quoted 7 lines · Yo'av Moshe
Quoted 5 lines · Sami Tolvanen
Would it be possible to just copy the 4-byte CFI hash prefix to OCRAM when relocating the function? If not, the
__nocfiapproach seems reasonable to me.Sami
Quoted 7 lines
I gave this a try - I tried copying 4 bytes from before
imx6_suspendinto OCRAM, but when I tested it on physical hardware (Kobo Clara HD), it still crashed on suspend.I suspect it's because
imx6_suspendis written in assembly (suspend-imx6.S) rather than C, so Clang doesn't emit a CFI hash prefix before it in the first place.
Does using SYM_TYPED_FUNC_START for imx6_suspend() make that work? Something like this builds fine for me and I see __kcfi_typeid_imx6_suspend generated by Clang.
arch/arm/mach-imx/suspend-imx6.S +3 -2
@@ -3,6 +3,7 @@ * Copyright 2014 Freescale Semiconductor, Inc. */ +#include <linux/cfi_types.h> #include <linux/linkage.h> #include <asm/assembler.h> #include <asm/asm-offsets.h>
@@ -148,7 +149,7 @@ .endm -ENTRY(imx6_suspend) +SYM_TYPED_FUNC_START(imx6_suspend) ldr r1, [r0, #PM_INFO_PBASE_OFFSET] ldr r2, [r0, #PM_INFO_RESUME_ADDR_OFFSET] ldr r3, [r0, #PM_INFO_DDR_TYPE_OFFSET]
@@ -329,4 +330,4 @@ resume: resume_mmdc ret lr -ENDPROC(imx6_suspend) +SYM_FUNC_END(imx6_suspend)
Signature · 2 lines
Cheers, Nathan
Quoted 3 lines · Nathan Chancellor
Does using
SYM_TYPED_FUNC_STARTforimx6_suspend()make that work? Something like this builds fine for me and I see__kcfi_typeid_imx6_suspendgenerated by Clang.
Hi Nathan,
Thanks for the suggestion!
Unfortunately my physical board died, so I couldn't test this on hardware. However, I gave your patch a try under QEMU emulation (with CONFIG_CFI=y and copying the 4-byte hash into OCRAM before fncpy), and it still triggered a CFI panic when suspending:
__und_svc_finish from imx6q_suspend_finish+0x78/0xd0
It seems like the hash generated for assembly doesn't match what the C code expects at the call site.
Best regards,
Yo'av
Quoted 14 lines · Yo'av Moshe
Quoted 3 lines · Nathan Chancellor
Does using
SYM_TYPED_FUNC_STARTforimx6_suspend()make that work? Something like this builds fine for me and I see__kcfi_typeid_imx6_suspendgenerated by Clang.Hi Nathan,
Thanks for the suggestion!
Unfortunately my physical board died, so I couldn't test this on hardware. However, I gave your patch a try under QEMU emulation (with
CONFIG_CFI=yand copying the 4-byte hash into OCRAM before fncpy), and it still triggered a CFI panic when suspending:
Quoted 4 lines
__und_svc_finish from imx6q_suspend_finish+0x78/0xd0It seems like the hash generated for assembly doesn't match what the C code expects at the call site.
Are relocations applied to this code before the function is copied? Because SYM_TYPED_FUNC_START doesn't embed the actual CFI hash in the binary, just a reference to a symbol that contains the hash, and it relies on the kernel to apply relocations before indirect calls are made to the function.
Sami
Quoted 5 lines · Sami Tolvanen
Are relocations applied to this code before the function is copied? Because
SYM_TYPED_FUNC_STARTdoesn't embed the actual CFI hash in the binary, just a reference to a symbol that contains the hash, and it relies on the kernel to apply relocations before indirect calls are made to the function.
Hi Sami,
AFAICT, kernel relocations are already applied at boot before fncpy() is called during init.
I think the issue might be a parameter mismatch: in C, the pointer is declared as taking a parameter void (*)(void __iomem *), but in assembly imx6_suspend has no parameters, so Clang generates two different hashes for them?
Best regards,
Yo'av
Quoted 5 lines · Yo'av Moshe
I think the issue might be a parameter mismatch: in C, the pointer is declared as taking a parameter
void (*)(void __iomem *), but in assemblyimx6_suspendhas no parameters, so Clang generates two different hashes for them?
As long as the C prototype for the assembly function matches the function pointer type, the hashes should also match. Can you dump the hash prefix from the OCRAM and compare it to the disassembly of the call site?
Sami
Quoted 4 lines · Sami Tolvanen
As long as the C prototype for the assembly function matches the function pointer type, the hashes should also match. Can you dump the hash prefix from the OCRAM and compare it to the disassembly of the call site?
Here is what I saw when checking the disassembly in vmlinux:
1. Call site expected hash in pm-imx6.c: 0xA488EBFC
2. SYM_TYPED_FUNC_START hash prefix in suspend-imx6.S: 0xd4d4d4d4
It looks like because suspend-imx6.S is an assembly file, Clang emits the default assembly type ID (0xd4d4d4d4) for it, which doesn't match the C call site expectation (0xA488EBFC).
Yo'av
Quoted 10 lines · Yo'av Moshe
Quoted 4 lines · Sami Tolvanen
As long as the C prototype for the assembly function matches the function pointer type, the hashes should also match. Can you dump the hash prefix from the OCRAM and compare it to the disassembly of the call site?
Here is what I saw when checking the disassembly in vmlinux:
1. Call site expected hash in
pm-imx6.c:0xA488EBFC
2.SYM_TYPED_FUNC_STARThash prefix insuspend-imx6.S:0xd4d4d4d4
What's the value for the __kcfi_typeid_imx6_suspend symbol in vmlinux.o? I would expect the hash value to look a bit more random.
Quoted 3 lines
It looks like because
suspend-imx6.Sis an assembly file, Clang emits the default assembly type ID (0xd4d4d4d4) for it, which doesn't match the C call site expectation (0xA488EBFC).
Clang doesn't actually emit type hashes for assembly functions. SYM_TYPED_FUNC_START just adds a reference to the __kcfi_typeid_<functionname> symbol that contains a hash calculated for the C prototype.
Sami
Quoted 2 lines · Sami Tolvanen
What's the value for the
__kcfi_typeid_imx6_suspendsymbol in vmlinux.o? I would expect the hash value to look a bit more random.
Checking vmlinux.o with llvm-readelf -s vmlinux.o indeed shows:
a488ebfc 0 NOTYPE WEAK DEFAULT ABS __kcfi_typeid_imx6_suspend
Yo'av
Quoted 7 lines · Yo'av Moshe
Quoted 2 lines · Sami Tolvanen
What's the value for the
__kcfi_typeid_imx6_suspendsymbol in vmlinux.o? I would expect the hash value to look a bit more random.Checking vmlinux.o with
llvm-readelf -s vmlinux.oindeed shows:
a488ebfc0 NOTYPE WEAK DEFAULT ABS__kcfi_typeid_imx6_suspend
OK, so the hashes should match. I can only assume that the hash prefix doesn't end up getting copied to OCRAM for some reason then, or perhaps the source of the copy operation doesn't have relocations applied? Either way, if this turns out to be infeasible, I think the __nocfi approach is reasonable in this case too.
Sami
Quoted 4 lines · Sami Tolvanen
Quoted 2 lines · Yo'av Moshe
On 2026-07-23 9:21 PM, Sami Tolvanen wrote:
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
What's the value for the
__kcfi_typeid_imx6_suspendsymbol in vmlinux.o? I would expect the hash value to look a bit more random.Checking vmlinux.o with
llvm-readelf -s vmlinux.oindeed shows:
a488ebfc0 NOTYPE WEAK DEFAULT ABS__kcfi_typeid_imx6_suspendOK, so the hashes should match. I can only assume that the hash prefix doesn't end up getting copied to OCRAM for some reason then, or
Yeah, Yo'av did you test further modifications that explicitly copy the hash prefix? I assumed Nathan's change _plus some additional tweak to copy not just imx6_suspend to OCRAM which I assume the code already does, but I would imagine you need another change to copy __kcfi_typeid_imx6_suspend into OCRAM just before the function. Was that tested?
Quoted 5 lines
perhaps the source of the copy operation doesn't have relocations applied? Either way, if this turns out to be infeasible, I think the
__nocfiapproach is reasonable in this case too.Sami
Signature · 2 lines
Thanks, ~Nick Desaulniers
Quoted 6 lines · Nick Desaulniers
Yeah, Yo'av did you test further modifications that explicitly copy the hash prefix? I assumed Nathan's change
_plussome additional tweak to copy not justimx6_suspendto OCRAM which I assume the code already does, but I would imagine you need another change to copy__kcfi_typeid_imx6_suspendinto OCRAM just before the function. Was that tested?
Yes, I tried that as well and it unfortunately didn't work.
I tested Nathan's change along with explicitly copying the 4-byte prefix from before imx6_suspend into OCRAM right before calling fncpy(). However, when checking the compiled binary, "&imx6_suspend - 4" just contained padding rather than the hash, so the copied 4-byte value didn't match what the call site expected.
Yo'av
Quoted 7 lines · Yo'av Moshe
Yes, I tried that as well and it unfortunately didn't work.
I tested Nathan's change along with explicitly copying the 4-byte prefix from before
imx6_suspendinto OCRAM right before callingfncpy(). However, when checking the compiled binary, "&imx6_suspend- 4" just contained padding rather than the hash, so the copied 4-byte value didn't match what the call site expected.
Hi Sascha, Frank,
Gentle ping on this patch. Is there anything else needed from my end?
Best regards,
Yo'av
Quoted 11 lines · Yo'av Moshe
Quoted 7 lines · Yo'av Moshe
Yes, I tried that as well and it unfortunately didn't work.
I tested Nathan's change along with explicitly copying the 4-byte prefix from before
imx6_suspendinto OCRAM right before callingfncpy(). However, when checking the compiled binary, "&imx6_suspend- 4" just contained padding rather than the hash, so the copied 4-byte value didn't match what the call site expected.Hi Sascha, Frank,
Gentle ping on this patch. Is there anything else needed from my end?
There are long discussion on this thread.
Sami Tolvanen and Nick Desaulniers:
Do you agree on Yo's implement?
Frank
Quoted 4 lines
Best regards,
Yo'av
Quoted 8 lines · Yo'av Moshe
Quoted 6 lines · Nick Desaulniers
Yeah, Yo'av did you test further modifications that explicitly copy the hash prefix? I assumed Nathan's change
_plussome additional tweak to copy not justimx6_suspendto OCRAM which I assume the code already does, but I would imagine you need another change to copy__kcfi_typeid_imx6_suspendinto OCRAM just before the function. Was that tested?
Quoted 4 lines
Yes, I tried that as well and it unfortunately didn't work.
I tested Nathan's change along with explicitly copying the 4-byte prefix from before
imx6_suspendinto OCRAM right before callingfncpy().
Can you share that diff? I would have expected that to work. Perhaps a minor mistake in your implementation?
Quoted 3 lines
However, when checking the compiled binary, "
&imx6_suspend- 4" just contained padding rather than the hash, so the copied 4-byte value didn't match what the call site expected.
How did you verify this? Can you share the command line invocations and output?
Quoted 2 lines
Yo'av
Signature · 2 lines
Thanks, ~Nick Desaulniers
Sorry it took so long, I finally got it to work using your method!
Quoted 2 lines · Nick Desaulniers
Can you share that diff? I would have expected that to work. Perhaps a minor mistake in your implementation?
I started with something very simple: using SYM_TYPED_FUNC_START and copying the 4-byte hash into OCRAM before fncpy(). But that didn't boot on my hardware (Kobo Clara HD, i.MX6SLL).
I think the issues were:
1. My first attempt wrote the hash to suspend_ocram_base + sizeof(*pm_info) - 4, which I suspect overwrote the last member of struct imx6_cpu_pm_info, corrupting the memory controller setup.
2. Moving the hash after pm_info and offsetting fncpy by +4 still failed — I think because fncpy requires 8-byte aligned source and destination addresses (FNCPY_ALIGN, with a BUG_ON check).
3. Offsetting by +8 for alignment also failed. I suspect it is because SYM_TYPED_FUNC_START emits the 4-byte hash after the .align directive, which shifts the imx6_suspend label out of 8-byte alignment, again triggering fncpy's source alignment BUG_ON.
The version that finally boots on hardware emits the hash manually with explicit alignment padding, instead of using SYM_TYPED_FUNC_START:
arch/arm/mach-imx/pm-imx6.c +9 -3
@@ -515,7 +515,7 @@ static int __init imx6q_suspend_init(const struct imx6_pm_socdata *socdata) pm_info = suspend_ocram_base; pm_info->pbase = ocram_pbase; pm_info->resume_addr =__pa_symbol (v7_cpu_resume); - pm_info->pm_info_size =sizeof (*pm_info); + pm_info->pm_info_size =sizeof (*pm_info) +8 ;/* * ccm physical address is not used by asm code currently,
@@ -568,10 +568,16 @@ static int __init imx6q_suspend_init(const struct imx6_pm_socdata *socdata) mmdc_offset_array[i]); } +/* Reserve 8 bytes between pm_info and the function copy in OCRAM: +* 4 bytes padding + 4 bytes kCFI type hash, so that the hash sits +* at fncpy_dest - 4 and fncpy_dest remains 8-byte aligned. */ + *(u32 *)(suspend_ocram_base +sizeof (*pm_info) +4 ) = + *(((u32 *)&imx6_suspend) -1 ); + imx6_suspend_in_ocram_fn =fncpy ( - suspend_ocram_base +sizeof (*pm_info), + suspend_ocram_base +sizeof (*pm_info) +8 , &imx6_suspend, -MX6Q_SUSPEND_OCRAM_SIZE -sizeof (*pm_info)); +MX6Q_SUSPEND_OCRAM_SIZE -sizeof (*pm_info) -8 );__arm_iomem_set_ro (suspend_ocram_base,MX6Q_SUSPEND_OCRAM_SIZE );
arch/arm/mach-imx/suspend-imx6.S +10 -0
@@ -3,6 +3,7 @@ * Copyright 2014 Freescale Semiconductor, Inc. */ +#include <linux/cfi_types.h> #include <linux/linkage.h> #include <asm/assembler.h> #include <asm/asm-offsets.h>
@@ -148,6 +149,15 @@ .endm +#ifdef CONFIG_CFI + /* + * Emit kCFI type hash before imx6_suspend with padding to preserve + * the 8-byte alignment that fncpy requires for the source address. + */ + .align 3 + .4byte 0 + __CFI_TYPE(imx6_suspend) +#endif ENTRY(imx6_suspend) ldr r1, [r0, #PM_INFO_PBASE_OFFSET] ldr r2, [r0, #PM_INFO_RESUME_ADDR_OFFSET]
Quoted 1 line
How did you verify this? Can you share the command line invocations and output?
I tested each iteration on postmarketOS by building the kernel with pmbootstrap (Clang/LLVM, CONFIG_CFI=y), replacing the vmlinuz on the SD card, and booting the Kobo Clara HD. The earlier attempts all failed to boot (though they worked fine in QEMU, which doesn't emulate the i.MX6 MMDC hardware I guess?).
I'm not really sure which approach is better now. The version above preserves CFI on the indirect call, but it's quite involved compared to the v3 __nocfi wrapper. If you think this is the better way to go, I'm happy to clean it up and resubmit as v4.
Yo'av
Quoted 2 lines · Yo'av Moshe
Sorry it took so long, I finally got it to work using your method!
Great! (It's somewhat funny to me the level of effort going into this; I love it. Keep up the good work). It's also good to hear that we can keep this working _with_ CFI still enabled.
Quoted 29 lines · Nick Desaulniers
Quoted 2 lines
Can you share that diff? I would have expected that to work. Perhaps a minor mistake in your implementation?
I started with something very simple: using
SYM_TYPED_FUNC_STARTand copying the 4-byte hash into OCRAM beforefncpy(). But that didn't boot on my hardware (Kobo Clara HD, i.MX6SLL).I think the issues were:
1. My first attempt wrote the hash tosuspend_ocram_base+sizeof(*pm_info)- 4, which I suspect overwrote the last member of structimx6_cpu_pm_info, corrupting the memory controller setup.
2. Moving the hash afterpm_infoand offsetting fncpy by +4 still failed — I think because fncpy requires 8-byte aligned source and destination addresses (FNCPY_ALIGN, with aBUG_ONcheck).
3. Offsetting by +8 for alignment also failed. I suspect it is becauseSYM_TYPED_FUNC_STARTemits the 4-byte hash after the .align directive, which shifts theimx6_suspendlabel out of 8-byte alignment, again triggering fncpy's source alignmentBUG_ON.The version that finally boots on hardware emits the hash manually with explicit alignment padding, instead of using
SYM_TYPED_FUNC_START:arch/arm/mach-imx/pm-imx6.c +0
-0
arch/arm/mach-imx/pm-imx6.c static int __init imx6q_suspend_init(const struct imx6_pm_socdata *socdata) · line 518
Quoted 5 lines
arch/arm/mach-imx/pm-imx6.c +1
-1@@ -515,7 +515,7 @@ static int __init imx6q_suspend_init(const struct imx6_pm_socdata *socdata) pm_info = suspend_ocram_base; pm_info->pbase = ocram_pbase; pm_info->resume_addr =__pa_symbol (v7_cpu_resume); - pm_info->pm_info_size =sizeof (*pm_info); + pm_info->pm_info_size =sizeof (*pm_info) +8 ;
I wonder if it would be better to modify the struct imx6_cpu_pm_info struct definition instead to add explicit padding there? The comment below about reserving padding seems to make more sense for this hunk here than down there.
Like add a u32 cfi_type; member last, maybe? That's not 8 bytes, but I think it fits in the padding, then you might not need +/- 8 in your pointer arithmetic anymore? I _think_.
arch/arm/mach-imx/pm-imx6.c static int __init imx6q_suspend_init(const struct imx6_pm_socdata *socdata) · line 575
Quoted 12 lines
arch/arm/mach-imx/pm-imx6.c +5
-0/* * ccm physical address is not used by asm code currently, @@ -568,10 +568,16 @@ static int __init imx6q_suspend_init(const struct imx6_pm_socdata *socdata) mmdc_offset_array[i]); } +/* Reserve 8 bytes between pm_info and the function copy in OCRAM: +* 4 bytes padding + 4 bytes kCFI type hash, so that the hash sits +* at fncpy_dest - 4 and fncpy_dest remains 8-byte aligned. */ + *(u32 *)(suspend_ocram_base +sizeof (*pm_info) +4 ) = + *(((u32 *)&imx6_suspend) -1 );
This is incredibly hard to follow. This is copying the hash, not reserving anything?
Is cfi_get_func_hash() something we can use here?
```
#include <linux/cfi.h>
#ifdef CONFIG_CFI
// Make sure to clear the thumb bit!
pm_info->cfi_type = cfi_get_func_hash((void
*)((uintptr_t)&imx6_suspend & ~1));
#endif
```then remove the below hunk?
Quoted 26 lines
arch/arm/mach-imx/pm-imx6.c +3
-2+ imx6_suspend_in_ocram_fn =fncpy ( - suspend_ocram_base +sizeof (*pm_info), + suspend_ocram_base +sizeof (*pm_info) +8 , &imx6_suspend, -MX6Q_SUSPEND_OCRAM_SIZE -sizeof (*pm_info)); +MX6Q_SUSPEND_OCRAM_SIZE -sizeof (*pm_info) -8 );__arm_iomem_set_ro (suspend_ocram_base,MX6Q_SUSPEND_OCRAM_SIZE );arch/arm/mach-imx/suspend-imx6.S +1
-0@@ -3,6 +3,7 @@ * Copyright 2014 Freescale Semiconductor, Inc. */ +#include <linux/cfi_types.h> #include <linux/linkage.h> #include <asm/assembler.h> #include <asm/asm-offsets.h>@@ -148,6 +149,15 @@ .endm
arch/arm/mach-imx/suspend-imx6.S line 161
Quoted 10 lines
arch/arm/mach-imx/suspend-imx6.S +9
-0+#ifdef CONFIG_CFI + /* + * Emit kCFI type hash before imx6_suspend with padding to preserve + * the 8-byte alignment that fncpy requires for the source address. + */ + .align 3 + .4byte 0 + __CFI_TYPE(imx6_suspend) +#endif ENTRY(imx6_suspend)
Should you replace ENTRY (and everything you added to this file) with SYM_TYPED_START? Or perhaps we should add a new macro to cfi_types.h for this? ENTRY_TYPED or something?
Quoted 11 lines
arch/arm/mach-imx/suspend-imx6.S +0
-0ldr r1, [r0, #PM_INFO_PBASE_OFFSET] ldr r2, [r0, #PM_INFO_RESUME_ADDR_OFFSET]Quoted 3 lines
How did you verify this? Can you share the command line invocations and output?
I tested each iteration on postmarketOS by building the kernel with pmbootstrap (Clang/LLVM,
CONFIG_CFI=y), replacing the vmlinuz on the SD card, and booting the Kobo Clara HD. The earlier attempts all failed to boot (though they worked fine in QEMU, which doesn't emulate the i.MX6 MMDC hardware I guess?).
Quoted 4 lines
I'm not really sure which approach is better now. The version above preserves CFI on the indirect call, but it's quite involved compared to the v3
__nocfiwrapper. If you think this is the better way to go, I'm happy to clean it up and resubmit as v4.
I do think it's better, but probably still needs revision.
FWIW, I still have a bunch of i.mx8 family boards in a box under my desk. I have a wafer gifted to me from my best man that we think is some i.mx chip from his time at NXP. Out of nostalgia for that line, let's get this wrapped up neatly with a bow.
Quoted 2 lines
Yo'av
Signature · 2 lines
Thanks, ~Nick Desaulniers
Quoted 1 line · Nick Desaulniers
I do think it's better, but probably still needs revision.
All three suggestions worked great. cfi_type fits exactly in the struct's tail padding so I removed the ±8 arithmetic, cfi_get_func_hash() works as you suggested, and I've switched to SYM_TYPED_FUNC_START/SYM_FUNC_END. I only kept a small manual alignment pad before the macro, since it aligns the type hash but not the function entry itself — I'll include details in the v4 notes.
Yo'av
PATCH v4 1/1 ARM: imx: Fix suspend/resume crash with Clang CFI
In reply to Yo'av MosheThe suspend code that runs from OCRAM is copied there with fncpy(), which does not copy the kCFI type hash preceding the function. With CONFIG_CFI=y the indirect call through imx6_suspend_in_ocram_fn therefore panics.
Keep the call covered by CFI instead of exempting it:
- Declare imx6_suspend() with SYM_TYPED_FUNC_START() so a type hash is emitted for it. fncpy() requires the entry point to be 8-byte aligned, and the macro's alignment applies to the hash rather than the entry that follows it, so pad manually before the macro.
- Add a cfi_type member at the end of struct imx6_cpu_pm_info, which directly precedes the OCRAM copy of the function. It fits in the struct's tail padding, so no sizes or offsets change. Fill it using cfi_get_func_hash(), putting the hash where the caller's CFI check expects it: four bytes before the function entry.
Also mark ccm_base, suspend_ocram_base and imx6_suspend_in_ocram_fn as __ro_after_init: they are only written during __init, and the function pointer in particular should not be writable afterwards.
---
Tested on a Kobo Clara HD (i.MX6SLL) running postmarketOS (clang/LLVM, CONFIG_CFI=y): suspend and resume work.
Changes in v4, all following Nick's suggestions:
- Drop the v3 __nocfi wrapper; keep the indirect call CFI-checked.
- Store the hash in a cfi_type member in the struct's tail padding instead of open-coded pointer arithmetic.
- Use cfi_get_func_hash() instead of reading the hash manually.
- Use SYM_TYPED_FUNC_START()/SYM_FUNC_END() instead of ENTRY()/ ENDPROC() with a hand-rolled __CFI_TYPE.
Note: linux/uaccess.h is included before linux/cfi.h because cfi_get_func_hash() uses get_kernel_nofault() and cfi.h does not include uaccess.h itself.
arch/arm/mach-imx/pm-imx6.c | 25 ++++++++++++++++++++++--- arch/arm/mach-imx/suspend-imx6.S | 14 ++++++++++++-- 2 files changed, 34 insertions(+), 5 deletions(-)
arch/arm/mach-imx/pm-imx6.c +22 -3
@@ -4,6 +4,8 @@* Copyright 2011 Linaro Ltd. */ +#include <linux/uaccess.h> +#include <linux/cfi.h> #include <linux/clk/imx.h> #include <linux/delay.h> #include <linux/init.h>
@@ -61,9 +63,9 @@#define MX6Q_SUSPEND_OCRAM_SIZE 0x1000#define MX6_MAX_MMDC_IO_NUM 33 -static void __iomem *ccm_base; -static void __iomem *suspend_ocram_base; -static void (*imx6_suspend_in_ocram_fn)(void __iomem *ocram_vbase); +static void __iomem *ccm_base__ro_after_init ; +static void __iomem *suspend_ocram_base__ro_after_init ; +static void (*imx6_suspend_in_ocram_fn)(void __iomem *ocram_vbase)__ro_after_init ;/* * suspend ocram space layout:
@@ -229,8 +231,18 @@ struct imx6_cpu_pm_info {struct imx6_pm_base l2_base;u32 mmdc_io_num;/* Number of MMDC IOs which need saved/restored. */ u32 mmdc_io_val[MX6_MAX_MMDC_IO_NUM ][2 ];/* To save offset and value */ +u32 cfi_type;/* kCFI type hash of imx6_suspend() */ }__aligned(8) ; +/* +* The ocram copy of imx6_suspend() starts right after struct imx6_cpu_pm_info, +* and the CFI check on the indirect call reads the kCFI type hash from the +* four bytes preceding the function entry, so cfi_type must occupy the last +* four bytes of the struct, i.e. fit into its tail padding. +*/ +static_assert(offsetofend(struct imx6_cpu_pm_info, cfi_type) == + sizeof(struct imx6_cpu_pm_info)); +void imx6_set_int_mem_clk_lpm (bool enable) {u32 val =readl_relaxed (ccm_base +CGPR );
@@ -568,6 +580,13 @@ static int __init imx6q_suspend_init(const struct imx6_pm_socdata *socdata) mmdc_offset_array[i]); } +/* +* Mask out the Thumb bit, as cfi_get_func_hash() expects the +* function's actual start address. Returns 0 if CONFIG_CFI=n. +*/ + pm_info->cfi_type = +cfi_get_func_hash ((void *)((uintptr_t )&imx6_suspend & ~1UL )); + imx6_suspend_in_ocram_fn = fncpy( suspend_ocram_base + sizeof(*pm_info), &imx6_suspend,
arch/arm/mach-imx/suspend-imx6.S +12 -2
@@ -3,6 +3,7 @@ * Copyright 2014 Freescale Semiconductor, Inc. */ +#include <linux/cfi_types.h> #include <linux/linkage.h> #include <asm/assembler.h> #include <asm/asm-offsets.h>
@@ -148,7 +149,16 @@ .endm -ENTRY(imx6_suspend) +#ifdef CONFIG_CFI + /* + * Pad the location counter so that the type hash emitted by + * SYM_TYPED_FUNC_START() below ends on an 8-byte boundary: + * fncpy() requires the function entry to be 8-byte aligned. + */ + .align 3 + .4byte 0 +#endif +SYM_TYPED_FUNC_START(imx6_suspend) ldr r1, [r0, #PM_INFO_PBASE_OFFSET] ldr r2, [r0, #PM_INFO_RESUME_ADDR_OFFSET] ldr r3, [r0, #PM_INFO_DDR_TYPE_OFFSET]
@@ -329,4 +339,4 @@ resume: resume_mmdc ret lr -ENDPROC(imx6_suspend) +SYM_FUNC_END(imx6_suspend)
Signature · 1 line
2.55.0
Quoted 7 lines · Yo'av Moshe
The suspend code that runs from OCRAM is copied there with
fncpy(), which does not copy the kCFI type hash preceding the function. WithCONFIG_CFI=ythe indirect call throughimx6_suspend_in_ocram_fntherefore panics.Keep the call covered by CFI instead of exempting it:
Nice! This is a pretty good progression.
Quoted 31 lines
- Declare
imx6_suspend()withSYM_TYPED_FUNC_START()so a type hash is emitted for it.fncpy()requires the entry point to be 8-byte aligned, and the macro's alignment applies to the hash rather than the entry that follows it, so pad manually before the macro.- Add a
cfi_typemember at the end of structimx6_cpu_pm_info, which directly precedes the OCRAM copy of the function. It fits in the struct's tail padding, so no sizes or offsets change. Fill it usingcfi_get_func_hash(), putting the hash where the caller's CFI check expects it: four bytes before the function entry.Also mark
ccm_base,suspend_ocram_baseandimx6_suspend_in_ocram_fnas__ro_after_init: they are only written during__init, and the function pointer in particular should not be writable afterwards.Suggested-by Nick Desaulniers <[email protected]>Signed-off-by Yo'av Moshe <[email protected]>---
Tested on a Kobo Clara HD (i.MX6SLL) running postmarketOS (clang/LLVM,CONFIG_CFI=y): suspend and resume work.Changes in v4, all following Nick's suggestions:
- Drop the v3__nocfiwrapper; keep the indirect call CFI-checked.
- Store the hash in acfi_typemember in the struct's tail padding instead of open-coded pointer arithmetic.
- Usecfi_get_func_hash()instead of reading the hash manually.
- Use SYM_TYPED_FUNC_START()/SYM_FUNC_END() instead of ENTRY()/ENDPROC()with a hand-rolled__CFI_TYPE.
Quoted 3 lines
Note:
linux/uaccess.his included beforelinux/cfi.hbecausecfi_get_func_hash()usesget_kernel_nofault()andcfi.hdoes not includeuaccess.hitself.
Ah, no, we (you) should fix that. include/linux/cfi.h should IWYU.
Quoted 10 lines
arch/arm/mach-imx/pm-imx6.c | 25 ++++++++++++++++++++++--- arch/arm/mach-imx/suspend-imx6.S | 14 ++++++++++++-- 2 files changed, 34 insertions(+), 5 deletions(-)arch/arm/mach-imx/pm-imx6.c +0
-0
arch/arm/mach-imx/pm-imx6.c line 8
Quoted 5 lines
arch/arm/mach-imx/pm-imx6.c +2
-0@@ -4,6 +4,8 @@* Copyright 2011 Linaro Ltd. */ +#include <linux/uaccess.h> +#include <linux/cfi.h>
This is a bug in include/linux/cfi.h; don't work around it. Fix it.
Quoted 21 lines
arch/arm/mach-imx/pm-imx6.c +4
-3#include <linux/clk/imx.h> #include <linux/delay.h> #include <linux/init.h> @@ -61,9 +63,9 @@#define MX6Q_SUSPEND_OCRAM_SIZE 0x1000#define MX6_MAX_MMDC_IO_NUM 33 -static void __iomem *ccm_base; -static void __iomem *suspend_ocram_base; -static void (*imx6_suspend_in_ocram_fn)(void __iomem *ocram_vbase); +static void __iomem *ccm_base__ro_after_init ; +static void __iomem *suspend_ocram_base__ro_after_init ; +static void (*imx6_suspend_in_ocram_fn)(void __iomem *ocram_vbase)__ro_after_init ;/* * suspend ocram space layout: @@ -229,8 +231,18 @@ struct imx6_cpu_pm_info {struct imx6_pm_base l2_base;u32 mmdc_io_num;/* Number of MMDC IOs which need saved/restored. */ u32 mmdc_io_val[MX6_MAX_MMDC_IO_NUM ][2 ];/* To save offset and value */ +u32 cfi_type;/* kCFI type hash of imx6_suspend() */
arch/arm/mach-imx/pm-imx6.c struct imx6_cpu_pm_info { · line 244
Quoted 10 lines
arch/arm/mach-imx/pm-imx6.c +8
-0}__aligned(8) ; +/* +* The ocram copy of imx6_suspend() starts right after struct imx6_cpu_pm_info, +* and the CFI check on the indirect call reads the kCFI type hash from the +* four bytes preceding the function entry, so cfi_type must occupy the last +* four bytes of the struct, i.e. fit into its tail padding. +*/ +static_assert(offsetofend(struct imx6_cpu_pm_info, cfi_type) == + sizeof(struct imx6_cpu_pm_info));
on this IWYU thread, shall we also include linux/build_bug.h for static_assert or linux/stddef.h for offsetofend?
Quoted 33 lines
arch/arm/mach-imx/pm-imx6.c +8
-0+void imx6_set_int_mem_clk_lpm (bool enable) {u32 val =readl_relaxed (ccm_base +CGPR );@@ -568,6 +580,13 @@ static int __init imx6q_suspend_init(const struct imx6_pm_socdata *socdata) mmdc_offset_array[i]); } +/* +* Mask out the Thumb bit, as cfi_get_func_hash() expects the +* function's actual start address. Returns 0 if CONFIG_CFI=n. +*/ + pm_info->cfi_type = +cfi_get_func_hash ((void *)((uintptr_t )&imx6_suspend & ~1UL )); + imx6_suspend_in_ocram_fn = fncpy( suspend_ocram_base + sizeof(*pm_info), &imx6_suspend,arch/arm/mach-imx/suspend-imx6.S +1
-0@@ -3,6 +3,7 @@ * Copyright 2014 Freescale Semiconductor, Inc. */ +#include <linux/cfi_types.h> #include <linux/linkage.h> #include <asm/assembler.h> #include <asm/asm-offsets.h>@@ -148,7 +149,16 @@ .endm
arch/arm/mach-imx/suspend-imx6.S line 159
Quoted 10 lines
arch/arm/mach-imx/suspend-imx6.S +8
-1-ENTRY(imx6_suspend) +#ifdef CONFIG_CFI + /* + * Pad the location counter so that the type hash emitted by + * SYM_TYPED_FUNC_START() below ends on an 8-byte boundary: + * fncpy() requires the function entry to be 8-byte aligned. + */ + .align 3 + .4byte 0
I still don't like these assembler directives inline like this; this feels like we should have perhaps a new macro in include/linux/cfi_types.h. Thoughts, Sami?
One that garuntees the 8B alignment of the symbol for w/e that function patching routine requires?
Quoted 14 lines
arch/arm/mach-imx/suspend-imx6.S +3
-2+#endif +SYM_TYPED_FUNC_START(imx6_suspend) ldr r1, [r0, #PM_INFO_PBASE_OFFSET] ldr r2, [r0, #PM_INFO_RESUME_ADDR_OFFSET] ldr r3, [r0, #PM_INFO_DDR_TYPE_OFFSET]@@ -329,4 +339,4 @@ resume: resume_mmdc ret lr -ENDPROC(imx6_suspend) +SYM_FUNC_END(imx6_suspend) --2.55.0
Signature · 2 lines
Thanks, ~Nick Desaulniers
Quoted 8 lines · Nick Desaulniers
Quoted 4 lines · Yo'av Moshe
Note:
linux/uaccess.his included beforelinux/cfi.hbecausecfi_get_func_hash()usesget_kernel_nofault()andcfi.hdoes not includeuaccess.hitself.Ah, no, we (you) should fix that.
include/linux/cfi.hshould IWYU.
Nathan fixed this here:
https://lore.kernel.org/lkml/20260604-tracing-fix-cfi-h-build-error-v1-1-b27015390901@kernel.org/
Quoted 14 lines
arch/arm/mach-imx/suspend-imx6.S line 159
Quoted 9 lines
arch/arm/mach-imx/suspend-imx6.S +8
-1-ENTRY(imx6_suspend) +#ifdef CONFIG_CFI + /* + * Pad the location counter so that the type hash emitted by + * SYM_TYPED_FUNC_START() below ends on an 8-byte boundary: + * fncpy() requires the function entry to be 8-byte aligned. + */ + .align 3 + .4byte 0I still don't like these assembler directives inline like this; this feels like we should have perhaps a new macro in
include/linux/cfi_types.h. Thoughts, Sami?
Quoted 2 lines
One that garuntees the 8B alignment of the symbol for w/e that function patching routine requires?
Sounds like arm should override SYM_TYPED_FUNC_START to add whatever alignment is needed?
Sami
Quoted 10 lines · Sami Tolvanen
Quoted 7 lines · Nick Desaulniers
Quoted 4 lines · Yo'av Moshe
Note:
linux/uaccess.his included beforelinux/cfi.hbecausecfi_get_func_hash()usesget_kernel_nofault()andcfi.hdoes not includeuaccess.hitself.
Quoted 5 lines
Quoted 1 line
Ah, no, we (you) should fix that.
include/linux/cfi.hshould IWYU.Nathan fixed this here:
https://lore.kernel.org/lkml/20260604-tracing-fix-cfi-h-build-error-v1-1-b27015390901@kernel.org/
Ah! Then this series should probably be rebased or include that patch.
Quoted 15 lines
Quoted 15 lines
arch/arm/mach-imx/suspend-imx6.S line 159
Quoted 10 lines
arch/arm/mach-imx/suspend-imx6.S +8
-1-ENTRY(imx6_suspend) +#ifdef CONFIG_CFI + /* + * Pad the location counter so that the type hash emitted by + * SYM_TYPED_FUNC_START() below ends on an 8-byte boundary: + * fncpy() requires the function entry to be 8-byte aligned. + */ + .align 3 + .4byte 0I still don't like these assembler directives inline like this; this feels like we should have perhaps a new macro in
include/linux/cfi_types.h. Thoughts, Sami?
Quoted 5 lines
Quoted 2 lines
One that garuntees the 8B alignment of the symbol for w/e that function patching routine requires?
Sounds like arm should override
SYM_TYPED_FUNC_STARTto add whatever alignment is needed?
The 8B alignment seems to be a requirement of fncopy() for 32b ARM. But functions in ARM can have 4B or even 2B alignment w/ Thumb, I think.
Doing such an override would add unnecessary padding for each use of SYM_TYPED_FUNC_START where probably none of them have this fncopy() constraint.
```
#ifdef CONFIG_CFI
#define SYM_TYPED_START_ALIGNED(name, linkage, align) \
linkage(name) ASM_NL \
.balign align ASM_NL \
.fill (align) - 4, 1, 0 ASM_NL \
__CFI_TYPE(name) ASM_NL \
name:
#else /* CONFIG_CFI */
#define SYM_TYPED_START_ALIGNED(name, linkage, align) \
SYM_START(name, linkage, .balign align)
#endif /* CONFIG_CFI */
#define SYM_TYPED_FUNC_START_ALIGNED(name, align) \
SYM_TYPED_START_ALIGNED(name, SYM_L_GLOBAL, align)
```
then this driver could do
```
// fncopy needs 8B alignment; see arch/arm/include/asm/fncpy.h.
SYM_TYPED_FUNC_START_ALIGNED(imx6_suspend, 8)
```Signature · 2 lines
Thanks, ~Nick Desaulniers
Quoted 28 lines · Nick Desaulniers
The 8B alignment seems to be a requirement of
fncopy()for 32b ARM. But functions in ARM can have 4B or even 2B alignment w/ Thumb, I think.Doing such an override would add unnecessary padding for each use of
SYM_TYPED_FUNC_STARTwhere probably none of them have thisfncopy()constraint.``` #ifdef CONFIG_CFI #define SYM_TYPED_START_ALIGNED(name, linkage, align) \ linkage(name) ASM_NL \ .balign align ASM_NL \ .fill (align) - 4, 1, 0 ASM_NL \ __CFI_TYPE(name) ASM_NL \ name: #else /* CONFIG_CFI */ #define SYM_TYPED_START_ALIGNED(name, linkage, align) \ SYM_START(name, linkage, .balign align) #endif /* CONFIG_CFI */ #define SYM_TYPED_FUNC_START_ALIGNED(name, align) \ SYM_TYPED_START_ALIGNED(name, SYM_L_GLOBAL, align) ```
Quoted 4 lines
then this driver could do
```
// fncopy needs 8B alignment; seearch/arm/include/asm/fncpy.h.SYM_TYPED_FUNC_START_ALIGNED(imx6_suspend, 8)
Looks reasonable, but this still feels ARM-specific to me. How about adding the macro to arch/arm/include/asm/linkage.h unless you have other users in mind?
Sami
Quoted 3 lines · Sami Tolvanen
Looks reasonable, but this still feels ARM-specific to me. How about adding the macro to
arch/arm/include/asm/linkage.hunless you have other users in mind?
I've now sent a v5 with Nick's macro added to arch/arm/include/asm/linkage.h as you suggested, plus the other fixes Nick asked for:
https://lore.kernel.org/all/[email protected]/
The sashiko-bot's comment on v5 indeed pointed out that imx5, at91 and omap1 might suffer from the same bug: their suspend routines are also fncpy()'d into SRAM without a type hash and called indirectly. They're all ARM, so asm/linkage.h still seems like the right place for the macro.
Do you think these need to be fixed as part of this patch, or as follow-ups? I only have my one imx6 device (a Kobo Clara HD), so I'm not sure how to test the fixes for the others.
Yo'av
Quoted 10 lines · Yo'av Moshe
The sashiko-bot's comment on v5 indeed pointed out that imx5, at91 and omap1 might suffer from the same bug: their suspend routines are also fncpy()'d into SRAM without a type hash and called indirectly. They're all ARM, so
asm/linkage.hstill seems like the right place for the macro.Do you think these need to be fixed as part of this patch, or as follow-ups? I only have my one imx6 device (a Kobo Clara HD), so I'm not sure how to test the fixes for the others.
I think separate patches for the others should be fine.
Sami
Quoted 9 lines · Sami Tolvanen
Quoted 7 lines · Yo'av Moshe
The sashiko-bot's comment on v5 indeed pointed out that imx5, at91 and omap1 might suffer from the same bug: their suspend routines are also fncpy()'d into SRAM without a type hash and called indirectly. They're all ARM, so
asm/linkage.hstill seems like the right place for the macro.
Quoted 5 lines
Quoted 3 lines
Do you think these need to be fixed as part of this patch, or as follow-ups? I only have my one imx6 device (a Kobo Clara HD), so I'm not sure how to test the fixes for the others.
I think separate patches for the others should be fine.
Without devices in hand to test on, I'd be hesitant to claim 100% confidence. Maybe reach out to the MAINTAINERS of those, and see if they have cycles to help test potential patches, before writing/sending patches that potentially don't get tested?
Signature · 2 lines
Thanks, ~Nick Desaulniers
Quoted 14 lines · Nick Desaulniers
Quoted 13 lines · Sami Tolvanen
Quoted 10 lines · Yo'av Moshe
The sashiko-bot's comment on v5 indeed pointed out that imx5, at91 and omap1 might suffer from the same bug: their suspend routines are also fncpy()'d into SRAM without a type hash and called indirectly. They're all ARM, so
asm/linkage.hstill seems like the right place for the macro.Do you think these need to be fixed as part of this patch, or as follow-ups? I only have my one imx6 device (a Kobo Clara HD), so I'm not sure how to test the fixes for the others.
Quoted 6 lines
Quoted 1 line
I think separate patches for the others should be fine.
Without devices in hand to test on, I'd be hesitant to claim 100% confidence. Maybe reach out to the
MAINTAINERSof those, and see if they have cycles to help test potential patches, before writing/sending patches that potentially don't get tested?
Nick Desaulniers:
Is this patch good to pickup?
Frank
Quoted 5 lines
--
Thanks,
~Nick Desaulniers
Nick Desaulniers
Reviewed-byQuoted 4 lines · Frank Li
Nick Desaulniers:
Is this patch good to pickup?
Yes.
It does depend on
https://lore.kernel.org/lkml/20260604-tracing-fix-cfi-h-build-error-v1-1-b27015390901@kernel.org/
though, so you might need to coordinate your PRs to Linus with Nathan.
Nathan, is https://lore.kernel.org/lkml/20260604-tracing-fix-cfi-h-build-error-v1-1-b27015390901@kernel.org/
going via kbuild tree or what?
Signature · 2 lines
Thanks, ~Nick Desaulniers
Quoted 9 lines · Nick Desaulniers
Quoted 4 lines · Frank Li
Nick Desaulniers:
Is this patch good to pickup?
Yes.
Quoted 7 lines
Reviewed-by Nick Desaulniers <[email protected]>It does depend on
https://lore.kernel.org/lkml/20260604-tracing-fix-cfi-h-build-error-v1-1-b27015390901@kernel.org/
though, so you might need to coordinate your PRs to Linus with Nathan.
Nathan, is https://lore.kernel.org/lkml/20260604-tracing-fix-cfi-h-build-error-v1-1-b27015390901@kernel.org/
going via kbuild tree or what?
And make sure to pick up v5, not v4. Was just looking at this thread on lore, and it seems like Frank you replied on v4, not v5. My RB tag is for v5, not v4.
https://lore.kernel.org/llvm/[email protected]/T/#mc0e4f1537fba074b962de6b4338f289dbb26c81e
Signature · 2 lines
Thanks, ~Nick Desaulniers
Quoted 5 lines · Nick Desaulniers
It does depend on
https://lore.kernel.org/lkml/20260604-tracing-fix-cfi-h-build-error-v1-1-b27015390901@kernel.org/
though, so you might need to coordinate your PRs to Linus with Nathan.
Nathan, is https://lore.kernel.org/lkml/20260604-tracing-fix-cfi-h-build-error-v1-1-b27015390901@kernel.org/
going via kbuild tree or what?
That change is already in mainline as commit 979c294509f9 ("cfi: Include uaccess.h for get_kernel_nofault"), so there should be nothing for upstream maintainers to worry about. If this patch needs to go back to stable past 6.18, then that change will need to be included with it.
Signature · 2 lines
Cheers, Nathan
PATCH v5 1/1 ARM: imx: Fix suspend/resume crash with Clang CFI
In reply to Yo'av MosheThe suspend code that runs from OCRAM is copied there with fncpy(), which does not copy the kCFI type hash preceding the function. With CONFIG_CFI=y the indirect call through imx6_suspend_in_ocram_fn therefore panics.
Keep the call covered by CFI instead of exempting it:
- Add SYM_TYPED_FUNC_START_ALIGNED(), a variant of SYM_TYPED_FUNC_START() that aligns the function entry rather than the type hash preceding it, and use it to declare imx6_suspend():
fncpy() requires the entry point of the function it copies to be 8-byte aligned. The macro lives in arch/arm/include/asm/linkage.h since the requirement comes from arm's fncpy().
- Add a cfi_type member at the end of struct imx6_cpu_pm_info, which directly precedes the OCRAM copy of the function. It fits in the struct's tail padding, so no sizes or offsets change. Fill it using cfi_get_func_hash(), putting the hash where the caller's CFI check expects it: four bytes before the function entry.
Also mark ccm_base, suspend_ocram_base and imx6_suspend_in_ocram_fn as __ro_after_init: they are only written during __init, and the function pointer in particular should not be writable afterwards.
---
Tested on a Kobo Clara HD (i.MX6SLL) running postmarketOS (clang/LLVM, CONFIG_CFI=y): suspend and resume work.
Note that with CONFIG_CFI=y this depends on commit 979c294509f9 ("cfi: Include uaccess.h for get_kernel_nofault()"), already in mainline, which stable backports would need as well.
Changes in v5:
- Drop the linux/uaccess.h include workaround, obsoleted by commit 979c294509f9 (Nick, Sami).
- Include linux/build_bug.h and linux/stddef.h for static_assert() and offsetofend() (Nick).
- Replace the open-coded alignment pad with a new SYM_TYPED_FUNC_START_ALIGNED() macro as sketched by Nick, placed in arch/arm/include/asm/linkage.h as suggested by Sami.
Changes in v4:
- Drop the v3 __nocfi wrapper; keep the indirect call CFI-checked.
- Store the hash in a cfi_type member in the struct's tail padding instead of open-coded pointer arithmetic.
- Use cfi_get_func_hash() instead of reading the hash manually.
- Use SYM_TYPED_FUNC_START()/SYM_FUNC_END() instead of ENTRY()/ ENDPROC() with a hand-rolled __CFI_TYPE.
arch/arm/include/asm/linkage.h | 29 +++++++++++++++++++++++++++++ arch/arm/mach-imx/pm-imx6.c | 26 +++++++++++++++++++++++--- arch/arm/mach-imx/suspend-imx6.S | 6 ++++-- 3 files changed, 56 insertions(+), 5 deletions(-)
arch/arm/include/asm/linkage.h +29 -0
@@ -9,4 +9,33 @@ .type name, %function; \ END(name) +#ifdef __ASSEMBLY__ + +/* +* Variants of SYM_TYPED_START/SYM_TYPED_FUNC_START that align the +* function entry itself instead of the kCFI type hash preceding it, +* for functions whose entry point must meet an alignment requirement, +* such as the 8-byte alignment fncpy() demands of its source. +*/ +#ifdef CONFIG_CFI + +#define SYM_TYPED_START_ALIGNED (name, linkage, align) \ + linkage(name) ASM_NL \ + .balign align ASM_NL \ + .fill (align) - 4, 1, 0 ASM_NL \ + __CFI_TYPE(name) ASM_NL \ + name: + +#else /* CONFIG_CFI */ + +#define SYM_TYPED_START_ALIGNED (name, linkage, align) \ + SYM_START(name, linkage, .balign align) + +#endif /* CONFIG_CFI */ + +#define SYM_TYPED_FUNC_START_ALIGNED (name, align) \ + SYM_TYPED_START_ALIGNED(name, SYM_L_GLOBAL, align) + +#endif /* __ASSEMBLY__ */ +#endif
arch/arm/mach-imx/pm-imx6.c +23 -3
@@ -4,6 +4,8 @@* Copyright 2011 Linaro Ltd. */ +#include <linux/build_bug.h> +#include <linux/cfi.h> #include <linux/clk/imx.h> #include <linux/delay.h> #include <linux/init.h>
@@ -18,6 +20,7 @@#include <linux/of_platform.h> #include <linux/platform_device.h> #include <linux/regmap.h> +#include <linux/stddef.h> #include <linux/suspend.h> #include <asm/cacheflush.h> #include <asm/fncpy.h>
@@ -61,9 +64,9 @@#define MX6Q_SUSPEND_OCRAM_SIZE 0x1000#define MX6_MAX_MMDC_IO_NUM 33 -static void __iomem *ccm_base; -static void __iomem *suspend_ocram_base; -static void (*imx6_suspend_in_ocram_fn)(void __iomem *ocram_vbase); +static void __iomem *ccm_base__ro_after_init ; +static void __iomem *suspend_ocram_base__ro_after_init ; +static void (*imx6_suspend_in_ocram_fn)(void __iomem *ocram_vbase)__ro_after_init ;/* * suspend ocram space layout:
@@ -229,8 +232,18 @@ struct imx6_cpu_pm_info {struct imx6_pm_base l2_base;u32 mmdc_io_num;/* Number of MMDC IOs which need saved/restored. */ u32 mmdc_io_val[MX6_MAX_MMDC_IO_NUM ][2 ];/* To save offset and value */ +u32 cfi_type;/* kCFI type hash of imx6_suspend() */ }__aligned(8) ; +/* +* The ocram copy of imx6_suspend() starts right after struct imx6_cpu_pm_info, +* and the CFI check on the indirect call reads the kCFI type hash from the +* four bytes preceding the function entry, so cfi_type must occupy the last +* four bytes of the struct, i.e. fit into its tail padding. +*/ +static_assert(offsetofend(struct imx6_cpu_pm_info, cfi_type) == + sizeof(struct imx6_cpu_pm_info)); +void imx6_set_int_mem_clk_lpm (bool enable) {u32 val =readl_relaxed (ccm_base +CGPR );
@@ -568,6 +581,13 @@ static int __init imx6q_suspend_init(const struct imx6_pm_socdata *socdata) mmdc_offset_array[i]); } +/* +* Mask out the Thumb bit, as cfi_get_func_hash() expects the +* function's actual start address. Returns 0 if CONFIG_CFI=n. +*/ + pm_info->cfi_type = +cfi_get_func_hash ((void *)((uintptr_t )&imx6_suspend & ~1UL )); + imx6_suspend_in_ocram_fn = fncpy( suspend_ocram_base + sizeof(*pm_info), &imx6_suspend,
arch/arm/mach-imx/suspend-imx6.S +4 -2
@@ -3,6 +3,7 @@ * Copyright 2014 Freescale Semiconductor, Inc. */ +#include <linux/cfi_types.h> #include <linux/linkage.h> #include <asm/assembler.h> #include <asm/asm-offsets.h>
@@ -148,7 +149,8 @@ .endm -ENTRY(imx6_suspend) +/* fncpy() requires an 8-byte-aligned entry; see arch/arm/include/asm/fncpy.h */ +SYM_TYPED_FUNC_START_ALIGNED(imx6_suspend, 8) ldr r1, [r0, #PM_INFO_PBASE_OFFSET] ldr r2, [r0, #PM_INFO_RESUME_ADDR_OFFSET] ldr r3, [r0, #PM_INFO_DDR_TYPE_OFFSET]
@@ -329,4 +331,4 @@ resume: resume_mmdc ret lr -ENDPROC(imx6_suspend) +SYM_FUNC_END(imx6_suspend)
Signature · 1 line
2.55.0
[email protected]
AI reviewThank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [High] Global CFI Check Failure on fncpy() copied Suspend/PM routines (IMX5, AT91, OMAP1) due to missing kCFI hashes. --
commit 24f4d3cbee4d4f64e23b67ce2fbbb6d3e3eb8db1
Author: Yo'av Moshe <[email protected]>
ARM: imx: Fix suspend/resume crash with Clang CFI
The suspend code that runs from OCRAM is copied there with fncpy(), which does not copy the kCFI type hash preceding the function. With CONFIG_CFI=y the indirect call through imx6_suspend_in_ocram_fn therefore panics.
Quoted 4 lines
arch/arm/mach-imx/pm-imx6.c +0
-0
[ ... ]
Quoted 4 lines
arch/arm/mach-imx/pm-imx6.c +0
-0@@ -568,6 +581,13 @@ static int __init imx6q_suspend_init(const struct imx6_pm_socdata *socdata) mmdc_offset_array[i]); }
arch/arm/mach-imx/pm-imx6.c static int __init imx6q_suspend_init(const struct imx6_pm_socdata *socdata) · line 593
Quoted 10 lines
arch/arm/mach-imx/pm-imx6.c +7
-0+/* +* Mask out the Thumb bit, as cfi_get_func_hash() expects the +* function's actual start address. Returns 0 if CONFIG_CFI=n. +*/ + pm_info->cfi_type = +cfi_get_func_hash ((void *)((uintptr_t )&imx6_suspend & ~1UL )); + imx6_suspend_in_ocram_fn = fncpy( suspend_ocram_base + sizeof(*pm_info), &imx6_suspend,
[Severity: High]
This isn't a bug introduced by this patch, but since fncpy() strips the 4-byte kCFI type hash during SRAM copying, wouldn't other ARM architectures using fncpy() for suspend routines also fail their CFI checks and panic in the exact same way?
For instance, this happens during suspend or idle when the copied PM routine is called indirectly.
In arch/arm/mach-imx/pm-imx5.c:mx5_suspend_enter():
if (imx5_suspend_in_ocram_fn)
imx5_suspend_in_ocram_fn(suspend_ocram_base);In arch/arm/mach-at91/pm.c:at91_pm_suspend():
at91_suspend_sram_fn(&soc_pm.data);
In arch/arm/mach-omap1/pm.c:omap1_pm_idle():
omap_sram_suspend(omap_readl(ARM_IDLECT1),
omap_readl(ARM_IDLECT2));In arch/arm/mach-omap1/pm.c:omap1_pm_suspend():
omap_sram_suspend(arg0, arg1);
Do these architectures need a similar fix to ensure the kCFI hash is physically located before the function copy in SRAM?
Quoted 1 line
MX6Q_SUSPEND_OCRAM_SIZE-sizeof(*pm_info));
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Quoted 21 lines · Yo'av Moshe
The suspend code that runs from OCRAM is copied there with
fncpy(), which does not copy the kCFI type hash preceding the function. WithCONFIG_CFI=ythe indirect call throughimx6_suspend_in_ocram_fntherefore panics.Keep the call covered by CFI instead of exempting it:
- Add
SYM_TYPED_FUNC_START_ALIGNED(), a variant ofSYM_TYPED_FUNC_START()that aligns the function entry rather than the type hash preceding it, and use it to declareimx6_suspend():
fncpy()requires the entry point of the function it copies to be 8-byte aligned. The macro lives inarch/arm/include/asm/linkage.hsince the requirement comes from arm'sfncpy().- Add a
cfi_typemember at the end of structimx6_cpu_pm_info, which directly precedes the OCRAM copy of the function. It fits in the struct's tail padding, so no sizes or offsets change. Fill it usingcfi_get_func_hash(), putting the hash where the caller's CFI check expects it: four bytes before the function entry.
Quoted 7 lines
Also mark
ccm_base,suspend_ocram_baseandimx6_suspend_in_ocram_fnas__ro_after_init: they are only written during__init, and the function pointer in particular should not be writable afterwards.Suggested-by Nick Desaulniers <[email protected]>Signed-off-by Yo'av Moshe <[email protected]>
Sami
[email protected]
AppliedFrom: Frank Li <[email protected]>
Quoted 8 lines · Yo'av Moshe
The suspend code that runs from OCRAM is copied there with
fncpy(), which does not copy the kCFI type hash preceding the function. WithCONFIG_CFI=ythe indirect call throughimx6_suspend_in_ocram_fntherefore panics.Keep the call covered by CFI instead of exempting it:
[...]
Applied, thanks!
[1/1] ARM: imx: Fix suspend/resume crash with Clang CFI
commit: 242757d109bd5c9b8fb7194fb4d44625be0b859a
Best regards,
Signature · 1 line
Frank Li <[email protected]>