Skip to contentSkip to conversation

xen/arm: set CPSR Z bit when creating aarch32 guests

PATCH In review Stefano Stabellini 4 participants 5 messages Mar 25, 2022

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 xen/arm: set CPSR Z bit when creating aarch32 guests

From: Stefano Stabellini <[email protected]>

The first 32 bytes of zImage are NOPs. When CONFIG_EFI is enabled in the kernel, certain versions of Linux will use an UNPREDICATABLE NOP encoding, sometimes resulting in an unbootable kernel. Whether the resulting kernel is bootable or not depends on the processor. See commit a92882a4d270 in the Linux kernel for all the details.

All kernel releases starting from Linux 4.9 without commit a92882a4d270 are affected.

Fortunately there is a simple workaround: setting the "Z" bit in CPSR make it so those invalid NOP instructions are never executed. That is because the instruction is conditional (not equal). So, on QEMU at least, the instruction will end up to be ignored and not generate an exception. Setting the "Z" bit makes those kernel versions bootable again and it is harmless in the other cases.

Note that both U-Boot and QEMU -kernel set the "Z" bit in CPSR when booting a zImage kernel on aarch32.

---
Changes in v3:
- improve commit message
- improve in-code comment

Changes in v2:
- improve commit message
- add in-code comment
- move PSR_Z to the beginning

---
 xen/include/public/arch-arm.h | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)
xen/include/public/arch-arm.h +10 -1
index 94b31511dd..c0c1149e27 100644
@@ -361,6 +361,7 @@ typedef uint64_t xen_callback_t;
 #define PSR_DBG_MASK    (1<<9)        /* arm64: Debug Exception mask */
 #define PSR_IT_MASK     (0x0600fc00)  /* Thumb If-Then Mask */
 #define PSR_JAZELLE     (1<<24)       /* Jazelle Mode */
+#define PSR_Z           (1<<30)       /* Zero condition flag */
 
 /* 32 bit modes */
 #define PSR_MODE_USR 0x10
@@ -383,7 +384,15 @@ typedef uint64_t xen_callback_t;
 #define PSR_MODE_EL1t 0x04
 #define PSR_MODE_EL0t 0x00
 
-#define PSR_GUEST32_INIT  (PSR_ABT_MASK|PSR_FIQ_MASK|PSR_IRQ_MASK|PSR_MODE_SVC)
+/*
+ * We set PSR_Z to be able to boot Linux kernel versions with an invalid
+ * encoding of the first 8 NOP instructions. See commit a92882a4d270 in
+ * Linux.
+ *
+ * Note that PSR_Z is also set by U-Boot and QEMU -kernel when loading
+ * zImage kernels on aarch32.
+ */
+#define PSR_GUEST32_INIT  (PSR_Z|PSR_ABT_MASK|PSR_FIQ_MASK|PSR_IRQ_MASK|PSR_MODE_SVC)
 #define PSR_GUEST64_INIT (PSR_ABT_MASK|PSR_FIQ_MASK|PSR_IRQ_MASK|PSR_MODE_EL1h)
 
 #define SCTLR_GUEST_INIT    xen_mk_ullong(0x00c50078)
Signature · 1 line
2.25.1
Reviewed-by on v1 1/1
#1 lore raw

Hi Stefano,

Quoted 74 lines

-----Original Message-----
From: Xen-devel <[email protected]> On Behalf Of Stefano Stabellini
Sent: 2022年3月25日 9:01
To: [email protected]
Cc: [email protected]; [email protected]; Bertrand Marquis <[email protected]>; [email protected]; Stefano Stabellini <[email protected]>
Subject: [PATCH] xen/arm: set CPSR Z bit when creating aarch32 guests

From: Stefano Stabellini <[email protected]>

The first 32 bytes of zImage are NOPs. When CONFIG_EFI is enabled in the kernel, certain versions of Linux will use an UNPREDICATABLE NOP encoding, sometimes resulting in an unbootable kernel. Whether the resulting kernel is bootable or not depends on the processor. See commit a92882a4d270 in the Linux kernel for all the details.

All kernel releases starting from Linux 4.9 without commit a92882a4d270 are affected.

Fortunately there is a simple workaround: setting the "Z" bit in CPSR make it so those invalid NOP instructions are never executed. That is because the instruction is conditional (not equal). So, on QEMU at least, the instruction will end up to be ignored and not generate an exception. Setting the "Z" bit makes those kernel versions bootable again and it is harmless in the other cases.

Note that both U-Boot and QEMU -kernel set the "Z" bit in CPSR when booting a zImage kernel on aarch32.

---
Changes in v3:
- improve commit message
- improve in-code comment

Changes in v2:
- improve commit message
- add in-code comment
- move PSR_Z to the beginning

---
 xen/include/public/arch-arm.h | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)
xen/include/public/arch-arm.h +10 -1
index 94b31511dd..c0c1149e27 100644
@@ -361,6 +361,7 @@ typedef uint64_t xen_callback_t;
 #define PSR_DBG_MASK    (1<<9)        /* arm64: Debug Exception mask */
 #define PSR_IT_MASK     (0x0600fc00)  /* Thumb If-Then Mask */
 #define PSR_JAZELLE     (1<<24)       /* Jazelle Mode */
+#define PSR_Z           (1<<30)       /* Zero condition flag */
 
 /* 32 bit modes */
 #define PSR_MODE_USR 0x10
@@ -383,7 +384,15 @@ typedef uint64_t xen_callback_t;
 #define PSR_MODE_EL1t 0x04
 #define PSR_MODE_EL0t 0x00
 
-#define PSR_GUEST32_INIT
 (PSR_ABT_MASK|PSR_FIQ_MASK|PSR_IRQ_MASK|PSR_MODE_SVC)
+/*
+ * We set PSR_Z to be able to boot Linux kernel versions with an invalid
+ * encoding of the first 8 NOP instructions. See commit a92882a4d270 in
+ * Linux.
+ *
+ * Note that PSR_Z is also set by U-Boot and QEMU -kernel when loading
+ * zImage kernels on aarch32.
+ */
+#define PSR_GUEST32_INIT
 (PSR_Z|PSR_ABT_MASK|PSR_FIQ_MASK|PSR_IRQ_MASK|PSR_MODE_SVC)
 #define PSR_GUEST64_INIT
Quoted 2 lines

(PSR_ABT_MASK|PSR_FIQ_MASK|PSR_IRQ_MASK|PSR_MODE_EL1h)

Maybe this is a good opportunity to fix the alignment of the two macros : )

Quoted 4 lines

#define SCTLR_GUEST_INIT xen_mk_ullong(0x00c50078)
--
2.25.1

Reviewed-by on v1 1/1
#3 lore raw
In reply to Stefano Stabellini

Hi Stefano,

Quoted 20 lines · Stefano Stabellini

From: Stefano Stabellini <[email protected]>

The first 32 bytes of zImage are NOPs. When CONFIG_EFI is enabled in the kernel, certain versions of Linux will use an UNPREDICATABLE NOP encoding, sometimes resulting in an unbootable kernel. Whether the resulting kernel is bootable or not depends on the processor. See commit a92882a4d270 in the Linux kernel for all the details.

All kernel releases starting from Linux 4.9 without commit a92882a4d270 are affected.

Fortunately there is a simple workaround: setting the "Z" bit in CPSR make it so those invalid NOP instructions are never executed. That is because the instruction is conditional (not equal). So, on QEMU at least, the instruction will end up to be ignored and not generate an exception. Setting the "Z" bit makes those kernel versions bootable again and it is harmless in the other cases.

Quoted 4 lines

Note that both U-Boot and QEMU -kernel set the "Z" bit in CPSR when booting a zImage kernel on aarch32.

Thanks for the comment and commit message fixes.

Cheers
Bertrand

Quoted 45 lines

---
Changes in v3:
- improve commit message
- improve in-code comment

Changes in v2:
- improve commit message
- add in-code comment
- move PSR_Z to the beginning
---
xen/include/public/arch-arm.h | 11 ++++++++++-

1 file changed, 10 insertions(+), 1 deletion(-)
xen/include/public/arch-arm.h +10 -1
index 94b31511dd..c0c1149e27 100644
@@ -361,6 +361,7 @@ typedef uint64_t xen_callback_t;
 #define PSR_DBG_MASK    (1<<9)        /* arm64: Debug Exception mask */
 #define PSR_IT_MASK     (0x0600fc00)  /* Thumb If-Then Mask */
 #define PSR_JAZELLE     (1<<24)       /* Jazelle Mode */
+#define PSR_Z           (1<<30)       /* Zero condition flag */
 
 /* 32 bit modes */
 #define PSR_MODE_USR 0x10
@@ -383,7 +384,15 @@ typedef uint64_t xen_callback_t;
 #define PSR_MODE_EL1t 0x04
 #define PSR_MODE_EL0t 0x00
 
-#define PSR_GUEST32_INIT  (PSR_ABT_MASK|PSR_FIQ_MASK|PSR_IRQ_MASK|PSR_MODE_SVC)
+/*
+ * We set PSR_Z to be able to boot Linux kernel versions with an invalid
+ * encoding of the first 8 NOP instructions. See commit a92882a4d270 in
+ * Linux.
+ *
+ * Note that PSR_Z is also set by U-Boot and QEMU -kernel when loading
+ * zImage kernels on aarch32.
+ */
+#define PSR_GUEST32_INIT  (PSR_Z|PSR_ABT_MASK|PSR_FIQ_MASK|PSR_IRQ_MASK|PSR_MODE_SVC)
 #define PSR_GUEST64_INIT (PSR_ABT_MASK|PSR_FIQ_MASK|PSR_IRQ_MASK|PSR_MODE_EL1h)
 
 #define SCTLR_GUEST_INIT    xen_mk_ullong(0x00c50078)
Signature · 1 line
2.25.1
#2 lore raw
In reply to Wei Chen

Hi Wei,

Quoted 70 lines · Wei Chen
Quoted 70 lines

-----Original Message-----
From: Xen-devel <[email protected]> On Behalf Of Stefano Stabellini
Sent: 2022年3月25日 9:01
To: [email protected]
Cc: [email protected]; [email protected]; Bertrand Marquis <[email protected]>; [email protected]; Stefano Stabellini <[email protected]>
Subject: [PATCH] xen/arm: set CPSR Z bit when creating aarch32 guests

From: Stefano Stabellini <[email protected]>

The first 32 bytes of zImage are NOPs. When CONFIG_EFI is enabled in the kernel, certain versions of Linux will use an UNPREDICATABLE NOP encoding, sometimes resulting in an unbootable kernel. Whether the resulting kernel is bootable or not depends on the processor. See commit a92882a4d270 in the Linux kernel for all the details.

All kernel releases starting from Linux 4.9 without commit a92882a4d270 are affected.

Fortunately there is a simple workaround: setting the "Z" bit in CPSR make it so those invalid NOP instructions are never executed. That is because the instruction is conditional (not equal). So, on QEMU at least, the instruction will end up to be ignored and not generate an exception. Setting the "Z" bit makes those kernel versions bootable again and it is harmless in the other cases.

Note that both U-Boot and QEMU -kernel set the "Z" bit in CPSR when booting a zImage kernel on aarch32.

---
Changes in v3:
- improve commit message
- improve in-code comment

Changes in v2:
- improve commit message
- add in-code comment
- move PSR_Z to the beginning

---
  xen/include/public/arch-arm.h | 11 ++++++++++-
  1 file changed, 10 insertions(+), 1 deletion(-)
xen/include/public/arch-arm.h +8 -1
index 94b31511dd..c0c1149e27 100644
@@ -361,6 +361,7 @@ typedef uint64_t xen_callback_t;
  #define PSR_DBG_MASK    (1<<9)        /* arm64: Debug Exception mask */
  #define PSR_IT_MASK     (0x0600fc00)  /* Thumb If-Then Mask */
  #define PSR_JAZELLE     (1<<24)       /* Jazelle Mode */
+#define PSR_Z           (1<<30)       /* Zero condition flag */
 
  /* 32 bit modes */
  #define PSR_MODE_USR 0x10
@@ -383,7 +384,15 @@ typedef uint64_t xen_callback_t;
  #define PSR_MODE_EL1t 0x04
  #define PSR_MODE_EL0t 0x00
 
-#define PSR_GUEST32_INIT
 (PSR_ABT_MASK|PSR_FIQ_MASK|PSR_IRQ_MASK|PSR_MODE_SVC)
+/*
+ * We set PSR_Z to be able to boot Linux kernel versions with an invalid
+ * encoding of the first 8 NOP instructions. See commit a92882a4d270 in
+ * Linux.
+ *
+ * Note that PSR_Z is also set by U-Boot and QEMU -kernel when loading
+ * zImage kernels on aarch32.
Quoted 8 lines
Quoted 6 lines
xen/include/public/arch-arm.h +2 -0
+ */
+#define PSR_GUEST32_INIT
 (PSR_Z|PSR_ABT_MASK|PSR_FIQ_MASK|PSR_IRQ_MASK|PSR_MODE_SVC)
  #define PSR_GUEST64_INIT

(PSR_ABT_MASK|PSR_FIQ_MASK|PSR_IRQ_MASK|PSR_MODE_EL1h)

Maybe this is a good opportunity to fix the alignment of the two macros : )

I have dropped one space for PSR_GUEST32_INIT and committed.

Cheers,

Signature · 1 line
Julien Grall