Skip to contentSkip to conversation

xen/arm: Emulate ID registers

PATCH Superseded Bertrand Marquis 2 participants 12 messages Nov 26, 2020 – Nov 30, 2020

v1 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

cover xen/arm: Emulate ID registers

The goal of this serie is to emulate coprocessor ID registers so that Xen only publish to guest features that are supported by Xen and can actually be used by guests.
One practical example where this is required are SVE support which is forbidden by Xen as it is not supported, but if Linux is compiled with it, it will crash on boot. An other one is AMU which is also forbidden by Xen but one Linux compiled with it would crash if the platform supports it.

To be able to emulate the coprocessor registers defining what features are supported by the hardware, the TID3 bit of HCR must be disabled and Xen must emulated the values of those registers when an exception is catched when a guest is accessing it.

This serie is first creating a guest cpuinfo structure which will contain the values that we want to publish to the guests and then provides the proper emulationg for those registers when Xen is getting an exception due to an access to any of those registers.

This is a first simple implementation to solve the problem and the way to define the values that we provide to guests and which features are disabled will be in a future patchset enhance so that we could decide per guest what can be used or not and depending on this deduce the bits to activate in HCR and the values that we must publish on ID registers.

Bertrand Marquis (7):
xen/arm: Add ID registers and complete cpufinfo xen/arm: Add arm64 ID registers definitions xen/arm: create a cpuinfo structure for guest xen/arm: Add handler for ID registers on arm64 xen/arm: Add handler for cp15 ID registers xen/arm: Add CP10 exception support to handle VMFR xen/arm: Activate TID3 in HCR_EL2

 xen/arch/arm/arm64/vsysreg.c        | 49 +++++++++++++++++++
 xen/arch/arm/cpufeature.c           | 67 ++++++++++++++++++++++++++
 xen/arch/arm/traps.c                |  7 ++-
 xen/arch/arm/vcpreg.c               | 73 +++++++++++++++++++++++++++++
 xen/include/asm-arm/arm64/hsr.h     | 37 +++++++++++++++
 xen/include/asm-arm/arm64/sysregs.h | 22 +++++++++
 xen/include/asm-arm/cpregs.h        | 10 ++++
 xen/include/asm-arm/cpufeature.h    | 63 +++++++++++++++++++++----
 xen/include/asm-arm/perfc_defn.h    |  1 +
 xen/include/asm-arm/traps.h         |  1 +
 10 files changed, 321 insertions(+), 9 deletions(-)
Signature · 1 line
2.17.1
#1 lore raw

PATCH 1/7 xen/arm: Add ID registers and complete cpufinfo

Add definition and entries in cpuinfo for ID registers introduced in newer Arm Architecture reference manual:
- ID_PFR2: processor feature register 2
- ID_DFR1: debug feature register 1
- ID_MMFR4 and ID_MMFR5: Memory model feature registers 4 and 5
- ID_ISA6: ISA Feature register 6
Add more bitfield definitions in PFR fields of cpuinfo.
Add MVFR2 register definition for aarch32.
Add mvfr values in cpuinfo.
Add some registers definition for arm64 in sysregs as some are not always know by compilers.
Initialize the new values added in cpuinfo in identify_cpu during init.

---
 xen/arch/arm/cpufeature.c           | 16 ++++++++
 xen/include/asm-arm/arm64/sysregs.h | 22 +++++++++++
 xen/include/asm-arm/cpregs.h        | 10 +++++
 xen/include/asm-arm/cpufeature.h    | 61 +++++++++++++++++++++++++----
 4 files changed, 101 insertions(+), 8 deletions(-)
xen/arch/arm/cpufeature.c +16 -0
index 44126dbf07..c0adf848c5 100644
@@ -114,13 +114,17 @@ void identify_cpu(struct cpuinfo_arm *c)
 
         c->mm64.bits[0]  = READ_SYSREG64(ID_AA64MMFR0_EL1);
         c->mm64.bits[1]  = READ_SYSREG64(ID_AA64MMFR1_EL1);
+        c->mm64.bits[2]  = READ_SYSREG64(ID_AA64MMFR2_EL1);
 
         c->isa64.bits[0] = READ_SYSREG64(ID_AA64ISAR0_EL1);
         c->isa64.bits[1] = READ_SYSREG64(ID_AA64ISAR1_EL1);
+
+        c->zfr64.bits[0] = READ_SYSREG64(ID_AA64ZFR0_EL1);
 #endif
 
         c->pfr32.bits[0] = READ_SYSREG32(ID_PFR0_EL1);
         c->pfr32.bits[1] = READ_SYSREG32(ID_PFR1_EL1);
+        c->pfr32.bits[2] = READ_SYSREG32(ID_PFR2_EL1);
 
         c->dbg32.bits[0] = READ_SYSREG32(ID_DFR0_EL1);
 
@@ -130,6 +134,8 @@ void identify_cpu(struct cpuinfo_arm *c)
         c->mm32.bits[1]  = READ_SYSREG32(ID_MMFR1_EL1);
         c->mm32.bits[2]  = READ_SYSREG32(ID_MMFR2_EL1);
         c->mm32.bits[3]  = READ_SYSREG32(ID_MMFR3_EL1);
+        c->mm32.bits[4]  = READ_SYSREG32(ID_MMFR4_EL1);
+        c->mm32.bits[5]  = READ_SYSREG32(ID_MMFR5_EL1);
 
         c->isa32.bits[0] = READ_SYSREG32(ID_ISAR0_EL1);
         c->isa32.bits[1] = READ_SYSREG32(ID_ISAR1_EL1);
@@ -137,6 +143,16 @@ void identify_cpu(struct cpuinfo_arm *c)
         c->isa32.bits[3] = READ_SYSREG32(ID_ISAR3_EL1);
         c->isa32.bits[4] = READ_SYSREG32(ID_ISAR4_EL1);
         c->isa32.bits[5] = READ_SYSREG32(ID_ISAR5_EL1);
+        c->isa32.bits[6] = READ_SYSREG32(ID_ISAR6_EL1);
+
+#ifdef CONFIG_ARM_64
+        c->mvfr.bits[0] = READ_SYSREG64(MVFR0_EL1);
+        c->mvfr.bits[1] = READ_SYSREG64(MVFR1_EL1);
+        c->mvfr.bits[2] = READ_SYSREG64(MVFR2_EL1);
+#else
+        c->mvfr.bits[0] = READ_CP32(MVFR0);
+        c->mvfr.bits[1] = READ_CP32(MVFR1);
+#endif
 }
 
 /*
xen/include/asm-arm/arm64/sysregs.h +22 -0
index c60029d38f..fd0b25b9c7 100644
@@ -57,6 +57,28 @@
 #define ICH_AP1R2_EL2             __AP1Rx_EL2(2)
 #define ICH_AP1R3_EL2             __AP1Rx_EL2(3)
 
+/*
+ * Define ID coprocessor registers if they are not
+ * already defined by the compiler.
+ *
+ * Values picked from linux kernel
+ */
+#ifndef ID_AA64MMFR2_EL1
+#define ID_AA64MMFR2_EL1            S3_0_C0_C7_2
+#endif
+#ifndef ID_PFR2_EL1
+#define ID_PFR2_EL1                 S3_0_C0_C3_4
+#endif
+#ifndef ID_MMFR5_EL1
+#define ID_MMFR5_EL1                S3_0_C0_C3_6
+#endif
+#ifndef ID_ISAR6_EL1
+#define ID_ISAR6_EL1                S3_0_C0_C2_7
+#endif
+#ifndef ID_AA64ZFR0_EL1
+#define ID_AA64ZFR0_EL1             S3_0_C0_C4_4
+#endif
+
 /* Access to system registers */
 
 #define READ_SYSREG32(name) ((uint32_t)READ_SYSREG64(name))
xen/include/asm-arm/cpregs.h +10 -0
index 8fd344146e..aa8fc1d4a3 100644
@@ -63,6 +63,7 @@
 #define FPSID           p10,7,c0,c0,0   /* Floating-Point System ID Register */
 #define FPSCR           p10,7,c1,c0,0   /* Floating-Point Status and Control Register */
 #define MVFR0           p10,7,c7,c0,0   /* Media and VFP Feature Register 0 */
+#define MVFR1           p10,7,c6,c0,0   /* Media and VFP Feature Register 1 */
 #define FPEXC           p10,7,c8,c0,0   /* Floating-Point Exception Control Register */
 #define FPINST          p10,7,c9,c0,0   /* Floating-Point Instruction Register */
 #define FPINST2         p10,7,c10,c0,0  /* Floating-point Instruction Register 2 */
@@ -108,18 +109,23 @@
 #define MPIDR           p15,0,c0,c0,5   /* Multiprocessor Affinity Register */
 #define ID_PFR0         p15,0,c0,c1,0   /* Processor Feature Register 0 */
 #define ID_PFR1         p15,0,c0,c1,1   /* Processor Feature Register 1 */
+#define ID_PFR2         p15,0,c0,c3,4   /* Processor Feature Register 2 */
 #define ID_DFR0         p15,0,c0,c1,2   /* Debug Feature Register 0 */
+#define ID_DFR1         p15,0,c0,c3,5   /* Debug Feature Register 1 */
 #define ID_AFR0         p15,0,c0,c1,3   /* Auxiliary Feature Register 0 */
 #define ID_MMFR0        p15,0,c0,c1,4   /* Memory Model Feature Register 0 */
 #define ID_MMFR1        p15,0,c0,c1,5   /* Memory Model Feature Register 1 */
 #define ID_MMFR2        p15,0,c0,c1,6   /* Memory Model Feature Register 2 */
 #define ID_MMFR3        p15,0,c0,c1,7   /* Memory Model Feature Register 3 */
+#define ID_MMFR4        p15,0,c0,c2,6   /* Memory Model Feature Register 4 */
+#define ID_MMFR5        p15,0,c0,c3,6   /* Memory Model Feature Register 5 */
 #define ID_ISAR0        p15,0,c0,c2,0   /* ISA Feature Register 0 */
 #define ID_ISAR1        p15,0,c0,c2,1   /* ISA Feature Register 1 */
 #define ID_ISAR2        p15,0,c0,c2,2   /* ISA Feature Register 2 */
 #define ID_ISAR3        p15,0,c0,c2,3   /* ISA Feature Register 3 */
 #define ID_ISAR4        p15,0,c0,c2,4   /* ISA Feature Register 4 */
 #define ID_ISAR5        p15,0,c0,c2,5   /* ISA Feature Register 5 */
+#define ID_ISAR6        p15,0,c0,c2,7   /* ISA Feature Register 6 */
 #define CCSIDR          p15,1,c0,c0,0   /* Cache Size ID Registers */
 #define CLIDR           p15,1,c0,c0,1   /* Cache Level ID Register */
 #define CSSELR          p15,2,c0,c0,0   /* Cache Size Selection Register */
@@ -318,12 +324,16 @@
 #define ID_ISAR3_EL1            ID_ISAR3
 #define ID_ISAR4_EL1            ID_ISAR4
 #define ID_ISAR5_EL1            ID_ISAR5
+#define ID_ISAR6_EL1            ID_ISAR6
 #define ID_MMFR0_EL1            ID_MMFR0
 #define ID_MMFR1_EL1            ID_MMFR1
 #define ID_MMFR2_EL1            ID_MMFR2
 #define ID_MMFR3_EL1            ID_MMFR3
+#define ID_MMFR4_EL1            ID_MMFR4
+#define ID_MMFR5_EL1            ID_MMFR5
 #define ID_PFR0_EL1             ID_PFR0
 #define ID_PFR1_EL1             ID_PFR1
+#define ID_PFR2_EL1             ID_PFR2
 #define IFSR32_EL2              IFSR
 #define MDCR_EL2                HDCR
 #define MIDR_EL1                MIDR
xen/include/asm-arm/cpufeature.h +53 -8

119 lines not on this page · raw message

Signature · 1 line
2.17.1
#2 lore raw

PATCH 2/7 xen/arm: Add arm64 ID registers definitions

In reply to Bertrand Marquis

Add coprocessor registers definitions for all ID registers trapped through the TID3 bit of HSR.
Those are the one that will be emulated in Xen to only publish to guests the features that are supported by Xen and that are accessible to guests.

---
 xen/include/asm-arm/arm64/hsr.h | 37 +++++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)
xen/include/asm-arm/arm64/hsr.h +37 -0
index ca931dd2fe..e691d41c17 100644
@@ -110,6 +110,43 @@
 #define HSR_SYSREG_CNTP_CTL_EL0   HSR_SYSREG(3,3,c14,c2,1)
 #define HSR_SYSREG_CNTP_CVAL_EL0  HSR_SYSREG(3,3,c14,c2,2)
 
+/* Those registers are used when HCR_EL2.TID3 is set */
+#define HSR_SYSREG_ID_PFR0_EL1    HSR_SYSREG(3,0,c0,c1,0)
+#define HSR_SYSREG_ID_PFR1_EL1    HSR_SYSREG(3,0,c0,c1,1)
+#define HSR_SYSREG_ID_PFR2_EL1    HSR_SYSREG(3,0,c0,c3,4)
+#define HSR_SYSREG_ID_DFR0_EL1    HSR_SYSREG(3,0,c0,c1,2)
+#define HSR_SYSREG_ID_DFR1_EL1    HSR_SYSREG(3,0,c0,c3,5)
+#define HSR_SYSREG_ID_AFR0_EL1    HSR_SYSREG(3,0,c0,c1,3)
+#define HSR_SYSREG_ID_MMFR0_EL1   HSR_SYSREG(3,0,c0,c1,4)
+#define HSR_SYSREG_ID_MMFR1_EL1   HSR_SYSREG(3,0,c0,c1,5)
+#define HSR_SYSREG_ID_MMFR2_EL1   HSR_SYSREG(3,0,c0,c1,6)
+#define HSR_SYSREG_ID_MMFR3_EL1   HSR_SYSREG(3,0,c0,c1,7)
+#define HSR_SYSREG_ID_MMFR4_EL1   HSR_SYSREG(3,0,c0,c2,6)
+#define HSR_SYSREG_ID_MMFR5_EL1   HSR_SYSREG(3,0,c0,c3,6)
+#define HSR_SYSREG_ID_ISAR0_EL1   HSR_SYSREG(3,0,c0,c2,0)
+#define HSR_SYSREG_ID_ISAR1_EL1   HSR_SYSREG(3,0,c0,c2,1)
+#define HSR_SYSREG_ID_ISAR2_EL1   HSR_SYSREG(3,0,c0,c2,2)
+#define HSR_SYSREG_ID_ISAR3_EL1   HSR_SYSREG(3,0,c0,c2,3)
+#define HSR_SYSREG_ID_ISAR4_EL1   HSR_SYSREG(3,0,c0,c2,4)
+#define HSR_SYSREG_ID_ISAR5_EL1   HSR_SYSREG(3,0,c0,c2,5)
+#define HSR_SYSREG_ID_ISAR6_EL1   HSR_SYSREG(3,0,c0,c2,7)
+#define HSR_SYSREG_MVFR0_EL1      HSR_SYSREG(3,0,c0,c3,0)
+#define HSR_SYSREG_MVFR1_EL1      HSR_SYSREG(3,0,c0,c3,1)
+#define HSR_SYSREG_MVFR2_EL1      HSR_SYSREG(3,0,c0,c3,2)
+
+#define HSR_SYSREG_ID_AA64PFR0_EL1   HSR_SYSREG(3,0,c0,c4,0)
+#define HSR_SYSREG_ID_AA64PFR1_EL1   HSR_SYSREG(3,0,c0,c4,1)
+#define HSR_SYSREG_ID_AA64DFR0_EL1   HSR_SYSREG(3,0,c0,c5,0)
+#define HSR_SYSREG_ID_AA64DFR1_EL1   HSR_SYSREG(3,0,c0,c5,1)
+#define HSR_SYSREG_ID_AA64ISAR0_EL1  HSR_SYSREG(3,0,c0,c6,0)
+#define HSR_SYSREG_ID_AA64ISAR1_EL1  HSR_SYSREG(3,0,c0,c6,1)
+#define HSR_SYSREG_ID_AA64MMFR0_EL1  HSR_SYSREG(3,0,c0,c7,0)
+#define HSR_SYSREG_ID_AA64MMFR1_EL1  HSR_SYSREG(3,0,c0,c7,1)
+#define HSR_SYSREG_ID_AA64MMFR2_EL1  HSR_SYSREG(3,0,c0,c7,2)
+#define HSR_SYSREG_ID_AA64AFR0_EL1   HSR_SYSREG(3,0,c0,c5,4)
+#define HSR_SYSREG_ID_AA64AFR1_EL1   HSR_SYSREG(3,0,c0,c5,5)
+#define HSR_SYSREG_ID_AA64ZFR0_EL1   HSR_SYSREG(3,0,c0,c4,4)
+
 #endif /* __ASM_ARM_ARM64_HSR_H */
 
 /*
Signature · 1 line
2.17.1
#3 lore raw

PATCH 3/7 xen/arm: create a cpuinfo structure for guest

In reply to Bertrand Marquis

Create a cpuinfo structure for guest and mask into it the features that we do not support in Xen or that we do not want to publish to guests.

Modify some values in the cpuinfo structure for guests to mask some features which we do not want to allow to guests (like AMU) or we do not support (like SVE).

The code is trying to group together registers modifications for the same feature to be able in the long term to easily enable/disable a feature depending on user parameters or add other registers modification in the same place (like enabling/disabling HCR bits).

---
 xen/arch/arm/cpufeature.c        | 51 ++++++++++++++++++++++++++++++++
 xen/include/asm-arm/cpufeature.h |  2 ++
 2 files changed, 53 insertions(+)
xen/arch/arm/cpufeature.c +51 -0
index 835c48ca28..4cc4ebc78a 100644
@@ -24,6 +24,8 @@
 
 DECLARE_BITMAP(cpu_hwcaps, ARM_NCAPS);
 
+struct cpuinfo_arm __read_mostly guest_cpuinfo;
+
 void update_cpu_capabilities(const struct arm_cpu_capabilities *caps,
                              const char *info)
 {
@@ -155,6 +157,55 @@ void identify_cpu(struct cpuinfo_arm *c)
 #endif
 }
 
+/*
+ * This function is creating a cpuinfo structure with values modified to mask
+ * all cpu features that should not be published to guest.
+ * The created structure is then used to provide ID registers values to guests.
+ */
+static int __init create_guest_cpuinfo(void)
+{
+    /*
+     * TODO: The code is currently using only the features detected on the boot
+     * core. In the long term we should try to compute values containing only
+     * features supported by all cores.
+     */
+    identify_cpu(&guest_cpuinfo);
+
+#ifdef CONFIG_ARM_64
+    /* Disable MPAM as xen does not support it */
+    guest_cpuinfo.pfr64.mpam = 0;
+    guest_cpuinfo.pfr64.mpam_frac = 0;
+
+    /* Disable SVE as Xen does not support it */
+    guest_cpuinfo.pfr64.sve = 0;
+    guest_cpuinfo.zfr64.bits[0] = 0;
+
+    /* Disable MTE as Xen does not support it */
+    guest_cpuinfo.pfr64.mte = 0;
+#endif
+
+    /* Disable AMU */
+#ifdef CONFIG_ARM_64
+    guest_cpuinfo.pfr64.amu = 0;
+#endif
+    guest_cpuinfo.pfr32.amu = 0;
+
+    /* Disable RAS as Xen does not support it */
+#ifdef CONFIG_ARM_64
+    guest_cpuinfo.pfr64.ras = 0;
+    guest_cpuinfo.pfr64.ras_frac = 0;
+#endif
+    guest_cpuinfo.pfr32.ras = 0;
+    guest_cpuinfo.pfr32.ras_frac = 0;
+
+    return 0;
+}
+/*
+ * This function needs to be run after all smp are started to have
+ * cpuinfo structures for all cores.
+ */
+__initcall(create_guest_cpuinfo);
+
 /*
  * Local variables:
  * mode: C
xen/include/asm-arm/cpufeature.h +2 -0
index 1862c14e37..f03aca3d90 100644
@@ -290,6 +290,8 @@ extern void identify_cpu(struct cpuinfo_arm *);
 extern struct cpuinfo_arm cpu_data[];
 #define current_cpu_data cpu_data[smp_processor_id()]
 
+extern struct cpuinfo_arm guest_cpuinfo;
+
 #endif /* __ASSEMBLY__ */
 
 #endif
Signature · 1 line
2.17.1
#4 lore raw

PATCH 4/7 xen/arm: Add handler for ID registers on arm64

In reply to Bertrand Marquis

Add vsysreg emulation for registers trapped when TID3 bit is activated in HSR.
The emulation is returning the value stored in cpuinfo_guest structure for most values and the hardware value for registers not stored in the structure (those are mostly registers existing only as a provision for feature use but who have no definition right now).

---
 xen/arch/arm/arm64/vsysreg.c | 49 ++++++++++++++++++++++++++++++++++++
 1 file changed, 49 insertions(+)
xen/arch/arm/arm64/vsysreg.c +49 -0
index 8a85507d9d..970ef51603 100644
@@ -69,6 +69,14 @@ TVM_REG(CONTEXTIDR_EL1)
         break;                                                          \
     }
 
+/* Macro to generate easily case for ID co-processor emulation */
+#define GENERATE_TID3_INFO(reg,field,offset)                            \
+    case HSR_SYSREG_##reg:                                              \
+    {                                                                   \
+        return handle_ro_read_val(regs, regidx, hsr.sysreg.read, hsr,   \
+                          1, guest_cpuinfo.field.bits[offset]);         \
+    }
+
 void do_sysreg(struct cpu_user_regs *regs,
                const union hsr hsr)
 {
@@ -259,6 +267,47 @@ void do_sysreg(struct cpu_user_regs *regs,
          */
         return handle_raz_wi(regs, regidx, hsr.sysreg.read, hsr, 1);
 
+    /*
+     * HCR_EL2.TID3
+     *
+     * This is trapping most Identification registers used by a guest
+     * to identify the processor features
+     */
+    GENERATE_TID3_INFO(ID_PFR0_EL1, pfr32, 0)
+    GENERATE_TID3_INFO(ID_PFR1_EL1, pfr32, 1)
+    GENERATE_TID3_INFO(ID_PFR2_EL1, pfr32, 2)
+    GENERATE_TID3_INFO(ID_DFR0_EL1, dbg32, 0)
+    GENERATE_TID3_INFO(ID_DFR1_EL1, dbg32, 1)
+    GENERATE_TID3_INFO(ID_AFR0_EL1, aux32, 0)
+    GENERATE_TID3_INFO(ID_MMFR0_EL1, mm32, 0)
+    GENERATE_TID3_INFO(ID_MMFR1_EL1, mm32, 1)
+    GENERATE_TID3_INFO(ID_MMFR2_EL1, mm32, 2)
+    GENERATE_TID3_INFO(ID_MMFR3_EL1, mm32, 3)
+    GENERATE_TID3_INFO(ID_MMFR4_EL1, mm32, 4)
+    GENERATE_TID3_INFO(ID_MMFR5_EL1, mm32, 5)
+    GENERATE_TID3_INFO(ID_ISAR0_EL1, isa32, 0)
+    GENERATE_TID3_INFO(ID_ISAR1_EL1, isa32, 1)
+    GENERATE_TID3_INFO(ID_ISAR2_EL1, isa32, 2)
+    GENERATE_TID3_INFO(ID_ISAR3_EL1, isa32, 3)
+    GENERATE_TID3_INFO(ID_ISAR4_EL1, isa32, 4)
+    GENERATE_TID3_INFO(ID_ISAR5_EL1, isa32, 5)
+    GENERATE_TID3_INFO(ID_ISAR6_EL1, isa32, 6)
+    GENERATE_TID3_INFO(MVFR0_EL1, mvfr, 0)
+    GENERATE_TID3_INFO(MVFR1_EL1, mvfr, 1)
+    GENERATE_TID3_INFO(MVFR2_EL1, mvfr, 2)
+    GENERATE_TID3_INFO(ID_AA64PFR0_EL1, pfr64, 0)
+    GENERATE_TID3_INFO(ID_AA64PFR1_EL1, pfr64, 1)
+    GENERATE_TID3_INFO(ID_AA64DFR0_EL1, dbg64, 0)
+    GENERATE_TID3_INFO(ID_AA64DFR1_EL1, dbg64, 1)
+    GENERATE_TID3_INFO(ID_AA64ISAR0_EL1, isa64, 0)
+    GENERATE_TID3_INFO(ID_AA64ISAR1_EL1, isa64, 1)
+    GENERATE_TID3_INFO(ID_AA64MMFR0_EL1, mm64, 0)
+    GENERATE_TID3_INFO(ID_AA64MMFR1_EL1, mm64, 1)
+    GENERATE_TID3_INFO(ID_AA64MMFR2_EL1, mm64, 2)
+    GENERATE_TID3_INFO(ID_AA64AFR0_EL1, aux64, 0)
+    GENERATE_TID3_INFO(ID_AA64AFR1_EL1, aux64, 1)
+    GENERATE_TID3_INFO(ID_AA64ZFR0_EL1, zfr64, 0)
+
     /*
      * HCR_EL2.TIDCP
      *
Signature · 1 line
2.17.1
#5 lore raw

PATCH 5/7 xen/arm: Add handler for cp15 ID registers

In reply to Bertrand Marquis

Add support for emulation of cp15 based ID registers (on arm32 or when running a 32bit guest on arm64).
The handlers are returning the values stored in the guest_cpuinfo structure.
In the current status the MVFR registers are no supported.

---
 xen/arch/arm/vcpreg.c | 35 +++++++++++++++++++++++++++++++++++
 1 file changed, 35 insertions(+)
xen/arch/arm/vcpreg.c +35 -0
index cdc91cdf5b..d0c6406f34 100644
@@ -155,6 +155,14 @@ TVM_REG32(CONTEXTIDR, CONTEXTIDR_EL1)
         break;                                                      \
     }
 
+/* Macro to generate easily case for ID co-processor emulation */
+#define GENERATE_TID3_INFO(reg,field,offset)                        \
+    case HSR_CPREG32(reg):                                          \
+    {                                                               \
+        return handle_ro_read_val(regs, regidx, cp32.read, hsr,     \
+                          1, guest_cpuinfo.field.bits[offset]);     \
+    }
+
 void do_cp15_32(struct cpu_user_regs *regs, const union hsr hsr)
 {
     const struct hsr_cp32 cp32 = hsr.cp32;
@@ -286,6 +294,33 @@ void do_cp15_32(struct cpu_user_regs *regs, const union hsr hsr)
          */
         return handle_raz_wi(regs, regidx, cp32.read, hsr, 1);
 
+    /*
+     * HCR_EL2.TID3
+     *
+     * This is trapping most Identification registers used by a guest
+     * to identify the processor features
+     */
+    GENERATE_TID3_INFO(ID_PFR0, pfr32, 0)
+    GENERATE_TID3_INFO(ID_PFR1, pfr32, 1)
+    GENERATE_TID3_INFO(ID_PFR2, pfr32, 2)
+    GENERATE_TID3_INFO(ID_DFR0, dbg32, 0)
+    GENERATE_TID3_INFO(ID_DFR1, dbg32, 1)
+    GENERATE_TID3_INFO(ID_AFR0, aux32, 0)
+    GENERATE_TID3_INFO(ID_MMFR0, mm32, 0)
+    GENERATE_TID3_INFO(ID_MMFR1, mm32, 1)
+    GENERATE_TID3_INFO(ID_MMFR2, mm32, 2)
+    GENERATE_TID3_INFO(ID_MMFR3, mm32, 3)
+    GENERATE_TID3_INFO(ID_MMFR4, mm32, 4)
+    GENERATE_TID3_INFO(ID_MMFR5, mm32, 5)
+    GENERATE_TID3_INFO(ID_ISAR0, isa32, 0)
+    GENERATE_TID3_INFO(ID_ISAR1, isa32, 1)
+    GENERATE_TID3_INFO(ID_ISAR2, isa32, 2)
+    GENERATE_TID3_INFO(ID_ISAR3, isa32, 3)
+    GENERATE_TID3_INFO(ID_ISAR4, isa32, 4)
+    GENERATE_TID3_INFO(ID_ISAR5, isa32, 5)
+    GENERATE_TID3_INFO(ID_ISAR6, isa32, 6)
+    /* MVFR registers are in cp10 no cp15 */
+
     /*
      * HCR_EL2.TIDCP
      *
Signature · 1 line
2.17.1
#6 lore raw

PATCH 6/7 xen/arm: Add CP10 exception support to handle VMFR

In reply to Bertrand Marquis

Add support for cp10 exceptions decoding to be able to emulate the values for VMFR0 and VMFR1 when TID3 bit of HSR is activated. This is required for aarch32 guests accessing VMFR0 and VMFR1 using vmrs and vmsr instructions.

---
 xen/arch/arm/traps.c             |  5 +++++
 xen/arch/arm/vcpreg.c            | 38 ++++++++++++++++++++++++++++++++
 xen/include/asm-arm/perfc_defn.h |  1 +
 xen/include/asm-arm/traps.h      |  1 +
 4 files changed, 45 insertions(+)
xen/arch/arm/traps.c +5 -0
index 22bd1bd4c6..28d9d64558 100644
@@ -2097,6 +2097,11 @@ void do_trap_guest_sync(struct cpu_user_regs *regs)
         perfc_incr(trap_cp14_dbg);
         do_cp14_dbg(regs, hsr);
         break;
+    case HSR_EC_CP10:
+        GUEST_BUG_ON(!psr_mode_is_32bit(regs));
+        perfc_incr(trap_cp10);
+        do_cp10(regs, hsr);
+        break;
     case HSR_EC_CP:
         GUEST_BUG_ON(!psr_mode_is_32bit(regs));
         perfc_incr(trap_cp);
xen/arch/arm/vcpreg.c +38 -0
index d0c6406f34..9d6a36ca5d 100644
@@ -634,6 +634,44 @@ void do_cp14_dbg(struct cpu_user_regs *regs, const union hsr hsr)
     inject_undef_exception(regs, hsr);
 }
 
+void do_cp10(struct cpu_user_regs *regs, const union hsr hsr)
+{
+    const struct hsr_cp32 cp32 = hsr.cp32;
+    int regidx = cp32.reg;
+
+    if ( !check_conditional_instr(regs, hsr) )
+    {
+        advance_pc(regs, hsr);
+        return;
+    }
+
+    switch ( hsr.bits & HSR_CP32_REGS_MASK )
+    {
+    /*
+     * HSR.TID3 is trapping access to MVFR register used to identify the
+     * VFP/Simd using VMRS/VMSR instructions.
+     * In this case MVFR2 is not supported as the instruction does not support
+     * it.
+     * Exception encoding is using MRC/MCR standard with the reg field in Crn
+     * as are declared MVFR0 and MVFR1 in cpregs.h
+     */
+    GENERATE_TID3_INFO(MVFR0, mvfr, 0)
+    GENERATE_TID3_INFO(MVFR1, mvfr, 1)
+
+    default:
+        gdprintk(XENLOG_ERR,
+                 "%s p10, %d, r%d, cr%d, cr%d, %d @ 0x%"PRIregister"\n",
+                 cp32.read ? "mrc" : "mcr",
+                 cp32.op1, cp32.reg, cp32.crn, cp32.crm, cp32.op2, regs->pc);
+        gdprintk(XENLOG_ERR, "unhandled 32-bit CP10 access %#x\n",
+                 hsr.bits & HSR_CP32_REGS_MASK);
+        inject_undef_exception(regs, hsr);
+        return;
+    }
+
+    advance_pc(regs, hsr);
+}
+
 void do_cp(struct cpu_user_regs *regs, const union hsr hsr)
 {
     const struct hsr_cp cp = hsr.cp;
xen/include/asm-arm/perfc_defn.h +1 -0
index 6a83185163..31f071222b 100644
@@ -11,6 +11,7 @@ PERFCOUNTER(trap_cp15_64,  "trap: cp15 64-bit access")
 PERFCOUNTER(trap_cp14_32,  "trap: cp14 32-bit access")
 PERFCOUNTER(trap_cp14_64,  "trap: cp14 64-bit access")
 PERFCOUNTER(trap_cp14_dbg, "trap: cp14 dbg access")
+PERFCOUNTER(trap_cp10,     "trap: cp10 access")
 PERFCOUNTER(trap_cp,       "trap: cp access")
 PERFCOUNTER(trap_smc32,    "trap: 32-bit smc")
 PERFCOUNTER(trap_hvc32,    "trap: 32-bit hvc")
xen/include/asm-arm/traps.h +1 -0
index 997c37884e..c4a3d0fb1b 100644
@@ -62,6 +62,7 @@ void do_cp15_64(struct cpu_user_regs *regs, const union hsr hsr);
 void do_cp14_32(struct cpu_user_regs *regs, const union hsr hsr);
 void do_cp14_64(struct cpu_user_regs *regs, const union hsr hsr);
 void do_cp14_dbg(struct cpu_user_regs *regs, const union hsr hsr);
+void do_cp10(struct cpu_user_regs *regs, const union hsr hsr);
 void do_cp(struct cpu_user_regs *regs, const union hsr hsr);
 
 /* SMCCC handling */
Signature · 1 line
2.17.1
#7 lore raw

PATCH 7/7 xen/arm: Activate TID3 in HCR_EL2

In reply to Bertrand Marquis

Activate TID3 bit in HSR register when starting a guest. This will trap all coprecessor ID registers so that we can give to guest values corresponding to what they can actually use and mask some features to guests even though they would be supported by the underlying hardware (like SVE or MPAM).

---
 xen/arch/arm/traps.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
xen/arch/arm/traps.c +1 -1
index 28d9d64558..c1a9ad6056 100644
@@ -98,7 +98,7 @@ register_t get_default_hcr_flags(void)
 {
     return  (HCR_PTW|HCR_BSU_INNER|HCR_AMO|HCR_IMO|HCR_FMO|HCR_VM|
              (vwfi != NATIVE ? (HCR_TWI|HCR_TWE) : 0) |
-             HCR_TSC|HCR_TAC|HCR_SWIO|HCR_TIDCP|HCR_FB|HCR_TSW);
+             HCR_TID3|HCR_TSC|HCR_TAC|HCR_SWIO|HCR_TIDCP|HCR_FB|HCR_TSW);
 }
 
 static enum {
Signature · 1 line
2.17.1
#8 lore raw
In reply to Bertrand Marquis
Quoted 25 lines · Bertrand Marquis

The goal of this serie is to emulate coprocessor ID registers so that Xen only publish to guest features that are supported by Xen and can actually be used by guests.
One practical example where this is required are SVE support which is forbidden by Xen as it is not supported, but if Linux is compiled with it, it will crash on boot. An other one is AMU which is also forbidden by Xen but one Linux compiled with it would crash if the platform supports it.

To be able to emulate the coprocessor registers defining what features are supported by the hardware, the TID3 bit of HCR must be disabled and Xen must emulated the values of those registers when an exception is catched when a guest is accessing it.

This serie is first creating a guest cpuinfo structure which will contain the values that we want to publish to the guests and then provides the proper emulationg for those registers when Xen is getting an exception due to an access to any of those registers.

This is a first simple implementation to solve the problem and the way to define the values that we provide to guests and which features are disabled will be in a future patchset enhance so that we could decide per guest what can be used or not and depending on this deduce the bits to activate in HCR and the values that we must publish on ID registers.

Quoted 8 lines

Bertrand Marquis (7):
xen/arm: Add ID registers and complete cpufinfo xen/arm: Add arm64 ID registers definitions xen/arm: create a cpuinfo structure for guest xen/arm: Add handler for ID registers on arm64 xen/arm: Add handler for cp15 ID registers xen/arm: Add CP10 exception support to handle VMFR xen/arm: Activate TID3 in HCR_EL2

CI found an ARM randconfig failure against this series.

https://gitlab.com/xen-project/patchew/xen/-/pipelines/221798884

I have admit that I can't spot an obvious connection so it might be collateral damage from elsewhere, but does need looking at irrespective.

~Andrew (in lieu of a real CI robot).

#9 lore raw

Hi Andrew,

Quoted 37 lines · Andrew Cooper
Quoted 25 lines · Bertrand Marquis

The goal of this serie is to emulate coprocessor ID registers so that Xen only publish to guest features that are supported by Xen and can actually be used by guests.
One practical example where this is required are SVE support which is forbidden by Xen as it is not supported, but if Linux is compiled with it, it will crash on boot. An other one is AMU which is also forbidden by Xen but one Linux compiled with it would crash if the platform supports it.

To be able to emulate the coprocessor registers defining what features are supported by the hardware, the TID3 bit of HCR must be disabled and Xen must emulated the values of those registers when an exception is catched when a guest is accessing it.

This serie is first creating a guest cpuinfo structure which will contain the values that we want to publish to the guests and then provides the proper emulationg for those registers when Xen is getting an exception due to an access to any of those registers.

This is a first simple implementation to solve the problem and the way to define the values that we provide to guests and which features are disabled will be in a future patchset enhance so that we could decide per guest what can be used or not and depending on this deduce the bits to activate in HCR and the values that we must publish on ID registers.

Quoted 8 lines

Bertrand Marquis (7):
xen/arm: Add ID registers and complete cpufinfo xen/arm: Add arm64 ID registers definitions xen/arm: create a cpuinfo structure for guest xen/arm: Add handler for ID registers on arm64 xen/arm: Add handler for cp15 ID registers xen/arm: Add CP10 exception support to handle VMFR xen/arm: Activate TID3 in HCR_EL2

Quoted 6 lines

CI found an ARM randconfig failure against this series.

https://gitlab.com/xen-project/patchew/xen/-/pipelines/221798884

I have admit that I can't spot an obvious connection so it might be collateral damage from elsewhere, but does need looking at irrespective.

This absolutely right, there is a bug in my code and i will send a V2 to fix it.

Very nice finding, i am wondering why my tests did not point this out.

Regards
Bertrand

Quoted 2 lines

~Andrew (in lieu of a real CI robot).

#10 lore raw
Quoted 40 lines · Bertrand Marquis

Hi Andrew,

Quoted 38 lines · Andrew Cooper
Quoted 25 lines · Bertrand Marquis

The goal of this serie is to emulate coprocessor ID registers so that Xen only publish to guest features that are supported by Xen and can actually be used by guests.
One practical example where this is required are SVE support which is forbidden by Xen as it is not supported, but if Linux is compiled with it, it will crash on boot. An other one is AMU which is also forbidden by Xen but one Linux compiled with it would crash if the platform supports it.

To be able to emulate the coprocessor registers defining what features are supported by the hardware, the TID3 bit of HCR must be disabled and Xen must emulated the values of those registers when an exception is catched when a guest is accessing it.

This serie is first creating a guest cpuinfo structure which will contain the values that we want to publish to the guests and then provides the proper emulationg for those registers when Xen is getting an exception due to an access to any of those registers.

This is a first simple implementation to solve the problem and the way to define the values that we provide to guests and which features are disabled will be in a future patchset enhance so that we could decide per guest what can be used or not and depending on this deduce the bits to activate in HCR and the values that we must publish on ID registers.

Quoted 8 lines

Bertrand Marquis (7):
xen/arm: Add ID registers and complete cpufinfo xen/arm: Add arm64 ID registers definitions xen/arm: create a cpuinfo structure for guest xen/arm: Add handler for ID registers on arm64 xen/arm: Add handler for cp15 ID registers xen/arm: Add CP10 exception support to handle VMFR xen/arm: Activate TID3 in HCR_EL2

CI found an ARM randconfig failure against this series.

Quoted 7 lines
Quoted 4 lines

https://gitlab.com/xen-project/patchew/xen/-/pipelines/221798884

I have admit that I can't spot an obvious connection so it might be collateral damage from elsewhere, but does need looking at irrespective.

This absolutely right, there is a bug in my code and i will send a V2 to fix it.

Very nice finding, i am wondering why my tests did not point this out.

Its randconfig, so every time the test runs, it picks a new random Kconfig configuration.

Sadly, it is non-deterministic, and not necessarily the fault of change the test ran against.  We're probably going to have to tweak how we run these tests before the CI goes too much further.

~Andrew

#11 lore raw

Hi Andrew,

Quoted 45 lines · Andrew Cooper
Quoted 42 lines · Bertrand Marquis

Hi Andrew,

Quoted 40 lines · Andrew Cooper
Quoted 25 lines · Bertrand Marquis

The goal of this serie is to emulate coprocessor ID registers so that Xen only publish to guest features that are supported by Xen and can actually be used by guests.
One practical example where this is required are SVE support which is forbidden by Xen as it is not supported, but if Linux is compiled with it, it will crash on boot. An other one is AMU which is also forbidden by Xen but one Linux compiled with it would crash if the platform supports it.

To be able to emulate the coprocessor registers defining what features are supported by the hardware, the TID3 bit of HCR must be disabled and Xen must emulated the values of those registers when an exception is catched when a guest is accessing it.

This serie is first creating a guest cpuinfo structure which will contain the values that we want to publish to the guests and then provides the proper emulationg for those registers when Xen is getting an exception due to an access to any of those registers.

This is a first simple implementation to solve the problem and the way to define the values that we provide to guests and which features are disabled will be in a future patchset enhance so that we could decide per guest what can be used or not and depending on this deduce the bits to activate in HCR and the values that we must publish on ID registers.

Quoted 8 lines

Bertrand Marquis (7):
xen/arm: Add ID registers and complete cpufinfo xen/arm: Add arm64 ID registers definitions xen/arm: create a cpuinfo structure for guest xen/arm: Add handler for ID registers on arm64 xen/arm: Add handler for cp15 ID registers xen/arm: Add CP10 exception support to handle VMFR xen/arm: Activate TID3 in HCR_EL2

CI found an ARM randconfig failure against this series.

https://gitlab.com/xen-project/patchew/xen/-/pipelines/221798884

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

I have admit that I can't spot an obvious connection so it might be collateral damage from elsewhere, but does need looking at irrespective.

This absolutely right, there is a bug in my code and i will send a V2 to fix it.

Very nice finding, i am wondering why my tests did not point this out.

Its randconfig, so every time the test runs, it picks a new random Kconfig configuration.

This could be consider as a fuzzer testing and can be very usefull.

Quoted 4 lines

Sadly, it is non-deterministic, and not necessarily the fault of change the test ran against. We're probably going to have to tweak how we run these tests before the CI goes too much further.

Agree but in this case the error would have been triggered for any Arm configuration with the right compiler flags.

Regards
Bertrand