Skip to contentSkip to conversation

arm64: Add KRYO4XX gold CPU core to spectre-v2 safe list

PATCH In review Sai Prakash Ranjan 3 participants 10 messages Aug 13, 2020 – Aug 14, 2020

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 arm64: Add KRYO4XX gold CPU core to spectre-v2 safe list

KRYO4XX gold/big CPU cores are based on Cortex-A76 which has CSV2 bits set and are spectre-v2 safe. But on big.LITTLE systems where they are coupled with other CPU cores such as the KRYO4XX silver based on Cortex-A55 which are spectre-v2 safe but do not have CSV2 bits set, the system wide safe value will be set to the lowest value of CSV2 bits as per FTR_LOWER_SAFE defined for CSV2 bits of register ID_AA64PFR0_EL1.

This is a problem when booting a guest kernel on gold CPU cores where it will incorrectly report ARM_SMCCC_ARCH_WORKAROUND_1 warning and consider them as vulnerable for Spectre variant 2 due to system wide safe value which is used in kvm emulation code when reading id registers. One wrong way of fixing this is to set the FTR_HIGHER_SAFE for CSV2 bits, so instead add the KRYO4XX gold CPU core to the safe list which will be consulted even when the sanitised read reports that CSV2 bits are not set for KRYO4XX gold cores.

---
 arch/arm64/kernel/cpu_errata.c | 1 +
 1 file changed, 1 insertion(+)
arch/arm64/kernel/cpu_errata.c +1 -0
index 6bd1d3ad037a..6cbdd2d98a2a 100644
@@ -545,6 +545,7 @@ static const struct midr_range spectre_v2_safe_list[] = {
 	MIDR_ALL_VERSIONS(MIDR_HISI_TSV110),
 	MIDR_ALL_VERSIONS(MIDR_QCOM_KRYO_3XX_SILVER),
 	MIDR_ALL_VERSIONS(MIDR_QCOM_KRYO_4XX_SILVER),
+	MIDR_ALL_VERSIONS(MIDR_QCOM_KRYO_4XX_GOLD),
 	{ /* sentinel */ }
 };
 
Signature · 2 lines
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member
of Code Aurora Forum, hosted by The Linux Foundation
maintainer
#1 lore raw
Quoted 28 lines · Sai Prakash Ranjan

KRYO4XX gold/big CPU cores are based on Cortex-A76 which has CSV2 bits set and are spectre-v2 safe. But on big.LITTLE systems where they are coupled with other CPU cores such as the KRYO4XX silver based on Cortex-A55 which are spectre-v2 safe but do not have CSV2 bits set, the system wide safe value will be set to the lowest value of CSV2 bits as per FTR_LOWER_SAFE defined for CSV2 bits of register ID_AA64PFR0_EL1.

This is a problem when booting a guest kernel on gold CPU cores where it will incorrectly report ARM_SMCCC_ARCH_WORKAROUND_1 warning and consider them as vulnerable for Spectre variant 2 due to system wide safe value which is used in kvm emulation code when reading id registers. One wrong way of fixing this is to set the FTR_HIGHER_SAFE for CSV2 bits, so instead add the KRYO4XX gold CPU core to the safe list which will be consulted even when the sanitised read reports that CSV2 bits are not set for KRYO4XX gold cores.

---
 arch/arm64/kernel/cpu_errata.c | 1 +
 1 file changed, 1 insertion(+)
arch/arm64/kernel/cpu_errata.c +0 -0
index 6bd1d3ad037a..6cbdd2d98a2a 100644

arch/arm64/kernel/cpu_errata.c static const struct midr_range spectre_v2_safe_list[] = { · line 548

Quoted 4 lines
arch/arm64/kernel/cpu_errata.c +1 -0
@@ -545,6 +545,7 @@ static const struct midr_range spectre_v2_safe_list[] = {
 	MIDR_ALL_VERSIONS(MIDR_HISI_TSV110),
 	MIDR_ALL_VERSIONS(MIDR_QCOM_KRYO_3XX_SILVER),
 	MIDR_ALL_VERSIONS(MIDR_QCOM_KRYO_4XX_SILVER),
+	MIDR_ALL_VERSIONS(MIDR_QCOM_KRYO_4XX_GOLD),

We shouldn't be putting CPUs in the safe list when they have CSV2 reporting that they are mitigated in hardware, so I don't think this is the right approach.

Sounds more like KVM should advertise CSV2 for the vCPUs if all of the physical CPUs without CSV2 set are on the safe list. But then again, KVM has always been slightly in denial about big.LITTLE because you can't sensibly expose it to a guest if there are detectable differences...

Will

#2 lore raw
Quoted 24 lines · Will Deacon
Quoted 23 lines · Sai Prakash Ranjan

KRYO4XX gold/big CPU cores are based on Cortex-A76 which has CSV2 bits set and are spectre-v2 safe. But on big.LITTLE systems where they are coupled with other CPU cores such as the KRYO4XX silver based on Cortex-A55 which are spectre-v2 safe but do not have CSV2 bits set, the system wide safe value will be set to the lowest value of CSV2 bits as per FTR_LOWER_SAFE defined for CSV2 bits of register ID_AA64PFR0_EL1.

This is a problem when booting a guest kernel on gold CPU cores where it will incorrectly report ARM_SMCCC_ARCH_WORKAROUND_1 warning and consider them as vulnerable for Spectre variant 2 due to system wide safe value which is used in kvm emulation code when reading id registers. One wrong way of fixing this is to set the FTR_HIGHER_SAFE for CSV2 bits, so instead add the KRYO4XX gold CPU core to the safe list which will be consulted even when the sanitised read reports that CSV2 bits are not set for KRYO4XX gold cores.

---
 arch/arm64/kernel/cpu_errata.c | 1 +
 1 file changed, 1 insertion(+)
Quoted 15 lines

arch/arm64/kernel/cpu_errata.c static const struct midr_range · line 549

Quoted 10 lines
arch/arm64/kernel/cpu_errata.c +1 -0
index 6bd1d3ad037a..6cbdd2d98a2a 100644
@@ -545,6 +545,7 @@ static const struct midr_range 
 spectre_v2_safe_list[] = {
 	MIDR_ALL_VERSIONS(MIDR_HISI_TSV110),
 	MIDR_ALL_VERSIONS(MIDR_QCOM_KRYO_3XX_SILVER),
 	MIDR_ALL_VERSIONS(MIDR_QCOM_KRYO_4XX_SILVER),
+	MIDR_ALL_VERSIONS(MIDR_QCOM_KRYO_4XX_GOLD),

We shouldn't be putting CPUs in the safe list when they have CSV2 reporting that they are mitigated in hardware, so I don't think this is the right approach.

Ok but the only thing I find wrong in this approach is that it is a redundant information because CSV2 is already advertising the mitigation, but again CSV2 check is done first so it doesn't really hurt to add it to the safe list because we already know that it is safe.

Quoted 5 lines

Sounds more like KVM should advertise CSV2 for the vCPUs if all of the physical CPUs without CSV2 set are on the safe list. But then again, KVM has always been slightly in denial about big.LITTLE because you can't sensibly expose it to a guest if there are detectable differences...

Sorry but I don't see how the guest kernel will see the CSV2 bits set for gold CPU cores without actually adding them to the safe list or reading the not sanitised value of ID_AA64PFR0_EL1 ?

Thanks,
Sai

Signature · 2 lines
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member
of Code Aurora Forum, hosted by The Linux Foundation
maintainer
#3 lore raw
Quoted 43 lines · Sai Prakash Ranjan
Quoted 24 lines · Will Deacon
Quoted 23 lines · Sai Prakash Ranjan

KRYO4XX gold/big CPU cores are based on Cortex-A76 which has CSV2 bits set and are spectre-v2 safe. But on big.LITTLE systems where they are coupled with other CPU cores such as the KRYO4XX silver based on Cortex-A55 which are spectre-v2 safe but do not have CSV2 bits set, the system wide safe value will be set to the lowest value of CSV2 bits as per FTR_LOWER_SAFE defined for CSV2 bits of register ID_AA64PFR0_EL1.

This is a problem when booting a guest kernel on gold CPU cores where it will incorrectly report ARM_SMCCC_ARCH_WORKAROUND_1 warning and consider them as vulnerable for Spectre variant 2 due to system wide safe value which is used in kvm emulation code when reading id registers. One wrong way of fixing this is to set the FTR_HIGHER_SAFE for CSV2 bits, so instead add the KRYO4XX gold CPU core to the safe list which will be consulted even when the sanitised read reports that CSV2 bits are not set for KRYO4XX gold cores.

---
 arch/arm64/kernel/cpu_errata.c | 1 +
 1 file changed, 1 insertion(+)
Quoted 17 lines

arch/arm64/kernel/cpu_errata.c static const struct midr_range · line 549

Quoted 11 lines
arch/arm64/kernel/cpu_errata.c +1 -0
index 6bd1d3ad037a..6cbdd2d98a2a 100644
@@ -545,6 +545,7 @@ static const struct midr_range
 spectre_v2_safe_list[] = {
 	MIDR_ALL_VERSIONS(MIDR_HISI_TSV110),
 	MIDR_ALL_VERSIONS(MIDR_QCOM_KRYO_3XX_SILVER),
 	MIDR_ALL_VERSIONS(MIDR_QCOM_KRYO_4XX_SILVER),
+	MIDR_ALL_VERSIONS(MIDR_QCOM_KRYO_4XX_GOLD),

We shouldn't be putting CPUs in the safe list when they have CSV2 reporting
that they are mitigated in hardware, so I don't think this is the right approach.

Quoted 5 lines

Ok but the only thing I find wrong in this approach is that it is a redundant
information because CSV2 is already advertising the mitigation, but again CSV2 check is done first so it doesn't really hurt to add it to the safe list because we already know that it is safe.

It simply doesn't scale. That's why CSV2 exists in the first place, so we don't have to modify the kernel everytime a new CPU is invented.

Quoted 9 lines
Quoted 5 lines

Sounds more like KVM should advertise CSV2 for the vCPUs if all of the physical CPUs without CSV2 set are on the safe list. But then again, KVM has always been slightly in denial about big.LITTLE because you can't sensibly expose it to a guest if there are detectable differences...

Sorry but I don't see how the guest kernel will see the CSV2 bits set for gold CPU cores without actually adding them to the safe list or reading the not sanitised value of ID_AA64PFR0_EL1 ?

Well that's for somebody to figure out in the patch. I'm just saying that adding cores to the safe list when they already have a CSV2 field conveying the same information is the wrong approach. The right appproach is for KVM to expose CSV2 as set when the system is not affected by the erratum.

FWIW, I'll be sending some patches soon to clean a lot of this up, so maybe it will become clearer after that.

Will

maintainer
#4 lore raw
Quoted 59 lines · Will Deacon
Quoted 43 lines · Sai Prakash Ranjan
Quoted 24 lines · Will Deacon
Quoted 23 lines · Sai Prakash Ranjan

KRYO4XX gold/big CPU cores are based on Cortex-A76 which has CSV2 bits set and are spectre-v2 safe. But on big.LITTLE systems where they are coupled with other CPU cores such as the KRYO4XX silver based on Cortex-A55 which are spectre-v2 safe but do not have CSV2 bits set, the system wide safe value will be set to the lowest value of CSV2 bits as per FTR_LOWER_SAFE defined for CSV2 bits of register ID_AA64PFR0_EL1.

This is a problem when booting a guest kernel on gold CPU cores where it will incorrectly report ARM_SMCCC_ARCH_WORKAROUND_1 warning and consider them as vulnerable for Spectre variant 2 due to system wide safe value which is used in kvm emulation code when reading id registers. One wrong way of fixing this is to set the FTR_HIGHER_SAFE for CSV2 bits, so instead add the KRYO4XX gold CPU core to the safe list which will be consulted even when the sanitised read reports that CSV2 bits are not set for KRYO4XX gold cores.

---
 arch/arm64/kernel/cpu_errata.c | 1 +
 1 file changed, 1 insertion(+)
Quoted 17 lines

arch/arm64/kernel/cpu_errata.c static const struct midr_range · line 549

Quoted 11 lines
arch/arm64/kernel/cpu_errata.c +1 -0
index 6bd1d3ad037a..6cbdd2d98a2a 100644
@@ -545,6 +545,7 @@ static const struct midr_range
 spectre_v2_safe_list[] = {
 	MIDR_ALL_VERSIONS(MIDR_HISI_TSV110),
 	MIDR_ALL_VERSIONS(MIDR_QCOM_KRYO_3XX_SILVER),
 	MIDR_ALL_VERSIONS(MIDR_QCOM_KRYO_4XX_SILVER),
+	MIDR_ALL_VERSIONS(MIDR_QCOM_KRYO_4XX_GOLD),

We shouldn't be putting CPUs in the safe list when they have CSV2 reporting
that they are mitigated in hardware, so I don't think this is the right approach.

Quoted 5 lines

Ok but the only thing I find wrong in this approach is that it is a redundant
information because CSV2 is already advertising the mitigation, but again CSV2 check is done first so it doesn't really hurt to add it to the safe list because we already know that it is safe.

It simply doesn't scale. That's why CSV2 exists in the first place, so we don't have to modify the kernel everytime a new CPU is invented.

Quoted 6 lines
Quoted 5 lines

Sounds more like KVM should advertise CSV2 for the vCPUs if all of the physical CPUs without CSV2 set are on the safe list. But then again, KVM has always been slightly in denial about big.LITTLE because you can't sensibly expose it to a guest if there are detectable differences...

Quoted 8 lines
Quoted 3 lines

Sorry but I don't see how the guest kernel will see the CSV2 bits set for gold CPU cores without actually adding them to the safe list or reading the not sanitised value of ID_AA64PFR0_EL1 ?

Well that's for somebody to figure out in the patch. I'm just saying that adding cores to the safe list when they already have a CSV2 field conveying the same information is the wrong approach. The right appproach is for KVM to expose CSV2 as set when the system is not affected by the erratum.

A sensible way to fix this would be with something like that:

arch/arm64/kvm/sys_regs.c +3 -0
index 077293b5115f..2735db21ff0d 100644
@@ -1131,6 +1131,9 @@ static u64 read_id_reg(const struct kvm_vcpu *vcpu,
 		if (!vcpu_has_sve(vcpu))
 			val &= ~(0xfUL << ID_AA64PFR0_SVE_SHIFT);
 		val &= ~(0xfUL << ID_AA64PFR0_AMU_SHIFT);
+		if (!(val & (0xfUL << ID_AA64PFR0_CSV2_SHIFT)) &&
+		    get_spectre_v2_workaround_state() == ARM64_BP_HARDEN_NOT_REQUIRED)
+			val |= (1UL << ID_AA64PFR0_CSV2_SHIFT);
 	} else if (id == SYS_ID_AA64ISAR1_EL1 && !vcpu_has_ptrauth(vcpu)) {
 		val &= ~((0xfUL << ID_AA64ISAR1_APA_SHIFT) |
 			 (0xfUL << ID_AA64ISAR1_API_SHIFT) |
        M.
Signature · 1 line
Jazz is not dead. It just smells funny...
author Tested-by
#5 lore raw
Quoted 86 lines · Marc Zyngier
Quoted 59 lines · Will Deacon
Quoted 43 lines · Sai Prakash Ranjan
Quoted 24 lines · Will Deacon
Quoted 23 lines · Sai Prakash Ranjan

KRYO4XX gold/big CPU cores are based on Cortex-A76 which has CSV2 bits set and are spectre-v2 safe. But on big.LITTLE systems where they are coupled with other CPU cores such as the KRYO4XX silver based on Cortex-A55 which are spectre-v2 safe but do not have CSV2 bits set, the system wide safe value will be set to the lowest value of CSV2 bits as per FTR_LOWER_SAFE defined for CSV2 bits of register ID_AA64PFR0_EL1.

This is a problem when booting a guest kernel on gold CPU cores where it will incorrectly report ARM_SMCCC_ARCH_WORKAROUND_1 warning and consider them as vulnerable for Spectre variant 2 due to system wide safe value which is used in kvm emulation code when reading id registers. One wrong way of fixing this is to set the FTR_HIGHER_SAFE for CSV2 bits, so instead add the KRYO4XX gold CPU core to the safe list which will be consulted even when the sanitised read reports that CSV2 bits are not set for KRYO4XX gold cores.

---
 arch/arm64/kernel/cpu_errata.c | 1 +
 1 file changed, 1 insertion(+)
Quoted 17 lines

arch/arm64/kernel/cpu_errata.c static const struct midr_range · line 549

Quoted 11 lines
arch/arm64/kernel/cpu_errata.c +1 -0
index 6bd1d3ad037a..6cbdd2d98a2a 100644
@@ -545,6 +545,7 @@ static const struct midr_range
 spectre_v2_safe_list[] = {
 	MIDR_ALL_VERSIONS(MIDR_HISI_TSV110),
 	MIDR_ALL_VERSIONS(MIDR_QCOM_KRYO_3XX_SILVER),
 	MIDR_ALL_VERSIONS(MIDR_QCOM_KRYO_4XX_SILVER),
+	MIDR_ALL_VERSIONS(MIDR_QCOM_KRYO_4XX_GOLD),

We shouldn't be putting CPUs in the safe list when they have CSV2 reporting
that they are mitigated in hardware, so I don't think this is the right approach.

Quoted 5 lines

Ok but the only thing I find wrong in this approach is that it is a redundant
information because CSV2 is already advertising the mitigation, but again CSV2 check is done first so it doesn't really hurt to add it to the safe list because we already know that it is safe.

It simply doesn't scale. That's why CSV2 exists in the first place, so we don't have to modify the kernel everytime a new CPU is invented.

Quoted 6 lines
Quoted 5 lines

Sounds more like KVM should advertise CSV2 for the vCPUs if all of the physical CPUs without CSV2 set are on the safe list. But then again, KVM has always been slightly in denial about big.LITTLE because you can't sensibly expose it to a guest if there are detectable differences...

Quoted 8 lines
Quoted 3 lines

Sorry but I don't see how the guest kernel will see the CSV2 bits set for gold CPU cores without actually adding them to the safe list or reading the not sanitised value of ID_AA64PFR0_EL1 ?

Well that's for somebody to figure out in the patch. I'm just saying that adding cores to the safe list when they already have a CSV2 field conveying the same information is the wrong approach. The right appproach is for KVM to expose CSV2 as set when the system is not affected by the erratum.

A sensible way to fix this would be with something like that:

arch/arm64/kvm/sys_regs.c +3 -0
index 077293b5115f..2735db21ff0d 100644
@@ -1131,6 +1131,9 @@ static u64 read_id_reg(const struct kvm_vcpu 
 *vcpu,
 		if (!vcpu_has_sve(vcpu))
 			val &= ~(0xfUL << ID_AA64PFR0_SVE_SHIFT);
 		val &= ~(0xfUL << ID_AA64PFR0_AMU_SHIFT);
+		if (!(val & (0xfUL << ID_AA64PFR0_CSV2_SHIFT)) &&
+		    get_spectre_v2_workaround_state() == 
 ARM64_BP_HARDEN_NOT_REQUIRED)
+			val |= (1UL << ID_AA64PFR0_CSV2_SHIFT);
 	} else if (id == SYS_ID_AA64ISAR1_EL1 && !vcpu_has_ptrauth(vcpu)) {

arch/arm64/kvm/sys_regs.c static u64 read_id_reg(const struct kvm_vcpu *vcpu, · line 1139

Quoted 2 lines
arch/arm64/kvm/sys_regs.c +0 -0
 		val &= ~((0xfUL << ID_AA64ISAR1_APA_SHIFT) |
 			 (0xfUL << ID_AA64ISAR1_API_SHIFT) |

Thanks Marc, I gave this a go on SC7180 where the issue was seen and this fix is good.

Thanks,
Sai

Signature · 2 lines
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member
of Code Aurora Forum, hosted by The Linux Foundation
maintainer
#6 lore raw
Quoted 93 lines · Sai Prakash Ranjan
Quoted 86 lines · Marc Zyngier
Quoted 59 lines · Will Deacon
Quoted 43 lines · Sai Prakash Ranjan
Quoted 24 lines · Will Deacon
Quoted 23 lines · Sai Prakash Ranjan

KRYO4XX gold/big CPU cores are based on Cortex-A76 which has CSV2 bits set and are spectre-v2 safe. But on big.LITTLE systems where they are coupled with other CPU cores such as the KRYO4XX silver based on Cortex-A55 which are spectre-v2 safe but do not have CSV2 bits set, the system wide safe value will be set to the lowest value of CSV2 bits as per FTR_LOWER_SAFE defined for CSV2 bits of register ID_AA64PFR0_EL1.

This is a problem when booting a guest kernel on gold CPU cores where it will incorrectly report ARM_SMCCC_ARCH_WORKAROUND_1 warning and consider them as vulnerable for Spectre variant 2 due to system wide safe value which is used in kvm emulation code when reading id registers. One wrong way of fixing this is to set the FTR_HIGHER_SAFE for CSV2 bits, so instead add the KRYO4XX gold CPU core to the safe list which will be consulted even when the sanitised read reports that CSV2 bits are not set for KRYO4XX gold cores.

---
 arch/arm64/kernel/cpu_errata.c | 1 +
 1 file changed, 1 insertion(+)
Quoted 17 lines

arch/arm64/kernel/cpu_errata.c static const struct midr_range · line 549

Quoted 11 lines
arch/arm64/kernel/cpu_errata.c +1 -0
index 6bd1d3ad037a..6cbdd2d98a2a 100644
@@ -545,6 +545,7 @@ static const struct midr_range
 spectre_v2_safe_list[] = {
 	MIDR_ALL_VERSIONS(MIDR_HISI_TSV110),
 	MIDR_ALL_VERSIONS(MIDR_QCOM_KRYO_3XX_SILVER),
 	MIDR_ALL_VERSIONS(MIDR_QCOM_KRYO_4XX_SILVER),
+	MIDR_ALL_VERSIONS(MIDR_QCOM_KRYO_4XX_GOLD),

We shouldn't be putting CPUs in the safe list when they have CSV2 reporting
that they are mitigated in hardware, so I don't think this is the right approach.

Quoted 5 lines

Ok but the only thing I find wrong in this approach is that it is a redundant
information because CSV2 is already advertising the mitigation, but again CSV2 check is done first so it doesn't really hurt to add it to the safe list because we already know that it is safe.

It simply doesn't scale. That's why CSV2 exists in the first place, so we don't have to modify the kernel everytime a new CPU is invented.

Quoted 6 lines
Quoted 5 lines

Sounds more like KVM should advertise CSV2 for the vCPUs if all of the physical CPUs without CSV2 set are on the safe list. But then again, KVM has always been slightly in denial about big.LITTLE because you can't sensibly expose it to a guest if there are detectable differences...

Quoted 8 lines
Quoted 3 lines

Sorry but I don't see how the guest kernel will see the CSV2 bits set for gold CPU cores without actually adding them to the safe list or reading the not sanitised value of ID_AA64PFR0_EL1 ?

Well that's for somebody to figure out in the patch. I'm just saying that adding cores to the safe list when they already have a CSV2 field conveying the same information is the wrong approach. The right appproach is for KVM to expose CSV2 as set when the system is not affected by the erratum.

A sensible way to fix this would be with something like that:

arch/arm64/kvm/sys_regs.c +3 -0
index 077293b5115f..2735db21ff0d 100644
@@ -1131,6 +1131,9 @@ static u64 read_id_reg(const struct kvm_vcpu 
 *vcpu,
 		if (!vcpu_has_sve(vcpu))
 			val &= ~(0xfUL << ID_AA64PFR0_SVE_SHIFT);
 		val &= ~(0xfUL << ID_AA64PFR0_AMU_SHIFT);
+		if (!(val & (0xfUL << ID_AA64PFR0_CSV2_SHIFT)) &&
+		    get_spectre_v2_workaround_state() == 
 ARM64_BP_HARDEN_NOT_REQUIRED)
+			val |= (1UL << ID_AA64PFR0_CSV2_SHIFT);
 	} else if (id == SYS_ID_AA64ISAR1_EL1 && !vcpu_has_ptrauth(vcpu)) {

arch/arm64/kvm/sys_regs.c static u64 read_id_reg(const struct kvm_vcpu *vcpu, · line 1139

Quoted 2 lines
arch/arm64/kvm/sys_regs.c +0 -0
 		val &= ~((0xfUL << ID_AA64ISAR1_APA_SHIFT) |
 			 (0xfUL << ID_AA64ISAR1_API_SHIFT) |

Thanks Marc, I gave this a go on SC7180 where the issue was seen and this fix is good.

Quoted 1 line

There is still a problem with this approach. A late CPU could come up after a guest has been started. If that CPU identified as vulnerable by the host kernel, get_spectre_v2_workaround_state() will return a different value, breaking the guest (or more likely, leaving it exposed to Spectre-v2 attacks).

We'd need to disable the late onlining of CPUs that would change the mitigation state, and this is... ugly.

        M.
Signature · 1 line
Jazz is not dead. It just smells funny...
#7 lore raw
Quoted 96 lines · Marc Zyngier
Quoted 93 lines · Sai Prakash Ranjan
Quoted 86 lines · Marc Zyngier
Quoted 59 lines · Will Deacon
Quoted 43 lines · Sai Prakash Ranjan
Quoted 24 lines · Will Deacon
Quoted 23 lines · Sai Prakash Ranjan

KRYO4XX gold/big CPU cores are based on Cortex-A76 which has CSV2 bits set and are spectre-v2 safe. But on big.LITTLE systems where they are coupled with other CPU cores such as the KRYO4XX silver based on Cortex-A55 which are spectre-v2 safe but do not have CSV2 bits set, the system wide safe value will be set to the lowest value of CSV2 bits as per FTR_LOWER_SAFE defined for CSV2 bits of register ID_AA64PFR0_EL1.

This is a problem when booting a guest kernel on gold CPU cores where it will incorrectly report ARM_SMCCC_ARCH_WORKAROUND_1 warning and consider them as vulnerable for Spectre variant 2 due to system wide safe value which is used in kvm emulation code when reading id registers. One wrong way of fixing this is to set the FTR_HIGHER_SAFE for CSV2 bits, so instead add the KRYO4XX gold CPU core to the safe list which will be consulted even when the sanitised read reports that CSV2 bits are not set for KRYO4XX gold cores.

---
 arch/arm64/kernel/cpu_errata.c | 1 +
 1 file changed, 1 insertion(+)
Quoted 17 lines

arch/arm64/kernel/cpu_errata.c static const struct midr_range · line 549

Quoted 11 lines
arch/arm64/kernel/cpu_errata.c +1 -0
index 6bd1d3ad037a..6cbdd2d98a2a 100644
@@ -545,6 +545,7 @@ static const struct midr_range
 spectre_v2_safe_list[] = {
 	MIDR_ALL_VERSIONS(MIDR_HISI_TSV110),
 	MIDR_ALL_VERSIONS(MIDR_QCOM_KRYO_3XX_SILVER),
 	MIDR_ALL_VERSIONS(MIDR_QCOM_KRYO_4XX_SILVER),
+	MIDR_ALL_VERSIONS(MIDR_QCOM_KRYO_4XX_GOLD),

We shouldn't be putting CPUs in the safe list when they have CSV2 reporting
that they are mitigated in hardware, so I don't think this is the right approach.

Quoted 5 lines

Ok but the only thing I find wrong in this approach is that it is a redundant
information because CSV2 is already advertising the mitigation, but again CSV2 check is done first so it doesn't really hurt to add it to the safe list because we already know that it is safe.

It simply doesn't scale. That's why CSV2 exists in the first place, so we don't have to modify the kernel everytime a new CPU is invented.

Quoted 6 lines
Quoted 5 lines

Sounds more like KVM should advertise CSV2 for the vCPUs if all of the physical CPUs without CSV2 set are on the safe list. But then again, KVM has always been slightly in denial about big.LITTLE because you can't sensibly expose it to a guest if there are detectable differences...

Quoted 8 lines
Quoted 3 lines

Sorry but I don't see how the guest kernel will see the CSV2 bits set for gold CPU cores without actually adding them to the safe list or reading the not sanitised value of ID_AA64PFR0_EL1 ?

Well that's for somebody to figure out in the patch. I'm just saying that adding cores to the safe list when they already have a CSV2 field conveying the same information is the wrong approach. The right appproach is for KVM to expose CSV2 as set when the system is not affected by the erratum.

A sensible way to fix this would be with something like that:

arch/arm64/kvm/sys_regs.c +3 -0
index 077293b5115f..2735db21ff0d 100644
@@ -1131,6 +1131,9 @@ static u64 read_id_reg(const struct kvm_vcpu 
 *vcpu,
 		if (!vcpu_has_sve(vcpu))
 			val &= ~(0xfUL << ID_AA64PFR0_SVE_SHIFT);
 		val &= ~(0xfUL << ID_AA64PFR0_AMU_SHIFT);
+		if (!(val & (0xfUL << ID_AA64PFR0_CSV2_SHIFT)) &&
+		    get_spectre_v2_workaround_state() == 
 ARM64_BP_HARDEN_NOT_REQUIRED)
+			val |= (1UL << ID_AA64PFR0_CSV2_SHIFT);
 	} else if (id == SYS_ID_AA64ISAR1_EL1 && !vcpu_has_ptrauth(vcpu)) {

arch/arm64/kvm/sys_regs.c static u64 read_id_reg(const struct kvm_vcpu *vcpu, · line 1139

Quoted 2 lines
arch/arm64/kvm/sys_regs.c +0 -0
 		val &= ~((0xfUL << ID_AA64ISAR1_APA_SHIFT) |
 			 (0xfUL << ID_AA64ISAR1_API_SHIFT) |

Thanks Marc, I gave this a go on SC7180 where the issue was seen and this fix is good.

Quoted 1 line
Quoted 9 lines

There is still a problem with this approach. A late CPU could come up after a guest has been started. If that CPU identified as vulnerable by the host kernel, get_spectre_v2_workaround_state() will return a different value, breaking the guest (or more likely, leaving it exposed to Spectre-v2 attacks).

We'd need to disable the late onlining of CPUs that would change the mitigation state, and this is... ugly.

Ugh, yes indeed and here I was thinking that these things are straightforward :(

Thanks,
Sai

Signature · 2 lines
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member
of Code Aurora Forum, hosted by The Linux Foundation
maintainer
#8 lore raw
Quoted 1 line · Sai Prakash Ranjan

On 2020-08-13 23:29, Marc Zyngier wrote:

[...]

Quoted 5 lines
Quoted 3 lines

We'd need to disable the late onlining of CPUs that would change the mitigation state, and this is... ugly.

Ugh, yes indeed and here I was thinking that these things are straightforward :(

It never is. Big-little is a broken concept. I'll try and think of something next week, as I'm pretty swamped at the moment.

        M.
Signature · 1 line
Jazz is not dead. It just smells funny...
#9 lore raw
Quoted 5 lines · Marc Zyngier
Quoted 1 line · Sai Prakash Ranjan

On 2020-08-13 23:29, Marc Zyngier wrote:

[...]

Quoted 9 lines
Quoted 5 lines
Quoted 3 lines

We'd need to disable the late onlining of CPUs that would change the mitigation state, and this is... ugly.

Ugh, yes indeed and here I was thinking that these things are straightforward :(

It never is. Big-little is a broken concept. I'll try and think of something next week, as I'm pretty swamped at the moment.

Sure no problem, thanks.

-Sai

Signature · 2 lines
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member
of Code Aurora Forum, hosted by The Linux Foundation