Skip to contentSkip to conversation

kvmppc: protection keys fix

RFC PATCH In review Ram Pai 5 participants 11 messages Jul 28, 2017 – Aug 31, 2017

Subsystems (2)
lore
Share & tools
lore

Ram Pai

author
#0 lore raw

RFC PATCH 1/1 kvmppc: protection keys fix

kvmppc_do_h_enter() hcall, clobbers the high-order two bits of the protection key in the lower pte (ptel). Hence any pkey number above 7 fails to behave.

The following patch, preserves the pkey bits.

---
 arch/powerpc/include/asm/book3s/64/mmu-hash.h |    1 +
 arch/powerpc/kvm/book3s_hv_rm_mmu.c           |    2 +-
 2 files changed, 2 insertions(+), 1 deletions(-)
arch/powerpc/include/asm/book3s/64/mmu-hash.h +1 -0
index 369f9ff..f07a1c0 100644
@@ -109,6 +109,7 @@
 #define HPTE_R_KEY_BIT2		ASM_CONST(0x0000000000000800)
 #define HPTE_R_KEY_BIT3		ASM_CONST(0x0000000000000400)
 #define HPTE_R_KEY_BIT4		ASM_CONST(0x0000000000000200)
+#define HPTE_R_KEY		(HPTE_R_KEY_LO | HPTE_R_KEY_HI)
 
 #define HPTE_V_1TB_SEG		ASM_CONST(0x4000000000000000)
 #define HPTE_V_VRMA_MASK	ASM_CONST(0x4001ffffff000000)
arch/powerpc/kvm/book3s_hv_rm_mmu.c +1 -1
index ce6f212..d9462ab 100644
@@ -268,7 +268,7 @@ long kvmppc_do_h_enter(struct kvm *kvm, unsigned long flags,
 	if (!realmode)
 		local_irq_restore(irq_flags);
 
-	ptel &= ~(HPTE_R_PP0 - psize);
+	ptel &= HPTE_R_KEY | HPTE_R_PP0 | (psize-1);
 	ptel |= pa;
 
 	if (pa)
Signature · 1 line
1.7.1
Acked-by on v1 1/1
#2 lore raw
In reply to Ram Pai
Quoted 3 lines · Ram Pai

kvmppc_do_h_enter() hcall, clobbers the high-order two bits of the protection key in the lower pte (ptel). Hence any pkey number above 7 fails to behave.

I am not sure behave is the right word.

Quoted 5 lines

The following patch, preserves the pkey bits.

---

Yep, Looks right. I would reword the change log though

#3 lore raw
In reply to Ram Pai
Quoted 29 lines · Ram Pai

kvmppc_do_h_enter() hcall, clobbers the high-order two bits of the protection key in the lower pte (ptel). Hence any pkey number above 7 fails to behave.

The following patch, preserves the pkey bits.

---
 arch/powerpc/include/asm/book3s/64/mmu-hash.h |    1 +
 arch/powerpc/kvm/book3s_hv_rm_mmu.c           |    2 +-
 2 files changed, 2 insertions(+), 1 deletions(-)
arch/powerpc/include/asm/book3s/64/mmu-hash.h +1 -0
index 369f9ff..f07a1c0 100644
@@ -109,6 +109,7 @@
 #define HPTE_R_KEY_BIT2		ASM_CONST(0x0000000000000800)
 #define HPTE_R_KEY_BIT3		ASM_CONST(0x0000000000000400)
 #define HPTE_R_KEY_BIT4		ASM_CONST(0x0000000000000200)
+#define HPTE_R_KEY		(HPTE_R_KEY_LO | HPTE_R_KEY_HI)
 
 #define HPTE_V_1TB_SEG		ASM_CONST(0x4000000000000000)
 #define HPTE_V_VRMA_MASK	ASM_CONST(0x4001ffffff000000)
arch/powerpc/kvm/book3s_hv_rm_mmu.c +0 -0
index ce6f212..d9462ab 100644

arch/powerpc/kvm/book3s_hv_rm_mmu.c long kvmppc_do_h_enter(struct kvm *kvm, unsigned long flags, · line 271

Quoted 5 lines
arch/powerpc/kvm/book3s_hv_rm_mmu.c +1 -1
@@ -268,7 +268,7 @@ long kvmppc_do_h_enter(struct kvm *kvm, unsigned long flags,
 	if (!realmode)
 		local_irq_restore(irq_flags);
 
-	ptel &= ~(HPTE_R_PP0 - psize);
+	ptel &= HPTE_R_KEY | HPTE_R_PP0 | (psize-1);

Is this all needed to get keys to work on kvm ? We use key value 31 and key value 0 along with absent and invalid hpte config to indicate mmio and host swapped pages ? Won't that need to be fixed ?

Quoted 5 lines
arch/powerpc/kvm/book3s_hv_rm_mmu.c +0 -0
 	ptel |= pa;
 
 	if (pa)
Signature · 1 line
1.7.1

-aneesh

#4 lore raw
In reply to Ram Pai
Quoted 34 lines · Aneesh Kumar K.V
Quoted 32 lines · Ram Pai

kvmppc_do_h_enter() hcall, clobbers the high-order two bits of the protection key in the lower pte (ptel). Hence any pkey number above 7 fails to behave.

The following patch, preserves the pkey bits.

---
 arch/powerpc/include/asm/book3s/64/mmu-hash.h |    1 +
 arch/powerpc/kvm/book3s_hv_rm_mmu.c           |    2 +-
 2 files changed, 2 insertions(+), 1 deletions(-)
arch/powerpc/include/asm/book3s/64/mmu-hash.h +1 -0
index 369f9ff..f07a1c0 100644
@@ -109,6 +109,7 @@
 #define HPTE_R_KEY_BIT2		ASM_CONST(0x0000000000000800)
 #define HPTE_R_KEY_BIT3		ASM_CONST(0x0000000000000400)
 #define HPTE_R_KEY_BIT4		ASM_CONST(0x0000000000000200)
+#define HPTE_R_KEY		(HPTE_R_KEY_LO | HPTE_R_KEY_HI)
 
 #define HPTE_V_1TB_SEG		ASM_CONST(0x4000000000000000)
 #define HPTE_V_VRMA_MASK	ASM_CONST(0x4001ffffff000000)
arch/powerpc/kvm/book3s_hv_rm_mmu.c +0 -0
index ce6f212..d9462ab 100644
@@ -268,7 +268,7 @@ long kvmppc_do_h_enter(struct kvm *kvm, unsigned long flags,
 	if (!realmode)
 		local_irq_restore(irq_flags);
 
Quoted 6 lines

arch/powerpc/kvm/book3s_hv_rm_mmu.c long kvmppc_do_h_enter(struct kvm *kvm, unsigned long flags, · line 271

Quoted 2 lines
arch/powerpc/kvm/book3s_hv_rm_mmu.c +1 -1
-	ptel &= ~(HPTE_R_PP0 - psize);
+	ptel &= HPTE_R_KEY | HPTE_R_PP0 | (psize-1);

Is this all needed to get keys to work on kvm ? We use key value 31 and key value 0 along with absent and invalid hpte config to indicate mmio and host swapped pages ? Won't that need to be fixed ?

It should be fine, though if you want to audit the code go ahead.

Yong Ji fixed it last year to check for key 31 only when the HPTE is absent, to clear the key when making a HPTE absent unless it is an MMIO HPTE (and then it's set to 31), and to set the key to the guest-requested key when making the HPTE present.

Paul.

author
#5 lore raw
In reply to Ram Pai
Quoted 34 lines · Aneesh Kumar K.V
Quoted 32 lines · Ram Pai

kvmppc_do_h_enter() hcall, clobbers the high-order two bits of the protection key in the lower pte (ptel). Hence any pkey number above 7 fails to behave.

The following patch, preserves the pkey bits.

---
 arch/powerpc/include/asm/book3s/64/mmu-hash.h |    1 +
 arch/powerpc/kvm/book3s_hv_rm_mmu.c           |    2 +-
 2 files changed, 2 insertions(+), 1 deletions(-)
arch/powerpc/include/asm/book3s/64/mmu-hash.h +1 -0
index 369f9ff..f07a1c0 100644
@@ -109,6 +109,7 @@
 #define HPTE_R_KEY_BIT2		ASM_CONST(0x0000000000000800)
 #define HPTE_R_KEY_BIT3		ASM_CONST(0x0000000000000400)
 #define HPTE_R_KEY_BIT4		ASM_CONST(0x0000000000000200)
+#define HPTE_R_KEY		(HPTE_R_KEY_LO | HPTE_R_KEY_HI)
 
 #define HPTE_V_1TB_SEG		ASM_CONST(0x4000000000000000)
 #define HPTE_V_VRMA_MASK	ASM_CONST(0x4001ffffff000000)
arch/powerpc/kvm/book3s_hv_rm_mmu.c +0 -0
index ce6f212..d9462ab 100644
@@ -268,7 +268,7 @@ long kvmppc_do_h_enter(struct kvm *kvm, unsigned long flags,
 	if (!realmode)
 		local_irq_restore(irq_flags);
 
Quoted 6 lines

arch/powerpc/kvm/book3s_hv_rm_mmu.c long kvmppc_do_h_enter(struct kvm *kvm, unsigned long flags, · line 271

Quoted 2 lines
arch/powerpc/kvm/book3s_hv_rm_mmu.c +1 -1
-	ptel &= ~(HPTE_R_PP0 - psize);
+	ptel &= HPTE_R_KEY | HPTE_R_PP0 | (psize-1);

Is this all needed to get keys to work on kvm ? We use key value 31 and key value 0 along with absent and invalid hpte config to indicate mmio and host swapped pages ? Won't that need to be fixed ?

The OS assumes key 0 and 1 are reserved and hence does not let applications use it. And the device tree tells us if key 31 is used or not. If it is used the OS does not let application use it either.

Now, the device tree for KVM guests do not expose the key property yet. That needs to be fixed.

RP

#6 lore raw
In reply to Ram Pai
Quoted 42 lines · Ram Pai
Quoted 34 lines · Aneesh Kumar K.V
Quoted 32 lines · Ram Pai

kvmppc_do_h_enter() hcall, clobbers the high-order two bits of the protection key in the lower pte (ptel). Hence any pkey number above 7 fails to behave.

The following patch, preserves the pkey bits.

---
 arch/powerpc/include/asm/book3s/64/mmu-hash.h |    1 +
 arch/powerpc/kvm/book3s_hv_rm_mmu.c           |    2 +-
 2 files changed, 2 insertions(+), 1 deletions(-)
arch/powerpc/include/asm/book3s/64/mmu-hash.h +1 -0
index 369f9ff..f07a1c0 100644
@@ -109,6 +109,7 @@
 #define HPTE_R_KEY_BIT2		ASM_CONST(0x0000000000000800)
 #define HPTE_R_KEY_BIT3		ASM_CONST(0x0000000000000400)
 #define HPTE_R_KEY_BIT4		ASM_CONST(0x0000000000000200)
+#define HPTE_R_KEY		(HPTE_R_KEY_LO | HPTE_R_KEY_HI)
 
 #define HPTE_V_1TB_SEG		ASM_CONST(0x4000000000000000)
 #define HPTE_V_VRMA_MASK	ASM_CONST(0x4001ffffff000000)
arch/powerpc/kvm/book3s_hv_rm_mmu.c +0 -0
index ce6f212..d9462ab 100644
@@ -268,7 +268,7 @@ long kvmppc_do_h_enter(struct kvm *kvm, unsigned long flags,
 	if (!realmode)
 		local_irq_restore(irq_flags);
 
Quoted 6 lines

arch/powerpc/kvm/book3s_hv_rm_mmu.c long kvmppc_do_h_enter(struct kvm *kvm, unsigned long flags, · line 271

Quoted 2 lines
arch/powerpc/kvm/book3s_hv_rm_mmu.c +1 -1
-	ptel &= ~(HPTE_R_PP0 - psize);
+	ptel &= HPTE_R_KEY | HPTE_R_PP0 | (psize-1);

Is this all needed to get keys to work on kvm ? We use key value 31 and key value 0 along with absent and invalid hpte config to indicate mmio and host swapped pages ? Won't that need to be fixed ?

Quoted 6 lines

The OS assumes key 0 and 1 are reserved and hence does not let applications use it. And the device tree tells us if key 31 is used or not. If it is used the OS does not let application use it either.

Now, the device tree for KVM guests do not expose the key property yet. That needs to be fixed.

I'm currently preparing a patch for that. One question: did I understand correctly that Paul Mackerras' answer on this thread¹ means that KVM doesn't need to reserve any keys for itself and can pass on 32 keys to the guest?

Signature · 4 lines
Thiago Jung Bauermann
IBM Linux Technology Center

¹ https://www.spinics.net/lists/kvm-ppc/msg12765.html
author
#7 lore raw
In reply to Ram Pai
Quoted 49 lines · Thiago Jung Bauermann
Quoted 46 lines · Ram Pai
Quoted 34 lines · Aneesh Kumar K.V
Quoted 32 lines · Ram Pai

kvmppc_do_h_enter() hcall, clobbers the high-order two bits of the protection key in the lower pte (ptel). Hence any pkey number above 7 fails to behave.

The following patch, preserves the pkey bits.

---
 arch/powerpc/include/asm/book3s/64/mmu-hash.h |    1 +
 arch/powerpc/kvm/book3s_hv_rm_mmu.c           |    2 +-
 2 files changed, 2 insertions(+), 1 deletions(-)
arch/powerpc/include/asm/book3s/64/mmu-hash.h +1 -0
index 369f9ff..f07a1c0 100644
@@ -109,6 +109,7 @@
 #define HPTE_R_KEY_BIT2		ASM_CONST(0x0000000000000800)
 #define HPTE_R_KEY_BIT3		ASM_CONST(0x0000000000000400)
 #define HPTE_R_KEY_BIT4		ASM_CONST(0x0000000000000200)
+#define HPTE_R_KEY		(HPTE_R_KEY_LO | HPTE_R_KEY_HI)
 
 #define HPTE_V_1TB_SEG		ASM_CONST(0x4000000000000000)
 #define HPTE_V_VRMA_MASK	ASM_CONST(0x4001ffffff000000)
arch/powerpc/kvm/book3s_hv_rm_mmu.c +0 -0
index ce6f212..d9462ab 100644
@@ -268,7 +268,7 @@ long kvmppc_do_h_enter(struct kvm *kvm, unsigned long flags,
 	if (!realmode)
 		local_irq_restore(irq_flags);
 
Quoted 6 lines

arch/powerpc/kvm/book3s_hv_rm_mmu.c long kvmppc_do_h_enter(struct kvm *kvm, unsigned long flags, · line 271

Quoted 2 lines
arch/powerpc/kvm/book3s_hv_rm_mmu.c +1 -1
-	ptel &= ~(HPTE_R_PP0 - psize);
+	ptel &= HPTE_R_KEY | HPTE_R_PP0 | (psize-1);

Is this all needed to get keys to work on kvm ? We use key value 31 and key value 0 along with absent and invalid hpte config to indicate mmio and host swapped pages ? Won't that need to be fixed ?

The OS assumes key 0 and 1 are reserved and hence does not let applications use it. And the device tree tells us if key 31 is used or not. If it is used the OS does not let application use it either.

Quoted 7 lines
Quoted 2 lines

Now, the device tree for KVM guests do not expose the key property yet. That needs to be fixed.

I'm currently preparing a patch for that. One question: did I understand correctly that Paul Mackerras' answer on this thread¹ means that KVM doesn't need to reserve any keys for itself and can pass on 32 keys to the guest?

No. KVM does use key 31 for emulated MMIO.

if (!(memslot && !(memslot->flags & KVM_MEMSLOT_INVALID))) {
	/* Emulated MMIO - mark this with key1 */
	pteh |= HPTE_V_ABSENT;
	ptel |= HPTE_R_KEY_HI |
	HPTE_R_KEY_LO;
	goto do_insert;
}

The device tree has to say 31 and not 32.

RP.

#8 lore raw
In reply to Ram Pai
Quoted 1 line · Ram Pai

On Thu, Aug 17, 2017 at 05:54:45PM -0300, Thiago Jung Bauermann wrote:

[snip]

Quoted 8 lines
Quoted 5 lines

I'm currently preparing a patch for that. One question: did I understand correctly that Paul Mackerras' answer on this thread¹ means that KVM doesn't need to reserve any keys for itself and can pass on 32 keys to the guest?

No. KVM does use key 31 for emulated MMIO.

Quoted 7 lines
if (!(memslot && !(memslot->flags & KVM_MEMSLOT_INVALID))) {
	/* Emulated MMIO - mark this with key1 */
	pteh |= HPTE_V_ABSENT;
	ptel |= HPTE_R_KEY_HI |
	HPTE_R_KEY_LO;
	goto do_insert;
}

Actually, the guest *can* use all 32 keys. We only use key 31 on absent HPTEs (see the HPTE_V_ABSENT flag above - these are HPTEs that the guest thinks are valid but are not actually mapped in the real HPT).

Paul.

author
#9 lore raw
In reply to Ram Pai
Quoted 19 lines · Paul Mackerras
Quoted 1 line · Ram Pai

On Thu, Aug 17, 2017 at 05:54:45PM -0300, Thiago Jung Bauermann wrote:

[snip]

Quoted 8 lines
Quoted 5 lines

I'm currently preparing a patch for that. One question: did I understand correctly that Paul Mackerras' answer on this thread¹ means that KVM doesn't need to reserve any keys for itself and can pass on 32 keys to the guest?

No. KVM does use key 31 for emulated MMIO.

Quoted 7 lines
if (!(memslot && !(memslot->flags & KVM_MEMSLOT_INVALID))) {
	/* Emulated MMIO - mark this with key1 */
	pteh |= HPTE_V_ABSENT;
	ptel |= HPTE_R_KEY_HI |
	HPTE_R_KEY_LO;
	goto do_insert;
}
Quoted 4 lines

Actually, the guest *can* use all 32 keys. We only use key 31 on absent HPTEs (see the HPTE_V_ABSENT flag above - these are HPTEs that the guest thinks are valid but are not actually mapped in the real HPT).

Ok. looks like you use the HPTE_V_ABSENT flag to determine if a key fault on key 31, needs to be redirected to the guest or handled internally.

RP

#10 lore raw
In reply to Ram Pai
Quoted 7 lines · Ram Pai

kvmppc_do_h_enter() hcall, clobbers the high-order two bits of the protection key in the lower pte (ptel). Hence any pkey number above 7 fails to behave.

The following patch, preserves the pkey bits.

Thanks, patch applied to my kvm-ppc-next branch (with rewritten patch description).

Paul.