x86/xen: avoid 32-bit writes to PTEs in PV PAE guests
Subsystems (2)
Juergen Gross
maintainercover x86/xen: avoid 32-bit writes to PTEs in PV PAE guests
While the hypervisor emulates plain writes to PTEs happily, this is much slower than issuing a hypercall for PTE modifcations. And writing a PTE via two 32-bit write instructions (especially when clearing the PTE) will result in an intermediate L1TF vulnerable PTE.
Writes to PAE PTEs should always be done with 64-bit writes or via hypercalls.
Juergen Gross (2):
x86/xen: don't write ptes directly in 32-bit PV guests x86/pae: use 64 bit atomic xchg function in native_ptep_get_and_clear
arch/x86/include/asm/pgtable-3level.h | 14 ++++++++------ arch/x86/xen/mmu_pv.c | 7 +++---- 2 files changed, 11 insertions(+), 10 deletions(-)
Signature · 1 line
2.13.7
Juergen Gross
maintainerPATCH 1/2 x86/xen: don't write ptes directly in 32-bit PV guests
In some cases 32-bit PAE PV guests still write PTEs directly instead of using hypercalls. This is especially bad when clearing a PTE as this is done via 32-bit writes which will produce intermediate L1TF attackable PTEs.
Change the code to use hypercalls instead.
--- arch/x86/xen/mmu_pv.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-)
arch/x86/xen/mmu_pv.c +3 -4
@@ -434,14 +434,13 @@ static void xen_set_pud(pud_t *ptr, pud_t val)static void xen_set_pte_atomic (pte_t *ptep,pte_t pte) {trace_xen_mmu_set_pte_atomic (ptep, pte); -set_64bit ((u64 *)ptep,native_pte_val (pte)); +__xen_set_pte (ptep, pte); }static void xen_pte_clear (struct mm_struct *mm,unsigned long addr,pte_t *ptep) {trace_xen_mmu_pte_clear (mm, addr, ptep); -if (!xen_batched_set_pte (ptep,native_make_pte (0 ))) -native_pte_clear (mm, addr, ptep); +__xen_set_pte (ptep,native_make_pte (0 )); }static void xen_pmd_clear (pmd_t *pmdp)
@@ -1569,7 +1568,7 @@ static void __init xen_set_pte_init(pte_t *ptep, pte_t pte) pte =__pte_ma (((pte_val_ma (*ptep) & _PAGE_RW) | ~_PAGE_RW) &pte_val_ma (pte));#endif -native_set_pte (ptep, pte); +__xen_set_pte (ptep, pte); }/* Early in boot, while setting up the initial pagetable, assume
Signature · 7 lines
2.13.7 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Juergen Gross
maintainerPATCH 1/2 x86/xen: don't write ptes directly in 32-bit PV guests
In reply to Juergen GrossIn some cases 32-bit PAE PV guests still write PTEs directly instead of using hypercalls. This is especially bad when clearing a PTE as this is done via 32-bit writes which will produce intermediate L1TF attackable PTEs.
Change the code to use hypercalls instead.
--- arch/x86/xen/mmu_pv.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-)
arch/x86/xen/mmu_pv.c +3 -4
@@ -434,14 +434,13 @@ static void xen_set_pud(pud_t *ptr, pud_t val)static void xen_set_pte_atomic (pte_t *ptep,pte_t pte) {trace_xen_mmu_set_pte_atomic (ptep, pte); -set_64bit ((u64 *)ptep,native_pte_val (pte)); +__xen_set_pte (ptep, pte); }static void xen_pte_clear (struct mm_struct *mm,unsigned long addr,pte_t *ptep) {trace_xen_mmu_pte_clear (mm, addr, ptep); -if (!xen_batched_set_pte (ptep,native_make_pte (0 ))) -native_pte_clear (mm, addr, ptep); +__xen_set_pte (ptep,native_make_pte (0 )); }static void xen_pmd_clear (pmd_t *pmdp)
@@ -1569,7 +1568,7 @@ static void __init xen_set_pte_init(pte_t *ptep, pte_t pte) pte =__pte_ma (((pte_val_ma (*ptep) & _PAGE_RW) | ~_PAGE_RW) &pte_val_ma (pte));#endif -native_set_pte (ptep, pte); +__xen_set_pte (ptep, pte); }/* Early in boot, while setting up the initial pagetable, assume
Signature · 1 line
2.13.7
Re: [PATCH 1/2] x86/xen: don't write ptes directly in 32-bit PV guests
Quoted 9 lines
Quoted 1 line
Quoted 1 line · On 20.08.18 at 07:14, <[email protected]> wrote:
In some cases 32-bit PAE PV guests still write PTEs directly instead of using hypercalls. This is especially bad when clearing a PTE as this is done via 32-bit writes which will produce intermediate L1TF attackable PTEs.
Change the code to use hypercalls instead.
Signed-off-by Juergen Gross <[email protected]>
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH 1/2] x86/xen: don't write ptes directly in 32-bit PV guests
In reply to Juergen GrossQuoted 9 lines
Quoted 1 line
Quoted 1 line · On 20.08.18 at 07:14, <[email protected]> wrote:
In some cases 32-bit PAE PV guests still write PTEs directly instead of using hypercalls. This is especially bad when clearing a PTE as this is done via 32-bit writes which will produce intermediate L1TF attackable PTEs.
Change the code to use hypercalls instead.
Signed-off-by Juergen Gross <[email protected]>
Juergen Gross
maintainerPATCH 2/2 x86/pae: use 64 bit atomic xchg function in native_ptep_get_and_clear
In reply to Juergen GrossUsing only 32-bit writes for the pte will result in an intermediate L1TF vulnerable PTE. When running as a Xen PV guest this will at once switch the guest to shadow mode resulting in a loss of performance.
Use arch_atomic64_xchg() instead which will perform the requested operation atomically with all 64 bits.
Some performance considerations according to:
The main number should be the latency, as there is no tight loop around native_ptep_get_and_clear().
"lock cmpxchg8b" has a latency of 20 cycles, while "lock xchg" (with a memory operand) isn't mentioned in that document. "lock xadd" (with xadd having 3 cycles less latency than xchg) has a latency of 11, so we can assume a latency of 14 for "lock xchg".
---
In case adding about 6 cycles for native_ptep_get_and_clear() is believed to be too bad I can modify the patch to add a paravirt function for that purpose in order to add the overhead for Xen guests only (in fact the overhead for Xen guests will be less, as only one instruction writing to the PTE has to be emulated by the hypervisor).
--- arch/x86/include/asm/pgtable-3level.h | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-)
arch/x86/include/asm/pgtable-3level.h +8 -6
@@ -2,6 +2,8 @@#ifndef _ASM_X86_PGTABLE_3LEVEL_H#define _ASM_X86_PGTABLE_3LEVEL_H +#include <asm/atomic64_32.h> +/* * Intel Physical Address Extension (PAE) Mode - three-level page * tables on PPro+ CPUs.
@@ -148,14 +150,14 @@ static inline void pud_clear(pud_t *pudp)#ifdef CONFIG_SMP static inline pte_t native_ptep_get_and_clear (pte_t *ptep) { -pte_t res; +union { +pte_t pte; +long long val; + } res; -/* xchg acts as a barrier before the setting of the high bits */ - res.pte_low =xchg (&ptep->pte_low,0 ); - res.pte_high = ptep->pte_high; - ptep->pte_high =0 ; + res.val =arch_atomic64_xchg ((atomic64_t *)ptep,0 ); -return res; +return res.pte; }#else #define native_ptep_get_and_clear (xp) native_local_ptep_get_and_clear(xp)
Signature · 7 lines
2.13.7 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Juergen Gross
maintainerPATCH 2/2 x86/pae: use 64 bit atomic xchg function in native_ptep_get_and_clear
In reply to Juergen GrossUsing only 32-bit writes for the pte will result in an intermediate L1TF vulnerable PTE. When running as a Xen PV guest this will at once switch the guest to shadow mode resulting in a loss of performance.
Use arch_atomic64_xchg() instead which will perform the requested operation atomically with all 64 bits.
Some performance considerations according to:
The main number should be the latency, as there is no tight loop around native_ptep_get_and_clear().
"lock cmpxchg8b" has a latency of 20 cycles, while "lock xchg" (with a memory operand) isn't mentioned in that document. "lock xadd" (with xadd having 3 cycles less latency than xchg) has a latency of 11, so we can assume a latency of 14 for "lock xchg".
---
In case adding about 6 cycles for native_ptep_get_and_clear() is believed to be too bad I can modify the patch to add a paravirt function for that purpose in order to add the overhead for Xen guests only (in fact the overhead for Xen guests will be less, as only one instruction writing to the PTE has to be emulated by the hypervisor).
--- arch/x86/include/asm/pgtable-3level.h | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-)
arch/x86/include/asm/pgtable-3level.h +8 -6
@@ -2,6 +2,8 @@#ifndef _ASM_X86_PGTABLE_3LEVEL_H#define _ASM_X86_PGTABLE_3LEVEL_H +#include <asm/atomic64_32.h> +/* * Intel Physical Address Extension (PAE) Mode - three-level page * tables on PPro+ CPUs.
@@ -148,14 +150,14 @@ static inline void pud_clear(pud_t *pudp)#ifdef CONFIG_SMP static inline pte_t native_ptep_get_and_clear (pte_t *ptep) { -pte_t res; +union { +pte_t pte; +long long val; + } res; -/* xchg acts as a barrier before the setting of the high bits */ - res.pte_low =xchg (&ptep->pte_low,0 ); - res.pte_high = ptep->pte_high; - ptep->pte_high =0 ; + res.val =arch_atomic64_xchg ((atomic64_t *)ptep,0 ); -return res; +return res.pte; }#else #define native_ptep_get_and_clear (xp) native_local_ptep_get_and_clear(xp)
Signature · 1 line
2.13.7
Re: [PATCH 2/2] x86/pae: use 64 bit atomic xchg function in native_ptep_get_and_clear
arch/x86/include/asm/pgtable-3level.h static inline void pud_clear(pud_t *pudp) · line 156
Quoted 10 lines
Quoted 1 line
Quoted 1 line · On 20.08.18 at 07:14, <[email protected]> wrote:
arch/x86/include/asm/pgtable-3level.h +4
-1@@ -148,14 +150,14 @@ static inline void pud_clear(pud_t *pudp)#ifdef CONFIG_SMP static inline pte_t native_ptep_get_and_clear (pte_t *ptep) { -pte_t res; +union { +pte_t pte; +long long val; + } res;
Why the union? pte_t already is one, with the pte field being what you're after ...
arch/x86/include/asm/pgtable-3level.h static inline void pud_clear(pud_t *pudp) · line 158
Quoted 5 lines
arch/x86/include/asm/pgtable-3level.h +1
-4-/* xchg acts as a barrier before the setting of the high bits */ - res.pte_low =xchg (&ptep->pte_low,0 ); - res.pte_high = ptep->pte_high; - ptep->pte_high =0 ; + res.val =arch_atomic64_xchg ((atomic64_t *)ptep,0 );
... here.
Jan
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH 2/2] x86/pae: use 64 bit atomic xchg function in native_ptep_get_and_clear
In reply to Juergen Grossarch/x86/include/asm/pgtable-3level.h static inline void pud_clear(pud_t *pudp) · line 156
Quoted 10 lines
Quoted 1 line
Quoted 1 line · On 20.08.18 at 07:14, <[email protected]> wrote:
arch/x86/include/asm/pgtable-3level.h +4
-1@@ -148,14 +150,14 @@ static inline void pud_clear(pud_t *pudp)#ifdef CONFIG_SMP static inline pte_t native_ptep_get_and_clear (pte_t *ptep) { -pte_t res; +union { +pte_t pte; +long long val; + } res;
Why the union? pte_t already is one, with the pte field being what you're after ...
arch/x86/include/asm/pgtable-3level.h static inline void pud_clear(pud_t *pudp) · line 158
Quoted 5 lines
arch/x86/include/asm/pgtable-3level.h +1
-4-/* xchg acts as a barrier before the setting of the high bits */ - res.pte_low =xchg (&ptep->pte_low,0 ); - res.pte_high = ptep->pte_high; - ptep->pte_high =0 ; + res.val =arch_atomic64_xchg ((atomic64_t *)ptep,0 );
... here.
Jan
Juergen Gross
maintainerRe: [PATCH 2/2] x86/pae: use 64 bit atomic xchg function in native_ptep_get_and_clear
Quoted 14 lines · Jan Beulich
arch/x86/include/asm/pgtable-3level.h static inline void pud_clear(pud_t *pudp) · line 156
Quoted 10 lines
Quoted 1 line
Quoted 1 line · On 20.08.18 at 07:14, <[email protected]> wrote:
arch/x86/include/asm/pgtable-3level.h +4
-1@@ -148,14 +150,14 @@ static inline void pud_clear(pud_t *pudp)#ifdef CONFIG_SMP static inline pte_t native_ptep_get_and_clear (pte_t *ptep) { -pte_t res; +union { +pte_t pte; +long long val; + } res;Why the union?
pte_talready is one, with the pte field being what you're after ...
Quoted 7 lines
arch/x86/include/asm/pgtable-3level.h static inline void pud_clear(pud_t *pudp) · line 158
Quoted 5 lines
arch/x86/include/asm/pgtable-3level.h +1
-4-/* xchg acts as a barrier before the setting of the high bits */ - res.pte_low =xchg (&ptep->pte_low,0 ); - res.pte_high = ptep->pte_high; - ptep->pte_high =0 ; + res.val =arch_atomic64_xchg ((atomic64_t *)ptep,0 );... here.
Uuh, yes.
I'm waiting for more comments, especially regarding the potential need for a paravirt function.
Juergen
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Juergen Gross
maintainerRe: [Xen-devel] [PATCH 2/2] x86/pae: use 64 bit atomic xchg function in native_ptep_get_and_clear
In reply to Jan BeulichQuoted 14 lines · Jan Beulich
arch/x86/include/asm/pgtable-3level.h static inline void pud_clear(pud_t *pudp) · line 156
Quoted 10 lines
Quoted 1 line
Quoted 1 line · On 20.08.18 at 07:14, <[email protected]> wrote:
arch/x86/include/asm/pgtable-3level.h +4
-1@@ -148,14 +150,14 @@ static inline void pud_clear(pud_t *pudp)#ifdef CONFIG_SMP static inline pte_t native_ptep_get_and_clear (pte_t *ptep) { -pte_t res; +union { +pte_t pte; +long long val; + } res;Why the union?
pte_talready is one, with the pte field being what you're after ...
Quoted 7 lines
arch/x86/include/asm/pgtable-3level.h static inline void pud_clear(pud_t *pudp) · line 158
Quoted 5 lines
arch/x86/include/asm/pgtable-3level.h +1
-4-/* xchg acts as a barrier before the setting of the high bits */ - res.pte_low =xchg (&ptep->pte_low,0 ); - res.pte_high = ptep->pte_high; - ptep->pte_high =0 ; + res.val =arch_atomic64_xchg ((atomic64_t *)ptep,0 );... here.
Uuh, yes.
I'm waiting for more comments, especially regarding the potential need for a paravirt function.
Juergen
Re: [PATCH 2/2] x86/pae: use 64 bit atomic xchg function in native_ptep_get_and_clear
In reply to Juergen GrossQuoted 5 lines · Juergen Gross
In case adding about 6 cycles for
native_ptep_get_and_clear()is believed to be too bad I can modify the patch to add a paravirt function for that purpose in order to add the overhead for Xen guests only (in fact the overhead for Xen guests will be less, as only one instruction writing to the PTE has to be emulated by the hypervisor).
I doubt that its worth the trouble of yet another paravirt thingy.
Quoted 22 lines
--- arch/x86/include/asm/pgtable-3level.h | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-)arch/x86/include/asm/pgtable-3level.h +2
-1@@ -2,6 +2,8 @@#ifndef _ASM_X86_PGTABLE_3LEVEL_H#define _ASM_X86_PGTABLE_3LEVEL_H +#include <asm/atomic64_32.h> +/* * Intel Physical Address Extension (PAE) Mode - three-level page * tables on PPro+ CPUs. @@ -148,14 +150,14 @@ static inline void pud_clear(pud_t *pudp)#ifdef CONFIG_SMP static inline pte_t native_ptep_get_and_clear (pte_t *ptep) { -pte_t res;
arch/x86/include/asm/pgtable-3level.h static inline void pud_clear(pud_t *pudp) · line 158
Quoted 10 lines
arch/x86/include/asm/pgtable-3level.h +5
-4+union { +pte_t pte; +long long val; + } res; -/* xchg acts as a barrier before the setting of the high bits */ - res.pte_low =xchg (&ptep->pte_low,0 ); - res.pte_high = ptep->pte_high; - ptep->pte_high =0 ; + res.val =arch_atomic64_xchg ((atomic64_t *)ptep,0 );
Couldn't you just keep
pte_t res;
and do:
res.pte = (pteval_t) arch_atomic64_xchg((atomic64_t *)ptep, 0);
Hmm?
Thanks,
tglx
Juergen Gross
maintainerRe: [PATCH 2/2] x86/pae: use 64 bit atomic xchg function in native_ptep_get_and_clear
Quoted 44 lines · Thomas Gleixner
Quoted 5 lines · Juergen Gross
In case adding about 6 cycles for
native_ptep_get_and_clear()is believed to be too bad I can modify the patch to add a paravirt function for that purpose in order to add the overhead for Xen guests only (in fact the overhead for Xen guests will be less, as only one instruction writing to the PTE has to be emulated by the hypervisor).I doubt that its worth the trouble of yet another paravirt thingy.
Quoted 22 lines
--- arch/x86/include/asm/pgtable-3level.h | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-)arch/x86/include/asm/pgtable-3level.h +2
-1@@ -2,6 +2,8 @@#ifndef _ASM_X86_PGTABLE_3LEVEL_H#define _ASM_X86_PGTABLE_3LEVEL_H +#include <asm/atomic64_32.h> +/* * Intel Physical Address Extension (PAE) Mode - three-level page * tables on PPro+ CPUs. @@ -148,14 +150,14 @@ static inline void pud_clear(pud_t *pudp)#ifdef CONFIG_SMP static inline pte_t native_ptep_get_and_clear (pte_t *ptep) { -pte_t res;arch/x86/include/asm/pgtable-3level.h static inline void pud_clear(pud_t *pudp) · line 158
Quoted 10 lines
arch/x86/include/asm/pgtable-3level.h +5
-4+union { +pte_t pte; +long long val; + } res; -/* xchg acts as a barrier before the setting of the high bits */ - res.pte_low =xchg (&ptep->pte_low,0 ); - res.pte_high = ptep->pte_high; - ptep->pte_high =0 ; + res.val =arch_atomic64_xchg ((atomic64_t *)ptep,0 );Couldn't you just keep
Quoted 7 lines
pte_t res;and do:
res.pte = (pteval_t) arch_atomic64_xchg((atomic64_t *)ptep, 0);Hmm?
Yes, got this suggestion already by Jan. I'm waiting with V2 until tomorrow to see whether someone has other complaints.
Juergen
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Juergen Gross
maintainerRe: [PATCH 2/2] x86/pae: use 64 bit atomic xchg function in native_ptep_get_and_clear
In reply to Thomas GleixnerQuoted 44 lines · Thomas Gleixner
Quoted 5 lines · Juergen Gross
In case adding about 6 cycles for
native_ptep_get_and_clear()is believed to be too bad I can modify the patch to add a paravirt function for that purpose in order to add the overhead for Xen guests only (in fact the overhead for Xen guests will be less, as only one instruction writing to the PTE has to be emulated by the hypervisor).I doubt that its worth the trouble of yet another paravirt thingy.
Quoted 22 lines
--- arch/x86/include/asm/pgtable-3level.h | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-)arch/x86/include/asm/pgtable-3level.h +2
-1@@ -2,6 +2,8 @@#ifndef _ASM_X86_PGTABLE_3LEVEL_H#define _ASM_X86_PGTABLE_3LEVEL_H +#include <asm/atomic64_32.h> +/* * Intel Physical Address Extension (PAE) Mode - three-level page * tables on PPro+ CPUs. @@ -148,14 +150,14 @@ static inline void pud_clear(pud_t *pudp)#ifdef CONFIG_SMP static inline pte_t native_ptep_get_and_clear (pte_t *ptep) { -pte_t res;arch/x86/include/asm/pgtable-3level.h static inline void pud_clear(pud_t *pudp) · line 158
Quoted 10 lines
arch/x86/include/asm/pgtable-3level.h +5
-4+union { +pte_t pte; +long long val; + } res; -/* xchg acts as a barrier before the setting of the high bits */ - res.pte_low =xchg (&ptep->pte_low,0 ); - res.pte_high = ptep->pte_high; - ptep->pte_high =0 ; + res.val =arch_atomic64_xchg ((atomic64_t *)ptep,0 );Couldn't you just keep
Quoted 7 lines
pte_t res;and do:
res.pte = (pteval_t) arch_atomic64_xchg((atomic64_t *)ptep, 0);Hmm?
Yes, got this suggestion already by Jan. I'm waiting with V2 until tomorrow to see whether someone has other complaints.
Juergen
Re: [PATCH 2/2] x86/pae: use 64 bit atomic xchg function in native_ptep_get_and_clear
In reply to Juergen GrossQuoted 5 lines · Juergen Gross
In case adding about 6 cycles for
native_ptep_get_and_clear()is believed to be too bad I can modify the patch to add a paravirt function for that purpose in order to add the overhead for Xen guests only (in fact the overhead for Xen guests will be less, as only one instruction writing to the PTE has to be emulated by the hypervisor).
I doubt that its worth the trouble of yet another paravirt thingy.
Quoted 22 lines
--- arch/x86/include/asm/pgtable-3level.h | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-)arch/x86/include/asm/pgtable-3level.h +2
-1@@ -2,6 +2,8 @@#ifndef _ASM_X86_PGTABLE_3LEVEL_H#define _ASM_X86_PGTABLE_3LEVEL_H +#include <asm/atomic64_32.h> +/* * Intel Physical Address Extension (PAE) Mode - three-level page * tables on PPro+ CPUs. @@ -148,14 +150,14 @@ static inline void pud_clear(pud_t *pudp)#ifdef CONFIG_SMP static inline pte_t native_ptep_get_and_clear (pte_t *ptep) { -pte_t res;
arch/x86/include/asm/pgtable-3level.h static inline void pud_clear(pud_t *pudp) · line 158
Quoted 10 lines
arch/x86/include/asm/pgtable-3level.h +5
-4+union { +pte_t pte; +long long val; + } res; -/* xchg acts as a barrier before the setting of the high bits */ - res.pte_low =xchg (&ptep->pte_low,0 ); - res.pte_high = ptep->pte_high; - ptep->pte_high =0 ; + res.val =arch_atomic64_xchg ((atomic64_t *)ptep,0 );
Couldn't you just keep
pte_t res;
and do:
res.pte = (pteval_t) arch_atomic64_xchg((atomic64_t *)ptep, 0);
Hmm?
Thanks,
tglx
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel