Skip to content

RFC x86: Use a single fixed ASID per domain

Teddy Astie xen-devel 2 versions – In review

Get series with b4
b4 am 1785509179.8631fc262581453bbf619ec5b2062170.19fb8a3efe8000e099@vates.tech

Versions newest first · linked by author and title

  1. v2 RFC Latest5 patches 9 repliesIn review
  2. v1 RFC7 patches 7 repliesSuperseded

Patches in v2 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/5vmx: Refactor vpid_sync_vcpu_gva()xen/arch/x86/include/asm/hvm/vmx/vmx.h+3 -63 replies
2/5vmx: Introduce vcpu single context VPID invalidationxen/arch/x86/include/asm/hvm/vmx/vmx.h+14 -00 replies
3/5x86: Track vcpu context switches and introduce needs_tlb_flush fieldxen/arch/x86/domain.c xen/arch/x86/include/asm/domain.h+32 -02 replies
4/5x86/hvm: Transition to needs_tlb_flush logic, use per-domain ASIDdocs/misc/xen-command-line.pandoc xen/arch/x86/flushtlb.c xen/arch/x86/hvm/asid.c +21 more+241 -2604 replies
5/5hvm: Allow specifying a prefered asid minimumxen/arch/x86/hvm/asid.c xen/arch/x86/include/asm/hvm/asid.h+9 -00 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v2 yet.

Read the v2 thread

Cover letter [RFC PATCH v2 0/5] x86: Use a single fixed ASID per domain

(ASID can be replaced with VPID for Intel)

Currently in Xen. In order to perform a TLB flush for a guest, we just increment the ASID, which has the same effect as flushing the TLB. When we run out of ASID, we flush the entire TLB and start over.
This is done per pCPU, thus the ASID management is done on a per-pCPU basis.

However, AMD SEV requires the ASID to be configured in advance and be unique per domain as it bound to the encryption key used for the domain.

This patch series propose a new ASID model where a fixed single ASID is attributed to the domain, thus all vCPU share such ASID. Moreover, the TLB for a vCPU needs to be flushed in these conditions :
* the vCPU is migrated to another pCPU
(remote pCPU TLB may be out of sync or lack flushes made previously)
* a different vCPU of the same domain previously ran on this pCPU
(TLB entries comes from the previous vCPU, which may have a different CR3)

Albeit related to a SEV vulnerability, [1] gives more justifications on the needs of these rules for correctness.

It is also important for making hypervisor-side guest broadcast TLB flushing realistic, as a single fixed ASID per domain allows a broacast TLB flush to use such ASID as target.

Moreover, AMD programmer manual also explicitely require this for correctly virtualizing INVLPGB in guests (through 090h:7 ("Enable INVLPGB/TLBSYNC.") in VMCB) :

Quoted 6 lines

A guest that executes a legal INVLPGB that is not intercepted will have the requested ASID field replaced by the current ASID and the valid ASID bit set before doing the broadcast invalidation. Because of its broadcast nature, the ASID field must be global and all processors must allocate the same ASID to the same Guest for proper operation. Hypervisors that do not support a global ASID must intercept the Guest usage of INVLPGB, if enabled, for proper behavior.

In order to avoid making ASID management too complex, transition to only using a fixed ASID per domain. If no ASID is available (e.g out of ASID or ASID usage is diabled), we use ASID=1 as a placeholder (which is still a valid ASID), and always flush the TLB on context switch if the vcpu is using ASID=1.

Also add a way to specify a prefered minimum ASID, it is meant to be used later on to avoid clobbering the low ASID space of SEV-ES and favor allocating ASIDs over that space.

[1] TLB Poisoning Attacks on AMD Secure Encrypted Virtualization
https://dl.acm.org/doi/epdf/10.1145/3485832.3485876

v1: https://lore.kernel.org/xen-devel/[email protected]/

Teddy Astie (5):
vmx: Refactor vpid_sync_vcpu_gva()
vmx: Introduce vcpu single context VPID invalidation
x86: Track vcpu context switches and introduce needs_tlb_flush field x86/hvm: Transition to needs_tlb_flush logic, use per-domain ASID
hvm: Allow specifying a prefered asid minimum

 docs/misc/xen-command-line.pandoc      |   2 +-
 xen/arch/x86/domain.c                  |  26 ++++
 xen/arch/x86/flushtlb.c                |  22 +--
 xen/arch/x86/hvm/asid.c                | 177 +++++++++++--------------
 xen/arch/x86/hvm/emulate.c             |   2 +-
 xen/arch/x86/hvm/hvm.c                 |  14 +-
 xen/arch/x86/hvm/nestedhvm.c           |   7 +-
 xen/arch/x86/hvm/svm/asid.c            |  67 +++++++---
 xen/arch/x86/hvm/svm/nestedsvm.c       |   2 +-
 xen/arch/x86/hvm/svm/svm.c             |  35 +++--
 xen/arch/x86/hvm/svm/svm.h             |   4 -
 xen/arch/x86/hvm/vmx/vmcs.c            |   6 +-
 xen/arch/x86/hvm/vmx/vmx.c             |  66 ++++-----
 xen/arch/x86/hvm/vmx/vvmx.c            |   4 +-
 xen/arch/x86/include/asm/domain.h      |   6 +
 xen/arch/x86/include/asm/flushtlb.h    |   7 -
 xen/arch/x86/include/asm/hvm/asid.h    |  39 +++---
 xen/arch/x86/include/asm/hvm/domain.h  |   1 +
 xen/arch/x86/include/asm/hvm/hvm.h     |  15 +--
 xen/arch/x86/include/asm/hvm/svm.h     |   5 +
 xen/arch/x86/include/asm/hvm/vcpu.h    |  10 +-
 xen/arch/x86/include/asm/hvm/vmx/vmx.h |  25 +++-
 xen/arch/x86/mm/hap/hap.c              |   9 +-
 xen/arch/x86/mm/p2m.c                  |   6 +-
 xen/arch/x86/mm/paging.c               |   2 +-
 xen/arch/x86/mm/shadow/multi.c         |  12 +-
 26 files changed, 302 insertions(+), 269 deletions(-)
Signature · 10 lines
2.54.0



--
Teddy Astie | Vates XCP-ng Developer

XCP-ng & Xen Orchestra - Vates solutions

web: https://vates.tech