Skip to contentSkip to conversation

x86: Use a single fixed ASID per domain

RFC PATCH Superseded Teddy Astie 2 participants 15 messages Apr 15 – Jun 16

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover x86: Use a single fixed ASID per domain

(ASID can be replaced with VPID for Intel)

This patch series is a reorganized version of [1] though, it share the same goal. Not a lot of logic remain from v1/v2 [2] and the patch series is structured quite differently to v3 [1].

Currently in Xen. In order to perform a TLB flush for a guest, we just increment the ASID, which has the same effect as flushing the TLB. When we run out of ASID, we flush the entire TLB and start over.
This is done per pCPU, thus the ASID management is done on a per-pCPU basis.

However, AMD SEV requires the ASID to be configured in advance and be unique per domain as it bound to the encryption key used for the domain.

This patch series propose a new ASID model where a fixed single ASID is attributed to the domain, thus all vCPU share such ASID. Moreover, the TLB for a vCPU is always flushed in these conditions :
* the vCPU is migrated to another pCPU
(remote pCPU TLB may be out of sync or lack flushes made previously)
* a different vCPU of the same domain previously ran on this pCPU
(TLB entries comes from the previous vCPU, which may have a different CR3)

Albeit related to a SEV vulnerability, [4] gives more justifications on the needs of these rules for correctness.

It is also important for making hypervisor-side guest broadcast TLB flushing realistic, as a single fixed ASID per domain allows a broacast TLB flush to use such ASID as target.

Moreover, AMD programmer manual also explicitely require this for correctly virtualizing INVLPGB in guests (through 090h:7 ("Enable INVLPGB/TLBSYNC.") in VMCB) :

Quoted 6 lines

A guest that executes a legal INVLPGB that is not intercepted will have the requested ASID field replaced by the current ASID and the valid ASID bit set before doing the broadcast invalidation. Because of its broadcast nature, the ASID field must be global and all processors must allocate the same ASID to the same Guest for proper operation. Hypervisors that do not support a global ASID must intercept the Guest usage of INVLPGB, if enabled, for proper behavior.

In order to avoid making ASID management too complex, transition to only using a fixed ASID per domain. If no ASID is available (e.g out of ASID or ASID usage is diabled), we use ASID=1 as a placeholder (which is still a valid ASID), and always flush the TLB on context switch if the vcpu is using ASID=1.

Also add a way to specify a prefered minimum ASID, it is meant to be used later on to avoid clobbering the low ASID space of SEV-ES and favor allocating ASIDs over that space.

[1] x86/hvm: Introduce Xen-wide ASID allocator
https://lore.kernel.org/xen-devel/[email protected]/

[2] x86/hvm: Introduce Xen-wide ASID allocator
https://lore.kernel.org/xen-devel/[email protected]/

[4] TLB Poisoning Attacks on AMD Secure Encrypted Virtualization
https://dl.acm.org/doi/epdf/10.1145/3485832.3485876

Teddy Astie (7):
vmx: Introduce vcpu single context VPID invalidation
common: Track latest pCPU that ran the vCPU
common: Introduce needs_tlb_flush vcpu field
x86: Set v->needs_tlb_flush when needed
x86/hvm: Flush TLB on vCPU overlaps on the same pCPU x86/hvm: Transition to needs_tlb_flush logic, use per-domain ASID
hvm: Allow specifying a prefered asid minimum

 docs/misc/xen-command-line.pandoc      |   2 +-
 xen/arch/x86/flushtlb.c                |  22 +--
 xen/arch/x86/hvm/asid.c                | 177 +++++++++++--------------
 xen/arch/x86/hvm/emulate.c             |   2 +-
 xen/arch/x86/hvm/hvm.c                 |  25 +++-
 xen/arch/x86/hvm/nestedhvm.c           |   7 +-
 xen/arch/x86/hvm/svm/asid.c            |  67 +++++++---
 xen/arch/x86/hvm/svm/nestedsvm.c       |   2 +-
 xen/arch/x86/hvm/svm/svm.c             |  47 +++++--
 xen/arch/x86/hvm/svm/svm.h             |   4 -
 xen/arch/x86/hvm/vmx/vmcs.c            |   6 +-
 xen/arch/x86/hvm/vmx/vmx.c             |  77 ++++++-----
 xen/arch/x86/hvm/vmx/vvmx.c            |   4 +-
 xen/arch/x86/include/asm/flushtlb.h    |   7 -
 xen/arch/x86/include/asm/hvm/asid.h    |  27 ++--
 xen/arch/x86/include/asm/hvm/domain.h  |   1 +
 xen/arch/x86/include/asm/hvm/hvm.h     |  15 +--
 xen/arch/x86/include/asm/hvm/svm.h     |   5 +
 xen/arch/x86/include/asm/hvm/vcpu.h    |  10 +-
 xen/arch/x86/include/asm/hvm/vmx/vmx.h |  23 +++-
 xen/arch/x86/mm/hap/hap.c              |   9 +-
 xen/arch/x86/mm/p2m.c                  |   7 +-
 xen/arch/x86/mm/paging.c               |   2 +-
 xen/arch/x86/mm/shadow/multi.c         |  12 +-
 xen/common/domain.c                    |   8 ++
 xen/common/sched/core.c                |   5 +
 xen/include/xen/sched.h                |   6 +
 27 files changed, 319 insertions(+), 260 deletions(-)
Signature · 10 lines
2.52.0



--
Teddy Astie | Vates XCP-ng Developer

XCP-ng & Xen Orchestra - Vates solutions

web: https://vates.tech
#1 lore raw

RFC PATCH 1/7 vmx: Introduce vcpu single context VPID invalidation

Introduce vpid_sync_vcpu_context to do a single-context invalidation on the vpid attached to the vcpu as a alternative to per-gva and all-context invlidations.

---
 xen/arch/x86/include/asm/hvm/vmx/vmx.h | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)
xen/arch/x86/include/asm/hvm/vmx/vmx.h +21 -0
index da04752e17..3524cb3536 100644
@@ -452,6 +452,27 @@ static inline void ept_sync_all(void)
 
 void ept_sync_domain(struct p2m_domain *p2m);
 
+static inline void vpid_sync_vcpu_context(const struct vcpu *v)
+{
+    int type = INVVPID_SINGLE_CONTEXT;
+
+    /*
+     * If single context invalidation is not supported, we escalate to
+     * use all context invalidation.
+     */
+    if ( likely(cpu_has_vmx_vpid_invvpid_single_context) )
+        goto execute_invvpid;
+
+    /*
+     * If single context invalidation is not supported, we escalate to
+     * use all context invalidation.
+     */
+    type = INVVPID_ALL_CONTEXT;
+
+execute_invvpid:
+    __invvpid(type, v->arch.hvm.n1asid.asid, 0);
+}
+
 static inline void vpid_sync_vcpu_gva(struct vcpu *v, unsigned long gva)
 {
     int type = INVVPID_INDIVIDUAL_ADDR;
Signature · 10 lines
2.52.0



--
Teddy Astie | Vates XCP-ng Developer

XCP-ng & Xen Orchestra - Vates solutions

web: https://vates.tech
#2 lore raw

Re: [RFC PATCH 1/7] vmx: Introduce vcpu single context VPID invalidation

Quoted 27 lines · Teddy Astie

Introduce vpid_sync_vcpu_context to do a single-context invalidation on the vpid attached to the vcpu as a alternative to per-gva and all-context invlidations.

---
 xen/arch/x86/include/asm/hvm/vmx/vmx.h | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)
xen/arch/x86/include/asm/hvm/vmx/vmx.h +10 -0
index da04752e17..3524cb3536 100644
@@ -452,6 +452,27 @@ static inline void ept_sync_all(void)
 
 void ept_sync_domain(struct p2m_domain *p2m);
 
+static inline void vpid_sync_vcpu_context(const struct vcpu *v)
+{
+    int type = INVVPID_SINGLE_CONTEXT;
+
+    /*
+     * If single context invalidation is not supported, we escalate to
+     * use all context invalidation.
+     */
+    if ( likely(cpu_has_vmx_vpid_invvpid_single_context) )
+        goto execute_invvpid;

xen/arch/x86/include/asm/hvm/vmx/vmx.h static inline void ept_sync_all(void) · line 474

Quoted 10 lines
xen/arch/x86/include/asm/hvm/vmx/vmx.h +10 -0
+
+    /*
+     * If single context invalidation is not supported, we escalate to
+     * use all context invalidation.
+     */
+    type = INVVPID_ALL_CONTEXT;
+
+execute_invvpid:
+    __invvpid(type, v->arch.hvm.n1asid.asid, 0);
+}

I think this (such) better would be introduced with a user (else the description wants to say what it's going to be needed for). I further think that this (such) would better be done without goto (else at the very least the label wants to conform to ./CODING_STYLE). And finally I think that the local variable would better be of an unsigned type.

Jan

#3 lore raw

RFC PATCH 2/7 common: Track latest pCPU that ran the vCPU

In reply to Teddy Astie

Track on which pCPU each vCPU of a domain ran. This will be used to know whether a TLB flush is required or not when the vCPU is migrated on another pCPU.

---
I wonder if there is a way to move

    curr->domain->latest_vcpu[cpu] = curr->vcpu_id

into a (at least more) common code ?

 xen/arch/x86/hvm/svm/svm.c | 3 +++
 xen/arch/x86/hvm/vmx/vmx.c | 3 +++
 xen/common/domain.c        | 8 ++++++++
 xen/include/xen/sched.h    | 4 ++++
 4 files changed, 18 insertions(+)
xen/arch/x86/hvm/svm/svm.c +3 -0
index ced6166847..58e927ae04 100644
@@ -967,6 +967,7 @@ void asmlinkage svm_vmenter_helper(void)
     const struct cpu_user_regs *regs = guest_cpu_user_regs();
     struct vcpu *curr = current;
     struct vmcb_struct *vmcb = curr->arch.hvm.svm.vmcb;
+    unsigned int cpu = smp_processor_id();
 
     ASSERT(hvmemul_cache_disabled(curr));
 
@@ -977,6 +978,8 @@ void asmlinkage svm_vmenter_helper(void)
 
     svm_sync_vmcb(curr, vmcb_needs_vmsave);
 
+    curr->domain->latest_vcpu[cpu] = curr->vcpu_id;
+
     vmcb->rax = regs->rax;
     vmcb->rip = regs->rip;
     vmcb->rsp = regs->rsp;
xen/arch/x86/hvm/vmx/vmx.c +3 -0
index 269ca56433..ec0a790336 100644
@@ -4934,6 +4934,7 @@ bool asmlinkage vmx_vmenter_helper(const struct cpu_user_regs *regs)
     u32 new_asid, old_asid;
     struct hvm_vcpu_asid *p_asid;
     bool need_flush;
+    unsigned int cpu = smp_processor_id();
 
     ASSERT(hvmemul_cache_disabled(curr));
 
@@ -4977,6 +4978,8 @@ bool asmlinkage vmx_vmenter_helper(const struct cpu_user_regs *regs)
     if ( unlikely(need_flush) )
         vpid_sync_all();
 
+    currd->latest_vcpu[cpu] = curr->vcpu_id;
+
     if ( paging_mode_hap(curr->domain) )
     {
         struct ept_data *ept = &p2m_get_hostp2m(currd)->ept;
xen/common/domain.c +8 -0
index bb9e210c28..7867166411 100644
@@ -758,6 +758,7 @@ static void _domain_destroy(struct domain *d)
     rangeset_domain_destroy(d);
 
     free_cpumask_var(d->dirty_cpumask);
+    xfree(d->latest_vcpu);
 
     xsm_free_security_domain(d);
 
@@ -992,6 +993,13 @@ struct domain *domain_create(domid_t domid,
     if ( !zalloc_cpumask_var(&d->dirty_cpumask) )
         goto fail;
 
+    err = -ENOMEM;
+    d->latest_vcpu = xmalloc_array(int, nr_cpu_ids);
+    if ( !d->latest_vcpu )
+        goto fail;
+    for (unsigned int i = 0; i < nr_cpu_ids; i++)
+        d->latest_vcpu[i] = -1;
+
     rangeset_domain_initialise(d);
 
     if ( is_idle_domain(d) )
xen/include/xen/sched.h +4 -0
index 212c7d765c..4b8ae21b51 100644
@@ -567,6 +567,10 @@ struct domain
     /* Bitmask of CPUs which are holding onto this domain's state. */
     cpumask_var_t    dirty_cpumask;
 
+    /* Mapping of the latest vCPU that ran on a specific CPU
+     * (-1 if the vCPU hasn't ran yet) */
+    int *latest_vcpu;
+
     struct arch_domain arch;
 
     void *ssid; /* sHype security subject identifier */
Signature · 10 lines
2.52.0



--
Teddy Astie | Vates XCP-ng Developer

XCP-ng & Xen Orchestra - Vates solutions

web: https://vates.tech
#4 lore raw

Re: [RFC PATCH 2/7] common: Track latest pCPU that ran the vCPU

Quoted 3 lines · Teddy Astie

Track on which pCPU each vCPU of a domain ran. This will be used to know whether a TLB flush is required or not when the vCPU is migrated on another pCPU.

Somewhat related tracking already exists - see the dirty_cpumask field. But what title and description say doesn't match ...

xen/arch/x86/hvm/svm/svm.c void asmlinkage svm_vmenter_helper(void) · line 981

Quoted 5 lines
xen/arch/x86/hvm/svm/svm.c +1 -0
@@ -977,6 +978,8 @@ void asmlinkage svm_vmenter_helper(void)
 
     svm_sync_vmcb(curr, vmcb_needs_vmsave);
 
+    curr->domain->latest_vcpu[cpu] = curr->vcpu_id;

... the implementation anyway: You track which vCPU last ran on a given pCPU. Since the same pCPU may have run multiple vCPU-s which then weren't scheduled again, you lose data afaict.

xen/common/domain.c struct domain *domain_create(domid_t domid, · line 997

Quoted 6 lines
xen/common/domain.c +2 -0
@@ -992,6 +993,13 @@ struct domain *domain_create(domid_t domid,
     if ( !zalloc_cpumask_var(&d->dirty_cpumask) )
         goto fail;
 
+    err = -ENOMEM;
+    d->latest_vcpu = xmalloc_array(int, nr_cpu_ids);

xvmalloc_array() please, as this can be huge. It possibly being huge is also of concern.

xen/include/xen/sched.h struct domain · line 572

Quoted 9 lines
xen/include/xen/sched.h +3 -0
@@ -567,6 +567,10 @@ struct domain
     /* Bitmask of CPUs which are holding onto this domain's state. */
     cpumask_var_t    dirty_cpumask;
 
+    /* Mapping of the latest vCPU that ran on a specific CPU
+     * (-1 if the vCPU hasn't ran yet) */
+    int *latest_vcpu;

Why plain int? You don't really leverage -1 as a sentinel, and any unsigned value >= nr_cpu_ids would do in its stead.

Jan

#5 lore raw

Re: [RFC PATCH 2/7] common: Track latest pCPU that ran the vCPU

Le 04/05/2026 à 17:51, Jan Beulich a écrit :

Quoted 6 lines · Teddy Astie
Quoted 3 lines

Track on which pCPU each vCPU of a domain ran. This will be used to know whether a TLB flush is required or not when the vCPU is migrated on another pCPU.

Somewhat related tracking already exists - see the dirty_cpumask field.

I've seen it, but I'm not sure how it can be leveraged here.

I will try to take a closer look if that could be used instead.

Quoted 4 lines

But what title and description say doesn't match ...

Quoted 2 lines
(quoted diff) +0 -0
@@ -977,6 +978,8 @@ void asmlinkage svm_vmenter_helper(void)
 
Quoted 8 lines

xen/arch/x86/hvm/svm/svm.c void asmlinkage svm_vmenter_helper(void) · line 981

Quoted 3 lines
xen/arch/x86/hvm/svm/svm.c +1 -0
      svm_sync_vmcb(curr, vmcb_needs_vmsave);
 
+    curr->domain->latest_vcpu[cpu] = curr->vcpu_id;

... the implementation anyway: You track which vCPU last ran on a given pCPU. Since the same pCPU may have run multiple vCPU-s which then weren't scheduled again, you lose data afaict.

I mixed up the wording. But the implementation is the proper intent.

It's more

Track which vCPU of the domain each pCPU ran.

Quoted 16 lines

xen/common/domain.c struct domain *domain_create(domid_t domid, · line 997

Quoted 6 lines
xen/common/domain.c +2 -0
@@ -992,6 +993,13 @@ struct domain *domain_create(domid_t domid,
      if ( !zalloc_cpumask_var(&d->dirty_cpumask) )
          goto fail;
 
+    err = -ENOMEM;
+    d->latest_vcpu = xmalloc_array(int, nr_cpu_ids);

xvmalloc_array() please, as this can be huge. It possibly being huge is also of concern.

Quoted 6 lines
xen/include/xen/sched.h +0 -0
@@ -567,6 +567,10 @@ struct domain
      /* Bitmask of CPUs which are holding onto this domain's state. */
      cpumask_var_t    dirty_cpumask;
 
Quoted 7 lines

xen/include/xen/sched.h struct domain · line 572

Quoted 3 lines
xen/include/xen/sched.h +3 -0
+    /* Mapping of the latest vCPU that ran on a specific CPU
+     * (-1 if the vCPU hasn't ran yet) */
+    int *latest_vcpu;

Why plain int? You don't really leverage -1 as a sentinel, and any unsigned value >= nr_cpu_ids would do in its stead.

int is not really required here in practice. It's more here to express a invalid state instead of leaving 0 (which would be first pCPU) even if it would not change the overall behavior. Using ~0 could also work alternatively.

Quoted 2 lines

Jan

Teddy

#6 lore raw

Re: [RFC PATCH 2/7] common: Track latest pCPU that ran the vCPU

Quoted 25 lines · Teddy Astie

Le 04/05/2026 à 17:51, Jan Beulich a écrit :

Quoted 6 lines · Teddy Astie
Quoted 3 lines

Track on which pCPU each vCPU of a domain ran. This will be used to know whether a TLB flush is required or not when the vCPU is migrated on another pCPU.

Somewhat related tracking already exists - see the dirty_cpumask field.

I've seen it, but I'm not sure how it can be leveraged here.

I will try to take a closer look if that could be used instead.

Quoted 4 lines

But what title and description say doesn't match ...

Quoted 2 lines
(quoted diff) +0 -0
@@ -977,6 +978,8 @@ void asmlinkage svm_vmenter_helper(void)
 
Quoted 8 lines

xen/arch/x86/hvm/svm/svm.c void asmlinkage svm_vmenter_helper(void) · line 981

Quoted 3 lines
xen/arch/x86/hvm/svm/svm.c +1 -0
      svm_sync_vmcb(curr, vmcb_needs_vmsave);
 
+    curr->domain->latest_vcpu[cpu] = curr->vcpu_id;

... the implementation anyway: You track which vCPU last ran on a given pCPU. Since the same pCPU may have run multiple vCPU-s which then weren't scheduled again, you lose data afaict.

Quoted 5 lines

I mixed up the wording. But the implementation is the proper intent.

It's more

Track which vCPU of the domain each pCPU ran.

Okay, yet then (as already pointed out) how do you know vCPU0 ran last on a given pCPU if after its de-scheduling vCPU1 (of the same domain) was put there. Your track record (after de-scheduling vCPU1) will say only vCPU1; information on vCPU0 will be lost. Yet then, as also indicated, it's not quite clear to me how exactly you mean to leverage this tracking.

Jan

#7 lore raw

Re: [RFC PATCH 2/7] common: Track latest pCPU that ran the vCPU

Le 05/05/2026 à 12:32, Jan Beulich a écrit :

Quoted 30 lines · Teddy Astie
Quoted 29 lines

Le 04/05/2026 à 17:51, Jan Beulich a écrit :

Quoted 6 lines · Teddy Astie
Quoted 3 lines

Track on which pCPU each vCPU of a domain ran. This will be used to know whether a TLB flush is required or not when the vCPU is migrated on another pCPU.

Somewhat related tracking already exists - see the dirty_cpumask field.

I've seen it, but I'm not sure how it can be leveraged here.

I will try to take a closer look if that could be used instead.

Quoted 4 lines

But what title and description say doesn't match ...

Quoted 2 lines
(quoted diff) +0 -0
@@ -977,6 +978,8 @@ void asmlinkage svm_vmenter_helper(void)
 
Quoted 8 lines

xen/arch/x86/hvm/svm/svm.c void asmlinkage svm_vmenter_helper(void) · line 981

Quoted 3 lines
xen/arch/x86/hvm/svm/svm.c +1 -0
       svm_sync_vmcb(curr, vmcb_needs_vmsave);
 
+    curr->domain->latest_vcpu[cpu] = curr->vcpu_id;

... the implementation anyway: You track which vCPU last ran on a given pCPU. Since the same pCPU may have run multiple vCPU-s which then weren't scheduled again, you lose data afaict.

I mixed up the wording. But the implementation is the proper intent.

It's more

Quoted 8 lines
Quoted 1 line

Track which vCPU of the domain each pCPU ran.

Okay, yet then (as already pointed out) how do you know vCPU0 ran last on a given pCPU if after its de-scheduling vCPU1 (of the same domain) was put there. Your track record (after de-scheduling vCPU1) will say only vCPU1; information on vCPU0 will be lost. Yet then, as also indicated, it's not quite clear to me how exactly you mean to leverage this tracking.

The idea is to know which vCPU of the domain the pCPU ASID is holding. Hence, we need to know which was the latest running vCPU of the domain on a per pCPU basis (now that each domain has it's own unique ASID, and each pCPU has its own TLB).

Intermediate vCPUs are not useful, given that the TLB is expected to be flushed when switching vCPUs of a same domain (as they share the same ASID).

So overall, we flush the TLB when either :
- switching betweens vCPU of a same domain on a single pCPU
- the vCPU was previously ran elsewhere (this is dealt in patch 3 through old_cpu != new_cpu check in sched_unit_migrate_finish())

I think the commit description is lacking the "last vCPU" idea.

Would this be better ? :

Introduce last_vcpu which maps each pCPU to the latest vCPU that ran on it. This will be used to know whether a TLB flush is required or not when the pCPU TLB doesn't hold this vCPU state anymore.

Quoted 2 lines

Jan

Teddy

#8 lore raw

Re: [RFC PATCH 2/7] common: Track latest pCPU that ran the vCPU

Quoted 33 lines · Teddy Astie

Le 05/05/2026 à 12:32, Jan Beulich a écrit :

Quoted 32 lines · Teddy Astie
Quoted 30 lines

Le 04/05/2026 à 17:51, Jan Beulich a écrit :

Quoted 6 lines · Teddy Astie
Quoted 3 lines

Track on which pCPU each vCPU of a domain ran. This will be used to know whether a TLB flush is required or not when the vCPU is migrated on another pCPU.

Somewhat related tracking already exists - see the dirty_cpumask field.

I've seen it, but I'm not sure how it can be leveraged here.

I will try to take a closer look if that could be used instead.

Quoted 4 lines

But what title and description say doesn't match ...

Quoted 2 lines
(quoted diff) +0 -0
@@ -977,6 +978,8 @@ void asmlinkage svm_vmenter_helper(void)
 
Quoted 8 lines

xen/arch/x86/hvm/svm/svm.c void asmlinkage svm_vmenter_helper(void) · line 981

Quoted 3 lines
xen/arch/x86/hvm/svm/svm.c +1 -0
       svm_sync_vmcb(curr, vmcb_needs_vmsave);
 
+    curr->domain->latest_vcpu[cpu] = curr->vcpu_id;

... the implementation anyway: You track which vCPU last ran on a given pCPU. Since the same pCPU may have run multiple vCPU-s which then weren't scheduled again, you lose data afaict.

I mixed up the wording. But the implementation is the proper intent.

It's more

Track which vCPU of the domain each pCPU ran.

Quoted 7 lines
Quoted 5 lines

Okay, yet then (as already pointed out) how do you know vCPU0 ran last on a given pCPU if after its de-scheduling vCPU1 (of the same domain) was put there. Your track record (after de-scheduling vCPU1) will say only vCPU1; information on vCPU0 will be lost. Yet then, as also indicated, it's not quite clear to me how exactly you mean to leverage this tracking.

The idea is to know which vCPU of the domain the pCPU ASID is holding.

I fear don't understand this, in particular since you also say ...

Quoted 6 lines

Hence, we need to know which was the latest running vCPU of the domain on a per pCPU basis (now that each domain has it's own unique ASID, and each pCPU has its own TLB).

Intermediate vCPUs are not useful, given that the TLB is expected to be flushed when switching vCPUs of a same domain (as they share the same ASID).

... this: If they all use the same ASID, why would any tracking be needed?

Quoted 4 lines

So overall, we flush the TLB when either :
- switching betweens vCPU of a same domain on a single pCPU
- the vCPU was previously ran elsewhere (this is dealt in patch 3 through old_cpu != new_cpu check in sched_unit_migrate_finish())

Luckily this is better understandable.

Quoted 7 lines

I think the commit description is lacking the "last vCPU" idea.

Would this be better ? :

Introduce last_vcpu which maps each pCPU to the latest vCPU that ran on it. This will be used to know whether a TLB flush is required or not when the pCPU TLB doesn't hold this vCPU state anymore.

Before we settle on any wording, I think it needs to become clear why / whether such tracking is really needed. It feels fragile to say the least. Might be best to start with flushing in too many cases, and only later see about optimizing things.

Jan

#9 lore raw

RFC PATCH 3/7 common: Introduce needs_tlb_flush vcpu field

In reply to Teddy Astie

This field is meant to be used to schedule a TLB flush on the vCPU before waking it up. This field can be set from another vCPU at any time.

Schedule a TLB flush when the vCPU is migrated to another CPU. This is needed as the vCPU-related TLB entries may be out of sync with what happened on another core.

Currently, no architecture use this mechanism, but it is meant to be used as a way to schedule a TLB flush on the vCPU.

---
 xen/common/sched/core.c | 5 +++++
 xen/include/xen/sched.h | 2 ++
 2 files changed, 7 insertions(+)
xen/common/sched/core.c +5 -0
index a57d5dd929..f8e615b3af 100644
@@ -1188,7 +1188,12 @@ static void sched_unit_migrate_finish(struct sched_unit *unit)
 
     /* Wake on new CPU. */
     for_each_sched_unit_vcpu ( unit, v )
+    {
+        if ( old_cpu != new_cpu )
+            /* Migrating to another CPU needs TLB flush */
+            v->needs_tlb_flush = true;
         vcpu_wake(v);
+    }
 }
 
 static bool sched_check_affinity_broken(const struct sched_unit *unit)
xen/include/xen/sched.h +2 -0
index 4b8ae21b51..a26c571015 100644
@@ -224,6 +224,8 @@ struct vcpu
     bool             defer_shutdown;
     /* VCPU is paused following shutdown request (d->is_shutting_down)? */
     bool             paused_for_shutdown;
+    /* VCPU needs its TLB flushed before waking. */
+    bool             needs_tlb_flush;
     /* VCPU need affinity restored */
     uint8_t          affinity_broken;
 #define VCPU_AFFINITY_OVERRIDE    0x01
Signature · 10 lines
2.52.0



--
Teddy Astie | Vates XCP-ng Developer

XCP-ng & Xen Orchestra - Vates solutions

web: https://vates.tech
#10 lore raw

Re: [RFC PATCH 3/7] common: Introduce needs_tlb_flush vcpu field

Quoted 7 lines · Teddy Astie

This field is meant to be used to schedule a TLB flush on the vCPU before waking it up. This field can be set from another vCPU at any time.

Schedule a TLB flush when the vCPU is migrated to another CPU. This is needed as the vCPU-related TLB entries may be out of sync with what happened on another core.

While not an issue with ...

xen/common/sched/core.c static void sched_unit_migrate_finish(struct sched_unit *unit) · line 1194

Quoted 10 lines
xen/common/sched/core.c +4 -0
@@ -1188,7 +1188,12 @@ static void sched_unit_migrate_finish(struct sched_unit *unit)
 
     /* Wake on new CPU. */
     for_each_sched_unit_vcpu ( unit, v )
+    {
+        if ( old_cpu != new_cpu )
+            /* Migrating to another CPU needs TLB flush */
+            v->needs_tlb_flush = true;

... this setting of the flag, for almost any other place (which apparently the next patch is going to introduce, judging by its title) the immediate question is: What if the vCPU is presently running? The flag is meant to only take effect on migration to another pCPU. IOW the overall purpose doesn't become quite clear.

xen/include/xen/sched.h struct vcpu · line 228

Quoted 8 lines
xen/include/xen/sched.h +2 -0
@@ -224,6 +224,8 @@ struct vcpu
     bool             defer_shutdown;
     /* VCPU is paused following shutdown request (d->is_shutting_down)? */
     bool             paused_for_shutdown;
+    /* VCPU needs its TLB flushed before waking. */
+    bool             needs_tlb_flush;

If only x86 is going to use the field, why is it being put here?

Jan

#11 lore raw

RFC PATCH 4/7 x86: Set v->needs_tlb_flush when needed

In reply to Teddy Astie

Sets v->needs_tlb_flush in where a tlb flush is expected to be scheduled on the vCPU.

---
The goal here is to avoid too much noise in [1], hence it currently cohexists with hvm_asid_flush_vcpu(), but [1] will drop hvm_asid_flush_vcpu() and only keep needs_tlb_flush.

[1] x86/hvm: Transition to needs_tlb_flush logic, use per-domain ASID

 xen/arch/x86/flushtlb.c        | 4 ++++
 xen/arch/x86/hvm/emulate.c     | 1 +
 xen/arch/x86/hvm/hvm.c         | 1 +
 xen/arch/x86/hvm/svm/svm.c     | 5 +++++
 xen/arch/x86/hvm/vmx/vmcs.c    | 1 +
 xen/arch/x86/hvm/vmx/vmx.c     | 3 +++
 xen/arch/x86/hvm/vmx/vvmx.c    | 1 +
 xen/arch/x86/mm/p2m.c          | 4 ++++
 xen/arch/x86/mm/paging.c       | 1 +
 xen/arch/x86/mm/shadow/multi.c | 1 +
 10 files changed, 22 insertions(+)
xen/arch/x86/flushtlb.c +4 -0
index 23721bb52c..8ee2385bba 100644
@@ -324,7 +324,11 @@ unsigned int guest_flush_tlb_flags(const struct domain *d)
 void guest_flush_tlb_mask(const struct domain *d, const cpumask_t *mask)
 {
     unsigned int flags = guest_flush_tlb_flags(d);
+    struct vcpu *v;
 
     if ( flags )
         flush_mask(mask, flags);
+
+    for_each_vcpu(d, v)
+        v->needs_tlb_flush = true;
 }
xen/arch/x86/hvm/emulate.c +1 -0
index f3aae158e9..3bc1d321cc 100644
@@ -2657,6 +2657,7 @@ static int cf_check hvmemul_tlb_op(
         if ( x86emul_invpcid_type(aux) != X86_INVPCID_INDIV_ADDR )
         {
             hvm_asid_flush_vcpu(current);
+            current->needs_tlb_flush = true;
             break;
         }
         aux = x86emul_invpcid_pcid(aux);
xen/arch/x86/hvm/hvm.c +1 -0
index 4a81afce02..0f0b0e242f 100644
@@ -1613,6 +1613,7 @@ int hvm_vcpu_initialise(struct vcpu *v)
     struct domain *d = v->domain;
 
     hvm_asid_flush_vcpu(v);
+    v->needs_tlb_flush = true;
 
     spin_lock_init(&v->arch.hvm.tm_lock);
     INIT_LIST_HEAD(&v->arch.hvm.tm_list);
xen/arch/x86/hvm/svm/svm.c +5 -0
index 58e927ae04..64c08432fd 100644
@@ -138,6 +138,8 @@ static void cf_check svm_update_guest_cr(
         {
             if ( !(flags & HVM_UPDATE_GUEST_CR3_NOFLUSH) )
                 hvm_asid_flush_vcpu(v);
+                
+            v->needs_tlb_flush = true;
         }
         else if ( nestedhvm_vmswitch_in_progress(v) )
             ; /* CR3 switches during VMRUN/VMEXIT do not flush the TLB. */
@@ -944,6 +946,7 @@ static void noreturn cf_check svm_do_resume(void)
         hvm_migrate_pirqs(v);
         /* Migrating to another ASID domain.  Request a new ASID. */
         hvm_asid_flush_vcpu(v);
+        v->needs_tlb_flush = true;
     }
 
     if ( !vcpu_guestmode && !vlapic_hw_disabled(vlapic) )
@@ -2306,6 +2309,8 @@ static void cf_check svm_invlpg(struct vcpu *v, unsigned long linear)
 {
     /* Safe fallback. Take a new ASID. */
     hvm_asid_flush_vcpu(v);
+    /* Schedule a tlb flush on the VCPU. */
+    v->needs_tlb_flush = true;
 }
 
 static bool cf_check svm_get_pending_event(
xen/arch/x86/hvm/vmx/vmcs.c +1 -0
index 8e52ef4d49..4efe13e07f 100644
@@ -1904,6 +1904,7 @@ void cf_check vmx_do_resume(void)
         v->arch.hvm.vmx.hostenv_migrated = 1;
 
         hvm_asid_flush_vcpu(v);
+        v->needs_tlb_flush = true;
     }
 
     debug_state = v->domain->debugger_attached
xen/arch/x86/hvm/vmx/vmx.c +3 -0
index ec0a790336..0e4f9f9c3d 100644
@@ -1511,6 +1511,7 @@ static void cf_check vmx_handle_cd(struct vcpu *v, unsigned long value)
 
             wbinvd();               /* flush possibly polluted cache */
             hvm_asid_flush_vcpu(v); /* invalidate memory type cached in TLB */
+            v->needs_tlb_flush = true; /* invalidate memory type cached in TLB */
             v->arch.hvm.vmx.cache_mode = CACHE_MODE_NO_FILL;
         }
         else
@@ -1520,6 +1521,7 @@ static void cf_check vmx_handle_cd(struct vcpu *v, unsigned long value)
             if ( !is_iommu_enabled(v->domain) || iommu_snoop )
                 vmx_clear_msr_intercept(v, MSR_IA32_CR_PAT, VMX_MSR_RW);
             hvm_asid_flush_vcpu(v); /* no need to flush cache */
+            v->needs_tlb_flush = true;
         }
     }
 }
@@ -1872,6 +1874,7 @@ static void cf_check vmx_update_guest_cr(
 
         if ( !(flags & HVM_UPDATE_GUEST_CR3_NOFLUSH) )
             hvm_asid_flush_vcpu(v);
+        v->needs_tlb_flush = true;
         break;
 
     default:
xen/arch/x86/hvm/vmx/vvmx.c +1 -0
index e4cdfe55c1..16d6f1d61b 100644
@@ -1254,6 +1254,7 @@ static void virtual_vmentry(struct cpu_user_regs *regs)
         if ( nvmx->guest_vpid != new_vpid )
         {
             hvm_asid_flush_vcpu_asid(&vcpu_nestedhvm(v).nv_n2asid);
+            v->needs_tlb_flush = true;
             nvmx->guest_vpid = new_vpid;
         }
     }
xen/arch/x86/mm/p2m.c +4 -0
index fddecdf978..910623ac93 100644
@@ -25,6 +25,7 @@
 #include <asm/p2m.h>
 #include <asm/mem_sharing.h>
 #include <asm/hvm/nestedhvm.h>
+#include <asm/hvm/vcpu.h>
 #include <asm/altp2m.h>
 #include <asm/vm_event.h>
 #include <xsm/xsm.h>
@@ -1439,6 +1440,7 @@ p2m_flush(struct vcpu *v, struct p2m_domain *p2m)
     vcpu_nestedhvm(v).nv_p2m = NULL;
     p2m_flush_table(p2m);
     hvm_asid_flush_vcpu(v);
+    v->needs_tlb_flush = true;
 }
 
 void
@@ -1498,6 +1500,7 @@ static void assign_np2m(struct vcpu *v, struct p2m_domain *p2m)
 static void nvcpu_flush(struct vcpu *v)
 {
     hvm_asid_flush_vcpu(v);
+    v->needs_tlb_flush = true;
     vcpu_nestedhvm(v).stale_np2m = true;
 }
 
@@ -1618,6 +1621,7 @@ void np2m_schedule(int dir)
             {
                 /* This vCPU's np2m was flushed while it was not runnable */
                 hvm_asid_flush_core();
+                curr->needs_tlb_flush = true;
                 vcpu_nestedhvm(curr).nv_p2m = NULL;
             }
             else
xen/arch/x86/mm/paging.c +1 -0
index 2396f81ad5..b0b3bef753 100644
@@ -939,6 +939,7 @@ void paging_update_nestedmode(struct vcpu *v)
         /* TODO: shadow-on-shadow */
         v->arch.paging.nestedmode = NULL;
     hvm_asid_flush_vcpu(v);
+    v->needs_tlb_flush = true;
 }
 
 int __init paging_set_allocation(struct domain *d, unsigned int pages,
xen/arch/x86/mm/shadow/multi.c +1 -0
index 80cd3299fa..2df2842138 100644
@@ -3165,6 +3165,7 @@ sh_update_linear_entries(struct vcpu *v)
      * without this change, it would fetch the wrong value due to a stale TLB.
      */
     sh_flush_local(d);
+    v->needs_tlb_flush = true;
 }
 
 static pagetable_t cf_check sh_update_cr3(struct vcpu *v, bool noflush)
Signature · 10 lines
2.52.0



--
Teddy Astie | Vates XCP-ng Developer

XCP-ng & Xen Orchestra - Vates solutions

web: https://vates.tech
#12 lore raw

RFC PATCH 5/7 x86/hvm: Flush TLB on vCPU overlaps on the same pCPU

In reply to Teddy Astie

When using the same ASID/VPID for all vCPU of a domain, we need to make sure that when context switching between 2 vCPUs of the same domain on the same pCPU don't miss a TLB flush, as they may be using different CR3.

Flush the TLB if the latest vCPU that ran on this core differ to the one we're currently on.

---
Would it be preferable to move it to common code (or shared logic for x86).

 xen/arch/x86/hvm/svm/svm.c | 8 ++++++++
 xen/arch/x86/hvm/vmx/vmx.c | 7 +++++++
 2 files changed, 15 insertions(+)
xen/arch/x86/hvm/svm/svm.c +8 -0
index 64c08432fd..8714fb18ec 100644
@@ -981,6 +981,14 @@ void asmlinkage svm_vmenter_helper(void)
 
     svm_sync_vmcb(curr, vmcb_needs_vmsave);
 
+    /**
+     * Check if we were the latest vCPU of this domain that ran on this pCPU.
+     * Flush the TLB if it is not, as the TLB entries are the ones from the previous
+     * vCPU. vCPU migration from a CPU to another always imply a TLB flush.
+     */
+    if ( curr->domain->latest_vcpu[cpu] != curr->vcpu_id )
+        curr->needs_tlb_flush = true;
+
     curr->domain->latest_vcpu[cpu] = curr->vcpu_id;
 
     vmcb->rax = regs->rax;
xen/arch/x86/hvm/vmx/vmx.c +7 -0
index 0e4f9f9c3d..dceff2f221 100644
@@ -4980,6 +4980,13 @@ bool asmlinkage vmx_vmenter_helper(const struct cpu_user_regs *regs)
 
     if ( unlikely(need_flush) )
         vpid_sync_all();
+    /**
+     * Check if we were the latest vCPU of this domain that ran on this pCPU.
+     * Flush the TLB if it is not, as the TLB entries are the ones from the previous
+     * vCPU. vCPU migration from a CPU to another always imply a TLB flush.
+     */
+    if ( currd->latest_vcpu[cpu] != curr->vcpu_id )
+        curr->needs_tlb_flush = true;
 
     currd->latest_vcpu[cpu] = curr->vcpu_id;
 
Signature · 10 lines
2.52.0



--
Teddy Astie | Vates XCP-ng Developer

XCP-ng & Xen Orchestra - Vates solutions

web: https://vates.tech
#13 lore raw

RFC PATCH 6/7 x86/hvm: Transition to needs_tlb_flush logic, use per-domain ASID

In reply to Teddy Astie

Change the ASID model where all vCPU of a domain share the same ASID as required by AMD SEV and broadcast TLB flushing features (AMD INVLPGB).

ASID 1 is reserved as a placeholder for "no domain ASID", and used when either ASID are not supported or no more ASID is available for use. In this case, we always flush the TLB when from and to such domain's vCPU.

Moreover, centralize the TLB flushing logic to use needs_tlb_flush, if a full TLB flush needs to be performed for the vCPU, either through SVM tlb_control or VMX invvpid before entering the guest.

As a result, drop ASID tickling logic, which is now redundant with the needs_tlb_flush mechanism introduced previously. Also take the opportunity to drop some now unused helpers now that FLUSH_HVM_ASID_CORE is dropped.

---
Nested virt makes some aspects tricky and is hard to reason about, especially since there are 2 ASID in such case and complex TLB flushing logic.

 docs/misc/xen-command-line.pandoc      |   2 +-
 xen/arch/x86/flushtlb.c                |  18 +--
 xen/arch/x86/hvm/asid.c                | 169 +++++++++++--------------
 xen/arch/x86/hvm/emulate.c             |   1 -
 xen/arch/x86/hvm/hvm.c                 |  24 +++-
 xen/arch/x86/hvm/nestedhvm.c           |   7 +-
 xen/arch/x86/hvm/svm/asid.c            |  67 +++++++---
 xen/arch/x86/hvm/svm/nestedsvm.c       |   2 +-
 xen/arch/x86/hvm/svm/svm.c             |  31 +++--
 xen/arch/x86/hvm/svm/svm.h             |   4 -
 xen/arch/x86/hvm/vmx/vmcs.c            |   5 +-
 xen/arch/x86/hvm/vmx/vmx.c             |  64 +++++-----
 xen/arch/x86/hvm/vmx/vvmx.c            |   3 +-
 xen/arch/x86/include/asm/flushtlb.h    |   7 -
 xen/arch/x86/include/asm/hvm/asid.h    |  26 ++--
 xen/arch/x86/include/asm/hvm/domain.h  |   1 +
 xen/arch/x86/include/asm/hvm/hvm.h     |  15 +--
 xen/arch/x86/include/asm/hvm/svm.h     |   5 +
 xen/arch/x86/include/asm/hvm/vcpu.h    |  10 +-
 xen/arch/x86/include/asm/hvm/vmx/vmx.h |   4 +-
 xen/arch/x86/mm/hap/hap.c              |   9 +-
 xen/arch/x86/mm/p2m.c                  |   3 -
 xen/arch/x86/mm/paging.c               |   1 -
 xen/arch/x86/mm/shadow/multi.c         |  11 +-
 24 files changed, 228 insertions(+), 261 deletions(-)
docs/misc/xen-command-line.pandoc +1 -1
index 6c77129732..099b718df9 100644
@@ -208,7 +208,7 @@ to appropriate auditing by Xen.  Argo is disabled by default.
 > Default: `true`
 
 Permit Xen to use Address Space Identifiers.  This is an optimisation which
-tags the TLB entries with an ID per vcpu.  This allows for guest TLB flushes
+tags the TLB entries with an ID per domain.  This allows for guest TLB flushes
 to be performed without the overhead of a complete TLB flush.
 
 ### async-show-all (x86)
xen/arch/x86/flushtlb.c +3 -15
index 8ee2385bba..f60d2df8fd 100644
@@ -13,6 +13,7 @@
 #include <xen/softirq.h>
 #include <asm/cache.h>
 #include <asm/flushtlb.h>
+#include <asm/hvm/hvm.h>
 #include <asm/invpcid.h>
 #include <asm/nops.h>
 #include <asm/page.h>
@@ -117,7 +118,6 @@ void switch_cr3_cr4(unsigned long cr3, unsigned long cr4)
 
     if ( tlb_clk_enabled )
         t = pre_flush();
-    hvm_flush_guest_tlbs();
 
     old_cr4 = read_cr4();
     ASSERT(!(old_cr4 & X86_CR4_PCIDE) || !(old_cr4 & X86_CR4_PGE));
@@ -221,9 +221,6 @@ unsigned int flush_area_local(const void *va, unsigned int flags)
             do_tlb_flush();
     }
 
-    if ( flags & FLUSH_HVM_ASID_CORE )
-        hvm_flush_guest_tlbs();
-
     if ( flags & (FLUSH_CACHE_EVICT | FLUSH_CACHE_WRITEBACK) )
     {
         const struct cpuinfo_x86 *c = &current_cpu_data;
@@ -313,21 +310,12 @@ void cache_writeback(const void *addr, unsigned int size)
     asm volatile ("sfence" ::: "memory");
 }
 
-unsigned int guest_flush_tlb_flags(const struct domain *d)
-{
-    bool shadow = paging_mode_shadow(d);
-    bool asid = is_hvm_domain(d) && (cpu_has_svm || shadow);
-
-    return (shadow ? FLUSH_TLB : 0) | (asid ? FLUSH_HVM_ASID_CORE : 0);
-}
-
 void guest_flush_tlb_mask(const struct domain *d, const cpumask_t *mask)
 {
-    unsigned int flags = guest_flush_tlb_flags(d);
     struct vcpu *v;
 
-    if ( flags )
-        flush_mask(mask, flags);
+    if ( paging_mode_shadow(d) )
+        flush_tlb_mask(mask);
 
     for_each_vcpu(d, v)
         v->needs_tlb_flush = true;
xen/arch/x86/hvm/asid.c +73 -96

211 lines not on this page · raw message

xen/arch/x86/hvm/emulate.c +0 -1
index 3bc1d321cc..efdf236df1 100644
@@ -2656,7 +2656,6 @@ static int cf_check hvmemul_tlb_op(
     case x86emul_invpcid:
         if ( x86emul_invpcid_type(aux) != X86_INVPCID_INDIV_ADDR )
         {
-            hvm_asid_flush_vcpu(current);
             current->needs_tlb_flush = true;
             break;
         }
xen/arch/x86/hvm/hvm.c +20 -4
index 0f0b0e242f..2ab625d6e0 100644
@@ -520,11 +520,13 @@ static bool hvm_get_pending_event(struct vcpu *v, struct x86_event *info)
 
 void hvm_do_resume(struct vcpu *v)
 {
+    struct domain *currd = v->domain;;
+
     check_wakeup_from_wait();
 
     pt_restore_timer(v);
 
-    if ( has_vpci(v->domain) && vpci_process_pending(v) )
+    if ( has_vpci(currd) && vpci_process_pending(v) )
     {
         raise_softirq(SCHEDULE_SOFTIRQ);
         return;
@@ -559,6 +561,13 @@ void hvm_do_resume(struct vcpu *v)
             v->arch.monitor.next_interrupt_enabled = false;
         }
     }
+
+    if ( unlikely(currd->arch.hvm.asid.asid == 1) )
+        /*
+         * As ASID=1 may be shared across multiples domains, we can't easily track the
+         * state of the TLB, we always flush the TLB before resuming this vCPU.
+         */
+        v->needs_tlb_flush = true;
 }
 
 static int cf_check hvm_print_line(
@@ -714,6 +723,10 @@ int hvm_domain_initialise(struct domain *d,
     if ( rc )
         goto fail2;
 
+    rc = hvm_asid_alloc(&d->arch.hvm.asid);
+    if ( rc )
+        goto fail2;
+
     rc = alternative_call(hvm_funcs.domain_initialise, d);
     if ( rc != 0 )
         goto fail2;
@@ -794,7 +807,7 @@ void hvm_domain_destroy(struct domain *d)
         list_del(&ioport->list);
         xfree(ioport);
     }
-
+    hvm_asid_free(&d->arch.hvm.asid);
     destroy_vpci_mmcfg(d);
 }
 
@@ -1612,7 +1625,6 @@ int hvm_vcpu_initialise(struct vcpu *v)
     int rc;
     struct domain *d = v->domain;
 
-    hvm_asid_flush_vcpu(v);
     v->needs_tlb_flush = true;
 
     spin_lock_init(&v->arch.hvm.tm_lock);
@@ -4085,6 +4097,11 @@ static void hvm_s3_resume(struct domain *d)
     }
 }
 
+int hvm_flush_tlb(const unsigned long *vcpu_bitmap)
+{
+    return current->domain->arch.paging.flush_tlb(vcpu_bitmap);
+}
+
 static int hvmop_flush_tlb_all(void)
 {
     if ( !is_hvm_domain(current->domain) )
@@ -5464,4 +5481,3 @@ int hvm_copy_context_and_params(struct domain *dst, struct domain *src)
  * indent-tabs-mode: nil
  * End:
  */
-
xen/arch/x86/hvm/nestedhvm.c +3 -4
index bddd77d810..61e866b771 100644
@@ -12,6 +12,7 @@
 #include <asm/hvm/nestedhvm.h>
 #include <asm/event.h>  /* for local_event_delivery_(en|dis)able */
 #include <asm/paging.h> /* for paging_mode_hap() */
+#include <asm/hvm/asid.h>
 
 static unsigned long *shadow_io_bitmap[3];
 
@@ -36,13 +37,11 @@ nestedhvm_vcpu_reset(struct vcpu *v)
     hvm_unmap_guest_frame(nv->nv_vvmcx, 1);
     nv->nv_vvmcx = NULL;
     nv->nv_vvmcxaddr = INVALID_PADDR;
-    nv->nv_flushp2m = 0;
+    nv->nv_flushp2m = true;
     nv->nv_p2m = NULL;
     nv->stale_np2m = false;
     nv->np2m_generation = 0;
 
-    hvm_asid_flush_vcpu_asid(&nv->nv_n2asid);
-
     alternative_vcall(hvm_funcs.nhvm_vcpu_reset, v);
 
     /* vcpu is in host mode */
@@ -86,7 +85,7 @@ static void cf_check nestedhvm_flushtlb_ipi(void *info)
      * This is cheaper than flush_tlb_local() and has
      * the same desired effect.
      */
-    hvm_asid_flush_core();
+    WARN_ON(hvm_flush_tlb(NULL));
     vcpu_nestedhvm(v).nv_p2m = NULL;
     vcpu_nestedhvm(v).stale_np2m = true;
 }
xen/arch/x86/hvm/svm/asid.c +47 -20

97 lines not on this page · raw message

xen/arch/x86/hvm/svm/nestedsvm.c +1 -1
index ef6fa5d23b..d9a84f9388 100644
@@ -5,6 +5,7 @@
  *
  */
 
+#include <asm/hvm/asid.h>
 #include <asm/hvm/support.h>
 #include <asm/hvm/svm.h>
 #include <asm/hvm/nestedhvm.h>
@@ -638,7 +639,6 @@ nsvm_vcpu_vmentry(struct vcpu *v, struct cpu_user_regs *regs,
     if ( svm->ns_asid != vmcb_get_asid(ns_vmcb))
     {
         nv->nv_flushp2m = 1;
-        hvm_asid_flush_vcpu_asid(&vcpu_nestedhvm(v).nv_n2asid);
         svm->ns_asid = vmcb_get_asid(ns_vmcb);
     }
 
xen/arch/x86/hvm/svm/svm.c +17 -14
index 8714fb18ec..ba22acb8a6 100644
@@ -27,6 +27,7 @@
 #include <asm/hvm/nestedhvm.h>
 #include <asm/hvm/support.h>
 #include <asm/hvm/svm.h>
+#include <asm/hvm/asid.h>
 #include <asm/i387.h>
 #include <asm/idt.h>
 #include <asm/iocap.h>
@@ -137,16 +138,18 @@ static void cf_check svm_update_guest_cr(
         if ( !nestedhvm_enabled(v->domain) )
         {
             if ( !(flags & HVM_UPDATE_GUEST_CR3_NOFLUSH) )
-                hvm_asid_flush_vcpu(v);
                 
             v->needs_tlb_flush = true;
         }
         else if ( nestedhvm_vmswitch_in_progress(v) )
             ; /* CR3 switches during VMRUN/VMEXIT do not flush the TLB. */
         else if ( !(flags & HVM_UPDATE_GUEST_CR3_NOFLUSH) )
-            hvm_asid_flush_vcpu_asid(
-                nestedhvm_vcpu_in_guestmode(v)
-                ? &vcpu_nestedhvm(v).nv_n2asid : &v->arch.hvm.n1asid);
+        {
+            if (nestedhvm_vcpu_in_guestmode(v))
+                vcpu_nestedhvm(v).nv_flushp2m = true;
+            else
+                v->needs_tlb_flush = true;
+        }
         break;
     case 4:
         value = HVM_CR4_HOST_MASK;
@@ -944,8 +947,6 @@ static void noreturn cf_check svm_do_resume(void)
         v->arch.hvm.svm.launch_core = smp_processor_id();
         hvm_migrate_timers(v);
         hvm_migrate_pirqs(v);
-        /* Migrating to another ASID domain.  Request a new ASID. */
-        hvm_asid_flush_vcpu(v);
         v->needs_tlb_flush = true;
     }
 
@@ -974,8 +975,6 @@ void asmlinkage svm_vmenter_helper(void)
 
     ASSERT(hvmemul_cache_disabled(curr));
 
-    svm_asid_handle_vmrun();
-
     TRACE_TIME(TRC_HVM_VMENTRY |
                (nestedhvm_vcpu_in_guestmode(curr) ? TRC_HVM_NESTEDFLAG : 0));
 
@@ -991,6 +990,9 @@ void asmlinkage svm_vmenter_helper(void)
 
     curr->domain->latest_vcpu[cpu] = curr->vcpu_id;
 
+    if ( test_and_clear_bool(curr->needs_tlb_flush) )
+        svm_vcpu_set_tlb_control(curr);
+
     vmcb->rax = regs->rax;
     vmcb->rip = regs->rip;
     vmcb->rsp = regs->rsp;
@@ -1111,6 +1113,8 @@ static int cf_check svm_vcpu_initialise(struct vcpu *v)
         return rc;
     }
 
+    svm_vcpu_assign_asid(v);
+
     return 0;
 }
 
@@ -1536,9 +1540,6 @@ static int _svm_cpu_up(bool bsp)
     /* check for erratum 383 */
     svm_init_erratum_383(c);
 
-    /* Initialize core's ASID handling. */
-    svm_asid_init(c);
-
     /* Initialize OSVW bits to be used by guests */
     svm_host_osvw_init();
 
@@ -2293,7 +2294,7 @@ static void svm_invlpga_intercept(
 {
     svm_invlpga(linear,
                 (asid == 0)
-                ? v->arch.hvm.n1asid.asid
+                ? v->domain->arch.hvm.asid.asid
                 : vcpu_nestedhvm(v).nv_n2asid.asid);
 }
 
@@ -2315,8 +2316,6 @@ static bool cf_check is_invlpg(
 
 static void cf_check svm_invlpg(struct vcpu *v, unsigned long linear)
 {
-    /* Safe fallback. Take a new ASID. */
-    hvm_asid_flush_vcpu(v);
     /* Schedule a tlb flush on the VCPU. */
     v->needs_tlb_flush = true;
 }
@@ -2486,6 +2485,8 @@ const struct hvm_function_table * __init start_svm(void)
     svm_function_table.caps.hap_superpage_2mb = true;
     svm_function_table.caps.hap_superpage_1gb = cpu_has_page1gb;
 
+    svm_asid_init();
+
     return &svm_function_table;
 }
 
@@ -2542,6 +2543,8 @@ void asmlinkage svm_vmexit_handler(void)
                    (vlapic_get_reg(vlapic, APIC_TASKPRI) & 0x0F));
     }
 
+    svm_vcpu_clear_tlb_control(v);
+
     exit_reason = vmcb->exitcode;
 
     if ( hvm_long_mode_active(v) )
xen/arch/x86/hvm/svm/svm.h +0 -4
index cfa411ad5a..901354e914 100644
@@ -12,12 +12,8 @@
 #include <xen/types.h>
 
 struct cpu_user_regs;
-struct cpuinfo_x86;
 struct vcpu;
 
-void svm_asid_init(const struct cpuinfo_x86 *c);
-void svm_asid_handle_vmrun(void);
-
 unsigned long *svm_msrbit(unsigned long *msr_bitmap, uint32_t msr);
 void __update_guest_eip(struct cpu_user_regs *regs, unsigned int inst_len);
 
xen/arch/x86/hvm/vmx/vmcs.c +1 -4
index 4efe13e07f..88808e26e7 100644
@@ -20,6 +20,7 @@
 #include <asm/current.h>
 #include <asm/flushtlb.h>
 #include <asm/hvm/hvm.h>
+#include <asm/hvm/asid.h>
 #include <asm/hvm/io.h>
 #include <asm/hvm/nestedhvm.h>
 #include <asm/hvm/vmx/vmcs.h>
@@ -778,8 +779,6 @@ static int _vmx_cpu_up(bool bsp)
 
     this_cpu(vmxon) = 1;
 
-    hvm_asid_init(cpu_has_vmx_vpid ? (1u << VMCS_VPID_WIDTH) : 0);
-
     if ( cpu_has_vmx_ept )
         ept_sync_all();
 
@@ -1903,7 +1902,6 @@ void cf_check vmx_do_resume(void)
          */
         v->arch.hvm.vmx.hostenv_migrated = 1;
 
-        hvm_asid_flush_vcpu(v);
         v->needs_tlb_flush = true;
     }
 
@@ -2117,7 +2115,6 @@ void vmcs_dump_vcpu(struct vcpu *v)
          (SECONDARY_EXEC_ENABLE_VPID | SECONDARY_EXEC_ENABLE_VM_FUNCTIONS) )
         printk("Virtual processor ID = 0x%04x VMfunc controls = %016lx\n",
                vmr16(VIRTUAL_PROCESSOR_ID), vmr(VM_FUNCTION_CONTROL));
-
     vmx_vmcs_exit(v);
 }
 
xen/arch/x86/hvm/vmx/vmx.c +31 -33

139 lines not on this page · raw message

xen/arch/x86/hvm/vmx/vvmx.c +1 -2
index 16d6f1d61b..7314aa5633 100644
@@ -1253,7 +1253,6 @@ static void virtual_vmentry(struct cpu_user_regs *regs)
 
         if ( nvmx->guest_vpid != new_vpid )
         {
-            hvm_asid_flush_vcpu_asid(&vcpu_nestedhvm(v).nv_n2asid);
             v->needs_tlb_flush = true;
             nvmx->guest_vpid = new_vpid;
         }
@@ -2053,7 +2052,7 @@ static int nvmx_handle_invvpid(struct cpu_user_regs *regs)
     case INVVPID_INDIVIDUAL_ADDR:
     case INVVPID_SINGLE_CONTEXT:
     case INVVPID_ALL_CONTEXT:
-        hvm_asid_flush_vcpu_asid(&vcpu_nestedhvm(current).nv_n2asid);
+        hvm_flush_tlb(NULL);
         break;
     default:
         vmfail(regs, VMX_INSN_INVEPT_INVVPID_INVALID_OP);
xen/arch/x86/include/asm/flushtlb.h +0 -7
index 7bcbca2b7f..d8167aca18 100644
@@ -125,12 +125,6 @@ void switch_cr3_cr4(unsigned long cr3, unsigned long cr4);
 #define FLUSH_VCPU_STATE 0x1000
  /* Flush the per-cpu root page table */
 #define FLUSH_ROOT_PGTBL 0x2000
-#if CONFIG_HVM
- /* Flush all HVM guests linear TLB (using ASID/VPID) */
-#define FLUSH_HVM_ASID_CORE 0x4000
-#else
-#define FLUSH_HVM_ASID_CORE 0
-#endif
 #if defined(CONFIG_PV) || defined(CONFIG_SHADOW_PAGING)
 /*
  * Adding this to the flags passed to flush_area_mask will prevent using the
@@ -190,7 +184,6 @@ void flush_area_mask(const cpumask_t *mask, const void *va,
 
 static inline void flush_page_to_ram(unsigned long mfn, bool sync_icache) {}
 
-unsigned int guest_flush_tlb_flags(const struct domain *d);
 void guest_flush_tlb_mask(const struct domain *d, const cpumask_t *mask);
 
 #endif /* __FLUSHTLB_H__ */
xen/arch/x86/include/asm/hvm/asid.h +11 -15
index 25ba57e768..13ea357f70 100644
@@ -8,25 +8,21 @@
 #ifndef __ASM_X86_HVM_ASID_H__
 #define __ASM_X86_HVM_ASID_H__
 
+#include <xen/stdbool.h>
+#include <xen/stdint.h>
 
-struct vcpu;
-struct hvm_vcpu_asid;
+struct hvm_asid {
+  uint32_t asid;
+};
 
-/* Initialise ASID management for the current physical CPU. */
-void hvm_asid_init(unsigned int nasids);
+extern bool asid_enabled;
 
-/* Invalidate a particular ASID allocation: forces re-allocation. */
-void hvm_asid_flush_vcpu_asid(struct hvm_vcpu_asid *asid);
+/* Initialise ASID management distributed across all CPUs. */
+int hvm_asid_init(unsigned long nasids);
 
-/* Invalidate all ASID allocations for specified VCPU: forces re-allocation. */
-void hvm_asid_flush_vcpu(struct vcpu *v);
-
-/* Flush all ASIDs on this processor core. */
-void hvm_asid_flush_core(void);
-
-/* Called before entry to guest context. Checks ASID allocation, returns a
- * boolean indicating whether all ASIDs must be flushed. */
-bool hvm_asid_handle_vmenter(struct hvm_vcpu_asid *asid);
+int hvm_asid_alloc(struct hvm_asid *asid);
+int hvm_asid_alloc_range(struct hvm_asid *asid, unsigned long min, unsigned long max);
+void hvm_asid_free(struct hvm_asid *asid);
 
 #endif /* __ASM_X86_HVM_ASID_H__ */
 
xen/arch/x86/include/asm/hvm/domain.h +1 -0
index abf9bc448d..03fe6c8f7b 100644
@@ -139,6 +139,7 @@ struct hvm_domain {
     } write_map;
 
     struct hvm_pi_ops pi_ops;
+    struct hvm_asid asid;
 
     union {
         struct vmx_domain vmx;
xen/arch/x86/include/asm/hvm/hvm.h +2 -13
index e7c1364802..935d9e7548 100644
@@ -274,6 +274,8 @@ int hvm_domain_initialise(struct domain *d,
 void hvm_domain_relinquish_resources(struct domain *d);
 void hvm_domain_destroy(struct domain *d);
 
+int hvm_flush_tlb(const unsigned long *vcpu_bitmap);
+
 int hvm_vcpu_initialise(struct vcpu *v);
 void hvm_vcpu_destroy(struct vcpu *v);
 void hvm_vcpu_down(struct vcpu *v);
@@ -497,17 +499,6 @@ static inline void hvm_set_tsc_offset(struct vcpu *v, uint64_t offset)
     alternative_vcall(hvm_funcs.set_tsc_offset, v, offset);
 }
 
-/*
- * Called to ensure than all guest-specific mappings in a tagged TLB are 
- * flushed; does *not* flush Xen's TLB entries, and on processors without a 
- * tagged TLB it will be a noop.
- */
-static inline void hvm_flush_guest_tlbs(void)
-{
-    if ( hvm_enabled )
-        hvm_asid_flush_core();
-}
-
 static inline unsigned int
 hvm_get_cpl(struct vcpu *v)
 {
@@ -901,8 +892,6 @@ static inline int hvm_cpu_up(void)
 
 static inline void hvm_cpu_down(void) {}
 
-static inline void hvm_flush_guest_tlbs(void) {}
-
 static inline void hvm_invlpg(const struct vcpu *v, unsigned long linear)
 {
     ASSERT_UNREACHABLE();
xen/arch/x86/include/asm/hvm/svm.h +5 -0
index a35a61273b..1877bb149a 100644
@@ -9,6 +9,11 @@
 #ifndef __ASM_X86_HVM_SVM_H__
 #define __ASM_X86_HVM_SVM_H__
 
+void svm_asid_init(void);
+void svm_vcpu_assign_asid(struct vcpu *v);
+void svm_vcpu_set_tlb_control(struct vcpu *v);
+void svm_vcpu_clear_tlb_control(struct vcpu *v);
+
 /*
  * PV context switch helpers.  Prefetching the VMCB area itself has been shown
  * to be useful for performance.
xen/arch/x86/include/asm/hvm/vcpu.h +2 -8
index 836138a4a6..7472cdd0f2 100644
@@ -9,6 +9,7 @@
 #define __ASM_X86_HVM_VCPU_H__
 
 #include <xen/tasklet.h>
+#include <asm/hvm/asid.h>
 #include <asm/hvm/vlapic.h>
 #include <asm/hvm/vmx/vmcs.h>
 #include <asm/hvm/vmx/vvmx.h>
@@ -16,11 +17,6 @@
 #include <asm/mtrr.h>
 #include <public/hvm/ioreq.h>
 
-struct hvm_vcpu_asid {
-    uint64_t generation;
-    uint32_t asid;
-};
-
 struct hvm_vcpu_io {
     /*
      * HVM emulation:
@@ -78,7 +74,7 @@ struct nestedvcpu {
     bool stale_np2m; /* True when p2m_base in VMCx02 is no longer valid */
     uint64_t np2m_generation;
 
-    struct hvm_vcpu_asid nv_n2asid;
+    struct hvm_asid nv_n2asid;
 
     bool nv_vmentry_pending;
     bool nv_vmexit_pending;
@@ -142,8 +138,6 @@ struct hvm_vcpu {
     /* (MFN) hypervisor page table */
     pagetable_t         monitor_table;
 
-    struct hvm_vcpu_asid n1asid;
-
     u64                 msr_tsc_adjust;
 
     union {
xen/arch/x86/include/asm/hvm/vmx/vmx.h +2 -2
index 3524cb3536..4f307ae123 100644
@@ -470,7 +470,7 @@ static inline void vpid_sync_vcpu_context(const struct vcpu *v)
     type = INVVPID_ALL_CONTEXT;
 
 execute_invvpid:
-    __invvpid(type, v->arch.hvm.n1asid.asid, 0);
+    __invvpid(type, v->domain->arch.hvm.asid.asid, 0);
 }
 
 static inline void vpid_sync_vcpu_gva(struct vcpu *v, unsigned long gva)
@@ -494,7 +494,7 @@ static inline void vpid_sync_vcpu_gva(struct vcpu *v, unsigned long gva)
         type = INVVPID_ALL_CONTEXT;
 
 execute_invvpid:
-    __invvpid(type, v->arch.hvm.n1asid.asid, (u64)gva);
+    __invvpid(type, v->domain->arch.hvm.asid.asid, (u64)gva);
 }
 
 static inline void vpid_sync_all(void)
xen/arch/x86/mm/hap/hap.c +4 -5
index 5ccb80bda5..156734d3e0 100644
@@ -27,6 +27,7 @@
 #include <asm/p2m.h>
 #include <asm/domain.h>
 #include <xen/numa.h>
+#include <asm/hvm/asid.h>
 #include <asm/hvm/nestedhvm.h>
 #include <public/sched.h>
 
@@ -750,18 +751,16 @@ static bool cf_check flush_tlb(const unsigned long *vcpu_bitmap)
         if ( !flush_vcpu(v, vcpu_bitmap) )
             continue;
 
-        hvm_asid_flush_vcpu(v);
-
         cpu = read_atomic(&v->dirty_cpu);
         if ( cpu != this_cpu && is_vcpu_dirty_cpu(cpu) && v->is_running )
             __cpumask_set_cpu(cpu, mask);
     }
 
+    guest_flush_tlb_mask(d, mask);
+
     /*
      * Trigger a vmexit on all pCPUs with dirty vCPU state in order to force an
-     * ASID/VPID change and hence accomplish a guest TLB flush. Note that vCPUs
-     * not currently running will already be flushed when scheduled because of
-     * the ASID tickle done in the loop above.
+     * ASID/VPID flush and hence accomplish a guest TLB flush.
      */
     on_selected_cpus(mask, NULL, NULL, 0);
 
xen/arch/x86/mm/p2m.c +0 -3
index 910623ac93..6eb6035f09 100644
@@ -1439,7 +1439,6 @@ p2m_flush(struct vcpu *v, struct p2m_domain *p2m)
     ASSERT(v->domain == p2m->domain);
     vcpu_nestedhvm(v).nv_p2m = NULL;
     p2m_flush_table(p2m);
-    hvm_asid_flush_vcpu(v);
     v->needs_tlb_flush = true;
 }
 
@@ -1499,7 +1498,6 @@ static void assign_np2m(struct vcpu *v, struct p2m_domain *p2m)
 
 static void nvcpu_flush(struct vcpu *v)
 {
-    hvm_asid_flush_vcpu(v);
     v->needs_tlb_flush = true;
     vcpu_nestedhvm(v).stale_np2m = true;
 }
@@ -1620,7 +1618,6 @@ void np2m_schedule(int dir)
             if ( !np2m_valid )
             {
                 /* This vCPU's np2m was flushed while it was not runnable */
-                hvm_asid_flush_core();
                 curr->needs_tlb_flush = true;
                 vcpu_nestedhvm(curr).nv_p2m = NULL;
             }
xen/arch/x86/mm/paging.c +0 -1
index b0b3bef753..af5584a777 100644
@@ -938,7 +938,6 @@ void paging_update_nestedmode(struct vcpu *v)
     else
         /* TODO: shadow-on-shadow */
         v->arch.paging.nestedmode = NULL;
-    hvm_asid_flush_vcpu(v);
     v->needs_tlb_flush = true;
 }
 
xen/arch/x86/mm/shadow/multi.c +3 -8
index 2df2842138..0eef92bc5b 100644
@@ -81,12 +81,6 @@ const char *const fetch_type_names[] = {
 
 static pagetable_t cf_check sh_update_cr3(struct vcpu *v, bool noflush);
 
-/* Helper to perform a local TLB flush. */
-static void sh_flush_local(const struct domain *d)
-{
-    flush_local(guest_flush_tlb_flags(d));
-}
-
 #if GUEST_PAGING_LEVELS >= 4 && defined(CONFIG_PV32)
 #define ASSERT_VALID_L2(t) \
     ASSERT((t) == SH_type_l2_shadow || (t) == SH_type_l2h_shadow)
@@ -2947,7 +2941,8 @@ static bool cf_check sh_invlpg(struct vcpu *v, unsigned long linear)
     if ( mfn_to_page(sl1mfn)->u.sh.type
          == SH_type_fl1_shadow )
     {
-        sh_flush_local(v->domain);
+        flush_tlb_local();
+        v->needs_tlb_flush = true;
         return false;
     }
 
@@ -3164,7 +3159,7 @@ sh_update_linear_entries(struct vcpu *v)
      * linear pagetable to read a top-level shadow page table entry. But,
      * without this change, it would fetch the wrong value due to a stale TLB.
      */
-    sh_flush_local(d);
+    flush_tlb_local();
     v->needs_tlb_flush = true;
 }
 
Signature · 10 lines
2.52.0



--
Teddy Astie | Vates XCP-ng Developer

XCP-ng & Xen Orchestra - Vates solutions

web: https://vates.tech
#14 lore raw

RFC PATCH 7/7 hvm: Allow specifying a prefered asid minimum

In reply to Teddy Astie

To avoid clobbering all ASIDs that are below SEV-enabled guest maximum, we want to allocate if possible all ASID over a "prefered minimum" and fallback to below otherwise.

---
 xen/arch/x86/hvm/asid.c             | 8 ++++++++
 xen/arch/x86/include/asm/hvm/asid.h | 1 +
 2 files changed, 9 insertions(+)
xen/arch/x86/hvm/asid.c +8 -0
index 1a21125161..4ad3200c96 100644
@@ -22,6 +22,9 @@ boolean_param("asid", opt_asid_enabled);
 bool __read_mostly asid_enabled = false;
 static unsigned long __ro_after_init *asid_bitmap;
 static unsigned long __ro_after_init asid_count;
+
+/* Default minimum ASID to use */
+unsigned long __read_mostly asid_default_min = 0;
 static DEFINE_SPINLOCK(asid_lock);
 
 /*
@@ -67,6 +70,11 @@ int hvm_asid_alloc(struct hvm_asid *asid)
         return 0;
     }
 
+    /* Try to allocate above default minimum */
+    if ( asid_default_min &&
+         !hvm_asid_alloc_range(asid, asid_default_min, asid_count) )
+        return 0;
+
     spin_lock(&asid_lock);
     new_asid = find_first_zero_bit(asid_bitmap, asid_count);
     if ( new_asid > asid_count )
xen/arch/x86/include/asm/hvm/asid.h +1 -0
index 13ea357f70..e989ebbe8c 100644
@@ -16,6 +16,7 @@ struct hvm_asid {
 };
 
 extern bool asid_enabled;
+extern unsigned long asid_default_min;
 
 /* Initialise ASID management distributed across all CPUs. */
 int hvm_asid_init(unsigned long nasids);
Signature · 10 lines
2.52.0



--
Teddy Astie | Vates XCP-ng Developer

XCP-ng & Xen Orchestra - Vates solutions

web: https://vates.tech