x86/hvm: Intercept RDPMC when vPMU is disabled
PATCH 1/1 x86/hvm: Intercept RDPMC when vPMU is disabled
vPMU isn't security supported, and in general guests can't access any of the performance counter MSRs. However, the RDPMC instruction isn't intercepted, meaning that guest software can read the instantaneous counter values.
When vPMU isn't configured, intercept RDPMC and unconditionally fail it as if software has requested a bad counter index (#GP fault). It is model specific as to which counters are available to begin with, and in levelled scenarios, this information may not be accurate in the first place.
This change isn't expected to have any impact on VMs. Userspace is not usually given access to RDPMC (Windows appear to completely prohibit it; Linux is restricted to root), and kernels won't be executing RDPMC instructions if their PMU drivers have failed to start.
---
CC: Jan Beulich <[email protected]>
CC: Wei Liu <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Jun Nakajima <[email protected]>
CC: Kevin Tian <[email protected]>
CC: Boris Ostrovsky <[email protected]>
CC: Suravee Suthikulpanit <[email protected]>
CC: Brian Woods <[email protected]>
CC: Juergen Gross <[email protected]>
This should be taken into Xen 4.12 and backported to the stable releases. While it isn't an XSA itself, it is an information leak (Xen's NMI watchdog in particular) which could be advantagous to an attacker trying to exploit a race condition.
The only other option is to emulate the reported family and offer back all 0's for the accessable counters. Obviously this is a non-starter.
This patch has had some basic testing in XenServer with no issues identified with any guests.
--- xen/arch/x86/hvm/svm/svm.c | 8 ++++++++ xen/arch/x86/hvm/svm/vmcb.c | 3 +++ xen/arch/x86/hvm/vmx/vmcs.c | 2 ++ xen/arch/x86/hvm/vmx/vmx.c | 9 +++++++++ 4 files changed, 22 insertions(+)
xen/arch/x86/hvm/svm/svm.c +8 -0
@@ -2984,6 +2984,14 @@ void svm_vmexit_handler(struct cpu_user_regs *regs)svm_vmexit_do_rdtsc (regs, exit_reason ==VMEXIT_RDTSCP );break ; +case VMEXIT_RDPMC : +/* +* Interception is only active when VPMU is disabled, and the guest +* mustn't be able to read the counters. Emulate "bad counter index". +*/ +hvm_inject_hw_exception (TRAP_gp_fault,0 ); +break ; +case VMEXIT_MONITOR :case VMEXIT_MWAIT :hvm_inject_hw_exception (TRAP_invalid_op,X86_EVENT_NO_EC );
xen/arch/x86/hvm/svm/vmcb.c +3 -0
@@ -67,6 +67,9 @@ static int construct_vmcb(struct vcpu *v)GENERAL1_INTERCEPT_INVLPGA |GENERAL1_INTERCEPT_IOIO_PROT |GENERAL1_INTERCEPT_MSR_PROT |GENERAL1_INTERCEPT_SHUTDOWN_EVT |GENERAL1_INTERCEPT_TASK_SWITCH ; +if ( !vpmu_available (v) ) + vmcb->_general1_intercepts |=GENERAL1_INTERCEPT_RDPMC ; + vmcb->_general2_intercepts =GENERAL2_INTERCEPT_VMRUN |GENERAL2_INTERCEPT_VMMCALL | GENERAL2_INTERCEPT_VMLOAD | GENERAL2_INTERCEPT_VMSAVE |
xen/arch/x86/hvm/vmx/vmcs.c +2 -0
@@ -994,6 +994,8 @@ static int construct_vmcs(struct vcpu *v)__vmwrite (PIN_BASED_VM_EXEC_CONTROL , vmx_pin_based_exec_control); v->arch.hvm.vmx.exec_control = vmx_cpu_based_exec_control; +if ( !vpmu_available (v) ) + v->arch.hvm.vmx.exec_control |=CPU_BASED_RDPMC_EXITING ;if ( d->arch.vtsc && !cpu_has_vmx_tsc_scaling ) v->arch.hvm.vmx.exec_control |=CPU_BASED_RDTSC_EXITING ;
xen/arch/x86/hvm/vmx/vmx.c +9 -0
@@ -3956,6 +3956,15 @@ void vmx_vmexit_handler(struct cpu_user_regs *regs)__vmread (EXIT_QUALIFICATION , &exit_qualification);vmx_invlpg_intercept (exit_qualification);break ; + +case EXIT_REASON_RDPMC : +/* +* Interception is only active when VPMU is disabled, and the guest +* mustn't be able to read the counters. Emulate "bad counter index". +*/ +hvm_inject_hw_exception (TRAP_gp_fault,0 ); +break ; +case EXIT_REASON_RDTSCP :if ( !currd->arch.cpuid->extd.rdtscp ) {
Signature · 7 lines
2.1.4 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Quoted 26 lines · Andrew Cooper
vPMU isn't security supported, and in general guests can't access any of the performance counter MSRs. However, the RDPMC instruction isn't intercepted, meaning that guest software can read the instantaneous counter values.
When vPMU isn't configured, intercept RDPMC and unconditionally fail it as if software has requested a bad counter index (#GP fault). It is model specific as to which counters are available to begin with, and in levelled scenarios, this information may not be accurate in the first place.
This change isn't expected to have any impact on VMs. Userspace is not usually given access to RDPMC (Windows appear to completely prohibit it; Linux is restricted to root), and kernels won't be executing RDPMC instructions if their PMU drivers have failed to start.
Signed-off-by Andrew Cooper <[email protected]>---
CC: Jan Beulich <[email protected]>
CC: Wei Liu <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Jun Nakajima <[email protected]>
CC: Kevin Tian <[email protected]>
CC: Boris Ostrovsky <[email protected]>
CC: Suravee Suthikulpanit <[email protected]>
CC: Brian Woods <[email protected]>
CC: Juergen Gross <[email protected]>
Quoted 7 lines
This should be taken into Xen 4.12 and backported to the stable releases. While it isn't an XSA itself, it is an information leak (Xen's NMI watchdog in particular) which could be advantagous to an attacker trying to exploit a race condition.
The only other option is to emulate the reported family and offer back all 0's for the accessable counters. Obviously this is a non-starter.
When VPMU is off MSR reads return zero. While it is debatable whether this the right action, shouldn't rdpmc behave in the same fashion?
-boris
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Quoted 27 lines · Boris Ostrovsky
Quoted 26 lines · Andrew Cooper
vPMU isn't security supported, and in general guests can't access any of the performance counter MSRs. However, the RDPMC instruction isn't intercepted, meaning that guest software can read the instantaneous counter values.
When vPMU isn't configured, intercept RDPMC and unconditionally fail it as if software has requested a bad counter index (#GP fault). It is model specific as to which counters are available to begin with, and in levelled scenarios, this information may not be accurate in the first place.
This change isn't expected to have any impact on VMs. Userspace is not usually given access to RDPMC (Windows appear to completely prohibit it; Linux is restricted to root), and kernels won't be executing RDPMC instructions if their PMU drivers have failed to start.
Signed-off-by Andrew Cooper <[email protected]>---
CC: Jan Beulich <[email protected]>
CC: Wei Liu <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Jun Nakajima <[email protected]>
CC: Kevin Tian <[email protected]>
CC: Boris Ostrovsky <[email protected]>
CC: Suravee Suthikulpanit <[email protected]>
CC: Brian Woods <[email protected]>
CC: Juergen Gross <[email protected]>
Quoted 8 lines
Quoted 7 lines
This should be taken into Xen 4.12 and backported to the stable releases. While it isn't an XSA itself, it is an information leak (Xen's NMI watchdog in particular) which could be advantagous to an attacker trying to exploit a race condition.
The only other option is to emulate the reported family and offer back all 0's for the accessable counters. Obviously this is a non-starter.
When VPMU is off MSR reads return zero.
That behaviour isn't long for this world.
Quoted 1 line
While it is debatable whether this the right action, shouldn't rdpmc behave in the same fashion?
I specifically don't want to propagate the "lets complete with zero" behaviour further, because it takes away #GP faults which the guest would otherwise get.
~Andrew
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Quoted 35 lines · Andrew Cooper
Quoted 34 lines · Boris Ostrovsky
Quoted 33 lines · Andrew Cooper
vPMU isn't security supported, and in general guests can't access any of the performance counter MSRs. However, the RDPMC instruction isn't intercepted, meaning that guest software can read the instantaneous counter values.
When vPMU isn't configured, intercept RDPMC and unconditionally fail it as if software has requested a bad counter index (#GP fault). It is model specific as to which counters are available to begin with, and in levelled scenarios, this information may not be accurate in the first place.
This change isn't expected to have any impact on VMs. Userspace is not usually given access to RDPMC (Windows appear to completely prohibit it; Linux is restricted to root), and kernels won't be executing RDPMC instructions if their PMU drivers have failed to start.
Signed-off-by Andrew Cooper <[email protected]>---
CC: Jan Beulich <[email protected]>
CC: Wei Liu <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Jun Nakajima <[email protected]>
CC: Kevin Tian <[email protected]>
CC: Boris Ostrovsky <[email protected]>
CC: Suravee Suthikulpanit <[email protected]>
CC: Brian Woods <[email protected]>
CC: Juergen Gross <[email protected]>This should be taken into Xen 4.12 and backported to the stable releases. While it isn't an XSA itself, it is an information leak (Xen's NMI watchdog in particular) which could be advantagous to an attacker trying to exploit a race condition.
The only other option is to emulate the reported family and offer back all 0's for the accessable counters. Obviously this is a non-starter.
Quoted 7 lines
Quoted 1 line
When VPMU is off MSR reads return zero.
That behaviour isn't long for this world.
Quoted 1 line
While it is debatable whether this the right action, shouldn't rdpmc behave in the same fashion?
I specifically don't want to propagate the "lets complete with zero" behaviour further, because it takes away #GP faults which the guest would otherwise get.
The guest should get a #GP on Intel if CPUID is not reporting any counters but not on AMD where the first 4 counters are architectural.
-boris
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Quoted 10 lines
Quoted 1 line
Quoted 1 line · On 23.02.19 at 00:48, <[email protected]> wrote:
Quoted 7 lines · Andrew Cooper
Quoted 6 lines · Boris Ostrovsky
Quoted 5 lines · Andrew Cooper
vPMU isn't security supported, and in general guests can't access any of the performance counter MSRs. However, the RDPMC instruction isn't intercepted, meaning that guest software can read the instantaneous counter values.
When vPMU isn't configured, intercept RDPMC and unconditionally fail it as
if
Quoted 30 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
software has requested a bad counter index (#GP fault). It is model
specific
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
as to which counters are available to begin with, and in levelled scenarios, this information may not be accurate in the first place.
This change isn't expected to have any impact on VMs. Userspace is not usually given access to RDPMC (Windows appear to completely prohibit it;
Linux
Quoted 22 lines
Quoted 22 lines
Quoted 22 lines
is restricted to root), and kernels won't be executing RDPMC instructions if their PMU drivers have failed to start.
Signed-off-by Andrew Cooper <[email protected]>---
CC: Jan Beulich <[email protected]>
CC: Wei Liu <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Jun Nakajima <[email protected]>
CC: Kevin Tian <[email protected]>
CC: Boris Ostrovsky <[email protected]>
CC: Suravee Suthikulpanit <[email protected]>
CC: Brian Woods <[email protected]>
CC: Juergen Gross <[email protected]>This should be taken into Xen 4.12 and backported to the stable releases. While it isn't an XSA itself, it is an information leak (Xen's NMI watchdog in particular) which could be advantagous to an attacker trying to exploit a race condition.
The only other option is to emulate the reported family and offer back all 0's for the accessable counters. Obviously this is a non-starter.
I don't really understand why you say this - Boris certainly has a point ...
Quoted 10 lines
Quoted 7 lines
Quoted 1 line
When VPMU is off MSR reads return zero.
That behaviour isn't long for this world.
Quoted 1 line
While it is debatable whether this the right action, shouldn't rdpmc behave in the same fashion?
I specifically don't want to propagate the "lets complete with zero" behaviour further, because it takes away #GP faults which the guest would otherwise get.
The guest should get a #GP on Intel if CPUID is not reporting any counters but not on AMD where the first 4 counters are architectural.
... here. For Intel, afaics, we indeed produce a blank CPUID leaf in all cases, so the behavior looks reasonably consistent. I would question though whether a blank CPUID leaf / the absence of any counters wouldn't call for #UD instead of #GP(0). Otherwise, along the lines of AMD, aren't the first two indexes uniformly valid for Intel?
Additionally aren't you invoking vpmu_available() before the data it examines actually got set? Afaics vpmu_initialise() gets called after hvm_vcpu_initialise(), yet the latter is where you add the intercept enabling.
Jan
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Quoted 45 lines · Jan Beulich
Quoted 10 lines
Quoted 1 line
Quoted 1 line · On 23.02.19 at 00:48, <[email protected]> wrote:
Quoted 7 lines · Andrew Cooper
Quoted 6 lines · Boris Ostrovsky
Quoted 5 lines · Andrew Cooper
vPMU isn't security supported, and in general guests can't access any of the performance counter MSRs. However, the RDPMC instruction isn't intercepted, meaning that guest software can read the instantaneous counter values.
When vPMU isn't configured, intercept RDPMC and unconditionally fail it as
if
Quoted 30 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
software has requested a bad counter index (#GP fault). It is model
specific
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
as to which counters are available to begin with, and in levelled scenarios, this information may not be accurate in the first place.
This change isn't expected to have any impact on VMs. Userspace is not usually given access to RDPMC (Windows appear to completely prohibit it;
Linux
Quoted 22 lines
Quoted 22 lines
Quoted 22 lines
is restricted to root), and kernels won't be executing RDPMC instructions if their PMU drivers have failed to start.
Signed-off-by Andrew Cooper <[email protected]>---
CC: Jan Beulich <[email protected]>
CC: Wei Liu <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Jun Nakajima <[email protected]>
CC: Kevin Tian <[email protected]>
CC: Boris Ostrovsky <[email protected]>
CC: Suravee Suthikulpanit <[email protected]>
CC: Brian Woods <[email protected]>
CC: Juergen Gross <[email protected]>This should be taken into Xen 4.12 and backported to the stable releases. While it isn't an XSA itself, it is an information leak (Xen's NMI watchdog in particular) which could be advantagous to an attacker trying to exploit a race condition.
The only other option is to emulate the reported family and offer back all 0's for the accessable counters. Obviously this is a non-starter.
I don't really understand why you say this - Boris certainly has a point ...
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
When VPMU is off MSR reads return zero.
That behaviour isn't long for this world.
Quoted 7 lines
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
While it is debatable whether this the right action, shouldn't rdpmc behave in the same fashion?
I specifically don't want to propagate the "lets complete with zero" behaviour further, because it takes away #GP faults which the guest would otherwise get.
The guest should get a #GP on Intel if CPUID is not reporting any counters but not on AMD where the first 4 counters are architectural.
... here.
No - just because something is architectural doesn't mean the guest gets to play with it.
Especially not for vPMU where the code is of such bad quality we had to disable in a security fix, and re-disable it again later in another security fix.
Quoted 4 lines
For Intel, afaics, we indeed produce a blank CPUID leaf in all cases, so the behavior looks reasonably consistent. I would question though whether a blank CPUID leaf / the absence of any counters wouldn't call for #UD instead of #GP(0).
RDPMC hasn't #UD'd in a quarter of a century, but does #GP in userspace outside of developer profiling scenarios.
Quoted 3 lines
Otherwise,
along the lines of AMD, aren't the first two indexes uniformly valid for Intel?
No - its model specific behaviour. The only difference for more modern systems is that they have agreed on a common behaviour.
And that is specifically why implementing 0's is a non-starter - it is not a remotely sensible use of time to build enough infrastructure to provide correct model-specific behaviour just for a corner case which operating systems don't encounter in practice.
Quoted 4 lines
Additionally aren't you invoking
vpmu_available()before the data it examines actually got set? Afaicsvpmu_initialise()gets called afterhvm_vcpu_initialise(), yet the latter is where you add the intercept enabling.
Hmm yes - that does appear to break the vpmu case. The order of initialisation will need tweaking as well.
~Andrew
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Quoted 9 lines
Quoted 1 line
Quoted 1 line · On 25.02.19 at 15:11, <[email protected]> wrote:
Quoted 4 lines · Jan Beulich
For Intel, afaics, we indeed produce a blank CPUID leaf in all cases, so the behavior looks reasonably consistent. I would question though whether a blank CPUID leaf / the absence of any counters wouldn't call for #UD instead of #GP(0).
RDPMC hasn't #UD'd in a quarter of a century, but does #GP in userspace outside of developer profiling scenarios.
I guess I could equally well say that RDPMC hasn't #GP'd for as long for indexes zero and one.
Quoted 11 lines
Quoted 3 lines
Otherwise,
along the lines of AMD, aren't the first two indexes uniformly valid for Intel?No - its model specific behaviour. The only difference for more modern systems is that they have agreed on a common behaviour.
And that is specifically why implementing 0's is a non-starter - it is not a remotely sensible use of time to build enough infrastructure to provide correct model-specific behaviour just for a corner case which operating systems don't encounter in practice.
No-one said you need to consider all cases. But returning zeros for the first four (AMD) or two (Intel) counters can hardly be that big of a problem.
Anyway - I'm not going to fight this much more, as vPMU clearly isn't my primary area of interest. But I'll listen to further comments from Boris, wrt giving an eventual ack.
Jan
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Quoted 21 lines · Jan Beulich
Quoted 8 lines
Quoted 1 line
Quoted 1 line · On 25.02.19 at 15:11, <[email protected]> wrote:
Quoted 4 lines · Jan Beulich
For Intel, afaics, we indeed produce a blank CPUID leaf in all cases, so the behavior looks reasonably consistent. I would question though whether a blank CPUID leaf / the absence of any counters wouldn't call for #UD instead of #GP(0).
RDPMC hasn't #UD'd in a quarter of a century, but does #GP in userspace outside of developer profiling scenarios.
I guess I could equally well say that RDPMC hasn't #GP'd for as long for indexes zero and one.
Quoted 10 lines
Quoted 3 lines
Otherwise,
along the lines of AMD, aren't the first two indexes uniformly valid for Intel?No - its model specific behaviour. The only difference for more modern systems is that they have agreed on a common behaviour.
And that is specifically why implementing 0's is a non-starter - it is not a remotely sensible use of time to build enough infrastructure to provide correct model-specific behaviour just for a corner case which operating systems don't encounter in practice.
Quoted 7 lines
No-one said you need to consider all cases. But returning zeros for the first four (AMD) or two (Intel) counters can hardly be that big of a problem.
Anyway - I'm not going to fight this much more, as vPMU clearly isn't my primary area of interest. But I'll listen to further comments from Boris, wrt giving an eventual ack.
The most important thing IMO is to make MSR accesses and rdpmc be consistent with each other.
As far as faulting on them as opposed to returning zero on reads and dropping writes --- neither is right I think. The difference is that with faulting a guest might suddenly start crashing. Not Linux since it does rdmsrl_safe() in check_hw_exists() but who knows what Windows do, for example.
-boris
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel