KVM: x86: Move kvm_check_request(KVM_REQ_NMI) after kvm_check_request(KVM_REQ_NMI)
PATCH 1/1 KVM: x86: Move kvm_check_request(KVM_REQ_NMI) after kvm_check_request(KVM_REQ_NMI)
Move kvm_check_request(KVM_REQ_NMI) after kvm_check_request(KVM_REQ_NMI). When vPMU is active use, processing each KVM_REQ_PMI will generate a KVM_REQ_NMI. Existing control flow after KVM_REQ_PMI finished will fail the guest enter, jump to kvm_x86_cancel_injection(), and re-enter vcpu_enter_guest(), this wasted lot of cycles and increase the overhead for vPMU as well as the virtualization.
So move the code snippet of kvm_check_request(KVM_REQ_NMI) to make KVM runloop more efficient with vPMU.
To evaluate the effectiveness of this change, we launch a 8-vcpu QEMU VM on an Intel SPR CPU. In the VM, we run perf with all 48 events Intel vtune uses. In addition, we use SPEC2017 benchmark programs as the workload with the setup of using single core, single thread.
At the host level, we probe the invocations to vmx_cancel_injection() with the following command:
$ perf probe -a vmx_cancel_injection
$ perf stat -a -e probe:vmx_cancel_injection -I 10000 # per 10 secondsThe following is the result that we collected at beginning of the spec2017 benchmark run (so mostly for 500.perlbench_r in spec2017). Kindly forgive the incompleteness.
On kernel without the change:
10.010018010 14254 probe:vmx_cancel_injection
20.037646388 15207 probe:vmx_cancel_injection
30.078739816 15261 probe:vmx_cancel_injection
40.114033258 15085 probe:vmx_cancel_injection
50.149297460 15112 probe:vmx_cancel_injection
60.185103088 15104 probe:vmx_cancel_injectionOn kernel with the change:
10.003595390 40 probe:vmx_cancel_injection
20.017855682 31 probe:vmx_cancel_injection
30.028355883 34 probe:vmx_cancel_injection
40.038686298 31 probe:vmx_cancel_injection
50.048795162 20 probe:vmx_cancel_injection
60.069057747 19 probe:vmx_cancel_injectionFrom the above, it is clear that we save 1500 invocations per vcpu per second to vmx_cancel_injection() for workloads like perlbench.
--- arch/x86/kvm/x86.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
arch/x86/kvm/x86.c +2 -2
@@ -10580,12 +10580,12 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)if (kvm_check_request (KVM_REQ_SMI , vcpu))process_smi (vcpu);#endif -if (kvm_check_request (KVM_REQ_NMI , vcpu)) -process_nmi (vcpu);if (kvm_check_request (KVM_REQ_PMU , vcpu))kvm_pmu_handle_event (vcpu);if (kvm_check_request (KVM_REQ_PMI , vcpu))kvm_pmu_deliver_pmi (vcpu); +if (kvm_check_request (KVM_REQ_NMI , vcpu)) +process_nmi (vcpu);if (kvm_check_request (KVM_REQ_IOAPIC_EOI_EXIT , vcpu)) {BUG_ON (vcpu->arch.pending_ioapic_eoi >255 ); if (test_bit(vcpu->arch.pending_ioapic_eoi,
base-commit: 73554b29bd70546c1a9efc9c160641ef1b849358
Signature · 1 line
2.42.0.515.g380fc7ccd1-goog
ah, typo in the subject: The 2nd KVM_REQ_NMI should be KVM_REQ_PMI. Sorry about that.
Quoted 74 lines · Mingwei Zhang
Move
kvm_check_request(KVM_REQ_NMI)afterkvm_check_request(KVM_REQ_NMI). When vPMU is active use, processing eachKVM_REQ_PMIwill generate aKVM_REQ_NMI. Existing control flow afterKVM_REQ_PMIfinished will fail the guest enter, jump tokvm_x86_cancel_injection(), and re-entervcpu_enter_guest(), this wasted lot of cycles and increase the overhead for vPMU as well as the virtualization.So move the code snippet of
kvm_check_request(KVM_REQ_NMI)to make KVM runloop more efficient with vPMU.To evaluate the effectiveness of this change, we launch a 8-vcpu QEMU VM on an Intel SPR CPU. In the VM, we run perf with all 48 events Intel vtune uses. In addition, we use SPEC2017 benchmark programs as the workload with the setup of using single core, single thread.
At the host level, we probe the invocations to
vmx_cancel_injection()with the following command:$ perf probe -a vmx_cancel_injection $ perf stat -a -e probe:vmx_cancel_injection -I 10000 # per 10 secondsThe following is the result that we collected at beginning of the spec2017 benchmark run (so mostly for 500.perlbench_r in spec2017). Kindly forgive the incompleteness.
On kernel without the change:
10.010018010 14254 probe:vmx_cancel_injection 20.037646388 15207 probe:vmx_cancel_injection 30.078739816 15261 probe:vmx_cancel_injection 40.114033258 15085 probe:vmx_cancel_injection 50.149297460 15112 probe:vmx_cancel_injection 60.185103088 15104 probe:vmx_cancel_injectionOn kernel with the change:
10.003595390 40 probe:vmx_cancel_injection 20.017855682 31 probe:vmx_cancel_injection 30.028355883 34 probe:vmx_cancel_injection 40.038686298 31 probe:vmx_cancel_injection 50.048795162 20 probe:vmx_cancel_injection 60.069057747 19 probe:vmx_cancel_injectionFrom the above, it is clear that we save 1500 invocations per vcpu per second to
vmx_cancel_injection()for workloads like perlbench.Signed-off-by Mingwei Zhang <[email protected]>--- arch/x86/kvm/x86.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)arch/x86/kvm/x86.c +2
-2@@ -10580,12 +10580,12 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)if (kvm_check_request (KVM_REQ_SMI , vcpu))process_smi (vcpu);#endif -if (kvm_check_request (KVM_REQ_NMI , vcpu)) -process_nmi (vcpu);if (kvm_check_request (KVM_REQ_PMU , vcpu))kvm_pmu_handle_event (vcpu);if (kvm_check_request (KVM_REQ_PMI , vcpu))kvm_pmu_deliver_pmi (vcpu); +if (kvm_check_request (KVM_REQ_NMI , vcpu)) +process_nmi (vcpu);if (kvm_check_request (KVM_REQ_IOAPIC_EOI_EXIT , vcpu)) {BUG_ON (vcpu->arch.pending_ioapic_eoi >255 ); if (test_bit(vcpu->arch.pending_ioapic_eoi,base-commit:
73554b29bd70546c1a9efc9c160641ef1b849358--
2.42.0.515.g380fc7ccd1-goog
Quoted 6 lines · Mingwei Zhang
ah, typo in the subject: The 2nd
KVM_REQ_NMIshould beKVM_REQ_PMI. Sorry about that.Quoted 2 lines · Mingwei Zhang
Move
kvm_check_request(KVM_REQ_NMI)afterkvm_check_request(KVM_REQ_NMI).
Please remove it, no need to repeat the subject.
Quoted 5 lines
Quoted 5 lines
When vPMU is active use, processing each
KVM_REQ_PMIwill generate aKVM_REQ_NMI. Existing control flow afterKVM_REQ_PMIfinished will fail the guest enter, jump tokvm_x86_cancel_injection(), and re-entervcpu_enter_guest(), this wasted lot of cycles and increase the overhead for vPMU as well as the virtualization.
Optimization is after correctness, so please explain if this is correct first!
Quoted 68 lines
Quoted 67 lines
So move the code snippet of
kvm_check_request(KVM_REQ_NMI)to make KVM runloop more efficient with vPMU.To evaluate the effectiveness of this change, we launch a 8-vcpu QEMU VM on an Intel SPR CPU. In the VM, we run perf with all 48 events Intel vtune uses. In addition, we use SPEC2017 benchmark programs as the workload with the setup of using single core, single thread.
At the host level, we probe the invocations to
vmx_cancel_injection()with the following command:$ perf probe -a vmx_cancel_injection $ perf stat -a -e probe:vmx_cancel_injection -I 10000 # per 10 secondsThe following is the result that we collected at beginning of the spec2017 benchmark run (so mostly for 500.perlbench_r in spec2017). Kindly forgive the incompleteness.
On kernel without the change:
10.010018010 14254 probe:vmx_cancel_injection 20.037646388 15207 probe:vmx_cancel_injection 30.078739816 15261 probe:vmx_cancel_injection 40.114033258 15085 probe:vmx_cancel_injection 50.149297460 15112 probe:vmx_cancel_injection 60.185103088 15104 probe:vmx_cancel_injectionOn kernel with the change:
10.003595390 40 probe:vmx_cancel_injection 20.017855682 31 probe:vmx_cancel_injection 30.028355883 34 probe:vmx_cancel_injection 40.038686298 31 probe:vmx_cancel_injection 50.048795162 20 probe:vmx_cancel_injection 60.069057747 19 probe:vmx_cancel_injectionFrom the above, it is clear that we save 1500 invocations per vcpu per
second tovmx_cancel_injection()for workloads like perlbench.Signed-off-by Mingwei Zhang <[email protected]>--- arch/x86/kvm/x86.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)arch/x86/kvm/x86.c +2
-2@@ -10580,12 +10580,12 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)if (kvm_check_request (KVM_REQ_SMI , vcpu))process_smi (vcpu);#endif -if (kvm_check_request (KVM_REQ_NMI , vcpu)) -process_nmi (vcpu);if (kvm_check_request (KVM_REQ_PMU , vcpu))kvm_pmu_handle_event (vcpu);if (kvm_check_request (KVM_REQ_PMI , vcpu))kvm_pmu_deliver_pmi (vcpu); +if (kvm_check_request (KVM_REQ_NMI , vcpu)) +process_nmi (vcpu);if (kvm_check_request (KVM_REQ_IOAPIC_EOI_EXIT , vcpu)) {BUG_ON (vcpu->arch.pending_ioapic_eoi >255 ); if (test_bit(vcpu->arch.pending_ioapic_eoi,base-commit:
73554b29bd70546c1a9efc9c160641ef1b849358--
2.42.0.515.g380fc7ccd1-goog
Signature · 2 lines
Thanks!
XinSean Christopherson
maintainerQuoted 9 lines · Xin Li
Quoted 6 lines · Mingwei Zhang
ah, typo in the subject: The 2nd
KVM_REQ_NMIshould beKVM_REQ_PMI. Sorry about that.Quoted 2 lines · Mingwei Zhang
Move
kvm_check_request(KVM_REQ_NMI)afterkvm_check_request(KVM_REQ_NMI).Please remove it, no need to repeat the subject.
Heh, from Documentation/process/maintainer-kvm-x86.rst:
Changelog
~~~~~~~~~
Most importantly, write changelogs using imperative mood and avoid pronouns.
See :ref:describe_changes for more information, with one amendment: lead with a short blurb on the actual changes, and then follow up with the context and background. Note! This order directly conflicts with the tip tree's preferred approach! Please follow the tip tree's preferred style when sending patches that primarily target arch/x86 code that is _NOT_ KVM code.
That said, I do prefer that the changelog intro isn't just a copy+paste of the shortlog, and the shortlog and changelog should use conversational language instead of describing the literal code movement.
Quoted 1 line
Quoted 1 line
Quoted 1 line
When vPMU is active use, processing each
KVM_REQ_PMIwill generate a
This is not guaranteed.
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
KVM_REQ_NMI. Existing control flow afterKVM_REQ_PMIfinished will fail the guest enter, jump tokvm_x86_cancel_injection(), and re-entervcpu_enter_guest(), this wasted lot of cycles and increase the overhead for vPMU as well as the virtualization.
As above, use conversational language, the changelog isn't meant to be a play-by-play.
E.g.
KVM: x86: Service NMI requests *after* PMI requests in VM-Enter path
Move the handling of NMI requests after PMI requests in the VM-Enter path so that KVM doesn't need to cancel and redo VM-Enter in the likely scenario that the vCPU has configured its LVPTC entry to generate an NMI.
Because APIC emulation "injects" NMIs via KVM_REQ_NMI, handling PMI requests after NMI requests means KVM won't detect the pending NMI request until the final check for outstanding requests. Detecting requests at the final stage is costly as KVM has already loaded guest state, potentially queued events for injection, disabled IRQs, dropped SRCU, etc., most of which needs to be unwound.
Quoted 2 lines
Optimization is after correctness, so please explain if this is correct first!
Not first. Leading with an in-depth description of KVM requests and NMI handling is not going to help understand *why* this change is being first. But I do agree that this should provide an analysis of why it's ok to swap the order, specificially why it's architecturally ok if KVM drops an NMI due to the swapped ordering, e.g. if the PMI is coincident with two other NMIs (or one other NMI and NMIs are blocked).
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
So move the code snippet of
kvm_check_request(KVM_REQ_NMI)to make KVM runloop more efficient with vPMU.To evaluate the effectiveness of this change, we launch a 8-vcpu QEMU VM on
Avoid pronouns. There's no need for all the "fluff", just state the setup, the test, and the results.
Really getting into the nits, but the whole "8-vcpu QEMU VM" versus "the setup of using single core, single thread" is confusing IMO. If there were potential performance downsides and/or tradeoffs, then getting the gory details might be necessary, but that's not the case here, and if it were really necessary to drill down that deep, then I would want to better quantify the impact, e.g. in terms latency.
E.g. on Intel SPR running SPEC2017 benchmark and Intel vtune in the guest, handling PMI requests before NMI requests reduces the number of canceled runs by ~1500 per second, per vCPU (counted by probing calls to vmx_cancel_injection()).
Quoted 30 lines
Quoted 30 lines
Quoted 30 lines
an Intel SPR CPU. In the VM, we run perf with all 48 events Intel vtune uses. In addition, we use SPEC2017 benchmark programs as the workload with the setup of using single core, single thread.
At the host level, we probe the invocations to
vmx_cancel_injection()with the following command:$ perf probe -a vmx_cancel_injection $ perf stat -a -e probe:vmx_cancel_injection -I 10000 # per 10 secondsThe following is the result that we collected at beginning of the spec2017 benchmark run (so mostly for 500.perlbench_r in spec2017). Kindly forgive the incompleteness.
On kernel without the change:
10.010018010 14254 probe:vmx_cancel_injection 20.037646388 15207 probe:vmx_cancel_injection 30.078739816 15261 probe:vmx_cancel_injection 40.114033258 15085 probe:vmx_cancel_injection 50.149297460 15112 probe:vmx_cancel_injection 60.185103088 15104 probe:vmx_cancel_injectionOn kernel with the change:
10.003595390 40 probe:vmx_cancel_injection 20.017855682 31 probe:vmx_cancel_injection 30.028355883 34 probe:vmx_cancel_injection 40.038686298 31 probe:vmx_cancel_injection 50.048795162 20 probe:vmx_cancel_injection 60.069057747 19 probe:vmx_cancel_injection
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
From the above, it is clear that we save 1500 invocations per vcpu per
second tovmx_cancel_injection()for workloads like perlbench.
Nit, this really should have:
I personally don't care about the attribution, but (a) others often do care and
(b) the added context is helpful. E.g. for bad/questionable suggestsions/ideas, knowing that person X was also involved helps direct and/or curate questions/comments accordingly.
Quoted 116 lines · Sean Christopherson
Quoted 9 lines · Xin Li
Quoted 6 lines · Mingwei Zhang
ah, typo in the subject: The 2nd
KVM_REQ_NMIshould beKVM_REQ_PMI. Sorry about that.Quoted 2 lines · Mingwei Zhang
Move
kvm_check_request(KVM_REQ_NMI)afterkvm_check_request(KVM_REQ_NMI).Please remove it, no need to repeat the subject.
Heh, from
Documentation/process/maintainer-kvm-x86.rst:Changelog
~~~~~~~~~
Most importantly, write changelogs using imperative mood and avoid pronouns.See :ref:
describe_changesfor more information, with one amendment: lead with a short blurb on the actual changes, and then follow up with the context and background. Note! This order directly conflicts with the tip tree's preferred approach! Please follow the tip tree's preferred style when sending patches that primarily targetarch/x86code that is _NOT_ KVM code.That said, I do prefer that the changelog intro isn't just a copy+paste of the shortlog, and the shortlog and changelog should use conversational language instead of describing the literal code movement.
Quoted 1 line
Quoted 1 line
Quoted 1 line
When vPMU is active use, processing each
KVM_REQ_PMIwill generate aThis is not guaranteed.
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
KVM_REQ_NMI. Existing control flow afterKVM_REQ_PMIfinished will fail the guest enter, jump tokvm_x86_cancel_injection(), and re-entervcpu_enter_guest(), this wasted lot of cycles and increase the overhead for vPMU as well as the virtualization.As above, use conversational language, the changelog isn't meant to be a play-by-play.
E.g.
KVM: x86: Service NMI requests *after* PMI requests in VM-Enter path
Move the handling of NMI requests after PMI requests in the VM-Enter path so that KVM doesn't need to cancel and redo VM-Enter in the likely scenario that the vCPU has configured its LVPTC entry to generate an NMI.
Because APIC emulation "injects" NMIs via
KVM_REQ_NMI, handling PMI requests after NMI requests means KVM won't detect the pending NMI request until the final check for outstanding requests. Detecting requests at the final stage is costly as KVM has already loaded guest state, potentially queued events for injection, disabled IRQs, dropped SRCU, etc., most of which needs to be unwound.Quoted 2 lines
Optimization is after correctness, so please explain if this is correct first!
Not first. Leading with an in-depth description of KVM requests and NMI handling is not going to help understand *why* this change is being first. But I do agree that this should provide an analysis of why it's ok to swap the order, specificially why it's architecturally ok if KVM drops an NMI due to the swapped ordering, e.g. if the PMI is coincident with two other NMIs (or one other NMI and NMIs are blocked).
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
So move the code snippet of
kvm_check_request(KVM_REQ_NMI)to make KVM runloop more efficient with vPMU.To evaluate the effectiveness of this change, we launch a 8-vcpu QEMU VM on
Avoid pronouns. There's no need for all the "fluff", just state the setup, the test, and the results.
Really getting into the nits, but the whole "8-vcpu QEMU VM" versus "the setup of using single core, single thread" is confusing IMO. If there were potential performance downsides and/or tradeoffs, then getting the gory details might be necessary, but that's not the case here, and if it were really necessary to drill down that deep, then I would want to better quantify the impact, e.g. in terms latency.
E.g. on Intel SPR running SPEC2017 benchmark and Intel vtune in the guest, handling PMI requests before NMI requests reduces the number of canceled runs by ~1500 per second, per vCPU (counted by probing calls to
vmx_cancel_injection()).Quoted 30 lines
Quoted 30 lines
Quoted 30 lines
an Intel SPR CPU. In the VM, we run perf with all 48 events Intel vtune uses. In addition, we use SPEC2017 benchmark programs as the workload with the setup of using single core, single thread.
At the host level, we probe the invocations to
vmx_cancel_injection()with the following command:$ perf probe -a vmx_cancel_injection $ perf stat -a -e probe:vmx_cancel_injection -I 10000 # per 10 secondsThe following is the result that we collected at beginning of the spec2017 benchmark run (so mostly for 500.perlbench_r in spec2017). Kindly forgive the incompleteness.
On kernel without the change:
10.010018010 14254 probe:vmx_cancel_injection 20.037646388 15207 probe:vmx_cancel_injection 30.078739816 15261 probe:vmx_cancel_injection 40.114033258 15085 probe:vmx_cancel_injection 50.149297460 15112 probe:vmx_cancel_injection 60.185103088 15104 probe:vmx_cancel_injectionOn kernel with the change:
10.003595390 40 probe:vmx_cancel_injection 20.017855682 31 probe:vmx_cancel_injection 30.028355883 34 probe:vmx_cancel_injection 40.038686298 31 probe:vmx_cancel_injection 50.048795162 20 probe:vmx_cancel_injection 60.069057747 19 probe:vmx_cancel_injectionQuoted 2 lines
Quoted 2 lines
Quoted 2 lines
From the above, it is clear that we save 1500 invocations per vcpu per
second tovmx_cancel_injection()for workloads like perlbench.
Quoted 8 lines
Nit, this really should have:
Suggested-by Sean Christopherson <[email protected]>I personally don't care about the attribution, but (a) others often do care and
(b) the added context is helpful. E.g. for bad/questionable suggestsions/ideas, knowing that person X was also involved helps direct and/or curate questions/comments accordingly.
For sure! I will also pay more attention to that in the future.
Thanks.
-Mingwei