KVM: PPC: Book3S HV: XIVE: assorted fixes on vCPU and RAM limits
Subsystems (3)
cover KVM: PPC: Book3S HV: XIVE: assorted fixes on vCPU and RAM limits
Hello,
Here are a couple of fixes for issues in the XIVE KVM device when testing the limits : RAM size and number of vCPUS.
Based on 5.2-rc1.
Available on GitHub:
https://github.com/legoater/linux/commits/xive-5.2
Thanks,
C.
Cédric Le Goater (3):
KVM: PPC: Book3S HV: XIVE: clear file mapping when device is released
KVM: PPC: Book3S HV: XIVE: do not test the EQ flag validity when
reseting
KVM: PPC: Book3S HV: XIVE: fix the enforced limit on the vCPU
identifier
arch/powerpc/kvm/book3s_xive_native.c | 46 ++++++++++++++++----------- 1 file changed, 27 insertions(+), 19 deletions(-)
Signature · 1 line
2.20.1
PATCH 1/3 KVM: PPC: Book3S HV: XIVE: clear file mapping when device is released
Improve the release of the XIVE KVM device by clearing the file address_space which is used to unmap the interrupt ESB pages when a device is passed-through.
--- arch/powerpc/kvm/book3s_xive_native.c | 8 ++++++++ 1 file changed, 8 insertions(+)
arch/powerpc/kvm/book3s_xive_native.c +8 -0
@@ -979,6 +979,14 @@ static void kvmppc_xive_native_release(struct kvm_device *dev)pr_devel ("Releasing xive native device\n" ); +/* +* Clear the KVM device file address_space which is used to +* unmap the ESB pages when a device is passed-through. +*/ +mutex_lock (&xive->mapping_lock); + xive->mapping =NULL ; +mutex_unlock (&xive->mapping_lock); +/* * Clearing mmu_ready temporarily while holding kvm->lock * is a way of ensuring that no vcpus can enter the guest
Signature · 1 line
2.20.1
PATCH 2/3 KVM: PPC: Book3S HV: XIVE: do not test the EQ flag validity when resetting
In reply to Cédric Le GoaterWhen a CPU is hot-unplugged, the EQ is deconfigured using a zero size and a zero address. In this case, there is no need to check the flag and queue size validity. Move the checks after the queue reset code section to fix CPU hot-unplug.
--- arch/powerpc/kvm/book3s_xive_native.c | 36 +++++++++++++-------------- 1 file changed, 18 insertions(+), 18 deletions(-)
arch/powerpc/kvm/book3s_xive_native.c +18 -18
@@ -565,24 +565,6 @@ static int kvmppc_xive_native_set_queue_config(struct kvmppc_xive *xive, __func__, server, priority, kvm_eq.flags, kvm_eq.qshift, kvm_eq.qaddr, kvm_eq.qtoggle, kvm_eq.qindex); -/* -* sPAPR specifies a "Unconditional Notify (n) flag" for the -* H_INT_SET_QUEUE_CONFIG hcall which forces notification -* without using the coalescing mechanisms provided by the -* XIVE END ESBs. This is required on KVM as notification -* using the END ESBs is not supported. -*/ -if (kvm_eq.flags !=KVM_XIVE_EQ_ALWAYS_NOTIFY ) { -pr_err ("invalid flags %d\n" , kvm_eq.flags); -return -EINVAL ; - } - - rc =xive_native_validate_queue_size (kvm_eq.qshift); -if (rc) { -pr_err ("invalid queue size %d\n" , kvm_eq.qshift); -return rc; - } -/* reset queue and disable queueing */ if (!kvm_eq.qshift) { q->guest_qaddr =0 ;
@@ -604,6 +586,24 @@ static int kvmppc_xive_native_set_queue_config(struct kvmppc_xive *xive,return 0 ; } +/* +* sPAPR specifies a "Unconditional Notify (n) flag" for the +* H_INT_SET_QUEUE_CONFIG hcall which forces notification +* without using the coalescing mechanisms provided by the +* XIVE END ESBs. This is required on KVM as notification +* using the END ESBs is not supported. +*/ +if (kvm_eq.flags !=KVM_XIVE_EQ_ALWAYS_NOTIFY ) { +pr_err ("invalid flags %d\n" , kvm_eq.flags); +return -EINVAL ; + } + + rc =xive_native_validate_queue_size (kvm_eq.qshift); +if (rc) { +pr_err ("invalid queue size %d\n" , kvm_eq.qshift); +return rc; + } +if (kvm_eq.qaddr & ((1ull << kvm_eq.qshift) -1 )) {pr_err ("queue page is not aligned %llx/%llx\n" , kvm_eq.qaddr,1ull << kvm_eq.qshift);
Signature · 1 line
2.20.1
Re: [PATCH 2/3] KVM: PPC: Book3S HV: XIVE: do not test the EQ flag validity when resetting
Quoted 9 lines · On Mon, 20 May 2019 09:15:13 +0200 Cédric Le Goater <[email protected]> wrote:
When a CPU is hot-unplugged, the EQ is deconfigured using a zero size and a zero address. In this case, there is no need to check the flag and queue size validity. Move the checks after the queue reset code section to fix CPU hot-unplug.
Reported-by Satheesh Rajendran <[email protected]>Tested-by Satheesh Rajendran <[email protected]>Signed-off-by Cédric Le Goater <[email protected]>---
Quoted 57 lines
arch/powerpc/kvm/book3s_xive_native.c | 36 +++++++++++++-------------- 1 file changed, 18 insertions(+), 18 deletions(-)arch/powerpc/kvm/book3s_xive_native.c +18
-18@@ -565,24 +565,6 @@ static int kvmppc_xive_native_set_queue_config(struct kvmppc_xive *xive, __func__, server, priority, kvm_eq.flags, kvm_eq.qshift, kvm_eq.qaddr, kvm_eq.qtoggle, kvm_eq.qindex); -/* -* sPAPR specifies a "Unconditional Notify (n) flag" for the -* H_INT_SET_QUEUE_CONFIG hcall which forces notification -* without using the coalescing mechanisms provided by the -* XIVE END ESBs. This is required on KVM as notification -* using the END ESBs is not supported. -*/ -if (kvm_eq.flags !=KVM_XIVE_EQ_ALWAYS_NOTIFY ) { -pr_err ("invalid flags %d\n" , kvm_eq.flags); -return -EINVAL ; - } - - rc =xive_native_validate_queue_size (kvm_eq.qshift); -if (rc) { -pr_err ("invalid queue size %d\n" , kvm_eq.qshift); -return rc; - } -/* reset queue and disable queueing */ if (!kvm_eq.qshift) { q->guest_qaddr =0 ;@@ -604,6 +586,24 @@ static int kvmppc_xive_native_set_queue_config(struct kvmppc_xive *xive,return 0 ; } +/* +* sPAPR specifies a "Unconditional Notify (n) flag" for the +* H_INT_SET_QUEUE_CONFIG hcall which forces notification +* without using the coalescing mechanisms provided by the +* XIVE END ESBs. This is required on KVM as notification +* using the END ESBs is not supported. +*/ +if (kvm_eq.flags !=KVM_XIVE_EQ_ALWAYS_NOTIFY ) { +pr_err ("invalid flags %d\n" , kvm_eq.flags); +return -EINVAL ; + } + + rc =xive_native_validate_queue_size (kvm_eq.qshift); +if (rc) { +pr_err ("invalid queue size %d\n" , kvm_eq.qshift); +return rc; + } +if (kvm_eq.qaddr & ((1ull << kvm_eq.qshift) -1 )) {pr_err ("queue page is not aligned %llx/%llx\n" , kvm_eq.qaddr,1ull << kvm_eq.qshift);
PATCH 3/3 KVM: PPC: Book3S HV: XIVE: fix the enforced limit on the vCPU identifier
In reply to Cédric Le GoaterWhen a vCPU is connected to the KVM device, it is done using its vCPU identifier in the guest. Fix the enforced limit on the vCPU identifier by taking into account the SMT mode.
--- arch/powerpc/kvm/book3s_xive_native.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/powerpc/kvm/book3s_xive_native.c +1 -1
@@ -109,7 +109,7 @@ int kvmppc_xive_native_connect_vcpu(struct kvm_device *dev,return -EPERM ;if (vcpu->arch.irq_type !=KVMPPC_IRQ_DEFAULT )return -EBUSY ; -if (server_num >=KVM_MAX_VCPUS ) { +if (server_num >= (KVM_MAX_VCPUS * vcpu->kvm->arch.emul_smt_mode)) {pr_devel ("Out of bounds !\n" );return -EINVAL ; }
Signature · 1 line
2.20.1
Re: [PATCH 3/3] KVM: PPC: Book3S HV: XIVE: fix the enforced limit on the vCPU identifier
Quoted 4 lines · On Mon, 20 May 2019 09:15:14 +0200 Cédric Le Goater <[email protected]> wrote:
When a vCPU is connected to the KVM device, it is done using its vCPU identifier in the guest. Fix the enforced limit on the vCPU identifier by taking into account the SMT mode.
Quoted 20 lines
Reported-by Satheesh Rajendran <[email protected]>Tested-by Satheesh Rajendran <[email protected]>Signed-off-by Cédric Le Goater <[email protected]>--- arch/powerpc/kvm/book3s_xive_native.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)arch/powerpc/kvm/book3s_xive_native.c +1
-1@@ -109,7 +109,7 @@ int kvmppc_xive_native_connect_vcpu(struct kvm_device *dev,return -EPERM ;if (vcpu->arch.irq_type !=KVMPPC_IRQ_DEFAULT )return -EBUSY ; -if (server_num >=KVM_MAX_VCPUS ) { +if (server_num >= (KVM_MAX_VCPUS * vcpu->kvm->arch.emul_smt_mode)) {pr_devel ("Out of bounds !\n" );return -EINVAL ; }
Quoted 4 lines · Cédric Le Goater
Hello,
Here are a couple of fixes for issues in the XIVE KVM device when testing the limits : RAM size and number of vCPUS.
How serious are the problems these patches fix? I'm wondering if I need to make a backport for RHEL8.1.
Quoted 21 lines
Based on 5.2-rc1.
Available on GitHub:
https://github.com/legoater/linux/commits/xive-5.2
Thanks,
C.
Cédric Le Goater (3):
KVM: PPC: Book3S HV: XIVE: clear file mapping when device is released
KVM: PPC: Book3S HV: XIVE: do not test the EQ flag validity when
reseting
KVM: PPC: Book3S HV: XIVE: fix the enforced limit on the vCPU
identifierarch/powerpc/kvm/book3s_xive_native.c | 46 ++++++++++++++++----------- 1 file changed, 27 insertions(+), 19 deletions(-)
Signature · 4 lines
David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson
Quoted 9 lines · On Thu, 23 May 2019 09:30:43 +1000 David Gibson <[email protected]>…
Quoted 4 lines · Cédric Le Goater
Hello,
Here are a couple of fixes for issues in the XIVE KVM device when testing the limits : RAM size and number of vCPUS.
How serious are the problems these patches fix? I'm wondering if I need to make a backport for RHEL8.1.
Patch 2/3 fixes a QEMU error when hot-unplugging a vCPU:
qemu-system-ppc64: KVM_SET_DEVICE_ATTR failed: Group 4 attr 0x0000000000000046: Invalid argument
Patch 3/3 fixes an issue where the guest freezes at some point when doing vCPU hot-plug/unplug in a loop.
Both issues have a BZ at IBM. They can be mirrored to RH if needed.
Quoted 22 lines
Quoted 21 lines
Based on 5.2-rc1.
Available on GitHub:
https://github.com/legoater/linux/commits/xive-5.2
Thanks,
C.
Cédric Le Goater (3):
KVM: PPC: Book3S HV: XIVE: clear file mapping when device is released
KVM: PPC: Book3S HV: XIVE: do not test the EQ flag validity when
reseting
KVM: PPC: Book3S HV: XIVE: fix the enforced limit on the vCPU
identifierarch/powerpc/kvm/book3s_xive_native.c | 46 ++++++++++++++++----------- 1 file changed, 27 insertions(+), 19 deletions(-)
Quoted 13 lines · Greg Kurz
Quoted 9 lines · On Thu, 23 May 2019 09:30:43 +1000 David Gibson <[email protected]>…
Quoted 4 lines · Cédric Le Goater
Hello,
Here are a couple of fixes for issues in the XIVE KVM device when testing the limits : RAM size and number of vCPUS.
How serious are the problems these patches fix? I'm wondering if I need to make a backport for RHEL8.1.
Quoted 7 lines
Patch 2/3 fixes a QEMU error when hot-unplugging a vCPU:
qemu-system-ppc64:
KVM_SET_DEVICE_ATTRfailed: Group 4 attr0x0000000000000046: Invalid argumentPatch 3/3 fixes an issue where the guest freezes at some point when doing vCPU hot-plug/unplug in a loop.
Oh.. weird. It's not clear to me how it would do that.
Quoted 1 line
Both issues have a BZ at IBM. They can be mirrored to RH if needed.
That would be helpful, thanks.
Quoted 24 lines
Quoted 23 lines
Quoted 22 lines
Based on 5.2-rc1.
Available on GitHub:
https://github.com/legoater/linux/commits/xive-5.2
Thanks,
C.
Cédric Le Goater (3):
KVM: PPC: Book3S HV: XIVE: clear file mapping when device is released
KVM: PPC: Book3S HV: XIVE: do not test the EQ flag validity when
reseting
KVM: PPC: Book3S HV: XIVE: fix the enforced limit on the vCPU
identifierarch/powerpc/kvm/book3s_xive_native.c | 46 ++++++++++++++++----------- 1 file changed, 27 insertions(+), 19 deletions(-)
Signature · 4 lines
David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson
Quoted 16 lines · On Thu, 23 May 2019 16:27:15 +1000 David Gibson <[email protected]>…
Quoted 15 lines · Greg Kurz
Quoted 9 lines · On Thu, 23 May 2019 09:30:43 +1000 David Gibson <[email protected]>…
Quoted 4 lines · Cédric Le Goater
Hello,
Here are a couple of fixes for issues in the XIVE KVM device when testing the limits : RAM size and number of vCPUS.
How serious are the problems these patches fix? I'm wondering if I need to make a backport for RHEL8.1.
Patch 2/3 fixes a QEMU error when hot-unplugging a vCPU:
Quoted 8 lines
Quoted 5 lines
qemu-system-ppc64:
KVM_SET_DEVICE_ATTRfailed: Group 4 attr0x0000000000000046: Invalid argumentPatch 3/3 fixes an issue where the guest freezes at some point when doing vCPU hot-plug/unplug in a loop.
Oh.. weird. It's not clear to me how it would do that.
Cedric provided a better description in some other mail: guest with 1024 vCPUs.
Quoted 4 lines
Quoted 1 line
Both issues have a BZ at IBM. They can be mirrored to RH if needed.
That would be helpful, thanks.
Ok, I'll take care of that.
Quoted 30 lines
Quoted 27 lines
Quoted 26 lines
Quoted 25 lines
Based on 5.2-rc1.
Available on GitHub:
https://github.com/legoater/linux/commits/xive-5.2
Thanks,
C.
Cédric Le Goater (3):
KVM: PPC: Book3S HV: XIVE: clear file mapping when device is released
KVM: PPC: Book3S HV: XIVE: do not test the EQ flag validity when
reseting
KVM: PPC: Book3S HV: XIVE: fix the enforced limit on the vCPU
identifierarch/powerpc/kvm/book3s_xive_native.c | 46 ++++++++++++++++----------- 1 file changed, 27 insertions(+), 19 deletions(-)
Quoted 5 lines · David Gibson
Quoted 4 lines · Cédric Le Goater
Hello,
Here are a couple of fixes for issues in the XIVE KVM device when testing the limits : RAM size and number of vCPUS.
This summary is wrong. RAM size was fixed in QEMU.
Quoted 2 lines
How serious are the problems these patches fix? I'm wondering if I need to make a backport for RHEL8.1.
Patch 1 is a cleanup patch. It does not fix any critical issues.
Patch 2 fixes CPU hotplug. The test on the EQ flag is at the wrong place :/ This is important I think.
Patch 3 fixes the maximum number of vCPUS supported. This one is less important maybe, unless we want to run a guest with 1024 vCPUs. Which is quite slow to run on most P9 systems.
QEMU emits a warning :
warning: Number of SMP cpus requested (1024) exceeds the recommended cpus supported by KVM (120)
May be we should refuse to run QEMU when that number is above a certain threshold ?
C.
Quoted 22 lines
Quoted 21 lines
Based on 5.2-rc1.
Available on GitHub:
https://github.com/legoater/linux/commits/xive-5.2
Thanks,
C.
Cédric Le Goater (3):
KVM: PPC: Book3S HV: XIVE: clear file mapping when device is released
KVM: PPC: Book3S HV: XIVE: do not test the EQ flag validity when
reseting
KVM: PPC: Book3S HV: XIVE: fix the enforced limit on the vCPU
identifierarch/powerpc/kvm/book3s_xive_native.c | 46 ++++++++++++++++----------- 1 file changed, 27 insertions(+), 19 deletions(-)
Quoted 13 lines · Cédric Le Goater
Hello,
Here are a couple of fixes for issues in the XIVE KVM device when testing the limits : RAM size and number of vCPUS.
Based on 5.2-rc1.
Available on GitHub:
https://github.com/legoater/linux/commits/xive-5.2
Thanks,
Quoted 8 lines
C.
Cédric Le Goater (3):
KVM: PPC: Book3S HV: XIVE: clear file mapping when device is released
KVM: PPC: Book3S HV: XIVE: do not test the EQ flag validity when
reseting
KVM: PPC: Book3S HV: XIVE: fix the enforced limit on the vCPU
identifier
Thanks, series applied to my kvm-ppc-fixes branch.
Paul.