Fix doorbell emulation for v2 API on PPC
Subsystems (5)
v2 cover Fix doorbell emulation for v2 API on PPC
Doorbell emulation for KVM on PAPR guests is broken as support for DPDES was not added in initial patch series [1].
Add DPDES support and doorbell handling support for V2 API.
[1] lore.kernel.org/linuxppc-dev/[email protected]
Changes in v2:
1. Split DPDES support into its own patch
Gautam Menghani (2):
arch/powerpc/kvm: Add DPDES support in helper library for Guest state
buffer
arch/powerpc/kvm: Fix doorbell emulation for v2 API
Documentation/arch/powerpc/kvm-nested.rst | 4 +++- arch/powerpc/include/asm/guest-state-buffer.h | 3 ++- arch/powerpc/include/asm/kvm_book3s.h | 1 + arch/powerpc/kvm/book3s_hv.c | 5 +++++ arch/powerpc/kvm/book3s_hv_nestedv2.c | 7 +++++++ arch/powerpc/kvm/test-guest-state-buffer.c | 2 +- 6 files changed, 19 insertions(+), 3 deletions(-)
Signature · 1 line
2.45.1
PATCH v2 1/2 arch/powerpc/kvm: Add DPDES support in helper library for Guest state buffer
Add support for using DPDES in the library for using guest state buffers. DPDES support is needed for enabling usage of doorbells in a L2 KVM on PAPR guest.
--- Documentation/arch/powerpc/kvm-nested.rst | 4 +++- arch/powerpc/include/asm/guest-state-buffer.h | 3 ++- arch/powerpc/include/asm/kvm_book3s.h | 1 + arch/powerpc/kvm/book3s_hv_nestedv2.c | 7 +++++++ arch/powerpc/kvm/test-guest-state-buffer.c | 2 +- 5 files changed, 14 insertions(+), 3 deletions(-)
Documentation/arch/powerpc/kvm-nested.rst +3 -1
@@ -546,7 +546,9 @@ table information. +--------+-------+----+--------+----------------------------------+ | 0x1052 | 0x08 | RW | T | CTRL | +--------+-------+----+--------+----------------------------------+ -| 0x1053-| | | | Reserved | +| 0x1053 | 0x08 | RW | T | DPDES | ++--------+-------+----+--------+----------------------------------+ +| 0x1054-| | | | Reserved | | 0x1FFF | | | | | +--------+-------+----+--------+----------------------------------+ | 0x2000 | 0x04 | RW | T | CR |
arch/powerpc/include/asm/guest-state-buffer.h +2 -1
@@ -81,6 +81,7 @@#define KVMPPC_GSID_HASHKEYR 0x1050#define KVMPPC_GSID_HASHPKEYR 0x1051#define KVMPPC_GSID_CTRL 0x1052 +#define KVMPPC_GSID_DPDES 0x1053#define KVMPPC_GSID_CR 0x2000#define KVMPPC_GSID_PIDR 0x2001
@@ -110,7 +111,7 @@#define KVMPPC_GSE_META_COUNT (KVMPPC_GSE_META_END - KVMPPC_GSE_META_START + 1)#define KVMPPC_GSE_DW_REGS_START KVMPPC_GSID_GPR(0) -#define KVMPPC_GSE_DW_REGS_END KVMPPC_GSID_CTRL +#define KVMPPC_GSE_DW_REGS_END KVMPPC_GSID_DPDES#define KVMPPC_GSE_DW_REGS_COUNT \ (KVMPPC_GSE_DW_REGS_END - KVMPPC_GSE_DW_REGS_START + 1)
arch/powerpc/include/asm/kvm_book3s.h +1 -0
@@ -594,6 +594,7 @@ static inline u##size kvmppc_get_##reg(struct kvm_vcpu *vcpu) \ KVMPPC_BOOK3S_VCORE_ACCESSOR(vtb, 64, KVMPPC_GSID_VTB) +KVMPPC_BOOK3S_VCORE_ACCESSOR(dpdes, 64, KVMPPC_GSID_DPDES) KVMPPC_BOOK3S_VCORE_ACCESSOR_GET(arch_compat, 32, KVMPPC_GSID_LOGICAL_PVR) KVMPPC_BOOK3S_VCORE_ACCESSOR_GET(lpcr, 64, KVMPPC_GSID_LPCR)KVMPPC_BOOK3S_VCORE_ACCESSOR_SET (tb_offset,64 ,KVMPPC_GSID_TB_OFFSET )
arch/powerpc/kvm/book3s_hv_nestedv2.c +7 -0
@@ -311,6 +311,10 @@ static int gs_msg_ops_vcpu_fill_info(struct kvmppc_gs_buff *gsb, rc =kvmppc_gse_put_u64 (gsb, iden, vcpu->arch.vcore->vtb);break ; +case KVMPPC_GSID_DPDES : + rc =kvmppc_gse_put_u64 (gsb, iden, + vcpu->arch.vcore->dpdes); +break ;case KVMPPC_GSID_LPCR : rc =kvmppc_gse_put_u64 (gsb, iden, vcpu->arch.vcore->lpcr);
@@ -543,6 +547,9 @@ static int gs_msg_ops_vcpu_refresh_info(struct kvmppc_gs_msg *gsm,case KVMPPC_GSID_VTB : vcpu->arch.vcore->vtb =kvmppc_gse_get_u64 (gse);break ; +case KVMPPC_GSID_DPDES : + vcpu->arch.vcore->dpdes =kvmppc_gse_get_u64 (gse); +break ;case KVMPPC_GSID_LPCR : vcpu->arch.vcore->lpcr =kvmppc_gse_get_u64 (gse);break ;
arch/powerpc/kvm/test-guest-state-buffer.c +1 -1
@@ -151,7 +151,7 @@ static void test_gs_bitmap(struct kunit *test) i++; } -for (u16 iden =KVMPPC_GSID_GPR (0 ); iden <=KVMPPC_GSID_CTRL ; iden++) { +for (u16 iden =KVMPPC_GSID_GPR (0 ); iden <=KVMPPC_GSE_DW_REGS_END ; iden++) {kvmppc_gsbm_set (&gsbm, iden);kvmppc_gsbm_set (&gsbm1, iden);KUNIT_EXPECT_TRUE (test,kvmppc_gsbm_test (&gsbm, iden));
Signature · 1 line
2.45.1
Re: [PATCH v2 1/2] arch/powerpc/kvm: Add DPDES support in helper library for Guest state buffer
Quoted 4 lines · Gautam Menghani
Add support for using DPDES in the library for using guest state buffers. DPDES support is needed for enabling usage of doorbells in a L2 KVM on PAPR guest.
Quoted 97 lines
Fixes 6ccbbc33f06a ("KVM: PPC: Add helper library for Guest State Buffers")Cc [email protected] # v6.7Signed-off-by Gautam Menghani <[email protected]>--- Documentation/arch/powerpc/kvm-nested.rst | 4 +++- arch/powerpc/include/asm/guest-state-buffer.h | 3 ++- arch/powerpc/include/asm/kvm_book3s.h | 1 + arch/powerpc/kvm/book3s_hv_nestedv2.c | 7 +++++++ arch/powerpc/kvm/test-guest-state-buffer.c | 2 +- 5 files changed, 14 insertions(+), 3 deletions(-)Documentation/arch/powerpc/kvm-nested.rst +3
-1@@ -546,7 +546,9 @@ table information. +--------+-------+----+--------+----------------------------------+ | 0x1052 | 0x08 | RW | T | CTRL | +--------+-------+----+--------+----------------------------------+ -| 0x1053-| | | | Reserved | +| 0x1053 | 0x08 | RW | T | DPDES | ++--------+-------+----+--------+----------------------------------+ +| 0x1054-| | | | Reserved | | 0x1FFF | | | | | +--------+-------+----+--------+----------------------------------+ | 0x2000 | 0x04 | RW | T | CR |arch/powerpc/include/asm/guest-state-buffer.h +2
-1@@ -81,6 +81,7 @@#define KVMPPC_GSID_HASHKEYR 0x1050#define KVMPPC_GSID_HASHPKEYR 0x1051#define KVMPPC_GSID_CTRL 0x1052 +#define KVMPPC_GSID_DPDES 0x1053#define KVMPPC_GSID_CR 0x2000#define KVMPPC_GSID_PIDR 0x2001@@ -110,7 +111,7 @@#define KVMPPC_GSE_META_COUNT (KVMPPC_GSE_META_END - KVMPPC_GSE_META_START + 1)#define KVMPPC_GSE_DW_REGS_START KVMPPC_GSID_GPR(0) -#define KVMPPC_GSE_DW_REGS_END KVMPPC_GSID_CTRL +#define KVMPPC_GSE_DW_REGS_END KVMPPC_GSID_DPDES#define KVMPPC_GSE_DW_REGS_COUNT \ (KVMPPC_GSE_DW_REGS_END - KVMPPC_GSE_DW_REGS_START + 1)arch/powerpc/include/asm/kvm_book3s.h +1
-0@@ -594,6 +594,7 @@ static inline u##size kvmppc_get_##reg(struct kvm_vcpu *vcpu) \ KVMPPC_BOOK3S_VCORE_ACCESSOR(vtb, 64, KVMPPC_GSID_VTB) +KVMPPC_BOOK3S_VCORE_ACCESSOR(dpdes, 64, KVMPPC_GSID_DPDES) KVMPPC_BOOK3S_VCORE_ACCESSOR_GET(arch_compat, 32, KVMPPC_GSID_LOGICAL_PVR) KVMPPC_BOOK3S_VCORE_ACCESSOR_GET(lpcr, 64, KVMPPC_GSID_LPCR)KVMPPC_BOOK3S_VCORE_ACCESSOR_SET (tb_offset,64 ,KVMPPC_GSID_TB_OFFSET )arch/powerpc/kvm/book3s_hv_nestedv2.c +7
-0@@ -311,6 +311,10 @@ static int gs_msg_ops_vcpu_fill_info(struct kvmppc_gs_buff *gsb, rc =kvmppc_gse_put_u64 (gsb, iden, vcpu->arch.vcore->vtb);break ; +case KVMPPC_GSID_DPDES : + rc =kvmppc_gse_put_u64 (gsb, iden, + vcpu->arch.vcore->dpdes); +break ;case KVMPPC_GSID_LPCR : rc =kvmppc_gse_put_u64 (gsb, iden, vcpu->arch.vcore->lpcr);@@ -543,6 +547,9 @@ static int gs_msg_ops_vcpu_refresh_info(struct kvmppc_gs_msg *gsm,case KVMPPC_GSID_VTB : vcpu->arch.vcore->vtb =kvmppc_gse_get_u64 (gse);break ; +case KVMPPC_GSID_DPDES : + vcpu->arch.vcore->dpdes =kvmppc_gse_get_u64 (gse); +break ;case KVMPPC_GSID_LPCR : vcpu->arch.vcore->lpcr =kvmppc_gse_get_u64 (gse);break ;arch/powerpc/kvm/test-guest-state-buffer.c +1
-1@@ -151,7 +151,7 @@ static void test_gs_bitmap(struct kunit *test) i++; } -for (u16 iden =KVMPPC_GSID_GPR (0 ); iden <=KVMPPC_GSID_CTRL ; iden++) { +for (u16 iden =KVMPPC_GSID_GPR (0 ); iden <=KVMPPC_GSE_DW_REGS_END ; iden++) {kvmppc_gsbm_set (&gsbm, iden);kvmppc_gsbm_set (&gsbm1, iden);KUNIT_EXPECT_TRUE (test,kvmppc_gsbm_test (&gsbm, iden));
PATCH v2 2/2 arch/powerpc/kvm: Fix doorbell emulation for v2 API
In reply to Gautam MenghaniDoorbell emulation is broken for KVM on PAPR guests as support for DPDES was not added in the initial patch series. Due to this, a KVM on PAPR guest with SMT > 1 cannot be booted with the XICS interrupt controller as doorbells are setup in the initial probe path when using XICS (pSeries_smp_probe()).
Command to replicate the above bug:
qemu-system-ppc64 \ -drive file=rhel.qcow2,format=qcow2 \ -m 20G \ -smp 8,cores=1,threads=8 \ -cpu host \ -nographic \ -machine pseries,ic-mode=xics -accel kvm
Add doorbell state handling support in the host
KVM code to fix doorbell emulation.
--- arch/powerpc/kvm/book3s_hv.c | 5 +++++ 1 file changed, 5 insertions(+)
arch/powerpc/kvm/book3s_hv.c +5 -0
@@ -4116,6 +4116,11 @@ static int kvmhv_vcpu_entry_nestedv2(struct kvm_vcpu *vcpu, u64 time_limit,int trap;long rc; +if (vcpu->arch.doorbell_request) { + vcpu->arch.doorbell_request =0 ; +kvmppc_set_dpdes (vcpu,1 ); + } + io = &vcpu->arch.nestedv2_io; msr =mfmsr ();
Signature · 1 line
2.45.1
Re: [PATCH v2 2/2] arch/powerpc/kvm: Fix doorbell emulation for v2 API
Quoted 16 lines · Gautam Menghani
Doorbell emulation is broken for KVM on PAPR guests as support for DPDES was not added in the initial patch series. Due to this, a KVM on PAPR guest with SMT > 1 cannot be booted with the XICS interrupt controller as doorbells are setup in the initial probe path when using XICS (
pSeries_smp_probe()).Command to replicate the above bug:
qemu-system-ppc64 \ -drive file=rhel.qcow2,format=qcow2 \ -m 20G \ -smp 8,cores=1,threads=8 \ -cpu host \ -nographic \ -machine pseries,ic-mode=xics -accel kvm
Quoted 2 lines
Add doorbell state handling support in the host
KVM code to fix doorbell emulation.
Quoted 24 lines
Fixes 19d31c5f1157 ("KVM: PPC: Add support for nestedv2 guests")Cc [email protected] # v6.7Signed-off-by Gautam Menghani <[email protected]>--- arch/powerpc/kvm/book3s_hv.c | 5 +++++ 1 file changed, 5 insertions(+)arch/powerpc/kvm/book3s_hv.c +5
-0@@ -4116,6 +4116,11 @@ static int kvmhv_vcpu_entry_nestedv2(struct kvm_vcpu *vcpu, u64 time_limit,int trap;long rc; +if (vcpu->arch.doorbell_request) { + vcpu->arch.doorbell_request =0 ; +kvmppc_set_dpdes (vcpu,1 ); + } + io = &vcpu->arch.nestedv2_io; msr =mfmsr ();
Nicholas Piggin
reviewerQuoted 3 lines · Gautam Menghani
Doorbell emulation for KVM on PAPR guests is broken as support for DPDES was not added in initial patch series [1].
Add DPDES support and doorbell handling support for V2 API.
Looks good, thanks. So fix for v1 doorbells is coming?
Thanks,
Nick
Quoted 18 lines
[1] lore.kernel.org/linuxppc-dev/[email protected]
Changes in v2:
1. Split DPDES support into its own patchGautam Menghani (2):
arch/powerpc/kvm: Add DPDES support in helper library for Guest state
buffer
arch/powerpc/kvm: Fix doorbell emulation for v2 APIDocumentation/arch/powerpc/kvm-nested.rst | 4 +++- arch/powerpc/include/asm/guest-state-buffer.h | 3 ++- arch/powerpc/include/asm/kvm_book3s.h | 1 + arch/powerpc/kvm/book3s_hv.c | 5 +++++ arch/powerpc/kvm/book3s_hv_nestedv2.c | 7 +++++++ arch/powerpc/kvm/test-guest-state-buffer.c | 2 +- 6 files changed, 19 insertions(+), 3 deletions(-)
Quoted 6 lines · Nicholas Piggin
Quoted 3 lines · Gautam Menghani
Doorbell emulation for KVM on PAPR guests is broken as support for DPDES was not added in initial patch series [1].
Add DPDES support and doorbell handling support for V2 API.Looks good, thanks. So fix for v1 doorbells is coming?
Yes I've root caused the doorbell breakage in V1 API to commit 7c3ded5735141ff4d049747c9f76672a8b737c49. I'll send out a fix soon.
Thanks,
Gautam
Quoted 14 lines · Gautam Menghani
Doorbell emulation for KVM on PAPR guests is broken as support for DPDES was not added in initial patch series [1].
Add DPDES support and doorbell handling support for V2 API.[1] lore.kernel.org/linuxppc-dev/[email protected]
Changes in v2:
1. Split DPDES support into its own patchGautam Menghani (2):
arch/powerpc/kvm: Add DPDES support in helper library for Guest state
buffer
arch/powerpc/kvm: Fix doorbell emulation for v2 API
Quoted 11 lines
Documentation/arch/powerpc/kvm-nested.rst | 4 +++- arch/powerpc/include/asm/guest-state-buffer.h | 3 ++- arch/powerpc/include/asm/kvm_book3s.h | 1 + arch/powerpc/kvm/book3s_hv.c | 5 +++++ arch/powerpc/kvm/book3s_hv_nestedv2.c | 7 +++++++ arch/powerpc/kvm/test-guest-state-buffer.c | 2 +- 6 files changed, 19 insertions(+), 3 deletions(-)Signature · 1 line
2.45.1
Hi Michael,
This patch series is to be backported for all kernels >= 6.7. So the tag should be
Cc: [email protected] # v6.7+
and not
Cc: [email protected] # v6.7
Should I send a new version of this series or can you please make this change when pulling in your tree?
Thanks,
Gautam
Michael Ellerman
reviewerQuoted 34 lines · Gautam Menghani
Quoted 14 lines · Gautam Menghani
Doorbell emulation for KVM on PAPR guests is broken as support for DPDES was not added in initial patch series [1].
Add DPDES support and doorbell handling support for V2 API.[1] lore.kernel.org/linuxppc-dev/[email protected]
Changes in v2:
1. Split DPDES support into its own patchGautam Menghani (2):
arch/powerpc/kvm: Add DPDES support in helper library for Guest state
buffer
arch/powerpc/kvm: Fix doorbell emulation for v2 APIQuoted 11 lines
Documentation/arch/powerpc/kvm-nested.rst | 4 +++- arch/powerpc/include/asm/guest-state-buffer.h | 3 ++- arch/powerpc/include/asm/kvm_book3s.h | 1 + arch/powerpc/kvm/book3s_hv.c | 5 +++++ arch/powerpc/kvm/book3s_hv_nestedv2.c | 7 +++++++ arch/powerpc/kvm/test-guest-state-buffer.c | 2 +- 6 files changed, 19 insertions(+), 3 deletions(-)Signature · 1 line
2.45.1Hi Michael,
This patch series is to be backported for all kernels >= 6.7. So the tag should be
Cc: [email protected] # v6.7+
Quoted 5 lines
and not
Cc: [email protected] # v6.7Should I send a new version of this series or can you please make this change when pulling in your tree?
I can make the change when applying.
cheers
Michael Ellerman
AppliedQuoted 10 lines · Gautam Menghani
Doorbell emulation for KVM on PAPR guests is broken as support for DPDES was not added in initial patch series [1].
Add DPDES support and doorbell handling support for V2 API.[1] lore.kernel.org/linuxppc-dev/[email protected]
Changes in v2:
1. Split DPDES support into its own patch[...]
Applied to powerpc/topic/ppc-kvm.
[1/2] arch/powerpc/kvm: Add DPDES support in helper library for Guest state buffer
https://git.kernel.org/powerpc/c/55dfb8bed6fe8bda390cc71cca878d11a9407099 [2/2] arch/powerpc/kvm: Fix doorbell emulation for v2 API https://git.kernel.org/powerpc/c/54ec2bd9e0173b75daf84675d07c56584f96564b
cheers