ppc: add native hash and radix support for POWER9
cover ppc: add native hash and radix support for POWER9
Hello,
Here is another series of fixes and extensions from Ben providing support for POWER9 native hash MMU and POWER9 native radix MMU. These prepare ground for the support of QEMU POWER9 PowerNV machines.
Quoted 1 line
From there, I hand over the patchset to Suraj for any follow-ups.
Thanks,
C.
Benjamin Herrenschmidt (12):
target/ppc/spapr: Set LPCR:HR when using Radix mode target/ppci/mmu: Use LPCR:HR to chose radix vs. hash translation target/ppc: Re-enable RMLS on POWER9 for virtual hypervisors target/ppc: Fix #include guard in mmu-book3s-v3.h target/ppc: Cleanup 64-bit MMU includes
target/ppc: Fix ordering of hash MMU accesses
target/ppc: Add basic support for "new format" HPTE as found on POWER9 target/ppc: Fix synchronization of mttcg with broadcast TLB flushes target/ppc: Flush the TLB locally when the LPIDR is written target/ppc: Rename PATB/PATBE -> PATE
target/ppc: Support for POWER9 native hash
target/ppc: Basic POWER9 bare-metal radix MMU support
include/hw/ppc/spapr.h | 1 + target/ppc/cpu.h | 7 ++- target/ppc/helper.h | 1 + target/ppc/helper_regs.h | 27 ++++------ target/ppc/mmu-book3s-v3.h | 89 +++++++++++++++++++++++++++--- target/ppc/mmu-hash64.h | 22 ++------ target/ppc/mmu-radix64.h | 4 +- hw/ppc/spapr.c | 86 ++++++++++++++++++++++++----- hw/ppc/spapr_hcall.c | 69 +++++++----------------- hw/ppc/spapr_rtas.c | 6 +-- target/ppc/misc_helper.c | 15 ++++++ target/ppc/mmu-book3s-v3.c | 31 +++++++++-- target/ppc/mmu-hash32.c | 6 +++ target/ppc/mmu-hash64.c | 28 ++++++++-- target/ppc/mmu-radix64.c | 96 ++++++++++++++++++++++++++------- target/ppc/mmu_helper.c | 9 +--- target/ppc/translate_init.inc.c | 7 ++- 17 files changed, 361 insertions(+), 143 deletions(-)
Signature · 1 line
2.20.1
PATCH 1/12 target/ppc/spapr: Set LPCR:HR when using Radix mode
From: Benjamin Herrenschmidt <[email protected]>
The HW relies on LPCR:HR along with the PATE to determine whether to use Radix or Hash mode. In fact it uses LPCR:HR more commonly than the PATE.
For us, it's also more efficient to do so, especially since unlike the HW we do not maintain a cache of the current PATE and HV PATE in a generic place.
Prepare the grounds for that by ensuring that LPCR:HR is set properly on SPAPR machines.
Another option would have been to use a callback to get the PATE but this gets messy when implementing bare metal support, it's much simpler (and faster) to use LPCR.
Since existing migration streams may not have it, fix it up in spapr_post_load() as well based on the pseudo-PATE entry that we keep.
--- include/hw/ppc/spapr.h | 1 + target/ppc/cpu.h | 1 + hw/ppc/spapr.c | 41 +++++++++++++++++++++++++++++++++++- hw/ppc/spapr_hcall.c | 46 +++++++---------------------------------- hw/ppc/spapr_rtas.c | 6 +++--- target/ppc/mmu-hash64.c | 2 +- 6 files changed, 54 insertions(+), 43 deletions(-)
include/hw/ppc/spapr.h +1 -0
@@ -840,4 +840,5 @@ void spapr_check_pagesize(sPAPRMachineState *spapr, hwaddr pagesize,#define SPAPR_OV5_XIVE_EXPLOIT 0x40#define SPAPR_OV5_XIVE_BOTH 0x80/* Only to advertise on the platform */ +void spapr_set_all_lpcrs (target_ulong value,target_ulong mask);#endif /* HW_SPAPR_H */
target/ppc/cpu.h +1 -0
@@ -385,6 +385,7 @@ struct ppc_slb_t {#define LPCR_AIL (3ull << LPCR_AIL_SHIFT)#define LPCR_UPRT PPC_BIT(41)/* Use Process Table */ #define LPCR_EVIRT PPC_BIT(42)/* Enhanced Virtualisation */ +#define LPCR_HR PPC_BIT(43)/* Host Radix */ #define LPCR_ONL PPC_BIT(45)#define LPCR_LD PPC_BIT(46)/* Large Decrementer */ #define LPCR_P7_PECE0 PPC_BIT(49)
hw/ppc/spapr.c +40 -1
65 lines not on this page · raw message
hw/ppc/spapr_hcall.c +8 -38
74 lines not on this page · raw message
hw/ppc/spapr_rtas.c +3 -3
@@ -172,10 +172,10 @@ static void rtas_start_cpu(PowerPCCPU *callcpu, sPAPRMachineState *spapr,* New cpus are expected to start in the same radix/hash mode * as the existing CPUs */ -if (ppc64_radix_guest (callcpu)) { - lpcr |=LPCR_UPRT |LPCR_GTSE ; +if (ppc64_v3_radix (callcpu)) { + lpcr |=LPCR_UPRT |LPCR_GTSE |LPCR_HR ; }else { - lpcr &= ~(LPCR_UPRT |LPCR_GTSE ); + lpcr &= ~(LPCR_UPRT |LPCR_GTSE |LPCR_HR ); } }ppc_store_lpcr (newcpu, lpcr);
target/ppc/mmu-hash64.c +1 -1
@@ -1084,7 +1084,7 @@ void ppc_store_lpcr(PowerPCCPU *cpu, target_ulong val)case POWERPC_MMU_3_00 :/* P9 */ lpcr = val & (LPCR_VPM1 |LPCR_ISL |LPCR_KBV |LPCR_DPFD | (LPCR_PECE_U_MASK &LPCR_HVEE ) |LPCR_ILE |LPCR_AIL | -LPCR_UPRT |LPCR_EVIRT |LPCR_ONL | +LPCR_UPRT |LPCR_EVIRT |LPCR_ONL |LPCR_HR | (LPCR_PECE_L_MASK & (LPCR_PDEE |LPCR_HDEE |LPCR_EEE |LPCR_DEE |LPCR_OEE )) |LPCR_MER |LPCR_GTSE |LPCR_TC |LPCR_HEIC |LPCR_LPES0 |LPCR_HVICE |LPCR_HDICE );
Signature · 1 line
2.20.1
Re: [Qemu-devel] [PATCH 01/12] target/ppc/spapr: Set LPCR:HR when using Radix mode
Quoted 97 lines · Cédric Le Goater
From: Benjamin Herrenschmidt <[email protected]>
The HW relies on LPCR:HR along with the PATE to determine whether to use Radix or Hash mode. In fact it uses LPCR:HR more commonly than the PATE.
For us, it's also more efficient to do so, especially since unlike the HW we do not maintain a cache of the current PATE and HV PATE in a generic place.
Prepare the grounds for that by ensuring that LPCR:HR is set properly on SPAPR machines.
Another option would have been to use a callback to get the PATE but this gets messy when implementing bare metal support, it's much simpler (and faster) to use LPCR.
Since existing migration streams may not have it, fix it up in
spapr_post_load()as well based on the pseudo-PATE entry that we keep.Signed-off-by Benjamin Herrenschmidt <[email protected]>Signed-off-by Cédric Le Goater <[email protected]>--- include/hw/ppc/spapr.h | 1 + target/ppc/cpu.h | 1 + hw/ppc/spapr.c | 41 +++++++++++++++++++++++++++++++++++- hw/ppc/spapr_hcall.c | 46 +++++++---------------------------------- hw/ppc/spapr_rtas.c | 6 +++--- target/ppc/mmu-hash64.c | 2 +- 6 files changed, 54 insertions(+), 43 deletions(-)include/hw/ppc/spapr.h +1
-0@@ -840,4 +840,5 @@ void spapr_check_pagesize(sPAPRMachineState *spapr, hwaddr pagesize,#define SPAPR_OV5_XIVE_EXPLOIT 0x40#define SPAPR_OV5_XIVE_BOTH 0x80/* Only to advertise on the platform */ +void spapr_set_all_lpcrs (target_ulong value,target_ulong mask);#endif /* HW_SPAPR_H */ target/ppc/cpu.h +1
-0@@ -385,6 +385,7 @@ struct ppc_slb_t {#define LPCR_AIL (3ull << LPCR_AIL_SHIFT)#define LPCR_UPRT PPC_BIT(41)/* Use Process Table */ #define LPCR_EVIRT PPC_BIT(42)/* Enhanced Virtualisation */ +#define LPCR_HR PPC_BIT(43)/* Host Radix */ #define LPCR_ONL PPC_BIT(45)#define LPCR_LD PPC_BIT(46)/* Large Decrementer */ #define LPCR_P7_PECE0 PPC_BIT(49)hw/ppc/spapr.c +31
-0@@ -1372,6 +1372,37 @@ static void emulate_spapr_hypercall(PPCVirtualHypervisor *vhyp, } } +struct LPCRSyncState { +target_ulong value; +target_ulong mask; +}; + +static void do_lpcr_sync (CPUState *cs,run_on_cpu_data arg) +{ +struct LPCRSyncState *s = arg.host_ptr; +PowerPCCPU *cpu =POWERPC_CPU (cs); +CPUPPCState *env = &cpu->env; +target_ulong lpcr; + +cpu_synchronize_state (cs); + lpcr = env->spr[SPR_LPCR ]; + lpcr &= ~s->mask; + lpcr |= s->value; +ppc_store_lpcr (cpu, lpcr); +} + +void spapr_set_all_lpcrs (target_ulong value,target_ulong mask) +{ +CPUState *cs; +struct LPCRSyncState s = { + .value = value, + .mask = mask + }; +CPU_FOREACH (cs) { +run_on_cpu (cs, do_lpcr_sync,RUN_ON_CPU_HOST_PTR (&s)); + } +} +static uint64_t spapr_get_patbe (PPCVirtualHypervisor *vhyp) {sPAPRMachineState *spapr =SPAPR_MACHINE (vhyp);
hw/ppc/spapr.c void spapr_reallocate_hpt(sPAPRMachineState *spapr, int shift, · line 1582
Quoted 5 lines
hw/ppc/spapr.c +1
-1@@ -1548,7 +1579,7 @@ void spapr_reallocate_hpt(sPAPRMachineState *spapr, int shift, } }/* We're setting up a hash table, so that means we're not radix */ - spapr->patb_entry =0 ; +spapr_set_all_lpcrs (0 ,LPCR_HR |LPCR_UPRT );
I think we should still set the patb_entry as well as the LPCR here. It might not matter, but it will be less confusing for debugging at the least. Apart from that nit,
Quoted 138 lines
hw/ppc/spapr.c +8
-0}void spapr_setup_hpt_and_vrma (sPAPRMachineState *spapr)@@ -1606,6 +1637,7 @@ static void spapr_machine_reset(void)* without a HPT because KVM will start them in radix mode. * Set the GR bit in PATB so that we know there is no HPT. */ spapr->patb_entry =PATBE1_GR ; +spapr_set_all_lpcrs (LPCR_HR |LPCR_UPRT ,LPCR_HR |LPCR_UPRT ); }else {spapr_setup_hpt_and_vrma (spapr); }@@ -1764,6 +1796,13 @@ static int spapr_post_load(void *opaque, int version_id)bool radix = !!(spapr->patb_entry &PATBE1_GR );bool gtse = !!(cpu->env.spr[SPR_LPCR ] &LPCR_GTSE ); +/* +* Update LPCR:HR and UPRT as they may not be set properly in +* the stream +*/ +spapr_set_all_lpcrs (radix ? (LPCR_HR |LPCR_UPRT ) :0 , +LPCR_HR |LPCR_UPRT ); + err =kvmppc_configure_v3_mmu (cpu, radix, gtse, spapr->patb_entry);if (err) {error_report ("Process table config unsupported by the host" );hw/ppc/spapr_hcall.c +8
-38@@ -17,37 +17,6 @@#include "mmu-book3s-v3.h" #include "hw/mem/memory-device.h" -struct LPCRSyncState { -target_ulong value; -target_ulong mask; -}; - -static void do_lpcr_sync (CPUState *cs,run_on_cpu_data arg) -{ -struct LPCRSyncState *s = arg.host_ptr; -PowerPCCPU *cpu =POWERPC_CPU (cs); -CPUPPCState *env = &cpu->env; -target_ulong lpcr; - -cpu_synchronize_state (cs); - lpcr = env->spr[SPR_LPCR ]; - lpcr &= ~s->mask; - lpcr |= s->value; -ppc_store_lpcr (cpu, lpcr); -} - -static void set_all_lpcrs (target_ulong value,target_ulong mask) -{ -CPUState *cs; -struct LPCRSyncState s = { - .value = value, - .mask = mask - }; -CPU_FOREACH (cs) { -run_on_cpu (cs, do_lpcr_sync,RUN_ON_CPU_HOST_PTR (&s)); - } -} -static bool has_spr (PowerPCCPU *cpu,int spr) {/* We can test whether the SPR is defined by checking for a valid name */ @@ -1255,12 +1224,12 @@ static target_ulong h_set_mode_resource_le(PowerPCCPU *cpu,switch (mflags) {case H_SET_MODE_ENDIAN_BIG : -set_all_lpcrs (0 ,LPCR_ILE ); +spapr_set_all_lpcrs (0 ,LPCR_ILE );spapr_pci_switch_vga (true);return H_SUCCESS ;case H_SET_MODE_ENDIAN_LITTLE : -set_all_lpcrs (LPCR_ILE ,LPCR_ILE ); +spapr_set_all_lpcrs (LPCR_ILE ,LPCR_ILE );spapr_pci_switch_vga (false);return H_SUCCESS ; }@@ -1289,7 +1258,7 @@ static target_ulong h_set_mode_resource_addr_trans_mode(PowerPCCPU *cpu,return H_UNSUPPORTED_FLAG ; } -set_all_lpcrs (mflags <<LPCR_AIL_SHIFT ,LPCR_AIL ); +spapr_set_all_lpcrs (mflags <<LPCR_AIL_SHIFT ,LPCR_AIL );return H_SUCCESS ; }@@ -1422,10 +1391,11 @@ static target_ulong h_register_process_table(PowerPCCPU *cpu, spapr->patb_entry = cproc;/* Save new process table */ -/* Update the UPRT and GTSE bits in the LPCR for all cpus */ -set_all_lpcrs (((flags & (FLAG_RADIX |FLAG_HASH_PROC_TBL )) ?LPCR_UPRT :0 ) | - ((flags &FLAG_GTSE ) ?LPCR_GTSE :0 ), -LPCR_UPRT |LPCR_GTSE ); +/* Update the UPRT, HR and GTSE bits in the LPCR for all cpus */ +spapr_set_all_lpcrs (((flags & (FLAG_RADIX |FLAG_HASH_PROC_TBL )) ? + (LPCR_UPRT |LPCR_HR ) :0 ) | + ((flags &FLAG_GTSE ) ?LPCR_GTSE :0 ), +LPCR_UPRT |LPCR_HR |LPCR_GTSE );if (kvm_enabled ()) { return kvmppc_configure_v3_mmu(cpu, flags & FLAG_RADIX,hw/ppc/spapr_rtas.c +3
-3@@ -172,10 +172,10 @@ static void rtas_start_cpu(PowerPCCPU *callcpu, sPAPRMachineState *spapr,* New cpus are expected to start in the same radix/hash mode * as the existing CPUs */ -if (ppc64_radix_guest (callcpu)) { - lpcr |=LPCR_UPRT |LPCR_GTSE ; +if (ppc64_v3_radix (callcpu)) { + lpcr |=LPCR_UPRT |LPCR_GTSE |LPCR_HR ; }else { - lpcr &= ~(LPCR_UPRT |LPCR_GTSE ); + lpcr &= ~(LPCR_UPRT |LPCR_GTSE |LPCR_HR ); } }ppc_store_lpcr (newcpu, lpcr);target/ppc/mmu-hash64.c +1
-1@@ -1084,7 +1084,7 @@ void ppc_store_lpcr(PowerPCCPU *cpu, target_ulong val)case POWERPC_MMU_3_00 :/* P9 */ lpcr = val & (LPCR_VPM1 |LPCR_ISL |LPCR_KBV |LPCR_DPFD | (LPCR_PECE_U_MASK &LPCR_HVEE ) |LPCR_ILE |LPCR_AIL | -LPCR_UPRT |LPCR_EVIRT |LPCR_ONL | +LPCR_UPRT |LPCR_EVIRT |LPCR_ONL |LPCR_HR | (LPCR_PECE_L_MASK & (LPCR_PDEE |LPCR_HDEE |LPCR_EEE |LPCR_DEE |LPCR_OEE )) |LPCR_MER |LPCR_GTSE |LPCR_TC |LPCR_HEIC |LPCR_LPES0 |LPCR_HVICE |LPCR_HDICE );
Signature · 4 lines
David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson
PATCH 2/12 target/ppci/mmu: Use LPCR:HR to chose radix vs. hash translation
In reply to Cédric Le GoaterFrom: Benjamin Herrenschmidt <[email protected]>
Now that LPCR:HR is set properly for SPAPR, use it for deciding the translation type, which also works for bare metal
--- target/ppc/mmu-book3s-v3.h | 14 +++++++++----- target/ppc/mmu-book3s-v3.c | 11 ++++++++++- target/ppc/mmu_helper.c | 9 ++------- 3 files changed, 21 insertions(+), 13 deletions(-)
target/ppc/mmu-book3s-v3.h +9 -5
@@ -43,14 +43,18 @@ static inline bool ppc64_use_proc_tbl(PowerPCCPU *cpu)return !!(cpu->env.spr[SPR_LPCR ] &LPCR_UPRT ); } -static inline bool ppc64_radix_guest (PowerPCCPU *cpu) +/* +* The LPCR:HR bit is a shortcut that avoids having to +* dig out the partition table in the fast path. This is +* also how the HW uses it. +*/ +static inline bool ppc64_v3_radix (PowerPCCPU *cpu) { -PPCVirtualHypervisorClass *vhc = -PPC_VIRTUAL_HYPERVISOR_GET_CLASS (cpu->vhyp); - -return !!(vhc->get_patbe (cpu->vhyp) &PATBE1_GR ); +return !!(cpu->env.spr[SPR_LPCR ] &LPCR_HR ); } +hwaddr ppc64_v3_get_phys_page_debug (PowerPCCPU *cpu,vaddr eaddr); +int ppc64_v3_handle_mmu_fault (PowerPCCPU *cpu,vaddr eaddr,int rwx,int mmu_idx);
target/ppc/mmu-book3s-v3.c +10 -1
@@ -26,9 +26,18 @@int ppc64_v3_handle_mmu_fault (PowerPCCPU *cpu,vaddr eaddr,int rwx,int mmu_idx) { -if (ppc64_radix_guest (cpu)) {/* Guest uses radix */ +if (ppc64_v3_radix (cpu)) {/* Guest uses radix */ return ppc_radix64_handle_mmu_fault (cpu, eaddr, rwx, mmu_idx); }else {/* Guest uses hash */ return ppc_hash64_handle_mmu_fault (cpu, eaddr, rwx, mmu_idx); } } + +hwaddr ppc64_v3_get_phys_page_debug (PowerPCCPU *cpu,vaddr eaddr) +{ +if (ppc64_v3_radix (cpu)) { +return ppc_radix64_get_phys_page_debug (cpu, eaddr); + }else { +return ppc_hash64_get_phys_page_debug (cpu, eaddr); + } +}
target/ppc/mmu_helper.c +2 -7
@@ -1342,7 +1342,7 @@ void dump_mmu(FILE *f, fprintf_function cpu_fprintf, CPUPPCState *env)dump_slb (f, cpu_fprintf,ppc_env_get_cpu (env));break ;case POWERPC_MMU_3_00 : -if (ppc64_radix_guest (ppc_env_get_cpu (env))) { +if (ppc64_v3_radix (ppc_env_get_cpu (env))) {/* TODO - Unsupported */ }else {dump_slb (f, cpu_fprintf,ppc_env_get_cpu (env));
@@ -1497,12 +1497,7 @@ hwaddr ppc_cpu_get_phys_page_debug(CPUState *cs, vaddr addr)case POWERPC_MMU_2_07 :return ppc_hash64_get_phys_page_debug (cpu, addr);case POWERPC_MMU_3_00 : -if (ppc64_radix_guest (ppc_env_get_cpu (env))) { -return ppc_radix64_get_phys_page_debug (cpu, addr); - }else { -return ppc_hash64_get_phys_page_debug (cpu, addr); - } -break ; +return ppc64_v3_get_phys_page_debug (cpu, addr);#endif case POWERPC_MMU_32B :
Signature · 1 line
2.20.1
Re: [Qemu-devel] [PATCH 02/12] target/ppci/mmu: Use LPCR:HR to chose radix vs. hash translation
Quoted 7 lines · Cédric Le Goater
From: Benjamin Herrenschmidt <[email protected]>
Now that LPCR:HR is set properly for SPAPR, use it for deciding the translation type, which also works for bare metal
Signed-off-by Benjamin Herrenschmidt <[email protected]>Signed-off-by Cédric Le Goater <[email protected]>
Quoted 85 lines
--- target/ppc/mmu-book3s-v3.h | 14 +++++++++----- target/ppc/mmu-book3s-v3.c | 11 ++++++++++- target/ppc/mmu_helper.c | 9 ++------- 3 files changed, 21 insertions(+), 13 deletions(-)target/ppc/mmu-book3s-v3.h +9
-5@@ -43,14 +43,18 @@ static inline bool ppc64_use_proc_tbl(PowerPCCPU *cpu)return !!(cpu->env.spr[SPR_LPCR ] &LPCR_UPRT ); } -static inline bool ppc64_radix_guest (PowerPCCPU *cpu) +/* +* The LPCR:HR bit is a shortcut that avoids having to +* dig out the partition table in the fast path. This is +* also how the HW uses it. +*/ +static inline bool ppc64_v3_radix (PowerPCCPU *cpu) { -PPCVirtualHypervisorClass *vhc = -PPC_VIRTUAL_HYPERVISOR_GET_CLASS (cpu->vhyp); - -return !!(vhc->get_patbe (cpu->vhyp) &PATBE1_GR ); +return !!(cpu->env.spr[SPR_LPCR ] &LPCR_HR ); } +hwaddr ppc64_v3_get_phys_page_debug (PowerPCCPU *cpu,vaddr eaddr); +int ppc64_v3_handle_mmu_fault (PowerPCCPU *cpu,vaddr eaddr,int rwx,int mmu_idx);target/ppc/mmu-book3s-v3.c +10
-1@@ -26,9 +26,18 @@int ppc64_v3_handle_mmu_fault (PowerPCCPU *cpu,vaddr eaddr,int rwx,int mmu_idx) { -if (ppc64_radix_guest (cpu)) {/* Guest uses radix */ +if (ppc64_v3_radix (cpu)) {/* Guest uses radix */ return ppc_radix64_handle_mmu_fault (cpu, eaddr, rwx, mmu_idx); }else {/* Guest uses hash */ return ppc_hash64_handle_mmu_fault (cpu, eaddr, rwx, mmu_idx); } } + +hwaddr ppc64_v3_get_phys_page_debug (PowerPCCPU *cpu,vaddr eaddr) +{ +if (ppc64_v3_radix (cpu)) { +return ppc_radix64_get_phys_page_debug (cpu, eaddr); + }else { +return ppc_hash64_get_phys_page_debug (cpu, eaddr); + } +}target/ppc/mmu_helper.c +2
-7@@ -1342,7 +1342,7 @@ void dump_mmu(FILE *f, fprintf_function cpu_fprintf, CPUPPCState *env)dump_slb (f, cpu_fprintf,ppc_env_get_cpu (env));break ;case POWERPC_MMU_3_00 : -if (ppc64_radix_guest (ppc_env_get_cpu (env))) { +if (ppc64_v3_radix (ppc_env_get_cpu (env))) {/* TODO - Unsupported */ }else {dump_slb (f, cpu_fprintf,ppc_env_get_cpu (env));@@ -1497,12 +1497,7 @@ hwaddr ppc_cpu_get_phys_page_debug(CPUState *cs, vaddr addr)case POWERPC_MMU_2_07 :return ppc_hash64_get_phys_page_debug (cpu, addr);case POWERPC_MMU_3_00 : -if (ppc64_radix_guest (ppc_env_get_cpu (env))) { -return ppc_radix64_get_phys_page_debug (cpu, addr); - }else { -return ppc_hash64_get_phys_page_debug (cpu, addr); - } -break ; +return ppc64_v3_get_phys_page_debug (cpu, addr);#endif case POWERPC_MMU_32B :
Signature · 4 lines
David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson
Re: [Qemu-devel] [PATCH 02/12] target/ppci/mmu: Use LPCR:HR to chose radix vs. hash translation
Quoted 5 lines · David Gibson
Quoted 4 lines · Cédric Le Goater
From: Benjamin Herrenschmidt <[email protected]>
Now that LPCR:HR is set properly for SPAPR, use it for deciding the translation type, which also works for bare metal
"ppci" -> "ppc" in patch title.
Quoted 92 lines
Quoted 3 lines
Signed-off-by Benjamin Herrenschmidt <[email protected]>Signed-off-by Cédric Le Goater <[email protected]>Reviewed-by David Gibson <[email protected]>Quoted 85 lines
--- target/ppc/mmu-book3s-v3.h | 14 +++++++++----- target/ppc/mmu-book3s-v3.c | 11 ++++++++++- target/ppc/mmu_helper.c | 9 ++------- 3 files changed, 21 insertions(+), 13 deletions(-)target/ppc/mmu-book3s-v3.h +9
-5@@ -43,14 +43,18 @@ static inline bool ppc64_use_proc_tbl(PowerPCCPU *cpu)return !!(cpu->env.spr[SPR_LPCR ] &LPCR_UPRT ); } -static inline bool ppc64_radix_guest (PowerPCCPU *cpu) +/* +* The LPCR:HR bit is a shortcut that avoids having to +* dig out the partition table in the fast path. This is +* also how the HW uses it. +*/ +static inline bool ppc64_v3_radix (PowerPCCPU *cpu) { -PPCVirtualHypervisorClass *vhc = -PPC_VIRTUAL_HYPERVISOR_GET_CLASS (cpu->vhyp); - -return !!(vhc->get_patbe (cpu->vhyp) &PATBE1_GR ); +return !!(cpu->env.spr[SPR_LPCR ] &LPCR_HR ); } +hwaddr ppc64_v3_get_phys_page_debug (PowerPCCPU *cpu,vaddr eaddr); +int ppc64_v3_handle_mmu_fault (PowerPCCPU *cpu,vaddr eaddr,int rwx,int mmu_idx);target/ppc/mmu-book3s-v3.c +10
-1@@ -26,9 +26,18 @@int ppc64_v3_handle_mmu_fault (PowerPCCPU *cpu,vaddr eaddr,int rwx,int mmu_idx) { -if (ppc64_radix_guest (cpu)) {/* Guest uses radix */ +if (ppc64_v3_radix (cpu)) {/* Guest uses radix */ return ppc_radix64_handle_mmu_fault (cpu, eaddr, rwx, mmu_idx); }else {/* Guest uses hash */ return ppc_hash64_handle_mmu_fault (cpu, eaddr, rwx, mmu_idx); } } + +hwaddr ppc64_v3_get_phys_page_debug (PowerPCCPU *cpu,vaddr eaddr) +{ +if (ppc64_v3_radix (cpu)) { +return ppc_radix64_get_phys_page_debug (cpu, eaddr); + }else { +return ppc_hash64_get_phys_page_debug (cpu, eaddr); + } +}target/ppc/mmu_helper.c +2
-7@@ -1342,7 +1342,7 @@ void dump_mmu(FILE *f, fprintf_function cpu_fprintf, CPUPPCState *env)dump_slb (f, cpu_fprintf,ppc_env_get_cpu (env));break ;case POWERPC_MMU_3_00 : -if (ppc64_radix_guest (ppc_env_get_cpu (env))) { +if (ppc64_v3_radix (ppc_env_get_cpu (env))) {/* TODO - Unsupported */ }else {dump_slb (f, cpu_fprintf,ppc_env_get_cpu (env));@@ -1497,12 +1497,7 @@ hwaddr ppc_cpu_get_phys_page_debug(CPUState *cs, vaddr addr)case POWERPC_MMU_2_07 :return ppc_hash64_get_phys_page_debug (cpu, addr);case POWERPC_MMU_3_00 : -if (ppc64_radix_guest (ppc_env_get_cpu (env))) { -return ppc_radix64_get_phys_page_debug (cpu, addr); - }else { -return ppc_hash64_get_phys_page_debug (cpu, addr); - } -break ; +return ppc64_v3_get_phys_page_debug (cpu, addr);#endif case POWERPC_MMU_32B :
Re: [Qemu-devel] [PATCH 02/12] target/ppci/mmu: Use LPCR:HR to chose radix vs. hash translation
Quoted 8 lines · Philippe Mathieu-Daudé
Quoted 5 lines · David Gibson
Quoted 4 lines · Cédric Le Goater
From: Benjamin Herrenschmidt <[email protected]>
Now that LPCR:HR is set properly for SPAPR, use it for deciding the translation type, which also works for bare metal
"ppci" -> "ppc" in patch title.
Fixed up inline.
Quoted 94 lines
Quoted 93 lines
Quoted 4 lines
Signed-off-by Benjamin Herrenschmidt <[email protected]>Signed-off-by Cédric Le Goater <[email protected]>Reviewed-by David Gibson <[email protected]>Quoted 85 lines
--- target/ppc/mmu-book3s-v3.h | 14 +++++++++----- target/ppc/mmu-book3s-v3.c | 11 ++++++++++- target/ppc/mmu_helper.c | 9 ++------- 3 files changed, 21 insertions(+), 13 deletions(-)target/ppc/mmu-book3s-v3.h +9
-5@@ -43,14 +43,18 @@ static inline bool ppc64_use_proc_tbl(PowerPCCPU *cpu)return !!(cpu->env.spr[SPR_LPCR ] &LPCR_UPRT ); } -static inline bool ppc64_radix_guest (PowerPCCPU *cpu) +/* +* The LPCR:HR bit is a shortcut that avoids having to +* dig out the partition table in the fast path. This is +* also how the HW uses it. +*/ +static inline bool ppc64_v3_radix (PowerPCCPU *cpu) { -PPCVirtualHypervisorClass *vhc = -PPC_VIRTUAL_HYPERVISOR_GET_CLASS (cpu->vhyp); - -return !!(vhc->get_patbe (cpu->vhyp) &PATBE1_GR ); +return !!(cpu->env.spr[SPR_LPCR ] &LPCR_HR ); } +hwaddr ppc64_v3_get_phys_page_debug (PowerPCCPU *cpu,vaddr eaddr); +int ppc64_v3_handle_mmu_fault (PowerPCCPU *cpu,vaddr eaddr,int rwx,int mmu_idx);target/ppc/mmu-book3s-v3.c +10
-1@@ -26,9 +26,18 @@int ppc64_v3_handle_mmu_fault (PowerPCCPU *cpu,vaddr eaddr,int rwx,int mmu_idx) { -if (ppc64_radix_guest (cpu)) {/* Guest uses radix */ +if (ppc64_v3_radix (cpu)) {/* Guest uses radix */ return ppc_radix64_handle_mmu_fault (cpu, eaddr, rwx, mmu_idx); }else {/* Guest uses hash */ return ppc_hash64_handle_mmu_fault (cpu, eaddr, rwx, mmu_idx); } } + +hwaddr ppc64_v3_get_phys_page_debug (PowerPCCPU *cpu,vaddr eaddr) +{ +if (ppc64_v3_radix (cpu)) { +return ppc_radix64_get_phys_page_debug (cpu, eaddr); + }else { +return ppc_hash64_get_phys_page_debug (cpu, eaddr); + } +}target/ppc/mmu_helper.c +2
-7@@ -1342,7 +1342,7 @@ void dump_mmu(FILE *f, fprintf_function cpu_fprintf, CPUPPCState *env)dump_slb (f, cpu_fprintf,ppc_env_get_cpu (env));break ;case POWERPC_MMU_3_00 : -if (ppc64_radix_guest (ppc_env_get_cpu (env))) { +if (ppc64_v3_radix (ppc_env_get_cpu (env))) {/* TODO - Unsupported */ }else {dump_slb (f, cpu_fprintf,ppc_env_get_cpu (env));@@ -1497,12 +1497,7 @@ hwaddr ppc_cpu_get_phys_page_debug(CPUState *cs, vaddr addr)case POWERPC_MMU_2_07 :return ppc_hash64_get_phys_page_debug (cpu, addr);case POWERPC_MMU_3_00 : -if (ppc64_radix_guest (ppc_env_get_cpu (env))) { -return ppc_radix64_get_phys_page_debug (cpu, addr); - }else { -return ppc_hash64_get_phys_page_debug (cpu, addr); - } -break ; +return ppc64_v3_get_phys_page_debug (cpu, addr);#endif case POWERPC_MMU_32B :
Signature · 4 lines
David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson
PATCH 3/12 target/ppc: Re-enable RMLS on POWER9 for virtual hypervisors
In reply to Cédric Le GoaterFrom: Benjamin Herrenschmidt <[email protected]>
Historically the 64-bit server MMU supports two way of configuring the guest "real mode" mapping:
- The "RMA" with is a single chunk of physically contiguous
memory remapped as guest real, and controlled by the RMLS field in the LPCR register and the RMOR register.
- The "VRMA" which uses special PTEs inserted in the partition
hash table by the hypervisor.
POWER9 deprecates the former, which is reflected by the filtering done in ppc_store_lpcr() which effectively prevents setting of the RMLS field.
However, when using fully emulated SPAPR machines, our qemu code currently only knows how to define the guest real mode memory using RMLS.
Thus you cannot run a SPAPR machine anymore with a POWER9 CPU model today.
This works around it with a quirk in ppc_store_lpcr() to continue allowing the RMLS field to be set when using a virtual hypervisor.
Ultimately we will want to implement configuring a VRMA instead which will also be necessary if we want to migrate a SPAPR guest between TCG and KVM but this is a lot more work.
--- target/ppc/mmu-hash64.c | 8 ++++++++ 1 file changed, 8 insertions(+)
target/ppc/mmu-hash64.c +8 -0
@@ -1088,6 +1088,14 @@ void ppc_store_lpcr(PowerPCCPU *cpu, target_ulong val) (LPCR_PECE_L_MASK & (LPCR_PDEE |LPCR_HDEE |LPCR_EEE |LPCR_DEE |LPCR_OEE )) |LPCR_MER |LPCR_GTSE |LPCR_TC | LPCR_HEIC | LPCR_LPES0 | LPCR_HVICE | LPCR_HDICE); +/* +* If we have a virtual hypervisor, we need to bring back RMLS. It +* doesn't exist on an actual P9 but that's all we know how to +* configure with softmmu at the moment +*/ +if (cpu->vhyp) { + lpcr |= (val &LPCR_RMLS ); + }break ;default : ;
Signature · 1 line
2.20.1
Re: [Qemu-devel] [PATCH 03/12] target/ppc: Re-enable RMLS on POWER9 for virtual hypervisors
Quoted 23 lines · Cédric Le Goater
From: Benjamin Herrenschmidt <[email protected]>
Historically the 64-bit server MMU supports two way of configuring the guest "real mode" mapping:
- The "RMA" with is a single chunk of physically contiguous
memory remapped as guest real, and controlled by the RMLS field in the LPCR register and the RMOR register.- The "VRMA" which uses special PTEs inserted in the partition
hash table by the hypervisor.POWER9 deprecates the former, which is reflected by the filtering done in
ppc_store_lpcr()which effectively prevents setting of the RMLS field.However, when using fully emulated SPAPR machines, our qemu code currently only knows how to define the guest real mode memory using RMLS.
Thus you cannot run a SPAPR machine anymore with a POWER9 CPU model today.
Quoted 6 lines
This works around it with a quirk in
ppc_store_lpcr()to continue allowing the RMLS field to be set when using a virtual hypervisor.Ultimately we will want to implement configuring a VRMA instead which will also be necessary if we want to migrate a SPAPR guest between TCG and KVM but this is a lot more work.
Urgh. I'm applying this because it fixes a real bug, but it's not really the right solution.
We will want to support VRMAs at some point in order to let emulated powernv machines run guests, but implementing VRMA doesn't really make sense in the context of a PAPR machine.
The real problem here is that we're using the LPCR - which notionally doesn't even exist in a PAPR machine - to trick the softmmu code, which is written from the point of view of bare metal - into doing more or less the right thing for a PAPR machine.
The correct solution, I think, is to put a test on cpu->vhyp into the guts of the softmmu code so that in non-HV real mode it doesn't even consult the LPCR and just goes directly to GPAs (which is the only think that the pseries machine knows about).
Although then I guess we'd need some way in the vhyp of representing the permissible max address of real mode accesses.
Signature · 4 lines
David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson
PATCH 4/12 target/ppc: Fix #include guard in mmu-book3s-v3.h
In reply to Cédric Le GoaterFrom: Benjamin Herrenschmidt <[email protected]>
--- target/ppc/mmu-book3s-v3.h | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
target/ppc/mmu-book3s-v3.h +3 -3
@@ -17,8 +17,8 @@* License along with this library; if not, see <http://www.gnu.org/licenses/>. */ -#ifndef MMU_H -#define MMU_H +#ifndef MMU_BOOOK3S_V3_H +#define MMU_BOOOK3S_V3_H #ifndef CONFIG_USER_ONLY
@@ -62,4 +62,4 @@ int ppc64_v3_handle_mmu_fault(PowerPCCPU *cpu, vaddr eaddr, int rwx,#endif /* CONFIG_USER_ONLY */ -#endif /* MMU_H */ +#endif /* MMU_BOOOK3S_V3_H */
Signature · 1 line
2.20.1
David Gibson
AppliedRe: [Qemu-devel] [PATCH 04/12] target/ppc: Fix #include guard in mmu-book3s-v3.h
Quoted 4 lines · Cédric Le Goater
From: Benjamin Herrenschmidt <[email protected]>
Signed-off-by Benjamin Herrenschmidt <[email protected]>Signed-off-by Cédric Le Goater <[email protected]>
Applied, thanks.
Quoted 25 lines
--- target/ppc/mmu-book3s-v3.h | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)target/ppc/mmu-book3s-v3.h +3
-3@@ -17,8 +17,8 @@* License along with this library; if not, see <http://www.gnu.org/licenses/>. */ -#ifndef MMU_H -#define MMU_H +#ifndef MMU_BOOOK3S_V3_H +#define MMU_BOOOK3S_V3_H #ifndef CONFIG_USER_ONLY @@ -62,4 +62,4 @@ int ppc64_v3_handle_mmu_fault(PowerPCCPU *cpu, vaddr eaddr, int rwx,#endif /* CONFIG_USER_ONLY */ -#endif /* MMU_H */ +#endif /* MMU_BOOOK3S_V3_H */
Signature · 4 lines
David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson
PATCH 5/12 target/ppc: Cleanup 64-bit MMU includes
In reply to Cédric Le GoaterFrom: Benjamin Herrenschmidt <[email protected]>
To enable inlining more things, move #include of mmu-hash64.h and mmu-radix64.h to mmu-book3s-v3.h
--- target/ppc/mmu-book3s-v3.h | 3 +++ hw/ppc/spapr_hcall.c | 1 - target/ppc/mmu-book3s-v3.c | 2 -- target/ppc/mmu-hash64.c | 1 - target/ppc/mmu-radix64.c | 1 - 5 files changed, 3 insertions(+), 5 deletions(-)
target/ppc/mmu-book3s-v3.h +3 -0
@@ -22,6 +22,9 @@#ifndef CONFIG_USER_ONLY +#include "mmu-hash64.h" +#include "mmu-radix64.h" +/* * Partition table definitions */
hw/ppc/spapr_hcall.c +0 -1
@@ -9,7 +9,6 @@#include "helper_regs.h" #include "hw/ppc/spapr.h" #include "hw/ppc/spapr_cpu_core.h" -#include "mmu-hash64.h" #include "cpu-models.h" #include "trace.h" #include "kvm_ppc.h"
target/ppc/mmu-book3s-v3.c +0 -2
@@ -19,9 +19,7 @@#include "qemu/osdep.h" #include "cpu.h" -#include "mmu-hash64.h" #include "mmu-book3s-v3.h" -#include "mmu-radix64.h" int ppc64_v3_handle_mmu_fault (PowerPCCPU *cpu,vaddr eaddr,int rwx,int mmu_idx)
target/ppc/mmu-hash64.c +0 -1
@@ -24,7 +24,6 @@#include "qemu/error-report.h" #include "sysemu/hw_accel.h" #include "kvm_ppc.h" -#include "mmu-hash64.h" #include "exec/log.h" #include "hw/hw.h" #include "mmu-book3s-v3.h"
target/ppc/mmu-radix64.c +0 -1
@@ -25,7 +25,6 @@#include "sysemu/kvm.h" #include "kvm_ppc.h" #include "exec/log.h" -#include "mmu-radix64.h" #include "mmu-book3s-v3.h" static bool ppc_radix64_get_fully_qualified_addr(CPUPPCState *env, vaddr eaddr,
Signature · 1 line
2.20.1
Re: [Qemu-devel] [PATCH 05/12] target/ppc: Cleanup 64-bit MMU includes
Quoted 7 lines · Cédric Le Goater
From: Benjamin Herrenschmidt <[email protected]>
To enable inlining more things, move #include of
mmu-hash64.handmmu-radix64.htommu-book3s-v3.hSigned-off-by Benjamin Herrenschmidt <[email protected]>Signed-off-by Cédric Le Goater <[email protected]>
I don't really get what this is in aid of, and it doesn't seem quite right.
It's fair enough in mmu-book3s-v3.c. But mmu-hash64.c handles 64-bit
hash MMUs earlier than V3 as well, so it doesn't really make sense for it to get to mmu-hash64.c via an indirect include.
Quoted 72 lines
--- target/ppc/mmu-book3s-v3.h | 3 +++ hw/ppc/spapr_hcall.c | 1 - target/ppc/mmu-book3s-v3.c | 2 -- target/ppc/mmu-hash64.c | 1 - target/ppc/mmu-radix64.c | 1 - 5 files changed, 3 insertions(+), 5 deletions(-)target/ppc/mmu-book3s-v3.h +3
-0@@ -22,6 +22,9 @@#ifndef CONFIG_USER_ONLY +#include "mmu-hash64.h" +#include "mmu-radix64.h" +/* * Partition table definitions */ hw/ppc/spapr_hcall.c +0
-1@@ -9,7 +9,6 @@#include "helper_regs.h" #include "hw/ppc/spapr.h" #include "hw/ppc/spapr_cpu_core.h" -#include "mmu-hash64.h" #include "cpu-models.h" #include "trace.h" #include "kvm_ppc.h" target/ppc/mmu-book3s-v3.c +0
-2@@ -19,9 +19,7 @@#include "qemu/osdep.h" #include "cpu.h" -#include "mmu-hash64.h" #include "mmu-book3s-v3.h" -#include "mmu-radix64.h" int ppc64_v3_handle_mmu_fault (PowerPCCPU *cpu,vaddr eaddr,int rwx,int mmu_idx)target/ppc/mmu-hash64.c +0
-1@@ -24,7 +24,6 @@#include "qemu/error-report.h" #include "sysemu/hw_accel.h" #include "kvm_ppc.h" -#include "mmu-hash64.h" #include "exec/log.h" #include "hw/hw.h" #include "mmu-book3s-v3.h" target/ppc/mmu-radix64.c +0
-1@@ -25,7 +25,6 @@#include "sysemu/kvm.h" #include "kvm_ppc.h" #include "exec/log.h" -#include "mmu-radix64.h" #include "mmu-book3s-v3.h" static bool ppc_radix64_get_fully_qualified_addr(CPUPPCState *env, vaddr eaddr,
Signature · 4 lines
David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson
PATCH 6/12 target/ppc: Fix ordering of hash MMU accesses
In reply to Cédric Le GoaterFrom: Benjamin Herrenschmidt <[email protected]>
With mttcg, we can have MMU lookups happening at the same time as the guest modifying the page tables.
Since the HPTEs of the hash table MMU contains two words (or double worlds on 64-bit), we need to make sure we read them in the right order, with the correct memory barrier.
Additionally, when using emulated SPAPR mode, the hypercalls writing to the hash table must also perform the udpates in the right order.
Note: This part is still not entirely correct
--- hw/ppc/spapr.c | 21 +++++++++++++++++++-- target/ppc/mmu-hash32.c | 6 ++++++ target/ppc/mmu-hash64.c | 6 ++++++ 3 files changed, 31 insertions(+), 2 deletions(-)
hw/ppc/spapr.c +19 -2
@@ -1507,8 +1507,25 @@ static void spapr_store_hpte(PPCVirtualHypervisor *vhyp, hwaddr ptex,if (!spapr->htab) {kvmppc_write_hpte (ptex, pte0, pte1); }else { -stq_p (spapr->htab + offset, pte0); -stq_p (spapr->htab + offset +HASH_PTE_SIZE_64 /2 , pte1); +if (pte0 &HPTE64_V_VALID ) { +stq_p (spapr->htab + offset +HASH_PTE_SIZE_64 /2 , pte1); +/* +* When setting valid, we write PTE1 first. This ensures +* proper synchronization with the reading code in +* ppc_hash64_pteg_search() +*/ +smp_wmb (); +stq_p (spapr->htab + offset, pte0); + }else { +stq_p (spapr->htab + offset, pte0); +/* +* When clearing it we set PTE0 first. This ensures proper +* synchronization with the reading code in +* ppc_hash64_pteg_search() +*/ +smp_wmb (); +stq_p (spapr->htab + offset +HASH_PTE_SIZE_64 /2 , pte1); + } } }
target/ppc/mmu-hash32.c +6 -0
@@ -319,6 +319,12 @@ static hwaddr ppc_hash32_pteg_search(PowerPCCPU *cpu, hwaddr pteg_off,for (i =0 ; i <HPTES_PER_GROUP ; i++) { pte0 =ppc_hash32_load_hpte0 (cpu, pte_offset); +/* +* pte0 contains the valid bit and must be read before pte1, +* otherwise we might see an old pte1 with a new valid bit and +* thus an inconsistent hpte value +*/ +smp_rmb (); pte1 =ppc_hash32_load_hpte1 (cpu, pte_offset); if ((pte0 & HPTE32_V_VALID)
target/ppc/mmu-hash64.c +6 -0
@@ -506,6 +506,12 @@ static hwaddr ppc_hash64_pteg_search(PowerPCCPU *cpu, hwaddr hash, }for (i =0 ; i <HPTES_PER_GROUP ; i++) { pte0 =ppc_hash64_hpte0 (cpu, pteg, i); +/* +* pte0 contains the valid bit and must be read before pte1, +* otherwise we might see an old pte1 with a new valid bit and +* thus an inconsistent hpte value +*/ +smp_rmb (); pte1 =ppc_hash64_hpte1 (cpu, pteg, i);/* This compares V, B, H (secondary) and the AVPN */
Signature · 1 line
2.20.1
David Gibson
AppliedRe: [Qemu-devel] [PATCH 06/12] target/ppc: Fix ordering of hash MMU accesses
Quoted 9 lines · Cédric Le Goater
From: Benjamin Herrenschmidt <[email protected]>
With mttcg, we can have MMU lookups happening at the same time as the guest modifying the page tables.
Since the HPTEs of the hash table MMU contains two words (or double worlds on 64-bit), we need to make sure we read them in the right order, with the correct memory barrier.
Quoted 8 lines
Additionally, when using emulated SPAPR mode, the hypercalls writing to the hash table must also perform the udpates in the right order.
Note: This part is still not entirely correct
Signed-off-by Benjamin Herrenschmidt <[email protected]>Signed-off-by Cédric Le Goater <[email protected]>
Yeah, this stuff was written under the assumption it was protected by the BQL, which is getting less true all the time.
Applied.
Quoted 72 lines
--- hw/ppc/spapr.c | 21 +++++++++++++++++++-- target/ppc/mmu-hash32.c | 6 ++++++ target/ppc/mmu-hash64.c | 6 ++++++ 3 files changed, 31 insertions(+), 2 deletions(-)hw/ppc/spapr.c +19
-2@@ -1507,8 +1507,25 @@ static void spapr_store_hpte(PPCVirtualHypervisor *vhyp, hwaddr ptex,if (!spapr->htab) {kvmppc_write_hpte (ptex, pte0, pte1); }else { -stq_p (spapr->htab + offset, pte0); -stq_p (spapr->htab + offset +HASH_PTE_SIZE_64 /2 , pte1); +if (pte0 &HPTE64_V_VALID ) { +stq_p (spapr->htab + offset +HASH_PTE_SIZE_64 /2 , pte1); +/* +* When setting valid, we write PTE1 first. This ensures +* proper synchronization with the reading code in +* ppc_hash64_pteg_search() +*/ +smp_wmb (); +stq_p (spapr->htab + offset, pte0); + }else { +stq_p (spapr->htab + offset, pte0); +/* +* When clearing it we set PTE0 first. This ensures proper +* synchronization with the reading code in +* ppc_hash64_pteg_search() +*/ +smp_wmb (); +stq_p (spapr->htab + offset +HASH_PTE_SIZE_64 /2 , pte1); + } } }target/ppc/mmu-hash32.c +6
-0@@ -319,6 +319,12 @@ static hwaddr ppc_hash32_pteg_search(PowerPCCPU *cpu, hwaddr pteg_off,for (i =0 ; i <HPTES_PER_GROUP ; i++) { pte0 =ppc_hash32_load_hpte0 (cpu, pte_offset); +/* +* pte0 contains the valid bit and must be read before pte1, +* otherwise we might see an old pte1 with a new valid bit and +* thus an inconsistent hpte value +*/ +smp_rmb (); pte1 =ppc_hash32_load_hpte1 (cpu, pte_offset); if ((pte0 & HPTE32_V_VALID)target/ppc/mmu-hash64.c +6
-0@@ -506,6 +506,12 @@ static hwaddr ppc_hash64_pteg_search(PowerPCCPU *cpu, hwaddr hash, }for (i =0 ; i <HPTES_PER_GROUP ; i++) { pte0 =ppc_hash64_hpte0 (cpu, pteg, i); +/* +* pte0 contains the valid bit and must be read before pte1, +* otherwise we might see an old pte1 with a new valid bit and +* thus an inconsistent hpte value +*/ +smp_rmb (); pte1 =ppc_hash64_hpte1 (cpu, pteg, i);/* This compares V, B, H (secondary) and the AVPN */
Signature · 4 lines
David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson
PATCH 7/12 target/ppc: Add basic support for "new format" HPTE as found on POWER9
In reply to Cédric Le GoaterFrom: Benjamin Herrenschmidt <[email protected]>
POWER9 (arch v3) slightly changes the HPTE format. The B bits move from the first to the second half of the HPTE, and the AVPN/ARPN are slightly shorter.
However, under SPAPR, the hypercalls still take the old format (and probably will for the foreseable future).
The simplest way to support this is thus to convert the HPTEs from new to old format when reading them if the MMU model is v3 and there is no virtual hypervisor, leaving the rest of the code unchanged.
--- target/ppc/mmu-book3s-v3.h | 12 ++++++++++++ target/ppc/mmu-hash64.h | 5 +++++ target/ppc/mmu-hash64.c | 5 +++++ 3 files changed, 22 insertions(+)
target/ppc/mmu-book3s-v3.h +12 -0
@@ -56,6 +56,18 @@ static inline bool ppc64_v3_radix(PowerPCCPU *cpu)return !!(cpu->env.spr[SPR_LPCR ] &LPCR_HR ); } +static inline void ppc64_v3_new_to_old_hpte (target_ulong *pte0, +target_ulong *pte1) +{ +/* Insert B into pte0 */ + *pte0 = (*pte0 &HPTE64_V_COMMON_BITS ) | + ((*pte1 &HPTE64_R_3_0_SSIZE_MASK ) << + (HPTE64_V_SSIZE_SHIFT -HPTE64_R_3_0_SSIZE_SHIFT )); + +/* Remove B from pte1 */ + *pte1 = *pte1 & ~HPTE64_R_3_0_SSIZE_MASK ; +} +hwaddr ppc64_v3_get_phys_page_debug (PowerPCCPU *cpu,vaddr eaddr); int ppc64_v3_handle_mmu_fault(PowerPCCPU *cpu, vaddr eaddr, int rwx,
target/ppc/mmu-hash64.h +5 -0
@@ -102,6 +102,11 @@ void ppc_hash64_filter_pagesizes(PowerPCCPU *cpu,#define HPTE64_V_1TB_SEG 0x4000000000000000ULL#define HPTE64_V_VRMA_MASK 0x4001ffffff000000ULL +/* Format changes for ARCH v3 */ +#define HPTE64_V_COMMON_BITS 0x000fffffffffffffULL +#define HPTE64_R_3_0_SSIZE_SHIFT 58 +#define HPTE64_R_3_0_SSIZE_MASK (3ULL << HPTE64_R_3_0_SSIZE_SHIFT) +static inline hwaddr ppc_hash64_hpt_base (PowerPCCPU *cpu) {if (cpu->vhyp) {
target/ppc/mmu-hash64.c +5 -0
@@ -514,6 +514,11 @@ static hwaddr ppc_hash64_pteg_search(PowerPCCPU *cpu, hwaddr hash,smp_rmb (); pte1 =ppc_hash64_hpte1 (cpu, pteg, i); +/* Convert format if necessary */ +if (cpu->env.mmu_model ==POWERPC_MMU_3_00 && !cpu->vhyp) { +ppc64_v3_new_to_old_hpte (&pte0, &pte1); + } +/* This compares V, B, H (secondary) and the AVPN */ if (HPTE64_V_COMPARE (pte0, ptem)) { *pshift =hpte_page_shift (sps, pte0, pte1);
Signature · 1 line
2.20.1
Re: [Qemu-devel] [PATCH 07/12] target/ppc: Add basic support for "new format" HPTE as found on POWER9
Quoted 9 lines · Cédric Le Goater
From: Benjamin Herrenschmidt <[email protected]>
POWER9 (arch v3) slightly changes the HPTE format. The B bits move from the first to the second half of the HPTE, and the AVPN/ARPN are slightly shorter.
However, under SPAPR, the hypercalls still take the old format (and probably will for the foreseable future).
Quoted 6 lines
The simplest way to support this is thus to convert the HPTEs from new to old format when reading them if the MMU model is v3 and there is no virtual hypervisor, leaving the rest of the code unchanged.
Signed-off-by Benjamin Herrenschmidt <[email protected]>Signed-off-by Cédric Le Goater <[email protected]>
Kinda hacky, but definitely the easiest way in the short to medium term.
Quoted 61 lines
--- target/ppc/mmu-book3s-v3.h | 12 ++++++++++++ target/ppc/mmu-hash64.h | 5 +++++ target/ppc/mmu-hash64.c | 5 +++++ 3 files changed, 22 insertions(+)target/ppc/mmu-book3s-v3.h +12
-0@@ -56,6 +56,18 @@ static inline bool ppc64_v3_radix(PowerPCCPU *cpu)return !!(cpu->env.spr[SPR_LPCR ] &LPCR_HR ); } +static inline void ppc64_v3_new_to_old_hpte (target_ulong *pte0, +target_ulong *pte1) +{ +/* Insert B into pte0 */ + *pte0 = (*pte0 &HPTE64_V_COMMON_BITS ) | + ((*pte1 &HPTE64_R_3_0_SSIZE_MASK ) << + (HPTE64_V_SSIZE_SHIFT -HPTE64_R_3_0_SSIZE_SHIFT )); + +/* Remove B from pte1 */ + *pte1 = *pte1 & ~HPTE64_R_3_0_SSIZE_MASK ; +} +hwaddr ppc64_v3_get_phys_page_debug (PowerPCCPU *cpu,vaddr eaddr); int ppc64_v3_handle_mmu_fault(PowerPCCPU *cpu, vaddr eaddr, int rwx,target/ppc/mmu-hash64.h +5
-0@@ -102,6 +102,11 @@ void ppc_hash64_filter_pagesizes(PowerPCCPU *cpu,#define HPTE64_V_1TB_SEG 0x4000000000000000ULL#define HPTE64_V_VRMA_MASK 0x4001ffffff000000ULL +/* Format changes for ARCH v3 */ +#define HPTE64_V_COMMON_BITS 0x000fffffffffffffULL +#define HPTE64_R_3_0_SSIZE_SHIFT 58 +#define HPTE64_R_3_0_SSIZE_MASK (3ULL << HPTE64_R_3_0_SSIZE_SHIFT) +static inline hwaddr ppc_hash64_hpt_base (PowerPCCPU *cpu) {if (cpu->vhyp) {target/ppc/mmu-hash64.c +5
-0@@ -514,6 +514,11 @@ static hwaddr ppc_hash64_pteg_search(PowerPCCPU *cpu, hwaddr hash,smp_rmb (); pte1 =ppc_hash64_hpte1 (cpu, pteg, i); +/* Convert format if necessary */ +if (cpu->env.mmu_model ==POWERPC_MMU_3_00 && !cpu->vhyp) { +ppc64_v3_new_to_old_hpte (&pte0, &pte1); + } +/* This compares V, B, H (secondary) and the AVPN */ if (HPTE64_V_COMPARE (pte0, ptem)) { *pshift =hpte_page_shift (sps, pte0, pte1);
Signature · 4 lines
David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson
Re: [Qemu-devel] [PATCH 07/12] target/ppc: Add basic support for "new format" HPTE as found on POWER9
Quoted 14 lines · David Gibson
Quoted 13 lines · Cédric Le Goater
From: Benjamin Herrenschmidt <[email protected]>
POWER9 (arch v3) slightly changes the HPTE format. The B bits move from the first to the second half of the HPTE, and the AVPN/ARPN are slightly shorter.
However, under SPAPR, the hypercalls still take the old format (and probably will for the foreseable future).
The simplest way to support this is thus to convert the HPTEs from new to old format when reading them if the MMU model is v3 and there is no virtual hypervisor, leaving the rest of the code unchanged.
Quoted 5 lines
Quoted 2 lines
Signed-off-by Benjamin Herrenschmidt <[email protected]>Signed-off-by Cédric Le Goater <[email protected]>Kinda hacky, but definitely the easiest way in the short to medium term.
So, this doesn't compile as-is without the include rearrangement I objected to earlier in the series.
But.. I'm not actually seeing any reason (here or later in the series) that ppc64_v3_new_to_old_hpte() really needs to be in the header. So I'm applying this with it moved into mmu-hash64.c.
Quoted 63 lines
Quoted 62 lines
--- target/ppc/mmu-book3s-v3.h | 12 ++++++++++++ target/ppc/mmu-hash64.h | 5 +++++ target/ppc/mmu-hash64.c | 5 +++++ 3 files changed, 22 insertions(+)target/ppc/mmu-book3s-v3.h +12
-0@@ -56,6 +56,18 @@ static inline bool ppc64_v3_radix(PowerPCCPU *cpu)return !!(cpu->env.spr[SPR_LPCR ] &LPCR_HR ); } +static inline void ppc64_v3_new_to_old_hpte (target_ulong *pte0, +target_ulong *pte1) +{ +/* Insert B into pte0 */ + *pte0 = (*pte0 &HPTE64_V_COMMON_BITS ) | + ((*pte1 &HPTE64_R_3_0_SSIZE_MASK ) << + (HPTE64_V_SSIZE_SHIFT -HPTE64_R_3_0_SSIZE_SHIFT )); + +/* Remove B from pte1 */ + *pte1 = *pte1 & ~HPTE64_R_3_0_SSIZE_MASK ; +} +hwaddr ppc64_v3_get_phys_page_debug (PowerPCCPU *cpu,vaddr eaddr); int ppc64_v3_handle_mmu_fault(PowerPCCPU *cpu, vaddr eaddr, int rwx,target/ppc/mmu-hash64.h +5
-0@@ -102,6 +102,11 @@ void ppc_hash64_filter_pagesizes(PowerPCCPU *cpu,#define HPTE64_V_1TB_SEG 0x4000000000000000ULL#define HPTE64_V_VRMA_MASK 0x4001ffffff000000ULL +/* Format changes for ARCH v3 */ +#define HPTE64_V_COMMON_BITS 0x000fffffffffffffULL +#define HPTE64_R_3_0_SSIZE_SHIFT 58 +#define HPTE64_R_3_0_SSIZE_MASK (3ULL << HPTE64_R_3_0_SSIZE_SHIFT) +static inline hwaddr ppc_hash64_hpt_base (PowerPCCPU *cpu) {if (cpu->vhyp) {target/ppc/mmu-hash64.c +5
-0@@ -514,6 +514,11 @@ static hwaddr ppc_hash64_pteg_search(PowerPCCPU *cpu, hwaddr hash,smp_rmb (); pte1 =ppc_hash64_hpte1 (cpu, pteg, i); +/* Convert format if necessary */ +if (cpu->env.mmu_model ==POWERPC_MMU_3_00 && !cpu->vhyp) { +ppc64_v3_new_to_old_hpte (&pte0, &pte1); + } +/* This compares V, B, H (secondary) and the AVPN */ if (HPTE64_V_COMPARE (pte0, ptem)) { *pshift =hpte_page_shift (sps, pte0, pte1);
Signature · 4 lines
David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson
PATCH 8/12 target/ppc: Fix synchronization of mttcg with broadcast TLB flushes
In reply to Cédric Le GoaterFrom: Benjamin Herrenschmidt <[email protected]>
Let's use the generic helper tlb_flush_all_cpus_synced() instead of iterating the CPUs ourselves.
We do lose the optimization of clearing the "other" CPUs "need flush" flags but this shouldn't be a problem in practice.
--- target/ppc/helper_regs.h | 27 ++++++++++----------------- 1 file changed, 10 insertions(+), 17 deletions(-)
target/ppc/helper_regs.h +10 -17
@@ -174,26 +174,19 @@ static inline int hreg_store_msr(CPUPPCState *env, target_ulong value,static inline void check_tlb_flush (CPUPPCState *env,bool global) {CPUState *cs =CPU (ppc_env_get_cpu (env)); -if (env->tlb_need_flush &TLB_NEED_LOCAL_FLUSH ) { -tlb_flush (cs); - env->tlb_need_flush &= ~TLB_NEED_LOCAL_FLUSH ; - } -/* Propagate TLB invalidations to other CPUs when the guest uses broadcast -* TLB invalidation instructions. -*/ +/* Handle global flushes first */ if (global && (env->tlb_need_flush &TLB_NEED_GLOBAL_FLUSH )) { -CPUState *other_cs; -CPU_FOREACH (other_cs) { -if (other_cs != cs) { -PowerPCCPU *cpu =POWERPC_CPU (other_cs); -CPUPPCState *other_env = &cpu->env; - - other_env->tlb_need_flush &= ~TLB_NEED_LOCAL_FLUSH ; -tlb_flush (other_cs); - } - } env->tlb_need_flush &= ~TLB_NEED_GLOBAL_FLUSH ; + env->tlb_need_flush &= ~TLB_NEED_LOCAL_FLUSH ; +tlb_flush_all_cpus_synced (cs); +return ; + } + +/* Then handle local ones */ +if (env->tlb_need_flush &TLB_NEED_LOCAL_FLUSH ) { + env->tlb_need_flush &= ~TLB_NEED_LOCAL_FLUSH ; +tlb_flush (cs); } }#else
Signature · 1 line
2.20.1
PATCH 9/12 target/ppc: Flush the TLB locally when the LPIDR is written
In reply to Cédric Le GoaterFrom: Benjamin Herrenschmidt <[email protected]>
Our TCG TLB only tags whether it's a HV vs a guest access, so it must be flushed when the LPIDR is changed.
--- target/ppc/helper.h | 1 + target/ppc/misc_helper.c | 15 +++++++++++++++ target/ppc/translate_init.inc.c | 7 ++++++- 3 files changed, 22 insertions(+), 1 deletion(-)
target/ppc/helper.h +1 -0
@@ -718,6 +718,7 @@ DEF_HELPER_2(store_ptcr, void, env, tl)#endif DEF_HELPER_2(store_sdr1, void, env, tl) DEF_HELPER_2(store_pidr, void, env, tl) +DEF_HELPER_2(store_lpidr, void, env, tl) DEF_HELPER_FLAGS_2(store_tbl, TCG_CALL_NO_RWG, void, env, tl) DEF_HELPER_FLAGS_2(store_tbu, TCG_CALL_NO_RWG, void, env, tl)DEF_HELPER_FLAGS_2 (store_atbl,TCG_CALL_NO_RWG , void, env, tl)
target/ppc/misc_helper.c +15 -0
@@ -117,6 +117,21 @@ void helper_store_pidr(CPUPPCState *env, target_ulong val)tlb_flush (CPU (cpu)); } +void helper_store_lpidr (CPUPPCState *env,target_ulong val) +{ +PowerPCCPU *cpu =ppc_env_get_cpu (env); + + env->spr[SPR_LPIDR ] = val; + +/* +* We need to flush the TLB on LPID changes as we only tag HV vs +* guest in TCG TLB. Also the quadrants means the HV will +* potentially access and cache entries for the current LPID as +* well. +*/ +tlb_flush (CPU (cpu)); +} +void helper_store_hid0_601 (CPUPPCState *env,target_ulong val) {target_ulong hid0;
target/ppc/translate_init.inc.c +6 -1
@@ -407,6 +407,11 @@ static void spr_write_pidr(DisasContext *ctx, int sprn, int gprn)gen_helper_store_pidr (cpu_env, cpu_gpr[gprn]); } +static void spr_write_lpidr (DisasContext *ctx,int sprn,int gprn) +{ +gen_helper_store_lpidr (cpu_env, cpu_gpr[gprn]); +} +static void spr_read_hior (DisasContext *ctx,int gprn,int sprn) {tcg_gen_ld_tl (cpu_gpr[gprn], cpu_env, offsetof(CPUPPCState , excp_prefix));
@@ -7876,7 +7881,7 @@ static void gen_spr_book3s_ids(CPUPPCState *env)spr_register_hv (env,SPR_LPIDR ,"LPIDR" ,SPR_NOACCESS ,SPR_NOACCESS ,SPR_NOACCESS ,SPR_NOACCESS , - &spr_read_generic, &spr_write_generic, + &spr_read_generic, &spr_write_lpidr,0x00000000 ); spr_register_hv(env, SPR_HFSCR, "HFSCR", SPR_NOACCESS, SPR_NOACCESS,
Signature · 1 line
2.20.1
PATCH 10/12 target/ppc: Rename PATB/PATBE -> PATE
In reply to Cédric Le GoaterFrom: Benjamin Herrenschmidt <[email protected]>
That "b" means "base address" and thus shouldn't be in the name of actual entries and related constants.
This patch keeps the synthetic patb_entry field of the spapr virtual hypervisor unchanged until I figure out if that has an impact on the migration stream.
--- target/ppc/cpu.h | 6 +++++- target/ppc/mmu-book3s-v3.h | 11 ++++++++++- target/ppc/mmu-radix64.h | 4 ++-- hw/ppc/spapr.c | 24 +++++++++++++++--------- hw/ppc/spapr_hcall.c | 22 ++++++++++++---------- target/ppc/mmu-radix64.c | 18 ++++++++++-------- 6 files changed, 54 insertions(+), 31 deletions(-)
target/ppc/cpu.h +5 -1
@@ -317,6 +317,10 @@ struct ppc_slb_t {#define SEGMENT_SHIFT_1T 40#define SEGMENT_MASK_1T (~((1ULL << SEGMENT_SHIFT_1T) - 1)) +typedef struct ppc_v3_pate_t { +uint64_t dw0; +uint64_t dw1; +}ppc_v3_pate_t ;/*****************************************************************************/ /* Machine state register bits definition */
@@ -1239,7 +1243,7 @@ struct PPCVirtualHypervisorClass { hwaddr ptex, int n);void (*store_hpte)(PPCVirtualHypervisor *vhyp,hwaddr ptex,uint64_t pte0,uint64_t pte1); -uint64_t (*get_patbe)(PPCVirtualHypervisor *vhyp); +void (*get_pate)(PPCVirtualHypervisor *vhyp,ppc_v3_pate_t *entry);target_ulong (*encode_hpt_for_kvm_pr)(PPCVirtualHypervisor *vhyp); };
target/ppc/mmu-book3s-v3.h +10 -1
@@ -32,7 +32,16 @@#define PTCR_PATS 0x000000000000001FULL/* Partition Table Size */ /* Partition Table Entry Fields */ -#define PATBE1_GR 0x8000000000000000 +#define PATE0_HR 0x8000000000000000 + +/* +* WARNING: This field doesn't actually exist in the final version of +* the architecture and is unused by hardware. However, qemu uses it +* as an indication of a radix guest in the pseudo-PATB entry that it +* maintains for SPAPR guests and in the migration stream, so we need +* to keep it around +*/ +#define PATE1_GR 0x8000000000000000/* Process Table Entry */ struct prtb_entry {
target/ppc/mmu-radix64.h +2 -2
@@ -12,8 +12,8 @@#define R_EADDR_QUADRANT3 0xC000000000000000/* Radix Partition Table Entry Fields */ -#define PATBE1_R_PRTB 0x0FFFFFFFFFFFF000 -#define PATBE1_R_PRTS 0x000000000000001F +#define PATE1_R_PRTB 0x0FFFFFFFFFFFF000 +#define PATE1_R_PRTS 0x000000000000001F/* Radix Process Table Entry Fields */ #define PRTBE_R_GET_RTS (rts) \
hw/ppc/spapr.c +15 -9
@@ -1403,11 +1403,13 @@ void spapr_set_all_lpcrs(target_ulong value, target_ulong mask) } } -static uint64_t spapr_get_patbe (PPCVirtualHypervisor *vhyp) +static void spapr_get_pate (PPCVirtualHypervisor *vhyp,ppc_v3_pate_t *entry) {sPAPRMachineState *spapr =SPAPR_MACHINE (vhyp); -return spapr->patb_entry; +/* Copy PATE1:GR into PATE0:HR */ + entry->dw0 = spapr->patb_entry &PATE0_HR ; + entry->dw1 = spapr->patb_entry; }#define HPTE (_table, _i) (void *)(((uint64_t *)(_table)) + ((_i) * 2))
@@ -1650,17 +1652,21 @@ static void spapr_machine_reset(void)if (kvm_enabled () &&kvmppc_has_cap_mmu_radix () &&ppc_type_check_compat (machine->cpu_type,CPU_POWERPC_LOGICAL_3_00 ,0 , spapr->max_compat_pvr)) { -/* If using KVM with radix mode available, VCPUs can be started +/* +* If using KVM with radix mode available, VCPUs can be started * without a HPT because KVM will start them in radix mode. -* Set the GR bit in PATB so that we know there is no HPT. */ - spapr->patb_entry =PATBE1_GR ; +* Set the GR bit in PATE so that we know there is no HPT. +*/ + spapr->patb_entry =PATE1_GR ;spapr_set_all_lpcrs (LPCR_HR |LPCR_UPRT ,LPCR_HR |LPCR_UPRT ); }else {spapr_setup_hpt_and_vrma (spapr); } -/* if this reset wasn't generated by CAS, we should reset our -* negotiated options and start from scratch */ +/* +* If this reset wasn't generated by CAS, we should reset our +* negotiated options and start from scratch +*/ if (!spapr->cas_reboot) {spapr_ovec_cleanup (spapr->ov5_cas); spapr->ov5_cas =spapr_ovec_new ();
@@ -1810,7 +1816,7 @@ static int spapr_post_load(void *opaque, int version_id)if (kvm_enabled () && spapr->patb_entry) {PowerPCCPU *cpu =POWERPC_CPU (first_cpu); -bool radix = !!(spapr->patb_entry &PATBE1_GR ); +bool radix = !!(spapr->patb_entry &PATE1_GR );bool gtse = !!(cpu->env.spr[SPR_LPCR ] &LPCR_GTSE );/*
@@ -4060,7 +4066,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data) vhc->map_hptes = spapr_map_hptes; vhc->unmap_hptes = spapr_unmap_hptes; vhc->store_hpte = spapr_store_hpte; - vhc->get_patbe = spapr_get_patbe; + vhc->get_pate = spapr_get_pate; vhc->encode_hpt_for_kvm_pr = spapr_encode_hpt_for_kvm_pr; xic->ics_get = spapr_ics_get; xic->ics_resend = spapr_ics_resend;
hw/ppc/spapr_hcall.c +12 -10
@@ -1310,12 +1310,12 @@ static void spapr_check_setup_free_hpt(sPAPRMachineState *spapr,* later and so assumed radix and now it's called H_REG_PROC_TBL */ -if ((patbe_old &PATBE1_GR ) == (patbe_new &PATBE1_GR )) { +if ((patbe_old &PATE1_GR ) == (patbe_new &PATE1_GR )) {/* We assume RADIX, so this catches all the "Do Nothing" cases */ - }else if (!(patbe_old &PATBE1_GR )) { + }else if (!(patbe_old &PATE1_GR )) {/* HASH->RADIX : Free HPT */ spapr_free_hpt (spapr); - }else if (!(patbe_new &PATBE1_GR )) { + }else if (!(patbe_new &PATE1_GR )) {/* RADIX->HASH || NOTHING->HASH : Allocate HPT */ spapr_setup_hpt_and_vrma (spapr); }
@@ -1353,7 +1353,7 @@ static target_ulong h_register_process_table(PowerPCCPU *cpu, }else if (table_size >24 ) {return H_P4 ; } - cproc =PATBE1_GR | proc_tbl | table_size; + cproc =PATE1_GR | proc_tbl | table_size; }else {/* Register new HPT process table */ if (flags &FLAG_HASH_PROC_TBL ) {/* Hash with Segment Tables */ /* TODO - Not Supported */
@@ -1372,13 +1372,15 @@ static target_ulong h_register_process_table(PowerPCCPU *cpu, } }else {/* Deregister current process table */ -/* Set to benign value: (current GR) | 0. This allows -* deregistration in KVM to succeed even if the radix bit in flags -* doesn't match the radix bit in the old PATB. */ - cproc = spapr->patb_entry &PATBE1_GR ; +/* +* Set to benign value: (current GR) | 0. This allows +* deregistration in KVM to succeed even if the radix bit +* in flags doesn't match the radix bit in the old PATE. +*/ + cproc = spapr->patb_entry &PATE1_GR ; } }else {/* Maintain current registration */ -if (!(flags &FLAG_RADIX ) != !(spapr->patb_entry &PATBE1_GR )) { +if (!(flags &FLAG_RADIX ) != !(spapr->patb_entry &PATE1_GR )) {/* Technically caused by flag bits => H_PARAMETER */ return H_PARAMETER ;/* Existing Process Table Mismatch */ }
@@ -1615,7 +1617,7 @@ static target_ulong h_client_architecture_support(PowerPCCPU *cpu,if (!spapr->cas_reboot) {/* If spapr_machine_reset() did not set up a HPT but one is necessary * (because the guest isn't going to use radix) then set it up here. */ -if ((spapr->patb_entry &PATBE1_GR ) && !guest_radix) { +if ((spapr->patb_entry &PATE1_GR ) && !guest_radix) {/* legacy hash or new hash: */ spapr_setup_hpt_and_vrma (spapr); }
target/ppc/mmu-radix64.c +10 -8
@@ -193,8 +193,9 @@ int ppc_radix64_handle_mmu_fault(PowerPCCPU *cpu, vaddr eaddr, int rwx,PPCVirtualHypervisorClass *vhc =PPC_VIRTUAL_HYPERVISOR_GET_CLASS (cpu->vhyp);hwaddr raddr, pte_addr; -uint64_t lpid =0 , pid =0 , offset, size, patbe, prtbe0, pte; +uint64_t lpid =0 , pid =0 , offset, size, prtbe0, pte;int page_size, prot, fault_cause =0 ; +ppc_v3_pate_t pate;assert ((rwx ==0 ) || (rwx ==1 ) || (rwx ==2 ));assert (!msr_hv);/* For now there is no Radix PowerNV Support */
@@ -219,17 +220,17 @@ int ppc_radix64_handle_mmu_fault(PowerPCCPU *cpu, vaddr eaddr, int rwx, }/* Get Process Table */ - patbe = vhc->get_patbe (cpu->vhyp); + vhc->get_pate (cpu->vhyp, &pate);/* Index Process Table by PID to Find Corresponding Process Table Entry */ offset = pid *sizeof (struct prtb_entry ); - size =1ULL << ((patbe &PATBE1_R_PRTS ) +12 ); + size =1ULL << ((pate.dw1 &PATE1_R_PRTS ) +12 );if (offset >= size) {/* offset exceeds size of the process table */ ppc_radix64_raise_si (cpu, rwx, eaddr,DSISR_NOPTE );return 1 ; } - prtbe0 =ldq_phys (cs->as, (patbe &PATBE1_R_PRTB ) + offset); + prtbe0 =ldq_phys (cs->as, (pate.dw1 &PATE1_R_PRTB ) + offset);/* Walk Radix Tree from Process Table Entry to Convert EA to RA */ page_size =PRTBE_R_GET_RTS (prtbe0);
@@ -257,8 +258,9 @@ hwaddr ppc_radix64_get_phys_page_debug(PowerPCCPU *cpu, target_ulong eaddr)PPCVirtualHypervisorClass *vhc =PPC_VIRTUAL_HYPERVISOR_GET_CLASS (cpu->vhyp);hwaddr raddr, pte_addr; -uint64_t lpid =0 , pid =0 , offset, size, patbe, prtbe0, pte; +uint64_t lpid =0 , pid =0 , offset, size, prtbe0, pte;int page_size, fault_cause =0 ; +ppc_v3_pate_t pate;/* Handle Real Mode */ if (msr_dr ==0 ) {
@@ -272,16 +274,16 @@ hwaddr ppc_radix64_get_phys_page_debug(PowerPCCPU *cpu, target_ulong eaddr) }/* Get Process Table */ - patbe = vhc->get_patbe (cpu->vhyp); + vhc->get_pate (cpu->vhyp, &pate);/* Index Process Table by PID to Find Corresponding Process Table Entry */ offset = pid *sizeof (struct prtb_entry ); - size =1ULL << ((patbe &PATBE1_R_PRTS ) +12 ); + size =1ULL << ((pate.dw1 &PATE1_R_PRTS ) +12 );if (offset >= size) {/* offset exceeds size of the process table */ return -1 ; } - prtbe0 =ldq_phys (cs->as, (patbe &PATBE1_R_PRTB ) + offset); + prtbe0 =ldq_phys (cs->as, (pate.dw1 &PATE1_R_PRTB ) + offset);/* Walk Radix Tree from Process Table Entry to Convert EA to RA */ page_size =PRTBE_R_GET_RTS (prtbe0);
Signature · 1 line
2.20.1
PATCH 11/12 target/ppc: Support for POWER9 native hash
In reply to Cédric Le GoaterFrom: Benjamin Herrenschmidt <[email protected]>
(Might need more patch splitting)
--- target/ppc/mmu-book3s-v3.h | 45 ++++++++++++++++++++++++++++++++++++++ target/ppc/mmu-hash64.h | 19 +--------------- target/ppc/mmu-book3s-v3.c | 18 +++++++++++++++ target/ppc/mmu-hash64.c | 6 +++-- 4 files changed, 68 insertions(+), 20 deletions(-)
target/ppc/mmu-book3s-v3.h +45 -0
@@ -55,6 +55,9 @@ static inline bool ppc64_use_proc_tbl(PowerPCCPU *cpu)return !!(cpu->env.spr[SPR_LPCR ] &LPCR_UPRT ); } +bool ppc64_v3_get_pate (PowerPCCPU *cpu,target_ulong lpid, +ppc_v3_pate_t *entry); +/* * The LPCR:HR bit is a shortcut that avoids having to * dig out the partition table in the fast path. This is
@@ -82,6 +85,48 @@ hwaddr ppc64_v3_get_phys_page_debug(PowerPCCPU *cpu, vaddr eaddr);int ppc64_v3_handle_mmu_fault (PowerPCCPU *cpu,vaddr eaddr,int rwx,int mmu_idx); +static inline hwaddr ppc_hash64_hpt_base (PowerPCCPU *cpu) +{ +uint64_t base; + +if (cpu->vhyp) { +return 0 ; + } +if (cpu->env.mmu_model ==POWERPC_MMU_3_00 ) { +ppc_v3_pate_t pate; + +if (!ppc64_v3_get_pate (cpu, cpu->env.spr[SPR_LPIDR ], &pate)) { +return 0 ; + } + base = pate.dw0; + }else { + base = cpu->env.spr[SPR_SDR1 ]; + } +return base &SDR_64_HTABORG ; +} + +static inline hwaddr ppc_hash64_hpt_mask (PowerPCCPU *cpu) +{ +uint64_t base; + +if (cpu->vhyp) { +PPCVirtualHypervisorClass *vhc = +PPC_VIRTUAL_HYPERVISOR_GET_CLASS (cpu->vhyp); +return vhc->hpt_mask (cpu->vhyp); + } +if (cpu->env.mmu_model ==POWERPC_MMU_3_00 ) { +ppc_v3_pate_t pate; + +if (!ppc64_v3_get_pate (cpu, cpu->env.spr[SPR_LPIDR ], &pate)) { +return 0 ; + } + base = pate.dw0; + }else { + base = cpu->env.spr[SPR_SDR1 ]; + } +return (1ULL << ((base &SDR_64_HTABSIZE ) +18 -7 )) -1 ; +} +#endif /* TARGET_PPC64 */ #endif /* CONFIG_USER_ONLY */
target/ppc/mmu-hash64.h +1 -18
@@ -63,6 +63,7 @@ void ppc_hash64_filter_pagesizes(PowerPCCPU *cpu,#define SDR_64_HTABORG 0x0FFFFFFFFFFC0000ULL#define SDR_64_HTABSIZE 0x000000000000001FULL +#define PATE0_HTABORG 0x0FFFFFFFFFFC0000ULL#define HPTES_PER_GROUP 8#define HASH_PTE_SIZE_64 16#define HASH_PTEG_SIZE_64 (HASH_PTE_SIZE_64 * HPTES_PER_GROUP)
@@ -107,24 +108,6 @@ void ppc_hash64_filter_pagesizes(PowerPCCPU *cpu,#define HPTE64_R_3_0_SSIZE_SHIFT 58#define HPTE64_R_3_0_SSIZE_MASK (3ULL << HPTE64_R_3_0_SSIZE_SHIFT) -static inline hwaddr ppc_hash64_hpt_base (PowerPCCPU *cpu) -{ -if (cpu->vhyp) { -return 0 ; - } -return cpu->env.spr[SPR_SDR1 ] &SDR_64_HTABORG ; -} - -static inline hwaddr ppc_hash64_hpt_mask (PowerPCCPU *cpu) -{ -if (cpu->vhyp) { -PPCVirtualHypervisorClass *vhc = -PPC_VIRTUAL_HYPERVISOR_GET_CLASS (cpu->vhyp); -return vhc->hpt_mask (cpu->vhyp); - } -return (1ULL << ((cpu->env.spr[SPR_SDR1 ] &SDR_64_HTABSIZE ) +18 -7 )) -1 ; -} -struct ppc_hash_pte64 {uint64_t pte0, pte1; };
target/ppc/mmu-book3s-v3.c +18 -0
@@ -39,3 +39,21 @@ hwaddr ppc64_v3_get_phys_page_debug(PowerPCCPU *cpu, vaddr eaddr)return ppc_hash64_get_phys_page_debug (cpu, eaddr); } } + +bool ppc64_v3_get_pate (PowerPCCPU *cpu,target_ulong lpid,ppc_v3_pate_t *entry) +{ +uint64_t patb = cpu->env.spr[SPR_PTCR ] &PTCR_PATB ; +uint64_t pats = cpu->env.spr[SPR_PTCR ] &PTCR_PATS ; + +/* Calculate number of entries */ + pats =1ull << (pats +12 -4 ); +if (pats <= lpid) { +return false; + } + +/* Grab entry */ + patb +=16 * lpid; + entry->dw0 =ldq_phys (CPU (cpu)->as, patb); + entry->dw1 =ldq_phys (CPU (cpu)->as, patb +8 ); +return true; +}
target/ppc/mmu-hash64.c +4 -2
@@ -416,7 +416,7 @@ const ppc_hash_pte64_t *ppc_hash64_map_hptes(PowerPCCPU *cpu, hwaddr ptex, int n) {hwaddr pte_offset = ptex *HASH_PTE_SIZE_64 ; -hwaddr base =ppc_hash64_hpt_base (cpu); +hwaddr base;hwaddr plen = n *HASH_PTE_SIZE_64 ;const ppc_hash_pte64_t *hptes;
@@ -425,6 +425,7 @@ const ppc_hash_pte64_t *ppc_hash64_map_hptes(PowerPCCPU *cpu,PPC_VIRTUAL_HYPERVISOR_GET_CLASS (cpu->vhyp);return vhc->map_hptes (cpu->vhyp, ptex, n); } + base =ppc_hash64_hpt_base (cpu);if (!base) {return NULL ;
@@ -928,7 +929,7 @@ hwaddr ppc_hash64_get_phys_page_debug(PowerPCCPU *cpu, target_ulong addr)void ppc_hash64_store_hpte (PowerPCCPU *cpu,hwaddr ptex,uint64_t pte0,uint64_t pte1) { -hwaddr base =ppc_hash64_hpt_base (cpu); +hwaddr base;hwaddr offset = ptex *HASH_PTE_SIZE_64 ;if (cpu->vhyp) {
@@ -937,6 +938,7 @@ void ppc_hash64_store_hpte(PowerPCCPU *cpu, hwaddr ptex, vhc->store_hpte (cpu->vhyp, ptex, pte0, pte1);return ; } + base =ppc_hash64_hpt_base (cpu);stq_phys (CPU (cpu)->as, base + offset, pte0);stq_phys (CPU (cpu)->as, base + offset +HASH_PTE_SIZE_64 /2 , pte1);
Signature · 1 line
2.20.1
PATCH 12/12 target/ppc: Basic POWER9 bare-metal radix MMU support
In reply to Cédric Le GoaterFrom: Benjamin Herrenschmidt <[email protected]>
No guest support yet
--- target/ppc/mmu-radix64.c | 81 ++++++++++++++++++++++++++++++++++------ 1 file changed, 69 insertions(+), 12 deletions(-)
target/ppc/mmu-radix64.c +69 -12
@@ -30,10 +30,26 @@static bool ppc_radix64_get_fully_qualified_addr (CPUPPCState *env,vaddr eaddr,uint64_t *lpid,uint64_t *pid) { -/* We don't have HV support yet and shouldn't get here with it set anyway */ -assert (!msr_hv); - -if (!msr_hv) {/* !MSR[HV] -> Guest */ +if (msr_hv) {/* MSR[HV] -> Hypervisor/bare metal */ +switch (eaddr &R_EADDR_QUADRANT ) { +case R_EADDR_QUADRANT0 : + *lpid =0 ; + *pid = env->spr[SPR_BOOKS_PID ]; +break ; +case R_EADDR_QUADRANT1 : + *lpid = env->spr[SPR_LPIDR ]; + *pid = env->spr[SPR_BOOKS_PID ]; +break ; +case R_EADDR_QUADRANT2 : + *lpid = env->spr[SPR_LPIDR ]; + *pid =0 ; +break ; +case R_EADDR_QUADRANT3 : + *lpid =0 ; + *pid =0 ; +break ; + } + }else {/* !MSR[HV] -> Guest */ switch (eaddr &R_EADDR_QUADRANT ) {case R_EADDR_QUADRANT0 :/* Guest application */ *lpid = env->spr[SPR_LPIDR ];
@@ -185,21 +201,32 @@ static uint64_t ppc_radix64_walk_tree(PowerPCCPU *cpu, vaddr eaddr, raddr, psize, fault_cause, pte_addr); } +static bool validate_pate (PowerPCCPU *cpu,uint64_t lpid,ppc_v3_pate_t *pate) +{ +CPUPPCState *env = &cpu->env; + +if (!(pate->dw0 &PATE0_HR )) { +return false; + } +if (lpid ==0 && !msr_hv) { +return false; + } +/* More checks ... */ +return true; +} +int ppc_radix64_handle_mmu_fault (PowerPCCPU *cpu,vaddr eaddr,int rwx,int mmu_idx) {CPUState *cs =CPU (cpu);CPUPPCState *env = &cpu->env; -PPCVirtualHypervisorClass *vhc = -PPC_VIRTUAL_HYPERVISOR_GET_CLASS (cpu->vhyp); +PPCVirtualHypervisorClass *vhc;hwaddr raddr, pte_addr;uint64_t lpid =0 , pid =0 , offset, size, prtbe0, pte;int page_size, prot, fault_cause =0 ;ppc_v3_pate_t pate;assert ((rwx ==0 ) || (rwx ==1 ) || (rwx ==2 )); -assert (!msr_hv);/* For now there is no Radix PowerNV Support */ -assert (cpu->vhyp);assert (ppc64_use_proc_tbl (cpu));/* Real Mode Access */
@@ -220,7 +247,23 @@ int ppc_radix64_handle_mmu_fault(PowerPCCPU *cpu, vaddr eaddr, int rwx, }/* Get Process Table */ - vhc->get_pate (cpu->vhyp, &pate); +if (cpu->vhyp) { + vhc =PPC_VIRTUAL_HYPERVISOR_GET_CLASS (cpu->vhyp); + vhc->get_pate (cpu->vhyp, &pate); + }else { +if (!ppc64_v3_get_pate (cpu, lpid, &pate)) { +ppc_radix64_raise_si (cpu, rwx, eaddr,DSISR_NOPTE ); +return 1 ; + } +if (!validate_pate (cpu, lpid, &pate)) { +ppc_radix64_raise_si (cpu, rwx, eaddr,DSISR_R_BADCONFIG ); + } +/* We don't support guest mode yet */ +if (lpid !=0 ) { +error_report ("PowerNV guest support Unimplemented" ); +exit (1 ); + } + }/* Index Process Table by PID to Find Corresponding Process Table Entry */ offset = pid *sizeof (struct prtb_entry );
@@ -255,8 +298,7 @@ hwaddr ppc_radix64_get_phys_page_debug(PowerPCCPU *cpu, target_ulong eaddr) {CPUState *cs =CPU (cpu);CPUPPCState *env = &cpu->env; -PPCVirtualHypervisorClass *vhc = -PPC_VIRTUAL_HYPERVISOR_GET_CLASS (cpu->vhyp); +PPCVirtualHypervisorClass *vhc;hwaddr raddr, pte_addr;uint64_t lpid =0 , pid =0 , offset, size, prtbe0, pte;int page_size, fault_cause =0 ;
@@ -274,7 +316,22 @@ hwaddr ppc_radix64_get_phys_page_debug(PowerPCCPU *cpu, target_ulong eaddr) }/* Get Process Table */ - vhc->get_pate (cpu->vhyp, &pate); +if (cpu->vhyp) { + vhc =PPC_VIRTUAL_HYPERVISOR_GET_CLASS (cpu->vhyp); + vhc->get_pate (cpu->vhyp, &pate); + }else { +if (!ppc64_v3_get_pate (cpu, lpid, &pate)) { +return -1 ; + } +if (!validate_pate (cpu, lpid, &pate)) { +return -1 ; + } +/* We don't support guest mode yet */ +if (lpid !=0 ) { +error_report ("PowerNV guest support Unimplemented" ); +exit (1 ); + } + }/* Index Process Table by PID to Find Corresponding Process Table Entry */ offset = pid *sizeof (struct prtb_entry );
Signature · 1 line
2.20.1
Quoted 7 lines · Cédric Le Goater
Hello,
Here is another series of fixes and extensions from Ben providing support for POWER9 native hash MMU and POWER9 native radix MMU. These prepare ground for the support of QEMU POWER9 PowerNV machines.
Quoted 1 line
From there, I hand over the patchset to Suraj for any follow-ups.
I've now merged all of these except the include cleanup, which I've partly done in a different way. A couple of the later patches are tweaked so that doesn't break compile.
Quoted 38 lines
Thanks,
C.
Benjamin Herrenschmidt (12):
target/ppc/spapr: Set LPCR:HR when using Radix mode target/ppci/mmu: Use LPCR:HR to chose radix vs. hash translation target/ppc: Re-enable RMLS on POWER9 for virtual hypervisors target/ppc: Fix #include guard inmmu-book3s-v3.htarget/ppc: Cleanup 64-bit MMU includes
target/ppc: Fix ordering of hash MMU accesses
target/ppc: Add basic support for "new format" HPTE as found on POWER9 target/ppc: Fix synchronization of mttcg with broadcast TLB flushes target/ppc: Flush the TLB locally when the LPIDR is written target/ppc: Rename PATB/PATBE -> PATE
target/ppc: Support for POWER9 native hash
target/ppc: Basic POWER9 bare-metal radix MMU supportinclude/hw/ppc/spapr.h | 1 + target/ppc/cpu.h | 7 ++- target/ppc/helper.h | 1 + target/ppc/helper_regs.h | 27 ++++------ target/ppc/mmu-book3s-v3.h | 89 +++++++++++++++++++++++++++--- target/ppc/mmu-hash64.h | 22 ++------ target/ppc/mmu-radix64.h | 4 +- hw/ppc/spapr.c | 86 ++++++++++++++++++++++++----- hw/ppc/spapr_hcall.c | 69 +++++++----------------- hw/ppc/spapr_rtas.c | 6 +-- target/ppc/misc_helper.c | 15 ++++++ target/ppc/mmu-book3s-v3.c | 31 +++++++++-- target/ppc/mmu-hash32.c | 6 +++ target/ppc/mmu-hash64.c | 28 ++++++++-- target/ppc/mmu-radix64.c | 96 ++++++++++++++++++++++++++------- target/ppc/mmu_helper.c | 9 +--- target/ppc/translate_init.inc.c | 7 ++- 17 files changed, 361 insertions(+), 143 deletions(-)
Signature · 4 lines
David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson