Skip to contentSkip to conversation

spapr: implement dispatch and suspend calls

PATCH Superseded Nicholas Piggin 3 participants 19 messages Jul 16, 2019 – Jul 17, 2019

v4 of 6 Superseded Latest is v6 →

lore
Share & tools
lore
#0 lore raw

v4 cover spapr: implement dispatch and suspend calls

This series follows on from the previous that added H_PROD and H_CONFER, but I've now aimed to make it conform better to PAPR.

It's still not completely there (as explained in comments), but it's better than before and actually better matches KVM that does implement the prod bit and dispatch counter.

The first 3 patches implement these splpar hcalls for tcg, as KVM implements its own H_PROD, H_CONFER, H_CEDE, and dispatch management. These would be nice to merge as they make qemu behave more like KVM and PowerVM with these calls.

The last 2 patches implement some parts of the guest suspend APIs I've been using to test Linux modifications to the pseries (and generic kernel) suspend/hibernate code, but they are not really useful to a Linux guest (yet) due to other missing bits.

I have some Linux code I'll try to gradually upstream to work around the missing bits and make this suspend on QEMU "work", which is at least useful for testing without having PowerVM.

Thanks,
Nick

Nicholas Piggin (5):
spapr: Implement dispatch counter and prod bit on tcg
spapr: Implement H_PROD
spapr: Implement H_CONFER
spapr: Implement H_JOIN
spapr: Implement ibm,suspend-me

 hw/ppc/spapr.c                  |  52 +++++++++++++
 hw/ppc/spapr_cpu_core.c         |   5 +-
 hw/ppc/spapr_hcall.c            | 126 ++++++++++++++++++++++++++++++--
 hw/ppc/spapr_rtas.c             |  32 ++++++++
 include/hw/ppc/spapr.h          |  14 +++-
 include/hw/ppc/spapr_cpu_core.h |   2 +
 target/ppc/cpu.h                |   2 +
 target/ppc/translate_init.inc.c |  25 +++++++
 8 files changed, 251 insertions(+), 7 deletions(-)
Signature · 1 line
2.20.1
#6 lore raw

PATCH v4 2/5 spapr: Implement H_PROD

H_PROD is added, and H_CEDE is modified to test the prod bit according to PAPR.

---
 hw/ppc/spapr_hcall.c | 29 +++++++++++++++++++++++++++++
 1 file changed, 29 insertions(+)
hw/ppc/spapr_hcall.c +29 -0
index e615881ac4..8b208ab259 100644
@@ -1050,14 +1050,41 @@ static target_ulong h_cede(PowerPCCPU *cpu, SpaprMachineState *spapr,
 {
     CPUPPCState *env = &cpu->env;
     CPUState *cs = CPU(cpu);
+    SpaprCpuState *spapr_cpu = spapr_cpu_state(cpu);
 
     env->msr |= (1ULL << MSR_EE);
     hreg_compute_hflags(env);
+
+    if (spapr_cpu->prod) {
+        spapr_cpu->prod = false;
+        return H_SUCCESS;
+    }
+
     if (!cpu_has_work(cs)) {
         cs->halted = 1;
         cs->exception_index = EXCP_HLT;
         cs->exit_request = 1;
     }
+
+    return H_SUCCESS;
+}
+
+static target_ulong h_prod(PowerPCCPU *cpu, SpaprMachineState *spapr,
+                           target_ulong opcode, target_ulong *args)
+{
+    target_long target = args[0];
+    CPUState *cs;
+    SpaprCpuState *spapr_cpu = spapr_cpu_state(cpu);
+
+    cs = CPU(spapr_find_cpu(target));
+    if (!cs) {
+        return H_PARAMETER;
+    }
+
+    spapr_cpu->prod = true;
+    cs->halted = 0;
+    qemu_cpu_kick(cs);
+
     return H_SUCCESS;
 }
 
@@ -1882,6 +1909,8 @@ static void hypercall_register_types(void)
     /* hcall-splpar */
     spapr_register_hypercall(H_REGISTER_VPA, h_register_vpa);
     spapr_register_hypercall(H_CEDE, h_cede);
+    spapr_register_hypercall(H_PROD, h_prod);
+
     spapr_register_hypercall(H_SIGNAL_SYS_RESET, h_signal_sys_reset);
 
     /* processor register resource access h-calls */
Signature · 1 line
2.20.1
#1 lore raw

PATCH v4 1/5 spapr: Implement dispatch counter and prod bit on tcg

In reply to Nicholas Piggin

Implement cpu_exec_enter/exit on ppc which calls into new methods of the same name in PPCVirtualHypervisorClass. These are used by spapr to implement these splpar elements, used in subsequent changes.

---
 hw/ppc/spapr.c                  | 25 +++++++++++++++++++++++++
 hw/ppc/spapr_cpu_core.c         |  5 ++++-
 hw/ppc/spapr_hcall.c            |  5 -----
 include/hw/ppc/spapr.h          |  7 +++++++
 include/hw/ppc/spapr_cpu_core.h |  2 ++
 target/ppc/cpu.h                |  2 ++
 target/ppc/translate_init.inc.c | 25 +++++++++++++++++++++++++
 7 files changed, 65 insertions(+), 6 deletions(-)
hw/ppc/spapr.c +25 -0
index 821f0d4a49..87b11e2484 100644
@@ -4302,6 +4302,29 @@ PowerPCCPU *spapr_find_cpu(int vcpu_id)
     return NULL;
 }
 
+static void spapr_cpu_exec_enter(PPCVirtualHypervisor *vhyp, PowerPCCPU *cpu)
+{
+    SpaprCpuState *spapr_cpu = spapr_cpu_state(cpu);
+
+    /* These are only called by TCG, KVM maintains dispatch state */
+
+    spapr_cpu->prod = false;
+    spapr_cpu->dispatch_counter++;
+    assert((spapr_cpu->dispatch_counter & 1) == 0);
+    if (spapr_cpu->vpa_addr) {
+        CPUState *cs = CPU(cpu);
+        stl_be_phys(cs->as, spapr_cpu->vpa_addr + VPA_DISPATCH_COUNTER, spapr_cpu->dispatch_counter);
+    }
+}
+
+static void spapr_cpu_exec_exit(PPCVirtualHypervisor *vhyp, PowerPCCPU *cpu)
+{
+    SpaprCpuState *spapr_cpu = spapr_cpu_state(cpu);
+
+    spapr_cpu->dispatch_counter++;
+    assert((spapr_cpu->dispatch_counter & 1) == 1);
+}
+
 static void spapr_machine_class_init(ObjectClass *oc, void *data)
 {
     MachineClass *mc = MACHINE_CLASS(oc);
@@ -4358,6 +4381,8 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)
     vhc->hpte_set_r = spapr_hpte_set_r;
     vhc->get_pate = spapr_get_pate;
     vhc->encode_hpt_for_kvm_pr = spapr_encode_hpt_for_kvm_pr;
+    vhc->cpu_exec_enter = spapr_cpu_exec_enter;
+    vhc->cpu_exec_exit = spapr_cpu_exec_exit;
     xic->ics_get = spapr_ics_get;
     xic->ics_resend = spapr_ics_resend;
     xic->icp_get = spapr_icp_get;
hw/ppc/spapr_cpu_core.c +4 -1
index 5621fb9a3d..fb2ed9e95d 100644
@@ -261,6 +261,7 @@ error:
 static PowerPCCPU *spapr_create_vcpu(SpaprCpuCore *sc, int i, Error **errp)
 {
     SpaprCpuCoreClass *scc = SPAPR_CPU_CORE_GET_CLASS(sc);
+    SpaprCpuState *spapr_cpu;
     CPUCore *cc = CPU_CORE(sc);
     Object *obj;
     char *id;
@@ -287,7 +288,9 @@ static PowerPCCPU *spapr_create_vcpu(SpaprCpuCore *sc, int i, Error **errp)
         goto err;
     }
 
-    cpu->machine_data = g_new0(SpaprCpuState, 1);
+    spapr_cpu = g_new0(SpaprCpuState, 1);
+    spapr_cpu->dispatch_counter = 1;
+    cpu->machine_data = spapr_cpu;
 
     object_unref(obj);
     return cpu;
hw/ppc/spapr_hcall.c +0 -5
index 6808d4cda8..e615881ac4 100644
@@ -874,11 +874,6 @@ unmap_out:
 #define FLAGS_DEREGISTER_DTL       0x0000c00000000000ULL
 #define FLAGS_DEREGISTER_SLBSHADOW 0x0000e00000000000ULL
 
-#define VPA_MIN_SIZE           640
-#define VPA_SIZE_OFFSET        0x4
-#define VPA_SHARED_PROC_OFFSET 0x9
-#define VPA_SHARED_PROC_VAL    0x2
-
 static target_ulong register_vpa(PowerPCCPU *cpu, target_ulong vpa)
 {
     CPUState *cs = CPU(cpu);
include/hw/ppc/spapr.h +7 -0
index 60553d32c4..5d36eec9d0 100644
@@ -525,6 +525,13 @@ void spapr_register_hypercall(target_ulong opcode, spapr_hcall_fn fn);
 target_ulong spapr_hypercall(PowerPCCPU *cpu, target_ulong opcode,
                              target_ulong *args);
 
+/* Virtual Processor Area structure constants */
+#define VPA_MIN_SIZE           640
+#define VPA_SIZE_OFFSET        0x4
+#define VPA_SHARED_PROC_OFFSET 0x9
+#define VPA_SHARED_PROC_VAL    0x2
+#define VPA_DISPATCH_COUNTER   0x100
+
 /* ibm,set-eeh-option */
 #define RTAS_EEH_DISABLE                 0
 #define RTAS_EEH_ENABLE                  1
include/hw/ppc/spapr_cpu_core.h +2 -0
index f9645a7290..3032dfa7ee 100644
@@ -46,6 +46,8 @@ typedef struct SpaprCpuState {
     uint64_t vpa_addr;
     uint64_t slb_shadow_addr, slb_shadow_size;
     uint64_t dtl_addr, dtl_size;
+    uint32_t dispatch_counter;
+    bool prod;
     struct ICPState *icp;
     struct XiveTCTX *tctx;
 } SpaprCpuState;
target/ppc/cpu.h +2 -0
index c9beba2a5c..78d6504acb 100644
@@ -1224,6 +1224,8 @@ struct PPCVirtualHypervisorClass {
     void (*hpte_set_r)(PPCVirtualHypervisor *vhyp, hwaddr ptex, uint64_t pte1);
     void (*get_pate)(PPCVirtualHypervisor *vhyp, ppc_v3_pate_t *entry);
     target_ulong (*encode_hpt_for_kvm_pr)(PPCVirtualHypervisor *vhyp);
+    void (*cpu_exec_enter)(PPCVirtualHypervisor *vhyp, PowerPCCPU *cpu);
+    void (*cpu_exec_exit)(PPCVirtualHypervisor *vhyp, PowerPCCPU *cpu);
 };
 
 #define TYPE_PPC_VIRTUAL_HYPERVISOR "ppc-virtual-hypervisor"
target/ppc/translate_init.inc.c +25 -0
index 86fc8f2e31..58d4a93b23 100644
@@ -10473,6 +10473,28 @@ static bool ppc_cpu_is_big_endian(CPUState *cs)
 }
 #endif
 
+static void ppc_cpu_exec_enter(CPUState *cs)
+{
+    PowerPCCPU *cpu = POWERPC_CPU(cs);
+
+    if (cpu->vhyp) {
+        PPCVirtualHypervisorClass *vhc =
+            PPC_VIRTUAL_HYPERVISOR_GET_CLASS(cpu->vhyp);
+        vhc->cpu_exec_enter(cpu->vhyp, cpu);
+    }
+}
+
+static void ppc_cpu_exec_exit(CPUState *cs)
+{
+    PowerPCCPU *cpu = POWERPC_CPU(cs);
+
+    if (cpu->vhyp) {
+        PPCVirtualHypervisorClass *vhc =
+            PPC_VIRTUAL_HYPERVISOR_GET_CLASS(cpu->vhyp);
+        vhc->cpu_exec_exit(cpu->vhyp, cpu);
+    }
+}
+
 static void ppc_cpu_instance_init(Object *obj)
 {
     PowerPCCPU *cpu = POWERPC_CPU(obj);
@@ -10624,6 +10646,9 @@ static void ppc_cpu_class_init(ObjectClass *oc, void *data)
     cc->tcg_initialize = ppc_translate_init;
     cc->tlb_fill = ppc_cpu_tlb_fill;
 #endif
+    cc->cpu_exec_enter = ppc_cpu_exec_enter;
+    cc->cpu_exec_exit = ppc_cpu_exec_exit;
+
     cc->disas_set_info = ppc_disas_set_info;
 
     dc->fw_name = "PowerPC,UNKNOWN";
Signature · 1 line
2.20.1
#14 lore raw

PATCH v4 5/5 spapr: Implement ibm,suspend-me

In reply to Nicholas Piggin

This has been useful to modify and test the Linux pseries suspend code but it requires modification to the guest to call it (due to being gated by other unimplemented features). It is not otherwise used by Linux yet, but work is slowly progressing there.

This allows a (lightly modified) guest kernel to suspend with echo mem > /sys/power/state and be resumed with system_wakeup monitor command.

---
 hw/ppc/spapr.c         | 26 ++++++++++++++++++++++++++
 hw/ppc/spapr_rtas.c    | 32 ++++++++++++++++++++++++++++++++
 include/hw/ppc/spapr.h |  7 ++++++-
 3 files changed, 64 insertions(+), 1 deletion(-)
hw/ppc/spapr.c +26 -0
index 5c54e1cb9a..b85d41bb1e 100644
@@ -1710,6 +1710,11 @@ static void spapr_machine_reset(MachineState *machine)
     void *fdt;
     int rc;
 
+    if (spapr->suspend_reset) {
+        spapr->suspend_reset = false;
+        return;
+    }
+
     spapr_caps_apply(spapr);
 
     first_ppc_cpu = POWERPC_CPU(first_cpu);
@@ -2721,6 +2726,23 @@ static PCIHostState *spapr_create_default_phb(void)
     return PCI_HOST_BRIDGE(dev);
 }
 
+static Notifier wakeup;
+static void spapr_notify_wakeup(Notifier *notifier, void *data)
+{
+    WakeupReason *reason = data;
+
+    switch (*reason) {
+    case QEMU_WAKEUP_REASON_RTC:
+        break;
+    case QEMU_WAKEUP_REASON_PMTIMER:
+        break;
+    case QEMU_WAKEUP_REASON_OTHER:
+        break;
+    default:
+        break;
+    }
+}
+
 /* pSeries LPAR / sPAPR hardware init */
 static void spapr_machine_init(MachineState *machine)
 {
@@ -3078,6 +3100,10 @@ static void spapr_machine_init(MachineState *machine)
 
     qemu_register_boot_set(spapr_boot_set, spapr);
 
+    wakeup.notify = spapr_notify_wakeup;
+    qemu_register_wakeup_notifier(&wakeup);
+    qemu_register_wakeup_support();
+
     if (kvm_enabled()) {
         /* to stop and start vmclock */
         qemu_add_vm_change_state_handler(cpu_ppc_clock_vm_state_change,
hw/ppc/spapr_rtas.c +32 -0
index a618a2ac0f..60a007ec38 100644
@@ -216,6 +216,36 @@ static void rtas_stop_self(PowerPCCPU *cpu, SpaprMachineState *spapr,
     qemu_cpu_kick(cs);
 }
 
+static void rtas_ibm_suspend_me(PowerPCCPU *cpu, SpaprMachineState *spapr,
+                           uint32_t token, uint32_t nargs,
+                           target_ulong args,
+                           uint32_t nret, target_ulong rets)
+{
+    CPUState *cs;
+
+    if (nargs != 0 || nret != 1) {
+        rtas_st(rets, 0, RTAS_OUT_PARAM_ERROR);
+        return;
+    }
+
+    CPU_FOREACH(cs) {
+        PowerPCCPU *c = POWERPC_CPU(cs);
+        CPUPPCState *e = &c->env;
+        if (c == cpu)
+            continue;
+
+	/* See h_join */
+        if (!cs->halted || (e->msr & (1ULL << MSR_EE))) {
+            rtas_st(rets, 0, H_MULTI_THREADS_ACTIVE);
+            return;
+        }
+    }
+
+    spapr->suspend_reset = true;
+    qemu_system_suspend_request();
+    rtas_st(rets, 0, RTAS_OUT_SUCCESS);
+}
+
 static inline int sysparm_st(target_ulong addr, target_ulong len,
                              const void *val, uint16_t vallen)
 {
@@ -483,6 +513,8 @@ static void core_rtas_register_types(void)
                         rtas_query_cpu_stopped_state);
     spapr_rtas_register(RTAS_START_CPU, "start-cpu", rtas_start_cpu);
     spapr_rtas_register(RTAS_STOP_SELF, "stop-self", rtas_stop_self);
+    spapr_rtas_register(RTAS_IBM_SUSPEND_ME, "ibm,suspend-me",
+                        rtas_ibm_suspend_me);
     spapr_rtas_register(RTAS_IBM_GET_SYSTEM_PARAMETER,
                         "ibm,get-system-parameter",
                         rtas_ibm_get_system_parameter);
include/hw/ppc/spapr.h +6 -1
index 5d36eec9d0..df0b0c15da 100644
@@ -171,6 +171,10 @@ struct SpaprMachineState {
     bool use_hotplug_event_source;
     SpaprEventSource *event_sources;
 
+    /* Machine has been suspended, so the next machine_reset should not
+     * reset state, but just return and allow execution to resume. */
+    bool suspend_reset;
+
     /* ibm,client-architecture-support option negotiation */
     bool cas_reboot;
     bool cas_legacy_guest_workaround;
@@ -631,8 +635,9 @@ target_ulong spapr_hypercall(PowerPCCPU *cpu, target_ulong opcode,
 #define RTAS_IBM_CREATE_PE_DMA_WINDOW           (RTAS_TOKEN_BASE + 0x27)
 #define RTAS_IBM_REMOVE_PE_DMA_WINDOW           (RTAS_TOKEN_BASE + 0x28)
 #define RTAS_IBM_RESET_PE_DMA_WINDOW            (RTAS_TOKEN_BASE + 0x29)
+#define RTAS_IBM_SUSPEND_ME                     (RTAS_TOKEN_BASE + 0x2A)
 
-#define RTAS_TOKEN_MAX                          (RTAS_TOKEN_BASE + 0x2A)
+#define RTAS_TOKEN_MAX                          (RTAS_TOKEN_BASE + 0x2B)
 
 /* RTAS ibm,get-system-parameter token values */
 #define RTAS_SYSPARM_SPLPAR_CHARACTERISTICS      20
Signature · 1 line
2.20.1
#12 lore raw

PATCH v4 4/5 spapr: Implement H_JOIN

In reply to Nicholas Piggin

This has been useful to modify and test the Linux pseries suspend code but it requires modification to the guest to call it (due to being gated by other unimplemented features). It is not otherwise used by Linux yet, but work is slowly progressing there.

---
 hw/ppc/spapr.c       |  1 +
 hw/ppc/spapr_hcall.c | 44 ++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 45 insertions(+)
hw/ppc/spapr.c +1 -0
index 87b11e2484..5c54e1cb9a 100644
@@ -1066,6 +1066,7 @@ static void spapr_dt_rtas(SpaprMachineState *spapr, void *fdt)
     add_str(hypertas, "hcall-tce");
     add_str(hypertas, "hcall-vio");
     add_str(hypertas, "hcall-splpar");
+    add_str(hypertas, "hcall-join");
     add_str(hypertas, "hcall-bulk");
     add_str(hypertas, "hcall-set-mode");
     add_str(hypertas, "hcall-sprg0");
hw/ppc/spapr_hcall.c +44 -0

56 lines not on this page · raw message

Signature · 1 line
2.20.1
#8 lore raw

PATCH v4 3/5 spapr: Implement H_CONFER

In reply to Nicholas Piggin

This does not do directed yielding and is not quite as strict as PAPR specifies in terms of precise dispatch behaviour. This generally will mean suboptimal performance, rather than guest misbehaviour. Linux does not rely on exact dispatch behaviour.

---
 hw/ppc/spapr_hcall.c | 48 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 48 insertions(+)
hw/ppc/spapr_hcall.c +48 -0
index 8b208ab259..28d58113be 100644
@@ -1069,6 +1069,53 @@ static target_ulong h_cede(PowerPCCPU *cpu, SpaprMachineState *spapr,
     return H_SUCCESS;
 }
 
+static target_ulong h_confer(PowerPCCPU *cpu, SpaprMachineState *spapr,
+                           target_ulong opcode, target_ulong *args)
+{
+    target_long target = args[0];
+    uint32_t dispatch = args[1];
+    PowerPCCPU *target_cpu = spapr_find_cpu(target);
+    CPUState *target_cs = CPU(target_cpu);
+    CPUState *cs = CPU(cpu);
+    SpaprCpuState *spapr_cpu;
+
+    /*
+     * This does not do a targeted yield or confer, but check the parameter
+     * anyway. -1 means confer to all/any other CPUs.
+     */
+    if (target != -1 && !target_cs) {
+        return H_PARAMETER;
+    }
+
+    spapr_cpu = spapr_cpu_state(target_cpu);
+
+    /*
+     * PAPR specifies waiting until proded in this case, without dispatch
+     * counter check.
+     */
+    if (cpu == target_cpu) {
+        if (spapr_cpu->prod) {
+            spapr_cpu->prod = false;
+            return H_SUCCESS;
+        }
+
+        cs->halted = 1;
+        cs->exception_index = EXCP_HALTED;
+        cs->exit_request = 1;
+
+        return H_SUCCESS;
+    }
+
+    if (spapr_cpu->dispatch_counter != dispatch || (dispatch & 1) == 0) {
+        return H_SUCCESS;
+    }
+
+    cs->exception_index = EXCP_YIELD;
+    cpu_loop_exit(cs);
+
+    return H_SUCCESS;
+}
+
 static target_ulong h_prod(PowerPCCPU *cpu, SpaprMachineState *spapr,
                            target_ulong opcode, target_ulong *args)
 {
@@ -1909,6 +1956,7 @@ static void hypercall_register_types(void)
     /* hcall-splpar */
     spapr_register_hypercall(H_REGISTER_VPA, h_register_vpa);
     spapr_register_hypercall(H_CEDE, h_cede);
+    spapr_register_hypercall(H_CONFER, h_confer);
     spapr_register_hypercall(H_PROD, h_prod);
 
     spapr_register_hypercall(H_SIGNAL_SYS_RESET, h_signal_sys_reset);
Signature · 1 line
2.20.1
#18 lore raw
In reply to Nicholas Piggin
Patchew URL: https://patchew.org/QEMU/[email protected]/

Patchew URL: https://patchew.org/QEMU/[email protected]/

Hi,

This series seems to have some coding style problems. See output below for more information:

Subject: [Qemu-devel] [PATCH v4 0/5] spapr: implement dispatch and suspend calls
Message-id: [email protected]
Type: series

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

Switched to a new branch 'test'
c432dea spapr: Implement ibm,suspend-me
09bae6b spapr: Implement H_JOIN
031bd3f spapr: Implement H_CONFER
a3d8bc2 spapr: Implement H_PROD
ed23e67 spapr: Implement dispatch counter and prod bit on tcg

=== OUTPUT BEGIN ===
1/5 Checking commit ed23e6798f18 (spapr: Implement dispatch counter and prod bit on tcg)
ERROR: line over 90 characters
#33: FILE: hw/ppc/spapr.c:4316:
+ stl_be_phys(cs->as, spapr_cpu->vpa_addr + VPA_DISPATCH_COUNTER, spapr_cpu->dispatch_counter);

total: 1 errors, 0 warnings, 131 lines checked

Patch 1/5 has style problems, please review. If any of these errors are false positives report them to the maintainer, see CHECKPATCH in MAINTAINERS.

2/5 Checking commit a3d8bc2510cc (spapr: Implement H_PROD) 3/5 Checking commit 031bd3f6986b (spapr: Implement H_CONFER) 4/5 Checking commit 09bae6bec0c1 (spapr: Implement H_JOIN)
ERROR: braces {} are necessary for all arms of this statement #55: FILE: hw/ppc/spapr_hcall.c:1092:
+ if (c == cpu)
[...]

ERROR: code indent should never use tabs
#58: FILE: hw/ppc/spapr_hcall.c:1095:
+^I/* Don't have a way to indicate joined, so use halted && MSR[EE]=0 */$

total: 2 errors, 0 warnings, 63 lines checked

Patch 4/5 has style problems, please review. If any of these errors are false positives report them to the maintainer, see CHECKPATCH in MAINTAINERS.

5/5 Checking commit c432deae5480 (spapr: Implement ibm,suspend-me)
ERROR: braces {} are necessary for all arms of this statement #93: FILE: hw/ppc/spapr_rtas.c:234:
+ if (c == cpu)
[...]

ERROR: code indent should never use tabs
#96: FILE: hw/ppc/spapr_rtas.c:237:
+^I/* See h_join */$

WARNING: Block comments use a leading /* on a separate line #128: FILE: include/hw/ppc/spapr.h:174:
+ /* Machine has been suspended, so the next machine_reset should not

WARNING: Block comments use a trailing */ on a separate line #129: FILE: include/hw/ppc/spapr.h:175:
+ * reset state, but just return and allow execution to resume. */

total: 2 errors, 2 warnings, 108 lines checked

Patch 5/5 has style problems, please review. If any of these errors are false positives report them to the maintainer, see CHECKPATCH in MAINTAINERS.

=== OUTPUT END ===

Test command exited with code: 1

The full log is available at
http://patchew.org/logs/[email protected]/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to [email protected]

#2 lore raw

Re: [Qemu-devel] [PATCH v4 1/5] spapr: Implement dispatch counter and prod bit on tcg

In reply to Nicholas Piggin
Quoted 5 lines · Nicholas Piggin

Implement cpu_exec_enter/exit on ppc which calls into new methods of the same name in PPCVirtualHypervisorClass. These are used by spapr to implement these splpar elements, used in subsequent changes.

Quoted 9 lines
---
 hw/ppc/spapr.c                  | 25 +++++++++++++++++++++++++
 hw/ppc/spapr_cpu_core.c         |  5 ++++-
 hw/ppc/spapr_hcall.c            |  5 -----
 include/hw/ppc/spapr.h          |  7 +++++++
 include/hw/ppc/spapr_cpu_core.h |  2 ++
 target/ppc/cpu.h                |  2 ++
 target/ppc/translate_init.inc.c | 25 +++++++++++++++++++++++++
 7 files changed, 65 insertions(+), 6 deletions(-)

Mostly LGTM. Please do address the style issues that the bot reported.

In addition, do the dispatch_counter and prod values need to be migrated?

Quoted 170 lines
hw/ppc/spapr.c +25 -0
index 821f0d4a49..87b11e2484 100644
@@ -4302,6 +4302,29 @@ PowerPCCPU *spapr_find_cpu(int vcpu_id)
     return NULL;
 }
 
+static void spapr_cpu_exec_enter(PPCVirtualHypervisor *vhyp, PowerPCCPU *cpu)
+{
+    SpaprCpuState *spapr_cpu = spapr_cpu_state(cpu);
+
+    /* These are only called by TCG, KVM maintains dispatch state */
+
+    spapr_cpu->prod = false;
+    spapr_cpu->dispatch_counter++;
+    assert((spapr_cpu->dispatch_counter & 1) == 0);
+    if (spapr_cpu->vpa_addr) {
+        CPUState *cs = CPU(cpu);
+        stl_be_phys(cs->as, spapr_cpu->vpa_addr + VPA_DISPATCH_COUNTER, spapr_cpu->dispatch_counter);
+    }
+}
+
+static void spapr_cpu_exec_exit(PPCVirtualHypervisor *vhyp, PowerPCCPU *cpu)
+{
+    SpaprCpuState *spapr_cpu = spapr_cpu_state(cpu);
+
+    spapr_cpu->dispatch_counter++;
+    assert((spapr_cpu->dispatch_counter & 1) == 1);
+}
+
 static void spapr_machine_class_init(ObjectClass *oc, void *data)
 {
     MachineClass *mc = MACHINE_CLASS(oc);
@@ -4358,6 +4381,8 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)
     vhc->hpte_set_r = spapr_hpte_set_r;
     vhc->get_pate = spapr_get_pate;
     vhc->encode_hpt_for_kvm_pr = spapr_encode_hpt_for_kvm_pr;
+    vhc->cpu_exec_enter = spapr_cpu_exec_enter;
+    vhc->cpu_exec_exit = spapr_cpu_exec_exit;
     xic->ics_get = spapr_ics_get;
     xic->ics_resend = spapr_ics_resend;
     xic->icp_get = spapr_icp_get;
hw/ppc/spapr_cpu_core.c +4 -1
index 5621fb9a3d..fb2ed9e95d 100644
@@ -261,6 +261,7 @@ error:
 static PowerPCCPU *spapr_create_vcpu(SpaprCpuCore *sc, int i, Error **errp)
 {
     SpaprCpuCoreClass *scc = SPAPR_CPU_CORE_GET_CLASS(sc);
+    SpaprCpuState *spapr_cpu;
     CPUCore *cc = CPU_CORE(sc);
     Object *obj;
     char *id;
@@ -287,7 +288,9 @@ static PowerPCCPU *spapr_create_vcpu(SpaprCpuCore *sc, int i, Error **errp)
         goto err;
     }
 
-    cpu->machine_data = g_new0(SpaprCpuState, 1);
+    spapr_cpu = g_new0(SpaprCpuState, 1);
+    spapr_cpu->dispatch_counter = 1;
+    cpu->machine_data = spapr_cpu;
 
     object_unref(obj);
     return cpu;
hw/ppc/spapr_hcall.c +0 -5
index 6808d4cda8..e615881ac4 100644
@@ -874,11 +874,6 @@ unmap_out:
 #define FLAGS_DEREGISTER_DTL       0x0000c00000000000ULL
 #define FLAGS_DEREGISTER_SLBSHADOW 0x0000e00000000000ULL
 
-#define VPA_MIN_SIZE           640
-#define VPA_SIZE_OFFSET        0x4
-#define VPA_SHARED_PROC_OFFSET 0x9
-#define VPA_SHARED_PROC_VAL    0x2
-
 static target_ulong register_vpa(PowerPCCPU *cpu, target_ulong vpa)
 {
     CPUState *cs = CPU(cpu);
include/hw/ppc/spapr.h +7 -0
index 60553d32c4..5d36eec9d0 100644
@@ -525,6 +525,13 @@ void spapr_register_hypercall(target_ulong opcode, spapr_hcall_fn fn);
 target_ulong spapr_hypercall(PowerPCCPU *cpu, target_ulong opcode,
                              target_ulong *args);
 
+/* Virtual Processor Area structure constants */
+#define VPA_MIN_SIZE           640
+#define VPA_SIZE_OFFSET        0x4
+#define VPA_SHARED_PROC_OFFSET 0x9
+#define VPA_SHARED_PROC_VAL    0x2
+#define VPA_DISPATCH_COUNTER   0x100
+
 /* ibm,set-eeh-option */
 #define RTAS_EEH_DISABLE                 0
 #define RTAS_EEH_ENABLE                  1
include/hw/ppc/spapr_cpu_core.h +2 -0
index f9645a7290..3032dfa7ee 100644
@@ -46,6 +46,8 @@ typedef struct SpaprCpuState {
     uint64_t vpa_addr;
     uint64_t slb_shadow_addr, slb_shadow_size;
     uint64_t dtl_addr, dtl_size;
+    uint32_t dispatch_counter;
+    bool prod;
     struct ICPState *icp;
     struct XiveTCTX *tctx;
 } SpaprCpuState;
target/ppc/cpu.h +2 -0
index c9beba2a5c..78d6504acb 100644
@@ -1224,6 +1224,8 @@ struct PPCVirtualHypervisorClass {
     void (*hpte_set_r)(PPCVirtualHypervisor *vhyp, hwaddr ptex, uint64_t pte1);
     void (*get_pate)(PPCVirtualHypervisor *vhyp, ppc_v3_pate_t *entry);
     target_ulong (*encode_hpt_for_kvm_pr)(PPCVirtualHypervisor *vhyp);
+    void (*cpu_exec_enter)(PPCVirtualHypervisor *vhyp, PowerPCCPU *cpu);
+    void (*cpu_exec_exit)(PPCVirtualHypervisor *vhyp, PowerPCCPU *cpu);
 };
 
 #define TYPE_PPC_VIRTUAL_HYPERVISOR "ppc-virtual-hypervisor"
target/ppc/translate_init.inc.c +25 -0
index 86fc8f2e31..58d4a93b23 100644
@@ -10473,6 +10473,28 @@ static bool ppc_cpu_is_big_endian(CPUState *cs)
 }
 #endif
 
+static void ppc_cpu_exec_enter(CPUState *cs)
+{
+    PowerPCCPU *cpu = POWERPC_CPU(cs);
+
+    if (cpu->vhyp) {
+        PPCVirtualHypervisorClass *vhc =
+            PPC_VIRTUAL_HYPERVISOR_GET_CLASS(cpu->vhyp);
+        vhc->cpu_exec_enter(cpu->vhyp, cpu);
+    }
+}
+
+static void ppc_cpu_exec_exit(CPUState *cs)
+{
+    PowerPCCPU *cpu = POWERPC_CPU(cs);
+
+    if (cpu->vhyp) {
+        PPCVirtualHypervisorClass *vhc =
+            PPC_VIRTUAL_HYPERVISOR_GET_CLASS(cpu->vhyp);
+        vhc->cpu_exec_exit(cpu->vhyp, cpu);
+    }
+}
+
 static void ppc_cpu_instance_init(Object *obj)
 {
     PowerPCCPU *cpu = POWERPC_CPU(obj);
@@ -10624,6 +10646,9 @@ static void ppc_cpu_class_init(ObjectClass *oc, void *data)
     cc->tcg_initialize = ppc_translate_init;
     cc->tlb_fill = ppc_cpu_tlb_fill;
 #endif
+    cc->cpu_exec_enter = ppc_cpu_exec_enter;
+    cc->cpu_exec_exit = ppc_cpu_exec_exit;
+
     cc->disas_set_info = ppc_disas_set_info;
 
     dc->fw_name = "PowerPC,UNKNOWN";
Signature · 4 lines
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson
#13 lore raw

Re: [Qemu-devel] [PATCH v4 4/5] spapr: Implement H_JOIN

In reply to Nicholas Piggin
Quoted 6 lines · Nicholas Piggin

This has been useful to modify and test the Linux pseries suspend code but it requires modification to the guest to call it (due to being gated by other unimplemented features). It is not otherwise used by Linux yet, but work is slowly progressing there.

LGTM apart from style issues as noted by the bot.

Quoted 79 lines
---
 hw/ppc/spapr.c       |  1 +
 hw/ppc/spapr_hcall.c | 44 ++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 45 insertions(+)
hw/ppc/spapr.c +1 -0
index 87b11e2484..5c54e1cb9a 100644
@@ -1066,6 +1066,7 @@ static void spapr_dt_rtas(SpaprMachineState *spapr, void *fdt)
     add_str(hypertas, "hcall-tce");
     add_str(hypertas, "hcall-vio");
     add_str(hypertas, "hcall-splpar");
+    add_str(hypertas, "hcall-join");
     add_str(hypertas, "hcall-bulk");
     add_str(hypertas, "hcall-set-mode");
     add_str(hypertas, "hcall-sprg0");
hw/ppc/spapr_hcall.c +44 -0
index 28d58113be..52847a7047 100644
@@ -1069,6 +1069,47 @@ static target_ulong h_cede(PowerPCCPU *cpu, SpaprMachineState *spapr,
     return H_SUCCESS;
 }
 
+static target_ulong h_join(PowerPCCPU *cpu, SpaprMachineState *spapr,
+                           target_ulong opcode, target_ulong *args)
+{
+    CPUPPCState *env = &cpu->env;
+    CPUState *cs;
+    SpaprCpuState *spapr_cpu = spapr_cpu_state(cpu);
+    bool last_unjoined = true;
+
+    if (env->msr & (1ULL << MSR_EE)) {
+        return H_BAD_MODE;
+    }
+
+    if (spapr_cpu->prod) {
+        spapr_cpu->prod = false;
+        return H_SUCCESS;
+    }
+
+    CPU_FOREACH(cs) {
+        PowerPCCPU *c = POWERPC_CPU(cs);
+        CPUPPCState *e = &c->env;
+        if (c == cpu)
+            continue;
+
+	/* Don't have a way to indicate joined, so use halted && MSR[EE]=0 */
+        if (!cs->halted || (e->msr & (1ULL << MSR_EE))) {
+            last_unjoined = false;
+            break;
+        }
+    }
+    if (last_unjoined) {
+        return H_CONTINUE;
+    }
+
+    cs = CPU(cpu);
+    cs->halted = 1;
+    cs->exception_index = EXCP_HALTED;
+    cs->exit_request = 1;
+
+    return H_SUCCESS;
+}
+
 static target_ulong h_confer(PowerPCCPU *cpu, SpaprMachineState *spapr,
                            target_ulong opcode, target_ulong *args)
 {
@@ -1959,6 +2000,9 @@ static void hypercall_register_types(void)
     spapr_register_hypercall(H_CONFER, h_confer);
     spapr_register_hypercall(H_PROD, h_prod);
 
+    /* hcall-join */
+    spapr_register_hypercall(H_JOIN, h_join);
+
     spapr_register_hypercall(H_SIGNAL_SYS_RESET, h_signal_sys_reset);
 
     /* processor register resource access h-calls */
Signature · 4 lines
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson
#7 lore raw

Re: [Qemu-devel] [PATCH v4 2/5] spapr: Implement H_PROD

In reply to Nicholas Piggin
Quoted 4 lines · Nicholas Piggin

H_PROD is added, and H_CEDE is modified to test the prod bit according to PAPR.

LGTM apart from the style issues the bot noted.

Quoted 59 lines
---
 hw/ppc/spapr_hcall.c | 29 +++++++++++++++++++++++++++++
 1 file changed, 29 insertions(+)
hw/ppc/spapr_hcall.c +29 -0
index e615881ac4..8b208ab259 100644
@@ -1050,14 +1050,41 @@ static target_ulong h_cede(PowerPCCPU *cpu, SpaprMachineState *spapr,
 {
     CPUPPCState *env = &cpu->env;
     CPUState *cs = CPU(cpu);
+    SpaprCpuState *spapr_cpu = spapr_cpu_state(cpu);
 
     env->msr |= (1ULL << MSR_EE);
     hreg_compute_hflags(env);
+
+    if (spapr_cpu->prod) {
+        spapr_cpu->prod = false;
+        return H_SUCCESS;
+    }
+
     if (!cpu_has_work(cs)) {
         cs->halted = 1;
         cs->exception_index = EXCP_HLT;
         cs->exit_request = 1;
     }
+
+    return H_SUCCESS;
+}
+
+static target_ulong h_prod(PowerPCCPU *cpu, SpaprMachineState *spapr,
+                           target_ulong opcode, target_ulong *args)
+{
+    target_long target = args[0];
+    CPUState *cs;
+    SpaprCpuState *spapr_cpu = spapr_cpu_state(cpu);
+
+    cs = CPU(spapr_find_cpu(target));
+    if (!cs) {
+        return H_PARAMETER;
+    }
+
+    spapr_cpu->prod = true;
+    cs->halted = 0;
+    qemu_cpu_kick(cs);
+
     return H_SUCCESS;
 }
 
@@ -1882,6 +1909,8 @@ static void hypercall_register_types(void)
     /* hcall-splpar */
     spapr_register_hypercall(H_REGISTER_VPA, h_register_vpa);
     spapr_register_hypercall(H_CEDE, h_cede);
+    spapr_register_hypercall(H_PROD, h_prod);
+
     spapr_register_hypercall(H_SIGNAL_SYS_RESET, h_signal_sys_reset);
 
     /* processor register resource access h-calls */
Signature · 4 lines
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson
#15 lore raw

Re: [Qemu-devel] [PATCH v4 5/5] spapr: Implement ibm,suspend-me

In reply to Nicholas Piggin
Quoted 21 lines · Nicholas Piggin

This has been useful to modify and test the Linux pseries suspend code but it requires modification to the guest to call it (due to being gated by other unimplemented features). It is not otherwise used by Linux yet, but work is slowly progressing there.

This allows a (lightly modified) guest kernel to suspend with echo mem > /sys/power/state and be resumed with system_wakeup monitor command.

---
 hw/ppc/spapr.c         | 26 ++++++++++++++++++++++++++
 hw/ppc/spapr_rtas.c    | 32 ++++++++++++++++++++++++++++++++
 include/hw/ppc/spapr.h |  7 ++++++-
 3 files changed, 64 insertions(+), 1 deletion(-)
hw/ppc/spapr.c +0 -0
index 5c54e1cb9a..b85d41bb1e 100644

hw/ppc/spapr.c static void spapr_machine_reset(MachineState *machine) · line 1714

Quoted 5 lines
hw/ppc/spapr.c +2 -0
@@ -1710,6 +1710,11 @@ static void spapr_machine_reset(MachineState *machine)
     void *fdt;
     int rc;
 
+    if (spapr->suspend_reset) {
+        spapr->suspend_reset = false;

Do we need to migrate this value?

hw/ppc/spapr.c static PCIHostState *spapr_create_default_phb(void) · line 2729

Quoted 11 lines
hw/ppc/spapr.c +4 -0
+        return;
+    }
+
     spapr_caps_apply(spapr);
 
     first_ppc_cpu = POWERPC_CPU(first_cpu);
@@ -2721,6 +2726,23 @@ static PCIHostState *spapr_create_default_phb(void)
     return PCI_HOST_BRIDGE(dev);
 }
 
+static Notifier wakeup;

I think this should be in sPAPRMachineState rather than global.

Quoted 4 lines
hw/ppc/spapr.c +4 -0
+static void spapr_notify_wakeup(Notifier *notifier, void *data)
+{
+    WakeupReason *reason = data;
+

hw/ppc/spapr.c static PCIHostState *spapr_create_default_phb(void) · line 2743

Quoted 10 lines
hw/ppc/spapr.c +10 -0
+    switch (*reason) {
+    case QEMU_WAKEUP_REASON_RTC:
+        break;
+    case QEMU_WAKEUP_REASON_PMTIMER:
+        break;
+    case QEMU_WAKEUP_REASON_OTHER:
+        break;
+    default:
+        break;
+    }

So.. you have a bunch of switch cases, all of which ignore the input..

Quoted 71 lines
hw/ppc/spapr.c +6 -0
+}
+
 /* pSeries LPAR / sPAPR hardware init */
 static void spapr_machine_init(MachineState *machine)
 {
@@ -3078,6 +3100,10 @@ static void spapr_machine_init(MachineState *machine)
 
     qemu_register_boot_set(spapr_boot_set, spapr);
 
+    wakeup.notify = spapr_notify_wakeup;
+    qemu_register_wakeup_notifier(&wakeup);
+    qemu_register_wakeup_support();
+
     if (kvm_enabled()) {
         /* to stop and start vmclock */
         qemu_add_vm_change_state_handler(cpu_ppc_clock_vm_state_change,
hw/ppc/spapr_rtas.c +32 -0
index a618a2ac0f..60a007ec38 100644
@@ -216,6 +216,36 @@ static void rtas_stop_self(PowerPCCPU *cpu, SpaprMachineState *spapr,
     qemu_cpu_kick(cs);
 }
 
+static void rtas_ibm_suspend_me(PowerPCCPU *cpu, SpaprMachineState *spapr,
+                           uint32_t token, uint32_t nargs,
+                           target_ulong args,
+                           uint32_t nret, target_ulong rets)
+{
+    CPUState *cs;
+
+    if (nargs != 0 || nret != 1) {
+        rtas_st(rets, 0, RTAS_OUT_PARAM_ERROR);
+        return;
+    }
+
+    CPU_FOREACH(cs) {
+        PowerPCCPU *c = POWERPC_CPU(cs);
+        CPUPPCState *e = &c->env;
+        if (c == cpu)
+            continue;
+
+	/* See h_join */
+        if (!cs->halted || (e->msr & (1ULL << MSR_EE))) {
+            rtas_st(rets, 0, H_MULTI_THREADS_ACTIVE);
+            return;
+        }
+    }
+
+    spapr->suspend_reset = true;
+    qemu_system_suspend_request();
+    rtas_st(rets, 0, RTAS_OUT_SUCCESS);
+}
+
 static inline int sysparm_st(target_ulong addr, target_ulong len,
                              const void *val, uint16_t vallen)
 {
@@ -483,6 +513,8 @@ static void core_rtas_register_types(void)
                         rtas_query_cpu_stopped_state);
     spapr_rtas_register(RTAS_START_CPU, "start-cpu", rtas_start_cpu);
     spapr_rtas_register(RTAS_STOP_SELF, "stop-self", rtas_stop_self);
+    spapr_rtas_register(RTAS_IBM_SUSPEND_ME, "ibm,suspend-me",
+                        rtas_ibm_suspend_me);
     spapr_rtas_register(RTAS_IBM_GET_SYSTEM_PARAMETER,
                         "ibm,get-system-parameter",
                         rtas_ibm_get_system_parameter);
include/hw/ppc/spapr.h +0 -0
index 5d36eec9d0..df0b0c15da 100644

include/hw/ppc/spapr.h struct SpaprMachineState { · line 176

Quoted 6 lines
include/hw/ppc/spapr.h +3 -0
@@ -171,6 +171,10 @@ struct SpaprMachineState {
     bool use_hotplug_event_source;
     SpaprEventSource *event_sources;
 
+    /* Machine has been suspended, so the next machine_reset should not
+     * reset state, but just return and allow execution to resume. */
+    bool suspend_reset;

Hrm, this seems odd, but maybe it's part of the existing suspend design. Why would system_reset resume a suspend, rather than having a specific operation for that.

Quoted 15 lines
include/hw/ppc/spapr.h +3 -1
+
     /* ibm,client-architecture-support option negotiation */
     bool cas_reboot;
     bool cas_legacy_guest_workaround;
@@ -631,8 +635,9 @@ target_ulong spapr_hypercall(PowerPCCPU *cpu, target_ulong opcode,
 #define RTAS_IBM_CREATE_PE_DMA_WINDOW           (RTAS_TOKEN_BASE + 0x27)
 #define RTAS_IBM_REMOVE_PE_DMA_WINDOW           (RTAS_TOKEN_BASE + 0x28)
 #define RTAS_IBM_RESET_PE_DMA_WINDOW            (RTAS_TOKEN_BASE + 0x29)
+#define RTAS_IBM_SUSPEND_ME                     (RTAS_TOKEN_BASE + 0x2A)
 
-#define RTAS_TOKEN_MAX                          (RTAS_TOKEN_BASE + 0x2A)
+#define RTAS_TOKEN_MAX                          (RTAS_TOKEN_BASE + 0x2B)
 
 /* RTAS ibm,get-system-parameter token values */
 #define RTAS_SYSPARM_SPLPAR_CHARACTERISTICS      20
Signature · 4 lines
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson
#9 lore raw

Re: [Qemu-devel] [PATCH v4 3/5] spapr: Implement H_CONFER

In reply to Nicholas Piggin
Quoted 25 lines · Nicholas Piggin

This does not do directed yielding and is not quite as strict as PAPR specifies in terms of precise dispatch behaviour. This generally will mean suboptimal performance, rather than guest misbehaviour. Linux does not rely on exact dispatch behaviour.

---
 hw/ppc/spapr_hcall.c | 48 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 48 insertions(+)
hw/ppc/spapr_hcall.c +7 -0
index 8b208ab259..28d58113be 100644
@@ -1069,6 +1069,53 @@ static target_ulong h_cede(PowerPCCPU *cpu, SpaprMachineState *spapr,
     return H_SUCCESS;
 }
 
+static target_ulong h_confer(PowerPCCPU *cpu, SpaprMachineState *spapr,
+                           target_ulong opcode, target_ulong *args)
+{
+    target_long target = args[0];
+    uint32_t dispatch = args[1];
+    PowerPCCPU *target_cpu = spapr_find_cpu(target);
+    CPUState *target_cs = CPU(target_cpu);

hw/ppc/spapr_hcall.c static target_ulong h_cede(PowerPCCPU *cpu, SpaprMachineState *spapr, · line 1088

Quoted 10 lines
hw/ppc/spapr_hcall.c +10 -0
+    CPUState *cs = CPU(cpu);
+    SpaprCpuState *spapr_cpu;
+
+    /*
+     * This does not do a targeted yield or confer, but check the parameter
+     * anyway. -1 means confer to all/any other CPUs.
+     */
+    if (target != -1 && !target_cs) {
+        return H_PARAMETER;
+    }

Should we return an error if a targeted yield is attempted, rather than pretend we've done it?

hw/ppc/spapr_hcall.c static target_ulong h_cede(PowerPCCPU *cpu, SpaprMachineState *spapr, · line 1093

Quoted 5 lines
hw/ppc/spapr_hcall.c +5 -0
+
+    spapr_cpu = spapr_cpu_state(target_cpu);
+
+    /*
+     * PAPR specifies waiting until proded in this case, without dispatch

s/proded/prodded/

hw/ppc/spapr_hcall.c static target_ulong h_cede(PowerPCCPU *cpu, SpaprMachineState *spapr, · line 1104

Quoted 11 lines
hw/ppc/spapr_hcall.c +11 -0
+     * counter check.
+     */
+    if (cpu == target_cpu) {
+        if (spapr_cpu->prod) {
+            spapr_cpu->prod = false;
+            return H_SUCCESS;
+        }
+
+        cs->halted = 1;
+        cs->exception_index = EXCP_HALTED;
+        cs->exit_request = 1;

Now that we're using this sequence in a bunch of places, I wonder if we want a little helper function.

Quoted 25 lines
hw/ppc/spapr_hcall.c +15 -0
+
+        return H_SUCCESS;
+    }
+
+    if (spapr_cpu->dispatch_counter != dispatch || (dispatch & 1) == 0) {
+        return H_SUCCESS;
+    }
+
+    cs->exception_index = EXCP_YIELD;
+    cpu_loop_exit(cs);
+
+    return H_SUCCESS;
+}
+
 static target_ulong h_prod(PowerPCCPU *cpu, SpaprMachineState *spapr,
                            target_ulong opcode, target_ulong *args)
 {
@@ -1909,6 +1956,7 @@ static void hypercall_register_types(void)
     /* hcall-splpar */
     spapr_register_hypercall(H_REGISTER_VPA, h_register_vpa);
     spapr_register_hypercall(H_CEDE, h_cede);
+    spapr_register_hypercall(H_CONFER, h_confer);
     spapr_register_hypercall(H_PROD, h_prod);
 
     spapr_register_hypercall(H_SIGNAL_SYS_RESET, h_signal_sys_reset);
Signature · 4 lines
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson
#3 lore raw

Re: [Qemu-devel] [PATCH v4 1/5] spapr: Implement dispatch counter and prod bit on tcg

In reply to David Gibson

David Gibson's on July 16, 2019 5:34 pm:

Quoted 16 lines · Nicholas Piggin
Quoted 5 lines

Implement cpu_exec_enter/exit on ppc which calls into new methods of the same name in PPCVirtualHypervisorClass. These are used by spapr to implement these splpar elements, used in subsequent changes.

Quoted 9 lines
---
 hw/ppc/spapr.c                  | 25 +++++++++++++++++++++++++
 hw/ppc/spapr_cpu_core.c         |  5 ++++-
 hw/ppc/spapr_hcall.c            |  5 -----
 include/hw/ppc/spapr.h          |  7 +++++++
 include/hw/ppc/spapr_cpu_core.h |  2 ++
 target/ppc/cpu.h                |  2 ++
 target/ppc/translate_init.inc.c | 25 +++++++++++++++++++++++++
 7 files changed, 65 insertions(+), 6 deletions(-)
Quoted 2 lines

Mostly LGTM. Please do address the style issues that the bot reported.

Will do.

Quoted 2 lines

In addition, do the dispatch_counter and prod values need to be migrated?

I was thinking no, it should be just a blip. But could the guest be reading the dispatch counter in the vpa for some other reason? There are other (unimplemented) dispatch statistics in there AFAIKS. Linux doesn't, but others might.

And what about KVM, does any of the VPA data get migrated for KVM guests?

Thanks,
Nick

#10 lore raw

Re: [Qemu-devel] [PATCH v4 3/5] spapr: Implement H_CONFER

In reply to David Gibson

David Gibson's on July 16, 2019 6:25 pm:

Quoted 31 lines · Nicholas Piggin
Quoted 30 lines

This does not do directed yielding and is not quite as strict as PAPR specifies in terms of precise dispatch behaviour. This generally will mean suboptimal performance, rather than guest misbehaviour. Linux does not rely on exact dispatch behaviour.

---
 hw/ppc/spapr_hcall.c | 48 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 48 insertions(+)
hw/ppc/spapr_hcall.c +12 -0
index 8b208ab259..28d58113be 100644
@@ -1069,6 +1069,53 @@ static target_ulong h_cede(PowerPCCPU *cpu, SpaprMachineState *spapr,
     return H_SUCCESS;
 }
 
+static target_ulong h_confer(PowerPCCPU *cpu, SpaprMachineState *spapr,
+                           target_ulong opcode, target_ulong *args)
+{
+    target_long target = args[0];
+    uint32_t dispatch = args[1];
+    PowerPCCPU *target_cpu = spapr_find_cpu(target);
+    CPUState *target_cs = CPU(target_cpu);
+    CPUState *cs = CPU(cpu);
+    SpaprCpuState *spapr_cpu;
+
+    /*
+     * This does not do a targeted yield or confer, but check the parameter
Quoted 8 lines

hw/ppc/spapr_hcall.c static target_ulong h_cede(PowerPCCPU *cpu, SpaprMachineState *spapr, · line 1088

Quoted 5 lines
hw/ppc/spapr_hcall.c +5 -0
+     * anyway. -1 means confer to all/any other CPUs.
+     */
+    if (target != -1 && !target_cs) {
+        return H_PARAMETER;
+    }

Should we return an error if a targeted yield is attempted, rather than pretend we've done it?

I don't think so, because we do _some_ kind of yield for the directed case which is probably better than nothing, and Linux won't fall back.

PAPR is much more strict about dispatching. The H_CONFERing vCPU must not run until the target(s) has been dispatched (if runnable), for example. So we don't really implement it to the letter, we just do "some kind of yield, whatever generic tcg code has implemented".

For single threaded tcg it seems a signifcant complication to the round robin algorithm to add a directed yield, yet simply yielding to the next vCPU is a good idea here because useful work will get done including by the lock holder before we run again.

If multi threaded tcg performance with lot of vCPUs and lock contention starts becoming more important I guess directed yielding might be something to look at.

Thanks,
Nick

#16 lore raw

Re: [Qemu-devel] [PATCH v4 5/5] spapr: Implement ibm,suspend-me

In reply to David Gibson

David Gibson's on July 16, 2019 6:30 pm:

Quoted 25 lines · Nicholas Piggin
Quoted 24 lines

This has been useful to modify and test the Linux pseries suspend code but it requires modification to the guest to call it (due to being gated by other unimplemented features). It is not otherwise used by Linux yet, but work is slowly progressing there.

This allows a (lightly modified) guest kernel to suspend with echo mem > /sys/power/state and be resumed with system_wakeup monitor command.

---
 hw/ppc/spapr.c         | 26 ++++++++++++++++++++++++++
 hw/ppc/spapr_rtas.c    | 32 ++++++++++++++++++++++++++++++++
 include/hw/ppc/spapr.h |  7 ++++++-
 3 files changed, 64 insertions(+), 1 deletion(-)
hw/ppc/spapr.c +0 -0
index 5c54e1cb9a..b85d41bb1e 100644
@@ -1710,6 +1710,11 @@ static void spapr_machine_reset(MachineState *machine)
     void *fdt;
     int rc;
 
Quoted 4 lines

hw/ppc/spapr.c static void spapr_machine_reset(MachineState *machine) · line 1714

Quoted 2 lines
hw/ppc/spapr.c +2 -0
+    if (spapr->suspend_reset) {
+        spapr->suspend_reset = false;

Do we need to migrate this value?

I suppose we do if we can migrate a suspended machine?

Quoted 6 lines
Quoted 6 lines
hw/ppc/spapr.c +3 -0
+        return;
+    }
+
     spapr_caps_apply(spapr);
 
Quoted 8 lines

hw/ppc/spapr.c static PCIHostState *spapr_create_default_phb(void) · line 2729

Quoted 6 lines
hw/ppc/spapr.c +1 -0
     first_ppc_cpu = POWERPC_CPU(first_cpu);
@@ -2721,6 +2726,23 @@ static PCIHostState *spapr_create_default_phb(void)
     return PCI_HOST_BRIDGE(dev);
 }
 
+static Notifier wakeup;

I think this should be in sPAPRMachineState rather than global.

Sure.

Quoted 17 lines
Quoted 5 lines
hw/ppc/spapr.c +4 -0
+static void spapr_notify_wakeup(Notifier *notifier, void *data)
+{
+    WakeupReason *reason = data;
+

hw/ppc/spapr.c static PCIHostState *spapr_create_default_phb(void) · line 2743

Quoted 10 lines
hw/ppc/spapr.c +10 -0
+    switch (*reason) {
+    case QEMU_WAKEUP_REASON_RTC:
+        break;
+    case QEMU_WAKEUP_REASON_PMTIMER:
+        break;
+    case QEMU_WAKEUP_REASON_OTHER:
+        break;
+    default:
+        break;
+    }

So.. you have a bunch of switch cases, all of which ignore the input..

Yeah I kind of just copy and pasted I think. This part of the patch may not have been quite as cooked as I remembered :\

Quoted 74 lines
Quoted 74 lines
hw/ppc/spapr.c +6 -0
+}
+
 /* pSeries LPAR / sPAPR hardware init */
 static void spapr_machine_init(MachineState *machine)
 {
@@ -3078,6 +3100,10 @@ static void spapr_machine_init(MachineState *machine)
 
     qemu_register_boot_set(spapr_boot_set, spapr);
 
+    wakeup.notify = spapr_notify_wakeup;
+    qemu_register_wakeup_notifier(&wakeup);
+    qemu_register_wakeup_support();
+
     if (kvm_enabled()) {
         /* to stop and start vmclock */
         qemu_add_vm_change_state_handler(cpu_ppc_clock_vm_state_change,
hw/ppc/spapr_rtas.c +32 -0
index a618a2ac0f..60a007ec38 100644
@@ -216,6 +216,36 @@ static void rtas_stop_self(PowerPCCPU *cpu, SpaprMachineState *spapr,
     qemu_cpu_kick(cs);
 }
 
+static void rtas_ibm_suspend_me(PowerPCCPU *cpu, SpaprMachineState *spapr,
+                           uint32_t token, uint32_t nargs,
+                           target_ulong args,
+                           uint32_t nret, target_ulong rets)
+{
+    CPUState *cs;
+
+    if (nargs != 0 || nret != 1) {
+        rtas_st(rets, 0, RTAS_OUT_PARAM_ERROR);
+        return;
+    }
+
+    CPU_FOREACH(cs) {
+        PowerPCCPU *c = POWERPC_CPU(cs);
+        CPUPPCState *e = &c->env;
+        if (c == cpu)
+            continue;
+
+	/* See h_join */
+        if (!cs->halted || (e->msr & (1ULL << MSR_EE))) {
+            rtas_st(rets, 0, H_MULTI_THREADS_ACTIVE);
+            return;
+        }
+    }
+
+    spapr->suspend_reset = true;
+    qemu_system_suspend_request();
+    rtas_st(rets, 0, RTAS_OUT_SUCCESS);
+}
+
 static inline int sysparm_st(target_ulong addr, target_ulong len,
                              const void *val, uint16_t vallen)
 {
@@ -483,6 +513,8 @@ static void core_rtas_register_types(void)
                         rtas_query_cpu_stopped_state);
     spapr_rtas_register(RTAS_START_CPU, "start-cpu", rtas_start_cpu);
     spapr_rtas_register(RTAS_STOP_SELF, "stop-self", rtas_stop_self);
+    spapr_rtas_register(RTAS_IBM_SUSPEND_ME, "ibm,suspend-me",
+                        rtas_ibm_suspend_me);
     spapr_rtas_register(RTAS_IBM_GET_SYSTEM_PARAMETER,
                         "ibm,get-system-parameter",
                         rtas_ibm_get_system_parameter);
include/hw/ppc/spapr.h +0 -0
index 5d36eec9d0..df0b0c15da 100644
@@ -171,6 +171,10 @@ struct SpaprMachineState {
     bool use_hotplug_event_source;
     SpaprEventSource *event_sources;
 
Quoted 7 lines

include/hw/ppc/spapr.h struct SpaprMachineState { · line 176

Quoted 3 lines
include/hw/ppc/spapr.h +3 -0
+    /* Machine has been suspended, so the next machine_reset should not
+     * reset state, but just return and allow execution to resume. */
+    bool suspend_reset;

Hrm, this seems odd, but maybe it's part of the existing suspend design. Why would system_reset resume a suspend, rather than having a specific operation for that.

It is where system_wakeup cmd pops out, via qemu_system_reset, main_loop_should_exit. I'm not sure if we have any existing state we can use. runstate_is_running() doesn't seem to work because of CAS I guess (maybe CAS is what makes spapr so much different from x86 in terms of resetting the world here?)

Thanks,
Nick

#11 lore raw

Re: [Qemu-devel] [PATCH v4 3/5] spapr: Implement H_CONFER

In reply to Nicholas Piggin
Quoted 49 lines · Nicholas Piggin

David Gibson's on July 16, 2019 6:25 pm:

Quoted 31 lines · Nicholas Piggin
Quoted 30 lines

This does not do directed yielding and is not quite as strict as PAPR specifies in terms of precise dispatch behaviour. This generally will mean suboptimal performance, rather than guest misbehaviour. Linux does not rely on exact dispatch behaviour.

---
 hw/ppc/spapr_hcall.c | 48 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 48 insertions(+)
hw/ppc/spapr_hcall.c +12 -0
index 8b208ab259..28d58113be 100644
@@ -1069,6 +1069,53 @@ static target_ulong h_cede(PowerPCCPU *cpu, SpaprMachineState *spapr,
     return H_SUCCESS;
 }
 
+static target_ulong h_confer(PowerPCCPU *cpu, SpaprMachineState *spapr,
+                           target_ulong opcode, target_ulong *args)
+{
+    target_long target = args[0];
+    uint32_t dispatch = args[1];
+    PowerPCCPU *target_cpu = spapr_find_cpu(target);
+    CPUState *target_cs = CPU(target_cpu);
+    CPUState *cs = CPU(cpu);
+    SpaprCpuState *spapr_cpu;
+
+    /*
+     * This does not do a targeted yield or confer, but check the parameter
Quoted 8 lines

hw/ppc/spapr_hcall.c static target_ulong h_cede(PowerPCCPU *cpu, SpaprMachineState *spapr, · line 1088

Quoted 5 lines
hw/ppc/spapr_hcall.c +5 -0
+     * anyway. -1 means confer to all/any other CPUs.
+     */
+    if (target != -1 && !target_cs) {
+        return H_PARAMETER;
+    }

Should we return an error if a targeted yield is attempted, rather than pretend we've done it?

I don't think so, because we do _some_ kind of yield for the directed case which is probably better than nothing, and Linux won't fall back.

PAPR is much more strict about dispatching. The H_CONFERing vCPU must not run until the target(s) has been dispatched (if runnable), for example. So we don't really implement it to the letter, we just do "some kind of yield, whatever generic tcg code has implemented".

Quoted 8 lines

For single threaded tcg it seems a signifcant complication to the round robin algorithm to add a directed yield, yet simply yielding to the next vCPU is a good idea here because useful work will get done including by the lock holder before we run again.

If multi threaded tcg performance with lot of vCPUs and lock contention starts becoming more important I guess directed yielding might be something to look at.

Ok, makes sense to me.

Signature · 4 lines
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson
#17 lore raw

Re: [Qemu-devel] [PATCH v4 5/5] spapr: Implement ibm,suspend-me

In reply to Nicholas Piggin

On Tue, Jul 16, 2019 at 09:15:23PM +1000, Nicholas Piggin wrote:
65;5603;1c> David Gibson's on July 16, 2019 6:30 pm:

Quoted 25 lines
Quoted 25 lines · Nicholas Piggin
Quoted 24 lines

This has been useful to modify and test the Linux pseries suspend code but it requires modification to the guest to call it (due to being gated by other unimplemented features). It is not otherwise used by Linux yet, but work is slowly progressing there.

This allows a (lightly modified) guest kernel to suspend with echo mem > /sys/power/state and be resumed with system_wakeup monitor command.

---
 hw/ppc/spapr.c         | 26 ++++++++++++++++++++++++++
 hw/ppc/spapr_rtas.c    | 32 ++++++++++++++++++++++++++++++++
 include/hw/ppc/spapr.h |  7 ++++++-
 3 files changed, 64 insertions(+), 1 deletion(-)
hw/ppc/spapr.c +0 -0
index 5c54e1cb9a..b85d41bb1e 100644
@@ -1710,6 +1710,11 @@ static void spapr_machine_reset(MachineState *machine)
     void *fdt;
     int rc;
 
Quoted 6 lines
Quoted 4 lines

hw/ppc/spapr.c static void spapr_machine_reset(MachineState *machine) · line 1714

Quoted 2 lines
hw/ppc/spapr.c +2 -0
+    if (spapr->suspend_reset) {
+        spapr->suspend_reset = false;

Do we need to migrate this value?

I suppose we do if we can migrate a suspended machine?

I don't see why we couldn't. And it might not happen because of explicit user choice in a managed environment like RHV or openstack.

Quoted 34 lines
Quoted 5 lines
Quoted 5 lines
hw/ppc/spapr.c +3 -0
+        return;
+    }
+
     spapr_caps_apply(spapr);
 
Quoted 8 lines

hw/ppc/spapr.c static PCIHostState *spapr_create_default_phb(void) · line 2729

Quoted 6 lines
hw/ppc/spapr.c +1 -0
     first_ppc_cpu = POWERPC_CPU(first_cpu);
@@ -2721,6 +2726,23 @@ static PCIHostState *spapr_create_default_phb(void)
     return PCI_HOST_BRIDGE(dev);
 }
 
+static Notifier wakeup;

I think this should be in sPAPRMachineState rather than global.

Sure.

Quoted 17 lines
Quoted 5 lines
hw/ppc/spapr.c +4 -0
+static void spapr_notify_wakeup(Notifier *notifier, void *data)
+{
+    WakeupReason *reason = data;
+

hw/ppc/spapr.c static PCIHostState *spapr_create_default_phb(void) · line 2743

Quoted 10 lines
hw/ppc/spapr.c +10 -0
+    switch (*reason) {
+    case QEMU_WAKEUP_REASON_RTC:
+        break;
+    case QEMU_WAKEUP_REASON_PMTIMER:
+        break;
+    case QEMU_WAKEUP_REASON_OTHER:
+        break;
+    default:
+        break;
+    }

So.. you have a bunch of switch cases, all of which ignore the input..

Quoted 2 lines

Yeah I kind of just copy and pasted I think. This part of the patch may not have been quite as cooked as I remembered :\

Heh :).

Quoted 82 lines
Quoted 74 lines
Quoted 74 lines
hw/ppc/spapr.c +6 -0
+}
+
 /* pSeries LPAR / sPAPR hardware init */
 static void spapr_machine_init(MachineState *machine)
 {
@@ -3078,6 +3100,10 @@ static void spapr_machine_init(MachineState *machine)
 
     qemu_register_boot_set(spapr_boot_set, spapr);
 
+    wakeup.notify = spapr_notify_wakeup;
+    qemu_register_wakeup_notifier(&wakeup);
+    qemu_register_wakeup_support();
+
     if (kvm_enabled()) {
         /* to stop and start vmclock */
         qemu_add_vm_change_state_handler(cpu_ppc_clock_vm_state_change,
hw/ppc/spapr_rtas.c +32 -0
index a618a2ac0f..60a007ec38 100644
@@ -216,6 +216,36 @@ static void rtas_stop_self(PowerPCCPU *cpu, SpaprMachineState *spapr,
     qemu_cpu_kick(cs);
 }
 
+static void rtas_ibm_suspend_me(PowerPCCPU *cpu, SpaprMachineState *spapr,
+                           uint32_t token, uint32_t nargs,
+                           target_ulong args,
+                           uint32_t nret, target_ulong rets)
+{
+    CPUState *cs;
+
+    if (nargs != 0 || nret != 1) {
+        rtas_st(rets, 0, RTAS_OUT_PARAM_ERROR);
+        return;
+    }
+
+    CPU_FOREACH(cs) {
+        PowerPCCPU *c = POWERPC_CPU(cs);
+        CPUPPCState *e = &c->env;
+        if (c == cpu)
+            continue;
+
+	/* See h_join */
+        if (!cs->halted || (e->msr & (1ULL << MSR_EE))) {
+            rtas_st(rets, 0, H_MULTI_THREADS_ACTIVE);
+            return;
+        }
+    }
+
+    spapr->suspend_reset = true;
+    qemu_system_suspend_request();
+    rtas_st(rets, 0, RTAS_OUT_SUCCESS);
+}
+
 static inline int sysparm_st(target_ulong addr, target_ulong len,
                              const void *val, uint16_t vallen)
 {
@@ -483,6 +513,8 @@ static void core_rtas_register_types(void)
                         rtas_query_cpu_stopped_state);
     spapr_rtas_register(RTAS_START_CPU, "start-cpu", rtas_start_cpu);
     spapr_rtas_register(RTAS_STOP_SELF, "stop-self", rtas_stop_self);
+    spapr_rtas_register(RTAS_IBM_SUSPEND_ME, "ibm,suspend-me",
+                        rtas_ibm_suspend_me);
     spapr_rtas_register(RTAS_IBM_GET_SYSTEM_PARAMETER,
                         "ibm,get-system-parameter",
                         rtas_ibm_get_system_parameter);
include/hw/ppc/spapr.h +0 -0
index 5d36eec9d0..df0b0c15da 100644
@@ -171,6 +171,10 @@ struct SpaprMachineState {
     bool use_hotplug_event_source;
     SpaprEventSource *event_sources;
 
Quoted 7 lines

include/hw/ppc/spapr.h struct SpaprMachineState { · line 176

Quoted 3 lines
include/hw/ppc/spapr.h +3 -0
+    /* Machine has been suspended, so the next machine_reset should not
+     * reset state, but just return and allow execution to resume. */
+    bool suspend_reset;

Hrm, this seems odd, but maybe it's part of the existing suspend design. Why would system_reset resume a suspend, rather than having a specific operation for that.

Quoted 5 lines

It is where system_wakeup cmd pops out, via qemu_system_reset, main_loop_should_exit. I'm not sure if we have any existing state we can use. runstate_is_running() doesn't seem to work because of CAS I guess (maybe CAS is what makes spapr so much different from x86 in terms of resetting the world here?)

CAS certainly complicates things.

Signature · 4 lines
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson
#4 lore raw

Re: [Qemu-devel] [PATCH v4 1/5] spapr: Implement dispatch counter and prod bit on tcg

In reply to Nicholas Piggin
Quoted 22 lines · Nicholas Piggin

David Gibson's on July 16, 2019 5:34 pm:

Quoted 16 lines · Nicholas Piggin
Quoted 5 lines

Implement cpu_exec_enter/exit on ppc which calls into new methods of the same name in PPCVirtualHypervisorClass. These are used by spapr to implement these splpar elements, used in subsequent changes.

Quoted 9 lines
---
 hw/ppc/spapr.c                  | 25 +++++++++++++++++++++++++
 hw/ppc/spapr_cpu_core.c         |  5 ++++-
 hw/ppc/spapr_hcall.c            |  5 -----
 include/hw/ppc/spapr.h          |  7 +++++++
 include/hw/ppc/spapr_cpu_core.h |  2 ++
 target/ppc/cpu.h                |  2 ++
 target/ppc/translate_init.inc.c | 25 +++++++++++++++++++++++++
 7 files changed, 65 insertions(+), 6 deletions(-)
Quoted 2 lines

Mostly LGTM. Please do address the style issues that the bot reported.

Will do.

Quoted 7 lines
Quoted 2 lines

In addition, do the dispatch_counter and prod values need to be migrated?

I was thinking no, it should be just a blip. But could the guest be reading the dispatch counter in the vpa for some other reason? There are other (unimplemented) dispatch statistics in there AFAIKS. Linux doesn't, but others might.

Right, I guess it's a question of whether a guest would get confused if the dispatch count suddenly resets.

Quoted 2 lines

And what about KVM, does any of the VPA data get migrated for KVM guests?

Well, the stuff that's actually written into the VPA is implicitly migrated, since it's part of guest memory.

Signature · 4 lines
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson
#5 lore raw

Re: [Qemu-devel] [PATCH v4 1/5] spapr: Implement dispatch counter and prod bit on tcg

David Gibson's on July 17, 2019 11:51 am:

Quoted 26 lines · Nicholas Piggin
Quoted 25 lines

David Gibson's on July 16, 2019 5:34 pm:

Quoted 16 lines · Nicholas Piggin
Quoted 5 lines

Implement cpu_exec_enter/exit on ppc which calls into new methods of the same name in PPCVirtualHypervisorClass. These are used by spapr to implement these splpar elements, used in subsequent changes.

Quoted 9 lines
---
 hw/ppc/spapr.c                  | 25 +++++++++++++++++++++++++
 hw/ppc/spapr_cpu_core.c         |  5 ++++-
 hw/ppc/spapr_hcall.c            |  5 -----
 include/hw/ppc/spapr.h          |  7 +++++++
 include/hw/ppc/spapr_cpu_core.h |  2 ++
 target/ppc/cpu.h                |  2 ++
 target/ppc/translate_init.inc.c | 25 +++++++++++++++++++++++++
 7 files changed, 65 insertions(+), 6 deletions(-)
Quoted 2 lines

Mostly LGTM. Please do address the style issues that the bot reported.

Will do.

Quoted 2 lines

In addition, do the dispatch_counter and prod values need to be migrated?

Quoted 7 lines
Quoted 4 lines

I was thinking no, it should be just a blip. But could the guest be reading the dispatch counter in the vpa for some other reason? There are other (unimplemented) dispatch statistics in there AFAIKS. Linux doesn't, but others might.

Right, I guess it's a question of whether a guest would get confused if the dispatch count suddenly resets.

Linux wouldn't AFAIKS, but...

Quoted 5 lines
Quoted 2 lines

And what about KVM, does any of the VPA data get migrated for KVM guests?

Well, the stuff that's actually written into the VPA is implicitly migrated, since it's part of guest memory.

Yeah I just sent a patch which uses the VPA directly, so it matches KVM and should get migrated.

Thanks,
Nick