Skip to contentSkip to conversation

record-replay: support SMP target machine

RFC PATCH Superseded Nicholas Piggin 1 participant 1 message Aug 5, 2023

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

RFC PATCH 1/1 record-replay: support SMP target machine

Hi, I have a use for RR in SMP machine. It seems conceptually simple to add, RR scheduler is driven in a deterministic way like everything else. In practice, I'm not entirely sure if I have all the locking and corner cases covered. The code is also a bit ugly at the moment in terms of CPU switching and recording, possibly an initial tidy patch to wrap up the CPU scheduling in a nice way would make that better.

In any case, this series works enough on a ppc smp 2 guest including step and reverse step of both threads when booting Linux (with some other ppc rr fix patches that have already been posted).

Any thoughts on the approach, or details?

Thanks
Nick

---
RR CPU switching is driven by timers and events so it is deterministic like everything else. Record a CPU switch event and use that to drive the CPU switch on replay.

Some of the RR CPU scheduling variables have to be reset to get an exact replay.

---
 accel/tcg/tcg-accel-ops-icount.c |  9 +++-
 accel/tcg/tcg-accel-ops-rr.c     | 73 ++++++++++++++++++++++++--------
 hw/ppc/spapr.c                   |  1 +
 include/exec/replay-core.h       |  3 ++
 replay/replay-internal.h         |  1 +
 replay/replay.c                  | 34 ++++++++++++++-
 scripts/replay-dump.py           |  5 +++
 softmmu/vl.c                     |  2 +-
 8 files changed, 107 insertions(+), 21 deletions(-)
accel/tcg/tcg-accel-ops-icount.c +7 -2
index 3d2cfbbc97..c26782a56a 100644
@@ -93,10 +93,15 @@ void icount_handle_deadline(void)
 int64_t icount_percpu_budget(int cpu_count)
 {
     int64_t limit = icount_get_limit();
-    int64_t timeslice = limit / cpu_count;
+    int64_t timeslice;
 
-    if (timeslice == 0) {
+    if (replay_mode == REPLAY_MODE_PLAY) {
         timeslice = limit;
+    } else {
+        timeslice = limit / cpu_count;
+        if (timeslice == 0) {
+            timeslice = limit;
+        }
     }
 
     return timeslice;
accel/tcg/tcg-accel-ops-rr.c +46 -12

117 lines not on this page · raw message

include/exec/replay-core.h +3 -0
index 244c77acce..543c129a1d 100644
@@ -52,6 +52,9 @@ void replay_gdb_attached(void);
 
 /* Interrupts and exceptions */
 
+bool replay_switch_cpu(void);
+bool replay_has_switch_cpu(void);
+
 /* Called by exception handler to write or read exception processing events */
 bool replay_exception(void);
 /*
replay/replay-internal.h +1 -0
index b6836354ac..95849e7461 100644
@@ -58,6 +58,7 @@ enum ReplayEvents {
     /* some of greater codes are reserved for checkpoints */
     EVENT_CHECKPOINT,
     EVENT_CHECKPOINT_LAST = EVENT_CHECKPOINT + CHECKPOINT_COUNT - 1,
+    EVENT_SWITCH_CPU,
     /* end of log event */
     EVENT_END,
     EVENT_COUNT
replay/replay.c +33 -1
index 0f7d766efe..ed0f020903 100644
@@ -98,9 +98,41 @@ void replay_account_executed_instructions(void)
     }
 }
 
-bool replay_exception(void)
+bool replay_switch_cpu(void)
+{
+    if (replay_mode == REPLAY_MODE_RECORD) {
+        g_assert(replay_mutex_locked());
+        replay_save_instructions();
+        replay_put_event(EVENT_SWITCH_CPU);
+        return true;
+    } else if (replay_mode == REPLAY_MODE_PLAY) {
+        bool res = replay_has_switch_cpu();
+        if (res) {
+            replay_finish_event();
+        } else {
+            assert(0);
+        }
+        return res;
+    }
+
+    return true;
+}
+
+bool replay_has_switch_cpu(void)
 {
+    bool res = false;
+    if (replay_mode == REPLAY_MODE_PLAY) {
+        g_assert(replay_mutex_locked());
+        replay_account_executed_instructions();
+        res = replay_next_event_is(EVENT_SWITCH_CPU);
+    }
+
+    return res;
+}
 
+
+bool replay_exception(void)
+{
     if (replay_mode == REPLAY_MODE_RECORD) {
         g_assert(replay_mutex_locked());
         replay_save_instructions();
softmmu/vl.c +1 -1
index 0f39996640..fe57170ed7 100644
@@ -1883,7 +1883,7 @@ static void qemu_apply_machine_options(QDict *qdict)
         semihosting_arg_fallback(current_machine->kernel_filename, current_machine->kernel_cmdline);
     }
 
-    if (current_machine->smp.cpus > 1) {
+    if (0 && current_machine->smp.cpus > 1) {
         replay_add_blocker("smp");
     }
 }
Signature · 1 line
2.40.1